2

I have a RoR application with installed Prometheus-client and Telegraf daemon with Prometheus input plugin working on the instance I want to monitor.

As far as I understand I need some kind of exporter middleware to collect metrics from Prometheus::Client.registry and expose them with /metrics HTTP endpoint.

What I don't really understand is how to pass all metrics from different envs (e.g from rake task and app's runtime code) into the same registry (it's an instance variable of Prometheus::Middleware::Exporter.new(registry)) of the same instance of Prometheus::Middleware::Exporter middleware?

Also, will urls = ["http://localhost:3000/metrics"] config of Prometheus input plugin for Telegraf work on EC2 instance for example?

Thank you for advices.

xamenrax
  • 1,724
  • 3
  • 27
  • 47

2 Answers2

2

Perhaps an easier way to go would be to setup a Telegraf client on the same host (with Prometheus output and statsd input) and then fire events from your application into Telegraf's input, in statsd format. Telegraf would then turn around and emit these metrics in Prometheus's format.

in this way you'll get both Telegraf's host-level metrics (free memory, disc usage, etc) AND your application's metrics, all exported in the same port. It doesn't require any Ruby-specific code, just the ability to fire UDP messages from your app into a local port.

FuzzyAmi
  • 7,543
  • 6
  • 45
  • 79
  • The thing is that I need to use Prometheus client written in ruby, Telegraf agent with Prometheus input plugin (grabs metrics from /metrics endpoint) installed on the same host and I need to send all these metrics to InfluxDB endpoint. Basically, my question is how to send metrics from both rake task and app's runtime code to the /metrics endpoint (created with Prometheus exporter) using Prometheus client. – xamenrax Apr 03 '17 at 16:24
  • why do you need to use Prometheus client in ruby? I'm suggesting you skip that technology and just send statsd-messages into telegraf from your code. it'll amount to the same thing, in the end. I dont know much ruby, but this page seems to show how its done: https://ruby-doc.org/stdlib-1.9.3/libdoc/socket/rdoc/UDPSocket.html – FuzzyAmi Apr 04 '17 at 05:50
  • Because we won't be using InfluxDB in the future and planning to use Prometheus server (currently we use Prometheus exporter to grab metrics from InfluxDB), also there is no histogram and summary metrics in StatsD ruby client as far as I can see. P.S I know all this is overcomplicated – xamenrax Apr 04 '17 at 07:25
  • Switching to Prometheus is not a problem: telegraf can output the metrics to whichever format you want (we use it for Prometheus). Regarding histogram/summary: you might have a point here: I'm not sure you could export that format to/from Telegraf. – FuzzyAmi Apr 04 '17 at 07:41
  • > I'm not sure you could export that format to/from Telegraf. Why not? I use telegraf + prometheus input plugin and prometheus exporter middleware to give metrics on /metrics route. – xamenrax Apr 04 '17 at 07:44
  • what i'm trying to say is this: You can use telegraf to export metrics. I use it to export Counters and Guagues. I *guess* you could use it to export historgram, but I dont know how to do that w/ telegraf. – FuzzyAmi Apr 04 '17 at 07:47
2

First of all, it's not recommended to use a catch-all exporter like Telegraf. You can read about some of the arguments in this blog post: https://www.robustperception.io/one-agent-to-rule-them-all/.

Then, if I understand your question correctly, it's not possible to use the same registry from multiple processes (like your Rails app and some rake task). Your Rails app will export its own metrics and you'll need to use a different approach for rake tasks.

As rake tasks are (usually) short-lived processes, they are not well suited to be pulled from. You have two options here, either you use the Pushgateway and the PGW-support in client_ruby to push all relevant metrics at the end of the rake task execution (like how long it took, how many items were processed, if there was any error, etc.). Alternatively, you can use the textfile collector in the node_exporter and write your metrics to disk at the end of your rake task execution. The node_exporter will then read that file and export the metrics when it gets scraped.

I don't actively monitor stackoverflow, you'll get more help with these questions on the prometheus-users mailing list, see https://prometheus.io/community/.

grobie
  • 148
  • 4