# nginx-lua-prometheus **Repository Path**: smilive/nginx-lua-prometheus ## Basic Information - **Project Name**: nginx-lua-prometheus - **Description**: nginx-lua-prometheus - **Primary Language**: Unknown - **License**: MIT - **Default Branch**: master - **Homepage**: None - **GVP Project**: No ## Statistics - **Stars**: 0 - **Forks**: 0 - **Created**: 2021-11-12 - **Last Updated**: 2021-11-12 ## Categories & Tags **Categories**: Uncategorized **Tags**: None ## README [![Build Status](https://secure.travis-ci.org/knyar/nginx-lua-prometheus.svg?branch=master)](http://travis-ci.org/knyar/nginx-lua-prometheus?branch=master) [![Coverage Status](https://coveralls.io/repos/github/knyar/nginx-lua-prometheus/badge.svg?branch=master)](https://coveralls.io/github/knyar/nginx-lua-prometheus?branch=master) # Prometheus metric library for Nginx This is a Lua library that can be used with Nginx to keep track of metrics and expose them on a separate web page to be pulled by [Prometheus](https://prometheus.io). ## Installation You need to install nginx package with lua support (`libnginx-mod-http-lua` on newer Debian versions, or `nginx-extras` on older ones). The library file, `prometheus.lua`, needs to be available in `LUA_PATH`. If this is the only Lua library you use, you can just point `lua_package_path` to the directory with this git repo checked out (see example below). OpenResty users will find this library in [opm](https://opm.openresty.org/). It is also available via [luarocks](https://luarocks.org/modules/knyar/nginx-lua-prometheus). ## Quick start guide To track request latency broken down by server name and request count broken down by server name and status, add the following to the `http` section of `nginx.conf`: ```nginx lua_shared_dict prometheus_metrics 10M; lua_package_path "/path/to/nginx-lua-prometheus/?.lua;;"; init_worker_by_lua_block { prometheus = require("prometheus").init("prometheus_metrics") metric_requests = prometheus:counter( "nginx_http_requests_total", "Number of HTTP requests", {"host", "status"}) metric_latency = prometheus:histogram( "nginx_http_request_duration_seconds", "HTTP request latency", {"host"}) metric_connections = prometheus:gauge( "nginx_http_connections", "Number of HTTP connections", {"state"}) } log_by_lua_block { metric_requests:inc(1, {ngx.var.server_name, ngx.var.status}) metric_latency:observe(tonumber(ngx.var.request_time), {ngx.var.server_name}) } ``` This: * configures a shared dictionary for your metrics called `prometheus_metrics` with a 10MB size limit; * registers a counter called `nginx_http_requests_total` with two labels: `host` and `status`; * registers a histogram called `nginx_http_request_duration_seconds` with one label `host`; * registers a gauge called `nginx_http_connections` with one label `state`; * on each HTTP request measures its latency, recording it in the histogram and increments the counter, setting current server name as the `host` label and HTTP status code as the `status` label. Last step is to configure a separate server that will expose the metrics. Please make sure to only make it reachable from your Prometheus server: ```nginx server { listen 9145; allow 192.168.0.0/16; deny all; location /metrics { content_by_lua_block { metric_connections:set(ngx.var.connections_reading, {"reading"}) metric_connections:set(ngx.var.connections_waiting, {"waiting"}) metric_connections:set(ngx.var.connections_writing, {"writing"}) prometheus:collect() } } } ``` Metrics will be available at `http://your.nginx:9145/metrics`. Note that the gauge metric in this example contains values obtained from nginx global state, so they get set immediately before metrics are returned to the client. ## API reference ### init() **syntax:** require("prometheus").init(*dict_name*, [*options*]]) Initializes the module. This should be called once from the [init_worker_by_lua_block](https://github.com/openresty/lua-nginx-module#init_worker_by_lua_block) section of nginx configuration. * `dict_name` is the name of the nginx shared dictionary which will be used to store all metrics. Defaults to `prometheus_metrics` if not specified. * `options` is a table of configuration options that can be provided. Accepted options are: * `prefix` (string): metric name prefix. This string will be prepended to metric names on output. * `error_metric_name` (string): Can be used to change the default name of error metric (see [Built-in metrics](#built-in-metrics) for details). * `sync_interval` (number): sets per-worker counter sync interval in seconds. This sets the boundary on eventual consistency of counter metrics. Defaults to 1. Returns a `prometheus` object that should be used to register metrics. Example: ```nginx init_worker_by_lua_block { prometheus = require("prometheus").init("prometheus_metrics", {sync_interval=3}) } ``` ### prometheus:counter() **syntax:** prometheus:counter(*name*, *description*, *label_names*) Registers a counter. Should be called once for each counter from the [init_worker_by_lua_block]( https://github.com/openresty/lua-nginx-module#init_worker_by_lua_block) section. * `name` is the name of the metric. * `description` is the text description that will be presented to Prometheus along with the metric. Optional (pass `nil` if you still need to define label names). * `label_names` is an array of label names for the metric. Optional. [Naming section](https://prometheus.io/docs/practices/naming/) of Prometheus documentation provides good guidelines on choosing metric and label names. Returns a `counter` object that can later be incremented. Example: ```nginx init_worker_by_lua_block { prometheus = require("prometheus").init("prometheus_metrics") metric_bytes = prometheus:counter( "nginx_http_request_size_bytes", "Total size of incoming requests") metric_requests = prometheus:counter( "nginx_http_requests_total", "Number of HTTP requests", {"host", "status"}) } ``` ### prometheus:gauge() **syntax:** prometheus:gauge(*name*, *description*, *label_names*) Registers a gauge. Should be called once for each gauge from the [init_worker_by_lua_block]( https://github.com/openresty/lua-nginx-module#init_worker_by_lua_block) section. * `name` is the name of the metric. * `description` is the text description that will be presented to Prometheus along with the metric. Optional (pass `nil` if you still need to define label names). * `label_names` is an array of label names for the metric. Optional. Returns a `gauge` object that can later be set. Example: ```nginx init_worker_by_lua_block { prometheus = require("prometheus").init("prometheus_metrics") metric_connections = prometheus:gauge( "nginx_http_connections", "Number of HTTP connections", {"state"}) } ``` ### prometheus:histogram() **syntax:** prometheus:histogram(*name*, *description*, *label_names*, *buckets*) Registers a histogram. Should be called once for each histogram from the [init_worker_by_lua_block]( https://github.com/openresty/lua-nginx-module#init_worker_by_lua_block) section. * `name` is the name of the metric. * `description` is the text description. Optional. * `label_names` is an array of label names for the metric. Optional. * `buckets` is an array of numbers defining bucket boundaries. Optional, defaults to 20 latency buckets covering a range from 5ms to 10s (in seconds). Returns a `histogram` object that can later be used to record samples. Example: ```nginx init_worker_by_lua_block { prometheus = require("prometheus").init("prometheus_metrics") metric_latency = prometheus:histogram( "nginx_http_request_duration_seconds", "HTTP request latency", {"host"}) metric_response_sizes = prometheus:histogram( "nginx_http_response_size_bytes", "Size of HTTP responses", nil, {10,100,1000,10000,100000,1000000}) } ``` ### prometheus:collect() **syntax:** prometheus:collect() Presents all metrics in a text format compatible with Prometheus. This should be called in [content_by_lua_block](https://github.com/openresty/lua-nginx-module#content_by_lua_block) to expose the metrics on a separate HTTP page. Example: ```nginx location /metrics { content_by_lua_block { prometheus:collect() } allow 192.168.0.0/16; deny all; } ``` ### prometheus:metric_data() **syntax:** prometheus:metric_data() Returns metric data as an array of strings. ### counter:inc() **syntax:** counter:inc(*value*, *label_values*) Increments a previously registered counter. This is usually called from [log_by_lua_block](https://github.com/openresty/lua-nginx-module#log_by_lua_block) globally or per server/location. * `value` is a value that should be added to the counter. Defaults to 1. * `label_values` is an array of label values. The number of label values should match the number of label names defined when the counter was registered using `prometheus:counter()`. No label values should be provided for counters with no labels. Non-printable characters will be stripped from label values. Example: ```nginx log_by_lua_block { metric_bytes:inc(tonumber(ngx.var.request_length)) metric_requests:inc(1, {ngx.var.server_name, ngx.var.status}) } ``` ### counter:del() **syntax:** counter:del(*label_values*) Delete a previously registered counter. This is usually called when you don't need to observe such counter (or a metric with specific label values in this counter) any more. If this counter has labels, you have to pass `label_values` to delete the specific metric of this counter. If you want to delete all the metrics of a counter with labels, you should call `Counter:reset()`. * `label_values` is an array of label values. The number of label values should match the number of label names defined when the counter was registered using `prometheus:counter()`. No label values should be provided for counters with no labels. Non-printable characters will be stripped from label values. This function will wait for `sync_interval` before deleting the metric to allow all workers to sync their counters. ### counter:reset() **syntax:** counter:reset() Delete all metrics for a previously registered counter. If this counter have no labels, it is just the same as `Counter:del()` function. If this counter have labels, it will delete all the metrics with different label values. This function will wait for `sync_interval` before deleting the metrics to allow all workers to sync their counters. ### gauge:set() **syntax:** gauge:set(*value*, *label_values*) Sets the current value of a previously registered gauge. This could be called from [log_by_lua_block](https://github.com/openresty/lua-nginx-module#log_by_lua_block) globally or per server/location to modify a gauge on each request, or from [content_by_lua_block](https://github.com/openresty/lua-nginx-module#content_by_lua_block) just before `prometheus::collect()` to return a real-time value. * `value` is a value that the gauge should be set to. Required. * `label_values` is an array of label values. ### gauge:inc() **syntax:** gauge:inc(*value*, *label_values*) Increments or decrements a previously registered gauge. This is usually called when you want to observe the real-time value of a metric that can both be increased and decreased. * `value` is a value that should be added to the gauge. It could be a negative value when you need to decrease the value of the gauge. Defaults to 1. * `label_values` is an array of label values. The number of label values should match the number of label names defined when the gauge was registered using `prometheus:gauge()`. No label values should be provided for gauges with no labels. Non-printable characters will be stripped from label values. ### gauge:del() **syntax:** gauge:del(*label_values*) Delete a previously registered gauge. This is usually called when you don't need to observe such gauge (or a metric with specific label values in this gauge) any more. If this gauge has labels, you have to pass `label_values` to delete the specific metric of this gauge. If you want to delete all the metrics of a gauge with labels, you should call `Gauge:reset()`. * `label_values` is an array of label values. The number of label values should match the number of label names defined when the gauge was registered using `prometheus:gauge()`. No label values should be provided for gauges with no labels. Non-printable characters will be stripped from label values. ### gauge:reset() **syntax:** gauge:reset() Delete all metrics for a previously registered gauge. If this gauge have no labels, it is just the same as `Gauge:del()` function. If this gauge have labels, it will delete all the metrics with different label values. ### histogram:observe() **syntax:** histogram:observe(*value*, *label_values*) Records a value in a previously registered histogram. Usually called from [log_by_lua_block](https://github.com/openresty/lua-nginx-module#log_by_lua_block) globally or per server/location. * `value` is a value that should be recorded. Required. * `label_values` is an array of label values. Example: ```nginx log_by_lua_block { metric_latency:observe(tonumber(ngx.var.request_time), {ngx.var.server_name}) metric_response_sizes:observe(tonumber(ngx.var.bytes_sent)) } ``` ### histogram:reset() **syntax:** histogram:reset() Delete all metrics for a previously registered histogram. This function will wait for `sync_interval` before deleting the metrics to allow all workers to sync their counters. ### Built-in metrics The module increments an error metric called `nginx_metric_errors_total` (unless another name was configured in [init()](#init)) if it encounters an error (for example, when `lua_shared_dict` becomes full). You might want to configure an alert on that metric. ## Caveats ### Usage in stream module For now, there is no way to share a dictionary between HTTP and Stream modules in Nginx. If you are using this library to collect metrics from stream module, you will need to configure a separate endpoint to return them. Here's an example. ```nginx server { listen 9145; content_by_lua_block { local sock = assert(ngx.req.socket(true)) local data = sock:receive() local location = "GET /metrics" if string.sub(data, 1, string.len(location)) == location then ngx.say("HTTP/1.1 200 OK") ngx.say("Content-Type: text/plain") ngx.say("") ngx.say(table.concat(prometheus:metric_data(), "")) else ngx.say("HTTP/1.1 404 Not Found") end } } ``` ## Troubleshooting ### Make sure that nginx lua module is enabled If you experience problems indicating that nginx doesn't know how to interpret lua scripts, please make sure that [the lua module](https://github.com/openresty/lua-nginx-module) is enabled. You might need something like this in your `nginx.conf`: ```nginx load_module modules/ndk_http_module.so; load_module modules/ngx_http_lua_module.so; ``` ### Keep lua code cache enabled This module expects the [lua_code_cache](https://github.com/openresty/lua-nginx-module#lua_code_cache) option to be `on` (which is the default). ### Try using an older version of the library If you are seeing library initialization errors, followed by errors for each metric change request (e.g. *attempt to index global '...' (a nil value)*), you are probably using an old version of lua-nginx-module. For example, this will happen if you try using the latest version of this library with the `nginx-extras` package shipped with Ubuntu 16.04. If you cannot upgrade nginx and lua-nginx-module, you can try using an older version of this library; it will not have the latest performance optimizations, but will still be functional. The recommended older release to use is [0.20181120](https://github.com/knyar/nginx-lua-prometheus/tree/0.20181120). ## Development ### Install dependencies for testing - `luarocks install luacheck` - `luarocks install luaunit` ### Run tests - `luacheck --globals ngx -- prometheus.lua` - `lua prometheus_test.lua` - `cd integration && ./test.sh` (requires Docker and Go) ### Releasing new version - update CHANGELOG.md - update version in the `dist.ini` - rename `.rockspec` file and update version inside it - commit changes - create a new Git tag: `git tag 0.XXXXXXXX && git push origin 0.XXXXXXXX` - push to luarocks: `luarocks upload nginx-lua-prometheus-0.20181120-1.rockspec` - upload to OPM: `opm build && opm upload` ## Credits - Created and maintained by Anton Tolchanov ([@knyar](https://github.com/knyar)) - Metrix prefix support contributed by david birdsong ([@davidbirdsong]( https://github.com/davidbirdsong)) - Gauge support contributed by Cosmo Petrich ([@cosmopetrich]( https://github.com/cosmopetrich)) - Performance improvements and per-worker counters are contributed by Wangchong Zhou ([@fffonion](https://github.com/fffonion)) / [@Kong]( https://github.com/Kong). - Metric name tracking improvements contributed by Jan DolinĂ¡r ([@dolik-rce]( https://github.com/dolik-rce)) ## License Licensed under MIT license. ### Third Party License Following third party modules are used in this library: - [Kong/lua-resty-counter](https://github.com/Kong/lua-resty-counter) This module is licensed under the Apache 2.0 license. Copyright (C) 2019, Kong Inc. All rights reserved.