Configure Nginx Reverse Proxy For Grafana Access

Proxying Grafana with Nginx is easy, but there are some small thins that needs to be considered.

If you have already tried this and received strange errors like:

192.168.1.1 – – [09/May/2019:10:26:58 +0300] “-” 000 0 “-” “-” “-“

499 errors in your nginx proxy logs

Grafana dashboard graphics not being loaded

most probably you have forgotten the tricky Nginx variables mentioned below.

1) Configure Grafana

These are the important settings inside grafana.ini you need to setup properly:

Open grafana config file: /etc/grafana/grafana.ini

[server]
http_port = 3000
domain = grafana.example.com
root_url = http://grafana.example.com

2) Configure Nginx

This is the tricky part from the setup. If you forget to put the header buffer settings in your Nginx vhost configs, you will end up with malfunctioning Grafana graphs.

The key parts from your Nginx configs are the usage of:

large_client_header_buffers

AND (for http2 over ssl, if you are using it)

http2_max_header_size

http2_max_field_size

upstream grafana {
        server 127.0.0.1:3000;
        keepalive 15;
}

# Non-https version to redirect 
    server {
        listen       80 default_server;
        server_name  graf.example.com;
        root         /usr/share/nginx/html;

        large_client_header_buffers 4 64k;

        location / {
                    proxy_pass http://grafana;
                    proxy_http_version 1.1;
                    proxy_hide_header Upgrade ;
                    proxy_set_header Connection "Keep-Alive";
                    proxy_set_header Host $host;
                    proxy_set_header Proxy-Connection "Keep-Alive";
        }

        error_page 404 /404.html;
            location = /40x.html {
        }

        error_page 500 502 503 504 /50x.html;
            location = /50x.html {
        }
    }
# Settings for a TLS enabled server.
#
server {
    listen       443 ssl http2 default_server;
    server_name  graf.example.com;
    root         /usr/share/nginx/html;

    large_client_header_buffers 4 256k;
    http2_max_header_size 256k; 
    http2_max_field_size  256k; 

    ssl_certificate "/etc/pki/tls/certs/graf.example.com.crt";
    ssl_certificate_key "/etc/pki/tls/private/graf.example.com.key";
    ssl_session_cache shared:SSL:1m;
    ssl_session_timeout  10m;
    ssl_ciphers HIGH:!aNULL:!MD5;
    ssl_prefer_server_ciphers on;


    location ~ / {
                proxy_pass http://grafana;
                proxy_http_version 1.1;
                proxy_hide_header Upgrade ;
                proxy_set_header Connection "Keep-Alive";
                proxy_set_header Proxy-Connection "Keep-Alive";
                proxy_set_header Host $host;
    }

    error_page 404 /404.html;
        location = /40x.html {
    }

    error_page 500 502 503 504 /50x.html;
        location = /50x.html {
    }
}

In the SSL vhost configuration I expect you have put your SSL certificates under:

ssl_certificate “/etc/pki/tls/certs/graf.example.com.crt”;

ssl_certificate_key “/etc/pki/tls/private/graf.example.com.key”;

Modify it to suite your case.

3) Some more information on Nginx Proxying

If you are interested in some more useful information regarding Nginx proxying (with caching), I’ve written some resources about it. Check it out.

How To Setup Nginx Reverse Proxy With Caching

Nginx Proxy Caching based on page size

Check If Request is being cached in Nginx

1) Configure Grafana

2) Configure Nginx

3) Some more information on Nginx Proxying

Related posts: