Sunday, March 31, 2013

Setting Up munin on Ubuntu Server 13.04 with nginx

This post explains how to configure munin on a single Ubuntu 13.04 server that uses nginx.

Monitoring is supposed to save time by debugging and predicting / avoiding catastrophes. However, setting up munin on Ubuntu was a time-consuming trial-and-error process for me. The official instructions and various blog posts that cover this topic skip important steps, such as having monit's FastCGI processes start automatically at boot time. I have documented the setup that worked for me, hoping that others can reuse my work.

Ubuntu packages

Run the following command to install the packages needed for munin.
sudo apt-get install munin munin-node spawn-fcgi libcgi-fast-perl

The following sections configure the munin packages.

munin configuration

Write the munin configuration below to /etc/munin/munin-conf.d/90-fcgi

graph_strategy cgi
html_strategy cgi
cgiurl_graph /munin/munin-cgi-graph

nginx configuration

Write the nginx configuration below to /etc/nginx/sites-enabled/munin.conf

server {
  listen 443 ssl;
  listen 80;
  charset utf-8;
  server_name munin.your-domain.com;

  location ~ ^/munin/munin-cgi-graph/ {
    fastcgi_split_path_info ^(/munin/munin-cgi-graph)(.*);
    fastcgi_param PATH_INFO $fastcgi_path_info;
    fastcgi_pass unix:/var/run/munin/fastcgi-graph.sock;
    include fastcgi_params;
  }

  location /munin/static/ {
    alias /etc/munin/static/;
    expires modified +1w;
  }

  location /munin/ {
    fastcgi_split_path_info ^(/munin)(.*);
    fastcgi_param PATH_INFO $fastcgi_path_info;
    fastcgi_pass unix:/var/run/munin/fastcgi-html.sock;
    include fastcgi_params;
  }

  location / {
    rewrite ^/$ munin/ redirect; break;
  }
}

This configuration assumes that you have a DNS entry set aside for reaching the monit pages. I have separate DNS entries for all my applications, and they're all CNAMEs for the (same) machine that they're running on.

Once you're done tweaking the script, reload nginx.
sudo /etc/init.d/nginx reload

FastCGI daemons

Write the script below to /etc/init.d/munin-fcgi

#!/bin/bash

### BEGIN INIT INFO
# Provides:          munin-fcgi
# Required-Start:    $remote_fs $syslog $network
# Required-Stop:     $remote_fs $syslog $network
# Default-Start:     2 3 4 5
# Default-Stop:      0 1 6
# Short-Description: Start munin FCGI processes at boot time
# Description:       Start the FCGI processes behind http://munin.*/
### END INIT INFO

graph_pidfile="/var/run/munin/fcgi_graph.pid"
# Ubuntu 12.10: /usr/lib/cgi-bin/munin-cgi-graph
graph_cgi="/usr/lib/munin/cgi/munin-cgi-graph"
html_pidfile="/var/run/munin/fcgi_html.pid"
# Ubuntu 12.10: /usr/lib/cgi-bin/munin-cgi-html
html_cgi="/usr/lib/munin/cgi/munin-cgi-html"

retval=0

. /lib/lsb/init-functions

start() {
  echo -n "Starting munin graph FastCGI: "
  start_daemon -p ${graph_pidfile} /usr/bin/spawn-fcgi -u munin -g munin \
      -s /var/run/munin/fastcgi-graph.sock -U www-data ${graph_cgi}
  echo
  echo -n "Starting munin html FastCGI: "
  start_daemon -p ${html_pidfile} /usr/bin/spawn-fcgi -u munin -g munin \
      -s /var/run/munin/fastcgi-html.sock -U www-data ${html_cgi}
  echo
  retval=$?
}
stop() {
  echo -n "Stopping munin graph FastCGI: "
  killproc -p ${graph_pidfile} ${graph_cgi} -QUIT
  echo
  echo -n "Stopping munin html FastCGI: "
  killproc -p ${html_pidfile} ${html_cgi} -QUIT
  echo
  retval=$?
}

case "$1" in
  start)
    start
  ;;
  stop)
    stop
  ;;
  restart)
    stop
    start
  ;;
  *)
    echo "Usage: munin-fcgi {start|stop|restart}"
    exit 1
  ;;
esac
exit $retval

Make the script executable, and fix some permissions while you're at it.

sudo chmod +x /etc/init.d/munin-fcgi
sudo chown munin /var/log/munin/munin-cgi-*

Now that the init.d script is in place, start it and have it run on every boot.

sudo /etc/init.d/munin-fcgi start
sudo update-rc.d munin-fcgi defaults

Debugging

You should be able to point your browser to http://munin.your-domain.com and see many graphs. If that doesn't work out, the logs below should give you a clue as to what went wrong.

  • /var/log/nginx/error.log
  • /var/log/munin/munin-cgi-graph.log
  • /var/log/munin/munin-cgi-html.log

Conclusion

I hope that you have found my post useful, and I hope that it helped you get your munin monitoring setup up and running in a manner of minutes. I look forward to your comments and feedback!

6 comments:

  1. thanks for article.
    My problem is in /var/log/nginx/error.log:
    2013/05/01 20:52:01 [error] 13076#0: *6 open() "/etc/nginx/html/cgi-bin/munin-cgi-graph/localdomain/local
    host.localdomain/uptime-week.png" failed (2: No such file or directory)

    html socket works good, but graphs not.


    /var/log/munin/munin-cgi-graph.log looks (probably) ok:

    2013/05/01 20:57:57 Request path is /localdomain/localhost.localdomain/diskstats_iops/xvda1-year.png
    2013/05/01 20:57:57 asked for (localdomain/localhost.localdomain, diskstats_iops, xvda1, year)
    2013/05/01 20:57:57 Starting munin-graph
    2013/05/01 20:57:57 [INFO] Looking into drawing /var/lib/munin/cgi-tmp/munin-cgi-graph/localdomain/localh
    ost.localdomain/diskstats_iops/xvda1-year.png
    2013/05/01 20:57:57 [INFO] Graphed service localdomain;localhost.localdomain:diskstats_iops.xvda1 (0.02 s
    ec for 1 graphs)
    2013/05/01 20:57:57 Munin-graph finished (0.02 sec)

    any advice please?

    ReplyDelete
  2. This worked perfectly! Thank you very much. :))

    ReplyDelete
  3. Cool, thanks. Goddammned munin devs broke something with Ubuntu 13.04 update. Fixed my one using this article.

    ReplyDelete
  4. this works! almost gave up until i saw this.

    ReplyDelete
  5. Nice - thanks - in particular for the init.d script.

    ReplyDelete
  6. Hi. I'm trying to setup my Ubuntu Server 12.04 according this tutorial. However when I try point my browser to http://munin.your-domain.com I get nginx error in error.log that looks like the following:

    crit] 1184#0: *1 connect() to unix:.var/run/munin/fastcgi-html.sock failed (2: No such file or directory)

    It seems that init.d script dosen't creates that file. Can you help resolve my problem please?

    ReplyDelete