Repository: shawn-sterling/graphios
Branch: master
Commit: f5f0a1da1a3b
Files: 18
Total size: 88.9 KB
Directory structure:
gitextract_hfsrrmqg/
├── .gitignore
├── .travis.yml
├── MANIFEST.in
├── README.md
├── graphios.cfg
├── graphios.py
├── graphios_backends.py
├── init/
│ ├── debian/
│ │ ├── graphios
│ │ └── graphios.conf
│ ├── rhel/
│ │ ├── graphios
│ │ ├── install
│ │ ├── postinstall
│ │ └── postuninstall
│ └── systemd/
│ └── graphios.service
├── nagios/
│ ├── graphios_commands.cfg
│ └── nagios_perfdata.cfg
├── setup.cfg
└── setup.py
================================================
FILE CONTENTS
================================================
================================================
FILE: .gitignore
================================================
# File artifacts which may by produced regular software on Windows, Mac OS X and Linux
Thumbs.db
.DS_Store
*.bak
# JetBrains PyCharm configuration and working files
.idea
*.py[co]
~temp/
# Distribution / packaging
.Python
env/
build/
develop-eggs/
dist/
downloads/
eggs/
lib/
lib64/
parts/
sdist/
var/
*.egg-info/
.installed.cfg
*.egg
# PyInstaller
# Usually these files are written by a python script from a template
# before PyInstaller builds the exe, so as to inject date/other infos into it.
*.manifest
*.spec
# Installer logs
pip-log.txt
pip-delete-this-directory.txt
# rope
.ropeproject/
================================================
FILE: .travis.yml
================================================
language: python
python:
- "2.6"
- "2.7"
install:
- pip install flake8
script: flake8 *.py
================================================
FILE: MANIFEST.in
================================================
include graphios.py
include graphios_backends.py
include graphios.cfg
graft init
graft nagios
================================================
FILE: README.md
================================================
Graphios
========
[](https://travis-ci.org/shawn-sterling/graphios)
*Oct 15, 2014*
New graphios 2.0!
What's new?
* Support for multiple backends (graphite, statsd, librato) (and multiples of
each backend if you want)
* Support for using your service descriptions instead of custom variables
* Install options (pip, setup.py, rpms)
* Bugfixes
* mulitple perfdata in 1 line sometimes did weird things
* quotes in your labels/metrics were sometimes in carbon
* labels with multiple '::' could mess up
# Introduction
Graphios is a script to emit nagios perfdata to various upstream metrics
processing and time-series (graphing) systems. It's currently compatible with
[graphite], [statsd], [Librato] and [InfluxDB], with possibly [Heka], and
[RRDTool] support coming soon. Graphios can emit Nagios metrics to any number
of supported upstream metrics systems simultaenously.
# Requirements
* A working nagios / icinga / naemon server
* A functional carbon or statsd daemon, and/or Librato credentials
* Python 2.6 or later (but not python 3.x) (Is anyone still using 2.4? Likely
very little work to make this work under 2.4 again if so. Let me know)
# License
Graphios is released under the [GPL v2](http://www.gnu.org/licenses/gpl-2.0.html).
# Documentation
The goal of graphios is to get nagios perf data into a graphing system like
graphite (carbon). Systems like these typically use a dot-delimited metric name
to store each metric hierarcicly, so it can be easily located later.
Graphios creates these metric names one of two ways.
1. by reading a pair of custom variables that
you configure for services and hosts called \_graphiteprefix and
\_graphitepostfix. Together, these custom variables enable you to control the
metric name that gets sent to whatever back-end metrics system you're using.
You don't have to set them both, but things will certainly be less confusing
for you if you set at least one or the other.
2. by using your service description in the format:
\_graphiteprefix.hostname.service-description.\_graphitepostfix.perfdata
so if you didn't feel like setting your graphiteprefix and postfix, it would
just use:
hostname.service-description.perfdata
If you are using option 2, that means EVERY service will be sent to graphite.
You will also want to make sure your service descriptions are consistant or
your backend naming will be really weird.
I think most people will use the first option, so let's work with that for a
bit. What gets sent to graphite is this:
graphiteprefix.hostname.graphitepostfix.perfdata
The specific content of the perfdata section depends on each particular Nagios
plugin's output.
Simple Example
--------------
A simple example is the check\_host\_alive command (which calls the check\_icmp
plugin by default). The check\_icmp plugin returns the following perfstring:
rta=4.029ms;10.000;30.000;0; pl=0%;5;10;; rtmax=4.996ms;;;; rtmin=3.066ms;;;;
If we configured a host with a custom graphiteprefix variable like this:
<pre>
define host {
host_name myhost
check_command check_host_alive
_graphiteprefix ops.nagios01.pingto
}
</pre>
Graphios will construct and emit the following metric name to the upstream metric system:
ops.nagios01.pingto.myhost.rta 4.029 nagios_timet
ops.nagios01.pingto.myhost.pl 0 nagios_timet
ops.nagios01.pingto.myhost.rtmax 4.996 nagios_timet
ops.nagios01.pingto.myhost.rtmin 3.066 nagios_timet
Where *nagios\_timet* is the a unix epoch time stamp from when the plugin
results were received by Nagios core. Your prefix is of course, entirely up to
you. In our example, our prefix refers to the Team that created the metric
(Ops), becuause our upstream metrics system is used by many different teams.
Afer the team name, we've identified the specific Nagios host that took this
measurement, because we actually have several Nagios boxes, and finally,
'pingto' is the name of this specific metric: the *ping* time from nagios01
*to* myhost.
Another example
---------------
Lets take a look at the check_load plugin, which returns the following
perfdata:
load1=8.41;20;22;; load5=6.06;18;20;; load15=5.58;16;18
Our service is defined like this:
<pre>
define service {
service_description Load
host_name myhost
_graphiteprefix datacenter01.webservers
_graphitepostfix nrdp.load
}
</pre>
With this confiuration, graphios generates the following metric names:
datacenter01.webservers.myhost.nrdp.load.load1 8.41 nagios_timet
datacenter01.webservers.myhost.nrdp.load.load5 6.06 nagios_timet
datacenter01.webservers.myhost.nrdp.load.load15 5.58 nagios_timet
As you can probably guess, our custom prefix in this example identifies the
specific data center, and server-type from which these metrics originated,
while our postfix refers to the check_nrdp plugin, which is the means by which
we collected the data, followed finally by the metric-type.
You should think carefully about how you name your metrics, because later on,
these names will enable you to easily combine metrics (like load1) across
various sources (like all webservers).
Using metric_base_path to add a universal prefix
------------------------------------------------
In an environment where multiple things are feeding metrics into your backend
service, it can be handy to differentiate by source. Normally, you would need
to prepend the graphiteprefix to all services and hosts, but in some cases, this
isn't possible or feasible.
When you want everything to be prepended with the same string, use the
metric_base_path setting:
metric_base_path = mycorp.nagios
Note that quotes will be preserved. Also, _graphiteprefix and _graphitepostfix
will be applied in addition to this string, so if you are already adding
mycorp.nagios to your prefix, you will end up with mycorp.nagios.mycorp.nagios.metricname
A few words on Naming things for Librato
----------------------------------------
The default configuration that works for Graphite also does what you'd expect
for Librato, so if you're just getting started, and you want to check out
Librato, don't worry about it, ignore this section and forge ahead.
But you're a power user, you should be aware that the Librato Backend is
actually generating a differet metric name than the other plugins.
Librato is a very metrics-centric platform. Metrics are the first-class entity,
and sources (like hosts), are actually a separate dimension in their system.
This is very cool when you're monitoring ephemeral things that aren't hosts,
like threads, or worker processes, but it slightly complicates things here.
So, for example, where the Graphite plugin generates a name like this (from the
example above):
datacenter01.webservers.myhost.nrdp.load.load1
The Librato plugin will generate a name that omits the hostname:
datacenter01.webservers.nrdp.load.load1
And then it will automatically send the hostname as the source dimension when
it emits the metric to Librato. For 99% of everyone, this is exactly what you
want. But if you're a 1%'er you can influence this behavior by modifying the
"namevals" and "sourcevals" lists in the librato section of the graphios.cfg
Automatic names
---------------
Version 2.0: Graphios now supports automatic names, because custom variables
are hard. :)
This is an all or nothing setting, meaning if you turn this on all services
will now send to graphios (instead of just the ones with the prefix and postfix
setup). This will work fine, so long as you have very consistent service
descriptions.
To turn this on, modify the graphios.cfg and change:
use_service_desc = False
to
use_service_desc = True
You can still use the graphite prefix and postfix variables but you don't have
to.
# Big Fat Warning
Graphios assumes your checks are using the same unit of measurement. Most
plugins support this, some do not. check\_icmp) always reports in ms for
example.
# Installation
This is recommended for intermediate+ Nagios administrators. If you are just
learning Nagios this might be a difficult pill to swallow depending on your
experience level.
Hundreds of people have emailed me their success stories on getting graphios
working. I have been using this in production on a medium size nagios
installation for a couple years.
There are now a few ways to get graphios installed.
1 - Use pypi
```
pip install graphios
```
NOTE: This will attempt to find your nagios.cfg and add the configuration
steps 1 and 2 for you (Don't worry we back up the file before touching it)
NOTE2: If you get the error:
Could not find a version that satisfies the requirement graphios
This is a because graphios is still in the beta category. I will remove
this in a few weeks, so until then you need to:
```
pip install --pre graphios
```
2 - Clone it yourself
```
git clone https://github.com/shawn-sterling/graphios.git
cd graphios
```
Then do one of the following three things (depending what you like best):
1 - Python setup
```
python setup.py install
```
2 - Create + Install RPM
```
python setup.py bdist_rpm
yum localinstall bdist/graphios-$version.rpm
```
3 - Copy the files where you want them to be
```
cp graphios*.py /my/dir
cp graphios.cfg /my/dir
```
# Configuration
Setting this up on the nagios front is very much like pnp4nagios with npcd.
(You do not need to have any pnp4nagios experience at all). If you are already
running pnp4nagios , check out my pnp4nagios notes (below).
Steps:
(1) graphios.cfg
----------------
The default location for graphios.cfg is in /etc/graphios/graphios.cfg, it
also checks the same directory as the graphios.py is.
Your graphios.cfg can live anywhere you want, but if it's not in the above
locations you will need to modify your init script to match.
Out of the box, it enables the carbon back-end and sends pickled metrics to
127.0.0.1:2004. It also specifies the location of the graphios log and spool
directories, and controls things like log levels, sleep intervals, and of
course, backends like carbon, statsd, and librato.
The config file is well commented, adding/changing backends is very simple.
(2) nagios.cfg
--------------
Your nagios.cfg is going to need to modified to send the graphite data to the
perfdata files. Depending on how you installed graphios this step may have been
done for you.
The following needs to be put into your nagios.cfg
<pre>
service_perfdata_file=/var/spool/nagios/graphios/service-perfdata
service_perfdata_file_template=DATATYPE::SERVICEPERFDATA\tTIMET::$TIMET$\tHOSTNAME::$HOSTNAME$\tSERVICEDESC::$SERVICEDESC$\tSERVICEPERFDATA::$SERVICEPERFDATA$\tSERVICECHECKCOMMAND::$SERVICECHECKCOMMAND$\tHOSTSTATE::$HOSTSTATE$\tHOSTSTATETYPE::$HOSTSTATETYPE$\tSERVICESTATE::$SERVICESTATE$\tSERVICESTATETYPE::$SERVICESTATETYPE$\tGRAPHITEPREFIX::$_SERVICEGRAPHITEPREFIX$\tGRAPHITEPOSTFIX::$_SERVICEGRAPHITEPOSTFIX$
service_perfdata_file_mode=a
service_perfdata_file_processing_interval=15
service_perfdata_file_processing_command=graphite_perf_service
host_perfdata_file=/var/spool/nagios/graphios/host-perfdata
host_perfdata_file_template=DATATYPE::HOSTPERFDATA\tTIMET::$TIMET$\tHOSTNAME::$HOSTNAME$\tHOSTPERFDATA::$HOSTPERFDATA$\tHOSTCHECKCOMMAND::$HOSTCHECKCOMMAND$\tHOSTSTATE::$HOSTSTATE$\tHOSTSTATETYPE::$HOSTSTATETYPE$\tGRAPHITEPREFIX::$_HOSTGRAPHITEPREFIX$\tGRAPHITEPOSTFIX::$_HOSTGRAPHITEPOSTFIX$
host_perfdata_file_mode=a
host_perfdata_file_processing_interval=15
host_perfdata_file_processing_command=graphite_perf_host
</pre>
Which sets up some custom variables, specifically:
for services:
$\_SERVICEGRAPHITEPREFIX
$\_SERVICEGRAPHITEPOSTFIX
for hosts:
$\_HOSTGRAPHITEPREFIX
$\_HOSTGRAPHITEPOSTFIX
The prepended HOST and SERVICE is just the way nagios works,
\_HOSTGRAPHITEPREFIX means it's the \_GRAPHITEPREFIX variable from host
configuration.
(3) nagios commands
-------------------
There are 2 commands we setup in the nagios.cfg, which if you used pip or the
rpm/deb may have already been setup for you. We need:
graphite\_perf\_service
graphite\_perf\_host
Which we now need to define:
I use include dirs, so I make a new file called graphios\_commands.cfg inside
my include dir. Do that, or add the below commands to one of your existing
nagios config files.
#### NOTE: Your spool directory may be different, this is setup in step (2) the service_perfdata_file, and host_perfdata_file.
<pre>
define command {
command_name graphite_perf_host
command_line /bin/mv /var/spool/nagios/graphios/host-perfdata /var/spool/nagios/graphios/host-perfdata.$TIMET$
}
define command {
command_name graphite_perf_service
command_line /bin/mv /var/spool/nagios/graphios/service-perfdata /var/spool/nagios/graphios/service-perfdata.$TIMET$
}
</pre>
All these commands do is move the current files to a different filename that we can process without interrupting nagios. This way nagios doesn't have to sit around waiting for us to process the results.
(4) Run it!
---------------
We recommend running graphios.py from the console for the first time, this will
make sure things are sending the way you think they are. A good example would
be:
./graphios.py --spool-directory /var/spool/nagios/graphios \
--log-file /tmp/graphios.log \
--backend carbon \
--server 127.0.0.1:2004 \
--test
and if there are problems add
--verbose
Other command line options:
<pre>
Usage: graphios.py [options]
sends nagios performance data to carbon.
Options:
-h, --help show this help message and exit
-v, --verbose sets logging to DEBUG level
--spool-directory=SPOOL_DIRECTORY
where to look for nagios performance data
--log-file=LOG_FILE file to log to
--backend=BACKEND sets which storage backend to use
--config=CONFIG set custom config file location
--test Turns on test mode, which won't send to backends
--replace_char=REPLACE_CHAR
Replacement Character (default '_'
--sleep_time=SLEEP_TIME
How much time to sleep between checks
--sleep_max=SLEEP_MAX
Max time to sleep between runs
--server=SERVER Server address (for backend)
--no_replace_hostname
Replace '.' in nagios hostnames, default on.
--reverse_hostname Reverse nagios hostname, default off.
)
</pre>
** NOTE: If you use --config on the command line, we ignore every other
command line, your --config will overwrite everything else.
(5) Optional init script: graphios
----------------------------------
Remember: *screen* is not a daemon management tool.
If you installed with pip/setup.py/rpm this part should be done for you!
Take a look in the init/ directory and find your OS of choice.
For debian/ubuntu:
cp init/debian/graphios /etc/init.d/
cp init/debian/graphios.conf /etc/init
chmod 755 /etc/init.d/graphios
For rhel/centos/sl < 6:
cp init/rhel/graphios /etc/init.d
chmod 755 /etc/init.d/graphios
for systems with systemd:
cp init/systemd/graphios.service /usr/lib/systemd/system
#### NOTE: You may need to change the location and username that the script runs as. this varies slightly depending on where you decided to put graphios.py
The lines you will likely have to change:
<pre>
prog="/opt/nagios/bin/graphios.py"
# or use the command line options:
#prog="/opt/nagios/bin/graphios.py --log-file=/dir/mylog.log --spool-directory=/dir/my/sool"
GRAPHIOS_USER="nagios"
</pre>
(6) Your host and service configs
---------------------------------
Once you have done the above you need to add a custom variable to the hosts and
services that you want sent to graphite. (Unless you are using service
descriptions, in which case you can skip this step)
The format that will be sent to carbon is:
<pre>
_graphiteprefix.hostname._graphitepostfix.perfdata
</pre>
You do not need to set both graphiteprefix and graphitepostfix. Just one or the
other will do. If you do not set at least one of them, the data will not be
sent to graphite at all (unless you are using the service descriptions)
Examples:
<pre>
define host {
name myhost
check_command check_host_alive
_graphiteprefix monitoring.nagios01.pingto
}
</pre>
Which would create the following graphite entries with data from the check\_host\_alive plugin:
monitoring.nagios01.pingto.myhost.rta
monitoring.nagios01.pingto.myhost.rtmin
monitoring.nagios01.pingto.myhost.rtmax
monitoring.nagios01.pingto.myhost.pl
<pre>
define service {
service_description MySQL threads connected
host_name myhost
check_command check_mysql_health_threshold!threads-connected!3306!1600!1800
_graphiteprefix monitoring.nagios01.mysql
}
</pre>
Which gives us:
monitoring.nagios01.mysql.myhost.threads_connected
See the Documentation (above) for more explanation on how this works.
# Upgrading
To upgrade from the old version of graphios, you need to:
1. Look at the things you changed in the old graphios.py (carbon_server,
spool_directory, log_file location, etc)
2. Edit your new graphios.cfg and put those options there instead. You should
NOT have to modify the new graphios.py.
*Why Upgrade?*
The new version has fixed some bugs, and has cooler optional backends; and
support for multiple backends, including multiple carbon servers. I don't think
any major performance increases have been made, so if it isn't broken don't fix
it.
# PNP4Nagios Notes:
Are you already running pnp4nagios? And want to just try this out and see if
you like it? Cool! This is very easy to do without breaking your PNP4Nagios
configuration (but do a backup just in case).
Steps:
(1) In your nagios.cfg:
-----------------------
Add the following at the end of your:
<pre>
host_perfdata_file_template
\tGRAPHITEPREFIX::$_HOSTGRAPHITEPREFIX$\tGRAPHITEPOSTFIX::$_HOSTGRAPHITEPOSTFIX$
service_perfdata_file_template
\tGRAPHITEPREFIX::$_SERVICEGRAPHITEPREFIX$\tGRAPHITEPOSTFIX::$_SERVICEGRAPHITEPOSTFIX$
</pre>
This will add the variables to your check results, and will be ignored by pnp4nagios.
(2) Change your commands:
-------------------------
(find your command names under host\_perfdata\_file\_processing\_command and service\_perfdata\_file\_processing\_command in your nagios.cfg)
You likely have 2 commands setup that look something like these two:
<pre>
define command{
command_name process-service-perfdata-file
command_line /bin/mv /usr/local/pnp4nagios/var/service-perfdata /usr/local/pnp4nagios/var/spool/service-perfdata.$TIMET$
}
define command{
command_name process-host-perfdata-file
command_line /bin/mv /usr/local/pnp4nagios/var/host-perfdata /usr/local/pnp4nagios/var/spool/host-perfdata.$TIMET$
}
</pre>
Instead of just moving the file; move it then copy it, then we can point graphios at the copy.
You can do this by either:
(1) Change the command\_line to something like:
<pre>
command_line "/bin/mv /usr/local/pnp4nagios/var/host-perfdata /usr/local/pnp4nagios/var/spool/host-perfdata.$TIMET$ && cp /usr/local/pnp4nagios/var/spool/host-perfdata.$TIMET$ /usr/local/pnp4nagios/var/spool/graphios"
</pre>
OR
(2) Make a script:
<pre>
#!/bin/bash
/bin/mv /usr/local/pnp4nagios/var/host-perfdata /usr/local/pnp4nagios/var/spool/host-perfdata.$TIMET$
cp /usr/local/pnp4nagios/var/spool/host-perfdata.$TIMET$ /usr/local/pnp4nagios/var/spool/graphios
change the command_line to be:
command_line /path/to/myscript.sh
</pre>
You should now be able to skip steps 2 and 3 on the configuration instructions.
# OMD (Open Monitoring Distribution) Notes:
These instructions are for OMD >= 1.2x (including the current nightly builds).
__Note:__ All steps below are assumed to be carried out under your OMD site's user.
(1) Change PNP4NAGIOS to use "NPCD with Bulk Mode" instead of NPCDMOD. This is done by redirecting the symlink for pnp4nagios.cfg:
<pre>
ln -sf ~/etc/pnp4nagios/nagios_npcd.cfg ~/etc/nagios/nagios.d/pnp4nagios.cfg
</pre>
(2) Update ~/etc/pnp4nagios/nagios_npcd.cfg (remember to replace SITENAME).
<pre>
#
# PNP4Nagios Bulk Mode with npcd
#
process_performance_data=1
#
# service performance data
#
service_perfdata_file=/omd/sites/SITENAME/var/pnp4nagios/service-perfdata
service_perfdata_file_template=DATATYPE::SERVICEPERFDATA\tTIMET::$TIMET$\tHOSTNAME::$HOSTNAME$\tSERVICEDESC::$SERVICEDESC$\tSERVICEPERFDATA::$SERVICEPERFDATA$\tSERVICECHECKCOMMAND::$SERVICECHECKCOMMAND$\tHOSTSTATE::$HOSTSTATE$\tHOSTSTATETYPE::$HOSTSTATETYPE$\tSERVICESTATE::$SERVICESTATE$\tSERVICESTATETYPE::$SERVICESTATETYPE$\tGRAPHITEPREFIX::$_SERVICEGRAPHITEPREFIX$\tGRAPHITEPOSTFIX::$_SERVICEGRAPHITEPOSTFIX$
service_perfdata_file_mode=a
service_perfdata_file_processing_interval=15
service_perfdata_file_processing_command=omd-process-service-perfdata-file
#
# host performance data
#
host_perfdata_file=/omd/sites/SITENAME/var/pnp4nagios/host-perfdata
host_perfdata_file_template=DATATYPE::HOSTPERFDATA\tTIMET::$TIMET$\tHOSTNAME::$HOSTNAME$\tHOSTPERFDATA::$HOSTPERFDATA$\tHOSTCHECKCOMMAND::$HOSTCHECKCOMMAND$\tHOSTSTATE::$HOSTSTATE$\tHOSTSTATETYPE::$HOSTSTATETYPE$\tGRAPHITEPREFIX::$_HOSTGRAPHITEPREFIX$\tGRAPHITEPOSTFIX::$_HOSTGRAPHITEPOSTFIX$
host_perfdata_file_mode=a
host_perfdata_file_processing_interval=15
host_perfdata_file_processing_command=omd-process-host-perfdata-file
</pre>
(3) Update etc/nagios/conf.d/pnp4nagios.cfg (remember to replace SITENAME).
<pre>
define command{
command_name omd-process-service-perfdata-file
command_line /bin/mv /omd/sites/SITENAME/var/pnp4nagios/service-perfdata /omd/sites/SITENAME/var/pnp4nagios/spool/service-perfdata.$TIMET$ && cp /omd/sites/SITENAME/var/pnp4nagios/spool/service-perfdata.$TIMET$ /omd/sites/SITENAME/var/graphios/spool/
}
define command{
command_name omd-process-host-perfdata-file
command_line /bin/mv /omd/sites/SITENAME/var/pnp4nagios/host-perfdata /omd/sites/SITENAME/var/pnp4nagios/spool/host-perfdata.$TIMET$ && cp /omd/sites/SITENAME/var/pnp4nagios/spool/host-perfdata.$TIMET$ /omd/sites/SITENAME/var/graphios/spool/
}
</pre>
(4) Optional: If you don't want PNP4NAGIOS to ever see perfdata for checks that Graphios is exporting data for, you can modify the ~/etc/nagios/conf.d/pnp4nagios.cfg command lines to remove data with a grep. In the below case, we grep out a specific string (GRAPHITEPREFIX\:\:lustre) to remove perfdata containing that string. This involves a little move moving around of files, but nothing excessive and stops PNP4NAGIOS from trying to genearte RRD files with that data. (Again remember to change SITENAME).
<pre>
define command{
command_name omd-process-service-perfdata-file
#command_line /bin/mv /omd/sites/SITENAME/var/pnp4nagios/service-perfdata /omd/sites/SITENAME/var/pnp4nagios/spool/service-perfdata.$TIMET$
###GRAPHITE SETTING### ADDED REDIRECTION TO REMOVE exportstats
command_line /bin/mv /omd/sites/SITENAME/var/pnp4nagios/service-perfdata /omd/sites/SITENAME/var/pnp4nagios/service-perfdata.$TIMET$ && /bin/cp /omd/sites/SITENAME/var/pnp4nagios/service-perfdata.$TIMET$ /omd/sites/SITENAME/var/graphios/spool/ && grep -v GRAPHITEPREFIX\:\:lustre /omd/sites/SITENAME/var/pnp4nagios/service-perfdata.$TIMET$ > /omd/sites/SITENAME/var/pnp4nagios/spool/service-perfdata.$TIMET$ && /bin/rm /omd/sites/SITENAME/var/pnp4nagios/service-perfdata.*
}
define command{
command_name omd-process-host-perfdata-file
#command_line /bin/mv /omd/sites/SITENAME/var/pnp4nagios/host-perfdata /omd/sites/SITENAME/var/pnp4nagios/spool/host-perfdata.$TIMET$
####GRAPHITE SETTING### ADDED REDIRECTION TO REMOVE exportstats
command_line /bin/mv /omd/sites/SITENAME/var/pnp4nagios/host-perfdata /omd/sites/SITENAME/var/pnp4nagios/host-perfdata.$TIMET$ && /bin/cp /omd/sites/SITENAME/var/pnp4nagios/host-perfdata.$TIMET$ /omd/sites/SITENAME/var/graphios/spool/ && grep -v GRAPHITEPREFIX\:\:lustre /omd/sites/SITENAME/var/pnp4nagios/host-perfdata.$TIMET$ > /omd/sites/SITENAME/var/pnp4nagios/spool/host-perfdata.$TIMET$ && /bin/rm /omd/sites/SITENAME/var/pnp4nagios/host-perfdata.*
</pre>
# Check_MK Notes:
How to set custom variables for services and hosts using check_mk config files. (For OMD please don't overlook the notes above).
(1) For host perf data just create a new file named "extra_host_conf.mk" inside your check_mk conf.d dir.
<pre>
extra_host_conf["_graphiteprefix"] = [
( "DESIREDPREFIX.ping", ALL_HOSTS),
]
</pre>
(2) Run check_mk -O to updated and reload Nagios.
(3) Test via "check_mk -N hostname | less", to see if your prefix or postfix is there.
For service perf data create a file called, "extra_service_conf.mk". Remember you can use your host tags or any of kinds of tricks with check_mk config files.
<pre>
extra_service_conf["_graphiteprefix"] = [
( "DESIREDPREFIX.check_mk", ALL_HOSTS, ["Check_MK"]),
( "DESIREDPREFIX.cpu.load", ALL_HOSTS, ["CPU load"]),
]
</pre>
- - -
__Tip__: An easy way to produce graphite keys in the format: `$company.$server.$metric` is:
(1) Set `metric_base_path` to $company in `graphios.cfg`.
(2) In your 'extra' check_mk config files set your graphiteprefix to $metric, and set no graphiteprefix.
<pre>
extra_host_conf["_graphitepostfix"] = [
# e.g. mycompany.server123.ping
( "ping", ALL_HOSTS),
]
extra_service_conf["_graphitepostfix"] = [
# e.g. mycompany.server123.cpu.load
( "cpu.load", ALL_HOSTS, ["CPU load"]),
]
</pre>
# Trouble getting it working?
Many people are running graphios now (cool!), but if you are having trouble
getting it working let me know. I am not offering to teach you how to setup
Nagios, this is for intermediate+ nagios users. Email me at
shawn@systemtemplar.org and I will do what I can to help.
# Got it working?
Cool! Drop me a line and let me know how it goes.
# Find a bug?
Open an Issue on github and I will try to fix it asap.
# Contributing
I'm open to any feedback / patches / suggestions.
# Special Thanks
Special thanks to Dave Josephsen who added the multiple backend support and
worked with me to design and build the new version of graphios.
Shawn Sterling shawn@systemtemplar.org
================================================
FILE: graphios.cfg
================================================
# Graphios config file
[graphios]
#------------------------------------------------------------------------------
# Global Details (you need these!)
#------------------------------------------------------------------------------
# Character to use as replacement for invalid characters in metric names
replacement_character = _
# nagios spool directory
spool_directory = /var/spool/nagios/graphios
# graphios log info
log_file = /usr/local/nagios/var/graphios.log
# max log size in megabytes (it will rotate the files)
log_max_size = 24
# available log levels:
# DEBUG, INFO, WARNING, ERROR, CRITICAL
# see https://docs.python.org/2/library/logging.html#logging-levels for details
# DEBUG is quite verbose
#log_level = logging.DEBUG
log_level = logging.INFO
# Disable this once you get it working.
debug = True
# How long to sleep between processing the spool directory
sleep_time = 15
# when we can't connect to carbon, the sleeptime is doubled until we hit max
sleep_max = 480
# test mode makes it so we print what we would add to carbon, and not delete
# any files from the spool directory. log_level must be DEBUG as well.
test_mode = False
# use service description, most people will NOT want this, read documentation!
use_service_desc = False
# replace "." in nagios hostnames? (so "my.host.name" becomes "my_host_name")
# (uses the replacement_character)
replace_hostname = True
# reverse hostname
# if you have:
# host.datacenter.company.tld
# as your nagios hostname you may prefer to have your metric stored as:
# tld.company.datacenter.host
reverse_hostname = False
# This string will be universally pre-pended to metrics, regardless of whether
# or not _graphiteprefix is set. (Quotes not required).
# metric_base_path = mycorp.nagios
#------------------------------------------------------------------------------
# Carbon Details (comment out if not using carbon)
#------------------------------------------------------------------------------
enable_carbon = False
# Defaults to using the pickle protocol. Set to True to use the plaintext protocol.
carbon_plaintext = False
# Comma separated list of carbon server IP:Port 's
carbon_servers = 127.0.0.1:2004
# The max amount of metrics to send to the carbon server at a time (def:200)
#carbon_max_metrics = 200
#flag the carbon backend as 'non essential' for the purposes of error checking
#nerf_carbon = False
#------------------------------------------------------------------------------
# Statsd Details (comment in if you are using statsd)
#------------------------------------------------------------------------------
enable_statsd = False
# Comma separated list of statsd server IP:Port 's
statsd_servers = 127.0.0.1:8125
#flag the statsd backend as 'non essential' for the purposes of error checking
#nerf_statsd = False
#------------------------------------------------------------------------------
# librato Details (comment in if you are using librato)
#------------------------------------------------------------------------------
enable_librato = False
# your (required) librato credentials here:
#librato_email = <your email>
#librato_token = <your api token>
#### ZOMG SUPER IMPORTANT SETTING THAT WILL SAVE YOU MONEY #####
# json-formmated RE patterns that match the names of metrics you want to emit
# to Librato the default list is [".*"] (send everything).
# Example:
# librato_whitelist = ["load","rta","swap"]
librato_whitelist = [".*"]
#OPTIONAL BELOW HERE, LEAVE COMMENTED UNLESS YOU REALLY WANT IT CHANGED
# floor_time_secs: Floor samples to this time (set to graphios sleep_time)
#librato_floor_time_secs = 15
# comma separated list of Nagios Macros we use to construct the metric name:
# librato_namevals = GRAPHITEPREFIX,SERVICEDESC,GRAPHITEPOSTFIX,LABEL
# comma separated list of Nagios Macros we use to construct the source value :
# librato_sourcevals = HOSTNAME
#flag the librato backend as 'non essential' for the purposes of error checking
#nerf_librato = False
#------------------------------------------------------------------------------
# InfluxDB Details (if you are using InfluxDB 0.8)
#------------------------------------------------------------------------------
enable_influxdb = False
#------------------------------------------------------------------------------
# InfluxDB Details (if you are using InfluxDB 0.9)
# This will work a bit differently because of the addition of tags in
# InfluxDB 0.9. Now the metric will be named after the service description,
# the perfdata field will be a tag, and the host name will be a tag. The value
# of the metric is stored in the 'value' column.
# Requires use_service_desc = True.
#------------------------------------------------------------------------------
enable_influxdb09 = False
# Extra tags to add to metrics, like data center location etc.
# Only valid for 0.9
#influxdb_extra_tags = {"location": "la"}
# Comma separated list of server:ports
# defaults to 127.0.0.1:8086 (:8087 if using SSL).
#influxdb_servers = 127.0.0.1:8087
# SSL, defaults to False
#influxdb_use_ssl = True
# Database-name, defaults to nagios
#influxdb_db = <your influxdb-database>
# Credentials (required)
#influxdb_user = <your username>
#influxdb_password = <your password>
# Max metrics to send / request, defaults to 250
#influxdb_max_metrics = 500
# Flag the InfluxDB backend as 'non essential' for the purposes of error checking
#nerf_influxdb = False
# enable Line Protocol, defaults to False
#influxdb_line_protocol = True
#------------------------------------------------------------------------------
# STDOUT Details (comment in if you are using STDOUT)
#------------------------------------------------------------------------------
#comment the line below to disable the STDOUT sender
enable_stdout = False
#flag the stdout backend as 'non essential' for the purposes of error checking
nerf_stdout = True
================================================
FILE: graphios.py
================================================
#!/usr/bin/python -tt
# vim: set ts=4 sw=4 tw=79 et :
# Copyright (C) 2011 Shawn Sterling <shawn@systemtemplar.org>
#
# With contributions from:
#
# Juan Jose Presa <juanjop@gmail.com>
# Ranjib Dey <dey.ranjib@gmail.com>
# Ryan Davis <https://github.com/ryepup>
# Alexey Diyan <alexey.diyan@gmail.com>
# Steffen Zieger <me@saz.sh>
# Nathan Bird <ecthellion@gmail.com>
# Dave Josephsen <dave@skeptech.org>
# Emil Thelin <https://github.com/gummiboll>
#
# This program is free software; you can redistribute it and/or
# modify it under the terms of the GNU General Public License
# as published by the Free Software Foundation; either version 2
# of the License, or (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA
# 02110-1301, USA.
#
# graphios: this program will read nagios host and service perfdata, and
# send it to a carbon server.
#
# The latest version of this code will be found on my github page:
# https://github.com/shawn-sterling
from ConfigParser import SafeConfigParser
from optparse import OptionParser
import copy
import graphios_backends as backends
import logging
import logging.handlers
import os
import os.path
import re
import sys
import time
# ##########################################################
# ### Do not edit this file, edit the graphios.cfg #####
# nagios spool directory
spool_directory = '/var/spool/nagios/graphios'
# graphios log info
log_file = ''
log_max_size = 24
# by default we will check the current path for graphios.cfg, if config_file
# is passed as a command line argument we will use that instead.
config_file = ''
# This is overridden via config file
debug = False
# config dictionary
cfg = {}
# backend global
be = ""
# available loglevels for graphios.cfg
loglevels = {
'logging.DEBUG': logging.DEBUG,
'logging.INFO': logging.INFO,
'logging.WARNING': logging.WARNING,
'logging.ERROR': logging.ERROR,
'logging.CRITICAL': logging.CRITICAL
}
# options parsing
parser = OptionParser("""usage: %prog [options]
sends nagios performance data to carbon.
""")
parser.add_option('-v', "--verbose", action="store_true", dest="verbose",
help="sets logging to DEBUG level")
parser.add_option('-q', "--quiet", action="store_true", dest="quiet",
help="sets logging to WARNING level")
parser.add_option("--spool-directory", dest="spool_directory",
default=spool_directory,
help="where to look for nagios performance data")
parser.add_option("--log-file", dest="log_file",
default=log_file,
help="file to log to")
parser.add_option("--backend", dest="backend", default="stdout",
help="sets which storage backend to use")
parser.add_option("--config_file", dest="config_file", default="",
help="set custom config file location")
parser.add_option("--test", action="store_true", dest="test", default="",
help="Turns on test mode, which won't send to backends")
parser.add_option("--replace_char", dest="replace_char", default="_",
help="Replacement Character (default '_'")
parser.add_option("--sleep_time", dest="sleep_time", default=15,
help="How much time to sleep between checks")
parser.add_option("--sleep_max", dest="sleep_max", default=480,
help="Max time to sleep between runs")
parser.add_option("--server", dest="server", default="",
help="Server address (for backend)")
parser.add_option("--no_replace_hostname", action="store_false",
dest="replace_hostname", default=True,
help="Replace '.' in nagios hostnames, default on.")
parser.add_option("--reverse_hostname", action="store_true",
dest="reverse_hostname",
help="Reverse nagios hostname, default off.")
log = logging.getLogger('log')
class GraphiosMetric(object):
def __init__(self):
self.LABEL = '' # The name in the perfdata from nagios
self.VALUE = '' # The measured value of that metric
self.UOM = '' # The unit of measure for the metric
self.DATATYPE = '' # HOSTPERFDATA|SERVICEPERFDATA
self.METRICTYPE = 'gauge' # gauge|counter|timer etc..
self.TIMET = '' # Epoc time the measurement was taken
self.HOSTNAME = '' # name of th host measured
self.SERVICEDESC = '' # nagios configured service description
self.PERFDATA = '' # the space-delimited raw perfdata
self.SERVICECHECKCOMMAND = '' # literal check command syntax
self.HOSTCHECKCOMMAND = '' # literal check command syntax
self.HOSTSTATE = '' # current state afa nagios is concerned
self.HOSTSTATETYPE = '' # HARD|SOFT
self.SERVICESTATE = '' # current state afa nagios is concerned
self.SERVICESTATETYPE = '' # HARD|SOFT
self.METRICBASEPATH = '' # Establishes a root base path
self.GRAPHITEPREFIX = '' # graphios prefix
self.GRAPHITEPOSTFIX = '' # graphios suffix
self.VALID = False # if this metric is valid
if 'metric_base_path' in cfg:
self.METRICBASEPATH = cfg['metric_base_path']
def validate(self):
# because we eliminated all whitespace, there shouldn't be any quotes
# this happens more with windows nagios plugins
re.sub("'", "", self.LABEL)
re.sub('"', "", self.LABEL)
re.sub("'", "", self.VALUE)
re.sub('"', "", self.VALUE)
self.check_adjust_hostname()
if (
self.TIMET is not '' and
self.PERFDATA is not '' and
self.HOSTNAME is not ''
):
if "use_service_desc" in cfg and cfg["use_service_desc"] is True:
if self.SERVICEDESC != '' or self.DATATYPE == 'HOSTPERFDATA':
self.VALID = True
else:
# not using service descriptions
if (
# We should keep this logic and not check for a
# base path here. Just because there's a base path
# doesn't mean the metric should be considered valid
self.GRAPHITEPREFIX == "" and
self.GRAPHITEPOSTFIX == ""
):
self.VALID = False
else:
self.VALID = True
def check_adjust_hostname(self):
if cfg["reverse_hostname"]:
self.HOSTNAME = '.'.join(reversed(self.HOSTNAME.split('.')))
if cfg["replace_hostname"]:
self.HOSTNAME = self.HOSTNAME.replace(".",
cfg["replacement_character"])
def chk_bool(value):
"""
checks if value is a stringified boolean
"""
if (value.lower() == "true"):
return True
elif (value.lower() == "false"):
return False
return value
def read_config(config_file):
"""
reads the config file
"""
if config_file == '':
# check same dir as graphios binary
my_file = "%s/graphios.cfg" % sys.path[0]
if os.path.isfile(my_file):
config_file = my_file
else:
# check /etc/graphios/graphios.cfg
config_file = "/etc/graphios/graphios.cfg"
config = SafeConfigParser()
# The logger won't be initialized yet, so we use print_debug
if os.path.isfile(config_file):
config.read(config_file)
config_dict = {}
for section in config.sections():
# there should only be 1 'graphios' section
print_debug("section: %s" % section)
config_dict['name'] = section
for name, value in config.items(section):
config_dict[name] = chk_bool(value)
print_debug("config[%s]=%s" % (name, value))
# print config_dict
return config_dict
else:
print_debug("Can't open config file: %s" % config_file)
print """\nEither modify the script at the config_file = '' line and
specify where you want your config file to be, or create a config file
in the above directory (which should be the same dir the graphios.py is in)
or you can specify --config=myconfigfilelocation at the command line."""
sys.exit(1)
def verify_config(config_dict):
"""
verifies the required config variables are found
"""
global spool_directory
ensure_list = ['replacement_character', 'log_file', 'log_max_size',
'log_level', 'sleep_time', 'sleep_max', 'test_mode',
'reverse_hostname', 'replace_hostname']
missing_values = []
for ensure in ensure_list:
if ensure not in config_dict:
missing_values.append(ensure)
if len(missing_values) > 0:
print "\nMust have value in config file for:\n"
for value in missing_values:
print "%s\n" % value
sys.exit(1)
if not config_dict['log_level'] in loglevels.keys():
print "Unknown loglevel: " + config_dict['log_level'] + '\n'
print "Available loglevels:"
print '\n'.join(sorted(loglevels.keys()))
sys.exit(1)
if "spool_directory" in config_dict:
spool_directory = config_dict['spool_directory']
def print_debug(msg):
"""
prints a debug message if global debug is True
"""
if debug:
print msg
def verify_options(opts):
"""
verify the passed command line options, puts into global cfg
"""
global cfg
global spool_directory
# because these have defaults in the parser section we know they will be
# set. So we don't have to do a bunch of ifs.
if "log_file" not in cfg:
cfg["log_file"] = opts.log_file
if cfg["log_file"] == "''" or cfg["log_file"] == "":
cfg["log_file"] = "%s/graphios.log" % sys.path[0]
cfg["log_max_size"] = 24
if opts.verbose:
cfg["debug"] = True
cfg["log_level"] = "logging.DEBUG"
elif opts.quiet:
cfg["debug"] = False
cfg["log_level"] = "logging.WARNING"
else:
cfg["debug"] = False
cfg["log_level"] = "logging.INFO"
if opts.test:
cfg["test_mode"] = True
else:
cfg["test_mode"] = False
cfg["replacement_character"] = opts.replace_char
cfg["spool_directory"] = opts.spool_directory
cfg["sleep_time"] = opts.sleep_time
cfg["sleep_max"] = opts.sleep_max
cfg["replace_hostname"] = opts.replace_hostname
cfg["reverse_hostname"] = opts.reverse_hostname
spool_directory = opts.spool_directory
# cfg["backend"] = opts.backend
handle_backends(opts)
# cfg["enable_carbon"] = True
return cfg
def handle_backends(opts):
global cfg
if opts.backend == "carbon" or opts.backend == "statsd":
if not opts.server:
print "Must also have --server for carbon or statsd."
sys.exit(1)
if opts.backend == "carbon":
cfg["enable_carbon"] = True
cfg["carbon_servers"] = opts.server
if opts.backend == "statsd":
cfg["enable_statsd"] = True
cfg["statsd_server"] = opts.server
if opts.backend == "librato":
print "Use graphios.cfg for librato."
sys.exit(1)
def configure():
"""
sets up graphios config
"""
global debug
try:
cfg["log_max_size"] = int(cfg["log_max_size"])
except ValueError:
print "log_max_size needs to be a integer"
sys.exit(1)
# Convert cfg["log_max_size"] to bytes. Assume its already in bytes
# if its > 1000000
if cfg["log_max_size"] < 1000000:
log_max_bytes = cfg["log_max_size"]*1024*1024
else:
log_max_bytes = cfg["log_max_size"]
log_handler = logging.handlers.RotatingFileHandler(
cfg["log_file"], maxBytes=log_max_bytes, backupCount=4,
# encoding='bz2')
)
formatter = logging.Formatter(
"%(asctime)s %(filename)s %(levelname)s %(message)s",
"%B %d %H:%M:%S")
log_handler.setFormatter(formatter)
log.addHandler(log_handler)
if cfg.get("debug") is True or cfg['log_level'] == 'logging.DEBUG':
log.debug("adding streamhandler")
log.setLevel(logging.DEBUG)
log.addHandler(logging.StreamHandler())
debug = True
else:
log.setLevel(loglevels[cfg['log_level']])
debug = False
def process_log(file_name):
""" process log lines into GraphiosMetric Objects.
input is a tab delimited series of key/values each of which are delimited
by '::' it looks like:
DATATYPE::HOSTPERFDATA TIMET::1399738074 etc..
"""
processed_objects = [] # the final list of metric objects we'll return
graphite_lines = 0 # count the number of valid lines we process
try:
host_data_file = open(file_name, "r")
file_array = host_data_file.readlines()
host_data_file.close()
except (IOError, OSError) as ex:
log.critical("Can't open file:%s error: %s" % (file_name, ex))
sys.exit(2)
# parse each line into a metric object
for line in file_array:
if not re.search("^DATATYPE::", line):
continue
# log.debug('parsing: %s' % line)
graphite_lines += 1
variables = line.split('\t')
mobj = get_mobj(variables)
if mobj:
# break out the metric object into one object per perfdata metric
# log.debug('perfdata:%s' % mobj.PERFDATA)
for metric in mobj.PERFDATA.split():
try:
nobj = copy.copy(mobj)
(nobj.LABEL, d) = metric.split('=')
v = d.split(';')[0]
u = v
nobj.VALUE = re.sub("[a-zA-Z%]", "", v)
nobj.UOM = re.sub("[^a-zA-Z]+", "", u)
processed_objects.append(nobj)
except:
log.critical("failed to parse label: '%s' part of perf"
"string '%s'" % (metric, nobj.PERFDATA))
continue
return processed_objects
def get_mobj(nag_array):
"""
takes a split array of nagios variables and returns a mobj if it's
valid. otherwise return False.
"""
mobj = GraphiosMetric()
for var in nag_array:
# drop the metric if we can't split it for any reason
try:
(var_name, value) = var.split('::', 1)
except:
log.warn("could not split value %s, dropping metric" % var)
return False
value = re.sub("/", cfg["replacement_character"], value)
if re.search("PERFDATA", var_name):
mobj.PERFDATA = value
elif re.search("^\$_", value):
continue
else:
value = re.sub("\s", "", value)
setattr(mobj, var_name, value)
mobj.validate()
if mobj.VALID is True:
return mobj
return False
def handle_file(file_name, graphite_lines):
"""
archive processed metric lines and delete the input log files
"""
if "test_mode" in cfg and cfg["test_mode"] is True:
log.debug("graphite_lines:%s" % graphite_lines)
else:
try:
os.remove(file_name)
except (OSError, IOError) as ex:
log.critical("couldn't remove file %s error:%s" % (file_name, ex))
else:
log.debug("deleted %s" % file_name)
def process_spool_dir(directory):
"""
processes the files in the spool directory
"""
global be
log.debug("Processing spool directory %s", directory)
num_files = 0
mobjs_len = 0
try:
perfdata_files = os.listdir(directory)
except (IOError, OSError) as e:
print "Exception '%s' reading spool directory: %s" % (e, directory)
print "Check if dir exists, or file permissions."
print "Exiting."
sys.exit(1)
for perfdata_file in perfdata_files:
mobjs = []
processed_dict = {}
all_done = True
file_dir = os.path.join(directory, perfdata_file)
if check_skip_file(perfdata_file, file_dir):
continue
num_files += 1
mobjs = process_log(file_dir)
mobjs_len = len(mobjs)
processed_dict = send_backends(mobjs)
# process the output from the backends and decide the fate of the file
for backend in be["essential_backends"]:
if processed_dict[backend] < mobjs_len:
log.critical("keeping %s, insufficent metrics sent from %s. \
Should be %s, got %s" % (file_dir, backend,
mobjs_len,
processed_dict[backend]))
all_done = False
if all_done is True:
handle_file(file_dir, len(mobjs))
log.info("Processed %s files (%s metrics) in %s" % (num_files,
mobjs_len, directory))
def check_skip_file(file_name, file_dir):
"""
checks if file should be skipped
"""
if (
file_name == "host-perfdata" or
file_name == "service-perfdata"
):
return True
elif re.match('^_', file_name):
return True
if os.stat(file_dir)[6] == 0:
# file was 0 bytes
handle_file(file_dir, 0)
return True
if os.path.isdir(file_dir):
return True
return False
def init_backends():
"""
I'm going to be a little forward thinking with this and build a global dict
of enabled back-ends whose values are instantiations of the back-end
objects themselves. I know, global bad, but hypothetically we could modify
this dict dynamically via a runtime-interface (graphiosctl?) to turn off/on
backends without having to restart the graphios process. I feel like
that's enough of a win to justify the global. If you build it, they will
come.
"""
global be
be = {} # a top-level global for important backend-related stuff
be["enabled_backends"] = {} # a dict of instantiated backend objects
be["essential_backends"] = [] # a list of backends we actually care about
# PLUGIN WRITERS! register your new backends by adding their obj name here
avail_backends = ("carbon",
"statsd",
"librato",
"influxdb",
"influxdb09",
"stdout",
)
# populate the controller dict from avail + config. this assumes you named
# your backend the same as the config option that enables your backend (eg.
# carbon and enable_carbon)
for backend in avail_backends:
cfg_option = "enable_%s" % (backend)
if cfg_option in cfg and cfg[cfg_option] is True:
backend_obj = getattr(backends, backend)
be["enabled_backends"][backend] = backend_obj(cfg)
nerf_option = "nerf_%s" % (backend)
if nerf_option in cfg:
if cfg[nerf_option] is False:
be["essential_backends"].append(backend)
else:
be["essential_backends"].append(backend)
# not proud of that slovenly conditional ^^
log.info("Enabled backends: %s" % be["enabled_backends"].keys())
def send_backends(metrics):
"""
use the enabled_backends dict to call into the backend send functions
"""
global be
if len(be["enabled_backends"]) < 1:
log.critical("At least one Back-end must be enabled in graphios.cfg")
sys.exit(1)
ret = {} # return a dict of who processed what
processed_lines = 0
for backend in be["enabled_backends"]:
processed_lines = be["enabled_backends"][backend].send(metrics)
# log.debug('%s processed %s metrics' % backend, processed_lines)
ret[backend] = processed_lines
return ret
def main():
log.info("graphios startup.")
try:
while True:
process_spool_dir(spool_directory)
log.debug("graphios sleeping.")
time.sleep(float(cfg["sleep_time"]))
except KeyboardInterrupt:
log.info("ctrl-c pressed. Exiting graphios.")
if __name__ == '__main__':
if len(sys.argv) > 1:
(options, args) = parser.parse_args()
# print options
if options.config_file:
cfg = read_config(options.config_file)
else:
cfg = verify_options(options)
else:
cfg = read_config(config_file)
verify_config(cfg)
configure()
# print cfg
init_backends()
main()
================================================
FILE: graphios_backends.py
================================================
# vim: set ts=4 sw=4 tw=79 et :
import socket
import cPickle as pickle
import struct
import re
import logging
import sys
import base64
import urllib2
import json
import os
import ast
# ###########################################################
# #### Librato Backend
class librato(object):
def __init__(self, cfg):
"""
Implements the librato backend-module
"""
self.log = logging.getLogger("log.backends.librato")
self.log.info("Librato Backend Initialized")
self.api = "https://metrics-api.librato.com"
self.sink_name = "graphios-librato"
self.sink_version = "0.0.1"
self.flush_timeout_secs = 5
self.gauges = {}
self.whitelist = []
self.metrics_sent = 0
self.max_metrics_payload = 500
try:
cfg["librato_email"]
except:
self.log.critical("please define librato_email in graphios.cfg")
sys.exit(1)
else:
self.email = cfg['librato_email']
try:
cfg["librato_token"]
except:
self.log.critical("please define librato_token in graphios.cfg")
sys.exit(1)
else:
self.token = cfg['librato_token']
try:
cfg['librato_namevals']
except:
self.namevals = ['METRICBASEPATH', 'GRAPHITEPREFIX', 'SERVICEDESC',
'GRAPHITEPOSTFIX', 'LABEL']
else:
self.namevals = cfg['librato_namevals'].split(",")
try:
cfg['librato_sourcevals']
except:
self.sourcevals = ['HOSTNAME']
else:
self.sourcevals = cfg['librato_sourcevals'].split(",")
try:
cfg["librato_floor_time_secs"]
except:
self.floor_time_secs = 15
else:
self.floor_time_secs = cfg["librato_floor_time_secs"]
try:
cfg["librato_whitelist"]
except:
self.whitelist = [re.compile(".*")]
else:
for pattern in json.loads(cfg["librato_whitelist"]):
self.log.debug("adding librato whitelist pattern %s" % pattern)
self.whitelist.append(re.compile(pattern))
def build_path(self, vals, m):
path = ''
for s in vals:
path += getattr(m, s)
path += '.'
path = re.sub(r"^\.", '', path) # fix sources that begin in dot
path = re.sub(r"\.$", '', path) # fix sources that end in dot
path = re.sub(r"\.\.", '.', path) # fix sources with double dots
path = re.sub(r"['\"]", '.', path) # fix sources with imbedded quotes
return path
def k_not_in_whitelist(self, k):
# return True if k isn't whitelisted
# wl_match = True
for pattern in self.whitelist:
if pattern.search(k) is not None:
return False
return True
def add_measure(self, m):
ts = int(m.TIMET)
if self.floor_time_secs is not None:
ts = (ts / self.floor_time_secs) * self.floor_time_secs
source = self.build_path(self.sourcevals, m)
name = self.build_path(self.namevals, m)
k = "%s\t%s" % (name, source)
# bail if this metric isn't whitelisted
if self.k_not_in_whitelist(k):
return None
# add the metric to our gauges dict
if k not in self.gauges:
self.gauges[k] = {
'name': name,
'source': source,
'measure_time': ts,
}
value = float(m.VALUE)
self.gauges[k]['value'] = value
def flush_payload(self, headers, g):
"""
POST a payload to Librato.
"""
body = json.dumps({'gauges': g})
url = "%s/v1/metrics" % (self.api)
req = urllib2.Request(url, body, headers)
try:
f = urllib2.urlopen(req, timeout=self.flush_timeout_secs)
# f.read()
f.close()
except urllib2.HTTPError as error:
self.metrics_sent = 0
body = error.read()
self.log.warning('Failed to send metrics to Librato: Code: \
%d . Response: %s' % (error.code, body))
except IOError as error:
self.metrics_sent = 0
if hasattr(error, 'reason'):
self.log.warning('Error when sending metrics Librato \
(%s)' % (error.reason))
elif hasattr(error, 'code'):
self.log.warning('Error when sending metrics Librato \
(%s)' % (error.code))
else:
self.log.warning('Error when sending metrics Librato \
and I dunno why')
def flush(self):
"""
POST a collection of gauges to Librato.
"""
# Nothing to do
if len(self.gauges) == 0:
return 0
# Limit our payload sizes
# max_metrics_payload = 500 # this is never used, delete it?
headers = {
'Content-Type': 'application/json',
'User-Agent': self.build_user_agent(),
'Authorization': 'Basic %s' % self.build_basic_auth()
}
metrics = []
count = 0
for g in self.gauges.values():
metrics.append(g)
count += 1
if count >= self.max_metrics_payload:
self.flush_payload(headers, metrics)
count = 0
metrics = []
if count > 0:
self.flush_payload(headers, metrics)
self.gauges = {}
def build_basic_auth(self):
base64string = base64.encodestring('%s:%s' % (self.email, self.token))
return base64string.translate(None, '\n')
def build_user_agent(self):
sink_name = "graphios-librato"
sink_version = "0.0.1"
try:
uname = os.uname()
system = "; ".join([uname[0], uname[4]])
except:
system = os.name()
pver = sys.version_info
user_agent = '%s/%s (%s) Python-Urllib2/%d.%d' % \
(sink_name, sink_version,
system, pver[0], pver[1])
return user_agent
def send(self, metrics):
self.metrics_sent = len(metrics)
# Construct the output
for m in metrics:
self.add_measure(m)
# Flush
self.flush()
return self.metrics_sent
############################################################
# #### Carbon back-end #####
class carbon(object):
def __init__(self, cfg):
self.log = logging.getLogger("log.backends.carbon")
self.log.info("Carbon Backend Initialized")
try:
cfg['carbon_servers']
except:
self.carbon_servers = '127.0.0.1'
else:
self.carbon_servers = cfg['carbon_servers']
try:
cfg['replacement_character']
except:
self.replacement_character = '_'
else:
self.replacement_character = cfg['replacement_character']
try:
cfg['carbon_max_metrics']
self.carbon_max_metrics = cfg['carbon_max_metrics']
except:
self.carbon_max_metrics = 200
try:
self.carbon_max_metrics = int(self.carbon_max_metrics)
except ValueError:
self.log.critical("carbon_max_metrics needs to be a integer")
sys.exit(1)
try:
cfg['use_service_desc']
self.use_service_desc = cfg['use_service_desc']
except:
self.use_service_desc = False
try:
cfg['metric_base_path']
self.metric_base_path = cfg['metric_base_path']
except:
self.metric_base_path = ''
try:
cfg['test_mode']
self.test_mode = cfg['test_mode']
except:
self.test_mode = False
# try:
# cfg['replace_hostname']
# self.replace_hostname = cfg['replace_hostname']
# except:
# self.replace_hostname = True
try:
cfg['carbon_plaintext']
self.carbon_plaintext = cfg['carbon_plaintext']
except:
self.carbon_plaintext = False
def convert_messages(self, metrics):
"""
Converts the metric obj list into graphite messages
"""
metric_list = []
messages = []
for m in metrics:
path = self.build_path(m)
value = m.VALUE
timestamp = m.TIMET
if self.carbon_plaintext:
metric_item = "%s %s %s\n" % (path, value, timestamp)
else:
metric_item = (path, (timestamp, value))
if self.test_mode:
print "%s %s %s" % (path, value, timestamp)
metric_list.append(metric_item)
for metric_list_chunk in self.chunks(metric_list,
self.carbon_max_metrics):
if self.carbon_plaintext:
messages.append("".join(metric_list_chunk))
else:
payload = pickle.dumps(metric_list_chunk)
header = struct.pack("!L", len(payload))
message = header + payload
messages.append(message)
return messages
def chunks(self, l, n):
""" Yield successive n-sized chunks from l.
"""
for i in xrange(0, len(l), n):
yield l[i:i + n]
def build_path(self, m):
"""
Builds a carbon metric
"""
pre = ""
if m.METRICBASEPATH != "":
pre = "%s." % m.METRICBASEPATH
if m.GRAPHITEPREFIX != "":
pre += "%s." % m.GRAPHITEPREFIX
if m.GRAPHITEPOSTFIX != "":
post = ".%s" % m.GRAPHITEPOSTFIX
else:
post = ""
# if self.replace_hostname:
# hostname = m.HOSTNAME.replace('.', self.replacement_character)
# else:
hostname = m.HOSTNAME
if self.use_service_desc:
# we want: (prefix.)hostname.service_desc(.postfix).perfdata
service_desc = self.fix_string(m.SERVICEDESC)
path = "%s%s.%s%s.%s" % (pre, hostname, service_desc, post,
m.LABEL)
else:
path = "%s%s%s.%s" % (pre, hostname, post, m.LABEL)
path = re.sub(r"\.$", '', path) # fix paths that end in dot
path = re.sub(r"\.\.", '.', path) # fix paths with double dots
path = self.fix_string(path)
return path
def fix_string(self, my_string):
"""
takes a string and replaces whitespace and invalid carbon chars with
the global replacement_character
"""
invalid_chars = '~!$:;%^*()+={}[]|\/<>'
my_string = re.sub("\s", self.replacement_character, my_string)
for char in invalid_chars:
my_string = my_string.replace(char, self.replacement_character)
return my_string
def send(self, metrics):
"""
Connect to the Carbon server
Send the metrics
"""
ret = 0
sock = socket.socket()
servers = self.carbon_servers.split(",")
for serv in servers:
if ":" in serv:
server, port = serv.split(":")
port = int(port)
else:
server = serv
if self.carbon_plaintext:
port = 2003
else:
port = 2004
self.log.debug("Connecting to carbon at %s:%s" % (server, port))
try:
sock.connect((socket.gethostbyname(server), port))
self.log.debug("connected")
except Exception, ex:
self.log.warning("Can't connect to carbon: %s:%s %s" % (
server, port, ex))
messages = self.convert_messages(metrics)
try:
for message in messages:
sock.sendall(message)
except Exception, ex:
self.log.critical("Can't send message to carbon error:%s" % ex)
sock.close()
return 0
# this only gets returned if nothing failed.
ret += len(metrics)
sock.close()
return ret
# ###########################################################
# #### statsd backend #######################################
class statsd(object):
def __init__(self, cfg):
self.log = logging.getLogger("log.backends.statsd")
self.log.info("Statsd backend initialized")
try:
cfg['statsd_servers']
except:
self.statsd_servers = '127.0.0.1'
else:
self.statsd_servers = cfg['statsd_servers']
def set_type(self, metric):
# detect and set the metric type
if re.search("gauge", metric.METRICTYPE):
return 'g'
elif re.search("counter", metric.METRICTYPE):
return 'c'
elif re.search("time", metric.METRICTYPE):
return 'ms'
elif re.search("set", metric.METRICTYPE):
return 's'
else:
return 'g' # default to gauge
def convert(self, metrics):
# Converts the metric object list into a list of statsd tuples
out_list = []
for m in metrics:
path = '%s.%s.%s.%s.%s' % (m.METRICBASEPATH, m.GRAPHITEPREFIX,
m.HOSTNAME, m.GRAPHITEPOSTFIX,
m.LABEL)
path = re.sub(r'\.$', '', path) # fix paths that end in dot
path = re.sub(r'\.\.', '.', path) # fix paths with empty values
mtype = self.set_type(m) # gauge|counter|timer|set
value = "%s|%s" % (m.VALUE, mtype) # emit literally this to statsd
metric_tuple = "%s:%s" % (path, value)
out_list.append(metric_tuple)
return out_list
def send(self, metrics):
# Fire metrics at the statsd server and hope for the best (loludp)
ret = True
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
mlist = self.convert(metrics)
ret = 0
servers = self.statsd_servers.split(",")
for serv in servers:
if ":" in serv:
server, port = serv.split(":")
port = int(port)
else:
server = serv
port = 8125
self.log.debug("sending to statsd at %s:%s" % (server, port))
for m in mlist:
try:
sock.sendto(m, (socket.gethostbyname(server), port))
except Exception, ex:
self.log.critical("Can't send metric to statsd error:%s"
% ex)
else:
ret += 1
return ret
# ###########################################################
# #### influxdb backend ####################################
class influxdb(object):
def __init__(self, cfg):
self.log = logging.getLogger("log.backends.influxdb")
self.log.info("InfluxDB backend initialized")
self.scheme = "http"
self.default_ports = {'https': 8087, 'http': 8086}
self.timeout = 5
if 'influxdb_use_ssl' in cfg:
if cfg['influxdb_use_ssl']:
self.scheme = "https"
if 'influxdb_servers' in cfg:
self.influxdb_servers = cfg['influxdb_servers'].split(',')
else:
self.influxdb_servers = ['127.0.0.1:%i' %
self.default_ports[self.scheme]]
if 'influxdb_user' in cfg:
self.influxdb_user = cfg['influxdb_user']
else:
self.log.critical("Missing influxdb_user in graphios.cfg")
sys.exit(1)
if 'influxdb_password' in cfg:
self.influxdb_password = cfg['influxdb_password']
else:
self.log.critical("Missing influxdb_password in graphios.cfg")
sys.exit(1)
if 'influxdb_db' in cfg:
self.influxdb_db = cfg['influxdb_db']
else:
self.influxdb_db = "nagios"
if 'influxdb_max_metrics' in cfg:
self.influxdb_max_metrics = cfg['influxdb_max_metrics']
else:
self.influxdb_max_metrics = 250
try:
self.influxdb_max_metrics = int(self.influxdb_max_metrics)
except ValueError:
self.log.critical("influxdb_max_metrics needs to be a integer")
sys.exit(1)
def build_url(self, server):
""" Returns a url to specified InfluxDB-server """
test_port = server.split(':')
if len(test_port) < 2:
server = "%s:%i" % (server, self.default_ports[self.scheme])
return "%s://%s/db/%s/series?u=%s&p=%s" % (self.scheme, server,
self.influxdb_db,
self.influxdb_user,
self.influxdb_password)
def build_path(self, m):
""" Returns a path """
path = ""
if m.METRICBASEPATH != "":
path += "%s." % m.METRICBASEPATH
if m.GRAPHITEPREFIX != "":
path += "%s." % m.GRAPHITEPREFIX
path += "%s." % m.HOSTNAME
if m.SERVICEDESC != "":
path += "%s." % m.SERVICEDESC
path = "%s%s" % (path, m.LABEL)
if m.GRAPHITEPOSTFIX != "":
path = "%s.%s" % (path, m.GRAPHITEPOSTFIX)
return path
def chunks(self, l, n):
""" Yield successive n-sized chunks from l. """
for i in xrange(0, len(l), n):
yield l[i:i+n]
def url_request(self, url, chunk):
json_body = json.dumps(chunk)
req = urllib2.Request(url, json_body)
req.add_header('Content-Type', 'application/json')
return req
def _send(self, server, chunk):
self.log.debug("Connecting to InfluxDB at %s" % server)
req = self.url_request(self.build_url(server), chunk)
try:
r = urllib2.urlopen(req, timeout=self.timeout)
r.close()
return True
except urllib2.HTTPError as e:
body = e.read()
self.log.warning('Failed to send metrics to InfluxDB. \
Status code: %d: %s' % (e.code, body))
return False
except IOError as e:
fail_string = "Failed to send metrics to InfluxDB. "
if hasattr(e, 'code'):
fail_string = fail_string + "Status code: %s" % e.code
if hasattr(e, 'reason'):
fail_string = fail_string + str(e.reason)
self.log.warning(fail_string)
return False
def send(self, metrics):
""" Connect to influxdb and send metrics """
ret = 0
perfdata = {}
series = []
for m in metrics:
ret += 1
path = self.build_path(m)
if path not in perfdata:
perfdata[path] = []
# influx assumes timestamp in milliseconds
timet_ms = int(m.TIMET)*1000
# Ensure a int/float gets passed
try:
value = int(m.VALUE)
except ValueError:
try:
value = float(m.VALUE)
except ValueError:
value = 0
perfdata[path].append([timet_ms, value])
for k, v in perfdata.iteritems():
series.append({"name": k, "columns": ["time", "value"],
"points": v})
series_chunks = self.chunks(series, self.influxdb_max_metrics)
for chunk in series_chunks:
for s in self.influxdb_servers:
if not self._send(s, chunk):
ret = 0
return ret
# ###########################################################
# #### influxdb-0.9 backend ####################################
class influxdb09(influxdb):
def __init__(self, cfg):
influxdb.__init__(self, cfg)
if 'influxdb_extra_tags' in cfg:
self.influxdb_extra_tags = ast.literal_eval(
cfg['influxdb_extra_tags'])
print self.influxdb_extra_tags
else:
self.influxdb_extra_tags = {}
try:
cfg['influxdb_line_protocol']
self.influxdb_line_protocol = cfg['influxdb_line_protocol']
except:
self.influxdb_line_protocol = False
def build_url(self, server):
""" Returns a url to specified InfluxDB-server """
test_port = server.split(':')
if len(test_port) < 2:
server = "%s:%i" % (server, self.default_ports[self.scheme])
if self.influxdb_line_protocol:
return "%s://%s/write?u=%s&p=%s&db=%s" % (self.scheme, server,
self.influxdb_user,
self.influxdb_password,
self.influxdb_db)
else:
return "%s://%s/write?u=%s&p=%s" % (self.scheme, server,
self.influxdb_user,
self.influxdb_password)
def url_request(self, url, chunk):
if self.influxdb_line_protocol:
req = urllib2.Request(url, chunk)
req.add_header('Content-Type', 'application/x-www-form-urlencoded')
return req
else:
return super(influxdb09, self).url_request(url, chunk)
def format_metric(self, timestamp, path, tags, value):
if not self.influxdb_line_protocol:
return {
"timestamp": timestamp,
"measurement": path,
"tags": tags,
"fields": {"value": value}}
return '%s,%s value=%s %d' % (
path,
','.join(['%s=%s' % (k, v) for k, v in tags.iteritems() if v]),
value,
timestamp * 10 ** 9
)
def format_series(self, chunk):
if self.influxdb_line_protocol:
return '\n'.join(chunk)
else:
return {"database": self.influxdb_db, "points": chunk}
def send(self, metrics):
""" Connect to influxdb and send metrics """
ret = 0
perfdata = []
for m in metrics:
ret += 1
if (m.SERVICEDESC == ''):
path = m.HOSTCHECKCOMMAND
else:
path = m.SERVICEDESC
# Ensure a int/float gets passed
try:
value = int(m.VALUE)
except ValueError:
try:
value = float(m.VALUE)
except ValueError:
value = 0
tags = {"check": m.LABEL, "host": m.HOSTNAME}
tags.update(self.influxdb_extra_tags)
perfdata.append(self.format_metric(int(m.TIMET), path,
tags, value))
series_chunks = self.chunks(perfdata, self.influxdb_max_metrics)
for chunk in series_chunks:
series = self.format_series(chunk)
for s in self.influxdb_servers:
if not self._send(s, series):
ret = 0
return ret
# ###########################################################
# #### stdout backend #######################################
class stdout(object):
def __init__(self, cfg):
self.log = logging.getLogger("log.backends.stdout")
self.log.info("STDOUT Backend Initialized")
def send(self, metrics):
ret = 0
for metric in metrics:
ret += 1
print("%s:%s" % ('LABEL', metric.LABEL))
print("%s:%s" % ('VALUE ', metric.VALUE))
print("%s:%s" % ('UOM ', metric.UOM))
print("%s:%s" % ('DATATYPE ', metric.DATATYPE))
print("%s:%s" % ('TIMET ', metric.TIMET))
print("%s:%s" % ('HOSTNAME ', metric.HOSTNAME))
print("%s:%s" % ('SERVICEDESC ', metric.SERVICEDESC))
print("%s:%s" % ('PERFDATA ', metric.PERFDATA))
print("%s:%s" % ('SERVICECHECKCOMMAND',
metric.SERVICECHECKCOMMAND))
print("%s:%s" % ('HOSTCHECKCOMMAND ', metric.HOSTCHECKCOMMAND))
print("%s:%s" % ('HOSTSTATE ', metric.HOSTSTATE))
print("%s:%s" % ('HOSTSTATETYPE ', metric.HOSTSTATETYPE))
print("%s:%s" % ('SERVICESTATE ', metric.SERVICESTATE))
print("%s:%s" % ('SERVICESTATETYPE ', metric.SERVICESTATETYPE))
print("%s:%s" % ('METRICBASEPATH ', metric.METRICBASEPATH))
print("%s:%s" % ('GRAPHITEPREFIX ', metric.GRAPHITEPREFIX))
print("%s:%s" % ('GRAPHITEPOSTFIX ', metric.GRAPHITEPOSTFIX))
print("-------")
return ret
# ###########################################################
# #### start here #######################################
if __name__ == "__main__":
print("I'm just a lowly module. Try calling graphios.py instead")
sys.exit(42)
================================================
FILE: init/debian/graphios
================================================
#! /bin/bash
### BEGIN INIT INFO
# Provides: graphios
# Required-Start: $local_fs $remote_fs $syslog $named $network $time nagios3
# Required-Stop: $local_fs $remote_fs $syslog $named $network
# Should-Start:
# Should-Stop:
# Default-Start: 2 3 4 5
# Default-Stop: 0 1 6
# Short-Description: graphios bridge between nagios and graphite
# Description: graphios feeds nagios performance data to graphite
### END INIT INFO
# File : graphios
# Source function library.
. /lib/lsb/init-functions
prog="/usr/local/bin/graphios.py"
# or use the command line options:
#prog="/opt/nagios/bin/graphios.py --log-file=/dir/mylog.log --spool-directory=/dir/my/sool"
GRAPHIOS_USER="nagios"
NAME="graphios"
DESC="nagios to graphite bridge"
RETVAL=0
start () {
log_daemon_msg "Starting $DESC" "$NAME"
start-stop-daemon --start --quiet --name $(basename $prog) --user $GRAPHIOS_USER \
--chuid $GRAPHIOS_USER --exec $prog --background
log_end_msg $?
echo
}
stop () {
log_daemon_msg "Stopping $DESC" "$NAME"
start-stop-daemon --stop --oknodo --quiet --name $(basename $prog) --user $GRAPHIOS_USER
log_end_msg $?
echo
}
restart () {
stop
start
}
# See how we are called.
case "$1" in
start)
start
;;
stop)
stop
;;
restart|reload)
restart
;;
status)
status $prog
RETVAL=$?
;;
*)
echo "Usage: service graphios {start|stop|restart|reload}"
RETVAL=2
;;
esac
exit $RETVAL
================================================
FILE: init/debian/graphios.conf
================================================
description "Graphios: Emit nagios perfdata to graphite/statsd/librato"
author "Shawn Sterling <shawn@systemtemplar.org>"
start on runlevel [234]
stop on runlevel [016]
exec /usr/local/bin/graphios.py
================================================
FILE: init/rhel/graphios
================================================
#!/bin/bash
#
# graphios start the graphios script
#
#
# chkconfig: 345 99 01
# description: graphios nagios -> graphite script
#
# File : graphios
# Source function library.
. /etc/init.d/functions
# Source networking configuration.
. /etc/sysconfig/network
# Check that networking is up.
[ "$NETWORKING" = "no" ] && exit 0
prog="/usr/bin/graphios"
# or use the command line options:
#prog="/usr/bin/graphios --log-file=/dir/mylog.log --spool-directory=/dir/my/sool"
GRAPHIOS_USER="nagios"
RETVAL=0
start () {
echo -n "Starting $prog"
/usr/bin/sudo -u ${GRAPHIOS_USER} ${prog} &
RETVAL=$?
[ $RETVAL -eq 0 ] && success || failure
echo
}
stop () {
echo -n "Stopping $prog"
killproc graphios
RETVAL=$?
[ $RETVAL -eq 0 ] && success || failure
echo
}
restart () {
stop
start
}
# See how we are called.
case "$1" in
start)
start
;;
stop)
stop
;;
restart|reload)
restart
;;
status)
status $prog
RETVAL=$?
;;
*)
echo "Usage: service graphios {start|stop|restart|reload}"
RETVAL=2
;;
esac
exit $RETVAL
================================================
FILE: init/rhel/install
================================================
python setup.py install --optimize 1 --root=$RPM_BUILD_ROOT --record=INSTALLED_FILES
================================================
FILE: init/rhel/postinstall
================================================
# add symlink
ln -s /usr/bin/graphios.py /usr/bin/graphios
# add graphios service
if [ ! -d /etc/systemd ]; then
# we are running init
if ! chkconfig --add graphios; then
logger -p user.err -s -t %name -- "Error adding graphios service."
exit 1
fi
fi
================================================
FILE: init/rhel/postuninstall
================================================
if [ ! -d /etc/systemd ]; then
# we are running init
service graphios stop
else
# we are running systemd
systemctl stop graphios.service
fi
unlink /usr/bin/graphios
================================================
FILE: init/systemd/graphios.service
================================================
[Unit]
Description=graphios - a script to emit nagios perfdata to various backends
[Service]
ExecStart=/usr/bin/graphios.py
Restart=on-abort
[Install]
WantedBy=multi-user.target
================================================
FILE: nagios/graphios_commands.cfg
================================================
define command {
command_name graphios_perf_host
command_line /bin/mv /var/spool/nagios/graphios/host-perfdata /var/spool/nagios/graphios/host-perfdata.$TIMET$
}
define command {
command_name graphios_perf_service
command_line /bin/mv /var/spool/nagios/graphios/service-perfdata /var/spool/nagios/graphios/service-perfdata.$TIMET$
}
================================================
FILE: nagios/nagios_perfdata.cfg
================================================
###### Auto-generated Graphios configs #######
process_performance_data=1
service_perfdata_file=/var/spool/nagios/graphios/service-perfdata
service_perfdata_file_template=DATATYPE::SERVICEPERFDATA\tTIMET::$TIMET$\tHOSTNAME::$HOSTNAME$\tSERVICEDESC::$SERVICEDESC$\tSERVICEPERFDATA::$SERVICEPERFDATA$\tSERVICECHECKCOMMAND::$SERVICECHECKCOMMAND$\tHOSTSTATE::$HOSTSTATE$\tHOSTSTATETYPE::$HOSTSTATETYPE$\tSERVICESTATE::$SERVICESTATE$\tSERVICESTATETYPE::$SERVICESTATETYPE$\tGRAPHITEPREFIX::$_SERVICEGRAPHITEPREFIX$\tGRAPHITEPOSTFIX::$_SERVICEGRAPHITEPOSTFIX$\tMETRICTYPE::$_SERVICEMETRICTYPE$
service_perfdata_file_mode=a
service_perfdata_file_processing_interval=15
service_perfdata_file_processing_command=graphios_perf_service
host_perfdata_file=/var/spool/nagios/graphios/host-perfdata
host_perfdata_file_template=DATATYPE::HOSTPERFDATA\tTIMET::$TIMET$\tHOSTNAME::$HOSTNAME$\tHOSTPERFDATA::$HOSTPERFDATA$\tHOSTCHECKCOMMAND::$HOSTCHECKCOMMAND$\tHOSTSTATE::$HOSTSTATE$\tHOSTSTATETYPE::$HOSTSTATETYPE$\tGRAPHITEPREFIX::$_HOSTGRAPHITEPREFIX$\tGRAPHITEPOSTFIX::$_HOSTGRAPHITEPOSTFIX$\tMETRICTYPE::$_HOSTMETRICTYPE$
host_perfdata_file_mode=a
host_perfdata_file_processing_interval=15
host_perfdata_file_processing_command=graphios_perf_host
================================================
FILE: setup.cfg
================================================
[bdist_rpm]
provides = graphios
post-install = init/rhel/postinstall
install-script = init/rhel/install
post-uninstall = init/rhel/postuninstall
================================================
FILE: setup.py
================================================
# vim: set ts=4 sw=4 tw=79 et :
from setuptools import setup
from setuptools.command.install import install as _install
from shutil import Error as FileError
from shutil import copy
from time import strftime
import os
import platform
import re
def find_nagios_cfg(lookin):
"""
finds the nagios.cfg given list of directories
"""
for path in lookin:
for root, dirs, files in os.walk(path):
if "nagios.cfg" in files:
return os.path.join(root, "nagios.cfg")
def parse_nagios_cfg(nag_cfg):
"""
parses the nagios.cfg
"""
nconfig = {}
inputfile = open(nag_cfg, 'r')
for line in inputfile:
if re.match('[a-zA-Z]', line[0]):
try:
option, value = line.split("=")
except:
continue
else:
nconfig[option.rstrip()] = value.rstrip()
inputfile.close()
return nconfig
def add_perfdata_config(nconfig, nag_cfg):
"""
adds the graphios perfdata cfg to the nagios.cfg
"""
with open('nagios/nagios_perfdata.cfg') as f:
main_config = f.read().splitlines()
with open('nagios/graphios_commands.cfg') as f:
commands = f.read().splitlines()
# add the graphios commands
cstat = os.stat(nag_cfg)
print("nagios uid: %s gid: %s" % (cstat.st_uid, cstat.st_gid))
if "cfg_dir" in nconfig:
command_file = os.path.join(nconfig["cfg_dir"],
'graphios_commands.cfg')
else:
command_dir = os.path.join(os.path.dirname(nag_cfg), "objects")
main_config.append("cfg_dir=%s" % command_dir)
if not os.path.exists(command_dir):
os.mkdir(command_dir)
os.chown(command_dir, cstat.st_uid, cstat.st_gid)
command_file = os.path.join(command_dir, "graphios_commands.cfg")
cfile = open(command_file, 'a')
for line in commands:
cfile.writelines("%s\n" % line)
cfile.close()
os.chown(command_file, cstat.st_uid, cstat.st_gid)
write_main_config(nconfig, nag_cfg, main_config)
def write_main_config(nconfig, nag_cfg, main_config):
"""
writes the nagios.cfg
"""
# now add the main config
nfile = open(nag_cfg, 'a')
if (
"process_performance_data" in nconfig and
nconfig["process_performance_data"] == "1"
):
print("pre-existing perfdata config detected")
for line in main_config:
nfile.writelines("# %s\n" % line)
else:
for line in main_config:
nfile.writelines("%s\n" % line)
nfile.close()
def _post_install():
"""
tries to find the nagios.cfg and insert graphios perf commands/cfg
"""
lookin = ['/etc/nagios/', '/opt/nagios/', '/usr/local/nagios',
'/usr/nagios']
nag_cfg = find_nagios_cfg(lookin)
if nag_cfg is None:
print("sorry I couldn't find the nagios.cfg file")
print("NO POST INSTALL COULD BE PERFORMED")
else:
print("found nagios.cfg in %s" % nag_cfg)
nconfig = parse_nagios_cfg(nag_cfg)
print("parsed nagcfg, nagios_log is at %s" % nconfig['log_file'])
if backup_file(nag_cfg):
add_perfdata_config(nconfig, nag_cfg)
else:
print("Backup failed, add modify nagios.cfg manually.")
def backup_file(file_name):
"""
backs up the nagios.cfg, incase we break something.
"""
my_time = strftime('%d-%m-%y')
new_file_name = "%s.%s" % (file_name, my_time)
print("backing up file:%s to %s" % (file_name, new_file_name))
try:
copy(file_name, new_file_name)
return True
except (FileError, IOError, OSError) as e:
print("Error:%s copying:%s to:%s" % (e, file_name, new_file_name))
return False
class my_install(_install):
"""
installs graphios
"""
def run(self):
_install.run(self)
self.execute(_post_install, [], msg="Running post install task")
data_files = [
(('/etc/graphios'), ["graphios.cfg"])
]
scripts = ["graphios.py"]
distro = platform.dist()[0]
distro_ver = int(platform.dist()[1].split('.')[0])
# print "using %s %s" % (distro, distro_ver)
if distro in ['Ubuntu', 'debian']:
data_files.append(('/etc/init/', ['init/debian/graphios.conf']))
data_files.append(('/usr/local/bin/', ['graphios.py']))
data_files.append(('/etc/init.d/', ['init/debian/graphios']))
elif distro in ['centos', 'redhat', 'fedora']:
data_files.append(('/usr/bin', ['graphios.py']))
if distro_ver >= 7:
data_files.append(('/usr/lib/systemd/system',
['init/systemd/graphios.service']))
elif distro_ver < 7:
data_files.append(('/etc/rc.d/init.d', ['init/rhel/graphios']))
# print data_files
setup(
name='graphios',
version='2.0.0b3',
description='Emit Nagios metrics to Graphite, Statsd, and Librato',
author='Shawn Sterling',
author_email='shawn@systemtemplar.org',
url='https://github.com/shawn-sterling/graphios',
license='GPL v2',
scripts=['graphios.py'],
data_files=data_files,
py_modules=['graphios_backends'],
cmdclass={'install': my_install},
classifiers=[
'Development Status :: 4 - Beta',
'Intended Audience :: System Administrators',
'Programming Language :: Python :: 2.7',
],
keywords='Nagios metrics graphing visualization',
long_description='Graphios is a script to send nagios perfdata to various\
backends like graphite, statsd, and librato. \
https://github.com/shawn-sterling/graphios'
)
gitextract_hfsrrmqg/ ├── .gitignore ├── .travis.yml ├── MANIFEST.in ├── README.md ├── graphios.cfg ├── graphios.py ├── graphios_backends.py ├── init/ │ ├── debian/ │ │ ├── graphios │ │ └── graphios.conf │ ├── rhel/ │ │ ├── graphios │ │ ├── install │ │ ├── postinstall │ │ └── postuninstall │ └── systemd/ │ └── graphios.service ├── nagios/ │ ├── graphios_commands.cfg │ └── nagios_perfdata.cfg ├── setup.cfg └── setup.py
SYMBOL INDEX (67 symbols across 3 files)
FILE: graphios.py
class GraphiosMetric (line 122) | class GraphiosMetric(object):
method __init__ (line 123) | def __init__(self):
method validate (line 147) | def validate(self):
method check_adjust_hostname (line 176) | def check_adjust_hostname(self):
function chk_bool (line 184) | def chk_bool(value):
function read_config (line 195) | def read_config(config_file):
function verify_config (line 230) | def verify_config(config_dict):
function print_debug (line 256) | def print_debug(msg):
function verify_options (line 264) | def verify_options(opts):
function handle_backends (line 303) | def handle_backends(opts):
function configure (line 320) | def configure():
function process_log (line 358) | def process_log(file_name):
function get_mobj (line 400) | def get_mobj(nag_array):
function handle_file (line 428) | def handle_file(file_name, graphite_lines):
function process_spool_dir (line 443) | def process_spool_dir(directory):
function check_skip_file (line 483) | def check_skip_file(file_name, file_dir):
function init_backends (line 504) | def init_backends():
function send_backends (line 544) | def send_backends(metrics):
function main (line 561) | def main():
FILE: graphios_backends.py
class librato (line 18) | class librato(object):
method __init__ (line 19) | def __init__(self, cfg):
method build_path (line 82) | def build_path(self, vals, m):
method k_not_in_whitelist (line 93) | def k_not_in_whitelist(self, k):
method add_measure (line 101) | def add_measure(self, m):
method flush_payload (line 126) | def flush_payload(self, headers, g):
method flush (line 155) | def flush(self):
method build_basic_auth (line 187) | def build_basic_auth(self):
method build_user_agent (line 192) | def build_user_agent(self):
method send (line 209) | def send(self, metrics):
class carbon (line 225) | class carbon(object):
method __init__ (line 226) | def __init__(self, cfg):
method convert_messages (line 285) | def convert_messages(self, metrics):
method chunks (line 313) | def chunks(self, l, n):
method build_path (line 319) | def build_path(self, m):
method fix_string (line 348) | def fix_string(self, my_string):
method send (line 359) | def send(self, metrics):
class statsd (line 402) | class statsd(object):
method __init__ (line 403) | def __init__(self, cfg):
method set_type (line 413) | def set_type(self, metric):
method convert (line 426) | def convert(self, metrics):
method send (line 442) | def send(self, metrics):
class influxdb (line 473) | class influxdb(object):
method __init__ (line 474) | def __init__(self, cfg):
method build_url (line 519) | def build_url(self, server):
method build_path (line 530) | def build_path(self, m):
method chunks (line 551) | def chunks(self, l, n):
method url_request (line 556) | def url_request(self, url, chunk):
method _send (line 562) | def _send(self, server, chunk):
method send (line 584) | def send(self, metrics):
class influxdb09 (line 627) | class influxdb09(influxdb):
method __init__ (line 628) | def __init__(self, cfg):
method build_url (line 643) | def build_url(self, server):
method url_request (line 659) | def url_request(self, url, chunk):
method format_metric (line 667) | def format_metric(self, timestamp, path, tags, value):
method format_series (line 681) | def format_series(self, chunk):
method send (line 687) | def send(self, metrics):
class stdout (line 727) | class stdout(object):
method __init__ (line 728) | def __init__(self, cfg):
method send (line 732) | def send(self, metrics):
FILE: setup.py
function find_nagios_cfg (line 12) | def find_nagios_cfg(lookin):
function parse_nagios_cfg (line 22) | def parse_nagios_cfg(nag_cfg):
function add_perfdata_config (line 40) | def add_perfdata_config(nconfig, nag_cfg):
function write_main_config (line 71) | def write_main_config(nconfig, nag_cfg, main_config):
function _post_install (line 90) | def _post_install():
function backup_file (line 110) | def backup_file(file_name):
class my_install (line 125) | class my_install(_install):
method run (line 129) | def run(self):
Condensed preview — 18 files, each showing path, character count, and a content snippet. Download the .json file or copy for the full structured content (96K chars).
[
{
"path": ".gitignore",
"chars": 603,
"preview": "# File artifacts which may by produced regular software on Windows, Mac OS X and Linux\nThumbs.db\n.DS_Store\n*.bak\n\n# JetB"
},
{
"path": ".travis.yml",
"chars": 100,
"preview": "language: python\n\npython:\n - \"2.6\"\n - \"2.7\"\n\ninstall:\n - pip install flake8\n\nscript: flake8 *.py\n"
},
{
"path": "MANIFEST.in",
"chars": 98,
"preview": "include graphios.py\ninclude graphios_backends.py\ninclude graphios.cfg\ngraft init\ngraft nagios\n"
},
{
"path": "README.md",
"chars": 26569,
"preview": "\nGraphios\n========\n\n[](https://travis-ci"
},
{
"path": "graphios.cfg",
"chars": 5910,
"preview": "# Graphios config file\n\n[graphios]\n\n#------------------------------------------------------------------------------\n# Gl"
},
{
"path": "graphios.py",
"chars": 21197,
"preview": "#!/usr/bin/python -tt\n# vim: set ts=4 sw=4 tw=79 et :\n# Copyright (C) 2011 Shawn Sterling <shawn@systemtemplar.org>\n#\n#"
},
{
"path": "graphios_backends.py",
"chars": 25524,
"preview": "# vim: set ts=4 sw=4 tw=79 et :\n\nimport socket\nimport cPickle as pickle\nimport struct\nimport re\nimport logging\nimport sy"
},
{
"path": "init/debian/graphios",
"chars": 1577,
"preview": "#! /bin/bash\n### BEGIN INIT INFO\n# Provides: graphios\n# Required-Start: $local_fs $remote_fs $syslog $named "
},
{
"path": "init/debian/graphios.conf",
"chars": 204,
"preview": "description \"Graphios: Emit nagios perfdata to graphite/statsd/librato\"\nauthor \"Shawn Sterling <shawn@systemtemplar.org"
},
{
"path": "init/rhel/graphios",
"chars": 1212,
"preview": "#!/bin/bash\n#\n# graphios start the graphios script\n#\n#\n# chkconfig: 345 99 01\n# description: graphios nagios -> gra"
},
{
"path": "init/rhel/install",
"chars": 85,
"preview": "python setup.py install --optimize 1 --root=$RPM_BUILD_ROOT --record=INSTALLED_FILES\n"
},
{
"path": "init/rhel/postinstall",
"chars": 280,
"preview": "# add symlink\nln -s /usr/bin/graphios.py /usr/bin/graphios\n\n# add graphios service\nif [ ! -d /etc/systemd ]; then\n # "
},
{
"path": "init/rhel/postuninstall",
"chars": 182,
"preview": "if [ ! -d /etc/systemd ]; then\n # we are running init\n service graphios stop\nelse\n # we are running systemd\n "
},
{
"path": "init/systemd/graphios.service",
"chars": 180,
"preview": "[Unit]\nDescription=graphios - a script to emit nagios perfdata to various backends\n\n[Service]\nExecStart=/usr/bin/graphio"
},
{
"path": "nagios/graphios_commands.cfg",
"chars": 394,
"preview": "define command {\n command_name graphios_perf_host\n command_line /bin/mv /var/spool/nagios/gr"
},
{
"path": "nagios/nagios_perfdata.cfg",
"chars": 1234,
"preview": "\n###### Auto-generated Graphios configs #######\nprocess_performance_data=1\nservice_perfdata_file=/var/spool/nagios/graph"
},
{
"path": "setup.cfg",
"chars": 145,
"preview": "[bdist_rpm]\nprovides = graphios\npost-install = init/rhel/postinstall\ninstall-script = init/rhel/install\npost-uninstall ="
},
{
"path": "setup.py",
"chars": 5581,
"preview": "# vim: set ts=4 sw=4 tw=79 et :\nfrom setuptools import setup\nfrom setuptools.command.install import install as _install\n"
}
]
About this extraction
This page contains the full source code of the shawn-sterling/graphios GitHub repository, extracted and formatted as plain text for AI agents and large language models (LLMs). The extraction includes 18 files (88.9 KB), approximately 22.5k tokens, and a symbol index with 67 extracted functions, classes, methods, constants, and types. Use this with OpenClaw, Claude, ChatGPT, Cursor, Windsurf, or any other AI tool that accepts text input. You can copy the full output to your clipboard or download it as a .txt file.
Extracted by GitExtract — free GitHub repo to text converter for AI. Built by Nikandr Surkov.