rrdcached (1) - Linux Manuals
rrdcached: Data caching daemon for rrdtool
NAME
rrdcached - Data caching daemon for rrdtool
SYNOPSIS
rrdcached [-PDESCRIPTION
rrdcached is a daemon that receives updates to existing RRD files, accumulates them and, if enough have been received or a defined time has passed, writes the updates to the RRD file. A flush command may be used to force writing of values to disk, so that graphing facilities and similar can work with up-to-date data.
The daemon was written with big setups in mind. Those setups usually run into
IO
For network sockets, a port may be specified by using the form
"[address]:port". If the address is an IPv4 address or a fully
qualified domain name (i. e. the address contains at least one dot
(".")), the square brackets can be omitted, resulting in the (simpler)
"address:port" pattern. The default port is 42217/tcp. If you
specify a network socket, it is mandatory to read the
``SECURITY CONSIDERATIONS'' section.
The following formats are accepted. Please note that the address of the UNIX
domain socket must start with a slash in the second case!
If the -l option is not specified the default address,
"unix:/tmp/rrdcached.sock", will be used.
This option affects the following UNIX socket addresses (the following
-l options) or the default socket (if no -l options have been
specified), i.e., you may specify different settings for different
sockets.
The default is not to change ownership or permissions of the socket and, thus,
use the system default.
Please note that not all systems honor this setting. On Linux, read/write
permissions are required to connect to a UNIX socket. However, many
BSD-derived systems ignore permissions for UNIX sockets. See unix(7) for
details.
This option affects the following UNIX socket addresses (the following
-l options) or the default socket (if no -l options have been
specified), i.e., you may specify different settings for different
sockets.
The default is not to change ownership or permissions of the socket and, thus,
use the system default.
The arguments given to the -P option is a comma separated list of commands.
For example, to allow the "FLUSH" and "PENDING" commands one could specify:
The -P option affects the following socket addresses (the following -l
options) or the default socket (if no -l options have been
specified). In the following example, only the IPv4 network socket (address
10.0.0.1) will be restricted to the "FLUSH" and "PENDING" commands:
A complete list of available commands can be found in the section
``Valid Commands'' below. There are two minor special exceptions:
Please also read ``SECURITY CONSIDERATIONS'' below.
On startup, the daemon will check for journal files in this directory. If
found, all updates therein will be read into memory before the daemon
starts accepting new connections.
The journal will be rotated with the same frequency as the flush timer
given by -f.
When journaling is enabled, the daemon will use a fast shutdown procedure.
Rather than flushing all files to disk, it will make sure the journal is
properly written and exit immediately. Although the RRD data files are
not fully up-to-date, no information is lost; all pending updates will be
replayed from the journal next time the daemon starts up.
To disable fast shutdown, use the -F option.
WARNING: The paths up to and including the base directory MUSTNOTBE
symbolic links. In other words, if the base directory is
specified as:
... then NONE of the following should be symbolic links:
The update command can send values to the daemon instead of writing them to
the disk itself. All other commands can send a FLUSH command (see below) to
the daemon before accessing the files, so they work with up-to-date data even
if the cache timeout is large.
When appending a value to a tree node, it is checked whether it's time to write
the values to disk. Values are written to disk if
"now()
A separate ``update thread'' constantly dequeues the first element in the update
queue and writes all its values to the appropriate file. So as long as the
update queue is not empty files are written at the highest possible rate.
Since the timeout of files is checked only when new values are added to the
file, ``dead'' files, i. e. files that are not updated anymore, would never
be written to disk. Therefore, every now and then, controlled by the -f
option, the entire tree is walked and all ``old'' values are enqueued. Since this
only affects ``dead'' files and walking the tree is relatively expensive, you
should set the ``flush interval'' to a reasonably high value. The default is
3600 seconds (one hour).
The downside of caching values is that they won't show up in graphs generated
from the RRD
The above diagram demonstrates:
If your rrdtool installation was built with libwrap then you can use
hosts_access to restrict client access to the rrdcache daemon (rrdcached). For more
information on how to use hosts_access to restrict access to the rrdcache
daemon you should read the hosts_access(5) man pages.
It is still highly recommended to install a packet filter or similar mechanism to
prevent unauthorized connections. Unless you have a dedicated VLAN or VPN for
this, using network sockets is probably a bad idea!
Authorization is currently done on a per-socket basis. That means each socket
has a list of commands it will accept and it will accept. It will accept only
those commands explicitly listed but it will (currently) accept these commands
from anyone reaching the socket.
If the networking sockets are to be used, it is necessary to restrict the
accepted commands to those needed by external clients. If, for example,
external clients want to draw graphs of the cached data, they should only be
allowed to use the "FLUSH" command.
Again, this may be added in the future, but for the time being it is your job
to keep your private data private. Install a VPN or an encrypted tunnel if you
statistics are confidential!
The daemon will blindly write to any file it gets told, so you really should
create a separate user just for this daemon. Also it does not do any sanity
checks, so if it gets told to write values for a time far in the future, your
files will be messed up good!
The protocol is line based, this means that each record consists of one or more
lines. A line is terminated by the line feed character 0x0A, commonly
written as "\n". In the examples below, this character will be written as
"<LF>" (``line feed'').
After the connection has been established, the client is expected to send a
``command''. A command consists of the command keyword, possibly some arguments,
and a terminating newline character. For a list of commands, see
``Valid Commands'' below.
Example:
The daemon answers with a line consisting of a status code and a short status
message, separated by one or more space characters. A negative status code
signals an error, a positive status code or zero signal success. If the status
code is greater than zero, it indicates the number of lines that follow the
status line.
Examples:
The format in which the values are returned is similar to many other line based
protocols: Each value is printed on a separate line, each consisting of the
name of the value, a colon, one or more spaces and the actual value.
Example:
Note that rrdcached only accepts absolute timestamps in the update values.
Updates strings like ``N:1:2:3'' are automatically converted to absolute
time by the RRD client library before sending to rrdcached.
All commands are executed just as they would be if given individually,
except for output to the user. Messages indicating success are
suppressed, and error messages are delayed until the client is finished.
Command processing is finished when the client sends a dot (``.'') on its
own line. After the client has finished, the server responds with an
error count and the list of error messages (if any). Each error messages
indicates the number of the command to which it corresponds, and the error
message itself. The first user command after BATCH is command number one.
Both rrdcached and this manual page have been written by Florian.
OPTIONS
unix:</path/to/unix.sock>
/<path/to/unix.sock>
<hostname-or-ip>
[<hostname-or-ip>]:<port>
<hostname-or-ipv4>:<port>
rrdcached -P FLUSH,PENDING $MORE_ARGUMENTS
rrdcached -l unix:/some/path -P FLUSH,PENDING -l 10.0.0.1
+------------------------+------------------------+
! Command line ! File updated !
+------------------------+------------------------+
! foo.rrd ! /tmp/foo.rrd !
! foo/bar.rrd ! /tmp/foo/bar.rrd !
! /var/lib/rrd/foo.rrd ! /var/lib/rrd/foo.rrd !
+------------------------+------------------------+
Paths given on the command line and paths actually
updated by the daemon, assuming the base directory
"/tmp".
-b /base/dir/somewhere
/base
/base/dir
/base/dir/somewhere
AFFECTED RRDTOOL COMMANDS
The following commands may be made aware of the rrdcached using the command
line argument --daemon or the environment variable RRDCACHED_ADDRESS:
ERROR REPORTING
The daemon reports errors in one of two ways: During startup, error messages
are printed to "STDERR". One of the steps when starting up is to fork to the
background and closing "STDERR" - after this writing directly to the user is
no longer possible. Once this has happened, the daemon will send log messages
to the system logging daemon using syslog(3). The facility used is
"LOG_DAEMON".
HOW IT WORKS
When receiving an update, rrdcached does not write to disk but looks for an
entry for that file in its internal tree. If not found, an entry is created
including the current time (called ``First'' in the diagram below). This time is
not the time specified on the command line but the time the operating system
considers to be ``now''. The value and time of the value (called ``Time'' in the
diagram below) are appended to the tree node.
+------+ +------+ +------+
! head ! ! root ! ! tail !
+---+--+ +---+--+ +---+--+
! /\ !
! / \ !
! /\ /\ !
! /\/\ \ `----------------- ... --------, !
V / `-------, ! V
+---+----+---+ +------+-----+ +---+----+---+
! File: foo ! ! File: bar ! ! File: qux !
! First: 101 ! ! First: 119 ! ! First: 180 !
! Next:&bar -+--->! Next:&... -+---> ... --->! Next:NULL !
| Prev:NULL !<---+-Prev:&foo !<--- ... ----+-Prev: &... !
+============+ +============+ +============+
! Time: 100 ! ! Time: 120 ! ! Time: 180 !
! Value: 10 ! ! Value: 0.1 ! ! Value: 2,2 !
+------------+ +------------+ +------------+
! Time: 110 ! ! Time: 130 ! ! Time: 190 !
! Value: 26 ! ! Value: 0.1 ! ! Value: 7,3 !
+------------+ +------------+ +------------+
: : : : : :
+------------+ +------------+ +------------+
! Time: 230 ! ! Time: 250 ! ! Time: 310 !
! Value: 42 ! ! Value: 0.2 ! ! Value: 1,2 !
+------------+ +------------+ +------------+
SECURITY CONSIDERATIONS
Authentication
If your rrdtool installation was built without libwrap there is no form of
authentication for clients connecting to the rrdcache daemon!
Authorization
There is minimal per-socket authorization.
Encryption
There is no encryption.
Sanity checking
There is no sanity checking.
Conclusion
PROTOCOL
The daemon communicates with clients using a line based ASCII protocol which is
easy to read and easy to type. This makes it easy for scripts to implement the
protocol and possible for users to use telnet to connect to the daemon
and test stuff ``by hand''.
FLUSH /tmp/foo.rrd<LF>
0 Success<LF>
2 Two lines follow<LF>
This is the first line<LF>
And this is the second line<LF>
Valid Commands
The following commands are understood by the daemon:
<num_vals> <file>
9 Statistics follow
QueueLength: 0
UpdatesReceived: 30
FlushesReceived: 2
UpdatesWritten: 13
DataSetsWritten: 390
TreeNodesNumber: 13
TreeDepth: 4
JournalBytes: 190
JournalRotate: 0
client: BATCH
server: 0 Go ahead. End with dot '.' on its own line.
client: UPDATE x.rrd 1223661439:1:2:3 <--- command #1
client: UPDATE y.rrd 1223661440:3:4:5 <--- command #2
client: and so on...
client: .
server: 2 Errors
server: 1 message for command 1
server: 12 message for command 12
Performance Values
The following counters are returned by the STATS command:
SIGNALS
BUGS
No known bugs at the moment.
AUTHOR
Florian Forster <octo at verplant.org>
CONTRIBUTORS
kevin brintnall <kbrint [at] rufus.net>