What is this?
SplitRotate is a small program written in C++, which is designed to be used with Apache (both 1.x and 2.x). It splits the log file produced by Apache into seperate files depending on which web-site (in a virtual host setup) the log comes from.
The advantages of this approach vs. an external splitter is that it's completely transparent to Apache, and as such the web-server doesn't need to be restarted or anything like that. Apache in fact ships with a perl-script that performs some of the same functions, but SplitRotate has a number of advantages which are outlined below.
The advantages of this approach vs. an external splitter is that it's completely transparent to Apache, and as such the web-server doesn't need to be restarted or anything like that. Apache in fact ships with a perl-script that performs some of the same functions, but SplitRotate has a number of advantages which are outlined below.
Features
- Small, fast and memory efficient; written in C++. No script-interpreter footprint or anything like that.
- Easy to install and use; three quick steps is all. Se below.
- Can split logs, rotate them when they grow too big, compress the rotated logs and optionally store compressed logs in a separate directory. Can also maintain a separate directory per site.
- Full customization of permissions on created files and directories. Together with the previous feature, this makes it easy to provide user read-only access to logs pertaining to the site they administer, without compromising the other logs. See also the Usage tips section below.
- Allows for uninterruptable running of Apache. No restart needed even during or after external rotation.
Installation
- Download and build the program. Just unpack, change into the directory and type make. GNU Make and gcc is required, but otherwise any POSIX-compliant operating system should work.
- Copy the executable somewhere, and change the CustomLog directive in the Apache configuration to: CustomLog "|/path/to/splitrotate" logstyle, where logstyle can be anything you want, as long as it starts with the virtual host as the first word. The following will produce output in the Common Log Format: LogFormat "%v %h %l %u %t \"%r\" %>s %b" logstyle. See the Apache manual for more details.
- Adjust the configuration file, /etc/splitrotate.conf, to your needs (a sample is included). In particular you may want to change the LogDir directive. For a full list of configuration options, see below. Finally, create a command FIFO if needed (see the configuration item Fifo below) and restart apache.
Download and license
This program is provided more or less as-is. You may do anything with it, but it would be appreciated if you would credit the original author if you do so. Also, it would be greatly appreciated if updates or changes to the program are suggested and sent to the author, so improvements can benefit a larger crowd. The source can be obtained here:
Pre-build configuration
Some options are currently compiled into the program executable, and can not be set via the configuration file. Have a look at the file config.h which should explain those settings.
Configuration items
The configuration file resides in /etc/splitrotate.conf. This can be changed by editing the file config.h, as explained above. As usual, empty lines and lines starting with # (optinally preceeded by white space) will be ignored. Configuration items are case insensitive.
Log files created by SplitRotate will be named after the host name of the site they come from. So the file receiving all log output pertaining to this site would be called cyanite.org.
Log files created by SplitRotate will be named after the host name of the site they come from. So the file receiving all log output pertaining to this site would be called cyanite.org.
- LogDir
- Sets the path to the base directory for the program. This is required to point to a valid directory, in which SplitRotate can create files (and possibly directories). Defaults to /var/log, but you should probably change this to a sub-directory of that directory. All other file names and paths in the configuration file (except Compressor) are considered relative to this directory.
- MaxSize
- If this is set to an integer number of bytes, log files will be rotated transparently when they grow above this size. Currently, no units such as k or M etc. are supported, but this is on the to-do. Rotated files are named site_hostname.n[.ext], where n is an integer, and ext is a possible extension added by the compression program (se below).
- LogFile
- If this is set, warnings and errors from SplitRotate are output to this file. Otherwise log ouput is done via the syslog interface, with facility daemon. Defaults to not set (i.e. defaults to using syslog).
- BackLogSize
- Sets the number of old log files to keep in connection with rotation. Files will be deleted after that. Defaults to 5. Set to 0 to disable removing of old log files.
- Compressor
- If rotation is enabled, this can be set to the full path of a compression program used to compress the rotated log files. See the next option for details on how the compression program is invoked from SplitRotate. Defaults to nothing, meaning that no compression is performed.
- CompressorOptions
- With this item, optional compression options can be set. The compressor is invoked as follows: <compressor> <compressor options> <file>. This should work for most common compressors such as gzip, bzip2 and similar. Defaults to nothing.
- PlainLog
- This option sets the name of the plain log file, used to output log entries that SplitRotate deems spurious and not belonging to a real site. Such entries are sometimes generated by Apache in response to long SEARCH queries (probably originating from viruses). Defaults to plain.log. If used with the NoSiteLogs option below, all log output will go to this file.
- OldLogsDir
- If set to a directory, rotated logs will be put in this directory instead of, as is default, residing in the base directory as set with the LogDir option. Remember the path in this option is relative to the base directory. Defaults to nothing. Note that this option is overrided by the SiteDirs option below.
- SiteDirs
- If this option is set (no value required), log files for the individual sites will be stored in separate directories with the site's host name as their name. The log file itself will still receive the default name (which is also the host name). When this options is given, rotated logs will be put in the same directory as the main log for each site, so the OldLogsDir option is ignored. Defaults to not set.
- FilePerms
- This sets the file permission used when SplitRotate creates new log files (initially, on demand and after rotations). Numbers can be given in decimal, hexedecimal prefixed with 0x or octal prefixed with 0. Defaults to 0600.
- DirPerms
- Same thing as above, but used for the directories SplitRotate creates. This is only relevant when the SiteDirs option is enabled. Defaults to 0655.
- Fifo
- Sets the (full) path for SplitRotate's command FIFO. See Command FIFO below for details. The FIFO must be created manually, but only once of course. Defaults to /var/run/splitrotate.control.
- NoFifo
- Setting this (no value required) disabled the command FIFO. The FIFO is mainly useful if external rotation based on e.g. age is needed. Defaults to not set (that is: the FIFO is used per default).
- NoSiteLogs
- If set, disables per-site logs altogether, and instead outputs everything to the plain log file (configured with the PlainLog option above). Everything will be output as if it were spurious entries, i.e. no procesessing is done at all.
- NoPlainLog
- If set, disables the plain log, which is normally used for log entries deemed spurious by the program. If the NoSiteLogs option is also set, do not set this option as all log entries will then simply be discarded.
Command FIFO
Per default, SplitRotate opens a command FIFO which can be used to control certain aspects of the operation run-time. The FIFO must be created with the mkfifo command before using SplitRotate. If it is not found in the expected place, it will act as if the NoFifo option is set (but issue a warning to the log).
The FIFO supports the following commands, which can be issued with echo <command> >/path/to/fifo. Make sure that the FIFO has proper permissions so unauthorized access is not possible. FIFO commands are case insensitive, and none of them accept arguments.
The FIFO supports the following commands, which can be issued with echo <command> >/path/to/fifo. Make sure that the FIFO has proper permissions so unauthorized access is not possible. FIFO commands are case insensitive, and none of them accept arguments.
- RotateOff
- Disables log rotation.
- RotateOn
- Enables log rotation. This can only be done if rotation was originally enabled with the MaxSize configuration item.
- Reset
-
Resets SplitRotate, closing all open files. This is the most useful command, and should be used when external rotation is performed. The optimal procedure is:
- Rename the log file to something else. SplitRotate will still have the file open at this point, and may append to it.
- Issue the reset command to the FIFO. As soon as more log input arrives, the log file will be recreated under the original name. No more output will be made to the renamed file.
- Compress the renamed file or do whatever you want with it, at this point.
- Quit
- Tells SplitRotate to quit. Normally, Apache will restart it right away, so there is probably not much use for this.
Known problems
The spurious entries occuring in v1.0.2 should be fixed now. The problem originates from Apache and its way of dealing with malformed SEARCH requests. I will add some more and stronger sanity checks in the near future.
Usage tips
The SiteDirs option makes it possible to give a user or a group of users that are administrators of a certain site, read-only or read-write access to their own log, while keeping other logs secret. This can be done by having a UNIX group for each web-site, and creating the site-dir yourself owned by that group. Then set the SETGUID bit on the directory (so files inside will also be owned by that group), and set the FilePerms option to e.g. 0640 which allowed read-write access for the owner (typically the user running Apache), and read-only access for the group owning the file, which will be the site-group.
Feedback, questions or suggestions
Got comments, need a feature, found a bug or need to reach for another reason? Write to . See the main page for more contact details.