pkg://chklogs-2.0-3.tar.gz:163018/
chklogs-2.0-3/
doc/chklogs.txt
downloads
C H K L O G S
U S E R ' S G U I D E
Version 2.0
Document Revision B
18 October 1997
Copyright (C)1994-1997 D. Emilio Grimaldo Tunon
*************************************
* T a b l e O f C o n t e n t s *
*************************************
I INTRODUCTION . . . . . . . . . . . . . . . . . . . . . . . 1
1.1 Features . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 Other sources of information . . . . . . . . . . . . . . 2
1.3 Ramblings . . . . . . . . . . . . . . . . . . . . . . . 2
II INSTALLATION . . . . . . . . . . . . . . . . . . . . . . 2
2.1 File list . . . . . . . . . . . . . . . . . . . . . . . 3
2.2 System Requirements . . . . . . . . . . . . . . . . . . 3
2.3 The installation script . . . . . . . . . . . . . . . . 3
2.3.1 Text console installation . . . . . . . . . . . . . . 4
2.3.2 X-based installation . . . . . . . . . . . . . . . . . 5
2.4 Upgrading from 1.8 or 1.9 . . . . . . . . . . . . . . . 5
2.5 Upgrading from much older versions . . . . . . . . . . . 6
III SPECIAL CONCEPTS . . . . . . . . . . . . . . . . . . . . 6
3.1 Log Thresholds . . . . . . . . . . . . . . . . . . . . . 6
3.2 Log Groups (1.8+) . . . . . . . . . . . . . . . . . . . 6
3.3 Repositories (1.8+) . . . . . . . . . . . . . . . . . . 6
3.3.1 Archival Method . . . . . . . . . . . . . . . . . . . 7
3.4 Lumping (1.8+) . . . . . . . . . . . . . . . . . . . . . 7
3.5 Plug-outs . . . . . . . . . . . . . . . . . . . . . . . 7
IV CONFIGURATION . . . . . . . . . . . . . . . . . . . . . . 7
4.1 Configuration variables . . . . . . . . . . . . . . . . 8
4.1.1 Description of the configuration variables . . . . . . 9
4.2 The Resource Files . . . . . . . . . . . . . . . . . . . 10
4.2.1 Semantics of the Resource File . . . . . . . . . . . . 10
4.3 The Configuration File . . . . . . . . . . . . . . . . . 11
4.3.1 Semantics of the configuration file . . . . . . . . . 11
4.3.1.1 Header section . . . . . . . . . . . . . . . . . . . 12
4.3.1.2 Definition section . . . . . . . . . . . . . . . . . 12
4.4 Plug-out interface . . . . . . . . . . . . . . . . . . . 13
4.4.1 Contributions . . . . . . . . . . . . . . . . . . . . 14
V THE ADMINISTRATIVE PROGRAM: ChklogsAdm . . . . . . . . . . 14
5.1 Creating a configuration: newconf . . . . . . . . . . . 14
5.2 Initializing the internal database: init . . . . . . . . 15
5.3 Initializing the repositories: initrepos . . . . . . . . 15
5.4 Synchronizing the package: sync . . . . . . . . . . . . 15
5.5 Managing log groups . . . . . . . . . . . . . . . . . . 16
5.5.1 Creating a log group: newgroup . . . . . . . . . . . . 16
5.5.2 Adding a log to the group: gadd . . . . . . . . . . . 17
5.5.3 Remove a group: rmgroup . . . . . . . . . . . . . . . 17
5.5.4 Deleting a log from the configuration: del . . . . . . 17
5.5.5 Modifying group attributes . . . . . . . . . . . . . . 18
5.5.6 Contributions . . . . . . . . . . . . . . . . . . . . 18
VI MANAGING YOUR LOGS: Chklogs . . . . . . . . . . . . . . . 18
6.1 Using chklogs with CRON . . . . . . . . . . . . . . . . 20
6.2 Shuffling During Archival . . . . . . . . . . . . . . . 20
6.3 Sample Output File . . . . . . . . . . . . . . . . . . . 21
VII OF BUGS, REGISTRATION AND OTHER DAEMONS . . . . . . . . 21
7.1 Registration . . . . . . . . . . . . . . . . . . . . . . 21
7.2 Bug Reports & Feature Requests . . . . . . . . . . . . . 22
7.3 Copyright . . . . . . . . . . . . . . . . . . . . . . . 22
7.4 Licensing terms . . . . . . . . . . . . . . . . . . . . 23
7.5 Credits and Acknowledments . . . . . . . . . . . . . . . 23
Appendix A Plug-outs and Execute Helpers . . . . . . . . . . 24
A.1 CdkWrap . . . . . . . . . . . . . . . . . . . . . . . . 24
A.2 LogSummary . . . . . . . . . . . . . . . . . . . . . . . 24
A.3 UuSummary . . . . . . . . . . . . . . . . . . . . . . . 25
A.4 Ck_Uutraf . . . . . . . . . . . . . . . . . . . . . . . 25
A.5 Ck_Xferlog . . . . . . . . . . . . . . . . . . . . . . . 25
Appendix B The Xchklogs Graphical User Interface . . . . . . 26
Chapter I INTRODUCTION
Sooner or later every system admininistrator has to check and purge-if
necessary-the system logs. These system logs are created by programs
such as cron, init, list managers, mail programs, news, uucp, etc. These
logs always grow, some faster than others but in the end they end up
using precious disk space. To avoid running out of disk space these
system logs have to be trimmed or purged when the information they
contain is no longer necessary. When archiving logs (supported
by the script) there is also a risk to waste too much disk space with
too many (read old) archived logs, to prevent this a shuffling
mechanism is used.
The location of the logs vary from system to system, and sometimes
the person installing the distribution has little knowledge about
the location (or even existence!) of these log files. This program
can either be part of the system administrator's tool set, or in
the case of newbies the program can be installed in such a way that
he receives the proper warnings when necessary.
It is very flexible and allows you to give thresholds either in
bytes (size) or age (days,months) and has several other features.
With this release I am bumping the major version number, a lot (a lot!)
of work has gone into this new release. Read on for more details, I
am afraid you actually need to read the documentation, it isn't a
simple package anymore.
I have changed the format of this user's manual, for information about
new features, changes as well as problems that have been solved please
read the release notes.
1.1 Features
--------------
These are some of the features that have made Chklogs a popular
solution for log management:
* Each log has its own specification of thresholds and action(s)
associated with it.
* Your choice of compression program
* You can manage them either by size or by age.
* Add in you own extensions.
* Logical Log Groups with pre and post processing
- 1 -
* Global and local repositories (Alternate Repository feature)
* Log shuffling (also known as log rotation).
* Directory lumping
* Requires no programming experience
* System and Personal resource files
* Fully based on Perl 5.003 with strict pragma
1.2 Other sources of information
----------------------------------
There is plenty of on-line information, The main web page is
reachable via:
http://www.IAEhv.nl/users/grimaldo/info/
here you can select the software package, read the main page
and get to other sub-pages such as The Frequently Asked Questions,
and Poll form:
http://www.IAEhv.nl/users/grimaldo/info/chklogs/faq-chklogs.html
http://www.IAEhv.nl/users/grimaldo/Linux/form-chklogs.html
Also you will find online release notes and manual pages (up-to-date).
Primary site:
ftp://ftp.iaehv.nl/pub/users/grimaldo/
Sometimes it might be needed to do minor corrections without a
version bump, therefore I use a release number and a build level.
For example chklogs-1.9-1.tar.gz refers to release 1.9 build
level 1.
I only upload the first build to the alternate sites (Funet, Sunsite),
if a rebuild (no release number change) are *only* uploaded to
the primary site mentioned above.
1.3 Ramblings
---------------
Documentation, hum, the most painful thing of every release :-)
I am still keeping the man pages updated, then the Html version
is generated with the nice man2html package. Unfortunately maintaining
the .txt and the web page still have separate lives, not to mention
a tutorial rewrite for the Linux Journal (Dec.'97 or Jan.'98 I hope).
Unfortunately SGML doesn't seem to have (am I wrong?) enough
pizzaz to make an attractive web page out of the document.
Chapter II INSTALLATION
Now that you have obtained package proceed to unpack it. There
are the following subdirectories:
doc
All the documentation in ASCII, groff and Html
bin
The necessary scripts and modules
plugs
The goodies scripts used as plug-outs
relnotes
Most recent release notes
support
Scripts and data used during installation
- 2 -
Additionally there are some files on the root directory, among which
the license text, sample configuration and resource an installation
script and a makefile.
2.1 File list
---------------
certificate A certificate of authenticity
chklogs The Perl script
chklogs.lsm Linux Software Map entry
chklogs.txt This document
chklogs.8 Manual page, chklogs
chklogs.conf.5 Manual page, chklogs config file format
chklogsadm.8 Manual page, chklogsadm
chklogsrc.5 Manual page, chklogs resource file
chklogs.conf A sample configuration file
chklogsrc A default resource file
Chklogs.pm Perl module. Chklogs general library
Smtp.pm Perl module. SMTP for Chklogs
DegtUtils.pm Perl module. Goodies Library
Interpret.pm Perl module. Resource file interpreter
GuideMe Installation Guidance script
SetupChklogs Graphical installation program
Register Registration mailer
makefile For intalling the software
bugreport.txt For reporting problems
chklogs.IAFA The IAFA document for Sunsite
xchklogs.txt documentation of GUI v1.1
ck_uutraf Plug-out for UUCP
ck_xferlog Plug-out for FTP
logsummary Plug-out for Majordomo
uusummary part of ck_uutraf
cdkwrap Plug-out wrapper for non-compliant scripts
rn-X.Y Release notes for version X.Y
2.2 System Requirements
-------------------------
* Perl 5.003 or higher
* Socket.pm Perl module (v1.5 or better)
* Fcntl.pm Perl module (v1.0 or better)
* Getopt::Long.pm Perl module (v2.1 or better)
Yes, I switched to 5.003 and using lots of its features. If you still
have an old Perl (4.036) and don't want to upgrade (not very wise)
then you must stick to Chklogs v1.9. However due to time limitations
I can only support the most recent version.
The other requirements are no problem either, except perhaps for the
Socket module. Recent versions (as required) have an extended interface
that allows to use the same parameters across platforms (Sun/Linux etc).
If you have a pre-1.5 version of that module I advise you to upgrade.
If you don't the built-in SMTP mailer of Chklogs will break, if you
still insist on not upgrading your Socket.pm, you must then disable the
built-in mailer as explained later.
2.3 The installation script
-----------------------------
There are now two ways to install Chklogs on your system. It is still
not perfect but I hope better than before. One of the methods is the
- 3 -
usual text console (as done up to v1.9), a newer method which I recommend
if you are not very experienced, is an X-based installation.
If you are in X-windows you must make sure you have your DISPLAY
environment variable set and that you are authorized to map a window
on that display.
Whichever method you always must start by executing the frontend
installation from the root directory:
./GuideMe
This script will do just that, it will scan your system for stuff that
Chklogs needs and give you an indication of what it found and what
might need to be changed in the default configuration.
I suggest keeping the default installation directories as that will
simplify your upgrade path whenever necessary. I chose the /usr/local
hierarchy which seems most appropriate.
After installation you have two programs that you are going to use,
chklogs which does the hard work, and the administrative
program chklogsadm .
One thing you might need to do is to change the magic line of the
scripts so that it reflects the location of the (Perl) interpreter
on your system. This corresponds to the PERL_BIN variable in
the makefile. For newbies that is the first line which looks like:
#!/usr/bin/perl
And last but not least, if you chose to install it on a different
directory other than /usr/local/lib/chklogs/ then you must
also instruct the scripts chklogs and chklogsadm where to
find the Chklogs libraries. The configuration variable in the makefile
is DEGT_LIB . In the script (should you decide to do it by hand
rather than using the makefile) the line looks like this:
use lib '/usr/local/lib/chklogs';
2.3.1 Text console installation
---------------------------------
The guidance script will not execute anything else, you have to make
sure you read the suggestions from the script and modify the pertinent
files as indicated. Do read the documentation, I receive many questions
from people that don't read it completely.
You can either modify the built-in defaults in the scripts as it was
done before, or modify the values in the sample resource file:
chklogsrc
I would recommend the latter method as it will greatly simplify your
maintenance and upgrades. This would become your Global Resource File
that will hopefully end up in /usr/local/lib/chklogs.
Once the installation guidance script is done, you will need to
run the makefile to get the current configuration installed on your
- 4 -
system. For this to work you must make sure that the necessary
configuration is specified in the makefile or specify the variables
in the `make target' command. These variables are (see makefile):
* <tt/DEGT_LIB/ where the Chklogs module and other private scripts
will be installed. I recommend /usr/local/lib/chklogs
* <tt/PERL_BIN/ The location and name of your Perl binary.
* <tt/GZIP_BIN/ The location and name of your Zipper/compressor.
* <tt/SYSLOG_PID/ The name of the file containing the Syslogd PID.
make target
make install
that would use the defaults given in the makefile, if you want to
override the defaults of the makefile you can use something like:
make target DEGT_LIB=/usr/lib/chklogs SYSLOG_PID=syslog.pid \
PERL_BIN=/usr/local/bin/perl \
GZIP_BIN=/usr/local/bin/gzip
make install
2.3.2 X-based installation
----------------------------
This is new on Chklogs-2000, this requires that you are able to
access the local display and that you have the wish (Tcl/Tk shell)
in your PATH. If you don't make sure you do or do a manual installation.
If it went well you will see that GuideMe has passed control to the
SetupChklogs script in the support directory. It is easy to use, most
important is the "Configuration" dialog invoked by pushing the button
of the same name.
The Configuration dialog will show you the entries of the Resource File,
as the defaults. In particular it will highlight those entries that
are invalid and that you need to modify on the dialog. Pressing ENTER
causes the script to verify the entry.
Once all your entries in the configuration dialog are ok, you can
proceed and confirm the installation. This will create a Global Resource
File in the /usr/local/lib/chklogs directory with the defaults you
have given. With this there is no need to modify the built-in defaults
in the scripts and make your upgrading easier.
Additionally there is no need to run 'make install' as this is done
by the SetupChklogs script once you confirm the correctness of your
configuration.
2.4 Upgrading from 1.8 or 1.9
-------------------------------
There have been some changes that require modification in your current
installation. In 1.9 the status file database was wrongly named 'resource
file' located in /var/log/.chklogs.rc , this has become
/usr/local/lib/chklogs/.chklogsdb . Make sure you move the file
to the appropriate directory with the new name. Also v1.9 used 2 digits
for the year thus unsuitable for the year 2000. The situation has been
alleviated by using 4 digits, old files are automatically updated when
Chklogs runs (not by the installation script).
- 5 -
Make sure you backup your existing configuration file.
2.5 Upgrading from much older versions
----------------------------------------
Consider this a new installation, start from scratch but remember what
you wanted to do with your logs. You cannot use the same configuration
file.
Chapter III SPECIAL CONCEPTS
Before going into more details about the program operations and
configuration I would like to introduce a couple of its features and
concepts: Log thresholds, Log groups, Repositories, Lumping and
Plug outs.
3.1 Log Thresholds
--------------------
For each and every log that you register you must specify a threshold,
when reached or exceeded ChkLogs will perform a well defined action of
your choice. The threshold can be either a size or an age.
If you wish to specify a threshold in bytes then this field is a
sequence of decimal digits representing the size, optionally followed
(immediately and without any spaces in between!) by a modifier. The only
valid modifier for size thresholds is for kilobytes: `k' or `K'.
Example: 20000, 3.5K, 7k
As of v1.8 ChkLogs also supports timed logs, in this case the threshold
field specifies an age in days or months in which case the modifier is
either `d' or `m' respectively and is case insensitive. Examples are
20d (20 days), 15D (15 days), 1m (1 month). A month is 30 days. These
represent the age of the log since the last time it was acted upon.
ChkLogs keeps track of this internally.
3.2 Log Groups (1.8+)
-----------------------
If you want to keep simple you will not bother with groups, but it might
be convenient if you administer a large site or a site with many
services.
Logs that are not put together in a logical group implicitely belong
to the `common' group, therefore this name is reserved.
Logical groups are particularly useful when using the Global Repository
option explained below.
Also, defining a logical group is useful when you have to perform a
specific action (or execute a program) before the group is processed or
checked and right after the group is processed/checked. These are known
as Pre and Post executions. The syslogd group of logs is already handled
internally by chklogs but you can define it for clarity.
You can easily create groups and add logs to groups (or the default
group `common') with the chklogsadm program using the newgroup and gadd
commands. It is also possible to modify the group attributes with the
same newgroup command.
3.3 Repositories (1.8+)
-------------------------
- 6 -
Previously (up to v1.7) ChkLogs used to keep the archived logs (*.gz)
in the same directory where the log resided, this resulted in cluttered
directories which is not very esthetical.
You can use either a Global Repository or a Local Repository by means
of options specified in the configuration file.
A Local Repository is simply a subdirectory (./OldLogs) below the
current directory where the log resides. So if Chklogs is in this mode
of operation and it is time to archive /var/log/samba.log then the
resulting archive samba.DDMMYY.gz will be stored in /var/log/OldLogs/.
The Global Repository is slightly more complicated, it is a hierarchical
collection of groups. If in Global Repository mode, and assuming the
Global Repository is /var/adm/chklogs/ then the archived logs will be
stored in subdirectories under the GR (/var/adm/chklogs in this example).
The names of the subdirectories are the same name of the defined groups
(i.e. /var/adm/chklogs/uucp if you defined an `uucp' group). All other
logs which are not in an explicit group fall into the `common'
subdirectory.
It is important to note that these (sub)directories *must* exist at the
time chklogs execute!. So make sure that is so even when you switch back
and forth between Global and Local options. You don't have to create
them by hand, the initrepos command of chklogsadm will do the job.
3.3.1 Archival Method
-----------------------
Archival takes place by invoking the compresser of your choice, I use
GNU Zip (gzip) but you are free to use whichever you have on your
system such as zip (.zip) or compress (.Z) as indicated by the $zipper
variable in the configuration section. If you decide to use any other
zipper/archiver make sure that you specify the right compressed file
extension in $zipext.
In earlier (pre 1.8) versions of ChkLogs the resulting archives where
left in the same directory where the log resided, now we make use of
the Repository concept explained earlier yielding to a more elegant
implementation.
3.4 Lumping (1.8+)
--------------------
Lumping is when you have a (relatively) large amount of log files of the
same kind, and for example today they can be 10 logs, next day 30 etc.
This feature is a request from a couple of large sites in which there
are just too many UUCP logs (for each leaf site for example) to list
separately.
In this case it is much easier to specify the directory name rather than
the logname (with full path). Chklogs will scan the whole directory -one
level deep- and consider any normal file (non-archive, non-directory) as
a log to control according to the specification.
3.5 Plug-outs
---------------
This is a similar concept to plug-ins except I didn't dare calling it
that. A plug out is simply an external program that is executed by
ChkLogs when needed or when certain conditions are fulfilled.
Chapter IV CONFIGURATION
- 7 -
Configuration of the Chklogs package has now become very flexible, now
there is no need to modify (only in extreme cases) the hard-coded
defaults because of the introduction of the Resource Files .
To clarify this point, Chklogs has four levels of configuration
precedence as follows:
Highest precedence: Command-line options
Midway precedence : Personal resource file
Lower precedence : Global resource file
Lowest precedence : In-script configuration
Table 4.1: Configuration options precedence
Meaning that whatever you configure on the script can be overriden by
the Global Resource File , which can in turn be overriden by the
Personal Resource File and so on. That is the reason why during
the (X-based) installation a global resource file is created and
installed, thus making it easier for you. Then you can proceed to tweak
the global one, or use a personal file in your home directory.
In order to make a good configuration you should consider the following:
1. Find your logs, some detective work needed by Unix newbies
2. Assign a (size/age) threshold for each one
3. Designate an action to perform when the threshold is reached
4. Where you want to put your archived logs
Before we continue into more details of the configuration, let's
introduce some terms. Make sure you get acquainted with them so that
Chklogs can do what you want. The rest of the sections tell you
more about the other files in the package (resource, configuration).
Then in the next chapter we will delve more into the details of
managing log groups and how it can be done with the administrative
program.
4.1 Configuration variables
-----------------------------
Though you are most likely to just make use of the Resource Files,
it might be a good idea to become acquainted with the mapping of
the configuration parameters of the resource files to the
configuration variables (hard coded defaults) used in the Perl
scripts, in particular chklogs ,chklogsadm and
Chklogs.pm .
Resource name Config variable Location
------------- --------------- ------------
ChklogsConf $ConfFile Chklogs.pm
ChklogsDb $ResrcFile Chklogs.pm
VarRun $VarRun Chklogs.pm
RelativePath $RelativePath Chklogs.pm
Admin $admin Chklogs.pm
SyslogConf $SyslogConf Chklogs.pm
MiniMail $useMiniMail Chklogs.pm
MailHost $mailhost Chklogs.pm
n.a. $zipper chklogs
n.a. $zipext chklogs
- 8 -
n.a. $syslogF chklogs
n.a. $maxlogs chklogs
n.a. $mailer chklogs
Table 4.2: Mapping of configuration variables
Those that have an equivalent in the resource file need no further
explanation as their purpose is clearly stated in the sample
resource presented in the respective section. Furthermore, the
configuration section of the script -if any- is clearly marked.
4.1.1 Description of the configuration variables
--------------------------------------------------
$zipper
The command used to invoke the program that will compress
the offending log. Defaults to /bin/gzip , but
/bin/compress also works. Notice that the PKZip
compatible version is not suitable to my interface.
$zipext
The extension given to the compressed file by the <tt/$zipper/.
Defaults to gz . If you chose compress for your zipper,
this must then be Z .
$syslogF
Location of syslog PID file. Default is <ttsyslog.pid/.
It is assumed to be in the path given by the VarRun
configuration variable/resource! (change with respect to 1.9),
however a full path will also be recognized for backwards
compatibility. In some systems (SunOS) this is
/etc/syslog.pid .
$maxlogs
Shuffling. Default Maximum nr. of logs allowed (per type)
before the oldest is removed in case the index file has
an illegal (or zero) value.
$mailer
This contains the command (with parameters) to invoke your
mail delivery agent (Sendmail in most systems). This is for
backwards compatibility with pre-2.0 versions. Chklogs is
configured to use its built-in mailer rather than spawn
this external program.
Do notice that if your system does not have an SMTP server locally or in
your localnet, the default configuration will not work as the built-in
mailer won't be able to connect. In this case you must disable it by
setting in the resource file(s) (outside the ignore block!):
set MiniMail no
Then Chklogs will fall back to the pre-2.0 mode and attempt to spawn the
mailer given in the $mailer configuration variable.
Other important configuration variables are:
$ConfFile
The configuration file. Defaults to `/etc/chklogs.conf'.
$ResrcFile
Not a resource file!. This is better named the status
database and defaults to
/usr/local/lib/chklogs/.chklogsdb Used to be
named .chklogs.rc .
- 9 -
$SyslogConf
Location of the <tt/syslogd/ configuration file, needed by
chklogs to arbitrate syslogd when it needs to work. It
defaults to /etc/syslog.conf which should work
on the great majority of systems.
$RelativePath
The relative name of the repository for newly created
archives if Local Repository option is chosen.
4.2 The Resource Files
------------------------
There can be two resource files, the global one which is by default
in /usr/local/lib/chklogs/chklogsrc , and the personal file
residing in your home directory (hidden file) as ~/.chklogsrc .
From now on I will use PRF and GRF as a short notation for Personal
Resource File and Global Resource File respectively.
It is not considered an error condition if any (or both) of the
resource files are missing -though I recommend having at least a
GRF.- If both are missing Chklogs will default to the
built-in (hard coded) values, so that is your choice and responsability
to properly configure the package.
In addition to making it easier to run and configure Chklogs, the PRF
makes it a snap to test Chklogs (as I do before releasing) on a
sandbox with dummy logs and dummy repositories before placing your
valuable logs in the hands of Chklogs. This is particularly good if you
do not have root permission!. I am sure you will have no problem trusting
your logs to Chklogs.
4.2.1 Semantics of the Resource File
--------------------------------------
A sample resource file might looks like this:
#
# Global Resource File for Chklogs 2.x
#
# -- ChklogsConf:
# Location of configuration file. Don't change.
set ChklogsConf /etc/chklogs.conf
# -- ChklogsDb:
# Location of history/age file.
set ChklogsDb /usr/local/lib/chklogs/.chklogsdb
# -- VarRun:
# Directory with the *.pid files, here we
# find syslog.pid and chklogs.pid
set VarRun /var/run
# -- RelativePath:
# Directory name (basename) for archived logs
# when -Options local is selected
set RelativePath OldLogs
# -- Admin:
# Email address of recipients of Chklogs
# activity summary
set Admin root
- 10 -
# -- MailHost:
# The fully qualified hostname to which we will connect
# via SMTP for mailing the report.
set MailHost localhost
# -- SyslogConf:
# Full path/filename of the Syslogd
# daemon configuration file
set SyslogConf /etc/syslog.conf
# -- Ignore Control
# Anything between `ignore on' and `ignore off'
# will *not* modify the configuration variable
# in question.
mode ignore on
# -- MiniMail:
# yes/no. Only use `no' as a fallback when
# you don't have SMTP server. In that case
# you must have Sendmail and configure the
# mailer on chklogs. Default is 'yes'
set MiniMail no
Please note that it has kind of a ticklish (Tcl) syntax, I personally
find the capitalistic use of dollar signs very annoying in Perl :-).
Also, if you do not have root permissions you can still install
and test Chklogs in the so called sandbox, for that I recommend
using a PRF where you override the ChklogsConf , ChklogsDb ,
VarRun and Admin variables.
Bear in mind that anything that is in between the ignore lines
is ignored. You can find this information in more detail in the
chklogsrc(5) manual page.
4.3 The Configuration File
----------------------------
If this is the first time you install ChkLogs you will have to do a bit
of investigative work and find out where your system logs are. A good
start might be with /etc/syslog.conf . These days (at least in Linux)
they seem to be in the /var/log directory, but they might be spread
all over or in a combination of that, /var/adm and other places.
Once you have listed the (fully qualified path) names of the system logs
you have to take a few other decisions about what to make with them,
this is explained in more detail later.
The configuration file location is specified with the $ConfFile
variable mentioned before. This file consists of single line entries,
you cannot span an entry into 2 or more lines.
4.3.1 Semantics of the configuration file
-------------------------------------------
There are three types of entries/lines in this file:
1. Comment lines, starting with `#' in the first column.
2. Instruction lines, beginning with the `#:' character combination and
are used for run time configuration and special commands. These
lines consist of `#:' immediately followed by a token-value pair.
As of v2.0 Chklogs also accepts `-' as the instruction line marker.
3. Specification lines describing a system log and associated actions.
- 11 -
Empty lines are also allowed in the configuration file. Furthermore, this
file is divided -without any particular delimiters- into two logical
sections: Header and Definitions.
4.3.1.1 Header section
------------------------
Needless to say the header section appears at the top of the
configuration file and usually looks like this:
#################################################################
# ChkLogs v1.9 configuration file for host panama
#
# Copyright (C)1995,1996,1997 D. Emilio Grimaldo T.
#
-Options global
-Global /var/log/chklogs
As shown in this example it has a comment line(s) of your choice and two
compulsory instruction lines in precisely that order.
Options
When set to `local' the generated archives of the given log will
be moved to the "./OldLogs" subdirectory referred as the Local
Repository. If the option is `global' chklogs will store the
archives into a Global Repository in a hierarchy of log groups.
You cannot specify both local and global at the same time.
Global
Specifies the location of the root of the Global Repository, it
is ignored if the `local' option is used. This must be an
absolute path, furthermore the directory must exist (see
chklogsadm)
In the future other options might be added.
4.3.1.2 Definition section
----------------------------
This section contains at least specification lines but can also have
instruction and comment/empty lines. Obviously groups and lumps are
defined in this section.
A specification line is defined as follows:
FULL-LOG-NAME THRESHOLD ACTION [ PARAMETER(S) ]
The first field is an absolute path. Normally it includes the log name.
If Chklogs finds this is a directory instead of a regular file (log) it
will recursively process all normal files in that directory as per the
given specification. This was explained in the Lumping section.
The second field is the threshold in size or age as explained in a
previous section.
The third field defines an action to be performed if the logname
fulfills the threshold condition. The action field may or may not be
followed by a parameter(s) depending on the action field. The action can
be archive, truncate or execute. The action name must be in lowercase.
Archive
- 12 -
When this action is specified ChkLogs will create a compressed
archive of the log and place it in the repository. This requires
a parameter indicating the maximum number of archives you want
to exist in the repository at any given time (for that log). If
no maximum is given, or set to zero then ChkLogs will use an
internal value specified by $maxlogs. Once that many archives
are present ChkLogs will do the so called shuffling or log
rotation therefore saving your disk space. This is advisable for
important logs which you may want/need to look in detail later.
Example:
/var/log/cron 50.5K archive 2
Truncate
This is normally used for logs you don't care much about. It
takes no parameters and simply means that when the threshold is
reached ChkLogs will TRUNCATE TO ZERO SIZE, no backups, no
archives. Example:
/var/log/Xerrors 3d truncate
Execute
Here is where the plug-outs come in. It takes at least one
parameter which indicates which script/program to execute when
the threshold is reached. This is useful when you want to do
your own handling of the log, or when a program has its own
specific log handling. Likewise you can use it coupled with any
of the above two actions in two consecutive entries, the example
below shows the use of a plug out to generate a summary based on
the contents of the log and then proceed to truncate/discard
(or possibly archive) the log:
/var/log/maillog 1m execute po_mailstats -f %L
/var/log/maillog 1m archive 2
Note that there is no mention of a Warn action, this is because that
is taken care of by a command line option. Command line options are
explained later.
4.4 Plug-out interface
------------------------
Plug-outs (no pun intended ;-) is ChkLogs way of adding new actions
and be as flexible as possible. For this to work a few rules have to
be followed:
1. Your program must not generate any output on STDOUT or you will
get problems if you run it on cron(8) and in the other case it
will spoil and clutter the report produced by chklogs. Best thing
is when output is stored in a file and possibly mailed.
2. ChkLogs will not pass any parameters of its own making, only
the parameters that you give in the specification line.
If you also need to pass your plug-out the name (including path) of
the log file you need not to write it all over again, simply put
a %L (uppercase l) at the parameter position you wish and ChkLogs will
replace it at run time by the fully qualified log name.
Usually the programs that generate the logs also have another program
(and sometimes an utility written by someone else) that takes a look
at the log and filters it out to produce a specialized report with
more meaningful data. Sometimes these programs are not directly
suitable as a ChkLogs plug-out, in such case you can get away by
- 13 -
wrapping it in a Perl or shell script.
As of v1.8 I have decided to distribute some plug-out examples
that I use on my own LAN. These are located in the plugout subdirectory.
If you have something you want to use but doesn't conform to the
above mentioned requirements, you can use with by means of the
wrapper (new in 2.0) program cdkwrap described in Appendix A. It
will even mail you the results.
Last but not least, every program spawned via the execute interface
(plug out) will have access to the following environment variables:
CDKROOT
Directory were the log is to be placed when archived.
CDKLOG
The fully qualified name of the log (path and basename).
CDKMAILTO
Who is to be the mail recipient of the results.
4.4.1 Contributions
---------------------
If you have a plugout of your own and think it is general and useful
to other ChkLogs users send me a mail describing it and where it
can be found, I will then add a link in my web page to your plugout.
Chapter V THE ADMINISTRATIVE PROGRAM: ChklogsAdm
In this chapter you will get acquainted with the Chklogs
Administrative program. Here there is only some extra or
complementary information, the rest you can find in the
chklogsadm(8) manual page.
Each of the sections will take you step by step through the most
important parts of the process of tailoring Chklogs to your
system or network.
The following subcommands are recognized by chklogsadm , those
marked with an asterisk indicate that there is no manual way to
do it, the others you can use the subcommand via chklogsadm or
edit the file yourself if you know what you are doing. I recommend
using the program as I get several reports of problems that are
caused by wrong configurations.
Command Action
---------- --------------------------
newconf Create a new configuration
init (*) Initialize package
initrepos (*) Initialize repositories
newgroup Create a new log group
rmgroup Remove a log group definition
gadd Add a log to the configuration
del Delete a log from the configuration
sync (*) Synchronize package
when (*) Query status database
syslog (*) Query syslog configuration
Table 5.1: Chklogsadm commands
5.1 Creating a configuration: newconf
---------------------------------------
- 14 -
You can do it by hand using your favourite editor to bring to life
a new chklogs.conf file. Another alternative is to use the
administrative program to do it. Say, if you want to create a
configuration using a global repository located at
/var/log/Chklogs/ you would type:
chklogsadm newconf -global /var/log/Chklogs
whereas if you want to use a local repository the command would be:
chklogsadm newconf -local
If there is already a chklogs.conf file the command will
abort, if you really want to wipe out your previous configuration
then use the -force option.
5.2 Initializing the internal database: init
----------------------------------------------
It is not very sensible to do it by hand. Furthermore you only
need to do this once, when you install Chklogs for the first time
(for versions 1.8 or greater). This is so that Chklogs knows when
it had done what to whom and is specially important if you use
log aging. If you don't you still need this command!. Simply type:
chklogsadm init
5.3 Initializing the repositories: initrepos
----------------------------------------------
This is not to be confused with init . You should perform this
operation everytime you:
* Created a group
* Switched between global and local repositories
* Added a log to the configuration
The command itself is very simple:
chklogsadm initrepos
If you don't Chklogs will not work as we all expect and will
leave the archived logs in their current directory, thus cluttering
your system. Not a pretty sight.
5.4 Synchronizing the package: sync
-------------------------------------
When changes are done with the administrative program, it is necessary
to resynchronize the status database with the following command:
chklogsadm sync
Conditions that require this step are:
* Added a log to the configuration
* Removed a log to the configuration
- 15 -
5.5 Managing log groups
-------------------------
By default all logs belong to the reserved group 'common' but you
can create your own logical groups, either for clarity (a comment
would do on this case) or to do some special processing. The following
rules apply:
1. The 'common' group name is reserved. It has no headers.
2. There is no limit in how many logs a group can have
3. The group header must be in the exact same order I describe.
4. The first empty line ChkLogs finds after the group header
is assumed to be the end-of-group marker. Therefore make sure
there is an empty line *after* the last specification line of
the group.
A group header and body looks like this:
#:Group test
#:Pre /bin/cdk_stub start
#:Post /bin/cdk_stub stop
/var/log/debug 20000 truncate
/var/log/maillog 5000 execute mailst -f %L
/var/log/samba.log 10000 truncate
The example shows the required empty line to mark the end of the
group. After that you can have yet another group header, or another
empty line or a comment line or a specification line. The samba log
does not belong to the 'test' group.
In this example I defined the 'test' group. To this group I have
associated two actions, one (Pre) is executed before the first
log is checked, and the other (Post) is executed *after* the last
log of the group is checked. You can specify parameters. In this
example I have two logs registered with the group.
You must still have a Pre and Post entry (exactly in that order!)
even if you do not want to make use of the feature. In that case
simply leave the parameter or Pre/Post empty. Otherwise Chklogs
will not consider this a group as its definition was incomplete,
reasonable enough isn't it?
Another example of the use of a group would be to have a group for
the WWW server (httpd and alike) in which you would include the
appropriate Pre and Post actions to control your daemon so that
it doesn't write to the log files while ChkLogs is processing them.
With the grouping the 'down' time of the daemon is minimized.
Now you can use Chklogs with anything.
5.5.1 Creating a log group: newgroup
--------------------------------------
Since there is some administrative info that ChkLogs must know
about it is better that you don't create groups by hand. Instead
you must use the Administrative script chklogsadm, for this
example I would do:
chklogsadm newgroup -g test -b "/bin/cdk_stub start" \
-a "/bin/cdk_stub stop"
- 16 -
Here the Pre and Post actions/parameters are specified via the
(b)efore and (a)fter command line options. All parameters are
compulsory. Do notice the use of quotes otherwise the shell will
split it further and confuse chklogsadm. If the script (cdk_stub)
uses no parameters then you don't need the quotes. If the group
already exists the operation results in an error.
The Pre/Post actions are the group attributes.
5.5.2 Adding a log to the group: gadd
---------------------------------------
chklogsadm gadd -g test -l /var/log/debug -s 20000 -t
chklogsadm gadd -g test -l /var/log/maillog -s 5000 -e "mailst -f %L"
Again notice the use of quotes when passing parameters to one of
the options.
When adding a log to an undefined group you must use `-g common'.
This is the generalized group.
Once done you must do two things to insure that the (new) repositories
that might be needed are created/initialized and that the internal
administration data is synchronized. The following two commands are
needed:
chklogsadm initrepos
chklogsadm sync
See the special instructions in the Installation section, you might
want to delay these two commands until later when you have created
a complete configuration file.
5.5.3 Remove a group: rmgroup
-------------------------------
If you only need to remove the group identification while keeping
the logs, simply edit the configuration file and remove the
group header.
As of release 1.9 it is also possible (recommended) to use the
administrative program to remove it. For example if you wish to
remove the 'test' group mentioned above this does the job:
chklogsadm rmgroup -g test
5.5.4 Deleting a log from the configuration: del
--------------------------------------------------
If on the other hand you want to remove the whole group, including
the log entries (unlikely I think) then can either edit the
configuration file by hand or run the administrative 'del' command
for each of the logs. For the test group mentioned above this
would be:
chklogsadm del -l /var/log/debug -t T
chklogsadm del -l /var/log/maillog -t E
- 17 -
Notice that in addition to the log path you *must* specify the
action tag, valid ones are A for archive, T for truncate and E
for execute. You cannot use the full tag (i.e. 'archive'). The
reason for this is that you might have two entries for the same
log, one for either truncate or archive and the other for execute.
The administrative program needs to know which one always, even
if there is only one (only one pass is done).
Whenever you add or remove logs to the configuration you must
use:
chklogsadm sync
For more information about the Administrative script see the
chklogsadm(8) manual page.
5.5.5 Modifying group attributes
----------------------------------
Exactly the same format of the newgroup command as described in
"Creating a new group" the only difference is the addition of the
-m (modify) flag. Even if you only want to change one of the
attributes (pre or post) you still need to specify the other.
For example if you want to modify the group attributes of the
test group from:
#:group test
#:pre /bin/cdk_stub start
#:post /bin/cdk_stub stop
to:
#:group test
#:pre /bin/my_plugout start
#:post /bin/another_plugout expire
the following administrative command will do it:
chklogsadm newgroup -m -g test -b "/bin/my_plugout start" \
-a "/bin/another_plugout expire"
5.5.6 Contributions
---------------------
If you have a Pre/Post program or script of your own and think it is
general and useful to other ChkLogs users send me a mail describing it
and where it can be found, I will then add a link in my web page to
your contribution.
Chapter VI MANAGING YOUR LOGS: Chklogs
So far we have covered how to adapt the ChkLogs scripts/library
for your system (where to find programs, interpreter etc.), we have
also discussed how to create a configuration file to describe
- 18 -
what and when to do something with your logs. Now it is a good time
to introduce the operational details, how to run ChkLogs.
Although most of the times you will run it from a crontab, you might
want to use it directly from the command line. The Warn option is
particularly useful if you want to know what ChkLogs would do to
your logs before actually commiting to a crontab. It is always a bit
scary to try a new program ;-)
At run time Chklogs will compare the maximum allowable sizes/ages
of each log -as given in the configuration file- against the actual
size/age of the log file being examined. Then a report is produced
that will include as part of the header:
a) The location of the configuration file
b) The date the report was produced, and the hostname
c) Column headings for the contents of the report
The body of the report is very much similar to the configuration
file but includes current information against the specified thresholds,
whether an action was taken or not and finally a trailer.
The syntax of the ChkLogs command is as follows:
chklogs
chklogs -t
chklogs -c
chklogs [ -m | -w | -a ]
In the first form (no options) chklogs will process all the logs
according to the default action specified in the configuration file and a
report will be produced on stdout. This reports includes the
following for *each* of the examined system logs:
1) The fully qualified filename of the system logfile
2) The current size/age of the log
3) The maximum allowed size/age as specified in the
configuration file
4) Either Action taken (archived, truncated, execute, ok) or
action to be taken (archive!, truncate!, execute, ok)
If a log has reached or exceeded the threshold, the action column
will show the action taken by the program, that is whether it was
archived, truncated or the execution of an external program/plug-out.
These actions and their interfaces have been described earlier.
The second form (-t option) will parse the configuration file and
produce a listing with the same header as above but with the following
info per log file:
1) Fully qualifie
