[122957] in North American Network Operators' Group
Re: log parsing tool?
daemon@ATHENA.MIT.EDU (Matthew Palmer)
Wed Feb 24 00:44:23 2010
Date: Wed, 24 Feb 2010 16:43:38 +1100
From: Matthew Palmer <mpalmer@hezmatt.org>
To: nanog@nanog.org
Mail-Followup-To: nanog@nanog.org
In-Reply-To: <f8bb772a1002221415s150a09e2k6a41cad8402237d6@mail.gmail.com>
Errors-To: nanog-bounces+nanog.discuss=bloom-picayune.mit.edu@nanog.org
On Mon, Feb 22, 2010 at 04:15:22PM -0600, fedora fedora wrote:
> Anyone has good recommendations for an open-sourced log parsing and
> analyzing application? It will be used to work with syslog-ng and other
> general syslog and application logs.
>
> I have been looking at swatch and logwatch, but would like to find out if
> there are other good choices, thanks
SEC does seem to be the "gold standard" in advanced log correlation beyond
that available in "grep | mail" type systems such as logwatch. However it
is incredibly arcane, and despite reading a lot of documentation for it I've
never really been able to wrap my head around it.
A colleague has started to write a SEC-like tool with (I hope) a more
approachable mental model; take a look at http://github.com/rodjek/grok. I
must (embarrasedly) admit I haven't looked at it yet, but he claims that he
reimplemented sshd_sentry (the fail2ban equivalent we use) in two lines of
rules, which seems like a nice (basic) demonstration.
- Matt
