# flume-syslog-source2

**Repository Path**: mirrors_spotify/flume-syslog-source2

## Basic Information

- **Project Name**: flume-syslog-source2
- **Description**: Improved Syslog source for Flume
- **Primary Language**: Unknown
- **License**: Not specified
- **Default Branch**: master
- **Homepage**: None
- **GVP Project**: No

## Statistics

- **Stars**: 0
- **Forks**: 0
- **Created**: 2020-08-18
- **Last Updated**: 2026-05-02

## Categories & Tags

**Categories**: Uncategorized

**Tags**: None

## README

Flume syslog source 2
=====================
Tommie Gannert <tommie@spotify.com>
2011-08-25


This project defines a plugin for Flume containing an improved Syslog source.

It parses http://tools.ietf.org/rfc/rfc3164.txt[RFC 3164] (BSD Syslog)
as well as the newer http://tools.ietf.org/rfc/rfc5424.txt[RFC 5424] (Syslog.)
The source goes to a greater length to create a Flume event than the built-in
version. All fields possible are parsed, and the body is just the body of
the message. Even the Tag and PID fields are parsed. This should leave clean
and normalized log messages, at the expense of more CPU cycles.

The data is interpreted as ASCII octets where possible. The notable exception
is the hostname, which Flume stores as a String, and must thus be decoded.
Where applicable, decoding to Unicode assumes the octet stream is encoded
in UTF-8.


Usage
-----
Build with `mvn package`. The produced Jar should be installed like any Flume
plugin, and the plugin class to be added to `flume.plugin.classes` in the
configuration file is called `com.spotify.flume.syslog2.SyslogSource2Plugin`.

The source is accessed using the name `syslog2`, and allows both TCP and UDP
connections.

For TCP, the syntax is

  syslog2(tcp[, host[, port]])

and for UDP

  syslog2(udp[, host[, port[, bufferSize]]])

The default port is 514, the well-known Syslog port. The buffer size (which is
the maximum size of an incoming UDP packet) is 64 kB unless overridden.


Caveats
-------
Note that the structured data of RFC 5424 is parsed, but not split into
multiple fields. This is for space effiency. If you need it split, add a
sink decorator for it.

The code hasn't been tested in Flume yet.


Implementation
--------------
The `SyslogParser` class is the main class and implements a recursive decent
parser based on an `InputStream`. Most bytes are handled without concern for
encoding, for speed and robustness. The philosophy is; it's better to get the
data into Flume, than rejecting it. This might not be suitable for you, but
then you have the option of writing a decorator to handle that.


License
-------
____
  Copyright 2011 Spotify Ltd

  Licensed under the Apache License, Version 2.0 (the "License");
  you may not use this file except in compliance with the License.
  You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

  Unless required by applicable law or agreed to in writing, software
  distributed under the License is distributed on an "AS IS" BASIS,
  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  See the License for the specific language governing permissions and
  limitations under the License.
____