1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
|
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html><head>
<meta http-equiv="Content-Language" content="en">
<title>Field Extraction Module (mmfields)</title></head>
<body>
<a href="rsyslog_conf_modules.html">back</a>
<h1>Fields Extraction Module (mmfields)</h1>
<p><b>Module Name: mmfields</b></p>
<p><b>Author: </b>Rainer Gerhards <rgerhards@adiscon.com></p>
<p><b>Available since</b>: 7.5.1</p>
<p><b>Description</b>:</p>
<p>The mmfield module permits to extract fields. It is an alternate to using
the property replacer field extraction capabilities. In contrast to the
property replacer, all fields are extracted as once and stored inside the
structured data part (more precisely: they become Lumberjack [JSON] properties).
<p>Using this module is of special advantage if a field-based log format
is to be processed, like for example CEF <b>and</b> and either a large
number of fields is needed or a specific field is used multiple times
inside filters. In these scenarios, mmfields potentially offers better
performance than the property replacer of the RainerScript field extraction
method. The reason is that mmfields extracts all fields as one big sweep,
whereas the other methods extract fields individually, which requires
multiple passes through the same data. On the other hand, adding field
content to the rsyslog property dictionary also has some overhead,
so for high-performance use cases it is suggested to do some performance
testing before finally deciding which method to use. This is most important
if only a smaller subset of the fields is actually needed.
<p>In any case, mmfields provides a very handy and easy to use way to
parse structured data into a it's individual data items. Again, a primiary
use case was support for CEF (Common Event Format), which is made
extremely easy to do with this module.
<p>This module is implemented via the action interface. Thus it
can be conditionally used depending on some prequisites.
<p> </p>
<p><b>Module Configuration Parameters</b>:</p>
<p>Currently none.
<p> </p>
<p><b>Action Confguration Parameters</b>:</p>
<ul>
<li><b>separator</b> - separatorChar (default ',')<br>
This is the character used to separate fields. Currently, only a single
character is permitted, while the RainerScript method permits to
specify multi-character separator strings. For CEF, this is not required.
If there is actual need to support multi-character separator strings,
support can relatively easy be added. It is suggested to request it on the
rsyslog mailing list, together with the use case - we intend to add
functionality only if there is a real use case behind the request
(in the past we too-often implemented things that actually never got used).
<br>The fields are named f<i>nbr</i>, where <i>nbr</i> is the field number
starting with one and being incremented for each field.
<li><b>jsonRoot</b> - path (default "!")<br>
This parameters specifies into which json path the extracted fields shall
be written. The default is to use the json root object itself.
</ul>
<p><b>Caveats/Known Bugs:</b>
<ul>
<li>Currently none.
</ul>
<p><b>Samples:</b></p>
<p>This is a very simple use case where each message is
parsed. The default separator character of comma is being used.
<p><textarea rows="5" cols="60">module(load="mmfields")
template(name="ftpl" type=string string="%$!%\n")
action(type="omfields")
action(type="omfile" file="/path/to/logfile" template="ftpl")
</textarea>
<p>The following sample is similar to the previous one, but
this time the colon is used as separator and data is written
into the "$!mmfields" json path.
<p><textarea rows="5" cols="60">module(load="mmfields")
template(name="ftpl" type=string string="%$!%\n")
action(type="omfields" separator=":" jsonRoot="!mmfields")
action(type="omfile" file="/path/to/logfile" template="ftpl")
</textarea>
<p>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>] [<a href="manual.html">manual
index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p>
<p><font size="2">This documentation is part of the
<a href="http://www.rsyslog.com/">rsyslog</a> project.<br>
Copyright © 2013 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and
<a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL
version 3 or higher.</font></p>
</body></html>
|
