diff options
author | Igor Pashev <pashev.igor@gmail.com> | 2014-10-01 17:56:20 +0400 |
---|---|---|
committer | Igor Pashev <pashev.igor@gmail.com> | 2014-10-01 17:56:20 +0400 |
commit | c046f7bcc92281465917e026f83fd0d38569cb06 (patch) | |
tree | 711f61cf319e171a5f41c469ef30e3298c8917f8 | |
parent | 17262528e2277c3d069c4a29ed098830d4fdbc08 (diff) | |
parent | 7ec8c6d6f9114765775ea5100af5b0b20af4502e (diff) | |
download | rsyslog-c046f7bcc92281465917e026f83fd0d38569cb06.tar.gz |
Merge branch 'master' of git://anonscm.debian.org/collab-maint/rsyslog
Conflicts:
debian/changelog
debian/patches/series
debian/rules
550 files changed, 28612 insertions, 27671 deletions
@@ -1,4 +1,1184 @@ --------------------------------------------------------------------------- +Version 8.4.1 [v8-stable] 2014-09-30 +- imudp: add for bracketing mode, which makes parsing stats easier +- permit at-sign in variable names + closes: https://github.com/rsyslog/rsyslog/issues/110 +- bugfix: fix syntax error in anon_cc_numbers.py script + Thanks to github user anthcourtney for the patch. + closes: https://github.com/rsyslog/rsyslog/issues/109 +- bugfix: ompgsql: don't loose uncomitted data on retry + Thanks to Jared Johnson and Axel Rau for the patch. +- bugfix: imfile: if a state file for a different file name was set, + that different file (name) was monitored instead of the configured + one. Now, the state file is deleted and the correct file monitored. + closes: https://github.com/rsyslog/rsyslog/issues/103 +- bugfix: omudpspoof: source port was invalid + Thanks to Pavel Levshin for the patch +- bugfix: build failure on systems which don't have json_tokener_errors + Older versions of json-c need to use a different API (which don't exists + on newer versions, unfortunately...) + Thanks to Thomas D. for reporting this problem. +- bugfix: omelasticsearch does not work with broken/changed ES 1.0+ API + closes: https://github.com/rsyslog/rsyslog/issues/104 +- bugfix: mmanon did not properly anonymize IP addresses starting with '9' + Thanks to defa-at-so36.net for reporting this problem. + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=529 +- bugfix: build problems on SuSe Linux + Thanks Andreas Stieger for the patch +- bugfix: omelasticsearch error file did not work correctly on ES 1.0+ + due to a breaking change in the ElasticSearch API. + see also: https://github.com/rsyslog/rsyslog/issues/104 +- bugfix: potential abort when a message with PRI > 191 was processed + if the "pri-text" property was used in active templates, this could + be abused to a remote denial of service from permitted senders + see also: CVE-2014-3634 +--------------------------------------------------------------------------- +Version 8.4.0 [v8-stable] 2014-08-18 +- this is the new stable branch, which incorporates all enhancements of + rsyslog 8.3. +--------------------------------------------------------------------------- +Version 8.3.5 [v8-devel] 2014-08-05 +- mmjsonparse: support selectable cookie and target containers + This permits to put different meanings into a json formatted syslog + message, e.g. the "traditional" cee or cim data. +- bugfix: mmjsonparse did not build with json-c < 0.10 + This was a regression introduced some time in the past in order to + support API changes in json-c. Now we check for the version and use + proper code. +- omprog: emit error message via syslog() if loading binary fails + This happens after forking, so omprog has no longer access to rsyslog's + regular error reporting functions. Previously, this meant any error + message was lost. Now it is emitted via regular syslog (which may end up + in a different instance, if multiple instances run...) +- couple of patches imported from v7-stable (7.6.4) +--------------------------------------------------------------------------- +Version 8.3.4 [v8-devel] 2014-07-11 +- new pmciscoios parser supporting various Cisco IOS formats +- RFC3164 timestamp parser now accepts timezones and subsecond resolution + ... at least for some common formats and where we could do so without + running risk of breaking proper formats (or introducing regressions) +- new parser config object -- permits to define custom parser definitions +- new tzinfo config object -- permits to define time zone offsets + This is a utility object that currently is being used by some parsers. +- bugfix: mishandling of input modules not supporting new input instances + If they did not support this, accidently the output module part of the + module union was written, leading to unpredictable results. Note: all + core modules do support this interface, but some contributed or very + old ones do not. +- bugfix: double-free when ruleset() parser parameters were used + While unlikely, this could cause stability issues even after the + config phase. +--------------------------------------------------------------------------- +Version 8.3.3 [v8-devel] 2014-06-26 +- unify input object naming + imudp now supports "name" paramter, as other inputs do. "inputname" has + been deprecated, but can still be used. Same applies to "appendport" + subparamter". Thanks to "Nick Syslog" for the suggestion. +- made the missing (contributed) modules build under v8 [import from 8.2.2] + Modules: + * mmrfc5424addhmac + * omrabbitmq + * omgssapi + * omhdfs + * omzmq3 +- added a cleanup process (janitor); permits to close omfile files after a + timeout +- make omgssapi build under v8.3 [import vom v8.2] + note that we could do this to the stable, because there is NO regression + chance at all: only omgssapi was changed, and this module did NOT work + previously. +- removed obsolete --disable-fsstnd configure option + Thanks to Thomas D. for alerting us. + Closes: https://github.com/rsyslog/rsyslog/issues/72 +--------------------------------------------------------------------------- +Version 8.3.2 [v8-devel] 2014-05-02 +- new template options for date extraction: + - year + - month + - day + - wday + - hour + - minute + - second + - tzoffshour + - tzoffsmin + - tzoffsdirection + - wdayname + For string templates, these are property options and they are + prefixed with "date-" (e.g. "date-year", "date-month", ...) + see also: https://github.com/rsyslog/rsyslog/issues/65 +- bugfix: mmexternal remove framing char before processing JSON reply + This did not have any real bad effects, but caused unnecessary + processing, as empty replies were not properly detected. Otherwise, + the bug was not noticible from the user's PoV. +- bugfix: mmexternal segfault due to invalid free in non-json input mode + closes: https://github.com/rsyslog/rsyslog/issues/70 +- bugfix: mmexternal segfault when external plugin sent invalid reply + ... or no reply at all. This happened if the reply was imporper JSON. + Now, we emit an error message in those cases. + see also: https://github.com/rsyslog/rsyslog/issues/69 +- bugfix: mmexternal did potentially pass incomplete data to restarted + external plugin + This could happen if EPIPE was returned "too late", in which case the + beginning of the data could be lost. +- bugfix: mmexternal did not properly process messages over 4KiB + The data to be passed to the external plugin was truncated after 4KiB. + see: https://github.com/rsyslog/rsyslog/issues/64 +- imrelp: added support for per-listener ruleset and inputname + see: https://github.com/rsyslog/rsyslog/pull/63 + Thanks to bobthesecurityguy github user for the patch +--------------------------------------------------------------------------- +Version 8.3.1 [v8-devel] 2014-04-24 +- external message modification interface now support modifying message PRI +- "jsonmesg" property will include uuid only if one was previously generated + This is primarily a performance optimization. Whenever the message uuid + is gotten, it is generated when not already present. As we used the + regular setter, this means that always the uuid was generated, which is + quite time-consuming. This has now been changed so that it only is + generated if it already exists. That also matches more closly the + semantics, as "jsonmesg" should not make modifications to the message. + Note that the same applies to "fulljson" passing mode for external + plugins. +- added plugin to rewrite message facility and/or severity + Name: fac-sever-rewrite.py +- permits to build against json-c 0.12 + Unfortunately, json-c had an ABI breakage, so this is necessary. Note + that versions prior to 0.12 had security issues (CVE-2013-6370, + CVE-2013-6371) and so it is desirable to link against the new version. + Thanks to Thomas D. for the patch. Note that at least some distros + have fixed the security issue in older versions of json-c, so this + seems to apply mostly when building from sources. +- bugfix: using UUID property could cause segfault +- bugfix/mmexternal: memory leak +- bugfix: memory leak when using "jsonmesg" property +- bugfix: mmutf8fix did not detect two invalid sequences + Thanks to Axel Rau for the patch. +- bugfix: build problems with lexer.l on some platforms + For some reason, the strdup() prototype and others are missing. I admit + that I don't know why, as this happens only in 8.3.0+ and there is no + indication of changes to the affected files. In any case, we need to + fix this, and the current solution works at least as an interim one. +--------------------------------------------------------------------------- +Version 8.3.0 [v8-devel] 2014-04-10 +- new plugin for anonymizing credit card numbers + Thanks to Peter Slavov for providing the code. +- external message modification modules are now supported + They are bound via the new native module "mmexternal". Also, a sample + skeleton for an external python message modification module has been + added. +- new $jsonmesg property with JSON representation of whole message object + closes: https://github.com/rsyslog/rsyslog/issues/19 +- improved error message for invalid field extraction in string template + see also: + http://kb.monitorware.com/problem-with-field-based-extraction-t12299.html +- fix build problems on Solaris +- NOTE: a json-c API that we begun to use requires the compiler to be in + c99 mode. By default, we select it automatically. If you modify this and + use gcc, be sure to include "-std=c99" in your compiler flags. This seems + to be necessary only for older versions of gcc. +--------------------------------------------------------------------------- +Version 8.2.3 [v8-stable] 2014-??-?? +- bugfix: ommysql: handle/mem leak upon termination of worker thread + This could become bad if the (instance) worker threads are often + started and terminated. But it takes quite a while to show effect. +--------------------------------------------------------------------------- +Version 8.2.2 [v8-stable] 2014-06-02 +- made the missing (contributed) modules build under v8 + Note that we could do this to the stable, because there is NO regression + chance at all: only the modules themselves were changed, and they did + NOT work at all previously. Please also note that most of these modules + did not yet receive real testing. As we don't have the necessary + environments (easily enough available), we depend on users submitting + error reports and helping to iron out any issues that may arise. + Modules: + * mmrfc5424addhmac + * omrabbitmq + * omgssapi + * omhdfs + * omzmq3 +--------------------------------------------------------------------------- +Version 8.2.1 [v8-stable] 2014-04-17 +- permits to build against json-c 0.12 + Unfortunately, json-c had an ABI breakage, so this is necessary. Note + that versions prior to 0.12 had security issues (CVE-2013-6370, + CVE-2013-6371) and so it is desirable to link against the new version. + Thanks to Thomas D. for the patch. Note that at least some distros + have fixed the security issue in older versions of json-c, so this + seems to apply mostly when building from sources. +- doc is no longer shipped as part of the rsyslog tarball + Instead, the rsyslog-doc project creates its own tarball. This is the + result of a mailing list discussion after the 8.2.0 release with a + tarball-in-tarball approach, which was disliked by almost all distro + maintainers. This move also has the advantage of de-coupling the + release cycles of both projects a bit (which turned out to be a bit + problematic in practice). +- bugfix: mmutf8fix did not detect two invalid sequences + Thanks to Axel Rau for the patch. +--------------------------------------------------------------------------- +Version 8.2.0 [v8-stable] 2014-04-02 +This starts a new stable branch based on 8.1.6 plus the following changes: +- we now use doc from the rsyslog-doc project + As such, the ./doc subtree has been removed. Instead, a cache of the + rsyslog-doc project's files has been included in ./rsyslog-doc.tar.gz. + Note that the exact distribution mode for the doc is still under + discussion and may change in future releases. + This was agreed upon on the rsyslog mailing list. For doc issues + and corrections, be sure to work with the rsyslog-doc project. It is + currently hosted at https://github.com/rsyslog/rsyslog-doc +- add support for specifying the liblogging-stdlog channel spec + new global parameter "stdlog.channelspec" +- add "defaultnetstreamdrivercertfile" global variable to set a default + for the certfile. + Thanks to Radu Gheorghe for the patch. +- omelasticsearch: add new "usehttps" parameter for secured connections + Thanks to Radu Gheorghe for the patch. +- "action resumed" message now also specifies module type + which makes troubleshooting a bit easier. Note that we cannot output all + the config details (like destination etc) as this would require much more + elaborate code changes, which we at least do not like to do in the + stable version. +- add capability to override GnuTLS path in build process + Thanks to Clayton Shotwell for the patch +- better and more consistent action naming, action queues now always + contain the word "queue" after the action name +- bugfix: ompipe did resume itself even when it was still in error + See: https://github.com/rsyslog/rsyslog/issues/35 + Thanks to github user schplat for reporting +- bugfix: ompipe used invalid default template + This is a regression from an old change (didn't track it down precisely, + but over a year ago). It used the Forwarding template instead of + the file template (so we have a full syslog header). This fix corrects + it back to previous behaviour, but new scripts that used the wrong + format may now need to have the RSYSLOG_ForwardingFormat template + explicitely be applied. + closes: https://github.com/rsyslog/rsyslog/issues/50 +--------------------------------------------------------------------------- +Version 8.1.6 [release candidate] 2014-02-20 +- omfile: permit to set global defaults for action parameters + Thanks to Nathan Brown for the patch. + See also: https://github.com/rsyslog/rsyslog/pull/23 +- add capability to escape control characters in the C way of doing it + adds new global parameter "parser.escapeControlCharactersCStyle" + Thanks to Nathan Brown for the patch. + See also: https://github.com/rsyslog/rsyslog/pull/13 +- parser global parameters can now be set using RainerScript global() + Thanks to Nathan Brown for the patch. + See also: https://github.com/rsyslog/rsyslog/pull/23 +- omprog: guard program-to-be-executed against CTL-C + This can frequently happen in debug mode, where rsyslog is terminated + by ctl-c. In any case, SIGINT is not meant to control the child process, + so it should be blocked. +- omprog bugfix: parameter "forceSingleInstance" is NOT mandatory +- add new jsonr property replacer option + Thanks to Nathan Brown for the patch. +- added external plugin interface +- ommongodb: add authentication support (untested) + Thanks to JT for the patch. + See also: https://github.com/rsyslog/rsyslog/pull/17 +- bugfix: json templates are improperly created + Strings miss the terminating NUL character, which obviously can lead + to all sorts of problems. + See also: https://github.com/rsyslog/rsyslog/issues/27 + Thanks to Alain for the analysis and the patch. +- ompgsql bugfix: improper handling of auto-backgrounding mode + If rsyslog was set to auto-background itself (default code behaviour, but + many distros now turn it off for good reason), ompgsql could not + properly connect. This could even lead to a segfault. The core reason + was that a PG session handle was kept open over a fork, something that + is explicitely forbidden in the PG API. + Thanks to Alain for the analysis and the patch. +- bugfix: ommongodb's template parameter was mandatory but should have + been optional + Thanks to Alain for the analysis and the patch. +- bugfix: end of batch processing was not 100% correct. Could lead to + outputs not properly wirting messages. At least omelasticsearch did not + write anything to the database due to this bug. + See: https://github.com/rsyslog/rsyslog/issues/10 + Thanks to Radu Gheorghe for reporting the issue. +--------------------------------------------------------------------------- +Version 8.1.5 [devel] 2014-01-24 +- omprog: ability to execute multiple program instances per action + It can now execute one program instance per worker thread. This is + generally a very good thing the have performance wise. Usually, this + should cause no problems with the invoked program. For that reason, + we have decided to make this the default mode of operation. If not + desired, it can be turned off via the 'forceSingleInstance="on"' + action parameter. + CHANGE OF BEHAVIOUR: previous versions did always execute only one + instance per action, no matter how many workers were active. If + your program has special needs, you need to change your configuration. +- imfile now supports inotify (but must be explicitely turned on) +- imfile no longer has a limit on number of monitored files +- added ProcessInternalMessages global system parameter + This permits to inject rsyslog status messages into *another* main + syslogd or the journal. +- new dependency: liblogging-stdlog (for submitting to external logger) +- bugfix: imuxsock input parameters were not accepted + due to copy&paste error. Thanks to Andy Goldstein for the fix. +--------------------------------------------------------------------------- +Version 8.1.4 [devel] 2014-01-10 +- add exec_template() RainerScript function +- imrelp: support for TCP KEEPALIVE added +- bumped librelp dependency to 1.2.2 to support new KEEPALIVE feature +- Add directives for numerically specifying GIDs/UIDs + The already present directives (FileOwner, FileGroup, DirOwner, + DirGroup) translate names to numerical IDs, which depends on the user + information being available during rsyslog's startup. This can fail if + the information is obtained over a network or from a service such as + SSSD. The new directives provide a way to specify the numerical IDs + directly and bypass the lookup. + Thanks to Tomas Heinrich for the patch. +- bugfix: action commitTransaction() processing did not properly handle + suspended actions +- bugfix: omelasticsearch fail.es stats counter was improperly maitained +--------------------------------------------------------------------------- +Version 8.1.3 [devel] 2013-12-06 + +THIS VERSION CAN BE CONSIDERED A "NORMAL" DEVEL RELEASE. It's no longer +highly experimental. This assertion is based on real-world feedback. + +- changes to the strgen module interface +- new output module interface for transactional modules +- performance improvements + * reduced number of malloc/frees due to further changes to the + output module interface + * reduced number of malloc/frees during string template processing + We now re-use once allocated string template memory for as long + as the worker thread exists. This saves us from doing new memory + allocs (and their free counterpart) when the next message is + processed. The drawback is that the cache always is the size of + the so-far largest message processed. This is not considered a + problem, as in any case a single messages' memory footprint should + be far lower than that of a whole set of messages (especially on + busy servers). + * used variable qualifiers (const, __restrict__) to hopefully help + the compiler generate somewhat faster code +- failed action detection more precisely for a number of actions + If an action uses string parameter passing but is non-transactional + it can be executed immediately, giving a quicker indicatio of + action failure. +- bugfix: limiting queue disk space did not work properly + * queue.maxdiskspace actually initializes queue.maxfilesize + * total size of queue files was not checked against + queue.maxdiskspace for disk assisted queues. + Thanks to Karol Jurak for the patch. +--------------------------------------------------------------------------- +Version 8.1.2 [experimental] 2013-11-28 +- support for liblognorm1 added - results in performance improvements + Thanks to Pavel Levshin for his work in this regard. +- support for jemalloc added via --enable-jemalloc + Thanks to Pavel Levshin for suggesting jemalloc + Note that build system is experimental at this stage. +- queue defaults have changed + * high water mark is now dynamically 90% of queue size + * low water makr is now dynamically 70% of queue size + * queue.discardMark is now dynamically 98% of queue size + * queue.workerThreadMinimumMessage set to queue.size / num workers + For queues with very low queue.maxSize (< 100), "emergency" defaults + will be used. +- bugfix: disk queues created files in wrong working directory + if the $WorkDirectory was changed multiple times, all queues only + used the last value set. +- bugfix: legacy directive $ActionQueueWorkerThreads was not honored +- bugfix: mmrfc5424addhmac: "key" parameter was not properly processed +--------------------------------------------------------------------------- +Version 8.1.1 [experimental] 2013-11-19 +- bugfix: STOP/discard(~) was mostly NOT honored + This lead to execution of config code that was not meant to be executed. +- bugfix: memory leak on worker thread termination +- bugfix: potential segfault in omfile under heavy load + Thanks to Pavel Levshin for alerting us. +- bugfix: mmsequence: instance mode did not work + Thanks to Pavel Levshin for the patch +- bugfix: segfault on startup when certain script constructs are used + e.g. "if not $msg ..." +- omhiredis: now supports v8 output module interface and works again + Thanks to Pavel Levshin for the patch +- mmaudit: now supports v8 output module interface and work again +- bugfix: potential abort on startup in debug mode + This depends on template type being used. The root cause was a + non-necessary debug output, which were at the wrong spot (leftover from + initial testing). + Thanks to Pavel Levshin for alerting us and providing a patch + proposal. +--------------------------------------------------------------------------- +Version 8.1.0 [experimental] 2013-11-15 +- rewritten core engine for higher performance and new features + In detail: + * completely rewritten rule execution engine + * completely changed output module interface + * remodelled output module interface + * enabled important output modules to support full concurrent + operation + The core engine has been considerably changed and must be considered + experimental at this stage. Note that it does not yet include all + features planned for v8, but is close to this goal. In theory, the + engine should perform much better, especially on complex configurations + and busy servers. Most importantly, actions instances can now be called + concurrently from worker threads and many important output modules + support multiple concurrent action instances natively. +- module omruleset is no longer enabled by default. + Note that it has been deprecated in v7 and been replaced by the "call" + statement. Also, it can still be build without problems, the option must + just explicitely be given. +--------------------------------------------------------------------------- +Version 7.6.6 [v7.6-stable] 2014-09-30 +- bugfix: potential abort when a message with PRI > 191 was processed + if the "pri-text" property was used in active templates, this could + be abused to a remote denial of service from permitted senders + see also: CVE-2014-3634 +- bugfix: potential segfault on startup on 64 bit systems + This happened immediately on startup during config processing. Once + rsyslog got past this stage, it could not happen. +- bugfix: build problems on SuSe Linux + Thanks Andreas Stieger for the patch +--------------------------------------------------------------------------- +Version 7.6.5 [v7.6-stable] 2014-09-17 +- bugfix: in 7.6.4, pri-based filters did not work correctly + messages were distributed to the wrong bins. +- bugfix: build problems on systems without atomic instructons + e.g. RHEL 5; backport from v8 +--------------------------------------------------------------------------- +Version 7.6.4 [v7.6-stable] 2014-09-12 +- add --enable-generate-man-pages configure switch (default: enabled) + This forces generation of man pages, even if cached ones exists. This + "fixes" a typical release tarball nit. While it is hackish, the + benefit is clear given the history of failed tarball releases since + we changed the cached man page handling. It was just too easy to get + that wrong. +- removed obsolete --disable-fsstnd configure option + Thanks to Thomas D. for alerting us. + Closes: https://github.com/rsyslog/rsyslog/issues/72 +- permits to build against json-c 0.12 + Unfortunately, json-c had an ABI breakage, so this is necessary. Note + that versions prior to 0.12 had security issues (CVE-2013-6370, + CVE-2013-6371) and so it is desirable to link against the new version. + Thanks to Thomas D. for the patch. Note that at least some distros + have fixed the security issue in older versions of json-c, so this + seems to apply mostly when building from sources. +- new omfile default module parameters + * filecreatemode + * fileowner + * fileownernum + * filegroup + * filegroupnum + * dirowner + * dirownernum + * dirgroup + * dirgroupnum + Thanks to Karol Jurak for the patch. +- bugfix: memory leak in TCP TLS mode +- bugfix: imfile: if a state file for a different file name was set, + that different file (name) was monitored instead of the configured + one. Now, the state file is deleted and the correct file monitored. + closes: https://github.com/rsyslog/rsyslog/issues/103 +- bugfix: using UUID property could cause segfault +- bugfix: mmutf8fix did not detect two invalid sequences + Thanks to Axel Rau for the patch. +- bugfix: file descriptor leak with Guardtime signatures + When a .gtstate file is opened it is never closed. This is especially + bad when dynafiles frequently get evicted from dynafile cache and be + re-opened again. +- bugfix: busy loop in tcp listener when running out of file descriptors + Thanks to Susant Sahani for the patch. +- bugfix: mishandling of input modules not supporting new input instances + If they did not support this, accidently the output module part of the + module union was written, leading to unpredictable results. Note: all + core modules do support this interface, but some contributed or very + old ones do not. +- bugfix: double-free when ruleset() parser parameters were used + While unlikely, this could cause stability issues even after the + config phase. +- bugfix: output modules with parameters with multiple passing modes + could caused strange behaviour including aborts + This was due to the fact that the action module only preserved and + processed the last set passing mode. Note that this was not a problem + for the plugins provided by the rsyslog git: none of them uses different + passing modes. + Thanks to Tomas Heinrich for providing a very detailled bug report. +- various fixes after coverty scan + These do not address issues seen in practice but those seen by the tool. + Some of them may affect practical deployments. + Thanks to Tomas Heinrich for the patches. +- bugfix imuxsock: "Last message repeated..." was not emitted at shutdown + The "Last message repeated..." notice didn't get printed if rsyslog was + shut down before the repetition was broken. + Thanks to Tomas Heinrich for the patch. +- bugfix: make dist failed when GUARDTIME or LIBGCRYPT feature was disabled +- bugfix: mmjsonparse did not build with json-c < 0.10 + This was a regression introduced some time in the past in order to + support API changes in json-c. Now we check for the version and use + proper code. +- bugfix: mmanon did not properly anonymize IP addresses starting with '9' + Thanks to defa-at-so36.net for reporting this problem. + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=529 +--------------------------------------------------------------------------- +Version 7.6.3 [v7.6-stable] 2014-03-27 +- add capability to override GnuTLS path in build process + Thanks to Clayton Shotwell for the patch +- support for librelp 1.2.5 + Support new return states of librelp 1.2.5 to emit better error messages + For obvious reasons, librelp 1.2.5 is now required. +- bugfix: ompipe used invalid default template + This is a regression from an old change (didn't track it down precisely, + but over a year ago). It used the Forwarding template instead of + the file template (so we have a full syslog header). This fix corrects + it back to previous behaviour, but new scripts that used the wrong + format may now need to have the RSYSLOG_ForwardingFormat template + explicitely be applied. + closes: https://github.com/rsyslog/rsyslog/issues/50 +- bugfix: ompipe did emit many suspension messages for /dev/xconsole + (hopefully now) closes: https://github.com/rsyslog/rsyslog/issues/35 + When it was present, but nobody reading from it. The problem + is the way the rsyslog v7 engine tries to resolve failures in outputs. + It does some retries, and along those lines some state information gets + lost and it is close to impossible to retain it. However, the actual + root problem is that ompipe does not reliably detect if it is able to + recover. The problem here is that it actually does not know this + before it does an actual write. These two things together mess up the + logic that suppresses invalid resumption/suspension messages + (actually, the plugin switches state really that often). + Nevertheless, the prime problem with /dev/xconsole (and probably + most other pipes as well) is that it gets full. So I have now added + code that checks, during resume processing, if the pipe is writable. + If it is not, resume is deferred. That should address the case. +--------------------------------------------------------------------------- +Version 7.6.2 [v7.6-stable] 2014-03-17 +- support for librelp 1.2.4 + This was necessary due to the problems with librelp 1.2.3 API stability. + We now use the new native 1.2.4 APIs to learn about the state of + librelp's TLS support. + For obvious reasons, librelp 1.2.4 is now required. +--------------------------------------------------------------------------- +Version 7.6.1 [v7.6-stable] 2014-03-13 +- added "action.reportSuspension" action parameter + This now permits to control handling on a per-action basis rather to + the previous "global setting only". +- "action resumed" message now also specifies module type + which makes troubleshooting a bit easier. Note that we cannot output all + the config details (like destination etc) as this would require much more + elaborate code changes, which we at least do not like to do in the + stable version. +- better and more consistent action naming, action queues now always + contain the word "queue" after the action name +- add support for "tls-less" librelp + we now require librelp 1.2.3, as we need the new error code definition + See also: https://github.com/rsyslog/librelp/issues/1 +- build system improvements + * autoconf subdir option + * support for newer json-c packages + Thanks to Michael Biebl for the patches. +- imjournal enhancements: + * log entries with empty message field are no longer ignored + * invalid facility and severity values are replaced by defaults + * new config parameters to set default facility and severity + Thanks to Tomas Heinrich for implementing this +- bugfix: ompipe did resume itself even when it was still in error + See: https://github.com/rsyslog/rsyslog/issues/35 + Thanks to github user schplat for reporting +- bugfix: "action xxx suspended" did report incorrect error code +- bugfix: ommongodb's template parameter was mandatory but should have + been optional + Thanks to Alain for the analysis and the patch. +- bugfix: only partial doc was put into distribution tarball + Thanks to Michael Biebl for alerting us. + see also: https://github.com/rsyslog/rsyslog/issues/31 +- bugfix: async ruleset did process already-deleted messages + Thanks to John Novotny for the patch. +--------------------------------------------------------------------------- +Version 7.6.0 [v7.6-stable] 2014-02-12 +This starts a new stable branch based on 7.5.8 plus the following changes: +- bugfix: imuxsock input parameters were not accepted + due to copy&paste error. Thanks to Andy Goldstein for the fix. +- added ProcessInternalMessages global system parameter + This permits to inject rsyslog status messages into *another* main + syslogd or the journal. +- new dependency: liblogging-stdlog (for submitting to external logger) +- bugfix: json templates are improperly created + Strings miss the terminating NUL character, which obviously can lead + to all sorts of problems. + See also: https://github.com/rsyslog/rsyslog/issues/27 + Thanks to Alain for the analysis and the patch. +- ompgsql bugfix: improper handling of auto-backgrounding mode + If rsyslog was set to auto-background itself (default code behaviour, but + many distros now turn it off for good reason), ompgsql could not + properly connect. This could even lead to a segfault. The core reason + was that a PG session handle was kept open over a fork, something that + is explicitely forbidden in the PG API. + Thanks to Alain for the analysis and the patch. +--------------------------------------------------------------------------- +Version 7.5.8 [v7-release candidate] 2014-01-09 +- add exec_template() RainerScript function +- add debug.onShutdown and debug.logFile global paramters + These enebale the new "debug on shutdown" mode, which can be used to + track hard to find problems that occur during system shutdown. +- Add directives for numerically specifying GIDs/UIDs + The already present directives (FileOwner, FileGroup, DirOwner, + DirGroup) translate names to numerical IDs, which depends on the user + information being available during rsyslog's startup. This can fail if + the information is obtained over a network or from a service such as + SSSD. The new directives provide a way to specify the numerical IDs + directly and bypass the lookup. + Thanks to Tomas Heinrich for the patch. +- actions now report if they suspend and resume themselves + this is by default on and controllable by the action.reportSuspension + global parameter +- bugfix: omelasticsearch fail.es stats counter was improperly maintained +- bugfix: mmrfc5424addhmac: "key" parameter was not properly processed +- add new impstats action counters: + * suspended + * suspended.duration + * resumed +--------------------------------------------------------------------------- +Version 7.5.7 [v7-devel] 2013-11-25 +- queue defaults have changed + * high water mark is now dynamically 90% of queue size + * low water makr is now dynamically 70% of queue size + * queue.discardMark is now dynamically 98% of queue size + * queue.workerThreadMinimumMessage set to queue.size / num workers + For queues with very low queue.maxSize (< 100), "emergency" defaults + will be used. +- worker thread pool handling has been improved + Among others, permits pool to actually shrink (was quite hard with + previous implementation. This will also improve performance and/or + lower system overhead on busy systems. + Thanks to Pavel Levshin for the enhancement. +- bugfix: mmpstrucdata generated inaccessible properties +- bugfix: RainerScript optimizer did not optimize PRI filters + things like "if $syslogfacility-text == "local3"" were not converted + to PRIFILT. This was a regression introduced in 7.5.6. +- bugfix: legacy directive $ActionQueueWorkerThreads was not honored +- bugfix: segfault on startup when certain script constructs are used + e.g. "if not $msg ..." +- bugfix: ommysql lost configfile/section parameters after first close + This means that when a connection was broken, it was probably + re-instantiated with different parameters than configured. +- bugfix: regression in template processing with subtrees in templates + Thanks to Pavel Levshin for the fix +- bugfix: regular worker threads are not properly (re)started if DA + mode is active. + This occurs only under rare conditions, but definitely is a bug that + needed to be addressed. It probably is present since version 4. + Note that this patch has not been applied to v7.4-stable, as it + is very unlikely to happen and the fix itself has some regression + potential (the fix looks very solid, but it addresses a core component). + Thanks to Pavel Levshin for the fix +- now emit warning message if om with msg passing mode uses action queue + These can modify the message, and this causes races. +- bugfix: $SystemLogUseSysTimeStamp/$SystemLogUsePIDFromSystem did not work + Thanks to Tomas Heinrich for the patch. +--------------------------------------------------------------------------- +Version 7.5.6 [devel] 2013-10-29 +- impstats: add capability to bind to a ruleset +- improved performance of RainerScript variable access + by refactoring the whole body of variable handling code. This also + solves some of the anomalies experienced in some versions of rsyslog. + All variable types are now handled in unified code, including + access via templates. +- RainerScript: make use of 64 bit for numbers where available + Thanks to Pavel Levshin for enhancement. +- slight performance optimization if GCC is used + We give branch prediction hints for the frequent RETiRet macro which is + used for error handling. Some slight performance gain is to be expected + from that. +- removed global variable support + The original idea was not well thought out and global variables, as + implemented, worked far different from what anybody would expect. As + such, we consider the current approach as an experiment that did not + work out and opt to removing it, clearing the way for a better future + solution. Note: global vars were introduced in 7.5.3 on Sept, 11th 2013. +- new module mmsequence, primarily used for action load balancing + Thanks to Pavel Levshin for contributing this module. +- bugfix: unset statement always worked on message var, even if local + var was given +- imudp: support for binding to ruleset added +- bugfix: segfault if variable was assigned to non-container subtree + Thanks to Pavel Levshin for the fix +- bugfix: imuxsock did not suport addtl sockets if syssock was disabled + Thanks to Pavel Levshin for the fix +- bugfix: running imupd on multiple threads lead to segfault if recvmmsg + is available +- bugfix: imudp when using recvmmsg could report wrong sender IP +- bugfix: segfault if re_extract() function was used and no match found +- bugfix: omelasticsearch did not compile on platforms without atomic + instructions +- bugfix: potential misadressing on startup if property-filter was used + This could happen if the property name was longer than 127 chars, a case + that would not happen in practice. +- bugfix: invalid property filter was not properly disabled in ruleset + Note: the cosmetic memory leak introduced with that patch in 7.4.5 is + now also fixed. +- imported bugfixes from 7.4.6 stable release +--------------------------------------------------------------------------- +Version 7.5.5 [devel] 2013-10-16 +- imfile: permit to monitor an unlimited number of files +- imptcp: add "defaultTZ" input parameter +- imudp: support for multiple receiver threads added +- imudp: add "dfltTZ" input config parameter +- bugfix: memory leak in mmnormalize +- bugfix: mmutf8fix did not properly handle invalid UTF-8 at END of message + if the very last character sequence was too long, this was not detected + Thanks to Risto Vaarandi for reporting this problem. +- mmanon: removed the check for specific "terminator characters" after + last octet. As it turned out, this didn't work in practice as there + was an enormous set of potential terminator chars -- so removing + them was the best thing to do. Note that this may change behaviour of + existing installations. Yet, we still consider this an important + bugfix, that should be applied to the stable branch. + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=477 + Thanks to Muri Cicanor for initiating the discussion +- now requires libestr 0.1.7 as early versions had a nasty bug in + string comparisons +- bugfix: mmanon did not detect all IP addresses in rewrite mode + The problem occured if two IPs were close to each other and the first one + was shrunk. + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=485 + Thanks to micah-at-riseup.net for reporting this bug +- bugfix: mmanon sometimes used invalid replacement char in simple mode + depending on configuration sequence, the replacement character was set + to 's' instead of the correct value. Most importantly, it was set to + 's' if simple mode was selected and no replacement char set. + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=484 + Thanks to micah-at-riseup.net for reporting this bug +- bugfix: memory leak in mmnormalize +- bugfix: array-based ==/!= comparisions lead to invalid results + This was a regression introduced in 7.3.5 bei the PRI optimizer +--------------------------------------------------------------------------- +Version 7.5.4 [devel] 2013-10-07 +- mmpstrucdata: new module to parse RFC5424 structured data into json + message properties +- change main/ruleset queue defaults to be more enterprise-like + new defaults are queue.size 100,000 max workers 2, worker + activation after 40,000 msgs are queued, batch size 256. These settings + are much more useful for enterprises and will not hurt low-end systems + that much. This is part of our re-focus on enterprise needs. +- omfwd: new action parameter "maxErrorMessages" added +- omfile: new module parameters to set action defaults added + * dirCreateMode + * fileCreateMode +- mmutf8fix: new module to fix invalid UTF-8 sequences +- imuxsock: handle unlimited number of additional listen sockets +- doc: improve usability by linking to relevant web ressources + The idea is to enable users to quickly find additional information, + samples, HOWTOs and the like on the main site. + At the same time, (very) slightly remove memory footprint when + few listeners are monitored. +- bugfix: omfwd parameter streamdrivermmode was not properly handled + it was always overwritten by whatever value was set via the + legacy directive $ActionSendStreamDriverMode +- imtcp: add streamdriver.name module parameter + permits overriding the system default stream driver (gtls, ptcp) +- bugfix: build system: libgcrypt.h needed even if libgrcypt was disabled + Thanks to Jonny Törnbom for reporting this problem +- imported bugfixes from 7.4.4 +--------------------------------------------------------------------------- +Version 7.5.3 [devel] 2013-09-11 +- imfile: support for escaping LF characters added + embedded LF in syslog messages cause a lot of trouble. imfile now has + the capability to escape them to "#012" (just like the regular control + character escape option). This requires new-style input statements to be + used. If legacy configuration statements are used, LF escaping is always + turned off to preserve compatibility. + NOTE: if input() statements were already used, there is a CHANGE OF + BEHAVIOUR: starting with this version, escaping is enabled by + default. So if you do not want it, you need to add + escapeLF="off" + to the input statement. Given the trouble LFs cause and the fact + that the majority of installations still use legacy config, we + considered this behaviour change acceptable and useful. + see also: http://blog.gerhards.net/2013/09/imfile-multi-line-messages.html +- add support for global and local variables +- bugfix: queue file size was not correctly processed + this could lead to using one queue file per message for sizes >2GiB + Thanks to Tomas Heinrich for the patch. +- add main_queue() configuration object to configure main message queue +- bugfix: stream compression in imptcp caused timestamp to be corrupted +- imudp: add ability to specify SO_RCVBUF size (rcvbufSize parameter) +- imudp: use inputname for statistics, if configured +- impstats: add process resource usage counters [via getrusage()] +- impstats: add paramter "resetCounters" to report delta values + possible for most, but not all, counters. See doc for details. +- librelp 1.2.0 is now required +- make use of new librelp generic error reporting facility + This leads to more error messages being passed to the user and + thus simplified troubleshooting. +- bugfix: very small memory leak in imrelp + more or less cosmetic, a single memory block was not freed, but this + only happens immediately before termination (when the OS automatically + frees all memory). Still an annoyance e.g. in valgrind. +- fix compile problem in debug build +- imported fixes from 7.4.4 +--------------------------------------------------------------------------- +Version 7.5.2 [devel] 2013-07-04 +- librelp 1.1.4 is now required + We use API extensions for better error reporting and higher performance. +- omrelp: use transactional mode to make imrelp emit bulk sends +- omrelp: add "windowSize" parameter to set custom RELP window size +- bugfix: double-free in omelasticsearch + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=461 + a security advisory for this bug is available at: + http://www.lsexperts.de/advisories/lse-2013-07-03.txt + CVE: CVE-2013-4758 + PLEASE NOTE: This issue only existed if omelasticsearch was used + in a non-default configuration, where the "errorfile" parameter + was specified. Without that parameter set, the bug could not + be triggered. + Thanks to Markus Vervier and Marius Ionescu for providing a detailled + bug report. Special thanks to Markus for coordinating his security + advisory with us. +- doc: fixed various typos + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=391 + Thanks to Georgi Georgiev for the patch. +--------------------------------------------------------------------------- +Version 7.5.1 [devel] 2013-06-26 +- librelp 1.1.3 is required - older versions can lead to a segfault +- add mmfields, which among others supports easy parsing of CEF messages +- omrelp: + * new parameter "compression.prioritystring" to control encryption + parameters used by GnuTLS +- imrelp: + * new parameter "compression.dhbits" to control the number of + bits being used for Diffie-Hellman key generation + * new parameter "compression.prioritystring" to control encryption + parameters used by GnuTLS + * support for impstats added + * support for setting permitted peers (client authentication) added + * bugfix: potential segfault at startup on invalid config parameters +- imjournal: imported patches from 7.4.1 +- omprog: add support for command line parameters +- added experimental TCP stream compression (imptcp only, currently) +- added BSD-specific syslog facilities + * "console" + * "bsd_security" - this is called "security" under BSD, but that name + was unfortunately already taken by some standard facility. So I + did the (hopefully) second-best thing and renamed it a little. +- imported fixes from 7.4.2 (especially build problems on FreeBSD) +- bugfix: imptcp did not properly initialize compression status variable + could lead to segfault if stream:always compression mode was selected +--------------------------------------------------------------------------- +Version 7.5.0 [devel] 2013-06-11 +- imrelp: implement "ruleset" module parameter +- imrelp/omrelp: add TLS & compression (zip) support +- omrelp: add "rebindInterval" parameter +- add -S command line option to specify IP address to use for RELP client + connections + Thanks to Axel Rau for the patch. +--------------------------------------------------------------------------- +Version 7.4.11 [v7.4-stable] *never released* +- imjournal enhancements: + * log entries with empty message field are no longer ignored + * invalid facility and severity values are replaced by defaults + * new config parameters to set default facility and severity + Thanks to Tomas Heinrich for implementing this +--------------------------------------------------------------------------- +Version 7.4.10 [v7.4-stable] 2014-02-12 +- bugfix: json templates are improperly created + Strings miss the terminating NUL character, which obviously can lead + to all sorts of problems. + See also: https://github.com/rsyslog/rsyslog/issues/27 + Thanks to Alain for the analysis and the patch. +- ompgsql bugfix: improper handling of auto-backgrounding mode + If rsyslog was set to auto-background itself (default code behaviour, but + many distros now turn it off for good reason), ompgsql could not + properly connect. This could even lead to a segfault. The core reason + was that a PG session handle was kept open over a fork, something that + is explicitely forbidden in the PG API. + Thanks to Alain for the analysis and the patch. +--------------------------------------------------------------------------- +Version 7.4.9 [v7.4-stable] 2014-01-22 +- added ProcessInternalMessages global system parameter + This permits to inject rsyslog status messages into *another* main + syslogd or the journal. +- new dependency: liblogging-stdlog (for submitting to external logger) +- bugfix: imuxsock input parameters were not accepted + due to copy&paste error. Thanks to Andy Goldstein for the fix. +- bugfix: potential double-free in RainerScript equal comparison + happens if the left-hand operand is JSON object and the right-hand + operand is a non-string that does not convert to a number (for + example, it can be another JSON object, probably the only case that + could happen in practice). This is very unlikely to be triggered. +- bugfix: some RainerScript Json(Variable)/string comparisons were wrong +--------------------------------------------------------------------------- +Version 7.4.8 [v7.4-stable] 2014-01-08 +- rsgtutil provides better error messages on unfinished signature blocks +- bugfix: guard against control characters in internal (error) messages + Thanks to Ahto Truu for alerting us. +- bugfix: immark did emit messages under kern.=info instead of syslog.=info + Note that his can potentially break exisiting configurations that + rely on immark sending as kern.=info. Unfortunately, we cannot leave + this unfixed as we never should emit messages under the kern facility. +--------------------------------------------------------------------------- +Version 7.4.7 [v7.4-stable] 2013-12-10 +- bugfix: limiting queue disk space did not work properly + * queue.maxdiskspace actually initializes queue.maxfilesize + * total size of queue files was not checked against + queue.maxdiskspace for disk assisted queues. + Thanks to Karol Jurak for the patch. +- bugfix: linux kernel-like ratelimiter did not work properly with all + inputs (for example, it did not work with imdup). The reason was that + the PRI value was used, but that needed parsing of the message, which + was done too late. +- bugfix: disk queues created files in wrong working directory + if the $WorkDirectory was changed multiple times, all queues only + used the last value set. +- bugfix: legacy directive $ActionQueueWorkerThreads was not honored +- bugfix: segfault on startup when certain script constructs are used + e.g. "if not $msg ..." +- bugfix: imuxsock: UseSysTimeStamp config parameter did not work correctly + Thanks to Tomas Heinrich for alerting us and provinding a solution + suggestion. +- bugfix: $SystemLogUseSysTimeStamp/$SystemLogUsePIDFromSystem did not work + Thanks to Tomas Heinrich for the patch. +- improved checking of queue config parameters on startup +- bugfix: call to ruleset with async queue did not use the queue + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=443 +- bugfix: if imtcp is loaded and no listeners are configured (which is + uncommon), rsyslog crashes during shutdown. +--------------------------------------------------------------------------- +Version 7.4.6 [v7.4-stable] 2013-10-31 +- bugfix: potential abort during HUP + This could happen when one of imklog, imzmq3, imkmsg, impstats, + imjournal, or imuxsock were under heavy load during a HUP. + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=489 + Thanks to Guy Rozendorn for reporting the problem and Peval Levhshin for + his analysis. +- bugfix: imtcp flowControl parameter incorrectly defaulted to "off" + This could cause message loss on systems under heavy load and was + a change-of-behaviour to previous version. This is a regression + most probably introduced in 5.9.0 (but did not try hard to find the + exact point of its introduction). +- now requires libestr 0.1.9 as earlier versions lead to problems with + number handling in RainerScript +- bugfix: memory leak in strlen() RainerScript function + Thanks to Gregoire Seux for reportig this bug. + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=486 +- bugfix: buffer overrun if re_extract function was called for submatch 50 + Thanks to Pavel Levshin for reporting the problem and its location. +- bugfix: memleak in re_extract() function + Thanks to Pavel Levshin for reporting this problem. +- bugfix: potential abort in RainerScript optimizer + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=488 + Thanks to Thomas Doll for reporting the problem and Pavel Levshin for + fixing it. +- bugfix: memory leak in omhiredis + Thanks to Pavel Levshin for the fix +- bugfix: segfault if variable was assigned to non-container subtree + Thanks to Pavel Levshin for the fix +--------------------------------------------------------------------------- +Version 7.4.5 [v7.4-stable] 2013-10-22 +- mmanon: removed the check for specific "terminator characters" after + last octet. As it turned out, this didn't work in practice as there + was an enormous set of potential terminator chars -- so removing + them was the best thing to do. Note that this may change behaviour of + existing installations. Yet, we still consider this an important + bugfix, that should be applied to the stable branch. + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=477 + Thanks to Muri Cicanor for initiating the discussion +- now requires libestr 0.1.8 as early versions had a nasty bug in + string comparisons +- omelasticsearch: add failed.httprequests stats counter +- bugfix: invalid property filter was not properly disabled in ruleset + Note that this bugfix introduces a very slight memory leak, which is + cosmetic, as it just holds data until termination that is no longer + needed. It is just the part of the config that was invalid. We will + "fix" this "issue" in the devel version first, as the fix is a bit + too intrusive to do without hard need in the stable version. +- bugfix: segfault if re_extract() function was used and no match found +- bugfix: potential misadressing on startup if property-filter was used + This could happen if the property name was longer than 127 chars, a case + that would not happen in practice. +- bugfix: omelasticsearch: correct failed.http stats counter +- bugfix: omelasticsearch: did not correctly initialize stats counters +- bugfix: omelasticsearch: failed.es counter was only maintained in bulk mode + This usually did not lead to any problems, because they are in static + memory, which is initialized to zero by the OS when the plugin is + loaded. But it may cause problems especially on systems that do not + support atomic instructions - in this case the associated mutexes also + did not get properly initialized. +- bugfix: mmanon did not detect all IP addresses in rewrite mode + The problem occured if two IPs were close to each other and the first one + was shrunk. + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=485 + Thanks to micah-at-riseup.net for reporting this bug +- bugfix: mmanon sometimes used invalid replacement char in simple mode + depending on configuration sequence, the replacement character was set + to 's' instead of the correct value. Most importantly, it was set to + 's' if simple mode was selected and no replacement char set. + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=484 + Thanks to micah-at-riseup.net for reporting this bug +- bugfix: memory leak in mmnormalize +- bugfix: array-based ==/!= comparisions lead to invalid results + This was a regression introduced in 7.3.5 bei the PRI optimizer +- bugfix: omprog blocked signals to executed programs + The made it impossible to send signals to programs executed via + omprog. + Thanks to Risto Vaarandi for the analysis and a patch. +- bugfix: doc: imuxsock legacy param $SystemLogSocketParseTrusted was + misspelled + Thanks to David Lang for alerting us +- bugfix: imfile "facility" input parameter improperly handled + caused facility not to be set, and severity to be overwritten with + the facility value. + Thanks to forum user dmunny for reporting this bug. +- bugfix: small memory leak in imfile when $ResetConfigVariables was used + Thanks to Grégory Nuyttens for reporting this bug and providig a fix +- bugfix: segfault on startup if TLS was used but no CA cert set +- bugfix: segfault on startup if TCP TLS was used but no cert or key set +- bugfix: some more build problems with newer json-c versions + Thanks to Michael Biebl for mentioning the problem. +- bugfix: build system: libgcrypt.h needed even if libgrcypt was disabled + Thanks to Jonny Törnbom for reporting this problem +--------------------------------------------------------------------------- +Version 7.4.4 [v7.4-stable] 2013-09-03 +- better error messages in GuardTime signature provider + Thanks to Ahto Truu for providing the patch. +- make rsyslog use the new json-c pkgconfig file if available + Thanks to the Gentoo team for the patches. +- bugfix: imfile parameter "persistStateInterval" was unusable + due to a case typo in imfile; work-around was to use legacy config + Thanks to Brandon Murphy for reporting this bug. +- bugfix: TLV16 flag encoding error in signature files from GT provider + This fixes a problem where the TLV16 flag was improperly encoded. + Unfortunately, existing files already have the bug and may not properly + be processed. The fix uses constants from the GuardTime API lib to + prevent such problems in the future. + Thanks to Ahto Truu for providing the patch. +- bugfix: slightly malformed SMTP handling in ommail +- bugfix: segfault in omprog if no template was provided (now dflt is used) +- bugfix: segfault in ompipe if no template was provided (now dflt is used) +- bugfix: segfault in omsnmp if no template was provided (now dflt is used) +- bugfix: some omsnmp optional config params were flagged as mandatory +- bugfix: segfault in omelasticsearch when resuming queued messages + after restarting Elasticsearch + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=464 +- bugfix: imtcp addtlframedelimiter could not be set to zero + Thanks to Chris Norton for alerting us. +- doc bugfix: remove no-longer existing omtemplate from developer doc + was specifically mentioned as a sample for creating new plugins + Thanks to Yannick Brosseau for alerting us of this problem. + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=473 +--------------------------------------------------------------------------- +Version 7.4.3 [v7.4-stable] 2013-07-18 +- bugfix: queue file size was not correctly processed + this could lead to using one queue file per message for sizes >2GiB + Thanks to Tomas Heinrich for the patch. +- bugfix: $QHOUR/$HHOUR were always "00" or "01" + regression some time between v5 and here + Thanks to forum user rjmcinty for reporting this bug +- bugfix: testbench tool chkseq did improperly report invalid file + This happened when permitted duplicate values existed in the very + last lines, right before end-of-file. + Thanks to Radu Gheorghe for reporting this bug. +--------------------------------------------------------------------------- +Version 7.4.3 [v7.4-stable] 2013-07-18 +- bugfix: memory leak if disk queues were used and json data present +- bugfix: CEE/json data was lost during disk queue operation +- bugfix: potential segfault during startup on invalid config + could happen if invalid actions were present, which could lead + to improper handling in optimizer. +- bugfix: 100% CPU utilization when DA queue became full +- bugfix: omlibdbi did not properly close connection on some errors + This happened to errors occuring in Begin/End Transaction entry + points. +- cosmetic bugfix: file name buffer was not freed on disk queue destruction + This was an extremely small one-time per run memleak, so nothing of + concern. However, it bugs under valgrind and similar memory debuggers. +- fix build on FreeBSD + Thanks to Christiano Rolim for the patch +--------------------------------------------------------------------------- +Version 7.4.2 [v7.4-stable] 2013-07-04 +- bugfix: in RFC5425 TLS, multiple wildcards in auth could cause segfault +- bugfix: RainerScript object required parameters were not properly + checked - this clould result to segfaults on startup if parameters + were missing. +- bugfix: double-free in omelasticsearch + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=461 + a security advisory for this bug is available at: + http://www.lsexperts.de/advisories/lse-2013-07-03.txt + CVE: CVE-2013-4758 + PLEASE NOTE: This issue only existed if omelasticsearch was used + in a non-default configuration, where the "errorfile" parameter + was specified. Without that parameter set, the bug could not + be triggered. + Thanks to Markus Vervier and Marius Ionescu for providing a detailled + bug report. Special thanks to Markus for coordinating his security + advisory with us. +- bugfix: omrelp potential segfault at startup on invalid config parameters +- bugfix: small memory leak when $uptime property was used +- bugfix: potential segfault on rsyslog termination in imudp + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=456 +- bugfix: lmsig_gt abort on invalid configuration parameters + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=448 + Thanks to Risto Laanoja for the patch. +- imtcp: fix typo in "listner" parameter, which is "listener" + Currently, both names are accepted. +- solved build problems on FreeBSD + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=457 + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=458 + Thanks to Christiano for reproting and suggesting patches +- solved build problems on CENTOS5 +--------------------------------------------------------------------------- +Version 7.4.1 [v7.4-stable] 2013-06-17 +- imjournal: add ratelimiting capability + The original imjournal code did not support ratelimiting at all. We + now have our own ratelimiter. This can mitigate against journal + database corruption, when the journal re-sends old data. This is a + current bug in systemd journal, but we won't outrule this to happen + in the future again. So it is better to have a safeguard in place. + By default, we permit 20,000 messages witin 10 minutes. This may + be a bit restrictive, but given the risk potential it seems reasonable. + Users requiring larger traffic flows can always adjust the value. +- bugfix: potential loop in rate limiting + if the message that tells about rate-limiting gets rate-limited itself, + it will potentially create and endless loop +- bugfix: potential segfault in imjournal if journal DB is corrupted +- bugfix: prevent a segfault in imjournal if state file is not defined +- bugfix imzmq3: potential segfault on startup + if no problem happend at startup, everything went fine + Thanks to Hongfei Cheng and Brian Knox for the patch +--------------------------------------------------------------------------- +Version 7.4.0 [v7.4-stable] 2013-06-06 +This starts a new stable branch based on 7.3.15 plus the following changes: +- add --enable-cached-man-pages ./configure option + permits to build rsyslog on a system where rst2man is not installed. In + that case, cached versions of the man pages are used (they were built + during "make dist", so they should be current for the version in + question. +- doc bugfix: ReadMode wrong in imfile doc, two values were swapped + Thanks to jokajak@gmail.com for mentioning this + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=450 +- imjournal: no longer do periodic wakeup +- bugfix: potential hang *in debug mode* on rsyslogd termination + This ONLY affected rsyslogd if it were running with debug output + enabled. +- bugfix: $template statement with multiple spaces lead to invalid tpl name + If multiple spaces were used in front of the template name, all but one + of them became actually part of the template name. So + $template a,"..." would be name " a", and as such "a" was not + available, e.g. in + *.* /var/log/file;a + This is a legacy config problem. As it was unreported for many years, + no backport of the fix to old versions will happen. + This is a long-standing bug that was only recently reported by forum + user mc-sim. + Reference: http://kb.monitorware.com/post23448.html +- 0mq fixes; credits to Hongfei Cheng and Brian Knox +--------------------------------------------------------------------------- +Version 7.3.15 [beta] 2013-05-15 +- bugfix: problem in build system (especially when cross-compiling) + Thanks to Tomas Heinrich and winfried_mb2@xmsnet.nl for the patch. + closes: http://bugzilla.adiscon.com/show_bug.cgi?id=445 +- bugfix: imjournal had problem with systemd journal API change +- imjournal: now obtain and include PID +- bugfix: .logsig files had tlv16 indicator bit at wrong offset +- bugfix: omrelp legacy config parameters set a timeout of zero + which lead the legacy config to be unusable. +- bugfix: segfault on startup if a disk queue was configure without file + name + Now this triggers an error message and the queue is changed to + linkedList type. +- bugfix: invalid addressing in string class (recent regression) +--------------------------------------------------------------------------- Version 7.3.14 [beta] 2013-05-06 - bugfix: some man pages were not properly installed either rscryutil or rsgtutil man was installed, but not both @@ -1133,8 +2313,6 @@ Version 6.2.0 [v6-stable], 2012-01-09 - bugfix: omfile returns fatal error code for things that go really wrong previously, RS_RET_RESUME was returned, which lead to a loop inside the rule engine as omfile could not really recover. -- bugfix: rsyslogd -v always said 64 atomics were not present - thanks to mono_matsuko for the patch - bugfix: potential abort after reading invalid X.509 certificate closes: http://bugzilla.adiscon.com/show_bug.cgi?id=290 Thanks to Tomas Heinrich for the patch @@ -1363,6 +2541,9 @@ expected that interfaces, even new ones, break during the initial [ported from v4] --------------------------------------------------------------------------- Version 5.10.2 [V5-STABLE], 201?-??-?? +- bugfix: queue file size was not correctly processed + this could lead to using one queue file per message for sizes >2GiB + Thanks to Tomas Heinrich for the patch. - updated systemd files to match current systemd source - bugfix: spurios error messages from imuxsock about (non-error) EAGAIN Thanks to Marius Tomaschewski for the patch. @@ -3394,6 +4575,8 @@ Version 3.22.4 [v3-stable] (rgerhards), 2010-??-?? closes: http://bugzilla.adiscon.com/show_bug.cgi?id=271 - improved some code based on clang static analyzer results - bugfix: potential misadressing in property replacer +- bugfix: improper handling of invalid PRI values + references: CVE-2014-3634 --------------------------------------------------------------------------- Version 3.22.3 [v3-stable] (rgerhards), 2010-11-24 - bugfix(important): problem in TLS handling could cause rsyslog to loop diff --git a/Makefile.am b/Makefile.am index ed3b54b..103aa6f 100644 --- a/Makefile.am +++ b/Makefile.am @@ -58,6 +58,7 @@ CLEANFILES = \ endif EXTRA_DIST = \ + README.md \ platform/README \ platform/freebsd/rsyslogd \ platform/slackware/rc.rsyslogd \ @@ -71,7 +72,9 @@ EXTRA_DIST = \ contrib/gnutls/key.pem \ rsyslog.service.in -SUBDIRS = doc compat runtime grammar . plugins/immark plugins/imuxsock plugins/imtcp plugins/imudp plugins/omtesting +SUBDIRS = compat runtime grammar . plugins/immark plugins/imuxsock plugins/imtcp plugins/imudp plugins/omtesting +# external plugin driver is always enabled (core component) +SUBDIRS += plugins/mmexternal if ENABLE_RSYSLOGD SUBDIRS += tools @@ -129,6 +132,10 @@ if ENABLE_PMCISCONAMES SUBDIRS += plugins/pmcisconames endif +if ENABLE_PMCISCOIOS +SUBDIRS += plugins/pmciscoios +endif + if ENABLE_PMAIXFORWARDEDFROM SUBDIRS += plugins/pmaixforwardedfrom endif @@ -241,6 +248,30 @@ if ENABLE_MMANON SUBDIRS += plugins/mmanon endif +if ENABLE_MMUTF8FIX +SUBDIRS += plugins/mmutf8fix +endif + +if ENABLE_MMCOUNT +SUBDIRS += plugins/mmcount +endif + +if ENABLE_MMSEQUENCE +SUBDIRS += plugins/mmsequence +endif + +if ENABLE_MMFIELDS +SUBDIRS += plugins/mmfields +endif + +if ENABLE_MMPSTRUCDATA +SUBDIRS += plugins/mmpstrucdata +endif + +if ENABLE_MMRFC5424ADDHMAC +SUBDIRS += plugins/mmrfc5424addhmac +endif + if ENABLE_ORACLE SUBDIRS += plugins/omoracle endif @@ -260,8 +291,7 @@ SUBDIRS += tests # temporarily be removed below. The intent behind forcing everthing to compile # in a make distcheck is so that we detect code that accidently was not updated # when some global update happened. -DISTCHECK_CONFIGURE_FLAGS= --enable-gssapi_krb5 \ - --enable-imfile \ +DISTCHECK_CONFIGURE_FLAGS= --enable-imfile \ --enable-snmp \ --enable-libdbi \ --enable-mysql \ @@ -284,10 +314,12 @@ DISTCHECK_CONFIGURE_FLAGS= --enable-gssapi_krb5 \ --enable-pmaixforwardedfrom \ --enable-pmcisconames \ --enable-pmsnare \ - --enable-mmsnmptrapd \ --enable-elasticsearch \ + --enable-valgrind \ --with-systemdsystemunitdir=$$dc_install_base/$(systemdsystemunitdir) # temporarily disable these checks for make distcheck 2012-09-06 rgerhards +# --enable-mmsnmptrapd \ +# --enable-gssapi_krb5 \ # --enable-extended-tests \ -# --enable-pgsql \ +# --enable-pgsql ACLOCAL_AMFLAGS = -I m4 diff --git a/Makefile.in b/Makefile.in index 5088b0b..260cc75 100644 --- a/Makefile.in +++ b/Makefile.in @@ -57,36 +57,43 @@ sbin_PROGRAMS = @ENABLE_SMCUSTBINDCDR_TRUE@am__append_14 = plugins/sm_cust_bindcdr @ENABLE_OMSTDOUT_TRUE@am__append_15 = plugins/omstdout @ENABLE_PMCISCONAMES_TRUE@am__append_16 = plugins/pmcisconames -@ENABLE_PMAIXFORWARDEDFROM_TRUE@am__append_17 = plugins/pmaixforwardedfrom -@ENABLE_PMSNARE_TRUE@am__append_18 = plugins/pmsnare -@ENABLE_PMLASTMSG_TRUE@am__append_19 = plugins/pmlastmsg -@ENABLE_PMRFC3164SD_TRUE@am__append_20 = plugins/pmrfc3164sd -@ENABLE_OMRULESET_TRUE@am__append_21 = plugins/omruleset -@ENABLE_OMUDPSPOOF_TRUE@am__append_22 = plugins/omudpspoof -@ENABLE_OMMONGODB_TRUE@am__append_23 = plugins/ommongodb -@ENABLE_OMHIREDIS_TRUE@am__append_24 = plugins/omhiredis -@ENABLE_OMZMQ3_TRUE@am__append_25 = plugins/omzmq3 -@ENABLE_OMRABBITMQ_TRUE@am__append_26 = plugins/omrabbitmq -@ENABLE_IMZMQ3_TRUE@am__append_27 = plugins/imzmq3 -@ENABLE_OMUXSOCK_TRUE@am__append_28 = plugins/omuxsock -@ENABLE_OMHDFS_TRUE@am__append_29 = plugins/omhdfs -@ENABLE_OMJOURNAL_TRUE@am__append_30 = plugins/omjournal -@ENABLE_IMJOURNAL_TRUE@am__append_31 = plugins/imjournal -@ENABLE_ELASTICSEARCH_TRUE@am__append_32 = plugins/omelasticsearch -@ENABLE_MMSNMPTRAPD_TRUE@am__append_33 = plugins/mmsnmptrapd -@ENABLE_IMFILE_TRUE@am__append_34 = plugins/imfile -@ENABLE_IMPTCP_TRUE@am__append_35 = plugins/imptcp -@ENABLE_IMTTCP_TRUE@am__append_36 = plugins/imttcp -@ENABLE_IMDIAG_TRUE@am__append_37 = plugins/imdiag -@ENABLE_MAIL_TRUE@am__append_38 = plugins/ommail -@ENABLE_OMPROG_TRUE@am__append_39 = plugins/omprog -@ENABLE_RFC3195_TRUE@am__append_40 = plugins/im3195 -@ENABLE_MMNORMALIZE_TRUE@am__append_41 = plugins/mmnormalize -@ENABLE_MMJSONPARSE_TRUE@am__append_42 = plugins/mmjsonparse -@ENABLE_MMAUDIT_TRUE@am__append_43 = plugins/mmaudit -@ENABLE_MMANON_TRUE@am__append_44 = plugins/mmanon -@ENABLE_ORACLE_TRUE@am__append_45 = plugins/omoracle -@ENABLE_GUI_TRUE@am__append_46 = java +@ENABLE_PMCISCOIOS_TRUE@am__append_17 = plugins/pmciscoios +@ENABLE_PMAIXFORWARDEDFROM_TRUE@am__append_18 = plugins/pmaixforwardedfrom +@ENABLE_PMSNARE_TRUE@am__append_19 = plugins/pmsnare +@ENABLE_PMLASTMSG_TRUE@am__append_20 = plugins/pmlastmsg +@ENABLE_PMRFC3164SD_TRUE@am__append_21 = plugins/pmrfc3164sd +@ENABLE_OMRULESET_TRUE@am__append_22 = plugins/omruleset +@ENABLE_OMUDPSPOOF_TRUE@am__append_23 = plugins/omudpspoof +@ENABLE_OMMONGODB_TRUE@am__append_24 = plugins/ommongodb +@ENABLE_OMHIREDIS_TRUE@am__append_25 = plugins/omhiredis +@ENABLE_OMZMQ3_TRUE@am__append_26 = plugins/omzmq3 +@ENABLE_OMRABBITMQ_TRUE@am__append_27 = plugins/omrabbitmq +@ENABLE_IMZMQ3_TRUE@am__append_28 = plugins/imzmq3 +@ENABLE_OMUXSOCK_TRUE@am__append_29 = plugins/omuxsock +@ENABLE_OMHDFS_TRUE@am__append_30 = plugins/omhdfs +@ENABLE_OMJOURNAL_TRUE@am__append_31 = plugins/omjournal +@ENABLE_IMJOURNAL_TRUE@am__append_32 = plugins/imjournal +@ENABLE_ELASTICSEARCH_TRUE@am__append_33 = plugins/omelasticsearch +@ENABLE_MMSNMPTRAPD_TRUE@am__append_34 = plugins/mmsnmptrapd +@ENABLE_IMFILE_TRUE@am__append_35 = plugins/imfile +@ENABLE_IMPTCP_TRUE@am__append_36 = plugins/imptcp +@ENABLE_IMTTCP_TRUE@am__append_37 = plugins/imttcp +@ENABLE_IMDIAG_TRUE@am__append_38 = plugins/imdiag +@ENABLE_MAIL_TRUE@am__append_39 = plugins/ommail +@ENABLE_OMPROG_TRUE@am__append_40 = plugins/omprog +@ENABLE_RFC3195_TRUE@am__append_41 = plugins/im3195 +@ENABLE_MMNORMALIZE_TRUE@am__append_42 = plugins/mmnormalize +@ENABLE_MMJSONPARSE_TRUE@am__append_43 = plugins/mmjsonparse +@ENABLE_MMAUDIT_TRUE@am__append_44 = plugins/mmaudit +@ENABLE_MMANON_TRUE@am__append_45 = plugins/mmanon +@ENABLE_MMUTF8FIX_TRUE@am__append_46 = plugins/mmutf8fix +@ENABLE_MMCOUNT_TRUE@am__append_47 = plugins/mmcount +@ENABLE_MMSEQUENCE_TRUE@am__append_48 = plugins/mmsequence +@ENABLE_MMFIELDS_TRUE@am__append_49 = plugins/mmfields +@ENABLE_MMPSTRUCDATA_TRUE@am__append_50 = plugins/mmpstrucdata +@ENABLE_MMRFC5424ADDHMAC_TRUE@am__append_51 = plugins/mmrfc5424addhmac +@ENABLE_ORACLE_TRUE@am__append_52 = plugins/omoracle +@ENABLE_GUI_TRUE@am__append_53 = java subdir = . DIST_COMMON = README $(am__configure_deps) $(srcdir)/Makefile.am \ $(srcdir)/Makefile.in $(srcdir)/config.h.in \ @@ -215,23 +222,25 @@ AM_RECURSIVE_TARGETS = $(RECURSIVE_TARGETS:-recursive=) \ distdir dist dist-all distcheck ETAGS = etags CTAGS = ctags -DIST_SUBDIRS = doc compat runtime grammar . plugins/immark \ +DIST_SUBDIRS = compat runtime grammar . plugins/immark \ plugins/imuxsock plugins/imtcp plugins/imudp plugins/omtesting \ - tools plugins/imklog plugins/imkmsg plugins/impstats \ - plugins/imsolaris plugins/omgssapi plugins/imgssapi \ - plugins/omrelp plugins/imrelp plugins/ommysql plugins/omlibdbi \ - plugins/ompgsql plugins/omsnmp plugins/sm_cust_bindcdr \ - plugins/omstdout plugins/pmcisconames \ - plugins/pmaixforwardedfrom plugins/pmsnare plugins/pmlastmsg \ - plugins/pmrfc3164sd plugins/omruleset plugins/omudpspoof \ - plugins/ommongodb plugins/omhiredis plugins/omzmq3 \ - plugins/omrabbitmq plugins/imzmq3 plugins/omuxsock \ - plugins/omhdfs plugins/omjournal plugins/imjournal \ - plugins/omelasticsearch plugins/mmsnmptrapd plugins/imfile \ - plugins/imptcp plugins/imttcp plugins/imdiag plugins/ommail \ - plugins/omprog plugins/im3195 plugins/mmnormalize \ - plugins/mmjsonparse plugins/mmaudit plugins/mmanon \ - plugins/omoracle java tests + plugins/mmexternal tools plugins/imklog plugins/imkmsg \ + plugins/impstats plugins/imsolaris plugins/omgssapi \ + plugins/imgssapi plugins/omrelp plugins/imrelp plugins/ommysql \ + plugins/omlibdbi plugins/ompgsql plugins/omsnmp \ + plugins/sm_cust_bindcdr plugins/omstdout plugins/pmcisconames \ + plugins/pmciscoios plugins/pmaixforwardedfrom plugins/pmsnare \ + plugins/pmlastmsg plugins/pmrfc3164sd plugins/omruleset \ + plugins/omudpspoof plugins/ommongodb plugins/omhiredis \ + plugins/omzmq3 plugins/omrabbitmq plugins/imzmq3 \ + plugins/omuxsock plugins/omhdfs plugins/omjournal \ + plugins/imjournal plugins/omelasticsearch plugins/mmsnmptrapd \ + plugins/imfile plugins/imptcp plugins/imttcp plugins/imdiag \ + plugins/ommail plugins/omprog plugins/im3195 \ + plugins/mmnormalize plugins/mmjsonparse plugins/mmaudit \ + plugins/mmanon plugins/mmutf8fix plugins/mmcount \ + plugins/mmsequence plugins/mmfields plugins/mmpstrucdata \ + plugins/mmrfc5424addhmac plugins/omoracle java tests DISTFILES = $(DIST_COMMON) $(DIST_SOURCES) $(TEXINFOS) $(EXTRA_DIST) distdir = $(PACKAGE)-$(VERSION) top_distdir = $(distdir) @@ -308,7 +317,6 @@ GREP = @GREP@ GSS_LIBS = @GSS_LIBS@ GUARDTIME_CFLAGS = @GUARDTIME_CFLAGS@ GUARDTIME_LIBS = @GUARDTIME_LIBS@ -HAVE_LIBGCRYPT_CONFIG = @HAVE_LIBGCRYPT_CONFIG@ HAVE_MYSQL_CONFIG = @HAVE_MYSQL_CONFIG@ HAVE_ORACLE_CONFIG = @HAVE_ORACLE_CONFIG@ HAVE_PGSQL_CONFIG = @HAVE_PGSQL_CONFIG@ @@ -329,14 +337,15 @@ LEXLIB = @LEXLIB@ LEX_OUTPUT_ROOT = @LEX_OUTPUT_ROOT@ LIBDBI_CFLAGS = @LIBDBI_CFLAGS@ LIBDBI_LIBS = @LIBDBI_LIBS@ -LIBEE_CFLAGS = @LIBEE_CFLAGS@ -LIBEE_LIBS = @LIBEE_LIBS@ LIBESTR_CFLAGS = @LIBESTR_CFLAGS@ LIBESTR_LIBS = @LIBESTR_LIBS@ LIBGCRYPT_CFLAGS = @LIBGCRYPT_CFLAGS@ +LIBGCRYPT_CONFIG = @LIBGCRYPT_CONFIG@ LIBGCRYPT_LIBS = @LIBGCRYPT_LIBS@ LIBLOGGING_CFLAGS = @LIBLOGGING_CFLAGS@ LIBLOGGING_LIBS = @LIBLOGGING_LIBS@ +LIBLOGGING_STDLOG_CFLAGS = @LIBLOGGING_STDLOG_CFLAGS@ +LIBLOGGING_STDLOG_LIBS = @LIBLOGGING_STDLOG_LIBS@ LIBLOGNORM_CFLAGS = @LIBLOGNORM_CFLAGS@ LIBLOGNORM_LIBS = @LIBLOGNORM_LIBS@ LIBM = @LIBM@ @@ -361,6 +370,8 @@ NM = @NM@ NMEDIT = @NMEDIT@ OBJDUMP = @OBJDUMP@ OBJEXT = @OBJEXT@ +OPENSSL_CFLAGS = @OPENSSL_CFLAGS@ +OPENSSL_LIBS = @OPENSSL_LIBS@ ORACLE_CFLAGS = @ORACLE_CFLAGS@ ORACLE_LIBS = @ORACLE_LIBS@ OTOOL = @OTOOL@ @@ -500,6 +511,7 @@ pkgconfigdir = $(libdir)/pkgconfig @HAVE_SYSTEMD_TRUE@ rsyslog.service EXTRA_DIST = \ + README.md \ platform/README \ platform/freebsd/rsyslogd \ platform/slackware/rc.rsyslogd \ @@ -513,26 +525,30 @@ EXTRA_DIST = \ contrib/gnutls/key.pem \ rsyslog.service.in +# external plugin driver is always enabled (core component) # tests are added as last element, because tests may need different # modules that need to be generated first -SUBDIRS = doc compat runtime grammar . plugins/immark plugins/imuxsock \ - plugins/imtcp plugins/imudp plugins/omtesting $(am__append_3) \ - $(am__append_4) $(am__append_5) $(am__append_6) \ - $(am__append_7) $(am__append_8) $(am__append_9) \ - $(am__append_10) $(am__append_11) $(am__append_12) \ - $(am__append_13) $(am__append_14) $(am__append_15) \ - $(am__append_16) $(am__append_17) $(am__append_18) \ - $(am__append_19) $(am__append_20) $(am__append_21) \ - $(am__append_22) $(am__append_23) $(am__append_24) \ - $(am__append_25) $(am__append_26) $(am__append_27) \ - $(am__append_28) $(am__append_29) $(am__append_30) \ - $(am__append_31) $(am__append_32) $(am__append_33) \ - $(am__append_34) $(am__append_35) $(am__append_36) \ - $(am__append_37) $(am__append_38) $(am__append_39) \ - $(am__append_40) $(am__append_41) $(am__append_42) \ - $(am__append_43) $(am__append_44) $(am__append_45) \ - $(am__append_46) tests +SUBDIRS = compat runtime grammar . plugins/immark plugins/imuxsock \ + plugins/imtcp plugins/imudp plugins/omtesting \ + plugins/mmexternal $(am__append_3) $(am__append_4) \ + $(am__append_5) $(am__append_6) $(am__append_7) \ + $(am__append_8) $(am__append_9) $(am__append_10) \ + $(am__append_11) $(am__append_12) $(am__append_13) \ + $(am__append_14) $(am__append_15) $(am__append_16) \ + $(am__append_17) $(am__append_18) $(am__append_19) \ + $(am__append_20) $(am__append_21) $(am__append_22) \ + $(am__append_23) $(am__append_24) $(am__append_25) \ + $(am__append_26) $(am__append_27) $(am__append_28) \ + $(am__append_29) $(am__append_30) $(am__append_31) \ + $(am__append_32) $(am__append_33) $(am__append_34) \ + $(am__append_35) $(am__append_36) $(am__append_37) \ + $(am__append_38) $(am__append_39) $(am__append_40) \ + $(am__append_41) $(am__append_42) $(am__append_43) \ + $(am__append_44) $(am__append_45) $(am__append_46) \ + $(am__append_47) $(am__append_48) $(am__append_49) \ + $(am__append_50) $(am__append_51) $(am__append_52) \ + $(am__append_53) tests # make sure "make distcheck" tries to build all modules. This means that # a developer must always have an environment where every supporting library @@ -540,8 +556,7 @@ SUBDIRS = doc compat runtime grammar . plugins/immark plugins/imuxsock \ # temporarily be removed below. The intent behind forcing everthing to compile # in a make distcheck is so that we detect code that accidently was not updated # when some global update happened. -DISTCHECK_CONFIGURE_FLAGS = --enable-gssapi_krb5 \ - --enable-imfile \ +DISTCHECK_CONFIGURE_FLAGS = --enable-imfile \ --enable-snmp \ --enable-libdbi \ --enable-mysql \ @@ -564,10 +579,16 @@ DISTCHECK_CONFIGURE_FLAGS = --enable-gssapi_krb5 \ --enable-pmaixforwardedfrom \ --enable-pmcisconames \ --enable-pmsnare \ - --enable-mmsnmptrapd \ --enable-elasticsearch \ + --enable-valgrind \ --with-systemdsystemunitdir=$$dc_install_base/$(systemdsystemunitdir) +# temporarily disable these checks for make distcheck 2012-09-06 rgerhards +# --enable-mmsnmptrapd \ +# --enable-gssapi_krb5 \ +# --enable-extended-tests \ +# --enable-pgsql +ACLOCAL_AMFLAGS = -I m4 all: config.h $(MAKE) $(AM_MAKEFLAGS) all-recursive @@ -715,22 +736,25 @@ distclean-compile: @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/lmtcpsrv_la-tcpsrv.Plo@am__quote@ .c.o: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(COMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ $< -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Po +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.o$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c $< +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ $< .c.obj: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(COMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ `$(CYGPATH_W) '$<'` -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Po +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.obj$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ `$(CYGPATH_W) '$<'` &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c `$(CYGPATH_W) '$<'` +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ `$(CYGPATH_W) '$<'` .c.lo: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(LTCOMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ $< -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Plo +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.lo$$||'`;\ +@am__fastdepCC_TRUE@ $(LTCOMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Plo @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=yes @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ @am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(LTCOMPILE) -c -o $@ $< @@ -1256,10 +1280,6 @@ uninstall-am: uninstall-nodist_systemdsystemunitDATA \ @HAVE_SYSTEMD_TRUE@%.service: %.service.in @HAVE_SYSTEMD_TRUE@ $(AM_V_GEN)sed -e 's,@sbindir\@,$(sbindir),g' $< > $@ -# temporarily disable these checks for make distcheck 2012-09-06 rgerhards -# --enable-extended-tests \ -# --enable-pgsql \ -#ACLOCAL_AMFLAGS = -I m4 # Tell versions [3.59,3.63) of GNU make to not export all variables. # Otherwise a system limit (for SysV at least) may be exceeded. @@ -1,5 +1 @@ -This file has been superseeded by the files in the doc folder. -Please see doc/manual.html for futher details. If you are -looking for install information doc/install.html is for you! -If you do not have the doc set, see - http://www.rsyslog.com/doc +see README.md -- this file here is just required for autotools diff --git a/README.md b/README.md new file mode 100644 index 0000000..bfe4efe --- /dev/null +++ b/README.md @@ -0,0 +1,93 @@ +Rsyslog - what is it? +===================== +Rsyslog is a **r**ocket-fast **sys**tem for **log** processing. + +It offers high-performance, great security features and a modular design. +While it started as a regular syslogd, rsyslog has evolved into a kind of swiss +army knife of logging, being able to accept inputs from a wide variety of sources, +transform them, and output to the results to diverse destinations. + +Rsyslog can deliver over one million messages per second to local destinations +when limited processing is applied (based on v7, December 2013). Even with +remote destinations and more elaborate processing the performance is usually +considered "stunning". + +Mailing List +============ +http://lists.adiscon.net/mailman/listinfo/rsyslog + +Installing rsyslog +================== +Most distributions carry rsyslog in their repository. So you usually just need +to use the package manager to install it. Note that on non-systemd systems (most +notably Ubuntu), rsyslog usually is already installed. + +Project-Provided Packages +---------------------------- +Unfortunately, distributions often do not catch up with the pace of rsyslog +development and as such only offer old versions. To solve that problem, we have +created packages for current versions ourselves. + +They are available for: + * RPM-based systems: http://www.rsyslog.com/rhelcentos-rpms/ + * Ubuntu: http://www.rsyslog.com/ubuntu-repository/ + * Debian: http://www.rsyslog.com/debian-repository/ + +Building from Source +-------------------- +Follow the instructions at: http://www.rsyslog.com/doc/build_from_repo.html + +Reporting Bugs +============== + +Talk to the mailing list if you think something is a bug. Often, it's just a +matter of doing some config trickery. + +File bugs at: https://github.com/rsyslog/rsyslog/issues + +How to Contribute +================= +Contributions to rsyslog are very welcome. Fork and send us your Pull Requests. + +For more information about contributing, see the +[CONTRIBUTING](CONTRIBUTING.md) file. + +Note that it is esay to add output plugins using languages like Python or +Perl. So if you need to connect to a system which is not yet supported, you +can easily do so via an external plugin. For more information see the +[README](plugins/external/README.md) file in the external plugin directory. + +Documentation +============= +The main rsyslog documentation is available in HTML format. To read +it, point your web browser to ./doc/manual.html. Alternatively, +you can view the documentation for *the most recent rsyslog version* +online at: http://www.rsyslog.com/doc + +Project Philosophy +================== +We are an open source project in all aspects and very open to outside feedback +and contribution. We base our work on standards and try to solve all real-world +needs (of course, we occasionally fail tackeling actually all needs ;)). While +the project is primarily sponsored by Adiscon, technical development is +independent from company goals and most decisions are solely based on mailing +list discussion results. There is an active commuity around rsyslog. + +There is no such thing like being an official member of the rsyslog team. The +closest to that is being subscribed to the mailing list: +http://lists.adiscon.net/mailman/listinfo/rsyslog + +This method of open discussions is modelled after the IETF process, which is +probably the best-known and most successive collaborative standards body. + +Project Funding +=============== +Rsyslog's main sponsor Adiscon tries to fund rsyslog by selling custom +development and support contracts. Adiscon does NOT license rsyslog under a +commercial license (this is simply impossible for anyone due to rsyslog's +license structure). + +Any third party is obviously also free to offer custom development, support +and rsyslog consulting. We gladly merge results of such third-party work into +the main repository (assuming it matches the few essential things written +down in our contribution policy). @@ -13,18 +13,15 @@ * The different modes (and calling sequence) are: * * if set iExecEveryNthOccur > 1 || iSecsExecOnceInterval - * - doSubmitToActionQComplexBatch - * - helperSubmitToActionQComplexBatch - * - doActionCallAction + * - doSubmitToActionQComplex * handles mark message reduction, but in essence calls * - actionWriteToAction * - qqueueEnqObj * (now queue engine processing) - * if(pThis->bWriteAllMarkMsgs == RSFALSE) - this is the DEFAULT - * - doSubmitToActionQNotAllMarkBatch - * - doSubmitToActionQBatch (and from here like in the else case below!) + * if(pThis->bWriteAllMarkMsgs == RSFALSE) + * - doSubmitToActionQNotAllMark + * - doSubmitToActionQ (and from here like in the else case below!) * else - * - doSubmitToActionQBatch * - doSubmitToActionQ * - qqueueEnqObj * (now queue engine processing) @@ -36,9 +33,6 @@ * * After dequeue, processing is as follows: * - processBatchMain - * - processAction - * - submitBatch - * - tryDoAction * - ... * * MORE ON PROCESSING, QUEUES and FILTERING @@ -69,7 +63,7 @@ * beast. * rgerhards, 2011-06-15 * - * Copyright 2007-2011 Rainer Gerhards and Adiscon GmbH. + * Copyright 2007-2013 Rainer Gerhards and Adiscon GmbH. * * This file is part of rsyslog. * @@ -98,7 +92,7 @@ #include <strings.h> #include <time.h> #include <errno.h> -#include <json/json.h> +#include <json.h> #include "dirty.h" #include "template.h" @@ -114,18 +108,18 @@ #include "unicode-helper.h" #include "atomic.h" #include "ruleset.h" +#include "parserif.h" #include "statsobj.h" #define NO_TIME_PROVIDED 0 /* indicate we do not provide any cached time */ /* forward definitions */ -static rsRetVal processBatchMain(action_t *pAction, batch_t *pBatch, int*); -static rsRetVal doSubmitToActionQComplexBatch(action_t *pAction, batch_t *pBatch); -static rsRetVal doSubmitToActionQNotAllMarkBatch(action_t *pAction, batch_t *pBatch); -static rsRetVal doSubmitToActionQBatch(action_t *pAction, batch_t *pBatch); +static rsRetVal processBatchMain(void *pVoid, batch_t *pBatch, wti_t * const pWti); +static rsRetVal doSubmitToActionQ(action_t * const pAction, wti_t * const pWti, msg_t*); +static rsRetVal doSubmitToActionQComplex(action_t * const pAction, wti_t * const pWti, msg_t*); +static rsRetVal doSubmitToActionQNotAllMark(action_t * const pAction, wti_t * const pWti, msg_t*); /* object static data (once for all instances) */ -/* TODO: make this an object! DEFobjStaticHelpers -- rgerhards, 2008-03-05 */ DEFobjCurrIf(obj) DEFobjCurrIf(datetime) DEFobjCurrIf(module) @@ -175,10 +169,11 @@ configSettings_t cs_save; /* our saved (scope!) config settings */ /* the counter below counts actions created. It is used to obtain unique IDs for the action. They * should not be relied on for any long-term activity (e.g. disk queue names!), but they are nice * to have during one instance of an rsyslogd run. For example, I use them to name actions when there - * is no better name available. Note that I do NOT recover previous numbers on HUP - we simply keep - * counting. -- rgerhards, 2008-01-29 + * is no better name available. */ -static int iActionNbr = 0; +int iActionNbr = 0; +int bActionReportSuspension = 1; +int bActionReportSuspensionCont = 0; /* tables for interfacing with the v6 config system */ static struct cnfparamdescr cnfparamdescr[] = { @@ -191,6 +186,8 @@ static struct cnfparamdescr cnfparamdescr[] = { { "action.execonlywhenpreviousissuspended", eCmdHdlrBinary, 0 }, /* legacy: actionexeconlywhenpreviousissuspended */ { "action.repeatedmsgcontainsoriginalmsg", eCmdHdlrBinary, 0 }, /* legacy: repeatedmsgcontainsoriginalmsg */ { "action.resumeretrycount", eCmdHdlrInt, 0 }, /* legacy: actionresumeretrycount */ + { "action.reportsuspension", eCmdHdlrBinary, 0 }, + { "action.reportsuspensioncontinuation", eCmdHdlrBinary, 0 }, { "action.resumeinterval", eCmdHdlrInt, 0 } }; static struct cnfparamblk pblk = @@ -224,7 +221,7 @@ static struct cnfparamblk pblk = * a lot. So we simply return the system time. */ static inline time_t -getActNow(action_t *pThis) +getActNow(action_t * const pThis) { assert(pThis != NULL); if(pThis->tActNow == -1) { @@ -253,8 +250,8 @@ actionResetQueueParams(void) cs.ActionQueType = QUEUETYPE_DIRECT; /* type of the main message queue above */ cs.iActionQueueSize = 1000; /* size of the main message queue above */ cs.iActionQueueDeqBatchSize = 16; /* default batch size */ - cs.iActionQHighWtrMark = 800; /* high water mark for disk-assisted queues */ - cs.iActionQLowWtrMark = 200; /* low water mark for disk-assisted queues */ + cs.iActionQHighWtrMark = -1; /* high water mark for disk-assisted queues */ + cs.iActionQLowWtrMark = -1; /* low water mark for disk-assisted queues */ cs.iActionQDiscardMark = 980; /* begin to discard messages */ cs.iActionQDiscardSeverity = 8; /* discard warning and above */ cs.iActionQueueNumWorkers = 1; /* number of worker threads for the mm queue above */ @@ -265,7 +262,7 @@ actionResetQueueParams(void) cs.iActionQtoActShutdown = 1000; /* action shutdown (in phase 2) */ cs.iActionQtoEnq = 50; /* timeout for queue enque */ cs.iActionQtoWrkShutdown = 60000; /* timeout for worker thread shutdown */ - cs.iActionQWrkMinMsgs = 100; /* minimum messages per worker needed to start a new one */ + cs.iActionQWrkMinMsgs = -1; /* minimum messages per worker needed to start a new one */ cs.bActionQSaveOnShutdown = 1; /* save queue on shutdown (when DA enabled)? */ cs.iActionQueMaxDiskSpace = 0; cs.iActionQueueDeqSlowdown = 0; @@ -284,7 +281,7 @@ actionResetQueueParams(void) /* destructs an action descriptor object * rgerhards, 2007-08-01 */ -rsRetVal actionDestruct(action_t *pThis) +rsRetVal actionDestruct(action_t * const pThis) { DEFiRet; ASSERT(pThis != NULL); @@ -304,13 +301,13 @@ rsRetVal actionDestruct(action_t *pThis) if(pThis->statsobj != NULL) statsobj.Destruct(&pThis->statsobj); - if(pThis->pMod != NULL) + if(pThis->pModData != NULL) pThis->pMod->freeInstance(pThis->pModData); pthread_mutex_destroy(&pThis->mutAction); - pthread_mutex_destroy(&pThis->mutActExec); d_free(pThis->pszName); d_free(pThis->ppTpl); + d_free(pThis->peParamPassing); finalize_it: d_free(pThis); @@ -318,6 +315,19 @@ finalize_it: } +/* Disable action, this means it will never again be usable + * until rsyslog is reloaded. Use only as a last resort, but + * depends on output module. + * rgerhards, 2007-08-02 + */ +static inline void +actionDisable(action_t *__restrict__ const pThis) +{ + pThis->bDisabled = 1; +} + + + /* create a new action descriptor object * rgerhards, 2007-08-01 * Note that it is vital to set proper initial values as the v6 config @@ -334,14 +344,18 @@ rsRetVal actionConstruct(action_t **ppThis) pThis->iResumeInterval = 30; pThis->iResumeRetryCount = 0; pThis->pszName = NULL; - pThis->bWriteAllMarkMsgs = RSFALSE; + pThis->bWriteAllMarkMsgs = 1; pThis->iExecEveryNthOccur = 0; pThis->iExecEveryNthOccurTO = 0; pThis->iSecsExecOnceInterval = 0; pThis->bExecWhenPrevSusp = 0; pThis->bRepMsgHasMsg = 0; + pThis->bDisabled = 0; + pThis->isTransactional = 0; + pThis->bReportSuspension = -1; /* indicate "not yet set" */ + pThis->bReportSuspensionCont = -1; /* indicate "not yet set" */ pThis->tLastOccur = datetime.GetTime(NULL); /* done once per action on startup only */ - pthread_mutex_init(&pThis->mutActExec, NULL); + pThis->iActionNbr = iActionNbr; pthread_mutex_init(&pThis->mutAction, NULL); INIT_ATOMIC_HELPER_MUT(pThis->mutCAS); @@ -357,13 +371,11 @@ finalize_it: /* action construction finalizer */ rsRetVal -actionConstructFinalize(action_t *pThis, struct cnfparamvals *queueParams) +actionConstructFinalize(action_t *__restrict__ const pThis, struct nvlst *lst) { DEFiRet; uchar pszAName[64]; /* friendly name of our action */ - ASSERT(pThis != NULL); - if(!strcmp((char*)modGetName(pThis->pMod), "builtin:omdiscard")) { /* discard actions will be optimized out */ FINALIZE; @@ -371,35 +383,60 @@ actionConstructFinalize(action_t *pThis, struct cnfparamvals *queueParams) /* generate a friendly name for us action stats */ if(pThis->pszName == NULL) { snprintf((char*) pszAName, sizeof(pszAName)/sizeof(uchar), "action %d", iActionNbr); - } else { - ustrncpy(pszAName, pThis->pszName, sizeof(pszAName)); - pszAName[sizeof(pszAName)-1] = '\0'; /* to be on the save side */ + pThis->pszName = ustrdup(pszAName); + } + + /* cache transactional attribute */ + pThis->isTransactional = pThis->pMod->mod.om.supportsTX; + if(pThis->isTransactional) { + int i; + for(i = 0 ; i < pThis->iNumTpls ; ++i) { + if(pThis->peParamPassing[i] != ACT_STRING_PASSING) { + errmsg.LogError(0, RS_RET_INVLD_OMOD, "action '%s'(%d) is transactional but " + "parameter %d " + "uses invalid paramter passing mode -- disabling " + "action. This is probably caused by a pre-v7 " + "output module that needs upgrade.", + pThis->pszName, pThis->iActionNbr, i); + actionDisable(pThis); + ABORT_FINALIZE(RS_RET_INVLD_OMOD); + + } + } } + /* support statistics gathering */ CHKiRet(statsobj.Construct(&pThis->statsobj)); - CHKiRet(statsobj.SetName(pThis->statsobj, pszAName)); + CHKiRet(statsobj.SetName(pThis->statsobj, pThis->pszName)); STATSCOUNTER_INIT(pThis->ctrProcessed, pThis->mutCtrProcessed); CHKiRet(statsobj.AddCounter(pThis->statsobj, UCHAR_CONSTANT("processed"), - ctrType_IntCtr, &pThis->ctrProcessed)); + ctrType_IntCtr, CTR_FLAG_RESETTABLE, &pThis->ctrProcessed)); STATSCOUNTER_INIT(pThis->ctrFail, pThis->mutCtrFail); CHKiRet(statsobj.AddCounter(pThis->statsobj, UCHAR_CONSTANT("failed"), - ctrType_IntCtr, &pThis->ctrFail)); + ctrType_IntCtr, CTR_FLAG_RESETTABLE, &pThis->ctrFail)); + + STATSCOUNTER_INIT(pThis->ctrSuspend, pThis->mutCtrSuspend); + CHKiRet(statsobj.AddCounter(pThis->statsobj, UCHAR_CONSTANT("suspended"), + ctrType_IntCtr, CTR_FLAG_RESETTABLE, &pThis->ctrSuspend)); + STATSCOUNTER_INIT(pThis->ctrSuspendDuration, pThis->mutCtrSuspendDuration); + CHKiRet(statsobj.AddCounter(pThis->statsobj, UCHAR_CONSTANT("suspended.duration"), + ctrType_IntCtr, 0, &pThis->ctrSuspendDuration)); + + STATSCOUNTER_INIT(pThis->ctrResume, pThis->mutCtrResume); + CHKiRet(statsobj.AddCounter(pThis->statsobj, UCHAR_CONSTANT("resumed"), + ctrType_IntCtr, CTR_FLAG_RESETTABLE, &pThis->ctrResume)); CHKiRet(statsobj.ConstructFinalize(pThis->statsobj)); /* create our queue */ /* generate a friendly name for the queue */ - if(pThis->pszName == NULL) { - snprintf((char*) pszAName, sizeof(pszAName)/sizeof(uchar), "action %d queue", - iActionNbr); - } else { - ustrncpy(pszAName, pThis->pszName, sizeof(pszAName)); - pszAName[63] = '\0'; /* to be on the save side */ - } + snprintf((char*) pszAName, sizeof(pszAName)/sizeof(uchar), "%s queue", + pThis->pszName); + /* now check if we can run the action in "firehose mode" during stage one of * its processing (that is before messages are enqueued into the action q). * This is only possible if some features, which require strict sequence, are @@ -412,13 +449,13 @@ actionConstructFinalize(action_t *pThis, struct cnfparamvals *queueParams) DBGPRINTF("info: firehose mode disabled for action because " "iExecEveryNthOccur=%d, iSecsExecOnceInterval=%d\n", pThis->iExecEveryNthOccur, pThis->iSecsExecOnceInterval); - pThis->submitToActQ = doSubmitToActionQComplexBatch; - } else if(pThis->bWriteAllMarkMsgs == RSFALSE) { - /* nearly full-speed submission mode, default case */ - pThis->submitToActQ = doSubmitToActionQNotAllMarkBatch; + pThis->submitToActQ = doSubmitToActionQComplex; + } else if(pThis->bWriteAllMarkMsgs) { + /* full firehose submission mode, default case*/ + pThis->submitToActQ = doSubmitToActionQ; } else { - /* full firehose submission mode */ - pThis->submitToActQ = doSubmitToActionQBatch; + /* nearly full-speed submission mode */ + pThis->submitToActQ = doSubmitToActionQNotAllMark; } /* create queue */ @@ -428,11 +465,11 @@ actionConstructFinalize(action_t *pThis, struct cnfparamvals *queueParams) * spec. -- rgerhards, 2008-01-30 */ CHKiRet(qqueueConstruct(&pThis->pQueue, cs.ActionQueType, 1, cs.iActionQueueSize, - (rsRetVal (*)(void*, batch_t*, int*))processBatchMain)); + processBatchMain)); obj.SetName((obj_t*) pThis->pQueue, pszAName); qqueueSetpAction(pThis->pQueue, pThis); - if(queueParams == NULL) { /* use legacy params? */ + if(lst == NULL) { /* use legacy params? */ /* ... set some properties ... */ # define setQPROP(func, directive, data) \ CHKiRet_Hdlr(func(pThis->pQueue, data)) { \ @@ -459,6 +496,7 @@ actionConstructFinalize(action_t *pThis, struct cnfparamvals *queueParams) setQPROP(qqueueSetiDiscardMrk, "$ActionQueueDiscardMark", cs.iActionQDiscardMark); setQPROP(qqueueSetiDiscardSeverity, "$ActionQueueDiscardSeverity", cs.iActionQDiscardSeverity); setQPROP(qqueueSetiMinMsgsPerWrkr, "$ActionQueueWorkerThreadMinimumMessages", cs.iActionQWrkMinMsgs); + setQPROP(qqueueSetiNumWorkerThreads, "$ActionQueueWorkerThreads", cs.iActionQueueNumWorkers); setQPROP(qqueueSetbSaveOnShutdown, "$ActionQueueSaveOnShutdown", cs.bActionQSaveOnShutdown); setQPROP(qqueueSetiDeqSlowdown, "$ActionQueueDequeueSlowdown", cs.iActionQueueDeqSlowdown); setQPROP(qqueueSetiDeqtWinFromHr, "$ActionQueueDequeueTimeBegin", cs.iActionQueueDeqtWinFromHr); @@ -466,7 +504,7 @@ actionConstructFinalize(action_t *pThis, struct cnfparamvals *queueParams) } else { /* we have v6-style config params */ qqueueSetDefaultsActionQueue(pThis->pQueue); - qqueueApplyCnfParam(pThis->pQueue, queueParams); + qqueueApplyCnfParam(pThis->pQueue, lst); } # undef setQPROP @@ -475,6 +513,12 @@ actionConstructFinalize(action_t *pThis, struct cnfparamvals *queueParams) qqueueDbgPrint(pThis->pQueue); DBGPRINTF("Action %p: queue %p created\n", pThis, pThis->pQueue); + + if(pThis->bUsesMsgPassingMode && pThis->pQueue->qType != QUEUETYPE_DIRECT) { + parser_warnmsg("module %s with message passing mode uses " + "non-direct queue. This most probably leads to undesired " + "results", (char*)modGetName(pThis->pMod)); + } /* and now reset the queue params (see comment in its function header!) */ actionResetQueueParams(); @@ -498,9 +542,9 @@ rsRetVal actionSetGlobalResumeInterval(int iNewVal) * returned string must not be modified. * rgerhards, 2009-05-07 */ -static uchar *getActStateName(action_t *pThis) +static uchar *getActStateName(action_t * const pThis, wti_t * const pWti) { - switch(pThis->eState) { + switch(getActionState(pWti, pThis)) { case ACT_STATE_RDY: return (uchar*) "rdy"; case ACT_STATE_ITX: @@ -509,8 +553,6 @@ static uchar *getActStateName(action_t *pThis) return (uchar*) "rtry"; case ACT_STATE_SUSP: return (uchar*) "susp"; - case ACT_STATE_DIED: - return (uchar*) "died"; case ACT_STATE_COMM: return (uchar*) "comm"; default: @@ -522,12 +564,12 @@ static uchar *getActStateName(action_t *pThis) /* returns a suitable return code based on action state * rgerhards, 2009-05-07 */ -static rsRetVal getReturnCode(action_t *pThis) +static rsRetVal getReturnCode(action_t * const pThis, wti_t * const pWti) { DEFiRet; ASSERT(pThis != NULL); - switch(pThis->eState) { + switch(getActionState(pWti, pThis)) { case ACT_STATE_RDY: iRet = RS_RET_OK; break; @@ -543,12 +585,11 @@ static rsRetVal getReturnCode(action_t *pThis) iRet = RS_RET_SUSPENDED; break; case ACT_STATE_SUSP: - case ACT_STATE_DIED: iRet = RS_RET_ACTION_FAILED; break; default: - DBGPRINTF("Invalid action engine state %d, program error\n", - (int) pThis->eState); + DBGPRINTF("Invalid action engine state %u, program error\n", + getActionState(pWti, pThis)); iRet = RS_RET_ERR; break; } @@ -560,44 +601,34 @@ static rsRetVal getReturnCode(action_t *pThis) /* set the action to a new state * rgerhards, 2007-08-02 */ -static inline void actionSetState(action_t *pThis, action_state_t newState) +static inline void +actionSetState(action_t * const pThis, wti_t * const pWti, uint8_t newState) { - pThis->eState = newState; - DBGPRINTF("Action %p transitioned to state: %s\n", pThis, getActStateName(pThis)); + setActionState(pWti, pThis, newState); + DBGPRINTF("Action %d transitioned to state: %s\n", + pThis->iActionNbr, getActStateName(pThis, pWti)); } /* Handles the transient commit state. So far, this is * mostly a dummy... * rgerhards, 2007-08-02 */ -static void actionCommitted(action_t *pThis) +static void actionCommitted(action_t * const pThis, wti_t * const pWti) { - actionSetState(pThis, ACT_STATE_RDY); + actionSetState(pThis, pWti, ACT_STATE_RDY); } /* set action to "rtry" state. * rgerhards, 2007-08-02 */ -static void actionRetry(action_t *pThis) +static void actionRetry(action_t * const pThis, wti_t * const pWti) { - actionSetState(pThis, ACT_STATE_RTRY); - pThis->iResumeOKinRow++; + actionSetState(pThis, pWti, ACT_STATE_RTRY); + incActionResumeInRow(pWti, pThis); } - -/* Disable action, this means it will never again be usable - * until rsyslog is reloaded. Use only as a last resort, but - * depends on output module. - * rgerhards, 2007-08-02 - */ -static void actionDisable(action_t *pThis) -{ - actionSetState(pThis, ACT_STATE_DIED); -} - - -/* Suspend action, this involves changing the acton state as well +/* Suspend action, this involves changing the action state as well * as setting the next retry time. * if we have more than 10 retries, we prolong the * retry interval. If something is really stalled, it will @@ -605,17 +636,50 @@ static void actionDisable(action_t *pThis) * CPU time. TODO: maybe a config option for that? * rgerhards, 2007-08-02 */ -static inline void actionSuspend(action_t *pThis) +static inline void +actionSuspend(action_t * const pThis, wti_t * const pWti) { time_t ttNow; + int suspendDuration; + char timebuf[32]; + + /* we need to defer setting the action's own bReportSuspension state until + * after the full config has been processed. So the most simple case to do + * that is here. It's not a performance problem, as it happens infrequently. + * it's not a threading race problem, as always the same value will be written. + */ + if(pThis->bReportSuspension == -1) + pThis->bReportSuspension = bActionReportSuspension; + if(pThis->bReportSuspensionCont == -1) { + pThis->bReportSuspensionCont = bActionReportSuspensionCont; + if(pThis->bReportSuspensionCont == -1) + pThis->bReportSuspension = 1; + } /* note: we can NOT use a cached timestamp, as time may have evolved * since caching, and this would break logic (and it actually did so!) */ datetime.GetTime(&ttNow); - pThis->ttResumeRtry = ttNow + pThis->iResumeInterval * (pThis->iNbrResRtry / 10 + 1); - actionSetState(pThis, ACT_STATE_SUSP); - DBGPRINTF("action suspended, earliest retry=%d\n", (int) pThis->ttResumeRtry); + suspendDuration = pThis->iResumeInterval * (getActionNbrResRtry(pWti, pThis) / 10 + 1); + pThis->ttResumeRtry = ttNow + suspendDuration; + actionSetState(pThis, pWti, ACT_STATE_SUSP); + pThis->ctrSuspendDuration += suspendDuration; + if(getActionNbrResRtry(pWti, pThis) == 0) { + STATSCOUNTER_INC(pThis->ctrSuspend, pThis->mutCtrSuspend); + } + + if( pThis->bReportSuspensionCont + || (pThis->bReportSuspension && getActionNbrResRtry(pWti, pThis) == 0) ) { + ctime_r(&pThis->ttResumeRtry, timebuf); + timebuf[strlen(timebuf)-1] = '\0'; /* strip LF */ + errmsg.LogMsg(0, RS_RET_SUSPENDED, LOG_WARNING, + "action '%s' suspended, next retry is %s", + pThis->pszName, timebuf); + } + DBGPRINTF("action '%s' suspended, earliest retry=%lld (now %lld), iNbrResRtry %d, " + "duration %d\n", + pThis->pszName, (long long) pThis->ttResumeRtry, (long long) ttNow, + getActionNbrResRtry(pWti, pThis), suspendDuration); } @@ -627,15 +691,15 @@ static inline void actionSuspend(action_t *pThis) * entry point. This is invalid, but has harsh consequences: it will cause the rsyslog * engine to go into a tight loop. That obviously is not acceptable. As such, we track the * count of iterations that a tryResume returning RS_RET_OK is immediately followed by - * an unsuccessful call to doAction(). If that happens more than 1,000 times, we assume + * an unsuccessful call to doAction(). If that happens more than 10 times, we assume * the return acutally is a RS_RET_SUSPENDED. In order to go through the various - * resumption stages, we do this for every 1000 requests. This magic number 1000 may + * resumption stages, we do this for every 10 requests. This magic number 10 may * not be the most appropriate, but it should be thought of a "if nothing else helps" * kind of facility: in the first place, the module should return a proper indication * of its inability to recover. -- rgerhards, 2010-04-26. */ -static inline rsRetVal -actionDoRetry(action_t *pThis, int *pbShutdownImmediate) +static rsRetVal +actionDoRetry(action_t * const pThis, wti_t * const pWti) { int iRetries; int iSleepPeriod; @@ -645,31 +709,40 @@ actionDoRetry(action_t *pThis, int *pbShutdownImmediate) ASSERT(pThis != NULL); iRetries = 0; - while((*pbShutdownImmediate == 0) && pThis->eState == ACT_STATE_RTRY) { - DBGPRINTF("actionDoRetry: enter loop, iRetries=%d\n", iRetries); - iRet = pThis->pMod->tryResume(pThis->pModData); - DBGPRINTF("actionDoRetry: action->tryResume returned %d\n", iRet); - if((pThis->iResumeOKinRow > 9) && (pThis->iResumeOKinRow % 10 == 0)) { + while((*pWti->pbShutdownImmediate == 0) && getActionState(pWti, pThis) == ACT_STATE_RTRY) { + DBGPRINTF("actionDoRetry: %s enter loop, iRetries=%d\n", pThis->pszName, iRetries); + iRet = pThis->pMod->tryResume(pWti->actWrkrInfo[pThis->iActionNbr].actWrkrData); + DBGPRINTF("actionDoRetry: %s action->tryResume returned %d\n", pThis->pszName, iRet); + if((getActionResumeInRow(pWti, pThis) > 9) && (getActionResumeInRow(pWti, pThis) % 10 == 0)) { bTreatOKasSusp = 1; - pThis->iResumeOKinRow = 0; + setActionResumeInRow(pWti, pThis, 0); } else { bTreatOKasSusp = 0; } if((iRet == RS_RET_OK) && (!bTreatOKasSusp)) { - DBGPRINTF("actionDoRetry: had success RDY again (iRet=%d)\n", iRet); - actionSetState(pThis, ACT_STATE_RDY); + DBGPRINTF("actionDoRetry: %s had success RDY again (iRet=%d)\n", + pThis->pszName, iRet); + if(pThis->bReportSuspension) { + errmsg.LogMsg(0, RS_RET_OK, LOG_INFO, "action '%s' " + "resumed (module '%s')", + pThis->pszName, pThis->pMod->pszName); + } + setActionJustResumed(pWti, pThis, 1); + actionSetState(pThis, pWti, ACT_STATE_RDY); } else if(iRet == RS_RET_SUSPENDED || bTreatOKasSusp) { /* max retries reached? */ - DBGPRINTF("actionDoRetry: check for max retries, iResumeRetryCount %d, iRetries %d\n", - pThis->iResumeRetryCount, iRetries); + DBGPRINTF("actionDoRetry: %s check for max retries, iResumeRetryCount " + "%d, iRetries %d\n", + pThis->pszName, pThis->iResumeRetryCount, iRetries); if((pThis->iResumeRetryCount != -1 && iRetries >= pThis->iResumeRetryCount)) { - actionSuspend(pThis); + actionSuspend(pThis, pWti); + if(getActionNbrResRtry(pWti, pThis) < 20) + incActionNbrResRtry(pWti, pThis); } else { - ++pThis->iNbrResRtry; ++iRetries; iSleepPeriod = pThis->iResumeInterval; srSleep(iSleepPeriod, 0); - if(*pbShutdownImmediate) { + if(*pWti->pbShutdownImmediate) { ABORT_FINALIZE(RS_RET_FORCE_TERM); } } @@ -678,8 +751,8 @@ actionDoRetry(action_t *pThis, int *pbShutdownImmediate) } } - if(pThis->eState == ACT_STATE_RDY) { - pThis->iNbrResRtry = 0; + if(getActionState(pWti, pThis) == ACT_STATE_RDY) { + setActionNbrResRtry(pWti, pThis, 0); } finalize_it: @@ -687,17 +760,32 @@ finalize_it: } +static rsRetVal +actionCheckAndCreateWrkrInstance(action_t * const pThis, wti_t * const pWti) +{ + DEFiRet; + if(pWti->actWrkrInfo[pThis->iActionNbr].actWrkrData == NULL) { + DBGPRINTF("wti %p: we need to create a new action worker instance for " + "action %d\n", pWti, pThis->iActionNbr); + CHKiRet(pThis->pMod->mod.om.createWrkrInstance(&(pWti->actWrkrInfo[pThis->iActionNbr].actWrkrData), + pThis->pModData)); + pWti->actWrkrInfo[pThis->iActionNbr].pAction = pThis; + setActionState(pWti, pThis, ACT_STATE_RDY); /* action is enabled */ + } +finalize_it: + RETiRet; +} + /* try to resume an action -- rgerhards, 2007-08-02 * changed to new action state engine -- rgerhards, 2009-05-07 */ -static rsRetVal actionTryResume(action_t *pThis, int *pbShutdownImmediate) +static rsRetVal +actionTryResume(action_t * const pThis, wti_t * const pWti) { DEFiRet; time_t ttNow = NO_TIME_PROVIDED; - ASSERT(pThis != NULL); - - if(pThis->eState == ACT_STATE_SUSP) { + if(getActionState(pWti, pThis) == ACT_STATE_SUSP) { /* if we are suspended, we need to check if the timeout expired. * for this handling, we must always obtain a fresh timestamp. We used * to use the action timestamp, but in this case we will never reach a @@ -707,19 +795,19 @@ static rsRetVal actionTryResume(action_t *pThis, int *pbShutdownImmediate) */ datetime.GetTime(&ttNow); /* cache "now" */ if(ttNow >= pThis->ttResumeRtry) { - actionSetState(pThis, ACT_STATE_RTRY); /* back to retries */ + actionSetState(pThis, pWti, ACT_STATE_RTRY); /* back to retries */ } } - if(pThis->eState == ACT_STATE_RTRY) { + if(getActionState(pWti, pThis) == ACT_STATE_RTRY) { if(ttNow == NO_TIME_PROVIDED) /* use cached result if we have it */ datetime.GetTime(&ttNow); - CHKiRet(actionDoRetry(pThis, pbShutdownImmediate)); + CHKiRet(actionDoRetry(pThis, pWti)); } - if(Debug && (pThis->eState == ACT_STATE_RTRY ||pThis->eState == ACT_STATE_SUSP)) { + if(Debug && (getActionState(pWti, pThis) == ACT_STATE_RTRY ||getActionState(pWti, pThis) == ACT_STATE_SUSP)) { DBGPRINTF("actionTryResume: action %p state: %s, next retry (if applicable): %u [now %u]\n", - pThis, getActStateName(pThis), (unsigned) pThis->ttResumeRtry, (unsigned) ttNow); + pThis, getActStateName(pThis, pWti), (unsigned) pThis->ttResumeRtry, (unsigned) ttNow); } finalize_it: @@ -731,24 +819,25 @@ finalize_it: * depending on its current state. * rgerhards, 2009-05-07 */ -static inline rsRetVal actionPrepare(action_t *pThis, int *pbShutdownImmediate) +static inline rsRetVal +actionPrepare(action_t *__restrict__ const pThis, wti_t *__restrict__ const pWti) { DEFiRet; - assert(pThis != NULL); - CHKiRet(actionTryResume(pThis, pbShutdownImmediate)); + CHKiRet(actionCheckAndCreateWrkrInstance(pThis, pWti)); + CHKiRet(actionTryResume(pThis, pWti)); /* if we are now ready, we initialize the transaction and advance * action state accordingly */ - if(pThis->eState == ACT_STATE_RDY) { - iRet = pThis->pMod->mod.om.beginTransaction(pThis->pModData); + if(getActionState(pWti, pThis) == ACT_STATE_RDY) { + iRet = pThis->pMod->mod.om.beginTransaction(pWti->actWrkrInfo[pThis->iActionNbr].actWrkrData); switch(iRet) { case RS_RET_OK: - actionSetState(pThis, ACT_STATE_ITX); + actionSetState(pThis, pWti, ACT_STATE_ITX); break; case RS_RET_SUSPENDED: - actionRetry(pThis); + actionRetry(pThis, pWti); break; case RS_RET_DISABLE_ACTION: actionDisable(pThis); @@ -762,10 +851,11 @@ finalize_it: } +#if 0 // TODO: remove? /* debug-print the contents of an action object * rgerhards, 2007-08-02 */ -rsRetVal actionDbgPrint(action_t *pThis) +static rsRetVal actionDbgPrint(action_t *pThis) { DEFiRet; char *sz; @@ -775,11 +865,12 @@ rsRetVal actionDbgPrint(action_t *pThis) dbgprintf("\n"); dbgprintf("\tInstance data: 0x%lx\n", (unsigned long) pThis->pModData); dbgprintf("\tResume Interval: %d\n", pThis->iResumeInterval); - if(pThis->eState == ACT_STATE_SUSP) { +#if 0 // do we need this ??? + if(getActionState(pWti, pThis) == ACT_STATE_SUSP) { dbgprintf("\tresume next retry: %u, number retries: %d", (unsigned) pThis->ttResumeRtry, pThis->iNbrResRtry); } - dbgprintf("\tState: %s\n", getActStateName(pThis)); +#endif dbgprintf("\tExec only when previous is suspended: %d\n", pThis->bExecWhenPrevSusp); if(pThis->submitToActQ == doSubmitToActionQComplexBatch) { sz = "slow, but feature-rich"; @@ -795,45 +886,55 @@ rsRetVal actionDbgPrint(action_t *pThis) RETiRet; } +#endif /* prepare the calling parameters for doAction() * rgerhards, 2009-05-07 */ static rsRetVal -prepareDoActionParams(action_t *pAction, batch_obj_t *pElem, struct syslogTime *ttNow) +prepareDoActionParams(action_t * __restrict__ const pAction, + wti_t * __restrict__ const pWti, + msg_t *__restrict__ const pMsg, + struct syslogTime *ttNow) { int i; - msg_t *pMsg; struct json_object *json; + actWrkrIParams_t *iparams; + actWrkrInfo_t *__restrict__ pWrkrInfo; DEFiRet; - ASSERT(pAction != NULL); - ASSERT(pElem != NULL); - - pMsg = pElem->pMsg; - /* here we must loop to process all requested strings */ - for(i = 0 ; i < pAction->iNumTpls ; ++i) { - switch(pAction->eParamPassing) { + pWrkrInfo = &(pWti->actWrkrInfo[pAction->iActionNbr]); + if(pAction->isTransactional) { + CHKiRet(wtiNewIParam(pWti, pAction, &iparams)); + for(i = 0 ; i < pAction->iNumTpls ; ++i) { + CHKiRet(tplToString(pAction->ppTpl[i], pMsg, + &actParam(iparams, pAction->iNumTpls, 0, i), + ttNow)); + } + } else { + for(i = 0 ; i < pAction->iNumTpls ; ++i) { + switch(pAction->peParamPassing[i]) { case ACT_STRING_PASSING: - CHKiRet(tplToString(pAction->ppTpl[i], pMsg, &(pElem->staticActStrings[i]), - &pElem->staticLenStrings[i], ttNow)); - pElem->staticActParams[i] = pElem->staticActStrings[i]; + CHKiRet(tplToString(pAction->ppTpl[i], pMsg, + &(pWrkrInfo->p.nontx.actParams[i]), + ttNow)); break; case ACT_ARRAY_PASSING: - CHKiRet(tplToArray(pAction->ppTpl[i], pMsg, (uchar***) &(pElem->staticActParams[i]), ttNow)); + CHKiRet(tplToArray(pAction->ppTpl[i], pMsg, + (uchar***) &(pWrkrInfo->p.nontx.actParams[i].param), ttNow)); break; case ACT_MSG_PASSING: - pElem->staticActParams[i] = (void*) pMsg; + pWrkrInfo->p.nontx.actParams[i].param = (void*) pMsg; break; case ACT_JSON_PASSING: CHKiRet(tplToJSON(pAction->ppTpl[i], pMsg, &json, ttNow)); - pElem->staticActParams[i] = (void*) json; + pWrkrInfo->p.nontx.actParams[i].param = (void*) json; break; - default:dbgprintf("software bug/error: unknown pAction->eParamPassing %d in prepareDoActionParams\n", - (int) pAction->eParamPassing); - assert(0); /* software bug if this happens! */ + default:dbgprintf("software bug/error: unknown pAction->peParamPassing[%d] %d in prepareDoActionParams\n", + i, (int) pAction->peParamPassing[i]); break; + } } } @@ -842,96 +943,87 @@ finalize_it: } -/* free a batches ressources, but not string buffers (because they will - * most probably be reused). String buffers are only deleted upon final - * destruction of the batch. - * This function here must be called only when the batch is actually no - * longer used, also not for retrying actions or such. It invalidates - * buffers. - * rgerhards, 2010-12-17 - */ -static rsRetVal releaseBatch(action_t *pAction, batch_t *pBatch) +static void +releaseDoActionParams(action_t *__restrict__ const pAction, wti_t *__restrict__ const pWti) { int jArr; - int i, j; - batch_obj_t *pElem; + int j; + actWrkrInfo_t *__restrict__ pWrkrInfo; uchar ***ppMsgs; - DEFiRet; - - ASSERT(pAction != NULL); - if(pAction->eParamPassing == ACT_STRING_PASSING || pAction->eParamPassing == ACT_MSG_PASSING) - goto done; /* we need to do nothing with these types! */ - - for(i = 0 ; i < batchNumMsgs(pBatch) && !*(pBatch->pbShutdownImmediate) ; ++i) { - pElem = &(pBatch->pElem[i]); - if(batchIsValidElem(pBatch, i)) { - switch(pAction->eParamPassing) { - case ACT_ARRAY_PASSING: - ppMsgs = (uchar***) pElem->staticActParams; - for(j = 0 ; j < pAction->iNumTpls ; ++j) { - if(((uchar**)ppMsgs)[j] != NULL) { - jArr = 0; - while(ppMsgs[j][jArr] != NULL) { - d_free(ppMsgs[j][jArr]); - ppMsgs[j][jArr] = NULL; - ++jArr; - } - d_free(((uchar**)ppMsgs)[j]); - ((uchar**)ppMsgs)[j] = NULL; - } + pWrkrInfo = &(pWti->actWrkrInfo[pAction->iActionNbr]); + for(j = 0 ; j < pAction->iNumTpls ; ++j) { + switch(pAction->peParamPassing[j]) { + case ACT_ARRAY_PASSING: + ppMsgs = (uchar***) pWrkrInfo->p.nontx.actParams[0].param; + if(((uchar**)ppMsgs)[j] != NULL) { + jArr = 0; + while(ppMsgs[j][jArr] != NULL) { + free(ppMsgs[j][jArr]); + ppMsgs[j][jArr] = NULL; + ++jArr; } - break; - case ACT_JSON_PASSING: - for(j = 0 ; j < pAction->iNumTpls ; ++j) { - json_object_put((struct json_object*) - pElem->staticActParams[j]); - pElem->staticActParams[j] = NULL; - } - break; - case ACT_STRING_PASSING: - case ACT_MSG_PASSING: - /* can never happen, just to keep compiler happy! */ - break; + free(((uchar**)ppMsgs)[j]); + ((uchar**)ppMsgs)[j] = NULL; } + break; + case ACT_JSON_PASSING: + json_object_put((struct json_object*) + pWrkrInfo->p.nontx.actParams[j].param); + pWrkrInfo->p.nontx.actParams[j].param = NULL; + break; + case ACT_STRING_PASSING: + case ACT_MSG_PASSING: + /* no need to do anything with these */ + break; } } -done: RETiRet; + return; } - -/* call the DoAction output plugin entry point - * rgerhards, 2008-01-28 +/* This is used in resume processing. We only finally know that a resume + * worked when we have been able to actually process a messages. As such, + * we need to do some cleanup and status tracking in that case. */ -rsRetVal -actionCallDoAction(action_t *pThis, msg_t *pMsg, void *actParams) +static void +actionSetActionWorked(action_t *__restrict__ const pThis, wti_t *__restrict__ const pWti) { - DEFiRet; - - ASSERT(pThis != NULL); - ISOBJ_TYPE_assert(pMsg, msg); - - DBGPRINTF("entering actionCalldoAction(), state: %s\n", getActStateName(pThis)); + setActionResumeInRow(pWti, pThis, 0); + + if(getActionJustResumed(pWti, pThis)) { + /* OK, we *really* could resume, so tell user! */ + if(pThis->bReportSuspension) { + errmsg.LogMsg(0, RS_RET_RESUMED, LOG_INFO, "action '%s' " + "resumed (module '%s')", + pThis->pszName, pThis->pMod->pszName); + } + setActionJustResumed(pWti, pThis, 0); + } +} - pThis->bHadAutoCommit = 0; - iRet = pThis->pMod->mod.om.doAction(actParams, pMsg->msgFlags, pThis->pModData); - switch(iRet) { +static rsRetVal +handleActionExecResult(action_t *__restrict__ const pThis, + wti_t *__restrict__ const pWti, + const rsRetVal ret) +{ + DEFiRet; + switch(ret) { case RS_RET_OK: - actionCommitted(pThis); - pThis->iResumeOKinRow = 0; /* we had a successful call! */ + actionCommitted(pThis, pWti); + actionSetActionWorked(pThis, pWti); /* we had a successful call! */ break; case RS_RET_DEFER_COMMIT: - pThis->iResumeOKinRow = 0; /* we had a successful call! */ + actionSetActionWorked(pThis, pWti); /* we had a successful call! */ /* we are done, action state remains the same */ break; case RS_RET_PREVIOUS_COMMITTED: /* action state remains the same, but we had a commit. */ pThis->bHadAutoCommit = 1; - pThis->iResumeOKinRow = 0; /* we had a successful call! */ + actionSetActionWorked(pThis, pWti); /* we had a successful call! */ break; case RS_RET_SUSPENDED: - actionRetry(pThis); + actionRetry(pThis, pWti); break; case RS_RET_DISABLE_ACTION: actionDisable(pThis); @@ -939,70 +1031,134 @@ actionCallDoAction(action_t *pThis, msg_t *pMsg, void *actParams) default:/* permanent failure of this message - no sense in retrying. This is * not yet handled (but easy TODO) */ + iRet = ret; FINALIZE; } - iRet = getReturnCode(pThis); + iRet = getReturnCode(pThis, pWti); finalize_it: RETiRet; } +/* call the DoAction output plugin entry point + * rgerhards, 2008-01-28 + */ +static rsRetVal +actionCallDoAction(action_t *__restrict__ const pThis, + actWrkrIParams_t *__restrict__ const iparams, + wti_t *__restrict__ const pWti) +{ + uchar *param[CONF_OMOD_NUMSTRINGS_MAXSIZE]; + int i; + DEFiRet; + + DBGPRINTF("entering actionCalldoAction(), state: %s, actionNbr %d\n", + getActStateName(pThis, pWti), pThis->iActionNbr); + + pThis->bHadAutoCommit = 0; + /* for this interface, we need to emulate the old style way + * of parameter passing. + */ + for(i = 0 ; i < pThis->iNumTpls ; ++i) { + param[i] = actParam(iparams, pThis->iNumTpls, 0, i).param; + } + + iRet = pThis->pMod->mod.om.doAction(param, + pWti->actWrkrInfo[pThis->iActionNbr].actWrkrData); + iRet = handleActionExecResult(pThis, pWti, iRet); + RETiRet; +} + + +/* call the commitTransaction output plugin entry point */ +static rsRetVal +actionCallCommitTransaction(action_t * const pThis, + const actWrkrInfo_t *const wrkrInfo, + wti_t *const pWti) +{ + DEFiRet; + + ASSERT(pThis != NULL); + + DBGPRINTF("entering actionCallCommitTransaction(), state: %s, actionNbr %d, " + "nMsgs %u\n", + getActStateName(pThis, pWti), pThis->iActionNbr, + wrkrInfo->p.tx.currIParam); + + iRet = pThis->pMod->mod.om.commitTransaction( + pWti->actWrkrInfo[pThis->iActionNbr].actWrkrData, + wrkrInfo->p.tx.iparams, wrkrInfo->p.tx.currIParam); + iRet = handleActionExecResult(pThis, pWti, iRet); + RETiRet; +} + /* process a message * this readies the action and then calls doAction() * rgerhards, 2008-01-28 */ -static inline rsRetVal -actionProcessMessage(action_t *pThis, msg_t *pMsg, void *actParams, int *pbShutdownImmediate) +rsRetVal +actionProcessMessage(action_t * const pThis, void *actParams, wti_t * const pWti) { DEFiRet; - ASSERT(pThis != NULL); - ISOBJ_TYPE_assert(pMsg, msg); - - CHKiRet(actionPrepare(pThis, pbShutdownImmediate)); + CHKiRet(actionPrepare(pThis, pWti)); if(pThis->pMod->mod.om.SetShutdownImmdtPtr != NULL) - pThis->pMod->mod.om.SetShutdownImmdtPtr(pThis->pModData, pbShutdownImmediate); - if(pThis->eState == ACT_STATE_ITX) - CHKiRet(actionCallDoAction(pThis, pMsg, actParams)); + pThis->pMod->mod.om.SetShutdownImmdtPtr(pThis->pModData, pWti->pbShutdownImmediate); + if(getActionState(pWti, pThis) == ACT_STATE_ITX) + CHKiRet(actionCallDoAction(pThis, actParams, pWti)); - iRet = getReturnCode(pThis); + iRet = getReturnCode(pThis, pWti); finalize_it: RETiRet; } -/* finish processing a batch. Most importantly, that means we commit if we - * need to do so. - * rgerhards, 2008-01-28 - */ +/* the following functions simulates a potential future new omo callback */ static rsRetVal -finishBatch(action_t *pThis, batch_t *pBatch) +doTransaction(action_t *__restrict__ const pThis, wti_t *__restrict__ const pWti) { + actWrkrInfo_t *wrkrInfo; int i; DEFiRet; - ASSERT(pThis != NULL); - - if(pThis->eState == ACT_STATE_RDY) { - /* we just need to flag the batch as commited */ - FINALIZE; /* nothing to do */ + wrkrInfo = &(pWti->actWrkrInfo[pThis->iActionNbr]); + if(pThis->pMod->mod.om.commitTransaction != NULL) { + DBGPRINTF("doTransaction: have commitTransaction IF, using that, pWrkrInfo %p\n", wrkrInfo); + CHKiRet(actionCallCommitTransaction(pThis, wrkrInfo, pWti)); + } else { /* note: this branch is for compatibility with old TX modules */ + DBGPRINTF("doTransaction: action %d, currIParam %d\n", + pThis->iActionNbr, wrkrInfo->p.tx.currIParam); + for(i = 0 ; i < wrkrInfo->p.tx.currIParam ; ++i) { + /* Note: we provide the message's base iparam - actionProcessMessage() + * uses this as *base* address. + */ + iRet = actionProcessMessage(pThis, + &actParam(wrkrInfo->p.tx.iparams, pThis->iNumTpls, i, 0), pWti); + } } +finalize_it: + RETiRet; +} + + +/* Commit try committing (do not handle retry processing and such) */ +static rsRetVal +actionTryCommit(action_t *__restrict__ const pThis, wti_t *__restrict__ const pWti) +{ + DEFiRet; + + doTransaction(pThis, pWti); - CHKiRet(actionPrepare(pThis, pBatch->pbShutdownImmediate)); - if(pThis->eState == ACT_STATE_ITX) { - iRet = pThis->pMod->mod.om.endTransaction(pThis->pModData); + CHKiRet(actionPrepare(pThis, pWti)); + if(getActionState(pWti, pThis) == ACT_STATE_ITX) { + iRet = pThis->pMod->mod.om.endTransaction(pWti->actWrkrInfo[pThis->iActionNbr].actWrkrData); switch(iRet) { case RS_RET_OK: - actionCommitted(pThis); - /* flag messages as committed */ - for(i = 0 ; i < pBatch->nElem ; ++i) { - batchSetElemState(pBatch, i, BATCH_STATE_COMM); - pBatch->pElem[i].bPrevWasSuspended = 0; /* we had success! */ - } + actionCommitted(pThis, pWti); break; case RS_RET_SUSPENDED: - actionRetry(pThis); + actionRetry(pThis, pWti); break; case RS_RET_DISABLE_ACTION: actionDisable(pThis); @@ -1010,12 +1166,12 @@ finishBatch(action_t *pThis, batch_t *pBatch) case RS_RET_DEFER_COMMIT: DBGPRINTF("output plugin error: endTransaction() returns RS_RET_DEFER_COMMIT " "- ignored\n"); - actionCommitted(pThis); + actionCommitted(pThis, pWti); break; case RS_RET_PREVIOUS_COMMITTED: DBGPRINTF("output plugin error: endTransaction() returns RS_RET_PREVIOUS_COMMITTED " "- ignored\n"); - actionCommitted(pThis); + actionCommitted(pThis, pWti); break; default:/* permanent failure of this message - no sense in retrying. This is * not yet handled (but easy TODO) @@ -1023,325 +1179,173 @@ finishBatch(action_t *pThis, batch_t *pBatch) FINALIZE; } } - iRet = getReturnCode(pThis); - -finalize_it: - RETiRet; -} - - -/* try to submit a partial batch of elements. - * rgerhards, 2009-05-12 - */ -static inline rsRetVal -tryDoAction(action_t *pAction, batch_t *pBatch, int *pnElem) -{ - int i; - int iElemProcessed; - int iCommittedUpTo; - msg_t *pMsg; - rsRetVal localRet; - DEFiRet; - - assert(pBatch != NULL); - assert(pnElem != NULL); - - i = pBatch->iDoneUpTo; /* all messages below that index are processed */ - iElemProcessed = 0; - iCommittedUpTo = i; - DBGPRINTF("tryDoAction %p, pnElem %d, nElem %d\n", pAction, *pnElem, pBatch->nElem); - while(iElemProcessed <= *pnElem && i < pBatch->nElem) { - if(*(pBatch->pbShutdownImmediate)) - ABORT_FINALIZE(RS_RET_FORCE_TERM); - /* NOTE: do NOT extend the filter below! Anything else must be done on the - * enq side of the queue (see file header comment)! -- rgerhards, 2011-06-15 - */ - if(batchIsValidElem(pBatch, i)) { - pMsg = pBatch->pElem[i].pMsg; - localRet = actionProcessMessage(pAction, pMsg, pBatch->pElem[i].staticActParams, - pBatch->pbShutdownImmediate); - DBGPRINTF("action %p call returned %d\n", pAction, localRet); - /* Note: we directly modify the batch object state, because we know that - * wo do not overwrite BATCH_STATE_DISC indicators! - */ - if(localRet == RS_RET_OK) { - /* mark messages as committed */ - while(iCommittedUpTo <= i) { - pBatch->pElem[iCommittedUpTo].bPrevWasSuspended = 0; /* we had success! */ - batchSetElemState(pBatch, iCommittedUpTo, BATCH_STATE_COMM); - ++iCommittedUpTo; - //pBatch->pElem[iCommittedUpTo++].state = BATCH_STATE_COMM; - } - } else if(localRet == RS_RET_PREVIOUS_COMMITTED) { - /* mark messages as committed */ - while(iCommittedUpTo < i) { - pBatch->pElem[iCommittedUpTo].bPrevWasSuspended = 0; /* we had success! */ - batchSetElemState(pBatch, iCommittedUpTo, BATCH_STATE_COMM); - ++iCommittedUpTo; - //pBatch->pElem[iCommittedUpTo++].state = BATCH_STATE_COMM; - } - pBatch->eltState[i] = BATCH_STATE_SUB; - } else if(localRet == RS_RET_DEFER_COMMIT) { - pBatch->eltState[i] = BATCH_STATE_SUB; - } else if(localRet == RS_RET_DISCARDMSG) { - pBatch->eltState[i] = BATCH_STATE_DISC; - } else { - dbgprintf("tryDoAction: unexpected error code %d[nElem %d, Commited UpTo %d], finalizing\n", - localRet, *pnElem, iCommittedUpTo); - iRet = localRet; - FINALIZE; - } - } - ++i; - ++iElemProcessed; - } + iRet = getReturnCode(pThis, pWti); finalize_it: - if(pBatch->iDoneUpTo != iCommittedUpTo) { - pBatch->iDoneUpTo = iCommittedUpTo; - } + pWti->actWrkrInfo[pThis->iActionNbr].p.tx.currIParam = 0; /* reset to beginning */ RETiRet; } -/* submit a batch for actual action processing. - * The first nElem elements are processed. This function calls itself - * recursively if it needs to handle errors. - * Note: we don't need the number of the first message to be processed as a parameter, - * because this is kept track of inside the batch itself (iDoneUpTo). - * rgerhards, 2009-05-12 +/* Note: we currently need to return an iRet, as this is used in + * direct mode. TODO: However, it may be worth further investigating this, + * as it looks like there is no ultimate consumer of this code. + * rgerhards, 2013-11-06 */ static rsRetVal -submitBatch(action_t *pAction, batch_t *pBatch, int nElem) +actionCommit(action_t *__restrict__ const pThis, wti_t *__restrict__ const pWti) { - int i; - int bDone; - rsRetVal localRet; - int wasDoneTo; + sbool bDone; DEFiRet; - assert(pBatch != NULL); + if(!pThis->isTransactional || + pWti->actWrkrInfo[pThis->iActionNbr].p.tx.currIParam == 0 || + getActionState(pWti, pThis) == ACT_STATE_SUSP + ) { + FINALIZE; + } - DBGPRINTF("submitBatch: enter, nElem %d\n", nElem); - wasDoneTo = pBatch->iDoneUpTo; + /* even more TODO: + This is the place where retry processing needs to go in. If the action + permanently fails, we should - as a new feature - add the capability to + write an error file. This is already done be omelasticsearch, and IMHO + pretty useful. + For the time being, I do NOT implement all of this (not even retry!) + as I want to get the rest of the engine to SISD (non-SIMD ;)) so that + I know any potential suprises and complications that arise out of this. + When this is done, I can come back here and complete this work. Obviously, + many features do not work in the mean time (but it is not planned to release + any of these partial implementations). + rgerhards, 2013-11-04 + */ bDone = 0; do { - localRet = tryDoAction(pAction, pBatch, &nElem); - if(localRet == RS_RET_FORCE_TERM) { + iRet = actionTryCommit(pThis, pWti); + DBGPRINTF("actionCommit, in retry loop, iRet %d\n", iRet); + if(iRet == RS_RET_FORCE_TERM) { ABORT_FINALIZE(RS_RET_FORCE_TERM); - } - if( localRet == RS_RET_OK - || localRet == RS_RET_PREVIOUS_COMMITTED - || localRet == RS_RET_DEFER_COMMIT) { - /* try commit transaction, once done, we can simply do so as if - * that return state was returned from tryDoAction(). - */ - localRet = finishBatch(pAction, pBatch); - } - - if( localRet == RS_RET_OK - || localRet == RS_RET_PREVIOUS_COMMITTED - || localRet == RS_RET_DEFER_COMMIT) { + } else if(iRet == RS_RET_OK || + iRet == RS_RET_SUSPENDED || + iRet == RS_RET_ACTION_FAILED) { bDone = 1; - } else if(localRet == RS_RET_SUSPENDED) { - DBGPRINTF("action ret RS_RET_SUSPENDED - retry full batch\n"); - /* do nothing, this will retry the full batch */ - } else if(localRet == RS_RET_ACTION_FAILED) { - /* in this case, everything not yet committed is BAD */ - for(i = pBatch->iDoneUpTo ; i < wasDoneTo + nElem ; ++i) { - if( pBatch->eltState[i] != BATCH_STATE_DISC - && pBatch->eltState[i] != BATCH_STATE_COMM ) { - pBatch->eltState[i] = BATCH_STATE_BAD; - pBatch->pElem[i].bPrevWasSuspended = 1; - STATSCOUNTER_INC(pAction->ctrFail, pAction->mutCtrFail); - } - } + } + if(getActionState(pWti, pThis) == ACT_STATE_RDY || + getActionState(pWti, pThis) == ACT_STATE_SUSP) { bDone = 1; - } else { - if(nElem == 1) { - batchSetElemState(pBatch, pBatch->iDoneUpTo, BATCH_STATE_BAD); - bDone = 1; - } else { - /* retry with half as much. Depth is log_2 batchsize, so recursion is not too deep */ - DBGPRINTF("submitBatch recursing trying to find and exclude the culprit " - "for iRet %d\n", localRet); - submitBatch(pAction, pBatch, nElem / 2); - submitBatch(pAction, pBatch, nElem - (nElem / 2)); - bDone = 1; - } } - } while(!bDone && !*(pBatch->pbShutdownImmediate)); /* do .. while()! */ - - if(*(pBatch->pbShutdownImmediate)) - ABORT_FINALIZE(RS_RET_FORCE_TERM); - + } while(!bDone); finalize_it: RETiRet; } - -/* copy "active" array of batch, as we need to modify it. The caller - * must make sure the new array is freed and the orginal batch - * pointer is restored (thus the caller must save it). If active - * is currently NULL, this is properly handled. - * Note: the batches active pointer is modified, so it must be - * saved BEFORE calling this function! - * rgerhards, 2012-09-12 - */ -static rsRetVal -copyActive(batch_t *pBatch) +/* Commit all active transactions in *DIRECT mode* */ +void +actionCommitAllDirect(wti_t *__restrict__ const pWti) { - sbool *active; - DEFiRet; + int i; + action_t *pAction; - CHKmalloc(active = malloc(batchNumMsgs(pBatch) * sizeof(sbool))); - if(pBatch->active == NULL) - memset(active, 1, batchNumMsgs(pBatch)); - else - memcpy(active, pBatch->active, batchNumMsgs(pBatch)); - pBatch->active = active; -finalize_it: - RETiRet; + for(i = 0 ; i < iActionNbr ; ++i) { + pAction = pWti->actWrkrInfo[i].pAction; + if(pAction == NULL) + continue; + DBGPRINTF("actionCommitAll: action %d, state %u, nbr to commit %d " + "isTransactional %d\n", + i, getActionStateByNbr(pWti, i), pWti->actWrkrInfo->p.tx.currIParam, + pAction->isTransactional); + if(pAction->pQueue->qType == QUEUETYPE_DIRECT) + actionCommit(pAction, pWti); + } } -/* The following function prepares a batch for processing, that it is - * reinitializes batch states, generates strings and does everything else - * that needs to be done in order to make the batch ready for submission to - * the actual output module. Note that we look at the precomputed - * filter OK condition and process only those messages, that actually matched - * the filter. - * rgerhards, 2010-06-14 +/* process a single message. This is both called if we run from the + * cosumer side of an action queue as well as directly from the main + * queue thread if the action queue is set to "direct". */ -static inline rsRetVal -prepareBatch(action_t *pAction, batch_t *pBatch, sbool **activeSave, int *bMustRestoreActivePtr) +static rsRetVal +processMsgMain(action_t *__restrict__ const pAction, + wti_t *__restrict__ const pWti, + msg_t *__restrict__ const pMsg, + struct syslogTime *ttNow) { - int i; - batch_obj_t *pElem; - struct syslogTime ttNow; DEFiRet; - /* indicate we have not yet read the date */ - ttNow.year = 0; - - pBatch->iDoneUpTo = 0; - for(i = 0 ; i < batchNumMsgs(pBatch) && !*(pBatch->pbShutdownImmediate) ; ++i) { - pElem = &(pBatch->pElem[i]); - if(batchIsValidElem(pBatch, i)) { - pBatch->eltState[i] = BATCH_STATE_RDY; - if(prepareDoActionParams(pAction, pElem, &ttNow) != RS_RET_OK) { - /* make sure we have our copy of "active" array */ - if(!*bMustRestoreActivePtr) { - *activeSave = pBatch->active; - copyActive(pBatch); - } - pBatch->active[i] = RSFALSE; - } - } + if(pAction->bExecWhenPrevSusp && !pWti->execState.bPrevWasSuspended) { + DBGPRINTF("action %d: NOT executing, as previous action was " + "not suspended\n", pAction->iActionNbr); + FINALIZE; } - RETiRet; -} - -/* receive a batch and process it. This includes retry handling. - * rgerhards, 2009-05-12 - */ -static inline rsRetVal -processAction(action_t *pAction, batch_t *pBatch) -{ - DEFiRet; + iRet = prepareDoActionParams(pAction, pWti, pMsg, ttNow); - assert(pBatch != NULL); - CHKiRet(submitBatch(pAction, pBatch, pBatch->nElem)); - iRet = finishBatch(pAction, pBatch); + if(pAction->isTransactional) { + pWti->actWrkrInfo[pAction->iActionNbr].pAction = pAction; + DBGPRINTF("action %d is transactional - executing in commit phase\n", pAction->iActionNbr); + actionPrepare(pAction, pWti); + iRet = getReturnCode(pAction, pWti); + FINALIZE; + } + iRet = actionProcessMessage(pAction, + pWti->actWrkrInfo[pAction->iActionNbr].p.nontx.actParams, + pWti); + if(pAction->bNeedReleaseBatch) + releaseDoActionParams(pAction, pWti); finalize_it: + if(iRet == RS_RET_OK) { + if(pWti->execState.bDoAutoCommit) + iRet = actionCommit(pAction, pWti); + } + pWti->execState.bPrevWasSuspended = (iRet == RS_RET_SUSPENDED || iRet == RS_RET_ACTION_FAILED); RETiRet; } - -#pragma GCC diagnostic ignored "-Wempty-body" -/* receive an array of to-process user pointers and submit them - * for processing. - * rgerhards, 2009-04-22 +/* This entry point is called by the ACTION queue (not main queue!) */ static rsRetVal -processBatchMain(action_t *pAction, batch_t *pBatch, int *pbShutdownImmediate) +processBatchMain(void *__restrict__ const pVoid, + batch_t *__restrict__ const pBatch, + wti_t *__restrict__ const pWti) { - int *pbShutdownImmdtSave; - sbool *activeSave; - int bMustRestoreActivePtr = 0; - rsRetVal localRet; + action_t *__restrict__ const pAction = (action_t*__restrict__ const) pVoid; + int i; + struct syslogTime ttNow; DEFiRet; - assert(pBatch != NULL); - - if(pbShutdownImmediate != NULL) { - pbShutdownImmdtSave = pBatch->pbShutdownImmediate; - pBatch->pbShutdownImmediate = pbShutdownImmediate; - } - CHKiRet(prepareBatch(pAction, pBatch, &activeSave, &bMustRestoreActivePtr)); - - /* We now must guard the output module against execution by multiple threads. The - * plugin interface specifies that output modules must not be thread-safe (except - * if they notify us they are - functionality not yet implemented...). - * rgerhards, 2008-01-30 - */ - d_pthread_mutex_lock(&pAction->mutActExec); - pthread_cleanup_push(mutexCancelCleanup, &pAction->mutActExec); - - iRet = processAction(pAction, pBatch); - - pthread_cleanup_pop(1); /* unlock mutex */ - - /* even if processAction failed, we need to release the batch (else we - * have a memory leak). So we do this first, and then check if we need to - * return an error code. If so, the code from processAction has priority. - * rgerhards, 2010-12-17 - */ - localRet = releaseBatch(pAction, pBatch); + wtiResetExecState(pWti, pBatch); + /* indicate we have not yet read the date */ + ttNow.year = 0; - if(iRet == RS_RET_OK) - iRet = localRet; - - if(bMustRestoreActivePtr) { - free(pBatch->active); - pBatch->active = activeSave; + for(i = 0 ; i < batchNumMsgs(pBatch) && !*pWti->pbShutdownImmediate ; ++i) { + if(batchIsValidElem(pBatch, i)) { + iRet = processMsgMain(pAction, pWti, pBatch->pElem[i].pMsg, &ttNow); + batchSetElemState(pBatch, i, BATCH_STATE_COMM); + } } -finalize_it: - if(pbShutdownImmediate != NULL) - pBatch->pbShutdownImmediate = pbShutdownImmdtSave; + iRet = actionCommit(pAction, pWti); RETiRet; } -#pragma GCC diagnostic warning "-Wempty-body" -/* call the HUP handler for a given action, if such a handler is defined. The - * action mutex is locked, because the HUP handler most probably needs to modify - * some internal state information. - * rgerhards, 2008-10-22 +/* call the HUP handler for a given action, if such a handler is defined. + * Note that the action must be able to service HUP requests concurrently + * to any current doAction() processing. */ -#pragma GCC diagnostic ignored "-Wempty-body" rsRetVal -actionCallHUPHdlr(action_t *pAction) +actionCallHUPHdlr(action_t * const pAction) { DEFiRet; ASSERT(pAction != NULL); DBGPRINTF("Action %p checks HUP hdlr: %p\n", pAction, pAction->pMod->doHUP); - if(pAction->pMod->doHUP == NULL) { - FINALIZE; /* no HUP handler, so we are done ;) */ + if(pAction->pMod->doHUP != NULL) { + CHKiRet(pAction->pMod->doHUP(pAction->pModData)); } - d_pthread_mutex_lock(&pAction->mutActExec); - pthread_cleanup_push(mutexCancelCleanup, &pAction->mutActExec); - CHKiRet(pAction->pMod->doHUP(pAction->pModData)); - pthread_cleanup_pop(1); /* unlock mutex */ - finalize_it: RETiRet; } -#pragma GCC diagnostic warning "-Wempty-body" /* set the action message queue mode @@ -1376,27 +1380,28 @@ static rsRetVal setActionQueType(void __attribute__((unused)) *pVal, uchar *pszT /* This submits the message to the action queue in case we do NOT need to handle repeat * message processing. That case permits us to gain lots of freedom during processing - * and thus speed. This is also utilized to submit messages in complex case once + * and thus speed. This is also utilized to submit messages in more complex cases once * the complex logic has been applied ;) * rgerhards, 2010-06-08 */ -static inline rsRetVal -doSubmitToActionQ(action_t *pAction, msg_t *pMsg) +static rsRetVal +doSubmitToActionQ(action_t * const pAction, wti_t * const pWti, msg_t *pMsg) { + struct syslogTime ttNow; // TODO: think if we can buffer this in pWti DEFiRet; - if(pAction->eState == ACT_STATE_DIED) { - DBGPRINTF("action %p died, do NOT execute\n", pAction); - FINALIZE; - } + DBGPRINTF("Called action, logging to %s\n", module.GetStateName(pAction->pMod)); STATSCOUNTER_INC(pAction->ctrProcessed, pAction->mutCtrProcessed); - if(pAction->pQueue->qType == QUEUETYPE_DIRECT) - iRet = qqueueEnqMsgDirect(pAction->pQueue, MsgAddRef(pMsg)); - else + if(pAction->pQueue->qType == QUEUETYPE_DIRECT) { + ttNow.year = 0; + iRet = processMsgMain(pAction, pWti, pMsg, &ttNow); + } else {/* in this case, we do single submits to the queue. + * TODO: optimize this, we may do at least a multi-submit! + */ iRet = qqueueEnqMsg(pAction->pQueue, eFLOWCTL_NO_DELAY, MsgAddRef(pMsg)); + } -finalize_it: RETiRet; } @@ -1409,7 +1414,7 @@ finalize_it: * be filtered out before calling us (what is done currently!). */ rsRetVal -actionWriteToAction(action_t *pAction, msg_t *pMsg) +actionWriteToAction(action_t * const pAction, msg_t *pMsg, wti_t * const pWti) { DEFiRet; @@ -1464,44 +1469,46 @@ actionWriteToAction(action_t *pAction, msg_t *pMsg) /* When we reach this point, we have a valid, non-disabled action. * So let's enqueue our message for execution. -- rgerhards, 2007-07-24 */ - iRet = doSubmitToActionQ(pAction, pMsg); + iRet = doSubmitToActionQ(pAction, pWti, pMsg); finalize_it: RETiRet; } -/* helper to actonCallAction, mostly needed because of this damn - * pthread_cleanup_push() POSIX macro... +/* Call configured action, most complex case with all features supported (and thus slow). + * rgerhards, 2010-06-08 */ -static inline rsRetVal -doActionCallAction(action_t *pAction, batch_t *pBatch, int idxBtch) +#pragma GCC diagnostic ignored "-Wempty-body" +static rsRetVal +doSubmitToActionQComplex(action_t * const pAction, wti_t * const pWti, msg_t *pMsg) { - msg_t *pMsg; DEFiRet; - pMsg = pBatch->pElem[idxBtch].pMsg; + d_pthread_mutex_lock(&pAction->mutAction); + pthread_cleanup_push(mutexCancelCleanup, &pAction->mutAction); + DBGPRINTF("Called action %p (complex case), logging to %s\n", + pAction, module.GetStateName(pAction->pMod)); + pAction->tActNow = -1; /* we do not yet know our current time (clear prev. value) */ + // TODO: can we optimize the "now" handling again (was batch, I guess...)? /* don't output marks to recently written outputs */ - if(pAction->bWriteAllMarkMsgs == RSFALSE + if(pAction->bWriteAllMarkMsgs == 0 && (pMsg->msgFlags & MARK) && (getActNow(pAction) - pAction->f_time) < MarkInterval / 2) { ABORT_FINALIZE(RS_RET_OK); } /* call the output driver */ - iRet = actionWriteToAction(pAction, pMsg); + iRet = actionWriteToAction(pAction, pMsg, pWti); finalize_it: - /* we need to update the batch to handle failover processing correctly */ - if(iRet == RS_RET_OK) { - pBatch->pElem[idxBtch].bPrevWasSuspended = 0; - } else if(iRet == RS_RET_ACTION_FAILED) { - pBatch->pElem[idxBtch].bPrevWasSuspended = 1; - } + d_pthread_mutex_unlock(&pAction->mutAction); + pthread_cleanup_pop(0); /* remove mutex cleanup handler */ RETiRet; } +#pragma GCC diagnostic warning "-Wempty-body" /* helper to activateActions, it activates a specific action. @@ -1509,7 +1516,7 @@ finalize_it: DEFFUNC_llExecFunc(doActivateActions) { rsRetVal localRet; - action_t *pThis = (action_t*) pData; + action_t * const pThis = (action_t*) pData; BEGINfunc localRet = qqueueStart(pThis->pQueue); if(localRet != RS_RET_OK) { @@ -1550,201 +1557,48 @@ activateActions(void) * rgerhards, 2010-06-08 */ static rsRetVal -doSubmitToActionQNotAllMarkBatch(action_t *pAction, batch_t *pBatch) +doSubmitToActionQNotAllMark(action_t * const pAction, wti_t * const pWti, msg_t * const pMsg) { - time_t now = 0; + int doProcess = 1; time_t lastAct; - int i; - sbool *activeSave; - DEFiRet; - - activeSave = pBatch->active; - copyActive(pBatch); - - for(i = 0 ; i < batchNumMsgs(pBatch) ; ++i) { - if((pBatch->eltState[i] == BATCH_STATE_DISC) || !pBatch->active[i]) - continue; - if(now == 0) { - now = datetime.GetTime(NULL); /* good time call - the only one done */ - } - /* CAS loop, we write back a bit early, but that's OK... */ - /* we use reception time, not dequeue time - this is considered more appropriate and - * also faster ;) -- rgerhards, 2008-09-17 */ - do { - lastAct = pAction->f_time; - if(pBatch->pElem[i].pMsg->msgFlags & MARK) { - if((now - lastAct) < MarkInterval / 2) { - pBatch->active[i] = 0; - DBGPRINTF("batch item %d: action was recently called, ignoring " - "mark message\n", i); - break; /* do not update timestamp for non-written mark messages */ - } - } - } while(ATOMIC_CAS_time_t(&pAction->f_time, lastAct, - pBatch->pElem[i].pMsg->ttGenTime, &pAction->mutCAS) == 0); - if(pBatch->active[i]) { - DBGPRINTF("Called action(NotAllMark), processing batch[%d] via '%s'\n", - i, module.GetStateName(pAction->pMod)); - } - } - - iRet = doSubmitToActionQBatch(pAction, pBatch); - - free(pBatch->active); - pBatch->active = activeSave; - - RETiRet; -} - -static inline void -countStatsBatchEnq(action_t *pAction, batch_t *pBatch) -{ - int i; - for(i = 0 ; i < batchNumMsgs(pBatch) && !*(pBatch->pbShutdownImmediate) ; ++i) { - if( batchIsValidElem(pBatch, i)) { - STATSCOUNTER_INC(pAction->ctrProcessed, pAction->mutCtrProcessed); - } - } -} - - -/* enqueue a batch in direct mode. We have put this into its own function just to avoid - * cluttering the actual submit function. - * rgerhards, 2011-06-16 - */ -static inline rsRetVal -doQueueEnqObjDirectBatch(action_t *pAction, batch_t *pBatch) -{ - sbool bNeedSubmit; - sbool *activeSave; - int i; DEFiRet; - activeSave = pBatch->active; - copyActive(pBatch); - - /* note: for direct mode, we need to adjust the filter property. For non-direct - * this is not necessary, because in that case we enqueue only what actually needs - * to be processed. + /* TODO: think about the whole logic. If messages come in out of order, things + * tend to become a bit unreliable. On the other hand, this only happens if we have + * very high traffic, in which this use case here is not really affected (as the + * MarkInterval is pretty corase). */ - if(pAction->bExecWhenPrevSusp) { - bNeedSubmit = 0; - for(i = 0 ; i < batchNumMsgs(pBatch) && !*(pBatch->pbShutdownImmediate) ; ++i) { - if(!pBatch->pElem[i].bPrevWasSuspended) { - DBGPRINTF("action enq stage: change active to 0 due to " - "failover case in elem %d\n", i); - pBatch->active[i] = 0; - } - if(batchIsValidElem(pBatch, i)) { - STATSCOUNTER_INC(pAction->ctrProcessed, pAction->mutCtrProcessed); - bNeedSubmit = 1; - } - DBGPRINTF("action %p[%d]: valid:%d state:%d execWhenPrev:%d prevWasSusp:%d\n", - pAction, i, batchIsValidElem(pBatch, i), pBatch->eltState[i], - pAction->bExecWhenPrevSusp, pBatch->pElem[i].bPrevWasSuspended); - } - if(bNeedSubmit) { - /* note: stats were already computed above */ - iRet = qqueueEnqObjDirectBatch(pAction->pQueue, pBatch); - } else { - DBGPRINTF("no need to submit batch, all invalid\n"); - } - } else { - if(GatherStats) - countStatsBatchEnq(pAction, pBatch); - iRet = qqueueEnqObjDirectBatch(pAction->pQueue, pBatch); - } - - free(pBatch->active); - pBatch->active = activeSave; - RETiRet; -} - -/* This submits the message to the action queue in case we do NOT need to handle repeat - * message processing. That case permits us to gain lots of freedom during processing - * and thus speed. - * rgerhards, 2010-06-08 - */ -static rsRetVal -doSubmitToActionQBatch(action_t *pAction, batch_t *pBatch) -{ - int i; - DEFiRet; - - DBGPRINTF("Called action(Batch), logging to %s\n", module.GetStateName(pAction->pMod)); - - if(pAction->pQueue->qType == QUEUETYPE_DIRECT) { - iRet = doQueueEnqObjDirectBatch(pAction, pBatch); - } else {/* in this case, we do single submits to the queue. - * TODO: optimize this, we may do at least a multi-submit! - */ - for(i = 0 ; i < batchNumMsgs(pBatch) && !*(pBatch->pbShutdownImmediate) ; ++i) { - DBGPRINTF("action %p: valid:%d state:%d execWhenPrev:%d prevWasSusp:%d\n", - pAction, batchIsValidElem(pBatch, i), pBatch->eltState[i], - pAction->bExecWhenPrevSusp, pBatch->pElem[i].bPrevWasSuspended); - if( batchIsValidElem(pBatch, i) - && (pAction->bExecWhenPrevSusp == 0 || pBatch->pElem[i].bPrevWasSuspended == 1)) { - doSubmitToActionQ(pAction, pBatch->pElem[i].pMsg); + /* CAS loop, we write back a bit early, but that's OK... */ + /* we use reception time, not dequeue time - this is considered more appropriate and + * also faster ;) -- rgerhards, 2008-09-17 */ + do { + lastAct = pAction->f_time; + if(pMsg->msgFlags & MARK) { + if((pMsg->ttGenTime - lastAct) < MarkInterval / 2) { + doProcess = 0; + DBGPRINTF("action was recently called, ignoring mark message\n"); + break; /* do not update timestamp for non-written mark messages */ } } - } + } while(ATOMIC_CAS_time_t(&pAction->f_time, lastAct, + pMsg->ttGenTime, &pAction->mutCAS) == 0); - RETiRet; -} - - - -/* Helper to submit a batch of actions to the engine. Note that we have rather - * complicated processing here, so we need to do this one message after another. - * rgerhards, 2010-06-23 - */ -static inline rsRetVal -helperSubmitToActionQComplexBatch(action_t *pAction, batch_t *pBatch) -{ - int i; - DEFiRet; - - DBGPRINTF("Called action %p (complex case), logging to %s\n", - pAction, module.GetStateName(pAction->pMod)); - for(i = 0 ; i < batchNumMsgs(pBatch) && !*(pBatch->pbShutdownImmediate) ; ++i) { - DBGPRINTF("action %p: valid:%d state:%d execWhenPrev:%d prevWasSusp:%d\n", - pAction, batchIsValidElem(pBatch, i), pBatch->eltState[i], - pAction->bExecWhenPrevSusp, pBatch->pElem[i].bPrevWasSuspended); - if( batchIsValidElem(pBatch, i) - && ((pAction->bExecWhenPrevSusp == 0) || pBatch->pElem[i].bPrevWasSuspended) ) { - doActionCallAction(pAction, pBatch, i); - } + if(doProcess) { + DBGPRINTF("Called action(NotAllMark), processing via '%s'\n", + module.GetStateName(pAction->pMod)); + iRet = doSubmitToActionQ(pAction, pWti, pMsg); } RETiRet; } -/* Call configured action, most complex case with all features supported (and thus slow). - * rgerhards, 2010-06-08 - */ -#pragma GCC diagnostic ignored "-Wempty-body" -static rsRetVal -doSubmitToActionQComplexBatch(action_t *pAction, batch_t *pBatch) -{ - DEFiRet; - - d_pthread_mutex_lock(&pAction->mutAction); - pthread_cleanup_push(mutexCancelCleanup, &pAction->mutAction); - iRet = helperSubmitToActionQComplexBatch(pAction, pBatch); - d_pthread_mutex_unlock(&pAction->mutAction); - pthread_cleanup_pop(0); /* remove mutex cleanup handler */ - - RETiRet; -} -#pragma GCC diagnostic warning "-Wempty-body" - /* apply all params from param block to action. This supports the v6 config system. * Defaults must have been set appropriately during action construct! * rgerhards, 2011-08-01 */ static rsRetVal -actionApplyCnfParam(action_t *pAction, struct cnfparamvals *pvals) +actionApplyCnfParam(action_t * const pAction, struct cnfparamvals * const pvals) { int i; @@ -1769,6 +1623,10 @@ actionApplyCnfParam(action_t *pAction, struct cnfparamvals *pvals) pAction->bRepMsgHasMsg = pvals[i].val.d.n; } else if(!strcmp(pblk.descr[i].name, "action.resumeretrycount")) { pAction->iResumeRetryCount = pvals[i].val.d.n; + } else if(!strcmp(pblk.descr[i].name, "action.reportsuspension")) { + pAction->bReportSuspension = (int) pvals[i].val.d.n; + } else if(!strcmp(pblk.descr[i].name, "action.reportsuspensioncontinuation")) { + pAction->bReportSuspensionCont = (int) pvals[i].val.d.n; } else if(!strcmp(pblk.descr[i].name, "action.resumeinterval")) { pAction->iResumeInterval = pvals[i].val.d.n; } else { @@ -1788,7 +1646,7 @@ actionApplyCnfParam(action_t *pAction, struct cnfparamvals *pvals) rsRetVal addAction(action_t **ppAction, modInfo_t *pMod, void *pModData, omodStringRequest_t *pOMSR, struct cnfparamvals *actParams, - struct cnfparamvals *queueParams, int bSuspended) + struct nvlst * const lst) { DEFiRet; int i; @@ -1817,7 +1675,7 @@ addAction(action_t **ppAction, modInfo_t *pMod, void *pModData, pAction->bRepMsgHasMsg = cs.bActionRepMsgHasMsg; cs.iActExecEveryNthOccur = 0; /* auto-reset */ cs.iActExecEveryNthOccurTO = 0; /* auto-reset */ - cs.bActionWriteAllMarkMsgs = RSFALSE; /* auto-reset */ + cs.bActionWriteAllMarkMsgs = 1; /* auto-reset */ cs.pszActionName = NULL; /* free again! */ } else { actionApplyCnfParam(pAction, actParams); @@ -1831,43 +1689,50 @@ addAction(action_t **ppAction, modInfo_t *pMod, void *pModData, * the discard action, which does not require a string. -- rgerhards, 2007-07-30 */ if(pAction->iNumTpls > 0) { - /* we first need to create the template pointer array */ + /* we first need to create the template arrays */ CHKmalloc(pAction->ppTpl = (struct template **)calloc(pAction->iNumTpls, sizeof(struct template *))); + CHKmalloc(pAction->peParamPassing = (paramPassing_t*)calloc(pAction->iNumTpls, sizeof(paramPassing_t))); } + pAction->bUsesMsgPassingMode = 0; + pAction->bNeedReleaseBatch = 0; for(i = 0 ; i < pAction->iNumTpls ; ++i) { CHKiRet(OMSRgetEntry(pOMSR, i, &pTplName, &iTplOpts)); /* Ok, we got everything, so it now is time to look up the template * (Hint: templates MUST be defined before they are used!) */ - if( !(iTplOpts & OMSR_TPL_AS_MSG) - && (pAction->ppTpl[i] = - tplFind(ourConf, (char*)pTplName, strlen((char*)pTplName))) == NULL) { - snprintf(errMsg, sizeof(errMsg) / sizeof(char), - " Could not find template '%s' - action disabled", - pTplName); - errno = 0; - errmsg.LogError(0, RS_RET_NOT_FOUND, "%s", errMsg); - ABORT_FINALIZE(RS_RET_NOT_FOUND); - } - /* check required template options */ - if( (iTplOpts & OMSR_RQD_TPL_OPT_SQL) - && (pAction->ppTpl[i]->optFormatEscape == 0)) { - errno = 0; - errmsg.LogError(0, RS_RET_RQD_TPLOPT_MISSING, "Action disabled. To use this action, you have to specify " - "the SQL or stdSQL option in your template!\n"); - ABORT_FINALIZE(RS_RET_RQD_TPLOPT_MISSING); + if(!(iTplOpts & OMSR_TPL_AS_MSG)) { + if((pAction->ppTpl[i] = + tplFind(ourConf, (char*)pTplName, strlen((char*)pTplName))) == NULL) { + snprintf(errMsg, sizeof(errMsg) / sizeof(char), + " Could not find template '%s' - action disabled", + pTplName); + errno = 0; + errmsg.LogError(0, RS_RET_NOT_FOUND, "%s", errMsg); + ABORT_FINALIZE(RS_RET_NOT_FOUND); + } + /* check required template options */ + if( (iTplOpts & OMSR_RQD_TPL_OPT_SQL) + && (pAction->ppTpl[i]->optFormatEscape == 0)) { + errno = 0; + errmsg.LogError(0, RS_RET_RQD_TPLOPT_MISSING, "Action disabled. To use this action, you have to specify " + "the SQL or stdSQL option in your template!\n"); + ABORT_FINALIZE(RS_RET_RQD_TPLOPT_MISSING); + } } /* set parameter-passing mode */ if(iTplOpts & OMSR_TPL_AS_ARRAY) { - pAction->eParamPassing = ACT_ARRAY_PASSING; + pAction->peParamPassing[i] = ACT_ARRAY_PASSING; + pAction->bNeedReleaseBatch = 1; } else if(iTplOpts & OMSR_TPL_AS_MSG) { - pAction->eParamPassing = ACT_MSG_PASSING; + pAction->peParamPassing[i] = ACT_MSG_PASSING; + pAction->bUsesMsgPassingMode = 1; } else if(iTplOpts & OMSR_TPL_AS_JSON) { - pAction->eParamPassing = ACT_JSON_PASSING; + pAction->peParamPassing[i] = ACT_JSON_PASSING; + pAction->bNeedReleaseBatch = 1; } else { - pAction->eParamPassing = ACT_STRING_PASSING; + pAction->peParamPassing[i] = ACT_STRING_PASSING; } DBGPRINTF("template: '%s' assigned\n", pTplName); @@ -1875,15 +1740,10 @@ addAction(action_t **ppAction, modInfo_t *pMod, void *pModData, pAction->pMod = pMod; pAction->pModData = pModData; - /* check if the module is compatible with select features (currently no such features exist) */ - pAction->eState = ACT_STATE_RDY; /* action is enabled */ - - if(bSuspended) - actionSuspend(pAction); - CHKiRet(actionConstructFinalize(pAction, queueParams)); + CHKiRet(actionConstructFinalize(pAction, lst)); - /* TODO: if we exit here, we have a memory leak... */ + /* TODO: if we exit here, we have a (quite acceptable...) memory leak */ *ppAction = pAction; /* finally store the action pointer */ @@ -1919,7 +1779,7 @@ resetConfigVariables(uchar __attribute__((unused)) *pp, void __attribute__((unus static inline void initConfigVariables(void) { - cs.bActionWriteAllMarkMsgs = RSFALSE; + cs.bActionWriteAllMarkMsgs = 1; cs.glbliActionResumeRetryCount = 0; cs.bActExecWhenPrevSusp = 0; cs.iActExecOnceInterval = 0; @@ -1940,47 +1800,34 @@ rsRetVal actionNewInst(struct nvlst *lst, action_t **ppAction) { struct cnfparamvals *paramvals; - struct cnfparamvals *queueParams; modInfo_t *pMod; uchar *cnfModName = NULL; omodStringRequest_t *pOMSR; void *pModData; action_t *pAction; - int typeIdx; DEFiRet; paramvals = nvlstGetParams(lst, &pblk, NULL); if(paramvals == NULL) { - ABORT_FINALIZE(RS_RET_ERR); + ABORT_FINALIZE(RS_RET_PARAM_ERROR); } dbgprintf("action param blk after actionNewInst:\n"); cnfparamsPrint(&pblk, paramvals); - typeIdx = cnfparamGetIdx(&pblk, "type"); - if(paramvals[typeIdx].bUsed == 0) { - errmsg.LogError(0, RS_RET_CONF_RQRD_PARAM_MISSING, "action type missing"); - ABORT_FINALIZE(RS_RET_CONF_RQRD_PARAM_MISSING); // TODO: move this into rainerscript handlers - } cnfModName = (uchar*)es_str2cstr(paramvals[cnfparamGetIdx(&pblk, ("type"))].val.d.estr, NULL); if((pMod = module.FindWithCnfName(loadConf, cnfModName, eMOD_OUT)) == NULL) { errmsg.LogError(0, RS_RET_MOD_UNKNOWN, "module name '%s' is unknown", cnfModName); ABORT_FINALIZE(RS_RET_MOD_UNKNOWN); } - iRet = pMod->mod.om.newActInst(cnfModName, lst, &pModData, &pOMSR); - // TODO: check if RS_RET_SUSPENDED is still valid in v6! - if(iRet != RS_RET_OK && iRet != RS_RET_SUSPENDED) { - FINALIZE; /* iRet is already set to error state */ - } - - qqueueDoCnfParams(lst, &queueParams); + CHKiRet(pMod->mod.om.newActInst(cnfModName, lst, &pModData, &pOMSR)); - if((iRet = addAction(&pAction, pMod, pModData, pOMSR, paramvals, queueParams, - (iRet == RS_RET_SUSPENDED)? 1 : 0)) == RS_RET_OK) { + if((iRet = addAction(&pAction, pMod, pModData, pOMSR, paramvals, lst)) == RS_RET_OK) { /* check if the module is compatible with select features * (currently no such features exist) */ - pAction->eState = ACT_STATE_RDY; /* action is enabled */ loadConf->actions.nbrActions++; /* one more active action! */ + *ppAction = pAction; + } else { + // TODO: cleanup } - *ppAction = pAction; finalize_it: free(cnfModName); @@ -1988,8 +1835,6 @@ finalize_it: RETiRet; } -/* TODO: we are not yet a real object, the ClassInit here just looks like it is.. - */ rsRetVal actionClassInit(void) { DEFiRet; @@ -4,7 +4,7 @@ * File begun on 2007-08-06 by RGerhards (extracted from syslogd.c, which * was under BSD license at the time of rsyslog fork) * - * Copyright 2007-2012 Adiscon GmbH. + * Copyright 2007-2013 Adiscon GmbH. * * This file is part of rsyslog. * @@ -28,22 +28,12 @@ #include "syslogd-types.h" #include "queue.h" -/* external data - this is to be removed when we change the action - * object interface (will happen some time..., at latest when the - * config file format is changed). -- rgerhards, 2008-01-28 - */ +/* external data */ extern int glbliActionResumeRetryCount; +extern int bActionReportSuspension; +extern int bActionReportSuspensionCont; -typedef enum { - ACT_STATE_DIED = 0, /* action permanently failed and now disabled - MUST BE ZERO! */ - ACT_STATE_RDY = 1, /* action ready, waiting for new transaction */ - ACT_STATE_ITX = 2, /* transaction active, waiting for new data or commit */ - ACT_STATE_COMM = 3, /* transaction finished (a transient state) */ - ACT_STATE_RTRY = 4, /* failure occured, trying to restablish ready state */ - ACT_STATE_SUSP = 5 /* suspended due to failure (return fail until timeout expired) */ -} action_state_t; - /* the following struct defines the action object data structure */ struct action_s { @@ -51,16 +41,18 @@ struct action_s { time_t tActNow; /* the current time for an action execution. Initially set to -1 and populated on an as-needed basis. This is a performance optimization. */ time_t tLastExec; /* time this action was last executed */ + int iActionNbr; /* this action's number (ID) */ sbool bExecWhenPrevSusp;/* execute only when previous action is suspended? */ sbool bWriteAllMarkMsgs;/* should all mark msgs be written (not matter how recent the action was executed)? */ - int iSecsExecOnceInterval; /* if non-zero, minimum seconds to wait until action is executed again */ - action_state_t eState; /* current state of action */ + sbool bReportSuspension;/* should suspension (and reactivation) of the action reported */ + sbool bReportSuspensionCont; sbool bHadAutoCommit; /* did an auto-commit happen during doAction()? */ + sbool bDisabled; + sbool isTransactional; + int iSecsExecOnceInterval; /* if non-zero, minimum seconds to wait until action is executed again */ time_t ttResumeRtry; /* when is it time to retry the resume? */ - int iResumeOKinRow; /* number of times in a row that resume said OK with an immediate failure following */ int iResumeInterval;/* resume interval for this action */ int iResumeRetryCount;/* how often shall we retry a suspended action? (-1 --> eternal) */ - int iNbrResRtry; /* number of retries since last suspend */ int iNbrNoExec; /* number of matches that did not yet yield to an exec */ int iExecEveryNthOccur;/* execute this action only every n-th occurence (with n=0,1 -> always) */ int iExecEveryNthOccurTO;/* timeout for n-th occurence feature */ @@ -68,40 +60,46 @@ struct action_s { struct modInfo_s *pMod;/* pointer to output module handling this selector */ void *pModData; /* pointer to module data - content is module-specific */ sbool bRepMsgHasMsg; /* "message repeated..." has msg fragment in it (0-no, 1-yes) */ - rsRetVal (*submitToActQ)(action_t *, batch_t *);/* function submit message to action queue */ + rsRetVal (*submitToActQ)(action_t *, wti_t*, msg_t*);/* function submit message to action queue */ rsRetVal (*qConstruct)(struct queue_s *pThis); - enum { ACT_STRING_PASSING = 0, ACT_ARRAY_PASSING = 1, ACT_MSG_PASSING = 2, - ACT_JSON_PASSING = 3} - eParamPassing; /* mode of parameter passing to action */ + sbool bUsesMsgPassingMode; + sbool bNeedReleaseBatch; /* do we need to release batch ressources? Depends on ParamPassig modes... */ int iNumTpls; /* number of array entries for template element below */ struct template **ppTpl;/* array of template to use - strings must be passed to doAction * in this order. */ + paramPassing_t *peParamPassing; /* mode of parameter passing to action for that template */ qqueue_t *pQueue; /* action queue */ pthread_mutex_t mutAction; /* primary action mutex */ - pthread_mutex_t mutActExec; /* mutex to guard actual execution of doAction for single-threaded modules */ - uchar *pszName; /* action name (for documentation) */ - DEF_ATOMIC_HELPER_MUT(mutCAS); + uchar *pszName; /* action name */ + DEF_ATOMIC_HELPER_MUT(mutCAS) /* for statistics subsystem */ statsobj_t *statsobj; - STATSCOUNTER_DEF(ctrProcessed, mutCtrProcessed); - STATSCOUNTER_DEF(ctrFail, mutCtrFail); + STATSCOUNTER_DEF(ctrProcessed, mutCtrProcessed) + STATSCOUNTER_DEF(ctrFail, mutCtrFail) + STATSCOUNTER_DEF(ctrSuspend, mutCtrSuspend) + STATSCOUNTER_DEF(ctrSuspendDuration, mutCtrSuspendDuration) + STATSCOUNTER_DEF(ctrResume, mutCtrResume) }; /* function prototypes */ rsRetVal actionConstruct(action_t **ppThis); -rsRetVal actionConstructFinalize(action_t *pThis, struct cnfparamvals *queueParams); +rsRetVal actionConstructFinalize(action_t *pThis, struct nvlst *lst); rsRetVal actionDestruct(action_t *pThis); -rsRetVal actionDbgPrint(action_t *pThis); +//rsRetVal actionDbgPrint(action_t *pThis); rsRetVal actionSetGlobalResumeInterval(int iNewVal); rsRetVal actionDoAction(action_t *pAction); -rsRetVal actionWriteToAction(action_t *pAction, msg_t *pMsg); +rsRetVal actionWriteToAction(action_t *pAction, msg_t *pMsg, wti_t*); rsRetVal actionCallHUPHdlr(action_t *pAction); rsRetVal actionClassInit(void); -rsRetVal addAction(action_t **ppAction, modInfo_t *pMod, void *pModData, omodStringRequest_t *pOMSR, struct cnfparamvals *actParams, struct cnfparamvals *queueParams, int bSuspended); +rsRetVal addAction(action_t **ppAction, modInfo_t *pMod, void *pModData, omodStringRequest_t *pOMSR, struct cnfparamvals *actParams, struct nvlst *lst); rsRetVal activateActions(void); rsRetVal actionNewInst(struct nvlst *lst, action_t **ppAction); rsRetVal actionProcessCnf(struct cnfobj *o); +void actionCommitAllDirect(wti_t *pWti); + +/* external data */ +extern int iActionNbr; #endif /* #ifndef ACTION_H_INCLUDED */ diff --git a/compat/Makefile.in b/compat/Makefile.in index 35efead..d2ff2ac 100644 --- a/compat/Makefile.in +++ b/compat/Makefile.in @@ -126,7 +126,6 @@ GREP = @GREP@ GSS_LIBS = @GSS_LIBS@ GUARDTIME_CFLAGS = @GUARDTIME_CFLAGS@ GUARDTIME_LIBS = @GUARDTIME_LIBS@ -HAVE_LIBGCRYPT_CONFIG = @HAVE_LIBGCRYPT_CONFIG@ HAVE_MYSQL_CONFIG = @HAVE_MYSQL_CONFIG@ HAVE_ORACLE_CONFIG = @HAVE_ORACLE_CONFIG@ HAVE_PGSQL_CONFIG = @HAVE_PGSQL_CONFIG@ @@ -147,14 +146,15 @@ LEXLIB = @LEXLIB@ LEX_OUTPUT_ROOT = @LEX_OUTPUT_ROOT@ LIBDBI_CFLAGS = @LIBDBI_CFLAGS@ LIBDBI_LIBS = @LIBDBI_LIBS@ -LIBEE_CFLAGS = @LIBEE_CFLAGS@ -LIBEE_LIBS = @LIBEE_LIBS@ LIBESTR_CFLAGS = @LIBESTR_CFLAGS@ LIBESTR_LIBS = @LIBESTR_LIBS@ LIBGCRYPT_CFLAGS = @LIBGCRYPT_CFLAGS@ +LIBGCRYPT_CONFIG = @LIBGCRYPT_CONFIG@ LIBGCRYPT_LIBS = @LIBGCRYPT_LIBS@ LIBLOGGING_CFLAGS = @LIBLOGGING_CFLAGS@ LIBLOGGING_LIBS = @LIBLOGGING_LIBS@ +LIBLOGGING_STDLOG_CFLAGS = @LIBLOGGING_STDLOG_CFLAGS@ +LIBLOGGING_STDLOG_LIBS = @LIBLOGGING_STDLOG_LIBS@ LIBLOGNORM_CFLAGS = @LIBLOGNORM_CFLAGS@ LIBLOGNORM_LIBS = @LIBLOGNORM_LIBS@ LIBM = @LIBM@ @@ -179,6 +179,8 @@ NM = @NM@ NMEDIT = @NMEDIT@ OBJDUMP = @OBJDUMP@ OBJEXT = @OBJEXT@ +OPENSSL_CFLAGS = @OPENSSL_CFLAGS@ +OPENSSL_LIBS = @OPENSSL_LIBS@ ORACLE_CFLAGS = @ORACLE_CFLAGS@ ORACLE_LIBS = @ORACLE_LIBS@ OTOOL = @OTOOL@ @@ -337,22 +339,25 @@ distclean-compile: @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/compat_la-getifaddrs.Plo@am__quote@ .c.o: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(COMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ $< -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Po +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.o$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c $< +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ $< .c.obj: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(COMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ `$(CYGPATH_W) '$<'` -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Po +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.obj$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ `$(CYGPATH_W) '$<'` &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c `$(CYGPATH_W) '$<'` +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ `$(CYGPATH_W) '$<'` .c.lo: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(LTCOMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ $< -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Plo +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.lo$$||'`;\ +@am__fastdepCC_TRUE@ $(LTCOMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Plo @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=yes @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ @am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(LTCOMPILE) -c -o $@ $< diff --git a/config.h.in b/config.h.in index f050273..06da93a 100644 --- a/config.h.in +++ b/config.h.in @@ -9,15 +9,15 @@ /* Indicator that GnuTLS is present */ #undef ENABLE_GNUTLS +/* Indicator that IMDIAG is present */ +#undef ENABLE_IMDIAG + /* Indicator that LIBGCRYPT is present */ #undef ENABLE_LIBGCRYPT /* Regular expressions support enabled. */ #undef FEATURE_REGEXP -/* Description */ -#undef FSSTND - /* Define to 1 if you have the `alarm' function. */ #undef HAVE_ALARM @@ -31,11 +31,14 @@ #undef HAVE_ATOMIC_BUILTINS /* Define if compiler provides 64 bit atomic builtins */ -#undef HAVE_ATOMIC_BUILTINS_64BIT +#undef HAVE_ATOMIC_BUILTINS64 /* Define to 1 if you have the `basename' function. */ #undef HAVE_BASENAME +/* Define to 1 if compiler supports __builtin_expect */ +#undef HAVE_BUILTIN_EXPECT + /* Define to 1 if your system has a working `chown' function. */ #undef HAVE_CHOWN @@ -97,12 +100,33 @@ /* set define */ #undef HAVE_GLOB_NOMAGIC +/* Define to 1 if you have the <hadoop/hdfs.h> header file. */ +#undef HAVE_HADOOP_HDFS_H + +/* Define to 1 if you have the <hdfs.h> header file. */ +#undef HAVE_HDFS_H + +/* Define to 1 if you have the `inotify_init' function. */ +#undef HAVE_INOTIFY_INIT + /* Define to 1 if you have the <inttypes.h> header file. */ #undef HAVE_INTTYPES_H +/* jemalloc support is integrated. */ +#undef HAVE_JEMALLOC + +/* Define to 1 if you have the `json_object_new_int64' function. */ +#undef HAVE_JSON_OBJECT_NEW_INT64 + +/* Define to 1 if you have the `json_tokener_error_desc' function. */ +#undef HAVE_JSON_TOKENER_ERROR_DESC + /* Define to 1 if you have the <libgen.h> header file. */ #undef HAVE_LIBGEN_H +/* Define to 1 if you have the `json-c' library (-ljson-c). */ +#undef HAVE_LIBJSON_C + /* Define to 1 if you have the <libnet.h> header file. */ #undef HAVE_LIBNET_H @@ -146,6 +170,9 @@ /* Define to 1 if you have the <net-snmp/net-snmp-config.h> header file. */ #undef HAVE_NET_SNMP_NET_SNMP_CONFIG_H +/* Define to 1 if the system has the type `off64_t'. */ +#undef HAVE_OFF64_T + /* Define to 1 if you have the <paths.h> header file. */ #undef HAVE_PATHS_H @@ -162,6 +189,9 @@ and to 0 otherwise. */ #undef HAVE_REALLOC +/* Define to 1 if you have the `recvmmsg' function. */ +#undef HAVE_RECVMMSG + /* Define to 1 if you have the `regcomp' function. */ #undef HAVE_REGCOMP @@ -183,8 +213,8 @@ /* Define to 1 if you have the <semaphore.h> header file. */ #undef HAVE_SEMAPHORE_H -/* Define to 1 if you have the `setid' function. */ -#undef HAVE_SETID +/* Define to 1 if you have the `setsid' function. */ +#undef HAVE_SETSID /* Define to 1 if you have the `socket' function. */ #undef HAVE_SOCKET @@ -253,15 +283,15 @@ /* set define */ #undef HAVE_SYSINFO_UPTIME -/* Define to 1 if you have the <syslog.h> header file. */ -#undef HAVE_SYSLOG_H - /* Define to 1 if you have the <sys/epoll.h> header file. */ #undef HAVE_SYS_EPOLL_H /* Define to 1 if you have the <sys/file.h> header file. */ #undef HAVE_SYS_FILE_H +/* Define to 1 if you have the <sys/inotify.h> header file. */ +#undef HAVE_SYS_INOTIFY_H + /* Define to 1 if you have the <sys/ioctl.h> header file. */ #undef HAVE_SYS_IOCTL_H @@ -298,9 +328,6 @@ /* Define to 1 if you have the `ttyname_r' function. */ #undef HAVE_TTYNAME_R -/* Define to 1 if typeof works with your compiler. */ -#undef HAVE_TYPEOF - /* Define to 1 if you have the `uname' function. */ #undef HAVE_UNAME @@ -518,9 +545,6 @@ /* Define to `int' if <sys/types.h> does not define. */ #undef ssize_t -/* Define to __typeof__ if your compiler spells it that way. */ -#undef typeof - /* Define to `int' if <sys/types.h> doesn't define. */ #undef uid_t @@ -1,6 +1,6 @@ #! /bin/sh # Guess values for system-dependent variables and create Makefiles. -# Generated by GNU Autoconf 2.68 for rsyslog 7.3.14. +# Generated by GNU Autoconf 2.68 for rsyslog 8.4.1. # # Report bugs to <rsyslog@lists.adiscon.com>. # @@ -570,8 +570,8 @@ MAKEFLAGS= # Identity of this package. PACKAGE_NAME='rsyslog' PACKAGE_TARNAME='rsyslog' -PACKAGE_VERSION='7.3.14' -PACKAGE_STRING='rsyslog 7.3.14' +PACKAGE_VERSION='8.4.1' +PACKAGE_STRING='rsyslog 8.4.1' PACKAGE_BUGREPORT='rsyslog@lists.adiscon.com' PACKAGE_URL='' @@ -615,6 +615,7 @@ ac_includes_default="\ ac_subst_vars='am__EXEEXT_FALSE am__EXEEXT_TRUE LTLIBOBJS +RST2MAN ENABLE_OMHIREDIS_FALSE ENABLE_OMHIREDIS_TRUE HIREDIS_LIBS @@ -651,6 +652,8 @@ ENABLE_PMSNARE_FALSE ENABLE_PMSNARE_TRUE ENABLE_PMAIXFORWARDEDFROM_FALSE ENABLE_PMAIXFORWARDEDFROM_TRUE +ENABLE_PMCISCOIOS_FALSE +ENABLE_PMCISCOIOS_TRUE ENABLE_PMCISCONAMES_FALSE ENABLE_PMCISCONAMES_TRUE ENABLE_PMLASTMSG_FALSE @@ -689,6 +692,20 @@ ENABLE_RELP_FALSE ENABLE_RELP_TRUE RELP_LIBS RELP_CFLAGS +ENABLE_MMRFC5424ADDHMAC_FALSE +ENABLE_MMRFC5424ADDHMAC_TRUE +OPENSSL_LIBS +OPENSSL_CFLAGS +ENABLE_MMPSTRUCDATA_FALSE +ENABLE_MMPSTRUCDATA_TRUE +ENABLE_MMFIELDS_FALSE +ENABLE_MMFIELDS_TRUE +ENABLE_MMSEQUENCE_FALSE +ENABLE_MMSEQUENCE_TRUE +ENABLE_MMCOUNT_FALSE +ENABLE_MMCOUNT_TRUE +ENABLE_MMUTF8FIX_FALSE +ENABLE_MMUTF8FIX_TRUE ENABLE_MMANON_FALSE ENABLE_MMANON_TRUE ENABLE_MMAUDIT_FALSE @@ -719,7 +736,7 @@ LIBGCRYPT_LIBS LIBGCRYPT_CFLAGS ENABLE_LIBGCRYPT_FALSE ENABLE_LIBGCRYPT_TRUE -HAVE_LIBGCRYPT_CONFIG +LIBGCRYPT_CONFIG ENABLE_GNUTLS_FALSE ENABLE_GNUTLS_TRUE GNUTLS_LIBS @@ -763,6 +780,8 @@ ENABLE_DIAGTOOLS_TRUE HAVE_SYSTEMD_FALSE HAVE_SYSTEMD_TRUE systemdsystemunitdir +ENABLE_JEMALLOC_FALSE +ENABLE_JEMALLOC_TRUE ENABLE_INET_FALSE ENABLE_INET_TRUE ENABLE_IMJOURNAL_FALSE @@ -799,14 +818,13 @@ RT_LIBS SOL_LIBS JSON_C_LIBS JSON_C_CFLAGS -LIBEE_LIBS -LIBEE_CFLAGS +LIBLOGGING_STDLOG_LIBS +LIBLOGGING_STDLOG_CFLAGS LIBESTR_LIBS LIBESTR_CFLAGS PKG_CONFIG_LIBDIR PKG_CONFIG_PATH PKG_CONFIG -RST2MAN OTOOL64 OTOOL LIPO @@ -948,7 +966,7 @@ enable_klog enable_kmsg enable_imjournal enable_inet -enable_fsstnd +enable_jemalloc enable_unlimited_select with_systemdsystemunitdir enable_debug @@ -977,6 +995,12 @@ enable_mmnormalize enable_mmjsonparse enable_mmaudit enable_mmanon +enable_mmutf8fix +enable_mmcount +enable_mmsequence +enable_mmfields +enable_mmpstrucdata +enable_mmrfc5424addhmac enable_relp enable_guardtime enable_rfc3195 @@ -992,6 +1016,7 @@ enable_omstdout enable_omjournal enable_pmlastmsg enable_pmcisconames +enable_pmciscoios enable_pmaixforwardedfrom enable_pmsnare enable_pmrfc3164sd @@ -1006,6 +1031,7 @@ enable_imzmq3 enable_omzmq3 enable_omrabbitmq enable_omhiredis +enable_generate_man_pages ' ac_precious_vars='build_alias host_alias @@ -1023,8 +1049,8 @@ PKG_CONFIG_PATH PKG_CONFIG_LIBDIR LIBESTR_CFLAGS LIBESTR_LIBS -LIBEE_CFLAGS -LIBEE_LIBS +LIBLOGGING_STDLOG_CFLAGS +LIBLOGGING_STDLOG_LIBS JSON_C_CFLAGS JSON_C_LIBS LIBSYSTEMD_JOURNAL_CFLAGS @@ -1037,6 +1063,8 @@ GNUTLS_CFLAGS GNUTLS_LIBS LIBLOGNORM_CFLAGS LIBLOGNORM_LIBS +OPENSSL_CFLAGS +OPENSSL_LIBS RELP_CFLAGS RELP_LIBS GUARDTIME_CFLAGS @@ -1593,7 +1621,7 @@ if test "$ac_init_help" = "long"; then # Omit some internal or obsolete options to make the list less imposing. # This message is too long to be a string in the A/UX 3.1 sh. cat <<_ACEOF -\`configure' configures rsyslog 7.3.14 to adapt to many kinds of systems. +\`configure' configures rsyslog 8.4.1 to adapt to many kinds of systems. Usage: $0 [OPTION]... [VAR=VALUE]... @@ -1663,7 +1691,7 @@ fi if test -n "$ac_init_help"; then case $ac_init_help in - short | recursive ) echo "Configuration of rsyslog 7.3.14:";; + short | recursive ) echo "Configuration of rsyslog 8.4.1:";; esac cat <<\_ACEOF @@ -1689,7 +1717,7 @@ Optional Features: [default=no] --enable-imjournal Systemd journal message import [default=no] --enable-inet Enable networking support [default=yes] - --disable-fsstnd Disable support for FSSTND + --enable-jemalloc Enable jemalloc support [default=no] --enable-unlimited-select Enable unlimited select() syscall [default=no] --enable-debug Enable debug mode [default=no] @@ -1721,6 +1749,14 @@ Optional Features: --enable-mmjsonparse Enable building mmjsonparse support [default=no] --enable-mmaudit Enable building mmaudit support [default=no] --enable-mmanon Enable building mmanon support [default=no] + --enable-mmutf8fix Enable building mmutf8fix support [default=no] + --enable-mmcount Enable message counting [default=no] + --enable-mmsequence Enable sequence generator [default=no] + --enable-mmfields Enable building mmfields support [default=no] + --enable-mmpstrucdata Enable building mmpstrucdata support [default=no] + --enable-mmrfc5424addhmac + Enable building mmrfc5424addhmac support + [default=no] --enable-relp Enable RELP support [default=no] --enable-guardtime Enable log file signing support (via GuardTime) [default=no] @@ -1737,6 +1773,7 @@ Optional Features: --enable-omjournal Compiles omjournal [default=no] --enable-pmlastmsg Compiles lastmsg parser module [default=no] --enable-pmcisconames Compiles cisconames parser module [default=no] + --enable-pmciscoios Compiles ciscoios parser module [default=no] --enable-pmaixforwardedfrom Compiles aixforwardedfrom parser module [default=no] --enable-pmsnare Compiles snare parser module [default=no] @@ -1753,6 +1790,8 @@ Optional Features: --enable-omzmq3 Compiles omzmq3 output module [default=no] --enable-omrabbitmq Compiles omrabbitmq output module [default=no] --enable-omhiredis Compiles omhiredis template module [default=no] + --enable-generate-man-pages + Generate man pages from source [default=yes] Optional Packages: --with-PACKAGE[=ARG] use PACKAGE [ARG=yes] @@ -1791,9 +1830,10 @@ Some influential environment variables: C compiler flags for LIBESTR, overriding pkg-config LIBESTR_LIBS linker flags for LIBESTR, overriding pkg-config - LIBEE_CFLAGS - C compiler flags for LIBEE, overriding pkg-config - LIBEE_LIBS linker flags for LIBEE, overriding pkg-config + LIBLOGGING_STDLOG_CFLAGS + C compiler flags for LIBLOGGING_STDLOG, overriding pkg-config + LIBLOGGING_STDLOG_LIBS + linker flags for LIBLOGGING_STDLOG, overriding pkg-config JSON_C_CFLAGS C compiler flags for JSON_C, overriding pkg-config JSON_C_LIBS linker flags for JSON_C, overriding pkg-config @@ -1814,6 +1854,10 @@ Some influential environment variables: C compiler flags for LIBLOGNORM, overriding pkg-config LIBLOGNORM_LIBS linker flags for LIBLOGNORM, overriding pkg-config + OPENSSL_CFLAGS + C compiler flags for OPENSSL, overriding pkg-config + OPENSSL_LIBS + linker flags for OPENSSL, overriding pkg-config RELP_CFLAGS C compiler flags for RELP, overriding pkg-config RELP_LIBS linker flags for RELP, overriding pkg-config GUARDTIME_CFLAGS @@ -1905,7 +1949,7 @@ fi test -n "$ac_init_help" && exit $ac_status if $ac_init_version; then cat <<\_ACEOF -rsyslog configure 7.3.14 +rsyslog configure 8.4.1 generated by GNU Autoconf 2.68 Copyright (C) 2010 Free Software Foundation, Inc. @@ -2484,7 +2528,7 @@ cat >config.log <<_ACEOF This file contains any messages produced by compilers while running configure, to aid debugging if configure makes a mistake. -It was created by rsyslog $as_me 7.3.14, which was +It was created by rsyslog $as_me 8.4.1, which was generated by GNU Autoconf 2.68. Invocation command line was $ $0 $@ @@ -3299,7 +3343,7 @@ fi # Define the identity of the package. PACKAGE='rsyslog' - VERSION='7.3.14' + VERSION='8.4.1' cat >>confdefs.h <<_ACEOF @@ -5870,9 +5914,6 @@ if test "$am_t" != yes; then fi -if test "$GCC" = "yes" -then CFLAGS="$CFLAGS -W -Wall -Wformat-security -Wshadow -Wcast-align -Wpointer-arith -Wmissing-format-attribute -g" -fi # Check whether --enable-static was given. if test "${enable_static+set}" = set; then : enableval=$enable_static; p=${PACKAGE-default} @@ -13075,46 +13116,6 @@ CC="$lt_save_CC" -# Extract the first word of "rst2man", so it can be a program name with args. -set dummy rst2man; ac_word=$2 -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 -$as_echo_n "checking for $ac_word... " >&6; } -if ${ac_cv_path_RST2MAN+:} false; then : - $as_echo_n "(cached) " >&6 -else - case $RST2MAN in - [\\/]* | ?:[\\/]*) - ac_cv_path_RST2MAN="$RST2MAN" # Let the user override the test with a path. - ;; - *) - as_save_IFS=$IFS; IFS=$PATH_SEPARATOR -for as_dir in $PATH -do - IFS=$as_save_IFS - test -z "$as_dir" && as_dir=. - for ac_exec_ext in '' $ac_executable_extensions; do - if { test -f "$as_dir/$ac_word$ac_exec_ext" && $as_test_x "$as_dir/$ac_word$ac_exec_ext"; }; then - ac_cv_path_RST2MAN="$as_dir/$ac_word$ac_exec_ext" - $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 - break 2 - fi -done - done -IFS=$as_save_IFS - - ;; -esac -fi -RST2MAN=$ac_cv_path_RST2MAN -if test -n "$RST2MAN"; then - { $as_echo "$as_me:${as_lineno-$LINENO}: result: $RST2MAN" >&5 -$as_echo "$RST2MAN" >&6; } -else - { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 -$as_echo "no" >&6; } -fi - - @@ -13247,12 +13248,12 @@ if test -n "$LIBESTR_CFLAGS"; then pkg_cv_LIBESTR_CFLAGS="$LIBESTR_CFLAGS" elif test -n "$PKG_CONFIG"; then if test -n "$PKG_CONFIG" && \ - { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"libestr >= 0.1.5\""; } >&5 - ($PKG_CONFIG --exists --print-errors "libestr >= 0.1.5") 2>&5 + { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"libestr >= 0.1.9\""; } >&5 + ($PKG_CONFIG --exists --print-errors "libestr >= 0.1.9") 2>&5 ac_status=$? $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 test $ac_status = 0; }; then - pkg_cv_LIBESTR_CFLAGS=`$PKG_CONFIG --cflags "libestr >= 0.1.5" 2>/dev/null` + pkg_cv_LIBESTR_CFLAGS=`$PKG_CONFIG --cflags "libestr >= 0.1.9" 2>/dev/null` test "x$?" != "x0" && pkg_failed=yes else pkg_failed=yes @@ -13264,12 +13265,12 @@ if test -n "$LIBESTR_LIBS"; then pkg_cv_LIBESTR_LIBS="$LIBESTR_LIBS" elif test -n "$PKG_CONFIG"; then if test -n "$PKG_CONFIG" && \ - { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"libestr >= 0.1.5\""; } >&5 - ($PKG_CONFIG --exists --print-errors "libestr >= 0.1.5") 2>&5 + { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"libestr >= 0.1.9\""; } >&5 + ($PKG_CONFIG --exists --print-errors "libestr >= 0.1.9") 2>&5 ac_status=$? $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 test $ac_status = 0; }; then - pkg_cv_LIBESTR_LIBS=`$PKG_CONFIG --libs "libestr >= 0.1.5" 2>/dev/null` + pkg_cv_LIBESTR_LIBS=`$PKG_CONFIG --libs "libestr >= 0.1.9" 2>/dev/null` test "x$?" != "x0" && pkg_failed=yes else pkg_failed=yes @@ -13290,14 +13291,14 @@ else _pkg_short_errors_supported=no fi if test $_pkg_short_errors_supported = yes; then - LIBESTR_PKG_ERRORS=`$PKG_CONFIG --short-errors --print-errors --cflags --libs "libestr >= 0.1.5" 2>&1` + LIBESTR_PKG_ERRORS=`$PKG_CONFIG --short-errors --print-errors --cflags --libs "libestr >= 0.1.9" 2>&1` else - LIBESTR_PKG_ERRORS=`$PKG_CONFIG --print-errors --cflags --libs "libestr >= 0.1.5" 2>&1` + LIBESTR_PKG_ERRORS=`$PKG_CONFIG --print-errors --cflags --libs "libestr >= 0.1.9" 2>&1` fi # Put the nasty error message in config.log where it belongs echo "$LIBESTR_PKG_ERRORS" >&5 - as_fn_error $? "Package requirements (libestr >= 0.1.5) were not met: + as_fn_error $? "Package requirements (libestr >= 0.1.9) were not met: $LIBESTR_PKG_ERRORS @@ -13331,19 +13332,19 @@ $as_echo "yes" >&6; } fi pkg_failed=no -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for LIBEE" >&5 -$as_echo_n "checking for LIBEE... " >&6; } +{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for LIBLOGGING_STDLOG" >&5 +$as_echo_n "checking for LIBLOGGING_STDLOG... " >&6; } -if test -n "$LIBEE_CFLAGS"; then - pkg_cv_LIBEE_CFLAGS="$LIBEE_CFLAGS" +if test -n "$LIBLOGGING_STDLOG_CFLAGS"; then + pkg_cv_LIBLOGGING_STDLOG_CFLAGS="$LIBLOGGING_STDLOG_CFLAGS" elif test -n "$PKG_CONFIG"; then if test -n "$PKG_CONFIG" && \ - { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"libee >= 0.4.0\""; } >&5 - ($PKG_CONFIG --exists --print-errors "libee >= 0.4.0") 2>&5 + { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"liblogging-stdlog >= 1.0.3\""; } >&5 + ($PKG_CONFIG --exists --print-errors "liblogging-stdlog >= 1.0.3") 2>&5 ac_status=$? $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 test $ac_status = 0; }; then - pkg_cv_LIBEE_CFLAGS=`$PKG_CONFIG --cflags "libee >= 0.4.0" 2>/dev/null` + pkg_cv_LIBLOGGING_STDLOG_CFLAGS=`$PKG_CONFIG --cflags "liblogging-stdlog >= 1.0.3" 2>/dev/null` test "x$?" != "x0" && pkg_failed=yes else pkg_failed=yes @@ -13351,16 +13352,16 @@ fi else pkg_failed=untried fi -if test -n "$LIBEE_LIBS"; then - pkg_cv_LIBEE_LIBS="$LIBEE_LIBS" +if test -n "$LIBLOGGING_STDLOG_LIBS"; then + pkg_cv_LIBLOGGING_STDLOG_LIBS="$LIBLOGGING_STDLOG_LIBS" elif test -n "$PKG_CONFIG"; then if test -n "$PKG_CONFIG" && \ - { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"libee >= 0.4.0\""; } >&5 - ($PKG_CONFIG --exists --print-errors "libee >= 0.4.0") 2>&5 + { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"liblogging-stdlog >= 1.0.3\""; } >&5 + ($PKG_CONFIG --exists --print-errors "liblogging-stdlog >= 1.0.3") 2>&5 ac_status=$? $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 test $ac_status = 0; }; then - pkg_cv_LIBEE_LIBS=`$PKG_CONFIG --libs "libee >= 0.4.0" 2>/dev/null` + pkg_cv_LIBLOGGING_STDLOG_LIBS=`$PKG_CONFIG --libs "liblogging-stdlog >= 1.0.3" 2>/dev/null` test "x$?" != "x0" && pkg_failed=yes else pkg_failed=yes @@ -13381,22 +13382,22 @@ else _pkg_short_errors_supported=no fi if test $_pkg_short_errors_supported = yes; then - LIBEE_PKG_ERRORS=`$PKG_CONFIG --short-errors --print-errors --cflags --libs "libee >= 0.4.0" 2>&1` + LIBLOGGING_STDLOG_PKG_ERRORS=`$PKG_CONFIG --short-errors --print-errors --cflags --libs "liblogging-stdlog >= 1.0.3" 2>&1` else - LIBEE_PKG_ERRORS=`$PKG_CONFIG --print-errors --cflags --libs "libee >= 0.4.0" 2>&1` + LIBLOGGING_STDLOG_PKG_ERRORS=`$PKG_CONFIG --print-errors --cflags --libs "liblogging-stdlog >= 1.0.3" 2>&1` fi # Put the nasty error message in config.log where it belongs - echo "$LIBEE_PKG_ERRORS" >&5 + echo "$LIBLOGGING_STDLOG_PKG_ERRORS" >&5 - as_fn_error $? "Package requirements (libee >= 0.4.0) were not met: + as_fn_error $? "Package requirements (liblogging-stdlog >= 1.0.3) were not met: -$LIBEE_PKG_ERRORS +$LIBLOGGING_STDLOG_PKG_ERRORS Consider adjusting the PKG_CONFIG_PATH environment variable if you installed software in a non-standard prefix. -Alternatively, you may set the environment variables LIBEE_CFLAGS -and LIBEE_LIBS to avoid the need to call pkg-config. +Alternatively, you may set the environment variables LIBLOGGING_STDLOG_CFLAGS +and LIBLOGGING_STDLOG_LIBS to avoid the need to call pkg-config. See the pkg-config man page for more details." "$LINENO" 5 elif test $pkg_failed = untried; then { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 @@ -13407,15 +13408,15 @@ as_fn_error $? "The pkg-config script could not be found or is too old. Make su is in your PATH or set the PKG_CONFIG environment variable to the full path to pkg-config. -Alternatively, you may set the environment variables LIBEE_CFLAGS -and LIBEE_LIBS to avoid the need to call pkg-config. +Alternatively, you may set the environment variables LIBLOGGING_STDLOG_CFLAGS +and LIBLOGGING_STDLOG_LIBS to avoid the need to call pkg-config. See the pkg-config man page for more details. To get pkg-config, see <http://pkg-config.freedesktop.org/>. See \`config.log' for more details" "$LINENO" 5; } else - LIBEE_CFLAGS=$pkg_cv_LIBEE_CFLAGS - LIBEE_LIBS=$pkg_cv_LIBEE_LIBS + LIBLOGGING_STDLOG_CFLAGS=$pkg_cv_LIBLOGGING_STDLOG_CFLAGS + LIBLOGGING_STDLOG_LIBS=$pkg_cv_LIBLOGGING_STDLOG_LIBS { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5 $as_echo "yes" >&6; } @@ -13479,7 +13480,163 @@ fi # Put the nasty error message in config.log where it belongs echo "$JSON_C_PKG_ERRORS" >&5 - as_fn_error $? "Package requirements (json) were not met: + + +pkg_failed=no +{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for JSON_C" >&5 +$as_echo_n "checking for JSON_C... " >&6; } + +if test -n "$JSON_C_CFLAGS"; then + pkg_cv_JSON_C_CFLAGS="$JSON_C_CFLAGS" + elif test -n "$PKG_CONFIG"; then + if test -n "$PKG_CONFIG" && \ + { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"json-c\""; } >&5 + ($PKG_CONFIG --exists --print-errors "json-c") 2>&5 + ac_status=$? + $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 + test $ac_status = 0; }; then + pkg_cv_JSON_C_CFLAGS=`$PKG_CONFIG --cflags "json-c" 2>/dev/null` + test "x$?" != "x0" && pkg_failed=yes +else + pkg_failed=yes +fi + else + pkg_failed=untried +fi +if test -n "$JSON_C_LIBS"; then + pkg_cv_JSON_C_LIBS="$JSON_C_LIBS" + elif test -n "$PKG_CONFIG"; then + if test -n "$PKG_CONFIG" && \ + { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"json-c\""; } >&5 + ($PKG_CONFIG --exists --print-errors "json-c") 2>&5 + ac_status=$? + $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 + test $ac_status = 0; }; then + pkg_cv_JSON_C_LIBS=`$PKG_CONFIG --libs "json-c" 2>/dev/null` + test "x$?" != "x0" && pkg_failed=yes +else + pkg_failed=yes +fi + else + pkg_failed=untried +fi + + + +if test $pkg_failed = yes; then + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 +$as_echo "no" >&6; } + +if $PKG_CONFIG --atleast-pkgconfig-version 0.20; then + _pkg_short_errors_supported=yes +else + _pkg_short_errors_supported=no +fi + if test $_pkg_short_errors_supported = yes; then + JSON_C_PKG_ERRORS=`$PKG_CONFIG --short-errors --print-errors --cflags --libs "json-c" 2>&1` + else + JSON_C_PKG_ERRORS=`$PKG_CONFIG --print-errors --cflags --libs "json-c" 2>&1` + fi + # Put the nasty error message in config.log where it belongs + echo "$JSON_C_PKG_ERRORS" >&5 + + as_fn_error $? "Package requirements (json-c) were not met: + +$JSON_C_PKG_ERRORS + +Consider adjusting the PKG_CONFIG_PATH environment variable if you +installed software in a non-standard prefix. + +Alternatively, you may set the environment variables JSON_C_CFLAGS +and JSON_C_LIBS to avoid the need to call pkg-config. +See the pkg-config man page for more details." "$LINENO" 5 +elif test $pkg_failed = untried; then + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 +$as_echo "no" >&6; } + { { $as_echo "$as_me:${as_lineno-$LINENO}: error: in \`$ac_pwd':" >&5 +$as_echo "$as_me: error: in \`$ac_pwd':" >&2;} +as_fn_error $? "The pkg-config script could not be found or is too old. Make sure it +is in your PATH or set the PKG_CONFIG environment variable to the full +path to pkg-config. + +Alternatively, you may set the environment variables JSON_C_CFLAGS +and JSON_C_LIBS to avoid the need to call pkg-config. +See the pkg-config man page for more details. + +To get pkg-config, see <http://pkg-config.freedesktop.org/>. +See \`config.log' for more details" "$LINENO" 5; } +else + JSON_C_CFLAGS=$pkg_cv_JSON_C_CFLAGS + JSON_C_LIBS=$pkg_cv_JSON_C_LIBS + { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5 +$as_echo "yes" >&6; } + +fi + +elif test $pkg_failed = untried; then + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 +$as_echo "no" >&6; } + + +pkg_failed=no +{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for JSON_C" >&5 +$as_echo_n "checking for JSON_C... " >&6; } + +if test -n "$JSON_C_CFLAGS"; then + pkg_cv_JSON_C_CFLAGS="$JSON_C_CFLAGS" + elif test -n "$PKG_CONFIG"; then + if test -n "$PKG_CONFIG" && \ + { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"json-c\""; } >&5 + ($PKG_CONFIG --exists --print-errors "json-c") 2>&5 + ac_status=$? + $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 + test $ac_status = 0; }; then + pkg_cv_JSON_C_CFLAGS=`$PKG_CONFIG --cflags "json-c" 2>/dev/null` + test "x$?" != "x0" && pkg_failed=yes +else + pkg_failed=yes +fi + else + pkg_failed=untried +fi +if test -n "$JSON_C_LIBS"; then + pkg_cv_JSON_C_LIBS="$JSON_C_LIBS" + elif test -n "$PKG_CONFIG"; then + if test -n "$PKG_CONFIG" && \ + { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"json-c\""; } >&5 + ($PKG_CONFIG --exists --print-errors "json-c") 2>&5 + ac_status=$? + $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 + test $ac_status = 0; }; then + pkg_cv_JSON_C_LIBS=`$PKG_CONFIG --libs "json-c" 2>/dev/null` + test "x$?" != "x0" && pkg_failed=yes +else + pkg_failed=yes +fi + else + pkg_failed=untried +fi + + + +if test $pkg_failed = yes; then + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 +$as_echo "no" >&6; } + +if $PKG_CONFIG --atleast-pkgconfig-version 0.20; then + _pkg_short_errors_supported=yes +else + _pkg_short_errors_supported=no +fi + if test $_pkg_short_errors_supported = yes; then + JSON_C_PKG_ERRORS=`$PKG_CONFIG --short-errors --print-errors --cflags --libs "json-c" 2>&1` + else + JSON_C_PKG_ERRORS=`$PKG_CONFIG --print-errors --cflags --libs "json-c" 2>&1` + fi + # Put the nasty error message in config.log where it belongs + echo "$JSON_C_PKG_ERRORS" >&5 + + as_fn_error $? "Package requirements (json-c) were not met: $JSON_C_PKG_ERRORS @@ -13512,6 +13669,85 @@ $as_echo "yes" >&6; } fi +else + JSON_C_CFLAGS=$pkg_cv_JSON_C_CFLAGS + JSON_C_LIBS=$pkg_cv_JSON_C_LIBS + { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5 +$as_echo "yes" >&6; } + +fi + +# if int64 is supported, use it +{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for json_object_new_object in -ljson-c" >&5 +$as_echo_n "checking for json_object_new_object in -ljson-c... " >&6; } +if ${ac_cv_lib_json_c_json_object_new_object+:} false; then : + $as_echo_n "(cached) " >&6 +else + ac_check_lib_save_LIBS=$LIBS +LIBS="-ljson-c $LIBS" +cat confdefs.h - <<_ACEOF >conftest.$ac_ext +/* end confdefs.h. */ + +/* Override any GCC internal prototype to avoid an error. + Use char because int might match the return type of a GCC + builtin and then its argument prototype would still apply. */ +#ifdef __cplusplus +extern "C" +#endif +char json_object_new_object (); +int +main () +{ +return json_object_new_object (); + ; + return 0; +} +_ACEOF +if ac_fn_c_try_link "$LINENO"; then : + ac_cv_lib_json_c_json_object_new_object=yes +else + ac_cv_lib_json_c_json_object_new_object=no +fi +rm -f core conftest.err conftest.$ac_objext \ + conftest$ac_exeext conftest.$ac_ext +LIBS=$ac_check_lib_save_LIBS +fi +{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_lib_json_c_json_object_new_object" >&5 +$as_echo "$ac_cv_lib_json_c_json_object_new_object" >&6; } +if test "x$ac_cv_lib_json_c_json_object_new_object" = xyes; then : + cat >>confdefs.h <<_ACEOF +#define HAVE_LIBJSON_C 1 +_ACEOF + + LIBS="-ljson-c $LIBS" + +fi + +for ac_func in json_object_new_int64 +do : + ac_fn_c_check_func "$LINENO" "json_object_new_int64" "ac_cv_func_json_object_new_int64" +if test "x$ac_cv_func_json_object_new_int64" = xyes; then : + cat >>confdefs.h <<_ACEOF +#define HAVE_JSON_OBJECT_NEW_INT64 1 +_ACEOF + +fi +done + + +# look for newer API +for ac_func in json_tokener_error_desc +do : + ac_fn_c_check_func "$LINENO" "json_tokener_error_desc" "ac_cv_func_json_tokener_error_desc" +if test "x$ac_cv_func_json_tokener_error_desc" = xyes; then : + cat >>confdefs.h <<_ACEOF +#define HAVE_JSON_TOKENER_ERROR_DESC 1 +_ACEOF + +fi +done + + case "${host}" in *-*-linux*) @@ -13913,7 +14149,7 @@ $as_echo "#define HAVE_SYS_WAIT_H 1" >>confdefs.h fi -for ac_header in arpa/inet.h libgen.h malloc.h fcntl.h locale.h netdb.h netinet/in.h paths.h stddef.h stdlib.h string.h sys/file.h sys/ioctl.h sys/param.h sys/socket.h sys/time.h sys/stat.h syslog.h unistd.h utmp.h utmpx.h sys/epoll.h sys/prctl.h +for ac_header in arpa/inet.h libgen.h malloc.h fcntl.h locale.h netdb.h netinet/in.h paths.h stddef.h stdlib.h string.h sys/file.h sys/ioctl.h sys/param.h sys/socket.h sys/time.h sys/stat.h sys/inotify.h unistd.h utmp.h utmpx.h sys/epoll.h sys/prctl.h do : as_ac_Header=`$as_echo "ac_cv_header_$ac_header" | $as_tr_sh` ac_fn_c_check_header_mongrel "$LINENO" "$ac_header" "$as_ac_Header" "$ac_includes_default" @@ -14254,60 +14490,6 @@ $as_echo "#define volatile /**/" >>confdefs.h fi - { $as_echo "$as_me:${as_lineno-$LINENO}: checking for typeof syntax and keyword spelling" >&5 -$as_echo_n "checking for typeof syntax and keyword spelling... " >&6; } -if ${ac_cv_c_typeof+:} false; then : - $as_echo_n "(cached) " >&6 -else - ac_cv_c_typeof=no - for ac_kw in typeof __typeof__ no; do - test $ac_kw = no && break - cat confdefs.h - <<_ACEOF >conftest.$ac_ext -/* end confdefs.h. */ - -int -main () -{ - - int value; - typedef struct { - char a [1 - + ! (($ac_kw (value)) - (($ac_kw (value)) 0 < ($ac_kw (value)) -1 - ? ($ac_kw (value)) - 1 - : ~ (~ ($ac_kw (value)) 0 - << sizeof ($ac_kw (value)))))]; } - ac__typeof_type_; - return - (! ((void) ((ac__typeof_type_ *) 0), 0)); - - ; - return 0; -} -_ACEOF -if ac_fn_c_try_compile "$LINENO"; then : - ac_cv_c_typeof=$ac_kw -fi -rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext - test $ac_cv_c_typeof != no && break - done -fi -{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_c_typeof" >&5 -$as_echo "$ac_cv_c_typeof" >&6; } - if test $ac_cv_c_typeof != no; then - -$as_echo "#define HAVE_TYPEOF 1" >>confdefs.h - - if test $ac_cv_c_typeof != typeof; then - -cat >>confdefs.h <<_ACEOF -#define typeof $ac_cv_c_typeof -_ACEOF - - fi - fi - - sa_includes="\ $ac_includes_default #if HAVE_SYS_SOCKET_H @@ -15108,7 +15290,7 @@ fi done -for ac_func in flock basename alarm clock_gettime gethostbyname gethostname gettimeofday localtime_r memset mkdir regcomp select setid socket strcasecmp strchr strdup strerror strndup strnlen strrchr strstr strtol strtoul uname ttyname_r getline malloc_trim prctl epoll_create epoll_create1 fdatasync syscall lseek64 +for ac_func in flock inotify_init recvmmsg basename alarm clock_gettime gethostbyname gethostname gettimeofday localtime_r memset mkdir regcomp select setsid socket strcasecmp strchr strdup strerror strndup strnlen strrchr strstr strtol strtoul uname ttyname_r getline malloc_trim prctl epoll_create epoll_create1 fdatasync syscall lseek64 do : as_ac_var=`$as_echo "ac_cv_func_$ac_func" | $as_tr_sh` ac_fn_c_check_func "$LINENO" "$ac_func" "$as_ac_var" @@ -15120,6 +15302,16 @@ _ACEOF fi done +ac_fn_c_check_type "$LINENO" "off64_t" "ac_cv_type_off64_t" "$ac_includes_default" +if test "x$ac_cv_type_off64_t" = xyes; then : + +cat >>confdefs.h <<_ACEOF +#define HAVE_OFF64_T 1 +_ACEOF + + +fi + # getifaddrs is in libc (mostly) or in libsocket (eg Solaris 11) or not defined (eg Solaris 10) { $as_echo "$as_me:${as_lineno-$LINENO}: checking for library containing getifaddrs" >&5 @@ -15262,6 +15454,33 @@ $as_echo "no; defined as 64" >&6; } fi rm -f core conftest.err conftest.$ac_objext conftest.$ac_ext +# Check for __builtin_expect() +{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for __builtin_expect()" >&5 +$as_echo_n "checking for __builtin_expect()... " >&6; } +cat confdefs.h - <<_ACEOF >conftest.$ac_ext +/* end confdefs.h. */ + +int +main () +{ +return __builtin_expect(main != 0, 1) + ; + return 0; +} +_ACEOF +if ac_fn_c_try_link "$LINENO"; then : + +$as_echo "#define HAVE_BUILTIN_EXPECT 1" >>confdefs.h + + { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5 +$as_echo "yes" >&6; } +else + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 +$as_echo "no" >&6; } +fi +rm -f core conftest.err conftest.$ac_objext \ + conftest$ac_exeext conftest.$ac_ext + # check for availability of atomic operations { $as_echo "$as_me:${as_lineno-$LINENO}: checking whether the compiler provides atomic builtins" >&5 $as_echo_n "checking whether the compiler provides atomic builtins... " >&6; } @@ -15392,7 +15611,7 @@ $as_echo "$ap_cv_atomic_builtins_64" >&6; } if test "$ap_cv_atomic_builtins_64" = "yes"; then -$as_echo "#define HAVE_ATOMIC_BUILTINS_64BIT 1" >>confdefs.h +$as_echo "#define HAVE_ATOMIC_BUILTINS64 1" >>confdefs.h fi @@ -16246,38 +16465,79 @@ $as_echo "#define SYSLOG_INET 1" >>confdefs.h fi +# jemalloc +# Check whether --enable-jemalloc was given. +if test "${enable_jemalloc+set}" = set; then : + enableval=$enable_jemalloc; case "${enableval}" in + yes) enable_jemalloc="yes" ;; + no) enable_jemalloc="no" ;; + *) as_fn_error $? "bad value ${enableval} for --enable-jemalloc" "$LINENO" 5 ;; + esac +else + enable_jemalloc="no" -# -# The following define determines whether the package adheres to the -# file system standard. -# -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for FSSTND support" >&5 -$as_echo_n "checking for FSSTND support... " >&6; } -# Check whether --enable-fsstnd was given. -if test "${enable_fsstnd+set}" = set; then : - enableval=$enable_fsstnd; - if test "x${enableval}" = "xyes"; then - { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5 -$as_echo "yes" >&6; } - -$as_echo "#define FSSTND 1" >>confdefs.h +fi - else - { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 -$as_echo "no" >&6; } - fi + if test x$enable_jemalloc = xyes; then + ENABLE_JEMALLOC_TRUE= + ENABLE_JEMALLOC_FALSE='#' +else + ENABLE_JEMALLOC_TRUE='#' + ENABLE_JEMALLOC_FALSE= +fi +if test "$enable_jemalloc" = "yes"; then + { $as_echo "$as_me:${as_lineno-$LINENO}: checking for malloc_stats_print in -ljemalloc" >&5 +$as_echo_n "checking for malloc_stats_print in -ljemalloc... " >&6; } +if ${ac_cv_lib_jemalloc_malloc_stats_print+:} false; then : + $as_echo_n "(cached) " >&6 else + ac_check_lib_save_LIBS=$LIBS +LIBS="-ljemalloc + $LIBS" +cat confdefs.h - <<_ACEOF >conftest.$ac_ext +/* end confdefs.h. */ - # enabled by default - { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5 -$as_echo "yes" >&6; } +/* Override any GCC internal prototype to avoid an error. + Use char because int might match the return type of a GCC + builtin and then its argument prototype would still apply. */ +#ifdef __cplusplus +extern "C" +#endif +char malloc_stats_print (); +int +main () +{ +return malloc_stats_print (); + ; + return 0; +} +_ACEOF +if ac_fn_c_try_link "$LINENO"; then : + ac_cv_lib_jemalloc_malloc_stats_print=yes +else + ac_cv_lib_jemalloc_malloc_stats_print=no +fi +rm -f core conftest.err conftest.$ac_objext \ + conftest$ac_exeext conftest.$ac_ext +LIBS=$ac_check_lib_save_LIBS +fi +{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_lib_jemalloc_malloc_stats_print" >&5 +$as_echo "$ac_cv_lib_jemalloc_malloc_stats_print" >&6; } +if test "x$ac_cv_lib_jemalloc_malloc_stats_print" = xyes; then : + RT_LIBS="$RT_LIBS -ljemalloc" -$as_echo "#define FSSTND 1" >>confdefs.h +$as_echo "#define HAVE_JEMALLOC 1" >>confdefs.h +else + { { $as_echo "$as_me:${as_lineno-$LINENO}: error: in \`$ac_pwd':" >&5 +$as_echo "$as_me: error: in \`$ac_pwd':" >&2;} +as_fn_error $? "jemalloc library is missing +See \`config.log' for more details" "$LINENO" 5; } fi +fi # support for unlimited select() syscall @@ -17731,24 +17991,26 @@ else fi if test "x$enable_libgcrypt" = "xyes"; then - # Extract the first word of "libgcrypt-config", so it can be a program name with args. + # Extract the first word of "libgcrypt-config", so it can be a program name with args. set dummy libgcrypt-config; ac_word=$2 { $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 $as_echo_n "checking for $ac_word... " >&6; } -if ${ac_cv_prog_HAVE_LIBGCRYPT_CONFIG+:} false; then : +if ${ac_cv_path_LIBGCRYPT_CONFIG+:} false; then : $as_echo_n "(cached) " >&6 else - if test -n "$HAVE_LIBGCRYPT_CONFIG"; then - ac_cv_prog_HAVE_LIBGCRYPT_CONFIG="$HAVE_LIBGCRYPT_CONFIG" # Let the user override the test. -else -as_save_IFS=$IFS; IFS=$PATH_SEPARATOR + case $LIBGCRYPT_CONFIG in + [\\/]* | ?:[\\/]*) + ac_cv_path_LIBGCRYPT_CONFIG="$LIBGCRYPT_CONFIG" # Let the user override the test with a path. + ;; + *) + as_save_IFS=$IFS; IFS=$PATH_SEPARATOR for as_dir in $PATH do IFS=$as_save_IFS test -z "$as_dir" && as_dir=. for ac_exec_ext in '' $ac_executable_extensions; do if { test -f "$as_dir/$ac_word$ac_exec_ext" && $as_test_x "$as_dir/$ac_word$ac_exec_ext"; }; then - ac_cv_prog_HAVE_LIBGCRYPT_CONFIG="yes" + ac_cv_path_LIBGCRYPT_CONFIG="$as_dir/$ac_word$ac_exec_ext" $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 break 2 fi @@ -17756,19 +18018,21 @@ done done IFS=$as_save_IFS + test -z "$ac_cv_path_LIBGCRYPT_CONFIG" && ac_cv_path_LIBGCRYPT_CONFIG="no" + ;; +esac fi -fi -HAVE_LIBGCRYPT_CONFIG=$ac_cv_prog_HAVE_LIBGCRYPT_CONFIG -if test -n "$HAVE_LIBGCRYPT_CONFIG"; then - { $as_echo "$as_me:${as_lineno-$LINENO}: result: $HAVE_LIBGCRYPT_CONFIG" >&5 -$as_echo "$HAVE_LIBGCRYPT_CONFIG" >&6; } +LIBGCRYPT_CONFIG=$ac_cv_path_LIBGCRYPT_CONFIG +if test -n "$LIBGCRYPT_CONFIG"; then + { $as_echo "$as_me:${as_lineno-$LINENO}: result: $LIBGCRYPT_CONFIG" >&5 +$as_echo "$LIBGCRYPT_CONFIG" >&6; } else { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 $as_echo "no" >&6; } fi - if test "x${HAVE_LIBGCRYPT_CONFIG}" != "xyes"; then + if test "x${LIBGCRYPT_CONFIG}" = "xno"; then { { $as_echo "$as_me:${as_lineno-$LINENO}: error: in \`$ac_pwd':" >&5 $as_echo "$as_me: error: in \`$ac_pwd':" >&2;} as_fn_error $? "libgcrypt-config not found in PATH @@ -17780,7 +18044,7 @@ if ${ac_cv_lib_gcrypt_gcry_cipher_open+:} false; then : $as_echo_n "(cached) " >&6 else ac_check_lib_save_LIBS=$LIBS -LIBS="-lgcrypt `libgcrypt-config --libs --cflags` +LIBS="-lgcrypt `${LIBGCRYPT_CONFIG} --libs --cflags` $LIBS" cat confdefs.h - <<_ACEOF >conftest.$ac_ext /* end confdefs.h. */ @@ -17812,8 +18076,8 @@ fi { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_lib_gcrypt_gcry_cipher_open" >&5 $as_echo "$ac_cv_lib_gcrypt_gcry_cipher_open" >&6; } if test "x$ac_cv_lib_gcrypt_gcry_cipher_open" = xyes; then : - LIBGCRYPT_CFLAGS="`libgcrypt-config --cflags`" - LIBGCRYPT_LIBS="`libgcrypt-config --libs`" + LIBGCRYPT_CFLAGS="`${LIBGCRYPT_CONFIG} --cflags`" + LIBGCRYPT_LIBS="`${LIBGCRYPT_CONFIG} --libs`" else { { $as_echo "$as_me:${as_lineno-$LINENO}: error: in \`$ac_pwd':" >&5 @@ -17863,7 +18127,10 @@ else ENABLE_RSYSLOGRT_FALSE= fi -RSRT_CFLAGS="\$(RSRT_CFLAGS1) \$(LIBESTR_CFLAGS) \$(JSON_C_FLAGS)" +RSRT_CFLAGS="\$(RSRT_CFLAGS1) \$(LIBESTR_CFLAGS) \$(JSON_C_CFLAGS)" +if test "$GCC" = "yes" +then RSRT_CFLAGS="$RSRT_CFLAGS -W -std=c99 -Wall -Wformat-security -Wshadow -Wcast-align -Wpointer-arith -Wmissing-format-attribute -g" +fi RSRT_LIBS="\$(RSRT_LIBS1) \$(LIBESTR_LIBS) \$(JSON_C_LIBS)" @@ -17971,11 +18238,9 @@ fi -# imdiag support (so far we do not need a library, but we need to turn this on and off) -# note that we enable this be default, because an important point is to make -# it available to users who do not know much about how to handle things. It -# would complicate things if we first needed to tell them how to enable imdiag. -# rgerhards, 2008-07-25 +# imdiag support +# This is a core testbench tool. You need to enable it if you want to +# use not only a small subset of the testbench. # Check whether --enable-imdiag was given. if test "${enable_imdiag+set}" = set; then : enableval=$enable_imdiag; case "${enableval}" in @@ -17988,6 +18253,11 @@ else fi +if test "x$enable_imdiag" = "xyes"; then + +$as_echo "#define ENABLE_IMDIAG 1" >>confdefs.h + +fi if test x$enable_imdiag = xyes; then ENABLE_IMDIAG_TRUE= ENABLE_IMDIAG_FALSE='#' @@ -18021,12 +18291,12 @@ if test -n "$LIBLOGNORM_CFLAGS"; then pkg_cv_LIBLOGNORM_CFLAGS="$LIBLOGNORM_CFLAGS" elif test -n "$PKG_CONFIG"; then if test -n "$PKG_CONFIG" && \ - { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"lognorm >= 0.3.1\""; } >&5 - ($PKG_CONFIG --exists --print-errors "lognorm >= 0.3.1") 2>&5 + { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"lognorm >= 1.0.0\""; } >&5 + ($PKG_CONFIG --exists --print-errors "lognorm >= 1.0.0") 2>&5 ac_status=$? $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 test $ac_status = 0; }; then - pkg_cv_LIBLOGNORM_CFLAGS=`$PKG_CONFIG --cflags "lognorm >= 0.3.1" 2>/dev/null` + pkg_cv_LIBLOGNORM_CFLAGS=`$PKG_CONFIG --cflags "lognorm >= 1.0.0" 2>/dev/null` test "x$?" != "x0" && pkg_failed=yes else pkg_failed=yes @@ -18038,12 +18308,12 @@ if test -n "$LIBLOGNORM_LIBS"; then pkg_cv_LIBLOGNORM_LIBS="$LIBLOGNORM_LIBS" elif test -n "$PKG_CONFIG"; then if test -n "$PKG_CONFIG" && \ - { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"lognorm >= 0.3.1\""; } >&5 - ($PKG_CONFIG --exists --print-errors "lognorm >= 0.3.1") 2>&5 + { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"lognorm >= 1.0.0\""; } >&5 + ($PKG_CONFIG --exists --print-errors "lognorm >= 1.0.0") 2>&5 ac_status=$? $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 test $ac_status = 0; }; then - pkg_cv_LIBLOGNORM_LIBS=`$PKG_CONFIG --libs "lognorm >= 0.3.1" 2>/dev/null` + pkg_cv_LIBLOGNORM_LIBS=`$PKG_CONFIG --libs "lognorm >= 1.0.0" 2>/dev/null` test "x$?" != "x0" && pkg_failed=yes else pkg_failed=yes @@ -18064,14 +18334,14 @@ else _pkg_short_errors_supported=no fi if test $_pkg_short_errors_supported = yes; then - LIBLOGNORM_PKG_ERRORS=`$PKG_CONFIG --short-errors --print-errors --cflags --libs "lognorm >= 0.3.1" 2>&1` + LIBLOGNORM_PKG_ERRORS=`$PKG_CONFIG --short-errors --print-errors --cflags --libs "lognorm >= 1.0.0" 2>&1` else - LIBLOGNORM_PKG_ERRORS=`$PKG_CONFIG --print-errors --cflags --libs "lognorm >= 0.3.1" 2>&1` + LIBLOGNORM_PKG_ERRORS=`$PKG_CONFIG --print-errors --cflags --libs "lognorm >= 1.0.0" 2>&1` fi # Put the nasty error message in config.log where it belongs echo "$LIBLOGNORM_PKG_ERRORS" >&5 - as_fn_error $? "Package requirements (lognorm >= 0.3.1) were not met: + as_fn_error $? "Package requirements (lognorm >= 1.0.0) were not met: $LIBLOGNORM_PKG_ERRORS @@ -18127,138 +18397,205 @@ else fi -if test "x$enable_mmjsonparse" = "xyes"; then + if test x$enable_mmjsonparse = xyes; then + ENABLE_MMJSONPARSE_TRUE= + ENABLE_MMJSONPARSE_FALSE='#' +else + ENABLE_MMJSONPARSE_TRUE='#' + ENABLE_MMJSONPARSE_FALSE= +fi -pkg_failed=no -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for LIBLOGNORM" >&5 -$as_echo_n "checking for LIBLOGNORM... " >&6; } -if test -n "$LIBLOGNORM_CFLAGS"; then - pkg_cv_LIBLOGNORM_CFLAGS="$LIBLOGNORM_CFLAGS" - elif test -n "$PKG_CONFIG"; then - if test -n "$PKG_CONFIG" && \ - { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"lognorm >= 0.3.1\""; } >&5 - ($PKG_CONFIG --exists --print-errors "lognorm >= 0.3.1") 2>&5 - ac_status=$? - $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 - test $ac_status = 0; }; then - pkg_cv_LIBLOGNORM_CFLAGS=`$PKG_CONFIG --cflags "lognorm >= 0.3.1" 2>/dev/null` - test "x$?" != "x0" && pkg_failed=yes + +# mmaudit +# Check whether --enable-mmaudit was given. +if test "${enable_mmaudit+set}" = set; then : + enableval=$enable_mmaudit; case "${enableval}" in + yes) enable_mmaudit="yes" ;; + no) enable_mmaudit="no" ;; + *) as_fn_error $? "bad value ${enableval} for --enable-mmaudit" "$LINENO" 5 ;; + esac else - pkg_failed=yes + enable_mmaudit=no + fi - else - pkg_failed=untried + + if test x$enable_mmaudit = xyes; then + ENABLE_MMAUDIT_TRUE= + ENABLE_MMAUDIT_FALSE='#' +else + ENABLE_MMAUDIT_TRUE='#' + ENABLE_MMAUDIT_FALSE= fi -if test -n "$LIBLOGNORM_LIBS"; then - pkg_cv_LIBLOGNORM_LIBS="$LIBLOGNORM_LIBS" - elif test -n "$PKG_CONFIG"; then - if test -n "$PKG_CONFIG" && \ - { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"lognorm >= 0.3.1\""; } >&5 - ($PKG_CONFIG --exists --print-errors "lognorm >= 0.3.1") 2>&5 - ac_status=$? - $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 - test $ac_status = 0; }; then - pkg_cv_LIBLOGNORM_LIBS=`$PKG_CONFIG --libs "lognorm >= 0.3.1" 2>/dev/null` - test "x$?" != "x0" && pkg_failed=yes + + + +# mmanon +# Check whether --enable-mmanon was given. +if test "${enable_mmanon+set}" = set; then : + enableval=$enable_mmanon; case "${enableval}" in + yes) enable_mmanon="yes" ;; + no) enable_mmanon="no" ;; + *) as_fn_error $? "bad value ${enableval} for --enable-mmanon" "$LINENO" 5 ;; + esac else - pkg_failed=yes + enable_mmanon=no + fi - else - pkg_failed=untried + + if test x$enable_mmanon = xyes; then + ENABLE_MMANON_TRUE= + ENABLE_MMANON_FALSE='#' +else + ENABLE_MMANON_TRUE='#' + ENABLE_MMANON_FALSE= fi -if test $pkg_failed = yes; then - { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 -$as_echo "no" >&6; } +# mmutf8fix +# Check whether --enable-mmutf8fix was given. +if test "${enable_mmutf8fix+set}" = set; then : + enableval=$enable_mmutf8fix; case "${enableval}" in + yes) enable_mmutf8fix="yes" ;; + no) enable_mmutf8fix="no" ;; + *) as_fn_error $? "bad value ${enableval} for --enable-mmutf8fix" "$LINENO" 5 ;; + esac +else + enable_mmutf8fix=no -if $PKG_CONFIG --atleast-pkgconfig-version 0.20; then - _pkg_short_errors_supported=yes +fi + + if test x$enable_mmutf8fix = xyes; then + ENABLE_MMUTF8FIX_TRUE= + ENABLE_MMUTF8FIX_FALSE='#' else - _pkg_short_errors_supported=no + ENABLE_MMUTF8FIX_TRUE='#' + ENABLE_MMUTF8FIX_FALSE= fi - if test $_pkg_short_errors_supported = yes; then - LIBLOGNORM_PKG_ERRORS=`$PKG_CONFIG --short-errors --print-errors --cflags --libs "lognorm >= 0.3.1" 2>&1` - else - LIBLOGNORM_PKG_ERRORS=`$PKG_CONFIG --print-errors --cflags --libs "lognorm >= 0.3.1" 2>&1` - fi - # Put the nasty error message in config.log where it belongs - echo "$LIBLOGNORM_PKG_ERRORS" >&5 - as_fn_error $? "Package requirements (lognorm >= 0.3.1) were not met: -$LIBLOGNORM_PKG_ERRORS -Consider adjusting the PKG_CONFIG_PATH environment variable if you -installed software in a non-standard prefix. +# mmcount +# Check whether --enable-mmcount was given. +if test "${enable_mmcount+set}" = set; then : + enableval=$enable_mmcount; case "${enableval}" in + yes) enable_mmcount="yes" ;; + no) enable_mmcount="no" ;; + *) as_fn_error $? "bad value ${enableval} for --enable-mmcount" "$LINENO" 5 ;; + esac +else + enable_mmcount=no -Alternatively, you may set the environment variables LIBLOGNORM_CFLAGS -and LIBLOGNORM_LIBS to avoid the need to call pkg-config. -See the pkg-config man page for more details." "$LINENO" 5 -elif test $pkg_failed = untried; then - { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 -$as_echo "no" >&6; } - { { $as_echo "$as_me:${as_lineno-$LINENO}: error: in \`$ac_pwd':" >&5 -$as_echo "$as_me: error: in \`$ac_pwd':" >&2;} -as_fn_error $? "The pkg-config script could not be found or is too old. Make sure it -is in your PATH or set the PKG_CONFIG environment variable to the full -path to pkg-config. +fi -Alternatively, you may set the environment variables LIBLOGNORM_CFLAGS -and LIBLOGNORM_LIBS to avoid the need to call pkg-config. -See the pkg-config man page for more details. + if test x$enable_mmcount = xyes; then + ENABLE_MMCOUNT_TRUE= + ENABLE_MMCOUNT_FALSE='#' +else + ENABLE_MMCOUNT_TRUE='#' + ENABLE_MMCOUNT_FALSE= +fi -To get pkg-config, see <http://pkg-config.freedesktop.org/>. -See \`config.log' for more details" "$LINENO" 5; } + + +# mmsequence +# Check whether --enable-mmsequence was given. +if test "${enable_mmsequence+set}" = set; then : + enableval=$enable_mmsequence; case "${enableval}" in + yes) enable_mmsequence="yes" ;; + no) enable_mmsequence="no" ;; + *) as_fn_error $? "bad value ${enableval} for --enable-mmsequence" "$LINENO" 5 ;; + esac else - LIBLOGNORM_CFLAGS=$pkg_cv_LIBLOGNORM_CFLAGS - LIBLOGNORM_LIBS=$pkg_cv_LIBLOGNORM_LIBS - { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5 -$as_echo "yes" >&6; } + enable_mmsequence=no fi + + if test x$enable_mmsequence = xyes; then + ENABLE_MMSEQUENCE_TRUE= + ENABLE_MMSEQUENCE_FALSE='#' +else + ENABLE_MMSEQUENCE_TRUE='#' + ENABLE_MMSEQUENCE_FALSE= fi - if test x$enable_mmjsonparse = xyes; then - ENABLE_MMJSONPARSE_TRUE= - ENABLE_MMJSONPARSE_FALSE='#' + + + +# mmfields +# Check whether --enable-mmfields was given. +if test "${enable_mmfields+set}" = set; then : + enableval=$enable_mmfields; case "${enableval}" in + yes) enable_mmfields="yes" ;; + no) enable_mmfields="no" ;; + *) as_fn_error $? "bad value ${enableval} for --enable-mmfields" "$LINENO" 5 ;; + esac else - ENABLE_MMJSONPARSE_TRUE='#' - ENABLE_MMJSONPARSE_FALSE= + enable_mmfields=no + fi + if test x$enable_mmfields = xyes; then + ENABLE_MMFIELDS_TRUE= + ENABLE_MMFIELDS_FALSE='#' +else + ENABLE_MMFIELDS_TRUE='#' + ENABLE_MMFIELDS_FALSE= +fi -# mmaudit -# Check whether --enable-mmaudit was given. -if test "${enable_mmaudit+set}" = set; then : - enableval=$enable_mmaudit; case "${enableval}" in - yes) enable_mmaudit="yes" ;; - no) enable_mmaudit="no" ;; - *) as_fn_error $? "bad value ${enableval} for --enable-mmaudit" "$LINENO" 5 ;; +# mmpstrucdata +# Check whether --enable-mmpstrucdata was given. +if test "${enable_mmpstrucdata+set}" = set; then : + enableval=$enable_mmpstrucdata; case "${enableval}" in + yes) enable_mmpstrucdata="yes" ;; + no) enable_mmpstrucdata="no" ;; + *) as_fn_error $? "bad value ${enableval} for --enable-mmpstrucdata" "$LINENO" 5 ;; esac else - enable_mmaudit=no + enable_mmpstrucdata=no + +fi + if test x$enable_mmpstrucdata = xyes; then + ENABLE_MMPSTRUCDATA_TRUE= + ENABLE_MMPSTRUCDATA_FALSE='#' +else + ENABLE_MMPSTRUCDATA_TRUE='#' + ENABLE_MMPSTRUCDATA_FALSE= fi -if test "x$enable_mmaudit" = "xyes"; then + + +# mmrfc5424addhmac +# Check whether --enable-mmrfc5424addhmac was given. +if test "${enable_mmrfc5424addhmac+set}" = set; then : + enableval=$enable_mmrfc5424addhmac; case "${enableval}" in + yes) enable_mmrfc5424addhmac="yes" ;; + no) enable_mmrfc5424addhmac="no" ;; + *) as_fn_error $? "bad value ${enableval} for --enable-mmrfc5424addhmac" "$LINENO" 5 ;; + esac +else + enable_mmrfc5424addhmac=no + +fi + +if test "x$enable_mmrfc5424addhmac" = "xyes"; then pkg_failed=no -{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for LIBLOGNORM" >&5 -$as_echo_n "checking for LIBLOGNORM... " >&6; } +{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for OPENSSL" >&5 +$as_echo_n "checking for OPENSSL... " >&6; } -if test -n "$LIBLOGNORM_CFLAGS"; then - pkg_cv_LIBLOGNORM_CFLAGS="$LIBLOGNORM_CFLAGS" +if test -n "$OPENSSL_CFLAGS"; then + pkg_cv_OPENSSL_CFLAGS="$OPENSSL_CFLAGS" elif test -n "$PKG_CONFIG"; then if test -n "$PKG_CONFIG" && \ - { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"lognorm >= 0.3.1\""; } >&5 - ($PKG_CONFIG --exists --print-errors "lognorm >= 0.3.1") 2>&5 + { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"openssl >= 0.9.7\""; } >&5 + ($PKG_CONFIG --exists --print-errors "openssl >= 0.9.7") 2>&5 ac_status=$? $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 test $ac_status = 0; }; then - pkg_cv_LIBLOGNORM_CFLAGS=`$PKG_CONFIG --cflags "lognorm >= 0.3.1" 2>/dev/null` + pkg_cv_OPENSSL_CFLAGS=`$PKG_CONFIG --cflags "openssl >= 0.9.7" 2>/dev/null` test "x$?" != "x0" && pkg_failed=yes else pkg_failed=yes @@ -18266,16 +18603,16 @@ fi else pkg_failed=untried fi -if test -n "$LIBLOGNORM_LIBS"; then - pkg_cv_LIBLOGNORM_LIBS="$LIBLOGNORM_LIBS" +if test -n "$OPENSSL_LIBS"; then + pkg_cv_OPENSSL_LIBS="$OPENSSL_LIBS" elif test -n "$PKG_CONFIG"; then if test -n "$PKG_CONFIG" && \ - { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"lognorm >= 0.3.1\""; } >&5 - ($PKG_CONFIG --exists --print-errors "lognorm >= 0.3.1") 2>&5 + { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"openssl >= 0.9.7\""; } >&5 + ($PKG_CONFIG --exists --print-errors "openssl >= 0.9.7") 2>&5 ac_status=$? $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 test $ac_status = 0; }; then - pkg_cv_LIBLOGNORM_LIBS=`$PKG_CONFIG --libs "lognorm >= 0.3.1" 2>/dev/null` + pkg_cv_OPENSSL_LIBS=`$PKG_CONFIG --libs "openssl >= 0.9.7" 2>/dev/null` test "x$?" != "x0" && pkg_failed=yes else pkg_failed=yes @@ -18296,22 +18633,22 @@ else _pkg_short_errors_supported=no fi if test $_pkg_short_errors_supported = yes; then - LIBLOGNORM_PKG_ERRORS=`$PKG_CONFIG --short-errors --print-errors --cflags --libs "lognorm >= 0.3.1" 2>&1` + OPENSSL_PKG_ERRORS=`$PKG_CONFIG --short-errors --print-errors --cflags --libs "openssl >= 0.9.7" 2>&1` else - LIBLOGNORM_PKG_ERRORS=`$PKG_CONFIG --print-errors --cflags --libs "lognorm >= 0.3.1" 2>&1` + OPENSSL_PKG_ERRORS=`$PKG_CONFIG --print-errors --cflags --libs "openssl >= 0.9.7" 2>&1` fi # Put the nasty error message in config.log where it belongs - echo "$LIBLOGNORM_PKG_ERRORS" >&5 + echo "$OPENSSL_PKG_ERRORS" >&5 - as_fn_error $? "Package requirements (lognorm >= 0.3.1) were not met: + as_fn_error $? "Package requirements (openssl >= 0.9.7) were not met: -$LIBLOGNORM_PKG_ERRORS +$OPENSSL_PKG_ERRORS Consider adjusting the PKG_CONFIG_PATH environment variable if you installed software in a non-standard prefix. -Alternatively, you may set the environment variables LIBLOGNORM_CFLAGS -and LIBLOGNORM_LIBS to avoid the need to call pkg-config. +Alternatively, you may set the environment variables OPENSSL_CFLAGS +and OPENSSL_LIBS to avoid the need to call pkg-config. See the pkg-config man page for more details." "$LINENO" 5 elif test $pkg_failed = untried; then { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 @@ -18322,49 +18659,29 @@ as_fn_error $? "The pkg-config script could not be found or is too old. Make su is in your PATH or set the PKG_CONFIG environment variable to the full path to pkg-config. -Alternatively, you may set the environment variables LIBLOGNORM_CFLAGS -and LIBLOGNORM_LIBS to avoid the need to call pkg-config. +Alternatively, you may set the environment variables OPENSSL_CFLAGS +and OPENSSL_LIBS to avoid the need to call pkg-config. See the pkg-config man page for more details. To get pkg-config, see <http://pkg-config.freedesktop.org/>. See \`config.log' for more details" "$LINENO" 5; } else - LIBLOGNORM_CFLAGS=$pkg_cv_LIBLOGNORM_CFLAGS - LIBLOGNORM_LIBS=$pkg_cv_LIBLOGNORM_LIBS + OPENSSL_CFLAGS=$pkg_cv_OPENSSL_CFLAGS + OPENSSL_LIBS=$pkg_cv_OPENSSL_LIBS { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5 $as_echo "yes" >&6; } fi +#AC_CHECK_LIB([crypto],[CRYPTO_new_ex_data], [], [AC_MSG_ERROR([OpenSSL libraries required])]) +#AC_CHECK_LIB([ssl],[SSL_library_init], [], [AC_MSG_ERROR([OpenSSL libraries required])]) +#AC_CHECK_HEADERS([openssl/crypto.h openssl/x509.h openssl/pem.h openssl/ssl.h openssl/err.h],[],[AC_MSG_ERROR([OpenSSL headers required])]) fi - if test x$enable_mmaudit = xyes; then - ENABLE_MMAUDIT_TRUE= - ENABLE_MMAUDIT_FALSE='#' + if test x$enable_mmrfc5424addhmac = xyes; then + ENABLE_MMRFC5424ADDHMAC_TRUE= + ENABLE_MMRFC5424ADDHMAC_FALSE='#' else - ENABLE_MMAUDIT_TRUE='#' - ENABLE_MMAUDIT_FALSE= -fi - - - -# mmanon -# Check whether --enable-mmanon was given. -if test "${enable_mmanon+set}" = set; then : - enableval=$enable_mmanon; case "${enableval}" in - yes) enable_mmanon="yes" ;; - no) enable_mmanon="no" ;; - *) as_fn_error $? "bad value ${enableval} for --enable-mmanon" "$LINENO" 5 ;; - esac -else - enable_mmanon=no - -fi - - if test x$enable_mmanon = xyes; then - ENABLE_MMANON_TRUE= - ENABLE_MMANON_FALSE='#' -else - ENABLE_MMANON_TRUE='#' - ENABLE_MMANON_FALSE= + ENABLE_MMRFC5424ADDHMAC_TRUE='#' + ENABLE_MMRFC5424ADDHMAC_FALSE= fi @@ -18392,12 +18709,12 @@ if test -n "$RELP_CFLAGS"; then pkg_cv_RELP_CFLAGS="$RELP_CFLAGS" elif test -n "$PKG_CONFIG"; then if test -n "$PKG_CONFIG" && \ - { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"relp >= 1.0.3\""; } >&5 - ($PKG_CONFIG --exists --print-errors "relp >= 1.0.3") 2>&5 + { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"relp >= 1.2.5\""; } >&5 + ($PKG_CONFIG --exists --print-errors "relp >= 1.2.5") 2>&5 ac_status=$? $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 test $ac_status = 0; }; then - pkg_cv_RELP_CFLAGS=`$PKG_CONFIG --cflags "relp >= 1.0.3" 2>/dev/null` + pkg_cv_RELP_CFLAGS=`$PKG_CONFIG --cflags "relp >= 1.2.5" 2>/dev/null` test "x$?" != "x0" && pkg_failed=yes else pkg_failed=yes @@ -18409,12 +18726,12 @@ if test -n "$RELP_LIBS"; then pkg_cv_RELP_LIBS="$RELP_LIBS" elif test -n "$PKG_CONFIG"; then if test -n "$PKG_CONFIG" && \ - { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"relp >= 1.0.3\""; } >&5 - ($PKG_CONFIG --exists --print-errors "relp >= 1.0.3") 2>&5 + { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"relp >= 1.2.5\""; } >&5 + ($PKG_CONFIG --exists --print-errors "relp >= 1.2.5") 2>&5 ac_status=$? $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 test $ac_status = 0; }; then - pkg_cv_RELP_LIBS=`$PKG_CONFIG --libs "relp >= 1.0.3" 2>/dev/null` + pkg_cv_RELP_LIBS=`$PKG_CONFIG --libs "relp >= 1.2.5" 2>/dev/null` test "x$?" != "x0" && pkg_failed=yes else pkg_failed=yes @@ -18435,14 +18752,14 @@ else _pkg_short_errors_supported=no fi if test $_pkg_short_errors_supported = yes; then - RELP_PKG_ERRORS=`$PKG_CONFIG --short-errors --print-errors --cflags --libs "relp >= 1.0.3" 2>&1` + RELP_PKG_ERRORS=`$PKG_CONFIG --short-errors --print-errors --cflags --libs "relp >= 1.2.5" 2>&1` else - RELP_PKG_ERRORS=`$PKG_CONFIG --print-errors --cflags --libs "relp >= 1.0.3" 2>&1` + RELP_PKG_ERRORS=`$PKG_CONFIG --print-errors --cflags --libs "relp >= 1.2.5" 2>&1` fi # Put the nasty error message in config.log where it belongs echo "$RELP_PKG_ERRORS" >&5 - as_fn_error $? "Package requirements (relp >= 1.0.3) were not met: + as_fn_error $? "Package requirements (relp >= 1.2.5) were not met: $RELP_PKG_ERRORS @@ -18600,6 +18917,7 @@ else fi + # RFC 3195 support # Check whether --enable-rfc3195 was given. if test "${enable_rfc3195+set}" = set; then : @@ -18623,12 +18941,12 @@ if test -n "$LIBLOGGING_CFLAGS"; then pkg_cv_LIBLOGGING_CFLAGS="$LIBLOGGING_CFLAGS" elif test -n "$PKG_CONFIG"; then if test -n "$PKG_CONFIG" && \ - { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"liblogging >= 0.7.1\""; } >&5 - ($PKG_CONFIG --exists --print-errors "liblogging >= 0.7.1") 2>&5 + { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"liblogging-rfc3195 >= 1.0.1\""; } >&5 + ($PKG_CONFIG --exists --print-errors "liblogging-rfc3195 >= 1.0.1") 2>&5 ac_status=$? $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 test $ac_status = 0; }; then - pkg_cv_LIBLOGGING_CFLAGS=`$PKG_CONFIG --cflags "liblogging >= 0.7.1" 2>/dev/null` + pkg_cv_LIBLOGGING_CFLAGS=`$PKG_CONFIG --cflags "liblogging-rfc3195 >= 1.0.1" 2>/dev/null` test "x$?" != "x0" && pkg_failed=yes else pkg_failed=yes @@ -18640,12 +18958,12 @@ if test -n "$LIBLOGGING_LIBS"; then pkg_cv_LIBLOGGING_LIBS="$LIBLOGGING_LIBS" elif test -n "$PKG_CONFIG"; then if test -n "$PKG_CONFIG" && \ - { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"liblogging >= 0.7.1\""; } >&5 - ($PKG_CONFIG --exists --print-errors "liblogging >= 0.7.1") 2>&5 + { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"liblogging-rfc3195 >= 1.0.1\""; } >&5 + ($PKG_CONFIG --exists --print-errors "liblogging-rfc3195 >= 1.0.1") 2>&5 ac_status=$? $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5 test $ac_status = 0; }; then - pkg_cv_LIBLOGGING_LIBS=`$PKG_CONFIG --libs "liblogging >= 0.7.1" 2>/dev/null` + pkg_cv_LIBLOGGING_LIBS=`$PKG_CONFIG --libs "liblogging-rfc3195 >= 1.0.1" 2>/dev/null` test "x$?" != "x0" && pkg_failed=yes else pkg_failed=yes @@ -18666,14 +18984,14 @@ else _pkg_short_errors_supported=no fi if test $_pkg_short_errors_supported = yes; then - LIBLOGGING_PKG_ERRORS=`$PKG_CONFIG --short-errors --print-errors --cflags --libs "liblogging >= 0.7.1" 2>&1` + LIBLOGGING_PKG_ERRORS=`$PKG_CONFIG --short-errors --print-errors --cflags --libs "liblogging-rfc3195 >= 1.0.1" 2>&1` else - LIBLOGGING_PKG_ERRORS=`$PKG_CONFIG --print-errors --cflags --libs "liblogging >= 0.7.1" 2>&1` + LIBLOGGING_PKG_ERRORS=`$PKG_CONFIG --print-errors --cflags --libs "liblogging-rfc3195 >= 1.0.1" 2>&1` fi # Put the nasty error message in config.log where it belongs echo "$LIBLOGGING_PKG_ERRORS" >&5 - as_fn_error $? "Package requirements (liblogging >= 0.7.1) were not met: + as_fn_error $? "Package requirements (liblogging-rfc3195 >= 1.0.1) were not met: $LIBLOGGING_PKG_ERRORS @@ -19155,6 +19473,29 @@ fi +# settings for pmciscoios +# Check whether --enable-pmciscoios was given. +if test "${enable_pmciscoios+set}" = set; then : + enableval=$enable_pmciscoios; case "${enableval}" in + yes) enable_pmciscoios="yes" ;; + no) enable_pmciscoios="no" ;; + *) as_fn_error $? "bad value ${enableval} for --enable-pmciscoios" "$LINENO" 5 ;; + esac +else + enable_pmciscoios=no + +fi + + if test x$enable_pmciscoios = xyes; then + ENABLE_PMCISCOIOS_TRUE= + ENABLE_PMCISCOIOS_FALSE='#' +else + ENABLE_PMCISCOIOS_TRUE='#' + ENABLE_PMCISCOIOS_FALSE= +fi + + + # settings for pmaixforwardedfrom # Check whether --enable-pmaixforwardedfrom was given. if test "${enable_pmaixforwardedfrom+set}" = set; then : @@ -19233,7 +19574,7 @@ if test "${enable_omruleset+set}" = set; then : *) as_fn_error $? "bad value ${enableval} for --enable-omruleset" "$LINENO" 5 ;; esac else - enable_omruleset=yes + enable_omruleset=no fi @@ -19358,10 +19699,21 @@ else fi -# -# you may want to do some library checks here - see snmp, mysql, pgsql modules -# for samples -# +if test "x$enable_omhdfs"; then + for ac_header in hdfs.h hadoop/hdfs.h +do : + as_ac_Header=`$as_echo "ac_cv_header_$ac_header" | $as_tr_sh` +ac_fn_c_check_header_mongrel "$LINENO" "$ac_header" "$as_ac_Header" "$ac_includes_default" +if eval test \"x\$"$as_ac_Header"\" = x"yes"; then : + cat >>confdefs.h <<_ACEOF +#define `$as_echo "HAVE_$ac_header" | $as_tr_cpp` 1 +_ACEOF + +fi + +done + +fi if test x$enable_omhdfs = xyes; then ENABLE_OMHDFS_TRUE= ENABLE_OMHDFS_FALSE='#' @@ -19962,7 +20314,200 @@ fi # END HIREDIS SUPPORT -ac_config_files="$ac_config_files Makefile runtime/Makefile compat/Makefile grammar/Makefile tools/Makefile doc/Makefile plugins/imudp/Makefile plugins/imtcp/Makefile plugins/im3195/Makefile plugins/imgssapi/Makefile plugins/imuxsock/Makefile plugins/imjournal/Makefile plugins/immark/Makefile plugins/imklog/Makefile plugins/imkmsg/Makefile plugins/omhdfs/Makefile plugins/omprog/Makefile plugins/omstdout/Makefile plugins/omjournal/Makefile plugins/pmrfc3164sd/Makefile plugins/pmlastmsg/Makefile plugins/pmcisconames/Makefile plugins/pmsnare/Makefile plugins/pmaixforwardedfrom/Makefile plugins/omruleset/Makefile plugins/omuxsock/Makefile plugins/imfile/Makefile plugins/imsolaris/Makefile plugins/imptcp/Makefile plugins/imttcp/Makefile plugins/impstats/Makefile plugins/imrelp/Makefile plugins/imdiag/Makefile plugins/imzmq3/Makefile plugins/omtesting/Makefile plugins/omgssapi/Makefile plugins/ommysql/Makefile plugins/ompgsql/Makefile plugins/omrelp/Makefile plugins/omlibdbi/Makefile plugins/ommail/Makefile plugins/omsnmp/Makefile plugins/omoracle/Makefile plugins/omudpspoof/Makefile plugins/ommongodb/Makefile plugins/omhiredis/Makefile plugins/omzmq3/Makefile plugins/omrabbitmq/Makefile plugins/mmnormalize/Makefile plugins/mmjsonparse/Makefile plugins/mmaudit/Makefile plugins/mmanon/Makefile plugins/omelasticsearch/Makefile plugins/sm_cust_bindcdr/Makefile plugins/mmsnmptrapd/Makefile java/Makefile tests/Makefile" + +{ $as_echo "$as_me:${as_lineno-$LINENO}: checking if required man pages already exist..." >&5 +$as_echo "$as_me: checking if required man pages already exist..." >&6;} +have_to_generate_man_pages="no" + +# man pages for libgcrypt module +if test "x$enable_usertools" = "xyes" && test "x$enable_libgcrypt" = "xyes"; then + as_ac_File=`$as_echo "ac_cv_file_"tools/rscryutil.1"" | $as_tr_sh` +{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for \"tools/rscryutil.1\"" >&5 +$as_echo_n "checking for \"tools/rscryutil.1\"... " >&6; } +if eval \${$as_ac_File+:} false; then : + $as_echo_n "(cached) " >&6 +else + test "$cross_compiling" = yes && + as_fn_error $? "cannot check for file existence when cross compiling" "$LINENO" 5 +if test -r ""tools/rscryutil.1""; then + eval "$as_ac_File=yes" +else + eval "$as_ac_File=no" +fi +fi +eval ac_res=\$$as_ac_File + { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_res" >&5 +$as_echo "$ac_res" >&6; } +if eval test \"x\$"$as_ac_File"\" = x"yes"; then : + +cat >>confdefs.h <<_ACEOF +#define `$as_echo "HAVE_"tools/rscryutil.1"" | $as_tr_cpp` 1 +_ACEOF + +else + have_to_generate_man_pages="yes" + +fi +as_ac_File=`$as_echo "ac_cv_file_"tools/rsgtutil.1"" | $as_tr_sh` +{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for \"tools/rsgtutil.1\"" >&5 +$as_echo_n "checking for \"tools/rsgtutil.1\"... " >&6; } +if eval \${$as_ac_File+:} false; then : + $as_echo_n "(cached) " >&6 +else + test "$cross_compiling" = yes && + as_fn_error $? "cannot check for file existence when cross compiling" "$LINENO" 5 +if test -r ""tools/rsgtutil.1""; then + eval "$as_ac_File=yes" +else + eval "$as_ac_File=no" +fi +fi +eval ac_res=\$$as_ac_File + { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_res" >&5 +$as_echo "$ac_res" >&6; } +if eval test \"x\$"$as_ac_File"\" = x"yes"; then : + +cat >>confdefs.h <<_ACEOF +#define `$as_echo "HAVE_"tools/rsgtutil.1"" | $as_tr_cpp` 1 +_ACEOF + +else + have_to_generate_man_pages="yes" + +fi + +fi + +# man pages for GuardTime module +if test "x$enable_usertools" = "xyes" && test "x$enable_guardtime" = "xyes"; then + as_ac_File=`$as_echo "ac_cv_file_"tools/rscryutil.1"" | $as_tr_sh` +{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for \"tools/rscryutil.1\"" >&5 +$as_echo_n "checking for \"tools/rscryutil.1\"... " >&6; } +if eval \${$as_ac_File+:} false; then : + $as_echo_n "(cached) " >&6 +else + test "$cross_compiling" = yes && + as_fn_error $? "cannot check for file existence when cross compiling" "$LINENO" 5 +if test -r ""tools/rscryutil.1""; then + eval "$as_ac_File=yes" +else + eval "$as_ac_File=no" +fi +fi +eval ac_res=\$$as_ac_File + { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_res" >&5 +$as_echo "$ac_res" >&6; } +if eval test \"x\$"$as_ac_File"\" = x"yes"; then : + +cat >>confdefs.h <<_ACEOF +#define `$as_echo "HAVE_"tools/rscryutil.1"" | $as_tr_cpp` 1 +_ACEOF + +else + have_to_generate_man_pages="yes" + +fi +as_ac_File=`$as_echo "ac_cv_file_"tools/rsgtutil.1"" | $as_tr_sh` +{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for \"tools/rsgtutil.1\"" >&5 +$as_echo_n "checking for \"tools/rsgtutil.1\"... " >&6; } +if eval \${$as_ac_File+:} false; then : + $as_echo_n "(cached) " >&6 +else + test "$cross_compiling" = yes && + as_fn_error $? "cannot check for file existence when cross compiling" "$LINENO" 5 +if test -r ""tools/rsgtutil.1""; then + eval "$as_ac_File=yes" +else + eval "$as_ac_File=no" +fi +fi +eval ac_res=\$$as_ac_File + { $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_res" >&5 +$as_echo "$ac_res" >&6; } +if eval test \"x\$"$as_ac_File"\" = x"yes"; then : + +cat >>confdefs.h <<_ACEOF +#define `$as_echo "HAVE_"tools/rsgtutil.1"" | $as_tr_cpp` 1 +_ACEOF + +else + have_to_generate_man_pages="yes" + +fi + +fi + +# Check whether --enable-generate-man-pages was given. +if test "${enable_generate_man_pages+set}" = set; then : + enableval=$enable_generate_man_pages; case "${enableval}" in + yes) have_to_generate_man_pages="yes" ;; + no) have_to_generate_man_pages="no" ;; + *) as_fn_error $? "bad value ${enableval} for --enable-generate-man-pages" "$LINENO" 5 ;; + esac +else + have_to_generate_man_pages=yes + +fi + + +if test "x$have_to_generate_man_pages" = "xyes"; then + { $as_echo "$as_me:${as_lineno-$LINENO}: result: Some man pages are missing or --enable-generate-man-pages given. We need rst2man to generate man pages from source..." >&5 +$as_echo "Some man pages are missing or --enable-generate-man-pages given. We need rst2man to generate man pages from source..." >&6; } +else + { $as_echo "$as_me:${as_lineno-$LINENO}: result: All required man pages found. We don't need rst2man!" >&5 +$as_echo "All required man pages found. We don't need rst2man!" >&6; } +fi + +if test "x$have_to_generate_man_pages" = "xyes"; then + # We need rst2man to generate our man pages + for ac_prog in rst2man rst2man.py +do + # Extract the first word of "$ac_prog", so it can be a program name with args. +set dummy $ac_prog; ac_word=$2 +{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5 +$as_echo_n "checking for $ac_word... " >&6; } +if ${ac_cv_prog_RST2MAN+:} false; then : + $as_echo_n "(cached) " >&6 +else + if test -n "$RST2MAN"; then + ac_cv_prog_RST2MAN="$RST2MAN" # Let the user override the test. +else +as_save_IFS=$IFS; IFS=$PATH_SEPARATOR +for as_dir in $PATH +do + IFS=$as_save_IFS + test -z "$as_dir" && as_dir=. + for ac_exec_ext in '' $ac_executable_extensions; do + if { test -f "$as_dir/$ac_word$ac_exec_ext" && $as_test_x "$as_dir/$ac_word$ac_exec_ext"; }; then + ac_cv_prog_RST2MAN="$ac_prog" + $as_echo "$as_me:${as_lineno-$LINENO}: found $as_dir/$ac_word$ac_exec_ext" >&5 + break 2 + fi +done + done +IFS=$as_save_IFS + +fi +fi +RST2MAN=$ac_cv_prog_RST2MAN +if test -n "$RST2MAN"; then + { $as_echo "$as_me:${as_lineno-$LINENO}: result: $RST2MAN" >&5 +$as_echo "$RST2MAN" >&6; } +else + { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5 +$as_echo "no" >&6; } +fi + + + test -n "$RST2MAN" && break +done + + if test -z "$RST2MAN"; then + as_fn_error $? "rst2man is required to build man pages. You can use the release tarball with pregenerated man pages to avoid this depedency. Use the --disable-generate-man-pages configure option in that case." "$LINENO" 5 + fi +fi + +ac_config_files="$ac_config_files Makefile runtime/Makefile compat/Makefile grammar/Makefile tools/Makefile plugins/imudp/Makefile plugins/imtcp/Makefile plugins/im3195/Makefile plugins/imgssapi/Makefile plugins/imuxsock/Makefile plugins/imjournal/Makefile plugins/immark/Makefile plugins/imklog/Makefile plugins/imkmsg/Makefile plugins/omhdfs/Makefile plugins/omprog/Makefile plugins/mmexternal/Makefile plugins/omstdout/Makefile plugins/omjournal/Makefile plugins/pmrfc3164sd/Makefile plugins/pmlastmsg/Makefile plugins/pmcisconames/Makefile plugins/pmciscoios/Makefile plugins/pmsnare/Makefile plugins/pmaixforwardedfrom/Makefile plugins/omruleset/Makefile plugins/omuxsock/Makefile plugins/imfile/Makefile plugins/imsolaris/Makefile plugins/imptcp/Makefile plugins/imttcp/Makefile plugins/impstats/Makefile plugins/imrelp/Makefile plugins/imdiag/Makefile plugins/imzmq3/Makefile plugins/omtesting/Makefile plugins/omgssapi/Makefile plugins/ommysql/Makefile plugins/ompgsql/Makefile plugins/omrelp/Makefile plugins/omlibdbi/Makefile plugins/ommail/Makefile plugins/omsnmp/Makefile plugins/omoracle/Makefile plugins/omudpspoof/Makefile plugins/ommongodb/Makefile plugins/omhiredis/Makefile plugins/omzmq3/Makefile plugins/omrabbitmq/Makefile plugins/mmnormalize/Makefile plugins/mmjsonparse/Makefile plugins/mmaudit/Makefile plugins/mmanon/Makefile plugins/mmutf8fix/Makefile plugins/mmcount/Makefile plugins/mmsequence/Makefile plugins/mmfields/Makefile plugins/mmpstrucdata/Makefile plugins/mmrfc5424addhmac/Makefile plugins/omelasticsearch/Makefile plugins/sm_cust_bindcdr/Makefile plugins/mmsnmptrapd/Makefile java/Makefile tests/Makefile" cat >confcache <<\_ACEOF # This file is a shell script that caches the results of configure @@ -20141,6 +20686,10 @@ if test -z "${ENABLE_INET_TRUE}" && test -z "${ENABLE_INET_FALSE}"; then as_fn_error $? "conditional \"ENABLE_INET\" was never defined. Usually this means the macro was only invoked conditionally." "$LINENO" 5 fi +if test -z "${ENABLE_JEMALLOC_TRUE}" && test -z "${ENABLE_JEMALLOC_FALSE}"; then + as_fn_error $? "conditional \"ENABLE_JEMALLOC\" was never defined. +Usually this means the macro was only invoked conditionally." "$LINENO" 5 +fi if test -z "${HAVE_SYSTEMD_TRUE}" && test -z "${HAVE_SYSTEMD_FALSE}"; then as_fn_error $? "conditional \"HAVE_SYSTEMD\" was never defined. Usually this means the macro was only invoked conditionally." "$LINENO" 5 @@ -20229,6 +20778,30 @@ if test -z "${ENABLE_MMANON_TRUE}" && test -z "${ENABLE_MMANON_FALSE}"; then as_fn_error $? "conditional \"ENABLE_MMANON\" was never defined. Usually this means the macro was only invoked conditionally." "$LINENO" 5 fi +if test -z "${ENABLE_MMUTF8FIX_TRUE}" && test -z "${ENABLE_MMUTF8FIX_FALSE}"; then + as_fn_error $? "conditional \"ENABLE_MMUTF8FIX\" was never defined. +Usually this means the macro was only invoked conditionally." "$LINENO" 5 +fi +if test -z "${ENABLE_MMCOUNT_TRUE}" && test -z "${ENABLE_MMCOUNT_FALSE}"; then + as_fn_error $? "conditional \"ENABLE_MMCOUNT\" was never defined. +Usually this means the macro was only invoked conditionally." "$LINENO" 5 +fi +if test -z "${ENABLE_MMSEQUENCE_TRUE}" && test -z "${ENABLE_MMSEQUENCE_FALSE}"; then + as_fn_error $? "conditional \"ENABLE_MMSEQUENCE\" was never defined. +Usually this means the macro was only invoked conditionally." "$LINENO" 5 +fi +if test -z "${ENABLE_MMFIELDS_TRUE}" && test -z "${ENABLE_MMFIELDS_FALSE}"; then + as_fn_error $? "conditional \"ENABLE_MMFIELDS\" was never defined. +Usually this means the macro was only invoked conditionally." "$LINENO" 5 +fi +if test -z "${ENABLE_MMPSTRUCDATA_TRUE}" && test -z "${ENABLE_MMPSTRUCDATA_FALSE}"; then + as_fn_error $? "conditional \"ENABLE_MMPSTRUCDATA\" was never defined. +Usually this means the macro was only invoked conditionally." "$LINENO" 5 +fi +if test -z "${ENABLE_MMRFC5424ADDHMAC_TRUE}" && test -z "${ENABLE_MMRFC5424ADDHMAC_FALSE}"; then + as_fn_error $? "conditional \"ENABLE_MMRFC5424ADDHMAC\" was never defined. +Usually this means the macro was only invoked conditionally." "$LINENO" 5 +fi if test -z "${ENABLE_RELP_TRUE}" && test -z "${ENABLE_RELP_FALSE}"; then as_fn_error $? "conditional \"ENABLE_RELP\" was never defined. Usually this means the macro was only invoked conditionally." "$LINENO" 5 @@ -20289,6 +20862,10 @@ if test -z "${ENABLE_PMCISCONAMES_TRUE}" && test -z "${ENABLE_PMCISCONAMES_FALSE as_fn_error $? "conditional \"ENABLE_PMCISCONAMES\" was never defined. Usually this means the macro was only invoked conditionally." "$LINENO" 5 fi +if test -z "${ENABLE_PMCISCOIOS_TRUE}" && test -z "${ENABLE_PMCISCOIOS_FALSE}"; then + as_fn_error $? "conditional \"ENABLE_PMCISCOIOS\" was never defined. +Usually this means the macro was only invoked conditionally." "$LINENO" 5 +fi if test -z "${ENABLE_PMAIXFORWARDEDFROM_TRUE}" && test -z "${ENABLE_PMAIXFORWARDEDFROM_FALSE}"; then as_fn_error $? "conditional \"ENABLE_PMAIXFORWARDEDFROM\" was never defined. Usually this means the macro was only invoked conditionally." "$LINENO" 5 @@ -20754,7 +21331,7 @@ cat >>$CONFIG_STATUS <<\_ACEOF || ac_write_fail=1 # report actual input values of CONFIG_FILES etc. instead of their # values after options handling. ac_log=" -This file was extended by rsyslog $as_me 7.3.14, which was +This file was extended by rsyslog $as_me 8.4.1, which was generated by GNU Autoconf 2.68. Invocation command line was CONFIG_FILES = $CONFIG_FILES @@ -20820,7 +21397,7 @@ _ACEOF cat >>$CONFIG_STATUS <<_ACEOF || ac_write_fail=1 ac_cs_config="`$as_echo "$ac_configure_args" | sed 's/^ //; s/[\\""\`\$]/\\\\&/g'`" ac_cs_version="\\ -rsyslog config.status 7.3.14 +rsyslog config.status 8.4.1 configured by $0, generated by GNU Autoconf 2.68, with options \\"\$ac_cs_config\\" @@ -21234,7 +21811,6 @@ do "compat/Makefile") CONFIG_FILES="$CONFIG_FILES compat/Makefile" ;; "grammar/Makefile") CONFIG_FILES="$CONFIG_FILES grammar/Makefile" ;; "tools/Makefile") CONFIG_FILES="$CONFIG_FILES tools/Makefile" ;; - "doc/Makefile") CONFIG_FILES="$CONFIG_FILES doc/Makefile" ;; "plugins/imudp/Makefile") CONFIG_FILES="$CONFIG_FILES plugins/imudp/Makefile" ;; "plugins/imtcp/Makefile") CONFIG_FILES="$CONFIG_FILES plugins/imtcp/Makefile" ;; "plugins/im3195/Makefile") CONFIG_FILES="$CONFIG_FILES plugins/im3195/Makefile" ;; @@ -21246,11 +21822,13 @@ do "plugins/imkmsg/Makefile") CONFIG_FILES="$CONFIG_FILES plugins/imkmsg/Makefile" ;; "plugins/omhdfs/Makefile") CONFIG_FILES="$CONFIG_FILES plugins/omhdfs/Makefile" ;; "plugins/omprog/Makefile") CONFIG_FILES="$CONFIG_FILES plugins/omprog/Makefile" ;; + "plugins/mmexternal/Makefile") CONFIG_FILES="$CONFIG_FILES plugins/mmexternal/Makefile" ;; "plugins/omstdout/Makefile") CONFIG_FILES="$CONFIG_FILES plugins/omstdout/Makefile" ;; "plugins/omjournal/Makefile") CONFIG_FILES="$CONFIG_FILES plugins/omjournal/Makefile" ;; "plugins/pmrfc3164sd/Makefile") CONFIG_FILES="$CONFIG_FILES plugins/pmrfc3164sd/Makefile" ;; "plugins/pmlastmsg/Makefile") CONFIG_FILES="$CONFIG_FILES plugins/pmlastmsg/Makefile" ;; "plugins/pmcisconames/Makefile") CONFIG_FILES="$CONFIG_FILES plugins/pmcisconames/Makefile" ;; + "plugins/pmciscoios/Makefile") CONFIG_FILES="$CONFIG_FILES plugins/pmciscoios/Makefile" ;; "plugins/pmsnare/Makefile") CONFIG_FILES="$CONFIG_FILES plugins/pmsnare/Makefile" ;; "plugins/pmaixforwardedfrom/Makefile") CONFIG_FILES="$CONFIG_FILES plugins/pmaixforwardedfrom/Makefile" ;; "plugins/omruleset/Makefile") CONFIG_FILES="$CONFIG_FILES plugins/omruleset/Makefile" ;; @@ -21281,6 +21859,12 @@ do "plugins/mmjsonparse/Makefile") CONFIG_FILES="$CONFIG_FILES plugins/mmjsonparse/Makefile" ;; "plugins/mmaudit/Makefile") CONFIG_FILES="$CONFIG_FILES plugins/mmaudit/Makefile" ;; "plugins/mmanon/Makefile") CONFIG_FILES="$CONFIG_FILES plugins/mmanon/Makefile" ;; + "plugins/mmutf8fix/Makefile") CONFIG_FILES="$CONFIG_FILES plugins/mmutf8fix/Makefile" ;; + "plugins/mmcount/Makefile") CONFIG_FILES="$CONFIG_FILES plugins/mmcount/Makefile" ;; + "plugins/mmsequence/Makefile") CONFIG_FILES="$CONFIG_FILES plugins/mmsequence/Makefile" ;; + "plugins/mmfields/Makefile") CONFIG_FILES="$CONFIG_FILES plugins/mmfields/Makefile" ;; + "plugins/mmpstrucdata/Makefile") CONFIG_FILES="$CONFIG_FILES plugins/mmpstrucdata/Makefile" ;; + "plugins/mmrfc5424addhmac/Makefile") CONFIG_FILES="$CONFIG_FILES plugins/mmrfc5424addhmac/Makefile" ;; "plugins/omelasticsearch/Makefile") CONFIG_FILES="$CONFIG_FILES plugins/omelasticsearch/Makefile" ;; "plugins/sm_cust_bindcdr/Makefile") CONFIG_FILES="$CONFIG_FILES plugins/sm_cust_bindcdr/Makefile" ;; "plugins/mmsnmptrapd/Makefile") CONFIG_FILES="$CONFIG_FILES plugins/mmsnmptrapd/Makefile" ;; @@ -22655,11 +23239,14 @@ echo " Zlib compression support enabled: $enable_zlib" echo " rsyslog runtime will be built: $enable_rsyslogrt" echo " rsyslogd will be built: $enable_rsyslogd" echo " GUI components will be built: $enable_gui" +echo " have to generate man pages: $have_to_generate_man_pages" echo " Unlimited select() support enabled: $enable_unlimited_select" echo " uuid support enabled: $enable_uuid" echo " Log file signing support: $enable_guardtime" echo " Log file encryption support: $enable_libgcrypt" echo " anonymization support enabled: $enable_mmanon" +echo " message counting support enabled: $enable_mmcount" +echo " mmfields enabled: $enable_mmfields" echo echo "---{ input plugins }---" echo " Klog functionality enabled: $enable_klog ($os_type)" @@ -22690,6 +23277,7 @@ echo "---{ parser modules }---" echo " pmrfc3164sd module will be compiled: $enable_pmrfc3164sd" echo " pmlastmsg module will be compiled: $enable_pmlastmsg" echo " pmcisconames module will be compiled: $enable_pmcisconames" +echo " pmciscoios module will be compiled: $enable_pmciscoios" echo " pmaixforwardedfrom module w.be compiled: $enable_pmaixforwardedfrom" echo " pmsnare module will be compiled: $enable_pmsnare" echo @@ -22698,6 +23286,10 @@ echo " mmnormalize module will be compiled: $enable_mmnormalize" echo " mmjsonparse module will be compiled: $enable_mmjsonparse" echo " mmjaduit module will be compiled: $enable_mmaudit" echo " mmsnmptrapd module will be compiled: $enable_mmsnmptrapd" +echo " mmutf8fix enabled: $enable_mmutf8fix" +echo " mmrfc5424addhmac enabled: $enable_mmrfc5424addhmac" +echo " mmpstrucdata enabled: $enable_mmpstrucdata" +echo " mmsequence enabled: $enable_mmsequence" echo echo "---{ strgen modules }---" echo " sm_cust_bindcdr module will be compiled: $enable_sm_cust_bindcdr" diff --git a/configure.ac b/configure.ac index e18a0a7..b6fc6e6 100644 --- a/configure.ac +++ b/configure.ac @@ -2,8 +2,8 @@ # Process this file with autoconf to produce a configure script. AC_PREREQ(2.61) -AC_INIT([rsyslog],[7.3.14],[rsyslog@lists.adiscon.com]) -AM_INIT_AUTOMAKE +AC_INIT([rsyslog],[8.4.1],[rsyslog@lists.adiscon.com]) +AM_INIT_AUTOMAKE([subdir-objects]) m4_ifdef([AM_SILENT_RULES], [AM_SILENT_RULES([yes])]) @@ -22,20 +22,25 @@ AC_PROG_LEX AC_PROG_YACC AC_PROG_CC AM_PROG_CC_C_O -if test "$GCC" = "yes" -then CFLAGS="$CFLAGS -W -Wall -Wformat-security -Wshadow -Wcast-align -Wpointer-arith -Wmissing-format-attribute -g" -fi AC_DISABLE_STATIC AC_PROG_LIBTOOL AC_CANONICAL_HOST -AC_PATH_PROG([RST2MAN], [rst2man]) PKG_PROG_PKG_CONFIG # modules we require -PKG_CHECK_MODULES(LIBESTR, libestr >= 0.1.5) -PKG_CHECK_MODULES(LIBEE, libee >= 0.4.0) -PKG_CHECK_MODULES([JSON_C], [json]) +PKG_CHECK_MODULES(LIBESTR, libestr >= 0.1.9) +PKG_CHECK_MODULES(LIBLOGGING_STDLOG, liblogging-stdlog >= 1.0.3) +PKG_CHECK_MODULES([JSON_C], [json],, [ + PKG_CHECK_MODULES([JSON_C], [json-c],,) +]) + +# if int64 is supported, use it +AC_CHECK_LIB(json-c, json_object_new_object,,) +AC_CHECK_FUNCS(json_object_new_int64,,) + +# look for newer API +AC_CHECK_FUNCS(json_tokener_error_desc,,) case "${host}" in *-*-linux*) @@ -85,7 +90,7 @@ AC_SUBST(DL_LIBS) AC_HEADER_RESOLV AC_HEADER_STDC AC_HEADER_SYS_WAIT -AC_CHECK_HEADERS([arpa/inet.h libgen.h malloc.h fcntl.h locale.h netdb.h netinet/in.h paths.h stddef.h stdlib.h string.h sys/file.h sys/ioctl.h sys/param.h sys/socket.h sys/time.h sys/stat.h syslog.h unistd.h utmp.h utmpx.h sys/epoll.h sys/prctl.h]) +AC_CHECK_HEADERS([arpa/inet.h libgen.h malloc.h fcntl.h locale.h netdb.h netinet/in.h paths.h stddef.h stdlib.h string.h sys/file.h sys/ioctl.h sys/param.h sys/socket.h sys/time.h sys/stat.h sys/inotify.h unistd.h utmp.h utmpx.h sys/epoll.h sys/prctl.h]) # Checks for typedefs, structures, and compiler characteristics. AC_C_CONST @@ -100,7 +105,6 @@ AC_TYPE_UINT8_T AC_HEADER_TIME AC_STRUCT_TM AC_C_VOLATILE -AC_C_TYPEOF sa_includes="\ $ac_includes_default @@ -121,7 +125,8 @@ AC_TYPE_SIGNAL AC_FUNC_STAT AC_FUNC_STRERROR_R AC_FUNC_VPRINTF -AC_CHECK_FUNCS([flock basename alarm clock_gettime gethostbyname gethostname gettimeofday localtime_r memset mkdir regcomp select setid socket strcasecmp strchr strdup strerror strndup strnlen strrchr strstr strtol strtoul uname ttyname_r getline malloc_trim prctl epoll_create epoll_create1 fdatasync syscall lseek64]) +AC_CHECK_FUNCS([flock inotify_init recvmmsg basename alarm clock_gettime gethostbyname gethostname gettimeofday localtime_r memset mkdir regcomp select setsid socket strcasecmp strchr strdup strerror strndup strnlen strrchr strstr strtol strtoul uname ttyname_r getline malloc_trim prctl epoll_create epoll_create1 fdatasync syscall lseek64]) +AC_CHECK_TYPES([off64_t]) # getifaddrs is in libc (mostly) or in libsocket (eg Solaris 11) or not defined (eg Solaris 10) AC_SEARCH_LIBS([getifaddrs], [socket], [AC_DEFINE(HAVE_GETIFADDRS, [1], [set define])]) @@ -153,6 +158,14 @@ AC_TRY_COMPILE([ AC_MSG_RESULT(no; defined as 64) ) +# Check for __builtin_expect() +AC_MSG_CHECKING([for __builtin_expect()]) +AC_LINK_IFELSE([AC_LANG_PROGRAM(, return __builtin_expect(main != 0, 1))], + [AC_DEFINE(HAVE_BUILTIN_EXPECT, 1, + Define to 1 if compiler supports __builtin_expect) + AC_MSG_RESULT([yes])], + [AC_MSG_RESULT([no])]) + # check for availability of atomic operations RS_ATOMIC_OPERATIONS RS_ATOMIC_OPERATIONS_64BIT @@ -367,27 +380,28 @@ if test "$enable_inet" = "yes"; then AC_DEFINE(SYSLOG_INET, 1, [network support is integrated.]) fi - -# -# The following define determines whether the package adheres to the -# file system standard. -# -AC_MSG_CHECKING(for FSSTND support) -AC_ARG_ENABLE([fsstnd], - [AS_HELP_STRING([--disable-fsstnd], [Disable support for FSSTND])], - [ - if test "x${enableval}" = "xyes"; then - AC_MSG_RESULT([yes]) - AC_DEFINE([FSSTND], [1], [Description]) - else - AC_MSG_RESULT([no]) - fi - ], - [ - # enabled by default - AC_MSG_RESULT([yes]) - AC_DEFINE([FSSTND], [1], [Description]) - ]) +# jemalloc +AC_ARG_ENABLE(jemalloc, + [AS_HELP_STRING([--enable-jemalloc],[Enable jemalloc support @<:@default=no@:>@])], + [case "${enableval}" in + yes) enable_jemalloc="yes" ;; + no) enable_jemalloc="no" ;; + *) AC_MSG_ERROR(bad value ${enableval} for --enable-jemalloc) ;; + esac], + [enable_jemalloc="no"] +) +AM_CONDITIONAL(ENABLE_JEMALLOC, test x$enable_jemalloc = xyes) +if test "$enable_jemalloc" = "yes"; then + AC_CHECK_LIB( + [jemalloc], + [malloc_stats_print], + [RT_LIBS="$RT_LIBS -ljemalloc" + AC_DEFINE(HAVE_JEMALLOC, 1, [jemalloc support is integrated.]) + ], + [AC_MSG_FAILURE([jemalloc library is missing])], + [] + ) +fi # support for unlimited select() syscall @@ -784,22 +798,18 @@ AC_ARG_ENABLE(libgcrypt, [enable_libgcrypt=yes] ) if test "x$enable_libgcrypt" = "xyes"; then - AC_CHECK_PROG( - [HAVE_LIBGCRYPT_CONFIG], - [libgcrypt-config], - [yes],,, - ) - if test "x${HAVE_LIBGCRYPT_CONFIG}" != "xyes"; then + AC_PATH_PROG([LIBGCRYPT_CONFIG],[libgcrypt-config],[no]) + if test "x${LIBGCRYPT_CONFIG}" = "xno"; then AC_MSG_FAILURE([libgcrypt-config not found in PATH]) fi AC_CHECK_LIB( [gcrypt], [gcry_cipher_open], - [LIBGCRYPT_CFLAGS="`libgcrypt-config --cflags`" - LIBGCRYPT_LIBS="`libgcrypt-config --libs`" + [LIBGCRYPT_CFLAGS="`${LIBGCRYPT_CONFIG} --cflags`" + LIBGCRYPT_LIBS="`${LIBGCRYPT_CONFIG} --libs`" ], [AC_MSG_FAILURE([libgcrypt is missing])], - [`libgcrypt-config --libs --cflags`] + [`${LIBGCRYPT_CONFIG} --libs --cflags`] ) AC_DEFINE([ENABLE_LIBGCRYPT], [1], [Indicator that LIBGCRYPT is present]) fi @@ -823,7 +833,10 @@ if test "x$enable_rsyslogrt" = "xyes"; then RSRT_LIBS1="\$(top_builddir)/runtime/librsyslog.la" fi AM_CONDITIONAL(ENABLE_RSYSLOGRT, test x$enable_rsyslogrt = xyes) -RSRT_CFLAGS="\$(RSRT_CFLAGS1) \$(LIBESTR_CFLAGS) \$(JSON_C_FLAGS)" +RSRT_CFLAGS="\$(RSRT_CFLAGS1) \$(LIBESTR_CFLAGS) \$(JSON_C_CFLAGS)" +if test "$GCC" = "yes" +then RSRT_CFLAGS="$RSRT_CFLAGS -W -std=c99 -Wall -Wformat-security -Wshadow -Wcast-align -Wpointer-arith -Wmissing-format-attribute -g" +fi RSRT_LIBS="\$(RSRT_LIBS1) \$(LIBESTR_LIBS) \$(JSON_C_LIBS)" AC_SUBST(RSRT_CFLAGS1) AC_SUBST(RSRT_LIBS1) @@ -891,11 +904,9 @@ AC_ARG_ENABLE(mail, AM_CONDITIONAL(ENABLE_MAIL, test x$enable_mail = xyes) -# imdiag support (so far we do not need a library, but we need to turn this on and off) -# note that we enable this be default, because an important point is to make -# it available to users who do not know much about how to handle things. It -# would complicate things if we first needed to tell them how to enable imdiag. -# rgerhards, 2008-07-25 +# imdiag support +# This is a core testbench tool. You need to enable it if you want to +# use not only a small subset of the testbench. AC_ARG_ENABLE(imdiag, [AS_HELP_STRING([--enable-imdiag],[Enable imdiag @<:@default=no@:>@])], [case "${enableval}" in @@ -905,6 +916,9 @@ AC_ARG_ENABLE(imdiag, esac], [enable_imdiag=no] ) +if test "x$enable_imdiag" = "xyes"; then + AC_DEFINE([ENABLE_IMDIAG], [1], [Indicator that IMDIAG is present]) +fi AM_CONDITIONAL(ENABLE_IMDIAG, test x$enable_imdiag = xyes) @@ -919,7 +933,7 @@ AC_ARG_ENABLE(mmnormalize, [enable_mmnormalize=no] ) if test "x$enable_mmnormalize" = "xyes"; then - PKG_CHECK_MODULES(LIBLOGNORM, lognorm >= 0.3.1) + PKG_CHECK_MODULES(LIBLOGNORM, lognorm >= 1.0.0) fi AM_CONDITIONAL(ENABLE_MMNORMALIZE, test x$enable_mmnormalize = xyes) @@ -934,9 +948,6 @@ AC_ARG_ENABLE(mmjsonparse, esac], [enable_mmjsonparse=no] ) -if test "x$enable_mmjsonparse" = "xyes"; then - PKG_CHECK_MODULES(LIBLOGNORM, lognorm >= 0.3.1) -fi AM_CONDITIONAL(ENABLE_MMJSONPARSE, test x$enable_mmjsonparse = xyes) @@ -950,9 +961,6 @@ AC_ARG_ENABLE(mmaudit, esac], [enable_mmaudit=no] ) -if test "x$enable_mmaudit" = "xyes"; then - PKG_CHECK_MODULES(LIBLOGNORM, lognorm >= 0.3.1) -fi AM_CONDITIONAL(ENABLE_MMAUDIT, test x$enable_mmaudit = xyes) @@ -969,6 +977,89 @@ AC_ARG_ENABLE(mmanon, AM_CONDITIONAL(ENABLE_MMANON, test x$enable_mmanon = xyes) +# mmutf8fix +AC_ARG_ENABLE(mmutf8fix, + [AS_HELP_STRING([--enable-mmutf8fix],[Enable building mmutf8fix support @<:@default=no@:>@])], + [case "${enableval}" in + yes) enable_mmutf8fix="yes" ;; + no) enable_mmutf8fix="no" ;; + *) AC_MSG_ERROR(bad value ${enableval} for --enable-mmutf8fix) ;; + esac], + [enable_mmutf8fix=no] +) +AM_CONDITIONAL(ENABLE_MMUTF8FIX, test x$enable_mmutf8fix = xyes) + + +# mmcount +AC_ARG_ENABLE(mmcount, + [AS_HELP_STRING([--enable-mmcount],[Enable message counting @<:@default=no@:>@])], + [case "${enableval}" in + yes) enable_mmcount="yes" ;; + no) enable_mmcount="no" ;; + *) AC_MSG_ERROR(bad value ${enableval} for --enable-mmcount) ;; + esac], + [enable_mmcount=no] +) +AM_CONDITIONAL(ENABLE_MMCOUNT, test x$enable_mmcount = xyes) + + +# mmsequence +AC_ARG_ENABLE(mmsequence, + [AS_HELP_STRING([--enable-mmsequence],[Enable sequence generator @<:@default=no@:>@])], + [case "${enableval}" in + yes) enable_mmsequence="yes" ;; + no) enable_mmsequence="no" ;; + *) AC_MSG_ERROR(bad value ${enableval} for --enable-mmsequence) ;; + esac], + [enable_mmsequence=no] +) +AM_CONDITIONAL(ENABLE_MMSEQUENCE, test x$enable_mmsequence = xyes) + + +# mmfields +AC_ARG_ENABLE(mmfields, + [AS_HELP_STRING([--enable-mmfields],[Enable building mmfields support @<:@default=no@:>@])], + [case "${enableval}" in + yes) enable_mmfields="yes" ;; + no) enable_mmfields="no" ;; + *) AC_MSG_ERROR(bad value ${enableval} for --enable-mmfields) ;; + esac], + [enable_mmfields=no] +) +AM_CONDITIONAL(ENABLE_MMFIELDS, test x$enable_mmfields = xyes) + +# mmpstrucdata +AC_ARG_ENABLE(mmpstrucdata, + [AS_HELP_STRING([--enable-mmpstrucdata],[Enable building mmpstrucdata support @<:@default=no@:>@])], + [case "${enableval}" in + yes) enable_mmpstrucdata="yes" ;; + no) enable_mmpstrucdata="no" ;; + *) AC_MSG_ERROR(bad value ${enableval} for --enable-mmpstrucdata) ;; + esac], + [enable_mmpstrucdata=no] +) +AM_CONDITIONAL(ENABLE_MMPSTRUCDATA, test x$enable_mmpstrucdata = xyes) + + +# mmrfc5424addhmac +AC_ARG_ENABLE(mmrfc5424addhmac, + [AS_HELP_STRING([--enable-mmrfc5424addhmac],[Enable building mmrfc5424addhmac support @<:@default=no@:>@])], + [case "${enableval}" in + yes) enable_mmrfc5424addhmac="yes" ;; + no) enable_mmrfc5424addhmac="no" ;; + *) AC_MSG_ERROR(bad value ${enableval} for --enable-mmrfc5424addhmac) ;; + esac], + [enable_mmrfc5424addhmac=no] +) +if test "x$enable_mmrfc5424addhmac" = "xyes"; then + PKG_CHECK_MODULES(OPENSSL, openssl >= 0.9.7) +#AC_CHECK_LIB([crypto],[CRYPTO_new_ex_data], [], [AC_MSG_ERROR([OpenSSL libraries required])]) +#AC_CHECK_LIB([ssl],[SSL_library_init], [], [AC_MSG_ERROR([OpenSSL libraries required])]) +#AC_CHECK_HEADERS([openssl/crypto.h openssl/x509.h openssl/pem.h openssl/ssl.h openssl/err.h],[],[AC_MSG_ERROR([OpenSSL headers required])]) +fi +AM_CONDITIONAL(ENABLE_MMRFC5424ADDHMAC, test x$enable_mmrfc5424addhmac = xyes) + + # RELP support AC_ARG_ENABLE(relp, [AS_HELP_STRING([--enable-relp],[Enable RELP support @<:@default=no@:>@])], @@ -980,7 +1071,7 @@ AC_ARG_ENABLE(relp, [enable_relp=no] ) if test "x$enable_relp" = "xyes"; then - PKG_CHECK_MODULES(RELP, relp >= 1.0.3) + PKG_CHECK_MODULES(RELP, relp >= 1.2.5) fi AM_CONDITIONAL(ENABLE_RELP, test x$enable_relp = xyes) @@ -1000,6 +1091,7 @@ if test "x$enable_guardtime" = "xyes"; then fi AM_CONDITIONAL(ENABLE_GUARDTIME, test x$enable_guardtime = xyes) + # RFC 3195 support AC_ARG_ENABLE(rfc3195, [AS_HELP_STRING([--enable-rfc3195],[Enable RFC3195 support @<:@default=no@:>@])], @@ -1011,7 +1103,7 @@ AC_ARG_ENABLE(rfc3195, [enable_rfc3195=no] ) if test "x$enable_rfc3195" = "xyes"; then - PKG_CHECK_MODULES(LIBLOGGING, liblogging >= 0.7.1) + PKG_CHECK_MODULES(LIBLOGGING, liblogging-rfc3195 >= 1.0.1) fi AM_CONDITIONAL(ENABLE_RFC3195, test x$enable_rfc3195 = xyes) @@ -1191,6 +1283,19 @@ AC_ARG_ENABLE(pmcisconames, AM_CONDITIONAL(ENABLE_PMCISCONAMES, test x$enable_pmcisconames = xyes) +# settings for pmciscoios +AC_ARG_ENABLE(pmciscoios, + [AS_HELP_STRING([--enable-pmciscoios],[Compiles ciscoios parser module @<:@default=no@:>@])], + [case "${enableval}" in + yes) enable_pmciscoios="yes" ;; + no) enable_pmciscoios="no" ;; + *) AC_MSG_ERROR(bad value ${enableval} for --enable-pmciscoios) ;; + esac], + [enable_pmciscoios=no] +) +AM_CONDITIONAL(ENABLE_PMCISCOIOS, test x$enable_pmciscoios = xyes) + + # settings for pmaixforwardedfrom AC_ARG_ENABLE(pmaixforwardedfrom, [AS_HELP_STRING([--enable-pmaixforwardedfrom],[Compiles aixforwardedfrom parser module @<:@default=no@:>@])], @@ -1238,7 +1343,7 @@ AC_ARG_ENABLE(omruleset, no) enable_omruleset="no" ;; *) AC_MSG_ERROR(bad value ${enableval} for --enable-omruleset) ;; esac], - [enable_omruleset=yes] + [enable_omruleset=no] ) AM_CONDITIONAL(ENABLE_OMRULESET, test x$enable_omruleset = xyes) @@ -1311,10 +1416,9 @@ AC_ARG_ENABLE(omhdfs, esac], [enable_omhdfs=no] ) -# -# you may want to do some library checks here - see snmp, mysql, pgsql modules -# for samples -# +if test "x$enable_omhdfs"; then + AC_CHECK_HEADERS([hdfs.h hadoop/hdfs.h]) +fi AM_CONDITIONAL(ENABLE_OMHDFS, test x$enable_omhdfs = xyes) @@ -1408,12 +1512,55 @@ AM_CONDITIONAL(ENABLE_OMHIREDIS, test x$enable_omhiredis = xyes) # END HIREDIS SUPPORT + +AC_CHECKING([if required man pages already exist]) +have_to_generate_man_pages="no" + +# man pages for libgcrypt module +if test "x$enable_usertools" = "xyes" && test "x$enable_libgcrypt" = "xyes"; then + AC_CHECK_FILES(["tools/rscryutil.1" "tools/rsgtutil.1"], + [], + [have_to_generate_man_pages="yes"] + ) +fi + +# man pages for GuardTime module +if test "x$enable_usertools" = "xyes" && test "x$enable_guardtime" = "xyes"; then + AC_CHECK_FILES(["tools/rscryutil.1" "tools/rsgtutil.1"], + [], + [have_to_generate_man_pages="yes"] + ) +fi + +AC_ARG_ENABLE(generate-man-pages, + [AS_HELP_STRING([--enable-generate-man-pages],[Generate man pages from source @<:@default=yes@:>@])], + [case "${enableval}" in + yes) have_to_generate_man_pages="yes" ;; + no) have_to_generate_man_pages="no" ;; + *) AC_MSG_ERROR(bad value ${enableval} for --enable-generate-man-pages) ;; + esac], + [have_to_generate_man_pages=yes] +) + +if test "x$have_to_generate_man_pages" = "xyes"; then + AC_MSG_RESULT([Some man pages are missing or --enable-generate-man-pages given. We need rst2man to generate man pages from source...]) +else + AC_MSG_RESULT([All required man pages found. We don't need rst2man!]) +fi + +if test "x$have_to_generate_man_pages" = "xyes"; then + # We need rst2man to generate our man pages + AC_CHECK_PROGS([RST2MAN], [rst2man rst2man.py], []) + if test -z "$RST2MAN"; then + AC_MSG_ERROR([rst2man is required to build man pages. You can use the release tarball with pregenerated man pages to avoid this depedency. Use the --disable-generate-man-pages configure option in that case.]) + fi +fi + AC_CONFIG_FILES([Makefile \ runtime/Makefile \ compat/Makefile \ grammar/Makefile \ tools/Makefile \ - doc/Makefile \ plugins/imudp/Makefile \ plugins/imtcp/Makefile \ plugins/im3195/Makefile \ @@ -1425,11 +1572,13 @@ AC_CONFIG_FILES([Makefile \ plugins/imkmsg/Makefile \ plugins/omhdfs/Makefile \ plugins/omprog/Makefile \ + plugins/mmexternal/Makefile \ plugins/omstdout/Makefile \ plugins/omjournal/Makefile \ plugins/pmrfc3164sd/Makefile \ plugins/pmlastmsg/Makefile \ plugins/pmcisconames/Makefile \ + plugins/pmciscoios/Makefile \ plugins/pmsnare/Makefile \ plugins/pmaixforwardedfrom/Makefile \ plugins/omruleset/Makefile \ @@ -1460,6 +1609,12 @@ AC_CONFIG_FILES([Makefile \ plugins/mmjsonparse/Makefile \ plugins/mmaudit/Makefile \ plugins/mmanon/Makefile \ + plugins/mmutf8fix/Makefile \ + plugins/mmcount/Makefile \ + plugins/mmsequence/Makefile \ + plugins/mmfields/Makefile \ + plugins/mmpstrucdata/Makefile \ + plugins/mmrfc5424addhmac/Makefile \ plugins/omelasticsearch/Makefile \ plugins/sm_cust_bindcdr/Makefile \ plugins/mmsnmptrapd/Makefile \ @@ -1477,11 +1632,14 @@ echo " Zlib compression support enabled: $enable_zlib" echo " rsyslog runtime will be built: $enable_rsyslogrt" echo " rsyslogd will be built: $enable_rsyslogd" echo " GUI components will be built: $enable_gui" +echo " have to generate man pages: $have_to_generate_man_pages" echo " Unlimited select() support enabled: $enable_unlimited_select" echo " uuid support enabled: $enable_uuid" echo " Log file signing support: $enable_guardtime" echo " Log file encryption support: $enable_libgcrypt" echo " anonymization support enabled: $enable_mmanon" +echo " message counting support enabled: $enable_mmcount" +echo " mmfields enabled: $enable_mmfields" echo echo "---{ input plugins }---" echo " Klog functionality enabled: $enable_klog ($os_type)" @@ -1512,6 +1670,7 @@ echo "---{ parser modules }---" echo " pmrfc3164sd module will be compiled: $enable_pmrfc3164sd" echo " pmlastmsg module will be compiled: $enable_pmlastmsg" echo " pmcisconames module will be compiled: $enable_pmcisconames" +echo " pmciscoios module will be compiled: $enable_pmciscoios" echo " pmaixforwardedfrom module w.be compiled: $enable_pmaixforwardedfrom" echo " pmsnare module will be compiled: $enable_pmsnare" echo @@ -1520,6 +1679,10 @@ echo " mmnormalize module will be compiled: $enable_mmnormalize" echo " mmjsonparse module will be compiled: $enable_mmjsonparse" echo " mmjaduit module will be compiled: $enable_mmaudit" echo " mmsnmptrapd module will be compiled: $enable_mmsnmptrapd" +echo " mmutf8fix enabled: $enable_mmutf8fix" +echo " mmrfc5424addhmac enabled: $enable_mmrfc5424addhmac" +echo " mmpstrucdata enabled: $enable_mmpstrucdata" +echo " mmsequence enabled: $enable_mmsequence" echo echo "---{ strgen modules }---" echo " sm_cust_bindcdr module will be compiled: $enable_sm_cust_bindcdr" diff --git a/debian/changelog b/debian/changelog index 2d40071..07fcdec 100644 --- a/debian/changelog +++ b/debian/changelog @@ -1,3 +1,197 @@ +rsyslog (8.4.1-1) unstable; urgency=high + + * New upstream release. + - CVE-2014-3634: Fix remote syslog vulnerability due to improper + handling of invalid PRI values. + * Drop patches which have been merged upstream. + * Urgency high for the security fix. + + -- Michael Biebl <biebl@debian.org> Tue, 30 Sep 2014 12:35:33 +0200 + +rsyslog (8.4.0-2) unstable; urgency=medium + + * Tweak rsyslog systemd service file. Add Documentation field and restart + the rsyslogd daemon on failures. + * Cherry-pick upstream patch to fix build failures on platforms without + atomic instructions. + + -- Michael Biebl <biebl@debian.org> Wed, 20 Aug 2014 19:36:33 +0200 + +rsyslog (8.4.0-1) unstable; urgency=medium + + * New upstream release. + * Update patches: + - Refresh 0001-Don-t-create-a-database.patch. + - Drop 0002-Fix-pidfile-location.patch, merged upstream. + * Add --disable-generate-man-pages to configure flags to avoid a build + dependency on python-docutils. We don't ship the generated man pages + anyway. + * Add support for external message modification modules which are bound via + the new native mmexternal module. + + -- Michael Biebl <biebl@debian.org> Mon, 18 Aug 2014 18:28:20 +0200 + +rsyslog (8.2.2-5) unstable; urgency=medium + + * Make sure to actually only create the temporary syslog.service symlink on + upgrades and avoid the usage of readlink. + + -- Michael Biebl <biebl@debian.org> Mon, 18 Aug 2014 02:43:31 +0200 + +rsyslog (8.2.2-4) unstable; urgency=medium + + * Fix wheezy → jessie upgrade failure when running under systemd due to the + syslog.service symlink temporarily being missing. + + -- Michael Biebl <biebl@debian.org> Mon, 18 Aug 2014 00:39:03 +0200 + +rsyslog (8.2.2-3) unstable; urgency=low + + * Upload to unstable. + + -- Michael Biebl <biebl@debian.org> Sun, 29 Jun 2014 14:40:56 +0200 + +rsyslog (8.2.2-2) experimental; urgency=medium + + * Merge changes from unstable branch. + + -- Michael Biebl <biebl@debian.org> Mon, 23 Jun 2014 13:52:12 +0200 + +rsyslog (8.2.2-1) experimental; urgency=medium + + * New major upstream release. + - Rewritten core engine providing higher performance and better + scalability with complex configurations. + - Simplified script execution. + - Global variable support. + - Output part was revamped completely and required changes to all output + modules. + - Support for external plugins which can be written in any language. + * Switch watch file to track v8 stable branch. + * Update Build-Depends: + - Bump librelp-dev to (>= 1.2.5). + - Bump liblogging-stdlog-dev to (>= 1.0.3). + - Bump liblognorm-dev to (>= 1.0.0). + - Drop libee-dev. + * Drop the omruleset module which has been deprecated and replaced by the + much more efficient RainerScript "call" statement. + * Remove the rsyslog-doc package which is now built from a separate source + package. + * Fix pidfile location. The pidfile should be stored in /var/run, not /etc. + + -- Michael Biebl <biebl@debian.org> Mon, 09 Jun 2014 21:16:19 +0200 + +rsyslog (7.6.3-3) unstable; urgency=medium + + * Build against libgnutls28-dev. (Closes: #752304) + * Use canonical URIs for Vcs-* fields. + + -- Michael Biebl <biebl@debian.org> Mon, 23 Jun 2014 13:44:55 +0200 + +rsyslog (7.6.3-2) unstable; urgency=medium + + * Update Build-Depends: + - Bump librelp-dev to (>= 1.2.5). + * Build omelasticsearch module which provides support for logging to an + Elasticsearch server. Split that module into a separate package called + rsyslog-elasticsearch. (Closes: #744951) + + -- Michael Biebl <biebl@debian.org> Fri, 16 May 2014 22:37:05 +0200 + +rsyslog (7.6.3-1) unstable; urgency=medium + + * New upstream release. + - Fixes ompipe to properly handle retries and not cause unnecessary + suspension messages. (Closes: #742113) + * Update watch file, track v7 stable branch. + + -- Michael Biebl <biebl@debian.org> Fri, 28 Mar 2014 12:36:45 +0100 + +rsyslog (7.6.2-1) unstable; urgency=medium + + * New upstream release. + * Update patches. + * Update Build-Depends: + - Bump librelp-dev to (>= 1.2.4). + - Add liblogging-stdlog-dev (>= 1.0.1). + * Fix typo in rsyslog.postrm: dissappear → disappear + * Drop useless "exit 0" from maintainer scripts. + * Enable and install the mmutf8fix, mmpstrucdata and mmsequence message + modification modules. + * Stop installing debian/sample.conf. Upstream ships a more up-to-date + example configuration file. + + -- Michael Biebl <biebl@debian.org> Mon, 17 Mar 2014 17:26:08 +0100 + +rsyslog (7.4.8-1) unstable; urgency=medium + + [ Michael Biebl ] + * New upstream release. + * Update Build-Depends: + - Bump libestr-dev to (>= 0.1.9). + - Tighten liblognorm-dev to (<< 1.0.0). + - Replace libjson0-dev with libjson-c-dev, we no longer need the + transitional package. + * Bump Standards-Version to 3.9.5. No further changes. + + [ Daniel Pocock ] + * Make template parameter not mandatory in mongodb output plugin. Patch + cherry-picked from upstream Git. (Closes: #740869, #721277) + * Ensure JSON templates are NUL terminated. Patch cherry-picked from + upstream Git. + + -- Michael Biebl <biebl@debian.org> Tue, 11 Mar 2014 19:52:49 +0100 + +rsyslog (7.4.4-1) unstable; urgency=low + + * New upstream release. + + -- Michael Biebl <biebl@debian.org> Tue, 03 Sep 2013 22:42:49 +0200 + +rsyslog (7.4.3-1) unstable; urgency=low + + * New upstream release. + + -- Michael Biebl <biebl@debian.org> Tue, 23 Jul 2013 01:01:40 +0200 + +rsyslog (7.4.2-1) unstable; urgency=low + + * New upstream release. + + -- Michael Biebl <biebl@debian.org> Sat, 06 Jul 2013 00:14:19 +0200 + +rsyslog (7.4.1-3) unstable; urgency=low + + * Bump Build-Depends on dh-systemd to (>= 1.4) to ensure we have a recent + enough version of init-system-helpers which properly deals with a missing + state directory. (Closes: #714265) + + -- Michael Biebl <biebl@debian.org> Thu, 27 Jun 2013 19:21:04 +0200 + +rsyslog (7.4.1-2) unstable; urgency=low + + * Use dh-systemd to setup the systemd service properly. + + -- Michael Biebl <biebl@debian.org> Wed, 26 Jun 2013 21:13:33 +0200 + +rsyslog (7.4.1-1) unstable; urgency=low + + * New upstream release. + + -- Michael Biebl <biebl@debian.org> Mon, 17 Jun 2013 23:38:51 +0200 + +rsyslog (7.4.0-1) unstable; urgency=low + + * New upstream release. + + -- Michael Biebl <biebl@debian.org> Thu, 06 Jun 2013 18:51:11 +0200 + +rsyslog (7.3.15-1) unstable; urgency=low + + * New upstream release. + + -- Michael Biebl <biebl@debian.org> Wed, 15 May 2013 18:21:02 +0200 + rsyslog (7.3.14-2+dyson2) unstable; urgency=low * Added dependency on /etc/rsyslog.confin SMF service and thus support for diff --git a/debian/control b/debian/control index 5461373..ca0e0e3 100644 --- a/debian/control +++ b/debian/control @@ -5,23 +5,25 @@ Maintainer: Michael Biebl <biebl@debian.org> Build-Depends: debhelper (>= 8), dpkg-dev (>= 1.16.1), dh-autoreconf, + dh-systemd (>= 1.4), zlib1g-dev, libmysqlclient-dev, libpq-dev, libmongo-client-dev (>= 0.1.4), + libcurl4-gnutls-dev, libkrb5-dev, - libgnutls-dev, - librelp-dev (>= 1.0.3), - libestr-dev (>= 0.1.5), - libee-dev (>= 0.4.0), - liblognorm-dev (>= 0.3.1), - libjson0-dev, + libgnutls28-dev, + librelp-dev (>= 1.2.5), + libestr-dev (>= 0.1.9), + liblognorm-dev (>= 1.0.0), + liblogging-stdlog-dev (>= 1.0.3), + libjson-c-dev, uuid-dev, pkg-config, bison -Standards-Version: 3.9.4 -Vcs-Git: git://git.debian.org/git/collab-maint/rsyslog.git -Vcs-Browser: http://git.debian.org/?p=collab-maint/rsyslog.git;a=summary +Standards-Version: 3.9.5 +Vcs-Git: git://anonscm.debian.org/collab-maint/rsyslog.git +Vcs-Browser: http://anonscm.debian.org/gitweb/?p=collab-maint/rsyslog.git;a=summary Homepage: http://www.rsyslog.com/ Package: rsyslog @@ -59,22 +61,6 @@ Description: reliable system and kernel logging daemon . It is the default syslogd on Debian systems. -Package: rsyslog-doc -Section: doc -Priority: extra -Architecture: all -Breaks: rsyslog (<< 2.0.1-2) -Replaces: rsyslog (<< 2.0.1-2) -Depends: ${misc:Depends} -Suggests: doc-base, - www-browser -Description: documentation for rsyslog - This package contains detailed HTML documentation for rsyslog. - . - It describes the general configuration file syntax for filters, actions, - templates, etc, and has detailed information for all available configuration - directives. - Package: rsyslog-mysql Architecture: any Priority: extra @@ -86,7 +72,7 @@ Depends: ${shlibs:Depends}, Recommends: mysql-client Suggests: mysql-server Description: MySQL output plugin for rsyslog - This plugin allows rsyslog to write the syslog messages into a MySQL database. + This plugin allows rsyslog to write syslog messages into a MySQL database. Package: rsyslog-pgsql Architecture: any @@ -99,7 +85,7 @@ Depends: ${shlibs:Depends}, Recommends: postgresql-client Suggests: postgresql Description: PostgreSQL output plugin for rsyslog - This plugin allows rsyslog to write the syslog messages into a PostgreSQL + This plugin allows rsyslog to write syslog messages into a PostgreSQL database. Package: rsyslog-mongodb @@ -110,9 +96,20 @@ Depends: ${shlibs:Depends}, rsyslog (= ${binary:Version}), Recommends: mongodb-server Description: MongoDB output plugin for rsyslog - This plugin allows rsyslog to write the syslog messages to MongoDB, a + This plugin allows rsyslog to write syslog messages to MongoDB, a scalable, high-performance, open source NoSQL database. +Package: rsyslog-elasticsearch +Architecture: any +Priority: extra +Depends: ${shlibs:Depends}, + ${misc:Depends}, + rsyslog (= ${binary:Version}), +Description: Elasticsearch output plugin for rsyslog + This plugin allows rsyslog to write syslog messages to Elasticsearch, a + distributed, multitenant-capable full-text search engine with a RESTful web + interface and schema-free JSON documents. + Package: rsyslog-gssapi Architecture: any Priority: extra @@ -143,6 +140,6 @@ Depends: ${shlibs:Depends}, ${misc:Depends}, rsyslog (= ${binary:Version}) Description: RELP protocol support for rsyslog - These plugins allows rsyslog to send and receive syslog messages via the + These plugins allow rsyslog to send and receive syslog messages via the RELP protocol. RELP ensures reliable transport over the network even on connection loss or if a peer becomes unavailable. diff --git a/debian/patches/0001-Don-t-create-a-database.patch b/debian/patches/0001-Don-t-create-a-database.patch new file mode 100644 index 0000000..b499376 --- /dev/null +++ b/debian/patches/0001-Don-t-create-a-database.patch @@ -0,0 +1,42 @@ +From: Michael Biebl <biebl@debian.org> +Date: Thu, 13 Mar 2014 17:58:08 +0100 +Subject: Don't create a database + +dbconfig-common, which is used by rsyslog-mysql and rsyslog-pgsql, takes +care of creating the database for us. +--- + plugins/ommysql/createDB.sql | 6 ++---- + plugins/ompgsql/createDB.sql | 2 -- + 2 files changed, 2 insertions(+), 6 deletions(-) + +diff --git a/plugins/ommysql/createDB.sql b/plugins/ommysql/createDB.sql +index 211cfb0..1dc6f5c 100644 +--- a/plugins/ommysql/createDB.sql ++++ b/plugins/ommysql/createDB.sql +@@ -1,6 +1,4 @@ +-CREATE DATABASE Syslog; +-USE Syslog; +-CREATE TABLE SystemEvents ++CREATE TABLE IF NOT EXISTS SystemEvents + ( + ID int unsigned not null auto_increment primary key, + CustomerID bigint, +@@ -28,7 +26,7 @@ CREATE TABLE SystemEvents + SystemID int NULL + ); + +-CREATE TABLE SystemEventsProperties ++CREATE TABLE IF NOT EXISTS SystemEventsProperties + ( + ID int unsigned not null auto_increment primary key, + SystemEventID int NULL , +diff --git a/plugins/ompgsql/createDB.sql b/plugins/ompgsql/createDB.sql +index 0c0f7e1..5dab0dd 100644 +--- a/plugins/ompgsql/createDB.sql ++++ b/plugins/ompgsql/createDB.sql +@@ -1,5 +1,3 @@ +-CREATE DATABASE "Syslog" WITH ENCODING 'SQL_ASCII' TEMPLATE template0; +-\c Syslog; + CREATE TABLE SystemEvents + ( + ID serial not null primary key, diff --git a/debian/patches/01-dont_create_db.patch b/debian/patches/01-dont_create_db.patch deleted file mode 100644 index efd0c75..0000000 --- a/debian/patches/01-dont_create_db.patch +++ /dev/null @@ -1,31 +0,0 @@ -Index: rsyslog/plugins/ommysql/createDB.sql -=================================================================== ---- rsyslog.orig/plugins/ommysql/createDB.sql 2007-12-12 18:18:29.000000000 +0100 -+++ rsyslog/plugins/ommysql/createDB.sql 2008-02-12 16:03:04.000000000 +0100 -@@ -1,6 +1,4 @@ --CREATE DATABASE Syslog; --USE Syslog; --CREATE TABLE SystemEvents -+CREATE TABLE IF NOT EXISTS SystemEvents - ( - ID int unsigned not null auto_increment primary key, - CustomerID bigint, -@@ -28,7 +26,7 @@ - SystemID int NULL - ); - --CREATE TABLE SystemEventsProperties -+CREATE TABLE IF NOT EXISTS SystemEventsProperties - ( - ID int unsigned not null auto_increment primary key, - SystemEventID int NULL , -Index: rsyslog/plugins/ompgsql/createDB.sql -=================================================================== ---- rsyslog.orig/plugins/ompgsql/createDB.sql 2007-12-12 18:18:29.000000000 +0100 -+++ rsyslog/plugins/ompgsql/createDB.sql 2008-02-12 16:03:04.000000000 +0100 -@@ -1,5 +1,3 @@ --CREATE DATABASE 'Syslog' WITH ENCODING 'SQL_ASCII'; --\c Syslog; - CREATE TABLE SystemEvents - ( - ID serial not null primary key, diff --git a/debian/patches/series b/debian/patches/series index abf808c..4a52108 100644 --- a/debian/patches/series +++ b/debian/patches/series @@ -1,4 +1,3 @@ -# Debian patches for rsyslog -dyson-imuxsock.patch -01-dont_create_db.patch +0001-Don-t-create-a-database.patch dyson-imsolaris-no-libdoor.patch +dyson-imuxsock.patch diff --git a/debian/rsyslog-doc.doc-base b/debian/rsyslog-doc.doc-base deleted file mode 100644 index 390179b..0000000 --- a/debian/rsyslog-doc.doc-base +++ /dev/null @@ -1,9 +0,0 @@ -Document: rsyslog-doc -Title: Rsyslog Documentation -Author: Rainer Gerhards -Abstract: This documentation covers the configuration of rsyslog. -Section: System/Administration - -Format: HTML -Index: /usr/share/doc/rsyslog-doc/html/manual.html -Files: /usr/share/doc/rsyslog-doc/html/*.html diff --git a/debian/rsyslog-doc.examples b/debian/rsyslog-doc.examples deleted file mode 100644 index 7ca2e48..0000000 --- a/debian/rsyslog-doc.examples +++ /dev/null @@ -1 +0,0 @@ -debian/sample.conf diff --git a/debian/rsyslog-doc.install b/debian/rsyslog-doc.install deleted file mode 100644 index 57a07f8..0000000 --- a/debian/rsyslog-doc.install +++ /dev/null @@ -1,4 +0,0 @@ -doc/*.html usr/share/doc/rsyslog-doc/html/ -doc/*.jpg usr/share/doc/rsyslog-doc/html/ -doc/*.png usr/share/doc/rsyslog-doc/html/ -doc/*.conf usr/share/doc/rsyslog-doc/html/ diff --git a/debian/rsyslog-elasticsearch.install b/debian/rsyslog-elasticsearch.install new file mode 100644 index 0000000..5173ef9 --- /dev/null +++ b/debian/rsyslog-elasticsearch.install @@ -0,0 +1 @@ +usr/lib/rsyslog/omelasticsearch.so diff --git a/debian/rsyslog-mysql.postinst b/debian/rsyslog-mysql.postinst index 2e08b9b..49a8fdf 100644 --- a/debian/rsyslog-mysql.postinst +++ b/debian/rsyslog-mysql.postinst @@ -41,5 +41,3 @@ esac #DEBHELPER# - -exit 0 diff --git a/debian/rsyslog-mysql.postrm b/debian/rsyslog-mysql.postrm index be625a5..103278e 100644 --- a/debian/rsyslog-mysql.postrm +++ b/debian/rsyslog-mysql.postrm @@ -51,5 +51,3 @@ esac #DEBHELPER# - -exit 0 diff --git a/debian/rsyslog-mysql.prerm b/debian/rsyslog-mysql.prerm index 2c8d719..5a70d81 100644 --- a/debian/rsyslog-mysql.prerm +++ b/debian/rsyslog-mysql.prerm @@ -8,5 +8,3 @@ set -e dbc_go rsyslog-mysql $@ #DEBHELPER# - -exit 0 diff --git a/debian/rsyslog-pgsql.postinst b/debian/rsyslog-pgsql.postinst index c11dd5a..77bb245 100644 --- a/debian/rsyslog-pgsql.postinst +++ b/debian/rsyslog-pgsql.postinst @@ -41,5 +41,3 @@ esac #DEBHELPER# - -exit 0 diff --git a/debian/rsyslog-pgsql.postrm b/debian/rsyslog-pgsql.postrm index d48ff21..5de1fb3 100644 --- a/debian/rsyslog-pgsql.postrm +++ b/debian/rsyslog-pgsql.postrm @@ -51,5 +51,3 @@ esac #DEBHELPER# - -exit 0 diff --git a/debian/rsyslog-pgsql.prerm b/debian/rsyslog-pgsql.prerm index 02d4ed4..e150dd3 100644 --- a/debian/rsyslog-pgsql.prerm +++ b/debian/rsyslog-pgsql.prerm @@ -8,5 +8,3 @@ set -e dbc_go rsyslog-pgsql $@ #DEBHELPER# - -exit 0 diff --git a/debian/rsyslog.install b/debian/rsyslog.install index a94fc1d..084c5bd 100644 --- a/debian/rsyslog.install +++ b/debian/rsyslog.install @@ -17,11 +17,14 @@ usr/lib/rsyslog/lmtcpclt.so usr/lib/rsyslog/lmtcpsrv.so usr/lib/rsyslog/lmzlibw.so usr/lib/rsyslog/mmanon.so +usr/lib/rsyslog/mmexternal.so usr/lib/rsyslog/mmnormalize.so usr/lib/rsyslog/mmjsonparse.so +usr/lib/rsyslog/mmutf8fix.so +usr/lib/rsyslog/mmpstrucdata.so +usr/lib/rsyslog/mmsequence.so usr/lib/rsyslog/ommail.so usr/lib/rsyslog/omprog.so usr/lib/rsyslog/omuxsock.so -usr/lib/rsyslog/omruleset.so usr/lib/rsyslog/pm*.so lib/systemd/system/rsyslog.service diff --git a/debian/rsyslog.links b/debian/rsyslog.links deleted file mode 100644 index c9d3179..0000000 --- a/debian/rsyslog.links +++ /dev/null @@ -1,2 +0,0 @@ -lib/systemd/system/rsyslog.service etc/systemd/system/multi-user.target.wants/rsyslog.service -lib/systemd/system/rsyslog.service etc/systemd/system/syslog.service diff --git a/debian/rsyslog.postinst b/debian/rsyslog.postinst index a4fb3a0..da1554a 100644 --- a/debian/rsyslog.postinst +++ b/debian/rsyslog.postinst @@ -23,6 +23,11 @@ case "$1" in # Fix permissions of the spool/work directory (Bug: #693099) chmod 700 /var/spool/rsyslog + + # Clean up temporary syslog.service symlink + if [ -d /run/systemd/system ] && dpkg --compare-versions "$2" lt-nl "7.4.1-2" ; then + rm -f /run/systemd/system/syslog.service + fi ;; abort-upgrade|abort-remove|abort-deconfigure) @@ -34,7 +39,4 @@ case "$1" in ;; esac - #DEBHELPER# - -exit 0 diff --git a/debian/rsyslog.postrm b/debian/rsyslog.postrm index 4d7ae11..f418ca3 100644 --- a/debian/rsyslog.postrm +++ b/debian/rsyslog.postrm @@ -6,7 +6,7 @@ if [ "$1" = "remove" ]; then [ -f /etc/logrotate.d/rsyslog ] && mv -f /etc/logrotate.d/rsyslog /etc/logrotate.d/rsyslog.disabled fi -if [ "$1" = "purge" -o "$1" = "dissappear" ]; then +if [ "$1" = "purge" ] || [ "$1" = "disappear" ]; then [ -f /etc/logrotate.d/rsyslog.disabled ] && rm -f /etc/logrotate.d/rsyslog.disabled fi @@ -16,5 +16,3 @@ if [ "$1" = "remove" ]; then fi #DEBHELPER# - -exit 0 diff --git a/debian/rsyslog.preinst b/debian/rsyslog.preinst index fce810f..8058953 100644 --- a/debian/rsyslog.preinst +++ b/debian/rsyslog.preinst @@ -2,7 +2,7 @@ set -e -if [ "$1" = "install" -a -n "$2" ] ; then +if [ "$1" = "install" ] && [ -n "$2" ] ; then [ -f /etc/logrotate.d/rsyslog.disabled ] && mv -f /etc/logrotate.d/rsyslog.disabled /etc/logrotate.d/rsyslog fi @@ -10,6 +10,16 @@ if [ "$1" = "upgrade" ] && dpkg --compare-versions "$2" lt "5.7.8-1" ; then rm -f /etc/systemd/system/sockets.target.wants/rsyslog.socket fi -#DEBHELPER# +# In wheezy we ship the syslog.service symlink directly in the package, in +# jessie we use init-system-helpers to create it dynamically in postinst. +# During the upgrade there is a time frame when the symlink does not exist. +# If systemd is reloaded at this point, it loses track of the rsyslogd process. +# To work around this problem, create a (runtime) copy of the syslog.service +# symlink before the upgrade and remove it again afterwards. See #724796 +if [ "$1" = "upgrade" ] && dpkg --compare-versions "$2" lt "7.4.1-2" ; then + if [ -d /run/systemd/system ] && [ -L /etc/systemd/system/syslog.service ] ; then + ln -sf /lib/systemd/system/rsyslog.service /run/systemd/system/syslog.service + fi +fi -exit 0 +#DEBHELPER# diff --git a/debian/rsyslog.prerm b/debian/rsyslog.prerm new file mode 100644 index 0000000..503b849 --- /dev/null +++ b/debian/rsyslog.prerm @@ -0,0 +1,10 @@ +#!/bin/sh + +set -e + +# Stop the socket on remove so rsyslog is not restarted via socket activation +if [ -d /run/systemd/system ] && [ "$1" = remove ] ; then + systemctl stop syslog.socket || true +fi + +#DEBHELPER# diff --git a/debian/rules b/debian/rules index 9e03c84..f41acfe 100755 --- a/debian/rules +++ b/debian/rules @@ -6,11 +6,12 @@ include /usr/share/dpkg/buildflags.mk DEB_HOST_ARCH_OS ?= $(shell dpkg-architecture -qDEB_HOST_ARCH_OS) %: - dh $@ --with autoreconf + dh $@ --with autoreconf,systemd ifeq ($(DEB_HOST_ARCH_OS), linux) confflags += --enable-imptcp \ - --enable-kmsg + --enable-kmsg \ + --with-systemdsystemunitdir=/lib/systemd/system endif ifeq (illumos,$(DEB_HOST_ARCH_OS)) @@ -23,6 +24,7 @@ override_dh_auto_configure: --enable-mysql \ --enable-pgsql \ --enable-ommongodb \ + --enable-elasticsearch \ --enable-mail \ --enable-imfile \ --enable-impstats \ @@ -40,10 +42,13 @@ override_dh_auto_configure: --enable-mmanon \ --enable-mmnormalize \ --enable-mmjsonparse \ + --enable-mmutf8fix \ + --enable-mmpstrucdata \ + --enable-mmsequence \ --disable-libgcrypt \ --disable-testbench \ - $(confflags) \ - --with-systemdsystemunitdir=/lib/systemd/system + --disable-generate-man-pages \ + $(confflags) override_dh_auto_install: dh_auto_install @@ -75,8 +80,5 @@ else echo '$$ModLoad imklog' > debian/rsyslog/etc/rsyslog.d/00-load-imklog.conf endif -override_dh_compress: - dh_compress -X rsyslog-example.conf - override_dh_installinit: dh_installinit -R diff --git a/debian/sample.conf b/debian/sample.conf deleted file mode 100644 index a27de0c..0000000 --- a/debian/sample.conf +++ /dev/null @@ -1,284 +0,0 @@ -# This is a sample configuation file for rsyslogd. See the -# doc/manual.html for details. If you can not find the -# manual set, please visit -# -# http://www.rsyslog.com/doc/ -# -# to obtain it online. -# -# WARNING: We do NOT keep the comments in this file always -# up to date. Be sure to consult the doc set that -# came with your package, especially the file on -# rsyslog.conf - it probably has some better information -# than is provided here in comments. The main purpose -# of sample.conf is to show you some actual directives, -# not to be the authorative doc source. -# -# Please note that rsyslogd by default -# reads /etc/rsyslogd.conf (and NOT /etc/syslogd.conf!). -# -# A commented sample configuration. More a man page than a real -# sample ;) -# -# We try to keep things as consistent with existing syslog implementation -# as possible. We use "$" to start lines that contain new directives. - -# We limit who can send us messages: -$AllowedSender UDP, 192.0.2.0/24, 10.0.0.1 # all machines in 192.0.2 as well as 10.0.0.1 -$AllowedSender TCP, 10.0.0.1 # for TCP, we allow only 10.0.0.1 -# remove the AllowedSender directives if you do not want to limit -# who can send rsyslogd messages (not recommended) - -# Templates are a key feature of rsyslog. They allow to specify any -# format a user might want. Every output in rsyslog uses templates - this -# holds true for files, user messages and so on. The database writer -# expects its template to be a proper SQL statement - so this is highly -# customizable too. You might ask how does all of this work when no templates -# at all are specified. Good question ;) The answer is simple, though. Templates -# compatible with the stock syslogd formats are hardcoded into rsyslog. So if -# no template is specified, we use one of these hardcoded templates. Search for -# "template_" in syslogd.c and you will find the hardcoded ones. -# -# A template consists of a template directive, a name, the actual template text -# and optional options. A sample is: -# -# $template MyTemplateName,"\7Text %property% some more text\n",<options> -# -# The "$template" is the template directive. It tells rsyslog that this -# line contains a template. -# -# "MyTemplateName" is the template name. All other config lines refer to -# this name. -# -# The text within quotes is the actual template text. The backslash is -# a escape character, much as in C. It does all these "cool" things. For -# example, \7 rings the bell (this is an ASCII value), \n is a new line. -# C programmers and perl coders have the advantage of knowing this, but the -# set in rsyslog is a bit restricted currently. All text in the template -# is used literally, except for things within percent signs. These are -# properties and allow you access to the contents of the syslog message. -# Properties are accessed via the property replacer (nice name, huh) and -# it can do cool things, too. For example, it can pick a substring or -# do date-specific formatting. More on this is below, on some lines of the -# property replacer. -# -# The <options> part is optional. It carries options that influence the -# template as whole. Details are below. Be sure NOT to mistake template -# options with property options - the later ones are processed by the -# property replacer and apply to a SINGLE property, only (and not the -# whole template). -# -# Template options are case-insensitive. Currently defined are: -# sql - format the string suitable for a SQL statement. This will replace single -# quotes ("'") by two single quotes ("''") inside each field. This option MUST -# be specified when a template is used for writing to a database, otherwise SQL -# injection might occur. -# -# Please note that the database writer *checks* that the sql option is -# present in the template. If it is not present, the write database action -# is disabled. This is to guard you against accidential forgetting it and -# then becoming vulnerable for SQL injection. -# The sql option can also be useful with files - especially if you want -# to run them on another machine for performance reasons. However, do NOT -# use it if you do not have a real need for it - among others, it takes -# some toll on the processing time. Not much, but on a really busy system -# you might notice it ;) -# -# To escape: -# % = \% -# \ = \\ -# --> '\' is used to escape (as in C) -#$template TraditionalFormat,%timegenerated% %HOSTNAME% %syslogtag%%msg%\n" -# -# Properties can be accessed by the property replacer. They are accessed -# inside the template by putting them between percent signs. Properties -# can be modifed by the property replacer. The full syntax is as follows: -# -# %propname:fromChar:toChar:options% -# -# propname is the name of the property to access. This IS case-sensitive! -# Currently supported are: -# msg the MSG part of the message (aka "the message" ;)) -# rawmsg the message excactly as it was received from the -# socket. Should be useful for debugging. -# UxTradMsg will disappear soon - do NOT use! -# HOSTNAME hostname from the message -# source alias for HOSTNAME -# syslogtag TAG from the message -# PRI PRI part of the message - undecoded (single value) -# IUT the monitorware InfoUnitType - used when talking to a -# MonitorWare backend (also for phpLogCon) -# syslogfacility the facility from the message - in numerical form -# syslogpriority the priority (actully severity!) from the -# message - in numerical form -# timegenerated timestamp when the message was RECEIVED. Always in high -# resolution -# timereported timestamp from the message. Resolution depends on what -# was provided in the message (in most cases, only seconds) -# TIMESTAMP alias for timereported -# -# Other properties might be available at the time you read this. Be sure -# to consult the property replacer documentation in the doc set for all -# properties. -# -# FromChar and toChar are used to build substrings. They specify the -# offset within the string that should be copied. Offset counting -# starts at 1, so if you need to obtain the first 2 characters of the -# message text, you can use this syntax: "%msg:1:2%". -# If you do not whish to specify from and to, but you want to -# specify options, you still need to include the colons. For example, -# if you would like to convert the full message text to lower case -# only, use "%msg:::lowercase%". -# -# property options are case-insensitive, currently defined are: -# uppercase convert property to lowercase only -# lowercase convert property text to uppercase only -# drop-last-lf The last LF in the message (if any), is dropped. -# Especially useful for PIX. -# date-mysql format as mysql date -# date-rfc3164 format as RFC 3164 date -# date-rfc3339 format as RFC 3339 date -# escape-cc NOT yet implemented - -# Below find some samples of what a template can do. Have a good -# time finding out what they do [or just tun them] ;) - -# A template that resambles traditional syslogd file output: -$template TraditionalFormat,"%timegenerated% %HOSTNAME% %syslogtag%%msg:::drop-last-lf%\n" - -# a template useful for debugging format issues -$template DEBUG,"Debug line with all properties:\nFROMHOST: '%FROMHOST%', HOSTNAME: '%HOSTNAME%', PRI: %PRI%,\nsyslogtag '%syslogtag%', programname: '%programname%', APP-NAME: '%APP-NAME%', PROCID: '%PROCID%', MSGID: '%MSGID%',\nTIMESTAMP: '%TIMESTAMP%', STRUCTURED-DATA: '%STRUCTURED-DATA%',\nmsg: '%msg%'\nescaped msg: '%msg:::drop-cc%'\nrawmsg: '%rawmsg%'\n\n" -# -# A template that resembles RFC 3164 on-the-wire format: -# (yes, there is NO space betwen syslogtag and msg! that's important!) -$template RFC3164fmt,"<%PRI%>%TIMESTAMP% %HOSTNAME% %syslogtag%%msg%" - -# a template resembling traditional wallmessage format: -$template wallmsg,"\r\n\7Message from syslogd@%HOSTNAME% at %timegenerated% ...\r\n %syslogtag%%msg%\n\r" - -# The template below emulates winsyslog format, but we need to check the time -# stamps used. for now, it is good enough ;) This format works best with -# other members of the MonitorWare product family. It is also a good sample -# where you can see the property replacer in action. -$template WinSyslogFmt,"%HOSTNAME%,%timegenerated:1:10:date-rfc3339%,%timegenerated:12:19:date-rfc3339%,%timegenerated:1:10:date-rfc3339%,%timegenerated:12:19:date-rfc3339%,%syslogfacility%,%syslogpriority%,%syslogtag%%msg%\n" - -# A template used for database writing (notice it *is* an actual -# sql-statement): -$template dbFormat,"insert into SystemEvents (Message, Facility,FromHost, Priority, DeviceReportedTime, ReceivedAt, InfoUnitID, SysLogTag) values ('%msg%', %syslogfacility%, '%HOSTNAME%',%syslogpriority%, '%timereported:::date-mysql%', '%timegenerated:::date-mysql%', %iut%, '%syslogtag%')",sql - -# Selector lines are somewhat different from stock syslogd. With -# rsyslog, you can add a semicolon ";" after the target and then -# the template name. That will assign this template to the respective -# action. If no template name is given, a hardcoded template is used. -# If a template name is given, but the template was not defined, the -# selector line is DEACTIVATED. -# -# ############# -# # IMPORTANT # -# ############# -# Templates MUST be defined BEFORE they are used! It is OK to -# intermix template definitions and selector lines within the -# config file, but each template MUST be defined before it is -# used the first time! -# - -# We have some very rough samples here - This sample file focusses -# on the new syntax, so we do NOT describe all possible selections. -# Use the syslog.conf if you are interested to see how to select -# based on facility and severits (aka priority). - -*.* /var/log/winsyslog-like.log;WinSyslogFmt - -# A selector using the traditional format defined above: -*.* /var/log/traditionalfile.log;TraditionalFormat - -# And another one using the hardcoded traditional format: -*.* /var/log/anothertraditionalfile.log - -# Templates are also fully supportd for forwarding: -*.* @172.19.2.16;RFC3164fmt - -# And this finally is a database action -# The semicolon at the end is not necessary, -# but some previous versions of rsyslogd had a bug that -# made them abort if it were missing. As Dennis Olvany -# pointed out, it would be extremely nice to have this -# semicolon in the sample conf - so we did in a previous -# version and it still sticks around ;) -*.* >hostname,dbname,userid,password; -# It uses the default schema (MonitorWare format). The parameters -# should be self-explanatory. - -# And this one uses the template defined above: -*.* >hostname,dbname,userid,password;dbFormat - - -# -# Rsyslog supports TCP-based syslog. To enable receiving TCP messages, -# use the -t <port> command line option (where port is the port it -# shall listen to. To forward messages to the remote host, you must -# specify a forwarding action and include the host and port. TCP -# and UDP-based forwarding has basically the same syntax, except that -# TCP delivery is triggered by specifying a second at-sign (@) in the -# message. -# This is UDP forwarding to port 514: -*.* @172.19.2.16 -# This is UDP forwarding to port 1514: -*.* @172.19.2.16:1514 -# This is TCP forwarding to port 1514: -*.* @@172.19.2.16:1514 -# The second @-sign is all you need (except, of course, a tcp-capable -# syslogd like rsyslogd ;)). -# Of course, you can also specify a template with TCP: -*.* @@172.19.2.16:1514;RFC3164Fmt -# There are also some options you can select. These come between -# paranthesis. Available are: -# z<number> - turn on compression, number is compression mode 0 - none, 9 max -# o - (tcp only) use octet counting for framing EXPERIMENTAL -# -# Forward via TCP with maximum compression and octet couting framing: -*.* @@(z9,o)172.19.2.16:1514;RFC3164Fmt -# Forward via UDP with maximum compression to port 1514 -*.* @(z9)172.19.2.16:1514 - -# We also support property-based filters, which allow for nice -# things. Let's for example assume that you receive a lot of -# nonsense messages with "ID-4711" in the message text. You know -# that you will never need these messages. So you simply discard them -:msg, contains, "ID-4711" ~ - -# or you would like to store messages from a specific host to -# a different file: -:FROMHOST, isequal,"myhost.example.com" /var/log/myhost.log - -# everyting that does not contain "error" should also be -# discarded -:msg, !contains, "error" ~ -# and the rest go to a seperate file -*.* /var/log/error -# (keep in mind that the two directives shown immediately -# above must be kept in that order to actually work) - -# you can also execute a script. Let's assume, for example, you need -# to execute "turn-diesel-generator-on" when "power failed" is contained -# in a message... ;) -:msg, contains, "power failed" ^turn-diesel-generator-on -# (The script is passed the syslog message as first and only parameter. -# Other parameters can currently not be specified.) - -# Note that boolean operations (other than not [!]) are not -# currently supported. As such, you can not filter out different -# facilities from different machines - hopefully later ;) - -# -# A final world. rsyslog is considered a part of Adiscon's MonitorWare product line. -# As such, you can find current information as well as information on the -# other product line members on http://www.monitorware.com. Please be warned, there -# are a number of closed-source commercial Windows applications among these products ;) -# -# You might want to check the GPL'ed phpLogCon (http://www.phplogcon.org) -# as a web-based front-end to a syslog message database. -# -# I hope this work is useful. -# 2005-09-27 Rainer Gerhards <rgerhards@adiscon.com> -# diff --git a/debian/watch b/debian/watch index b6f8643..dd306eb 100644 --- a/debian/watch +++ b/debian/watch @@ -1,2 +1,2 @@ version=3 -http://rsyslog.com/tag/beta/ .*rsyslog-(.*)\.tar\.gz +http://rsyslog.com/downloads/download-v8-stable/ .*/rsyslog-([\d\.]*)\.tar\.gz @@ -5,7 +5,7 @@ * yet a runtime library, because it depends on some functionality * residing somewhere else. * - * Copyright 2007, 2008 Rainer Gerhards and Adiscon GmbH. + * Copyright 2007-2013 Rainer Gerhards and Adiscon GmbH. * * This file is part of rsyslog. * @@ -32,10 +32,10 @@ rsRetVal multiSubmitMsg2(multi_submit_t *pMultiSub); /* friends only! */ rsRetVal submitMsg2(msg_t *pMsg); rsRetVal __attribute__((deprecated)) submitMsg(msg_t *pMsg); rsRetVal multiSubmitFlush(multi_submit_t *pMultiSub); -rsRetVal logmsgInternal(int iErr, int pri, uchar *msg, int flags); +rsRetVal logmsgInternal(const int iErr, const int pri, const uchar *const msg, int flags); rsRetVal __attribute__((deprecated)) parseAndSubmitMessage(uchar *hname, uchar *hnameIP, uchar *msg, int len, int flags, flowControl_t flowCtlTypeu, prop_t *pInputName, struct syslogTime *stTime, time_t ttGenTime, ruleset_t *pRuleset); -rsRetVal diagGetMainMsgQSize(int *piSize); /* for imdiag */ -rsRetVal createMainQueue(qqueue_t **ppQueue, uchar *pszQueueName, struct cnfparamvals *queueParams); +rsRetVal createMainQueue(qqueue_t **ppQueue, uchar *pszQueueName, struct nvlst *lst); +rsRetVal startMainQueue(qqueue_t *pQueue); extern int MarkInterval; extern qqueue_t *pMsgQueue; /* the main message queue */ diff --git a/doc/Makefile.am b/doc/Makefile.am deleted file mode 100644 index 46afd90..0000000 --- a/doc/Makefile.am +++ /dev/null @@ -1,175 +0,0 @@ -html_files = \ - index.html \ - bugs.html \ - debug.html \ - features.html \ - generic_design.html \ - expression.html \ - droppriv.html \ - history.html \ - how2help.html \ - install.html \ - build_from_repo.html \ - ipv6.html \ - log_rotation_fix_size.html \ - manual.html \ - modules.html \ - property_replacer.html \ - rsyslog_ng_comparison.html \ - rsyslog_conf.html \ - rsyslog-example.conf \ - rsyslog_mysql.html \ - rsyslog_pgsql.html \ - rsyslog_packages.html \ - rsyslog_high_database_rate.html \ - rsyslog_php_syslog_ng.html \ - rsyslog_recording_pri.html \ - rsyslog_tls.html \ - rsyslog_reliable_forwarding.html \ - rsyslog_stunnel.html \ - syslog_protocol.html \ - version_naming.html \ - contributors.html \ - dev_queue.html \ - ompipe.html \ - omfwd.html \ - omfile.html \ - omjournal.html \ - mmanon.html \ - omusrmsg.html \ - omstdout.html \ - omudpspoof.html \ - omruleset.html \ - omsnmp.html \ - sigprov_gt.html \ - ommysql.html \ - omoracle.html \ - omlibdbi.html \ - imfile.html \ - imtcp.html \ - imptcp.html \ - impstats.html \ - imgssapi.html \ - imrelp.html \ - imsolaris.html \ - imuxsock.html \ - imklog.html \ - pmlastmsg.html \ - mmsnmptrapd.html \ - queues.html \ - src/queueWorkerLogic.dia \ - queueWorkerLogic.jpg \ - queueWorkerLogic_small.jpg \ - tls_cert_100.jpg \ - tls_cert_ca.jpg \ - tls_cert.jpg \ - tls_cert_errmsgs.html \ - rsyslog_secure_tls.html \ - tls_cert_server.html \ - tls_cert_ca.html \ - tls_cert_summary.html \ - tls_cert_machine.html \ - tls_cert_udp_relay.html \ - tls_cert_client.html \ - tls_cert_scenario.html \ - rainerscript.html \ - lookup_tables.html \ - rscript_abnf.html \ - rsconf1_actionexeconlywhenpreviousissuspended.html \ - rsconf1_actionresumeinterval.html \ - rsconf1_allowedsender.html \ - rsconf1_controlcharacterescapeprefix.html \ - rsconf1_escape8bitcharsonreceive.html \ - rsconf1_debugprintcfsyslinehandlerlist.html \ - rsconf1_debugprintmodulelist.html \ - rsconf1_debugprinttemplatelist.html \ - rsconf1_dircreatemode.html \ - rsconf1_dirgroup.html \ - rsconf1_dirowner.html \ - rsconf1_dropmsgswithmaliciousdnsptrrecords.html \ - rsconf1_droptrailinglfonreception.html \ - rsconf1_dynafilecachesize.html \ - rsconf1_escapecontrolcharactersonreceive.html \ - rsconf1_failonchownfailure.html \ - rsconf1_filecreatemode.html \ - rsconf1_filegroup.html \ - rsconf1_fileowner.html \ - rsconf1_generateconfiggraph.html \ - rsconf1_gssforwardservicename.html \ - rsconf1_gsslistenservicename.html \ - rsconf1_gssmode.html \ - rsconf1_includeconfig.html \ - rsconf1_mainmsgqueuesize.html \ - rsconf1_markmessageperiod.html \ - rsconf1_modload.html \ - rsconf1_moddir.html \ - rsconf1_repeatedmsgreduction.html \ - rsconf1_resetconfigvariables.html \ - rsconf1_rulesetcreatemainqueue.html \ - rsconf1_umask.html \ - rsconf1_rulesetparser.html \ - v3compatibility.html \ - v4compatibility.html \ - v5compatibility.html \ - im3195.html \ - netstream.html \ - ns_gtls.html \ - ns_ptcp.html \ - src/tls_cert.dia \ - gssapi.html \ - licensing.html \ - mmnormalize.html \ - mmjsonparse.html \ - ommail.html \ - omuxsock.html \ - omrelp.html \ - syslog_parsing.html \ - troubleshoot.html \ - rsyslog_conf_actions.html \ - rsyslog_conf_filter.html \ - rsyslog_conf_global.html \ - rsyslog_conf_modules.html \ - rsyslog_conf_output.html \ - rsyslog_conf_templates.html \ - rsyslog_conf_nomatch.html \ - queues_analogy.html \ - multi_ruleset.html \ - multi_ruleset_legacy_format.html \ - dev_oplugins.html \ - free_support.html \ - imudp.html \ - messageparser.html \ - omhdfs.html \ - omprog.html \ - queue_msg_state.jpeg \ - rsconf1_abortonuncleanconfig.html \ - rsconf1_maxopenfiles.html \ - rsconf1_omfileforcechown.html \ - rsyslog_queue_pointers.jpeg \ - rsyslog_queue_pointers2.jpeg \ - v6compatibility.html \ - v7compatibility.html \ - rsyslog_conf_basic_structure.html \ - rsyslog_conf_sysklogd_compatibility.html \ - imkmsg.html \ - src/classes.dia - -grfx_files = \ - rsyslog_confgraph_complex.png\ - rsyslog_confgraph_std.png \ - module_workflow.png \ - direct_queue0.png \ - direct_queue1.png \ - direct_queue2.png \ - direct_queue3.png \ - direct_queue_rsyslog.png \ - direct_queue_rsyslog2.png \ - direct_queue_directq.png \ - dataflow.png \ - queue_analogy_tv.png \ - gssapi.png \ - rfc5424layers.png \ - src/rfc5424layers.dia \ - rsyslog-vers.png - -EXTRA_DIST = $(html_files) $(grfx_files) diff --git a/doc/bugs.html b/doc/bugs.html deleted file mode 100644 index a12c43f..0000000 --- a/doc/bugs.html +++ /dev/null @@ -1,32 +0,0 @@ -<html> -<head> -<title>rsyslog bugs and annoyances</title> -</head> -<body> -<h1>rsyslog bugs and annoyances</h1> -<p><b>This page lists the known bugs rsyslog has to offer.</b> It lists -old and esoteric bugs. A live list of bugs is contained in our bugzilla. <b> -<font color="#FF0000">Please visit </font><a href="http://www.rsyslog.com/bugs"> -<font color="#FF0000">http://www.rsyslog.com/bugs</font></a></b> to see what we -have. There, you can also open your own bug report if you think you found one.</p> -<p>This list has last been updated on 2008-02-12 by -<a href="http://www.adiscon.com/en/people/rainer-gerhards.php">Rainer Gerhards</a>.</p> -<h1>rsyslogd</h1> -<h2>EQUALLY-NAMED TEMPLATES</h2> -<p>If multiple templates with the SAME name are created, all but the - first definition is IGNORED. So you can NOT (yet) replace a - template definition. I also strongly doubt I will ever support this, because -it does not make an awful lot of sense (after all, why not use two template -names...).</p> -<h2>WALLMSG FORMAT (* selector) - </h2> -<p>This format is actually not 100% compatible with stock syslogd - the - date is missing. Will be fixed soon and can also be fixed just via - the proper template. Anyone up for this? ;)</p> -<h2>MULTIPLE INSTANCES</h2> -<p>If multiple instances are running on a single machine, the one with - the -r switch must start first. Also, UDP-based syslog forwarding between the -instances does not work. Use TCP instead.</p> - -</body> -</html> diff --git a/doc/build_from_repo.html b/doc/build_from_repo.html deleted file mode 100644 index a06863e..0000000 --- a/doc/build_from_repo.html +++ /dev/null @@ -1,79 +0,0 @@ -<html><head> -<title>Building rsyslog from the source repository</title> -</head> -<body> -<h1>Building rsyslog from the source repository</h1> -<p>In most cases, people install rsyslog either via a package or use an "official" -distribution tarball to generate it. But there may be situations where it is desirable -to build directly from the source repository. This is useful for people who would like to -participate in development or who would like to use the latest, not-yet-released code. -The later may especially be the case if you are asked to try out an experimental version. -<p>Building from the repsitory is not much different than building from the source -tarball, but some files are missing because they are output files and thus do not -belong into the repository. -<h2>Obtaining the Source</h2> -<p>First of all, you need to download the sources. Rsyslog is currently kept in a git -repository. You can clone this repository either via http or git protocol (with the later -being much faster. URLS are: -<ul> -<li>git://git.adiscon.com/git/rsyslog.git -<li>http://git.adiscon.com/git/rsyslog.git -</ul> -<p>There is also a browsable version (gitweb) available at -<a href="http://git.adiscon.com/?p=rsyslog.git;a=summary">http://git.adiscon.com/?p=rsyslog.git;a=summary</a>. -This version also offers snapshots of each commit for easy download. You can use these if -you do not have git present on your system. -<p>After you have cloned the repository, you are in the master branch by default. This -is where we keep the devel branch. If you need any other branch, you need to do -a "git checkout --track -b branch origin/branch". For example, the command to check out -the beta branch is "git checkout --track -b beta origin/beta". -<h2>Prequisites</h2> -<p>To build the compilation system, you need the <b>pkg-config</b> package (an utility for -autotools) present on your system. Otherwise, configure will fail with something like -<pre><code> -checking for SYSLOG_UNIXAF support... yes -checking for FSSTND support... yes -./configure: line 25895: syntax error near unexpected token `RELP,' -./configure: line 25895: ` PKG_CHECK_MODULES(RELP, relp >= 0.1.1)' -</code></pre> -<h2>Creating the Build Environment</h2> -<p>This is fairly easy: just issue "<b>autoreconf -fvi</b>", which should do everything you need. -Once this is done, you can follow the usual ./configure steps just like when -you downloaded an official distribution tarball (see the -<a href="install.html">rsyslog install guide</a>, starting at step 2, -for further details about that). - -<h2>Special Compile-Time Options</h2> -<p>On some platforms, compile-time issues occur, like the one shown below: -<p><pre><code> -make[2]: Entering directory `/home/az/RSyslog/rsyslog-5.5.0/tools' - CCLD rsyslogd -rsyslogd-omfile.o: In function `getClockFileAccess': -/home/az/RSyslog/rsyslog-5.5.0/tools/omfile.c:91: undefined reference to `__sync_fetch_and_add_8' -/home/az/RSyslog/rsyslog-5.5.0/tools/omfile.c:91: undefined reference to `__sync_fetch_and_add_8' -/home/az/RSyslog/rsyslog-5.5.0/tools/omfile.c:91: undefined reference to `__sync_fetch_and_add_8' -</code></pre> -<p>Note that the exact error messages can be different. These type of errors stem down to -atomic instruction support in GCC, which is somewhat depending on the machine architecture it -compiles code for. Very old machines (like the original i386) do not even at all provide support -for these instructions. -<p>The availability of atomic instructions is vital for rsyslog - it can not be built without them. -Consequently, there is a configure check included for them. But under some circumstances, -GCC seems to report they are available, but does not provide implementations for -all of them (at least this is my observation...). The simple cure is to make sure that -GCC generates code for a modern-enough architecture. This, for example, can be done as -follows: -<p><pre><code> -./configure CFLAGS="-march=i586 -mcpu=i686" --enable-imfile ... (whatever you need) -</code></pre> -<p>These settings should resolve the issue . - -<p>[<a href="manual.html">manual index</a>] -[<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> -<p><font size="2">This documentation is part of the -<a href="http://www.rsyslog.com/">rsyslog</a> project.<br> -Copyright © 2008, 2009 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and -<a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL -version 3 or higher.</font></p> -</body> -</html> diff --git a/doc/contributors.html b/doc/contributors.html deleted file mode 100644 index 713c329..0000000 --- a/doc/contributors.html +++ /dev/null @@ -1,60 +0,0 @@ -<html> -<head> -<title>Contributor Hall of Fame</title> -</head> -<body> -<h2>Contributor Hall of Fame</h2> -<p><b>This page is dedicated to all the people who helped make -<a href="http://www.rsyslog.com/">rsyslog</a> become a reality.</b> -Unfortunately, I have begun this page in July of 2007, long after the project -started. I try to extract all past contributor information from CVS, readme's, -code etc - but I may fail. If you contributed and do not find yourself listed -below, please accept my sincere apologies and drop me a line.</p> -<p>Please also note that I will do the checks for past contributors once the -current very busy development phase is over, so it may take a few weeks to fully -populate this file.</p> -<p>Contributors are listed in alphabetical order. If I know an Alias only, that -alias is used as heading. Else the real name is used. If I know first and last -name, they are listed in that order ("Rainer Gerhards" and not "Gerhards, -Rainer"). I tend to be sparse with information on contributors, at least without -their permission. If you contribute, let me know if I may include your email -and/or web page address.</p> -<p>Thanks to all past, present and future contributors!</p> -<p><a href="http://www.gerhards.net/rainer">Rainer Gerhards</a><br> -Project Initiator and Maintainer</p> -<h2>Bartosz Kuzma</h2> -<ul> - <li>provided many contributions before I started this list, so there are - probably some missing</li> - <li>suggested the use of autotools in parallel to Peter Vrabec and helped me - get it going</li> - <li>sent a number of patches (see cvs log for details)</li> -</ul> -<h2>Michel Samia</h2> -<ul> - <li>provided patch with regex functionality for filters on 2007-07-14, first - seen in 1.16.1</li> -</ul> -<h2>mildew@gmail.com</h2> -<ul> - <li>provided a large patch to enhance $AllowedSender directive for IPv6 as - well as DNS names</li> -</ul> -<h2>Peter Vrabec</h2> -<ul> - <li>provided many contributions before I started this list, so there are - probably some missing</li> - <li>provided basic IPv6 support</li> - <li>convinced me to use autotools and provided the first working config for - it</li> - <li>provided Rainer with ongoing support, inspiration and motivation</li> -</ul> -<h2>varmojfekoj@gmail.com</h2> -<ul> - <li>provided contributions before I started this list, so there are probably - some missing</li> - <li>provided patches for several memory leaks</li> -</ul> -<p><font size="2">Last Updated: 2007-07-19</font></p> -</body> -</html> diff --git a/doc/dataflow.png b/doc/dataflow.png Binary files differdeleted file mode 100644 index fd614d8..0000000 --- a/doc/dataflow.png +++ /dev/null diff --git a/doc/debug.html b/doc/debug.html deleted file mode 100644 index 996bf5c..0000000 --- a/doc/debug.html +++ /dev/null @@ -1,170 +0,0 @@ -<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> -<html> -<head> -<meta http-equiv="Content-Language" content="en"> -<title>Rsyslog Debug Support</title></head> -<body> -<h1>Rsyslog Debug Support</h1> -<p> -Rsyslog provides a number of debug aides. Some of them are activated by -adding the --enable-rtinst ./configure option ("rtinst" means runtime -instrumentation). Turning debugging on obviously costs some performance -(in some cases considerable). -</p> -<p>This is document is just being created and thus terse.</p> -<h2>Signals supported</h2> -<p><b>SIGUSR1</b> - turns debug messages on and off. Note that for this -signal to work, rsyslogd must be running with debugging enabled, either -via the -d command line switch or the environment options specified below. -It is <b>not</b> required that rsyslog was compiled with debugging enabled -(but depending on the settings this may lead to better debug info). -<p><b>SIGUSR2</b> - outputs debug information (including active threads -and a call stack) for the state when SIGUSR2 was received. This is a -one-time output. Can be sent as often as the user likes. -<p><b>Note:</b> this signal <b>may go away</b> in later releases and may -be replaced by something else.</p> -<h2>Environment Variables</h2> -<p>There are two environment variables that set several debug settings: -<ul> -<li>The "RSYSLOG_DEBUGLOG" (sample: RSYSLOG_DEBUGLOG="/path/to/debuglog/") -writes (allmost) -all debug message to the specified log file in addition to stdout. Some -system messages (e.g. segfault or abort message) are not written to the -file as we can not capture them. -<li>Runtime debug support is controlled by "RSYSLOG_DEBUG". -<p>The "RSYSLOG_DEBUG" environment variable contains an option string with the following -options possible (all are case insensitive):</p> -<ul> -<li><b>LogFuncFlow</b> - print out the logical flow of functions (entering and exiting them)</li> -<li><b>FileTrace</b> - specifies which files to trace LogFuncFlow. If <b>not</b> -set (the default), a LogFuncFlow trace is provided for all files. Set -to limit it to the files specified. FileTrace may be specified multiple -times, one file each (e.g. export RSYSLOG_DEBUG="LogFuncFlow -FileTrace=vm.c FileTrace=expr.c"</li> -<li><b>PrintFuncDB</b> - print the content of the debug function database whenever debug information is printed (e.g. abort case)!</li> -<li><b>PrintAllDebugInfoOnExit</b> - print all debug information immediately before rsyslogd exits (<span style="font-weight: bold; font-style: italic;">currently not implemented!</span>)</li> -<li><b>PrintMutexAction</b> - print mutex action as it happens. Useful for finding deadlocks and such.</li> -<li><b>NoLogTimeStamp</b> - do not prefix log lines with a timestamp (default is to do that).</li> -<li><b>NoStdOut</b> - do not emit debug messages to stdout. If RSYSLOG_DEBUGLOG is not set, this means no messages will be displayed at all.</li> -<li><b>Debug</b> - if present, turns on the debug system and enables debug output -<li><b>DebugOnDemand</b> - if present, turns on the debug system but does not enable -debug output itself. You need to send SIGUSR1 to turn it on when desired. -<li><b>OutputTidToStderr</b> - if present, makes rsyslog output information about -the thread id (tid) of newly create processesto stderr. Note that not necessarily -all new threads are reported (depends on the code, e.g. of plugins). This is -only available under Linux. This usually does NOT work when privileges have -been dropped (that's not a bug, but the way it is). -<li><b>help</b> - display a very short list of commands - hopefully a life saver if you can't access the documentation...</li> -</ul> -</ul> -<h3>Why Environment Variables?</h3> -<p>You may ask why we use environment variables for debug-system parameters and not -the usual rsyslog.conf configuration commands. After all, environment variables force one -to change distro-specific configuration files, whereas regular configuration directives -would fit nicely into the one central rsyslog.conf. -<p>The problem here is that many settings of the debug system must be initialized -before the full rsyslog engine starts up. At that point, there is no such thing like -rsyslog.conf or the objects needed to process it present in an running instance. -And even if we would enable to change settings some time later, that would mean that -we can not correctly monitor (and debug) the initial startup phase of rsyslogd. What -makes matters worse is that during this startup phase (and never again later!) some -of the base debug structure needs to be created, at least if the build is -configured for that (many of these things only happen in --enable-rtinst mode). So -if we do not initialize the debug system <b>before</b> actually startig up the -rsyslog core, we get a number of data structures wrong. -<p>For these reasons, we utilize environment variables to initialize and configure -the debugging system. We understand this may be somewhat painful, but now you know -there are at least some good reasons for doing so. -<p>HOWEVER, if you have a too hard time to set debug instructions using the environment -variables, there is a cure, described in the next paragraph. - -<h2>Enabling Debug via rsyslog.conf</h2> -<p>As described in the previous paragraph, enabling debug via rsyslog.conf -may not be perfect for some debugging needs, but basic debug output will work - and -that is what most often is requried. There are limited options available, but these -cover the most important use cases. -<p>Debug processing is done via legacy config statements. There currently -is no plan to move these over to the v6+ config system. Availabe settings are -<ul> -<li>$DebugFile <filename> - sets the debug file name -<li>$DebugLevel <0|1|2> - sets the respective debug level, where -0 means debug off, 1 is debug on demand activated (but debug mode off) -and 2 is full debug mode. -</ul> -<p>Note that in theory it is forbidden to specify these parameters more -than once. However, we do not enforce that and if it happens results -are undefined. - -<h2>Getting debug information from a running Instance</h2> -<p>It is possible to obtain debugging information from a running instance, but this requires -some setup. We assume that the instance runs in the background, so debug output to -stdout is not desired. As such, all debug information needs to go into a log file. -<p>To create this setup, you need to -<ul> -<li>point the RSYSLOG_DEBUGLOG environment variable to a file that is accessible -during the while runtime (we strongly suggest a file in the local file system!) -<li>set RSYSLOG_DEBUG at least to "DebugOnDeman NoStdOut" -<li>make sure these environment variables are set in the correct (distro-specifc) -startup script if you do not run rsyslogd interactively -</ul> -<p>These settings enable the capability to react to SIGUSR1. The signal will toggle -debug status when received. So send it one to turn debug loggin on, and send it again -to turn debug logging off again. The third time it will be turned on again ... and so on. -<p>On a typical system, you can signal rsyslogd as follows: -<pre> -kill -USR1 `cat /var/run/rsyslogd.pid` -</pre> -Important: there are backticks around the "cat"-command. If you use the regular -quote it won't work. The debug log will show whether debug logging has been turned -on or off. There is no other indication of the status. -<p>Note: running with DebugOnDemand by itself does in practice not have any performance -toll. However, switching debug logging on has a severe performance toll. Also, debug -logging synchronizes much of the code, removing a lot of concurrency and thus -potential race conditions. As such, the very same running instance may behave -very differently with debug logging turned on vs. off. The on-demand debug log -functionality is considered to be very valuable to analyze hard-to-find bugs that -only manifest after a long runtime. Turning debug logging on a failing instance -may reveal the cause of the failure. However, depending on the failure, debug logging -may not even be successfully be turned on. Also note that with this rsyslog version we cannot -obtain any debug information on events that happened <i>before</i> debug logging was -turned on. -<p>If an instance hangs, it is possible to obtain some useful information about the current -threads and their calling stack by sending SIGUSR2. However, the usefulness of that -information is very much depending on rsyslog compile-time settings, must importantly -the --enable-rtinst configure flag. Note that activating this option causes additional overhead -and slows down rsyslgod considerable. So if you do that, you need to check if it is -capable to handle the workload. Also, threading behavior is modified by the -runtime instrumentation. -<p>Sending SIGUSR2 writes new process state information to the log file each time -it is sent. So it may be useful to do that from time to time. It probably is most -useful if the process seems to hang, in which case it may (may!) be able to output -some diagnostic information on the current processing state. In that case, turning -on the mutex debugging options (see above) is probably useful. -<h2>Interpreting the Logs</h2> -<p>Debug logs are primarily meant for rsyslog developers. But they may still provide valuable -information to users. Just be warned that logs sometimes contains informaton the looks like -an error, but actually is none. We put a lot of extra information into the logs, and there -are some cases where it is OK for an error to happen, we just wanted to record it inside -the log. The code handles many cases automatically. So, in short, the log may not make sense to -you, but it (hopefully) makes sense to a developer. Note that we developers often need -many lines of the log file, it is relatively rare that a problem can be diagnosed by -looking at just a couple of (hundered) log records. -<h2>Security Risks</h2> -<p>The debug log will reveal potentially sensible information, including user accounts and -passwords, to anyone able to read the log file. As such, it is recommended to properly -guard access to the log file. Also, an instance running with debug log enabled runs much -slower than one without. An attacker may use this to place carry out a denial-of-service -attack or try to hide some information from the log file. As such, it is suggested to -enable DebugOnDemand mode only for a reason. Note that when no debug mode is enabled, -SIGUSR1 and SIGUSR2 are completely ignored. -<p>When running in any of the debug modes (including on demand mode), an interactive -instance of rsyslogd can be aborted by pressing ctl-c. -<p> -<p>[<a href="manual.html">manual index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> -<p><font size="2">This documentation is part of the -<a href="http://www.rsyslog.com/">rsyslog</a> project.<br> -Copyright © 2008-2013 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and -<a href="http://www.adiscon.com/">Adiscon</a>. -Released under the GNU GPL version 3 or higher.</font></p> -</body> -</html> diff --git a/doc/dev_oplugins.html b/doc/dev_oplugins.html deleted file mode 100644 index b33b67f..0000000 --- a/doc/dev_oplugins.html +++ /dev/null @@ -1,336 +0,0 @@ -<html> -<head> -<title>writing rsyslog output plugins (developer's guide)</title> -</head> -<body> -<h1>Writing Rsyslog Output Plugins</h1> -<p>This page is the begin of some developer documentation for writing output -plugins. Doing so is quite easy (and that was a design goal), but there currently -is only sparse documentation on the process available. I was tempted NOT to -write this guide here because I know I will most probably not be able to -write a complete guide. -<p>However, I finally concluded that it may be better to have same information -and pointers than to have nothing. -<h2>Getting Started and Samples</h2> -<p>The best to get started with rsyslog plugin development is by looking at -existing plugins. All that start with "om" are <b>o</b>utput <b>m</b>odules. That -means they are primarily thought of being message sinks. In theory, however, output -plugins may aggergate other functionality, too. Nobody has taken this route so far -so if you would like to do that, it is highly suggested to post your plan on the -rsyslog mailing list, first (so that we can offer advise). -<p>The rsyslog distribution tarball contains two plugins that are extremely well -targeted for getting started: -<ul> -<li>omtemplate -<li>omstdout -</ul> -Plugin omtemplate was specifically created to provide a copy template for new output -plugins. It is bare of real functionality but has ample comments. Even if you decide -to start from another plugin (or even from scratch), be sure to read omtemplate source -and comments first. The omstdout is primarily a testing aide, but offers support for -the two different parameter-passing conventions plugins can use (plus the way to -differentiate between the two). It also is not bare of functionaly, only mostly -bare of it ;). But you can actually execute it and play with it. -<p>In any case, you should also read the comments in ./runtime/module-template.h. -Output plugins are build based on a large set of code-generating macros. These -macros handle most of the plumbing needed by the interface. As long as no -special callback to rsyslog is needed (it typically is not), an output plugin does -not really need to be aware that it is executed by rsyslog. As a plug-in programmer, -you can (in most cases) "code as usual". However, all macros and entry points need to be -provided and thus reading the code comments in the files mentioned is highly suggested. -<p>In short, the best idea is to start with a template. Let's assume you start by -copying omtemplate. Then, the basic steps you need to do are: -<ul> -<li>cp ./plugins/omtemplate ./plugins/your-plugin -<li>mv cd ./plugins/your-plugin -<li>vi Makefile.am, adjust to your-plugin -<li>mv omtemplate.c your-plugin.c -<li>cd ../.. -<li>vi Makefile.am configure.ac -<br>search for omtemplate, copy and modify (follow comments) -</ul> -<p>Basically, this is all you need to do ... Well, except, of course, coding -your plugin ;). For testing, you need rsyslog's debugging support. Some useful -information is given in "<a href="troubleshoot.html">troubleshooting rsyslog</a> -from the doc set. -<h2>Special Topics</h2> -<h3>Threading</h3> -<p>Rsyslog uses massive parallel processing and multithreading. However, a plugin's entry -points are guaranteed to be never called concurrently <b>for the same action</b>. -That means your plugin must be able to be called concurrently by two or more -threads, but you can be sure that for the same instance no concurrent calls -happen. This is guaranteed by the interface specification and the rsyslog core -guards against multiple concurrent calls. An instance, in simple words, is one -that shares a single instanceData structure. -<p>So as long as you do not mess around with global data, you do not need -to think about multithreading (and can apply a purely sequential programming -methodology). -<p>Please note that duringt the configuraton parsing stage of execution, access to -global variables for the configuration system is safe. In that stage, the core will -only call sequentially into the plugin. -<h3>Getting Message Data</h3> -<p>The doAction() entry point of your plugin is provided with messages to be processed. -It will only be activated after filtering and all other conditions, so you do not need -to apply any other conditional but can simply process the message. -<p>Note that you do NOT receive the full internal representation of the message -object. There are various (including historical) reasons for this and, among -others, this is a design decision based on security. -<p>Your plugin will only receive what the end user has configured in a $template -statement. However, starting with 4.1.6, there are two ways of receiving the -template content. The default mode, and in most cases sufficient and optimal, -is to receive a single string with the expanded template. As I said, this is usually -optimal, think about writing things to files, emailing content or forwarding it. -<p>The important philosophy is that a plugin should <b>never</b> reformat any -of such strings - that would either remove the user's ability to fully control -message formats or it would lead to duplicating code that is already present in the -core. If you need some formatting that is not yet present in the core, suggest it -to the rsyslog project, best done by sending a patch ;), and we will try hard to -get it into the core (so far, we could accept all such suggestions - no promise, though). -<p>If a single string seems not suitable for your application, the plugin can also -request access to the template components. The typical use case seems to be databases, where -you would like to access properties via specific fields. With that mode, you receive a -char ** array, where each array element points to one field from the template (from -left to right). Fields start at arrray index 0 and a NULL pointer means you have -reached the end of the array (the typical Unix "poor man's linked list in an array" -design). Note, however, that each of the individual components is a string. It is -not a date stamp, number or whatever, but a string. This is because rsyslog processes -strings (from a high-level design look at it) and so this is the natural data type. -Feel free to convert to whatever you need, but keep in mind that malformed packets -may have lead to field contents you'd never expected... -<p>If you like to use the array-based parameter passing method, think that it -is only available in rsyslog 4.1.6 and above. If you can accept that your plugin -will not be working with previous versions, you do not need to handle pre 4.1.6 cases. -However, it would be "nice" if you shut down yourself in these cases - otherwise the -older rsyslog core engine will pass you a string where you expect the array of pointers, -what most probably results in a segfault. To check whether or not the core supports the -functionality, you can use this code sequence: -<pre> -<code> -BEGINmodInit() - rsRetVal localRet; - rsRetVal (*pomsrGetSupportedTplOpts)(unsigned long *pOpts); - unsigned long opts; - int bArrayPassingSupported; /* does core support template passing as an array? */ -CODESTARTmodInit - *ipIFVersProvided = CURR_MOD_IF_VERSION; /* we only support the current interface specification */ -CODEmodInit_QueryRegCFSLineHdlr - /* check if the rsyslog core supports parameter passing code */ - bArrayPassingSupported = 0; - localRet = pHostQueryEtryPt((uchar*)"OMSRgetSupportedTplOpts", &pomsrGetSupportedTplOpts); - if(localRet == RS_RET_OK) { - /* found entry point, so let's see if core supports array passing */ - CHKiRet((*pomsrGetSupportedTplOpts)(&opts)); - if(opts & OMSR_TPL_AS_ARRAY) - bArrayPassingSupported = 1; - } else if(localRet != RS_RET_ENTRY_POINT_NOT_FOUND) { - ABORT_FINALIZE(localRet); /* Something else went wrong, what is not acceptable */ - } - DBGPRINTF("omstdout: array-passing is %ssupported by rsyslog core.\n", bArrayPassingSupported ? "" : "not "); - - if(!bArrayPassingSupported) { - DBGPRINTF("rsyslog core too old, shutting down this plug-in\n"); - ABORT_FINALIZE(RS_RET_ERR); - } - -</code> -</pre> -<p>The code first checks if the core supports the OMSRgetSupportedTplOpts() API (which is -also not present in all versions!) and, if so, queries the core if the OMSR_TPL_AS_ARRAY mode -is supported. If either does not exits, the core is too old for this functionality. The sample -snippet above then shuts down, but a plugin may instead just do things different. In -omstdout, you can see how a plugin may deal with the situation. -<p><b>In any case, it is recommended that at least a graceful shutdown is made and the -array-passing capability not blindly be used.</b> In such cases, we can not guard the -plugin from segfaulting and if the plugin (as currently always) is run within -rsyslog's process space, that results in a segfault for rsyslog. So do not do this. -<p>Another possible mode is OMSR_TPL_AS_JSON, where instead of the template -a json-c memory object tree is passed to the module. The module can extract data -via json-c API calls. It MUST NOT modify the provided structure. This mode is -primarily aimed at plugins that need to process tree-like data, as found -for example in MongoDB or ElasticSearch. -<h3>Batching of Messages</h3> -<p>Starting with rsyslog 4.3.x, batching of output messages is supported. Previously, only -a single-message interface was supported. -<p>With the <b>single message</b> plugin interface, each message is passed via a separate call to the plugin. -Most importantly, the rsyslog engine assumes that each call to the plugin is a complete transaction -and as such assumes that messages be properly commited after the plugin returns to the engine. -<p>With the <b>batching</b> interface, rsyslog employs something along the line of -"transactions". Obviously, the rsyslog core can not make non-transactional outputs -to be fully transactional. But what it can is support that the output tells the core which -messages have been commited by the output and which not yet. The core can than take care -of those uncommited messages when problems occur. For example, if a plugin has received -50 messages but not yet told the core that it commited them, and then returns an error state, the -core assumes that all these 50 messages were <b>not</b> written to the output. The core then -requeues all 50 messages and does the usual retry processing. Once the output plugin tells the -core that it is ready again to accept messages, the rsyslog core will provide it with these 50 -not yet commited messages again (actually, at this point, the rsyslog core no longer knows that -it is re-submiting the messages). If, in contrary, the plugin had told rsyslog that 40 of these 50 -messages were commited (before it failed), then only 10 would have been requeued and resubmitted. -<p>In order to provide an efficient implementation, there are some (mild) constraints in that -transactional model: first of all, rsyslog itself specifies the ultimate transaction boundaries. -That is, it tells the plugin when a transaction begins and when it must finish. The plugin -is free to commit messages in between, but it <b>must</b> commit all work done when the core -tells it that the transaction ends. All messages passed in between a begin and end transaction -notification are called a batch of messages. They are passed in one by one, just as without -transaction support. Note that batch sizes are variable within the range of 1 to a user configured -maximum limit. Most importantly, that means that plugins may receive batches of single messages, -so they are required to commit each message individually. If the plugin tries to be "smarter" -than the rsyslog engine and does not commit messages in those cases (for example), the plugin -puts message stream integrity at risk: once rsyslog has notified the plugin of transacton end, -it discards all messages as it considers them committed and save. If now something goes wrong, -the rsyslog core does not try to recover lost messages (and keep in mind that "goes wrong" -includes such uncontrollable things like connection loss to a database server). So it is -highly recommended to fully abide to the plugin interface details, even though you may -think you can do it better. The second reason for that is that the core engine will -have configuration settings that enable the user to tune commit rate to their use-case -specific needs. And, as a relief: why would rsyslog ever decide to use batches of one? -There is a trivial case and that is when we have very low activity so that no queue of -messages builds up, in which case it makes sense to commit work as it arrives. -(As a side-note, there are some valid cases where a timeout-based commit feature makes sense. -This is also under evaluation and, once decided, the core will offer an interface plus a way -to preserve message stream integrity for properly-crafted plugins). -<p>The second restriction is that if a plugin makes commits in between (what is perfectly -legal) those commits must be in-order. So if a commit is made for message ten out of 50, -this means that messages one to nine are also commited. It would be possible to remove -this restriction, but we have decided to deliberately introduce it to simpify things. -<h3>Output Plugin Transaction Interface</h3> -<p>In order to keep compatible with existing output plugins (and because it introduces -no complexity), the transactional plugin interface is build on the traditional -non-transactional one. Well... actually the traditional interface was transactional -since its introduction, in the sense that each message was processed in its own -transaction. -<p>So the current <code>doAction()</b> entry point can be considered to have this -structure (from the transactional interface point of view): -<p><pre><code> -doAction() - { - beginTransaction() - ProcessMessage() - endTransaction() - } - </code></pre> -<p>For the <b>transactional interface</b>, we now move these implicit <code>beginTransaction()</code> -and <code>endTransaction(()</code> call out of the message processing body, resulting is such -a structure: -<p><pre><code> -beginTransaction() - { - /* prepare for transaction */ - } - -doAction() - { - ProcessMessage() - /* maybe do partial commits */ - } - -endTransaction() - { - /* commit (rest of) batch */ - } -</code></pre> -<p>And this calling structure actually is the transactional interface! It is as simple as this. -For the new interface, the core calls a <code>beginTransaction()</code> entry point inside the -plugin at the start of the batch. Similarly, the core call <code>endTransaction()</code> at the -end of the batch. The plugin must implement these entry points according to its needs. -<p>But how does the core know when to use the old or the new calling interface? This is rather -easy: when loading a plugin, the core queries the plugin for the <code>beginTransaction()</code> -and <code>endTransaction()</code> entry points. If the plugin supports these, the new interface is -used. If the plugin does not support them, the old interface is used and rsyslog implies that -a commit is done after each message. Note that there is no special "downlevel" handling -necessary to support this. In the case of the non-transactional interface, rsyslog considers -each completed call to <code>doAction</code> as partial commit up to the current message. -So implementation inside the core is very straightforward. -<p>Actually, <b>we recommend that the transactional entry points only be defined by those -plugins that actually need them</b>. All others should not define them in which case -the default commit behaviour inside rsyslog will apply (thus removing complexity from the -plugin). -<p>In order to support partial commits, special return codes must be defined for -<code>doAction</code>. All those return codes mean that processing completed successfully. -But they convey additional information about the commit status as follows: -<p> -<table border="0"> -<tr> -<td valign="top"><i>RS_RET_OK</i></td> -<td>The record and all previous inside the batch has been commited. -<i>Note:</i> this definition is what makes integrating plugins without the -transaction being/end calls so easy - this is the traditional "success" return -state and if every call returns it, there is no need for actually calling -<code>endTransaction()</code>, because there is no transaction open).</td> -</tr> -<tr> -<td valign="top"><i>RS_RET_DEFER_COMMIT</i></td> -<td>The record has been processed, but is not yet commited. This is the -expected state for transactional-aware plugins.</td> -</tr> -<tr> -<td valign="top"><i>RS_RET_PREVIOUS_COMMITTED</i></td> -<td>The <b>previous</b> record inside the batch has been committed, but the -current one not yet. This state is introduced to support sources that fill up -buffers and commit once a buffer is completely filled. That may occur halfway -in the next record, so it may be important to be able to tell the -engine the everything up to the previouos record is commited</td> -</tr> -</table> -<p>Note that the typical <b>calling cycle</b> is <code>beginTransaction()</code>, -followed by <i>n</i> times -<code>doAction()</code></n> followed by <code>endTransaction()</code>. However, if either -<code>beginTransaction()</code> or <code>doAction()</code> return back an error state -(including RS_RET_SUSPENDED), then the transaction is considered aborted. In result, the -remaining calls in this cycle (e.g. <code>endTransaction()</code>) are never made and a -new cycle (starting with <code>beginTransaction()</code> is begun when processing resumes. -So an output plugin must expect and handle those partial cycles gracefully. -<p><b>The question remains how can a plugin know if the core supports batching?</b> -First of all, even if the engine would not know it, the plugin would return with RS_RET_DEFER_COMMIT, -what then would be treated as an error by the engine. This would effectively disable the -output, but cause no further harm (but may be harm enough in itself). -<p>The real solution is to enable the plugin to query the rsyslog core if this feature is -supported or not. At the time of the introduction of batching, no such query-interface -exists. So we introduce it with that release. What the means is if a rsyslog core can -not provide this query interface, it is a core that was build before batching support -was available. So the absence of a query interface indicates that the transactional -interface is not available. One might now be tempted the think there is no need to do -the actual check, but is is recommended to ask the rsyslog engine explicitely if -the transactional interface is present and will be honored. This enables us to -create versions in the future which have, for whatever reason we do not yet know, no -support for this interface. -<p>The logic to do these checks is contained in the <code>INITChkCoreFeature</code> macro, -which can be used as follows: -<p><pre><code> -INITChkCoreFeature(bCoreSupportsBatching, CORE_FEATURE_BATCHING); -</code></pre> -<p>Here, bCoreSupportsBatching is a plugin-defined integer which after execution is -1 if batches (and thus the transational interface) is supported and 0 otherwise. -CORE_FEATURE_BATCHING is the feature we are interested in. Future versions of rsyslog -may contain additional feature-test-macros (you can see all of them in -./runtime/rsyslog.h). -<p>Note that the ompsql output plugin supports transactional mode in a hybrid way and -thus can be considered good example code. - -<h2>Open Issues</h2> -<ul> -<li>Processing errors handling -<li>reliable re-queue during error handling and queue termination -</ul> - - - -<h3>Licensing</h3> -<p>From the rsyslog point of view, plugins constitute separate projects. As such, -we think plugins are not required to be compatible with GPLv3. However, this is -no legal advise. If you intend to release something under a non-GPLV3 compatible license -it is probably best to consult with your lawyer. -<p>Most importantly, and this is definite, the rsyslog team does not expect -or require you to contribute your plugin to the rsyslog project (but of course -we are happy if you do). -<h2>Copyright</h2> -<p>Copyright (c) 2009 <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> -and <a href="http://www.adiscon.com/en/">Adiscon</a>.</p> -<p>Permission is granted to copy, distribute and/or modify this document under -the terms of the GNU Free Documentation License, Version 1.2 or any later -version published by the Free Software Foundation; with no Invariant Sections, -no Front-Cover Texts, and no Back-Cover Texts. A copy of the license can be -viewed at <a href="http://www.gnu.org/copyleft/fdl.html"> -http://www.gnu.org/copyleft/fdl.html</a>.</p> -</body> -</html> diff --git a/doc/dev_queue.html b/doc/dev_queue.html deleted file mode 100644 index bf2af7f..0000000 --- a/doc/dev_queue.html +++ /dev/null @@ -1,250 +0,0 @@ -<html> -<head> -<title>rsyslog queue object</title> -</head> -<body> -<h1>The rsyslog queue object</h1> -<p>This page reflects the status as of 2008-01-17. The documentation is still incomplete. -Target audience is developers and users who would like to get an in-depth understanding of -queues as used in <a href="http://www.rsyslog.com/">rsyslog</a>.</p> -<p><b>Please note that this document is outdated and does not longer reflect the -specifics of the queue object. However, I have decided to leave it in the doc -set, as the overall picture provided still is quite OK. I intend to update this -document somewhat later when I have reached the "store-and-forward" milestone.</b></p> -<h1>Some definitions</h1> -<p>A queue is DA-enabled if it is configured to use disk-assisted mode when -there is need to. A queue is in DA mode (or DA run mode), when it actually runs -disk assisted.</p> -<h1>Implementation Details</h1> -<h2>Disk-Assisted Mode</h2> -<p>Memory-Type queues may utilize disk-assisted (DA) mode. DA mode is enabled -whenever a queue file name prefix is provided. This is called DA-enabled mode. -If DA-enabled, the queue operates as a regular memory queue until a high water -mark is reached. If that happens, the queue activates disk assistance (called -"runs disk assisted" or "runs DA" - you can find that often in source file -comments). To do so, it creates a helper queue instance (the DA queue). At that -point, there are two queues running - the primary queue's consumer changes to a -shuffle-to-DA-queue consumer and the original primary consumer is assigned to -the DA queue. Existing and new messages are spooled to the disk queue, where the -DA worker takes them from and passes them for execution to the actual consumer. -In essence, the primary queue has now become a memory buffer for the DA queue. -The primary queue will be drained until a low water mark is reached. At that -point, processing is held. New messages enqueued to the primary queue will not -be processed but kept in memory. Processing resumes when either the high water -mark is reached again or the DA queue indicates it is empty. If the DA queue is -empty, it is shut down and processing of the primary queue continues as a -regular in-memory queue (aka "DA mode is shut down"). The whole thing iterates -once the high water mark is hit again.</p> -<p>There is one special case: if the primary queue is shut down and could not -finish processing all messages within the configured timeout periods, the DA -queue is instantiated to take up the remaining messages. These will be preserved -and be processed during the next run. During that period, the DA queue runs in -"enqueue-only" mode and does not execute any consumer. Draining the primary -queue is typically very fast. If that behaviour is not desired, it can be turned -of via parameters. In that case, any remaining in-memory messages are lost.</p> -<p>Due to the fact that when running DA two queues work closely together and -worker threads (including the DA worker) may shut down at any time (due to -timeout), processing synchronization and startup and shutdown is somewhat -complex. I'll outline the exact conditions and steps down here. I also do this -so that I know clearly what to develop to, so please be patient if the -information is a bit too in-depth ;)</p> -<h2>DA Run Mode Initialization</h2> -<p>Three cases:</p> -<ol> - <li>any time during queueEnqObj() when the high water mark is hit</li> - <li>at queue startup if there is an on-disk queue present (presence of QI - file indicates presence of queue data)</li> - <li>at queue shutdown if remaining in-memory data needs to be persisted to - disk</li> -</ol> -<p>In <b>case 1</b>, the worker pool is running. When switching to DA mode, all -regular workers are sent termination commands. The DA worker is initiated. -Regular workers may run in parallel to the DA worker until they terminate. -Regular workers shall terminate as soon as their current consumer has completed. -They shall not execute the DA consumer.</p> -<p>In <b>case 2</b>, the worker pool is not yet running and is NOT started. The -DA worker is initiated.</p> -<p>In <b>case 3</b>, the worker pool is already shut down. The DA worker is -initiated. The DA queue runs in enqueue-only mode.</p> -<p>In all cases, the DA worker starts up and checks if DA mode is already fully -initialized. If not, it initializes it, what most importantly means construction -of the queue.</p> -<p>Then, regular worker processing is carried out. That is, the queue worker -will wait on empty queue and terminate after an timeout. However, If any message -is received, the DA consumer is executed. That consumer checks the low water -mark. If the low water mark is reached, it stops processing until either the -high water mark is reached again or the DA queue indicates it is empty (there is -a pthread_cond_t for this synchronization).</p> -<p>In theory, a <b>case-2</b> startup could lead to the worker becoming inactive -and terminating while waiting on the primary queue to fill. In practice, this is -highly unlikely (but only for the main message queue) because rsyslog issues a -startup message. HOWEVER, we can not rely on that, it would introduce a race. If -the primary rsyslog thread (the one that issues the message) is scheduled very -late and there is a low inactivty timeout for queue workers, the queue worker -may terminate before the startup message is issued. And if the on-disk queue -holds only a few messages, it may become empty before the DA worker is -re-initiated again. So it is possible that the DA run mode termination criteria -occurs while no DA worker is running on the primary queue.</p> -<p>In cases 1 and 3, the DA worker can never become inactive without hitting the -DA shutdown criteria. In <b>case 1</b>, it either shuffles messages from the -primary to the DA queue or it waits because it has the hit low water mark. </p> -<p>In <b>case 3</b>, it always shuffles messages between the queues (because, -that's the sole purpose of that run). In order for this to happen, the high -water mark has been set to the value of 1 when DA run mode has been initialized. -This ensures that the regular logic can be applied to drain the primary queue. -To prevent a hold due to reaching the low water mark, that mark must be changed -to 0 before the DA worker starts.</p> -<h2>DA Run Mode Shutdown</h2> -<p>In essence, DA run mode is terminated when the DA queue is empty and the -primary worker queue size is below the high water mark. It is also terminated -when the primary queue is shut down. The decision to switch back to regular -(non-DA) run mode is typically made by the DA worker. If it switches, the DA -queue is destructed and the regular worker pool is restarted. In some cases, the -queue shutdown process may initiate the "switch" (in this case more or less a -clean shutdown of the DA queue).</p> -<p>One might think that it would be more natural for the DA queue to detect -being idle and shut down itself. However, there are some issues associated with -that. Most importantly, all queue worker threads need to be shut down during -queue destruction. Only after that has happend, final destruction steps can -happen (else we would have a myriad of races). However, it is the DA queues -worker thread that detects it is empty (empty queue detection always happens at -the consumer side and must so). That would lead to the DA queue worker thread to -initiate DA queue destruction which in turn would lead to that very same thread -being canceled (because workers must shut down before the queue can be -destructed). Obviously, this does not work out (and I didn't even mention the -other issues - so let's forget about it). As such, the thread that enqueues -messages must destruct the queue - and that is the primary queue's DA worker -thread.</p> -<p>There are some subleties due to thread synchronization and the fact that the -DA consumer may not be running (in a <b>case-2 startup</b>). So it is not -trivial to reliably change the queue back from DA run mode to regular run mode. -The priority is a clean switch. We accept the fact that there may be situations -where we cleanly shut down DA run mode, just to re-enable it with the very next -message being enqueued. While unlikely, this will happen from time to time and -is considered perfectly legal. We can't predict the future and it would -introduce too great complexity to try to do something against that (that would -most probably even lead to worse performance under regular conditions).</p> -<p>The primary queue's DA worker thread may wait at two different places:</p> -<ol> - <li>after reaching the low water mark and waiting for either high water or - DA queue empty</li> - <li>at the regular pthread_cond_wait() on an empty primary queue</li> -</ol> -<p>Case 2 is unlikely, but may happen (see info above on a case 2 startup).</p> -<p><b>The DA worker may also not wait at all,</b> because it is actively -executing and shuffeling messages between the queues. In that case, however, the -program flow passes both of the two wait conditions but simply does not wait.</p> -<p><b>Finally, the DA worker may be inactive </b>(again, with a case-2 startup). -In that case no work(er) at all is executed. Most importantly, without the DA -worker being active, nobody will ever detect the need to change back to regular -mode. If we have this situation, the very next message enqueued will cause the -switch, because then the DA run mode shutdown criteria is met. However, it may -take close to eternal for this message to arrive. During that time, disk and -memory resources for the DA queue remain allocated. This also leaves processing -in a sub-optimal state and it may take longer than necessary to switch back to -regular queue mode when a message burst happens. In extreme cases, this could -even lead to shutdown of DA run mode, which takes so long that the high water -mark is passed and DA run mode is immediately re-initialized - while with an -immediate switch, the message burst may have been able to be processed by the -in-memory queue without DA support.</p> -<p>So in short, it is desirable switch to regular run mode as soon as possible. -To do this, we need an active DA worker. The easy solution is to initiate DA -worker startup from the DA queue's worker once it detects empty condition. To do -so, the DA queue's worker must call into a "<i>DA worker startup initiation</i>" -routine inside the main queue. As a reminder, the DA worker will most probably -not receive the "DA queue empty" signal in that case, because it will be long -sent (in most cases) before the DA worker even waits for it. So <b>it is vital -that DA run mode termination checks be done in the DA worker before it goes into -any wait condition</b>.</p> -<p>Please note that the "<i>DA worker startup initiation</i>" routine may be -called concurrently from multiple initiators. <b>To prevent a race, it must be -guarded by the queue mutex </b>and return without any action (and no error -code!) if the DA worker is already initiated.</p> -<p>All other cases can be handled by checking the termination criteria -immediately at the start of the worker and then once again for each run. The -logic follows this simplified flow diagram:</p> -<p align="center"><a href="queueWorkerLogic.jpg"> -<img border="0" src="queueWorkerLogic_small.jpg" width="431" height="605"></a></p> -<p>Some of the more subtle aspects of worker processing (e.g. enqueue thread -signaling and other fine things) have been left out in order to get the big -picture. What is called "check DA mode switchback..." right after "worker init" -is actually a check for the worker's termination criteria. Typically, <b>the -worker termination criteria is a shutdown request</b>. However, <b>for a DA -worker, termination is also requested if the queue size is below the high water -mark AND the DA queue is empty</b>. There is also a third termination criteria -and it is not even on the chart: that is the inactivity timeout, which exists in -all modes. Note that while the inactivity timeout shuts down a thread, it -logically does not terminate the worker pool (or DA worker): workers are -restarted on an as-needed basis. However, inactivity timeouts are very important -because they require us to restart workers in some situations where we may -expect a running one. So always keep them on your mind.</p> -<h2>Queue Destruction</h2> -<p>Now let's consider <b>the case of destruction of the primary queue. </b>During -destruction, our focus is on loosing as few messages as possible. If the -queue is not DA-enabled, there is nothing but the configured timeouts to handle -that situation. However, with a DA-enabled queue there are more options.</p> -<p>If the queue is DA-enabled, it may be <i>configured to persist messages to -disk before it is terminated</i>. In that case, loss of messages never occurs -(at the price of a potentially lengthy shutdown). Even if that setting is not -applied, the queue should drain as many messages as possible to the disk. For -that reason, it makes no sense to wait on a low water mark. Also, if the queue -is already in DA run mode, it does not make any sense to switch back to regular -run mode during termination and then try to process some messages via the -regular consumer. It is much more appropriate the try completely drain the queue -during the remaining timeout period. For the same reason, it is preferred that -no new consumers be activated (via the DA queue's worker), as they only cost -valuable CPU cycles and, more importantly, would potentially be long(er)-running -and possibly be needed to be cancelled. To prevent all of that, <b>queue -parameters are changed for DA-enabled queues:</b> the high water mark is to 1 -and the low water mark to 0 on the primary queue. The DA queue is commanded to -run in enqueue-only mode. If the primary queue is <i>configured to persist -messages to disk before it is terminated</i>, its SHUTDOWN timeout is changed to -to eternal. These parameters will cause the queue to drain as much as possible -to disk (and they may cause a case 3 DA run mode initiation). Please note that -once the primary queue has been drained, the DA queue's worker will -automatically switch back to regular (non-DA) run mode. <b>It must be ensured -that no worker cancellation occurs during that switchback</b>. Please note that -the queue may not switch back to regular run mode if it is not <i>configured to -persist messages to disk before it is terminated</i>. In order to apply the new -parameters, <b>worker threads must be awakened.</b> Remember we may not be in DA -run mode at this stage. In that case, the regular workers must be awakened, which -then will switch to DA run mode. No worker may be active, in that case one must -be initiated. If in DA run mode and the DA worker is inactive, the "<i>DA -worker startup initiation</i>" must be called to activate it. That routine -ensures only one DA worker is started even with multiple concurrent callers - -this may be the case here. The DA queue's worker may have requested DA worker -startup in order to terminate on empty queue (which will probably not be honored -as we have changed the low water mark).</p> -<p>After all this is done, the queue destructor requests termination of the -queue's worker threads. It will use the normal timeouts and potentially cancel -too-long running worker threads. <b>The shutdown process must ensure that all -workers reach running state before they are commanded to terminate</b>. -Otherwise it may run into a race condition that could lead to a false shutdown -with workers running asynchronously. As a few workers may have just been started -to initialize (to apply new parameter settings), the probability for this race -condition is extremely high, especially on single-CPU systems.</p> -<p>After all workers have been shut down (or cancelled), the queue may still be -in DA run mode. If so, this must be terminated, which now can simply be done by -destructing the DA queue object. This is not a real switchback to regular run -mode, but that doesn't matter because the queue object will soon be gone away.</p> -<p>Finally, the queue is mostly shut down and ready to be actually destructed. -As a last try, the queuePersists() entry point is called. It is used to persists -a non-DA-enabled queue in whatever way is possible for that queue. There may be -no implementation for the specific queue type. Please note that this is not just -a theoretical construct. This is an extremely important code path when the DA -queue itself is destructed. Remember that it is a queue object in its own right. -The DA queue is obviously not DA-enabled, so it calls into queuePersists() -during its destruction - this is what enables us to persist the disk queue!</p> -<p>After that point, left over queue resources (mutexes, dynamic memory, ...) -are freed and the queue object is actually destructed.</p> -<h2>Copyright</h2> -<p>Copyright (c) 2008 <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> -and <a href="http://www.adiscon.com/en/">Adiscon</a>.</p> -<p>Permission is granted to copy, distribute and/or modify this document under -the terms of the GNU Free Documentation License, Version 1.2 or any later -version published by the Free Software Foundation; with no Invariant Sections, -no Front-Cover Texts, and no Back-Cover Texts. A copy of the license can be -viewed at <a href="http://www.gnu.org/copyleft/fdl.html"> -http://www.gnu.org/copyleft/fdl.html</a>.</p> -</body> -</html>
\ No newline at end of file diff --git a/doc/direct_queue0.png b/doc/direct_queue0.png Binary files differdeleted file mode 100644 index 6d1b373..0000000 --- a/doc/direct_queue0.png +++ /dev/null diff --git a/doc/direct_queue1.png b/doc/direct_queue1.png Binary files differdeleted file mode 100644 index 503f815..0000000 --- a/doc/direct_queue1.png +++ /dev/null diff --git a/doc/direct_queue2.png b/doc/direct_queue2.png Binary files differdeleted file mode 100644 index cbb99af..0000000 --- a/doc/direct_queue2.png +++ /dev/null diff --git a/doc/direct_queue3.png b/doc/direct_queue3.png Binary files differdeleted file mode 100644 index cc49299..0000000 --- a/doc/direct_queue3.png +++ /dev/null diff --git a/doc/direct_queue_directq.png b/doc/direct_queue_directq.png Binary files differdeleted file mode 100644 index c5d8769..0000000 --- a/doc/direct_queue_directq.png +++ /dev/null diff --git a/doc/direct_queue_rsyslog.png b/doc/direct_queue_rsyslog.png Binary files differdeleted file mode 100644 index 6150222..0000000 --- a/doc/direct_queue_rsyslog.png +++ /dev/null diff --git a/doc/direct_queue_rsyslog2.png b/doc/direct_queue_rsyslog2.png Binary files differdeleted file mode 100644 index 807b064..0000000 --- a/doc/direct_queue_rsyslog2.png +++ /dev/null diff --git a/doc/droppriv.html b/doc/droppriv.html deleted file mode 100644 index 7293e87..0000000 --- a/doc/droppriv.html +++ /dev/null @@ -1,60 +0,0 @@ -<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> -<html><head><title>dropping privileges in rsyslog</title> -</head> -<body> -<h1>Dropping privileges in rsyslog</h1> -<p><b>Available since: </b> 4.1.1</p> -<p><b>Description</b>:</p> -<p> -Rsyslogd provides the ability to drop privileges by -impersonating as another user and/or group after startup. - -<p>Please note that due to POSIX standards, rsyslogd always needs to start -up as root if there is a listener who must bind to a network port below 1024. -For example, the UDP listener usually needs to listen to 514 and as such -rsyslogd needs to start up as root. - -<p>If you do not need this functionality, you can start rsyslog directly as an ordinary -user. That is probably the safest way of operations. However, if a startup as -root is required, you can use the $PrivDropToGroup and $PrivDropToUser config -directives to specify a group and/or user that rsyslogd should drop to after initialization. -Once this happend, the daemon runs without high privileges (depending, of -course, on the permissions of the user account you specified). -<p>There is some additional information available in the -<a href="http://wiki.rsyslog.com/index.php/Security#Dropping_Privileges">rsyslog wiki</a>. -<p><b>Configuration Directives</b>:</p> -<ul> -<li><b>$PrivDropToUser</b><br> -Name of the user rsyslog should run under after startup. Please note that -this user is looked up in the system tables. If the lookup fails, privileges are -NOT dropped. Thus it is advisable to use the less convenient $PrivDropToUserID directive. -If the user id can be looked up, but can not be set, rsyslog aborts. -<br> -</li> -<li><b>$PrivDropToUserID</b><br> -Much the same as $PrivDropToUser, except that a numerical user id instead of a name -is specified.Thus, privilege drop will always happen. -rsyslogd aborts. -<li><b>$PrivDropToGroup</b><br> -Name of the group rsyslog should run under after startup. Please note that -this user is looked up in the system tables. If the lookup fails, privileges are -NOT dropped. Thus it is advisable to use the less convenient $PrivDropToGroupID directive. -Note that all supplementary groups are removed from the process if $PrivDropToGroup is -specified. -If the group id can be looked up, but can not be set, rsyslog aborts. -<br> -</li> -<li><b>$PrivDropToGroupID</b><br> -Much the same as $PrivDropToGroup, except that a numerical group id instead of a name -is specified. Thus, privilege drop will always happen. -</ul> -<p>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>] -[<a href="manual.html">manual index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> -<p><font size="2">This documentation is part of the <a href="http://www.rsyslog.com/">rsyslog</a> -project.<br> -Copyright © 2008 by <a href="http://www.gerhards.net/rainer">Rainer -Gerhards</a> and -<a href="http://www.adiscon.com/">Adiscon</a>. -Released under the GNU GPL version 3 or higher.</font></p> - -</body></html> diff --git a/doc/expression.html b/doc/expression.html deleted file mode 100644 index c401d9a..0000000 --- a/doc/expression.html +++ /dev/null @@ -1,22 +0,0 @@ -<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> -<html><head> -<meta http-equiv="Content-Language" content="en"> -<title>Expressions in rsyslog</title></head> -<body> -<a href="rsyslog_conf_filter.html">back to rsyslog filter conditions</a> -<h1>Expressions in rsyslog</h1> -<p>Rsyslog supports expressions at a growing number of places. So -far, they are supported for filtering messages.</p> -<p>Expression support is provided by RainerScript. Please see the -<a href="rainerscript.html">RainerScript documentation</a> for more details.</p> -<p>C-like comments (/* some comment */) are supported <b>inside</b> the expression, -but not yet in the rest of the configuration file.</p> - -<p>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>] -[<a href="manual.html">manual index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> -<p><font size="2">This documentation is part of the -<a href="http://www.rsyslog.com/">rsyslog</a> project.<br> -Copyright © 2008, 2009 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and -<a href="http://www.adiscon.com/">Adiscon</a>. -Released under the GNU GPL version 3 or higher.</font></p> -</body></html> diff --git a/doc/features.html b/doc/features.html deleted file mode 100644 index 626ff65..0000000 --- a/doc/features.html +++ /dev/null @@ -1,158 +0,0 @@ -<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> -<html><head><title>rsyslog features</title> -</head> -<body> -<a href="rsyslog_conf.html">back</a> -<h1>RSyslog - Features</h1> -<p><b>This page lists both current features as well as -those being considered for future versions of rsyslog.</b> If you -think a feature is missing, drop -<a href="mailto:rgerhards@adiscon.com">Rainer</a> a -note. Rsyslog is a vital project. Features are added each few days. If -you would like to keep up of what is going on, you can also subscribe -to the <a href="http://lists.adiscon.net/mailman/listinfo/rsyslog">rsyslog -mailing list</a>.</p> -<p><span style="font-weight: bold;">A better -structured feature list is now contained in our </span><a style="font-weight: bold;" href="rsyslog_ng_comparison.html">rsyslog -vs. syslog-ng comparison</a><span style="font-weight: bold;">. -</span>Probably that page will replace this one in the -future. -</p> -<h2>Current Features</h2> -<ul> -<li>native support for <a href="rsyslog_mysql.html">writing -to MySQL databases</a></li> -<li> native support for writing to Postgres databases</li> -<li>direct support for Firebird/Interbase, -OpenTDS (MS SQL, Sybase), SQLLite, Ingres, Oracle, and mSQL via libdbi, -a database abstraction layer (almost as good as native)</li> -<li>native support for <a href="ommail.html">sending -mail messages</a> (first seen in 3.17.0)</li> -<li>support for (plain) tcp based syslog - much better -reliability</li> -<li>support for sending and receiving compressed syslog messages</li> -<li>support for on-demand on-disk spooling of messages that can -not be processed fast enough (a great feature for <a href="rsyslog_high_database_rate.html">writing massive -amounts of syslog messages to a database</a>)</li> -<li>support for selectively <a href="http://wiki.rsyslog.com/index.php/OffPeakHours">processing -messages only during specific timeframes</a> and spooling them to -disk otherwise</li> -<li>ability to monitor text files and convert their contents -into syslog messages (one per line)</li> -<li>ability to configure backup syslog/database servers - if -the primary fails, control is switched to a prioritized list of backups</li> -<li>support for receiving messages via reliable <a href="http://www.monitorware.com/Common/en/glossary/rfc3195.php"> -RFC 3195</a> delivery (a bit clumpsy to build right now...)</li> -<li>ability to generate file names and directories (log -targets) dynamically, based on many different properties</li> -<li>control of log output format, including ability to present -channel and priority as visible log data</li> -<li>good timestamp format control; at a minimum, ISO 8601/RFC -3339 second-resolution UTC zone</li> -<li>ability to reformat message contents and work with -substrings</li> -<li>support for log files larger than 2gb</li> -<li>support for file size limitation and automatic rollover -command execution</li> -<li>support for running multiple rsyslogd instances on a single -machine</li> -<li>support for <a href="rsyslog_tls.html">TLS-protected -syslog</a> (both <a href="rsyslog_tls.html">natively</a> -and via <a href="rsyslog_stunnel.html">stunnel</a>)</li> -<li>ability to filter on any part of the message, not just -facility and severity</li> -<li>ability to use regular expressions in filters</li> -<li>support for discarding messages based on filters</li> -<li>ability to execute shell scripts on received messages</li> -<li>control of whether the local hostname or the hostname of -the origin of the data is shown as the hostname in the output</li> -<li>ability to preserve the original hostname in NAT -environments and relay chains </li> -<li>ability to limit the allowed network senders</li> -<li>powerful BSD-style hostname and program name blocks for -easy multi-host support</li> -<li> massively multi-threaded with dynamic work thread pools -that start up and shut themselves down on an as-needed basis (great for -high log volume on multicore machines)</li> -<li>very experimental and volatile support for <a href="syslog_protocol.html">syslog-protocol</a> -compliant messages (it is volatile because standardization is currently -underway and this is a proof-of-concept implementation to aid this -effort)</li> -<li> world's first implementation of syslog-transport-tls</li> -<li> the sysklogd's klogd functionality is implemented as the <i>imklog</i> -input plug-in. So rsyslog is a full replacement for the sysklogd package</li> -<li> support for IPv6</li> -<li> ability to control repeated line reduction ("last message -repeated n times") on a per selector-line basis</li> -<li> supports sub-configuration files, which can be -automatically read from directories. Includes are specified in the main -configuration file</li> -<li> supports multiple actions per selector/filter condition</li> -<li> MySQL and Postgres SQL functionality as a dynamically -loadable plug-in</li> -<li> modular design for inputs and outputs - easily extensible -via custom plugins</li> -<li> an easy-to-write to plugin interface</li> -<li> ability to send SNMP trap messages</li> -<li> ability to filter out messages based on sequence of arrival</li> -<li>support for comma-seperated-values (CSV) output generation -(via the "csv" property replace option). The -CSV format supported is that from RFC 4180.</li> -<li>support for arbitrary complex boolean, string and -arithmetic expressions in message filters</li> -</ul> -<h2>World's first</h2> -Rsyslog has an interesting number of "world's firsts" - things that -were implemented for the first time ever in rsyslog. Some of them are still features not available elsewhere.<br><ul> -<li>world's first implementation of IETF I-D syslog-protocol (February 2006, version 1.12.2 and above), now RFC5424</li><li>world's first implementation of dynamic syslog on-the-wire compression (December 2006, version 1.13.0 and above)</li><li>world's first open-source implementation of a disk-queueing syslogd (January 2008, version 3.11.0 and above)</li> -<li>world's first implementation of IETF I-D -syslog-transport-tls (May 2008, version 3.19.0 and above)</li> -</ul> -<h2>Upcoming Features</h2> -<p>The list below is something like a repository of ideas we'd -like to implement. Features on this list are typically NOT scheduled -for immediate inclusion. We maintain a -<a href="http://bugzilla.adiscon.com/rsyslog-feature.html">feature -request tracker at our bugzilla</a>. This tracker has things -typically within reach of implementation. Users are encouraged to -submit feature requests there (or via our forums). If we like them but -they look quite long-lived (aka "not soon to be implemented"), they -will possibly be migrated to this list here and at some time moved back -to the bugzilla tracker.</p> -<p><b>Note that we also maintain a -<a href="http://www.rsyslog.com/sponsor_feature">list of features that are looking for sponsors</a>. -If you are interested in any of these features, or any other feature, you may consider sponsoring -the implementation. This is also a great way to show your commitment to the open source -community. Plus, it can be financially attractive: just think about how much less it may -be to sponsor a feature instead of purchasing a commercial implementation. Also, the benefit -of being recognised as a sponsor may even drive new customers to your business!</b> -<ul> -<li>port it to more *nix variants (eg AIX and HP UX) - this -needs volunteers with access to those machines and knowledge </li> -<li>pcre filtering - maybe (depending on feedback) - -simple regex already partly added. So far, this seems sufficient so -that there is no urgent need to do pcre. If done, it will be a loadable RainerScript function.</li> -<li>support for <a href="http://www.monitorware.com/Common/en/glossary/rfc3195.php">RFC -3195</a> as a sender - this is currently unlikely to happen, -because there is no real demand for it. Any work on RFC 3195 has been -suspend until we see some real interest in it. It is probably -much better to use TCP-based syslog, which is interoperable with a -large number of applications. You may also read my blog post on the -future of liblogging, which contains interesting information about the <a href="http://rgerhards.blogspot.com/2007/09/where-is-liblogging-heading-to.html"> -future of RFC 3195 in rsyslog</a>.</li> -</ul> -<p>To see when each feature was added, see the -<a href="http://www.rsyslog.com/Topic4.phtml">rsyslog -change log</a> (online only).</p> - -<p>[<a href="manual.html">manual index</a>] -[<a href="rsyslog_conf.html">rsyslog.conf</a>] -[<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> -<p><font size="2">This documentation is part of the -<a href="http://www.rsyslog.com/">rsyslog</a> project.<br> -Copyright © 2008 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and -<a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL -version 2 or higher.</font></p> - -</body></html> - diff --git a/doc/free_support.html b/doc/free_support.html deleted file mode 100644 index 182a82c..0000000 --- a/doc/free_support.html +++ /dev/null @@ -1,56 +0,0 @@ -<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> -<html><head> -<meta http-equiv="Content-Language" content="en"> -<title>Free Support for Rsyslog</title> - -</head> -<body> -<h1>Free Services for Rsyslog</h1> -<p><i>A personal word from Rainer, the lead developer of rsyslog:</i> -<p><b>The rsyslog community provides ample free support resources. Please see our -<a href="troubleshoot.html">troubleshooting guide</a> to get started.</b></p> -<p>Every now and then I receive private mail with support questions. I appreciate -any feedback, but I must limit my resources so that I can help driver a great logging -system forward. -<p>To do so, I have decided not to reply to unsolicited support emails, at least not -with a solution (but rather a link to this page ;)). I hope this does not offend you. The -reason is quite simple: If I do personal support, you gain some advantage without -contributing something back. Think about it: if you ask your question on the public -forum or mailing list, other with the same problem can you and, most importantly, even -years later find your post (and the answer) and get the problem solved. So by -solving your issue in public, you help create a great community ressource and also -help your fellow users finding solutions quicker. In the long term, this -also contributes to improved code because the more questions users can find -solutions to themselves, the fewer I need to look at. -<p>But it comes even better: the rsyslog community is much broader than Rainer ;) - there -are helpful other members hanging around at the public places. They often answer -questions, so that I do not need to look at them (btw, once again a big "thank you", folks!). -And, more important, those folks have different background than me. So they often -either know better how to solve your problem (e.g. because it is distro-specific) -or they know how to better phrase it (after all, I like abstract terms and concepts ;)). -So you do yourself a favor if you use the public places. -<p>An excellent place to go to is the -<a href="http://kb.monitorware.com/rsyslog-f40.html">rsyslog forum</a> inside the -knowledge base (which in itself is a great place to visit!). For those used to -mailing lists, the -<a href="http://lists.adiscon.net/mailman/listinfo/rsyslog">rsyslog mailing list</a> -also offers excellent advise. -<p><b>Don't like to post your question in a public place?</b> Well, then you should -consider purchasing <a href="professional_support.html">rsyslog professional support</a>. -The fees are very low and help fund the project. If you use rsyslog seriously inside -a corporate environment, there is no excuse for not getting one of the support -packages ;) -<p>Of course, things are different when I ask you to mail me privately. I'll usually do -that when I think it makes sense, for example when we exchange debug logs. -<p>I hope you now understand the free support options and the reasoning for them. -I hope I haven't offended you with my words - this is not my intension. I just needed to -make clear why there are some limits on my responsiveness. Happy logging! -<p>[<a href="manual.html">manual index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> -<p><font size="2">This documentation is part of the -<a href="http://www.rsyslog.com/">rsyslog</a> -project.<br> -Copyright © 2008 by <a href="http://www.gerhards.net/rainer">Rainer -Gerhards</a> and -<a href="http://www.adiscon.com/">Adiscon</a>. -Released under the GNU GPL version 3 or higher.</font></p> -</body></html> diff --git a/doc/generic_design.html b/doc/generic_design.html deleted file mode 100644 index 74dbd80..0000000 --- a/doc/generic_design.html +++ /dev/null @@ -1,149 +0,0 @@ -<html>
-<head>
-<title>syslogd generic design</title>
-</head>
-<body>
-<h2>Generic design of a syslogd</h2>
-<p>Written 2007-04-10 by
-<a href="http://www.adiscon.com/en/people/rainer-gerhards.php">Rainer Gerhards</a></p>
-<p>The text below describes a generic approach on how a syslogd can be
-implemented. I created this description for some other project, where it was not
-used. Instead of throwing it away, I thought it would be a good addition to the
-rsyslog documentation. While rsyslog differs in details from the description
-below, it is sufficiently close to it. Further development of rsyslog will
-probably match it even closer to the description.</p>
-<p>If you intend to read the rsyslog source code, I recommend reading this
-document here first. You will not find the same names and not all of the
-concepts inside rsyslog. However, I think your understanding will benefit from
-knowing the generic architecture.</p>
-<font size="3"><pre>
-
- +-----------------+
- | "remote" PLOrig |
- +-----------------+
- |
- I +--------+-----+-----+ +-----+-------+------+-----+
- P | PLOrig | GWI | ... | | GWO | Store | Disc | ... |
- C +--------+-----+-----+ +-----+-------+------+-----+
- | | ^
- v v |
- +--------------+ +------------+ +--------------+
- | PLGenerator | | RelayEng | | CollectorEng |
- +--------------+ +------------+ +--------------+
- | ^ ^
- | | |
- v v |
- +-------------+ +------------+ +--------------+
- | PLG Ext | | RelEng Ext | | CollcEng Ext |
- +-------------+ +------------+ +--------------+
- | ^ ^
- | | |
- v v |
- +--------------------------------------------------------------+
- | Message Router |
- +--------------------------------------------------------------+
- | ^
- v |
- +--------------------------------------------------------------+
- | Message CoDec (e.g. RFC 3164, RFCYYYY) |
- +--------------------------------------------------------------+
- | ^
- v |
- +---------------------+-----------------------+----------------+
- | transport UDP | transport TLS | ... |
- +---------------------+-----------------------+----------------+
-
- Generic Syslog Application Architecture
-</pre></font>
-<ul>
- <li>A "syslog application" is an application whose purpose is the
-processing of syslog messages. It may be part of a larger
-application with a broader purpose. An example: a database
-application might come with its own syslog send subsystem and not
-go through a central syslog application. In the sense of this
-document, that application is called a "syslog application" even
-though a casual observer might correctly call it a database
-application and may not even know that it supports sending of
-syslog messages.</li>
- <li>Payload is the information that is to be conveyed. Payload by
-itself may have any format and is totally independent from to
-format specified in this document. The "Message CoDec" of the
-syslog application will bring it into the required format.</li>
- <li>Payload Originators ("PLOrig") are the original creators of payload.
-Typically, these are application programs.</li>
- <li>A "Remote PLOrig" is a payload originator residing in a different
-application than the syslog application itself. That application
-may reside on a different machine and may talk to the syslog
-application via RPC.</li>
- <li>A "PLOrig" is a payload originator residing within the syslog
-application itself. Typically, this PLOrig emits syslog
-application startup, shutdown, error and status log messages.</li>
- <li>A "GWI" is a inbound gateway. For example, a SNMP-to-syslog
-gateway may receive SNMP messages and translate them into syslog.</li>
- <li>The ellipsis after "GWI" indicates that there are potentially a
-variety of different other ways to originally generate payload.</li>
- <li>A "PLGenerator" is a payload generator. It takes the information
-from the payload-generating source and integrates it into the
-syslog subsystem of the application. This is a highly theoretical
-concept. In practice, there may not actually be any such
-component. Instead, the payload generators (or other parts like
-the GWI) may talk directly to the syslog subsystem. Conceptually,
-the "PLGenerator" is the first component where the information is
-actually syslog content.</li>
- <li>A "PLG Ext" is a payload generator extension. It is used to
-modify the syslog information. An example of a "PLG Ext" might be
-the addition of cryptographic signatures to the syslog
-information.</li>
- <li>A "Message Router" is a component that accepts in- and outbound
-syslog information and routes it to the proper next destination
-inside the syslog application. The routing information itself is
-expected to be learnt by operator configuration.</li>
- <li>A "Message CoDec" is the message encoder/decoder. The encoder
-takes syslog information and encodes them into the required format<br>for a syslog message. The decoder takes a syslog message and
-decodes it into syslog information. Codecs for multiple syslog
-formats may be present inside a single syslog application.</li>
- <li>A transport (UDP, TLS, yet-to-be-defined ones) sends and receives
-syslog messages. Multiple transports may be used by a single<br>syslog application at the same time. A single transport instance
-may be used for both sending and receiving. Alternatively, a
-single instance might be used for sending and receiving
-exclusively. Multiple instances may be used for different
-listener ports and receivers.</li>
- <li>A "RelayEng" is the relaying engine. It provides functionality
-necessary for receiving syslog information and sending it to
-another syslog application.</li>
- <li>A "RelEng Ext" is an extension that processes syslog information
-as it enters or exits a RelayEng. An example of such a component
-might be a relay cryptographically signing received syslog
-messages. Such a function might be useful to guarantee authenticity
-starting from a given point inside a relay chain.</li>
- <li>A "CollectorEng" is a collector engine. At this component, syslog
-information leaves the syslog system and is translated into some
-other form. After the CollectorEng, the information is no longer
-defined to be of native syslog type.</li>
- <li>A "CollcEng Ext" is a collector engine extension. It modifies
-syslog information before it is passed on to the CollectorEng. An
-example for this might be the verification of cryptographically
-signed syslog message information. Please note that another
-implementation approach would be to do the verification outside of
-the syslog application or in a stage after "CollectorEng".</li>
- <li>A "GWO" is an outbound gateway. An example of this might be the
-forwarding of syslog information via SNMP or SMTP. Please note
-that when a GWO directly connects to a GWI on a different syslog
-application, no native exchange of syslog information takes place.
-Instead, the native protocol of these gateways (e.g. SNMP) is
-used. The syslog information is embedded inside that protocol.
-Depending on protocol and gateway implementation, some of the
-native syslog information might be lost.</li>
- <li>A "Store" is any way to persistently store the extracted syslog
-information, e.g. to the file system or to a data base.</li>
- <li>"Disc" means the discarding of messages. Operators often find it
-useful to discard noise messages and so most syslog applications<br>contain a way to do that.</li>
- <li>The ellipsis after "Disc" indicates that there are potentially a variety of different other ways to consume syslog information.</li>
- <li>There may be multiple instances of each of the described
-components in a single syslog application.</li>
- <li>A syslog application is made up of all or some of the above
-mentioned components.</li>
-</ul>
-</p>
-</body>
-</html>
diff --git a/doc/gssapi.html b/doc/gssapi.html deleted file mode 100644 index 3ad7d07..0000000 --- a/doc/gssapi.html +++ /dev/null @@ -1,118 +0,0 @@ -<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> -<html><head><title>GSSAPI module support in rsyslog v3</title> - -</head> -<body> -<h1>GSSAPI module support in rsyslog v3</h1> -<p style="font-weight: bold;">What is it good for.</p> -<ul style="margin-left: 1.25cm;"> -<li> -client-serverauthentication </li> -<li> -Log -messages encryption </li> -</ul> -<p class="P5"> </p> -<p class="P3"><span style="font-weight: bold;">Requirements.</span> -</p> -<ul> -<li>Kerberos infrastructure</li> -<li>rsyslog, rsyslog-gssapi</li> -</ul> -<p> </p> -<p><span style="font-weight: bold;">Configuration.</span> -</p> -<p>Let's assume there are 3 machines in kerberos Realm: </p> -<ul> -<li>the -first is running KDC (Kerberos Authentication Service and Key -Distribution Center),</li> -<li>the second is a client sending its logs to the server,</li> -<li>the third is receiver, gathering all logs.</li> -</ul> -<p class="P7"> </p> -<p class="P10"><span style="font-style: italic;">1. -KDC:</span> </p> -<ul> -<li>Kerberos -database must be properly set-up on KDC machine first. Use -kadmin/kadmin.local to do that. Two principals need to be add in our -case:</li> -</ul> -<ol style="margin-left: 1.25cm; list-style-type: decimal;"> -<li> -<p>sender@REALM.ORG -</p> -</li> -</ol> -<ul> -<li>client must have ticket for pricipal sender</li> -<li>REALM.ORG is kerberos Realm</li> -</ul> -<ol style="margin-left: 1.25cm; list-style-type: decimal;"> -<li>host/receiver.mydomain.com@REALM.ORG - service principal</li> -</ol> -<ul> -<li>Use ktadd to export service principal and transfer it to -/etc/krb5.keytab -on receiver </li> -</ul> -<p><span style="font-style: italic;">2. CLIENT:</span> -</p> -<ul> -<li>set-up rsyslog, in /etc/rsyslog.conf</li> -<li>$ModLoad omgssapi - load output gss module </li> -<li>$GSSForwardServiceName -otherThanHost - set the name of service principal, "host" is the -default one</li> -<li>*.* :omgssapi:receiver.mydomain.com - action line, forward -logs to receiver</li> -<li>kinit root - get the TGT ticket</li> -<li>service rsyslog start -<p class="P14" style="margin-left: 0.25cm;"> </p> -</li> -</ul> -<p><span style="font-style: italic;">3. SERVER:</span> -</p> -<ul> -<li class="P14" style="margin-left: 0cm;"> -<p class="P14" style="margin-left: 0.25cm;">set-up -rsyslog, in /etc/rsyslog.conf </p> -</li> -<li class="P16"> -<p class="P16" style="margin-left: 0.25cm;">$ModLoad -<a href="imgssapi.html">imgssapi</a> - load input gss module </p> -</li> -<li class="P16"> -<p class="P16" style="margin-left: 0.25cm;">$InputGSSServerServiceName -otherThanHost - set the name of service principal, "host" is the -default one </p> -</li> -<li class="P16"> -<p class="P16" style="margin-left: 0.25cm;">$InputGSSServerPermitPlainTCP -on - accept GSS and TCP connections (not authenticated senders), off by -default </p> -</li> -<li class="P16"> -<p class="P16" style="margin-left: 0.25cm;">$InputGSSServerRun -514 - run server on port </p> -</li> -<li class="P14" style="margin-left: 0cm;"> -<p class="P14" style="margin-left: 0.25cm;">service -rsyslog start </p> -</li> -</ul> -<span style="font-weight: bold;">The picture demonstrate -how things work.</span> -<p class="P18"> </p> -<img src="gssapi.png" alt="rsyslog gssapi support"> -<p>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>] -[<a href="manual.html">manual index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> -<p><font size="2">This documentation is part of the -<a href="http://www.rsyslog.com/">rsyslog</a> -project.<br> -Copyright © 2008 by <a href="http://www.gerhards.net/rainer">Rainer -Gerhards</a> and -<a href="http://www.adiscon.com/">Adiscon</a>. -Released under the GNU GPL version 3 or higher.</font></p> -</body></html> diff --git a/doc/gssapi.png b/doc/gssapi.png Binary files differdeleted file mode 100644 index c82baa5..0000000 --- a/doc/gssapi.png +++ /dev/null diff --git a/doc/history.html b/doc/history.html deleted file mode 100644 index 57b6400..0000000 --- a/doc/history.html +++ /dev/null @@ -1,147 +0,0 @@ -<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> -<html><head> -<title>rsyslog history</title></head> -<body> -<h1>RSyslog - History</h1> - -<b>Rsyslog is a GPL-ed, enhanced syslogd. Among others, it offers support for -reliable syslog over TCP, writing to -MySQL databases and fully configurable output formats (including great timestamps).</b> -Rsyslog was initiated by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a>. -If you are interested to learn why Rainer initiated the project, you -may want to read his blog posting on "<a href="http://rgerhards.blogspot.com/2007/08/why-does-world-need-another-syslogd.html">why -the world needs another syslogd</a>".<p>Rsyslog has -been forked in <b>2004</b> from the <a href="http://www.infodrom.org/projects/sysklogd/">sysklogd standard package</a>. -The goal of the -rsyslog project is to provide a feature-richer and reliable -syslog daemon while retaining drop-in replacement capabilities to stock -syslogd. By "reliable", we mean support for reliable transmission -modes like TCP or <a href="http://www.monitorware.com/Common/en/glossary/rfc3195.php">RFC 3195</a> -(syslog-reliable). We do NOT imply that the sysklogd package is unreliable.</p> -<p>The name "rsyslog" stems back to the -planned support for syslog-reliable. Ironically, the initial release -of rsyslog did NEITHER support syslog-reliable NOR tcp based syslog. -Instead, it contained enhanced configurability and other enhancements -(like database support). The reason for this is that full support for -RFC 3195 would require even more changes and especially fundamental -architectural -changes. Also, questions asked on the loganalysis list and at other -places indicated that RFC3195 is NOT a prime priority for users, but -rather better control over the output format. So there we were, with -a rsyslogd that covers a lot of enhancements, but not a single one -of these that made its name ;) Since version 0.9.2, receiving syslog -messages via plain tcp is finally supported, a bit later sending via -TCP, too. Starting with 1.11.0, RFC 3195 is finally supported at the -receiving side (a.k.a. "listener"). Support for sending via RFC 3195 is -still due. Anyhow, rsyslog has come much closer to what it name -promises.</p> -<p> -The database support was initially included so that our web-based syslog -interface could be used. This is another open source project which can be found -under <a href="http://www.phplogcon.org">http://www.phplogcon.org</a>. We highly recommend having a look at -it. It might not work for you if you expect thousands of messages per -second (because your database won't be able to provide adequate performance), -but in many cases it is a very handy analysis and troubleshooting tool. - -In the mean time, of course, lots of people have found many applications for -writing to databases, so the prime focus is no longer on phpLogcon. - -</p> -<p>Rsyslogd supports an enhanced syslog.conf file format, and also works -with the standard syslog.conf. In theory, it should be possible to simply replace -the syslogd binary with the one that comes with rsyslog. Of course, in order -to use any of the new features, you must re-write your syslog.conf. To learn -how to do this, please review our commented <a href="sample.conf.php">sample.conf</a> -file. It outlines the enhancements over stock syslogd. Discussion has often -arisen of whether having an "old syslogd" logfile format is good or evil. So -far, this has not been solved (but Rainer likes the idea of a new format), so we -need to live with it for the time being. It is planned to be reconsidered in the -3.x release time frame. -</p><p>If you are interested in the <a href="http://en.wikipedia.org/wiki/IHE">IHE</a> -environment, you might be interested to hear that rsyslog supports message with -sizes of 32k and more. This feature has been tested, but by default is turned off -(as it has some memory footprint that we didn't want to put on users not -actually requiring it). Search the file syslogd.c and search for "IHE" - you -will find easy and precise instructions on what you need to change (it's just -one line of code!). Please note that RFC 3195/COOKED supports 1K message sizes -only. It'll probably support longer messages in the future, but it is our -believe that using larger messages with current RFC 3195 is a violation of the -standard.</p><p>In <b>February 2007</b>, 1.13.1 was released and served for quite a -while as a stable reference. Unfortunately, it was not later released as stable, -so the stable build became quite outdated.</p><p>In <b>June 2007</b>, Peter Vrabec from Red Hat helped us to create -RPM files for Fedora as well as supporting IPv6. There also seemed to be some -interest from the Red Hat community. This interest and new ideas resulted in a -very busy time with many great additions.</p><p>In <b>July 2007</b>, Andrew -Pantyukhin added BSD ports files for rsyslog and liblogging. We were strongly -encouraged by this too. It looks like rsyslog is getting more and more momentum. -Let's see what comes next...</p><p>Also in <b>July 2007</b> (and beginning of -August), Rainer remodeled the output part of rsyslog. It got a clean object model -and is now prepared for a plug-in architecture. During that time, some base -ideas for the overall new object model appeared.</p><p>In <b>August 2007</b> -community involvement grew more and more. Also, more packages appeared. We were -quite happy about that. To facilitate user contributions, we set up a -<a href="http://wiki.rsyslog.com/">wiki</a> on August 10th, 2007. Also in August -2007, rsyslog 1.18.2 appeared, which is deemed to be quite close to the final -2.0.0 release. With its appearance, the pace of changes was deliberately reduced, -in order to allow it to mature (see Rainers's -<a href="http://rgerhards.blogspot.com/2007/07/pace-of-changes-in-rsyslog.html"> -blog post</a> on this topic, written a bit early, but covering the essence).</p><p> -In <b>November 2007</b>, rsyslog became the default syslogd in Fedora 8. -Obviously, that was something we *really* liked. Community involvement also is -still growing. There is one sad thing to note: ever since summer, there is an -extremely hard to find segfault bug. It happens on very rare occasions only and -never in lab. We are hunting this bug for month now, but still could not get -hold of it. Unfortunately, this also affects the new features schedule. It makes -limited sense to implement new features if problems with existing ones are not -really understood.</p><p><b>December 2007</b> showed the appearance of a postgres -output module, contributed by sur5r. With 1.20.0, December is also the first -time since the bug hunt that we introduce other new features. It has been decided -that we carefully will add features in order to not affect the overall project -by these rare bugs. Still, the bug hunt is top priority, but we need to have more -data to analyze. At then end of December, it looked like the bug was found (a -race condition), but further confirmation from the field is required before -declaring victory. December also brings the initial development on <b>rsyslog v3</b>, -resulting in loadable input modules, now running on a separate thread each.</p><p>On -<b>January, 2nd 2008</b>, rsyslog 1.21.2 is re-released as rsyslog v2.0.0 -stable. This is a major milestone as far as the stable build is concerned. v3 is -not yet officially announced. Other than the stable v2 build, v3 will not be -backwards compatibile (including missing compatibility to stock sysklogd) for -quite a while. Config file changes are required and some command line options do -no longer work due to the new design.</p><p>On <span style="font-weight: bold;">January, 31st 2008</span> -the new massively-multithreaded queue engine was released for the first -time. It was a major milestone, implementing a feature I dreamed of for -more than a year.</p><p>End of <span style="font-weight: bold;">February 2008</span> -saw the first note about RainerScript, a way to configure rsyslogd via -a script-language like configuration format. This effort evolved out of -the need to have complex expression support, which was also the first -use case. On February, 28th rsyslog 3.12.0 was released, the first -version to contain expression support. This also meant that rsyslog -from that date on supported all syslog-ng major features, but had a -number of major features exlusive to it. With 3.12.0, I consider -rsyslog fully superior to syslog-ng (except for platform support).</p> - -<p>Following the Fedora Developer's conference in Brno <b>2012</b>, rsyslog -got very serious on implementing <b>structured logging</b> in -project Lumberjack (CEE) style. Project Lumberjack was a much broader -effort and brought closer collaboration with the syslog-ng folks, which -helped to maintain and improve interoperability. In the -<b>late winter/spring/summer 2012</b> timeframe numerous engine enhancements -were made and plugins written (among them the first "official" interfaces -to the Linux audit subsystem). At the end of the year, this culminated in the -rsyslog 7, which not only implemented Lumberjack but also was the first one -to support full condition nesting in rsyslog.conf (and a ton of other features as -well). - -<p>In <b>spring 2013</b> major new security features were engineered, -namely anonymization support, as well as log file signing and -encryption capabilities. - -<p>Be sure to visit Rainer's <a href="http://rgerhards.blogspot.com/">syslog blog</a> -to get some more insight into the development and futures of rsyslog and syslog in general. -Don't be shy to post to either the blog or the -<a href="http://www.rsyslog.com/PNphpBB2.phtml">rsyslog forums</a>.</p> -<h2>Some useful links</h2> -<ul> - <li><a href="http://www.rsyslog.com/Topic4.phtml">the rsyslog change log</a></li> -</ul> -</body></html> diff --git a/doc/how2help.html b/doc/how2help.html deleted file mode 100644 index 7fda694..0000000 --- a/doc/how2help.html +++ /dev/null @@ -1,60 +0,0 @@ -<html> -<head> -<title>How you can Help</title> -</head> -<body> -<h2>How you can Help</h2> -<p><b>You like rsyslog and would like to lend us a helping hand?</b> This page -tells you how easy it is to help a little bit. You can contribute to the project -even with a single mouse click! If you could pick a single item from the -wish list, that would be awfully helpful!</p> -<p>This is our wish list:</p> -<ul> - <li>let others know how great rsyslog is<ul> - <li>spread word about rsyslog in forums and newsgroups</li> - <li>place a link to <a href="http://www.rsyslog.com">www.rsyslog.com</a> - from your home page</li> - <li>you may also want to tell others about the - <a href="http://loganalyzer.adiscon.com">log analyzer tool - created by the same folks as rsyslog</a> - at least, if you like it ;) - </ul> - </li> - <li>let us know about rsyslog - we are eager for feedback<ul> - <li>tell us what you like and what you not like - so that we can include - that into development</li> - <li>tell us what you use rsyslog for - especially if you have high - traffic volume or an otherwise "uncommon" deployment. We are looking for - case studies and experience how rsyslog performs in unusual scenarios.</li> - <li>allow us to post your thoughts and experiences as a "user story" on - the web site (so far, none are there ;))</li> - </ul> - </li> - <li>if you know how to create packages (rpm, deb, ...)<ul> - <li>we would very much appreciate your help with package creation. We know - that it is important to have good binary packages for a product to - spread widely. Yet, we do not have the knowledge to do it all ourselves. - <a href="mailto:rgerhards@adiscon.com">Drop Rainer a note </a>if you - could help us out.</li> - </ul> - </li> - <li>if you have configured a device for sending syslog data, and that device - is not in our - <a href="http://www.monitorware.com/en/syslog-enabled-products/">syslog - configuration database</a>, you might want to tell us how to configure it.</li> - <li>if you are a corporate user<ul> - <li>you might consider <a href="http://www.adiscon.com">Adiscon</a>'s - commercial <a href="http://www.monitorware.com/">MonitorWare products</a> - for Windows, e.g. to deliver Windows Event Log data to rsyslogd (sales - of the commercial products funds the open source development - and they - also work very well).</li> - <li>you might be interested in - <a href="http://www.adiscon.com/Common/en/Products/techsup.php"> - purchasing professional support or add-on development</a> for rsyslog</li> - </ul> - </li> -</ul> -<p><b>We appreciate your help very much.</b> A big thank you for anything you -might do!</p> - -</body> -</html> diff --git a/doc/im3195.html b/doc/im3195.html deleted file mode 100644 index aad9f3d..0000000 --- a/doc/im3195.html +++ /dev/null @@ -1,48 +0,0 @@ -<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> -<html><head> -<title>RFC3195 Input Module (im3195)</title> - -</head> -<body> -<a href="rsyslog_conf_modules.html">back</a> - -<h1>RFC3195 Input Module</h1> -<p><b>Module Name: im3195</b></p> -<p><b>Author: </b>Rainer Gerhards -<rgerhards@adiscon.com></p> -<p><b>Description</b>:</p> -<p>Receives syslog messages via RFC 3195. The RAW profile is fully implemented and the -COOKED profile is provided in an experimental state. This module uses -<a href="http://www.liblogging.org">liblogging</a> for the actual protocol handling.</p> -<p><b>Configuration Directives</b>:</p> -<ul> -<li><strong>$Input3195ListenPort <port></strong><br> -The port on which imklog listens for RFC 3195 messages. The default port is 601 -(the IANA-assigned port)</li> -</ul> -<b>Caveats/Known Bugs:</b> -<p>Due to no demand at all for RFC3195, we have converted rfc3195d -to this input module, but we have NOT conducted any testing. Also, -the module does not yet properly handle the recovery case. If someone -intends to put this module into production, good testing should be -cunducted. It also is a good idea to notify the rsyslog project that you intend to use -it in production. In this case, we'll probably give the module another -cleanup. We don't do this now because so far it looks just like a big -waste of time. -<p>Currently only a single listener can be defined. That one binds to all interfaces.</p> -<p><b>Sample:</b></p> -<p>The following sample accepts syslog messages via RFC 3195 on port 1601. -<br> -</p> -<textarea rows="15" cols="60">$ModLoad im3195 -$Input3195ListenPort 1601 -</textarea> -<p>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>] -[<a href="manual.html">manual index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> -<p><font size="2">This documentation is part of the -<a href="http://www.rsyslog.com/">rsyslog</a> project.<br> -Copyright © 2008 by <a href="http://www.gerhards.net/rainer">Rainer -Gerhards</a> and -<a href="http://www.adiscon.com/">Adiscon</a>. -Released under the GNU GPL version 3 or higher.</font></p> -</body></html> diff --git a/doc/imfile.html b/doc/imfile.html deleted file mode 100644 index f37f705..0000000 --- a/doc/imfile.html +++ /dev/null @@ -1,240 +0,0 @@ -<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> -<html><head> -<meta http-equiv="Content-Language" content="en"><title>Text File Input Monitor</title></head> -<body> -<a href="rsyslog_conf_modules.html">back</a> - -<h1>Text File Input Module</h1> -<p><b>Module Name: imfile</b></p> -<p><b>Author: </b>Rainer Gerhards -<rgerhards@adiscon.com></p> -<p><b>Description</b>:</p> -<p>Provides the ability to convert any standard text file into -a syslog message. A standard -text file is a file consisting of printable characters with lines -being delimited by LF.</p> -<p>The file is read line-by-line and any line read is passed to -rsyslog's rule engine. The rule engine applies filter conditons and -selects which actions needs to be carried out. Empty lines are <b>not</b> -processed, as they would result in empty syslog records. They are simply -ignored.</p> -<p>As new lines are written they are taken from the file and -processed. Please note that this happens based on a polling interval -and not immediately. The file monitor support file rotation. To fully -work, rsyslogd must run while the file is rotated. Then, any remaining -lines from the old file are read and processed and when done with that, -the new file is being processed from the beginning. If rsyslogd is -stopped during rotation, the new file is read, but any not-yet-reported -lines from the previous file can no longer be obtained.</p> -<p>When rsyslogd is stopped while monitoring a text file, it -records the last processed location and continues to work from there -upon restart. So no data is lost during a restart (except, as noted -above, if the file is rotated just in this very moment).</p> -<p>Currently, the file must have a fixed name and location -(directory). It is planned to add support for dynamically generating -file names in the future.</p> -<p>Multiple files may be monitored by specifying -$InputRunFileMonitor multiple times. -</p> - -<p><b>Configuration Directives</b>:</p> -<p><b>Module Directives</b></p> -<ul> -<li><span style="font-weight: bold;">PollingInterval -seconds</span><br> -This is a global setting. It specifies how often files are to be polled -for new data. The time specified is in seconds. The <span style="font-weight: bold;">default value</span> is 10 -seconds. Please note that future -releases of imfile may support per-file polling intervals, but -currently this is not the case. If multiple PollingInterval -statements are present in rsyslog.conf, only the last one is used.<br> -A short poll interval provides more rapid message forwarding, but -requires more system ressources. While it is possible, we stongly -recommend not to set the polling interval to 0 seconds. That will make -rsyslogd become a CPU hog, taking up considerable ressources. It is -supported, however, for the few very unusual situations where this -level may be needed. Even if you need quick response, 1 seconds should -be well enough. Please note that imfile keeps reading files as long as -there is any data in them. So a "polling sleep" will only happen when -nothing is left to be processed.</li> -</ul> - -<p><b>Action Directives</b></p> -<ul> -<li><strong>File /path/to/file</strong><br> -The file being monitored. So far, this must be an absolute name (no -macros or templates)</li> -<li><span style="font-weight: bold;">Tag -tag:</span><br> -The tag to be used for messages that originate from this file. If you -would like to see the colon after the tag, you need to specify it here -(as shown above).</li> -<li><span style="font-weight: bold;">StateFile -<name-of-state-file></span><br> -Rsyslog must keep track of which parts of the to be monitored file it -already processed. This is done in the state file. This file always is -created in the rsyslog working directory (configurable via -$WorkDirectory). Be careful to use unique names for different files -being monitored. If there are duplicates, all sorts of "interesting" -things may happen. Rsyslog currently does not check if a name is -specified multiple times. -Note that when $WorkDirectory is not set or set to a non-writable -location, the state file will not be generated.</li> -<li><span style="font-weight: bold;">Facility -facility</span><br> -The syslog facility to be assigned to lines read. Can be specified in -textual form (e.g. "local0", "local1", ...) or as numbers (e.g. 128 for -"local0"). Textual form is suggested. <span style="font-weight: bold;">Default</span> is -"local0".<span style="font-weight: bold;"></span></li> -<li><span style="font-weight: bold;">Severity</span><br> -The -syslog severity to be assigned to lines read. Can be specified in -textual form (e.g. "info", "warning", ...) or as numbers (e.g. 4 for -"info"). Textual form is suggested. <span style="font-weight: bold;">Default</span> -is "notice".</li> -<li><b>PersistStateInterval</b> [lines]</b><br> -Specifies how often the state file shall be written when processing the input -file. The default value is 0, which means a new state file is only written when -the monitored files is being closed (end of rsyslogd execution). Any other -value n means that the state file is written every time n file lines have -been processed. This setting can be used to guard against message duplication due -to fatal errors (like power fail). Note that this setting affects imfile -performance, especially when set to a low value. Frequently writing the state -file is very time consuming. -<li><b>ReadMode</b> [mode]</b><br> -This mode should defined when having multiline messages. The value can range from 0-2 and determines the multiline detection method. -<br>0 (default) - line based (Each line is a new message) -<br>1 - indented (New log messages start at the beginning of a line. If a line starts with a space it is part of the log message before it) -<br>2 - paragraph (There is a blank line between log messages) -<li><b>MaxLinesAtOnce</b> [number]</b> -<br> -This is useful if multiple files need to be monitored. If set to 0, each file -will be fully processed and then processing switches to the next file -(this was the default in previous versions). If it is set, a maximum of -[number] lines is processed in sequence for each file, and then the file is -switched. This provides a kind of mutiplexing the load of multiple files and -probably leads to a more natural distribution of events when multiple busy files -are monitored. The default is 1024. -<li><b>MaxSubmitAtOnce</b> [number]</b> -<br> -This is an expert option. It can be used to set the maximum input batch size that -imfile can generate. The default is 1024, which is suitable for a wide range of -applications. Be sure to understand rsyslog message batch processing before you -modify this option. If you do not know what this doc here talks about, this is a -good indication that you should NOT modify the default. -<li><b>Ruleset</b> <ruleset> -Binds the listener to a specific <a href="multi_ruleset.html">ruleset</a>.</li> -</ul> -<b>Caveats/Known Bugs:</b> -<p>So far, only 100 files can be monitored. If more are needed, -the source needs to be patched. See define MAX_INPUT_FILES in imfile.c</p><p>Powertop -users may want to notice that imfile utilizes polling. Thus, it is no -good citizen when it comes to conserving system power consumption. We -are currently evaluating to move to inotify(). However, there are a -number of subtle issues, which needs to be worked out first. We will -make the change as soon as we can. If you can afford it, we recommend -using a long polling interval in the mean time. -</p> -<p><b>Sample:</b></p> -<p>The following sample monitors two files. If you need just one, -remove the second one. If you need more, add them according to the -sample ;). This code must be placed in /etc/rsyslog.conf (or wherever -your distro puts rsyslog's config files). Note that only commands -actually needed need to be specified. The second file uses less -commands and uses defaults instead.<br> -</p> -<textarea rows="15" cols="60">module(load="imfile" PollingInterval="10") #needs to be done just once -# File 1 -input(type="imfile" File="/path/to/file1" - Tag="tag1" - StateFile="/var/spool/rsyslog/statefile1" - Severity="error" - Facility="local7") -# File 2 -input(type="imfile" File="/path/to/file2" - Tag="tag2" - StateFile="/var/spool/rsyslog/statefile2") -# ... and so on ... -# -</textarea> - - -<p><b>Legacy Configuration Directives</b>:</p> -<ul> -<li><strong>$InputFileName /path/to/file</strong><br> -equivalent to: File </li> -<li><span style="font-weight: bold;">$InputFileTag -tag:</span><br> -equivalent to: Tag </li> -<li><span style="font-weight: bold;">$InputFileStateFile -<name-of-state-file></span><br> -equivalent to: StateFile </li> -<li><span style="font-weight: bold;">$InputFileFacility -facility</span><br> -equivalent to: Facility </span></li> -<li><span style="font-weight: bold;">$InputFileSeverity</span><br> -equivalent to: Severity</li> -<li><span style="font-weight: bold;">$InputRunFileMonitor</span><br> -This <span style="font-weight: bold;">activates</span> -the current monitor. It has no parameters. If you forget this -directive, no file monitoring will take place.</li> -<li><span style="font-weight: bold;">$InputFilePollInterval -seconds</span><br> -equivalent to: PollingInterva</li> -<li><b>$InputFilePersistStateInterval</b> [lines]</b><br> -Available in 4.7.3+, 5.6.2+<br> -equivalent to: PersistStateInterval -<li><b>$InputFileReadMode</b> [mode]</b><br> -Available in 5.7.5+<br> -equivalent to: ReadMode -<li><b>$InputFileMaxLinesAtOnce</b> [number]</b><br> -Available in 5.9.0+<br> -equivalent to: MaxLinesAtOnce -<li>$InputFileBindRuleset <ruleset><br> -Available in 5.7.5+, 6.1.5+<br> -equivalent to: Ruleset </li> -</ul> -<b>Caveats/Known Bugs:</b> -<p>So far, only 100 files can be monitored. If more are needed, -the source needs to be patched. See define MAX_INPUT_FILES in imfile.c</p><p>Powertop -users may want to notice that imfile utilizes polling. Thus, it is no -good citizen when it comes to conserving system power consumption. We -are currently evaluating to move to inotify(). However, there are a -number of subtle issues, which needs to be worked out first. We will -make the change as soon as we can. If you can afford it, we recommend -using a long polling interval in the mean time. -</p> -<p><b>Sample:</b></p> -<p>The following sample monitors two files. If you need just one, -remove the second one. If you need more, add them according to the -sample ;). This code must be placed in /etc/rsyslog.conf (or wherever -your distro puts rsyslog's config files). Note that only commands -actually needed need to be specified. The second file uses less -commands and uses defaults instead.<br> -</p> -<textarea rows="15" cols="60">$ModLoad imfile # needs to be done just once -# File 1 -$InputFileName /path/to/file1 -$InputFileTag tag1: -$InputFileStateFile stat-file1 -$InputFileSeverity error -$InputFileFacility local7 -$InputRunFileMonitor -# File 2 -$InputFileName /path/to/file2 -$InputFileTag tag2: -$InputFileStateFile stat-file2 -$InputRunFileMonitor -# ... and so on ... -# -# check for new lines every 10 seconds -$InputFilePollingInterval 10 -</textarea> -<p>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>] -[<a href="manual.html">manual index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> -<p><font size="2">This documentation is part of the -<a href="http://www.rsyslog.com/">rsyslog</a> project.<br> -Copyright © 2008 by <a href="http://www.gerhards.net/rainer">Rainer -Gerhards</a> and <a href="http://www.adiscon.com/">Adiscon</a>. -Released under the GNU GPL version 3 or higher.</font></p> -</body></html> diff --git a/doc/imgssapi.html b/doc/imgssapi.html deleted file mode 100644 index dd90fec..0000000 --- a/doc/imgssapi.html +++ /dev/null @@ -1,53 +0,0 @@ -<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> -<html><head> -<meta http-equiv="Content-Language" content="en"><title>GSSAPI Syslog Input Module</title> - -</head> -<body> -<a href="rsyslog_conf_modules.html">back</a> - -<h1>GSSAPI Syslog Input Module</h1> -<p><b>Module Name: imgssapi</b></p> -<p><b>Author: </b>varmojfekoj</p> -<p><b>Description</b>:</p> -<p>Provides the ability to receive syslog messages from the -network protected via Kerberos 5 encryption and authentication. This -module also accept plain tcp syslog messages on the same port if configured to do so. If you need just plain tcp, use <a href="imtcp.html">imtcp</a> instead.</p> -<p>There is also an <a href="gssapi.html">overview of gssapi support in rsyslog</a> available. We recommend reading -it before digging into the configuration parameters.</p> -<p><b>Configuration Directives</b>:</p> -<ul> -<li>InputGSSServerRun <port><br> -Starts a GSSAPI server on selected port - note that this runs -independently from the TCP server.</li> -<li>InputGSSServerServiceName <name><br> -The service name to use for the GSS server.</li> -<li>$InputGSSServerPermitPlainTCP on|<span style="font-weight: bold;">off</span><br> -Permits the server to receive plain tcp syslog (without GSS) on the -same port</li> -<li>$InputGSSServerMaxSessions <number><br> -Sets the maximum number of sessions supported</li> -</ul> -<b>Caveats/Known Bugs:</b> -<ul> -<li>module always binds to all interfaces</li> -<li>only a single listener can be bound</li> - -</ul> -<p><b>Sample:</b></p> -<p>This sets up a GSS server on port 1514 that also permits to -receive plain tcp syslog messages (on the same port):<br> -</p> -<textarea rows="15" cols="60">$ModLoad imgssapi # needs to be done just once -$InputGSSServerRun 1514 -$InputGSSServerPermitPlainTCP on -</textarea> -<p>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>] -[<a href="manual.html">manual index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> -<p><font size="2">This documentation is part of the -<a href="http://www.rsyslog.com/">rsyslog</a> -project.<br> -Copyright © 2008-2011 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and -<a href="http://www.adiscon.com/">Adiscon</a>. -Released under the GNU GPL version 3 or higher.</font></p> -</body></html> diff --git a/doc/imklog.html b/doc/imklog.html deleted file mode 100644 index 1f195b1..0000000 --- a/doc/imklog.html +++ /dev/null @@ -1,119 +0,0 @@ -<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> -<html><head> -<meta http-equiv="Content-Language" content="en"><title>Kernel Log Input Module (imklog)</title> - -</head> -<body> -<a href="rsyslog_conf_modules.html">back</a> - -<h1>Kernel Log Input Module</h1> -<p><b>Module Name: imklog</b></p> -<p><b>Author: </b>Rainer Gerhards -<rgerhards@adiscon.com></p> -<p><b>Description</b>:</p> -<p>Reads messages from the kernel log and submits them to the -syslog engine.</p> -<p><b>Configuration Directives</b>:</p> -<ul> -<li><strong>LogPath</strong><br> -The path to the Kernel log. This value should only be changed if you really know what -you are doing.</li> -<li><strong>InternalMsgFacility -<facility></strong><br> -The facility which messages internally generated by imklog will have. -imklog generates some messages of itself (e.g. on problems, startup and -shutdown) and these do not stem from the kernel. Historically, under -Linux, these too have "kern" facility. Thus, on Linux platforms the -default is "kern" while on others it is "syslogd". You usually do not -need to specify this configuratin directive - it is included primarily -for few limited cases where it is needed for good reason. Bottom line: -if you don't have a good idea why you should use this setting, do not -touch it.</li> -<li><b>PermitNonKernelFacility [on/<i>off</i>]</b><br> -At least under BSD the kernel log may contain entries -with non-kernel facilities. This setting controls how those are -handled. The default is "off", in which case these messages are -ignored. Switch it to on to submit non-kernel messages to rsyslog -processing.</li> -<li><b>ParseKernelTimeStamp</b> [on/<b>off</b>]<br> -If enabled and the kernel creates a timestamp for its log messages, this timestamp will be -parsed and converted into regular message time instead to use the receive time of the kernel -message (as in 5.8.x and before). Default is to not parse the kernel timestamp, because the -clock used by the kernel to create the timestamps is not supposed to be as accurate as the -monotonic clock required to convert it. Depending on the hardware and kernel, it can result -in message time differences between kernel and system messages which occurred at same time. -<li><b>KeepKernelTimeStamp</b> [on/<b>off</b>]<br> -If enabled, this option causes to keep the [timestamp] provided by the kernel at the begin -of in each message rather than to remove it, when it could be parsed and converted into -local time for use as regular message time. Only used when <b>ParseKernelTimestamp</b> is on. -<li><b>ConsoleLogLevel</b> [<i>number</i>] -(former klogd -c option) -- sets the console log level. If specified, only messages with -up to the specified level are printed to the console. The default is -1, which means that -the current settings are not modified. To get this behavior, do not specify -ConsoleLogLevel in the configuration file. Note that this is a global parameter. Each time -it is changed, the previous definition is re-set. The one activate will be that one that is -active when imklog actually starts processing. In short words: do not specify this -directive more than once! -</ul> -<b>Caveats/Known Bugs:</b> -<p>This is obviously platform specific and requires platform -drivers. -Currently, imklog functionality is available on Linux and BSD.</p> -<p>This module is <b>not supported on Solaris</b> and not needed there. -For Solaris kernel input, use <a href="imsolaris.html">imsolaris</a>.</p> -<p><b>Sample:</b></p> -<p>The following sample pulls messages from the kernel log. All -parameters are left by default, which is usually a good idea. Please -note that loading the plugin is sufficient to activate it. No directive -is needed to start pulling kernel messages.<br> -</p> -<textarea rows="4" cols="60">module(load="imklog") -</textarea> -<p><b>Legacy Configuration Directives</b>:</p> -<ul> -<li><strong>$KLogInternalMsgFacility -<facility></strong><br> -equivalent to: InternalMsgFacility</li> -<li><span style="font-weight: bold;">$KLogPermitNonKernelFacility -[on/<span style="font-style: italic;">off</span>]<br> -equivalent to: PermitNonKernelFacility</li> -<li><span style="font-weight: bold;"></span>$DebugPrintKernelSymbols -[on/<b>off</b>]<br> -Linux only, ignored on other platforms (but may be specified)</li> -<li><b>$klogLocalIPIF</b> [interface name] - (available since 5.9.6) - if provided, the IP of the specified -interface (e.g. "eth0") shall be used as fromhost-ip for imklog-originating messages. -If this directive is not given OR the interface cannot be found (or has no IP address), -the default of "127.0.0.1" is used. -</li> -<li>$klogSymbolLookup [on/<b>off</b>] -- -disables imklog kernel symbol translation (former klogd -x option). NOTE that -this option is counter-productive on recent kernels (>= 2.6) because the -kernel already does the symbol translation and this option breaks the information.<br> -<b>This option is scheduled for removal, probably with version 4.x.</b> Do not use -it except if you have a very good reason. If you have one, let us know -because otherwise new versions will no longer support it.<br> -Linux only, ignored on other platforms (but may be specified)</li> -<li><b>$klogConsoleLogLevel</b> [<i>number</i>] -<br>equivalent to: ConsoleLogLevel</li> -<li><b>$klogUseSyscallInterface</b> [on/<b>off</b>] --- former klogd -s option<br> -Linux only, ignored on other platforms (but may be specified)</li> -<li>$klogSymbolsTwice [on/<b>off</b>] -- -former klogd -2 option<br> -Linux only, ignored on other platforms (but may be specified)<br style="font-weight: bold;"> -</li> -<li><b>$klogParseKernelTimeStamp</b> [on/<b>off</b>]<br> -equivalent to: ParseKernelTimeStamp</li> -<li><b>$klogKeepKernelTimeStamp</b> [on/<b>off</b>]<br> -equivalent to: KeepKernelTimeStamp</li> -</ul> -<p>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>] -[<a href="manual.html">manual index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> -<p><font size="2">This documentation is part of the -<a href="http://www.rsyslog.com/">rsyslog</a> -project.<br> -Copyright © 2008-2012 by <a href="http://www.gerhards.net/rainer">Rainer -Gerhards</a> and -<a href="http://www.adiscon.com/">Adiscon</a>. -Released under the GNU GPL version 3 or higher.</font></p> -</body></html> diff --git a/doc/imkmsg.html b/doc/imkmsg.html deleted file mode 100644 index 23b9614..0000000 --- a/doc/imkmsg.html +++ /dev/null @@ -1,50 +0,0 @@ -<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> -<html><head> -<meta http-equiv="Content-Language" content="en"><title>/dev/kmsg Log Input Module (imkmsg)</title> - -</head> -<body> -<a href="rsyslog_conf_modules.html">back</a> - -<h1>/dev/kmsg Log Input Module</h1> -<p><b>Module Name: imkmsg</b></p> -<p><b>Authors: </b>Rainer Gerhards -<rgerhards@adiscon.com><br /> -Milan Bartos -<mbartos@redhat.com></p> -<p><b>Description</b>:</p> -<p>Reads messages from the /dev/kmsg structured kernel log and submits them to the -syslog engine.</p> -<p> -The printk log buffer constains log records. These records are exported by /dev/kmsg -device as structured data in the following format:<br /> - "level,sequnum,timestamp;<message text>\n"<br /> -There could be continuation lines starting with space that contains key/value pairs.<br /> -<br /> -Log messages are parsed as necessary into rsyslog msg_t structure. Continuation lines are parsed -as json key/value pairs and added into rsyslog's message json representation. -</p> -<p><b>Configuration Directives</b>:</p> -<p>This module has no configuration directives.</p> -<b>Caveats/Known Bugs:</b> -<p>This module can't be used together with imklog module. When using one of them, make sure the other -one is not enabled.</p> -<p>This is Linux specific module and requires /dev/kmsg device with structured kernel logs.</p> -<p><b>Sample:</b></p> -<p>The following sample pulls messages from the /dev/kmsg log device. All -parameters are left by default, which is usually a good idea. Please -note that loading the plugin is sufficient to activate it. No directive -is needed to start pulling messages.<br> -</p> -<textarea rows="15" cols="60">$ModLoad imkmsg -</textarea> -<p>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>] -[<a href="manual.html">manual index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> -<p><font size="2">This documentation is part of the -<a href="http://www.rsyslog.com/">rsyslog</a> -project.<br> -Copyright © 2008-2009 by <a href="http://www.gerhards.net/rainer">Rainer -Gerhards</a> and -<a href="http://www.adiscon.com/">Adiscon</a>. -Released under the GNU GPL version 3 or higher.</font></p> -</body></html> diff --git a/doc/impstats.html b/doc/impstats.html deleted file mode 100644 index 8db9c6f..0000000 --- a/doc/impstats.html +++ /dev/null @@ -1,130 +0,0 @@ -<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> -<html><head> -<title>Periodic Statistics of Internal Counters (impstats)</title> -</head> -<body> -<a href="rsyslog_conf_modules.html">back</a> - -<h1>Input Module to Generate Periodic Statistics of Internal Counters</h1> -<p><b>Module Name: impstats</b></p> -<p><b>Available since: </b>5.7.0+, 6.1.1+ -<p><b>Author: </b>Rainer Gerhards <rgerhards@adiscon.com></p> -<p><b>Description</b>:</p> -<p>This module provides periodic output of rsyslog internal counters. -Note that the whole statistics system is currently under development. So -availabilty and format of counters may change and is not yet stable (so be -prepared to change your trending scripts when you upgrade to a newer rsyslog version). -<p>The set of available counters will be output as a set of syslog messages. This -output is periodic, with the interval being configurable (default is 5 minutes). -Be sure that your configuration records the counter messages (default is syslog.=info). -Besides logging to the regular syslog stream, the module can also be configured to -write statistics data into a (local) file. -<p>Note that loading this module has impact on rsyslog performance. Depending on -settings, this impact may be noticable (for high-load environments). -<p>The rsyslog website has an updated overview of available -<a href="http://rsyslog.com/rsyslog-statistic-counter/">rsyslog statistic counters</a>. -</p> -<p><b>Module Confguration Parameters</b>:</p> -<p>This module supports module parameters, only. -<ul> - <li><strong>interval </strong>[seconds] (default 300 [5minutes])<br> - Sets the interval, in <b>seconds</b> at which messages are generated. Please note that the - actual interval may be a bit longer. We do not try to be precise and so the interval is - actually a sleep period which is entered after generating all messages. So the actual - interval is what is configured here plus the actual time required to generate messages. - In general, the difference should not really matter. - <br></li> - <li><strong>facility </strong>[templateName]<br> - The numerical syslog facility code to be used for generated messages. Default - is 5 (syslog). This is useful for filtering messages. - <br></li> - <li><strong>severity </strong>[templateName]<br> - The numerical syslog severity code to be used for generated messages. Default - is 6 (info).This is useful for filtering messages. - <br></li> - <li><strong>format </strong>[json/cee/<b>legacy</b>](rsyslog v6.3.8+ only)<br> - Specifies the format of emitted stats messages. The default of "legacy" is - compatible with pre v6-rsyslog. The other options provide support for - structured formats (note the "cee" is actually "project lumberack" logging). - <br></li> - <li><strong>log.syslog </strong>[<b>on</b>/off] - available since 7.3.6<br> - This is a boolean setting specifying if data should be sent - to the usual syslog stream. This is useful if custom formatting - or more elaborate processing is desired. However, output is placed - under the same restrictions as regular syslog data, especially in - regard to the queue position (stats data may sit for an extended - period of time in queues if they are full).<br></li> - <li><strong>log.file </strong>[file name] - available since 7.3.6<br> - If specified, statistics data is written the specified file. For - robustness, this should be a local file. The file format cannot be - customized, it consists of a date header, followed by a colon, - followed by the actual statistics record, all on one line. Only - very limited error handling is done, so if things go wrong stats - records will probably be lost. Logging to file an be a useful - alternative if for some reasons (e.g. full queues) the regular - syslog stream method shall not be used solely. Note that turning - on file logging does NOT turn of syslog logging. If that is desired - log.syslog="off" must be explicitely set. - <br></li> - -</ul> -<p><b>Legacx Configuration Directives</b>:</p> -A limited set of parameters can also be set via the legacy configuration -syntax. Note that this is intended as an upward compatibilit layer, so -newer features are intentionally <b>not</b> available via legacy directives. -<ul> -<li>$PStatInterval <Seconds> - same as the "interval" parameter. -<li>$PStatFacility <numerical facility> - same as the "facility" parameter. -<li>$PStatSeverity <numerical severity> - same as the "severity" parameter. -<li>$PStatJSON <on/<b>off</b>> (rsyslog v6.3.8+ only)<br> -If set to on, stats messages are emitted as structured cee-enhanced syslog. If -set to off, legacy format is used (which is compatible with pre v6-rsyslog). -</li> -</ul> -<b>Caveats/Known Bugs:</b> -<ul> -<li>This module MUST be loaded right at the top of rsyslog.conf, otherwise -stats may not get turned on in all places.</li> -</ul> -<p><b>Samples:</b></p> -<p>This activates the module and records messages to /var/log/rsyslog-stats in 10 minute intervals:<br> -</p> -<textarea rows="5" cols="60">module(load="impstats" interval="600" severity="7") - -# to actually gather the data: -syslog.=debug /var/log/rsyslog-stats -</textarea> -<p><b>Legacy Sample:</b></p> -<p>This activates the module and records messages to /var/log/rsyslog-stats in 10 minute intervals:</p> -<textarea rows="6" cols="60">$ModLoad impstats -$PStatInterval 600 -$PStatSeverity 7 - -syslog.=debug /var/log/rsyslog-stats -</textarea> -<p>In the next sample, the default interval of 5 minutes is used. However, this time -stats data is NOT emitted to the syslog stream but to a local file instead. -<p> -<textarea rows="3" cols="70">module(load="impstats" interval="600" severity="7" - log.syslog="off" /* need to turn log stream logging off! */ - log.file="/path/to/local/stats.log") -</textarea> -<p>And finally, we log to both the regular syslog log stream as well as a file. -Within the log stream, we forward the data records to another server: -<p> -<textarea rows="4" cols="70">module(load="impstats" interval="600" severity="7" - log.file="/path/to/local/stats.log") - -syslog.=debug @central.example.net -</textarea> - -<p>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>] -[<a href="manual.html">manual index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> -<p><font size="2">This documentation is part of the -<a href="http://www.rsyslog.com/">rsyslog</a> -project.<br> -Copyright © 2013 by <a href="http://www.gerhards.net/rainer">Rainer -Gerhards</a> and -<a href="http://www.adiscon.com/">Adiscon</a>. -Released under the GNU GPL version 3 or higher.</font></p> -</body></html> diff --git a/doc/imptcp.html b/doc/imptcp.html deleted file mode 100644 index b5bd097..0000000 --- a/doc/imptcp.html +++ /dev/null @@ -1,166 +0,0 @@ -<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> -<html><head> -<meta http-equiv="Content-Language" content="en"> -<title>Plain TCP Syslog Input Module (imptcp)</title></head> -<body> -<a href="rsyslog_conf_modules.html">back</a> - -<h1>Plain TCP Syslog Input Module</h1> -<p><b>Module Name: imptcp</b></p> -<p><b>Available since: </b>4.7.3+, 5.5.8+ -<p><b>Author: </b>Rainer Gerhards -<rgerhards@adiscon.com></p> -<p><b>Description</b>:</p> -<p>Provides the ability to receive syslog messages via plain TCP syslog. -This is a specialised input plugin tailored for high performance on Linux. It will -probably not run on any other platform. Also, it does not provide TLS services. -Encryption can be provided by using <a href="rsyslog_stunnel.html">stunnel</a>. -<p>This module has no limit on the number of listeners and sessions that can be used. -</p> - -<p><b>Configuration Directives</b>:</p> -<p>This plugin has config directives similar named as imtcp, but they all have <b>P</b>TCP in -their name instead of just TCP. Note that only a subset of the parameters are supported. -<ul> - -<p><b>Module Parameters</b>:</p> -<p>These paramters can be used with the "module()" statement. They apply -globaly to all inputs defined by the module. -<ul> -<li>Threads <number><br> -Number of helper worker threads to process incoming messages. These -threads are utilized to pull data off the network. On a busy system, additional -helper threads (but not more than there are CPUs/Cores) can help improving -performance. The default value is two. -</ul> -<p><b>Input Parameters</b>:</p> -<p>These parameters can be used with the "input()" statement. They apply to the -input they are specified with. -<ul> -<li><b>AddtlFrameDelimiter</b> <Delimiter><br> -This directive permits to specify an additional frame delimiter for plain tcp syslog. -The industry-standard specifies using the LF character as frame delimiter. Some vendors, -notable Juniper in their NetScreen products, use an invalid frame delimiter, in Juniper's -case the NUL character. This directive permits to specify the ASCII value of the delimiter -in question. Please note that this does not guarantee that all wrong implementations can -be cured with this directive. It is not even a sure fix with all versions of NetScreen, -as I suggest the NUL character is the effect of a (common) coding error and thus will -probably go away at some time in the future. But for the time being, the value 0 can -probably be used to make rsyslog handle NetScreen's invalid syslog/tcp framing. -For additional information, see this -<a href="http://kb.monitorware.com/problem-with-netscreen-log-t1652.html">forum thread</a>. -<br><b>If this doesn't work for you, please do not blame the rsyslog team. Instead file -a bug report with Juniper!</b> -<br>Note that a similar, but worse, issue exists with Cisco's IOS implementation. They do -not use any framing at all. This is confirmed from Cisco's side, but there seems to be -very limited interest in fixing this issue. This directive <b>can not</b> fix the Cisco bug. -That would require much more code changes, which I was unable to do so far. Full details -can be found at the <a href="http://www.rsyslog.com/Article321.phtml">Cisco tcp syslog anomaly</a> -page. -<li><b>SupportOctetCountedFraming</b> <<b>on</b>|off><br> -If set to "on", the legacy octed-counted framing (similar to RFC5425 framing) is -activated. This is the default and should be left unchanged until you know -very well what you do. It may be useful to turn it off, if you know this framing -is not used and some senders emit multi-line messages into the message stream. -</li> -<li><b>ServerNotifyOnConnectionClose</b> [on/<b>off</b>]<br> -instructs imptcp to emit a message if the remote peer closes a connection.<br> -<li><b>KeepAlive</b> <on/<b>off</b>><br> -enable of disable keep-alive packets at the tcp socket layer. The default is -to disable them.</li> -<li><b>KeepAlive.Probes</b> <number><br> -The number of unacknowledged probes to send before considering the connection dead and notifying the application layer. -The default, 0, means that the operating system defaults are used. This has only -effect if keep-alive is enabled. The functionality may not be available on -all platforms. -<li><b>KeepAlive.Interval</b> <number><br> -The interval between subsequential keepalive probes, regardless of what the connection has exchanged in the meantime. -The default, 0, means that the operating system defaults are used. This has only -effect if keep-alive is enabled. The functionality may not be available on -all platforms. -<li><b>KeepAlive.Time</b> <number><br> -The interval between the last data packet sent (simple ACKs are not considered data) and the first keepalive probe; after the connection is marked to need keepalive, this counter is not used any further. -The default, 0, means that the operating system defaults are used. This has only -effect if keep-alive is enabled. The functionality may not be available on -all platforms. -<li><b>Port</b> <number><br> -Select a port to listen on</li> -<li><b>Name</b> <name><br> -Sets a name for the inputname property. If no name is set "imptcp" is used by default. Setting a -name is not strictly necessary, but can be useful to apply filtering based on which input -the message was received from. -<li><b>Ruleset</b> <name><br> -Binds specified ruleset to next server defined. -<li><b>Address</b> <name><br> -On multi-homed machines, specifies to which local address the listerner should be bound. -<li><b>RateLimit.Interval</b> [number] - (available since 7.3.1) specifies the rate-limiting -interval in seconds. Default value is 0, which turns off rate limiting. Set it to a number -of seconds (5 recommended) to activate rate-limiting. -</li> -<li><b>RateLimit.Burst</b> [number] - (available since 7.3.1) specifies the rate-limiting -burst in number of messages. Default is 10,000. -</li> -</ul> -<b>Caveats/Known Bugs:</b> -<ul> -<li>module always binds to all interfaces</li> -</ul> -<p><b>Sample:</b></p> -<p>This sets up a TCP server on port 514:<br> -</p> -<textarea rows="4" cols="60">module(load="/folder/to/rsyslog/plugins/imptcp/.libs/imptcp") # needs to be done just once -input(type="imptcp" port="514") -</textarea> - -<p><b>Legacy Configuration Directives</b>:</p> -<ul> -<li>$InputPTCPServerAddtlFrameDelimiter <Delimiter><br> -Equivalent to: AddTLFrameDelimiter</li> -<li><b>$InputPTCPSupportOctetCountedFraming</b> <<b>on</b>|off><br> -Equivalent to: SupportOctetCountedFraming -</li> -<li>$InputPTCPServerNotifyOnConnectionClose [on/<b>off</b>]<br> -Equivalent to: ServerNotifyOnConnectionClose.<br></li> -<li><b>$InputPTCPServerKeepAlive</b> <on/<b>off</b>><br> -Equivalent to: KeepAlive </li> -<li><b>$InputPTCPServerKeepAlive_probes</b> <number><br> -Equivalent to: KeepAlive.Probes</li> -<li><b>$InputPTCPServerKeepAlive_intvl</b> <number><br> -Equivalent to: KeepAlive.Interval </li> -<li><b>$InputPTCPServerKeepAlive_time</b> <number><br> -Equivalent to: KeepAlive.Time</li> -<li><b>$InputPTCPServerRun</b> <port><br> -Equivalent to: Port </li> -<li>$InputPTCPServerInputName <name><br> -Equivalent to: Name </li> -<li>$InputPTCPServerBindRuleset <name><br> -Equivalent to: Ruleset </li> -<li>$InputPTCPHelperThreads <number><br> -Number of helper worker threads to process incoming messages. These -threads are utilized to pull data off the network. On a busy system, additional -helper threads (but not more than there are CPUs/Cores) can help improving -performance. The default value is two. -<li>$InputPTCPServerListenIP <name><br> -Equivalent to: Address </li> -</ul> -<b>Caveats/Known Bugs:</b> -<ul> -<li>module always binds to all interfaces</li> -</ul> -<p><b>Sample:</b></p> -<p>This sets up a TCP server on port 514:<br> -</p> -<textarea rows="3" cols="60">$ModLoad imptcp # -needs to be done just once -$InputPTCPServerRun 514 -</textarea> -<p>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>] -[<a href="manual.html">manual index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> -<p><font size="2">This documentation is part of the -<a href="http://www.rsyslog.com/">rsyslog</a> -project.<br> -Copyright © 2010-2012 by <a href="http://www.gerhards.net/rainer">Rainer -Gerhards</a> and -<a href="http://www.adiscon.com/">Adiscon</a>. -Released under the GNU GPL version 3 or higher.</font></p> -</body></html> diff --git a/doc/imrelp.html b/doc/imrelp.html deleted file mode 100644 index 9f3e487..0000000 --- a/doc/imrelp.html +++ /dev/null @@ -1,84 +0,0 @@ -<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> -<html><head> -<meta http-equiv="Content-Language" content="en"><title>RELP Input Module</title> - -</head> -<body> -<a href="rsyslog_conf_modules.html">back</a> - -<h1>RELP Input Module</h1> -<p><b>Module Name: imrelp</b></p> -<p><b>Author: Rainer Gerhards</b></p> -<p><b>Description</b>:</p> -<p>Provides the ability to receive syslog messages via the -reliable RELP protocol. This module requires <a href="http://www.librelp.com">librelp</a> to be -present on the system. From the user's point of view, imrelp works much -like imtcp or imgssapi, except that no message loss can occur. Please -note that with the currently supported relp protocol version, a minor -message duplication may occur if a network connection between the relp -client and relp server breaks after the client could successfully send -some messages but the server could not acknowledge them. The window of -opportunity is very slim, but in theory this is possible. Future -versions of RELP will prevent this. Please also note that rsyslogd may -lose a few messages if rsyslog is shutdown while a network conneciton -to the server is broken and could not yet be recovered. Future version -of RELP support in rsyslog will prevent that. Please note that both -scenarios also exists with plain tcp syslog. RELP, even with the small -nits outlined above, is a much more reliable solution than plain tcp -syslog and so it is highly suggested to use RELP instead of plain tcp. -Clients send messages to the RELP server via omrelp.</p> - -<p><b>Configuration Directives</b>:</p> -<ul> -<li><b>Ruleset</b> <name></br> -Binds the specified ruleset to all RELP listeners. -<li><b>Port</b> <port><br> -Starts a RELP server on selected port</li> -</ul> -<b>Caveats/Known Bugs:</b> -<ul> -<li>see description</li> -<li>To obtain the remote system's IP address, you need to have at least -librelp 1.0.0 installed. Versions below it return the hostname instead -of the IP address.</li> -<li>Contrary to other inputs, the ruleset can only be bound to all listeners, -not specific ones. This is due to a currently existing limitation in librelp. -</ul> -<p><b>Sample:</b></p> -<p>This sets up a RELP server on port 20514.<br> -</p> -<textarea rows="15" cols="60">module(load="imrelp") # needs to be done just once -input(type="imrelp" port="20514") -</textarea> - -<p><b>Legacy Configuration Directives</b>:</p> -<ul> -<li>InputRELPServerBindRuleset <name> (available in 6.3.6+)</br> -equivalent to: RuleSet -<li>InputRELPServerRun <port><br> -equivalent to: Port</li> -</ul> -<b>Caveats/Known Bugs:</b> -<ul> -<li>see description</li> -<li>To obtain the remote system's IP address, you need to have at least -librelp 1.0.0 installed. Versions below it return the hostname instead -of the IP address.</li> -<li>Contrary to other inputs, the ruleset can only be bound to all listeners, -not specific ones. This is due to a currently existing limitation in librelp. -</ul> -<p><b>Sample:</b></p> -<p>This sets up a RELP server on port 20514.<br> -</p> -<textarea rows="15" cols="60">$ModLoad imrelp # needs to be done just once -$InputRELPServerRun 20514 -</textarea> -<p>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>] -[<a href="manual.html">manual index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> -<p><font size="2">This documentation is part of the -<a href="http://www.rsyslog.com/">rsyslog</a> project.<br> -Copyright © 2008-2011 by <a href="http://www.gerhards.net/rainer">Rainer -Gerhards</a> and -<a href="http://www.adiscon.com/">Adiscon</a>. -Released under the GNU GPL version 3 or higher.</font></p> -</body></html> diff --git a/doc/imsolaris.html b/doc/imsolaris.html deleted file mode 100644 index ce0e7e8..0000000 --- a/doc/imsolaris.html +++ /dev/null @@ -1,47 +0,0 @@ -<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> -<html><head> -<meta http-equiv="Content-Language" content="en"> -<title>Solaris Input Module (imsolaris)</title> - -</head> -<body> -<a href="rsyslog_conf_modules.html">back</a> - -<h1>Solaris Input Module</h1> -<p><b>Module Name: imsolaris</b></p> -<p><b>Author: </b>Rainer Gerhards -<rgerhards@adiscon.com></p> -<p><b>Description</b>:</p> -<p>Reads local Solaris log messages including the kernel log.</p> -<p>This module is specifically tailored for Solaris. Under Solaris, there -is no special kernel input device. Instead, both kernel messages as well as -messages emitted via syslog() are received from a single source. -<p>This module obeys the Solaris door() mechanism to detect a running syslogd -instance. As such, only one can be active at one time. If it detects another -active intance at startup, the module disables itself, but rsyslog will -continue to run. -<p><b>Configuration Directives</b>:</p> -<ul> -<li><strong>$IMSolarisLogSocketName <name></strong><br> -This is the name of the log socket (stream) to read. If not given, /dev/log -is read. -</li> -</ul> -<b>Caveats/Known Bugs:</b> -<p>None currently known. For obvious reasons, works on Solaris, only (and compilation -will most probably fail on any other platform). -<p><b>Sample:</b></p> -<p>The following sample pulls messages from the default log source -<br> -</p> -<textarea rows="15" cols="60">$ModLoad imsolaris -</textarea> -<p>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>] -[<a href="manual.html">manual index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> -<p><font size="2">This documentation is part of the -<a href="http://www.rsyslog.com/">rsyslog</a> -project.<br> -Copyright © 2010 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and -<a href="http://www.adiscon.com/">Adiscon</a>. -Released under the GNU GPL version 3 or higher.</font></p> -</body></html> diff --git a/doc/imtcp.html b/doc/imtcp.html deleted file mode 100644 index b9f0b05..0000000 --- a/doc/imtcp.html +++ /dev/null @@ -1,180 +0,0 @@ -<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> -<html> -<head> -<meta http-equiv="Content-Language" content="en"> -<title>TCP Syslog Input Module</title> -</head> - -<body> -<a href="rsyslog_conf_modules.html">back to rsyslog module overview</a> - -<h1>TCP Syslog Input Module</h1> -<p><b>Module Name: imtcp</b></p> -<p><b>Author: </b>Rainer Gerhards <rgerhards@adiscon.com></p> -<p><b>Multi-Ruleset Support: </b>since 4.5.0 and 5.1.1 -<p><b>Description</b>:</p> -<p>Provides the ability to receive syslog messages via TCP. -Encryption is natively provided by selecting the approprioate network stream driver and -can also be provided by using <a href="rsyslog_stunnel.html">stunnel</a> -(an alternative is the use the <a href="imgssapi.html">imgssapi</a> module).</p> - -<p><b>Configuration Directives</b>:</p> -<p><b>Global Directives</b>:</p> -<ul> -<li><b>AddtlFrameDelimiter</b> <Delimiter><br> -This directive permits to specify an additional frame delimiter for plain tcp syslog. -The industry-standard specifies using the LF character as frame delimiter. Some vendors, -notable Juniper in their NetScreen products, use an invalid frame delimiter, in Juniper's -case the NUL character. This directive permits to specify the ASCII value of the delimiter -in question. Please note that this does not guarantee that all wrong implementations can -be cured with this directive. It is not even a sure fix with all versions of NetScreen, -as I suggest the NUL character is the effect of a (common) coding error and thus will -probably go away at some time in the future. But for the time being, the value 0 can -probably be used to make rsyslog handle NetScreen's invalid syslog/tcp framing. -For additional information, see this -<a href="http://kb.monitorware.com/problem-with-netscreen-log-t1652.html">forum thread</a>. -<br><b>If this doesn't work for you, please do not blame the rsyslog team. Instead file -a bug report with Juniper!</b> -<br>Note that a similar, but worse, issue exists with Cisco's IOS implementation. They do -not use any framing at all. This is confirmed from Cisco's side, but there seems to be -very limited interest in fixing this issue. This directive <b>can not</b> fix the Cisco bug. -That would require much more code changes, which I was unable to do so far. Full details -can be found at the <a href="http://www.rsyslog.com/Article321.phtml">Cisco tcp syslog anomaly</a> -page. -<li><b>DisableLFDelimiter</b> <on/<b>off</b>><br> -Industry-strandard plain text tcp syslog uses the LF to delimit syslog frames. However, -some users brought up the case that it may be useful to define a different delimiter and -totally disable LF as a delimiter (the use case named were multi-line messages). This mode -is non-standard and will probably come with a lot of problems. However, as there is need -for it and it is relatively easy to support, we do so. Be sure to turn this setting to -"on" only if you exactly know what you are doing. You may run into all sorts of troubles, -so be prepared to wrangle with that! -<li><b>NotifyOnConnectionClose</b> [on/<b>off</b>]<br> -instructs imtcp to emit a message if the remote peer closes a connection.<br> -<b>Important:</b> This directive is global to all listeners and must be given right -after loading imtcp, otherwise it may have no effect.</li> -<li><b>KeepAlive</b> <on/<b>off</b>><br> -enable of disable keep-alive packets at the tcp socket layer. The default is -to disable them.</li> -<li><b>FlowControl</b> <<b>on</b>/off><br> -This setting specifies whether some message flow control shall be exercised on the -related TCP input. If set to on, messages are handled as "light delayable", which means -the sender is throttled a bit when the queue becomes near-full. This is done in order -to preserve some queue space for inputs that can not throttle (like UDP), but it -may have some undesired effect in some configurations. Still, we consider this as -a useful setting and thus it is the default. To turn the handling off, simply -configure that explicitely. -</li> -<li><b>MaxListeners</b> <number><br> -Sets the maximum number of listeners (server ports) supported. Default is 20. This must be set before the first $InputTCPServerRun directive.</li> -<li><b>MaxSessions</b> <number><br> Sets the maximum number of sessions supported. Default is 200. This must be set before the first $InputTCPServerRun directive</li> -<li><b>StreamDriver.Mode</b> <number><br> -Sets the driver mode for the currently selected <a href="netstream.html">network stream driver</a>. <number> is driver specifc.</li> -<li><b>StreamDriver.AuthMode</b> <mode-string><br> -Sets the authentication mode for the currently selected <a href="netstream.html">network stream driver</a>. <mode-string> is driver specifc.</li> -<li><b>PermittedPeer</b> <id-string><br> -Sets permitted peer IDs. Only these peers are able to connect to the -listener. <id-string> semantics depend on the currently selected -AuthMode and <a href="netstream.html">network stream driver</a>. PermittedPeer may not be set in anonymous modes. -<br>PermittedPeer may be set either to a single peer or an array of peers either of type IP or name, depending on the tls certificate. -<br>Single peer: PermittedPeer="127.0.0.1" -<br>Array of peers: PermittedPeer=["test1.example.net","10.1.2.3","test2.example.net","..."]</li> -</ul> -<p><b>Action Directives</b>:</p> -<ul> -<li><b>Port</b> <port><br> -Starts a TCP server on selected port</li> -<li><b>Name</b> <name><br> -Sets a name for the inputname property. If no name is set "imtcp" is used by default. Setting a -name is not strictly necessary, but can be useful to apply filtering based on which input -the message was received from. -<li><b>Ruleset</b> <ruleset><br> -Binds the listener to a specific <a href="multi_ruleset.html">ruleset</a>.</li> -<li><b>SupportOctetCountedFraming</b> <<b>on</b>|off><br> -If set to "on", the legacy octed-counted framing (similar to RFC5425 framing) is -activated. This is the default and should be left unchanged until you know -very well what you do. It may be useful to turn it off, if you know this framing -is not used and some senders emit multi-line messages into the message stream. -</li> -<li><b>RateLimit.Interval</b> [number] - (available since 7.3.1) specifies the rate-limiting -interval in seconds. Default value is 0, which turns off rate limiting. Set it to a number -of seconds (5 recommended) to activate rate-limiting. -</li> -<li><b>RateLimit.Burst</b> [number] - (available since 7.3.1) specifies the rate-limiting -burst in number of messages. Default is 10,000. -</li> -</ul> -<b>Caveats/Known Bugs:</b> -<ul> -<li>module always binds to all interfaces</li> -<li>can not be loaded together with <a href="imgssapi.html">imgssapi</a> -(which includes the functionality of imtcp)</li> -</ul> -<p><b>Example:</b></p> -<p>This sets up a TCP server on port 514 and permits it to accept up to 500 connections:<br> -</p> -<textarea rows="15" cols="60">module(load="imtcp" MaxSessions="500") -input(type="imtcp" port="514") -</textarea> -<p>Note that the global parameters (here: max sessions) need to be set when the module is loaded. Otherwise, the parameters will not apply. -</p> - -<p><b>Legacy Configuration Directives</b>:</p> -<ul> -<li><b>$InputTCPServerAddtlFrameDelimiter <Delimiter></b><br> -equivalent to: AddtlFrameDelimiter -<li><b>$InputTCPServerDisableLFDelimiter</b> <on/<b>off</b>> (available since 5.5.3)<br> -equivalent to: DisableLFDelimiter -<li><b>$InputTCPServerNotifyOnConnectionClose</b> [on/<b>off</b>] (available since 4.5.5)<br> -equivalent to: NotifyOnConnectionClose<br> -</li> -<li><b>$InputTCPServerKeepAlive</b> <on/<b>off</b>><br> -equivalent to: KeepAlive</li> -<li><b>$InputTCPServerRun</b> <port><br> -equivalent to: Port</li> -<li><b>$InputTCPFlowControl</b> <<b>on</b>/off><br> -equivalent to: FlowControl -</li> -<li><b>$InputTCPMaxListeners</b> <number><br> -equivalent to: MaxListeners</li> -<li><b>$InputTCPMaxSessions</b> <number><br> -equivalent to: MaxSessions</li> -<li><b>$InputTCPServerStreamDriverMode</b> <number><br> -equivalent to: StreamDriver.Mode</li> -<li><b>$InputTCPServerInputName</b> <name><br> -equivalent to: Name -<li><b>$InputTCPServerStreamDriverAuthMode</b> <mode-string><br> -equivalent to: StreamDriver.AuthMode</li> -<li><b>$InputTCPServerStreamDriverPermittedPeer</b> <id-string><br> -equivalent to: PermittedPeer.</li> -<li><b>$InputTCPServerBindRuleset</b> <ruleset><br> -equivalent to: Ruleset</a>.</li> -<li><b>$InputTCPSupportOctetCountedFraming</b> <<b>on</b>|off><br> -equivalent to: SupportOctetCountedFraming -</li> -</ul> -<b>Caveats/Known Bugs:</b> -<ul> -<li>module always binds to all interfaces</li> -<li>can not be loaded together with <a href="imgssapi.html">imgssapi</a> -(which includes the functionality of imtcp)</li> -</ul> -<p><b>Example:</b></p> -<p>This sets up a TCP server on port 514 and permits it to accept up to 500 connections:<br> -</p> -<textarea rows="15" cols="60">$ModLoad imtcp # needs to be done just once -$InputTCPMaxSessions 500 -$InputTCPServerRun 514 -</textarea> -<p>Note that the parameters (here: max sessions) need to be set <b>before</b> the listener -is activated. Otherwise, the parameters will not apply. -</p> -<p>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>] -[<a href="manual.html">manual index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> -<p><font size="2">This documentation is part of the <a href="http://www.rsyslog.com/">rsyslog</a> -project.<br> -Copyright © 2008,2009 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and -<a href="http://www.adiscon.com/">Adiscon</a>. -Released under the GNU GPL version 3 or higher.</font></p> -</body> -</html> diff --git a/doc/imudp.html b/doc/imudp.html deleted file mode 100644 index 6c94953..0000000 --- a/doc/imudp.html +++ /dev/null @@ -1,148 +0,0 @@ -<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> -<html> -<head> -<meta http-equiv="Content-Language" content="en"> -<title>UDP Syslog Input Module (imudp)</title> -</head> - -<body> -<a href="rsyslog_conf_modules.html">back to rsyslog module overview</a> - -<h1>UDP Syslog Input Module</h1> -<p><b>Module Name: imudp</b></p> -<p><b>Author: </b>Rainer Gerhards <rgerhards@adiscon.com></p> -<p><b>Multi-Ruleset Support: </b>since 5.3.2 -<p><b>Description</b>:</p> -<p>Provides the ability to receive syslog messages via UDP. -<p>Multiple receivers may be configured by specifying -multiple input actions. -</p> - -<p><b>Configuration Parameters</b>:</p> -<p><b>Module Parameters</b>:</p> -<ul> -<li><b>TimeRequery</b> <nbr-of-times><br> -this is a performance -optimization. Getting the system time is very costly. With this setting, imudp can -be instructed to obtain the precise time only once every n-times. This logic is -only activated if messages come in at a very fast rate, so doing less frequent -time calls should usually be acceptable. The default value is two, because we have -seen that even without optimization the kernel often returns twice the identical time. -You can set this value as high as you like, but do so at your own risk. The higher -the value, the less precise the timestamp. -<li><b>SchedulingPolicy</b> <rr/fifo/other><br> -Can be used the set the scheduler priority, if the necessary functionality -is provided by the platform. Most useful to select "fifo" for real-time -processing under Linux (and thus reduce chance of packet loss). -<li><b>SchedulingPriority</b> <number><br> -Scheduling priority to use. -</ul> -<p><b>Input Parameters</b>:</p> -<ul> -<li><b>Address</b> <IP><br> -local IP address (or name) the UDP listens should bind to</li> -<li><b>Port</b> <port><br> -default 514, start UDP server on this port. Either a single port can be specified or an array of ports. If multiple ports are specified, a listener will be automatically started for each port. Thus, no additional inputs need to be configured. -<br>Single port: Port="514" -<br>Array of ports: Port=["514","515","10514","..."]</li> -<li><b>Ruleset</b> <ruleset><br> -Binds the listener to a specific <a href="multi_ruleset.html">ruleset</a>.</li> -<li><b>RateLimit.Interval</b> [number] - (available since 7.3.1) specifies the rate-limiting -interval in seconds. Default value is 0, which turns off rate limiting. Set it to a number -of seconds (5 recommended) to activate rate-limiting. -</li> -<li><b>RateLimit.Burst</b> [number] - (available since 7.3.1) specifies the rate-limiting -burst in number of messages. Default is 10,000. -</li> -<li><b>InputName</b> [name] - (available since 7.3.9) specifies the value of -the inputname. In older versions, this was always "imudp" for all listeners, -which still i the default. -Starting with 7.3.9 it can be set to different values for each listener. -Note that when a single input statement defines multipe listner ports, the -inputname will be the same for all of them. If you want to differentiate in that -case, use "InputName.AppendPort" to make them unique. -Note that the "InputName" parameter can be an empty string. In that case, the -corresponding inputname property will obviously also be the empty string. This -is primarily meant to be used togehter with "InputName.AppendPort" to set the -inputname equal to the port. -</li> -<li><b>InputName.AppendPort</b> [on/<b>off</b>] - (available since 7.3.9) -appends the port the the inputname. Note that when no inputname is specified, -the default of "imudp" is used and the port is appended to that default. So, -for example, a listner port of 514 in that case will lead to an inputname -of "imudp514". The ability to append a port is most useful when multiple ports -are defined for a single input and each of the inputnames shall be unique. -Note that there currently is no differentiation between IPv4/v6 listners on -the same port. -</li> -</ul> -<b>Caveats/Known Bugs:</b> -<ul> -<li>Scheduling parameters are set <b>after</b> privileges have been dropped. -In most cases, this means that setting them will not be possible after -privilege drop. This may be worked around by using a sufficiently-privileged -user account. -</li> -</ul> -<p><b>Samples:</b></p> -<p>This sets up an UPD server on port 514:<br> -</p> -<textarea rows="3" cols="60">module(load="imudp") # needs to be done just once -input(type="imudp" port="514") -</textarea> - -<p>In the next example, we set up three listners at ports 10514, 10515 and 10516 -and assign a listner name of "udp" to it, followed by the port number: -</p> -<textarea rows="4" cols="60">module(load="imudp") -input(type="imudp" port=["10514","10515","10516"] - inputname="udp" inputname.appendPort="on") -</textarea> - -<p>The next example is almost equal to the previous one, but -now the inputname property will just be set to the port number. -So if a message was received on port 10515, the input name will be -"10515" in this example whereas it was "udp10515" in the previous one. -Note that to do that we set the inputname to the empty string. -</p> -<textarea rows="4" cols="60">module(load="imudp") -input(type="imudp" port=["10514","10515","10516"] - inputname="" inputname.appendPort="on") -</textarea> - - -<p><b>Legacy Configuration Directives</b>:</p> -<p>Multiple receivers may be configured by specifying -$UDPServerRun multiple times. -</p> -<ul> -<li>$UDPServerAddress <IP><br> -equivalent to: Address </li> -<li>$UDPServerRun <port><br> -equivalent to: Port </li> -<li>$UDPServerTimeRequery <nbr-of-times><br> -equivalent to: TimeRequery -<li>$InputUDPServerBindRuleset <ruleset><br> -equivalent to: Ruleset </li> -<li>$IMUDPSchedulingPolicy <rr/fifo/other> Available since 4.7.4+, 5.7.3+, 6.1.3+.<br> -equivalent to: SchedulingPolicy -<li>$IMUDPSchedulingPriority <number> Available since 4.7.4+, 5.7.3+, 6.1.3+.<br> -equivalent to: SchedulingPriority -</ul> -<p><b>Sample:</b></p> -<p>This sets up an UPD server on port 514:<br> -</p> -<textarea rows="3" cols="60">$ModLoad imudp # needs to be done just once -$UDPServerRun 514 -</textarea> - -<p>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>] -[<a href="manual.html">manual index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> -<p><font size="2">This documentation is part of the -<a href="http://www.rsyslog.com/">rsyslog</a> -project.<br> -Copyright © 2009-2013 by -<a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and -<a href="http://www.adiscon.com/">Adiscon</a>. -Released under the GNU GPL version 3 or higher.</font></p> -</body></html> diff --git a/doc/imuxsock.html b/doc/imuxsock.html deleted file mode 100644 index 0affe8c..0000000 --- a/doc/imuxsock.html +++ /dev/null @@ -1,330 +0,0 @@ -<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> -<html><head> -<meta http-equiv="Content-Language" content="en"> -<title>Unix Socket Input</title> -</head> -<body> -<a href="rsyslog_conf_modules.html">back</a> - -<h1>Unix Socket Input</h1> -<p><b>Module Name: imuxsock</b></p> -<p><b>Author: </b>Rainer Gerhards -<rgerhards@adiscon.com></p> -<p><b>Description</b>:</p> -<p><b>Provides the ability to accept syslog messages via local Unix -sockets. Most importantly, this is the mechanism by which the syslog(3) -call delivers syslog messages to rsyslogd.</b> So you need to have this -module loaded to read the system log socket and be able to process log -messages from applications running on the local system.</p> -<p><b>Application-provided -timestamps are ignored by default.</b> This is needed, as some programs -(e.g. sshd) log with inconsistent timezone information, what -messes up the local logs (which by default don't even contain time zone -information). This seems to be consistent with what sysklogd did for -the past four years. Alternate behaviour may be desirable if -gateway-like processes send messages via the local log slot - in this -case, it can be enabled via the -IgnoreTimestamp and SysSock.IgnoreTimestamp config directives</p> -<p><b>There is input rate limiting available,</b> (since 5.7.1) to guard you against -the problems of a wild running logging process. -If more than SysSock.RateLimit.Interval * SysSock.RateLimit.Burst log messages are emitted -from the same process, those messages with SysSock.RateLimit.Severity or lower will be -dropped. It is not possible to recover anything about these messages, but imuxsock will -tell you how many it has dropped one the interval has expired AND the next message -is logged. Rate-limiting depends on SCM_CREDENTIALS. If the platform does not support -this socket option, rate limiting is turned off. If multiple sockets are configured, -rate limiting works independently on each of them (that should be what you usually expect). -The same functionality is available for additional log sockets, in which case the -config statements just use -the prefix RateLimit... but otherwise works exactly the same. -When working with severities, please keep in mind that higher severity numbers mean lower -severity and configure things accordingly. -To turn off rate limiting, set the interval to zero. -<p><b>Unix log sockets can be flow-controlled.</b> That is, if processing queues fill up, -the unix socket reader is blocked for a short while. This may be useful to prevent overruning -the queues (which may cause exessive disk-io where it actually would not be needed). However, -flow-controlling a log socket (and especially the system log socket) can lead to a very -unresponsive system. As such, flow control is disabled by default. That means any log records -are places as quickly as possible into the processing queues. If you would like to have -flow control, you need to enable it via the SysSock.FlowControl and -FlowControl config directives. Just make sure you thought about -the implications. Note that for many systems, turning on flow control does not hurt. -<p>Starting with rsyslog 5.9.4, -<b><a href="http://www.rsyslog.com/what-are-trusted-properties/">trusted syslog properties</a> -are available</b>. These require a recent enough Linux Kernel and access to the /proc file -system. In other words, this may not work on all platforms and may not work fully when -privileges are dropped (depending on how they are dropped). Note that trusted properties -can be very useful, but also typically cause the message to grow rather large. Also, the -format of log messages is obviously changed by adding the trusted properties at the end. -For these reasons, the feature is <b>not enabled by default</b>. If you want to use it, -you must turn it on (via SysSock.Annotate and Annotate). - -<p><b>Configuration Directives</b>:</p> -<p><b>Global Parameters</b></p> -<ul> -<li><b>SysSock.IgnoreTimestamp</b> [<b>on</b>/off]<br> -Ignore timestamps included in the messages, applies to messages received via the system log socket. -</li> -<li><b>SysSock.IgnoreOwnMessages</b> [<b>on</b>/off] (available since 7.3.7)<br> -Ignores messages that originated from the same instance of rsyslogd. There usually -is no reason to receive messages from ourselfs. This setting is vital -when writing messages to the Linux journal. See <a href="omjournal.html">omjournal</a> -module documentation for a more in-depth description. -</li> -<li><b>SysSock.Use</b> (imuxsock) [on/<b>off</b>] -do NOT listen for the local log socket. This is most useful if you run multiple -instances of rsyslogd where only one shall handle the system log socket. -</li> -<li><b>SysSock.Name</b> <name-of-socket> -</li> -<li><b>SysSock.FlowControl</b> [on/<b>off</b>] - specifies if flow control should be applied -to the system log socket. -</li> -<li><b>SysSock.UsePIDFromSystem</b> [on/<b>off</b>] - specifies if the pid being logged shall -be obtained from the log socket itself. If so, the TAG part of the message is rewritten. -It is recommended to turn this option on, but the default is "off" to keep compatible -with earlier versions of rsyslog. -</li> -<li><b>SysSock.RateLimit.Interval</b> [number] - specifies the rate-limiting -interval in seconds. Default value is 5 seconds. Set it to 0 to turn rate limiting off. -</li> -<li><b>SysSock.RateLimit.Burst</b> [number] - specifies the rate-limiting -burst in number of messages. Default is 200. -</li> -<li><b>SysSock.RateLimit.Severity</b> [numerical severity] - specifies the severity of -messages that shall be rate-limited. -</li> -<li><b>SysSock.UseSysTimeStamp</b> [<b>on</b>/off] the same as $InputUnixListenSocketUseSysTimeStamp, but for the system log socket. -</li> -<li><b>SysSock.Annotate</b> <on/<b>off</b>> turn on annotation/trusted -properties for the system log socket.</li> -<li><b>SysSock.ParseTrusted</b> <on/<b>off</b>> if Annotation is turned on, create -JSON/lumberjack properties out of the trusted properties (which can be accessed -via RainerScript JSON Variables, e.g. "$!pid") instead of adding them to the message. -</li> -<li><b>SysSock.Unlink</b> <<b>on</b>/off> (available since 7.3.9)<br> -if turned on (default), the system socket is unlinked and re-created when -opened and also unlinked when finally closed. Note that this setting has -no effect when running under systemd control (because systemd handles -the socket). -</li> -</ul> - -<p><b>Input Instance Parameters</b></p> -<ul> -<li><b>IgnoreTimestamp</b> [<b>on</b>/off] -<br>Ignore timestamps included in the message. Applies to the next socket being added.</li> -<li><b>IgnoreOwnMessages</b> [<b>on</b>/off] (available since 7.3.7)<br> -Ignore messages that originated from the same instance of rsyslogd. There usually -is no reason to receive messages from ourselfs. This setting is vital -when writing messages to the Linux journal. See <a href="omjournal.html">omjournal</a> -module documentation for a more in-depth description. -</li> -<li><b>FlowControl</b> [on/<b>off</b>] - specifies if flow control should be applied -to the next socket.</li> -<li><b>RateLimit.Interval</b> [number] - specifies the rate-limiting -interval in seconds. Default value is 0, which turns off rate limiting. Set it to a number -of seconds (5 recommended) to activate rate-limiting. The default of 0 has been choosen -as people experienced problems with this feature activated by default. Now it needs an -explicit opt-in by setting this parameter. -</li> -<li><b>RateLimit.Burst</b> [number] - specifies the rate-limiting -burst in number of messages. Default is 200. -</li> -<li><b>RateLimit.Severity</b> [numerical severity] - specifies the severity of -messages that shall be rate-limited. -</li> -<!--<li><b>LocalIPIF</b> [interface name] - if provided, the IP of the specified -interface (e.g. "eth0") shall be used as fromhost-ip for imuxsock-originating messages. -If this directive is not given OR the interface cannot be found (or has no IP address), -the default of "127.0.0.1" is used. -</li>--> -<li><b>UsePIDFromSystem</b> [on/<b>off</b>] - specifies if the pid being logged shall -be obtained from the log socket itself. If so, the TAG part of the message is rewritten. -It is recommended to turn this option on, but the default is "off" to keep compatible -with earlier versions of rsyslog. </li> -<li><b>UseSysTimeStamp</b> [<b>on</b>/off] instructs imuxsock -to obtain message time from the system (via control messages) insted of using time -recorded inside the message. This may be most useful in combination with systemd. Note: -this option was introduced with version 5.9.1. Due to the usefulness of it, we -decided to enable it by default. As such, 5.9.1 and above behave slightly different -than previous versions. However, we do not see how this could negatively affect -existing environments.<br> -<li><b>CreatePath</b> [on/<b>off</b>] - create directories in the socket path -if they do not already exist. They are created with 0755 permissions with the owner being the process under -which rsyslogd runs. The default is not to create directories. Keep in mind, though, that rsyslogd always -creates the socket itself if it does not exist (just not the directories by default). -<br>Note that this statement affects the -next Socket directive that follows in sequence in the configuration file. It never works -on the system log socket (where it is deemed unnecessary). Also note that it is automatically -being reset to "off" after the Socket directive, so if you would have it active -for two additional listen sockets, you need to specify it in front of each one. This option is primarily considered -useful for defining additional sockets that reside on non-permanent file systems. As rsyslogd probably starts -up before the daemons that create these sockets, it is a vehicle to enable rsyslogd to listen to those -sockets even though their directories do not yet exist.</li> -<li><b>Socket</b> <name-of-socket> adds additional unix socket, default none -- former -a option</li> -<li><b>HostName</b> <hostname> permits to override the hostname that -shall be used inside messages taken from the <b>next</b> Socket socket. Note that -the hostname must be specified before the $AddUnixListenSocket configuration directive, and it -will only affect the next one and then automatically be reset. This functionality is provided so -that the local hostname can be overridden in cases where that is desired.</li> -<li><b>Annotate</b> <on/<b>off</b>> turn on annotation/trusted -properties for the non-system log socket in question.</li> -<li><b>ParseTrusted</b> <on/<b>off</b>> equivalent to the SysSock.ParseTrusted module -parameter, but applies to the input that is being defined. -<li><b>Unlink</b> <<b>on</b>/off> (available since 7.3.9)<br> -if turned on (default), the socket is unlinked and re-created when -opened and also unlinked when finally closed. Set it to off if you -handle socket creation yourself. Note that handling socket creation -oneself has the advantage that a limited amount of messages may be -queued by the OS if rsyslog is not running. -</li> -</ul> - -<b>Caveats/Known Bugs:</b><br> -<ul> -<li>There is a compile-time limit of 50 concurrent sockets. If you need more, you need to -change the array size in imuxsock.c. -<li>This documentation is sparse and incomplete. -</ul> -<p><b>Sample:</b></p> -<p>The following sample is the minimum setup required to accept syslog messages from applications running -on the local system.<br> -</p> -<textarea rows="2" cols="70">module(load="imuxsock" # needs to be done just once -SysSock.FlowControl="on") # enable flow control (use if needed) -</textarea> - -<p>The following sample is similiar to the first one, but enables trusted -properties, which are put into JSON/lumberjack variables. -<br> -</p> -<textarea rows="2" cols="70">module(load="imuxsock" SysSock.Annotate="on" SysSock.ParseTrusted="on") -</textarea> - -<p>The following sample is a configuration where rsyslogd pulls logs from two -jails, and assigns different hostnames to each of the jails: </p> -<textarea rows="6" cols="70">module(load="imuxsock") # needs to be done just once - -input(type="imuxsock" HostName="jail1.example.net" Socket="/jail/1/dev/log") -input(type="imuxsock" HostName="jail2.example.net" Socket="/jail/2/dev/log") -</textarea> -<p>The following sample is a configuration where rsyslogd reads the openssh log -messages via a separate socket, but this socket is created on a temporary file -system. As rsyslogd starts up before the sshd, it needs to create the socket -directories, because it otherwise can not open the socket and thus not listen -to openssh messages. Note that it is vital not to place any other socket between -the CreatePath and the Socket.</p> -<textarea rows="6" cols="70">module(load="imuxsock") # needs to be done just once - -input(type="imuxsock" Socket="/var/run/sshd/dev/log" CreatePath="on") -</textarea> -<p>The following sample is used to turn off input rate limiting on the system log -socket. -<textarea rows="4" cols="70">module(load="imuxsock" # needs to be done just once -SysSock.RateLimit.Interval="0") # turn off rate limiting -</textarea> -<p>The following sample is used activate message annotation and thus trusted properties -on the system log socket. -<textarea rows="4" cols="70">module(load="imuxsock" # needs to be done just once -SysSock.Annotate="on") -</textarea> - - -<p><b>Legacy Configuration Directives</b>:</p> -<ul> -<li><b>$InputUnixListenSocketIgnoreMsgTimestamp</b> [<b>on</b>/off] -<br>equivalent to: IgnoreTimestamp.</li> -<li><b>$InputUnixListenSocketFlowControl</b> [on/<b>off</b>] - equivalent to: FlowControl .</li> -<li><b>$IMUXSockRateLimitInterval</b> [number] - equivalent to: RateLimit.Interval -</li> -<li><b>$IMUXSockRateLimitBurst</b> [number] - equivalent to: RateLimit.Burst -</li> -<li><b>$IMUXSockRateLimitSeverity</b> [numerical severity] - equivalent to: RateLimit.Severity -</li> -<li><b>$IMUXSockLocalIPIF</b> [interface name] - (available since 5.9.6) - if provided, the IP of the specified -interface (e.g. "eth0") shall be used as fromhost-ip for imuxsock-originating messages. -If this directive is not given OR the interface cannot be found (or has no IP address), -the default of "127.0.0.1" is used. -</li> -<li><b>$InputUnixListenSocketUsePIDFromSystem</b> [on/<b>off</b>] - equivalent to: UsePIDFromSystem. -<br>This option was introduced in 5.7.0.</li> -<li><b>$InputUnixListenSocketUseSysTimeStamp</b> [<b>on</b>/off] equivalent to: UseSysTimeStamp .<br> -<li><b>$SystemLogSocketIgnoreMsgTimestamp</b> [<b>on</b>/off]<br> -equivalent to: SysSock.IgnoreTimestamp.</li> -<li><b>$OmitLocalLogging</b> (imuxsock) [on/<b>off</b>] equivalent to: SysSock.Use</li> -<li><b>$SystemLogSocketName</b> <name-of-socket> equivalent to: SysSock.Name</li> -<li><b>$SystemLogFlowControl</b> [on/<b>off</b>] - equivalent to: SysSock.FlowControl.</li> -<li><b>$SystemLogUsePIDFromSystem</b> [on/<b>off</b>] - equivalent to: SysSock.UsePIDFromSystem. -<br>This option was introduced in 5.7.0.</li> -<li><b>$SystemLogRateLimitInterval</b> [number] - equivalent to: SysSock.RateLimit.Interval. -</li> -<li><b>$SystemLogRateLimitBurst</b> [number] - equivalent to: SysSock.RateLimit.Burst -</li> -<li><b>$SystemLogRateLimitSeverity</b> [numerical severity] - equivalent to: SysSock.RateLimit.Severity -</li> -<li><b>$SystemLogUseSysTimeStamp</b> [<b>on</b>/off] equivalent to: SysSock.UseSysTimeStamp. -<li><b>$InputUnixListenSocketCreatePath</b> [on/<b>off</b>] - equivalent to: CreatePath -<br>[available since 4.7.0 and 5.3.0]</li> -<li><b>$AddUnixListenSocket</b> <name-of-socket> equivalent to: Socket </li> -<li><b>$InputUnixListenSocketHostName</b> <hostname> equivalent to: HostName.</li> -<li><b>$InputUnixListenSocketAnnotate</b> <on/<b>off</b>> equivalent to: Annotate.</li> -<li><b>$SystemLogSocketAnnotate</b> <on/<b>off</b>> equivalent to: SysSock.Annotate.</li> -<li><b>$SystemLogSocketParseTrusted</b> <on/<b>off</b>> equivalent to: SysSock.ParseTrusted.</li> -</ul> - -<b>Caveats/Known Bugs:</b><br> -<ul> -<li>There is a compile-time limit of 50 concurrent sockets. If you need more, you need to -change the array size in imuxsock.c. -<li>This documentation is sparse and incomplete. -</ul> -<p><b>Sample:</b></p> -<p>The following sample is the minimum setup required to accept syslog messages from applications running -on the local system.<br> -</p> -<textarea rows="2" cols="70">$ModLoad imuxsock # needs to be done just once -$SystemLogSocketFlowControl on # enable flow control (use if needed) -</textarea> -<p>The following sample is a configuration where rsyslogd pulls logs from two -jails, and assigns different hostnames to each of the jails: </p> -<textarea rows="6" cols="70">$ModLoad imuxsock # needs to be done just once - -$InputUnixListenSocketHostName jail1.example.net -$AddUnixListenSocket /jail/1/dev/log -$InputUnixListenSocketHostName jail2.example.net -$AddUnixListenSocket /jail/2/dev/log -</textarea> -<p>The following sample is a configuration where rsyslogd reads the openssh log -messages via a separate socket, but this socket is created on a temporary file -system. As rsyslogd starts up before the sshd, it needs to create the socket -directories, because it otherwise can not open the socket and thus not listen -to openssh messages. Note that it is vital not to place any other socket between -the $InputUnixListenSocketCreatePath and the $InputUnixListenSocketHostName.</p> -<textarea rows="6" cols="70">$ModLoad imuxsock # needs to be done just once - -$InputUnixListenSocketCreatePath on # turn on for *next* socket -$InputUnixListenSocket /var/run/sshd/dev/log -</textarea> -<p>The following sample is used to turn off input rate limiting on the system log -socket. -<textarea rows="4" cols="70">$ModLoad imuxsock # needs to be done just once - -$SystemLogRateLimitInterval 0 # turn off rate limiting -</textarea> -<p>The following sample is used activate message annotation and thus trusted properties -on the system log socket. -<textarea rows="4" cols="70">$ModLoad imuxsock # needs to be done just once - -$SystemLogSocketAnnotate on -</textarea> -<p>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>] -[<a href="manual.html">manual index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> -<p><font size="2">This documentation is part of the -<a href="http://www.rsyslog.com/">rsyslog</a> -project.<br> -Copyright © 2008-2013 by <a href="http://www.gerhards.net/rainer">Rainer -Gerhards</a> and -<a href="http://www.adiscon.com/">Adiscon</a>. -Released under the GNU GPL version 3 or higher.</font></p> -</body></html> diff --git a/doc/index.html b/doc/index.html deleted file mode 100644 index d753e2e..0000000 --- a/doc/index.html +++ /dev/null @@ -1,31 +0,0 @@ -<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> -<html><head><title>Welcome to rsyslog</title></head> -<body> -<h1>Welcome to rsyslog</h1> -<p><b><a href="http://www.rsyslog.com/">Rsyslog</a> -is an enhanced syslogd suitable both for small systems as -well as large enterprises.</b> -<p>This page provide a few quick pointers which hopefully make your -experience with rsyslog a pleasant one. These are -<ul> -<li><b>Most importantly, the <a href="manual.html">rsyslog manual</a></b> - this points to locally -installed documentation which exactly matches the version you have installed. -It is highly suggested to at least briefly look over these files. -<li>The <a href="http://www.rsyslog.com">rsyslog web site</a> which offers -probably every information you'll ever need (ok, just kidding...). -<li>The <a href="http://www.rsyslog.com/status">project status page</a> provides -information on current releases -<li>and the <a href="troubleshoot.html">troubleshooting guide</a> hopefully helps if -things do not immediately work out -</ul> -<p>In general, rsyslog supports plain old syslog.conf format, except that the -config file is now called rsyslog.conf. This should help you get started -quickly. -To do the really cool things, though, -you need to learn a bit about its new features. -The man pages offer a bare minimum of information (and are still quite long). Read the -<a href="manual.html">html documentation</a> instead. -When you change the configuration, remember to restart rsyslogd, because otherwise -it will not use your new settings (and you'll end up totally puzzled why this great -config of yours does not even work a bit...;)) -</body></html> diff --git a/doc/install.html b/doc/install.html deleted file mode 100644 index 48b7f64..0000000 --- a/doc/install.html +++ /dev/null @@ -1,180 +0,0 @@ -<html><head> -<title>A guide on HOWTO install rsyslog</title> -<meta name="KEYWORDS" content="syslog encryption, rsyslog, stunnel, secure syslog, tcp, reliable, howto, ssl"> -</head> -<body> -<h1>HOWTO install rsyslog</h1> - <P><small><i>Written by - <a href="http://www.adiscon.com/en/people/rainer-gerhards.php">Rainer - Gerhards</a></i></small></P> -<h2>Abstract</h2> -<p><i><b>In this paper, I describe how to install -<a href="http://www.rsyslog.com/">rsyslog</a>.</b> It is intentionally a brief -step-by-step guide, targeted to those who want to quickly get it up and running. -For more elaborate information, please consult the rest of the -<a href="manual.html">manual set</a>.</i></p> -<h2>How to make your life easier...</h2> -<p>Some folks have thankfully created <a href="rsyslog_packages.html"> -RPMs/packages for rsyslog</a>. If you use them, you can spare yourself many of -the steps below. This is highly recommended if there is a package for your -distribution available.</p> -<h2>Steps To Do</h2> -<p>Rsyslog does currently only have very limited availability as a package (if -you volunteer to create one, <a href="mailto:rgerhards@adiscon.com">drop me a -line</a>). Thus, this guide focuses on installing from the source, which -thankfully is <b>quite easy</b>.</p> -<h3>Step 1 - Download Software</h3> -<p>For obvious reasons, you need to download rsyslog. Here, I assume that you -use a distribution tarball. If you would like to use a version directly from -the repository, see <a href="build_from_repo.html">build rsyslog from repository</a> -instead. -<p>Load the most recent build -from <a href="http://www.rsyslog.com/downloads">http://www.rsyslog.com/downloads</a>. -Extract the software with "tar xzf -nameOfDownloadSet-". This will create a new -subdirectory rsyslog-version in the current working directory. CD into that. </p> -<p>Depending on your system configuration, you also need to install some build -tools, most importantly make, the gcc compiler and the MySQL development system -(if you intend to use MySQL - the package is often named "mysql-dev"). On many systems, these things should already be -present. If you don't know exactly, simply skip this step for now and see if -nice error messages pop up during the compile process. If they do, you can still -install the missing build environment tools. So this is nothing that you need to -look at very carefully.</p> -<h3>Step 2 - Run ./configure</h3> -<p>Run ./configure to adopt rsyslog to your environment. While doing so, you can -also enable options. Configure will display selected options when it is -finished. For example, to enable MySQL support, run</p> -<p>./configure --enable-mysql</p> -<p>Please note that MySQL support by default is NOT disabled.</p> -<h3>Step 3 - Compile</h3> -<p>That is easy. Just type "make" and let the compiler work. On any recent -system, that should be a very quick task, on many systems just a matter of a few -seconds. If an error message comes up, most probably a part of your build -environment is not installed. Check with step 1 in those cases. </p> -<h3>Step 4 - Install</h3> -<p>Again, that is quite easy. All it takes is a "make install". That will copy -the rsyslogd and the man pages to the relevant directories.</p> -<h3>Step 5 - Configure rsyslogd</h3> -<p>In this step, you tell rsyslogd what to do with received messages. If you are -upgrading from stock syslogd, /etc/syslog.conf is probably a good starting -point. Rsyslogd understands stock syslogd syntax, so you can simply copy over -/etc/syslog.conf to /etc/rsyslog.conf. Note since version 3 rsyslog requires -to load plug-in modules to perform useful work (more about -<a href="v3compatibility.html">compatibilty notes v3</a>). To load the most common plug-ins, -add the following to the top of rsyslog.conf:</p> -<p> -$ModLoad immark # provides --MARK-- message capability <br /> -$ModLoad imudp # provides UDP syslog reception <br /> -$ModLoad imtcp # provides TCP syslog reception and GSS-API (if compiled to support it) <br /> -$ModLoad imuxsock # provides support for local system logging (e.g. via logger command) <br /> -$ModLoad imklog # provides kernel logging support (previously done by rklogd) <br /> -</p> -Change rsyslog.conf for any further -enhancements you would like to see. For example, you can add database writing as -outlined in the paper "<a href="rsyslog_mysql.html">Writing syslog Data to MySQL</a>" -(remember you need to enable MySQL support during step 2 if you want to do -that!).</p> -<h3>Step 6 - Disable stock syslogd</h3> -<p>In almost all cases, there already is stock syslogd installed. Because both -it and rsyslogd listen to the same sockets, they can NOT be run concurrently. So -you need to disable the stock syslogd. To do this, you typically must change -your rc.d startup scripts.</p> -<p>For example, under <a href="http://www.debian.org/">Debian</a> this must be -done as follows: The default runlevel is 2. We modify the init scripts for -runlevel 2 - in practice, you need to do this for all run levels you will ever -use (which probably means all). Under /etc/rc2.d there is a S10sysklogd script (actually -a symlink). Change the name to _S10sysklogd (this keeps the symlink in place, -but will prevent further execution - effectively disabling it).</p> -<h3>Step 7 - Enable rsyslogd Autostart</h3> -<p>This step is very close to step 3. Now, we want to enable rsyslogd to start -automatically. The rsyslog package contains a (currently small) number of -startup scripts. They are inside the distro-specific directory (e.g. debian). If -there is nothing for your operating system, you can simply copy the stock -syslogd startup script and make the minor modifications to run rsyslogd (the -samples should be of help if you intend to do this).</p> -<p>In our Debian example, the actual scripts are stored in /etc/init.d. Copy the -standard script to that location. Then, you need to add a symlink to it in the -respective rc.d directory. In our sample, we modify rc2.d, and can do this via -the command "ln -s ../init.d/rsyslogd S10rsyslogd". Please note that the S10 -prefix tells the system to start rsyslogd at the same time stock sysklogd was -started.</p> -<p><b>Important:</b> if you use the database functionality, you should make sure -that MySQL starts before rsyslogd. If it starts later, you will receive an error -message during each restart (this might be acceptable to you). To do so, either -move MySQL's start order before rsyslogd or rsyslogd's after MySQL.</p> -<h3>Step 8 - Check daily cron scripts</h3> -<p>Most distributions come pre-configured with some daily scripts for log -rotation. As long as you use the same log file names, the log rotation scripts -will probably work quite well. There is one caveat, though. The scripts need to -tell syslogd that the files have been rotated. To do this, they typically have a -part using syslogd's init script to do that. Obviously, the default scripts do -not know about rsyslogd, so they manipulate syslogd. If that happens, in most -cases an additional instance of stock syslogd is started (in almost all cases, -this was not functional, but it is at least distracting). It also means that -rsyslogd is not properly told about the log rotation, which will lead it to -continue to write to the now-rotated files.</p> -<p>So you need to fix these scripts. See your distro-specific documentation how -they are located. Under most Linuxes, the primary script to modify is /etc/cron.daily/sysklogd. -Watch for a comment "Restart syslogd" (usually at the very end of the file). The -restart command must be changed to use rsyslogd's rc script.</p> -<p>Also, if you use klogd together with rsyslogd (under most Linuxes you will do -that), you need to make sure that klogd is restarted after rsyslogd is restarted. -So it might be a good idea to put a klogd reload-or-restart command right after -the rsyslogd command in your daily script. This can save you lots of troubles.</p> -<h3>Done</h3> -<p>This concludes the steps necessary to install rsyslogd. Of course, it is -always a good idea to test everything thoroughly. At a minimalist level, you -should do a reboot and after that check if everything has come up correctly. Pay -attention not only to running processes, but also check if the log files (or the -database) are correctly being populated.</p> -<p>If rsyslogd encounters any serious errors during startup, you should be able -to see them at least on the system console. They might not be in log file, as -errors might occur before the log file rules are in place. So it is always a -good idea to check system console output when things don't go smooth. In some -rare cases, enabling debug logging (-d option) in rsyslogd can be helpful. If -all fails, go to <a href="http://www.rsyslog.com">www.rsyslog.com</a> and check -the forum or mailing list for help with your issue.</p> -<h2>Housekeeping stuff</h2> -<p>This section and its subsections contain all these nice things that you -usually need to read only if you are really curios ;)</p> -<h3>Feedback requested</h3> -<P>I would appreciate feedback on this tutorial. It is still in its infancy, so additional ideas, -comments or bug sighting reports are very welcome. Please -<a href="mailto:rgerhards@adiscon.com">let me know</a> about them.</P> -<h3>Revision History</h3> -<ul> - <li>2005-08-08 * - <a href="http://www.adiscon.com/en/people/rainer-gerhards.php">Rainer Gerhards</a> * Initial - version created</li> - <li>2005-08-09 * - <a href="http://www.adiscon.com/en/people/rainer-gerhards.php">Rainer Gerhards</a> - * updated to include distro-specific directories, which are now mandatory</li> - <li>2005-09-06 * - <a href="http://www.adiscon.com/en/people/rainer-gerhards.php">Rainer Gerhards</a> - * added information on log rotation scripts</li> - <li>2007-07-13 * - <a href="http://www.adiscon.com/en/people/rainer-gerhards.php">Rainer Gerhards</a> - * updated to new autotools-based build system</li> - <li>2008-10-01 * - <a href="http://www.adiscon.com/en/people/rainer-gerhards.php">Rainer Gerhards</a> - * added info on building from source repository</li> -</ul> -<h3>Copyright</h3> -<p>Copyright © 2005-2008 -<a href="http://www.adiscon.com/en/people/rainer-gerhards.php">Rainer Gerhards</a> and -<a href="http://www.adiscon.com/en/">Adiscon</a>.</p> -<p> Permission is granted to copy, distribute and/or modify this document - under the terms of the GNU Free Documentation License, Version 1.2 - or any later version published by the Free Software Foundation; - with no Invariant Sections, no Front-Cover Texts, and no Back-Cover - Texts. A copy of the license can be viewed at -<a href="http://www.gnu.org/copyleft/fdl.html"> -http://www.gnu.org/copyleft/fdl.html</a>.</p> -<p>[<a href="manual.html">manual index</a>] -[<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> -<p><font size="2">This documentation is part of the -<a href="http://www.rsyslog.com/">rsyslog</a> project.<br> -Copyright © 2008 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and -<a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL -version 1.2 or higher.</font></p> -</body> -</html> diff --git a/doc/ipv6.html b/doc/ipv6.html deleted file mode 100644 index 67c8e1f..0000000 --- a/doc/ipv6.html +++ /dev/null @@ -1,48 +0,0 @@ -<html> -<head> -<title>Notes on IPv6 Handling in Rsyslog</title> -</head> -<body> -<h1>Notes on IPv6 Handling in Rsyslog</h1> -<p><b>Rsyslog fully* supports sending and receiving syslog messages via both -IPv4 and IPv6.</b> IPv6 is natively supported for both UDP and TCP. However, -there are some options that control handling of IPv6 operations. I thought it is -is a good idea to elaborate a little about them, so that you can probably find -your way somewhat easier.</p> -<p>First of all, you can restrict rsyslog to using IPv4 or IPv6 addresses only -by specifying the -4 or -6 command line option (now guess which one does -what...). If you do not provide any command line option, rsyslog uses IPv4 and -IPv6 addresses concurrently. In practice, that means the listener binds to both -addresses (provided they are configured). When sending syslog messages, rsyslog -uses IPv4 addresses when the receiver can be reached via IPv4 and IPv6 addresses -if it can be reached via IPv6. If it can be reached on either IPv4 and v6, -rsyslog leaves the choice to the socket layer. The important point to know is -that it uses whatever connectivity is available to reach the destination.</p> -<p><b>There is one subtle difference between UDP and TCP.</b> With the new -IPv4/v6 ignorant code, rsyslog has potentially different ways to reach -destinations. The socket layer returns all of these paths in a sorted array. -For TCP, rsyslog loops through this array until a successful TCP connect can be -made. If that happens, the other addresses are ignored and messages are sent via -the successfully-connected socket.</p> -<p>For UDP, there is no such definite success indicator. Sure, the socket layer -may detect some errors, but it may not notice other errors (due to the -unreliable nature of UDP). By default, the UDP sender also tries one entry after -the other in the sorted array of destination addresses. When a send fails, the -next address is tried. When the send function finally succeeds, rsyslogd assumes -the UDP packet has reached its final destination. However, if rsyslogd is -started with the "-A" (capital A!) was given on the command line, rsyslogd will -continue to send messages until the end of the destination address array is -reached. This may result in duplicate messages, but it also provides some -additional reliability in case a message could not be received. You need to be -sure about the implications before applying this option. In general, it is NOT -recommended to use the -A option.</p> -<p><i><b>*</b>rsyslog does not support RFC 3195 over IPv6. The reason is that -the RFC 3195 library, <a href="http://www.liblogging.org/">liblogging</a>, -supports IPv4, only. Currently, there are no plans to update either rsyslog to -another RFC 3195 stack or update liblogging. There is simply no demand for 3195 -solutions.</i></p> -<p><font size="2">Last Updated: 2007-07-02<br> -Copyright © 2007 by Rainer Gerhards, released under the GNU GPL V2 or later.</font></p> - -</body> -</html> diff --git a/doc/licensing.html b/doc/licensing.html deleted file mode 100644 index 93a5093..0000000 --- a/doc/licensing.html +++ /dev/null @@ -1,72 +0,0 @@ -<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> -<html><head> -<title>rsyslog licensing</title> - -</head> -<body> -<h1>rsyslog licensing</h1> -<p><b>Most important things first: if you intend to use rsyslog inside a GPLv3 compatible project, you are free to do so.</b> You don't even need to continue reading. -If you intend to use rsyslog inside a non-GPLv3 -compatible project, rsyslog offers you some liberties to do that, too. However, you then need -to study the licensing details in depth. -<p>The project hopes this is a good compromise, which also gives a boost to fellow free -software developers who release under GPLv3. -<p>And now on to the dirty and boring license details, still on a executive summary level. For the -real details, check source files and the files COPYING and COPYING.LESSER inside the distribution. -<p>The rsyslog package contains several components: -<ul> -<li>the rsyslog core programs (like rsyslogd) -<li>plugins (like imklog, omrelp, ...) -<li>the rsyslog runtime library -</ul> -<p>Each of these components can be thought of as individual projects. In fact, some of the -plugins have different main authors than the rest of the rsyslog package. All of these -components are currently put together into a single "rsyslog" package (tarball) for -convinience: this makes it easier to distribute a consistent version where everything -is included (and in the right versions) to build a full system. Platform package -maintainers in general take the overall package and split off the individual components, so that -users can install only what they need. In source installations, this can be done via the -proper ./configure switches. -<p>However, while it is convenient to package all parts in a single tarball, it does not -imply all of them are necessarily covered by the same license. Traditionally, GPL licenses -are used for rsyslog, because the project would like to provide free software. GPLv3 has been -used since around 2008 to help fight for our freedom. All rsyslog core programs are -released under GPLv3. But, from the beginning on, plugins were separate projects and we did not -impose and license restrictions on them. So even though all plugins that currently ship with -the rsyslog package are also placed under GPLv3, this can not taken for granted. You need -to check each plugins license terms if in question - this is especially important for -plugins that do NOT ship as part of the rsyslog tarball. -<p>In order to make rsyslog technology available to a broader range of applications, -the rsyslog runtime is, at least partly, licensed under LGPL. If in doubt, check the source file -licensing comments. As of now, the following files are licensed under LGPL: -<ul> -<li>queue.c/.h -<li>wti.c/.h -<li>wtp.c/.h -<li>vm.c/.h -<li>vmop.c/.h -<li>vmprg.c/.h -<li>vmstk.c/.h -<li>expr.c/.h -<li>sysvar.c/.h -<li>ctok.c/.h -<li>ctok_token.c/.h -<li>regexp.c/.h -<li>sync.c/.h -<li>stream.c/.h -<li>var.c/.h -</ul> -This list will change as time of the runtime modularization. At some point in the future, there will -be a well-designed set of files inside a runtime library branch and all of these will be LGPL. Some -select extras will probably still be covered by GPL. We are following a similar licensing -model in GnuTLS, which makes effort to reserve some functionality exclusively to open source -projects. -<p>[<a href="manual.html">manual index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> -<p><font size="2">This documentation is part of the -<a href="http://www.rsyslog.com/">rsyslog</a> -project.<br> -Copyright © 2008 by <a href="http://www.gerhards.net/rainer">Rainer -Gerhards</a> and -<a href="http://www.adiscon.com/">Adiscon</a>. Last Update: 2008-04-15. -Released under the GNU GPL version 3 or higher.</font></p> -</body></html> diff --git a/doc/log_rotation_fix_size.html b/doc/log_rotation_fix_size.html deleted file mode 100644 index 51edf03..0000000 --- a/doc/log_rotation_fix_size.html +++ /dev/null @@ -1,69 +0,0 @@ -<html><head> -<title>Keep the log file size accurate with log rotation</title> -<meta name="KEYWORDS" content="log rotation, howto, guide, fixed-size log"> -</head> -<body> -<a href="rsyslog_conf_output.html">back</a> - -<h1>Log rotation with rsyslog</h1> - <P><small><i>Written by - Michael Meckelein</i></small></P> -<h2>Situation</h2> - -<p>Your environment does not allow you to store tons of logs? -You have limited disc space available for logging, for example -you want to log to a 124 MB RAM usb stick? Or you do not want to -keep all the logs for months, logs from the last days is sufficient? -Think about log rotation.</p> - -<h2>Log rotation based on a fixed log size</h2> - -<p>This small but hopefully useful article will show you the way -to keep your logs at a given size. The following sample is based on -rsyslog illustrating a simple but effective log rotation with a -maximum size condition.</p> - -<h2>Use Output Channels for fixed-length syslog files</h2> - -<p>Lets assume you do not want to spend more than 100 MB hard -disc space for you logs. With rsyslog you can configure Output -Channels to achieve this. Putting the following directive</p> - -<p><pre> -# start log rotation via outchannel -# outchannel definiation -$outchannel log_rotation,/var/log/log_rotation.log, 52428800,/home/me/./log_rotation_script -# activate the channel and log everything to it -*.* :omfile:$log_rotation -# end log rotation via outchannel -</pre></p> - -<p>to ryslog.conf instruct rsyslog to log everything to the destination file -'/var/log/log_rotation.log' until the give file size of 50 MB is reached. If -the max file size is reached it will perform an action. In our case it executes -the script /home/me/log_rotation_script which contains a single command:</p> - -<p><pre> -mv -f /var/log/log_rotation.log /var/log/log_rotation.log.1 -</p></pre> - -<p>This moves the original log to a kind of backup log file. -After the action was successfully performed rsyslog creates a new /var/log/log_rotation.log -file and fill it up with new logs. So the latest logs are always in log_roatation.log.</p> - -<h2>Conclusion</h2> - -<p>With this approach two files for logging are used, each with a maximum size of 50 MB. So -we can say we have successfully configured a log rotation which satisfies our requirement. -We keep the logs at a fixed-size level of100 MB.</p> -<p>[<a href="manual.html">manual index</a>] -[<a href="rsyslog_conf.html">rsyslog.conf</a>] -[<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> -<p><font size="2">This documentation is part of the -<a href="http://www.rsyslog.com/">rsyslog</a> project.<br> -Copyright © 2008 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and -<a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL -version 2 or higher.</font></p> - -</body> -</html> diff --git a/doc/lookup_tables.html b/doc/lookup_tables.html deleted file mode 100644 index d72810f..0000000 --- a/doc/lookup_tables.html +++ /dev/null @@ -1,205 +0,0 @@ -<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> -<html><head> -<title>Lookup Tables</title> -</head> - -<body> -<h1>Lookup Tables</h1> - -<p><b><font color="red">NOTE: this is</font> proposed functionality, which is -<font color="red">NOT YET IMPLEMENTED</font>!</b> - -<p><b>Lookup tables</a> are a powerful construct -to obtain "class" information based on message content (e.g. to build -log file names for different server types, departments or remote -offices).</b> -<p>The base idea is to use a message variable as an index into a table which then -returns another value. For example, $fromhost-ip could be used as an index, with -the table value representing the type of server or the department or remote office -it is located in. A main point with lookup tables is that the lookup is very fast. -So while lookup tables can be emulated with if-elseif constructs, they are generally -much faster. Also, it is possible to reload lookup tables during rsyslog runtime without -the need for a full restart. -<p>The lookup tables itself exists in a separate configuration file (one per table). This -file is loaded on rsyslog startup and when a reload is requested. -<p>There are different types of lookup tables: -<ul> -<li><b>string</b> - the value to be looked up is an arbitrary string. Only exact -some strings match. -<li><b>array</b> - the value to be looked up is an integer number from a consequtive set. -The set does not need to start at zero or one, but there must be no number missing. So, for example -5,6,7,8,9 would be a valid set of index values, while 1,2,4,5 would not be (due to missing -2). -A match happens if the requested number is present. -<li><b>sparseArray</b> - the value to be looked up is an integer value, but there may -be gaps inside the set of values (usually there are large gaps). A typical use case would -be the matching of IPv4 address information. A match happens on the first value that is -less than or equal to the requested value. -</ul> -<p>Note that index integer numbers are represented by unsigned 32 bits. -<p>Lookup tables can be access via the lookup() built-in function. The core idea is to -set a local variable to the lookup result and later on use that local variable in templates. -<p>More details on usage now follow. -<h2>Lookup Table File Format</h2> -<p>Lookup table files contain a single JSON object. This object contains of a header and a -table part. -<h3>Header</h3> -<p>The header is the top-level json. It has paramters "version", "nomatch", and "type". -The version parameter -must be given and must always be one for this version of rsyslog. The nomatch -parameter is optional. If specified, it contains the value to be used if lookup() -is provided an index value for which no entry exists. The default for -"nomatch" is the empty string. Type specifies the type of lookup to be done. -<h3>Table</h3> -This must be an array of elements, even if only a single value exists (for obvious -reasons, we do not expect this to occur often). Each array element must contain two -fields "index" and "value". -<h3>Example</h3> -<p>This is a sample of how an ip-to-office mapping may look like: -<pre> -{ "version":1, "nomatch":"unk", "type":"string", - "table":[ {"index":"10.0.1.1", "value":"A" }, - {"index":"10.0.1.2", "value":"A" }, - {"index":"10.0.1.3", "value":"A" }, - {"index":"10.0.2.1", "value":"B" }, - {"index":"10.0.2.2", "value":"B" }, - {"index":"10.0.2.3", "value":"B" } - ] -} -</pre> -Note: if a different IP comes in, the value "unk" -is returend thanks to the nomatch parameter in -the first line. -<p> -<h2>RainerScript Statements</h2> -<h3>lookup_table() Object</h3> -<p>This statement defines and intially loads a lookup table. Its format is -as follows: -<pre> -lookup_table(name="name" file="/path/to/file" reloadOnHUP="on|off") -</pre> -<h4>Parameters</h4> -<ul> - <li><b>name</b> (mandatory)<br> - Defines the name of lookup table for further reference - inside the configuration. Names must be unique. Note that - it is possible, though not advisible, to have different - names for the same file. - <li><b>file</b> (mandatory)<br> - Specifies the full path for the lookup table file. This file - must be readable for the user rsyslog is run under (important - when dropping privileges). It must point to a valid lookup - table file as described above. - <li><b>reloadOnHUP</b> (optional, default "on")<br> - Specifies if the table shall automatically be reloaded - as part of HUP processing. For static tables, the - default is "off" and specifying "on" triggers an - error message. Note that the default of "on" may be - somewhat suboptimal performance-wise, but probably - is what the user intuitively expects. Turn it off - if you know that you do not need the automatic - reload capability. -</ul> - -<h3>lookup() Function</h3> -<p>This function is used to actually do the table lookup. Format: -<pre> -lookup_table("name", indexvalue) -</pre> -<h4>Parameters</h4> -<ul> - <li><b>return value</b><br> - The function returns the string that is associated with the - given indexvalue. If the indexvalue is not present inside the - lookup table, the "nomatch" string is returned (or an empty string - if it is not defined). - <li><b>name</b> (constant string)<br> - The lookup table to be used. Note that this must be specificed as a - constant. In theory, variable table names could be made possible, but - their runtime behaviour is not as good as for static names, and we do - not (yet) see good use cases where dynamic table names could be useful. - <li><b>indexvalue</b> (expression)<br> - The value to be looked up. While this is an arbitrary RainerScript expression, - it's final value is always converted to a string in order to conduct - the lookup. For example, "lookup(table, 3+4)" would be exactly the same - as "lookup(table, "7")". In most cases, indexvalue will probably be - a single variable, but it could also be the result of all RainerScript-supported - expression types (like string concatenation or substring extraction). - Valid samples are "lookup(name, $fromhost-ip & $hostname)" or - "lookup(name, substr($fromhost-ip, 0, 5))" as well as of course the - usual "lookup(table, $fromhost-ip)". -</ul> - - -<h3>load_lookup_table Statement</h3> - -<p><b>Note: in the final implementation, this MAY be implemented as an action. -This is a low-level decesion that must be made during the detail development -process. Parameters and semantics will remain the same of this happens.</b> - -<p>This statement is used to reload a lookup table. It will fail if -the table is static. While this statement is executed, lookups to this table -are temporarily blocked. So for large tables, there may be a slight performance -hit during the load phase. It is assume that always a triggering condition -is used to load the table. -<pre> -load_lookup_table(name="name" errOnFail="on|off" valueOnFail="value") -</pre> -<h4>Parameters</h4> -<ul> - <li><b>name</b> (string)<br> - The lookup table to be used. - <li><b>errOnFail</b> (boolean, default "on")<br> - Specifies whether or not an error message is to be emitted if - there are any problems reloading the lookup table. - <li><b>valueOnFail</b> (optional, string)<br> - This parameter affects processing if the lookup table cannot - be loaded for some reason: If the parameter is not present, - the previous table will be kept in use. If the parameter is - given, the previous table will no longer be used, and instead - an empty table be with nomath=valueOnFail be generated. In short, - that means when the parameter is set and the reload fails, - all matches will always return what is specified in valueOnFail. -</ul> - -<h3>Usage example</h3> -<p>For clarity, we show only those parts of rsyslog.conf that affect -lookup tables. We use the remote office example that an example lookup -table file is given above for. -<pre> -lookup_table(name="ip2office" file="/path/to/ipoffice.lu" - reloadOnHUP="off") - - -template(name="depfile" type="string" - string="/var/log/%$usr.dep%/messages") - -set $usr.dep = lookup("ip2office", $fromhost-ip); -action(type="omfile" dynfile="depfile") - -# support for reload "commands" -if $fromhost-ip == "10.0.1.123" - and $msg contains "reload office lookup table" - then - load_lookup_table(name="ip2office" errOnFail="on") -</pre> - -<p>Note: for performance reasons, it makes sense to put the reload command into -a dedicated ruleset, bound to a specific listener - which than should also -be sufficiently secured, e.g. via TLS mutual auth. - -<h2>Implementation Details</h2> -<p>The lookup table functionality is implemented via highly efficient algorithms. -The string lookup is based on a parse tree and has O(1) time complexity. The array -lookup is also O(1). In case of sparseArray, we have O(log n). -<p>To preserve space and, more important, increase cache hit performance, equal -data values are only stored once, no matter how often a lookup index points to them. -<p>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>] -[<a href="manual.html">manual index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> -<p><font size="2">This documentation is part of the -<a href="http://www.rsyslog.com/">rsyslog</a> project.<br> -Copyright © 2013 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and -<a href="http://www.adiscon.com/">Adiscon</a>. -Released under the GNU GPL version 3 or higher.</font></p> -</body> -</html> diff --git a/doc/manual.html b/doc/manual.html deleted file mode 100644 index 5a74c8c..0000000 --- a/doc/manual.html +++ /dev/null @@ -1,123 +0,0 @@ -<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> -<html><head><title>rsyslog documentation</title></head> -<body> -<h1>RSyslog - Documentation</h1> -<p><b><a href="http://www.rsyslog.com/">Rsyslog</a> -is an enhanced syslogd -supporting, among others, <a href="rsyslog_mysql.html">MySQL</a>, -PostgreSQL, <a href="http://wiki.rsyslog.com/index.php/FailoverSyslogServer">failover -log destinations</a>, syslog/tcp, fine grain output format -control, high precision timestamps, queued operations and the ability to filter on any message -part.</b> -It is quite compatible to stock sysklogd and can be used as a drop-in -replacement. Its <a href="features.html"> -advanced features</a> make it suitable for enterprise-class, <a href="rsyslog_tls.html">encryption protected syslog</a> -relay chains while at the same time being very easy to setup for the -novice user. And as we know what enterprise users really need, there are -also <a href="http://www.rsyslog.com/professional-services"> rsyslog -professional services</a> available directly from the source!</p> -<p><b>Please visit the <a href="http://www.rsyslog.com/sponsors">rsyslog sponsor's page</a> -to honor the project sponsors or become one yourself!</b> We are very grateful for any help towards the -project goals.</p> -<p><b>This documentation is for version 7.3.14 (beta branch) of rsyslog.</b> -Visit the <i><a href="http://www.rsyslog.com/status">rsyslog status page</a></i></b> -to obtain current version information and project status. -</p><p><b>If you like rsyslog, you might -want to lend us a helping hand. </b>It doesn't require a lot of -time - even a single mouse click helps. Learn <a href="how2help.html">how to help the rsyslog project</a>. -Due to popular demand, there is now a <a href="rsyslog_ng_comparison.html">side-by-side comparison -between rsyslog and syslog-ng</a>.</p> -<p>If you are upgrading from rsyslog v2 or stock sysklogd, -<a href="v3compatibility.html">be sure to read the rsyslog v3 compatibility notes</a>, -and if you are upgrading from v3, read the -<a href="v4compatibility.html">rsyslog v4 compatibility notes</a>, -if you upgrade from v4, read the -<a href="v5compatibility.html">rsyslog v5 compatibility notes</a>, and -if you upgrade from v5, read the -<a href="v6compatibility.html">rsyslog v6 compatibility notes</a>. -if you upgrade from v6, read the -<a href="v7compatibility.html">rsyslog v7 compatibility notes</a>. -<p>Rsyslog will work even -if you do not read the doc, but doing so will definitely improve your experience.</p> -<p><b>Follow the links below for the</b></p> -<ul> -<li><a href="troubleshoot.html">troubleshooting rsyslog problems</a></li> -<li><a href="rsyslog_conf.html">configuration file format (rsyslog.conf)</a></li> -<li><a href="http://www.rsyslog.com/tool-regex">a regular expression checker/generator tool for rsyslog</a></li> -<li> <a href="property_replacer.html">property replacer, an important core component</a></li> -<li><a href="bugs.html">rsyslog bug list</a></li> -<li><a href="messageparser.html">understanding rsyslog message parsers</a></li> -<li><a href="generic_design.html">backgrounder on generic syslog application design</a></li> -<li><a href="modules.html">description of rsyslog modules</a></li> -<li><a href="rsyslog_packages.html">rsyslog packages</a></li> -</ul> -<p><b>To keep current on rsyslog development, follow -<a href="http://twitter.com/rgerhards">Rainer's twitter feed</a>.</b></p> -<p><b>We have some in-depth papers on</b></p> -<ul> -<li><a href="install.html">installing rsyslog</a></li> -<li><a href="build_from_repo.html">obtaining rsyslog from the source repository</a></li> -<li><a href="ipv6.html">rsyslog and IPv6</a> (which is fully supported)</li> -<li><a href="rsyslog_secure_tls.html">native TLS encryption for syslog</a></li> -<li><a href="multi_ruleset.html">using multiple rule sets in rsyslog</a></li> -<li><a href="rsyslog_stunnel.html">ssl-encrypting syslog with stunnel</a></li> -<li><a href="rsyslog_mysql.html">writing syslog messages to MySQL (and other databases as well)</a></li> -<li><a href="rsyslog_pgsql.html">writing syslog messages to PostgreSQL (and other databases as well)</a></li> -<li><a href="rsyslog_high_database_rate.html">writing massive amounts of syslog messages to a database</a></li> -<li><a href="rsyslog_reliable_forwarding.html">reliable forwarding to a remote server</a></li> -<li><a href="rsyslog_php_syslog_ng.html">using -php-syslog-ng with rsyslog</a></li> -<li><a href="rsyslog_recording_pri.html">recording -the syslog priority (severity and facility) to the log file</a></li> -<li><a href="http://www.rsyslog.com/Article19.phtml">preserving -syslog sender over NAT</a> (online only)</li> -<li><a href="gssapi.html">an overview and howto of rsyslog gssapi support</a></li> -<li><a href="debug.html">debug support in rsyslog</a></li> -<li>Developer Documentation - <ul> - <li><a href="build_from_repo.html">building rsyslog from the source repository</a></li> - <li><a href="dev_oplugins.html">writing rsyslog output plugins</a></li> - <li><a href="dev_queue.html">the rsyslog message queue object (developer's view)</a></li> - </ul></li> -</ul> -<p>Our <a href="history.html">rsyslog history</a> -page is for you if you would like to learn a little more -on why there is an rsyslog at all. If you are interested why you should -care about rsyslog at all, you may want to read Rainer's essay on "<a href="http://rgerhards.blogspot.com/2007/08/why-does-world-need-another-syslogd.html">why -the world needs another syslogd</a>".</p> -<p>Documentation is added continuously. Please note that the -documentation here -matches only the current version of rsyslog. If you use an older -version, be sure to use the doc that came with it.</p> -<p><b>You can also browse the following online resources:</b></p> -<ul> -<li>the <a href="http://wiki.rsyslog.com/">rsyslog -wiki</a>, a community resource which includes <a href="http://wiki.rsyslog.com/index.php/Configuration_Samples">rsyslog configuration examples</a></li> -<li><a href="http://www.rsyslog.com/module-Static_Docs-view-f-manual.html.phtml">rsyslog -online documentation (most current version only)</a></li> - -<li><a href="http://kb.monitorware.com/rsyslog-f40.html">rsyslog discussion forum - use this for technical support</a></li> -<li><a href="http://www.rsyslog.com/Topic8.phtml">rsyslog video tutorials</a></li> -<li><a href="http://www.rsyslog.com/Topic4.phtml">rsyslog change log</a></li> -<li><a href="http://www.rsyslog.com/Topic3.phtml">rsyslog FAQ</a></li> -<li><a href="http://www.monitorware.com/en/syslog-enabled-products/">syslog device configuration guide</a> (off-site)</li> -<li><a href="http://www.rsyslog.com/PNphpBB2.phtml">rsyslog discussion forum - use this for technical support</a></li> -<li><a href="http://kb.monitorware.com/rsyslog-f49.html">deutsches rsyslog forum</a> (forum in German language)</li> -</ul> -<p>And don't forget about the <a href="http://lists.adiscon.net/mailman/listinfo/rsyslog">rsyslog -mailing list</a>. If you are interested in the "backstage", you -may find -<a href="http://www.gerhards.net/rainer">Rainer</a>'s -<a href="http://blog.gerhards.net/">blog</a> an -interesting read (filter on syslog and rsyslog tags). -Or meet <a href="http://www.facebook.com/people/Rainer-Gerhards/1349393098">Rainer Gerhards at Facebook</a> -or <a href="https://plus.google.com/112402185904751517878/posts">Google+</a>. -If you would like to use rsyslog source code inside your open source project, you can do that without -any restriction as long as your license is GPLv3 compatible. If your license is incompatible to GPLv3, -you may even be still permitted to use rsyslog source code. However, then you need to look at the way -<a href="licensing.html">rsyslog is licensed</a>.</p> -<p>Feedback is always welcome, but if you have a support question, please do not -mail Rainer directly (<a href="free_support.html">why not?</a>) - use the -<a href="http://lists.adiscon.net/mailman/listinfo/rsyslog">rsyslog mailing list</a> -or <a href="http://kb.monitorware.com/rsyslog-f40.html">rsyslog formum</a> instead. -</body></html> diff --git a/doc/messageparser.html b/doc/messageparser.html deleted file mode 100644 index 370db59..0000000 --- a/doc/messageparser.html +++ /dev/null @@ -1,222 +0,0 @@ -<html> -<head> -<title>Message parsers in rsyslog</title> -</head> -<body> -<a href="manual.html">rsyslog documentation</a> - -<h1>Message parsers in rsyslog</h1> -<p><small><i>Written by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> -(2009-11-06)</i></small></p> -<h2>Intro</h2> -<p>Message parsers are a feature of rsyslog 5.3.4 and above. In this article, I describe what -message parsers are, what they can do and how they relate to the relevant standards. I will -also describe what you can not do with time. Finally, I give some advice on implementing your -own custom parser. - -<h2>What are message parsers?</h2> -<p>Well, the quick answer is that message parsers are the component of rsyslog that -parses the syslog message after it is being received. Prior to rsyslog 5.3.4, message parsers -where built in into the rsyslog core itself and could not be modified (other than by modifying -the rsyslog code). -<p>In 5.3.4, we changed that: message parsers are now loadable modules (just -like input and output modules). That means that new message parsers can be added without -modifying the rsyslog core, even without contributing something back to the -project. -<p>But that doesn't answer what a message parser really is. What does ist mean to "parse a -message" and, maybe more importantly, what is a message? To answer these questions correctly, -we need to dig down into the relevant standards. -<a href="http://tools.ietf.org/html/rfc5424">RFC5424</a> specifies a layered architecture -for the syslog protocol: -<p align="center"><img src="rfc5424layers.png" alt="RFC5424 syslog protocol layers"> -<p>For us important is the distinction between the syslog transport and the upper layers. -The transport layer specifies how a stream of messages is assembled at the sender side and -how this stream of messages is disassembled into the individual messages at the receiver -side. In networking terminology, this is called "framing". The core idea is that -each message is put into a so-called "frame", which then is transmitted over the communications -link. -<p>The framing used is depending on the protocol. For example, in UDP the "frame"-equivalent is -a packet that is being sent (this also means that no two messages can travel within a single -UDP packet). In "plain tcp syslog", the industry standard, LF is used as a frame delimiter -(which also means that no multi-line message can properly be transmitted, a "design" flaw -in plain tcp syslog). In <a href="http://tools.ietf.org/html/rfc5425">RFC5425</a> there is -a header in front of each frame that contains the size of the message. With this framing, -any message content can properly be transferred. -<p>And now comes the important part: <b>message parsers do NOT operate at the transport -layer</b>, they operate, as their name implies, on messages. So we can not use message -parsers to change the underlying framing. For example, if a sender splits (for whatever -reason) a single message into two and encapsulates these into two frames, there is no way -a message parser could undo that. -<p>A typical example may be a multi-line message: let's assume some originator has generated -a message for the format "A\nB" (where \n means LF). If that message is being transmitted -via plain tcp syslog, the frame delimiter is LF. So the sender will delimite the frame with -LF, but otherwise send the message unmodified onto the wire (because that is how things are --unfortunately- done in plain tcp syslog...). So wire will see "A\nB\n". When this -arrives at the receiver, the transport layer will undo the framing. When it sees the LF -after A, it thinks it finds a valid frame delimiter (in fact, this is the correct view!). So -the receive will extract one complete message A and one complete message B, not knowing -that they once were both part of a large multi-line message. These two messages are then -passed to the upper layers, where the message parsers receive them and extract information. -However, the message parsers never know (or even have a chance to see) that A and B -belonged together. Even further, in rsyslog there is no guarnatee that A will be parsed -before B - concurrent operations may cause the reverse order (and do so very validly). -<p>The important lesson is: <b>message parsers can not be used to fix a broken framing</b>. -You need a full protocol implementation to do that, what is the domain of input and -output modules. -<p>I have now told you what you can not do with message parsers. But what they are good for? -Thankfully, broken framing is not the primary problem of the syslog world. A wealth of different -formats is. Unfortunately, many real-world implementations violate the relevant standards -in one way or another. That makes it often very hard to extract meaningful information from -a message or to process messages from different sources by the same rules. In my article -<a href="syslog_parsing.html">syslog parsing in rsyslog</a> I have elaborated on all -the real-world evil that you can usually see. So I won't repeat that here. But in short, the -real problem is not the framing, but how to make malformed messages well-looking. -<p><b>This is what message parsers permit you to do: take a (well-known) malformed message, parse -it according to its semantics and generate perfectly valid internal message representations -from it.</b> So as long as messages are consistenly in the same wrong format (and they usually -are!), a message parser can look at that format, parse it, and make the message processable just -like it were wellformed in the first place. Plus, one can abuse the interface to do some other -"intersting" tricks, but that would take us to far. -<p>While this functionality may not sound exciting, it actually solves a very big issue (that you -only really understand if you have managed a system with various different syslog sources). -Note that we were often able to process malformed messages in the past with the help of the -property replacer and regular expressions. While this is nice, it has a performance hit. A -message parser is a C code, compiled to native language, and thus typically much faster than -any regular expression based method (depending, of course, on the quality of the implementation...). - -<h2>How are message parsers used?</h2> -<p>In a simlified view, rsyslog -<ol> -<li>first receives messages (via the input module), -<li><i>then parses them (at the message level!)</i> and -<li>then processes them (operating on the internal message representation). -</ol> -Message parsers are utilized in the second step (written in italics). -Thus, they take the raw message (NOT frame!) received from the remote system and create -the internal structure out of it that the other parts of rsyslog need in order to perform -their processing. Parsing is vital, because an unparsed message can not be processed in the -third stage, the actual application-level processing (like forwarding or writing to files). -<h3>Parser Chains and how they Operate</h3> -Rsyslog chains parsers together to provide flexibility. -A <b>parser chain</b> -contains all parsers that can potentially be used to parse a message. -It is assumed that there is some -way a parser can detect if the message it is being presented is supported by it. If so, the parser -will tell the rsyslog engine and parse the message. The rsyslog engine now calls each parser -inside the chain (in sequence!) until the first parser is able to parse the message. After one -parser has been found, the message is considered parsed and no others parsers are called on that -message. -<p>Side-note: this method implies there are some "not-so-dirty" tricks available to modify -the message by a parser module that declares itself as "unable to parse" but still does -some message modification. This was not a primary design goal, but may be utilized, and the -interface probably extended, to support generic filter modules. These would need to go -to the root of the parser chain. As mentioned, the current system already supports this. -<p>The position inside the parser chain can be thought of as a priority: parser sitting -earlier in the chain take precedence over those sitting later in it. So more specific -parser should go ealier in the chain. A good example of how this works is the default parser -set provided by rsyslog: rsyslog.rfc5424 and rsyslog.rfc3164, each one parses according to the -rfc that has named it. RFC5424 was designed to be distinguishable from RFC3164 message by the -sequence "1 " immediately after the so-called PRI-part (don't worry about these words, it is -sufficient if you understand there is a well-defined sequence used to indentify RFC5424 -messages). In contrary, RFC3164 actually permits everything as a valid message. Thus the -RFC3164 parser will always parse a message, sometimes with quite unexpected outcome (there is -a lot of guesswork involved in that parser, which unfortunately is unavoidable due to -existing techology limits). So the default parser chain is to try the RFC5424 parser first -and after it the RFC3164 parser. If we have a 5424-formatted message, that parser will -identify and parse it and the rsyslog engine will stop processing. But if we receive a -legacy syslog message, the RFC5424 will detect that it can not parse it, return this status -to the engine which then calls the next parser inside the chain. That usually happens to be -the RFC3164 parser, which will always process the message. But there could also be any other -parser inside the chain, and then each one would be called unless one that is able to parse -can be found. -<p>If we reversed the parser order, RFC5424 messages would incorrectly parsed. Why? Because the -RFC3164 parser will always parse every message, so if it were asked first, it would parse -(and misinterpret) the 5424-formatted message, return it did so and the rsyslog engine would -never call the 5424 parser. So oder of sequence is very important. -<p>What happens if no parser in the chain could parse a message? Well, then we could not -obtain the in-memory representation that is needed to further process the message. In that -case, rsyslog has no other choice than to discard the message. If it does so, it will emit -a warning message, but only in the first 1,000 incidents. This limit is a safety measure -against message-loops, which otherwise could quickly result from a parser chain -misconfiguration. <b>If you do not tolerate loss of unparsable messages, you must ensure -that each message can be parsed.</b> You can easily achive this by always using the -"rsyslog-rfc3164" parser as the <i>last</i> parser inside parser chains. That may result -in invalid parsing, but you will have a chance to see the invalid message (in debug mode, -a warning message will be written to the debug log each time a message is dropped due to -inability to parse it). -<h3>Where are parser chains used?</h3> -<p>We now know what parser chains are and how they operate. The question is now how many -parser chains can be active and how it is decicded which parser chain is used on which message. -This is controlled via <a href="multi_ruleset.html">rsyslog's rulesets</a>. In short, multiple -rulesets can be defined and there always exist at least one ruleset (for specifcs, follow -the <a href="multi_ruleset.html">link</a>). A parser chain is bound to a specific ruleset. -This is done by virtue of defining parsers via the -<a href="rsconf1_rulesetparser.html">$RulesetParser</a> configuration directive (for specifics, -see there). If no such directive is specified, the default parser chain is used. As of this -writing, the default parser chain always consists of "rsyslog.rfc5424", "rsyslog.rfc3164", in -that order. As soon as a parser is configured, the default list is cleared and the new parser -is added to the end of the (initially empty) ruleset's parser chain. -<p>The important point to know is that parser chains are defined on a per-ruleset basis. -<h3>Can I use different parser chains for different devices?</h3> -<p>The correct answer is: generally yes, but it depends. First of all, remember that input -modules (and specific listeners) may be bound to specific rulesets. As parser chains "reside" -in rulesets, binding to a ruleset also binds to the parser chain that is bound to that ruleset. -As a number one prequisite, the input module must support binding to different rulesets. Not -all do, but their number is growing. For example, the important -<a href="imudp.html">imudp</a> and <a href="imtcp.html">imtcp</a> input modules support -that functionality. Those that do not (for example <a href="im3195">im3195</a>) can only -utilize the default ruleset and thus the parser chain defined in that ruleset. -<p>If you do not know if the input module in question supports ruleset binding, check -its documentation page. Those that support it have the requiered directives. -<p>Note that it is currently under evaluation if rsyslog will support binding parser chains -to specific inputs directly, without depending on the ruleset. There are some concerns that -this may not be necessary but adds considerable complexity to the configuration. So this may -or may not be possible in the future. In any case, if we decide to add it, input modules -need to support it, so this functionality would require some time to implement. -<p>The coockbook recipe for using different parsers for different devices is given -as an actual in-depth example in the <a href="rscon1_rulesetsparser.html">$RulesetParser</a> -configuration directive doc page. In short, it is acomplished by defining specific rulesets -for the required parser chains, definining different listener ports for each of the devices -with different format and binding these listeners to the correct ruleset (and thus parser -chains). Using that approach, a variety of different message formats can be supported -via a single rsyslog instance. - -<h2>Which message parsers are available</h2> -<p>As of this writing, there exist only two message parsers, one for RFC5424 format and one for -legacy syslog (loosely described in -<a href="http://tools.ietf.org/html/rfc3164">RFC3164</a>). These parsers are built-in and -must not be explicitely loaded. However, message parsers can be added with relative ease -by anyone knowing to code in C. Then, they can be loaded via $ModLoad just like any -other loadable module. It is expected that the rsyslog project will be contributed additional -message parsers over time, so that at some point there hopefully is a rich choice of them -(I intend to add a browsable repository as soon as new parsers pop up). -<h3>How to write a message parser?</h3> -<p>As a prequisite, you need to know the exact format that the device is sending. Then, you need -moderate C coding skills, and a little bit of rsyslog internals. I guess the rsyslog specific part -should not be that hard, as almost all information can be gained from the existing parsers. They -are rather simple in structure and can be found under the "./tools" directory. They are named -pmrfc3164.c and pmrfc5424.c. You need to follow the usual loadable module guidelines. -It is my expectation that writing a parser should typically not take longer than a single -day, with maybe a day more to get aquainted with rsyslog. Of course, I am not sure if the number -is actually right. -<p>If you can not program or have no time to do it, Adiscon can also write a message parser -for you as -part of the <a href="http://www.rsyslog/professional-services">rsyslog professional services -offering</a>. -<h2>Conclusion</h2> -<p>Malformed syslog messages are a pain and unfortunately often seen in practice. Message parsers -provide a fast and efficient solution for this problem. Different parsers can be defined for -different devices, and they all convert message information into rsyslog's well-defined -internal format. Message parsers were first introduced in rsyslog 5.3.4 and also offer -some interesting ideas that may be explored in the future - up to full message normalization -capabilities. It is strongly recommended that anyone with a heterogenous environment take -a look at message parser capabilities. - -<p>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>] [<a href="manual.html">manual -index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> -<p><font size="2">This documentation is part of the -<a href="http://www.rsyslog.com/">rsyslog</a> project.<br> -Copyright © 2009 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and -<a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL version 3 or higher.</font></p> -</body> -</html> diff --git a/doc/mmanon.html b/doc/mmanon.html deleted file mode 100644 index 16065a1..0000000 --- a/doc/mmanon.html +++ /dev/null @@ -1,119 +0,0 @@ -<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> -<html><head> -<meta http-equiv="Content-Language" content="en"> -<title>IP Address Anonimization Module (mmanon)</title></head> - -<body> -<a href="rsyslog_conf_modules.html">back</a> - -<h1>IP Address Anonimization Module (mmanon)</h1> -<p><b>Module Name: mmanon</b></p> -<p><b>Author: </b>Rainer Gerhards <rgerhards@adiscon.com></p> -<p><b>Available since</b>: 7.3.7</p> -<p><b>Description</b>:</p> -<p>The mmanon module permits to anonymize IP addresses. It is a message -modification module that actually changes the IP address inside the message, -so after calling mmanon, the original message can no longer be obtained. -Note that anonymization will break digital signatures on the message, if -they exist. -<p><i>How are IP-Addresses defined?</i> -<p>We assume that an IP address consists of four octets in dotted notation, -where each of the octets has a value between 0 and 255, inclusively. After -the last octet, there must be either a space or a colon. So, for example, -"1.2.3.4 Test" and "1.2.3.4:514 Test" are detected as containing valid IP -addresses, whereas this is not the case for "1.2.300.4 Test" or -"1.2.3.4-Test". The message text may contain multiple addresses. If so, -each of them is anonimized (according to the same rules). -<b>Important:</b> We may change the set of acceptable characters after -the last octet in the future, if there are good reasons to do so. -<p> </p> - -<p><b>Module Configuration Parameters</b>:</p> -<p>Currently none. -<p> </p> -<p><b>Action Confguration Parameters</b>:</p> -<ul> -<li><b>mode</b> - default "rewrite"<br> -There exists the "simple" and "rewrite" mode. In simple mode, only octets -as whole can be anonymized and the length of the message is never changed. -This means that when the last three octets of the address 10.1.12.123 are -anonymized, the result will be 10.0.00.000. This means that the length of the -original octets is still visible and may be used to draw some privacy-evasive -conclusions. This mode is slightly faster than "overwrite" mode, and this -may matter in high throughput environments.<br> -The default "rewrite" mode will do full anonymization of any number of bits -and it will also normlize the address, so that no information about the -original IP address is available. So in the above example, 10.1.12.123 would -be anonymized to 10.0.0.0. -<li><b>ipv4.bits</b> - default 16<br> -This set the number of bits that should be anonymized (bits are from the -right, so lower bits are anonymized first). This setting permits to save -network information while still anonymizing user-specific data. The more -bits you discard, the better the anonymization obviously is. The default -of 16 bits reflects what German data privacy rules consider as being -sufficinetly anonymized. We assume, this can also be used as a rough -but conservative guideline for other countries.<br> -Note: when in simple mode, only bits on a byte boundary can be specified. -As such, any value other than 8, 16, 24 or 32 is invalid. If an invalid -value is given, it is rounded to the next byte boundary (so we favor stronger -anonymization in that case). For example, a bit value of 12 will become 16 in -simple mode (an error message is also emitted). -<li><b>replacementChar</b> - default "x"<br> -In simple mode, this sets the character -that the to-be-anonymized part of the IP address is to be overwritten -with. In rewrite mode, this parameter is <b>not permitted</b>, as in -this case we need not necessarily rewrite full octets. As such, the anonymized -part is always zero-filled and replacementChar is of no use. If it is -specified, an error message is emitted and the parameter ignored. -</ul> - -<p><b>Caveats/Known Bugs:</b> -<ul> -<li><b>only IPv4</b> is supported -</ul> - -<p><b>Samples:</b></p> -<p>In this snippet, we write one file without anonymization and another one -with the message anonymized. Note that once mmanon has run, access to the -original message is no longer possible (execept if stored in user -variables before anonymization). -<p><textarea rows="5" cols="60">module(load="mmanon") -action(type="omfile" file="/path/to/non-anon.log") -action(type="mmanon") -action(type="omfile" file="/path/to/anon.log") -</textarea> - -<p>This next snippet is almost identical to the first one, but -here we anonymize the full IPv4 address. Note that by -modifying the number of bits, you can anonymize different parts -of the address. Keep in mind that in simple mode (used here), the bit values -must match IP address bytes, so for IPv4 only the values 8, 16, 24 and -32 are valid. Also, in this example the replacement is done -via asterisks instead of lower-case "x"-letters. Also keep in mind that -"replacementChar" can only be set in simple mode. -<p><textarea rows="5" cols="60">module(load="mmanon") -action(type="omfile" file="/path/to/non-anon.log") -action(type="mmanon" ipv4.bits="32" mode="simple" replacementChar="*") -action(type="omfile" file="/path/to/anon.log") -</textarea> - -<p>The next snippet is also based on the first one, but anonimzes an -"odd" number of bits, 12. The value of 12 is used by some folks as a -compromise between keeping privacy and still permiting to gain some -more in-depth insight from log files. Note that anonymizing 12 bits -may be insufficient to fulfill legal requirements (if such exist). -<p><textarea rows="5" cols="60">module(load="mmanon") -action(type="omfile" file="/path/to/non-anon.log") -action(type="mmanon" ipv4.bits="12") -action(type="omfile" file="/path/to/anon.log") -</textarea> - -<p>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>] [<a href="manual.html">manual -index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> -<p><font size="2">This documentation is part of the -<a href="http://www.rsyslog.com/">rsyslog</a> project.<br> -Copyright © 2008-2013 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and -<a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL -version 3 or higher.</font></p> - -</body></html> diff --git a/doc/mmjsonparse.html b/doc/mmjsonparse.html deleted file mode 100644 index c2c862d..0000000 --- a/doc/mmjsonparse.html +++ /dev/null @@ -1,45 +0,0 @@ -<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> -<html><head> -<title>CEE/lumberjack JSON support Module (mmjsonparse)</title> -</head> -<body> -<a href="rsyslog_conf_modules.html">back</a> - -<h1>Log Message Normalization Module</h1> -<p><b>Module Name: mmjsonparse</b></p> -<p><b>Available since: </b>6.6.0+ -<p><b>Author: </b>Rainer Gerhards <rgerhards@adiscon.com></p> -<p><b>Description</b>:</p> -<p>This module provides support for parsing structured log messages -that follow the CEE/lumberjack spec. The so-called "CEE cookie" is checked -and, if present, the JSON-encoded structured message content is parsed. -The properties are than available as original message properties. -</p> -<p><b>Action specific Configuration Directives</b>:</p> -<p>currently none -<ul> -<p><b>Legacy Configuration Directives</b>:</p> -<p>none -<b>Caveats/Known Bugs:</b> -<p>None known at this time. -</ul> -<p><b>Sample:</b></p> -<p>This activates the module and applies normalization to all messages:<br> -</p> -<textarea rows="2" cols="60">module(load="mmjsonparse") -action(type="mmjsonparse") -</textarea> -<p>The same in legacy format:</p> -<textarea rows="2" cols="60">$ModLoad mmjsonparse -*.* :mmjsonparse: -</textarea> -<p>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>] -[<a href="manual.html">manual index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> -<p><font size="2">This documentation is part of the -<a href="http://www.rsyslog.com/">rsyslog</a> -project.<br> -Copyright © 2012 by <a href="http://www.gerhards.net/rainer">Rainer -Gerhards</a> and -<a href="http://www.adiscon.com/">Adiscon</a>. -Released under the GNU GPL version 3 or higher.</font></p> -</body></html> diff --git a/doc/mmnormalize.html b/doc/mmnormalize.html deleted file mode 100644 index 787bd95..0000000 --- a/doc/mmnormalize.html +++ /dev/null @@ -1,72 +0,0 @@ -<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> -<html><head> -<title>Log Message Normalization Module (mmnormalize)</title> -</head> -<body> -<a href="rsyslog_conf_modules.html">back</a> - -<h1>Log Message Normalization Module</h1> -<p><b>Module Name: mmnormalize</b></p> -<p><b>Available since: </b>6.1.2+ -<p><b>Author: </b>Rainer Gerhards <rgerhards@adiscon.com></p> -<p><b>Description</b>:</p> -<p>This module provides the capability to normalize log messages via -<a href="http://www.liblognorm.com">liblognorm</a>. Thanks to liblognorm, unstructured text, -like usually found in log messages, can very quickly be parsed and put into -a normal form. This is done so quickly, that it should be possible -to normalize events in realtime. -<p>This module is implemented via the output module interface. This means that -mmnormalize should be called just like an action. After it has been called, -the normalized message properties are avaialable and can be accessed. These properties -are called the "CEE/lumberjack" properties, because liblognorm creates a format that is -inspired by the CEE/lumberjack approach. -<p><b>Please note:</b> CEE/lumberjack properties are different from regular properties. -They have always "$!" prepended to the property name given in the rulebase. Such a -property needs to be called with <b>%$!propertyname%</b>. -<p>Note that mmnormalize should only be called once on each message. Behaviour is -undefined if multiple calls to mmnormalize happen for the same message. -</p> -<p><b>Action Parameters</b>:</p> -<ul> -<li><b>ruleBase</b> [word]<br> -Specifies which rulebase file is to use. If there are -multiple mmnormalize instances, each one can use a different file. However, -a single instance can use only a single file. This parameter MUST be given, -because normalization can only happen based on a rulebase. It is recommended -that an absolute path name is given. Information on how to create the rulebase -can be found in the <a href="http://www.liblognorm.com/files/manual/index.html">liblognorm manual</a>. -<li><b>useRawMsg</b> [boolean]<br> -Specifies if the raw message should be used for normalization (on) or just the -MSG part of the message (off). Default is "off". -</ul> -<p><b>Legacy Configuration Directives</b>:</p> -<ul> -<li>$mmnormalizeRuleBase <rulebase-file> - equivalent to the "ruleBase" -parameter. -<li>$mmnormalizeUseRawMsg <on/off> - equivalent to the "useRawMsg" -parameter. -</ul> -<b>Caveats/Known Bugs:</b> -<p>None known at this time. -</ul> -<p><b>Sample:</b></p> -<p>This activates the module and applies normalization to all messages:<br> -</p> -<textarea rows="2" cols="60">module(load="mmnormalize") -action(type="mmnormalize" ruleBase="/path/to/rulebase.rb") -</textarea> -<p>The same in legacy format:</p> -<textarea rows="3" cols="60">$ModLoad mmnormalize -$mmnormalizeRuleBase /path/to/rulebase.rb -*.* :mmnormalize: -</textarea> -<p>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>] -[<a href="manual.html">manual index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> -<p><font size="2">This documentation is part of the -<a href="http://www.rsyslog.com/">rsyslog</a> -project.<br> -Copyright © 2010-2012 by <a href="http://www.gerhards.net/rainer">Rainer -Gerhards</a> and -<a href="http://www.adiscon.com/">Adiscon</a>. -Released under the GNU GPL version 3 or higher.</font></p> -</body></html> diff --git a/doc/mmsnmptrapd.html b/doc/mmsnmptrapd.html deleted file mode 100644 index 699049d..0000000 --- a/doc/mmsnmptrapd.html +++ /dev/null @@ -1,95 +0,0 @@ -<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> -<html> -<head> -<meta http-equiv="Content-Language" content="en"> -<title>mmsnmptrapd message modification module</title> -</head> - -<body> -<a href="rsyslog_conf_modules.html">back to rsyslog module overview</a> - -<h1>mmsnmptrapd message modification module</h1> -<p><b>Module Name: imtcp</b></p> -<p><b>Author: </b>Rainer Gerhards <rgerhards@adiscon.com> (custom-created)</p> -<p><b>Multi-Ruleset Support: </b>since 5.8.1 -<p><b>Description</b>:</p> -<p>This module uses a specific configuration of snmptrapd's tag values to -obtain information of the original source system and the severity present inside the -original SNMP trap. It then replaces these fields inside the syslog message. -<p>Let's look at an example. Essentially, SNMPTT will invoke something like this: -<pre>logger -t snmptrapd/warning/realhost Host 003c.abcd.ffff in vlan 17 is flapping between port Gi4/1 and port Gi3/2 -</pre> -<p> -This message modification module will change the tag (removing the additional information), -hostname and severity (not shown in example), so the log entry will look as follows: -<pre> -2011-04-21T16:43:09.101633+02:00 realhost snmptrapd: Host 003c.abcd.ffff in vlan 122 is flapping between port Gi4/1 and port Gi3/2 -</pre> -The following logic is applied to all message being processed: -<ol> -<li>The module checks incoming syslog entries. If their TAG field starts with "snmptrapd/" -(configurable), they are modified, otherwise not. If the are modified, this happens as follows: -<li>It will derive the hostname from the tag field which has format snmptrapd/severity/hostname -<li>It should derive the severity from the tag field which has format -snmptrapd/severity/hostname. A configurable mapping table will be used to drive a new -severity value from that severity string. If no mapping has been defined, the original -severity is not changed. -<li>It replaces the "FromHost" value with the derived value from step2 -<li>It replaces the "Severity" value with the derived value from step 3 -</ol> -<p>Note that the placement of this module inside the configuration is important. All actions -before this modules is called will work on the unmodified message. All messages after it's call -will work on the modified message. Please also note that there is some extra power in case it -is required: as this module is implemented via the output module interface, a filter -can be used (actually must be used) in order to tell when it is called. Usually, the catch-all -filter (*.*) is used, but more specific filters are fully supported. So it is possible to define -different parameters for this module depending on different filters. It is also possible to -just run messages from one remote system through this module, with the help of filters or -multiple rulesets and ruleset bindings. In short words, all capabilities rsyslog offers -to control output modules are also available to mmsnmptrapd. -<p><b>Configuration Directives</b>:</p> -<ul> -<li><b>$mmsnmptrapdTag</b> [tagname]<br> -tells the module which start string inside the tag to look for. The default is -"snmptrapd". Note that a slash is automatically added to this tag when it comes to -matching incoming messages. It MUST not be given, except if two slashes are required -for whatever reasons (so "tag/" results in a check for "tag//" at the start of -the tag field). -<li><b>$mmsnmptrapdSeverityMapping</b> [severtiymap]<br> -This specifies the severity mapping table. It needs to be specified as a list. Note that -due to the current config system <b>no whitespace</b> is supported inside the list, so be -sure not to use any whitespace inside it.<br> -The list is constructed of Severtiy-Name/Severity-Value pairs, delimited by comma. -Severity-Name is a case-sensitive string, e.g. "warning" and an associated -numerical value (e.g. 4). -Possible values are in the rage 0..7 and are defined in RFC5424, table 2. The -given sample would be specified as "warning/4".<br> -If multiple instances of mmsnmptrapd are used, each instance uses the most recently -defined $mmsnmptrapdSeverityMapping before itself. -</ul> -<b>Caveats/Known Bugs:</b> -<ul> -<li>currently none known</li> -</ul> -<p><b>Example:</b></p> -<p>This enables to rewrite messages from snmptrapd and configures error and warning -severities. The default tag is used.<br> -</p> -<textarea rows="10" cols="80">$ModLoad mmsnmptrapd # needs to be done just once -# ... other module loads and listener setup ... -*.* /path/to/file/with/orignalMessage # this file receives *un*modified messages -$mmsnmptrapdSeverityMapping warning/4,error/3 -*.* :mmsnmptrapd: # *now* message is modified -*.* /path/to/file/with/modifiedMessage # this file receives modified messages -# ... rest of config ... -</textarea> -</p> -<p>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>] -[<a href="manual.html">manual index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> -<p><font size="2">This documentation is part of the <a href="http://www.rsyslog.com/">rsyslog</a> -project.<br> -Copyright © 2011 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and -<a href="http://www.adiscon.com/">Adiscon</a>. -Released under the GNU GPL version 3 or higher.</font></p> -</body> -</html> diff --git a/doc/module_workflow.png b/doc/module_workflow.png Binary files differdeleted file mode 100644 index e1a72e9..0000000 --- a/doc/module_workflow.png +++ /dev/null diff --git a/doc/modules.html b/doc/modules.html deleted file mode 100644 index 4eae6db..0000000 --- a/doc/modules.html +++ /dev/null @@ -1,94 +0,0 @@ -<html><head> -<title>Writing syslog Data to MySQL</title> -<meta name="KEYWORDS" content="syslog, mysql, syslog to mysql, howto"> -</head> -<body> -<h1>About rsyslog Modules</h1> -<P><small><i>Written by -<a href="http://www.adiscon.com/en/people/rainer-gerhards.php">Rainer Gerhards</a> (2007-07-28)</i></small></P> -<p><font color="#FF0000"><b>This document is incomplete. The module interface is -also quite incomplete and under development. Do not currently use it!</b></font> -You may want to visit <a href="http://rgerhards.blogspot.com/">Rainer's blog</a> -to learn what's going on.</p> -<h2>Overview</h2> -<p>In theory, modules provide input and output, among other functions, in -rsyslog. In practice, modules are only utilized for output in the current -release. The module interface is not yet completed and a moving target. We do -not recommend to write a module based on the current specification. If you do, -please be prepared that future released of rsyslog will probably break your -module. </p> -<p>A goal of modularization is to provide an easy to use plug-in interface. -However, this goal is not yet reached and all modules must be statically linked.</p> -<h2>Module "generation"</h2> -<p>There is a lot of plumbing that is always the same in all modules. For -example, the interface definitions, answering function pointer queries and such. -To get rid of these laborious things, I generate most of them automatically from -a single file. This file is named module-template.h. It also contains the -current best description of the interface "specification".</p> -<p>One thing that can also be achieved with it is the capability to cope with a -rapidly changing interface specification. The module interface is evolving. -Currently, it is far from being finished. As I moved the monolithic code to -modules, I needed (and still need) to make many "non-clean" code hacks, just to -get it working. These things are now gradually being removed. However, this -requires frequent changes to the interfaces, as things move in and out while -working towards a clean interface. All the interim is necessary to reach the -goal. This volatility of specifications is the number one reasons I currently -advise against implementing your own modules (hint: if you do, be sure to use -module-template.h and be prepared to fix newly appearing and disappearing data -elements).</p> -<h2>Naming Conventions</h2> -<h3>Source</h3> -<p>Output modules, and only output modules, should start with a file name of -"om" (e.g. "omfile.c", "omshell.c"). Similarly, input modules will use "im" and -filter modules "fm". The third character shall not be a hyphen.</p> -<h2>Module Security</h2> -<p>Modules are directly loaded into rsyslog's address space. As such, any module -is provided a big level of trust. Please note that further module interfaces -might provide a way to load a module into an isolated address space. This, -however, is far from being completed. So the number one rule about module -security is to run only code that you know you can trust.</p> -<p>To minimize the security risks associated with modules, rsyslog provides only -the most minimalistic access to data structures to its modules. For that reason, -the output modules do not receive any direct pointers to the selector_t -structure, the syslogd action structures and - most importantly - the msg -structure itself. Access to these structures would enable modules to access data -that is none of their business, creating a potential security weakness.</p> -<p>Not having access to these structures also simplifies further queueing and -error handling cases. As we do not need to provide e.g. full access to the msg -object itself, we do not need to serialize and cache it. Instead, strings needed -by the module are created by syslogd and then the final result is provided to -the module. That, for example, means that in a queued case $NOW is the actual -timestamp of when the message was processed, which may be even days before it -being dequeued. Think about it: If we wouldn't cache the resulting string, $NOW -would be the actual date if the action were suspended and messages queued for -some time. That could potentially result in big confusion.</p> -<p>It is thought that if an output module actually needs access to the while msg -object, we will (then) introduce a way to serialize it (e.g. to XML) in the -property replacer. Then, the output module can work with this serialized object. -The key point is that output modules never deal directly with msg objects (and -other internal structures). Besides security, this also greatly simplifies the -job of the output module developer.</p> -<h2>Action Selectors</h2> -<p>Modules (and rsyslog) need to know when they are called. For this, there must -a an action identification in selector lines. There are two syntaxes: the -single-character syntax, where a single characters identifies a module (e.g. "*" -for a wall message) and the modules designator syntax, where the module name is -given between colons (e.g. ":ommysql:"). The single character syntax is -depreciated and should not be used for new plugins.</p> -<p>An in-depth discussion of module designation in action selectors can be found -in this forum thread:</p> -<p> -<a href="http://www.rsyslog.com/index.php?name=PNphpBB2&file=viewtopic&p=678#678"> -http://www.rsyslog.com/index.php?name=PNphpBB2&file=viewtopic&p=678#678</a></p> -<h2>Copyright</h2> -<p>Copyright (c) 2007 -<a href="http://www.adiscon.com/en/people/rainer-gerhards.php">Rainer Gerhards</a> -and <a href="http://www.adiscon.com/en/">Adiscon</a>.</p> -<p>Permission is granted to copy, distribute and/or modify this document under -the terms of the GNU Free Documentation License, Version 1.2 or any later -version published by the Free Software Foundation; with no Invariant Sections, -no Front-Cover Texts, and no Back-Cover Texts. A copy of the license can be -viewed at <a href="http://www.gnu.org/copyleft/fdl.html"> -http://www.gnu.org/copyleft/fdl.html</a>.</p> -</body> -</html> diff --git a/doc/multi_ruleset.html b/doc/multi_ruleset.html deleted file mode 100644 index 37c5406..0000000 --- a/doc/multi_ruleset.html +++ /dev/null @@ -1,238 +0,0 @@ -<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> -<html><head> -<title>Multiple Rulesets in rsyslog</title></head> -<body> -<h1>Multiple Rulesets in rsyslog</h1> -<p>Starting with version 4.5.0 and 5.1.1, <a href="http://www.rsyslog.com">rsyslog</a> supports -multiple rulesets within a single configuration. -This is especially useful for routing the recpetion of remote messages to a set of specific rules. -Note that the input module must support binding to non-standard rulesets, so the functionality -may not be available with all inputs. -<p>In this document, I am using <a href="imtcp.html">imtcp</a>, an input module -that supports binding to non-standard rulesets since rsyslog started to support them. -<h2>What is a Ruleset?</h2> -If you have worked with (r)syslog.conf, you know that it is made up of what I call rules (others -tend to call them selectors, a sysklogd term). Each rule consist of a filter and one or more -actions to be carried out when the filter evaluates to true. A filter may be as simple as a -traditional -syslog priority based filter (like "*.*" or "mail.info" or a as complex as a -script-like expression. Details on that are covered in the config file documentation. After the -filter come action specifiers, and an action is something that does something to a message, e.g. -write it to a file or forward it to a remote logging server. - -<p>A traditional configuration file is made up of one or more of these rules. When a new -message arrives, its processing starts with the first rule (in order of appearance in -rsyslog.conf) and continues for each rule until either all rules have been processed or -a so-called "discard" action happens, in which case processing stops and the -message is thrown away (what also happens after the last rule has been processed). - -<p>The <b>multi-ruleset</b> support now permits to specify more than one such rule sequence. -You can think of a traditional config file just as a single default rule set, which is -automatically bound to each of the inputs. This is even what actually happens. When -rsyslog.conf is processed, the config file parser looks for the directive - -<pre>ruleset(name="rulesetname"); -</pre> - -<p>Where name is any name the user likes (but must not start with "RSYSLOG_", which -is the name space reserved for rsyslog use). If it finds this directive, it begins a new -rule set (if the name was not yet know) or switches to an already-existing one (if the name -was known). All rules defined between this $RuleSet directive and the next one are appended -to the named ruleset. Note that the reserved name "RSYSLOG_DefaultRuleset" is used to -specify rsyslogd's default ruleset. You can use that name whereever you can use a ruleset name, -including when binding an input to it. - -<p>Inside a ruleset, messages are processed as described above: they start with the first rule -and rules are processed in the order of appearance of the configuration file until either -there are no more rules or the discard action is executed. Note that with multiple rulesets -no longer <b>all</b> rsyslog.conf rules are executed but <b>only</b> those that are -contained within the specific ruleset. - -<p>Inputs must explicitely bind to rulesets. If they don't do, the default ruleset is bound. - -<p>This brings up the next question: - -<h2>What does "To bind to a Ruleset" mean?</h2> -<p>This term is used in the same sense as "to bind an IP address to an interface": -it means that a specific input, or part of an input (like a tcp listener) will use a specific -ruleset to "pass its messages to". So when a new message arrives, it will be processed -via the bound ruleset. Rule from all other rulesets are irrelevant and will never be processed. -<p>This makes multiple rulesets very handy to process local and remote message via -seperate means: bind the respective receivers to different rule sets, and you do not need -to seperate the messages by any other method. - -<p>Binding to rulesets is input-specifc. For imtcp, this is done via the - -<pre>input(type="imptcp" port="514" ruleset="rulesetname"); -</pre> - -directive. Note that "name" must be the name of a ruleset that is already defined -at the time the bind directive is given. There are many ways to make sure this happens, but -I personally think that it is best to define all rule sets at the top of rsyslog.conf and -define the inputs at the bottom. This kind of reverses the traditional recommended ordering, but -seems to be a really useful and straightforward way of doing things. -<h2>Why are rulesets important for different parser configurations?</h2> -<p>Custom message parsers, used to handle differnet (and potentially otherwise-invalid) -message formats, can be bound to rulesets. So multiple rulesets can be a very useful -way to handle devices sending messages in different malformed formats in a consistent -way. Unfortunately, this is not uncommon in the syslog world. An in-depth explanation -with configuration sample can be found at the -<a href="rsconf1_rulesetparser.html">$RulesetParser</a> configuration directive. -<h2>Can I use a different Ruleset as the default?</h2> -<p>This is possible by using the - -<pre>$DefaultRuleset <name> -</pre> - -Directive. Please note, however, that this directive is actually global: that is, it does not -modify the ruleset to which the next input is bound but rather provides a system-wide -default rule set for those inputs that did not explicitly bind to one. As such, the directive -can not be used as a work-around to bind inputs to non-default rulesets that do not support -ruleset binding. -<h2>Examples</h2> -<h3>Split local and remote logging</h3> -<p>Let's say you have a pretty standard system that logs its local messages to the usual -bunch of files that are specified in the default rsyslog.conf. As an example, your rsyslog.conf -might look like this: - -<pre> -# ... module loading ... -# The authpriv file has restricted access. -authpriv.* /var/log/secure -# Log all the mail messages in one place. -mail.* /var/log/maillog -# Log cron stuff -cron.* /var/log/cron -# Everybody gets emergency messages -*.emerg * -... more ... -</pre> - -<p>Now, you want to add receive messages from a remote system and log these to -a special file, but you do not want to have these messages written to the files -specified above. The traditional approach is to add a rule in front of all others that -filters on the message, processes it and then discards it: - -<pre> -# ... module loading ... -# process remote messages -if $fromhost-ip == '192.168.152.137' then { - action(type="omfile" file="/var/log/remotefile02") - stop - } - - -# only messages not from 192.0.21 make it past this point - -# The authpriv file has restricted access. -authpriv.* /var/log/secure -# Log all the mail messages in one place. -mail.* /var/log/maillog -# Log cron stuff -cron.* /var/log/cron -# Everybody gets emergency messages -*.emerg * -... more ... -</pre> - -<p>Note that "stop" is the discard action!. Also note that we assume that -192.0.2.1 is the sole remote sender (to keep it simple). - -<p>With multiple rulesets, we can simply define a dedicated ruleset for the remote reception -case and bind it to the receiver. This may be written as follows: - -<pre> -# ... module loading ... -# process remote messages -# define new ruleset and add rules to it: -ruleset(name="remote"){ - action(type="omfile" file="/var/log/remotefile") -} -# only messages not from 192.0.21 make it past this point - -# bind ruleset to tcp listener and activate it: -input(type="imptcp" port="10514" ruleset="remote") -</pre> - -<h3>Split local and remote logging for three different ports</h3> -<p>This example is almost like the first one, but it extends it a little bit. While it is -very similar, I hope it is different enough to provide a useful example why you may want -to have more than two rulesets. - -<p>Again, we would like to use the "regular" log files for local logging, only. But -this time we set up three syslog/tcp listeners, each one listening to a different -port (in this example 10514, 10515, and 10516). Logs received from these receivers shall go into -different files. Also, logs received from 10516 (and only from that port!) with -"mail.*" priority, shall be written into a specif file and <b>not</b> be -written to 10516's general log file. - -<p>This is the config: - -<pre> -# ... module loading ... -# process remote messages - -ruleset(name="remote10514"){ - action(type="omfile" file="/var/log/remote10514") -} - -ruleset(name="remote10515"){ - action(type="omfile" file="/var/log/remote10515") -} - -ruleset(name="test1"){ - if prifilt("mail.*") then { - /var/log/mail10516 - stop - # note that the stop-command will prevent this message from - # being written to the remote10516 file - as usual... - } - /var/log/remote10516 -} - - -# and now define listners bound to the relevant ruleset -input(type="imptcp" port="10514" ruleset="remote10514") -input(type="imptcp" port="10515" ruleset="remote10515") -input(type="imptcp" port="10516" ruleset="remote10516") -</pre> - - - - -<h2>Performance</h2> -<h3>Fewer Filters</h3> -<p>No rule processing can be faster than not processing a rule at all. As such, it is useful -for a high performance system to identify disjunct actions and try to split these off to -different rule sets. In the example section, we had a case where three different tcp listeners -need to write to three different files. This is a perfect example of where multiple rule sets -are easier to use and offer more performance. The performance is better simply because there -is no need to check the reception service - instead messages are automatically pushed to the -right rule set and can be processed by very simple rules (maybe even with -"*.*"-filters, the fastest ones available). - -<h3>Partitioning of Input Data</h3> -<p>Starting with rsyslog 5.3.4, rulesets permit higher concurrency. They offer the ability -to run on their own "main" queue. What that means is that a own queue is associated -with a specific rule set. That means that inputs bound to that ruleset do no longer need -to compete with each other when they enqueue a data element into the queue. Instead, enqueue -operations can be completed in parallel. -<p>An example: let us assume we have three TCP listeners. Without rulesets, each of them -needs to insert messages into the main message queue. So if each of them wants to -submit a newly arrived message into the queue at the same time, only one can do so -while the others need to wait. With multiple rulesets, its own queue can be created for each -ruleset. If now each listener is bound to its own ruleset, concurrent message submission is -possible. On a machine with a sufficiently large number of corse, this can result in -dramatic performance improvement. -<p>It is highly advised that high-performance systems define a dedicated ruleset, with a -dedicated queue for each of the inputs. -<p>By default, rulesets do <b>not</b> have their own queue. It must be activated via the -<a href="rsconf1_rulesetcreatemainqueue.html">$RulesetCreateMainQueue</a> directive. - -<p>[<a href="manual.html">manual index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> -<p><font size="2">This documentation is part of the <a href="http://www.rsyslog.com/">rsyslog</a> -project.<br> -Copyright © 2009 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and -<a href="http://www.adiscon.com/">Adiscon</a>. -Released under the GNU GPL version 3 or higher.</font></p> -</body></html> diff --git a/doc/multi_ruleset_legacy_format.html b/doc/multi_ruleset_legacy_format.html deleted file mode 100644 index 5a9e7a4..0000000 --- a/doc/multi_ruleset_legacy_format.html +++ /dev/null @@ -1,192 +0,0 @@ -<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> -<html><head> -<title>Multiple Rulesets in legacy format</title></head> -<body> -<h1>Multiple Rulesets in rsyslog</h1> -<p>Starting with version 4.5.0 and 5.1.1, <a href="http://www.rsyslog.com">rsyslog</a> supports -multiple rulesets within a single configuration. -This is especially useful for routing the recpetion of remote messages to a set of specific rules. -Note that the input module must support binding to non-standard rulesets, so the functionality -may not be available with all inputs.<p> -<b>Attention: this guide is shortened and only contains the samples in legacy format.</b> -Please follow this link to the full guide in the new config format "list": <a href="http://www.rsyslog.com/doc/multi_ruleset.html">http://www.rsyslog.com/doc/multi_ruleset.html<a> - - -<h2>Examples</h2> -<h3>Split local and remote logging</h3> -<p>Let's say you have a pretty standard system that logs its local messages to the usual -bunch of files that are specified in the default rsyslog.conf. As an example, your rsyslog.conf -might look like this: - -<pre> -# ... module loading ... -# The authpriv file has restricted access. -authpriv.* /var/log/secure -# Log all the mail messages in one place. -mail.* /var/log/maillog -# Log cron stuff -cron.* /var/log/cron -# Everybody gets emergency messages -*.emerg * -... more ... -</pre> - -<p>Now, you want to add receive messages from a remote system and log these to -a special file, but you do not want to have these messages written to the files -specified above. The traditional approach is to add a rule in front of all others that -filters on the message, processes it and then discards it: - -<pre> -# ... module loading ... -# process remote messages -:fromhost-ip, isequal, "192.0.2.1" /var/log/remotefile -& ~ -# only messages not from 192.0.21 make it past this point - -# The authpriv file has restricted access. -authpriv.* /var/log/secure -# Log all the mail messages in one place. -mail.* /var/log/maillog -# Log cron stuff -cron.* /var/log/cron -# Everybody gets emergency messages -*.emerg * -... more ... -</pre> - -<p>Note the tilde character, which is the discard action!. Also note that we assume that -192.0.2.1 is the sole remote sender (to keep it simple). - -<p>With multiple rulesets, we can simply define a dedicated ruleset for the remote reception -case and bind it to the receiver. This may be written as follows: - -<pre> -# ... module loading ... -# process remote messages -# define new ruleset and add rules to it: -$RuleSet remote -*.* /var/log/remotefile -# only messages not from 192.0.21 make it past this point - -# bind ruleset to tcp listener -$InputTCPServerBindRuleset remote -# and activate it: -$InputTCPServerRun 10514 - -# switch back to the default ruleset: -$RuleSet RSYSLOG_DefaultRuleset -# The authpriv file has restricted access. -authpriv.* /var/log/secure -# Log all the mail messages in one place. -mail.* /var/log/maillog -# Log cron stuff -cron.* /var/log/cron -# Everybody gets emergency messages -*.emerg * -... more ... -</pre> - -<p>Here, we need to switch back to the default ruleset after we have defined our custom -one. This is why I recommend a different ordering, which I find more intuitive. The sample -below has it, and it leads to the same results: - -<pre> -# ... module loading ... -# at first, this is a copy of the unmodified rsyslog.conf -# The authpriv file has restricted access. -authpriv.* /var/log/secure -# Log all the mail messages in one place. -mail.* /var/log/maillog -# Log cron stuff -cron.* /var/log/cron -# Everybody gets emergency messages -*.emerg * -... more ... -# end of the "regular" rsyslog.conf. Now come the new definitions: - -# process remote messages -# define new ruleset and add rules to it: -$RuleSet remote -*.* /var/log/remotefile - -# bind ruleset to tcp listener -$InputTCPServerBindRuleset remote -# and activate it: -$InputTCPServerRun 10514 -</pre> - -<p>Here, we do not switch back to the default ruleset, because this is not needed as it is -completely defined when we begin the "remote" ruleset. - -<p>Now look at the examples and compare them to the single-ruleset solution. You will notice -that we do <b>not</b> need a real filter in the multi-ruleset case: we can simply use -"*.*" as all messages now means all messages that are being processed by this -rule set and all of them come in via the TCP receiver! This is what makes using multiple -rulesets so much easier. - -<h3>Split local and remote logging for three different ports</h3> -<p>This example is almost like the first one, but it extends it a little bit. While it is -very similar, I hope it is different enough to provide a useful example why you may want -to have more than two rulesets. - -<p>Again, we would like to use the "regular" log files for local logging, only. But -this time we set up three syslog/tcp listeners, each one listening to a different -port (in this example 10514, 10515, and 10516). Logs received from these receivers shall go into -different files. Also, logs received from 10516 (and only from that port!) with -"mail.*" priority, shall be written into a specif file and <b>not</b> be -written to 10516's general log file. - -<p>This is the config: - -<pre> -# ... module loading ... -# at first, this is a copy of the unmodified rsyslog.conf -# The authpriv file has restricted access. -authpriv.* /var/log/secure -# Log all the mail messages in one place. -mail.* /var/log/maillog -# Log cron stuff -cron.* /var/log/cron -# Everybody gets emergency messages -*.emerg * -... more ... -# end of the "regular" rsyslog.conf. Now come the new definitions: - -# process remote messages - -#define rulesets first -$RuleSet remote10514 -*.* /var/log/remote10514 - -$RuleSet remote10515 -*.* /var/log/remote10515 - -$RuleSet remote10516 -mail.* /var/log/mail10516 -& ~ -# note that the discard-action will prevent this messag from -# being written to the remote10516 file - as usual... -*.* /var/log/remote10516 - -# and now define listners bound to the relevant ruleset -$InputTCPServerBindRuleset remote10514 -$InputTCPServerRun 10514 - -$InputTCPServerBindRuleset remote10515 -$InputTCPServerRun 10515 - -$InputTCPServerBindRuleset remote10516 -$InputTCPServerRun 10516 -</pre> - -<p>Note that the "mail.*" rule inside the "remote10516" ruleset does -not affect processing inside any other rule set, including the default rule set. - - -<p>[<a href="manual.html">manual index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> -<p><font size="2">This documentation is part of the <a href="http://www.rsyslog.com/">rsyslog</a> -project.<br> -Copyright © 2009 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and -<a href="http://www.adiscon.com/">Adiscon</a>. -Released under the GNU GPL version 3 or higher.</font></p> -</body></html> diff --git a/doc/netstream.html b/doc/netstream.html deleted file mode 100644 index cbfa12a..0000000 --- a/doc/netstream.html +++ /dev/null @@ -1,23 +0,0 @@ -<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> -<html><head><title>Network Stream Drivers</title> - -</head> -<body> -<a href="rsyslog_conf_global.html">back</a> - -<h1>Network Stream Drivers</h1><p>Network stream drivers are a layer -between various parts of rsyslogd (e.g. the imtcp module) and the -transport layer. They provide sequenced delivery, authentication and -confidentiality to the upper layers. Drivers implement different -capabilities.</p><p> Users need to know about netstream drivers because -they need to configure the proper driver, and proper driver properties, -to achieve desired results (e.g. a <a href="rsyslog_tls.html">TLS-protected syslog transmission</a>).</p><p>The following drivers exist:</p><ul><li><a href="ns_ptcp.html">ptcp</a> - the plain tcp network transport (no security)</li><li><a href="ns_gtls.html">gtls</a> - a secure TLS transport implemented via the GnuTLS library</li></ul>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>] -[<a href="manual.html">manual index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>] -<p><font size="2">This documentation is part of the -<a href="http://www.rsyslog.com/">rsyslog</a> -project.<br> -Copyright © 2008 by <a href="http://www.gerhards.net/rainer">Rainer -Gerhards</a> and -<a href="http://www.adiscon.com/">Adiscon</a>. -Released under the GNU GPL version 3 or higher.</font></p> -</body></html> diff --git a/doc/ns_gtls.html b/doc/ns_gtls.html deleted file mode 100644 index 0d02ad0..0000000 --- a/doc/ns_gtls.html +++ /dev/null @@ -1,59 +0,0 @@ -<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> -<html><head><title>gtls Network Stream Driver</title> - -</head> -<body> -<h1>gtls Network Stream Driver</h1> -<p>This <a href="netstream.html">network stream -driver</a> implements a TLS protected transport via the <a href="http://www.gnu.org/software/gnutls/" target="_blank">GnuTLS -library</a>.</p> -<p><b>Available since:</b> 3.19.0 (suggested minimum 3.19.8 and above)</p> -<p style="font-weight: bold;">Supported Driver Modes</p> -<ul> -<li>0 - unencrypted trasmission (just like <a href="ns_ptcp.html">ptcp</a> driver)</li> -<li>1 - TLS-protected operation</li> -</ul> -Note: mode 0 does not provide any benefit over the ptcp driver. This -mode exists for technical reasons, but should not be used. It may be -removed in the future.<br> -<span style="font-weight: bold;">Supported Authentication -Modes</span><br> -<ul> -<li><span style="font-weight: bold;">anon</span> -- anonymous authentication as -described in IETF's draft-ietf-syslog-transport-tls-12 Internet draft</li> -<li><span style="font-weight: bold;">x509/fingerprint</span> -- certificate fingerprint authentication as -described in IETF's draft-ietf-syslog-transport-tls-12 Internet draft</li> -<li><span style="font-weight: bold;">x509/certvalid</span> -- certificate validation only</li> -<li><span style="font-weight: bold;">x509/name</span> -- certificate validation and subject name authentication as -described in IETF's draft-ietf-syslog-transport-tls-12 Internet draft -</li> -</ul> -Note: "anon" does not permit to authenticate the remote peer. As such, -this mode is vulnerable to man in the middle attacks as well as -unauthorized access. It is recommended NOT to use this mode.</p> -<p>x509/certvalid is a nonstandard mode. It validates the remote -peers certificate, but does not check the subject name. This is -weak authentication that may be useful in scenarios where multiple -devices are deployed and it is sufficient proof of authenticy when -their certificates are signed by the CA the server trusts. This is -better than anon authentication, but still not recommended. -<b>Known Problems</b><br> -<p>Even in x509/fingerprint mode, both the client and sever -certificate currently must be signed by the same root CA. This is an -artifact of the underlying GnuTLS library and the way we use it. It is -expected that we can resolve this issue in the future.</p> -<p>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>] -[<a href="manual.html">manual index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>] -</p> -<p><font size="2">This documentation is part of the -<a href="http://www.rsyslog.com/">rsyslog</a> -project.<br> -Copyright © 2008 by <a href="http://www.gerhards.net/rainer">Rainer -Gerhards</a> and -<a href="http://www.adiscon.com/">Adiscon</a>. -Released under the GNU GPL version 3 or higher.</font></p> -</body></html> diff --git a/doc/ns_ptcp.html b/doc/ns_ptcp.html deleted file mode 100644 index c028d6c..0000000 --- a/doc/ns_ptcp.html +++ /dev/null @@ -1,16 +0,0 @@ -<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> -<html><head><title>ptcp Network Stream Driver</title> - -</head> -<body> -<h1>ptcp Network Stream Driver</h1> -<p>This <a href="netstream.html">network stream driver</a> implement a plain tcp transport without security properties.</p><p>Supported Driver Modes</p><ul><li>0 - unencrypted trasmission</li></ul>Supported Authentication Modes<br><ul><li>"anon" - no authentication</li></ul>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>] -[<a href="manual.html">manual index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>] -<p><font size="2">This documentation is part of the -<a href="http://www.rsyslog.com/">rsyslog</a> -project.<br> -Copyright © 2008 by <a href="http://www.gerhards.net/rainer">Rainer -Gerhards</a> and -<a href="http://www.adiscon.com/">Adiscon</a>. -Released under the GNU GPL version 3 or higher.</font></p> -</body></html>
\ No newline at end of file diff --git a/doc/omfile.html b/doc/omfile.html deleted file mode 100644 index 3966ab1..0000000 --- a/doc/omfile.html +++ /dev/null @@ -1,187 +0,0 @@ -<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> -<html><head> -<meta http-equiv="Content-Language" content="en"> -<title>File Output Module</title></head> - -<body> -<a href="rsyslog_conf_modules.html">back</a> - -<h1>File Output Module</h1> -<p><b>Module Name: omfile</b></p> -<p><b>Author: </b>Rainer Gerhards <rgergards@adiscon.com></p> -<p><b>Description</b>:</p> -<p>The omfile plug-in provides the core functionality of writing messages to files residing inside the local file system (which may actually be remote if methods like NFS are used). Both files named with static names as well files with names based on message content are supported by this module. It is a built-in module that does not need to be loaded. </p> -<p> </p> - -<p><b>Module Parameters</b>:</p> -<ul> - <li><strong>Template </strong>[templateName]<br> - sets a new default template for file actions.<br></li> - -</ul> -<p> </p> -<p><b>Action Parameters</b>:</p> -<ul> - <li><strong>DynaFileCacheSize </strong>(not mandatory, default will be used)<br> - Defines a template to be used for the output. <br></li><br> - - <li><strong>ZipLevel </strong>0..9 [default 0]<br> - if greater 0, turns on gzip compression of the output file. The higher the number, the better the compression, but also the more CPU is required for zipping.<br></li><br> - - <li><b>VeryRobustZip</b> [<b>on</b>/off] (v7.3.0+) - if ZipLevel is greater 0, - then this setting controls if extra headers are written to make the resulting file - extra hardened against malfunction. If set to off, data appended to previously unclean - closed files may not be accessible without extra tools. - Note that this risk is usually expected to be bearable, and thus "off" is the default mode. - The extra headers considerably - degrade compression, files with this option set to "on" may be four to five times as - large as files processed in "off" mode. - </li><br> - - <li><strong>FlushInterval </strong>(not mandatory, default will be used)<br> - Defines a template to be used for the output. <br></li><br> - - <li><strong>ASyncWriting </strong>on/off [default off]<br> - if turned on, the files will be written in asynchronous mode via a separate thread. In that case, double buffers will be used so that one buffer can be filled while the other buffer is being written. Note that in order to enable FlushInterval, AsyncWriting must be set to "on". Otherwise, the flush interval will be ignored. Also note that when FlushOnTXEnd is "on" but AsyncWriting is off, output will only be written when the buffer is full. This may take several hours, or even require a rsyslog shutdown. However, a buffer flush can be forced in that case by sending rsyslogd a HUP signal. <br></li><br> - - <li><strong>FlushOnTXEnd </strong>on/off [default on]<br> - Omfile has the capability to write output using a buffered writer. Disk writes are only done when the buffer is full. So if an error happens during that write, data is potentially lost. In cases where this is unacceptable, set FlushOnTXEnd to on. Then, data is written at the end of each transaction (for pre-v5 this means after each log message) and the usual error recovery thus can handle write errors without data loss. Note that this option severely reduces the effect of zip compression and should be switched to off for that use case. Note that the default -on- is primarily an aid to preserve the traditional syslogd behaviour.<br></li><br> - - <li><strong>IOBufferSize </strong><size_nbr>, default 4k<br> - size of the buffer used to writing output data. The larger the buffer, the potentially better performance is. The default of 4k is quite conservative, it is useful to go up to 64k, and 128K if you used gzip compression (then, even higher sizes may make sense)<br></li><br> - - <li><strong>DirOwner </strong><br> - Set the file owner for directories newly created. Please note that this setting does not affect the owner of directories already existing. The parameter is a user name, for which the userid is obtained by rsyslogd during startup processing. Interim changes to the user mapping are not detected.<br></li><br> - - <li><strong>DirGroup </strong><br> - Set the group for directories newly created. Please note that this setting does not affect the group of directories already existing. The parameter is a group name, for which the groupid is obtained by rsyslogd on during startup processing. Interim changes to the user mapping are not detected.<br></li><br> - - <li><strong>FileOwner </strong><br> - Set the file owner for dynaFiles newly created. Please note that this setting does not affect the owner of files already existing. The parameter is a user name, for which the userid is obtained by rsyslogd during startup processing. Interim changes to the user mapping are not detected.<br></li><br> - - <li><strong>FileGroup </strong><br> - Set the group for dynaFiles newly created. Please note that this setting does not affect the group of files already existing. The parameter is a group name, for which the groupid is obtained by rsyslogd during startup processing. Interim changes to the user mapping are not detected.<br></li><br> - - <li><strong>DirCreateMode </strong>[defaul 0700]<br> - This is the same as $FileCreateMode, but for directories automatically generated.<br></li><br> - - <li><strong>FileCreateMode </strong>[default 0644]<br> - The FileCreateMode directive allows to specify the creation mode with which rsyslogd creates new files. If not specified, the value 0644 is used (which retains backward-compatibility with earlier releases). The value given must always be a 4-digit octal number, with the initial digit being zero. <br>Please note that the actual permission depend on rsyslogd's process umask. If in doubt, use "$umask 0000" right at the beginning of the configuration file to remove any restrictions. <br>FileCreateMode may be specified multiple times. If so, it specifies the creation mode for all selector lines that follow until the next $FileCreateMode directive. Order of lines is vitally important.<br></li><br> - - <li><strong>FailOnCHOwnFailure </strong>on/off [default on]<br> - This option modifies behaviour of dynaFile creation. If different owners or groups are specified for new files or directories and rsyslogd fails to set these new owners or groups, it will log an error and NOT write to the file in question if that option is set to "on". If it is set to "off", the error will be ignored and processing continues. Keep in mind, that the files in this case may be (in)accessible by people who should not have permission. The default is "on".<br></li><br> - - <li><strong>CreateDirs </strong>on/off [default on]<br> - create directories on an as-needed basis<br></li><br> - - <li><strong>Sync </strong>on/off [default off]<br> - enables file syncing capability of omfile.<br></li><br> - - <li><strong>File </strong><br> - If the file already exists, new data is appended to it. Existing data is not truncated. If the file does not already exist, it is created. Files are kept open as long as rsyslogd is active. This conflicts with external log file rotation. In order to close a file after rotation, send rsyslogd a HUP signal after the file has been rotated away. <br></li><br> - - <li><strong>DynaFile </strong><br> - For each message, the file name is generated based on the given template. Then, this file is opened. As with the ``file'' property, data is appended if the file already exists. If the file does not exist, a new file is created. A cache of recent files is kept. Note that this cache can consume quite some memory (especially if large buffer sizes are used). Files are kept open as long as they stay inside the cache. Currently, files are only evicted from the cache when there is need to do so (due to insufficient cache size). To force-close (and evict) a dynafile from cache, send a HUP signal to rsyslogd. <br></li><br> - - <li><b>Sig.Provider </b>[ProviderName]<br> - Selects a signature provider for log signing. Currently, - there only is one provider called - "<a href="sigprov_gt.html">gt</a>".<br></li><br> - - <li><b>Cry.Provider </b>[ProviderName]<br> - Selects a crypto provider for log encryption. Currently, - there only is one provider called - "<a href="cryprov_gcry.html">gcry</a>".<br></li><br> - - <li><strong>Template </strong>[templateName]<br> - sets a new default template for file actions.<br></li><br> - -</ul> -<p><b>Caveats/Known Bugs:</b></p><ul><li>None.</li></ul> -<p><b>Sample:</b></p> -<p>The following command writes all syslog messages into a file.</p> -<textarea rows="5" cols="60">Module (load="builtin:omfile") -*.* action(type="omfile" -DirCreateMode="0700" -FileCreateMode="0644" -File="/var/log/messages") -</textarea> - -<br><br> - -<p><b>Legacy Configuration Directives</b>:</p> -<ul> - <li><strong>$DynaFileCacheSize </strong>(not mandatory, default will be used)<br> - Defines a template to be used for the output. <br></li><br> - - <li><strong>$OMFileZipLevel </strong>0..9 [default 0]<br> - if greater 0, turns on gzip compression of the output file. The higher the number, the better the compression, but also the more CPU is required for zipping.<br></li><br> - - <li><strong>$OMFileFlushInterval </strong>(not mandatory, default will be used)<br> - Defines a template to be used for the output. <br></li><br> - - <li><strong>$OMFileASyncWriting </strong>on/off [default off]<br> - if turned on, the files will be written in asynchronous mode via a separate thread. In that case, double buffers will be used so that one buffer can be filled while the other buffer is being written. Note that in order to enable FlushInterval, AsyncWriting must be set to "on". Otherwise, the flush interval will be ignored. Also note that when FlushOnTXEnd is "on" but AsyncWriting is off, output will only be written when the buffer is full. This may take several hours, or even require a rsyslog shutdown. However, a buffer flush can be forced in that case by sending rsyslogd a HUP signal. <br></li><br> - - <li><strong>$OMFileFlushOnTXEnd </strong>on/off [default on]<br> - Omfile has the capability to write output using a buffered writer. Disk writes are only done when the buffer is full. So if an error happens during that write, data is potentially lost. In cases where this is unacceptable, set FlushOnTXEnd to on. Then, data is written at the end of each transaction (for pre-v5 this means after each log message) and the usual error recovery thus can handle write errors without data loss. Note that this option severely reduces the effect of zip compression and should be switched to off for that use case. Note that the default -on- is primarily an aid to preserve the traditional syslogd behaviour.<br></li><br> - - <li><strong>$OMFileIOBufferSize </strong><size_nbr>, default 4k<br> - size of the buffer used to writing output data. The larger the buffer, the potentially better performance is. The default of 4k is quite conservative, it is useful to go up to 64k, and 128K if you used gzip compression (then, even higher sizes may make sense)<br></li><br> - - <li><strong>$DirOwner </strong><br> - Set the file owner for directories newly created. Please note that this setting does not affect the owner of directories already existing. The parameter is a user name, for which the userid is obtained by rsyslogd during startup processing. Interim changes to the user mapping are not detected.<br></li><br> - - <li><strong>$DirGroup </strong><br> - Set the group for directories newly created. Please note that this setting does not affect the group of directories already existing. The parameter is a group name, for which the groupid is obtained by rsyslogd on during startup processing. Interim changes to the user mapping are not detected.<br></li><br> - - <li><strong>$FileOwner </strong><br> - Set the file owner for dynaFiles newly created. Please note that this setting does not affect the owner of files already existing. The parameter is a user name, for which the userid is obtained by rsyslogd during startup processing. Interim changes to the user mapping are not detected.<br></li><br> - - <li><strong>$FileGroup </strong><br> - Set the group for dynaFiles newly created. Please note that this setting does not affect the group of files already existing. The parameter is a group name, for which the groupid is obtained by rsyslogd during startup processing. Interim changes to the user mapping are not detected.<br></li><br> - - <li><strong>$DirCreateMode </strong>[defaul 0700]<br> - This is the same as $FileCreateMode, but for directories automatically generated.<br></li><br> - - <li><strong>$FileCreateMode </strong>[default 0644]<br> - The FileCreateMode directive allows to specify the creation mode with which rsyslogd creates new files. If not specified, the value 0644 is used (which retains backward-compatibility with earlier releases). The value given must always be a 4-digit octal number, with the initial digit being zero. <br>Please note that the actual permission depend on rsyslogd's process umask. If in doubt, use "$umask 0000" right at the beginning of the configuration file to remove any restrictions. <br>FileCreateMode may be specified multiple times. If so, it specifies the creation mode for all selector lines that follow until the next $FileCreateMode directive. Order of lines is vitally important.<br></li><br> - - <li><strong>$FailOnCHOwnFailure </strong>on/off [default on]<br> - This option modifies behaviour of dynaFile creation. If different owners or groups are specified for new files or directories and rsyslogd fails to set these new owners or groups, it will log an error and NOT write to the file in question if that option is set to "on". If it is set to "off", the error will be ignored and processing continues. Keep in mind, that the files in this case may be (in)accessible by people who should not have permission. The default is "on".<br></li><br> - - <li><strong>$F$OMFileForceCHOwn </strong><br> - force ownership change for all files<br></li><br> - - <li><strong>$CreateDirs </strong>on/off [default on]<br> - create directories on an as-needed basis<br></li><br> - - <li><strong>$ActionFileEnableSync </strong>on/off [default off]<br> - enables file syncing capability of omfile.<br></li><br> - - <li><strong>$ActionFileDefaultTemplate </strong>[templateName]<br> - sets a new default template for file actions.<br></li><br> - - <li><strong>$ResetConfigVariables </strong><br> - Resets all configuration variables to their default value. Any settings made will not be applied to configuration lines following the $ResetConfigVariables. This is a good method to make sure no side-effects exists from previous directives. This directive has no parameters.<br></li><br> - -</ul> - -<p><b>Legacy Sample:</b></p> -<p>The following command writes all syslog messages into a file.</p> -<textarea rows="5" cols="60">$ModLoad omfile -$DirCreateMode 0700 -$FileCreateMode 0644 -*.* /var/log/messages -</textarea> - - -<p>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>] [<a href="manual.html">manual -index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> -<p><font size="2">This documentation is part of the -<a href="http://www.rsyslog.com/">rsyslog</a> project.<br> -Copyright © 2008-2013 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and -<a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL -version 3 or higher.</font></p> - -</body></html> diff --git a/doc/omfwd.html b/doc/omfwd.html deleted file mode 100644 index 53f9e52..0000000 --- a/doc/omfwd.html +++ /dev/null @@ -1,118 +0,0 @@ -<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> -<html><head> -<meta http-equiv="Content-Language" content="en"> -<title>Forwarding Output Module</title></head> - -<body> -<a href="rsyslog_conf_modules.html">back</a> - -<h1>Forwarding Output Module</h1> -<p><b>Module Name: omfwd</b></p> -<p><b>Author: </b>Rainer Gerhards <rgergards@adiscon.com></p> -<p><b>Description</b>:</p> -<p>The omfwd plug-in provides the core functionality of traditional message forwarding via UDP and plain TCP. It is a built-in module that does not need to be loaded. </p> -<p> </p> - -<p><b>Global Configuration Directives</b>:</p> -<ul> - <li><strong>Template </strong>[templateName]<br> - sets a non-standard default template for this module.<br></li> - -</ul> -<p> </p> -<p><b>Action specific Configuration Directives</b>:</p> -<ul> - <li><strong>Target </strong>string<br> - Name or IP-Address of the system that shall receive messages. Any resolvable name is fine. <br></li><br> - - <li><strong>Port </strong>[Default 514]<br> - Name or numerical value of port to use when connecting to target. <br></li><br> - - <li><strong>Protocol </strong>udp/tcp [default udp]<br> - Type of protocol to use for forwarding. Note that ``tcp'' means both legacy plain tcp syslog as well as RFC5425-based TCL-encrypted syslog. Which one is selected depends on the protocol drivers set before the action commend. Note that as of 6.3.6, there is no way to specify this within the action itself. <br></li><br> - - <li><strong>TCP_Framing </strong>``traditional'' or ``octet-counted'' [default traditional]<br> - Framing-Mode to be for forwarding. This affects only TCP-based protocols. It is ignored for UDP. In protocol engineering, ``framing'' means how multiple messages over the same connection are separated. Usually, this is transparent to users. Unfortunately, the early syslog protocol evolved, and so there are cases where users need to specify the framing. The traditional framing is nontransparent. With it, messages are end when a LF (aka ``line break'', ``return'') is encountered, and the next message starts immediately after the LF. If multi-line messages are received, these are essentially broken up into multiple message, usually with all but the first message segment being incorrectly formatted. The octet-counting framing solves this issue. With it, each message is prefixed with the actual message length, so that a receivers knows exactly where the message ends. Multi-line messages cause no problem here. This mode is very close to the method described in RFC5425 for TLS-enabled syslog. Unfortunately, only few syslogd implementations support octet-counted framing. As such, the traditional framing is set as default, even though it has defects. If it is known that the receiver supports octet-counted framing, it is suggested to use that framing mode. <br></li><br> - - <li><strong>ZipLevel </strong>0..9 [default 0]<br> - Compression level for messages. Rsyslog implements a proprietary capability to zip transmitted messages. Note that compression happens on a message-per-message basis. As such, there is a performance gain only for larger messages. Before compressing a message, rsyslog checks if there is some gain by compression. If so, the message is sent compressed. If not, it is sent uncompressed. As such, it is totally valid that compressed and uncompressed messages are intermixed within a conversation. <br>The compression level is specified via the usual factor of 0 to 9, with 9 being the strongest compression (taking up most processing time) and 0 being no compression at all (taking up no extra processing time). <br></li><br> - - <li><strong>RebindInterval </strong>integer<br> - Permits to specify an interval at which the current connection is broken and re-established. This setting is primarily an aid to load balancers. After the configured number of messages has been transmitted, the current connection is terminated and a new one started. Note that this setting applies to both TCP and UDP traffic. For UDP, the new ``connection'' uses a different source port (ports are cycled and not reused too frequently). This usually is perceived as a ``new connection'' by load balancers, which in turn forward messages to another physical target system. <br></li><br> - - <li><strong>StreamDriver </strong>string<br> - Set the file owner for directories newly created. Please note that this setting does not affect the owner of directories already existing. The parameter is a user name, for which the userid is obtained by rsyslogd during startup processing. Interim changes to the user mapping are not detected.<br></li><br> - - <li><strong>StreamDriverMode </strong>integer [default 0]<br> - mode to use with the stream driver (driver-specific)<br></li><br> - - <li><strong>StreamDriverAuthMode </strong>string<br> - authentication mode to use with the stream driver. Note that this directive requires TLS netstream drivers. For all others, it will be ignored. (driver-specific).<br></li><br> - - <li><strong>StreamDriverPermittedPeers </strong>string<br> - accepted fingerprint (SHA1) or name of remote peer. Note that this directive requires TLS netstream drivers. For all others, it will be ignored. (driver-specific)<br></li><br> - - <li><strong>ResendLastMSGOnReconnect </strong>on/off<br> - Permits to resend the last message when a connection is reconnected. This setting affects TCP-based syslog, only. It is most useful for traditional, plain TCP syslog. Using this protocol, it is not always possible to know which messages were successfully transmitted to the receiver when a connection breaks. In many cases, the last message sent is lost. By switching this setting to "yes", rsyslog will always retransmit the last message when a connection is reestablished. This reduces potential message loss, but comes at the price that some messages may be duplicated (what usually is more acceptable). <br></li><br> - -</ul> -<p><b>Caveats/Known Bugs:</b></p><ul><li>None.</li></ul> -<p><b>Sample:</b></p> -<p>The following command sends all syslog messages to a remote server via TCP port 10514.</p> -<textarea rows="5" cols="60">Module (load="builtin:omfwd") -*.* action(type="omfwd" -Target="192.168.2.11" -Port="10514" -Protocol="tcp" -) -</textarea> - -<br><br> - -<p><b>Legacy Configuration Directives</b>:</p> -<ul> - <li><strong>$ActionForwardDefaultTemplateName </strong>string [templatename]<br> - sets a new default template for UDP and plain TCP forwarding action<br></li><br> - - <li><strong>$ActionSendTCPRebindInterval </strong>integer<br> - instructs the TCP send action to close and re-open the connection to the remote host every nbr of messages sent. Zero, the default, means that no such processing is done. This directive is useful for use with load-balancers. Note that there is some performance overhead associated with it, so it is advisable to not too often "rebind" the connection (what "too often" actually means depends on your configuration, a rule of thumb is that it should be not be much more often than once per second).<br></li><br> - - <li><strong>$ActionSendUDPRebindInterval </strong>integer<br> - instructs the UDP send action to rebind the send socket every nbr of messages sent. Zero, the default, means that no rebind is done. This directive is useful for use with load-balancers.<br></li><br> - - <li><strong>$ActionSendStreamDriver </strong><driver basename><br> - just like $DefaultNetstreamDriver, but for the specific action <br></li><br> - - <li><strong>$ActionSendStreamDriverMode </strong><mode> [default 0]<br> - mode to use with the stream driver (driver-specific)<br></li><br> - - <li><strong>$ActionSendStreamDriverAuthMode </strong><mode><br> - authentication mode to use with the stream driver. Note that this directive requires TLS netstream drivers. For all others, it will be ignored. (driver-specific))<br></li><br> - - <li><strong>$ActionSendStreamDriverPermittedPeers </strong><ID><br> - accepted fingerprint (SHA1) or name of remote peer. Note that this directive requires TLS netstream drivers. For all others, it will be ignored. (driver-specific) <br></li><br> - - <li><strong>$ActionSendResendLastMsgOnReconnect </strong>on/off [default off]<br> - specifies if the last message is to be resend when a connecition breaks and has been reconnected. May increase reliability, but comes at the risk of message duplication. <br></li><br> - - <li><strong>$ResetConfigVariables </strong><br> - Resets all configuration variables to their default value. Any settings made will not be applied to configuration lines following the $ResetConfigVariables. This is a good method to make sure no side-effects exists from previous directives. This directive has no parameters.<br></li><br> - -</ul> - -<p><b>Legacy Sample:</b></p> -<p>The following command sends all syslog messages to a remote server via TCP port 10514.</p> -<textarea rows="5" cols="60">$ModLoad omfwd -*.* @@192.168.2.11:10514 -</textarea> - - -<p>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>] [<a href="manual.html">manual -index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> -<p><font size="2">This documentation is part of the -<a href="http://www.rsyslog.com/">rsyslog</a> project.<br> -Copyright © 2008 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and -<a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL -version 3 or higher.</font></p> - -</body></html> diff --git a/doc/omhdfs.html b/doc/omhdfs.html deleted file mode 100644 index ef7e60c..0000000 --- a/doc/omhdfs.html +++ /dev/null @@ -1,69 +0,0 @@ -<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> -<html><head><title>rsyslog output module for HDFS (omhdfs)</title> -<a href="features.html">back</a> -</head> -<body> -<h1>Unix sockets Output Module (omhdfs)</h1> -<p><b>Module Name: omhdfs</b></p> -<p><b>Available since: </b> 5.7.1</p> -<p><b>Author: </b>Rainer Gerhards <rgerhards@adiscon.com></p> -<p><b>Description</b>:</p> -<p>This module supports writing message into files on Hadoop's HDFS -file system. -<p><b>Configuration Directives</b>:</p> -<ul> -<li><b>$OMHDFSFileName</b> [name]<br> -The name of the file to which the output data shall be written. -</li> -<li><b>$OMHDFSHost</b> [name]<br> -Name or IP address of the HDFS host to connect to. -</li> -<li><b>$OMHDFSPort</b> [name]<br> -Port on which to connect to the HDFS host. -</li> -<li><b>$OMHDFSDefaultTemplate</b> [name]<br> -Default template to be used when none is specified. This saves the work of -specifying the same template ever and ever again. Of course, the default -template can be overwritten via the usual method. -</li> -</ul> -<b>Caveats/Known Bugs:</b> -<p>Building omhdfs is a challenge because we could not yet find out how -to integrate Java properly into the autotools build process. The issue is -that HDFS is written in Java and libhdfs uses JNI to talk to it. That requires -that various system-specific environment options and pathes be set correctly. At -this point, we leave this to the user. If someone know how to do it better, -please drop us a line! -<ul> -<li>In order to build, you need to set these environment variables BEFORE running -./configure: -<ul> -<li>JAVA_INCLUDES - must have all include pathes that are needed to build -JNI C programms, including the -I options necessary for gcc. An example is<br> -# export JAVA_INCLUDES="-I/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_64/include -I/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_64/include/linux" -<li>JAVA_LIBS - must have all library pathes that are needed to build -JNI C programms, including the -l/-L options necessary for gcc. An example is<br> -# export export JAVA_LIBS="-L/usr/java/jdk1.6.0_21/jre/lib/amd64 -L/usr/java/jdk1.6.0_21/jre/lib/amd64/server -ljava -ljvm -lverify" -</ul> - -<li>As of HDFS architecture, you must make sure that all relevant environment -variables (the usual Java stuff and HADOOP's home directory) are properly set. -<li>As it looks, libhdfs makes Java throw exceptions to stdout. There is no -known work-around for this (and it usually should not case any troubles. -</ul> -<p><b>Sample:</b></p> -<p> -</p> -<textarea rows="4" cols="80">$ModLoad omhdfs - -$OMHDFSFileName /var/log/logfile -*.* :omhdfs: -</textarea> -[<a href="manual.html">manual index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> -<p><font size="2">This documentation is part of the <a href="http://www.rsyslog.com/">rsyslog</a> -project.<br> -Copyright © 2010 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and -<a href="http://www.adiscon.com/">Adiscon</a>. -Released under the GNU GPL version 3 or higher.</font></p> - -</body></html> diff --git a/doc/omjournal.html b/doc/omjournal.html deleted file mode 100644 index c42d984..0000000 --- a/doc/omjournal.html +++ /dev/null @@ -1,83 +0,0 @@ -<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> -<html><head> -<meta http-equiv="Content-Language" content="en"> -<title>Linux Journal Output Module (omjournal)</title></head> - -<body> -<a href="rsyslog_conf_modules.html">back</a> - -<h1>Linux Journal Output Module (omjournal)</h1> -<p><b>Module Name: omjournal</b></p> -<p><b>Author: </b>Rainer Gerhards <rgerhards@adiscon.com></p> -<p><b>Available since</b>: 7.3.7</p> -<p><b>Description</b>:</p> -<p>The omjournal output module provides an interface to the Linux journal. -It is meant to be used in those cases where the Linux journal is being used -as the sole system log database. With omjournal, messages from various -sources (e.g. files and remote devices) can also be written to the journal -and processed by its tools. -<p>A typical use case we had on our mind is a SOHO environment, where the -user wants to include syslog data obtained from the local router to be -part of the journal data. -<p> </p> - -<p><b>Module Configuration Parameters</b>:</p> -<p>Currently none. -<p> </p> -<p><b>Action Confguration Parameters</b>:</p> -<p>Currently none. - -<p><b>Caveats/Known Bugs:</b> -<ul> -<li>One needs to be careful that no message routing loop is created. The -systemd journal forwards messages it receives to the traditional syslog -system (if present). That means rsyslog will receive the same message that -it just wrote as new input on imuxsock. If not handled specially and assuming -all messages be written to the journal, the message would be emitted to the -journal again and a deadly loop is started. -<p>To prevent that, imuxsock by default does not accept messages originating -from its own process ID, aka it ignores messages from the current instance of -rsyslogd. However, this setting can be changed, and if so the problem may occur. -</ul> - -<p><b>Sample:</b></p> -<p>We assume we have a DSL router inside the network and would like to -receive its syslog message into the journal. Note that this configuration can be -used without havoing any other syslog functionality at all (most importantly, there -is no need to write any file to /var/log!). We assume syslog over UDP, as this -is the most probable choice for the SOHO environment that this use case reflects. -To log to syslog data to the journal, add the following snippet to rsyslog.conf: -<textarea rows="20" cols="60">/* first, we make sure all necessary - * modules are present: - */ -module(load="imudp") # input module for UDP syslog -module(load="omjournal") # output module for journal - -/* then, define the actual server that listens to the - * router. Note that 514 is the default port for UDP - * syslog and that we use a dedicated ruleset to - * avoid mixing messages with the local log stream - * (if there is any). - */ -input(type="imudp" port="514" ruleset="writeToJournal") - -/* inside that ruleset, we just write data to the journal: */ -ruleset(name="writeToJournal") { - action(type="omjournal") -} -</textarea> -<p>Note that this can be your sole rsyslog.conf if you do not use rsyslog -for anything else than receving the router syslog messages. -<p>If you do not receive messages, <b>you probably need to enable inbound UDP -syslog traffic in your firewall</b>. - - -<p>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>] [<a href="manual.html">manual -index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> -<p><font size="2">This documentation is part of the -<a href="http://www.rsyslog.com/">rsyslog</a> project.<br> -Copyright © 2008-2013 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and -<a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL -version 3 or higher.</font></p> - -</body></html> diff --git a/doc/omlibdbi.html b/doc/omlibdbi.html deleted file mode 100644 index e47c7f5..0000000 --- a/doc/omlibdbi.html +++ /dev/null @@ -1,150 +0,0 @@ -<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> -<html><head> -<meta http-equiv="Content-Language" content="en"><title>Generic Database Output Module (omlibdbi)</title> - -</head> -<body> -<a href="rsyslog_conf_modules.html">back</a> - -<h1>Generic Database Output Module (omlibdbi)</h1> -<p><b>Module Name: omlibdbi</b></p> -<p><b>Author: </b>Rainer Gerhards -<rgerhards@adiscon.com></p> -<p><b>Description</b>:</p> -<p>This modules supports a large number of database systems via <a href="http://libdbi.sourceforge.net/">libdbi</a>. -Libdbi abstracts the database layer and provides drivers for many -systems. Drivers are available via the <a href="http://libdbi-drivers.sourceforge.net/">libdbi-drivers</a> -project. As of this writing, the following drivers are available:</p> -<ul> -<li><a href="http://www.firebird.sourceforge.net/">Firebird/Interbase</a></li> -<li><a href="http://www.freetds.org/">FreeTDS</a> -(provides access to <a href="http://www.microsoft.com/sql">MS -SQL Server</a> and <a href="http://www.sybase.com/products/informationmanagement/adaptiveserverenterprise">Sybase</a>)</li> -<li><a href="http://www.mysql.com/">MySQL</a> -(also -supported via the native ommysql plugin in rsyslog)</li> -<li><a href="http://www.postgresql.org/">PostgreSQL</a>(also -supported via the native -ommysql plugin in rsyslog)</li> -<li><a href="http://www.sqlite.org/">SQLite/SQLite3</a></li> -</ul> -<p>The following drivers are in various stages of completion:</p> -<ul> -<li><a href="http://ingres.com/">Ingres</a></li> -<li><a href="http://www.hughes.com.au/">mSQL</a></li> -<li><a href="http://www.oracle.com/">Oracle</a></li> -</ul> -<p>These drivers seem to be quite usable, at -least from an rsyslog point of view.</p> -<p>Libdbi provides a slim layer between rsyslog and the actual -database engine. We have not yet done any performance testing (e.g. -omlibdbi vs. ommysql) but honestly believe that the performance impact -should be irrelevant, if at all measurable. Part of that assumption is -that rsyslog just does the "insert" and most of the time is spent -either in the database engine or rsyslog itself. It's hard to think of -any considerable time spent in the libdbi abstraction layer.</p> -<p><span style="font-weight: bold;">Setup</span></p> -<p>In order for this plugin to work, you need to have libdbi, the -libdbi driver for your database backend and the client software for -your database backend installed. There are libdbi packages for many -distributions. Please note that rsyslogd requires a quite recent -version (0.8.3) of libdbi. It may work with older versions, but these -need some special ./configure options to support being called from a -dlopen()ed plugin (as omlibdbi is). So in short, you probably save you -a lot of headache if you make sure you have at least libdbi version -0.8.3 on your system. -</p> -<p><b>Module Parameters</b></p> -<ul> -<li><b>template</b><br> -The default template to use. This template is used when no template is -explicitely specified in the action() statement. -<li><b>driverdirectory</b><br> -Path to the libdbi drivers. Usually, -you do not need to set it. If you installed libdbi-drivers at a -non-standard location, you may need to specify the directory here. If -you are unsure, do <b>not</b> use this configuration directive. -Usually, everything works just fine. -Note that this was an action() paramter in rsyslog versions below 7.3.0. -However, only the first action's driverdirectory parameter was actually used. -This has been cleaned up in 7.3.0, where this now is a module paramter. -</li> -</ul> -<p><b>Action Parameters</b></p> -<ul> -<li><b>server</b><br>Name or address of the MySQL server -<li><b>db</b><br>Database to use -<li><b>uid</b><br>logon userid used to connect to server. Must have proper permissions. -<li><b>pwd</b><br>the user's password -<li><b>template</b><br>Template to use when submitting messages. -<li><b>driver</b><br> -Name of the dbidriver to use, see libdbi-drivers documentation. As a -quick excerpt, at least those were available at the time of this -writiting "mysql" (suggest to use ommysql instead), "firebird" (Firbird -and InterBase), "ingres", "msql", "Oracle", "sqlite", "sqlite3", -"freetds" (for Microsoft SQL and Sybase) and "pgsql" (suggest to use -ompgsql instead).</li> -</ul> -<p><b>Legacy (pre-v6) Configuration Directives</b>:</p> -<p>It is strongly recommended NOT to use legacy format. -<ul> -<li><i>$ActionLibdbiDriverDirectory /path/to/dbd/drivers</i> -- like the driverdirectory action parameter. -<li><i>$ActionLibdbiDriver drivername</i> - like the drivername action parameter -<li><i>$ActionLibdbiHost hostname</i> - like the server action parameter -<li><i>$ActionLibdbiUserName user</i> - like the uid action parameter -<li><i>$ActionlibdbiPassword</i> - like the pwd action parameter -<li><i>$ActionlibdbiDBName db</i> - like the db action parameter -<li><i>selector line: :omlibdbi:<code>;template</code></i><br> -executes the recently configured omlibdbi action. The ;template part is -optional. If no template is provided, a default template is used (which -is currently optimized for MySQL - sorry, folks...)</li> -</ul> -<b>Caveats/Known Bugs:</b> -<p>You must make sure that any templates used for omlibdbi -properly escape strings. This is usually done by supplying the SQL (or -STDSQL) option to the template. Omlibdbi rejects templates without this -option for security reasons. However, omlibdbi does not detect if you -used the right option for your backend. Future versions of rsyslog -(with full expression support) will provide advanced -ways of handling this situation. So far, you must be careful. The -default template provided by rsyslog is suitable for MySQL, but not -necessarily for your database backend. Be careful!</p> -<p>If you receive the rsyslog error message "libdbi or libdbi -drivers not present on this system" you may either not have libdbi and -its drivers installed or (very probably) the version is earlier than -0.8.3. In this case, you need to make sure you have at least 0.8.3 and -the libdbi driver for your database backend present on your system.</p><p>I -do not have most of the database supported by omlibdbi in my lab. So it -received limited cross-platform tests. If you run into troubles, be -sure the let us know at <a href="http://www.rsyslog.com">http://www.rsyslog.com</a>.</p> -<p><b>Sample:</b></p> -<p>The following sample writes all syslog messages to the -database "syslog_db" on mysqlsever.example.com. The server is MySQL and -being accessed under the account of "user" with password "pwd" (if you -have empty passwords, just remove the $ActionLibdbiPassword line).<br> -</p> -<textarea rows="5" cols="60">module(load="omlibdbi") -*.* action(type="omlibdbi" driver="mysql" - server="mysqlserver.example.com" db="syslog_db" - uid="user" pwd="pwd" -</textarea> -<p><b>Legacy Sample:</b></p> -<p>The same as above, but in legacy config format (pre rsyslog-v6): -<textarea rows="8" cols="60">$ModLoad omlibdbi -$ActionLibdbiDriver mysql -$ActionLibdbiHost mysqlserver.example.com -$ActionLibdbiUserName user -$ActionLibdbiPassword pwd -$ActionLibdbiDBName syslog_db -*.* :omlibdbi: -</textarea> -<p>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>] -[<a href="manual.html">manual index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> -<p><font size="2">This documentation is part of the -<a href="http://www.rsyslog.com/">rsyslog</a> -project.<br> -Copyright © 2008-2012 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and -<a href="http://www.adiscon.com/">Adiscon</a>. -Released under the ASL 2.0.</font></p> -</body></html> diff --git a/doc/ommail.html b/doc/ommail.html deleted file mode 100644 index 0841dc9..0000000 --- a/doc/ommail.html +++ /dev/null @@ -1,146 +0,0 @@ -<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> -<html><head><title>mail output module - sending syslog messages via mail</title> -<a href="features.html">back</a> -</head> -<body> -<h1>Mail Output Module (ommail)</h1> -<p><b>Module Name: ommail</b></p> -<p><b>Available since: </b> 3.17.0</p> -<p><b>Author: </b>Rainer Gerhards -<rgerhards@adiscon.com></p> -<p><b>Description</b>:</p> -<p>This module supports sending syslog messages via mail. Each -syslog message is sent via its own mail. Obviously, you will want to -apply rigorous filtering, otherwise your mailbox (and mail server) will -be heavily spammed. The ommail plugin is primarily meant for alerting -users. As such, it is assume that mails will only be sent in an -extremely limited number of cases.</p> -<p>Please note that ommail is especially well-suited to work in -tandem with <a href="imfile.html">imfile</a> to -watch files for the occurence of specific things to be alerted on. So -its scope is far broader than forwarding syslog messages to mail -recipients.</p> -Ommail uses two templates, one for the mail body and one for the -subject line. If neither is provided, a quite meaningless subject line -is used and the mail body will be a syslog message just as if it were -written to a file. It is expected that the users customizes both -messages. In an effort to support cell phones (including SMS gateways), -there is an option to turn off the body part at all. This is considered -to be useful to send a short alert to a pager-like device.<br> -<br> -It is highly recommended to use the "<span style="font-weight: bold;">$ActionExecOnlyOnceEveryInterval -<seconds></span>" directive to limit the amount of -mails that potentially be generated. With it, mails are sent at most in -a <seconds> interval. This may be your life safer. And -remember that an hour has 3,600 seconds, so if you would like to -receive mails at most once every two hours, include a -"$ActionExecOnlyOnceEveryInterval 7200" immediately before the ommail -action. Messages sent more frequently are simpy discarded.<span style="font-weight: bold;"></span> -<p><b>Configuration Directives</b>:</p> -<ul> -<li><span style="font-weight: bold;">$ActionMailSMTPServer</span><br> -Name or IP address of the SMTP server to be used. Must currently be -set. The default is 127.0.0.1, the SMTP server on the local machine. -Obviously it is not good to expect one to be present on each machine, -so this value should be specified.<br> -</li> -<li><span style="font-weight: bold;">$ActionMailSMTPPort</span><br> -Port number or name of the SMTP port to be used. The default is 25, the -standard SMTP port.</li> -<li><span style="font-weight: bold;">$ActionMailFrom</span><br> -The email address used as the senders address. There is no default.</li> -<li><span style="font-weight: bold;">$ActionMailTo</span><br> -The recipient email addresses. There is no default. To specify multiple -recpients, repeat this directive as often as needed. Note: <b>This directive -must be specified for each new action and is automatically reset.</b> -[Multiple recipients are supported for 3.21.2 and above.]</li> -<li><span style="font-weight: bold;">$ActionMailSubject</span><br> -The name of the <span style="font-weight: bold;">template</span> -to be used as the mail subject. If this is not specified, a more or -less meaningless mail subject is generated (we don't tell you the exact -text because that can change - if you want to have something specific, -configure it!).</li> -<li><span style="font-weight: bold;">$ActionMailEnableBody</span><br> -Setting this to "off" permits to exclude the actual message body. This -may be useful for pager-like devices or cell phone SMS messages. The -default is "on", which is appropriate for allmost all cases. Turn it -off only if you know exactly what you do!</li> -</ul> -<b>Caveats/Known Bugs:</b> -<p>The current ommail implementation supports <span style="font-weight: bold;">SMTP-direct mode</span> -only. In that mode, the plugin talks to the mail server via SMTP -protocol. No other process is involved. This mode offers best -reliability as it is not depending on any external entity except the -mail server. Mail server downtime is acceptable if the action is put -onto its own action queue, so that it may wait for the SMTP server to -come back online. However, the module implements only the bare SMTP -essentials. Most importantly, it does not provide any authentication -capabilities. So your mail server must be configured to accept incoming -mail from ommail without any authentication needs (this may be change -in the future as need arises, but you may also be referred to -sendmail-mode).</p> -<p>In theory, ommail should also offer a mode where it uses the -sendmail utility to send its mail (<span style="font-weight: bold;">sendmail-mode</span>). -This is somewhat less reliable (because we depend on an entity we do -not have close control over - sendmail). It also requires dramatically -more system ressources, as we need to load the external process (but -that should be no problem given the expected infrequent number of calls -into this plugin). The big advantage of sendmail mode is that it -supports all the bells and whistles of a full-blown SMTP implementation -and may even work for local delivery without a SMTP server being -present. Sendmail mode will be implemented as need arises. So if you -need it, please drop us a line (I nobody does, sendmail mode will -probably never be implemented).</p> -<p><b>Sample:</b></p> -<p>The following sample alerts the operator if the string "hard -disk fatal failure" is present inside a syslog message. The mail server -at mail.example.net is used and the subject shall be "disk problem on -<hostname>". Note how \r\n is included inside the body -text -to create line breaks. A message is sent at most once every 6 hours, -any other messages are silently discarded (or, to be precise, not being -forwarded - they are still being processed by the rest of the -configuration file).<br> -</p> -<textarea rows="15" cols="80">$ModLoad ommail -$ActionMailSMTPServer mail.example.net -$ActionMailFrom rsyslog@example.net -$ActionMailTo operator@example.net -$template mailSubject,"disk problem on %hostname%" -$template mailBody,"RSYSLOG Alert\r\nmsg='%msg%'" -$ActionMailSubject mailSubject -# make sure we receive a mail only once in six -# hours (21,600 seconds ;)) -$ActionExecOnlyOnceEveryInterval 21600 -# the if ... then ... mailBody mus be on one line! -if $msg contains 'hard disk fatal failure' then :ommail:;mailBody -</textarea> -<p>The sample below is the same, but sends mail to two recipients:</p> -<textarea rows="15" cols="80">$ModLoad ommail -$ActionMailSMTPServer mail.example.net -$ActionMailFrom rsyslog@example.net -$ActionMailTo operator@example.net -$ActionMailTo admin@example.net -$template mailSubject,"disk problem on %hostname%" -$template mailBody,"RSYSLOG Alert\r\nmsg='%msg%'" -$ActionMailSubject mailSubject -# make sure we receive a mail only once in six -# hours (21,600 seconds ;)) -$ActionExecOnlyOnceEveryInterval 21600 -# the if ... then ... mailBody mus be on one line! -if $msg contains 'hard disk fatal failure' then :ommail:;mailBody -</textarea> -<p>A more advanced example plus a discussion on using the email feature -inside a reliable system can be found in Rainer's blogpost -"<a style="font-style: italic;" href="http://rgerhards.blogspot.com/2008/04/why-is-native-email-capability.html">Why -is native email capability an advantage for a syslogd?</a>" -<p>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>] -[<a href="manual.html">manual index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> -<p><font size="2">This documentation is part of the <a href="http://www.rsyslog.com/">rsyslog</a> -project.<br> -Copyright © 2008 by <a href="http://www.gerhards.net/rainer">Rainer -Gerhards</a> and -<a href="http://www.adiscon.com/">Adiscon</a>. -Released under the GNU GPL version 3 or higher.</font></p> - -</body></html> diff --git a/doc/ommysql.html b/doc/ommysql.html deleted file mode 100644 index 7769fb8..0000000 --- a/doc/ommysql.html +++ /dev/null @@ -1,85 +0,0 @@ -<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> -<html><head> -<meta http-equiv="Content-Language" content="en"> -<title>MySQL Database Output Module</title> -</head> - -<body> -<a href="rsyslog_conf_modules.html">back</a> - -<h1>MySQL Database Output Module</h1> -<p><b>Module Name: ommysql</b></p> -<p><b>Author: </b>Michael Meckelein (Initial Author) / Rainer Gerhards -<rgerhards@adiscon.com></p> -<p><b>Description</b>:</p> -<p>This module provides native support for logging to MySQL databases. It offers -superior performance over the more generic <a href="omlibdbi.html">omlibdbi</a> module. -</p> -<p><b>Action Parameters</b>:</p> -<ul> -<li><b>server</b><br>Name or address of the MySQL server -<li><b>serverport</b><br>Permits to select -a non-standard port for the MySQL server. The default is 0, which means the -system default port is used. There is no need to specify this parameter unless -you know the server is running on a non-standard listen port. -<li><b>db</b><br>Database to use -<li><b>uid</b><br>logon userid used to connect to server. Must have proper permissions. -<li><b>pwd</b><br>the user's password -<li><b>template</b><br>Template to use when submitting messages. -<li><b>mysqlconfig.file</b><br>Permits the selection -of an optional MySQL Client Library configuration file (my.cnf) for extended -configuration functionality. The use of this configuration directive is necessary -only if you have a non-standard environment or if fine-grained control over the -database connection is desired.</li> -<li><b>mysqlconfig.section</b><br>Permits the selection of the -section within the configuration file specified by the <b>myselconfig.file</b> parameter. -<br>This will likely only be used where the database administrator provides a single -configuration file with multiple profiles. -<br>This configuration parameter is ignored unless <b>mysqlconfig.file</b> is also used. -<br>If omitted, the MySQL Client Library default of "client" will be used.</li> -</ul> -<p><b>Legacy (pre-v6) Configuration Directives</b>:</p> -<p>ommysql mostly uses the "very old style" (v0) configuration, with almost everything on the -action line itself. -<ul> -<li><b>$ActionOmmysqlServerPort <port></b> - like the "serverport" action parameter. -<li><b>$OmMySQLConfigFile <file name></b> - like the "mysqlconfig.file" action parameter. -<li><b>$OmMySQLConfigSection <string></b> - like the "mysqlconfig.file" action parameter. -<li>Action line: -<br><b>:ommysql:database-server,database-name,database-userid,database-password</b> -<br>All parameters should be filled in for a successful connect. -</ul> -<p>Note rsyslog contains a canned default template to write to the MySQL -database. It works on the MonitorWare schema. This template is: -<p> -<textarea rows="5" cols="80">$template tpl,"insert into SystemEvents (Message, Facility, FromHost, Priority, DeviceReportedTime, ReceivedAt, InfoUnitID, SysLogTag) values ('%msg%', %syslogfacility%, '%HOSTNAME%', %syslogpriority%, '%timereported:::date-mysql%', '%timegenerated:::date-mysql%', %iut%, '%syslogtag%')",SQL -</textarea> -<p>As you can see, the template is an actual SQL statement. Note the ",SQL" option: it tells the -template processor that the template is used for SQL processing, thus quote characters are quoted -to prevent security issues. You can not assign a template without ",SQL" to a MySQL output action. -<p>If you would like to change fields contents or add or delete your own fields, you -can simply do so by modifying the schema (if required) and creating your own custom -template. -<p><b>Sample:</b></p> -<p>The following sample writes all syslog messages to the -database "syslog_db" on mysqlsever.example.com. The server is -being accessed under the account of "user" with password "pwd". -</p> -<textarea rows="5" cols="80">$ModLoad ommysql -*.* action(type="ommysql" server="mysqlserver.example.com" serverport="1234" - db="syslog_db" uid="user" pwd="pwd") -</textarea> -<p><b>Legacy Sample:</b></p> -<p>The same as above, but in legacy config format (pre rsyslog-v6): -<textarea rows="5" cols="80">$ModLoad ommysql -$ActionOmmysqlServerPort 1234 # use non-standard port -*.* :ommysql:mysqlserver.example.com,syslog_db,user,pwd -</textarea> -<p>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>] -[<a href="manual.html">manual index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> -<p><font size="2">This documentation is part of the -<a href="http://www.rsyslog.com/">rsyslog</a> project.<br> -Copyright © 2008-2012 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and -<a href="http://www.adiscon.com/">Adiscon</a>. -Released under the ASL 2.0.</font></p> -</body></html> diff --git a/doc/omoracle.html b/doc/omoracle.html deleted file mode 100644 index 2bb6aa5..0000000 --- a/doc/omoracle.html +++ /dev/null @@ -1,201 +0,0 @@ -<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> -<html><head> -<meta http-equiv="Content-Language" content="en"> -<title>Oracle Database Output Module</title> -</head> - -<body> -<a href="rsyslog_conf_modules.html">rsyslog module reference</a> - -<h1>Oracle Database Output Module</h1> -<p><b>Module Name: omoracle</b></p> -<p><b>Author: </b>Luis Fernando Muñoz Mejías <Luis.Fernando.Munoz.Mejias@cern.ch></p> -<p><b>Available since: </b>: 4.3.0 -<p><b>Status: </b>: contributed module, not maitained by rsyslog core authors -<p><b>Description</b>:</p> -<p>This module provides native support for logging to Oracle -databases. It offers superior performance over the more -generic <a href="omlibdbi.html">omlibdbi</a> module. It also includes -a number of enhancements, most importantly prepared statements and -batching, what provides a big performance improvement. -</p> -<p>Note that this module is maintained by its original author. If you need assistance with it, -it is suggested to post questions to the -<a href="http://lists.adiscon.net/mailman/listinfo/rsyslog">rsyslog mailing list</a>. -<p>From the header comments of this module: -<p><pre> - - This is an output module feeding directly to an Oracle - database. It uses Oracle Call Interface, a propietary module - provided by Oracle. - - Selector lines to be used are of this form: - - :omoracle:;TemplateName - - The module gets its configuration via rsyslog $... directives, - namely: - - $OmoracleDBUser: user name to log in on the database. - - $OmoracleDBPassword: password to log in on the database. - - $OmoracleDB: connection string (an Oracle easy connect or a db - name as specified by tnsnames.ora) - - $OmoracleBatchSize: Number of elements to send to the DB on each - transaction. - - $OmoracleStatement: Statement to be prepared and executed in - batches. Please note that Oracle's prepared statements have their - placeholders as ':identifier', and this module uses the colon to - guess how many placeholders there will be. - - All these directives are mandatory. The dbstring can be an Oracle - easystring or a DB name, as present in the tnsnames.ora file. - - The form of the template is just a list of strings you want - inserted to the DB, for instance: - - $template TestStmt,"%hostname%%msg%" - - Will provide the arguments to a statement like - - $OmoracleStatement \ - insert into foo(hostname,message)values(:host,:message) - - Also note that identifiers to placeholders are arbitrary. You - need to define the properties on the template in the correct order - you want them passed to the statement! -</pre> -<p>Some additional documentation contributed by Ronny Egner: -<pre> -REQUIREMENTS: --------------- - -- Oracle Instantclient 10g (NOT 11g) Base + Devel - (if you´re on 64-bit linux you should choose the 64-bit libs!) -- JDK 1.6 (not neccessary for oracle plugin but "make" didd not finsished successfully without it) - -- "oracle-instantclient-config" script - (seems to shipped with instantclient 10g Release 1 but i was unable to find it for 10g Release 2 so here it is) - - -====================== /usr/local/bin/oracle-instantclient-config ===================== -#!/bin/sh -# -# Oracle InstantClient SDK config file -# Jean-Christophe Duberga - Bordeaux 2 University -# - -# just adapt it to your environment -incdirs="-I/usr/include/oracle/10.2.0.4/client64" -libdirs="-L/usr/lib/oracle/10.2.0.4/client64/lib" - -usage="\ -Usage: oracle-instantclient-config [--prefix[=DIR]] [--exec-prefix[=DIR]] [--version] [--cflags] [--libs] [--static-libs]" - -if test $# -eq 0; then - echo "${usage}" 1>&2 - exit 1 -fi - -while test $# -gt 0; do - case "$1" in - -*=*) optarg=`echo "$1" | sed 's/[-_a-zA-Z0-9]*=//'` ;; - *) optarg= ;; - esac - - case $1 in - --prefix=*) - prefix=$optarg - if test $exec_prefix_set = no ; then - exec_prefix=$optarg - fi - ;; - --prefix) - echo $prefix - ;; - --exec-prefix=*) - exec_prefix=$optarg - exec_prefix_set=yes - ;; - --exec-prefix) - echo ${exec_prefix} - ;; - --version) - echo ${version} - ;; - --cflags) - echo ${incdirs} - ;; - --libs) - echo $libdirs -lclntsh -lnnz10 -locci -lociei -locijdbc10 - ;; - --static-libs) - echo "No static libs" 1>&2 - exit 1 - ;; - *) - echo "${usage}" 1>&2 - exit 1 - ;; - esac - shift -done - -=============== END ============== - - - - -COMPILING RSYSLOGD -------------------- - - -./configure --enable-oracle - - - - -RUNNING -------- - -- make sure rsyslogd is able to locate the oracle libs (either via LD_LIBRARY_PATH or /etc/ld.so.conf) -- set TNS_ADMIN to point to your tnsnames.ora -- create a tnsnames.ora and test you are able to connect to the database - -- create user in oracle as shown in the following example: - create user syslog identified by syslog default tablespace users quota unlimited on users; - grant create session to syslog; - create role syslog_role; - grant syslog_role to syslog; - grant create table to syslog_role; - grant create sequence to syslog_role; - -- create tables as needed - -- configure rsyslog as shown in the following example - $ModLoad omoracle - - $OmoracleDBUser syslog - $OmoracleDBPassword syslog - $OmoracleDB syslog - $OmoracleBatchSize 1 - $OmoracleBatchItemSize 4096 - - $OmoracleStatementTemplate OmoracleStatement - $template OmoracleStatement,"insert into foo(hostname,message) values (:host,:message)" - $template TestStmt,"%hostname%%msg%" - *.* :omoracle:;TestStmt - (you guess it: username = password = database = "syslog".... see $rsyslogd_source/plugins/omoracle/omoracle.c for me info) -</pre> -<p>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>] -[<a href="manual.html">manual index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> -<p><font size="2">This documentation is part of the -<a href="http://www.rsyslog.com/">rsyslog</a> -project.<br> -Copyright © 2008, 2009 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and -<a href="http://www.adiscon.com/">Adiscon</a>. -Released under the GNU GPL version 3 or higher.</font></p> -</body></html> diff --git a/doc/ompipe.html b/doc/ompipe.html deleted file mode 100644 index 49915b7..0000000 --- a/doc/ompipe.html +++ /dev/null @@ -1,62 +0,0 @@ -<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> -<html><head> -<meta http-equiv="Content-Language" content="en"> -<title>Pipe Output Module</title></head> - -<body> -<a href="rsyslog_conf_modules.html">back</a> - -<h1>Pipe Output Module</h1> -<p><b>Module Name: omfwd</b></p> -<p><b>Author: </b>Rainer Gerhards <rgergards@adiscon.com></p> -<p><b>Description</b>:</p> -<p>The ompipe plug-in provides the core functionality for logging output to named pipes (fifos). It is a built-in module that does not need to be loaded. </p> -<p> </p> - -<p><b>Global Configuration Directives</b>:</p> -<ul> - <li><strong>Template </strong>[templateName]<br> - sets a new default template for file actions.<br></li> - -</ul> -<p> </p> -<p><b>Action specific Configuration Directives</b>:</p> -<ul> - <li><strong>Pipe </strong>string<br> - A fifo or named pipe can be used as a destination for log messages.<br></li><br> - - - -</ul> -<p><b>Caveats/Known Bugs:</b></p><ul><li>None.</li></ul> -<p><b>Sample:</b></p> -<p>The following command sends all syslog messages to a remote server via TCP port 10514.</p> -<textarea rows="5" cols="60">Module (path="builtin:ompipe") -*.* action(type="ompipe" -Pipe="NameofPipe" -) -</textarea> - -<br><br> - -<p><b>Legacy Configuration Directives</b>:</p> -<p>rsyslog has support for logging output to named pipes (fifos). A fifo or named pipe can be used as a destination for log messages by prepending a pipe symbol ("|'') to the name of the file. This is handy for debugging. Note that the fifo must be created with the mkfifo(1) command before rsyslogd is started. - -</p> - -<p><b>Legacy Sample:</b></p> -<p>The following command sends all syslog messages to a remote server via TCP port 10514.</p> -<textarea rows="5" cols="60">$ModLoad ompipe -*.* |/var/log/pipe -</textarea> - - -<p>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>] [<a href="manual.html">manual -index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> -<p><font size="2">This documentation is part of the -<a href="http://www.rsyslog.com/">rsyslog</a> project.<br> -Copyright © 2008 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and -<a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL -version 3 or higher.</font></p> - -</body></html> diff --git a/doc/omprog.html b/doc/omprog.html deleted file mode 100644 index 471ab22..0000000 --- a/doc/omprog.html +++ /dev/null @@ -1,43 +0,0 @@ -<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> -<html><head><title>omprog output module - sending messages to a program</title> -<a href="features.html">back</a> -</head> -<body> -<h1>Program integration Output module</h1> -<p><b>Module Name: omprog</b></p> -<p><b>Available since: </b> 4.3.0</p> -<p><b>Author: </b>Rainer Gerhards -<rgerhards@adiscon.com></p> -<p><b>Description</b>:</p> -<p>This module permits to integrate arbitrary external programs into rsyslog's -logging. It is similar to the "execute program (^)" action, but offers better security -and much higher performance. While "execute program (^)" can be a useful tool for -executing programs if rare events occur, omprog can be used to provide massive -amounts of log data to a program. -<p>Executes the configured program and feeds log messages to that binary via -stdin. The binary is free to do whatever it wants with the supplied data. -If the program terminates, it is re-started. If rsyslog terminates, the -program's stdin will see EOF. The program must than terminate. The message format -passed to the program can, as usual, be modified by defining rsyslog templates. -<p>Note that each time an omprog action is defined, the corresponding programm -is invoked. A single instance is <b>not</b> being re-used. There are arguments pro and -con re-using existing binaries. For the time being, it simply is not done. In the future, -we may add an option for such pooling, provided that some demand for that is voiced. -You can also mimic the same effect by defining multiple rulesets and including them (at -the price of some slight performance loss). -<p><b>Configuration Directives</b>:</p> -<ul> -<li><b>$ActionOMProgBinary</b> <binary><br> -The binary program to be executed. -</ul> -<b>Caveats/Known Bugs:</b> -<p>Currently none known. -<p>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>] -[<a href="manual.html">manual index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> -<p><font size="2">This documentation is part of the <a href="http://www.rsyslog.com/">rsyslog</a> -project.<br> -Copyright © 2008-2011 by <a href="http://www.gerhards.net/rainer">Rainer -Gerhards</a> and -<a href="http://www.adiscon.com/">Adiscon</a>. -Released under the GNU GPL version 3 or higher.</font></p> -</body></html> diff --git a/doc/omrelp.html b/doc/omrelp.html deleted file mode 100644 index 8858f88..0000000 --- a/doc/omrelp.html +++ /dev/null @@ -1,78 +0,0 @@ -<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> -<html><head> -<meta http-equiv="Content-Language" content="en"><title>RELP Output Module (omrelp)</title> - -</head> -<body> -<a href="rsyslog_conf_modules.html">back to rsyslog module documentation</a> - -<h1>RELP Output Module (omrelp)</h1> -<p><b>Module Name: omrelp</b></p> -<p><b>Author: </b>Rainer Gerhards -<rgerhards@adiscon.com></p> -<p><b>Description</b>:</p> -<p>This module supports sending syslog messages over the reliable -RELP protocol. For RELP's advantages over plain tcp syslog, please see -the documentation for <a href="imrelp.html">imrelp</a> -(the server counterpart). </p> -<span style="font-weight: bold;">Setup</span> -<p>Please note that <a href="http://www.librelp.com">librelp</a> -is required for imrelp (it provides the core relp protocol -implementation).</p> -<p><b>Action Configuration Parameters</b>:</p> -<p>This module supports RainerScript configuration starting with -rsyslog 7.3.10. For older versions, legacy configuration directives -must be used. -<ul> - <li><b>target </b>(mandatory)<br> - The target server to connect to. - </li> - <li><b>template </b>(not mandatory, default "RSYSLOG_ForwardFormat")<br> - Defines the template to be used for the output. - </li> - <li><b>timeout </b>(not mandatory, default 90)<br> - Timeout for relp sessions. If set too low, valid sessions - may be considered dead and tried to recover. - </li> -</ul> -<p><b>Sample:</b></p> -<p>The following sample sends all messages to the central server -"centralserv" at port 2514 (note that that server must run imrelp on -port 2514). -</p> -<textarea rows="3" cols="60">module(load="omrelp") -action(type="omrelp" target="centralserv" port="2514") -</textarea> -<p><b>Legacy Configuration Directives</b>:</p> -<p>This module uses old-style action configuration to keep -consistent with the forwarding rule. So far, no additional -configuration directives can be specified. To send a message via RELP, -use</p> -<p>*.* - :omrelp:<sever>:<port>;<template></p> -<p>just as you use </p> -<p>*.* - @@<sever>:<port>;<template></p> -<p>to forward a message via plain tcp syslog.</p> -<b>Caveats/Known Bugs:</b> -<p>See <a href="imrelp.html">imrelp</a>, -which documents them. </p> -<p><b>Legacy Sample:</b></p> -<p>The following sample sends all messages to the central server -"centralserv" at port 2514 (note that that server must run imrelp on -port 2514). -</p> -<textarea rows="3" cols="60">$ModLoad omrelp -*.* :omrelp:centralserv:2514 -</textarea> -<p>Note: to use IPv6 addresses, encode them in [::1] format. -<p>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>] -[<a href="manual.html">manual index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> -<p><font size="2">This documentation is part of the -<a href="http://www.rsyslog.com/">rsyslog</a> -project.<br> -Copyright © 2008 by <a href="http://www.gerhards.net/rainer">Rainer -Gerhards</a> and -<a href="http://www.adiscon.com/">Adiscon</a>. -Released under the GNU GPL version 3 or higher.</font></p> -</body></html> diff --git a/doc/omruleset.html b/doc/omruleset.html deleted file mode 100644 index 41d6ccf..0000000 --- a/doc/omruleset.html +++ /dev/null @@ -1,140 +0,0 @@ -<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> -<html><head> -<meta http-equiv="Content-Language" content="en"> -<title>ruleset output module (omruleset)</title> -</head> -<body> -<a href="rsyslog_conf_modules.html">rsyslog module reference</a> - -<h1>ruleset output/including module (omruleset)</h1> -<p><b>Module Name: omruleset</b></p> -<p><b>Author: </b>Rainer Gerhards <rgerhards@adiscon.com></p> -<p><b>Available Since</b>: 5.3.4</p> -<p><b>Description</b>:</p> -<p>This is a very special "output" module. It permits to pass a message object -to another rule set. While this is a very simple action, it enables very -complex configurations, e.g. it supports high-speed "and" conditions, sending -data to the same file in a non-racy way, include-ruleset functionality as well as -some high-performance optimizations (in case the rule sets have the necessary -queue definitions). -<p>While it leads to a lot of power, this output module offers seamingly easy functionaltiy. -The complexity (and capablities) arise from how everthing can be combined. -<p>With this module, a message can be sent to processing to another ruleset. This is somewhat -similar to a "#include" in the C programming language. However, one needs to keep -on the mind that a ruleset can contain its own queue and that a queue can run in various modes. -<p>Note that if no queue is defined in the ruleset, the message is enqueued into the main message -queue. This most often is not optimal and means that message processing may be severely defered. -Also note that when the ruleset's target queue is full and no free space can be aquired -within the usual timeout, the message object may actually be lost. This is an extreme scenario, -but users building an audit-grade system need to know this restriction. For regular installations, -it should not really be relevant. -<p><b>At minimum, be sure you understand the -<a href="rsconf1_rulesetcreatemainqueue.html">$RulesetCreateMainQueue</a> -directive as well as the importance of statement order in rsyslog.conf before using omruleset!</b> -<p><b>Recommended Use:</b> -<ul> -<li>create rulesets specifically for omruleset -<li>create these rulesets with their own main queue -<li> decent queueing parameters (sizes, threads, etc) should be used -for the ruleset main queue. If in doubt, use the same parameters as for the -overall main queue. -<li>if you use multiple levels of ruleset nesting, double check for endless loops - the rsyslog -engine does not detect these -</ul> - -<p><b>Configuration Directives</b>:</p> -<ul> -<li><b>$ActionOmrulesetRulesetName</b> ruleset-to-submit-to<br> -This directive specifies the name of the ruleset that the message -provided to omruleset should be submitted to. This ruleset must already have -been defined. Note that the directive is automatically reset after each -:omruleset: action and there is no default. This is done to prevent accidential -loops in ruleset definition, what can happen very quickly. -The :omruleset: action will NOT be honored if no ruleset name has been defined. As usual, -the ruleset name must be specified in front of the action that it modifies. -</ul> -<p><b>Examples:</b></p> -<p>This example creates a ruleset for a write-to-file action. The idea here -is that the same file is written based on multiple filters, problems occur if the file is used -together with a buffer. That is because file buffers are action-specific, and so some partial -buffers would be written. With omruleset, we create a single action inside its own ruleset and -then pass all messages to it whenever we need to do so. Of course, such a simple situation could -also be solved by a more complex filter, but the method used here can also be utilized in -more complex scenarios (e.g. with multiple listeners). The example tries to keep it simple. -Note that we create a ruleset-specific main queue (for simplicity with the default main queue -parameters) in order to avoid re-queueing messages back into the main queue. -</p> -<textarea rows="30" cols="80">$ModLoad omruleset - -# define ruleset for commonly written file -$RuleSet commonAction -$RulesetCreateMainQueue on -*.* /path/to/file.log - -#switch back to default ruleset -$ruleset RSYSLOG_DefaultRuleset - -# begin first action -# note that we must first specify which ruleset to use for omruleset: -$ActionOmrulesetRulesetName CommonAction -mail.info :omruleset: -#end first action - -# begin second action -# note that we must first specify which ruleset to use for omruleset: -$ActionOmrulesetRulesetName CommonAction -:FROMHOST, isequal, "myhost.example.com" :omruleset: -#end second action - -# of course, we can have "regular" actions alongside :omrulset: actions -*.* /path/to/general-message-file.log -</textarea> -<p>The next example is used to creat a high-performance nested and filter condition. Here, -it is first checked if the message contains a string "error". If so, the message is forwarded -to another ruleset which then applies some filters. The advantage of this is that we can use -high-performance filters where we otherwise would need to use the (much slower) expression-based -filters. Also, this enables pipeline processing, in that second ruleset is executed in -parallel to the first one.</p> -<textarea rows="30" cols="80">$ModLoad omruleset - -# define "second" ruleset -$RuleSet nested -$RulesetCreateMainQueue on # again, we use our own queue -mail.* /path/to/mailerr.log -kernel.* /path/to/kernelerr.log -auth.* /path/to/autherr.log - -#switch back to default ruleset -$ruleset RSYSLOG_DefaultRuleset - -# begin first action - here we filter on "error" -# note that we must first specify which ruleset to use for omruleset: -$ActionOmrulesetRulesetName nested -:msg, contains, "error :omruleset: -#end first action - -# begin second action - as an example we can do anything else in -# this processing. Note that these actions are processed concurrently -# to the ruleset "nested" -:FROMHOST, isequal, "myhost.example.com" /path/to/host.log -#end second action - -# of course, we can have "regular" actions alongside :omrulset: actions -*.* /path/to/general-message-file.log -</textarea> -<p><b>Caveats/Known Bugs:</b> -<p>The current configuration file language is not really adequate for a complex construct -like omruleset. Unfortunately, more important work is currently preventing me from redoing the -config language. So use extreme care when nesting rulesets and be sure to test-run your -config before putting it into production, ensuring you have a suffciently large probe -of the traffic run over it. If problems arise, the -<a href="troubleshoot.html">rsyslog debug log</a> is your friend. -<p>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>] -[<a href="manual.html">manual index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> -<p><font size="2">This documentation is part of the -<a href="http://www.rsyslog.com/">rsyslog</a> -project.<br> -Copyright © 2009 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and -<a href="http://www.adiscon.com/">Adiscon</a>. -Released under the GNU GPL version 3 or higher.</font></p> -</body></html> diff --git a/doc/omsnmp.html b/doc/omsnmp.html deleted file mode 100644 index 202bb5b..0000000 --- a/doc/omsnmp.html +++ /dev/null @@ -1,323 +0,0 @@ -<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> -<html><head> -<meta http-equiv="Content-Language" content="en"> -<title>SNMP Output Module</title></head> - -<body> -<a href="rsyslog_conf_modules.html">back</a> - -<h1>SNMP Output Module</h1> -<p><b>Module Name: omsnmp</b></p> -<p><b>Author: Andre Lorbach <alorbach@adiscon.com></b></p> -<p><b>Description</b>:</p> -<p>Provides the ability to send syslog messages as an SNMPv1 & v2c traps. By -default, SNMPv2c is preferred. The syslog message is wrapped into a OCTED -STRING variable. This module uses the <a target="_blank" href="http://net-snmp.sourceforge.net/"> -NET-SNMP</a> library. In order to compile this module, you will need to have the -<a target="_blank" href="http://net-snmp.sourceforge.net/">NET-SNMP</a> -developer (headers) package installed. </p> -<p> </p> -<p><b>Action Line:</b></p> -<p>%omsnmp% without any further parameters.</p> -<p> </p> -<p><b>Configuration Directives</b>:</p> -<ul> - <li><strong>transport </strong>(This parameter is optional, the - default value is "udp")<br> - <br> - Defines the transport type you wish to use. Technically we can support all - transport types which are supported by NET-SNMP. <br> - To name a few possible values: <br> - <br> - udp, tcp, udp6, tcp6, icmp, icmp6 ...<br> - <br> - Example: <strong>transport udp<br> - </strong></li> - <li><strong>server</strong><br> - <br> - This can be a hostname or ip address, and is our snmp target host. This - parameter is required, if the snmptarget is not defined, nothing will be - send. <br> - <br> - Example: <strong>server server.domain.xxx</strong><br> - </li> - <li><strong>port </strong>(This parameter is optional, the - default value is "162")<br> - <br> - The port which will be used, common values are port 162 or 161. <br> - <br> - Example: <strong>port 162</strong><br> - </li> - <li><strong>version </strong>(This parameter is optional, the - default value is "1")<br> - <br> - There can only be two choices for this parameter for now. <br> - 0 means SNMPv1 will be used.<br> - 1 means SNMPv2c will be used. <br> - Any other value will default to 1. <br> - <br> - Example: <strong>version 1</strong><br> - </li> - <li><strong>community </strong>(This parameter is optional, the - default value is "public")<br> - <br> - This sets the used SNMP Community.<br> - <br> - Example:<strong> community public<br> - </strong><br> - </li> - <li><strong>trapoid </strong>(This parameter is - optional, the default value is "1.3.6.1.4.1.19406.1.2.1" which means - "ADISCON-MONITORWARE-MIB::syslogtrap")<br> - This configuration parameter is used for <strong>SNMPv2</strong> only.<br> - <br> - This is the OID which defines the trap-type, or notifcation-type rsyslog - uses to send the trap. <br> - In order to decode this OID, you will need to have the - ADISCON-MONITORWARE-MIB and ADISCON-MIB mibs installed on the receiver side. Downloads of these mib files - can be found here: <br> - <a href="http://www.adiscon.org/download/ADISCON-MIB.txt"> - http://www.adiscon.org/download/ADISCON-MIB.txt</a><br> - <a href="http://www.adiscon.org/download/ADISCON-MONITORWARE-MIB.txt"> - http://www.adiscon.org/download/ADISCON-MONITORWARE-MIB.txt</a><br> - <br> - Thanks to the net-snmp - mailinglist for the help and the recommendations ;).<br> - <br> - Example: <strong>trapoid 1.3.6.1.4.1.19406.1.2.1<br> - </strong>If you have this MIBS installed, you can also configured with the - OID Name: <strong>trapoid ADISCON-MONITORWARE-MIB::syslogtrap<br> - </strong> - </li> - <li><strong>messageoid </strong>(This parameter is - optional, the default value is "1.3.6.1.4.1.19406.1.1.2.1" which means - "ADISCON-MONITORWARE-MIB::syslogMsg")<br> - <br> - This OID will be used as a variable, type "OCTET STRING". This variable will - contain up to 255 characters of the original syslog message including syslog header. It is recommend to - use the default OID. <br> - In order to decode this OID, you will need to have the - ADISCON-MONITORWARE-MIB and ADISCON-MIB mibs installed on the receiver side. - To download these custom mibs, see the description of <strong>$actionsnmptrapoid. - </strong><br> - <br> - Example: <strong>messageoid 1.3.6.1.4.1.19406.1.1.2.1<br> - </strong>If you have this MIBS installed, you can also configured with the - OID Name: <strong>messageoid - ADISCON-MONITORWARE-MIB::syslogMsg<br> - </strong><br> - </li> - <li><strong>enterpriseoid </strong>(This parameter is optional, - the default value is "1.3.6.1.4.1.3.1.1" which means "enterprises.cmu.1.1")<br> - <br> - Customize this value if needed. I recommend to use the default value unless - you require to use a different OID. <br> - This configuration parameter is used for <strong>SNMPv1</strong> only. It - has no effect if <strong>SNMPv2</strong> is used. <br> - <br> - Example: <strong>enterpriseoid 1.3.6.1.4.1.3.1.1 <br> - </strong><br> - </li> - <li><strong>specifictype </strong>(This parameter is optional, - the default value is "0")<strong> </strong><br> - <br> - This is the specific trap number. This configuration parameter is used for - <strong>SNMPv1</strong> only. It has no effect if <strong>SNMPv2</strong> is - used. <br> - <br> - Example: <strong>specifictype 0<br> - </strong><br> - </li> - <li><strong>traptype</strong> (This parameter is optional, the - default value is "6" which means SNMP_TRAP_ENTERPRISESPECIFIC) <br> - <br> - There are only 7 Possible trap types defined which can be used here. These - trap types are: <br> - 0 = SNMP_TRAP_COLDSTART<br> - 1 = SNMP_TRAP_WARMSTART<br> - 2 = SNMP_TRAP_LINKDOWN<br> - 3 = SNMP_TRAP_LINKUP<br> - 4 = SNMP_TRAP_AUTHFAIL<br> - 5 = SNMP_TRAP_EGPNEIGHBORLOSS<br> - 6 = SNMP_TRAP_ENTERPRISESPECIFIC<br> - <br> - Any other value will default to 6 automatically. This configuration - parameter is used for <strong>SNMPv1</strong> only. It has no effect if - <strong>SNMPv2</strong> is used. <br> - <br> - Example: <strong>traptype 6</strong><br> - </li> - <li><strong>template </strong>[templateName]<strong> </strong><br> - <br> - sets a new default template for file actions. - </li> -</ul> -<p> </p> -<p><b>Caveats/Known Bugs:</b></p><ul><li>In order to decode the custom OIDs, you - will need to have the adiscon mibs installed. </li></ul> -<p><b>Sample:</b></p> -<p>The following commands send every message as a snmp trap.</p> -<textarea rows="10" cols="60">Module (path="omsnmp") -*.* action( type="omsnmp" -transport="udp" -target="localhost" -targetport="162" -version="1" -community="public") - -</textarea> - -<p><b>Legacy Configuration Directives</b>:</p> -<ul> - <li><strong>$actionsnmptransport </strong>(This parameter is optional, the - default value is "udp")<br> - <br> - Defines the transport type you wish to use. Technically we can support all - transport types which are supported by NET-SNMP. <br> - To name a few possible values: <br> - <br> - udp, tcp, udp6, tcp6, icmp, icmp6 ...<br> - <br> - Example: <strong>$actionsnmptransport udp<br> - </strong></li> - <li><strong>$actionsnmptarget</strong><br> - <br> - This can be a hostname or ip address, and is our snmp target host. This - parameter is required, if the snmptarget is not defined, nothing will be - send. <br> - <br> - Example: <strong>$actionsnmptarget server.domain.xxx</strong><br> - </li> - <li><strong>$actionsnmptargetport </strong>(This parameter is optional, the - default value is "162")<br> - <br> - The port which will be used, common values are port 162 or 161. <br> - <br> - Example: <strong>$actionsnmptargetport 162</strong><br> - </li> - <li><strong>$actionsnmpversion </strong>(This parameter is optional, the - default value is "1")<br> - <br> - There can only be two choices for this parameter for now. <br> - 0 means SNMPv1 will be used.<br> - 1 means SNMPv2c will be used. <br> - Any other value will default to 1. <br> - <br> - Example: <strong>$actionsnmpversion 1</strong><br> - </li> - <li><strong>$actionsnmpcommunity </strong>(This parameter is optional, the - default value is "public")<br> - <br> - This sets the used SNMP Community.<br> - <br> - Example:<strong> $actionsnmpcommunity public<br> - </strong><br> - </li> - <li><strong>$actionsnmptrapoid </strong>(This parameter is - optional, the default value is "1.3.6.1.4.1.19406.1.2.1" which means - "ADISCON-MONITORWARE-MIB::syslogtrap")<br> - This configuration parameter is used for <strong>SNMPv2</strong> only.<br> - <br> - This is the OID which defines the trap-type, or notifcation-type rsyslog - uses to send the trap. <br> - In order to decode this OID, you will need to have the - ADISCON-MONITORWARE-MIB and ADISCON-MIB mibs installed on the receiver side. Downloads of these mib files - can be found here: <br> - <a href="http://www.adiscon.org/download/ADISCON-MIB.txt"> - http://www.adiscon.org/download/ADISCON-MIB.txt</a><br> - <a href="http://www.adiscon.org/download/ADISCON-MONITORWARE-MIB.txt"> - http://www.adiscon.org/download/ADISCON-MONITORWARE-MIB.txt</a><br> - <br> - Thanks to the net-snmp - mailinglist for the help and the recommendations ;).<br> - <br> - Example: <strong>$actionsnmptrapoid 1.3.6.1.4.1.19406.1.2.1<br> - </strong>If you have this MIBS installed, you can also configured with the - OID Name: <strong>$actionsnmptrapoid ADISCON-MONITORWARE-MIB::syslogtrap<br> - </strong> - </li> - <li><strong>$actionsnmpsyslogmessageoid </strong>(This parameter is - optional, the default value is "1.3.6.1.4.1.19406.1.1.2.1" which means - "ADISCON-MONITORWARE-MIB::syslogMsg")<br> - <br> - This OID will be used as a variable, type "OCTET STRING". This variable will - contain up to 255 characters of the original syslog message including syslog header. It is recommend to - use the default OID. <br> - In order to decode this OID, you will need to have the - ADISCON-MONITORWARE-MIB and ADISCON-MIB mibs installed on the receiver side. - To download these custom mibs, see the description of <strong>$actionsnmptrapoid. - </strong><br> - <br> - Example: <strong>$actionsnmpsyslogmessageoid 1.3.6.1.4.1.19406.1.1.2.1<br> - </strong>If you have this MIBS installed, you can also configured with the - OID Name: <strong>$actionsnmpsyslogmessageoid - ADISCON-MONITORWARE-MIB::syslogMsg<br> - </strong><br> - </li> - <li><strong>$actionsnmpenterpriseoid </strong>(This parameter is optional, - the default value is "1.3.6.1.4.1.3.1.1" which means "enterprises.cmu.1.1")<br> - <br> - Customize this value if needed. I recommend to use the default value unless - you require to use a different OID. <br> - This configuration parameter is used for <strong>SNMPv1</strong> only. It - has no effect if <strong>SNMPv2</strong> is used. <br> - <br> - Example: <strong>$actionsnmpenterpriseoid 1.3.6.1.4.1.3.1.1 <br> - </strong><br> - </li> - <li><strong>$actionsnmpspecifictype </strong>(This parameter is optional, - the default value is "0")<strong> </strong><br> - <br> - This is the specific trap number. This configuration parameter is used for - <strong>SNMPv1</strong> only. It has no effect if <strong>SNMPv2</strong> is - used. <br> - <br> - Example: <strong>$actionsnmpspecifictype 0<br> - </strong><br> - </li> - <li><strong>$actionsnmptraptype</strong> (This parameter is optional, the - default value is "6" which means SNMP_TRAP_ENTERPRISESPECIFIC) <br> - <br> - There are only 7 Possible trap types defined which can be used here. These - trap types are: <br> - 0 = SNMP_TRAP_COLDSTART<br> - 1 = SNMP_TRAP_WARMSTART<br> - 2 = SNMP_TRAP_LINKDOWN<br> - 3 = SNMP_TRAP_LINKUP<br> - 4 = SNMP_TRAP_AUTHFAIL<br> - 5 = SNMP_TRAP_EGPNEIGHBORLOSS<br> - 6 = SNMP_TRAP_ENTERPRISESPECIFIC<br> - <br> - Any other value will default to 6 automatically. This configuration - parameter is used for <strong>SNMPv1</strong> only. It has no effect if - <strong>SNMPv2</strong> is used. <br> - <br> - Example: <strong>$actionsnmptraptype 6</strong><br> - </li> -</ul> -<p> </p> -<p><b>Caveats/Known Bugs:</b></p><ul><li>In order to decode the custom OIDs, you - will need to have the adiscon mibs installed. </li></ul> -<p><b>Sample:</b></p> -<p>The following commands send every message as a snmp trap.</p> -<textarea rows="10" cols="60">$ModLoad omsnmp - -$actionsnmptransport udp -$actionsnmptarget localhost -$actionsnmptargetport 162 -$actionsnmpversion 1 -$actionsnmpcommunity public - -*.* :omsnmp: -</textarea> - - -<p>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>] [<a href="manual.html">manual -index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> -<p><font size="2">This documentation is part of the -<a href="http://www.rsyslog.com/">rsyslog</a> project.<br> -Copyright © 2008 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and -<a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL -version 3 or higher.</font></p> - -</body></html> diff --git a/doc/omstdout.html b/doc/omstdout.html deleted file mode 100644 index 0bd10cf..0000000 --- a/doc/omstdout.html +++ /dev/null @@ -1,42 +0,0 @@ -<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> -<html><head> -<meta http-equiv="Content-Language" content="en"> -<title>stdout output module (omstdout)</title> -</head> -<body> -<a href="rsyslog_conf_modules.html">rsyslog module reference</a> - -<h1>stdout output module (stdout)</h1> -<p><b>Module Name: omstdout</b></p> -<p><b>Author: </b>Rainer Gerhards -<rgerhards@adiscon.com></p> -<p><b>Available Since</b>: 4.1.6</p> -<p><b>Description</b>:</p> -<p>This module writes any messages that are passed to it to stdout. -It was developed for the rsyslog test suite. However, there may -(limited) other uses exists. Please not that we do not put too much -effort into the quality of this module as we do not expect it to -be used in real deployments. If you do, please drop us a note so -that we can enhance its priority! -<p><b>Configuration Directives</b>:</p> -<ul> -<li><b>$ActionOMStdoutArrayInterface</b> [on|<b>off</b><br> -This setting instructs omstdout to use the alternate -array based method of parameter passing. If used, the values -will be output with commas between the values but no other padding bytes. -This is a test aid for the alternate calling interface. -<li><b>$ActionOMStdoutEnsureLFEnding</b> [<b>on</b>|off<br> -Makes sure that each message is written with a terminating LF. This is needed for -the automatted tests. If the message contains a trailing LF, none is added. -</ul> -<b>Caveats/Known Bugs:</b> -<p>Currently none known. -<p>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>] -[<a href="manual.html">manual index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> -<p><font size="2">This documentation is part of the -<a href="http://www.rsyslog.com/">rsyslog</a> -project.<br> -Copyright © 2009 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and -<a href="http://www.adiscon.com/">Adiscon</a>. -Released under the GNU GPL version 3 or higher.</font></p> -</body></html> diff --git a/doc/omudpspoof.html b/doc/omudpspoof.html deleted file mode 100644 index 930412c..0000000 --- a/doc/omudpspoof.html +++ /dev/null @@ -1,207 +0,0 @@ -<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> -<html><head> -<title>UDP spoofing output module (omudpspoof)</title> -</head> -<body> -<a href="rsyslog_conf_modules.html">rsyslog module reference</a> - -<h1>UDP spoofing output module (omudpspoof)</h1> -<p><b>Module Name: omstdout</b></p> -<p><b>Authors: </b>Rainer Gerhards <rgerhards@adiscon.com> -and David Lang <david@lang.hm> -</p> -<p><b>Available Since</b>: 5.1.3 / v7 config since 7.2.5</p> -<p><b>Description</b>:</p> -<p>This module is similar to the regular UDP forwarder, but permits to -spoof the sender address. Also, it enables to circle through a number of -source ports. -<p><b>Important:</b> This module requires root priveleges for its low-level -socket access. As such, the <b>module will not work if rsyslog is configured to -drop privileges</b>. - -<p><b>load() Parameters</b>:</p> -<ul> - <li><strong>Template </strong>[templateName]<br> - sets a non-standard default template for this module.<br></li> - -</ul> -<p> </p> -<p><b>action() parameters</b>:</p> -<ul> - <li><strong>Target </strong>string<br> - Name or IP-Address of the system that shall receive messages. Any resolvable name is fine. <br></li><br> - - <li><strong>Port </strong>[Integer, Default 514]<br> - Name or numerical value of port to use when connecting to target. <br></li><br> - - <li><b>Template</b>[Word]<br> - Template to use as message text. - <br></li><br> - - <li><strong>SourceTemplate </strong>[Word]<br> - This is the name of the template that contains a - numerical IP address that is to be used as the source system IP address. - While it may often be a constant value, it can be generated as usual via the - property replacer, as long as it is a valid IPv4 address. If not specified, the - build-in default template RSYSLOG_omudpspoofDfltSourceTpl is used. This template is defined - as follows:<br> - template(name="RSYSLOG_omudpspoofDfltSourceTpl" type="string" string="%fromhost-ip%")<br> - So in essence, the default template spoofs the address of the system the message - was received from. This is considered the most important use case. - <br></li><br> - - <li><b>SourcePortStart</b>[Word]<br> - Specifies the start value for circeling the source ports. Must be less than or - equal to the end value. Default is 32000. - <br></li><br> - - <li><b>SourcePortEnd</b>[Word]<br> - Specifies the ending value for circeling the source ports. Must be less than or - equal to the start value. Default is 42000. - <br></li><br> - - <li><b>mtu</b>[Integer, default 1500]<br> - Maximum MTU supported by the network. Default respects Ethernet and must - usually not be adjusted. Setting a too-high MTU can lead to message loss, - too low to excess message fragmentation. Change only if you really know what - you are doing. This is always given in number of bytes. - <br></li><br> -</ul> -<p><b>pre-v7 Configuration Directives</b>:</p> -<ul> -<li><b>$ActionOMOMUDPSpoofSourceNameTemplate</b> <templatename> -- equivalent to the "sourceTemplate" parameter. -<li><b>$ActionOMUDPSpoofTargetHost</b> <hostname> - equivalent to the "target" parameter. -<li><b>$ActionOMUDPSpoofTargetPort</b> <port> - equivalent to the "target" parameter. -<li><b>$ActionOMUDPSpoofDefaultTemplate</b> <templatename> -- equivalent to the "template" load() parameter. -<li><b>$ActionOMUDPSpoofSourcePortStart</b> <number> -- equivalent to the "SourcePortStart" parameter. -<li><b>$ActionOMUDPSpoofSourcePortEnd</b> <number> -- equivalent to the "SourcePortEnd" parameter. -</ul> -<b>Caveats/Known Bugs:</b> -<ul> -<li><b>IPv6</b> is currently not supported. If you need this capability, please let us -know via the rsyslog mailing list. -<li>Versions shipped prior to rsyslog 7.2.5 do not support message sizes over 1472 bytes (more -pricesely: over the network-supported MTU). Starting with 7.2.5, those messages will be -fragmented, up to a total upper limit of 64K (induced by UDP). Message sizes over -64K will be truncated. For older versions, messages over 1472 may be totally discarded -or truncated, depending on version and environment. -</ul> - -<p><b>Config Samples</b></p> -<p>The following sample forwards all syslog messages in standard form to the -remote server server.example.com. The original sender's address is used. We do not -care about the source port. This example is considered the typical use case for -omudpspoof. -</p> -<textarea rows="3" cols="80">module(load="omudpspoof") -action(type="omudpspoof" target="server.example.com") -</textarea> - -<p>The following sample forwards all syslog messages in unmodified form to the -remote server server.example.com. The sender address 192.0.2.1 with fixed -source port 514 is used. -</p> -<textarea rows="7" cols="80">module(load="omudpspoof") -template(name="spoofaddr" type="string" string="192.0.2.1") -template(name="spooftemplate" type="string" string="%rawmsg%") -action(type="omudpspoof" target="server.example.com" - sourcetemplate="spoofaddr" template="spooftemplate" - sourceport.start="514" sourceport.end="514) -</textarea> -<p>The following sample is exatly like the previous, but it specifies a larger size -MTU. If, for example, the envrionment supports Jumbo Ethernet frames, increasing the -MTU is useful as it reduces packet fragmentation, which most often is the source of -problems. Note that setting the MTU to a value larger than the local-attached network -supports will lead to send errors and loss of message. So use with care! -</p> -<textarea rows="8" cols="80">module(load="omudpspoof") -template(name="spoofaddr" type="string" string="192.0.2.1") -template(name="spooftemplate" type="string" string="%rawmsg%") -action(type="omudpspoof" target="server.example.com" - sourcetemplate="spoofaddr" template="spooftemplate" - sourceport.start="514" sourceport.end="514 - mtu="8000") -</textarea> -<p>Of course, the action can be combined with any type of filter, for -example a tradition PRI filter:</p> -<textarea rows="8" cols="80">module(load="omudpspoof") -template(name="spoofaddr" type="string" string="192.0.2.1") -template(name="spooftemplate" type="string" string="%rawmsg%") -local0.* action(type="omudpspoof" target="server.example.com" - sourcetemplate="spoofaddr" template="spooftemplate" - sourceport.start="514" sourceport.end="514 - mtu="8000") -</textarea> -<p>... or any complex expression-based filter:</p> -<textarea rows="8" cols="80">module(load="omudpspoof") -template(name="spoofaddr" type="string" string="192.0.2.1") -template(name="spooftemplate" type="string" string="%rawmsg%") -if prifilt("local0.*") and $msg contains "error" then - action(type="omudpspoof" target="server.example.com" - sourcetemplate="spoofaddr" template="spooftemplate" - sourceport.start="514" sourceport.end="514 - mtu="8000") -</textarea> -<p>and of course it can also be combined with as many other actions -as one likes:</p> -<textarea rows="11" cols="80">module(load="omudpspoof") -template(name="spoofaddr" type="string" string="192.0.2.1") -template(name="spooftemplate" type="string" string="%rawmsg%") -if prifilt("local0.*") and $msg contains "error" then { - action(type="omudpspoof" target="server.example.com" - sourcetemplate="spoofaddr" template="spooftemplate" - sourceport.start="514" sourceport.end="514 - mtu="8000") - action(type="omfile" file="/var/log/somelog") - stop # or whatever... -} -</textarea> - -<p><b>Legacy Sample (pre-v7):</b></p> -<p>The following sample forwards all syslog messages in standard form to the -remote server server.example.com. The original sender's address is used. We do not -care about the source port. This example is considered the typical use case for -omudpspoof. -</p> -<textarea rows="5" cols="80">$ModLoad omudpspoof -$ActionOMUDPSpoofTargetHost server.example.com -*.* :omudpspoof: -</textarea> - -<p>The following sample forwards all syslog messages in unmodified form to the -remote server server.example.com. The sender address 192.0.2.1 with fixed -source port 514 is used. -</p> -<textarea rows="8" cols="80">$ModLoad omudpspoof -$template spoofaddr,"192.0.2.1" -$template spooftemplate,"%rawmsg%" -$ActionOMUDPSpoofSourceNameTemplate spoofaddr -$ActionOMUDPSpoofTargetHost server.example.com -$ActionOMUDPSpoofSourcePortStart 514 -$ActionOMUDPSpoofSourcePortEnd 514 -*.* :omudpspoof:;spooftemplate -</textarea> -<p>The following sample is similar to the previous, but uses as many defaults as possible. -In that sample, a source port in the range 32000..42000 is used. The message is formatted -according to rsyslog's canned default forwarding format. Note that if any parameters -have been changed, the previously set defaults will be used! -</p> -<textarea rows="5" cols="80">$ModLoad omudpspoof -$template spoofaddr,"192.0.2.1" -$ActionOMUDPSpoofSourceNameTemplate spoofaddr -$ActionOMUDPSpoofTargetHost server.example.com -*.* :omudpspoof: -</textarea> -<p>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>] -[<a href="manual.html">manual index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> -<p><font size="2">This documentation is part of the -<a href="http://www.rsyslog.com/">rsyslog</a> -project.<br> -Copyright © 2009-2012 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and -<a href="http://www.adiscon.com/">Adiscon</a>. -Released under the GNU GPL version 3 or higher.</font></p> -</body></html> diff --git a/doc/omusrmsg.html b/doc/omusrmsg.html deleted file mode 100644 index eccfef2..0000000 --- a/doc/omusrmsg.html +++ /dev/null @@ -1,64 +0,0 @@ -<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> -<html><head> -<meta http-equiv="Content-Language" content="en"> -<title>User Message Output Module</title></head> - -<body> -<a href="rsyslog_conf_modules.html">back</a> - -<h1>User Message Output Module</h1> -<p><b>Module Name: omusrmsg</b></p> -<p><b>Author: </b>Rainer Gerhards <rgergards@adiscon.com></p> -<p><b>Description</b>:</p> -<p>The omusrmsg plug-in provides the core functionality for logging output to a logged on user. It is a built-in module that does not need to be loaded. </p> -<p> </p> - -<p><b>Global Configuration Directives</b>:</p> -<ul> - <li><strong>Template </strong>[templateName]<br> - sets a new default template for file actions.<br></li> - -</ul> -<p> </p> -<p><b>Action specific Configuration Directives</b>:</p> -<ul> - <li><strong>Users </strong>string<br> - Must be a valid user name or root.<br></li><br> - - - -</ul> -<p><b>Caveats/Known Bugs:</b></p><ul><li>None.</li></ul> -<p><b>Sample:</b></p> -<p>The following command sends all critical syslog messages to a user and to root.</p> -<textarea rows="5" cols="60">Module (path="builtin:omusrmsg") -*.=crit action(type="omusrmsg" -Users="ExampleUser" -Users="root" -) -</textarea> - -<br><br> - -<p><b>Legacy Configuration Directives</b>:</p> -<p> - No specific configuration directives available. See configuration sample below for details on using the plugin. -</p> - -<p><b>Legacy Sample:</b></p> -<p>The following command sends all critical syslog messages to a user and to root.</p> -<textarea rows="5" cols="60">$ModLoad omusrmsg -*.=crit :omusrmsg:exampleuser -& root -</textarea> - - -<p>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>] [<a href="manual.html">manual -index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> -<p><font size="2">This documentation is part of the -<a href="http://www.rsyslog.com/">rsyslog</a> project.<br> -Copyright © 2008 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and -<a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL -version 3 or higher.</font></p> - -</body></html> diff --git a/doc/omuxsock.html b/doc/omuxsock.html deleted file mode 100644 index a1c0922..0000000 --- a/doc/omuxsock.html +++ /dev/null @@ -1,43 +0,0 @@ -<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> -<html><head><title>Unix sockets output module (omuxsock) - sending syslog messages to local socket</title> -<a href="features.html">back</a> -</head> -<body> -<h1>Unix sockets Output Module (omuxsock)</h1> -<p><b>Module Name: omuxsock</b></p> -<p><b>Available since: </b> 4.7.3, 5.5.7</p> -<p><b>Author: </b>Rainer Gerhards <rgerhards@adiscon.com></p> -<p><b>Description</b>:</p> -<p>This module supports sending syslog messages to local Unix sockets. -Thus it provided a fast message-passing interface between different rsyslog -instances. The counterpart to omuxsock is <a href="imuxsock.html">imuxsock</a>. -Note that the template used together with omuxsock must be suitable to be -processed by the receiver. -<p><b>Configuration Directives</b>:</p> -<ul> -<li><b>$OMUxSockSocket</b><br> -Name of the socket to send data to. This has no default and <b>must</b> -be set. -</li> -<li><b>$OMUxSockDefaultTemplate</b><br> -This can be used to override the default template to be used together -with omuxsock. This is primarily useful if there are many forwarding -actions and each of them should use the same template.</li> -</ul> -<b>Caveats/Known Bugs:</b> -<p>Currently, only datagram sockets are supported. -<p><b>Sample:</b></p> -<p>The following sample writes all messages to the "/tmp/socksample" socket. -</p> -<textarea rows="4" cols="80">$ModLoad omuxsock -$OMUxSockSocket /tmp/socksample -*.* :omuxsock: -</textarea> -[<a href="manual.html">manual index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> -<p><font size="2">This documentation is part of the <a href="http://www.rsyslog.com/">rsyslog</a> -project.<br> -Copyright © 2010 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and -<a href="http://www.adiscon.com/">Adiscon</a>. -Released under the GNU GPL version 3 or higher.</font></p> - -</body></html> diff --git a/doc/pmlastmsg.html b/doc/pmlastmsg.html deleted file mode 100644 index fd26dbd..0000000 --- a/doc/pmlastmsg.html +++ /dev/null @@ -1,69 +0,0 @@ -<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> -<html><head> -<meta http-equiv="Content-Language" content="en"> -<title>parser module for "last message repeated n times" (pmlastmsg)</title> -</head> -<body> -<a href="rsyslog_conf_modules.html">rsyslog module reference</a> - -<h1>parser module for "last message repeated n times" (pmlastmsg)</h1> -<p><b>Module Name: pmlastmsg</b></p> -<p><b>Module Type: parser module</b></p> -<p><b>Author: </b>Rainer Gerhards <rgerhards@adiscon.com></p> -<p><b>Available Since</b>: 5.5.6</p> -<p><b>Description</b>:</p> -<p>Some syslogds are known to emit severily malformed messages with content -"last message repeated n times". These messages can mess up message reception, as -they lead to wrong interpretation with the standard RFC3164 parser. Rather than -trying to fix this issue in pmrfc3164, we have created a new parser module -specifically for these messages. The reason is that some processing overhead is -involved in processing these messages (they must be recognized) and we would -not like to place this toll on every user but only on those actually in need -of the feature. Note that the performance toll is not large -- but if you expect -a very high message rate with tenthousands of messages per second, you will notice -a difference. -<p>This module should be loaded first inside <a href="messageparser.html">rsyslog's -parser chain</a>. It processes all those messages that contain a PRI, then none or -some spaces and then the exact text (case-insensitive) "last message repeated n times" -where n must be an integer. All other messages are left untouched. - -<p><b>Please note:</b> this parser module makes it possible that these messages -are properly detected. It does <b>not</b> drop them. If you intend to drop those -messages, you need to use the usual filter logic in combination with the discard -action. As a side-note, please keep on your mind that the sender discarded messages -when the "last message repeated n times" message is emited. You want to consider if -that really is what you intend to happen. If not, go change the sender. - -<p><b>Configuration Directives</b>:</p> -<p>There do not currently exist any configuration directives for this module. -<p><b>Examples:</b></p> -<p>This example is the typical use case, where some systems emit malformed -"repeated msg" messages. Other than that, the default RFC5424 and RFC3164 parsers -should be used. Note that when a parser is specified, the default parser chain -is removed, so we need to specify all three parsers. We use this together with the -default ruleset. -</p> -<textarea rows="15" cols="80">$ModLoad pmlastmsg # this parser is NOT a built-in module - -# note that parser are tried in the -# order they appear in rsyslog.conf, so put pmlastmsg first -$RulesetParser rsyslog.lastline -# as we have removed the default parser chain, we -# need to add the default parsers as well. -$RulesetParser rsyslog.rfc5424 -$RulesetParser rsyslog.rfc3164 - -# now come the typical rules, like... -*.* /path/to/file.log -</textarea> -<p><b>Caveats/Known Bugs:</b> -<p>currently none -<p>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>] -[<a href="manual.html">manual index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> -<p><font size="2">This documentation is part of the -<a href="http://www.rsyslog.com/">rsyslog</a> -project.<br> -Copyright © 2010 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and -<a href="http://www.adiscon.com/">Adiscon</a>. -Released under the GNU GPL version 3 or higher.</font></p> -</body></html> diff --git a/doc/property_replacer.html b/doc/property_replacer.html deleted file mode 100644 index 13ff41c..0000000 --- a/doc/property_replacer.html +++ /dev/null @@ -1,766 +0,0 @@ -<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> -<html><head><title>The Rsyslogd Property Replacer</title></head> -<body> -<a href="rsyslog_conf_templates.html">back</a> -<h1>The Property Replacer</h1> -<p><b>The property replacer is a core component in -rsyslogd's output system.</b> A syslog message has a number of -well-defined properties (see below). Each of this properties can be -accessed <b>and</b> manipulated by the property replacer. -With it, it is easy to use only part of a property value or manipulate -the value, e.g. by converting all characters to lower case.</p> -<h1>Accessing Properties</h1> -<p>Syslog message properties are used inside templates. They are -accessed by putting them between percent signs. Properties can be -modified by the property replacer. The full syntax is as follows:</p> -<blockquote><b><code>%propname:fromChar:toChar:options:fieldname%</code></b></blockquote> -<h2>Available Properties</h2> -<p><b><code>propname</code></b> is the -name of the property to access. It is case-insensitive (prior to 3.17.0, they were case-senstive). -Currently supported are:</p> -<table> -<tbody> -<tr> -<td><b>msg</b></td> -<td>the MSG part of the message (aka "the message" ;))</td> -</tr> -<tr> -<td><b>rawmsg</b></td> -<td>the message excactly as it was received from the -socket. Should be useful for debugging.</td> -</tr> -<tr> -<td><b>hostname</b></td> -<td>hostname from the message</td> -</tr> -<tr> -<td><b>source</b></td> -<td>alias for HOSTNAME</td> -</tr> -<tr> -<td><b>fromhost</b></td> -<td>hostname of the system the message was received from -(in a relay chain, this is the system immediately in front of us and -not necessarily the original sender). This is a DNS-resolved name, except -if that is not possible or DNS resolution has been disabled.</td> -</tr> -<tr> -<td><b>fromhost-ip</b></td> -<td>The same as fromhost, but alsways as an IP address. Local inputs -(like imklog) use 127.0.0.1 in this property.</td> -</tr> -<tr> -<td><b>syslogtag</b></td> -<td>TAG from the message</td> -</tr> -<tr> -<td><b>programname</b></td> -<td>the "static" part of the tag, as defined by -BSD syslogd. For example, when TAG is "named[12345]", programname is -"named".</td> -</tr> -<tr> -<td><b>pri</b></td> -<td>PRI part of the message - undecoded (single value)</td> -</tr> -<tr> -<td><b>pri-text</b></td> -<td>the PRI part of the message in textual form (e.g. "syslog.info")</td> -</tr> -<tr> -<td><b>iut</b></td> -<td>the monitorware InfoUnitType - used when talking -to a <a href="http://www.monitorware.com">MonitorWare</a> -backend (also for <a href="http://www.phplogcon.org/">phpLogCon</a>)</td> -</tr> -<tr> -<td><b>syslogfacility</b></td> -<td>the facility from the message - in numerical form</td> -</tr> -<tr> -<td><b>syslogfacility-text</b></td> -<td>the facility from the message - in text form</td> -</tr> -<tr> -<td><b>syslogseverity</b></td> -<td>severity from the message - in numerical form</td> -</tr> -<tr> -<td><b>syslogseverity-text</b></td> -<td>severity from the message - in text form</td> -</tr> -<tr> -<td><b>syslogpriority</b></td> -<td>an alias for syslogseverity - included for historical -reasons (be careful: it still is the severity, not PRI!)</td> -</tr> -<tr> -<td><b>syslogpriority-text</b></td> -<td>an alias for syslogseverity-text</td> -</tr> -<tr> -<td><b>timegenerated</b></td> -<td>timestamp when the message was RECEIVED. Always in high -resolution</td> -</tr> -<tr> -<td><b>timereported</b></td> -<td>timestamp from the message. Resolution depends on -what was provided in the message (in most cases, -only seconds)</td> -</tr> -<tr> -<td><b>timestamp</b></td> -<td>alias for timereported</td> -</tr> -<tr> -<td><b>protocol-version</b></td> -<td>The contents of the PROTCOL-VERSION field from IETF -draft draft-ietf-syslog-protcol</td> -</tr> -<tr> -<td><b>structured-data</b></td> -<td>The contents of the STRUCTURED-DATA field from IETF -draft draft-ietf-syslog-protocol</td> -</tr> -<tr> -<td><b>app-name</b></td> -<td>The contents of the APP-NAME field from IETF draft -draft-ietf-syslog-protocol</td> -</tr> -<tr> -<td><b>procid</b></td> -<td>The contents of the PROCID field from IETF draft -draft-ietf-syslog-protocol</td> -</tr> -<tr> -<td><b>msgid</b></td> -<td>The contents of the MSGID field from -IETF draft draft-ietf-syslog-protocol</td> -</tr> -<tr> -<td><b>parsesuccess</b></td> -<td>This returns the status of the <b>last</b> called higher level parser, -like mmjsonparse. A higher level parser parses the actual message for additional -structured data and maintains an extra property table while doing so (this is -often referred to as "cee data" because the idea was originally rooted in the -cee effort, only (but has been extended since then). Note that higher level -parsers must explicitely support (and set) this property. So, depending on the -parser, it may not be set correctly. -<br>If the parser properly supports it, the value "OK" means that parsing was -successfull, while "FAIL" means the parser could not successfully obtain any data. -Failure state is not necessarily an error. For example, it may simple indicate -that the cee-enhanced syslog parser (mmjsonparse) did not detect cee-enhanced format, -what can be totally valid. Using this property, further processing of the message -can be directed based on this parsing outcome. If no parser has been called at the -time this property is accessed, it will contain "FAIL". -<br><b>This property is available since version 6.3.8.</b> -</td> -</tr> -<td><b>inputname</b></td> -<td>The name of the input module that generated the -message (e.g. "imuxsock", "imudp"). Note that not all modules -necessarily provide this property. If not provided, it is an -empty string. Also note that the input module may provide -any value of its liking. Most importantly, it is <b>not</b> -necessarily the module input name. Internal sources can also -provide inputnames. Currently, "rsyslogd" is defined as inputname -for messages internally generated by rsyslogd, for example startup -and shutdown and error messages. -This property is considered useful when trying to filter messages -based on where they originated - e.g. locally generated messages -("rsyslogd", "imuxsock", "imklog") should go to a different place -than messages generated somewhere. -</td> -</tr> -<tr> -<td><b>$bom</b></td> -<td>The UTF-8 encoded Unicode byte-order mask (BOM). This may be useful -in templates for RFC5424 support, when the character set is know to be -Unicode.</td> -</tr> -<td><b>$uptime</b></td> -<td>system-uptime in seconds (as reported by operating system). -</td> -</tr> -<tr> -<td><b>$now</b></td> -<td>The current date stamp in the format YYYY-MM-DD</td> -</tr> -<tr> -<td><b>$year</b></td> -<td>The current year (4-digit)</td> -</tr> -<tr> -<td><b>$month</b></td> -<td>The current month (2-digit)</td> -</tr> -<tr> -<td><b>$day</b></td> -<td>The current day of the month (2-digit)</td> -</tr> -<tr> -<td><b>$hour</b></td> -<td>The current hour in military (24 hour) time (2-digit)</td> -</tr> -<tr> -<td><b>$hhour</b></td> -<td>The current half hour we are in. From minute 0 to 29, -this is always 0 while -from 30 to 59 it is always 1.</td> -</tr> -<tr> -<td><b>$qhour</b></td> -<td>The current quarter hour we are in. Much like $HHOUR, but values -range from 0 to 3 (for the four quater hours that are in each hour)</td> -</tr> -<tr> -<td><b>$minute</b></td> -<td>The current minute (2-digit)</td> -</tr> -<tr> -<td><b>$myhostname</b></td> -<td>The name of the current host as it knows itself (probably useful -for filtering in a generic way)</td> -</tr> -<tr> -<td><b>$!<name></b></td> -<td>This is the "bridge" to syslog message normalization (via -<a href="mmnormalize.html">mmnormalize</a>): name is a name defined -inside the normalization rule. It has the value selected by the rule -or none if no rule with this field did match. You can also use these -properties to specify JSON fields from the CEE-enhanced syslog -message, once you parse it with <a href="mmjsonparse.html">mmjsonparse</a> -</td> -</tr> -<tr> -<td><b>$!all-json</b></td> -<td>This is the JSON part of the CEE-enhanced syslog message, which -can be parsed with <a href="mmjsonparse.html">mmjsonparse</a> -</td> -</tr> -</tbody> -</table> -<p>Properties starting with a $-sign are so-called system -properties. These do NOT stem from the message but are rather -internally-generated.</p> -<h2>Legacy Character Positions</h2> -<p><b><code>FromChar</code></b> and <b><code>toChar</code></b> -are used to build substrings. They specify the offset within the string -that should be copied. Offset counting starts at 1, so if you need to -obtain the first 2 characters of the message text, you can use this -syntax: "%msg:1:2%". If you do not whish to specify from and to, but -you want to specify options, you still need to include the colons. For -example, if you would like to convert the full message text to lower -case, use "%msg:::lowercase%". If you would like to extract from a -position until the end of the string, you can place a dollar-sign ("$") -in toChar (e.g. %msg:10:$%, which will extract from position 10 to the -end of the string).</p> -<p>There is also support for <b>regular expressions</b>. -To use them, you need to place a "R" into FromChar. This tells rsyslog -that a regular expression instead of position-based extraction is -desired. The actual regular expression must then be provided in toChar. -The regular expression <b>must</b> be followed by the -string "--end". It denotes the end of the regular expression and will -not become part of it. If you are using regular expressions, the -property replacer will return the part of the property text that -matches the regular expression. An example for a property replacer -sequence with a regular expression is: "%msg:R:.*Sev:. \(.*\) -\[.*--end%"</p> -<p>It is possible to specify some parametes after the "R". These are -comma-separated. They are: -<p>R,<regexp-type>,<submatch>,<<a href="rsyslog_conf_nomatch.html">nomatch</a>>,<match-number> -<p>regexp-type is either "BRE" for Posix basic regular expressions or -"ERE" for extended ones. The string must be given in upper case. The -default is "BRE" to be consistent with earlier versions of rsyslog that -did not support ERE. The submatch identifies the submatch to be used -with the result. A single digit is supported. Match 0 is the full match, -while 1 to 9 are the acutal submatches. The match-number identifies which match to -use, if the expression occurs more than once inside the string. Please note -that the first match is number 0, the second 1 and so on. Up to 10 matches -(up to number 9) are supported. Please note that it would be more -natural to have the match-number in front of submatch, but this would break -backward-compatibility. So the match-number must be specified after "nomatch". -<p><a href="rsyslog_conf_nomatch.html">nomatch</a> specifies what should -be used in case no match is found. -<p>The following is a sample of an ERE expression that takes the first -submatch from the message string and replaces the expression with -the full field if no match is found: -<p>%msg:R,ERE,1,FIELD:for (vlan[0-9]*):--end% -<p>and this takes the first submatch of the second match of said expression: -<p>%msg:R,ERE,1,FIELD,1:for (vlan[0-9]*):--end% -<p><b>Please note: there is also a -<a href="http://www.rsyslog.com/tool-regex">rsyslog regular expression checker/generator</a> -online tool available.</b> With that tool, you can check your regular expressions and -also generate a valid property replacer sequence. Usage of this tool is recommended. -Depending on the version offered, the tool may not cover all subleties that can -be done with the property replacer. It concentrates on the most often used cases. So it -is still useful to hand-craft expressions for demanding environments. -<p><b>Also, extraction can be done based on so-called -"fields"</b>. To do so, place a "F" into FromChar. A field in its -current definition is anything that is delimited by a delimiter -character. The delimiter by default is TAB (US-ASCII value 9). However, -if can be changed to any other US-ASCII character by specifying a comma -and the <b>decimal</b> US-ASCII value of the delimiter -immediately after the "F". For example, to use comma (",") as a -delimiter, use this field specifier: "F,44". If your syslog -data is delimited, this is a quicker way to extract than via regular -expressions (actually, a *much* quicker way). Field counting starts at -1. Field zero is accepted, but will always lead to a "field not found" -error. The same happens if a field number higher than the number of -fields in the property is requested. The field number must be placed in -the "ToChar" parameter. An example where the 3rd field (delimited by -TAB) from the msg property is extracted is as follows: "%msg:F:3%". The -same example with semicolon as delimiter is "%msg:F,59:3%".</p> -<p>Please note that the special characters "F" and "R" are -case-sensitive. Only upper case works, lower case will return an error. -There are no white spaces permitted inside the sequence (that will lead -to error messages and will NOT provide the intended result).</p> -<p>Each occurence of the field delimiter starts a new field. However, -if you add a plus sign ("+") after the field delimiter, multiple -delimiters, one immediately after the others, are treated as separate -fields. This can be useful in cases where the syslog message contains -such sequences. A frequent case may be with code that is written as -follows:</p> -<code><pre> -int n, m; -... -syslog(LOG_ERR, "%d test %6d", n, m); -</pre></code> -<p>This will result into things like this in syslog messages: -"1 test 2", -"1 test 23", -"1 test 234567" -<p>As you can see, the fields are delimited by space characters, but -their exact number is unknown. They can properly be extracted as follows: -<p> -"%msg:F,32:2%" to "%msg:F,32+:2%". -<p>This feature was suggested by Zhuang Yuyao and implemented by him. -It is modeled after perl compatible regular expressions. -</p> - -<h2>Property Options</h2> -<b><code>property options</code></b> are -case-insensitive. They are available as of version 6.5.0. -Currently, the following options are defined: -<p></p> -<table> -<tbody> -<tr> -<td><b>Name</b></td> -<td>New format. Name of the template / property / constant.</td> -</tr> -<tr> -<td><b>Outname</b></td> -<td>This field permits to specify a field name for structured-data emitting property replacer options. -It is most useful to set, for example, the name for JSON-based fields (like used in ommngodb). For -text-based modules, it is simply ignored. -If not specified, the original property name is used, with the exception of properties starting with -"$!", where that prefix is removed. Note that unnamaned constants are NOT forwarded to output modules -that expect structure (like ommnogodb). To pass constants, an outname must be set. -</tr> -<tr> -<td><b>CaseConversion</b></td> -<td>New format. Additional values below.</td> -</tr> -<tr> -<td>upper</td> -<td>convert property to lowercase only</td> -</tr> -<tr> -<td>lower</td> -<td>convert property text to uppercase only</td> -</tr> -<tr> -<td><b>DateFormat</b></td> -<td>New format, additional parameter is needed. See below.</td> -</tr> -<tr> -<td>mysql</td> -<td>format as mysql date</td> -</tr> -<tr> -<td>pgsql</td> -<td>format as pgsql date</td> -</tr> -<tr> -<td>rfc3164</td> -<td>format as RFC 3164 date</td> -</tr> -<tr> -<tr> -<td valign="top">rfc3164-buggyday</td> -<td>similar to date-rfc3164, but emulates a common coding error: RFC 3164 demands -that a space is written for single-digit days. With this option, a zero is -written instead. This format seems to be used by syslog-ng and the -date-rfc3164-buggyday option can be used in migration scenarios where otherwise -lots of scripts would need to be adjusted. It is recommended <i>not</i> to use this -option when forwarding to remote hosts - they may treat the date as invalid -(especially when parsing strictly according to RFC 3164).</td> -<br><i>This feature was introduced in rsyslog 4.6.2 and v4 versions above and -5.5.3 and all versions above.</i> -</tr> -<tr> -<td>rfc3339</td> -<td>format as RFC 3339 date</td> -</tr> -<tr> -<td>unixtimestamp</td> -<td>format as unix timestamp (seconds since epoch)</td> -</tr> -<tr> -<td>subseconds</td> -<td>just the subseconds of a timestamp (always 0 for a low precision timestamp)</td> -</tr> -<tr> -<td>pos-end-relative</td> - <td>the from and to position is relative to the end of the string - instead of the usual start of string. (available since rsyslog v7.3.10) - </td> -</tr> -<tr> -<td><b>ControlCharacters</b></td> -<td>Option values for how to process control characters</td> -</tr> -<tr> -<td valign="top">escape</td> -<td>replace control characters (ASCII value 127 and values -less then 32) with an escape sequence. The sequnce is -"#<charval>" where charval is the 3-digit decimal value -of the control character. For example, a tabulator would be replaced by -"#009".<br> -Note: using this option requires that <a href="rsconf1_escapecontrolcharactersonreceive.html">$EscapeControlCharactersOnReceive</a> -is set to off.</td> -</tr> -<tr> -<td valign="top">space</td> -<td>replace control characters by spaces<br> -Note: using this option requires that <a href="rsconf1_escapecontrolcharactersonreceive.html">$EscapeControlCharactersOnReceive</a> -is set to off.</td> -</tr> -<tr> -<td valign="top">drop</td> -<td>drop control characters - the resulting string will -neither contain control characters, escape sequences nor any other -replacement character like space.<br> -Note: using this option requires that <a href="rsconf1_escapecontrolcharactersonreceive.html">$EscapeControlCharactersOnReceive</a> -is set to off.</td> -</tr> -<tr> -<td><b>SecurePath</b></td> -<td>Option values for securing path templates.</td> -</tr> -<tr> -<td valign="top">drop</td> -<td>Drops slashes inside the field (e.g. "a/b" becomes "ab"). -Useful for secure pathname generation (with dynafiles). -</td> -</tr> -<tr> -<td valign="top">replace</td> -<td>Replace slashes inside the field by an underscore. (e.g. "a/b" becomes "a_b"). -Useful for secure pathname generation (with dynafiles). -</td> -</tr> -<tr> -<td><b>Format</b></td> -<td>Option values for the general output format.</td> -</tr> -<tr> -<td>json</td> -<td>encode the value so that it can be used inside a JSON field. This means -that several characters (according to the JSON spec) are being escaped, for -example US-ASCII LF is replaced by "\n". -The json option cannot be used together with either jsonf or csv options. -</td> -</tr> -<tr> -<td>jsonf</td> -<td><i>(available in 6.3.9+)</i> -This signifies that the property should be expressed as a json <b>f</b>ield. -That means not only the property is written, but rather a complete json field in -the format<br> -"fieldname"="value"</b> -where "filedname" is the assigend field name (or the property name if none was assigned) -and value is the end result of property replacer operation. Note that value supports -all property replacer options, like substrings, case converson and the like. -Values are properly json-escaped. However, field names are (currently) not. It is -expected that proper field names are configured. -The jsonf option cannot be used together with either json or csv options. -</td> -</tr> -<tr> -<td valign="top">csv</td> -<td>formats the resulting field (after all modifications) in CSV format -as specified in <a href="http://www.ietf.org/rfc/rfc4180.txt">RFC 4180</a>. -Rsyslog will always use double quotes. Note that in order to have full CSV-formatted -text, you need to define a proper template. An example is this one: -<br>$template csvline,"%syslogtag:::csv%,%msg:::csv%" -<br>Most importantly, you need to provide the commas between the fields -inside the template. -The csv option cannot be used together with either json or jsonf options. -<br><i>This feature was introduced in rsyslog 4.1.6.</i> -</td> -</tr> -<tr> -<td><b>droplastlf</b></td> -<td>The last LF in the message (if any), is dropped. -Especially useful for PIX.</td> -</tr> -<tr> -<td valign="top"><b>spifno1stsp</b></td> -<td>This option looks scary and should probably not be used by a user. For any field -given, it returns either a single space character or no character at all. Field content -is never returned. A space is returned if (and only if) the first character of the -field's content is NOT a space. This option is kind of a hack to solve a problem rooted -in RFC 3164: 3164 specifies no delimiter between the syslog tag sequence and the actual -message text. Almost all implementation in fact delemit the two by a space. As of -RFC 3164, this space is part of the message text itself. This leads to a problem when -building the message (e.g. when writing to disk or forwarding). Should a delimiting -space be included if the message does not start with one? If not, the tag is immediately -followed by another non-space character, which can lead some log parsers to misinterpret -what is the tag and what the message. The problem finally surfaced when the klog module -was restructured and the tag correctly written. It exists with other message sources, -too. The solution was the introduction of this special property replacer option. Now, -the default template can contain a conditional space, which exists only if the -message does not start with one. While this does not solve all issues, it should -work good enough in the far majority of all cases. If you read this text and have -no idea of what it is talking about - relax: this is a good indication you will never -need this option. Simply forget about it ;) -</td> -</tr> -<tr> -<td></td> -<td></td> -</tr> -<tr> -<td><b>New character position</b></td> -<td>In addition to the above mentioned Character Positions in the legacy format, -positions can be determined by specifying the correct options for the properties. -Again, this is mostly for using the list format.</td> -</tr> -<tr> -<td>position.From</td> -<td>Character position in the property to start from.</td> -</tr> -<tr> -<td>position.To</td> -<td>Character position that determines the end for extraction. If the value is "$" -then the end of the string will be used.</td> -</tr> -<tr> -<td>field.Number</td> -<td>The number of the field, which should be used for the search operation with Regex.</td> -</tr> -<tr> -<td>field.Delimiter</td> -<td>The Character that should delimit a field. Example: ",". Everything in a -property until this character is considered a field.</td> -</tr> -<tr> -<td>regex.Expression</td> -<td>Value to be compared to property.</td> -</tr> -<tr> -<td>regex.Type</td> -<td>Values BRE or ERE</td> -</tr> -<tr> -<td>regex.NoMatchMode</td> -<td>DFLT, BLANK, ZERO, FIELD</td> -</tr> -<tr> -<td>regex.Match</td> -<td>Match to use.</td> -</tr> -<tr> -<td>regex.Submatch</td> -<td>Submatch to use. Values 0-9 whereas 0 = All</td> -</tr> -</tbody> -</table> - - - -<h2>Legacy Property Options</h2> -<b><code>property options</code></b> are -case-insensitive. Currently, the following options are defined: -<p></p> -<table> -<tbody> -<tr> -<td><b>uppercase</b></td> -<td>convert property to lowercase only</td> -</tr> -<tr> -<td><b>lowercase</b></td> -<td>convert property text to uppercase only</td> -</tr> -<tr> -<td><b>json</b></td> -<td>encode the value so that it can be used inside a JSON field. This means -that several characters (according to the JSON spec) are being escaped, for -example US-ASCII LF is replaced by "\n". -The json option cannot be used together with either jsonf or csv options. -</td> -</tr> -<tr> -<td><b>jsonf</b></td> -<td><i>(available in 6.3.9+)</i> -This signifies that the property should be expressed as a json <b>f</b>ield. -That means not only the property is written, but rather a complete json field in -the format<br> -"fieldname"="value"</b> -where "filedname" is the assigend field name (or the property name if none was assigned) -and value is the end result of property replacer operation. Note that value supports -all property replacer options, like substrings, case converson and the like. -Values are properly json-escaped. However, field names are (currently) not. It is -expected that proper field names are configured. -The jsonf option cannot be used together with either json or csv options. -</td> -</tr> -<tr> -<td valign="top"><b>csv</b></td> -<td>formats the resulting field (after all modifications) in CSV format -as specified in <a href="http://www.ietf.org/rfc/rfc4180.txt">RFC 4180</a>. -Rsyslog will always use double quotes. Note that in order to have full CSV-formatted -text, you need to define a proper template. An example is this one: -<br>$template csvline,"%syslogtag:::csv%,%msg:::csv%" -<br>Most importantly, you need to provide the commas between the fields -inside the template. -The csv option cannot be used together with either json or jsonf options. -<br><i>This feature was introduced in rsyslog 4.1.6.</i> -</td> -</tr> -<tr> -<td><b>drop-last-lf</b></td> -<td>The last LF in the message (if any), is dropped. -Especially useful for PIX.</td> -</tr> -<tr> -<td><b>date-mysql</b></td> -<td>format as mysql date</td> -</tr> -<tr> -<td><b>date-rfc3164</b></td> -<td>format as RFC 3164 date</td> -</tr> -<tr> -<tr> -<td valign="top"><b>date-rfc3164-buggyday</b></td> -<td>similar to date-rfc3164, but emulates a common coding error: RFC 3164 demands -that a space is written for single-digit days. With this option, a zero is -written instead. This format seems to be used by syslog-ng and the -date-rfc3164-buggyday option can be used in migration scenarios where otherwise -lots of scripts would need to be adjusted. It is recommended <i>not</i> to use this -option when forwarding to remote hosts - they may treat the date as invalid -(especially when parsing strictly according to RFC 3164).</td> -<br><i>This feature was introduced in rsyslog 4.6.2 and v4 versions above and -5.5.3 and all versions above.</i> -</tr> -<tr> -<td><b>date-rfc3339</b></td> -<td>format as RFC 3339 date</td> -</tr> -<tr> -<td><b>date-unixtimestamp</b></td> -<td>format as unix timestamp (seconds since epoch)</td> -</tr> -<tr> -<td><b>date-subseconds</b></td> -<td>just the subseconds of a timestamp (always 0 for a low precision timestamp)</td> -</tr> -<tr> -<td valign="top"><b>escape-cc</b></td> -<td>replace control characters (ASCII value 127 and values -less then 32) with an escape sequence. The sequnce is -"#<charval>" where charval is the 3-digit decimal value -of the control character. For example, a tabulator would be replaced by -"#009".<br> -Note: using this option requires that <a href="rsconf1_escapecontrolcharactersonreceive.html">$EscapeControlCharactersOnReceive</a> -is set to off.</td> -</tr> -<tr> -<td valign="top"><b>space-cc</b></td> -<td>replace control characters by spaces<br> -Note: using this option requires that <a href="rsconf1_escapecontrolcharactersonreceive.html">$EscapeControlCharactersOnReceive</a> -is set to off.</td> -</tr> -<tr> -<td valign="top"><b>drop-cc</b></td> -<td>drop control characters - the resulting string will -neither contain control characters, escape sequences nor any other -replacement character like space.<br> -Note: using this option requires that <a href="rsconf1_escapecontrolcharactersonreceive.html">$EscapeControlCharactersOnReceive</a> -is set to off.</td> -</tr> -<tr> -<td valign="top"><b>sp-if-no-1st-sp</b></td> -<td>This option looks scary and should probably not be used by a user. For any field -given, it returns either a single space character or no character at all. Field content -is never returned. A space is returned if (and only if) the first character of the -field's content is NOT a space. This option is kind of a hack to solve a problem rooted -in RFC 3164: 3164 specifies no delimiter between the syslog tag sequence and the actual -message text. Almost all implementation in fact delemit the two by a space. As of -RFC 3164, this space is part of the message text itself. This leads to a problem when -building the message (e.g. when writing to disk or forwarding). Should a delimiting -space be included if the message does not start with one? If not, the tag is immediately -followed by another non-space character, which can lead some log parsers to misinterpret -what is the tag and what the message. The problem finally surfaced when the klog module -was restructured and the tag correctly written. It exists with other message sources, -too. The solution was the introduction of this special property replacer option. Now, -the default template can contain a conditional space, which exists only if the -message does not start with one. While this does not solve all issues, it should -work good enough in the far majority of all cases. If you read this text and have -no idea of what it is talking about - relax: this is a good indication you will never -need this option. Simply forget about it ;) -</td> -</tr> -<tr> -<td valign="top"><b>secpath-drop</b></td> -<td>Drops slashes inside the field (e.g. "a/b" becomes "ab"). -Useful for secure pathname generation (with dynafiles). -</td> -</tr> -<tr> -<td valign="top"><b>secpath-replace</b></td> -<td>Replace slashes inside the field by an underscore. (e.g. "a/b" becomes "a_b"). -Useful for secure pathname generation (with dynafiles). -</td> -</tr> -<tr> -<td><b>mandatory-field</b></td> -<td>In templates that are used for building field lists (in particular, ommongodb), include -this field, even if it is empty (or NULL). If not set, the field will be removed from -the output field set if empty. The latter is the default case. -</tr> -</tbody> -</table> -<p>To use multiple options, simply place them one after each other with a comma delmimiting -them. For example "escape-cc,sp-if-no-1st-sp". If you use conflicting options together, -the last one will override the previous one. For example, using "escape-cc,drop-cc" will -use drop-cc and "drop-cc,escape-cc" will use escape-cc mode. -<h2>Fieldname</h2> -<p><i>(available in 6.3.9+)</i> -<p>This field permits to specify a field name for structured-data emitting property replacer -options. It was initially introduced to support the "jsonf" option, for which it provides -the capability to set an alternative field name. If it is not specified, it defaults to -the property name. -<h2>Further Links</h2> -<ul> -<li>Article on "<a href="rsyslog_recording_pri.html">Recording -the Priority of Syslog Messages</a>" (describes use of templates -to record severity and facility of a message)</li> -<li><a href="rsyslog_conf.html">Configuration file -format</a>, this is where you actually use the property replacer.</li> -</ul> -<p>[<a href="manual.html">manual index</a>] -[<a href="rsyslog_conf.html">rsyslog.conf</a>] -[<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> -<p><font size="2">This documentation is part of the -<a href="http://www.rsyslog.com/">rsyslog</a> project.<br> -Copyright © 2008, 2009 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and -<a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL -version 2 or higher.</font></p> - -</body></html> diff --git a/doc/queueWorkerLogic.jpg b/doc/queueWorkerLogic.jpg Binary files differdeleted file mode 100644 index fb143c4..0000000 --- a/doc/queueWorkerLogic.jpg +++ /dev/null diff --git a/doc/queueWorkerLogic_small.jpg b/doc/queueWorkerLogic_small.jpg Binary files differdeleted file mode 100644 index 4fae6d2..0000000 --- a/doc/queueWorkerLogic_small.jpg +++ /dev/null diff --git a/doc/queue_analogy_tv.png b/doc/queue_analogy_tv.png Binary files differdeleted file mode 100644 index fedcb55..0000000 --- a/doc/queue_analogy_tv.png +++ /dev/null diff --git a/doc/queue_msg_state.jpeg b/doc/queue_msg_state.jpeg Binary files differdeleted file mode 100644 index a215f00..0000000 --- a/doc/queue_msg_state.jpeg +++ /dev/null diff --git a/doc/queues.html b/doc/queues.html deleted file mode 100644 index 75b70fb..0000000 --- a/doc/queues.html +++ /dev/null @@ -1,398 +0,0 @@ -<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> -<html><head> -<title>Understanding rsyslog queues</title></head> -<body> -<a href="rsyslog_conf_global.html">back</a> - -<h1>Understanding rsyslog Queues</h1> -<p>Rsyslog uses queues whenever two activities need to be loosely coupled. With a -queue, one part of the system "produces" something while another part "consumes" -this something. The "something" is most often syslog messages, but queues may -also be used for other purposes.</p> -<p>This document provides a good insight into technical details, operation modes -and implications. In addition to it, an -<a href="queues_analogy.html">rsyslog queue concepts overview</a> document -exists which tries to explain queues with the help of some analogies. This may -probably be a better place to start reading about queues. I assume that once you -have understood that document, the material here will be much easier to grasp -and look much more natural. -<p>The most prominent example is the main message queue. Whenever rsyslog -receives a message (e.g. locally, via UDP, TCP or in whatever else way), it -places these messages into the main message queue. Later, it is dequeued by the -rule processor, which then evaluates which actions are to be carried out. In -front of each action, there is also a queue, which potentially de-couples the -filter processing from the actual action (e.g. writing to file, database or -forwarding to another host).</p> -<h1>Where are Queues Used?</h1> -<p>Currently, queues are used for the main message queue and for the -actions.</p> -<p>There is a single main message queue inside rsyslog. Each input module -delivers messages to it. The main message queue worker filters messages based on -rules specified in rsyslog.conf and dispatches them to the individual action -queues. Once a message is in an action queue, it is deleted from the main -message queue.</p> -<p>There are multiple action queues, one for each configured action. By default, -these queues operate in direct (non-queueing) mode. Action queues are fully -configurable and thus can be changed to whatever is best for the given use case.</p> -<p>Future versions of rsyslog will most probably utilize queues at other places, -too.</p> -<p> Wherever "<i><object></i>" is used in the config file -statements, substitute "<i><object></i>" with either "MainMsg" or "Action". The -former will set main message queue -parameters, the later parameters for the next action that will be -created. Action queue parameters can not be modified once the action has been -specified. For example, to tell the main message queue to save its content on -shutdown, use <i>$MainMsgQueueSaveOnShutdown on</i>".</p> -<p>If the same parameter is specified multiple times before a queue is created, -the last one specified takes precedence. The main message queue is created after -parsing the config file and all of its potential includes. An action queue is -created each time an action selector is specified. Action queue parameters are -reset to default after an action queue has been created (to provide a clean -environment for the next action).</p> -<p>Not all queues necessarily support the full set of queue configuration -parameters, because not all are applicable. For example, in current output -module design, actions do not support multi-threading. Consequently, the number -of worker threads is fixed to one for action queues and can not be changed.</p> -<h1>Queue Modes</h1> -<p>Rsyslog supports different queue modes, some with submodes. Each of them has -specific advantages and disadvantages. Selecting the right queue mode is quite -important when tuning rsyslogd. The queue mode (aka "type") is set via the "<i>$<object>QueueType</i>" -config directive.</p> -<h2>Direct Queues</h2> -<p>Direct queues are <b>non</b>-queuing queues. A queue in direct mode does -neither queue nor buffer any of the queue elements but rather passes the element -directly (and immediately) from the producer to the consumer. This sounds strange, -but there is a good reason for this queue type.</p> -<p>Direct mode queues allow to use queues generically, even in places where -queuing is not always desired. A good example is the queue in front of output -actions. While it makes perfect sense to buffer forwarding actions or database -writes, it makes only limited sense to build up a queue in front of simple local -file writes. Yet, rsyslog still has a queue in front of every action. So for -file writes, the queue mode can simply be set to "direct", in which case no -queuing happens.</p> -<p>Please note that a direct queue also is the only queue type that passes back -the execution return code (success/failure) from the consumer to the producer. -This, for example, is needed for the backup action logic. Consequently, backup -actions require the to-be-checked action to use a "direct" mode queue.</p> -<p>To create a direct queue, use the "<i>$<object>QueueType Direct</i>" config -directive.</p> -<h2>Disk Queues</h2> -<p>Disk queues use disk drives for buffering. The important fact is that the -always use the disk and do not buffer anything in memory. Thus, the queue is -ultra-reliable, but by far the slowest mode. For regular use cases, this queue -mode is not recommended. It is useful if log data is so important that it must -not be lost, even in extreme cases.</p> -<p>When a disk queue is written, it is done in chunks. Each chunk receives its -individual file. Files are named with a prefix (set via the "<i>$<object>QueueFilename</i>" -config directive) and followed by a 7-digit number (starting at one and -incremented for each file). Chunks are 10mb by default, a different size can be -set via the"<i>$<object>QueueMaxFileSize</i>" config directive. Note that -the size limit is not a sharp one: rsyslog always writes one complete queue -entry, even if it violates the size limit. So chunks are actually a little but -(usually less than 1k) larger then the configured size. Each chunk also has a -different size for the same reason. If you observe different chunk sizes, you -can relax: this is not a problem.</p> -<p>Writing in chunks is used so that processed data can quickly be deleted and -is free for other uses - while at the same time keeping no artificial upper -limit on disk space used. If a disk quota is set (instructions further below), -be sure that the quota/chunk size allows at least two chunks to be written. -Rsyslog currently does not check that and will fail miserably if a single chunk -is over the quota.</p> -<p>Creating new chunks costs performance but provides quicker ability to free -disk space. The 10mb default is considered a good compromise between these two. -However, it may make sense to adapt these settings to local policies. For -example, if a disk queue is written on a dedicated 200gb disk, it may make sense -to use a 2gb (or even larger) chunk size.</p> -<p>Please note, however, that the disk queue by default does not update its -housekeeping structures every time it writes to disk. This is for performance -reasons. In the event of failure, data will still be lost (except when manually -is mangled with the file structures). However, disk queues can be set to write -bookkeeping information on checkpoints (every n records), so that this can be -made ultra-reliable, too. If the checkpoint interval is set to one, no data can -be lost, but the queue is exceptionally slow.</p> -<p>Each queue can be placed on a different disk for best performance and/or -isolation. This is currently selected by specifying different <i>$WorkDirectory</i> -config directives before the queue creation statement.</p> -<p>To create a disk queue, use the "<i>$<object>QueueType Disk</i>" config -directive. Checkpoint intervals can be specified via "<i>$<object>QueueCheckpointInterval</i>", -with 0 meaning no checkpoints. Note that disk-based queues can be made very reliable -by issuing a (f)sync after each write operation. Starting with version 4.3.2, this can -be requested via "<i><object>QueueSyncQueueFiles on/off</i> with the -default being off. Activating this option has a performance penalty, so it should -not be turned on without reason.</p> -<h2>In-Memory Queues</h2> -<p>In-memory queue mode is what most people have on their mind when they think -about computing queues. Here, the enqueued data elements are held in memory. -Consequently, in-memory queues are very fast. But of course, they do not survive -any program or operating system abort (what usually is tolerable and unlikely). -Be sure to use an UPS if you use in-memory mode and your log data is important -to you. Note that even in-memory queues may hold data for an infinite amount of -time when e.g. an output destination system is down and there is no reason to move -the data out of memory (lying around in memory for an extended period of time is -NOT a reason). Pure in-memory queues can't even store queue elements anywhere -else than in core memory. </p> -<p>There exist two different in-memory queue modes: LinkedList and FixedArray. -Both are quite similar from the user's point of view, but utilize different -algorithms. </p> -<p>A FixedArray queue uses a fixed, pre-allocated array that holds pointers to -queue elements. The majority of space is taken up by the actual user data -elements, to which the pointers in the array point. The pointer array itself is -comparatively small. However, it has a certain memory footprint even if the -queue is empty. As there is no need to dynamically allocate any housekeeping -structures, FixedArray offers the best run time performance (uses the least CPU -cycle). FixedArray is best if there is a relatively low number of queue elements -expected and performance is desired. It is the default mode for the main message -queue (with a limit of 10,000 elements).</p> -<p>A LinkedList queue is quite the opposite. All housekeeping structures are -dynamically allocated (in a linked list, as its name implies). This requires -somewhat more runtime processing overhead, but ensures that memory is only -allocated in cases where it is needed. LinkedList queues are especially -well-suited for queues where only occasionally a than-high number of elements -need to be queued. A use case may be occasional message burst. Memory -permitting, it could be limited to e.g. 200,000 elements which would take up -only memory if in use. A FixedArray queue may have a too large static memory -footprint in such cases.</p> -<p><b>In general, it is advised to use LinkedList mode if in doubt</b>. The -processing overhead compared to FixedArray is low and may be -outweigh by the reduction in memory use. Paging in most-often-unused -pointer array pages can be much slower than dynamically allocating them.</p> -<p>To create an in-memory queue, use the "<i>$<object>QueueType LinkedList</i>" -or "<i>$<object>QueueType FixedArray</i>" config directive.</p> -<h3>Disk-Assisted Memory Queues</h3> -<p>If a disk queue name is defined for in-memory queues (via <i> -$<object>QueueFileName</i>), they automatically -become "disk-assisted" (DA). In that mode, data is written to disk (and read -back) on an as-needed basis.</p> -<p>Actually, the regular memory queue (called the -"primary queue") and a disk queue (called the "DA queue") work in tandem in this -mode. Most importantly, the disk queue is activated if the primary queue is full -or needs to be persisted on shutdown. Disk-assisted queues combine the -advantages of pure memory queues with those of pure disk queues. Under normal -operations, they are very fast and messages will never touch the disk. But if -there is need to, an unlimited amount of messages can be buffered (actually -limited by free disk space only) and data can be persisted between rsyslogd runs.</p> -<p>With a DA-queue, both disk-specific and in-memory specific configuration -parameters can be set. From the user's point of view, think of a DA queue like a -"super-queue" which does all within a single queue [from the code perspective, -there is some specific handling for this case, so it is actually much like a -single object].</p> -<p>DA queues are typically used to de-couple potentially long-running and -unreliable actions (to make them reliable). For example, it is recommended to -use a disk-assisted linked list in-memory queue in front of each database and -"send via tcp" action. Doing so makes these actions reliable and de-couples -their potential low execution speed from the rest of your rules (e.g. the local -file writes). There is a howto on <a href="rsyslog_high_database_rate.html"> -massive database inserts</a> which nicely describes this use case. It may even -be a good read if you do not intend to use databases.</p> -<p>With DA queues, we do not simply write out everything to disk and then run as -a disk queue once the in-memory queue is full. A much smarter algorithm is used, -which involves a "high watermark" and a "low watermark". Both specify numbers of -queued items. If the queue size reaches high watermark elements, the queue -begins to write data elements to disk. It does so until it reaches the low water -mark elements. At this point, it stops writing until either high water mark is -reached again or the on-disk queue becomes empty, in which case the queue -reverts back to in-memory mode, only. While holding at the low watermark, new -elements are actually enqueued in memory. They are eventually written to disk, -but only if the high water mark is ever reached again. If it isn't, these items -never touch the disk. So even when a queue runs disk-assisted, there is -in-memory data present (this is a big difference to pure disk queues!).</p> -<p>This algorithm prevents unnecessary disk writes, but also leaves some -additional buffer space for message bursts. Remember that creating disk files -and writing to them is a lengthy operation. It is too lengthy to e.g. block -receiving UDP messages. Doing so would result in message loss. Thus, the queue -initiates DA mode, but still is able to receive messages and enqueue them - as -long as the maximum queue size is not reached. The number of elements between -the high water mark and the maximum queue size serves as this "emergency -buffer". Size it according to your needs, if traffic is very bursty you will -probably need a large buffer here. Keep in mind, though, that under normal -operations these queue elements will probably never be used. Setting the high -water mark too low will cause disk-assistance to be turned on more often than -actually needed.</p> -<p>The water marks can be set via the "<i>$<object>QueueHighWatermark</i>" and -"<i>$<object>QueueHighWatermark</i>" configuration file directives. Note that -these are actual numbers, not precentages. Be sure they make sense (also in -respect to "<i>$<object>QueueSize</i>"), as rsyslodg does currently not perform -any checks on the numbers provided. It is easy to screw up the system here (yes, -a feature enhancement request is filed ;)).</p> -<h1>Limiting the Queue Size</h1> -<p>All queues, including disk queues, have a limit of the number of elements -they can enqueue. This is set via the "<i>$<object>QueueSize</i>" config -parameter. Note that the size is specified in number of enqueued elements, not -their actual memory size. Memory size limits can not be set. A conservative -assumption is that a single syslog messages takes up 512 bytes on average -(in-memory, NOT on the wire, this *is* a difference).</p> -<p>Disk assisted queues are special in that they do <b>not</b> have any size -limit. The enqueue an unlimited amount of elements. To prevent running out of -space, disk and disk-assisted queues can be size-limited via the "<i>$<object>QueueMaxDiskSpace</i>" -configuration parameter. If it is not set, the limit is only available free -space (and reaching this limit is currently not very gracefully handled, so -avoid running into it!). If a limit is set, the queue can not grow larger than -it. Note, however, that the limit is approximate. The engine always writes -complete records. As such, it is possible that slightly more than the set limit -is used (usually less than 1k, given the average message size). Keeping strictly -on the limit would be a performance hurt, and thus the design decision was to -favour performance. If you don't like that policy, simply specify a slightly -lower limit (e.g. 999,999K instead of 1G).</p> -<p>In general, it is a good idea to limit the pysical disk space even if you -dedicate a whole disk to rsyslog. That way, you prevent it from running out of -space (future version will have an auto-size-limit logic, that then kicks in in -such situations).</p> -<h1>Worker Thread Pools</h1> -<p>Each queue (except in "direct" mode) has an associated pool of worker -threads. Worker threads carry out the action to be performed on the data -elements enqueued. As an actual sample, the main message queue's worker task is -to apply filter logic to each incoming message and enqueue them to the relevant -output queues (actions).</p> -<p>Worker threads are started and stopped on an as-needed basis. On a system -without activity, there may be no worker at all running. One is automatically -started when a message comes in. Similarily, additional workers are started if -the queue grows above a specific size. The "<i>$<object>QueueWorkerThreadMinimumMessages</i>" -config parameter controls worker startup. If it is set to the minimum number of -elements that must be enqueued in order to justify a new worker startup. For -example, let's assume it is set to 100. As long as no more than 100 messages are -in the queue, a single worker will be used. When more than 100 messages arrive, -a new worker thread is automatically started. Similarily, a third worker will be -started when there are at least 300 messages, a forth when reaching 400 and so -on.</p> -<p>It, however, does not make sense to have too many worker threads running in -parall. Thus, the upper limit ca be set via "<i>$<object>QueueWorkerThreads</i>". -If it, for example, is set to four, no more than four workers will ever be -started, no matter how many elements are enqueued. </p> -<p>Worker threads that have been started are kept running until an inactivity -timeout happens. The timeout can be set via "<i>$<object>QueueWorkerTimeoutThreadShutdown</i>" -and is specified in milliseconds. If you do not like to keep the workers -running, simply set it to 0, which means immediate timeout and thus immediate -shutdown. But consider that creating threads involves some overhead, and this is -why we keep them running. If you would like to never shutdown any worker -threads, specify -1 for this parameter.</p> -<h2>Discarding Messages</h2> -<p>If the queue reaches the so called "discard watermark" (a number of queued -elements), less important messages can automatically be discarded. This is in an -effort to save queue space for more important messages, which you even less like -to loose. Please note that whenever there are more than "discard watermark" -messages, both newly incoming as well as already enqueued low-priority messages -are discarded. The algorithm discards messages newly coming in and those at the -front of the queue.</p> -<p>The discard watermark is a last resort setting. It should be set sufficiently -high, but low enough to allow for large message burst. Please note that it take -effect immediately and thus shows effect promptly - but that doesn't help if the -burst mainly consist of high-priority messages...</p> -<p>The discard watermark is set via the "<i>$<object>QueueDiscardMark</i>" -directive. The priority of messages to be discarded is set via "<i>$<object>QueueDiscardSeverity</i>". -This directive accepts both the usual textual severity as well as a -numerical one. To understand it, you must be aware of the numerical -severity values. They are defined in RFC 3164:</p> -<pre> Numerical Severity<br> Code<br><br> 0 Emergency: system is unusable<br> 1 Alert: action must be taken immediately<br> 2 Critical: critical conditions<br> 3 Error: error conditions<br> 4 Warning: warning conditions<br> 5 Notice: normal but significant condition<br> 6 Informational: informational messages<br> 7 Debug: debug-level messages</pre> -<p>Anything of the specified severity and (numerically) above it is -discarded. To turn message discarding off, simply specify the discard -watermark to be higher than the queue size. An alternative is to -specify the numerical value 8 as DiscardSeverity. This is also the -default setting to prevent unintentional message loss. So if you would -like to use message discarding, you need to set" <i>$<object>QueueDiscardSeverity</i>" to an actual value.</p> -<p>An interesting application is with disk-assisted queues: if the discard -watermark is set lower than the high watermark, message discarding will start -before the queue becomes disk-assisted. This may be a good thing if you would -like to switch to disk-assisted mode only in cases where it is absolutely -unavoidable and you prefer to discard less important messages first.</p> -<h1>Filled-Up Queues</h1> -<p>If the queue has either reached its configured maximum number of entries or -disk space, it is finally full. If so, rsyslogd throttles the data element -submitter. If that, for example, is a reliable input (TCP, local log socket), -that will slow down the message originator which is a good -resolution for this scenario.</p> -<p>During throtteling, a disk-assisted queue continues to write to disk and -messages are also discarded based on severity as well as regular dequeuing and -processing continues. So chances are good the situation will be resolved by -simply throttling. Note, though, that throtteling is highly undesirable for -unreliable sources, like UDP message reception. So it is not a good thing to run -into throtteling mode at all.</p> -<p>We can not hold processing -infinitely, not even when throtteling. For example, throtteling the local -log socket too long would cause the system at whole come to a standstill. To -prevent this, rsyslogd times out after a configured period ("<i>$<object>QueueTimeoutEnqueue</i>", -specified in milliseconds) if no space becomes available. As a last resort, it -then discards the newly arrived message.</p> -<p>If you do not like throtteling, set the timeout to 0 - the message will then -immediately be discarded. If you use a high timeout, be sure you know what you -do. If a high main message queue enqueue timeout is set, it can lead to -something like a complete hang of the system. The same problem does not apply to -action queues.</p> -<h2>Rate Limiting</h2> -<p>Rate limiting provides a way to prevent rsyslogd from processing things too -fast. It can, for example, prevent overruning a receiver system.</p> -<p>Currently, there are only limited rate-limiting features available. The "<i>$<object>QueueDequeueSlowdown</i>" -directive allows to specify how long (in microseconds) dequeueing should be -delayed. While simple, it still is powerful. For example, using a -DequeueSlowdown delay of 1,000 microseconds on a UDP send action ensures that no -more than 1,000 messages can be sent within a second (actually less, as there is -also some time needed for the processing itself).</p><h2>Processing Timeframes</h2><p>Queues -can be set to dequeue (process) messages only during certain -timeframes. This is useful if you, for example, would like to transfer -the bulk of messages only during off-peak hours, e.g. when you have -only limited bandwidth on the network path the the central server.</p><p>Currently, -only a single timeframe is supported and, even worse, it can only be -specified by the hour. It is not hard to extend rsyslog's capabilities -in this regard - it was just not requested so far. So if you need more -fine-grained control, let us know and we'll probably implement it. -There are two configuration directives, both should be used together or -results are unpredictable:" <i>$<object>QueueDequeueTimeBegin <hour></i>" and "<i>$<object>QueueDequeueTimeEnd <hour></i>". The hour parameter must be specified in 24-hour format (so 10pm is 22). A use case for this parameter can be found in the <a href="http://wiki.rsyslog.com/index.php/OffPeakHours">rsyslog wiki</a>. </p> -<h2>Performance</h2> -<p>The locking involved with maintaining the queue has a potentially large -performance impact. How large this is, and if it exists at all, depends much on -the configuration and actual use case. However, the queue is able to work on -so-called "batches" when dequeueing data elements. With batches, -multiple data elements are dequeued at once (with a single locking call). -The queue dequeues all available elements up to a configured upper -limit (<i><object>DequeueBatchSize <number></i>). It is important -to note that the actual upper limit is dictated by availability. The queue engine -will never wait for a batch to fill. So even if a high upper limit is configured, -batches may consist of fewer elements, even just one, if there are no more elements -waiting in the queue. -<p>Batching -can improve performance considerably. Note, however, that it affects the -order in which messages are passed to the queue worker threads, as each worker -now receive as batch of messages. Also, the larger the batch size and the higher -the maximum number of permitted worker threads, the more main memory is needed. -For a busy server, large batch sizes (around 1,000 or even more elements) may be useful. -Please note that with batching, the main memory must hold BatchSize * NumOfWorkers -objects in memory (worst-case scenario), even if running in disk-only mode. So if you -use the default 5 workers at the main message queue and set the batch size to 1,000, you need -to be prepared that the main message queue holds up to 5,000 messages in main memory -<b>in addition</b> to the configured queue size limits! -<p>The queue object's default maximum batch size -is eight, but there exists different defaults for the actual parts of -rsyslog processing that utilize queues. So you need to check these object's -defaults. -<h2>Terminating Queues</h2> -<p>Terminating a process sounds easy, but can be complex. -Terminating a running queue is in fact the most complex operation a queue -object can perform. You don't see that from a user's point of view, but its -quite hard work for the developer to do everything in the right order.</p> -<p>The complexity arises when the queue has still data enqueued when it -finishes. Rsyslog tries to preserve as much of it as possible. As a first -measure, there is a regular queue time out ("<i>$<object>QueueTimeoutShutdown</i>", -specified in milliseconds): the queue workers are given that time period to -finish processing the queue.</p> -<p>If after that period there is still data in the queue, workers are instructed -to finish the current data element and then terminate. This essentially means -any other data is lost. There is another timeout ("<i>$<object>QueueTimeoutActionCompletion</i>", -also specified in milliseconds) that specifies how long the workers have to -finish the current element. If that timeout expires, any remaining workers are -cancelled and the queue is brought down.</p> -<p>If you do not like to lose data on shutdown, the "<i>$<object>QueueSaveOnShutdown</i>" -parameter can be set to "on". This requires either a disk or disk-assisted -queue. If set, rsyslogd ensures that any queue elements are saved to disk before -it terminates. This includes data elements there were begun being processed by -workers that needed to be cancelled due to too-long processing. For a large -queue, this operation may be lengthy. No timeout applies to a required shutdown -save.</p> -[<a href="manual.html">manual index</a>] -[<a href="rsyslog_conf.html">rsyslog.conf</a>] -[<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> -<p><font size="2">This documentation is part of the -<a href="http://www.rsyslog.com/">rsyslog</a> project.<br> -Copyright © 2008, 2009 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and -<a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL -version 3 or higher.</font></p> - -</body></html> diff --git a/doc/queues_analogy.html b/doc/queues_analogy.html deleted file mode 100644 index d7533ad..0000000 --- a/doc/queues_analogy.html +++ /dev/null @@ -1,259 +0,0 @@ -<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> -<html><head> -<title>turning lanes and rsyslog queues - an analogy</title></head> -<body> -<a href="rsyslog_conf_global.html">back</a> - -<h1>Turning Lanes and Rsyslog Queues - an Analogy</h1> -<p>If there is a single object absolutely vital to understanding the way -rsyslog works, this object is queues. Queues offer a variety of services, -including support for multithreading. While there is elaborate in-depth -documentation on the ins and outs of <a href="queues.html">rsyslog queues</a>, -some of the concepts are hard to grasp even for experienced people. I think this -is because rsyslog uses a very high layer of abstraction which includes things -that look quite unnatural, like queues that do <b>not</b> actually queue... -<p>With this document, I take a different approach: I will not describe every specific -detail of queue operation but hope to be able to provide the core idea of how -queues are used in rsyslog by using an analogy. I will compare the rsyslog data flow -with real-life traffic flowing at an intersection. -<p>But first let's set the stage for the rsyslog part. The graphic below describes -the data flow inside rsyslog: -<p align="center"><img src="dataflow.png" alt="rsyslog data flow"> -<p>Note that there is a <a href="http://www.rsyslog.com/Article350.phtml">video tutorial</a> -available on the data flow. It is not perfect, but may aid in understanding this picture. -<p>For our needs, the important fact to know is that messages enter rsyslog on "the -left side" (for example, via UDP), are being preprocessed, put into the -so-called main queue, taken off that queue, be filtered and be placed into one or -several action queues (depending on filter results). They leave rsyslog on "the -right side" where output modules (like the file or database writer) consume them. -<p>So there are always <b>two</b> stages where a message (conceptually) is queued - first -in the main queue and later on in <i>n</i> action specific queues (with <i>n</i> being the number of -actions that the message in question needs to be processed by, what is being decided -by the "Filter Engine"). As such, a message will be in at least two queues -during its lifetime (with the exception of messages being discarded by the queue itself, -but for the purpose of this document, we will ignore that possibility). -<p>Also, it is vitally -important to understand that <b>each</b> action has a queue sitting in front of it. -If you have dug into the details of rsyslog configuration, you have probably seen that -a queue mode can be set for each action. And the default queue mode is the so-called -"direct mode", in which "the queue does not actually enqueue data". -That sounds silly, but is not. It is an important abstraction that helps keep the code clean. -<p>To understand this, we first need to look at who is the active component. In our data flow, -the active part always sits to the left of the object. For example, the "Preprocessor" -is being called by the inputs and calls itself into the main message queue. That is, the queue -receiver is called, it is passive. One might think that the "Parser & Filter Engine" -is an active component that actively pulls messages from the queue. This is wrong! Actually, -it is the queue that has a pool of worker threads, and these workers pull data from the queue -and then call the passively waiting Parser and Filter Engine with those messages. So the -main message queue is the active part, the Parser and Filter Engine is passive. -<p>Let's now try an analogy analogy for this part: Think about a TV show. The show is produced -in some TV studio, from there sent (actively) to a radio tower. The radio tower passively -receives from the studio and then actively sends out a signal, which is passively received -by your TV set. In our simplified view, we have the following picture: -<p align="center"><img src="queue_analogy_tv.png" alt="rsyslog queues and TV analogy"> -<p>The lower part of the picture lists the equivalent rsyslog entities, in an abstracted way. -Every queue has a producer (in the above sample the input) and a consumer (in the above sample the Parser -and Filter Engine). Their active and passive functions are equivalent to the TV entities -that are listed on top of the rsyslog entity. For example, a rsyslog consumer can never -actively initiate reception of a message in the same way a TV set cannot actively -"initiate" a TV show - both can only "handle" (display or process) -what is sent to them. -<p>Now let's look at the action queues: here, the active part, the producer, is the -Parser and Filter Engine. The passive part is the Action Processor. The later does any -processing that is necessary to call the output plugin, in particular it processes the template -to create the plugin calling parameters (either a string or vector of arguments). From the -action queue's point of view, Action Processor and Output form a single entity. Again, the -TV set analogy holds. The Output <b>does not</b> actively ask the queue for data, but -rather passively waits until the queue itself pushes some data to it. - -<p>Armed with this knowledge, we can now look at the way action queue modes work. My analogy here -is a junction, as shown below (note that the colors in the pictures below are <b>not</b> related to -the colors in the pictures above!): -<p align="center"><img src="direct_queue0.png"> -<p>This is a very simple real-life traffic case: one road joins another. We look at -traffic on the straight road, here shown by blue and green arrows. Traffic in the -opposing direction is shown in blue. Traffic flows without -any delays as long as nobody takes turns. To be more precise, if the opposing traffic takes -a (right) turn, traffic still continues to flow without delay. However, if a car in the red traffic -flow intends to do a (left, then) turn, the situation changes: -<p align="center"><img src="direct_queue1.png"> -<p>The turning car is represented by the green arrow. It cannot turn unless there is a gap -in the "blue traffic stream". And as this car blocks the roadway, the remaining -traffic (now shown in red, which should indicate the block condition), -must wait until the "green" car has made its turn. So -a queue will build up on that lane, waiting for the turn to be completed. -Note that in the examples below I do not care that much about the properties of the -opposing traffic. That is, because its structure is not really important for what I intend to -show. Think about the blue arrow as being a traffic stream that most of the time blocks -left-turners, but from time to time has a gap that is sufficiently large for a left-turn -to complete. -<p>Our road network designers know that this may be unfortunate, and for more important roads -and junctions, they came up with the concept of turning lanes: -<p align="center"><img src="direct_queue2.png"> -<p>Now, the car taking the turn can wait in a special area, the turning lane. As such, -the "straight" traffic is no longer blocked and can flow in parallel to the -turning lane (indicated by a now-green-again arrow). - -<p>However, the turning lane offers only finite space. So if too many cars intend to -take a left turn, and there is no gap in the "blue" traffic, we end up with -this well-known situation: -<p align="center"><img src="direct_queue3.png"> -<p>The turning lane is now filled up, resulting in a tailback of cars intending to -left turn on the main driving lane. The end result is that "straight" traffic -is again being blocked, just as in our initial problem case without the turning lane. -In essence, the turning lane has provided some relief, but only for a limited amount of -cars. Street system designers now try to weight cost vs. benefit and create (costly) -turning lanes that are sufficiently large to prevent traffic jams in most, but not all -cases. -<p><b>Now let's dig a bit into the mathematical properties of turning lanes.</b> We assume that -cars all have the same length. So, units of cars, the length is alsways one (which is nice, -as we don't need to care about that factor any longer ;)). A turning lane has finite capacity of -<i>n</i> cars. As long as the number of cars wanting to take a turn is less than or eqal -to <i>n</i>, "straigth traffic" is not blocked (or the other way round, traffic -is blocked if at least <i>n + 1</i> cars want to take a turn!). We can now find an optimal -value for <i>n</i>: it is a function of the probability that a car wants to turn -and the cost of the turning lane -(as well as the probability there is a gap in the "blue" traffic, but we ignore this -in our simple sample). -If we start from some finite upper bound of <i>n</i>, we can decrease -<i>n</i> to a point where it reaches zero. But let's first look at <i>n = 1</i>, in which case exactly -one car can wait on the turning lane. More than one car, and the rest of the traffic is blocked. -Our everyday logic indicates that this is actually the lowest boundary for <i>n</i>. -<p>In an abstract view, however, <i>n</i> can be zero and that works nicely. There still can be -<i>n</i> cars at any given time on the turning lane, it just happens that this means there can -be no car at all on it. And, as usual, if we have at least <i>n + 1</i> cars wanting to turn, -the main traffic flow is blocked. True, but <i>n + 1 = 0 + 1 = 1</i> so as soon as there is any -car wanting to take a turn, the main traffic flow is blocked (remember, in all cases, I assume -no sufficiently large gaps in the opposing traffic). -<p>This is the situation our everyday perception calls "road without turning lane". -In my math model, it is a "road with turning lane of size 0". The subtle difference -is important: my math model guarantees that, in an abstract sense, there always is a turning -lane, it may just be too short. But it exists, even though we don't see it. And now I can -claim that even in my small home village, all roads have turning lanes, which is rather -impressive, isn't it? ;) -<p><b>And now we finally have arrived at rsyslog's queues!</b> Rsyslog action queues exists for -all actions just like all roads in my village have turning lanes! And as in this real-life sample, -it may be hard to see the action queues for that reason. In rsyslog, the "direct" queue -mode is the equivalent to the 0-sized turning lane. And actions queues are the equivalent to turning -lanes in general, with our real-life <i>n</i> being the maximum queue size. -The main traffic line (which sometimes is blocked) is the equivalent to the -main message queue. And the periods without gaps in the opposing traffic are equivalent to -execution time of an action. In a rough sketch, the rsyslog main and action queues look like in the -following picture. -<p align="center"><img src="direct_queue_rsyslog.png"> -<p>We need to read this picture from right to left (otherwise I would need to redo all -the graphics ;)). In action 3, you see a 0-sized turning lane, aka an action queue in "direct" -mode. All other queues are run in non-direct modes, but with different sizes greater than 0. -<p>Let us first use our car analogy: -Assume we are in a car on the main lane that wants to take turn into the "action 4" -road. We pass action 1, where a number of cars wait in the turning lane and we pass -action 2, which has a slightly smaller, but still not filled up turning lane. So we pass that -without delay, too. Then we come to "action 3", which has no turning lane. Unfortunately, -the car in front of us wants to turn left into that road, so it blocks the main lane. So, this time -we need to wait. An observer standing on the sidewalk may see that while we need to wait, there are -still some cars in the "action 4" turning lane. As such, even though no new cars can -arrive on the main lane, cars still turn into the "action 4" lane. In other words, -an observer standing in "action 4" road is unable to see that traffic on the main lane -is blocked. -<p>Now on to rsyslog: Other than in the real-world traffic example, messages in rsyslog -can - at more or less the -same time - "take turns" into several roads at once. This is done by duplicating the message -if the road has a non-zero-sized "turning lane" - or in rsyslog terms a queue that is -running in any non-direct mode. If so, a deep copy of the message object is made, that placed into -the action queue and then the initial message proceeds on the "main lane". The action -queue then pushes the duplicates through action processing. This is also the reason why a -discard action inside a non-direct queue does not seem to have an effect. Actually, it discards the -copy that was just created, but the original message object continues to flow. -<p> -In action 1, we have some entries in the action queue, as we have in action 2 (where the queue is -slightly shorter). As we have seen, new messages pass action one and two almost instantaneously. -However, when a messages reaches action 3, its flow is blocked. Now, message processing must wait -for the action to complete. Processing flow in a direct mode queue is something like a U-turn: - -<p align="center"><img src="direct_queue_directq.png" alt="message processing in an rsyslog action queue in direct mode"> -<p>The message starts to execute the action and once this is done, processing flow continues. -In a real-life analogy, this may be the route of a delivery man who needs to drop a parcel -in a side street before he continues driving on the main route. As a side-note, think of what happens -with the rest of the delivery route, at least for today, if the delivery truck has a serious accident -in the side street. The rest of the parcels won't be delivered today, will they? This is exactly how the -discard action works. It drops the message object inside the action and thus the message will no -longer be available for further delivery - but as I said, only if the discard is done in a -direct mode queue (I am stressing this example because it often causes a lot of confusion). -<p>Back to the overall scenario. We have seen that messages need to wait for action 3 to -complete. Does this necessarily mean that at the same time no messages can be processed -in action 4? Well, it depends. As in the real-life scenario, action 4 will continue to -receive traffic as long as its action queue ("turn lane") is not drained. In -our drawing, it is not. So action 4 will be executed while messages still wait for action 3 -to be completed. -<p>Now look at the overall picture from a slightly different angle: -<p align="center"><img src="direct_queue_rsyslog2.png" alt="message processing in an rsyslog action queue in direct mode"> -<p>The number of all connected green and red arrows is four - one each for action 1, 2 and 3 -(this one is dotted as action 4 was a special case) and one for the "main lane" as -well as action 3 (this one contains the sole red arrow). <b>This number is the lower bound for -the number of threads in rsyslog's output system ("right-hand part" of the main message -queue)!</b> Each of the connected arrows is a continuous thread and each "turn lane" is -a place where processing is forked onto a new thread. Also, note that in action 3 the processing -is carried out on the main thread, but not in the non-direct queue modes. -<p>I have said this is "the lower bound for the number of threads...". This is with -good reason: the main queue may have more than one worker thread (individual action queues -currently do not support this, but could do in the future - there are good reasons for that, too -but exploring why would finally take us away from what we intend to see). Note that you -configure an upper bound for the number of main message queue worker threads. The actual number -varies depending on a lot of operational variables, most importantly the number of messages -inside the queue. The number <i>t_m</i> of actually running threads is within the integer-interval -[0,confLimit] (with confLimit being the operator configured limit, which defaults to 5). -Output plugins may have more than one thread created by themselves. It is quite unusual for an -output plugin to create such threads and so I assume we do not have any of these. -Then, the overall number of threads in rsyslog's filtering and output system is -<i>t_total = t_m + number of actions in non-direct modes</i>. Add the number of -inputs configured to that and you have the total number of threads running in rsyslog at -a given time (assuming again that inputs utilize only one thread per plugin, a not-so-safe -assumption). -<p>A quick side-note: I gave the lower bound for <i>t_m</i> as zero, which is somewhat in contrast -to what I wrote at the begin of the last paragraph. Zero is actually correct, because rsyslog -stops all worker threads when there is no work to do. This is also true for the action queues. -So the ultimate lower bound for a rsyslog output system without any work to carry out actually is zero. -But this bound will never be reached when there is continuous flow of activity. And, if you are -curios: if the number of workers is zero, the worker wakeup process is actually handled within the -threading context of the "left-hand-side" (or producer) of the queue. After being -started, the worker begins to play the active queue component again. All of this, of course, -can be overridden with configuration directives. -<p>When looking at the threading model, one can simply add n lanes to the main lane but otherwise -retain the traffic analogy. This is a very good description of the actual process (think what this -means to the "turning lanes"; hint: there still is only one per action!). -<p><b>Let's try to do a warp-up:</b> I have hopefully been able to show that in rsyslog, an action -queue "sits in front of" each output plugin. Messages are received and flow, from input -to output, over various stages and two level of queues to the outputs. Actions queues are always -present, but may not easily be visible when in direct mode (where no actual queuing takes place). -The "road junction with turning lane" analogy well describes the way - and intent - of the various -queue levels in rsyslog. -<p>On the output side, the queue is the active component, <b>not</b> the consumer. As such, the consumer -cannot ask the queue for anything (like n number of messages) but rather is activated by the queue -itself. As such, a queue somewhat resembles a "living thing" whereas the outputs are -just tools that this "living thing" uses. -<p><b>Note that I left out a couple of subtleties</b>, especially when it comes -to error handling and terminating -a queue (you hopefully have now at least a rough idea why I say "terminating <b>a queue</b>" -and not "terminating an action" - <i>who is the "living thing"?</i>). An action returns -a status to the queue, but it is the queue that ultimately decides which messages can finally be -considered processed and which not. Please note that the queue may even cancel an output right in -the middle of its action. This happens, if configured, if an output needs more than a configured -maximum processing time and is a guard condition to prevent slow outputs from deferring a rsyslog -restart for too long. Especially in this case re-queuing and cleanup is not trivial. Also, note that -I did not discuss disk-assisted queue modes. The basic rules apply, but there are some additional -constraints, especially in regard to the threading model. Transitioning between actual -disk-assisted mode and pure-in-memory-mode (which is done automatically when needed) is also far from -trivial and a real joy for an implementer to work on ;). -<p>If you have not done so before, it may be worth reading the -<a href="queues.html">rsyslog queue user's guide,</a> which most importantly lists all -the knobs you can turn to tweak queue operation. -<p>[<a href="manual.html">manual index</a>] -[<a href="rsyslog_conf.html">rsyslog.conf</a>] -[<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> -<p><font size="2">This documentation is part of the -<a href="http://www.rsyslog.com/">rsyslog</a> project.<br> -Copyright © 2009 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and -<a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL -version 3 or higher.</font></p> -</body> -</html> diff --git a/doc/rainerscript.html b/doc/rainerscript.html deleted file mode 100644 index 7cbbfa9..0000000 --- a/doc/rainerscript.html +++ /dev/null @@ -1,110 +0,0 @@ -<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> -<html><head> -<meta http-equiv="Content-Language" content="en"><title>RainerScript</title> - -</head> -<body> -<h1>RainerScript</h1> -<p><b>RainerScript is a scripting language specifically -designed and well-suited -for processing network events and configuring event processors</b> -(with the most prominent sample being syslog). While RainerScript is -theoritically usable with various softwares, it currently is being -used, and developed for, rsyslog. Please note that RainerScript may not -be abreviated as rscript, because that's somebody elses trademark.</p> -<p>RainerScript is currently under development. It has its first -appearance in rsyslog 3.12.0, where it provides complex expression -support. However, this is only a very partial implementatio of the -scripting language. Due to technical restrictions, the final -implementation will have a slightly different syntax. So while you are -invited to use the full power of expresssions, you unfortunatley need -to be prepared to change your configuration files at some later points. -Maintaining backwards-compatibility at this point would cause us to -make too much compromise. Defering the release until everything is -perfect is also not a good option. So use your own judgement.</p> -<p>A formal definition of the language can be found in <a href="rscript_abnf.html">RainerScript ABNF</a>. The -rest of this document describes the language from the user's point of -view. Please note that this doc is also currently under development and -can (and will) probably improve as time progresses. If you have -questions, use the rsyslog forum. Feedback is also always welcome.</p> -<h2>Data Types</h2> -RainerScript is a typeless language. That doesn't imply you don't need -to care about types. Of course, expressions like "A" + "B" will not -return a valid result, as you can't really add two letters (to -concatenate them, use the concatenation operator &). - However, all type conversions are automatically done by the -script interpreter when there is need to do so.<br> -<h2>Expressions</h2> -The language supports arbitrary complex expressions. All usual -operators are supported. The precedence of operations is as follows -(with operations being higher in the list being carried out before -those lower in the list, e.g. multiplications are done before additions.<br> -<ul> -<li>expressions in parenthesis</li><li>not, unary minus</li><li>*, /, % (modulus, as in C)</li><li>+, -, & (string concatenation)</li><li>==, !=, <>, <, >, <=, >=, contains (strings!), startswith (strings!)</li><li>and</li><li>or</li> -</ul>For example, "not a == b" probably returns not what you intended. -The script processor will first evaluate "not a" and then compare the -resulting boolean to the value of b. What you probably intended to do -is "not (a == b)". And if you just want to test for inequality, we -highly suggest to use "!=" or "<>". Both are exactly the same and -are provided so that you can pick whichever you like best. So inquality -of a and b should be tested as "a <> b". The "not" operator -should be reserved to cases where it actually is needed to form a -complex boolean expression. In those cases, parenthesis are highly -recommended. -<h2>Lookup Tables</h2> -<p><a href="lookup_tables.html">Lookup tables</a> are a powerful construct -to obtain "class" information based on message content (e.g. to build -log file names for different server types, departments or remote -offices). -<h2>Functions</h2> -<p>RainerScript supports a currently quite limited set of functions: -<ul> -<li>getenv(str) - like the OS call, returns the value of the environment -variable, if it exists. Returns an empty string if it does not exist. -<li>strlen(str) - returns the length of the provided string -<li>tolower(str) - converts the provided string into lowercase -<li>cstr(expr) - converts expr to a string value -<li>cnum(expr) - converts expr to a number (integer) -<li>re_match(expr, re) - returns 1, if expr matches re, 0 otherwise -<li>re_extract(expr, re, match, submatch, no-found) - extracts -data from a string (property) via a regular expression match. -POSIX ERE regular expressions are used. The variable "match" contains -the number of the match to use. This permits to pick up more than the -first expression match. Submatch is the submatch to match (max 50 supported). -The "no-found" parameter specifies which string is to be returned in case when -the regular expression is not found. Note that match and submatch start with -zero. It currently is not possible to extract more than one submatch with -a single call. -<li>field(str, delim, matchnbr) - returns a field-based substring. str is the string -to search, delim is the delimiter and matchnbr is the match to search -for (the first match starts at 1). This works similar as the field based -property-replacer option. -Versions prior to 7.3.7 only support a single character as delimiter character. -Starting with version 7.3.7, a full string can be used as delimiter. If a single -character is being used as delimiter, delim is the numerical ascii value of the -field delimiter character (so that non-printable characters can by specified). If a -string is used as delmiter, a multi-character string (e.g. "#011") is to be -specified. Samples:<br> -set $!usr!field = field($msg, 32, 3); -- the third field, delimited by space<br> -set $!usr!field = field($msg, "#011", 3); -- the third field, delmited by "#011"<br> -Note that when a single character is specified as string [field($msg, ",", 3)] a -string-based extraction is done, which is more performance intense than the -equivalent single-character [field($msg, 44 ,3)] extraction. -<li>prifilt(constant) - mimics a traditional PRI-based filter (like "*.*" or -"mail.info"). The traditional filter string must be given as a <b>constant string</b>. -Dynamic string evaluation is not permitted (for performance reasons). -</ul> -<p>The following example can be used to build a dynamic filter based on some environment -variable: -<pre> -if $msg contains getenv('TRIGGERVAR') then /path/to/errfile -</pre> -<p>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>] -[<a href="manual.html">manual index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> -<p><font size="2">This documentation is part of the -<a href="http://www.rsyslog.com/">rsyslog</a> -project.<br> -Copyright © 2008-2013 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and -<a href="http://www.adiscon.com/">Adiscon</a>. -Released under the GNU GPL version 3 or higher.</font></p> -</body></html> diff --git a/doc/rfc5424layers.png b/doc/rfc5424layers.png Binary files differdeleted file mode 100644 index 70192cc..0000000 --- a/doc/rfc5424layers.png +++ /dev/null diff --git a/doc/rsconf1_abortonuncleanconfig.html b/doc/rsconf1_abortonuncleanconfig.html deleted file mode 100644 index 77526c0..0000000 --- a/doc/rsconf1_abortonuncleanconfig.html +++ /dev/null @@ -1,37 +0,0 @@ -<html> -<head> -<title>rsyslog.conf file</title> -</head> -<body> -<a href="rsyslog_conf_global.html">rsyslog.conf configuration directive</a> - -<h2>$AboortOnUncleanConfig</h2> -<p><b>Type:</b> global configuration directive</p> -<p><b>Parameter Values:</b> boolean (on/off, yes/no)</p> -<p><b>Available since:</b> 5.3.1+</p> -<p><b>Default:</b> off</p> -<p><b>Description:</b></p> -<p>This directive permits to prevent rsyslog from running when the configuration file -is not clean. "Not Clean" means there are errors or some other annoyances that rsyslgod -reports on startup. This is a user-requested feature to have a strict startup mode. Note -that with the current code base it is not always possible to differentiate between an -real error and a warning-like condition. As such, the startup will also prevented if -warnings are present. I consider this a good thing in being "strict", but I admit -there also currently is no other way of doing it. -<p><b>Caveats:</b></p> -Note that the consequences of a failed rsyslogd startup can be much more serious than a -startup with only partial configuration. For example, log data may be lost or systems that -depend on the log server in question will not be able to send logs, what in the ultimate -result could result in a system hang on those systems. Also, the local system may hang when -the local log socket has become full and is not read. There exist many such scenarios. -As such, it is strongly recommended not to turn on this directive. - -<p>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>] [<a href="manual.html">manual -index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> -<p><font size="2">This documentation is part of the -<a href="http://www.rsyslog.com/">rsyslog</a> project.<br> -Copyright © 2009 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and -<a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL -version 2 or higher.</font></p> -</body> -</html> diff --git a/doc/rsconf1_actionexeconlywhenpreviousissuspended.html b/doc/rsconf1_actionexeconlywhenpreviousissuspended.html deleted file mode 100644 index 1626b4c..0000000 --- a/doc/rsconf1_actionexeconlywhenpreviousissuspended.html +++ /dev/null @@ -1,31 +0,0 @@ -<html> -<head> -<title>rsyslog.conf file</title> -</head> -<body> -<a href="rsyslog_conf_global.html">back</a> - -<h2>$ActionExecOnlyWhenPreviousIsSuspended</h2> -<p><b>Type:</b> global configuration directive</p> -<p><b>Default:</b> off</p> -<p><b>Description:</b></p> -<p>This directive allows to specify if actions should always be executed ("off," the default) or only if the previous action is suspended ("on"). This directive works hand-in-hand with the multiple actions per selector feature. It can be used, for example, to create rules that automatically switch destination servers or databases to a (set of) backup(s), if the primary server fails. Note that this feature depends on proper implementation of the suspend feature in the output module. All built-in output modules properly support it (most importantly the database write and the syslog message forwarder).</p> -<p>This selector processes all messages it receives (*.*). It tries to forward every message to primary-syslog.example.com (via tcp). If it can not reach that server, it tries secondary-1-syslog.example.com, if that fails too, it tries secondary-2-syslog.example.com. If neither of these servers can be connected, the data is stored in /var/log/localbuffer. Please note that the secondaries and the local log buffer are only used if the one before them does not work. So ideally, /var/log/localbuffer will never receive a message. If one of the servers resumes operation, it automatically takes over processing again.</p> -<p>We strongly advise not to use repeated line reduction together with ActionExecOnlyWhenPreviousIsSuspended. It may lead to "interesting" and undesired results (but you can try it if you like).</p> -<p><b>Sample:</b></p> -<p><code><b>*.* @@primary-syslog.example.com -<br>$ActionExecOnlyWhenPreviousIsSuspended on -<br>& @@secondary-1-syslog.example.com # & is used to have more than one action for -<br>& @@secondary-2-syslog.example.com # the same selector - the mult-action feature -<br>& /var/log/localbuffer -<br>$ActionExecOnlyWhenPreviousIsSuspended off # to re-set it for the next selector </b></code></p> - -<p>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>] [<a href="manual.html">manual -index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> -<p><font size="2">This documentation is part of the -<a href="http://www.rsyslog.com/">rsyslog</a> project.<br> -Copyright © 2007 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and -<a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL -version 2 or higher.</font></p> -</body> -</html> diff --git a/doc/rsconf1_actionresumeinterval.html b/doc/rsconf1_actionresumeinterval.html deleted file mode 100644 index c036547..0000000 --- a/doc/rsconf1_actionresumeinterval.html +++ /dev/null @@ -1,32 +0,0 @@ -<html> -<head> -<title>rsyslog.conf file</title> -</head> -<body> -<a href="rsyslog_conf_global.html">back</a> - -<h2>$ActionResumeInterval</h2> -<p><b>Type:</b> global configuration directive</p> -<p><b>Default:</b> 30</p> -<p><b>Description:</b></p> -<p>Sets the ActionResumeInterval for all following actions. The interval -provided is always in seconds. Thus, multiply by 60 if you need minutes and -3,600 if you need hours (not recommended).</p> -<p>When an action is suspended (e.g. destination can not be connected), the -action is resumed for the configured interval. Thereafter, it is retried. If -multiple retires fail, the interval is automatically extended. This is to -prevent excessive ressource use for retires. After each 10 retries, the interval -is extended by itself. To be precise, the actual interval is (numRetries / 10 + -1) * $ActionResumeInterval. so after the 10th try, it by default is 60 and after -the 100th try it is 330.</p> -<p><b>Sample:</b></p> -<p><code><b>$ActionResumeInterval 30 </b></code></p> -<p>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>] [<a href="manual.html">manual -index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> -<p><font size="2">This documentation is part of the -<a href="http://www.rsyslog.com/">rsyslog</a> project.<br> -Copyright © 2007 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and -<a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL -version 2 or higher.</font></p> -</body> -</html> diff --git a/doc/rsconf1_allowedsender.html b/doc/rsconf1_allowedsender.html deleted file mode 100644 index ac39e26..0000000 --- a/doc/rsconf1_allowedsender.html +++ /dev/null @@ -1,30 +0,0 @@ -<html> -<head> -<title>rsyslog.conf file</title> -</head> -<body> -<a href="rsyslog_conf_global.html">back</a> - -<h2>$AllowedSender</h2> -<p><b>Type:</b> global configuration directive</p> -<p><b>Default:</b> all allowed</p> -<p><b>Description:</b></p> -<p>Allowed sender lists can be used to specify which remote systems are allowed to send syslog messages to rsyslogd. With them, further hurdles can be placed between an attacker and rsyslogd. If a message from a system not in the allowed sender list is received, that message is discarded. A diagnostic message is logged, so that the fact is recorded (this message can be turned off with the "-w" rsyslogd command line option).</p> -<p>Allowed sender lists can be defined for UDP and TCP senders separately. There can be as many allowed senders as needed. The syntax to specify them is:</p> -<p><code><b>$AllowedSender <protocol>, ip[/bits], ip[/bits]</b></code></p> -<p>"$AllowedSender" is the directive - it must be written exactly as shown and the $ must start at the first column of the line. "<protocol>" is either "UDP" or "TCP". It must immediately be followed by the comma, else you will receive an error message. "ip[/bits]" is a machine or network ip address as in "192.0.2.0/24" or "127.0.0.1". If the "/bits" part is omitted, a single host is assumed (32 bits or mask 255.255.255.255). "/0" is not allowed, because that would match any sending system. If you intend to do that, just remove all $AllowedSender directives. If more than 32 bits are requested with IPv4, they are adjusted to 32. For IPv6, the limit is 128 for obvious reasons. Hostnames, with and without wildcards, may also be provided. If so, the result of revers DNS resolution is used for filtering. Multiple allowed senders can be specified in a comma-delimited list. Also, multiple $AllowedSender lines can be given. They are all combined into one UDP and one TCP list. Performance-wise, it is good to specify those allowed senders with high traffic volume before those with lower volume. As soon as a match is found, no further evaluation is necessary and so you can save CPU cycles.</p> -<p>Rsyslogd handles allowed sender detection very early in the code, nearly as the first action after receiving a message. This keeps the access to potential vulnerable code in rsyslog at a minimum. However, it is still a good idea to impose allowed sender limitations via firewalling.</p> -<p><b>WARNING:</b> by UDP design, rsyslogd can not identify a spoofed sender address in UDP syslog packets. As such, a malicious person could spoof the address of an allowed sender, send such packets to rsyslogd and rsyslogd would accept them as being from the faked sender. To prevent this, use syslog via TCP exclusively. If you need to use UDP-based syslog, make sure that you do proper egress and ingress filtering at the firewall and router level.</p> -<p>Rsyslog also detects some kind of malicious reverse DNS entries. In any case, using DNS names adds an extra layer of vulnerability. We recommend to stick with hard-coded IP addresses wherever possible.</p> -<p><b>Sample:</b></p> -<p><code><b>$AllowedSender UDP, 127.0.0.1, 192.0.2.0/24, [::1]/128, *.example.net, somehost.example.com</b></code></p> - -<p>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>] [<a href="manual.html">manual -index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> -<p><font size="2">This documentation is part of the -<a href="http://www.rsyslog.com/">rsyslog</a> project.<br> -Copyright © 2007 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and -<a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL -version 2 or higher.</font></p> -</body> -</html> diff --git a/doc/rsconf1_controlcharacterescapeprefix.html b/doc/rsconf1_controlcharacterescapeprefix.html deleted file mode 100644 index 45cd923..0000000 --- a/doc/rsconf1_controlcharacterescapeprefix.html +++ /dev/null @@ -1,25 +0,0 @@ -<html> -<head> -<title>rsyslog.conf file</title> -</head> -<body> -<a href="rsyslog_conf_global.html">back</a> - -<h2>$ControlCharacterEscapePrefix</h2> -<p><b>Type:</b> global configuration directive</p> -<p><b>Default:</b> \</p> -<p><b>Description:</b></p> -<p>This option specifies the prefix character to be used for control character escaping (see option $EscapeControlCharactersOnReceive). By default, it is '\', which is backwards-compatible with sysklogd. Change it to '#' in order to be compliant to the value that is somewhat suggested by Internet-Draft syslog-protocol.</p> -<p><b>IMPORTANT</b>: do not use the ' character. This is reserved and will most probably be used in the future as a character delimiter. For the same reason, the syntax of this directive will probably change in future releases.</p> -<p><b>Sample:</b></p> -<p><code><b>$EscapeControlCharactersOnReceive # # as of syslog-protocol</b></code></p> - -<p>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>] [<a href="manual.html">manual -index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> -<p><font size="2">This documentation is part of the -<a href="http://www.rsyslog.com/">rsyslog</a> project.<br> -Copyright © 2007 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and -<a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL -version 2 or higher.</font></p> -</body> -</html> diff --git a/doc/rsconf1_debugprintcfsyslinehandlerlist.html b/doc/rsconf1_debugprintcfsyslinehandlerlist.html deleted file mode 100644 index e158de4..0000000 --- a/doc/rsconf1_debugprintcfsyslinehandlerlist.html +++ /dev/null @@ -1,24 +0,0 @@ -<html> -<head> -<title>rsyslog.conf file</title> -</head> -<body> -<a href="rsyslog_conf_global.html">back</a> - -<h2>$DebugPrintCFSyslineHandlerList</h2> -<p><b>Type:</b> global configuration directive</p> -<p><b>Default:</b> on</p> -<p><b>Description:</b></p> -<p>Specifies whether or not the configuration file sysline handler list should be written to the debug log. Possible values: on/off. Default is on. Does not affect operation if debugging is disabled.</p> -<p><b>Sample:</b></p> -<p><code><b></b></code></p> - -<p>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>] [<a href="manual.html">manual -index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> -<p><font size="2">This documentation is part of the -<a href="http://www.rsyslog.com/">rsyslog</a> project.<br> -Copyright © 2007 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and -<a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL -version 2 or higher.</font></p> -</body> -</html> diff --git a/doc/rsconf1_debugprintmodulelist.html b/doc/rsconf1_debugprintmodulelist.html deleted file mode 100644 index f25663f..0000000 --- a/doc/rsconf1_debugprintmodulelist.html +++ /dev/null @@ -1,23 +0,0 @@ -<html> -<head> -<title>rsyslog.conf file</title> -</head> -<body> -<a href="rsyslog_conf_global.html">back</a> -<h2>$DebugPrintModuleList</h2> -<p><b>Type:</b> global configuration directive</p> -<p><b>Default:</b> on</p> -<p><b>Description:</b></p> -<p>Specifies whether or not the module list should be written to the debug log. Possible values: on/off. Default is on. Does not affect operation if debugging is disabled.</p> -<p><b>Sample:</b></p> -<p><code><b></b></code></p> - -<p>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>] [<a href="manual.html">manual -index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> -<p><font size="2">This documentation is part of the -<a href="http://www.rsyslog.com/">rsyslog</a> project.<br> -Copyright © 2007 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and -<a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL -version 2 or higher.</font></p> -</body> -</html> diff --git a/doc/rsconf1_debugprinttemplatelist.html b/doc/rsconf1_debugprinttemplatelist.html deleted file mode 100644 index b5f1f28..0000000 --- a/doc/rsconf1_debugprinttemplatelist.html +++ /dev/null @@ -1,24 +0,0 @@ -<html> -<head> -<title>rsyslog.conf file</title> -</head> -<body> -<a href="rsyslog_conf_global.html">back</a> - -<h2>$DebugPrintTemplateList</h2> -<p><b>Type:</b> global configuration directive</p> -<p><b>Default:</b> on</p> -<p><b>Description:</b></p> -<p>Specifies whether or not the template list should be written to the debug log. Possible values: on/off. Default is on. Does not affect operation if debugging is disabled..</p> -<p><b>Sample:</b></p> -<p><code><b></b></code></p> - -<p>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>] [<a href="manual.html">manual -index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> -<p><font size="2">This documentation is part of the -<a href="http://www.rsyslog.com/">rsyslog</a> project.<br> -Copyright © 2007 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and -<a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL -version 2 or higher.</font></p> -</body> -</html> diff --git a/doc/rsconf1_dircreatemode.html b/doc/rsconf1_dircreatemode.html deleted file mode 100644 index b22b6c5..0000000 --- a/doc/rsconf1_dircreatemode.html +++ /dev/null @@ -1,28 +0,0 @@ -<html> -<head> -<title>rsyslog.conf file</title> -</head> -<body> -<a href="rsyslog_conf_global.html">back</a> - -<h2>$DirCreateMode</h2> -<p><b>Type:</b> global configuration directive</p> -<p><b>Default:</b> 0700</p> -<p><b>Description:</b></p> -<p>This is the same as $FileCreateMode, but for directories automatically generated.</p> -<p>Please visit the -<a target="_blank" href="http://lists.adiscon.net/pipermail/rsyslog/2009-April/001986.html">rsyslog mailing list -archive</a> -to understand why the default is so restrictive.</p> -<p><b>Sample:</b></p> -<p><code><b></b></code></p> - -<p>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>] [<a href="manual.html">manual -index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> -<p><font size="2">This documentation is part of the -<a href="http://www.rsyslog.com/">rsyslog</a> project.<br> -Copyright © 2007-2009 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and -<a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL -version 3 or higher.</font></p> -</body> -</html> diff --git a/doc/rsconf1_dirgroup.html b/doc/rsconf1_dirgroup.html deleted file mode 100644 index 4bc8692..0000000 --- a/doc/rsconf1_dirgroup.html +++ /dev/null @@ -1,24 +0,0 @@ -<html> -<head> -<title>rsyslog.conf file</title> -</head> -<body> -<a href="rsyslog_conf_global.html">back</a> - -<h2>$DirGroup</h2> -<p><b>Type:</b> global configuration directive</p> -<p><b>Default:</b> </p> -<p><b>Description:</b></p> -<p>Set the group for directories newly created. Please note that this setting does not affect the group of directories already existing. The parameter is a group name, for which the groupid is obtained by rsyslogd on during startup processing. Interim changes to the user mapping are not detected.</p> -<p><b>Sample:</b></p> -<p><code><b>$DirGroup loggroup</b></code></p> - -<p>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>] [<a href="manual.html">manual -index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> -<p><font size="2">This documentation is part of the -<a href="http://www.rsyslog.com/">rsyslog</a> project.<br> -Copyright © 2007 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and -<a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL -version 2 or higher.</font></p> -</body> -</html> diff --git a/doc/rsconf1_dirowner.html b/doc/rsconf1_dirowner.html deleted file mode 100644 index f779c00..0000000 --- a/doc/rsconf1_dirowner.html +++ /dev/null @@ -1,24 +0,0 @@ -<html> -<head> -<title>rsyslog.conf file</title> -</head> -<body> -<a href="rsyslog_conf_global.html">back</a> - -<h2>$DirOwner</h2> -<p><b>Type:</b> global configuration directive</p> -<p><b>Default:</b> </p> -<p><b>Description:</b></p> -<p>Set the file owner for directories newly created. Please note that this setting does not affect the owner of directories already existing. The parameter is a user name, for which the userid is obtained by rsyslogd during startup processing. Interim changes to the user mapping are not detected.</p> -<p><b>Sample:</b></p> -<p><code><b>$DirOwner loguser</b></code></p> - -<p>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>] [<a href="manual.html">manual -index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> -<p><font size="2">This documentation is part of the -<a href="http://www.rsyslog.com/">rsyslog</a> project.<br> -Copyright © 2007 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and -<a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL -version 2 or higher.</font></p> -</body> -</html> diff --git a/doc/rsconf1_dropmsgswithmaliciousdnsptrrecords.html b/doc/rsconf1_dropmsgswithmaliciousdnsptrrecords.html deleted file mode 100644 index 95027a7..0000000 --- a/doc/rsconf1_dropmsgswithmaliciousdnsptrrecords.html +++ /dev/null @@ -1,24 +0,0 @@ -<html> -<head> -<title>rsyslog.conf file</title> -</head> -<body> -<a href="rsyslog_conf_global.html">back</a> - -<h2>$DropMsgsWithMaliciousDnsPTRRecords</h2> -<p><b>Type:</b> global configuration directive</p> -<p><b>Default:</b> off</p> -<p><b>Description:</b></p> -<p>Rsyslog contains code to detect malicious DNS PTR records (reverse name resolution). An attacker might use specially-crafted DNS entries to make you think that a message might have originated on another IP address. Rsyslog can detect those cases. It will log an error message in any case. If this option here is set to "on", the malicious message will be completely dropped from your logs. If the option is set to "off", the message will be logged, but the original IP will be used instead of the DNS name.</p> -<p><b>Sample:</b></p> -<p><code><b>$DropMsgsWithMaliciousDnsPTRRecords on</b></code></p> - -<p>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>] [<a href="manual.html">manual -index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> -<p><font size="2">This documentation is part of the -<a href="http://www.rsyslog.com/">rsyslog</a> project.<br> -Copyright © 2007 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and -<a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL -version 2 or higher.</font></p> -</body> -</html> diff --git a/doc/rsconf1_droptrailinglfonreception.html b/doc/rsconf1_droptrailinglfonreception.html deleted file mode 100644 index fb59b87..0000000 --- a/doc/rsconf1_droptrailinglfonreception.html +++ /dev/null @@ -1,24 +0,0 @@ -<html> -<head> -<title>rsyslog.conf file</title> -</head> -<body> -<a href="rsyslog_conf_global.html">back</a> - -<h2>$DropTrailingLFOnReception</h2> -<p><b>Type:</b> global configuration directive</p> -<p><b>Default:</b> on</p> -<p><b>Description:</b></p> -<p>Syslog messages frequently have the line feed character (LF) as the last character of the message. In almost all cases, this LF should not really become part of the message. However, recent IETF syslog standardization recommends against modifying syslog messages (e.g. to keep digital signatures valid). This option allows to specify if trailing LFs should be dropped or not. The default is to drop them, which is consistent with what sysklogd does.</p> -<p><b>Sample:</b></p> -<p><code><b>$DropTrailingLFOnRecption on</b></code></p> - -<p>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>] [<a href="manual.html">manual -index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> -<p><font size="2">This documentation is part of the -<a href="http://www.rsyslog.com/">rsyslog</a> project.<br> -Copyright © 2007 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and -<a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL -version 2 or higher.</font></p> -</body> -</html> diff --git a/doc/rsconf1_dynafilecachesize.html b/doc/rsconf1_dynafilecachesize.html deleted file mode 100644 index cacbf6e..0000000 --- a/doc/rsconf1_dynafilecachesize.html +++ /dev/null @@ -1,25 +0,0 @@ -<html> -<head> -<title>rsyslog.conf file</title> -</head> -<body> -<a href="rsyslog_conf_global.html">back</a> - -<h2>$DynaFileCacheSize</h2> -<p><b>Type:</b> global configuration directive</p> -<p><b>Default:</b> 10</p> -<p><b>Description:</b></p> -<p>This directive specifies the maximum size of the cache for dynamically-generated file names. Selector lines with dynamic files names ('?' indicator) support writing to multiple files with a single selector line. This setting specifies how many open file handles should be cached. If, for example, the file name is generated with the hostname in it and you have 100 different hosts, a cache size of 100 would ensure that files are opened once and then stay open. This can be a great way to increase performance. If the cache size is lower than the number of different files, the least recently used one is discarded (and the file closed). The hardcoded maximum is 10,000 - a value that we assume should already be very extreme. Please note that if you expect to run with a very large number of files, you probably need to reconfigure the kernel to support such a large number. In practice, we do NOT recommend to use a cache of more than 1,000 entries. The cache lookup would probably require more time than the open and close operations. The minimum value is 1.</p> -<p>Numbers are always in decimal. Leading zeros should be avoided (in some later version, they may be mis-interpreted as being octal). Multiple directives may be given. They are applied to selector lines based on order of appearance.</p> -<p><b>Sample:</b></p> -<p><code><b>$DynaFileCacheSize 100 # a cache of 100 files at most</b></code></p> - -<p>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>] [<a href="manual.html">manual -index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> -<p><font size="2">This documentation is part of the -<a href="http://www.rsyslog.com/">rsyslog</a> project.<br> -Copyright © 2007 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and -<a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL -version 2 or higher.</font></p> -</body> -</html> diff --git a/doc/rsconf1_escape8bitcharsonreceive.html b/doc/rsconf1_escape8bitcharsonreceive.html deleted file mode 100644 index 408851c..0000000 --- a/doc/rsconf1_escape8bitcharsonreceive.html +++ /dev/null @@ -1,44 +0,0 @@ -<html> -<head> -<title>rsyslog.conf file</title> -</head> -<body> -<a href="rsyslog_conf_global.html">back</a> - -<h2>$Escape8BitCharactersOnReceive</h2> -<p><b>Type:</b> global configuration directive</p> -<p><b>Default:</b> off</p> -<p><b>Available Since:</b> 5.5.2</p> -<p><b>Description:</b></p> -<p>This directive instructs rsyslogd to replace non US-ASCII characters (those that -have the 8th bit set) during reception of the message. -This may be useful for some systems. -Please note that this escaping breaks Unicode and many other encodings. Most importantly, -it can be assumed that Asian and European characters will be rendered hardly readable by -this settings. However, it may still be useful when the logs themself are primarily -in English and only occasionally contain local script. -If this option is turned on, all control-characters are converted to a 3-digit octal number and be prefixed with the $ControlCharacterEscapePrefix character (being '#' by default). -<p><b>Warning:</b></p> -<ul> - <li>turning on this option most probably destroys non-western character sets - (like Japanese, Chinese and Korean) as well as European character sets.</li> - <li>turning on this option destroys digital signatures if such exists inside - the message</li> - <li>if turned on, the drop-cc, space-cc and escape-cc - <a href="property_replacer.html">property replacer</a> options do not work - as expected because control characters are already removed upon message - reception. If you intend to use these property replacer options, you must - turn off $Escape8BitCharactersOnReceive.</li> -</ul> -<p><b>Sample:</b></p> -<p><code><b>$Escape8BitCharactersOnReceive on</b></code></p> - -<p>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>] [<a href="manual.html">manual -index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> -<p><font size="2">This documentation is part of the -<a href="http://www.rsyslog.com/">rsyslog</a> project.<br> -Copyright © 2010 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and -<a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL -version 3 or higher.</font></p> -</body> -</html> diff --git a/doc/rsconf1_escapecontrolcharactersonreceive.html b/doc/rsconf1_escapecontrolcharactersonreceive.html deleted file mode 100644 index 178f9a6..0000000 --- a/doc/rsconf1_escapecontrolcharactersonreceive.html +++ /dev/null @@ -1,36 +0,0 @@ -<html> -<head> -<title>rsyslog.conf file</title> -</head> -<body> -<a href="rsyslog_conf_global.html">back</a> - -<h2>$EscapeControlCharactersOnReceive</h2> -<p><b>Type:</b> global configuration directive</p> -<p><b>Default:</b> on</p> -<p><b>Description:</b></p> -<p>This directive instructs rsyslogd to replace control characters during reception of the message. The intent is to provide a way to stop non-printable messages from entering the syslog system as whole. If this option is turned on, all control-characters are converted to a 3-digit octal number and be prefixed with the $ControlCharacterEscapePrefix character (being '\' by default). For example, if the BEL character (ctrl-g) is included in the message, it would be converted to "\007". To be compatible to sysklogd, this option must be turned on.</p> -<p><b>Warning:</b></p> -<ul> - <li>turning on this option most probably destroys non-western character sets - (like Japanese, Chinese and Korean)</li> - <li>turning on this option destroys digital signatures if such exists inside - the message</li> - <li>if turned on, the drop-cc, space-cc and escape-cc - <a href="property_replacer.html">property replacer</a> options do not work - as expected because control characters are already removed upon message - reception. If you intend to use these property replacer options, you must - turn off $EscapeControlCharactersOnReceive.</li> -</ul> -<p><b>Sample:</b></p> -<p><code><b>$EscapeControlCharactersOnReceive on</b></code></p> - -<p>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>] [<a href="manual.html">manual -index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> -<p><font size="2">This documentation is part of the -<a href="http://www.rsyslog.com/">rsyslog</a> project.<br> -Copyright © 2007 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and -<a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL -version 2 or higher.</font></p> -</body> -</html> diff --git a/doc/rsconf1_failonchownfailure.html b/doc/rsconf1_failonchownfailure.html deleted file mode 100644 index d8bbab8..0000000 --- a/doc/rsconf1_failonchownfailure.html +++ /dev/null @@ -1,24 +0,0 @@ -<html> -<head> -<title>rsyslog.conf file</title> -</head> -<body> -<a href="rsyslog_conf_global.html">back</a> - -<h2>$FailOnChownFailure</h2> -<p><b>Type:</b> global configuration directive</p> -<p><b>Default:</b> on</p> -<p><b>Description:</b></p> -<p>This option modifies behaviour of dynaFile creation. If different owners or groups are specified for new files or directories and rsyslogd fails to set these new owners or groups, it will log an error and NOT write to the file in question if that option is set to "on". If it is set to "off", the error will be ignored and processing continues. Keep in mind, that the files in this case may be (in)accessible by people who should not have permission. The default is "on".</p> -<p><b>Sample:</b></p> -<p><code><b>$FailOnChownFailure off</b></code></p> - -<p>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>] [<a href="manual.html">manual -index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> -<p><font size="2">This documentation is part of the -<a href="http://www.rsyslog.com/">rsyslog</a> project.<br> -Copyright © 2007 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and -<a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL -version 2 or higher.</font></p> -</body> -</html> diff --git a/doc/rsconf1_filecreatemode.html b/doc/rsconf1_filecreatemode.html deleted file mode 100644 index 10b0317..0000000 --- a/doc/rsconf1_filecreatemode.html +++ /dev/null @@ -1,37 +0,0 @@ -<html> -<head> -<title>rsyslog.conf file</title> -</head> -<body> -<a href="rsyslog_conf_global.html">back</a> - -<h2>$FileCreateMode</h2> -<p><b>Type:</b> global configuration directive</p> -<p><b>Default:</b> 0644</p> -<p><b>Description:</b></p> -<p>The $FileCreateMode directive allows to specify the creation mode with which rsyslogd creates new files. If not specified, the value 0644 is used (which retains backward-compatibility with earlier releases). The value given must always be a 4-digit octal number, with the initial digit being zero.</p> -<p>Please note that the actual permission depend on rsyslogd's process umask. If in doubt, use "$umask 0000" right at the beginning of the configuration file to remove any restrictions.</p> -<p>$FileCreateMode may be specified multiple times. If so, it specifies the creation mode for all selector lines that follow until the next $FileCreateMode directive. Order of lines is vitally important.</p> -<p><b>Sample:</b></p> -<p><code><b>$FileCreateMode 0600</b></code></p> -<p>This sample lets rsyslog create files with read and write access only for the users it runs under.</p> -<p>The following sample is deemed to be a complete rsyslog.conf: -<p><code><b>$umask 0000 # make sure nothing interferes with the following -definitions<br> -*.* /var/log/file-with-0644-default<br> -$FileCreateMode 0600<br> -*.* /var/log/file-with-0600<br> -$FileCreateMode 0644<br> -*.* /var/log/file-with-0644</b></code></p> -<p>As you can see, open modes depend on position in the config file. Note the -first line, which is created with the hardcoded default creation mode.</p> - -<p>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>] [<a href="manual.html">manual -index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> -<p><font size="2">This documentation is part of the -<a href="http://www.rsyslog.com/">rsyslog</a> project.<br> -Copyright © 2007 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and -<a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL -version 3 or higher.</font></p> -</body> -</html> diff --git a/doc/rsconf1_filegroup.html b/doc/rsconf1_filegroup.html deleted file mode 100644 index 935f074..0000000 --- a/doc/rsconf1_filegroup.html +++ /dev/null @@ -1,24 +0,0 @@ -<html> -<head> -<title>rsyslog.conf file</title> -</head> -<body> -<a href="rsyslog_conf_global.html">back</a> - -<h2>$FileGroup</h2> -<p><b>Type:</b> global configuration directive</p> -<p><b>Default:</b> </p> -<p><b>Description:</b></p> -<p>Set the group for dynaFiles newly created. Please note that this setting does not affect the group of files already existing. The parameter is a group name, for which the groupid is obtained by rsyslogd during startup processing. Interim changes to the user mapping are not detected.</p> -<p><b>Sample:</b></p> -<p><code><b>$FileGroup loggroup</b></code></p> - -<p>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>] [<a href="manual.html">manual -index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> -<p><font size="2">This documentation is part of the -<a href="http://www.rsyslog.com/">rsyslog</a> project.<br> -Copyright © 2007 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and -<a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL -version 2 or higher.</font></p> -</body> -</html> diff --git a/doc/rsconf1_fileowner.html b/doc/rsconf1_fileowner.html deleted file mode 100644 index 62125c8..0000000 --- a/doc/rsconf1_fileowner.html +++ /dev/null @@ -1,24 +0,0 @@ -<html> -<head> -<title>rsyslog.conf file</title> -</head> -<body> -<a href="rsyslog_conf_global.html">back</a> - -<h2>$FileOwner</h2> -<p><b>Type:</b> global configuration directive</p> -<p><b>Default:</b> </p> -<p><b>Description:</b></p> -<p>Set the file owner for dynaFiles newly created. Please note that this setting does not affect the owner of files already existing. The parameter is a user name, for which the userid is obtained by rsyslogd during startup processing. Interim changes to the user mapping are not detected.</p> -<p><b>Sample:</b></p> -<p><code><b>$FileOwner loguser</b></code></p> - -<p>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>] [<a href="manual.html">manual -index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> -<p><font size="2">This documentation is part of the -<a href="http://www.rsyslog.com/">rsyslog</a> project.<br> -Copyright © 2007 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and -<a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL -version 2 or higher.</font></p> -</body> -</html> diff --git a/doc/rsconf1_generateconfiggraph.html b/doc/rsconf1_generateconfiggraph.html deleted file mode 100644 index 3f0fd66..0000000 --- a/doc/rsconf1_generateconfiggraph.html +++ /dev/null @@ -1,127 +0,0 @@ -<html> -<head> -<title>rsyslog.conf file</title> -</head> -<body> -<a href="rsyslog_conf_global.html">back</a> - -<h2>$GenerateConfigGraph</h2> -<p><b>Type:</b> global configuration directive</p> -<p><b>Default:</b> </p> -<p><b>Available Since:</b> 4.3.1 <b>CURRENTLY NOT AVAILABLE</b></p> -<p><b>Description:</b></p> -<b>This directive is currently not supported. We had to disable it when we improved the -rule engine. It is considerable effort to re-enable it. On the other hand, we are about -to add a new config system, which will make yet another config graph method necessary. -As such we have decided to currently disable this functionality and re-introduce it when -the new config system has been instantiated. -</b></p> -<p>This directive permits to create (hopefully) good-looking visualizations of rsyslogd's -configuration. It does not affect rsyslog operation. If the directive is specified multiple -times, all but the last are ignored. If it is specified, a graph is created. This happens -both during a regular startup as well a config check run. It is recommended to include -this directive only for documentation purposes and remove it from a production -configuraton. -<p>The graph is not drawn by rsyslog itself. Instead, it uses the great open source tool -<a href="http://www.graphviz.org">Graphviz</a> to do the actual drawing. This has at least -two advantages: -<ul> -<li>the graph drawing support code in rsyslog is extremly slim and without overhead -<li>the user may change or further annotate the generated file, thus potentially -improving his documentation -</ul> -The drawback, of course, is that you need to run Graphviz once you have generated -the control file with rsyslog. Fortunately, the process to do so is rather easy: -<ol> -<li>add "$GenerateConfigGraph /path/to/file.dot" to rsyslog.conf (from now on, I -will call the file just file.dot). Optionally, add "$ActionName" statement -<b>in front of</b> those actions that you like to use friendly names with. If you do -this, keep the names short. -<li>run rsyslog at least once (either in regular or configuration check mode) -<li>remember to remove the $GenerateConfigGraph directive when you no longer need it (or -comment it out) -<li>change your working directory to where you place the dot file -<li>if you would like to edit the rsyslog-generated file, now is the time to do so -<li>do "dot -Tpng file.dot > file.png" -<li>remember that you can use "convert -resize 50% file.png resized.png" if -dot's output is too large (likely) or too small. Resizing can be especially useful if -you intend to get a rough overview over your configuration. -</ol> -After completing these steps, you should have a nice graph of your configuration. Details -are missing, but that is exactly the point. At the start of the graph is always (at least -in this version, could be improved) a node called "inputs" in a tripple hexagon -shape. This represents all inputs active in the system (assuming you have defined some, -what the current version does not check). Next comes the main queue. It is given in a -hexagon shape. That shape indicates that a queue is peresent and used to de-couple -the inbound from the outbound part of the graph. In technical terms, here is a -threading boundary. Action with "real" queues (other than in direct mode) -also utilize this shape. For actions, notice that a "hexagon action" creates -a deep copy of the message. As such, a "discard hexagon action" actually does -nothing, because it duplicates the message and then discards <b>the duplicate</b>. -At the end of the diagram, you always see a "discard" action. This indicates -that rsyslog discards messages which have been run through all available rules. -<p>Edges are labeled with information about when they are taken. For filters, the type of -filter, but not any specifics, are given. It is also indicated if no filter is -applied in the configuration file (by using a "*.*" selector). Edges without -labels are unconditionally taken. The actions themselfs are labeled with the name of -the output module that handles them. If provided, the name given via -"ActionName" is used instead. No further details are provided. -<p>If there is anything in red, this should draw your attention. In this case, rsyslogd -has detected something that does not look quite right. A typical example is a discard -action which is followed by some other actions in an action unit. Even though something -may be red, it can be valid - rsyslogd's graph generator does not yet check each and -every speciality, so the configuration may just cover a very uncommon case. -<p>Now let's look at some examples. The graph below was generated on a fairly standard -Fedora rsyslog.conf file. It had only the usually commented-out last forwarding action -activated: -<p align="center"> -<img src="rsyslog_confgraph_std.png" alt="rsyslog configuration graph for a default fedora rsyslog.conf"> -<p>This is the typical structure for a simple rsyslog configuration. There are a couple of -actions, each guarded by a filter. Messages run from top to bottom and control branches -whenever a filter evaluates to true. As there is no discard action, all messages will -run through all filters and discarded in the system default discard action right after -all configured actions. -</p> -<p>A more complex example can be seen in the next graph. This is a configuration I -created for testing the graph-creation features, so it contains a little bit of -everything. However, real-world configurations can look quite complex, too (and I -wouldn't say this one is very complex): -<p align="center"> -<img src="rsyslog_confgraph_complex.png"> -</p> -<p>Here, we have a user-defined discard action. You can immediately see this because -processing branches after the first "builtin-file" action. Those messages -where the filter evaluates to true for will never run through the left-hand action -branch. However, there is also a configuration error present: there are two more -actions (now shown red) after the discard action. As the message is discarded, these will -never be executed. Note that the discard branch contains no further filters. This is -because these actions are all part of the same action unit, which is guarded only by -an entry filter. The same is present a bit further down at the node labeled -"write_system_log_2". This note has one more special feature, that is label -was set via "ActionName", thus is does not have standard form (the same -happened to the node named "Forward" right at the top of the diagram. -Inside this diagram, the "Forward" node is executed asynchonously on its own -queue. All others are executed synchronously. -<p>Configuration graphs are useful for documenting a setup, but are also a great -<a href="troubleshoot.html">troubleshooting</a> resource. It is important to -remember that <b>these graphs are generated -from rsyslogd's in-memory action processing structures</b>. You can not get closer -to understanding on how rsyslog interpreted its configuration files. -So if the graph does not look -what you intended to do, there is probably something worng in rsyslog.conf. -<p>If something is not working as expected, but you do not spot the error immediately, -I recommend to generate a graph and zoom it so that you see all of it in one great picture. -You may not be able to read anything, but the structure should look good to you and -so you can zoom into those areas that draw your attention. -<p><b>Sample:</b></p> -<p><code><b>$DirOwner /path/to/graphfile-file.dot</b></code></p> - -<p>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>] [<a href="manual.html">manual -index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> -<p><font size="2">This documentation is part of the -<a href="http://www.rsyslog.com/">rsyslog</a> project.<br> -Copyright © 2009 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and -<a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL -version 2 or higher.</font></p> -</body> -</html> diff --git a/doc/rsconf1_gssforwardservicename.html b/doc/rsconf1_gssforwardservicename.html deleted file mode 100644 index 45d9ba9..0000000 --- a/doc/rsconf1_gssforwardservicename.html +++ /dev/null @@ -1,26 +0,0 @@ -<html> -<head> -<title>rsyslog.conf file</title> -</head> -<body> -<a href="rsyslog_conf_global.html">back</a> - -<h2>$GssForwardServiceName</h2> -<p><b>Type:</b> global configuration directive</p> -<p><b>Default:</b> host</p> -<p><b>Provided by:</b> <i>omgssapi</i></p> -<p><b>Description:</b></p> -<p>Specifies the service name used by the client when forwarding GSS-API wrapped messages.</p> -<p>The GSS-API service names are constructed by appending '@' and a hostname following "@@" in each selector.</p> -<p><b>Sample:</b></p> -<p><code><b>$GssForwardServiceName rsyslog</b></code></p> - -<p>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>] [<a href="manual.html">manual -index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> -<p><font size="2">This documentation is part of the -<a href="http://www.rsyslog.com/">rsyslog</a> project.<br> -Copyright © 2007 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and -<a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL -version 2 or higher.</font></p> -</body> -</html> diff --git a/doc/rsconf1_gsslistenservicename.html b/doc/rsconf1_gsslistenservicename.html deleted file mode 100644 index 5fdf3ed..0000000 --- a/doc/rsconf1_gsslistenservicename.html +++ /dev/null @@ -1,24 +0,0 @@ -<html> -<head> -<title>rsyslog.conf file</title> -</head> -<body> -<a href="rsyslog_conf_global.html">back</a> - -<h2>$GssListenServiceName</h2> -<p><b>Type:</b> global configuration directive</p> -<p><b>Default:</b> host</p> -<p><b>Description:</b></p> -<p>Specifies the service name used by the server when listening for GSS-API wrapped messages.</p> -<p><b>Sample:</b></p> -<p><code><b>$GssForwardServiceName rsyslog</b></code></p> - -<p>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>] [<a href="manual.html">manual -index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> -<p><font size="2">This documentation is part of the -<a href="http://www.rsyslog.com/">rsyslog</a> project.<br> -Copyright © 2007 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and -<a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL -version 2 or higher.</font></p> -</body> -</html> diff --git a/doc/rsconf1_gssmode.html b/doc/rsconf1_gssmode.html deleted file mode 100644 index 2b1d565..0000000 --- a/doc/rsconf1_gssmode.html +++ /dev/null @@ -1,26 +0,0 @@ -<html> -<head> -<title>rsyslog.conf file</title> -</head> -<body> -<a href="rsyslog_conf_global.html">back</a> - -<h2>$GssMode</h2> -<p><b>Type:</b> global configuration directive</p> -<p><b>Default:</b> encryption</p> -<p><b>Provided by:</b> <i>omgssapi</i></p> -<p><b>Description:</b></p> -<p>Specifies GSS-API mode to use, which can be "<b>integrity</b>" - clients are authenticated and - messages are checked for integrity, "<b>encryption</b>" - same as - "integrity", but messages are also encrypted if both sides support it.<p><b>Sample:</b></p> -<p><code><b>$GssMode Encryption</b></code></p> - -<p>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>] [<a href="manual.html">manual -index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> -<p><font size="2">This documentation is part of the -<a href="http://www.rsyslog.com/">rsyslog</a> project.<br> -Copyright © 2007 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and -<a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL -version 2 or higher.</font></p> -</body> -</html> diff --git a/doc/rsconf1_includeconfig.html b/doc/rsconf1_includeconfig.html deleted file mode 100644 index 132cee6..0000000 --- a/doc/rsconf1_includeconfig.html +++ /dev/null @@ -1,48 +0,0 @@ -<html> -<head> -<title>rsyslog.conf file</title> -</head> -<body> -<a href="rsyslog_conf_global.html">back</a> - -<h2>$IncludeConfig</h2> -<p><b>Type:</b> global configuration directive</p> -<p><b>Default:</b> </p> -<p><b>Description:</b></p> -<p>This directive allows to include other files into the main configuration file. As soon as an IncludeConfig directive is found, the contents of the new file is processed. IncludeConfigs can be nested. Please note that from a logical point of view the files are merged. Thus, if the include modifies some parameters (e.g. $DynaFileChacheSize), these new parameters are in place for the "calling" configuration file when the include is completed. To avoid any side effects, do a $ResetConfigVariables after the $IncludeConfig. It may also be a good idea to do a $ResetConfigVariables right at the start of the include, so that the module knows exactly what it does. Of course, one might specifically NOT do this to inherit parameters from the main file. As always, use it as it best fits...</p> -<p>If all regular files in the /etc/rsyslog.d directory are included, then files starting with "." are ignored - so you can use them to place comments into the dir (e.g. "/etc/rsyslog.d/.mycomment" will be ignored). <a href="http://sourceforge.net/tracker/index.php?func=detail&aid=1764088&group_id=123448&atid=696555">Michael Biebl had the idea to this functionality</a>. Let me quote hím:</p> -<blockquote> -<p><i>Say you can add an option<br> -$IncludeConfig /etc/rsyslog.d/<br> -(which probably would make a good default)<br> -to /etc/rsyslog.conf, which would then merge and include all *.conf files<br> -in /etc/rsyslog.d/.<br> -<br> -This way, a distribution can modify its packages easily to drop a simple<br> -config file into this directory upon installation.<br> -<br> -As an example, the network-manager package could install a simple config<br> -file /etc/rsyslog.d/network-manager.conf which would contain.<br> -:programname, contains, "NetworkManager" -/var/log/NetworkManager.log<br> -<br> -Upon uninstallation, the file could be easily removed again. This approach<br> -would be much cleaner and less error prone, than having to munge around<br> -with the /etc/rsyslog.conf file directly.</i></p> -</blockquote> -<p><b>Sample:</b></p> -<p><code><b>$IncludeConfig /etc/some-included-file.conf</b></code></p> -<p>Directories can also be included. To do so, the name must end on a slash:</p> -<p><code><b>$IncludeConfig /etc/rsyslog.d/</b></code></p> -<p><b>And finally, only specific files matching a wildcard my be included -from a directory:</b></p> -<p><code><b>$IncludeConfig /etc/rsyslog.d/*.conf</b></code></p> - -<p>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>] [<a href="manual.html">manual -index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> -<p><font size="2">This documentation is part of the -<a href="http://www.rsyslog.com/">rsyslog</a> project.<br> -Copyright © 2007 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and -<a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL -version 2 or higher.</font></p> -</body> -</html> diff --git a/doc/rsconf1_mainmsgqueuesize.html b/doc/rsconf1_mainmsgqueuesize.html deleted file mode 100644 index ffed1c0..0000000 --- a/doc/rsconf1_mainmsgqueuesize.html +++ /dev/null @@ -1,24 +0,0 @@ -<html> -<head> -<title>rsyslog.conf file</title> -</head> -<body> -<a href="rsyslog_conf_global.html">back</a> - -<h2>$MainMsgQueueSize</h2> -<p><b>Type:</b> global configuration directive</p> -<p><b>Default:</b> 10000</p> -<p><b>Description:</b></p> -<p>This allows to specify the maximum size of the message queue. This directive is only available when rsyslogd has been compiled with multithreading support. In this mode, receiver and output modules are de-coupled via an in-memory queue. This queue buffers messages when the output modules are not capable to process them as fast as they are received. Once the queue size is exhausted, messages will be dropped. The slower the output (e.g. MySQL), the larger the queue should be. Buffer space for the actual queue entries is allocated on an as-needed basis. Please keep in mind that a very large queue may exhaust available system memory and swap space. Keep this in mind when configuring the max size. The actual size of a message depends largely on its content and the originator. As a rule of thumb, typically messages should not take up more then roughly 1k (this is the memory structure, not what you see in a network dump!). For typical linux messages, 512 bytes should be a good bet. Please also note that there is a minimal amount of memory taken for each queue entry, no matter if it is used or not. This is one pointer value, so on 32bit systems, it should typically be 4 bytes and on 64bit systems it should typically be 8 bytes. For example, the default queue size of 10,000 entries needs roughly 40k fixed overhead on a 32 bit system.</p> -<p><b>Sample:</b></p> -<p><code><b>$MainMsgQueueSize 100000 # 100,000 may be a value to handle burst traffic</b></code></p> - -<p>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>] [<a href="manual.html">manual -index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> -<p><font size="2">This documentation is part of the -<a href="http://www.rsyslog.com/">rsyslog</a> project.<br> -Copyright © 2007 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and -<a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL -version 2 or higher.</font></p> -</body> -</html> diff --git a/doc/rsconf1_markmessageperiod.html b/doc/rsconf1_markmessageperiod.html deleted file mode 100644 index a6486ba..0000000 --- a/doc/rsconf1_markmessageperiod.html +++ /dev/null @@ -1,32 +0,0 @@ -<html> -<head> -<title>rsyslog.conf file</title> -</head> -<body> -<a href="rsyslog_conf_global.html">back</a> - -<h2>$MarkMessagePeriod</h2> -<p><b>Type:</b> specific to immark input module</p> -<p><b>Default:</b> 1200 (20 minutes)</p> -<p><b>Description:</b></p> -<p>This specifies when mark messages are to be written to output modules. The -time specified is in seconds. Specifying 0 is possible and disables mark -messages. In that case, however, it is more efficient to NOT load the immark -input module.</p> -<p>So far, there is only one mark message process and any subsequent -$MarkMessagePeriod overwrites the previous.</p> -<p><b>This directive is only available after the immark input module has been -loaded.</b></p> -<p><b>Sample:</b></p> -<p><code><b>$MarkMessagePeriod 600 # mark messages appear every 10 Minutes</b></code></p> -<p><b>Available since:</b> rsyslog 3.0.0</p> - -<p>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>] [<a href="manual.html">manual -index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> -<p><font size="2">This documentation is part of the -<a href="http://www.rsyslog.com/">rsyslog</a> project.<br> -Copyright © 2007 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and -<a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL -version 2 or higher.</font></p> -</body> -</html> diff --git a/doc/rsconf1_maxopenfiles.html b/doc/rsconf1_maxopenfiles.html deleted file mode 100644 index b6c9cc0..0000000 --- a/doc/rsconf1_maxopenfiles.html +++ /dev/null @@ -1,35 +0,0 @@ -<html> -<head> -<title>$MaxOpenFiles - rsyslog.conf file</title> -</head> -<body> -<a href="rsyslog_conf_global.html">[rsyslog configuration directive overview]</a> - -<h2>$MaxOpenFiles</h2> -<p><b>Available Since:</b> 4.3.0</p> -<p><b>Type:</b> global configuration directive</p> -<p><b>Default:</b> <i>operating system default</i></p> -<p><b>Description:</b></p> -<p>Set the maximum number of files that the rsyslog process can have open at any given -time. Note that this includes open tcp sockets, so this setting is the upper limit for -the number of open TCP connections as well. If you expect a large nubmer of concurrent -connections, it is suggested that the number is set to the max number connected plus 1000. -Please note that each dynafile also requires up to 100 open file handles. -<p>The setting is similar to running "ulimit -n number-of-files". -<p>Please note that depending on permissions and operating system configuration, the -setrlimit() request issued by rsyslog may fail, in which case the previous limit is kept -in effect. Rsyslog will emit a warning message in this case. -<p><b>Sample:</b></p> -<p><code><b>$MaxOpenFiles 2000</b></code></p> -<p><b>Bugs:</b></p> -<p>For some reason, this settings seems not to work on all platforms. If you experience -problems, please let us know so that we can (hopefully) narrow down the issue. -<p>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>] [<a href="manual.html">manual -index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> -<p><font size="2">This documentation is part of the -<a href="http://www.rsyslog.com/">rsyslog</a> project.<br> -Copyright © 2009 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and -<a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL -version 3 or higher.</font></p> -</body> -</html> diff --git a/doc/rsconf1_moddir.html b/doc/rsconf1_moddir.html deleted file mode 100644 index 889de05..0000000 --- a/doc/rsconf1_moddir.html +++ /dev/null @@ -1,29 +0,0 @@ -<html> -<head> -<title>rsyslog.conf file</title> -</head> -<body> -<a href="rsyslog_conf_global.html">back</a> - -<h2>$ModDir</h2> -<p><b>Type:</b> global configuration directive</p> -<p><b>Default:</b> system default for user libraries, e.g. -/usr/local/lib/rsyslog/</p> -<p><b>Description:</b></p> -<p>Provides the default directory in which loadable modules reside. This may be -used to specify an alternate location that is not based on the system default. -If the system default is used, there is no need to specify this directive. Please -note that it is vitally important to end the path name with a slash, else module -loads will fail.</p> -<p><b>Sample:</b></p> -<p><code><b>$ModDir /usr/rsyslog/libs/ # note the trailing slash!</b></code></p> - -<p>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>] [<a href="manual.html">manual -index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> -<p><font size="2">This documentation is part of the -<a href="http://www.rsyslog.com/">rsyslog</a> project.<br> -Copyright © 2007 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and -<a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL -version 2 or higher.</font></p> -</body> -</html> diff --git a/doc/rsconf1_modload.html b/doc/rsconf1_modload.html deleted file mode 100644 index ce457ea..0000000 --- a/doc/rsconf1_modload.html +++ /dev/null @@ -1,34 +0,0 @@ -<html> -<head> -<title>rsyslog.conf file</title> -</head> -<body> -<a href="rsyslog_conf_global.html">back</a> - -<h2>$ModLoad</h2> -<p><b>Type:</b> global configuration directive</p> -<p><b>Default:</b> </p> -<p><b>Description:</b></p> -<p>Dynamically loads a plug-in into rsyslog's address space and activates it. -The plug-in must obey the rsyslog module API. Currently, only MySQL and Postgres -output modules are available -as a plugins, but users may create their own. A plug-in must be loaded BEFORE -any configuration file lines that reference it.</p> -<p>Modules must be present in the system default destination for rsyslog -modules. You can also set the directory via the <a href="rsconf1_moddir.html"> -$ModDir</a> directive.</p> -<p>If a full path name is specified, the module is loaded from that path. The -default module directory is ignored in that case.</p> -<p><b>Sample:</b></p> -<p><code><b>$ModLoad ommysql # load MySQL functionality<br> -$ModLoad /rsyslog/modules/ompgsql.so # load the postgres module via absolute path</b></code></p> - -<p>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>] [<a href="manual.html">manual -index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> -<p><font size="2">This documentation is part of the -<a href="http://www.rsyslog.com/">rsyslog</a> project.<br> -Copyright © 2007 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and -<a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL -version 2 or higher.</font></p> -</body> -</html> diff --git a/doc/rsconf1_omfileforcechown.html b/doc/rsconf1_omfileforcechown.html deleted file mode 100644 index a680810..0000000 --- a/doc/rsconf1_omfileforcechown.html +++ /dev/null @@ -1,67 +0,0 @@ -<html> -<head> -<title>rsyslog.conf file</title> -</head> -<body> -<a href="rsyslog_conf_global.html">back</a> - -<h2>$omfileForceChown</h2> -<p><b>Type:</b> global configuration directive</p> -<p><b>Parameter Values:</b> boolean (on/off, yes/no)</p> -<p><b>Available:</b> 4.7.0+, 5.3.0-5.8.x, <b>NOT</b> available in 5.9.x or higher</p> -<p><b>Note: this directive has been removed and is no longer available. The -documentation is currently being retained for historical reaons.</b> Expect -it to go away at some later stage as well. -<p><b>Default:</b> off</p> -<p><b>Description:</b></p> -<p>Forces rsyslogd to change the ownership for output files that already exist. Please note -that this tries to fix a potential problem that exists outside the scope of rsyslog. Actually, -it tries to fix invalid ownership/permission settings set by the original file creator. -<p>Rsyslog changes the ownership during initial execution with root privileges. When a privelege -drop is configured, privileges are dropped after the file owner ship is changed. Not that this currently -is a limitation in rsyslog's privilege drop code, which is on the TODO list to be removed. See Caveats -section below for the important implications. -<p><b>Caveats:</b></p> -<p>This directive tries to fix a problem that actually is outside the scope of rsyslog. As such, -there are a couple of restrictions and situations in which it will not work. <b>Users are strongly -encouraged to fix their system instead of turning this directive on</b> - it should only be used -as a last resort. -<p>At least in the following scenario, this directive will fail expectedly: -<p>It does not address -the situation that someone changes the ownership *after* rsyslogd has started. -Let's, for example, consider a log rotation script. -<ul> -<li>rsyslog is started -<li>ownership is changed -<li>privileges dropped -<li>log rotation (lr) script starts -<li>lr removes files -<li>lr creates new files with root:adm (or whatever else) -<li>lr HUPs rsyslogd -<li>rsyslogd closes files -<li>rsyslogd tries to open files -<li>rsyslogd tries to change ownership --> fail as we are non-root now -<li>file open fails -</ul> - -Please note that once the privilege drop code is refactored, this directive will -no longer work, because then privileges will be dropped before any action is performed, -and thus we will no longer be able to chown files that do not belong to the -user rsyslogd is configured to run under. - -<p>So <b>expect the directive to go away</b>. It will not -be removed in version 4, but may disappear at any time for any version greater than 4. - -<p><b>Sample:</b></p> -<p><code><b>$FileOwner loguser</b> -<br><b>$omfileForceChown on</b></code></p> - -<p>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>] [<a href="manual.html">manual -index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> -<p><font size="2">This documentation is part of the -<a href="http://www.rsyslog.com/">rsyslog</a> project.<br> -Copyright © 2007 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and -<a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL -version 2 or higher.</font></p> -</body> -</html> diff --git a/doc/rsconf1_repeatedmsgreduction.html b/doc/rsconf1_repeatedmsgreduction.html deleted file mode 100644 index 248e834..0000000 --- a/doc/rsconf1_repeatedmsgreduction.html +++ /dev/null @@ -1,25 +0,0 @@ -<html> -<head> -<title>rsyslog.conf file</title> -</head> -<body> -<a href="rsyslog_conf_global.html">back</a> - -<h2>$RepeatedMsgReduction</h2> -<p><b>Type:</b> global configuration directive</p> -<p><b>Default:</b> depending on -e</p> -<p><b>Description:</b></p> -<p>This directive specifies whether or not repeated messages should be reduced (this is the "Last line repeated n times" feature). If set to on, repeated messages are reduced. If set to off, every message is logged. Please note that this directive overrides the -e command line option. In case -e is given, it is just the default value until the first RepeatedMsgReduction directive is encountered.</p> -<p>This directives affects selector lines until a new directive is specified.</p> -<p><b>Sample:</b></p> -<p><code><b>$RepeatedMsgReduction off # log every message</b></code></p> - -<p>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>] [<a href="manual.html">manual -index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> -<p><font size="2">This documentation is part of the -<a href="http://www.rsyslog.com/">rsyslog</a> project.<br> -Copyright © 2007 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and -<a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL -version 2 or higher.</font></p> -</body> -</html> diff --git a/doc/rsconf1_resetconfigvariables.html b/doc/rsconf1_resetconfigvariables.html deleted file mode 100644 index 46cf0bd..0000000 --- a/doc/rsconf1_resetconfigvariables.html +++ /dev/null @@ -1,24 +0,0 @@ -<html> -<head> -<title>rsyslog.conf file</title> -</head> -<body> -<a href="rsyslog_conf_global.html">back</a> - -<h2>$ResetConfigVariables</h2> -<p><b>Type:</b> global configuration directive</p> -<p><b>Default:</b> </p> -<p><b>Description:</b></p> -<p>Resets all configuration variables to their default value. Any settings made will not be applied to configuration lines following the $ResetConfigVariables. This is a good method to make sure no side-effects exists from previous directives. This directive has no parameters.</p> -<p><b>Sample:</b></p> -<p><code><b>$ResetConfigVariables</b></code></p> - -<p>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>] [<a href="manual.html">manual -index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> -<p><font size="2">This documentation is part of the -<a href="http://www.rsyslog.com/">rsyslog</a> project.<br> -Copyright © 2007 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and -<a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL -version 2 or higher.</font></p> -</body> -</html> diff --git a/doc/rsconf1_rulesetcreatemainqueue.html b/doc/rsconf1_rulesetcreatemainqueue.html deleted file mode 100644 index 5c1e0de..0000000 --- a/doc/rsconf1_rulesetcreatemainqueue.html +++ /dev/null @@ -1,83 +0,0 @@ -<html> -<head> -<title>RulesetCreateMainQueue - rsyslog.conf file</title> -</head> -<body> -<a href="rsyslog_conf_global.html">rsyslog.conf configuration directive</a> - -<h2>$RulesetCreateMainQueue</h2> -<p><b>Type:</b> ruleset-specific configuration directive</p> -<p><b>Parameter Values:</b> boolean (on/off, yes/no)</p> -<p><b>Available since:</b> 5.3.5+</p> -<p><b>Default:</b> off</p> -<p><b>Description:</b></p> -<p> -Rulesets may use their own "main" message queue for message submission. Specifying -this directive, <b>inside a ruleset definition</b>, turns this on. This is both a performance -enhancement and also permits different rulesets (and thus different inputs within the same -rsyslogd instance) to use different types of main message queues. -<p>The ruleset queue is created with the parameters that are specified for the main message -queue at the time the directive is given. If different queue configurations are desired, -different main message queue directives must be used in front of the $RulesetCreateMainQueue -directive. Note that this directive may only be given once per ruleset. If multiple statements -are specified, only the first is used and for the others error messages are emitted. -<p>Note that the final set of ruleset configuration directives specifies the parameters for -the default main message queue. -<p>To learn more about this feature, please be sure to read about -<a href="multi_ruleset.html">multi-ruleset support in rsyslog</a>. -<p><b>Caveats:</b></p> -The configuration statement "$RulesetCreateMainQueue off" has no effect at all. -The capability to specify this is an artifact of the current (ugly!) configuration -language. - -<p><b>Example:</b></p> -<p>This example sets up a tcp server with three listeners. Each of these -three listener is bound to a specific ruleset. As a performance optimization, -the rulesets all receive their own private queue. The result is that received messages -can be independently processed. With only a single main message queue, we would have -some lock contention between the messages. This does not happen here. Note that in this -example, we use different processing. Of course, all messages could also have been -processed in the same way ($IncludeConfig may be useful in that case!). -</p> -<textarea rows="30" cols="60">$ModLoad imtcp -# at first, this is a copy of the unmodified rsyslog.conf -#define rulesets first -$RuleSet remote10514 -$RulesetCreateMainQueue on # create ruleset-specific queue -*.* /var/log/remote10514 - -$RuleSet remote10515 -$RulesetCreateMainQueue on # create ruleset-specific queue -*.* /var/log/remote10515 - -$RuleSet remote10516 -$RulesetCreateMainQueue on # create ruleset-specific queue -mail.* /var/log/mail10516 -& ~ -# note that the discard-action will prevent this messag from -# being written to the remote10516 file - as usual... -*.* /var/log/remote10516 - -# and now define listners bound to the relevant ruleset -$InputTCPServerBindRuleset remote10514 -$InputTCPServerRun 10514 - -$InputTCPServerBindRuleset remote10515 -$InputTCPServerRun 10515 - -$InputTCPServerBindRuleset remote10516 -$InputTCPServerRun 10516 -</textarea> -<p>Note the positions of the directives. With the current config language, -position is very important. This is ugly, but unfortunately the way it currently -works. -</p> - -<p>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>] [<a href="manual.html">manual -index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> -<p><font size="2">This documentation is part of the -<a href="http://www.rsyslog.com/">rsyslog</a> project.<br> -Copyright © 2009 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and -<a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL version 2 or higher.</font></p> -</body> -</html> diff --git a/doc/rsconf1_rulesetparser.html b/doc/rsconf1_rulesetparser.html deleted file mode 100644 index 433456c..0000000 --- a/doc/rsconf1_rulesetparser.html +++ /dev/null @@ -1,123 +0,0 @@ -<html> -<head> -<title>RulesetParser - rsyslog.conf file</title> -</head> -<body> -<a href="rsyslog_conf_global.html">rsyslog.conf configuration directive</a> - -<h2>$RulesetParser</h2> -<p><b>Type:</b> ruleset-specific configuration directive</p> -<p><b>Parameter Values:</b> string</p> -<p><b>Available since:</b> 5.3.4+</p> -<p><b>Default:</b> rsyslog.rfc5424 followed by rsyslog.rfc3164</p> -<p><b>Description:</b></p> -<p> -This directive permits to specify which -<a href="messageparser.html">message parsers</a> should be used for the ruleset -in question. It no ruleset is explicitely specified, the default ruleset is used. Message -parsers are contained in (loadable) parser modules with the most common cases -(RFC3164 and RFC5424) being build-in into rsyslogd. -<p>When this directive is specified the first time for a ruleset, it will not only add the -parser to the ruleset's parser chain, it will also wipe out the default parser chain. -So if you need to have -them in addition to the custom parser, you need to specify those as well. -<p>Order of directives is important. Parsers are tried one after another, in the order -they are specified inside the config. As soon as a parser is able to parse the message, -it will do so and no other parsers will be executed. If no matching parser can be found, -the message will be discarded and a warning message be issued (but only for the first -1,000 instances of this problem, to prevent message generation loops). -<p>Note that the rfc3164 parser will <b>always</b> be able to parse a message - it may -just not be the format that you like. This has two important implications: 1) always place -that parser at the END of the parser list, or the other parsers after it will never -be tried and 2) if you would like to make sure no message is lost, placing the rfc3164 -parser at the end of the parser list ensures that. -<p>Multiple parser modules are very useful if you have various devices that emit -messages that are malformed in various ways. The route to take then is -<ul> -<li>make sure you find a custom parser for that device; if there is no one, you -may consider writing one yourself (it is not that hard) or getting one written -as part of -<a href="http://www.rsyslog.com/professional-services">Adiscon's professional services -for rsyslog</a>. -<li>load your custom parsers via $ModLoad -<li>create a ruleset for each malformed format; assign the custom parser to it -<li>create a specific listening port for all devices that emit the same -malformed format -<li>bind the listener to the ruleset with the required parser -</ul> -<p>Note that it may be cumbersome to add all rules to all rulesets. To avoid this, -you can either use $Include or <a href="omruleset.html">omruleset</a> -(what probably provides the best solution). -<p>More information about rulesets in general can be found in -<a href="multi_ruleset.html">multi-ruleset support in rsyslog</a>. -<p><b>Caveats:</b></p> -<p>currently none known</p> - -<p><b>Example:</b></p> -<p>This example assumes there are two devices emiting malformed messages via UDP. -We have two custom parsers for them, named "device1.parser" and -"device2.parser". In addition to that, we have a number of other -devices sending wellformed messages, also via UDP. -<p>The solution is to listen for data from the two devices on two special -ports (10514 and 10515 in this example), create a ruleset for each and -assign the custom parsers to them. The rest of the messages are received via -port 514 using the regular parsers. Processing shall be equal for all messages. -So we simply forward the malformed messages to the regular queue once they are parsed (keep -in mind that a message is never again parsed once any parser properly processed it). -</p> -<textarea rows="40" cols="80">$ModLoad imudp -$ModLoad pmdevice1 # load parser "device1.parser" for device 1 -$ModLoad pmdevice2 # load parser "device2.parser" for device 2 - -# define ruleset for the first device sending malformed data -$Ruleset maldev1 -$RulesetCreateMainQueue on # create ruleset-specific queue -$RulesetParser "device1.parser" # note: this deactivates the default parsers -# forward all messages to default ruleset: -$ActionOmrulesetRulesetName RSYSLOG_DefaultRuleset -*.* :omruleset: - -# define ruleset for the second device sending malformed data -$Ruleset maldev2 -$RulesetCreateMainQueue on # create ruleset-specific queue -$RulesetParser "device2.parser" # note: this deactivates the default parsers -# forward all messages to default ruleset: -$ActionOmrulesetRulesetName RSYSLOG_DefaultRuleset -*.* :omruleset: - -# switch back to default ruleset -$Ruleset RSYSLOG_DefaultRuleset -*.* /path/to/file -auth.info @authlogger.example.net -# whatever else you usually do... - - -# now define the inputs and bind them to the rulesets -# first the default listener (utilizing the default ruleset) -$UDPServerRun 514 - -# now the one with the parser for device type 1: -$InputUDPServerBindRuleset maldev1 -$UDPServerRun 10514 - -# and finally the one for device type 2: -$InputUDPServerBindRuleset maldev2 -$UDPServerRun 10515 -</textarea> - -<p>For an example of how multiple parser can be chained (and an actual use case), please see -the example section on the <a href="pmlastmsg.html">pmlastmsg</a> parser -module. -<p>Note the positions of the directives. With the current config language, -<b>sequence of statements is very important</b>. This is ugly, but unfortunately -the way it currently works. -</p> - -<p>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>] [<a href="manual.html">manual -index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> -<p><font size="2">This documentation is part of the -<a href="http://www.rsyslog.com/">rsyslog</a> project.<br> -Copyright © 2009 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and -<a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL version 2 or higher.</font></p> -</body> -</html> diff --git a/doc/rsconf1_umask.html b/doc/rsconf1_umask.html deleted file mode 100644 index 8e41e67..0000000 --- a/doc/rsconf1_umask.html +++ /dev/null @@ -1,26 +0,0 @@ -<html> -<head> -<title>rsyslog.conf file</title> -</head> -<body> -<a href="rsyslog_conf_global.html">back</a> - -<h2>$UMASK</h2> -<p><b>Type:</b> global configuration directive</p> -<p><b>Default:</b> </p> -<p><b>Description:</b></p> -<p>The $umask directive allows to specify the rsyslogd processes' umask. If not specified, the system-provided default is used. The value given must always be a 4-digit octal number, with the initial digit being zero.</p> -<p>If $umask is specified multiple times in the configuration file, results may be somewhat unpredictable. It is recommended to specify it only once.</p> -<p><b>Sample:</b></p> -<p><code><b>$umask 0000</b></code></p> -<p>This sample removes all restrictions.</p> - -<p>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>] [<a href="manual.html">manual -index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> -<p><font size="2">This documentation is part of the -<a href="http://www.rsyslog.com/">rsyslog</a> project.<br> -Copyright © 2007 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and -<a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL -version 2 or higher.</font></p> -</body> -</html> diff --git a/doc/rscript_abnf.html b/doc/rscript_abnf.html deleted file mode 100644 index 9172d94..0000000 --- a/doc/rscript_abnf.html +++ /dev/null @@ -1,94 +0,0 @@ -<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> -<html><head> -<meta http-equiv="Content-Language" content="en"><title>RainerScript ABNF</title></head> -<body> -<h1>RainerScript ABNF</h1> -<p>This is the formal definition of RainerScript, as supported by -rsyslog configuration. Please note that this currently is working -document and the actual implementation may be quite different.</p> -<p>The -first glimpse of RainerScript will be available as part of rsyslog -3.12.0 expression support. However, the 3.12. series of rsyslog will -have a partial script implementaiton, which will not necessariy be -compatible with the later full implementation. So if you use it, be -prepared for some config file changes as RainerScript evolves.</p> -<p>C-like comments (/* some comment */) are supported in all pure -RainerScript lines. However, legacy-mapped lines do not support them. -All lines support the hash mark "#" as a comment initiator. Everything -between the hash and the end of line is a comment (just like // in C++ -and many other languages).</p> -<h2>Formal Definition</h2> -<p>Below is the formal language definitionin ABNF (RFC 2234) -format: <br> -</p> -<pre>; <span style="font-weight: bold;">all of this is a working document and may change!</span> -- rgerhards, 2008-02-24<br> -<br> -script := *stmt<br> -stmt := (if_stmt / block / vardef / run_s / load_s)<br> -vardef := "var" ["scope" = ("global" / "event")] <br> -block := "begin" stmt "end"<br> -load_s := "load" constraint ("module") modpath params ; load mod only if expr is true<br> -run_s := "run" constraint ("input") name<br> -constraint:= "if" expr ; constrains some one-time commands<br> -modpath := expr<br> -params := ["params" *1param *("," param) "endparams"]<br> -param := paramname) "=" expr<br> -paramname := [*(obqualifier ".") name]<br> -modpath:= ; path to module<br> -?line? := cfsysline / cfli<br> -cfsysline:= BOL "$" *char EOL ; how to handle the first line? (no EOL in front!)<br> -BOL := ; Begin of Line - implicitely set on file beginning and after each EOL<br> -EOL := 0x0a ;LF<br> -if_stmt := "if" expr "then"<br> -old_filter:= BOL facility "." severity ; no whitespace allowed between BOL and facility!<br> -facility := "*" / "auth" / "authpriv" / "cron" / "daemon" / "kern" / "lpr" / <br> -"mail" / "mark" / "news" / "security" / "syslog" / "user" / "uucp" / <br> -"local0" .. "local7" / "mark"<br> -; The keyword security should not be used anymore<br> -; mark is just internal<br> -severity := TBD ; not really relevant in this context<br> -<br> -; and now the actual expression<br> -expr := e_and *("or" e_and)<br> -e_and := e_cmp *("and" e_cmp)<br> -e_cmp := val 0*1(cmp_op val)<br> -val := term *(("+" / "-" / "&") term)<br> -term := factor *(("*" / "/" / "%") factor)<br> -factor := ["not"] ["-"] terminal<br> -terminal := var / constant / function / ( "(" expr ")" )<br> -function := name "(" *("," expr) ")"<br> -var := "$" varname<br> -varname := msgvar / sysvar / ceevar<br> -msgvar := name<br> -ceevar := "!" name<br> -sysvar := "$" name<br> -name := alpha *(alnum)<br> -constant := string / number<br> -string := simpstr / tplstr ; tplstr will be implemented in next phase<br> -simpstr := "'" *char "'" ; use your imagination for char ;)<br> -tplstr := '"' template '"' ; not initially implemented<br> -number := ["-"] 1*digit ; 0nn = octal, 0xnn = hex, nn = decimal<br> -cmp_op := "==" / "!=" / "<>" / "<" / ">" / "<=" / ">=" / "contains" / "contains_i" / "startswith" / "startswith_i"<br> -digit := %x30-39<br> -alpha := "a" ... "z" # all letters<br> -alnum :* alpha / digit / "_" /"-" # "-" necessary to cover currently-existing message properties<br> -</pre> -<h2>Samples</h2> -<p>Some samples of RainerScript:</p><p>define function IsLinux<br>begin<br> if $environ contains "linux" then return true else return false<br>end</p><p>load if IsLinux() 'imklog.so' params name='klog' endparams /* load klog under linux only */<br>run if IsLinux() input 'klog'<br>load 'ommysql.so'</p><p>if $message contains "error" then<br> action<br> type='ommysql.so', queue.mode='disk', queue.highwatermark = 300,<br> action.dbname='events', action.dbuser='uid',<br> - [?action.template='templatename'?] or [?action.sql='insert into -table... values('&$facility&','&$severity&...?]<br> endaction<br><br>... or ...</p><p>define action writeMySQL<br> type='ommysql.so', queue.mode='disk', queue.highwatermark = 300,<br> action.dbname='events', action.dbuser='uid',<br> [?action.template='templatename'?] or [?action.sql='insert into table... values('<span style="font-family: monospace;"> &</span> $facility & ',' & $severity &...?]<br> endaction</p><p>if $message contains "error" then action writeMySQL</p><p>ALTERNATE APPROACH</p><p>define function IsLinux(<br> if $environ contains "linux" then return true else return false<br>)</p><p>load if IsLinux() 'imklog.so' params name='klog' endparams /* load klog under linux only */<br>run if IsLinux() input 'klog'<br>load 'ommysql.so'</p><p>if $message contains "error" then<br> action(<br> type='ommysql.so', queue.mode='disk', queue.highwatermark = 300,<br> action.dbname='events', action.dbuser='uid',<br> - [?action.template='templatename'?] or [?action.sql='insert into -table... values('&$facility&','&$severity&...?]<br> )<br><br>... or ...</p><p>define action writeMySQL(<br> type='ommysql.so', queue.mode='disk', queue.highwatermark = 300,<br> action.dbname='events', action.dbuser='uid',<br> - [?action.template='templatename'?] or [?action.sql='insert into -table... values('&$facility&','&$severity&...?]<br> )</p><p>if $message contains "error" then action writeMySQL(action.dbname='differentDB')</p><p></p><p>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>] -<h2>Implementation</h2> -RainerScript will be implemented via a hand-crafted LL(1) parser. I was tempted to use yacc, but it turned out the resulting code was not thread-safe and as such did not fit within the context of rsyslog. Also, limited error handling is not a real problem for us: if there is a problem in parsing the configuration file, we stop processing. Guessing what was meant and trying to recover would IMHO not be good choices for something like a syslogd. -[<a href="manual.html">manual index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> -<p><font size="2">This documentation is part of the -<a href="http://www.rsyslog.com/">rsyslog</a> -project.<br> -Copyright © 2008 by <a href="http://www.gerhards.net/rainer">Rainer -Gerhards</a> and -<a href="http://www.adiscon.com/">Adiscon</a>. -Released under the GNU GPL version 3 or higher.</font></p> -</body></html> diff --git a/doc/rsyslog-example.conf b/doc/rsyslog-example.conf deleted file mode 100644 index a3ec2f1..0000000 --- a/doc/rsyslog-example.conf +++ /dev/null @@ -1,163 +0,0 @@ -# A commented quick reference and sample configuration -# WARNING: This is not a manual, the full manual of rsyslog configuration is in -# rsyslog.conf (5) manpage -# -# "$" starts lines that contain new directives. The full list of directives -# can be found in /usr/share/doc/rsyslog-1.19.6/doc/rsyslog_conf.html or online -# at http://www.rsyslog.com/doc if you do not have (or find) a local copy. -# -# Set syslogd options - -# Some global directives -# ---------------------- - -# $AllowedSender - specifies which remote systems are allowed to send syslog messages to rsyslogd -# -------------- -$AllowedSender UDP, 127.0.0.1, 192.0.2.0/24, [::1]/128, *.example.net, somehost.example.com - -# $UMASK - specifies the rsyslogd processes' umask -# ------ -$umask 0000 - -# $FileGroup - Set the group for dynaFiles newly created -# ---------- -$FileGroup loggroup - -# $FileOwner - Set the file owner for dynaFiles newly created. -# ---------- -$FileOwner loguser - -# $IncludeConfig - include other files into the main configuration file -# -------------- -$IncludeConfig /etc/some-included-file.conf # one file -$IncludeConfig /etc/rsyslog.d/ # whole directory (must contain the final slash) - -# $ModLoad - Dynamically loads a plug-in and activates it -# -------- -$ModLoad ommysql # load MySQL functionality -$ModLoad /rsyslog/modules/somemodule.so # load a module via absolute path - - - -# Templates -# --------- - -# Templates allow to specify any format a user might want. -# They MUST be defined BEFORE they are used. - -# A template consists of a template directive, a name, the actual template text -# and optional options. A sample is: -# -$template MyTemplateName,"\7Text %property% some more text\n", - -# where: -# * $template - tells rsyslog that this line contains a template. -# * MyTemplateName - template name. All other config lines refer to this name. -# * "\7Text %property% some more text\n" - templage text - -# The backslash is an escape character, i.e. \7 rings the bell, \n is a new line. -# To escape: -# % = \% -# \ = \\ - -# Template options are case-insensitive. Currently defined are: -# sql format the string suitable for a SQL statement. This will replace single -# quotes ("'") by two single quotes ("''") to prevent the SQL injection -# (NO_BACKSLASH_ESCAPES turned off) -# stdsql - format the string suitable for a SQL statement that is to -# be sent to a standards-compliant sql server. -# (NO_BACKSLASH_ESCAPES turned on) - - - -# Properties inside templates -# --------------------------- - -# Properties can be modified by the property replacer. They are accessed -# inside the template by putting them between percent signs. The full syntax is as follows: - -# %propname:fromChar:toChar:options% - -# FromChar and toChar are used to build substrings. -# If you need to obtain the first 2 characters of the -# message text, you can use this syntax: -"%msg:1:2%". -# If you do not whish to specify from and to, but you want to -# specify options, you still need to include the colons. - -# For example, to convert the full message text to lower case only, use -# "%msg:::lowercase%". - -# The full list of property options can be found in rsyslog.conf(5) manpage - - - -# Samples of template definitions -# ------------------------------- - -# A template that resambles traditional syslogd file output: -$template TraditionalFormat,"%timegenerated% %HOSTNAME% %syslogtag%%msg:::drop-last-lf%\n" - -# A more verbose template: -$template precise,"%syslogpriority%,%syslogfacility%,%timegenerated::fulltime%,%HOSTNAME%,%syslogtag%,%msg%\n" - -# A template that resembles RFC 3164 on-the-wire format: -# (yes, there is NO space betwen syslogtag and msg! that's important!) -$template RFC3164fmt,"<%PRI%>%TIMESTAMP% %HOSTNAME% %syslogtag%%msg%" - -# a template resembling traditional wallmessage format: -$template wallmsg,"\r\n\7Message from syslogd@%HOSTNAME% at %timegenerated% ...\r\n %syslogtag%%msg%\n\r" - -# The template below emulates winsyslog format, but we need to check the time -# stamps used. It is also a good sampleof the property replacer in action. -$template WinSyslogFmt,"%HOSTNAME%,%timegenerated:1:10:date-rfc3339%,%timegenerated:12:19:date-rfc3339%,%timegenerated:1:10:date-rfc3339%,%timegenerated:12:19:date-rfc3339%,%syslogfacility%,%syslogpriority%,%syslogtag%%msg%\n" - -# A template used for database writing (notice it *is* an actual -# sql-statement): -$template dbFormat,"insert into SystemEvents (Message, Facility,FromHost, Priority, DeviceReportedTime, ReceivedAt, InfoUnitID, SysLogTag) values ('%msg%', %syslogfacility%, '%HOSTNAME%',%syslogpriority%, '%timereported:::date-mysql%', '%timegenerated:::date-mysql%', %iut%, '%syslogtag%')",sql - - - -# Samples of rules -# ---------------- -# Regular file -# ------------ -*.* /var/log/traditionalfile.log;TraditionalFormat # log to a file in the traditional format - -# Forwarding to remote machine -# ---------------------------- -*.* @172.19.2.16 # udp (standard for syslog) -*.* @@172.19.2.17 # tcp - -# Database action -# --------------- -# (you must have rsyslog-mysql package installed) -# !!! Don't forget to set permission of rsyslog.conf to 600 !!! -*.* >hostname,dbname,userid,password # (default Monitorware schema, can be created by /usr/share/doc/rsyslog-mysql-1.19.6/createDB.sql) - -# And this one uses the template defined above: -*.* >hostname,dbname,userid,password;dbFormat - -# Program to execute -# ------------------ -*.* ^alsaunmute # set default volume to soundcard - -# Filter using regex -# ------------------ -# if the user logges word rulez or rulezz or rulezzz or..., then we will shut down his pc -# (note, that + have to be double backslashed...) -:msg, regex, "rulez\\+" ^poweroff - -# A more complex example -# ---------------------- -$template bla_logged,"%timegenerated% the BLA was logged" -:msg, contains, "bla" ^logger;bla_logged - -# Pipes -# ----- -# first we need to create pipe by # mkfifo /a_big_pipe -*.* |/a_big_pipe - -# Discarding -# ---------- -*.* ~ # discards everything diff --git a/doc/rsyslog-vers.png b/doc/rsyslog-vers.png Binary files differdeleted file mode 100644 index e8ec8b8..0000000 --- a/doc/rsyslog-vers.png +++ /dev/null diff --git a/doc/rsyslog_conf.html b/doc/rsyslog_conf.html deleted file mode 100644 index c5f4d2e..0000000 --- a/doc/rsyslog_conf.html +++ /dev/null @@ -1,45 +0,0 @@ -<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> -<html><head><title>rsyslog.conf configuration file</title></head> -<body> -<h1>rsyslog.conf configuration file</h1> -<p><b>Rsyslog is configured via the rsyslog.conf file</b>, -typically found in /etc. By default, rsyslogd reads the file -/etc/rsyslog.conf. This may be changed by command line option "-f".</p> -<p><a href="http://wiki.rsyslog.com/index.php/Configuration_Samples"> -Configuration file examples can be found in the rsyslog wiki</a>. Also -keep the -<a href="http://www.rsyslog.com/config-snippets/">rsyslog config snippets</a> -on your mind. These are ready-to-use -real building blocks for rsyslog configuration. -</p> -<p>While rsyslogd contains enhancements over standard syslogd, -efforts have been made to keep the configuration file as compatible as -possible. While, for obvious reasons, <a href="features.html">enhanced -features</a> require a different config file syntax, rsyslogd -should be able to work with a standard syslog.conf file. This is -especially useful while you are migrating from syslogd to rsyslogd.</p> - -<p><b>Follow the links below to learn more about specific topics:</b></p> -<ul> -<li><a href="rsyslog_conf_basic_structure.html">Basic Structure</a></li> -<li><a href="rsyslog_conf_modules.html">Modules</a></li> -<li><a href="rsyslog_conf_templates.html">Templates</a></li> -<li><a href="rsyslog_conf_filter.html">Filter Conditions</a></li> -<li><a href="rsyslog_conf_actions.html">Actions (legacy format)</a></li> -<li><a href="rsyslog_conf_output.html">Output Channels</a></li> -<!--<li><a href="rsyslog_conf_examples.html">Examples</a></li>--> -<li><a href="rsyslog_conf_global.html">Legacy Configuration Directives</a></li> -<li><a href="rsyslog_conf_sysklogd_compatibility.html">sysklogd compatibility</a></li> -</ul> - -<p>[<a href="rsyslog_conf.html">back to top</a>] -[<a href="manual.html">manual index</a>] -[<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> -<p><font size="2">This documentation is part of the -<a href="http://www.rsyslog.com/">rsyslog</a> project.<br> -Copyright © 2008-2013 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and -<a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL -version 3 or higher.</font></p> -</body> -</html> - diff --git a/doc/rsyslog_conf_actions.html b/doc/rsyslog_conf_actions.html deleted file mode 100644 index fa240d9..0000000 --- a/doc/rsyslog_conf_actions.html +++ /dev/null @@ -1,414 +0,0 @@ -<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> -<html><head><title>Actions - rsyslog.conf</title></head> -<body> -<p>This is a part of the rsyslog.conf documentation.</p> -<a href="rsyslog_conf.html">back</a> -<h2>Actions</h2> -Action object describe what is to be done with a message. They are -implemented via <a href="rsyslog_conf_modules.html#om">outpout modules</a>. -<p>The action object has different parameters: -<ul> -<li>those that apply to all actions and are action specific. These - are documented below. -<li>parameters for the action queue. While they also apply to - all parameters, they are queue-specific, not action-specific (they - are the same that are used in rulesets, for example). -<li>action-specific parameters. These are specific to a certain - type of actions. They are documented by the output module - in question. -</ul> -<h3>General Action Parameters</h3> -<ul> - <li><b>name</b> word - <br>used for statistics gathering and documentation - <li><b>type</b> string - <br>Mandatory parameter for every action. The name of the module that should be used. </li> - <li><b>action.writeAllMarkMessages</b> on/off - <br>Normally, mark messages are written to actions only if the action was not recently executed (by default, recently means within the past 20 minutes). If this setting is switched to "on", mark messages are always sent to actions, no matter how recently they have been executed. In this mode, mark messages can be used as a kind of heartbeat. Note that this option auto-resets to "off", so if you intend to use it with multiple actions, it must be specified in front off all selector lines that should provide this functionality. </li> - <li><b>action.execOnlyEveryNthTime</b> integer - <br>If configured, the next action will only be executed every n-th time. For example, if configured to 3, the first two messages that go into the action will be dropped, the 3rd will actually cause the action to execute, the 4th and 5th will be dropped, the 6th executed under the action, ... and so on. Note: this setting is automatically re-set when the actual action is defined.</li> - <li><b>action.execOnlyEveryNthTimeout</b> integer - <br>Has a meaning only if Action.ExecOnlyEveryNthTime is also configured for the same action. If so, the timeout setting specifies after which period the counting of "previous actions" expires and a new action count is begun. Specify 0 (the default) to disable timeouts. -Why is this option needed? Consider this case: a message comes in at, eg., 10am. That's count 1. Then, nothing happens for the next 10 hours. At 8pm, the next one occurs. That's count 2. Another 5 hours later, the next message occurs, bringing the total count to 3. Thus, this message now triggers the rule. -The question is if this is desired behavior? Or should the rule only be triggered if the messages occur within an e.g. 20 minute window? If the later is the case, you need a -<br>Action.ExecOnlyEveryNthTimeTimeout="1200" -<br>This directive will timeout previous messages seen if they are older than 20 minutes. In the example above, the count would now be always 1 and consequently no rule would ever be triggered. </li> - <li><b>action.execOnlyOnceEveryInterval</b> integer - <br>Execute action only if the last execute is at last <seconds> seconds in the past (more info in ommail, but may be used with any action)</li> - <li><b>action.execOnlyWhenpReviousIsSuspended</b> on/off - <br>This directive allows to specify if actions should always be executed ("off," the default) or only if the previous action is suspended ("on"). This directive works hand-in-hand with the multiple actions per selector feature. It can be used, for example, to create rules that automatically switch destination servers or databases to a (set of) backup(s), if the primary server fails. Note that this feature depends on proper implementation of the suspend feature in the output module. All built-in output modules properly support it (most importantly the database write and the syslog message forwarder).</li> - <li><b>action.repeatedmsgcontainsoriginalmsg</b> on/off - <br>"last message repeated n times" messages, if generated, have a different format that contains the message that is being repeated. Note that only the first "n" characters are included, with n to be at least 80 characters, most probably more (this may change from version to version, thus no specific limit is given). The bottom line is that n is large enough to get a good idea which message was repeated but it is not necessarily large enough for the whole message. (Introduced with 4.1.5). Once set, it affects all following actions.</li> - <li><b>action.resumeRetryCount</b> integer - <br>[default 0, -1 means eternal]</li> - <li><b>action.resumeInterval</b> integer - <br>Sets the ActionResumeInterval for the action. The interval provided is always in seconds. Thus, multiply by 60 if you need minutes and 3,600 if you need hours (not recommended). -When an action is suspended (e.g. destination can not be connected), the action is resumed for the configured interval. Thereafter, it is retried. If multiple retires fail, the interval is automatically extended. This is to prevent excessive ressource use for retires. After each 10 retries, the interval is extended by itself. To be precise, the actual interval is (numRetries / 10 + 1) * Action.ResumeInterval. so after the 10th try, it by default is 60 and after the 100th try it is 330.</li> -</ul> - - -<h2>Legacy Format</h2> -<p><b>Be warned that legacy action format is hard to get right. It is -recommended to use RainerScript-Style action format whenever possible!</b> -A key problem with legacy format is that a single action is defined via -multiple configurations lines, which may be spread all across rsyslog.conf. -Even the definition of multiple actions may be intermixed (often not -intentional!). If legacy actions format needs to be used (e.g. some modules -may not yet implement the RainerScript format), it is strongly recommended -to place all configuration statements pertaining to a single action -closely together. -<p>Please also note that legacy action parameters <b>do not</b> affect -RainerScript action objects. So if you define for example: - -<code><pre> -$actionResumeRetryCount 10 -action(type="omfwd" target="server1.example.net") -@@server2.example.net -</pre></code> - -server1's "action.resumeRetryCount" parameter is <b>not</b> set, instead -server2's is! -<p>A goal of the new RainerScript action format was to avoid confusion -which parameters are actually used. As such, it would be counter-productive -to honor legacy action parameters inside a RainerScript definition. As -result, both types of action definitions are strictly (and nicely) -separated from each other. The bottom line is that if RainerScript actions -are used, one does not need to care about which legacy action parameters may -(still...) be in effect. -<p> -<p>Note that not all modules necessarily support legacy action format. -Especially newer modules are recommended to NOT support it. -<h3>Legacy Description</h3> -<p>Templates can be used with many actions. If used, the specified template -is used to generate the message content (instead of the default -template). To specify a template, write a semicolon after the action -value immediately followed by the template name.<br> -<br> -Beware: templates MUST be defined BEFORE they are used. It is OK to -define some templates, then use them in selector lines, define more -templates and use use them in the following selector lines. But it is -NOT permitted to use a template in a selector line that is above its -definition. If you do this, the action will be ignored.</p> -<p><b>You can have multiple actions for a single selector </b> (or -more precisely a single filter of such a selector line). Each action -must be on its own line and the line must start with an ampersand -('&') character and have no filters. An example would be</p> -<p><code><b>*.=crit :omusrmsg:rger<br> -& root<br> -& /var/log/critmsgs</b></code></p> -<p>These three lines send critical messages to the user rger and -root and also store them in /var/log/critmsgs. <b>Using multiple -actions per selector is</b> convenient and also <b>offers -a performance benefit</b>. As the filter needs to be evaluated -only once, there is less computation required to process the directive -compared to the otherwise-equal config directives below:</p> -<p><code><b>*.=crit :omusrmsg:rger<br> -*.=crit root<br> -*.=crit /var/log/critmsgs</b></code></p> -<p> </p> -<h3>Regular File</h3> -<p>Typically messages are logged to real files. The file usually is -specified by full pathname, beginning with a slash "/". -Starting with version 4.6.2 and 5.4.1 (previous v5 version do NOT support this) -relative file names can also be specified. To do so, these must begin with a -dot. For example, use "./file-in-current-dir.log" to specify a file in the -current directory. Please note that rsyslogd usually changes its working -directory to the root, so relative file names must be tested with care (they -were introduced primarily as a debugging vehicle, but may have useful other applications -as well).<br> -<br> -<br> -You may prefix each entry with the minus "-'' sign to omit syncing the -file after every logging. Note that you might lose information if the -system crashes right behind a write attempt. Nevertheless this might -give you back some performance, especially if you run programs that use -logging in a very verbose manner.</p> -<p>If your system is connected to a reliable UPS and you receive -lots of log data (e.g. firewall logs), it might be a very good idea to -turn of -syncing by specifying the "-" in front of the file name. </p> -<p><b>The filename can be either static </b>(always -the same) or <b>dynamic</b> (different based on message -received). The later is useful if you would automatically split -messages into different files based on some message criteria. For -example, dynamic file name selectors allow you to split messages into -different files based on the host that sent them. With dynamic file -names, everything is automatic and you do not need any filters. </p> -<p>It works via the template system. First, you define a template -for the file name. An example can be seen above in the description of -template. We will use the "DynFile" template defined there. Dynamic -filenames are indicated by specifying a questions mark "?" instead of a -slash, followed by the template name. Thus, the selector line for our -dynamic file name would look as follows:</p> -<blockquote> -<code>*.* ?DynFile</code> -</blockquote> -<p>That's all you need to do. Rsyslog will now automatically -generate file names for you and store the right messages into the right -files. Please note that the minus sign also works with dynamic file -name selectors. Thus, to avoid syncing, you may use</p> -<blockquote> -<code>*.* -?DynFile</code></blockquote> -<p>And of course you can use templates to specify the output -format:</p> -<blockquote> -<code>*.* ?DynFile;MyTemplate</code></blockquote> -<p><b>A word of caution:</b> rsyslog creates files as -needed. So if a new host is using your syslog server, rsyslog will -automatically create a new file for it.</p> -<p><b>Creating directories is also supported</b>. For -example you can use the hostname as directory and the program name as -file name:</p> -<blockquote> -<code>$template DynFile,"/var/log/%HOSTNAME%/%programname%.log"</code></blockquote> -<h3>Named Pipes</h3> -<p>This version of rsyslogd(8) has support for logging output to -named pipes (fifos). A fifo or named pipe can be used as a destination -for log messages by prepending a pipe symbol ("|'') to the name of the -file. This is handy for debugging. Note that the fifo must be created -with the mkfifo(1) command before rsyslogd(8) is started.</p> -<h3>Terminal and Console</h3> -<p>If the file you specified is a tty, special tty-handling is -done, same with /dev/console.</p> -<h3>Remote Machine</h3> -<p>Rsyslogd provides full remote logging, i.e. is able to send -messages to a remote host running rsyslogd(8) and to receive messages -from remote hosts. Using this feature you're able to control all syslog -messages on one host, if all other machines will log remotely to that. -This tears down administration needs.</p> -<p>To forward messages to another host, prepend the hostname with -the at sign ("@"). A single at sign means that messages will -be forwarded via UDP protocol (the standard for syslog). If you prepend -two at signs ("@@"), the messages will be transmitted via TCP. Please -note that plain TCP based syslog is not officially standardized, but -most major syslogds support it (e.g. syslog-ng or -<a href="http://www.winsyslog.com/">WinSyslog</a>). The -forwarding action indicator (at-sign) can be followed by one or more -options. If they are given, they must be immediately (without a space) -following the final at sign and be enclosed in parenthesis. The -individual options must be separated by commas. The following options -are right now defined:</p> -<table id="table2" border="1" width="100%"> -<tbody> -<tr> -<td> -<p align="center"><b>z<number></b></p> -</td> -<td>Enable zlib-compression for the message. The -<number> is the compression level. It can be 1 (lowest -gain, lowest CPU overhead) to 9 (maximum compression, highest CPU -overhead). The level can also be 0, which means "no compression". If -given, the "z" option is ignored. So this does not make an awful lot of -sense. There is hardly a difference between level 1 and 9 for typical -syslog messages. You can expect a compression gain between 0% and 30% -for typical messages. Very chatty messages may compress up to 50%, but -this is seldom seen with typically traffic. Please note that rsyslogd -checks the compression gain. Messages with 60 bytes or less will never -be compressed. This is because compression gain is pretty unlikely and -we prefer to save CPU cycles. Messages over that size are always -compressed. However, it is checked if there is a gain in compression -and only if there is, the compressed message is transmitted. Otherwise, -the uncompressed messages is transmitted. This saves the receiver CPU -cycles for decompression. It also prevents small message to actually -become larger in compressed form. -<p><b>Please note that when a TCP transport is used, -compression will also turn on syslog-transport-tls framing. See the "o" -option for important information on the implications.</b></p> -<p>Compressed messages are automatically detected and -decompressed by the receiver. There is nothing that needs to be -configured on the receiver side.</p> -</td> -</tr> -<tr> -<td> -<p align="center"><b>o</b></p> -</td> -<td><b>This option is experimental. Use at your own -risk and only if you know why you need it! If in doubt, do NOT turn it -on.</b> -<p>This option is only valid for plain TCP based -transports. It selects a different framing based on IETF internet draft -syslog-transport-tls-06. This framing offers some benefits over -traditional LF-based framing. However, the standardization effort is -not yet complete. There may be changes in upcoming versions of this -standard. Rsyslog will be kept in line with the standard. There is some -chance that upcoming changes will be incompatible to the current -specification. In this case, all systems using -transport-tls framing -must be upgraded. There will be no effort made to retain compatibility -between different versions of rsyslog. The primary reason for that is -that it seems technically impossible to provide compatibility between -some of those changes. So you should take this note very serious. It is -not something we do not *like* to do (and may change our mind if enough -people beg...), it is something we most probably *can not* do for -technical reasons (aka: you can beg as much as you like, it won't -change anything...).</p> -<p>The most important implication is that compressed syslog -messages via TCP must be considered with care. Unfortunately, it is -technically impossible to transfer compressed records over traditional -syslog plain tcp transports, so you are left with two evil choices...</p> -</td> -</tr> -</tbody> -</table> -<p><br> -The hostname may be followed by a colon and the destination port.</p> -<p>The following is an example selector line with forwarding:</p> -<p>*.* @@(o,z9)192.168.0.1:1470</p> -<p>In this example, messages are forwarded via plain TCP with -experimental framing and maximum compression to the host 192.168.0.1 at -port 1470.</p> -<p>*.* @192.168.0.1</p> -<p>In the example above, messages are forwarded via UDP to the -machine 192.168.0.1, the destination port defaults to 514. Messages -will not be compressed.</p> -<p>Note that IPv6 addresses contain colons. So if an IPv6 address is specified -in the hostname part, rsyslogd could not detect where the IP address ends -and where the port starts. There is a syntax extension to support this: -put squary brackets around the address (e.g. "[2001::1]"). Square -brackets also work with real host names and IPv4 addresses, too. -<p>A valid sample to send messages to the IPv6 host 2001::1 at port 515 -is as follows: -<p>*.* @[2001::1]:515 -<p>This works with TCP, too. -<p><b>Note to sysklogd users:</b> sysklogd does <b>not</b> -support RFC 3164 format, which is the default forwarding template in -rsyslog. As such, you will experience duplicate hostnames if rsyslog is -the sender and sysklogd is the receiver. The fix is simple: you need to -use a different template. Use that one:</p> -<p class="MsoPlainText">$template -sysklogd,"<%PRI%>%TIMESTAMP% %syslogtag%%msg%\""<br> -*.* @192.168.0.1;sysklogd</p> -<h3>List of Users</h3> -<p>Usually critical messages are also directed to "root'' on -that machine. You can specify a list of users that shall get the -message by simply writing ":omusrmsg: followed by the login name. For example, -the send messages to root, use ":omusrmsg:root". -You may specify more than one user -by separating them with commas (",''). Do not repeat the ":omusrmsg:" prefix in -this case. For example, to send data to users root and rger, use -":omusrmsg:root,rger" (do not use ":omusrmsg:root,:omusrmsg:rger", this is invalid). -If they're logged in they get -the message.</p> -<h3>Everyone logged on</h3> -<p>Emergency messages often go to all users currently online to -notify them that something strange is happening with the system. To -specify this wall(1)-feature use an asterisk as the user message -destination(":omusrmsg:*'').</p> -<h3>Call Plugin</h3> -<p>This is a generic way to call an output plugin. The plugin -must support this functionality. Actual parameters depend on the -module, so see the module's doc on what to supply. The general syntax -is as follows:</p> -<p>:modname:params;template</p> -<p>Currently, the ommysql database output module supports this -syntax (in addtion to the ">" syntax it traditionally -supported). For ommysql, the module name is "ommysql" and the params -are the traditional ones. The ;template part is not module specific, it -is generic rsyslog functionality available to all modules.</p> -<p>As an example, the ommysql module may be called as follows:</p> -<p>:ommysql:dbhost,dbname,dbuser,dbpassword;dbtemplate</p> -<p>For details, please see the "Database Table" section of this -documentation.</p> -<p>Note: as of this writing, the ":modname:" part is hardcoded -into the module. So the name to use is not necessarily the name the -module's plugin file is called.</p> -<h3>Database Table</h3> -<p>This allows logging of the message to a database table. -Currently, only MySQL databases are supported. However, other database -drivers will most probably be developed as plugins. By default, a <a href="http://www.monitorware.com/">MonitorWare</a>-compatible -schema is required for this to work. You can create that schema with -the createDB.SQL file that came with the rsyslog package. You can also<br> -use any other schema of your liking - you just need to define a proper -template and assign this template to the action.<br> -<br> -The database writer is called by specifying a greater-then sign -(">") in front of the database connect information. Immediately -after that<br> -sign the database host name must be given, a comma, the database name, -another comma, the database user, a comma and then the user's password. -If a specific template is to be used, a semicolon followed by the -template name can follow the connect information. This is as follows:<br> -<br> ->dbhost,dbname,dbuser,dbpassword;dbtemplate</p> -<p><b>Important: to use the database functionality, the -MySQL output module must be loaded in the config file</b> BEFORE -the first database table action is used. This is done by placing the</p> -<p><code><b>$ModLoad ommysql</b></code></p> -<p>directive some place above the first use of the database write -(we recommend doing at the the beginning of the config file).</p> -<h3>Discard</h3> -<p>If the discard action is carried out, the received message is -immediately discarded. No further processing of it occurs. Discard has -primarily been added to filter out messages before carrying on any -further processing. For obvious reasons, the results of "discard" are -depending on where in the configuration file it is being used. Please -note that once a message has been discarded there is no way to retrieve -it in later configuration file lines.</p> -<p>Discard can be highly effective if you want to filter out some -annoying messages that otherwise would fill your log files. To do that, -place the discard actions early in your log files. This often plays -well with property-based filters, giving you great freedom in -specifying what you do not want.</p> -<p>Discard is just the single tilde character with no further -parameters:</p> -<p>~</p> -<p>For example,</p> -<p>*.* ~</p> -<p>discards everything (ok, you can achive the same by not -running rsyslogd at all...).</p> -<h3>Output Channel</h3> -<p>Binds an output channel definition (see there for details) to -this action. Output channel actions must start with a $-sign, e.g. if -you would like to bind your output channel definition "mychannel" to -the action, use "$mychannel". Output channels support template -definitions like all all other actions.</p> -<h3>Shell Execute</h3> -<p>This executes a program in a subshell. The program is passed -the template-generated message as the only command line parameter. -Rsyslog waits until the program terminates and only then continues to -run.</p> -<p>^program-to-execute;template</p> -<p>The program-to-execute can be any valid executable. It -receives the template string as a single parameter (argv[1]).</p> -<p><b>WARNING:</b> The Shell Execute action was added -to serve an urgent need. While it is considered reasonable save when -used with some thinking, its implications must be considered. The -current implementation uses a system() call to execute the command. -This is not the best way to do it (and will hopefully changed in -further releases). Also, proper escaping of special characters is done -to prevent command injection. However, attackers always find smart ways -to circumvent escaping, so we can not say if the escaping applied will -really safe you from all hassles. Lastly, rsyslog will wait until the -shell command terminates. Thus, a program error in it (e.g. an infinite -loop) can actually disable rsyslog. Even without that, during the -programs run-time no messages are processed by rsyslog. As the IP -stacks buffers are quickly overflowed, this bears an increased risk of -message loss. You must be aware of these implications. Even though they -are severe, there are several cases where the "shell execute" action is -very useful. This is the reason why we have included it in its current -form. To mitigate its risks, always a) test your program thoroughly, b) -make sure its runtime is as short as possible (if it requires a longer -run-time, you might want to spawn your own sub-shell asynchronously), -c) apply proper firewalling so that only known senders can send syslog -messages to rsyslog. Point c) is especially important: if rsyslog is -accepting message from any hosts, chances are much higher that an -attacker might try to exploit the "shell execute" action.</p> -<h3>Template Name</h3> -<p>Every ACTION can be followed by a template name. If so, that -template is used for message formatting. If no name is given, a -hard-coded default template is used for the action. There can only be -one template name for each given action. The default template is -specific to each action. For a description of what a template is and -what you can do with it, see "TEMPLATES" at the top of this document.</p> - -<p>[<a href="manual.html">manual index</a>] -[<a href="rsyslog_conf.html">rsyslog.conf</a>] -[<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> -<p><font size="2">This documentation is part of the -<a href="http://www.rsyslog.com/">rsyslog</a> project.<br> -Copyright © 2008-2013 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and -<a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL -version 2 or higher.</font></p> -</body> -</html> - diff --git a/doc/rsyslog_conf_basic_structure.html b/doc/rsyslog_conf_basic_structure.html deleted file mode 100644 index f5d4891..0000000 --- a/doc/rsyslog_conf_basic_structure.html +++ /dev/null @@ -1,103 +0,0 @@ -<html><head><title>Basic Structure - rsyslog.conf</title></head> -<body> -<h1>Basic rsyslog.conf Structure</h1> -<p>This is a part of the rsyslog.conf documentation.</p> -<a href="rsyslog_conf.html">Back to rsyslog.conf manual</a> -<p>Rsyslog supports three different types of configuration statements -concurrently: -<ul> -<li><b>sysklogd</b> - this is the plain old format, thaught everywhere -and still pretty useful for simple use cases. Note that some very -few constructs are no longer supported because they are incompatible -with newer features. These are mentioned in the compatibility docs. -<li><b>legacy rsyslog</b> - these are statements that begin with a dollar -sign. They set some configuration parameters and modify e.g. the way -actions operate. This is the only format supported in pre-v6 versions of -rsyslog. It is still fully supported in v6 and above. Note that some -plugins and features may still only be available through legacy format -(because plugins need to be explicitely upgraded to use the new style -format, and this hasn't happened to all plugins). -<li><b>RainerScript</b> - the new style format. This is the best and most -precise format to be used for more complex cases. The rest of this page -assumes RainerScript based rsyslog.conf. -</ul> -<p>The rsyslog.conf files consists of statements. For old style (sysklogd & legacy -rsyslog), lines do matter. For new style (RainerScript) line spacing is irrelevant. -Most importantly, this means with new style actions and all other objects can split -across lines as users want to. -<h2>Comments</h2> -<p>There are two types of comments: -<ul> -<li><b>#-Comments</b> - start with a hash sign (#) and run to the end of the line -<li><b>C-style Comments</b> - start with /* and end with */, just like in the C -programming language. They can be used to comment out multiple lines at one. Comment -nesting is not supported, but #-Comments can be contained inside a C-style comment. -</ul> - -<h2>Processing Order</h2> -<p>Directives are processed from the top of rsyslog.conf to the bottom. Sequence -matters. For example, if you stop processing of a message, obviously all statements -after the stop statement are never evaluated. - -<h3>Flow Control Statements</h3> -<ul> -<li><b>if expr then ... else ...</b> - conditional execution -<li><b>stop</b> - stops processing the current message -<li><b>call</b> - calls a ruleset (just like a subroutine call) -<li><b>continue</b> - a NOP, useful e.g. inside the then part of an if -</ul> - -<h3>Data Manipulation Statements</h3> -<ul> -<li><b>set</b> - <a href="http://www.rsyslog.com/how-to-set-variables-in-rsyslog-v7/">sets</a> -a user variable -<li><b>unset</b> - deletes a previously set user variable -</ul> - -<h2>Inputs</h2> -<p>Every input requires an input module to be loaded and a listener defined for it. -Full details can be found inside the <a href="rsyslog_conf_modules.html">rsyslog -modules</a> documentation. Once loaded, inputs are defined via the -<b>input()</b> object. - -<h2>Outputs</h2> -<p>Outputs are also called "actions". A small set of actions is pre-loaded (like -the output file writer, which is used in almost every rsyslog.conf), others must -be loaded just like inputs. -<p>An action is invoked via the <b>action(type="type" ...)</b> object. Type is -mandatory and must contain the name of the plugin to be called (e.g. "omfile" or -"ommongodb"). Other paramters may be present. Their type and use depends on -the output plugin in question. - -<h2>Rulesets and Rules</h2> -<p>Rulesets and rules form the basis of rsyslog processing. In short, a rule -is a way how rsyslog shall process a specific message. Usually, there is a type -of filter (if-statement) in front of the rule. Complex nesting of rules is possible, -much like in a programming language. -<p>Rulesets are containers for rules. A single ruleset can contain many rules. In -the programming language analogy, one may think of a ruleset like being a program. -A ruleset can be "bound" (assigned) to a specific input. In the analogy, this means that when -a message comes in via that input, the "program" (ruleset) bound to it will be executed -(but not any other!). -<p>There is detail documentation available for -<a href="multi_ruleset.html">rsyslog rulesets</a>. -<p>For quick reference, rulesets are defined as follows: -<pre> -ruleset(name="rulesetname") { - action(type="omfile" file="/path/to/file") - action(type="..." ...) - /* and so on... */ -} -</pre> - -<p>[<a href="manual.html">manual index</a>] -[<a href="rsyslog_conf.html">rsyslog.conf</a>] -[<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> -<p><font size="2">This documentation is part of the -<a href="http://www.rsyslog.com/">rsyslog</a> project.<br> -Copyright © 2008-2013 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and -<a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL -version 3 or higher.</font></p> -</body> -</html> - diff --git a/doc/rsyslog_conf_filter.html b/doc/rsyslog_conf_filter.html deleted file mode 100644 index a795193..0000000 --- a/doc/rsyslog_conf_filter.html +++ /dev/null @@ -1,288 +0,0 @@ -<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> -<html><head><title>Filter Conditions - rsyslog.conf</title></head> -<body> -<p>This is a part of the rsyslog.conf documentation.</p> -<a href="rsyslog_conf.html">back</a> -<h2>Filter Conditions</h2> -<p>Rsyslog offers three different types "filter conditions":</p> -<ul> -<li><a href="http://www.rainerscript.com/">RainerScript</a>-based filters</li> -<li>"traditional" severity and facility based selectors</li> -<li>property-based filters</li> -</ul> -<h3>RainerScript-Based Filters</h3> -RainerScript based filters are the prime means of creating complex rsyslog configuration. -The permit filtering on arbitrary complex expressions, which can include boolean, -arithmetic and string operations. They also support full nesting of filters, just -as you know from other scripting environments. -<br> -Scripts based filters are indicated by the keyword "if", as usual. -They have this format:<br> -<br> -if expr then block else block -<br> -"If" and "then" are fixed keywords that mus be present. "expr" is a -(potentially quite complex) expression. So the <a href="expression.html">expression documentation</a> for -details. -The keyword "else" and its associated block is optional. Note that a block can contain either -a single action (chain), or an arbitrary complex script enclosed in curly braces, e.g.: -<br> -<pre> -if $programname == 'prog1' then { - action(type="omfile" file="/var/log/prog1.log") - if $msg contains 'test' then - action(type="omfile" file="/var/log/prog1test.log") - else - action(type="omfile" file="/var/log/prog1notest.log") -} -</pre> -<br> -Other types of filtes can also be combined with the pure RainerScript ones. This makes -it particularly easy to migrate from early config files to RainerScript. Also, the traditional -syslog PRI-based filters are a good and easy to use addition. While they are legacy, we still -recommend there use where they are up to the job. We do NOT, however, recommend property-based -filters any longer. As an example, the following is perfectly valid: -<br> -<pre> -if $fromhost == 'host1' then { - mail.* action(type="omfile" file="/var/log/host1/mail.log") - *.err /var/log/host1/errlog # this is also still valid - # - # more "old-style rules" ... - # -} else { - mail.* action(type="omfile" file="/var/log/mail.log") - *.err /var/log/errlog - # - # more "old-style rules" ... - # -} -</pre> -<br> - -Right now, you need to specify numerical values if you would like to -check for facilities and severity. These can be found in <a href="http://www.ietf.org/rfc/rfc3164.txt">RFC 3164</a>. -If you don't like that, you can of course also use the textual property -- just be sure to use the right one. As expression support is enhanced, -this will change. For example, if you would like to filter on message -that have facility local0, start with "DEVNAME" and have either -"error1" or "error0" in their message content, you could use the -following filter:<br> -<br> -<code> -if $syslogfacility-text == 'local0' and $msg -startswith 'DEVNAME' and ($msg contains 'error1' or $msg contains -'error0') then /var/log/somelog<br> -</code> -<br> -Please note that the above <span style="font-weight: bold;">must -all be on one line</span>! And if you would like to store all -messages except those that contain "error1" or "error0", you just need -to add a "not":<br> -<br> -<code> -if $syslogfacility-text == 'local0' and $msg -startswith 'DEVNAME' and <span style="font-weight: bold;">not</span> -($msg contains 'error1' or $msg contains -'error0') then /var/log/somelog<br> -</code> -<br> -If you would like to do case-insensitive comparisons, use -"contains_i" instead of "contains" and "startswith_i" instead of -"startswith".<br> -<br> -Regular expressions are supported via functions (see function list). - -<h3>Selectors</h3> -<p><b>Selectors are the traditional way of filtering syslog -messages.</b> They have been kept in rsyslog with their original -syntax, because it is well-known, highly effective and also needed for -compatibility with stock syslogd configuration files. If you just need -to filter based on priority and facility, you should do this with -selector lines. They are <b>not</b> second-class citizens -in rsyslog and offer the best performance for this job.</p> -<p>The selector field itself again consists of two parts, a -facility and a priority, separated by a period (".''). Both parts are -case insensitive and can also be specified as decimal numbers, but -don't do that, you have been warned. Both facilities and priorities are -described in syslog(3). The names mentioned below correspond to the -similar LOG_-values in /usr/include/syslog.h.<br> -<br> -The facility is one of the following keywords: auth, authpriv, cron, -daemon, kern, lpr, mail, mark, news, security (same as auth), syslog, -user, uucp and local0 through local7. The keyword security should not -be used anymore and mark is only for internal use and therefore should -not be used in applications. Anyway, you may want to specify and -redirect these messages here. The facility specifies the subsystem that -produced the message, i.e. all mail programs log with the mail facility -(LOG_MAIL) if they log using syslog.<br> -<br> -The priority is one of the following keywords, in ascending order: -debug, info, notice, warning, warn (same as warning), err, error (same -as err), crit, alert, emerg, panic (same as emerg). The keywords error, -warn and panic are deprecated and should not be used anymore. The -priority defines the severity of the message.<br> -<br> -The behavior of the original BSD syslogd is that all messages of the -specified priority and higher are logged according to the given action. -Rsyslogd behaves the same, but has some extensions.<br> -<br> -In addition to the above mentioned names the rsyslogd(8) understands -the following extensions: An asterisk ("*'') stands for all facilities -or all priorities, depending on where it is used (before or after the -period). The keyword none stands for no priority of the given facility.<br> -<br> -You can specify multiple facilities with the same priority pattern in -one statement using the comma (",'') operator. You may specify as much -facilities as you want. Remember that only the facility part from such -a statement is taken, a priority part would be skipped.</p> -<p>Multiple selectors may be specified for a single action using -the semicolon (";'') separator. Remember that each selector in the -selector field is capable to overwrite the preceding ones. Using this -behavior you can exclude some priorities from the pattern.</p> -<p>Rsyslogd has a syntax extension to the original BSD source, -that makes its use more intuitively. You may precede every priority -with an equals sign ("='') to specify only this single priority and -not any of the above. You may also (both is valid, too) precede the -priority with an exclamation mark ("!'') to ignore all that -priorities, either exact this one or this and any higher priority. If -you use both extensions than the exclamation mark must occur before the -equals sign, just use it intuitively.</p> -<h3>Property-Based Filters</h3> -<p>Property-based filters are unique to rsyslogd. They allow to -filter on any property, like HOSTNAME, syslogtag and msg. A list of all -currently-supported properties can be found in the <a href="property_replacer.html">property replacer documentation</a> -(but keep in mind that only the properties, not the replacer is -supported). With this filter, each properties can be checked against a -specified value, using a specified compare operation.</p> -<p>A property-based filter must start with a colon in column 0. -This tells rsyslogd that it is the new filter type. The colon must be -followed by the property name, a comma, the name of the compare -operation to carry out, another comma and then the value to compare -against. This value must be quoted. There can be spaces and tabs -between the commas. Property names and compare operations are -case-sensitive, so "msg" works, while "MSG" is an invalid property -name. In brief, the syntax is as follows:</p> -<p><code><b>:property, [!]compare-operation, "value"</b></code></p> -<p>The following <b>compare-operations</b> are -currently supported:</p> -<table id="table1" border="1" width="100%"> -<tbody> -<tr> -<td>contains</td> -<td>Checks if the string provided in value is contained in -the property. There must be an exact match, wildcards are not supported.</td> -</tr> -<tr> -<td>isempty</td> -<td>Checks if the property is empty. The value is discarded. This is -especially useful when working with normalized data, where some fields -may be populated based on normalization result. -Available since 6.6.2. -</tr> -<tr> -<td>isequal</td> -<td>Compares the "value" string provided and the property -contents. These two values must be exactly equal to match. The -difference to contains is that contains searches for the value anywhere -inside the property value, whereas all characters must be identical for -isequal. As such, isequal is most useful for fields like syslogtag or -FROMHOST, where you probably know the exact contents.</td> -</tr> -<tr> -<td>startswith</td> -<td>Checks if the value is found exactly at the beginning -of the property value. For example, if you search for "val" with -<p><code><b>:msg, startswith, "val"</b></code></p> -<p>it will be a match if msg contains "values are in this -message" but it won't match if the msg contains "There are values in -this message" (in the later case, contains would match). Please note -that "startswith" is by far faster than regular expressions. So -it makes very much sense (performance-wise) to use "startswith".</p> -<p>Note: when processing syslog messages, please note that $msg usually -starts with a space. The reason for this is RFC3164. Please read the -<a href="http://www.rsyslog.com/log-normalization-and-the-leading-space/">detail -description</a> of what that means to you. In short, you need to make sure -that you include the first space if you use "startswith", otherwise you will -not get matches. -</td> -</tr> -<tr> -<td>regex</td> -<td>Compares the property against the provided POSIX -BRE regular -expression.</td> -</tr> -<tr> -<td>ereregex</td> -<td>Compares the property against the provided POSIX -ERE regular -expression.</td> -</tr> -</tbody> -</table> -<p>You can use the bang-character (!) immediately in front of a -compare-operation, the outcome of this operation is negated. For -example, if msg contains "This is an informative message", the -following sample would not match:</p> -<p><code><b>:msg, contains, "error"</b></code></p> -<p>but this one matches:</p> -<p><code><b>:msg, !contains, "error"</b></code></p> -<p>Using negation can be useful if you would like to do some -generic processing but exclude some specific events. You can use the -discard action in conjunction with that. A sample would be:</p> -<p><code><b>*.* -/var/log/allmsgs-including-informational.log<br> -:msg, contains, "informational" <font color="#ff0000" size="4">~</font> -<br> -*.* /var/log/allmsgs-but-informational.log</b></code></p> -<p>Do not overlook the red tilde in line 2! In this sample, all -messages are written to the file allmsgs-including-informational.log. -Then, all messages containing the string "informational" are discarded. -That means the config file lines below the "discard line" (number 2 in -our sample) will not be applied to this message. Then, all remaining -lines will also be written to the file allmsgs-but-informational.log.</p> -<p><b>Value</b> is a quoted string. It supports some -escape sequences:</p> -<p>\" - the quote character (e.g. "String with \"Quotes\"")<br> -\\ - the backslash character (e.g. "C:\\tmp")</p> -<p>Escape sequences always start with a backslash. Additional -escape sequences might be added in the future. Backslash characters <b>must</b> -be escaped. Any other sequence then those outlined above is invalid and -may lead to unpredictable results.</p> -<p>Probably, "msg" is the most prominent use case of property -based filters. It is the actual message text. If you would like to -filter based on some message content (e.g. the presence of a specific -code), this can be done easily by:</p> -<p><code><b>:msg, contains, "ID-4711"</b></code></p> -<p>This filter will match when the message contains the string -"ID-4711". Please note that the comparison is case-sensitive, so it -would not match if "id-4711" would be contained in the message.</p> -<p><code><b>:msg, regex, "fatal .* error"</b></code></p> -<p>This filter uses a POSIX regular expression. It matches when -the -string contains the words "fatal" and "error" with anything in between -(e.g. "fatal net error" and "fatal lib error" but not "fatal error" as -two spaces are required by the regular expression!).</p> -<p>Getting property-based filters right can sometimes be -challenging. In order to help you do it with as minimal effort as -possible, rsyslogd spits out debug information for all property-based -filters during their evaluation. To enable this, run rsyslogd in -foreground and specify the "-d" option.</p> -<p>Boolean operations inside property based filters (like -'message contains "ID17" or message contains "ID18"') are currently not -supported (except for "not" as outlined above). Please note that while -it is possible to query facility and severity via property-based -filters, it is far more advisable to use classic selectors (see above) -for those cases.</p> -<p>[<a href="manual.html">manual index</a>] -[<a href="rsyslog_conf.html">rsyslog.conf</a>] -[<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> -<p><font size="2">This documentation is part of the -<a href="http://www.rsyslog.com/">rsyslog</a> project.<br> -Copyright © 2008 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and -<a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL -version 2 or higher.</font></p> -</body> -</html> - diff --git a/doc/rsyslog_conf_global.html b/doc/rsyslog_conf_global.html deleted file mode 100644 index 651808f..0000000 --- a/doc/rsyslog_conf_global.html +++ /dev/null @@ -1,333 +0,0 @@ -<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> -<html><head><title>Configuration Directives - rsyslog.conf</title></head> -<body> -<p>This is a part of the rsyslog.conf documentation.</p> -<a href="rsyslog_conf.html">back</a> -<h2>Configuration Directives</h2> -<p>All configuration directives need to be specified on a line by their -own and must start with a dollar-sign. Note that those starting with -the word "Action" modify the next action and should be specified -in front of it. -<p>Here is a list in alphabetical order. Follow links for a description.</p> -<p>Not all directives have an in-depth description right now. -Default values for them are in bold. A more in-depth description will -appear as implementation progresses. -</p> -<p><b>Be sure to read information about <a href="queues.html">queues in rsyslog</a></b> - -many parameter settings modify queue parameters. If in doubt, use the -default, it is usually well-chosen and applicable in most cases.</p> -<ul> -<li><a href="rsconf1_abortonuncleanconfig.html">$AbortOnUncleanConfig</a> - abort startup if there is -any issue with the config file</li> -<li><a href="rsconf1_actionexeconlywhenpreviousissuspended.html">$ActionExecOnlyWhenPreviousIsSuspended</a></li> -<li>$ActionName <a_single_word> - used primarily for documentation, e.g. when -generating a configuration graph. Available sice 4.3.1. -<li>$ActionExecOnlyOnceEveryInterval <seconds> - -execute action only if the last execute is at last -<seconds> seconds in the past (more info in <a href="ommail.html">ommail</a>, -but may be used with any action)</li> -<li><i><b>$ActionExecOnlyEveryNthTime</b> <number></i> - If configured, the next action will -only be executed every n-th time. For example, if configured to 3, the first two messages -that go into the action will be dropped, the 3rd will actually cause the action to execute, -the 4th and 5th will be dropped, the 6th executed under the action, ... and so on. Note: -this setting is automatically re-set when the actual action is defined.</li> -<li><i><b>$ActionExecOnlyEveryNthTimeTimeout</b> <number-of-seconds></i> - has a meaning only if -$ActionExecOnlyEveryNthTime is also configured for the same action. If so, the timeout -setting specifies after which period the counting of "previous actions" expires and -a new action count is begun. Specify 0 (the default) to disable timeouts. -<br> -<i>Why is this option needed?</i> Consider this case: a message comes in at, eg., 10am. That's -count 1. Then, nothing happens for the next 10 hours. At 8pm, the next -one occurs. That's count 2. Another 5 hours later, the next message -occurs, bringing the total count to 3. Thus, this message now triggers -the rule. -<br> -The question is if this is desired behavior? Or should the rule only be -triggered if the messages occur within an e.g. 20 minute window? If the -later is the case, you need a -<br> -$ActionExecOnlyEveryNthTimeTimeout 1200 -<br> -This directive will timeout previous messages seen if they are older -than 20 minutes. In the example above, the count would now be always 1 -and consequently no rule would ever be triggered. - -<li><a href="omfile.html">$ActionFileDefaultTemplate</a> [templateName] - sets a new default template for file actions</li> -<li><a href="omfile.html">$ActionFileEnableSync</a> [on/<span style="font-weight: bold;">off</span>] - enables file -syncing capability of omfile</li> -<li><a href="omfwd.html">$ActionForwardDefaultTemplate</a> [templateName] - sets a new -default template for UDP and plain TCP forwarding action</li> -<li>$ActionGSSForwardDefaultTemplate [templateName] - sets a -new default template for GSS-API forwarding action</li> -<li>$ActionQueueCheckpointInterval <number></li> -<li>$ActionQueueDequeueBatchSize <number> [default 16]</li> -<li>$ActionQueueDequeueSlowdown <number> [number -is timeout in <i> micro</i>seconds (1000000us is 1sec!), -default 0 (no delay). Simple rate-limiting!]</li> -<li>$ActionQueueDiscardMark <number> [default -9750]</li> -<li>$ActionQueueDiscardSeverity <number> -[*numerical* severity! default 8 (nothing discarded)]</li> -<li>$ActionQueueFileName <name></li> -<li>$ActionQueueHighWaterMark <number> [default -8000]</li> -<li>$ActionQueueImmediateShutdown [on/<b>off</b>]</li> -<li>$ActionQueueSize <number></li> -<li>$ActionQueueLowWaterMark <number> [default -2000]</li> -<li>$ActionQueueMaxFileSize <size_nbr>, default 1m</li> -<li>$ActionQueueTimeoutActionCompletion <number> -[number is timeout in ms (1000ms is 1sec!), default 1000, 0 means -immediate!]</li> -<li>$ActionQueueTimeoutEnqueue <number> [number -is timeout in ms (1000ms is 1sec!), default 2000, 0 means indefinite]</li> -<li>$ActionQueueTimeoutShutdown <number> [number -is timeout in ms (1000ms is 1sec!), default 0 (indefinite)]</li> -<li>$ActionQueueWorkerTimeoutThreadShutdown -<number> [number is timeout in ms (1000ms is 1sec!), -default 60000 (1 minute)]</li> -<li>$ActionQueueType [FixedArray/LinkedList/<b>Direct</b>/Disk]</li> -<li>$ActionQueueSaveOnShutdown [on/<b>off</b>] -</li> -<li>$ActionQueueWorkerThreads <number>, num worker threads, default 1, recommended 1</li> -<li>$ActionQueueWorkerThreadMinumumMessages <number>, default 100</li> -<li><a href="rsconf1_actionresumeinterval.html">$ActionResumeInterval</a></li> -<li>$ActionResumeRetryCount <number> [default 0, -1 means eternal]</li> -<li><a href="omfwd.html">$ActionSendResendLastMsgOnReconnect</a> <[on/<b>off</b>]> specifies if the last message is to be resend when a connecition breaks and has been reconnected. May increase reliability, but comes at the risk of message duplication. -<li><a href="omfwd.html">$ActionSendStreamDriver</a> <driver basename> just like $DefaultNetstreamDriver, but for the specific action</li> -<li><a href="omfwd.html">$ActionSendStreamDriverMode</a> <mode>, default 0, mode to use with the stream driver (driver-specific)</li> -<li><a href="omfwd.html">$ActionSendStreamDriverAuthMode</a> <mode>, authentication mode to use with the stream driver. Note that this directive requires TLS -netstream drivers. For all others, it will be ignored. -(driver-specific)</li> -<li><a href="omfwd.html">$ActionSendStreamDriverPermittedPeer</a> <ID>, accepted fingerprint (SHA1) or name of remote peer. Note that this directive requires TLS -netstream drivers. For all others, it will be ignored. -(driver-specific) -<span style="font-weight: bold;"> directive may go away</span>!</li> -<li><a href="omfwd.html"><b>$ActionSendTCPRebindInterval</b> nbr</a>- [available since 4.5.1] - instructs the TCP send -action to close and re-open the connection to the remote host every nbr of messages sent. -Zero, the default, means that no such processing is done. This directive is useful for -use with load-balancers. Note that there is some performance overhead associated with it, -so it is advisable to not too often "rebind" the connection (what -"too often" actually means depends on your configuration, a rule of thumb is -that it should be not be much more often than once per second).</li> -<li><a href="omfwd.html"><b>$ActionSendUDPRebindInterval</b> nbr</a>- [available since 4.3.2] - instructs the UDP send -action to rebind the send socket every nbr of messages sent. Zero, the default, means -that no rebind is done. This directive is useful for use with load-balancers.</li> -<li><b>$ActionWriteAllMarkMessages</b> [on/<b>off</b>]- [available since 5.1.5] - normally, mark messages -are written to actions only if the action was not recently executed (by default, recently means within the -past 20 minutes). If this setting is switched to "on", mark messages are always sent to actions, -no matter how recently they have been executed. In this mode, mark messages can be used as a kind of -heartbeat. Note that this option auto-resets to "off", so if you intend to use it with multiple -actions, it must be specified in front off <b>all</b> selector lines that should provide this -functionality. -</li> -<li><a href="rsconf1_allowedsender.html">$AllowedSender</a></li> -<li><a href="rsconf1_controlcharacterescapeprefix.html">$ControlCharacterEscapePrefix</a></li> -<li><a href="rsconf1_debugprintcfsyslinehandlerlist.html">$DebugPrintCFSyslineHandlerList</a></li> - -<li><a href="rsconf1_debugprintmodulelist.html">$DebugPrintModuleList</a></li> -<li><a href="rsconf1_debugprinttemplatelist.html">$DebugPrintTemplateList</a></li> -<li>$DefaultNetstreamDriver <drivername>, the default <a href="netstream.html">network stream driver</a> to use. Defaults to ptcp.$DefaultNetstreamDriverCAFile </path/to/cafile.pem></li> -<li>$DefaultNetstreamDriverCertFile </path/to/certfile.pem></li> -<li>$DefaultNetstreamDriverKeyFile </path/to/keyfile.pem></li> -<li><b>$DefaultRuleset</b> <i>name</i> - changes the default ruleset for unbound inputs to -the provided <i>name</i> (the default default ruleset is named -"RSYSLOG_DefaultRuleset"). It is advised to also read -our paper on <a href="multi_ruleset.html">using multiple rule sets in rsyslog</a>.</li> -<li><a href="omfile.html"><b>$CreateDirs</b></a> [<b>on</b>/off] - create directories on an as-needed basis</li> -<li><a href="omfile.html">$DirCreateMode</a></li> -<li><a href="omfile.html">$DirGroup</a></li> -<li><a href="omfile.html">$DirOwner</a></li> -<li><a href="rsconf1_dropmsgswithmaliciousdnsptrrecords.html">$DropMsgsWithMaliciousDnsPTRRecords</a></li> -<li><a href="rsconf1_droptrailinglfonreception.html">$DropTrailingLFOnReception</a></li> -<li><a href="omfile.html">$DynaFileCacheSize</a></li> -<li><a href="rsconf1_escape8bitcharsonreceive.html">$Escape8BitCharactersOnReceive</a></li> -<li><a href="rsconf1_escapecontrolcharactersonreceive.html">$EscapeControlCharactersOnReceive</a></li> -<li><b>$EscapeControlCharactersOnReceive</b> [<b>on</b>|off] - escape USASCII HT character</li> -<li>$SpaceLFOnReceive [on/<b>off</b>] - instructs rsyslogd to replace LF with spaces during message reception (sysklogd compatibility aid)</li> -<li>$ErrorMessagesToStderr [<b>on</b>|off] - direct rsyslogd error message to stderr (in addition to other targets)</li> -<li><a href="omfile.html">$FailOnChownFailure</a></li> -<li><a href="omfile.html">$FileCreateMode</a></li> -<li><a href="omfile.html">$FileGroup</a></li> -<li><a href="omfile.html">$FileOwner</a></li> -<li><a href="rsconf1_generateconfiggraph.html">$GenerateConfigGraph</a></li> -<li><a href="rsconf1_gssforwardservicename.html">$GssForwardServiceName</a></li> -<li><a href="rsconf1_gsslistenservicename.html">$GssListenServiceName</a></li> -<li><a href="rsconf1_gssmode.html">$GssMode</a></li> -<li><a href="rsconf1_includeconfig.html">$IncludeConfig</a></li><li>MainMsgQueueCheckpointInterval <number></li> -<li><b>$LocalHostName</b> [name] - this directive permits to overwrite the system -hostname with the one specified in the directive. If the directive is given -multiple times, all but the last one will be ignored. Please note that startup -error messages may be issued with the real hostname. This is by design and not -a bug (but one may argue if the design should be changed ;)). Available since -4.7.4+, 5.7.3+, 6.1.3+. -<li><b>$LogRSyslogStatusMessages</b> [<b>on</b>/off] - If set to on (the default), -rsyslog emits message on startup and shutdown as well as when it is HUPed. -This information might be needed by some log analyzers. If set to off, no such -status messages are logged, what may be useful for other scenarios. -[available since 4.7.0 and 5.3.0] -<li><b>$MainMsgQueueDequeueBatchSize</b> <number> [default 32]</li> -<li>$MainMsgQueueDequeueSlowdown <number> [number -is timeout in <i> micro</i>seconds (1000000us is 1sec!), -default 0 (no delay). Simple rate-limiting!]</li> -<li>$MainMsgQueueDiscardMark <number> [default 9750]</li> -<li>$MainMsgQueueDiscardSeverity <severity> -[either a textual or numerical severity! default 4 (warning)]</li> -<li>$MainMsgQueueFileName <name></li> -<li>$MainMsgQueueHighWaterMark <number> [default -8000]</li> -<li>$MainMsgQueueImmediateShutdown [on/<b>off</b>]</li> -<li><a href="rsconf1_mainmsgqueuesize.html">$MainMsgQueueSize</a></li> -<li>$MainMsgQueueLowWaterMark <number> [default -2000]</li> -<li>$MainMsgQueueMaxFileSize <size_nbr>, default -1m</li> -<li>$MainMsgQueueTimeoutActionCompletion -<number> [number is timeout in ms (1000ms is 1sec!), -default -1000, 0 means immediate!]</li> -<li>$MainMsgQueueTimeoutEnqueue <number> [number -is timeout in ms (1000ms is 1sec!), default 2000, 0 means indefinite]</li> -<li>$MainMsgQueueTimeoutShutdown <number> [number -is timeout in ms (1000ms is 1sec!), default 0 (indefinite)]</li> -<li>$MainMsgQueueWorkerTimeoutThreadShutdown -<number> [number is timeout in ms (1000ms is 1sec!), -default 60000 (1 minute)]</li> -<li>$MainMsgQueueType [<b>FixedArray</b>/LinkedList/Direct/Disk]</li> -<li>$MainMsgQueueSaveOnShutdown [on/<b>off</b>] -</li> -<li>$MainMsgQueueWorkerThreads <number>, num -worker threads, default 1, recommended 1</li> -<li>$MainMsgQueueWorkerThreadMinumumMessages <number>, default 100</li> -<li><a href="rsconf1_markmessageperiod.html">$MarkMessagePeriod</a> (immark)</li> -<li><b><i>$MaxMessageSize</i></b> <size_nbr>, default 2k - allows to specify maximum supported message size -(both for sending and receiving). The default -should be sufficient for almost all cases. Do not set this below 1k, as it would cause -interoperability problems with other syslog implementations.<br> -Change the setting to e.g. 32768 if you would like to -support large message sizes for IHE (32k is the current maximum -needed for IHE). I was initially tempted to set the default to 32k, -but there is a some memory footprint with the current -implementation in rsyslog. -<br>If you intend to receive Windows Event Log data (e.g. via -<a href="http://www.eventreporter.com/">EventReporter</a>), you might want to -increase this number to an even higher value, as event -log messages can be very lengthy ("$MaxMessageSize 64k" is not a bad idea). -Note: testing showed that 4k seems to be -the typical maximum for <b>UDP</b> based syslog. This is an IP stack -restriction. Not always ... but very often. If you go beyond -that value, be sure to test that rsyslogd actually does what -you think it should do ;) It is highly suggested to use a TCP based transport -instead of UDP (plain TCP syslog, RELP). This resolves the UDP stack size restrictions. -<br>Note that 2k, the current default, is the smallest size that must be -supported in order to be compliant to the upcoming new syslog RFC series. -</li> -<li><a href="rsconf1_maxopenfiles.html">$MaxOpenFiles</a></li> -<li><a href="rsconf1_moddir.html">$ModDir</a></li> -<li><a href="rsconf1_modload.html">$ModLoad</a></li> -<li><a href="omfile.html"><b>$OMFileAsyncWriting</b></a> [on/<b>off</b>], if turned on, the files will be written -in asynchronous mode via a separate thread. In that case, double buffers will be used so -that one buffer can be filled while the other buffer is being written. Note that in order -to enable $OMFileFlushInterval, $OMFileAsyncWriting must be set to "on". Otherwise, the flush -interval will be ignored. Also note that when $OMFileFlushOnTXEnd is "on" but -$OMFileAsyncWriting is off, output will only be written when the buffer is full. This may take -several hours, or even require a rsyslog shutdown. However, a buffer flush can be forced -in that case by sending rsyslogd a HUP signal. -<li><a href="omfile.html"><b>$OMFileZipLevel</b></a> 0..9 [default 0] - if greater 0, turns on gzip compression -of the output file. The higher the number, the better the compression, but also the -more CPU is required for zipping.</li> -<li><a href="omfile.html"><b>$OMFileIOBufferSize</b></a> <size_nbr>, default 4k, size of the buffer used to writing output data. The larger the buffer, the potentially better performance is. The default of 4k is quite conservative, it is useful to go up to 64k, and 128K if you used gzip compression (then, even higher sizes may make sense)</li> -<li><a href="omfile.html"><b>$OMFileFlushOnTXEnd</b></a> <[<b>on</b>/off]>, default on. Omfile has the -capability to -write output using a buffered writer. Disk writes are only done when the buffer is -full. So if an error happens during that write, data is potentially lost. In cases where -this is unacceptable, set $OMFileFlushOnTXEnd to on. Then, data is written at the end -of each transaction (for pre-v5 this means after <b>each</b> log message) and the usual -error recovery thus can handle write errors without data loss. Note that this option -severely reduces the effect of zip compression and should be switched to off -for that use case. Note that the default -on- is primarily an aid to preserve -the traditional syslogd behaviour.</li> -<li><a href="omfile.html">$omfileForceChown</a> - force ownership change for all files</li> -<li><b>$RepeatedMsgContainsOriginalMsg</b> [on/<b>off</b>] - "last message repeated n times" messages, if generated, -have a different format that contains the message that is being repeated. -Note that only the first "n" characters are included, with n to be at least 80 characters, most -probably more (this may change from version to version, thus no specific limit is given). The bottom -line is that n is large enough to get a good idea which message was repeated but it is not necessarily -large enough for the whole message. (Introduced with 4.1.5). Once set, it affects all following actions.</li> -<li><a href="rsconf1_repeatedmsgreduction.html">$RepeatedMsgReduction</a></li> -<li><a href="rsconf1_resetconfigvariables.html">$ResetConfigVariables</a></li> -<li><b>$Ruleset</b> <i>name</i> - starts a new ruleset or switches back to one already defined. -All following actions belong to that new rule set. -the <i>name</i> does not yet exist, it is created. To switch back to rsyslog's -default ruleset, specify "RSYSLOG_DefaultRuleset") as the name. -All following actions belong to that new rule set. It is advised to also read -our paper on <a href="multi_ruleset.html">using multiple rule sets in rsyslog</a>.</li> -<li><b><a href="rsconf1_rulesetcreatemainqueue.html">$RulesetCreateMainQueue</a></b> on - creates -a ruleset-specific main queue. -<li><b><a href="rsconf1_rulesetparser.html">$RulesetParser</a></b> - enables to set -a specific (list of) message parsers to be used with the ruleset. -<li><b>$OptimizeForUniprocessor</b> [on/<b>off</b>] - turns on optimizatons which lead to better -performance on uniprocessors. If you run on multicore-machiens, turning this off lessens CPU load. The -default may change as uniprocessor systems become less common. [available since 4.1.0]</li> -<li>$PreserveFQDN [on/<b>off</b>) - if set to off (legacy default to remain compatible -to sysklogd), the domain part from a name that is within the same domain as the receiving -system is stripped. If set to on, full names are always used.</li> -<li>$WorkDirectory <name> (directory for spool and other work files. -Do <b>not</b> use trailing slashes)</li> -<li>$UDPServerAddress <IP> (imudp) -- local IP -address (or name) the UDP listens should bind to</li> -<li>$UDPServerRun <port> (imudp) -- former --r<port> option, default 514, start UDP server on this -port, "*" means all addresses</li> -<li>$UDPServerTimeRequery <nbr-of-times> (imudp) -- this is a performance -optimization. Getting the system time is very costly. With this setting, imudp can -be instructed to obtain the precise time only once every n-times. This logic is -only activated if messages come in at a very fast rate, so doing less frequent -time calls should usually be acceptable. The default value is two, because we have -seen that even without optimization the kernel often returns twice the identical time. -You can set this value as high as you like, but do so at your own risk. The higher -the value, the less precise the timestamp. -<li><a href="droppriv.html">$PrivDropToGroup</a></li> -<li><a href="droppriv.html">$PrivDropToGroupID</a></li> -<li><a href="droppriv.html">$PrivDropToUser</a></li> -<li><a href="droppriv.html">$PrivDropToUserID</a></li> -<li><b>$Sleep</b> <seconds> - puts the rsyslog main thread to sleep for the specified -number of seconds immediately when the directive is encountered. You should have a -good reason for using this directive!</li> -<li><b>$LocalHostIPIF</b> <interface name> - (available since 5.9.6) - if provided, the IP of the specified -interface (e.g. "eth0") shall be used as fromhost-ip for locall-originating messages. -If this directive is not given OR the interface cannot be found (or has no IP address), -the default of "127.0.0.1" is used. Note that this directive can be given only -once. Trying to reset will result in an error message and the new value will -be ignored. Please note that modules must have support for obtaining the local -IP address set via this directive. While this is the case for rsyslog-provided -modules, it may not always be the case for contributed plugins. -<br><b>Important:</b> This directive shall be placed <b>right at the top of -rsyslog.conf</b>. Otherwise, if error messages are triggered before this directive -is processed, rsyslog will fix the local host IP to "127.0.0.1", what than can -not be reset. -</li> -<li><a href="rsconf1_umask.html">$UMASK</a></li> -</ul> -<p><b>Where <size_nbr> or integers are specified above,</b> -modifiers can be used after the number part. For example, 1k means -1024. Supported are k(ilo), m(ega), g(iga), t(era), p(eta) and e(xa). -Lower case letters refer to the traditional binary defintion (e.g. 1m -equals 1,048,576) whereas upper case letters refer to their new -1000-based definition (e.g 1M equals 1,000,000).</p> -<p>Numbers may include '.' and ',' for readability. So you can -for example specify either "1000" or "1,000" with the same result. -Please note that rsyslogd simply ignores the punctuation. From it's -point of view, "1,,0.0.,.,0" also has the value 1000. </p> - -<p>[<a href="manual.html">manual index</a>] -[<a href="rsyslog_conf.html">rsyslog.conf</a>] -[<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> -<p><font size="2">This documentation is part of the -<a href="http://www.rsyslog.com/">rsyslog</a> project.<br> -Copyright © 2008-2010 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and -<a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL -version 3 or higher.</font></p> -</body> -</html> - - diff --git a/doc/rsyslog_conf_modules.html b/doc/rsyslog_conf_modules.html deleted file mode 100644 index 2668bf2..0000000 --- a/doc/rsyslog_conf_modules.html +++ /dev/null @@ -1,190 +0,0 @@ -<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> -<html><head><title>Modules - rsyslog.conf</title></head> -<body> -<p>This is a part of the rsyslog.conf documentation.</p> -<a href="rsyslog_conf.html">Back to rsyslog.conf manual</a> -<h1>Modules</h1> -<p>Rsyslog has a modular design. This enables functionality to be -dynamically loaded from modules, which may also be written by any -third party. Rsyslog itself offers all non-core functionality as -modules. Consequently, there is a growing -number of modules. Here is the entry point to their documentation and -what they do (list is currently not complete)</p> -<p>Please note that each module provides configuration -directives, which are NOT necessarily being listed below. Also -remember, that a modules configuration directive (and functionality) is -only available if it has been loaded (using $ModLoad).</p> -<p>It is relatively easy to write a rsyslog module. <b>If none of the provided -modules solve your need, you may consider writing one or have one written -for you by -<a href="http://www.rsyslog.com/professional-services">Adiscon's professional services for rsyslog</a> -</b>(this often is a very cost-effective and efficient way of getting what you need). -<p>There exist different classes of loadable modules: -<ul> -<li><a href="rsyslog_conf_modules.html#im">Input Modules</a> -<li><a href="rsyslog_conf_modules.html#om">Output Modules</a> -<li><a href="rsyslog_conf_modules.html#pm">Parser Modules</a> -<li><a href="rsyslog_conf_modules.html#mm">Message Modification Modules</a> -<li><a href="rsyslog_conf_modules.html#sm">String Generator Modules</a> -<li><a href="rsyslog_conf_modules.html#lm">Library Modules</a> -</ul> - -<a name"im"></a><h2>Input Modules</h2> -<p>Input modules are used to gather messages from various sources. They interface -to message generators. -<ul> -<li><a href="imfile.html">imfile</a> - input module for text files</li> -<li><a href="imrelp.html">imrelp</a> - RELP input module</li> -<li><a href="imudp.html">imudp</a> - udp syslog message input</li> -<li><a href="imtcp.html">imtcp</a> - input plugin for tcp syslog</li> -<li><a href="imptcp.html">imptcp</a> - input plugin for plain tcp syslog (no TLS but faster)</li> -<li><a href="imgssapi.html">imgssapi</a> - input plugin for plain tcp and GSS-enabled syslog</li> -<li>immark - support for mark messages</li> -<li><a href="imklog.html">imklog</a> - kernel logging</li> -<li><a href="imuxsock.html">imuxsock</a> - unix sockets, including the system log socket</li> -<li><a href="imsolaris.html">imsolaris</a> - input for the Sun Solaris system log source</li> -<li><a href="im3195.html">im3195</a> - accepts syslog messages via RFC 3195</li> -<li><a href="impstats.html">impstats</a> - provides periodic statistics of rsyslog internal counters</li> -</ul> - -<a name"om"></a><h2>Output Modules</h2> -<p>Output modules process messages. With them, message formats can be transformed -and messages be transmitted to various different targets. -<ul> -<li><a href="omfile.html">omfile</a> - file output module</li> -<li><a href="omfwd.html">omfwd</a> - syslog forwarding output module</li> -<li><a href="omjournal.html">omjournal</a> - Linux journal output module</li> -<li><a href="ompipe.html">ompipe</a> - named pipe output module</li> -<li><a href="omusrmsg.html">omusrmsg</a> - user message output module</li> -<li><a href="omsnmp.html">omsnmp</a> - SNMP trap output module</li> -<li><a href="omstdout.html">omtdout</a> - stdout output module (mainly a test tool)</li> -<li><a href="omrelp.html">omrelp</a> - RELP output module</li> -<li><a href="omruleset.html">omruleset</a> - forward message to another ruleset</li> -<li>omgssapi - output module for GSS-enabled syslog</li> -<li><a href="ommysql.html">ommysql</a> - output module for MySQL</li> -<li>ompgsql - output module for PostgreSQL</li> -<li><a href="omlibdbi.html">omlibdbi</a> - -generic database output module (Firebird/Interbase, MS SQL, Sybase, -SQLLite, Ingres, Oracle, mSQL)</li> -<li><a href="ommail.html">ommail</a> - -permits rsyslog to alert folks by mail if something important happens</li> -<li><a href="omprog.html">omprog</a> - permits sending messages to a program for custom processing</li> -<li><a href="omoracle.html">omoracle</a> - output module for Oracle (native OCI interface)</li> -<li><a href="omudpspoof.html">omudpspoof</a> - output module sending UDP syslog messages with a spoofed address</li> -<li><a href="omuxsock.html">omuxsock</a> - output module Unix domain sockets</li> -<li><a href="omhdfs.html">omhdfs</a> - output module for Hadoop's HDFS file system</li> -<li><a href="ommongodb.html">ommongodb</a> - output module for MongoDB</li> -<li><a href="omelasticsearch.html">omelasticsearch</a> - output module for ElasticSearch</li> -</ul> - -<a name="pm"></a><h2>Parser Modules</h2> -<p>Parser modules are used to parse message content, once the message has been -received. They can be used to process custom message formats or invalidly formatted -messages. For details, please see the <a href="messageparser.html">rsyslog -message parser documentation</a>. -<p>The current modules are currently provided as part of rsyslog: -<ul> -<li>pmrfc5424[builtin] - rsyslog.rfc5424 - -parses RFC5424-formatted messages (the new syslog standard) -<li>pmrfc3164[builtin] - rsyslog.rfc3164 - -the traditional/legacy syslog parser -<li>pmrfc3164sd - rsyslog.rfc3164sd - -a contributed module supporting RFC5424 structured data inside -RFC3164 messages (not supported by the rsyslog team) -<li><a href="pmlastmsg.html">pmlastmsg</a> - rsyslog.lastmsg - -a parser module that handles the typically malformed "last messages -repated n times" messages emitted by some syslogds. -</ul> - -<a name="mm"></a><h2>Message Modification Modules</h2> -<p>Message modification modules are used to change the content of messages being processed. -They can be implemented using either the output module or the parser module interface. -From the rsyslog core's point of view, they actually are output or parser modules, it is their -implementation that makes them special. -<p>Currently, there exists only a limited set of such modules, but new ones could be written with -the methods the engine provides. They could be used, for example, to -add dynamically computed content to message (fields). -<p>Message modification modules are usually written for one specific task and thus -usually are not generic enough to be reused. However, existing module's code is -probably an excellent starting base for writing a new module. Currently, the following -modules exist inside the source tree: -<ul> -<li><a href="mmanon.html">mmanon</a> - used to anonymize log messages. -<li><a href="mmnormalize.html">mmnormalize</a> - used to normalize log messages. -Note that this actually is a <b>generic</b> module. -<li><a href="mmjsonparse.html">mmjsonparse</a> - used to interpret CEE/lumberjack -enabled structured log messages. -<li><a href="mmsnmptrapd.html">mmsnmptrapd</a> - uses information provided by snmptrapd inside -the tag to correct the original sender system and priority of messages. Implemented via -the output module interface. -</ul> - -<a name="lm"></a><h2>String Generator Modules</h2> -<p>String generator modules are used, as the name implies, to generate strings based -on the message content. They are currently tightly coupled with the template system. -Their primary use is to speed up template processing by providing a native C -interface to template generation. These modules exist since 5.5.6. To get an idea -of the potential speedup, the default file format, when generated by a string generator, -provides a roughly 5% speedup. For more complex strings, especially those that include -multiple regular expressions, the speedup may be considerably higher. -<p>String generator modules are written to a quite simple interface. However, a word of -caution is due: they access the rsyslog message object via a low-level interface. -That interface is not guaranteed yet to stay stable. So it may be necessary to -modify string generator modules if the interface changes. Obviously, we will not do that -without good reason, but it may happen. -<p>Rsyslog comes with a set of core, build-in string generators, which are used -to provide those default templates that we consider to be time-critical: -<ul> -<li>smfile - the default rsyslog file format -<li>smfwd - the default rsyslog (network) forwarding format -<li>smtradfile - the traditional syslog file format -<li>smfwd - the traditional syslog (network) forwarding format -</ul> -<p>Note that when you replace these defaults be some custom strings, you will -loose some performance (around 5%). For typical systems, this is not really relevant. -But for a high-performance systems, it may be very relevant. To solve that issue, create -a new string generator module for your custom format, starting out from one of the -default generators provided. If you can not do this yourself, you may want to -contact <a href="mailto:info%40adiscon.com">Adiscon</a> as we offer custom development -of string generators at a very low price. -<p>Note that string generator modules can be dynamically loaded. However, the default -ones provided are so important that they are build right into the executable. But this -does not need to be done that way (and it is straightforward to do it dynamic). - - -<a name="lm"></a><h2>Library Modules</h2> -<p>Library modules provide dynamically loadable functionality for parts of rsyslog, -most often for other loadable modules. They can not be user-configured and are loaded -automatically by some components. They are just mentioned so that error messages that -point to library moduls can be understood. No module list is provided. - -<h2>Where are the modules integrated into the Message Flow?</h2> -<p>Depending on their module type, modules may access and/or modify messages at -various stages during rsyslog's processing. Note that only the "core type" (e.g. input, -output) but not any type derived from it (message modification module) specifies when -a module is called. -<p>The simplified workflow is as follows: -<p align="center"> -<img src="module_workflow.png" alt"rsyslog: loadable modules and message flow"> -<p>As can be seen, messages are received by input modules, then passed to one or many -parser modules, which generate the in-memory representation of the message and may -also modify the message itself. The, the internal representation is passed to -output modules, which may output a message and (with the interfaces newly introduced -in v5) may also modify messageo object content. -<p>String generator modules are not included inside this picture, because they are -not a required part of the workflow. If used, they operate "in front of" the -output modules, because they are called during template generation. -<p>Note that the actual flow is much more complex and depends a lot on queue and -filter settings. This graphic above is a high-level message flow diagram. - -<p>[<a href="manual.html">manual index</a>] -[<a href="rsyslog_conf.html">rsyslog.conf</a>] -[<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> -<p><font size="2">This documentation is part of the -<a href="http://www.rsyslog.com/">rsyslog</a> project.<br> -Copyright © 2008-2013 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and -<a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL -version 3 or higher.</font></p> -</body> -</html> - diff --git a/doc/rsyslog_conf_nomatch.html b/doc/rsyslog_conf_nomatch.html deleted file mode 100644 index 5f25f3e..0000000 --- a/doc/rsyslog_conf_nomatch.html +++ /dev/null @@ -1,48 +0,0 @@ -<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> -<html><head><title>nomatch mode - property replacer - rsyslog.conf</title></head> -<body> -<h1>nomatch mode - property replacer - rsyslog.con</h1> -<p>This is a part of the <a href="rsyslog_conf.html">rsyslog.conf documentation</a> -of the <a href="property_replacer.html">property replacer</a>.</p> -<p><b>The "nomatch-Mode" specifies which string the property replacer -shall return if a regular expression did not find the search string.</b>. Traditionally, -the string "**NO MATCH**" was returned, but many people complained this was almost never useful. -Still, this mode is support as "<b>DFLT</b>" for legacy configurations. -<p>Three additional and potentially useful modes exist: in one (<b>BLANK</b>) a blank string -is returned. This is probably useful for inserting values into databases where no -value shall be inserted if the expression could not be found. -<p>A similar mode is "<b>ZERO</b>" where the string "0" is returned. This is suitable -for numerical values. A use case may be -that you record a traffic log based on firewall rules and the "bytes transmitted" counter -is extracted via a regular expression. If no "bytes transmitted" counter is available -in the current message, it is probably a good idea to return an empty string, which the -database layer can turn into a zero. -<p>The other mode is "<b>FIELD</b>", in which the complete field is returned. This may be useful -in cases where absense of a match is considered a failure and the message that triggered -it shall be logged. -<p>If in doubt, <b>it is highly suggested to use the -<a href="http://www.rsyslog.com/tool-regex">rsyslog online regular expression -checker and generator</a> to see these options in action</b>. With that online tool, -you can craft regular expressions based on samples and try out the different modes. - -<h2>Summary of nomatch Modes</h2> -<table border="1" cellspacing="0"> -<tr><td><b>Mode</b></td><td><b>Returned</b></td></tr> -<tr><td>DFLT</td><td>"**NO MATCH**"</td></tr> -<tr><td>BLANK</td><td>"" (empty string)</td></tr> -<tr><td>ZERO</td><td>"0"</td></tr> -<tr><td>FIELD</td><td>full content of original field</td></tr> -<tr><td> </td><td><a href="http://www.rsyslog.com/tool-regex">Interactive Tool</a></td></tr> -</table> -<p>[<a href="manual.html">manual index</a>] -[<a href="rsyslog_conf.html">rsyslog.conf</a>] -[<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> -<p><font size="2">This documentation is part of the -<a href="http://www.rsyslog.com/">rsyslog</a> project.<br> -Copyright © 2008 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and -<a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL -version 2 or higher.</font></p> -</body> -</html> - - diff --git a/doc/rsyslog_conf_output.html b/doc/rsyslog_conf_output.html deleted file mode 100644 index 426f2f2..0000000 --- a/doc/rsyslog_conf_output.html +++ /dev/null @@ -1,81 +0,0 @@ -<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> -<html><head><title>Output Channels - rsyslog.conf</title></head> -<body> -<p>This is a part of the rsyslog.conf documentation.</p> -<a href="rsyslog_conf.html">back</a> -<h2>Output Channels</h2> -<p>Output Channels are a new concept first introduced in rsyslog -0.9.0. <b>As of this writing, it is most likely that they will -be replaced by something different in the future.</b> So if you -use them, be prepared to change you configuration file syntax when you -upgrade to a later release.<br> -<br> -The idea behind output channel definitions is that it shall provide an -umbrella for any type of output that the user might want. In essence,<br> -this is the "file" part of selector lines (and this is why we are not -sure output channel syntax will stay after the next review). There is a<br> -difference, though: selector channels both have filter conditions -(currently facility and severity) as well as the output destination. -they can only be used to write to files - not pipes, ttys or whatever -Output channels define the output definition, only. As of this build, -else. If we stick with output channels, this will change over time.</p> -<p>In concept, an output channel includes everything needed to -know about an output actions. In practice, the current implementation -only carries<br> -a filename, a maximum file size and a command to be issued when this -file size is reached. More things might be present in future version, -which might also change the syntax of the directive.</p> -<p>Output channels are defined via an $outchannel directive. It's -syntax is as follows:<br> -<br> -$outchannel name,file-name,max-size,action-on-max-size<br> -<br> -name is the name of the output channel (not the file), file-name is the -file name to be written to, max-size the maximum allowed size and -action-on-max-size a command to be issued when the max size is reached. -This command always has exactly one parameter. The binary is that part -of action-on-max-size before the first space, its parameter is -everything behind that space.<br> -<br> -Please note that max-size is queried BEFORE writing the log message to -the file. So be sure to set this limit reasonably low so that any -message might fit. For the current release, setting it 1k lower than -you expected is helpful. The max-size must always be specified in bytes -- there are no special symbols (like 1k, 1m,...) at this point of -development.<br> -<br> -Keep in mind that $outchannel just defines a channel with "name". It -does not activate it. To do so, you must use a selector line (see -below). That selector line includes the channel name plus an $ sign in -front of it. A sample might be:<br> -<br> -*.* :omfile:$mychannel<br> -<br> -In its current form, output channels primarily provide the ability to -size-limit an output file. To do so, specify a maximum size. When this -size is reached, rsyslogd will execute the action-on-max-size command -and then reopen the file and retry. The command should be something -like a <a href="log_rotation_fix_size.html">log rotation -script</a> or a similar thing.</p> -<p>If there is no action-on-max-size command or the command did -not resolve the situation, the file is closed and never reopened by -rsyslogd (except, of course, by huping it). This logic was integrated -when we first experienced severe issues with files larger 2gb, which -could lead to rsyslogd dumping core. In such cases, it is more -appropriate to stop writing to a single file. Meanwhile, rsyslogd has -been fixed to support files larger 2gb, but obviously only on file -systems and operating system versions that do so. So it can still make -sense to enforce a 2gb file size limit.</p> - -<p>[<a href="manual.html">manual index</a>] -[<a href="rsyslog_conf.html">rsyslog.conf</a>] -[<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> -<p><font size="2">This documentation is part of the -<a href="http://www.rsyslog.com/">rsyslog</a> project.<br> -Copyright © 2008 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and -<a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL -version 2 or higher.</font></p> -</body> -</html> - - diff --git a/doc/rsyslog_conf_sysklogd_compatibility.html b/doc/rsyslog_conf_sysklogd_compatibility.html deleted file mode 100644 index c95d6fd..0000000 --- a/doc/rsyslog_conf_sysklogd_compatibility.html +++ /dev/null @@ -1,31 +0,0 @@ -<html><head><title>sysklogdcompatibility - rsyslog.conf</title></head> -<body> -<h1>sysklogd compatibility</h1> -<p>This is a part of the rsyslog.conf documentation.</p> -<a href="rsyslog_conf.html">Back to rsyslog.conf manual</a> -<p>Rsyslog supports standard sysklogd's configuration file format -and extends it. So in general, you can take a "normal" syslog.conf and -use it together with rsyslogd. It will understand everything. However, -to use most of rsyslogd's unique features, you need to add extended -configuration directives.</p> -<p>Rsyslogd supports the classical, selector-based rule lines. -They are still at the heart of it and all actions are initiated via -rule lines. -However, there are ample new directives, either in rsyslog traditional -format (starting with a dollar sign) or in RainerScript format. These -work together with sysklogd statements. A few select statements are -no longer supported and may generate error messages. They are mentioned -in the compatibility notes. -</p> - -<p>[<a href="manual.html">manual index</a>] -[<a href="rsyslog_conf.html">rsyslog.conf</a>] -[<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> -<p><font size="2">This documentation is part of the -<a href="http://www.rsyslog.com/">rsyslog</a> project.<br> -Copyright © 2008-2013 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and -<a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL -version 3 or higher.</font></p> -</body> -</html> - diff --git a/doc/rsyslog_conf_templates.html b/doc/rsyslog_conf_templates.html deleted file mode 100644 index 9a6e161..0000000 --- a/doc/rsyslog_conf_templates.html +++ /dev/null @@ -1,538 +0,0 @@ -<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> -<html><head><title>Templates - rsyslog.conf</title></head> -<body> -<p>This is a part of the rsyslog.conf - documentation.</p> -<a href="rsyslog_conf.html">back</a> -<h1>Templates</h1> -<p>Templates are a key feature of rsyslog. They allow to specify -any -format a user might want. They are also used for dynamic file name -generation. Every output in rsyslog uses templates - this holds true -for files, user messages and so on. The database writer expects its -template to be a proper SQL statement - so this is highly customizable -too. You might ask how does all of this work when no templates at all -are specified. Good question ;) The answer is simple, though. Templates -compatible with the stock syslogd formats are hardcoded into rsyslogd. -So if no template is specified, we use one of these hardcoded -templates. Search for "template_" in syslogd.c and you will find the -hardcoded ones.</p> -<p>Templates are specified by template() statements. They can also be specified -via $Template legacy statements. Note that these are scheduled for removal in -later versions of rsyslog, so it is probably a good idea to avoid them -for new uses. -<h2>The template() statement</h2> -<p>The template() statement is used to define templates. Note that it is a -<b>static</b> statement, that means all templates are defined when rsyslog -reads the config file. As such, templates are not affected by if-statements -or config nesting. -<p>The basic structure of the template statement is as follows: -<br><br> -<code>template(parameters)</code> -<br><br> -In addition to this simpler syntax, list templates (to be described below) -support an extended syntax: -<br><br> -<code>template(parameters) { list-descriptions }</code> -<p>Each template has a parameter <b>name</b>, which specifies the templates -name, and a parameter <b>type</b>, which specifies the template type. The name -parameter must be unique, and behaviour is unpredictable if it is not. The <b>type</b> -parameter specifies different template types. Different types simply enable -different ways to specify the template content. The template type <b>does not</b> -affect what an (output) plugin can do with it. So use the type that best fits your -needs (from a config writing point of view!). The following types are available: -<ul> -<li>list -<li>subtree -<li>string -<li>plugin -</ul> -The various types are described below. - -<h3>list</h3> -<p>In this case, the template is generated by a list of constant and -variable statements. These follow the template spec in curly braces. This type is -also primarily meant for use with structure-aware outputs, like ommongodb. However, -it also works perfectly with text-based outputs. We recommend to use this mode -if more complex property substitutions needs to be done. In that case, the list-based -template syntax is much clearer than the simple string-based one. -<p>The list template contains the template header (with <b>type="list"</b>) and is followed -by <b>constant</b> and <b>property</b> statements, given in curly braces to signify -the template statement they belong to. As the name says, <b>constant</b> statements -describe constant text and <b>property</b> describes property access. There are many options -to <b>property</b>, described further below. Most of these options are used to extract -only partial property contents or to modify the text obtained (like to change its case -to upper or lower case, only). -<p>To grasp the idea, an actual sample is: -<br><pre><code>template(name="tpl1" type="list") { - constant(value="Syslog MSG is: '") - property(name="msg") - constant(value="', ") - property(name="timereported" dateFormat="rfc3339" caseConversion="lower") - constant(value="\n") - } -</code></pre> -<br>This sample is probably primarily targeted at the usual file-based output.</p> - - -<h4>constant statement</h4> -<p>This provides a way to specify constant text. The text is used literally. It is -primarily intended for text-based output, so that some constant text can be included. For -example, if a complex template is build for file output, one usually needs to finish it -by a newline, which can be introduced by a constant statement. Here is an actual sample -of that use case from the rsylsog testbench: -<br><pre><code>template(name="outfmt" type="list") { - property(name="$!usr!msgnum") - constant(value="\n") -}</code></pre> -The following escape sequences are recogniced inside the constant text: -<ul> -<li>\\ - single backslash -<li>\n - LF -<li>\ooo - (three octal digits) - represents character with this numerical value (e.g. \101 -equals "A"). Note that three -octal digits must be given (in contrast to some languagues, where between one and three are valid). -While we support octal notation, we recommend to use hex notation as this is better known. -<li>\xhh - (where h is a hex digit) - represents character with this numerical value (e.g. \x41 -equals "A"). Note that two hexadecimal digits must be given (in contrast to some languagues -where one or two are valid). -<li>... some others ... list needs to be extended -</ul> -<p>Note: if an unsupported character follows a backslash, this is treated as an error. Behaviour -is unpredictable in this case. -<p>To aid usage of the same template both for text-based outputs and structured ones, constant -text without an "outname" parameter will be ignored when creating the name/value tree -for structured outputs. So if you want to supply some constant text e.g. to mongodb, you must -include an outname, as can be seen here: -<br><pre><code>template(name="outfmt" type="list") { - property(name="$!usr!msgnum") - constant(value="\n" <b>outname="IWantThisInMyDB"</b>) -}</code></pre> - -The "constant" statement supports the following parameters: -<ul> -<li>value - the constant value to use -<li>outname - output field name (for structured outputs) -</ul> - - -<h4>property statement</h4> -<p>This statement is used to include property text. It can access all properties. Also, -options permit to specify picking only part of a property or modifying it. -It supports the following parameters: -<ul> -<li>name - the name of the property to access -<li>outname - output field name (for structured outputs) -<li>dateformat - date format to use (only for date-related properties) -<li>caseconversion - permits to convert case of the text. supported values are -"lower" and "upper" -<li>controlcharacters - specifies how to handle control characters. Supported values are -"escape", which escapes them, "space", which replaces them by a single space, and -"drop", which simply removes them from the string. -<li>securepath - used for creating pathnames suitable for use in dynafile templates -<li>format - specifiy format on a field basis. Supported values are "csv", for use when -csv-data is generated, "json", which formats proper json content (but without a field -header) and "jsonf", which formats as a complete json field. -<li>position.from - obtain substring starting from this position (1 is the first position) -<li>position.to - obtain substring up to this position -<li>position.relativeToEnd - the from and to position is relative to the end of the string - instead of the usual start of string. (available since rsyslog v7.3.10) -<li>field.number - obtain this field match -<li>field.delimiter - decimal value of delimiter character for field extraction -<li>regex.expression - expression to use -<li>regex.type - either ERE or BRE -<li>regex.nomatchmode - what to do if we have no match -<li>regex.match - match to use -<li>regex.submatch - submatch to use -<li>droplastlf - drop a trailing LF, if it is present -<li>mandatory - signifies a field as mandatory. If set to "on", this field will always -be present in data passed to structured outputs, even if it is empty. If "off" (the default) -empty fields will not be passed to structured outputs. This is especially useful for outputs -that support dynamic schemas (like ommongodb). -<li>spifno1stsp - expert options for RFC3164 template processing -</ul> - - -<h3>subtree</h3> -<p>Available since rsyslog 7.1.4 -<p> -In this case, the template is generated based on a complete -(CEE) subtree. This type of template is most useful for outputs that know how to -process hierarchical structure, like ommongodb. With that type, the parameter -<b>subtree</b> must be specified, which tells which subtree to use. For example -template(name="tpl1" type="subtree" subtree="$!") includes all CEE data, while -template(name="tpl2" type="subtree" subtree="$!usr!tpl2") includes only the -subtree starting at $!usr!tpl2. The core idea when using this type of template -is that the actual data is prefabricated via set and unset script statements, -and the resulting strucuture is then used inside the template. This method MUST -be used if a complete subtree needs to be placed <i>directly</i> into the -object's root. With all other template types, only subcontainers can be generated. -Note that subtree type can also be used with text-based outputs, like omfile. HOWEVER, -you do not have any capability to specify constant text, and as such cannot include -line breaks. As a consequence, using this template type for text outputs is usually -only useful for debugging or very special cases (e.g. where the text is interpreted -by a JSON parser later on). -<h4>Use case</h4> -<p>A typical use case is to first create a custom subtree and then include it into -the template, like in this small example: -<br><blockquote><code>set $!usr!tpl2!msg = $msg; -<br>set $!usr!tpl2!dataflow = field($msg, 58, 2); -<br>template(name="tpl2" type="subtree" subtree="$!usr!tpl2") -</code></blockquote> -<p>Here, we assume that $msg contains various fields, and the data from a field -is to be extracted and stored - together with the message - as field content. -<h3>string</h3> -<p>This closely resembles the legacy template statement. It -has a mandatory parameter <b>string</b>, which holds the template string to be -applied. A template string is a mix of constant text and replacement variables -(see property replacer). These variables are taken from message or other dynamic -content when the final string to be passed to a plugin is generated. String-based -templates are a great way to specify textual content, especially if no complex -manipulation to properties is necessary. Full details on how to specify template -text can be found below. -<br>Config example: -<br><blockquote><code>template(name="tpl3" type="string" string="%TIMESTAMP:::date-rfc3339% %HOSTNAME% %syslogtag%%msg:::sp-if-no-1st-sp%%msg:::drop-last-lf%\n") -</code></blockquote> -<h3>plugin</h3> -In this case, the template is generated by a plugin (which -is then called -a "strgen" or "string generator"). The format is fix as it is coded. While this -is inflexible, it provides superior performance, and is often used for that -reason (not that "regular" templates are slow - but in very demanding environments -that "last bit" can make a difference). Refer to the plugin's documentation -for further details. For this type, the paramter <b>plugin</b> must be specified and -must contain the name of the plugin as it identifies itself. Note that the -plugin must be loaded prior to being used inside a template. -<br>Config example: -<br><blockquote><code>template(name="tpl4" type="plugin" plugin="mystrgen") -</code></blockquote> - -<h3>options</h3> -The <options> part is optional. It carries options -influencing the template as whole and is part of the template parameters. -See details below. Be sure NOT to mistake template options with property -options - the latter ones are processed by the property replacer and -apply to a SINGLE property, only (and not the whole template).<br> -<br> -Template options are case-insensitive. Currently defined are: </p> -<p><b>option.sql</b> - format the string suitable for a SQL -statement in MySQL format. This will replace single quotes ("'") and -the backslash character by their backslash-escaped counterpart ("\'" -and "\\") inside each field. Please note that in MySQL configuration, -the <code class="literal">NO_BACKSLASH_ESCAPES</code> -mode must be turned off for this format to work (this is the default).</p> -<p><b>option.stdsql</b> - format the string suitable for a -SQL statement that is to be sent to a standards-compliant sql server. -This will replace single quotes ("'") by two single quotes ("''") -inside each field. You must use stdsql together with MySQL if in MySQL -configuration the -<code class="literal">NO_BACKSLASH_ESCAPES</code> is -turned on.</p> -<p><b>option.json</b> - format the string suitable for a -json statement. -This will replace single quotes ("'") by two single quotes ("''") -inside each field.</p> -<p>At no time, multiple template option should be used. This can cause -unpredictable behaviour and is against all logic.</p> -<p>Either the <b>sql</b> or <b>stdsql</b> -option <b>must</b> be specified when a template is used -for writing to a database, otherwise injection might occur. Please note -that due to the unfortunate fact that several vendors have violated the -sql standard and introduced their own escape methods, it is impossible -to have a single option doing all the work. So you yourself -must make sure you are using the right format. <b>If you choose -the wrong one, you are still vulnerable to sql injection.</b><br> -<br> -Please note that the database writer *checks* that the sql option is -present in the template. If it is not present, the write database -action is disabled. This is to guard you against accidental forgetting -it and then becoming vulnerable to SQL injection. The sql option can -also be useful with files - especially if you want to import them into -a database on another machine for performance reasons. However, do NOT -use it if you do not have a real need for it - among others, it takes -some toll on the processing time. Not much, but on a really busy system -you might notice it ;)</p> -<p>The default template for the write to database action has the -sql option set. As we currently support only MySQL and the sql option -matches the default MySQL configuration, this is a good choice. -However, if you have turned on -<code class="literal">NO_BACKSLASH_ESCAPES</code> in -your MySQL config, you need to supply a template with the stdsql -option. Otherwise you will become vulnerable to SQL injection. <br> -<br> -To escape:<br> -% = \%<br> -\ = \\ --> '\' is used to escape (as in C)<br> -template (name="TraditionalFormat" type="string" string="%timegenerated% %HOSTNAME% %syslogtag%%msg%\n"<br> -<br> - -<h3>Examples</h3> -<h4>Standard Template for Writing to Files</h4> -<p><pre><code>template(name="FileFormat" type="list") { - property(name="timestamp" dateFormat="rfc3339") - constant(value=" ") - property(name="hostname") - constant(value=" ") - property(name="syslogtag") - constant(value=" ") - property(name="msg" spifno1stsp="on" ) - property(name="msg" droplastlf="on" ) - constant(value="\n") - } -</code></pre> -<p>The equivalent string template looks like this: -<br><pre><code>template(name="FileFormat" type="string" - string= "%TIMESTAMP% %HOSTNAME% %syslogtag%%msg:::sp-if-no-1st-sp%%msg:::drop-last-lf%\n" -)</code></pre> -Note that the template string itself must be on a single line. - -<h4>Standard Template for Forwarding to a Remote Host (RFC3164 mode)</h4> -<p><pre><code>template(name="ForwardFormat" type="list") { - constant(value="<") - property(name="PRI") - constant(value="<") - property(name="timestamp" dateFormat="rfc3339") - constant(value=" ") - property(name="hostname") - constant(value=" ") - property(name="syslogtag" position.from="1" position.to="32") - constant(value=" ") - property(name="msg" spifno1stsp="on" ) - } -</code></pre> -<p>The equivalent string template looks like this: -<br><pre><code>template(name="forwardFormat" type="string" - string="<%PRI%>%TIMESTAMP:::date-rfc3339% %HOSTNAME% %syslogtag:1:32%%msg:::sp-if-no-1st-sp%%msg%" -)</code></pre> -Note that the template string itself must be on a single line. - -<h4>Standard Template for write to the MySQL database</h4> -<p><pre><code>template(name="StdSQLformat" type="list" option.sql="on") { - constant(value="insert into SystemEvents (Message, Facility, FromHost, Priority, DeviceReportedTime, ReceivedAt, InfoUnitID, SysLogTag)") - constant(value=" values ('") - property(name="msg") - constant(value="', ") - property(name="syslogfacility") - constant(value=", '") - property(name="hostname") - constant(value="', ") - property(name="syslogpriority") - constant(value=", '") - property(name="timereported" dateFormat="mysql") - constant(value="', '") - property(name="timegenerated" dateFormat="mysql") - constant(value="', ") - property(name="iut") - constant(value=", '") - property(name="syslogtag") - constant(value="')") - } -</code></pre> -<p>The equivalent string template looks like this: -<br><pre><code>template(name="stdSQLformat" type="string" option.sql="on" - string="insert into SystemEvents (Message, Facility, FromHost, Priority, DeviceReportedTime, ReceivedAt, InfoUnitID, SysLogTag) values ('%msg%', %syslogfacility%, '%HOSTNAME%', %syslogpriority%, '%timereported:::date-mysql%', '%timegenerated:::date-mysql%', %iut%, '%syslogtag%')" -)</code></pre> -Note that the template string itself must be on a single line. - -<h2>legacy format</h2> -<p>In pre v6-versions of rsyslog, you need to use the <code>$template</code> -statement to configure templates. They provide the equivalent to string- and -plugin-based templates. The legacy syntax continous to work in v7, however -we recommend to avoid legacy format for newly written config files. Legacy and -current config statements can coexist within the same config file. -<p>The general format is -<br><br><code>$template name,param[,options]</code></br></br> -where "name" is the template name and "param" is a single parameter -that specifies template content. The optional "options" part is used to -set template options. -<h3>string</h3> -The parameter is the same string that with the current-style format you -specify in the <b>string</b> parameter, for example: -<br><br><code>$template strtpl,"PRI: %pri%, MSG: %msg%\n"</code> -<p>Note that list templates are not available in legacy format, so you need -to use complex property replacer constructs to do complex things. - -<h3>plugin</h3> -This is equivalent to the "plugin"-type template directive. Here, the -parameter is the plugin name, with an equal sign prepended. An example -is: -<br><br><code>$template plugintpl,=myplugin</code> - -<h2>Reserved Template Names</h2> -<p>Template -names beginning with "RSYSLOG_" are reserved for rsyslog use. Do NOT -use them if, otherwise you may receive a conflict in the future (and -quite unpredictable behaviour). There is a small set of pre-defined -templates that you can use without the need to define it:</p> -<ul> -<li><span style="font-weight: bold;">RSYSLOG_TraditionalFileFormat</span> -- the "old style" default log file format with low-precision timestamps</li> -<li><span style="font-weight: bold;">RSYSLOG_FileFormat</span> -- a modern-style logfile format similar to TraditionalFileFormat, buth -with high-precision timestamps and timezone information</li> -<li><span style="font-weight: bold;">RSYSLOG_TraditionalForwardFormat</span> -- the traditional forwarding format with low-precision timestamps. Most -useful if you send messages to other syslogd's or rsyslogd -below -version 3.12.5.</li> -<li><span style="font-weight: bold;">RSYSLOG_SysklogdFileFormat</span> -- sysklogd compatible log file format. If used with options: $SpaceLFOnReceive on; -$EscapeControlCharactersOnReceive off; $DropTrailingLFOnReception off, -the log format will conform to sysklogd log format.</li> -<li><span style="font-weight: bold;">RSYSLOG_ForwardFormat</span> -- a new high-precision forwarding format very similar to the -traditional one, but with high-precision timestamps and timezone -information. Recommended to be used when sending messages to rsyslog -3.12.5 or above.</li> -<li><span style="font-weight: bold;">RSYSLOG_SyslogProtocol23Format</span> -- the format specified in IETF's internet-draft -ietf-syslog-protocol-23, which is assumed to be come the new syslog -standard RFC. This format includes several improvements. The rsyslog -message parser understands this format, so you can use it together with -all relatively recent versions of rsyslog. Other syslogd's may get -hopelessly confused if receiving that format, so check before you use -it. Note that the format is unlikely to change when the final RFC comes -out, but this may happen.</li> -<li><span style="font-weight: bold;">RSYSLOG_DebugFormat</span> -- a special format used for troubleshooting property problems. This format -is meant to be written to a log file. Do <b>not</b> use for production or remote -forwarding.</li> -</ul> - -<h2>The following is legacy documentation soon to be integrated.</h2> - -<!--<table> -<tr><td>param name</td><td>meaning</td></tr> -<tr><td>name</td><td>name of the template</td></tr> -</table> ---> - -<p>Starting with 5.5.6, there are actually two different types of template: -<ul> -<li>string based -<li>string-generator module based -</ul> -<p><a href="rsyslog_conf_modules.html#sm">String-generator module</a> based templates -have been introduced in 5.5.6. They permit a string generator, actually a C "program", -the generate a format. Obviously, it is more work required to code such a generator, -but the reward is speed improvement. If you do not need the ultimate throughput, you -can forget about string generators (so most people never need to know what they are). -You may just be interested in learning that for the most important default formats, -rsyslog already contains highly optimized string generators and these are called -without any need to configure anything. But if you have written (or purchased) a -string generator module, you need to know how to call it. Each such module has a name, -which you need to know (look it up in the module doc or ask the developer). Let's assume -that "mystrgen" is the module name. Then you can define a template for that strgen -in the following way: - -<blockquote><code>template(name="MyTemplateName" type="plugin" string="mystrgen")</code></blockquote> -<p>Legacy example:</p> -<blockquote><code>$template MyTemplateName,=mystrgen</code></blockquote> -(Of course, you must have first loaded the module via $ModLoad). -<p>The important part is the equal sign in the legacy format: it tells the rsyslog config parser that -no string follows but a strgen module name. -<p>There are no additional parameters but the module name supported. This is because -there is no way to customize anything inside such a "template" other than by -modifying the code of the string generator. - -<p>So for most use cases, string-generator module based templates are <b>not</b> -the route to take. Usually, we use <b>string based templates</b> instead. -This is what the rest of the documentation now talks about. - -<p>A template consists of a template directive, a name, the -actual template text and optional options. A sample is:</p> -<blockquote><code>template(name="MyTemplateName" type="string" string="Example: Text %property% some more text\n" options)</code></blockquote> -<p>Legacy example:</p> -<blockquote><code>$template MyTemplateName,"\7Text -%property% some more text\n",<options></code></blockquote> -<p>The "template" (legacy: $template) is the template directive. It tells rsyslog -that this line contains a template. "MyTemplateName" is the template -name. All -other config lines refer to this name. The text within "string" is the -actual template text. The backslash is an escape character, much as it -is in C. It does all these "cool" things. For example, \7 rings the -bell (this is an ASCII value), \n is a new line. C programmers and perl -coders have the advantage of knowing this, but the set in rsyslog is a -bit restricted currently. -</p> -<p>All text in the template is used literally, except for things -within percent signs. These are properties and allow you access to the -contents of the syslog message. Properties are accessed via the -<a href="property_replacer.html">property replacer</a> -(nice name, huh) and it can do cool things, too. For -example, it can pick a substring or do date-specific formatting. More -on this is below, on some lines of the property replacer.<br> -<br> - -<br> -Properties can be accessed by the <a href="property_replacer.html">property -replacer</a> (see there for details).</p> -<p>Templates can be used in the form of a <b>list</b> as well. This has been -introduced with <b>6.5.0</b> The list consists of two parts which are either -a <b>constant</b> or a <b>property</b>. The constants -are taking the part of "text" that you usually enter in string-based templates. -The properties stay variable, as they are a substitute for different values of a -certain type. This type of template is extremely useful for complicated cases, -as it helps you to easily keep an overview over the template. Though, it has -the disadvantage of needing more effort to create it.</p> -<br>Config example: -<br><blockquote><code>template(name="MyTemplate" type="list" option.json="off") { - <br>constant(value="Test: ") - <br>property(name="msg" outname="mymessage") - <br>constant(value=" --!!!-- ") - <br>property(name="timereported" dateFormat="rfc3339" caseConversion="lower") - <br>constant(value="\n") - <br>} -</code></blockquote> -<p>First, the general template option will be defined. The values of the template -itself get defined in the curly brackets. As it can be seen, we have constants -and properties in exchange. Whereas constants will be filled with a value and probably -some options, properties do direct to a property and the options that could be needed -additional format definitions.</p> -<p>We suggest to use separate lines for all constants and properties. This -helps to keep a good overview over the different parts of the template. -Though, writing it in a single line will work, it is much harder to debug -if anything goes wrong with the template. </p> - -<p><b>Please note that templates can also be -used to generate selector lines with dynamic file names.</b> For -example, if you would like to split syslog messages from different -hosts to different files (one per host), you can define the following -template:</p> -<blockquote><code>template (name="DynFile" type="string" string="/var/log/system-%HOSTNAME%.log")</code></blockquote> -<p>Legacy example:</p> -<blockquote><code>$template -DynFile,"/var/log/system-%HOSTNAME%.log"</code></blockquote> -<p>This template can then be used when defining an output -selector line. It will result in something like -"/var/log/system-localhost.log"</p> -<h3>Legacy String-based Template Samples</h3> -<p>This section provides some default templates in legacy format, as used in rsyslog -previous to version 6. Note that this format is still supported, so there is no hard need -to upgrade existing configurations. However, it is strongly recommended that the legacy -constructs are not used when crafting new templates. -Note that each $Template statement is on a <b>single</b> line, but probably broken -accross several lines for display purposes by your browsers. Lines are separated by -empty lines. Keep in mind, that line breaks are important in legacy format. -<p><code> -$template FileFormat,"%TIMESTAMP:::date-rfc3339% %HOSTNAME% %syslogtag%%msg:::sp-if-no-1st-sp%%msg:::drop-last-lf%\n" -<br><br> -$template TraditionalFileFormat,"%TIMESTAMP% %HOSTNAME% %syslogtag%%msg:::sp-if-no-1st-sp%%msg:::drop-last-lf%\n" -<br><br> -$template ForwardFormat,"<%PRI%>%TIMESTAMP:::date-rfc3339% %HOSTNAME% %syslogtag:1:32%%msg:::sp-if-no-1st-sp%%msg%" -<br><br> -$template TraditionalForwardFormat,"<%PRI%>%TIMESTAMP% %HOSTNAME% %syslogtag:1:32%%msg:::sp-if-no-1st-sp%%msg%" -<br><br> -$template StdSQLFormat,"insert into SystemEvents (Message, Facility, FromHost, Priority, DeviceReportedTime, ReceivedAt, InfoUnitID, SysLogTag) values ('%msg%', %syslogfacility%, '%HOSTNAME%', %syslogpriority%, '%timereported:::date-mysql%', '%timegenerated:::date-mysql%', %iut%, '%syslogtag%')",SQL -</code></p> - -<p>[<a href="manual.html">manual index</a>] -[<a href="rsyslog_conf.html">rsyslog.conf</a>] -[<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> -<p><font size="2">This documentation is part of the -<a href="http://www.rsyslog.com/">rsyslog</a> project.<br> -Copyright © 2008-2012 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and -<a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL -version 2 or higher.</font></p> -</body> -</html> - diff --git a/doc/rsyslog_confgraph_complex.png b/doc/rsyslog_confgraph_complex.png Binary files differdeleted file mode 100644 index 21c04c5..0000000 --- a/doc/rsyslog_confgraph_complex.png +++ /dev/null diff --git a/doc/rsyslog_confgraph_std.png b/doc/rsyslog_confgraph_std.png Binary files differdeleted file mode 100644 index 655a7f8..0000000 --- a/doc/rsyslog_confgraph_std.png +++ /dev/null diff --git a/doc/rsyslog_high_database_rate.html b/doc/rsyslog_high_database_rate.html deleted file mode 100644 index 2bae58c..0000000 --- a/doc/rsyslog_high_database_rate.html +++ /dev/null @@ -1,186 +0,0 @@ -<html><head> - -<title>Handling a massive syslog database insert rate with Rsyslog</title> - -<meta name="KEYWORDS" content="syslog, rsyslog, reliable, howto, database, postgresql, mysql, buffering, disk, queue"> - -</head> - -<body> -<a href="features.html">back</a> - -<h1>Handling a massive syslog database insert rate with Rsyslog</h1> - - <P><small><i>Written by - - <a href="http://www.gerhards.net/rainer">Rainer - - Gerhards</a> (2008-01-31)</i></small></P> - -<h2>Abstract</h2> - -<p><i><b>In this paper, I describe how log massive amounts of -<a href="http://www.monitorware.com/en/topics/syslog/">syslog</a> - -messages to a database. </b>This HOWTO is currently under development and thus a -bit brief. Updates are promised ;).</i></p> - -<h2>The Intention</h2> - -<p>Database updates are inherently slow when it comes to storing syslog -messages. However, there are a number of applications where it is handy to have -the message inside a database. Rsyslog supports native database writing via -output plugins. As of this writing, there are plugins available for MySQL an -PostgreSQL. Maybe additional plugins have become available by the time you read -this. Be sure to check.</p> -<p>In order to successfully write messages to a database backend, the backend -must be capable to record messages at the expected average arrival rate. This is -the rate if you take all messages that can arrive within a day and divide it by -86400 (the number of seconds per day). Let's say you expect 43,200,000 messages -per day. That's an average rate of 500 messages per second (mps). Your database -server MUST be able to handle that amount of message per second on a sustained -rate. If it doesn't, you either need to add an additional server, lower the -number of message - or forget about it.</p> -<p>However, this is probably not your peak rate. Let's simply assume your -systems work only half a day, that's 12 hours (and, yes, I know this is -unrealistic, but you'll get the point soon). So your average rate is actually -1,000 mps during work hours and 0 mps during non-work hours. To make matters -worse, workload is not divided evenly during the day. So you may have peaks of -up to 10,000mps while at other times the load may go down to maybe just 100mps. -Peaks may stay well above 2,000mps for a few minutes.</p> -<p>So how the hack you will be able to handle all of this traffic (including the -peaks) with a database server that is just capable of inserting a maximum of -500mps?</p> -<p>The key here is buffering. Messages that the database server is not capable -to handle will be buffered until it is. Of course, that means database insert -are NOT real-time. If you need real-time inserts, you need to make sure your -database server can handle traffic at the actual peak rate. But lets assume you -are OK with some delay.</p> -<p>Buffering is fine. But how about these massive amounts of data? That can't be -hold in memory, so don't we run out of luck with buffering? The key here is that -rsyslog can not only buffer in memory but also buffer to disk (this may remind -you of "spooling" which gets you the right idea). There are several queuing -modes available, offering differnent throughput. In general, the idea is to -buffer in memory until the memory buffer is exhausted and switch to -disk-buffering when needed (and only as long as needed). All of this is handled -automatically and transparently by rsyslog.</p> -<p>With our above scenario, the disk buffer would build up during the day and -rsyslog would use the night to drain it. Obviously, this is an extreme example, -but it shows what can be done. Please note that queue content survies rsyslogd -restarts, so even a reboot of the system will not cause any message loss.</p> -<h2>How To Setup</h2> -<p>Frankly, it's quite easy. You just need to do is instruct rsyslog to use a -disk queue and then configure your action. There is nothing else to do. With the -following simple config file, you log anything you receive to a MySQL database -and have buffering applied automatically.</p> -<textarea rows="11" cols="80"> -$ModLoad ommysql # load the output driver (use ompgsql for PostgreSQL) -$ModLoad imudp # network reception -$UDPServerRun 514 # start a udp server at port 514 -$ModLoad imuxsock # local message reception - -$WorkDirectory /rsyslog/work # default location for work (spool) files -$MainMsgQueueFileName mainq # set file name, also enables disk mode - -$ActionResumeRetryCount -1 # infinite retries on insert failure -# for PostgreSQL replace :ommysql: by :ompgsql: below: -*.* :ommysql:hostname,dbname,userid,password; -</textarea> -<p>The simple setup above has one drawback: the write database action is -executed together with all other actions. Typically, local files are also -written. These local file writes are now bound to the speed of the database -action. So if the database is down, or threre is a large backlog, local files -are also not (or late) written.</p> -<p><b>There is an easy way to avoid this with rsyslog.</b> It involves a -slightly more complicated setup. In rsyslog, each action can utilize its own -queue. If so, messages are simply pulled over from the main queue and then the -action queue handles action processing on its own. This way, main processing and -the action are de-coupled. In the above example, this means that local file -writes will happen immediately while the database writes are queued. As a -side-note, each action can have its own queue, so if you would like to more than -a single database or send messages reliably to another host, you can do all of -this on their own queues, de-coupling their processing speeds.</p> -<p>The configuration for the de-coupled database write involves just a few more -commands:</p> -<textarea rows="11" cols="80"> -$ModLoad ommysql # load the output driver (use ompgsql for PostgreSQL) -$ModLoad imudp # network reception -$UDPServerRun 514 # start a udp server at port 514 -$ModLoad imuxsock # local message reception - -$WorkDirectory /rsyslog/work # default location for work (spool) files - -$ActionQueueType LinkedList # use asynchronous processing -$ActionQueueFileName dbq # set file name, also enables disk mode -$ActionResumeRetryCount -1 # infinite retries on insert failure -# for PostgreSQL replace :ommysql: by :ompgsql: below: -*.* :ommysql:hostname,dbname,userid,password; -</textarea> -<p><b>This is the recommended configuration for this use case.</b> It requires -rsyslog 3.11.0 or above.</p> -<p>In this example, the main message queue is NOT disk-assisted (there is no -$MainMsgQueueFileName directive). We still could do that, but have not done it -because there seems to be no need. The only slow running action is the database -writer and it has its own queue. So there is no real reason to use a large main -message queue (except, of course, if you expect *really* heavy traffic bursts).</p> -<p>Note that you can modify a lot of queue performance parameters, but the above -config will get you going with default values. If you consider using this on a real -busy server, it is strongly recommended to invest some time in setting the tuning -parameters to appropriate values.</p> - -<h3>Feedback requested</h3> - -<P>I would appreciate feedback on this tutorial. If you have additional ideas, - -comments or find bugs (I *do* bugs - no way... ;)), please - -<a href="mailto:rgerhards@adiscon.com">let me know</a>.</P> - -<h2>Revision History</h2> - -<ul> - - <li>2008-01-28 * - - <a href="http://www.adiscon.com/en/people/rainer-gerhards.php">Rainer Gerhards</a> * Initial Version created</li> - <li>2008-01-28 * - - <a href="http://www.adiscon.com/en/people/rainer-gerhards.php">Rainer Gerhards</a> - * Updated to new v3.11.0 capabilities</li> - -</ul> -<h2>Copyright</h2> - -<p>Copyright (c) 2008 - -<a href="http://www.adiscon.com/en/people/rainer-gerhards.php">Rainer Gerhards</a> and - -<a href="http://www.adiscon.com/en/">Adiscon</a>.</p> - -<p> Permission is granted to copy, distribute and/or modify this document - - under the terms of the GNU Free Documentation License, Version 1.2 - - or any later version published by the Free Software Foundation; - - with no Invariant Sections, no Front-Cover Texts, and no Back-Cover - - Texts. A copy of the license can be viewed at - -<a href="http://www.gnu.org/copyleft/fdl.html"> - -http://www.gnu.org/copyleft/fdl.html</a>.</p> - - -<p>[<a href="manual.html">manual index</a>] -[<a href="rsyslog_conf.html">rsyslog.conf</a>] -[<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> -<p><font size="2">This documentation is part of the -<a href="http://www.rsyslog.com/">rsyslog</a> project.<br> -Copyright © 2008 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and -<a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL -version 2 or higher.</font></p> - -</body> - -</html> diff --git a/doc/rsyslog_mysql.html b/doc/rsyslog_mysql.html deleted file mode 100644 index a27bd59..0000000 --- a/doc/rsyslog_mysql.html +++ /dev/null @@ -1,271 +0,0 @@ -<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> -<html><head><title>Writing syslog Data to MySQL</title> -<a href="features.html">back</a> -<meta name="KEYWORDS" content="syslog, mysql, syslog to mysql, howto"></head> -<body> -<h1>Writing syslog messages to MySQL</h1> -<p><small><i>Written by <a href="http://www.adiscon.com/en/people/rainer-gerhards.php">Rainer -Gerhards</a> (2008-02-28)</i></small></p> -<h2>Abstract</h2> -<p><i><b>In this paper, I describe how to write -<a href="http://www.monitorware.com/en/topics/syslog/">syslog</a> -messages to a <a href="http://www.mysql.com">MySQL</a> -database.</b> Having syslog messages in a database is often -handy, especially when you intend to set up a front-end for viewing -them. This paper describes an approach with <a href="http://www.rsyslog.com/">rsyslogd</a>, -an -alternative enhanced syslog daemon natively supporting MySQL. I -describe the components needed to be installed and how to configure -them. Please note that as of this writing, rsyslog supports a variety -of databases. While this guide is still MySQL-focussed, you -can probably use it together with other ones too. You just need to -modify a few settings.</i></p> -<h2>Background</h2> -<p>In many cases, syslog data is simply written to text files. -This approach has some advantages, most notably it is very fast and -efficient. However, data stored in text files is not readily accessible -for real-time viewing and analysis. To do that, the messages need to be -in a database. There are various ways to store syslog messages in a -database. For example, some have the syslogd write text files which are -later feed via a separate script into the database. Others have written -scripts taking the data (via a pipe) from a non-database-aware syslogd -and store them as they appear. Some others use database-aware syslogds -and make them write the data directly to the database. In this paper, I -use that "direct write" approach. I think it is superior, because the -syslogd itself knows the status of the database connection and thus can -handle it intelligently (well ... hopefully ;)). I use rsyslogd to -acomplish this, simply because I have initiated the rsyslog project -with database-awareness as one goal.</p> -<p><b>One word of caution:</b> while message storage -in the database provides an excellent foundation for interactive -analysis, it comes at a cost. Database i/o is considerably slower than -text file i/o. As such, directly writing to the database makes sense -only if your message volume is low enough to allow a) the syslogd, b) -the network, and c) the database server to catch up with it. Some time -ago, I have written a paper on -<a href="http://www.monitorware.com/Common/en/Articles/performance-optimizing-syslog-server.php">optimizing -syslog server performance</a>. While this paper talks about -Window-based solutions, the ideas in it are generic enough to apply -here, too. So it might be worth reading if you anticipate medium high -to high traffic. If you anticipate really high traffic (or very large -traffic spikes), you should seriously consider forgetting about direct -database writes - in my opinion, such a situation needs either a very -specialised system or a different approach (the text-file-to-database -approach might work better for you in this case). -</p> -<h2>Overall System Setup</h2> -<p>In this paper, I concentrate on the server side. If you are -thinking about interactive syslog message review, you probably want to -centralize syslog. In such a scenario, you have multiple machines (the -so-called clients) send their data to a central machine (called server -in this context). While I expect such a setup to be typical when you -are interested in storing messages in the database, I do not describe -how to set it up. This is beyond the scope of this paper. If you search -a little, you will probably find many good descriptions on how to -centralize syslog. If you do that, it might be a good idea to do it -securely, so you might also be interested in my paper on <a href="rsyslog_stunnel.html"> -ssl-encrypting syslog message transfer</a>.</p> -<p>No matter how the messages arrive at the server, their -processing is always the same. So you can use this paper in combination -with any description for centralized syslog reporting.</p> -<p>As I already said, I use rsyslogd on the server. It has -intrinsic support for talking to MySQL databases. For obvious reasons, -we also need an instance of MySQL running. To keep us focussed, the -setup of MySQL itself is also beyond the scope of this paper. I assume -that you have successfully installed MySQL and also have a front-end at -hand to work with it (for example, -<a href="http://www.phpmyadmin.net/">phpMyAdmin</a>). -Please make sure that this is installed, actually working and you have -a basic understanding of how to handle it.</p> -<h2>Setting up the system</h2> -<p>You need to download and install rsyslogd first. Obtain it -from the -<a href="http://www.rsyslog.com/">rsyslog site</a>. -Make sure that you disable stock syslogd, otherwise you will experience -some difficulties. On some distributions (Fedora 8 and above, for -example), rsyslog may already by the default syslogd, in which case you -obviously do not need to do anything specific. For many others, there -are prebuild packages available. If you use either, please make sure -that you have the required database plugins for your database -available. It usually is a separate package and typically <span style="font-weight: bold;">not</span> installed by default.</p> -<p>It is important to understand how rsyslogd talks to the -database. In rsyslogd, there is the concept of "templates". Basically, -a template is a string that includes some replacement characters, which -are called "properties" in rsyslog. Properties are accessed via the "<a href="property_replacer.html">Property Replacer</a>". -Simply said, you access properties by including their name between -percent signs inside the template. For example, if the syslog message -is "Test", the template "%msg%" would be expanded to "Test". Rsyslogd -supports sending template text as a SQL statement to MySQL. As such, -the template must be a valid SQL statement. There is no limit in what -the statement might be, but there are some obvious and not so obvious -choices. For example, a template "drop table xxx" is possible, but does -not make an awful lot of sense. In practice, you will always use an -"insert" statment inside the template.</p> -<p>An example: if you would just like to store the msg part of -the full syslog message, you have probably created a table "syslog" -with a single column "message". In such a case, a good template would -be "insert into syslog(message) values ('%msg%')". With the example -above, that would be expanded to "insert into syslog(message) -values('Test')". This expanded string is then sent to the database. -It's that easy, no special magic. The only thing you must ensure is -that your template expands to a proper SQL statement and that this -statement matches your database design.</p> -<p>Does that mean you need to create database schema yourself and -also must fully understand rsyslogd's properties? No, that's not -needed. Because we anticipated that folks are probably more interested -in getting things going instead of designing them from scratch. So we -have provided a default schema as well as build-in support for it. This -schema also offers an additional benefit: rsyslog is part of <a href="http://www.adiscon.com/en/">Adiscon</a>'s -<a href="http://www.monitorware.com/en/">MonitorWare -product line</a> (which includes open source and closed source -members). All of these tools share the same default schema and know how -to operate on it. For this reason, the default schema is also called -the "MonitorWare Schema". If you use it, you can simply add <a href="http://www.phplogcon.org/">phpLogCon, a GPLed syslog -web interface</a>, to your system and have instant interactive -access to your database. So there are some benefits in using the -provided schema.</p> -<p>The schema definition is contained in the file "createDB.sql". -It comes with the rsyslog package. Review it to check that the database -name is acceptable for you. Be sure to leave the table and field names -unmodified, because otherwise you need to customize rsyslogd's default -sql template, which we do not do in this paper. Then, run the script -with your favourite MySQL tool. Double-check that the table was -successfully created.</p> -<p>MySQL support in rsyslog is integrated via a loadable plug-in -module. To use the database -functionality, MySQL must be enabled in the config file BEFORE the -first database table action is -used. This is done by placing the</p> -<blockquote> -<p><code>$ModLoad ommysql</code></p> -</blockquote> -<p>directive at the begining of /etc/rsyslog.conf. For other databases, use their plugin name (e.g. ompgsql).</p> -<p>Next, we need to tell rsyslogd to write data to the database. -As we use the default schema, we do NOT need to define a template for -this. We can use the hardcoded one (rsyslogd handles the proper -template linking). So all we need to do is add a simple selector line -to /etc/rsyslog.conf:</p> -<blockquote> -<p><code>*.* :ommysql:database-server,database-name,database-userid,database-password</code></p> -</blockquote> -<p>Again, other databases have other selector names, e.g. ":ompgsql:" -instead of ":ommysql:". See the output plugin's documentation for -details.</p><p>In many cases, MySQL will run on the local machine. In this -case, you can simply use "127.0.0.1" for <i>database-server</i>. -This can be especially advisable, if you do not need to expose MySQL to -any process outside of the local machine. In this case, you can simply -bind it to 127.0.0.1, which provides a quite secure setup. Of course, -also supports remote MySQL instances. In that case, use the remote -server name (e.g. mysql.example.com) or IP-address. The <i> -database-name</i> by default is "syslog". If you have modified -the default, use your name here. <i>Database-userid</i> -and <i>-password</i> are the credentials used to connect -to the database. As they are stored in clear text in rsyslog.conf, that -user should have only the least possible privileges. It is sufficient -to grant it INSERT privileges to the systemevents table, only. As a -side note, it is strongly advisable to make the rsyslog.conf file -readable by root only - if you make it world-readable, everybody could -obtain the password (and eventually other vital information from it). -In our example, let's assume you have created a MySQL user named -"syslogwriter" with a password of "topsecret" (just to say it bluntly: -such a password is NOT a good idea...). If your MySQL database is on -the local machine, your rsyslog.conf line might look like in this -sample:</p> -<blockquote> -<p><code>*.* :ommysql:127.0.0.1,Syslog,syslogwriter,topsecret</code></p> -</blockquote> -<p>Save rsyslog.conf, restart rsyslogd - and you should see -syslog messages being stored in the "systemevents" table!</p> -<p>The example line stores every message to the database. -Especially if you have a high traffic volume, you will probably limit -the amount of messages being logged. This is easy to acomplish: the -"write database" action is just a regular selector line. As such, you -can apply normal selector-line filtering. If, for example, you are only -interested in messages from the mail subsystem, you can use the -following selector line:</p> -<blockquote> -<p><code>mail.* </code><code>:ommysql:</code><code>127.0.0.1,syslog,syslogwriter,topsecret</code></p> -</blockquote> -<p>Review the <a href="rsyslog_conf.html">rsyslog.conf</a> -documentation for details on selector lines and their filtering.</p> -<p><b>You have now completed everything necessary to store -syslog messages to the MySQL database.</b> If you would like to -try out a front-end, you might want to look at <a href="http://www.phplogcon.org/">phpLogCon</a>, which -displays syslog data in a browser. As of this writing, phpLogCon is not -yet a powerful tool, but it's open source, so it might be a starting -point for your own solution.</p> -<h2>On Reliability...</h2> -<p>Rsyslogd writes syslog messages directly to the database. This -implies that the database must be available at the time of message -arrival. If the database is offline, no space is left or something else -goes wrong - rsyslogd can not write the database record. If rsyslogd is -unable to store a message, it performs one retry. This is helpful if -the database server was restarted. In this case, the previous -connection was broken but a reconnect immediately succeeds. However, if -the database is down for an extended period of time, an immediate retry -does not help.</p> -<p>Message loss in this scenario can easily be prevented with -rsyslog. All you need to do is run the database writer in queued mode. -This is now described in a generic way and I do not intend to duplicate -it here. So please be sure to read "<a href="rsyslog_high_database_rate.html">Handling a massive -syslog database insert rate with Rsyslog</a>", which describes -the scenario and also includes configuration examples.</p> -<h2>Conclusion</h2> -<p>With minimal effort, you can use rsyslogd to write syslog -messages to a MySQL database. You can even make it absolutely fail-safe -and protect it against database server downtime. Once the messages are -arrived there, you -can interactivley review and analyse them. In practice, the messages -are also stored in text files for longer-term archival and the -databases are cleared out after some time (to avoid becoming too slow). -If you expect an extremely high syslog message volume, storing it in -real-time to the database may outperform your database server. In such -cases, either filter out some messages or used queued mode (which in -general is recommended with databases).</p> -<p>The method outlined in this paper provides an easy to setup -and maintain solution for most use cases.</p> -<h3>Feedback Requested</h3> -<p>I would appreciate feedback on this paper. If you have -additional ideas, comments or find bugs, please -<a href="mailto:rgerhards@adiscon.com">let me know</a>.</p> -<h2>References and Additional Material</h2> -<ul> -<li><a href="http://www.rsyslog.com">www.rsyslog.com</a> -- the rsyslog site</li> -<li> <a href="http://www.monitorware.com/Common/en/Articles/performance-optimizing-syslog-server.php"> -Paper on Syslog Server Optimization</a></li> -</ul> -<h2>Revision History</h2> -<ul> -<li>2005-08-02 * <a href="http://www.adiscon.com/en/people/rainer-gerhards.php">Rainer -Gerhards</a> * initial version created</li> -<li>2005-08-03 * <a href="http://www.adiscon.com/en/people/rainer-gerhards.php">Rainer -Gerhards</a> * added references to demo site</li> -<li>2007-06-13 * <a href="http://www.adiscon.com/en/people/rainer-gerhards.php">Rainer -Gerhards</a> * removed demo site - was torn down because too -expensive for usage count</li> -<li>2008-02-21 * <a href="http://www.adiscon.com/en/people/rainer-gerhards.php">Rainer -Gerhards</a> * updated reliability section, can now be done with -on-demand disk queues</li><li>2008-02-28 * <a href="http://www.adiscon.com/en/people/rainer-gerhards.php">Rainer -Gerhards</a> * added info on other databases, updated syntax to more recent one</li> -</ul> -<h2>Copyright</h2> -<p>Copyright (c) 2005-2008 -<a href="http://www.adiscon.com/en/people/rainer-gerhards.php">Rainer -Gerhards</a> and <a href="http://www.adiscon.com/en/">Adiscon</a>.</p> -<p>Permission is granted to copy, distribute and/or modify this -document under the terms of the GNU Free Documentation License, Version -1.2 or any later version published by the Free Software Foundation; -with no Invariant Sections, no Front-Cover Texts, and no Back-Cover -Texts. A copy of the license can be viewed at <a href="http://www.gnu.org/copyleft/fdl.html"> -http://www.gnu.org/copyleft/fdl.html</a>.</p> -<p>[<a href="manual.html">manual index</a>] -[<a href="rsyslog_conf.html">rsyslog.conf</a>] -[<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> -<p><font size="2">This documentation is part of the -<a href="http://www.rsyslog.com/">rsyslog</a> project.<br> -Copyright © 2008 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and -<a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL -version 2 or higher.</font></p> - -</body></html> diff --git a/doc/rsyslog_ng_comparison.html b/doc/rsyslog_ng_comparison.html deleted file mode 100644 index 44c895f..0000000 --- a/doc/rsyslog_ng_comparison.html +++ /dev/null @@ -1,613 +0,0 @@ -<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> -<html><head><title>rsyslog vs. syslog-ng - a comparison</title></head> -<body> -<a href="features.html">back</a> -<h1>rsyslog vs. syslog-ng</h1> -<p><small><i>Written by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> -(2008-05-06), slightly updated 2012-01-09</i></small></p> -<p><b>This comparison page is rooted nearly 5 years in the past and has become severely -outdated since then.</b> It was unmaintained for several years and contained false -information on both syslog-ng and rsyslog as technology had advanced so much. -<p>This page was initially written because so many people asked about a comparison when -rsyslog was in its infancy. So I tried to create one, but it was hard to maintain as both -projects grew and added feature after feature. I have to admit we did not try hard to keep -it current -- there were many other priorities. I even had forgetten about this page, when I -saw that Peter Czanik blogged about its -<a href="http://blogs.balabit.com/2012/01/05/rsyslog-vs-syslog-ng/">incorrectness</a> (it must be noted -that Peter is wrong on RELP -- it is well alive). I now remember -that he asked me some time ago about this page, what I somehow lost... I guess he must have been -rather grumpy about that :-( -<p>Visiting this page after so many years is interesting, because it shows how much has changed since then. -Obviously, one of my main goals in regard to syslog-ng is reached: in 2007, I blogged that -<a href="http://blog.gerhards.net/2007/08/why-does-world-need-another-syslogd.html">the -world needs another syslogd</a> in order to have healthy competition and a greate feature -set in the free editions. In my opinion, the timeline clearly tells that rsyslog's competition -has driven more syslog-ng features from the commercial to the free edition. Also, I found -it interesting to see that syslog-ng has adapted rsyslog's licensing scheme, modular design and -multi-threadedness. On the other hand, the Balabit folks have obviously done a quicker and -better move on log normalization with what they call patterndb (it is very roughly equivalent -to what rsyslog has just recently introduced with the help of liblognorm). - -<p>To that account, I think the projects are closer together than 5 years ago. I should now -go ahead and create a new feature comparison. Given previous experience, I think this does not -work out. In the future, we will probably focus on some top features, as Balabit does. However, -that requires some time and I have to admit I do not like to drop this page that has a lot of -inbound links. So I think I do the useful thing by providing these notes and removing the -syslog-ng information. So it can't be wrong on syslog-ng any more. Note that it still contains -some incorrect information about rsyslog (it's the state it had 5 years ago!). The core idea is -to start with updating the <a href="features.html">rsyslog feature sheet</a> and from there -on work to a complete comparision. Of course, feel free to read on if you like to get some sense -of history (and inspiration on what you can still do -- but more ;)). -<br><br> -Thanks,<br> -Rainer Gerhards -<p> - -<table border="1"> -<tbody> -<tr> -<td valign="top"><b>Feature</b></td> -<td valign="top"><b>rsyslog</b></td> -<td valign="top"><b>syslog-ng</b></td> -</tr> -<tr> -<td colspan="3" valign="top"><br> -<b>Input Sources</b><br> -</td> -</tr> -<tr> -<td valign="top">UNIX domain socket</td> -<td valign="top">yes</td> -<td valign="top"></td> -<td></td> -</tr> -<tr> -<td valign="top">UDP</td> -<td valign="top">yes</td> -<td valign="top"></td> -<td></td> -</tr> -<tr> -<td valign="top">TCP</td> -<td valign="top">yes</td> -<td valign="top"></td> -<td></td> -</tr> -<tr> -<td valign="top"><a href="http://www.librelp.com">RELP</a></td> -<td valign="top">yes</td> -<td valign="top"></td> -<td></td> -</tr> -<tr> -<td valign="top">RFC 3195/BEEP</td> -<td valign="top">yes (via <a href="im3195.html">im3195</a>)</td> -<td valign="top"></td> -<td></td> -</tr> -<tr> -<td valign="top">kernel log</td> -<td valign="top">yes</td> -<td valign="top"></td> -<td></td> -</tr> -<tr> -<td valign="top">file</td> -<td valign="top">yes</td> -<td valign="top"></td> -<td></td> -</tr> -<tr> -<td valign="top">mark message generator as an -optional input</td> -<td valign="top">yes</td> -<td valign="top"></td> -<td></td> -</tr> -<tr> -<td valign="top">Windows Event Log</td> -<td valign="top">via a Windows event logging software such as -<a href="http://www.eventreporter.com">EventReporter</a> -or <a href="http://www.mwagent.com">MonitorWare Agent</a> -(both commercial software, both fund rsyslog development)</td> -<td valign="top"></td> -</tr> -<tr> -<td colspan="3" valign="top"><b><br> -Network (Protocol) Support</b><br> -</td> -</tr> -<tr> -<td valign="top">support for (plain) tcp based syslog</td> -<td valign="top">yes</td> -<td valign="top"></td> -</tr> -<tr> -<td valign="top">support for GSS-API</td> -<td valign="top">yes</td> -<td valign="top"></td> -</tr> -<tr> -<td valign="top">ability to limit the allowed -network senders (syslog ACLs)</td> -<td valign="top">yes</td> -<td valign="top"></td> -</tr> -<tr> -<td valign="top">support for syslog-transport-tls -based framing on syslog/tcp connections</td> -<td valign="top">yes</td> -<td valign="top"></td> -</tr> -<tr> -<td valign="top">udp syslog</td> -<td valign="top">yes</td> -<td valign="top"></td> -</tr> -<tr> -<td valign="top">syslog over RELP<br> -truly reliable message delivery (<a href="http://blog.gerhards.net/2008/05/why-you-cant-build-reliable-tcp.html">Why -is plain tcp syslog not reliable?</a>)</td> -<td valign="top">yes</td> -<td valign="top"></td> -</tr> -<tr> -<td valign="top">on the wire (zlib) message -compression</td> -<td valign="top">yes</td> -<td valign="top"></td> -</tr> -<tr> -<td valign="top">support for receiving messages via -reliable <a href="http://www.monitorware.com/Common/en/glossary/rfc3195.php">RFC -3195</a> delivery</td> -<td valign="top">yes</td> -<td valign="top"></td> -</tr> -<tr> -<td valign="top">support for <a href="rsyslog_tls.html">TLS/SSL-protected -syslog</a> </td> -<td valign="top"><a href="rsyslog_tls.html">natively</a> (since 3.19.0)<br><a href="rsyslog_stunnel.html">via -stunnel</a></td> -<td valign="top"></td> -</tr> -<tr> -<td valign="top">support for IETF's new syslog-protocol draft</td> -<td valign="top">yes</td> -<td valign="top"></td> -</tr> -<tr> -<td valign="top">support for IETF's new syslog-transport-tls draft</td> -<td valign="top">yes<br>(since 3.19.0 - world's first implementation)</td> -<td valign="top"></td> -</tr> -<tr> -<td valign="top">support for IPv6</td> -<td valign="top">yes</td> -<td valign="top"></td> -</tr> -<tr> -<td valign="top">native ability to send SNMP traps</td> -<td valign="top">yes</td> -<td valign="top"></td> -</tr> -<tr> -<td valign="top">ability to preserve the original -hostname in NAT environments and relay chains</td> -<td valign="top">yes</td> -<td valign="top"></td> -</tr> -<tr> -<td colspan="3" valign="top"><br> -<b>Message Filtering</b><br> -</td> -</tr> -<tr> -<td valign="top">Filtering for syslog facility and -priority</td> -<td valign="top">yes</td> -<td valign="top"></td> -<td></td> -</tr> -<tr> -<td valign="top">Filtering for hostname</td> -<td valign="top">yes</td> -<td valign="top"></td> -<td></td> -</tr> -<tr> -<td valign="top">Filtering for application</td> -<td valign="top">yes</td> -<td valign="top"></td> -<td></td> -</tr> -<tr> -<td valign="top">Filtering for message contents</td> -<td valign="top">yes</td> -<td valign="top"></td> -<td></td> -</tr> -<tr> -<td valign="top">Filtering for sending IP address</td> -<td valign="top">yes</td> -<td valign="top"></td> -<td></td> -</tr> -<tr> -<td valign="top">ability to filter on any other message -field not mentioned above (including substrings and the like)</td> -<td valign="top">yes</td> -<td valign="top"></td> -</tr> -<tr> -<td>support for complex filters, using full boolean algebra -with and/or/not operators and parenthesis</td> -<td>yes</td> -<td></td> -</tr> -<tr> -<td>Support for reusable filters: specify a filter once and -use it in multiple selector lines</td> -<td>no</td> -<td></td> -</tr> -<tr> -<td>support for arbritrary complex arithmetic and string -expressions inside filters</td> -<td>yes</td> -<td></td> -</tr> -<tr> -<td valign="top">ability to use regular expressions -in filters</td> -<td valign="top">yes</td> -<td valign="top"></td> -</tr> -<tr> -<td valign="top">support for discarding messages -based on filters</td> -<td valign="top">yes</td> -<td valign="top"></td> -<td></td> -</tr> -<tr> -<td valign="top">ability to filter out messages based on sequence of appearing</td> -<td valign="top">yes (starting with 3.21.3)</td> -<td valign="top"></td> -<td></td> -</tr> -<tr> -<td valign="top">powerful BSD-style hostname and -program name blocks for easy multi-host support</td> -<td valign="top">yes</td> -<td valign="top"></td> -</tr> -<tr> -<td></td> -<td></td> -<td></td> -</tr> -<tr> -<td colspan="3" valign="top"><br> -<b>Supported Database Outputs</b><br> -</td> -</tr> -<tr> -<td valign="top">MySQL</td> -<td valign="top"><a href="rsyslog_mysql.html">yes</a> -(native ommysql, <a href="omlibdbi.html">omlibdbi</a>)</td> -<td valign="top"></td> -</tr> -<tr> -<td valign="top">PostgreSQL</td> -<td valign="top">yes (native ompgsql, <a href="omlibdbi.html">omlibdbi</a>)</td> -<td valign="top"></td> -</tr> -<tr> -<td valign="top">Oracle</td> -<td valign="top">yes (<a href="omlibdbi.html">omlibdbi</a>)</td> -<td valign="top"></td> -</tr> -<tr> -<td valign="top">SQLite</td> -<td valign="top">yes (<a href="omlibdbi.html">omlibdbi</a>)</td> -<td valign="top"></td> -</tr> -<tr> -<td valign="top">Microsoft SQL (Open TDS)</td> -<td valign="top">yes (<a href="omlibdbi.html">omlibdbi</a>)</td> -<td valign="top"></td> -</tr> -<tr> -<td valign="top">Sybase (Open TDS)</td> -<td valign="top">yes (<a href="omlibdbi.html">omlibdbi</a>)</td> -<td valign="top"></td> -</tr> -<tr> -<td valign="top">Firebird/Interbase</td> -<td valign="top">yes (<a href="omlibdbi.html">omlibdbi</a>)</td> -<td valign="top"></td> -</tr> -<tr> -<td valign="top">Ingres</td> -<td valign="top">yes (<a href="omlibdbi.html">omlibdbi</a>)</td> -<td valign="top"></td> -</tr> -<tr> -<td valign="top">mSQL</td> -<td valign="top">yes (<a href="omlibdbi.html">omlibdbi</a>)</td> -<td valign="top"></td> -</tr> -<tr> -<td colspan="3" valign="top"><br> -<b>Enterprise Features</b><br> -</td> -</tr> -<tr> -<td valign="top">support for on-demand on-disk -spooling of messages</td> -<td valign="top">yes</td> -<td valign="top"></td> -</tr> -<tr> -<td valign="top">ability to limit disk space used -by spool files</td> -<td valign="top">yes</td> -<td valign="top"></td> -</tr> -<tr> -<td valign="top">each action can use its own, -independant -set of spool files</td> -<td valign="top">yes</td> -<td valign="top"></td> -</tr> -<tr> -<td valign="top">different sets of spool files can -be placed on different disk</td> -<td valign="top">yes</td> -<td valign="top"></td> -</tr> -<tr> -<td valign="top">ability to process spooled -messages only during a configured timeframe (e.g. process messages only -during off-peak hours, during peak hours they are enqueued only)</td> -<td valign="top"><a href="http://wiki.rsyslog.com/index.php/OffPeakHours">yes</a><br> -(can independently be configured for the main queue and each action -queue)</td> -<td valign="top"></td> -</tr> -<tr> -<td valign="top">ability to configure backup -syslog/database servers </td> -<td valign="top">yes</td> -<td valign="top"></td> -</tr> -<tr> -<td>Professional Support</td> -<td><a href="professional_support.html">yes</a></td> -<td></td> -</tr> -<tr> -<td colspan="3" valign="top"><br> -<b>Config File</b><br> -</td> -</tr> -<tr> -<td valign="top">config file format</td> -<td valign="top">compatible to legacy syslogd but -ugly</td> -<td valign="top"></td> -</tr> -<tr> -<td valign="top">ability to include config file from -within other config files</td> -<td valign="top">yes</td> -<td valign="top"></td> -</tr> -<tr> -<td height="25" valign="top">ability to -include all config files -existing in a specific directory</td> -<td height="25" valign="top">yes</td> -<td height="25" valign="top"></td> -</tr> -<tr> -<td colspan="3" valign="top"><br> -<b>Extensibility</b><br> -</td> -</tr> -<tr> -<td valign="top">Functionality split in separately -loadable -modules</td> -<td valign="top">yes</td> -<td valign="top"></td> -</tr> -<tr> -<td valign="top">Support for third-party input -plugins</td> -<td valign="top">yes</td> -<td valign="top"></td> -</tr> -<tr> -</tr> -<tr> -<td valign="top">Support for third-party output -plugins</td> -<td valign="top">yes</td> -<td valign="top"></td> -</tr> -<tr> -<td colspan="3" valign="top"><br> -<b>Other Features</b><br> -</td> -</tr> -<tr> -</tr> -<tr> -<td valign="top">ability to generate file names and -directories (log targets) dynamically</td> -<td valign="top">yes</td> -<td valign="top"></td> -</tr> -<tr> -<td valign="top">control of log output format, -including ability to present channel and priority as visible log data</td> -<td valign="top">yes</td> -<td valign="top"></td> -</tr> -<tr><td valign="top">native ability to send mail messages</td> -<td valign="top">yes (<a href="ommail.html">ommail</a>, introduced in 3.17.0)</td> -<td valign="top"></td> -</tr> -<tr> -<td valign="top">good timestamp format control; at a -minimum, ISO 8601/RFC 3339 second-resolution UTC zone</td> -<td valign="top">yes</td> -<td valign="top"></td> -</tr> -<tr> -<td valign="top">ability to reformat message -contents and work with substrings</td> -<td valign="top">yes</td> -<td valign="top"></td> -</tr> -<tr> -<td valign="top">support for log files larger than -2gb</td> -<td valign="top">yes</td> -<td valign="top"></td> -</tr> -<tr> -<td valign="top">support for log file size -limitation -and automatic rollover command execution</td> -<td valign="top">yes</td> -<td valign="top"></td> -</tr> -<tr> -<td valign="top">support for running multiple -syslogd instances on a single machine</td> -<td valign="top">yes</td> -<td valign="top"></td> -</tr> -<tr> -<td valign="top">ability to execute shell scripts on -received messages</td> -<td valign="top"></td> -<td valign="top">yes</td> -</tr> -<tr> -<td valign="top">ability to pipe messages to a -continously running program</td> -<td valign="top"></td> -<td valign="top"></td> -</tr> -<tr> -<td valign="top">massively multi-threaded for -tomorrow's multi-core machines</td> -<td valign="top">yes</td> -<td valign="top"></td> -</tr> -<tr> -<td valign="top">ability to control repeated line -reduction ("last message repeated n times") on a per selector-line basis</td> -<td valign="top">yes</td> -<td valign="top"></td> -</tr> -<tr> -<td valign="top">supports multiple actions per -selector/filter condition</td> -<td valign="top">yes</td> -<td valign="top"></td> -<td></td> -</tr> -<tr> -<td valign="top">web interface</td> -<td valign="top"><a href="http://www.phplogcon.org">phpLogCon</a><br> -[also works with <a href="http://freshmeat.net/projects/php-syslog-ng/"> -php-syslog-ng</a>]</td> -<td valign="top"></td> -</tr> -<tr> -<td valign="top">using text files as input source</td> -<td valign="top">yes</td> -<td valign="top"></td> -</tr> -<tr> -<td valign="top">rate-limiting output actions</td> -<td valign="top">yes</td> -<td valign="top"></td> -</tr> -<tr> -<td valign="top">discard low-priority messages under -system stress</td> -<td valign="top">yes</td> -<td valign="top"></td> -</tr> -<tr> -<td height="43" valign="top">flow control -(slow down message reception when system is busy)</td> -<td height="43" valign="top">yes (advanced, -with multiple ways to slow down inputs depending on individual input -capabilities, based on watermarks)</td> -<td height="43" valign="top"></td> -</tr> -<tr> -<td valign="top">rewriting messages</td> -<td valign="top">yes</td> -<td valign="top"></td> -</tr> -<tr> -<td valign="top">output data into various formats</td> -<td valign="top">yes</td> -<td valign="top"></td> -</tr> -<tr> -<td valign="top">ability to control "message -repeated n times" generation</td> -<td valign="top">yes</td> -<td valign="top"></td> -</tr> -<tr> -<td valign="top">license</td> -<td valign="top">GPLv3 (GPLv2 for v2 branch)</td> -<td valign="top"></td> -</tr> -<tr> -<td valign="top">supported platforms</td> -<td valign="top">Linux, BSD, anecdotical seen on -Solaris; compilation and basic testing done on HP UX</td> -<td valign="top"></td> -</tr> -<tr> -<td valign="top">DNS cache</td> -<td valign="top"></td> -<td valign="top"></td> -</tr> -</tbody> -</table> -<p>While the <span style="font-weight: bold;">rsyslog</span> -project was initiated in 2004, it <span style="font-weight: bold;">is -build on the main author's (Rainer Gerhards) 12+ years of -logging experience</span>. Rainer, for example, also -wrote the first <a href="http://www.winsyslog.com/Common/en/News/WinSyslog-1996-03-31.php">Windows -syslog server</a> in early 1996 and invented the <a href="http://www.eventreporter.com/Common/en/News/EvntSLog-1997-03-23.php">eventlog-to-syslog</a> -class of applications in early 1997. He did custom logging development -and consulting even before he wrote these products. Rsyslog draws on -that vast experience and sometimes even on the code.</p> -<p>Based on a discussion I had, I also wrote about the <b>political -argument why it is good to have another strong syslogd besides syslog-ng</b>. -You may want to read it at my blog at "<a href="http://rgerhards.blogspot.com/2007/08/why-does-world-need-another-syslogd.html">Why -does the world need another syslogd?</a>".</p> -<p>[<a href="manual.html">manual index</a>] -[<a href="rsyslog_conf.html">rsyslog.conf</a>] -[<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> -<p><font size="2">This documentation is part of the -<a href="http://www.rsyslog.com/">rsyslog</a> project.<br> -Copyright © 2008 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and -<a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL -version 2 or higher.</font></p> - -</body></html> diff --git a/doc/rsyslog_packages.html b/doc/rsyslog_packages.html deleted file mode 100644 index 80ba96c..0000000 --- a/doc/rsyslog_packages.html +++ /dev/null @@ -1,76 +0,0 @@ -<html> -<head> -<title>rsyslog precompiled packages (RPM and such...)</title> -</head> -<body> -<h1>rsyslog packages</h1> -<p><b>Thanks to some volunteers, rsyslog is also available in package form on -some distributions.</b> All currently known packages are listed below. If I have forgotten -one or if you would -like to maintain a package for a new distribution, please mail me at -<a href="mailto:rgerhards@adiscon.com">rgerhards@adiscon.com</a>. Any help is *deeply* -appreciated. While I create the core daemon, the package maintainers are really -filling it with life, making it available to the average user. I am very -grateful for that!</p> -<p>This list has last been updated on 2008-07-11 by -<a href="http://www.adiscon.com/en/people/rainer-gerhards.php">Rainer Gerhards</a>. -New packages may appear at any time, so be sure to check this page whenever you -need a new one.</p> -<ul> -<li><b>BSD</b> (maintained by infofarmer) - <ul> - <li><a href="http://www.freshports.org/sysutils/rsyslog/"> http://www.freshports.org/sysutils/rsyslog/</a> - </ul> - -<li><b>CentOS 4.3</b> (maintained by James Bergamin) - <ul> - <li><a href="http://www.se-community.com/~james/rsyslog/"> -http://www.se-community.com/~james/rsyslog/</a> - </ul> - -<li><b>Debian</b> (maintained by Michael Biebl) - <ul> - <li><a href="http://packages.debian.org/sid/rsyslog">http://packages.debian.org/sid/rsyslog</a> - </ul> - -<li><b>Fedora</b> - <ul> - <li>Starting with Fedora 8, rsyslog is available as part of the core distribution. - </ul> - -<li><b>openSUSE</b> (maintained by darix) - <ul> - <li><a href="http://download.opensuse.org/repositories/home:/darix/">http://download.opensuse.org/repositories/home:/darix/</a> - </ul> - -<li><b>Red Hat Enterprise Linux</b> - <ul> - <li>Starting with RHEL 5.2, rsyslog is available as part of the core distribution. - </ul> - -<li><b>Ubuntu</b> - <ul> - <li>Starting with hardy, rsyslog is available from the universe repository. - </ul> - -<li>Almost any Linux</h2> - <ul> - <li>Bennet Todd maintains packages that should work on almost any Linux. -He keeps a current i386 tree. There is also a PPC tree, but that one is not paid -much attention for (anyhow, it is known to typically work well, too). -Please visit <a href="http://bent.latency.net/bent/"> -http://bent.latency.net/bent/</a>, select the relevant tree and then do a search -for rsyslog. -Please note, however, that as of this writing the versions in this repository -have been aged a bit. So it may be worth trying to find some other places first. - </ul> -</ul> - -<p>Just in case you are interested, the list of distribution is sorted by alphabetic order -of the distribution name. - -<p>If you do not find a suitable package for your distribution, there is no reason -to panic. It is quite simple to install rsyslog from the source tarball, so you -should consider that. -</body> -</html> diff --git a/doc/rsyslog_pgsql.html b/doc/rsyslog_pgsql.html deleted file mode 100644 index 21516ec..0000000 --- a/doc/rsyslog_pgsql.html +++ /dev/null @@ -1,336 +0,0 @@ -<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN"> -<HTML> -<HEAD> - <META HTTP-EQUIV="CONTENT-TYPE" CONTENT="text/html; charset=utf-8"> - <TITLE></TITLE> - <META NAME="GENERATOR" CONTENT="OpenOffice.org 3.1 (Unix)"> - <META NAME="AUTHOR" CONTENT="Marc Schiffbauer"> - <META NAME="CREATED" CONTENT="20100129;15054500"> - <META NAME="CHANGEDBY" CONTENT="Marc Schiffbauer"> - <META NAME="CHANGED" CONTENT="20100129;16035000"> - <META NAME="Info 1" CONTENT=""> - <META NAME="Info 2" CONTENT=""> - <META NAME="Info 3" CONTENT=""> - <META NAME="Info 4" CONTENT=""> - <STYLE TYPE="text/css"> - <!-- - @page { size: 8.27in 11.69in; margin: 0.79in } - P { margin-bottom: 0.08in } - P.western { font-family: "Arial", sans-serif } - H1 { margin-bottom: 0.08in } - H1.western { font-family: "Times New Roman", serif } - H1.cjk { font-family: "DejaVu Sans" } - H1.ctl { font-family: "DejaVu Sans" } - H2 { margin-bottom: 0.08in } - H2.western { font-family: "Times New Roman", serif } - BLOCKQUOTE.western { font-family: "Arial", sans-serif } - H3 { margin-bottom: 0.08in } - H3.western { font-family: "Times New Roman", serif } - A:link { so-language: zxx } - --> - </STYLE> -</HEAD> -<BODY> -<H1 CLASS="western"><SPAN LANG="en-US">Writing </SPAN>syslog messages -to MySQL, PostgreSQL or any other supported Database</H1> -<P CLASS="western"><FONT SIZE=2><I>Written by </I></FONT><A HREF="http://www.adiscon.com/en/people/rainer-gerhards.php"><FONT SIZE=2><I>Rainer -Gerhards</I></FONT></A><FONT SIZE=2><I> with some additions by Marc -Schiffbauer (2008-02-28)</I></FONT></P> -<H2 CLASS="western">Abstract</H2> -<P CLASS="western"><SPAN LANG="en-US"><I><B>In this paper, I describe -how to write </B></I></SPAN><A HREF="http://www.monitorware.com/en/topics/syslog/">syslog</A><SPAN LANG="en-US"><I><B> -messages to a </B></I></SPAN><A HREF="http://www.mysql.com/">MySQL</A><SPAN LANG="en-US"><I><B> -or </B></I></SPAN><A HREF="http://www.postgresql.org/">PostgreSQL</A><SPAN LANG="en-US"><I><B> -database.</B></I></SPAN><SPAN LANG="en-US"><I> Having syslog messages -in a database is often handy, especially when you intend to set up a -front-end for viewing them. This paper describes an approach with -</I></SPAN><A HREF="http://www.rsyslog.com/">rsyslogd</A><SPAN LANG="en-US"><I>, -an alternative enhanced syslog daemon natively supporting MySQL and -PostgreSQL. I describe the components needed to be installed and how -to configure them. Please note that as of this writing, rsyslog -supports a variety of databases. While this guide is still MySQL- and -PostgreSQL-focused, you can probably use it together with other ones -too. You just need to modify a few settings.</I></SPAN></P> -<H2 CLASS="western">Background</H2> -<P LANG="en-US" CLASS="western">In many cases, syslog data is simply -written to text files. This approach has some advantages, most -notably it is very fast and efficient. However, data stored in text -files is not readily accessible for real-time viewing and analysis. -To do that, the messages need to be in a database. There are various -ways to store syslog messages in a database. For example, some have -the syslogd write text files which are later feed via a separate -script into the database. Others have written scripts taking the data -(via a pipe) from a non-database-aware syslogd and store them as they -appear. Some others use database-aware syslogds and make them write -the data directly to the database. In this paper, I use that "direct -write" approach. I think it is superior, because the syslogd -itself knows the status of the database connection and thus can -handle it intelligently (well ... hopefully ;)). I use rsyslogd to -acomplish this, simply because I have initiated the rsyslog project -with database-awareness as one goal.</P> -<P CLASS="western"><SPAN LANG="en-US"><B>One word of caution:</B></SPAN><SPAN LANG="en-US"> -while message storage in the database provides an excellent -foundation for interactive analysis, it comes at a cost. Database i/o -is considerably slower than text file i/o. As such, directly writing -to the database makes sense only if your message volume is low enough -to allow a) the syslogd, b) the network, and c) the database server -to catch up with it. Some time ago, I have written a paper on -</SPAN><A HREF="http://www.monitorware.com/Common/en/Articles/performance-optimizing-syslog-server.php">optimizing -syslog server performance</A><SPAN LANG="en-US">. While this paper -talks about Window-based solutions, the ideas in it are generic -enough to apply here, too. So it might be worth reading if you -anticipate medium high to high traffic. If you anticipate really high -traffic (or very large traffic spikes), you should seriously consider -forgetting about direct database writes - in my opinion, such a -situation needs either a very specialized system or a different -approach (the text-file-to-database approach might work better for -you in this case). </SPAN> -</P> -<H2 CLASS="western">Overall System Setup</H2> -<P CLASS="western"><SPAN LANG="en-US">In this paper, I concentrate on -the server side. If you are thinking about interactive syslog message -review, you probably want to centralize syslog. In such a scenario, -you have multiple machines (the so-called clients) send their data to -a central machine (called server in this context). While I expect -such a setup to be typical when you are interested in storing -messages in the database, I do not describe how to set it up. This is -beyond the scope of this paper. If you search a little, you will -probably find many good descriptions on </SPAN><SPAN LANG="en-US">how -to centralize syslog. If you do that, it might be a good idea to do -it securely, so you might also be interested in my paper on -</SPAN><A HREF="http://www.rsyslog.com/doc-rsyslog_stunnel.html">ssl-encrypting -syslog message transfer</A><SPAN LANG="en-US">.</SPAN></P> -<P LANG="en-US" CLASS="western">No matter how the messages arrive at -the server, their processing is always the same. So you can use this -paper in combination with any description for centralized syslog -reporting.</P> -<P CLASS="western"><SPAN LANG="en-US">As I already said, I use -rsyslogd on the server. It has intrinsic support for talking to the -supported databases. For obvious reasons, we also need an instance of -MySQL or PostgreSQL running. To keep us focused, the setup of the -database itself is also beyond the scope of this paper. I assume that -you have successfully installed the database and also have a -front-end at hand to work with it (for example, </SPAN><A HREF="http://www.phpmyadmin.net/">phpMyAdmin</A><SPAN LANG="en-US"> -or </SPAN><A HREF="http://phppgadmin.sourceforge.net/">phpPgAdmin</A><SPAN LANG="en-US">. -Please make sure that this is installed, actually working and you -have a basic understanding of how to handle it.</SPAN></P> -<H2 CLASS="western">Setting up the system</H2> -<P CLASS="western"><SPAN LANG="en-US">You need to download and -install rsyslogd first. Obtain it from the </SPAN><A HREF="http://www.rsyslog.com/">rsyslog -site</A><SPAN LANG="en-US">. Make sure that you disable stock -syslogd, otherwise you will experience some difficulties. On some -distributions (Fedora 8 and above, for example), rsyslog may -already by the default syslogd, in which case you obviously do not -need to do anything specific. For many others, there are prebuild -packages available. If you use either, please make sure that you have -the required database plugins for your database available. It usually -is a separate package and typically </SPAN><SPAN LANG="en-US"><B>not</B></SPAN><SPAN LANG="en-US"> -installed by default.</SPAN></P> -<P CLASS="western"><SPAN LANG="en-US">It is important to understand -how rsyslogd talks to the database. In rsyslogd, there is the concept -of "templates". Basically, a template is a string that -includes some replacement characters, which are called "properties" -in rsyslog. Properties are accessed via the "</SPAN><A HREF="http://www.rsyslog.com/doc-property_replacer.html">Property -Replacer</A><SPAN LANG="en-US">". Simply said, you access -properties by including their name between percent signs inside the -template. For example, if the syslog message is "Test", the -template "%msg%" would be expanded to "Test". -Rsyslogd supports sending template text as a SQL statement to the -database. As such, the template must be a valid SQL statement. There -is no limit in what the statement might be, but there are some -obvious and not so obvious choices. For example, a template "drop -table xxx" is possible, but does not make an awful lot of sense. -In practice, you will always use an "insert" statement -inside the template.</SPAN></P> -<P LANG="en-US" CLASS="western">An example: if you would just like to -store the msg part of the full syslog message, you have probably -created a table "syslog" with a single column "message". -In such a case, a good template would be "insert into -syslog(message) values ('%msg%')". With the example above, that -would be expanded to "insert into syslog(message) -values('Test')". This expanded string is then sent to the -database. It's that easy, no special magic. The only thing you must -ensure is that your template expands to a proper SQL statement and -that this statement matches your database design.</P> -<P CLASS="western"><SPAN LANG="en-US">Does that mean you need to -create database schema yourself and also must fully understand -rsyslogd's properties? No, that's not needed. Because we anticipated -that folks are probably more interested in getting things going -instead of designing them from scratch. So we have provided a default -schema as well as build-in support for it. This schema also offers an -additional benefit: rsyslog is part of </SPAN><A HREF="http://www.adiscon.com/en/">Adiscon</A><SPAN LANG="en-US">'s -</SPAN><A HREF="http://www.monitorware.com/en/">MonitorWare product -line</A><SPAN LANG="en-US"> (which includes open source and closed -source members). All of these tools share the same default schema and -know how to operate on it. For this reason, the default schema is -also called the "MonitorWare Schema". If you use it, you -can simply add </SPAN><A HREF="http://www.phplogcon.org/">phpLogCon, -a GPLed syslog web interface</A><SPAN LANG="en-US">, to your system -and have instant interactive access to your database. So there are -some benefits in using the provided schema.</SPAN></P> -<P LANG="en-US" CLASS="western">The schema definition is contained in -the file "createDB.sql". It comes with the rsyslog package -and one can be found for each supported database type (in the plugins -directory). Review it to check that the database name is acceptable -for you. Be sure to leave the table and field names unmodified, -because otherwise you need to customize rsyslogd's default sql -template, which we do not do in this paper. Then, run the script with -your favorite SQL client. Double-check that the table was -successfully created.</P> -<P LANG="en-US" CLASS="western">It is important to note that the -correct database encoding must be used so that the database will -accept strings independend of the string encoding. This is an -important part because it can not be guarantied that all syslog -messages will have a defined character encoding. This is especially -true if the rsyslog-Server will collect messages from different -clients and different products. -</P> -<P LANG="en-US" CLASS="western">For example PostgreSQL may refuse to -accept messages if you would set the database encoding to “UTF8†-while a client is sending invalid byte sequences for that encoding. -</P> -<P LANG="en-US" CLASS="western">Database support in rsyslog is -integrated via loadable plugin modules. To use the database -functionality, the database plugin must be enabled in the config file -BEFORE the first database table action is used. This is done by -placing the</P> -<BLOCKQUOTE CLASS="western"><CODE>$ModLoad ommysql</CODE></BLOCKQUOTE> -<P CLASS="western">directive at the begining of /etc/rsyslog.conf for -MySQL and</P> -<BLOCKQUOTE CLASS="western"><CODE>$ModLoad ompgsql</CODE></BLOCKQUOTE> -<P CLASS="western"><CODE><FONT FACE="Arial, sans-serif">for -PostgreSQL.</FONT></CODE></P> -<P LANG="en-US" CLASS="western"><FONT FACE="Arial, sans-serif">For -other databases, use their plugin name (e.g. omoracle).</FONT></P> -<P CLASS="western">Next, we need to tell rsyslogd to write data to -the database. As we use the default schema, we do NOT need to define -a template for this. We can use the hardcoded one (rsyslogd handles -the proper template linking). So all we need to do e.g. for MySQL is -add a simple selector line to /etc/rsyslog.conf:</P> -<BLOCKQUOTE CLASS="western"><CODE>*.* -:ommysql:database-server,database-name,database-userid,database-password</CODE></BLOCKQUOTE> -<P CLASS="western">Again, other databases have other selector names, -e.g. ":ompgsql:" instead of ":ommysql:". See the -output plugin's documentation for details.</P> -<P LANG="en-US" CLASS="western">In many cases, the database will run -on the local machine. In this case, you can simply use "127.0.0.1" -for <I>database-server</I>. This can be especially advisable, if you -do not need to expose the database to any process outside of the -local machine. In this case, you can simply bind it to 127.0.0.1, -which provides a quite secure setup. Of course, rsyslog also supports -remote database instances. In that case, use the remote server name -(e.g. mydb.example.com) or IP-address. The <I>database-name</I> by -default is "Syslog". If you have modified the default, use -your name here. <I>Database-userid</I> and <I>-password</I> are the -credentials used to connect to the database. As they are stored in -clear text in rsyslog.conf, that user should have only the least -possible privileges. It is sufficient to grant it INSERT privileges -to the systemevents table, only. As a side note, it is strongly -advisable to make the rsyslog.conf file readable by root only - if -you make it world-readable, everybody could obtain the password (and -eventually other vital information from it). In our example, let's -assume you have created a database user named "syslogwriter" -with a password of "topsecret" (just to say it bluntly: -such a password is NOT a good idea...). If your database is on the -local machine, your rsyslog.conf line might look like in this sample:</P> -<BLOCKQUOTE CLASS="western"><CODE>*.* -:ommysql:127.0.0.1,Syslog,syslogwriter,topsecret</CODE></BLOCKQUOTE> -<P CLASS="western">Save rsyslog.conf, restart rsyslogd - and you -should see syslog messages being stored in the "systemevents" -table!</P> -<P LANG="en-US" CLASS="western">The example line stores every message -to the database. Especially if you have a high traffic volume, you -will probably limit the amount of messages being logged. This is easy -to accomplish: the "write database" action is just a -regular selector line. As such, you can apply normal selector-line -filtering. If, for example, you are only interested in messages from -the mail subsystem, you can use the following selector line:</P> -<BLOCKQUOTE CLASS="western"><CODE>mail.* :ommysql:127.0.0.1,syslog,syslogwriter,topsecret</CODE></BLOCKQUOTE> -<P CLASS="western">Review the <A HREF="http://www.rsyslog.com/doc-rsyslog_conf.html">rsyslog.conf</A> -documentation for details on selector lines and their filtering.</P> -<P CLASS="western"><SPAN LANG="en-US"><B>You have now completed -everything necessary to store syslog messages to the a database.</B></SPAN><SPAN LANG="en-US"> -If you would like to try out a front-end, you might want to look at -</SPAN><A HREF="http://www.phplogcon.org/">phpLogCon</A><SPAN LANG="en-US">, -which displays syslog data in a browser. As of this writing, -phpLogCon is not yet a powerful tool, but it's open source, so it -might be a starting point for your own solution.</SPAN></P> -<H2 CLASS="western">On Reliability...</H2> -<P LANG="en-US" CLASS="western">Rsyslogd writes syslog messages -directly to the database. This implies that the database must be -available at the time of message arrival. If the database is offline, -no space is left or something else goes wrong - rsyslogd can not -write the database record. If rsyslogd is unable to store a message, -it performs one retry. This is helpful if the database server was -restarted. In this case, the previous connection was broken but a -reconnect immediately succeeds. However, if the database is down for -an extended period of time, an immediate retry does not help.</P> -<P CLASS="western"><SPAN LANG="en-US">Message loss in this scenario -can easily be prevented with rsyslog. All you need to do is run the -database writer in queued mode. This is now described in a generic -way and I do not intend to duplicate it here. So please be sure to -read "</SPAN><A HREF="http://www.rsyslog.com/doc-rsyslog_high_database_rate.html">Handling -a massive syslog database insert rate with Rsyslog</A><SPAN LANG="en-US">", -which describes the scenario and also includes configuration -examples.</SPAN></P> -<H2 CLASS="western">Conclusion</H2> -<P LANG="en-US" CLASS="western">With minimal effort, you can use -rsyslogd to write syslog messages to a database. You can even make it -absolutely fail-safe and protect it against database server downtime. -Once the messages are arrived there, you can interactively review and -analyze them. In practice, the messages are also stored in text files -for longer-term archival and the databases are cleared out after some -time (to avoid becoming too slow). If you expect an extremely high -syslog message volume, storing it in real-time to the database may -outperform your database server. In such cases, either filter out -some messages or used queued mode (which in general is recommended -with databases).</P> -<P LANG="en-US" CLASS="western">The method outlined in this paper -provides an easy to setup and maintain solution for most use cases.</P> -<H3 CLASS="western">Feedback Requested</H3> -<P CLASS="western">I would appreciate feedback on this paper. If you -have additional ideas, comments or find bugs, please <A HREF="mailto:rgerhards@adiscon.com">let -me know</A>.</P> -<H2 CLASS="western">References and Additional Material</H2> -<UL> - <LI><P CLASS="western" STYLE="margin-bottom: 0in"><A HREF="http://www.rsyslog.com/">www.rsyslog.com</A> - - the rsyslog site - </P> - <LI><P CLASS="western"><A HREF="http://www.monitorware.com/Common/en/Articles/performance-optimizing-syslog-server.php">Paper - on Syslog Server Optimization</A> - </P> -</UL> -<H2 CLASS="western">Revision History</H2> -<UL> - <LI><P CLASS="western" STYLE="margin-bottom: 0in">2005-08-02 * - <A HREF="http://www.adiscon.com/en/people/rainer-gerhards.php">Rainer - Gerhards</A> * initial version created - </P> - <LI><P CLASS="western" STYLE="margin-bottom: 0in">2005-08-03 * - <A HREF="http://www.adiscon.com/en/people/rainer-gerhards.php">Rainer - Gerhards</A> * added references to demo site - </P> - <LI><P CLASS="western" STYLE="margin-bottom: 0in">2007-06-13 * - <A HREF="http://www.adiscon.com/en/people/rainer-gerhards.php">Rainer - Gerhards</A> * removed demo site - was torn down because too - expensive for usage count - </P> - <LI><P CLASS="western" STYLE="margin-bottom: 0in">2008-02-21 * - <A HREF="http://www.adiscon.com/en/people/rainer-gerhards.php">Rainer - Gerhards</A> * updated reliability section, can now be done with - on-demand disk queues</P> - <LI><P CLASS="western">2008-02-28 * <A HREF="http://www.adiscon.com/en/people/rainer-gerhards.php">Rainer - Gerhards</A> * added info on other databases, updated syntax to more - recent one - </P> - <LI><P CLASS="western">2010-01-29 * Marc Schiffbauer * added some - PostgreSQL stuff, made wording more database generic, fixed some - typos</P> -</UL> -<H2 CLASS="western">Copyright</H2> -<P CLASS="western">Copyright (c) 2005-2010 <A HREF="http://www.adiscon.com/en/people/rainer-gerhards.php">Rainer -Gerhards</A>, Marc Schiffbauer and <A HREF="http://www.adiscon.com/en/">Adiscon</A>.</P> -<P CLASS="western"><BR><BR> -</P> -</BODY> -</HTML> diff --git a/doc/rsyslog_php_syslog_ng.html b/doc/rsyslog_php_syslog_ng.html deleted file mode 100644 index ed4d72f..0000000 --- a/doc/rsyslog_php_syslog_ng.html +++ /dev/null @@ -1,153 +0,0 @@ -<html><head> -<title>Using php-syslog-ng with rsyslog</title> -<meta name="KEYWORDS" content="syslog, php-syslog-ng, mysql, howto, rsyslog"> -</head> -<body> -<h1>Using php-syslog-ng with rsyslog</h1> - <P><small><i>Written by - <a href="http://www.adiscon.com/en/people/rainer-gerhards.php">Rainer - Gerhards</a> (2005-08-04)</i></small></P> -<p>Note: it has been reported that this guide is somewhat outdated. Please -use with care. Also, please note that <b>rsyslog's "native" web frontend is -<a href="http://www.phplogcon.org">phpLogCon</a></b>, which provides best integration -and a lot of extra functionality.</p> -<h2>Abstract</h2> -<p><i><b>In this paper, I describe how to use -<a href="http://www.vermeer.org/projects/php-syslog-ng">php-syslog-ng</a> with -<a href="http://www.rsyslog.com/">rsyslogd</a>. </b> Php-syslog-ng is a -popular web interface to syslog data. Its name stem from the fact that it -usually picks up its data from a database created by -<a href="http://www.balabit.com/products/syslog_ng/">syslog-ng</a> and some -helper scripts. However, there is nothing syslog-ng specific in the database. -With rsyslogd's high customizability, it is easy to write to a syslog-ng like -schema. I will tell you how to do this, enabling you to use php-syslog-ng as a -front-end for rsyslogd - or save the hassle with syslog-ng database -configuration and simply go ahead and use rsyslogd instead.</i></p> -<h2>Overall System Setup</h2> -<p>The setup is pretty straightforward. Basically, php-syslog-ng's interface to -the syslogd is the database. We use the schema that php-syslog-ng expects and -make rsyslogd write to it in its format. Because of this, php-syslog-ng does not -even know there is no syslog-ng present.</p> -<h2>Setting up the system</h2> -<p>For php-syslog-ng, you can follow its usual setup instructions. Just skip any -steps refering to configure syslog-ng. Make sure you create the database schema -in <a href="http://www.mysql.com/">MySQL</a>. As of this writing, the expected schema can be created via this script:</p> -<blockquote> - <code>CREATE DATABASE syslog<br> - !<br> - USE syslog<br> - !<br> - CREATE TABLE logs (<br> - host varchar(32) default NULL,<br> - facility varchar(10) default NULL,<br> - priority varchar(10) default NULL,<br> - level varchar(10) default NULL,<br> - tag varchar(10) default NULL,<br> - date date default NULL,<br> - time time default NULL,<br> - program varchar(15) default NULL,<br> - msg text,<br> - seq int(10) unsigned NOT NULL auto_increment,<br> - PRIMARY KEY (seq),<br> - KEY host (host),<br> - KEY seq (seq),<br> - KEY program (program),<br> - KEY time (time),<br> - KEY date (date),<br> - KEY priority (priority),<br> - KEY facility (facility)<br> - ) TYPE=MyISAM;</code> -</blockquote> -<p>Please note that at the time you are reading this paper, the schema might have changed. -Check for any differences. As we customize rsyslogd to the schema, it is vital -to have the correct one. If this paper is outdated, -<a href="mailto:rgerhards@adiscon.com">let me know</a> so that I can fix it.</p> -<p>Once this schema is created, we simply instruct rsyslogd to store received -data in it. I wont go into too much detail here. If you are interested in some -more details, you might find my paper "<a href="rsyslog_mysql.html">Writing -syslog messages to MySQL</a>" worth reading. For this article, we simply modify -<a href="rsyslog_conf.html">rsyslog.conf </a>so that it writes to the database. -That is easy. Just these two lines are needed:</p> -<blockquote> - <code><font color="green">$template syslog-ng,"insert into logs(host, facility, priority, tag, date, - time, msg) values ('%HOSTNAME%', %syslogfacility%, %syslogpriority%, - '%syslogtag%', '%timereported:::date-mysql%', '%timereported:::date-mysql%', - '%msg%')", SQL</font> <br> - <font color="red">*.* >mysql-server,syslog,user,pass;syslog-ng</font> - </code> -</blockquote> -<p>These are just <b>two</b> lines. I have color-coded them so that you see what -belongs together (the colors have no other meaning). The green line is the -actual SQL statement being used to take care of the syslog-ng schema. Rsyslogd -allows you to fully control the statement sent to the database. This allows you -to write to any database format, including your homegrown one (if you so desire). -Please note that there is a small inefficiency in our current usage: the - <code><font color="green">'%timereported:::date-mysql%'</font></code> -property is used for both the time and the date (if you wonder about what all -these funny characters mean, see the <a href="property_replacer.html">rsyslogd -property replacer manual</a>) . We could have extracted just the date and time -parts of the respective properties. However, this is more complicated and also -adds processing time to rsyslogd's processing (substrings must be extracted). So we take a full mysql-formatted timestamp and supply it to MySQL. The sql engine in turn -discards the unneeded part. It works pretty well. As of my understanding, the -inefficiency of discarding the unneeded part in MySQL is lower than the -effciency gain from using the full timestamp in rsyslogd. So it is most probably -the best solution.</p> -<p>Please note that rsyslogd knows two different timestamp properties: one is -timereported, used here. It is the timestamp from the message itself. Sometimes -that is a good choice, in other cases not. It depends on your environment. The other one is the timegenerated -property. This is the time when rsyslogd received the message. For obvious -reasons, that timestamp is consistent, even when your devices are in multiple -time zones or their clocks are off. However, it is not "the real thing". It's -your choice which one you prefer. If you prefer timegenerated ... simply use it -;)</p> -<p>The line in red tells rsyslogd which messages to log and where to store it. -The "*.*" selects all messages. You can use standard syslog selector line filters here if -you do not like to see everything in your database. The ">" tells -rsyslogd that a MySQL connection -must be established. Then, "mysql-server" is the name or IP address of the -server machine, "syslog" is the database name (default from the schema) and "user" -and "pass" are the logon credentials. Use a user with low privileges, insert into the -logs table is sufficient. "syslog-ng" is the template name and tells rsyslogd to -use the SQL statement shown above.</p> -<p>Once you have made the changes, all you need to do is restart -rsyslogd. Then, you should see syslog messages flow into your database - and -show up in php-syslog-ng.</p> -<h2>Conclusion</h2> -<P>With minumal effort, you can use php-syslog-ng together with rsyslogd. For -those unfamiliar with syslog-ng, this configuration is probably easier to set up -then switching to syslog-ng. For existing rsyslogd users, php-syslog-ng might be a nice -add-on to their logging infrastructure.</P> -<P>Please note that the <a href="http://www.monitorware.com/en/">MonitorWare family</a> (to which rsyslog belongs) also -offers a web-interface: <a href="http://www.phplogcon.org/">phpLogCon</a>. -From my point of view, obviously, <b>phpLogCon is the more natural choice for a web interface -to be used together with rsyslog</b>. It also offers superb functionality and provides, -for example,native display of Windows event log entries. -I have set up a <a href="http://demo.phplogcon.org/">demo server</a>., -You can have a peek at it -without installing anything.</P> -<h2>Feedback Requested</h2> -<P>I would appreciate feedback on this paper. If you have additional ideas, -comments or find bugs, please -<a href="mailto:rgerhards@adiscon.com">let me know</a>.</P> -<h2>References and Additional Material</h2> -<ul> - <li><a href="http://www.vermeer.org/projects/php-syslog-ng">php-syslog-ng</a></li> -</ul> -<h2>Revision History</h2> -<ul> - <li>2005-08-04 * - <a href="http://www.adiscon.com/en/people/rainer-gerhards.php">Rainer Gerhards</a> * - initial version created</li> -</ul> -<h2>Copyright</h2> -<p>Copyright (c) 2005 -<a href="http://www.adiscon.com/en/people/rainer-gerhards.php">Rainer Gerhards</a> -and <a href="http://www.adiscon.com/en/">Adiscon</a>.</p> -<p>Permission is granted to copy, distribute and/or modify this document under -the terms of the GNU Free Documentation License, Version 1.2 or any later -version published by the Free Software Foundation; with no Invariant Sections, -no Front-Cover Texts, and no Back-Cover Texts. A copy of the license can be -viewed at <a href="http://www.gnu.org/copyleft/fdl.html"> -http://www.gnu.org/copyleft/fdl.html</a>.</p> -</body> -</html> diff --git a/doc/rsyslog_queue_pointers.jpeg b/doc/rsyslog_queue_pointers.jpeg Binary files differdeleted file mode 100644 index 809dd44..0000000 --- a/doc/rsyslog_queue_pointers.jpeg +++ /dev/null diff --git a/doc/rsyslog_queue_pointers2.jpeg b/doc/rsyslog_queue_pointers2.jpeg Binary files differdeleted file mode 100644 index 2ad6011..0000000 --- a/doc/rsyslog_queue_pointers2.jpeg +++ /dev/null diff --git a/doc/rsyslog_recording_pri.html b/doc/rsyslog_recording_pri.html deleted file mode 100644 index abcadf2..0000000 --- a/doc/rsyslog_recording_pri.html +++ /dev/null @@ -1,134 +0,0 @@ -<html><head> -<title>Recording the Priority of Syslog Messages</title> -<meta name="KEYWORDS" content="syslog, mysql, syslog to mysql, howto"> -</head> -<body> -<h1>Recording the Priority of Syslog Messages</h1> - <P><small><i>Written by - <a href="http://www.adiscon.com/en/people/rainer-gerhards.php">Rainer - Gerhards</a> (2007-06-18)</i></small></P> -<h2>Abstract</h2> -<p><i><b>The so-called priority (PRI) is very important in syslog messages, -because almost all filtering in syslog.conf is based on it.</b> However, many -syslogds (including the Linux stock sysklogd) do not provide a way to record -that value. In this article, I'll give a brief overview of how PRI can be -written to a log file.</i></p> -<h2>Background</h2> -<p>The PRI value is a combination of so-called severity and facility. The -facility indicates where the message originated from (e.g. kernel, mail -subsystem) while the severity provides a glimpse of how important the message -might be (e.g. error or informational). Be careful with these values: they are -in no way consistent across applications (especially severity). However, they -still form the basis of most filtering in syslog.conf. For example, the -directive (aka "selector line)</p> -<p align="center"> -<code>mail.* /var/log/mail.log</code> -</p> -<p>means that messages with the mail facility should be stored to -/var/log/mail.log, no matter which severity indicator they have (that is telling -us the asterisk). If you set up complex conditions, it can be annoying to find -out which PRI value a specific syslog message has. Most stock syslogds do not -provide any way to record them.</p> -<h2>How is it done?</h2> -<p>With <a href="http://www.rsyslog.com/">rsyslog</a>, PRI recording is simple. -All you need is the correct template. Even if you do not use rsyslog on a regular -basis, it might be a handy tool for finding out the priority.</p> -<p>Rsyslog provides a flexible system to specify the output formats. It is -template-based. A template with the traditional syslog format looks as follows:</p> -<p align="center"> -<code>$template TraditionalFormat,"%timegenerated% %HOSTNAME% %syslogtag%%msg:::drop-last-lf%\n"</code> -</p> -<p>The part in quotes is the output formats. Things between percent-signs are -so-called <a href="property_replacer.html">messages properties</a>. They are replaced with the respective content -from the syslog message when output is written. Everything outside of the -percent signs is literal text, which is simply written as specified.</p> -<p>Thankfully, rsyslog provides message properties for the priority. These are -called "PRI", "syslogfacility" and "syslogpriority" (case is important!). They are numerical -values. Starting with rsyslog 1.13.4, there is also a property "pri-text", which -contains the priority in friendly text format (e.g. "syslog.info"). For the rest -of this article, I assume that you run version 1.13.4 or higher.</p> -<p>Recording the priority is now a simple matter of adding the respective field -to the template. It now looks like this:</p> -<p align="center"> -<code>$template TraditionalFormatWithPRI,"%pri-text%: %timegenerated% %HOSTNAME% %syslogtag%%msg:::drop-last-lf%\n"</code> -</p> -<p>Now we have the right template - but how to write it to a file? You probably -have a line like this in your syslog.conf:</p> -<p align="center"><code>*.* -/var/log/messages.log</code></p> -<p>It does not specify a template. Consequently, rsyslog uses the traditional -format. In order to use some other format, simply specify the template after the -semicolon:</p> -<p align="center"><code>*.* -/var/log/messages.log;TraditionalFormatWithPRI</code></p> -<p>That's all you need to do. There is one common pitfall: you need to define -the template before you use it in a selector line. Otherwise, you will receive -an error.</p> -<p>Once you have applied the changes, you need to restart rsyslogd. It -will then pick the new configuration.</p> -<h2>What if I do not want rsyslogd to be the standard syslogd?</h2> -<p>If you do not want to switch to rsyslog, you can still use it as a setup aid. -A little bit of configuration is required.</p> -<ol> - <li>Download, make and install rsyslog</li> - <li>copy your syslog.conf over to rsyslog.conf</li> - <li>add the template described above to it; select the file that should use - it</li> - <li>stop your regular syslog daemon for the time being</li> - <li>run rsyslogd (you may even do this interactively by calling it with the - -n additional option from a shell)</li> - <li>stop rsyslogd (press ctrl-c when running interactively)</li> - <li>restart your regular syslogd</li> -</ol> -<p>That's it - you can now review the priorities.</p> -<h2>Some Sample Data</h2> -<p>Below is some sample data created with the template specified above. Note the -priority recording at the start of each line.</p> -<p> -<code>kern.info: Jun 15 18:10:38 host kernel: PCI: Sharing IRQ 11 with 00:04.0<br> -kern.info: Jun 15 18:10:38 host kernel: PCI: Sharing IRQ 11 with 01:00.0<br> -kern.warn: Jun 15 18:10:38 host kernel: Yenta IRQ list 06b8, PCI irq11<br> -kern.warn: Jun 15 18:10:38 host kernel: Socket status: 30000006<br> -kern.warn: Jun 15 18:10:38 host kernel: Yenta IRQ list 06b8, PCI irq11<br> -kern.warn: Jun 15 18:10:38 host kernel: Socket status: 30000010<br> -kern.info: Jun 15 18:10:38 host kernel: cs: IO port probe 0x0c00-0x0cff: clean.<br> -kern.info: Jun 15 18:10:38 host kernel: cs: IO port probe 0x0100-0x04ff: excluding 0x100-0x107 0x378-0x37f 0x4d0-0x4d7<br> -kern.info: Jun 15 18:10:38 host kernel: cs: IO port probe 0x0a00-0x0aff: clean.<br> -local7.notice: Jun 15 18:17:24 host dd: 1+0 records out<br> -local7.notice: Jun 15 18:17:24 host random: Saving random seed: succeeded<br> -local7.notice: Jun 15 18:17:25 host portmap: portmap shutdown succeeded<br> -local7.notice: Jun 15 18:17:25 host network: Shutting down interface eth1: succeeded<br> -local7.notice: Jun 15 18:17:25 host network: Shutting down loopback interface: succeeded<br> -local7.notice: Jun 15 18:17:25 host pcmcia: Shutting down PCMCIA services: cardmgr<br> -user.notice: Jun 15 18:17:25 host /etc/hotplug/net.agent: NET unregister event not supported<br> -local7.notice: Jun 15 18:17:27 host pcmcia: modules.<br> -local7.notice: Jun 15 18:17:29 host rc: Stopping pcmcia: succeeded<br> -local7.notice: Jun 15 18:17:30 host rc: Starting killall: succeeded<br> -syslog.info: Jun 15 18:17:33 host [origin software="rsyslogd" swVersion="1.13.3" x-pid="2464"] exiting on signal 15.<br> -syslog.info: Jun 18 10:55:47 host [origin software="rsyslogd" swVersion="1.13.3" x-pid="2367"][x-configInfo udpReception="Yes" udpPort="514" tcpReception="Yes" tcpPort="1470"] restart<br> -user.notice: Jun 18 10:55:50 host rger: test<br> -syslog.info: Jun 18 10:55:52 host [origin software="rsyslogd" swVersion="1.13.3" x-pid="2367"] exiting on signal 2.</code></p> -<h2>Feedback Requested</h2> -<P>I would appreciate feedback on this paper. If you have additional ideas, -comments or find bugs, please -<a href="mailto:rgerhards@adiscon.com">let me know</a>.</P> -<h2>References and Additional Material</h2> -<ul> - <li><a href="http://www.rsyslog.com">www.rsyslog.com</a> - the rsyslog site</li> -</ul> -<h2>Revision History</h2> -<ul> - <li>2007-06-18 * - <a href="http://www.adiscon.com/en/people/rainer-gerhards.php">Rainer Gerhards</a> - * initial version created</li> -</ul> -<h2>Copyright</h2> -<p>Copyright (c) 2007 -<a href="http://www.adiscon.com/en/people/rainer-gerhards.php">Rainer Gerhards</a> -and <a href="http://www.adiscon.com/en/">Adiscon</a>.</p> -<p>Permission is granted to copy, distribute and/or modify this document under -the terms of the GNU Free Documentation License, Version 1.2 or any later -version published by the Free Software Foundation; with no Invariant Sections, -no Front-Cover Texts, and no Back-Cover Texts. A copy of the license can be -viewed at <a href="http://www.gnu.org/copyleft/fdl.html"> -http://www.gnu.org/copyleft/fdl.html</a>.</p> -</body> -</html> diff --git a/doc/rsyslog_reliable_forwarding.html b/doc/rsyslog_reliable_forwarding.html deleted file mode 100644 index d04d9ea..0000000 --- a/doc/rsyslog_reliable_forwarding.html +++ /dev/null @@ -1,152 +0,0 @@ -<html><head> -<title>Reliable Forwarding of syslog Messages (via plain TCP syslog)</title> -</head> -<body> -<h1>Reliable Forwarding of syslog Messages with Rsyslog</h1> - <P><small><i>Written by - <a href="http://www.gerhards.net/rainer">Rainer - Gerhards</a> (2008-06-27)</i></small></P> -<h2>Abstract</h2> -<p><i><b>In this paper, I describe how to forward -<a href="http://www.monitorware.com/en/topics/syslog/">syslog</a> - - messages (quite) reliable to a central rsyslog server.</b> -This depends on rsyslog being installed on the client system and -it is recommended to have it installed on the server system. Please note -that industry-standard -<a href="http://blog.gerhards.net/2008/04/on-unreliability-of-plain-tcp-syslog.html">plain TCP syslog protocol is not fully reliable</a> -(thus the "quite reliable"). If you need a truely reliable solution, you need -to look into RELP (natively supported by rsyslog).</i></p> - -<h2>The Intention</h2> -<p>Whenever two systems talk over a network, something can go wrong. -For example, the communications link may go down, or a client or server may abort. -Even in regular cases, the server may be offline for a short period of time -because of routine maintenance. -<p>A logging system should be capable of avoiding message loss in situations where the -server is not reachable. To do so, unsent data needs to be buffered at the client while the -server is offline. Then, once the server is up again, this data is to be sent. -<p>This can easily be acomplished by rsyslog. In rsyslog, every action runs on its own queue -and each queue can be set to buffer data if the action is not ready. Of course, -you must be able to detect that "the action is not ready", which means the remote -server is offline. This can be detected with plain TCP syslog and RELP, but not with UDP. -So you need to use either of the two. In this howto, we use plain TCP syslog. -<p>Please note that we are using rsyslog-specific features. The are required on the -client, but not on the server. So the client system must run rsyslog (at least version 3.12.0), while on the -server another syslogd may be running, as long as it supports plain tcp syslog. -<p><b>The rsyslog queueing subsystem tries to buffer to memory. So even if the -remote server goes -offline, no disk file is generated.</b> File on disk are created only if there is -need to, for example if rsyslog runs out of (configured) memory queue space or needs -to shutdown (and thus persist yet unsent messages). Using main memory and going to the -disk when needed is a huge performance benefit. You do not need to care about it, -because, all of it is handled automatically and transparently by rsyslog.</p> -<h2>How To Setup</h2> -<p>First, you need to create a working directory for rsyslog. This is where it -stores its queue files (should need arise). You may use any location on your -local system. -<p>Next, you need to do is instruct rsyslog to use a -disk queue and then configure your action. There is nothing else to do. With the -following simple config file, you forward anything you receive to a remote server -and have buffering applied automatically when it goes down. This must be done on the -client machine.</p> -<textarea rows="9" cols="80"> -$ModLoad imuxsock # local message reception - -$WorkDirectory /rsyslog/work # default location for work (spool) files - -$ActionQueueType LinkedList # use asynchronous processing -$ActionQueueFileName srvrfwd # set file name, also enables disk mode -$ActionResumeRetryCount -1 # infinite retries on insert failure -$ActionQueueSaveOnShutdown on # save in-memory data if rsyslog shuts down -*.* @@server:port -</textarea> -<p>The port given above is optional. It may not be specified, in which case you only -provide the server name. The "$ActionQueueFileName" is used to create queue files, should need -arise. This value must be unique inside rsyslog.conf. No two rules must use the same queue file. -Also, for obvious reasons, it must only contain those characters that can be used inside a -valid file name. Rsyslog possibly adds some characters in front and/or at the end of that name -when it creates files. So that name should not be at the file size name length limit (which -should not be a problem these days). -<p>Please note that actual spool files are only created if the remote server is down -<b>and</b> there is no more space in the in-memory queue. By default, a short failure -of the remote server will never result in the creation of a disk file as a couple of -hundered messages can be held in memory by default. [These parameters can be fine-tuned. However, -then you need to either fully understand how the queue works -(<a href="http://www.rsyslog.com/doc-queues.html">read elaborate doc</a>) or -use <a href="http://www.rsyslog.com/doc-professional_support.html">professional services</a> -to have it done based on -your specs ;) - what that means is that fine-tuning queue parameters is far from -being trivial...] -<p>If you would like to test if your buffering scenario works, you need to -stop, wait a while and restart you central server. Do <b>not</b> watch for files being created, -as this usually does not happen and never happens immediately. - -<h3>Forwarding to More than One Server</h3> -<p>If you have more than one server you would like to forward to, that's quickly done. -Rsyslog has no limit on the number or type of actions, so you can define as many targets -as you like. What is important to know, however, is that the full set of directives make -up an action. So you can not simply add (just) a second forwarding rule, but need to -duplicate the rule configuration as well. Be careful that you use different queue -file names for the second action, else you will mess up your system. -<p>A sample for forwarding to two hosts looks like this: -<p> -<textarea rows="20" cols="80"> -$ModLoad imuxsock # local message reception - -$WorkDirectory /rsyslog/work # default location for work (spool) files - -# start forwarding rule 1 -$ActionQueueType LinkedList # use asynchronous processing -$ActionQueueFileName srvrfwd1 # set file name, also enables disk mode -$ActionResumeRetryCount -1 # infinite retries on insert failure -$ActionQueueSaveOnShutdown on # save in-memory data if rsyslog shuts down -*.* @@server1:port -# end forwarding rule 1 - -# start forwarding rule 2 -$ActionQueueType LinkedList # use asynchronous processing -$ActionQueueFileName srvrfwd2 # set file name, also enables disk mode -$ActionResumeRetryCount -1 # infinite retries on insert failure -$ActionQueueSaveOnShutdown on # save in-memory data if rsyslog shuts down -*.* @@server2 -# end forwarding rule 2 -</textarea> -<p>Note the filename used for the first rule it is "srvrfwd1" and for the second it -is "srvrfwd2". I have used a server without port name in the second forwarding rule. -This was just to illustrate how this can be done. You can also specify a port there -(or drop the port from server1). -<p>When there are multiple action queues, they all work independently. Thus, if server1 -goes down, server2 still receives data in real-time. The client will <b>not</b> block -and wait for server1 to come back online. Similarily, server1's operation will not -be affected by server2's state. - -<h2>Some Final Words on Reliability ...</h2> -<p>Using plain TCP syslog provides a lot of reliability over UDP syslog. However, -plain TCP syslog is <b>not</b> a fully reliable transport. In order to get full reliability, -you need to use the RELP protocol. -<p>Folow the next link to learn more about -<a href="http://blog.gerhards.net/2008/04/on-unreliability-of-plain-tcp-syslog.html">the -problems you may encounter with plain tcp syslog</a>. -<h3>Feedback requested</h3> -<P>I would appreciate feedback on this tutorial. If you have additional ideas, -comments or find bugs (I *do* bugs - no way... ;)), please -<a href="mailto:rgerhards@adiscon.com">let me know</a>.</P> -<h2>Revision History</h2> -<ul> - <li>2008-06-27 * - <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> * Initial Version created</li> -</ul> -<h2>Copyright</h2> -<p>Copyright (c) 2008 -<a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and -<a href="http://www.adiscon.com/en/">Adiscon</a>.</p> -<p> Permission is granted to copy, distribute and/or modify this document - under the terms of the GNU Free Documentation License, Version 1.2 - or any later version published by the Free Software Foundation; - with no Invariant Sections, no Front-Cover Texts, and no Back-Cover - Texts. A copy of the license can be viewed at -<a href="http://www.gnu.org/copyleft/fdl.html"> -http://www.gnu.org/copyleft/fdl.html</a>.</p> -</body> -</html> diff --git a/doc/rsyslog_secure_tls.html b/doc/rsyslog_secure_tls.html deleted file mode 100644 index b15e5a4..0000000 --- a/doc/rsyslog_secure_tls.html +++ /dev/null @@ -1,127 +0,0 @@ -<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> -<html><head><title>TLS-protected syslog: recommended scenario</title> -</head> -<body> - -<h1>Encrypting Syslog Traffic with TLS (SSL)</h1> -<p><small><i>Written by <a href="http://www.adiscon.com/en/people/rainer-gerhards.php">Rainer -Gerhards</a> (2008-06-17)</i></small></p> -<ul> -<li><a href="rsyslog_secure_tls.html">Overview</a> -<li><a href="tls_cert_scenario.html">Sample Scenario</a> -<li><a href="tls_cert_ca.html">Setting up the CA</a> -<li><a href="tls_cert_machine.html">Generating Machine Certificates</a> -<li><a href="tls_cert_server.html">Setting up the Central Server</a> -<li><a href="tls_cert_client.html">Setting up syslog Clients</a> -<li><a href="tls_cert_udp_relay.html">Setting up the UDP syslog relay</a> -<li><a href="tls_cert_summary.html">Wrapping it all up</a> -<li><a href="tls_cert_errmsgs.html">Frequently seen Error Messages</a> -</ul> - -<h2>Overview</h2> -<p>This document describes a secure way to set up rsyslog TLS. A secure logging -environment requires more than just encrypting the transmission channel. This document -provides one possible way to create such a secure system. -<p>Rsyslog's TLS authentication can be used very flexible and thus supports a -wide range of security policies. This section tries to give some advise on a -scenario that works well for many environments. However, it may not be suitable -for you - please assess you security needs before using the recommendations -below. Do not blame us if it doesn't provide what you need ;)</p> -<p>Our policy offers these security benefits:</p> -<ul> - <li>syslog messages are encrypted while traveling on the wire</li> - <li>the syslog sender authenticates to the syslog receiver; thus, the - receiver knows who is talking to it</li> - <li>the syslog receiver authenticates to the syslog sender; thus, the sender - can check if it indeed is sending to the expected receiver</li> - <li>the mutual authentication prevents man-in-the-middle attacks</li> -</ul> -<p>Our secrity goals are achived via public/private key security. As such, it is -vital that private keys are well protected and not accessible to third parties. -<span style="float: left"> -<script type="text/javascript"><!-- -google_ad_client = "pub-3204610807458280"; -/* rsyslog doc inline */ -google_ad_slot = "5958614527"; -google_ad_width = 125; -google_ad_height = 125; -//--> -</script> -<script type="text/javascript" -src="http://pagead2.googlesyndication.com/pagead/show_ads.js"> -</script> -</span> -If private keys have become known to third parties, the system does not provide -any security at all. Also, our solution bases on X.509 certificates and a (very -limited) chain of trust. We have one instance (the CA) that issues all machine -certificates. The machine certificate indentifies a particular machine. hile in -theory (and practice), there could be several "sub-CA" that issues machine -certificates for a specific adminitrative domain, we do not include this in our -"simple yet secure" setup. If you intend to use this, rsyslog supports it, but -then you need to dig a bit more into the documentation (or use the forum to ask). -In general, if you depart from our simple model, you should have good reasons -for doing so and know quite well what you are doing - otherwise you may -compromise your system security.</p> -<p>Please note that security never comes without effort. In the scenario -described here, we have limited the effort as much as possible. What remains is -some setup work for the central CA, the certificate setup for each machine as -well as a few configuration commands that need to be applied to all of them. -Proably the most important limiting factor in our setup is that all senders and -receivers must support IETF's syslog-transport-tls standard (which is not -finalized yet). We use mandatory-to-implement technology, yet you may have -trouble finding all required features in some implementations. More often, -unfortunately, you will find that an implementation does not support the -upcoming IETF standard at all - especially in the "early days" (starting May -2008) when rsyslog is the only implementation of said standard.</p> -<p>Fortunately, rsyslog supports allmost every protocol that is out there in the -syslog world. So in cases where transport-tls is not available on a sender, we -recommend to use rsyslog as the initial relay. In that mode, the not-capabe -sender sends to rsyslog via another protocol, which then relays the message via -transport-tls to either another interim relay or the final destination (which, -of course, must by transport-tls capable). In such a scenario, it is best to try -see what the sender support. Maybe it is possible to use industry-standard plain -tcp syslog with it. Often you can even combine it with stunnel, which then, too, -enables a secure delivery to the first rsyslog relay. If all of that is not -possible, you can (and often must...) resort to UDP. Even though this is now -lossy and insecure, this is better than not having the ability to listen to that -device at all. It may even be reasonale secure if the uncapable sender and the -first rsyslog relay communicate via a private channel, e.g. a dedicated network -link.</p> -<p>One final word of caution: transport-tls protects the connection between the -sender and the receiver. It does not necessarily protect against attacks that -are present in the message itself. Especially in a relay environment, the -message may have been originated from a malicious system, which placed invalid -hostnames and/or other content into it. If there is no provisioning against such -things, these records may show up in the receivers' repository. -transport-tls -does not protect against this (but it may help, properly used). Keep in mind -that syslog-transport-tls provides hop-by-hop security. It does not provide -end-to-end security and it does not authenticate the message itself (just the -last sender).</p> -<h3>A very quick Intro</h3> -<p>If you'd like to get all information very rapidly, the graphic below contains -everything you need to know (from the certificate perspective) in a very condensed -manner. It is no surprise if the graphic puzzles you. In this case, <a href="tls_cert_scenario.html">simply read on</a> -for full instructions. -<p> -<img align="center" alt="TLS/SSL protected syslog" src="tls_cert.jpg"> -<h3>Feedback requested</h3> -<p>I would appreciate feedback on this tutorial. If you have -additional ideas, comments or find bugs (I *do* bugs - no way... ;)), -please -<a href="mailto:rgerhards@adiscon.com">let me know</a>.</p> -<h2>Revision History</h2> -<ul> -<li>2008-06-06 * <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> * Initial Version created</li> -<li>2008-06-18 * <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> * Greatly enhanced and modularized the doc</li> -</ul> -<h2>Copyright</h2> -<p>Copyright (c) 2008 <a href="http://www.adiscon.com/en/people/rainer-gerhards.php">Rainer -Gerhards</a> and -<a href="http://www.adiscon.com/en/">Adiscon</a>.</p> -<p> Permission is granted to copy, distribute and/or modify this -document under the terms of the GNU Free Documentation License, Version -1.2 or any later version published by the Free Software Foundation; -with no Invariant Sections, no Front-Cover Texts, and no Back-Cover -Texts. A copy of the license can be viewed at -<a href="http://www.gnu.org/copyleft/fdl.html">http://www.gnu.org/copyleft/fdl.html</a>.</p> -</body></html> diff --git a/doc/rsyslog_stunnel.html b/doc/rsyslog_stunnel.html deleted file mode 100644 index f0c0b3a..0000000 --- a/doc/rsyslog_stunnel.html +++ /dev/null @@ -1,249 +0,0 @@ -<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> -<html><head> -<a href="features.html">back</a> - -<title>SSL Encrypting syslog with stunnel</title><meta name="KEYWORDS" content="syslog encryption, rsyslog, stunnel, secure syslog, tcp, reliable, howto, ssl"></head><body> -<h1>SSL Encrypting Syslog with Stunnel</h1> - <p><small><i>Written by - <a href="http://www.adiscon.com/en/people/rainer-gerhards.php">Rainer - Gerhards</a> (2005-07-22)</i></small></p> -<h2>Abstract</h2> -<p><i><b>In this paper, I describe how to encrypt <a href="http://www.monitorware.com/en/topics/syslog/">syslog</a> -messages on the network.</b> Encryption -is vital to keep the confidiental content of syslog messages secure. I describe the overall -approach and provide an HOWTO do it with the help of -<a href="http://www.rsyslog.com">rsyslogd</a> and <a href="http://www.stunnel.org">stunnel</a>.</i></p><p><span style="font-weight: bold;">Please note that starting with rsyslog 3.19.0, </span><a style="font-weight: bold;" href="rsyslog_tls.html">rsyslog provides native TLS/SSL encryption</a><span style="font-weight: bold;"> <span style="font-style: italic;">without</span> the need of stunnel. </span>I -strongly recomend to use that feature instead of stunnel. The stunnel -documentation here is mostly provided for backwards compatibility. New -deployments are advised to use native TLS mode.<i></i></p> -<h2>Background</h2> -<p><b>Syslog is a -clear-text protocol. That means anyone with a sniffer can have -a peek at your data.</b> In some environments, this is no problem at all. In -others, it is a huge setback, probably even preventing deployment of syslog -solutions. Thankfully, there is an easy way to encrypt syslog communication. I -will describe one approach in this paper.</p> -<p>The most straightforward solution would be that the syslogd itself encrypts -messages. Unfortuantely, encryption is only standardized in -<a href="http://www.monitorware.com/Common/en/glossary/rfc3195.php">RFC 3195</a>. But there -is currently no syslogd that implements RFC 3195's encryption features, -so this route leads to nothing. Another approach would be to use vendor- or -project-specific syslog extensions. There are a few around, but the problem here -is that they have compatibility issues. However, there is one surprisingly easy -and interoperable solution: though not standardized, many vendors and projects -implement plain tcp syslog. In a nutshell, plain tcp syslog is a mode where -standard syslog messages are transmitted via tcp and records are separated by -newline characters. This mode is supported by all major syslogd's (both on Linux/Unix -and Windows) as well as log sources (for example, -<a href="http://www.eventreporter.com/en/">EventReporter</a> for Windows -Event Log forwarding). Plain tcp syslog offers reliability, but it does not -offer encryption in itself. However, since it operates on a tcp stream, it is now easy -to add encryption. There are various ways to do that. In this paper, I will -describe how it is done with stunnel (an -other alternative would be <a href="http://en.wikipedia.org/wiki/IPSec">IPSec</a>, for example).</p> -<p>Stunnel is open source and it is available both for Unix/Linux and Windows. -It provides a way to - use ssl communication for any non-ssl aware client and server - in this case, - our syslogd.</p> - <p>Stunnel works much like a wrapper. Both on the client and on the server machine, - tunnel portals are created. The non-ssl aware client and server software is - configured to not directly talk to the remote partner, but to the local - (s)tunnel portal instead. Stunnel, in turn, takes the data received from the - client, encrypts it via ssl, sends it to the remote tunnel portal and that - remote portal sends it to the recipient process on the remote machine. The - transfer to the portals is done via unencrypted communication. As such, - it is vital that - the portal and the respective program that is talking to it are on the same - machine, otherwise data would travel partly unencrypted. Tunneling, as done by stunnel, - requires connection oriented communication. This is why you need to use - tcp-based syslog. As a side-note, you can also encrypt a plain-text RFC - 3195 session via stunnel, though this definitely is not what the - protocol designers had on their mind ;)</p> -<p>In the rest of this document, I assume that you use rsyslog on both the -client and the server. For the samples, I use <a href="http://www.debian.org/">Debian</a>. -Interestingly, there are -some annoying differences between stunnel implementations. For example, on -Debian a comment line starts with a semicolon (';'). On -<a href="http://www.redhat.com">Red Hat</a>, it starts with -a hash sign ('#'). So you need to watch out for subtle issues when setting up -your system.</p> -<h2>Overall System Setup</h2> -<p>In ths paper, I assume two machines, one named "client" and the other named "server". -It is obvious that, in practice, you will probably have multiple clients but -only one server. Syslog traffic shall be transmitted via stunnel over the -network. Port 60514 is to be used for that purpose. The machines are set up as -follows:</p> -<p><b>Client</b></p> -<ul> - <li>rsyslog forwards message to stunnel local portal at port 61514</li> - <li>local stunnel forwards data via the network to port 60514 to its remote - peer</li> -</ul> -<p><b>Server</b></p> -<ul> - <li>stunnel listens on port 60514 to connections from its client peers</li> - <li>all connections are forwarded to the locally-running rsyslog listening - at port 61514</li> -</ul> -<h2>Setting up the system</h2> -<p>For Debian, you need the "stunnel4" package. The "stunnel" package is the -older 3.x release, which will not support the configuration I describe below. -Other distributions might have other names. For example, on Red Hat it is just "stunnel". -Make sure that you install the appropriate package on both the client and the -server. It is also a good idea to check if there are updates for either stunnel -or openssl (which stunnel uses) - there are often security fixes available and -often the latest fixes are not included in the default package.</p> -<p>In my sample setup, I use only the bare minimum of options. For example, I do -not make the server check client cerficiates. Also, I do not talk much about -certificates at all. If you intend to really secure your system, you should -probably learn about certificates and how to manage and deploy them. This is -beyond the scope of this paper. For additional information, -<a href="http://www.stunnel.org/faq/certs.html"> -http://www.stunnel.org/faq/certs.html</a> is a good starting point.</p> -<p>You also need to install rsyslogd on both machines. Do this before starting -with the configuration. You should also familarize yourself with its -configuration file syntax, so that you know which actions you can trigger with -it. Rsyslogd can work as a drop-in replacement for stock -<a href="http://www.infodrom.org/projects/sysklogd/">sysklogd</a>. So if you know -the standard syslog.conf syntax, you do not need to learn any more to follow -this paper.</p> -<h3>Server Setup</h3> -<p>At the server, you need to have a digital certificate. That certificate -enables SSL operation, as it provides the necessary crypto keys being used to -secure the connection. Many versions of stunnel come with a default certificate, -often found in /etc/stunnel/stunnel.pem. If you have it, it is good for testing -only. If you use it in production, it is very easy to break into your secure -channel as everybody is able to get hold of your private key. I didn't find an -stunnel.pem on my Debian machine. I guess the Debian folks removed it because of -its insecurity.</p> -<p>You can create your own certificate with a simple openssl tool - you need to -do it if you have none and I highly recommend to create one in any case. To -create it, cd to /etc/stunnel and type:</p> -<p></p><blockquote><code>openssl req -new -x509 -days 3650 -nodes -out -stunnel.pem -keyout stunnel.pem</code></blockquote><p></p> -<p>That command will ask you a number of questions. Provide some answer for -them. If you are unsure, read -<a href="http://www.stunnel.org/faq/certs.html"> -http://www.stunnel.org/faq/certs.html</a>. After the command has finished, you -should have a usable stunnel.pem in your working directory.</p> -<p>Next is to create a configuration file for stunnel. It will direct stunnel -what to do. You can used the following basic file:</p> -<p></p><blockquote><code></code><pre>; Certificate/key is needed in server mode<br>cert = /etc/stunnel/stunnel.pem<br><br><i>; Some debugging stuff useful for troubleshooting<br>debug = 7<br>foreground=yes</i> - -[ssyslog] -accept = 60514 -connect = 61514</pre> -</blockquote><p></p> -<p>Save this file to e.g. /etc/stunnel/syslog-server.conf. Please note that the -settings in <i>italics</i> are for debugging only. They run stunnel -with a lot of debug information in the foreground. This is very valuable while -you setup the system - and very useless once everything works well. So be sure -to remove these lines when going to production.</p> -<p>Finally, you need to start the stunnel daemon. Under Debian, this is done via -"stunnel /etc/stunnel/syslog.server.conf". If you have enabled the debug -settings, you will immediately see a lot of nice messages.</p> -<p>Now you have stunnel running, but it obviously unable to talk to rsyslog - -because it is not yet running. If not already done, configure it so that it does -everything you want. If in doubt, you can simply copy /etc/syslog.conf to /etc/rsyslog.conf -and you probably have what you want. The really important thing in rsyslogd -configuration is that you must make it listen to tcp port 61514 (remember: this -is where stunnel send the messages to). Thankfully, this is easy to achive: just -add "-t 61514" to the rsyslogd startup options in your system startup script. -After done so, start (or restart) rsyslogd.</p> -<p>The server should now be fully operational.</p> -<h3>Client Setup</h3> -<p>The client setup is simpler. Most importantly, you do not need a certificate -(of course, you can use one if you would like to authenticate the client, but -this is beyond the scope of this paper). So the basic thing you need to do is -create the stunnel configuration file.</p> -<p></p><blockquote><code></code><pre><i>; Some debugging stuff useful for troubleshooting<br>debug = 7<br>foreground=yes</i> - -<b>client=yes</b> - -[ssyslog] -accept = 127.0.0.1:61514 -connect = <font color="#ff0000">192.0.2.1</font>:60514<br></pre> -</blockquote><p></p> -<p>Again, the text in <i>italics</i> is for debugging purposes only. I suggest -you leave it in during your initial testing and then remove it. The most -important difference to the server configuration outlined above is the "client=yes" -directive. It is what makes this stunnel behave like a client. The accept -directive binds stunnel only to the local host, so that it is protected from -receiving messages from the network (somebody might fake to be the local sender). -The address "192.0.2.1" is the address of the server machine. You must change it -to match your configuration. Save this file to /etc/stunnel/syslog-client.conf.</p> -<p>Then, start stunnel via "stunnel4 /etc/stunnel/syslog-client.conf". Now -you should see some startup messages. If no errors appear, you have a running -client stunnel instance.</p> -<p>Finally, you need to tell rsyslogd to send data to the remote host. In stock -syslogd, you do this via the "@host" forwarding directive. The same works with -rsyslog, but it suppports extensions to use tcp. Add the following line to your -/etc/rsyslog.conf:</p> -<p></p><blockquote><code></code><pre>*.* @<font color="#ff0000">@</font>127.0.0.1:61514<br></pre> -</blockquote><i><p></p> - -</i> - -<p>Please note the double at-sign (@@). This is no typo. It tells rsyslog to use -tcp instead of udp delivery. In this sample, all messages are forwarded to the -remote host. Obviously, you may want to limit this via the usual rsyslog.conf -settings (if in doubt, use man rsyslog.con).</p> -<p>You do not need to add any special startup settings to rsyslog on the client. -Start or restart rsyslog so that the new configuration setting takes place.</p> -<h3>Done</h3> -<p>After following these steps, you should have a working secure syslog -forwarding system. To verify, you can type "logger test" or a similar smart -command on the client. It should show up in the respective server log file. If -you dig out you sniffer, you should see that the traffic on the wire is actually -protected. In the configuration use above, the two stunnel endpoints should be -quite chatty, so that you can follow the action going on on your system.</p> -<p>If you have only basic security needs, you can probably just remove the debug -settings and take the rest of the configuration to production. If you are -security-sensitve, you should have a look at the various stunnel settings that -help you further secure the system.</p> -<h2>Preventing Systems from talking directly to the rsyslog Server</h2> -<p>It is possible that remote systems (or attackers) talk to the rsyslog server -by directly connecting to its port 61514. Currently (July of 2005), rsyslog does -not offer the ability to bind to the local host, only. This feature is planned, -but as long as it is missing, rsyslog must be protected via a firewall. This can -easily be done via e.g iptables. Just be sure not to forget it.</p> -<h2>Conclusion</h2> -<p>With minumal effort, you can set up a secure logging infrastructure employing -ssl encrypted syslog message transmission. As a side note, you also have the -benefit of reliable tcp delivery which is far less prone to message loss than -udp.</p> -<h3>Feedback requested</h3> -<p>I would appreciate feedback on this tutorial. If you have additional ideas, -comments or find bugs (I *do* bugs - no way... ;)), please -<a href="mailto:rgerhards@adiscon.com">let me know</a>.</p> -<h2>Revision History</h2> -<ul> - <li>2005-07-22 * - <a href="http://www.adiscon.com/en/people/rainer-gerhards.php">Rainer Gerhards</a> * Initial Version created</li> - <li>2005-07-26 * - <a href="http://www.adiscon.com/en/people/rainer-gerhards.php">Rainer Gerhards</a> * Some text brush-up, hyperlinks added</li> - <li>2005-08-03 * - <a href="http://www.adiscon.com/en/people/rainer-gerhards.php">Rainer Gerhards</a> - * license added</li><li>2008-05-05 * <a href="http://www.adiscon.com/en/people/rainer-gerhards.php">Rainer Gerhards</a> - * updated to reflect native TLS capability of rsyslog 3.19.0 and above</li> -</ul> -<h2>Copyright</h2> -<p>Copyright (c) 2008 <a href="http://www.adiscon.com/en/people/rainer-gerhards.php">Rainer Gerhards</a> and -<a href="http://www.adiscon.com/en/">Adiscon</a>.</p> -<p> Permission is granted to copy, distribute and/or modify this document - under the terms of the GNU Free Documentation License, Version 1.2 - or any later version published by the Free Software Foundation; - with no Invariant Sections, no Front-Cover Texts, and no Back-Cover - Texts. A copy of the license can be viewed at -<a href="http://www.gnu.org/copyleft/fdl.html"> -http://www.gnu.org/copyleft/fdl.html</a>.</p> -<p>[<a href="manual.html">manual index</a>] -[<a href="rsyslog_conf.html">rsyslog.conf</a>] -[<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> -<p><font size="2">This documentation is part of the -<a href="http://www.rsyslog.com/">rsyslog</a> project.<br> -Copyright © 2008 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and -<a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL -version 2 or higher.</font></p> - -</body></html> diff --git a/doc/rsyslog_tls.html b/doc/rsyslog_tls.html deleted file mode 100644 index 286660d..0000000 --- a/doc/rsyslog_tls.html +++ /dev/null @@ -1,303 +0,0 @@ -<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> -<html><head><title>TLS (SSL) Encrypting syslog</title> -<a href="features.html">back</a> - -<meta name="KEYWORDS" content="syslog encryption, rsyslog, secure syslog, tcp, reliable, howto, ssl, tls"> -</head> -<body> -<h1>Encrypting Syslog Traffic with TLS (SSL)</h1> -<p><small><i>Written by <a href="http://www.adiscon.com/en/people/rainer-gerhards.php">Rainer -Gerhards</a> (2008-05-06)</i></small></p> -<h2>Abstract</h2> -<p><i><b>In this paper, I describe how to encrypt <a href="http://www.monitorware.com/en/topics/syslog/">syslog</a> -messages on the network.</b> Encryption -is vital to keep the confidiental content of syslog messages secure. I -describe the overall -approach and provide an HOWTO do it with <a href="http://www.rsyslog.com">rsyslog's</a> TLS -features. </i></p> -<p>Please -note that TLS is the more secure successor of SSL. While people often -talk about "SSL encryption" they actually mean "TLS encryption". So -don't look any further if you look for how to SSL-encrypt syslog. You -have found the right spot.</p> -<p>This is a quick guide. There is a more elaborate guide currently -under construction which provides a much more secure environment. It -is highly recommended to -<a href="rsyslog_secure_tls.html">at least have a look at it</a>. -<h2>Background</h2> -<p><b>Traditional syslog is a clear-text protocol. That -means anyone with a sniffer can have a peek at your data.</b> In -some environments, this is no problem at all. In others, it is a huge -setback, probably even preventing deployment of syslog solutions. -Thankfully, there are easy ways to encrypt syslog -communication. </p> -The traditional approach involves <a href="rsyslog_stunnel.html">running -a wrapper like stunnel around the syslog session</a>. This works -quite well and is in widespread use. However, it is not thightly -coupled with the main syslogd and some, even severe, problems can -result from this (follow a mailing list thread that describes <a href="http://lists.adiscon.net/pipermail/rsyslog/2008-March/000580.html">total -loss of syslog messages due to stunnel mode</a> and the <a href="http://rgerhards.blogspot.com/2008/04/on-unreliability-of-plain-tcp-syslog.html">unreliability -of TCP syslog</a>). -<p><a href="gssapi.html">Rsyslog supports syslog via -GSSAP</a>I since long to overcome these limitatinos. However, -syslog via GSSAPI is a rsyslog-exclusive transfer mode and it requires -a proper Kerberos environment. As such, it isn't a really universal -solution. The <a href="http://www.ietf.org/">IETF</a> -has begun standardizing syslog over plain tcp over -TLS for a while now. While I am not fully satisfied with the results so -far, this obviously has the potential to become the long-term -solution. The Internet Draft in question, syslog-transport-tls has been -dormant for some time but is now (May of 2008) again being worked on. I -expect it to turn into a RFC within the next 12 month (but don't take -this for granted ;)). I didn't want to wait for it, because there -obviously is need for TLS syslog right now (and, honestly, I have -waited long enough...). Consequently, I have -implemented the current draft, with some interpretations I made (there -will be a compliance doc soon). So in essence, a TLS-protected syslog -transfer mode is available right now. As a side-note, Rsyslog -is the world's first -implementation of syslog-transport-tls.</p> -<p>Please note that in theory it should be compatible with other, -non IETF syslog-transport-tls implementations. If you would like to run -it with something else, please let us know so that we can create a -compatibility list (and implement compatbility where it doesn't yet -exist). </p> -<h2>Overall System Setup</h2> -<p>Encryption requires a reliable stream. So It will not work -over UDP syslog. In rsyslog, network transports utilize a so-called -"network stream layer" (netstream for short). This layer provides a -unified view of the transport to the application layer. The plain TCP -syslog sender and receiver are the upper layer. The driver layer -currently consists of the "ptcp" and "gtls" library plugins. "ptcp" -stands for "plain tcp" and is used for unencrypted message transfer. It -is also used internally by the gtls driver, so it must always be -present on a system. The "gtls" driver is for GnutTLS, a TLS library. -It is used for encrypted message transfer. In the future, additional -drivers will become available (most importantly, we would like to -include a driver for NSS).</p> -<p>What you need to do to build an encrypted syslog channel is to -simply use the proper netstream drivers on both the client and the -server. Client, in the sense of this document, is the rsyslog system -that is sending syslog messages to a remote (central) loghost, which is -called the server. In short, the setup is as follows:</p> -<p><b>Client</b></p> -<ul> -<li>forwards messages via plain tcp syslog using gtls netstream -driver to central sever on port 10514<br> -</li> -</ul> -<p><b>Server</b></p> -<ul> -<li>accept incoming messages via plain tcp syslog using gtls -netstream driver on port 10514</li> -</ul> -<h2>Setting up the system</h2> -<h3>Server Setup</h3> -<p>At the server, you need to have a digital certificate. That -certificate enables SSL operation, as it provides the necessary crypto -keys being used to secure the connection. There is a set of default -certificates in ./contrib/gnutls. These are key.pem and cert.pem. These -are good for testing. If you use it in production, -it is very easy to break into your secure channel as everybody is able -to get hold of your private key. So it is a good idea to -generate the key and certificate yourself.</p> -<p>You also need a root CA certificate. Again, there is a sample -CA certificate in ./contrib/gnutls, named ca.cert. It is suggested to -generate your own.</p> -<p>To configure the server, you need to tell it where are its -certificate files, to use the gtls driver and start up a listener. This -is done as follows:<br> -</p> -<blockquote><code></code> -<pre># make gtls driver the default -$DefaultNetstreamDriver gtls - -# certificate files -$DefaultNetstreamDriverCAFile /path/to/contrib/gnutls/ca.pem -$DefaultNetstreamDriverCertFile /path/to/contrib/gnutls/cert.pem -$DefaultNetstreamDriverKeyFile /path/to/contrib/gnutls/key.pem - -$ModLoad imtcp # load TCP listener - -$InputTCPServerStreamDriverMode 1 # run driver in TLS-only mode -$InputTCPServerStreamDriverAuthMode anon # client is NOT authenticated -$InputTCPServerRun 10514 # start up listener at port 10514 -</pre> -</blockquote> -This is all you need to do. You can use the rest of your rsyslog.conf -together with this configuration. The way messages are received does -not interfer with any other option, so you are able to do anything else -you like without any restrictions. -<p>Restart rsyslogd. The server should now be fully -operational.</p> -<h3>Client Setup</h3> -<p>The client setup is equally simple. You need less -certificates, just the CA cert. </p> -<blockquote> -<pre># certificate files - just CA for a client -$DefaultNetstreamDriverCAFile /path/to/contrib/gnutls/ca.pem - -# set up the action -$DefaultNetstreamDriver gtls # use gtls netstream driver -$ActionSendStreamDriverMode 1 # require TLS for the connection -$ActionSendStreamDriverAuthMode anon # server is NOT authenticated -*.* @@(o)server.example.net:10514 # send (all) messages - -</pre> -</blockquote> -<p>Note that we use the regular TCP forwarding syntax (@@) here. -There is nothing special, because the encryption is handled by the -netstream driver. So I have just forwarded every message (*.*) for -simplicity - you can use any of rsyslog's filtering capabilities (like -epxression-based filters or regular expressions). Note that the "(o)" -part is not strictly necessary. It selects octet-based framing, which -provides compatiblity to IETF's syslog-transport-tls draft. Besides -compatibility, this is also a more reliable transfer mode, so I suggest -to always use it.</p> -<h3>Done</h3> -<p>After -following these steps, you should have a working secure -syslog forwarding system. To verify, you can type "logger test" or a -similar "smart" command on the client. It should show up in the -respective server log file. If you dig out your sniffer, you should see -that the traffic on the wire is actually protected.</p> -<h3>Limitations</h3> -<p>The -RELP transport can currently not be protected by TLS. A work-around is -to use stunnel. TLS support for RELP will be added once plain TCP -syslog has sufficiently matured and there either is some time left to do this -or we find a sponsor ;).</p> -<h2>Certificates</h2> -<p>In order to be really secure, certificates are needed. This is -a short summary on how to generate the necessary certificates with -GnuTLS' certtool. You can also generate certificates via other tools, -but as we currently support GnuTLS as the only TLS library, we thought -it is a good idea to use their tools.<br> -</p> -<p>Note that this section aims at people who are not involved -with PKI at all. The main goal is to get them going in a reasonable -secure way. </p> -<h3>CA Certificate</h3> -<p>This is used to sign all of your other certificates. The CA -cert must be trusted by all clients and servers. The private key must -be well-protected and not given to any third parties. The certificate -itself can (and must) be distributed. To generate it, do the following:</p> -<ol> -<li>generate the private key: -<pre>certtool --generate-privkey --outfile ca-key.pem</pre> -<br> -This takes a short while. Be sure to do some work on your workstation, -it waits for radom input. Switching between windows is sufficient ;) -</li> -<li>now create the (self-signed) CA certificate itself:<br> -<pre>certtool --generate-self-signed --load-privkey ca-key.pem --outfile ca.pem</pre> -This generates the CA certificate. This command queries you for a -number of things. Use appropriate responses. When it comes to -certificate validity, keep in mind that you need to recreate all -certificates when this one expires. So it may be a good idea to use a -long period, eg. 3650 days (roughly 10 years). You need to specify that -the certificates belongs to an authrity. The certificate is used to -sign other certificates.<br> -</li> -<li>You need to distribute this certificate -to all peers and you need to point to it via the -$DefaultNetstreamDriverCAFile config directive. All other certificates -will be issued by this CA.<br> -Important: do only distribute the ca.pem, NOT ca-key.pem (the private -key). Distributing the CA private key would totally breach security as -everybody could issue new certificates on the behalf of this CA. -</li> -</ol> -<h3>Individual Peer Certificate</h3> -<p>Each peer (be it client, server or both), needs a certificate -that conveys its identity. Access control is based on these -certificates. You can, for example, configure a server to accept -connections only from configured clients. The client ID is taken from -the client instances certificate. So as a general rule of thumb, you -need to create a certificate for each instance of rsyslogd that you -run. That instance also needs the private key, so that it can properly -decrypt the traffic. Safeguard the peer's private key file. If somebody -gets hold of it, it can malicously pretend to be the compromised host. -If such happens, regenerate the certificate and make sure you use a -different name instead of the compromised one (if you use name-based -authentication). </p> -<p>These are the steps to generate the indivudual certificates -(repeat: you need to do this for every instance, do NOT share the -certificates created in this step):</p> -<ol> -<li>generate a private key (do NOT mistake this with the CA's -private key - this one is different):<br> -<pre>certtool --generate-privkey --outfile key.pem</pre> -Again, this takes a short while.</li> -<li>generate a certificate request:<br> -<pre>certtool --generate-request --load-privkey key.pem --outfile request.pem</pre> -If you do not have the CA's private key (because you are not authorized -for this), you can send the certificate request to the responsible -person. If you do this, you can skip the remaining steps, as the CA -will provide you with the final certificate. If you submit the request -to the CA, you need to tell the CA the answers that you would normally -provide in step 3 below. -</li> -<li>Sign (validate, authorize) the certificate request and -generate the instances certificate. You need to have the CA's -certificate and private key for this:<br> -<pre>certtool --generate-certificate --load-request request.pem --outfile cert.pem \<br> --load-ca-certificate ca.pem --load-ca-privkey ca-key.pem</pre> -Answer questions as follows: Cert does not belogn to an authority; it -is a TLS web server and client certificate; the dnsName MUST be the -name of the peer in question (e.g. centralserver.example.net) - this is -the name used for authenticating the peers. Please note that you may -use an IP address in dnsName. This is a good idea if you would like to -use default server authentication and you use selector lines with IP -addresses (e.g. "*.* @@192.168.0.1") - in that case you need to select -a dnsName of 192.168.0.1. But, of course, changing the server IP then -requires generating a new certificate.</li> -</ol> -After you have generated the certificate, you need to place it onto the -local machine running rsyslogd. Specify the certificate and key via the -$DefaultNetstreamDriverCertFile /path/to/cert.pem and -$DefaultNetstreamDriverKeyFile /path/to/key.pem configuration -directives. Make sure that nobody has access to key.pem, as that would -breach security. And, once again: do NOT use these files on more than -one instance. Doing so would prevent you from distinguising between the -instances and thus would disable useful authentication. -<h3>Troubleshooting Certificates</h3> -<p>If you experience trouble with your certificate setup, it may -be -useful to get some information on what is contained in a specific -certificate (file). To obtain that information, do </p> -<pre>$ certtool --certificate-info --infile cert.pem</pre> -<p>where "cert.pem" can be replaced by the various certificate pem files (but it does not work with the key files).</p> -<h2>Conclusion</h2> -<p>With minumal effort, you can set up a secure logging -infrastructure employing TLS encrypted syslog message transmission.</p> -<h3>Feedback requested</h3> -<p>I would appreciate feedback on this tutorial. If you have -additional ideas, comments or find bugs (I *do* bugs - no way... ;)), -please -<a href="mailto:rgerhards@adiscon.com">let me know</a>.</p> -<h2>Revision History</h2> -<ul> -<li>2008-05-06 * <a href="http://www.gerhards.net/rainer">Rainer -Gerhards</a> * Initial Version created</li><li>2008-05-26 * <a href="http://www.gerhards.net/rainer">Rainer -Gerhards</a> * added information about certificates</li> -</ul> -<h2>Copyright</h2> -<p>Copyright (c) 2008 <a href="http://www.adiscon.com/en/people/rainer-gerhards.php">Rainer -Gerhards</a> and -<a href="http://www.adiscon.com/en/">Adiscon</a>.</p> -<p> Permission is granted to copy, distribute and/or modify this -document under the terms of the GNU Free Documentation License, Version -1.2 or any later version published by the Free Software Foundation; -with no Invariant Sections, no Front-Cover Texts, and no Back-Cover -Texts. A copy of the license can be viewed at -<a href="http://www.gnu.org/copyleft/fdl.html">http://www.gnu.org/copyleft/fdl.html</a>.</p> -<p>[<a href="manual.html">manual index</a>] -[<a href="rsyslog_conf.html">rsyslog.conf</a>] -[<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> -<p><font size="2">This documentation is part of the -<a href="http://www.rsyslog.com/">rsyslog</a> project.<br> -Copyright © 2008 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and -<a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL -version 2 or higher.</font></p> - -</body></html> diff --git a/doc/sigprov_gt.html b/doc/sigprov_gt.html deleted file mode 100644 index caeee11..0000000 --- a/doc/sigprov_gt.html +++ /dev/null @@ -1,100 +0,0 @@ -<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> -<html> -<head> -<meta http-equiv="Content-Language" content="en"> -<title>GuardTime Log Signature Provider (gt)</title> -</head> - -<body> -<a href="rsyslog_conf_modules.html">back to rsyslog module overview</a> - -<h1>GuardTime Log Signature Provider (gt)</h1> -<p><b>Signature Provider Name: gt</b></p> -<p><b>Author: </b>Rainer Gerhards <rgerhards@adiscon.com></p> -<p><b>Supported Since: </b>since 7.3.9 -<p><b>Description</b>:</p> -<p>Provides the ability to sign syslog messages via the -GuardTime signature services. -</p> - -<p><b>Configuration Parameters</b>:</p> -<p>Signature providers are loaded by omfile, when the -provider is selected in its "sig.providerName" parameter. -Parameters for the provider are given in the omfile action instance -line. -<p>This provider creates a signature file with the same base name but -the extension ".gtsig" for each log file (both for fixed-name files -as well as dynafiles). Both files together form a set. So you need to -archive both in order to prove integrity. -<ul> -<li><b>sig.hashFunction</b> <Hash Algorithm><br> -The following hash algorithms are currently supported: - <ul> - <li>SHA1 - <li>RIPEMD-160 - <li>SHA2-224 - <li>SHA2-256 - <li>SHA2-384 - <li>SHA2-512 - </ul> -</li> -<li><b>sig.timestampService</b> <timestamper URL><br> -This provides the URL of the timestamper service. If not selected, -a default server is selected. This may not necessarily be a good -one for your region. -</li> -<li><b>sig.block.sizeLimit</b> <nbr-records><br> -The maximum number of records inside a single signature block. By -default, there is no size limit, so the signature is only written -on file closure. Note that a signature request typically takes between -one and two seconds. So signing to frequently is probably not a good -idea. -</li> -<li><b>sig.keepRecordHashes</b> <on/<b>off</b>><br> -Controls if record hashes are written to the .gtsig file. This -enhances the ability to spot the location of a signature breach, -but costs considerable disk space (65 bytes for each log record -for SHA2-512 hashes, for example). -</li> -<li><b>sig.keepTreeHashes</b> <on/<b>off</b>><br> -Controls if tree (intermediate) hashes are written to the .gtsig file. This -enhances the ability to spot the location of a signature breach, -but costs considerable disk space (a bit mire than the amount -sig.keepRecordHashes requries). Note that both Tree and Record -hashes can be kept inside the signature file. -</li> -</ul> -<b>Caveats/Known Bugs:</b> -<ul> -<li>currently none known -</li> -</ul> -<p><b>Samples:</b></p> -<p>This writes a log file with it's associated signature file. Default -parameters are used. -</p> -<textarea rows="3" cols="60"> -action(type="omfile" file="/var/log/somelog" - sig.provider="gt") -</textarea> - -<p>In the next sample, we use the more secure SHA2-512 hash function, -sign every 10,000 records and Tree and Record hashes are kept. -<textarea rows="3" cols="60"> -action(type="omfile" file="/var/log/somelog" - sig.provider="gt" sig.hashfunction="SHA2-512" - sig.block.sizelimit="10000" - sig.keepTreeHashes="on" sig.keepRecordHashes="on") -</textarea> - - -<p>[<a href="rsyslog_conf.html">rsyslog.conf overview</a>] -[<a href="manual.html">manual index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> -<p><font size="2">This documentation is part of the -<a href="http://www.rsyslog.com/">rsyslog</a> -project.<br> -Copyright © 2013 by -<a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and -<a href="http://www.adiscon.com/">Adiscon</a>. -Released under the GNU GPL version 3 or higher.</font></p> -</body></html> diff --git a/doc/src/classes.dia b/doc/src/classes.dia Binary files differdeleted file mode 100644 index 8cfcbd0..0000000 --- a/doc/src/classes.dia +++ /dev/null diff --git a/doc/src/queueWorkerLogic.dia b/doc/src/queueWorkerLogic.dia Binary files differdeleted file mode 100644 index 068ea50..0000000 --- a/doc/src/queueWorkerLogic.dia +++ /dev/null diff --git a/doc/src/rfc5424layers.dia b/doc/src/rfc5424layers.dia Binary files differdeleted file mode 100644 index 300b779..0000000 --- a/doc/src/rfc5424layers.dia +++ /dev/null diff --git a/doc/src/tls_cert.dia b/doc/src/tls_cert.dia Binary files differdeleted file mode 100644 index e76431d..0000000 --- a/doc/src/tls_cert.dia +++ /dev/null diff --git a/doc/syslog_parsing.html b/doc/syslog_parsing.html deleted file mode 100644 index 1ccec6f..0000000 --- a/doc/syslog_parsing.html +++ /dev/null @@ -1,210 +0,0 @@ -<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> -<html><head><title>syslog parsing in rsyslog</title> -</head> -<body> -<h1>syslog parsing in rsyslog</h1> -<p><small><i>Written by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> -(2008-09-23)</i></small></p> -<p><b>We regularly receive messages asking why <a href="http://www.rsyslog.com">rsyslog</a> -parses this or that message incorrectly.</b> Of course, it turns out that rsyslog does -the right thing, but the message sender does not. And also of course, this is not even -of the slightest help to the end user experiencing the problem ;). So I thought I write this -paper. It describes the problem source and shows potential solutions (aha!). -<h2>Syslog Standardization</h2> -The syslog protocol has not been standardized until relatively recently.The first document "smelling" a bit -like a standard is <a href="http://www.ietf.org/rfc/rfc3164.txt">RFC 3164</a>, which dates back -to August 2001. The problem is that this document is no real standard. It has assigned "informational" -status by the <a href="http://www.ietf.org">IETF</a> which means it provides some hopefully -useful information but does not demand anything. It is impossible to "comply" to an informational -document. This, of course, doesn't stop marketing guys from telling they comply to RFC3164 and -it also does not stop some techs to tell you "this and that does not comply to RFC3164, so it is -<anybody else but them>'s fault". -<p>Then, there is <a href="http://www.ietf.org/rfc/rfc3195.txt">RFC3195</a>, which is -a real standard. In it's section 3 it makes (a somewhat questionable) reference to (informational) -RFC 3164 which may be interpreted in a way that RFC3195 standardizes the format layed out -in RFC 3164 by virtue of referencing them. So RFC3195 seems to extend its standardization -domain to the concepts layed out in RFC 3164 (which is why I tend to find that refrence -questionable). In that sense, RFC3195 standardizes the format informationally described in -RFC3164, Section 4. But it demands it only for the scope of RFC3195, which is syslog over -BEEP - and NOT syslog over UDP. So one may argue whether or not the RFC3164 format could -be considered a standard for any non-BEEP (including UDP) syslog, too. In the strict view -I tend to have, it does not. Refering to the RFC3195 context usually does not help, -because there are virtually no RFC3195 implementations available (at this time, -I would consider this RFC a failure). -<p>Now let's for a short moment assume that RFC3195 would somehow be able to demand -RFC3164 format for non-BEEP syslog. So we could use RFC3164 format as a standard. But does -that really help? Let's cite RFC 3164, right at the begining of section 4 (actually, this -is the first sentence): -<blockquote> -<pre> - The payload of any IP packet that has a UDP destination port of 514 - MUST be treated as a syslog message. -<pre> -</blockquote> -<p>Think a bit about it: this means that whatever is send to port 514 must be considered -a valid syslog message. No format at all is demanded. So if "this is junk" is sent to -UDP port 514 - voila, we have a valid message (interestingly, it is no longer a syslog -message if it is sent to port 515 ;)). You may now argue that I am overdoing. So let's -cite RFC 3164, Section 5.4, Example 2: -<blockquote> -<pre> - Example 2 - - Use the BFG! - - While this is a valid message, it has extraordinarily little useful - information. -</pre> -</blockquote> -<p>As you can see, RFC3164 explicitely states that no format at all is required. -<p>Now a side-note is due: all of this does not mean that the RFC3164 authors -did not know what they were doing. No, right the contrary is true: RFC3164 mission -is to describe what has been seen in practice as syslog messages and the -conclusion is quite right that there is no common understanding on the -message format. This is also the reason why RFC3164 is an informational document: -it provides useful information, but does not precisely specify anything. -<p>After all of this bashing, I now have to admit that RFC3164 has some format -recommendations layed out in section 4. The format described has quite some -value in it and implementors recently try to follow it. This format is usually meant -when someone tells you that a software is "RFC3164 compliant" or expects "RFC3164 compliant messages". -I also have to admit that rsyslog also uses this format and, in the sense outlined here, -expects messages received to be "RFC3164 compliant" (knowingly that such a beast does not -exist - I am simply lying here ;)). -<p>Please note that there is some relief of the situation in reach. There is a new normative -syslog RFC series upcoming, and it specifies a standard message format. At the time of -this writing, the main documents are sitting in the RFC editor queue waiting for a transport -mapping to be completed. I personally expect them to be assigned RFC numbers in 2009. -<h2>Practical Format Requirements</h2> -<p>From a practical point of view, the message format expected (and generated by -default in legacy mode) is: -<pre><code> -<PRI>TIMESTAMP SP HOST SP TAG MSG(Freetext) -</code></pre> -<p>SP is the ASCII "space" character and the definition of the rest of the fields -can be taken from RFC3164. Please note that there also is a lot of confusion on what -syntax and semantics the TAG actually has. This format is called "legacy syslog" because -it is not well specified (as you know by now) and has been "inherited from the real world". -<p>Rsyslog offers two parsers: one for the upcoming RFC series and one for legacy format. We -concentrate on the later. That parser applies some logic to detect missing hostnames, -is able to handle various ways the TIMESTAMP is typically malformed. In short it applies -a lot of guesswork in trying to figure out what a message really means. I am sure the -guessing algorithm can be improved, and I am always trying that when I see new malformed -messages (and there is an ample set of them...). However, this finds its limits where -it is not possible to differentiate between two entities which could be either. -For example, look at this message: -<pre><code> -<144>Tue Sep 23 11:40:01 taghost sample message -</code></pre> -<p>Does it contain a hostname? Mabye. The value "taghost" is a valid hostname. Of course, it is -also a valid tag. If it is a hostname, the tag's value is "sample" and the msg value is "message". -Or is the hostname missing, the tag is "taghost" and msg is "sample message"? As a human, I tend -to say the later interpretation is correct. But that's hard to tell the message parser (and, no, I do -not intend to apply artificial intelligence just to guess what the hostname value is...). -<p>One approach is to configure the parser so that it never expects hostnames. This becomes problematic -if you receive messages from multiple devices. Over time, I may implement parser conditionals, -but this is not yet available and I am not really sure if it is needed comlexity... -<p>Things like this, happen. Even more scary formats happen in practice. Even from mainstream -vendors. For example, I was just asked about this message (which, btw, finally made me -write this article here): -<pre></code> -"<130> [ERROR] iapp_socket_task.c 399: iappSocketTask: iappRecvPkt returned error" -</code></pre> -<p>If you compare it with the format RFC3164 "suggests", you'll quickly notice that -the message is "a bit" malformed. Actually, even my human intelligence is not sufficient -to guess if there is a TAG or not (is "[ERROR]" a tag or part of the message). I may not be -the smartest guy, but don't expect me to program a parser that is smarter than me. -<p>To the best of my konwledge, these vendor's device's syslog format can be configured, so it -would proabably be a good idea to include a (sufficiently well-formed) timestamp, -the sending hostname and (maybe?) a tag to make this message well parseable. -I will also once again take this sample and see if we can apply some guesswork. -For example, "[" can not be part of a well-formed TIMESTAMP, so logic can conclude -there is not TIMESTAMP. Also, "[" can not be used inside a valid hostname, so -logic can conclude that the message contains no hostname. Even if I implement this -logic (which I will probably do), this is a partial solution: it is impossible to -guess if there is a tag or not (honestly!). And, even worse, it is a solution only for -those set of messages that can be handled by the logic described. Now consider this -hypothetical message: -<pre></code> -"<130> [ERROR] host.example.net 2008-09-23 11-40-22 PST iapp_socket_task.c 399: iappSocketTask: iappRecvPkt returned error" -</code></pre> -<p>Obviously, it requires additional guesswork. If we iterate over all the cases, we -can very quickly see that it is impossible to guess everything correct. In the example above -we can not even surely tell if PST should be a timezone or some other message property. -<p>A potential solution is to generate a parser-table based parser, but this requires -considerable effort and also has quite some runtime overhead. I try to avoid this for -now (but I may do it, especially if someone sponsors this work ;)). Side-note: if you want -to be a bit scared about potential formats, you may want to have a look at my paper -<i>"<a href="http://www.monitorware.com/en/workinprogress/nature-of-syslog-data.php">On the Nature of Syslog Data</a>"</i>. -<h2>Work-Around</h2> -<p><b>The number one work-around is to configure your devices so that they emit -(sufficiently) well-formed messages.</b> You should by now know what these look -like. -<p>If that cure is not available, there are some things you can do in rsyslog to -handle the situation. First of all, be sure to read about -<a href="rsyslog_conf.html">rsyslog.conf format</a> -and the <a href="property_replacer.html">property replacer and properties</a> specifically. -You need to understand that everything is configured in rsyslog. And that the message is parsed -into properties. There are also properties available which do not stem back directly to parsing. -Most importantly, %fromhost% property holds the name of the system rsyslog received -the message from. In non-relay cases, this can be used instead of hostname. In relay cases, -there is no cure other than to either fix the orginal sender or at least one of the -relays in front of the rsyslog instance in question. Similarly, you can use %timegenerated% -instead of %timereported%. Timegenerated is the time the message hit rsyslog for the first -time. For non-relayed, locally connected peers, Timegenerated should be a very close approximation -of the actual time a message was formed at the sender (depending, of course, on potential -internal queueing inside the sender). -Also, you may use the -%rawmsg% property together with the several extraction modes the property replacer supports. -Rawmsg contains the message as it is received from the remote peer. In a sense, you can -implement a post-parser with this method. -<p>To use these properties, you need to define your own templates and assign them. Details -can be found in the above-quoted documentation. Just let's do a quick example. Let's say -you have the horrible message shown above and can not fix the sending device for -some good reason. In rsyslog.conf, you used to say: -<pre><code> -*.* /var/log/somefile -</code></pre> -<p>Of course, things do not work out well with that ill-formed message. So you decide -to dump the rawmsg to the file and pull the remote host and time of message generation -from rsyslog's internal properties (which, btw, is clever, because otherwise there is no -indication of these two properties...). So you need to define a template for that and -make sure the template is used with your file logging action. This is how it may look: -<pre><code> -$template, MalfromedMsgFormater,"%timegenerated% %fromhost% %rawmsg:::drop-last-lf%\n" -*.* /var/log/somefile;MalformedMsgFormatter -</code></pre> -<p>This will make your log much nicer, but not look perfect. Experiment a bit -with the available properties and replacer extraction options to fine-tune it -to your needs. -<h2>The Ultimate Solution...</h2> -<p>Is available with rsyslog 5.3.4 and above. Here, we can define so-called custom -parsers. These are plugin modules, written in C and adapted to a specific message format -need. The big plus of custom parsers is that they offer excellent performance and unlimited -possibilities - far better than any work-around could do. Custom parsers can be -<a href="rsconf1_rulesetparser.html">bound to specific rule sets</a> -(and thus listening) ports with relative ease. The only con is that they must be written. -However, if you are lucky, a parser for your device may already exist. If not, you can -opt to write it yourself, what is not too hard if you know some C. Alternatively, -Adiscon can program one for you as part of the -<a href="http://www.rsyslog.com/professional-services">rsyslog professional services offering</a>. -In any case, you should seriously consider custom parsers as an alternative if you can not -reconfigure your device to send decent message format. -<h2>Wrap-Up</h2> -<p>Syslog message format is not sufficiently standardized. There exists a weak -"standard" format, which is used by a good number of implementations. However, there -exist many others, including mainstream vendor implementations, which have a -(sometimes horribly) different format. Rsyslog tries to deal with anomalies but -can not guess right in all instances. If possible, the sender should be configured -to submit well-formed messages. If that is not possible, you can work around these -issues with rsyslog's property replacer and template system. Or you can use a suitable -message parser or write one for your needs. -<p>I hope this is a useful guide. You may also have a look at the -<a href="troubleshoot.html">rsyslog troubleshooting guide</a> for further help and places where -to ask questions. -<p>[<a href="manual.html">manual index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> -<p><font size="2">This documentation is part of the -<a href="http://www.rsyslog.com/">rsyslog</a> project.<br> -Copyright © 2009 by <a href="http://www.gerhards.net/rainer">Rainer -Gerhards</a> and <a href="http://www.adiscon.com/">Adiscon</a>. -Released under the GNU GPL version 3 or higher.</font></p> -</body></html> diff --git a/doc/syslog_protocol.html b/doc/syslog_protocol.html deleted file mode 100644 index 57eb9ff..0000000 --- a/doc/syslog_protocol.html +++ /dev/null @@ -1,205 +0,0 @@ -<html> -<head> -<title>syslog-protocol support in rsyslog</title> -</head> -<body> -<a href="features.html">back</a> -<h1>syslog-protocol support in rsyslog</h1> -<p><b><a href="http://www.rsyslog.com/">Rsyslog</a> provides a trial -implementation of the proposed -<a href="http://www.monitorware.com/Common/en/glossary/syslog-protocol.php"> -syslog-protocol</a> standard.</b> The intention of this implementation is to -find out what inside syslog-protocol is causing problems during implementation. -As syslog-protocol is a standard under development, its support in rsyslog is -highly volatile. It may change from release to release. So while it provides -some advantages in the real world, users are cautioned against using it right -now. If you do, be prepared that you will probably need to update all of your -rsyslogds with each new release. If you try it anyhow, please provide feedback -as that would be most beneficial for us.</p> -<h2>Currently supported message format</h2> -<p>Due to recent discussion on syslog-protocol, we do not follow any specific -revision of the draft but rather the candidate ideas. The format supported -currently is:</p> -<p><b><code><PRI>VERSION SP TIMESTAMP SP HOSTNAME SP APP-NAME SP PROCID SP MSGID SP [SD-ID]s -SP MSG</code></b></p> -<p>Field syntax and semantics are as defined in IETF I-D syslog-protocol-15.</p> -<h2>Capabilities Implemented</h2> -<ul> - <li>receiving message in the supported format (see above)</li> - <li>sending messages in the supported format</li> - <li>relaying messages</li> - <li>receiving messages in either legacy or -protocol format and transforming - them into the other one</li> - <li>virtual availability of TAG, PROCID, APP-NAME, MSGID, SD-ID no matter if - the message was received via legacy format, API or syslog-protocol format (non-present - fields are being emulated with great success)</li> - <li>maximum message size is set via preprocessor #define</li> - <li>syslog-protocol messages can be transmitted both over UDP and plain TCP - with some restrictions on compliance in the case of TCP</li> -</ul> -<h2>Findings</h2> -<p>This lists what has been found during implementation:</p> -<ul> - <li>The same receiver must be able to support both legacy and - syslog-protocol syslog messages. Anything else would be a big inconvenience - to users and would make deployment much harder. The detection must be done - automatically (see below on how easy that is).</li> - <li><b>NUL characters inside MSG</b> cause the message to be truncated at - that point. This is probably a major point for many C-based implementations. - No measures have yet been taken against this. Modifying the code to "cleanly" - support NUL characters is non-trivial, even though rsyslogd already has some - byte-counted string library (but this is new and not yet available - everywhere).</li> - <li><b>character encoding in MSG</b>: is is problematic to do the right - UTF-8 encoding. The reason is that we pick up the MSG from the local domain - socket (which got it from the syslog(3) API). The text obtained does not - include any encoding information, but it does include non US-ASCII - characters. It may also include any other encoding. Other than by guessing - based on the provided text, I have no way to find out what it is. In order - to make the syslogd do anything useful, I have now simply taken the message - as is and stuffed it into the MSG part. Please note that I think this will - be a route that other implementors would take, too.</li> - <li>A minimal parser is easy to implement. It took me roughly 2 hours to add - it to rsyslogd. This includes the time for restructuring the code to be able - to parse both legacy syslog as well as syslog-protocol. The parser has some - restrictions, though<ul> - <li>STRUCTURED-DATA field is extracted, but not validated. Structured data - "[test ]]" is not caught as an error. Nor are any other errors caught. For - my needs with this syslogd, that level of structured data processing is - probably sufficient. I do not want to parse/validate it in all cases. This - is also a performance issue. I think other implementors could have the same - view. As such, we should not make validation a requirement.</li> - <li>MSG is not further processed (e.g. Unicode not being validated)</li> - <li>the other header fields are also extracted, but no validation is - performed right now. At least some validation should be easy to add (not - done this because it is a proof-of-concept and scheduled to change).</li> -</ul> - </li> - <li>Universal access to all syslog fields (missing ones being emulated) was - also quite easy. It took me around another 2 hours to integrate emulation of - non-present fields into the code base.</li> - <li>The version at the start of the message makes it easy to detect if we - have legacy syslog or syslog-protocol. Do NOT move it to somewhere inside - the middle of the message, that would complicate things. It might not be - totally fail-safe to just rely on "1 " as the "cookie" for a syslog-protocol. - Eventually, it would be good to add some more uniqueness, e.g. "@#1 ".</li> - <li>I have no (easy) way to detect truncation if that happens on the UDP - stack. All I see is that I receive e.g. a 4K message. If the message was e.g. - 6K, I received two chunks. The first chunk (4K) is correctly detected as a - syslog-protocol message, the second (2K) as legacy syslog. I do not see what - we could do against this. This questions the usefulness of the TRUNCATE bit. - Eventually, I could look at the UDP headers and see that it is a fragment. I - have looked at a network sniffer log of the conversation. This looks like - two totally-independent messages were sent by the sender stack.</li> - <li>The maximum message size is currently being configured via a - preprocessor #define. It can easily be set to 2K or 4K, but more than 4K is - not possible because of UDP stack limitations. Eventually, this can be - worked around, but I have not done this yet.</li> - <li>rsyslogd can accept syslog-protocol formatted messages but is able to - relay them in legacy format. I find this a must in real-life deployments. - For this, I needed to do some field mapping so that APP-NAME/PROCID are - mapped into a TAG.</li> - <li>rsyslogd can also accept legacy syslog message and relay them in - syslog-protocol format. For this, I needed to apply some sub-parsing of the - TAG, which on most occasions provides correct results. There might be some - misinterpretations but I consider these to be mostly non-intrusive. </li> - <li>Messages received from the syslog API (the normal case under *nix) also - do not have APP-NAME and PROCID and I must parse them out of TAG as - described directly above. As such, this algorithm is absolutely vital to - make things work on *nix.</li> - <li>I have an issue with messages received via the syslog(3) API (or, to be - more precise, via the local domain socket this API writes to): These - messages contain a timestamp, but that timestamp does neither have the year - nor the high-resolution time. The year is no real issue, I just take the - year of the reception of that message. There is a very small window of - exposure for messages read from the log immediately after midnight Jan 1st. - The message in the domain socket might have been written immediately before - midnight in the old year. I think this is acceptable. However, I can not - assign a high-precision timestamp, at least it is somewhat off if I take the - timestamp from message reception on the local socket. An alternative might - be to ignore the timestamp present and instead use that one when the message - is pulled from the local socket (I am talking about IPC, not the network - - just a reminder...). This is doable, but eventually not advisable. It looks - like this needs to be resolved via a configuration option.</li> - <li>rsyslogd already advertised its origin information on application - startup (in a syslog-protocol-14 compatible format). It is fairly easy to - include that with any message if desired (not currently done).</li> - <li>A big problem I noticed are malformed messages. In -syslog-protocol, we - recommend/require to discard malformed messages. However, in practice users - would like to see everything that the syslogd receives, even if it is in - error. For the first version, I have not included any error handling at all. - However, I think I would deliberately ignore any "discard" requirement. My - current point of view is that in my code I would eventually flag a message - as being invalid and allow the user to filter on this invalidness. So these - invalid messages could be redirected into special bins.</li> - <li>The error logging recommendations (those I insisted on;)) are not really - practical. My application has its own error logging philosophy and I will - not change this to follow a draft.</li> - <li>Relevance of support for leap seconds and senders without knowledge of - time is questionable. I have not made any specific provisions in the code - nor would I know how to handle that differently. I could, however, pull the - local reception timestamp in this case, so it might be useful to have this - feature. I do not think any more about this for the initial proof-of-concept. - Note it as a potential problem area, especially when logging to databases.</li> - <li>The HOSTNAME field for internally generated messages currently contains - the hostname part only, not the FQDN. This can be changed inside the code - base, but it requires some thinking so that thinks are kept compatible with - legacy syslog. I have not done this for the proof-of-concept, but I think it - is not really bad. Maybe an hour or half a day of thinking.</li> - <li>It is possible that I did not receive a TAG with legacy syslog or via - the syslog API. In this case, I can not generate the APP-NAME. For - consistency, I have used "-" in such cases (just like in PROCID, MSGID and - STRUCTURED-DATA).</li> - <li>As an architectural side-effect, syslog-protocol formatted messages can - also be transmitted over non-standard syslog/raw tcp. This implementation - uses the industry-standard LF termination of tcp syslog records. As such, - syslog-protocol messages containing a LF will be broken invalidly. There is - nothing that can be done against this without specifying a TCP transport. - This issue might be more important than one thinks on first thought. The - reason is the wide deployment of syslog/tcp via industry standard.</li> -</ul> -<p><b>Some notes on syslog-transport-udp-06</b></p> -<ul> - <li>I did not make any low-level modifications to the UDP code and think I - am still basically covered with this I-D.</li> - <li>I deliberately violate section 3.3 insofar as that I do not necessarily - accept messages destined to port 514. This feature is user-required and a - must. The same applies to the destination port. I am not sure if the "MUST" - in section 3.3 was meant that this MUST be an option, but not necessarily be - active. The wording should be clarified.</li> - <li>section 3.6: I do not check checksums. See the issue with discarding - messages above. The same solution will probably be applied in my code.</li> -</ul> -<p> </p> -<h2>Conlusions/Suggestions</h2> -<p>These are my personal conclusions and suggestions. Obviously, they must be -discussed ;)</p> -<ul> - <li>NUL should be disallowed in MSG</li> - <li>As it is not possible to definitely know the character encoding of the - application-provided message, MSG should <b>not</b> be specified to use UTF-8 - exclusively. Instead, it is suggested that any encoding may be used but - UTF-8 is preferred. To detect UTF-8, the MSG should start with the UTF-8 - byte order mask of "EF BB BF" if it is UTF-8 encoded (see section 155.9 of - <a href="http://www.unicode.org/versions/Unicode4.0.0/ch15.pdf"> - http://www.unicode.org/versions/Unicode4.0.0/ch15.pdf</a>) </li> - <li>Requirements to drop messages should be reconsidered. I guess I would - not be the only implementor ignoring them.</li> - <li>Logging requirements should be reconsidered and probably be removed.</li> - <li>It would be advisable to specify "-" for APP-NAME is the name is not - known to the sender.</li> - <li>The implications of the current syslog/tcp industry standard on - syslog-protocol should be further evaluated and be fully understood</li> -</ul> -<p> </p> -<p>[<a href="manual.html">manual index</a>] -[<a href="rsyslog_conf.html">rsyslog.conf</a>] -[<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> -<p><font size="2">This documentation is part of the -<a href="http://www.rsyslog.com/">rsyslog</a> project.<br> -Copyright © 2008 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and -<a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL -version 2 or higher.</font></p> -</body> -</html> - diff --git a/doc/tls_cert.jpg b/doc/tls_cert.jpg Binary files differdeleted file mode 100644 index 920e998..0000000 --- a/doc/tls_cert.jpg +++ /dev/null diff --git a/doc/tls_cert_100.jpg b/doc/tls_cert_100.jpg Binary files differdeleted file mode 100644 index beeedc5..0000000 --- a/doc/tls_cert_100.jpg +++ /dev/null diff --git a/doc/tls_cert_ca.html b/doc/tls_cert_ca.html deleted file mode 100644 index 2cae404..0000000 --- a/doc/tls_cert_ca.html +++ /dev/null @@ -1,168 +0,0 @@ -<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> -<html><head><title>TLS-protected syslog: scenario</title> -</head> -<body> - -<h1>Encrypting Syslog Traffic with TLS (SSL)</h1> -<p><small><i>Written by <a href="http://www.adiscon.com/en/people/rainer-gerhards.php">Rainer -Gerhards</a> (2008-06-17)</i></small></p> - -<ul> -<li><a href="rsyslog_secure_tls.html">Overview</a> -<li><a href="tls_cert_scenario.html">Sample Scenario</a> -<li><a href="tls_cert_ca.html">Setting up the CA</a> -<li><a href="tls_cert_machine.html">Generating Machine Certificates</a> -<li><a href="tls_cert_server.html">Setting up the Central Server</a> -<li><a href="tls_cert_client.html">Setting up syslog Clients</a> -<li><a href="tls_cert_udp_relay.html">Setting up the UDP syslog relay</a> -<li><a href="tls_cert_summary.html">Wrapping it all up</a> -</ul> - -<h3>Setting up the CA</h3> -<p>The first step is to set up a certificate authority (CA). It must be -maintained by a trustworthy person (or group) and approves the indentities of -all machines. It does so by issuing their certificates. In a small setup, the -administrator can provide the CA function. What is important is the the CA's -<span style="float: left"> -<script type="text/javascript"><!-- -google_ad_client = "pub-3204610807458280"; -/* rsyslog doc inline */ -google_ad_slot = "5958614527"; -google_ad_width = 125; -google_ad_height = 125; -//--> -</script> -<script type="text/javascript" -src="http://pagead2.googlesyndication.com/pagead/show_ads.js"> -</script> -</span> -private key is well-protocted and machine certificates are only issued if it is -know they are valid (in a single-admin case that means the admin should not -issue certificates to anyone else except himself).</p> -<p>The CA creates a so-called self-signed certificate. That is, it approves its -own authenticy. This sounds useless, but the key point to understand is that -every machine will be provided a copy of the CA's certificate. Accepting this -certificate is a matter of trust. So by configuring the CA certificate, the -administrator tells <a href="http://www.rsyslog.com">rsyslog</a> which certificates to trust. This is the root of all -trust under this model. That is why the CA's private key is so important - -everyone getting hold of it is trusted by our rsyslog instances.</p> -<center><img src="tls_cert_ca.jpg"></center> -<p>To create a self-signed certificate, use the following commands with GnuTLS (which -is currently the only supported TLS library, what may change in the future). -Please note that GnuTLS' tools are not installed by default on many platforms. Also, -the tools do not necessarily come with the GnuTLS core package. If you do not -have certtool on your system, check if there is package for the GnuTLS tools available -(under Fedora, for example, this is named gnutls-utils-<version> and -it is NOT installed by default). </p> -<ol> -<li>generate the private key: -<pre>certtool --generate-privkey --outfile ca-key.pem</pre> -<br> -This takes a short while. Be sure to do some work on your workstation, -it waits for radom input. Switching between windows is sufficient ;) -</li> -<li>now create the (self-signed) CA certificate itself:<br> -<pre>certtool --generate-self-signed --load-privkey ca-key.pem --outfile ca.pem</pre> -This generates the CA certificate. This command queries you for a -number of things. Use appropriate responses. When it comes to -certificate validity, keep in mind that you need to recreate all -certificates when this one expires. So it may be a good idea to use a -long period, eg. 3650 days (roughly 10 years). You need to specify that -the certificates belongs to an authority. The certificate is used to -sign other certificates.<br> -</li> -</ol> -<h3>Sample Screen Session</h3> -<p>Text in red is user input. Please note that for some questions, there is no -user input given. This means the default was accepted by simply pressing the -enter key. -<code><pre> -[root@rgf9dev sample]# <font color="red">certtool --generate-privkey --outfile ca-key.pem --bits 2048</font> -Generating a 2048 bit RSA private key... -[root@rgf9dev sample]# <font color="red">certtool --generate-self-signed --load-privkey ca-key.pem --outfile ca.pem</font> -Generating a self signed certificate... -Please enter the details of the certificate's distinguished name. Just press enter to ignore a field. -Country name (2 chars): <font color="red">US</font> -Organization name: <font color="red">SomeOrg</font> -Organizational unit name: <font color="red">SomeOU</font> -Locality name: <font color="red">Somewhere</font> -State or province name: <font color="red">CA</font> -Common name: <font color="red">someName (not necessarily DNS!)</font> -UID: -This field should not be used in new certificates. -E-mail: -Enter the certificate's serial number (decimal): - - -Activation/Expiration time. -The certificate will expire in (days): <font color="red">3650</font> - - -Extensions. -Does the certificate belong to an authority? (Y/N): <font color="red">y</font> -Path length constraint (decimal, -1 for no constraint): -Is this a TLS web client certificate? (Y/N): -Is this also a TLS web server certificate? (Y/N): -Enter the e-mail of the subject of the certificate: <font color="red">someone@example.net</font> -Will the certificate be used to sign other certificates? (Y/N): <font color="red">y</font> -Will the certificate be used to sign CRLs? (Y/N): -Will the certificate be used to sign code? (Y/N): -Will the certificate be used to sign OCSP requests? (Y/N): -Will the certificate be used for time stamping? (Y/N): -Enter the URI of the CRL distribution point: -X.509 Certificate Information: - Version: 3 - Serial Number (hex): 485a365e - Validity: - Not Before: Thu Jun 19 10:35:12 UTC 2008 - Not After: Sun Jun 17 10:35:25 UTC 2018 - Subject: C=US,O=SomeOrg,OU=SomeOU,L=Somewhere,ST=CA,CN=someName (not necessarily DNS!) - Subject Public Key Algorithm: RSA - Modulus (bits 2048): - d9:9c:82:46:24:7f:34:8f:60:cf:05:77:71:82:61:66 - 05:13:28:06:7a:70:41:bf:32:85:12:5c:25:a7:1a:5a - 28:11:02:1a:78:c1:da:34:ee:b4:7e:12:9b:81:24:70 - ff:e4:89:88:ca:05:30:0a:3f:d7:58:0b:38:24:a9:b7 - 2e:a2:b6:8a:1d:60:53:2f:ec:e9:38:36:3b:9b:77:93 - 5d:64:76:31:07:30:a5:31:0c:e2:ec:e3:8d:5d:13:01 - 11:3d:0b:5e:3c:4a:32:d8:f3:b3:56:22:32:cb:de:7d - 64:9a:2b:91:d9:f0:0b:82:c1:29:d4:15:2c:41:0b:97 - Exponent: - 01:00:01 - Extensions: - Basic Constraints (critical): - Certificate Authority (CA): TRUE - Subject Alternative Name (not critical): - RFC822name: someone@example.net - Key Usage (critical): - Certificate signing. - Subject Key Identifier (not critical): - fbfe968d10a73ae5b70d7b434886c8f872997b89 -Other Information: - Public Key Id: - fbfe968d10a73ae5b70d7b434886c8f872997b89 - -Is the above information ok? (Y/N): <font color="red">y</font> - - -Signing certificate... -[root@rgf9dev sample]# <font color="red">chmod 400 ca-key.pem</font> -[root@rgf9dev sample]# <font color="red">ls -l</font> -total 8 --r-------- 1 root root 887 2008-06-19 12:33 ca-key.pem --rw-r--r-- 1 root root 1029 2008-06-19 12:36 ca.pem -[root@rgf9dev sample]# -</pre></code> -<p><font color="red"><b>Be sure to safeguard ca-key.pem!</b> Nobody except the CA itself -needs to have it. If some third party obtains it, you security is broken!</font> -<h2>Copyright</h2> -<p>Copyright (c) 2008 <a href="http://www.adiscon.com/en/people/rainer-gerhards.php">Rainer -Gerhards</a> and -<a href="http://www.adiscon.com/en/">Adiscon</a>.</p> -<p> Permission is granted to copy, distribute and/or modify this -document under the terms of the GNU Free Documentation License, Version -1.2 or any later version published by the Free Software Foundation; -with no Invariant Sections, no Front-Cover Texts, and no Back-Cover -Texts. A copy of the license can be viewed at -<a href="http://www.gnu.org/copyleft/fdl.html">http://www.gnu.org/copyleft/fdl.html</a>.</p> -</body></html> diff --git a/doc/tls_cert_ca.jpg b/doc/tls_cert_ca.jpg Binary files differdeleted file mode 100644 index f2da045..0000000 --- a/doc/tls_cert_ca.jpg +++ /dev/null diff --git a/doc/tls_cert_client.html b/doc/tls_cert_client.html deleted file mode 100644 index dbe7961..0000000 --- a/doc/tls_cert_client.html +++ /dev/null @@ -1,91 +0,0 @@ -<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> -<html><head><title>TLS-protected syslog: client setup</title> -</head> -<body> - -<h1>Encrypting Syslog Traffic with TLS (SSL)</h1> -<p><small><i>Written by <a href="http://www.adiscon.com/en/people/rainer-gerhards.php">Rainer -Gerhards</a> (2008-07-03)</i></small></p> - -<ul> -<li><a href="rsyslog_secure_tls.html">Overview</a> -<li><a href="tls_cert_scenario.html">Sample Scenario</a> -<li><a href="tls_cert_ca.html">Setting up the CA</a> -<li><a href="tls_cert_machine.html">Generating Machine Certificates</a> -<li><a href="tls_cert_server.html">Setting up the Central Server</a> -<li><a href="tls_cert_client.html">Setting up syslog Clients</a> -<li><a href="tls_cert_udp_relay.html">Setting up the UDP syslog relay</a> -<li><a href="tls_cert_summary.html">Wrapping it all up</a> -</ul> - -<h3>Setting up a client</h3> -<p>In this step, we configure a client machine. We from our scenario, we use -zuse.example.net. You need to do the same steps for all other clients, too (in the -example, that meanst turng.example.net). The client check's the server's identity and -talks to it only if it is the expected server. This is a very important step. -Without it, you would not detect man-in-the-middle attacks or simple malicious servers -who try to get hold of your valuable log data. -<span style="float: left"> -<script type="text/javascript"><!-- -google_ad_client = "pub-3204610807458280"; -/* rsyslog doc inline */ -google_ad_slot = "5958614527"; -google_ad_width = 125; -google_ad_height = 125; -//--> -</script> -<script type="text/javascript" -src="http://pagead2.googlesyndication.com/pagead/show_ads.js"> -</script> -</span> -<p><center><img src="tls_cert_100.jpg"></center> -<p>Steps to do: -<ul> -<li>make sure you have a functional CA (<a href="tls_cert_ca.html">Setting up the CA</a>) -<li>generate a machine certificate for zuse.example.net (follow instructions in - <a href="tls_cert_machine.html">Generating Machine Certificates</a>) -<li>make sure you copy over ca.pem, machine-key.pem ad machine-cert.pem to the client. -Ensure that no user except root can access them (<b>even read permissions are really bad</b>). -<li>configure the client so that it checks the server identity and sends messages only -if the server identity is known. Please note that you have the same options as when -configuring a server. However, we now use a single name only, because there is only one -central server. No using wildcards make sure that we will exclusively talk to that server -(otherwise, a compromised client may take over its role). If you load-balance to different -server identies, you obviously need to allow all of them. It still is suggested to use -explcit names. -</ul> -<p><b>At this point, please be reminded once again that your security needs may be quite different from -what we assume in this tutorial. Evaluate your options based on your security needs.</b> -<h3>Sample syslog.conf</h3> -<p>Keep in mind that this rsyslog.conf sends messages via TCP, only. Also, we do not -show any rules to write local files. Feel free to add them. -<code><pre> -# make gtls driver the default -$DefaultNetstreamDriver gtls - -# certificate files -$DefaultNetstreamDriverCAFile /rsyslog/protected/ca.pem -$DefaultNetstreamDriverCertFile /rsyslog/protected/machine-cert.pem -$DefaultNetstreamDriverKeyFile /rsyslog/protected/machine-key.pem - -$ActionSendStreamDriverAuthMode x509/name -$ActionSendStreamDriverPermittedPeer central.example.net -$ActionSendStreamDriverMode 1 # run driver in TLS-only mode -*.* @@central.example.net:10514 # forward everything to remote server -</pre></code> -<p>Note: the example above forwards every message to the remote server. Of course, -you can use the normal filters to restrict the set of information that is sent. -Depending on your message volume and needs, this may be a smart thing to do. -<p><font color="red"><b>Be sure to safeguard at least the private key (machine-key.pem)!</b> -If some third party obtains it, you security is broken!</font> -<h2>Copyright</h2> -<p>Copyright © 2008 <a href="http://www.adiscon.com/en/people/rainer-gerhards.php">Rainer -Gerhards</a> and -<a href="http://www.adiscon.com/en/">Adiscon</a>.</p> -<p> Permission is granted to copy, distribute and/or modify this -document under the terms of the GNU Free Documentation License, Version -1.2 or any later version published by the Free Software Foundation; -with no Invariant Sections, no Front-Cover Texts, and no Back-Cover -Texts. A copy of the license can be viewed at -<a href="http://www.gnu.org/copyleft/fdl.html">http://www.gnu.org/copyleft/fdl.html</a>.</p> -</body></html> diff --git a/doc/tls_cert_errmsgs.html b/doc/tls_cert_errmsgs.html deleted file mode 100644 index d002174..0000000 --- a/doc/tls_cert_errmsgs.html +++ /dev/null @@ -1,103 +0,0 @@ -<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> -<html><head><title>TLS-protected syslog: error messages</title> -</head> -<body> - -<h1>Encrypting Syslog Traffic with TLS (SSL)</h1> -<p><small><i>Written by <a href="http://www.adiscon.com/en/people/rainer-gerhards.php">Rainer -Gerhards</a> (2008-06-17)</i></small></p> - -<ul> -<li><a href="rsyslog_secure_tls.html">Overview</a> -<li><a href="tls_cert_scenario.html">Sample Scenario</a> -<li><a href="tls_cert_ca.html">Setting up the CA</a> -<li><a href="tls_cert_machine.html">Generating Machine Certificates</a> -<li><a href="tls_cert_server.html">Setting up the Central Server</a> -<li><a href="tls_cert_client.html">Setting up syslog Clients</a> -<li><a href="tls_cert_udp_relay.html">Setting up the UDP syslog relay</a> -<li><a href="tls_cert_summary.html">Wrapping it all up</a> -<li><a href="tls_cert_errmsgs.html">Frequently seen Error Messages</a> -</ul> - -<h3>Error Messages</h3> -<p>This page covers error message you may see when setting up -<span style="float: left"> -<script type="text/javascript"><!-- -google_ad_client = "pub-3204610807458280"; -/* rsyslog doc inline */ -google_ad_slot = "5958614527"; -google_ad_width = 125; -google_ad_height = 125; -//--> -</script> -<script type="text/javascript" -src="http://pagead2.googlesyndication.com/pagead/show_ads.js"> -</script> -</span> -<a href="http://www.rsyslog.com">rsyslog</a> with TLS. Please note that many -of the message stem back to the TLS library being used. In those cases, there is -not always a good explanation available in rsyslog alone. -<p>A single error typically results in two or more message being emitted: (at -least) one is the actual error cause, followed by usually one message with additional -information (like certificate contents). In a typical system, these message should -immediately follow each other in your log. Kepp in mind that they are reported -as syslog.err, so you need to capture these to actually see errors (the default -rsyslog.conf's shipped by many systems will do that, recording them e.g. in -/etc/messages). -<h3>certificate invalid</h3> -<p>Sample: -<code> -not permitted to talk to peer, certificate invalid: <font color="red">insecure algorithm</font> -</code> -<p>This message may occur during connection setup. It indicates that the remote peer's -certificate can not be accepted. The reason for this is given in the message part that -is shown in red. Please note that this red part directly stems back to the TLS library, -so rsyslog does acutally not have any more information about the reason. -<p>With GnuTLS, the following reasons have been seen in practice: -<h4>insecure algorith</h4> -<p>The certificate contains information on which encryption algorithms are to be used. -This information is entered when the certificate is created. -Some older alogrithms are no longer secure and the TLS library does not accept -them. Thus the connection request failed. The cure is to use a certificate with sufficiently secure -alogorithms. -<p>Please note that noi encryption algorithm is totally secure. It only is secure based -on our current knowledge AND on computing power available. As computers get more and more -powerful, previously secure algorithms become insecure over time. As such, algorithms -considered secure today may not be accepted by the TLS library in the future. -<p>So in theory, after a system upgrade, a connection request may fail with the "insecure -algorithm" failure without any change in rsyslog configuration or certificates. This could be -caused by a new perception of the TLS library of what is secure and what not. -<h3>GnuTLS error -64</h3> -<p>Sample: <code>unexpected GnuTLS error -64 in nsd_gtls.c:517: Error while reading file.</code> -<p>This error points to an encoding error witht the pem file in question. It means "base 64 encoding error". -From my experience, it can be caused by a couple of things, some of them not obvious: -<ul> -<li>You specified a wrong file, which is not actually in .pem format -<li>The file was incorrectly generated -<li>I think I have also seen this when I accidently swapped private key files and -certificate files. So double-check the type of file you are using. -<li>It may even be a result of an access (permission) problem. In theory, that -should lead to another error, but in practice it sometimes seems to lead to -this -64 error. -</ul> -<h3>info on invalid cert</h3> -<p>Sample: -<code> -info on invalid cert: peer provided 1 certificate(s). Certificate 1 info: certificate valid from Wed Jun 18 11:45:44 2008 to Sat Jun 16 11:45:53 2018; Certificate public key: RSA; DN: C=US,O=Sample Corp,OU=Certs,L=Somehwere,ST=CA,CN=somename; Issuer DN: C=US,O=Sample Corp,OU=Certs,L=Somewhere,ST=CA,CN=somename,EMAIL=xxx@example.com; SAN:DNSname: machine.example.net; -</code> -<p>This is <b>not</b> an error message in itself. It always follows the actual error message and -tells you what is seen in the peer's certificate. This is done to give you a chance to evaluate -the certificate and better understand why the initial error message was issued. -<p>Please note that you can NOT diagnose problems based on this message alone. It follows -in a number of error cases and does not pinpoint any problems by itself. -<h2>Copyright</h2> -<p>Copyright (c) 2008 <a href="http://www.adiscon.com/en/people/rainer-gerhards.php">Rainer -Gerhards</a> and -<a href="http://www.adiscon.com/en/">Adiscon</a>.</p> -<p> Permission is granted to copy, distribute and/or modify this -document under the terms of the GNU Free Documentation License, Version -1.2 or any later version published by the Free Software Foundation; -with no Invariant Sections, no Front-Cover Texts, and no Back-Cover -Texts. A copy of the license can be viewed at -<a href="http://www.gnu.org/copyleft/fdl.html">http://www.gnu.org/copyleft/fdl.html</a>.</p> -</body></html> diff --git a/doc/tls_cert_machine.html b/doc/tls_cert_machine.html deleted file mode 100644 index 095e15c..0000000 --- a/doc/tls_cert_machine.html +++ /dev/null @@ -1,182 +0,0 @@ -<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> -<html><head><title>TLS-protected syslog: generating the machine certificate</title> -</head> -<body> - -<h1>Encrypting Syslog Traffic with TLS (SSL)</h1> -<p><small><i>Written by <a href="http://www.adiscon.com/en/people/rainer-gerhards.php">Rainer -Gerhards</a> (2008-06-18)</i></small></p> - -<ul> -<li><a href="rsyslog_secure_tls.html">Overview</a> -<li><a href="tls_cert_scenario.html">Sample Scenario</a> -<li><a href="tls_cert_ca.html">Setting up the CA</a> -<li><a href="tls_cert_machine.html">Generating Machine Certificates</a> -<li><a href="tls_cert_server.html">Setting up the Central Server</a> -<li><a href="tls_cert_client.html">Setting up syslog Clients</a> -<li><a href="tls_cert_udp_relay.html">Setting up the UDP syslog relay</a> -<li><a href="tls_cert_summary.html">Wrapping it all up</a> -</ul> - -<h3>generating the machine certificate</h3> -<p>In this step, we generate certificates for each of the machines. Please note -that both clients and servers need certificates. The certificate identifies each -machine to the remote peer. The DNSName specified inside the certificate can -<span style="float: left"> -<script type="text/javascript"><!-- -google_ad_client = "pub-3204610807458280"; -/* rsyslog doc inline */ -google_ad_slot = "5958614527"; -google_ad_width = 125; -google_ad_height = 125; -//--> -</script> -<script type="text/javascript" -src="http://pagead2.googlesyndication.com/pagead/show_ads.js"> -</script> -</span> -be specified inside the $<object>PermittedPeer config statements. -<p>For now, we assume that a single person (or group) is responsible for the whole -rsyslog system and thus it is OK if that single person is in posession of all -machine's private keys. This simplification permits us to use a somewhat less -complicated way of generating the machine certificates. So, we generate both the private -and public key on the CA (which is NOT a server!) and then copy them over to the -respective machines. -<p>If the roles of machine and CA administrators are split, the private key must -be generated by the machine administrator. This is done via a certificate request. -This request is then sent to the CA admin, which in turn generates the certificate -(containing the public key). The CA admin then sends back the certificate to the -machine admin, who installs it. That way, the CA admin never get's hold of the -machine's private key. Instructions for this mode will be given in a later revision -of this document. -<p><b>In any case, it is vital that the machine's private key is protected. Anybody -able to obtain that private key can imporsonate as the machine to which it belongs, thus -breaching your security.</b> -<h3>Sample Screen Session</h3> -<p>Text in red is user input. Please note that for some questions, there is no -user input given. This means the default was accepted by simply pressing the -enter key. -<p><b>Please note:</b> you need to substitute the names specified below with values -that match your environment. Most importantly, machine.example.net must be replaced -by the actual name of the machine that will be using this certificate. For example, -if you generate a certificate for a machine named "server.example.com", you need -to use that name. If you generate a certificate for "client.example.com", you need -to use this name. Make sure that each machine certificate has a unique name. If not, -you can not apply proper access control. -<code><pre> -[root@rgf9dev sample]# <font color="red">certtool --generate-privkey --outfile key.pem --bits 2048</font> -Generating a 2048 bit RSA private key... -[root@rgf9dev sample]# <font color="red">certtool --generate-request --load-privkey key.pem --outfile request.pem</font> -Generating a PKCS #10 certificate request... -Country name (2 chars): <font color="red">US</font> -Organization name: <font color="red">SomeOrg</font> -Organizational unit name: <font color="red">SomeOU</font> -Locality name: <font color="red">Somewhere</font> -State or province name: <font color="red">CA</font> -Common name: <font color="red">machine.example.net</font> -UID: -Enter a dnsName of the subject of the certificate: -Enter the IP address of the subject of the certificate: -Enter the e-mail of the subject of the certificate: -Enter a challange password: -Does the certificate belong to an authority? (y/N): <font color="red">n</font> -Will the certificate be used for signing (DHE and RSA-EXPORT ciphersuites)? (y/N): -Will the certificate be used for encryption (RSA ciphersuites)? (y/N): -Is this a TLS web client certificate? (y/N): <font color="red">y</font> -Is this also a TLS web server certificate? (y/N): <font color="red">y</font> -[root@rgf9dev sample]# <font color="red">certtool --generate-certificate --load-request request.pem --outfile cert.pem --load-ca-certificate ca.pem --load-ca-privkey ca-key.pem</font> -Generating a signed certificate... -Enter the certificate's serial number (decimal): - - -Activation/Expiration time. -The certificate will expire in (days): 1000 - - -Extensions. -Do you want to honour the extensions from the request? (y/N): -Does the certificate belong to an authority? (Y/N): <font color="red">n</font> -Is this a TLS web client certificate? (Y/N): <font color="red">y</font> -Is this also a TLS web server certificate? (Y/N): <font color="red">y</font> -Enter the dnsName of the subject of the certificate: <font color="red">machine.example.net</font> <i>{This is the name of the machine that will use the certificate}</i> -Enter the IP address of the subject of certificate: -Will the certificate be used for signing (DHE and RSA-EXPORT ciphersuites)? (Y/N): -Will the certificate be used for encryption (RSA ciphersuites)? (Y/N): -X.509 Certificate Information: - Version: 3 - Serial Number (hex): 485a3819 - Validity: - Not Before: Thu Jun 19 10:42:54 UTC 2008 - Not After: Wed Mar 16 10:42:57 UTC 2011 - Subject: C=US,O=SomeOrg,OU=SomeOU,L=Somewhere,ST=CA,CN=machine.example.net - Subject Public Key Algorithm: RSA - Modulus (bits 2048): - b2:4e:5b:a9:48:1e:ff:2e:73:a1:33:ee:d8:a2:af:ae - 2f:23:76:91:b8:39:94:00:23:f2:6f:25:ad:c9:6a:ab - 2d:e6:f3:62:d8:3e:6e:8a:d6:1e:3f:72:e5:d8:b9:e0 - d0:79:c2:94:21:65:0b:10:53:66:b0:36:a6:a7:cd:46 - 1e:2c:6a:9b:79:c6:ee:c6:e2:ed:b0:a9:59:e2:49:da - c7:e3:f0:1c:e0:53:98:87:0d:d5:28:db:a4:82:36:ed - 3a:1e:d1:5c:07:13:95:5d:b3:28:05:17:2a:2b:b6:8e - 8e:78:d2:cf:ac:87:13:15:fc:17:43:6b:15:c3:7d:b9 - Exponent: - 01:00:01 - Extensions: - Basic Constraints (critical): - Certificate Authority (CA): FALSE - Key Purpose (not critical): - TLS WWW Client. - TLS WWW Server. - Subject Alternative Name (not critical): - DNSname: machine.example.net - Subject Key Identifier (not critical): - 0ce1c3dbd19d31fa035b07afe2e0ef22d90b28ac - Authority Key Identifier (not critical): - fbfe968d10a73ae5b70d7b434886c8f872997b89 -Other Information: - Public Key Id: - 0ce1c3dbd19d31fa035b07afe2e0ef22d90b28ac - -Is the above information ok? (Y/N): <font color="red">y</font> - - -Signing certificate... -[root@rgf9dev sample]# <font color="red">rm -f request.pem</font> -[root@rgf9dev sample]# <font color="red">ls -l</font> -total 16 --r-------- 1 root root 887 2008-06-19 12:33 ca-key.pem --rw-r--r-- 1 root root 1029 2008-06-19 12:36 ca.pem --rw-r--r-- 1 root root 1074 2008-06-19 12:43 cert.pem --rw-r--r-- 1 root root 887 2008-06-19 12:40 key.pem -[root@rgf9dev sample]# # it may be a good idea to rename the files to indicate where they belong to -[root@rgf9dev sample]# <font color="red">mv cert.pem machine-cert.pem</font> -[root@rgf9dev sample]# <font color="red">mv key.pem machine-key.pem</font> -[root@rgf9dev sample]# -</pre></code> -<h3>Distributing Files</h3> -<p>Provide the machine with: -<ul> -<li>a copy of ca.pem -<li>cert.pem -<li>key.pem -</ul> -<p>This is how the relevant part of rsyslog.conf looks on the target machine: -<p> -<code><pre> -$DefaultNetstreamDriverCAFile /home/rger/proj/rsyslog/sample/ca.pem -$DefaultNetstreamDriverCertFile /home/rger/proj/rsyslog/sample/machine-cert.pem -$DefaultNetstreamDriverKeyFile /home/rger/proj/rsyslog/sample/machine-key.pem -</pre></code> -<p><b><font color="red">Never</font> provide anyone with ca-key.pem!</b> Also, make sure -nobody but the machine in question gets hold of key.pem. -<h2>Copyright</h2> -<p>Copyright (c) 2008 <a href="http://www.adiscon.com/en/people/rainer-gerhards.php">Rainer -Gerhards</a> and -<a href="http://www.adiscon.com/en/">Adiscon</a>.</p> -<p> Permission is granted to copy, distribute and/or modify this -document under the terms of the GNU Free Documentation License, Version -1.2 or any later version published by the Free Software Foundation; -with no Invariant Sections, no Front-Cover Texts, and no Back-Cover -Texts. A copy of the license can be viewed at -<a href="http://www.gnu.org/copyleft/fdl.html">http://www.gnu.org/copyleft/fdl.html</a>.</p> -</body></html> diff --git a/doc/tls_cert_scenario.html b/doc/tls_cert_scenario.html deleted file mode 100644 index 7973532..0000000 --- a/doc/tls_cert_scenario.html +++ /dev/null @@ -1,63 +0,0 @@ -<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> -<html><head><title>TLS-protected syslog: scenario</title> -</head> -<body> - -<h1>Encrypting Syslog Traffic with TLS (SSL)</h1> -<p><small><i>Written by <a href="http://www.adiscon.com/en/people/rainer-gerhards.php">Rainer -Gerhards</a> (2008-06-17)</i></small></p> - -<ul> -<li><a href="rsyslog_secure_tls.html">Overview</a> -<li><a href="tls_cert_scenario.html">Sample Scenario</a> -<li><a href="tls_cert_ca.html">Setting up the CA</a> -<li><a href="tls_cert_machine.html">Generating Machine Certificates</a> -<li><a href="tls_cert_server.html">Setting up the Central Server</a> -<li><a href="tls_cert_client.html">Setting up syslog Clients</a> -<li><a href="tls_cert_udp_relay.html">Setting up the UDP syslog relay</a> -<li><a href="tls_cert_summary.html">Wrapping it all up</a> -<li><a href="tls_cert_errmsgs.html">Frequently seen Error Messages</a> -</ul> - -<h3>Sample Scenario</h3> -<p>We have a quite simple scenario. There is one central syslog server, -<span style="float: left"> -<script type="text/javascript"><!-- -google_ad_client = "pub-3204610807458280"; -/* rsyslog doc inline */ -google_ad_slot = "5958614527"; -google_ad_width = 125; -google_ad_height = 125; -//--> -</script> -<script type="text/javascript" -src="http://pagead2.googlesyndication.com/pagead/show_ads.js"> -</script> -</span> -named central.example.net. These server is being reported to by two Linux -machines with name zuse.example.net and turing.example.net. Also, there is a -third client - ada.example.net - which send both its own messages to the central -server but also forwards messages receive from an UDP-only capable router. We -hav decided to use ada.example.net because it is in the same local network -segment as the router and so we enjoy TLS' security benefits for forwarding the -router messages inside the corporate network. All systems (except the router) use -<a href="http://www.rsyslog.com/">rsyslog</a> as the syslog software.</p> -<p><center><img src="tls_cert_100.jpg"></center> -<p>Please note that the CA must not necessarily be connected to the rest of the -network. Actually, it may be considered a security plus if it is not. If the CA -is reachable via the regular network, it should be sufficiently secured (firewal -rules et al). Keep in mind that if the CA's security is breached, your overall -system security is breached. -<p>In case the CA is compromised, you need to regenerate the CA's certificate as well -as all individual machines certificates. -<h2>Copyright</h2> -<p>Copyright (c) 2008 <a href="http://www.adiscon.com/en/people/rainer-gerhards.php">Rainer -Gerhards</a> and -<a href="http://www.adiscon.com/en/">Adiscon</a>.</p> -<p> Permission is granted to copy, distribute and/or modify this -document under the terms of the GNU Free Documentation License, Version -1.2 or any later version published by the Free Software Foundation; -with no Invariant Sections, no Front-Cover Texts, and no Back-Cover -Texts. A copy of the license can be viewed at -<a href="http://www.gnu.org/copyleft/fdl.html">http://www.gnu.org/copyleft/fdl.html</a>.</p> -</body></html> diff --git a/doc/tls_cert_server.html b/doc/tls_cert_server.html deleted file mode 100644 index 9c024bc..0000000 --- a/doc/tls_cert_server.html +++ /dev/null @@ -1,127 +0,0 @@ -<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> -<html><head><title>TLS-protected syslog: central server setup</title> -</head> -<body> - -<h1>Encrypting Syslog Traffic with TLS (SSL)</h1> -<p><small><i>Written by <a href="http://www.adiscon.com/en/people/rainer-gerhards.php">Rainer -Gerhards</a> (2008-06-18)</i></small></p> - -<ul> -<li><a href="rsyslog_secure_tls.html">Overview</a> -<li><a href="tls_cert_scenario.html">Sample Scenario</a> -<li><a href="tls_cert_ca.html">Setting up the CA</a> -<li><a href="tls_cert_machine.html">Generating Machine Certificates</a> -<li><a href="tls_cert_server.html">Setting up the Central Server</a> -<li><a href="tls_cert_client.html">Setting up syslog Clients</a> -<li><a href="tls_cert_udp_relay.html">Setting up the UDP syslog relay</a> -<li><a href="tls_cert_summary.html">Wrapping it all up</a> -</ul> - -<h3>Setting up the Central Server</h3> -<p>In this step, we configure the central server. We assume it accepts messages only -via TLS protected plain tcp based syslog from those peers that are explicitely permitted -to send to it. The picture below show our configuration. This step configures -the server central.example.net. -<span style="float: left"> -<script type="text/javascript"><!-- -google_ad_client = "pub-3204610807458280"; -/* rsyslog doc inline */ -google_ad_slot = "5958614527"; -google_ad_width = 125; -google_ad_height = 125; -//--> -</script> -<script type="text/javascript" -src="http://pagead2.googlesyndication.com/pagead/show_ads.js"> -</script> -</span> -<p><center><img src="tls_cert_100.jpg"></center> -<p><i><font color="red"><b>Important:</b> Keep in mind that the order of configuration directives -is very important in rsyslog. As such, the samples given below do only work if the given -order is preserved.</font> Re-ordering the directives can break configurations and has broken them -in practice. If you intend to re-order them, please be sure that you fully understand how -the configuration language works and, most importantly, which statements form a block together. -Please also note that we understand the the current configuration file format is -ugly. However, there has been more important work in the way of enhancing it. If you would like -to contribute some time to improve the config file language, please let us know. Any help -is appreciated (be it doc or coding work!).</i> -<p>Steps to do: -<ul> -<li>make sure you have a functional CA (<a href="tls_cert_ca.html">Setting up the CA</a>) -<li>generate a machine certificate for central.example.net (follow instructions in - <a href="tls_cert_machine.html">Generating Machine Certificates</a>) -<li>make sure you copy over ca.pem, machine-key.pem ad machine-cert.pem to the central server. -Ensure that no user except root can access them (<b>even read permissions are really bad</b>). -<li>configure the server so that it accepts messages from all machines in the -example.net domain that have certificates from your CA. Alternatively, you may also -precisely define from which machine names messages are accepted. See sample rsyslog.conf -below. -</ul> -In this setup, we use wildcards to ease adding new systems. We permit the server to accept -messages from systems whos names match *.example.net. -<pre><code> -$InputTCPServerStreamDriverPermittedPeer *.example.net -</code></pre> -This will match zuse.example.net and -turing.example.net, but NOT pascal.otherdepartment.example.net. If the later would be desired, -you can (and need) to include additional permitted peer config statments: -<pre><code> -$InputTCPServerStreamDriverPermittedPeer *.example.net -$InputTCPServerStreamDriverPermittedPeer *.otherdepartment.example.net -$InputTCPServerStreamDriverPermittedPeer *.example.com -</code></pre> -<p>As can be seen with example.com, the different permitted peers need NOT to be in a single -domain tree. Also, individual machines can be configured. For example, if only zuse, turing -and ada should be able to talk to the server, you can achive this by: -<pre><code> -$InputTCPServerStreamDriverPermittedPeer zuse.example.net -$InputTCPServerStreamDriverPermittedPeer turing.example.net -$InputTCPServerStreamDriverPermittedPeer ada.example.net -</code></pre> -<p>As an extension to the (upcoming) IETF syslog/tls standard, you can specify some text -together with a domain component wildcard. So "*server.example.net", "server*.example.net" -are valid permitted peers. However "server*Fix.example.net" is NOT a valid wildcard. The -IETF standard permits no text along the wildcards. -<p>The reason we use wildcards in the default setup is that it makes it easy to add systems -without the need to change the central server's configuration. It is important to understand that -the central server will accept names <b>only</b> (no exception) if the client certificate was -signed by the CA we set up. So if someone tries to create a malicious certificate with -a name "zuse.example.net", the server will <b>not</b> accept it. So a wildcard is safe -as long as you ensure CA security is not breached. Actually, you authorize a client by issuing -the certificate to it. -<p><b>At this point, please be reminded once again that your security needs may be quite different from -what we assume in this tutorial. Evaluate your options based on your security needs.</b> -<h3>Sample syslog.conf</h3> -<p>Keep in mind that this rsyslog.conf accepts messages via TCP, only. The only other -source accepted is messages from the server itself. -<code><pre> -$ModLoad imuxsock # local messages -$ModLoad imtcp # TCP listener - -# make gtls driver the default -$DefaultNetstreamDriver gtls - -# certificate files -$DefaultNetstreamDriverCAFile /rsyslog/protected/ca.pem -$DefaultNetstreamDriverCertFile /rsyslog/protected/machine-cert.pem -$DefaultNetstreamDriverKeyFile /rsyslog/protected/machine-key.pem - -$InputTCPServerStreamDriverAuthMode x509/name -$InputTCPServerStreamDriverPermittedPeer *.example.net -$InputTCPServerStreamDriverMode 1 # run driver in TLS-only mode -$InputTCPServerRun 10514 # start up listener at port 10514 -</pre></code> -<p><font color="red"><b>Be sure to safeguard at least the private key (machine-key.pem)!</b> -If some third party obtains it, you security is broken!</font> -<h2>Copyright</h2> -<p>Copyright (c) 2008 <a href="http://www.adiscon.com/en/people/rainer-gerhards.php">Rainer -Gerhards</a> and -<a href="http://www.adiscon.com/en/">Adiscon</a>.</p> -<p> Permission is granted to copy, distribute and/or modify this -document under the terms of the GNU Free Documentation License, Version -1.2 or any later version published by the Free Software Foundation; -with no Invariant Sections, no Front-Cover Texts, and no Back-Cover -Texts. A copy of the license can be viewed at -<a href="http://www.gnu.org/copyleft/fdl.html">http://www.gnu.org/copyleft/fdl.html</a>.</p> -</body></html> diff --git a/doc/tls_cert_summary.html b/doc/tls_cert_summary.html deleted file mode 100644 index 8e003bc..0000000 --- a/doc/tls_cert_summary.html +++ /dev/null @@ -1,66 +0,0 @@ -<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> -<html><head><title>TLS-protected syslog: Summary</title> -</head> -<body> - -<h1>Encrypting Syslog Traffic with TLS (SSL)</h1> -<p><small><i>Written by <a href="http://www.adiscon.com/en/people/rainer-gerhards.php">Rainer -Gerhards</a> (2008-07-03)</i></small></p> - -<ul> -<li><a href="rsyslog_secure_tls.html">Overview</a> -<li><a href="tls_cert_scenario.html">Sample Scenario</a> -<li><a href="tls_cert_ca.html">Setting up the CA</a> -<li><a href="tls_cert_machine.html">Generating Machine Certificates</a> -<li><a href="tls_cert_server.html">Setting up the Central Server</a> -<li><a href="tls_cert_client.html">Setting up syslog Clients</a> -<li><a href="tls_cert_udp_relay.html">Setting up the UDP syslog relay</a> -<li><a href="tls_cert_summary.html">Wrapping it all up</a> -</ul> - -<h3>Summary</h3> -<p>If you followed the steps outlined in this documentation set, you now have -<span style="float: left"> -<script type="text/javascript"><!-- -google_ad_client = "pub-3204610807458280"; -/* rsyslog doc inline */ -google_ad_slot = "5958614527"; -google_ad_width = 125; -google_ad_height = 125; -//--> -</script> -<script type="text/javascript" -src="http://pagead2.googlesyndication.com/pagead/show_ads.js"> -</script> -</span> -a reasonable (for most needs) secure setup for the following environment: -<center><img src="tls_cert_100.jpg"></center> -<p>You have learned about the security decisions involved and which we -made in this example. <b>Be once again reminded that you must make sure yourself -that whatever you do matches your security needs!</b> There is no guarantee that -what we generally find useful actually is. It may even be totally unsuitable for -your environment. -<p>In the example, we created a rsyslog certificate authority (CA). Guard the CA's -files. You need them whenever you need to create a new machine certificate. We also saw how -to generate the machine certificates themselfs and distribute them to the individual -machines. Also, you have found some configuration samples for a sever, a client and -a syslog relay. Hopefully, this will enable you to set up a similar system in many -environments. -<p>Please be warned that you defined some expiration dates for the certificates. -After they are reached, the certificates are no longer valid and rsyslog will NOT -accept them. At that point, syslog messages will no longer be transmitted (and rsyslogd -will heavily begin to complain). So it is a good idea to make sure that you renew the -certificates before they expire. Recording a reminder somewhere is probably a good -idea. -<p>If you have any more questions, please visit the <a href="http://kb.monitorware.com/rsyslog-f40.html">rsyslog forum</a> and simply ask ;) -<h2>Copyright</h2> -<p>Copyright (c) 2008 <a href="http://www.adiscon.com/en/people/rainer-gerhards.php">Rainer -Gerhards</a> and -<a href="http://www.adiscon.com/en/">Adiscon</a>.</p> -<p> Permission is granted to copy, distribute and/or modify this -document under the terms of the GNU Free Documentation License, Version -1.2 or any later version published by the Free Software Foundation; -with no Invariant Sections, no Front-Cover Texts, and no Back-Cover -Texts. A copy of the license can be viewed at -<a href="http://www.gnu.org/copyleft/fdl.html">http://www.gnu.org/copyleft/fdl.html</a>.</p> -</body></html> diff --git a/doc/tls_cert_udp_relay.html b/doc/tls_cert_udp_relay.html deleted file mode 100644 index f4740ce..0000000 --- a/doc/tls_cert_udp_relay.html +++ /dev/null @@ -1,105 +0,0 @@ -<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> -<html><head><title>TLS-protected syslog: UDP relay setup</title> -</head> -<body> - -<h1>Encrypting Syslog Traffic with TLS (SSL)</h1> -<p><small><i>Written by <a href="http://www.adiscon.com/en/people/rainer-gerhards.php">Rainer -Gerhards</a> (2008-07-03)</i></small></p> - -<ul> -<li><a href="rsyslog_secure_tls.html">Overview</a> -<li><a href="tls_cert_scenario.html">Sample Scenario</a> -<li><a href="tls_cert_ca.html">Setting up the CA</a> -<li><a href="tls_cert_machine.html">Generating Machine Certificates</a> -<li><a href="tls_cert_server.html">Setting up the Central Server</a> -<li><a href="tls_cert_client.html">Setting up syslog Clients</a> -<li><a href="tls_cert_udp_relay.html">Setting up the UDP syslog relay</a> -<li><a href="tls_cert_summary.html">Wrapping it all up</a> -</ul> - -<h3>Setting up the UDP syslog relay</h3> -<p>In this step, we configure the UDP relay ada.example.net. -As a reminder, that machine relays messages from a local router, which only -supports UDP syslog, to the central syslog server. The router does not talk -directly to it, because we would like to have TLS protection for its sensitve -logs. If the router and the syslog relay are on a sufficiently secure private -network, this setup can be considered reasonable secure. In any case, it is the -best alternative among the possible configuration scenarios. -<span style="float: left"> -<script type="text/javascript"><!-- -google_ad_client = "pub-3204610807458280"; -/* rsyslog doc inline */ -google_ad_slot = "5958614527"; -google_ad_width = 125; -google_ad_height = 125; -//--> -</script> -<script type="text/javascript" -src="http://pagead2.googlesyndication.com/pagead/show_ads.js"> -</script> -</span> -<p><center><img src="tls_cert_100.jpg"></center> -<p>Steps to do: -<ul> -<li>make sure you have a functional CA (<a href="tls_cert_ca.html">Setting up the CA</a>) -<li>generate a machine certificate for ada.example.net (follow instructions in - <a href="tls_cert_machine.html">Generating Machine Certificates</a>) -<li>make sure you copy over ca.pem, machine-key.pem ad machine-cert.pem to the client. -Ensure that no user except root can access them (<b>even read permissions are really bad</b>). -<li>configure the client so that it checks the server identity and sends messages only -if the server identity is known. -</ul> -<p>These were essentially the same steps as for any -<a href="tls_cert_client.html">TLS syslog client</a>. We now need to add the -capability to forward the router logs: -<ul> -<li>make sure that the firewall rules permit message recpetion on UDP port 514 (if you use -a non-standard port for UDP syslog, make sure that port number is permitted). -<li>you may want to limit who can send syslog messages via UDP. A great place to do this -is inside the firewall, but you can also do it in rsyslog.conf via an $AllowedSender -directive. We have used one in the sample config below. Please be aware that this is -a kind of weak authentication, but definitely better than nothing... -<li>add the UDP input plugin to rsyslog's config and start a UDP listener -<li>make sure that your forwarding-filter permits to forward messages received -from the remote router to the server. In our sample scenario, we do not need to -add anything special, because all messages are forwarded. This includes messages -received from remote hosts. -</ul> -<p><b>At this point, please be reminded once again that your security needs may be quite different from -what we assume in this tutorial. Evaluate your options based on your security needs.</b> -<h3>Sample syslog.conf</h3> -<p>Keep in mind that this rsyslog.conf sends messages via TCP, only. Also, we do not -show any rules to write local files. Feel free to add them. -<code><pre> -# start a UDP listener for the remote router -$ModLoad imudp # load UDP server plugin -$AllowedSender UDP, 192.0.2.1 # permit only the router -$UDPServerRun 514 # listen on default syslog UDP port 514 - -# make gtls driver the default -$DefaultNetstreamDriver gtls - -# certificate files -$DefaultNetstreamDriverCAFile /rsyslog/protected/ca.pem -$DefaultNetstreamDriverCertFile /rsyslog/protected/machine-cert.pem -$DefaultNetstreamDriverKeyFile /rsyslog/protected/machine-key.pem - -$ActionSendStreamDriverAuthMode x509/name -$ActionSendStreamDriverPermittedPeer central.example.net -$ActionSendStreamDriverMode 1 # run driver in TLS-only mode -*.* @@central.example.net:10514 # forward everything to remote server -</pre></code> -<p><font color="red"><b>Be sure to safeguard at least the private key (machine-key.pem)!</b> -If some third party obtains it, you security is broken!</font> -<h2>Copyright</h2> -<p>Copyright © 2008 <a href="http://www.adiscon.com/en/people/rainer-gerhards.php">Rainer -Gerhards</a> and -<a href="http://www.adiscon.com/en/">Adiscon</a>.</p> -<p> Permission is granted to copy, distribute and/or modify this -document under the terms of the GNU Free Documentation License, Version -1.2 or any later version published by the Free Software Foundation; -with no Invariant Sections, no Front-Cover Texts, and no Back-Cover -Texts. A copy of the license can be viewed at -<a href="http://www.gnu.org/copyleft/fdl.html">http://www.gnu.org/copyleft/fdl.html</a>.</p> -</body></html> diff --git a/doc/troubleshoot.html b/doc/troubleshoot.html deleted file mode 100644 index 0f0c7fc..0000000 --- a/doc/troubleshoot.html +++ /dev/null @@ -1,164 +0,0 @@ -<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> -<html><head><title>troubleshooting rsyslog</title></head> -<body> -<h2>troubleshooting rsyslog</h2> -<p><b>Having trouble with <a href="http://www.rsyslog.com">rsyslog</a>?</b> -This page provides some tips on where to look for help and what to do -if you need to ask for assistance. This page is continously being expanded. -<p>Useful troubleshooting ressources are: -<ul> -<li>The <a href="http://www.rsyslog.com/doc">rsyslog documentation</a> - note that the online version always covers -the most recent development version. However, there is a version-specific -doc set in each tarball. If you installed rsyslog from a package, there usually -is a rsyslog-doc package, that often needs to be installed separately. -<li>The <a href="http://wiki.rsyslog.com">rsyslog wiki</a> provides user tips and experiences. -<li>Check <a href="http://bugzilla.adiscon.com">the bugzilla</a> to see if your problem is a known -(and even fixed ;)) bug. -</ul> -<p><b>Malformed Messages and Message Properties</b> -<p>A common trouble source are <a href="syslog_parsing.html">ill-formed syslog messages</a>, which -lead to to all sorts of interesting problems, including malformed hostnames and dates. -Read the quoted guide to find relief. A common symptom is that the %HOSTNAME% property is -used for generating dynafile names, but some glibberish shows up. This is caused by the -malformed syslog messages, so be sure to read the -<a href="syslog_parsing.html">guide</a> if you face that problem. Just let me add that the -common work-around is to use %FROMHOST% or %FROMHOST-IP% instead. These do not take the -hostname from the message, but rather use the host that sent the message (taken from -the socket layer). Of course, this does not work over NAT or relay chains, where the -only cure is to make sure senders emit well-formed messages. -<p><b>Configuration Problems</b> -<p>Rsyslog 3.21.1 and above has been enhanced to support extended configuration checking. -It offers a special command line switch (-N1) that puts it into "config verfication mode". -In that mode, it interprets and check the configuration file, but does not startup. This -mode can be used in parallel to a running instance of rsyslogd. -<p>To enable it, run rsyslog interactively as follows: -<p><b><i>/path/to/rsyslogd -f/path/to/config-file -N1</i></b> -<p>You should also specify other options you usually give (like -c3 and whatever else). -Any problems experienced are reported to stderr [aka "your screen" (if not redirected)]. -<p><b>Configuration Graphs</b> -<p>Starting with rsyslog 4.3.1, the -"<a href="rsconf1_generateconfiggraph.html">$GenerateConfigGraph</a>" -command is supported, a very valuable troubleshooting tool. It permits to -generate a graph of how rsyslogd understood its configuration file. It is assumed that -many configuration issues can easily be detected just by looking at the configuration graph. -Full details of how to generate the graphs, and what to look for can be found in the -"<a href="rsconf1_generateconfiggraph.html">$GenerateConfigGraph</a>" -manual page. -<p><b>Asking for Help</b> -<p>If you can't find the answer yourself, you should look at these places for -community help. -<ul> -<li>The <a href="http://kb.monitorware.com/rsyslog-f40.html">rsyslog forum</a>. This is -the preferred method of obtaining support. -<li>The <a href="http://lists.adiscon.net/mailman/listinfo/rsyslog">rsyslog mailing list</a>. -This is a low-volume list which occasional gets traffic spikes. -The mailing list is probably a good place for complex questions. -</ul> -<p><b>Debug Log</b> -<p>If you ask for help, there are chances that we need to ask for an rsyslog debug log. -The debug log is a detailled report of what rsyslog does during processing. As such, it may -even be useful for your very own troubleshooting. People have seen things inside their debug -log that enabled them to find problems they did not see before. So having a look at the -debug log, even before asking for help, may be useful. -<p>Note that the debug log contains most of those things we consider useful. This is a lot -of information, but may still be too few. So it sometimes may happen that you will be asked -to run a specific version which has additional debug output. Also, we revise from time to -time what is worth putting into the standard debug log. As such, log content may change -from version to version. We do not guarantee any specific debug log contents, so do not -rely on that. The amount of debug logging can also be controlled via some environment -options. Please see <a href="debug.html">debugging support</a> for further details. -<p>In general, it is advisable to run rsyslogd in the foreground to obtain the log. -To do so, make sure you know which options are usually used when you start rsyslogd -as a background daemon. Let's assume "-c3" is the only option used. Then, do the following: -<ul> -<li>make sure rsyslogd as a daemon is stopped (verify with ps -ef|grep rsyslogd) -<li>make sure you have a console session with root permissions -<li>run rsyslogd interactively: /sbin/rsyslogd ..your options.. -dn > logfile -<br>where "your options" is what you usually use. /sbin/rsyslogd is the full path -to the rsyslogd binary (location different depending on distro). -In our case, the command would be -<br>/sbin/rsyslogd -c3 -dn > logfile -<li>press ctrl-C when you have sufficient data (e.g. a device logged a record) -<br><b>NOTE: rsyslogd will NOT stop automatically - you need to ctrl-c out of it!</b> -<li>Once you have done all that, you can review logfile. It contains the debug output. -<li>When you are done, make sure you re-enable (and start) the background daemon! -</ul> -<p>If you need to submit the logfile, you may want to check if it contains any -passwords or other sensitive data. If it does, you can change it to some <b>consistent</b> -meaningless value. <b>Do not delete the lines</b>, as this renders the debug log -unusable (and makes Rainer quite angry for wasted time, aka significantly reduces the chance -he will remain motivated to look at your problem ;)). For the same reason, make sure -whatever you change is change consistently. Really! -<p>Debug log file can get quite large. Before submitting them, it is a good idea to zip them. -Rainer has handled files of around 1 to 2 GB. If your's is larger ask before submitting. Often, -it is sufficient to submit the first 2,000 lines of the log file and around another 1,000 around -the area where you see a problem. Also, -ask you can submit a file via private mail. Private mail is usually a good way to go for large files -or files with sensitive content. However, do NOT send anything sensitive that you do not want -the outside to be known. While Rainer so far made effort no to leak any sensitive information, -there is no guarantee that doesn't happen. If you need a guarantee, you are probably a -candidate for a <a href="professional_support.html">commercial support contract</a>. Free support -comes without any guarantees, include no guarantee on confidentiality -[aka "we don't want to be sued for work were are not even paid for ;)]. -<b>So if you submit debug logs, do so at your sole risk</b>. By submitting them, you accept -this policy. -<p><b>Segmentation Faults</b> -<p>Rsyslog has a very rapid development process, complex capabilities and now gradually gets -more and more exposure. While we are happy about this, it also has some bad effects: some -deployment scenarios have probably never been tested and it may be impossible to test -them for the development team because of resources needed. So while we try to avoid this, -you may see a serious problem during deployments in demanding, non-standard, environments -(hopefully not with a stable version, but chances are good you'll run into troubles with -the development versions). -<p>Active support from the user base is very important to help us track down those things. -Most often, serious problems are the result of some memory misadressing. During development, -we routinely use valgrind, a very well and capable memory debugger. This helps us to create -pretty clean code. But valgrind can not detect everything, most importantly not code pathes -that are never executed. So of most use for us is information about aborts and abort locations. -<p>Unforutnately, faults rooted in adressing errors typically show up only later, so the -actual abort location is in an unrelated spot. To help track down the original spot, -<a href="http://www.gnu.org/software/hello/manual/libc/Heap-Consistency-Checking.html">libc -later than 5.4.23 offers support</a> for finding, and possible temporary relief from it, -by means of the MALLOC_CHECK_ environment variable. Setting it to 2 is a useful troubleshooting -aid for us. It will make the program abort as soon as the check routines detect anything -suspicious (unfortunately, this may still not be the root cause, but hopefully closer to it). -Setting it to 0 may even make some problems disappear (but it will NOT fix them!). -With functionality comes cost, and so exporting MALLOC_CHECK_ without need comes at -a performance penalty. However, we strongly recommend adding this instrumentation to your -test environment should you see any serious problems. Chances are good it will help us -interpret a dump better, and thus be able to quicker craft a fix. -<p>In order to get useful information, we need some backtrace of the abort. First, you need -to make sure that a core file is created. Under Fedora, for example, that means you need -to have an "ulimit -c unlimited" in place. -<p>Now let's assume you got a core file (e.g. in /core.1234). So what to do next? Sending a -core file to us is most often pointless - we need to have the exact same system configuration in -order to interpret it correctly. Obviously, chances are extremely slim for this to be. So we would -appreciate if you could extract the most important information. This is done as follows: -<ul> -<li>$gdb /path/to/rsyslogd -<li>$info thread -<li>you'll see a number of threads (in the range 0 to n with n being the highest number). For - <b>each</b> of them, do the following (let's assume that i is the thread number): - <ul> - <li>$ thread i (e.g. thread 0, thread 1, ...) - <li>$bt - </ul> -<li>then you can quit gdb with "$q" -</ul> -<p>Then please send all information that gdb spit out to the development team. It is best to first -ask on the forum or mailing list on how to do that. The developers will keep in contact with you -and, I fear, will probably ask for other things as well ;) -<p>Note that we strive for highest reliability of the engine even in unusual deployment scenarios. -Unfortunately, this is hard to achieve, especially with limited resources. So we are depending on -cooperation from users. This is your chance to make a big contribution to the project without the -need to program or do anything else except get a problem solved ;) -<p>[<a href="manual.html">manual index</a>] -[<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> -<p><font size="2">This documentation is part of the -<a href="http://www.rsyslog.com/">rsyslog</a> project.<br> -Copyright © 2008-2010 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and -<a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL -version 3 or higher.</font></p> -</body> -</html> - diff --git a/doc/v3compatibility.html b/doc/v3compatibility.html deleted file mode 100644 index 1c15350..0000000 --- a/doc/v3compatibility.html +++ /dev/null @@ -1,196 +0,0 @@ -<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> -<html><head><title>Compatibility notes for rsyslog v3</title> - -<meta name="KEYWORDS" content="syslog, mysql, syslog to mysql, howto"></head> -<body> -<h1>Compatibility Notes for rsyslog v3</h1> -<p><small><i>Written by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> -(2008-03-28)</i></small></p> -<p>Rsyslog aims to be a drop-in replacement for sysklogd. -However, version 3 has some considerable enhancements, which lead to -some backward compatibility issues both in regard to sysklogd and -rsyslog v1 and v2. Most of these issues are avoided by default by not -specifying the -c option on the rsyslog command line. That will enable -backwards-compatibility mode. However, please note that things may be -suboptimal in backward compatibility mode, so the advise is to work -through this document, update your rsyslog.conf, remove the no longer -supported startup options and then add -c3 as the first option to the -rsyslog command line. That will enable native mode.</p> -<p>Please note that rsyslogd helps you during that process by -logging appropriate messages about compatibility mode and -backwards-compatibility statemtents automatically generated. You may -want your syslogd log for those. They immediately follow rsyslogd's -startup message.</p> -<h2>Inputs</h2> -<p>With v2 and below, inputs were automatically started together -with rsyslog. In v3, inputs are optional! They come in the form of -plug-in modules. -<font color="#ff0000"><b>At least one input module -must be loaded to make rsyslog do any useful work.</b></font> -The config file directives doc briefly lists which config statements -are available by which modules.</p> -<p>It is suggested that input modules be loaded in the top part -of the config file. Here is an example, also highlighting the most -important modules:</p> -<p><b>$ModLoad immark # provides --MARK-- -message capability<br> -$ModLoad imudp # provides UDP syslog reception<br> -$ModLoad imtcp # provides TCP syslog reception<br> -</b><b>$ModLoad imgssapi # provides GSSAPI syslog -reception<br> -</b><b>$ModLoad imuxsock # provides support for local -system logging (e.g. -via logger command)<br> -$ModLoad imklog # provides kernel logging support (previously done -by rklogd)</b></p> -<h2>Command Line Options</h2> -<p>A number of command line options have been removed. New config -file directives have been added for them. The -h and -e option have -been removed even in compatibility mode. They are ignored but an -informative message is logged. Please note that -h was never supported -in v2, but was silently ignored. It disappeared some time ago in the -final v1 builds. It can be replaced by applying proper filtering inside -syslog.conf.</p> -<h2>-c option / Compatibility Mode</h2> -<p>The -c option is new and tells rsyslogd about the desired -backward compatibility mode. It must always be the first option on the -command line, as it influences processing of the other options. To use -the rsyslog v3 native -interface, specify -c3. To use compatibility mode , -either do not use -c at all or use -c<vers> where vers is -the -rsyslog version that it shall be compatible to. Use -c0 to be -command-line compatible to sysklogd.</p><p><span style="font-weight: bold;">Please note that rsyslogd issues warning messages if the -c3 command line option is not given.</span> -This is to alert you that your are running in compatibility mode. -Compatibility mode interfers with you rsyslog.conf commands and may -cause some undesired side-effects. It is meant to be used with a plain -old rsyslog.conf - if you use new features, things become messy. So the -best advise is to work through this document, convert your options and -config file and then use rsyslog in native mode. In order to aid you in -this process, rsyslog logs every compatibility-mode config file -directive it has generated. So you can simply copy them from your -logfile and paste them to the config.</p> -<h2>-e Option</h2> -This option is no longer supported, as the "last message repeated n -times" feature is now turned off by default. We changed this default -because this feature is causing a lot of trouble and we need to make it -either go away or change the way it works. For more information, please -see our dedicted <a href="http://www.rsyslog.com/PNphpBB2-viewtopic-p-1130.phtml">forum -thread on "last message repeated n times"</a>. This thread also -contains information on how to configure rsyslogd so that it continues -to support this feature (as long as it is not totally removed). -<h2>-m Option</h2> -<p>The -m command line option is emulated in compatibiltiy mode. -To replace it, use the following config directives (compatibility mode -auto-generates them):</p> -<p><b>$ModLoad immark<br> -$MarkMessagePeriod 1800 # 30 minutes</b></p> -<h2>-r Option</h2> -<p>Is no longer available in native mode. However, it -is -understood in compatibility mode (if no -c option is given). Use the <b>$UDPSeverRun -<port></b> config file directives. You can now also -set the local address the server should listen to via <b>$UDPServerAddress -<ip></b> config directive.</p> -<p>The following example configures an UDP syslog server at the -local address 192.0.2.1 on port 514:</p> -<p><b>$ModLoad imudp<br> -$UDPServerAddress 192.0.2.1 # this MUST be before the $UDPServerRun -directive!<br> -$UDPServerRun 514</b></p> -<p>"$UDPServerAddress *" means listen on all local interfaces. -This is the default if no directive is specified.</p> -<p>Please note that now multiple listeners are supported. For -example, you can do the following:</p> -<p><b>$ModLoad imudp<br> -$UDPServerAddress 192.0.2.1 # this MUST be before the $UDPServerRun -directive!<br> -$UDPServerRun 514<br> -$UDPServerAddress * # all local interfaces<br> -$UDPServerRun 1514</b></p> -<p>These config file settings run two listeners: one -at 192.0.2.1:514 and one on port 1514, which listens on all local -interfaces.</p> -<h2>Default port for UDP (and TCP) Servers</h2> -<p>Please note that with pre-v3 rsyslogd, a service database -lookup was made when a UDP server was started and no port was -configured. Only if that failed, the IANA default of 514 was used. For -TCP servers, this lookup was never done and 514 always used if no -specific port was configured. For consitency, both TCP and UDP now use -port 514 as default. If a lookup is desired, you need to specify it in -the "Run" directive, e.g. "<i>$UDPServerRun syslog</i>".</p> -<h2>klogd</h2> -<p>klogd has (finally) been replaced by a loadable input module. -To enable klogd functionality, do</p> -<p><b>$ModLoad imklog</b></p> -<p>Note that this can not be handled by the compatibility layer, -as klogd was a separate binary.A limited set of klogd command line -settings is now supported -via rsyslog.conf. That set of configuration directives is to be -expanded. </p> -<h2>Output File Syncing</h2> -Rsyslogd tries to keep as compatible to -stock syslogd as possible. As such, it retained stock syslogd's default -of syncing every file write if not specified otherwise (by placing a -dash in front of the output file name). While this was a useful feature -in past days where hardware was much less reliable and UPS seldom, this -no longer is useful in today's worl. Instead, the syncing is a high -performace hit. With it, rsyslogd writes files around 50 *times* slower -than without it. It also affects overall system performance due to the -high IO activity. In rsyslog v3, syncing has been turned off by -default. This is done via a specific configuration directive -"$ActionFileEnableSync on/off" which is off by default. So even if -rsyslogd finds sync selector lines, it ignores them by default. In -order to enable file syncing, the administrator must specify -"$ActionFileEnableSync on" at the top of rsyslog.conf. This ensures -that syncing only happens in some installations where the administrator -actually wanted that (performance-intense) feature. In the fast -majority of cases (if not all), this dramatically increases rsyslogd -performance without any negative effects. -<h2>Output File Format</h2> -<p>Rsyslog supports high precision RFC 3339 timestamps and puts these into -local log files by default. This is a departure from previous syslogd -behaviour. We decided to sacrify some backward-compatibility in an -effort to provide a better logging solution. Rsyslog has been -supporting the high-precision timestamps for over three years as of -this writing, but nobody used them because they were not default (one -may also assume that most people didn't even know about them). Now, we -are writing the great high-precision time stamps, which greatly aid in -getting the right sequence of logging events. If you do not like that, -you can easily turn them off by placing -</p><p style="font-weight: bold;"><code>$ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat</code> -</p><p>right at the start of your rsyslog.conf. This will use the -previous format. Please note that the name is case-sensitive and must -be specificed exactly as shown above. Please also note that you can of -course use any other format of your liking. To do so, simply specify -the template to use or set a new default template via the -$ActionFileDefaultTemplate directive. Keep in mind, though, that -templates must be defined before they are used.</p><p>Keep in mind that -when receiving messages from remote hosts, the timestamp is just as -precise as the remote host provided it. In most cases, this means you -will only a receive a standard timestamp with second precision. If -rsyslog is running at the remote end, you can configure it to provide -high-precision timestamps (see below).</p><h2>Forwarding Format</h2><p>When -forwarding messages to remote syslog servers, rsyslogd by default uses -the plain old syslog format with second-level resolution inside the -timestamps. We could have made it emit high precision timestamps. -However, that would have broken almost all receivers, including earlier -versions of rsyslog. To avoid this hassle, high-precision timestamps -need to be explicitely enabled. To make this as painless as possible, -rsyslog comes with a canned template that contains everything -necessary. To enable high-precision timestamps, just use:</p><p style="font-weight: bold;"><code>$ActionForwardDefaultTemplate RSYSLOG_ForwardFormat # for plain TCP and UDP</code></p><p style="font-weight: bold;"><code>$ActionGSSForwardDefaultTemplate RSYSLOG_ForwardFormat # for GSS-API</code></p><p>And, of course, you can always set different forwarding formats by just specifying the right template.</p><p>If -you are running in a system with only rsyslog 3.12.5 and above in the -receiver roles, it is suggested to add one (or both) of the above -statements to the top of your rsyslog.conf (but after the $ModLoad's!) -- that will enable you to use the best in timestamp support availble. -Please note that when you use this format with other receivers, they -will probably become pretty confused and not detect the timestamp at -all. In earlier rsyslog versions, for example, that leads to -duplication of timestamp and hostname fields and disables the detection -of the orignal hostname in a relayed/NATed environment. So use the new -format with care. </p><h2>Queue Modes for the Main Message Queue</h2> -<p>Either "FixedArray" or "LinkedList" is recommended. "Direct" -is available, but should not be used except for a very good reason -("Direct" disables queueing and will potentially lead to message loss -on the input side).</p> -</body></html> diff --git a/doc/v4compatibility.html b/doc/v4compatibility.html deleted file mode 100644 index 2a51ade..0000000 --- a/doc/v4compatibility.html +++ /dev/null @@ -1,96 +0,0 @@ -<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> -<html><head><title>Compatibility notes for rsyslog v4</title> -</head> -<body> -<h1>Compatibility Notes for rsyslog v4</h1> -<p><small><i>Written by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> -(2009-07-15)</i></small></p> -<p>The changes introduced in rsyslog v4 are numerous, but not very intrusive. -This document describes things to keep in mind when moving from v3 to v4. It -does not list enhancements nor does it talk about compatibility concerns introduced -by v3 (for this, see the <a href="v3compatibility.html">rsyslog v3 compatibility notes</a>). -<h2>HUP processing</h2> -<p>With v3 and below, rsyslog used the traditional HUP behaviour. That meant that -all output files are closed and the configuration file is re-read and the new configuration -applied. -<p>With a program as simple and static as sysklogd, this was not much of an issue. The -most important config settings (like udp reception) of a traditional syslogd can not be -modified via the configuration file. So a config file reload only meant setting up a new set of filters. It also didn't account as problem that while doing so messages may be lost - without -any threading and queuing model, a traditional syslogd will potentially always loose -messages, so it is irrelevant if this happens, too, during the short config re-read -phase. -<p>In rsyslog, things are quite different: the program is more or less a framework into -which loadable modules are loaded as needed for a particular configuration. The software -that will acutally be running is taylored via the config file. Thus, a re-read of -the config file requires a full, very heavy restart, because the software acutally -running with the new config can be totally different from what ran with the old config. -<p>Consequently, the traditional HUP is a very heavy operation and may even cause some -data loss because queues must be shut down, listeners stopped and so on. Some of these -operations (depending on their configuration) involve intentional message loss. The operation -also takes up a lot of system resources and needs quite some time (maybe seconds) to be -completed. During this restart period, the syslog subsytem is not fully available. -<p>From the software developer's point of view, the full restart done by a HUP is rather complex, -especially if user-timeout limits set on action completion are taken into consideration (for -those in the know: at the extreme ends this means we need to cancel threads as a last resort, -but than we need to make sure that such cancellation does not happen at points where it -would be fatal for a restart). A regular restart, where the process is actually terminated, is -much less complex, because the operating system does a full cleanup after process termination, -so rsyslogd does not need to take care for exotic cleanup cases and leave that to the OS. -In the end result, restart-type HUPs clutter the code, increase complexity (read: add bugs) -and cost performance. -<p>On the contrary, a HUP is typically needed for log rotation, and the real desire is -to close files. This is a non-disruptive and very lightweigth operation. -<p>Many people have said that they are used to HUP the syslogd to apply configuration -changes. This is true, but it is questionable if that really justifies all the cost that -comes with it. After all, it is the difference between typing -<pre> -$ kill -HUP `cat /var/run/rsyslogd.pid` -</pre> -versus -<pre> -$ /etc/init.d/rsyslog restart -</pre> -Semantically, both is mostly the same thing. The only difference is that with the restart -command rsyslogd can spit config error message to stderr, so that the user is able to see -any problems and fix them. With a HUP, we do not have access to stderr and thus can log -error messages only to their configured destinations; exprience tells that most users -will never find them there. What, by the way, is another strong argument against -restarting rsyslogd by HUPing it. -<p>So a restart via HUP is not strictly necessary -and most other deamons require that a restart command is typed in if a restart is required. -<p>Rsyslog will follow this paradigm in the next versions, resulting in many benefits. In v4, -we provide some support for the old-style semantics. We introduced a setting $HUPisRestart -which may be set to "on" (tradional, heavy operation) -or "off" (new, lightweight "file close only" operation). -The initial versions had the default set to traditional behavior, but starting with 4.5.1 -we are now using the new behavior as the default. -<p>Most importantly, <b>this may break some scripts</b>, but my sincere belief is that -there are very few scripts that automatically <b>change</b> rsyslog's config and then do a -HUP to reload it. Anyhow, if you have some of these, it may be a good idea to change -them now instead of turning restart-type HUPs on. Other than that, one mainly needs -to change the habit of how to restart rsyslog after a configuration change. -<p><b>Please note that restart-type HUP is depricated and will go away in rsyslog v5.</b> -So it is a good idea to become ready for the new version now and also enjoy some of the -benefits of the "real restart", like the better error-reporting capability. -<p>Note that code complexity reduction (and thus performance improvement) needs the restart-type -HUP code to be removed, so these changes can (and will) only happen in version 5. -<h2>outchannels</h2> -Note: as always documented, outchannels are an experimental feature that may be -removed and/or changed in the future. -There is one concrete change done starting with 4.6.7: let's assume an -outchannel "mychannel" was defined. Then, this channel could be used inside an -<code> -*.* $mychannel -</code> -This is still supported and will remain to be supported in v4. However, there is -a new variant which explicitely tells this is to be handled by omfile. This new -syntax is as follows: -<code> -*.* :omfile:$mychannel -</code> -Note that future versions, specifically starting with v6, the older syntax is no -longer supported. So users are strongly advised to switch to the new syntax. As an -aid to the conversion process, rsyslog 4.7.4 and above issue a warning message -if the old-style directive is seen -- but still accept the old syntax without -any problems. -</body></html> diff --git a/doc/v5compatibility.html b/doc/v5compatibility.html deleted file mode 100644 index fc4289c..0000000 --- a/doc/v5compatibility.html +++ /dev/null @@ -1,36 +0,0 @@ -<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> -<html><head><title>Compatibility notes for rsyslog v5</title> -</head> -<body> -<h1>Compatibility Notes for rsyslog v5</h1> -<p><small><i>Written by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> -(2009-07-15)</i></small></p> -<p>The changes introduced in rsyslog v5 are numerous, but not very intrusive. -This document describes things to keep in mind when moving from v4 to v5. It -does not list enhancements nor does it talk about compatibility concerns introduced -by earlier versions (for this, see their respective compatibility documents). -<h2>HUP processing</h2> -<p>The $HUPisRestart directive is supported by some early v5 versions, but has been removed -in 5.1.3 and above. That means that restart-type HUP processing is no longer -available. This processing was redundant and had a lot a drawbacks. -For details, please see the -<a href="v4compatibility.html">rsyslog v4 compatibility notes</a> which elaborate -on the reasons and the (few) things you may need to change. -<p>Please note that starting with 5.8.11 HUP will also requery the local hostname. -<h2>Queue on-disk format</h2> -<p>The queue information file format has been changed. When upgrading from v4 to -v5, make sure that the queue is emptied and no on-disk structure present. We did -not go great length in understanding the old format, as there was too little demand -for that (and it being quite some effort if done right). -<h2>Queue Worker Thread Shutdown</h2> -<p>Previous rsyslog versions had the capability to "run" on zero queue worker -if no work was required. This was done to save a very limited number of resources. However, -it came at the price of great complexity. In v5, we have decided to let a minium of one -worker run all the time. The additional resource consumption is probably not noticable at -all, however, this enabled us to do some important code cleanups, resulting in faster -and more reliable code (complex code is hard to maintain and error-prone). From the -regular user's point of view, this change should be barely noticable. I am including the -note for expert users, who will notice it in rsyslog debug output and other analysis tools. -So it is no error if each queue in non-direct mode now always runs at least one worker -thread. -</body></html> diff --git a/doc/v6compatibility.html b/doc/v6compatibility.html deleted file mode 100644 index 7ce8c00..0000000 --- a/doc/v6compatibility.html +++ /dev/null @@ -1,198 +0,0 @@ -<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> -<html><head><title>Compatibility notes for rsyslog v6</title> -</head> -<body> -<h1>Compatibility Notes for rsyslog v6</h1> -<p><small><i>Written by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> -(2011-10-27)</i></small></p> -<p> -This document describes things to keep in mind when moving from v5 to v6. It -does not list enhancements nor does it talk about compatibility concerns introduced -by earlier versions (for this, see their respective compatibility documents). Its focus -is primarily on what you need to know if you used a previous version and want to use the -current one without hassle. -<p>Version 6 offers a better config language and some other improvements. -As the config system has many ties into the rsyslog engine AND all plugins, -the changes are somewhat intrusive. Note, however, that core processing has -not been changed much in v6 and will not. So once the configuration is loaded, -the stability of v6 is quite comparable to v5. -</p> -<h2>Property "pri-text"</h2> -<p>Traditionally, this property did not only return the textual form -of the pri ("local0.err"), but also appended the numerical value to it -("local0.err<133>"). This sounds odd and was left unnoticed for some years. -In October 2011, this odd behaviour was brought up on the rsyslog mailing list -by Gregory K. Ruiz-Ade. Code review showed that the behaviour was intentional, -but no trace of what the intention was when it was introduced could be found. -The documentation was also unclear, it said no numerical value was present, -but the samples had it. We agreed that the additional numerical value is -of disadvantage. We also guessed that this property is very rarely being used, -otherwise the problem should have been raised much earlier. However, we -didn't want to change behaviour in older builds. So v6 was set to clean up -the situation. In v6, text-pri will always return the textual part only -("local0.err") and the numerical value will not be contained any longer inside -the string. If you actually need that value, it can fairly easily be added -via the template system. -<p><b>If you have used this property previously and relied on the numerical -part, you need to update your rsyslog configuration files.</b> -<h2>Plugin ABI</h2> -<p>The plugin interface has considerably been changed to support the new -config language. All plugins need to be upgraded. This usually does not require -much coding. However, if the new config language shall be supported, more -changes must be made to plugin code. All project-supported plugins have been -upgraded, so this compatibility issue is only of interest for you if you have -custom plugins or use some user-contributed plugins from the rsyslog project -that are not maintained by the project itself (omoracle is an example). Please -expect some further plugin instablity during the initial v6 releases. -<h2>RainerScript based rsyslog.conf</h2> -<p>A better config format was the main release target for rsyslog v6. It comes in the -flavor of so-called RainerScript -(<a href="http://blog.gerhards.net/2008/02/introducing-rainerscript-and-some.html">why the -name RainerScript?</a>). RainerScript supports legacy syslog.conf format, much as you know it -from other syslogd's (like sysklogd or the BSD syslogd's) as well as previous versions -of rsyslog. Initial work on RainerScript began in v4, and the if-construct was already -supported in v4 and v5. Version 6 has now taken this further. After long discussions we -decided to use the legacy format as a basis, and lightly extend it by native RainerScript -constructs. The main goal was to make sure that previous knowledge and config systems -could still be used while offering a much more intuitive and powerful way of configuring -rsyslog. -<p>RainerScript has been implemented from scratch and with new tools (flex/bison, for those in the -know). Starting with 6.3.3, this new config file processor replaces the legacy one. Note that -the new processor handles all formats, extended RainerScript as well as legacy syslog.conf format. -There are some legacy construct that were especially hard to translate. You'll read about them in -other parts of this document (especially outchannels, which require a format change). - -<p>In v6, all legacy formats are supported. In the long term, we may remove some of the ugly -rsyslog-specific constructs. Good candidates are all configuration commands starting with -a dollar sign, like "$ActionFileDefaultTemplate"). However, this will not be the case before -rsyslog v7 or (much more likely) v8/9. Right now, you also need to use these commands, because -not all have already been converted to the new RainerScript format. - -<p>In 6.3.3, the new parser is used, but almost none of the extended RainerScript capabilities -are available. They will incrementally be introduced with the following releases. Note that for -some features (most importantly if-then-else nested blocks), the v6 core engine is not -capable enough. It is our aim to provide a much better config language to as many rsyslog -users as quickly as possible. As such, we refrain from doing big engine changes in v6. This -in turn means we cannot introduce some features into RainerScript that we really want to see. -These features will come up with rsyslog v7, which will have even better flow control -capabilities inside the core engine. Note that v7 will fully support v6 RainerScript. -Let us also say that the v6 version is not a low-end quick hack: it offers full-fledged -syslog message processing control, capable of doing the best you can find inside the -industry. We just say that v7 will come up with even more advanced capabilites. -<p>Please note that we tried hard to make the RainerScript parser compatible with -all legacy config files. However, we may have failed in one case or another. So if you -experience problems during config processing, chances are there may be a problem -on the rsyslog side. In that case, please let us know. - -<p>Please see the -<a href="http://blog.gerhards.net/2011/07/rsyslog-633-config-format-improvements.html">blog -post about rsyslog 6.3.3 config format</a> for details of what is currently supported. - -<h2>compatibility mode</h2> -<p>Compatibility mode (specified via -c option) has been removed. This was a migration aid from -sysklogd and very early versions of rsyslog. As all major distros now have rsyslog as their -default, and thus ship rsyslog-compliant config files, there is no longer a need for -compatibility mode. Removing it provides easier to maintain code. Also, practice has shown -that many users were confused by compatibility mode (and even some package maintainers got -it wrong). So this not only cleans up the code but rather removes a frequent source of -error. -<p>It must be noted, though, that this means rsyslog is no longer a 100% drop-in replacement -for sysklogd. If you convert an extremely old system, you need to checks its config and -probably need to apply some very mild changes to the config file. -<h2>abort on config errors</h2> -<p>Previous versions accepted some malformedness inside the config file without aborting. This -could lead to some uncertainty about which configuration was actually running. In v6 there -are some situations where config file errors can not be ignored. In these cases rsyslog -emits error messages to stderr, and then exists with a non-zero exit code. It is important -to check for those cases as this means log data is potentially lost. -Please note that -the root problem is the same for earlier versions as well. With them, it was just harder -to spot why things went wrong (and if at all). -<h2>Default Batch Sizes</h2> -<p>Due to their positive effect on performance and comparatively low overhead, -default batch sizes have been increased. Starting with 6.3.4, the action queues -have a default batch size of 128 messages. -<h2>Default action queue enqueue timeout</h2> -<p>This timeout previously was 2seconds, and has been reduced to 50ms (starting with 6.5.0). This change -was made as a long timeout will caused delays in the associated main queue, something -that was quite unexpected to users. Now, this can still happen, but the effect is much -less harsh (but still considerable on a busy system). Also, 50ms should be fairly enough -for most output sources, except when they are really broken (like network disconnect). If -they are really broken, even a 2second timeout does not help, so we hopefully get the best -of both worlds with the new timeout. A specific timeout can of course still be configured, -it is just the timeout that changed. -<h2>outchannels</h2> -<p>Outchannels are a to-be-removed feature of rsyslog, at least as far as the config -syntax is concerned. Nevertheless, v6 still supports it, but a new syntax is required -for the action. Let's assume your outchannel is named "channel". The previous syntax was -<blockquote><code> -*.* $channel -</code> </blockquote> -This was deprecated in v5 and no longer works in v6. Instead, you need to specify -<blockquote><code> -*.* :omfile:$channel -</code></blockquote> -Note that this syntax is available starting with rsyslog v4. It is important to keep on your -mind that future versions of rsyslog will require different syntax and/or drop outchannel support -completely. So if at all possible, avoid using this feature. If you must use it, be prepared for -future changes and watch announcements very carefully. -<h2>ompipe default template</h2> -<p>Starting with 6.5.0, ompipe does no longer use the omfile default template. -Instead, the default template must be set via the module load statement. -An example is -<blockquote><code> -module(load="builtin:ompipe" template="myDefaultTemplate") -</code> </blockquote> -<p>For obvious reasons, the default template must be defined somewhere in -the config file, otherwise errors will happen during the config load -phase. -<h2>omusrmsg</h2> -<p>The omusrmsg module is used to send messages to users. In legacy-legacy -config format (that is the very old sysklogd style), it was suffucient to use -just the user name to call this action, like in this example: -<blockquote><code> -*.* rgerhards -</code> </blockquote> -This format is very ambigious and causes headache (see -<a href="http://blog.gerhards.net/2011/07/why-omusrmsg-is-evil-and-how-it-is.html">blog post -on omusrmsg</a> for details). Thus the format has been superseded by this syntax -(which is legacy format ;-)): -<blockquote><code> -*.* :omusrmsg:rgerhards -</code> </blockquote> -That syntax is supported since later subversions of version 4. -<p>Rsyslog v6 still supports the legacy-legacy format, but in a very strict -sense. For example, if multiple users or templates are given, no spaces -must be included in the action line. For example, this works up to v5, but no -longer in v6: -<blockquote><code> -*.* rgerhards, bgerhards -</code> </blockquote> -To fix it in a way that is compatible with pre-v4, use (note the removed space!): -<blockquote><code> -*.* rgerhards,bgerhards -</code> </blockquote> -Of course, it probably is better to understand in native v6 format: -<blockquote><code> -*.* action(type="omusrmsg" users="rgerhards, bgerhards") -</code> </blockquote> -As you see, here you may include spaces between user names. -<p>In the long term, legacy-legacy format will most probably totally disappear, -so it is a wise decision to change config files at least to the legacy -format (with ":omusrmsg:" in front of the name). - -<h2>Escape Sequences in Script-Based Filters</h2> -<p>In v5, escape sequences were very simplistic. Inside a string, "\x" meant -"x" with x being any character. This has been changed so that the usual set of -escapes is supported, must importantly "\n", "\t", "\xhh" (with hh being hex digits) -and "\ooo" with (o being octal digits). So if one of these sequences was used -previously, results are obviously different. However, that should not create any -real problems, because it is hard to envision why someone should have done that -(why write "\n" when you can also write "n"?). -<p>[<a href="manual.html">manual index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> -<p><font size="2">This documentation is part of the -<a href="http://www.rsyslog.com/">rsyslog</a> project.<br> -Copyright © 2011 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and -<a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL -version 2 or higher.</font></p> -</body></html> diff --git a/doc/v7compatibility.html b/doc/v7compatibility.html deleted file mode 100644 index da4772f..0000000 --- a/doc/v7compatibility.html +++ /dev/null @@ -1,138 +0,0 @@ -<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> -<html><head><title>Compatibility notes for rsyslog v7</title> -</head> -<body> -<h1>Compatibility Notes for rsyslog v7</h1> -This document describes things to keep in mind when moving from v6 to v7. It -does not list enhancements nor does it talk about compatibility concerns introduced -by earlier versions (for this, see their respective compatibility documents). Its focus -is primarily on what you need to know if you used v6 and want to use v7 without hassle. -<p>Version 7 builds on the new config language introduced in v6 and extends it. -Other than v6, it not just only extends the config language, but provides -considerable changes to core elements as well. The result is much more power and -ease of use as well (this time that is not contradictionary). -</p> -<h2>BSD-Style blocks</h2> -BSD style blocks are no longer supported (for good reason). See the -<a href="http://www.rsyslog.com/g/BSD">rsyslog BSD blocks info</a> -page for more information and how to upgrade your config. -<p>[<a href="manual.html">manual index</a>] [<a href="http://www.rsyslog.com/">rsyslog site</a>]</p> - -<h2>CEE-Properties</h2> -In rsyslog v6, CEE properties could not be used across disk-based queues. If this was -done, there content was reset. This was a missing feature in v6. In v7, this feature -has been implemented. Consequently, situations where the previous behaviour were -desired need now to be solved differently. We do not think that this will cause any -problems to anyone, especially as in v6 this was announced as a missing feature. - -<h2>omusrmsg: using just a username or "*" is deprecated</h2> -<p>In legacy config format, the asterisk denotes writing the message to all users. -This is usually used for emergency messages and configured like this: -<pre> -*.emerg * -</pre> -<p>Unfortunately, the use of this single character conflicts with other uses, for -example with the multiplication operator. While rsyslog up to versions v7.4 preserves the meaning of -asterisk as an action, it is deprecated and will probably be removed in future versions. -Consequently, a warning message is emitted. To make this warning go away, the action must -be explicitly given, as follows: -<pre> -*.emerg :omusrmsg:* -</pre> -<p>The same holds true for user names. For example -<pre> -*.emerg john -</pre> -<p>at a minimum should be rewritten as -<pre> -*.emerg :omusrmsg:john -</pre> -<p>Of course, for even more clarity the new RainerScript style of action can -also be used: -<pre> -*.emerg action(type="omusrmsg" users="john") -</pre> -<p>In Rainer's blog, there is more -<a href="http://blog.gerhards.net/2011/07/why-omusrmsg-is-evil-and-how-it-is.html">background -information on why omusrmsg needed to be changed</a> available. - -<h2>omruleset and discard (~) action are deprecated</h2> -<p>Both continue to work, but have been replaced by better alternatives. -<p>The discard action (tilde character) has been replaced by the "stop" -RainerScript directive. It is considered more intuitive and offers slightly -better performance. -<p>The omruleset module has been replaced by the "call" RainerScript directive. -Call permits to execute a ruleset like a subroutine, and does so with much -higher performance than omruleset did. Note that omruleset could be run off -an async queue. This was more a side than a desired effect and is not supported -by the call statement. If that effect was needed, it can simply be simulated by -running the called rulesets actions asynchronously (what in any case is the right -way to handle this). -<p>Note that the deprecated modules emit warning messages when being used. -They tell that the construct is deprecated and which statement is to be used -as replacement. This does <b>not</b> affect operations: both modules are still -fully operational and will not be removed in the v7 timeframe. - -<h2>Retries of output plugins that do not do proper replies</h2> -<p>Some output plugins may not be able to detect if their target is capable of -accepting data again after an error (technically, they always return OK when -TryResume is called). Previously, the rsyslog core engine suspended such an action -after 1000 succesive failures. This lead to potentially a large amount of -errors and error messages. Starting with 7.2.1, this has been reduced to 10 -successive failures. This still gives the plugin a chance to recover. In extreme -cases, a plugin may now enter suspend mode where it previously did not do so. -In practice, we do NOT expect that. -<h1>Notes for the 7.3/7.4 branch</h1> -<h2>"last message repeated n times" Processing</h2> -<p>This processing has been optimized and moved to the input side. This results -in usually far better performance and also de-couples different sources -from the same -processing. It is now also integrated in to the more generic rate-limiting -processing. -<h3>User-Noticable Changes</h3> -The code works almost as before, with two exceptions: -<ul> -<li>The supression amount can be different, as the new algorithm - precisely check's a single source, and while that source is being - read. The previous algorithm worked on a set of mixed messages - from multiple sources. -<li>The previous algorithm wrote a "last message repeated n times" message - at least every 60 seconds. For performance reasons, we do no longer do - this but write this message only when a new message arrives or rsyslog - is shut down. -</ul> -<p>Note that the new algorithms needs support from input modules. If old -modules which do not have the necessary support are used, duplicate -messages will most probably not be detected. Upgrading the module code is -simple, and all rsyslog-provided plugins support the new method, so this -should not be a real problem (crafting a solution would result in rather -complex code - for a case that most probably would never happen). -<h3>Performance Implications</h3> -<p>In general, the new method enables far faster output procesing. However, it -needs to be noted that the "last message repeated n" processing needs parsed -messages in order to detect duplicated. Consequently, if it is enabled the -parser step cannot be deferred to the main queue processing thread and -thus must be done during input processing. The changes workload distribution -and may have (good or bad) effect on the overall performance. If you have -a very high performance installation, it is suggested to check the performance -profile before deploying the new version. Note: for high-performance -environments it is highly recommended NOT to use "last message repeated n times" -processing but rather the other (more efficient) rate-limiting methods. These -also do NOT require the parsing step to be done during input processing. - -<h2>Stricter string-template Processing</h2> -<p>Previously, no error message for invalid string template parameters -was generated. -Rather a malformed template was generated, and error information emitted -at runtime. However, this could be quite confusing. Note that the new code -changes user experience: formerly, rsyslog and the affected -actions properly started up, but the actions did not produce proper -data. Now, there are startup error messages and the actions are NOT -executed (due to missing template due to template error). - -<p><font size="2">This documentation is part of the -<a href="http://www.rsyslog.com/">rsyslog</a> project.<br> -Copyright © 2011-2013 by <a href="http://www.gerhards.net/rainer">Rainer Gerhards</a> and -<a href="http://www.adiscon.com/">Adiscon</a>. Released under the GNU GPL -version 2 or higher.</font></p> -</body></html> diff --git a/doc/version_naming.html b/doc/version_naming.html deleted file mode 100644 index 3bfa19b..0000000 --- a/doc/version_naming.html +++ /dev/null @@ -1,130 +0,0 @@ -<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> -<html><head><title>rsyslog version naming</title></head> -<body> -<h1>Version Naming</h1> -<p style="font-weight: bold;">This is the proposal on how versions should be named in the future:</p><p>Rsyslog version naming has undergone a number of changes in -the past. Our sincere hopes is that the scheme outlined here will serve -us well for the future. In general, a three-number versioning scheme -with a potential development state indication is used. It follows this -pattern:</p> -<p>major.minor.patchlevel[-devstate]</p> -<p>where devstate has some forther structure: --<releaseReason><releaseNumber></p> -<p>All stable builds come without the devstate part. All unstable -development version come with it.</p> -<p>The <span style="font-weight: bold;">major</span> -version is incremented whenever something really important happens. A -single new feature, even if important, does not justify an increase in -the major version. There is no hard rule when the major version needs -an increment. It mostly is a soft factor, when the developers and/or -the community think there has been sufficient change to justify that. -Major version increments are expected to happen quite infrequently, -maybe around once a year. A major version increment has important -implications from the support side: without support contracts, the -current major version's last stable release and the last stable release -of the version immediately below it are supported (Adiscon, the rsyslog -sponsor, offers <a href="professional_support.html">support contracts</a> covering all other versions).</p> -<p>The <span style="font-weight: bold;">minor</span> version is -incremented whenever a non-trivial new feature is planned to be added. -Triviality of a feature is simply determined by time estimated to -implement a feature. If that's more than a few days, it is considered a -non-trivial feature. Whenever a new minor version is begun, the desired -feature is identified and will be the primary focus of that major.minor -version. Trivial features may justify a new minor version if they -either do not look trivial from the user's point of view or change -something quite considerable (so we need to alert users). A minor -version increment may also be done for some other good reasons that the -developers have.</p> -<p>The <span style="font-weight: bold;">patchlevel</span> is incremented whenever there is a bugfix or very minor feature added to a (stable or development) release.</p><p>The <span style="font-weight: bold;">devstate</span> -is important during development of a feature. It helps the developers -to release versions with new features to the general public and in the -hope that this will result in some testing. To understand how it works, -we need to look at the release cycle: As already said, at the start of -a new minor version, a new non-trivial feature to be implemented in -that version is selected. Development on this feature begins. At the -current pace of development, getting initial support for such a -non-trivial feature typically takes between two and four weeks. During -this time, new feature requests come in. Also, we may find out that it -may be just the right time to implement some not yet targeted feature -requests. A reason for this is that the minor release's feature focus -is easier to implement if the other feature is implemented first. This -is a quite common thing to happen. So development on the primary focus -may hold for a short period while we implement something else. Even -unrelated, but very trivial feature requests (maybe an hour's worth of -time to implement), may be done in between. Once we have implemented -these things, we would like to release as quickly as possible (even -more if someone has asked for the feature). So we do not like to wait -for the original focus feature to be ready (what could take maybe three -more weeks). As a result, we release the new features. But that version -will also include partial code of the focus feature. Typically this -doesn't hurt as long as noone tries to use it (what of course would -miserably fail). But still, part of the new code is already in it. When -we release such a "minor-feature enhanced" but "focus-feature not yet -completed" version, we need a way to flag it. In current thinking, that -is using a "<span style="font-weight: bold;">-mf<version></span>" <span style="font-weight: bold;">devstate</span> -in the version number ("mf" stands for "minor feature"). Version -numbers for -mf releases start at 0 for the first release and are -monotonically incremented. Once the focus feature has been fully -implemented, a new version now actually supporting that feature will be -released. Now, the release reason is changed to the well-know "<span style="font-weight: bold;">-rc<version></span>" -where "rc" stands for release candidate. For the first release -candidate, the version starts at 0 again and is incremented -monotonically for each subsequent release. Please note that a -rc0 may -only have bare functionality but later -rc's have a richer one. If new -minor features are implemented and released once we have reached rc -stage, still a new rc version is issued. The difference between "mf" -and "rc" is simply the presence of the desired feature. No support is -provided for -mf versions once the first -rc version has been released. -And only the most current -rc version is supported.</p><p>The -rc is -removed and the version declared stable when we think it has undergone -sufficient testing and look sufficiently well. Then, it'll turn into a -stable release. Stable minor releases never receive non-trivial new -features. There may be more than one -rc releases without a stable -release present at the same time. In fact, most often we will work on -the next minor development version while the previous minor version is -still a -rc because it is not yet considered sufficiently stable.</p><p>Note: <span style="font-weight: bold;">the -absence of the -devstate part indicates that a release is stable. -Following the same logic, any release with a -devstate part is unstable.</span></p><p>A quick sample: </p><p>4.0.0 -is the stable release. We begin to implement relp, moving to -major.minor to 4.1. While we develop it, someone requests a trivial -feature, which we implement. We need to release, so we will have -4.1.0-mf0. Another new feature is requested, move to 4.1.0-mf2. A first -version of RELP is implemented: 4.1.0-rc0. A new trivial feature is -implemented: 4.1.0-rc1. Relp is being enhanced: 4.1.0-rc2. We now feel -RELP is good enough for the time being and begin to implement TLS on -plain /Tcp syslog: logical increment to 4.2. Now another new feature in -that tree: 4.2.0-mf0. Note that we now have 4.0.0 (stable) and -4.1.0-rc2 and 4.1.0-mf0 (both devel). We find a big bug in RELP coding. -Two new releases: 4.1.0-rc3, 4.2.0-mf1 (the bug fix acts like a -non-focus feature change). We release TLS: 4.2.0-rc0. Another RELP bug -fix 4.1.0-rc4, 4.2.0-rc1. After a while, RELP is matured: 4.1.0 -(stable). Now support for 4.0.x stable ends. It, however, is still -provided for 3.x.x (in the actual case 2.x.x, because v3 was under the -old naming scheme and now stable v3 was ever released).</p><p style="font-weight: bold;">This is how it is done so far:</p><p>This document briefly outlines the strategy for naming -versions. It applies to versions 1.0.0 and above. Versions below that -are all unstable and have a different naming schema.</p> -<p><b>Please note that version naming is currently being -changed. There is a -<a href="http://blog.gerhards.net/2007/08/on-rsyslog-versions.html">blog -post about future rsyslog versions</a>.</b></p> -<p>The major version is incremented whenever a considerate, major -features have been added. This is expected to happen quite infrequently.</p> -<p>The minor version number is incremented whenever there is -"sufficient need" (at the discretion of the developers). There is a -notable difference between stable and unstable branches. The <b>stable -branch</b> always has a minor version number in the range from 0 -to 9. It is expected that the stable branch will receive bug and -security fixes only. So the range of minor version numbers should be -quite sufficient.</p> -<p>For the <b>unstable branch</b>, minor version -numbers always start at 10 and are incremented as needed (again, at the -discretion of the developers). Here, new minor versions include both -fixes as well as new features (hopefully most of the time). They are -expected to be released quite often.</p> -<p>The patch level (third number) is incremented whenever a -really minor thing must be added to an existing version. This is -expected to happen quite infrequently.</p> -<p>In general, the unstable branch carries all new development. -Once it concludes with a sufficiently-enhanced, quite stable version, a -new major stable version is assigned.</p> -</body></html> diff --git a/grammar/Makefile.in b/grammar/Makefile.in index 73648f1..da227b1 100644 --- a/grammar/Makefile.in +++ b/grammar/Makefile.in @@ -137,7 +137,6 @@ GREP = @GREP@ GSS_LIBS = @GSS_LIBS@ GUARDTIME_CFLAGS = @GUARDTIME_CFLAGS@ GUARDTIME_LIBS = @GUARDTIME_LIBS@ -HAVE_LIBGCRYPT_CONFIG = @HAVE_LIBGCRYPT_CONFIG@ HAVE_MYSQL_CONFIG = @HAVE_MYSQL_CONFIG@ HAVE_ORACLE_CONFIG = @HAVE_ORACLE_CONFIG@ HAVE_PGSQL_CONFIG = @HAVE_PGSQL_CONFIG@ @@ -158,14 +157,15 @@ LEXLIB = @LEXLIB@ LEX_OUTPUT_ROOT = @LEX_OUTPUT_ROOT@ LIBDBI_CFLAGS = @LIBDBI_CFLAGS@ LIBDBI_LIBS = @LIBDBI_LIBS@ -LIBEE_CFLAGS = @LIBEE_CFLAGS@ -LIBEE_LIBS = @LIBEE_LIBS@ LIBESTR_CFLAGS = @LIBESTR_CFLAGS@ LIBESTR_LIBS = @LIBESTR_LIBS@ LIBGCRYPT_CFLAGS = @LIBGCRYPT_CFLAGS@ +LIBGCRYPT_CONFIG = @LIBGCRYPT_CONFIG@ LIBGCRYPT_LIBS = @LIBGCRYPT_LIBS@ LIBLOGGING_CFLAGS = @LIBLOGGING_CFLAGS@ LIBLOGGING_LIBS = @LIBLOGGING_LIBS@ +LIBLOGGING_STDLOG_CFLAGS = @LIBLOGGING_STDLOG_CFLAGS@ +LIBLOGGING_STDLOG_LIBS = @LIBLOGGING_STDLOG_LIBS@ LIBLOGNORM_CFLAGS = @LIBLOGNORM_CFLAGS@ LIBLOGNORM_LIBS = @LIBLOGNORM_LIBS@ LIBM = @LIBM@ @@ -190,6 +190,8 @@ NM = @NM@ NMEDIT = @NMEDIT@ OBJDUMP = @OBJDUMP@ OBJEXT = @OBJEXT@ +OPENSSL_CFLAGS = @OPENSSL_CFLAGS@ +OPENSSL_LIBS = @OPENSSL_LIBS@ ORACLE_CFLAGS = @ORACLE_CFLAGS@ ORACLE_LIBS = @ORACLE_LIBS@ OTOOL = @OTOOL@ @@ -363,22 +365,25 @@ distclean-compile: @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/libgrammar_la-rainerscript.Plo@am__quote@ .c.o: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(COMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ $< -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Po +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.o$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c $< +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ $< .c.obj: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(COMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ `$(CYGPATH_W) '$<'` -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Po +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.obj$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ `$(CYGPATH_W) '$<'` &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c `$(CYGPATH_W) '$<'` +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ `$(CYGPATH_W) '$<'` .c.lo: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(LTCOMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ $< -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Plo +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.lo$$||'`;\ +@am__fastdepCC_TRUE@ $(LTCOMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Plo @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=yes @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ @am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(LTCOMPILE) -c -o $@ $< diff --git a/grammar/grammar.c b/grammar/grammar.c index 4a70e3d..3aa9c43 100644 --- a/grammar/grammar.c +++ b/grammar/grammar.c @@ -66,8 +66,9 @@ /* Copy the first part of user declarations. */ /* Line 268 of yacc.c */ -#line 31 "grammar.y" +#line 29 "grammar.y" +#include "config.h" #include <stdio.h> #include <libestr.h> #include "rainerscript.h" @@ -81,7 +82,7 @@ extern int yyerror(char*); /* Line 268 of yacc.c */ -#line 85 "grammar.c" +#line 86 "grammar.c" /* Enabling traces. */ #ifndef YYDEBUG @@ -200,7 +201,7 @@ typedef union YYSTYPE { /* Line 293 of yacc.c */ -#line 44 "grammar.y" +#line 43 "grammar.y" char *s; long long n; @@ -218,7 +219,7 @@ typedef union YYSTYPE /* Line 293 of yacc.c */ -#line 222 "grammar.c" +#line 223 "grammar.c" } YYSTYPE; # define YYSTYPE_IS_TRIVIAL 1 # define yystype YYSTYPE /* obsolescent; will be withdrawn */ @@ -230,7 +231,7 @@ typedef union YYSTYPE /* Line 343 of yacc.c */ -#line 234 "grammar.c" +#line 235 "grammar.c" #ifdef short # undef short @@ -550,14 +551,14 @@ static const yytype_int8 yyrhs[] = /* YYRLINE[YYN] -- source line where rule number YYN was defined. */ static const yytype_uint8 yyrline[] = { - 0, 122, 122, 123, 124, 125, 126, 127, 128, 129, - 130, 134, 138, 139, 140, 141, 142, 143, 144, 145, - 146, 147, 148, 149, 150, 151, 155, 159, 160, 161, - 162, 163, 164, 165, 166, 168, 169, 170, 171, 172, - 173, 174, 175, 176, 177, 178, 179, 180, 181, 182, - 183, 184, 185, 186, 187, 188, 189, 190, 191, 192, - 193, 194, 195, 196, 197, 198, 199, 200, 201, 202, - 203, 204 + 0, 121, 121, 122, 123, 124, 125, 126, 127, 128, + 129, 133, 137, 138, 139, 140, 141, 142, 143, 144, + 145, 146, 147, 148, 149, 150, 154, 158, 159, 160, + 161, 162, 163, 164, 165, 167, 168, 169, 170, 171, + 172, 173, 174, 175, 176, 177, 178, 179, 180, 181, + 182, 183, 184, 185, 186, 187, 188, 189, 190, 191, + 192, 193, 194, 195, 196, 197, 198, 199, 200, 201, + 202, 203 }; #endif @@ -1611,56 +1612,56 @@ yyreduce: case 3: /* Line 1806 of yacc.c */ -#line 123 "grammar.y" +#line 122 "grammar.y" { cnfDoObj((yyvsp[(2) - (2)].obj)); } break; case 4: /* Line 1806 of yacc.c */ -#line 124 "grammar.y" +#line 123 "grammar.y" { cnfDoScript((yyvsp[(2) - (2)].stmt)); } break; case 5: /* Line 1806 of yacc.c */ -#line 125 "grammar.y" +#line 124 "grammar.y" { cnfDoCfsysline((yyvsp[(2) - (2)].s)); } break; case 6: /* Line 1806 of yacc.c */ -#line 126 "grammar.y" +#line 125 "grammar.y" { cnfDoBSDTag((yyvsp[(2) - (2)].s)); } break; case 7: /* Line 1806 of yacc.c */ -#line 127 "grammar.y" +#line 126 "grammar.y" { cnfDoBSDHost((yyvsp[(2) - (2)].s)); } break; case 8: /* Line 1806 of yacc.c */ -#line 128 "grammar.y" +#line 127 "grammar.y" { (yyval.obj) = cnfobjNew((yyvsp[(1) - (3)].objType), (yyvsp[(2) - (3)].nvlst)); } break; case 9: /* Line 1806 of yacc.c */ -#line 129 "grammar.y" +#line 128 "grammar.y" { (yyval.obj) = cnfobjNew(CNFOBJ_TPL, (yyvsp[(2) - (3)].nvlst)); } break; case 10: /* Line 1806 of yacc.c */ -#line 131 "grammar.y" +#line 130 "grammar.y" { (yyval.obj) = cnfobjNew(CNFOBJ_TPL, (yyvsp[(2) - (6)].nvlst)); (yyval.obj)->subobjs = (yyvsp[(5) - (6)].objlst); } @@ -1669,7 +1670,7 @@ yyreduce: case 11: /* Line 1806 of yacc.c */ -#line 135 "grammar.y" +#line 134 "grammar.y" { (yyval.obj) = cnfobjNew(CNFOBJ_RULESET, (yyvsp[(2) - (6)].nvlst)); (yyval.obj)->script = (yyvsp[(5) - (6)].stmt); } @@ -1678,98 +1679,98 @@ yyreduce: case 12: /* Line 1806 of yacc.c */ -#line 138 "grammar.y" +#line 137 "grammar.y" { (yyval.objlst) = NULL; } break; case 13: /* Line 1806 of yacc.c */ -#line 139 "grammar.y" +#line 138 "grammar.y" { (yyval.objlst) = objlstAdd((yyvsp[(1) - (2)].objlst), (yyvsp[(2) - (2)].obj)); } break; case 14: /* Line 1806 of yacc.c */ -#line 140 "grammar.y" +#line 139 "grammar.y" { (yyval.objlst) = objlstAdd((yyvsp[(1) - (2)].objlst), (yyvsp[(2) - (2)].obj)); } break; case 15: /* Line 1806 of yacc.c */ -#line 141 "grammar.y" +#line 140 "grammar.y" { (yyval.obj) = cnfobjNew(CNFOBJ_PROPERTY, (yyvsp[(2) - (3)].nvlst)); } break; case 16: /* Line 1806 of yacc.c */ -#line 142 "grammar.y" +#line 141 "grammar.y" { (yyval.obj) = cnfobjNew(CNFOBJ_CONSTANT, (yyvsp[(2) - (3)].nvlst)); } break; case 17: /* Line 1806 of yacc.c */ -#line 143 "grammar.y" +#line 142 "grammar.y" { (yyval.nvlst) = NULL; } break; case 18: /* Line 1806 of yacc.c */ -#line 144 "grammar.y" +#line 143 "grammar.y" { (yyvsp[(2) - (2)].nvlst)->next = (yyvsp[(1) - (2)].nvlst); (yyval.nvlst) = (yyvsp[(2) - (2)].nvlst); } break; case 19: /* Line 1806 of yacc.c */ -#line 145 "grammar.y" +#line 144 "grammar.y" { (yyval.nvlst) = nvlstSetName((yyvsp[(3) - (3)].nvlst), (yyvsp[(1) - (3)].estr)); } break; case 20: /* Line 1806 of yacc.c */ -#line 146 "grammar.y" +#line 145 "grammar.y" { (yyval.nvlst) = nvlstNewStr((yyvsp[(1) - (1)].estr)); } break; case 21: /* Line 1806 of yacc.c */ -#line 147 "grammar.y" +#line 146 "grammar.y" { (yyval.nvlst) = nvlstNewArray((yyvsp[(1) - (1)].arr)); } break; case 22: /* Line 1806 of yacc.c */ -#line 148 "grammar.y" +#line 147 "grammar.y" { (yyval.stmt) = (yyvsp[(1) - (1)].stmt); } break; case 23: /* Line 1806 of yacc.c */ -#line 149 "grammar.y" +#line 148 "grammar.y" { (yyval.stmt) = scriptAddStmt((yyvsp[(1) - (2)].stmt), (yyvsp[(2) - (2)].stmt)); } break; case 24: /* Line 1806 of yacc.c */ -#line 150 "grammar.y" +#line 149 "grammar.y" { (yyval.stmt) = (yyvsp[(1) - (1)].stmt); } break; case 25: /* Line 1806 of yacc.c */ -#line 151 "grammar.y" +#line 150 "grammar.y" { (yyval.stmt) = cnfstmtNew(S_IF); (yyval.stmt)->d.s_if.expr = (yyvsp[(2) - (4)].expr); (yyval.stmt)->d.s_if.t_then = (yyvsp[(4) - (4)].stmt); @@ -1779,7 +1780,7 @@ yyreduce: case 26: /* Line 1806 of yacc.c */ -#line 155 "grammar.y" +#line 154 "grammar.y" { (yyval.stmt) = cnfstmtNew(S_IF); (yyval.stmt)->d.s_if.expr = (yyvsp[(2) - (6)].expr); (yyval.stmt)->d.s_if.t_then = (yyvsp[(4) - (6)].stmt); @@ -1789,322 +1790,322 @@ yyreduce: case 27: /* Line 1806 of yacc.c */ -#line 159 "grammar.y" +#line 158 "grammar.y" { (yyval.stmt) = cnfstmtNewSet((yyvsp[(2) - (5)].s), (yyvsp[(4) - (5)].expr)); } break; case 28: /* Line 1806 of yacc.c */ -#line 160 "grammar.y" +#line 159 "grammar.y" { (yyval.stmt) = cnfstmtNewUnset((yyvsp[(2) - (3)].s)); } break; case 29: /* Line 1806 of yacc.c */ -#line 161 "grammar.y" +#line 160 "grammar.y" { (yyval.stmt) = cnfstmtNewPRIFILT((yyvsp[(1) - (2)].s), (yyvsp[(2) - (2)].stmt)); } break; case 30: /* Line 1806 of yacc.c */ -#line 162 "grammar.y" +#line 161 "grammar.y" { (yyval.stmt) = cnfstmtNewPROPFILT((yyvsp[(1) - (2)].s), (yyvsp[(2) - (2)].stmt)); } break; case 31: /* Line 1806 of yacc.c */ -#line 163 "grammar.y" +#line 162 "grammar.y" { (yyval.stmt) = (yyvsp[(1) - (1)].stmt); } break; case 32: /* Line 1806 of yacc.c */ -#line 164 "grammar.y" +#line 163 "grammar.y" { (yyval.stmt) = (yyvsp[(2) - (3)].stmt); } break; case 33: /* Line 1806 of yacc.c */ -#line 165 "grammar.y" +#line 164 "grammar.y" { (yyval.stmt) = (yyvsp[(1) - (1)].stmt); } break; case 34: /* Line 1806 of yacc.c */ -#line 166 "grammar.y" +#line 165 "grammar.y" { (yyval.stmt) = scriptAddStmt((yyvsp[(1) - (3)].stmt), (yyvsp[(3) - (3)].stmt)); } break; case 35: /* Line 1806 of yacc.c */ -#line 168 "grammar.y" +#line 167 "grammar.y" { (yyval.stmt) = cnfstmtNewAct((yyvsp[(2) - (3)].nvlst)); } break; case 36: /* Line 1806 of yacc.c */ -#line 169 "grammar.y" +#line 168 "grammar.y" { (yyval.stmt) = cnfstmtNewLegaAct((yyvsp[(1) - (1)].s)); } break; case 37: /* Line 1806 of yacc.c */ -#line 170 "grammar.y" +#line 169 "grammar.y" { (yyval.stmt) = cnfstmtNew(S_STOP); } break; case 38: /* Line 1806 of yacc.c */ -#line 171 "grammar.y" +#line 170 "grammar.y" { (yyval.stmt) = cnfstmtNewCall((yyvsp[(2) - (2)].estr)); } break; case 39: /* Line 1806 of yacc.c */ -#line 172 "grammar.y" +#line 171 "grammar.y" { (yyval.stmt) = cnfstmtNewContinue(); } break; case 40: /* Line 1806 of yacc.c */ -#line 173 "grammar.y" +#line 172 "grammar.y" { (yyval.expr) = cnfexprNew(AND, (yyvsp[(1) - (3)].expr), (yyvsp[(3) - (3)].expr)); } break; case 41: /* Line 1806 of yacc.c */ -#line 174 "grammar.y" +#line 173 "grammar.y" { (yyval.expr) = cnfexprNew(OR, (yyvsp[(1) - (3)].expr), (yyvsp[(3) - (3)].expr)); } break; case 42: /* Line 1806 of yacc.c */ -#line 175 "grammar.y" +#line 174 "grammar.y" { (yyval.expr) = cnfexprNew(NOT, NULL, (yyvsp[(2) - (2)].expr)); } break; case 43: /* Line 1806 of yacc.c */ -#line 176 "grammar.y" +#line 175 "grammar.y" { (yyval.expr) = cnfexprNew(CMP_EQ, (yyvsp[(1) - (3)].expr), (yyvsp[(3) - (3)].expr)); } break; case 44: /* Line 1806 of yacc.c */ -#line 177 "grammar.y" +#line 176 "grammar.y" { (yyval.expr) = cnfexprNew(CMP_NE, (yyvsp[(1) - (3)].expr), (yyvsp[(3) - (3)].expr)); } break; case 45: /* Line 1806 of yacc.c */ -#line 178 "grammar.y" +#line 177 "grammar.y" { (yyval.expr) = cnfexprNew(CMP_LE, (yyvsp[(1) - (3)].expr), (yyvsp[(3) - (3)].expr)); } break; case 46: /* Line 1806 of yacc.c */ -#line 179 "grammar.y" +#line 178 "grammar.y" { (yyval.expr) = cnfexprNew(CMP_GE, (yyvsp[(1) - (3)].expr), (yyvsp[(3) - (3)].expr)); } break; case 47: /* Line 1806 of yacc.c */ -#line 180 "grammar.y" +#line 179 "grammar.y" { (yyval.expr) = cnfexprNew(CMP_LT, (yyvsp[(1) - (3)].expr), (yyvsp[(3) - (3)].expr)); } break; case 48: /* Line 1806 of yacc.c */ -#line 181 "grammar.y" +#line 180 "grammar.y" { (yyval.expr) = cnfexprNew(CMP_GT, (yyvsp[(1) - (3)].expr), (yyvsp[(3) - (3)].expr)); } break; case 49: /* Line 1806 of yacc.c */ -#line 182 "grammar.y" +#line 181 "grammar.y" { (yyval.expr) = cnfexprNew(CMP_CONTAINS, (yyvsp[(1) - (3)].expr), (yyvsp[(3) - (3)].expr)); } break; case 50: /* Line 1806 of yacc.c */ -#line 183 "grammar.y" +#line 182 "grammar.y" { (yyval.expr) = cnfexprNew(CMP_CONTAINSI, (yyvsp[(1) - (3)].expr), (yyvsp[(3) - (3)].expr)); } break; case 51: /* Line 1806 of yacc.c */ -#line 184 "grammar.y" +#line 183 "grammar.y" { (yyval.expr) = cnfexprNew(CMP_STARTSWITH, (yyvsp[(1) - (3)].expr), (yyvsp[(3) - (3)].expr)); } break; case 52: /* Line 1806 of yacc.c */ -#line 185 "grammar.y" +#line 184 "grammar.y" { (yyval.expr) = cnfexprNew(CMP_STARTSWITHI, (yyvsp[(1) - (3)].expr), (yyvsp[(3) - (3)].expr)); } break; case 53: /* Line 1806 of yacc.c */ -#line 186 "grammar.y" +#line 185 "grammar.y" { (yyval.expr) = cnfexprNew('&', (yyvsp[(1) - (3)].expr), (yyvsp[(3) - (3)].expr)); } break; case 54: /* Line 1806 of yacc.c */ -#line 187 "grammar.y" +#line 186 "grammar.y" { (yyval.expr) = cnfexprNew('+', (yyvsp[(1) - (3)].expr), (yyvsp[(3) - (3)].expr)); } break; case 55: /* Line 1806 of yacc.c */ -#line 188 "grammar.y" +#line 187 "grammar.y" { (yyval.expr) = cnfexprNew('-', (yyvsp[(1) - (3)].expr), (yyvsp[(3) - (3)].expr)); } break; case 56: /* Line 1806 of yacc.c */ -#line 189 "grammar.y" +#line 188 "grammar.y" { (yyval.expr) = cnfexprNew('*', (yyvsp[(1) - (3)].expr), (yyvsp[(3) - (3)].expr)); } break; case 57: /* Line 1806 of yacc.c */ -#line 190 "grammar.y" +#line 189 "grammar.y" { (yyval.expr) = cnfexprNew('/', (yyvsp[(1) - (3)].expr), (yyvsp[(3) - (3)].expr)); } break; case 58: /* Line 1806 of yacc.c */ -#line 191 "grammar.y" +#line 190 "grammar.y" { (yyval.expr) = cnfexprNew('%', (yyvsp[(1) - (3)].expr), (yyvsp[(3) - (3)].expr)); } break; case 59: /* Line 1806 of yacc.c */ -#line 192 "grammar.y" +#line 191 "grammar.y" { (yyval.expr) = (yyvsp[(2) - (3)].expr); } break; case 60: /* Line 1806 of yacc.c */ -#line 193 "grammar.y" +#line 192 "grammar.y" { (yyval.expr) = cnfexprNew('M', NULL, (yyvsp[(2) - (2)].expr)); } break; case 61: /* Line 1806 of yacc.c */ -#line 194 "grammar.y" +#line 193 "grammar.y" { (yyval.expr) = (struct cnfexpr*) cnffuncNew((yyvsp[(1) - (3)].estr), NULL); } break; case 62: /* Line 1806 of yacc.c */ -#line 195 "grammar.y" +#line 194 "grammar.y" { (yyval.expr) = (struct cnfexpr*) cnffuncNew((yyvsp[(1) - (4)].estr), (yyvsp[(3) - (4)].fparams)); } break; case 63: /* Line 1806 of yacc.c */ -#line 196 "grammar.y" +#line 195 "grammar.y" { (yyval.expr) = (struct cnfexpr*) cnfnumvalNew((yyvsp[(1) - (1)].n)); } break; case 64: /* Line 1806 of yacc.c */ -#line 197 "grammar.y" +#line 196 "grammar.y" { (yyval.expr) = (struct cnfexpr*) cnfstringvalNew((yyvsp[(1) - (1)].estr)); } break; case 65: /* Line 1806 of yacc.c */ -#line 198 "grammar.y" +#line 197 "grammar.y" { (yyval.expr) = (struct cnfexpr*) cnfvarNew((yyvsp[(1) - (1)].s)); } break; case 66: /* Line 1806 of yacc.c */ -#line 199 "grammar.y" +#line 198 "grammar.y" { (yyval.expr) = (struct cnfexpr*) (yyvsp[(1) - (1)].arr); } break; case 67: /* Line 1806 of yacc.c */ -#line 200 "grammar.y" +#line 199 "grammar.y" { (yyval.fparams) = cnffparamlstNew((yyvsp[(1) - (1)].expr), NULL); } break; case 68: /* Line 1806 of yacc.c */ -#line 201 "grammar.y" +#line 200 "grammar.y" { (yyval.fparams) = cnffparamlstNew((yyvsp[(1) - (3)].expr), (yyvsp[(3) - (3)].fparams)); } break; case 69: /* Line 1806 of yacc.c */ -#line 202 "grammar.y" +#line 201 "grammar.y" { (yyval.arr) = (yyvsp[(2) - (3)].arr); } break; case 70: /* Line 1806 of yacc.c */ -#line 203 "grammar.y" +#line 202 "grammar.y" { (yyval.arr) = cnfarrayNew((yyvsp[(1) - (1)].estr)); } break; case 71: /* Line 1806 of yacc.c */ -#line 204 "grammar.y" +#line 203 "grammar.y" { (yyval.arr) = cnfarrayAdd((yyvsp[(1) - (3)].arr), (yyvsp[(3) - (3)].estr)); } break; /* Line 1806 of yacc.c */ -#line 2108 "grammar.c" +#line 2109 "grammar.c" default: break; } /* User semantic actions sometimes alter yychar, and that requires @@ -2335,7 +2336,7 @@ yyreturn: /* Line 2067 of yacc.c */ -#line 206 "grammar.y" +#line 205 "grammar.y" /* int yyerror(char *s) diff --git a/grammar/grammar.h b/grammar/grammar.h index cad1d92..c9bb3db 100644 --- a/grammar/grammar.h +++ b/grammar/grammar.h @@ -129,7 +129,7 @@ typedef union YYSTYPE { /* Line 2068 of yacc.c */ -#line 44 "grammar.y" +#line 43 "grammar.y" char *s; long long n; diff --git a/grammar/grammar.y b/grammar/grammar.y index c5bad68..488d332 100644 --- a/grammar/grammar.y +++ b/grammar/grammar.y @@ -1,34 +1,33 @@ - /* Bison file for rsyslog config format v2 (RainerScript). - * Please note: this file introduces the new config format, but maintains - * backward compatibility. In order to do so, the grammar is not 100% clean, - * but IMHO still sufficiently easy both to understand for programmers - * maitaining the code as well as users writing the config file. Users are, - * of course, encouraged to use new constructs only. But it needs to be noted - * that some of the legacy constructs (specifically the in-front-of-action - * PRI filter) are very hard to beat in ease of use, at least for simpler - * cases. - * - * Copyright 2011-2012 Rainer Gerhards and Adiscon GmbH. - * - * This file is part of the rsyslog runtime library. - * - * The rsyslog runtime library is free software: you can redistribute it and/or modify - * it under the terms of the GNU Lesser General Public License as published by - * the Free Software Foundation, either version 3 of the License, or - * (at your option) any later version. - * - * The rsyslog runtime library is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU Lesser General Public License for more details. - * - * You should have received a copy of the GNU Lesser General Public License - * along with the rsyslog runtime library. If not, see <http://www.gnu.org/licenses/>. - * - * A copy of the GPL can be found in the file "COPYING" in this distribution. - * A copy of the LGPL can be found in the file "COPYING.LESSER" in this distribution. - */ +/* Bison file for rsyslog config format v2 (RainerScript). + * Please note: this file introduces the new config format, but maintains + * backward compatibility. In order to do so, the grammar is not 100% clean, + * but IMHO still sufficiently easy both to understand for programmers + * maitaining the code as well as users writing the config file. Users are, + * of course, encouraged to use new constructs only. But it needs to be noted + * that some of the legacy constructs (specifically the in-front-of-action + * PRI filter) are very hard to beat in ease of use, at least for simpler + * cases. + * + * Copyright 2011-2014 Rainer Gerhards and Adiscon GmbH. + * + * This file is part of the rsyslog runtime library. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * -or- + * see COPYING.ASL20 in the source distribution + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ %{ +#include "config.h" #include <stdio.h> #include <libestr.h> #include "rainerscript.h" diff --git a/grammar/lexer.c b/grammar/lexer.c index b1af617..a498352 100644 --- a/grammar/lexer.c +++ b/grammar/lexer.c @@ -384,8 +384,8 @@ static void yy_fatal_error (yyconst char msg[] ); *yy_cp = '\0'; \ (yy_c_buf_p) = yy_cp; -#define YY_NUM_RULES 97 -#define YY_END_OF_BUFFER 98 +#define YY_NUM_RULES 101 +#define YY_END_OF_BUFFER 102 /* This struct is not used in this scanner, but its presence is necessary. */ struct yy_trans_info @@ -393,57 +393,64 @@ struct yy_trans_info flex_int32_t yy_verify; flex_int32_t yy_nxt; }; -static yyconst flex_int16_t yy_accept[443] = +static yyconst flex_int16_t yy_accept[505] = { 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, - 0, 0, 0, 0, 98, 96, 95, 95, 96, 96, - 96, 42, 68, 96, 96, 96, 96, 73, 96, 73, - 73, 73, 73, 73, 73, 73, 73, 73, 73, 73, - 43, 44, 67, 95, 68, 96, 96, 96, 73, 73, - 73, 73, 73, 73, 73, 73, 73, 73, 73, 73, - 88, 87, 87, 88, 88, 74, 76, 88, 79, 75, - 77, 78, 85, 85, 85, 55, 55, 54, 53, 52, - 40, 39, 39, 40, 38, 36, 36, 38, 38, 38, - 11, 13, 38, 17, 18, 9, 12, 8, 14, 10, - - 32, 32, 3, 24, 7, 25, 37, 37, 37, 37, - 37, 37, 37, 15, 16, 0, 0, 94, 0, 0, - 72, 72, 72, 70, 81, 73, 0, 73, 73, 73, - 73, 73, 1, 73, 73, 73, 73, 73, 73, 73, - 73, 0, 0, 0, 0, 0, 0, 0, 0, 72, - 0, 0, 0, 73, 73, 73, 73, 73, 73, 73, - 73, 1, 73, 73, 73, 73, 73, 73, 73, 73, - 0, 80, 0, 0, 86, 82, 75, 85, 85, 84, + 0, 0, 0, 0, 102, 100, 99, 99, 100, 100, + 100, 42, 72, 100, 100, 100, 100, 77, 100, 77, + 77, 77, 77, 77, 77, 77, 77, 77, 77, 77, + 77, 43, 44, 71, 99, 72, 100, 100, 100, 77, + 77, 77, 77, 77, 77, 77, 77, 77, 77, 77, + 77, 77, 92, 91, 91, 92, 92, 78, 80, 92, + 83, 79, 81, 82, 89, 89, 89, 55, 55, 54, + 53, 52, 40, 39, 39, 40, 38, 36, 36, 38, + 38, 38, 11, 13, 38, 17, 18, 9, 12, 8, + + 14, 10, 32, 32, 3, 24, 7, 25, 37, 37, + 37, 37, 37, 37, 37, 15, 16, 0, 0, 98, + 0, 0, 76, 76, 76, 74, 85, 77, 0, 77, + 77, 77, 77, 77, 1, 77, 77, 77, 77, 77, + 77, 77, 77, 77, 77, 77, 77, 0, 0, 0, + 0, 0, 0, 0, 0, 76, 0, 0, 0, 77, + 77, 77, 77, 77, 77, 77, 77, 1, 77, 77, + 77, 77, 77, 77, 77, 77, 77, 77, 77, 77, + 0, 84, 0, 0, 90, 86, 79, 89, 89, 88, 56, 52, 41, 22, 0, 35, 0, 0, 33, 0, - 34, 0, 83, 30, 0, 32, 20, 23, 19, 21, - - 37, 37, 37, 37, 4, 37, 37, 0, 90, 0, - 89, 0, 0, 92, 0, 72, 72, 69, 69, 70, - 71, 73, 73, 73, 73, 73, 73, 73, 73, 73, - 73, 48, 73, 73, 73, 66, 0, 93, 72, 0, - 0, 66, 73, 73, 73, 73, 73, 73, 73, 73, - 73, 73, 48, 73, 73, 73, 0, 0, 0, 33, - 0, 0, 31, 5, 37, 6, 37, 37, 0, 91, - 72, 69, 69, 71, 73, 47, 73, 73, 46, 73, - 73, 73, 73, 73, 73, 45, 73, 73, 0, 0, - 0, 73, 47, 73, 73, 46, 73, 73, 73, 73, - - 73, 73, 45, 73, 73, 0, 0, 0, 0, 0, - 37, 37, 2, 73, 73, 73, 73, 73, 73, 73, - 73, 73, 73, 49, 0, 0, 73, 73, 73, 73, - 73, 73, 73, 73, 73, 73, 49, 0, 37, 37, - 73, 73, 73, 73, 0, 62, 73, 73, 73, 73, - 73, 0, 0, 73, 73, 73, 73, 73, 73, 73, - 73, 73, 37, 37, 0, 64, 73, 73, 0, 57, - 0, 63, 73, 73, 73, 73, 0, 65, 0, 73, - 73, 73, 73, 73, 73, 37, 37, 73, 50, 73, - 73, 0, 59, 73, 65, 73, 50, 73, 73, 73, - - 26, 37, 0, 61, 73, 0, 60, 0, 58, 73, - 37, 37, 73, 73, 27, 28, 73, 73, 37, 73, - 73, 29, 73, 73, 73, 73, 73, 73, 73, 73, - 73, 73, 73, 73, 73, 73, 73, 73, 73, 73, - 51, 0 + + 34, 0, 87, 30, 0, 32, 20, 23, 19, 21, + 37, 37, 37, 37, 4, 37, 37, 0, 94, 0, + 93, 0, 0, 96, 0, 76, 76, 73, 73, 74, + 75, 77, 77, 77, 77, 77, 77, 77, 77, 77, + 77, 77, 77, 77, 48, 77, 77, 77, 77, 70, + 0, 97, 76, 0, 0, 70, 77, 77, 77, 77, + 77, 77, 77, 77, 77, 77, 77, 77, 77, 48, + 77, 77, 77, 77, 0, 0, 0, 33, 0, 0, + 31, 5, 37, 6, 37, 37, 0, 95, 76, 73, + 73, 75, 77, 47, 77, 77, 46, 77, 77, 77, + + 77, 77, 77, 77, 77, 77, 45, 77, 77, 77, + 0, 0, 0, 77, 47, 77, 77, 46, 77, 77, + 77, 77, 77, 77, 77, 77, 77, 45, 77, 77, + 77, 0, 0, 0, 0, 0, 37, 37, 2, 77, + 77, 77, 77, 77, 77, 77, 77, 77, 77, 77, + 77, 77, 77, 49, 0, 0, 77, 77, 77, 77, + 77, 77, 77, 77, 77, 77, 77, 77, 77, 49, + 0, 37, 37, 77, 77, 77, 77, 0, 65, 77, + 77, 77, 77, 77, 77, 77, 77, 77, 0, 0, + 77, 77, 77, 77, 77, 77, 77, 77, 77, 77, + + 77, 77, 37, 37, 0, 68, 77, 77, 0, 60, + 77, 77, 0, 66, 0, 59, 77, 77, 77, 77, + 77, 0, 69, 0, 77, 77, 77, 77, 77, 77, + 77, 37, 37, 77, 50, 77, 77, 77, 77, 0, + 62, 77, 77, 69, 77, 50, 77, 77, 77, 77, + 26, 37, 0, 64, 77, 77, 77, 0, 63, 0, + 61, 0, 58, 77, 37, 37, 77, 77, 77, 77, + 27, 28, 77, 0, 57, 77, 77, 37, 77, 77, + 77, 29, 0, 67, 77, 77, 77, 77, 77, 77, + 77, 77, 77, 77, 77, 77, 77, 77, 77, 77, + + 77, 77, 51, 0 } ; static yyconst flex_int32_t yy_ec[256] = @@ -455,13 +462,13 @@ static yyconst flex_int32_t yy_ec[256] = 13, 14, 15, 16, 17, 18, 19, 20, 21, 21, 21, 21, 21, 21, 21, 22, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, - 36, 37, 38, 39, 39, 40, 41, 42, 43, 44, - 39, 45, 46, 47, 48, 49, 50, 51, 52, 39, - 53, 54, 55, 29, 56, 1, 57, 58, 59, 60, + 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, + 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, + 56, 57, 58, 59, 60, 1, 61, 62, 63, 64, - 61, 62, 63, 64, 65, 39, 39, 66, 67, 68, - 69, 70, 39, 71, 72, 73, 74, 49, 75, 76, - 77, 39, 78, 29, 79, 80, 1, 1, 1, 1, + 65, 66, 67, 68, 69, 39, 70, 71, 72, 73, + 74, 75, 76, 77, 78, 79, 80, 51, 81, 82, + 83, 84, 85, 59, 86, 87, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, @@ -478,504 +485,565 @@ static yyconst flex_int32_t yy_ec[256] = 1, 1, 1, 1, 1 } ; -static yyconst flex_int32_t yy_meta[81] = +static yyconst flex_int32_t yy_meta[88] = { 0, 1, 2, 3, 4, 5, 6, 1, 7, 1, 1, - 6, 1, 1, 8, 9, 10, 11, 12, 1, 13, - 13, 14, 15, 16, 1, 17, 1, 18, 1, 19, - 20, 21, 21, 21, 19, 22, 22, 22, 22, 22, - 22, 23, 22, 22, 23, 22, 23, 22, 24, 22, - 25, 22, 1, 6, 1, 26, 19, 20, 21, 21, - 21, 19, 22, 22, 22, 22, 22, 23, 22, 22, - 23, 22, 23, 22, 22, 25, 22, 1, 1, 1 + 6, 1, 1, 8, 9, 10, 11, 12, 13, 14, + 14, 15, 16, 17, 1, 18, 1, 19, 20, 21, + 22, 23, 23, 23, 21, 24, 24, 24, 24, 24, + 24, 24, 25, 24, 24, 24, 25, 24, 25, 24, + 26, 24, 27, 24, 24, 1, 6, 1, 1, 28, + 21, 22, 23, 23, 23, 21, 24, 24, 24, 24, + 24, 24, 25, 24, 24, 24, 25, 24, 25, 24, + 24, 27, 24, 24, 1, 1, 1 } ; -static yyconst flex_int16_t yy_base[515] = +static yyconst flex_int16_t yy_base[580] = { 0, - 0, 80, 160, 0, 238, 239, 241, 244, 236, 241, - 263, 0, 343, 0, 1068, 3182, 3182, 3182, 0, 1059, - 0, 3182, 3182, 953, 236, 0, 942, 0, 0, 392, - 395, 387, 388, 394, 388, 387, 385, 400, 405, 395, - 3182, 3182, 0, 463, 424, 932, 486, 462, 502, 529, - 572, 538, 540, 568, 505, 569, 570, 612, 613, 592, - 3182, 3182, 3182, 420, 933, 3182, 3182, 904, 3182, 0, - 3182, 3182, 904, 899, 863, 0, 3182, 3182, 3182, 428, - 3182, 3182, 3182, 0, 3182, 3182, 3182, 852, 435, 438, - 3182, 3182, 449, 3182, 3182, 3182, 3182, 3182, 3182, 830, - - 455, 487, 3182, 233, 798, 783, 413, 0, 461, 485, - 484, 486, 488, 3182, 3182, 534, 778, 3182, 685, 569, - 0, 617, 518, 0, 3182, 0, 754, 509, 535, 537, - 545, 588, 0, 588, 524, 611, 610, 604, 609, 612, - 720, 765, 646, 0, 767, 752, 0, 758, 781, 792, - 676, 667, 806, 835, 814, 859, 872, 796, 791, 740, - 794, 742, 795, 851, 861, 859, 778, 854, 862, 866, - 554, 3182, 448, 665, 3182, 3182, 0, 630, 613, 3182, - 0, 834, 0, 3182, 860, 3182, 896, 0, 0, 760, - 3182, 900, 3182, 451, 0, 920, 3182, 3182, 3182, 3182, - - 0, 798, 858, 858, 0, 880, 899, 946, 3182, 574, - 3182, 976, 950, 959, 560, 966, 524, 0, 0, 0, - 0, 906, 1017, 1012, 1026, 1030, 1015, 1016, 1021, 1022, - 1033, 0, 1025, 1026, 1037, 0, 1070, 3182, 1073, 939, - 786, 1088, 1101, 1112, 1101, 1046, 1080, 1095, 1097, 1038, - 1109, 1116, 1114, 1115, 1118, 1130, 1166, 938, 0, 0, - 944, 0, 3182, 0, 1129, 0, 1138, 1147, 482, 3182, - 474, 0, 0, 0, 1147, 0, 1145, 1059, 0, 1083, - 1146, 1155, 1152, 1164, 1154, 0, 1161, 1156, 970, 1095, - 1201, 1164, 1168, 1184, 1172, 1173, 1192, 1188, 1174, 1196, - - 1208, 1205, 1204, 1206, 1212, 1259, 1235, 0, 1243, 0, - 1230, 1200, 0, 1224, 1238, 1233, 1236, 1279, 1250, 1243, - 1243, 1255, 1260, 0, 1195, 563, 1254, 1267, 1263, 1253, - 1296, 1281, 1266, 1278, 1294, 1297, 1264, 1304, 1287, 1292, - 1363, 1301, 1297, 1370, 1374, 3182, 1377, 1294, 1310, 1312, - 1314, 659, 1334, 1388, 1328, 1353, 1395, 1400, 1373, 1368, - 1370, 1371, 1353, 573, 1421, 3182, 1369, 1386, 1446, 3182, - 1453, 3182, 1391, 808, 1460, 1412, 1416, 3182, 1423, 1405, - 1434, 1440, 1427, 1479, 1450, 1413, 1422, 1495, 0, 1431, - 1506, 1510, 3182, 1521, 1432, 1524, 1449, 1465, 1535, 1538, - - 379, 1447, 1549, 3182, 1477, 1553, 3182, 1556, 3182, 1504, - 1483, 1498, 1512, 1532, 0, 374, 1514, 1523, 1511, 1529, - 1531, 0, 1532, 1537, 1547, 1548, 1542, 1545, 1538, 1551, - 1551, 1566, 1564, 1584, 1568, 1586, 1574, 1583, 239, 1604, - 3182, 3182, 1670, 1696, 1722, 1748, 1774, 1782, 1807, 1833, - 1851, 1865, 1889, 1907, 1925, 1950, 1970, 1988, 2014, 2040, - 2055, 2081, 2107, 2122, 2148, 2170, 2195, 2209, 2235, 2261, - 2275, 2299, 2317, 2332, 2351, 2372, 2397, 2420, 2441, 2460, - 2486, 2507, 2531, 2546, 2572, 2598, 2613, 2639, 2660, 2667, - 2688, 2714, 2735, 2748, 2757, 2783, 2809, 2835, 2861, 2887, - - 2913, 2939, 2965, 2983, 3005, 3027, 3048, 3067, 3093, 3107, - 3116, 3125, 3134, 3155 + 0, 87, 174, 0, 259, 260, 262, 265, 257, 262, + 284, 0, 371, 0, 1310, 3608, 3608, 3608, 0, 1257, + 0, 3608, 3608, 1232, 257, 0, 1192, 0, 0, 427, + 430, 421, 422, 429, 422, 437, 438, 420, 445, 444, + 428, 3608, 3608, 0, 503, 468, 1191, 526, 539, 555, + 574, 621, 579, 586, 619, 519, 570, 622, 626, 654, + 674, 628, 3608, 3608, 3608, 455, 1198, 3608, 3608, 1062, + 3608, 0, 3608, 3608, 1059, 1057, 974, 0, 3608, 3608, + 3608, 455, 3608, 3608, 3608, 0, 3608, 3608, 3608, 894, + 459, 614, 3608, 3608, 462, 3608, 3608, 3608, 3608, 3608, + + 3608, 901, 544, 501, 3608, 254, 849, 822, 444, 0, + 470, 504, 527, 550, 580, 3608, 3608, 564, 834, 3608, + 748, 599, 0, 671, 466, 0, 3608, 0, 703, 576, + 620, 621, 619, 633, 0, 634, 642, 590, 657, 663, + 802, 670, 665, 673, 796, 797, 792, 839, 704, 0, + 715, 711, 0, 829, 734, 876, 705, 726, 890, 904, + 837, 923, 935, 875, 850, 849, 922, 793, 923, 925, + 937, 909, 930, 948, 929, 795, 936, 945, 963, 941, + 500, 3608, 587, 702, 3608, 3608, 0, 665, 643, 3608, + 0, 839, 0, 3608, 857, 3608, 1005, 0, 0, 472, + + 3608, 1008, 3608, 725, 0, 844, 3608, 3608, 3608, 3608, + 0, 884, 938, 942, 0, 973, 984, 1028, 3608, 621, + 3608, 1090, 1034, 1041, 618, 1048, 602, 0, 0, 0, + 0, 985, 992, 991, 1012, 1016, 1003, 1016, 1016, 1128, + 1131, 1135, 1136, 1023, 0, 1137, 1138, 1150, 1151, 0, + 1062, 3608, 1069, 542, 847, 1181, 1204, 1215, 1199, 1209, + 1214, 1173, 1211, 1216, 1208, 1156, 1220, 1221, 1249, 1146, + 1222, 1227, 1260, 1265, 1297, 1047, 0, 0, 1054, 0, + 3608, 0, 1205, 0, 1257, 1262, 559, 3608, 540, 0, + 0, 0, 1263, 0, 1259, 1200, 0, 1279, 1262, 1262, + + 465, 1272, 1281, 1272, 1282, 1273, 0, 1281, 526, 1274, + 1075, 1077, 1184, 1285, 1286, 1309, 1307, 1291, 1303, 1314, + 1319, 1324, 1320, 1332, 1330, 1341, 1331, 1327, 1329, 1353, + 1340, 1407, 1064, 0, 1170, 0, 1353, 1337, 0, 1344, + 1362, 1353, 1363, 1190, 1360, 1364, 1381, 1371, 1376, 1374, + 1388, 1395, 1385, 0, 892, 867, 1389, 1406, 1390, 1397, + 1440, 1400, 1424, 1408, 1413, 1425, 1435, 1452, 1447, 1432, + 919, 1401, 1426, 1495, 1435, 1430, 1512, 1523, 3608, 448, + 1440, 1526, 1529, 1454, 1443, 1460, 1463, 1476, 961, 1477, + 1541, 1505, 1506, 1548, 1495, 1555, 1559, 1532, 1531, 1534, + + 1538, 1547, 1497, 879, 1589, 3608, 1505, 1503, 1595, 3608, + 1525, 1551, 1619, 3608, 1625, 3608, 1559, 1217, 1630, 1574, + 1580, 1600, 3608, 1605, 1594, 1606, 1603, 1595, 1648, 1614, + 1622, 1576, 1577, 1678, 0, 1611, 1603, 1623, 1692, 1695, + 3608, 1698, 1709, 1652, 1712, 1634, 1665, 1723, 1726, 1729, + 443, 1639, 1740, 3608, 1655, 1657, 1679, 1743, 3608, 1754, + 3608, 1757, 3608, 1707, 1670, 1686, 1708, 1769, 1703, 1741, + 0, 437, 1728, 1783, 3608, 1733, 1736, 1727, 1786, 1742, + 1761, 0, 1806, 3608, 1741, 1760, 1757, 1779, 1762, 1774, + 1766, 1778, 1779, 1782, 1794, 1795, 1795, 1800, 1790, 1801, + + 260, 1827, 3608, 3608, 1898, 1926, 1954, 1982, 2010, 2018, + 2045, 2073, 2093, 2108, 2134, 2154, 2174, 2201, 2223, 2243, + 2271, 2299, 2316, 2344, 2372, 2389, 2417, 2441, 2468, 2483, + 2511, 2539, 2567, 2595, 2615, 2630, 2656, 2676, 2693, 2714, + 2737, 2764, 2789, 2812, 2833, 2861, 2884, 2910, 2927, 2955, + 2983, 3000, 3028, 3051, 3059, 3082, 3110, 3133, 1841, 3147, + 3175, 3203, 3231, 3259, 3287, 3315, 3343, 3371, 3391, 3415, + 3439, 3462, 3483, 3511, 3526, 3536, 3546, 3556, 3579 } ; -static yyconst flex_int16_t yy_def[515] = +static yyconst flex_int16_t yy_def[580] = { 0, - 442, 442, 442, 3, 443, 443, 444, 444, 445, 445, - 442, 11, 442, 13, 442, 442, 442, 442, 446, 447, - 448, 442, 442, 442, 449, 449, 450, 451, 452, 451, - 451, 451, 451, 451, 451, 451, 451, 451, 451, 451, - 442, 442, 449, 453, 454, 455, 456, 457, 458, 458, - 458, 51, 51, 51, 51, 51, 51, 51, 51, 51, - 442, 442, 442, 459, 460, 442, 442, 442, 442, 461, - 442, 442, 462, 462, 442, 463, 442, 442, 442, 442, - 442, 442, 442, 464, 442, 442, 442, 442, 465, 466, - 442, 442, 467, 442, 442, 442, 442, 442, 442, 442, - - 442, 442, 442, 442, 442, 442, 468, 468, 468, 468, - 468, 468, 468, 442, 442, 446, 447, 442, 442, 442, - 449, 449, 469, 470, 442, 451, 471, 451, 451, 451, - 451, 451, 451, 451, 451, 451, 451, 451, 451, 451, - 451, 472, 473, 474, 475, 473, 476, 442, 474, 477, - 475, 475, 478, 479, 451, 480, 480, 157, 157, 157, - 157, 157, 157, 157, 157, 157, 157, 157, 157, 157, - 481, 442, 482, 483, 442, 442, 484, 485, 485, 442, - 486, 442, 487, 442, 488, 442, 489, 490, 491, 492, - 442, 493, 442, 442, 494, 442, 442, 442, 442, 442, - - 495, 495, 495, 495, 495, 495, 495, 442, 442, 496, - 442, 442, 442, 442, 497, 498, 499, 500, 501, 502, - 503, 504, 504, 504, 504, 504, 504, 504, 504, 504, - 504, 504, 504, 504, 504, 505, 442, 442, 498, 442, - 506, 507, 508, 508, 244, 244, 244, 244, 244, 244, - 244, 244, 244, 244, 244, 244, 509, 442, 510, 491, - 442, 511, 442, 495, 495, 495, 495, 495, 497, 442, - 499, 500, 501, 503, 504, 504, 504, 504, 504, 504, - 504, 504, 504, 504, 504, 504, 504, 504, 506, 506, - 506, 244, 244, 244, 244, 244, 244, 244, 244, 244, - - 244, 244, 244, 244, 244, 509, 442, 512, 442, 513, - 495, 495, 495, 504, 504, 504, 504, 504, 504, 504, - 504, 504, 504, 504, 442, 442, 244, 244, 244, 244, - 244, 244, 244, 244, 244, 244, 244, 509, 495, 495, - 504, 504, 504, 504, 442, 442, 504, 504, 504, 504, - 504, 442, 514, 244, 244, 244, 244, 244, 244, 244, - 244, 244, 495, 495, 442, 442, 504, 504, 442, 442, - 442, 442, 504, 504, 504, 504, 514, 442, 514, 244, - 244, 244, 244, 244, 244, 495, 495, 504, 504, 504, - 504, 442, 442, 504, 514, 244, 244, 244, 244, 244, - - 495, 495, 442, 442, 504, 442, 442, 442, 442, 244, - 495, 495, 504, 244, 495, 495, 504, 244, 495, 504, - 244, 495, 504, 244, 504, 244, 504, 244, 504, 244, - 504, 244, 504, 244, 504, 244, 504, 244, 504, 244, - 442, 0, 442, 442, 442, 442, 442, 442, 442, 442, - 442, 442, 442, 442, 442, 442, 442, 442, 442, 442, - 442, 442, 442, 442, 442, 442, 442, 442, 442, 442, - 442, 442, 442, 442, 442, 442, 442, 442, 442, 442, - 442, 442, 442, 442, 442, 442, 442, 442, 442, 442, - 442, 442, 442, 442, 442, 442, 442, 442, 442, 442, - - 442, 442, 442, 442, 442, 442, 442, 442, 442, 442, - 442, 442, 442, 442 + 504, 504, 504, 3, 505, 505, 506, 506, 507, 507, + 504, 11, 504, 13, 504, 504, 504, 504, 508, 509, + 510, 504, 504, 504, 511, 511, 512, 513, 514, 513, + 513, 513, 513, 513, 513, 513, 513, 513, 513, 513, + 513, 504, 504, 511, 515, 516, 517, 518, 519, 520, + 520, 520, 52, 52, 52, 52, 52, 52, 52, 52, + 52, 52, 504, 504, 504, 521, 522, 504, 504, 504, + 504, 523, 504, 504, 524, 524, 504, 525, 504, 504, + 504, 504, 504, 504, 504, 526, 504, 504, 504, 504, + 527, 528, 504, 504, 529, 504, 504, 504, 504, 504, + + 504, 504, 504, 504, 504, 504, 504, 504, 530, 530, + 530, 530, 530, 530, 530, 504, 504, 531, 532, 504, + 504, 504, 511, 511, 533, 534, 504, 535, 536, 535, + 535, 535, 535, 535, 535, 535, 535, 535, 535, 535, + 535, 535, 535, 535, 535, 535, 535, 537, 538, 539, + 540, 538, 541, 504, 539, 542, 540, 540, 543, 544, + 535, 545, 545, 163, 163, 163, 163, 163, 163, 163, + 163, 163, 163, 163, 163, 163, 163, 163, 163, 163, + 546, 504, 547, 548, 504, 504, 549, 550, 550, 504, + 551, 504, 552, 504, 553, 504, 554, 555, 556, 557, + + 504, 558, 504, 504, 559, 504, 504, 504, 504, 504, + 560, 560, 560, 560, 560, 560, 560, 504, 504, 561, + 504, 504, 504, 504, 562, 563, 564, 565, 566, 567, + 568, 569, 569, 569, 569, 569, 569, 569, 569, 569, + 569, 569, 569, 569, 569, 569, 569, 569, 569, 570, + 504, 504, 563, 504, 571, 572, 573, 573, 258, 258, + 258, 258, 258, 258, 258, 258, 258, 258, 258, 258, + 258, 258, 258, 258, 574, 504, 575, 556, 504, 576, + 504, 560, 560, 560, 560, 560, 562, 504, 564, 565, + 566, 568, 569, 569, 569, 569, 569, 569, 569, 569, + + 569, 569, 569, 569, 569, 569, 569, 569, 569, 569, + 571, 571, 571, 258, 258, 258, 258, 258, 258, 258, + 258, 258, 258, 258, 258, 258, 258, 258, 258, 258, + 258, 574, 504, 577, 504, 578, 560, 560, 560, 569, + 569, 569, 569, 569, 569, 569, 569, 569, 569, 569, + 569, 569, 569, 569, 504, 504, 258, 258, 258, 258, + 258, 258, 258, 258, 258, 258, 258, 258, 258, 258, + 574, 560, 560, 569, 569, 569, 569, 504, 504, 569, + 569, 569, 569, 569, 569, 569, 569, 569, 504, 579, + 258, 258, 258, 258, 258, 258, 258, 258, 258, 258, + + 258, 258, 560, 560, 504, 504, 569, 569, 504, 504, + 569, 569, 504, 504, 504, 504, 569, 569, 569, 569, + 569, 579, 504, 579, 258, 258, 258, 258, 258, 258, + 258, 560, 560, 569, 569, 569, 569, 569, 569, 504, + 504, 569, 569, 579, 258, 258, 258, 258, 258, 258, + 560, 560, 504, 504, 569, 569, 569, 504, 504, 504, + 504, 504, 504, 258, 560, 560, 569, 569, 569, 258, + 560, 560, 569, 504, 504, 569, 258, 560, 569, 569, + 258, 560, 504, 504, 569, 258, 569, 258, 569, 258, + 569, 258, 569, 258, 569, 258, 569, 258, 569, 258, + + 569, 258, 504, 0, 504, 504, 504, 504, 504, 504, + 504, 504, 504, 504, 504, 504, 504, 504, 504, 504, + 504, 504, 504, 504, 504, 504, 504, 504, 504, 504, + 504, 504, 504, 504, 504, 504, 504, 504, 504, 504, + 504, 504, 504, 504, 504, 504, 504, 504, 504, 504, + 504, 504, 504, 504, 504, 504, 504, 504, 504, 504, + 504, 504, 504, 504, 504, 504, 504, 504, 504, 504, + 504, 504, 504, 504, 504, 504, 504, 504, 504 } ; -static yyconst flex_int16_t yy_nxt[3263] = +static yyconst flex_int16_t yy_nxt[3696] = { 0, 16, 17, 18, 17, 19, 16, 20, 21, 16, 22, 16, 16, 16, 23, 24, 16, 25, 26, 27, 28, 28, 28, 29, 16, 16, 16, 26, 26, 26, 30, 28, 31, 28, 32, 28, 33, 28, 34, 28, 28, - 35, 28, 28, 36, 37, 38, 39, 40, 28, 28, - 28, 28, 16, 16, 16, 28, 30, 28, 31, 28, - 32, 28, 33, 28, 34, 28, 35, 28, 28, 36, - 37, 38, 39, 40, 28, 28, 28, 41, 42, 43, - 16, 44, 18, 44, 19, 16, 20, 21, 16, 22, - 16, 16, 16, 45, 46, 16, 47, 26, 27, 28, - - 28, 28, 48, 16, 16, 16, 26, 26, 26, 49, - 50, 51, 50, 52, 50, 53, 50, 54, 50, 50, - 55, 50, 50, 56, 57, 58, 59, 60, 50, 50, - 50, 50, 16, 16, 16, 28, 49, 50, 51, 50, - 52, 50, 53, 50, 54, 50, 55, 50, 50, 56, - 57, 58, 59, 60, 50, 50, 50, 41, 42, 43, - 61, 62, 63, 62, 61, 64, 65, 61, 61, 61, - 61, 61, 66, 61, 61, 67, 61, 61, 68, 61, - 61, 61, 61, 61, 61, 69, 61, 61, 61, 70, - 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, - - 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, - 70, 70, 71, 61, 72, 61, 70, 70, 70, 70, - 70, 70, 70, 70, 70, 70, 70, 70, 70, 70, - 70, 70, 70, 70, 70, 70, 70, 61, 61, 61, - 74, 74, 77, 77, 77, 77, 77, 77, 79, 122, - 441, 75, 75, 79, 123, 80, 80, 80, 197, 198, - 80, 80, 80, 81, 82, 83, 82, 81, 81, 81, - 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, - 81, 81, 81, 81, 81, 81, 81, 81, 81, 81, - 81, 81, 84, 84, 84, 84, 84, 84, 84, 84, - - 84, 84, 84, 84, 84, 84, 84, 84, 84, 84, - 84, 84, 84, 84, 84, 81, 81, 81, 81, 84, - 84, 84, 84, 84, 84, 84, 84, 84, 84, 84, - 84, 84, 84, 84, 84, 84, 84, 84, 84, 84, - 81, 81, 81, 85, 86, 87, 86, 88, 89, 85, - 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, - 85, 100, 101, 102, 102, 85, 103, 104, 105, 106, - 85, 85, 107, 108, 109, 108, 108, 108, 108, 108, - 108, 108, 108, 108, 110, 111, 108, 108, 112, 113, - 108, 108, 108, 108, 108, 114, 85, 115, 85, 107, - - 108, 109, 108, 108, 108, 108, 108, 108, 108, 108, - 110, 111, 108, 108, 112, 113, 108, 108, 108, 108, - 85, 85, 85, 128, 129, 172, 131, 132, 133, 419, - 135, 136, 137, 138, 411, 134, 141, 130, 140, 147, - 186, 147, 188, 148, 148, 188, 139, 182, 182, 182, - 128, 129, 131, 132, 202, 133, 135, 136, 137, 191, - 138, 134, 141, 130, 142, 140, 142, 257, 257, 151, - 194, 194, 139, 173, 194, 194, 270, 144, 152, 144, - 202, 127, 127, 127, 270, 145, 121, 121, 187, 121, - 121, 121, 121, 121, 121, 121, 121, 121, 121, 122, - - 121, 121, 192, 203, 123, 195, 196, 196, 196, 121, - 121, 121, 121, 121, 121, 146, 126, 154, 126, 147, - 219, 155, 155, 126, 207, 126, 270, 204, 205, 203, - 195, 121, 206, 157, 156, 208, 209, 208, 121, 121, - 121, 121, 146, 126, 154, 126, 147, 164, 155, 155, - 126, 207, 126, 204, 205, 222, 228, 126, 206, 172, - 157, 156, 270, 121, 121, 121, 352, 156, 353, 156, - 213, 214, 213, 164, 223, 215, 211, 160, 224, 161, - 156, 222, 156, 228, 126, 146, 126, 154, 126, 147, - 225, 155, 155, 126, 156, 126, 156, 156, 156, 156, - - 223, 158, 162, 160, 224, 161, 156, 173, 156, 163, - 156, 156, 156, 165, 159, 179, 225, 166, 216, 214, - 216, 156, 387, 217, 156, 156, 156, 126, 158, 162, - 226, 227, 179, 170, 156, 163, 156, 156, 156, 165, - 159, 156, 156, 166, 229, 167, 169, 387, 156, 231, - 232, 233, 234, 230, 156, 156, 226, 227, 168, 170, - 156, 147, 352, 147, 353, 148, 148, 175, 156, 156, - 240, 229, 167, 169, 442, 231, 232, 233, 234, 230, - 156, 156, 241, 442, 168, 210, 210, 211, 210, 210, - 210, 210, 210, 210, 210, 210, 210, 210, 210, 210, - - 210, 210, 210, 210, 210, 210, 210, 210, 210, 210, - 210, 210, 210, 210, 212, 212, 212, 212, 212, 212, - 212, 212, 212, 212, 212, 212, 212, 212, 212, 212, - 212, 212, 212, 212, 212, 212, 212, 210, 210, 210, - 210, 212, 212, 212, 212, 212, 212, 212, 212, 212, - 212, 212, 212, 212, 212, 212, 212, 212, 212, 212, - 212, 212, 210, 210, 210, 235, 142, 147, 142, 147, - 191, 148, 148, 147, 151, 147, 221, 148, 148, 144, - 118, 144, 237, 238, 237, 246, 156, 145, 156, 289, - 290, 235, 121, 239, 238, 239, 121, 121, 121, 121, - - 121, 121, 121, 121, 121, 121, 121, 121, 200, 240, - 121, 246, 156, 192, 156, 121, 121, 121, 121, 121, - 121, 241, 152, 199, 253, 127, 127, 127, 221, 154, - 264, 147, 245, 155, 155, 244, 247, 156, 248, 236, - 156, 156, 156, 193, 121, 121, 121, 121, 236, 126, - 253, 126, 236, 182, 182, 182, 126, 264, 245, 391, - 236, 244, 247, 156, 248, 186, 156, 156, 156, 121, - 121, 121, 146, 126, 154, 126, 147, 184, 155, 155, - 126, 180, 126, 249, 391, 146, 126, 154, 126, 147, - 126, 155, 155, 126, 250, 126, 254, 156, 252, 265, - - 156, 179, 255, 251, 266, 156, 179, 156, 156, 267, - 249, 256, 156, 187, 126, 258, 258, 176, 243, 261, - 261, 250, 254, 156, 252, 265, 156, 126, 255, 251, - 266, 156, 268, 156, 156, 175, 267, 256, 156, 196, - 196, 196, 240, 275, 243, 120, 259, 208, 209, 208, - 262, 213, 214, 213, 241, 125, 215, 307, 307, 268, - 213, 214, 213, 309, 309, 215, 120, 216, 214, 216, - 275, 259, 217, 289, 290, 262, 210, 210, 211, 210, - 210, 210, 210, 210, 210, 210, 210, 210, 210, 210, - 210, 210, 210, 210, 210, 210, 210, 210, 210, 210, - - 210, 210, 210, 210, 210, 212, 212, 212, 212, 212, - 212, 212, 212, 212, 212, 212, 212, 212, 212, 212, - 212, 212, 212, 212, 212, 212, 212, 212, 210, 210, - 210, 210, 212, 212, 212, 212, 212, 212, 212, 212, - 212, 212, 212, 212, 212, 212, 212, 212, 212, 212, - 212, 212, 212, 210, 210, 210, 276, 277, 278, 279, - 280, 118, 281, 282, 283, 284, 285, 442, 286, 287, - 288, 237, 238, 237, 239, 238, 239, 156, 442, 296, - 442, 300, 276, 277, 278, 156, 279, 280, 281, 282, - 283, 284, 236, 285, 286, 287, 316, 288, 442, 442, - - 442, 236, 126, 156, 126, 236, 296, 300, 442, 126, - 297, 156, 317, 236, 146, 126, 154, 126, 147, 156, - 155, 155, 126, 316, 126, 146, 126, 154, 126, 147, - 442, 155, 155, 126, 156, 126, 156, 297, 292, 317, - 156, 442, 298, 126, 299, 156, 294, 295, 156, 302, - 442, 293, 301, 156, 156, 156, 126, 156, 303, 442, - 156, 304, 156, 305, 442, 292, 156, 126, 298, 156, - 299, 172, 294, 295, 156, 311, 302, 293, 301, 156, - 156, 156, 312, 156, 303, 306, 306, 304, 313, 314, - 305, 315, 318, 442, 319, 156, 320, 321, 325, 322, - - 323, 311, 324, 156, 325, 442, 327, 156, 312, 329, - 326, 156, 156, 332, 313, 314, 326, 315, 318, 173, - 319, 330, 320, 156, 321, 322, 323, 156, 324, 156, - 328, 156, 327, 156, 331, 156, 329, 156, 156, 332, - 333, 334, 442, 156, 156, 336, 340, 156, 330, 156, - 335, 156, 442, 156, 185, 185, 328, 156, 337, 339, - 331, 156, 190, 190, 172, 341, 333, 342, 334, 156, - 156, 336, 340, 156, 343, 344, 335, 156, 338, 338, - 345, 345, 345, 347, 337, 348, 339, 349, 350, 351, - 346, 341, 357, 156, 342, 354, 355, 345, 345, 345, - - 343, 344, 156, 156, 356, 156, 156, 346, 359, 172, - 347, 348, 173, 349, 358, 350, 351, 156, 357, 156, - 156, 354, 360, 355, 363, 373, 362, 361, 156, 156, - 356, 156, 156, 156, 359, 156, 156, 364, 442, 378, - 442, 358, 367, 156, 368, 442, 156, 442, 360, 442, - 442, 363, 373, 362, 361, 442, 374, 173, 375, 156, - 376, 156, 156, 364, 365, 365, 365, 156, 367, 380, - 368, 369, 369, 369, 366, 345, 345, 345, 371, 371, - 371, 370, 374, 442, 375, 346, 376, 379, 372, 365, - 365, 365, 156, 156, 386, 380, 369, 369, 369, 366, - - 381, 371, 371, 371, 382, 442, 370, 156, 442, 156, - 156, 372, 156, 442, 383, 388, 384, 385, 156, 389, - 386, 378, 365, 365, 365, 390, 381, 156, 395, 442, - 442, 382, 366, 156, 156, 156, 156, 378, 156, 156, - 383, 388, 384, 385, 156, 394, 389, 369, 369, 369, - 442, 396, 390, 156, 371, 371, 371, 370, 401, 402, - 156, 392, 392, 392, 372, 156, 156, 397, 405, 379, - 156, 393, 394, 156, 398, 442, 379, 396, 399, 156, - 392, 392, 392, 400, 401, 379, 402, 442, 156, 156, - 393, 442, 156, 412, 397, 405, 403, 403, 403, 156, - - 442, 398, 410, 399, 156, 156, 404, 406, 406, 406, - 400, 392, 392, 392, 156, 156, 413, 407, 156, 412, - 415, 393, 408, 408, 408, 403, 403, 403, 442, 410, - 156, 442, 409, 442, 416, 404, 406, 406, 406, 408, - 408, 408, 413, 414, 156, 417, 407, 415, 422, 409, - 403, 403, 403, 420, 406, 406, 406, 408, 408, 408, - 404, 416, 421, 156, 407, 418, 423, 409, 424, 414, - 156, 156, 417, 425, 156, 422, 156, 156, 426, 420, - 427, 428, 442, 429, 156, 431, 430, 156, 421, 156, - 156, 433, 418, 423, 435, 424, 156, 156, 432, 425, - - 156, 437, 156, 156, 426, 156, 434, 427, 428, 429, - 156, 431, 430, 156, 436, 441, 156, 433, 439, 438, - 442, 435, 156, 156, 432, 156, 442, 440, 437, 442, - 442, 156, 434, 442, 442, 442, 442, 442, 442, 442, - 442, 436, 442, 156, 439, 442, 438, 442, 156, 156, - 442, 156, 442, 440, 442, 442, 442, 442, 442, 442, - 442, 442, 442, 442, 442, 442, 442, 442, 442, 156, - 73, 73, 73, 73, 73, 73, 73, 73, 73, 73, - 73, 73, 73, 73, 73, 73, 73, 73, 73, 73, - 73, 73, 73, 73, 73, 73, 76, 76, 76, 76, - - 76, 76, 76, 76, 76, 76, 76, 76, 76, 76, - 76, 76, 76, 76, 76, 76, 76, 76, 76, 76, - 76, 76, 78, 78, 78, 78, 78, 78, 78, 78, + 35, 36, 28, 28, 37, 28, 38, 39, 40, 41, + 28, 28, 28, 28, 28, 16, 16, 16, 26, 28, + 30, 28, 31, 28, 32, 28, 33, 28, 34, 28, + 35, 36, 28, 28, 37, 28, 38, 39, 40, 41, + 28, 28, 28, 28, 42, 43, 44, 16, 45, 18, + 45, 19, 16, 20, 21, 16, 22, 16, 16, 16, + + 46, 47, 16, 48, 26, 27, 28, 28, 28, 49, + 16, 16, 16, 26, 26, 26, 50, 51, 52, 51, + 53, 51, 54, 51, 55, 51, 51, 56, 57, 51, + 51, 58, 51, 59, 60, 61, 62, 51, 51, 51, + 51, 51, 16, 16, 16, 26, 28, 50, 51, 52, + 51, 53, 51, 54, 51, 55, 51, 56, 57, 51, + 51, 58, 51, 59, 60, 61, 62, 51, 51, 51, + 51, 42, 43, 44, 63, 64, 65, 64, 63, 66, + 67, 63, 63, 63, 63, 63, 68, 63, 63, 69, + 63, 63, 70, 63, 63, 63, 63, 63, 63, 71, + + 63, 63, 63, 72, 72, 72, 72, 72, 72, 72, + 72, 72, 72, 72, 72, 72, 72, 72, 72, 72, + 72, 72, 72, 72, 72, 72, 72, 72, 72, 73, + 63, 74, 63, 63, 72, 72, 72, 72, 72, 72, + 72, 72, 72, 72, 72, 72, 72, 72, 72, 72, + 72, 72, 72, 72, 72, 72, 72, 72, 63, 63, + 63, 76, 76, 79, 79, 79, 79, 79, 79, 81, + 124, 503, 77, 77, 81, 125, 82, 82, 82, 207, + 208, 82, 82, 82, 83, 84, 85, 84, 83, 83, + 83, 83, 83, 83, 83, 83, 83, 83, 83, 83, + + 83, 83, 83, 83, 83, 83, 83, 83, 83, 83, + 83, 83, 83, 86, 86, 86, 86, 86, 86, 86, + 86, 86, 86, 86, 86, 86, 86, 86, 86, 86, + 86, 86, 86, 86, 86, 86, 86, 86, 86, 83, + 83, 83, 83, 83, 86, 86, 86, 86, 86, 86, + 86, 86, 86, 86, 86, 86, 86, 86, 86, 86, + 86, 86, 86, 86, 86, 86, 86, 86, 83, 83, + 83, 87, 88, 89, 88, 90, 91, 87, 92, 93, + 94, 95, 96, 97, 98, 99, 100, 101, 87, 102, + 103, 104, 104, 87, 105, 106, 107, 108, 87, 87, + + 109, 110, 111, 110, 110, 110, 110, 110, 110, 110, + 110, 110, 110, 112, 113, 110, 110, 110, 114, 115, + 110, 110, 110, 110, 110, 110, 116, 87, 117, 87, + 87, 109, 110, 111, 110, 110, 110, 110, 110, 110, + 110, 110, 110, 112, 113, 110, 110, 110, 114, 115, + 110, 110, 110, 110, 110, 87, 87, 87, 130, 131, + 182, 133, 134, 135, 196, 137, 138, 140, 229, 142, + 147, 136, 201, 132, 192, 192, 192, 145, 143, 123, + 139, 146, 201, 153, 141, 153, 212, 154, 154, 130, + 131, 133, 134, 144, 135, 137, 478, 138, 140, 142, + + 147, 136, 465, 132, 148, 182, 148, 411, 145, 143, + 139, 183, 146, 213, 141, 197, 212, 150, 202, 150, + 206, 206, 206, 144, 346, 151, 123, 123, 202, 123, + 123, 123, 123, 123, 123, 123, 123, 123, 123, 124, + 123, 123, 288, 213, 125, 254, 157, 214, 162, 123, + 123, 123, 123, 123, 123, 158, 183, 255, 129, 129, + 129, 288, 170, 204, 204, 218, 219, 218, 152, 128, + 160, 128, 153, 215, 161, 161, 128, 214, 128, 162, + 353, 123, 123, 123, 123, 123, 163, 152, 128, 160, + 128, 153, 170, 161, 161, 128, 205, 128, 216, 171, + + 223, 224, 223, 215, 288, 225, 275, 275, 162, 353, + 123, 123, 123, 172, 128, 162, 217, 163, 198, 166, + 288, 198, 162, 221, 232, 205, 167, 239, 216, 162, + 171, 198, 198, 128, 152, 128, 160, 128, 153, 162, + 161, 161, 128, 172, 128, 189, 162, 217, 162, 166, + 164, 173, 162, 168, 232, 162, 167, 162, 239, 162, + 233, 169, 162, 234, 165, 162, 235, 189, 174, 162, + 180, 162, 226, 224, 226, 175, 236, 227, 237, 162, + 128, 164, 173, 162, 168, 238, 162, 176, 162, 240, + 233, 169, 162, 234, 165, 162, 235, 162, 174, 162, + + 180, 162, 177, 162, 185, 175, 236, 178, 237, 241, + 244, 179, 504, 245, 162, 238, 246, 162, 176, 153, + 240, 153, 157, 154, 154, 231, 153, 162, 153, 254, + 154, 154, 177, 504, 162, 251, 252, 251, 178, 241, + 244, 255, 179, 245, 204, 204, 246, 162, 220, 220, + 221, 220, 220, 220, 220, 220, 220, 220, 220, 220, + 220, 220, 220, 220, 220, 220, 220, 220, 220, 220, + 220, 220, 220, 220, 220, 220, 220, 222, 222, 222, + 222, 222, 222, 222, 222, 222, 222, 222, 222, 222, + 222, 222, 222, 222, 222, 222, 222, 222, 222, 222, + + 222, 222, 222, 220, 220, 220, 220, 220, 222, 222, + 222, 222, 222, 222, 222, 222, 222, 222, 222, 222, + 222, 222, 222, 222, 222, 222, 222, 222, 222, 222, + 222, 222, 220, 220, 220, 242, 120, 247, 248, 249, + 148, 162, 148, 270, 153, 243, 153, 210, 154, 154, + 311, 312, 160, 150, 153, 150, 161, 161, 192, 192, + 192, 151, 196, 206, 206, 206, 242, 247, 248, 249, + 389, 162, 390, 270, 209, 243, 123, 253, 252, 253, + 123, 123, 123, 123, 123, 123, 123, 123, 123, 123, + 123, 123, 259, 254, 123, 355, 260, 162, 162, 123, + + 123, 123, 123, 123, 123, 255, 158, 356, 250, 129, + 129, 129, 231, 197, 203, 258, 282, 250, 128, 194, + 128, 250, 259, 162, 182, 128, 260, 162, 162, 250, + 433, 123, 123, 123, 123, 123, 152, 128, 160, 128, + 153, 265, 161, 161, 128, 258, 128, 282, 152, 128, + 160, 128, 153, 162, 161, 161, 128, 162, 128, 433, + 123, 123, 123, 128, 389, 261, 390, 262, 263, 269, + 162, 162, 265, 162, 264, 183, 266, 162, 162, 271, + 283, 267, 128, 257, 162, 162, 272, 162, 274, 162, + 284, 268, 190, 162, 128, 261, 162, 262, 263, 269, + + 162, 162, 285, 162, 273, 264, 266, 162, 162, 271, + 283, 162, 267, 257, 162, 162, 272, 286, 274, 162, + 284, 268, 293, 162, 276, 276, 162, 279, 279, 218, + 219, 218, 294, 285, 273, 223, 224, 223, 295, 296, + 225, 162, 223, 224, 223, 297, 298, 225, 286, 226, + 224, 226, 299, 293, 227, 300, 306, 277, 301, 189, + 280, 189, 294, 251, 252, 251, 333, 333, 295, 296, + 253, 252, 253, 335, 335, 186, 297, 298, 311, 312, + 504, 504, 299, 195, 195, 300, 277, 306, 301, 280, + 220, 220, 221, 220, 220, 220, 220, 220, 220, 220, + + 220, 220, 220, 220, 220, 220, 220, 220, 220, 220, + 220, 220, 220, 220, 220, 220, 220, 220, 220, 222, + 222, 222, 222, 222, 222, 222, 222, 222, 222, 222, + 222, 222, 222, 222, 222, 222, 222, 222, 222, 222, + 222, 222, 222, 222, 222, 220, 220, 220, 220, 220, + 222, 222, 222, 222, 222, 222, 222, 222, 222, 222, + 222, 222, 222, 222, 222, 222, 222, 222, 222, 222, + 222, 222, 222, 222, 220, 220, 220, 302, 303, 304, + 305, 307, 308, 309, 310, 250, 162, 355, 504, 200, + 200, 378, 378, 378, 250, 128, 162, 128, 250, 356, + + 185, 379, 128, 324, 122, 127, 250, 302, 303, 304, + 305, 307, 308, 162, 309, 310, 162, 152, 128, 160, + 128, 153, 320, 161, 161, 128, 162, 128, 152, 128, + 160, 128, 153, 324, 161, 161, 128, 342, 128, 162, + 128, 314, 318, 162, 319, 122, 316, 317, 162, 162, + 321, 162, 320, 337, 162, 315, 162, 323, 322, 120, + 162, 162, 162, 128, 325, 326, 328, 162, 342, 162, + 439, 329, 314, 318, 128, 319, 316, 317, 162, 162, + 321, 162, 327, 337, 162, 315, 162, 323, 322, 162, + 162, 162, 162, 330, 325, 326, 328, 162, 331, 439, + + 162, 329, 182, 338, 339, 162, 340, 341, 343, 504, + 344, 345, 347, 327, 348, 350, 332, 332, 349, 162, + 351, 352, 354, 504, 330, 162, 162, 504, 357, 331, + 162, 162, 360, 338, 339, 162, 340, 341, 504, 343, + 344, 345, 347, 162, 359, 348, 350, 162, 349, 162, + 351, 352, 354, 183, 162, 162, 162, 358, 357, 162, + 363, 162, 361, 360, 162, 364, 504, 162, 362, 368, + 162, 162, 162, 162, 366, 359, 365, 162, 367, 162, + 162, 162, 372, 346, 162, 373, 374, 358, 370, 162, + 363, 375, 361, 162, 162, 376, 364, 162, 362, 368, + + 162, 162, 162, 377, 380, 366, 365, 369, 367, 381, + 162, 162, 182, 372, 382, 373, 374, 383, 370, 384, + 385, 386, 375, 162, 387, 376, 371, 371, 388, 162, + 162, 391, 393, 377, 380, 392, 369, 394, 403, 381, + 162, 378, 378, 378, 395, 382, 162, 383, 162, 384, + 385, 379, 386, 162, 397, 387, 398, 396, 388, 162, + 162, 391, 393, 183, 162, 162, 392, 394, 400, 403, + 162, 399, 162, 404, 395, 162, 162, 407, 162, 408, + 162, 401, 423, 162, 397, 417, 398, 162, 396, 412, + 402, 418, 162, 504, 162, 162, 405, 405, 405, 400, + + 504, 399, 162, 404, 504, 162, 406, 407, 419, 408, + 162, 420, 401, 409, 409, 409, 417, 162, 421, 412, + 402, 418, 162, 410, 378, 378, 378, 413, 413, 413, + 415, 415, 415, 424, 379, 162, 435, 414, 419, 432, + 416, 420, 405, 405, 405, 162, 162, 425, 421, 409, + 409, 409, 406, 434, 411, 426, 413, 413, 413, 410, + 415, 415, 415, 427, 504, 162, 414, 435, 504, 432, + 416, 162, 162, 436, 162, 162, 162, 425, 162, 428, + 504, 162, 429, 434, 437, 426, 430, 162, 162, 431, + 405, 405, 405, 438, 427, 162, 409, 409, 409, 162, + + 406, 162, 162, 436, 162, 423, 410, 442, 162, 428, + 444, 162, 429, 443, 452, 437, 430, 162, 162, 431, + 413, 413, 413, 451, 438, 162, 415, 415, 415, 162, + 414, 440, 440, 440, 162, 162, 416, 447, 442, 446, + 455, 441, 445, 162, 443, 452, 162, 449, 448, 440, + 440, 440, 456, 451, 162, 450, 424, 423, 504, 441, + 457, 424, 162, 504, 162, 162, 504, 504, 447, 504, + 446, 455, 445, 162, 162, 504, 162, 448, 449, 453, + 453, 453, 456, 504, 162, 467, 450, 466, 162, 454, + 468, 457, 162, 458, 458, 458, 440, 440, 440, 460, + + 460, 460, 464, 459, 162, 162, 441, 471, 424, 461, + 462, 462, 462, 453, 453, 453, 467, 466, 162, 469, + 463, 468, 472, 454, 458, 458, 458, 460, 460, 460, + 462, 462, 462, 464, 459, 162, 476, 461, 471, 504, + 463, 453, 453, 453, 458, 458, 458, 470, 473, 469, + 504, 454, 162, 472, 459, 460, 460, 460, 462, 462, + 462, 479, 504, 162, 482, 461, 162, 476, 463, 162, + 474, 474, 474, 480, 477, 504, 481, 470, 473, 485, + 475, 162, 162, 487, 474, 474, 474, 483, 483, 483, + 489, 504, 479, 162, 475, 482, 162, 484, 486, 162, + + 162, 162, 488, 480, 491, 477, 481, 483, 483, 483, + 485, 162, 490, 487, 162, 493, 492, 484, 162, 162, + 495, 489, 162, 496, 497, 498, 504, 494, 499, 486, + 162, 162, 488, 500, 491, 162, 501, 504, 503, 504, + 162, 162, 504, 490, 162, 493, 492, 502, 162, 162, + 495, 504, 162, 496, 281, 497, 498, 494, 504, 499, + 504, 281, 281, 281, 500, 162, 501, 162, 504, 504, + 162, 162, 504, 504, 504, 504, 504, 502, 504, 504, + 504, 504, 504, 504, 504, 504, 504, 504, 504, 504, + 504, 504, 504, 504, 504, 504, 504, 162, 75, 75, + + 75, 75, 75, 75, 75, 75, 75, 75, 75, 75, + 75, 75, 75, 75, 75, 75, 75, 75, 75, 75, + 75, 75, 75, 75, 75, 75, 78, 78, 78, 78, 78, 78, 78, 78, 78, 78, 78, 78, 78, 78, - 78, 78, 78, 78, 78, 78, 78, 78, 116, 442, - 442, 442, 116, 116, 116, 116, 116, 116, 116, 116, - 116, 116, 116, 116, 116, 116, 116, 116, 116, 116, - 116, 116, 116, 116, 117, 117, 117, 117, 117, 117, - 117, 117, 117, 117, 117, 117, 117, 117, 117, 117, - 117, 117, 117, 117, 117, 117, 117, 117, 117, 117, - - 119, 119, 119, 119, 119, 119, 119, 121, 121, 442, - 121, 121, 121, 121, 121, 121, 121, 121, 121, 121, - 121, 121, 121, 121, 121, 121, 121, 121, 121, 121, - 121, 121, 121, 124, 124, 124, 124, 124, 124, 124, - 124, 124, 124, 124, 124, 124, 124, 124, 124, 124, - 124, 124, 124, 124, 124, 124, 124, 124, 124, 126, - 126, 126, 442, 126, 126, 442, 126, 442, 442, 126, - 126, 126, 126, 126, 126, 126, 126, 127, 127, 442, - 442, 442, 442, 127, 127, 127, 127, 127, 127, 127, - 143, 442, 143, 442, 442, 442, 143, 143, 442, 143, - - 442, 442, 442, 143, 442, 442, 442, 143, 143, 143, - 143, 143, 143, 143, 146, 442, 146, 442, 146, 146, - 442, 442, 442, 442, 442, 146, 146, 146, 146, 146, - 146, 146, 149, 442, 442, 149, 149, 149, 149, 149, - 442, 442, 442, 149, 149, 149, 149, 149, 149, 149, - 150, 150, 442, 150, 150, 150, 150, 150, 150, 150, - 150, 150, 150, 150, 150, 150, 150, 150, 150, 150, - 150, 150, 150, 150, 150, 150, 153, 442, 442, 442, - 153, 442, 153, 153, 442, 442, 442, 442, 153, 153, - 153, 153, 153, 153, 153, 156, 156, 156, 156, 156, - - 156, 156, 442, 156, 442, 442, 156, 156, 156, 156, - 156, 156, 156, 156, 171, 171, 171, 171, 171, 171, - 171, 171, 171, 171, 171, 171, 171, 171, 171, 171, - 171, 171, 171, 171, 171, 171, 171, 171, 171, 171, - 174, 174, 174, 174, 174, 174, 174, 174, 174, 174, - 174, 174, 174, 174, 174, 174, 174, 174, 174, 174, - 174, 174, 174, 174, 174, 174, 177, 177, 177, 442, - 442, 442, 442, 177, 177, 177, 177, 177, 177, 177, - 177, 178, 178, 178, 178, 178, 178, 178, 442, 178, - 178, 178, 178, 178, 178, 178, 178, 178, 178, 178, - - 178, 178, 178, 178, 178, 178, 178, 181, 442, 442, - 442, 181, 181, 181, 181, 181, 181, 181, 181, 181, + 78, 78, 78, 78, 78, 78, 78, 78, 78, 78, + 78, 78, 78, 78, 80, 80, 80, 80, 80, 80, + 80, 80, 80, 80, 80, 80, 80, 80, 80, 80, + 80, 80, 80, 80, 80, 80, 80, 80, 80, 80, + 80, 80, 118, 504, 504, 504, 118, 118, 118, 118, + 118, 118, 118, 118, 118, 118, 118, 118, 118, 118, + + 118, 118, 118, 118, 118, 118, 118, 118, 118, 118, + 119, 119, 119, 119, 119, 119, 119, 119, 119, 119, + 119, 119, 119, 119, 119, 119, 119, 119, 119, 119, + 119, 119, 119, 119, 119, 119, 119, 119, 121, 121, + 121, 121, 121, 121, 121, 123, 123, 504, 123, 123, + 123, 123, 123, 123, 123, 123, 123, 123, 123, 123, + 123, 123, 123, 123, 123, 123, 123, 123, 123, 123, + 123, 123, 123, 126, 126, 126, 126, 126, 126, 126, + 126, 126, 126, 126, 126, 126, 126, 126, 126, 126, + 126, 126, 126, 126, 126, 126, 126, 126, 126, 126, + + 126, 128, 128, 128, 504, 504, 128, 128, 504, 128, + 504, 504, 504, 128, 128, 128, 128, 128, 128, 128, + 128, 129, 129, 504, 504, 504, 504, 504, 129, 129, + 129, 129, 129, 129, 129, 149, 504, 149, 504, 504, + 504, 149, 149, 504, 149, 504, 504, 504, 504, 149, + 504, 504, 504, 504, 149, 149, 149, 149, 149, 149, + 149, 152, 504, 152, 504, 152, 504, 152, 504, 504, + 504, 504, 504, 504, 152, 152, 152, 152, 152, 152, + 152, 155, 504, 504, 155, 155, 504, 155, 155, 155, + 504, 504, 504, 504, 155, 155, 155, 155, 155, 155, + + 155, 156, 156, 504, 156, 156, 156, 156, 156, 156, + 156, 156, 156, 156, 156, 156, 156, 156, 156, 156, + 156, 156, 156, 156, 156, 156, 156, 156, 156, 159, + 504, 504, 504, 159, 504, 504, 159, 159, 504, 504, + 504, 504, 504, 159, 159, 159, 159, 159, 159, 159, + 162, 162, 162, 162, 162, 504, 162, 162, 504, 162, + 504, 504, 504, 162, 162, 162, 162, 162, 162, 162, + 162, 181, 181, 181, 181, 181, 181, 181, 181, 181, 181, 181, 181, 181, 181, 181, 181, 181, 181, 181, - 181, 181, 181, 183, 183, 183, 442, 442, 442, 442, - 183, 183, 183, 183, 183, 183, 183, 183, 185, 185, - 185, 185, 185, 185, 442, 185, 185, 185, 185, 185, - 185, 185, 185, 185, 185, 185, 185, 185, 185, 185, - 185, 185, 185, 185, 189, 442, 189, 442, 442, 442, - 442, 442, 442, 442, 442, 442, 442, 442, 189, 189, - 189, 189, 189, 189, 189, 190, 190, 190, 190, 190, - - 190, 190, 190, 190, 190, 190, 190, 190, 190, 190, - 190, 190, 190, 190, 190, 190, 190, 190, 190, 190, - 190, 201, 201, 442, 442, 442, 442, 201, 201, 201, - 201, 201, 201, 201, 201, 218, 218, 218, 218, 218, - 218, 218, 218, 218, 218, 218, 218, 218, 218, 218, - 218, 218, 218, 218, 218, 218, 218, 218, 218, 218, - 218, 220, 220, 442, 220, 220, 220, 220, 220, 220, - 220, 220, 220, 220, 220, 220, 220, 220, 220, 220, - 220, 220, 220, 220, 220, 220, 220, 127, 127, 127, - 442, 442, 442, 127, 127, 127, 127, 127, 127, 127, - - 143, 442, 143, 442, 442, 442, 143, 143, 442, 143, - 442, 442, 442, 143, 442, 442, 442, 143, 143, 143, - 143, 143, 143, 143, 146, 442, 146, 442, 146, 146, - 442, 442, 442, 442, 442, 146, 146, 146, 146, 146, - 146, 146, 149, 149, 149, 149, 149, 442, 442, 442, - 149, 149, 149, 149, 149, 149, 149, 152, 442, 442, - 442, 152, 442, 442, 442, 442, 442, 442, 442, 152, - 152, 152, 152, 152, 152, 152, 236, 442, 442, 236, - 442, 236, 442, 236, 236, 442, 442, 236, 236, 442, - 236, 236, 236, 236, 236, 236, 236, 150, 150, 150, - - 150, 150, 150, 150, 150, 150, 150, 150, 150, 150, - 150, 150, 150, 150, 150, 150, 150, 150, 150, 150, - 150, 150, 150, 153, 442, 442, 442, 442, 442, 153, - 153, 442, 153, 153, 153, 442, 442, 442, 153, 153, - 153, 153, 153, 153, 153, 242, 442, 442, 242, 242, - 242, 242, 242, 242, 242, 442, 242, 242, 442, 242, - 242, 242, 242, 242, 242, 242, 242, 156, 156, 156, - 156, 156, 156, 156, 442, 156, 442, 442, 156, 156, - 156, 156, 156, 156, 156, 156, 171, 171, 171, 171, - 171, 171, 171, 171, 171, 171, 171, 171, 171, 171, - - 171, 171, 171, 171, 171, 171, 171, 171, 171, 171, - 171, 171, 171, 442, 442, 442, 442, 442, 442, 171, - 442, 442, 442, 442, 171, 171, 171, 442, 442, 171, - 171, 174, 174, 174, 174, 174, 174, 174, 174, 174, - 174, 174, 174, 174, 174, 174, 174, 174, 174, 174, - 174, 174, 174, 174, 174, 174, 174, 177, 177, 177, - 442, 442, 442, 442, 177, 177, 177, 177, 177, 177, - 177, 177, 178, 178, 178, 178, 178, 178, 178, 442, - 178, 178, 178, 178, 178, 178, 178, 178, 178, 178, - 178, 178, 178, 178, 178, 178, 178, 178, 181, 442, - - 442, 442, 181, 181, 181, 181, 181, 181, 181, 181, + 181, 181, 181, 181, 181, 181, 181, 181, 181, 184, + + 184, 184, 184, 184, 184, 184, 184, 184, 184, 184, + 184, 184, 184, 184, 184, 184, 184, 184, 184, 184, + 184, 184, 184, 184, 184, 184, 184, 187, 504, 187, + 187, 504, 504, 504, 504, 504, 187, 187, 187, 187, + 187, 187, 187, 187, 188, 188, 188, 188, 188, 188, + 188, 504, 188, 188, 188, 188, 188, 188, 188, 188, + 188, 188, 188, 188, 188, 188, 188, 188, 188, 188, + 188, 188, 191, 504, 504, 504, 191, 191, 191, 191, + 191, 191, 191, 191, 191, 191, 191, 191, 191, 191, + 191, 191, 191, 191, 191, 191, 191, 191, 191, 191, + + 193, 504, 193, 193, 504, 504, 504, 504, 504, 193, + 193, 193, 193, 193, 193, 193, 193, 195, 195, 195, + 195, 195, 195, 504, 195, 195, 195, 195, 195, 195, + 195, 195, 195, 195, 195, 195, 195, 195, 195, 195, + 195, 195, 195, 195, 195, 199, 504, 199, 504, 504, + 504, 504, 199, 199, 504, 504, 504, 504, 504, 504, + 199, 199, 199, 199, 199, 199, 199, 199, 200, 200, + 200, 200, 200, 200, 200, 200, 200, 200, 200, 200, + 200, 200, 200, 200, 200, 200, 200, 200, 200, 200, + 200, 200, 200, 200, 200, 200, 211, 211, 504, 504, + + 504, 504, 504, 211, 211, 211, 211, 211, 211, 211, + 211, 118, 118, 118, 118, 118, 118, 118, 118, 118, + 118, 118, 118, 118, 118, 118, 118, 118, 118, 118, + 118, 118, 118, 118, 118, 118, 118, 118, 118, 119, + 119, 119, 119, 119, 119, 119, 119, 119, 119, 119, + 119, 119, 119, 119, 119, 119, 119, 119, 119, 119, + 119, 119, 119, 119, 119, 119, 119, 228, 228, 228, + 228, 228, 228, 228, 228, 228, 228, 228, 228, 228, + 228, 228, 228, 228, 228, 228, 228, 228, 228, 228, + 228, 228, 228, 228, 228, 230, 230, 504, 230, 230, + + 230, 230, 230, 230, 230, 230, 230, 230, 230, 230, + 230, 230, 230, 230, 230, 230, 230, 230, 230, 230, + 230, 230, 230, 128, 128, 128, 504, 504, 128, 128, + 504, 128, 504, 504, 504, 128, 128, 128, 128, 128, + 128, 128, 128, 129, 129, 129, 504, 504, 504, 504, + 129, 129, 129, 129, 129, 129, 129, 149, 504, 149, + 504, 504, 504, 149, 149, 504, 149, 504, 504, 504, + 504, 149, 504, 504, 504, 504, 149, 149, 149, 149, + 149, 149, 149, 152, 504, 152, 504, 152, 504, 152, + 504, 504, 504, 504, 504, 504, 152, 152, 152, 152, + + 152, 152, 152, 155, 155, 504, 155, 155, 155, 504, + 504, 504, 504, 155, 155, 155, 155, 155, 155, 155, + 158, 504, 504, 504, 158, 504, 504, 504, 504, 504, + 504, 504, 504, 504, 158, 158, 158, 158, 158, 158, + 158, 250, 504, 504, 250, 504, 250, 504, 250, 504, + 250, 504, 504, 250, 250, 504, 504, 250, 250, 250, + 250, 250, 250, 250, 156, 156, 156, 156, 156, 156, + 156, 156, 156, 156, 156, 156, 156, 156, 156, 156, + 156, 156, 156, 156, 156, 156, 156, 156, 156, 156, + 156, 156, 159, 504, 504, 504, 504, 504, 159, 159, + + 504, 504, 159, 159, 159, 504, 504, 504, 504, 159, + 159, 159, 159, 159, 159, 159, 256, 504, 504, 256, + 256, 256, 256, 256, 504, 256, 256, 504, 256, 256, + 504, 504, 256, 256, 256, 256, 256, 256, 256, 256, + 162, 162, 162, 162, 162, 504, 162, 162, 504, 162, + 504, 504, 504, 162, 162, 162, 162, 162, 162, 162, + 162, 181, 181, 181, 181, 181, 181, 181, 181, 181, 181, 181, 181, 181, 181, 181, 181, 181, 181, 181, - 181, 181, 181, 181, 183, 183, 183, 442, 442, 442, - 442, 183, 183, 183, 183, 183, 183, 183, 183, 185, - 185, 185, 185, 185, 185, 442, 185, 185, 185, 185, - 185, 185, 185, 185, 185, 185, 185, 185, 185, 185, - 185, 185, 185, 185, 185, 185, 185, 442, 442, 442, - 442, 442, 185, 442, 442, 442, 442, 442, 442, 185, - 442, 442, 185, 442, 185, 189, 189, 189, 189, 189, - 189, 189, 260, 442, 442, 442, 442, 442, 260, 260, - - 260, 260, 442, 442, 442, 442, 260, 260, 260, 260, - 260, 260, 260, 260, 190, 190, 190, 190, 190, 190, - 190, 190, 190, 190, 190, 190, 190, 190, 190, 190, - 190, 190, 190, 190, 190, 190, 190, 190, 190, 190, - 190, 190, 442, 442, 442, 442, 442, 190, 442, 442, - 442, 442, 442, 442, 190, 442, 442, 190, 442, 190, - 263, 442, 442, 442, 442, 442, 263, 263, 263, 201, - 201, 442, 442, 442, 442, 201, 201, 201, 201, 201, - 201, 201, 201, 210, 210, 210, 210, 210, 210, 210, - 210, 210, 210, 210, 210, 210, 210, 210, 210, 210, - - 210, 210, 210, 210, 210, 210, 210, 210, 210, 269, - 269, 269, 269, 269, 269, 269, 269, 269, 269, 269, - 269, 269, 269, 269, 269, 269, 269, 269, 269, 269, - 269, 269, 269, 269, 269, 121, 121, 121, 121, 121, - 121, 121, 121, 121, 121, 121, 121, 121, 121, 121, - 121, 121, 121, 121, 121, 121, 121, 121, 121, 121, - 121, 271, 271, 271, 271, 271, 271, 271, 271, 271, - 271, 271, 271, 271, 271, 271, 271, 271, 271, 271, - 271, 271, 271, 271, 271, 271, 271, 272, 272, 442, - 272, 272, 272, 272, 272, 272, 272, 272, 272, 272, - - 272, 272, 272, 272, 272, 272, 272, 272, 272, 272, - 272, 272, 272, 273, 273, 442, 273, 273, 273, 273, - 273, 273, 273, 273, 273, 273, 273, 273, 273, 273, - 273, 273, 273, 273, 273, 273, 273, 273, 273, 220, - 220, 442, 220, 220, 220, 220, 220, 220, 220, 220, + 181, 181, 181, 181, 181, 181, 181, 181, 181, 181, + 504, 504, 504, 504, 504, 504, 504, 181, 504, 504, + + 504, 504, 181, 504, 181, 181, 504, 504, 181, 181, + 184, 184, 184, 184, 184, 184, 184, 184, 184, 184, + 184, 184, 184, 184, 184, 184, 184, 184, 184, 184, + 184, 184, 184, 184, 184, 184, 184, 184, 187, 504, + 187, 187, 504, 504, 504, 504, 504, 187, 187, 187, + 187, 187, 187, 187, 187, 188, 188, 188, 188, 188, + 188, 188, 504, 188, 188, 188, 188, 188, 188, 188, + 188, 188, 188, 188, 188, 188, 188, 188, 188, 188, + 188, 188, 188, 191, 504, 504, 504, 191, 191, 191, + 191, 191, 191, 191, 191, 191, 191, 191, 191, 191, + + 191, 191, 191, 191, 191, 191, 191, 191, 191, 191, + 191, 193, 504, 193, 193, 504, 504, 504, 504, 504, + 193, 193, 193, 193, 193, 193, 193, 193, 195, 195, + 195, 195, 195, 195, 504, 195, 195, 195, 195, 195, + 195, 195, 195, 195, 195, 195, 195, 195, 195, 195, + 195, 195, 195, 195, 195, 195, 195, 195, 504, 504, + 504, 504, 504, 504, 195, 504, 504, 504, 504, 504, + 504, 504, 195, 504, 504, 195, 504, 195, 199, 199, + 199, 199, 199, 199, 199, 199, 278, 504, 504, 504, + 504, 504, 278, 278, 504, 278, 278, 504, 504, 504, + + 504, 278, 278, 278, 278, 278, 278, 278, 278, 278, + 200, 200, 200, 200, 200, 200, 200, 200, 200, 200, + 200, 200, 200, 200, 200, 200, 200, 200, 200, 200, + 200, 200, 200, 200, 200, 200, 200, 200, 200, 200, + 504, 504, 504, 504, 504, 504, 200, 504, 504, 504, + 504, 504, 504, 504, 200, 504, 504, 200, 504, 200, + 211, 211, 504, 504, 504, 504, 504, 211, 211, 211, + 211, 211, 211, 211, 211, 220, 220, 220, 220, 220, 220, 220, 220, 220, 220, 220, 220, 220, 220, 220, - 220, 220, 220, 220, 220, 274, 274, 442, 274, 274, - 274, 274, 274, 274, 274, 274, 274, 274, 274, 274, - 274, 274, 274, 274, 274, 274, 274, 274, 274, 274, - 274, 126, 126, 126, 442, 126, 126, 442, 126, 442, - - 442, 126, 126, 126, 126, 126, 126, 126, 126, 236, - 442, 442, 236, 442, 236, 442, 236, 236, 442, 442, - 236, 236, 442, 236, 236, 236, 236, 236, 236, 236, - 291, 291, 442, 442, 442, 442, 442, 442, 442, 442, - 442, 442, 442, 442, 442, 291, 291, 291, 291, 291, - 291, 291, 242, 442, 442, 242, 242, 242, 242, 242, - 242, 242, 442, 242, 242, 442, 242, 242, 242, 242, - 242, 242, 242, 242, 156, 156, 156, 156, 156, 156, - 156, 442, 156, 442, 442, 156, 156, 156, 156, 156, - 156, 156, 156, 171, 171, 171, 171, 171, 171, 171, - - 171, 171, 171, 171, 171, 171, 171, 171, 171, 171, - 171, 171, 171, 171, 171, 171, 171, 171, 171, 308, - 308, 442, 442, 442, 442, 308, 308, 308, 310, 310, - 442, 442, 442, 442, 310, 310, 310, 185, 185, 442, - 442, 442, 442, 185, 185, 185, 190, 190, 442, 442, - 442, 442, 190, 190, 190, 377, 377, 377, 377, 377, - 377, 377, 377, 377, 377, 377, 377, 377, 377, 377, - 377, 377, 377, 377, 377, 377, 377, 377, 377, 377, - 377, 15, 442, 442, 442, 442, 442, 442, 442, 442, - 442, 442, 442, 442, 442, 442, 442, 442, 442, 442, - - 442, 442, 442, 442, 442, 442, 442, 442, 442, 442, - 442, 442, 442, 442, 442, 442, 442, 442, 442, 442, - 442, 442, 442, 442, 442, 442, 442, 442, 442, 442, - 442, 442, 442, 442, 442, 442, 442, 442, 442, 442, - 442, 442, 442, 442, 442, 442, 442, 442, 442, 442, - 442, 442, 442, 442, 442, 442, 442, 442, 442, 442, - 442, 442 + 220, 220, 220, 220, 220, 220, 220, 220, 220, 220, + + 220, 220, 220, 287, 287, 287, 287, 287, 287, 287, + 287, 287, 287, 287, 287, 287, 287, 287, 287, 287, + 287, 287, 287, 287, 287, 287, 287, 287, 287, 287, + 287, 123, 123, 123, 123, 123, 123, 123, 123, 123, + 123, 123, 123, 123, 123, 123, 123, 123, 123, 123, + 123, 123, 123, 123, 123, 123, 123, 123, 123, 289, + 289, 289, 289, 289, 289, 289, 289, 289, 289, 289, + 289, 289, 289, 289, 289, 289, 289, 289, 289, 289, + 289, 289, 289, 289, 289, 289, 289, 290, 290, 504, + 290, 290, 290, 290, 290, 290, 290, 290, 290, 290, + + 290, 290, 290, 290, 290, 290, 290, 290, 290, 290, + 290, 290, 290, 290, 290, 291, 291, 504, 291, 291, + 291, 291, 291, 291, 291, 291, 291, 291, 291, 291, + 291, 291, 291, 291, 291, 291, 291, 291, 291, 291, + 291, 291, 291, 230, 230, 504, 230, 230, 230, 230, + 230, 230, 230, 230, 230, 230, 230, 230, 230, 230, + 230, 230, 230, 230, 230, 230, 230, 230, 230, 230, + 230, 292, 292, 504, 292, 292, 292, 292, 292, 292, + 292, 292, 292, 292, 292, 292, 292, 292, 292, 292, + 292, 292, 292, 292, 292, 292, 292, 292, 292, 128, + + 128, 128, 504, 504, 128, 128, 504, 128, 504, 504, + 504, 128, 128, 128, 128, 128, 128, 128, 128, 250, + 504, 504, 250, 504, 250, 504, 250, 504, 250, 504, + 504, 250, 250, 504, 504, 250, 250, 250, 250, 250, + 250, 250, 313, 313, 504, 504, 504, 504, 504, 504, + 504, 504, 504, 504, 504, 504, 504, 504, 504, 313, + 313, 313, 313, 313, 313, 313, 256, 504, 504, 256, + 256, 256, 256, 256, 504, 256, 256, 504, 256, 256, + 504, 504, 256, 256, 256, 256, 256, 256, 256, 256, + 162, 162, 162, 162, 162, 504, 162, 162, 504, 162, + + 504, 504, 504, 162, 162, 162, 162, 162, 162, 162, + 162, 181, 181, 181, 181, 181, 181, 181, 181, 181, + 181, 181, 181, 181, 181, 181, 181, 181, 181, 181, + 181, 181, 181, 181, 181, 181, 181, 181, 181, 334, + 334, 504, 504, 504, 504, 504, 334, 334, 334, 336, + 336, 504, 504, 504, 504, 504, 336, 336, 336, 195, + 195, 504, 504, 504, 504, 504, 195, 195, 195, 200, + 200, 504, 504, 504, 504, 504, 200, 200, 200, 422, + 422, 422, 422, 422, 422, 422, 422, 422, 422, 422, + 422, 422, 422, 422, 422, 422, 422, 422, 422, 422, + + 422, 422, 422, 422, 422, 422, 422, 15, 504, 504, + 504, 504, 504, 504, 504, 504, 504, 504, 504, 504, + 504, 504, 504, 504, 504, 504, 504, 504, 504, 504, + 504, 504, 504, 504, 504, 504, 504, 504, 504, 504, + 504, 504, 504, 504, 504, 504, 504, 504, 504, 504, + 504, 504, 504, 504, 504, 504, 504, 504, 504, 504, + 504, 504, 504, 504, 504, 504, 504, 504, 504, 504, + 504, 504, 504, 504, 504, 504, 504, 504, 504, 504, + 504, 504, 504, 504, 504, 504, 504, 504, 504, 504, + 504, 504, 504, 504, 504 + } ; -static yyconst flex_int16_t yy_chk[3263] = +static yyconst flex_int16_t yy_chk[3696] = { 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, @@ -985,7 +1053,7 @@ static yyconst flex_int16_t yy_chk[3263] = 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, - 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, + 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, @@ -994,358 +1062,407 @@ static yyconst flex_int16_t yy_chk[3263] = 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, + 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, + 2, 2, 2, 2, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, + 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, - 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, - 5, 6, 7, 7, 7, 8, 8, 8, 9, 25, - 439, 5, 6, 10, 25, 9, 9, 9, 104, 104, - 10, 10, 10, 11, 11, 11, 11, 11, 11, 11, + 3, 5, 6, 7, 7, 7, 8, 8, 8, 9, + 25, 501, 5, 6, 10, 25, 9, 9, 9, 106, + 106, 10, 10, 10, 11, 11, 11, 11, 11, 11, + 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, + 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, - 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, 11, - 11, 11, 11, 13, 13, 13, 13, 13, 13, 13, + 11, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, + 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, - 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, - 13, 13, 13, 30, 31, 64, 32, 33, 34, 416, - 35, 36, 37, 38, 401, 34, 40, 31, 39, 45, - 89, 45, 90, 45, 45, 90, 38, 80, 80, 80, - 30, 31, 32, 33, 107, 34, 35, 36, 37, 93, - 38, 34, 40, 31, 44, 39, 44, 173, 173, 48, - 194, 194, 38, 64, 101, 101, 271, 44, 48, 44, - 107, 48, 48, 48, 269, 44, 47, 47, 89, 47, - 47, 47, 47, 47, 47, 47, 47, 47, 47, 47, - - 47, 47, 93, 109, 47, 101, 102, 102, 102, 47, - 47, 47, 47, 47, 47, 49, 49, 49, 49, 49, - 123, 49, 49, 49, 113, 49, 217, 110, 111, 109, - 101, 123, 112, 49, 55, 116, 116, 116, 47, 47, - 47, 47, 50, 50, 50, 50, 50, 55, 50, 50, - 50, 113, 50, 110, 111, 128, 135, 49, 112, 171, - 49, 55, 215, 47, 47, 47, 326, 52, 326, 53, - 120, 120, 120, 55, 129, 120, 210, 52, 130, 53, - 52, 128, 53, 135, 50, 51, 51, 51, 51, 51, - 131, 51, 51, 51, 52, 51, 53, 54, 56, 57, - - 129, 51, 54, 52, 130, 53, 52, 171, 53, 54, - 54, 56, 57, 56, 51, 179, 131, 57, 122, 122, - 122, 60, 364, 122, 54, 56, 57, 51, 51, 54, - 132, 134, 178, 60, 60, 54, 54, 56, 57, 56, - 51, 58, 59, 57, 136, 58, 59, 364, 60, 137, - 138, 139, 140, 136, 58, 59, 132, 134, 58, 60, - 60, 143, 352, 143, 352, 143, 143, 174, 58, 59, - 152, 136, 58, 59, 152, 137, 138, 139, 140, 136, - 58, 59, 152, 151, 58, 119, 119, 119, 119, 119, - 119, 119, 119, 119, 119, 119, 119, 119, 119, 119, - - 119, 119, 119, 119, 119, 119, 119, 119, 119, 119, - 119, 119, 119, 119, 119, 119, 119, 119, 119, 119, - 119, 119, 119, 119, 119, 119, 119, 119, 119, 119, - 119, 119, 119, 119, 119, 119, 119, 119, 119, 119, - 119, 119, 119, 119, 119, 119, 119, 119, 119, 119, - 119, 119, 119, 119, 119, 119, 119, 119, 119, 119, - 119, 119, 119, 119, 119, 141, 142, 146, 142, 146, - 190, 146, 146, 148, 145, 148, 127, 148, 148, 142, - 117, 142, 149, 149, 149, 160, 160, 142, 162, 241, - 241, 141, 150, 150, 150, 150, 150, 150, 150, 150, - - 150, 150, 150, 150, 150, 150, 150, 150, 106, 153, - 150, 160, 160, 190, 162, 150, 150, 150, 150, 150, - 150, 153, 153, 105, 167, 153, 153, 153, 153, 155, - 202, 155, 159, 155, 155, 158, 161, 159, 163, 154, - 161, 163, 158, 100, 150, 150, 150, 150, 154, 154, - 167, 154, 154, 182, 182, 182, 154, 202, 159, 374, - 154, 158, 161, 159, 163, 185, 161, 163, 158, 150, - 150, 150, 156, 156, 156, 156, 156, 88, 156, 156, - 156, 75, 156, 164, 374, 157, 157, 157, 157, 157, - 154, 157, 157, 157, 165, 157, 168, 164, 166, 203, - - 168, 74, 169, 165, 204, 166, 73, 165, 169, 206, - 164, 170, 170, 185, 156, 187, 187, 68, 157, 192, - 192, 165, 168, 164, 166, 203, 168, 157, 169, 165, - 204, 166, 207, 165, 169, 65, 206, 170, 170, 196, - 196, 196, 240, 222, 157, 46, 187, 208, 208, 208, - 192, 213, 213, 213, 240, 27, 213, 258, 258, 207, - 214, 214, 214, 261, 261, 214, 24, 216, 216, 216, - 222, 187, 216, 289, 289, 192, 212, 212, 212, 212, - 212, 212, 212, 212, 212, 212, 212, 212, 212, 212, - 212, 212, 212, 212, 212, 212, 212, 212, 212, 212, - - 212, 212, 212, 212, 212, 212, 212, 212, 212, 212, - 212, 212, 212, 212, 212, 212, 212, 212, 212, 212, - 212, 212, 212, 212, 212, 212, 212, 212, 212, 212, - 212, 212, 212, 212, 212, 212, 212, 212, 212, 212, - 212, 212, 212, 212, 212, 212, 212, 212, 212, 212, - 212, 212, 212, 212, 212, 212, 223, 224, 224, 225, - 226, 20, 227, 228, 229, 230, 231, 15, 233, 234, - 235, 237, 237, 237, 239, 239, 239, 250, 0, 246, - 0, 250, 223, 224, 224, 246, 225, 226, 227, 228, - 229, 230, 242, 231, 233, 234, 278, 235, 290, 290, - - 0, 242, 242, 250, 242, 242, 246, 250, 0, 242, - 247, 246, 280, 242, 243, 243, 243, 243, 243, 247, - 243, 243, 243, 278, 243, 244, 244, 244, 244, 244, - 0, 244, 244, 244, 248, 244, 249, 247, 243, 280, - 245, 0, 248, 242, 249, 247, 245, 245, 251, 252, - 0, 244, 251, 253, 254, 252, 243, 255, 254, 0, - 248, 255, 249, 256, 0, 243, 245, 244, 248, 256, - 249, 257, 245, 245, 251, 265, 252, 244, 251, 253, - 254, 252, 267, 255, 254, 257, 257, 255, 268, 275, - 256, 277, 281, 0, 282, 256, 283, 284, 325, 285, - - 287, 265, 288, 292, 291, 291, 292, 293, 267, 295, - 325, 295, 296, 299, 268, 275, 291, 277, 281, 257, - 282, 297, 283, 294, 284, 285, 287, 298, 288, 292, - 294, 297, 292, 293, 298, 300, 295, 295, 296, 299, - 300, 301, 0, 303, 302, 304, 312, 301, 297, 294, - 302, 305, 0, 298, 307, 307, 294, 297, 305, 311, - 298, 300, 309, 309, 306, 314, 300, 315, 301, 303, - 302, 304, 312, 301, 316, 317, 302, 305, 306, 306, - 318, 318, 318, 319, 305, 320, 311, 321, 322, 323, - 318, 314, 330, 327, 315, 327, 328, 331, 331, 331, - - 316, 317, 329, 337, 329, 333, 328, 331, 333, 338, - 319, 320, 306, 321, 332, 322, 323, 334, 330, 327, - 332, 327, 334, 328, 339, 348, 336, 335, 329, 337, - 329, 333, 328, 335, 333, 331, 336, 340, 0, 353, - 0, 332, 342, 334, 343, 0, 332, 0, 334, 0, - 0, 339, 348, 336, 335, 0, 349, 338, 350, 335, - 351, 331, 336, 340, 341, 341, 341, 355, 342, 355, - 343, 344, 344, 344, 341, 345, 345, 345, 347, 347, - 347, 344, 349, 0, 350, 345, 351, 353, 347, 354, - 354, 354, 356, 355, 363, 355, 357, 357, 357, 354, - - 356, 358, 358, 358, 359, 0, 357, 360, 0, 361, - 362, 358, 359, 0, 360, 367, 361, 362, 356, 368, - 363, 377, 365, 365, 365, 373, 356, 354, 379, 0, - 0, 359, 365, 360, 357, 361, 362, 395, 359, 358, - 360, 367, 361, 362, 380, 376, 368, 369, 369, 369, - 0, 380, 373, 354, 371, 371, 371, 369, 386, 387, - 357, 375, 375, 375, 371, 358, 383, 381, 390, 377, - 380, 375, 376, 381, 382, 0, 379, 380, 383, 382, - 384, 384, 384, 385, 386, 395, 387, 0, 397, 385, - 384, 0, 383, 402, 381, 390, 388, 388, 388, 381, - - 0, 382, 398, 383, 398, 382, 388, 391, 391, 391, - 385, 392, 392, 392, 397, 385, 405, 391, 384, 402, - 411, 392, 394, 394, 394, 396, 396, 396, 0, 398, - 398, 0, 394, 0, 412, 396, 399, 399, 399, 400, - 400, 400, 405, 410, 384, 413, 399, 411, 419, 400, - 403, 403, 403, 417, 406, 406, 406, 408, 408, 408, - 403, 412, 418, 396, 406, 414, 420, 408, 421, 410, - 421, 414, 413, 423, 399, 419, 424, 400, 424, 417, - 425, 426, 0, 427, 428, 429, 428, 426, 418, 396, - 430, 431, 414, 420, 433, 421, 421, 414, 430, 423, - - 399, 435, 424, 400, 424, 432, 432, 425, 426, 427, - 428, 429, 428, 426, 434, 440, 430, 431, 437, 436, - 0, 433, 438, 434, 430, 436, 0, 438, 435, 0, - 0, 432, 432, 0, 0, 0, 0, 0, 0, 0, - 0, 434, 0, 440, 437, 0, 436, 0, 438, 434, - 0, 436, 0, 438, 0, 0, 0, 0, 0, 0, - 0, 0, 0, 0, 0, 0, 0, 0, 0, 440, - 443, 443, 443, 443, 443, 443, 443, 443, 443, 443, - 443, 443, 443, 443, 443, 443, 443, 443, 443, 443, - 443, 443, 443, 443, 443, 443, 444, 444, 444, 444, - - 444, 444, 444, 444, 444, 444, 444, 444, 444, 444, - 444, 444, 444, 444, 444, 444, 444, 444, 444, 444, - 444, 444, 445, 445, 445, 445, 445, 445, 445, 445, - 445, 445, 445, 445, 445, 445, 445, 445, 445, 445, - 445, 445, 445, 445, 445, 445, 445, 445, 446, 0, - 0, 0, 446, 446, 446, 446, 446, 446, 446, 446, - 446, 446, 446, 446, 446, 446, 446, 446, 446, 446, - 446, 446, 446, 446, 447, 447, 447, 447, 447, 447, - 447, 447, 447, 447, 447, 447, 447, 447, 447, 447, - 447, 447, 447, 447, 447, 447, 447, 447, 447, 447, - - 448, 448, 448, 448, 448, 448, 448, 449, 449, 0, - 449, 449, 449, 449, 449, 449, 449, 449, 449, 449, - 449, 449, 449, 449, 449, 449, 449, 449, 449, 449, - 449, 449, 449, 450, 450, 450, 450, 450, 450, 450, - 450, 450, 450, 450, 450, 450, 450, 450, 450, 450, - 450, 450, 450, 450, 450, 450, 450, 450, 450, 451, - 451, 451, 0, 451, 451, 0, 451, 0, 0, 451, - 451, 451, 451, 451, 451, 451, 451, 452, 452, 0, - 0, 0, 0, 452, 452, 452, 452, 452, 452, 452, - 453, 0, 453, 0, 0, 0, 453, 453, 0, 453, - - 0, 0, 0, 453, 0, 0, 0, 453, 453, 453, - 453, 453, 453, 453, 454, 0, 454, 0, 454, 454, - 0, 0, 0, 0, 0, 454, 454, 454, 454, 454, - 454, 454, 455, 0, 0, 455, 455, 455, 455, 455, - 0, 0, 0, 455, 455, 455, 455, 455, 455, 455, - 456, 456, 0, 456, 456, 456, 456, 456, 456, 456, - 456, 456, 456, 456, 456, 456, 456, 456, 456, 456, - 456, 456, 456, 456, 456, 456, 457, 0, 0, 0, - 457, 0, 457, 457, 0, 0, 0, 0, 457, 457, - 457, 457, 457, 457, 457, 458, 458, 458, 458, 458, - - 458, 458, 0, 458, 0, 0, 458, 458, 458, 458, - 458, 458, 458, 458, 459, 459, 459, 459, 459, 459, - 459, 459, 459, 459, 459, 459, 459, 459, 459, 459, - 459, 459, 459, 459, 459, 459, 459, 459, 459, 459, - 460, 460, 460, 460, 460, 460, 460, 460, 460, 460, - 460, 460, 460, 460, 460, 460, 460, 460, 460, 460, - 460, 460, 460, 460, 460, 460, 461, 461, 461, 0, - 0, 0, 0, 461, 461, 461, 461, 461, 461, 461, - 461, 462, 462, 462, 462, 462, 462, 462, 0, 462, - 462, 462, 462, 462, 462, 462, 462, 462, 462, 462, - - 462, 462, 462, 462, 462, 462, 462, 463, 0, 0, - 0, 463, 463, 463, 463, 463, 463, 463, 463, 463, - 463, 463, 463, 463, 463, 463, 463, 463, 463, 463, - 463, 463, 463, 464, 464, 464, 0, 0, 0, 0, - 464, 464, 464, 464, 464, 464, 464, 464, 465, 465, - 465, 465, 465, 465, 0, 465, 465, 465, 465, 465, - 465, 465, 465, 465, 465, 465, 465, 465, 465, 465, - 465, 465, 465, 465, 466, 0, 466, 0, 0, 0, - 0, 0, 0, 0, 0, 0, 0, 0, 466, 466, - 466, 466, 466, 466, 466, 467, 467, 467, 467, 467, - - 467, 467, 467, 467, 467, 467, 467, 467, 467, 467, - 467, 467, 467, 467, 467, 467, 467, 467, 467, 467, - 467, 468, 468, 0, 0, 0, 0, 468, 468, 468, - 468, 468, 468, 468, 468, 469, 469, 469, 469, 469, - 469, 469, 469, 469, 469, 469, 469, 469, 469, 469, - 469, 469, 469, 469, 469, 469, 469, 469, 469, 469, - 469, 470, 470, 0, 470, 470, 470, 470, 470, 470, - 470, 470, 470, 470, 470, 470, 470, 470, 470, 470, - 470, 470, 470, 470, 470, 470, 470, 471, 471, 471, - 0, 0, 0, 471, 471, 471, 471, 471, 471, 471, - - 472, 0, 472, 0, 0, 0, 472, 472, 0, 472, - 0, 0, 0, 472, 0, 0, 0, 472, 472, 472, - 472, 472, 472, 472, 473, 0, 473, 0, 473, 473, - 0, 0, 0, 0, 0, 473, 473, 473, 473, 473, - 473, 473, 474, 474, 474, 474, 474, 0, 0, 0, - 474, 474, 474, 474, 474, 474, 474, 475, 0, 0, - 0, 475, 0, 0, 0, 0, 0, 0, 0, 475, - 475, 475, 475, 475, 475, 475, 476, 0, 0, 476, - 0, 476, 0, 476, 476, 0, 0, 476, 476, 0, - 476, 476, 476, 476, 476, 476, 476, 477, 477, 477, - - 477, 477, 477, 477, 477, 477, 477, 477, 477, 477, - 477, 477, 477, 477, 477, 477, 477, 477, 477, 477, - 477, 477, 477, 478, 0, 0, 0, 0, 0, 478, - 478, 0, 478, 478, 478, 0, 0, 0, 478, 478, - 478, 478, 478, 478, 478, 479, 0, 0, 479, 479, - 479, 479, 479, 479, 479, 0, 479, 479, 0, 479, - 479, 479, 479, 479, 479, 479, 479, 480, 480, 480, - 480, 480, 480, 480, 0, 480, 0, 0, 480, 480, - 480, 480, 480, 480, 480, 480, 481, 481, 481, 481, - 481, 481, 481, 481, 481, 481, 481, 481, 481, 481, - - 481, 481, 481, 481, 481, 481, 481, 481, 481, 481, - 481, 481, 482, 0, 0, 0, 0, 0, 0, 482, - 0, 0, 0, 0, 482, 482, 482, 0, 0, 482, - 482, 483, 483, 483, 483, 483, 483, 483, 483, 483, - 483, 483, 483, 483, 483, 483, 483, 483, 483, 483, - 483, 483, 483, 483, 483, 483, 483, 484, 484, 484, - 0, 0, 0, 0, 484, 484, 484, 484, 484, 484, - 484, 484, 485, 485, 485, 485, 485, 485, 485, 0, - 485, 485, 485, 485, 485, 485, 485, 485, 485, 485, - 485, 485, 485, 485, 485, 485, 485, 485, 486, 0, - - 0, 0, 486, 486, 486, 486, 486, 486, 486, 486, - 486, 486, 486, 486, 486, 486, 486, 486, 486, 486, - 486, 486, 486, 486, 487, 487, 487, 0, 0, 0, - 0, 487, 487, 487, 487, 487, 487, 487, 487, 488, - 488, 488, 488, 488, 488, 0, 488, 488, 488, 488, - 488, 488, 488, 488, 488, 488, 488, 488, 488, 488, - 488, 488, 488, 488, 488, 489, 489, 0, 0, 0, - 0, 0, 489, 0, 0, 0, 0, 0, 0, 489, - 0, 0, 489, 0, 489, 490, 490, 490, 490, 490, - 490, 490, 491, 0, 0, 0, 0, 0, 491, 491, - - 491, 491, 0, 0, 0, 0, 491, 491, 491, 491, - 491, 491, 491, 491, 492, 492, 492, 492, 492, 492, - 492, 492, 492, 492, 492, 492, 492, 492, 492, 492, - 492, 492, 492, 492, 492, 492, 492, 492, 492, 492, - 493, 493, 0, 0, 0, 0, 0, 493, 0, 0, - 0, 0, 0, 0, 493, 0, 0, 493, 0, 493, - 494, 0, 0, 0, 0, 0, 494, 494, 494, 495, - 495, 0, 0, 0, 0, 495, 495, 495, 495, 495, - 495, 495, 495, 496, 496, 496, 496, 496, 496, 496, - 496, 496, 496, 496, 496, 496, 496, 496, 496, 496, - - 496, 496, 496, 496, 496, 496, 496, 496, 496, 497, - 497, 497, 497, 497, 497, 497, 497, 497, 497, 497, - 497, 497, 497, 497, 497, 497, 497, 497, 497, 497, - 497, 497, 497, 497, 497, 498, 498, 498, 498, 498, - 498, 498, 498, 498, 498, 498, 498, 498, 498, 498, - 498, 498, 498, 498, 498, 498, 498, 498, 498, 498, - 498, 499, 499, 499, 499, 499, 499, 499, 499, 499, - 499, 499, 499, 499, 499, 499, 499, 499, 499, 499, - 499, 499, 499, 499, 499, 499, 499, 500, 500, 0, - 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, - - 500, 500, 500, 500, 500, 500, 500, 500, 500, 500, - 500, 500, 500, 501, 501, 0, 501, 501, 501, 501, - 501, 501, 501, 501, 501, 501, 501, 501, 501, 501, - 501, 501, 501, 501, 501, 501, 501, 501, 501, 502, - 502, 0, 502, 502, 502, 502, 502, 502, 502, 502, - 502, 502, 502, 502, 502, 502, 502, 502, 502, 502, - 502, 502, 502, 502, 502, 503, 503, 0, 503, 503, - 503, 503, 503, 503, 503, 503, 503, 503, 503, 503, - 503, 503, 503, 503, 503, 503, 503, 503, 503, 503, - 503, 504, 504, 504, 0, 504, 504, 0, 504, 0, - - 0, 504, 504, 504, 504, 504, 504, 504, 504, 505, - 0, 0, 505, 0, 505, 0, 505, 505, 0, 0, - 505, 505, 0, 505, 505, 505, 505, 505, 505, 505, - 506, 506, 0, 0, 0, 0, 0, 0, 0, 0, - 0, 0, 0, 0, 0, 506, 506, 506, 506, 506, - 506, 506, 507, 0, 0, 507, 507, 507, 507, 507, - 507, 507, 0, 507, 507, 0, 507, 507, 507, 507, - 507, 507, 507, 507, 508, 508, 508, 508, 508, 508, - 508, 0, 508, 0, 0, 508, 508, 508, 508, 508, - 508, 508, 508, 509, 509, 509, 509, 509, 509, 509, + 13, 13, 13, 13, 13, 13, 13, 13, 30, 31, + 66, 32, 33, 34, 91, 35, 36, 37, 125, 38, + 41, 34, 95, 31, 82, 82, 82, 40, 39, 125, + 36, 40, 200, 46, 37, 46, 109, 46, 46, 30, + 31, 32, 33, 39, 34, 35, 472, 36, 37, 38, + + 41, 34, 451, 31, 45, 181, 45, 380, 40, 39, + 36, 66, 40, 111, 37, 91, 109, 45, 95, 45, + 104, 104, 104, 39, 301, 45, 48, 48, 200, 48, + 48, 48, 48, 48, 48, 48, 48, 48, 48, 48, + 48, 48, 289, 111, 48, 254, 49, 112, 56, 48, + 48, 48, 48, 48, 48, 49, 181, 254, 49, 49, + 49, 287, 56, 103, 103, 118, 118, 118, 50, 50, + 50, 50, 50, 113, 50, 50, 50, 112, 50, 56, + 309, 48, 48, 48, 48, 48, 50, 51, 51, 51, + 51, 51, 56, 51, 51, 51, 103, 51, 114, 57, + + 122, 122, 122, 113, 227, 122, 183, 183, 53, 309, + 48, 48, 48, 57, 50, 54, 115, 50, 92, 53, + 225, 92, 53, 220, 130, 103, 54, 138, 114, 54, + 57, 92, 92, 51, 52, 52, 52, 52, 52, 53, + 52, 52, 52, 57, 52, 189, 54, 115, 55, 53, + 52, 58, 53, 55, 130, 59, 54, 62, 138, 54, + 131, 55, 55, 132, 52, 58, 133, 188, 58, 59, + 62, 62, 124, 124, 124, 59, 134, 124, 136, 55, + 52, 52, 58, 60, 55, 137, 59, 60, 62, 139, + 131, 55, 55, 132, 52, 58, 133, 60, 58, 59, + + 62, 62, 60, 61, 184, 59, 134, 61, 136, 140, + 142, 61, 157, 143, 60, 137, 144, 61, 60, 149, + 139, 149, 151, 149, 149, 129, 152, 60, 152, 158, + 152, 152, 60, 158, 61, 155, 155, 155, 61, 140, + 142, 158, 61, 143, 204, 204, 144, 61, 121, 121, + 121, 121, 121, 121, 121, 121, 121, 121, 121, 121, + 121, 121, 121, 121, 121, 121, 121, 121, 121, 121, + 121, 121, 121, 121, 121, 121, 121, 121, 121, 121, + 121, 121, 121, 121, 121, 121, 121, 121, 121, 121, + 121, 121, 121, 121, 121, 121, 121, 121, 121, 121, + 121, 121, 121, 121, 121, 121, 121, 121, 121, 121, + 121, 121, 121, 121, 121, 121, 121, 121, 121, 121, + 121, 121, 121, 121, 121, 121, 121, 121, 121, 121, + 121, 121, 121, 121, 121, 141, 119, 145, 146, 147, + 148, 168, 148, 176, 154, 141, 154, 108, 154, 154, + 255, 255, 161, 148, 161, 148, 161, 161, 192, 192, + 192, 148, 195, 206, 206, 206, 141, 145, 146, 147, + 356, 168, 356, 176, 107, 141, 156, 156, 156, 156, + 156, 156, 156, 156, 156, 156, 156, 156, 156, 156, + 156, 156, 165, 159, 156, 355, 166, 166, 165, 156, + + 156, 156, 156, 156, 156, 159, 159, 355, 160, 159, + 159, 159, 159, 195, 102, 164, 212, 160, 160, 90, + 160, 160, 165, 164, 371, 160, 166, 166, 165, 160, + 404, 156, 156, 156, 156, 156, 162, 162, 162, 162, + 162, 172, 162, 162, 162, 164, 162, 212, 163, 163, + 163, 163, 163, 164, 163, 163, 163, 172, 163, 404, + 156, 156, 156, 160, 389, 167, 389, 169, 170, 175, + 167, 169, 172, 170, 171, 371, 173, 175, 173, 177, + 213, 174, 162, 163, 177, 171, 178, 172, 180, 180, + 214, 174, 77, 178, 163, 167, 174, 169, 170, 175, + + 167, 169, 216, 170, 179, 171, 173, 175, 173, 177, + 213, 179, 174, 163, 177, 171, 178, 217, 180, 180, + 214, 174, 232, 178, 197, 197, 174, 202, 202, 218, + 218, 218, 233, 216, 179, 223, 223, 223, 234, 234, + 223, 179, 224, 224, 224, 235, 236, 224, 217, 226, + 226, 226, 237, 232, 226, 238, 244, 197, 239, 76, + 202, 75, 233, 251, 251, 251, 276, 276, 234, 234, + 253, 253, 253, 279, 279, 70, 235, 236, 311, 311, + 312, 312, 237, 333, 333, 238, 197, 244, 239, 202, + 222, 222, 222, 222, 222, 222, 222, 222, 222, 222, + + 222, 222, 222, 222, 222, 222, 222, 222, 222, 222, + 222, 222, 222, 222, 222, 222, 222, 222, 222, 222, + 222, 222, 222, 222, 222, 222, 222, 222, 222, 222, + 222, 222, 222, 222, 222, 222, 222, 222, 222, 222, + 222, 222, 222, 222, 222, 222, 222, 222, 222, 222, + 222, 222, 222, 222, 222, 222, 222, 222, 222, 222, + 222, 222, 222, 222, 222, 222, 222, 222, 222, 222, + 222, 222, 222, 222, 222, 222, 222, 240, 241, 242, + 243, 246, 247, 248, 249, 256, 270, 313, 313, 335, + 335, 344, 344, 344, 256, 256, 266, 256, 256, 313, + + 67, 344, 256, 266, 47, 27, 256, 240, 241, 242, + 243, 246, 247, 262, 248, 249, 270, 257, 257, 257, + 257, 257, 262, 257, 257, 257, 266, 257, 258, 258, + 258, 258, 258, 266, 258, 258, 258, 296, 258, 259, + 256, 257, 260, 262, 261, 24, 259, 259, 265, 260, + 263, 263, 262, 283, 261, 258, 264, 265, 264, 20, + 267, 268, 271, 257, 267, 268, 271, 272, 296, 259, + 418, 272, 257, 260, 258, 261, 259, 259, 265, 260, + 263, 263, 269, 283, 261, 258, 264, 265, 264, 269, + 267, 268, 271, 273, 267, 268, 271, 272, 274, 418, + + 273, 272, 275, 285, 286, 274, 293, 295, 298, 15, + 299, 300, 302, 269, 303, 305, 275, 275, 304, 269, + 306, 308, 310, 0, 273, 314, 315, 0, 314, 274, + 273, 318, 319, 285, 286, 274, 293, 295, 0, 298, + 299, 300, 302, 319, 317, 303, 305, 317, 304, 316, + 306, 308, 310, 275, 320, 314, 315, 316, 314, 321, + 323, 318, 320, 319, 322, 324, 0, 328, 321, 329, + 325, 327, 324, 319, 326, 317, 325, 317, 327, 316, + 331, 326, 337, 322, 320, 338, 340, 316, 331, 321, + 323, 341, 320, 330, 322, 342, 324, 328, 321, 329, + + 325, 327, 324, 343, 345, 326, 325, 330, 327, 346, + 331, 326, 332, 337, 347, 338, 340, 348, 331, 349, + 350, 351, 341, 330, 352, 342, 332, 332, 353, 357, + 359, 357, 359, 343, 345, 358, 330, 360, 372, 346, + 362, 361, 361, 361, 362, 347, 358, 348, 364, 349, + 350, 361, 351, 365, 364, 352, 365, 363, 353, 357, + 359, 357, 359, 332, 363, 366, 358, 360, 367, 372, + 362, 366, 370, 373, 362, 367, 358, 375, 364, 376, + 361, 368, 390, 365, 364, 384, 365, 369, 363, 381, + 369, 385, 368, 0, 363, 366, 374, 374, 374, 367, + + 0, 366, 370, 373, 0, 367, 374, 375, 386, 376, + 361, 387, 368, 377, 377, 377, 384, 369, 388, 381, + 369, 385, 368, 377, 378, 378, 378, 382, 382, 382, + 383, 383, 383, 390, 378, 395, 408, 382, 386, 403, + 383, 387, 391, 391, 391, 392, 393, 392, 388, 394, + 394, 394, 391, 407, 395, 393, 396, 396, 396, 394, + 397, 397, 397, 398, 0, 395, 396, 408, 0, 403, + 397, 399, 398, 411, 400, 392, 393, 392, 401, 399, + 0, 391, 400, 407, 412, 393, 401, 402, 394, 402, + 405, 405, 405, 417, 398, 396, 409, 409, 409, 397, + + 405, 399, 398, 411, 400, 422, 409, 420, 401, 399, + 424, 391, 400, 421, 433, 412, 401, 402, 394, 402, + 413, 413, 413, 432, 417, 396, 415, 415, 415, 397, + 413, 419, 419, 419, 425, 428, 415, 427, 420, 426, + 436, 419, 425, 427, 421, 433, 426, 430, 428, 429, + 429, 429, 437, 432, 430, 431, 422, 444, 0, 429, + 438, 424, 431, 0, 425, 428, 0, 0, 427, 0, + 426, 436, 425, 427, 446, 0, 426, 428, 430, 434, + 434, 434, 437, 0, 430, 455, 431, 452, 429, 434, + 456, 438, 431, 439, 439, 439, 440, 440, 440, 442, + + 442, 442, 447, 439, 446, 447, 440, 465, 444, 442, + 443, 443, 443, 445, 445, 445, 455, 452, 429, 457, + 443, 456, 466, 445, 448, 448, 448, 449, 449, 449, + 450, 450, 450, 447, 448, 447, 469, 449, 465, 0, + 450, 453, 453, 453, 458, 458, 458, 464, 467, 457, + 0, 453, 445, 466, 458, 460, 460, 460, 462, 462, + 462, 473, 0, 448, 478, 460, 449, 469, 462, 450, + 468, 468, 468, 476, 470, 0, 477, 464, 467, 480, + 468, 470, 445, 485, 474, 474, 474, 479, 479, 479, + 487, 0, 473, 448, 474, 478, 449, 479, 481, 450, + + 486, 481, 486, 476, 489, 470, 477, 483, 483, 483, + 480, 470, 488, 485, 490, 491, 490, 483, 492, 488, + 493, 487, 494, 494, 495, 496, 0, 492, 497, 481, + 486, 481, 486, 498, 489, 496, 499, 0, 502, 0, + 498, 500, 0, 488, 490, 491, 490, 500, 492, 488, + 493, 0, 494, 494, 559, 495, 496, 492, 0, 497, + 0, 559, 559, 559, 498, 496, 499, 502, 0, 0, + 498, 500, 0, 0, 0, 0, 0, 500, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 502, 505, 505, + + 505, 505, 505, 505, 505, 505, 505, 505, 505, 505, + 505, 505, 505, 505, 505, 505, 505, 505, 505, 505, + 505, 505, 505, 505, 505, 505, 506, 506, 506, 506, + 506, 506, 506, 506, 506, 506, 506, 506, 506, 506, + 506, 506, 506, 506, 506, 506, 506, 506, 506, 506, + 506, 506, 506, 506, 507, 507, 507, 507, 507, 507, + 507, 507, 507, 507, 507, 507, 507, 507, 507, 507, + 507, 507, 507, 507, 507, 507, 507, 507, 507, 507, + 507, 507, 508, 0, 0, 0, 508, 508, 508, 508, + 508, 508, 508, 508, 508, 508, 508, 508, 508, 508, + + 508, 508, 508, 508, 508, 508, 508, 508, 508, 508, + 509, 509, 509, 509, 509, 509, 509, 509, 509, 509, 509, 509, 509, 509, 509, 509, 509, 509, 509, 509, - 509, 509, 509, 509, 509, 509, 509, 509, 509, 510, - 510, 0, 0, 0, 0, 510, 510, 510, 511, 511, - 0, 0, 0, 0, 511, 511, 511, 512, 512, 0, - 0, 0, 0, 512, 512, 512, 513, 513, 0, 0, - 0, 0, 513, 513, 513, 514, 514, 514, 514, 514, - 514, 514, 514, 514, 514, 514, 514, 514, 514, 514, - 514, 514, 514, 514, 514, 514, 514, 514, 514, 514, - 514, 442, 442, 442, 442, 442, 442, 442, 442, 442, - 442, 442, 442, 442, 442, 442, 442, 442, 442, 442, - - 442, 442, 442, 442, 442, 442, 442, 442, 442, 442, - 442, 442, 442, 442, 442, 442, 442, 442, 442, 442, - 442, 442, 442, 442, 442, 442, 442, 442, 442, 442, - 442, 442, 442, 442, 442, 442, 442, 442, 442, 442, - 442, 442, 442, 442, 442, 442, 442, 442, 442, 442, - 442, 442, 442, 442, 442, 442, 442, 442, 442, 442, - 442, 442 + 509, 509, 509, 509, 509, 509, 509, 509, 510, 510, + 510, 510, 510, 510, 510, 511, 511, 0, 511, 511, + 511, 511, 511, 511, 511, 511, 511, 511, 511, 511, + 511, 511, 511, 511, 511, 511, 511, 511, 511, 511, + 511, 511, 511, 512, 512, 512, 512, 512, 512, 512, + 512, 512, 512, 512, 512, 512, 512, 512, 512, 512, + 512, 512, 512, 512, 512, 512, 512, 512, 512, 512, + + 512, 513, 513, 513, 0, 0, 513, 513, 0, 513, + 0, 0, 0, 513, 513, 513, 513, 513, 513, 513, + 513, 514, 514, 0, 0, 0, 0, 0, 514, 514, + 514, 514, 514, 514, 514, 515, 0, 515, 0, 0, + 0, 515, 515, 0, 515, 0, 0, 0, 0, 515, + 0, 0, 0, 0, 515, 515, 515, 515, 515, 515, + 515, 516, 0, 516, 0, 516, 0, 516, 0, 0, + 0, 0, 0, 0, 516, 516, 516, 516, 516, 516, + 516, 517, 0, 0, 517, 517, 0, 517, 517, 517, + 0, 0, 0, 0, 517, 517, 517, 517, 517, 517, + + 517, 518, 518, 0, 518, 518, 518, 518, 518, 518, + 518, 518, 518, 518, 518, 518, 518, 518, 518, 518, + 518, 518, 518, 518, 518, 518, 518, 518, 518, 519, + 0, 0, 0, 519, 0, 0, 519, 519, 0, 0, + 0, 0, 0, 519, 519, 519, 519, 519, 519, 519, + 520, 520, 520, 520, 520, 0, 520, 520, 0, 520, + 0, 0, 0, 520, 520, 520, 520, 520, 520, 520, + 520, 521, 521, 521, 521, 521, 521, 521, 521, 521, + 521, 521, 521, 521, 521, 521, 521, 521, 521, 521, + 521, 521, 521, 521, 521, 521, 521, 521, 521, 522, + + 522, 522, 522, 522, 522, 522, 522, 522, 522, 522, + 522, 522, 522, 522, 522, 522, 522, 522, 522, 522, + 522, 522, 522, 522, 522, 522, 522, 523, 0, 523, + 523, 0, 0, 0, 0, 0, 523, 523, 523, 523, + 523, 523, 523, 523, 524, 524, 524, 524, 524, 524, + 524, 0, 524, 524, 524, 524, 524, 524, 524, 524, + 524, 524, 524, 524, 524, 524, 524, 524, 524, 524, + 524, 524, 525, 0, 0, 0, 525, 525, 525, 525, + 525, 525, 525, 525, 525, 525, 525, 525, 525, 525, + 525, 525, 525, 525, 525, 525, 525, 525, 525, 525, + + 526, 0, 526, 526, 0, 0, 0, 0, 0, 526, + 526, 526, 526, 526, 526, 526, 526, 527, 527, 527, + 527, 527, 527, 0, 527, 527, 527, 527, 527, 527, + 527, 527, 527, 527, 527, 527, 527, 527, 527, 527, + 527, 527, 527, 527, 527, 528, 0, 528, 0, 0, + 0, 0, 528, 528, 0, 0, 0, 0, 0, 0, + 528, 528, 528, 528, 528, 528, 528, 528, 529, 529, + 529, 529, 529, 529, 529, 529, 529, 529, 529, 529, + 529, 529, 529, 529, 529, 529, 529, 529, 529, 529, + 529, 529, 529, 529, 529, 529, 530, 530, 0, 0, + + 0, 0, 0, 530, 530, 530, 530, 530, 530, 530, + 530, 531, 531, 531, 531, 531, 531, 531, 531, 531, + 531, 531, 531, 531, 531, 531, 531, 531, 531, 531, + 531, 531, 531, 531, 531, 531, 531, 531, 531, 532, + 532, 532, 532, 532, 532, 532, 532, 532, 532, 532, + 532, 532, 532, 532, 532, 532, 532, 532, 532, 532, + 532, 532, 532, 532, 532, 532, 532, 533, 533, 533, + 533, 533, 533, 533, 533, 533, 533, 533, 533, 533, + 533, 533, 533, 533, 533, 533, 533, 533, 533, 533, + 533, 533, 533, 533, 533, 534, 534, 0, 534, 534, + + 534, 534, 534, 534, 534, 534, 534, 534, 534, 534, + 534, 534, 534, 534, 534, 534, 534, 534, 534, 534, + 534, 534, 534, 535, 535, 535, 0, 0, 535, 535, + 0, 535, 0, 0, 0, 535, 535, 535, 535, 535, + 535, 535, 535, 536, 536, 536, 0, 0, 0, 0, + 536, 536, 536, 536, 536, 536, 536, 537, 0, 537, + 0, 0, 0, 537, 537, 0, 537, 0, 0, 0, + 0, 537, 0, 0, 0, 0, 537, 537, 537, 537, + 537, 537, 537, 538, 0, 538, 0, 538, 0, 538, + 0, 0, 0, 0, 0, 0, 538, 538, 538, 538, + + 538, 538, 538, 539, 539, 0, 539, 539, 539, 0, + 0, 0, 0, 539, 539, 539, 539, 539, 539, 539, + 540, 0, 0, 0, 540, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 540, 540, 540, 540, 540, 540, + 540, 541, 0, 0, 541, 0, 541, 0, 541, 0, + 541, 0, 0, 541, 541, 0, 0, 541, 541, 541, + 541, 541, 541, 541, 542, 542, 542, 542, 542, 542, + 542, 542, 542, 542, 542, 542, 542, 542, 542, 542, + 542, 542, 542, 542, 542, 542, 542, 542, 542, 542, + 542, 542, 543, 0, 0, 0, 0, 0, 543, 543, + + 0, 0, 543, 543, 543, 0, 0, 0, 0, 543, + 543, 543, 543, 543, 543, 543, 544, 0, 0, 544, + 544, 544, 544, 544, 0, 544, 544, 0, 544, 544, + 0, 0, 544, 544, 544, 544, 544, 544, 544, 544, + 545, 545, 545, 545, 545, 0, 545, 545, 0, 545, + 0, 0, 0, 545, 545, 545, 545, 545, 545, 545, + 545, 546, 546, 546, 546, 546, 546, 546, 546, 546, + 546, 546, 546, 546, 546, 546, 546, 546, 546, 546, + 546, 546, 546, 546, 546, 546, 546, 546, 546, 547, + 0, 0, 0, 0, 0, 0, 0, 547, 0, 0, + + 0, 0, 547, 0, 547, 547, 0, 0, 547, 547, + 548, 548, 548, 548, 548, 548, 548, 548, 548, 548, + 548, 548, 548, 548, 548, 548, 548, 548, 548, 548, + 548, 548, 548, 548, 548, 548, 548, 548, 549, 0, + 549, 549, 0, 0, 0, 0, 0, 549, 549, 549, + 549, 549, 549, 549, 549, 550, 550, 550, 550, 550, + 550, 550, 0, 550, 550, 550, 550, 550, 550, 550, + 550, 550, 550, 550, 550, 550, 550, 550, 550, 550, + 550, 550, 550, 551, 0, 0, 0, 551, 551, 551, + 551, 551, 551, 551, 551, 551, 551, 551, 551, 551, + + 551, 551, 551, 551, 551, 551, 551, 551, 551, 551, + 551, 552, 0, 552, 552, 0, 0, 0, 0, 0, + 552, 552, 552, 552, 552, 552, 552, 552, 553, 553, + 553, 553, 553, 553, 0, 553, 553, 553, 553, 553, + 553, 553, 553, 553, 553, 553, 553, 553, 553, 553, + 553, 553, 553, 553, 553, 553, 554, 554, 0, 0, + 0, 0, 0, 0, 554, 0, 0, 0, 0, 0, + 0, 0, 554, 0, 0, 554, 0, 554, 555, 555, + 555, 555, 555, 555, 555, 555, 556, 0, 0, 0, + 0, 0, 556, 556, 0, 556, 556, 0, 0, 0, + + 0, 556, 556, 556, 556, 556, 556, 556, 556, 556, + 557, 557, 557, 557, 557, 557, 557, 557, 557, 557, + 557, 557, 557, 557, 557, 557, 557, 557, 557, 557, + 557, 557, 557, 557, 557, 557, 557, 557, 558, 558, + 0, 0, 0, 0, 0, 0, 558, 0, 0, 0, + 0, 0, 0, 0, 558, 0, 0, 558, 0, 558, + 560, 560, 0, 0, 0, 0, 0, 560, 560, 560, + 560, 560, 560, 560, 560, 561, 561, 561, 561, 561, + 561, 561, 561, 561, 561, 561, 561, 561, 561, 561, + 561, 561, 561, 561, 561, 561, 561, 561, 561, 561, + + 561, 561, 561, 562, 562, 562, 562, 562, 562, 562, + 562, 562, 562, 562, 562, 562, 562, 562, 562, 562, + 562, 562, 562, 562, 562, 562, 562, 562, 562, 562, + 562, 563, 563, 563, 563, 563, 563, 563, 563, 563, + 563, 563, 563, 563, 563, 563, 563, 563, 563, 563, + 563, 563, 563, 563, 563, 563, 563, 563, 563, 564, + 564, 564, 564, 564, 564, 564, 564, 564, 564, 564, + 564, 564, 564, 564, 564, 564, 564, 564, 564, 564, + 564, 564, 564, 564, 564, 564, 564, 565, 565, 0, + 565, 565, 565, 565, 565, 565, 565, 565, 565, 565, + + 565, 565, 565, 565, 565, 565, 565, 565, 565, 565, + 565, 565, 565, 565, 565, 566, 566, 0, 566, 566, + 566, 566, 566, 566, 566, 566, 566, 566, 566, 566, + 566, 566, 566, 566, 566, 566, 566, 566, 566, 566, + 566, 566, 566, 567, 567, 0, 567, 567, 567, 567, + 567, 567, 567, 567, 567, 567, 567, 567, 567, 567, + 567, 567, 567, 567, 567, 567, 567, 567, 567, 567, + 567, 568, 568, 0, 568, 568, 568, 568, 568, 568, + 568, 568, 568, 568, 568, 568, 568, 568, 568, 568, + 568, 568, 568, 568, 568, 568, 568, 568, 568, 569, + + 569, 569, 0, 0, 569, 569, 0, 569, 0, 0, + 0, 569, 569, 569, 569, 569, 569, 569, 569, 570, + 0, 0, 570, 0, 570, 0, 570, 0, 570, 0, + 0, 570, 570, 0, 0, 570, 570, 570, 570, 570, + 570, 570, 571, 571, 0, 0, 0, 0, 0, 0, + 0, 0, 0, 0, 0, 0, 0, 0, 0, 571, + 571, 571, 571, 571, 571, 571, 572, 0, 0, 572, + 572, 572, 572, 572, 0, 572, 572, 0, 572, 572, + 0, 0, 572, 572, 572, 572, 572, 572, 572, 572, + 573, 573, 573, 573, 573, 0, 573, 573, 0, 573, + + 0, 0, 0, 573, 573, 573, 573, 573, 573, 573, + 573, 574, 574, 574, 574, 574, 574, 574, 574, 574, + 574, 574, 574, 574, 574, 574, 574, 574, 574, 574, + 574, 574, 574, 574, 574, 574, 574, 574, 574, 575, + 575, 0, 0, 0, 0, 0, 575, 575, 575, 576, + 576, 0, 0, 0, 0, 0, 576, 576, 576, 577, + 577, 0, 0, 0, 0, 0, 577, 577, 577, 578, + 578, 0, 0, 0, 0, 0, 578, 578, 578, 579, + 579, 579, 579, 579, 579, 579, 579, 579, 579, 579, + 579, 579, 579, 579, 579, 579, 579, 579, 579, 579, + + 579, 579, 579, 579, 579, 579, 579, 504, 504, 504, + 504, 504, 504, 504, 504, 504, 504, 504, 504, 504, + 504, 504, 504, 504, 504, 504, 504, 504, 504, 504, + 504, 504, 504, 504, 504, 504, 504, 504, 504, 504, + 504, 504, 504, 504, 504, 504, 504, 504, 504, 504, + 504, 504, 504, 504, 504, 504, 504, 504, 504, 504, + 504, 504, 504, 504, 504, 504, 504, 504, 504, 504, + 504, 504, 504, 504, 504, 504, 504, 504, 504, 504, + 504, 504, 504, 504, 504, 504, 504, 504, 504, 504, + 504, 504, 504, 504, 504 + } ; /* Table of booleans, true if rule could match eol. */ -static yyconst flex_int32_t yy_rule_can_match_eol[98] = +static yyconst flex_int32_t yy_rule_can_match_eol[102] = { 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 1, 1, 1, - 1, 1, 1, 1, 1, 1, 0, 0, 0, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, - 1, 0, 0, 0, 0, 1, 0, 1, 0, 0, 0, 1, 1, 0, 1, 1, 0, 0, }; + 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 1, 1, 1, 1, 1, 0, 0, + 0, 0, 0, 0, 1, 0, 0, 0, 0, 1, 0, 1, 0, 0, 0, 1, 1, 0, 1, 1, + 0, 0, }; static yy_state_type yy_last_accepting_state; static char *yy_last_accepting_cpos; @@ -1373,26 +1490,28 @@ char *yytext; * cases. So while we hope that cfsysline support can be dropped some time in * the future, we will probably keep these useful constructs. * -* Copyright 2011-2012 Rainer Gerhards and Adiscon GmbH. +* Copyright 2011-2014 Rainer Gerhards and Adiscon GmbH. * * This file is part of the rsyslog runtime library. * -* The rsyslog runtime library is free software: you can redistribute it and/or modify -* it under the terms of the GNU Lesser General Public License as published by -* the Free Software Foundation, either version 3 of the License, or -* (at your option) any later version. -* -* The rsyslog runtime library is distributed in the hope that it will be useful, -* but WITHOUT ANY WARRANTY; without even the implied warranty of -* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -* GNU Lesser General Public License for more details. -* -* You should have received a copy of the GNU Lesser General Public License -* along with the rsyslog runtime library. If not, see <http://www.gnu.org/licenses/>. -* -* A copy of the GPL can be found in the file "COPYING" in this distribution. -* A copy of the LGPL can be found in the file "COPYING.LESSER" in this distribution. +* Licensed under the Apache License, Version 2.0 (the "License"); +* you may not use this file except in compliance with the License. +* You may obtain a copy of the License at +* +* http://www.apache.org/licenses/LICENSE-2.0 +* -or- +* see COPYING.ASL20 in the source distribution +* +* Unless required by applicable law or agreed to in writing, software +* distributed under the License is distributed on an "AS IS" BASIS, +* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +* See the License for the specific language governing permissions and +* limitations under the License. */ +#line 32 "lexer.l" +#include "config.h" +#include "parserif.h" +extern char *strdup(char*); /* somehow we do not get this from string.h... */ /*%option noyywrap nodefault case-insensitive */ /* avoid compiler warning: `yyunput' defined but not used */ #define YY_NO_INPUT 1 @@ -1418,7 +1537,7 @@ char *yytext; * wrote this ugly, but the price needed to pay in order to remain * compatible to the previous format. */ -#line 63 "lexer.l" +#line 67 "lexer.l" #include <ctype.h> #include <stdio.h> #include <stdlib.h> @@ -1447,9 +1566,11 @@ extern int yydebug; /* somehow, I need these prototype even though the headers are * included. I guess that's some autotools magic I don't understand... */ +#if !defined(__FreeBSD__) int fileno(FILE *stream); +#endif -#line 1453 "lexer.c" +#line 1574 "lexer.c" #define INITIAL 0 #define INOBJ 1 @@ -1643,11 +1764,11 @@ YY_DECL register char *yy_cp, *yy_bp; register int yy_act; -#line 95 "lexer.l" +#line 101 "lexer.l" /* keywords */ -#line 1651 "lexer.c" +#line 1772 "lexer.c" if ( !(yy_init) ) { @@ -1701,13 +1822,13 @@ yy_match: while ( yy_chk[yy_base[yy_current_state] + yy_c] != yy_current_state ) { yy_current_state = (int) yy_def[yy_current_state]; - if ( yy_current_state >= 443 ) + if ( yy_current_state >= 505 ) yy_c = yy_meta[(unsigned int) yy_c]; } yy_current_state = yy_nxt[yy_base[yy_current_state] + (unsigned int) yy_c]; ++yy_cp; } - while ( yy_base[yy_current_state] != 3182 ); + while ( yy_base[yy_current_state] != 3608 ); yy_find_action: yy_act = yy_accept[yy_current_state]; @@ -1743,131 +1864,131 @@ do_action: /* This label is used only to access EOF actions. */ case 1: YY_RULE_SETUP -#line 98 "lexer.l" +#line 104 "lexer.l" { BEGIN EXPR; return IF; } YY_BREAK case 2: YY_RULE_SETUP -#line 99 "lexer.l" +#line 105 "lexer.l" { BEGIN INITIAL; return THEN; } YY_BREAK case 3: YY_RULE_SETUP -#line 100 "lexer.l" +#line 106 "lexer.l" { BEGIN INITIAL; return ';'; } YY_BREAK case 4: YY_RULE_SETUP -#line 101 "lexer.l" +#line 107 "lexer.l" { return OR; } YY_BREAK case 5: YY_RULE_SETUP -#line 102 "lexer.l" +#line 108 "lexer.l" { return AND; } YY_BREAK case 6: YY_RULE_SETUP -#line 103 "lexer.l" +#line 109 "lexer.l" { return NOT; } YY_BREAK case 7: -#line 105 "lexer.l" +#line 111 "lexer.l" case 8: -#line 106 "lexer.l" +#line 112 "lexer.l" case 9: -#line 107 "lexer.l" +#line 113 "lexer.l" case 10: -#line 108 "lexer.l" +#line 114 "lexer.l" case 11: -#line 109 "lexer.l" +#line 115 "lexer.l" case 12: -#line 110 "lexer.l" +#line 116 "lexer.l" case 13: -#line 111 "lexer.l" +#line 117 "lexer.l" case 14: -#line 112 "lexer.l" +#line 118 "lexer.l" case 15: -#line 113 "lexer.l" +#line 119 "lexer.l" case 16: -#line 114 "lexer.l" +#line 120 "lexer.l" case 17: -#line 115 "lexer.l" +#line 121 "lexer.l" case 18: YY_RULE_SETUP -#line 115 "lexer.l" +#line 121 "lexer.l" { return yytext[0]; } YY_BREAK case 19: YY_RULE_SETUP -#line 116 "lexer.l" +#line 122 "lexer.l" { return CMP_EQ; } YY_BREAK case 20: YY_RULE_SETUP -#line 117 "lexer.l" +#line 123 "lexer.l" { return CMP_LE; } YY_BREAK case 21: YY_RULE_SETUP -#line 118 "lexer.l" +#line 124 "lexer.l" { return CMP_GE; } YY_BREAK case 22: -#line 120 "lexer.l" +#line 126 "lexer.l" case 23: YY_RULE_SETUP -#line 120 "lexer.l" +#line 126 "lexer.l" { return CMP_NE; } YY_BREAK case 24: YY_RULE_SETUP -#line 121 "lexer.l" +#line 127 "lexer.l" { return CMP_LT; } YY_BREAK case 25: YY_RULE_SETUP -#line 122 "lexer.l" +#line 128 "lexer.l" { return CMP_GT; } YY_BREAK case 26: YY_RULE_SETUP -#line 123 "lexer.l" +#line 129 "lexer.l" { return CMP_CONTAINS; } YY_BREAK case 27: YY_RULE_SETUP -#line 124 "lexer.l" +#line 130 "lexer.l" { return CMP_CONTAINSI; } YY_BREAK case 28: YY_RULE_SETUP -#line 125 "lexer.l" +#line 131 "lexer.l" { return CMP_STARTSWITH; } YY_BREAK case 29: YY_RULE_SETUP -#line 126 "lexer.l" +#line 132 "lexer.l" { return CMP_STARTSWITHI; } YY_BREAK case 30: -#line 128 "lexer.l" +#line 134 "lexer.l" case 31: -#line 129 "lexer.l" +#line 135 "lexer.l" case 32: YY_RULE_SETUP -#line 129 "lexer.l" +#line 135 "lexer.l" { yylval.n = strtoll(yytext, NULL, 0); return NUMBER; } YY_BREAK case 33: YY_RULE_SETUP -#line 130 "lexer.l" -{ yylval.s = strdup(yytext); return VAR; } +#line 136 "lexer.l" +{ yylval.s = strdup(yytext+1); return VAR; } YY_BREAK case 34: /* rule 34 can match eol */ YY_RULE_SETUP -#line 131 "lexer.l" +#line 137 "lexer.l" { yytext[yyleng-1] = '\0'; unescapeStr((uchar*)yytext+1, yyleng-2); @@ -1877,7 +1998,7 @@ YY_RULE_SETUP case 35: /* rule 35 can match eol */ YY_RULE_SETUP -#line 136 "lexer.l" +#line 142 "lexer.l" { yytext[yyleng-1] = '\0'; unescapeStr((uchar*)yytext+1, yyleng-2); @@ -1887,18 +2008,18 @@ YY_RULE_SETUP case 36: /* rule 36 can match eol */ YY_RULE_SETUP -#line 141 "lexer.l" +#line 147 "lexer.l" YY_BREAK case 37: YY_RULE_SETUP -#line 142 "lexer.l" +#line 148 "lexer.l" { yylval.estr = es_newStrFromCStr(yytext, yyleng); return FUNC; } YY_BREAK case 38: YY_RULE_SETUP -#line 144 "lexer.l" +#line 150 "lexer.l" { parser_errmsg("invalid character '%s' in expression " "- is there an invalid escape sequence somewhere?", yytext); } @@ -1906,66 +2027,66 @@ YY_RULE_SETUP case 39: /* rule 39 can match eol */ YY_RULE_SETUP -#line 147 "lexer.l" +#line 153 "lexer.l" YY_BREAK case 40: YY_RULE_SETUP -#line 148 "lexer.l" +#line 154 "lexer.l" { parser_errmsg("invalid character '%s' in 'call' statement" "- is there an invalid escape sequence somewhere?", yytext); } YY_BREAK case 41: YY_RULE_SETUP -#line 151 "lexer.l" +#line 157 "lexer.l" { yylval.estr = es_newStrFromCStr(yytext, yyleng); BEGIN INITIAL; return NAME; } YY_BREAK case 42: YY_RULE_SETUP -#line 154 "lexer.l" +#line 160 "lexer.l" { return '&'; } YY_BREAK case 43: YY_RULE_SETUP -#line 155 "lexer.l" +#line 161 "lexer.l" { return '{'; } YY_BREAK case 44: YY_RULE_SETUP -#line 156 "lexer.l" +#line 162 "lexer.l" { return '}'; } YY_BREAK case 45: YY_RULE_SETUP -#line 157 "lexer.l" +#line 163 "lexer.l" { return STOP; } YY_BREAK case 46: YY_RULE_SETUP -#line 158 "lexer.l" +#line 164 "lexer.l" { return ELSE; } YY_BREAK case 47: YY_RULE_SETUP -#line 159 "lexer.l" +#line 165 "lexer.l" { BEGIN INCALL; return CALL; } YY_BREAK case 48: YY_RULE_SETUP -#line 160 "lexer.l" +#line 166 "lexer.l" { BEGIN EXPR; return SET; } YY_BREAK case 49: YY_RULE_SETUP -#line 161 "lexer.l" +#line 167 "lexer.l" { BEGIN EXPR; return UNSET; } YY_BREAK case 50: YY_RULE_SETUP -#line 162 "lexer.l" +#line 168 "lexer.l" { return CONTINUE; } YY_BREAK /* line number support because the "preprocessor" combines lines and so needs @@ -1973,23 +2094,23 @@ YY_RULE_SETUP */ case 51: YY_RULE_SETUP -#line 166 "lexer.l" +#line 172 "lexer.l" { BEGIN LINENO; } YY_BREAK case 52: YY_RULE_SETUP -#line 167 "lexer.l" +#line 173 "lexer.l" { yylineno = atoi(yytext) - 1; } YY_BREAK case 53: YY_RULE_SETUP -#line 168 "lexer.l" +#line 174 "lexer.l" { BEGIN INITIAL; } YY_BREAK case 54: /* rule 54 can match eol */ YY_RULE_SETUP -#line 169 "lexer.l" +#line 175 "lexer.l" YY_BREAK /* $IncludeConfig must be detected as part of CFSYSLINE, because this is @@ -1998,12 +2119,12 @@ YY_RULE_SETUP case 55: /* rule 55 can match eol */ YY_RULE_SETUP -#line 173 "lexer.l" +#line 179 "lexer.l" YY_BREAK case 56: YY_RULE_SETUP -#line 174 "lexer.l" +#line 180 "lexer.l" { if(cnfDoInclude(yytext) != 0) yyterminate(); BEGIN INITIAL; } @@ -2011,121 +2132,149 @@ YY_RULE_SETUP case 57: /* rule 57 can match eol */ YY_RULE_SETUP -#line 177 "lexer.l" -{ yylval.objType = CNFOBJ_GLOBAL; +#line 183 "lexer.l" +{ yylval.objType = CNFOBJ_MAINQ; BEGIN INOBJ; return BEGINOBJ; } YY_BREAK case 58: /* rule 58 can match eol */ YY_RULE_SETUP -#line 179 "lexer.l" -{ yylval.objType = CNFOBJ_TPL; - BEGIN INOBJ; return BEGIN_TPL; } +#line 185 "lexer.l" +{ yylval.objType = CNFOBJ_TIMEZONE; + BEGIN INOBJ; return BEGINOBJ; } YY_BREAK case 59: /* rule 59 can match eol */ YY_RULE_SETUP -#line 181 "lexer.l" -{ yylval.objType = CNFOBJ_RULESET; - BEGIN INOBJ; return BEGIN_RULESET; } +#line 187 "lexer.l" +{ yylval.objType = CNFOBJ_PARSER; + BEGIN INOBJ; return BEGINOBJ; } YY_BREAK case 60: /* rule 60 can match eol */ YY_RULE_SETUP -#line 183 "lexer.l" -{ yylval.objType = CNFOBJ_PROPERTY; - BEGIN INOBJ; return BEGIN_PROPERTY; } +#line 189 "lexer.l" +{ yylval.objType = CNFOBJ_GLOBAL; + BEGIN INOBJ; return BEGINOBJ; } YY_BREAK case 61: /* rule 61 can match eol */ YY_RULE_SETUP -#line 185 "lexer.l" -{ yylval.objType = CNFOBJ_CONSTANT; - BEGIN INOBJ; return BEGIN_CONSTANT; } +#line 191 "lexer.l" +{ yylval.objType = CNFOBJ_TPL; + BEGIN INOBJ; return BEGIN_TPL; } YY_BREAK case 62: /* rule 62 can match eol */ YY_RULE_SETUP -#line 187 "lexer.l" -{ yylval.objType = CNFOBJ_INPUT; - BEGIN INOBJ; return BEGINOBJ; } +#line 193 "lexer.l" +{ yylval.objType = CNFOBJ_RULESET; + BEGIN INOBJ; return BEGIN_RULESET; } YY_BREAK case 63: /* rule 63 can match eol */ YY_RULE_SETUP -#line 189 "lexer.l" -{ yylval.objType = CNFOBJ_MODULE; - BEGIN INOBJ; return BEGINOBJ; } +#line 195 "lexer.l" +{ yylval.objType = CNFOBJ_PROPERTY; + BEGIN INOBJ; return BEGIN_PROPERTY; } YY_BREAK case 64: /* rule 64 can match eol */ YY_RULE_SETUP -#line 191 "lexer.l" -{ BEGIN INOBJ; return BEGIN_ACTION; } +#line 197 "lexer.l" +{ yylval.objType = CNFOBJ_CONSTANT; + BEGIN INOBJ; return BEGIN_CONSTANT; } YY_BREAK case 65: /* rule 65 can match eol */ YY_RULE_SETUP -#line 192 "lexer.l" -{ - yylval.s = strdup(rmLeadingSpace(yytext)); - dbgprintf("lexer: propfilt is '%s'\n", yylval.s); - return PROPFILT; - } +#line 199 "lexer.l" +{ yylval.objType = CNFOBJ_INPUT; + BEGIN INOBJ; return BEGINOBJ; } YY_BREAK case 66: +/* rule 66 can match eol */ YY_RULE_SETUP -#line 197 "lexer.l" -{ yylval.s = strdup(rmLeadingSpace(yytext)); return PRIFILT; } +#line 201 "lexer.l" +{ yylval.objType = CNFOBJ_MODULE; + BEGIN INOBJ; return BEGINOBJ; } YY_BREAK case 67: -#line 199 "lexer.l" +/* rule 67 can match eol */ +YY_RULE_SETUP +#line 203 "lexer.l" +{ yylval.objType = CNFOBJ_LOOKUP_TABLE; + BEGIN INOBJ; return BEGINOBJ; } + YY_BREAK case 68: -#line 200 "lexer.l" +/* rule 68 can match eol */ +YY_RULE_SETUP +#line 205 "lexer.l" +{ BEGIN INOBJ; return BEGIN_ACTION; } + YY_BREAK case 69: /* rule 69 can match eol */ -#line 201 "lexer.l" +YY_RULE_SETUP +#line 206 "lexer.l" +{ + yylval.s = strdup(rmLeadingSpace(yytext)); + dbgprintf("lexer: propfilt is '%s'\n", yylval.s); + return PROPFILT; + } + YY_BREAK case 70: -/* rule 70 can match eol */ -#line 202 "lexer.l" +YY_RULE_SETUP +#line 211 "lexer.l" +{ yylval.s = strdup(rmLeadingSpace(yytext)); return PRIFILT; } + YY_BREAK case 71: -/* rule 71 can match eol */ -#line 203 "lexer.l" +#line 213 "lexer.l" case 72: -/* rule 72 can match eol */ -#line 204 "lexer.l" +#line 214 "lexer.l" case 73: /* rule 73 can match eol */ +#line 215 "lexer.l" +case 74: +/* rule 74 can match eol */ +#line 216 "lexer.l" +case 75: +/* rule 75 can match eol */ +#line 217 "lexer.l" +case 76: +/* rule 76 can match eol */ +#line 218 "lexer.l" +case 77: +/* rule 77 can match eol */ YY_RULE_SETUP -#line 204 "lexer.l" +#line 218 "lexer.l" { yylval.s = yytext; return LEGACY_ACTION; } YY_BREAK -case 74: +case 78: YY_RULE_SETUP -#line 205 "lexer.l" +#line 219 "lexer.l" { BEGIN INITIAL; return ENDOBJ; } YY_BREAK -case 75: +case 79: YY_RULE_SETUP -#line 206 "lexer.l" +#line 220 "lexer.l" { yylval.estr = es_newStrFromCStr(yytext, yyleng); return NAME; } YY_BREAK -case 76: -#line 209 "lexer.l" -case 77: -#line 210 "lexer.l" -case 78: -#line 211 "lexer.l" -case 79: +case 80: +#line 223 "lexer.l" +case 81: +#line 224 "lexer.l" +case 82: +#line 225 "lexer.l" +case 83: YY_RULE_SETUP -#line 211 "lexer.l" +#line 225 "lexer.l" { return(yytext[0]); } YY_BREAK -case 80: -/* rule 80 can match eol */ +case 84: +/* rule 84 can match eol */ YY_RULE_SETUP -#line 212 "lexer.l" +#line 226 "lexer.l" { yytext[yyleng-1] = '\0'; unescapeStr((uchar*)yytext+1, yyleng-2); @@ -2134,59 +2283,59 @@ YY_RULE_SETUP YY_BREAK /*yylval.estr = es_newStrFromBuf(yytext+1, yyleng-2); return VALUE; }*/ -case 81: +case 85: YY_RULE_SETUP -#line 219 "lexer.l" +#line 233 "lexer.l" { preCommentState = YY_START; BEGIN COMMENT; } YY_BREAK -case 82: +case 86: YY_RULE_SETUP -#line 220 "lexer.l" +#line 234 "lexer.l" { preCommentState = YY_START; BEGIN COMMENT; } YY_BREAK -case 83: +case 87: YY_RULE_SETUP -#line 221 "lexer.l" +#line 235 "lexer.l" { preCommentState = YY_START; BEGIN COMMENT; } YY_BREAK -case 84: +case 88: YY_RULE_SETUP -#line 222 "lexer.l" +#line 236 "lexer.l" { BEGIN preCommentState; } YY_BREAK -case 85: -/* rule 85 can match eol */ +case 89: +/* rule 89 can match eol */ YY_RULE_SETUP -#line 223 "lexer.l" +#line 237 "lexer.l" YY_BREAK -case 86: +case 90: *yy_cp = (yy_hold_char); /* undo effects of setting up yytext */ (yy_c_buf_p) = yy_cp -= 1; YY_DO_BEFORE_ACTION; /* set up yytext again */ YY_RULE_SETUP -#line 224 "lexer.l" +#line 238 "lexer.l" /* skip comments in input */ YY_BREAK -case 87: -/* rule 87 can match eol */ +case 91: +/* rule 91 can match eol */ YY_RULE_SETUP -#line 225 "lexer.l" +#line 239 "lexer.l" YY_BREAK -case 88: +case 92: YY_RULE_SETUP -#line 226 "lexer.l" +#line 240 "lexer.l" { parser_errmsg("invalid character '%s' in object definition " "- is there an invalid escape sequence somewhere?", yytext); } YY_BREAK -case 89: +case 93: *yy_cp = (yy_hold_char); /* undo effects of setting up yytext */ (yy_c_buf_p) = yy_cp -= 1; YY_DO_BEFORE_ACTION; /* set up yytext again */ YY_RULE_SETUP -#line 229 "lexer.l" +#line 243 "lexer.l" { /* see comment on $IncludeConfig above */ if(!strncasecmp(yytext, "$includeconfig ", 14)) { yyless(14); @@ -2199,55 +2348,55 @@ YY_RULE_SETUP } } YY_BREAK -case 90: +case 94: *yy_cp = (yy_hold_char); /* undo effects of setting up yytext */ (yy_c_buf_p) = yy_cp -= 1; YY_DO_BEFORE_ACTION; /* set up yytext again */ YY_RULE_SETUP -#line 240 "lexer.l" +#line 254 "lexer.l" { yylval.s = strdup(yytext); return BSD_TAG_SELECTOR; } YY_BREAK -case 91: -/* rule 91 can match eol */ +case 95: +/* rule 95 can match eol */ *yy_cp = (yy_hold_char); /* undo effects of setting up yytext */ (yy_c_buf_p) = yy_cp -= 1; YY_DO_BEFORE_ACTION; /* set up yytext again */ YY_RULE_SETUP -#line 241 "lexer.l" +#line 255 "lexer.l" { yylval.s = strdup(yytext); return BSD_HOST_SELECTOR; } YY_BREAK -case 92: -/* rule 92 can match eol */ +case 96: +/* rule 96 can match eol */ *yy_cp = (yy_hold_char); /* undo effects of setting up yytext */ (yy_c_buf_p) = yy_cp -= 1; YY_DO_BEFORE_ACTION; /* set up yytext again */ YY_RULE_SETUP -#line 242 "lexer.l" +#line 256 "lexer.l" { yylval.s = strdup(yytext); return BSD_HOST_SELECTOR; } YY_BREAK -case 93: +case 97: *yy_cp = (yy_hold_char); /* undo effects of setting up yytext */ (yy_c_buf_p) = yy_cp -= 1; YY_DO_BEFORE_ACTION; /* set up yytext again */ YY_RULE_SETUP -#line 243 "lexer.l" +#line 257 "lexer.l" { yylval.s = strdup(yytext); return BSD_HOST_SELECTOR; } YY_BREAK -case 94: -/* rule 94 can match eol */ +case 98: +/* rule 98 can match eol */ YY_RULE_SETUP -#line 244 "lexer.l" +#line 258 "lexer.l" /* skip comments in input */ YY_BREAK -case 95: -/* rule 95 can match eol */ +case 99: +/* rule 99 can match eol */ YY_RULE_SETUP -#line 245 "lexer.l" +#line 259 "lexer.l" /* drop whitespace */ YY_BREAK -case 96: +case 100: YY_RULE_SETUP -#line 246 "lexer.l" +#line 260 "lexer.l" { parser_errmsg("invalid character '%s' " "- is there an invalid escape sequence somewhere?", yytext); } @@ -2259,15 +2408,15 @@ case YY_STATE_EOF(INCL): case YY_STATE_EOF(LINENO): case YY_STATE_EOF(INCALL): case YY_STATE_EOF(EXPR): -#line 249 "lexer.l" +#line 263 "lexer.l" { if(popfile() != 0) yyterminate(); } YY_BREAK -case 97: +case 101: YY_RULE_SETUP -#line 251 "lexer.l" +#line 265 "lexer.l" YY_FATAL_ERROR( "flex scanner jammed" ); YY_BREAK -#line 2271 "lexer.c" +#line 2420 "lexer.c" case YY_END_OF_BUFFER: { @@ -2558,7 +2707,7 @@ static int yy_get_next_buffer (void) while ( yy_chk[yy_base[yy_current_state] + yy_c] != yy_current_state ) { yy_current_state = (int) yy_def[yy_current_state]; - if ( yy_current_state >= 443 ) + if ( yy_current_state >= 505 ) yy_c = yy_meta[(unsigned int) yy_c]; } yy_current_state = yy_nxt[yy_base[yy_current_state] + (unsigned int) yy_c]; @@ -2586,11 +2735,11 @@ static int yy_get_next_buffer (void) while ( yy_chk[yy_base[yy_current_state] + yy_c] != yy_current_state ) { yy_current_state = (int) yy_def[yy_current_state]; - if ( yy_current_state >= 443 ) + if ( yy_current_state >= 505 ) yy_c = yy_meta[(unsigned int) yy_c]; } yy_current_state = yy_nxt[yy_base[yy_current_state] + (unsigned int) yy_c]; - yy_is_jam = (yy_current_state == 442); + yy_is_jam = (yy_current_state == 504); return yy_is_jam ? 0 : yy_current_state; } @@ -3236,7 +3385,7 @@ void yyfree (void * ptr ) #define YYTABLES_NAME "yytables" -#line 251 "lexer.l" +#line 265 "lexer.l" int diff --git a/grammar/lexer.l b/grammar/lexer.l index 237eb2a..796815c 100644 --- a/grammar/lexer.l +++ b/grammar/lexer.l @@ -9,27 +9,31 @@ * cases. So while we hope that cfsysline support can be dropped some time in * the future, we will probably keep these useful constructs. * - * Copyright 2011-2012 Rainer Gerhards and Adiscon GmbH. + * Copyright 2011-2014 Rainer Gerhards and Adiscon GmbH. * * This file is part of the rsyslog runtime library. * - * The rsyslog runtime library is free software: you can redistribute it and/or modify - * it under the terms of the GNU Lesser General Public License as published by - * the Free Software Foundation, either version 3 of the License, or - * (at your option) any later version. - * - * The rsyslog runtime library is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU Lesser General Public License for more details. - * - * You should have received a copy of the GNU Lesser General Public License - * along with the rsyslog runtime library. If not, see <http://www.gnu.org/licenses/>. - * - * A copy of the GPL can be found in the file "COPYING" in this distribution. - * A copy of the LGPL can be found in the file "COPYING.LESSER" in this distribution. + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * -or- + * see COPYING.ASL20 in the source distribution + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. */ +%{ +#include "config.h" +#include "parserif.h" +extern char *strdup(char*); /* somehow we do not get this from string.h... */ +%} + %option noyywrap nodefault case-insensitive yylineno /*%option noyywrap nodefault case-insensitive */ @@ -88,7 +92,9 @@ extern int yydebug; /* somehow, I need these prototype even though the headers are * included. I guess that's some autotools magic I don't understand... */ +#if !defined(__FreeBSD__) int fileno(FILE *stream); +#endif %} @@ -127,7 +133,7 @@ int fileno(FILE *stream); <EXPR>0[0-7]+ | /* octal number */ <EXPR>0x[0-7a-f] | /* hex number, following rule is dec; strtoll handles all! */ <EXPR>([1-9][0-9]*|0) { yylval.n = strtoll(yytext, NULL, 0); return NUMBER; } -<EXPR>\$[$!]{0,1}[a-z][!a-z0-9\-_\.]* { yylval.s = strdup(yytext); return VAR; } +<EXPR>\$[$!./]{0,1}[@a-z][!@a-z0-9\-_\.]* { yylval.s = strdup(yytext+1); return VAR; } <EXPR>\'([^'\\]|\\['"\\$bntr]|\\x[0-9a-f][0-9a-f]|\\[0-7][0-7][0-7])*\' { yytext[yyleng-1] = '\0'; unescapeStr((uchar*)yytext+1, yyleng-2); @@ -174,6 +180,12 @@ int fileno(FILE *stream); <INCL>[^ \t\n]+ { if(cnfDoInclude(yytext) != 0) yyterminate(); BEGIN INITIAL; } +"main_queue"[ \n\t]*"(" { yylval.objType = CNFOBJ_MAINQ; + BEGIN INOBJ; return BEGINOBJ; } +"timezone"[ \n\t]*"(" { yylval.objType = CNFOBJ_TIMEZONE; + BEGIN INOBJ; return BEGINOBJ; } +"parser"[ \n\t]*"(" { yylval.objType = CNFOBJ_PARSER; + BEGIN INOBJ; return BEGINOBJ; } "global"[ \n\t]*"(" { yylval.objType = CNFOBJ_GLOBAL; BEGIN INOBJ; return BEGINOBJ; } "template"[ \n\t]*"(" { yylval.objType = CNFOBJ_TPL; @@ -188,6 +200,8 @@ int fileno(FILE *stream); BEGIN INOBJ; return BEGINOBJ; } "module"[ \n\t]*"(" { yylval.objType = CNFOBJ_MODULE; BEGIN INOBJ; return BEGINOBJ; } +"lookup_table"[ \n\t]*"(" { yylval.objType = CNFOBJ_LOOKUP_TABLE; + BEGIN INOBJ; return BEGINOBJ; } "action"[ \n\t]*"(" { BEGIN INOBJ; return BEGIN_ACTION; } ^[ \t]*:\$?[a-z\-]+[ ]*,[ ]*!?[a-z]+[ ]*,[ ]*\"(\\\"|[^\"])*\" { yylval.s = strdup(rmLeadingSpace(yytext)); diff --git a/grammar/parserif.h b/grammar/parserif.h index aa271ec..b66023d 100644 --- a/grammar/parserif.h +++ b/grammar/parserif.h @@ -1,3 +1,21 @@ +/* rsyslog parser interface. + * + * Copyright 2011-2014 Rainer Gerhards + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * -or- + * see COPYING.ASL20 in the source distribution + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ #ifndef PARSERIF_H_DEFINED #define PARSERIF_H_DEFINED #include "rainerscript.h" @@ -6,6 +24,7 @@ int yyparse(); char *cnfcurrfn; void dbgprintf(char *fmt, ...) __attribute__((format(printf, 1, 2))); void parser_errmsg(char *fmt, ...) __attribute__((format(printf, 1, 2))); +void parser_warnmsg(char *fmt, ...) __attribute__((format(printf, 1, 2))); void tellLexEndParsing(void); extern int yydebug; extern int yylineno; @@ -19,5 +38,4 @@ void cnfDoScript(struct cnfstmt *script); void cnfDoCfsysline(char *ln); void cnfDoBSDTag(char *ln); void cnfDoBSDHost(char *ln); -es_str_t *cnfGetVar(char *name, void *usrptr); -#endif + #endif diff --git a/grammar/rainerscript.c b/grammar/rainerscript.c index 89cf946..ccfa0e2 100644 --- a/grammar/rainerscript.c +++ b/grammar/rainerscript.c @@ -2,7 +2,7 @@ * * Module begun 2011-07-01 by Rainer Gerhards * - * Copyright 2011-2013 Rainer Gerhards and Adiscon GmbH. + * Copyright 2011-2014 Rainer Gerhards and Adiscon GmbH. * * This file is part of the rsyslog runtime library. * @@ -38,6 +38,7 @@ #include "rainerscript.h" #include "conf.h" #include "parserif.h" +#include "parse.h" #include "rsconf.h" #include "grammar.h" #include "queue.h" @@ -46,6 +47,9 @@ #include "obj.h" #include "modules.h" #include "ruleset.h" +#include "msg.h" +#include "wti.h" +#include "unicode-helper.h" DEFobjCurrIf(obj) DEFobjCurrIf(regexp) @@ -62,8 +66,8 @@ struct cnffunc * cnffuncNew_prifilt(int fac); * NOTE: This function MUST be updated if new tokens are defined in the * grammar. */ -char * -tokenToString(int token) +const char * +tokenToString(const int token) { char *tokstr; static char tokbuf[512]; @@ -114,8 +118,8 @@ tokenToString(int token) } -char* -getFIOPName(unsigned iFIOP) +const char* +getFIOPName(const unsigned iFIOP) { char *pRet; switch(iFIOP) { @@ -144,8 +148,106 @@ getFIOPName(unsigned iFIOP) return pRet; } + +/* This function takes the filter part of a property + * based filter and decodes it. It processes the line up to the beginning + * of the action part. + */ +static rsRetVal +DecodePropFilter(uchar *pline, struct cnfstmt *stmt) +{ + rsParsObj *pPars = NULL; + cstr_t *pCSCompOp = NULL; + cstr_t *pCSPropName = NULL; + int iOffset; /* for compare operations */ + DEFiRet; + + ASSERT(pline != NULL); + + DBGPRINTF("Decoding property-based filter '%s'\n", pline); + + /* create parser object starting with line string without leading colon */ + if((iRet = rsParsConstructFromSz(&pPars, pline+1)) != RS_RET_OK) { + parser_errmsg("error %d constructing parser object", iRet); + ABORT_FINALIZE(iRet); + } + + /* read property */ + iRet = parsDelimCStr(pPars, &pCSPropName, ',', 1, 1, 1); + if(iRet != RS_RET_OK) { + parser_errmsg("error %d parsing filter property", iRet); + rsParsDestruct(pPars); + ABORT_FINALIZE(iRet); + } + CHKiRet(msgPropDescrFill(&stmt->d.s_propfilt.prop, cstrGetSzStrNoNULL(pCSPropName), + cstrLen(pCSPropName))); + + /* read operation */ + iRet = parsDelimCStr(pPars, &pCSCompOp, ',', 1, 1, 1); + if(iRet != RS_RET_OK) { + parser_errmsg("error %d compare operation property - ignoring selector", iRet); + rsParsDestruct(pPars); + ABORT_FINALIZE(iRet); + } + + /* we now first check if the condition is to be negated. To do so, we first + * must make sure we have at least one char in the param and then check the + * first one. + * rgerhards, 2005-09-26 + */ + if(rsCStrLen(pCSCompOp) > 0) { + if(*rsCStrGetBufBeg(pCSCompOp) == '!') { + stmt->d.s_propfilt.isNegated = 1; + iOffset = 1; /* ignore '!' */ + } else { + stmt->d.s_propfilt.isNegated = 0; + iOffset = 0; + } + } else { + stmt->d.s_propfilt.isNegated = 0; + iOffset = 0; + } + + if(!rsCStrOffsetSzStrCmp(pCSCompOp, iOffset, (uchar*) "contains", 8)) { + stmt->d.s_propfilt.operation = FIOP_CONTAINS; + } else if(!rsCStrOffsetSzStrCmp(pCSCompOp, iOffset, (uchar*) "isequal", 7)) { + stmt->d.s_propfilt.operation = FIOP_ISEQUAL; + } else if(!rsCStrOffsetSzStrCmp(pCSCompOp, iOffset, (uchar*) "isempty", 7)) { + stmt->d.s_propfilt.operation = FIOP_ISEMPTY; + } else if(!rsCStrOffsetSzStrCmp(pCSCompOp, iOffset, (uchar*) "startswith", 10)) { + stmt->d.s_propfilt.operation = FIOP_STARTSWITH; + } else if(!rsCStrOffsetSzStrCmp(pCSCompOp, iOffset, (unsigned char*) "regex", 5)) { + stmt->d.s_propfilt.operation = FIOP_REGEX; + } else if(!rsCStrOffsetSzStrCmp(pCSCompOp, iOffset, (unsigned char*) "ereregex", 8)) { + stmt->d.s_propfilt.operation = FIOP_EREREGEX; + } else { + parser_errmsg("error: invalid compare operation '%s'", + (char*) rsCStrGetSzStrNoNULL(pCSCompOp)); + ABORT_FINALIZE(RS_RET_ERR); + } + + if(stmt->d.s_propfilt.operation != FIOP_ISEMPTY) { + /* read compare value */ + iRet = parsQuotedCStr(pPars, &stmt->d.s_propfilt.pCSCompValue); + if(iRet != RS_RET_OK) { + parser_errmsg("error %d compare value property", iRet); + rsParsDestruct(pPars); + ABORT_FINALIZE(iRet); + } + } + +finalize_it: + if(pPars != NULL) + rsParsDestruct(pPars); + if(pCSCompOp != NULL) + rsCStrDestruct(&pCSCompOp); + if(pCSPropName != NULL) + cstrDestruct(&pCSPropName); + RETiRet; +} + static void -prifiltInvert(struct funcData_prifilt *prifilt) +prifiltInvert(struct funcData_prifilt *__restrict__ const prifilt) { int i; for(i = 0 ; i < LOG_NFACILITIES+1 ; ++i) { @@ -187,7 +289,7 @@ prifiltSetSeverity(struct funcData_prifilt *prifilt, int sev, int mode) * NOTE: fac MUST be in the range 0..24 (not multiplied by 8)! */ static void -prifiltSetFacility(struct funcData_prifilt *prifilt, int fac, int mode) +prifiltSetFacility(struct funcData_prifilt *__restrict__ const prifilt, const int fac, const int mode) { int i; @@ -224,7 +326,9 @@ prifiltSetFacility(struct funcData_prifilt *prifilt, int fac, int mode) * used to keep things simple). */ static void -prifiltCombine(struct funcData_prifilt *prifilt, struct funcData_prifilt *prifilt2, int mode) +prifiltCombine(struct funcData_prifilt *__restrict__ const prifilt, + struct funcData_prifilt *__restrict__ const prifilt2, + const int mode) { int i; for(i = 0 ; i < LOG_NFACILITIES+1 ; ++i) { @@ -237,7 +341,7 @@ prifiltCombine(struct funcData_prifilt *prifilt, struct funcData_prifilt *prifil void -readConfFile(FILE *fp, es_str_t **str) +readConfFile(FILE * const fp, es_str_t **str) { char ln[10240]; char buf[512]; @@ -854,7 +958,7 @@ nvlstGetParam(struct nvlst *valnode, struct cnfparamdescr *param, r = doGetInt(valnode, param, val); break; case eCmdHdlrNonNegInt: - r = doGetPositiveInt(valnode, param, val); + r = doGetNonNegInt(valnode, param, val); break; case eCmdHdlrPositiveInt: r = doGetPositiveInt(valnode, param, val); @@ -938,8 +1042,14 @@ nvlstGetParams(struct nvlst *lst, struct cnfparamblk *params, for(i = 0 ; i < params->nParams ; ++i) { param = params->descr + i; - if((valnode = nvlstFindNameCStr(lst, param->name)) == NULL) + if((valnode = nvlstFindNameCStr(lst, param->name)) == NULL) { + if(param->flags & CNFPARAM_REQUIRED) { + parser_errmsg("parameter '%s' required but not specified - " + "fix config", param->name); + bInError = 1; + } continue; + } if(vals[i].bUsed) { parser_errmsg("parameter '%s' specified more than once - " "one instance is ignored. Fix config", param->name); @@ -950,7 +1060,6 @@ nvlstGetParams(struct nvlst *lst, struct cnfparamblk *params, } } - if(bInError) { if(bValsWasNULL) cnfparamvalsDestruct(vals, params); @@ -1023,7 +1132,7 @@ cnfobjNew(enum cnfobjType objType, struct nvlst *lst) { struct cnfobj *o; - if((o = malloc(sizeof(struct nvlst))) != NULL) { + if((o = malloc(sizeof(struct cnfobj))) != NULL) { nvlstChkDupes(lst); o->objType = objType; o->nvlst = lst; @@ -1086,7 +1195,11 @@ var2Number(struct var *r, int *bSuccess) n = es_str2num(r->d.estr, bSuccess); } else { if(r->datatype == 'J') { +#ifdef HAVE_JSON_OBJECT_NEW_INT64 + n = (r->d.json == NULL) ? 0 : json_object_get_int64(r->d.json); +#else /* HAVE_JSON_OBJECT_NEW_INT64 */ n = (r->d.json == NULL) ? 0 : json_object_get_int(r->d.json); +#endif /* HAVE_JSON_OBJECT_NEW_INT64 */ } else { n = r->d.n; } @@ -1098,8 +1211,8 @@ var2Number(struct var *r, int *bSuccess) /* ensure that retval is a string */ -static inline es_str_t * -var2String(struct var *r, int *bMustFree) +static es_str_t * +var2String(struct var *__restrict__ const r, int *__restrict__ const bMustFree) { es_str_t *estr; char *cstr; @@ -1125,7 +1238,7 @@ var2String(struct var *r, int *bMustFree) } static uchar* -var2CString(struct var *r, int *bMustFree) +var2CString(struct var *__restrict__ const r, int *__restrict__ const bMustFree) { uchar *cstr; es_str_t *estr; @@ -1137,8 +1250,25 @@ var2CString(struct var *r, int *bMustFree) return cstr; } +/* frees struct var members, but not the struct itself. This is because + * it usually is allocated on the stack. Callers why dynamically allocate + * struct var need to free the struct themselfes! + */ +static void +varFreeMembers(struct var *r) +{ + /* Note: we do NOT need to free JSON objects, as we use + * json_object_object_get() to obtain the values, which does not + * increment the reference count. So json_object_put() [free] is + * neither required nor permitted (would free the original object!). + * So for the time being the string data type is the only one that + * we currently need to free. + */ + if(r->datatype == 'S') es_deleteStr(r->d.estr); +} + static rsRetVal -doExtractFieldByChar(uchar *str, uchar delim, int matchnbr, uchar **resstr) +doExtractFieldByChar(uchar *str, uchar delim, const int matchnbr, uchar **resstr) { int iCurrFld; int iLen; @@ -1171,7 +1301,7 @@ doExtractFieldByChar(uchar *str, uchar delim, int matchnbr, uchar **resstr) * step back a little not to copy it as part of the field. */ /* we got our end pointer, now do the copy */ iLen = pFldEnd - pFld + 1; /* the +1 is for an actual char, NOT \0! */ - CHKmalloc(pBuf = MALLOC((iLen + 1) * sizeof(char))); + CHKmalloc(pBuf = MALLOC((iLen + 1) * sizeof(uchar))); /* now copy */ memcpy(pBuf, pFld, iLen); pBuf[iLen] = '\0'; /* terminate it */ @@ -1185,7 +1315,7 @@ finalize_it: static rsRetVal -doExtractFieldByStr(uchar *str, char *delim, rs_size_t lenDelim, int matchnbr, uchar **resstr) +doExtractFieldByStr(uchar *str, char *delim, const rs_size_t lenDelim, const int matchnbr, uchar **resstr) { int iCurrFld; int iLen; @@ -1194,6 +1324,9 @@ doExtractFieldByStr(uchar *str, char *delim, rs_size_t lenDelim, int matchnbr, u uchar *pFldEnd; DEFiRet; + if (str == NULL || delim == NULL) + ABORT_FINALIZE(RS_RET_FIELD_NOT_FOUND); + /* first, skip to the field in question */ iCurrFld = 1; pFld = str; @@ -1216,7 +1349,7 @@ doExtractFieldByStr(uchar *str, char *delim, rs_size_t lenDelim, int matchnbr, u iLen = pFldEnd - pFld; } /* we got our end pointer, now do the copy */ - CHKmalloc(pBuf = MALLOC((iLen + 1) * sizeof(char))); + CHKmalloc(pBuf = MALLOC((iLen + 1) * sizeof(uchar))); /* now copy */ memcpy(pBuf, pFld, iLen); pBuf[iLen] = '\0'; /* terminate it */ @@ -1235,7 +1368,7 @@ doFunc_re_extract(struct cnffunc *func, struct var *ret, void* usrptr) short matchnbr; regmatch_t pmatch[50]; int bMustFree; - es_str_t *estr; + es_str_t *estr = NULL; /* init just to keep compiler happy */ char *str; struct var r[CNFFUNC_MAX_ARGS]; int iLenBuf; @@ -1254,8 +1387,8 @@ doFunc_re_extract(struct cnffunc *func, struct var *ret, void* usrptr) str = (char*) var2CString(&r[0], &bMustFree); matchnbr = (short) var2Number(&r[2], NULL); submatchnbr = (size_t) var2Number(&r[3], NULL); - if(submatchnbr > sizeof(pmatch)/sizeof(regmatch_t)) { - DBGPRINTF("re_extract() submatch %d is too large\n", submatchnbr); + if(submatchnbr >= sizeof(pmatch)/sizeof(regmatch_t)) { + DBGPRINTF("re_extract() submatch %zd is too large\n", submatchnbr); bHadNoMatch = 1; goto finalize_it; } @@ -1302,15 +1435,19 @@ doFunc_re_extract(struct cnffunc *func, struct var *ret, void* usrptr) iLenBuf); } - if(bMustFree) free(str); - if(r[0].datatype == 'S') es_deleteStr(r[0].d.estr); - if(r[2].datatype == 'S') es_deleteStr(r[2].d.estr); - if(r[3].datatype == 'S') es_deleteStr(r[3].d.estr); finalize_it: + if(bMustFree) free(str); + varFreeMembers(&r[0]); + varFreeMembers(&r[2]); + varFreeMembers(&r[3]); + if(bHadNoMatch) { cnfexprEval(func->expr[4], &r[4], usrptr); estr = var2String(&r[4], &bMustFree); - if(r[4].datatype == 'S') es_deleteStr(r[4].d.estr); + /* Note that we do NOT free the string that was returned/created + * for r[4]. We pass it to the caller, which in turn frees it. + * This saves us doing one unnecessary memory alloc & write. + */ } ret->datatype = 'S'; ret->d.estr = estr; @@ -1318,11 +1455,39 @@ finalize_it: } +/* note that we do not need to evaluate any parameters, as the template pointer + * is set during initialization(). + * TODO: think if we can keep our buffer; but that may not be trival thinking about + * multiple threads. + */ +static void +doFunc_exec_template(struct cnffunc *__restrict__ const func, + struct var *__restrict__ const ret, + msg_t *const pMsg) +{ + rsRetVal localRet; + actWrkrIParams_t iparam; + + wtiInitIParam(&iparam); + localRet = tplToString(func->funcdata, pMsg, &iparam, NULL); + if(localRet == RS_RET_OK) { + ret->d.estr = es_newStrFromCStr((char*)iparam.param, iparam.lenStr); + } else { + ret->d.estr = es_newStrFromCStr("", 0); + } + ret->datatype = 'S'; + free(iparam.param); + + return; +} + + /* Perform a function call. This has been moved out of cnfExprEval in order * to keep the code small and easier to maintain. */ static inline void -doFuncCall(struct cnffunc *func, struct var *ret, void* usrptr) +doFuncCall(struct cnffunc *__restrict__ const func, struct var *__restrict__ const ret, + void *__restrict__ const usrptr) { char *fname; char *envvar; @@ -1350,6 +1515,7 @@ doFuncCall(struct cnffunc *func, struct var *ret, void* usrptr) estr = var2String(&r[0], &bMustFree); ret->d.n = es_strlen(estr); if(bMustFree) es_deleteStr(estr); + if(r[0].datatype == 'S') es_deleteStr(r[0].d.estr); } ret->datatype = 'N'; break; @@ -1370,7 +1536,7 @@ doFuncCall(struct cnffunc *func, struct var *ret, void* usrptr) } ret->datatype = 'S'; if(bMustFree) es_deleteStr(estr); - if(r[0].datatype == 'S') es_deleteStr(r[0].d.estr); + varFreeMembers(&r[0]); free(str); break; case CNFFUNC_TOLOWER: @@ -1381,7 +1547,7 @@ doFuncCall(struct cnffunc *func, struct var *ret, void* usrptr) es_tolower(estr); ret->datatype = 'S'; ret->d.estr = estr; - if(r[0].datatype == 'S') es_deleteStr(r[0].d.estr); + varFreeMembers(&r[0]); break; case CNFFUNC_CSTR: cnfexprEval(func->expr[0], &r[0], usrptr); @@ -1390,7 +1556,7 @@ doFuncCall(struct cnffunc *func, struct var *ret, void* usrptr) estr = es_strdup(estr); ret->datatype = 'S'; ret->d.estr = estr; - if(r[0].datatype == 'S') es_deleteStr(r[0].d.estr); + varFreeMembers(&r[0]); break; case CNFFUNC_CNUM: if(func->expr[0]->nodetype == 'N') { @@ -1401,7 +1567,7 @@ doFuncCall(struct cnffunc *func, struct var *ret, void* usrptr) } else { cnfexprEval(func->expr[0], &r[0], usrptr); ret->d.n = var2Number(&r[0], NULL); - if(r[0].datatype == 'S') es_deleteStr(r[0].d.estr); + varFreeMembers(&r[0]); } ret->datatype = 'N'; break; @@ -1419,11 +1585,14 @@ doFuncCall(struct cnffunc *func, struct var *ret, void* usrptr) } ret->datatype = 'N'; if(bMustFree) free(str); - if(r[0].datatype == 'S') es_deleteStr(r[0].d.estr); + varFreeMembers(&r[0]); break; case CNFFUNC_RE_EXTRACT: doFunc_re_extract(func, ret, usrptr); break; + case CNFFUNC_EXEC_TEMPLATE: + doFunc_exec_template(func, ret, (msg_t*) usrptr); + break; case CNFFUNC_FIELD: cnfexprEval(func->expr[0], &r[0], usrptr); cnfexprEval(func->expr[1], &r[1], usrptr); @@ -1452,9 +1621,9 @@ doFuncCall(struct cnffunc *func, struct var *ret, void* usrptr) } ret->datatype = 'S'; if(bMustFree) free(str); - if(r[0].datatype == 'S') es_deleteStr(r[0].d.estr); - if(r[1].datatype == 'S') es_deleteStr(r[1].d.estr); - if(r[2].datatype == 'S') es_deleteStr(r[2].d.estr); + varFreeMembers(&r[0]); + varFreeMembers(&r[1]); + varFreeMembers(&r[2]); break; case CNFFUNC_PRIFILT: pPrifilt = (struct funcData_prifilt*) func->funcdata; @@ -1466,6 +1635,19 @@ doFuncCall(struct cnffunc *func, struct var *ret, void* usrptr) ret->d.n = 1; ret->datatype = 'N'; break; + case CNFFUNC_LOOKUP: +dbgprintf("DDDD: executing lookup\n"); + ret->datatype = 'S'; + if(func->funcdata == NULL) { + ret->d.estr = es_newStrFromCStr("TABLE-NOT-FOUND", sizeof("TABLE-NOT-FOUND")-1); + break; + } + cnfexprEval(func->expr[1], &r[1], usrptr); + str = (char*) var2CString(&r[1], &bMustFree); + ret->d.estr = lookupKey_estr(func->funcdata, (uchar*)str); + if(bMustFree) free(str); + if(r[1].datatype == 'S') es_deleteStr(r[1].d.estr); + break; default: if(Debug) { fname = es_str2cstr(func->fname, NULL); @@ -1479,22 +1661,31 @@ doFuncCall(struct cnffunc *func, struct var *ret, void* usrptr) } static inline void -evalVar(struct cnfvar *var, void *usrptr, struct var *ret) +evalVar(struct cnfvar *__restrict__ const var, void *__restrict__ const usrptr, + struct var *__restrict__ const ret) { + rs_size_t propLen; + uchar *pszProp = NULL; + unsigned short bMustBeFreed = 0; rsRetVal localRet; - es_str_t *estr; struct json_object *json; - if(var->name[0] == '$' && var->name[1] == '!') { - /* TODO: unify string libs */ - estr = es_newStrFromBuf(var->name+1, strlen(var->name)-1); - localRet = msgGetCEEPropJSON((msg_t*)usrptr, estr, &json); - es_deleteStr(estr); + if(var->prop.id == PROP_CEE || + var->prop.id == PROP_LOCAL_VAR || + var->prop.id == PROP_GLOBAL_VAR ) { + localRet = msgGetJSONPropJSON((msg_t*)usrptr, &var->prop, &json); ret->datatype = 'J'; ret->d.json = (localRet == RS_RET_OK) ? json : NULL; + + DBGPRINTF("rainerscript: var %d:%s: '%s'\n", var->prop.id, var->prop.name, + (ret->d.json == NULL) ? "" : json_object_get_string(ret->d.json)); } else { ret->datatype = 'S'; - ret->d.estr = cnfGetVar(var->name, usrptr); + pszProp = (uchar*) MsgGetProp((msg_t*)usrptr, NULL, &var->prop, &propLen, &bMustBeFreed, NULL); + ret->d.estr = es_newStrFromCStr((char*)pszProp, propLen); + DBGPRINTF("rainerscript: var %d: '%s'\n", var->prop.id, pszProp); + if(bMustBeFreed) + free(pszProp); } } @@ -1507,7 +1698,8 @@ evalVar(struct cnfvar *var, void *usrptr, struct var *ret) * and it was generally 5 to 10 times SLOWER than what we do here... */ static int -evalStrArrayCmp(es_str_t *estr_l, struct cnfarray* ar, int cmpop) +evalStrArrayCmp(es_str_t *const estr_l, struct cnfarray *__restrict__ const ar, + const int cmpop) { int i; int r = 0; @@ -1540,8 +1732,8 @@ evalStrArrayCmp(es_str_t *estr_l, struct cnfarray* ar, int cmpop) } #define FREE_BOTH_RET \ - if(r.datatype == 'S') es_deleteStr(r.d.estr); \ - if(l.datatype == 'S') es_deleteStr(l.d.estr) + varFreeMembers(&r); \ + varFreeMembers(&l) #define COMP_NUM_BINOP(x) \ cnfexprEval(expr->l, &l, usrptr); \ @@ -1550,6 +1742,17 @@ evalStrArrayCmp(es_str_t *estr_l, struct cnfarray* ar, int cmpop) ret->d.n = var2Number(&l, &convok_l) x var2Number(&r, &convok_r); \ FREE_BOTH_RET +#define COMP_NUM_BINOP_DIV(x) \ + cnfexprEval(expr->l, &l, usrptr); \ + cnfexprEval(expr->r, &r, usrptr); \ + ret->datatype = 'N'; \ + if((ret->d.n = var2Number(&r, &convok_r)) == 0) { \ + /* division by zero */ \ + } else { \ + ret->d.n = var2Number(&l, &convok_l) x ret->d.n; \ + } \ + FREE_BOTH_RET + /* NOTE: array as right-hand argument MUST be handled by user */ #define PREP_TWO_STRINGS \ cnfexprEval(expr->l, &l, usrptr); \ @@ -1568,9 +1771,9 @@ evalStrArrayCmp(es_str_t *estr_l, struct cnfarray* ar, int cmpop) #define FREE_TWO_STRINGS \ if(bMustFree) es_deleteStr(estr_r); \ - if(expr->r->nodetype != 'S' && expr->r->nodetype != 'A' && r.datatype == 'S') es_deleteStr(r.d.estr); \ + if(expr->r->nodetype != 'S' && expr->r->nodetype != 'A') varFreeMembers(&r); \ if(bMustFree2) es_deleteStr(estr_l); \ - if(l.datatype == 'S') es_deleteStr(l.d.estr) + varFreeMembers(&l) /* evaluate an expression. * Note that we try to avoid malloc whenever possible (because of @@ -1583,15 +1786,16 @@ evalStrArrayCmp(es_str_t *estr_l, struct cnfarray* ar, int cmpop) * simply is no case where full evaluation would make any sense at all. */ void -cnfexprEval(struct cnfexpr *expr, struct var *ret, void* usrptr) +cnfexprEval(const struct cnfexpr *__restrict__ const expr, struct var *__restrict__ const ret, + void *__restrict__ const usrptr) { struct var r, l; /* memory for subexpression results */ - es_str_t *estr_r, *estr_l; + es_str_t *__restrict__ estr_r, *__restrict__ estr_l; int convok_r, convok_l; int bMustFree, bMustFree2; long long n_r, n_l; - dbgprintf("eval expr %p, type '%s'\n", expr, tokenToString(expr->nodetype)); + DBGPRINTF("eval expr %p, type '%s'\n", expr, tokenToString(expr->nodetype)); switch(expr->nodetype) { /* note: comparison operations are extremely similar. The code can be copyied, only * places flagged with "CMP" need to be changed. @@ -1621,7 +1825,7 @@ cnfexprEval(struct cnfexpr *expr, struct var *ret, void* usrptr) if(bMustFree) es_deleteStr(estr_r); } } - if(r.datatype == 'S') es_deleteStr(r.d.estr); + varFreeMembers(&r); } } else if(l.datatype == 'J') { estr_l = var2String(&l, &bMustFree); @@ -1638,12 +1842,12 @@ cnfexprEval(struct cnfexpr *expr, struct var *ret, void* usrptr) if(convok_l) { ret->d.n = (n_l == r.d.n); /*CMP*/ } else { - estr_r = var2String(&r, &bMustFree); + estr_r = var2String(&r, &bMustFree2); ret->d.n = !es_strcmp(estr_l, estr_r); /*CMP*/ - if(bMustFree) es_deleteStr(estr_r); + if(bMustFree2) es_deleteStr(estr_r); } } - if(r.datatype == 'S') es_deleteStr(r.d.estr); + varFreeMembers(&r); } if(bMustFree) es_deleteStr(estr_l); } else { @@ -1660,9 +1864,9 @@ cnfexprEval(struct cnfexpr *expr, struct var *ret, void* usrptr) } else { ret->d.n = (l.d.n == r.d.n); /*CMP*/ } - if(r.datatype == 'S') es_deleteStr(r.d.estr); + varFreeMembers(&r); } - if(l.datatype == 'S') es_deleteStr(l.d.estr); + varFreeMembers(&l); break; case CMP_NE: cnfexprEval(expr->l, &l, usrptr); @@ -1687,6 +1891,21 @@ cnfexprEval(struct cnfexpr *expr, struct var *ret, void* usrptr) } } } + } else if(l.datatype == 'J') { + estr_l = var2String(&l, &bMustFree); + if(r.datatype == 'S') { + ret->d.n = es_strcmp(estr_l, r.d.estr); /*CMP*/ + } else { + n_l = var2Number(&l, &convok_l); + if(convok_l) { + ret->d.n = (n_l != r.d.n); /*CMP*/ + } else { + estr_r = var2String(&r, &bMustFree2); + ret->d.n = es_strcmp(estr_l, estr_r); /*CMP*/ + if(bMustFree2) es_deleteStr(estr_r); + } + } + if(bMustFree) es_deleteStr(estr_l); } else { if(r.datatype == 'S') { n_r = var2Number(&r, &convok_r); @@ -1720,6 +1939,21 @@ cnfexprEval(struct cnfexpr *expr, struct var *ret, void* usrptr) if(bMustFree) es_deleteStr(estr_r); } } + } else if(l.datatype == 'J') { + estr_l = var2String(&l, &bMustFree); + if(r.datatype == 'S') { + ret->d.n = es_strcmp(estr_l, r.d.estr) <= 0; /*CMP*/ + } else { + n_l = var2Number(&l, &convok_l); + if(convok_l) { + ret->d.n = (n_l <= r.d.n); /*CMP*/ + } else { + estr_r = var2String(&r, &bMustFree2); + ret->d.n = es_strcmp(estr_l, estr_r) <= 0; /*CMP*/ + if(bMustFree2) es_deleteStr(estr_r); + } + } + if(bMustFree) es_deleteStr(estr_l); } else { if(r.datatype == 'S') { n_r = var2Number(&r, &convok_r); @@ -1753,6 +1987,21 @@ cnfexprEval(struct cnfexpr *expr, struct var *ret, void* usrptr) if(bMustFree) es_deleteStr(estr_r); } } + } else if(l.datatype == 'J') { + estr_l = var2String(&l, &bMustFree); + if(r.datatype == 'S') { + ret->d.n = es_strcmp(estr_l, r.d.estr) >= 0; /*CMP*/ + } else { + n_l = var2Number(&l, &convok_l); + if(convok_l) { + ret->d.n = (n_l >= r.d.n); /*CMP*/ + } else { + estr_r = var2String(&r, &bMustFree2); + ret->d.n = es_strcmp(estr_l, estr_r) >= 0; /*CMP*/ + if(bMustFree2) es_deleteStr(estr_r); + } + } + if(bMustFree) es_deleteStr(estr_l); } else { if(r.datatype == 'S') { n_r = var2Number(&r, &convok_r); @@ -1786,6 +2035,21 @@ cnfexprEval(struct cnfexpr *expr, struct var *ret, void* usrptr) if(bMustFree) es_deleteStr(estr_r); } } + } else if(l.datatype == 'J') { + estr_l = var2String(&l, &bMustFree); + if(r.datatype == 'S') { + ret->d.n = es_strcmp(estr_l, r.d.estr) < 0; /*CMP*/ + } else { + n_l = var2Number(&l, &convok_l); + if(convok_l) { + ret->d.n = (n_l < r.d.n); /*CMP*/ + } else { + estr_r = var2String(&r, &bMustFree2); + ret->d.n = es_strcmp(estr_l, estr_r) < 0; /*CMP*/ + if(bMustFree2) es_deleteStr(estr_r); + } + } + if(bMustFree) es_deleteStr(estr_l); } else { if(r.datatype == 'S') { n_r = var2Number(&r, &convok_r); @@ -1819,6 +2083,21 @@ cnfexprEval(struct cnfexpr *expr, struct var *ret, void* usrptr) if(bMustFree) es_deleteStr(estr_r); } } + } else if(l.datatype == 'J') { + estr_l = var2String(&l, &bMustFree); + if(r.datatype == 'S') { + ret->d.n = es_strcmp(estr_l, r.d.estr) > 0; /*CMP*/ + } else { + n_l = var2Number(&l, &convok_l); + if(convok_l) { + ret->d.n = (n_l > r.d.n); /*CMP*/ + } else { + estr_r = var2String(&r, &bMustFree2); + ret->d.n = es_strcmp(estr_l, estr_r) > 0; /*CMP*/ + if(bMustFree2) es_deleteStr(estr_r); + } + } + if(bMustFree) es_deleteStr(estr_l); } else { if(r.datatype == 'S') { n_r = var2Number(&r, &convok_r); @@ -1890,9 +2169,9 @@ cnfexprEval(struct cnfexpr *expr, struct var *ret, void* usrptr) ret->d.n = 1ll; else ret->d.n = 0ll; - if(r.datatype == 'S') es_deleteStr(r.d.estr); + varFreeMembers(&r); } - if(l.datatype == 'S') es_deleteStr(l.d.estr); + varFreeMembers(&l); break; case AND: cnfexprEval(expr->l, &l, usrptr); @@ -1903,17 +2182,17 @@ cnfexprEval(struct cnfexpr *expr, struct var *ret, void* usrptr) ret->d.n = 1ll; else ret->d.n = 0ll; - if(r.datatype == 'S') es_deleteStr(r.d.estr); + varFreeMembers(&r); } else { ret->d.n = 0ll; } - if(l.datatype == 'S') es_deleteStr(l.d.estr); + varFreeMembers(&l); break; case NOT: cnfexprEval(expr->r, &r, usrptr); ret->datatype = 'N'; ret->d.n = !var2Number(&r, &convok_r); - if(r.datatype == 'S') es_deleteStr(r.d.estr); + varFreeMembers(&r); break; case 'N': ret->datatype = 'N'; @@ -1955,16 +2234,16 @@ cnfexprEval(struct cnfexpr *expr, struct var *ret, void* usrptr) COMP_NUM_BINOP(*); break; case '/': - COMP_NUM_BINOP(/); + COMP_NUM_BINOP_DIV(/); break; case '%': - COMP_NUM_BINOP(%); + COMP_NUM_BINOP_DIV(%); break; case 'M': cnfexprEval(expr->r, &r, usrptr); ret->datatype = 'N'; ret->d.n = -var2Number(&r, &convok_r); - if(r.datatype == 'S') es_deleteStr(r.d.estr); + varFreeMembers(&r); break; case 'F': doFuncCall((struct cnffunc*) expr, ret, usrptr); @@ -1976,6 +2255,7 @@ cnfexprEval(struct cnfexpr *expr, struct var *ret, void* usrptr) (unsigned) expr->nodetype, (char) expr->nodetype); break; } + DBGPRINTF("eval expr %p, return datatype '%c'\n", expr, ret->datatype); } //--------------------------------------------------------- @@ -2007,14 +2287,15 @@ cnffuncDestruct(struct cnffunc *func) break; default:break; } - free(func->funcdata); + if(func->fID != CNFFUNC_EXEC_TEMPLATE) + free(func->funcdata); free(func->fname); } /* Destruct an expression and all sub-expressions contained in it. */ void -cnfexprDestruct(struct cnfexpr *expr) +cnfexprDestruct(struct cnfexpr *__restrict__ const expr) { if(expr == NULL) { @@ -2057,6 +2338,7 @@ cnfexprDestruct(struct cnfexpr *expr) break; case 'V': free(((struct cnfvar*)expr)->name); + msgPropDescrDestruct(&(((struct cnfvar*)expr)->prop)); break; case 'F': cnffuncDestruct((struct cnffunc*)expr); @@ -2077,7 +2359,7 @@ cnfexprDestruct(struct cnfexpr *expr) * important. */ int -cnfexprEvalBool(struct cnfexpr *expr, void *usrptr) +cnfexprEvalBool(struct cnfexpr *__restrict__ const expr, void *__restrict__ const usrptr) { int convok; struct var ret; @@ -2269,11 +2551,13 @@ cnfstmtPrintOnly(struct cnfstmt *stmt, int indent, sbool subtree) break; case S_CALL: cstr = es_str2cstr(stmt->d.s_call.name, NULL); - doIndent(indent); dbgprintf("CALL [%s]\n", cstr); + doIndent(indent); dbgprintf("CALL [%s, queue:%d]\n", cstr, + stmt->d.s_call.ruleset == NULL ? 0 : 1); free(cstr); break; case S_ACT: - doIndent(indent); dbgprintf("ACTION %p [%s]\n", stmt->d.act, stmt->printable); + doIndent(indent); dbgprintf("ACTION %d [%s:%s]\n", stmt->d.act->iActionNbr, + modGetName(stmt->d.act->pMod), stmt->printable); break; case S_IF: doIndent(indent); dbgprintf("IF\n"); @@ -2313,12 +2597,12 @@ cnfstmtPrintOnly(struct cnfstmt *stmt, int indent, sbool subtree) case S_PROPFILT: doIndent(indent); dbgprintf("PROPFILT\n"); doIndent(indent); dbgprintf("\tProperty.: '%s'\n", - propIDToName(stmt->d.s_propfilt.propID)); - if(stmt->d.s_propfilt.propName != NULL) { - cstr = es_str2cstr(stmt->d.s_propfilt.propName, NULL); + propIDToName(stmt->d.s_propfilt.prop.id)); + if(stmt->d.s_propfilt.prop.id == PROP_CEE || + stmt->d.s_propfilt.prop.id == PROP_LOCAL_VAR || + stmt->d.s_propfilt.prop.id == PROP_GLOBAL_VAR) { doIndent(indent); - dbgprintf("\tCEE-Prop.: '%s'\n", cstr); - free(cstr); + dbgprintf("\tCEE-Prop.: '%s'\n", stmt->d.s_propfilt.prop.name); } doIndent(indent); dbgprintf("\tOperation: "); if(stmt->d.s_propfilt.isNegated) @@ -2351,7 +2635,7 @@ cnfstmtPrint(struct cnfstmt *root, int indent) } struct cnfnumval* -cnfnumvalNew(long long val) +cnfnumvalNew(const long long val) { struct cnfnumval *numval; if((numval = malloc(sizeof(struct cnfnumval))) != NULL) { @@ -2362,7 +2646,7 @@ cnfnumvalNew(long long val) } struct cnfstringval* -cnfstringvalNew(es_str_t *estr) +cnfstringvalNew(es_str_t *const estr) { struct cnfstringval *strval; if((strval = malloc(sizeof(struct cnfstringval))) != NULL) { @@ -2391,7 +2675,7 @@ done: return ar; } struct cnfarray* -cnfarrayAdd(struct cnfarray *ar, es_str_t *val) +cnfarrayAdd(struct cnfarray *__restrict__ const ar, es_str_t *__restrict__ val) { es_str_t **newptr; if((newptr = realloc(ar->arr, (ar->nmemb+1)*sizeof(es_str_t*))) == NULL) { @@ -2425,6 +2709,7 @@ cnfvarNew(char *name) if((var = malloc(sizeof(struct cnfvar))) != NULL) { var->nodetype = 'V'; var->name = name; + msgPropDescrFill(&var->prop, (uchar*)var->name, strlen(var->name)); } return var; } @@ -2441,59 +2726,68 @@ cnfstmtNew(unsigned s_type) return cnfstmt; } +void cnfstmtDestructLst(struct cnfstmt *root); + +/* delete a single stmt */ +static void +cnfstmtDestruct(struct cnfstmt *stmt) +{ + switch(stmt->nodetype) { + case S_NOP: + case S_STOP: + break; + case S_CALL: + es_deleteStr(stmt->d.s_call.name); + break; + case S_ACT: + actionDestruct(stmt->d.act); + break; + case S_IF: + cnfexprDestruct(stmt->d.s_if.expr); + if(stmt->d.s_if.t_then != NULL) { + cnfstmtDestructLst(stmt->d.s_if.t_then); + } + if(stmt->d.s_if.t_else != NULL) { + cnfstmtDestructLst(stmt->d.s_if.t_else); + } + break; + case S_SET: + free(stmt->d.s_set.varname); + cnfexprDestruct(stmt->d.s_set.expr); + break; + case S_UNSET: + free(stmt->d.s_set.varname); + break; + case S_PRIFILT: + cnfstmtDestructLst(stmt->d.s_prifilt.t_then); + cnfstmtDestructLst(stmt->d.s_prifilt.t_else); + break; + case S_PROPFILT: + msgPropDescrDestruct(&stmt->d.s_propfilt.prop); + if(stmt->d.s_propfilt.regex_cache != NULL) + rsCStrRegexDestruct(&stmt->d.s_propfilt.regex_cache); + if(stmt->d.s_propfilt.pCSCompValue != NULL) + cstrDestruct(&stmt->d.s_propfilt.pCSCompValue); + cnfstmtDestructLst(stmt->d.s_propfilt.t_then); + break; + default: + dbgprintf("error: unknown stmt type during destruct %u\n", + (unsigned) stmt->nodetype); + break; + } + free(stmt->printable); + free(stmt); +} + +/* delete a stmt and all others following it */ void -cnfstmtDestruct(struct cnfstmt *root) +cnfstmtDestructLst(struct cnfstmt *root) { struct cnfstmt *stmt, *todel; for(stmt = root ; stmt != NULL ; ) { - switch(stmt->nodetype) { - case S_NOP: - case S_STOP: - break; - case S_CALL: - es_deleteStr(stmt->d.s_call.name); - break; - case S_ACT: - actionDestruct(stmt->d.act); - break; - case S_IF: - cnfexprDestruct(stmt->d.s_if.expr); - if(stmt->d.s_if.t_then != NULL) { - cnfstmtDestruct(stmt->d.s_if.t_then); - } - if(stmt->d.s_if.t_else != NULL) { - cnfstmtDestruct(stmt->d.s_if.t_else); - } - break; - case S_SET: - free(stmt->d.s_set.varname); - cnfexprDestruct(stmt->d.s_set.expr); - break; - case S_UNSET: - free(stmt->d.s_set.varname); - break; - case S_PRIFILT: - cnfstmtDestruct(stmt->d.s_prifilt.t_then); - cnfstmtDestruct(stmt->d.s_prifilt.t_else); - break; - case S_PROPFILT: - if(stmt->d.s_propfilt.propName != NULL) - es_deleteStr(stmt->d.s_propfilt.propName); - if(stmt->d.s_propfilt.regex_cache != NULL) - rsCStrRegexDestruct(&stmt->d.s_propfilt.regex_cache); - if(stmt->d.s_propfilt.pCSCompValue != NULL) - cstrDestruct(&stmt->d.s_propfilt.pCSCompValue); - cnfstmtDestruct(stmt->d.s_propfilt.t_then); - break; - default: - dbgprintf("error: unknown stmt type during destruct %u\n", - (unsigned) stmt->nodetype); - break; - } - free(stmt->printable); todel = stmt; stmt = stmt->next; - free(todel); + cnfstmtDestruct(todel); } } @@ -2551,14 +2845,15 @@ struct cnfstmt * cnfstmtNewPROPFILT(char *propfilt, struct cnfstmt *t_then) { struct cnfstmt* cnfstmt; - rsRetVal lRet; if((cnfstmt = cnfstmtNew(S_PROPFILT)) != NULL) { cnfstmt->printable = (uchar*)propfilt; cnfstmt->d.s_propfilt.t_then = t_then; - cnfstmt->d.s_propfilt.propName = NULL; cnfstmt->d.s_propfilt.regex_cache = NULL; cnfstmt->d.s_propfilt.pCSCompValue = NULL; - lRet = DecodePropFilter((uchar*)propfilt, cnfstmt); + if(DecodePropFilter((uchar*)propfilt, cnfstmt) != RS_RET_OK) { + cnfstmt->nodetype = S_NOP; /* disable action! */ + cnfstmtDestructLst(t_then); /* we do no longer need this */ + } } return cnfstmt; } @@ -2599,7 +2894,7 @@ cnfstmtNewLegaAct(char *actline) goto done; cnfstmt->printable = (uchar*)strdup((char*)actline); localRet = cflineDoAction(loadConf, (uchar**)&actline, &cnfstmt->d.act); - if(localRet != RS_RET_OK && localRet != RS_RET_OK_WARN) { + if(localRet != RS_RET_OK) { parser_errmsg("%s occured in file '%s' around line %d", (localRet == RS_RET_OK_WARN) ? "warnings" : "errors", cnfcurrfn, yylineno); @@ -2714,7 +3009,10 @@ cnfexprOptimize_CMP_severity_facility(struct cnfexpr *expr) { struct cnffunc *func; - if(!strcmp("$syslogseverity", ((struct cnfvar*)expr->l)->name)) { + if(expr->l->nodetype != 'V') + FINALIZE; + + if(!strcmp("syslogseverity", ((struct cnfvar*)expr->l)->name)) { if(expr->r->nodetype == 'N') { int sev = (int) ((struct cnfnumval*)expr->r)->val; if(sev >= 0 && sev <= 7) { @@ -2728,7 +3026,7 @@ cnfexprOptimize_CMP_severity_facility(struct cnfexpr *expr) "evaluate to FALSE", sev); } } - } else if(!strcmp("$syslogfacility", ((struct cnfvar*)expr->l)->name)) { + } else if(!strcmp("syslogfacility", ((struct cnfvar*)expr->l)->name)) { if(expr->r->nodetype == 'N') { int fac = (int) ((struct cnfnumval*)expr->r)->val; if(fac >= 0 && fac <= 24) { @@ -2743,6 +3041,7 @@ cnfexprOptimize_CMP_severity_facility(struct cnfexpr *expr) } } } +finalize_it: return expr; } @@ -2755,7 +3054,7 @@ cnfexprOptimize_CMP_var(struct cnfexpr *expr) { struct cnffunc *func; - if(!strcmp("$syslogfacility-text", ((struct cnfvar*)expr->l)->name)) { + if(!strcmp("syslogfacility-text", ((struct cnfvar*)expr->l)->name)) { if(expr->r->nodetype == 'S') { char *cstr = es_str2cstr(((struct cnfstringval*)expr->r)->estr, NULL); int fac = decodeSyslogName((uchar*)cstr, syslogFacNames); @@ -2763,7 +3062,7 @@ cnfexprOptimize_CMP_var(struct cnfexpr *expr) parser_errmsg("invalid facility '%s', expression will always " "evaluate to FALSE", cstr); } else { - /* we can acutally optimize! */ + /* we can actually optimize! */ DBGPRINTF("optimizer: change comparison OP to FUNC prifilt()\n"); func = cnffuncNew_prifilt(fac); if(expr->nodetype == CMP_NE) @@ -2773,7 +3072,7 @@ cnfexprOptimize_CMP_var(struct cnfexpr *expr) } free(cstr); } - } else if(!strcmp("$syslogseverity-text", ((struct cnfvar*)expr->l)->name)) { + } else if(!strcmp("syslogseverity-text", ((struct cnfvar*)expr->l)->name)) { if(expr->r->nodetype == 'S') { char *cstr = es_str2cstr(((struct cnfstringval*)expr->r)->estr, NULL); int sev = decodeSyslogName((uchar*)cstr, syslogPriNames); @@ -2842,7 +3141,7 @@ cnfexprOptimize_AND_OR(struct cnfexpr *expr) static inline void cnfexprOptimize_CMPEQ_arr(struct cnfarray *arr) { - DBGPRINTF("optimizer: sorting array for CMP_EQ/NEQ comparison\n"); + DBGPRINTF("optimizer: sorting array of %d members for CMP_EQ/NEQ comparison\n", arr->nmemb); qsort(arr->arr, arr->nmemb, sizeof(es_str_t*), qs_arrcmp); } @@ -2880,13 +3179,23 @@ cnfexprOptimize(struct cnfexpr *expr) case '/': if(getConstNumber(expr, &ln, &rn)) { expr->nodetype = 'N'; - ((struct cnfnumval*)expr)->val = ln / rn; + if(rn == 0) { + /* division by zero */ + ((struct cnfnumval*)expr)->val = 0; + } else { + ((struct cnfnumval*)expr)->val = ln / rn; + } } break; case '%': if(getConstNumber(expr, &ln, &rn)) { expr->nodetype = 'N'; - ((struct cnfnumval*)expr)->val = ln % rn; + if(rn == 0) { + /* division by zero */ + ((struct cnfnumval*)expr)->val = 0; + } else { + ((struct cnfnumval*)expr)->val = ln % rn; + } } break; case CMP_NE: @@ -2904,10 +3213,14 @@ cnfexprOptimize(struct cnfexpr *expr) expr->r = exprswap; } } + if(expr->r->nodetype == 'A') { + cnfexprOptimize_CMPEQ_arr((struct cnfarray *)expr->r); + } + /* This should be evaluated last because it may change expr + * to a function. + */ if(expr->l->nodetype == 'V') { expr = cnfexprOptimize_CMP_var(expr); - } else if(expr->r->nodetype == 'A') { - cnfexprOptimize_CMPEQ_arr((struct cnfarray *)expr->r); } break; case CMP_LE: @@ -3044,7 +3357,7 @@ cnfstmtOptimizePRIFilt(struct cnfstmt *stmt) DBGPRINTF("optimizer: removing always-true PRIFILT %p\n", stmt); if(stmt->d.s_prifilt.t_else != NULL) { parser_errmsg("error: always-true PRI filter has else part!\n"); - cnfstmtDestruct(stmt->d.s_prifilt.t_else); + cnfstmtDestructLst(stmt->d.s_prifilt.t_else); } free(stmt->printable); stmt->printable = NULL; @@ -3084,8 +3397,14 @@ cnfstmtOptimizeCall(struct cnfstmt *stmt) stmt->nodetype = S_NOP; goto done; } - DBGPRINTF("CALL obtained ruleset ptr %p for ruleset %s\n", pRuleset, rsName); - stmt->d.s_call.stmt = pRuleset->root; + DBGPRINTF("CALL obtained ruleset ptr %p for ruleset %s [hasQueue:%d]\n", + pRuleset, rsName, rulesetHasQueue(pRuleset)); + if(rulesetHasQueue(pRuleset)) { + stmt->d.s_call.ruleset = pRuleset; + } else { + stmt->d.s_call.ruleset = NULL; + stmt->d.s_call.stmt = pRuleset->root; + } done: free(rsName); return; @@ -3210,6 +3529,13 @@ funcName2ID(es_str_t *fname, unsigned short nParams) return CNFFUNC_INVALID; } return CNFFUNC_FIELD; + } else if(!es_strbufcmp(fname, (unsigned char*)"exec_template", sizeof("exec_template") - 1)) { + if(nParams != 1) { + parser_errmsg("number of parameters for exec-template() must be one " + "but is %d.", nParams); + return CNFFUNC_INVALID; + } + return CNFFUNC_EXEC_TEMPLATE; } else if(!es_strbufcmp(fname, (unsigned char*)"prifilt", sizeof("prifilt") - 1)) { if(nParams != 1) { parser_errmsg("number of parameters for prifilt() must be one " @@ -3217,6 +3543,13 @@ funcName2ID(es_str_t *fname, unsigned short nParams) return CNFFUNC_INVALID; } return CNFFUNC_PRIFILT; + } else if(!es_strbufcmp(fname, (unsigned char*)"lookup", sizeof("lookup") - 1)) { + if(nParams != 2) { + parser_errmsg("number of parameters for lookup() must be two " + "but is %d.", nParams); + return CNFFUNC_INVALID; + } + return CNFFUNC_LOOKUP; } else { return CNFFUNC_INVALID; } @@ -3258,6 +3591,31 @@ finalize_it: } +static rsRetVal +initFunc_exec_template(struct cnffunc *func) +{ + char *tplName = NULL; + DEFiRet; + + if(func->expr[0]->nodetype != 'S') { + parser_errmsg("exec_template(): param 1 must be a constant string"); + FINALIZE; + } + + tplName = es_str2cstr(((struct cnfstringval*) func->expr[0])->estr, NULL); + func->funcdata = tplFind(ourConf, tplName, strlen(tplName)); + if(func->funcdata == NULL) { + parser_errmsg("exec_template(): template '%s' could not be found", tplName); + FINALIZE; + } + + +finalize_it: + free(tplName); + RETiRet; +} + + static inline rsRetVal initFunc_prifilt(struct cnffunc *func) { @@ -3281,6 +3639,30 @@ finalize_it: } +static inline rsRetVal +initFunc_lookup(struct cnffunc *func) +{ + uchar *cstr = NULL; + DEFiRet; + + func->funcdata = NULL; + if(func->expr[0]->nodetype != 'S') { + parser_errmsg("table name (param 1) of lookup() must be a constant string"); + FINALIZE; + } + + cstr = (uchar*)es_str2cstr(((struct cnfstringval*) func->expr[0])->estr, NULL); + if((func->funcdata = lookupFindTable(cstr)) == NULL) { + parser_errmsg("lookup table '%s' not found", cstr); + FINALIZE; + } + +finalize_it: + free(cstr); + RETiRet; +} + + struct cnffunc * cnffuncNew(es_str_t *fname, struct cnffparamlst* paramlst) { @@ -3318,6 +3700,12 @@ cnffuncNew(es_str_t *fname, struct cnffparamlst* paramlst) case CNFFUNC_PRIFILT: initFunc_prifilt(func); break; + case CNFFUNC_LOOKUP: + initFunc_lookup(func); + break; + case CNFFUNC_EXEC_TEMPLATE: + initFunc_exec_template(func); + break; default:break; } } @@ -3333,13 +3721,20 @@ cnffuncNew_prifilt(int fac) { struct cnffunc* func; + fac >>= 3; + if (fac >= LOG_NFACILITIES + 1 || fac < 0) + return NULL; + if((func = malloc(sizeof(struct cnffunc))) != NULL) { + if ((func->funcdata = calloc(1, sizeof(struct funcData_prifilt))) == NULL) { + free(func); + return NULL; + } func->nodetype = 'F'; func->fname = es_newStrFromCStr("prifilt", sizeof("prifilt")-1); func->nParams = 0; func->fID = CNFFUNC_PRIFILT; - func->funcdata = calloc(1, sizeof(struct funcData_prifilt)); - ((struct funcData_prifilt *)func->funcdata)->pmask[fac >> 3] = TABLE_ALLPRI; + ((struct funcData_prifilt *)func->funcdata)->pmask[fac] = TABLE_ALLPRI; } return func; } @@ -3444,6 +3839,8 @@ void cnfparamvalsDestruct(struct cnfparamvals *paramvals, struct cnfparamblk *blk) { int i; + if(paramvals == NULL) + return; for(i = 0 ; i < blk->nParams ; ++i) { if(paramvals[i].bUsed) { varDelete(¶mvals[i].val); diff --git a/grammar/rainerscript.h b/grammar/rainerscript.h index 31b2eb9..453b0f3 100644 --- a/grammar/rainerscript.h +++ b/grammar/rainerscript.h @@ -1,3 +1,21 @@ +/* rsyslog rainerscript definitions + * + * Copyright 2011-2014 Rainer Gerhards + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * -or- + * see COPYING.ASL20 in the source distribution + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ #ifndef INC_UTILS_H #define INC_UTILS_H #include <stdio.h> @@ -5,9 +23,9 @@ #include <typedefs.h> #include <sys/types.h> #include <regex.h> +#include "typedefs.h" - -#define LOG_NFACILITIES 24 /* current number of syslog facilities */ +#define LOG_NFACILITIES 24+1 /* we copy&paste this as including rsyslog.h gets us in off64_t trouble... :-( */ #define CNFFUNC_MAX_ARGS 32 /**< maximum number of arguments that any function can have (among * others, this is used to size data structures). @@ -24,6 +42,10 @@ enum cnfobjType { CNFOBJ_TPL, CNFOBJ_PROPERTY, CNFOBJ_CONSTANT, + CNFOBJ_MAINQ, + CNFOBJ_LOOKUP_TABLE, + CNFOBJ_PARSER, + CNFOBJ_TIMEZONE, CNFOBJ_INVALID = 0 }; @@ -55,6 +77,11 @@ cnfobjType2str(enum cnfobjType ot) case CNFOBJ_CONSTANT: return "constant"; break; + case CNFOBJ_MAINQ: + return "main_queue"; + case CNFOBJ_LOOKUP_TABLE: + return "lookup_table"; + break; default:return "error: invalid cnfobjType"; } } @@ -164,6 +191,7 @@ struct cnfstmt { struct { es_str_t *name; struct cnfstmt *stmt; + ruleset_t *ruleset; /* non-NULL if the ruleset has a queue assigned */ } s_call; struct { uchar pmask[LOG_NFACILITIES+1]; /* priority mask */ @@ -175,8 +203,7 @@ struct cnfstmt { regex_t *regex_cache;/* cache for compiled REs, if used */ struct cstr_s *pCSCompValue;/* value to "compare" against */ sbool isNegated; - uintTiny propID;/* ID of the requested property */ - es_str_t *propName;/* name of property for CEE-based filters */ + msgPropDescr_t prop; /* requested property */ struct cnfstmt *t_then; struct cnfstmt *t_else; } s_propfilt; @@ -203,6 +230,7 @@ struct cnfstringval { struct cnfvar { unsigned nodetype; char *name; + msgPropDescr_t prop; }; struct cnfarray { @@ -228,7 +256,9 @@ enum cnffuncid { CNFFUNC_RE_MATCH, CNFFUNC_RE_EXTRACT, CNFFUNC_FIELD, - CNFFUNC_PRIFILT + CNFFUNC_PRIFILT, + CNFFUNC_LOOKUP, + CNFFUNC_EXEC_TEMPLATE }; struct cnffunc { @@ -301,7 +331,7 @@ void cnfobjDestruct(struct cnfobj *o); void cnfobjPrint(struct cnfobj *o); struct cnfexpr* cnfexprNew(unsigned nodetype, struct cnfexpr *l, struct cnfexpr *r); void cnfexprPrint(struct cnfexpr *expr, int indent); -void cnfexprEval(struct cnfexpr *expr, struct var *ret, void *pusr); +void cnfexprEval(const struct cnfexpr *const expr, struct var *ret, void *pusr); int cnfexprEvalBool(struct cnfexpr *expr, void *usrptr); void cnfexprDestruct(struct cnfexpr *expr); struct cnfnumval* cnfnumvalNew(long long val); @@ -331,13 +361,13 @@ struct cnfstmt * cnfstmtNewSet(char *var, struct cnfexpr *expr); struct cnfstmt * cnfstmtNewUnset(char *var); struct cnfstmt * cnfstmtNewCall(es_str_t *name); struct cnfstmt * cnfstmtNewContinue(void); -void cnfstmtDestruct(struct cnfstmt *root); +void cnfstmtDestructLst(struct cnfstmt *root); void cnfstmtOptimize(struct cnfstmt *root); struct cnfarray* cnfarrayNew(es_str_t *val); struct cnfarray* cnfarrayDup(struct cnfarray *old); struct cnfarray* cnfarrayAdd(struct cnfarray *ar, es_str_t *val); void cnfarrayContentDestruct(struct cnfarray *ar); -char* getFIOPName(unsigned iFIOP); +const char* getFIOPName(unsigned iFIOP); rsRetVal initRainerscript(void); void unescapeStr(uchar *s, int len); char * tokenval2str(int tok); @@ -116,6 +116,9 @@ static int read_all(int fd, char *buf, unsigned int nbyte) struct timeval tv; #ifdef USE_UNLIMITED_SELECT fd_set *pRfds = malloc(glbl.GetFdSetSize()); + + if (pRfds == NULL) + return -1; #else fd_set rfds; fd_set *pRfds = &rfds; diff --git a/java/Makefile.in b/java/Makefile.in index 97d6734..4083830 100644 --- a/java/Makefile.in +++ b/java/Makefile.in @@ -107,7 +107,6 @@ GREP = @GREP@ GSS_LIBS = @GSS_LIBS@ GUARDTIME_CFLAGS = @GUARDTIME_CFLAGS@ GUARDTIME_LIBS = @GUARDTIME_LIBS@ -HAVE_LIBGCRYPT_CONFIG = @HAVE_LIBGCRYPT_CONFIG@ HAVE_MYSQL_CONFIG = @HAVE_MYSQL_CONFIG@ HAVE_ORACLE_CONFIG = @HAVE_ORACLE_CONFIG@ HAVE_PGSQL_CONFIG = @HAVE_PGSQL_CONFIG@ @@ -128,14 +127,15 @@ LEXLIB = @LEXLIB@ LEX_OUTPUT_ROOT = @LEX_OUTPUT_ROOT@ LIBDBI_CFLAGS = @LIBDBI_CFLAGS@ LIBDBI_LIBS = @LIBDBI_LIBS@ -LIBEE_CFLAGS = @LIBEE_CFLAGS@ -LIBEE_LIBS = @LIBEE_LIBS@ LIBESTR_CFLAGS = @LIBESTR_CFLAGS@ LIBESTR_LIBS = @LIBESTR_LIBS@ LIBGCRYPT_CFLAGS = @LIBGCRYPT_CFLAGS@ +LIBGCRYPT_CONFIG = @LIBGCRYPT_CONFIG@ LIBGCRYPT_LIBS = @LIBGCRYPT_LIBS@ LIBLOGGING_CFLAGS = @LIBLOGGING_CFLAGS@ LIBLOGGING_LIBS = @LIBLOGGING_LIBS@ +LIBLOGGING_STDLOG_CFLAGS = @LIBLOGGING_STDLOG_CFLAGS@ +LIBLOGGING_STDLOG_LIBS = @LIBLOGGING_STDLOG_LIBS@ LIBLOGNORM_CFLAGS = @LIBLOGNORM_CFLAGS@ LIBLOGNORM_LIBS = @LIBLOGNORM_LIBS@ LIBM = @LIBM@ @@ -160,6 +160,8 @@ NM = @NM@ NMEDIT = @NMEDIT@ OBJDUMP = @OBJDUMP@ OBJEXT = @OBJEXT@ +OPENSSL_CFLAGS = @OPENSSL_CFLAGS@ +OPENSSL_LIBS = @OPENSSL_LIBS@ ORACLE_CFLAGS = @ORACLE_CFLAGS@ ORACLE_LIBS = @ORACLE_LIBS@ OTOOL = @OTOOL@ diff --git a/m4/atomic_operations_64bit.m4 b/m4/atomic_operations_64bit.m4 index 3121cf1..9fbef0a 100644 --- a/m4/atomic_operations_64bit.m4 +++ b/m4/atomic_operations_64bit.m4 @@ -47,7 +47,7 @@ int main() }], [ap_cv_atomic_builtins_64=yes], [ap_cv_atomic_builtins_64=no], [ap_cv_atomic_builtins_64=no])]) if test "$ap_cv_atomic_builtins_64" = "yes"; then - AC_DEFINE(HAVE_ATOMIC_BUILTINS_64BIT, 1, [Define if compiler provides 64 bit atomic builtins]) + AC_DEFINE(HAVE_ATOMIC_BUILTINS64, 1, [Define if compiler provides 64 bit atomic builtins]) fi ]) diff --git a/outchannel.c b/outchannel.c index c97d220..ac2e087 100644 --- a/outchannel.c +++ b/outchannel.c @@ -110,7 +110,7 @@ static rsRetVal get_Field(uchar **pp, uchar **pField) *pp = p; CHKiRet(cstrFinalize(pStrB)); - CHKiRet(cstrConvSzStrAndDestruct(pStrB, pField, 0)); + CHKiRet(cstrConvSzStrAndDestruct(&pStrB, pField, 0)); finalize_it: if(iRet != RS_RET_OK) { @@ -178,7 +178,7 @@ static inline rsRetVal get_restOfLine(uchar **pp, uchar **pBuf) *pp = p; CHKiRet(cstrFinalize(pStrB)); - CHKiRet(cstrConvSzStrAndDestruct(pStrB, pBuf, 0)); + CHKiRet(cstrConvSzStrAndDestruct(&pStrB, pBuf, 0)); finalize_it: if(iRet != RS_RET_OK) { @@ -414,14 +414,16 @@ rsRetVal parsAddrWithBits(rsParsObj *pThis, struct NetAddr **pIP, int *pBits) /* now we have the string and must check/convert it to * an NetAddr structure. */ - CHKiRet(cstrConvSzStrAndDestruct(pCStr, &pszIP, 0)); + CHKiRet(cstrConvSzStrAndDestruct(&pCStr, &pszIP, 0)); - *pIP = calloc(1, sizeof(struct NetAddr)); + if((*pIP = calloc(1, sizeof(struct NetAddr))) == NULL) + ABORT_FINALIZE(RS_RET_OUT_OF_MEMORY); if (*((char*)pszIP) == '[') { pszTmp = (uchar*)strchr ((char*)pszIP, ']'); if (pszTmp == NULL) { free (pszIP); + free (*pIP); ABORT_FINALIZE(RS_RET_INVALID_IP); } *pszTmp = '\0'; diff --git a/plugins/im3195/Makefile.in b/plugins/im3195/Makefile.in index 94f22b1..ee26cb8 100644 --- a/plugins/im3195/Makefile.in +++ b/plugins/im3195/Makefile.in @@ -154,7 +154,6 @@ GREP = @GREP@ GSS_LIBS = @GSS_LIBS@ GUARDTIME_CFLAGS = @GUARDTIME_CFLAGS@ GUARDTIME_LIBS = @GUARDTIME_LIBS@ -HAVE_LIBGCRYPT_CONFIG = @HAVE_LIBGCRYPT_CONFIG@ HAVE_MYSQL_CONFIG = @HAVE_MYSQL_CONFIG@ HAVE_ORACLE_CONFIG = @HAVE_ORACLE_CONFIG@ HAVE_PGSQL_CONFIG = @HAVE_PGSQL_CONFIG@ @@ -175,14 +174,15 @@ LEXLIB = @LEXLIB@ LEX_OUTPUT_ROOT = @LEX_OUTPUT_ROOT@ LIBDBI_CFLAGS = @LIBDBI_CFLAGS@ LIBDBI_LIBS = @LIBDBI_LIBS@ -LIBEE_CFLAGS = @LIBEE_CFLAGS@ -LIBEE_LIBS = @LIBEE_LIBS@ LIBESTR_CFLAGS = @LIBESTR_CFLAGS@ LIBESTR_LIBS = @LIBESTR_LIBS@ LIBGCRYPT_CFLAGS = @LIBGCRYPT_CFLAGS@ +LIBGCRYPT_CONFIG = @LIBGCRYPT_CONFIG@ LIBGCRYPT_LIBS = @LIBGCRYPT_LIBS@ LIBLOGGING_CFLAGS = @LIBLOGGING_CFLAGS@ LIBLOGGING_LIBS = @LIBLOGGING_LIBS@ +LIBLOGGING_STDLOG_CFLAGS = @LIBLOGGING_STDLOG_CFLAGS@ +LIBLOGGING_STDLOG_LIBS = @LIBLOGGING_STDLOG_LIBS@ LIBLOGNORM_CFLAGS = @LIBLOGNORM_CFLAGS@ LIBLOGNORM_LIBS = @LIBLOGNORM_LIBS@ LIBM = @LIBM@ @@ -207,6 +207,8 @@ NM = @NM@ NMEDIT = @NMEDIT@ OBJDUMP = @OBJDUMP@ OBJEXT = @OBJEXT@ +OPENSSL_CFLAGS = @OPENSSL_CFLAGS@ +OPENSSL_LIBS = @OPENSSL_LIBS@ ORACLE_CFLAGS = @ORACLE_CFLAGS@ ORACLE_LIBS = @ORACLE_LIBS@ OTOOL = @OTOOL@ @@ -388,22 +390,25 @@ distclean-compile: @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/im3195_la-im3195.Plo@am__quote@ .c.o: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(COMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ $< -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Po +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.o$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c $< +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ $< .c.obj: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(COMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ `$(CYGPATH_W) '$<'` -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Po +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.obj$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ `$(CYGPATH_W) '$<'` &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c `$(CYGPATH_W) '$<'` +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ `$(CYGPATH_W) '$<'` .c.lo: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(LTCOMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ $< -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Plo +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.lo$$||'`;\ +@am__fastdepCC_TRUE@ $(LTCOMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Plo @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=yes @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ @am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(LTCOMPILE) -c -o $@ $< diff --git a/plugins/imdiag/Makefile.in b/plugins/imdiag/Makefile.in index 9e0933a..7fd2c1e 100644 --- a/plugins/imdiag/Makefile.in +++ b/plugins/imdiag/Makefile.in @@ -153,7 +153,6 @@ GREP = @GREP@ GSS_LIBS = @GSS_LIBS@ GUARDTIME_CFLAGS = @GUARDTIME_CFLAGS@ GUARDTIME_LIBS = @GUARDTIME_LIBS@ -HAVE_LIBGCRYPT_CONFIG = @HAVE_LIBGCRYPT_CONFIG@ HAVE_MYSQL_CONFIG = @HAVE_MYSQL_CONFIG@ HAVE_ORACLE_CONFIG = @HAVE_ORACLE_CONFIG@ HAVE_PGSQL_CONFIG = @HAVE_PGSQL_CONFIG@ @@ -174,14 +173,15 @@ LEXLIB = @LEXLIB@ LEX_OUTPUT_ROOT = @LEX_OUTPUT_ROOT@ LIBDBI_CFLAGS = @LIBDBI_CFLAGS@ LIBDBI_LIBS = @LIBDBI_LIBS@ -LIBEE_CFLAGS = @LIBEE_CFLAGS@ -LIBEE_LIBS = @LIBEE_LIBS@ LIBESTR_CFLAGS = @LIBESTR_CFLAGS@ LIBESTR_LIBS = @LIBESTR_LIBS@ LIBGCRYPT_CFLAGS = @LIBGCRYPT_CFLAGS@ +LIBGCRYPT_CONFIG = @LIBGCRYPT_CONFIG@ LIBGCRYPT_LIBS = @LIBGCRYPT_LIBS@ LIBLOGGING_CFLAGS = @LIBLOGGING_CFLAGS@ LIBLOGGING_LIBS = @LIBLOGGING_LIBS@ +LIBLOGGING_STDLOG_CFLAGS = @LIBLOGGING_STDLOG_CFLAGS@ +LIBLOGGING_STDLOG_LIBS = @LIBLOGGING_STDLOG_LIBS@ LIBLOGNORM_CFLAGS = @LIBLOGNORM_CFLAGS@ LIBLOGNORM_LIBS = @LIBLOGNORM_LIBS@ LIBM = @LIBM@ @@ -206,6 +206,8 @@ NM = @NM@ NMEDIT = @NMEDIT@ OBJDUMP = @OBJDUMP@ OBJEXT = @OBJEXT@ +OPENSSL_CFLAGS = @OPENSSL_CFLAGS@ +OPENSSL_LIBS = @OPENSSL_LIBS@ ORACLE_CFLAGS = @ORACLE_CFLAGS@ ORACLE_LIBS = @ORACLE_LIBS@ OTOOL = @OTOOL@ @@ -386,22 +388,25 @@ distclean-compile: @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/imdiag_la-imdiag.Plo@am__quote@ .c.o: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(COMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ $< -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Po +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.o$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c $< +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ $< .c.obj: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(COMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ `$(CYGPATH_W) '$<'` -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Po +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.obj$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ `$(CYGPATH_W) '$<'` &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c `$(CYGPATH_W) '$<'` +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ `$(CYGPATH_W) '$<'` .c.lo: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(LTCOMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ $< -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Plo +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.lo$$||'`;\ +@am__fastdepCC_TRUE@ $(LTCOMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Plo @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=yes @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ @am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(LTCOMPILE) -c -o $@ $< diff --git a/plugins/imdiag/imdiag.c b/plugins/imdiag/imdiag.c index 5fdc6ef..023d6f3 100644 --- a/plugins/imdiag/imdiag.c +++ b/plugins/imdiag/imdiag.c @@ -1,13 +1,11 @@ /* imdiag.c - * This is a diagnostics module, primarily meant for troubleshooting - * and information about the runtime state of rsyslog. It is implemented - * as an input plugin, because that interface best suits our needs - * and also enables us to inject test messages (something not yet - * implemented). + * This is a testbench tool. It started out with a broader scope, + * but we dropped this idea. To learn about rsyslog runtime statistics + * have a look at impstats. * * File begun on 2008-07-25 by RGerhards * - * Copyright 2008-2012 Adiscon GmbH. + * Copyright 2008-2014 Adiscon GmbH. * * This file is part of rsyslog. * @@ -54,6 +52,7 @@ #include "msg.h" #include "datetime.h" #include "ratelimit.h" +#include "queue.h" #include "net.h" /* for permittedPeers, may be removed when this is removed */ MODULE_TYPE_INPUT @@ -238,7 +237,7 @@ injectMsg(uchar *pszCmd, tcps_sess_t *pSess) int iFrom; int nMsgs; int i; - ratelimit_t *ratelimit; + ratelimit_t *ratelimit = NULL; DEFiRet; /* we do not check errors here! */ @@ -246,7 +245,7 @@ injectMsg(uchar *pszCmd, tcps_sess_t *pSess) iFrom = atoi((char*)wordBuf); getFirstWord(&pszCmd, wordBuf, sizeof(wordBuf)/sizeof(uchar), TO_LOWERCASE); nMsgs = atoi((char*)wordBuf); - ratelimitNew(&ratelimit, "imdiag", "injectmsg"); + CHKiRet(ratelimitNew(&ratelimit, "imdiag", "injectmsg")); for(i = 0 ; i < nMsgs ; ++i) { doInjectMsg(i + iFrom, ratelimit); @@ -254,41 +253,41 @@ injectMsg(uchar *pszCmd, tcps_sess_t *pSess) CHKiRet(sendResponse(pSess, "%d messages injected\n", nMsgs)); DBGPRINTF("imdiag: %d messages injected\n", nMsgs); - ratelimitDestruct(ratelimit); finalize_it: + if(ratelimit != NULL) + ratelimitDestruct(ratelimit); RETiRet; } -/* This function waits until the main queue is drained (size = 0) +/* This function waits until all queues are drained (size = 0) * To make sure it really is drained, we check three times. Otherwise we * may just see races. + * Note: until 2014--07-13, this checked just the main queue. However, + * the testbench was the sole user and checking all queues makes much more + * sense. So we change function semantics instead of carrying the old + * semantics over and crafting a new function. -- rgerhards */ static rsRetVal waitMainQEmpty(tcps_sess_t *pSess) { - int iMsgQueueSize; int iPrint = 0; DEFiRet; - CHKiRet(diagGetMainMsgQSize(&iMsgQueueSize)); while(1) { - if(iMsgQueueSize == 0) { + if(iOverallQueueSize == 0) { /* verify that queue is still empty (else it could just be a race!) */ srSleep(0,250000);/* wait a little bit */ - CHKiRet(diagGetMainMsgQSize(&iMsgQueueSize)); - if(iMsgQueueSize == 0) { + if(iOverallQueueSize == 0) { srSleep(0,500000);/* wait a little bit */ - CHKiRet(diagGetMainMsgQSize(&iMsgQueueSize)); } } - if(iMsgQueueSize == 0) + if(iOverallQueueSize == 0) break; if(iPrint++ % 500 == 0) - dbgprintf("imdiag sleeping, wait mainq drain, curr size %d\n", iMsgQueueSize); + dbgprintf("imdiag sleeping, wait queues drain, curr size %d\n", iOverallQueueSize); srSleep(0,200000);/* wait a little bit */ - CHKiRet(diagGetMainMsgQSize(&iMsgQueueSize)); } CHKiRet(sendResponse(pSess, "mainqueue empty\n")); @@ -304,7 +303,6 @@ finalize_it: static rsRetVal OnMsgReceived(tcps_sess_t *pSess, uchar *pRcv, int iLenMsg) { - int iMsgQueueSize; uchar *pszMsg; uchar *pToFree = NULL; uchar cmdBuf[1024]; @@ -326,9 +324,8 @@ OnMsgReceived(tcps_sess_t *pSess, uchar *pRcv, int iLenMsg) dbgprintf("imdiag received command '%s'\n", cmdBuf); if(!ustrcmp(cmdBuf, UCHAR_CONSTANT("getmainmsgqueuesize"))) { - CHKiRet(diagGetMainMsgQSize(&iMsgQueueSize)); - CHKiRet(sendResponse(pSess, "%d\n", iMsgQueueSize)); - DBGPRINTF("imdiag: %d messages in main queue\n", iMsgQueueSize); + CHKiRet(sendResponse(pSess, "%d\n", iOverallQueueSize)); + DBGPRINTF("imdiag: %d messages in main queue\n", iOverallQueueSize); } else if(!ustrcmp(cmdBuf, UCHAR_CONSTANT("waitmainqueueempty"))) { CHKiRet(waitMainQEmpty(pSess)); } else if(!ustrcmp(cmdBuf, UCHAR_CONSTANT("injectmsg"))) { diff --git a/plugins/imfile/Makefile.in b/plugins/imfile/Makefile.in index bce4baa..3e8aab2 100644 --- a/plugins/imfile/Makefile.in +++ b/plugins/imfile/Makefile.in @@ -153,7 +153,6 @@ GREP = @GREP@ GSS_LIBS = @GSS_LIBS@ GUARDTIME_CFLAGS = @GUARDTIME_CFLAGS@ GUARDTIME_LIBS = @GUARDTIME_LIBS@ -HAVE_LIBGCRYPT_CONFIG = @HAVE_LIBGCRYPT_CONFIG@ HAVE_MYSQL_CONFIG = @HAVE_MYSQL_CONFIG@ HAVE_ORACLE_CONFIG = @HAVE_ORACLE_CONFIG@ HAVE_PGSQL_CONFIG = @HAVE_PGSQL_CONFIG@ @@ -174,14 +173,15 @@ LEXLIB = @LEXLIB@ LEX_OUTPUT_ROOT = @LEX_OUTPUT_ROOT@ LIBDBI_CFLAGS = @LIBDBI_CFLAGS@ LIBDBI_LIBS = @LIBDBI_LIBS@ -LIBEE_CFLAGS = @LIBEE_CFLAGS@ -LIBEE_LIBS = @LIBEE_LIBS@ LIBESTR_CFLAGS = @LIBESTR_CFLAGS@ LIBESTR_LIBS = @LIBESTR_LIBS@ LIBGCRYPT_CFLAGS = @LIBGCRYPT_CFLAGS@ +LIBGCRYPT_CONFIG = @LIBGCRYPT_CONFIG@ LIBGCRYPT_LIBS = @LIBGCRYPT_LIBS@ LIBLOGGING_CFLAGS = @LIBLOGGING_CFLAGS@ LIBLOGGING_LIBS = @LIBLOGGING_LIBS@ +LIBLOGGING_STDLOG_CFLAGS = @LIBLOGGING_STDLOG_CFLAGS@ +LIBLOGGING_STDLOG_LIBS = @LIBLOGGING_STDLOG_LIBS@ LIBLOGNORM_CFLAGS = @LIBLOGNORM_CFLAGS@ LIBLOGNORM_LIBS = @LIBLOGNORM_LIBS@ LIBM = @LIBM@ @@ -206,6 +206,8 @@ NM = @NM@ NMEDIT = @NMEDIT@ OBJDUMP = @OBJDUMP@ OBJEXT = @OBJEXT@ +OPENSSL_CFLAGS = @OPENSSL_CFLAGS@ +OPENSSL_LIBS = @OPENSSL_LIBS@ ORACLE_CFLAGS = @ORACLE_CFLAGS@ ORACLE_LIBS = @ORACLE_LIBS@ OTOOL = @OTOOL@ @@ -386,22 +388,25 @@ distclean-compile: @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/imfile_la-imfile.Plo@am__quote@ .c.o: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(COMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ $< -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Po +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.o$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c $< +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ $< .c.obj: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(COMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ `$(CYGPATH_W) '$<'` -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Po +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.obj$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ `$(CYGPATH_W) '$<'` &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c `$(CYGPATH_W) '$<'` +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ `$(CYGPATH_W) '$<'` .c.lo: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(LTCOMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ $< -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Plo +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.lo$$||'`;\ +@am__fastdepCC_TRUE@ $(LTCOMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Plo @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=yes @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ @am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(LTCOMPILE) -c -o $@ $< diff --git a/plugins/imfile/imfile.c b/plugins/imfile/imfile.c index 349acea..d37cb03 100644 --- a/plugins/imfile/imfile.c +++ b/plugins/imfile/imfile.c @@ -5,7 +5,7 @@ * * Work originally begun on 2008-02-01 by Rainer Gerhards * - * Copyright 2008-2012 Adiscon GmbH. + * Copyright 2008-2014 Adiscon GmbH. * * This file is part of rsyslog. * @@ -23,7 +23,7 @@ * See the License for the specific language governing permissions and * limitations under the License. */ -#include "config.h" /* this is for autotools and always must be the first include */ +#include "config.h" #include <stdio.h> #include <stdlib.h> #include <assert.h> @@ -31,6 +31,12 @@ #include <errno.h> #include <fcntl.h> #include <pthread.h> /* do NOT remove: will soon be done by the module generation macros */ +#include <sys/types.h> +#include <unistd.h> +#include <fnmatch.h> +#ifdef HAVE_SYS_INOTIFY_H +#include <sys/inotify.h> +#endif #ifdef HAVE_SYS_STAT_H # include <sys/stat.h> #endif @@ -70,8 +76,17 @@ static int bLegacyCnfModGlobalsPermitted;/* are legacy module-global config para #define NUM_MULTISUB 1024 /* default max number of submits */ #define DFLT_PollInterval 10 +#define INIT_FILE_TAB_SIZE 4 /* default file table size - is extended as needed, use 2^x value */ +#define INIT_FILE_IN_DIR_TAB_SIZE 1 /* initial size for "associated files tab" in directory table */ +#define INIT_WDMAP_TAB_SIZE 1 /* default wdMap table size - is extended as needed, use 2^x value */ + +/* this structure is used in pure polling mode as well one of the support + * structures for inotify. + */ typedef struct fileInfo_s { uchar *pszFileName; + uchar *pszDirName; + uchar *pszBaseName; uchar *pszTag; size_t lenTag; uchar *pszStateFile; /* file in which state between runs is to be stored */ @@ -81,7 +96,8 @@ typedef struct fileInfo_s { int nRecords; /**< How many records did we process before persisting the stream? */ int iPersistStateInterval; /**< how often should state be persisted? (0=on close only) */ strm_t *pStrm; /* its stream (NULL if not assigned) */ - int readMode; /* which mode to use in ReadMulteLine call? */ + uint8_t readMode; /* which mode to use in ReadMulteLine call? */ + sbool escapeLF; /* escape LF inside the MSG content? */ ruleset_t *pRuleset; /* ruleset to bind listener to (use system default if unspecified) */ ratelimit_t *ratelimiter; multi_submit_t multiSub; @@ -103,6 +119,8 @@ static struct configSettings_s { struct instanceConf_s { uchar *pszFileName; + uchar *pszDirName; + uchar *pszFileBaseName; uchar *pszTag; uchar *pszStateFile; uchar *pszBindRuleset; @@ -110,7 +128,8 @@ struct instanceConf_s { int iPersistStateInterval; int iFacility; int iSeverity; - int readMode; + uint8_t readMode; + sbool escapeLF; int maxLinesAtOnce; ruleset_t *pBindRuleset; /* ruleset to bind listener to (use system default if unspecified) */ struct instanceConf_s *next; @@ -121,25 +140,76 @@ struct instanceConf_s { static rsRetVal persistStrmState(fileInfo_t *pInfo); static rsRetVal resetConfigVariables(uchar __attribute__((unused)) *pp, void __attribute__((unused)) *pVal); + +#define OPMODE_POLLING 0 +#define OPMODE_INOTIFY 1 + /* config variables */ struct modConfData_s { rsconf_t *pConf; /* our overall config object */ int iPollInterval; /* number of seconds to sleep when there was no file activity */ instanceConf_t *root, *tail; + uint8_t opMode; sbool configSetViaV2Method; }; static modConfData_t *loadModConf = NULL;/* modConf ptr to use for the current load process */ static modConfData_t *runModConf = NULL;/* modConf ptr to use for the current load process */ static int iFilPtr = 0; /* number of files to be monitored; pointer to next free spot during config */ -#define MAX_INPUT_FILES 100 -static fileInfo_t files[MAX_INPUT_FILES]; +static fileInfo_t *files = NULL; +static int allocMaxFiles; /* max file table size currently allocated */ + +#if HAVE_INOTIFY_INIT +/* support for inotify mode */ + +/* we need to track directories */ +struct dirInfoFiles_s { /* associated files */ + int idx; + int refcnt; /* due to inotify's async nature, we may have multiple + * references to a single file inside our cache - e.g. when + * inodes are removed, and the file name is re-created BUT another + * process (like rsyslogd ;)) holds open the old inode. + */ +}; +typedef struct dirInfoFiles_s dirInfoFiles_t; + +struct dirInfo_s { + uchar *dirName; + dirInfoFiles_t *files; /* associated file entries */ + int currMaxFiles; + int allocMaxFiles; +}; +typedef struct dirInfo_s dirInfo_t; +static dirInfo_t *dirs = NULL; +static int allocMaxDirs; +static int currMaxDirs; + +/* We need to map watch descriptors to our actual objects. Unfortunately, the + * inotify API does not provide us with any cookie, so a simple O(1) algorithm + * cannot be done (what a shame...). We assume that maintaining the array is much + * less often done than looking it up, so we keep the array sorted by watch descriptor + * and do a binary search on the wd we get back. This is at least O(log n), which + * is not too bad for the anticipated use case. + */ +struct wd_map_s { + int wd; /* ascending sort key */ + int fileIdx; /* -1, if this is a dir entry, otherwise index into files table */ + int dirIdx; /* index into dirs table, undefined if fileIdx != -1 */ +}; +typedef struct wd_map_s wd_map_t; +static wd_map_t *wdmap = NULL; +static int nWdmap; +static int allocMaxWdmap; +static int ino_fd; /* fd for inotify calls */ + +#endif /* #if HAVE_INOTIFY_INIT -------------------------------------------------- */ static prop_t *pInputName = NULL; /* there is only one global inputName for all messages generated by this input */ /* module-global parameters */ static struct cnfparamdescr modpdescr[] = { - { "pollinginterval", eCmdHdlrPositiveInt, 0 } + { "pollinginterval", eCmdHdlrPositiveInt, 0 }, + { "mode", eCmdHdlrGetWord, 0 } }; static struct cnfparamblk modpblk = { CNFPARAMBLK_VERSION, @@ -156,6 +226,7 @@ static struct cnfparamdescr inppdescr[] = { { "facility", eCmdHdlrFacility, 0 }, { "ruleset", eCmdHdlrString, 0 }, { "readmode", eCmdHdlrInt, 0 }, + { "escapelf", eCmdHdlrBinary, 0 }, { "maxlinesatonce", eCmdHdlrInt, 0 }, { "maxsubmitatonce", eCmdHdlrInt, 0 }, { "persiststateinterval", eCmdHdlrInt, 0 } @@ -168,6 +239,123 @@ static struct cnfparamblk inppblk = #include "im-helper.h" /* must be included AFTER the type definitions! */ + +#if HAVE_INOTIFY_INIT +/* support for inotify mode */ + +#if 0 /* enable if you need this for debugging */ +static void +dbg_wdmapPrint(char *msg) +{ + int i; + dbgprintf("%s\n", msg); + for(i = 0 ; i < nWdmap ; ++i) + dbgprintf("wdmap[%d]: wd: %d, file %d, dir %d\n", i, + wdmap[i].wd, wdmap[i].fileIdx, wdmap[i].dirIdx); +} +#endif + +static inline rsRetVal +wdmapInit(void) +{ + DEFiRet; + free(wdmap); + CHKmalloc(wdmap = malloc(sizeof(wd_map_t) * INIT_WDMAP_TAB_SIZE)); + allocMaxWdmap = INIT_WDMAP_TAB_SIZE; + nWdmap = 0; +finalize_it: + RETiRet; +} + + +/* compare function for bsearch() */ +static int +wdmap_cmp(const void *k, const void *a) +{ + int key = *((int*) k); + wd_map_t *etry = (wd_map_t*) a; + if(key < etry->wd) + return -1; + else if(key > etry->wd) + return 1; + else + return 0; +} +/* looks up a wdmap entry and returns it's index if found + * or -1 if not found. + */ +static wd_map_t * +wdmapLookup(int wd) +{ + return bsearch(&wd, wdmap, nWdmap, sizeof(wd_map_t), wdmap_cmp); +} + +/* note: we search backwards, as inotify tends to return increasing wd's */ +static rsRetVal +wdmapAdd(int wd, int dirIdx, int fileIdx) +{ + wd_map_t *newmap; + int newmapsize; + int i; + DEFiRet; + + for(i = nWdmap-1 ; i >= 0 && wdmap[i].wd > wd ; --i) + ; /* just scan */ + if(i >= 0 && wdmap[i].wd == wd) { + DBGPRINTF("imfile: wd %d already in wdmap!\n", wd); + FINALIZE; + } + ++i; + /* i now points to the entry that is to be moved upwards (or end of map) */ + if(nWdmap == allocMaxWdmap) { + newmapsize = 2 * allocMaxWdmap; + CHKmalloc(newmap = realloc(wdmap, sizeof(wd_map_t) * newmapsize)); + // TODO: handle the error more intelligently? At all possible? -- 2013-10-15 + wdmap = newmap; + allocMaxWdmap = newmapsize; + } + if(i < nWdmap) { + /* we need to shift to make room for new entry */ + dbgprintf("DDDD: imfile doing wdmap mmemmov(%d, %d, %d) for ADD\n", i,i+1,nWdmap-i); + memmove(wdmap + i, wdmap + i + 1, nWdmap - i); + } + wdmap[i].wd = wd; + wdmap[i].dirIdx = dirIdx; + wdmap[i].fileIdx = fileIdx; + ++nWdmap; + dbgprintf("DDDD: imfile: enter into wdmap[%d]: wd %d, dir %d, file %d\n",i,wd,dirIdx,fileIdx); + +finalize_it: + RETiRet; +} + +static rsRetVal +wdmapDel(int wd) +{ + int i; + DEFiRet; + + for(i = 0 ; i < nWdmap && wdmap[i].wd < wd ; ++i) + ; /* just scan */ + if(i == nWdmap || wdmap[i].wd != wd) { + DBGPRINTF("imfile: wd %d shall be deleted but not in wdmap!\n", wd); + FINALIZE; + } + if(i < nWdmap-1) { + /* we need to shift to delete it (see comment at wdmap definition) */ + dbgprintf("DDDD: imfile doing wdmap mmemmov(%d, %d, %d) for DEL\n", i,i+1,nWdmap-i-1); + memmove(wdmap + i, wdmap + i+1, nWdmap - i-1); + } + --nWdmap; + dbgprintf("DDDD: imfile: wd %d deleted, was idx %d\n", wd, i); + +finalize_it: + RETiRet; +} + +#endif /* #if HAVE_INOTIFY_INIT */ + + /* enqueue the read file line as a message. The provided string is * not freed - thuis must be done by the caller. */ @@ -188,8 +376,8 @@ static rsRetVal enqLine(fileInfo_t *pInfo, cstr_t *cstrLine) MsgSetMSGoffs(pMsg, 0); /* we do not have a header... */ MsgSetHOSTNAME(pMsg, glbl.GetLocalHostName(), ustrlen(glbl.GetLocalHostName())); MsgSetTAG(pMsg, pInfo->pszTag, pInfo->lenTag); - pMsg->iFacility = LOG_FAC(pInfo->iFacility); - pMsg->iSeverity = LOG_PRI(pInfo->iSeverity); + pMsg->iFacility = pri2fac(pInfo->iFacility); + pMsg->iSeverity = pri2sev(pInfo->iSeverity); MsgSetRuleset(pMsg, pInfo->pRuleset); ratelimitAddMsg(pInfo->ratelimiter, &pInfo->multiSub, pMsg); finalize_it: @@ -234,6 +422,18 @@ openFile(fileInfo_t *pThis) /* read back in the object */ CHKiRet(obj.Deserialize(&pThis->pStrm, (uchar*) "strm", psSF, NULL, pThis)); + DBGPRINTF("imfile: deserialized state file, state file base name '%s', " + "configured base name '%s'\n", pThis->pStrm->pszFName, + pThis->pszFileName); + if(ustrcmp(pThis->pStrm->pszFName, pThis->pszFileName)) { + errmsg.LogError(0, RS_RET_STATEFILE_WRONG_FNAME, "imfile: state file '%s' " + "contains file name '%s', but is used for file '%s'. State " + "file deleted, starting from begin of file.", + pszSFNam, pThis->pStrm->pszFName, pThis->pszFileName); + + unlink((char*)pszSFNam); + ABORT_FINALIZE(RS_RET_STATEFILE_WRONG_FNAME); + } strm.CheckFileChange(pThis->pStrm); CHKiRet(strm.SeekCurrOffs(pThis->pStrm)); @@ -281,8 +481,6 @@ static rsRetVal pollFile(fileInfo_t *pThis, int *pbHadFileData) int nProcessed = 0; DEFiRet; - ASSERT(pbHadFileData != NULL); - /* Note: we must do pthread_cleanup_push() immediately, because the POXIS macros * otherwise do not work if I include the _cleanup_pop() inside an if... -- rgerhards, 2008-08-14 */ @@ -295,9 +493,10 @@ static rsRetVal pollFile(fileInfo_t *pThis, int *pbHadFileData) while(glbl.GetGlobalInputTermState() == 0) { if(pThis->maxLinesAtOnce != 0 && nProcessed >= pThis->maxLinesAtOnce) break; - CHKiRet(strm.ReadLine(pThis->pStrm, &pCStr, pThis->readMode)); + CHKiRet(strm.ReadLine(pThis->pStrm, &pCStr, pThis->readMode, pThis->escapeLF)); ++nProcessed; - *pbHadFileData = 1; /* this is just a flag, so set it and forget it */ + if(pbHadFileData != NULL) + *pbHadFileData = 1; /* this is just a flag, so set it and forget it */ CHKiRet(enqLine(pThis, pCStr)); /* process line */ rsCStrDestruct(&pCStr); /* discard string (must be done by us!) */ if(pThis->iPersistStateInterval > 0 && pThis->nRecords++ >= pThis->iPersistStateInterval) { @@ -341,6 +540,7 @@ createInstance(instanceConf_t **pinst) inst->maxLinesAtOnce = 10240; inst->iPersistStateInterval = 0; inst->readMode = 0; + inst->escapeLF = 1; /* node created, let's add to config */ if(loadModConf->tail == NULL) { @@ -356,6 +556,65 @@ finalize_it: } +/* this function checks instance parameters and does some required pre-processing + * (e.g. split filename in path and actual name) + * Note: we do NOT use dirname()/basename() as they have portability problems. + */ +static rsRetVal +checkInstance(instanceConf_t *inst) +{ + char dirn[MAXFNAME]; + char basen[MAXFNAME]; + int i; + int lenName; + struct stat sb; + int r; + int eno; + char errStr[512]; + DEFiRet; + + lenName = ustrlen(inst->pszFileName); + for(i = lenName ; i >= 0 ; --i) { + if(inst->pszFileName[i] == '/') { + /* found basename component */ + if(i == lenName) + basen[0] = '\0'; + else { + memcpy(basen, inst->pszFileName+i+1, lenName-i); + /* Note \0 is copied above! */ + //basen[(lenName-i+1)+1] = '\0'; + } + break; + } + } + memcpy(dirn, inst->pszFileName, i); /* do not copy slash */ + dirn[i] = '\0'; + CHKmalloc(inst->pszFileBaseName = (uchar*) strdup(basen)); + CHKmalloc(inst->pszDirName = (uchar*) strdup(dirn)); + + if(dirn[0] == '\0') { + dirn[0] = '/'; + dirn[1] = '\0'; + } + r = stat(dirn, &sb); + if(r != 0) { + eno = errno; + rs_strerror_r(eno, errStr, sizeof(errStr)); + errmsg.LogError(0, RS_RET_CONFIG_ERROR, "imfile warning: directory '%s': %s", + dirn, errStr); + ABORT_FINALIZE(RS_RET_CONFIG_ERROR); + } + if(!S_ISDIR(sb.st_mode)) { + errmsg.LogError(0, RS_RET_CONFIG_ERROR, "imfile warning: configured directory " + "'%s' is NOT a directory", dirn); + ABORT_FINALIZE(RS_RET_CONFIG_ERROR); + } + +finalize_it: + RETiRet; +} + + /* add a new monitor */ static rsRetVal addInstance(void __attribute__((unused)) *pVal, uchar *pNewVal) { @@ -389,6 +648,9 @@ static rsRetVal addInstance(void __attribute__((unused)) *pVal, uchar *pNewVal) inst->maxLinesAtOnce = cs.maxLinesAtOnce; inst->iPersistStateInterval = cs.iPersistStateInterval; inst->readMode = cs.readMode; + inst->escapeLF = 0; + + CHKiRet(checkInstance(inst)); /* reset legacy system */ cs.iPersistStateInterval = 0; @@ -405,33 +667,46 @@ static inline rsRetVal addListner(instanceConf_t *inst) { DEFiRet; + int newMax; + fileInfo_t *newFileTab; fileInfo_t *pThis; - if(iFilPtr < MAX_INPUT_FILES) { - pThis = &files[iFilPtr]; - //TODO: optimize, save strdup? - pThis->pszFileName = (uchar*) strdup((char*) inst->pszFileName); - pThis->pszTag = (uchar*) strdup((char*) inst->pszTag); - pThis->lenTag = ustrlen(pThis->pszTag); - pThis->pszStateFile = (uchar*) strdup((char*) inst->pszStateFile); - - CHKiRet(ratelimitNew(&pThis->ratelimiter, "imfile", (char*)inst->pszFileName)); - CHKmalloc(pThis->multiSub.ppMsgs = MALLOC(inst->nMultiSub * sizeof(msg_t*))); - pThis->multiSub.maxElem = inst->nMultiSub; - pThis->multiSub.nElem = 0; - pThis->iSeverity = inst->iSeverity; - pThis->iFacility = inst->iFacility; - pThis->maxLinesAtOnce = inst->maxLinesAtOnce; - pThis->iPersistStateInterval = inst->iPersistStateInterval; - pThis->readMode = inst->readMode; - pThis->pRuleset = inst->pBindRuleset; - pThis->nRecords = 0; - } else { - errmsg.LogError(0, RS_RET_OUT_OF_DESRIPTORS, - "Too many file monitors configured - ignoring %s", - inst->pszFileName); - ABORT_FINALIZE(RS_RET_OUT_OF_DESRIPTORS); + if(iFilPtr == allocMaxFiles) { + newMax = 2 * allocMaxFiles; + newFileTab = realloc(files, newMax * sizeof(fileInfo_t)); + if(newFileTab == NULL) { + errmsg.LogError(0, RS_RET_OUT_OF_MEMORY, + "cannot alloc memory to monitor file '%s' - ignoring", + inst->pszFileName); + ABORT_FINALIZE(RS_RET_OUT_OF_MEMORY); + } + files = newFileTab; + allocMaxFiles = newMax; + DBGPRINTF("imfile: increased file table to %d entries\n", allocMaxFiles); } + + /* if we reach this point, there is space in the file table for the new entry */ + pThis = &files[iFilPtr]; + pThis->pszFileName = (uchar*) strdup((char*) inst->pszFileName); + pThis->pszDirName = inst->pszDirName; /* use value from inst! */ + pThis->pszBaseName = inst->pszFileBaseName; /* use value from inst! */ + pThis->pszTag = (uchar*) strdup((char*) inst->pszTag); + pThis->lenTag = ustrlen(pThis->pszTag); + pThis->pszStateFile = (uchar*) strdup((char*) inst->pszStateFile); + + CHKiRet(ratelimitNew(&pThis->ratelimiter, "imfile", (char*)inst->pszFileName)); + CHKmalloc(pThis->multiSub.ppMsgs = MALLOC(inst->nMultiSub * sizeof(msg_t*))); + pThis->multiSub.maxElem = inst->nMultiSub; + pThis->multiSub.nElem = 0; + pThis->iSeverity = inst->iSeverity; + pThis->iFacility = inst->iFacility; + pThis->maxLinesAtOnce = inst->maxLinesAtOnce; + pThis->iPersistStateInterval = inst->iPersistStateInterval; + pThis->readMode = inst->readMode; + pThis->escapeLF = inst->escapeLF; + pThis->pRuleset = inst->pBindRuleset; + pThis->nRecords = 0; + pThis->pStrm = NULL; ++iFilPtr; /* we got a new file to monitor */ resetConfigVariables(NULL, NULL); /* values are both dummies */ @@ -449,8 +724,6 @@ CODESTARTnewInpInst pvals = nvlstGetParams(lst, &inppblk, NULL); if(pvals == NULL) { - errmsg.LogError(0, RS_RET_MISSING_CNFPARAMS, - "imfile: required parameter are missing\n"); ABORT_FINALIZE(RS_RET_MISSING_CNFPARAMS); } @@ -475,12 +748,14 @@ CODESTARTnewInpInst } else if(!strcmp(inppblk.descr[i].name, "severity")) { inst->iSeverity = pvals[i].val.d.n; } else if(!strcmp(inppblk.descr[i].name, "facility")) { - inst->iSeverity = pvals[i].val.d.n; + inst->iFacility = pvals[i].val.d.n; } else if(!strcmp(inppblk.descr[i].name, "readmode")) { - inst->readMode = pvals[i].val.d.n; + inst->readMode = (uint8_t) pvals[i].val.d.n; + } else if(!strcmp(inppblk.descr[i].name, "escapelf")) { + inst->escapeLF = (sbool) pvals[i].val.d.n; } else if(!strcmp(inppblk.descr[i].name, "maxlinesatonce")) { inst->maxLinesAtOnce = pvals[i].val.d.n; - } else if(!strcmp(inppblk.descr[i].name, "persistStateInterval")) { + } else if(!strcmp(inppblk.descr[i].name, "persiststateinterval")) { inst->iPersistStateInterval = pvals[i].val.d.n; } else if(!strcmp(inppblk.descr[i].name, "maxsubmitatonce")) { inst->nMultiSub = pvals[i].val.d.n; @@ -489,6 +764,7 @@ CODESTARTnewInpInst "param '%s'\n", inppblk.descr[i].name); } } + CHKiRet(checkInstance(inst)); finalize_it: CODE_STD_FINALIZERnewInpInst cnfparamvalsDestruct(pvals, &inppblk); @@ -499,6 +775,7 @@ CODESTARTbeginCnfLoad loadModConf = pModConf; pModConf->pConf = pConf; /* init our settings */ + loadModConf->opMode = OPMODE_POLLING; loadModConf->iPollInterval = DFLT_PollInterval; loadModConf->configSetViaV2Method = 0; bLegacyCnfModGlobalsPermitted = 1; @@ -520,6 +797,7 @@ BEGINsetModCnf struct cnfparamvals *pvals = NULL; int i; CODESTARTsetModCnf + loadModConf->opMode = OPMODE_INOTIFY; /* new style config has different default! */ pvals = nvlstGetParams(lst, &modpblk, NULL); if(pvals == NULL) { errmsg.LogError(0, RS_RET_MISSING_CNFPARAMS, "imfile: error processing module " @@ -537,6 +815,17 @@ CODESTARTsetModCnf continue; if(!strcmp(modpblk.descr[i].name, "pollinginterval")) { loadModConf->iPollInterval = (int) pvals[i].val.d.n; + } else if(!strcmp(modpblk.descr[i].name, "mode")) { + if(!es_strconstcmp(pvals[i].val.d.estr, "polling")) + loadModConf->opMode = OPMODE_POLLING; + else if(!es_strconstcmp(pvals[i].val.d.estr, "inotify")) + loadModConf->opMode = OPMODE_INOTIFY; + else { + char *cstr = es_str2cstr(pvals[i].val.d.estr, NULL); + errmsg.LogError(0, RS_RET_PARAM_ERROR, "imfile: unknown " + "mode '%s'", cstr); + free(cstr); + } } else { dbgprintf("imfile: program error, non-handled " "param '%s' in beginCnfLoad\n", modpblk.descr[i].name); @@ -561,7 +850,9 @@ CODESTARTendCnfLoad /* persist module-specific settings from legacy config system */ loadModConf->iPollInterval = cs.iPollInterval; } - dbgprintf("imfile: polling interval is %d\n", loadModConf->iPollInterval); + dbgprintf("imfile: opmode is %d, polling interval is %d\n", + loadModConf->opMode, + loadModConf->iPollInterval); loadModConf = NULL; /* done loading */ /* free legacy config vars */ @@ -593,10 +884,16 @@ BEGINactivateCnf instanceConf_t *inst; CODESTARTactivateCnf runModConf = pModConf; + free(files); /* clear any previous instance */ + CHKmalloc(files = (fileInfo_t*) malloc(sizeof(fileInfo_t) * INIT_FILE_TAB_SIZE)); + allocMaxFiles = INIT_FILE_TAB_SIZE; + iFilPtr = 0; + for(inst = runModConf->root ; inst != NULL ; inst = inst->next) { addListner(inst); } - /* if we could not set up any listners, there is no point in running... */ + + /* if we could not set up any listeners, there is no point in running... */ if(iFilPtr == 0) { errmsg.LogError(0, NO_ERRCODE, "imfile: no file monitors could be started, " "input not activated.\n"); @@ -612,39 +909,19 @@ CODESTARTfreeCnf for(inst = pModConf->root ; inst != NULL ; ) { free(inst->pszBindRuleset); free(inst->pszFileName); + free(inst->pszDirName); + free(inst->pszFileBaseName); free(inst->pszTag); free(inst->pszStateFile); del = inst; inst = inst->next; free(del); } + free(files); ENDfreeCnf - -/* This function is the cancel cleanup handler. It is called when rsyslog decides the - * module must be stopped, what most probably happens during shutdown of rsyslogd. When - * this function is called, the runInput() function (below) is already terminated - somewhere - * in the middle of what it was doing. The cancel cleanup handler below should take - * care of any locked mutexes and such, things that really need to be cleaned up - * before processing continues. In general, many plugins do not need to provide - * any code at all here. - * - * IMPORTANT: the calling interface of this function can NOT be modified. It actually is - * called by pthreads. The provided argument is currently not being used. - */ -static void -inputModuleCleanup(void __attribute__((unused)) *arg) -{ - BEGINfunc - ENDfunc -} - - -/* This function is called by the framework to gather the input. The module stays - * most of its lifetime inside this function. It MUST NEVER exit this function. Doing - * so would end module processing and rsyslog would NOT reschedule the module. If - * you exit from this function, you violate the interface specification! +/* Monitor files in traditional polling mode. * * We go through all files and remember if at least one had data. If so, we do * another run (until no data was present in any file). Then we sleep for @@ -661,12 +938,12 @@ inputModuleCleanup(void __attribute__((unused)) *arg) * On spamming the main queue: keep in mind that it will automatically rate-limit * ourselfes if we begin to overrun it. So we really do not need to care here. */ -#pragma GCC diagnostic ignored "-Wempty-body" -BEGINrunInput +static rsRetVal +doPolling(void) +{ int i; int bHadFileData; /* were there at least one file with data during this run? */ -CODESTARTrunInput - pthread_cleanup_push(inputModuleCleanup, NULL); + DEFiRet; while(glbl.GetGlobalInputTermState() == 0) { do { bHadFileData = 0; @@ -675,24 +952,421 @@ CODESTARTrunInput break; /* terminate input! */ pollFile(&files[i], &bHadFileData); } - } while(iFilPtr > 1 && bHadFileData == 1 && glbl.GetGlobalInputTermState() == 0); /* warning: do...while()! */ + } while(iFilPtr > 1 && bHadFileData == 1 && glbl.GetGlobalInputTermState() == 0); + /* warning: do...while()! */ - /* Note: the additional 10ns wait is vitally important. It guards rsyslog against totally - * hogging the CPU if the users selects a polling interval of 0 seconds. It doesn't hurt any - * other valid scenario. So do not remove. -- rgerhards, 2008-02-14 + /* Note: the additional 10ns wait is vitally important. It guards rsyslog + * against totally hogging the CPU if the users selects a polling interval + * of 0 seconds. It doesn't hurt any other valid scenario. So do not remove. + * rgerhards, 2008-02-14 */ if(glbl.GetGlobalInputTermState() == 0) srSleep(runModConf->iPollInterval, 10); } - DBGPRINTF("imfile: terminating upon request of rsyslog core\n"); - pthread_cleanup_pop(0); /* just for completeness, but never called... */ - RETiRet; /* use it to make sure the housekeeping is done! */ -ENDrunInput -#pragma GCC diagnostic warning "-Wempty-body" - /* END no-touch zone * - * ------------------------------------------------------------------------------------------ */ + RETiRet; +} + + +#if HAVE_INOTIFY_INIT +/* add entry to dirs array */ +static rsRetVal +dirsAdd(uchar *dirName) +{ + int newMax; + dirInfo_t *newDirTab; + DEFiRet; + if(currMaxDirs == allocMaxDirs) { + newMax = 2 * allocMaxDirs; + newDirTab = realloc(dirs, newMax * sizeof(dirInfo_t)); + if(newDirTab == NULL) { + errmsg.LogError(0, RS_RET_OUT_OF_MEMORY, + "cannot alloc memory to monitor directory '%s' - ignoring", + dirName); + } + dirs = newDirTab; + allocMaxDirs = newMax; + DBGPRINTF("imfile: increased dir table to %d entries\n", allocMaxDirs); + } + + /* if we reach this point, there is space in the file table for the new entry */ + dirs[currMaxDirs].dirName = dirName; + CHKmalloc(dirs[currMaxDirs].files= malloc(sizeof(dirInfoFiles_t) * INIT_FILE_IN_DIR_TAB_SIZE)); + dirs[currMaxDirs].allocMaxFiles = INIT_FILE_IN_DIR_TAB_SIZE; + dirs[currMaxDirs].currMaxFiles= 0; + + ++currMaxDirs; +finalize_it: + RETiRet; +} + +/* checks if a file name is already inside the dirs array. Note that wildcards + * apply. Returns either the array index or -1 if not found. + * i is the index of the dir entry to search. + */ +static int +dirsFindFile(int i, uchar *fn) +{ + int f; + uchar *baseName; + + for(f = 0 ; f < dirs[i].currMaxFiles ; ++f) { + baseName = files[dirs[i].files[f].idx].pszBaseName; + if(!fnmatch((char*)fn, (char*)baseName, FNM_PATHNAME | FNM_PERIOD)) + break; /* found */ + } + if(f == dirs[i].currMaxFiles) + f = -1; + //dbgprintf("DDDD: dir '%s', file '%s', found:%d\n", dirs[i].dirName, fn, f); + return f; +} + +/* checks if a dir name is already inside the dirs array. If so, returns + * its index. If not present, -1 is returned. + */ +static int +dirsFindDir(uchar *dir) +{ + int i; + + for(i = 0 ; i < currMaxDirs && ustrcmp(dir, dirs[i].dirName) ; ++i) + ; /* just scan, all done in for() */ + if(i == currMaxDirs) + i = -1; + //dbgprintf("DDDD: dir '%s', found:%d\n", dir, i); + return i; +} + +static rsRetVal +dirsInit(void) +{ + instanceConf_t *inst; + DEFiRet; + + free(dirs); + CHKmalloc(dirs = malloc(sizeof(dirInfo_t) * INIT_FILE_TAB_SIZE)); + allocMaxDirs = INIT_FILE_TAB_SIZE; + currMaxDirs = 0; + + for(inst = runModConf->root ; inst != NULL ; inst = inst->next) { + if(dirsFindDir(inst->pszDirName) == -1) + dirsAdd(inst->pszDirName); + } + +finalize_it: + RETiRet; +} + +/* add file to directory (create association) + * i is index into file table, all other information is pulled from that table. + */ +static rsRetVal +dirsAddFile(int i) +{ + int dirIdx; + int j; + int newMax; + dirInfoFiles_t *newFileTab; + dirInfo_t *dir; + DEFiRet; + + dirIdx = dirsFindDir(files[i].pszDirName); + if(dirIdx == -1) { + errmsg.LogError(0, RS_RET_INTERNAL_ERROR, "imfile: could not find " + "directory '%s' in dirs array - ignoring", + files[i].pszDirName); + FINALIZE; + } + + dir = dirs + dirIdx; + for(j = 0 ; j < dir->currMaxFiles && dir->files[j].idx != i ; ++j) + ; /* just scan */ + if(j < dir->currMaxFiles) { + /* this is not important enough to send an user error, as all will + * continue to work. */ + ++dir->files[j].refcnt; + DBGPRINTF("imfile: file '%s' already registered in directory '%s', recnt now %d\n", + files[i].pszFileName, dir->dirName, dir->files[j].refcnt); + FINALIZE; + } + + if(dir->currMaxFiles == dir->allocMaxFiles) { + newMax = 2 * allocMaxFiles; + newFileTab = realloc(dirs, newMax * sizeof(dirInfoFiles_t)); + if(newFileTab == NULL) { + errmsg.LogError(0, RS_RET_OUT_OF_MEMORY, + "cannot alloc memory to map directory '%s' file relationship " + "'%s' - ignoring", files[i].pszFileName, dir->dirName); + ABORT_FINALIZE(RS_RET_OUT_OF_MEMORY); + } + dir->files = newFileTab; + dir->allocMaxFiles = newMax; + DBGPRINTF("imfile: increased dir table to %d entries\n", allocMaxDirs); + } + + dir->files[dir->currMaxFiles].idx = i; + dir->files[dir->currMaxFiles].refcnt = 1; + dbgprintf("DDDD: associated file %d[%s] to directory %d[%s]\n", + i, files[i].pszFileName, dirIdx, dir->dirName); + ++dir->currMaxFiles; +finalize_it: + RETiRet; +} + +/* delete a file from directory (remove association) + * fIdx is index into file table, all other information is pulled from that table. + */ +static rsRetVal +dirsDelFile(int fIdx) +{ + int dirIdx; + int j; + dirInfo_t *dir; + DEFiRet; + + dirIdx = dirsFindDir(files[fIdx].pszDirName); + if(dirIdx == -1) { + DBGPRINTF("imfile: could not find directory '%s' in dirs array - ignoring", + files[fIdx].pszDirName); + FINALIZE; + } + + dir = dirs + dirIdx; + for(j = 0 ; j < dir->currMaxFiles && dir->files[j].idx != fIdx ; ++j) + ; /* just scan */ + if(j == dir->currMaxFiles) { + DBGPRINTF("imfile: no association for file '%s' in directory '%s' " + "found - ignoring\n", files[fIdx].pszFileName, dir->dirName); + FINALIZE; + } + dir->files[j].refcnt--; + if(dir->files[j].refcnt == 0) { + /* we remove that entry (but we never shrink the table) */ + if(j < dir->currMaxFiles - 1) { + /* entry in middle - need to move others */ + memmove(dir->files+j, dir->files+j+1, + (dir->currMaxFiles -j-1) * sizeof(dirInfoFiles_t)); + } + --dir->currMaxFiles; + } + DBGPRINTF("imfile: removed association of file '%s' to directory '%s'\n", + files[fIdx].pszFileName, dir->dirName); + +finalize_it: + RETiRet; +} + +static void +in_setupDirWatch(int i) +{ + int wd; + wd = inotify_add_watch(ino_fd, (char*)dirs[i].dirName, IN_CREATE); + if(wd < 0) { + DBGPRINTF("imfile: could not create dir watch for '%s'\n", + files[i].pszFileName); + goto done; + } + wdmapAdd(wd, i, -1); + dbgprintf("DDDD: watch %d added for dir %s\n", wd, dirs[i].dirName); +done: return; +} + +/* Setup a new file watch. + * Note: we need to try to read this file, as it may already contain data this + * needs to be processed, and we won't get an event for that as notifications + * happen only for things after the watch has been activated. + */ +static void +in_setupFileWatch(int i) +{ + int wd; + wd = inotify_add_watch(ino_fd, (char*)files[i].pszFileName, IN_MODIFY); + if(wd < 0) { + DBGPRINTF("imfile: could not create initial file for '%s'\n", + files[i].pszFileName); + goto done; + } + wdmapAdd(wd, -1, i); + dbgprintf("DDDD: watch %d added for file %s\n", wd, files[i].pszFileName); + dirsAddFile(i); + pollFile(&files[i], NULL); +done: return; +} + +/* setup our initial set of watches, based on user config */ +static rsRetVal +in_setupInitialWatches() +{ + int i; + DEFiRet; + + for(i = 0 ; i < currMaxDirs ; ++i) { + in_setupDirWatch(i); + } + for(i = 0 ; i < iFilPtr ; ++i) { + in_setupFileWatch(i); + } + RETiRet; +} + +static void +in_dbg_showEv(struct inotify_event *ev) +{ + if(ev->mask & IN_IGNORED) { + dbgprintf("watch was REMOVED\n"); + } else if(ev->mask & IN_MODIFY) { + dbgprintf("watch was MODIFID\n"); + } else if(ev->mask & IN_ACCESS) { + dbgprintf("watch IN_ACCESS\n"); + } else if(ev->mask & IN_ATTRIB) { + dbgprintf("watch IN_ATTRIB\n"); + } else if(ev->mask & IN_CLOSE_WRITE) { + dbgprintf("watch IN_CLOSE_WRITE\n"); + } else if(ev->mask & IN_CLOSE_NOWRITE) { + dbgprintf("watch IN_CLOSE_NOWRITE\n"); + } else if(ev->mask & IN_CREATE) { + dbgprintf("file was CREATED: %s\n", ev->name); + } else if(ev->mask & IN_DELETE) { + dbgprintf("watch IN_DELETE\n"); + } else if(ev->mask & IN_DELETE_SELF) { + dbgprintf("watch IN_DELETE_SELF\n"); + } else if(ev->mask & IN_MOVE_SELF) { + dbgprintf("watch IN_MOVE_SELF\n"); + } else if(ev->mask & IN_MOVED_FROM) { + dbgprintf("watch IN_MOVED_FROM\n"); + } else if(ev->mask & IN_MOVED_TO) { + dbgprintf("watch IN_MOVED_TO\n"); + } else if(ev->mask & IN_OPEN) { + dbgprintf("watch IN_OPEN\n"); + } else if(ev->mask & IN_ISDIR) { + dbgprintf("watch IN_ISDIR\n"); + } else { + dbgprintf("unknown mask code %8.8x\n", ev->mask); + } +} + +static void +in_handleDirEvent(struct inotify_event *ev, int dirIdx) +{ + int fileIdx; + dbgprintf("DDDD: handle dir event for %s\n", dirs[dirIdx].dirName); + if(!(ev->mask & IN_CREATE)) { + DBGPRINTF("imfile: got non-expected inotify event:\n"); + in_dbg_showEv(ev); + goto done; + } + fileIdx = dirsFindFile(dirIdx, (uchar*)ev->name); + if(fileIdx == -1) { + dbgprintf("imfile: file '%s' not associated with dir '%s'\n", + ev->name, dirs[dirIdx].dirName); + goto done; + } + dbgprintf("DDDD: file '%s' associated with dir '%s'\n", ev->name, dirs[dirIdx].dirName); + in_setupFileWatch(fileIdx); +done: return; +} + +/* inotify told us that a file's wd was closed. We now need to remove + * the file from our internal structures. Remember that a different inode + * with the same name may already be in processing. + */ +static void +in_removeFile(struct inotify_event *ev, int fIdx) +{ + wdmapDel(ev->wd); + dirsDelFile(fIdx); +} + + +static void +in_handleFileEvent(struct inotify_event *ev, int fIdx) +{ + if(ev->mask & IN_MODIFY) { + pollFile(&files[fIdx], NULL); + } else if(ev->mask & IN_IGNORED) { + in_removeFile(ev, fIdx); + } else { + DBGPRINTF("imfile: got non-expected inotify event:\n"); + in_dbg_showEv(ev); + } +} + +static void +in_processEvent(struct inotify_event *ev) +{ + wd_map_t *etry; + + etry = wdmapLookup(ev->wd); + if(etry == NULL) { + DBGPRINTF("imfile: could not lookup wd %d\n", ev->wd); + goto done; + } + dbgprintf("DDDD: imfile: wd %d got file %d, dir %d\n", ev->wd, etry->fileIdx, etry->dirIdx); + if(etry->fileIdx == -1) { /* directory? */ + in_handleDirEvent(ev, etry->dirIdx); + } else { + in_handleFileEvent(ev, etry->fileIdx); + } +done: return; +} + +/* Monitor files in inotify mode */ +static rsRetVal +do_inotify() +{ + char iobuf[8192]; + struct inotify_event *ev; + int rd; + int currev; + DEFiRet; + + CHKiRet(wdmapInit()); + CHKiRet(dirsInit()); + ino_fd = inotify_init(); + DBGPRINTF("imfile: inotify fd %d\n", ino_fd); + CHKiRet(in_setupInitialWatches()); + + while(glbl.GetGlobalInputTermState() == 0) { + rd = read(ino_fd, iobuf, sizeof(iobuf)); + if(rd < 0) { + perror("inotify read"); exit(1); + } + currev = 0; + while(currev < rd) { + ev = (struct inotify_event*) (iobuf+currev); + dbgprintf("DDDD: imfile event notification: rd %d[%d], wd (%d, mask " + "%8.8x, cookie %4.4x, len %d)\n", + (int) rd, currev, ev->wd, ev->mask, ev->cookie, ev->len); + in_dbg_showEv(ev); + in_processEvent(ev); + currev += sizeof(struct inotify_event) + ev->len; + } + } + +finalize_it: + close(ino_fd); + RETiRet; +} + +#endif /* #if HAVE_INOTIFY_INIT */ + +/* This function is called by the framework to gather the input. The module stays + * most of its lifetime inside this function. It MUST NEVER exit this function. Doing + * so would end module processing and rsyslog would NOT reschedule the module. If + * you exit from this function, you violate the interface specification! + */ +BEGINrunInput +CODESTARTrunInput + DBGPRINTF("imfile: working in %s mode\n", + (runModConf->opMode == OPMODE_POLLING) ? "polling" : "inotify"); + if(runModConf->opMode == OPMODE_POLLING) + iRet = doPolling(); + else + iRet = do_inotify(); + + DBGPRINTF("imfile: terminating upon request of rsyslog core\n"); +ENDrunInput /* The function is called by rsyslog before runInput() is called. It is a last chance @@ -834,8 +1508,8 @@ resetConfigVariables(uchar __attribute__((unused)) *pp, void __attribute__((unus cs.pszFileName = NULL; free(cs.pszFileTag); cs.pszFileTag = NULL; - free(cs.pszFileTag); - cs.pszFileTag = NULL; + free(cs.pszStateFile); + cs.pszStateFile = NULL; /* set defaults... */ cs.iPollInterval = DFLT_PollInterval; @@ -862,8 +1536,7 @@ std_checkRuleset_genErrMsg(__attribute__((unused)) modConfData_t *modConf, insta * complexity of processing is depending on the actual module. However, only * thing absolutely necessary should be done here. Actual app-level processing * is to be performed in runInput(). A good sample of what to do here may be to - * set some variable defaults. The most important thing probably is registration - * of config command handlers. + * set some variable defaults. */ BEGINmodInit() CODESTARTmodInit diff --git a/plugins/imgssapi/Makefile.in b/plugins/imgssapi/Makefile.in index 5061bde..09070d1 100644 --- a/plugins/imgssapi/Makefile.in +++ b/plugins/imgssapi/Makefile.in @@ -154,7 +154,6 @@ GREP = @GREP@ GSS_LIBS = @GSS_LIBS@ GUARDTIME_CFLAGS = @GUARDTIME_CFLAGS@ GUARDTIME_LIBS = @GUARDTIME_LIBS@ -HAVE_LIBGCRYPT_CONFIG = @HAVE_LIBGCRYPT_CONFIG@ HAVE_MYSQL_CONFIG = @HAVE_MYSQL_CONFIG@ HAVE_ORACLE_CONFIG = @HAVE_ORACLE_CONFIG@ HAVE_PGSQL_CONFIG = @HAVE_PGSQL_CONFIG@ @@ -175,14 +174,15 @@ LEXLIB = @LEXLIB@ LEX_OUTPUT_ROOT = @LEX_OUTPUT_ROOT@ LIBDBI_CFLAGS = @LIBDBI_CFLAGS@ LIBDBI_LIBS = @LIBDBI_LIBS@ -LIBEE_CFLAGS = @LIBEE_CFLAGS@ -LIBEE_LIBS = @LIBEE_LIBS@ LIBESTR_CFLAGS = @LIBESTR_CFLAGS@ LIBESTR_LIBS = @LIBESTR_LIBS@ LIBGCRYPT_CFLAGS = @LIBGCRYPT_CFLAGS@ +LIBGCRYPT_CONFIG = @LIBGCRYPT_CONFIG@ LIBGCRYPT_LIBS = @LIBGCRYPT_LIBS@ LIBLOGGING_CFLAGS = @LIBLOGGING_CFLAGS@ LIBLOGGING_LIBS = @LIBLOGGING_LIBS@ +LIBLOGGING_STDLOG_CFLAGS = @LIBLOGGING_STDLOG_CFLAGS@ +LIBLOGGING_STDLOG_LIBS = @LIBLOGGING_STDLOG_LIBS@ LIBLOGNORM_CFLAGS = @LIBLOGNORM_CFLAGS@ LIBLOGNORM_LIBS = @LIBLOGNORM_LIBS@ LIBM = @LIBM@ @@ -207,6 +207,8 @@ NM = @NM@ NMEDIT = @NMEDIT@ OBJDUMP = @OBJDUMP@ OBJEXT = @OBJEXT@ +OPENSSL_CFLAGS = @OPENSSL_CFLAGS@ +OPENSSL_LIBS = @OPENSSL_LIBS@ ORACLE_CFLAGS = @ORACLE_CFLAGS@ ORACLE_LIBS = @ORACLE_LIBS@ OTOOL = @OTOOL@ @@ -387,22 +389,25 @@ distclean-compile: @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/imgssapi_la-imgssapi.Plo@am__quote@ .c.o: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(COMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ $< -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Po +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.o$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c $< +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ $< .c.obj: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(COMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ `$(CYGPATH_W) '$<'` -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Po +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.obj$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ `$(CYGPATH_W) '$<'` &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c `$(CYGPATH_W) '$<'` +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ `$(CYGPATH_W) '$<'` .c.lo: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(LTCOMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ $< -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Plo +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.lo$$||'`;\ +@am__fastdepCC_TRUE@ $(LTCOMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Plo @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=yes @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ @am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(LTCOMPILE) -c -o $@ $< diff --git a/plugins/imgssapi/imgssapi.c b/plugins/imgssapi/imgssapi.c index 4e3a70a..15d994c 100644 --- a/plugins/imgssapi/imgssapi.c +++ b/plugins/imgssapi/imgssapi.c @@ -72,6 +72,7 @@ MODULE_CNFNAME("imgssapi") /* some forward definitions - they may go away when we no longer include imtcp.c */ static rsRetVal addGSSListener(void __attribute__((unused)) *pVal, uchar *pNewVal); +static rsRetVal actGSSListener(uchar *port); static int TCPSessGSSInit(void); static void TCPSessGSSClose(tcps_sess_t* pSess); static rsRetVal TCPSessGSSRecv(tcps_sess_t *pSess, void *buf, size_t buf_len, ssize_t *); @@ -90,6 +91,7 @@ DEFobjCurrIf(glbl) static tcpsrv_t *pOurTcpsrv = NULL; /* our TCP server(listener) TODO: change for multiple instances */ static gss_cred_id_t gss_server_creds = GSS_C_NO_CREDENTIAL; +static uchar *srvPort; /* our usr structure for the tcpsrv object */ typedef struct gsssrv_s { @@ -317,6 +319,16 @@ static rsRetVal addGSSListener(void __attribute__((unused)) *pVal, uchar *pNewVal) { DEFiRet; + + srvPort = pNewVal; + + RETiRet; +} + +static rsRetVal +actGSSListener(uchar *port) +{ + DEFiRet; gsssrv_t *pGSrv; if(pOurTcpsrv == NULL) { @@ -340,7 +352,7 @@ addGSSListener(void __attribute__((unused)) *pVal, uchar *pNewVal) CHKiRet(tcpsrv.SetCBOnRegularClose(pOurTcpsrv, onRegularClose)); CHKiRet(tcpsrv.SetCBOnErrClose(pOurTcpsrv, onErrClose)); CHKiRet(tcpsrv.SetInputName(pOurTcpsrv, UCHAR_CONSTANT("imgssapi"))); - tcpsrv.configureTCPListen(pOurTcpsrv, pNewVal, 1); + tcpsrv.configureTCPListen(pOurTcpsrv, port, 1); CHKiRet(tcpsrv.ConstructFinalize(pOurTcpsrv)); } @@ -676,6 +688,11 @@ ENDfreeCnf */ BEGINrunInput CODESTARTrunInput + /* This will fail if the priviledges are dropped. Should be + * moved to the '*activateCnfPrePrivDrop' section eventually. + */ + actGSSListener(srvPort); + iRet = tcpsrv.Run(pOurTcpsrv); ENDrunInput @@ -683,7 +700,7 @@ ENDrunInput /* initialize and return if will run or not */ BEGINwillRun CODESTARTwillRun - if(pOurTcpsrv == NULL) + if(srvPort == NULL) ABORT_FINALIZE(RS_RET_NO_RUN); net.PrintAllowedSenders(2); /* TCP */ diff --git a/plugins/imjournal/Makefile.in b/plugins/imjournal/Makefile.in index d181e2e..5644fa5 100644 --- a/plugins/imjournal/Makefile.in +++ b/plugins/imjournal/Makefile.in @@ -154,7 +154,6 @@ GREP = @GREP@ GSS_LIBS = @GSS_LIBS@ GUARDTIME_CFLAGS = @GUARDTIME_CFLAGS@ GUARDTIME_LIBS = @GUARDTIME_LIBS@ -HAVE_LIBGCRYPT_CONFIG = @HAVE_LIBGCRYPT_CONFIG@ HAVE_MYSQL_CONFIG = @HAVE_MYSQL_CONFIG@ HAVE_ORACLE_CONFIG = @HAVE_ORACLE_CONFIG@ HAVE_PGSQL_CONFIG = @HAVE_PGSQL_CONFIG@ @@ -175,14 +174,15 @@ LEXLIB = @LEXLIB@ LEX_OUTPUT_ROOT = @LEX_OUTPUT_ROOT@ LIBDBI_CFLAGS = @LIBDBI_CFLAGS@ LIBDBI_LIBS = @LIBDBI_LIBS@ -LIBEE_CFLAGS = @LIBEE_CFLAGS@ -LIBEE_LIBS = @LIBEE_LIBS@ LIBESTR_CFLAGS = @LIBESTR_CFLAGS@ LIBESTR_LIBS = @LIBESTR_LIBS@ LIBGCRYPT_CFLAGS = @LIBGCRYPT_CFLAGS@ +LIBGCRYPT_CONFIG = @LIBGCRYPT_CONFIG@ LIBGCRYPT_LIBS = @LIBGCRYPT_LIBS@ LIBLOGGING_CFLAGS = @LIBLOGGING_CFLAGS@ LIBLOGGING_LIBS = @LIBLOGGING_LIBS@ +LIBLOGGING_STDLOG_CFLAGS = @LIBLOGGING_STDLOG_CFLAGS@ +LIBLOGGING_STDLOG_LIBS = @LIBLOGGING_STDLOG_LIBS@ LIBLOGNORM_CFLAGS = @LIBLOGNORM_CFLAGS@ LIBLOGNORM_LIBS = @LIBLOGNORM_LIBS@ LIBM = @LIBM@ @@ -207,6 +207,8 @@ NM = @NM@ NMEDIT = @NMEDIT@ OBJDUMP = @OBJDUMP@ OBJEXT = @OBJEXT@ +OPENSSL_CFLAGS = @OPENSSL_CFLAGS@ +OPENSSL_LIBS = @OPENSSL_LIBS@ ORACLE_CFLAGS = @ORACLE_CFLAGS@ ORACLE_LIBS = @ORACLE_LIBS@ OTOOL = @OTOOL@ @@ -388,22 +390,25 @@ distclean-compile: @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/imjournal_la-imjournal.Plo@am__quote@ .c.o: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(COMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ $< -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Po +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.o$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c $< +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ $< .c.obj: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(COMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ `$(CYGPATH_W) '$<'` -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Po +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.obj$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ `$(CYGPATH_W) '$<'` &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c `$(CYGPATH_W) '$<'` +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ `$(CYGPATH_W) '$<'` .c.lo: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(LTCOMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ $< -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Plo +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.lo$$||'`;\ +@am__fastdepCC_TRUE@ $(LTCOMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Plo @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=yes @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ @am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(LTCOMPILE) -c -o $@ $< diff --git a/plugins/imjournal/imjournal.c b/plugins/imjournal/imjournal.c index 2af1958..6a97b25 100644 --- a/plugins/imjournal/imjournal.c +++ b/plugins/imjournal/imjournal.c @@ -3,7 +3,7 @@ * To test under Linux: * emmit log message into systemd journal * - * Copyright (C) 2008-2012 Adiscon GmbH + * Copyright (C) 2008-2013 Adiscon GmbH * * This file is part of rsyslog. * @@ -30,7 +30,10 @@ #include <ctype.h> #include <stdlib.h> #include <time.h> +#include <sys/poll.h> #include <sys/socket.h> +#include <errno.h> +#include <systemd/sd-journal.h> #include "dirty.h" #include "cfsysline.h" @@ -43,8 +46,9 @@ #include "glbl.h" #include "prop.h" #include "errmsg.h" +#include "srUtils.h" #include "unicode-helper.h" -#include <systemd/sd-journal.h> +#include "ratelimit.h" MODULE_TYPE_INPUT MODULE_TYPE_NOKEEP @@ -61,12 +65,24 @@ DEFobjCurrIf(errmsg) static struct configSettings_s { char *stateFile; int iPersistStateInterval; + int ratelimitInterval; + int ratelimitBurst; + int bIgnorePrevious; + int iDfltSeverity; + int iDfltFacility; } cs; -/* module-gloval parameters */ +static rsRetVal facilityHdlr(uchar **pp, void *pVal); + +/* module-global parameters */ static struct cnfparamdescr modpdescr[] = { { "statefile", eCmdHdlrGetWord, 0 }, - { "persiststateinterval", eCmdHdlrInt, 0 } + { "ratelimit.interval", eCmdHdlrInt, 0 }, + { "ratelimit.burst", eCmdHdlrInt, 0 }, + { "persiststateinterval", eCmdHdlrInt, 0 }, + { "ignorepreviousmessages", eCmdHdlrBinary, 0 }, + { "defaultseverity", eCmdHdlrSeverity, 0 }, + { "defaultfacility", eCmdHdlrString, 0 } }; static struct cnfparamblk modpblk = { CNFPARAMBLK_VERSION, @@ -75,14 +91,51 @@ static struct cnfparamblk modpblk = }; #define DFLT_persiststateinterval 10 +#define DFLT_SEVERITY pr2fac(LOG_NOTICE) +#define DFLT_FACILITY pri2sev(LOG_USER) -static int bLegacyCnfModGlobalsPermitted = 0;/* are legacy module-global config parameters permitted? */ +static int bLegacyCnfModGlobalsPermitted = 1;/* are legacy module-global config parameters permitted? */ static prop_t *pInputName = NULL; /* there is only one global inputName for all messages generated by this module */ static prop_t *pLocalHostIP = NULL; /* a pseudo-constant propterty for 127.0.0.1 */ +static ratelimit_t *ratelimiter = NULL; static sd_journal *j; + +/* ugly workaround to handle facility numbers; values + * derived from names need to be eight times smaller, + * i.e.: 0..23 + */ +static rsRetVal facilityHdlr(uchar **pp, void *pVal) +{ + DEFiRet; + char *p; + + skipWhiteSpace(pp); + p = (char *) *pp; + + if (isdigit((int) *p)) { + *((int *) pVal) = (int) strtol(p, (char **) pp, 10); + } else { + int len; + syslogName_t *c; + + for (len = 0; p[len] && !isspace((int) p[len]); len++) + /* noop */; + for (c = syslogFacNames; c->c_name; c++) { + if (!strncasecmp(p, (char *) c->c_name, len)) { + *((int *) pVal) = pri2fac(c->c_val); + break; + } + } + *pp += len; + } + + RETiRet; +} + + /* enqueue the the journal message into the message queue. * The provided msg string is not freed - thus must be done * by the caller. @@ -118,7 +171,7 @@ enqMsg(uchar *msg, uchar *pszTag, int iFacility, int iSeverity, struct timeval * msgAddJSON(pMsg, (uchar*)"!", json); } - CHKiRet(submitMsg2(pMsg)); + CHKiRet(ratelimitAddMsg(ratelimiter, NULL, pMsg)); finalize_it: RETiRet; @@ -136,16 +189,19 @@ readjournal() { uint64_t timestamp; struct json_object *json = NULL; + int r; /* Information from messages */ char *message; + char *sys_pid; char *sys_iden; char *sys_iden_help; const void *get; + const void *pidget; char *parse; - char *get2; size_t length; + size_t pidlength; const void *equal_sign; struct json_object *jval; @@ -155,57 +211,57 @@ readjournal() { long prefixlen = 0; - int priority = 0; - int facility = 0; - - /* Get next journal message, if there is none, wait a second */ - if (sd_journal_next(j) == 0) { - sleep(1); - iRet = RS_RET_OK; - goto ret; - } + int severity = cs.iDfltSeverity; + int facility = cs.iDfltFacility; /* Get message text */ if (sd_journal_get_data(j, "MESSAGE", &get, &length) < 0) { - logmsgInternal(NO_ERRCODE, LOG_SYSLOG|LOG_INFO, (uchar *)"log message from journal doesn't have MESSAGE", 0); - iRet = RS_RET_OK; - goto ret; - } - message = strndup(get+8, length-8); - if (message == NULL) { - iRet = RS_RET_OUT_OF_MEMORY; - goto ret; + message = strdup(""); + } else { + message = strndup(((const char*)get)+8, length-8); + if (message == NULL) { + iRet = RS_RET_OUT_OF_MEMORY; + goto ret; + } } - /* Get message priority */ + /* Get message severity ("priority" in journald's terminology) */ if (sd_journal_get_data(j, "PRIORITY", &get, &length) >= 0) { - get2 = strndup(get, length); - priority = ((char *)get2)[9] - '0'; - free (get2); + if (length == 10) { + severity = ((char *)get)[9] - '0'; + if (severity < 0 || 7 < severity) { + dbgprintf("The value of the 'PRIORITY' field is " + "out of bounds: %d, resetting\n", severity); + severity = cs.iDfltSeverity; + } + } else { + dbgprintf("The value of the 'PRIORITY' field has an " + "unexpected length: %d\n", length); + } } /* Get syslog facility */ if (sd_journal_get_data(j, "SYSLOG_FACILITY", &get, &length) >= 0) { - get2 = strndup(get, length); - char f = ((char *)get2)[16]; - if (f >= '0' && f <= '9') { - facility += f - '0'; - } - f = ((char *)get2)[17]; - if (f >= '0' && f <= '9') { - facility *= 10; - facility += (f - '0'); + if (length == 17 || length == 18) { + facility = ((char *)get)[16] - '0'; + if (length == 18) { + facility *= 10; + facility += ((char *)get)[17] - '0'; + } + if (facility < 0 || 23 < facility) { + dbgprintf("The value of the 'FACILITY' field is " + "out of bounds: %d, resetting\n", facility); + facility = cs.iDfltFacility; + } + } else { + dbgprintf("The value of the 'FACILITY' field has an " + "unexpected length: %d\n", length); } - free (get2); - } else { - /* message is missing facility -> internal systemd journal msg, drop */ - iRet = RS_RET_OK; - goto free_message; } - /* Get message identifier and add ':' */ + /* Get message identifier, client pid and add ':' */ if (sd_journal_get_data(j, "SYSLOG_IDENTIFIER", &get, &length) >= 0) { - sys_iden = strndup(get+18, length-18); + sys_iden = strndup(((const char*)get)+18, length-18); } else { sys_iden = strdup("journal"); } @@ -214,19 +270,43 @@ readjournal() { goto free_message; } - asprintf(&sys_iden_help, "%s:", sys_iden); - if (sys_iden_help == NULL) { + if (sd_journal_get_data(j, "SYSLOG_PID", &pidget, &pidlength) >= 0) { + sys_pid = strndup(((const char*)pidget)+11, pidlength-11); + if (sys_pid == NULL) { + iRet = RS_RET_OUT_OF_MEMORY; + free (sys_iden); + goto free_message; + } + } else { + sys_pid = NULL; + } + + if (sys_pid) { + r = asprintf(&sys_iden_help, "%s[%s]:", sys_iden, sys_pid); + } else { + r = asprintf(&sys_iden_help, "%s:", sys_iden); + } + + free (sys_iden); + free (sys_pid); + + if (-1 == r) { iRet = RS_RET_OUT_OF_MEMORY; goto finalize_it; } - free (sys_iden); json = json_object_new_object(); SD_JOURNAL_FOREACH_DATA(j, get, l) { /* locate equal sign, this is always present */ equal_sign = memchr(get, '=', l); - assert (equal_sign != NULL); + + /* ... but we know better than to trust the specs */ + if (equal_sign == NULL) { + errmsg.LogError(0, RS_RET_ERR, "SD_JOURNAL_FOREACH_DATA()" + "returned a malformed field (has no '='): '%s'", (char*)get); + continue; /* skip the entry */ + } /* get length of journal data prefix */ prefixlen = ((char *)equal_sign - (char *)get); @@ -296,7 +376,7 @@ readjournal() { prefixlen++; /* remove '=' */ - data = strndup(get + prefixlen, l - prefixlen); + data = strndup(((const char*)get) + prefixlen, l - prefixlen); if (data == NULL) { iRet = RS_RET_OUT_OF_MEMORY; free (name); @@ -317,7 +397,7 @@ readjournal() { } /* submit message */ - enqMsg((uchar *)message, (uchar *) sys_iden_help, facility, priority, &tv, json); + enqMsg((uchar *)message, (uchar *) sys_iden_help, facility, severity, &tv, json); finalize_it: free(sys_iden_help); @@ -337,7 +417,9 @@ persistJournalState () { char *cursor; int ret = 0; - if ((ret = sd_journal_get_cursor(j, &cursor)) > 0) { + /* On success, sd_journal_get_cursor() returns 1 in systemd + 197 or older and 0 in systemd 198 or newer */ + if ((ret = sd_journal_get_cursor(j, &cursor)) >= 0) { if ((sf = fopen(cs.stateFile, "wb")) != NULL) { if (fprintf(sf, "%s", cursor) < 0) { iRet = RS_RET_IO_ERROR; @@ -345,29 +427,93 @@ persistJournalState () { fclose(sf); free(cursor); } else { + char errStr[256]; + rs_strerror_r(errno, errStr, sizeof(errStr)); + errmsg.LogError(0, RS_RET_FOPEN_FAILURE, "fopen() failed: " + "'%s', path: '%s'\n", errStr, cs.stateFile); iRet = RS_RET_FOPEN_FAILURE; } } else { + char errStr[256]; + rs_strerror_r(-(ret), errStr, sizeof(errStr)); + errmsg.LogError(0, RS_RET_ERR, "sd_journal_get_cursor() failed: '%s'\n", errStr); iRet = RS_RET_ERR; } RETiRet; } -BEGINrunInput -CODESTARTrunInput - /* this is an endless loop - it is terminated when the thread is - * signalled to do so. This, however, is handled by the framework, - * right into the sleep below. - */ - int count = 0; +/* Polls the journal for new messages. Similar to sd_journal_wait() + * except for the special handling of EINTR. + */ +static rsRetVal +pollJournal() +{ + DEFiRet; + struct pollfd pollfd; + int r; + + pollfd.fd = sd_journal_get_fd(j); + pollfd.events = sd_journal_get_events(j); + r = poll(&pollfd, 1, -1); + if (r == -1) { + if (errno == EINTR) { + /* EINTR is also received during termination + * so return now to check the term state. + */ + ABORT_FINALIZE(RS_RET_OK); + } else { + char errStr[256]; + + rs_strerror_r(errno, errStr, sizeof(errStr)); + errmsg.LogError(0, RS_RET_ERR, + "poll() failed: '%s'", errStr); + ABORT_FINALIZE(RS_RET_ERR); + } + } + + assert(r == 1); + + r = sd_journal_process(j); + if (r < 0) { + char errStr[256]; + + rs_strerror_r(errno, errStr, sizeof(errStr)); + errmsg.LogError(0, RS_RET_ERR, + "sd_journal_process() failed: '%s'", errStr); + ABORT_FINALIZE(RS_RET_ERR); + } + +finalize_it: + RETiRet; +} + + +/* This function loads a journal cursor from the state file. + */ +static rsRetVal +loadJournalState() +{ + DEFiRet; - char readCursor[128 + 1]; - FILE *r_sf; + if (cs.stateFile[0] != '/') { + char *new_stateFile; + + if (-1 == asprintf(&new_stateFile, "%s/%s", (char *)glbl.GetWorkDir(), cs.stateFile)) { + errmsg.LogError(0, RS_RET_OUT_OF_MEMORY, "imjournal: asprintf failed\n"); + ABORT_FINALIZE(RS_RET_OUT_OF_MEMORY); + } + free (cs.stateFile); + cs.stateFile = new_stateFile; + } /* if state file exists, set cursor to appropriate position */ if (access(cs.stateFile, F_OK|R_OK) != -1) { + FILE *r_sf; + if ((r_sf = fopen(cs.stateFile, "rb")) != NULL) { + char readCursor[128 + 1]; + if (fscanf(r_sf, "%128s\n", readCursor) != EOF) { if (sd_journal_seek_cursor(j, readCursor) != 0) { errmsg.LogError(0, RS_RET_ERR, "imjournal: " @@ -387,17 +533,77 @@ CODESTARTrunInput errmsg.LogError(0, RS_RET_FOPEN_FAILURE, "imjournal: " "open on state file `%s' failed\n", cs.stateFile); } + } else { + /* when IgnorePrevious, seek to the end of journal */ + if (cs.bIgnorePrevious) { + if (sd_journal_seek_tail(j) < 0) { + char errStr[256]; + + rs_strerror_r(errno, errStr, sizeof(errStr)); + errmsg.LogError(0, RS_RET_ERR, + "sd_journal_seek_tail() failed: '%s'", errStr); + ABORT_FINALIZE(RS_RET_ERR); + } + + if (sd_journal_previous(j) < 0) { + char errStr[256]; + + rs_strerror_r(errno, errStr, sizeof(errStr)); + errmsg.LogError(0, RS_RET_ERR, + "sd_journal_previous() failed: '%s'", errStr); + ABORT_FINALIZE(RS_RET_ERR); + } + } + } + +finalize_it: + RETiRet; +} + +BEGINrunInput +CODESTARTrunInput + CHKiRet(ratelimitNew(&ratelimiter, "imjournal", NULL)); + dbgprintf("imjournal: ratelimiting burst %d, interval %d\n", cs.ratelimitBurst, + cs.ratelimitInterval); + ratelimitSetLinuxLike(ratelimiter, cs.ratelimitInterval, cs.ratelimitBurst); + ratelimitSetNoTimeCache(ratelimiter); + + if (cs.stateFile) { + CHKiRet(loadJournalState()); } + /* this is an endless loop - it is terminated when the thread is + * signalled to do so. This, however, is handled by the framework. + */ while (glbl.GetGlobalInputTermState() == 0) { + int count = 0, r; + + r = sd_journal_next(j); + if (r < 0) { + char errStr[256]; + + rs_strerror_r(errno, errStr, sizeof(errStr)); + errmsg.LogError(0, RS_RET_ERR, + "sd_journal_next() failed: '%s'", errStr); + ABORT_FINALIZE(RS_RET_ERR); + } + + if (r == 0) { + /* No new messages, wait for activity. */ + CHKiRet(pollJournal()); + continue; + } + CHKiRet(readjournal()); - count++; - if (count == cs.iPersistStateInterval) { - count = 0; - persistJournalState(); + if (cs.stateFile) { /* can't persist without a state file */ + /* TODO: This could use some finer metric. */ + count++; + if (count == cs.iPersistStateInterval) { + count = 0; + persistJournalState(); + } } } - persistJournalState(); finalize_it: ENDrunInput @@ -409,6 +615,10 @@ CODESTARTbeginCnfLoad cs.iPersistStateInterval = DFLT_persiststateinterval; cs.stateFile = NULL; + cs.ratelimitBurst = 20000; + cs.ratelimitInterval = 600; + cs.iDfltSeverity = DFLT_SEVERITY; + cs.iDfltFacility = DFLT_FACILITY; ENDbeginCnfLoad @@ -444,7 +654,11 @@ ENDwillRun /* close journal */ BEGINafterRun CODESTARTafterRun + if (cs.stateFile) { /* can't persist without a state file */ + persistJournalState(); + } sd_journal_close(j); + ratelimitDestruct(ratelimiter); ENDafterRun @@ -487,6 +701,23 @@ CODESTARTsetModCnf cs.iPersistStateInterval = (int) pvals[i].val.d.n; } else if (!strcmp(modpblk.descr[i].name, "statefile")) { cs.stateFile = (char *)es_str2cstr(pvals[i].val.d.estr, NULL); + } else if(!strcmp(modpblk.descr[i].name, "ratelimit.burst")) { + cs.ratelimitBurst = (int) pvals[i].val.d.n; + } else if(!strcmp(modpblk.descr[i].name, "ratelimit.interval")) { + cs.ratelimitInterval = (int) pvals[i].val.d.n; + } else if (!strcmp(modpblk.descr[i].name, "ignorepreviousmessages")) { + cs.bIgnorePrevious = (int) pvals[i].val.d.n; + } else if (!strcmp(modpblk.descr[i].name, "defaultseverity")) { + cs.iDfltSeverity = (int) pvals[i].val.d.n; + } else if (!strcmp(modpblk.descr[i].name, "defaultfacility")) { + /* ugly workaround to handle facility numbers; values + derived from names need to be eight times smaller */ + + char *fac, *p; + + fac = p = es_str2cstr(pvals[i].val.d.estr, NULL); + facilityHdlr((uchar **) &p, (void *) &cs.iDfltFacility); + free(fac); } else { dbgprintf("imjournal: program error, non-handled " "param '%s' in beginCnfLoad\n", modpblk.descr[i].name); @@ -500,16 +731,22 @@ finalize_it: ENDsetModCnf +BEGINisCompatibleWithFeature +CODESTARTisCompatibleWithFeature + if(eFeat == sFEATURENonCancelInputTermination) + iRet = RS_RET_OK; +ENDisCompatibleWithFeature + + BEGINqueryEtryPt CODESTARTqueryEtryPt CODEqueryEtryPt_STD_IMOD_QUERIES CODEqueryEtryPt_STD_CONF2_QUERIES CODEqueryEtryPt_STD_CONF2_setModCnf_QUERIES +CODEqueryEtryPt_IsCompatibleWithFeature_IF_OMOD_QUERIES ENDqueryEtryPt - - BEGINmodInit() CODESTARTmodInit *ipIFVersProvided = CURR_MOD_IF_VERSION; /* we only support the current interface specification */ @@ -524,10 +761,20 @@ CODEmodInit_QueryRegCFSLineHdlr CHKiRet(prop.CreateStringProp(&pInputName, UCHAR_CONSTANT("imjournal"), sizeof("imjournal") - 1)); CHKiRet(prop.CreateStringProp(&pLocalHostIP, UCHAR_CONSTANT("127.0.0.1"), sizeof("127.0.0.1") - 1)); - CHKiRet(omsdRegCFSLineHdlr((uchar *)"persiststateinterval", 0, eCmdHdlrInt, + CHKiRet(omsdRegCFSLineHdlr((uchar *)"imjournalpersiststateinterval", 0, eCmdHdlrInt, NULL, &cs.iPersistStateInterval, STD_LOADABLE_MODULE_ID)); - CHKiRet(omsdRegCFSLineHdlr((uchar *)"statefile", 0, eCmdHdlrGetWord, + CHKiRet(omsdRegCFSLineHdlr((uchar *)"imjournalratelimitinterval", 0, eCmdHdlrInt, + NULL, &cs.ratelimitInterval, STD_LOADABLE_MODULE_ID)); + CHKiRet(omsdRegCFSLineHdlr((uchar *)"imjournalratelimitburst", 0, eCmdHdlrInt, + NULL, &cs.ratelimitBurst, STD_LOADABLE_MODULE_ID)); + CHKiRet(omsdRegCFSLineHdlr((uchar *)"imjournalstatefile", 0, eCmdHdlrGetWord, NULL, &cs.stateFile, STD_LOADABLE_MODULE_ID)); + CHKiRet(omsdRegCFSLineHdlr((uchar *)"imjournalignorepreviousmessages", 0, eCmdHdlrBinary, + NULL, &cs.bIgnorePrevious, STD_LOADABLE_MODULE_ID)); + CHKiRet(omsdRegCFSLineHdlr((uchar *)"imjournaldefaultseverity", 0, eCmdHdlrSeverity, + NULL, &cs.iDfltSeverity, STD_LOADABLE_MODULE_ID)); + CHKiRet(omsdRegCFSLineHdlr((uchar *)"imjournaldefaultfacility", 0, eCmdHdlrCustomHandler, + facilityHdlr, &cs.iDfltFacility, STD_LOADABLE_MODULE_ID)); ENDmodInit diff --git a/plugins/imklog/Makefile.in b/plugins/imklog/Makefile.in index 6e3d358..297787e 100644 --- a/plugins/imklog/Makefile.in +++ b/plugins/imklog/Makefile.in @@ -161,7 +161,6 @@ GREP = @GREP@ GSS_LIBS = @GSS_LIBS@ GUARDTIME_CFLAGS = @GUARDTIME_CFLAGS@ GUARDTIME_LIBS = @GUARDTIME_LIBS@ -HAVE_LIBGCRYPT_CONFIG = @HAVE_LIBGCRYPT_CONFIG@ HAVE_MYSQL_CONFIG = @HAVE_MYSQL_CONFIG@ HAVE_ORACLE_CONFIG = @HAVE_ORACLE_CONFIG@ HAVE_PGSQL_CONFIG = @HAVE_PGSQL_CONFIG@ @@ -182,14 +181,15 @@ LEXLIB = @LEXLIB@ LEX_OUTPUT_ROOT = @LEX_OUTPUT_ROOT@ LIBDBI_CFLAGS = @LIBDBI_CFLAGS@ LIBDBI_LIBS = @LIBDBI_LIBS@ -LIBEE_CFLAGS = @LIBEE_CFLAGS@ -LIBEE_LIBS = @LIBEE_LIBS@ LIBESTR_CFLAGS = @LIBESTR_CFLAGS@ LIBESTR_LIBS = @LIBESTR_LIBS@ LIBGCRYPT_CFLAGS = @LIBGCRYPT_CFLAGS@ +LIBGCRYPT_CONFIG = @LIBGCRYPT_CONFIG@ LIBGCRYPT_LIBS = @LIBGCRYPT_LIBS@ LIBLOGGING_CFLAGS = @LIBLOGGING_CFLAGS@ LIBLOGGING_LIBS = @LIBLOGGING_LIBS@ +LIBLOGGING_STDLOG_CFLAGS = @LIBLOGGING_STDLOG_CFLAGS@ +LIBLOGGING_STDLOG_LIBS = @LIBLOGGING_STDLOG_LIBS@ LIBLOGNORM_CFLAGS = @LIBLOGNORM_CFLAGS@ LIBLOGNORM_LIBS = @LIBLOGNORM_LIBS@ LIBM = @LIBM@ @@ -214,6 +214,8 @@ NM = @NM@ NMEDIT = @NMEDIT@ OBJDUMP = @OBJDUMP@ OBJEXT = @OBJEXT@ +OPENSSL_CFLAGS = @OPENSSL_CFLAGS@ +OPENSSL_LIBS = @OPENSSL_LIBS@ ORACLE_CFLAGS = @ORACLE_CFLAGS@ ORACLE_LIBS = @ORACLE_LIBS@ OTOOL = @OTOOL@ @@ -395,22 +397,25 @@ distclean-compile: @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/imklog_la-imklog.Plo@am__quote@ .c.o: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(COMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ $< -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Po +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.o$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c $< +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ $< .c.obj: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(COMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ `$(CYGPATH_W) '$<'` -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Po +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.obj$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ `$(CYGPATH_W) '$<'` &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c `$(CYGPATH_W) '$<'` +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ `$(CYGPATH_W) '$<'` .c.lo: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(LTCOMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ $< -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Plo +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.lo$$||'`;\ +@am__fastdepCC_TRUE@ $(LTCOMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Plo @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=yes @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ @am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(LTCOMPILE) -c -o $@ $< diff --git a/plugins/imklog/bsd.c b/plugins/imklog/bsd.c index 9c2eebb..8dcc230 100644 --- a/plugins/imklog/bsd.c +++ b/plugins/imklog/bsd.c @@ -218,8 +218,10 @@ readklog(modConfData_t *pModConf) if((size_t) iMaxLine < sizeof(bufRcv) - 1) { pRcv = bufRcv; } else { - if((pRcv = (uchar*) MALLOC(sizeof(uchar) * (iMaxLine + 1))) == NULL) + if((pRcv = (uchar*) MALLOC(sizeof(uchar) * (iMaxLine + 1))) == NULL) { iMaxLine = sizeof(bufRcv) - 1; /* better this than noting */ + pRcv = bufRcv; + } } len = 0; diff --git a/plugins/imklog/imklog.c b/plugins/imklog/imklog.c index 810ac26..4a76340 100644 --- a/plugins/imklog/imklog.c +++ b/plugins/imklog/imklog.c @@ -21,7 +21,7 @@ * To test under Linux: * echo test1 > /dev/kmsg * - * Copyright (C) 2008-2012 Adiscon GmbH + * Copyright (C) 2008-2014 Adiscon GmbH * * This file is part of rsyslog. * @@ -247,10 +247,10 @@ rsRetVal Syslog(int priority, uchar *pMsg, struct timeval *tp) /* if we don't get the pri, we use whatever we were supplied */ /* ignore non-kernel messages if not permitted */ - if(cs.bPermitNonKernel == 0 && LOG_FAC(priority) != LOG_KERN) + if(cs.bPermitNonKernel == 0 && pri2fac(priority) != LOG_KERN) FINALIZE; /* silently ignore */ - iRet = enqMsg((uchar*)pMsg, (uchar*) "kernel:", LOG_FAC(priority), LOG_PRI(priority), tp); + iRet = enqMsg((uchar*)pMsg, (uchar*) "kernel:", pri2fac(priority), pri2sev(priority), tp); finalize_it: RETiRet; diff --git a/plugins/imkmsg/Makefile.in b/plugins/imkmsg/Makefile.in index a604afc..7e3fa2b 100644 --- a/plugins/imkmsg/Makefile.in +++ b/plugins/imkmsg/Makefile.in @@ -153,7 +153,6 @@ GREP = @GREP@ GSS_LIBS = @GSS_LIBS@ GUARDTIME_CFLAGS = @GUARDTIME_CFLAGS@ GUARDTIME_LIBS = @GUARDTIME_LIBS@ -HAVE_LIBGCRYPT_CONFIG = @HAVE_LIBGCRYPT_CONFIG@ HAVE_MYSQL_CONFIG = @HAVE_MYSQL_CONFIG@ HAVE_ORACLE_CONFIG = @HAVE_ORACLE_CONFIG@ HAVE_PGSQL_CONFIG = @HAVE_PGSQL_CONFIG@ @@ -174,14 +173,15 @@ LEXLIB = @LEXLIB@ LEX_OUTPUT_ROOT = @LEX_OUTPUT_ROOT@ LIBDBI_CFLAGS = @LIBDBI_CFLAGS@ LIBDBI_LIBS = @LIBDBI_LIBS@ -LIBEE_CFLAGS = @LIBEE_CFLAGS@ -LIBEE_LIBS = @LIBEE_LIBS@ LIBESTR_CFLAGS = @LIBESTR_CFLAGS@ LIBESTR_LIBS = @LIBESTR_LIBS@ LIBGCRYPT_CFLAGS = @LIBGCRYPT_CFLAGS@ +LIBGCRYPT_CONFIG = @LIBGCRYPT_CONFIG@ LIBGCRYPT_LIBS = @LIBGCRYPT_LIBS@ LIBLOGGING_CFLAGS = @LIBLOGGING_CFLAGS@ LIBLOGGING_LIBS = @LIBLOGGING_LIBS@ +LIBLOGGING_STDLOG_CFLAGS = @LIBLOGGING_STDLOG_CFLAGS@ +LIBLOGGING_STDLOG_LIBS = @LIBLOGGING_STDLOG_LIBS@ LIBLOGNORM_CFLAGS = @LIBLOGNORM_CFLAGS@ LIBLOGNORM_LIBS = @LIBLOGNORM_LIBS@ LIBM = @LIBM@ @@ -206,6 +206,8 @@ NM = @NM@ NMEDIT = @NMEDIT@ OBJDUMP = @OBJDUMP@ OBJEXT = @OBJEXT@ +OPENSSL_CFLAGS = @OPENSSL_CFLAGS@ +OPENSSL_LIBS = @OPENSSL_LIBS@ ORACLE_CFLAGS = @ORACLE_CFLAGS@ ORACLE_LIBS = @ORACLE_LIBS@ OTOOL = @OTOOL@ @@ -387,22 +389,25 @@ distclean-compile: @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/imkmsg_la-kmsg.Plo@am__quote@ .c.o: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(COMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ $< -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Po +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.o$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c $< +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ $< .c.obj: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(COMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ `$(CYGPATH_W) '$<'` -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Po +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.obj$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ `$(CYGPATH_W) '$<'` &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c `$(CYGPATH_W) '$<'` +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ `$(CYGPATH_W) '$<'` .c.lo: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(LTCOMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ $< -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Plo +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.lo$$||'`;\ +@am__fastdepCC_TRUE@ $(LTCOMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Plo @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=yes @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ @am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(LTCOMPILE) -c -o $@ $< diff --git a/plugins/imkmsg/imkmsg.c b/plugins/imkmsg/imkmsg.c index 2a97f82..8588223 100644 --- a/plugins/imkmsg/imkmsg.c +++ b/plugins/imkmsg/imkmsg.c @@ -144,7 +144,7 @@ rsRetVal imkmsgLogIntMsg(int priority, char *fmt, ...) rsRetVal Syslog(int priority, uchar *pMsg, struct timeval *tp, struct json_object *json) { DEFiRet; - iRet = enqMsg((uchar*)pMsg, (uchar*) "kernel:", LOG_FAC(priority), LOG_PRI(priority), tp, json); + iRet = enqMsg((uchar*)pMsg, (uchar*) "kernel:", pri2fac(priority), pri2sev(priority), tp, json); RETiRet; } diff --git a/plugins/imkmsg/kmsg.c b/plugins/imkmsg/kmsg.c index 822d3db..3d9b78e 100644 --- a/plugins/imkmsg/kmsg.c +++ b/plugins/imkmsg/kmsg.c @@ -34,7 +34,8 @@ #include <ctype.h> #include <sys/klog.h> #include <sys/sysinfo.h> -#include <json/json.h> +#include <sys/time.h> +#include <json.h> #include "rsyslog.h" #include "srUtils.h" diff --git a/plugins/immark/Makefile.in b/plugins/immark/Makefile.in index d4ea9fd..c74c0b7 100644 --- a/plugins/immark/Makefile.in +++ b/plugins/immark/Makefile.in @@ -153,7 +153,6 @@ GREP = @GREP@ GSS_LIBS = @GSS_LIBS@ GUARDTIME_CFLAGS = @GUARDTIME_CFLAGS@ GUARDTIME_LIBS = @GUARDTIME_LIBS@ -HAVE_LIBGCRYPT_CONFIG = @HAVE_LIBGCRYPT_CONFIG@ HAVE_MYSQL_CONFIG = @HAVE_MYSQL_CONFIG@ HAVE_ORACLE_CONFIG = @HAVE_ORACLE_CONFIG@ HAVE_PGSQL_CONFIG = @HAVE_PGSQL_CONFIG@ @@ -174,14 +173,15 @@ LEXLIB = @LEXLIB@ LEX_OUTPUT_ROOT = @LEX_OUTPUT_ROOT@ LIBDBI_CFLAGS = @LIBDBI_CFLAGS@ LIBDBI_LIBS = @LIBDBI_LIBS@ -LIBEE_CFLAGS = @LIBEE_CFLAGS@ -LIBEE_LIBS = @LIBEE_LIBS@ LIBESTR_CFLAGS = @LIBESTR_CFLAGS@ LIBESTR_LIBS = @LIBESTR_LIBS@ LIBGCRYPT_CFLAGS = @LIBGCRYPT_CFLAGS@ +LIBGCRYPT_CONFIG = @LIBGCRYPT_CONFIG@ LIBGCRYPT_LIBS = @LIBGCRYPT_LIBS@ LIBLOGGING_CFLAGS = @LIBLOGGING_CFLAGS@ LIBLOGGING_LIBS = @LIBLOGGING_LIBS@ +LIBLOGGING_STDLOG_CFLAGS = @LIBLOGGING_STDLOG_CFLAGS@ +LIBLOGGING_STDLOG_LIBS = @LIBLOGGING_STDLOG_LIBS@ LIBLOGNORM_CFLAGS = @LIBLOGNORM_CFLAGS@ LIBLOGNORM_LIBS = @LIBLOGNORM_LIBS@ LIBM = @LIBM@ @@ -206,6 +206,8 @@ NM = @NM@ NMEDIT = @NMEDIT@ OBJDUMP = @OBJDUMP@ OBJEXT = @OBJEXT@ +OPENSSL_CFLAGS = @OPENSSL_CFLAGS@ +OPENSSL_LIBS = @OPENSSL_LIBS@ ORACLE_CFLAGS = @ORACLE_CFLAGS@ ORACLE_LIBS = @ORACLE_LIBS@ OTOOL = @OTOOL@ @@ -386,22 +388,25 @@ distclean-compile: @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/immark_la-immark.Plo@am__quote@ .c.o: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(COMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ $< -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Po +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.o$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c $< +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ $< .c.obj: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(COMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ `$(CYGPATH_W) '$<'` -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Po +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.obj$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ `$(CYGPATH_W) '$<'` &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c `$(CYGPATH_W) '$<'` +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ `$(CYGPATH_W) '$<'` .c.lo: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(LTCOMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ $< -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Plo +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.lo$$||'`;\ +@am__fastdepCC_TRUE@ $(LTCOMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Plo @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=yes @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ @am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(LTCOMPILE) -c -o $@ $< diff --git a/plugins/immark/immark.c b/plugins/immark/immark.c index 0e946c0..ec38f4c 100644 --- a/plugins/immark/immark.c +++ b/plugins/immark/immark.c @@ -193,7 +193,7 @@ CODESTARTrunInput break; /* terminate input! */ dbgprintf("immark: injecting mark message\n"); - logmsgInternal(NO_ERRCODE, LOG_INFO, (uchar*)"-- MARK --", MARK); + logmsgInternal(NO_ERRCODE, LOG_SYSLOG|LOG_INFO, (uchar*)"-- MARK --", MARK); } ENDrunInput diff --git a/plugins/impstats/Makefile.in b/plugins/impstats/Makefile.in index 185eb80..c7bb0f7 100644 --- a/plugins/impstats/Makefile.in +++ b/plugins/impstats/Makefile.in @@ -153,7 +153,6 @@ GREP = @GREP@ GSS_LIBS = @GSS_LIBS@ GUARDTIME_CFLAGS = @GUARDTIME_CFLAGS@ GUARDTIME_LIBS = @GUARDTIME_LIBS@ -HAVE_LIBGCRYPT_CONFIG = @HAVE_LIBGCRYPT_CONFIG@ HAVE_MYSQL_CONFIG = @HAVE_MYSQL_CONFIG@ HAVE_ORACLE_CONFIG = @HAVE_ORACLE_CONFIG@ HAVE_PGSQL_CONFIG = @HAVE_PGSQL_CONFIG@ @@ -174,14 +173,15 @@ LEXLIB = @LEXLIB@ LEX_OUTPUT_ROOT = @LEX_OUTPUT_ROOT@ LIBDBI_CFLAGS = @LIBDBI_CFLAGS@ LIBDBI_LIBS = @LIBDBI_LIBS@ -LIBEE_CFLAGS = @LIBEE_CFLAGS@ -LIBEE_LIBS = @LIBEE_LIBS@ LIBESTR_CFLAGS = @LIBESTR_CFLAGS@ LIBESTR_LIBS = @LIBESTR_LIBS@ LIBGCRYPT_CFLAGS = @LIBGCRYPT_CFLAGS@ +LIBGCRYPT_CONFIG = @LIBGCRYPT_CONFIG@ LIBGCRYPT_LIBS = @LIBGCRYPT_LIBS@ LIBLOGGING_CFLAGS = @LIBLOGGING_CFLAGS@ LIBLOGGING_LIBS = @LIBLOGGING_LIBS@ +LIBLOGGING_STDLOG_CFLAGS = @LIBLOGGING_STDLOG_CFLAGS@ +LIBLOGGING_STDLOG_LIBS = @LIBLOGGING_STDLOG_LIBS@ LIBLOGNORM_CFLAGS = @LIBLOGNORM_CFLAGS@ LIBLOGNORM_LIBS = @LIBLOGNORM_LIBS@ LIBM = @LIBM@ @@ -206,6 +206,8 @@ NM = @NM@ NMEDIT = @NMEDIT@ OBJDUMP = @OBJDUMP@ OBJEXT = @OBJEXT@ +OPENSSL_CFLAGS = @OPENSSL_CFLAGS@ +OPENSSL_LIBS = @OPENSSL_LIBS@ ORACLE_CFLAGS = @ORACLE_CFLAGS@ ORACLE_LIBS = @ORACLE_LIBS@ OTOOL = @OTOOL@ @@ -386,22 +388,25 @@ distclean-compile: @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/impstats_la-impstats.Plo@am__quote@ .c.o: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(COMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ $< -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Po +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.o$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c $< +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ $< .c.obj: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(COMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ `$(CYGPATH_W) '$<'` -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Po +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.obj$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ `$(CYGPATH_W) '$<'` &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c `$(CYGPATH_W) '$<'` +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ `$(CYGPATH_W) '$<'` .c.lo: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(LTCOMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ $< -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Plo +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.lo$$||'`;\ +@am__fastdepCC_TRUE@ $(LTCOMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Plo @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=yes @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ @am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(LTCOMPILE) -c -o $@ $< diff --git a/plugins/impstats/impstats.c b/plugins/impstats/impstats.c index cdd205f..ede7c60 100644 --- a/plugins/impstats/impstats.c +++ b/plugins/impstats/impstats.c @@ -1,7 +1,7 @@ /* impstats.c * A module to periodically output statistics gathered by rsyslog. * - * Copyright 2010-2012 Adiscon GmbH. + * Copyright 2010-2013 Adiscon GmbH. * * This file is part of rsyslog. * @@ -29,6 +29,13 @@ #include <pthread.h> #include <fcntl.h> #include <sys/uio.h> +#if defined(__FreeBSD__) +#include <sys/stat.h> +#endif +#include <errno.h> +#include <sys/time.h> +#include <sys/resource.h> + #include "dirty.h" #include "cfsysline.h" #include "module-template.h" @@ -39,6 +46,7 @@ #include "glbl.h" #include "statsobj.h" #include "prop.h" +#include "ruleset.h" MODULE_TYPE_INPUT MODULE_TYPE_NOKEEP @@ -55,6 +63,7 @@ DEFobjCurrIf(glbl) DEFobjCurrIf(prop) DEFobjCurrIf(statsobj) DEFobjCurrIf(errmsg) +DEFobjCurrIf(ruleset) typedef struct configSettings_s { int iStatsInterval; @@ -70,10 +79,14 @@ struct modConfData_s { int iFacility; int iSeverity; int logfd; /* fd if logging to file, or -1 if closed */ + ruleset_t *pBindRuleset; /* ruleset to bind listener to (use system default if unspecified) */ statsFmtType_t statsFmt; sbool bLogToSyslog; + sbool bResetCtrs; + sbool bBracketing; char *logfile; sbool configSetViaV2Method; + uchar *pszBindRuleset; /* name of ruleset to bind to */ }; static modConfData_t *loadModConf = NULL;/* modConf ptr to use for the current load process */ static modConfData_t *runModConf = NULL;/* modConf ptr to use for the current load process */ @@ -87,9 +100,12 @@ static struct cnfparamdescr modpdescr[] = { { "interval", eCmdHdlrInt, 0 }, { "facility", eCmdHdlrInt, 0 }, { "severity", eCmdHdlrInt, 0 }, + { "bracketing", eCmdHdlrBinary, 0 }, { "log.syslog", eCmdHdlrBinary, 0 }, + { "resetcounters", eCmdHdlrBinary, 0 }, { "log.file", eCmdHdlrGetWord, 0 }, - { "format", eCmdHdlrGetWord, 0 } + { "format", eCmdHdlrGetWord, 0 }, + { "ruleset", eCmdHdlrString, 0 } }; static struct cnfparamblk modpblk = { CNFPARAMBLK_VERSION, @@ -97,6 +113,19 @@ static struct cnfparamblk modpblk = modpdescr }; + +/* resource use stats counters */ +static intctr_t st_ru_utime; +static intctr_t st_ru_stime; +static int st_ru_maxrss; +static int st_ru_minflt; +static int st_ru_majflt; +static int st_ru_inblock; +static int st_ru_oublock; +static int st_ru_nvcsw; +static int st_ru_nivcsw; +static statsobj_t *statsobj_resources; + BEGINmodExit CODESTARTmodExit prop.Destruct(&pInputName); @@ -105,6 +134,7 @@ CODESTARTmodExit objRelease(prop, CORE_COMPONENT); objRelease(errmsg, CORE_COMPONENT); objRelease(statsobj, CORE_COMPONENT); + objRelease(ruleset, CORE_COMPONENT); ENDmodExit @@ -131,15 +161,16 @@ static inline void doSubmitMsg(uchar *line) { msg_t *pMsg; - DEFiRet; - CHKiRet(msgConstruct(&pMsg)); + if(msgConstruct(&pMsg) != RS_RET_OK) + goto finalize_it; MsgSetInputName(pMsg, pInputName); MsgSetRawMsgWOSize(pMsg, (char*)line); MsgSetHOSTNAME(pMsg, glbl.GetLocalHostName(), ustrlen(glbl.GetLocalHostName())); MsgSetRcvFrom(pMsg, glbl.GetLocalHostNameProp()); MsgSetRcvFromIP(pMsg, glbl.GetLocalHostIP()); MsgSetMSGoffs(pMsg, 0); + MsgSetRuleset(pMsg, runModConf->pBindRuleset); MsgSetTAG(pMsg, UCHAR_CONSTANT("rsyslogd-pstats:"), sizeof("rsyslogd-pstats:") - 1); pMsg->iFacility = runModConf->iFacility; pMsg->iSeverity = runModConf->iSeverity; @@ -157,7 +188,7 @@ finalize_it: /* log stats message to file; limited error handling done */ static inline void -doLogToFile(cstr_t *cstr) +doLogToFile(uchar *ln, size_t lenLn) { struct iovec iov[4]; ssize_t nwritten; @@ -165,7 +196,7 @@ doLogToFile(cstr_t *cstr) time_t t; char timebuf[32]; - if(cstrLen(cstr) == 0) + if(lenLn == 0) goto done; if(runModConf->logfd == -1) { runModConf->logfd = open(runModConf->logfile, O_WRONLY|O_CREAT|O_APPEND|O_CLOEXEC, S_IRUSR|S_IWUSR); @@ -181,9 +212,9 @@ doLogToFile(cstr_t *cstr) iov[1].iov_base = ": "; iov[1].iov_len = 2; nexpect += 2; - iov[2].iov_base = rsCStrGetSzStrNoNULL(cstr); - iov[2].iov_len = (size_t) cstrLen(cstr); - nexpect += cstrLen(cstr); + iov[2].iov_base = ln; + iov[2].iov_len = lenLn; + nexpect += lenLn; iov[3].iov_base = "\n"; iov[3].iov_len = 1; nexpect++; @@ -197,6 +228,20 @@ done: return; } +/* submit a line to our log destinations. Line must be fully formatted as + * required (but may be a simple verb like "BEGIN" and "END". + */ +static rsRetVal +submitLine(uchar *const ln, const size_t lenLn) +{ + DEFiRet; + if(runModConf->bLogToSyslog) + doSubmitMsg(ln); + if(runModConf->logfile != NULL) + doLogToFile(ln, lenLn); + RETiRet; +} + /* callback for statsobj * Note: usrptr exists only to satisfy requirements of statsobj callback interface! */ @@ -204,10 +249,7 @@ static rsRetVal doStatsLine(void __attribute__((unused)) *usrptr, cstr_t *cstr) { DEFiRet; - if(runModConf->bLogToSyslog) - doSubmitMsg(rsCStrGetSzStrNoNULL(cstr)); - if(runModConf->logfile != NULL) - doLogToFile(cstr); + iRet = submitLine(rsCStrGetSzStrNoNULL(cstr), cstrLen(cstr)); RETiRet; } @@ -218,7 +260,23 @@ doStatsLine(void __attribute__((unused)) *usrptr, cstr_t *cstr) static inline void generateStatsMsgs(void) { - statsobj.GetAllStatsLines(doStatsLine, NULL, runModConf->statsFmt); + struct rusage ru; + int r; + r = getrusage(RUSAGE_SELF, &ru); + if(r != 0) { + dbgprintf("impstats: getrusage() failed with error %d, zeroing out\n", errno); + memset(&ru, 0, sizeof(ru)); + } + st_ru_utime = ru.ru_utime.tv_sec * 1000000 + ru.ru_utime.tv_usec; + st_ru_stime = ru.ru_stime.tv_sec * 1000000 + ru.ru_stime.tv_usec; + st_ru_maxrss = ru.ru_maxrss; + st_ru_minflt = ru.ru_minflt; + st_ru_majflt = ru.ru_majflt; + st_ru_inblock = ru.ru_inblock; + st_ru_oublock = ru.ru_oublock; + st_ru_nvcsw = ru.ru_nvcsw; + st_ru_nivcsw = ru.ru_nivcsw; + statsobj.GetAllStatsLines(doStatsLine, NULL, runModConf->statsFmt, runModConf->bResetCtrs); } @@ -234,7 +292,10 @@ CODESTARTbeginCnfLoad loadModConf->statsFmt = statsFmt_Legacy; loadModConf->logfd = -1; loadModConf->logfile = NULL; + loadModConf->pszBindRuleset = NULL; loadModConf->bLogToSyslog = 1; + loadModConf->bBracketing = 0; + loadModConf->bResetCtrs = 0; bLegacyCnfModGlobalsPermitted = 1; /* init legacy config vars */ initConfigSettings(); @@ -267,8 +328,12 @@ CODESTARTsetModCnf loadModConf->iFacility = (int) pvals[i].val.d.n; } else if(!strcmp(modpblk.descr[i].name, "severity")) { loadModConf->iSeverity = (int) pvals[i].val.d.n; + } else if(!strcmp(modpblk.descr[i].name, "bracketing")) { + loadModConf->bBracketing = (sbool) pvals[i].val.d.n; } else if(!strcmp(modpblk.descr[i].name, "log.syslog")) { loadModConf->bLogToSyslog = (sbool) pvals[i].val.d.n; + } else if(!strcmp(modpblk.descr[i].name, "resetcounters")) { + loadModConf->bResetCtrs = (sbool) pvals[i].val.d.n; } else if(!strcmp(modpblk.descr[i].name, "log.file")) { loadModConf->logfile = es_str2cstr(pvals[i].val.d.estr, NULL); } else if(!strcmp(modpblk.descr[i].name, "format")) { @@ -284,6 +349,8 @@ CODESTARTsetModCnf mode); } free(mode); + } else if(!strcmp(modpblk.descr[i].name, "ruleset")) { + loadModConf->pszBindRuleset = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); } else { dbgprintf("impstats: program error, non-handled " "param '%s' in beginCnfLoad\n", modpblk.descr[i].name); @@ -317,6 +384,30 @@ CODESTARTendCnfLoad ENDendCnfLoad +/* we need our special version of checkRuleset(), as we do not have any instances */ +static inline rsRetVal +checkRuleset(modConfData_t *modConf) +{ + ruleset_t *pRuleset; + rsRetVal localRet; + DEFiRet; + + modConf->pBindRuleset = NULL; /* assume default ruleset */ + + if(modConf->pszBindRuleset == NULL) + FINALIZE; + + localRet = ruleset.GetRuleset(modConf->pConf, &pRuleset, modConf->pszBindRuleset); + if(localRet == RS_RET_NOT_FOUND) { + errmsg.LogError(0, NO_ERRCODE, "impstats: ruleset '%s' not found - " + "using default ruleset instead", modConf->pszBindRuleset); + } + CHKiRet(localRet); + modConf->pBindRuleset = pRuleset; +finalize_it: + RETiRet; +} + BEGINcheckCnf CODESTARTcheckCnf if(pModConf->iStatsInterval == 0) { @@ -324,6 +415,7 @@ CODESTARTcheckCnf "default of %d seconds", DEFAULT_STATS_PERIOD); pModConf->iStatsInterval = DEFAULT_STATS_PERIOD; } + iRet = checkRuleset(pModConf); ENDcheckCnf @@ -331,15 +423,41 @@ BEGINactivateCnf rsRetVal localRet; CODESTARTactivateCnf runModConf = pModConf; - DBGPRINTF("impstats: stats interval %d seconds, logToSyslog %d, logFile %s\n", - runModConf->iStatsInterval, runModConf->bLogToSyslog, + DBGPRINTF("impstats: stats interval %d seconds, reset %d, logToSyslog %d, logFile %s\n", + runModConf->iStatsInterval, runModConf->bResetCtrs, runModConf->bLogToSyslog, runModConf->logfile == NULL ? "deactivated" : (char*)runModConf->logfile); localRet = statsobj.EnableStats(); if(localRet != RS_RET_OK) { errmsg.LogError(0, localRet, "impstats: error enabling statistics gathering"); ABORT_FINALIZE(RS_RET_NO_RUN); } + /* initialize our own counters */ + CHKiRet(statsobj.Construct(&statsobj_resources)); + CHKiRet(statsobj.SetName(statsobj_resources, (uchar*)"resource-usage")); + CHKiRet(statsobj.AddCounter(statsobj_resources, UCHAR_CONSTANT("utime"), + ctrType_IntCtr, CTR_FLAG_NONE, &st_ru_utime)); + CHKiRet(statsobj.AddCounter(statsobj_resources, UCHAR_CONSTANT("stime"), + ctrType_IntCtr, CTR_FLAG_NONE, &st_ru_stime)); + CHKiRet(statsobj.AddCounter(statsobj_resources, UCHAR_CONSTANT("maxrss"), + ctrType_Int, CTR_FLAG_NONE, &st_ru_maxrss)); + CHKiRet(statsobj.AddCounter(statsobj_resources, UCHAR_CONSTANT("minflt"), + ctrType_Int, CTR_FLAG_NONE, &st_ru_minflt)); + CHKiRet(statsobj.AddCounter(statsobj_resources, UCHAR_CONSTANT("majflt"), + ctrType_Int, CTR_FLAG_NONE, &st_ru_majflt)); + CHKiRet(statsobj.AddCounter(statsobj_resources, UCHAR_CONSTANT("inblock"), + ctrType_Int, CTR_FLAG_NONE, &st_ru_inblock)); + CHKiRet(statsobj.AddCounter(statsobj_resources, UCHAR_CONSTANT("oublock"), + ctrType_Int, CTR_FLAG_NONE, &st_ru_oublock)); + CHKiRet(statsobj.AddCounter(statsobj_resources, UCHAR_CONSTANT("nvcsw"), + ctrType_Int, CTR_FLAG_NONE, &st_ru_nvcsw)); + CHKiRet(statsobj.AddCounter(statsobj_resources, UCHAR_CONSTANT("nivcsw"), + ctrType_Int, CTR_FLAG_NONE, &st_ru_nivcsw)); + CHKiRet(statsobj.ConstructFinalize(statsobj_resources)); finalize_it: + if(iRet != RS_RET_OK) { + errmsg.LogError(0, iRet, "impstats: error activating module"); + iRet = RS_RET_NO_RUN; + } ENDactivateCnf @@ -355,16 +473,18 @@ BEGINrunInput CODESTARTrunInput /* this is an endless loop - it is terminated when the thread is * signalled to do so. This, however, is handled by the framework, - * right into the sleep below. + * right into the sleep below. Note that we DELIBERATLY output + * final set of stats counters on termination request. Depending + * on configuration, they may not make it to the final destination... */ - while(1) { + while(glbl.GetGlobalInputTermState() == 0) { srSleep(runModConf->iStatsInterval, 0); /* seconds, micro seconds */ - - if(glbl.GetGlobalInputTermState() == 1) - break; /* terminate input! */ - DBGPRINTF("impstats: woke up, generating messages\n"); + if(runModConf->bBracketing) + submitLine((uchar*)"BEGIN", sizeof("BEGIN")-1); generateStatsMsgs(); + if(runModConf->bBracketing) + submitLine((uchar*)"END", sizeof("END")-1); } ENDrunInput @@ -404,6 +524,7 @@ CODEmodInit_QueryRegCFSLineHdlr CHKiRet(objUse(prop, CORE_COMPONENT)); CHKiRet(objUse(errmsg, CORE_COMPONENT)); CHKiRet(objUse(statsobj, CORE_COMPONENT)); + CHKiRet(objUse(ruleset, CORE_COMPONENT)); /* the pstatsinverval is an alias to support a previous screwed-up syntax... */ CHKiRet(regCfSysLineHdlr2((uchar *)"pstatsinterval", 0, eCmdHdlrInt, NULL, &cs.iStatsInterval, STD_LOADABLE_MODULE_ID, &bLegacyCnfModGlobalsPermitted)); CHKiRet(regCfSysLineHdlr2((uchar *)"pstatinterval", 0, eCmdHdlrInt, NULL, &cs.iStatsInterval, STD_LOADABLE_MODULE_ID, &bLegacyCnfModGlobalsPermitted)); diff --git a/plugins/imptcp/Makefile.in b/plugins/imptcp/Makefile.in index 3e2488a..d34a3f3 100644 --- a/plugins/imptcp/Makefile.in +++ b/plugins/imptcp/Makefile.in @@ -153,7 +153,6 @@ GREP = @GREP@ GSS_LIBS = @GSS_LIBS@ GUARDTIME_CFLAGS = @GUARDTIME_CFLAGS@ GUARDTIME_LIBS = @GUARDTIME_LIBS@ -HAVE_LIBGCRYPT_CONFIG = @HAVE_LIBGCRYPT_CONFIG@ HAVE_MYSQL_CONFIG = @HAVE_MYSQL_CONFIG@ HAVE_ORACLE_CONFIG = @HAVE_ORACLE_CONFIG@ HAVE_PGSQL_CONFIG = @HAVE_PGSQL_CONFIG@ @@ -174,14 +173,15 @@ LEXLIB = @LEXLIB@ LEX_OUTPUT_ROOT = @LEX_OUTPUT_ROOT@ LIBDBI_CFLAGS = @LIBDBI_CFLAGS@ LIBDBI_LIBS = @LIBDBI_LIBS@ -LIBEE_CFLAGS = @LIBEE_CFLAGS@ -LIBEE_LIBS = @LIBEE_LIBS@ LIBESTR_CFLAGS = @LIBESTR_CFLAGS@ LIBESTR_LIBS = @LIBESTR_LIBS@ LIBGCRYPT_CFLAGS = @LIBGCRYPT_CFLAGS@ +LIBGCRYPT_CONFIG = @LIBGCRYPT_CONFIG@ LIBGCRYPT_LIBS = @LIBGCRYPT_LIBS@ LIBLOGGING_CFLAGS = @LIBLOGGING_CFLAGS@ LIBLOGGING_LIBS = @LIBLOGGING_LIBS@ +LIBLOGGING_STDLOG_CFLAGS = @LIBLOGGING_STDLOG_CFLAGS@ +LIBLOGGING_STDLOG_LIBS = @LIBLOGGING_STDLOG_LIBS@ LIBLOGNORM_CFLAGS = @LIBLOGNORM_CFLAGS@ LIBLOGNORM_LIBS = @LIBLOGNORM_LIBS@ LIBM = @LIBM@ @@ -206,6 +206,8 @@ NM = @NM@ NMEDIT = @NMEDIT@ OBJDUMP = @OBJDUMP@ OBJEXT = @OBJEXT@ +OPENSSL_CFLAGS = @OPENSSL_CFLAGS@ +OPENSSL_LIBS = @OPENSSL_LIBS@ ORACLE_CFLAGS = @ORACLE_CFLAGS@ ORACLE_LIBS = @ORACLE_LIBS@ OTOOL = @OTOOL@ @@ -386,22 +388,25 @@ distclean-compile: @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/imptcp_la-imptcp.Plo@am__quote@ .c.o: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(COMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ $< -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Po +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.o$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c $< +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ $< .c.obj: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(COMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ `$(CYGPATH_W) '$<'` -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Po +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.obj$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ `$(CYGPATH_W) '$<'` &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c `$(CYGPATH_W) '$<'` +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ `$(CYGPATH_W) '$<'` .c.lo: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(LTCOMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ $< -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Plo +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.lo$$||'`;\ +@am__fastdepCC_TRUE@ $(LTCOMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Plo @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=yes @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ @am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(LTCOMPILE) -c -o $@ $< diff --git a/plugins/imptcp/imptcp.c b/plugins/imptcp/imptcp.c index 5c8bb67..3025c39 100644 --- a/plugins/imptcp/imptcp.c +++ b/plugins/imptcp/imptcp.c @@ -10,7 +10,7 @@ * * File begun on 2010-08-10 by RGerhards * - * Copyright 2007-2012 Rainer Gerhards and Adiscon GmbH. + * Copyright 2007-2013 Rainer Gerhards and Adiscon GmbH. * * This file is part of rsyslog. * @@ -50,6 +50,8 @@ #include <sys/socket.h> #include <sys/epoll.h> #include <netinet/tcp.h> +#include <stdint.h> +#include <zlib.h> #if HAVE_FCNTL_H #include <fcntl.h> #endif @@ -93,6 +95,11 @@ static void * wrkr(void *myself); #define DFLT_wrkrMax 2 +#define COMPRESS_NEVER 0 +#define COMPRESS_SINGLE_MSG 1 /* old, single-message compression */ +/* all other settings are for stream-compression */ +#define COMPRESS_STREAM_ALWAYS 2 + /* config settings */ typedef struct configSettings_s { int bKeepAlive; /* support keep-alive packets */ @@ -117,11 +124,13 @@ struct instanceConf_s { int bEmitMsgOnClose; int bSuppOctetFram; /* support octet-counted framing? */ int iAddtlFrameDelim; + uint8_t compressionMode; uchar *pszBindPort; /* port to bind to */ uchar *pszBindAddr; /* IP to bind socket to */ uchar *pszBindRuleset; /* name of ruleset to bind to */ uchar *pszInputName; /* value for inputname property, NULL is OK and handled by core engine */ ruleset_t *pBindRuleset; /* ruleset to bind listener to (use system default if unspecified) */ + uchar *dfltTZ; int ratelimitInterval; int ratelimitBurst; struct instanceConf_s *next; @@ -154,8 +163,10 @@ static struct cnfparamdescr inppdescr[] = { { "address", eCmdHdlrString, 0 }, { "name", eCmdHdlrString, 0 }, { "ruleset", eCmdHdlrString, 0 }, + { "defaulttz", eCmdHdlrString, 0 }, { "supportoctetcountedframing", eCmdHdlrBinary, 0 }, { "notifyonconnectionclose", eCmdHdlrBinary, 0 }, + { "compression.mode", eCmdHdlrGetWord, 0 }, { "keepalive", eCmdHdlrBinary, 0 }, { "keepalive.probes", eCmdHdlrInt, 0 }, { "keepalive.time", eCmdHdlrInt, 0 }, @@ -191,7 +202,9 @@ struct ptcpsrv_s { int iKeepAliveIntvl; int iKeepAliveProbes; int iKeepAliveTime; + uint8_t compressionMode; uchar *pszInputName; + uchar *dfltTZ; prop_t *pInputName; /* InputName in (fast to process) property format */ ruleset_t *pRuleset; ptcplstn_t *pLstn; /* root of our listeners */ @@ -207,11 +220,13 @@ struct ptcpsrv_s { * includes support for doubly-linked list. */ struct ptcpsess_s { -// ptcpsrv_t *pSrv; /* our server TODO: check remove! */ ptcplstn_t *pLstn; /* our listener */ ptcpsess_t *prev, *next; int sock; epolld_t *epd; + sbool bzInitDone; /* did we do an init of zstrm already? */ + z_stream zstrm; /* zip stream to use for tcp compression */ + uint8_t compressionMode; //--- from tcps_sess.h int iMsg; /* index of next char to store in msg */ int bAtStrtOfFram; /* are we at the very beginning of a new frame? */ @@ -239,6 +254,8 @@ struct ptcplstn_s { sbool bSuppOctetFram; epolld_t *epd; statsobj_t *stats; /* listener stats */ + intctr_t rcvdBytes; + intctr_t rcvdDecompressed; STATSCOUNTER_DEF(ctrSubmit, mutCtrSubmit) }; @@ -301,11 +318,17 @@ destructSess(ptcpsess_t *pSess) static void destructSrv(ptcpsrv_t *pSrv) { - ratelimitDestruct(pSrv->ratelimiter); - prop.Destruct(&pSrv->pInputName); + if(pSrv->ratelimiter != NULL) + ratelimitDestruct(pSrv->ratelimiter); + if(pSrv->pInputName != NULL) + prop.Destruct(&pSrv->pInputName); pthread_mutex_destroy(&pSrv->mutSessLst); - free(pSrv->pszInputName); - free(pSrv->port); + if(pSrv->pszInputName != NULL) + free(pSrv->pszInputName); + if(pSrv->port != NULL) + free(pSrv->port); + if(pSrv->lstnIP != NULL) + free(pSrv->lstnIP); free(pSrv); } @@ -481,11 +504,14 @@ getPeerNames(prop_t **peerName, prop_t **peerIP, struct sockaddr *pAddr) uchar szIP[NI_MAXHOST] = ""; uchar szHname[NI_MAXHOST] = ""; struct addrinfo hints, *res; + sbool bMaliciousHName = 0; DEFiRet; - error = getnameinfo(pAddr, SALEN(pAddr), (char*)szIP, sizeof(szIP), NULL, 0, NI_NUMERICHOST); + *peerName = NULL; + *peerIP = NULL; + error = getnameinfo(pAddr, SALEN(pAddr), (char*)szIP, sizeof(szIP), NULL, 0, NI_NUMERICHOST); if(error) { DBGPRINTF("Malformed from address %s\n", gai_strerror(error)); strcpy((char*)szHname, "???"); @@ -508,7 +534,7 @@ getPeerNames(prop_t **peerName, prop_t **peerIP, struct sockaddr *pAddr) /* OK, we know we have evil, so let's indicate this to our caller */ snprintf((char*)szHname, NI_MAXHOST, "[MALICIOUS:IP=%s]", szIP); DBGPRINTF("Malicious PTR record, IP = \"%s\" HOST = \"%s\"", szIP, szHname); - iRet = RS_RET_MALICIOUS_HNAME; + bMaliciousHName = 1; } } else { strcpy((char*)szHname, (char*)szIP); @@ -526,6 +552,14 @@ getPeerNames(prop_t **peerName, prop_t **peerIP, struct sockaddr *pAddr) CHKiRet(prop.ConstructFinalize(*peerIP)); finalize_it: + if(iRet != RS_RET_OK) { + if(*peerName != NULL) + prop.Destruct(peerName); + if(*peerIP != NULL) + prop.Destruct(peerIP); + } + if(bMaliciousHName) + iRet = RS_RET_MALICIOUS_HNAME; RETiRet; } @@ -614,7 +648,7 @@ AcceptConnReq(ptcplstn_t *pLstn, int *newSock, prop_t **peerName, prop_t **peerI iNewSock = accept(pLstn->sock, (struct sockaddr*) &addr, &addrlen); if(iNewSock < 0) { - if(errno == EAGAIN || errno == EWOULDBLOCK) + if(errno == EAGAIN || errno == EWOULDBLOCK || errno == EMFILE) ABORT_FINALIZE(RS_RET_NO_MORE_DATA); ABORT_FINALIZE(RS_RET_ACCEPT_ERR); } @@ -635,6 +669,8 @@ AcceptConnReq(ptcplstn_t *pLstn, int *newSock, prop_t **peerName, prop_t **peerI } if(sockflags == -1) { DBGPRINTF("error %d setting fcntl(O_NONBLOCK) on tcp socket %d", errno, iNewSock); + prop.Destruct(peerName); + prop.Destruct(peerIP); ABORT_FINALIZE(RS_RET_IO_ERROR); } @@ -680,6 +716,8 @@ doSubmitMsg(ptcpsess_t *pThis, struct syslogTime *stTime, time_t ttGenTime, mult MsgSetRawMsg(pMsg, (char*)pThis->pMsg, pThis->iMsg); MsgSetInputName(pMsg, pSrv->pInputName); MsgSetFlowControlType(pMsg, eFLOWCTL_LIGHT_DELAY); + if(pSrv->dfltTZ != NULL) + MsgSetDfltTZ(pMsg, (char*) pSrv->dfltTZ); pMsg->msgFlags = NEEDS_PARSING | PARSE_HOSTNAME; MsgSetRcvFrom(pMsg, pThis->peerName); CHKiRet(MsgSetRcvFromIP(pMsg, pThis->peerIP)); @@ -727,13 +765,13 @@ processDataRcvd(ptcpsess_t *pThis, char c, struct syslogTime *stTime, time_t ttG DBGPRINTF("TCP Message with octet-counter, size %d.\n", pThis->iOctetsRemain); if(c != ' ') { errmsg.LogError(0, NO_ERRCODE, "Framing Error in received TCP message: " - "delimiter is not SP but has ASCII value %d.\n", c); + "delimiter is not SP but has ASCII value %d.", c); } if(pThis->iOctetsRemain < 1) { /* TODO: handle the case where the octet count is 0! */ DBGPRINTF("Framing Error: invalid octet count\n"); errmsg.LogError(0, NO_ERRCODE, "Framing Error in received TCP message: " - "invalid octet count %d.\n", pThis->iOctetsRemain); + "invalid octet count %d.", pThis->iOctetsRemain); } else if(pThis->iOctetsRemain > iMaxLine) { /* while we can not do anything against it, we can at least log an indication * that something went wrong) -- rgerhards, 2008-03-14 @@ -741,7 +779,7 @@ processDataRcvd(ptcpsess_t *pThis, char c, struct syslogTime *stTime, time_t ttG DBGPRINTF("truncating message with %d octets - max msg size is %d\n", pThis->iOctetsRemain, iMaxLine); errmsg.LogError(0, NO_ERRCODE, "received oversize message: size is %d bytes, " - "max msg size is %d, truncating...\n", pThis->iOctetsRemain, iMaxLine); + "max msg size is %d, truncating...", pThis->iOctetsRemain, iMaxLine); } pThis->inputState = eInMsg; } @@ -806,19 +844,18 @@ processDataRcvd(ptcpsess_t *pThis, char c, struct syslogTime *stTime, time_t ttG * EXTRACT from tcps_sess.c */ static rsRetVal -DataRcvd(ptcpsess_t *pThis, char *pData, size_t iLen) +DataRcvdUncompressed(ptcpsess_t *pThis, char *pData, size_t iLen, struct syslogTime *stTime, time_t ttGenTime) { multi_submit_t multiSub; msg_t *pMsgs[CONF_NUM_MULTISUB]; - struct syslogTime stTime; - time_t ttGenTime; char *pEnd; DEFiRet; assert(pData != NULL); assert(iLen > 0); - datetime.getCurrTime(&stTime, &ttGenTime); + if(ttGenTime == 0) + datetime.getCurrTime(stTime, &ttGenTime); multiSub.ppMsgs = pMsgs; multiSub.maxElem = CONF_NUM_MULTISUB; multiSub.nElem = 0; @@ -827,7 +864,7 @@ DataRcvd(ptcpsess_t *pThis, char *pData, size_t iLen) pEnd = pData + iLen; /* this is one off, which is intensional */ while(pData < pEnd) { - CHKiRet(processDataRcvd(pThis, *pData++, &stTime, ttGenTime, &multiSub)); + CHKiRet(processDataRcvd(pThis, *pData++, stTime, ttGenTime, &multiSub)); } iRet = multiSubmitFlush(&multiSub); @@ -836,6 +873,71 @@ finalize_it: RETiRet; } +static rsRetVal +DataRcvdCompressed(ptcpsess_t *pThis, char *buf, size_t len) +{ + struct syslogTime stTime; + time_t ttGenTime; + int zRet; /* zlib return state */ + unsigned outavail; + uchar zipBuf[64*1024]; // TODO: alloc on heap, and much larger (512KiB? batch size!) + DEFiRet; + // TODO: can we do stats counters? Even if they are not 100% correct under all cases, + // by simply updating the input and output sizes? + uint64_t outtotal; + + datetime.getCurrTime(&stTime, &ttGenTime); + outtotal = 0; + + if(!pThis->bzInitDone) { + /* allocate deflate state */ + pThis->zstrm.zalloc = Z_NULL; + pThis->zstrm.zfree = Z_NULL; + pThis->zstrm.opaque = Z_NULL; + zRet = inflateInit(&pThis->zstrm); + if(zRet != Z_OK) { + DBGPRINTF("imptcp: error %d returned from zlib/inflateInit()\n", zRet); + ABORT_FINALIZE(RS_RET_ZLIB_ERR); + } + pThis->bzInitDone = RSTRUE; + } + + pThis->zstrm.next_in = (Bytef*) buf; + pThis->zstrm.avail_in = len; + /* run inflate() on buffer until everything has been uncompressed */ + do { + DBGPRINTF("imptcp: in inflate() loop, avail_in %d, total_in %ld\n", pThis->zstrm.avail_in, pThis->zstrm.total_in); + pThis->zstrm.avail_out = sizeof(zipBuf); + pThis->zstrm.next_out = zipBuf; + zRet = inflate(&pThis->zstrm, Z_SYNC_FLUSH); /* no bad return value */ + //zRet = inflate(&pThis->zstrm, Z_NO_FLUSH); /* no bad return value */ + DBGPRINTF("after inflate, ret %d, avail_out %d\n", zRet, pThis->zstrm.avail_out); + outavail = sizeof(zipBuf) - pThis->zstrm.avail_out; + if(outavail != 0) { + outtotal += outavail; + pThis->pLstn->rcvdDecompressed += outavail; + CHKiRet(DataRcvdUncompressed(pThis, (char*)zipBuf, outavail, &stTime, ttGenTime)); + } + } while (pThis->zstrm.avail_out == 0); + + dbgprintf("end of DataRcvCompress, sizes: in %lld, out %llu\n", (long long) len, (long long unsigned) outtotal); +finalize_it: + RETiRet; +} + +static rsRetVal +DataRcvd(ptcpsess_t *pThis, char *pData, size_t iLen) +{ + struct syslogTime stTime; + DEFiRet; + pThis->pLstn->rcvdBytes += iLen; + if(pThis->compressionMode >= COMPRESS_STREAM_ALWAYS) + iRet = DataRcvdCompressed(pThis, pData, iLen); + else + iRet = DataRcvdUncompressed(pThis, pData, iLen, &stTime, 0); + RETiRet; +} + /****************************************** --END-- TCP SUPPORT FUNCTIONS ***********************************/ @@ -919,10 +1021,10 @@ static rsRetVal addLstn(ptcpsrv_t *pSrv, int sock, int isIPv6) { DEFiRet; - ptcplstn_t *pLstn; + ptcplstn_t *pLstn = NULL; uchar statname[64]; - CHKmalloc(pLstn = malloc(sizeof(ptcplstn_t))); + CHKmalloc(pLstn = calloc(1, sizeof(ptcplstn_t))); pLstn->pSrv = pSrv; pLstn->bSuppOctetFram = pSrv->bSuppOctetFram; pLstn->sock = sock; @@ -935,9 +1037,19 @@ addLstn(ptcpsrv_t *pSrv, int sock, int isIPv6) CHKiRet(statsobj.SetName(pLstn->stats, statname)); STATSCOUNTER_INIT(pLstn->ctrSubmit, pLstn->mutCtrSubmit); CHKiRet(statsobj.AddCounter(pLstn->stats, UCHAR_CONSTANT("submitted"), - ctrType_IntCtr, &(pLstn->ctrSubmit))); + ctrType_IntCtr, CTR_FLAG_RESETTABLE, &(pLstn->ctrSubmit))); + /* the following counters are not protected by mutexes; we accept + * that they may not be 100% correct */ + pLstn->rcvdBytes = 0, + pLstn->rcvdDecompressed = 0; + CHKiRet(statsobj.AddCounter(pLstn->stats, UCHAR_CONSTANT("bytes.received"), + ctrType_IntCtr, CTR_FLAG_RESETTABLE, &(pLstn->rcvdBytes))); + CHKiRet(statsobj.AddCounter(pLstn->stats, UCHAR_CONSTANT("bytes.decompressed"), + ctrType_IntCtr, CTR_FLAG_RESETTABLE, &(pLstn->rcvdDecompressed))); CHKiRet(statsobj.ConstructFinalize(pLstn->stats)); + CHKiRet(addEPollSock(epolld_lstn, pLstn, sock, &pLstn->epd)); + /* add to start of server's listener list */ pLstn->prev = NULL; pLstn->next = pSrv->pLstn; @@ -945,9 +1057,15 @@ addLstn(ptcpsrv_t *pSrv, int sock, int isIPv6) pSrv->pLstn->prev = pLstn; pSrv->pLstn = pLstn; - iRet = addEPollSock(epolld_lstn, pLstn, sock, &pLstn->epd); - finalize_it: + if(iRet != RS_RET_OK) { + if(pLstn != NULL) { + if(pLstn->stats != NULL) + statsobj.Destruct(&(pLstn->stats)); + free(pLstn); + } + } + RETiRet; } @@ -968,9 +1086,13 @@ addSess(ptcplstn_t *pLstn, int sock, prop_t *peerName, prop_t *peerIP) pSess->bSuppOctetFram = pLstn->bSuppOctetFram; pSess->inputState = eAtStrtFram; pSess->iMsg = 0; + pSess->bzInitDone = 0; pSess->bAtStrtOfFram = 1; pSess->peerName = peerName; pSess->peerIP = peerIP; + pSess->compressionMode = pLstn->pSrv->compressionMode; + + CHKiRet(addEPollSock(epolld_sess, pSess, sock, &pSess->epd)); /* add to start of server's listener list */ pSess->prev = NULL; @@ -981,13 +1103,58 @@ addSess(ptcplstn_t *pLstn, int sock, prop_t *peerName, prop_t *peerIP) pSrv->pSess = pSess; pthread_mutex_unlock(&pSrv->mutSessLst); - iRet = addEPollSock(epolld_sess, pSess, sock, &pSess->epd); - finalize_it: + if(iRet != RS_RET_OK) { + if(pSess != NULL) { + if(pSess->pMsg != NULL) + free(pSess->pMsg); + free(pSess); + } + } + RETiRet; } +/* finish zlib buffer, to be called before closing the session. + */ +static rsRetVal +doZipFinish(ptcpsess_t *pSess) +{ + int zRet; /* zlib return state */ + DEFiRet; + unsigned outavail; + struct syslogTime stTime; + uchar zipBuf[32*1024]; // TODO: use "global" one from pSess + + if(!pSess->bzInitDone) + goto done; + + pSess->zstrm.avail_in = 0; + /* run inflate() on buffer until everything has been compressed */ + do { + DBGPRINTF("doZipFinish: in inflate() loop, avail_in %d, total_in %ld\n", pSess->zstrm.avail_in, pSess->zstrm.total_in); + pSess->zstrm.avail_out = sizeof(zipBuf); + pSess->zstrm.next_out = zipBuf; + zRet = inflate(&pSess->zstrm, Z_FINISH); /* no bad return value */ + DBGPRINTF("after inflate, ret %d, avail_out %d\n", zRet, pSess->zstrm.avail_out); + outavail = sizeof(zipBuf) - pSess->zstrm.avail_out; + if(outavail != 0) { + pSess->pLstn->rcvdDecompressed += outavail; + CHKiRet(DataRcvdUncompressed(pSess, (char*)zipBuf, outavail, &stTime, 0)); // TODO: query time! + } + } while (pSess->zstrm.avail_out == 0); + +finalize_it: + zRet = inflateEnd(&pSess->zstrm); + if(zRet != Z_OK) { + DBGPRINTF("imptcp: error %d returned from zlib/inflateEnd()\n", zRet); + } + + pSess->bzInitDone = 0; +done: RETiRet; +} + /* close/remove a session * NOTE: we must first remove the fd from the epoll set and then close it -- else we * get an error "bad file descriptor" from epoll. @@ -998,6 +1165,9 @@ closeSess(ptcpsess_t *pSess) int sock; DEFiRet; + if(pSess->compressionMode >= COMPRESS_STREAM_ALWAYS) + doZipFinish(pSess); + sock = pSess->sock; CHKiRet(removeEPollSock(sock, pSess->epd)); close(sock); @@ -1044,10 +1214,12 @@ createInstance(instanceConf_t **pinst) inst->iKeepAliveProbes = 0; inst->iKeepAliveTime = 0; inst->bEmitMsgOnClose = 0; + inst->dfltTZ = NULL; inst->iAddtlFrameDelim = TCPSRV_NO_ADDTL_DELIMITER; inst->pBindRuleset = NULL; inst->ratelimitBurst = 10000; /* arbitrary high limit */ inst->ratelimitInterval = 0; /* off */ + inst->compressionMode = COMPRESS_SINGLE_MSG; /* node created, let's add to config */ if(loadModConf->tail == NULL) { @@ -1115,9 +1287,9 @@ static inline rsRetVal addListner(modConfData_t __attribute__((unused)) *modConf, instanceConf_t *inst) { DEFiRet; - ptcpsrv_t *pSrv; + ptcpsrv_t *pSrv = NULL; - CHKmalloc(pSrv = MALLOC(sizeof(ptcpsrv_t))); + CHKmalloc(pSrv = calloc(1, sizeof(ptcpsrv_t))); pthread_mutex_init(&pSrv->mutSessLst, NULL); pSrv->pSess = NULL; pSrv->pLstn = NULL; @@ -1127,6 +1299,8 @@ addListner(modConfData_t __attribute__((unused)) *modConf, instanceConf_t *inst) pSrv->iKeepAliveProbes = inst->iKeepAliveProbes; pSrv->iKeepAliveTime = inst->iKeepAliveTime; pSrv->bEmitMsgOnClose = inst->bEmitMsgOnClose; + pSrv->compressionMode = inst->compressionMode; + pSrv->dfltTZ = inst->dfltTZ; CHKiRet(ratelimitNew(&pSrv->ratelimiter, "imtcp", (char*)inst->pszBindPort)); ratelimitSetLinuxLike(pSrv->ratelimiter, inst->ratelimitInterval, inst->ratelimitBurst); ratelimitSetThreadSafe(pSrv->ratelimiter); @@ -1155,6 +1329,9 @@ addListner(modConfData_t __attribute__((unused)) *modConf, instanceConf_t *inst) finalize_it: if(iRet != RS_RET_OK) { errmsg.LogError(0, NO_ERRCODE, "error %d trying to add listener", iRet); + if(pSrv != NULL) { + destructSrv(pSrv); + } } RETiRet; } @@ -1253,7 +1430,13 @@ lstnActivity(ptcplstn_t *pLstn) if(localRet == RS_RET_NO_MORE_DATA || glbl.GetGlobalInputTermState() == 1) break; CHKiRet(localRet); - CHKiRet(addSess(pLstn, newSock, peerName, peerIP)); + localRet = addSess(pLstn, newSock, peerName, peerIP); + if(localRet != RS_RET_OK) { + close(newSock); + prop.Destruct(&peerName); + prop.Destruct(&peerIP); + ABORT_FINALIZE(localRet); + } } finalize_it: @@ -1269,6 +1452,10 @@ sessActivity(ptcpsess_t *pSess) { int lenRcv; int lenBuf; + uchar *peerName; + int lenPeer; + int remsock = 0; /* init just to keep compiler happy... :-( */ + sbool bEmitOnClose = 0; char rcvBuf[128*1024]; DEFiRet; @@ -1285,13 +1472,15 @@ sessActivity(ptcpsess_t *pSess) } else if (lenRcv == 0) { /* session was closed, do clean-up */ if(pSess->pLstn->pSrv->bEmitMsgOnClose) { - uchar *peerName; - int lenPeer; - prop.GetString(pSess->peerName, &peerName, &lenPeer); - errmsg.LogError(0, RS_RET_PEER_CLOSED_CONN, "imptcp session %d closed by remote peer %s.\n", - pSess->sock, peerName); + prop.GetString(pSess->peerName, &peerName, &lenPeer), + remsock = pSess->sock; + bEmitOnClose = 1; + } + CHKiRet(closeSess(pSess)); /* close may emit more messages in strmzip mode! */ + if(bEmitOnClose) { + errmsg.LogError(0, RS_RET_PEER_CLOSED_CONN, "imptcp session %d closed by " + "remote peer %s.", remsock, peerName); } - CHKiRet(closeSess(pSess)); break; } else { if(errno == EAGAIN || errno == EWOULDBLOCK) @@ -1415,14 +1604,12 @@ wrkr(void *myself) BEGINnewInpInst struct cnfparamvals *pvals; instanceConf_t *inst; + char *cstr; int i; CODESTARTnewInpInst DBGPRINTF("newInpInst (imptcp)\n"); - pvals = nvlstGetParams(lst, &inppblk, NULL); - if(pvals == NULL) { - errmsg.LogError(0, RS_RET_MISSING_CNFPARAMS, - "imptcp: required parameter are missing\n"); + if((pvals = nvlstGetParams(lst, &inppblk, NULL)) == NULL) { ABORT_FINALIZE(RS_RET_MISSING_CNFPARAMS); } @@ -1446,6 +1633,19 @@ CODESTARTnewInpInst inst->pszBindRuleset = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); } else if(!strcmp(inppblk.descr[i].name, "supportoctetcountedframing")) { inst->bSuppOctetFram = (int) pvals[i].val.d.n; + } else if(!strcmp(inppblk.descr[i].name, "compression.mode")) { + cstr = es_str2cstr(pvals[i].val.d.estr, NULL); + if(!strcasecmp(cstr, "stream:always")) { + inst->compressionMode = COMPRESS_STREAM_ALWAYS; + } else if(!strcasecmp(cstr, "none")) { + inst->compressionMode = COMPRESS_NEVER; + } else { + errmsg.LogError(0, RS_RET_PARAM_ERROR, "omfwd: invalid value for 'compression.mode' " + "parameter (given is '%s')", cstr); + free(cstr); + ABORT_FINALIZE(RS_RET_PARAM_ERROR); + } + free(cstr); } else if(!strcmp(inppblk.descr[i].name, "keepalive")) { inst->bKeepAlive = (int) pvals[i].val.d.n; } else if(!strcmp(inppblk.descr[i].name, "keepalive.probes")) { @@ -1458,6 +1658,8 @@ CODESTARTnewInpInst inst->iAddtlFrameDelim = (int) pvals[i].val.d.n; } else if(!strcmp(inppblk.descr[i].name, "notifyonconnectionclose")) { inst->bEmitMsgOnClose = (int) pvals[i].val.d.n; + } else if(!strcmp(inppblk.descr[i].name, "defaulttz")) { + inst->dfltTZ = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); } else if(!strcmp(inppblk.descr[i].name, "ratelimit.burst")) { inst->ratelimitBurst = (int) pvals[i].val.d.n; } else if(!strcmp(inppblk.descr[i].name, "ratelimit.interval")) { @@ -1562,6 +1764,7 @@ BEGINactivateCnfPrePrivDrop instanceConf_t *inst; CODESTARTactivateCnfPrePrivDrop iMaxLine = glbl.GetMaxLine(); /* get maximum size we currently support */ + DBGPRINTF("imptcp: config params iMaxLine %d\n", iMaxLine); runModConf = pModConf; for(inst = runModConf->root ; inst != NULL ; inst = inst->next) { @@ -1612,6 +1815,7 @@ CODESTARTfreeCnf free(inst->pszBindAddr); free(inst->pszBindRuleset); free(inst->pszInputName); + free(inst->dfltTZ); del = inst; inst = inst->next; free(del); @@ -1661,7 +1865,9 @@ shutdownSrv(ptcpsrv_t *pSrv) /* now unlink listner */ lstnDel = pLstn; pLstn = pLstn->next; - DBGPRINTF("imptcp shutdown listen socket %d\n", lstnDel->sock); + DBGPRINTF("imptcp shutdown listen socket %d (rcvd %lld bytes, " + "decompressed %lld)\n", lstnDel->sock, lstnDel->rcvdBytes, + lstnDel->rcvdDecompressed); free(lstnDel->epd); free(lstnDel); } diff --git a/plugins/imrelp/Makefile.in b/plugins/imrelp/Makefile.in index 566efab..ec45d31 100644 --- a/plugins/imrelp/Makefile.in +++ b/plugins/imrelp/Makefile.in @@ -154,7 +154,6 @@ GREP = @GREP@ GSS_LIBS = @GSS_LIBS@ GUARDTIME_CFLAGS = @GUARDTIME_CFLAGS@ GUARDTIME_LIBS = @GUARDTIME_LIBS@ -HAVE_LIBGCRYPT_CONFIG = @HAVE_LIBGCRYPT_CONFIG@ HAVE_MYSQL_CONFIG = @HAVE_MYSQL_CONFIG@ HAVE_ORACLE_CONFIG = @HAVE_ORACLE_CONFIG@ HAVE_PGSQL_CONFIG = @HAVE_PGSQL_CONFIG@ @@ -175,14 +174,15 @@ LEXLIB = @LEXLIB@ LEX_OUTPUT_ROOT = @LEX_OUTPUT_ROOT@ LIBDBI_CFLAGS = @LIBDBI_CFLAGS@ LIBDBI_LIBS = @LIBDBI_LIBS@ -LIBEE_CFLAGS = @LIBEE_CFLAGS@ -LIBEE_LIBS = @LIBEE_LIBS@ LIBESTR_CFLAGS = @LIBESTR_CFLAGS@ LIBESTR_LIBS = @LIBESTR_LIBS@ LIBGCRYPT_CFLAGS = @LIBGCRYPT_CFLAGS@ +LIBGCRYPT_CONFIG = @LIBGCRYPT_CONFIG@ LIBGCRYPT_LIBS = @LIBGCRYPT_LIBS@ LIBLOGGING_CFLAGS = @LIBLOGGING_CFLAGS@ LIBLOGGING_LIBS = @LIBLOGGING_LIBS@ +LIBLOGGING_STDLOG_CFLAGS = @LIBLOGGING_STDLOG_CFLAGS@ +LIBLOGGING_STDLOG_LIBS = @LIBLOGGING_STDLOG_LIBS@ LIBLOGNORM_CFLAGS = @LIBLOGNORM_CFLAGS@ LIBLOGNORM_LIBS = @LIBLOGNORM_LIBS@ LIBM = @LIBM@ @@ -207,6 +207,8 @@ NM = @NM@ NMEDIT = @NMEDIT@ OBJDUMP = @OBJDUMP@ OBJEXT = @OBJEXT@ +OPENSSL_CFLAGS = @OPENSSL_CFLAGS@ +OPENSSL_LIBS = @OPENSSL_LIBS@ ORACLE_CFLAGS = @ORACLE_CFLAGS@ ORACLE_LIBS = @ORACLE_LIBS@ OTOOL = @OTOOL@ @@ -387,22 +389,25 @@ distclean-compile: @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/imrelp_la-imrelp.Plo@am__quote@ .c.o: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(COMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ $< -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Po +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.o$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c $< +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ $< .c.obj: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(COMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ `$(CYGPATH_W) '$<'` -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Po +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.obj$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ `$(CYGPATH_W) '$<'` &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c `$(CYGPATH_W) '$<'` +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ `$(CYGPATH_W) '$<'` .c.lo: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(LTCOMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ $< -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Plo +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.lo$$||'`;\ +@am__fastdepCC_TRUE@ $(LTCOMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Plo @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=yes @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ @am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(LTCOMPILE) -c -o $@ $< diff --git a/plugins/imrelp/imrelp.c b/plugins/imrelp/imrelp.c index 5e0ae55..6787f91 100644 --- a/plugins/imrelp/imrelp.c +++ b/plugins/imrelp/imrelp.c @@ -4,7 +4,7 @@ * * File begun on 2008-03-13 by RGerhards * - * Copyright 2008-2012 Adiscon GmbH. + * Copyright 2008-2014 Adiscon GmbH. * * This file is part of rsyslog. * @@ -47,6 +47,7 @@ #include "prop.h" #include "ruleset.h" #include "glbl.h" +#include "statsobj.h" MODULE_TYPE_INPUT MODULE_TYPE_NOKEEP @@ -59,6 +60,7 @@ DEFobjCurrIf(prop) DEFobjCurrIf(errmsg) DEFobjCurrIf(ruleset) DEFobjCurrIf(glbl) +DEFobjCurrIf(statsobj) /* forward definitions */ static rsRetVal resetConfigVariables(uchar __attribute__((unused)) *pp, void __attribute__((unused)) *pVal); @@ -67,30 +69,88 @@ static rsRetVal resetConfigVariables(uchar __attribute__((unused)) *pp, void __a /* Module static data */ /* config vars for legacy config system */ static relpEngine_t *pRelpEngine; /* our relp engine */ -static prop_t *pInputName = NULL; /* there is only one global inputName for all messages generated by this module */ -static struct configSettings_s { + +/* config settings */ +typedef struct configSettings_s { uchar *pszBindRuleset; /* name of Ruleset to bind to */ -} cs; +} configSettings_t; +static configSettings_t cs; struct instanceConf_s { uchar *pszBindPort; /* port to bind to */ + uchar *pszBindRuleset; /* name of ruleset to bind to */ + uchar *pszInputName; /* value for inputname property */ + prop_t *pInputName; /* InputName in property format for fast access */ + ruleset_t *pBindRuleset; /* ruleset to bind listener to */ + sbool bKeepAlive; /* support keep-alive packets */ + sbool bEnableTLS; + sbool bEnableTLSZip; + int dhBits; + uchar *pristring; /* GnuTLS priority string (NULL if not to be provided) */ + uchar *authmode; /* TLS auth mode */ + uchar *caCertFile; + uchar *myCertFile; + uchar *myPrivKeyFile; + int iKeepAliveIntvl; + int iKeepAliveProbes; + int iKeepAliveTime; + struct { + int nmemb; + uchar **name; + } permittedPeers; + struct instanceConf_s *next; + /* with librelp, this module does not have any own specific session + * or listener active data item. As a "work-around", we keep some + * data items inside the configuration object. To keep things + * decently clean, we put them all into their dedicated struct. So + * it is easy to judge what is actual configuration and what is + * dynamic runtime data. -- rgerhards, 2013-06-18 + */ + struct { + statsobj_t *stats; /* listener stats */ + STATSCOUNTER_DEF(ctrSubmit, mutCtrSubmit) + } data; }; struct modConfData_s { rsconf_t *pConf; /* our overall config object */ instanceConf_t *root, *tail; - uchar *pszBindRuleset; /* name of Ruleset to bind to */ - ruleset_t *pBindRuleset; /* due to librelp limitation, we need to bind all listerns to the same set */ + uchar *pszBindRuleset; /* default name of Ruleset to bind to */ }; static modConfData_t *loadModConf = NULL;/* modConf ptr to use for the current load process */ static modConfData_t *runModConf = NULL;/* modConf ptr to use for the current load process */ +/* module-global parameters */ +static struct cnfparamdescr modpdescr[] = { + { "ruleset", eCmdHdlrGetWord, 0 }, +}; +static struct cnfparamblk modpblk = + { CNFPARAMBLK_VERSION, + sizeof(modpdescr)/sizeof(struct cnfparamdescr), + modpdescr + }; + /* input instance parameters */ static struct cnfparamdescr inppdescr[] = { - { "port", eCmdHdlrString, CNFPARAM_REQUIRED } + { "port", eCmdHdlrString, CNFPARAM_REQUIRED }, + { "name", eCmdHdlrString, 0 }, + { "ruleset", eCmdHdlrString, 0 }, + { "keepalive", eCmdHdlrBinary, 0 }, + { "keepalive.probes", eCmdHdlrInt, 0 }, + { "keepalive.time", eCmdHdlrInt, 0 }, + { "keepalive.interval", eCmdHdlrInt, 0 }, + { "tls", eCmdHdlrBinary, 0 }, + { "tls.permittedpeer", eCmdHdlrArray, 0 }, + { "tls.authmode", eCmdHdlrString, 0 }, + { "tls.dhbits", eCmdHdlrInt, 0 }, + { "tls.prioritystring", eCmdHdlrString, 0 }, + { "tls.cacert", eCmdHdlrString, 0 }, + { "tls.mycert", eCmdHdlrString, 0 }, + { "tls.myprivkey", eCmdHdlrString, 0 }, + { "tls.compression", eCmdHdlrBinary, 0 } }; static struct cnfparamblk inppblk = { CNFPARAMBLK_VERSION, @@ -98,10 +158,35 @@ static struct cnfparamblk inppblk = inppdescr }; - +#include "im-helper.h" /* must be included AFTER the type definitions! */ +static int bLegacyCnfModGlobalsPermitted;/* are legacy module-global config parameters permitted? */ /* ------------------------------ callbacks ------------------------------ */ +static void +onErr(void *pUsr, char *objinfo, char* errmesg, __attribute__((unused)) relpRetVal errcode) +{ + instanceConf_t *inst = (instanceConf_t*) pUsr; + errmsg.LogError(0, RS_RET_RELP_AUTH_FAIL, "imrelp[%s]: error '%s', object " + " '%s' - input may not work as intended", + inst->pszBindPort, errmesg, objinfo); +} + +static void +onGenericErr(char *objinfo, char* errmesg, __attribute__((unused)) relpRetVal errcode) +{ + errmsg.LogError(0, RS_RET_RELP_ERR, "imrelp: librelp error '%s', object " + " '%s' - input may not work as intended", errmesg, objinfo); +} + +static void +onAuthErr(void *pUsr, char *authinfo, char* errmesg, __attribute__((unused)) relpRetVal errcode) +{ + instanceConf_t *inst = (instanceConf_t*) pUsr; + errmsg.LogError(0, RS_RET_RELP_AUTH_FAIL, "imrelp[%s]: authentication error '%s', peer " + "is '%s'", inst->pszBindPort, errmesg, authinfo); +} + /* callback for receiving syslog messages. This function is invoked from the * RELP engine when a syslog message arrived. It must return a relpRetVal, * with anything else but RELP_RET_OK terminating the relp session. Please note @@ -113,27 +198,27 @@ static struct cnfparamblk inppblk = * we will only see the hostname (twice). -- rgerhards, 2009-10-14 */ static relpRetVal -onSyslogRcv(uchar *pHostname, uchar *pIP, uchar *msg, size_t lenMsg) +onSyslogRcv(void *pUsr, uchar *pHostname, uchar *pIP, uchar *msg, size_t lenMsg) { prop_t *pProp = NULL; msg_t *pMsg; + instanceConf_t *inst = (instanceConf_t*) pUsr; DEFiRet; CHKiRet(msgConstruct(&pMsg)); - MsgSetInputName(pMsg, pInputName); + MsgSetInputName(pMsg, inst->pInputName); MsgSetRawMsg(pMsg, (char*)msg, lenMsg); MsgSetFlowControlType(pMsg, eFLOWCTL_LIGHT_DELAY); - MsgSetRuleset(pMsg, runModConf->pBindRuleset); + MsgSetRuleset(pMsg, inst->pBindRuleset); pMsg->msgFlags = PARSE_HOSTNAME | NEEDS_PARSING; - /* TODO: optimize this, we can store it inside the session, requires - * changes to librelp --> next librelp iteration?. rgerhards, 2012-10-29 - */ + /* TODO: optimize this, we can store it inside the session */ MsgSetRcvFromStr(pMsg, pHostname, ustrlen(pHostname), &pProp); CHKiRet(prop.Destruct(&pProp)); CHKiRet(MsgSetRcvFromIPStr(pMsg, pIP, ustrlen(pIP), &pProp)); CHKiRet(prop.Destruct(&pProp)); CHKiRet(submitMsg2(pMsg)); + STATSCOUNTER_INC(inst->data.ctrSubmit, inst->data.mutCtrSubmit); finalize_it: @@ -155,6 +240,22 @@ createInstance(instanceConf_t **pinst) inst->next = NULL; inst->pszBindPort = NULL; + inst->pszBindRuleset = NULL; + inst->pszInputName = NULL; + inst->pBindRuleset = NULL; + inst->bKeepAlive = 0; + inst->iKeepAliveIntvl = 0; + inst->iKeepAliveProbes = 0; + inst->iKeepAliveTime = 0; + inst->bEnableTLS = 0; + inst->bEnableTLSZip = 0; + inst->dhBits = 0; + inst->pristring = NULL; + inst->authmode = NULL; + inst->permittedPeers.nmemb = 0; + inst->caCertFile = NULL; + inst->myCertFile = NULL; + inst->myPrivKeyFile = NULL; /* node created, let's add to config */ if(loadModConf->tail == NULL) { @@ -170,16 +271,17 @@ finalize_it: } -/* modified to work for module, not instance (as usual) */ +/* function to generate an error message if the ruleset cannot be found */ static inline void -std_checkRuleset_genErrMsg(modConfData_t *modConf, __attribute__((unused)) instanceConf_t *inst) +std_checkRuleset_genErrMsg(__attribute__((unused)) modConfData_t *modConf, instanceConf_t *inst) { - errmsg.LogError(0, NO_ERRCODE, "imrelp: ruleset '%s' not found - " - "using default ruleset instead", modConf->pszBindRuleset); + errmsg.LogError(0, NO_ERRCODE, "imrelp[%s]: ruleset '%s' not found - " + "using default ruleset instead", + inst->pszBindPort, inst->pszBindRuleset); } -/* This function is called when a new listener instace shall be added to +/* This function is called when a new listener instance shall be added to * the current config object via the legacy config system. It just shuffles * all parameters to the listener in-memory instance. * rgerhards, 2011-05-04 @@ -194,9 +296,19 @@ static rsRetVal addInstance(void __attribute__((unused)) *pVal, uchar *pNewVal) if(pNewVal == NULL || *pNewVal == '\0') { errmsg.LogError(0, NO_ERRCODE, "imrelp: port number must be specified, listener ignored"); } - inst->pszBindPort = pNewVal; - + if((pNewVal == NULL) || (pNewVal == '\0')) { + inst->pszBindPort = NULL; + } else { + CHKmalloc(inst->pszBindPort = ustrdup(pNewVal)); + } + if((cs.pszBindRuleset == NULL) || (cs.pszBindRuleset[0] == '\0')) { + inst->pszBindRuleset = NULL; + } else { + CHKmalloc(inst->pszBindRuleset = ustrdup(cs.pszBindRuleset)); + } + inst->pBindRuleset = NULL; finalize_it: + free(pNewVal); RETiRet; } @@ -204,19 +316,88 @@ finalize_it: static rsRetVal addListner(modConfData_t __attribute__((unused)) *modConf, instanceConf_t *inst) { + relpSrv_t *pSrv; + int relpRet; + uchar statname[64]; + int i; DEFiRet; if(pRelpEngine == NULL) { CHKiRet(relpEngineConstruct(&pRelpEngine)); CHKiRet(relpEngineSetDbgprint(pRelpEngine, dbgprintf)); CHKiRet(relpEngineSetFamily(pRelpEngine, glbl.GetDefPFFamily())); CHKiRet(relpEngineSetEnableCmd(pRelpEngine, (uchar*) "syslog", eRelpCmdState_Required)); - CHKiRet(relpEngineSetSyslogRcv(pRelpEngine, onSyslogRcv)); + CHKiRet(relpEngineSetSyslogRcv2(pRelpEngine, onSyslogRcv)); + CHKiRet(relpEngineSetOnErr(pRelpEngine, onErr)); + CHKiRet(relpEngineSetOnGenericErr(pRelpEngine, onGenericErr)); + CHKiRet(relpEngineSetOnAuthErr(pRelpEngine, onAuthErr)); if (!glbl.GetDisableDNS()) { CHKiRet(relpEngineSetDnsLookupMode(pRelpEngine, 1)); } } - CHKiRet(relpEngineAddListner(pRelpEngine, inst->pszBindPort)); + CHKiRet(relpEngineListnerConstruct(pRelpEngine, &pSrv)); + CHKiRet(relpSrvSetLstnPort(pSrv, inst->pszBindPort)); + inst->pszInputName = ustrdup((inst->pszInputName == NULL) ? UCHAR_CONSTANT("imrelp") : inst->pszInputName); + CHKiRet(prop.Construct(&inst->pInputName)); + CHKiRet(prop.SetString(inst->pInputName, inst->pszInputName, ustrlen(inst->pszInputName))); + CHKiRet(prop.ConstructFinalize(inst->pInputName)); + /* support statistics gathering */ + CHKiRet(statsobj.Construct(&(inst->data.stats))); + snprintf((char*)statname, sizeof(statname), "imrelp[%s]", + inst->pszBindPort); + statname[sizeof(statname)-1] = '\0'; /* just to be on the save side... */ + CHKiRet(statsobj.SetName(inst->data.stats, statname)); + STATSCOUNTER_INIT(inst->data.ctrSubmit, inst->data.mutCtrSubmit); + CHKiRet(statsobj.AddCounter(inst->data.stats, UCHAR_CONSTANT("submitted"), + ctrType_IntCtr, CTR_FLAG_RESETTABLE, &(inst->data.ctrSubmit))); + CHKiRet(statsobj.ConstructFinalize(inst->data.stats)); + /* end stats counters */ + relpSrvSetUsrPtr(pSrv, inst); + relpSrvSetKeepAlive(pSrv, inst->bKeepAlive, inst->iKeepAliveIntvl, + inst->iKeepAliveProbes, inst->iKeepAliveTime); + if(inst->bEnableTLS) { + relpRet = relpSrvEnableTLS2(pSrv); + if(relpRet == RELP_RET_ERR_NO_TLS) { + errmsg.LogError(0, RS_RET_RELP_NO_TLS, + "imrelp: could not activate relp TLS, librelp " + "does not support it (most probably GnuTLS lib " + "is too old)!"); + ABORT_FINALIZE(RS_RET_RELP_NO_TLS); + } else if(relpRet != RELP_RET_OK) { + errmsg.LogError(0, RS_RET_RELP_ERR, + "imrelp: could not activate relp TLS, code %d", relpRet); + ABORT_FINALIZE(RS_RET_RELP_ERR); + } + if(inst->bEnableTLSZip) { + relpSrvEnableTLSZip2(pSrv); + } + if(inst->dhBits) { + relpSrvSetDHBits(pSrv, inst->dhBits); + } + relpSrvSetGnuTLSPriString(pSrv, (char*)inst->pristring); + if(relpSrvSetAuthMode(pSrv, (char*)inst->authmode) != RELP_RET_OK) { + errmsg.LogError(0, RS_RET_RELP_ERR, + "imrelp: invalid auth mode '%s'", inst->authmode); + ABORT_FINALIZE(RS_RET_RELP_ERR); + } + if(relpSrvSetCACert(pSrv, (char*) inst->caCertFile) != RELP_RET_OK) + ABORT_FINALIZE(RS_RET_RELP_ERR); + if(relpSrvSetOwnCert(pSrv, (char*) inst->myCertFile) != RELP_RET_OK) + ABORT_FINALIZE(RS_RET_RELP_ERR); + if(relpSrvSetPrivKey(pSrv, (char*) inst->myPrivKeyFile) != RELP_RET_OK) + ABORT_FINALIZE(RS_RET_RELP_ERR); + for(i = 0 ; i < inst->permittedPeers.nmemb ; ++i) { + relpSrvAddPermittedPeer(pSrv, (char*)inst->permittedPeers.name[i]); + } + } + relpRet = relpEngineListnerConstructFinalize(pRelpEngine, pSrv); + if(relpRet != RELP_RET_OK) { + errmsg.LogError(0, RS_RET_RELP_ERR, + "imrelp: could not activate relp listner, code %d", relpRet); + ABORT_FINALIZE(RS_RET_RELP_ERR); + } + + resetConfigVariables(NULL,NULL); finalize_it: RETiRet; @@ -226,7 +407,7 @@ finalize_it: BEGINnewInpInst struct cnfparamvals *pvals; instanceConf_t *inst; - int i; + int i,j; CODESTARTnewInpInst DBGPRINTF("newInpInst (imrelp)\n"); @@ -249,6 +430,41 @@ CODESTARTnewInpInst continue; if(!strcmp(inppblk.descr[i].name, "port")) { inst->pszBindPort = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + } else if(!strcmp(inppblk.descr[i].name, "name")) { + inst->pszInputName = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + } else if(!strcmp(inppblk.descr[i].name, "ruleset")) { + inst->pszBindRuleset = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + } else if(!strcmp(inppblk.descr[i].name, "keepalive")) { + inst->bKeepAlive = (sbool) pvals[i].val.d.n; + } else if(!strcmp(inppblk.descr[i].name, "keepalive.probes")) { + inst->iKeepAliveProbes = (int) pvals[i].val.d.n; + } else if(!strcmp(inppblk.descr[i].name, "keepalive.time")) { + inst->iKeepAliveTime = (int) pvals[i].val.d.n; + } else if(!strcmp(inppblk.descr[i].name, "keepalive.interval")) { + inst->iKeepAliveIntvl = (int) pvals[i].val.d.n; + } else if(!strcmp(inppblk.descr[i].name, "tls")) { + inst->bEnableTLS = (unsigned) pvals[i].val.d.n; + } else if(!strcmp(inppblk.descr[i].name, "tls.dhbits")) { + inst->dhBits = (unsigned) pvals[i].val.d.n; + } else if(!strcmp(inppblk.descr[i].name, "tls.prioritystring")) { + inst->pristring = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + } else if(!strcmp(inppblk.descr[i].name, "tls.authmode")) { + inst->authmode = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + } else if(!strcmp(inppblk.descr[i].name, "tls.compression")) { + inst->bEnableTLSZip = (unsigned) pvals[i].val.d.n; + } else if(!strcmp(inppblk.descr[i].name, "tls.cacert")) { + inst->caCertFile = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + } else if(!strcmp(inppblk.descr[i].name, "tls.mycert")) { + inst->myCertFile = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + } else if(!strcmp(inppblk.descr[i].name, "tls.myprivkey")) { + inst->myPrivKeyFile = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + } else if(!strcmp(inppblk.descr[i].name, "tls.permittedpeer")) { + inst->permittedPeers.nmemb = pvals[i].val.d.ar->nmemb; + CHKmalloc(inst->permittedPeers.name = + malloc(sizeof(uchar*) * inst->permittedPeers.nmemb)); + for(j = 0 ; j < pvals[i].val.d.ar->nmemb ; ++j) { + inst->permittedPeers.name[j] = (uchar*)es_str2cstr(pvals[i].val.d.ar->arr[j], NULL); + } } else { dbgprintf("imrelp: program error, non-handled " "param '%s'\n", inppblk.descr[i].name); @@ -264,41 +480,76 @@ BEGINbeginCnfLoad CODESTARTbeginCnfLoad loadModConf = pModConf; pModConf->pConf = pConf; + pModConf->pszBindRuleset = NULL; /* init legacy config variables */ cs.pszBindRuleset = NULL; + bLegacyCnfModGlobalsPermitted = 1; ENDbeginCnfLoad +BEGINsetModCnf + struct cnfparamvals *pvals = NULL; + int i; +CODESTARTsetModCnf + pvals = nvlstGetParams(lst, &modpblk, NULL); + if(pvals == NULL) { + errmsg.LogError(0, RS_RET_MISSING_CNFPARAMS, "error processing module " + "config parameters [module(...)]"); + ABORT_FINALIZE(RS_RET_MISSING_CNFPARAMS); + } + + if(Debug) { + dbgprintf("module (global) param blk for imrelp:\n"); + cnfparamsPrint(&modpblk, pvals); + } + + for(i = 0 ; i < modpblk.nParams ; ++i) { + if(!pvals[i].bUsed) + continue; + if(!strcmp(modpblk.descr[i].name, "ruleset")) { + loadModConf->pszBindRuleset = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + } else { + dbgprintf("imrelp: program error, non-handled " + "param '%s' in beginCnfLoad\n", modpblk.descr[i].name); + } + } + /* remove all of our legacy module handlers, as they can not used in addition + * the the new-style config method. + */ + bLegacyCnfModGlobalsPermitted = 0; +finalize_it: + if(pvals != NULL) + cnfparamvalsDestruct(pvals, &modpblk); +ENDsetModCnf + BEGINendCnfLoad CODESTARTendCnfLoad - if((cs.pszBindRuleset == NULL) || (cs.pszBindRuleset[0] == '\0')) { - loadModConf->pszBindRuleset = NULL; + if(loadModConf->pszBindRuleset == NULL) { + if((cs.pszBindRuleset == NULL) || (cs.pszBindRuleset[0] == '\0')) { + loadModConf->pszBindRuleset = NULL; + } else { + CHKmalloc(loadModConf->pszBindRuleset = ustrdup(cs.pszBindRuleset)); + } } else { - CHKmalloc(loadModConf->pszBindRuleset = ustrdup(cs.pszBindRuleset)); + if((cs.pszBindRuleset != NULL) && (cs.pszBindRuleset[0] != '\0')) { + errmsg.LogError(0, RS_RET_DUP_PARAM, "imrelp: warning: ruleset " + "set via legacy directive ignored"); + } } - loadModConf->pBindRuleset = NULL; finalize_it: free(cs.pszBindRuleset); + cs.pszBindRuleset = NULL; loadModConf = NULL; /* done loading */ ENDendCnfLoad - BEGINcheckCnf - rsRetVal localRet; - ruleset_t *pRuleset; + instanceConf_t *inst; CODESTARTcheckCnf - /* we emulate the standard "ruleset query" code provided by the framework - * for *instances* (which we can currently not support due to librelp). - */ - if(pModConf->pszBindRuleset == NULL) { - pModConf->pBindRuleset = NULL; - } else { - localRet = ruleset.GetRuleset(pModConf->pConf, &pRuleset, pModConf->pszBindRuleset); - if(localRet == RS_RET_NOT_FOUND) { - std_checkRuleset_genErrMsg(pModConf, NULL); + for(inst = pModConf->root ; inst != NULL ; inst = inst->next) { + if(inst->pszBindRuleset == NULL && pModConf->pszBindRuleset != NULL) { + CHKmalloc(inst->pszBindRuleset = ustrdup(pModConf->pszBindRuleset)); } - CHKiRet(localRet); - pModConf->pBindRuleset = pRuleset; + std_checkRuleset(pModConf, inst); } finalize_it: ENDcheckCnf @@ -311,8 +562,10 @@ CODESTARTactivateCnfPrePrivDrop for(inst = runModConf->root ; inst != NULL ; inst = inst->next) { addListner(pModConf, inst); } - if(pRelpEngine == NULL) + if(pRelpEngine == NULL) { + errmsg.LogError(0, RS_RET_NO_LSTN_DEFINED, "imrelp: no RELP listener defined, module can not run."); ABORT_FINALIZE(RS_RET_NO_RUN); + } finalize_it: ENDactivateCnfPrePrivDrop @@ -323,13 +576,23 @@ ENDactivateCnf BEGINfreeCnf instanceConf_t *inst, *del; + int i; CODESTARTfreeCnf for(inst = pModConf->root ; inst != NULL ; ) { free(inst->pszBindPort); + free(inst->pszBindRuleset); + free(inst->pszInputName); + free(inst->pristring); + free(inst->authmode); + statsobj.Destruct(&(inst->data.stats)); + for(i = 0 ; i < inst->permittedPeers.nmemb ; ++i) { + free(inst->permittedPeers.name[i]); + } del = inst; inst = inst->next; free(del); } + free(pModConf->pszBindRuleset); ENDfreeCnf /* This is used to terminate the plugin. Note that the signal handler blocks @@ -385,11 +648,8 @@ CODESTARTmodExit if(pRelpEngine != NULL) iRet = relpEngineDestruct(&pRelpEngine); - /* global variable cleanup */ - if(pInputName != NULL) - prop.Destruct(&pInputName); - /* release objects we used */ + objRelease(statsobj, CORE_COMPONENT); objRelease(ruleset, CORE_COMPONENT); objRelease(glbl, CORE_COMPONENT); objRelease(prop, CORE_COMPONENT); @@ -420,6 +680,7 @@ CODEqueryEtryPt_STD_IMOD_QUERIES CODEqueryEtryPt_STD_CONF2_QUERIES CODEqueryEtryPt_STD_CONF2_PREPRIVDROP_QUERIES CODEqueryEtryPt_STD_CONF2_IMOD_QUERIES +CODEqueryEtryPt_STD_CONF2_setModCnf_QUERIES CODEqueryEtryPt_IsCompatibleWithFeature_IF_OMOD_QUERIES ENDqueryEtryPt @@ -435,19 +696,15 @@ CODEmodInit_QueryRegCFSLineHdlr CHKiRet(objUse(errmsg, CORE_COMPONENT)); CHKiRet(objUse(net, LM_NET_FILENAME)); CHKiRet(objUse(ruleset, CORE_COMPONENT)); + CHKiRet(objUse(statsobj, CORE_COMPONENT)); /* register config file handlers */ - CHKiRet(omsdRegCFSLineHdlr((uchar *)"inputrelpserverbindruleset", 0, eCmdHdlrGetWord, - NULL, &cs.pszBindRuleset, STD_LOADABLE_MODULE_ID)); + CHKiRet(regCfSysLineHdlr2((uchar*)"inputrelpserverbindruleset", 0, eCmdHdlrGetWord, + NULL, &cs.pszBindRuleset, STD_LOADABLE_MODULE_ID, &bLegacyCnfModGlobalsPermitted)); CHKiRet(omsdRegCFSLineHdlr((uchar *)"inputrelpserverrun", 0, eCmdHdlrGetWord, addInstance, NULL, STD_LOADABLE_MODULE_ID)); CHKiRet(omsdRegCFSLineHdlr((uchar *)"resetconfigvariables", 1, eCmdHdlrCustomHandler, resetConfigVariables, NULL, STD_LOADABLE_MODULE_ID)); - - /* we need to create the inputName property (only once during our lifetime) */ - CHKiRet(prop.Construct(&pInputName)); - CHKiRet(prop.SetString(pInputName, UCHAR_CONSTANT("imrelp"), sizeof("imrelp") - 1)); - CHKiRet(prop.ConstructFinalize(pInputName)); ENDmodInit diff --git a/plugins/imsolaris/Makefile.in b/plugins/imsolaris/Makefile.in index 0ff286d..d8173c2 100644 --- a/plugins/imsolaris/Makefile.in +++ b/plugins/imsolaris/Makefile.in @@ -154,7 +154,6 @@ GREP = @GREP@ GSS_LIBS = @GSS_LIBS@ GUARDTIME_CFLAGS = @GUARDTIME_CFLAGS@ GUARDTIME_LIBS = @GUARDTIME_LIBS@ -HAVE_LIBGCRYPT_CONFIG = @HAVE_LIBGCRYPT_CONFIG@ HAVE_MYSQL_CONFIG = @HAVE_MYSQL_CONFIG@ HAVE_ORACLE_CONFIG = @HAVE_ORACLE_CONFIG@ HAVE_PGSQL_CONFIG = @HAVE_PGSQL_CONFIG@ @@ -175,14 +174,15 @@ LEXLIB = @LEXLIB@ LEX_OUTPUT_ROOT = @LEX_OUTPUT_ROOT@ LIBDBI_CFLAGS = @LIBDBI_CFLAGS@ LIBDBI_LIBS = @LIBDBI_LIBS@ -LIBEE_CFLAGS = @LIBEE_CFLAGS@ -LIBEE_LIBS = @LIBEE_LIBS@ LIBESTR_CFLAGS = @LIBESTR_CFLAGS@ LIBESTR_LIBS = @LIBESTR_LIBS@ LIBGCRYPT_CFLAGS = @LIBGCRYPT_CFLAGS@ +LIBGCRYPT_CONFIG = @LIBGCRYPT_CONFIG@ LIBGCRYPT_LIBS = @LIBGCRYPT_LIBS@ LIBLOGGING_CFLAGS = @LIBLOGGING_CFLAGS@ LIBLOGGING_LIBS = @LIBLOGGING_LIBS@ +LIBLOGGING_STDLOG_CFLAGS = @LIBLOGGING_STDLOG_CFLAGS@ +LIBLOGGING_STDLOG_LIBS = @LIBLOGGING_STDLOG_LIBS@ LIBLOGNORM_CFLAGS = @LIBLOGNORM_CFLAGS@ LIBLOGNORM_LIBS = @LIBLOGNORM_LIBS@ LIBM = @LIBM@ @@ -207,6 +207,8 @@ NM = @NM@ NMEDIT = @NMEDIT@ OBJDUMP = @OBJDUMP@ OBJEXT = @OBJEXT@ +OPENSSL_CFLAGS = @OPENSSL_CFLAGS@ +OPENSSL_LIBS = @OPENSSL_LIBS@ ORACLE_CFLAGS = @ORACLE_CFLAGS@ ORACLE_LIBS = @ORACLE_LIBS@ OTOOL = @OTOOL@ @@ -388,22 +390,25 @@ distclean-compile: @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/imsolaris_la-sun_cddl.Plo@am__quote@ .c.o: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(COMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ $< -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Po +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.o$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c $< +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ $< .c.obj: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(COMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ `$(CYGPATH_W) '$<'` -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Po +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.obj$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ `$(CYGPATH_W) '$<'` &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c `$(CYGPATH_W) '$<'` +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ `$(CYGPATH_W) '$<'` .c.lo: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(LTCOMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ $< -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Plo +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.lo$$||'`;\ +@am__fastdepCC_TRUE@ $(LTCOMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Plo @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=yes @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ @am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(LTCOMPILE) -c -o $@ $< diff --git a/plugins/imsolaris/imsolaris.c b/plugins/imsolaris/imsolaris.c index a220e72..36677e7 100644 --- a/plugins/imsolaris/imsolaris.c +++ b/plugins/imsolaris/imsolaris.c @@ -209,8 +209,8 @@ readLog(int fd, uchar *pRcv, int iMaxLine) MsgSetInputName(pMsg, pInputName); MsgSetRawMsg(pMsg, (char*)pRcv, strlen((char*)pRcv)); MsgSetHOSTNAME(pMsg, glbl.GetLocalHostName(), ustrlen(glbl.GetLocalHostName())); - pMsg->iFacility = LOG_FAC(hdr.pri); - pMsg->iSeverity = LOG_PRI(hdr.pri); + pMsg->iFacility = pri2fac(hdr.pri); + pMsg->iSeverity = pri2sev(hdr.pri); pMsg->msgFlags = NEEDS_PARSING | NO_PRI_IN_RAW; CHKiRet(submitMsg(pMsg)); } diff --git a/plugins/imsolaris/sun_cddl.c b/plugins/imsolaris/sun_cddl.c index 6d49c8b..e7f3fa7 100644 --- a/plugins/imsolaris/sun_cddl.c +++ b/plugins/imsolaris/sun_cddl.c @@ -18,7 +18,7 @@ * * CDDL HEADER END */ -/* Portions Copyright 2010 by Rainer Gerhards and Adiscon +/* Portions Copyright 2010-2014 by Rainer Gerhards and Adiscon */ /* * Copyright 2008 Sun Microsystems, Inc. All rights reserved. @@ -39,6 +39,7 @@ * software developed by the University of California, Berkeley, and its * contributors. */ +#include "config.h" #include <unistd.h> #include <errno.h> #include <sys/types.h> diff --git a/plugins/imtcp/Makefile.in b/plugins/imtcp/Makefile.in index 454a265..b1faba1 100644 --- a/plugins/imtcp/Makefile.in +++ b/plugins/imtcp/Makefile.in @@ -153,7 +153,6 @@ GREP = @GREP@ GSS_LIBS = @GSS_LIBS@ GUARDTIME_CFLAGS = @GUARDTIME_CFLAGS@ GUARDTIME_LIBS = @GUARDTIME_LIBS@ -HAVE_LIBGCRYPT_CONFIG = @HAVE_LIBGCRYPT_CONFIG@ HAVE_MYSQL_CONFIG = @HAVE_MYSQL_CONFIG@ HAVE_ORACLE_CONFIG = @HAVE_ORACLE_CONFIG@ HAVE_PGSQL_CONFIG = @HAVE_PGSQL_CONFIG@ @@ -174,14 +173,15 @@ LEXLIB = @LEXLIB@ LEX_OUTPUT_ROOT = @LEX_OUTPUT_ROOT@ LIBDBI_CFLAGS = @LIBDBI_CFLAGS@ LIBDBI_LIBS = @LIBDBI_LIBS@ -LIBEE_CFLAGS = @LIBEE_CFLAGS@ -LIBEE_LIBS = @LIBEE_LIBS@ LIBESTR_CFLAGS = @LIBESTR_CFLAGS@ LIBESTR_LIBS = @LIBESTR_LIBS@ LIBGCRYPT_CFLAGS = @LIBGCRYPT_CFLAGS@ +LIBGCRYPT_CONFIG = @LIBGCRYPT_CONFIG@ LIBGCRYPT_LIBS = @LIBGCRYPT_LIBS@ LIBLOGGING_CFLAGS = @LIBLOGGING_CFLAGS@ LIBLOGGING_LIBS = @LIBLOGGING_LIBS@ +LIBLOGGING_STDLOG_CFLAGS = @LIBLOGGING_STDLOG_CFLAGS@ +LIBLOGGING_STDLOG_LIBS = @LIBLOGGING_STDLOG_LIBS@ LIBLOGNORM_CFLAGS = @LIBLOGNORM_CFLAGS@ LIBLOGNORM_LIBS = @LIBLOGNORM_LIBS@ LIBM = @LIBM@ @@ -206,6 +206,8 @@ NM = @NM@ NMEDIT = @NMEDIT@ OBJDUMP = @OBJDUMP@ OBJEXT = @OBJEXT@ +OPENSSL_CFLAGS = @OPENSSL_CFLAGS@ +OPENSSL_LIBS = @OPENSSL_LIBS@ ORACLE_CFLAGS = @ORACLE_CFLAGS@ ORACLE_LIBS = @ORACLE_LIBS@ OTOOL = @OTOOL@ @@ -386,22 +388,25 @@ distclean-compile: @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/imtcp_la-imtcp.Plo@am__quote@ .c.o: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(COMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ $< -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Po +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.o$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c $< +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ $< .c.obj: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(COMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ `$(CYGPATH_W) '$<'` -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Po +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.obj$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ `$(CYGPATH_W) '$<'` &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c `$(CYGPATH_W) '$<'` +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ `$(CYGPATH_W) '$<'` .c.lo: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(LTCOMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ $< -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Plo +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.lo$$||'`;\ +@am__fastdepCC_TRUE@ $(LTCOMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Plo @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=yes @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ @am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(LTCOMPILE) -c -o $@ $< diff --git a/plugins/imtcp/imtcp.c b/plugins/imtcp/imtcp.c index fc22d45..5169757 100644 --- a/plugins/imtcp/imtcp.c +++ b/plugins/imtcp/imtcp.c @@ -4,7 +4,7 @@ * File begun on 2007-12-21 by RGerhards (extracted from syslogd.c, * which at the time of the rsyslog fork was BSD-licensed) * - * Copyright 2007-2012 Adiscon GmbH. + * Copyright 2007-2013 Adiscon GmbH. * * This file is part of rsyslog. * @@ -105,6 +105,7 @@ struct instanceConf_s { uchar *pszBindRuleset; /* name of ruleset to bind to */ ruleset_t *pBindRuleset; /* ruleset to bind listener to (use system default if unspecified) */ uchar *pszInputName; /* value for inputname property, NULL is OK and handled by core engine */ + uchar *dfltTZ; int ratelimitInterval; int ratelimitBurst; int bSuppOctetFram; @@ -124,6 +125,7 @@ struct modConfData_s { sbool bUseFlowControl; /* use flow control, what means indicate ourselfs a "light delayable" */ sbool bKeepAlive; sbool bEmitMsgOnClose; /* emit an informational message on close by remote peer */ + uchar *pszStrmDrvrName; /* stream driver to use */ uchar *pszStrmDrvrAuthMode; /* authentication mode to use */ struct cnfarray *permittedPeers; sbool configSetViaV2Method; @@ -138,11 +140,13 @@ static struct cnfparamdescr modpdescr[] = { { "disablelfdelimiter", eCmdHdlrBinary, 0 }, { "octetcountedframing", eCmdHdlrBinary, 0 }, { "notifyonconnectionclose", eCmdHdlrBinary, 0 }, - { "addtlframedelimiter", eCmdHdlrPositiveInt, 0 }, + { "addtlframedelimiter", eCmdHdlrNonNegInt, 0 }, { "maxsessions", eCmdHdlrPositiveInt, 0 }, { "maxlistners", eCmdHdlrPositiveInt, 0 }, + { "maxlisteners", eCmdHdlrPositiveInt, 0 }, { "streamdriver.mode", eCmdHdlrPositiveInt, 0 }, { "streamdriver.authmode", eCmdHdlrString, 0 }, + { "streamdriver.name", eCmdHdlrString, 0 }, { "permittedpeer", eCmdHdlrArray, 0 }, { "keepalive", eCmdHdlrBinary, 0 } }; @@ -156,6 +160,7 @@ static struct cnfparamblk modpblk = static struct cnfparamdescr inppdescr[] = { { "port", eCmdHdlrString, CNFPARAM_REQUIRED }, /* legacy: InputTCPServerRun */ { "name", eCmdHdlrString, 0 }, + { "defaulttz", eCmdHdlrString, 0 }, { "ruleset", eCmdHdlrString, 0 }, { "supportOctetCountedFraming", eCmdHdlrBinary, 0 }, { "ratelimit.interval", eCmdHdlrInt, 0 }, @@ -254,6 +259,7 @@ createInstance(instanceConf_t **pinst) inst->next = NULL; inst->pszBindRuleset = NULL; inst->pszInputName = NULL; + inst->dfltTZ = NULL; inst->bSuppOctetFram = 1; inst->ratelimitInterval = 0; inst->ratelimitBurst = 10000; @@ -327,6 +333,9 @@ addListner(modConfData_t *modConf, instanceConf_t *inst) CHKiRet(tcpsrv.SetbDisableLFDelim(pOurTcpsrv, modConf->bDisableLFDelim)); CHKiRet(tcpsrv.SetNotificationOnRemoteClose(pOurTcpsrv, modConf->bEmitMsgOnClose)); /* now set optional params, but only if they were actually configured */ + if(modConf->pszStrmDrvrName != NULL) { + CHKiRet(tcpsrv.SetDrvrName(pOurTcpsrv, modConf->pszStrmDrvrName)); + } if(modConf->pszStrmDrvrAuthMode != NULL) { CHKiRet(tcpsrv.SetDrvrAuthMode(pOurTcpsrv, modConf->pszStrmDrvrAuthMode)); } @@ -340,6 +349,7 @@ addListner(modConfData_t *modConf, instanceConf_t *inst) CHKiRet(tcpsrv.SetRuleset(pOurTcpsrv, inst->pBindRuleset)); CHKiRet(tcpsrv.SetInputName(pOurTcpsrv, inst->pszInputName == NULL ? UCHAR_CONSTANT("imtcp") : inst->pszInputName)); + CHKiRet(tcpsrv.SetDfltTZ(pOurTcpsrv, (inst->dfltTZ == NULL) ? (uchar*)"" : inst->dfltTZ)); CHKiRet(tcpsrv.SetLinuxLikeRatelimiters(pOurTcpsrv, inst->ratelimitInterval, inst->ratelimitBurst)); tcpsrv.configureTCPListen(pOurTcpsrv, inst->pszBindPort, inst->bSuppOctetFram); @@ -379,6 +389,8 @@ CODESTARTnewInpInst inst->pszBindPort = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); } else if(!strcmp(inppblk.descr[i].name, "name")) { inst->pszInputName = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + } else if(!strcmp(inppblk.descr[i].name, "defaulttz")) { + inst->dfltTZ = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); } else if(!strcmp(inppblk.descr[i].name, "ruleset")) { inst->pszBindRuleset = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); } else if(!strcmp(inppblk.descr[i].name, "supportOctetCountedFraming")) { @@ -407,11 +419,12 @@ CODESTARTbeginCnfLoad loadModConf->iTCPLstnMax = 20; loadModConf->bSuppOctetFram = 1; loadModConf->iStrmDrvrMode = 0; - loadModConf->bUseFlowControl = 0; + loadModConf->bUseFlowControl = 1; loadModConf->bKeepAlive = 0; loadModConf->bEmitMsgOnClose = 0; loadModConf->iAddtlFrameDelim = TCPSRV_NO_ADDTL_DELIMITER; loadModConf->bDisableLFDelim = 0; + loadModConf->pszStrmDrvrName = NULL; loadModConf->pszStrmDrvrAuthMode = NULL; loadModConf->permittedPeers = NULL; loadModConf->configSetViaV2Method = 0; @@ -453,7 +466,8 @@ CODESTARTsetModCnf loadModConf->iAddtlFrameDelim = (int) pvals[i].val.d.n; } else if(!strcmp(modpblk.descr[i].name, "maxsessions")) { loadModConf->iTCPSessMax = (int) pvals[i].val.d.n; - } else if(!strcmp(modpblk.descr[i].name, "maxlistners")) { + } else if(!strcmp(modpblk.descr[i].name, "maxlisteners") || + !strcmp(modpblk.descr[i].name, "maxlistners")) { /* keep old name for a while */ loadModConf->iTCPLstnMax = (int) pvals[i].val.d.n; } else if(!strcmp(modpblk.descr[i].name, "keepalive")) { loadModConf->bKeepAlive = (int) pvals[i].val.d.n; @@ -461,6 +475,8 @@ CODESTARTsetModCnf loadModConf->iStrmDrvrMode = (int) pvals[i].val.d.n; } else if(!strcmp(modpblk.descr[i].name, "streamdriver.authmode")) { loadModConf->pszStrmDrvrAuthMode = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + } else if(!strcmp(modpblk.descr[i].name, "streamdriver.name")) { + loadModConf->pszStrmDrvrName = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); } else if(!strcmp(modpblk.descr[i].name, "permittedpeer")) { loadModConf->permittedPeers = cnfarrayDup(pvals[i].val.d.ar); } else { @@ -561,6 +577,7 @@ ENDactivateCnf BEGINfreeCnf instanceConf_t *inst, *del; CODESTARTfreeCnf + free(pModConf->pszStrmDrvrName); free(pModConf->pszStrmDrvrAuthMode); if(pModConf->permittedPeers != NULL) { cnfarrayContentDestruct(pModConf->permittedPeers); @@ -569,6 +586,7 @@ CODESTARTfreeCnf for(inst = pModConf->root ; inst != NULL ; ) { free(inst->pszBindPort); free(inst->pszInputName); + free(inst->dfltTZ); del = inst; inst = inst->next; free(del); @@ -629,7 +647,7 @@ resetConfigVariables(uchar __attribute__((unused)) *pp, void __attribute__((unus cs.iTCPLstnMax = 20; cs.bSuppOctetFram = 1; cs.iStrmDrvrMode = 0; - cs.bUseFlowControl = 0; + cs.bUseFlowControl = 1; cs.bKeepAlive = 0; cs.bEmitMsgOnClose = 0; cs.iAddtlFrameDelim = TCPSRV_NO_ADDTL_DELIMITER; diff --git a/plugins/imttcp/Makefile.in b/plugins/imttcp/Makefile.in index e46d222..eef1989 100644 --- a/plugins/imttcp/Makefile.in +++ b/plugins/imttcp/Makefile.in @@ -153,7 +153,6 @@ GREP = @GREP@ GSS_LIBS = @GSS_LIBS@ GUARDTIME_CFLAGS = @GUARDTIME_CFLAGS@ GUARDTIME_LIBS = @GUARDTIME_LIBS@ -HAVE_LIBGCRYPT_CONFIG = @HAVE_LIBGCRYPT_CONFIG@ HAVE_MYSQL_CONFIG = @HAVE_MYSQL_CONFIG@ HAVE_ORACLE_CONFIG = @HAVE_ORACLE_CONFIG@ HAVE_PGSQL_CONFIG = @HAVE_PGSQL_CONFIG@ @@ -174,14 +173,15 @@ LEXLIB = @LEXLIB@ LEX_OUTPUT_ROOT = @LEX_OUTPUT_ROOT@ LIBDBI_CFLAGS = @LIBDBI_CFLAGS@ LIBDBI_LIBS = @LIBDBI_LIBS@ -LIBEE_CFLAGS = @LIBEE_CFLAGS@ -LIBEE_LIBS = @LIBEE_LIBS@ LIBESTR_CFLAGS = @LIBESTR_CFLAGS@ LIBESTR_LIBS = @LIBESTR_LIBS@ LIBGCRYPT_CFLAGS = @LIBGCRYPT_CFLAGS@ +LIBGCRYPT_CONFIG = @LIBGCRYPT_CONFIG@ LIBGCRYPT_LIBS = @LIBGCRYPT_LIBS@ LIBLOGGING_CFLAGS = @LIBLOGGING_CFLAGS@ LIBLOGGING_LIBS = @LIBLOGGING_LIBS@ +LIBLOGGING_STDLOG_CFLAGS = @LIBLOGGING_STDLOG_CFLAGS@ +LIBLOGGING_STDLOG_LIBS = @LIBLOGGING_STDLOG_LIBS@ LIBLOGNORM_CFLAGS = @LIBLOGNORM_CFLAGS@ LIBLOGNORM_LIBS = @LIBLOGNORM_LIBS@ LIBM = @LIBM@ @@ -206,6 +206,8 @@ NM = @NM@ NMEDIT = @NMEDIT@ OBJDUMP = @OBJDUMP@ OBJEXT = @OBJEXT@ +OPENSSL_CFLAGS = @OPENSSL_CFLAGS@ +OPENSSL_LIBS = @OPENSSL_LIBS@ ORACLE_CFLAGS = @ORACLE_CFLAGS@ ORACLE_LIBS = @ORACLE_LIBS@ OTOOL = @OTOOL@ @@ -386,22 +388,25 @@ distclean-compile: @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/imttcp_la-imttcp.Plo@am__quote@ .c.o: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(COMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ $< -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Po +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.o$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c $< +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ $< .c.obj: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(COMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ `$(CYGPATH_W) '$<'` -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Po +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.obj$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ `$(CYGPATH_W) '$<'` &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c `$(CYGPATH_W) '$<'` +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ `$(CYGPATH_W) '$<'` .c.lo: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(LTCOMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ $< -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Plo +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.lo$$||'`;\ +@am__fastdepCC_TRUE@ $(LTCOMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Plo @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=yes @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ @am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(LTCOMPILE) -c -o $@ $< diff --git a/plugins/imttcp/imttcp.c b/plugins/imttcp/imttcp.c index 9bd11f7..4bd44dd 100644 --- a/plugins/imttcp/imttcp.c +++ b/plugins/imttcp/imttcp.c @@ -589,13 +589,13 @@ processDataRcvd(ttcpsess_t *pThis, char c, struct syslogTime *stTime, time_t ttG DBGPRINTF("TCP Message with octet-counter, size %d.\n", pThis->iOctetsRemain); if(c != ' ') { errmsg.LogError(0, NO_ERRCODE, "Framing Error in received TCP message: " - "delimiter is not SP but has ASCII value %d.\n", c); + "delimiter is not SP but has ASCII value %d.", c); } if(pThis->iOctetsRemain < 1) { /* TODO: handle the case where the octet count is 0! */ DBGPRINTF("Framing Error: invalid octet count\n"); errmsg.LogError(0, NO_ERRCODE, "Framing Error in received TCP message: " - "invalid octet count %d.\n", pThis->iOctetsRemain); + "invalid octet count %d.", pThis->iOctetsRemain); } else if(pThis->iOctetsRemain > iMaxLine) { /* while we can not do anything against it, we can at least log an indication * that something went wrong) -- rgerhards, 2008-03-14 @@ -603,7 +603,7 @@ processDataRcvd(ttcpsess_t *pThis, char c, struct syslogTime *stTime, time_t ttG DBGPRINTF("truncating message with %d octets - max msg size is %d\n", pThis->iOctetsRemain, iMaxLine); errmsg.LogError(0, NO_ERRCODE, "received oversize message: size is %d bytes, " - "max msg size is %d, truncating...\n", pThis->iOctetsRemain, iMaxLine); + "max msg size is %d, truncating...", pThis->iOctetsRemain, iMaxLine); } pThis->inputState = eInMsg; } @@ -953,7 +953,7 @@ sessThrd(void *arg) uchar *peerName; int lenPeer; prop.GetString(pSess->peerName, &peerName, &lenPeer); - errmsg.LogError(0, RS_RET_PEER_CLOSED_CONN, "imttcp session %d closed by remote peer %s.\n", + errmsg.LogError(0, RS_RET_PEER_CLOSED_CONN, "imttcp session %d closed by remote peer %s.", pSess->sock, peerName); } break; diff --git a/plugins/imudp/Makefile.in b/plugins/imudp/Makefile.in index 93898db..756251b 100644 --- a/plugins/imudp/Makefile.in +++ b/plugins/imudp/Makefile.in @@ -154,7 +154,6 @@ GREP = @GREP@ GSS_LIBS = @GSS_LIBS@ GUARDTIME_CFLAGS = @GUARDTIME_CFLAGS@ GUARDTIME_LIBS = @GUARDTIME_LIBS@ -HAVE_LIBGCRYPT_CONFIG = @HAVE_LIBGCRYPT_CONFIG@ HAVE_MYSQL_CONFIG = @HAVE_MYSQL_CONFIG@ HAVE_ORACLE_CONFIG = @HAVE_ORACLE_CONFIG@ HAVE_PGSQL_CONFIG = @HAVE_PGSQL_CONFIG@ @@ -175,14 +174,15 @@ LEXLIB = @LEXLIB@ LEX_OUTPUT_ROOT = @LEX_OUTPUT_ROOT@ LIBDBI_CFLAGS = @LIBDBI_CFLAGS@ LIBDBI_LIBS = @LIBDBI_LIBS@ -LIBEE_CFLAGS = @LIBEE_CFLAGS@ -LIBEE_LIBS = @LIBEE_LIBS@ LIBESTR_CFLAGS = @LIBESTR_CFLAGS@ LIBESTR_LIBS = @LIBESTR_LIBS@ LIBGCRYPT_CFLAGS = @LIBGCRYPT_CFLAGS@ +LIBGCRYPT_CONFIG = @LIBGCRYPT_CONFIG@ LIBGCRYPT_LIBS = @LIBGCRYPT_LIBS@ LIBLOGGING_CFLAGS = @LIBLOGGING_CFLAGS@ LIBLOGGING_LIBS = @LIBLOGGING_LIBS@ +LIBLOGGING_STDLOG_CFLAGS = @LIBLOGGING_STDLOG_CFLAGS@ +LIBLOGGING_STDLOG_LIBS = @LIBLOGGING_STDLOG_LIBS@ LIBLOGNORM_CFLAGS = @LIBLOGNORM_CFLAGS@ LIBLOGNORM_LIBS = @LIBLOGNORM_LIBS@ LIBM = @LIBM@ @@ -207,6 +207,8 @@ NM = @NM@ NMEDIT = @NMEDIT@ OBJDUMP = @OBJDUMP@ OBJEXT = @OBJEXT@ +OPENSSL_CFLAGS = @OPENSSL_CFLAGS@ +OPENSSL_LIBS = @OPENSSL_LIBS@ ORACLE_CFLAGS = @ORACLE_CFLAGS@ ORACLE_LIBS = @ORACLE_LIBS@ OTOOL = @OTOOL@ @@ -387,22 +389,25 @@ distclean-compile: @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/imudp_la-imudp.Plo@am__quote@ .c.o: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(COMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ $< -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Po +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.o$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c $< +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ $< .c.obj: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(COMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ `$(CYGPATH_W) '$<'` -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Po +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.obj$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ `$(CYGPATH_W) '$<'` &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c `$(CYGPATH_W) '$<'` +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ `$(CYGPATH_W) '$<'` .c.lo: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(LTCOMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ $< -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Plo +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.lo$$||'`;\ +@am__fastdepCC_TRUE@ $(LTCOMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Plo @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=yes @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ @am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(LTCOMPILE) -c -o $@ $< diff --git a/plugins/imudp/imudp.c b/plugins/imudp/imudp.c index 7bf1473..f087196 100644 --- a/plugins/imudp/imudp.c +++ b/plugins/imudp/imudp.c @@ -4,7 +4,7 @@ * NOTE: read comments in module-template.h to understand how this file * works! * - * Copyright 2007-2012 Rainer Gerhards and Adiscon GmbH. + * Copyright 2007-2014 Rainer Gerhards and Adiscon GmbH. * * This file is part of rsyslog. * @@ -31,6 +31,8 @@ #include <errno.h> #include <unistd.h> #include <netdb.h> +#include <sys/socket.h> +#include <pthread.h> #if HAVE_SYS_EPOLL_H # include <sys/epoll.h> #endif @@ -59,6 +61,7 @@ MODULE_TYPE_NOKEEP MODULE_CNFNAME("imudp") /* defines */ +#define MAX_WRKR_THREADS 32 /* Module static data */ DEF_IMOD_STATIC_DATA @@ -78,9 +81,11 @@ static struct lstn_s { prop_t *pInputName; statsobj_t *stats; /* listener stats */ ratelimit_t *ratelimiter; + uchar *dfltTZ; STATSCOUNTER_DEF(ctrSubmit, mutCtrSubmit) } *lcnfRoot = NULL, *lcnfLast = NULL; + static int bLegacyCnfModGlobalsPermitted;/* are legacy module-global config parameters permitted? */ static int bDoACLCheck; /* are ACL checks neeed? Cached once immediately before listener startup */ static int iMaxLine; /* maximum UDP message size supported */ @@ -88,11 +93,7 @@ static time_t ttLastDiscard = 0; /* timestamp when a message from a non-permitte * This shall prevent remote DoS when the "discard on disallowed sender" * message is configured to be logged on occurance of such a case. */ -static uchar *pRcvBuf = NULL; /* receive buffer (for a single packet). We use a global and alloc - * it so that we can check available memory in willRun() and request - * termination if we can not get it. -- rgerhards, 2007-12-27 - */ - +#define BATCH_SIZE_DFLT 32 /* do not overdo, has heavy toll on memory, especially with large msgs */ #define TIME_REQUERY_DFLT 2 #define SCHED_PRIO_UNSET -12345678 /* a value that indicates that the scheduling priority has not been set */ /* config vars for legacy config system */ @@ -110,12 +111,33 @@ struct instanceConf_s { uchar *pszBindRuleset; /* name of ruleset to bind to */ uchar *inputname; ruleset_t *pBindRuleset; /* ruleset to bind listener to (use system default if unspecified) */ + uchar *dfltTZ; int ratelimitInterval; int ratelimitBurst; + int rcvbuf; /* 0 means: do not set, keep OS default */ struct instanceConf_s *next; sbool bAppendPortToInpname; }; +/* The following structure controls the worker threads. Global data is + * needed for their access. + */ +static struct wrkrInfo_s { + pthread_t tid; /* the worker's thread ID */ + int id; + thrdInfo_t *pThrd; + statsobj_t *stats; /* worker thread stats */ + STATSCOUNTER_DEF(ctrCall_recvmmsg, mutCtrCall_recvmmsg) + STATSCOUNTER_DEF(ctrCall_recvmsg, mutCtrCall_recvmsg) + STATSCOUNTER_DEF(ctrMsgsRcvd, mutCtrMsgsRcvd) + uchar *pRcvBuf; /* receive buffer (for a single packet) */ +# ifdef HAVE_RECVMMSG + struct sockaddr_storage *frominet; + struct mmsghdr *recvmsg_mmh; + struct iovec *recvmsg_iov; +# endif +} wrkrInfo[MAX_WRKR_THREADS]; + struct modConfData_s { rsconf_t *pConf; /* our overall config object */ instanceConf_t *root, *tail; @@ -123,6 +145,8 @@ struct modConfData_s { int iSchedPolicy; /* scheduling policy as SCHED_xxx */ int iSchedPrio; /* scheduling priority */ int iTimeRequery; /* how often is time to be queried inside tight recv loop? 0=always */ + int batchSize; /* max nbr of input batch --> also recvmmsg() max count */ + int8_t wrkrMax; /* max nbr of worker threads */ sbool configSetViaV2Method; }; static modConfData_t *loadModConf = NULL;/* modConf ptr to use for the current load process */ @@ -132,6 +156,8 @@ static modConfData_t *runModConf = NULL;/* modConf ptr to use for the current lo static struct cnfparamdescr modpdescr[] = { { "schedulingpolicy", eCmdHdlrGetWord, 0 }, { "schedulingpriority", eCmdHdlrInt, 0 }, + { "batchsize", eCmdHdlrInt, 0 }, + { "threads", eCmdHdlrPositiveInt, 0 }, { "timerequery", eCmdHdlrInt, 0 } }; static struct cnfparamblk modpblk = @@ -143,12 +169,16 @@ static struct cnfparamblk modpblk = /* input instance parameters */ static struct cnfparamdescr inppdescr[] = { { "port", eCmdHdlrArray, CNFPARAM_REQUIRED }, /* legacy: InputTCPServerRun */ + { "defaulttz", eCmdHdlrString, 0 }, { "inputname", eCmdHdlrGetWord, 0 }, { "inputname.appendport", eCmdHdlrBinary, 0 }, + { "name", eCmdHdlrGetWord, 0 }, + { "name.appendport", eCmdHdlrBinary, 0 }, { "address", eCmdHdlrString, 0 }, - { "ruleset", eCmdHdlrString, 0 }, { "ratelimit.interval", eCmdHdlrInt, 0 }, - { "ratelimit.burst", eCmdHdlrInt, 0 } + { "ratelimit.burst", eCmdHdlrInt, 0 }, + { "rcvbufsize", eCmdHdlrSize, 0 }, + { "ruleset", eCmdHdlrString, 0 } }; static struct cnfparamblk inppblk = { CNFPARAMBLK_VERSION, @@ -177,6 +207,8 @@ createInstance(instanceConf_t **pinst) inst->bAppendPortToInpname = 0; inst->ratelimitBurst = 10000; /* arbitrary high limit */ inst->ratelimitInterval = 0; /* off */ + inst->rcvbuf = 0; + inst->dfltTZ = NULL; /* node created, let's add to config */ if(loadModConf->tail == NULL) { @@ -252,23 +284,24 @@ addListner(instanceConf_t *inst) DBGPRINTF("Trying to open syslog UDP ports at %s:%s.\n", bindName, inst->pszBindPort); - newSocks = net.create_udp_socket(bindAddr, port, 1); + newSocks = net.create_udp_socket(bindAddr, port, 1, inst->rcvbuf); if(newSocks != NULL) { /* we now need to add the new sockets to the existing set */ /* ready to copy */ for(iSrc = 1 ; iSrc <= newSocks[0] ; ++iSrc) { - CHKmalloc(newlcnfinfo = (struct lstn_s*) MALLOC(sizeof(struct lstn_s))); + CHKmalloc(newlcnfinfo = (struct lstn_s*) calloc(1, sizeof(struct lstn_s))); newlcnfinfo->next = NULL; newlcnfinfo->sock = newSocks[iSrc]; newlcnfinfo->pRuleset = inst->pBindRuleset; - snprintf((char*)dispname, sizeof(dispname), "imudp(%s:%s)", bindName, port); - dispname[sizeof(dispname)-1] = '\0'; /* just to be on the save side... */ - CHKiRet(ratelimitNew(&newlcnfinfo->ratelimiter, (char*)dispname, NULL)); + newlcnfinfo->dfltTZ = inst->dfltTZ; if(inst->inputname == NULL) { inputname = (uchar*)"imudp"; } else { inputname = inst->inputname; } + snprintf((char*)dispname, sizeof(dispname), "%s(%s:%s)", inputname, bindName, port); + dispname[sizeof(dispname)-1] = '\0'; /* just to be on the save side... */ + CHKiRet(ratelimitNew(&newlcnfinfo->ratelimiter, (char*)dispname, NULL)); if(inst->bAppendPortToInpname) { snprintf((char*)inpnameBuf, sizeof(inpnameBuf), "%s%s", inputname, port); @@ -286,7 +319,7 @@ addListner(instanceConf_t *inst) CHKiRet(statsobj.SetName(newlcnfinfo->stats, dispname)); STATSCOUNTER_INIT(newlcnfinfo->ctrSubmit, newlcnfinfo->mutCtrSubmit); CHKiRet(statsobj.AddCounter(newlcnfinfo->stats, UCHAR_CONSTANT("submitted"), - ctrType_IntCtr, &(newlcnfinfo->ctrSubmit))); + ctrType_IntCtr, CTR_FLAG_RESETTABLE, &(newlcnfinfo->ctrSubmit))); CHKiRet(statsobj.ConstructFinalize(newlcnfinfo->stats)); /* link to list. Order must be preserved to take care for * conflicting matches. @@ -303,6 +336,23 @@ addListner(instanceConf_t *inst) } finalize_it: + if(iRet != RS_RET_OK) { + if(newlcnfinfo != NULL) { + if(newlcnfinfo->ratelimiter != NULL) + ratelimitDestruct(newlcnfinfo->ratelimiter); + if(newlcnfinfo->pInputName != NULL) + prop.Destruct(&newlcnfinfo->pInputName); + if(newlcnfinfo->stats != NULL) + statsobj.Destruct(&newlcnfinfo->stats); + free(newlcnfinfo); + } + /* close the rest of the open sockets as there's + nowhere to put them */ + for(; iSrc <= newSocks[0]; iSrc++) { + close(newSocks[iSrc]); + } + } + free(newSocks); RETiRet; } @@ -318,6 +368,161 @@ std_checkRuleset_genErrMsg(__attribute__((unused)) modConfData_t *modConf, insta } +/* This function processes received data. It provides unified handling + * in cases where recvmmsg() is available and not. + */ +static inline rsRetVal +processPacket(thrdInfo_t *pThrd, struct lstn_s *lstn, struct sockaddr_storage *frominetPrev, int *pbIsPermitted, + uchar *rcvBuf, ssize_t lenRcvBuf, struct syslogTime *stTime, time_t ttGenTime, + struct sockaddr_storage *frominet, socklen_t socklen, multi_submit_t *multiSub) +{ + DEFiRet; + msg_t *pMsg = NULL; + + assert(pThrd != NULL); + + if(lenRcvBuf == 0) + FINALIZE; /* this looks a bit strange, but practice shows it happens... */ + + /* if we reach this point, we had a good receive and can process the packet received */ + /* check if we have a different sender than before, if so, we need to query some new values */ + if(bDoACLCheck) { + socklen = sizeof(struct sockaddr_storage); + if(net.CmpHost(frominet, frominetPrev, socklen) != 0) { + memcpy(frominetPrev, frominet, socklen); /* update cache indicator */ + /* Here we check if a host is permitted to send us syslog messages. If it isn't, + * we do not further process the message but log a warning (if we are + * configured to do this). However, if the check would require name resolution, + * it is postponed to the main queue. See also my blog post at + * http://blog.gerhards.net/2009/11/acls-imudp-and-accepting-messages.html + * rgerhards, 2009-11-16 + */ + *pbIsPermitted = net.isAllowedSender2((uchar*)"UDP", + (struct sockaddr *)frominet, "", 0); + + if(*pbIsPermitted == 0) { + DBGPRINTF("msg is not from an allowed sender\n"); + if(glbl.GetOption_DisallowWarning) { + time_t tt; + datetime.GetTime(&tt); + if(tt > ttLastDiscard + 60) { + ttLastDiscard = tt; + errmsg.LogError(0, NO_ERRCODE, + "UDP message from disallowed sender discarded"); + } + } + } + } + } else { + *pbIsPermitted = 1; /* no check -> everything permitted */ + } + + DBGPRINTF("recv(%d,%d),acl:%d,msg:%.128s\n", lstn->sock, (int) lenRcvBuf, *pbIsPermitted, rcvBuf); + + if(*pbIsPermitted != 0) { + /* we now create our own message object and submit it to the queue */ + CHKiRet(msgConstructWithTime(&pMsg, stTime, ttGenTime)); + MsgSetRawMsg(pMsg, (char*)rcvBuf, lenRcvBuf); + MsgSetInputName(pMsg, lstn->pInputName); + MsgSetRuleset(pMsg, lstn->pRuleset); + MsgSetFlowControlType(pMsg, eFLOWCTL_NO_DELAY); + if(lstn->dfltTZ != NULL) + MsgSetDfltTZ(pMsg, (char*) lstn->dfltTZ); + pMsg->msgFlags = NEEDS_PARSING | PARSE_HOSTNAME | NEEDS_DNSRESOL; + if(*pbIsPermitted == 2) + pMsg->msgFlags |= NEEDS_ACLCHK_U; /* request ACL check after resolution */ + CHKiRet(msgSetFromSockinfo(pMsg, frominet)); + CHKiRet(ratelimitAddMsg(lstn->ratelimiter, multiSub, pMsg)); + STATSCOUNTER_INC(lstn->ctrSubmit, lstn->mutCtrSubmit); + } + +finalize_it: + if(iRet != RS_RET_OK) { + if(pMsg != NULL) { + msgDestruct(&pMsg); + } + } + + RETiRet; +} + + + + +/* The following "two" functions are helpers to runInput. Actually, it is + * just one function. Depending on whether or not we have recvmmsg(), + * an appropriate version is compiled (as such we need to maintain both!). + */ +#ifdef HAVE_RECVMMSG +static inline rsRetVal +processSocket(struct wrkrInfo_s *pWrkr, struct lstn_s *lstn, struct sockaddr_storage *frominetPrev, int *pbIsPermitted) +{ + DEFiRet; + int iNbrTimeUsed; + time_t ttGenTime; + struct syslogTime stTime; + char errStr[1024]; + msg_t *pMsgs[CONF_NUM_MULTISUB]; + multi_submit_t multiSub; + int nelem; + int i; + + multiSub.ppMsgs = pMsgs; + multiSub.maxElem = CONF_NUM_MULTISUB; + multiSub.nElem = 0; + iNbrTimeUsed = 0; + while(1) { /* loop is terminated if we have a "bad" receive, done below in the body */ + if(pWrkr->pThrd->bShallStop == RSTRUE) + ABORT_FINALIZE(RS_RET_FORCE_TERM); + memset(pWrkr->recvmsg_iov, 0, runModConf->batchSize * sizeof(struct iovec)); + memset(pWrkr->recvmsg_mmh, 0, runModConf->batchSize * sizeof(struct mmsghdr)); + for(i = 0 ; i < runModConf->batchSize ; ++i) { + pWrkr->recvmsg_iov[i].iov_base = pWrkr->pRcvBuf+(i*(iMaxLine+1)); + pWrkr->recvmsg_iov[i].iov_len = iMaxLine; + pWrkr->recvmsg_mmh[i].msg_hdr.msg_namelen = sizeof(struct sockaddr_storage); + pWrkr->recvmsg_mmh[i].msg_hdr.msg_name = &(pWrkr->frominet[i]); + pWrkr->recvmsg_mmh[i].msg_hdr.msg_iov = &(pWrkr->recvmsg_iov[i]); + pWrkr->recvmsg_mmh[i].msg_hdr.msg_iovlen = 1; + } + nelem = recvmmsg(lstn->sock, pWrkr->recvmsg_mmh, runModConf->batchSize, 0, NULL); + STATSCOUNTER_INC(pWrkr->ctrCall_recvmmsg, pWrkr->mutCtrCall_recvmmsg); + DBGPRINTF("imudp: recvmmsg returned %d\n", nelem); + if(nelem < 0 && errno == ENOSYS) { + /* be careful: some versions of valgrind do not support recvmmsg()! */ + DBGPRINTF("imudp: error ENOSYS on call to recvmmsg() - fall back to recvmsg\n"); + nelem = recvmsg(lstn->sock, &(pWrkr->recvmsg_mmh[0].msg_hdr), 0); + STATSCOUNTER_INC(pWrkr->ctrCall_recvmsg, pWrkr->mutCtrCall_recvmsg); + if(nelem >= 0) { + pWrkr->recvmsg_mmh[0].msg_len = nelem; + nelem = 1; + } + } + if(nelem < 0) { + if(errno != EINTR && errno != EAGAIN) { + rs_strerror_r(errno, errStr, sizeof(errStr)); + DBGPRINTF("INET socket error: %d = %s.\n", errno, errStr); + errmsg.LogError(errno, NO_ERRCODE, "imudp: error receiving on socket: %s", errStr); + } + ABORT_FINALIZE(RS_RET_ERR); // this most often is NOT an error, state is not checked by caller! + } + + if((runModConf->iTimeRequery == 0) || (iNbrTimeUsed++ % runModConf->iTimeRequery) == 0) { + datetime.getCurrTime(&stTime, &ttGenTime); + } + + pWrkr->ctrMsgsRcvd += nelem; + for(i = 0 ; i < nelem ; ++i) { + processPacket(pWrkr->pThrd, lstn, frominetPrev, pbIsPermitted, pWrkr->recvmsg_mmh[i].msg_hdr.msg_iov->iov_base, + pWrkr->recvmsg_mmh[i].msg_len, &stTime, ttGenTime, &(pWrkr->frominet[i]), + pWrkr->recvmsg_mmh[i].msg_hdr.msg_namelen, &multiSub); + } + } + +finalize_it: + multiSubmitFlush(&multiSub); + RETiRet; +} +#else /* we do not have recvmmsg() */ /* This function is a helper to runInput. I have extracted it * from the main loop just so that we do not have that large amount of code * in a single place. This function takes a socket and pulls messages from @@ -333,108 +538,61 @@ std_checkRuleset_genErrMsg(__attribute__((unused)) modConfData_t *modConf, insta * on scheduling order. -- rgerhards, 2008-10-02 */ static inline rsRetVal -processSocket(thrdInfo_t *pThrd, struct lstn_s *lstn, struct sockaddr_storage *frominetPrev, int *pbIsPermitted) +processSocket(struct wrkrInfo_s *pWrkr, struct lstn_s *lstn, struct sockaddr_storage *frominetPrev, int *pbIsPermitted) { int iNbrTimeUsed; time_t ttGenTime; struct syslogTime stTime; - socklen_t socklen; ssize_t lenRcvBuf; struct sockaddr_storage frominet; - msg_t *pMsg; - prop_t *propFromHost = NULL; - prop_t *propFromHostIP = NULL; multi_submit_t multiSub; msg_t *pMsgs[CONF_NUM_MULTISUB]; char errStr[1024]; + struct msghdr mh; + struct iovec iov[1]; DEFiRet; - assert(pThrd != NULL); multiSub.ppMsgs = pMsgs; multiSub.maxElem = CONF_NUM_MULTISUB; multiSub.nElem = 0; iNbrTimeUsed = 0; while(1) { /* loop is terminated if we have a bad receive, done below in the body */ - if(pThrd->bShallStop == RSTRUE) + if(pWrkr->pThrd->bShallStop == RSTRUE) ABORT_FINALIZE(RS_RET_FORCE_TERM); - socklen = sizeof(struct sockaddr_storage); - lenRcvBuf = recvfrom(lstn->sock, (char*) pRcvBuf, iMaxLine, 0, (struct sockaddr *)&frominet, &socklen); + memset(iov, 0, sizeof(iov)); + iov[0].iov_base = pWrkr->pRcvBuf; + iov[0].iov_len = iMaxLine; + memset(&mh, 0, sizeof(mh)); + mh.msg_name = &frominet; + mh.msg_namelen = sizeof(struct sockaddr_storage); + mh.msg_iov = iov; + mh.msg_iovlen = 1; + lenRcvBuf = recvmsg(lstn->sock, &mh, 0); + STATSCOUNTER_INC(pWrkr->ctrCall_recvmsg, pWrkr->mutCtrCall_recvmsg); if(lenRcvBuf < 0) { if(errno != EINTR && errno != EAGAIN) { rs_strerror_r(errno, errStr, sizeof(errStr)); DBGPRINTF("INET socket error: %d = %s.\n", errno, errStr); - errmsg.LogError(errno, NO_ERRCODE, "recvfrom inet"); + errmsg.LogError(errno, NO_ERRCODE, "imudp: error receiving on socket: %s", errStr); } ABORT_FINALIZE(RS_RET_ERR); // this most often is NOT an error, state is not checked by caller! } - if(lenRcvBuf == 0) - continue; /* this looks a bit strange, but practice shows it happens... */ - - /* if we reach this point, we had a good receive and can process the packet received */ - /* check if we have a different sender than before, if so, we need to query some new values */ - if(bDoACLCheck) { - if(net.CmpHost(&frominet, frominetPrev, socklen) != 0) { - memcpy(frominetPrev, &frominet, socklen); /* update cache indicator */ - /* Here we check if a host is permitted to send us syslog messages. If it isn't, - * we do not further process the message but log a warning (if we are - * configured to do this). However, if the check would require name resolution, - * it is postponed to the main queue. See also my blog post at - * http://blog.gerhards.net/2009/11/acls-imudp-and-accepting-messages.html - * rgerhards, 2009-11-16 - */ - *pbIsPermitted = net.isAllowedSender2((uchar*)"UDP", - (struct sockaddr *)&frominet, "", 0); - - if(*pbIsPermitted == 0) { - DBGPRINTF("msg is not from an allowed sender\n"); - if(glbl.GetOption_DisallowWarning) { - time_t tt; - datetime.GetTime(&tt); - if(tt > ttLastDiscard + 60) { - ttLastDiscard = tt; - errmsg.LogError(0, NO_ERRCODE, - "UDP message from disallowed sender discarded"); - } - } - } - } - } else { - *pbIsPermitted = 1; /* no check -> everything permitted */ + ++pWrkr->ctrMsgsRcvd; + if((runModConf->iTimeRequery == 0) || (iNbrTimeUsed++ % runModConf->iTimeRequery) == 0) { + datetime.getCurrTime(&stTime, &ttGenTime); } - DBGPRINTF("imudp:recv(%d,%d),acl:%d,msg:%s\n", lstn->sock, (int) lenRcvBuf, *pbIsPermitted, pRcvBuf); - - if(*pbIsPermitted != 0) { - if((runModConf->iTimeRequery == 0) || (iNbrTimeUsed++ % runModConf->iTimeRequery) == 0) { - datetime.getCurrTime(&stTime, &ttGenTime); - } - /* we now create our own message object and submit it to the queue */ - CHKiRet(msgConstructWithTime(&pMsg, &stTime, ttGenTime)); - MsgSetRawMsg(pMsg, (char*)pRcvBuf, lenRcvBuf); - MsgSetInputName(pMsg, lstn->pInputName); - MsgSetRuleset(pMsg, lstn->pRuleset); - MsgSetFlowControlType(pMsg, eFLOWCTL_NO_DELAY); - pMsg->msgFlags = NEEDS_PARSING | PARSE_HOSTNAME | NEEDS_DNSRESOL; - if(*pbIsPermitted == 2) - pMsg->msgFlags |= NEEDS_ACLCHK_U; /* request ACL check after resolution */ - CHKiRet(msgSetFromSockinfo(pMsg, &frominet)); - CHKiRet(ratelimitAddMsg(lstn->ratelimiter, &multiSub, pMsg)); - STATSCOUNTER_INC(lstn->ctrSubmit, lstn->mutCtrSubmit); - } + CHKiRet(processPacket(pWrkr->pThrd, lstn, frominetPrev, pbIsPermitted, pWrkr->pRcvBuf, lenRcvBuf, &stTime, + ttGenTime, &frominet, mh.msg_namelen, &multiSub)); } finalize_it: multiSubmitFlush(&multiSub); - - if(propFromHost != NULL) - prop.Destruct(&propFromHost); - if(propFromHostIP != NULL) - prop.Destruct(&propFromHostIP); - RETiRet; } +#endif /* #ifdef HAVE_RECVMMSG */ /* check configured scheduling priority. @@ -565,7 +723,7 @@ finalize_it: */ #if defined(HAVE_EPOLL_CREATE1) || defined(HAVE_EPOLL_CREATE) #define NUM_EPOLL_EVENTS 10 -rsRetVal rcvMainLoop(thrdInfo_t *pThrd) +rsRetVal rcvMainLoop(struct wrkrInfo_s *pWrkr) { DEFiRet; int nfds; @@ -628,11 +786,11 @@ rsRetVal rcvMainLoop(thrdInfo_t *pThrd) nfds = epoll_wait(efd, currEvt, NUM_EPOLL_EVENTS, -1); DBGPRINTF("imudp: epoll_wait() returned with %d fds\n", nfds); - if(pThrd->bShallStop == RSTRUE) + if(pWrkr->pThrd->bShallStop == RSTRUE) break; /* terminate input! */ for(i = 0 ; i < nfds ; ++i) { - processSocket(pThrd, currEvt[i].data.ptr, &frominetPrev, &bIsPermitted); + processSocket(pWrkr, currEvt[i].data.ptr, &frominetPrev, &bIsPermitted); } } @@ -644,7 +802,7 @@ finalize_it: } #else /* #if HAVE_EPOLL_CREATE1 */ /* this is the code for the select() interface */ -rsRetVal rcvMainLoop(thrdInfo_t *pThrd) +rsRetVal rcvMainLoop(thrdInfo_t *pWrkr) { DEFiRet; int maxfds; @@ -691,7 +849,7 @@ rsRetVal rcvMainLoop(thrdInfo_t *pThrd) for(lstn = lcnfRoot ; nfds && lstn != NULL ; lstn = lstn->next) { if(FD_ISSET(lstn->sock, &readfds)) { - processSocket(pThrd, lstn, &frominetPrev, &bIsPermitted); + processSocket(pWrkr, lstn, &frominetPrev, &bIsPermitted); --nfds; /* indicate we have processed one descriptor */ } } @@ -708,6 +866,7 @@ createListner(es_str_t *port, struct cnfparamvals *pvals) { instanceConf_t *inst; int i; + int bAppendPortUsed = 0; DEFiRet; CHKiRet(createInstance(&inst)); @@ -717,10 +876,42 @@ createListner(es_str_t *port, struct cnfparamvals *pvals) continue; if(!strcmp(inppblk.descr[i].name, "port")) { continue; /* array, handled by caller */ + } else if(!strcmp(inppblk.descr[i].name, "name")) { + if(inst->inputname != NULL) { + errmsg.LogError(0, RS_RET_INVALID_PARAMS, "imudp: name and inputname " + "paramter specified - only one can be used"); + ABORT_FINALIZE(RS_RET_INVALID_PARAMS); + } + inst->inputname = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + } else if(!strcmp(inppblk.descr[i].name, "name.appendport")) { + if(bAppendPortUsed) { + errmsg.LogError(0, RS_RET_INVALID_PARAMS, "imudp: name.appendport and " + "inputname.appendport paramter specified - only one can be used"); + ABORT_FINALIZE(RS_RET_INVALID_PARAMS); + } + inst->bAppendPortToInpname = (int) pvals[i].val.d.n; + bAppendPortUsed = 1; } else if(!strcmp(inppblk.descr[i].name, "inputname")) { + errmsg.LogError(0, RS_RET_DEPRECATED , "imudp: deprecated parameter inputname " + "used. Suggest to use name instead"); + if(inst->inputname != NULL) { + errmsg.LogError(0, RS_RET_INVALID_PARAMS, "imudp: name and inputname " + "parameter specified - only one can be used"); + ABORT_FINALIZE(RS_RET_INVALID_PARAMS); + } inst->inputname = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); } else if(!strcmp(inppblk.descr[i].name, "inputname.appendport")) { + errmsg.LogError(0, RS_RET_DEPRECATED , "imudp: deprecated parameter inputname.appendport " + "used. Suggest to use name.appendport instead"); + if(bAppendPortUsed) { + errmsg.LogError(0, RS_RET_INVALID_PARAMS, "imudp: name.appendport and " + "inputname.appendport parameter specified - only one can be used"); + ABORT_FINALIZE(RS_RET_INVALID_PARAMS); + } + bAppendPortUsed = 1; inst->bAppendPortToInpname = (int) pvals[i].val.d.n; + } else if(!strcmp(inppblk.descr[i].name, "defaulttz")) { + inst->dfltTZ = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); } else if(!strcmp(inppblk.descr[i].name, "address")) { inst->pszBindAddr = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); } else if(!strcmp(inppblk.descr[i].name, "ruleset")) { @@ -729,6 +920,8 @@ createListner(es_str_t *port, struct cnfparamvals *pvals) inst->ratelimitBurst = (int) pvals[i].val.d.n; } else if(!strcmp(inppblk.descr[i].name, "ratelimit.interval")) { inst->ratelimitInterval = (int) pvals[i].val.d.n; + } else if(!strcmp(inppblk.descr[i].name, "rcvbufsize")) { + inst->rcvbuf = (int) pvals[i].val.d.n; } else { dbgprintf("imudp: program error, non-handled " "param '%s'\n", inppblk.descr[i].name); @@ -746,10 +939,7 @@ BEGINnewInpInst CODESTARTnewInpInst DBGPRINTF("newInpInst (imudp)\n"); - pvals = nvlstGetParams(lst, &inppblk, NULL); - if(pvals == NULL) { - errmsg.LogError(0, RS_RET_MISSING_CNFPARAMS, - "imudp: required parameter are missing\n"); + if((pvals = nvlstGetParams(lst, &inppblk, NULL)) == NULL) { ABORT_FINALIZE(RS_RET_MISSING_CNFPARAMS); } if(Debug) { @@ -775,6 +965,8 @@ CODESTARTbeginCnfLoad pModConf->pConf = pConf; /* init our settings */ loadModConf->configSetViaV2Method = 0; + loadModConf->wrkrMax = 1; /* conservative, but least msg reordering */ + loadModConf->batchSize = BATCH_SIZE_DFLT; loadModConf->iTimeRequery = TIME_REQUERY_DFLT; loadModConf->iSchedPrio = SCHED_PRIO_UNSET; loadModConf->pszSchedPolicy = NULL; @@ -791,6 +983,7 @@ ENDbeginCnfLoad BEGINsetModCnf struct cnfparamvals *pvals = NULL; int i; + int wrkrMax; CODESTARTsetModCnf pvals = nvlstGetParams(lst, &modpblk, NULL); if(pvals == NULL) { @@ -809,10 +1002,22 @@ CODESTARTsetModCnf continue; if(!strcmp(modpblk.descr[i].name, "timerequery")) { loadModConf->iTimeRequery = (int) pvals[i].val.d.n; + } else if(!strcmp(modpblk.descr[i].name, "batchsize")) { + loadModConf->batchSize = (int) pvals[i].val.d.n; } else if(!strcmp(modpblk.descr[i].name, "schedulingpriority")) { loadModConf->iSchedPrio = (int) pvals[i].val.d.n; } else if(!strcmp(modpblk.descr[i].name, "schedulingpolicy")) { loadModConf->pszSchedPolicy = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + } else if(!strcmp(modpblk.descr[i].name, "threads")) { + wrkrMax = (int) pvals[i].val.d.n; + if(wrkrMax > MAX_WRKR_THREADS) { + errmsg.LogError(0, RS_RET_PARAM_ERROR, "imudp: configured for %d" + "worker threads, but maximum permitted is %d", + wrkrMax, MAX_WRKR_THREADS); + loadModConf->wrkrMax = MAX_WRKR_THREADS; + } else { + loadModConf->wrkrMax = wrkrMax; + } } else { dbgprintf("imudp: program error, non-handled " "param '%s' in beginCnfLoad\n", modpblk.descr[i].name); @@ -872,7 +1077,7 @@ CODESTARTactivateCnfPrePrivDrop for(inst = runModConf->root ; inst != NULL ; inst = inst->next) { addListner(inst); } - /* if we could not set up any listners, there is no point in running... */ + /* if we could not set up any listeners, there is no point in running... */ if(lcnfRoot == NULL) { errmsg.LogError(0, NO_ERRCODE, "imudp: no listeners could be started, " "input not activated.\n"); @@ -884,10 +1089,25 @@ ENDactivateCnfPrePrivDrop BEGINactivateCnf + int i; + int lenRcvBuf; CODESTARTactivateCnf /* caching various settings */ iMaxLine = glbl.GetMaxLine(); - CHKmalloc(pRcvBuf = MALLOC((iMaxLine + 1) * sizeof(char))); + lenRcvBuf = (iMaxLine + 1) * sizeof(char); +# ifdef HAVE_RECVMMSG + lenRcvBuf *= runModConf->batchSize; +# endif + DBGPRINTF("imudp: config params iMaxLine %d, lenRcvBuf %d\n", iMaxLine, lenRcvBuf); + for(i = 0 ; i < runModConf->wrkrMax ; ++i) { +# ifdef HAVE_RECVMMSG + CHKmalloc(wrkrInfo[i].recvmsg_iov = MALLOC(runModConf->batchSize * sizeof(struct iovec))); + CHKmalloc(wrkrInfo[i].recvmsg_mmh = MALLOC(runModConf->batchSize * sizeof(struct mmsghdr))); + CHKmalloc(wrkrInfo[i].frominet = MALLOC(runModConf->batchSize * sizeof(struct sockaddr_storage))); +# endif + CHKmalloc(wrkrInfo[i].pRcvBuf = MALLOC(lenRcvBuf)); + wrkrInfo[i].id = i; + } finalize_it: ENDactivateCnf @@ -898,22 +1118,35 @@ CODESTARTfreeCnf for(inst = pModConf->root ; inst != NULL ; ) { free(inst->pszBindPort); free(inst->pszBindAddr); - free(inst->pBindRuleset); free(inst->inputname); + free(inst->dfltTZ); del = inst; inst = inst->next; free(del); } ENDfreeCnf -/* This function is called to gather input. - * Note that sock must be non-NULL because otherwise we would not have - * indicated that we want to run (or we have a programming error ;)). -- rgerhards, 2008-10-02 - */ -BEGINrunInput -CODESTARTrunInput + +static void * +wrkr(void *myself) +{ + struct wrkrInfo_s *pWrkr = (struct wrkrInfo_s*) myself; +# if HAVE_PRCTL && defined PR_SET_NAME + uchar *pszDbgHdr; +# endif + uchar thrdName[32]; + + snprintf((char*)thrdName, sizeof(thrdName), "imudp(w%d)", pWrkr->id); +# if HAVE_PRCTL && defined PR_SET_NAME + /* set thread name - we ignore if the call fails, has no harsh consequences... */ + if(prctl(PR_SET_NAME, thrdName, 0, 0, 0) != 0) { + DBGPRINTF("prctl failed, not setting thread name for '%s'\n", thrdName); + } +# endif + dbgOutputTID((char*)thrdName); + /* Note well: the setting of scheduling parameters will not work - * when we dropped privileges (if the user is not sufficently + * when we dropped privileges (if the user is not sufficiently * privileged, of course). Howerver, we can't change the * scheduling params in PrePrivDrop(), as at that point our thread * is not yet created. So at least as an interim solution, we do @@ -921,7 +1154,51 @@ CODESTARTrunInput * privileges within the same instance. */ setSchedParams(runModConf); - iRet = rcvMainLoop(pThrd); + + /* support statistics gathering */ + statsobj.Construct(&(pWrkr->stats)); + statsobj.SetName(pWrkr->stats, thrdName); + STATSCOUNTER_INIT(pWrkr->ctrCall_recvmmsg, pWrkr->mutCtrCall_recvmmsg); + statsobj.AddCounter(pWrkr->stats, UCHAR_CONSTANT("called.recvmmsg"), + ctrType_IntCtr, CTR_FLAG_RESETTABLE, &(pWrkr->ctrCall_recvmmsg)); + STATSCOUNTER_INIT(pWrkr->ctrCall_recvmsg, pWrkr->mutCtrCall_recvmsg); + statsobj.AddCounter(pWrkr->stats, UCHAR_CONSTANT("called.recvmsg"), + ctrType_IntCtr, CTR_FLAG_RESETTABLE, &(pWrkr->ctrCall_recvmsg)); + STATSCOUNTER_INIT(pWrkr->ctrMsgsRcvd, pWrkr->mutCtrMsgsRcvd); + statsobj.AddCounter(pWrkr->stats, UCHAR_CONSTANT("msgs.received"), + ctrType_IntCtr, CTR_FLAG_RESETTABLE, &(pWrkr->ctrMsgsRcvd)); + statsobj.ConstructFinalize(pWrkr->stats); + + rcvMainLoop(pWrkr); + + /* cleanup */ + return NULL; +} + +/* This function is called to gather input. + * In essence, it just starts the pool of workers. To save resources, + * we run one of the workers on our own thread -- otherwise that thread would + * just idle around and wait for the workers to finish. + */ +BEGINrunInput + int i; + pthread_attr_t wrkrThrdAttr; +CODESTARTrunInput + pthread_attr_init(&wrkrThrdAttr); + pthread_attr_setstacksize(&wrkrThrdAttr, 4096*1024); + for(i = 0 ; i < runModConf->wrkrMax - 1 ; ++i) { + wrkrInfo[i].pThrd = pThrd; + pthread_create(&wrkrInfo[i].tid, &wrkrThrdAttr, wrkr, &(wrkrInfo[i])); + } + pthread_attr_destroy(&wrkrThrdAttr); + + wrkrInfo[i].pThrd = pThrd; + wrkrInfo[i].id = i; + wrkr(&wrkrInfo[i]); + + for(i = 0 ; i < runModConf->wrkrMax - 1 ; ++i) { + pthread_join(wrkrInfo[i].tid, NULL); + } ENDrunInput @@ -935,6 +1212,7 @@ ENDwillRun BEGINafterRun struct lstn_s *lstn, *lstnDel; + int i; CODESTARTafterRun /* do cleanup here */ net.clearAllowedSenders((uchar*)"UDP"); @@ -948,9 +1226,13 @@ CODESTARTafterRun free(lstnDel); } lcnfRoot = lcnfLast = NULL; - if(pRcvBuf != NULL) { - free(pRcvBuf); - pRcvBuf = NULL; + for(i = 0 ; i < runModConf->wrkrMax ; ++i) { +# ifdef HAVE_RECVMMSG + free(wrkrInfo[i].recvmsg_iov); + free(wrkrInfo[i].recvmsg_mmh); + free(wrkrInfo[i].frominet); +# endif + free(wrkrInfo[i].pRcvBuf); } ENDafterRun @@ -1011,6 +1293,11 @@ CODEmodInit_QueryRegCFSLineHdlr CHKiRet(objUse(ruleset, CORE_COMPONENT)); CHKiRet(objUse(net, LM_NET_FILENAME)); + DBGPRINTF("imudp: version %s initializing\n", VERSION); +# ifdef HAVE_RECVMMSG + DBGPRINTF("imdup: support for recvmmsg() present\n"); +# endif + /* register config file handlers */ CHKiRet(omsdRegCFSLineHdlr((uchar *)"inputudpserverbindruleset", 0, eCmdHdlrGetWord, NULL, &cs.pszBindRuleset, STD_LOADABLE_MODULE_ID)); diff --git a/plugins/imuxsock/Makefile.in b/plugins/imuxsock/Makefile.in index eb9eb43..593b248 100644 --- a/plugins/imuxsock/Makefile.in +++ b/plugins/imuxsock/Makefile.in @@ -154,7 +154,6 @@ GREP = @GREP@ GSS_LIBS = @GSS_LIBS@ GUARDTIME_CFLAGS = @GUARDTIME_CFLAGS@ GUARDTIME_LIBS = @GUARDTIME_LIBS@ -HAVE_LIBGCRYPT_CONFIG = @HAVE_LIBGCRYPT_CONFIG@ HAVE_MYSQL_CONFIG = @HAVE_MYSQL_CONFIG@ HAVE_ORACLE_CONFIG = @HAVE_ORACLE_CONFIG@ HAVE_PGSQL_CONFIG = @HAVE_PGSQL_CONFIG@ @@ -175,14 +174,15 @@ LEXLIB = @LEXLIB@ LEX_OUTPUT_ROOT = @LEX_OUTPUT_ROOT@ LIBDBI_CFLAGS = @LIBDBI_CFLAGS@ LIBDBI_LIBS = @LIBDBI_LIBS@ -LIBEE_CFLAGS = @LIBEE_CFLAGS@ -LIBEE_LIBS = @LIBEE_LIBS@ LIBESTR_CFLAGS = @LIBESTR_CFLAGS@ LIBESTR_LIBS = @LIBESTR_LIBS@ LIBGCRYPT_CFLAGS = @LIBGCRYPT_CFLAGS@ +LIBGCRYPT_CONFIG = @LIBGCRYPT_CONFIG@ LIBGCRYPT_LIBS = @LIBGCRYPT_LIBS@ LIBLOGGING_CFLAGS = @LIBLOGGING_CFLAGS@ LIBLOGGING_LIBS = @LIBLOGGING_LIBS@ +LIBLOGGING_STDLOG_CFLAGS = @LIBLOGGING_STDLOG_CFLAGS@ +LIBLOGGING_STDLOG_LIBS = @LIBLOGGING_STDLOG_LIBS@ LIBLOGNORM_CFLAGS = @LIBLOGNORM_CFLAGS@ LIBLOGNORM_LIBS = @LIBLOGNORM_LIBS@ LIBM = @LIBM@ @@ -207,6 +207,8 @@ NM = @NM@ NMEDIT = @NMEDIT@ OBJDUMP = @OBJDUMP@ OBJEXT = @OBJEXT@ +OPENSSL_CFLAGS = @OPENSSL_CFLAGS@ +OPENSSL_LIBS = @OPENSSL_LIBS@ ORACLE_CFLAGS = @ORACLE_CFLAGS@ ORACLE_LIBS = @ORACLE_LIBS@ OTOOL = @OTOOL@ @@ -387,22 +389,25 @@ distclean-compile: @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/imuxsock_la-imuxsock.Plo@am__quote@ .c.o: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(COMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ $< -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Po +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.o$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c $< +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ $< .c.obj: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(COMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ `$(CYGPATH_W) '$<'` -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Po +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.obj$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ `$(CYGPATH_W) '$<'` &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c `$(CYGPATH_W) '$<'` +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ `$(CYGPATH_W) '$<'` .c.lo: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(LTCOMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ $< -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Plo +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.lo$$||'`;\ +@am__fastdepCC_TRUE@ $(LTCOMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Plo @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=yes @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ @am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(LTCOMPILE) -c -o $@ $< diff --git a/plugins/imuxsock/imuxsock.c b/plugins/imuxsock/imuxsock.c index c503852..d99bed8 100644 --- a/plugins/imuxsock/imuxsock.c +++ b/plugins/imuxsock/imuxsock.c @@ -6,7 +6,7 @@ * * File begun on 2007-12-20 by RGerhards (extracted from syslogd.c) * - * Copyright 2007-2013 Rainer Gerhards and Adiscon GmbH. + * Copyright 2007-2014 Rainer Gerhards and Adiscon GmbH. * * This file is part of rsyslog. * @@ -62,7 +62,6 @@ MODULE_TYPE_NOKEEP MODULE_CNFNAME("imuxsock") /* defines */ -#define MAXFUNIX 50 #ifndef _PATH_LOG #ifdef BSD #define _PATH_LOG "/var/run/log" @@ -148,7 +147,7 @@ typedef struct lstn_s { sbool bUseSysTimeStamp; /* use timestamp from system (instead of from message) */ sbool bUnlink; /* unlink&re-create socket at start and end of processing */ } lstn_t; -static lstn_t listeners[MAXFUNIX]; +static lstn_t *listeners; static prop_t *pLocalHostIP = NULL; /* there is only one global IP for all internally-generated messages */ static prop_t *pInputName = NULL; /* our inputName currently is always "imudp", and this will hold it */ @@ -156,14 +155,15 @@ static int startIndexUxLocalSockets; /* process fd from that index on (used to * suppress local logging. rgerhards 2005-08-01 * read-only after startup */ -static int nfd = 1; /* number of Unix sockets open / read-only after startup */ +static int nfd = 1; /* number of active unix sockets (socket 0 is always reserved for the system + socket, even if it is not enabled. */ static int sd_fds = 0; /* number of systemd activated sockets */ -/* config vars for legacy config system */ #define DFLT_bCreatePath 0 #define DFLT_ratelimitInterval 0 #define DFLT_ratelimitBurst 200 #define DFLT_ratelimitSeverity 1 /* do not rate-limit emergency messages */ +/* config vars for the legacy config system */ static struct configSettings_s { int bOmitLocalLogging; uchar *pLogSockName; @@ -188,6 +188,7 @@ static struct configSettings_s { int bParseTrusted; /* parse trusted properties */ } cs; +/* config vars for the v2 config system (rsyslog v6+) */ struct instanceConf_s { uchar *sockName; uchar *pLogHostName; /* host name to use with this socket */ @@ -359,12 +360,7 @@ finalize_it: } -/* add an additional listen socket. Socket names are added - * until the array is filled up. It is never reset, only at - * module unload. - * TODO: we should change the array to a list so that we - * can support any number of listen socket names. - * rgerhards, 2007-12-20 +/* add an additional listen socket. * added capability to specify hostname for socket -- rgerhards, 2008-08-01 */ static rsRetVal @@ -372,66 +368,70 @@ addListner(instanceConf_t *inst) { DEFiRet; - if(nfd < MAXFUNIX) { - if(*inst->sockName == ':') { - listeners[nfd].bParseHost = 1; - } else { - listeners[nfd].bParseHost = 0; - } - if(inst->pLogHostName == NULL) { - listeners[nfd].hostName = NULL; - } else { - CHKiRet(prop.Construct(&(listeners[nfd].hostName))); - CHKiRet(prop.SetString(listeners[nfd].hostName, inst->pLogHostName, ustrlen(inst->pLogHostName))); - CHKiRet(prop.ConstructFinalize(listeners[nfd].hostName)); - } - if(inst->ratelimitInterval > 0) { - if((listeners[nfd].ht = create_hashtable(100, hash_from_key_fn, key_equals_fn, - (void(*)(void*))ratelimitDestruct)) == NULL) { - /* in this case, we simply turn off rate-limiting */ - DBGPRINTF("imuxsock: turning off rate limiting because we could not " - "create hash table\n"); - inst->ratelimitInterval = 0; - } + if(*inst->sockName == ':') { + listeners[nfd].bParseHost = 1; + } else { + listeners[nfd].bParseHost = 0; + } + if(inst->pLogHostName == NULL) { + listeners[nfd].hostName = NULL; + } else { + CHKiRet(prop.Construct(&(listeners[nfd].hostName))); + CHKiRet(prop.SetString(listeners[nfd].hostName, inst->pLogHostName, ustrlen(inst->pLogHostName))); + CHKiRet(prop.ConstructFinalize(listeners[nfd].hostName)); + } + if(inst->ratelimitInterval > 0) { + if((listeners[nfd].ht = create_hashtable(100, hash_from_key_fn, key_equals_fn, + (void(*)(void*))ratelimitDestruct)) == NULL) { + /* in this case, we simply turn off rate-limiting */ + DBGPRINTF("imuxsock: turning off rate limiting because we could not " + "create hash table\n"); + inst->ratelimitInterval = 0; } - listeners[nfd].ratelimitInterval = inst->ratelimitInterval; - listeners[nfd].ratelimitBurst = inst->ratelimitBurst; - listeners[nfd].ratelimitSev = inst->ratelimitSeverity; - listeners[nfd].flowCtl = inst->bUseFlowCtl ? eFLOWCTL_LIGHT_DELAY : eFLOWCTL_NO_DELAY; - listeners[nfd].flags = inst->bIgnoreTimestamp ? IGNDATE : NOFLAG; - listeners[nfd].bCreatePath = inst->bCreatePath; - listeners[nfd].sockName = ustrdup(inst->sockName); - listeners[nfd].bUseCreds = (inst->bDiscardOwnMsgs || inst->bWritePid || inst->ratelimitInterval || inst->bAnnotate) ? 1 : 0; - listeners[nfd].bAnnotate = inst->bAnnotate; - listeners[nfd].bParseTrusted = inst->bParseTrusted; - listeners[nfd].bDiscardOwnMsgs = inst->bDiscardOwnMsgs; - listeners[nfd].bUnlink = inst->bUnlink; - listeners[nfd].bWritePid = inst->bWritePid; - listeners[nfd].bUseSysTimeStamp = inst->bUseSysTimeStamp; - CHKiRet(ratelimitNew(&listeners[nfd].dflt_ratelimiter, "imuxsock", NULL)); - ratelimitSetLinuxLike(listeners[nfd].dflt_ratelimiter, - listeners[nfd].ratelimitInterval, - listeners[nfd].ratelimitBurst); - ratelimitSetSeverity(listeners[nfd].dflt_ratelimiter, - listeners[nfd].ratelimitSev); - nfd++; } else { - errmsg.LogError(0, NO_ERRCODE, "Out of unix socket name descriptors, ignoring %s\n", - inst->sockName); + listeners[nfd].ht = NULL; } + listeners[nfd].ratelimitInterval = inst->ratelimitInterval; + listeners[nfd].ratelimitBurst = inst->ratelimitBurst; + listeners[nfd].ratelimitSev = inst->ratelimitSeverity; + listeners[nfd].flowCtl = inst->bUseFlowCtl ? eFLOWCTL_LIGHT_DELAY : eFLOWCTL_NO_DELAY; + listeners[nfd].flags = inst->bIgnoreTimestamp ? IGNDATE : NOFLAG; + listeners[nfd].bCreatePath = inst->bCreatePath; + listeners[nfd].sockName = ustrdup(inst->sockName); + listeners[nfd].bUseCreds = (inst->bDiscardOwnMsgs || inst->bWritePid || inst->ratelimitInterval || inst->bAnnotate || inst->bUseSysTimeStamp) ? 1 : 0; + listeners[nfd].bAnnotate = inst->bAnnotate; + listeners[nfd].bParseTrusted = inst->bParseTrusted; + listeners[nfd].bDiscardOwnMsgs = inst->bDiscardOwnMsgs; + listeners[nfd].bUnlink = inst->bUnlink; + listeners[nfd].bWritePid = inst->bWritePid; + listeners[nfd].bUseSysTimeStamp = inst->bUseSysTimeStamp; + CHKiRet(ratelimitNew(&listeners[nfd].dflt_ratelimiter, "imuxsock", NULL)); + ratelimitSetLinuxLike(listeners[nfd].dflt_ratelimiter, + listeners[nfd].ratelimitInterval, + listeners[nfd].ratelimitBurst); + ratelimitSetSeverity(listeners[nfd].dflt_ratelimiter, + listeners[nfd].ratelimitSev); + nfd++; finalize_it: RETiRet; } -/* discard/Destruct all log sockets except for "socket" 0. Data for it comes from - * the constant memory pool - and if not, it is freeed via some other pointer. - */ static rsRetVal discardLogSockets(void) { int i; + /* Check whether the system socket is in use */ + if(startIndexUxLocalSockets == 0) { + /* Clean up rate limiting data for the system socket */ + if(listeners[0].ht != NULL) { + hashtable_destroy(listeners[0].ht, 1); /* 1 => free all values automatically */ + } + ratelimitDestruct(listeners[0].dflt_ratelimiter); + } + + /* Clean up all other sockets */ for (i = 1; i < nfd; i++) { if(listeners[i].sockName != NULL) { free(listeners[i].sockName); @@ -558,7 +558,7 @@ finalize_it: static inline rsRetVal findRatelimiter(lstn_t *pLstn, struct ucred *cred, ratelimit_t **prl) { - ratelimit_t *rl; + ratelimit_t *rl = NULL; int r; pid_t *keybuf; char pidbuf[256]; @@ -597,8 +597,11 @@ findRatelimiter(lstn_t *pLstn, struct ucred *cred, ratelimit_t **prl) } *prl = rl; + rl = NULL; finalize_it: + if(rl != NULL) + ratelimitDestruct(rl); if(*prl == NULL) *prl = pLstn->dflt_ratelimiter; RETiRet; @@ -734,7 +737,7 @@ copyescaped(uchar *dstbuf, uchar *inbuf, int inlen) static inline rsRetVal SubmitMsg(uchar *pRcv, int lenRcv, lstn_t *pLstn, struct ucred *cred, struct timeval *ts) { - msg_t *pMsg; + msg_t *pMsg = NULL; int lenMsg; int offs; int i; @@ -745,14 +748,8 @@ SubmitMsg(uchar *pRcv, int lenRcv, lstn_t *pLstn, struct ucred *cred, struct tim uchar bufParseTAG[CONF_TAG_MAXSIZE]; struct syslogTime st; time_t tt; - int lenProp; ratelimit_t *ratelimiter = NULL; - uchar propBuf[1024]; - uchar msgbuf[8192]; - uchar *pmsgbuf; - int toffs; /* offset for trusted properties */ struct syslogTime dummyTS; - struct json_object *json = NULL, *jval; DEFiRet; if(pLstn->bDiscardOwnMsgs && cred != NULL && cred->pid == glblGetOurPid()) { @@ -775,8 +772,8 @@ SubmitMsg(uchar *pRcv, int lenRcv, lstn_t *pLstn, struct ucred *cred, struct tim ++parse; ++offs; } - facil = LOG_FAC(pri); - sever = LOG_PRI(pri); + facil = pri2fac(pri); + sever = pri2sev(pri); findRatelimiter(pLstn, cred, &ratelimiter); /* ignore error, better so than others... */ @@ -794,36 +791,60 @@ SubmitMsg(uchar *pRcv, int lenRcv, lstn_t *pLstn, struct ucred *cred, struct tim } #endif + /* we now create our own message object and submit it to the queue */ + CHKiRet(msgConstructWithTime(&pMsg, &st, tt)); + /* created trusted properties */ if(cred != NULL && pLstn->bAnnotate) { - if((unsigned) (lenRcv + 4096) < sizeof(msgbuf)) { - pmsgbuf = msgbuf; - } else { - CHKmalloc(pmsgbuf = malloc(lenRcv+4096)); - } + uchar propBuf[1024]; + int lenProp; if (pLstn->bParseTrusted) { - json = json_object_new_object(); + struct json_object *json, *jval; + +#define CHKjson(operation, toBeFreed) \ + if((operation) == NULL) { \ + json_object_put(toBeFreed); \ + ABORT_FINALIZE(RS_RET_OUT_OF_MEMORY); \ + } + + CHKmalloc(json = json_object_new_object()); /* create value string, create field, and add it */ - jval = json_object_new_int(cred->pid); + CHKjson(jval = json_object_new_int(cred->pid), json); json_object_object_add(json, "pid", jval); - jval = json_object_new_int(cred->uid); + CHKjson(jval = json_object_new_int(cred->uid), json); json_object_object_add(json, "uid", jval); - jval = json_object_new_int(cred->gid); + CHKjson(jval = json_object_new_int(cred->gid), json); json_object_object_add(json, "gid", jval); if(getTrustedProp(cred, "comm", propBuf, sizeof(propBuf), &lenProp) == RS_RET_OK) { - jval = json_object_new_string((char*)propBuf); + CHKjson(jval = json_object_new_string((char*)propBuf), json); json_object_object_add(json, "appname", jval); } if(getTrustedExe(cred, propBuf, sizeof(propBuf), &lenProp) == RS_RET_OK) { - jval = json_object_new_string((char*)propBuf); + CHKjson(jval = json_object_new_string((char*)propBuf), json); json_object_object_add(json, "exe", jval); } if(getTrustedProp(cred, "cmdline", propBuf, sizeof(propBuf), &lenProp) == RS_RET_OK) { - jval = json_object_new_string((char*)propBuf); + CHKjson(jval = json_object_new_string((char*)propBuf), json); json_object_object_add(json, "cmd", jval); } +#undef CHKjson + + /* as per lumberjack spec, these properties need to go into + * the CEE root. + */ + msgAddJSON(pMsg, (uchar*)"!", json); + + MsgSetRawMsg(pMsg, (char*)pRcv, lenRcv); } else { + uchar msgbuf[8192]; + uchar *pmsgbuf = msgbuf; + int toffs; /* offset for trusted properties */ + + if((unsigned) (lenRcv + 4096) >= sizeof(msgbuf)) { + CHKmalloc(pmsgbuf = malloc(lenRcv+4096)); + } + memcpy(pmsgbuf, pRcv, lenRcv); memcpy(pmsgbuf+lenRcv, " @[", 3); toffs = lenRcv + 3; /* next free location */ @@ -853,14 +874,13 @@ SubmitMsg(uchar *pRcv, int lenRcv, lstn_t *pLstn, struct ucred *cred, struct tim pmsgbuf[toffs] = ']'; pmsgbuf[toffs+1] = '\0'; - pRcv = pmsgbuf; - lenRcv = toffs + 1; + MsgSetRawMsg(pMsg, (char*)pmsgbuf, toffs + 1); } + } else { + /* just add the unmodified message */ + MsgSetRawMsg(pMsg, (char*)pRcv, lenRcv); } - /* we now create our own message object and submit it to the queue */ - CHKiRet(msgConstructWithTime(&pMsg, &st, tt)); - MsgSetRawMsg(pMsg, (char*)pRcv, lenRcv); parser.SanitizeMsg(pMsg); lenMsg = pMsg->iLenRawMsg - offs; /* SanitizeMsg() may have changed the size */ MsgSetInputName(pMsg, pInputName); @@ -872,31 +892,24 @@ SubmitMsg(uchar *pRcv, int lenRcv, lstn_t *pLstn, struct ucred *cred, struct tim parse++; lenMsg--; /* '>' */ - if(json != NULL) { - /* as per lumberjack spec, these properties need to go into - * the CEE root. - */ - msgAddJSON(pMsg, (uchar*)"!", json); - } - if(ts == NULL) { if((pLstn->flags & IGNDATE)) { /* in this case, we still need to find out if we have a valid * datestamp or not .. and advance the parse pointer accordingly. */ if (datetime.ParseTIMESTAMP3339(&dummyTS, &parse, &lenMsg) != RS_RET_OK) { - datetime.ParseTIMESTAMP3164(&dummyTS, &parse, &lenMsg); + datetime.ParseTIMESTAMP3164(&dummyTS, &parse, &lenMsg, NO_PARSE3164_TZSTRING); } } else { if(datetime.ParseTIMESTAMP3339(&(pMsg->tTIMESTAMP), &parse, &lenMsg) != RS_RET_OK && - datetime.ParseTIMESTAMP3164(&(pMsg->tTIMESTAMP), &parse, &lenMsg) != RS_RET_OK) { + datetime.ParseTIMESTAMP3164(&(pMsg->tTIMESTAMP), &parse, &lenMsg, NO_PARSE3164_TZSTRING) != RS_RET_OK) { DBGPRINTF("we have a problem, invalid timestamp in msg!\n"); } } } else { /* if we pulled the time from the system, we need to update the message text */ uchar *tmpParse = parse; /* just to check correctness of TS */ if(datetime.ParseTIMESTAMP3339(&dummyTS, &tmpParse, &lenMsg) == RS_RET_OK || - datetime.ParseTIMESTAMP3164(&dummyTS, &tmpParse, &lenMsg) == RS_RET_OK) { + datetime.ParseTIMESTAMP3164(&dummyTS, &tmpParse, &lenMsg, NO_PARSE3164_TZSTRING) == RS_RET_OK) { /* We modify the message only if it contained a valid timestamp, * otherwise we do not touch it at all. */ datetime.formatTimestamp3164(&st, (char*)parse, 0); @@ -930,6 +943,10 @@ SubmitMsg(uchar *pRcv, int lenRcv, lstn_t *pLstn, struct ucred *cred, struct tim ratelimitAddMsg(ratelimiter, NULL, pMsg); STATSCOUNTER_INC(ctrSubmit, mutCtrSubmit); finalize_it: + if(iRet != RS_RET_OK) { + if(pMsg != NULL) + msgDestruct(&pMsg); + } RETiRet; } @@ -992,7 +1009,7 @@ static rsRetVal readSocket(lstn_t *pLstn) if(iRcvd > 0) { cred = NULL; ts = NULL; - if(pLstn->bUseCreds || pLstn->bUseSysTimeStamp) { + if(pLstn->bUseCreds) { for(cm = CMSG_FIRSTHDR(&msgh); cm; cm = CMSG_NXTHDR(&msgh, cm)) { # if HAVE_SCM_CREDENTIALS if( pLstn->bUseCreds @@ -1032,50 +1049,50 @@ activateListeners() int actSocks; DEFiRet; - /* first apply some config settings */ -# ifdef OS_SOLARIS - /* under solaris, we must NEVER process the local log socket, because - * it is implemented there differently. If we used it, we would actually - * delete it and render the system partly unusable. So don't do that. - * rgerhards, 2010-03-26 - */ - startIndexUxLocalSockets = 1; -# else - startIndexUxLocalSockets = runModConf->bOmitLocalLogging ? 1 : 0; -# endif - if(runModConf->pLogSockName != NULL) - listeners[0].sockName = runModConf->pLogSockName; - else if(sd_booted()) { - struct stat st; - if(stat(SYSTEMD_PATH_LOG, &st) != -1 && S_ISSOCK(st.st_mode)) { - listeners[0].sockName = (uchar*) SYSTEMD_PATH_LOG; + /* Initialize the system socket only if it's in use */ + if(startIndexUxLocalSockets == 0) { + /* first apply some config settings */ + listeners[0].sockName = UCHAR_CONSTANT(_PATH_LOG); + if(runModConf->pLogSockName != NULL) + listeners[0].sockName = runModConf->pLogSockName; + else if(sd_booted()) { + struct stat st; + if(stat(SYSTEMD_PATH_LOG, &st) != -1 && S_ISSOCK(st.st_mode)) { + listeners[0].sockName = (uchar*) SYSTEMD_PATH_LOG; + } } - } - if(runModConf->ratelimitIntervalSysSock > 0) { - if((listeners[0].ht = create_hashtable(100, hash_from_key_fn, key_equals_fn, NULL)) == NULL) { - /* in this case, we simply turn of rate-limiting */ - errmsg.LogError(0, NO_ERRCODE, "imuxsock: turning off rate limiting because we could not " - "create hash table\n"); - runModConf->ratelimitIntervalSysSock = 0; + if(runModConf->ratelimitIntervalSysSock > 0) { + if((listeners[0].ht = create_hashtable(100, hash_from_key_fn, key_equals_fn, NULL)) == NULL) { + /* in this case, we simply turn of rate-limiting */ + errmsg.LogError(0, NO_ERRCODE, "imuxsock: turning off rate limiting because we could not " + "create hash table\n"); + runModConf->ratelimitIntervalSysSock = 0; + } + } else { + listeners[0].ht = NULL; } + listeners[0].fd = -1; + listeners[0].hostName = NULL; + listeners[0].bParseHost = 0; + listeners[0].bCreatePath = 0; + listeners[0].ratelimitInterval = runModConf->ratelimitIntervalSysSock; + listeners[0].ratelimitBurst = runModConf->ratelimitBurstSysSock; + listeners[0].ratelimitSev = runModConf->ratelimitSeveritySysSock; + listeners[0].bUseCreds = (runModConf->bWritePidSysSock || runModConf->ratelimitIntervalSysSock || runModConf->bAnnotateSysSock || runModConf->bDiscardOwnMsgs || runModConf->bUseSysTimeStamp) ? 1 : 0; + listeners[0].bWritePid = runModConf->bWritePidSysSock; + listeners[0].bAnnotate = runModConf->bAnnotateSysSock; + listeners[0].bParseTrusted = runModConf->bParseTrusted; + listeners[0].bDiscardOwnMsgs = runModConf->bDiscardOwnMsgs; + listeners[0].bUnlink = runModConf->bUnlink; + listeners[0].bUseSysTimeStamp = runModConf->bUseSysTimeStamp; + listeners[0].flags = runModConf->bIgnoreTimestamp ? IGNDATE : NOFLAG; + listeners[0].flowCtl = runModConf->bUseFlowCtl ? eFLOWCTL_LIGHT_DELAY : eFLOWCTL_NO_DELAY; + CHKiRet(ratelimitNew(&listeners[0].dflt_ratelimiter, "imuxsock", NULL)); + ratelimitSetLinuxLike(listeners[0].dflt_ratelimiter, + listeners[0].ratelimitInterval, + listeners[0].ratelimitBurst); + ratelimitSetSeverity(listeners[0].dflt_ratelimiter,listeners[0].ratelimitSev); } - listeners[0].ratelimitInterval = runModConf->ratelimitIntervalSysSock; - listeners[0].ratelimitBurst = runModConf->ratelimitBurstSysSock; - listeners[0].ratelimitSev = runModConf->ratelimitSeveritySysSock; - listeners[0].bUseCreds = (runModConf->bWritePidSysSock || runModConf->ratelimitIntervalSysSock || runModConf->bAnnotateSysSock || runModConf->bDiscardOwnMsgs) ? 1 : 0; - listeners[0].bWritePid = runModConf->bWritePidSysSock; - listeners[0].bAnnotate = runModConf->bAnnotateSysSock; - listeners[0].bParseTrusted = runModConf->bParseTrusted; - listeners[0].bDiscardOwnMsgs = runModConf->bDiscardOwnMsgs; - listeners[0].bUnlink = runModConf->bUnlink; - listeners[0].bUseSysTimeStamp = runModConf->bUseSysTimeStamp; - listeners[0].flags = runModConf->bIgnoreTimestamp ? IGNDATE : NOFLAG; - listeners[0].flowCtl = runModConf->bUseFlowCtl ? eFLOWCTL_LIGHT_DELAY : eFLOWCTL_NO_DELAY; - CHKiRet(ratelimitNew(&listeners[0].dflt_ratelimiter, "imuxsock", NULL)); - ratelimitSetLinuxLike(listeners[0].dflt_ratelimiter, - listeners[0].ratelimitInterval, - listeners[0].ratelimitBurst); - ratelimitSetSeverity(listeners[0].dflt_ratelimiter,listeners[0].ratelimitSev); sd_fds = sd_listen_fds(0); if(sd_fds < 0) { @@ -1215,31 +1232,31 @@ CODESTARTnewInpInst continue; if(!strcmp(inppblk.descr[i].name, "socket")) { inst->sockName = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); - } else if(!strcmp(modpblk.descr[i].name, "createpath")) { + } else if(!strcmp(inppblk.descr[i].name, "createpath")) { inst->bCreatePath = (int) pvals[i].val.d.n; - } else if(!strcmp(modpblk.descr[i].name, "parsetrusted")) { + } else if(!strcmp(inppblk.descr[i].name, "parsetrusted")) { inst->bParseTrusted = (int) pvals[i].val.d.n; - } else if(!strcmp(modpblk.descr[i].name, "ignoreownmessages")) { + } else if(!strcmp(inppblk.descr[i].name, "ignoreownmessages")) { inst->bDiscardOwnMsgs = (int) pvals[i].val.d.n; - } else if(!strcmp(modpblk.descr[i].name, "unlink")) { + } else if(!strcmp(inppblk.descr[i].name, "unlink")) { inst->bUnlink = (int) pvals[i].val.d.n; - } else if(!strcmp(modpblk.descr[i].name, "hostname")) { + } else if(!strcmp(inppblk.descr[i].name, "hostname")) { inst->pLogHostName = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); - } else if(!strcmp(modpblk.descr[i].name, "ignoretimestamp")) { + } else if(!strcmp(inppblk.descr[i].name, "ignoretimestamp")) { inst->bIgnoreTimestamp = (int) pvals[i].val.d.n; - } else if(!strcmp(modpblk.descr[i].name, "flowcontrol")) { + } else if(!strcmp(inppblk.descr[i].name, "flowcontrol")) { inst->bUseFlowCtl = (int) pvals[i].val.d.n; - } else if(!strcmp(modpblk.descr[i].name, "usesystimestamp")) { + } else if(!strcmp(inppblk.descr[i].name, "usesystimestamp")) { inst->bUseSysTimeStamp = (int) pvals[i].val.d.n; - } else if(!strcmp(modpblk.descr[i].name, "annotate")) { + } else if(!strcmp(inppblk.descr[i].name, "annotate")) { inst->bAnnotate = (int) pvals[i].val.d.n; - } else if(!strcmp(modpblk.descr[i].name, "usepidfromsystem")) { + } else if(!strcmp(inppblk.descr[i].name, "usepidfromsystem")) { inst->bWritePid = (int) pvals[i].val.d.n; - } else if(!strcmp(modpblk.descr[i].name, "ratelimit.interval")) { + } else if(!strcmp(inppblk.descr[i].name, "ratelimit.interval")) { inst->ratelimitInterval = (int) pvals[i].val.d.n; - } else if(!strcmp(modpblk.descr[i].name, "ratelimit.burst")) { + } else if(!strcmp(inppblk.descr[i].name, "ratelimit.burst")) { inst->ratelimitBurst = (int) pvals[i].val.d.n; - } else if(!strcmp(modpblk.descr[i].name, "ratelimit.severity")) { + } else if(!strcmp(inppblk.descr[i].name, "ratelimit.severity")) { inst->ratelimitSeverity = (int) pvals[i].val.d.n; } else { dbgprintf("imuxsock: program error, non-handled " @@ -1256,11 +1273,14 @@ BEGINendCnfLoad CODESTARTendCnfLoad if(!loadModConf->configSetViaV2Method) { /* persist module-specific settings from legacy config system */ + /* these are used to initialize the system log socket (listeners[0]) */ loadModConf->bOmitLocalLogging = cs.bOmitLocalLogging; loadModConf->pLogSockName = cs.pLogSockName; loadModConf->bIgnoreTimestamp = cs.bIgnoreTimestampSysSock; + loadModConf->bUseSysTimeStamp = cs.bUseSysTimeStampSysSock; loadModConf->bUseFlowCtl = cs.bUseFlowCtlSysSock; loadModConf->bAnnotateSysSock = cs.bAnnotateSysSock; + loadModConf->bWritePidSysSock = cs.bWritePidSysSock; loadModConf->bParseTrusted = cs.bParseTrusted; loadModConf->ratelimitIntervalSysSock = cs.ratelimitIntervalSysSock; loadModConf->ratelimitBurstSysSock = cs.ratelimitBurstSysSock; @@ -1282,12 +1302,40 @@ ENDcheckCnf BEGINactivateCnfPrePrivDrop instanceConf_t *inst; + int nLstn; + int i; CODESTARTactivateCnfPrePrivDrop runModConf = pModConf; +# ifdef OS_SOLARIS + /* under solaris, we must NEVER process the local log socket, because + * it is implemented there differently. If we used it, we would actually + * delete it and render the system partly unusable. So don't do that. + * rgerhards, 2010-03-26 + */ + startIndexUxLocalSockets = 1; +# else + startIndexUxLocalSockets = runModConf->bOmitLocalLogging ? 1 : 0; +# endif + /* we first calculate the number of listeners so that we can + * appropriately size the listener array. Note that we will + * always allocate memory for the system log socket. + */ + nLstn = 0; for(inst = runModConf->root ; inst != NULL ; inst = inst->next) { - addListner(inst); + ++nLstn; + } + if(nLstn > 0 || startIndexUxLocalSockets == 0) { + DBGPRINTF("imuxsock: allocating memory for %d listeners\n", nLstn); + CHKmalloc(listeners = realloc(listeners, (1+nLstn)*sizeof(lstn_t))); + for(i = 1 ; i < nLstn ; ++i) { + listeners[i].sockName = NULL; + listeners[i].fd = -1; + } + for(inst = runModConf->root ; inst != NULL ; inst = inst->next) { + addListner(inst); + } + CHKiRet(activateListeners()); } - CHKiRet(activateListeners()); finalize_it: ENDactivateCnfPrePrivDrop @@ -1325,6 +1373,11 @@ BEGINrunInput #endif CODESTARTrunInput + CHKmalloc(pReadfds); + if(startIndexUxLocalSockets == 1 && nfd == 1) { + /* No sockets were configured, no reason to run. */ + ABORT_FINALIZE(RS_RET_OK); + } /* this is an endless loop - it is terminated when the thread is * signalled to do so. This, however, is handled by the framework, * right into the sleep below. @@ -1372,7 +1425,6 @@ CODESTARTrunInput finalize_it: freeFdSet(pReadfds); - RETiRet; ENDrunInput @@ -1415,6 +1467,7 @@ ENDafterRun BEGINmodExit CODESTARTmodExit + free(listeners); if(pInputName != NULL) prop.Destruct(&pInputName); @@ -1477,7 +1530,6 @@ static rsRetVal resetConfigVariables(uchar __attribute__((unused)) *pp, void __a BEGINmodInit() - int i; CODESTARTmodInit *ipIFVersProvided = CURR_MOD_IF_VERSION; /* we only support the current interface specification */ CODEmodInit_QueryRegCFSLineHdlr @@ -1507,34 +1559,6 @@ CODEmodInit_QueryRegCFSLineHdlr */ pLocalHostIP = glbl.GetLocalHostIP(); - /* init system log socket settings */ - listeners[0].flags = IGNDATE; - listeners[0].sockName = UCHAR_CONSTANT(_PATH_LOG); - listeners[0].hostName = NULL; - listeners[0].flowCtl = eFLOWCTL_NO_DELAY; - listeners[0].fd = -1; - listeners[0].bParseHost = 0; - listeners[0].bUseCreds = 0; - listeners[0].bAnnotate = 0; - listeners[0].bParseTrusted = 0; - listeners[0].bDiscardOwnMsgs = 1; - listeners[0].bUnlink = 1; - listeners[0].bCreatePath = 0; - listeners[0].bUseSysTimeStamp = 1; - if((listeners[0].ht = create_hashtable(100, hash_from_key_fn, key_equals_fn, - (void(*)(void*))ratelimitDestruct)) == NULL) { - /* in this case, we simply turn off rate-limiting */ - DBGPRINTF("imuxsock: turning off rate limiting for system socket " - "because we could not create hash table\n"); - listeners[0].ratelimitInterval = 0; - } - - /* initialize socket names */ - for(i = 1 ; i < MAXFUNIX ; ++i) { - listeners[i].sockName = NULL; - listeners[i].fd = -1; - } - /* register config file handlers */ CHKiRet(omsdRegCFSLineHdlr((uchar *)"inputunixlistensocketignoremsgtimestamp", 0, eCmdHdlrBinary, NULL, &cs.bIgnoreTimestamp, STD_LOADABLE_MODULE_ID)); @@ -1594,13 +1618,13 @@ CODEmodInit_QueryRegCFSLineHdlr CHKiRet(statsobj.SetName(modStats, UCHAR_CONSTANT("imuxsock"))); STATSCOUNTER_INIT(ctrSubmit, mutCtrSubmit); CHKiRet(statsobj.AddCounter(modStats, UCHAR_CONSTANT("submitted"), - ctrType_IntCtr, &ctrSubmit)); + ctrType_IntCtr, CTR_FLAG_RESETTABLE, &ctrSubmit)); STATSCOUNTER_INIT(ctrLostRatelimit, mutCtrLostRatelimit); CHKiRet(statsobj.AddCounter(modStats, UCHAR_CONSTANT("ratelimit.discarded"), - ctrType_IntCtr, &ctrLostRatelimit)); + ctrType_IntCtr, CTR_FLAG_RESETTABLE, &ctrLostRatelimit)); STATSCOUNTER_INIT(ctrNumRatelimiters, mutCtrNumRatelimiters); CHKiRet(statsobj.AddCounter(modStats, UCHAR_CONSTANT("ratelimit.numratelimiters"), - ctrType_IntCtr, &ctrNumRatelimiters)); + ctrType_IntCtr, CTR_FLAG_RESETTABLE, &ctrNumRatelimiters)); CHKiRet(statsobj.ConstructFinalize(modStats)); ENDmodInit diff --git a/plugins/imzmq3/Makefile.in b/plugins/imzmq3/Makefile.in index bee6d05..761c4f6 100644 --- a/plugins/imzmq3/Makefile.in +++ b/plugins/imzmq3/Makefile.in @@ -154,7 +154,6 @@ GREP = @GREP@ GSS_LIBS = @GSS_LIBS@ GUARDTIME_CFLAGS = @GUARDTIME_CFLAGS@ GUARDTIME_LIBS = @GUARDTIME_LIBS@ -HAVE_LIBGCRYPT_CONFIG = @HAVE_LIBGCRYPT_CONFIG@ HAVE_MYSQL_CONFIG = @HAVE_MYSQL_CONFIG@ HAVE_ORACLE_CONFIG = @HAVE_ORACLE_CONFIG@ HAVE_PGSQL_CONFIG = @HAVE_PGSQL_CONFIG@ @@ -175,14 +174,15 @@ LEXLIB = @LEXLIB@ LEX_OUTPUT_ROOT = @LEX_OUTPUT_ROOT@ LIBDBI_CFLAGS = @LIBDBI_CFLAGS@ LIBDBI_LIBS = @LIBDBI_LIBS@ -LIBEE_CFLAGS = @LIBEE_CFLAGS@ -LIBEE_LIBS = @LIBEE_LIBS@ LIBESTR_CFLAGS = @LIBESTR_CFLAGS@ LIBESTR_LIBS = @LIBESTR_LIBS@ LIBGCRYPT_CFLAGS = @LIBGCRYPT_CFLAGS@ +LIBGCRYPT_CONFIG = @LIBGCRYPT_CONFIG@ LIBGCRYPT_LIBS = @LIBGCRYPT_LIBS@ LIBLOGGING_CFLAGS = @LIBLOGGING_CFLAGS@ LIBLOGGING_LIBS = @LIBLOGGING_LIBS@ +LIBLOGGING_STDLOG_CFLAGS = @LIBLOGGING_STDLOG_CFLAGS@ +LIBLOGGING_STDLOG_LIBS = @LIBLOGGING_STDLOG_LIBS@ LIBLOGNORM_CFLAGS = @LIBLOGNORM_CFLAGS@ LIBLOGNORM_LIBS = @LIBLOGNORM_LIBS@ LIBM = @LIBM@ @@ -207,6 +207,8 @@ NM = @NM@ NMEDIT = @NMEDIT@ OBJDUMP = @OBJDUMP@ OBJEXT = @OBJEXT@ +OPENSSL_CFLAGS = @OPENSSL_CFLAGS@ +OPENSSL_LIBS = @OPENSSL_LIBS@ ORACLE_CFLAGS = @ORACLE_CFLAGS@ ORACLE_LIBS = @ORACLE_LIBS@ OTOOL = @OTOOL@ @@ -388,22 +390,25 @@ distclean-compile: @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/imzmq3_la-imzmq3.Plo@am__quote@ .c.o: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(COMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ $< -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Po +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.o$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c $< +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ $< .c.obj: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(COMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ `$(CYGPATH_W) '$<'` -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Po +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.obj$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ `$(CYGPATH_W) '$<'` &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c `$(CYGPATH_W) '$<'` +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ `$(CYGPATH_W) '$<'` .c.lo: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(LTCOMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ $< -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Plo +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.lo$$||'`;\ +@am__fastdepCC_TRUE@ $(LTCOMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Plo @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=yes @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ @am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(LTCOMPILE) -c -o $@ $< diff --git a/plugins/imzmq3/README b/plugins/imzmq3/README index 88653b8..9a108a0 100644 --- a/plugins/imzmq3/README +++ b/plugins/imzmq3/README @@ -1,24 +1,59 @@ ZeroMQ 3.x Input Plugin Building this plugin: -Requires libzmq and libczmq. First, install libzmq from the HEAD on github: -http://github.com/zeromq/libzmq. You can clone the repository, build, then -install it. The directions for doing so are there in the readme. Then, do -the same for libczmq: http://github.com/zeromq/czmq. At some point, the 3.1 -version of libzmq will be released, and a supporting version of libczmq. -At that time, you could simply download and install the tarballs instead of -using git to clone the repositories. Those tarballs (when available) can -be found at http://download.zeromq.org. As of this writing (5/31/2012), the -most recent version of czmq (1.1.0) and libzmq (3.1.0-beta) will not compile -properly. +Requires libzmq and libczmq. First, download the tarballs of both libzmq +and its supporting libczmq from http://download.zeromq.org. As of this +writing (04/23/2013), the most recent versions of libzmq and czmq are +3.2.2 and 1.3.2 respectively. Configure, build, and then install both libs. Imzmq3 allows you to push data into rsyslog from a zeromq socket. The example below binds a SUB socket to port 7172, and then any messages with the topic "foo" will be pushed into rsyslog. +Please note: +This plugin only supports the newer (v7) config format. Legacy config support +was removed. + Example Rsyslog.conf snippet: ------------------------------------------------------------------------------- - -$InputZmq3ServerRun action=BIND,type=SUB,description=tcp://*:7172,subscribe=foo +module(load="imzmq3" ioThreads="1") +input(type="imzmq3" action="CONNECT" socktype="SUB" description="tcp://*:7172" subscribe="foo,bar") ------------------------------------------------------------------------------- +Note you can specify multiple subscriptions with a comma-delimited list, with +no spaces between values. + +The only global parameter for this plugin is ioThreads, which is optional and +probably best left to the zmq default unless you know exactly what you are +doing. + +The instance-level parameters are: + +Required +description +subscribe (required if the sockType is SUB) + +Optional +sockType (defaults to SUB) +action (defaults to BIND +sndHWM +rcvHWM +identity +sndBuf +rcvBuf +linger +backlog +sndTimeout +rcvTimeout +maxMsgSize +rate +recoveryIVL +multicastHops +reconnectIVL +reconnectIVLMax +ipv4Only +affinity + +These all correspond to zmq optional settings. Except where noted, the defaults +are the zmq defaults if not set. See http://api.zeromq.org/3-2:zmq-setsockopt +for info on these. diff --git a/plugins/imzmq3/imzmq3.c b/plugins/imzmq3/imzmq3.c index 52c12a5..08b1dbe 100644 --- a/plugins/imzmq3/imzmq3.c +++ b/plugins/imzmq3/imzmq3.c @@ -19,20 +19,21 @@ * License along with this program. If not, see * <http://www.gnu.org/licenses/>. * - * Author: David Kelly - * <davidk@talksum.com> + * Authors: + * David Kelly <davidk@talksum.com> + * Hongfei Cheng <hongfeic@talksum.com> */ + +#include "config.h" +#include "rsyslog.h" + #include <assert.h> #include <errno.h> #include <stdlib.h> #include <string.h> #include <unistd.h> - -#include "rsyslog.h" - #include "cfsysline.h" -#include "config.h" #include "dirty.h" #include "errmsg.h" #include "glbl.h" @@ -49,6 +50,7 @@ MODULE_TYPE_INPUT MODULE_TYPE_NOKEEP +MODULE_CNFNAME("imzmq3"); /* convienent symbols to denote a socket we want to bind * vs one we want to just connect to @@ -83,47 +85,67 @@ typedef struct _poller_data { thrdInfo_t* thread; } poller_data; -typedef struct _socket_info { - int type; - int action; - char* description; - int sndHWM; /* if you want more than 2^32 messages, */ - int rcvHWM; /* then pass in 0 (the default). */ - char* identity; - char** subscriptions; - ruleset_t* ruleset; - int sndBuf; - int rcvBuf; - int linger; - int backlog; - int sndTimeout; - int rcvTimeout; - int maxMsgSize; - int rate; - int recoveryIVL; - int multicastHops; - int reconnectIVL; - int reconnectIVLMax; - int ipv4Only; - int affinity; - -} socket_info; +/* a linked-list of subscription topics */ +typedef struct sublist_t { + char* subscribe; + struct sublist_t* next; +} sublist; + +struct instanceConf_s { + int type; + int action; + char* description; + int sndHWM; /* if you want more than 2^32 messages, */ + int rcvHWM; /* then pass in 0 (the default). */ + char* identity; + sublist* subscriptions; + int sndBuf; + int rcvBuf; + int linger; + int backlog; + int sndTimeout; + int rcvTimeout; + int maxMsgSize; + int rate; + int recoveryIVL; + int multicastHops; + int reconnectIVL; + int reconnectIVLMax; + int ipv4Only; + int affinity; + uchar* pszBindRuleset; + ruleset_t* pBindRuleset; + struct instanceConf_s* next; + +}; + +struct modConfData_s { + rsconf_t* pConf; + instanceConf_t* root; + instanceConf_t* tail; + int io_threads; +}; +struct lstn_s { + struct lstn_s* next; + void* sock; + ruleset_t* pRuleset; +}; /* ---------------------------------------------------------------------------- * Static definitions/initializations. */ -static socket_info* s_socketInfo = NULL; -static size_t s_nitems = 0; -static prop_t * s_namep = NULL; +static modConfData_t* runModConf = NULL; +static struct lstn_s* lcnfRoot = NULL; +static struct lstn_s* lcnfLast = NULL; +static prop_t* s_namep = NULL; static zloop_t* s_zloop = NULL; -static int s_io_threads = 1; static zctx_t* s_context = NULL; -static ruleset_t* s_ruleset = NULL; static socket_type socketTypes[] = { - {"SUB", ZMQ_SUB }, - {"PULL", ZMQ_PULL }, - {"XSUB", ZMQ_XSUB } + {"SUB", ZMQ_SUB }, + {"PULL", ZMQ_PULL }, + {"ROUTER", ZMQ_ROUTER }, + {"XSUB", ZMQ_XSUB } }; static socket_action socketActions[] = { @@ -131,6 +153,48 @@ static socket_action socketActions[] = { {"CONNECT", ACTION_CONNECT}, }; +static struct cnfparamdescr modpdescr[] = { + { "ioThreads", eCmdHdlrInt, 0 }, +}; + +static struct cnfparamblk modpblk = { + CNFPARAMBLK_VERSION, + sizeof(modpdescr)/sizeof(struct cnfparamdescr), + modpdescr +}; + +static struct cnfparamdescr inppdescr[] = { + { "description", eCmdHdlrGetWord, 0 }, + { "sockType", eCmdHdlrGetWord, 0 }, + { "subscribe", eCmdHdlrGetWord, 0 }, + { "ruleset", eCmdHdlrGetWord, 0 }, + { "action", eCmdHdlrGetWord, 0 }, + { "sndHWM", eCmdHdlrInt, 0 }, + { "rcvHWM", eCmdHdlrInt, 0 }, + { "identity", eCmdHdlrGetWord, 0 }, + { "sndBuf", eCmdHdlrInt, 0 }, + { "rcvBuf", eCmdHdlrInt, 0 }, + { "linger", eCmdHdlrInt, 0 }, + { "backlog", eCmdHdlrInt, 0 }, + { "sndTimeout", eCmdHdlrInt, 0 }, + { "rcvTimeout", eCmdHdlrInt, 0 }, + { "maxMsgSize", eCmdHdlrInt, 0 }, + { "rate", eCmdHdlrInt, 0 }, + { "recoveryIVL", eCmdHdlrInt, 0 }, + { "multicastHops", eCmdHdlrInt, 0 }, + { "reconnectIVL", eCmdHdlrInt, 0 }, + { "reconnectIVLMax", eCmdHdlrInt, 0 }, + { "ipv4Only", eCmdHdlrInt, 0 }, + { "affinity", eCmdHdlrInt, 0 } +}; + +static struct cnfparamblk inppblk = { + CNFPARAMBLK_VERSION, + sizeof(inppdescr)/sizeof(struct cnfparamdescr), + inppdescr +}; + +#include "im-helper.h" /* must be included AFTER the type definitions! */ /* ---------------------------------------------------------------------------- * Helper functions @@ -179,15 +243,16 @@ static int getSocketAction(char* name) { } -static void setDefaults(socket_info* info) { - info->type = ZMQ_SUB; - info->action = ACTION_BIND; +static void setDefaults(instanceConf_t* info) { + info->type = -1; + info->action = -1; info->description = NULL; - info->sndHWM = 0; - info->rcvHWM = 0; + info->sndHWM = -1; + info->rcvHWM = -1; info->identity = NULL; info->subscriptions = NULL; - info->ruleset = NULL; + info->pszBindRuleset = NULL; + info->pBindRuleset = NULL; info->sndBuf = -1; info->rcvBuf = -1; info->linger = -1; @@ -202,93 +267,49 @@ static void setDefaults(socket_info* info) { info->reconnectIVLMax = -1; info->ipv4Only = -1; info->affinity = -1; - + info->next = NULL; }; - -/* The config string should look like: - * "action=AAA,type=TTT,description=DDD,sndHWM=SSS,rcvHWM=RRR,subscribe='xxx',subscribe='yyy'" - * +/* given a comma separated list of subscriptions, create a char* array of them + * to set later */ -static rsRetVal parseConfig(char* config, socket_info* info) { - int nsubs = 0; - - char* binding; - char* ptr1; - for (binding = strtok_r(config, ",", &ptr1); - binding != NULL; - binding = strtok_r(NULL, ",", &ptr1)) { - - /* Each binding looks like foo=bar */ - char * sep = strchr(binding, '='); - if (sep == NULL) - { - errmsg.LogError(0, NO_ERRCODE, - "Invalid argument format %s, ignoring ...", - binding); - continue; - } +static rsRetVal parseSubscriptions(char* subscribes, sublist** subList){ + char* tok = strtok(subscribes, ","); + sublist* currentSub; + sublist* head; + DEFiRet; - /* Replace '=' with '\0'. */ - *sep = '\0'; - - char * val = sep + 1; - - if (strcmp(binding, "action") == 0) { - info->action = getSocketAction(val); - } else if (strcmp(binding, "type") == 0) { - info->type = getSocketType(val); - } else if (strcmp(binding, "description") == 0) { - info->description = strdup(val); - } else if (strcmp(binding, "sndHWM") == 0) { - info->sndHWM = atoi(val); - } else if (strcmp(binding, "rcvHWM") == 0) { - info->sndHWM = atoi(val); - } else if (strcmp(binding, "subscribe") == 0) { - /* Add the subscription value to the list.*/ - char * substr = NULL; - substr = strdup(val); - info->subscriptions = realloc(info->subscriptions, sizeof(char *) * nsubs + 1); - info->subscriptions[nsubs] = substr; - ++nsubs; - } else if (strcmp(binding, "sndBuf") == 0) { - info->sndBuf = atoi(val); - } else if (strcmp(binding, "rcvBuf") == 0) { - info->rcvBuf = atoi(val); - } else if (strcmp(binding, "linger") == 0) { - info->linger = atoi(val); - } else if (strcmp(binding, "backlog") == 0) { - info->backlog = atoi(val); - } else if (strcmp(binding, "sndTimeout") == 0) { - info->sndTimeout = atoi(val); - } else if (strcmp(binding, "rcvTimeout") == 0) { - info->rcvTimeout = atoi(val); - } else if (strcmp(binding, "maxMsgSize") == 0) { - info->maxMsgSize = atoi(val); - } else if (strcmp(binding, "rate") == 0) { - info->rate = atoi(val); - } else if (strcmp(binding, "recoveryIVL") == 0) { - info->recoveryIVL = atoi(val); - } else if (strcmp(binding, "multicastHops") == 0) { - info->multicastHops = atoi(val); - } else if (strcmp(binding, "reconnectIVL") == 0) { - info->reconnectIVL = atoi(val); - } else if (strcmp(binding, "reconnectIVLMax") == 0) { - info->reconnectIVLMax = atoi(val); - } else if (strcmp(binding, "ipv4Only") == 0) { - info->ipv4Only = atoi(val); - } else if (strcmp(binding, "affinity") == 0) { - info->affinity = atoi(val); - } else { - errmsg.LogError(0, NO_ERRCODE, "Unknown argument %s", binding); - return RS_RET_INVALID_PARAMS; + /* create empty list */ + CHKmalloc(*subList = (sublist*)MALLOC(sizeof(sublist))); + head = *subList; + head->next = NULL; + head->subscribe=NULL; + currentSub=head; + + if(tok) { + head->subscribe=strdup(tok); + for(tok=strtok(NULL, ","); tok!=NULL;tok=strtok(NULL, ",")) { + CHKmalloc(currentSub->next = (sublist*)MALLOC(sizeof(sublist))); + currentSub=currentSub->next; + currentSub->subscribe=strdup(tok); + currentSub->next=NULL; } + } else { + /* make empty subscription ie subscribe="" */ + head->subscribe=strdup(""); } - - return RS_RET_OK; + /* TODO: temporary logging */ + currentSub = head; + DBGPRINTF("imzmq3: Subscriptions:"); + for(currentSub = head; currentSub != NULL; currentSub=currentSub->next) { + DBGPRINTF("'%s'", currentSub->subscribe); + } + DBGPRINTF("\n"); +finalize_it: + RETiRet; } -static rsRetVal validateConfig(socket_info* info) { +static rsRetVal validateConfig(instanceConf_t* info) { if (info->type == -1) { errmsg.LogError(0, RS_RET_INVALID_PARAMS, @@ -307,7 +328,7 @@ static rsRetVal validateConfig(socket_info* info) { } if(info->type == ZMQ_SUB && info->subscriptions == NULL) { errmsg.LogError(0, RS_RET_INVALID_PARAMS, - "SUB sockets need at least one subscription"); + "SUB sockets need a subscription"); return RS_RET_INVALID_PARAMS; } if(info->type != ZMQ_SUB && info->subscriptions != NULL) { @@ -320,39 +341,40 @@ static rsRetVal validateConfig(socket_info* info) { static rsRetVal createContext() { if (s_context == NULL) { - errmsg.LogError(0, NO_ERRCODE, "creating zctx."); + DBGPRINTF("imzmq3: creating zctx..."); + zsys_handler_set(NULL); s_context = zctx_new(); if (s_context == NULL) { errmsg.LogError(0, RS_RET_INVALID_PARAMS, "zctx_new failed: %s", - strerror(errno)); + zmq_strerror(errno)); /* DK: really should do better than invalid params...*/ return RS_RET_INVALID_PARAMS; } - - if (s_io_threads > 1) { - errmsg.LogError(0, NO_ERRCODE, "setting io worker threads to %d", s_io_threads); - zctx_set_iothreads(s_context, s_io_threads); + DBGPRINTF("success!\n"); + if (runModConf->io_threads > 1) { + DBGPRINTF("setting io worker threads to %d\n", runModConf->io_threads); + zctx_set_iothreads(s_context, runModConf->io_threads); } } return RS_RET_OK; } -static rsRetVal createSocket(socket_info* info, void** sock) { - size_t ii; +static rsRetVal createSocket(instanceConf_t* info, void** sock) { int rv; + sublist* sub; *sock = zsocket_new(s_context, info->type); if (!sock) { - errmsg.LogError(0, + errmsg.LogError(0, RS_RET_INVALID_PARAMS, "zsocket_new failed: %s, for type %d", - strerror(errno),info->type); - /* DK: invalid params seems right here */ + zmq_strerror(errno),info->type); + /* DK: invalid params seems right here */ return RS_RET_INVALID_PARAMS; } - + DBGPRINTF("imzmq3: socket of type %d created successfully\n", info->type) /* Set options *before* the connect/bind. */ if (info->identity) zsocket_set_identity(*sock, info->identity); if (info->sndBuf > -1) zsocket_set_sndbuf(*sock, info->sndBuf); @@ -369,38 +391,36 @@ static rsRetVal createSocket(socket_info* info, void** sock) { if (info->reconnectIVLMax > -1) zsocket_set_reconnect_ivl_max(*sock, info->reconnectIVLMax); if (info->ipv4Only > -1) zsocket_set_ipv4only(*sock, info->ipv4Only); if (info->affinity > -1) zsocket_set_affinity(*sock, info->affinity); - - /* since HWM have defaults, we always set them. No return codes to check, either.*/ - zsocket_set_sndhwm(*sock, info->sndHWM); - zsocket_set_rcvhwm(*sock, info->rcvHWM); - + if (info->sndHWM > -1 ) zsocket_set_sndhwm(*sock, info->sndHWM); + if (info->rcvHWM > -1 ) zsocket_set_rcvhwm(*sock, info->rcvHWM); /* Set subscriptions.*/ if (info->type == ZMQ_SUB) { - for (ii = 0; ii < sizeof(info->subscriptions)/sizeof(char*); ++ii) - zsocket_set_subscribe(*sock, info->subscriptions[ii]); + for(sub = info->subscriptions; sub!=NULL; sub=sub->next) { + zsocket_set_subscribe(*sock, sub->subscribe); + } } - - /* Do the bind/connect... */ if (info->action==ACTION_CONNECT) { rv = zsocket_connect(*sock, info->description); - if (rv < 0) { + if (rv == -1) { errmsg.LogError(0, RS_RET_INVALID_PARAMS, "zmq_connect using %s failed: %s", - info->description, strerror(errno)); + info->description, zmq_strerror(errno)); return RS_RET_INVALID_PARAMS; } + DBGPRINTF("imzmq3: connect for %s successful\n",info->description); } else { rv = zsocket_bind(*sock, info->description); - if (rv <= 0) { + if (rv == -1) { errmsg.LogError(0, RS_RET_INVALID_PARAMS, "zmq_bind using %s failed: %s", - info->description, strerror(errno)); + info->description, zmq_strerror(errno)); return RS_RET_INVALID_PARAMS; } + DBGPRINTF("imzmq3: bind for %s successful\n",info->description); } return RS_RET_OK; } @@ -409,89 +429,138 @@ static rsRetVal createSocket(socket_info* info, void** sock) { * Module endpoints */ -/* accept a new ruleset to bind. Checks if it exists and complains, if not. Note - * that this makes the assumption that after the bind ruleset is called in the config, - * another call will be made to add an endpoint. -*/ -static rsRetVal -set_ruleset(void __attribute__((unused)) *pVal, uchar *pszName) { - ruleset_t* ruleset_ptr; - rsRetVal localRet; - DEFiRet; - - localRet = ruleset.GetRuleset(ourConf, &ruleset_ptr, pszName); - if(localRet == RS_RET_NOT_FOUND) { - errmsg.LogError(0, NO_ERRCODE, "error: " - "ruleset '%s' not found - ignored", pszName); - } - CHKiRet(localRet); - s_ruleset = ruleset_ptr; - DBGPRINTF("imzmq3 current bind ruleset '%s'\n", pszName); - -finalize_it: - free(pszName); /* no longer needed */ - RETiRet; -} /* add an actual endpoint */ -static rsRetVal add_endpoint(void __attribute__((unused)) * oldp, uchar * valp) { +static rsRetVal createInstance(instanceConf_t** pinst) { DEFiRet; + instanceConf_t* inst; + CHKmalloc(inst = MALLOC(sizeof(instanceConf_t))); - /* increment number of items and store old num items, as it will be handy.*/ - size_t idx = s_nitems++; - - /* allocate a new socket_info array to accomidate this new endpoint*/ - socket_info* tmpSocketInfo; - CHKmalloc(tmpSocketInfo = (socket_info*)MALLOC(sizeof(socket_info) * s_nitems)); + /* set defaults into new instance config struct */ + setDefaults(inst); - /* copy existing socket_info across into new array, if any, and free old storage*/ - if(idx) { - memcpy(tmpSocketInfo, s_socketInfo, sizeof(socket_info) * idx); - free(s_socketInfo); + /* add this to the config */ + if (runModConf->root == NULL || runModConf->tail == NULL) { + runModConf->tail = runModConf->root = inst; + } else { + runModConf->tail->next = inst; + runModConf->tail = inst; } + *pinst = inst; +finalize_it: + RETiRet; +} - /* set the static to hold the new array */ - s_socketInfo = tmpSocketInfo; - - /* point to the new one */ - socket_info* sockInfo = &s_socketInfo[idx]; - - /* set defaults for the new socket info */ - setDefaults(sockInfo); - - /* Make a writeable copy of the string so we can use strtok - in the parseConfig call */ - char * copy = NULL; - CHKmalloc(copy = strdup((char *) valp)); +static rsRetVal createListener(struct cnfparamvals* pvals) { + instanceConf_t* inst; + int i; + DEFiRet; - /* parse the config string */ - CHKiRet(parseConfig(copy, sockInfo)); + CHKiRet(createInstance(&inst)); + for(i = 0 ; i < inppblk.nParams ; ++i) { + if(!pvals[i].bUsed) + continue; + if(!strcmp(inppblk.descr[i].name, "ruleset")) { + inst->pszBindRuleset = (uchar *)es_str2cstr(pvals[i].val.d.estr, NULL); + } else if(!strcmp(inppblk.descr[i].name, "description")) { + inst->description = es_str2cstr(pvals[i].val.d.estr, NULL); + } else if(!strcmp(inppblk.descr[i].name, "sockType")){ + inst->type = getSocketType(es_str2cstr(pvals[i].val.d.estr, NULL)); + } else if(!strcmp(inppblk.descr[i].name, "action")){ + inst->action = getSocketAction(es_str2cstr(pvals[i].val.d.estr, NULL)); + } else if(!strcmp(inppblk.descr[i].name, "sndHWM")) { + inst->sndHWM = (int) pvals[i].val.d.n; + } else if(!strcmp(inppblk.descr[i].name, "rcvHWM")) { + inst->rcvHWM = (int) pvals[i].val.d.n; + } else if(!strcmp(inppblk.descr[i].name, "subscribe")) { + CHKiRet(parseSubscriptions(es_str2cstr(pvals[i].val.d.estr, NULL), + &inst->subscriptions)); + } else if(!strcmp(inppblk.descr[i].name, "identity")){ + inst->identity = es_str2cstr(pvals[i].val.d.estr, NULL); + } else if(!strcmp(inppblk.descr[i].name, "sndBuf")) { + inst->sndBuf = (int) pvals[i].val.d.n; + } else if(!strcmp(inppblk.descr[i].name, "rcvBuf")) { + inst->rcvBuf = (int) pvals[i].val.d.n; + } else if(!strcmp(inppblk.descr[i].name, "linger")) { + inst->linger = (int) pvals[i].val.d.n; + } else if(!strcmp(inppblk.descr[i].name, "backlog")) { + inst->backlog = (int) pvals[i].val.d.n; + } else if(!strcmp(inppblk.descr[i].name, "sndTimeout")) { + inst->sndTimeout = (int) pvals[i].val.d.n; + } else if(!strcmp(inppblk.descr[i].name, "rcvTimeout")) { + inst->rcvTimeout = (int) pvals[i].val.d.n; + } else if(!strcmp(inppblk.descr[i].name, "maxMsgSize")) { + inst->maxMsgSize = (int) pvals[i].val.d.n; + } else if(!strcmp(inppblk.descr[i].name, "rate")) { + inst->rate = (int) pvals[i].val.d.n; + } else if(!strcmp(inppblk.descr[i].name, "recoveryIVL")) { + inst->recoveryIVL = (int) pvals[i].val.d.n; + } else if(!strcmp(inppblk.descr[i].name, "multicastHops")) { + inst->multicastHops = (int) pvals[i].val.d.n; + } else if(!strcmp(inppblk.descr[i].name, "reconnectIVL")) { + inst->reconnectIVL = (int) pvals[i].val.d.n; + } else if(!strcmp(inppblk.descr[i].name, "reconnectIVLMax")) { + inst->reconnectIVLMax = (int) pvals[i].val.d.n; + } else if(!strcmp(inppblk.descr[i].name, "ipv4Only")) { + inst->ipv4Only = (int) pvals[i].val.d.n; + } else if(!strcmp(inppblk.descr[i].name, "affinity")) { + inst->affinity = (int) pvals[i].val.d.n; + } else { + errmsg.LogError(0, NO_ERRCODE, "imzmq3: program error, non-handled " + "param '%s'\n", inppblk.descr[i].name); + } - /* validate it */ - CHKiRet(validateConfig(sockInfo)); + } +finalize_it: + RETiRet; +} + +static rsRetVal addListener(instanceConf_t* inst){ + /* create the socket */ + void* sock; + struct lstn_s* newcnfinfo; + DEFiRet; - /* bind to the current ruleset (if any)*/ - sockInfo->ruleset = s_ruleset; + CHKiRet(createSocket(inst, &sock)); + + /* now create new lstn_s struct */ + CHKmalloc(newcnfinfo=(struct lstn_s*)MALLOC(sizeof(struct lstn_s))); + newcnfinfo->next = NULL; + newcnfinfo->sock = sock; + newcnfinfo->pRuleset = inst->pBindRuleset; + /* add this struct to the global */ + if(lcnfRoot == NULL) { + lcnfRoot = newcnfinfo; + } + if(lcnfLast == NULL) { + lcnfLast = newcnfinfo; + } else { + lcnfLast->next = newcnfinfo; + lcnfLast = newcnfinfo; + } + finalize_it: - free(valp); /* in any case, this is no longer needed */ - RETiRet; + RETiRet; } - static int handlePoll(zloop_t __attribute__((unused)) * loop, zmq_pollitem_t *poller, void* pd) { - msg_t* logmsg; + msg_t* pMsg; poller_data* pollerData = (poller_data*)pd; char* buf = zstr_recv(poller->socket); - if (msgConstruct(&logmsg) == RS_RET_OK) { - MsgSetRawMsg(logmsg, buf, strlen(buf)); - MsgSetInputName(logmsg, s_namep); - MsgSetFlowControlType(logmsg, eFLOWCTL_NO_DELAY); - MsgSetRuleset(logmsg, pollerData->ruleset); - logmsg->msgFlags = NEEDS_PARSING; - submitMsg(logmsg); + if (msgConstruct(&pMsg) == RS_RET_OK) { + MsgSetRawMsg(pMsg, buf, strlen(buf)); + MsgSetInputName(pMsg, s_namep); + MsgSetHOSTNAME(pMsg, glbl.GetLocalHostName(), ustrlen(glbl.GetLocalHostName())); + MsgSetRcvFrom(pMsg, glbl.GetLocalHostNameProp()); + MsgSetRcvFromIP(pMsg, glbl.GetLocalHostIP()); + MsgSetMSGoffs(pMsg, 0); + MsgSetFlowControlType(pMsg, eFLOWCTL_NO_DELAY); + MsgSetRuleset(pMsg, pollerData->ruleset); + pMsg->msgFlags = NEEDS_PARSING | PARSE_HOSTNAME; + submitMsg2(pMsg); } /* gotta free the string returned from zstr_recv() */ @@ -510,51 +579,65 @@ static int handlePoll(zloop_t __attribute__((unused)) * loop, zmq_pollitem_t *po /* called when runInput is called by rsyslog */ static rsRetVal rcv_loop(thrdInfo_t* pThrd){ + size_t n_items = 0; size_t i; int rv; - zmq_pollitem_t* items; - poller_data* pollerData; - + zmq_pollitem_t* items = NULL; + poller_data* pollerData = NULL; + struct lstn_s* current; + instanceConf_t* inst; DEFiRet; - - /* create the context*/ - CHKiRet(createContext()); + /* now add listeners. This actually creates the sockets, etc... */ + for (inst = runModConf->root; inst != NULL; inst=inst->next) { + addListener(inst); + } + if (lcnfRoot == NULL) { + errmsg.LogError(0, NO_ERRCODE, "imzmq3: no listeners were " + "started, input not activated.\n"); + ABORT_FINALIZE(RS_RET_NO_RUN); + } + + /* count the # of items first */ + for(current=lcnfRoot;current!=NULL;current=current->next) + n_items++; + + /* make arrays of pollitems, pollerdata so they are easy to delete later */ + /* create the poll items*/ - CHKmalloc(items = (zmq_pollitem_t*)MALLOC(sizeof(zmq_pollitem_t)*s_nitems)); + CHKmalloc(items = (zmq_pollitem_t*)MALLOC(sizeof(zmq_pollitem_t)*n_items)); /* create poller data (stuff to pass into the zmq closure called when we get a message)*/ - CHKmalloc(pollerData = (poller_data*)MALLOC(sizeof(poller_data)*s_nitems)); + CHKmalloc(pollerData = (poller_data*)MALLOC(sizeof(poller_data)*n_items)); /* loop through and initialize the poll items and poller_data arrays...*/ - for(i=0; i<s_nitems;++i) { + for(i=0, current = lcnfRoot; current != NULL; current = current->next, i++) { /* create the socket, update items.*/ - createSocket(&s_socketInfo[i], &items[i].socket); + items[i].socket=current->sock; items[i].events = ZMQ_POLLIN; /* now update the poller_data for this item */ pollerData[i].thread = pThrd; - pollerData[i].ruleset = s_socketInfo[i].ruleset; + pollerData[i].ruleset = current->pRuleset; } - + s_zloop = zloop_new(); - for(i=0; i<s_nitems; ++i) { + for(i=0; i<n_items; ++i) { rv = zloop_poller(s_zloop, &items[i], handlePoll, &pollerData[i]); if (rv) { - errmsg.LogError(0, NO_ERRCODE, "imzmq3: zloop_poller failed for item %zu", i); + errmsg.LogError(0, NO_ERRCODE, "imzmq3: zloop_poller failed for item %zu: %s", i, zmq_strerror(errno)); } } + DBGPRINTF("imzmq3: zloop_poller starting..."); zloop_start(s_zloop); zloop_destroy(&s_zloop); - finalize_it: - for(i=0; i< s_nitems; ++i) { - zsocket_destroy(s_context, items[i].socket); - } - + DBGPRINTF("imzmq3: zloop_poller stopped."); +finalize_it: zctx_destroy(&s_context); free(items); + free(pollerData); RETiRet; } @@ -564,7 +647,8 @@ static rsRetVal rcv_loop(thrdInfo_t* pThrd){ BEGINrunInput CODESTARTrunInput - iRet = rcv_loop(pThrd); + CHKiRet(rcv_loop(pThrd)); +finalize_it: RETiRet; ENDrunInput @@ -572,17 +656,13 @@ ENDrunInput /* initialize and return if will run or not */ BEGINwillRun CODESTARTwillRun - /* we need to create the inputName property (only once during our + /* we need to create the inputName property (only once during our lifetime) */ - CHKiRet(prop.Construct(&s_namep)); - CHKiRet(prop.SetString(s_namep, + CHKiRet(prop.Construct(&s_namep)); + CHKiRet(prop.SetString(s_namep, UCHAR_CONSTANT("imzmq3"), sizeof("imzmq3") - 1)); - CHKiRet(prop.ConstructFinalize(s_namep)); - -/* If there are no endpoints this is pointless ...*/ - if (s_nitems == 0) - ABORT_FINALIZE(RS_RET_NO_RUN); + CHKiRet(prop.ConstructFinalize(s_namep)); finalize_it: ENDwillRun @@ -590,70 +670,207 @@ ENDwillRun BEGINafterRun CODESTARTafterRun - /* do cleanup here */ - if(s_namep != NULL) - prop.Destruct(&s_namep); + /* do cleanup here */ + if (s_namep != NULL) + prop.Destruct(&s_namep); ENDafterRun BEGINmodExit CODESTARTmodExit - /* release what we no longer need */ - objRelease(errmsg, CORE_COMPONENT); - objRelease(glbl, CORE_COMPONENT); - objRelease(prop, CORE_COMPONENT); + /* release what we no longer need */ + objRelease(errmsg, CORE_COMPONENT); + objRelease(glbl, CORE_COMPONENT); + objRelease(prop, CORE_COMPONENT); objRelease(ruleset, CORE_COMPONENT); ENDmodExit BEGINisCompatibleWithFeature CODESTARTisCompatibleWithFeature - if(eFeat == sFEATURENonCancelInputTermination) - iRet = RS_RET_OK; + if (eFeat == sFEATURENonCancelInputTermination) + iRet = RS_RET_OK; ENDisCompatibleWithFeature +BEGINbeginCnfLoad +CODESTARTbeginCnfLoad + /* After endCnfLoad() (BEGINendCnfLoad...ENDendCnfLoad) is called, + * the pModConf pointer must not be used to change the in-memory + * config object. It's safe to use the same pointer for accessing + * the config object until freeCnf() (BEGINfreeCnf...ENDfreeCnf). */ + runModConf = pModConf; + runModConf->pConf = pConf; + /* init module config */ + runModConf->io_threads = 0; /* 0 means don't set it */ +ENDbeginCnfLoad + + +BEGINsetModCnf + struct cnfparamvals* pvals = NULL; + int i; +CODESTARTsetModCnf + pvals = nvlstGetParams(lst, &modpblk, NULL); + if (NULL == pvals) { + errmsg.LogError(0, RS_RET_MISSING_CNFPARAMS, "imzmq3: error processing module " + " config parameters ['module(...)']"); + ABORT_FINALIZE(RS_RET_MISSING_CNFPARAMS); + } + + for (i=0; i < modpblk.nParams; ++i) { + if (!pvals[i].bUsed) + continue; + if (!strcmp(modpblk.descr[i].name, "ioThreads")) { + runModConf->io_threads = (int)pvals[i].val.d.n; + } else { + errmsg.LogError(0, RS_RET_INVALID_PARAMS, + "imzmq3: config error, unknown " + "param %s in setModCnf\n", + modpblk.descr[i].name); + } + } + +finalize_it: + if (pvals != NULL) + cnfparamvalsDestruct(pvals, &modpblk); +ENDsetModCnf + + +BEGINendCnfLoad +CODESTARTendCnfLoad + /* Last chance to make changes to the in-memory config object for this + * input module. After this call, the config object must no longer be + * changed. */ + if (pModConf != runModConf) { + errmsg.LogError(0, NO_ERRCODE, "imzmq3: pointer of in-memory config object has " + "changed - pModConf=%p, runModConf=%p", pModConf, runModConf); + } + assert(pModConf == runModConf); +ENDendCnfLoad + + +/* function to generate error message if framework does not find requested ruleset */ +static inline void +std_checkRuleset_genErrMsg(__attribute__((unused)) modConfData_t *modConf, instanceConf_t *inst) +{ + errmsg.LogError(0, NO_ERRCODE, "imzmq3: ruleset '%s' for socket %s not found - " + "using default ruleset instead", inst->pszBindRuleset, + inst->description); +} + + +BEGINcheckCnf +instanceConf_t* inst; +CODESTARTcheckCnf + for(inst = pModConf->root; inst!=NULL; inst=inst->next) { + std_checkRuleset(pModConf, inst); + /* now, validate the instanceConf */ + CHKiRet(validateConfig(inst)); + } +finalize_it: + RETiRet; +ENDcheckCnf + + +BEGINactivateCnfPrePrivDrop +CODESTARTactivateCnfPrePrivDrop + if (pModConf != runModConf) { + errmsg.LogError(0, NO_ERRCODE, "imzmq3: pointer of in-memory config object has " + "changed - pModConf=%p, runModConf=%p", pModConf, runModConf); + } + assert(pModConf == runModConf); + + /* first create the context */ + createContext(); + + /* could setup context here, and set the global worker threads + and so on... */ +ENDactivateCnfPrePrivDrop + + +BEGINactivateCnf +CODESTARTactivateCnf + if (pModConf != runModConf) { + errmsg.LogError(0, NO_ERRCODE, "imzmq3: pointer of in-memory config object has " + "changed - pModConf=%p, runModConf=%p", pModConf, runModConf); + } + assert(pModConf == runModConf); +ENDactivateCnf + + +BEGINfreeCnf + struct lstn_s *lstn, *lstn_r; + instanceConf_t *inst, *inst_r; + sublist *sub, *sub_r; +CODESTARTfreeCnf + DBGPRINTF("imzmq3: BEGINfreeCnf ...\n"); + if (pModConf != runModConf) { + errmsg.LogError(0, NO_ERRCODE, "imzmq3: pointer of in-memory config object has " + "changed - pModConf=%p, runModConf=%p", pModConf, runModConf); + } + for (lstn = lcnfRoot; lstn != NULL; ) { + lstn_r = lstn; + lstn = lstn_r->next; + free(lstn_r); + } + for (inst = pModConf->root ; inst != NULL ; ) { + for (sub = inst->subscriptions; sub != NULL; ) { + free(sub->subscribe); + sub_r = sub; + sub = sub_r->next; + free(sub_r); + } + free(inst->pszBindRuleset); + inst_r = inst; + inst = inst->next; + free(inst_r); + } +ENDfreeCnf + + +BEGINnewInpInst + struct cnfparamvals* pvals; +CODESTARTnewInpInst + + DBGPRINTF("newInpInst (imzmq3)\n"); + pvals = nvlstGetParams(lst, &inppblk, NULL); + if(NULL==pvals) { + errmsg.LogError(0, RS_RET_MISSING_CNFPARAMS, + "imzmq3: required parameters are missing\n"); + ABORT_FINALIZE(RS_RET_MISSING_CNFPARAMS); + } + DBGPRINTF("imzmq3: input param blk:\n"); + cnfparamsPrint(&inppblk, pvals); + + /* now, parse the config params and so on... */ + CHKiRet(createListener(pvals)); + +finalize_it: +CODE_STD_FINALIZERnewInpInst + cnfparamvalsDestruct(pvals, &inppblk); +ENDnewInpInst + + BEGINqueryEtryPt CODESTARTqueryEtryPt CODEqueryEtryPt_STD_IMOD_QUERIES +CODEqueryEtryPt_STD_CONF2_QUERIES +CODEqueryEtryPt_STD_CONF2_setModCnf_QUERIES +CODEqueryEtryPt_STD_CONF2_PREPRIVDROP_QUERIES +CODEqueryEtryPt_STD_CONF2_IMOD_QUERIES CODEqueryEtryPt_IsCompatibleWithFeature_IF_OMOD_QUERIES ENDqueryEtryPt -static rsRetVal resetConfigVariables(uchar __attribute__((unused)) *pp, - void __attribute__((unused)) *pVal) { - return RS_RET_OK; -} -static rsRetVal setGlobalWorkerThreads(uchar __attribute__((unused)) *pp, int val) { - errmsg.LogError(0, NO_ERRCODE, "setGlobalWorkerThreads called with %d",val); - s_io_threads = val; - return RS_RET_OK; -} BEGINmodInit() CODESTARTmodInit /* we only support the current interface specification */ - *ipIFVersProvided = CURR_MOD_IF_VERSION; + *ipIFVersProvided = CURR_MOD_IF_VERSION; CODEmodInit_QueryRegCFSLineHdlr - CHKiRet(objUse(errmsg, CORE_COMPONENT)); - CHKiRet(objUse(glbl, CORE_COMPONENT)); - CHKiRet(objUse(prop, CORE_COMPONENT)); + CHKiRet(objUse(errmsg, CORE_COMPONENT)); + CHKiRet(objUse(glbl, CORE_COMPONENT)); + CHKiRet(objUse(prop, CORE_COMPONENT)); CHKiRet(objUse(ruleset, CORE_COMPONENT)); - - /* register config file handlers */ - CHKiRet(omsdRegCFSLineHdlr((uchar *)"inputzmq3serverbindruleset", - 0, eCmdHdlrGetWord, - set_ruleset, NULL, - STD_LOADABLE_MODULE_ID)); - CHKiRet(omsdRegCFSLineHdlr((uchar *)"inputzmq3serverrun", - 0, eCmdHdlrGetWord, - add_endpoint, NULL, - STD_LOADABLE_MODULE_ID)); - CHKiRet(omsdRegCFSLineHdlr((uchar *)"resetconfigvariables", - 1, eCmdHdlrCustomHandler, - resetConfigVariables, NULL, - STD_LOADABLE_MODULE_ID)); - CHKiRet(omsdRegCFSLineHdlr((uchar *)"inputzmq3globalWorkerThreads", - 1, eCmdHdlrInt, - setGlobalWorkerThreads, NULL, - STD_LOADABLE_MODULE_ID)); ENDmodInit + + diff --git a/plugins/mmanon/Makefile.in b/plugins/mmanon/Makefile.in index f39134b..81afe4c 100644 --- a/plugins/mmanon/Makefile.in +++ b/plugins/mmanon/Makefile.in @@ -153,7 +153,6 @@ GREP = @GREP@ GSS_LIBS = @GSS_LIBS@ GUARDTIME_CFLAGS = @GUARDTIME_CFLAGS@ GUARDTIME_LIBS = @GUARDTIME_LIBS@ -HAVE_LIBGCRYPT_CONFIG = @HAVE_LIBGCRYPT_CONFIG@ HAVE_MYSQL_CONFIG = @HAVE_MYSQL_CONFIG@ HAVE_ORACLE_CONFIG = @HAVE_ORACLE_CONFIG@ HAVE_PGSQL_CONFIG = @HAVE_PGSQL_CONFIG@ @@ -174,14 +173,15 @@ LEXLIB = @LEXLIB@ LEX_OUTPUT_ROOT = @LEX_OUTPUT_ROOT@ LIBDBI_CFLAGS = @LIBDBI_CFLAGS@ LIBDBI_LIBS = @LIBDBI_LIBS@ -LIBEE_CFLAGS = @LIBEE_CFLAGS@ -LIBEE_LIBS = @LIBEE_LIBS@ LIBESTR_CFLAGS = @LIBESTR_CFLAGS@ LIBESTR_LIBS = @LIBESTR_LIBS@ LIBGCRYPT_CFLAGS = @LIBGCRYPT_CFLAGS@ +LIBGCRYPT_CONFIG = @LIBGCRYPT_CONFIG@ LIBGCRYPT_LIBS = @LIBGCRYPT_LIBS@ LIBLOGGING_CFLAGS = @LIBLOGGING_CFLAGS@ LIBLOGGING_LIBS = @LIBLOGGING_LIBS@ +LIBLOGGING_STDLOG_CFLAGS = @LIBLOGGING_STDLOG_CFLAGS@ +LIBLOGGING_STDLOG_LIBS = @LIBLOGGING_STDLOG_LIBS@ LIBLOGNORM_CFLAGS = @LIBLOGNORM_CFLAGS@ LIBLOGNORM_LIBS = @LIBLOGNORM_LIBS@ LIBM = @LIBM@ @@ -206,6 +206,8 @@ NM = @NM@ NMEDIT = @NMEDIT@ OBJDUMP = @OBJDUMP@ OBJEXT = @OBJEXT@ +OPENSSL_CFLAGS = @OPENSSL_CFLAGS@ +OPENSSL_LIBS = @OPENSSL_LIBS@ ORACLE_CFLAGS = @ORACLE_CFLAGS@ ORACLE_LIBS = @ORACLE_LIBS@ OTOOL = @OTOOL@ @@ -387,22 +389,25 @@ distclean-compile: @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/mmanon_la-mmanon.Plo@am__quote@ .c.o: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(COMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ $< -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Po +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.o$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c $< +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ $< .c.obj: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(COMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ `$(CYGPATH_W) '$<'` -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Po +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.obj$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ `$(CYGPATH_W) '$<'` &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c `$(CYGPATH_W) '$<'` +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ `$(CYGPATH_W) '$<'` .c.lo: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(LTCOMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ $< -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Plo +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.lo$$||'`;\ +@am__fastdepCC_TRUE@ $(LTCOMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Plo @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=yes @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ @am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(LTCOMPILE) -c -o $@ $< diff --git a/plugins/mmanon/mmanon.c b/plugins/mmanon/mmanon.c index a1c99d0..a85faa1 100644 --- a/plugins/mmanon/mmanon.c +++ b/plugins/mmanon/mmanon.c @@ -71,6 +71,10 @@ typedef struct _instanceData { } ipv4; } instanceData; +typedef struct wrkrInstanceData { + instanceData *pData; +} wrkrInstanceData_t; + struct modConfData_s { rsconf_t *pConf; /* our overall config object */ }; @@ -119,6 +123,10 @@ BEGINcreateInstance CODESTARTcreateInstance ENDcreateInstance +BEGINcreateWrkrInstance +CODESTARTcreateWrkrInstance +ENDcreateWrkrInstance + BEGINisCompatibleWithFeature CODESTARTisCompatibleWithFeature @@ -130,6 +138,11 @@ CODESTARTfreeInstance ENDfreeInstance +BEGINfreeWrkrInstance +CODESTARTfreeWrkrInstance +ENDfreeWrkrInstance + + static inline void setInstParamDefaults(instanceData *pData) { @@ -170,7 +183,6 @@ CODESTARTnewActInst cstr); free(cstr); } - pData->replChar = es_getBufAddr(pvals[i].val.d.estr)[0]; } else if(!strcmp(actpblk.descr[i].name, "replacementchar")) { pData->replChar = es_getBufAddr(pvals[i].val.d.estr)[0]; } else if(!strcmp(actpblk.descr[i].name, "ipv4.bits")) { @@ -283,7 +295,7 @@ anonip(instanceData *pData, uchar *msg, int *pLenMsg, int *idx) int endpos; int lenMsg = *pLenMsg; - while(i < lenMsg && (msg[i] <= '0' || msg[i] >= '9')) { + while(i < lenMsg && (msg[i] <= '0' || msg[i] > '9')) { ++i; /* skip to first number */ } if(i >= lenMsg) @@ -307,7 +319,7 @@ anonip(instanceData *pData, uchar *msg, int *pLenMsg, int *idx) ++i; ipstart[3] = i; octet = getnum(msg, lenMsg, &i); - if(octet > 255 || !(msg[i] == ' ' || msg[i] == ':')) goto done; + if(octet > 255) goto done; ipv4addr |= octet; /* OK, we now found an ip address */ @@ -339,6 +351,8 @@ anonip(instanceData *pData, uchar *msg, int *pLenMsg, int *idx) if(i - endpos > 0) { *pLenMsg = lenMsg - (i - endpos); memmove(msg+endpos, msg+i, lenMsg - i + 1); + /* correct index for next search! */ + i -= (i - endpos); } } @@ -357,7 +371,7 @@ CODESTARTdoAction lenMsg = getMSGLen(pMsg); msg = getMSG(pMsg); for(i = 0 ; i < lenMsg ; ++i) { - anonip(pData, msg, &lenMsg, &i); + anonip(pWrkrData->pData, msg, &lenMsg, &i); } if(lenMsg != getMSGLen(pMsg)) setMSGLen(pMsg, lenMsg); @@ -386,6 +400,7 @@ ENDmodExit BEGINqueryEtryPt CODESTARTqueryEtryPt CODEqueryEtryPt_STD_OMOD_QUERIES +CODEqueryEtryPt_STD_OMOD8_QUERIES CODEqueryEtryPt_STD_CONF2_OMOD_QUERIES CODEqueryEtryPt_STD_CONF2_QUERIES ENDqueryEtryPt diff --git a/plugins/mmaudit/Makefile.am b/plugins/mmaudit/Makefile.am index c64d082..77b2e85 100644 --- a/plugins/mmaudit/Makefile.am +++ b/plugins/mmaudit/Makefile.am @@ -1,8 +1,8 @@ pkglib_LTLIBRARIES = mmaudit.la mmaudit_la_SOURCES = mmaudit.c -mmaudit_la_CPPFLAGS = $(RSRT_CFLAGS) $(PTHREADS_CFLAGS) $(LIBLOGNORM_CFLAGS) $(LIBEE_CFLAGS) -mmaudit_la_LDFLAGS = -module -avoid-version $(LIBLOGNORM_LIBS) $(LIBEE_LIBS) +mmaudit_la_CPPFLAGS = $(RSRT_CFLAGS) $(PTHREADS_CFLAGS) +mmaudit_la_LDFLAGS = -module -avoid-version mmaudit_la_LIBADD = EXTRA_DIST = diff --git a/plugins/mmaudit/Makefile.in b/plugins/mmaudit/Makefile.in index fd8e8fb..20e2529 100644 --- a/plugins/mmaudit/Makefile.in +++ b/plugins/mmaudit/Makefile.in @@ -153,7 +153,6 @@ GREP = @GREP@ GSS_LIBS = @GSS_LIBS@ GUARDTIME_CFLAGS = @GUARDTIME_CFLAGS@ GUARDTIME_LIBS = @GUARDTIME_LIBS@ -HAVE_LIBGCRYPT_CONFIG = @HAVE_LIBGCRYPT_CONFIG@ HAVE_MYSQL_CONFIG = @HAVE_MYSQL_CONFIG@ HAVE_ORACLE_CONFIG = @HAVE_ORACLE_CONFIG@ HAVE_PGSQL_CONFIG = @HAVE_PGSQL_CONFIG@ @@ -174,14 +173,15 @@ LEXLIB = @LEXLIB@ LEX_OUTPUT_ROOT = @LEX_OUTPUT_ROOT@ LIBDBI_CFLAGS = @LIBDBI_CFLAGS@ LIBDBI_LIBS = @LIBDBI_LIBS@ -LIBEE_CFLAGS = @LIBEE_CFLAGS@ -LIBEE_LIBS = @LIBEE_LIBS@ LIBESTR_CFLAGS = @LIBESTR_CFLAGS@ LIBESTR_LIBS = @LIBESTR_LIBS@ LIBGCRYPT_CFLAGS = @LIBGCRYPT_CFLAGS@ +LIBGCRYPT_CONFIG = @LIBGCRYPT_CONFIG@ LIBGCRYPT_LIBS = @LIBGCRYPT_LIBS@ LIBLOGGING_CFLAGS = @LIBLOGGING_CFLAGS@ LIBLOGGING_LIBS = @LIBLOGGING_LIBS@ +LIBLOGGING_STDLOG_CFLAGS = @LIBLOGGING_STDLOG_CFLAGS@ +LIBLOGGING_STDLOG_LIBS = @LIBLOGGING_STDLOG_LIBS@ LIBLOGNORM_CFLAGS = @LIBLOGNORM_CFLAGS@ LIBLOGNORM_LIBS = @LIBLOGNORM_LIBS@ LIBM = @LIBM@ @@ -206,6 +206,8 @@ NM = @NM@ NMEDIT = @NMEDIT@ OBJDUMP = @OBJDUMP@ OBJEXT = @OBJEXT@ +OPENSSL_CFLAGS = @OPENSSL_CFLAGS@ +OPENSSL_LIBS = @OPENSSL_LIBS@ ORACLE_CFLAGS = @ORACLE_CFLAGS@ ORACLE_LIBS = @ORACLE_LIBS@ OTOOL = @OTOOL@ @@ -306,8 +308,8 @@ top_builddir = @top_builddir@ top_srcdir = @top_srcdir@ pkglib_LTLIBRARIES = mmaudit.la mmaudit_la_SOURCES = mmaudit.c -mmaudit_la_CPPFLAGS = $(RSRT_CFLAGS) $(PTHREADS_CFLAGS) $(LIBLOGNORM_CFLAGS) $(LIBEE_CFLAGS) -mmaudit_la_LDFLAGS = -module -avoid-version $(LIBLOGNORM_LIBS) $(LIBEE_LIBS) +mmaudit_la_CPPFLAGS = $(RSRT_CFLAGS) $(PTHREADS_CFLAGS) +mmaudit_la_LDFLAGS = -module -avoid-version mmaudit_la_LIBADD = EXTRA_DIST = all: all-am @@ -387,22 +389,25 @@ distclean-compile: @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/mmaudit_la-mmaudit.Plo@am__quote@ .c.o: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(COMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ $< -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Po +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.o$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c $< +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ $< .c.obj: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(COMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ `$(CYGPATH_W) '$<'` -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Po +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.obj$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ `$(CYGPATH_W) '$<'` &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c `$(CYGPATH_W) '$<'` +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ `$(CYGPATH_W) '$<'` .c.lo: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(LTCOMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ $< -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Plo +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.lo$$||'`;\ +@am__fastdepCC_TRUE@ $(LTCOMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Plo @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=yes @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ @am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(LTCOMPILE) -c -o $@ $< diff --git a/plugins/mmaudit/mmaudit.c b/plugins/mmaudit/mmaudit.c index 018e177..75f8dd4 100644 --- a/plugins/mmaudit/mmaudit.c +++ b/plugins/mmaudit/mmaudit.c @@ -14,7 +14,7 @@ * * File begun on 2012-02-23 by RGerhards * - * Copyright 2012 Adiscon GmbH. + * Copyright 2013 Adiscon GmbH. * * This file is part of rsyslog. * @@ -43,8 +43,7 @@ #include <errno.h> #include <unistd.h> #include <ctype.h> -#include <libestr.h> -#include <libee/libee.h> +#include <json.h> #include "conf.h" #include "syslogd-types.h" #include "template.h" @@ -70,6 +69,11 @@ typedef struct _instanceData { int dummy; /* remove when the first real parameter is needed */ } instanceData; +typedef struct wrkrInstanceData { + instanceData *pData; +} wrkrInstanceData_t; + + BEGINinitConfVars /* (re)set config variables to default values */ CODESTARTinitConfVars resetConfigVariables(NULL, NULL); @@ -80,6 +84,10 @@ BEGINcreateInstance CODESTARTcreateInstance ENDcreateInstance +BEGINcreateWrkrInstance +CODESTARTcreateWrkrInstance +ENDcreateWrkrInstance + BEGINisCompatibleWithFeature CODESTARTisCompatibleWithFeature @@ -90,6 +98,10 @@ BEGINfreeInstance CODESTARTfreeInstance ENDfreeInstance +BEGINfreeWrkrInstance +CODESTARTfreeWrkrInstance +ENDfreeWrkrInstance + BEGINdbgPrintInstInfo CODESTARTdbgPrintInstInfo @@ -303,6 +315,7 @@ ENDmodExit BEGINqueryEtryPt CODESTARTqueryEtryPt CODEqueryEtryPt_STD_OMOD_QUERIES +CODEqueryEtryPt_STD_OMOD8_QUERIES ENDqueryEtryPt diff --git a/plugins/mmcount/Makefile.am b/plugins/mmcount/Makefile.am new file mode 100644 index 0000000..9c8c99d --- /dev/null +++ b/plugins/mmcount/Makefile.am @@ -0,0 +1,8 @@ +pkglib_LTLIBRARIES = mmcount.la + +mmcount_la_SOURCES = mmcount.c +mmcount_la_CPPFLAGS = $(RSRT_CFLAGS) $(PTHREADS_CFLAGS) +mmcount_la_LDFLAGS = -module -avoid-version +mmcount_la_LIBADD = + +EXTRA_DIST = diff --git a/doc/Makefile.in b/plugins/mmcount/Makefile.in index e83e568..ec47ec0 100644 --- a/doc/Makefile.in +++ b/plugins/mmcount/Makefile.in @@ -14,6 +14,7 @@ # PARTICULAR PURPOSE. @SET_MAKE@ + VPATH = @srcdir@ pkgdatadir = $(datadir)/@PACKAGE@ pkgincludedir = $(includedir)/@PACKAGE@ @@ -33,7 +34,7 @@ PRE_UNINSTALL = : POST_UNINSTALL = : build_triplet = @build@ host_triplet = @host@ -subdir = doc +subdir = plugins/mmcount DIST_COMMON = $(srcdir)/Makefile.am $(srcdir)/Makefile.in ACLOCAL_M4 = $(top_srcdir)/aclocal.m4 am__aclocal_m4_deps = $(top_srcdir)/m4/atomic_operations.m4 \ @@ -47,14 +48,74 @@ mkinstalldirs = $(install_sh) -d CONFIG_HEADER = $(top_builddir)/config.h CONFIG_CLEAN_FILES = CONFIG_CLEAN_VPATH_FILES = -AM_V_GEN = $(am__v_GEN_@AM_V@) -am__v_GEN_ = $(am__v_GEN_@AM_DEFAULT_V@) -am__v_GEN_0 = @echo " GEN " $@; +am__vpath_adj_setup = srcdirstrip=`echo "$(srcdir)" | sed 's|.|.|g'`; +am__vpath_adj = case $$p in \ + $(srcdir)/*) f=`echo "$$p" | sed "s|^$$srcdirstrip/||"`;; \ + *) f=$$p;; \ + esac; +am__strip_dir = f=`echo $$p | sed -e 's|^.*/||'`; +am__install_max = 40 +am__nobase_strip_setup = \ + srcdirstrip=`echo "$(srcdir)" | sed 's/[].[^$$\\*|]/\\\\&/g'` +am__nobase_strip = \ + for p in $$list; do echo "$$p"; done | sed -e "s|$$srcdirstrip/||" +am__nobase_list = $(am__nobase_strip_setup); \ + for p in $$list; do echo "$$p $$p"; done | \ + sed "s| $$srcdirstrip/| |;"' / .*\//!s/ .*/ ./; s,\( .*\)/[^/]*$$,\1,' | \ + $(AWK) 'BEGIN { files["."] = "" } { files[$$2] = files[$$2] " " $$1; \ + if (++n[$$2] == $(am__install_max)) \ + { print $$2, files[$$2]; n[$$2] = 0; files[$$2] = "" } } \ + END { for (dir in files) print dir, files[dir] }' +am__base_list = \ + sed '$$!N;$$!N;$$!N;$$!N;$$!N;$$!N;$$!N;s/\n/ /g' | \ + sed '$$!N;$$!N;$$!N;$$!N;s/\n/ /g' +am__uninstall_files_from_dir = { \ + test -z "$$files" \ + || { test ! -d "$$dir" && test ! -f "$$dir" && test ! -r "$$dir"; } \ + || { echo " ( cd '$$dir' && rm -f" $$files ")"; \ + $(am__cd) "$$dir" && rm -f $$files; }; \ + } +am__installdirs = "$(DESTDIR)$(pkglibdir)" +LTLIBRARIES = $(pkglib_LTLIBRARIES) +mmcount_la_DEPENDENCIES = +am_mmcount_la_OBJECTS = mmcount_la-mmcount.lo +mmcount_la_OBJECTS = $(am_mmcount_la_OBJECTS) +AM_V_lt = $(am__v_lt_@AM_V@) +am__v_lt_ = $(am__v_lt_@AM_DEFAULT_V@) +am__v_lt_0 = --silent +mmcount_la_LINK = $(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \ + $(LIBTOOLFLAGS) --mode=link $(CCLD) $(AM_CFLAGS) $(CFLAGS) \ + $(mmcount_la_LDFLAGS) $(LDFLAGS) -o $@ +DEFAULT_INCLUDES = -I.@am__isrc@ -I$(top_builddir) +depcomp = $(SHELL) $(top_srcdir)/depcomp +am__depfiles_maybe = depfiles +am__mv = mv -f +COMPILE = $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(AM_CPPFLAGS) \ + $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) +LTCOMPILE = $(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \ + $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) \ + $(DEFAULT_INCLUDES) $(INCLUDES) $(AM_CPPFLAGS) $(CPPFLAGS) \ + $(AM_CFLAGS) $(CFLAGS) +AM_V_CC = $(am__v_CC_@AM_V@) +am__v_CC_ = $(am__v_CC_@AM_DEFAULT_V@) +am__v_CC_0 = @echo " CC " $@; AM_V_at = $(am__v_at_@AM_V@) am__v_at_ = $(am__v_at_@AM_DEFAULT_V@) am__v_at_0 = @ -SOURCES = -DIST_SOURCES = +CCLD = $(CC) +LINK = $(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \ + $(LIBTOOLFLAGS) --mode=link $(CCLD) $(AM_CFLAGS) $(CFLAGS) \ + $(AM_LDFLAGS) $(LDFLAGS) -o $@ +AM_V_CCLD = $(am__v_CCLD_@AM_V@) +am__v_CCLD_ = $(am__v_CCLD_@AM_DEFAULT_V@) +am__v_CCLD_0 = @echo " CCLD " $@; +AM_V_GEN = $(am__v_GEN_@AM_V@) +am__v_GEN_ = $(am__v_GEN_@AM_DEFAULT_V@) +am__v_GEN_0 = @echo " GEN " $@; +SOURCES = $(mmcount_la_SOURCES) +DIST_SOURCES = $(mmcount_la_SOURCES) +ETAGS = etags +CTAGS = ctags DISTFILES = $(DIST_COMMON) $(DIST_SOURCES) $(TEXINFOS) $(EXTRA_DIST) ACLOCAL = @ACLOCAL@ AMTAR = @AMTAR@ @@ -92,7 +153,6 @@ GREP = @GREP@ GSS_LIBS = @GSS_LIBS@ GUARDTIME_CFLAGS = @GUARDTIME_CFLAGS@ GUARDTIME_LIBS = @GUARDTIME_LIBS@ -HAVE_LIBGCRYPT_CONFIG = @HAVE_LIBGCRYPT_CONFIG@ HAVE_MYSQL_CONFIG = @HAVE_MYSQL_CONFIG@ HAVE_ORACLE_CONFIG = @HAVE_ORACLE_CONFIG@ HAVE_PGSQL_CONFIG = @HAVE_PGSQL_CONFIG@ @@ -113,14 +173,15 @@ LEXLIB = @LEXLIB@ LEX_OUTPUT_ROOT = @LEX_OUTPUT_ROOT@ LIBDBI_CFLAGS = @LIBDBI_CFLAGS@ LIBDBI_LIBS = @LIBDBI_LIBS@ -LIBEE_CFLAGS = @LIBEE_CFLAGS@ -LIBEE_LIBS = @LIBEE_LIBS@ LIBESTR_CFLAGS = @LIBESTR_CFLAGS@ LIBESTR_LIBS = @LIBESTR_LIBS@ LIBGCRYPT_CFLAGS = @LIBGCRYPT_CFLAGS@ +LIBGCRYPT_CONFIG = @LIBGCRYPT_CONFIG@ LIBGCRYPT_LIBS = @LIBGCRYPT_LIBS@ LIBLOGGING_CFLAGS = @LIBLOGGING_CFLAGS@ LIBLOGGING_LIBS = @LIBLOGGING_LIBS@ +LIBLOGGING_STDLOG_CFLAGS = @LIBLOGGING_STDLOG_CFLAGS@ +LIBLOGGING_STDLOG_LIBS = @LIBLOGGING_STDLOG_LIBS@ LIBLOGNORM_CFLAGS = @LIBLOGNORM_CFLAGS@ LIBLOGNORM_LIBS = @LIBLOGNORM_LIBS@ LIBM = @LIBM@ @@ -145,6 +206,8 @@ NM = @NM@ NMEDIT = @NMEDIT@ OBJDUMP = @OBJDUMP@ OBJEXT = @OBJEXT@ +OPENSSL_CFLAGS = @OPENSSL_CFLAGS@ +OPENSSL_LIBS = @OPENSSL_LIBS@ ORACLE_CFLAGS = @ORACLE_CFLAGS@ ORACLE_LIBS = @ORACLE_LIBS@ OTOOL = @OTOOL@ @@ -243,184 +306,16 @@ target_alias = @target_alias@ top_build_prefix = @top_build_prefix@ top_builddir = @top_builddir@ top_srcdir = @top_srcdir@ -html_files = \ - index.html \ - bugs.html \ - debug.html \ - features.html \ - generic_design.html \ - expression.html \ - droppriv.html \ - history.html \ - how2help.html \ - install.html \ - build_from_repo.html \ - ipv6.html \ - log_rotation_fix_size.html \ - manual.html \ - modules.html \ - property_replacer.html \ - rsyslog_ng_comparison.html \ - rsyslog_conf.html \ - rsyslog-example.conf \ - rsyslog_mysql.html \ - rsyslog_pgsql.html \ - rsyslog_packages.html \ - rsyslog_high_database_rate.html \ - rsyslog_php_syslog_ng.html \ - rsyslog_recording_pri.html \ - rsyslog_tls.html \ - rsyslog_reliable_forwarding.html \ - rsyslog_stunnel.html \ - syslog_protocol.html \ - version_naming.html \ - contributors.html \ - dev_queue.html \ - ompipe.html \ - omfwd.html \ - omfile.html \ - omjournal.html \ - mmanon.html \ - omusrmsg.html \ - omstdout.html \ - omudpspoof.html \ - omruleset.html \ - omsnmp.html \ - sigprov_gt.html \ - ommysql.html \ - omoracle.html \ - omlibdbi.html \ - imfile.html \ - imtcp.html \ - imptcp.html \ - impstats.html \ - imgssapi.html \ - imrelp.html \ - imsolaris.html \ - imuxsock.html \ - imklog.html \ - pmlastmsg.html \ - mmsnmptrapd.html \ - queues.html \ - src/queueWorkerLogic.dia \ - queueWorkerLogic.jpg \ - queueWorkerLogic_small.jpg \ - tls_cert_100.jpg \ - tls_cert_ca.jpg \ - tls_cert.jpg \ - tls_cert_errmsgs.html \ - rsyslog_secure_tls.html \ - tls_cert_server.html \ - tls_cert_ca.html \ - tls_cert_summary.html \ - tls_cert_machine.html \ - tls_cert_udp_relay.html \ - tls_cert_client.html \ - tls_cert_scenario.html \ - rainerscript.html \ - lookup_tables.html \ - rscript_abnf.html \ - rsconf1_actionexeconlywhenpreviousissuspended.html \ - rsconf1_actionresumeinterval.html \ - rsconf1_allowedsender.html \ - rsconf1_controlcharacterescapeprefix.html \ - rsconf1_escape8bitcharsonreceive.html \ - rsconf1_debugprintcfsyslinehandlerlist.html \ - rsconf1_debugprintmodulelist.html \ - rsconf1_debugprinttemplatelist.html \ - rsconf1_dircreatemode.html \ - rsconf1_dirgroup.html \ - rsconf1_dirowner.html \ - rsconf1_dropmsgswithmaliciousdnsptrrecords.html \ - rsconf1_droptrailinglfonreception.html \ - rsconf1_dynafilecachesize.html \ - rsconf1_escapecontrolcharactersonreceive.html \ - rsconf1_failonchownfailure.html \ - rsconf1_filecreatemode.html \ - rsconf1_filegroup.html \ - rsconf1_fileowner.html \ - rsconf1_generateconfiggraph.html \ - rsconf1_gssforwardservicename.html \ - rsconf1_gsslistenservicename.html \ - rsconf1_gssmode.html \ - rsconf1_includeconfig.html \ - rsconf1_mainmsgqueuesize.html \ - rsconf1_markmessageperiod.html \ - rsconf1_modload.html \ - rsconf1_moddir.html \ - rsconf1_repeatedmsgreduction.html \ - rsconf1_resetconfigvariables.html \ - rsconf1_rulesetcreatemainqueue.html \ - rsconf1_umask.html \ - rsconf1_rulesetparser.html \ - v3compatibility.html \ - v4compatibility.html \ - v5compatibility.html \ - im3195.html \ - netstream.html \ - ns_gtls.html \ - ns_ptcp.html \ - src/tls_cert.dia \ - gssapi.html \ - licensing.html \ - mmnormalize.html \ - mmjsonparse.html \ - ommail.html \ - omuxsock.html \ - omrelp.html \ - syslog_parsing.html \ - troubleshoot.html \ - rsyslog_conf_actions.html \ - rsyslog_conf_filter.html \ - rsyslog_conf_global.html \ - rsyslog_conf_modules.html \ - rsyslog_conf_output.html \ - rsyslog_conf_templates.html \ - rsyslog_conf_nomatch.html \ - queues_analogy.html \ - multi_ruleset.html \ - multi_ruleset_legacy_format.html \ - dev_oplugins.html \ - free_support.html \ - imudp.html \ - messageparser.html \ - omhdfs.html \ - omprog.html \ - queue_msg_state.jpeg \ - rsconf1_abortonuncleanconfig.html \ - rsconf1_maxopenfiles.html \ - rsconf1_omfileforcechown.html \ - rsyslog_queue_pointers.jpeg \ - rsyslog_queue_pointers2.jpeg \ - v6compatibility.html \ - v7compatibility.html \ - rsyslog_conf_basic_structure.html \ - rsyslog_conf_sysklogd_compatibility.html \ - imkmsg.html \ - src/classes.dia - -grfx_files = \ - rsyslog_confgraph_complex.png\ - rsyslog_confgraph_std.png \ - module_workflow.png \ - direct_queue0.png \ - direct_queue1.png \ - direct_queue2.png \ - direct_queue3.png \ - direct_queue_rsyslog.png \ - direct_queue_rsyslog2.png \ - direct_queue_directq.png \ - dataflow.png \ - queue_analogy_tv.png \ - gssapi.png \ - rfc5424layers.png \ - src/rfc5424layers.dia \ - rsyslog-vers.png - -EXTRA_DIST = $(html_files) $(grfx_files) +pkglib_LTLIBRARIES = mmcount.la +mmcount_la_SOURCES = mmcount.c +mmcount_la_CPPFLAGS = $(RSRT_CFLAGS) $(PTHREADS_CFLAGS) +mmcount_la_LDFLAGS = -module -avoid-version +mmcount_la_LIBADD = +EXTRA_DIST = all: all-am .SUFFIXES: +.SUFFIXES: .c .lo .o .obj $(srcdir)/Makefile.in: $(srcdir)/Makefile.am $(am__configure_deps) @for dep in $?; do \ case '$(am__configure_deps)' in \ @@ -430,9 +325,9 @@ $(srcdir)/Makefile.in: $(srcdir)/Makefile.am $(am__configure_deps) exit 1;; \ esac; \ done; \ - echo ' cd $(top_srcdir) && $(AUTOMAKE) --gnu doc/Makefile'; \ + echo ' cd $(top_srcdir) && $(AUTOMAKE) --gnu plugins/mmcount/Makefile'; \ $(am__cd) $(top_srcdir) && \ - $(AUTOMAKE) --gnu doc/Makefile + $(AUTOMAKE) --gnu plugins/mmcount/Makefile .PRECIOUS: Makefile Makefile: $(srcdir)/Makefile.in $(top_builddir)/config.status @case '$?' in \ @@ -451,18 +346,136 @@ $(top_srcdir)/configure: $(am__configure_deps) $(ACLOCAL_M4): $(am__aclocal_m4_deps) cd $(top_builddir) && $(MAKE) $(AM_MAKEFLAGS) am--refresh $(am__aclocal_m4_deps): +install-pkglibLTLIBRARIES: $(pkglib_LTLIBRARIES) + @$(NORMAL_INSTALL) + test -z "$(pkglibdir)" || $(MKDIR_P) "$(DESTDIR)$(pkglibdir)" + @list='$(pkglib_LTLIBRARIES)'; test -n "$(pkglibdir)" || list=; \ + list2=; for p in $$list; do \ + if test -f $$p; then \ + list2="$$list2 $$p"; \ + else :; fi; \ + done; \ + test -z "$$list2" || { \ + echo " $(LIBTOOL) $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=install $(INSTALL) $(INSTALL_STRIP_FLAG) $$list2 '$(DESTDIR)$(pkglibdir)'"; \ + $(LIBTOOL) $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=install $(INSTALL) $(INSTALL_STRIP_FLAG) $$list2 "$(DESTDIR)$(pkglibdir)"; \ + } + +uninstall-pkglibLTLIBRARIES: + @$(NORMAL_UNINSTALL) + @list='$(pkglib_LTLIBRARIES)'; test -n "$(pkglibdir)" || list=; \ + for p in $$list; do \ + $(am__strip_dir) \ + echo " $(LIBTOOL) $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=uninstall rm -f '$(DESTDIR)$(pkglibdir)/$$f'"; \ + $(LIBTOOL) $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=uninstall rm -f "$(DESTDIR)$(pkglibdir)/$$f"; \ + done + +clean-pkglibLTLIBRARIES: + -test -z "$(pkglib_LTLIBRARIES)" || rm -f $(pkglib_LTLIBRARIES) + @list='$(pkglib_LTLIBRARIES)'; for p in $$list; do \ + dir="`echo $$p | sed -e 's|/[^/]*$$||'`"; \ + test "$$dir" != "$$p" || dir=.; \ + echo "rm -f \"$${dir}/so_locations\""; \ + rm -f "$${dir}/so_locations"; \ + done +mmcount.la: $(mmcount_la_OBJECTS) $(mmcount_la_DEPENDENCIES) $(EXTRA_mmcount_la_DEPENDENCIES) + $(AM_V_CCLD)$(mmcount_la_LINK) -rpath $(pkglibdir) $(mmcount_la_OBJECTS) $(mmcount_la_LIBADD) $(LIBS) + +mostlyclean-compile: + -rm -f *.$(OBJEXT) + +distclean-compile: + -rm -f *.tab.c + +@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/mmcount_la-mmcount.Plo@am__quote@ + +.c.o: +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.o$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ $< + +.c.obj: +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.obj$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ `$(CYGPATH_W) '$<'` &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ `$(CYGPATH_W) '$<'` + +.c.lo: +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.lo$$||'`;\ +@am__fastdepCC_TRUE@ $(LTCOMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Plo +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=yes @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(LTCOMPILE) -c -o $@ $< + +mmcount_la-mmcount.lo: mmcount.c +@am__fastdepCC_TRUE@ $(AM_V_CC)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(mmcount_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT mmcount_la-mmcount.lo -MD -MP -MF $(DEPDIR)/mmcount_la-mmcount.Tpo -c -o mmcount_la-mmcount.lo `test -f 'mmcount.c' || echo '$(srcdir)/'`mmcount.c +@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/mmcount_la-mmcount.Tpo $(DEPDIR)/mmcount_la-mmcount.Plo +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='mmcount.c' object='mmcount_la-mmcount.lo' libtool=yes @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(mmcount_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o mmcount_la-mmcount.lo `test -f 'mmcount.c' || echo '$(srcdir)/'`mmcount.c mostlyclean-libtool: -rm -f *.lo clean-libtool: -rm -rf .libs _libs + +ID: $(HEADERS) $(SOURCES) $(LISP) $(TAGS_FILES) + list='$(SOURCES) $(HEADERS) $(LISP) $(TAGS_FILES)'; \ + unique=`for i in $$list; do \ + if test -f "$$i"; then echo $$i; else echo $(srcdir)/$$i; fi; \ + done | \ + $(AWK) '{ files[$$0] = 1; nonempty = 1; } \ + END { if (nonempty) { for (i in files) print i; }; }'`; \ + mkid -fID $$unique tags: TAGS -TAGS: +TAGS: $(HEADERS) $(SOURCES) $(TAGS_DEPENDENCIES) \ + $(TAGS_FILES) $(LISP) + set x; \ + here=`pwd`; \ + list='$(SOURCES) $(HEADERS) $(LISP) $(TAGS_FILES)'; \ + unique=`for i in $$list; do \ + if test -f "$$i"; then echo $$i; else echo $(srcdir)/$$i; fi; \ + done | \ + $(AWK) '{ files[$$0] = 1; nonempty = 1; } \ + END { if (nonempty) { for (i in files) print i; }; }'`; \ + shift; \ + if test -z "$(ETAGS_ARGS)$$*$$unique"; then :; else \ + test -n "$$unique" || unique=$$empty_fix; \ + if test $$# -gt 0; then \ + $(ETAGS) $(ETAGSFLAGS) $(AM_ETAGSFLAGS) $(ETAGS_ARGS) \ + "$$@" $$unique; \ + else \ + $(ETAGS) $(ETAGSFLAGS) $(AM_ETAGSFLAGS) $(ETAGS_ARGS) \ + $$unique; \ + fi; \ + fi ctags: CTAGS -CTAGS: - +CTAGS: $(HEADERS) $(SOURCES) $(TAGS_DEPENDENCIES) \ + $(TAGS_FILES) $(LISP) + list='$(SOURCES) $(HEADERS) $(LISP) $(TAGS_FILES)'; \ + unique=`for i in $$list; do \ + if test -f "$$i"; then echo $$i; else echo $(srcdir)/$$i; fi; \ + done | \ + $(AWK) '{ files[$$0] = 1; nonempty = 1; } \ + END { if (nonempty) { for (i in files) print i; }; }'`; \ + test -z "$(CTAGS_ARGS)$$unique" \ + || $(CTAGS) $(CTAGSFLAGS) $(AM_CTAGSFLAGS) $(CTAGS_ARGS) \ + $$unique + +GTAGS: + here=`$(am__cd) $(top_builddir) && pwd` \ + && $(am__cd) $(top_srcdir) \ + && gtags -i $(GTAGS_ARGS) "$$here" + +distclean-tags: + -rm -f TAGS ID GTAGS GRTAGS GSYMS GPATH tags distdir: $(DISTFILES) @srcdirstrip=`echo "$(srcdir)" | sed 's/[].[^$$\\*]/\\\\&/g'`; \ @@ -496,8 +509,11 @@ distdir: $(DISTFILES) done check-am: all-am check: check-am -all-am: Makefile +all-am: Makefile $(LTLIBRARIES) installdirs: + for dir in "$(DESTDIR)$(pkglibdir)"; do \ + test -z "$$dir" || $(MKDIR_P) "$$dir"; \ + done install: install-am install-exec: install-exec-am install-data: install-data-am @@ -530,11 +546,14 @@ maintainer-clean-generic: @echo "it deletes files that may require special tools to rebuild." clean: clean-am -clean-am: clean-generic clean-libtool mostlyclean-am +clean-am: clean-generic clean-libtool clean-pkglibLTLIBRARIES \ + mostlyclean-am distclean: distclean-am + -rm -rf ./$(DEPDIR) -rm -f Makefile -distclean-am: clean-am distclean-generic +distclean-am: clean-am distclean-compile distclean-generic \ + distclean-tags dvi: dvi-am @@ -554,7 +573,7 @@ install-dvi: install-dvi-am install-dvi-am: -install-exec-am: +install-exec-am: install-pkglibLTLIBRARIES install-html: install-html-am @@ -577,12 +596,14 @@ install-ps-am: installcheck-am: maintainer-clean: maintainer-clean-am + -rm -rf ./$(DEPDIR) -rm -f Makefile maintainer-clean-am: distclean-am maintainer-clean-generic mostlyclean: mostlyclean-am -mostlyclean-am: mostlyclean-generic mostlyclean-libtool +mostlyclean-am: mostlyclean-compile mostlyclean-generic \ + mostlyclean-libtool pdf: pdf-am @@ -592,20 +613,23 @@ ps: ps-am ps-am: -uninstall-am: +uninstall-am: uninstall-pkglibLTLIBRARIES .MAKE: install-am install-strip -.PHONY: all all-am check check-am clean clean-generic clean-libtool \ - distclean distclean-generic distclean-libtool distdir dvi \ - dvi-am html html-am info info-am install install-am \ - install-data install-data-am install-dvi install-dvi-am \ - install-exec install-exec-am install-html install-html-am \ - install-info install-info-am install-man install-pdf \ - install-pdf-am install-ps install-ps-am install-strip \ - installcheck installcheck-am installdirs maintainer-clean \ - maintainer-clean-generic mostlyclean mostlyclean-generic \ - mostlyclean-libtool pdf pdf-am ps ps-am uninstall uninstall-am +.PHONY: CTAGS GTAGS all all-am check check-am clean clean-generic \ + clean-libtool clean-pkglibLTLIBRARIES ctags distclean \ + distclean-compile distclean-generic distclean-libtool \ + distclean-tags distdir dvi dvi-am html html-am info info-am \ + install install-am install-data install-data-am install-dvi \ + install-dvi-am install-exec install-exec-am install-html \ + install-html-am install-info install-info-am install-man \ + install-pdf install-pdf-am install-pkglibLTLIBRARIES \ + install-ps install-ps-am install-strip installcheck \ + installcheck-am installdirs maintainer-clean \ + maintainer-clean-generic mostlyclean mostlyclean-compile \ + mostlyclean-generic mostlyclean-libtool pdf pdf-am ps ps-am \ + tags uninstall uninstall-am uninstall-pkglibLTLIBRARIES # Tell versions [3.59,3.63) of GNU make to not export all variables. diff --git a/plugins/mmcount/mmcount.c b/plugins/mmcount/mmcount.c new file mode 100644 index 0000000..64b296a --- /dev/null +++ b/plugins/mmcount/mmcount.c @@ -0,0 +1,360 @@ +/* mmcount.c + * count messages by priority or json property of given app-name. + * + * Copyright 2013 Red Hat Inc. + * + * This file is part of rsyslog. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * -or- + * see COPYING.ASL20 in the source distribution + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +#include "config.h" +#include "rsyslog.h" +#include <stdio.h> +#include <stdarg.h> +#include <stdlib.h> +#include <string.h> +#include <assert.h> +#include <signal.h> +#include <errno.h> +#include <unistd.h> +#include <stdint.h> +#include <json.h> +#include "conf.h" +#include "syslogd-types.h" +#include "srUtils.h" +#include "template.h" +#include "module-template.h" +#include "errmsg.h" +#include "hashtable.h" + +#define JSON_COUNT_NAME "!mmcount" +#define SEVERITY_COUNT 8 + +MODULE_TYPE_OUTPUT +MODULE_TYPE_NOKEEP +MODULE_CNFNAME("mmcount") + + +DEFobjCurrIf(errmsg); +DEF_OMOD_STATIC_DATA + +/* config variables */ + +typedef struct _instanceData { + char *pszAppName; + int severity[SEVERITY_COUNT]; + char *pszKey; + char *pszValue; + int valueCounter; + struct hashtable *ht; + pthread_mutex_t mut; +} instanceData; + +typedef struct wrkrInstanceData { + instanceData *pData; +} wrkrInstanceData_t; + +struct modConfData_s { + rsconf_t *pConf; /* our overall config object */ +}; +static modConfData_t *loadModConf = NULL;/* modConf ptr to use for the current load process */ +static modConfData_t *runModConf = NULL;/* modConf ptr to use for the current exec process */ + + +/* tables for interfacing with the v6 config system */ +/* action (instance) parameters */ +static struct cnfparamdescr actpdescr[] = { + { "appname", eCmdHdlrGetWord, 0 }, + { "key", eCmdHdlrGetWord, 0 }, + { "value", eCmdHdlrGetWord, 0 }, +}; +static struct cnfparamblk actpblk = + { CNFPARAMBLK_VERSION, + sizeof(actpdescr)/sizeof(struct cnfparamdescr), + actpdescr + }; + +BEGINbeginCnfLoad +CODESTARTbeginCnfLoad + loadModConf = pModConf; + pModConf->pConf = pConf; +ENDbeginCnfLoad + +BEGINendCnfLoad +CODESTARTendCnfLoad +ENDendCnfLoad + +BEGINcheckCnf +CODESTARTcheckCnf +ENDcheckCnf + +BEGINactivateCnf +CODESTARTactivateCnf + runModConf = pModConf; +ENDactivateCnf + +BEGINfreeCnf +CODESTARTfreeCnf +ENDfreeCnf + + +BEGINcreateInstance +CODESTARTcreateInstance + pthread_mutex_init(&pData->mut, NULL); +ENDcreateInstance + +BEGINcreateWrkrInstance +CODESTARTcreateWrkrInstance +ENDcreateWrkrInstance + + +BEGINisCompatibleWithFeature +CODESTARTisCompatibleWithFeature +ENDisCompatibleWithFeature + + +BEGINfreeInstance +CODESTARTfreeInstance +ENDfreeInstance + + +BEGINfreeWrkrInstance +CODESTARTfreeWrkrInstance +ENDfreeWrkrInstance + +static inline void +setInstParamDefaults(instanceData *pData) +{ + int i; + + pData->pszAppName = NULL; + for (i = 0; i < SEVERITY_COUNT; i++) + pData->severity[i] = 0; + pData->pszKey = NULL; + pData->pszValue = NULL; + pData->valueCounter = 0; + pData->ht = NULL; +} + +static unsigned int +hash_from_key_fn(void *k) +{ + return *(unsigned int *)k; +} + +static int +key_equals_fn(void *k1, void *k2) +{ + return (*(unsigned int *)k1 == *(unsigned int *)k2); +} + +BEGINnewActInst + struct cnfparamvals *pvals; + int i; +CODESTARTnewActInst + DBGPRINTF("newActInst (mmcount)\n"); + if((pvals = nvlstGetParams(lst, &actpblk, NULL)) == NULL) { + ABORT_FINALIZE(RS_RET_MISSING_CNFPARAMS); + } + + CODE_STD_STRING_REQUESTnewActInst(1) + CHKiRet(OMSRsetEntry(*ppOMSR, 0, NULL, OMSR_TPL_AS_MSG)); + CHKiRet(createInstance(&pData)); + setInstParamDefaults(pData); + + for(i = 0 ; i < actpblk.nParams ; ++i) { + if(!pvals[i].bUsed) + continue; + if(!strcmp(actpblk.descr[i].name, "appname")) { + pData->pszAppName = es_str2cstr(pvals[i].val.d.estr, NULL); + continue; + } + if(!strcmp(actpblk.descr[i].name, "key")) { + pData->pszKey = es_str2cstr(pvals[i].val.d.estr, NULL); + continue; + } + if(!strcmp(actpblk.descr[i].name, "value")) { + pData->pszValue = es_str2cstr(pvals[i].val.d.estr, NULL); + continue; + } + dbgprintf("mmcount: program error, non-handled " + "param '%s'\n", actpblk.descr[i].name); + } + + if(pData->pszAppName == NULL) { + dbgprintf("mmcount: action requires a appname"); + ABORT_FINALIZE(RS_RET_MISSING_CNFPARAMS); + } + + if(pData->pszKey != NULL && pData->pszValue == NULL) { + if(NULL == (pData->ht = create_hashtable(100, hash_from_key_fn, key_equals_fn, NULL))) { + DBGPRINTF("mmcount: error creating hash table!\n"); + ABORT_FINALIZE(RS_RET_ERR); + } + } +CODE_STD_FINALIZERnewActInst + cnfparamvalsDestruct(pvals, &actpblk); +ENDnewActInst + + +BEGINdbgPrintInstInfo +CODESTARTdbgPrintInstInfo +ENDdbgPrintInstInfo + + +BEGINtryResume +CODESTARTtryResume +ENDtryResume + +static int * +getCounter(struct hashtable *ht, char *str) { + unsigned int key; + int *pCounter; + unsigned int *pKey; + + /* we dont store str as key, instead we store hash of the str + as key to reduce memory usage */ + key = hash_from_string(str); + pCounter = hashtable_search(ht, &key); + if(pCounter) { + return pCounter; + } + + /* counter is not found for the str, so add new entry and + return the counter */ + if(NULL == (pKey = (unsigned int*)malloc(sizeof(unsigned int)))) { + DBGPRINTF("mmcount: memory allocation for key failed\n"); + return NULL; + } + *pKey = key; + + if(NULL == (pCounter = (int*)malloc(sizeof(int)))) { + DBGPRINTF("mmcount: memory allocation for value failed\n"); + free(pKey); + return NULL; + } + *pCounter = 0; + + if(!hashtable_insert(ht, pKey, pCounter)) { + DBGPRINTF("mmcount: inserting element into hashtable failed\n"); + free(pKey); + free(pCounter); + return NULL; + } + return pCounter; +} + +BEGINdoAction + msg_t *pMsg; + char *appname; + struct json_object *json = NULL; + es_str_t *estr = NULL; + struct json_object *keyjson = NULL; + char *pszValue; + int *pCounter; + instanceData *const pData = pWrkrData->pData; +CODESTARTdoAction + pMsg = (msg_t*) ppString[0]; + appname = getAPPNAME(pMsg, LOCK_MUTEX); + + pthread_mutex_lock(&pData->mut); + if(0 != strcmp(appname, pData->pszAppName)) { + /* we are not working for this appname. nothing to do */ + ABORT_FINALIZE(RS_RET_OK); + } + + if(!pData->pszKey) { + /* no key given for count, so we count severity */ + if(pMsg->iSeverity < SEVERITY_COUNT) { + pData->severity[pMsg->iSeverity]++; + json = json_object_new_int(pData->severity[pMsg->iSeverity]); + } + ABORT_FINALIZE(RS_RET_OK); + } + + /* key is given, so get the property json */ + estr = es_newStrFromBuf(pData->pszKey, strlen(pData->pszKey)); + if(msgGetCEEPropJSON(pMsg, estr, &keyjson) != RS_RET_OK) { + /* key not found in the message. nothing to do */ + ABORT_FINALIZE(RS_RET_OK); + } + + /* key found, so get the value */ + pszValue = (char*)json_object_get_string(keyjson); + + if(pData->pszValue) { + /* value also given for count */ + if(!strcmp(pszValue, pData->pszValue)) { + /* count for (value and key and appname) matched */ + pData->valueCounter++; + json = json_object_new_int(pData->valueCounter); + } + ABORT_FINALIZE(RS_RET_OK); + } + + /* value is not given, so we count for each value of given key */ + pCounter = getCounter(pData->ht, pszValue); + if(pCounter) { + (*pCounter)++; + json = json_object_new_int(*pCounter); + } +finalize_it: + pthread_mutex_unlock(&pData->mut); + if(estr) { + es_deleteStr(estr); + } + + if(json) { + msgAddJSON(pMsg, (uchar *)JSON_COUNT_NAME, json); + } +ENDdoAction + + +BEGINparseSelectorAct +CODESTARTparseSelectorAct +CODE_STD_STRING_REQUESTparseSelectorAct(1) + if(strncmp((char*) p, ":mmcount:", sizeof(":mmcount:") - 1)) { + errmsg.LogError(0, RS_RET_LEGA_ACT_NOT_SUPPORTED, + "mmcount supports only v6+ config format, use: " + "action(type=\"mmcount\" ...)"); + } + ABORT_FINALIZE(RS_RET_CONFLINE_UNPROCESSED); +CODE_STD_FINALIZERparseSelectorAct +ENDparseSelectorAct + + +BEGINmodExit +CODESTARTmodExit + objRelease(errmsg, CORE_COMPONENT); +ENDmodExit + + +BEGINqueryEtryPt +CODESTARTqueryEtryPt +CODEqueryEtryPt_STD_OMOD_QUERIES +CODEqueryEtryPt_STD_OMOD8_QUERIES +CODEqueryEtryPt_STD_CONF2_OMOD_QUERIES +CODEqueryEtryPt_STD_CONF2_QUERIES +ENDqueryEtryPt + + + +BEGINmodInit() +CODESTARTmodInit + *ipIFVersProvided = CURR_MOD_IF_VERSION; /* we only support the current interface specification */ +CODEmodInit_QueryRegCFSLineHdlr + DBGPRINTF("mmcount: module compiled with rsyslog version %s.\n", VERSION); + CHKiRet(objUse(errmsg, CORE_COMPONENT)); +ENDmodInit diff --git a/plugins/mmexternal/Makefile.am b/plugins/mmexternal/Makefile.am new file mode 100644 index 0000000..61631b3 --- /dev/null +++ b/plugins/mmexternal/Makefile.am @@ -0,0 +1,8 @@ +pkglib_LTLIBRARIES = mmexternal.la + +mmexternal_la_SOURCES = mmexternal.c +mmexternal_la_CPPFLAGS = $(RSRT_CFLAGS) $(PTHREADS_CFLAGS) +mmexternal_la_LDFLAGS = -module -avoid-version +mmexternal_la_LIBADD = + +EXTRA_DIST = diff --git a/plugins/mmexternal/Makefile.in b/plugins/mmexternal/Makefile.in new file mode 100644 index 0000000..14e3063 --- /dev/null +++ b/plugins/mmexternal/Makefile.in @@ -0,0 +1,637 @@ +# Makefile.in generated by automake 1.11.3 from Makefile.am. +# @configure_input@ + +# Copyright (C) 1994, 1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, +# 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010, 2011 Free Software +# Foundation, Inc. +# This Makefile.in is free software; the Free Software Foundation +# gives unlimited permission to copy and/or distribute it, +# with or without modifications, as long as this notice is preserved. + +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY, to the extent permitted by law; without +# even the implied warranty of MERCHANTABILITY or FITNESS FOR A +# PARTICULAR PURPOSE. + +@SET_MAKE@ + +VPATH = @srcdir@ +pkgdatadir = $(datadir)/@PACKAGE@ +pkgincludedir = $(includedir)/@PACKAGE@ +pkglibdir = $(libdir)/@PACKAGE@ +pkglibexecdir = $(libexecdir)/@PACKAGE@ +am__cd = CDPATH="$${ZSH_VERSION+.}$(PATH_SEPARATOR)" && cd +install_sh_DATA = $(install_sh) -c -m 644 +install_sh_PROGRAM = $(install_sh) -c +install_sh_SCRIPT = $(install_sh) -c +INSTALL_HEADER = $(INSTALL_DATA) +transform = $(program_transform_name) +NORMAL_INSTALL = : +PRE_INSTALL = : +POST_INSTALL = : +NORMAL_UNINSTALL = : +PRE_UNINSTALL = : +POST_UNINSTALL = : +build_triplet = @build@ +host_triplet = @host@ +subdir = plugins/mmexternal +DIST_COMMON = $(srcdir)/Makefile.am $(srcdir)/Makefile.in +ACLOCAL_M4 = $(top_srcdir)/aclocal.m4 +am__aclocal_m4_deps = $(top_srcdir)/m4/atomic_operations.m4 \ + $(top_srcdir)/m4/atomic_operations_64bit.m4 \ + $(top_srcdir)/m4/libtool.m4 $(top_srcdir)/m4/ltoptions.m4 \ + $(top_srcdir)/m4/ltsugar.m4 $(top_srcdir)/m4/ltversion.m4 \ + $(top_srcdir)/m4/lt~obsolete.m4 $(top_srcdir)/configure.ac +am__configure_deps = $(am__aclocal_m4_deps) $(CONFIGURE_DEPENDENCIES) \ + $(ACLOCAL_M4) +mkinstalldirs = $(install_sh) -d +CONFIG_HEADER = $(top_builddir)/config.h +CONFIG_CLEAN_FILES = +CONFIG_CLEAN_VPATH_FILES = +am__vpath_adj_setup = srcdirstrip=`echo "$(srcdir)" | sed 's|.|.|g'`; +am__vpath_adj = case $$p in \ + $(srcdir)/*) f=`echo "$$p" | sed "s|^$$srcdirstrip/||"`;; \ + *) f=$$p;; \ + esac; +am__strip_dir = f=`echo $$p | sed -e 's|^.*/||'`; +am__install_max = 40 +am__nobase_strip_setup = \ + srcdirstrip=`echo "$(srcdir)" | sed 's/[].[^$$\\*|]/\\\\&/g'` +am__nobase_strip = \ + for p in $$list; do echo "$$p"; done | sed -e "s|$$srcdirstrip/||" +am__nobase_list = $(am__nobase_strip_setup); \ + for p in $$list; do echo "$$p $$p"; done | \ + sed "s| $$srcdirstrip/| |;"' / .*\//!s/ .*/ ./; s,\( .*\)/[^/]*$$,\1,' | \ + $(AWK) 'BEGIN { files["."] = "" } { files[$$2] = files[$$2] " " $$1; \ + if (++n[$$2] == $(am__install_max)) \ + { print $$2, files[$$2]; n[$$2] = 0; files[$$2] = "" } } \ + END { for (dir in files) print dir, files[dir] }' +am__base_list = \ + sed '$$!N;$$!N;$$!N;$$!N;$$!N;$$!N;$$!N;s/\n/ /g' | \ + sed '$$!N;$$!N;$$!N;$$!N;s/\n/ /g' +am__uninstall_files_from_dir = { \ + test -z "$$files" \ + || { test ! -d "$$dir" && test ! -f "$$dir" && test ! -r "$$dir"; } \ + || { echo " ( cd '$$dir' && rm -f" $$files ")"; \ + $(am__cd) "$$dir" && rm -f $$files; }; \ + } +am__installdirs = "$(DESTDIR)$(pkglibdir)" +LTLIBRARIES = $(pkglib_LTLIBRARIES) +mmexternal_la_DEPENDENCIES = +am_mmexternal_la_OBJECTS = mmexternal_la-mmexternal.lo +mmexternal_la_OBJECTS = $(am_mmexternal_la_OBJECTS) +AM_V_lt = $(am__v_lt_@AM_V@) +am__v_lt_ = $(am__v_lt_@AM_DEFAULT_V@) +am__v_lt_0 = --silent +mmexternal_la_LINK = $(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \ + $(LIBTOOLFLAGS) --mode=link $(CCLD) $(AM_CFLAGS) $(CFLAGS) \ + $(mmexternal_la_LDFLAGS) $(LDFLAGS) -o $@ +DEFAULT_INCLUDES = -I.@am__isrc@ -I$(top_builddir) +depcomp = $(SHELL) $(top_srcdir)/depcomp +am__depfiles_maybe = depfiles +am__mv = mv -f +COMPILE = $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(AM_CPPFLAGS) \ + $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) +LTCOMPILE = $(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \ + $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) \ + $(DEFAULT_INCLUDES) $(INCLUDES) $(AM_CPPFLAGS) $(CPPFLAGS) \ + $(AM_CFLAGS) $(CFLAGS) +AM_V_CC = $(am__v_CC_@AM_V@) +am__v_CC_ = $(am__v_CC_@AM_DEFAULT_V@) +am__v_CC_0 = @echo " CC " $@; +AM_V_at = $(am__v_at_@AM_V@) +am__v_at_ = $(am__v_at_@AM_DEFAULT_V@) +am__v_at_0 = @ +CCLD = $(CC) +LINK = $(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \ + $(LIBTOOLFLAGS) --mode=link $(CCLD) $(AM_CFLAGS) $(CFLAGS) \ + $(AM_LDFLAGS) $(LDFLAGS) -o $@ +AM_V_CCLD = $(am__v_CCLD_@AM_V@) +am__v_CCLD_ = $(am__v_CCLD_@AM_DEFAULT_V@) +am__v_CCLD_0 = @echo " CCLD " $@; +AM_V_GEN = $(am__v_GEN_@AM_V@) +am__v_GEN_ = $(am__v_GEN_@AM_DEFAULT_V@) +am__v_GEN_0 = @echo " GEN " $@; +SOURCES = $(mmexternal_la_SOURCES) +DIST_SOURCES = $(mmexternal_la_SOURCES) +ETAGS = etags +CTAGS = ctags +DISTFILES = $(DIST_COMMON) $(DIST_SOURCES) $(TEXINFOS) $(EXTRA_DIST) +ACLOCAL = @ACLOCAL@ +AMTAR = @AMTAR@ +AM_DEFAULT_VERBOSITY = @AM_DEFAULT_VERBOSITY@ +AR = @AR@ +AUTOCONF = @AUTOCONF@ +AUTOHEADER = @AUTOHEADER@ +AUTOMAKE = @AUTOMAKE@ +AWK = @AWK@ +CC = @CC@ +CCDEPMODE = @CCDEPMODE@ +CFLAGS = @CFLAGS@ +CPP = @CPP@ +CPPFLAGS = @CPPFLAGS@ +CURL_CFLAGS = @CURL_CFLAGS@ +CURL_LIBS = @CURL_LIBS@ +CYGPATH_W = @CYGPATH_W@ +CZMQ_CFLAGS = @CZMQ_CFLAGS@ +CZMQ_LIBS = @CZMQ_LIBS@ +DEFS = @DEFS@ +DEPDIR = @DEPDIR@ +DLLTOOL = @DLLTOOL@ +DL_LIBS = @DL_LIBS@ +DSYMUTIL = @DSYMUTIL@ +DUMPBIN = @DUMPBIN@ +ECHO_C = @ECHO_C@ +ECHO_N = @ECHO_N@ +ECHO_T = @ECHO_T@ +EGREP = @EGREP@ +EXEEXT = @EXEEXT@ +FGREP = @FGREP@ +GNUTLS_CFLAGS = @GNUTLS_CFLAGS@ +GNUTLS_LIBS = @GNUTLS_LIBS@ +GREP = @GREP@ +GSS_LIBS = @GSS_LIBS@ +GUARDTIME_CFLAGS = @GUARDTIME_CFLAGS@ +GUARDTIME_LIBS = @GUARDTIME_LIBS@ +HAVE_MYSQL_CONFIG = @HAVE_MYSQL_CONFIG@ +HAVE_ORACLE_CONFIG = @HAVE_ORACLE_CONFIG@ +HAVE_PGSQL_CONFIG = @HAVE_PGSQL_CONFIG@ +HIREDIS_CFLAGS = @HIREDIS_CFLAGS@ +HIREDIS_LIBS = @HIREDIS_LIBS@ +IMUDP_LIBS = @IMUDP_LIBS@ +INSTALL = @INSTALL@ +INSTALL_DATA = @INSTALL_DATA@ +INSTALL_PROGRAM = @INSTALL_PROGRAM@ +INSTALL_SCRIPT = @INSTALL_SCRIPT@ +INSTALL_STRIP_PROGRAM = @INSTALL_STRIP_PROGRAM@ +JSON_C_CFLAGS = @JSON_C_CFLAGS@ +JSON_C_LIBS = @JSON_C_LIBS@ +LD = @LD@ +LDFLAGS = @LDFLAGS@ +LEX = @LEX@ +LEXLIB = @LEXLIB@ +LEX_OUTPUT_ROOT = @LEX_OUTPUT_ROOT@ +LIBDBI_CFLAGS = @LIBDBI_CFLAGS@ +LIBDBI_LIBS = @LIBDBI_LIBS@ +LIBESTR_CFLAGS = @LIBESTR_CFLAGS@ +LIBESTR_LIBS = @LIBESTR_LIBS@ +LIBGCRYPT_CFLAGS = @LIBGCRYPT_CFLAGS@ +LIBGCRYPT_CONFIG = @LIBGCRYPT_CONFIG@ +LIBGCRYPT_LIBS = @LIBGCRYPT_LIBS@ +LIBLOGGING_CFLAGS = @LIBLOGGING_CFLAGS@ +LIBLOGGING_LIBS = @LIBLOGGING_LIBS@ +LIBLOGGING_STDLOG_CFLAGS = @LIBLOGGING_STDLOG_CFLAGS@ +LIBLOGGING_STDLOG_LIBS = @LIBLOGGING_STDLOG_LIBS@ +LIBLOGNORM_CFLAGS = @LIBLOGNORM_CFLAGS@ +LIBLOGNORM_LIBS = @LIBLOGNORM_LIBS@ +LIBM = @LIBM@ +LIBMONGO_CLIENT_CFLAGS = @LIBMONGO_CLIENT_CFLAGS@ +LIBMONGO_CLIENT_LIBS = @LIBMONGO_CLIENT_LIBS@ +LIBOBJS = @LIBOBJS@ +LIBS = @LIBS@ +LIBSYSTEMD_JOURNAL_CFLAGS = @LIBSYSTEMD_JOURNAL_CFLAGS@ +LIBSYSTEMD_JOURNAL_LIBS = @LIBSYSTEMD_JOURNAL_LIBS@ +LIBTOOL = @LIBTOOL@ +LIBUUID_CFLAGS = @LIBUUID_CFLAGS@ +LIBUUID_LIBS = @LIBUUID_LIBS@ +LIPO = @LIPO@ +LN_S = @LN_S@ +LTLIBOBJS = @LTLIBOBJS@ +MAKEINFO = @MAKEINFO@ +MANIFEST_TOOL = @MANIFEST_TOOL@ +MKDIR_P = @MKDIR_P@ +MYSQL_CFLAGS = @MYSQL_CFLAGS@ +MYSQL_LIBS = @MYSQL_LIBS@ +NM = @NM@ +NMEDIT = @NMEDIT@ +OBJDUMP = @OBJDUMP@ +OBJEXT = @OBJEXT@ +OPENSSL_CFLAGS = @OPENSSL_CFLAGS@ +OPENSSL_LIBS = @OPENSSL_LIBS@ +ORACLE_CFLAGS = @ORACLE_CFLAGS@ +ORACLE_LIBS = @ORACLE_LIBS@ +OTOOL = @OTOOL@ +OTOOL64 = @OTOOL64@ +PACKAGE = @PACKAGE@ +PACKAGE_BUGREPORT = @PACKAGE_BUGREPORT@ +PACKAGE_NAME = @PACKAGE_NAME@ +PACKAGE_STRING = @PACKAGE_STRING@ +PACKAGE_TARNAME = @PACKAGE_TARNAME@ +PACKAGE_URL = @PACKAGE_URL@ +PACKAGE_VERSION = @PACKAGE_VERSION@ +PATH_SEPARATOR = @PATH_SEPARATOR@ +PGSQL_CFLAGS = @PGSQL_CFLAGS@ +PGSQL_LIBS = @PGSQL_LIBS@ +PKG_CONFIG = @PKG_CONFIG@ +PKG_CONFIG_LIBDIR = @PKG_CONFIG_LIBDIR@ +PKG_CONFIG_PATH = @PKG_CONFIG_PATH@ +PTHREADS_CFLAGS = @PTHREADS_CFLAGS@ +PTHREADS_LIBS = @PTHREADS_LIBS@ +RABBITMQ_CFLAGS = @RABBITMQ_CFLAGS@ +RABBITMQ_LIBS = @RABBITMQ_LIBS@ +RANLIB = @RANLIB@ +RELP_CFLAGS = @RELP_CFLAGS@ +RELP_LIBS = @RELP_LIBS@ +RSRT_CFLAGS = @RSRT_CFLAGS@ +RSRT_CFLAGS1 = @RSRT_CFLAGS1@ +RSRT_LIBS = @RSRT_LIBS@ +RSRT_LIBS1 = @RSRT_LIBS1@ +RST2MAN = @RST2MAN@ +RT_LIBS = @RT_LIBS@ +SED = @SED@ +SET_MAKE = @SET_MAKE@ +SHELL = @SHELL@ +SNMP_CFLAGS = @SNMP_CFLAGS@ +SNMP_LIBS = @SNMP_LIBS@ +SOL_LIBS = @SOL_LIBS@ +STRIP = @STRIP@ +UDPSPOOF_CFLAGS = @UDPSPOOF_CFLAGS@ +UDPSPOOF_LIBS = @UDPSPOOF_LIBS@ +VERSION = @VERSION@ +YACC = @YACC@ +YFLAGS = @YFLAGS@ +ZLIB_LIBS = @ZLIB_LIBS@ +abs_builddir = @abs_builddir@ +abs_srcdir = @abs_srcdir@ +abs_top_builddir = @abs_top_builddir@ +abs_top_srcdir = @abs_top_srcdir@ +ac_ct_AR = @ac_ct_AR@ +ac_ct_CC = @ac_ct_CC@ +ac_ct_DUMPBIN = @ac_ct_DUMPBIN@ +am__include = @am__include@ +am__leading_dot = @am__leading_dot@ +am__quote = @am__quote@ +am__tar = @am__tar@ +am__untar = @am__untar@ +bindir = @bindir@ +build = @build@ +build_alias = @build_alias@ +build_cpu = @build_cpu@ +build_os = @build_os@ +build_vendor = @build_vendor@ +builddir = @builddir@ +datadir = @datadir@ +datarootdir = @datarootdir@ +docdir = @docdir@ +dvidir = @dvidir@ +exec_prefix = @exec_prefix@ +have_valgrind = @have_valgrind@ +host = @host@ +host_alias = @host_alias@ +host_cpu = @host_cpu@ +host_os = @host_os@ +host_vendor = @host_vendor@ +htmldir = @htmldir@ +includedir = @includedir@ +infodir = @infodir@ +install_sh = @install_sh@ +libdir = @libdir@ +libexecdir = @libexecdir@ +localedir = @localedir@ +localstatedir = @localstatedir@ +mandir = @mandir@ +mkdir_p = @mkdir_p@ +moddirs = @moddirs@ +oldincludedir = @oldincludedir@ +pdfdir = @pdfdir@ +prefix = @prefix@ +program_transform_name = @program_transform_name@ +psdir = @psdir@ +sbindir = @sbindir@ +sharedstatedir = @sharedstatedir@ +srcdir = @srcdir@ +sysconfdir = @sysconfdir@ +systemdsystemunitdir = @systemdsystemunitdir@ +target_alias = @target_alias@ +top_build_prefix = @top_build_prefix@ +top_builddir = @top_builddir@ +top_srcdir = @top_srcdir@ +pkglib_LTLIBRARIES = mmexternal.la +mmexternal_la_SOURCES = mmexternal.c +mmexternal_la_CPPFLAGS = $(RSRT_CFLAGS) $(PTHREADS_CFLAGS) +mmexternal_la_LDFLAGS = -module -avoid-version +mmexternal_la_LIBADD = +EXTRA_DIST = +all: all-am + +.SUFFIXES: +.SUFFIXES: .c .lo .o .obj +$(srcdir)/Makefile.in: $(srcdir)/Makefile.am $(am__configure_deps) + @for dep in $?; do \ + case '$(am__configure_deps)' in \ + *$$dep*) \ + ( cd $(top_builddir) && $(MAKE) $(AM_MAKEFLAGS) am--refresh ) \ + && { if test -f $@; then exit 0; else break; fi; }; \ + exit 1;; \ + esac; \ + done; \ + echo ' cd $(top_srcdir) && $(AUTOMAKE) --gnu plugins/mmexternal/Makefile'; \ + $(am__cd) $(top_srcdir) && \ + $(AUTOMAKE) --gnu plugins/mmexternal/Makefile +.PRECIOUS: Makefile +Makefile: $(srcdir)/Makefile.in $(top_builddir)/config.status + @case '$?' in \ + *config.status*) \ + cd $(top_builddir) && $(MAKE) $(AM_MAKEFLAGS) am--refresh;; \ + *) \ + echo ' cd $(top_builddir) && $(SHELL) ./config.status $(subdir)/$@ $(am__depfiles_maybe)'; \ + cd $(top_builddir) && $(SHELL) ./config.status $(subdir)/$@ $(am__depfiles_maybe);; \ + esac; + +$(top_builddir)/config.status: $(top_srcdir)/configure $(CONFIG_STATUS_DEPENDENCIES) + cd $(top_builddir) && $(MAKE) $(AM_MAKEFLAGS) am--refresh + +$(top_srcdir)/configure: $(am__configure_deps) + cd $(top_builddir) && $(MAKE) $(AM_MAKEFLAGS) am--refresh +$(ACLOCAL_M4): $(am__aclocal_m4_deps) + cd $(top_builddir) && $(MAKE) $(AM_MAKEFLAGS) am--refresh +$(am__aclocal_m4_deps): +install-pkglibLTLIBRARIES: $(pkglib_LTLIBRARIES) + @$(NORMAL_INSTALL) + test -z "$(pkglibdir)" || $(MKDIR_P) "$(DESTDIR)$(pkglibdir)" + @list='$(pkglib_LTLIBRARIES)'; test -n "$(pkglibdir)" || list=; \ + list2=; for p in $$list; do \ + if test -f $$p; then \ + list2="$$list2 $$p"; \ + else :; fi; \ + done; \ + test -z "$$list2" || { \ + echo " $(LIBTOOL) $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=install $(INSTALL) $(INSTALL_STRIP_FLAG) $$list2 '$(DESTDIR)$(pkglibdir)'"; \ + $(LIBTOOL) $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=install $(INSTALL) $(INSTALL_STRIP_FLAG) $$list2 "$(DESTDIR)$(pkglibdir)"; \ + } + +uninstall-pkglibLTLIBRARIES: + @$(NORMAL_UNINSTALL) + @list='$(pkglib_LTLIBRARIES)'; test -n "$(pkglibdir)" || list=; \ + for p in $$list; do \ + $(am__strip_dir) \ + echo " $(LIBTOOL) $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=uninstall rm -f '$(DESTDIR)$(pkglibdir)/$$f'"; \ + $(LIBTOOL) $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=uninstall rm -f "$(DESTDIR)$(pkglibdir)/$$f"; \ + done + +clean-pkglibLTLIBRARIES: + -test -z "$(pkglib_LTLIBRARIES)" || rm -f $(pkglib_LTLIBRARIES) + @list='$(pkglib_LTLIBRARIES)'; for p in $$list; do \ + dir="`echo $$p | sed -e 's|/[^/]*$$||'`"; \ + test "$$dir" != "$$p" || dir=.; \ + echo "rm -f \"$${dir}/so_locations\""; \ + rm -f "$${dir}/so_locations"; \ + done +mmexternal.la: $(mmexternal_la_OBJECTS) $(mmexternal_la_DEPENDENCIES) $(EXTRA_mmexternal_la_DEPENDENCIES) + $(AM_V_CCLD)$(mmexternal_la_LINK) -rpath $(pkglibdir) $(mmexternal_la_OBJECTS) $(mmexternal_la_LIBADD) $(LIBS) + +mostlyclean-compile: + -rm -f *.$(OBJEXT) + +distclean-compile: + -rm -f *.tab.c + +@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/mmexternal_la-mmexternal.Plo@am__quote@ + +.c.o: +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.o$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ $< + +.c.obj: +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.obj$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ `$(CYGPATH_W) '$<'` &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ `$(CYGPATH_W) '$<'` + +.c.lo: +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.lo$$||'`;\ +@am__fastdepCC_TRUE@ $(LTCOMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Plo +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=yes @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(LTCOMPILE) -c -o $@ $< + +mmexternal_la-mmexternal.lo: mmexternal.c +@am__fastdepCC_TRUE@ $(AM_V_CC)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(mmexternal_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT mmexternal_la-mmexternal.lo -MD -MP -MF $(DEPDIR)/mmexternal_la-mmexternal.Tpo -c -o mmexternal_la-mmexternal.lo `test -f 'mmexternal.c' || echo '$(srcdir)/'`mmexternal.c +@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/mmexternal_la-mmexternal.Tpo $(DEPDIR)/mmexternal_la-mmexternal.Plo +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='mmexternal.c' object='mmexternal_la-mmexternal.lo' libtool=yes @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(mmexternal_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o mmexternal_la-mmexternal.lo `test -f 'mmexternal.c' || echo '$(srcdir)/'`mmexternal.c + +mostlyclean-libtool: + -rm -f *.lo + +clean-libtool: + -rm -rf .libs _libs + +ID: $(HEADERS) $(SOURCES) $(LISP) $(TAGS_FILES) + list='$(SOURCES) $(HEADERS) $(LISP) $(TAGS_FILES)'; \ + unique=`for i in $$list; do \ + if test -f "$$i"; then echo $$i; else echo $(srcdir)/$$i; fi; \ + done | \ + $(AWK) '{ files[$$0] = 1; nonempty = 1; } \ + END { if (nonempty) { for (i in files) print i; }; }'`; \ + mkid -fID $$unique +tags: TAGS + +TAGS: $(HEADERS) $(SOURCES) $(TAGS_DEPENDENCIES) \ + $(TAGS_FILES) $(LISP) + set x; \ + here=`pwd`; \ + list='$(SOURCES) $(HEADERS) $(LISP) $(TAGS_FILES)'; \ + unique=`for i in $$list; do \ + if test -f "$$i"; then echo $$i; else echo $(srcdir)/$$i; fi; \ + done | \ + $(AWK) '{ files[$$0] = 1; nonempty = 1; } \ + END { if (nonempty) { for (i in files) print i; }; }'`; \ + shift; \ + if test -z "$(ETAGS_ARGS)$$*$$unique"; then :; else \ + test -n "$$unique" || unique=$$empty_fix; \ + if test $$# -gt 0; then \ + $(ETAGS) $(ETAGSFLAGS) $(AM_ETAGSFLAGS) $(ETAGS_ARGS) \ + "$$@" $$unique; \ + else \ + $(ETAGS) $(ETAGSFLAGS) $(AM_ETAGSFLAGS) $(ETAGS_ARGS) \ + $$unique; \ + fi; \ + fi +ctags: CTAGS +CTAGS: $(HEADERS) $(SOURCES) $(TAGS_DEPENDENCIES) \ + $(TAGS_FILES) $(LISP) + list='$(SOURCES) $(HEADERS) $(LISP) $(TAGS_FILES)'; \ + unique=`for i in $$list; do \ + if test -f "$$i"; then echo $$i; else echo $(srcdir)/$$i; fi; \ + done | \ + $(AWK) '{ files[$$0] = 1; nonempty = 1; } \ + END { if (nonempty) { for (i in files) print i; }; }'`; \ + test -z "$(CTAGS_ARGS)$$unique" \ + || $(CTAGS) $(CTAGSFLAGS) $(AM_CTAGSFLAGS) $(CTAGS_ARGS) \ + $$unique + +GTAGS: + here=`$(am__cd) $(top_builddir) && pwd` \ + && $(am__cd) $(top_srcdir) \ + && gtags -i $(GTAGS_ARGS) "$$here" + +distclean-tags: + -rm -f TAGS ID GTAGS GRTAGS GSYMS GPATH tags + +distdir: $(DISTFILES) + @srcdirstrip=`echo "$(srcdir)" | sed 's/[].[^$$\\*]/\\\\&/g'`; \ + topsrcdirstrip=`echo "$(top_srcdir)" | sed 's/[].[^$$\\*]/\\\\&/g'`; \ + list='$(DISTFILES)'; \ + dist_files=`for file in $$list; do echo $$file; done | \ + sed -e "s|^$$srcdirstrip/||;t" \ + -e "s|^$$topsrcdirstrip/|$(top_builddir)/|;t"`; \ + case $$dist_files in \ + */*) $(MKDIR_P) `echo "$$dist_files" | \ + sed '/\//!d;s|^|$(distdir)/|;s,/[^/]*$$,,' | \ + sort -u` ;; \ + esac; \ + for file in $$dist_files; do \ + if test -f $$file || test -d $$file; then d=.; else d=$(srcdir); fi; \ + if test -d $$d/$$file; then \ + dir=`echo "/$$file" | sed -e 's,/[^/]*$$,,'`; \ + if test -d "$(distdir)/$$file"; then \ + find "$(distdir)/$$file" -type d ! -perm -700 -exec chmod u+rwx {} \;; \ + fi; \ + if test -d $(srcdir)/$$file && test $$d != $(srcdir); then \ + cp -fpR $(srcdir)/$$file "$(distdir)$$dir" || exit 1; \ + find "$(distdir)/$$file" -type d ! -perm -700 -exec chmod u+rwx {} \;; \ + fi; \ + cp -fpR $$d/$$file "$(distdir)$$dir" || exit 1; \ + else \ + test -f "$(distdir)/$$file" \ + || cp -p $$d/$$file "$(distdir)/$$file" \ + || exit 1; \ + fi; \ + done +check-am: all-am +check: check-am +all-am: Makefile $(LTLIBRARIES) +installdirs: + for dir in "$(DESTDIR)$(pkglibdir)"; do \ + test -z "$$dir" || $(MKDIR_P) "$$dir"; \ + done +install: install-am +install-exec: install-exec-am +install-data: install-data-am +uninstall: uninstall-am + +install-am: all-am + @$(MAKE) $(AM_MAKEFLAGS) install-exec-am install-data-am + +installcheck: installcheck-am +install-strip: + if test -z '$(STRIP)'; then \ + $(MAKE) $(AM_MAKEFLAGS) INSTALL_PROGRAM="$(INSTALL_STRIP_PROGRAM)" \ + install_sh_PROGRAM="$(INSTALL_STRIP_PROGRAM)" INSTALL_STRIP_FLAG=-s \ + install; \ + else \ + $(MAKE) $(AM_MAKEFLAGS) INSTALL_PROGRAM="$(INSTALL_STRIP_PROGRAM)" \ + install_sh_PROGRAM="$(INSTALL_STRIP_PROGRAM)" INSTALL_STRIP_FLAG=-s \ + "INSTALL_PROGRAM_ENV=STRIPPROG='$(STRIP)'" install; \ + fi +mostlyclean-generic: + +clean-generic: + +distclean-generic: + -test -z "$(CONFIG_CLEAN_FILES)" || rm -f $(CONFIG_CLEAN_FILES) + -test . = "$(srcdir)" || test -z "$(CONFIG_CLEAN_VPATH_FILES)" || rm -f $(CONFIG_CLEAN_VPATH_FILES) + +maintainer-clean-generic: + @echo "This command is intended for maintainers to use" + @echo "it deletes files that may require special tools to rebuild." +clean: clean-am + +clean-am: clean-generic clean-libtool clean-pkglibLTLIBRARIES \ + mostlyclean-am + +distclean: distclean-am + -rm -rf ./$(DEPDIR) + -rm -f Makefile +distclean-am: clean-am distclean-compile distclean-generic \ + distclean-tags + +dvi: dvi-am + +dvi-am: + +html: html-am + +html-am: + +info: info-am + +info-am: + +install-data-am: + +install-dvi: install-dvi-am + +install-dvi-am: + +install-exec-am: install-pkglibLTLIBRARIES + +install-html: install-html-am + +install-html-am: + +install-info: install-info-am + +install-info-am: + +install-man: + +install-pdf: install-pdf-am + +install-pdf-am: + +install-ps: install-ps-am + +install-ps-am: + +installcheck-am: + +maintainer-clean: maintainer-clean-am + -rm -rf ./$(DEPDIR) + -rm -f Makefile +maintainer-clean-am: distclean-am maintainer-clean-generic + +mostlyclean: mostlyclean-am + +mostlyclean-am: mostlyclean-compile mostlyclean-generic \ + mostlyclean-libtool + +pdf: pdf-am + +pdf-am: + +ps: ps-am + +ps-am: + +uninstall-am: uninstall-pkglibLTLIBRARIES + +.MAKE: install-am install-strip + +.PHONY: CTAGS GTAGS all all-am check check-am clean clean-generic \ + clean-libtool clean-pkglibLTLIBRARIES ctags distclean \ + distclean-compile distclean-generic distclean-libtool \ + distclean-tags distdir dvi dvi-am html html-am info info-am \ + install install-am install-data install-data-am install-dvi \ + install-dvi-am install-exec install-exec-am install-html \ + install-html-am install-info install-info-am install-man \ + install-pdf install-pdf-am install-pkglibLTLIBRARIES \ + install-ps install-ps-am install-strip installcheck \ + installcheck-am installdirs maintainer-clean \ + maintainer-clean-generic mostlyclean mostlyclean-compile \ + mostlyclean-generic mostlyclean-libtool pdf pdf-am ps ps-am \ + tags uninstall uninstall-am uninstall-pkglibLTLIBRARIES + + +# Tell versions [3.59,3.63) of GNU make to not export all variables. +# Otherwise a system limit (for SysV at least) may be exceeded. +.NOEXPORT: diff --git a/plugins/mmexternal/mmexternal.c b/plugins/mmexternal/mmexternal.c new file mode 100644 index 0000000..04cf6fc --- /dev/null +++ b/plugins/mmexternal/mmexternal.c @@ -0,0 +1,708 @@ +/* mmexternal.c + * This core plugin is an interface module to message modification + * modules written in languages other than C. + * + * Copyright 2014 by Rainer Gerhards + * + * This file is part of rsyslog. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * -or- + * see COPYING.ASL20 in the source distribution + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +#include "config.h" +#include "rsyslog.h" +#include <stdio.h> +#include <stdarg.h> +#include <stdlib.h> +#include <string.h> +#include <assert.h> +#include <signal.h> +#include <errno.h> +#include <unistd.h> +#include <fcntl.h> +#include <wait.h> +#include <sys/uio.h> +#include "conf.h" +#include "syslogd-types.h" +#include "srUtils.h" +#include "module-template.h" +#include "msg.h" +#include "errmsg.h" +#include "cfsysline.h" + +MODULE_TYPE_OUTPUT +MODULE_TYPE_NOKEEP +MODULE_CNFNAME("mmexternal") + +/* internal structures + */ +DEF_OMOD_STATIC_DATA +DEFobjCurrIf(errmsg) + +typedef struct _instanceData { + uchar *szBinary; /* name of binary to call */ + char **aParams; /* Optional Parameters for binary command */ + int iParams; /* Holds the count of parameters if set*/ + int bForceSingleInst; /* only a single wrkr instance of program permitted? */ + int inputProp; /* what to provide as input to the external program? */ +#define INPUT_MSG 0 +#define INPUT_RAWMSG 1 +#define INPUT_JSON 2 + uchar *outputFileName; /* name of file for std[out/err] or NULL if to discard */ + pthread_mutex_t mut; /* make sure only one instance is active */ +} instanceData; + +typedef struct wrkrInstanceData { + instanceData *pData; + pid_t pid; /* pid of currently running process */ + int fdOutput; /* it's fd (-1 if closed) */ + int fdPipeOut; /* file descriptor to write to */ + int fdPipeIn; /* fd we receive messages from the program (if we want to) */ + int bIsRunning; /* is binary currently running? 0-no, 1-yes */ + char *respBuf; /* buffer to read exernal plugin's response */ + int maxLenRespBuf; /* (current) maximum length of response buffer */ + int lenRespBuf; /* actual nbr of chars in response buffer */ + int idxRespBuf; /* last char read from response buffer */ +} wrkrInstanceData_t; + +typedef struct configSettings_s { + uchar *szBinary; /* name of binary to call */ +} configSettings_t; +static configSettings_t cs; + + +/* tables for interfacing with the v6 config system */ +/* action (instance) parameters */ +static struct cnfparamdescr actpdescr[] = { + { "binary", eCmdHdlrString, CNFPARAM_REQUIRED }, + { "interface.input", eCmdHdlrString, 0 }, + { "output", eCmdHdlrString, 0 }, + { "forcesingleinstance", eCmdHdlrBinary, 0 } +}; +static struct cnfparamblk actpblk = + { CNFPARAMBLK_VERSION, + sizeof(actpdescr)/sizeof(struct cnfparamdescr), + actpdescr + }; + +BEGINinitConfVars /* (re)set config variables to default values */ +CODESTARTinitConfVars + cs.szBinary = NULL; /* name of binary to call */ +ENDinitConfVars + +/* config settings */ + +BEGINcreateInstance +CODESTARTcreateInstance + pData->inputProp = INPUT_MSG; + pthread_mutex_init(&pData->mut, NULL); +ENDcreateInstance + +BEGINcreateWrkrInstance +CODESTARTcreateWrkrInstance + pWrkrData->fdPipeIn = -1; + pWrkrData->fdPipeOut = -1; + pWrkrData->fdOutput = -1; + pWrkrData->bIsRunning = 0; + pWrkrData->respBuf = NULL; + pWrkrData->maxLenRespBuf = 0; + pWrkrData->lenRespBuf = 0; + pWrkrData->idxRespBuf = 0; +ENDcreateWrkrInstance + + +BEGINisCompatibleWithFeature +CODESTARTisCompatibleWithFeature + if(eFeat == sFEATURERepeatedMsgReduction) + iRet = RS_RET_OK; +ENDisCompatibleWithFeature + + +BEGINfreeInstance + int i; +CODESTARTfreeInstance + pthread_mutex_destroy(&pData->mut); + free(pData->szBinary); + free(pData->outputFileName); + if(pData->aParams != NULL) { + for (i = 0; i < pData->iParams; i++) { + free(pData->aParams[i]); + } + free(pData->aParams); + } +ENDfreeInstance + +BEGINfreeWrkrInstance +CODESTARTfreeWrkrInstance + free(pWrkrData->respBuf); +ENDfreeWrkrInstance + + +BEGINdbgPrintInstInfo +CODESTARTdbgPrintInstInfo +ENDdbgPrintInstInfo + + +BEGINtryResume +CODESTARTtryResume +ENDtryResume + + +/* As this is just a debug function, we only make + * best effort to write the message but do *not* try very + * hard to handle errors. -- rgerhards, 2014-01-16 + */ +static void +writeOutputDebug(wrkrInstanceData_t *__restrict__ const pWrkrData, + const char *__restrict__ const buf, + const ssize_t lenBuf) +{ + char errStr[1024]; + ssize_t r; + + if(pWrkrData->pData->outputFileName == NULL) + goto done; + + if(pWrkrData->fdOutput == -1) { + pWrkrData->fdOutput = open((char*)pWrkrData->pData->outputFileName, + O_WRONLY | O_APPEND | O_CREAT, 0600); + if(pWrkrData->fdOutput == -1) { + DBGPRINTF("mmexternal: error opening output file %s: %s\n", + pWrkrData->pData->outputFileName, + rs_strerror_r(errno, errStr, sizeof(errStr))); + goto done; + } + } + + r = write(pWrkrData->fdOutput, buf, (size_t) lenBuf); + if(r != lenBuf) { + DBGPRINTF("mmexternal: problem writing output file %s: bytes " + "requested %lld, written %lld, msg: %s\n", + pWrkrData->pData->outputFileName, (long long) lenBuf, (long long) r, + rs_strerror_r(errno, errStr, sizeof(errStr))); + } +done: return; +} + + +/* Get reply from external program. Note that we *must* receive one + * reply for each message sent (half-duplex protocol). As such, the last + * char we read MUST be \n ... we cannot have multiple LF as this is + * forbidden by the plugin interface. We cannot have multiple responses + * for multiple messages, as we are in half-duplex mode! This makes + * things quite a bit simpler. So don't think the simple code does + * not handle those border-cases that are describe to cannot exist! + */ +static void +processProgramReply(wrkrInstanceData_t *__restrict__ const pWrkrData, msg_t *const pMsg) +{ + rsRetVal iRet; + char errStr[1024]; + ssize_t r; + int numCharsRead; + char *newptr; + +dbgprintf("mmexternal: checking prog output, fd %d\n", pWrkrData->fdPipeIn); + numCharsRead = 0; + do { + if(pWrkrData->maxLenRespBuf < numCharsRead + 256) { /* 256 to permit at least a decent read */ + pWrkrData->maxLenRespBuf += 4096; + if((newptr = realloc(pWrkrData->respBuf, pWrkrData->maxLenRespBuf)) == NULL) { + DBGPRINTF("mmexternal: error realloc responseBuf: %s\n", + rs_strerror_r(errno, errStr, sizeof(errStr))); + /* emergency - fake no update */ + strcpy(pWrkrData->respBuf, "{}\n"); + numCharsRead = 3; + break; + } + pWrkrData->respBuf = newptr; + } + r = read(pWrkrData->fdPipeIn, pWrkrData->respBuf+numCharsRead, + pWrkrData->maxLenRespBuf-numCharsRead-1); + if(r > 0) { + numCharsRead += r; + pWrkrData->respBuf[numCharsRead] = '\0'; /* space reserved in read! */ + } else { + /* emergency - fake no update */ + strcpy(pWrkrData->respBuf, "{}\n"); + numCharsRead = 3; + } +dbgprintf("mmexternal: read state %lld, data '%s'\n", (long long) r, pWrkrData->respBuf); + if(Debug && r == -1) { + DBGPRINTF("mmexternal: error reading from external program: %s\n", + rs_strerror_r(errno, errStr, sizeof(errStr))); + } + } while(pWrkrData->respBuf[numCharsRead-1] != '\n'); + + writeOutputDebug(pWrkrData, pWrkrData->respBuf, numCharsRead); + /* strip LF, which is not part of the JSON message but framing */ + pWrkrData->respBuf[numCharsRead-1] = '\0'; + iRet = MsgSetPropsViaJSON(pMsg, (uchar*)pWrkrData->respBuf); + if(iRet != RS_RET_OK) { + errmsg.LogError(0, iRet, "mmexternal: invalid reply '%s' from program '%s'", + pWrkrData->respBuf, pWrkrData->pData->szBinary); + } + + return; +} + + + +/* execute the child process (must be called in child context + * after fork). + */ +static void +execBinary(wrkrInstanceData_t *pWrkrData, int fdStdin, int fdStdOutErr) +{ + int i, iRet; + struct sigaction sigAct; + sigset_t set; + char errStr[1024]; + char *newenviron[] = { NULL }; + + fclose(stdin); + if(dup(fdStdin) == -1) { + DBGPRINTF("mmexternal: dup() stdin failed\n"); + } + close(1); + if(dup(fdStdOutErr) == -1) { + DBGPRINTF("mmexternal: dup() stdout failed\n"); + } + /* todo: different pipe for stderr? */ + close(2); + if(dup(fdStdOutErr) == -1) { + DBGPRINTF("mmexternal: dup() stderr failed\n"); + } + + /* we close all file handles as we fork soon + * Is there a better way to do this? - mail me! rgerhards@adiscon.com + */ +# ifndef VALGRIND /* we can not use this with valgrind - too many errors... */ + for(i = 3 ; i <= 65535 ; ++i) + close(i); +# endif + + /* reset signal handlers to default */ + memset(&sigAct, 0, sizeof(sigAct)); + sigemptyset(&sigAct.sa_mask); + sigAct.sa_handler = SIG_DFL; + for(i = 1 ; i < NSIG ; ++i) + sigaction(i, &sigAct, NULL); + /* we need to block SIGINT, otherwise the external program is cancelled when we are + * stopped in debug mode. + */ + sigAct.sa_handler = SIG_IGN; + sigaction(SIGINT, &sigAct, NULL); + sigemptyset(&set); + sigprocmask(SIG_SETMASK, &set, NULL); + + alarm(0); + + /* finally exec child */ + iRet = execve((char*)pWrkrData->pData->szBinary, pWrkrData->pData->aParams, newenviron); + if(iRet == -1) { + /* Note: this will go to stdout of the **child**, so rsyslog will never + * see it except when stdout is captured. If we use the plugin interface, + * we can use this to convey a proper status back! + */ + rs_strerror_r(errno, errStr, sizeof(errStr)); + DBGPRINTF("mmexternal: failed to execute binary '%s': %s\n", + pWrkrData->pData->szBinary, errStr); + } + + /* we should never reach this point, but if we do, we terminate */ + exit(1); +} + + +/* creates a pipe and starts program, uses pipe as stdin for program. + * rgerhards, 2009-04-01 + */ +static rsRetVal +openPipe(wrkrInstanceData_t *pWrkrData) +{ + int pipestdin[2]; + int pipestdout[2]; + pid_t cpid; + DEFiRet; + + if(pipe(pipestdin) == -1) { + ABORT_FINALIZE(RS_RET_ERR_CREAT_PIPE); + } + if(pipe(pipestdout) == -1) { + ABORT_FINALIZE(RS_RET_ERR_CREAT_PIPE); + } + + DBGPRINTF("mmexternal: executing program '%s' with '%d' parameters\n", + pWrkrData->pData->szBinary, pWrkrData->pData->iParams); + + /* NO OUTPUT AFTER FORK! */ + + cpid = fork(); + if(cpid == -1) { + ABORT_FINALIZE(RS_RET_ERR_FORK); + } + pWrkrData->pid = cpid; + + if(cpid == 0) { + /* we are now the child, just exec the binary. */ + close(pipestdin[1]); /* close those pipe "ports" that */ + close(pipestdout[0]); /* we don't need */ + execBinary(pWrkrData, pipestdin[0], pipestdout[1]); + /*NO CODE HERE - WILL NEVER BE REACHED!*/ + } + + DBGPRINTF("mmexternal: child has pid %d\n", (int) cpid); + pWrkrData->fdPipeIn = dup(pipestdout[0]); + close(pipestdin[0]); + close(pipestdout[1]); + pWrkrData->pid = cpid; + pWrkrData->fdPipeOut = pipestdin[1]; + pWrkrData->bIsRunning = 1; +finalize_it: + RETiRet; +} + + +/* clean up after a terminated child + */ +static inline rsRetVal +cleanup(wrkrInstanceData_t *pWrkrData) +{ + int status; + int ret; + char errStr[1024]; + DEFiRet; + + assert(pWrkrData->bIsRunning == 1); + ret = waitpid(pWrkrData->pid, &status, 0); + if(ret != pWrkrData->pid) { + /* if waitpid() fails, we can not do much - try to ignore it... */ + DBGPRINTF("mmexternal: waitpid() returned state %d[%s], future malfunction may happen\n", ret, + rs_strerror_r(errno, errStr, sizeof(errStr))); + } else { + /* check if we should print out some diagnostic information */ + DBGPRINTF("mmexternal: waitpid status return for program '%s': %2.2x\n", + pWrkrData->pData->szBinary, status); + if(WIFEXITED(status)) { + errmsg.LogError(0, NO_ERRCODE, "program '%s' exited normally, state %d", + pWrkrData->pData->szBinary, WEXITSTATUS(status)); + } else if(WIFSIGNALED(status)) { + errmsg.LogError(0, NO_ERRCODE, "program '%s' terminated by signal %d.", + pWrkrData->pData->szBinary, WTERMSIG(status)); + } + } + + if(pWrkrData->fdOutput != -1) { + close(pWrkrData->fdOutput); + pWrkrData->fdOutput = -1; + } + if(pWrkrData->fdPipeIn != -1) { + close(pWrkrData->fdPipeIn); + pWrkrData->fdPipeIn = -1; + } + if(pWrkrData->fdPipeOut != -1) { + close(pWrkrData->fdPipeOut); + pWrkrData->fdPipeOut = -1; + } + pWrkrData->bIsRunning = 0; + pWrkrData->bIsRunning = 0; + RETiRet; +} + + +/* try to restart the binary when it has stopped. + */ +static inline rsRetVal +tryRestart(wrkrInstanceData_t *pWrkrData) +{ + DEFiRet; + assert(pWrkrData->bIsRunning == 0); + + iRet = openPipe(pWrkrData); + RETiRet; +} + +/* write to pipe + * note that we do not try to run block-free. If the users fears something + * may block (and this not be acceptable), the action should be run on its + * own action queue. + */ +static rsRetVal +callExtProg(wrkrInstanceData_t *__restrict__ const pWrkrData, msg_t *__restrict__ const pMsg) +{ + int lenWritten; + int lenWrite; + int writeOffset; + int i_iov; + char errStr[1024]; + struct iovec iov[2]; + const uchar *inputstr = NULL; /* string to be processed by external program */ + DEFiRet; + + if(pWrkrData->pData->inputProp == INPUT_MSG) { + inputstr = getMSG(pMsg); + lenWrite = getMSGLen(pMsg); + } else if(pWrkrData->pData->inputProp == INPUT_RAWMSG) { + getRawMsg(pMsg, (uchar**)&inputstr, &lenWrite); + } else { + inputstr = msgGetJSONMESG(pMsg); + lenWrite = strlen((const char*)inputstr); + } + + writeOffset = 0; + do { + DBGPRINTF("mmexternal: writing to prog (fd %d, offset %d): %s\n", + pWrkrData->fdPipeOut, (int) writeOffset, inputstr); + i_iov = 0; + if(writeOffset < lenWrite) { + iov[0].iov_base = (char*)inputstr+writeOffset; + iov[0].iov_len = lenWrite - writeOffset; + ++i_iov; + } + iov[i_iov].iov_base = "\n"; + iov[i_iov].iov_len = 1; + lenWritten = writev(pWrkrData->fdPipeOut, iov, i_iov+1); + if(lenWritten == -1) { + switch(errno) { + case EPIPE: + DBGPRINTF("mmexternal: program '%s' terminated, trying to restart\n", + pWrkrData->pData->szBinary); + CHKiRet(cleanup(pWrkrData)); + CHKiRet(tryRestart(pWrkrData)); + writeOffset = 0; + break; + default: + DBGPRINTF("mmexternal: error %d writing to pipe: %s\n", errno, + rs_strerror_r(errno, errStr, sizeof(errStr))); + ABORT_FINALIZE(RS_RET_ERR_WRITE_PIPE); + break; + } + } else { + writeOffset += lenWritten; + } + } while(lenWritten != lenWrite+1); + + processProgramReply(pWrkrData, pMsg); + +finalize_it: + /* we need to free json input strings, only. All others point to memory + * inside the msg object, which is destroyed when the msg is destroyed. + */ + if(pWrkrData->pData->inputProp == INPUT_JSON) + free((void*)inputstr); + RETiRet; +} + + +BEGINdoAction + instanceData *pData; +CODESTARTdoAction +dbgprintf("DDDD:mmexternal processing message\n"); + pData = pWrkrData->pData; + if(pData->bForceSingleInst) + pthread_mutex_lock(&pData->mut); + if(pWrkrData->bIsRunning == 0) { + openPipe(pWrkrData); + } + + iRet = callExtProg(pWrkrData, (msg_t*)ppString[0]); + + if(iRet != RS_RET_OK) + iRet = RS_RET_SUSPENDED; + if(pData->bForceSingleInst) + pthread_mutex_unlock(&pData->mut); +dbgprintf("DDDD:mmexternal DONE processing message\n"); +ENDdoAction + + +static inline void +setInstParamDefaults(instanceData *pData) +{ + pData->szBinary = NULL; + pData->aParams = NULL; + pData->outputFileName = NULL; + pData->iParams = 0; + pData->bForceSingleInst = 0; +} + +BEGINnewActInst + struct cnfparamvals *pvals; + sbool bInQuotes; + int i; + int iPrm; + unsigned char *c; + es_size_t iCnt; + es_size_t iStr; + es_str_t *estrBinary; + es_str_t *estrParams; + es_str_t *estrTmp; + const char *cstr = NULL; +CODESTARTnewActInst + if((pvals = nvlstGetParams(lst, &actpblk, NULL)) == NULL) { + ABORT_FINALIZE(RS_RET_MISSING_CNFPARAMS); + } + + CHKiRet(createInstance(&pData)); + setInstParamDefaults(pData); + + CODE_STD_STRING_REQUESTnewActInst(1) + for(i = 0 ; i < actpblk.nParams ; ++i) { + if(!pvals[i].bUsed) + continue; + if(!strcmp(actpblk.descr[i].name, "binary")) { + estrBinary = pvals[i].val.d.estr; + estrParams = NULL; + + /* Search for space */ + c = es_getBufAddr(pvals[i].val.d.estr); + iCnt = 0; + while(iCnt < es_strlen(pvals[i].val.d.estr) ) { + if (c[iCnt] == ' ') { + /* Split binary name from parameters */ + estrBinary = es_newStrFromSubStr ( pvals[i].val.d.estr, 0, iCnt ); + estrParams = es_newStrFromSubStr ( pvals[i].val.d.estr, iCnt+1, es_strlen(pvals[i].val.d.estr)); + break; + } + iCnt++; + } + /* Assign binary and params */ + pData->szBinary = (uchar*)es_str2cstr(estrBinary, NULL); + dbgprintf("mmexternal: szBinary = '%s'\n", pData->szBinary); + /* Check for Params! */ + if (estrParams != NULL) { + dbgprintf("mmexternal: szParams = '%s'\n", es_str2cstr(estrParams, NULL) ); + + /* Count parameters if set */ + c = es_getBufAddr(estrParams); /* Reset to beginning */ + pData->iParams = 2; /* Set default to 2, first parameter for binary and second parameter at least from config*/ + iCnt = 0; + while(iCnt < es_strlen(estrParams) ) { + if (c[iCnt] == ' ' && c[iCnt-1] != '\\') + pData->iParams++; + iCnt++; + } + dbgprintf("mmexternal: iParams = '%d'\n", pData->iParams); + + /* Create argv Array */ + CHKmalloc(pData->aParams = malloc( (pData->iParams+1) * sizeof(char*))); /* One more for first param */ + + /* Second Loop, create parameter array*/ + c = es_getBufAddr(estrParams); /* Reset to beginning */ + iCnt = iStr = iPrm = 0; + estrTmp = NULL; + bInQuotes = FALSE; + /* Set first parameter to binary */ + pData->aParams[iPrm] = strdup((char*)pData->szBinary); + dbgprintf("mmexternal: Param (%d): '%s'\n", iPrm, pData->aParams[iPrm]); + iPrm++; + while(iCnt < es_strlen(estrParams) ) { + if ( c[iCnt] == ' ' && !bInQuotes ) { + /* Copy into Param Array! */ + estrTmp = es_newStrFromSubStr( estrParams, iStr, iCnt-iStr); + } + else if ( iCnt+1 >= es_strlen(estrParams) ) { + /* Copy rest of string into Param Array! */ + estrTmp = es_newStrFromSubStr( estrParams, iStr, iCnt-iStr+1); + } + else if (c[iCnt] == '"') { + /* switch inQuotes Mode */ + bInQuotes = !bInQuotes; + } + + if ( estrTmp != NULL ) { + pData->aParams[iPrm] = es_str2cstr(estrTmp, NULL); + iStr = iCnt+1; /* Set new start */ + dbgprintf("mmexternal: Param (%d): '%s'\n", iPrm, pData->aParams[iPrm]); + es_deleteStr( estrTmp ); + estrTmp = NULL; + iPrm++; + } + + /*Next char*/ + iCnt++; + } + /* NULL last parameter! */ + pData->aParams[iPrm] = NULL; + + } + } else if(!strcmp(actpblk.descr[i].name, "output")) { + pData->outputFileName = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + } else if(!strcmp(actpblk.descr[i].name, "forcesingleinstance")) { + pData->bForceSingleInst = (int) pvals[i].val.d.n; + } else if(!strcmp(actpblk.descr[i].name, "interface.input")) { + cstr = es_str2cstr(pvals[i].val.d.estr, NULL); + if(!strcmp(cstr, "msg")) + pData->inputProp = INPUT_MSG; + else if(!strcmp(cstr, "rawmsg")) + pData->inputProp = INPUT_RAWMSG; + else if(!strcmp(cstr, "fulljson")) + pData->inputProp = INPUT_JSON; + else { + errmsg.LogError(0, RS_RET_INVLD_INTERFACE_INPUT, + "mmexternal: invalid interface.input parameter '%s'", + cstr); + ABORT_FINALIZE(RS_RET_INVLD_INTERFACE_INPUT); + } + } else { + dbgprintf("mmexternal: program error, non-handled param '%s'\n", actpblk.descr[i].name); + } + } + + CHKiRet(OMSRsetEntry(*ppOMSR, 0, NULL, OMSR_TPL_AS_MSG)); + DBGPRINTF("mmexternal: bForceSingleInst %d\n", pData->bForceSingleInst); + DBGPRINTF("mmexternal: interface.input '%s', mode %d\n", cstr, pData->inputProp); +CODE_STD_FINALIZERnewActInst + free((void*)cstr); + cnfparamvalsDestruct(pvals, &actpblk); +ENDnewActInst + +BEGINparseSelectorAct +CODESTARTparseSelectorAct +CODE_STD_STRING_REQUESTparseSelectorAct(1) + if(!strncmp((char*) p, ":mmexternal:", sizeof(":mmexternal:") - 1)) { + errmsg.LogError(0, RS_RET_LEGA_ACT_NOT_SUPPORTED, + "mmexternal supports only v6+ config format, use: " + "action(type=\"mmexternal\" binary=...)"); + } + ABORT_FINALIZE(RS_RET_CONFLINE_UNPROCESSED); +CODE_STD_FINALIZERparseSelectorAct +ENDparseSelectorAct + + +BEGINmodExit +CODESTARTmodExit + free(cs.szBinary); + cs.szBinary = NULL; + CHKiRet(objRelease(errmsg, CORE_COMPONENT)); +finalize_it: +ENDmodExit + + +BEGINqueryEtryPt +CODESTARTqueryEtryPt +CODEqueryEtryPt_STD_OMOD_QUERIES +CODEqueryEtryPt_STD_OMOD8_QUERIES +CODEqueryEtryPt_STD_CONF2_CNFNAME_QUERIES +CODEqueryEtryPt_STD_CONF2_OMOD_QUERIES +ENDqueryEtryPt + +BEGINmodInit() +CODESTARTmodInit +INITLegCnfVars + *ipIFVersProvided = CURR_MOD_IF_VERSION; /* we only support the current interface specification */ +CODEmodInit_QueryRegCFSLineHdlr + CHKiRet(objUse(errmsg, CORE_COMPONENT)); +CODEmodInit_QueryRegCFSLineHdlr +ENDmodInit diff --git a/plugins/mmfields/Makefile.am b/plugins/mmfields/Makefile.am new file mode 100644 index 0000000..08170d5 --- /dev/null +++ b/plugins/mmfields/Makefile.am @@ -0,0 +1,8 @@ +pkglib_LTLIBRARIES = mmfields.la + +mmfields_la_SOURCES = mmfields.c +mmfields_la_CPPFLAGS = $(RSRT_CFLAGS) $(PTHREADS_CFLAGS) +mmfields_la_LDFLAGS = -module -avoid-version +mmfields_la_LIBADD = + +EXTRA_DIST = diff --git a/plugins/mmfields/Makefile.in b/plugins/mmfields/Makefile.in new file mode 100644 index 0000000..00a1b79 --- /dev/null +++ b/plugins/mmfields/Makefile.in @@ -0,0 +1,637 @@ +# Makefile.in generated by automake 1.11.3 from Makefile.am. +# @configure_input@ + +# Copyright (C) 1994, 1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, +# 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010, 2011 Free Software +# Foundation, Inc. +# This Makefile.in is free software; the Free Software Foundation +# gives unlimited permission to copy and/or distribute it, +# with or without modifications, as long as this notice is preserved. + +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY, to the extent permitted by law; without +# even the implied warranty of MERCHANTABILITY or FITNESS FOR A +# PARTICULAR PURPOSE. + +@SET_MAKE@ + +VPATH = @srcdir@ +pkgdatadir = $(datadir)/@PACKAGE@ +pkgincludedir = $(includedir)/@PACKAGE@ +pkglibdir = $(libdir)/@PACKAGE@ +pkglibexecdir = $(libexecdir)/@PACKAGE@ +am__cd = CDPATH="$${ZSH_VERSION+.}$(PATH_SEPARATOR)" && cd +install_sh_DATA = $(install_sh) -c -m 644 +install_sh_PROGRAM = $(install_sh) -c +install_sh_SCRIPT = $(install_sh) -c +INSTALL_HEADER = $(INSTALL_DATA) +transform = $(program_transform_name) +NORMAL_INSTALL = : +PRE_INSTALL = : +POST_INSTALL = : +NORMAL_UNINSTALL = : +PRE_UNINSTALL = : +POST_UNINSTALL = : +build_triplet = @build@ +host_triplet = @host@ +subdir = plugins/mmfields +DIST_COMMON = $(srcdir)/Makefile.am $(srcdir)/Makefile.in +ACLOCAL_M4 = $(top_srcdir)/aclocal.m4 +am__aclocal_m4_deps = $(top_srcdir)/m4/atomic_operations.m4 \ + $(top_srcdir)/m4/atomic_operations_64bit.m4 \ + $(top_srcdir)/m4/libtool.m4 $(top_srcdir)/m4/ltoptions.m4 \ + $(top_srcdir)/m4/ltsugar.m4 $(top_srcdir)/m4/ltversion.m4 \ + $(top_srcdir)/m4/lt~obsolete.m4 $(top_srcdir)/configure.ac +am__configure_deps = $(am__aclocal_m4_deps) $(CONFIGURE_DEPENDENCIES) \ + $(ACLOCAL_M4) +mkinstalldirs = $(install_sh) -d +CONFIG_HEADER = $(top_builddir)/config.h +CONFIG_CLEAN_FILES = +CONFIG_CLEAN_VPATH_FILES = +am__vpath_adj_setup = srcdirstrip=`echo "$(srcdir)" | sed 's|.|.|g'`; +am__vpath_adj = case $$p in \ + $(srcdir)/*) f=`echo "$$p" | sed "s|^$$srcdirstrip/||"`;; \ + *) f=$$p;; \ + esac; +am__strip_dir = f=`echo $$p | sed -e 's|^.*/||'`; +am__install_max = 40 +am__nobase_strip_setup = \ + srcdirstrip=`echo "$(srcdir)" | sed 's/[].[^$$\\*|]/\\\\&/g'` +am__nobase_strip = \ + for p in $$list; do echo "$$p"; done | sed -e "s|$$srcdirstrip/||" +am__nobase_list = $(am__nobase_strip_setup); \ + for p in $$list; do echo "$$p $$p"; done | \ + sed "s| $$srcdirstrip/| |;"' / .*\//!s/ .*/ ./; s,\( .*\)/[^/]*$$,\1,' | \ + $(AWK) 'BEGIN { files["."] = "" } { files[$$2] = files[$$2] " " $$1; \ + if (++n[$$2] == $(am__install_max)) \ + { print $$2, files[$$2]; n[$$2] = 0; files[$$2] = "" } } \ + END { for (dir in files) print dir, files[dir] }' +am__base_list = \ + sed '$$!N;$$!N;$$!N;$$!N;$$!N;$$!N;$$!N;s/\n/ /g' | \ + sed '$$!N;$$!N;$$!N;$$!N;s/\n/ /g' +am__uninstall_files_from_dir = { \ + test -z "$$files" \ + || { test ! -d "$$dir" && test ! -f "$$dir" && test ! -r "$$dir"; } \ + || { echo " ( cd '$$dir' && rm -f" $$files ")"; \ + $(am__cd) "$$dir" && rm -f $$files; }; \ + } +am__installdirs = "$(DESTDIR)$(pkglibdir)" +LTLIBRARIES = $(pkglib_LTLIBRARIES) +mmfields_la_DEPENDENCIES = +am_mmfields_la_OBJECTS = mmfields_la-mmfields.lo +mmfields_la_OBJECTS = $(am_mmfields_la_OBJECTS) +AM_V_lt = $(am__v_lt_@AM_V@) +am__v_lt_ = $(am__v_lt_@AM_DEFAULT_V@) +am__v_lt_0 = --silent +mmfields_la_LINK = $(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \ + $(LIBTOOLFLAGS) --mode=link $(CCLD) $(AM_CFLAGS) $(CFLAGS) \ + $(mmfields_la_LDFLAGS) $(LDFLAGS) -o $@ +DEFAULT_INCLUDES = -I.@am__isrc@ -I$(top_builddir) +depcomp = $(SHELL) $(top_srcdir)/depcomp +am__depfiles_maybe = depfiles +am__mv = mv -f +COMPILE = $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(AM_CPPFLAGS) \ + $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) +LTCOMPILE = $(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \ + $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) \ + $(DEFAULT_INCLUDES) $(INCLUDES) $(AM_CPPFLAGS) $(CPPFLAGS) \ + $(AM_CFLAGS) $(CFLAGS) +AM_V_CC = $(am__v_CC_@AM_V@) +am__v_CC_ = $(am__v_CC_@AM_DEFAULT_V@) +am__v_CC_0 = @echo " CC " $@; +AM_V_at = $(am__v_at_@AM_V@) +am__v_at_ = $(am__v_at_@AM_DEFAULT_V@) +am__v_at_0 = @ +CCLD = $(CC) +LINK = $(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \ + $(LIBTOOLFLAGS) --mode=link $(CCLD) $(AM_CFLAGS) $(CFLAGS) \ + $(AM_LDFLAGS) $(LDFLAGS) -o $@ +AM_V_CCLD = $(am__v_CCLD_@AM_V@) +am__v_CCLD_ = $(am__v_CCLD_@AM_DEFAULT_V@) +am__v_CCLD_0 = @echo " CCLD " $@; +AM_V_GEN = $(am__v_GEN_@AM_V@) +am__v_GEN_ = $(am__v_GEN_@AM_DEFAULT_V@) +am__v_GEN_0 = @echo " GEN " $@; +SOURCES = $(mmfields_la_SOURCES) +DIST_SOURCES = $(mmfields_la_SOURCES) +ETAGS = etags +CTAGS = ctags +DISTFILES = $(DIST_COMMON) $(DIST_SOURCES) $(TEXINFOS) $(EXTRA_DIST) +ACLOCAL = @ACLOCAL@ +AMTAR = @AMTAR@ +AM_DEFAULT_VERBOSITY = @AM_DEFAULT_VERBOSITY@ +AR = @AR@ +AUTOCONF = @AUTOCONF@ +AUTOHEADER = @AUTOHEADER@ +AUTOMAKE = @AUTOMAKE@ +AWK = @AWK@ +CC = @CC@ +CCDEPMODE = @CCDEPMODE@ +CFLAGS = @CFLAGS@ +CPP = @CPP@ +CPPFLAGS = @CPPFLAGS@ +CURL_CFLAGS = @CURL_CFLAGS@ +CURL_LIBS = @CURL_LIBS@ +CYGPATH_W = @CYGPATH_W@ +CZMQ_CFLAGS = @CZMQ_CFLAGS@ +CZMQ_LIBS = @CZMQ_LIBS@ +DEFS = @DEFS@ +DEPDIR = @DEPDIR@ +DLLTOOL = @DLLTOOL@ +DL_LIBS = @DL_LIBS@ +DSYMUTIL = @DSYMUTIL@ +DUMPBIN = @DUMPBIN@ +ECHO_C = @ECHO_C@ +ECHO_N = @ECHO_N@ +ECHO_T = @ECHO_T@ +EGREP = @EGREP@ +EXEEXT = @EXEEXT@ +FGREP = @FGREP@ +GNUTLS_CFLAGS = @GNUTLS_CFLAGS@ +GNUTLS_LIBS = @GNUTLS_LIBS@ +GREP = @GREP@ +GSS_LIBS = @GSS_LIBS@ +GUARDTIME_CFLAGS = @GUARDTIME_CFLAGS@ +GUARDTIME_LIBS = @GUARDTIME_LIBS@ +HAVE_MYSQL_CONFIG = @HAVE_MYSQL_CONFIG@ +HAVE_ORACLE_CONFIG = @HAVE_ORACLE_CONFIG@ +HAVE_PGSQL_CONFIG = @HAVE_PGSQL_CONFIG@ +HIREDIS_CFLAGS = @HIREDIS_CFLAGS@ +HIREDIS_LIBS = @HIREDIS_LIBS@ +IMUDP_LIBS = @IMUDP_LIBS@ +INSTALL = @INSTALL@ +INSTALL_DATA = @INSTALL_DATA@ +INSTALL_PROGRAM = @INSTALL_PROGRAM@ +INSTALL_SCRIPT = @INSTALL_SCRIPT@ +INSTALL_STRIP_PROGRAM = @INSTALL_STRIP_PROGRAM@ +JSON_C_CFLAGS = @JSON_C_CFLAGS@ +JSON_C_LIBS = @JSON_C_LIBS@ +LD = @LD@ +LDFLAGS = @LDFLAGS@ +LEX = @LEX@ +LEXLIB = @LEXLIB@ +LEX_OUTPUT_ROOT = @LEX_OUTPUT_ROOT@ +LIBDBI_CFLAGS = @LIBDBI_CFLAGS@ +LIBDBI_LIBS = @LIBDBI_LIBS@ +LIBESTR_CFLAGS = @LIBESTR_CFLAGS@ +LIBESTR_LIBS = @LIBESTR_LIBS@ +LIBGCRYPT_CFLAGS = @LIBGCRYPT_CFLAGS@ +LIBGCRYPT_CONFIG = @LIBGCRYPT_CONFIG@ +LIBGCRYPT_LIBS = @LIBGCRYPT_LIBS@ +LIBLOGGING_CFLAGS = @LIBLOGGING_CFLAGS@ +LIBLOGGING_LIBS = @LIBLOGGING_LIBS@ +LIBLOGGING_STDLOG_CFLAGS = @LIBLOGGING_STDLOG_CFLAGS@ +LIBLOGGING_STDLOG_LIBS = @LIBLOGGING_STDLOG_LIBS@ +LIBLOGNORM_CFLAGS = @LIBLOGNORM_CFLAGS@ +LIBLOGNORM_LIBS = @LIBLOGNORM_LIBS@ +LIBM = @LIBM@ +LIBMONGO_CLIENT_CFLAGS = @LIBMONGO_CLIENT_CFLAGS@ +LIBMONGO_CLIENT_LIBS = @LIBMONGO_CLIENT_LIBS@ +LIBOBJS = @LIBOBJS@ +LIBS = @LIBS@ +LIBSYSTEMD_JOURNAL_CFLAGS = @LIBSYSTEMD_JOURNAL_CFLAGS@ +LIBSYSTEMD_JOURNAL_LIBS = @LIBSYSTEMD_JOURNAL_LIBS@ +LIBTOOL = @LIBTOOL@ +LIBUUID_CFLAGS = @LIBUUID_CFLAGS@ +LIBUUID_LIBS = @LIBUUID_LIBS@ +LIPO = @LIPO@ +LN_S = @LN_S@ +LTLIBOBJS = @LTLIBOBJS@ +MAKEINFO = @MAKEINFO@ +MANIFEST_TOOL = @MANIFEST_TOOL@ +MKDIR_P = @MKDIR_P@ +MYSQL_CFLAGS = @MYSQL_CFLAGS@ +MYSQL_LIBS = @MYSQL_LIBS@ +NM = @NM@ +NMEDIT = @NMEDIT@ +OBJDUMP = @OBJDUMP@ +OBJEXT = @OBJEXT@ +OPENSSL_CFLAGS = @OPENSSL_CFLAGS@ +OPENSSL_LIBS = @OPENSSL_LIBS@ +ORACLE_CFLAGS = @ORACLE_CFLAGS@ +ORACLE_LIBS = @ORACLE_LIBS@ +OTOOL = @OTOOL@ +OTOOL64 = @OTOOL64@ +PACKAGE = @PACKAGE@ +PACKAGE_BUGREPORT = @PACKAGE_BUGREPORT@ +PACKAGE_NAME = @PACKAGE_NAME@ +PACKAGE_STRING = @PACKAGE_STRING@ +PACKAGE_TARNAME = @PACKAGE_TARNAME@ +PACKAGE_URL = @PACKAGE_URL@ +PACKAGE_VERSION = @PACKAGE_VERSION@ +PATH_SEPARATOR = @PATH_SEPARATOR@ +PGSQL_CFLAGS = @PGSQL_CFLAGS@ +PGSQL_LIBS = @PGSQL_LIBS@ +PKG_CONFIG = @PKG_CONFIG@ +PKG_CONFIG_LIBDIR = @PKG_CONFIG_LIBDIR@ +PKG_CONFIG_PATH = @PKG_CONFIG_PATH@ +PTHREADS_CFLAGS = @PTHREADS_CFLAGS@ +PTHREADS_LIBS = @PTHREADS_LIBS@ +RABBITMQ_CFLAGS = @RABBITMQ_CFLAGS@ +RABBITMQ_LIBS = @RABBITMQ_LIBS@ +RANLIB = @RANLIB@ +RELP_CFLAGS = @RELP_CFLAGS@ +RELP_LIBS = @RELP_LIBS@ +RSRT_CFLAGS = @RSRT_CFLAGS@ +RSRT_CFLAGS1 = @RSRT_CFLAGS1@ +RSRT_LIBS = @RSRT_LIBS@ +RSRT_LIBS1 = @RSRT_LIBS1@ +RST2MAN = @RST2MAN@ +RT_LIBS = @RT_LIBS@ +SED = @SED@ +SET_MAKE = @SET_MAKE@ +SHELL = @SHELL@ +SNMP_CFLAGS = @SNMP_CFLAGS@ +SNMP_LIBS = @SNMP_LIBS@ +SOL_LIBS = @SOL_LIBS@ +STRIP = @STRIP@ +UDPSPOOF_CFLAGS = @UDPSPOOF_CFLAGS@ +UDPSPOOF_LIBS = @UDPSPOOF_LIBS@ +VERSION = @VERSION@ +YACC = @YACC@ +YFLAGS = @YFLAGS@ +ZLIB_LIBS = @ZLIB_LIBS@ +abs_builddir = @abs_builddir@ +abs_srcdir = @abs_srcdir@ +abs_top_builddir = @abs_top_builddir@ +abs_top_srcdir = @abs_top_srcdir@ +ac_ct_AR = @ac_ct_AR@ +ac_ct_CC = @ac_ct_CC@ +ac_ct_DUMPBIN = @ac_ct_DUMPBIN@ +am__include = @am__include@ +am__leading_dot = @am__leading_dot@ +am__quote = @am__quote@ +am__tar = @am__tar@ +am__untar = @am__untar@ +bindir = @bindir@ +build = @build@ +build_alias = @build_alias@ +build_cpu = @build_cpu@ +build_os = @build_os@ +build_vendor = @build_vendor@ +builddir = @builddir@ +datadir = @datadir@ +datarootdir = @datarootdir@ +docdir = @docdir@ +dvidir = @dvidir@ +exec_prefix = @exec_prefix@ +have_valgrind = @have_valgrind@ +host = @host@ +host_alias = @host_alias@ +host_cpu = @host_cpu@ +host_os = @host_os@ +host_vendor = @host_vendor@ +htmldir = @htmldir@ +includedir = @includedir@ +infodir = @infodir@ +install_sh = @install_sh@ +libdir = @libdir@ +libexecdir = @libexecdir@ +localedir = @localedir@ +localstatedir = @localstatedir@ +mandir = @mandir@ +mkdir_p = @mkdir_p@ +moddirs = @moddirs@ +oldincludedir = @oldincludedir@ +pdfdir = @pdfdir@ +prefix = @prefix@ +program_transform_name = @program_transform_name@ +psdir = @psdir@ +sbindir = @sbindir@ +sharedstatedir = @sharedstatedir@ +srcdir = @srcdir@ +sysconfdir = @sysconfdir@ +systemdsystemunitdir = @systemdsystemunitdir@ +target_alias = @target_alias@ +top_build_prefix = @top_build_prefix@ +top_builddir = @top_builddir@ +top_srcdir = @top_srcdir@ +pkglib_LTLIBRARIES = mmfields.la +mmfields_la_SOURCES = mmfields.c +mmfields_la_CPPFLAGS = $(RSRT_CFLAGS) $(PTHREADS_CFLAGS) +mmfields_la_LDFLAGS = -module -avoid-version +mmfields_la_LIBADD = +EXTRA_DIST = +all: all-am + +.SUFFIXES: +.SUFFIXES: .c .lo .o .obj +$(srcdir)/Makefile.in: $(srcdir)/Makefile.am $(am__configure_deps) + @for dep in $?; do \ + case '$(am__configure_deps)' in \ + *$$dep*) \ + ( cd $(top_builddir) && $(MAKE) $(AM_MAKEFLAGS) am--refresh ) \ + && { if test -f $@; then exit 0; else break; fi; }; \ + exit 1;; \ + esac; \ + done; \ + echo ' cd $(top_srcdir) && $(AUTOMAKE) --gnu plugins/mmfields/Makefile'; \ + $(am__cd) $(top_srcdir) && \ + $(AUTOMAKE) --gnu plugins/mmfields/Makefile +.PRECIOUS: Makefile +Makefile: $(srcdir)/Makefile.in $(top_builddir)/config.status + @case '$?' in \ + *config.status*) \ + cd $(top_builddir) && $(MAKE) $(AM_MAKEFLAGS) am--refresh;; \ + *) \ + echo ' cd $(top_builddir) && $(SHELL) ./config.status $(subdir)/$@ $(am__depfiles_maybe)'; \ + cd $(top_builddir) && $(SHELL) ./config.status $(subdir)/$@ $(am__depfiles_maybe);; \ + esac; + +$(top_builddir)/config.status: $(top_srcdir)/configure $(CONFIG_STATUS_DEPENDENCIES) + cd $(top_builddir) && $(MAKE) $(AM_MAKEFLAGS) am--refresh + +$(top_srcdir)/configure: $(am__configure_deps) + cd $(top_builddir) && $(MAKE) $(AM_MAKEFLAGS) am--refresh +$(ACLOCAL_M4): $(am__aclocal_m4_deps) + cd $(top_builddir) && $(MAKE) $(AM_MAKEFLAGS) am--refresh +$(am__aclocal_m4_deps): +install-pkglibLTLIBRARIES: $(pkglib_LTLIBRARIES) + @$(NORMAL_INSTALL) + test -z "$(pkglibdir)" || $(MKDIR_P) "$(DESTDIR)$(pkglibdir)" + @list='$(pkglib_LTLIBRARIES)'; test -n "$(pkglibdir)" || list=; \ + list2=; for p in $$list; do \ + if test -f $$p; then \ + list2="$$list2 $$p"; \ + else :; fi; \ + done; \ + test -z "$$list2" || { \ + echo " $(LIBTOOL) $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=install $(INSTALL) $(INSTALL_STRIP_FLAG) $$list2 '$(DESTDIR)$(pkglibdir)'"; \ + $(LIBTOOL) $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=install $(INSTALL) $(INSTALL_STRIP_FLAG) $$list2 "$(DESTDIR)$(pkglibdir)"; \ + } + +uninstall-pkglibLTLIBRARIES: + @$(NORMAL_UNINSTALL) + @list='$(pkglib_LTLIBRARIES)'; test -n "$(pkglibdir)" || list=; \ + for p in $$list; do \ + $(am__strip_dir) \ + echo " $(LIBTOOL) $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=uninstall rm -f '$(DESTDIR)$(pkglibdir)/$$f'"; \ + $(LIBTOOL) $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=uninstall rm -f "$(DESTDIR)$(pkglibdir)/$$f"; \ + done + +clean-pkglibLTLIBRARIES: + -test -z "$(pkglib_LTLIBRARIES)" || rm -f $(pkglib_LTLIBRARIES) + @list='$(pkglib_LTLIBRARIES)'; for p in $$list; do \ + dir="`echo $$p | sed -e 's|/[^/]*$$||'`"; \ + test "$$dir" != "$$p" || dir=.; \ + echo "rm -f \"$${dir}/so_locations\""; \ + rm -f "$${dir}/so_locations"; \ + done +mmfields.la: $(mmfields_la_OBJECTS) $(mmfields_la_DEPENDENCIES) $(EXTRA_mmfields_la_DEPENDENCIES) + $(AM_V_CCLD)$(mmfields_la_LINK) -rpath $(pkglibdir) $(mmfields_la_OBJECTS) $(mmfields_la_LIBADD) $(LIBS) + +mostlyclean-compile: + -rm -f *.$(OBJEXT) + +distclean-compile: + -rm -f *.tab.c + +@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/mmfields_la-mmfields.Plo@am__quote@ + +.c.o: +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.o$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ $< + +.c.obj: +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.obj$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ `$(CYGPATH_W) '$<'` &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ `$(CYGPATH_W) '$<'` + +.c.lo: +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.lo$$||'`;\ +@am__fastdepCC_TRUE@ $(LTCOMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Plo +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=yes @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(LTCOMPILE) -c -o $@ $< + +mmfields_la-mmfields.lo: mmfields.c +@am__fastdepCC_TRUE@ $(AM_V_CC)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(mmfields_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT mmfields_la-mmfields.lo -MD -MP -MF $(DEPDIR)/mmfields_la-mmfields.Tpo -c -o mmfields_la-mmfields.lo `test -f 'mmfields.c' || echo '$(srcdir)/'`mmfields.c +@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/mmfields_la-mmfields.Tpo $(DEPDIR)/mmfields_la-mmfields.Plo +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='mmfields.c' object='mmfields_la-mmfields.lo' libtool=yes @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(mmfields_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o mmfields_la-mmfields.lo `test -f 'mmfields.c' || echo '$(srcdir)/'`mmfields.c + +mostlyclean-libtool: + -rm -f *.lo + +clean-libtool: + -rm -rf .libs _libs + +ID: $(HEADERS) $(SOURCES) $(LISP) $(TAGS_FILES) + list='$(SOURCES) $(HEADERS) $(LISP) $(TAGS_FILES)'; \ + unique=`for i in $$list; do \ + if test -f "$$i"; then echo $$i; else echo $(srcdir)/$$i; fi; \ + done | \ + $(AWK) '{ files[$$0] = 1; nonempty = 1; } \ + END { if (nonempty) { for (i in files) print i; }; }'`; \ + mkid -fID $$unique +tags: TAGS + +TAGS: $(HEADERS) $(SOURCES) $(TAGS_DEPENDENCIES) \ + $(TAGS_FILES) $(LISP) + set x; \ + here=`pwd`; \ + list='$(SOURCES) $(HEADERS) $(LISP) $(TAGS_FILES)'; \ + unique=`for i in $$list; do \ + if test -f "$$i"; then echo $$i; else echo $(srcdir)/$$i; fi; \ + done | \ + $(AWK) '{ files[$$0] = 1; nonempty = 1; } \ + END { if (nonempty) { for (i in files) print i; }; }'`; \ + shift; \ + if test -z "$(ETAGS_ARGS)$$*$$unique"; then :; else \ + test -n "$$unique" || unique=$$empty_fix; \ + if test $$# -gt 0; then \ + $(ETAGS) $(ETAGSFLAGS) $(AM_ETAGSFLAGS) $(ETAGS_ARGS) \ + "$$@" $$unique; \ + else \ + $(ETAGS) $(ETAGSFLAGS) $(AM_ETAGSFLAGS) $(ETAGS_ARGS) \ + $$unique; \ + fi; \ + fi +ctags: CTAGS +CTAGS: $(HEADERS) $(SOURCES) $(TAGS_DEPENDENCIES) \ + $(TAGS_FILES) $(LISP) + list='$(SOURCES) $(HEADERS) $(LISP) $(TAGS_FILES)'; \ + unique=`for i in $$list; do \ + if test -f "$$i"; then echo $$i; else echo $(srcdir)/$$i; fi; \ + done | \ + $(AWK) '{ files[$$0] = 1; nonempty = 1; } \ + END { if (nonempty) { for (i in files) print i; }; }'`; \ + test -z "$(CTAGS_ARGS)$$unique" \ + || $(CTAGS) $(CTAGSFLAGS) $(AM_CTAGSFLAGS) $(CTAGS_ARGS) \ + $$unique + +GTAGS: + here=`$(am__cd) $(top_builddir) && pwd` \ + && $(am__cd) $(top_srcdir) \ + && gtags -i $(GTAGS_ARGS) "$$here" + +distclean-tags: + -rm -f TAGS ID GTAGS GRTAGS GSYMS GPATH tags + +distdir: $(DISTFILES) + @srcdirstrip=`echo "$(srcdir)" | sed 's/[].[^$$\\*]/\\\\&/g'`; \ + topsrcdirstrip=`echo "$(top_srcdir)" | sed 's/[].[^$$\\*]/\\\\&/g'`; \ + list='$(DISTFILES)'; \ + dist_files=`for file in $$list; do echo $$file; done | \ + sed -e "s|^$$srcdirstrip/||;t" \ + -e "s|^$$topsrcdirstrip/|$(top_builddir)/|;t"`; \ + case $$dist_files in \ + */*) $(MKDIR_P) `echo "$$dist_files" | \ + sed '/\//!d;s|^|$(distdir)/|;s,/[^/]*$$,,' | \ + sort -u` ;; \ + esac; \ + for file in $$dist_files; do \ + if test -f $$file || test -d $$file; then d=.; else d=$(srcdir); fi; \ + if test -d $$d/$$file; then \ + dir=`echo "/$$file" | sed -e 's,/[^/]*$$,,'`; \ + if test -d "$(distdir)/$$file"; then \ + find "$(distdir)/$$file" -type d ! -perm -700 -exec chmod u+rwx {} \;; \ + fi; \ + if test -d $(srcdir)/$$file && test $$d != $(srcdir); then \ + cp -fpR $(srcdir)/$$file "$(distdir)$$dir" || exit 1; \ + find "$(distdir)/$$file" -type d ! -perm -700 -exec chmod u+rwx {} \;; \ + fi; \ + cp -fpR $$d/$$file "$(distdir)$$dir" || exit 1; \ + else \ + test -f "$(distdir)/$$file" \ + || cp -p $$d/$$file "$(distdir)/$$file" \ + || exit 1; \ + fi; \ + done +check-am: all-am +check: check-am +all-am: Makefile $(LTLIBRARIES) +installdirs: + for dir in "$(DESTDIR)$(pkglibdir)"; do \ + test -z "$$dir" || $(MKDIR_P) "$$dir"; \ + done +install: install-am +install-exec: install-exec-am +install-data: install-data-am +uninstall: uninstall-am + +install-am: all-am + @$(MAKE) $(AM_MAKEFLAGS) install-exec-am install-data-am + +installcheck: installcheck-am +install-strip: + if test -z '$(STRIP)'; then \ + $(MAKE) $(AM_MAKEFLAGS) INSTALL_PROGRAM="$(INSTALL_STRIP_PROGRAM)" \ + install_sh_PROGRAM="$(INSTALL_STRIP_PROGRAM)" INSTALL_STRIP_FLAG=-s \ + install; \ + else \ + $(MAKE) $(AM_MAKEFLAGS) INSTALL_PROGRAM="$(INSTALL_STRIP_PROGRAM)" \ + install_sh_PROGRAM="$(INSTALL_STRIP_PROGRAM)" INSTALL_STRIP_FLAG=-s \ + "INSTALL_PROGRAM_ENV=STRIPPROG='$(STRIP)'" install; \ + fi +mostlyclean-generic: + +clean-generic: + +distclean-generic: + -test -z "$(CONFIG_CLEAN_FILES)" || rm -f $(CONFIG_CLEAN_FILES) + -test . = "$(srcdir)" || test -z "$(CONFIG_CLEAN_VPATH_FILES)" || rm -f $(CONFIG_CLEAN_VPATH_FILES) + +maintainer-clean-generic: + @echo "This command is intended for maintainers to use" + @echo "it deletes files that may require special tools to rebuild." +clean: clean-am + +clean-am: clean-generic clean-libtool clean-pkglibLTLIBRARIES \ + mostlyclean-am + +distclean: distclean-am + -rm -rf ./$(DEPDIR) + -rm -f Makefile +distclean-am: clean-am distclean-compile distclean-generic \ + distclean-tags + +dvi: dvi-am + +dvi-am: + +html: html-am + +html-am: + +info: info-am + +info-am: + +install-data-am: + +install-dvi: install-dvi-am + +install-dvi-am: + +install-exec-am: install-pkglibLTLIBRARIES + +install-html: install-html-am + +install-html-am: + +install-info: install-info-am + +install-info-am: + +install-man: + +install-pdf: install-pdf-am + +install-pdf-am: + +install-ps: install-ps-am + +install-ps-am: + +installcheck-am: + +maintainer-clean: maintainer-clean-am + -rm -rf ./$(DEPDIR) + -rm -f Makefile +maintainer-clean-am: distclean-am maintainer-clean-generic + +mostlyclean: mostlyclean-am + +mostlyclean-am: mostlyclean-compile mostlyclean-generic \ + mostlyclean-libtool + +pdf: pdf-am + +pdf-am: + +ps: ps-am + +ps-am: + +uninstall-am: uninstall-pkglibLTLIBRARIES + +.MAKE: install-am install-strip + +.PHONY: CTAGS GTAGS all all-am check check-am clean clean-generic \ + clean-libtool clean-pkglibLTLIBRARIES ctags distclean \ + distclean-compile distclean-generic distclean-libtool \ + distclean-tags distdir dvi dvi-am html html-am info info-am \ + install install-am install-data install-data-am install-dvi \ + install-dvi-am install-exec install-exec-am install-html \ + install-html-am install-info install-info-am install-man \ + install-pdf install-pdf-am install-pkglibLTLIBRARIES \ + install-ps install-ps-am install-strip installcheck \ + installcheck-am installdirs maintainer-clean \ + maintainer-clean-generic mostlyclean mostlyclean-compile \ + mostlyclean-generic mostlyclean-libtool pdf pdf-am ps ps-am \ + tags uninstall uninstall-am uninstall-pkglibLTLIBRARIES + + +# Tell versions [3.59,3.63) of GNU make to not export all variables. +# Otherwise a system limit (for SysV at least) may be exceeded. +.NOEXPORT: diff --git a/plugins/mmfields/mmfields.c b/plugins/mmfields/mmfields.c new file mode 100644 index 0000000..cdce199 --- /dev/null +++ b/plugins/mmfields/mmfields.c @@ -0,0 +1,287 @@ +/* mmfields.c + * Parse all fields of the message into structured data inside the + * JSON tree. + * + * Copyright 2013 Adiscon GmbH. + * + * This file is part of rsyslog. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * -or- + * see COPYING.ASL20 in the source distribution + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +#include "config.h" +#include "rsyslog.h" +#include <stdio.h> +#include <stdarg.h> +#include <stdlib.h> +#include <string.h> +#include <assert.h> +#include <signal.h> +#include <errno.h> +#include <unistd.h> +#include <stdint.h> +#include "conf.h" +#include "syslogd-types.h" +#include "srUtils.h" +#include "template.h" +#include "module-template.h" +#include "errmsg.h" + +MODULE_TYPE_OUTPUT +MODULE_TYPE_NOKEEP +MODULE_CNFNAME("mmfields") + + +DEFobjCurrIf(errmsg) +DEF_OMOD_STATIC_DATA + +/* config variables */ + +/* define operation modes we have */ +#define SIMPLE_MODE 0 /* just overwrite */ +#define REWRITE_MODE 1 /* rewrite IP address, canoninized */ +typedef struct _instanceData { + char separator; + uchar *jsonRoot; /**< container where to store fields */ +} instanceData; + +typedef struct wrkrInstanceData { + instanceData *pData; +} wrkrInstanceData_t; + +struct modConfData_s { + rsconf_t *pConf; /* our overall config object */ +}; +static modConfData_t *loadModConf = NULL;/* modConf ptr to use for the current load process */ +static modConfData_t *runModConf = NULL;/* modConf ptr to use for the current exec process */ + + +/* tables for interfacing with the v6 config system */ +/* action (instance) parameters */ +static struct cnfparamdescr actpdescr[] = { + { "separator", eCmdHdlrGetChar, 0 }, + { "jsonroot", eCmdHdlrString, 0 } +}; +static struct cnfparamblk actpblk = + { CNFPARAMBLK_VERSION, + sizeof(actpdescr)/sizeof(struct cnfparamdescr), + actpdescr + }; + +BEGINbeginCnfLoad +CODESTARTbeginCnfLoad + loadModConf = pModConf; + pModConf->pConf = pConf; +ENDbeginCnfLoad + +BEGINendCnfLoad +CODESTARTendCnfLoad +ENDendCnfLoad + +BEGINcheckCnf +CODESTARTcheckCnf +ENDcheckCnf + +BEGINactivateCnf +CODESTARTactivateCnf + runModConf = pModConf; +ENDactivateCnf + +BEGINfreeCnf +CODESTARTfreeCnf +ENDfreeCnf + + +BEGINcreateInstance +CODESTARTcreateInstance +ENDcreateInstance + +BEGINcreateWrkrInstance +CODESTARTcreateWrkrInstance +ENDcreateWrkrInstance + + +BEGINisCompatibleWithFeature +CODESTARTisCompatibleWithFeature +ENDisCompatibleWithFeature + + +BEGINfreeInstance +CODESTARTfreeInstance + free(pData->jsonRoot); +ENDfreeInstance + +BEGINfreeWrkrInstance +CODESTARTfreeWrkrInstance +ENDfreeWrkrInstance + + +static inline void +setInstParamDefaults(instanceData *pData) +{ + pData->separator = ','; + pData->jsonRoot = NULL; +} + +BEGINnewActInst + struct cnfparamvals *pvals; + int i; +CODESTARTnewActInst + DBGPRINTF("newActInst (mmfields)\n"); + if((pvals = nvlstGetParams(lst, &actpblk, NULL)) == NULL) { + ABORT_FINALIZE(RS_RET_MISSING_CNFPARAMS); + } + + CODE_STD_STRING_REQUESTnewActInst(1) + CHKiRet(OMSRsetEntry(*ppOMSR, 0, NULL, OMSR_TPL_AS_MSG)); + CHKiRet(createInstance(&pData)); + setInstParamDefaults(pData); + + for(i = 0 ; i < actpblk.nParams ; ++i) { + if(!pvals[i].bUsed) + continue; + if(!strcmp(actpblk.descr[i].name, "separator")) { + pData->separator = es_getBufAddr(pvals[i].val.d.estr)[0]; + } else if(!strcmp(actpblk.descr[i].name, "jsonroot")) { + pData->jsonRoot = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + } else { + dbgprintf("mmfields: program error, non-handled " + "param '%s'\n", actpblk.descr[i].name); + } + } + if(pData->jsonRoot == NULL) { + CHKmalloc(pData->jsonRoot = (uchar*) strdup("!")); + } + +CODE_STD_FINALIZERnewActInst + cnfparamvalsDestruct(pvals, &actpblk); +ENDnewActInst + + +BEGINdbgPrintInstInfo +CODESTARTdbgPrintInstInfo +ENDdbgPrintInstInfo + + +BEGINtryResume +CODESTARTtryResume +ENDtryResume + + +static inline rsRetVal +extractField(instanceData *pData, uchar *msgtext, int lenMsg, int *curridx, uchar *fieldbuf) +{ + int i, j; + DEFiRet; + i = *curridx; + j = 0; + while(i < lenMsg && msgtext[i] != pData->separator) { + fieldbuf[j++] = msgtext[i++]; + } + fieldbuf[j] = '\0'; + if(i < lenMsg) + ++i; + *curridx = i; + + RETiRet; +} + + +static inline rsRetVal +parse_fields(instanceData *pData, msg_t *pMsg, uchar *msgtext, int lenMsg) +{ + uchar fieldbuf[32*1024]; + uchar fieldname[512]; + struct json_object *json; + struct json_object *jval; + int field; + uchar *buf; + int currIdx = 0; + DEFiRet; + + if(lenMsg < (int) sizeof(fieldbuf)) { + buf = fieldbuf; + } else { + CHKmalloc(buf = malloc(lenMsg+1)); + } + + json = json_object_new_object(); + if(json == NULL) { + ABORT_FINALIZE(RS_RET_ERR); + } + field = 1; + while(currIdx < lenMsg) { + CHKiRet(extractField(pData, msgtext, lenMsg, &currIdx, buf)); + DBGPRINTF("mmfields: field %d: '%s'\n", field, buf); + snprintf((char*)fieldname, sizeof(fieldname), "f%d", field); + fieldname[sizeof(fieldname)-1] = '\0'; + jval = json_object_new_string((char*)fieldbuf); + json_object_object_add(json, (char*)fieldname, jval); + field++; + } + msgAddJSON(pMsg, pData->jsonRoot, json); +finalize_it: + RETiRet; +} + + +BEGINdoAction + msg_t *pMsg; + uchar *msg; + int lenMsg; +CODESTARTdoAction + pMsg = (msg_t*) ppString[0]; + lenMsg = getMSGLen(pMsg); + msg = getMSG(pMsg); + CHKiRet(parse_fields(pWrkrData->pData, pMsg, msg, lenMsg)); +finalize_it: +ENDdoAction + + +BEGINparseSelectorAct +CODESTARTparseSelectorAct +CODE_STD_STRING_REQUESTparseSelectorAct(1) + if(strncmp((char*) p, ":mmfields:", sizeof(":mmfields:") - 1)) { + errmsg.LogError(0, RS_RET_LEGA_ACT_NOT_SUPPORTED, + "mmfields supports only v6+ config format, use: " + "action(type=\"mmfields\" ...)"); + } + ABORT_FINALIZE(RS_RET_CONFLINE_UNPROCESSED); +CODE_STD_FINALIZERparseSelectorAct +ENDparseSelectorAct + + +BEGINmodExit +CODESTARTmodExit + objRelease(errmsg, CORE_COMPONENT); +ENDmodExit + + +BEGINqueryEtryPt +CODESTARTqueryEtryPt +CODEqueryEtryPt_STD_OMOD_QUERIES +CODEqueryEtryPt_STD_OMOD8_QUERIES +CODEqueryEtryPt_STD_CONF2_OMOD_QUERIES +CODEqueryEtryPt_STD_CONF2_QUERIES +ENDqueryEtryPt + + + +BEGINmodInit() +CODESTARTmodInit + *ipIFVersProvided = CURR_MOD_IF_VERSION; /* we only support the current interface specification */ +CODEmodInit_QueryRegCFSLineHdlr + DBGPRINTF("mmfields: module compiled with rsyslog version %s.\n", VERSION); + CHKiRet(objUse(errmsg, CORE_COMPONENT)); +ENDmodInit diff --git a/plugins/mmjsonparse/Makefile.am b/plugins/mmjsonparse/Makefile.am index 5175fe8..ef39163 100644 --- a/plugins/mmjsonparse/Makefile.am +++ b/plugins/mmjsonparse/Makefile.am @@ -1,8 +1,8 @@ pkglib_LTLIBRARIES = mmjsonparse.la mmjsonparse_la_SOURCES = mmjsonparse.c -mmjsonparse_la_CPPFLAGS = $(RSRT_CFLAGS) $(PTHREADS_CFLAGS) $(LIBLOGNORM_CFLAGS) $(LIBEE_CFLAGS) -mmjsonparse_la_LDFLAGS = -module -avoid-version $(LIBLOGNORM_LIBS) $(LIBEE_LIBS) +mmjsonparse_la_CPPFLAGS = $(RSRT_CFLAGS) $(PTHREADS_CFLAGS) +mmjsonparse_la_LDFLAGS = -module -avoid-version mmjsonparse_la_LIBADD = EXTRA_DIST = diff --git a/plugins/mmjsonparse/Makefile.in b/plugins/mmjsonparse/Makefile.in index dbfd71a..1fe1189 100644 --- a/plugins/mmjsonparse/Makefile.in +++ b/plugins/mmjsonparse/Makefile.in @@ -154,7 +154,6 @@ GREP = @GREP@ GSS_LIBS = @GSS_LIBS@ GUARDTIME_CFLAGS = @GUARDTIME_CFLAGS@ GUARDTIME_LIBS = @GUARDTIME_LIBS@ -HAVE_LIBGCRYPT_CONFIG = @HAVE_LIBGCRYPT_CONFIG@ HAVE_MYSQL_CONFIG = @HAVE_MYSQL_CONFIG@ HAVE_ORACLE_CONFIG = @HAVE_ORACLE_CONFIG@ HAVE_PGSQL_CONFIG = @HAVE_PGSQL_CONFIG@ @@ -175,14 +174,15 @@ LEXLIB = @LEXLIB@ LEX_OUTPUT_ROOT = @LEX_OUTPUT_ROOT@ LIBDBI_CFLAGS = @LIBDBI_CFLAGS@ LIBDBI_LIBS = @LIBDBI_LIBS@ -LIBEE_CFLAGS = @LIBEE_CFLAGS@ -LIBEE_LIBS = @LIBEE_LIBS@ LIBESTR_CFLAGS = @LIBESTR_CFLAGS@ LIBESTR_LIBS = @LIBESTR_LIBS@ LIBGCRYPT_CFLAGS = @LIBGCRYPT_CFLAGS@ +LIBGCRYPT_CONFIG = @LIBGCRYPT_CONFIG@ LIBGCRYPT_LIBS = @LIBGCRYPT_LIBS@ LIBLOGGING_CFLAGS = @LIBLOGGING_CFLAGS@ LIBLOGGING_LIBS = @LIBLOGGING_LIBS@ +LIBLOGGING_STDLOG_CFLAGS = @LIBLOGGING_STDLOG_CFLAGS@ +LIBLOGGING_STDLOG_LIBS = @LIBLOGGING_STDLOG_LIBS@ LIBLOGNORM_CFLAGS = @LIBLOGNORM_CFLAGS@ LIBLOGNORM_LIBS = @LIBLOGNORM_LIBS@ LIBM = @LIBM@ @@ -207,6 +207,8 @@ NM = @NM@ NMEDIT = @NMEDIT@ OBJDUMP = @OBJDUMP@ OBJEXT = @OBJEXT@ +OPENSSL_CFLAGS = @OPENSSL_CFLAGS@ +OPENSSL_LIBS = @OPENSSL_LIBS@ ORACLE_CFLAGS = @ORACLE_CFLAGS@ ORACLE_LIBS = @ORACLE_LIBS@ OTOOL = @OTOOL@ @@ -307,8 +309,8 @@ top_builddir = @top_builddir@ top_srcdir = @top_srcdir@ pkglib_LTLIBRARIES = mmjsonparse.la mmjsonparse_la_SOURCES = mmjsonparse.c -mmjsonparse_la_CPPFLAGS = $(RSRT_CFLAGS) $(PTHREADS_CFLAGS) $(LIBLOGNORM_CFLAGS) $(LIBEE_CFLAGS) -mmjsonparse_la_LDFLAGS = -module -avoid-version $(LIBLOGNORM_LIBS) $(LIBEE_LIBS) +mmjsonparse_la_CPPFLAGS = $(RSRT_CFLAGS) $(PTHREADS_CFLAGS) +mmjsonparse_la_LDFLAGS = -module -avoid-version mmjsonparse_la_LIBADD = EXTRA_DIST = all: all-am @@ -388,22 +390,25 @@ distclean-compile: @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/mmjsonparse_la-mmjsonparse.Plo@am__quote@ .c.o: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(COMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ $< -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Po +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.o$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c $< +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ $< .c.obj: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(COMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ `$(CYGPATH_W) '$<'` -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Po +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.obj$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ `$(CYGPATH_W) '$<'` &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c `$(CYGPATH_W) '$<'` +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ `$(CYGPATH_W) '$<'` .c.lo: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(LTCOMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ $< -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Plo +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.lo$$||'`;\ +@am__fastdepCC_TRUE@ $(LTCOMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Plo @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=yes @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ @am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(LTCOMPILE) -c -o $@ $< diff --git a/plugins/mmjsonparse/mmjsonparse.c b/plugins/mmjsonparse/mmjsonparse.c index c47aceb..ba942f8 100644 --- a/plugins/mmjsonparse/mmjsonparse.c +++ b/plugins/mmjsonparse/mmjsonparse.c @@ -35,8 +35,7 @@ #include <errno.h> #include <unistd.h> #include <ctype.h> -#include <libestr.h> -#include <json/json.h> +#include <json.h> #include "conf.h" #include "syslogd-types.h" #include "template.h" @@ -59,15 +58,36 @@ DEFobjCurrIf(errmsg); DEF_OMOD_STATIC_DATA typedef struct _instanceData { - struct json_tokener *tokener; + char *cookie; + uchar *container; + int lenCookie; + /* REMOVE dummy when real data items are to be added! */ } instanceData; +typedef struct wrkrInstanceData { + instanceData *pData; + struct json_tokener *tokener; +} wrkrInstanceData_t; + struct modConfData_s { rsconf_t *pConf; /* our overall config object */ }; static modConfData_t *loadModConf = NULL;/* modConf ptr to use for the current load process */ static modConfData_t *runModConf = NULL;/* modConf ptr to use for the current exec process */ +/* tables for interfacing with the v6 config system */ +/* action (instance) parameters */ +static struct cnfparamdescr actpdescr[] = { + { "cookie", eCmdHdlrString, 0 }, + { "container", eCmdHdlrString, 0 } +}; +static struct cnfparamblk actpblk = + { CNFPARAMBLK_VERSION, + sizeof(actpdescr)/sizeof(struct cnfparamdescr), + actpdescr + }; + + BEGINbeginCnfLoad CODESTARTbeginCnfLoad @@ -95,14 +115,22 @@ ENDfreeCnf BEGINcreateInstance CODESTARTcreateInstance - pData->tokener = json_tokener_new(); - if(pData->tokener == NULL) { + CHKmalloc(pData->container = (uchar*)strdup("!")); + CHKmalloc(pData->cookie = strdup("@cee:")); + pData->lenCookie = strlen(pData->cookie); +finalize_it: +ENDcreateInstance + +BEGINcreateWrkrInstance +CODESTARTcreateWrkrInstance + pWrkrData->tokener = json_tokener_new(); + if(pWrkrData->tokener == NULL) { errmsg.LogError(0, RS_RET_ERR, "error: could not create json " - "tokener, cannot activate action"); + "tokener, cannot activate instance"); ABORT_FINALIZE(RS_RET_ERR); } finalize_it: -ENDcreateInstance +ENDcreateWrkrInstance BEGINisCompatibleWithFeature @@ -112,10 +140,16 @@ ENDisCompatibleWithFeature BEGINfreeInstance CODESTARTfreeInstance - if(pData->tokener != NULL) - json_tokener_free(pData->tokener); + free(pData->cookie); + free(pData->container); ENDfreeInstance +BEGINfreeWrkrInstance +CODESTARTfreeWrkrInstance + if(pWrkrData->tokener != NULL) + json_tokener_free(pWrkrData->tokener); +ENDfreeWrkrInstance + BEGINdbgPrintInstInfo CODESTARTdbgPrintInstInfo @@ -129,28 +163,32 @@ ENDtryResume static rsRetVal -processJSON(instanceData *pData, msg_t *pMsg, char *buf, size_t lenBuf) +processJSON(wrkrInstanceData_t *pWrkrData, msg_t *pMsg, char *buf, size_t lenBuf) { struct json_object *json; const char *errMsg; DEFiRet; - assert(pData->tokener != NULL); + assert(pWrkrData->tokener != NULL); DBGPRINTF("mmjsonparse: toParse: '%s'\n", buf); - json_tokener_reset(pData->tokener); + json_tokener_reset(pWrkrData->tokener); - json = json_tokener_parse_ex(pData->tokener, buf, lenBuf); + json = json_tokener_parse_ex(pWrkrData->tokener, buf, lenBuf); if(Debug) { errMsg = NULL; if(json == NULL) { enum json_tokener_error err; - err = pData->tokener->err; + err = pWrkrData->tokener->err; if(err != json_tokener_continue) - errMsg = json_tokener_errors[err]; +# if HAVE_JSON_TOKENER_ERROR_DESC + errMsg = json_tokener_error_desc(err); +# else + errMsg = json_tokener_errors[err]; +# endif else errMsg = "Unterminated input"; - } else if((size_t)pData->tokener->char_offset < lenBuf) + } else if((size_t)pWrkrData->tokener->char_offset < lenBuf) errMsg = "Extra characters after JSON object"; else if(!json_object_is_type(json, json_type_object)) errMsg = "JSON value is not an object"; @@ -160,25 +198,25 @@ processJSON(instanceData *pData, msg_t *pMsg, char *buf, size_t lenBuf) } } if(json == NULL - || ((size_t)pData->tokener->char_offset < lenBuf) + || ((size_t)pWrkrData->tokener->char_offset < lenBuf) || (!json_object_is_type(json, json_type_object))) { ABORT_FINALIZE(RS_RET_NO_CEE_MSG); } - msgAddJSON(pMsg, (uchar*)"!", json); + msgAddJSON(pMsg, pWrkrData->pData->container, json); finalize_it: RETiRet; } -#define COOKIE "@cee:" -#define LEN_COOKIE (sizeof(COOKIE)-1) BEGINdoAction msg_t *pMsg; uchar *buf; int bSuccess = 0; struct json_object *jval; struct json_object *json; + instanceData *pData; CODESTARTdoAction + pData = pWrkrData->pData; pMsg = (msg_t*) ppString[0]; /* note that we can performance-optimize the interface, but this also * requires changes to the libraries. For now, we accept message @@ -190,12 +228,12 @@ CODESTARTdoAction ++buf; } - if(*buf == '\0' || strncmp((char*)buf, COOKIE, LEN_COOKIE)) { + if(*buf == '\0' || strncmp((char*)buf, pData->cookie, pData->lenCookie)) { DBGPRINTF("mmjsonparse: no JSON cookie: '%s'\n", buf); ABORT_FINALIZE(RS_RET_NO_CEE_MSG); } - buf += LEN_COOKIE; - CHKiRet(processJSON(pData, pMsg, (char*) buf, strlen((char*)buf))); + buf += pData->lenCookie; + CHKiRet(processJSON(pWrkrData, pMsg, (char*) buf, strlen((char*)buf))); bSuccess = 1; finalize_it: if(iRet == RS_RET_NO_CEE_MSG) { @@ -203,27 +241,54 @@ finalize_it: json = json_object_new_object(); jval = json_object_new_string((char*)buf); json_object_object_add(json, "msg", jval); - msgAddJSON(pMsg, (uchar*)"!", json); + msgAddJSON(pMsg, pData->container, json); iRet = RS_RET_OK; } MsgSetParseSuccess(pMsg, bSuccess); ENDdoAction +static inline void +setInstParamDefaults(instanceData *pData) +{ + pData->cookie = NULL; +} + BEGINnewActInst + struct cnfparamvals *pvals; + int i; CODESTARTnewActInst - /* Note: we currently do not have any parameters, so we do not need - * the lst ptr. However, we will most probably need params in the - * future. - */ DBGPRINTF("newActInst (mmjsonparse)\n"); + if((pvals = nvlstGetParams(lst, &actpblk, NULL)) == NULL) { + ABORT_FINALIZE(RS_RET_MISSING_CNFPARAMS); + } + CODE_STD_STRING_REQUESTnewActInst(1) CHKiRet(OMSRsetEntry(*ppOMSR, 0, NULL, OMSR_TPL_AS_MSG)); CHKiRet(createInstance(&pData)); - /*setInstParamDefaults(pData);*/ + setInstParamDefaults(pData); + + for(i = 0 ; i < actpblk.nParams ; ++i) { + if(!pvals[i].bUsed) + continue; + if(!strcmp(actpblk.descr[i].name, "cookie")) { + free(pData->cookie); + pData->cookie = es_str2cstr(pvals[i].val.d.estr, NULL); + } else if(!strcmp(actpblk.descr[i].name, "container")) { + free(pData->container); + pData->container = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + } else { + dbgprintf("mmjsonparse: program error, non-handled param '%s'\n", actpblk.descr[i].name); + } + } + if(pData->container == NULL) + CHKmalloc(pData->container = (uchar*) strdup("!")); + if(pData->cookie == NULL) + CHKmalloc(pData->cookie = strdup("@cee:")); + pData->lenCookie = strlen(pData->cookie); CODE_STD_FINALIZERnewActInst -/* cnfparamvalsDestruct(pvals, &actpblk);*/ + cnfparamvalsDestruct(pvals, &actpblk); ENDnewActInst BEGINparseSelectorAct @@ -258,6 +323,7 @@ ENDmodExit BEGINqueryEtryPt CODESTARTqueryEtryPt CODEqueryEtryPt_STD_OMOD_QUERIES +CODEqueryEtryPt_STD_OMOD8_QUERIES CODEqueryEtryPt_STD_CONF2_OMOD_QUERIES CODEqueryEtryPt_STD_CONF2_QUERIES ENDqueryEtryPt diff --git a/plugins/mmnormalize/Makefile.am b/plugins/mmnormalize/Makefile.am index 0a3b5ba..6a50264 100644 --- a/plugins/mmnormalize/Makefile.am +++ b/plugins/mmnormalize/Makefile.am @@ -1,8 +1,8 @@ pkglib_LTLIBRARIES = mmnormalize.la mmnormalize_la_SOURCES = mmnormalize.c -mmnormalize_la_CPPFLAGS = $(RSRT_CFLAGS) $(PTHREADS_CFLAGS) $(LIBLOGNORM_CFLAGS) $(LIBEE_CFLAGS) -mmnormalize_la_LDFLAGS = -module -avoid-version $(LIBLOGNORM_LIBS) $(LIBEE_LIBS) +mmnormalize_la_CPPFLAGS = $(RSRT_CFLAGS) $(PTHREADS_CFLAGS) $(LIBLOGNORM_CFLAGS) +mmnormalize_la_LDFLAGS = -module -avoid-version $(LIBLOGNORM_LIBS) mmnormalize_la_LIBADD = EXTRA_DIST = diff --git a/plugins/mmnormalize/Makefile.in b/plugins/mmnormalize/Makefile.in index 521e60b..4d3c7ea 100644 --- a/plugins/mmnormalize/Makefile.in +++ b/plugins/mmnormalize/Makefile.in @@ -154,7 +154,6 @@ GREP = @GREP@ GSS_LIBS = @GSS_LIBS@ GUARDTIME_CFLAGS = @GUARDTIME_CFLAGS@ GUARDTIME_LIBS = @GUARDTIME_LIBS@ -HAVE_LIBGCRYPT_CONFIG = @HAVE_LIBGCRYPT_CONFIG@ HAVE_MYSQL_CONFIG = @HAVE_MYSQL_CONFIG@ HAVE_ORACLE_CONFIG = @HAVE_ORACLE_CONFIG@ HAVE_PGSQL_CONFIG = @HAVE_PGSQL_CONFIG@ @@ -175,14 +174,15 @@ LEXLIB = @LEXLIB@ LEX_OUTPUT_ROOT = @LEX_OUTPUT_ROOT@ LIBDBI_CFLAGS = @LIBDBI_CFLAGS@ LIBDBI_LIBS = @LIBDBI_LIBS@ -LIBEE_CFLAGS = @LIBEE_CFLAGS@ -LIBEE_LIBS = @LIBEE_LIBS@ LIBESTR_CFLAGS = @LIBESTR_CFLAGS@ LIBESTR_LIBS = @LIBESTR_LIBS@ LIBGCRYPT_CFLAGS = @LIBGCRYPT_CFLAGS@ +LIBGCRYPT_CONFIG = @LIBGCRYPT_CONFIG@ LIBGCRYPT_LIBS = @LIBGCRYPT_LIBS@ LIBLOGGING_CFLAGS = @LIBLOGGING_CFLAGS@ LIBLOGGING_LIBS = @LIBLOGGING_LIBS@ +LIBLOGGING_STDLOG_CFLAGS = @LIBLOGGING_STDLOG_CFLAGS@ +LIBLOGGING_STDLOG_LIBS = @LIBLOGGING_STDLOG_LIBS@ LIBLOGNORM_CFLAGS = @LIBLOGNORM_CFLAGS@ LIBLOGNORM_LIBS = @LIBLOGNORM_LIBS@ LIBM = @LIBM@ @@ -207,6 +207,8 @@ NM = @NM@ NMEDIT = @NMEDIT@ OBJDUMP = @OBJDUMP@ OBJEXT = @OBJEXT@ +OPENSSL_CFLAGS = @OPENSSL_CFLAGS@ +OPENSSL_LIBS = @OPENSSL_LIBS@ ORACLE_CFLAGS = @ORACLE_CFLAGS@ ORACLE_LIBS = @ORACLE_LIBS@ OTOOL = @OTOOL@ @@ -307,8 +309,8 @@ top_builddir = @top_builddir@ top_srcdir = @top_srcdir@ pkglib_LTLIBRARIES = mmnormalize.la mmnormalize_la_SOURCES = mmnormalize.c -mmnormalize_la_CPPFLAGS = $(RSRT_CFLAGS) $(PTHREADS_CFLAGS) $(LIBLOGNORM_CFLAGS) $(LIBEE_CFLAGS) -mmnormalize_la_LDFLAGS = -module -avoid-version $(LIBLOGNORM_LIBS) $(LIBEE_LIBS) +mmnormalize_la_CPPFLAGS = $(RSRT_CFLAGS) $(PTHREADS_CFLAGS) $(LIBLOGNORM_CFLAGS) +mmnormalize_la_LDFLAGS = -module -avoid-version $(LIBLOGNORM_LIBS) mmnormalize_la_LIBADD = EXTRA_DIST = all: all-am @@ -388,22 +390,25 @@ distclean-compile: @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/mmnormalize_la-mmnormalize.Plo@am__quote@ .c.o: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(COMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ $< -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Po +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.o$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c $< +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ $< .c.obj: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(COMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ `$(CYGPATH_W) '$<'` -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Po +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.obj$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ `$(CYGPATH_W) '$<'` &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c `$(CYGPATH_W) '$<'` +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ `$(CYGPATH_W) '$<'` .c.lo: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(LTCOMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ $< -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Plo +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.lo$$||'`;\ +@am__fastdepCC_TRUE@ $(LTCOMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Plo @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=yes @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ @am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(LTCOMPILE) -c -o $@ $< diff --git a/plugins/mmnormalize/mmnormalize.c b/plugins/mmnormalize/mmnormalize.c index fcadc32..ba2e730 100644 --- a/plugins/mmnormalize/mmnormalize.c +++ b/plugins/mmnormalize/mmnormalize.c @@ -1,15 +1,12 @@ /* mmnormalize.c * This is a message modification module. It normalizes the input message with - * the help of liblognorm. The messages EE event structure is updated. + * the help of liblognorm. The message's JSON variables are updated. * * NOTE: read comments in module-template.h for details on the calling interface! * - * TODO: check if we can replace libee via JSON system - currently that part - * is pretty inefficient... rgerhards, 2012-08-27 - * * File begun on 2010-01-01 by RGerhards * - * Copyright 2010-2012 Rainer Gerhards and Adiscon GmbH. + * Copyright 2010-2013 Rainer Gerhards and Adiscon GmbH. * * This file is part of rsyslog. * @@ -39,8 +36,7 @@ #include <errno.h> #include <unistd.h> #include <libestr.h> -#include <libee/libee.h> -#include <json/json.h> +#include <json.h> #include <liblognorm.h> #include "conf.h" #include "syslogd-types.h" @@ -67,9 +63,13 @@ typedef struct _instanceData { sbool bUseRawMsg; /**< use %rawmsg% instead of %msg% */ uchar *rulebase; /**< name of rulebase to use */ ln_ctx ctxln; /**< context to be used for liblognorm */ - ee_ctx ctxee; /**< context to be used for libee */ + char *pszPath; /**< path of normalized data */ } instanceData; +typedef struct wrkrInstanceData { + instanceData *pData; +} wrkrInstanceData_t; + typedef struct configSettings_s { uchar *rulebase; /**< name of normalization rulebase to use */ int bUseRawMsg; /**< use %rawmsg% instead of %msg% */ @@ -80,6 +80,7 @@ static configSettings_t cs; /* action (instance) parameters */ static struct cnfparamdescr actpdescr[] = { { "rulebase", eCmdHdlrGetWord, 1 }, + { "path", eCmdHdlrGetWord, 0 }, { "userawmsg", eCmdHdlrBinary, 0 } }; static struct cnfparamblk actpblk = @@ -96,30 +97,21 @@ static modConfData_t *loadModConf = NULL;/* modConf ptr to use for the current l static modConfData_t *runModConf = NULL;/* modConf ptr to use for the current exec process */ -/* to be called to build the libee part of the instance ONCE ALL PARAMETERS ARE CORRECT +/* to be called to build the liblognorm part of the instance ONCE ALL PARAMETERS ARE CORRECT * (and set within pData!). */ static rsRetVal buildInstance(instanceData *pData) { DEFiRet; - if((pData->ctxee = ee_initCtx()) == NULL) { - errmsg.LogError(0, RS_RET_ERR_LIBEE_INIT, "error: could not initialize libee " - "ctx, cannot activate action"); - ABORT_FINALIZE(RS_RET_ERR_LIBEE_INIT); - } - if((pData->ctxln = ln_initCtx()) == NULL) { errmsg.LogError(0, RS_RET_ERR_LIBLOGNORM_INIT, "error: could not initialize " "liblognorm ctx, cannot activate action"); - ee_exitCtx(pData->ctxee); ABORT_FINALIZE(RS_RET_ERR_LIBLOGNORM_INIT); } - ln_setEECtx(pData->ctxln, pData->ctxee); if(ln_loadSamples(pData->ctxln, (char*) pData->rulebase) != 0) { errmsg.LogError(0, RS_RET_NO_RULEBASE, "error: normalization rulebase '%s' " - "could not be loaded cannot activate action", cs.rulebase); - ee_exitCtx(pData->ctxee); + "could not be loaded cannot activate action", pData->rulebase); ln_exitCtx(pData->ctxln); ABORT_FINALIZE(RS_RET_ERR_LIBLOGNORM_SAMPDB_LOAD); } @@ -139,6 +131,11 @@ CODESTARTcreateInstance ENDcreateInstance +BEGINcreateWrkrInstance +CODESTARTcreateWrkrInstance +ENDcreateWrkrInstance + + BEGINbeginCnfLoad CODESTARTbeginCnfLoad loadModConf = pModConf; @@ -176,11 +173,16 @@ ENDisCompatibleWithFeature BEGINfreeInstance CODESTARTfreeInstance free(pData->rulebase); - ee_exitCtx(pData->ctxee); ln_exitCtx(pData->ctxln); + free(pData->pszPath); ENDfreeInstance +BEGINfreeWrkrInstance +CODESTARTfreeWrkrInstance +ENDfreeWrkrInstance + + BEGINdbgPrintInstInfo CODESTARTdbgPrintInstInfo dbgprintf("mmnormalize\n"); @@ -193,49 +195,28 @@ ENDtryResume BEGINdoAction msg_t *pMsg; - es_str_t *str; uchar *buf; - char *cstrJSON; int len; int r; - struct ee_event *event = NULL; - struct json_tokener *tokener; - struct json_object *json; + struct json_object *json = NULL; CODESTARTdoAction pMsg = (msg_t*) ppString[0]; - /* note that we can performance-optimize the interface, but this also - * requires changes to the libraries. For now, we accept message - * duplication. -- rgerhards, 2010-12-01 - */ - if(pData->bUseRawMsg) { + if(pWrkrData->pData->bUseRawMsg) { getRawMsg(pMsg, &buf, &len); } else { buf = getMSG(pMsg); len = getMSGLen(pMsg); } - str = es_newStrFromCStr((char*)buf, len); - r = ln_normalize(pData->ctxln, str, &event); + r = ln_normalize(pWrkrData->pData->ctxln, (char*)buf, len, &json); if(r != 0) { DBGPRINTF("error %d during ln_normalize\n", r); MsgSetParseSuccess(pMsg, 0); } else { MsgSetParseSuccess(pMsg, 1); } - es_deleteStr(str); - /* reformat to our json data struct */ - /* TODO: this is all extremly ineffcient! */ - ee_fmtEventToJSON(event, &str); - cstrJSON = es_str2cstr(str, NULL); - dbgprintf("mmnormalize generated: %s\n", cstrJSON); + msgAddJSON(pMsg, (uchar*)pWrkrData->pData->pszPath + 1, json); - tokener = json_tokener_new(); - json = json_tokener_parse_ex(tokener, cstrJSON, strlen((char*)cstrJSON)); - json_tokener_free(tokener); - msgAddJSON(pMsg, (uchar*)"!", json); - - free(cstrJSON); - es_deleteStr(str); ENDdoAction @@ -244,12 +225,14 @@ setInstParamDefaults(instanceData *pData) { pData->rulebase = NULL; pData->bUseRawMsg = 0; + pData->pszPath = strdup("$!"); } BEGINnewActInst struct cnfparamvals *pvals; int i; int bDestructPValsOnExit; + char *cstr; CODESTARTnewActInst DBGPRINTF("newActInst (mmnormalize)\n"); @@ -277,6 +260,23 @@ CODESTARTnewActInst pData->rulebase = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); } else if(!strcmp(actpblk.descr[i].name, "userawmsg")) { pData->bUseRawMsg = (int) pvals[i].val.d.n; + } else if(!strcmp(actpblk.descr[i].name, "path")) { + cstr = es_str2cstr(pvals[i].val.d.estr, NULL); + if (strlen(cstr) < 2) { + errmsg.LogError(0, RS_RET_VALUE_NOT_SUPPORTED, + "mmnormalize: valid path name should be at least " + "2 symbols long, got %s", cstr); + free(cstr); + } else if (cstr[0] != '$') { + errmsg.LogError(0, RS_RET_VALUE_NOT_SUPPORTED, + "mmnormalize: valid path name should start with $," + "got %s", cstr); + free(cstr); + } else { + free(pData->pszPath); + pData->pszPath = cstr; + } + continue; } else { DBGPRINTF("mmnormalize: program error, non-handled " "param '%s'\n", actpblk.descr[i].name); @@ -312,6 +312,7 @@ CODE_STD_STRING_REQUESTparseSelectorAct(1) pData->rulebase = cs.rulebase; pData->bUseRawMsg = cs.bUseRawMsg; + pData->pszPath = strdup("$!"); /* old interface does not support this feature */ /* all config vars auto-reset! */ cs.bUseRawMsg = 0; cs.rulebase = NULL; /* we used it up! */ @@ -337,6 +338,7 @@ ENDmodExit BEGINqueryEtryPt CODESTARTqueryEtryPt CODEqueryEtryPt_STD_OMOD_QUERIES +CODEqueryEtryPt_STD_OMOD8_QUERIES CODEqueryEtryPt_STD_CONF2_QUERIES CODEqueryEtryPt_STD_CONF2_OMOD_QUERIES ENDqueryEtryPt diff --git a/plugins/mmpstrucdata/Makefile.am b/plugins/mmpstrucdata/Makefile.am new file mode 100644 index 0000000..090150a --- /dev/null +++ b/plugins/mmpstrucdata/Makefile.am @@ -0,0 +1,8 @@ +pkglib_LTLIBRARIES = mmpstrucdata.la + +mmpstrucdata_la_SOURCES = mmpstrucdata.c +mmpstrucdata_la_CPPFLAGS = $(RSRT_CFLAGS) $(PTHREADS_CFLAGS) +mmpstrucdata_la_LDFLAGS = -module -avoid-version +mmpstrucdata_la_LIBADD = + +EXTRA_DIST = diff --git a/plugins/mmpstrucdata/Makefile.in b/plugins/mmpstrucdata/Makefile.in new file mode 100644 index 0000000..350e1bf --- /dev/null +++ b/plugins/mmpstrucdata/Makefile.in @@ -0,0 +1,638 @@ +# Makefile.in generated by automake 1.11.3 from Makefile.am. +# @configure_input@ + +# Copyright (C) 1994, 1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, +# 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010, 2011 Free Software +# Foundation, Inc. +# This Makefile.in is free software; the Free Software Foundation +# gives unlimited permission to copy and/or distribute it, +# with or without modifications, as long as this notice is preserved. + +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY, to the extent permitted by law; without +# even the implied warranty of MERCHANTABILITY or FITNESS FOR A +# PARTICULAR PURPOSE. + +@SET_MAKE@ + +VPATH = @srcdir@ +pkgdatadir = $(datadir)/@PACKAGE@ +pkgincludedir = $(includedir)/@PACKAGE@ +pkglibdir = $(libdir)/@PACKAGE@ +pkglibexecdir = $(libexecdir)/@PACKAGE@ +am__cd = CDPATH="$${ZSH_VERSION+.}$(PATH_SEPARATOR)" && cd +install_sh_DATA = $(install_sh) -c -m 644 +install_sh_PROGRAM = $(install_sh) -c +install_sh_SCRIPT = $(install_sh) -c +INSTALL_HEADER = $(INSTALL_DATA) +transform = $(program_transform_name) +NORMAL_INSTALL = : +PRE_INSTALL = : +POST_INSTALL = : +NORMAL_UNINSTALL = : +PRE_UNINSTALL = : +POST_UNINSTALL = : +build_triplet = @build@ +host_triplet = @host@ +subdir = plugins/mmpstrucdata +DIST_COMMON = $(srcdir)/Makefile.am $(srcdir)/Makefile.in +ACLOCAL_M4 = $(top_srcdir)/aclocal.m4 +am__aclocal_m4_deps = $(top_srcdir)/m4/atomic_operations.m4 \ + $(top_srcdir)/m4/atomic_operations_64bit.m4 \ + $(top_srcdir)/m4/libtool.m4 $(top_srcdir)/m4/ltoptions.m4 \ + $(top_srcdir)/m4/ltsugar.m4 $(top_srcdir)/m4/ltversion.m4 \ + $(top_srcdir)/m4/lt~obsolete.m4 $(top_srcdir)/configure.ac +am__configure_deps = $(am__aclocal_m4_deps) $(CONFIGURE_DEPENDENCIES) \ + $(ACLOCAL_M4) +mkinstalldirs = $(install_sh) -d +CONFIG_HEADER = $(top_builddir)/config.h +CONFIG_CLEAN_FILES = +CONFIG_CLEAN_VPATH_FILES = +am__vpath_adj_setup = srcdirstrip=`echo "$(srcdir)" | sed 's|.|.|g'`; +am__vpath_adj = case $$p in \ + $(srcdir)/*) f=`echo "$$p" | sed "s|^$$srcdirstrip/||"`;; \ + *) f=$$p;; \ + esac; +am__strip_dir = f=`echo $$p | sed -e 's|^.*/||'`; +am__install_max = 40 +am__nobase_strip_setup = \ + srcdirstrip=`echo "$(srcdir)" | sed 's/[].[^$$\\*|]/\\\\&/g'` +am__nobase_strip = \ + for p in $$list; do echo "$$p"; done | sed -e "s|$$srcdirstrip/||" +am__nobase_list = $(am__nobase_strip_setup); \ + for p in $$list; do echo "$$p $$p"; done | \ + sed "s| $$srcdirstrip/| |;"' / .*\//!s/ .*/ ./; s,\( .*\)/[^/]*$$,\1,' | \ + $(AWK) 'BEGIN { files["."] = "" } { files[$$2] = files[$$2] " " $$1; \ + if (++n[$$2] == $(am__install_max)) \ + { print $$2, files[$$2]; n[$$2] = 0; files[$$2] = "" } } \ + END { for (dir in files) print dir, files[dir] }' +am__base_list = \ + sed '$$!N;$$!N;$$!N;$$!N;$$!N;$$!N;$$!N;s/\n/ /g' | \ + sed '$$!N;$$!N;$$!N;$$!N;s/\n/ /g' +am__uninstall_files_from_dir = { \ + test -z "$$files" \ + || { test ! -d "$$dir" && test ! -f "$$dir" && test ! -r "$$dir"; } \ + || { echo " ( cd '$$dir' && rm -f" $$files ")"; \ + $(am__cd) "$$dir" && rm -f $$files; }; \ + } +am__installdirs = "$(DESTDIR)$(pkglibdir)" +LTLIBRARIES = $(pkglib_LTLIBRARIES) +mmpstrucdata_la_DEPENDENCIES = +am_mmpstrucdata_la_OBJECTS = mmpstrucdata_la-mmpstrucdata.lo +mmpstrucdata_la_OBJECTS = $(am_mmpstrucdata_la_OBJECTS) +AM_V_lt = $(am__v_lt_@AM_V@) +am__v_lt_ = $(am__v_lt_@AM_DEFAULT_V@) +am__v_lt_0 = --silent +mmpstrucdata_la_LINK = $(LIBTOOL) $(AM_V_lt) --tag=CC \ + $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=link $(CCLD) \ + $(AM_CFLAGS) $(CFLAGS) $(mmpstrucdata_la_LDFLAGS) $(LDFLAGS) \ + -o $@ +DEFAULT_INCLUDES = -I.@am__isrc@ -I$(top_builddir) +depcomp = $(SHELL) $(top_srcdir)/depcomp +am__depfiles_maybe = depfiles +am__mv = mv -f +COMPILE = $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(AM_CPPFLAGS) \ + $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) +LTCOMPILE = $(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \ + $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) \ + $(DEFAULT_INCLUDES) $(INCLUDES) $(AM_CPPFLAGS) $(CPPFLAGS) \ + $(AM_CFLAGS) $(CFLAGS) +AM_V_CC = $(am__v_CC_@AM_V@) +am__v_CC_ = $(am__v_CC_@AM_DEFAULT_V@) +am__v_CC_0 = @echo " CC " $@; +AM_V_at = $(am__v_at_@AM_V@) +am__v_at_ = $(am__v_at_@AM_DEFAULT_V@) +am__v_at_0 = @ +CCLD = $(CC) +LINK = $(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \ + $(LIBTOOLFLAGS) --mode=link $(CCLD) $(AM_CFLAGS) $(CFLAGS) \ + $(AM_LDFLAGS) $(LDFLAGS) -o $@ +AM_V_CCLD = $(am__v_CCLD_@AM_V@) +am__v_CCLD_ = $(am__v_CCLD_@AM_DEFAULT_V@) +am__v_CCLD_0 = @echo " CCLD " $@; +AM_V_GEN = $(am__v_GEN_@AM_V@) +am__v_GEN_ = $(am__v_GEN_@AM_DEFAULT_V@) +am__v_GEN_0 = @echo " GEN " $@; +SOURCES = $(mmpstrucdata_la_SOURCES) +DIST_SOURCES = $(mmpstrucdata_la_SOURCES) +ETAGS = etags +CTAGS = ctags +DISTFILES = $(DIST_COMMON) $(DIST_SOURCES) $(TEXINFOS) $(EXTRA_DIST) +ACLOCAL = @ACLOCAL@ +AMTAR = @AMTAR@ +AM_DEFAULT_VERBOSITY = @AM_DEFAULT_VERBOSITY@ +AR = @AR@ +AUTOCONF = @AUTOCONF@ +AUTOHEADER = @AUTOHEADER@ +AUTOMAKE = @AUTOMAKE@ +AWK = @AWK@ +CC = @CC@ +CCDEPMODE = @CCDEPMODE@ +CFLAGS = @CFLAGS@ +CPP = @CPP@ +CPPFLAGS = @CPPFLAGS@ +CURL_CFLAGS = @CURL_CFLAGS@ +CURL_LIBS = @CURL_LIBS@ +CYGPATH_W = @CYGPATH_W@ +CZMQ_CFLAGS = @CZMQ_CFLAGS@ +CZMQ_LIBS = @CZMQ_LIBS@ +DEFS = @DEFS@ +DEPDIR = @DEPDIR@ +DLLTOOL = @DLLTOOL@ +DL_LIBS = @DL_LIBS@ +DSYMUTIL = @DSYMUTIL@ +DUMPBIN = @DUMPBIN@ +ECHO_C = @ECHO_C@ +ECHO_N = @ECHO_N@ +ECHO_T = @ECHO_T@ +EGREP = @EGREP@ +EXEEXT = @EXEEXT@ +FGREP = @FGREP@ +GNUTLS_CFLAGS = @GNUTLS_CFLAGS@ +GNUTLS_LIBS = @GNUTLS_LIBS@ +GREP = @GREP@ +GSS_LIBS = @GSS_LIBS@ +GUARDTIME_CFLAGS = @GUARDTIME_CFLAGS@ +GUARDTIME_LIBS = @GUARDTIME_LIBS@ +HAVE_MYSQL_CONFIG = @HAVE_MYSQL_CONFIG@ +HAVE_ORACLE_CONFIG = @HAVE_ORACLE_CONFIG@ +HAVE_PGSQL_CONFIG = @HAVE_PGSQL_CONFIG@ +HIREDIS_CFLAGS = @HIREDIS_CFLAGS@ +HIREDIS_LIBS = @HIREDIS_LIBS@ +IMUDP_LIBS = @IMUDP_LIBS@ +INSTALL = @INSTALL@ +INSTALL_DATA = @INSTALL_DATA@ +INSTALL_PROGRAM = @INSTALL_PROGRAM@ +INSTALL_SCRIPT = @INSTALL_SCRIPT@ +INSTALL_STRIP_PROGRAM = @INSTALL_STRIP_PROGRAM@ +JSON_C_CFLAGS = @JSON_C_CFLAGS@ +JSON_C_LIBS = @JSON_C_LIBS@ +LD = @LD@ +LDFLAGS = @LDFLAGS@ +LEX = @LEX@ +LEXLIB = @LEXLIB@ +LEX_OUTPUT_ROOT = @LEX_OUTPUT_ROOT@ +LIBDBI_CFLAGS = @LIBDBI_CFLAGS@ +LIBDBI_LIBS = @LIBDBI_LIBS@ +LIBESTR_CFLAGS = @LIBESTR_CFLAGS@ +LIBESTR_LIBS = @LIBESTR_LIBS@ +LIBGCRYPT_CFLAGS = @LIBGCRYPT_CFLAGS@ +LIBGCRYPT_CONFIG = @LIBGCRYPT_CONFIG@ +LIBGCRYPT_LIBS = @LIBGCRYPT_LIBS@ +LIBLOGGING_CFLAGS = @LIBLOGGING_CFLAGS@ +LIBLOGGING_LIBS = @LIBLOGGING_LIBS@ +LIBLOGGING_STDLOG_CFLAGS = @LIBLOGGING_STDLOG_CFLAGS@ +LIBLOGGING_STDLOG_LIBS = @LIBLOGGING_STDLOG_LIBS@ +LIBLOGNORM_CFLAGS = @LIBLOGNORM_CFLAGS@ +LIBLOGNORM_LIBS = @LIBLOGNORM_LIBS@ +LIBM = @LIBM@ +LIBMONGO_CLIENT_CFLAGS = @LIBMONGO_CLIENT_CFLAGS@ +LIBMONGO_CLIENT_LIBS = @LIBMONGO_CLIENT_LIBS@ +LIBOBJS = @LIBOBJS@ +LIBS = @LIBS@ +LIBSYSTEMD_JOURNAL_CFLAGS = @LIBSYSTEMD_JOURNAL_CFLAGS@ +LIBSYSTEMD_JOURNAL_LIBS = @LIBSYSTEMD_JOURNAL_LIBS@ +LIBTOOL = @LIBTOOL@ +LIBUUID_CFLAGS = @LIBUUID_CFLAGS@ +LIBUUID_LIBS = @LIBUUID_LIBS@ +LIPO = @LIPO@ +LN_S = @LN_S@ +LTLIBOBJS = @LTLIBOBJS@ +MAKEINFO = @MAKEINFO@ +MANIFEST_TOOL = @MANIFEST_TOOL@ +MKDIR_P = @MKDIR_P@ +MYSQL_CFLAGS = @MYSQL_CFLAGS@ +MYSQL_LIBS = @MYSQL_LIBS@ +NM = @NM@ +NMEDIT = @NMEDIT@ +OBJDUMP = @OBJDUMP@ +OBJEXT = @OBJEXT@ +OPENSSL_CFLAGS = @OPENSSL_CFLAGS@ +OPENSSL_LIBS = @OPENSSL_LIBS@ +ORACLE_CFLAGS = @ORACLE_CFLAGS@ +ORACLE_LIBS = @ORACLE_LIBS@ +OTOOL = @OTOOL@ +OTOOL64 = @OTOOL64@ +PACKAGE = @PACKAGE@ +PACKAGE_BUGREPORT = @PACKAGE_BUGREPORT@ +PACKAGE_NAME = @PACKAGE_NAME@ +PACKAGE_STRING = @PACKAGE_STRING@ +PACKAGE_TARNAME = @PACKAGE_TARNAME@ +PACKAGE_URL = @PACKAGE_URL@ +PACKAGE_VERSION = @PACKAGE_VERSION@ +PATH_SEPARATOR = @PATH_SEPARATOR@ +PGSQL_CFLAGS = @PGSQL_CFLAGS@ +PGSQL_LIBS = @PGSQL_LIBS@ +PKG_CONFIG = @PKG_CONFIG@ +PKG_CONFIG_LIBDIR = @PKG_CONFIG_LIBDIR@ +PKG_CONFIG_PATH = @PKG_CONFIG_PATH@ +PTHREADS_CFLAGS = @PTHREADS_CFLAGS@ +PTHREADS_LIBS = @PTHREADS_LIBS@ +RABBITMQ_CFLAGS = @RABBITMQ_CFLAGS@ +RABBITMQ_LIBS = @RABBITMQ_LIBS@ +RANLIB = @RANLIB@ +RELP_CFLAGS = @RELP_CFLAGS@ +RELP_LIBS = @RELP_LIBS@ +RSRT_CFLAGS = @RSRT_CFLAGS@ +RSRT_CFLAGS1 = @RSRT_CFLAGS1@ +RSRT_LIBS = @RSRT_LIBS@ +RSRT_LIBS1 = @RSRT_LIBS1@ +RST2MAN = @RST2MAN@ +RT_LIBS = @RT_LIBS@ +SED = @SED@ +SET_MAKE = @SET_MAKE@ +SHELL = @SHELL@ +SNMP_CFLAGS = @SNMP_CFLAGS@ +SNMP_LIBS = @SNMP_LIBS@ +SOL_LIBS = @SOL_LIBS@ +STRIP = @STRIP@ +UDPSPOOF_CFLAGS = @UDPSPOOF_CFLAGS@ +UDPSPOOF_LIBS = @UDPSPOOF_LIBS@ +VERSION = @VERSION@ +YACC = @YACC@ +YFLAGS = @YFLAGS@ +ZLIB_LIBS = @ZLIB_LIBS@ +abs_builddir = @abs_builddir@ +abs_srcdir = @abs_srcdir@ +abs_top_builddir = @abs_top_builddir@ +abs_top_srcdir = @abs_top_srcdir@ +ac_ct_AR = @ac_ct_AR@ +ac_ct_CC = @ac_ct_CC@ +ac_ct_DUMPBIN = @ac_ct_DUMPBIN@ +am__include = @am__include@ +am__leading_dot = @am__leading_dot@ +am__quote = @am__quote@ +am__tar = @am__tar@ +am__untar = @am__untar@ +bindir = @bindir@ +build = @build@ +build_alias = @build_alias@ +build_cpu = @build_cpu@ +build_os = @build_os@ +build_vendor = @build_vendor@ +builddir = @builddir@ +datadir = @datadir@ +datarootdir = @datarootdir@ +docdir = @docdir@ +dvidir = @dvidir@ +exec_prefix = @exec_prefix@ +have_valgrind = @have_valgrind@ +host = @host@ +host_alias = @host_alias@ +host_cpu = @host_cpu@ +host_os = @host_os@ +host_vendor = @host_vendor@ +htmldir = @htmldir@ +includedir = @includedir@ +infodir = @infodir@ +install_sh = @install_sh@ +libdir = @libdir@ +libexecdir = @libexecdir@ +localedir = @localedir@ +localstatedir = @localstatedir@ +mandir = @mandir@ +mkdir_p = @mkdir_p@ +moddirs = @moddirs@ +oldincludedir = @oldincludedir@ +pdfdir = @pdfdir@ +prefix = @prefix@ +program_transform_name = @program_transform_name@ +psdir = @psdir@ +sbindir = @sbindir@ +sharedstatedir = @sharedstatedir@ +srcdir = @srcdir@ +sysconfdir = @sysconfdir@ +systemdsystemunitdir = @systemdsystemunitdir@ +target_alias = @target_alias@ +top_build_prefix = @top_build_prefix@ +top_builddir = @top_builddir@ +top_srcdir = @top_srcdir@ +pkglib_LTLIBRARIES = mmpstrucdata.la +mmpstrucdata_la_SOURCES = mmpstrucdata.c +mmpstrucdata_la_CPPFLAGS = $(RSRT_CFLAGS) $(PTHREADS_CFLAGS) +mmpstrucdata_la_LDFLAGS = -module -avoid-version +mmpstrucdata_la_LIBADD = +EXTRA_DIST = +all: all-am + +.SUFFIXES: +.SUFFIXES: .c .lo .o .obj +$(srcdir)/Makefile.in: $(srcdir)/Makefile.am $(am__configure_deps) + @for dep in $?; do \ + case '$(am__configure_deps)' in \ + *$$dep*) \ + ( cd $(top_builddir) && $(MAKE) $(AM_MAKEFLAGS) am--refresh ) \ + && { if test -f $@; then exit 0; else break; fi; }; \ + exit 1;; \ + esac; \ + done; \ + echo ' cd $(top_srcdir) && $(AUTOMAKE) --gnu plugins/mmpstrucdata/Makefile'; \ + $(am__cd) $(top_srcdir) && \ + $(AUTOMAKE) --gnu plugins/mmpstrucdata/Makefile +.PRECIOUS: Makefile +Makefile: $(srcdir)/Makefile.in $(top_builddir)/config.status + @case '$?' in \ + *config.status*) \ + cd $(top_builddir) && $(MAKE) $(AM_MAKEFLAGS) am--refresh;; \ + *) \ + echo ' cd $(top_builddir) && $(SHELL) ./config.status $(subdir)/$@ $(am__depfiles_maybe)'; \ + cd $(top_builddir) && $(SHELL) ./config.status $(subdir)/$@ $(am__depfiles_maybe);; \ + esac; + +$(top_builddir)/config.status: $(top_srcdir)/configure $(CONFIG_STATUS_DEPENDENCIES) + cd $(top_builddir) && $(MAKE) $(AM_MAKEFLAGS) am--refresh + +$(top_srcdir)/configure: $(am__configure_deps) + cd $(top_builddir) && $(MAKE) $(AM_MAKEFLAGS) am--refresh +$(ACLOCAL_M4): $(am__aclocal_m4_deps) + cd $(top_builddir) && $(MAKE) $(AM_MAKEFLAGS) am--refresh +$(am__aclocal_m4_deps): +install-pkglibLTLIBRARIES: $(pkglib_LTLIBRARIES) + @$(NORMAL_INSTALL) + test -z "$(pkglibdir)" || $(MKDIR_P) "$(DESTDIR)$(pkglibdir)" + @list='$(pkglib_LTLIBRARIES)'; test -n "$(pkglibdir)" || list=; \ + list2=; for p in $$list; do \ + if test -f $$p; then \ + list2="$$list2 $$p"; \ + else :; fi; \ + done; \ + test -z "$$list2" || { \ + echo " $(LIBTOOL) $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=install $(INSTALL) $(INSTALL_STRIP_FLAG) $$list2 '$(DESTDIR)$(pkglibdir)'"; \ + $(LIBTOOL) $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=install $(INSTALL) $(INSTALL_STRIP_FLAG) $$list2 "$(DESTDIR)$(pkglibdir)"; \ + } + +uninstall-pkglibLTLIBRARIES: + @$(NORMAL_UNINSTALL) + @list='$(pkglib_LTLIBRARIES)'; test -n "$(pkglibdir)" || list=; \ + for p in $$list; do \ + $(am__strip_dir) \ + echo " $(LIBTOOL) $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=uninstall rm -f '$(DESTDIR)$(pkglibdir)/$$f'"; \ + $(LIBTOOL) $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=uninstall rm -f "$(DESTDIR)$(pkglibdir)/$$f"; \ + done + +clean-pkglibLTLIBRARIES: + -test -z "$(pkglib_LTLIBRARIES)" || rm -f $(pkglib_LTLIBRARIES) + @list='$(pkglib_LTLIBRARIES)'; for p in $$list; do \ + dir="`echo $$p | sed -e 's|/[^/]*$$||'`"; \ + test "$$dir" != "$$p" || dir=.; \ + echo "rm -f \"$${dir}/so_locations\""; \ + rm -f "$${dir}/so_locations"; \ + done +mmpstrucdata.la: $(mmpstrucdata_la_OBJECTS) $(mmpstrucdata_la_DEPENDENCIES) $(EXTRA_mmpstrucdata_la_DEPENDENCIES) + $(AM_V_CCLD)$(mmpstrucdata_la_LINK) -rpath $(pkglibdir) $(mmpstrucdata_la_OBJECTS) $(mmpstrucdata_la_LIBADD) $(LIBS) + +mostlyclean-compile: + -rm -f *.$(OBJEXT) + +distclean-compile: + -rm -f *.tab.c + +@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/mmpstrucdata_la-mmpstrucdata.Plo@am__quote@ + +.c.o: +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.o$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ $< + +.c.obj: +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.obj$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ `$(CYGPATH_W) '$<'` &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ `$(CYGPATH_W) '$<'` + +.c.lo: +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.lo$$||'`;\ +@am__fastdepCC_TRUE@ $(LTCOMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Plo +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=yes @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(LTCOMPILE) -c -o $@ $< + +mmpstrucdata_la-mmpstrucdata.lo: mmpstrucdata.c +@am__fastdepCC_TRUE@ $(AM_V_CC)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(mmpstrucdata_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT mmpstrucdata_la-mmpstrucdata.lo -MD -MP -MF $(DEPDIR)/mmpstrucdata_la-mmpstrucdata.Tpo -c -o mmpstrucdata_la-mmpstrucdata.lo `test -f 'mmpstrucdata.c' || echo '$(srcdir)/'`mmpstrucdata.c +@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/mmpstrucdata_la-mmpstrucdata.Tpo $(DEPDIR)/mmpstrucdata_la-mmpstrucdata.Plo +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='mmpstrucdata.c' object='mmpstrucdata_la-mmpstrucdata.lo' libtool=yes @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(mmpstrucdata_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o mmpstrucdata_la-mmpstrucdata.lo `test -f 'mmpstrucdata.c' || echo '$(srcdir)/'`mmpstrucdata.c + +mostlyclean-libtool: + -rm -f *.lo + +clean-libtool: + -rm -rf .libs _libs + +ID: $(HEADERS) $(SOURCES) $(LISP) $(TAGS_FILES) + list='$(SOURCES) $(HEADERS) $(LISP) $(TAGS_FILES)'; \ + unique=`for i in $$list; do \ + if test -f "$$i"; then echo $$i; else echo $(srcdir)/$$i; fi; \ + done | \ + $(AWK) '{ files[$$0] = 1; nonempty = 1; } \ + END { if (nonempty) { for (i in files) print i; }; }'`; \ + mkid -fID $$unique +tags: TAGS + +TAGS: $(HEADERS) $(SOURCES) $(TAGS_DEPENDENCIES) \ + $(TAGS_FILES) $(LISP) + set x; \ + here=`pwd`; \ + list='$(SOURCES) $(HEADERS) $(LISP) $(TAGS_FILES)'; \ + unique=`for i in $$list; do \ + if test -f "$$i"; then echo $$i; else echo $(srcdir)/$$i; fi; \ + done | \ + $(AWK) '{ files[$$0] = 1; nonempty = 1; } \ + END { if (nonempty) { for (i in files) print i; }; }'`; \ + shift; \ + if test -z "$(ETAGS_ARGS)$$*$$unique"; then :; else \ + test -n "$$unique" || unique=$$empty_fix; \ + if test $$# -gt 0; then \ + $(ETAGS) $(ETAGSFLAGS) $(AM_ETAGSFLAGS) $(ETAGS_ARGS) \ + "$$@" $$unique; \ + else \ + $(ETAGS) $(ETAGSFLAGS) $(AM_ETAGSFLAGS) $(ETAGS_ARGS) \ + $$unique; \ + fi; \ + fi +ctags: CTAGS +CTAGS: $(HEADERS) $(SOURCES) $(TAGS_DEPENDENCIES) \ + $(TAGS_FILES) $(LISP) + list='$(SOURCES) $(HEADERS) $(LISP) $(TAGS_FILES)'; \ + unique=`for i in $$list; do \ + if test -f "$$i"; then echo $$i; else echo $(srcdir)/$$i; fi; \ + done | \ + $(AWK) '{ files[$$0] = 1; nonempty = 1; } \ + END { if (nonempty) { for (i in files) print i; }; }'`; \ + test -z "$(CTAGS_ARGS)$$unique" \ + || $(CTAGS) $(CTAGSFLAGS) $(AM_CTAGSFLAGS) $(CTAGS_ARGS) \ + $$unique + +GTAGS: + here=`$(am__cd) $(top_builddir) && pwd` \ + && $(am__cd) $(top_srcdir) \ + && gtags -i $(GTAGS_ARGS) "$$here" + +distclean-tags: + -rm -f TAGS ID GTAGS GRTAGS GSYMS GPATH tags + +distdir: $(DISTFILES) + @srcdirstrip=`echo "$(srcdir)" | sed 's/[].[^$$\\*]/\\\\&/g'`; \ + topsrcdirstrip=`echo "$(top_srcdir)" | sed 's/[].[^$$\\*]/\\\\&/g'`; \ + list='$(DISTFILES)'; \ + dist_files=`for file in $$list; do echo $$file; done | \ + sed -e "s|^$$srcdirstrip/||;t" \ + -e "s|^$$topsrcdirstrip/|$(top_builddir)/|;t"`; \ + case $$dist_files in \ + */*) $(MKDIR_P) `echo "$$dist_files" | \ + sed '/\//!d;s|^|$(distdir)/|;s,/[^/]*$$,,' | \ + sort -u` ;; \ + esac; \ + for file in $$dist_files; do \ + if test -f $$file || test -d $$file; then d=.; else d=$(srcdir); fi; \ + if test -d $$d/$$file; then \ + dir=`echo "/$$file" | sed -e 's,/[^/]*$$,,'`; \ + if test -d "$(distdir)/$$file"; then \ + find "$(distdir)/$$file" -type d ! -perm -700 -exec chmod u+rwx {} \;; \ + fi; \ + if test -d $(srcdir)/$$file && test $$d != $(srcdir); then \ + cp -fpR $(srcdir)/$$file "$(distdir)$$dir" || exit 1; \ + find "$(distdir)/$$file" -type d ! -perm -700 -exec chmod u+rwx {} \;; \ + fi; \ + cp -fpR $$d/$$file "$(distdir)$$dir" || exit 1; \ + else \ + test -f "$(distdir)/$$file" \ + || cp -p $$d/$$file "$(distdir)/$$file" \ + || exit 1; \ + fi; \ + done +check-am: all-am +check: check-am +all-am: Makefile $(LTLIBRARIES) +installdirs: + for dir in "$(DESTDIR)$(pkglibdir)"; do \ + test -z "$$dir" || $(MKDIR_P) "$$dir"; \ + done +install: install-am +install-exec: install-exec-am +install-data: install-data-am +uninstall: uninstall-am + +install-am: all-am + @$(MAKE) $(AM_MAKEFLAGS) install-exec-am install-data-am + +installcheck: installcheck-am +install-strip: + if test -z '$(STRIP)'; then \ + $(MAKE) $(AM_MAKEFLAGS) INSTALL_PROGRAM="$(INSTALL_STRIP_PROGRAM)" \ + install_sh_PROGRAM="$(INSTALL_STRIP_PROGRAM)" INSTALL_STRIP_FLAG=-s \ + install; \ + else \ + $(MAKE) $(AM_MAKEFLAGS) INSTALL_PROGRAM="$(INSTALL_STRIP_PROGRAM)" \ + install_sh_PROGRAM="$(INSTALL_STRIP_PROGRAM)" INSTALL_STRIP_FLAG=-s \ + "INSTALL_PROGRAM_ENV=STRIPPROG='$(STRIP)'" install; \ + fi +mostlyclean-generic: + +clean-generic: + +distclean-generic: + -test -z "$(CONFIG_CLEAN_FILES)" || rm -f $(CONFIG_CLEAN_FILES) + -test . = "$(srcdir)" || test -z "$(CONFIG_CLEAN_VPATH_FILES)" || rm -f $(CONFIG_CLEAN_VPATH_FILES) + +maintainer-clean-generic: + @echo "This command is intended for maintainers to use" + @echo "it deletes files that may require special tools to rebuild." +clean: clean-am + +clean-am: clean-generic clean-libtool clean-pkglibLTLIBRARIES \ + mostlyclean-am + +distclean: distclean-am + -rm -rf ./$(DEPDIR) + -rm -f Makefile +distclean-am: clean-am distclean-compile distclean-generic \ + distclean-tags + +dvi: dvi-am + +dvi-am: + +html: html-am + +html-am: + +info: info-am + +info-am: + +install-data-am: + +install-dvi: install-dvi-am + +install-dvi-am: + +install-exec-am: install-pkglibLTLIBRARIES + +install-html: install-html-am + +install-html-am: + +install-info: install-info-am + +install-info-am: + +install-man: + +install-pdf: install-pdf-am + +install-pdf-am: + +install-ps: install-ps-am + +install-ps-am: + +installcheck-am: + +maintainer-clean: maintainer-clean-am + -rm -rf ./$(DEPDIR) + -rm -f Makefile +maintainer-clean-am: distclean-am maintainer-clean-generic + +mostlyclean: mostlyclean-am + +mostlyclean-am: mostlyclean-compile mostlyclean-generic \ + mostlyclean-libtool + +pdf: pdf-am + +pdf-am: + +ps: ps-am + +ps-am: + +uninstall-am: uninstall-pkglibLTLIBRARIES + +.MAKE: install-am install-strip + +.PHONY: CTAGS GTAGS all all-am check check-am clean clean-generic \ + clean-libtool clean-pkglibLTLIBRARIES ctags distclean \ + distclean-compile distclean-generic distclean-libtool \ + distclean-tags distdir dvi dvi-am html html-am info info-am \ + install install-am install-data install-data-am install-dvi \ + install-dvi-am install-exec install-exec-am install-html \ + install-html-am install-info install-info-am install-man \ + install-pdf install-pdf-am install-pkglibLTLIBRARIES \ + install-ps install-ps-am install-strip installcheck \ + installcheck-am installdirs maintainer-clean \ + maintainer-clean-generic mostlyclean mostlyclean-compile \ + mostlyclean-generic mostlyclean-libtool pdf pdf-am ps ps-am \ + tags uninstall uninstall-am uninstall-pkglibLTLIBRARIES + + +# Tell versions [3.59,3.63) of GNU make to not export all variables. +# Otherwise a system limit (for SysV at least) may be exceeded. +.NOEXPORT: diff --git a/plugins/mmpstrucdata/mmpstrucdata.c b/plugins/mmpstrucdata/mmpstrucdata.c new file mode 100644 index 0000000..680ba92 --- /dev/null +++ b/plugins/mmpstrucdata/mmpstrucdata.c @@ -0,0 +1,415 @@ +/* mmpstrucdata.c + * Parse all fields of the message into structured data inside the + * JSON tree. + * + * Copyright 2013 Adiscon GmbH. + * + * This file is part of rsyslog. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * -or- + * see COPYING.ASL20 in the source distribution + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +#include "config.h" +#include "rsyslog.h" +#include <stdio.h> +#include <stdarg.h> +#include <stdlib.h> +#include <string.h> +#include <assert.h> +#include <signal.h> +#include <errno.h> +#include <unistd.h> +#include <stdint.h> +#include <ctype.h> +#include "conf.h" +#include "syslogd-types.h" +#include "srUtils.h" +#include "template.h" +#include "module-template.h" +#include "errmsg.h" + +MODULE_TYPE_OUTPUT +MODULE_TYPE_NOKEEP +MODULE_CNFNAME("mmpstrucdata") + + +DEFobjCurrIf(errmsg); +DEF_OMOD_STATIC_DATA + +/* config variables */ + +typedef struct _instanceData { + uchar *jsonRoot; /**< container where to store fields */ +} instanceData; + +typedef struct wrkrInstanceData { + instanceData *pData; +} wrkrInstanceData_t; + +struct modConfData_s { + rsconf_t *pConf; /* our overall config object */ +}; +static modConfData_t *loadModConf = NULL;/* modConf ptr to use for the current load process */ +static modConfData_t *runModConf = NULL;/* modConf ptr to use for the current exec process */ + + +/* tables for interfacing with the v6 config system */ +/* action (instance) parameters */ +static struct cnfparamdescr actpdescr[] = { + { "jsonroot", eCmdHdlrString, 0 } +}; +static struct cnfparamblk actpblk = + { CNFPARAMBLK_VERSION, + sizeof(actpdescr)/sizeof(struct cnfparamdescr), + actpdescr + }; + +BEGINbeginCnfLoad +CODESTARTbeginCnfLoad + loadModConf = pModConf; + pModConf->pConf = pConf; +ENDbeginCnfLoad + +BEGINendCnfLoad +CODESTARTendCnfLoad +ENDendCnfLoad + +BEGINcheckCnf +CODESTARTcheckCnf +ENDcheckCnf + +BEGINactivateCnf +CODESTARTactivateCnf + runModConf = pModConf; +ENDactivateCnf + +BEGINfreeCnf +CODESTARTfreeCnf +ENDfreeCnf + + +BEGINcreateInstance +CODESTARTcreateInstance +ENDcreateInstance + +BEGINcreateWrkrInstance +CODESTARTcreateWrkrInstance +ENDcreateWrkrInstance + + +BEGINisCompatibleWithFeature +CODESTARTisCompatibleWithFeature +ENDisCompatibleWithFeature + + +BEGINfreeInstance +CODESTARTfreeInstance + free(pData->jsonRoot); +ENDfreeInstance + +BEGINfreeWrkrInstance +CODESTARTfreeWrkrInstance +ENDfreeWrkrInstance + + +static inline void +setInstParamDefaults(instanceData *pData) +{ + pData->jsonRoot = NULL; +} + +BEGINnewActInst + struct cnfparamvals *pvals; + int i; +CODESTARTnewActInst + DBGPRINTF("newActInst (mmpstrucdata)\n"); + if((pvals = nvlstGetParams(lst, &actpblk, NULL)) == NULL) { + ABORT_FINALIZE(RS_RET_MISSING_CNFPARAMS); + } + + CODE_STD_STRING_REQUESTnewActInst(1) + CHKiRet(OMSRsetEntry(*ppOMSR, 0, NULL, OMSR_TPL_AS_MSG)); + CHKiRet(createInstance(&pData)); + setInstParamDefaults(pData); + + for(i = 0 ; i < actpblk.nParams ; ++i) { + if(!pvals[i].bUsed) + continue; + if(!strcmp(actpblk.descr[i].name, "jsonroot")) { + pData->jsonRoot = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + } else { + dbgprintf("mmpstrucdata: program error, non-handled " + "param '%s'\n", actpblk.descr[i].name); + } + } + if(pData->jsonRoot == NULL) { + CHKmalloc(pData->jsonRoot = (uchar*) strdup("!")); + } + +CODE_STD_FINALIZERnewActInst + cnfparamvalsDestruct(pvals, &actpblk); +ENDnewActInst + + +BEGINdbgPrintInstInfo +CODESTARTdbgPrintInstInfo +ENDdbgPrintInstInfo + + +BEGINtryResume +CODESTARTtryResume +ENDtryResume + + +static inline rsRetVal +parsePARAM_VALUE(uchar *sdbuf, int lenbuf, int *curridx, uchar *fieldbuf) +{ + int i, j; + DEFiRet; +dbgprintf("DDDD: parsePARAM_VALUE\n"); + i = *curridx; + j = 0; + while(i < lenbuf && sdbuf[i] != '"') { + if(sdbuf[i] == '\\') { + if(++i == lenbuf) { + fieldbuf[j++] = '\\'; + } else { + if(sdbuf[i] == '"') { + fieldbuf[j++] = '"'; + } else if(sdbuf[i] == '\\') { + fieldbuf[j++] = '\\'; + } else if(sdbuf[i] == ']') { + fieldbuf[j++] = '"'; + } else { + fieldbuf[j++] = '\\'; + fieldbuf[j++] = sdbuf[i]; + } + ++i; + } + } else { + fieldbuf[j++] = sdbuf[i++]; + } + } + fieldbuf[j] = '\0'; + *curridx = i; +dbgprintf("DDDD: parsePARAM_VALUE: '%s'\n", fieldbuf); + RETiRet; +} + + +static rsRetVal +parseSD_NAME(uchar *sdbuf, int lenbuf, int *curridx, uchar *namebuf) +{ + int i, j; + DEFiRet; +dbgprintf("DDDD: parseSD_NAME %s\n", sdbuf+*curridx); + i = *curridx; + for(j = 0 ; i < lenbuf && j < 32; ++j) { + if( sdbuf[i] == '=' || sdbuf[i] == '"' + || sdbuf[i] == ']' || sdbuf[i] == ' ') + break; + namebuf[j] = tolower(sdbuf[i]); + ++i; + } + namebuf[j] = '\0'; +dbgprintf("DDDD: parseSD_NAME, NAME: '%s'\n", namebuf); + *curridx = i; + RETiRet; +} + + +static inline rsRetVal +parseSD_PARAM(instanceData *pData, uchar *sdbuf, int lenbuf, int *curridx, struct json_object *jroot) +{ + int i; + uchar pName[33]; + uchar pVal[32*1024]; + struct json_object *jval; + DEFiRet; +dbgprintf("DDDD: parseSD_PARAM %s\n", sdbuf+*curridx); + + i = *curridx; + CHKiRet(parseSD_NAME(sdbuf, lenbuf, &i, pName)); + if(sdbuf[i] != '=') { + ABORT_FINALIZE(RS_RET_STRUC_DATA_INVLD); + } + ++i; + if(sdbuf[i] != '"') { + ABORT_FINALIZE(RS_RET_STRUC_DATA_INVLD); + } + ++i; + CHKiRet(parsePARAM_VALUE(sdbuf, lenbuf, &i, pVal)); + if(sdbuf[i] != '"') { + ABORT_FINALIZE(RS_RET_STRUC_DATA_INVLD); + } + ++i; + + jval = json_object_new_string((char*)pVal); + json_object_object_add(jroot, (char*)pName, jval); + + *curridx = i; +finalize_it: + RETiRet; +} + + +static inline rsRetVal +parseSD_ELEMENT(instanceData *pData, uchar *sdbuf, int lenbuf, int *curridx, struct json_object *jroot) +{ + int i; + uchar sd_id[33]; + struct json_object *json; + DEFiRet; +dbgprintf("DDDD: parseSD_ELEMENT: %s\n", sdbuf+*curridx); + + i = *curridx; + if(sdbuf[i] != '[') { + ABORT_FINALIZE(RS_RET_STRUC_DATA_INVLD); + } + ++i; /* eat '[' */ + + CHKiRet(parseSD_NAME(sdbuf, lenbuf, &i, sd_id)); + json = json_object_new_object(); + + while(i < lenbuf) { + if(sdbuf[i] == ']') { + break; + } else if(sdbuf[i] != ' ') { + ABORT_FINALIZE(RS_RET_STRUC_DATA_INVLD); + } + ++i; + while(i < lenbuf && sdbuf[i] == ' ') + ++i; + CHKiRet(parseSD_PARAM(pData, sdbuf, lenbuf, &i, json)); +dbgprintf("DDDD: done parseSD_PARAM, in loop, i:%d, lenbuf:%d, rest: %s\n", i, lenbuf, sdbuf+i); + } + + if(sdbuf[i] != ']') { + DBGPRINTF("mmpstrucdata: SD-ELEMENT does not terminate with " + "']': '%s'\n", sdbuf+i); + ABORT_FINALIZE(RS_RET_STRUC_DATA_INVLD); + } + ++i; /* eat ']' */ + *curridx = i; + json_object_object_add(jroot, (char*)sd_id, json); +dbgprintf("DDDD: SD_ELEMENT: json: '%s'\n", json_object_get_string(json)); +dbgprintf("DDDD: SD_ELEMENT: jroot '%s'\n", json_object_get_string(json)); +finalize_it: +dbgprintf("DDDD: parseSD_ELEMENT iRet:%d, i:%d, *curridx:%d\n", iRet, i, *curridx); + RETiRet; +} + +static inline rsRetVal +parse_sd(instanceData *pData, msg_t *pMsg) +{ +#if 0 + uchar fieldbuf[32*1024]; + uchar fieldname[512]; + struct json_object *json; + struct json_object *jval; + int field; + uchar *buf; +#endif + struct json_object *json, *jroot; + uchar *sdbuf; + int lenbuf; + int i = 0; + DEFiRet; + +#if 0 + if(lenMsg < (int) sizeof(fieldbuf)) { + buf = fieldbuf; + } else { + CHKmalloc(buf = malloc(lenMsg+1)); + } +#endif + +dbgprintf("DDDD: parse_sd\n"); + json = json_object_new_object(); + if(json == NULL) { + ABORT_FINALIZE(RS_RET_ERR); + } + MsgGetStructuredData(pMsg, &sdbuf,&lenbuf); + while(i < lenbuf) { + CHKiRet(parseSD_ELEMENT(pData, sdbuf, lenbuf, &i, json)); +dbgprintf("DDDD: parse_sd, i:%d\n", i); + } +dbgprintf("DDDD: json: '%s'\n", json_object_get_string(json)); + + jroot = json_object_new_object(); + if(jroot == NULL) { + ABORT_FINALIZE(RS_RET_ERR); + } + json_object_object_add(jroot, "rfc5424-sd", json); + msgAddJSON(pMsg, pData->jsonRoot, jroot); +finalize_it: + RETiRet; +} + + +BEGINdoAction + msg_t *pMsg; +CODESTARTdoAction +dbgprintf("DDDD: enter mmpstrucdata\n"); + pMsg = (msg_t*) ppString[0]; + if(!MsgHasStructuredData(pMsg)) { + DBGPRINTF("mmpstrucdata: message does not have structured data\n"); + FINALIZE; + } +dbgprintf("DDDD: parse mmpstrucdata\n"); + /* don't check return code - we never want rsyslog to retry + * or suspend this action! + */ + parse_sd(pWrkrData->pData, pMsg); +dbgprintf("DDDD: done parse mmpstrucdata\n"); +finalize_it: +ENDdoAction + + +BEGINparseSelectorAct +CODESTARTparseSelectorAct +CODE_STD_STRING_REQUESTparseSelectorAct(1) + if(strncmp((char*) p, ":mmpstrucdata:", sizeof(":mmpstrucdata:") - 1)) { + errmsg.LogError(0, RS_RET_LEGA_ACT_NOT_SUPPORTED, + "mmpstrucdata supports only v6+ config format, use: " + "action(type=\"mmpstrucdata\" ...)"); + } + ABORT_FINALIZE(RS_RET_CONFLINE_UNPROCESSED); +CODE_STD_FINALIZERparseSelectorAct +ENDparseSelectorAct + + +BEGINmodExit +CODESTARTmodExit + objRelease(errmsg, CORE_COMPONENT); +ENDmodExit + + +BEGINqueryEtryPt +CODESTARTqueryEtryPt +CODEqueryEtryPt_STD_OMOD_QUERIES +CODEqueryEtryPt_STD_OMOD8_QUERIES +CODEqueryEtryPt_STD_CONF2_OMOD_QUERIES +CODEqueryEtryPt_STD_CONF2_QUERIES +ENDqueryEtryPt + + + +BEGINmodInit() +CODESTARTmodInit + *ipIFVersProvided = CURR_MOD_IF_VERSION; /* we only support the current interface specification */ +CODEmodInit_QueryRegCFSLineHdlr + DBGPRINTF("mmpstrucdata: module compiled with rsyslog version %s.\n", VERSION); + CHKiRet(objUse(errmsg, CORE_COMPONENT)); +ENDmodInit diff --git a/plugins/mmrfc5424addhmac/Makefile.am b/plugins/mmrfc5424addhmac/Makefile.am new file mode 100644 index 0000000..6567def --- /dev/null +++ b/plugins/mmrfc5424addhmac/Makefile.am @@ -0,0 +1,8 @@ +pkglib_LTLIBRARIES = mmrfc5424addhmac.la + +mmrfc5424addhmac_la_SOURCES = mmrfc5424addhmac.c +mmrfc5424addhmac_la_CPPFLAGS = $(OPENSSL_CFLAGS) $(RSRT_CFLAGS) $(PTHREADS_CFLAGS) +mmrfc5424addhmac_la_LDFLAGS = -module -avoid-version +mmrfc5424addhmac_la_LIBADD = $(OPENSSL_LIBS) + +EXTRA_DIST = diff --git a/plugins/mmrfc5424addhmac/Makefile.in b/plugins/mmrfc5424addhmac/Makefile.in new file mode 100644 index 0000000..026c549 --- /dev/null +++ b/plugins/mmrfc5424addhmac/Makefile.in @@ -0,0 +1,640 @@ +# Makefile.in generated by automake 1.11.3 from Makefile.am. +# @configure_input@ + +# Copyright (C) 1994, 1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, +# 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010, 2011 Free Software +# Foundation, Inc. +# This Makefile.in is free software; the Free Software Foundation +# gives unlimited permission to copy and/or distribute it, +# with or without modifications, as long as this notice is preserved. + +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY, to the extent permitted by law; without +# even the implied warranty of MERCHANTABILITY or FITNESS FOR A +# PARTICULAR PURPOSE. + +@SET_MAKE@ + +VPATH = @srcdir@ +pkgdatadir = $(datadir)/@PACKAGE@ +pkgincludedir = $(includedir)/@PACKAGE@ +pkglibdir = $(libdir)/@PACKAGE@ +pkglibexecdir = $(libexecdir)/@PACKAGE@ +am__cd = CDPATH="$${ZSH_VERSION+.}$(PATH_SEPARATOR)" && cd +install_sh_DATA = $(install_sh) -c -m 644 +install_sh_PROGRAM = $(install_sh) -c +install_sh_SCRIPT = $(install_sh) -c +INSTALL_HEADER = $(INSTALL_DATA) +transform = $(program_transform_name) +NORMAL_INSTALL = : +PRE_INSTALL = : +POST_INSTALL = : +NORMAL_UNINSTALL = : +PRE_UNINSTALL = : +POST_UNINSTALL = : +build_triplet = @build@ +host_triplet = @host@ +subdir = plugins/mmrfc5424addhmac +DIST_COMMON = $(srcdir)/Makefile.am $(srcdir)/Makefile.in +ACLOCAL_M4 = $(top_srcdir)/aclocal.m4 +am__aclocal_m4_deps = $(top_srcdir)/m4/atomic_operations.m4 \ + $(top_srcdir)/m4/atomic_operations_64bit.m4 \ + $(top_srcdir)/m4/libtool.m4 $(top_srcdir)/m4/ltoptions.m4 \ + $(top_srcdir)/m4/ltsugar.m4 $(top_srcdir)/m4/ltversion.m4 \ + $(top_srcdir)/m4/lt~obsolete.m4 $(top_srcdir)/configure.ac +am__configure_deps = $(am__aclocal_m4_deps) $(CONFIGURE_DEPENDENCIES) \ + $(ACLOCAL_M4) +mkinstalldirs = $(install_sh) -d +CONFIG_HEADER = $(top_builddir)/config.h +CONFIG_CLEAN_FILES = +CONFIG_CLEAN_VPATH_FILES = +am__vpath_adj_setup = srcdirstrip=`echo "$(srcdir)" | sed 's|.|.|g'`; +am__vpath_adj = case $$p in \ + $(srcdir)/*) f=`echo "$$p" | sed "s|^$$srcdirstrip/||"`;; \ + *) f=$$p;; \ + esac; +am__strip_dir = f=`echo $$p | sed -e 's|^.*/||'`; +am__install_max = 40 +am__nobase_strip_setup = \ + srcdirstrip=`echo "$(srcdir)" | sed 's/[].[^$$\\*|]/\\\\&/g'` +am__nobase_strip = \ + for p in $$list; do echo "$$p"; done | sed -e "s|$$srcdirstrip/||" +am__nobase_list = $(am__nobase_strip_setup); \ + for p in $$list; do echo "$$p $$p"; done | \ + sed "s| $$srcdirstrip/| |;"' / .*\//!s/ .*/ ./; s,\( .*\)/[^/]*$$,\1,' | \ + $(AWK) 'BEGIN { files["."] = "" } { files[$$2] = files[$$2] " " $$1; \ + if (++n[$$2] == $(am__install_max)) \ + { print $$2, files[$$2]; n[$$2] = 0; files[$$2] = "" } } \ + END { for (dir in files) print dir, files[dir] }' +am__base_list = \ + sed '$$!N;$$!N;$$!N;$$!N;$$!N;$$!N;$$!N;s/\n/ /g' | \ + sed '$$!N;$$!N;$$!N;$$!N;s/\n/ /g' +am__uninstall_files_from_dir = { \ + test -z "$$files" \ + || { test ! -d "$$dir" && test ! -f "$$dir" && test ! -r "$$dir"; } \ + || { echo " ( cd '$$dir' && rm -f" $$files ")"; \ + $(am__cd) "$$dir" && rm -f $$files; }; \ + } +am__installdirs = "$(DESTDIR)$(pkglibdir)" +LTLIBRARIES = $(pkglib_LTLIBRARIES) +am__DEPENDENCIES_1 = +mmrfc5424addhmac_la_DEPENDENCIES = $(am__DEPENDENCIES_1) +am_mmrfc5424addhmac_la_OBJECTS = \ + mmrfc5424addhmac_la-mmrfc5424addhmac.lo +mmrfc5424addhmac_la_OBJECTS = $(am_mmrfc5424addhmac_la_OBJECTS) +AM_V_lt = $(am__v_lt_@AM_V@) +am__v_lt_ = $(am__v_lt_@AM_DEFAULT_V@) +am__v_lt_0 = --silent +mmrfc5424addhmac_la_LINK = $(LIBTOOL) $(AM_V_lt) --tag=CC \ + $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=link $(CCLD) \ + $(AM_CFLAGS) $(CFLAGS) $(mmrfc5424addhmac_la_LDFLAGS) \ + $(LDFLAGS) -o $@ +DEFAULT_INCLUDES = -I.@am__isrc@ -I$(top_builddir) +depcomp = $(SHELL) $(top_srcdir)/depcomp +am__depfiles_maybe = depfiles +am__mv = mv -f +COMPILE = $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(AM_CPPFLAGS) \ + $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) +LTCOMPILE = $(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \ + $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) \ + $(DEFAULT_INCLUDES) $(INCLUDES) $(AM_CPPFLAGS) $(CPPFLAGS) \ + $(AM_CFLAGS) $(CFLAGS) +AM_V_CC = $(am__v_CC_@AM_V@) +am__v_CC_ = $(am__v_CC_@AM_DEFAULT_V@) +am__v_CC_0 = @echo " CC " $@; +AM_V_at = $(am__v_at_@AM_V@) +am__v_at_ = $(am__v_at_@AM_DEFAULT_V@) +am__v_at_0 = @ +CCLD = $(CC) +LINK = $(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \ + $(LIBTOOLFLAGS) --mode=link $(CCLD) $(AM_CFLAGS) $(CFLAGS) \ + $(AM_LDFLAGS) $(LDFLAGS) -o $@ +AM_V_CCLD = $(am__v_CCLD_@AM_V@) +am__v_CCLD_ = $(am__v_CCLD_@AM_DEFAULT_V@) +am__v_CCLD_0 = @echo " CCLD " $@; +AM_V_GEN = $(am__v_GEN_@AM_V@) +am__v_GEN_ = $(am__v_GEN_@AM_DEFAULT_V@) +am__v_GEN_0 = @echo " GEN " $@; +SOURCES = $(mmrfc5424addhmac_la_SOURCES) +DIST_SOURCES = $(mmrfc5424addhmac_la_SOURCES) +ETAGS = etags +CTAGS = ctags +DISTFILES = $(DIST_COMMON) $(DIST_SOURCES) $(TEXINFOS) $(EXTRA_DIST) +ACLOCAL = @ACLOCAL@ +AMTAR = @AMTAR@ +AM_DEFAULT_VERBOSITY = @AM_DEFAULT_VERBOSITY@ +AR = @AR@ +AUTOCONF = @AUTOCONF@ +AUTOHEADER = @AUTOHEADER@ +AUTOMAKE = @AUTOMAKE@ +AWK = @AWK@ +CC = @CC@ +CCDEPMODE = @CCDEPMODE@ +CFLAGS = @CFLAGS@ +CPP = @CPP@ +CPPFLAGS = @CPPFLAGS@ +CURL_CFLAGS = @CURL_CFLAGS@ +CURL_LIBS = @CURL_LIBS@ +CYGPATH_W = @CYGPATH_W@ +CZMQ_CFLAGS = @CZMQ_CFLAGS@ +CZMQ_LIBS = @CZMQ_LIBS@ +DEFS = @DEFS@ +DEPDIR = @DEPDIR@ +DLLTOOL = @DLLTOOL@ +DL_LIBS = @DL_LIBS@ +DSYMUTIL = @DSYMUTIL@ +DUMPBIN = @DUMPBIN@ +ECHO_C = @ECHO_C@ +ECHO_N = @ECHO_N@ +ECHO_T = @ECHO_T@ +EGREP = @EGREP@ +EXEEXT = @EXEEXT@ +FGREP = @FGREP@ +GNUTLS_CFLAGS = @GNUTLS_CFLAGS@ +GNUTLS_LIBS = @GNUTLS_LIBS@ +GREP = @GREP@ +GSS_LIBS = @GSS_LIBS@ +GUARDTIME_CFLAGS = @GUARDTIME_CFLAGS@ +GUARDTIME_LIBS = @GUARDTIME_LIBS@ +HAVE_MYSQL_CONFIG = @HAVE_MYSQL_CONFIG@ +HAVE_ORACLE_CONFIG = @HAVE_ORACLE_CONFIG@ +HAVE_PGSQL_CONFIG = @HAVE_PGSQL_CONFIG@ +HIREDIS_CFLAGS = @HIREDIS_CFLAGS@ +HIREDIS_LIBS = @HIREDIS_LIBS@ +IMUDP_LIBS = @IMUDP_LIBS@ +INSTALL = @INSTALL@ +INSTALL_DATA = @INSTALL_DATA@ +INSTALL_PROGRAM = @INSTALL_PROGRAM@ +INSTALL_SCRIPT = @INSTALL_SCRIPT@ +INSTALL_STRIP_PROGRAM = @INSTALL_STRIP_PROGRAM@ +JSON_C_CFLAGS = @JSON_C_CFLAGS@ +JSON_C_LIBS = @JSON_C_LIBS@ +LD = @LD@ +LDFLAGS = @LDFLAGS@ +LEX = @LEX@ +LEXLIB = @LEXLIB@ +LEX_OUTPUT_ROOT = @LEX_OUTPUT_ROOT@ +LIBDBI_CFLAGS = @LIBDBI_CFLAGS@ +LIBDBI_LIBS = @LIBDBI_LIBS@ +LIBESTR_CFLAGS = @LIBESTR_CFLAGS@ +LIBESTR_LIBS = @LIBESTR_LIBS@ +LIBGCRYPT_CFLAGS = @LIBGCRYPT_CFLAGS@ +LIBGCRYPT_CONFIG = @LIBGCRYPT_CONFIG@ +LIBGCRYPT_LIBS = @LIBGCRYPT_LIBS@ +LIBLOGGING_CFLAGS = @LIBLOGGING_CFLAGS@ +LIBLOGGING_LIBS = @LIBLOGGING_LIBS@ +LIBLOGGING_STDLOG_CFLAGS = @LIBLOGGING_STDLOG_CFLAGS@ +LIBLOGGING_STDLOG_LIBS = @LIBLOGGING_STDLOG_LIBS@ +LIBLOGNORM_CFLAGS = @LIBLOGNORM_CFLAGS@ +LIBLOGNORM_LIBS = @LIBLOGNORM_LIBS@ +LIBM = @LIBM@ +LIBMONGO_CLIENT_CFLAGS = @LIBMONGO_CLIENT_CFLAGS@ +LIBMONGO_CLIENT_LIBS = @LIBMONGO_CLIENT_LIBS@ +LIBOBJS = @LIBOBJS@ +LIBS = @LIBS@ +LIBSYSTEMD_JOURNAL_CFLAGS = @LIBSYSTEMD_JOURNAL_CFLAGS@ +LIBSYSTEMD_JOURNAL_LIBS = @LIBSYSTEMD_JOURNAL_LIBS@ +LIBTOOL = @LIBTOOL@ +LIBUUID_CFLAGS = @LIBUUID_CFLAGS@ +LIBUUID_LIBS = @LIBUUID_LIBS@ +LIPO = @LIPO@ +LN_S = @LN_S@ +LTLIBOBJS = @LTLIBOBJS@ +MAKEINFO = @MAKEINFO@ +MANIFEST_TOOL = @MANIFEST_TOOL@ +MKDIR_P = @MKDIR_P@ +MYSQL_CFLAGS = @MYSQL_CFLAGS@ +MYSQL_LIBS = @MYSQL_LIBS@ +NM = @NM@ +NMEDIT = @NMEDIT@ +OBJDUMP = @OBJDUMP@ +OBJEXT = @OBJEXT@ +OPENSSL_CFLAGS = @OPENSSL_CFLAGS@ +OPENSSL_LIBS = @OPENSSL_LIBS@ +ORACLE_CFLAGS = @ORACLE_CFLAGS@ +ORACLE_LIBS = @ORACLE_LIBS@ +OTOOL = @OTOOL@ +OTOOL64 = @OTOOL64@ +PACKAGE = @PACKAGE@ +PACKAGE_BUGREPORT = @PACKAGE_BUGREPORT@ +PACKAGE_NAME = @PACKAGE_NAME@ +PACKAGE_STRING = @PACKAGE_STRING@ +PACKAGE_TARNAME = @PACKAGE_TARNAME@ +PACKAGE_URL = @PACKAGE_URL@ +PACKAGE_VERSION = @PACKAGE_VERSION@ +PATH_SEPARATOR = @PATH_SEPARATOR@ +PGSQL_CFLAGS = @PGSQL_CFLAGS@ +PGSQL_LIBS = @PGSQL_LIBS@ +PKG_CONFIG = @PKG_CONFIG@ +PKG_CONFIG_LIBDIR = @PKG_CONFIG_LIBDIR@ +PKG_CONFIG_PATH = @PKG_CONFIG_PATH@ +PTHREADS_CFLAGS = @PTHREADS_CFLAGS@ +PTHREADS_LIBS = @PTHREADS_LIBS@ +RABBITMQ_CFLAGS = @RABBITMQ_CFLAGS@ +RABBITMQ_LIBS = @RABBITMQ_LIBS@ +RANLIB = @RANLIB@ +RELP_CFLAGS = @RELP_CFLAGS@ +RELP_LIBS = @RELP_LIBS@ +RSRT_CFLAGS = @RSRT_CFLAGS@ +RSRT_CFLAGS1 = @RSRT_CFLAGS1@ +RSRT_LIBS = @RSRT_LIBS@ +RSRT_LIBS1 = @RSRT_LIBS1@ +RST2MAN = @RST2MAN@ +RT_LIBS = @RT_LIBS@ +SED = @SED@ +SET_MAKE = @SET_MAKE@ +SHELL = @SHELL@ +SNMP_CFLAGS = @SNMP_CFLAGS@ +SNMP_LIBS = @SNMP_LIBS@ +SOL_LIBS = @SOL_LIBS@ +STRIP = @STRIP@ +UDPSPOOF_CFLAGS = @UDPSPOOF_CFLAGS@ +UDPSPOOF_LIBS = @UDPSPOOF_LIBS@ +VERSION = @VERSION@ +YACC = @YACC@ +YFLAGS = @YFLAGS@ +ZLIB_LIBS = @ZLIB_LIBS@ +abs_builddir = @abs_builddir@ +abs_srcdir = @abs_srcdir@ +abs_top_builddir = @abs_top_builddir@ +abs_top_srcdir = @abs_top_srcdir@ +ac_ct_AR = @ac_ct_AR@ +ac_ct_CC = @ac_ct_CC@ +ac_ct_DUMPBIN = @ac_ct_DUMPBIN@ +am__include = @am__include@ +am__leading_dot = @am__leading_dot@ +am__quote = @am__quote@ +am__tar = @am__tar@ +am__untar = @am__untar@ +bindir = @bindir@ +build = @build@ +build_alias = @build_alias@ +build_cpu = @build_cpu@ +build_os = @build_os@ +build_vendor = @build_vendor@ +builddir = @builddir@ +datadir = @datadir@ +datarootdir = @datarootdir@ +docdir = @docdir@ +dvidir = @dvidir@ +exec_prefix = @exec_prefix@ +have_valgrind = @have_valgrind@ +host = @host@ +host_alias = @host_alias@ +host_cpu = @host_cpu@ +host_os = @host_os@ +host_vendor = @host_vendor@ +htmldir = @htmldir@ +includedir = @includedir@ +infodir = @infodir@ +install_sh = @install_sh@ +libdir = @libdir@ +libexecdir = @libexecdir@ +localedir = @localedir@ +localstatedir = @localstatedir@ +mandir = @mandir@ +mkdir_p = @mkdir_p@ +moddirs = @moddirs@ +oldincludedir = @oldincludedir@ +pdfdir = @pdfdir@ +prefix = @prefix@ +program_transform_name = @program_transform_name@ +psdir = @psdir@ +sbindir = @sbindir@ +sharedstatedir = @sharedstatedir@ +srcdir = @srcdir@ +sysconfdir = @sysconfdir@ +systemdsystemunitdir = @systemdsystemunitdir@ +target_alias = @target_alias@ +top_build_prefix = @top_build_prefix@ +top_builddir = @top_builddir@ +top_srcdir = @top_srcdir@ +pkglib_LTLIBRARIES = mmrfc5424addhmac.la +mmrfc5424addhmac_la_SOURCES = mmrfc5424addhmac.c +mmrfc5424addhmac_la_CPPFLAGS = $(OPENSSL_CFLAGS) $(RSRT_CFLAGS) $(PTHREADS_CFLAGS) +mmrfc5424addhmac_la_LDFLAGS = -module -avoid-version +mmrfc5424addhmac_la_LIBADD = $(OPENSSL_LIBS) +EXTRA_DIST = +all: all-am + +.SUFFIXES: +.SUFFIXES: .c .lo .o .obj +$(srcdir)/Makefile.in: $(srcdir)/Makefile.am $(am__configure_deps) + @for dep in $?; do \ + case '$(am__configure_deps)' in \ + *$$dep*) \ + ( cd $(top_builddir) && $(MAKE) $(AM_MAKEFLAGS) am--refresh ) \ + && { if test -f $@; then exit 0; else break; fi; }; \ + exit 1;; \ + esac; \ + done; \ + echo ' cd $(top_srcdir) && $(AUTOMAKE) --gnu plugins/mmrfc5424addhmac/Makefile'; \ + $(am__cd) $(top_srcdir) && \ + $(AUTOMAKE) --gnu plugins/mmrfc5424addhmac/Makefile +.PRECIOUS: Makefile +Makefile: $(srcdir)/Makefile.in $(top_builddir)/config.status + @case '$?' in \ + *config.status*) \ + cd $(top_builddir) && $(MAKE) $(AM_MAKEFLAGS) am--refresh;; \ + *) \ + echo ' cd $(top_builddir) && $(SHELL) ./config.status $(subdir)/$@ $(am__depfiles_maybe)'; \ + cd $(top_builddir) && $(SHELL) ./config.status $(subdir)/$@ $(am__depfiles_maybe);; \ + esac; + +$(top_builddir)/config.status: $(top_srcdir)/configure $(CONFIG_STATUS_DEPENDENCIES) + cd $(top_builddir) && $(MAKE) $(AM_MAKEFLAGS) am--refresh + +$(top_srcdir)/configure: $(am__configure_deps) + cd $(top_builddir) && $(MAKE) $(AM_MAKEFLAGS) am--refresh +$(ACLOCAL_M4): $(am__aclocal_m4_deps) + cd $(top_builddir) && $(MAKE) $(AM_MAKEFLAGS) am--refresh +$(am__aclocal_m4_deps): +install-pkglibLTLIBRARIES: $(pkglib_LTLIBRARIES) + @$(NORMAL_INSTALL) + test -z "$(pkglibdir)" || $(MKDIR_P) "$(DESTDIR)$(pkglibdir)" + @list='$(pkglib_LTLIBRARIES)'; test -n "$(pkglibdir)" || list=; \ + list2=; for p in $$list; do \ + if test -f $$p; then \ + list2="$$list2 $$p"; \ + else :; fi; \ + done; \ + test -z "$$list2" || { \ + echo " $(LIBTOOL) $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=install $(INSTALL) $(INSTALL_STRIP_FLAG) $$list2 '$(DESTDIR)$(pkglibdir)'"; \ + $(LIBTOOL) $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=install $(INSTALL) $(INSTALL_STRIP_FLAG) $$list2 "$(DESTDIR)$(pkglibdir)"; \ + } + +uninstall-pkglibLTLIBRARIES: + @$(NORMAL_UNINSTALL) + @list='$(pkglib_LTLIBRARIES)'; test -n "$(pkglibdir)" || list=; \ + for p in $$list; do \ + $(am__strip_dir) \ + echo " $(LIBTOOL) $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=uninstall rm -f '$(DESTDIR)$(pkglibdir)/$$f'"; \ + $(LIBTOOL) $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=uninstall rm -f "$(DESTDIR)$(pkglibdir)/$$f"; \ + done + +clean-pkglibLTLIBRARIES: + -test -z "$(pkglib_LTLIBRARIES)" || rm -f $(pkglib_LTLIBRARIES) + @list='$(pkglib_LTLIBRARIES)'; for p in $$list; do \ + dir="`echo $$p | sed -e 's|/[^/]*$$||'`"; \ + test "$$dir" != "$$p" || dir=.; \ + echo "rm -f \"$${dir}/so_locations\""; \ + rm -f "$${dir}/so_locations"; \ + done +mmrfc5424addhmac.la: $(mmrfc5424addhmac_la_OBJECTS) $(mmrfc5424addhmac_la_DEPENDENCIES) $(EXTRA_mmrfc5424addhmac_la_DEPENDENCIES) + $(AM_V_CCLD)$(mmrfc5424addhmac_la_LINK) -rpath $(pkglibdir) $(mmrfc5424addhmac_la_OBJECTS) $(mmrfc5424addhmac_la_LIBADD) $(LIBS) + +mostlyclean-compile: + -rm -f *.$(OBJEXT) + +distclean-compile: + -rm -f *.tab.c + +@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/mmrfc5424addhmac_la-mmrfc5424addhmac.Plo@am__quote@ + +.c.o: +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.o$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ $< + +.c.obj: +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.obj$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ `$(CYGPATH_W) '$<'` &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ `$(CYGPATH_W) '$<'` + +.c.lo: +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.lo$$||'`;\ +@am__fastdepCC_TRUE@ $(LTCOMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Plo +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=yes @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(LTCOMPILE) -c -o $@ $< + +mmrfc5424addhmac_la-mmrfc5424addhmac.lo: mmrfc5424addhmac.c +@am__fastdepCC_TRUE@ $(AM_V_CC)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(mmrfc5424addhmac_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT mmrfc5424addhmac_la-mmrfc5424addhmac.lo -MD -MP -MF $(DEPDIR)/mmrfc5424addhmac_la-mmrfc5424addhmac.Tpo -c -o mmrfc5424addhmac_la-mmrfc5424addhmac.lo `test -f 'mmrfc5424addhmac.c' || echo '$(srcdir)/'`mmrfc5424addhmac.c +@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/mmrfc5424addhmac_la-mmrfc5424addhmac.Tpo $(DEPDIR)/mmrfc5424addhmac_la-mmrfc5424addhmac.Plo +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='mmrfc5424addhmac.c' object='mmrfc5424addhmac_la-mmrfc5424addhmac.lo' libtool=yes @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(mmrfc5424addhmac_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o mmrfc5424addhmac_la-mmrfc5424addhmac.lo `test -f 'mmrfc5424addhmac.c' || echo '$(srcdir)/'`mmrfc5424addhmac.c + +mostlyclean-libtool: + -rm -f *.lo + +clean-libtool: + -rm -rf .libs _libs + +ID: $(HEADERS) $(SOURCES) $(LISP) $(TAGS_FILES) + list='$(SOURCES) $(HEADERS) $(LISP) $(TAGS_FILES)'; \ + unique=`for i in $$list; do \ + if test -f "$$i"; then echo $$i; else echo $(srcdir)/$$i; fi; \ + done | \ + $(AWK) '{ files[$$0] = 1; nonempty = 1; } \ + END { if (nonempty) { for (i in files) print i; }; }'`; \ + mkid -fID $$unique +tags: TAGS + +TAGS: $(HEADERS) $(SOURCES) $(TAGS_DEPENDENCIES) \ + $(TAGS_FILES) $(LISP) + set x; \ + here=`pwd`; \ + list='$(SOURCES) $(HEADERS) $(LISP) $(TAGS_FILES)'; \ + unique=`for i in $$list; do \ + if test -f "$$i"; then echo $$i; else echo $(srcdir)/$$i; fi; \ + done | \ + $(AWK) '{ files[$$0] = 1; nonempty = 1; } \ + END { if (nonempty) { for (i in files) print i; }; }'`; \ + shift; \ + if test -z "$(ETAGS_ARGS)$$*$$unique"; then :; else \ + test -n "$$unique" || unique=$$empty_fix; \ + if test $$# -gt 0; then \ + $(ETAGS) $(ETAGSFLAGS) $(AM_ETAGSFLAGS) $(ETAGS_ARGS) \ + "$$@" $$unique; \ + else \ + $(ETAGS) $(ETAGSFLAGS) $(AM_ETAGSFLAGS) $(ETAGS_ARGS) \ + $$unique; \ + fi; \ + fi +ctags: CTAGS +CTAGS: $(HEADERS) $(SOURCES) $(TAGS_DEPENDENCIES) \ + $(TAGS_FILES) $(LISP) + list='$(SOURCES) $(HEADERS) $(LISP) $(TAGS_FILES)'; \ + unique=`for i in $$list; do \ + if test -f "$$i"; then echo $$i; else echo $(srcdir)/$$i; fi; \ + done | \ + $(AWK) '{ files[$$0] = 1; nonempty = 1; } \ + END { if (nonempty) { for (i in files) print i; }; }'`; \ + test -z "$(CTAGS_ARGS)$$unique" \ + || $(CTAGS) $(CTAGSFLAGS) $(AM_CTAGSFLAGS) $(CTAGS_ARGS) \ + $$unique + +GTAGS: + here=`$(am__cd) $(top_builddir) && pwd` \ + && $(am__cd) $(top_srcdir) \ + && gtags -i $(GTAGS_ARGS) "$$here" + +distclean-tags: + -rm -f TAGS ID GTAGS GRTAGS GSYMS GPATH tags + +distdir: $(DISTFILES) + @srcdirstrip=`echo "$(srcdir)" | sed 's/[].[^$$\\*]/\\\\&/g'`; \ + topsrcdirstrip=`echo "$(top_srcdir)" | sed 's/[].[^$$\\*]/\\\\&/g'`; \ + list='$(DISTFILES)'; \ + dist_files=`for file in $$list; do echo $$file; done | \ + sed -e "s|^$$srcdirstrip/||;t" \ + -e "s|^$$topsrcdirstrip/|$(top_builddir)/|;t"`; \ + case $$dist_files in \ + */*) $(MKDIR_P) `echo "$$dist_files" | \ + sed '/\//!d;s|^|$(distdir)/|;s,/[^/]*$$,,' | \ + sort -u` ;; \ + esac; \ + for file in $$dist_files; do \ + if test -f $$file || test -d $$file; then d=.; else d=$(srcdir); fi; \ + if test -d $$d/$$file; then \ + dir=`echo "/$$file" | sed -e 's,/[^/]*$$,,'`; \ + if test -d "$(distdir)/$$file"; then \ + find "$(distdir)/$$file" -type d ! -perm -700 -exec chmod u+rwx {} \;; \ + fi; \ + if test -d $(srcdir)/$$file && test $$d != $(srcdir); then \ + cp -fpR $(srcdir)/$$file "$(distdir)$$dir" || exit 1; \ + find "$(distdir)/$$file" -type d ! -perm -700 -exec chmod u+rwx {} \;; \ + fi; \ + cp -fpR $$d/$$file "$(distdir)$$dir" || exit 1; \ + else \ + test -f "$(distdir)/$$file" \ + || cp -p $$d/$$file "$(distdir)/$$file" \ + || exit 1; \ + fi; \ + done +check-am: all-am +check: check-am +all-am: Makefile $(LTLIBRARIES) +installdirs: + for dir in "$(DESTDIR)$(pkglibdir)"; do \ + test -z "$$dir" || $(MKDIR_P) "$$dir"; \ + done +install: install-am +install-exec: install-exec-am +install-data: install-data-am +uninstall: uninstall-am + +install-am: all-am + @$(MAKE) $(AM_MAKEFLAGS) install-exec-am install-data-am + +installcheck: installcheck-am +install-strip: + if test -z '$(STRIP)'; then \ + $(MAKE) $(AM_MAKEFLAGS) INSTALL_PROGRAM="$(INSTALL_STRIP_PROGRAM)" \ + install_sh_PROGRAM="$(INSTALL_STRIP_PROGRAM)" INSTALL_STRIP_FLAG=-s \ + install; \ + else \ + $(MAKE) $(AM_MAKEFLAGS) INSTALL_PROGRAM="$(INSTALL_STRIP_PROGRAM)" \ + install_sh_PROGRAM="$(INSTALL_STRIP_PROGRAM)" INSTALL_STRIP_FLAG=-s \ + "INSTALL_PROGRAM_ENV=STRIPPROG='$(STRIP)'" install; \ + fi +mostlyclean-generic: + +clean-generic: + +distclean-generic: + -test -z "$(CONFIG_CLEAN_FILES)" || rm -f $(CONFIG_CLEAN_FILES) + -test . = "$(srcdir)" || test -z "$(CONFIG_CLEAN_VPATH_FILES)" || rm -f $(CONFIG_CLEAN_VPATH_FILES) + +maintainer-clean-generic: + @echo "This command is intended for maintainers to use" + @echo "it deletes files that may require special tools to rebuild." +clean: clean-am + +clean-am: clean-generic clean-libtool clean-pkglibLTLIBRARIES \ + mostlyclean-am + +distclean: distclean-am + -rm -rf ./$(DEPDIR) + -rm -f Makefile +distclean-am: clean-am distclean-compile distclean-generic \ + distclean-tags + +dvi: dvi-am + +dvi-am: + +html: html-am + +html-am: + +info: info-am + +info-am: + +install-data-am: + +install-dvi: install-dvi-am + +install-dvi-am: + +install-exec-am: install-pkglibLTLIBRARIES + +install-html: install-html-am + +install-html-am: + +install-info: install-info-am + +install-info-am: + +install-man: + +install-pdf: install-pdf-am + +install-pdf-am: + +install-ps: install-ps-am + +install-ps-am: + +installcheck-am: + +maintainer-clean: maintainer-clean-am + -rm -rf ./$(DEPDIR) + -rm -f Makefile +maintainer-clean-am: distclean-am maintainer-clean-generic + +mostlyclean: mostlyclean-am + +mostlyclean-am: mostlyclean-compile mostlyclean-generic \ + mostlyclean-libtool + +pdf: pdf-am + +pdf-am: + +ps: ps-am + +ps-am: + +uninstall-am: uninstall-pkglibLTLIBRARIES + +.MAKE: install-am install-strip + +.PHONY: CTAGS GTAGS all all-am check check-am clean clean-generic \ + clean-libtool clean-pkglibLTLIBRARIES ctags distclean \ + distclean-compile distclean-generic distclean-libtool \ + distclean-tags distdir dvi dvi-am html html-am info info-am \ + install install-am install-data install-data-am install-dvi \ + install-dvi-am install-exec install-exec-am install-html \ + install-html-am install-info install-info-am install-man \ + install-pdf install-pdf-am install-pkglibLTLIBRARIES \ + install-ps install-ps-am install-strip installcheck \ + installcheck-am installdirs maintainer-clean \ + maintainer-clean-generic mostlyclean mostlyclean-compile \ + mostlyclean-generic mostlyclean-libtool pdf pdf-am ps ps-am \ + tags uninstall uninstall-am uninstall-pkglibLTLIBRARIES + + +# Tell versions [3.59,3.63) of GNU make to not export all variables. +# Otherwise a system limit (for SysV at least) may be exceeded. +.NOEXPORT: diff --git a/plugins/mmrfc5424addhmac/mmrfc5424addhmac.c b/plugins/mmrfc5424addhmac/mmrfc5424addhmac.c new file mode 100644 index 0000000..3a58edd --- /dev/null +++ b/plugins/mmrfc5424addhmac/mmrfc5424addhmac.c @@ -0,0 +1,395 @@ +/* mmrfc5424addhmac.c + * custom module: add hmac to RFC5424 messages + * + * Note on important design decision: This module is fully self-contained. + * Most importantly, it does not rely on mmpstrucdata to populate the + * structured data portion of the messages JSON. There are two reasons + * for this: + * 1. robustness + * - this guard against misconfiguration + * - it permits us to be more liberal in regard to malformed + * structured data + * - it permits us to handle border-cases (like duplicate + * SD-IDs) with much less complexity + * 2. performance + * With being "on the spot" of what we need we can reduce memory + * reads and writes. This is a considerable save if the JSON representation + * is not otherwise needed. + * + * Note that the recommended calling sequence if both of these modules + * are used is + * + * 1. mmrfc5424addhmac + * 2. mmpstrucdata + * + * This sequence permits mmpstrucdata to pick up the modifications we + * made in this module here. + * + * Copyright 2013 Adiscon GmbH. + * + * This file is part of rsyslog. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * -or- + * see COPYING.ASL20 in the source distribution + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +#include "config.h" +#include "rsyslog.h" +#include <stdio.h> +#include <stdarg.h> +#include <stdlib.h> +#include <string.h> +#include <assert.h> +#include <signal.h> +#include <errno.h> +#include <unistd.h> +#include <stdint.h> +#include <openssl/hmac.h> +#include "conf.h" +#include "syslogd-types.h" +#include "srUtils.h" +#include "template.h" +#include "module-template.h" +#include "errmsg.h" + +MODULE_TYPE_OUTPUT +MODULE_TYPE_NOKEEP +MODULE_CNFNAME("mmrfc5424addhmac") + + +DEFobjCurrIf(errmsg); +DEF_OMOD_STATIC_DATA + +/* config variables */ + +typedef struct _instanceData { + uchar *key; + int16_t keylen; /* cached length of key, to avoid recomputation */ + uchar *sdid; /* SD-ID to be used to persist the hmac */ + int16_t sdidLen; + const EVP_MD *algo; +} instanceData; + +typedef struct wrkrInstanceData { + instanceData *pData; +} wrkrInstanceData_t; + +struct modConfData_s { + rsconf_t *pConf; /* our overall config object */ +}; +static modConfData_t *loadModConf = NULL;/* modConf ptr to use for the current load process */ +static modConfData_t *runModConf = NULL;/* modConf ptr to use for the current exec process */ + + +/* tables for interfacing with the v6 config system */ +/* action (instance) parameters */ +static struct cnfparamdescr actpdescr[] = { + { "key", eCmdHdlrString, 1 }, + { "hashfunction", eCmdHdlrString, 1 }, + { "sd_id", eCmdHdlrGetWord, 1 } +}; +static struct cnfparamblk actpblk = + { CNFPARAMBLK_VERSION, + sizeof(actpdescr)/sizeof(struct cnfparamdescr), + actpdescr + }; + +BEGINbeginCnfLoad +CODESTARTbeginCnfLoad + loadModConf = pModConf; + pModConf->pConf = pConf; +ENDbeginCnfLoad + +BEGINendCnfLoad +CODESTARTendCnfLoad +ENDendCnfLoad + +BEGINcheckCnf +CODESTARTcheckCnf +ENDcheckCnf + +BEGINactivateCnf +CODESTARTactivateCnf + runModConf = pModConf; +ENDactivateCnf + +BEGINfreeCnf +CODESTARTfreeCnf +ENDfreeCnf + + +BEGINcreateInstance +CODESTARTcreateInstance +ENDcreateInstance + + +BEGINcreateWrkrInstance +CODESTARTcreateWrkrInstance +ENDcreateWrkrInstance + + +BEGINisCompatibleWithFeature +CODESTARTisCompatibleWithFeature +ENDisCompatibleWithFeature + + +BEGINfreeInstance +CODESTARTfreeInstance +ENDfreeInstance + + +BEGINfreeWrkrInstance +CODESTARTfreeWrkrInstance +ENDfreeWrkrInstance + + +static inline void +setInstParamDefaults(instanceData *pData) +{ + pData->key = NULL; +} + +BEGINnewActInst + struct cnfparamvals *pvals; + char *ciphername; + int i; +CODESTARTnewActInst + DBGPRINTF("newActInst (mmrfc5424addhmac)\n"); + if((pvals = nvlstGetParams(lst, &actpblk, NULL)) == NULL) { + ABORT_FINALIZE(RS_RET_MISSING_CNFPARAMS); + } + + CODE_STD_STRING_REQUESTnewActInst(1) + CHKiRet(OMSRsetEntry(*ppOMSR, 0, NULL, OMSR_TPL_AS_MSG)); + CHKiRet(createInstance(&pData)); + setInstParamDefaults(pData); + + for(i = 0 ; i < actpblk.nParams ; ++i) { + if(!pvals[i].bUsed) + continue; + if(!strcmp(actpblk.descr[i].name, "key")) { + pData->key = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + pData->keylen = es_strlen(pvals[i].val.d.estr); + } else if(!strcmp(actpblk.descr[i].name, "hashfunction")) { + ciphername = es_str2cstr(pvals[i].val.d.estr, NULL); + pData->algo = EVP_get_digestbyname(ciphername); + if(pData->algo == NULL) { + errmsg.LogError(0, RS_RET_CRY_INVLD_ALGO, + "hashFunction '%s' unknown to openssl - " + "cannot continue", ciphername); + free(ciphername); + ABORT_FINALIZE(RS_RET_CRY_INVLD_ALGO); + } + free(ciphername); + } else if(!strcmp(actpblk.descr[i].name, "sd_id")) { + pData->sdid = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + pData->sdidLen = es_strlen(pvals[i].val.d.estr); + } else { + dbgprintf("mmrfc5424addhmac: program error, non-handled " + "param '%s'\n", actpblk.descr[i].name); + } + } + +CODE_STD_FINALIZERnewActInst + cnfparamvalsDestruct(pvals, &actpblk); +ENDnewActInst + + +BEGINdbgPrintInstInfo +CODESTARTdbgPrintInstInfo +ENDdbgPrintInstInfo + + +BEGINtryResume +CODESTARTtryResume +ENDtryResume + + +/* turn the binary data in bin of length len into a + * printable hex string. "print" must be 2*len+1 (for \0) + */ +static inline void +hexify(uchar *bin, int len, uchar *print) +{ + static const char hexchars[16] = + {'0','1','2','3','4','5','6','7','8','9','a','b','c','d','e','f'}; + int iSrc, iDst; + + for(iSrc = iDst = 0 ; iSrc < len ; ++iSrc) { + print[iDst++] = hexchars[bin[iSrc]>>4]; + print[iDst++] = hexchars[bin[iSrc]&0x0f]; + } + print[iDst] = '\0'; +} + + +/* skip to end of current SD-ID. This function can be improved + * in regard to fully parsing based on RFC5424, HOWEVER, this would + * also reduce performance. So we consider the current implementation + * to be superior. + */ +static inline void +skipSDID(uchar *sdbuf, int sdlen, int *rootIdx) +{ + int i; + i = *rootIdx; + while(i < sdlen) { + if(sdbuf[i] == ']') { + if(i > *rootIdx && sdbuf[i-1] == '\\') { + ; /* escaped, nothing to do! */ + } else { + ++i; /* eat ']' */ + break; + } + } + ++i; + } + *rootIdx = i; +} + +static inline void +getSDID(uchar *sdbuf, int sdlen, int *rootIdx, uchar *sdid) +{ + int i, j; + i = *rootIdx; + j = 0; + + if(sdbuf[i] != '[') { + ++i; + goto done; + } + + ++i; + while(i < sdlen && sdbuf[i] != '=' && sdbuf[i] != ' ' + && sdbuf[i] != ']' && sdbuf[i] != '"') { + sdid[j++] = sdbuf[i++]; + } +done: + sdid[j] = '\0'; + *rootIdx = i; +} + +/* check if "our" hmac is already present */ +static inline sbool +isHmacPresent(instanceData *pData, msg_t *pMsg) +{ + uchar *sdbuf; + rs_size_t sdlen; + sbool found; + int i; + uchar sdid[33]; /* RFC-based size limit */ + + MsgGetStructuredData(pMsg, &sdbuf, &sdlen); + found = 0; + + if(sdbuf[0] == '-') /* RFC: struc data is empty! */ + goto done; + + i = 0; + while(i < sdlen && !found) { + getSDID(sdbuf, sdlen, &i, sdid); + if(!strcmp((char*)pData->sdid, (char*)sdid)) { + found = 1; + break; + } + skipSDID(sdbuf, sdlen, &i); + } + +done: + return found; +} + +static inline rsRetVal +hashMsg(instanceData *pData, msg_t *pMsg) +{ + uchar *pRawMsg; + int lenRawMsg; + uchar *sdbuf; + rs_size_t sdlen; + unsigned int hashlen; + uchar hash[EVP_MAX_MD_SIZE]; + uchar hashPrintable[2*EVP_MAX_MD_SIZE+1]; + uchar newsd[64*1024]; /* we assume this is sufficient... */ + int lenNewsd; + DEFiRet; + + MsgGetStructuredData(pMsg, &sdbuf, &sdlen); + getRawMsg(pMsg, &pRawMsg, &lenRawMsg); + HMAC(pData->algo, pData->key, pData->keylen, + pRawMsg, lenRawMsg, hash, &hashlen); + hexify(hash, hashlen, hashPrintable); + lenNewsd = snprintf((char*)newsd, sizeof(newsd), "[%s hash=\"%s\"]", + (char*)pData->sdid, (char*)hashPrintable); + MsgAddToStructuredData(pMsg, newsd, lenNewsd); + RETiRet; +} + + +BEGINdoAction + instanceData *pData = pWrkrData->pData; + msg_t *pMsg; +CODESTARTdoAction + pMsg = (msg_t*) ppString[0]; + if( msgGetProtocolVersion(pMsg) == MSG_RFC5424_PROTOCOL + && !isHmacPresent(pData, pMsg)) { + hashMsg(pData, pMsg); + } else { + if(Debug) { + uchar *pRawMsg; + int lenRawMsg; + getRawMsg(pMsg, &pRawMsg, &lenRawMsg); + dbgprintf("mmrfc5424addhmac: non-rfc5424 or HMAC already " + "present: %.256s\n", pRawMsg); + } + } +ENDdoAction + + +BEGINparseSelectorAct +CODESTARTparseSelectorAct +CODE_STD_STRING_REQUESTparseSelectorAct(1) + if(strncmp((char*) p, ":mmrfc5424addhmac:", sizeof(":mmrfc5424addhmac:") - 1)) { + errmsg.LogError(0, RS_RET_LEGA_ACT_NOT_SUPPORTED, + "mmrfc5424addhmac supports only v6+ config format, use: " + "action(type=\"mmrfc5424addhmac\" ...)"); + } + ABORT_FINALIZE(RS_RET_CONFLINE_UNPROCESSED); +CODE_STD_FINALIZERparseSelectorAct +ENDparseSelectorAct + + +BEGINmodExit +CODESTARTmodExit + objRelease(errmsg, CORE_COMPONENT); + EVP_cleanup(); +ENDmodExit + + +BEGINqueryEtryPt +CODESTARTqueryEtryPt +CODEqueryEtryPt_STD_OMOD_QUERIES +CODEqueryEtryPt_STD_CONF2_OMOD_QUERIES +CODEqueryEtryPt_STD_OMOD8_QUERIES +CODEqueryEtryPt_STD_CONF2_QUERIES +ENDqueryEtryPt + + + +BEGINmodInit() +CODESTARTmodInit + *ipIFVersProvided = CURR_MOD_IF_VERSION; +CODEmodInit_QueryRegCFSLineHdlr + DBGPRINTF("mmrfc5424addhmac: module compiled with rsyslog version %s.\n", VERSION); + OpenSSL_add_all_digests(); + CHKiRet(objUse(errmsg, CORE_COMPONENT)); +ENDmodInit diff --git a/plugins/mmsequence/Makefile.am b/plugins/mmsequence/Makefile.am new file mode 100644 index 0000000..543d6d8 --- /dev/null +++ b/plugins/mmsequence/Makefile.am @@ -0,0 +1,8 @@ +pkglib_LTLIBRARIES = mmsequence.la + +mmsequence_la_SOURCES = mmsequence.c +mmsequence_la_CPPFLAGS = $(RSRT_CFLAGS) $(PTHREADS_CFLAGS) +mmsequence_la_LDFLAGS = -module -avoid-version +mmsequence_la_LIBADD = + +EXTRA_DIST = diff --git a/plugins/mmsequence/Makefile.in b/plugins/mmsequence/Makefile.in new file mode 100644 index 0000000..a9b631e --- /dev/null +++ b/plugins/mmsequence/Makefile.in @@ -0,0 +1,637 @@ +# Makefile.in generated by automake 1.11.3 from Makefile.am. +# @configure_input@ + +# Copyright (C) 1994, 1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, +# 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010, 2011 Free Software +# Foundation, Inc. +# This Makefile.in is free software; the Free Software Foundation +# gives unlimited permission to copy and/or distribute it, +# with or without modifications, as long as this notice is preserved. + +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY, to the extent permitted by law; without +# even the implied warranty of MERCHANTABILITY or FITNESS FOR A +# PARTICULAR PURPOSE. + +@SET_MAKE@ + +VPATH = @srcdir@ +pkgdatadir = $(datadir)/@PACKAGE@ +pkgincludedir = $(includedir)/@PACKAGE@ +pkglibdir = $(libdir)/@PACKAGE@ +pkglibexecdir = $(libexecdir)/@PACKAGE@ +am__cd = CDPATH="$${ZSH_VERSION+.}$(PATH_SEPARATOR)" && cd +install_sh_DATA = $(install_sh) -c -m 644 +install_sh_PROGRAM = $(install_sh) -c +install_sh_SCRIPT = $(install_sh) -c +INSTALL_HEADER = $(INSTALL_DATA) +transform = $(program_transform_name) +NORMAL_INSTALL = : +PRE_INSTALL = : +POST_INSTALL = : +NORMAL_UNINSTALL = : +PRE_UNINSTALL = : +POST_UNINSTALL = : +build_triplet = @build@ +host_triplet = @host@ +subdir = plugins/mmsequence +DIST_COMMON = $(srcdir)/Makefile.am $(srcdir)/Makefile.in +ACLOCAL_M4 = $(top_srcdir)/aclocal.m4 +am__aclocal_m4_deps = $(top_srcdir)/m4/atomic_operations.m4 \ + $(top_srcdir)/m4/atomic_operations_64bit.m4 \ + $(top_srcdir)/m4/libtool.m4 $(top_srcdir)/m4/ltoptions.m4 \ + $(top_srcdir)/m4/ltsugar.m4 $(top_srcdir)/m4/ltversion.m4 \ + $(top_srcdir)/m4/lt~obsolete.m4 $(top_srcdir)/configure.ac +am__configure_deps = $(am__aclocal_m4_deps) $(CONFIGURE_DEPENDENCIES) \ + $(ACLOCAL_M4) +mkinstalldirs = $(install_sh) -d +CONFIG_HEADER = $(top_builddir)/config.h +CONFIG_CLEAN_FILES = +CONFIG_CLEAN_VPATH_FILES = +am__vpath_adj_setup = srcdirstrip=`echo "$(srcdir)" | sed 's|.|.|g'`; +am__vpath_adj = case $$p in \ + $(srcdir)/*) f=`echo "$$p" | sed "s|^$$srcdirstrip/||"`;; \ + *) f=$$p;; \ + esac; +am__strip_dir = f=`echo $$p | sed -e 's|^.*/||'`; +am__install_max = 40 +am__nobase_strip_setup = \ + srcdirstrip=`echo "$(srcdir)" | sed 's/[].[^$$\\*|]/\\\\&/g'` +am__nobase_strip = \ + for p in $$list; do echo "$$p"; done | sed -e "s|$$srcdirstrip/||" +am__nobase_list = $(am__nobase_strip_setup); \ + for p in $$list; do echo "$$p $$p"; done | \ + sed "s| $$srcdirstrip/| |;"' / .*\//!s/ .*/ ./; s,\( .*\)/[^/]*$$,\1,' | \ + $(AWK) 'BEGIN { files["."] = "" } { files[$$2] = files[$$2] " " $$1; \ + if (++n[$$2] == $(am__install_max)) \ + { print $$2, files[$$2]; n[$$2] = 0; files[$$2] = "" } } \ + END { for (dir in files) print dir, files[dir] }' +am__base_list = \ + sed '$$!N;$$!N;$$!N;$$!N;$$!N;$$!N;$$!N;s/\n/ /g' | \ + sed '$$!N;$$!N;$$!N;$$!N;s/\n/ /g' +am__uninstall_files_from_dir = { \ + test -z "$$files" \ + || { test ! -d "$$dir" && test ! -f "$$dir" && test ! -r "$$dir"; } \ + || { echo " ( cd '$$dir' && rm -f" $$files ")"; \ + $(am__cd) "$$dir" && rm -f $$files; }; \ + } +am__installdirs = "$(DESTDIR)$(pkglibdir)" +LTLIBRARIES = $(pkglib_LTLIBRARIES) +mmsequence_la_DEPENDENCIES = +am_mmsequence_la_OBJECTS = mmsequence_la-mmsequence.lo +mmsequence_la_OBJECTS = $(am_mmsequence_la_OBJECTS) +AM_V_lt = $(am__v_lt_@AM_V@) +am__v_lt_ = $(am__v_lt_@AM_DEFAULT_V@) +am__v_lt_0 = --silent +mmsequence_la_LINK = $(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \ + $(LIBTOOLFLAGS) --mode=link $(CCLD) $(AM_CFLAGS) $(CFLAGS) \ + $(mmsequence_la_LDFLAGS) $(LDFLAGS) -o $@ +DEFAULT_INCLUDES = -I.@am__isrc@ -I$(top_builddir) +depcomp = $(SHELL) $(top_srcdir)/depcomp +am__depfiles_maybe = depfiles +am__mv = mv -f +COMPILE = $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(AM_CPPFLAGS) \ + $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) +LTCOMPILE = $(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \ + $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) \ + $(DEFAULT_INCLUDES) $(INCLUDES) $(AM_CPPFLAGS) $(CPPFLAGS) \ + $(AM_CFLAGS) $(CFLAGS) +AM_V_CC = $(am__v_CC_@AM_V@) +am__v_CC_ = $(am__v_CC_@AM_DEFAULT_V@) +am__v_CC_0 = @echo " CC " $@; +AM_V_at = $(am__v_at_@AM_V@) +am__v_at_ = $(am__v_at_@AM_DEFAULT_V@) +am__v_at_0 = @ +CCLD = $(CC) +LINK = $(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \ + $(LIBTOOLFLAGS) --mode=link $(CCLD) $(AM_CFLAGS) $(CFLAGS) \ + $(AM_LDFLAGS) $(LDFLAGS) -o $@ +AM_V_CCLD = $(am__v_CCLD_@AM_V@) +am__v_CCLD_ = $(am__v_CCLD_@AM_DEFAULT_V@) +am__v_CCLD_0 = @echo " CCLD " $@; +AM_V_GEN = $(am__v_GEN_@AM_V@) +am__v_GEN_ = $(am__v_GEN_@AM_DEFAULT_V@) +am__v_GEN_0 = @echo " GEN " $@; +SOURCES = $(mmsequence_la_SOURCES) +DIST_SOURCES = $(mmsequence_la_SOURCES) +ETAGS = etags +CTAGS = ctags +DISTFILES = $(DIST_COMMON) $(DIST_SOURCES) $(TEXINFOS) $(EXTRA_DIST) +ACLOCAL = @ACLOCAL@ +AMTAR = @AMTAR@ +AM_DEFAULT_VERBOSITY = @AM_DEFAULT_VERBOSITY@ +AR = @AR@ +AUTOCONF = @AUTOCONF@ +AUTOHEADER = @AUTOHEADER@ +AUTOMAKE = @AUTOMAKE@ +AWK = @AWK@ +CC = @CC@ +CCDEPMODE = @CCDEPMODE@ +CFLAGS = @CFLAGS@ +CPP = @CPP@ +CPPFLAGS = @CPPFLAGS@ +CURL_CFLAGS = @CURL_CFLAGS@ +CURL_LIBS = @CURL_LIBS@ +CYGPATH_W = @CYGPATH_W@ +CZMQ_CFLAGS = @CZMQ_CFLAGS@ +CZMQ_LIBS = @CZMQ_LIBS@ +DEFS = @DEFS@ +DEPDIR = @DEPDIR@ +DLLTOOL = @DLLTOOL@ +DL_LIBS = @DL_LIBS@ +DSYMUTIL = @DSYMUTIL@ +DUMPBIN = @DUMPBIN@ +ECHO_C = @ECHO_C@ +ECHO_N = @ECHO_N@ +ECHO_T = @ECHO_T@ +EGREP = @EGREP@ +EXEEXT = @EXEEXT@ +FGREP = @FGREP@ +GNUTLS_CFLAGS = @GNUTLS_CFLAGS@ +GNUTLS_LIBS = @GNUTLS_LIBS@ +GREP = @GREP@ +GSS_LIBS = @GSS_LIBS@ +GUARDTIME_CFLAGS = @GUARDTIME_CFLAGS@ +GUARDTIME_LIBS = @GUARDTIME_LIBS@ +HAVE_MYSQL_CONFIG = @HAVE_MYSQL_CONFIG@ +HAVE_ORACLE_CONFIG = @HAVE_ORACLE_CONFIG@ +HAVE_PGSQL_CONFIG = @HAVE_PGSQL_CONFIG@ +HIREDIS_CFLAGS = @HIREDIS_CFLAGS@ +HIREDIS_LIBS = @HIREDIS_LIBS@ +IMUDP_LIBS = @IMUDP_LIBS@ +INSTALL = @INSTALL@ +INSTALL_DATA = @INSTALL_DATA@ +INSTALL_PROGRAM = @INSTALL_PROGRAM@ +INSTALL_SCRIPT = @INSTALL_SCRIPT@ +INSTALL_STRIP_PROGRAM = @INSTALL_STRIP_PROGRAM@ +JSON_C_CFLAGS = @JSON_C_CFLAGS@ +JSON_C_LIBS = @JSON_C_LIBS@ +LD = @LD@ +LDFLAGS = @LDFLAGS@ +LEX = @LEX@ +LEXLIB = @LEXLIB@ +LEX_OUTPUT_ROOT = @LEX_OUTPUT_ROOT@ +LIBDBI_CFLAGS = @LIBDBI_CFLAGS@ +LIBDBI_LIBS = @LIBDBI_LIBS@ +LIBESTR_CFLAGS = @LIBESTR_CFLAGS@ +LIBESTR_LIBS = @LIBESTR_LIBS@ +LIBGCRYPT_CFLAGS = @LIBGCRYPT_CFLAGS@ +LIBGCRYPT_CONFIG = @LIBGCRYPT_CONFIG@ +LIBGCRYPT_LIBS = @LIBGCRYPT_LIBS@ +LIBLOGGING_CFLAGS = @LIBLOGGING_CFLAGS@ +LIBLOGGING_LIBS = @LIBLOGGING_LIBS@ +LIBLOGGING_STDLOG_CFLAGS = @LIBLOGGING_STDLOG_CFLAGS@ +LIBLOGGING_STDLOG_LIBS = @LIBLOGGING_STDLOG_LIBS@ +LIBLOGNORM_CFLAGS = @LIBLOGNORM_CFLAGS@ +LIBLOGNORM_LIBS = @LIBLOGNORM_LIBS@ +LIBM = @LIBM@ +LIBMONGO_CLIENT_CFLAGS = @LIBMONGO_CLIENT_CFLAGS@ +LIBMONGO_CLIENT_LIBS = @LIBMONGO_CLIENT_LIBS@ +LIBOBJS = @LIBOBJS@ +LIBS = @LIBS@ +LIBSYSTEMD_JOURNAL_CFLAGS = @LIBSYSTEMD_JOURNAL_CFLAGS@ +LIBSYSTEMD_JOURNAL_LIBS = @LIBSYSTEMD_JOURNAL_LIBS@ +LIBTOOL = @LIBTOOL@ +LIBUUID_CFLAGS = @LIBUUID_CFLAGS@ +LIBUUID_LIBS = @LIBUUID_LIBS@ +LIPO = @LIPO@ +LN_S = @LN_S@ +LTLIBOBJS = @LTLIBOBJS@ +MAKEINFO = @MAKEINFO@ +MANIFEST_TOOL = @MANIFEST_TOOL@ +MKDIR_P = @MKDIR_P@ +MYSQL_CFLAGS = @MYSQL_CFLAGS@ +MYSQL_LIBS = @MYSQL_LIBS@ +NM = @NM@ +NMEDIT = @NMEDIT@ +OBJDUMP = @OBJDUMP@ +OBJEXT = @OBJEXT@ +OPENSSL_CFLAGS = @OPENSSL_CFLAGS@ +OPENSSL_LIBS = @OPENSSL_LIBS@ +ORACLE_CFLAGS = @ORACLE_CFLAGS@ +ORACLE_LIBS = @ORACLE_LIBS@ +OTOOL = @OTOOL@ +OTOOL64 = @OTOOL64@ +PACKAGE = @PACKAGE@ +PACKAGE_BUGREPORT = @PACKAGE_BUGREPORT@ +PACKAGE_NAME = @PACKAGE_NAME@ +PACKAGE_STRING = @PACKAGE_STRING@ +PACKAGE_TARNAME = @PACKAGE_TARNAME@ +PACKAGE_URL = @PACKAGE_URL@ +PACKAGE_VERSION = @PACKAGE_VERSION@ +PATH_SEPARATOR = @PATH_SEPARATOR@ +PGSQL_CFLAGS = @PGSQL_CFLAGS@ +PGSQL_LIBS = @PGSQL_LIBS@ +PKG_CONFIG = @PKG_CONFIG@ +PKG_CONFIG_LIBDIR = @PKG_CONFIG_LIBDIR@ +PKG_CONFIG_PATH = @PKG_CONFIG_PATH@ +PTHREADS_CFLAGS = @PTHREADS_CFLAGS@ +PTHREADS_LIBS = @PTHREADS_LIBS@ +RABBITMQ_CFLAGS = @RABBITMQ_CFLAGS@ +RABBITMQ_LIBS = @RABBITMQ_LIBS@ +RANLIB = @RANLIB@ +RELP_CFLAGS = @RELP_CFLAGS@ +RELP_LIBS = @RELP_LIBS@ +RSRT_CFLAGS = @RSRT_CFLAGS@ +RSRT_CFLAGS1 = @RSRT_CFLAGS1@ +RSRT_LIBS = @RSRT_LIBS@ +RSRT_LIBS1 = @RSRT_LIBS1@ +RST2MAN = @RST2MAN@ +RT_LIBS = @RT_LIBS@ +SED = @SED@ +SET_MAKE = @SET_MAKE@ +SHELL = @SHELL@ +SNMP_CFLAGS = @SNMP_CFLAGS@ +SNMP_LIBS = @SNMP_LIBS@ +SOL_LIBS = @SOL_LIBS@ +STRIP = @STRIP@ +UDPSPOOF_CFLAGS = @UDPSPOOF_CFLAGS@ +UDPSPOOF_LIBS = @UDPSPOOF_LIBS@ +VERSION = @VERSION@ +YACC = @YACC@ +YFLAGS = @YFLAGS@ +ZLIB_LIBS = @ZLIB_LIBS@ +abs_builddir = @abs_builddir@ +abs_srcdir = @abs_srcdir@ +abs_top_builddir = @abs_top_builddir@ +abs_top_srcdir = @abs_top_srcdir@ +ac_ct_AR = @ac_ct_AR@ +ac_ct_CC = @ac_ct_CC@ +ac_ct_DUMPBIN = @ac_ct_DUMPBIN@ +am__include = @am__include@ +am__leading_dot = @am__leading_dot@ +am__quote = @am__quote@ +am__tar = @am__tar@ +am__untar = @am__untar@ +bindir = @bindir@ +build = @build@ +build_alias = @build_alias@ +build_cpu = @build_cpu@ +build_os = @build_os@ +build_vendor = @build_vendor@ +builddir = @builddir@ +datadir = @datadir@ +datarootdir = @datarootdir@ +docdir = @docdir@ +dvidir = @dvidir@ +exec_prefix = @exec_prefix@ +have_valgrind = @have_valgrind@ +host = @host@ +host_alias = @host_alias@ +host_cpu = @host_cpu@ +host_os = @host_os@ +host_vendor = @host_vendor@ +htmldir = @htmldir@ +includedir = @includedir@ +infodir = @infodir@ +install_sh = @install_sh@ +libdir = @libdir@ +libexecdir = @libexecdir@ +localedir = @localedir@ +localstatedir = @localstatedir@ +mandir = @mandir@ +mkdir_p = @mkdir_p@ +moddirs = @moddirs@ +oldincludedir = @oldincludedir@ +pdfdir = @pdfdir@ +prefix = @prefix@ +program_transform_name = @program_transform_name@ +psdir = @psdir@ +sbindir = @sbindir@ +sharedstatedir = @sharedstatedir@ +srcdir = @srcdir@ +sysconfdir = @sysconfdir@ +systemdsystemunitdir = @systemdsystemunitdir@ +target_alias = @target_alias@ +top_build_prefix = @top_build_prefix@ +top_builddir = @top_builddir@ +top_srcdir = @top_srcdir@ +pkglib_LTLIBRARIES = mmsequence.la +mmsequence_la_SOURCES = mmsequence.c +mmsequence_la_CPPFLAGS = $(RSRT_CFLAGS) $(PTHREADS_CFLAGS) +mmsequence_la_LDFLAGS = -module -avoid-version +mmsequence_la_LIBADD = +EXTRA_DIST = +all: all-am + +.SUFFIXES: +.SUFFIXES: .c .lo .o .obj +$(srcdir)/Makefile.in: $(srcdir)/Makefile.am $(am__configure_deps) + @for dep in $?; do \ + case '$(am__configure_deps)' in \ + *$$dep*) \ + ( cd $(top_builddir) && $(MAKE) $(AM_MAKEFLAGS) am--refresh ) \ + && { if test -f $@; then exit 0; else break; fi; }; \ + exit 1;; \ + esac; \ + done; \ + echo ' cd $(top_srcdir) && $(AUTOMAKE) --gnu plugins/mmsequence/Makefile'; \ + $(am__cd) $(top_srcdir) && \ + $(AUTOMAKE) --gnu plugins/mmsequence/Makefile +.PRECIOUS: Makefile +Makefile: $(srcdir)/Makefile.in $(top_builddir)/config.status + @case '$?' in \ + *config.status*) \ + cd $(top_builddir) && $(MAKE) $(AM_MAKEFLAGS) am--refresh;; \ + *) \ + echo ' cd $(top_builddir) && $(SHELL) ./config.status $(subdir)/$@ $(am__depfiles_maybe)'; \ + cd $(top_builddir) && $(SHELL) ./config.status $(subdir)/$@ $(am__depfiles_maybe);; \ + esac; + +$(top_builddir)/config.status: $(top_srcdir)/configure $(CONFIG_STATUS_DEPENDENCIES) + cd $(top_builddir) && $(MAKE) $(AM_MAKEFLAGS) am--refresh + +$(top_srcdir)/configure: $(am__configure_deps) + cd $(top_builddir) && $(MAKE) $(AM_MAKEFLAGS) am--refresh +$(ACLOCAL_M4): $(am__aclocal_m4_deps) + cd $(top_builddir) && $(MAKE) $(AM_MAKEFLAGS) am--refresh +$(am__aclocal_m4_deps): +install-pkglibLTLIBRARIES: $(pkglib_LTLIBRARIES) + @$(NORMAL_INSTALL) + test -z "$(pkglibdir)" || $(MKDIR_P) "$(DESTDIR)$(pkglibdir)" + @list='$(pkglib_LTLIBRARIES)'; test -n "$(pkglibdir)" || list=; \ + list2=; for p in $$list; do \ + if test -f $$p; then \ + list2="$$list2 $$p"; \ + else :; fi; \ + done; \ + test -z "$$list2" || { \ + echo " $(LIBTOOL) $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=install $(INSTALL) $(INSTALL_STRIP_FLAG) $$list2 '$(DESTDIR)$(pkglibdir)'"; \ + $(LIBTOOL) $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=install $(INSTALL) $(INSTALL_STRIP_FLAG) $$list2 "$(DESTDIR)$(pkglibdir)"; \ + } + +uninstall-pkglibLTLIBRARIES: + @$(NORMAL_UNINSTALL) + @list='$(pkglib_LTLIBRARIES)'; test -n "$(pkglibdir)" || list=; \ + for p in $$list; do \ + $(am__strip_dir) \ + echo " $(LIBTOOL) $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=uninstall rm -f '$(DESTDIR)$(pkglibdir)/$$f'"; \ + $(LIBTOOL) $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=uninstall rm -f "$(DESTDIR)$(pkglibdir)/$$f"; \ + done + +clean-pkglibLTLIBRARIES: + -test -z "$(pkglib_LTLIBRARIES)" || rm -f $(pkglib_LTLIBRARIES) + @list='$(pkglib_LTLIBRARIES)'; for p in $$list; do \ + dir="`echo $$p | sed -e 's|/[^/]*$$||'`"; \ + test "$$dir" != "$$p" || dir=.; \ + echo "rm -f \"$${dir}/so_locations\""; \ + rm -f "$${dir}/so_locations"; \ + done +mmsequence.la: $(mmsequence_la_OBJECTS) $(mmsequence_la_DEPENDENCIES) $(EXTRA_mmsequence_la_DEPENDENCIES) + $(AM_V_CCLD)$(mmsequence_la_LINK) -rpath $(pkglibdir) $(mmsequence_la_OBJECTS) $(mmsequence_la_LIBADD) $(LIBS) + +mostlyclean-compile: + -rm -f *.$(OBJEXT) + +distclean-compile: + -rm -f *.tab.c + +@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/mmsequence_la-mmsequence.Plo@am__quote@ + +.c.o: +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.o$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ $< + +.c.obj: +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.obj$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ `$(CYGPATH_W) '$<'` &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ `$(CYGPATH_W) '$<'` + +.c.lo: +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.lo$$||'`;\ +@am__fastdepCC_TRUE@ $(LTCOMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Plo +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=yes @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(LTCOMPILE) -c -o $@ $< + +mmsequence_la-mmsequence.lo: mmsequence.c +@am__fastdepCC_TRUE@ $(AM_V_CC)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(mmsequence_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT mmsequence_la-mmsequence.lo -MD -MP -MF $(DEPDIR)/mmsequence_la-mmsequence.Tpo -c -o mmsequence_la-mmsequence.lo `test -f 'mmsequence.c' || echo '$(srcdir)/'`mmsequence.c +@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/mmsequence_la-mmsequence.Tpo $(DEPDIR)/mmsequence_la-mmsequence.Plo +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='mmsequence.c' object='mmsequence_la-mmsequence.lo' libtool=yes @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(mmsequence_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o mmsequence_la-mmsequence.lo `test -f 'mmsequence.c' || echo '$(srcdir)/'`mmsequence.c + +mostlyclean-libtool: + -rm -f *.lo + +clean-libtool: + -rm -rf .libs _libs + +ID: $(HEADERS) $(SOURCES) $(LISP) $(TAGS_FILES) + list='$(SOURCES) $(HEADERS) $(LISP) $(TAGS_FILES)'; \ + unique=`for i in $$list; do \ + if test -f "$$i"; then echo $$i; else echo $(srcdir)/$$i; fi; \ + done | \ + $(AWK) '{ files[$$0] = 1; nonempty = 1; } \ + END { if (nonempty) { for (i in files) print i; }; }'`; \ + mkid -fID $$unique +tags: TAGS + +TAGS: $(HEADERS) $(SOURCES) $(TAGS_DEPENDENCIES) \ + $(TAGS_FILES) $(LISP) + set x; \ + here=`pwd`; \ + list='$(SOURCES) $(HEADERS) $(LISP) $(TAGS_FILES)'; \ + unique=`for i in $$list; do \ + if test -f "$$i"; then echo $$i; else echo $(srcdir)/$$i; fi; \ + done | \ + $(AWK) '{ files[$$0] = 1; nonempty = 1; } \ + END { if (nonempty) { for (i in files) print i; }; }'`; \ + shift; \ + if test -z "$(ETAGS_ARGS)$$*$$unique"; then :; else \ + test -n "$$unique" || unique=$$empty_fix; \ + if test $$# -gt 0; then \ + $(ETAGS) $(ETAGSFLAGS) $(AM_ETAGSFLAGS) $(ETAGS_ARGS) \ + "$$@" $$unique; \ + else \ + $(ETAGS) $(ETAGSFLAGS) $(AM_ETAGSFLAGS) $(ETAGS_ARGS) \ + $$unique; \ + fi; \ + fi +ctags: CTAGS +CTAGS: $(HEADERS) $(SOURCES) $(TAGS_DEPENDENCIES) \ + $(TAGS_FILES) $(LISP) + list='$(SOURCES) $(HEADERS) $(LISP) $(TAGS_FILES)'; \ + unique=`for i in $$list; do \ + if test -f "$$i"; then echo $$i; else echo $(srcdir)/$$i; fi; \ + done | \ + $(AWK) '{ files[$$0] = 1; nonempty = 1; } \ + END { if (nonempty) { for (i in files) print i; }; }'`; \ + test -z "$(CTAGS_ARGS)$$unique" \ + || $(CTAGS) $(CTAGSFLAGS) $(AM_CTAGSFLAGS) $(CTAGS_ARGS) \ + $$unique + +GTAGS: + here=`$(am__cd) $(top_builddir) && pwd` \ + && $(am__cd) $(top_srcdir) \ + && gtags -i $(GTAGS_ARGS) "$$here" + +distclean-tags: + -rm -f TAGS ID GTAGS GRTAGS GSYMS GPATH tags + +distdir: $(DISTFILES) + @srcdirstrip=`echo "$(srcdir)" | sed 's/[].[^$$\\*]/\\\\&/g'`; \ + topsrcdirstrip=`echo "$(top_srcdir)" | sed 's/[].[^$$\\*]/\\\\&/g'`; \ + list='$(DISTFILES)'; \ + dist_files=`for file in $$list; do echo $$file; done | \ + sed -e "s|^$$srcdirstrip/||;t" \ + -e "s|^$$topsrcdirstrip/|$(top_builddir)/|;t"`; \ + case $$dist_files in \ + */*) $(MKDIR_P) `echo "$$dist_files" | \ + sed '/\//!d;s|^|$(distdir)/|;s,/[^/]*$$,,' | \ + sort -u` ;; \ + esac; \ + for file in $$dist_files; do \ + if test -f $$file || test -d $$file; then d=.; else d=$(srcdir); fi; \ + if test -d $$d/$$file; then \ + dir=`echo "/$$file" | sed -e 's,/[^/]*$$,,'`; \ + if test -d "$(distdir)/$$file"; then \ + find "$(distdir)/$$file" -type d ! -perm -700 -exec chmod u+rwx {} \;; \ + fi; \ + if test -d $(srcdir)/$$file && test $$d != $(srcdir); then \ + cp -fpR $(srcdir)/$$file "$(distdir)$$dir" || exit 1; \ + find "$(distdir)/$$file" -type d ! -perm -700 -exec chmod u+rwx {} \;; \ + fi; \ + cp -fpR $$d/$$file "$(distdir)$$dir" || exit 1; \ + else \ + test -f "$(distdir)/$$file" \ + || cp -p $$d/$$file "$(distdir)/$$file" \ + || exit 1; \ + fi; \ + done +check-am: all-am +check: check-am +all-am: Makefile $(LTLIBRARIES) +installdirs: + for dir in "$(DESTDIR)$(pkglibdir)"; do \ + test -z "$$dir" || $(MKDIR_P) "$$dir"; \ + done +install: install-am +install-exec: install-exec-am +install-data: install-data-am +uninstall: uninstall-am + +install-am: all-am + @$(MAKE) $(AM_MAKEFLAGS) install-exec-am install-data-am + +installcheck: installcheck-am +install-strip: + if test -z '$(STRIP)'; then \ + $(MAKE) $(AM_MAKEFLAGS) INSTALL_PROGRAM="$(INSTALL_STRIP_PROGRAM)" \ + install_sh_PROGRAM="$(INSTALL_STRIP_PROGRAM)" INSTALL_STRIP_FLAG=-s \ + install; \ + else \ + $(MAKE) $(AM_MAKEFLAGS) INSTALL_PROGRAM="$(INSTALL_STRIP_PROGRAM)" \ + install_sh_PROGRAM="$(INSTALL_STRIP_PROGRAM)" INSTALL_STRIP_FLAG=-s \ + "INSTALL_PROGRAM_ENV=STRIPPROG='$(STRIP)'" install; \ + fi +mostlyclean-generic: + +clean-generic: + +distclean-generic: + -test -z "$(CONFIG_CLEAN_FILES)" || rm -f $(CONFIG_CLEAN_FILES) + -test . = "$(srcdir)" || test -z "$(CONFIG_CLEAN_VPATH_FILES)" || rm -f $(CONFIG_CLEAN_VPATH_FILES) + +maintainer-clean-generic: + @echo "This command is intended for maintainers to use" + @echo "it deletes files that may require special tools to rebuild." +clean: clean-am + +clean-am: clean-generic clean-libtool clean-pkglibLTLIBRARIES \ + mostlyclean-am + +distclean: distclean-am + -rm -rf ./$(DEPDIR) + -rm -f Makefile +distclean-am: clean-am distclean-compile distclean-generic \ + distclean-tags + +dvi: dvi-am + +dvi-am: + +html: html-am + +html-am: + +info: info-am + +info-am: + +install-data-am: + +install-dvi: install-dvi-am + +install-dvi-am: + +install-exec-am: install-pkglibLTLIBRARIES + +install-html: install-html-am + +install-html-am: + +install-info: install-info-am + +install-info-am: + +install-man: + +install-pdf: install-pdf-am + +install-pdf-am: + +install-ps: install-ps-am + +install-ps-am: + +installcheck-am: + +maintainer-clean: maintainer-clean-am + -rm -rf ./$(DEPDIR) + -rm -f Makefile +maintainer-clean-am: distclean-am maintainer-clean-generic + +mostlyclean: mostlyclean-am + +mostlyclean-am: mostlyclean-compile mostlyclean-generic \ + mostlyclean-libtool + +pdf: pdf-am + +pdf-am: + +ps: ps-am + +ps-am: + +uninstall-am: uninstall-pkglibLTLIBRARIES + +.MAKE: install-am install-strip + +.PHONY: CTAGS GTAGS all all-am check check-am clean clean-generic \ + clean-libtool clean-pkglibLTLIBRARIES ctags distclean \ + distclean-compile distclean-generic distclean-libtool \ + distclean-tags distdir dvi dvi-am html html-am info info-am \ + install install-am install-data install-data-am install-dvi \ + install-dvi-am install-exec install-exec-am install-html \ + install-html-am install-info install-info-am install-man \ + install-pdf install-pdf-am install-pkglibLTLIBRARIES \ + install-ps install-ps-am install-strip installcheck \ + installcheck-am installdirs maintainer-clean \ + maintainer-clean-generic mostlyclean mostlyclean-compile \ + mostlyclean-generic mostlyclean-libtool pdf pdf-am ps ps-am \ + tags uninstall uninstall-am uninstall-pkglibLTLIBRARIES + + +# Tell versions [3.59,3.63) of GNU make to not export all variables. +# Otherwise a system limit (for SysV at least) may be exceeded. +.NOEXPORT: diff --git a/plugins/mmsequence/mmsequence.c b/plugins/mmsequence/mmsequence.c new file mode 100644 index 0000000..ee45587 --- /dev/null +++ b/plugins/mmsequence/mmsequence.c @@ -0,0 +1,419 @@ +/* mmsequence.c + * Generate a number based on some sequence. + * + * Copyright 2013 pavel@levshin.spb.ru. + * + * Based on: mmcount.c + * Copyright 2013 Red Hat Inc. + * + * This file is part of rsyslog. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * -or- + * see COPYING.ASL20 in the source distribution + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +#include "config.h" +#include "rsyslog.h" +#include <stdio.h> +#include <stdarg.h> +#include <stdlib.h> +#include <string.h> +#include <assert.h> +#include <signal.h> +#include <errno.h> +#include <unistd.h> +#include <stdint.h> +#include <time.h> +#include <limits.h> +#include <json.h> +#include <pthread.h> +#include "conf.h" +#include "syslogd-types.h" +#include "srUtils.h" +#include "template.h" +#include "module-template.h" +#include "errmsg.h" +#include "hashtable.h" + +#define JSON_VAR_NAME "$!mmsequence" + +enum mmSequenceModes { + mmSequenceRandom, + mmSequencePerInstance, + mmSequencePerKey +}; + +MODULE_TYPE_OUTPUT +MODULE_TYPE_NOKEEP +MODULE_CNFNAME("mmsequence") + + +DEFobjCurrIf(errmsg); +DEF_OMOD_STATIC_DATA + +/* config variables */ + +typedef struct _instanceData { + enum mmSequenceModes mode; + int valueFrom; + int valueTo; + int step; + unsigned int seed; + int value; + char *pszKey; + char *pszVar; +} instanceData; + +typedef struct wrkrInstanceData { + instanceData *pData; +} wrkrInstanceData_t; + +struct modConfData_s { + rsconf_t *pConf; /* our overall config object */ +}; +static modConfData_t *loadModConf = NULL;/* modConf ptr to use for the current load process */ +static modConfData_t *runModConf = NULL;/* modConf ptr to use for the current exec process */ + + +/* tables for interfacing with the v6 config system */ +/* action (instance) parameters */ +static struct cnfparamdescr actpdescr[] = { + { "mode", eCmdHdlrGetWord, 0 }, + { "from", eCmdHdlrNonNegInt, 0 }, + { "to", eCmdHdlrPositiveInt, 0 }, + { "step", eCmdHdlrNonNegInt, 0 }, + { "key", eCmdHdlrGetWord, 0 }, + { "var", eCmdHdlrGetWord, 0 }, +}; +static struct cnfparamblk actpblk = + { CNFPARAMBLK_VERSION, + sizeof(actpdescr)/sizeof(struct cnfparamdescr), + actpdescr + }; + +/* table for key-counter pairs */ +static struct hashtable *ght; +static pthread_mutex_t ght_mutex = PTHREAD_MUTEX_INITIALIZER; + +static pthread_mutex_t inst_mutex = PTHREAD_MUTEX_INITIALIZER; + +BEGINbeginCnfLoad +CODESTARTbeginCnfLoad + loadModConf = pModConf; + pModConf->pConf = pConf; +ENDbeginCnfLoad + +BEGINendCnfLoad +CODESTARTendCnfLoad +ENDendCnfLoad + +BEGINcheckCnf +CODESTARTcheckCnf +ENDcheckCnf + +BEGINactivateCnf +CODESTARTactivateCnf + runModConf = pModConf; +ENDactivateCnf + +BEGINfreeCnf +CODESTARTfreeCnf +ENDfreeCnf + + +BEGINcreateInstance +CODESTARTcreateInstance +ENDcreateInstance + +BEGINcreateWrkrInstance +CODESTARTcreateWrkrInstance +ENDcreateWrkrInstance + + +BEGINisCompatibleWithFeature +CODESTARTisCompatibleWithFeature +ENDisCompatibleWithFeature + + +BEGINfreeInstance +CODESTARTfreeInstance +ENDfreeInstance + +BEGINfreeWrkrInstance +CODESTARTfreeWrkrInstance +ENDfreeWrkrInstance + + +static inline void +setInstParamDefaults(instanceData *pData) +{ + pData->mode = mmSequencePerInstance; + pData->valueFrom = 0; + pData->valueTo = INT_MAX; + pData->step = 1; + pData->pszKey = ""; + pData->pszVar = JSON_VAR_NAME; +} + +BEGINnewActInst + struct cnfparamvals *pvals; + int i; + char *cstr; +CODESTARTnewActInst + DBGPRINTF("newActInst (mmsequence)\n"); + if((pvals = nvlstGetParams(lst, &actpblk, NULL)) == NULL) { + ABORT_FINALIZE(RS_RET_MISSING_CNFPARAMS); + } + + CODE_STD_STRING_REQUESTnewActInst(1) + CHKiRet(OMSRsetEntry(*ppOMSR, 0, NULL, OMSR_TPL_AS_MSG)); + CHKiRet(createInstance(&pData)); + setInstParamDefaults(pData); + + for(i = 0 ; i < actpblk.nParams ; ++i) { + if(!pvals[i].bUsed) + continue; + if(!strcmp(actpblk.descr[i].name, "mode")) { + if(!es_strbufcmp(pvals[i].val.d.estr, (uchar*)"random", + sizeof("random")-1)) { + pData->mode = mmSequenceRandom; + } else if (!es_strbufcmp(pvals[i].val.d.estr, (uchar*)"instance", + sizeof("instance")-1)) { + pData->mode = mmSequencePerInstance; + } else if (!es_strbufcmp(pvals[i].val.d.estr, (uchar*)"key", + sizeof("key")-1)) { + pData->mode = mmSequencePerKey; + } else { + cstr = es_str2cstr(pvals[i].val.d.estr, NULL); + errmsg.LogError(0, RS_RET_INVLD_MODE, + "mmsequence: invalid mode '%s' - ignored", + cstr); + free(cstr); + } + continue; + } + if(!strcmp(actpblk.descr[i].name, "from")) { + pData->valueFrom = pvals[i].val.d.n; + continue; + } + if(!strcmp(actpblk.descr[i].name, "to")) { + pData->valueTo = pvals[i].val.d.n; + continue; + } + if(!strcmp(actpblk.descr[i].name, "step")) { + pData->step = pvals[i].val.d.n; + continue; + } + if(!strcmp(actpblk.descr[i].name, "key")) { + pData->pszKey = es_str2cstr(pvals[i].val.d.estr, NULL); + continue; + } + if(!strcmp(actpblk.descr[i].name, "var")) { + cstr = es_str2cstr(pvals[i].val.d.estr, NULL); + if (strlen(cstr) < 3) { + errmsg.LogError(0, RS_RET_VALUE_NOT_SUPPORTED, + "mmsequence: valid variable name should be at least " + "3 symbols long, got %s", cstr); + free(cstr); + } else if (cstr[0] != '$') { + errmsg.LogError(0, RS_RET_VALUE_NOT_SUPPORTED, + "mmsequence: valid variable name should start with $," + "got %s", cstr); + free(cstr); + } else { + pData->pszVar = cstr; + } + continue; + } + dbgprintf("mmsequence: program error, non-handled " + "param '%s'\n", actpblk.descr[i].name); + } + switch(pData->mode) { + case mmSequenceRandom: + pData->seed = (unsigned int)(intptr_t)pData ^ (unsigned int)time(NULL); + break; + case mmSequencePerInstance: + pData->value = pData->valueTo; + break; + case mmSequencePerKey: + if (pthread_mutex_lock(&ght_mutex)) { + DBGPRINTF("mmsequence: mutex lock has failed!\n"); + ABORT_FINALIZE(RS_RET_ERR); + } + if (ght == NULL) { + if(NULL == (ght = create_hashtable(100, hash_from_string, key_equals_string, NULL))) { + pthread_mutex_unlock(&ght_mutex); + DBGPRINTF("mmsequence: error creating hash table!\n"); + ABORT_FINALIZE(RS_RET_ERR); + } + } + pthread_mutex_unlock(&ght_mutex); + break; + default: + errmsg.LogError(0, RS_RET_INVLD_MODE, + "mmsequence: this mode is not currently implemented"); + } + +CODE_STD_FINALIZERnewActInst + cnfparamvalsDestruct(pvals, &actpblk); +ENDnewActInst + + +BEGINdbgPrintInstInfo +CODESTARTdbgPrintInstInfo +ENDdbgPrintInstInfo + + +BEGINtryResume +CODESTARTtryResume +ENDtryResume + +static int * +getCounter(struct hashtable *ht, char *str, int initial) { + int *pCounter; + char *pStr; + + pCounter = hashtable_search(ht, str); + if(pCounter) { + return pCounter; + } + + /* counter is not found for the str, so add new entry and + return the counter */ + if(NULL == (pStr = strdup(str))) { + DBGPRINTF("mmsequence: memory allocation for key failed\n"); + return NULL; + } + + if(NULL == (pCounter = (int*)malloc(sizeof(*pCounter)))) { + DBGPRINTF("mmsequence: memory allocation for value failed\n"); + free(pStr); + return NULL; + } + *pCounter = initial; + + if(!hashtable_insert(ht, pStr, pCounter)) { + DBGPRINTF("mmsequence: inserting element into hashtable failed\n"); + free(pStr); + free(pCounter); + return NULL; + } + return pCounter; +} + + +BEGINdoAction + msg_t *pMsg; + struct json_object *json; + int val = 0; + int *pCounter; + instanceData *pData; +CODESTARTdoAction + pData = pWrkrData->pData; + pMsg = (msg_t*) ppString[0]; + + switch(pData->mode) { + case mmSequenceRandom: + val = pData->valueFrom + (rand_r(&pData->seed) % + (pData->valueTo - pData->valueFrom)); + break; + case mmSequencePerInstance: + if (!pthread_mutex_lock(&inst_mutex)) { + if (pData->value >= pData->valueTo - pData->step) { + pData->value = pData->valueFrom; + } else { + pData->value += pData->step; + } + val = pData->value; + pthread_mutex_unlock(&inst_mutex); + } else { + errmsg.LogError(0, RS_RET_ERR, + "mmsequence: mutex lock has failed!"); + } + break; + case mmSequencePerKey: + if (!pthread_mutex_lock(&ght_mutex)) { + pCounter = getCounter(ght, pData->pszKey, pData->valueTo); + if(pCounter) { + if (*pCounter >= pData->valueTo - pData->step + || *pCounter < pData->valueFrom ) { + *pCounter = pData->valueFrom; + } else { + *pCounter += pData->step; + } + val = *pCounter; + } else { + errmsg.LogError(0, RS_RET_NOT_FOUND, + "mmsequence: unable to fetch the counter from hash"); + } + pthread_mutex_unlock(&ght_mutex); + } else { + errmsg.LogError(0, RS_RET_ERR, + "mmsequence: mutex lock has failed!"); + } + + break; + default: + errmsg.LogError(0, RS_RET_NOT_IMPLEMENTED, + "mmsequence: this mode is not currently implemented"); + } + + /* finalize_it: */ + json = json_object_new_int(val); + if (json == NULL) { + errmsg.LogError(0, RS_RET_OBJ_CREATION_FAILED, + "mmsequence: unable to create JSON"); + } else if (RS_RET_OK != msgAddJSON(pMsg, (uchar *)pData->pszVar + 1, json)) { + errmsg.LogError(0, RS_RET_OBJ_CREATION_FAILED, + "mmsequence: unable to pass out the value"); + json_object_put(json); + } +ENDdoAction + + +BEGINparseSelectorAct +CODESTARTparseSelectorAct +CODE_STD_STRING_REQUESTparseSelectorAct(1) + if(strncmp((char*) p, ":mmsequence:", sizeof(":mmsequence:") - 1)) { + errmsg.LogError(0, RS_RET_LEGA_ACT_NOT_SUPPORTED, + "mmsequence supports only v6+ config format, use: " + "action(type=\"mmsequence\" ...)"); + } + ABORT_FINALIZE(RS_RET_CONFLINE_UNPROCESSED); +CODE_STD_FINALIZERparseSelectorAct +ENDparseSelectorAct + + +BEGINmodExit +CODESTARTmodExit + objRelease(errmsg, CORE_COMPONENT); +ENDmodExit + + +BEGINqueryEtryPt +CODESTARTqueryEtryPt +CODEqueryEtryPt_STD_OMOD_QUERIES +CODEqueryEtryPt_STD_OMOD8_QUERIES +CODEqueryEtryPt_STD_CONF2_OMOD_QUERIES +CODEqueryEtryPt_STD_CONF2_QUERIES +ENDqueryEtryPt + + + +BEGINmodInit() +CODESTARTmodInit + *ipIFVersProvided = CURR_MOD_IF_VERSION; /* we only support the current interface specification */ +CODEmodInit_QueryRegCFSLineHdlr + DBGPRINTF("mmsequence: module compiled with rsyslog version %s.\n", VERSION); + CHKiRet(objUse(errmsg, CORE_COMPONENT)); +ENDmodInit diff --git a/plugins/mmsnmptrapd/Makefile.in b/plugins/mmsnmptrapd/Makefile.in index 92c7e9c..484cdb4 100644 --- a/plugins/mmsnmptrapd/Makefile.in +++ b/plugins/mmsnmptrapd/Makefile.in @@ -154,7 +154,6 @@ GREP = @GREP@ GSS_LIBS = @GSS_LIBS@ GUARDTIME_CFLAGS = @GUARDTIME_CFLAGS@ GUARDTIME_LIBS = @GUARDTIME_LIBS@ -HAVE_LIBGCRYPT_CONFIG = @HAVE_LIBGCRYPT_CONFIG@ HAVE_MYSQL_CONFIG = @HAVE_MYSQL_CONFIG@ HAVE_ORACLE_CONFIG = @HAVE_ORACLE_CONFIG@ HAVE_PGSQL_CONFIG = @HAVE_PGSQL_CONFIG@ @@ -175,14 +174,15 @@ LEXLIB = @LEXLIB@ LEX_OUTPUT_ROOT = @LEX_OUTPUT_ROOT@ LIBDBI_CFLAGS = @LIBDBI_CFLAGS@ LIBDBI_LIBS = @LIBDBI_LIBS@ -LIBEE_CFLAGS = @LIBEE_CFLAGS@ -LIBEE_LIBS = @LIBEE_LIBS@ LIBESTR_CFLAGS = @LIBESTR_CFLAGS@ LIBESTR_LIBS = @LIBESTR_LIBS@ LIBGCRYPT_CFLAGS = @LIBGCRYPT_CFLAGS@ +LIBGCRYPT_CONFIG = @LIBGCRYPT_CONFIG@ LIBGCRYPT_LIBS = @LIBGCRYPT_LIBS@ LIBLOGGING_CFLAGS = @LIBLOGGING_CFLAGS@ LIBLOGGING_LIBS = @LIBLOGGING_LIBS@ +LIBLOGGING_STDLOG_CFLAGS = @LIBLOGGING_STDLOG_CFLAGS@ +LIBLOGGING_STDLOG_LIBS = @LIBLOGGING_STDLOG_LIBS@ LIBLOGNORM_CFLAGS = @LIBLOGNORM_CFLAGS@ LIBLOGNORM_LIBS = @LIBLOGNORM_LIBS@ LIBM = @LIBM@ @@ -207,6 +207,8 @@ NM = @NM@ NMEDIT = @NMEDIT@ OBJDUMP = @OBJDUMP@ OBJEXT = @OBJEXT@ +OPENSSL_CFLAGS = @OPENSSL_CFLAGS@ +OPENSSL_LIBS = @OPENSSL_LIBS@ ORACLE_CFLAGS = @ORACLE_CFLAGS@ ORACLE_LIBS = @ORACLE_LIBS@ OTOOL = @OTOOL@ @@ -388,22 +390,25 @@ distclean-compile: @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/mmsnmptrapd_la-mmsnmptrapd.Plo@am__quote@ .c.o: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(COMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ $< -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Po +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.o$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c $< +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ $< .c.obj: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(COMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ `$(CYGPATH_W) '$<'` -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Po +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.obj$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ `$(CYGPATH_W) '$<'` &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c `$(CYGPATH_W) '$<'` +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ `$(CYGPATH_W) '$<'` .c.lo: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(LTCOMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ $< -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Plo +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.lo$$||'`;\ +@am__fastdepCC_TRUE@ $(LTCOMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Plo @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=yes @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ @am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(LTCOMPILE) -c -o $@ $< diff --git a/plugins/mmsnmptrapd/mmsnmptrapd.c b/plugins/mmsnmptrapd/mmsnmptrapd.c index b79a311..de63cce 100644 --- a/plugins/mmsnmptrapd/mmsnmptrapd.c +++ b/plugins/mmsnmptrapd/mmsnmptrapd.c @@ -68,11 +68,15 @@ struct severMap_s { typedef struct _instanceData { uchar *pszTagName; - uchar *pszTagID; /* chaced: name plus trailing shlash (for compares) */ - int lenTagID; /* cached length of tag ID, for performance reasons */ + uchar *pszTagID; /* cached: name plus trailing shlash (for compares) */ + int lenTagID; /* cached: length of tag ID, for performance reasons */ struct severMap_s *severMap; } instanceData; +typedef struct wrkrInstanceData { + instanceData *pData; +} wrkrInstanceData_t; + typedef struct configSettings_s { uchar *pszTagName; /**< name of tag start value that indicates snmptrapd initiated message */ uchar *pszSeverityMapping; /**< severitystring to numerical code mapping for snmptrapd string */ @@ -92,6 +96,10 @@ CODESTARTcreateInstance ENDcreateInstance +BEGINcreateWrkrInstance +CODESTARTcreateWrkrInstance +ENDcreateWrkrInstance + BEGINisCompatibleWithFeature CODESTARTisCompatibleWithFeature ENDisCompatibleWithFeature @@ -110,6 +118,10 @@ CODESTARTfreeInstance free(pData->pszTagID); ENDfreeInstance +BEGINfreeWrkrInstance +CODESTARTfreeWrkrInstance +ENDfreeWrkrInstance + BEGINdbgPrintInstInfo CODESTARTdbgPrintInstInfo @@ -190,7 +202,6 @@ getTagComponent(uchar *tag, uchar *dst, int *lenDst) ++i; } dst[i] = '\0'; -dbgprintf("XXXX: getTagComponent dst on output: '%s', len %d\n", dst, i); *lenDst = i; done: return i; @@ -225,9 +236,10 @@ BEGINdoAction uchar *pszTag; uchar pszSever[512]; uchar pszHost[512]; + instanceData *pData; CODESTARTdoAction + pData = pWrkrData->pData; pMsg = (msg_t*) ppString[0]; - dbgprintf("XXXX: mmsnmptrapd called with pMsg %p\n", pMsg); getTAG(pMsg, &pszTag, &lenTAG); if(strncmp((char*)pszTag, (char*)pData->pszTagID, pData->lenTagID)) { DBGPRINTF("tag '%s' not matching, mmsnmptrapd ignoring this message\n", @@ -236,18 +248,16 @@ CODESTARTdoAction } lenSever = sizeof(pszSever); -dbgprintf("XXXX: pszTag: '%s', lenID %d\n", pszTag, pData->lenTagID); getTagComponent(pszTag+pData->lenTagID-1, pszSever, &lenSever); lenHost = sizeof(pszHost); getTagComponent(pszTag+pData->lenTagID+lenSever, pszHost, &lenHost); - dbgprintf("XXXX: mmsnmptrapd sever '%s'(%d), host '%s'(%d)\n", pszSever, lenSever, pszHost,lenHost); + DBGPRINTF("mmsnmptrapd: sever '%s'(%d), host '%s'(%d)\n", pszSever, lenSever, pszHost,lenHost); if(pszHost[lenHost-1] == ':') { pszHost[lenHost-1] = '\0'; --lenHost; } sevCode = lookupSeverityCode(pData, pszSever); -dbgprintf("XXXX: severity for message is %d\n", sevCode); /* now apply new settings */ MsgSetTAG(pMsg, pData->pszTagName, pData->lenTagID); MsgSetHOSTNAME(pMsg, pszHost, lenHost); @@ -267,7 +277,7 @@ buildSeverityMapping(instanceData *pData) uchar pszSevCode[512]; int sevCode; uchar *mapping; - struct severMap_s *node; + struct severMap_s *node = NULL; DEFiRet; mapping = cs.pszSeverityMapping; @@ -300,6 +310,10 @@ buildSeverityMapping(instanceData *pData) } finalize_it: + if(iRet != RS_RET_OK) { + if(node != NULL) + free(node); + } RETiRet; } @@ -362,6 +376,7 @@ ENDmodExit BEGINqueryEtryPt CODESTARTqueryEtryPt CODEqueryEtryPt_STD_OMOD_QUERIES +CODEqueryEtryPt_STD_OMOD8_QUERIES CODEqueryEtryPt_STD_CONF2_CNFNAME_QUERIES ENDqueryEtryPt diff --git a/plugins/mmutf8fix/Makefile.am b/plugins/mmutf8fix/Makefile.am new file mode 100644 index 0000000..2c0f283 --- /dev/null +++ b/plugins/mmutf8fix/Makefile.am @@ -0,0 +1,8 @@ +pkglib_LTLIBRARIES = mmutf8fix.la + +mmutf8fix_la_SOURCES = mmutf8fix.c +mmutf8fix_la_CPPFLAGS = $(RSRT_CFLAGS) $(PTHREADS_CFLAGS) +mmutf8fix_la_LDFLAGS = -module -avoid-version +mmutf8fix_la_LIBADD = + +EXTRA_DIST = diff --git a/plugins/mmutf8fix/Makefile.in b/plugins/mmutf8fix/Makefile.in new file mode 100644 index 0000000..95ec28b --- /dev/null +++ b/plugins/mmutf8fix/Makefile.in @@ -0,0 +1,637 @@ +# Makefile.in generated by automake 1.11.3 from Makefile.am. +# @configure_input@ + +# Copyright (C) 1994, 1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, +# 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010, 2011 Free Software +# Foundation, Inc. +# This Makefile.in is free software; the Free Software Foundation +# gives unlimited permission to copy and/or distribute it, +# with or without modifications, as long as this notice is preserved. + +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY, to the extent permitted by law; without +# even the implied warranty of MERCHANTABILITY or FITNESS FOR A +# PARTICULAR PURPOSE. + +@SET_MAKE@ + +VPATH = @srcdir@ +pkgdatadir = $(datadir)/@PACKAGE@ +pkgincludedir = $(includedir)/@PACKAGE@ +pkglibdir = $(libdir)/@PACKAGE@ +pkglibexecdir = $(libexecdir)/@PACKAGE@ +am__cd = CDPATH="$${ZSH_VERSION+.}$(PATH_SEPARATOR)" && cd +install_sh_DATA = $(install_sh) -c -m 644 +install_sh_PROGRAM = $(install_sh) -c +install_sh_SCRIPT = $(install_sh) -c +INSTALL_HEADER = $(INSTALL_DATA) +transform = $(program_transform_name) +NORMAL_INSTALL = : +PRE_INSTALL = : +POST_INSTALL = : +NORMAL_UNINSTALL = : +PRE_UNINSTALL = : +POST_UNINSTALL = : +build_triplet = @build@ +host_triplet = @host@ +subdir = plugins/mmutf8fix +DIST_COMMON = $(srcdir)/Makefile.am $(srcdir)/Makefile.in +ACLOCAL_M4 = $(top_srcdir)/aclocal.m4 +am__aclocal_m4_deps = $(top_srcdir)/m4/atomic_operations.m4 \ + $(top_srcdir)/m4/atomic_operations_64bit.m4 \ + $(top_srcdir)/m4/libtool.m4 $(top_srcdir)/m4/ltoptions.m4 \ + $(top_srcdir)/m4/ltsugar.m4 $(top_srcdir)/m4/ltversion.m4 \ + $(top_srcdir)/m4/lt~obsolete.m4 $(top_srcdir)/configure.ac +am__configure_deps = $(am__aclocal_m4_deps) $(CONFIGURE_DEPENDENCIES) \ + $(ACLOCAL_M4) +mkinstalldirs = $(install_sh) -d +CONFIG_HEADER = $(top_builddir)/config.h +CONFIG_CLEAN_FILES = +CONFIG_CLEAN_VPATH_FILES = +am__vpath_adj_setup = srcdirstrip=`echo "$(srcdir)" | sed 's|.|.|g'`; +am__vpath_adj = case $$p in \ + $(srcdir)/*) f=`echo "$$p" | sed "s|^$$srcdirstrip/||"`;; \ + *) f=$$p;; \ + esac; +am__strip_dir = f=`echo $$p | sed -e 's|^.*/||'`; +am__install_max = 40 +am__nobase_strip_setup = \ + srcdirstrip=`echo "$(srcdir)" | sed 's/[].[^$$\\*|]/\\\\&/g'` +am__nobase_strip = \ + for p in $$list; do echo "$$p"; done | sed -e "s|$$srcdirstrip/||" +am__nobase_list = $(am__nobase_strip_setup); \ + for p in $$list; do echo "$$p $$p"; done | \ + sed "s| $$srcdirstrip/| |;"' / .*\//!s/ .*/ ./; s,\( .*\)/[^/]*$$,\1,' | \ + $(AWK) 'BEGIN { files["."] = "" } { files[$$2] = files[$$2] " " $$1; \ + if (++n[$$2] == $(am__install_max)) \ + { print $$2, files[$$2]; n[$$2] = 0; files[$$2] = "" } } \ + END { for (dir in files) print dir, files[dir] }' +am__base_list = \ + sed '$$!N;$$!N;$$!N;$$!N;$$!N;$$!N;$$!N;s/\n/ /g' | \ + sed '$$!N;$$!N;$$!N;$$!N;s/\n/ /g' +am__uninstall_files_from_dir = { \ + test -z "$$files" \ + || { test ! -d "$$dir" && test ! -f "$$dir" && test ! -r "$$dir"; } \ + || { echo " ( cd '$$dir' && rm -f" $$files ")"; \ + $(am__cd) "$$dir" && rm -f $$files; }; \ + } +am__installdirs = "$(DESTDIR)$(pkglibdir)" +LTLIBRARIES = $(pkglib_LTLIBRARIES) +mmutf8fix_la_DEPENDENCIES = +am_mmutf8fix_la_OBJECTS = mmutf8fix_la-mmutf8fix.lo +mmutf8fix_la_OBJECTS = $(am_mmutf8fix_la_OBJECTS) +AM_V_lt = $(am__v_lt_@AM_V@) +am__v_lt_ = $(am__v_lt_@AM_DEFAULT_V@) +am__v_lt_0 = --silent +mmutf8fix_la_LINK = $(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \ + $(LIBTOOLFLAGS) --mode=link $(CCLD) $(AM_CFLAGS) $(CFLAGS) \ + $(mmutf8fix_la_LDFLAGS) $(LDFLAGS) -o $@ +DEFAULT_INCLUDES = -I.@am__isrc@ -I$(top_builddir) +depcomp = $(SHELL) $(top_srcdir)/depcomp +am__depfiles_maybe = depfiles +am__mv = mv -f +COMPILE = $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(AM_CPPFLAGS) \ + $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) +LTCOMPILE = $(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \ + $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) \ + $(DEFAULT_INCLUDES) $(INCLUDES) $(AM_CPPFLAGS) $(CPPFLAGS) \ + $(AM_CFLAGS) $(CFLAGS) +AM_V_CC = $(am__v_CC_@AM_V@) +am__v_CC_ = $(am__v_CC_@AM_DEFAULT_V@) +am__v_CC_0 = @echo " CC " $@; +AM_V_at = $(am__v_at_@AM_V@) +am__v_at_ = $(am__v_at_@AM_DEFAULT_V@) +am__v_at_0 = @ +CCLD = $(CC) +LINK = $(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \ + $(LIBTOOLFLAGS) --mode=link $(CCLD) $(AM_CFLAGS) $(CFLAGS) \ + $(AM_LDFLAGS) $(LDFLAGS) -o $@ +AM_V_CCLD = $(am__v_CCLD_@AM_V@) +am__v_CCLD_ = $(am__v_CCLD_@AM_DEFAULT_V@) +am__v_CCLD_0 = @echo " CCLD " $@; +AM_V_GEN = $(am__v_GEN_@AM_V@) +am__v_GEN_ = $(am__v_GEN_@AM_DEFAULT_V@) +am__v_GEN_0 = @echo " GEN " $@; +SOURCES = $(mmutf8fix_la_SOURCES) +DIST_SOURCES = $(mmutf8fix_la_SOURCES) +ETAGS = etags +CTAGS = ctags +DISTFILES = $(DIST_COMMON) $(DIST_SOURCES) $(TEXINFOS) $(EXTRA_DIST) +ACLOCAL = @ACLOCAL@ +AMTAR = @AMTAR@ +AM_DEFAULT_VERBOSITY = @AM_DEFAULT_VERBOSITY@ +AR = @AR@ +AUTOCONF = @AUTOCONF@ +AUTOHEADER = @AUTOHEADER@ +AUTOMAKE = @AUTOMAKE@ +AWK = @AWK@ +CC = @CC@ +CCDEPMODE = @CCDEPMODE@ +CFLAGS = @CFLAGS@ +CPP = @CPP@ +CPPFLAGS = @CPPFLAGS@ +CURL_CFLAGS = @CURL_CFLAGS@ +CURL_LIBS = @CURL_LIBS@ +CYGPATH_W = @CYGPATH_W@ +CZMQ_CFLAGS = @CZMQ_CFLAGS@ +CZMQ_LIBS = @CZMQ_LIBS@ +DEFS = @DEFS@ +DEPDIR = @DEPDIR@ +DLLTOOL = @DLLTOOL@ +DL_LIBS = @DL_LIBS@ +DSYMUTIL = @DSYMUTIL@ +DUMPBIN = @DUMPBIN@ +ECHO_C = @ECHO_C@ +ECHO_N = @ECHO_N@ +ECHO_T = @ECHO_T@ +EGREP = @EGREP@ +EXEEXT = @EXEEXT@ +FGREP = @FGREP@ +GNUTLS_CFLAGS = @GNUTLS_CFLAGS@ +GNUTLS_LIBS = @GNUTLS_LIBS@ +GREP = @GREP@ +GSS_LIBS = @GSS_LIBS@ +GUARDTIME_CFLAGS = @GUARDTIME_CFLAGS@ +GUARDTIME_LIBS = @GUARDTIME_LIBS@ +HAVE_MYSQL_CONFIG = @HAVE_MYSQL_CONFIG@ +HAVE_ORACLE_CONFIG = @HAVE_ORACLE_CONFIG@ +HAVE_PGSQL_CONFIG = @HAVE_PGSQL_CONFIG@ +HIREDIS_CFLAGS = @HIREDIS_CFLAGS@ +HIREDIS_LIBS = @HIREDIS_LIBS@ +IMUDP_LIBS = @IMUDP_LIBS@ +INSTALL = @INSTALL@ +INSTALL_DATA = @INSTALL_DATA@ +INSTALL_PROGRAM = @INSTALL_PROGRAM@ +INSTALL_SCRIPT = @INSTALL_SCRIPT@ +INSTALL_STRIP_PROGRAM = @INSTALL_STRIP_PROGRAM@ +JSON_C_CFLAGS = @JSON_C_CFLAGS@ +JSON_C_LIBS = @JSON_C_LIBS@ +LD = @LD@ +LDFLAGS = @LDFLAGS@ +LEX = @LEX@ +LEXLIB = @LEXLIB@ +LEX_OUTPUT_ROOT = @LEX_OUTPUT_ROOT@ +LIBDBI_CFLAGS = @LIBDBI_CFLAGS@ +LIBDBI_LIBS = @LIBDBI_LIBS@ +LIBESTR_CFLAGS = @LIBESTR_CFLAGS@ +LIBESTR_LIBS = @LIBESTR_LIBS@ +LIBGCRYPT_CFLAGS = @LIBGCRYPT_CFLAGS@ +LIBGCRYPT_CONFIG = @LIBGCRYPT_CONFIG@ +LIBGCRYPT_LIBS = @LIBGCRYPT_LIBS@ +LIBLOGGING_CFLAGS = @LIBLOGGING_CFLAGS@ +LIBLOGGING_LIBS = @LIBLOGGING_LIBS@ +LIBLOGGING_STDLOG_CFLAGS = @LIBLOGGING_STDLOG_CFLAGS@ +LIBLOGGING_STDLOG_LIBS = @LIBLOGGING_STDLOG_LIBS@ +LIBLOGNORM_CFLAGS = @LIBLOGNORM_CFLAGS@ +LIBLOGNORM_LIBS = @LIBLOGNORM_LIBS@ +LIBM = @LIBM@ +LIBMONGO_CLIENT_CFLAGS = @LIBMONGO_CLIENT_CFLAGS@ +LIBMONGO_CLIENT_LIBS = @LIBMONGO_CLIENT_LIBS@ +LIBOBJS = @LIBOBJS@ +LIBS = @LIBS@ +LIBSYSTEMD_JOURNAL_CFLAGS = @LIBSYSTEMD_JOURNAL_CFLAGS@ +LIBSYSTEMD_JOURNAL_LIBS = @LIBSYSTEMD_JOURNAL_LIBS@ +LIBTOOL = @LIBTOOL@ +LIBUUID_CFLAGS = @LIBUUID_CFLAGS@ +LIBUUID_LIBS = @LIBUUID_LIBS@ +LIPO = @LIPO@ +LN_S = @LN_S@ +LTLIBOBJS = @LTLIBOBJS@ +MAKEINFO = @MAKEINFO@ +MANIFEST_TOOL = @MANIFEST_TOOL@ +MKDIR_P = @MKDIR_P@ +MYSQL_CFLAGS = @MYSQL_CFLAGS@ +MYSQL_LIBS = @MYSQL_LIBS@ +NM = @NM@ +NMEDIT = @NMEDIT@ +OBJDUMP = @OBJDUMP@ +OBJEXT = @OBJEXT@ +OPENSSL_CFLAGS = @OPENSSL_CFLAGS@ +OPENSSL_LIBS = @OPENSSL_LIBS@ +ORACLE_CFLAGS = @ORACLE_CFLAGS@ +ORACLE_LIBS = @ORACLE_LIBS@ +OTOOL = @OTOOL@ +OTOOL64 = @OTOOL64@ +PACKAGE = @PACKAGE@ +PACKAGE_BUGREPORT = @PACKAGE_BUGREPORT@ +PACKAGE_NAME = @PACKAGE_NAME@ +PACKAGE_STRING = @PACKAGE_STRING@ +PACKAGE_TARNAME = @PACKAGE_TARNAME@ +PACKAGE_URL = @PACKAGE_URL@ +PACKAGE_VERSION = @PACKAGE_VERSION@ +PATH_SEPARATOR = @PATH_SEPARATOR@ +PGSQL_CFLAGS = @PGSQL_CFLAGS@ +PGSQL_LIBS = @PGSQL_LIBS@ +PKG_CONFIG = @PKG_CONFIG@ +PKG_CONFIG_LIBDIR = @PKG_CONFIG_LIBDIR@ +PKG_CONFIG_PATH = @PKG_CONFIG_PATH@ +PTHREADS_CFLAGS = @PTHREADS_CFLAGS@ +PTHREADS_LIBS = @PTHREADS_LIBS@ +RABBITMQ_CFLAGS = @RABBITMQ_CFLAGS@ +RABBITMQ_LIBS = @RABBITMQ_LIBS@ +RANLIB = @RANLIB@ +RELP_CFLAGS = @RELP_CFLAGS@ +RELP_LIBS = @RELP_LIBS@ +RSRT_CFLAGS = @RSRT_CFLAGS@ +RSRT_CFLAGS1 = @RSRT_CFLAGS1@ +RSRT_LIBS = @RSRT_LIBS@ +RSRT_LIBS1 = @RSRT_LIBS1@ +RST2MAN = @RST2MAN@ +RT_LIBS = @RT_LIBS@ +SED = @SED@ +SET_MAKE = @SET_MAKE@ +SHELL = @SHELL@ +SNMP_CFLAGS = @SNMP_CFLAGS@ +SNMP_LIBS = @SNMP_LIBS@ +SOL_LIBS = @SOL_LIBS@ +STRIP = @STRIP@ +UDPSPOOF_CFLAGS = @UDPSPOOF_CFLAGS@ +UDPSPOOF_LIBS = @UDPSPOOF_LIBS@ +VERSION = @VERSION@ +YACC = @YACC@ +YFLAGS = @YFLAGS@ +ZLIB_LIBS = @ZLIB_LIBS@ +abs_builddir = @abs_builddir@ +abs_srcdir = @abs_srcdir@ +abs_top_builddir = @abs_top_builddir@ +abs_top_srcdir = @abs_top_srcdir@ +ac_ct_AR = @ac_ct_AR@ +ac_ct_CC = @ac_ct_CC@ +ac_ct_DUMPBIN = @ac_ct_DUMPBIN@ +am__include = @am__include@ +am__leading_dot = @am__leading_dot@ +am__quote = @am__quote@ +am__tar = @am__tar@ +am__untar = @am__untar@ +bindir = @bindir@ +build = @build@ +build_alias = @build_alias@ +build_cpu = @build_cpu@ +build_os = @build_os@ +build_vendor = @build_vendor@ +builddir = @builddir@ +datadir = @datadir@ +datarootdir = @datarootdir@ +docdir = @docdir@ +dvidir = @dvidir@ +exec_prefix = @exec_prefix@ +have_valgrind = @have_valgrind@ +host = @host@ +host_alias = @host_alias@ +host_cpu = @host_cpu@ +host_os = @host_os@ +host_vendor = @host_vendor@ +htmldir = @htmldir@ +includedir = @includedir@ +infodir = @infodir@ +install_sh = @install_sh@ +libdir = @libdir@ +libexecdir = @libexecdir@ +localedir = @localedir@ +localstatedir = @localstatedir@ +mandir = @mandir@ +mkdir_p = @mkdir_p@ +moddirs = @moddirs@ +oldincludedir = @oldincludedir@ +pdfdir = @pdfdir@ +prefix = @prefix@ +program_transform_name = @program_transform_name@ +psdir = @psdir@ +sbindir = @sbindir@ +sharedstatedir = @sharedstatedir@ +srcdir = @srcdir@ +sysconfdir = @sysconfdir@ +systemdsystemunitdir = @systemdsystemunitdir@ +target_alias = @target_alias@ +top_build_prefix = @top_build_prefix@ +top_builddir = @top_builddir@ +top_srcdir = @top_srcdir@ +pkglib_LTLIBRARIES = mmutf8fix.la +mmutf8fix_la_SOURCES = mmutf8fix.c +mmutf8fix_la_CPPFLAGS = $(RSRT_CFLAGS) $(PTHREADS_CFLAGS) +mmutf8fix_la_LDFLAGS = -module -avoid-version +mmutf8fix_la_LIBADD = +EXTRA_DIST = +all: all-am + +.SUFFIXES: +.SUFFIXES: .c .lo .o .obj +$(srcdir)/Makefile.in: $(srcdir)/Makefile.am $(am__configure_deps) + @for dep in $?; do \ + case '$(am__configure_deps)' in \ + *$$dep*) \ + ( cd $(top_builddir) && $(MAKE) $(AM_MAKEFLAGS) am--refresh ) \ + && { if test -f $@; then exit 0; else break; fi; }; \ + exit 1;; \ + esac; \ + done; \ + echo ' cd $(top_srcdir) && $(AUTOMAKE) --gnu plugins/mmutf8fix/Makefile'; \ + $(am__cd) $(top_srcdir) && \ + $(AUTOMAKE) --gnu plugins/mmutf8fix/Makefile +.PRECIOUS: Makefile +Makefile: $(srcdir)/Makefile.in $(top_builddir)/config.status + @case '$?' in \ + *config.status*) \ + cd $(top_builddir) && $(MAKE) $(AM_MAKEFLAGS) am--refresh;; \ + *) \ + echo ' cd $(top_builddir) && $(SHELL) ./config.status $(subdir)/$@ $(am__depfiles_maybe)'; \ + cd $(top_builddir) && $(SHELL) ./config.status $(subdir)/$@ $(am__depfiles_maybe);; \ + esac; + +$(top_builddir)/config.status: $(top_srcdir)/configure $(CONFIG_STATUS_DEPENDENCIES) + cd $(top_builddir) && $(MAKE) $(AM_MAKEFLAGS) am--refresh + +$(top_srcdir)/configure: $(am__configure_deps) + cd $(top_builddir) && $(MAKE) $(AM_MAKEFLAGS) am--refresh +$(ACLOCAL_M4): $(am__aclocal_m4_deps) + cd $(top_builddir) && $(MAKE) $(AM_MAKEFLAGS) am--refresh +$(am__aclocal_m4_deps): +install-pkglibLTLIBRARIES: $(pkglib_LTLIBRARIES) + @$(NORMAL_INSTALL) + test -z "$(pkglibdir)" || $(MKDIR_P) "$(DESTDIR)$(pkglibdir)" + @list='$(pkglib_LTLIBRARIES)'; test -n "$(pkglibdir)" || list=; \ + list2=; for p in $$list; do \ + if test -f $$p; then \ + list2="$$list2 $$p"; \ + else :; fi; \ + done; \ + test -z "$$list2" || { \ + echo " $(LIBTOOL) $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=install $(INSTALL) $(INSTALL_STRIP_FLAG) $$list2 '$(DESTDIR)$(pkglibdir)'"; \ + $(LIBTOOL) $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=install $(INSTALL) $(INSTALL_STRIP_FLAG) $$list2 "$(DESTDIR)$(pkglibdir)"; \ + } + +uninstall-pkglibLTLIBRARIES: + @$(NORMAL_UNINSTALL) + @list='$(pkglib_LTLIBRARIES)'; test -n "$(pkglibdir)" || list=; \ + for p in $$list; do \ + $(am__strip_dir) \ + echo " $(LIBTOOL) $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=uninstall rm -f '$(DESTDIR)$(pkglibdir)/$$f'"; \ + $(LIBTOOL) $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=uninstall rm -f "$(DESTDIR)$(pkglibdir)/$$f"; \ + done + +clean-pkglibLTLIBRARIES: + -test -z "$(pkglib_LTLIBRARIES)" || rm -f $(pkglib_LTLIBRARIES) + @list='$(pkglib_LTLIBRARIES)'; for p in $$list; do \ + dir="`echo $$p | sed -e 's|/[^/]*$$||'`"; \ + test "$$dir" != "$$p" || dir=.; \ + echo "rm -f \"$${dir}/so_locations\""; \ + rm -f "$${dir}/so_locations"; \ + done +mmutf8fix.la: $(mmutf8fix_la_OBJECTS) $(mmutf8fix_la_DEPENDENCIES) $(EXTRA_mmutf8fix_la_DEPENDENCIES) + $(AM_V_CCLD)$(mmutf8fix_la_LINK) -rpath $(pkglibdir) $(mmutf8fix_la_OBJECTS) $(mmutf8fix_la_LIBADD) $(LIBS) + +mostlyclean-compile: + -rm -f *.$(OBJEXT) + +distclean-compile: + -rm -f *.tab.c + +@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/mmutf8fix_la-mmutf8fix.Plo@am__quote@ + +.c.o: +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.o$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ $< + +.c.obj: +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.obj$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ `$(CYGPATH_W) '$<'` &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ `$(CYGPATH_W) '$<'` + +.c.lo: +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.lo$$||'`;\ +@am__fastdepCC_TRUE@ $(LTCOMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Plo +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=yes @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(LTCOMPILE) -c -o $@ $< + +mmutf8fix_la-mmutf8fix.lo: mmutf8fix.c +@am__fastdepCC_TRUE@ $(AM_V_CC)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(mmutf8fix_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT mmutf8fix_la-mmutf8fix.lo -MD -MP -MF $(DEPDIR)/mmutf8fix_la-mmutf8fix.Tpo -c -o mmutf8fix_la-mmutf8fix.lo `test -f 'mmutf8fix.c' || echo '$(srcdir)/'`mmutf8fix.c +@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/mmutf8fix_la-mmutf8fix.Tpo $(DEPDIR)/mmutf8fix_la-mmutf8fix.Plo +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='mmutf8fix.c' object='mmutf8fix_la-mmutf8fix.lo' libtool=yes @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(mmutf8fix_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o mmutf8fix_la-mmutf8fix.lo `test -f 'mmutf8fix.c' || echo '$(srcdir)/'`mmutf8fix.c + +mostlyclean-libtool: + -rm -f *.lo + +clean-libtool: + -rm -rf .libs _libs + +ID: $(HEADERS) $(SOURCES) $(LISP) $(TAGS_FILES) + list='$(SOURCES) $(HEADERS) $(LISP) $(TAGS_FILES)'; \ + unique=`for i in $$list; do \ + if test -f "$$i"; then echo $$i; else echo $(srcdir)/$$i; fi; \ + done | \ + $(AWK) '{ files[$$0] = 1; nonempty = 1; } \ + END { if (nonempty) { for (i in files) print i; }; }'`; \ + mkid -fID $$unique +tags: TAGS + +TAGS: $(HEADERS) $(SOURCES) $(TAGS_DEPENDENCIES) \ + $(TAGS_FILES) $(LISP) + set x; \ + here=`pwd`; \ + list='$(SOURCES) $(HEADERS) $(LISP) $(TAGS_FILES)'; \ + unique=`for i in $$list; do \ + if test -f "$$i"; then echo $$i; else echo $(srcdir)/$$i; fi; \ + done | \ + $(AWK) '{ files[$$0] = 1; nonempty = 1; } \ + END { if (nonempty) { for (i in files) print i; }; }'`; \ + shift; \ + if test -z "$(ETAGS_ARGS)$$*$$unique"; then :; else \ + test -n "$$unique" || unique=$$empty_fix; \ + if test $$# -gt 0; then \ + $(ETAGS) $(ETAGSFLAGS) $(AM_ETAGSFLAGS) $(ETAGS_ARGS) \ + "$$@" $$unique; \ + else \ + $(ETAGS) $(ETAGSFLAGS) $(AM_ETAGSFLAGS) $(ETAGS_ARGS) \ + $$unique; \ + fi; \ + fi +ctags: CTAGS +CTAGS: $(HEADERS) $(SOURCES) $(TAGS_DEPENDENCIES) \ + $(TAGS_FILES) $(LISP) + list='$(SOURCES) $(HEADERS) $(LISP) $(TAGS_FILES)'; \ + unique=`for i in $$list; do \ + if test -f "$$i"; then echo $$i; else echo $(srcdir)/$$i; fi; \ + done | \ + $(AWK) '{ files[$$0] = 1; nonempty = 1; } \ + END { if (nonempty) { for (i in files) print i; }; }'`; \ + test -z "$(CTAGS_ARGS)$$unique" \ + || $(CTAGS) $(CTAGSFLAGS) $(AM_CTAGSFLAGS) $(CTAGS_ARGS) \ + $$unique + +GTAGS: + here=`$(am__cd) $(top_builddir) && pwd` \ + && $(am__cd) $(top_srcdir) \ + && gtags -i $(GTAGS_ARGS) "$$here" + +distclean-tags: + -rm -f TAGS ID GTAGS GRTAGS GSYMS GPATH tags + +distdir: $(DISTFILES) + @srcdirstrip=`echo "$(srcdir)" | sed 's/[].[^$$\\*]/\\\\&/g'`; \ + topsrcdirstrip=`echo "$(top_srcdir)" | sed 's/[].[^$$\\*]/\\\\&/g'`; \ + list='$(DISTFILES)'; \ + dist_files=`for file in $$list; do echo $$file; done | \ + sed -e "s|^$$srcdirstrip/||;t" \ + -e "s|^$$topsrcdirstrip/|$(top_builddir)/|;t"`; \ + case $$dist_files in \ + */*) $(MKDIR_P) `echo "$$dist_files" | \ + sed '/\//!d;s|^|$(distdir)/|;s,/[^/]*$$,,' | \ + sort -u` ;; \ + esac; \ + for file in $$dist_files; do \ + if test -f $$file || test -d $$file; then d=.; else d=$(srcdir); fi; \ + if test -d $$d/$$file; then \ + dir=`echo "/$$file" | sed -e 's,/[^/]*$$,,'`; \ + if test -d "$(distdir)/$$file"; then \ + find "$(distdir)/$$file" -type d ! -perm -700 -exec chmod u+rwx {} \;; \ + fi; \ + if test -d $(srcdir)/$$file && test $$d != $(srcdir); then \ + cp -fpR $(srcdir)/$$file "$(distdir)$$dir" || exit 1; \ + find "$(distdir)/$$file" -type d ! -perm -700 -exec chmod u+rwx {} \;; \ + fi; \ + cp -fpR $$d/$$file "$(distdir)$$dir" || exit 1; \ + else \ + test -f "$(distdir)/$$file" \ + || cp -p $$d/$$file "$(distdir)/$$file" \ + || exit 1; \ + fi; \ + done +check-am: all-am +check: check-am +all-am: Makefile $(LTLIBRARIES) +installdirs: + for dir in "$(DESTDIR)$(pkglibdir)"; do \ + test -z "$$dir" || $(MKDIR_P) "$$dir"; \ + done +install: install-am +install-exec: install-exec-am +install-data: install-data-am +uninstall: uninstall-am + +install-am: all-am + @$(MAKE) $(AM_MAKEFLAGS) install-exec-am install-data-am + +installcheck: installcheck-am +install-strip: + if test -z '$(STRIP)'; then \ + $(MAKE) $(AM_MAKEFLAGS) INSTALL_PROGRAM="$(INSTALL_STRIP_PROGRAM)" \ + install_sh_PROGRAM="$(INSTALL_STRIP_PROGRAM)" INSTALL_STRIP_FLAG=-s \ + install; \ + else \ + $(MAKE) $(AM_MAKEFLAGS) INSTALL_PROGRAM="$(INSTALL_STRIP_PROGRAM)" \ + install_sh_PROGRAM="$(INSTALL_STRIP_PROGRAM)" INSTALL_STRIP_FLAG=-s \ + "INSTALL_PROGRAM_ENV=STRIPPROG='$(STRIP)'" install; \ + fi +mostlyclean-generic: + +clean-generic: + +distclean-generic: + -test -z "$(CONFIG_CLEAN_FILES)" || rm -f $(CONFIG_CLEAN_FILES) + -test . = "$(srcdir)" || test -z "$(CONFIG_CLEAN_VPATH_FILES)" || rm -f $(CONFIG_CLEAN_VPATH_FILES) + +maintainer-clean-generic: + @echo "This command is intended for maintainers to use" + @echo "it deletes files that may require special tools to rebuild." +clean: clean-am + +clean-am: clean-generic clean-libtool clean-pkglibLTLIBRARIES \ + mostlyclean-am + +distclean: distclean-am + -rm -rf ./$(DEPDIR) + -rm -f Makefile +distclean-am: clean-am distclean-compile distclean-generic \ + distclean-tags + +dvi: dvi-am + +dvi-am: + +html: html-am + +html-am: + +info: info-am + +info-am: + +install-data-am: + +install-dvi: install-dvi-am + +install-dvi-am: + +install-exec-am: install-pkglibLTLIBRARIES + +install-html: install-html-am + +install-html-am: + +install-info: install-info-am + +install-info-am: + +install-man: + +install-pdf: install-pdf-am + +install-pdf-am: + +install-ps: install-ps-am + +install-ps-am: + +installcheck-am: + +maintainer-clean: maintainer-clean-am + -rm -rf ./$(DEPDIR) + -rm -f Makefile +maintainer-clean-am: distclean-am maintainer-clean-generic + +mostlyclean: mostlyclean-am + +mostlyclean-am: mostlyclean-compile mostlyclean-generic \ + mostlyclean-libtool + +pdf: pdf-am + +pdf-am: + +ps: ps-am + +ps-am: + +uninstall-am: uninstall-pkglibLTLIBRARIES + +.MAKE: install-am install-strip + +.PHONY: CTAGS GTAGS all all-am check check-am clean clean-generic \ + clean-libtool clean-pkglibLTLIBRARIES ctags distclean \ + distclean-compile distclean-generic distclean-libtool \ + distclean-tags distdir dvi dvi-am html html-am info info-am \ + install install-am install-data install-data-am install-dvi \ + install-dvi-am install-exec install-exec-am install-html \ + install-html-am install-info install-info-am install-man \ + install-pdf install-pdf-am install-pkglibLTLIBRARIES \ + install-ps install-ps-am install-strip installcheck \ + installcheck-am installdirs maintainer-clean \ + maintainer-clean-generic mostlyclean mostlyclean-compile \ + mostlyclean-generic mostlyclean-libtool pdf pdf-am ps ps-am \ + tags uninstall uninstall-am uninstall-pkglibLTLIBRARIES + + +# Tell versions [3.59,3.63) of GNU make to not export all variables. +# Otherwise a system limit (for SysV at least) may be exceeded. +.NOEXPORT: diff --git a/plugins/mmutf8fix/mmutf8fix.c b/plugins/mmutf8fix/mmutf8fix.c new file mode 100644 index 0000000..e529686 --- /dev/null +++ b/plugins/mmutf8fix/mmutf8fix.c @@ -0,0 +1,338 @@ +/* mmutf8fix.c + * fix invalid UTF8 sequences. This is begun as a very simple replacer + * of non-control characters, and actually breaks some UTF-8 encoding + * right now. If the module turns out to be useful, it should be enhanced + * to support modes that really detect invalid UTF8. In the longer term + * it could also be evolved into an any-charset-to-UTF8 converter. But + * first let's see if it really gets into widespread enough use. + * + * Copyright 2013 Adiscon GmbH. + * + * This file is part of rsyslog. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * -or- + * see COPYING.ASL20 in the source distribution + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +#include "config.h" +#include "rsyslog.h" +#include <stdio.h> +#include <stdarg.h> +#include <stdlib.h> +#include <string.h> +#include <assert.h> +#include <signal.h> +#include <errno.h> +#include <unistd.h> +#include <stdint.h> +#include "conf.h" +#include "syslogd-types.h" +#include "srUtils.h" +#include "template.h" +#include "module-template.h" +#include "errmsg.h" + +MODULE_TYPE_OUTPUT +MODULE_TYPE_NOKEEP +MODULE_CNFNAME("mmutf8fix") + + +DEFobjCurrIf(errmsg); +DEF_OMOD_STATIC_DATA + +/* define operation modes we have */ +#define MODE_CC 0 /* just fix control characters */ +#define MODE_UTF8 1 /* do real UTF-8 fixing */ + +/* config variables */ +typedef struct _instanceData { + uchar replChar; + uint8_t mode; /* operations mode */ +} instanceData; + +typedef struct wrkrInstanceData { + instanceData *pData; +} wrkrInstanceData_t; + +struct modConfData_s { + rsconf_t *pConf; /* our overall config object */ +}; +static modConfData_t *loadModConf = NULL;/* modConf ptr to use for the current load process */ +static modConfData_t *runModConf = NULL;/* modConf ptr to use for the current exec process */ + + +/* tables for interfacing with the v6 config system */ +/* action (instance) parameters */ +static struct cnfparamdescr actpdescr[] = { + { "mode", eCmdHdlrGetWord, 0 }, + { "replacementchar", eCmdHdlrGetChar, 0 } +}; +static struct cnfparamblk actpblk = + { CNFPARAMBLK_VERSION, + sizeof(actpdescr)/sizeof(struct cnfparamdescr), + actpdescr + }; + +BEGINbeginCnfLoad +CODESTARTbeginCnfLoad + loadModConf = pModConf; + pModConf->pConf = pConf; +ENDbeginCnfLoad + +BEGINendCnfLoad +CODESTARTendCnfLoad +ENDendCnfLoad + +BEGINcheckCnf +CODESTARTcheckCnf +ENDcheckCnf + +BEGINactivateCnf +CODESTARTactivateCnf + runModConf = pModConf; +ENDactivateCnf + +BEGINfreeCnf +CODESTARTfreeCnf +ENDfreeCnf + + +BEGINcreateInstance +CODESTARTcreateInstance +ENDcreateInstance + + +BEGINcreateWrkrInstance +CODESTARTcreateWrkrInstance +ENDcreateWrkrInstance + + +BEGINisCompatibleWithFeature +CODESTARTisCompatibleWithFeature +ENDisCompatibleWithFeature + + +BEGINfreeInstance +CODESTARTfreeInstance +ENDfreeInstance + + +BEGINfreeWrkrInstance +CODESTARTfreeWrkrInstance +ENDfreeWrkrInstance + + +static inline void +setInstParamDefaults(instanceData *pData) +{ + pData->mode = MODE_UTF8; + pData->replChar = ' '; +} + +BEGINnewActInst + struct cnfparamvals *pvals; + int i; +CODESTARTnewActInst + DBGPRINTF("newActInst (mmutf8fix)\n"); + if((pvals = nvlstGetParams(lst, &actpblk, NULL)) == NULL) { + ABORT_FINALIZE(RS_RET_MISSING_CNFPARAMS); + } + + CODE_STD_STRING_REQUESTnewActInst(1) + CHKiRet(OMSRsetEntry(*ppOMSR, 0, NULL, OMSR_TPL_AS_MSG)); + CHKiRet(createInstance(&pData)); + setInstParamDefaults(pData); + + for(i = 0 ; i < actpblk.nParams ; ++i) { + if(!pvals[i].bUsed) + continue; + if(!strcmp(actpblk.descr[i].name, "mode")) { + if(!es_strbufcmp(pvals[i].val.d.estr, (uchar*)"utf-8", + sizeof("utf-8")-1)) { + pData->mode = MODE_UTF8; + } else if(!es_strbufcmp(pvals[i].val.d.estr, (uchar*)"controlcharacters", + sizeof("controlcharacters")-1)) { + pData->mode = MODE_CC; + } else { + char *cstr = es_str2cstr(pvals[i].val.d.estr, NULL); + errmsg.LogError(0, RS_RET_INVLD_MODE, + "mmutf8fix: invalid mode '%s' - ignored", + cstr); + free(cstr); + } + } else if(!strcmp(actpblk.descr[i].name, "replacementchar")) { + pData->replChar = es_getBufAddr(pvals[i].val.d.estr)[0]; + } else { + dbgprintf("mmutf8fix: program error, non-handled " + "param '%s'\n", actpblk.descr[i].name); + } + } + +CODE_STD_FINALIZERnewActInst + cnfparamvalsDestruct(pvals, &actpblk); +ENDnewActInst + + +BEGINdbgPrintInstInfo +CODESTARTdbgPrintInstInfo +ENDdbgPrintInstInfo + + +BEGINtryResume +CODESTARTtryResume +ENDtryResume + + +static inline void +doCC(instanceData *pData, uchar *msg, int lenMsg) +{ + int i; + + for(i = 0 ; i < lenMsg ; ++i) { + if(msg[i] < 32 || msg[i] > 126) { + msg[i] = pData->replChar; + } + } +} + +/* fix an invalid multibyte sequence */ +static inline void +fixInvldMBSeq(instanceData *pData, uchar *msg, int lenMsg, int strtIdx, int *endIdx, int8_t seqLen) +{ + int i; + + *endIdx = strtIdx + seqLen; + if(*endIdx > lenMsg) + *endIdx = lenMsg; + for(i = strtIdx ; i < *endIdx ; ++i) + msg[i] = pData->replChar; +} + +static inline void +doUTF8(instanceData *pData, uchar *msg, int lenMsg) +{ + uchar c; + int8_t seqLen, bytesLeft = 0; + uint32_t codepoint; + int strtIdx, endIdx; + int i; + + for(i = 0 ; i < lenMsg ; ++i) { + c = msg[i]; + if(bytesLeft) { + if((c & 0xc0) != 0x80) { + /* sequence invalid, invalidate all bytes */ + fixInvldMBSeq(pData, msg, lenMsg, strtIdx, &endIdx, + seqLen); + i = endIdx - 1; + bytesLeft = 0; + } else { + codepoint = (codepoint << 6) | (c & 0x3f); + --bytesLeft; + if(bytesLeft == 0) { + /* too-large codepoint? */ + if(codepoint > 0x10FFFF) { + fixInvldMBSeq(pData, msg, lenMsg, + strtIdx, &endIdx, + seqLen); + } + } + } + } else { + if((c & 0x80) == 0) { + /* 1-byte sequence, US-ASCII */ + ; /* nothing to do, all well */ + } else if((c & 0xe0) == 0xc0) { + /* 2-byte sequence */ + /* 0xc0 and 0xc1 are illegal */ + if(c == 0xc0 || c == 0xc1) { + msg[i] = pData->replChar; + } else { + strtIdx = i; + seqLen = bytesLeft = 1; + codepoint = c & 0x1f; + } + } else if((c & 0xf0) == 0xe0) { + /* 3-byte sequence */ + strtIdx = i; + seqLen = bytesLeft = 2; + codepoint = c & 0x0f; + } else if((c & 0xf8) == 0xf0) { + /* 4-byte sequence */ + strtIdx = i; + seqLen = bytesLeft = 3; + codepoint = c & 0x07; + } else { /* invalid (5&6 byte forbidden by RFC3629) */ + msg[i] = pData->replChar; + } + if(i+bytesLeft >= lenMsg) { + int dummy = lenMsg; + /* invalid, as rest of message cannot contain full char */ + fixInvldMBSeq(pData, msg, lenMsg, strtIdx, &dummy, seqLen); + i = lenMsg - 1; + } + } + } +} + +BEGINdoAction + msg_t *pMsg; + uchar *msg; + int lenMsg; +CODESTARTdoAction + pMsg = (msg_t*) ppString[0]; + lenMsg = getMSGLen(pMsg); + msg = getMSG(pMsg); + if(pWrkrData->pData->mode == MODE_CC) { + doCC(pWrkrData->pData, msg, lenMsg); + } else { + doUTF8(pWrkrData->pData, msg, lenMsg); + } +ENDdoAction + + +BEGINparseSelectorAct +CODESTARTparseSelectorAct +CODE_STD_STRING_REQUESTparseSelectorAct(1) + if(strncmp((char*) p, ":mmutf8fix:", sizeof(":mmutf8fix:") - 1)) { + errmsg.LogError(0, RS_RET_LEGA_ACT_NOT_SUPPORTED, + "mmutf8fix supports only v6+ config format, use: " + "action(type=\"mmutf8fix\" ...)"); + } + ABORT_FINALIZE(RS_RET_CONFLINE_UNPROCESSED); +CODE_STD_FINALIZERparseSelectorAct +ENDparseSelectorAct + + +BEGINmodExit +CODESTARTmodExit + objRelease(errmsg, CORE_COMPONENT); +ENDmodExit + + +BEGINqueryEtryPt +CODESTARTqueryEtryPt +CODEqueryEtryPt_STD_OMOD_QUERIES +CODEqueryEtryPt_STD_OMOD8_QUERIES +CODEqueryEtryPt_STD_CONF2_OMOD_QUERIES +CODEqueryEtryPt_STD_CONF2_QUERIES +ENDqueryEtryPt + + +BEGINmodInit() +CODESTARTmodInit + *ipIFVersProvided = CURR_MOD_IF_VERSION; /* we only support the current interface specification */ +CODEmodInit_QueryRegCFSLineHdlr + DBGPRINTF("mmutf8fix: module compiled with rsyslog version %s.\n", VERSION); + CHKiRet(objUse(errmsg, CORE_COMPONENT)); +ENDmodInit diff --git a/plugins/omelasticsearch/Makefile.in b/plugins/omelasticsearch/Makefile.in index 8752fcd..b953d7a 100644 --- a/plugins/omelasticsearch/Makefile.in +++ b/plugins/omelasticsearch/Makefile.in @@ -80,8 +80,9 @@ LTLIBRARIES = $(pkglib_LTLIBRARIES) am__DEPENDENCIES_1 = omelasticsearch_la_DEPENDENCIES = $(am__DEPENDENCIES_1) \ $(am__DEPENDENCIES_1) +am__dirstamp = $(am__leading_dot)dirstamp am_omelasticsearch_la_OBJECTS = omelasticsearch_la-omelasticsearch.lo \ - omelasticsearch_la-cjson.lo + cJSON/omelasticsearch_la-cjson.lo omelasticsearch_la_OBJECTS = $(am_omelasticsearch_la_OBJECTS) AM_V_lt = $(am__v_lt_@AM_V@) am__v_lt_ = $(am__v_lt_@AM_DEFAULT_V@) @@ -157,7 +158,6 @@ GREP = @GREP@ GSS_LIBS = @GSS_LIBS@ GUARDTIME_CFLAGS = @GUARDTIME_CFLAGS@ GUARDTIME_LIBS = @GUARDTIME_LIBS@ -HAVE_LIBGCRYPT_CONFIG = @HAVE_LIBGCRYPT_CONFIG@ HAVE_MYSQL_CONFIG = @HAVE_MYSQL_CONFIG@ HAVE_ORACLE_CONFIG = @HAVE_ORACLE_CONFIG@ HAVE_PGSQL_CONFIG = @HAVE_PGSQL_CONFIG@ @@ -178,14 +178,15 @@ LEXLIB = @LEXLIB@ LEX_OUTPUT_ROOT = @LEX_OUTPUT_ROOT@ LIBDBI_CFLAGS = @LIBDBI_CFLAGS@ LIBDBI_LIBS = @LIBDBI_LIBS@ -LIBEE_CFLAGS = @LIBEE_CFLAGS@ -LIBEE_LIBS = @LIBEE_LIBS@ LIBESTR_CFLAGS = @LIBESTR_CFLAGS@ LIBESTR_LIBS = @LIBESTR_LIBS@ LIBGCRYPT_CFLAGS = @LIBGCRYPT_CFLAGS@ +LIBGCRYPT_CONFIG = @LIBGCRYPT_CONFIG@ LIBGCRYPT_LIBS = @LIBGCRYPT_LIBS@ LIBLOGGING_CFLAGS = @LIBLOGGING_CFLAGS@ LIBLOGGING_LIBS = @LIBLOGGING_LIBS@ +LIBLOGGING_STDLOG_CFLAGS = @LIBLOGGING_STDLOG_CFLAGS@ +LIBLOGGING_STDLOG_LIBS = @LIBLOGGING_STDLOG_LIBS@ LIBLOGNORM_CFLAGS = @LIBLOGNORM_CFLAGS@ LIBLOGNORM_LIBS = @LIBLOGNORM_LIBS@ LIBM = @LIBM@ @@ -210,6 +211,8 @@ NM = @NM@ NMEDIT = @NMEDIT@ OBJDUMP = @OBJDUMP@ OBJEXT = @OBJEXT@ +OPENSSL_CFLAGS = @OPENSSL_CFLAGS@ +OPENSSL_LIBS = @OPENSSL_LIBS@ ORACLE_CFLAGS = @ORACLE_CFLAGS@ ORACLE_LIBS = @ORACLE_LIBS@ OTOOL = @OTOOL@ @@ -381,35 +384,48 @@ clean-pkglibLTLIBRARIES: echo "rm -f \"$${dir}/so_locations\""; \ rm -f "$${dir}/so_locations"; \ done +cJSON/$(am__dirstamp): + @$(MKDIR_P) cJSON + @: > cJSON/$(am__dirstamp) +cJSON/$(DEPDIR)/$(am__dirstamp): + @$(MKDIR_P) cJSON/$(DEPDIR) + @: > cJSON/$(DEPDIR)/$(am__dirstamp) +cJSON/omelasticsearch_la-cjson.lo: cJSON/$(am__dirstamp) \ + cJSON/$(DEPDIR)/$(am__dirstamp) omelasticsearch.la: $(omelasticsearch_la_OBJECTS) $(omelasticsearch_la_DEPENDENCIES) $(EXTRA_omelasticsearch_la_DEPENDENCIES) $(AM_V_CCLD)$(omelasticsearch_la_LINK) -rpath $(pkglibdir) $(omelasticsearch_la_OBJECTS) $(omelasticsearch_la_LIBADD) $(LIBS) mostlyclean-compile: -rm -f *.$(OBJEXT) + -rm -f cJSON/omelasticsearch_la-cjson.$(OBJEXT) + -rm -f cJSON/omelasticsearch_la-cjson.lo distclean-compile: -rm -f *.tab.c -@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/omelasticsearch_la-cjson.Plo@am__quote@ @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/omelasticsearch_la-omelasticsearch.Plo@am__quote@ +@AMDEP_TRUE@@am__include@ @am__quote@cJSON/$(DEPDIR)/omelasticsearch_la-cjson.Plo@am__quote@ .c.o: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(COMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ $< -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Po +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.o$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c $< +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ $< .c.obj: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(COMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ `$(CYGPATH_W) '$<'` -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Po +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.obj$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ `$(CYGPATH_W) '$<'` &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c `$(CYGPATH_W) '$<'` +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ `$(CYGPATH_W) '$<'` .c.lo: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(LTCOMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ $< -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Plo +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.lo$$||'`;\ +@am__fastdepCC_TRUE@ $(LTCOMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Plo @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=yes @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ @am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(LTCOMPILE) -c -o $@ $< @@ -421,18 +437,19 @@ omelasticsearch_la-omelasticsearch.lo: omelasticsearch.c @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ @am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(omelasticsearch_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o omelasticsearch_la-omelasticsearch.lo `test -f 'omelasticsearch.c' || echo '$(srcdir)/'`omelasticsearch.c -omelasticsearch_la-cjson.lo: cJSON/cjson.c -@am__fastdepCC_TRUE@ $(AM_V_CC)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(omelasticsearch_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT omelasticsearch_la-cjson.lo -MD -MP -MF $(DEPDIR)/omelasticsearch_la-cjson.Tpo -c -o omelasticsearch_la-cjson.lo `test -f 'cJSON/cjson.c' || echo '$(srcdir)/'`cJSON/cjson.c -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/omelasticsearch_la-cjson.Tpo $(DEPDIR)/omelasticsearch_la-cjson.Plo -@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='cJSON/cjson.c' object='omelasticsearch_la-cjson.lo' libtool=yes @AMDEPBACKSLASH@ +cJSON/omelasticsearch_la-cjson.lo: cJSON/cjson.c +@am__fastdepCC_TRUE@ $(AM_V_CC)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(omelasticsearch_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT cJSON/omelasticsearch_la-cjson.lo -MD -MP -MF cJSON/$(DEPDIR)/omelasticsearch_la-cjson.Tpo -c -o cJSON/omelasticsearch_la-cjson.lo `test -f 'cJSON/cjson.c' || echo '$(srcdir)/'`cJSON/cjson.c +@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) cJSON/$(DEPDIR)/omelasticsearch_la-cjson.Tpo cJSON/$(DEPDIR)/omelasticsearch_la-cjson.Plo +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='cJSON/cjson.c' object='cJSON/omelasticsearch_la-cjson.lo' libtool=yes @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(omelasticsearch_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o omelasticsearch_la-cjson.lo `test -f 'cJSON/cjson.c' || echo '$(srcdir)/'`cJSON/cjson.c +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(omelasticsearch_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o cJSON/omelasticsearch_la-cjson.lo `test -f 'cJSON/cjson.c' || echo '$(srcdir)/'`cJSON/cjson.c mostlyclean-libtool: -rm -f *.lo clean-libtool: -rm -rf .libs _libs + -rm -rf cJSON/.libs cJSON/_libs ID: $(HEADERS) $(SOURCES) $(LISP) $(TAGS_FILES) list='$(SOURCES) $(HEADERS) $(LISP) $(TAGS_FILES)'; \ @@ -549,6 +566,8 @@ clean-generic: distclean-generic: -test -z "$(CONFIG_CLEAN_FILES)" || rm -f $(CONFIG_CLEAN_FILES) -test . = "$(srcdir)" || test -z "$(CONFIG_CLEAN_VPATH_FILES)" || rm -f $(CONFIG_CLEAN_VPATH_FILES) + -rm -f cJSON/$(DEPDIR)/$(am__dirstamp) + -rm -f cJSON/$(am__dirstamp) maintainer-clean-generic: @echo "This command is intended for maintainers to use" @@ -559,7 +578,7 @@ clean-am: clean-generic clean-libtool clean-pkglibLTLIBRARIES \ mostlyclean-am distclean: distclean-am - -rm -rf ./$(DEPDIR) + -rm -rf ./$(DEPDIR) cJSON/$(DEPDIR) -rm -f Makefile distclean-am: clean-am distclean-compile distclean-generic \ distclean-tags @@ -605,7 +624,7 @@ install-ps-am: installcheck-am: maintainer-clean: maintainer-clean-am - -rm -rf ./$(DEPDIR) + -rm -rf ./$(DEPDIR) cJSON/$(DEPDIR) -rm -f Makefile maintainer-clean-am: distclean-am maintainer-clean-generic diff --git a/plugins/omelasticsearch/README b/plugins/omelasticsearch/README index 9021bc0..b8bf415 100644 --- a/plugins/omelasticsearch/README +++ b/plugins/omelasticsearch/README @@ -1,3 +1,7 @@ +How to access ElasticSearch on local machine (for testing): +=========================================================== +see: https://github.com/mobz/elasticsearch-head + How to produce an error: ======================== It's quite easy to get 400, if you put a wrong mapping to your diff --git a/plugins/omelasticsearch/omelasticsearch.c b/plugins/omelasticsearch/omelasticsearch.c index 33e58c1..68fb3c1 100644 --- a/plugins/omelasticsearch/omelasticsearch.c +++ b/plugins/omelasticsearch/omelasticsearch.c @@ -4,7 +4,7 @@ * NOTE: read comments in module-template.h for more specifics! * * Copyright 2011 Nathan Scott. - * Copyright 2009-2012 Rainer Gerhards and Adiscon GmbH. + * Copyright 2009-2013 Rainer Gerhards and Adiscon GmbH. * * This file is part of rsyslog. * @@ -58,10 +58,10 @@ DEFobjCurrIf(errmsg) DEFobjCurrIf(statsobj) statsobj_t *indexStats; -STATSCOUNTER_DEF(indexConFail, mutIndexConFail) STATSCOUNTER_DEF(indexSubmit, mutIndexSubmit) -STATSCOUNTER_DEF(indexFailed, mutIndexFailed) -STATSCOUNTER_DEF(indexSuccess, mutIndexSuccess) +STATSCOUNTER_DEF(indexHTTPFail, mutIndexHTTPFail) +STATSCOUNTER_DEF(indexHTTPReqFail, mutIndexHTTPReqFail) +STATSCOUNTER_DEF(indexESFail, mutIndexESFail) /* REST API for elasticsearch hits this URL: * http://<hostName>:<restPort>/<searchIndex>/<searchType> @@ -69,8 +69,8 @@ STATSCOUNTER_DEF(indexSuccess, mutIndexSuccess) typedef struct curl_slist HEADER; typedef struct _instanceData { int port; - int replyLen; int fdErrFile; /* error file fd or -1 if not open */ + pthread_mutex_t mutErrFile; uchar *server; uchar *uid; uchar *pwd; @@ -80,23 +80,30 @@ typedef struct _instanceData { uchar *tplName; uchar *timeout; uchar *bulkId; - uchar *restURL; /* last used URL for error reporting */ uchar *errorFile; - char *reply; sbool dynSrchIdx; sbool dynSrchType; sbool dynParent; sbool dynBulkId; sbool bulkmode; sbool asyncRepl; + sbool useHttps; +} instanceData; + +typedef struct wrkrInstanceData { + instanceData *pData; + int replyLen; + char *reply; + CURL *curlHandle; /* libcurl session handle */ + HEADER *postHeader; /* json POST request info */ + uchar *restURL; /* last used URL for error reporting */ struct { es_str_t *data; + int nmemb; /* number of messages in batch (for statistics counting) */ uchar *currTpl1; uchar *currTpl2; } batch; - CURL *curlHandle; /* libcurl session handle */ - HEADER *postHeader; /* json POST request info */ -} instanceData; +} wrkrInstanceData_t; /* tables for interfacing with the v6 config system */ @@ -114,9 +121,10 @@ static struct cnfparamdescr actpdescr[] = { { "dynparent", eCmdHdlrBinary, 0 }, { "bulkmode", eCmdHdlrBinary, 0 }, { "asyncrepl", eCmdHdlrBinary, 0 }, + { "usehttps", eCmdHdlrBinary, 0 }, { "timeout", eCmdHdlrGetWord, 0 }, { "errorfile", eCmdHdlrGetWord, 0 }, - { "template", eCmdHdlrGetWord, 1 }, + { "template", eCmdHdlrGetWord, 0 }, { "dynbulkid", eCmdHdlrBinary, 0 }, { "bulkid", eCmdHdlrGetWord, 0 }, }; @@ -126,12 +134,32 @@ static struct cnfparamblk actpblk = actpdescr }; +static rsRetVal curlSetup(wrkrInstanceData_t *pWrkrData, instanceData *pData); + BEGINcreateInstance CODESTARTcreateInstance - pData->restURL = NULL; pData->fdErrFile = -1; + pthread_mutex_init(&pData->mutErrFile, NULL); ENDcreateInstance +BEGINcreateWrkrInstance +CODESTARTcreateWrkrInstance +dbgprintf("omelasticsearch: createWrkrInstance\n"); + pWrkrData->restURL = NULL; + if(pData->bulkmode) { + pWrkrData->batch.currTpl1 = NULL; + pWrkrData->batch.currTpl2 = NULL; + if((pWrkrData->batch.data = es_newStr(1024)) == NULL) { + DBGPRINTF("omelasticsearch: error creating batch string " + "turned off bulk mode\n"); + pData->bulkmode = 0; /* at least it works */ + } + } + CHKiRet(curlSetup(pWrkrData, pWrkrData->pData)); +finalize_it: +dbgprintf("DDDD: createWrkrInstance,pData %p/%p, pWrkrData %p\n", pData, pWrkrData->pData, pWrkrData); +ENDcreateWrkrInstance + BEGINisCompatibleWithFeature CODESTARTisCompatibleWithFeature if(eFeat == sFEATURERepeatedMsgReduction) @@ -140,16 +168,9 @@ ENDisCompatibleWithFeature BEGINfreeInstance CODESTARTfreeInstance - if (pData->postHeader) { - curl_slist_free_all(pData->postHeader); - pData->postHeader = NULL; - } - if (pData->curlHandle) { - curl_easy_cleanup(pData->curlHandle); - pData->curlHandle = NULL; - } if(pData->fdErrFile != -1) close(pData->fdErrFile); + pthread_mutex_destroy(&pData->mutErrFile); free(pData->server); free(pData->uid); free(pData->pwd); @@ -158,11 +179,23 @@ CODESTARTfreeInstance free(pData->parent); free(pData->tplName); free(pData->timeout); - free(pData->restURL); free(pData->errorFile); free(pData->bulkId); ENDfreeInstance +BEGINfreeWrkrInstance +CODESTARTfreeWrkrInstance + if(pWrkrData->postHeader) { + curl_slist_free_all(pWrkrData->postHeader); + pWrkrData->postHeader = NULL; + } + if(pWrkrData->curlHandle) { + curl_easy_cleanup(pWrkrData->curlHandle); + pWrkrData->curlHandle = NULL; + } + free(pWrkrData->restURL); +ENDfreeWrkrInstance + BEGINdbgPrintInstInfo CODESTARTdbgPrintInstInfo dbgprintf("omelasticsearch\n"); @@ -179,6 +212,7 @@ CODESTARTdbgPrintInstInfo dbgprintf("\tdynamic search type=%d\n", pData->dynSrchType); dbgprintf("\tdynamic parent=%d\n", pData->dynParent); dbgprintf("\tasync replication=%d\n", pData->asyncRepl); + dbgprintf("\tuse https=%d\n", pData->useHttps); dbgprintf("\tbulkmode=%d\n", pData->bulkmode); dbgprintf("\terrorfile='%s'\n", pData->errorFile == NULL ? (uchar*)"(not configured)" : pData->errorFile); @@ -200,17 +234,22 @@ setBaseURL(instanceData *pData, es_str_t **url) *url = es_newStr(128); snprintf(portBuf, sizeof(portBuf), "%d", pData->port); - r = es_addBuf(url, "http://", sizeof("http://")-1); + if (pData->useHttps) { + r = es_addBuf(url, "https://", sizeof("https://")-1); + } + else { + r = es_addBuf(url, "http://", sizeof("http://")-1); + } if(r == 0) r = es_addBuf(url, (char*)pData->server, strlen((char*)pData->server)); if(r == 0) r = es_addChar(url, ':'); if(r == 0) r = es_addBuf(url, portBuf, strlen(portBuf)); - if(r == 0) r = es_addChar(url, '/'); + if(r == 0) es_addChar(url, '/'); RETiRet; } static inline rsRetVal -checkConn(instanceData *pData) +checkConn(wrkrInstanceData_t *pWrkrData) { es_str_t *url; CURL *curl = NULL; @@ -218,26 +257,32 @@ checkConn(instanceData *pData) char *cstr; DEFiRet; - setBaseURL(pData, &url); + setBaseURL(pWrkrData->pData, &url); curl = curl_easy_init(); if(curl == NULL) { DBGPRINTF("omelasticsearch: checkConn() curl_easy_init() failed\n"); ABORT_FINALIZE(RS_RET_SUSPENDED); } + /* Bodypart of request not needed, so set curl opt to nobody and httpget, otherwise lib-curl could sigsegv */ + curl_easy_setopt(curl, CURLOPT_HTTPGET, TRUE); + curl_easy_setopt(curl, CURLOPT_NOBODY, TRUE); + /* Only enable for debugging + curl_easy_setopt(curl, CURLOPT_VERBOSE, TRUE); */ + cstr = es_str2cstr(url, NULL); curl_easy_setopt(curl, CURLOPT_URL, cstr); free(cstr); - pData->reply = NULL; - pData->replyLen = 0; - curl_easy_setopt(curl, CURLOPT_WRITEDATA, pData); + pWrkrData->reply = NULL; + pWrkrData->replyLen = 0; + curl_easy_setopt(curl, CURLOPT_WRITEDATA, pWrkrData); res = curl_easy_perform(curl); if(res != CURLE_OK) { DBGPRINTF("omelasticsearch: checkConn() curl_easy_perform() " "failed: %s\n", curl_easy_strerror(res)); ABORT_FINALIZE(RS_RET_SUSPENDED); } - free(pData->reply); + free(pWrkrData->reply); DBGPRINTF("omelasticsearch: checkConn() completed with success\n"); finalize_it: @@ -250,7 +295,7 @@ finalize_it: BEGINtryResume CODESTARTtryResume DBGPRINTF("omelasticsearch: tryResume called\n"); - iRet = checkConn(pData); + iRet = checkConn(pWrkrData); ENDtryResume @@ -323,7 +368,7 @@ getIndexTypeAndParent(instanceData *pData, uchar **tpls, static rsRetVal -setCurlURL(instanceData *pData, uchar **tpls) +setCurlURL(wrkrInstanceData_t *pWrkrData, instanceData *pData, uchar **tpls) { char authBuf[1024]; uchar *searchIndex; @@ -358,14 +403,14 @@ setCurlURL(instanceData *pData, uchar **tpls) } if(parent != NULL) { if(r == 0) r = es_addBuf(&url, "parent=", sizeof("parent=")-1); - if(r == 0) r = es_addBuf(&url, (char*)parent, ustrlen(parent)); + if(r == 0) es_addBuf(&url, (char*)parent, ustrlen(parent)); } - free(pData->restURL); - pData->restURL = (uchar*)es_str2cstr(url, NULL); - curl_easy_setopt(pData->curlHandle, CURLOPT_URL, pData->restURL); + free(pWrkrData->restURL); + pWrkrData->restURL = (uchar*)es_str2cstr(url, NULL); + curl_easy_setopt(pWrkrData->curlHandle, CURLOPT_URL, pWrkrData->restURL); es_deleteStr(url); - DBGPRINTF("omelasticsearch: using REST URL: '%s'\n", pData->restURL); + DBGPRINTF("omelasticsearch: using REST URL: '%s'\n", pWrkrData->restURL); if(pData->uid != NULL) { rLocal = snprintf(authBuf, sizeof(authBuf), "%s:%s", pData->uid, @@ -376,8 +421,8 @@ setCurlURL(instanceData *pData, uchar **tpls) rLocal); ABORT_FINALIZE(RS_RET_ERR); } - curl_easy_setopt(pData->curlHandle, CURLOPT_USERPWD, authBuf); - curl_easy_setopt(pData->curlHandle, CURLOPT_PROXYAUTH, CURLAUTH_ANY); + curl_easy_setopt(pWrkrData->curlHandle, CURLOPT_USERPWD, authBuf); + curl_easy_setopt(pWrkrData->curlHandle, CURLOPT_PROXYAUTH, CURLAUTH_ANY); } finalize_it: RETiRet; @@ -389,7 +434,7 @@ finalize_it: * index changes. */ static rsRetVal -buildBatch(instanceData *pData, uchar *message, uchar **tpls) +buildBatch(wrkrInstanceData_t *pWrkrData, uchar *message, uchar **tpls) { int length = strlen((char *)message); int r; @@ -404,28 +449,29 @@ buildBatch(instanceData *pData, uchar *message, uchar **tpls) # define META_ID "\", \"_id\":\"" # define META_END "\"}}\n" - getIndexTypeAndParent(pData, tpls, &searchIndex, &searchType, &parent, &bulkId); - r = es_addBuf(&pData->batch.data, META_STRT, sizeof(META_STRT)-1); - if(r == 0) r = es_addBuf(&pData->batch.data, (char*)searchIndex, + getIndexTypeAndParent(pWrkrData->pData, tpls, &searchIndex, &searchType, &parent, &bulkId); + r = es_addBuf(&pWrkrData->batch.data, META_STRT, sizeof(META_STRT)-1); + if(r == 0) r = es_addBuf(&pWrkrData->batch.data, (char*)searchIndex, ustrlen(searchIndex)); - if(r == 0) r = es_addBuf(&pData->batch.data, META_TYPE, sizeof(META_TYPE)-1); - if(r == 0) r = es_addBuf(&pData->batch.data, (char*)searchType, + if(r == 0) r = es_addBuf(&pWrkrData->batch.data, META_TYPE, sizeof(META_TYPE)-1); + if(r == 0) r = es_addBuf(&pWrkrData->batch.data, (char*)searchType, ustrlen(searchType)); if(parent != NULL) { - if(r == 0) r = es_addBuf(&pData->batch.data, META_PARENT, sizeof(META_PARENT)-1); - if(r == 0) r = es_addBuf(&pData->batch.data, (char*)parent, ustrlen(parent)); + if(r == 0) r = es_addBuf(&pWrkrData->batch.data, META_PARENT, sizeof(META_PARENT)-1); + if(r == 0) r = es_addBuf(&pWrkrData->batch.data, (char*)parent, ustrlen(parent)); } if(bulkId != NULL) { - if(r == 0) r = es_addBuf(&pData->batch.data, META_ID, sizeof(META_ID)-1); - if(r == 0) r = es_addBuf(&pData->batch.data, (char*)bulkId, ustrlen(bulkId)); + if(r == 0) r = es_addBuf(&pWrkrData->batch.data, META_ID, sizeof(META_ID)-1); + if(r == 0) r = es_addBuf(&pWrkrData->batch.data, (char*)bulkId, ustrlen(bulkId)); } - if(r == 0) r = es_addBuf(&pData->batch.data, META_END, sizeof(META_END)-1); - if(r == 0) r = es_addBuf(&pData->batch.data, (char*)message, length); - if(r == 0) r = es_addBuf(&pData->batch.data, "\n", sizeof("\n")-1); + if(r == 0) r = es_addBuf(&pWrkrData->batch.data, META_END, sizeof(META_END)-1); + if(r == 0) r = es_addBuf(&pWrkrData->batch.data, (char*)message, length); + if(r == 0) r = es_addBuf(&pWrkrData->batch.data, "\n", sizeof("\n")-1); if(r != 0) { DBGPRINTF("omelasticsearch: growing batch failed with code %d\n", r); ABORT_FINALIZE(RS_RET_ERR); } + ++pWrkrData->batch.nmemb; iRet = RS_RET_DEFER_COMMIT; finalize_it: @@ -438,7 +484,7 @@ finalize_it: * needs to be closed, HUP must be sent. */ static inline rsRetVal -writeDataError(instanceData *pData, cJSON **pReplyRoot, uchar *reqmsg) +writeDataError(wrkrInstanceData_t *pWrkrData, instanceData *pData, cJSON **pReplyRoot, uchar *reqmsg) { char *rendered = NULL; cJSON *errRoot; @@ -446,6 +492,7 @@ writeDataError(instanceData *pData, cJSON **pReplyRoot, uchar *reqmsg) cJSON *replyRoot = *pReplyRoot; size_t toWrite; ssize_t wrRet; + sbool bMutLocked = 0; char errStr[1024]; DEFiRet; @@ -455,6 +502,9 @@ writeDataError(instanceData *pData, cJSON **pReplyRoot, uchar *reqmsg) FINALIZE; } + pthread_mutex_lock(&pData->mutErrFile); + bMutLocked = 1; + if(pData->fdErrFile == -1) { pData->fdErrFile = open((char*)pData->errorFile, O_WRONLY|O_CREAT|O_APPEND|O_LARGEFILE|O_CLOEXEC, @@ -466,7 +516,7 @@ writeDataError(instanceData *pData, cJSON **pReplyRoot, uchar *reqmsg) } } if((req=cJSON_CreateObject()) == NULL) ABORT_FINALIZE(RS_RET_ERR); - cJSON_AddItemToObject(req, "url", cJSON_CreateString((char*)pData->restURL)); + cJSON_AddItemToObject(req, "url", cJSON_CreateString((char*)pWrkrData->restURL)); cJSON_AddItemToObject(req, "postdata", cJSON_CreateString((char*)reqmsg)); if((errRoot=cJSON_CreateObject()) == NULL) ABORT_FINALIZE(RS_RET_ERR); @@ -483,19 +533,19 @@ writeDataError(instanceData *pData, cJSON **pReplyRoot, uchar *reqmsg) DBGPRINTF("omelasticsearch: error %d writing error file, write returns %lld\n", errno, (long long) wrRet); } - free(rendered); cJSON_Delete(errRoot); *pReplyRoot = NULL; /* tell caller not to delete once again! */ finalize_it: - if(rendered != NULL) - free(rendered); + if(bMutLocked) + pthread_mutex_unlock(&pData->mutErrFile); + free(rendered); RETiRet; } static inline rsRetVal -checkResultBulkmode(instanceData *pData, cJSON *root) +checkResultBulkmode(wrkrInstanceData_t *pWrkrData, cJSON *root) { int i; int numitems; @@ -509,7 +559,7 @@ checkResultBulkmode(instanceData *pData, cJSON *root) if(items == NULL || items->type != cJSON_Array) { DBGPRINTF("omelasticsearch: error in elasticsearch reply: " "bulkmode insert does not return array, reply is: %s\n", - pData->reply); + pWrkrData->reply); ABORT_FINALIZE(RS_RET_DATAFAIL); } numitems = cJSON_GetArraySize(items); @@ -527,10 +577,10 @@ DBGPRINTF("omelasticsearch: %d items in reply\n", numitems); "cannot obtain 'create' item for #%d\n", i); ABORT_FINALIZE(RS_RET_DATAFAIL); } - ok = cJSON_GetObjectItem(create, "ok"); - if(ok == NULL || ok->type != cJSON_True) { + ok = cJSON_GetObjectItem(create, "status"); + if(ok == NULL || ok->type != cJSON_Number || ok->valueint < 0 || ok->valueint > 299) { DBGPRINTF("omelasticsearch: error in elasticsearch reply: " - "item %d, prop ok (%p) not ok\n", i, ok); + "item %d, status is %d\n", i, ok->valueint); ABORT_FINALIZE(RS_RET_DATAFAIL); } } @@ -541,23 +591,25 @@ finalize_it: static inline rsRetVal -checkResult(instanceData *pData, uchar *reqmsg) +checkResult(wrkrInstanceData_t *pWrkrData, uchar *reqmsg) { cJSON *root; - cJSON *ok; + cJSON *status; DEFiRet; - root = cJSON_Parse(pData->reply); + root = cJSON_Parse(pWrkrData->reply); if(root == NULL) { DBGPRINTF("omelasticsearch: could not parse JSON result \n"); ABORT_FINALIZE(RS_RET_ERR); } - if(pData->bulkmode) { - iRet = checkResultBulkmode(pData, root); + if(pWrkrData->pData->bulkmode) { + iRet = checkResultBulkmode(pWrkrData, root); } else { - ok = cJSON_GetObjectItem(root, "ok"); - if(ok == NULL || ok->type != cJSON_True) { + status = cJSON_GetObjectItem(root, "status"); + /* as far as we know, no "status" means all went well */ + if(status != NULL && + (status->type == cJSON_Number || status->valueint >= 0 || status->valueint <= 299)) { iRet = RS_RET_DATAFAIL; } } @@ -566,31 +618,35 @@ checkResult(instanceData *pData, uchar *reqmsg) * these in any case. */ if(iRet == RS_RET_DATAFAIL) { - writeDataError(pData, &root, reqmsg); + STATSCOUNTER_INC(indexESFail, mutIndexESFail); + writeDataError(pWrkrData, pWrkrData->pData, &root, reqmsg); iRet = RS_RET_OK; /* we have handled the problem! */ } finalize_it: if(root != NULL) cJSON_Delete(root); + if(iRet != RS_RET_OK) { + STATSCOUNTER_INC(indexESFail, mutIndexESFail); + } RETiRet; } static rsRetVal -curlPost(instanceData *pData, uchar *message, int msglen, uchar **tpls) +curlPost(wrkrInstanceData_t *pWrkrData, uchar *message, int msglen, uchar **tpls, int nmsgs) { CURLcode code; - CURL *curl = pData->curlHandle; + CURL *curl = pWrkrData->curlHandle; DEFiRet; - pData->reply = NULL; - pData->replyLen = 0; + pWrkrData->reply = NULL; + pWrkrData->replyLen = 0; - if(pData->dynSrchIdx || pData->dynSrchType || pData->dynParent) - CHKiRet(setCurlURL(pData, tpls)); + if(pWrkrData->pData->dynSrchIdx || pWrkrData->pData->dynSrchType || pWrkrData->pData->dynParent) + CHKiRet(setCurlURL(pWrkrData, pWrkrData->pData, tpls)); - curl_easy_setopt(curl, CURLOPT_WRITEDATA, pData); + curl_easy_setopt(curl, CURLOPT_WRITEDATA, pWrkrData); curl_easy_setopt(curl, CURLOPT_POSTFIELDS, (char *)message); curl_easy_setopt(curl, CURLOPT_POSTFIELDSIZE, msglen); code = curl_easy_perform(curl); @@ -599,57 +655,67 @@ curlPost(instanceData *pData, uchar *message, int msglen, uchar **tpls) case CURLE_COULDNT_RESOLVE_PROXY: case CURLE_COULDNT_CONNECT: case CURLE_WRITE_ERROR: - STATSCOUNTER_INC(indexConFail, mutIndexConFail); + STATSCOUNTER_INC(indexHTTPReqFail, mutIndexHTTPReqFail); + indexHTTPFail += nmsgs; DBGPRINTF("omelasticsearch: we are suspending ourselfs due " "to failure %lld of curl_easy_perform()\n", (long long) code); ABORT_FINALIZE(RS_RET_SUSPENDED); default: - STATSCOUNTER_INC(indexSubmit, mutIndexSubmit); break; } - pData->reply[pData->replyLen] = '\0'; /* byte has been reserved in malloc */ - DBGPRINTF("omelasticsearch: es reply: '%s'\n", pData->reply); + DBGPRINTF("omelasticsearch: pWrkrData replyLen = '%d'\n", pWrkrData->replyLen); + if(pWrkrData->replyLen > 0) { + pWrkrData->reply[pWrkrData->replyLen] = '\0'; /* Append 0 Byte if replyLen is above 0 - byte has been reserved in malloc */ + } + DBGPRINTF("omelasticsearch: pWrkrData reply: '%s'\n", pWrkrData->reply); - CHKiRet(checkResult(pData, message)); + CHKiRet(checkResult(pWrkrData, message)); finalize_it: - free(pData->reply); + free(pWrkrData->reply); RETiRet; } BEGINbeginTransaction CODESTARTbeginTransaction -dbgprintf("omelasticsearch: beginTransaction\n"); - if(!pData->bulkmode) { +dbgprintf("omelasticsearch: beginTransaction, pWrkrData %p, pData %p\n", pWrkrData, pWrkrData->pData); + if(!pWrkrData->pData->bulkmode) { FINALIZE; } - es_emptyStr(pData->batch.data); + es_emptyStr(pWrkrData->batch.data); + pWrkrData->batch.nmemb = 0; finalize_it: ENDbeginTransaction BEGINdoAction CODESTARTdoAction - if(pData->bulkmode) { - CHKiRet(buildBatch(pData, ppString[0], ppString)); + STATSCOUNTER_INC(indexSubmit, mutIndexSubmit); + if(pWrkrData->pData->bulkmode) { + CHKiRet(buildBatch(pWrkrData, ppString[0], ppString)); } else { - CHKiRet(curlPost(pData, ppString[0], strlen((char*)ppString[0]), - ppString)); + CHKiRet(curlPost(pWrkrData, ppString[0], strlen((char*)ppString[0]), + ppString, 1)); } finalize_it: -dbgprintf("omelasticsearch: result doAction: %d (bulkmode %d)\n", iRet, pData->bulkmode); +dbgprintf("omelasticsearch: result doAction: %d (bulkmode %d)\n", iRet, pWrkrData->pData->bulkmode); ENDdoAction BEGINendTransaction - char *cstr; + char *cstr = NULL; CODESTARTendTransaction dbgprintf("omelasticsearch: endTransaction init\n"); - cstr = es_str2cstr(pData->batch.data, NULL); - dbgprintf("omelasticsearch: endTransaction, batch: '%s'\n", cstr); - CHKiRet(curlPost(pData, (uchar*) cstr, strlen(cstr), NULL)); + /* End Transaction only if batch data is not empty */ + if (pWrkrData->batch.data != NULL ) { + cstr = es_str2cstr(pWrkrData->batch.data, NULL); + dbgprintf("omelasticsearch: endTransaction, batch: '%s'\n", cstr); + CHKiRet(curlPost(pWrkrData, (uchar*) cstr, strlen(cstr), NULL, pWrkrData->batch.nmemb)); + } + else + dbgprintf("omelasticsearch: endTransaction, pWrkrData->batch.data is NULL, nothing to send. \n"); finalize_it: free(cstr); dbgprintf("omelasticsearch: endTransaction done with %d\n", iRet); @@ -660,24 +726,24 @@ size_t curlResult(void *ptr, size_t size, size_t nmemb, void *userdata) { char *p = (char *)ptr; - instanceData *pData = (instanceData*) userdata; + wrkrInstanceData_t *pWrkrData = (wrkrInstanceData_t*) userdata; char *buf; size_t newlen; - newlen = pData->replyLen + size*nmemb; - if((buf = realloc(pData->reply, newlen + 1)) == NULL) { + newlen = pWrkrData->replyLen + size*nmemb; + if((buf = realloc(pWrkrData->reply, newlen + 1)) == NULL) { DBGPRINTF("omelasticsearch: realloc failed in curlResult\n"); return 0; /* abort due to failure */ } - memcpy(buf+pData->replyLen, p, size*nmemb); - pData->replyLen = newlen; - pData->reply = buf; + memcpy(buf+pWrkrData->replyLen, p, size*nmemb); + pWrkrData->replyLen = newlen; + pWrkrData->reply = buf; return size*nmemb; } static rsRetVal -curlSetup(instanceData *pData) +curlSetup(wrkrInstanceData_t *pWrkrData, instanceData *pData) { HEADER *header; CURL *handle; @@ -693,13 +759,13 @@ curlSetup(instanceData *pData) curl_easy_setopt(handle, CURLOPT_WRITEFUNCTION, curlResult); curl_easy_setopt(handle, CURLOPT_POST, 1); - pData->curlHandle = handle; - pData->postHeader = header; + pWrkrData->curlHandle = handle; + pWrkrData->postHeader = header; if( pData->bulkmode || (pData->dynSrchIdx == 0 && pData->dynSrchType == 0 && pData->dynParent == 0)) { /* in this case, we know no tpls are involved in the request-->NULL OK! */ - setCurlURL(pData, NULL); + setCurlURL(pWrkrData, pData, NULL); } if(Debug) { @@ -726,6 +792,7 @@ setInstParamDefaults(instanceData *pData) pData->dynSrchType = 0; pData->dynParent = 0; pData->asyncRepl = 0; + pData->useHttps = 0; pData->bulkmode = 0; pData->tplName = NULL; pData->errorFile = NULL; @@ -776,6 +843,8 @@ CODESTARTnewActInst pData->timeout = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); } else if(!strcmp(actpblk.descr[i].name, "asyncrepl")) { pData->asyncRepl = pvals[i].val.d.n; + } else if(!strcmp(actpblk.descr[i].name, "usehttps")) { + pData->useHttps = pvals[i].val.d.n; } else if(!strcmp(actpblk.descr[i].name, "template")) { pData->tplName = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); } else if(!strcmp(actpblk.descr[i].name, "dynbulkid")) { @@ -819,16 +888,6 @@ CODESTARTnewActInst ABORT_FINALIZE(RS_RET_CONFIG_ERROR); } - if(pData->bulkmode) { - pData->batch.currTpl1 = NULL; - pData->batch.currTpl2 = NULL; - if((pData->batch.data = es_newStr(1024)) == NULL) { - DBGPRINTF("omelasticsearch: error creating batch string " - "turned off bulk mode\n"); - pData->bulkmode = 0; /* at least it works */ - } - } - iNumTpls = 1; if(pData->dynSrchIdx) ++iNumTpls; if(pData->dynSrchType) ++iNumTpls; @@ -920,9 +979,6 @@ CODESTARTnewActInst pData->searchIndex = (uchar*) strdup("system"); if(pData->searchType == NULL) pData->searchType = (uchar*) strdup("events"); - - CHKiRet(curlSetup(pData)); - CODE_STD_FINALIZERnewActInst cnfparamvalsDestruct(pvals, &actpblk); ENDnewActInst @@ -960,6 +1016,7 @@ ENDmodExit BEGINqueryEtryPt CODESTARTqueryEtryPt CODEqueryEtryPt_STD_OMOD_QUERIES +CODEqueryEtryPt_STD_OMOD8_QUERIES CODEqueryEtryPt_IsCompatibleWithFeature_IF_OMOD_QUERIES CODEqueryEtryPt_STD_CONF2_OMOD_QUERIES CODEqueryEtryPt_doHUP @@ -981,15 +1038,19 @@ CODEmodInit_QueryRegCFSLineHdlr /* support statistics gathering */ CHKiRet(statsobj.Construct(&indexStats)); - CHKiRet(statsobj.SetName(indexStats, (uchar *)"elasticsearch")); - CHKiRet(statsobj.AddCounter(indexStats, (uchar *)"connfail", - ctrType_IntCtr, &indexConFail)); - CHKiRet(statsobj.AddCounter(indexStats, (uchar *)"submits", - ctrType_IntCtr, &indexSubmit)); - CHKiRet(statsobj.AddCounter(indexStats, (uchar *)"failed", - ctrType_IntCtr, &indexFailed)); - CHKiRet(statsobj.AddCounter(indexStats, (uchar *)"success", - ctrType_IntCtr, &indexSuccess)); + CHKiRet(statsobj.SetName(indexStats, (uchar *)"omelasticsearch")); + STATSCOUNTER_INIT(indexSubmit, mutIndexSubmit); + CHKiRet(statsobj.AddCounter(indexStats, (uchar *)"submitted", + ctrType_IntCtr, CTR_FLAG_RESETTABLE, &indexSubmit)); + STATSCOUNTER_INIT(indexHTTPFail, mutIndexHTTPFail); + CHKiRet(statsobj.AddCounter(indexStats, (uchar *)"failed.http", + ctrType_IntCtr, CTR_FLAG_RESETTABLE, &indexHTTPFail)); + STATSCOUNTER_INIT(indexHTTPReqFail, mutIndexHTTPReqFail); + CHKiRet(statsobj.AddCounter(indexStats, (uchar *)"failed.httprequests", + ctrType_IntCtr, CTR_FLAG_RESETTABLE, &indexHTTPReqFail)); + STATSCOUNTER_INIT(indexESFail, mutIndexESFail); + CHKiRet(statsobj.AddCounter(indexStats, (uchar *)"failed.es", + ctrType_IntCtr, CTR_FLAG_RESETTABLE, &indexESFail)); CHKiRet(statsobj.ConstructFinalize(indexStats)); ENDmodInit diff --git a/plugins/omgssapi/Makefile.in b/plugins/omgssapi/Makefile.in index 3bc7c54..8841bd3 100644 --- a/plugins/omgssapi/Makefile.in +++ b/plugins/omgssapi/Makefile.in @@ -154,7 +154,6 @@ GREP = @GREP@ GSS_LIBS = @GSS_LIBS@ GUARDTIME_CFLAGS = @GUARDTIME_CFLAGS@ GUARDTIME_LIBS = @GUARDTIME_LIBS@ -HAVE_LIBGCRYPT_CONFIG = @HAVE_LIBGCRYPT_CONFIG@ HAVE_MYSQL_CONFIG = @HAVE_MYSQL_CONFIG@ HAVE_ORACLE_CONFIG = @HAVE_ORACLE_CONFIG@ HAVE_PGSQL_CONFIG = @HAVE_PGSQL_CONFIG@ @@ -175,14 +174,15 @@ LEXLIB = @LEXLIB@ LEX_OUTPUT_ROOT = @LEX_OUTPUT_ROOT@ LIBDBI_CFLAGS = @LIBDBI_CFLAGS@ LIBDBI_LIBS = @LIBDBI_LIBS@ -LIBEE_CFLAGS = @LIBEE_CFLAGS@ -LIBEE_LIBS = @LIBEE_LIBS@ LIBESTR_CFLAGS = @LIBESTR_CFLAGS@ LIBESTR_LIBS = @LIBESTR_LIBS@ LIBGCRYPT_CFLAGS = @LIBGCRYPT_CFLAGS@ +LIBGCRYPT_CONFIG = @LIBGCRYPT_CONFIG@ LIBGCRYPT_LIBS = @LIBGCRYPT_LIBS@ LIBLOGGING_CFLAGS = @LIBLOGGING_CFLAGS@ LIBLOGGING_LIBS = @LIBLOGGING_LIBS@ +LIBLOGGING_STDLOG_CFLAGS = @LIBLOGGING_STDLOG_CFLAGS@ +LIBLOGGING_STDLOG_LIBS = @LIBLOGGING_STDLOG_LIBS@ LIBLOGNORM_CFLAGS = @LIBLOGNORM_CFLAGS@ LIBLOGNORM_LIBS = @LIBLOGNORM_LIBS@ LIBM = @LIBM@ @@ -207,6 +207,8 @@ NM = @NM@ NMEDIT = @NMEDIT@ OBJDUMP = @OBJDUMP@ OBJEXT = @OBJEXT@ +OPENSSL_CFLAGS = @OPENSSL_CFLAGS@ +OPENSSL_LIBS = @OPENSSL_LIBS@ ORACLE_CFLAGS = @ORACLE_CFLAGS@ ORACLE_LIBS = @ORACLE_LIBS@ OTOOL = @OTOOL@ @@ -387,22 +389,25 @@ distclean-compile: @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/omgssapi_la-omgssapi.Plo@am__quote@ .c.o: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(COMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ $< -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Po +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.o$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c $< +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ $< .c.obj: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(COMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ `$(CYGPATH_W) '$<'` -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Po +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.obj$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ `$(CYGPATH_W) '$<'` &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c `$(CYGPATH_W) '$<'` +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ `$(CYGPATH_W) '$<'` .c.lo: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(LTCOMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ $< -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Plo +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.lo$$||'`;\ +@am__fastdepCC_TRUE@ $(LTCOMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Plo @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=yes @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ @am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(LTCOMPILE) -c -o $@ $< diff --git a/plugins/omgssapi/omgssapi.c b/plugins/omgssapi/omgssapi.c index 818a7cf..25400c7 100644 --- a/plugins/omgssapi/omgssapi.c +++ b/plugins/omgssapi/omgssapi.c @@ -4,7 +4,7 @@ * NOTE: read comments in module-template.h to understand how this file * works! * - * Copyright 2007, 2008 Rainer Gerhards and Adiscon GmbH. + * Copyright 2007-2014 Rainer Gerhards and Adiscon GmbH. * * This file is part of rsyslog. * @@ -88,6 +88,10 @@ typedef struct _instanceData { OM_uint32 gss_flags; } instanceData; +typedef struct wrkrInstanceData { + instanceData *pData; +} wrkrInstanceData_t; + /* config data */ typedef enum gss_mode_e { @@ -101,6 +105,7 @@ static struct configSettings_s { gss_mode_t gss_mode; } cs; +static pthread_mutex_t mutDoAct = PTHREAD_MUTEX_INITIALIZER; /* get the syslog forward port from selector_t. The passed in * struct must be one that is setup for forwarding. @@ -122,6 +127,11 @@ CODESTARTcreateInstance ENDcreateInstance +BEGINcreateWrkrInstance +CODESTARTcreateWrkrInstance +ENDcreateWrkrInstance + + BEGINisCompatibleWithFeature CODESTARTisCompatibleWithFeature if(eFeat == sFEATURERepeatedMsgReduction) @@ -163,6 +173,9 @@ CODESTARTfreeInstance free(pData->f_hname); ENDfreeInstance +BEGINfreeWrkrInstance +CODESTARTfreeWrkrInstance +ENDfreeWrkrInstance BEGINdbgPrintInstInfo CODESTARTdbgPrintInstInfo @@ -379,14 +392,19 @@ static rsRetVal doTryResume(instanceData *pData) BEGINtryResume CODESTARTtryResume - iRet = doTryResume(pData); + pthread_mutex_lock(&mutDoAct); + iRet = doTryResume(pWrkrData->pData); + pthread_mutex_unlock(&mutDoAct); ENDtryResume BEGINdoAction char *psz = NULL; /* temporary buffering */ register unsigned l; int iMaxLine; + instanceData *pData; CODESTARTdoAction + pthread_mutex_lock(&mutDoAct); + pData = pWrkrData->pData; switch (pData->eDestState) { case eDestFORW_SUSP: dbgprintf("internal error in omgssapi.c, eDestFORW_SUSP in doAction()!\n"); @@ -465,6 +483,7 @@ finalize_it: free(psz); } # endif + pthread_mutex_unlock(&mutDoAct); ENDdoAction @@ -656,6 +675,7 @@ ENDmodExit BEGINqueryEtryPt CODESTARTqueryEtryPt CODEqueryEtryPt_STD_OMOD_QUERIES +CODEqueryEtryPt_STD_OMOD8_QUERIES ENDqueryEtryPt diff --git a/plugins/omhdfs/Makefile.in b/plugins/omhdfs/Makefile.in index 85b6a2c..a54a4bd 100644 --- a/plugins/omhdfs/Makefile.in +++ b/plugins/omhdfs/Makefile.in @@ -154,7 +154,6 @@ GREP = @GREP@ GSS_LIBS = @GSS_LIBS@ GUARDTIME_CFLAGS = @GUARDTIME_CFLAGS@ GUARDTIME_LIBS = @GUARDTIME_LIBS@ -HAVE_LIBGCRYPT_CONFIG = @HAVE_LIBGCRYPT_CONFIG@ HAVE_MYSQL_CONFIG = @HAVE_MYSQL_CONFIG@ HAVE_ORACLE_CONFIG = @HAVE_ORACLE_CONFIG@ HAVE_PGSQL_CONFIG = @HAVE_PGSQL_CONFIG@ @@ -175,14 +174,15 @@ LEXLIB = @LEXLIB@ LEX_OUTPUT_ROOT = @LEX_OUTPUT_ROOT@ LIBDBI_CFLAGS = @LIBDBI_CFLAGS@ LIBDBI_LIBS = @LIBDBI_LIBS@ -LIBEE_CFLAGS = @LIBEE_CFLAGS@ -LIBEE_LIBS = @LIBEE_LIBS@ LIBESTR_CFLAGS = @LIBESTR_CFLAGS@ LIBESTR_LIBS = @LIBESTR_LIBS@ LIBGCRYPT_CFLAGS = @LIBGCRYPT_CFLAGS@ +LIBGCRYPT_CONFIG = @LIBGCRYPT_CONFIG@ LIBGCRYPT_LIBS = @LIBGCRYPT_LIBS@ LIBLOGGING_CFLAGS = @LIBLOGGING_CFLAGS@ LIBLOGGING_LIBS = @LIBLOGGING_LIBS@ +LIBLOGGING_STDLOG_CFLAGS = @LIBLOGGING_STDLOG_CFLAGS@ +LIBLOGGING_STDLOG_LIBS = @LIBLOGGING_STDLOG_LIBS@ LIBLOGNORM_CFLAGS = @LIBLOGNORM_CFLAGS@ LIBLOGNORM_LIBS = @LIBLOGNORM_LIBS@ LIBM = @LIBM@ @@ -207,6 +207,8 @@ NM = @NM@ NMEDIT = @NMEDIT@ OBJDUMP = @OBJDUMP@ OBJEXT = @OBJEXT@ +OPENSSL_CFLAGS = @OPENSSL_CFLAGS@ +OPENSSL_LIBS = @OPENSSL_LIBS@ ORACLE_CFLAGS = @ORACLE_CFLAGS@ ORACLE_LIBS = @ORACLE_LIBS@ OTOOL = @OTOOL@ @@ -387,22 +389,25 @@ distclean-compile: @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/omhdfs_la-omhdfs.Plo@am__quote@ .c.o: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(COMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ $< -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Po +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.o$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c $< +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ $< .c.obj: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(COMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ `$(CYGPATH_W) '$<'` -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Po +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.obj$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ `$(CYGPATH_W) '$<'` &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c `$(CYGPATH_W) '$<'` +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ `$(CYGPATH_W) '$<'` .c.lo: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(LTCOMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ $< -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Plo +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.lo$$||'`;\ +@am__fastdepCC_TRUE@ $(LTCOMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Plo @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=yes @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ @am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(LTCOMPILE) -c -o $@ $< diff --git a/plugins/omhdfs/omhdfs.c b/plugins/omhdfs/omhdfs.c index f8a7e73..e173fb3 100644 --- a/plugins/omhdfs/omhdfs.c +++ b/plugins/omhdfs/omhdfs.c @@ -4,7 +4,7 @@ * NOTE: read comments in module-template.h to understand how this file * works! * - * Copyright 2010 Rainer Gerhards and Adiscon GmbH. + * Copyright 2010-2014 Rainer Gerhards and Adiscon GmbH. * * This program is free software; you can redistribute it and/or * modify it under the terms of the GNU General Public License @@ -36,7 +36,12 @@ #include <unistd.h> #include <sys/file.h> #include <pthread.h> -#include <hdfs.h> +#ifdef HAVE_HDFS_H +# include <hdfs.h> +#endif +#ifdef HAVE_HADOOP_HDFS_H +# include <hadoop/hdfs.h> +#endif #include "syslogd-types.h" #include "srUtils.h" @@ -51,7 +56,7 @@ MODULE_TYPE_OUTPUT MODULE_TYPE_NOKEEP -MODULE_CNFNAME("omhdfs") +/* MODULE_CNFNAME("omhdfs") we need this only when we convert the module to v2 config system */ /* internal structures */ @@ -60,6 +65,7 @@ DEFobjCurrIf(errmsg) /* global data */ static struct hashtable *files; /* holds all file objects that we know */ +static pthread_mutex_t mutDoAct = PTHREAD_MUTEX_INITIALIZER; typedef struct configSettings_s { uchar *fileName; @@ -69,11 +75,6 @@ typedef struct configSettings_s { } configSettings_t; static configSettings_t cs; - -BEGINinitConfVars /* (re)set config variables to default values */ -CODESTARTinitConfVars -ENDinitConfVars - typedef struct { uchar *name; hdfsFS fs; @@ -91,6 +92,10 @@ typedef struct _instanceData { unsigned offsBuf; } instanceData; +typedef struct wrkrInstanceData { + instanceData *pData; +} wrkrInstanceData_t; + /* forward definitions (down here, need data types) */ static inline rsRetVal fileClose(file_t *pFile); @@ -387,6 +392,11 @@ CODESTARTcreateInstance ENDcreateInstance +BEGINcreateWrkrInstance +CODESTARTcreateWrkrInstance +ENDcreateWrkrInstance + + BEGINfreeInstance CODESTARTfreeInstance if(pData->pFile != NULL) @@ -394,8 +404,15 @@ CODESTARTfreeInstance ENDfreeInstance +BEGINfreeWrkrInstance +CODESTARTfreeWrkrInstance +ENDfreeWrkrInstance + + BEGINtryResume + instanceData *pData = pWrkrData->pData; CODESTARTtryResume + pthread_mutex_lock(&mutDoAct); fileClose(pData->pFile); fileOpen(pData->pFile); if(pData->pFile->fh == NULL){ @@ -403,6 +420,7 @@ CODESTARTtryResume pData->pFile->name); iRet = RS_RET_SUSPENDED; } + pthread_mutex_unlock(&mutDoAct); ENDtryResume @@ -413,20 +431,26 @@ ENDbeginTransaction BEGINdoAction + instanceData *pData = pWrkrData->pData; CODESTARTdoAction DBGPRINTF("omhdfs: action to to write to %s\n", pData->pFile->name); + pthread_mutex_lock(&mutDoAct); iRet = addData(pData, ppString[0]); -dbgprintf("omhdfs: done doAction\n"); + DBGPRINTF("omhdfs: done doAction\n"); + pthread_mutex_unlock(&mutDoAct); ENDdoAction BEGINendTransaction + instanceData *pData = pWrkrData->pData; CODESTARTendTransaction dbgprintf("omhdfs: endTransaction\n"); + pthread_mutex_lock(&mutDoAct); if(pData->offsBuf != 0) { DBGPRINTF("omhdfs: data unwritten at end of transaction, persisting...\n"); iRet = fileWrite(pData->pFile, pData->ioBuf, &pData->offsBuf); } + pthread_mutex_unlock(&mutDoAct); ENDendTransaction @@ -526,6 +550,7 @@ BEGINqueryEtryPt CODESTARTqueryEtryPt CODEqueryEtryPt_STD_OMOD_QUERIES CODEqueryEtryPt_TXIF_OMOD_QUERIES /* we support the transactional interface! */ +CODEqueryEtryPt_STD_OMOD8_QUERIES CODEqueryEtryPt_doHUP ENDqueryEtryPt diff --git a/plugins/omhiredis/Makefile.in b/plugins/omhiredis/Makefile.in index 7dfc453..6ca8153 100644 --- a/plugins/omhiredis/Makefile.in +++ b/plugins/omhiredis/Makefile.in @@ -155,7 +155,6 @@ GREP = @GREP@ GSS_LIBS = @GSS_LIBS@ GUARDTIME_CFLAGS = @GUARDTIME_CFLAGS@ GUARDTIME_LIBS = @GUARDTIME_LIBS@ -HAVE_LIBGCRYPT_CONFIG = @HAVE_LIBGCRYPT_CONFIG@ HAVE_MYSQL_CONFIG = @HAVE_MYSQL_CONFIG@ HAVE_ORACLE_CONFIG = @HAVE_ORACLE_CONFIG@ HAVE_PGSQL_CONFIG = @HAVE_PGSQL_CONFIG@ @@ -176,14 +175,15 @@ LEXLIB = @LEXLIB@ LEX_OUTPUT_ROOT = @LEX_OUTPUT_ROOT@ LIBDBI_CFLAGS = @LIBDBI_CFLAGS@ LIBDBI_LIBS = @LIBDBI_LIBS@ -LIBEE_CFLAGS = @LIBEE_CFLAGS@ -LIBEE_LIBS = @LIBEE_LIBS@ LIBESTR_CFLAGS = @LIBESTR_CFLAGS@ LIBESTR_LIBS = @LIBESTR_LIBS@ LIBGCRYPT_CFLAGS = @LIBGCRYPT_CFLAGS@ +LIBGCRYPT_CONFIG = @LIBGCRYPT_CONFIG@ LIBGCRYPT_LIBS = @LIBGCRYPT_LIBS@ LIBLOGGING_CFLAGS = @LIBLOGGING_CFLAGS@ LIBLOGGING_LIBS = @LIBLOGGING_LIBS@ +LIBLOGGING_STDLOG_CFLAGS = @LIBLOGGING_STDLOG_CFLAGS@ +LIBLOGGING_STDLOG_LIBS = @LIBLOGGING_STDLOG_LIBS@ LIBLOGNORM_CFLAGS = @LIBLOGNORM_CFLAGS@ LIBLOGNORM_LIBS = @LIBLOGNORM_LIBS@ LIBM = @LIBM@ @@ -208,6 +208,8 @@ NM = @NM@ NMEDIT = @NMEDIT@ OBJDUMP = @OBJDUMP@ OBJEXT = @OBJEXT@ +OPENSSL_CFLAGS = @OPENSSL_CFLAGS@ +OPENSSL_LIBS = @OPENSSL_LIBS@ ORACLE_CFLAGS = @ORACLE_CFLAGS@ ORACLE_LIBS = @ORACLE_LIBS@ OTOOL = @OTOOL@ @@ -389,22 +391,25 @@ distclean-compile: @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/omhiredis_la-omhiredis.Plo@am__quote@ .c.o: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(COMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ $< -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Po +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.o$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c $< +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ $< .c.obj: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(COMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ `$(CYGPATH_W) '$<'` -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Po +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.obj$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ `$(CYGPATH_W) '$<'` &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c `$(CYGPATH_W) '$<'` +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ `$(CYGPATH_W) '$<'` .c.lo: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(LTCOMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ $< -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Plo +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.lo$$||'`;\ +@am__fastdepCC_TRUE@ $(LTCOMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Plo @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=yes @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ @am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(LTCOMPILE) -c -o $@ $< diff --git a/plugins/omhiredis/omhiredis.c b/plugins/omhiredis/omhiredis.c index 051ac0b..28a1b9c 100644 --- a/plugins/omhiredis/omhiredis.c +++ b/plugins/omhiredis/omhiredis.c @@ -52,14 +52,17 @@ DEFobjCurrIf(errmsg) * this will be accessable * via pData */ typedef struct _instanceData { - redisContext *conn; /* redis connection */ uchar *server; /* redis server address */ int port; /* redis port */ uchar *tplName; /* template name */ - redisReply **replies; /* array to hold replies from redis */ - int count; /* count of command sent for current batch */ } instanceData; +typedef struct wrkrInstanceData { + instanceData *pData; + redisContext *conn; /* redis connection */ + redisReply **replies; /* array to hold replies from redis */ + int count; /* count of command sent for current batch */ +} wrkrInstanceData_t; static struct cnfparamdescr actpdescr[] = { { "server", eCmdHdlrGetWord, 0 }, @@ -76,6 +79,11 @@ BEGINcreateInstance CODESTARTcreateInstance ENDcreateInstance +BEGINcreateWrkrInstance +CODESTARTcreateWrkrInstance + pWrkrData->conn = NULL; /* Connect later */ +ENDcreateWrkrInstance + BEGINisCompatibleWithFeature CODESTARTisCompatibleWithFeature if(eFeat == sFEATURERepeatedMsgReduction) @@ -83,11 +91,11 @@ CODESTARTisCompatibleWithFeature ENDisCompatibleWithFeature /* called when closing */ -static void closeHiredis(instanceData *pData) +static void closeHiredis(wrkrInstanceData_t *pWrkrData) { - if(pData->conn != NULL) { - redisFree(pData->conn); - pData->conn = NULL; + if(pWrkrData->conn != NULL) { + redisFree(pWrkrData->conn); + pWrkrData->conn = NULL; } } @@ -95,11 +103,15 @@ static void closeHiredis(instanceData *pData) * TODO: free **replies */ BEGINfreeInstance CODESTARTfreeInstance - closeHiredis(pData); - free(pData->server); - free(pData->tplName); + if (pData->server != NULL) { + free(pData->server); + } ENDfreeInstance +BEGINfreeWrkrInstance +CODESTARTfreeWrkrInstance + closeHiredis(pWrkrData); +ENDfreeWrkrInstance BEGINdbgPrintInstInfo CODESTARTdbgPrintInstInfo @@ -107,17 +119,20 @@ CODESTARTdbgPrintInstInfo ENDdbgPrintInstInfo /* establish our connection to redis */ -static rsRetVal initHiredis(instanceData *pData, int bSilent) +static rsRetVal initHiredis(wrkrInstanceData_t *pWrkrData, int bSilent) { char *server; DEFiRet; - server = (pData->server == NULL) ? "127.0.0.1" : (char*) pData->server; - DBGPRINTF("omhiredis: trying connect to '%s' at port %d\n", server, pData->port); - + server = (pWrkrData->pData->server == NULL) ? "127.0.0.1" : + (char*) pWrkrData->pData->server; + DBGPRINTF("omhiredis: trying connect to '%s' at port %d\n", server, + pWrkrData->pData->port); + struct timeval timeout = { 1, 500000 }; /* 1.5 seconds */ - pData->conn = redisConnectWithTimeout(server, pData->port, timeout); - if (pData->conn->err) { + pWrkrData->conn = redisConnectWithTimeout(server, pWrkrData->pData->port, + timeout); + if (pWrkrData->conn->err) { if(!bSilent) errmsg.LogError(0, RS_RET_SUSPENDED, "can not initialize redis handle"); @@ -127,29 +142,29 @@ finalize_it: RETiRet; } -rsRetVal writeHiredis(uchar *message, instanceData *pData) +rsRetVal writeHiredis(uchar *message, wrkrInstanceData_t *pWrkrData) { DEFiRet; - /* if we do not have a redis connection, call - * initHiredis and try to establish one */ - if(pData->conn == NULL) - CHKiRet(initHiredis(pData, 0)); + /* if we do not have a redis connection, call + * initHiredis and try to establish one */ + if(pWrkrData->conn == NULL) + CHKiRet(initHiredis(pWrkrData, 0)); - /* try to append the command to the pipeline. - * REDIS_ERR reply indicates something bad - * happened, in which case abort. otherwise - * increase our current pipeline count - * by 1 and continue. */ + /* try to append the command to the pipeline. + * REDIS_ERR reply indicates something bad + * happened, in which case abort. otherwise + * increase our current pipeline count + * by 1 and continue. */ int rc; - rc = redisAppendCommand(pData->conn, (char*)message); + rc = redisAppendCommand(pWrkrData->conn, (char*)message); if (rc == REDIS_ERR) { - errmsg.LogError(0, NO_ERRCODE, "omhiredis: %s", pData->conn->errstr); - dbgprintf("omhiredis: %s\n", pData->conn->errstr); + errmsg.LogError(0, NO_ERRCODE, "omhiredis: %s", pWrkrData->conn->errstr); + dbgprintf("omhiredis: %s\n", pWrkrData->conn->errstr); ABORT_FINALIZE(RS_RET_ERR); } else { - pData->count++; - } + pWrkrData->count++; + } finalize_it: RETiRet; @@ -159,17 +174,18 @@ finalize_it: * try to restablish our connection to redis */ BEGINtryResume CODESTARTtryResume - if(pData->conn == NULL) - iRet = initHiredis(pData, 0); + if(pWrkrData->conn == NULL) + iRet = initHiredis(pWrkrData, 0); ENDtryResume -/* begin a transaction. for now does nothing. +/* begin a transaction. * if I decide to use MULTI ... EXEC in the - * fture, this block should send the + * future, this block should send the * MULTI command to redis. */ BEGINbeginTransaction CODESTARTbeginTransaction - dbgprintf("omhiredis: beginTransaction called\n"); + dbgprintf("omhiredis: beginTransaction called\n"); + pWrkrData->count = 0; ENDbeginTransaction /* call writeHiredis for this log line, @@ -177,8 +193,8 @@ ENDbeginTransaction * current pipeline */ BEGINdoAction CODESTARTdoAction - CHKiRet(writeHiredis(ppString[0], pData)); - iRet = RS_RET_DEFER_COMMIT; + CHKiRet(writeHiredis(ppString[0], pWrkrData)); + iRet = RS_RET_DEFER_COMMIT; finalize_it: ENDdoAction @@ -190,16 +206,15 @@ ENDdoAction * which should be fixed */ BEGINendTransaction CODESTARTendTransaction - dbgprintf("omhiredis: endTransaction called\n"); - int i; - pData->replies = malloc ( sizeof ( redisReply* ) * pData->count ); - for ( i = 0; i < pData->count; i++ ) { - redisGetReply ( pData->conn, (void *)&pData->replies[i] ); - /* TODO: add error checking here! */ - free ( pData->replies[i] ); - } - free ( pData->replies ); - pData->count = 0; + dbgprintf("omhiredis: endTransaction called\n"); + int i; + pWrkrData->replies = malloc ( sizeof ( redisReply* ) * pWrkrData->count ); + for ( i = 0; i < pWrkrData->count; i++ ) { + redisGetReply ( pWrkrData->conn, (void *)&pWrkrData->replies[i] ); + /* TODO: add error checking here! */ + freeReplyObject ( pWrkrData->replies[i] ); + } + free ( pWrkrData->replies ); ENDendTransaction /* set defaults. note server is set to NULL @@ -212,7 +227,6 @@ setInstParamDefaults(instanceData *pData) pData->server = NULL; pData->port = 6379; pData->tplName = NULL; - pData->count = 0; } /* here is where the work to set up a new instance @@ -282,6 +296,7 @@ ENDmodExit BEGINqueryEtryPt CODESTARTqueryEtryPt CODEqueryEtryPt_STD_OMOD_QUERIES +CODEqueryEtryPt_STD_OMOD8_QUERIES CODEqueryEtryPt_STD_CONF2_OMOD_QUERIES CODEqueryEtryPt_TXIF_OMOD_QUERIES /* supports transaction interface */ ENDqueryEtryPt @@ -293,9 +308,9 @@ CODESTARTmodInit CODEmodInit_QueryRegCFSLineHdlr CHKiRet(objUse(errmsg, CORE_COMPONENT)); INITChkCoreFeature(bCoreSupportsBatching, CORE_FEATURE_BATCHING); - if (!bCoreSupportsBatching) { - errmsg.LogError(0, NO_ERRCODE, "omhiredis: rsyslog core does not support batching - abort"); - ABORT_FINALIZE(RS_RET_ERR); - } + if (!bCoreSupportsBatching) { + errmsg.LogError(0, NO_ERRCODE, "omhiredis: rsyslog core does not support batching - abort"); + ABORT_FINALIZE(RS_RET_ERR); + } DBGPRINTF("omhiredis: module compiled with rsyslog version %s.\n", VERSION); ENDmodInit diff --git a/plugins/omjournal/Makefile.in b/plugins/omjournal/Makefile.in index 9a9f494..6a503a6 100644 --- a/plugins/omjournal/Makefile.in +++ b/plugins/omjournal/Makefile.in @@ -154,7 +154,6 @@ GREP = @GREP@ GSS_LIBS = @GSS_LIBS@ GUARDTIME_CFLAGS = @GUARDTIME_CFLAGS@ GUARDTIME_LIBS = @GUARDTIME_LIBS@ -HAVE_LIBGCRYPT_CONFIG = @HAVE_LIBGCRYPT_CONFIG@ HAVE_MYSQL_CONFIG = @HAVE_MYSQL_CONFIG@ HAVE_ORACLE_CONFIG = @HAVE_ORACLE_CONFIG@ HAVE_PGSQL_CONFIG = @HAVE_PGSQL_CONFIG@ @@ -175,14 +174,15 @@ LEXLIB = @LEXLIB@ LEX_OUTPUT_ROOT = @LEX_OUTPUT_ROOT@ LIBDBI_CFLAGS = @LIBDBI_CFLAGS@ LIBDBI_LIBS = @LIBDBI_LIBS@ -LIBEE_CFLAGS = @LIBEE_CFLAGS@ -LIBEE_LIBS = @LIBEE_LIBS@ LIBESTR_CFLAGS = @LIBESTR_CFLAGS@ LIBESTR_LIBS = @LIBESTR_LIBS@ LIBGCRYPT_CFLAGS = @LIBGCRYPT_CFLAGS@ +LIBGCRYPT_CONFIG = @LIBGCRYPT_CONFIG@ LIBGCRYPT_LIBS = @LIBGCRYPT_LIBS@ LIBLOGGING_CFLAGS = @LIBLOGGING_CFLAGS@ LIBLOGGING_LIBS = @LIBLOGGING_LIBS@ +LIBLOGGING_STDLOG_CFLAGS = @LIBLOGGING_STDLOG_CFLAGS@ +LIBLOGGING_STDLOG_LIBS = @LIBLOGGING_STDLOG_LIBS@ LIBLOGNORM_CFLAGS = @LIBLOGNORM_CFLAGS@ LIBLOGNORM_LIBS = @LIBLOGNORM_LIBS@ LIBM = @LIBM@ @@ -207,6 +207,8 @@ NM = @NM@ NMEDIT = @NMEDIT@ OBJDUMP = @OBJDUMP@ OBJEXT = @OBJEXT@ +OPENSSL_CFLAGS = @OPENSSL_CFLAGS@ +OPENSSL_LIBS = @OPENSSL_LIBS@ ORACLE_CFLAGS = @ORACLE_CFLAGS@ ORACLE_LIBS = @ORACLE_LIBS@ OTOOL = @OTOOL@ @@ -388,22 +390,25 @@ distclean-compile: @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/omjournal_la-omjournal.Plo@am__quote@ .c.o: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(COMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ $< -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Po +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.o$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c $< +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ $< .c.obj: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(COMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ `$(CYGPATH_W) '$<'` -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Po +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.obj$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ `$(CYGPATH_W) '$<'` &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c `$(CYGPATH_W) '$<'` +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ `$(CYGPATH_W) '$<'` .c.lo: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(LTCOMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ $< -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Plo +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.lo$$||'`;\ +@am__fastdepCC_TRUE@ $(LTCOMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Plo @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=yes @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ @am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(LTCOMPILE) -c -o $@ $< diff --git a/plugins/omjournal/omjournal.c b/plugins/omjournal/omjournal.c index c340287..5ca7982 100644 --- a/plugins/omjournal/omjournal.c +++ b/plugins/omjournal/omjournal.c @@ -56,6 +56,10 @@ DEF_OMOD_STATIC_DATA typedef struct _instanceData { } instanceData; +typedef struct wrkrInstanceData { + instanceData *pData; +} wrkrInstanceData_t; + struct modConfData_s { rsconf_t *pConf; /* our overall config object */ }; @@ -91,6 +95,11 @@ CODESTARTcreateInstance ENDcreateInstance +BEGINcreateWrkrInstance +CODESTARTcreateWrkrInstance +ENDcreateWrkrInstance + + BEGINisCompatibleWithFeature CODESTARTisCompatibleWithFeature ENDisCompatibleWithFeature @@ -101,12 +110,18 @@ CODESTARTfreeInstance ENDfreeInstance +BEGINfreeWrkrInstance +CODESTARTfreeWrkrInstance +ENDfreeWrkrInstance + + BEGINnewActInst CODESTARTnewActInst /* Note: we currently do not have any parameters, so we do not need * the lst ptr. However, we will most probably need params in the * future. */ + (void) lst; /* prevent compiler warning */ DBGPRINTF("newActInst (mmjournal)\n"); CODE_STD_STRING_REQUESTnewActInst(1) CHKiRet(OMSRsetEntry(*ppOMSR, 0, NULL, OMSR_TPL_AS_MSG)); @@ -145,13 +160,14 @@ CODESTARTdoAction "SYSLOG_IDENTIFIER=%s", tag, NULL); /* FIXME: think about what to do with errors ;) */ + (void) r; /* prevent compiler warning */ ENDdoAction BEGINparseSelectorAct CODESTARTparseSelectorAct CODE_STD_STRING_REQUESTparseSelectorAct(1) - if(strncmp((char*) p, ":omjournal:", sizeof(":omjournal:") - 1)) { + if(!strncmp((char*) p, ":omjournal:", sizeof(":omjournal:") - 1)) { errmsg.LogError(0, RS_RET_LEGA_ACT_NOT_SUPPORTED, "omjournal supports only v6+ config format, use: " "action(type=\"omjournal\" ...)"); @@ -170,6 +186,7 @@ ENDmodExit BEGINqueryEtryPt CODESTARTqueryEtryPt CODEqueryEtryPt_STD_OMOD_QUERIES +CODEqueryEtryPt_STD_OMOD8_QUERIES CODEqueryEtryPt_STD_CONF2_OMOD_QUERIES CODEqueryEtryPt_STD_CONF2_QUERIES ENDqueryEtryPt diff --git a/plugins/omlibdbi/Makefile.in b/plugins/omlibdbi/Makefile.in index 5408df9..8217570 100644 --- a/plugins/omlibdbi/Makefile.in +++ b/plugins/omlibdbi/Makefile.in @@ -154,7 +154,6 @@ GREP = @GREP@ GSS_LIBS = @GSS_LIBS@ GUARDTIME_CFLAGS = @GUARDTIME_CFLAGS@ GUARDTIME_LIBS = @GUARDTIME_LIBS@ -HAVE_LIBGCRYPT_CONFIG = @HAVE_LIBGCRYPT_CONFIG@ HAVE_MYSQL_CONFIG = @HAVE_MYSQL_CONFIG@ HAVE_ORACLE_CONFIG = @HAVE_ORACLE_CONFIG@ HAVE_PGSQL_CONFIG = @HAVE_PGSQL_CONFIG@ @@ -175,14 +174,15 @@ LEXLIB = @LEXLIB@ LEX_OUTPUT_ROOT = @LEX_OUTPUT_ROOT@ LIBDBI_CFLAGS = @LIBDBI_CFLAGS@ LIBDBI_LIBS = @LIBDBI_LIBS@ -LIBEE_CFLAGS = @LIBEE_CFLAGS@ -LIBEE_LIBS = @LIBEE_LIBS@ LIBESTR_CFLAGS = @LIBESTR_CFLAGS@ LIBESTR_LIBS = @LIBESTR_LIBS@ LIBGCRYPT_CFLAGS = @LIBGCRYPT_CFLAGS@ +LIBGCRYPT_CONFIG = @LIBGCRYPT_CONFIG@ LIBGCRYPT_LIBS = @LIBGCRYPT_LIBS@ LIBLOGGING_CFLAGS = @LIBLOGGING_CFLAGS@ LIBLOGGING_LIBS = @LIBLOGGING_LIBS@ +LIBLOGGING_STDLOG_CFLAGS = @LIBLOGGING_STDLOG_CFLAGS@ +LIBLOGGING_STDLOG_LIBS = @LIBLOGGING_STDLOG_LIBS@ LIBLOGNORM_CFLAGS = @LIBLOGNORM_CFLAGS@ LIBLOGNORM_LIBS = @LIBLOGNORM_LIBS@ LIBM = @LIBM@ @@ -207,6 +207,8 @@ NM = @NM@ NMEDIT = @NMEDIT@ OBJDUMP = @OBJDUMP@ OBJEXT = @OBJEXT@ +OPENSSL_CFLAGS = @OPENSSL_CFLAGS@ +OPENSSL_LIBS = @OPENSSL_LIBS@ ORACLE_CFLAGS = @ORACLE_CFLAGS@ ORACLE_LIBS = @ORACLE_LIBS@ OTOOL = @OTOOL@ @@ -387,22 +389,25 @@ distclean-compile: @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/omlibdbi_la-omlibdbi.Plo@am__quote@ .c.o: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(COMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ $< -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Po +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.o$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c $< +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ $< .c.obj: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(COMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ `$(CYGPATH_W) '$<'` -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Po +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.obj$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ `$(CYGPATH_W) '$<'` &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c `$(CYGPATH_W) '$<'` +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ `$(CYGPATH_W) '$<'` .c.lo: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(LTCOMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ $< -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Plo +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.lo$$||'`;\ +@am__fastdepCC_TRUE@ $(LTCOMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Plo @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=yes @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ @am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(LTCOMPILE) -c -o $@ $< diff --git a/plugins/omlibdbi/omlibdbi.c b/plugins/omlibdbi/omlibdbi.c index 6e27ad2..c203b4a 100644 --- a/plugins/omlibdbi/omlibdbi.c +++ b/plugins/omlibdbi/omlibdbi.c @@ -10,7 +10,7 @@ * * File begun on 2008-02-14 by RGerhards (extracted from syslogd.c) * - * Copyright 2008-2012 Adiscon GmbH. + * Copyright 2008-2013 Adiscon GmbH. * * This file is part of rsyslog. * @@ -50,6 +50,9 @@ #include "errmsg.h" #include "conf.h" +#undef HAVE_DBI_TXSUPP +#warning transaction support disabled in v8 -- TODO: reenable + MODULE_TYPE_OUTPUT MODULE_TYPE_NOKEEP MODULE_CNFNAME("omlibdbi") @@ -73,6 +76,10 @@ typedef struct _instanceData { int txSupport; /* transaction support */ } instanceData; +typedef struct wrkrInstanceData { + instanceData *pData; +} wrkrInstanceData_t; + typedef struct configSettings_s { uchar *dbiDrvrDir; /* global: where do the dbi drivers reside? */ uchar *drvrName; /* driver to use */ @@ -94,6 +101,8 @@ static modConfData_t *loadModConf = NULL;/* modConf ptr to use for the current l static modConfData_t *runModConf = NULL;/* modConf ptr to use for the current exec process */ static int bLegacyCnfModGlobalsPermitted;/* are legacy module-global config parameters permitted? */ +static pthread_mutex_t mutDoAct = PTHREAD_MUTEX_INITIALIZER; + /* tables for interfacing with the v6 config system */ /* module-global parameters */ @@ -157,6 +166,10 @@ BEGINcreateInstance CODESTARTcreateInstance ENDcreateInstance +BEGINcreateWrkrInstance +CODESTARTcreateWrkrInstance +ENDcreateWrkrInstance + BEGINisCompatibleWithFeature CODESTARTisCompatibleWithFeature @@ -187,6 +200,9 @@ CODESTARTfreeInstance free(pData->dbName); ENDfreeInstance +BEGINfreeWrkrInstance +CODESTARTfreeWrkrInstance +ENDfreeWrkrInstance BEGINdbgPrintInstInfo CODESTARTdbgPrintInstInfo @@ -326,22 +342,26 @@ finalize_it: BEGINtryResume CODESTARTtryResume - if(pData->conn == NULL) { - iRet = initConn(pData, 1); + if(pWrkrData->pData->conn == NULL) { + iRet = initConn(pWrkrData->pData, 1); } ENDtryResume /* transaction support 2013-03 */ BEGINbeginTransaction CODESTARTbeginTransaction - if(pData->conn == NULL) { - CHKiRet(initConn(pData, 0)); + if(pWrkrData->pData->conn == NULL) { + CHKiRet(initConn(pWrkrData->pData, 0)); } # if HAVE_DBI_TXSUPP if (pData->txSupport == 1) { if (dbi_conn_transaction_begin(pData->conn) != 0) { - dbgprintf("libdbi server error: begin transaction not successful\n"); - iRet = RS_RET_SUSPENDED; + const char *emsg; + dbi_conn_error(pData->conn, &emsg); + dbgprintf("libdbi server error: begin transaction " + "not successful: %s\n", emsg); + closeConn(pData); + ABORT_FINALIZE(RS_RET_SUSPENDED); } } # endif @@ -351,13 +371,15 @@ ENDbeginTransaction BEGINdoAction CODESTARTdoAction - CHKiRet(writeDB(ppString[0], pData)); + pthread_mutex_lock(&mutDoAct); + CHKiRet(writeDB(ppString[0], pWrkrData->pData)); # if HAVE_DBI_TXSUPP if (pData->txSupport == 1) { iRet = RS_RET_DEFER_COMMIT; } # endif finalize_it: + pthread_mutex_unlock(&mutDoAct); ENDdoAction /* transaction support 2013-03 */ @@ -365,7 +387,11 @@ BEGINendTransaction CODESTARTendTransaction # if HAVE_DBI_TXSUPP if (dbi_conn_transaction_commit(pData->conn) != 0) { - dbgprintf("libdbi server error: transaction not committed\n"); + const char *emsg; + dbi_conn_error(pData->conn, &emsg); + dbgprintf("libdbi server error: transaction not committed: %s\n", + emsg); + closeConn(pData); iRet = RS_RET_SUSPENDED; } # endif @@ -544,6 +570,7 @@ ENDmodExit BEGINqueryEtryPt CODESTARTqueryEtryPt CODEqueryEtryPt_STD_OMOD_QUERIES +CODEqueryEtryPt_STD_OMOD8_QUERIES CODEqueryEtryPt_STD_CONF2_QUERIES CODEqueryEtryPt_STD_CONF2_setModCnf_QUERIES CODEqueryEtryPt_STD_CONF2_OMOD_QUERIES diff --git a/plugins/ommail/Makefile.in b/plugins/ommail/Makefile.in index ae7b411..a5bf269 100644 --- a/plugins/ommail/Makefile.in +++ b/plugins/ommail/Makefile.in @@ -153,7 +153,6 @@ GREP = @GREP@ GSS_LIBS = @GSS_LIBS@ GUARDTIME_CFLAGS = @GUARDTIME_CFLAGS@ GUARDTIME_LIBS = @GUARDTIME_LIBS@ -HAVE_LIBGCRYPT_CONFIG = @HAVE_LIBGCRYPT_CONFIG@ HAVE_MYSQL_CONFIG = @HAVE_MYSQL_CONFIG@ HAVE_ORACLE_CONFIG = @HAVE_ORACLE_CONFIG@ HAVE_PGSQL_CONFIG = @HAVE_PGSQL_CONFIG@ @@ -174,14 +173,15 @@ LEXLIB = @LEXLIB@ LEX_OUTPUT_ROOT = @LEX_OUTPUT_ROOT@ LIBDBI_CFLAGS = @LIBDBI_CFLAGS@ LIBDBI_LIBS = @LIBDBI_LIBS@ -LIBEE_CFLAGS = @LIBEE_CFLAGS@ -LIBEE_LIBS = @LIBEE_LIBS@ LIBESTR_CFLAGS = @LIBESTR_CFLAGS@ LIBESTR_LIBS = @LIBESTR_LIBS@ LIBGCRYPT_CFLAGS = @LIBGCRYPT_CFLAGS@ +LIBGCRYPT_CONFIG = @LIBGCRYPT_CONFIG@ LIBGCRYPT_LIBS = @LIBGCRYPT_LIBS@ LIBLOGGING_CFLAGS = @LIBLOGGING_CFLAGS@ LIBLOGGING_LIBS = @LIBLOGGING_LIBS@ +LIBLOGGING_STDLOG_CFLAGS = @LIBLOGGING_STDLOG_CFLAGS@ +LIBLOGGING_STDLOG_LIBS = @LIBLOGGING_STDLOG_LIBS@ LIBLOGNORM_CFLAGS = @LIBLOGNORM_CFLAGS@ LIBLOGNORM_LIBS = @LIBLOGNORM_LIBS@ LIBM = @LIBM@ @@ -206,6 +206,8 @@ NM = @NM@ NMEDIT = @NMEDIT@ OBJDUMP = @OBJDUMP@ OBJEXT = @OBJEXT@ +OPENSSL_CFLAGS = @OPENSSL_CFLAGS@ +OPENSSL_LIBS = @OPENSSL_LIBS@ ORACLE_CFLAGS = @ORACLE_CFLAGS@ ORACLE_LIBS = @ORACLE_LIBS@ OTOOL = @OTOOL@ @@ -386,22 +388,25 @@ distclean-compile: @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/ommail_la-ommail.Plo@am__quote@ .c.o: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(COMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ $< -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Po +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.o$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c $< +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ $< .c.obj: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(COMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ `$(CYGPATH_W) '$<'` -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Po +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.obj$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ `$(CYGPATH_W) '$<'` &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c `$(CYGPATH_W) '$<'` +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ `$(CYGPATH_W) '$<'` .c.lo: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(LTCOMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ $< -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Plo +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.lo$$||'`;\ +@am__fastdepCC_TRUE@ $(LTCOMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Plo @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=yes @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ @am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(LTCOMPILE) -c -o $@ $< diff --git a/plugins/ommail/ommail.c b/plugins/ommail/ommail.c index 6044d2e..113385f 100644 --- a/plugins/ommail/ommail.c +++ b/plugins/ommail/ommail.c @@ -13,7 +13,7 @@ * * File begun on 2008-04-04 by RGerhards * - * Copyright 2008-2012 Adiscon GmbH. + * Copyright 2008-2013 Adiscon GmbH. * * This file is part of rsyslog. * @@ -82,13 +82,21 @@ typedef struct _instanceData { uchar *pszSrvPort; uchar *pszFrom; toRcpt_t *lstRcpt; + } smtp; + } md; /* mode-specific data */ +} instanceData; + +typedef struct wrkrInstanceData { + instanceData *pData; + union { + struct { char RcvBuf[1024]; /* buffer for receiving server responses */ size_t lenRcvBuf; size_t iRcvBuf; /* current index into the rcvBuf (buf empty if iRcvBuf == lenRcvBuf) */ int sock; /* socket to this server (most important when we do multiple msgs per mail) */ } smtp; } md; /* mode-specific data */ -} instanceData; +} wrkrInstanceData_t; typedef struct configSettings_s { toRcpt_t *lstRcpt; @@ -112,7 +120,7 @@ ENDinitConfVars /* forward definitions (as few as possible) */ static rsRetVal Send(int sock, char *msg, size_t len); -static rsRetVal readResponse(instanceData *pData, int *piState, int iExpected); +static rsRetVal readResponse(wrkrInstanceData_t *pWrkrData, int *piState, int iExpected); /* helpers for handling the recipient lists */ @@ -150,8 +158,6 @@ addRcpt(void __attribute__((unused)) *pVal, uchar *pNewVal) finalize_it: if(iRet != RS_RET_OK) { - if(pNew != NULL) - free(pNew); free(pNewVal); /* in any case, this is no longer needed */ } @@ -163,24 +169,22 @@ finalize_it: * iStatusToCheck < 0 means no checking should happen */ static rsRetVal -WriteRcpts(instanceData *pData, uchar *pszOp, size_t lenOp, int iStatusToCheck) +WriteRcpts(wrkrInstanceData_t *pWrkrData, uchar *pszOp, size_t lenOp, int iStatusToCheck) { toRcpt_t *pRcpt; int iState; DEFiRet; - assert(pData != NULL); - assert(pszOp != NULL); assert(lenOp != 0); - for(pRcpt = pData->md.smtp.lstRcpt ; pRcpt != NULL ; pRcpt = pRcpt->pNext) { + for(pRcpt = pWrkrData->pData->md.smtp.lstRcpt ; pRcpt != NULL ; pRcpt = pRcpt->pNext) { dbgprintf("Sending '%s: <%s>'\n", pszOp, pRcpt->pszTo); - CHKiRet(Send(pData->md.smtp.sock, (char*)pszOp, lenOp)); - CHKiRet(Send(pData->md.smtp.sock, ": <", sizeof(": <") - 1)); - CHKiRet(Send(pData->md.smtp.sock, (char*)pRcpt->pszTo, strlen((char*)pRcpt->pszTo))); - CHKiRet(Send(pData->md.smtp.sock, ">\r\n", sizeof(">\r\n") - 1)); + CHKiRet(Send(pWrkrData->md.smtp.sock, (char*)pszOp, lenOp)); + CHKiRet(Send(pWrkrData->md.smtp.sock, ":<", sizeof(":<") - 1)); + CHKiRet(Send(pWrkrData->md.smtp.sock, (char*)pRcpt->pszTo, strlen((char*)pRcpt->pszTo))); + CHKiRet(Send(pWrkrData->md.smtp.sock, ">\r\n", sizeof(">\r\n") - 1)); if(iStatusToCheck >= 0) - CHKiRet(readResponse(pData, &iState, iStatusToCheck)); + CHKiRet(readResponse(pWrkrData, &iState, iStatusToCheck)); } finalize_it: @@ -193,6 +197,11 @@ CODESTARTcreateInstance ENDcreateInstance +BEGINcreateWrkrInstance +CODESTARTcreateWrkrInstance +ENDcreateWrkrInstance + + BEGINisCompatibleWithFeature CODESTARTisCompatibleWithFeature if(eFeat == sFEATURERepeatedMsgReduction) @@ -203,17 +212,19 @@ ENDisCompatibleWithFeature BEGINfreeInstance CODESTARTfreeInstance if(pData->iMode == 0) { - if(pData->md.smtp.pszSrv != NULL) - free(pData->md.smtp.pszSrv); - if(pData->md.smtp.pszSrvPort != NULL) - free(pData->md.smtp.pszSrvPort); - if(pData->md.smtp.pszFrom != NULL) - free(pData->md.smtp.pszFrom); + free(pData->md.smtp.pszSrv); + free(pData->md.smtp.pszSrvPort); + free(pData->md.smtp.pszFrom); lstRcptDestruct(pData->md.smtp.lstRcpt); } ENDfreeInstance +BEGINfreeWrkrInstance +CODESTARTfreeWrkrInstance +ENDfreeWrkrInstance + + BEGINdbgPrintInstInfo CODESTARTdbgPrintInstInfo printf("mail"); /* TODO: extend! */ @@ -229,16 +240,16 @@ ENDdbgPrintInstInfo * rgerhards, 2008-04-04 */ static rsRetVal -getRcvChar(instanceData *pData, char *pC) +getRcvChar(wrkrInstanceData_t *pWrkrData, char *pC) { DEFiRet; ssize_t lenBuf; - assert(pData != NULL); - if(pData->md.smtp.iRcvBuf == pData->md.smtp.lenRcvBuf) { /* buffer empty? */ + if(pWrkrData->md.smtp.iRcvBuf == pWrkrData->md.smtp.lenRcvBuf) { /* buffer empty? */ /* yes, we need to read the next server response */ do { - lenBuf = recv(pData->md.smtp.sock, pData->md.smtp.RcvBuf, sizeof(pData->md.smtp.RcvBuf), 0); + lenBuf = recv(pWrkrData->md.smtp.sock, pWrkrData->md.smtp.RcvBuf, + sizeof(pWrkrData->md.smtp.RcvBuf), 0); if(lenBuf == 0) { ABORT_FINALIZE(RS_RET_NO_MORE_DATA); } else if(lenBuf < 0) { @@ -247,15 +258,15 @@ getRcvChar(instanceData *pData, char *pC) } } else { /* good read */ - pData->md.smtp.iRcvBuf = 0; - pData->md.smtp.lenRcvBuf = lenBuf; + pWrkrData->md.smtp.iRcvBuf = 0; + pWrkrData->md.smtp.lenRcvBuf = lenBuf; } } while(lenBuf < 1); } /* when we reach this point, we have a non-empty buffer */ - *pC = pData->md.smtp.RcvBuf[pData->md.smtp.iRcvBuf++]; + *pC = pWrkrData->md.smtp.RcvBuf[pWrkrData->md.smtp.iRcvBuf++]; finalize_it: RETiRet; @@ -266,14 +277,14 @@ finalize_it: * rgerhards, 2008-04-08 */ static rsRetVal -serverDisconnect(instanceData *pData) +serverDisconnect(wrkrInstanceData_t *pWrkrData) { DEFiRet; - assert(pData != NULL); + assert(pWrkrData != NULL); - if(pData->md.smtp.sock != -1) { - close(pData->md.smtp.sock); - pData->md.smtp.sock = -1; + if(pWrkrData->md.smtp.sock != -1) { + close(pWrkrData->md.smtp.sock); + pWrkrData->md.smtp.sock = -1; } RETiRet; @@ -284,16 +295,17 @@ serverDisconnect(instanceData *pData) * rgerhards, 2008-04-04 */ static rsRetVal -serverConnect(instanceData *pData) +serverConnect(wrkrInstanceData_t *pWrkrData) { struct addrinfo *res = NULL; struct addrinfo hints; char *smtpPort; char *smtpSrv; char errStr[1024]; - + instanceData *pData; DEFiRet; - assert(pData != NULL); + + pData = pWrkrData->pData; if(pData->md.smtp.pszSrv == NULL) smtpSrv = "127.0.0.1"; @@ -313,12 +325,12 @@ serverConnect(instanceData *pData) ABORT_FINALIZE(RS_RET_IO_ERROR); } - if((pData->md.smtp.sock = socket(res->ai_family, res->ai_socktype, res->ai_protocol)) == -1) { + if((pWrkrData->md.smtp.sock = socket(res->ai_family, res->ai_socktype, res->ai_protocol)) == -1) { dbgprintf("couldn't create send socket, reason %s", rs_strerror_r(errno, errStr, sizeof(errStr))); ABORT_FINALIZE(RS_RET_IO_ERROR); } - if(connect(pData->md.smtp.sock, res->ai_addr, res->ai_addrlen) != 0) { + if(connect(pWrkrData->md.smtp.sock, res->ai_addr, res->ai_addrlen) != 0) { dbgprintf("create tcp connection failed, reason %s", rs_strerror_r(errno, errStr, sizeof(errStr))); ABORT_FINALIZE(RS_RET_IO_ERROR); } @@ -328,9 +340,9 @@ finalize_it: freeaddrinfo(res); if(iRet != RS_RET_OK) { - if(pData->md.smtp.sock != -1) { - close(pData->md.smtp.sock); - pData->md.smtp.sock = -1; + if(pWrkrData->md.smtp.sock != -1) { + close(pWrkrData->md.smtp.sock); + pWrkrData->md.smtp.sock = -1; } } @@ -374,7 +386,7 @@ finalize_it: * The body is special in that we must escape a leading dot inside a line */ static rsRetVal -bodySend(instanceData *pData, char *msg, size_t len) +bodySend(wrkrInstanceData_t *pWrkrData, char *msg, size_t len) { DEFiRet; char szBuf[2048]; @@ -383,12 +395,12 @@ bodySend(instanceData *pData, char *msg, size_t len) int bHadCR = 0; int bInStartOfLine = 1; - assert(pData != NULL); + assert(pWrkrData != NULL); assert(msg != NULL); for(iSrc = 0 ; iSrc < len ; ++iSrc) { if(iBuf >= sizeof(szBuf) - 1) { /* one is reserved for our extra dot */ - CHKiRet(Send(pData->md.smtp.sock, szBuf, iBuf)); + CHKiRet(Send(pWrkrData->md.smtp.sock, szBuf, iBuf)); iBuf = 0; } szBuf[iBuf++] = msg[iSrc]; @@ -413,7 +425,7 @@ bodySend(instanceData *pData, char *msg, size_t len) } if(iBuf > 0) { /* incomplete buffer to send (the *usual* case)? */ - CHKiRet(Send(pData->md.smtp.sock, szBuf, iBuf)); + CHKiRet(Send(pWrkrData->md.smtp.sock, szBuf, iBuf)); } finalize_it: @@ -424,17 +436,17 @@ finalize_it: /* read response line from server */ static rsRetVal -readResponseLn(instanceData *pData, char *pLn, size_t lenLn) +readResponseLn(wrkrInstanceData_t *pWrkrData, char *pLn, size_t lenLn) { DEFiRet; size_t i = 0; char c; - assert(pData != NULL); + assert(pWrkrData != NULL); assert(pLn != NULL); do { - CHKiRet(getRcvChar(pData, &c)); + CHKiRet(getRcvChar(pWrkrData, &c)); if(c == '\n') break; if(i < (lenLn - 1)) /* if line is too long, we simply discard the rest */ @@ -453,18 +465,18 @@ finalize_it: * rgerhards, 2008-04-07 */ static rsRetVal -readResponse(instanceData *pData, int *piState, int iExpected) +readResponse(wrkrInstanceData_t *pWrkrData, int *piState, int iExpected) { DEFiRet; int bCont; char buf[128]; - assert(pData != NULL); + assert(pWrkrData != NULL); assert(piState != NULL); bCont = 1; do { - CHKiRet(readResponseLn(pData, buf, sizeof(buf))); + CHKiRet(readResponseLn(pWrkrData, buf, sizeof(buf))); /* note: the code below is not 100% clean as we may have received less than 4 characters. * However, as we have a fixed size this will not create a vulnerability. An error will * also most likely be generated, so it is quite acceptable IMHO -- rgerhards, 2008-04-08 @@ -506,64 +518,65 @@ mkSMTPTimestamp(uchar *pszBuf, size_t lenBuf) * rgerhards, 2008-04-04 */ static rsRetVal -sendSMTP(instanceData *pData, uchar *body, uchar *subject) +sendSMTP(wrkrInstanceData_t *pWrkrData, uchar *body, uchar *subject) { DEFiRet; int iState; /* SMTP state */ + instanceData *pData; uchar szDateBuf[64]; - assert(pData != NULL); + pData = pWrkrData->pData; - CHKiRet(serverConnect(pData)); - CHKiRet(readResponse(pData, &iState, 220)); + CHKiRet(serverConnect(pWrkrData)); + CHKiRet(readResponse(pWrkrData, &iState, 220)); - CHKiRet(Send(pData->md.smtp.sock, "HELO ", 5)); - CHKiRet(Send(pData->md.smtp.sock, (char*)glbl.GetLocalHostName(), strlen((char*)glbl.GetLocalHostName()))); - CHKiRet(Send(pData->md.smtp.sock, "\r\n", sizeof("\r\n") - 1)); - CHKiRet(readResponse(pData, &iState, 250)); + CHKiRet(Send(pWrkrData->md.smtp.sock, "HELO ", 5)); + CHKiRet(Send(pWrkrData->md.smtp.sock, (char*)glbl.GetLocalHostName(), strlen((char*)glbl.GetLocalHostName()))); + CHKiRet(Send(pWrkrData->md.smtp.sock, "\r\n", sizeof("\r\n") - 1)); + CHKiRet(readResponse(pWrkrData, &iState, 250)); - CHKiRet(Send(pData->md.smtp.sock, "MAIL FROM: <", sizeof("MAIL FROM: <") - 1)); - CHKiRet(Send(pData->md.smtp.sock, (char*)pData->md.smtp.pszFrom, strlen((char*)pData->md.smtp.pszFrom))); - CHKiRet(Send(pData->md.smtp.sock, ">\r\n", sizeof(">\r\n") - 1)); - CHKiRet(readResponse(pData, &iState, 250)); + CHKiRet(Send(pWrkrData->md.smtp.sock, "MAIL FROM:<", sizeof("MAIL FROM:<") - 1)); + CHKiRet(Send(pWrkrData->md.smtp.sock, (char*)pData->md.smtp.pszFrom, strlen((char*)pData->md.smtp.pszFrom))); + CHKiRet(Send(pWrkrData->md.smtp.sock, ">\r\n", sizeof(">\r\n") - 1)); + CHKiRet(readResponse(pWrkrData, &iState, 250)); - CHKiRet(WriteRcpts(pData, (uchar*)"RCPT TO", sizeof("RCPT TO") - 1, 250)); + CHKiRet(WriteRcpts(pWrkrData, (uchar*)"RCPT TO", sizeof("RCPT TO") - 1, 250)); - CHKiRet(Send(pData->md.smtp.sock, "DATA\r\n", sizeof("DATA\r\n") - 1)); - CHKiRet(readResponse(pData, &iState, 354)); + CHKiRet(Send(pWrkrData->md.smtp.sock, "DATA\r\n", sizeof("DATA\r\n") - 1)); + CHKiRet(readResponse(pWrkrData, &iState, 354)); /* now come the data part */ /* header */ mkSMTPTimestamp(szDateBuf, sizeof(szDateBuf)); - CHKiRet(Send(pData->md.smtp.sock, (char*)szDateBuf, strlen((char*)szDateBuf))); + CHKiRet(Send(pWrkrData->md.smtp.sock, (char*)szDateBuf, strlen((char*)szDateBuf))); - CHKiRet(Send(pData->md.smtp.sock, "From: <", sizeof("From: <") - 1)); - CHKiRet(Send(pData->md.smtp.sock, (char*)pData->md.smtp.pszFrom, strlen((char*)pData->md.smtp.pszFrom))); - CHKiRet(Send(pData->md.smtp.sock, ">\r\n", sizeof(">\r\n") - 1)); + CHKiRet(Send(pWrkrData->md.smtp.sock, "From: <", sizeof("From: <") - 1)); + CHKiRet(Send(pWrkrData->md.smtp.sock, (char*)pData->md.smtp.pszFrom, strlen((char*)pData->md.smtp.pszFrom))); + CHKiRet(Send(pWrkrData->md.smtp.sock, ">\r\n", sizeof(">\r\n") - 1)); - CHKiRet(WriteRcpts(pData, (uchar*)"To", sizeof("To") - 1, -1)); + CHKiRet(WriteRcpts(pWrkrData, (uchar*)"To", sizeof("To") - 1, -1)); - CHKiRet(Send(pData->md.smtp.sock, "Subject: ", sizeof("Subject: ") - 1)); - CHKiRet(Send(pData->md.smtp.sock, (char*)subject, strlen((char*)subject))); - CHKiRet(Send(pData->md.smtp.sock, "\r\n", sizeof("\r\n") - 1)); + CHKiRet(Send(pWrkrData->md.smtp.sock, "Subject: ", sizeof("Subject: ") - 1)); + CHKiRet(Send(pWrkrData->md.smtp.sock, (char*)subject, strlen((char*)subject))); + CHKiRet(Send(pWrkrData->md.smtp.sock, "\r\n", sizeof("\r\n") - 1)); - CHKiRet(Send(pData->md.smtp.sock, "X-Mailer: rsyslog-immail\r\n", sizeof("x-mailer: rsyslog-immail\r\n") - 1)); + CHKiRet(Send(pWrkrData->md.smtp.sock, "X-Mailer: rsyslog-immail\r\n", sizeof("x-mailer: rsyslog-immail\r\n") - 1)); - CHKiRet(Send(pData->md.smtp.sock, "\r\n", sizeof("\r\n") - 1)); /* indicate end of header */ + CHKiRet(Send(pWrkrData->md.smtp.sock, "\r\n", sizeof("\r\n") - 1)); /* indicate end of header */ /* body */ if(pData->bEnableBody) - CHKiRet(bodySend(pData, (char*)body, strlen((char*) body))); + CHKiRet(bodySend(pWrkrData, (char*)body, strlen((char*) body))); /* end of data, back to envelope transaction */ - CHKiRet(Send(pData->md.smtp.sock, "\r\n.\r\n", sizeof("\r\n.\r\n") - 1)); - CHKiRet(readResponse(pData, &iState, 250)); + CHKiRet(Send(pWrkrData->md.smtp.sock, "\r\n.\r\n", sizeof("\r\n.\r\n") - 1)); + CHKiRet(readResponse(pWrkrData, &iState, 250)); - CHKiRet(Send(pData->md.smtp.sock, "QUIT\r\n", sizeof("QUIT\r\n") - 1)); - CHKiRet(readResponse(pData, &iState, 221)); + CHKiRet(Send(pWrkrData->md.smtp.sock, "QUIT\r\n", sizeof("QUIT\r\n") - 1)); + CHKiRet(readResponse(pWrkrData, &iState, 221)); /* we are finished, a new connection is created for each request, so let's close it now */ - CHKiRet(serverDisconnect(pData)); + CHKiRet(serverDisconnect(pWrkrData)); finalize_it: RETiRet; @@ -583,8 +596,8 @@ finalize_it: */ BEGINtryResume CODESTARTtryResume - CHKiRet(serverConnect(pData)); - CHKiRet(serverDisconnect(pData)); /* if we fail, we will never reach this line */ + CHKiRet(serverConnect(pWrkrData)); + CHKiRet(serverDisconnect(pWrkrData)); /* if we fail, we will never reach this line */ finalize_it: if(iRet == RS_RET_IO_ERROR) iRet = RS_RET_SUSPENDED; @@ -593,17 +606,14 @@ ENDtryResume BEGINdoAction CODESTARTdoAction - dbgprintf(" Mail\n"); + DBGPRINTF(" Mail\n"); - /* forward */ - if(pData->bHaveSubject) - iRet = sendSMTP(pData, ppString[0], ppString[1]); - else - iRet = sendSMTP(pData, ppString[0], (uchar*)"message from rsyslog"); + iRet = sendSMTP(pWrkrData, ppString[0], + (pWrkrData->pData->bHaveSubject) ? + ppString[1] : (uchar*)"message from rsyslog"); if(iRet != RS_RET_OK) { - /* error! */ - dbgprintf("error sending mail, suspending\n"); + DBGPRINTF("error sending mail, suspending\n"); iRet = RS_RET_SUSPENDED; } ENDdoAction @@ -689,6 +699,7 @@ ENDmodExit BEGINqueryEtryPt CODESTARTqueryEtryPt CODEqueryEtryPt_STD_OMOD_QUERIES +CODEqueryEtryPt_STD_OMOD8_QUERIES CODEqueryEtryPt_STD_CONF2_CNFNAME_QUERIES ENDqueryEtryPt diff --git a/plugins/ommongodb/Makefile.in b/plugins/ommongodb/Makefile.in index f93db2b..ae41023 100644 --- a/plugins/ommongodb/Makefile.in +++ b/plugins/ommongodb/Makefile.in @@ -154,7 +154,6 @@ GREP = @GREP@ GSS_LIBS = @GSS_LIBS@ GUARDTIME_CFLAGS = @GUARDTIME_CFLAGS@ GUARDTIME_LIBS = @GUARDTIME_LIBS@ -HAVE_LIBGCRYPT_CONFIG = @HAVE_LIBGCRYPT_CONFIG@ HAVE_MYSQL_CONFIG = @HAVE_MYSQL_CONFIG@ HAVE_ORACLE_CONFIG = @HAVE_ORACLE_CONFIG@ HAVE_PGSQL_CONFIG = @HAVE_PGSQL_CONFIG@ @@ -175,14 +174,15 @@ LEXLIB = @LEXLIB@ LEX_OUTPUT_ROOT = @LEX_OUTPUT_ROOT@ LIBDBI_CFLAGS = @LIBDBI_CFLAGS@ LIBDBI_LIBS = @LIBDBI_LIBS@ -LIBEE_CFLAGS = @LIBEE_CFLAGS@ -LIBEE_LIBS = @LIBEE_LIBS@ LIBESTR_CFLAGS = @LIBESTR_CFLAGS@ LIBESTR_LIBS = @LIBESTR_LIBS@ LIBGCRYPT_CFLAGS = @LIBGCRYPT_CFLAGS@ +LIBGCRYPT_CONFIG = @LIBGCRYPT_CONFIG@ LIBGCRYPT_LIBS = @LIBGCRYPT_LIBS@ LIBLOGGING_CFLAGS = @LIBLOGGING_CFLAGS@ LIBLOGGING_LIBS = @LIBLOGGING_LIBS@ +LIBLOGGING_STDLOG_CFLAGS = @LIBLOGGING_STDLOG_CFLAGS@ +LIBLOGGING_STDLOG_LIBS = @LIBLOGGING_STDLOG_LIBS@ LIBLOGNORM_CFLAGS = @LIBLOGNORM_CFLAGS@ LIBLOGNORM_LIBS = @LIBLOGNORM_LIBS@ LIBM = @LIBM@ @@ -207,6 +207,8 @@ NM = @NM@ NMEDIT = @NMEDIT@ OBJDUMP = @OBJDUMP@ OBJEXT = @OBJEXT@ +OPENSSL_CFLAGS = @OPENSSL_CFLAGS@ +OPENSSL_LIBS = @OPENSSL_LIBS@ ORACLE_CFLAGS = @ORACLE_CFLAGS@ ORACLE_LIBS = @ORACLE_LIBS@ OTOOL = @OTOOL@ @@ -388,22 +390,25 @@ distclean-compile: @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/ommongodb_la-ommongodb.Plo@am__quote@ .c.o: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(COMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ $< -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Po +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.o$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c $< +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ $< .c.obj: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(COMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ `$(CYGPATH_W) '$<'` -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Po +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.obj$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ `$(CYGPATH_W) '$<'` &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c `$(CYGPATH_W) '$<'` +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ `$(CYGPATH_W) '$<'` .c.lo: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(LTCOMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ $< -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Plo +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.lo$$||'`;\ +@am__fastdepCC_TRUE@ $(LTCOMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Plo @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=yes @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ @am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(LTCOMPILE) -c -o $@ $< diff --git a/plugins/ommongodb/README b/plugins/ommongodb/README index ad4a8ea..1d9567a 100644 --- a/plugins/ommongodb/README +++ b/plugins/ommongodb/README @@ -16,3 +16,13 @@ changed in v7. If templates are used, it is suggested to use list-based templates. Constants can ONLY be inserted with list-based templates, as only these provide the capability to specify a field name (outname parameter). + +A very basic example is: + +*.* action(type="ommongodb" db="logs" collection="syslog") + +Please see the script clean-mongo-syslog for an example of how to +purge old records from MongoDB using PyMongo. It can be run +daily or weekly from cron. + + diff --git a/plugins/ommongodb/ommongodb.c b/plugins/ommongodb/ommongodb.c index dd99741..14309ac 100644 --- a/plugins/ommongodb/ommongodb.c +++ b/plugins/ommongodb/ommongodb.c @@ -4,7 +4,7 @@ * mongodb C interface is crap. Obtain the library here: * https://github.com/algernon/libmongo-client * - * Copyright 2007-2012 Rainer Gerhards and Adiscon GmbH. + * Copyright 2007-2013 Rainer Gerhards and Adiscon GmbH. * * This file is part of rsyslog. * @@ -33,9 +33,9 @@ #include <stdint.h> #include <time.h> #include <mongo.h> -#include <json/json.h> +#include <json.h> /* For struct json_object_iter, should not be necessary in future versions */ -#include <json/json_object_private.h> +#include <json_object_private.h> #include "rsyslog.h" #include "conf.h" @@ -71,6 +71,10 @@ typedef struct _instanceData { int bErrMsgPermitted; /* only one errmsg permitted per connection */ } instanceData; +typedef struct wrkrInstanceData { + instanceData *pData; +} wrkrInstanceData_t; + /* tables for interfacing with the v6 config system */ /* action (instance) parameters */ @@ -81,7 +85,7 @@ static struct cnfparamdescr actpdescr[] = { { "collection", eCmdHdlrGetWord, 0 }, { "uid", eCmdHdlrGetWord, 0 }, { "pwd", eCmdHdlrGetWord, 0 }, - { "template", eCmdHdlrGetWord, 1 } + { "template", eCmdHdlrGetWord, 0 } }; static struct cnfparamblk actpblk = { CNFPARAMBLK_VERSION, @@ -89,10 +93,16 @@ static struct cnfparamblk actpblk = actpdescr }; +static pthread_mutex_t mutDoAct = PTHREAD_MUTEX_INITIALIZER; + BEGINcreateInstance CODESTARTcreateInstance ENDcreateInstance +BEGINcreateWrkrInstance +CODESTARTcreateWrkrInstance +ENDcreateWrkrInstance + BEGINisCompatibleWithFeature CODESTARTisCompatibleWithFeature /* use this to specify if select features are supported by this @@ -126,6 +136,10 @@ CODESTARTfreeInstance free(pData->tplName); ENDfreeInstance +BEGINfreeWrkrInstance +CODESTARTfreeWrkrInstance +ENDfreeWrkrInstance + BEGINdbgPrintInstInfo CODESTARTdbgPrintInstInfo @@ -179,6 +193,29 @@ static rsRetVal initMongoDB(instanceData *pData, int bSilent) ABORT_FINALIZE(RS_RET_SUSPENDED); } + /* perform authentication */ + if(pData->uid && pData->pwd) { + + /* require both uid and pwd before attempting authentication */ + if(!pData->uid || !pData->pwd) { + dbgprintf("ommongodb: authentication requires uid and pwd attributes set; skipping"); + } + else if(!mongo_sync_cmd_authenticate(pData->conn, (const gchar*)pData->db, + (const gchar*)pData->uid, (const gchar*)pData->pwd)) { + if(!bSilent) { + reportMongoError(pData); + dbgprintf("ommongodb: could not authenticate %s against '%s'", pData->uid, pData->db); + } + + /* no point in continuing with an unauthenticated connection */ + closeMongoDB(pData); + ABORT_FINALIZE(RS_RET_SUSPENDED); + } + else { + dbgprintf("ommongodb: authenticated with %s against '%s'", pData->uid, pData->db); + } + } + finalize_it: RETiRet; } @@ -235,14 +272,20 @@ getDefaultBSON(msg_t *pMsg) int severity, facil; gint64 ts_gen, ts_rcv; /* timestamps: generated, received */ int secfrac; - - procid = MsgGetProp(pMsg, NULL, PROP_PROGRAMNAME, NULL, &procid_len, &procid_free, NULL); - tag = MsgGetProp(pMsg, NULL, PROP_SYSLOGTAG, NULL, &tag_len, &tag_free, NULL); - pid = MsgGetProp(pMsg, NULL, PROP_PROCID, NULL, &pid_len, &pid_free, NULL); - sys = MsgGetProp(pMsg, NULL, PROP_HOSTNAME, NULL, &sys_len, &sys_free, NULL); - msg = MsgGetProp(pMsg, NULL, PROP_MSG, NULL, &msg_len, &msg_free, NULL); - - // TODO: move to datetime? Refactor in any case! rgerhards, 2012-03-30 + msgPropDescr_t cProp; /* we use internal implementation knowledge... */ + + cProp.id = PROP_PROGRAMNAME; + procid = MsgGetProp(pMsg, NULL, &cProp, &procid_len, &procid_free, NULL); + cProp.id = PROP_SYSLOGTAG; + tag = MsgGetProp(pMsg, NULL, &cProp, &tag_len, &tag_free, NULL); + cProp.id = PROP_PROCID; + pid = MsgGetProp(pMsg, NULL, &cProp, &pid_len, &pid_free, NULL); + cProp.id = PROP_HOSTNAME; + sys = MsgGetProp(pMsg, NULL, &cProp, &sys_len, &sys_free, NULL); + cProp.id = PROP_MSG; + msg = MsgGetProp(pMsg, NULL, &cProp, &msg_len, &msg_free, NULL); + + /* TODO: move to datetime? Refactor in any case! rgerhards, 2012-03-30 */ ts_gen = (gint64) datetime.syslogTime2time_t(&pMsg->tTIMESTAMP) * 1000; /* ms! */ dbgprintf("ommongodb: ts_gen is %lld\n", (long long) ts_gen); dbgprintf("ommongodb: secfrac is %d, precision %d\n", pMsg->tTIMESTAMP.secfrac, pMsg->tTIMESTAMP.secfracPrecision); @@ -311,8 +354,11 @@ BSONAppendJSONObject(bson *doc, const gchar *name, struct json_object *json) case json_type_int: { int64_t i; - /* FIXME: the future version will have get_int64 */ +#ifdef HAVE_JSON_OBJECT_NEW_INT64 + i = json_object_get_int64(json); +#else /* HAVE_JSON_OBJECT_NEW_INT64 */ i = json_object_get_int(json); +#endif /* HAVE_JSON_OBJECT_NEW_INT64 */ if (i >= INT32_MIN && i <= INT32_MAX) return bson_append_int32(doc, name, i); else @@ -413,14 +459,17 @@ error: BEGINtryResume CODESTARTtryResume - if(pData->conn == NULL) { - iRet = initMongoDB(pData, 1); + if(pWrkrData->pData->conn == NULL) { + iRet = initMongoDB(pWrkrData->pData, 1); } ENDtryResume BEGINdoAction bson *doc = NULL; + instanceData *pData; CODESTARTdoAction + pthread_mutex_lock(&mutDoAct); + pData = pWrkrData->pData; /* see if we are ready to proceed */ if(pData->conn == NULL) { CHKiRet(initMongoDB(pData, 0)); @@ -445,6 +494,7 @@ CODESTARTdoAction } finalize_it: + pthread_mutex_unlock(&mutDoAct); if(doc != NULL) bson_free(doc); ENDdoAction @@ -551,6 +601,7 @@ ENDmodExit BEGINqueryEtryPt CODESTARTqueryEtryPt CODEqueryEtryPt_STD_OMOD_QUERIES +CODEqueryEtryPt_STD_OMOD8_QUERIES CODEqueryEtryPt_STD_CONF2_OMOD_QUERIES ENDqueryEtryPt diff --git a/plugins/ommysql/Makefile.am b/plugins/ommysql/Makefile.am index e253b9d..f621a1b 100644 --- a/plugins/ommysql/Makefile.am +++ b/plugins/ommysql/Makefile.am @@ -1,6 +1,6 @@ pkglib_LTLIBRARIES = ommysql.la -ommysql_la_SOURCES = ommysql.c ommysql.h +ommysql_la_SOURCES = ommysql.c ommysql_la_CPPFLAGS = $(RSRT_CFLAGS) $(MYSQL_CFLAGS) $(PTHREADS_CFLAGS) ommysql_la_LDFLAGS = -module -avoid-version ommysql_la_LIBADD = $(MYSQL_LIBS) diff --git a/plugins/ommysql/Makefile.in b/plugins/ommysql/Makefile.in index 3052c22..741de85 100644 --- a/plugins/ommysql/Makefile.in +++ b/plugins/ommysql/Makefile.in @@ -154,7 +154,6 @@ GREP = @GREP@ GSS_LIBS = @GSS_LIBS@ GUARDTIME_CFLAGS = @GUARDTIME_CFLAGS@ GUARDTIME_LIBS = @GUARDTIME_LIBS@ -HAVE_LIBGCRYPT_CONFIG = @HAVE_LIBGCRYPT_CONFIG@ HAVE_MYSQL_CONFIG = @HAVE_MYSQL_CONFIG@ HAVE_ORACLE_CONFIG = @HAVE_ORACLE_CONFIG@ HAVE_PGSQL_CONFIG = @HAVE_PGSQL_CONFIG@ @@ -175,14 +174,15 @@ LEXLIB = @LEXLIB@ LEX_OUTPUT_ROOT = @LEX_OUTPUT_ROOT@ LIBDBI_CFLAGS = @LIBDBI_CFLAGS@ LIBDBI_LIBS = @LIBDBI_LIBS@ -LIBEE_CFLAGS = @LIBEE_CFLAGS@ -LIBEE_LIBS = @LIBEE_LIBS@ LIBESTR_CFLAGS = @LIBESTR_CFLAGS@ LIBESTR_LIBS = @LIBESTR_LIBS@ LIBGCRYPT_CFLAGS = @LIBGCRYPT_CFLAGS@ +LIBGCRYPT_CONFIG = @LIBGCRYPT_CONFIG@ LIBGCRYPT_LIBS = @LIBGCRYPT_LIBS@ LIBLOGGING_CFLAGS = @LIBLOGGING_CFLAGS@ LIBLOGGING_LIBS = @LIBLOGGING_LIBS@ +LIBLOGGING_STDLOG_CFLAGS = @LIBLOGGING_STDLOG_CFLAGS@ +LIBLOGGING_STDLOG_LIBS = @LIBLOGGING_STDLOG_LIBS@ LIBLOGNORM_CFLAGS = @LIBLOGNORM_CFLAGS@ LIBLOGNORM_LIBS = @LIBLOGNORM_LIBS@ LIBM = @LIBM@ @@ -207,6 +207,8 @@ NM = @NM@ NMEDIT = @NMEDIT@ OBJDUMP = @OBJDUMP@ OBJEXT = @OBJEXT@ +OPENSSL_CFLAGS = @OPENSSL_CFLAGS@ +OPENSSL_LIBS = @OPENSSL_LIBS@ ORACLE_CFLAGS = @ORACLE_CFLAGS@ ORACLE_LIBS = @ORACLE_LIBS@ OTOOL = @OTOOL@ @@ -306,7 +308,7 @@ top_build_prefix = @top_build_prefix@ top_builddir = @top_builddir@ top_srcdir = @top_srcdir@ pkglib_LTLIBRARIES = ommysql.la -ommysql_la_SOURCES = ommysql.c ommysql.h +ommysql_la_SOURCES = ommysql.c ommysql_la_CPPFLAGS = $(RSRT_CFLAGS) $(MYSQL_CFLAGS) $(PTHREADS_CFLAGS) ommysql_la_LDFLAGS = -module -avoid-version ommysql_la_LIBADD = $(MYSQL_LIBS) @@ -388,22 +390,25 @@ distclean-compile: @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/ommysql_la-ommysql.Plo@am__quote@ .c.o: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(COMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ $< -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Po +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.o$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c $< +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ $< .c.obj: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(COMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ `$(CYGPATH_W) '$<'` -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Po +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.obj$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ `$(CYGPATH_W) '$<'` &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c `$(CYGPATH_W) '$<'` +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ `$(CYGPATH_W) '$<'` .c.lo: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(LTCOMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ $< -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Plo +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.lo$$||'`;\ +@am__fastdepCC_TRUE@ $(LTCOMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Plo @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=yes @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ @am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(LTCOMPILE) -c -o $@ $< diff --git a/plugins/ommysql/ommysql.c b/plugins/ommysql/ommysql.c index 2dfa29d..5a6fbb1 100644 --- a/plugins/ommysql/ommysql.c +++ b/plugins/ommysql/ommysql.c @@ -6,7 +6,7 @@ * * File begun on 2007-07-20 by RGerhards (extracted from syslogd.c) * - * Copyright 2007-2012 Adiscon GmbH. + * Copyright 2007-2014 Adiscon GmbH. * * This file is part of rsyslog. * Licensed under the Apache License, Version 2.0 (the "License"); @@ -38,7 +38,6 @@ #include "syslogd-types.h" #include "srUtils.h" #include "template.h" -#include "ommysql.h" #include "module-template.h" #include "errmsg.h" #include "cfsysline.h" @@ -55,18 +54,22 @@ DEF_OMOD_STATIC_DATA DEFobjCurrIf(errmsg) typedef struct _instanceData { - MYSQL *f_hmysql; /* handle to MySQL */ - char f_dbsrv[MAXHOSTNAMELEN+1]; /* IP or hostname of DB server*/ - unsigned int f_dbsrvPort; /* port of MySQL server */ - char f_dbname[_DB_MAXDBLEN+1]; /* DB name */ - char f_dbuid[_DB_MAXUNAMELEN+1]; /* DB user */ - char f_dbpwd[_DB_MAXPWDLEN+1]; /* DB user's password */ - unsigned uLastMySQLErrno; /* last errno returned by MySQL or 0 if all is well */ - uchar * f_configfile; /* MySQL Client Configuration File */ - uchar * f_configsection; /* MySQL Client Configuration Section */ - uchar *tplName; /* format template to use */ + char dbsrv[MAXHOSTNAMELEN+1]; /* IP or hostname of DB server*/ + unsigned int dbsrvPort; /* port of MySQL server */ + char dbname[_DB_MAXDBLEN+1]; /* DB name */ + char dbuid[_DB_MAXUNAMELEN+1]; /* DB user */ + char dbpwd[_DB_MAXPWDLEN+1]; /* DB user's password */ + uchar *configfile; /* MySQL Client Configuration File */ + uchar *configsection; /* MySQL Client Configuration Section */ + uchar *tplName; /* format template to use */ } instanceData; +typedef struct wrkrInstanceData { + instanceData *pData; + MYSQL *hmysql; /* handle to MySQL */ + unsigned uLastMySQLErrno; /* last errno returned by MySQL or 0 if all is well */ +} wrkrInstanceData_t; + typedef struct configSettings_s { int iSrvPort; /* database server port */ uchar *pszMySQLConfigFile; /* MySQL Client Configuration File */ @@ -104,6 +107,12 @@ CODESTARTcreateInstance ENDcreateInstance +BEGINcreateWrkrInstance +CODESTARTcreateWrkrInstance + pWrkrData->hmysql = NULL; +ENDcreateWrkrInstance + + BEGINisCompatibleWithFeature CODESTARTisCompatibleWithFeature if(eFeat == sFEATURERepeatedMsgReduction) @@ -115,33 +124,29 @@ ENDisCompatibleWithFeature * MySQL connection. * Initially added 2004-10-28 */ -static void closeMySQL(instanceData *pData) +static void closeMySQL(wrkrInstanceData_t *pWrkrData) { - ASSERT(pData != NULL); - - if(pData->f_hmysql != NULL) { /* just to be on the safe side... */ - mysql_close(pData->f_hmysql); - pData->f_hmysql = NULL; - } - if(pData->f_configfile!=NULL){ - free(pData->f_configfile); - pData->f_configfile=NULL; - } - if(pData->f_configsection!=NULL){ - free(pData->f_configsection); - pData->f_configsection=NULL; + if(pWrkrData->hmysql != NULL) { /* just to be on the safe side... */ + mysql_close(pWrkrData->hmysql); + pWrkrData->hmysql = NULL; } } BEGINfreeInstance CODESTARTfreeInstance - free(pData->f_configfile); - free(pData->f_configsection); + free(pData->configfile); + free(pData->configsection); free(pData->tplName); - closeMySQL(pData); ENDfreeInstance +BEGINfreeWrkrInstance +CODESTARTfreeWrkrInstance + closeMySQL(pWrkrData); + mysql_thread_end(); +ENDfreeWrkrInstance + + BEGINdbgPrintInstInfo CODESTARTdbgPrintInstInfo /* nothing special here */ @@ -152,25 +157,23 @@ ENDdbgPrintInstInfo * We check if we have a valid MySQL handle. If not, we simply * report an error, but can not be specific. RGerhards, 2007-01-30 */ -static void reportDBError(instanceData *pData, int bSilent) +static void reportDBError(wrkrInstanceData_t *pWrkrData, int bSilent) { char errMsg[512]; unsigned uMySQLErrno; - ASSERT(pData != NULL); - /* output log message */ errno = 0; - if(pData->f_hmysql == NULL) { + if(pWrkrData->hmysql == NULL) { errmsg.LogError(0, NO_ERRCODE, "unknown DB error occured - could not obtain MySQL handle"); } else { /* we can ask mysql for the error description... */ - uMySQLErrno = mysql_errno(pData->f_hmysql); + uMySQLErrno = mysql_errno(pWrkrData->hmysql); snprintf(errMsg, sizeof(errMsg)/sizeof(char), "db error (%d): %s\n", uMySQLErrno, - mysql_error(pData->f_hmysql)); - if(bSilent || uMySQLErrno == pData->uLastMySQLErrno) + mysql_error(pWrkrData->hmysql)); + if(bSilent || uMySQLErrno == pWrkrData->uLastMySQLErrno) dbgprintf("mysql, DBError(silent): %s\n", errMsg); else { - pData->uLastMySQLErrno = uMySQLErrno; + pWrkrData->uLastMySQLErrno = uMySQLErrno; errmsg.LogError(0, NO_ERRCODE, "%s", errMsg); } } @@ -183,25 +186,26 @@ static void reportDBError(instanceData *pData, int bSilent) * MySQL connection. * Initially added 2004-10-28 mmeckelein */ -static rsRetVal initMySQL(instanceData *pData, int bSilent) +static rsRetVal initMySQL(wrkrInstanceData_t *pWrkrData, int bSilent) { + instanceData *pData; DEFiRet; - ASSERT(pData != NULL); - ASSERT(pData->f_hmysql == NULL); - pData->f_hmysql = mysql_init(NULL); - if(pData->f_hmysql == NULL) { + ASSERT(pWrkrData->hmysql == NULL); + pData = pWrkrData->pData; + pWrkrData->hmysql = mysql_init(NULL); + if(pWrkrData->hmysql == NULL) { errmsg.LogError(0, RS_RET_SUSPENDED, "can not initialize MySQL handle"); iRet = RS_RET_SUSPENDED; } else { /* we could get the handle, now on with work... */ - mysql_options(pData->f_hmysql,MYSQL_READ_DEFAULT_GROUP,((pData->f_configsection!=NULL)?(char*)pData->f_configsection:"client")); - if(pData->f_configfile!=NULL){ + mysql_options(pWrkrData->hmysql,MYSQL_READ_DEFAULT_GROUP,((pData->configsection!=NULL)?(char*)pData->configsection:"client")); + if(pData->configfile!=NULL){ FILE * fp; - fp=fopen((char*)pData->f_configfile,"r"); + fp=fopen((char*)pData->configfile,"r"); int err=errno; if(fp==NULL){ char msg[512]; - snprintf(msg,sizeof(msg)/sizeof(char),"Could not open '%s' for reading",pData->f_configfile); + snprintf(msg,sizeof(msg)/sizeof(char),"Could not open '%s' for reading",pData->configfile); if(bSilent) { char errStr[512]; rs_strerror_r(err, errStr, sizeof(errStr)); @@ -210,17 +214,17 @@ static rsRetVal initMySQL(instanceData *pData, int bSilent) errmsg.LogError(err,NO_ERRCODE,"mysql configuration error: %s\n",msg); } else { fclose(fp); - mysql_options(pData->f_hmysql,MYSQL_READ_DEFAULT_FILE,pData->f_configfile); + mysql_options(pWrkrData->hmysql,MYSQL_READ_DEFAULT_FILE,pData->configfile); } } /* Connect to database */ - if(mysql_real_connect(pData->f_hmysql, pData->f_dbsrv, pData->f_dbuid, - pData->f_dbpwd, pData->f_dbname, pData->f_dbsrvPort, NULL, 0) == NULL) { - reportDBError(pData, bSilent); - closeMySQL(pData); /* ignore any error we may get */ + if(mysql_real_connect(pWrkrData->hmysql, pData->dbsrv, pData->dbuid, + pData->dbpwd, pData->dbname, pData->dbsrvPort, NULL, 0) == NULL) { + reportDBError(pWrkrData, bSilent); + closeMySQL(pWrkrData); /* ignore any error we may get */ ABORT_FINALIZE(RS_RET_SUSPENDED); } - mysql_autocommit(pData->f_hmysql, 0); + mysql_autocommit(pWrkrData->hmysql, 0); } finalize_it: @@ -232,35 +236,32 @@ finalize_it: * to an established MySQL session. * Initially added 2004-10-28 mmeckelein */ -rsRetVal writeMySQL(uchar *psz, instanceData *pData) +rsRetVal writeMySQL(wrkrInstanceData_t *pWrkrData, uchar *psz) { DEFiRet; - ASSERT(psz != NULL); - ASSERT(pData != NULL); - /* see if we are ready to proceed */ - if(pData->f_hmysql == NULL) { - CHKiRet(initMySQL(pData, 0)); + if(pWrkrData->hmysql == NULL) { + CHKiRet(initMySQL(pWrkrData, 0)); } /* try insert */ - if(mysql_query(pData->f_hmysql, (char*)psz)) { + if(mysql_query(pWrkrData->hmysql, (char*)psz)) { /* error occured, try to re-init connection and retry */ - closeMySQL(pData); /* close the current handle */ - CHKiRet(initMySQL(pData, 0)); /* try to re-open */ - if(mysql_query(pData->f_hmysql, (char*)psz)) { /* re-try insert */ + closeMySQL(pWrkrData); /* close the current handle */ + CHKiRet(initMySQL(pWrkrData, 0)); /* try to re-open */ + if(mysql_query(pWrkrData->hmysql, (char*)psz)) { /* re-try insert */ /* we failed, giving up for now */ - reportDBError(pData, 0); - closeMySQL(pData); /* free ressources */ + reportDBError(pWrkrData, 0); + closeMySQL(pWrkrData); /* free ressources */ ABORT_FINALIZE(RS_RET_SUSPENDED); } } finalize_it: if(iRet == RS_RET_OK) { - pData->uLastMySQLErrno = 0; /* reset error for error supression */ + pWrkrData->uLastMySQLErrno = 0; /* reset error for error supression */ } RETiRet; @@ -269,28 +270,28 @@ finalize_it: BEGINtryResume CODESTARTtryResume - if(pData->f_hmysql == NULL) { - iRet = initMySQL(pData, 1); + if(pWrkrData->hmysql == NULL) { + iRet = initMySQL(pWrkrData, 1); } ENDtryResume BEGINbeginTransaction CODESTARTbeginTransaction - CHKiRet(writeMySQL((uchar*)"START TRANSACTION", pData)); + CHKiRet(writeMySQL(pWrkrData, (uchar*)"START TRANSACTION")); finalize_it: ENDbeginTransaction BEGINdoAction CODESTARTdoAction dbgprintf("\n"); - CHKiRet(writeMySQL(ppString[0], pData)); + CHKiRet(writeMySQL(pWrkrData, ppString[0])); iRet = RS_RET_DEFER_COMMIT; finalize_it: ENDdoAction BEGINendTransaction CODESTARTendTransaction - if (mysql_commit(pData->f_hmysql) != 0) { + if(mysql_commit(pWrkrData->hmysql) != 0) { dbgprintf("mysql server error: transaction not committed\n"); iRet = RS_RET_SUSPENDED; } @@ -300,11 +301,10 @@ ENDendTransaction static inline void setInstParamDefaults(instanceData *pData) { - pData->f_dbsrvPort = 0; - pData->f_configfile = NULL; - pData->f_configsection = NULL; + pData->dbsrvPort = 0; + pData->configfile = NULL; + pData->configsection = NULL; pData->tplName = NULL; - pData->f_hmysql = NULL; /* initialize, but connect only on first message (important for queued mode!) */ } @@ -329,26 +329,26 @@ CODESTARTnewActInst continue; if(!strcmp(actpblk.descr[i].name, "server")) { cstr = es_str2cstr(pvals[i].val.d.estr, NULL); - strncpy(pData->f_dbsrv, cstr, sizeof(pData->f_dbsrv)); + strncpy(pData->dbsrv, cstr, sizeof(pData->dbsrv)); free(cstr); } else if(!strcmp(actpblk.descr[i].name, "serverport")) { - pData->f_dbsrvPort = (int) pvals[i].val.d.n, NULL; + pData->dbsrvPort = (int) pvals[i].val.d.n, NULL; } else if(!strcmp(actpblk.descr[i].name, "db")) { cstr = es_str2cstr(pvals[i].val.d.estr, NULL); - strncpy(pData->f_dbname, cstr, sizeof(pData->f_dbname)); + strncpy(pData->dbname, cstr, sizeof(pData->dbname)); free(cstr); } else if(!strcmp(actpblk.descr[i].name, "uid")) { cstr = es_str2cstr(pvals[i].val.d.estr, NULL); - strncpy(pData->f_dbuid, cstr, sizeof(pData->f_dbuid)); + strncpy(pData->dbuid, cstr, sizeof(pData->dbuid)); free(cstr); } else if(!strcmp(actpblk.descr[i].name, "pwd")) { cstr = es_str2cstr(pvals[i].val.d.estr, NULL); - strncpy(pData->f_dbpwd, cstr, sizeof(pData->f_dbpwd)); + strncpy(pData->dbpwd, cstr, sizeof(pData->dbpwd)); free(cstr); } else if(!strcmp(actpblk.descr[i].name, "mysqlconfig.file")) { - pData->f_configfile = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + pData->configfile = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); } else if(!strcmp(actpblk.descr[i].name, "mysqlconfig.section")) { - pData->f_configsection = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + pData->configsection = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); } else if(!strcmp(actpblk.descr[i].name, "template")) { pData->tplName = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); } else { @@ -399,19 +399,19 @@ CODE_STD_STRING_REQUESTparseSelectorAct(1) * Now we read the MySQL connection properties * and verify that the properties are valid. */ - if(getSubString(&p, pData->f_dbsrv, MAXHOSTNAMELEN+1, ',')) + if(getSubString(&p, pData->dbsrv, MAXHOSTNAMELEN+1, ',')) iMySQLPropErr++; - if(*pData->f_dbsrv == '\0') + if(*pData->dbsrv == '\0') iMySQLPropErr++; - if(getSubString(&p, pData->f_dbname, _DB_MAXDBLEN+1, ',')) + if(getSubString(&p, pData->dbname, _DB_MAXDBLEN+1, ',')) iMySQLPropErr++; - if(*pData->f_dbname == '\0') + if(*pData->dbname == '\0') iMySQLPropErr++; - if(getSubString(&p, pData->f_dbuid, _DB_MAXUNAMELEN+1, ',')) + if(getSubString(&p, pData->dbuid, _DB_MAXUNAMELEN+1, ',')) iMySQLPropErr++; - if(*pData->f_dbuid == '\0') + if(*pData->dbuid == '\0') iMySQLPropErr++; - if(getSubString(&p, pData->f_dbpwd, _DB_MAXPWDLEN+1, ';')) + if(getSubString(&p, pData->dbpwd, _DB_MAXPWDLEN+1, ';')) iMySQLPropErr++; /* now check for template * We specify that the SQL option must be present in the template. @@ -431,10 +431,9 @@ CODE_STD_STRING_REQUESTparseSelectorAct(1) errmsg.LogError(0, RS_RET_INVALID_PARAMS, "Trouble with MySQL connection properties. -MySQL logging disabled"); ABORT_FINALIZE(RS_RET_INVALID_PARAMS); } else { - pData->f_dbsrvPort = (unsigned) cs.iSrvPort; /* set configured port */ - pData->f_configfile = cs.pszMySQLConfigFile; - pData->f_configsection = cs.pszMySQLConfigSection; - pData->f_hmysql = NULL; /* initialize, but connect only on first message (important for queued mode!) */ + pData->dbsrvPort = (unsigned) cs.iSrvPort; /* set configured port */ + pData->configfile = cs.pszMySQLConfigFile; + pData->configsection = cs.pszMySQLConfigSection; } CODE_STD_FINALIZERparseSelectorAct @@ -454,6 +453,7 @@ ENDmodExit BEGINqueryEtryPt CODESTARTqueryEtryPt CODEqueryEtryPt_STD_OMOD_QUERIES +CODEqueryEtryPt_STD_OMOD8_QUERIES CODEqueryEtryPt_STD_CONF2_OMOD_QUERIES CODEqueryEtryPt_TXIF_OMOD_QUERIES /* we support the transactional interface! */ ENDqueryEtryPt @@ -492,7 +492,7 @@ CODEmodInit_QueryRegCFSLineHdlr mysql_server_init(0, NULL, NULL) # endif ) { - errmsg.LogError(0, NO_ERRCODE, "ommysql: mysql_server_init() failed, plugin " + errmsg.LogError(0, NO_ERRCODE, "ommysql: intializing mysql client failed, plugin " "can not run"); ABORT_FINALIZE(RS_RET_ERR); } diff --git a/plugins/ommysql/ommysql.h b/plugins/ommysql/ommysql.h deleted file mode 100644 index d807578..0000000 --- a/plugins/ommysql/ommysql.h +++ /dev/null @@ -1,31 +0,0 @@ -/* omusrmsg.c - * These are the definitions for the build-in MySQL output module. - * - * File begun on 2007-07-13 by RGerhards (extracted from syslogd.c) - * - * Copyright 2007 Rainer Gerhards and Adiscon GmbH. - * - * This file is part of rsyslog. - * - * Rsyslog is free software: you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation, either version 3 of the License, or - * (at your option) any later version. - * - * Rsyslog is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * - * You should have received a copy of the GNU General Public License - * along with Rsyslog. If not, see <http://www.gnu.org/licenses/>. - * - * A copy of the GPL can be found in the file "COPYING" in this distribution. - */ -#ifndef OMMYSQL_H_INCLUDED -#define OMMYSQL_H_INCLUDED 1 - -#endif /* #ifndef OMMYSQL_H_INCLUDED */ -/* - * vi:set ai: - */ diff --git a/plugins/omoracle/Makefile.in b/plugins/omoracle/Makefile.in index 5fe0944..d27899f 100644 --- a/plugins/omoracle/Makefile.in +++ b/plugins/omoracle/Makefile.in @@ -154,7 +154,6 @@ GREP = @GREP@ GSS_LIBS = @GSS_LIBS@ GUARDTIME_CFLAGS = @GUARDTIME_CFLAGS@ GUARDTIME_LIBS = @GUARDTIME_LIBS@ -HAVE_LIBGCRYPT_CONFIG = @HAVE_LIBGCRYPT_CONFIG@ HAVE_MYSQL_CONFIG = @HAVE_MYSQL_CONFIG@ HAVE_ORACLE_CONFIG = @HAVE_ORACLE_CONFIG@ HAVE_PGSQL_CONFIG = @HAVE_PGSQL_CONFIG@ @@ -175,14 +174,15 @@ LEXLIB = @LEXLIB@ LEX_OUTPUT_ROOT = @LEX_OUTPUT_ROOT@ LIBDBI_CFLAGS = @LIBDBI_CFLAGS@ LIBDBI_LIBS = @LIBDBI_LIBS@ -LIBEE_CFLAGS = @LIBEE_CFLAGS@ -LIBEE_LIBS = @LIBEE_LIBS@ LIBESTR_CFLAGS = @LIBESTR_CFLAGS@ LIBESTR_LIBS = @LIBESTR_LIBS@ LIBGCRYPT_CFLAGS = @LIBGCRYPT_CFLAGS@ +LIBGCRYPT_CONFIG = @LIBGCRYPT_CONFIG@ LIBGCRYPT_LIBS = @LIBGCRYPT_LIBS@ LIBLOGGING_CFLAGS = @LIBLOGGING_CFLAGS@ LIBLOGGING_LIBS = @LIBLOGGING_LIBS@ +LIBLOGGING_STDLOG_CFLAGS = @LIBLOGGING_STDLOG_CFLAGS@ +LIBLOGGING_STDLOG_LIBS = @LIBLOGGING_STDLOG_LIBS@ LIBLOGNORM_CFLAGS = @LIBLOGNORM_CFLAGS@ LIBLOGNORM_LIBS = @LIBLOGNORM_LIBS@ LIBM = @LIBM@ @@ -207,6 +207,8 @@ NM = @NM@ NMEDIT = @NMEDIT@ OBJDUMP = @OBJDUMP@ OBJEXT = @OBJEXT@ +OPENSSL_CFLAGS = @OPENSSL_CFLAGS@ +OPENSSL_LIBS = @OPENSSL_LIBS@ ORACLE_CFLAGS = @ORACLE_CFLAGS@ ORACLE_LIBS = @ORACLE_LIBS@ OTOOL = @OTOOL@ @@ -387,22 +389,25 @@ distclean-compile: @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/omoracle_la-omoracle.Plo@am__quote@ .c.o: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(COMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ $< -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Po +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.o$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c $< +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ $< .c.obj: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(COMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ `$(CYGPATH_W) '$<'` -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Po +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.obj$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ `$(CYGPATH_W) '$<'` &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c `$(CYGPATH_W) '$<'` +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ `$(CYGPATH_W) '$<'` .c.lo: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(LTCOMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ $< -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Plo +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.lo$$||'`;\ +@am__fastdepCC_TRUE@ $(LTCOMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Plo @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=yes @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ @am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(LTCOMPILE) -c -o $@ $< diff --git a/plugins/ompgsql/Makefile.in b/plugins/ompgsql/Makefile.in index 798096d..2a58bd6 100644 --- a/plugins/ompgsql/Makefile.in +++ b/plugins/ompgsql/Makefile.in @@ -154,7 +154,6 @@ GREP = @GREP@ GSS_LIBS = @GSS_LIBS@ GUARDTIME_CFLAGS = @GUARDTIME_CFLAGS@ GUARDTIME_LIBS = @GUARDTIME_LIBS@ -HAVE_LIBGCRYPT_CONFIG = @HAVE_LIBGCRYPT_CONFIG@ HAVE_MYSQL_CONFIG = @HAVE_MYSQL_CONFIG@ HAVE_ORACLE_CONFIG = @HAVE_ORACLE_CONFIG@ HAVE_PGSQL_CONFIG = @HAVE_PGSQL_CONFIG@ @@ -175,14 +174,15 @@ LEXLIB = @LEXLIB@ LEX_OUTPUT_ROOT = @LEX_OUTPUT_ROOT@ LIBDBI_CFLAGS = @LIBDBI_CFLAGS@ LIBDBI_LIBS = @LIBDBI_LIBS@ -LIBEE_CFLAGS = @LIBEE_CFLAGS@ -LIBEE_LIBS = @LIBEE_LIBS@ LIBESTR_CFLAGS = @LIBESTR_CFLAGS@ LIBESTR_LIBS = @LIBESTR_LIBS@ LIBGCRYPT_CFLAGS = @LIBGCRYPT_CFLAGS@ +LIBGCRYPT_CONFIG = @LIBGCRYPT_CONFIG@ LIBGCRYPT_LIBS = @LIBGCRYPT_LIBS@ LIBLOGGING_CFLAGS = @LIBLOGGING_CFLAGS@ LIBLOGGING_LIBS = @LIBLOGGING_LIBS@ +LIBLOGGING_STDLOG_CFLAGS = @LIBLOGGING_STDLOG_CFLAGS@ +LIBLOGGING_STDLOG_LIBS = @LIBLOGGING_STDLOG_LIBS@ LIBLOGNORM_CFLAGS = @LIBLOGNORM_CFLAGS@ LIBLOGNORM_LIBS = @LIBLOGNORM_LIBS@ LIBM = @LIBM@ @@ -207,6 +207,8 @@ NM = @NM@ NMEDIT = @NMEDIT@ OBJDUMP = @OBJDUMP@ OBJEXT = @OBJEXT@ +OPENSSL_CFLAGS = @OPENSSL_CFLAGS@ +OPENSSL_LIBS = @OPENSSL_LIBS@ ORACLE_CFLAGS = @ORACLE_CFLAGS@ ORACLE_LIBS = @ORACLE_LIBS@ OTOOL = @OTOOL@ @@ -388,22 +390,25 @@ distclean-compile: @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/ompgsql_la-ompgsql.Plo@am__quote@ .c.o: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(COMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ $< -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Po +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.o$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c $< +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ $< .c.obj: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(COMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ `$(CYGPATH_W) '$<'` -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Po +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.obj$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ `$(CYGPATH_W) '$<'` &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c `$(CYGPATH_W) '$<'` +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ `$(CYGPATH_W) '$<'` .c.lo: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(LTCOMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ $< -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Plo +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.lo$$||'`;\ +@am__fastdepCC_TRUE@ $(LTCOMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Plo @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=yes @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ @am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(LTCOMPILE) -c -o $@ $< diff --git a/plugins/ompgsql/createDB.sql b/plugins/ompgsql/createDB.sql index 2f72a0a..0c0f7e1 100644 --- a/plugins/ompgsql/createDB.sql +++ b/plugins/ompgsql/createDB.sql @@ -1,4 +1,4 @@ -CREATE DATABASE 'Syslog' WITH ENCODING 'SQL_ASCII'; +CREATE DATABASE "Syslog" WITH ENCODING 'SQL_ASCII' TEMPLATE template0; \c Syslog; CREATE TABLE SystemEvents ( diff --git a/plugins/ompgsql/ompgsql.c b/plugins/ompgsql/ompgsql.c index 11f346f..6b89717 100644 --- a/plugins/ompgsql/ompgsql.c +++ b/plugins/ompgsql/ompgsql.c @@ -6,7 +6,7 @@ * * File begun on 2007-10-18 by sur5r (converted from ommysql.c) * - * Copyright 2007, 2009 Rainer Gerhards and Adiscon GmbH. + * Copyright 2007, 2013 Rainer Gerhards and Adiscon GmbH. * * The following link my be useful for the not-so-postgres literate * when setting up a test environment (on Fedora): @@ -66,11 +66,17 @@ typedef struct _instanceData { ConnStatusType eLastPgSQLStatus; /* last status from postgres */ } instanceData; +typedef struct wrkrInstanceData { + instanceData *pData; +} wrkrInstanceData_t; + typedef struct configSettings_s { EMPTY_STRUCT } configSettings_t; static configSettings_t __attribute__((unused)) cs; +static pthread_mutex_t mutDoAct = PTHREAD_MUTEX_INITIALIZER; + BEGINinitConfVars /* (re)set config variables to default values */ CODESTARTinitConfVars ENDinitConfVars @@ -82,6 +88,10 @@ BEGINcreateInstance CODESTARTcreateInstance ENDcreateInstance +BEGINcreateWrkrInstance +CODESTARTcreateWrkrInstance +ENDcreateWrkrInstance + BEGINisCompatibleWithFeature CODESTARTisCompatibleWithFeature @@ -108,6 +118,9 @@ CODESTARTfreeInstance closePgSQL(pData); ENDfreeInstance +BEGINfreeWrkrInstance +CODESTARTfreeWrkrInstance +ENDfreeWrkrInstance BEGINdbgPrintInstInfo CODESTARTdbgPrintInstInfo @@ -222,9 +235,18 @@ writePgSQL(uchar *psz, instanceData *pData) if(bHadError || (PQstatus(pData->f_hpgsql) != CONNECTION_OK)) { /* error occured, try to re-init connection and retry */ - closePgSQL(pData); /* close the current handle */ - CHKiRet(initPgSQL(pData, 0)); /* try to re-open */ - bHadError = tryExec(psz, pData); /* retry */ + int inTransaction = 0; + if(pData->f_hpgsql != NULL) { + PGTransactionStatusType xactStatus = PQtransactionStatus(pData->f_hpgsql); + if((xactStatus == PQTRANS_INTRANS) || (xactStatus == PQTRANS_ACTIVE)) { + inTransaction = 1; + } + } + if ( inTransaction == 0 ) { + closePgSQL(pData); /* close the current handle */ + CHKiRet(initPgSQL(pData, 0)); /* try to re-open */ + bHadError = tryExec(psz, pData); /* retry */ + } if(bHadError || (PQstatus(pData->f_hpgsql) != CONNECTION_OK)) { /* we failed, giving up for now */ reportDBError(pData, 0); @@ -244,8 +266,8 @@ finalize_it: BEGINtryResume CODESTARTtryResume - if(pData->f_hpgsql == NULL) { - iRet = initPgSQL(pData, 1); + if(pWrkrData->pData->f_hpgsql == NULL) { + iRet = initPgSQL(pWrkrData->pData, 1); if(iRet == RS_RET_OK) { /* the code above seems not to actually connect to the database. As such, we do a * dummy statement (a pointless select...) to verify the connection and return @@ -253,7 +275,7 @@ CODESTARTtryResume * PostgreSQL expert, so any patch that does the desired result in a more * intelligent way is highly welcome. -- rgerhards, 2009-12-16 */ - iRet = writePgSQL((uchar*)"select 'a' as a", pData); + iRet = writePgSQL((uchar*)"select 'a' as a", pWrkrData->pData); } } @@ -262,24 +284,28 @@ ENDtryResume BEGINbeginTransaction CODESTARTbeginTransaction -dbgprintf("ompgsql: beginTransaction\n"); - iRet = writePgSQL((uchar*) "begin", pData); /* TODO: make user-configurable */ + dbgprintf("ompgsql: beginTransaction\n"); + if(pWrkrData->pData->f_hpgsql == NULL) + initPgSQL(pWrkrData->pData, 0); + iRet = writePgSQL((uchar*) "begin", pWrkrData->pData); /* TODO: make user-configurable */ ENDbeginTransaction BEGINdoAction CODESTARTdoAction + pthread_mutex_lock(&mutDoAct); dbgprintf("\n"); - CHKiRet(writePgSQL(ppString[0], pData)); + CHKiRet(writePgSQL(ppString[0], pWrkrData->pData)); if(bCoreSupportsBatching) iRet = RS_RET_DEFER_COMMIT; finalize_it: + pthread_mutex_unlock(&mutDoAct); ENDdoAction BEGINendTransaction CODESTARTendTransaction - iRet = writePgSQL((uchar*) "commit;", pData); /* TODO: make user-configurable */ + iRet = writePgSQL((uchar*) "commit;", pWrkrData->pData); /* TODO: make user-configurable */ dbgprintf("ompgsql: endTransaction\n"); ENDendTransaction @@ -346,8 +372,6 @@ CODE_STD_STRING_REQUESTparseSelectorAct(1) if (iPgSQLPropErr) { errmsg.LogError(0, RS_RET_INVALID_PARAMS, "Trouble with PgSQL connection properties. -PgSQL logging disabled"); ABORT_FINALIZE(RS_RET_INVALID_PARAMS); - } else { - CHKiRet(initPgSQL(pData, 0)); } CODE_STD_FINALIZERparseSelectorAct @@ -361,6 +385,7 @@ ENDmodExit BEGINqueryEtryPt CODESTARTqueryEtryPt CODEqueryEtryPt_STD_OMOD_QUERIES +CODEqueryEtryPt_STD_OMOD8_QUERIES CODEqueryEtryPt_TXIF_OMOD_QUERIES /* we support the transactional interface! */ ENDqueryEtryPt @@ -372,6 +397,11 @@ INITLegCnfVars CODEmodInit_QueryRegCFSLineHdlr CHKiRet(objUse(errmsg, CORE_COMPONENT)); INITChkCoreFeature(bCoreSupportsBatching, CORE_FEATURE_BATCHING); + +# warning: transaction support missing for v8 + bCoreSupportsBatching= 0; + DBGPRINTF("ompgsql: transactions are not yet supported on v8\n"); + DBGPRINTF("ompgsql: module compiled with rsyslog version %s.\n", VERSION); DBGPRINTF("ompgsql: %susing transactional output interface.\n", bCoreSupportsBatching ? "" : "not "); ENDmodInit diff --git a/plugins/omprog/Makefile.in b/plugins/omprog/Makefile.in index eb3e421..c08423c 100644 --- a/plugins/omprog/Makefile.in +++ b/plugins/omprog/Makefile.in @@ -153,7 +153,6 @@ GREP = @GREP@ GSS_LIBS = @GSS_LIBS@ GUARDTIME_CFLAGS = @GUARDTIME_CFLAGS@ GUARDTIME_LIBS = @GUARDTIME_LIBS@ -HAVE_LIBGCRYPT_CONFIG = @HAVE_LIBGCRYPT_CONFIG@ HAVE_MYSQL_CONFIG = @HAVE_MYSQL_CONFIG@ HAVE_ORACLE_CONFIG = @HAVE_ORACLE_CONFIG@ HAVE_PGSQL_CONFIG = @HAVE_PGSQL_CONFIG@ @@ -174,14 +173,15 @@ LEXLIB = @LEXLIB@ LEX_OUTPUT_ROOT = @LEX_OUTPUT_ROOT@ LIBDBI_CFLAGS = @LIBDBI_CFLAGS@ LIBDBI_LIBS = @LIBDBI_LIBS@ -LIBEE_CFLAGS = @LIBEE_CFLAGS@ -LIBEE_LIBS = @LIBEE_LIBS@ LIBESTR_CFLAGS = @LIBESTR_CFLAGS@ LIBESTR_LIBS = @LIBESTR_LIBS@ LIBGCRYPT_CFLAGS = @LIBGCRYPT_CFLAGS@ +LIBGCRYPT_CONFIG = @LIBGCRYPT_CONFIG@ LIBGCRYPT_LIBS = @LIBGCRYPT_LIBS@ LIBLOGGING_CFLAGS = @LIBLOGGING_CFLAGS@ LIBLOGGING_LIBS = @LIBLOGGING_LIBS@ +LIBLOGGING_STDLOG_CFLAGS = @LIBLOGGING_STDLOG_CFLAGS@ +LIBLOGGING_STDLOG_LIBS = @LIBLOGGING_STDLOG_LIBS@ LIBLOGNORM_CFLAGS = @LIBLOGNORM_CFLAGS@ LIBLOGNORM_LIBS = @LIBLOGNORM_LIBS@ LIBM = @LIBM@ @@ -206,6 +206,8 @@ NM = @NM@ NMEDIT = @NMEDIT@ OBJDUMP = @OBJDUMP@ OBJEXT = @OBJEXT@ +OPENSSL_CFLAGS = @OPENSSL_CFLAGS@ +OPENSSL_LIBS = @OPENSSL_LIBS@ ORACLE_CFLAGS = @ORACLE_CFLAGS@ ORACLE_LIBS = @ORACLE_LIBS@ OTOOL = @OTOOL@ @@ -387,22 +389,25 @@ distclean-compile: @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/omprog_la-omprog.Plo@am__quote@ .c.o: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(COMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ $< -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Po +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.o$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c $< +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ $< .c.obj: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(COMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ `$(CYGPATH_W) '$<'` -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Po +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.obj$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ `$(CYGPATH_W) '$<'` &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c `$(CYGPATH_W) '$<'` +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ `$(CYGPATH_W) '$<'` .c.lo: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(LTCOMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ $< -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Plo +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.lo$$||'`;\ +@am__fastdepCC_TRUE@ $(LTCOMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Plo @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=yes @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ @am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(LTCOMPILE) -c -o $@ $< diff --git a/plugins/omprog/omprog.c b/plugins/omprog/omprog.c index e425b42..2ba0b94 100644 --- a/plugins/omprog/omprog.c +++ b/plugins/omprog/omprog.c @@ -6,7 +6,7 @@ * * File begun on 2009-04-01 by RGerhards * - * Copyright 2009-2012 Adiscon GmbH. + * Copyright 2009-2014 Adiscon GmbH. * * This file is part of rsyslog. * @@ -27,6 +27,7 @@ #include "config.h" #include "rsyslog.h" #include <stdio.h> +#include <syslog.h> #include <stdarg.h> #include <stdlib.h> #include <string.h> @@ -34,7 +35,9 @@ #include <signal.h> #include <errno.h> #include <unistd.h> +#include <fcntl.h> #include <wait.h> +#include <pthread.h> #include "conf.h" #include "syslogd-types.h" #include "srUtils.h" @@ -54,11 +57,22 @@ DEFobjCurrIf(errmsg) typedef struct _instanceData { uchar *szBinary; /* name of binary to call */ + char **aParams; /* Optional Parameters for binary command */ uchar *tplName; /* assigned output template */ + int iParams; /* Holds the count of parameters if set*/ + int bForceSingleInst; /* only a single wrkr instance of program permitted? */ + uchar *outputFileName; /* name of file for std[out/err] or NULL if to discard */ + pthread_mutex_t mut; /* make sure only one instance is active */ +} instanceData; + +typedef struct wrkrInstanceData { + instanceData *pData; pid_t pid; /* pid of currently running process */ - int fdPipe; /* file descriptor to write to */ + int fdOutput; /* it's fd (-1 if closed) */ + int fdPipeOut; /* file descriptor to write to */ + int fdPipeIn; /* fd we receive messages from the program (if we want to) */ int bIsRunning; /* is binary currently running? 0-no, 1-yes */ -} instanceData; +} wrkrInstanceData_t; typedef struct configSettings_s { uchar *szBinary; /* name of binary to call */ @@ -70,6 +84,8 @@ static configSettings_t cs; /* action (instance) parameters */ static struct cnfparamdescr actpdescr[] = { { "binary", eCmdHdlrString, CNFPARAM_REQUIRED }, + { "output", eCmdHdlrString, 0 }, + { "forcesingleinstance", eCmdHdlrBinary, 0 }, { "template", eCmdHdlrGetWord, 0 } }; static struct cnfparamblk actpblk = @@ -87,8 +103,17 @@ ENDinitConfVars BEGINcreateInstance CODESTARTcreateInstance + pthread_mutex_init(&pData->mut, NULL); ENDcreateInstance +BEGINcreateWrkrInstance +CODESTARTcreateWrkrInstance + pWrkrData->fdPipeIn = -1; + pWrkrData->fdPipeOut = -1; + pWrkrData->fdOutput = -1; + pWrkrData->bIsRunning = 0; +ENDcreateWrkrInstance + BEGINisCompatibleWithFeature CODESTARTisCompatibleWithFeature @@ -98,11 +123,23 @@ ENDisCompatibleWithFeature BEGINfreeInstance + int i; CODESTARTfreeInstance - if(pData->szBinary != NULL) - free(pData->szBinary); + pthread_mutex_destroy(&pData->mut); + free(pData->szBinary); + free(pData->outputFileName); + if(pData->aParams != NULL) { + for (i = 0; i < pData->iParams; i++) { + free(pData->aParams[i]); + } + free(pData->aParams); + } ENDfreeInstance +BEGINfreeWrkrInstance +CODESTARTfreeWrkrInstance +ENDfreeWrkrInstance + BEGINdbgPrintInstInfo CODESTARTdbgPrintInstInfo @@ -114,27 +151,99 @@ CODESTARTtryResume ENDtryResume +/* As this is assume to be a debug function, we only make + * best effort to write the message but do *not* try very + * hard to handle errors. -- rgerhards, 2014-01-16 + */ +static void +writeProgramOutput(wrkrInstanceData_t *__restrict__ const pWrkrData, + const char *__restrict__ const buf, + const ssize_t lenBuf) +{ + char errStr[1024]; + ssize_t r; + +dbgprintf("omprog: writeProgramOutput, fd %d\n", pWrkrData->fdOutput); + if(pWrkrData->fdOutput == -1) { + pWrkrData->fdOutput = open((char*)pWrkrData->pData->outputFileName, + O_WRONLY | O_APPEND | O_CREAT, 0600); + if(pWrkrData->fdOutput == -1) { + DBGPRINTF("omprog: error opening output file %s: %s\n", + pWrkrData->pData->outputFileName, + rs_strerror_r(errno, errStr, sizeof(errStr))); + goto done; + } + } + + r = write(pWrkrData->fdOutput, buf, (size_t) lenBuf); + if(r != lenBuf) { + DBGPRINTF("omprog: problem writing output file %s: bytes " + "requested %lld, written %lld, msg: %s\n", + pWrkrData->pData->outputFileName, (long long) lenBuf, (long long) r, + rs_strerror_r(errno, errStr, sizeof(errStr))); + } +done: return; +} + + +/* check output of the executed program + * If configured to care about the output, we check if there is some and, + * if so, properly handle it. + */ +static void +checkProgramOutput(wrkrInstanceData_t *__restrict__ const pWrkrData) +{ + char buf[4096]; + ssize_t r; + +dbgprintf("omprog: checking prog output, fd %d\n", pWrkrData->fdPipeIn); + if(pWrkrData->fdPipeIn == -1) + goto done; + + do { +memset(buf, 0, sizeof(buf)); + r = read(pWrkrData->fdPipeIn, buf, sizeof(buf)); +dbgprintf("omprog: read state %lld, data '%s'\n", (long long) r, buf); + if(r > 0) + writeProgramOutput(pWrkrData, buf, r); + } while(r > 0); + +done: return; +} + + + /* execute the child process (must be called in child context * after fork). */ - -static void execBinary(instanceData *pData, int fdStdin) +static void +execBinary(wrkrInstanceData_t *pWrkrData, int fdStdin, int fdStdOutErr) { - int i; + int i, iRet; struct sigaction sigAct; - char *newargv[] = { NULL }; + sigset_t set; + char errStr[1024]; char *newenviron[] = { NULL }; - assert(pData != NULL); - fclose(stdin); if(dup(fdStdin) == -1) { - DBGPRINTF("omprog: dup() failed\n"); + DBGPRINTF("omprog: dup() stdin failed\n"); /* do some more error handling here? Maybe if the module * gets some more widespread use... */ } - //fclose(stdout); + if(pWrkrData->pData->outputFileName == NULL) { + close(fdStdOutErr); + } else { + close(1); + if(dup(fdStdOutErr) == -1) { + DBGPRINTF("omprog: dup() stdout failed\n"); + } + close(2); + if(dup(fdStdOutErr) == -1) { + DBGPRINTF("omprog: dup() stderr failed\n"); + } + } /* we close all file handles as we fork soon * Is there a better way to do this? - mail me! rgerhards@adiscon.com @@ -146,19 +255,35 @@ static void execBinary(instanceData *pData, int fdStdin) /* reset signal handlers to default */ memset(&sigAct, 0, sizeof(sigAct)); - sigfillset(&sigAct.sa_mask); + sigemptyset(&sigAct.sa_mask); sigAct.sa_handler = SIG_DFL; for(i = 1 ; i < NSIG ; ++i) sigaction(i, &sigAct, NULL); + /* we need to block SIGINT, otherwise our program is cancelled when we are + * stopped in debug mode. + */ + sigAct.sa_handler = SIG_IGN; + sigaction(SIGINT, &sigAct, NULL); + sigemptyset(&set); + sigprocmask(SIG_SETMASK, &set, NULL); alarm(0); /* finally exec child */ - execve((char*)pData->szBinary, newargv, newenviron); - /* switch to? - execlp((char*)program, (char*) program, (char*)arg, NULL); - */ - + iRet = execve((char*)pWrkrData->pData->szBinary, pWrkrData->pData->aParams, newenviron); + if(iRet == -1) { + /* Note: this will go to stdout of the **child**, so rsyslog will never + * see it except when stdout is captured. If we use the plugin interface, + * we can use this to convey a proper status back! + */ + rs_strerror_r(errno, errStr, sizeof(errStr)); + DBGPRINTF("omprog: failed to execute binary '%s': %s\n", + pWrkrData->pData->szBinary, errStr); + openlog("rsyslogd", 0, LOG_SYSLOG); + syslog(LOG_ERR, "omprog: failed to execute binary '%s': %s\n", + pWrkrData->pData->szBinary, errStr); + } + /* we should never reach this point, but if we do, we terminate */ exit(1); } @@ -168,19 +293,23 @@ static void execBinary(instanceData *pData, int fdStdin) * rgerhards, 2009-04-01 */ static rsRetVal -openPipe(instanceData *pData) +openPipe(wrkrInstanceData_t *pWrkrData) { - int pipefd[2]; + int pipestdin[2]; + int pipestdout[2]; pid_t cpid; + int flags; DEFiRet; - assert(pData != NULL); - - if(pipe(pipefd) == -1) { + if(pipe(pipestdin) == -1) { + ABORT_FINALIZE(RS_RET_ERR_CREAT_PIPE); + } + if(pipe(pipestdout) == -1) { ABORT_FINALIZE(RS_RET_ERR_CREAT_PIPE); } - DBGPRINTF("executing program '%s'\n", pData->szBinary); + DBGPRINTF("omprog: executing program '%s' with '%d' parameters\n", + pWrkrData->pData->szBinary, pWrkrData->pData->iParams); /* NO OUTPUT AFTER FORK! */ @@ -188,21 +317,31 @@ openPipe(instanceData *pData) if(cpid == -1) { ABORT_FINALIZE(RS_RET_ERR_FORK); } + pWrkrData->pid = cpid; if(cpid == 0) { - /* we are now the child, just set the right selectors and - * exec the binary. If that fails, there is not much we can do. - */ - close(pipefd[1]); - execBinary(pData, pipefd[0]); + /* we are now the child, just exec the binary. */ + close(pipestdin[1]); /* close those pipe "ports" that */ + close(pipestdout[0]); /* we don't need */ + execBinary(pWrkrData, pipestdin[0], pipestdout[1]); /*NO CODE HERE - WILL NEVER BE REACHED!*/ } - DBGPRINTF("child has pid %d\n", (int) cpid); - pData->fdPipe = pipefd[1]; - pData->pid = cpid; - close(pipefd[0]); - pData->bIsRunning = 1; + DBGPRINTF("omprog: child has pid %d\n", (int) cpid); + if(pWrkrData->pData->outputFileName != NULL) { + pWrkrData->fdPipeIn = dup(pipestdout[0]); + /* we need to set our fd to be non-blocking! */ + flags = fcntl(pWrkrData->fdPipeIn, F_GETFL); + flags |= O_NONBLOCK; + fcntl(pWrkrData->fdPipeIn, F_SETFL, flags); + } else { + pWrkrData->fdPipeIn = -1; + } + close(pipestdin[0]); + close(pipestdout[1]); + pWrkrData->pid = cpid; + pWrkrData->fdPipeOut = pipestdin[1]; + pWrkrData->bIsRunning = 1; finalize_it: RETiRet; } @@ -211,34 +350,48 @@ finalize_it: /* clean up after a terminated child */ static inline rsRetVal -cleanup(instanceData *pData) +cleanup(wrkrInstanceData_t *pWrkrData) { int status; int ret; char errStr[1024]; DEFiRet; - assert(pData != NULL); - assert(pData->bIsRunning == 1); - ret = waitpid(pData->pid, &status, 0); - if(ret != pData->pid) { + assert(pWrkrData->bIsRunning == 1); + ret = waitpid(pWrkrData->pid, &status, 0); + if(ret != pWrkrData->pid) { /* if waitpid() fails, we can not do much - try to ignore it... */ - DBGPRINTF("waitpid() returned state %d[%s], future malfunction may happen\n", ret, + DBGPRINTF("omprog: waitpid() returned state %d[%s], future malfunction may happen\n", ret, rs_strerror_r(errno, errStr, sizeof(errStr))); } else { /* check if we should print out some diagnostic information */ - DBGPRINTF("waitpid status return for program '%s': %2.2x\n", - pData->szBinary, status); + DBGPRINTF("omprog: waitpid status return for program '%s': %2.2x\n", + pWrkrData->pData->szBinary, status); if(WIFEXITED(status)) { errmsg.LogError(0, NO_ERRCODE, "program '%s' exited normally, state %d", - pData->szBinary, WEXITSTATUS(status)); + pWrkrData->pData->szBinary, WEXITSTATUS(status)); } else if(WIFSIGNALED(status)) { errmsg.LogError(0, NO_ERRCODE, "program '%s' terminated by signal %d.", - pData->szBinary, WTERMSIG(status)); + pWrkrData->pData->szBinary, WTERMSIG(status)); } } - pData->bIsRunning = 0; + checkProgramOutput(pWrkrData); /* try to catch any late messages */ + + if(pWrkrData->fdOutput != -1) { + close(pWrkrData->fdOutput); + pWrkrData->fdOutput = -1; + } + if(pWrkrData->fdPipeIn != -1) { + close(pWrkrData->fdPipeIn); + pWrkrData->fdPipeIn = -1; + } + if(pWrkrData->fdPipeOut != -1) { + close(pWrkrData->fdPipeOut); + pWrkrData->fdPipeOut = -1; + } + pWrkrData->bIsRunning = 0; + pWrkrData->bIsRunning = 0; RETiRet; } @@ -246,24 +399,22 @@ cleanup(instanceData *pData) /* try to restart the binary when it has stopped. */ static inline rsRetVal -tryRestart(instanceData *pData) +tryRestart(wrkrInstanceData_t *pWrkrData) { DEFiRet; - assert(pData != NULL); - assert(pData->bIsRunning == 0); + assert(pWrkrData->bIsRunning == 0); - iRet = openPipe(pData); + iRet = openPipe(pWrkrData); RETiRet; } - /* write to pipe * note that we do not try to run block-free. If the users fears something * may block (and this not be acceptable), the action should be run on its * own action queue. */ static rsRetVal -writePipe(instanceData *pData, uchar *szMsg) +writePipe(wrkrInstanceData_t *pWrkrData, uchar *szMsg) { int lenWritten; int lenWrite; @@ -271,33 +422,33 @@ writePipe(instanceData *pData, uchar *szMsg) char errStr[1024]; DEFiRet; - assert(pData != NULL); - lenWrite = strlen((char*)szMsg); writeOffset = 0; - do - { - lenWritten = write(pData->fdPipe, ((char*)szMsg)+writeOffset, lenWrite); + do { + checkProgramOutput(pWrkrData); +dbgprintf("omprog: writing to prog (fd %d): %s\n", pWrkrData->fdPipeOut, szMsg); + lenWritten = write(pWrkrData->fdPipeOut, ((char*)szMsg)+writeOffset, lenWrite); if(lenWritten == -1) { switch(errno) { - case EPIPE: - DBGPRINTF("Program '%s' terminated, trying to restart\n", - pData->szBinary); - CHKiRet(cleanup(pData)); - CHKiRet(tryRestart(pData)); - break; - default: - DBGPRINTF("error %d writing to pipe: %s\n", errno, - rs_strerror_r(errno, errStr, sizeof(errStr))); - ABORT_FINALIZE(RS_RET_ERR_WRITE_PIPE); - break; + case EPIPE: + DBGPRINTF("omprog: program '%s' terminated, trying to restart\n", + pWrkrData->pData->szBinary); + CHKiRet(cleanup(pWrkrData)); + CHKiRet(tryRestart(pWrkrData)); + break; + default: + DBGPRINTF("omprog: error %d writing to pipe: %s\n", errno, + rs_strerror_r(errno, errStr, sizeof(errStr))); + ABORT_FINALIZE(RS_RET_ERR_WRITE_PIPE); + break; } } else { writeOffset += lenWritten; } } while(lenWritten != lenWrite); + checkProgramOutput(pWrkrData); finalize_it: RETiRet; @@ -305,15 +456,22 @@ finalize_it: BEGINdoAction + instanceData *pData; CODESTARTdoAction - if(pData->bIsRunning == 0) { - openPipe(pData); +dbgprintf("DDDD:omprog processing message\n"); + pData = pWrkrData->pData; + if(pData->bForceSingleInst) + pthread_mutex_lock(&pData->mut); + if(pWrkrData->bIsRunning == 0) { + openPipe(pWrkrData); } - iRet = writePipe(pData, ppString[0]); + iRet = writePipe(pWrkrData, ppString[0]); if(iRet != RS_RET_OK) iRet = RS_RET_SUSPENDED; + if(pData->bForceSingleInst) + pthread_mutex_unlock(&pData->mut); ENDdoAction @@ -321,13 +479,23 @@ static inline void setInstParamDefaults(instanceData *pData) { pData->szBinary = NULL; - pData->fdPipe = -1; - pData->bIsRunning = 0; + pData->aParams = NULL; + pData->outputFileName = NULL; + pData->iParams = 0; + pData->bForceSingleInst = 0; } BEGINnewActInst struct cnfparamvals *pvals; + sbool bInQuotes; int i; + int iPrm; + unsigned char *c; + es_size_t iCnt; + es_size_t iStr; + es_str_t *estrBinary; + es_str_t *estrParams; + es_str_t *estrTmp; CODESTARTnewActInst if((pvals = nvlstGetParams(lst, &actpblk, NULL)) == NULL) { ABORT_FINALIZE(RS_RET_MISSING_CNFPARAMS); @@ -341,23 +509,96 @@ CODESTARTnewActInst if(!pvals[i].bUsed) continue; if(!strcmp(actpblk.descr[i].name, "binary")) { - pData->szBinary = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + estrBinary = pvals[i].val.d.estr; + estrParams = NULL; + + /* Search for space */ + c = es_getBufAddr(pvals[i].val.d.estr); + iCnt = 0; + while(iCnt < es_strlen(pvals[i].val.d.estr) ) { + if (c[iCnt] == ' ') { + /* Split binary name from parameters */ + estrBinary = es_newStrFromSubStr ( pvals[i].val.d.estr, 0, iCnt ); + estrParams = es_newStrFromSubStr ( pvals[i].val.d.estr, iCnt+1, es_strlen(pvals[i].val.d.estr)); + break; + } + iCnt++; + } + /* Assign binary and params */ + pData->szBinary = (uchar*)es_str2cstr(estrBinary, NULL); + dbgprintf("omprog: szBinary = '%s'\n", pData->szBinary); + /* Check for Params! */ + if (estrParams != NULL) { + dbgprintf("omprog: szParams = '%s'\n", es_str2cstr(estrParams, NULL) ); + + /* Count parameters if set */ + c = es_getBufAddr(estrParams); /* Reset to beginning */ + pData->iParams = 2; /* Set default to 2, first parameter for binary and second parameter at least from config*/ + iCnt = 0; + while(iCnt < es_strlen(estrParams) ) { + if (c[iCnt] == ' ' && c[iCnt-1] != '\\') + pData->iParams++; + iCnt++; + } + dbgprintf("omprog: iParams = '%d'\n", pData->iParams); + + /* Create argv Array */ + CHKmalloc(pData->aParams = malloc( (pData->iParams+1) * sizeof(char*))); /* One more for first param */ + + /* Second Loop, create parameter array*/ + c = es_getBufAddr(estrParams); /* Reset to beginning */ + iCnt = iStr = iPrm = 0; + estrTmp = NULL; + bInQuotes = FALSE; + /* Set first parameter to binary */ + pData->aParams[iPrm] = strdup((char*)pData->szBinary); + dbgprintf("omprog: Param (%d): '%s'\n", iPrm, pData->aParams[iPrm]); + iPrm++; + while(iCnt < es_strlen(estrParams) ) { + if ( c[iCnt] == ' ' && !bInQuotes ) { + /* Copy into Param Array! */ + estrTmp = es_newStrFromSubStr( estrParams, iStr, iCnt-iStr); + } + else if ( iCnt+1 >= es_strlen(estrParams) ) { + /* Copy rest of string into Param Array! */ + estrTmp = es_newStrFromSubStr( estrParams, iStr, iCnt-iStr+1); + } + else if (c[iCnt] == '"') { + /* switch inQuotes Mode */ + bInQuotes = !bInQuotes; + } + + if ( estrTmp != NULL ) { + pData->aParams[iPrm] = es_str2cstr(estrTmp, NULL); + iStr = iCnt+1; /* Set new start */ + dbgprintf("omprog: Param (%d): '%s'\n", iPrm, pData->aParams[iPrm]); + es_deleteStr( estrTmp ); + estrTmp = NULL; + iPrm++; + } + + /*Next char*/ + iCnt++; + } + /* NULL last parameter! */ + pData->aParams[iPrm] = NULL; + + } + } else if(!strcmp(actpblk.descr[i].name, "output")) { + pData->outputFileName = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + } else if(!strcmp(actpblk.descr[i].name, "forcesingleinstance")) { + pData->bForceSingleInst = (int) pvals[i].val.d.n; } else if(!strcmp(actpblk.descr[i].name, "template")) { pData->tplName = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); } else { - dbgprintf("omprog: program error, non-handled " - "param '%s'\n", actpblk.descr[i].name); + dbgprintf("omprog: program error, non-handled param '%s'\n", actpblk.descr[i].name); } } - if(pData->tplName == NULL) { - CHKiRet(OMSRsetEntry(*ppOMSR, 0, (uchar*) "RSYSLOG_FileFormat", - OMSR_NO_RQD_TPL_OPTS)); - } else { - CHKiRet(OMSRsetEntry(*ppOMSR, 0, - (uchar*) strdup((char*) pData->tplName), - OMSR_NO_RQD_TPL_OPTS)); - } + CHKiRet(OMSRsetEntry(*ppOMSR, 0, (uchar*)strdup((pData->tplName == NULL) ? + "RSYSLOG_FileFormat" : (char*)pData->tplName), + OMSR_NO_RQD_TPL_OPTS)); + DBGPRINTF("omprog: bForceSingleInst %d\n", pData->bForceSingleInst); CODE_STD_FINALIZERnewActInst cnfparamvalsDestruct(pvals, &actpblk); ENDnewActInst @@ -407,6 +648,7 @@ ENDmodExit BEGINqueryEtryPt CODESTARTqueryEtryPt CODEqueryEtryPt_STD_OMOD_QUERIES +CODEqueryEtryPt_STD_OMOD8_QUERIES CODEqueryEtryPt_STD_CONF2_CNFNAME_QUERIES CODEqueryEtryPt_STD_CONF2_OMOD_QUERIES ENDqueryEtryPt diff --git a/plugins/omrabbitmq/Makefile.in b/plugins/omrabbitmq/Makefile.in index c1df8bd..482e784 100644 --- a/plugins/omrabbitmq/Makefile.in +++ b/plugins/omrabbitmq/Makefile.in @@ -154,7 +154,6 @@ GREP = @GREP@ GSS_LIBS = @GSS_LIBS@ GUARDTIME_CFLAGS = @GUARDTIME_CFLAGS@ GUARDTIME_LIBS = @GUARDTIME_LIBS@ -HAVE_LIBGCRYPT_CONFIG = @HAVE_LIBGCRYPT_CONFIG@ HAVE_MYSQL_CONFIG = @HAVE_MYSQL_CONFIG@ HAVE_ORACLE_CONFIG = @HAVE_ORACLE_CONFIG@ HAVE_PGSQL_CONFIG = @HAVE_PGSQL_CONFIG@ @@ -175,14 +174,15 @@ LEXLIB = @LEXLIB@ LEX_OUTPUT_ROOT = @LEX_OUTPUT_ROOT@ LIBDBI_CFLAGS = @LIBDBI_CFLAGS@ LIBDBI_LIBS = @LIBDBI_LIBS@ -LIBEE_CFLAGS = @LIBEE_CFLAGS@ -LIBEE_LIBS = @LIBEE_LIBS@ LIBESTR_CFLAGS = @LIBESTR_CFLAGS@ LIBESTR_LIBS = @LIBESTR_LIBS@ LIBGCRYPT_CFLAGS = @LIBGCRYPT_CFLAGS@ +LIBGCRYPT_CONFIG = @LIBGCRYPT_CONFIG@ LIBGCRYPT_LIBS = @LIBGCRYPT_LIBS@ LIBLOGGING_CFLAGS = @LIBLOGGING_CFLAGS@ LIBLOGGING_LIBS = @LIBLOGGING_LIBS@ +LIBLOGGING_STDLOG_CFLAGS = @LIBLOGGING_STDLOG_CFLAGS@ +LIBLOGGING_STDLOG_LIBS = @LIBLOGGING_STDLOG_LIBS@ LIBLOGNORM_CFLAGS = @LIBLOGNORM_CFLAGS@ LIBLOGNORM_LIBS = @LIBLOGNORM_LIBS@ LIBM = @LIBM@ @@ -207,6 +207,8 @@ NM = @NM@ NMEDIT = @NMEDIT@ OBJDUMP = @OBJDUMP@ OBJEXT = @OBJEXT@ +OPENSSL_CFLAGS = @OPENSSL_CFLAGS@ +OPENSSL_LIBS = @OPENSSL_LIBS@ ORACLE_CFLAGS = @ORACLE_CFLAGS@ ORACLE_LIBS = @ORACLE_LIBS@ OTOOL = @OTOOL@ @@ -388,22 +390,25 @@ distclean-compile: @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/omrabbitmq_la-omrabbitmq.Plo@am__quote@ .c.o: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(COMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ $< -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Po +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.o$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c $< +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ $< .c.obj: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(COMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ `$(CYGPATH_W) '$<'` -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Po +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.obj$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ `$(CYGPATH_W) '$<'` &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c `$(CYGPATH_W) '$<'` +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ `$(CYGPATH_W) '$<'` .c.lo: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(LTCOMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ $< -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Plo +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.lo$$||'`;\ +@am__fastdepCC_TRUE@ $(LTCOMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Plo @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=yes @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ @am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(LTCOMPILE) -c -o $@ $< diff --git a/plugins/omrabbitmq/omrabbitmq.c b/plugins/omrabbitmq/omrabbitmq.c index 7ea7793..8ea7e62 100644 --- a/plugins/omrabbitmq/omrabbitmq.c +++ b/plugins/omrabbitmq/omrabbitmq.c @@ -3,6 +3,7 @@ * This output plugin enables rsyslog to send messages to the RabbitMQ. * * Copyright 2012-2013 Vaclav Tomec + * Copyright 2014 Rainer Gerhards * * This program is free software: you can redistribute it and/or * modify it under the terms of the GNU Lesser General Public License @@ -52,6 +53,7 @@ MODULE_CNFNAME("omrabbitmq") DEF_OMOD_STATIC_DATA DEFobjCurrIf(errmsg) +static pthread_mutex_t mutDoAct = PTHREAD_MUTEX_INITIALIZER; typedef struct _instanceData { /* here you need to define all action-specific data. A record of type @@ -72,6 +74,10 @@ typedef struct _instanceData { uchar *tplName; } instanceData; +typedef struct wrkrInstanceData { + instanceData *pData; +} wrkrInstanceData_t; + /* tables for interfacing with the v6 config system */ /* action (instance) parameters */ @@ -226,6 +232,11 @@ CODESTARTcreateInstance ENDcreateInstance +BEGINcreateWrkrInstance +CODESTARTcreateWrkrInstance +ENDcreateWrkrInstance + + BEGINisCompatibleWithFeature CODESTARTisCompatibleWithFeature /* use this to specify if select features are supported by this @@ -254,6 +265,10 @@ CODESTARTfreeInstance ENDfreeInstance +BEGINfreeWrkrInstance +CODESTARTfreeWrkrInstance +ENDfreeWrkrInstance + BEGINdbgPrintInstInfo CODESTARTdbgPrintInstInfo /* permits to spit out some debug info */ @@ -270,6 +285,7 @@ ENDdbgPrintInstInfo BEGINtryResume + instanceData *pData = pWrkrData->pData; CODESTARTtryResume /* this is called when an action has been suspended and the * rsyslog core tries to resume it. The action must then @@ -293,14 +309,17 @@ CODESTARTtryResume * not always be the case. */ + pthread_mutex_lock(&mutDoAct); if (pData->conn == NULL) { iRet = initRabbitMQ(pData); } + pthread_mutex_unlock(&mutDoAct); ENDtryResume BEGINdoAction + instanceData *pData = pWrkrData->pData; CODESTARTdoAction /* this is where you receive the message and need to carry out the * action. Data is provided in ppString[i] where 0 <= i <= num of strings @@ -315,6 +334,7 @@ CODESTARTdoAction amqp_bytes_t body_bytes; + pthread_mutex_lock(&mutDoAct); if (pData->conn == NULL) { CHKiRet(initRabbitMQ(pData)); } @@ -330,7 +350,7 @@ CODESTARTdoAction } finalize_it: - + pthread_mutex_unlock(&mutDoAct); ENDdoAction @@ -455,6 +475,7 @@ BEGINqueryEtryPt CODESTARTqueryEtryPt CODEqueryEtryPt_STD_OMOD_QUERIES CODEqueryEtryPt_STD_CONF2_OMOD_QUERIES + CODEqueryEtryPt_STD_OMOD8_QUERIES ENDqueryEtryPt diff --git a/plugins/omrelp/Makefile.in b/plugins/omrelp/Makefile.in index 6c4f93d..31ca744 100644 --- a/plugins/omrelp/Makefile.in +++ b/plugins/omrelp/Makefile.in @@ -154,7 +154,6 @@ GREP = @GREP@ GSS_LIBS = @GSS_LIBS@ GUARDTIME_CFLAGS = @GUARDTIME_CFLAGS@ GUARDTIME_LIBS = @GUARDTIME_LIBS@ -HAVE_LIBGCRYPT_CONFIG = @HAVE_LIBGCRYPT_CONFIG@ HAVE_MYSQL_CONFIG = @HAVE_MYSQL_CONFIG@ HAVE_ORACLE_CONFIG = @HAVE_ORACLE_CONFIG@ HAVE_PGSQL_CONFIG = @HAVE_PGSQL_CONFIG@ @@ -175,14 +174,15 @@ LEXLIB = @LEXLIB@ LEX_OUTPUT_ROOT = @LEX_OUTPUT_ROOT@ LIBDBI_CFLAGS = @LIBDBI_CFLAGS@ LIBDBI_LIBS = @LIBDBI_LIBS@ -LIBEE_CFLAGS = @LIBEE_CFLAGS@ -LIBEE_LIBS = @LIBEE_LIBS@ LIBESTR_CFLAGS = @LIBESTR_CFLAGS@ LIBESTR_LIBS = @LIBESTR_LIBS@ LIBGCRYPT_CFLAGS = @LIBGCRYPT_CFLAGS@ +LIBGCRYPT_CONFIG = @LIBGCRYPT_CONFIG@ LIBGCRYPT_LIBS = @LIBGCRYPT_LIBS@ LIBLOGGING_CFLAGS = @LIBLOGGING_CFLAGS@ LIBLOGGING_LIBS = @LIBLOGGING_LIBS@ +LIBLOGGING_STDLOG_CFLAGS = @LIBLOGGING_STDLOG_CFLAGS@ +LIBLOGGING_STDLOG_LIBS = @LIBLOGGING_STDLOG_LIBS@ LIBLOGNORM_CFLAGS = @LIBLOGNORM_CFLAGS@ LIBLOGNORM_LIBS = @LIBLOGNORM_LIBS@ LIBM = @LIBM@ @@ -207,6 +207,8 @@ NM = @NM@ NMEDIT = @NMEDIT@ OBJDUMP = @OBJDUMP@ OBJEXT = @OBJEXT@ +OPENSSL_CFLAGS = @OPENSSL_CFLAGS@ +OPENSSL_LIBS = @OPENSSL_LIBS@ ORACLE_CFLAGS = @ORACLE_CFLAGS@ ORACLE_LIBS = @ORACLE_LIBS@ OTOOL = @OTOOL@ @@ -387,22 +389,25 @@ distclean-compile: @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/omrelp_la-omrelp.Plo@am__quote@ .c.o: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(COMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ $< -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Po +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.o$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c $< +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ $< .c.obj: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(COMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ `$(CYGPATH_W) '$<'` -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Po +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.obj$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ `$(CYGPATH_W) '$<'` &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c `$(CYGPATH_W) '$<'` +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ `$(CYGPATH_W) '$<'` .c.lo: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(LTCOMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ $< -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Plo +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.lo$$||'`;\ +@am__fastdepCC_TRUE@ $(LTCOMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Plo @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=yes @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ @am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(LTCOMPILE) -c -o $@ $< diff --git a/plugins/omrelp/omrelp.c b/plugins/omrelp/omrelp.c index c9e3244..d7d9374 100644 --- a/plugins/omrelp/omrelp.c +++ b/plugins/omrelp/omrelp.c @@ -2,12 +2,21 @@ * * This is the implementation of the RELP output module. * - * NOTE: read comments in module-template.h to understand how this file - * works! + * Note that when multiple action workers are activated, we currently + * also create multiple actions. This may be the source of some mild + * message loss (!) if the worker instance is shut down while the + * connection to the remote system is in retry state. + * TODO: think if we should implement a mode where we do NOT + * support multiple action worker instances. This would be + * slower, but not have this loss opportunity. But it should + * definitely be optional and by default off due to the + * performance implications (and given the fact that message + * loss is pretty unlikely in usual cases). + * * * File begun on 2008-03-13 by RGerhards * - * Copyright 2008-2013 Adiscon GmbH. + * Copyright 2008-2014 Adiscon GmbH. * * This file is part of rsyslog. * @@ -55,32 +64,64 @@ DEF_OMOD_STATIC_DATA DEFobjCurrIf(errmsg) DEFobjCurrIf(glbl) +#define DFLT_ENABLE_TLS 0 +#define DFLT_ENABLE_TLSZIP 0 + static relpEngine_t *pRelpEngine; /* our relp engine */ typedef struct _instanceData { uchar *target; - int compressionLevel; /* 0 - no compression, else level for zlib */ uchar *port; - int bInitialConnect; /* is this the initial connection request of our module? (0-no, 1-yes) */ - int bIsConnected; /* currently connected to server? 0 - no, 1 - yes */ + int sizeWindow; /**< the RELP window size - 0=use default */ unsigned timeout; - relpClt_t *pRelpClt; /* relp client for this instance */ + unsigned rebindInterval; + sbool bEnableTLS; + sbool bEnableTLSZip; + sbool bHadAuthFail; /**< set on auth failure, will cause retry to disable action */ + uchar *pristring; /* GnuTLS priority string (NULL if not to be provided) */ + uchar *authmode; + uchar *caCertFile; + uchar *myCertFile; + uchar *myPrivKeyFile; uchar *tplName; + struct { + int nmemb; + uchar **name; + } permittedPeers; } instanceData; +typedef struct wrkrInstanceData { + instanceData *pData; + int bInitialConnect; /* is this the initial connection request of our module? (0-no, 1-yes) */ + int bIsConnected; /* currently connected to server? 0 - no, 1 - yes */ + relpClt_t *pRelpClt; /* relp client for this instance */ + unsigned nSent; /* number msgs sent - for rebind support */ +} wrkrInstanceData_t; + typedef struct configSettings_s { EMPTY_STRUCT } configSettings_t; static configSettings_t __attribute__((unused)) cs; +static rsRetVal doCreateRelpClient(wrkrInstanceData_t *pWrkrData); /* tables for interfacing with the v6 config system */ /* action (instance) parameters */ static struct cnfparamdescr actpdescr[] = { { "target", eCmdHdlrGetWord, 1 }, + { "tls", eCmdHdlrBinary, 0 }, + { "tls.compression", eCmdHdlrBinary, 0 }, + { "tls.prioritystring", eCmdHdlrString, 0 }, + { "tls.cacert", eCmdHdlrString, 0 }, + { "tls.mycert", eCmdHdlrString, 0 }, + { "tls.myprivkey", eCmdHdlrString, 0 }, + { "tls.authmode", eCmdHdlrString, 0 }, + { "tls.permittedpeer", eCmdHdlrArray, 0 }, { "port", eCmdHdlrGetWord, 0 }, + { "rebindinterval", eCmdHdlrInt, 0 }, + { "windowsize", eCmdHdlrInt, 0 }, { "timeout", eCmdHdlrInt, 0 }, - { "template", eCmdHdlrGetWord, 1 } + { "template", eCmdHdlrGetWord, 0 } }; static struct cnfparamblk actpblk = { CNFPARAMBLK_VERSION, @@ -105,33 +146,126 @@ static uchar *getRelpPt(instanceData *pData) return(pData->port); } -static inline rsRetVal -doCreateRelpClient(instanceData *pData) +static void +onErr(void *pUsr, char *objinfo, char* errmesg, __attribute__((unused)) relpRetVal errcode) +{ + wrkrInstanceData_t *pWrkrData = (wrkrInstanceData_t*) pUsr; + errmsg.LogError(0, RS_RET_RELP_AUTH_FAIL, "omrelp[%s:%s]: error '%s', object " + " '%s' - action may not work as intended", + pWrkrData->pData->target, pWrkrData->pData->port, errmesg, objinfo); +} + +static void +onGenericErr(char *objinfo, char* errmesg, __attribute__((unused)) relpRetVal errcode) +{ + errmsg.LogError(0, RS_RET_RELP_ERR, "omrelp: librelp error '%s', object " + "'%s' - action may not work as intended", + errmesg, objinfo); +} + +static void +onAuthErr(void *pUsr, char *authinfo, char* errmesg, __attribute__((unused)) relpRetVal errcode) +{ + instanceData *pData = ((wrkrInstanceData_t*) pUsr)->pData; + errmsg.LogError(0, RS_RET_RELP_AUTH_FAIL, "omrelp[%s:%s]: authentication error '%s', peer " + "is '%s' - DISABLING action", pData->target, pData->port, errmesg, authinfo); + pData->bHadAuthFail = 1; +} + +static rsRetVal +doCreateRelpClient(wrkrInstanceData_t *pWrkrData) { + int i; + instanceData *pData; DEFiRet; - if(relpEngineCltConstruct(pRelpEngine, &pData->pRelpClt) != RELP_RET_OK) + + pData = pWrkrData->pData; + if(relpEngineCltConstruct(pRelpEngine, &pWrkrData->pRelpClt) != RELP_RET_OK) + ABORT_FINALIZE(RS_RET_RELP_ERR); + if(relpCltSetTimeout(pWrkrData->pRelpClt, pData->timeout) != RELP_RET_OK) + ABORT_FINALIZE(RS_RET_RELP_ERR); + if(relpCltSetWindowSize(pWrkrData->pRelpClt, pData->sizeWindow) != RELP_RET_OK) ABORT_FINALIZE(RS_RET_RELP_ERR); - if(relpCltSetTimeout(pData->pRelpClt, pData->timeout) != RELP_RET_OK) + if(relpCltSetUsrPtr(pWrkrData->pRelpClt, pWrkrData) != RELP_RET_OK) ABORT_FINALIZE(RS_RET_RELP_ERR); + if(pData->bEnableTLS) { + if(relpCltEnableTLS(pWrkrData->pRelpClt) != RELP_RET_OK) + ABORT_FINALIZE(RS_RET_RELP_ERR); + if(pData->bEnableTLSZip) { + if(relpCltEnableTLSZip(pWrkrData->pRelpClt) != RELP_RET_OK) + ABORT_FINALIZE(RS_RET_RELP_ERR); + } + if(relpCltSetGnuTLSPriString(pWrkrData->pRelpClt, (char*) pData->pristring) != RELP_RET_OK) + ABORT_FINALIZE(RS_RET_RELP_ERR); + if(relpCltSetAuthMode(pWrkrData->pRelpClt, (char*) pData->authmode) != RELP_RET_OK) { + errmsg.LogError(0, RS_RET_RELP_ERR, + "omrelp: invalid auth mode '%s'\n", pData->authmode); + ABORT_FINALIZE(RS_RET_RELP_ERR); + } + if(relpCltSetCACert(pWrkrData->pRelpClt, (char*) pData->caCertFile) != RELP_RET_OK) + ABORT_FINALIZE(RS_RET_RELP_ERR); + if(relpCltSetOwnCert(pWrkrData->pRelpClt, (char*) pData->myCertFile) != RELP_RET_OK) + ABORT_FINALIZE(RS_RET_RELP_ERR); + if(relpCltSetPrivKey(pWrkrData->pRelpClt, (char*) pData->myPrivKeyFile) != RELP_RET_OK) + ABORT_FINALIZE(RS_RET_RELP_ERR); + for(i = 0 ; i < pData->permittedPeers.nmemb ; ++i) { + relpCltAddPermittedPeer(pWrkrData->pRelpClt, (char*)pData->permittedPeers.name[i]); + } + } + if(glbl.GetSourceIPofLocalClient() == NULL) { /* ar Do we have a client IP set? */ + if(relpCltSetClientIP(pWrkrData->pRelpClt, glbl.GetSourceIPofLocalClient()) != RELP_RET_OK) + ABORT_FINALIZE(RS_RET_RELP_ERR); + } + pWrkrData->bInitialConnect = 1; + pWrkrData->nSent = 0; finalize_it: RETiRet; } - BEGINcreateInstance CODESTARTcreateInstance - pData->bInitialConnect = 1; + pData->sizeWindow = 0; + pData->timeout = 90; + pData->rebindInterval = 0; + pData->bEnableTLS = DFLT_ENABLE_TLS; + pData->bEnableTLSZip = DFLT_ENABLE_TLSZIP; + pData->bHadAuthFail = 0; + pData->pristring = NULL; + pData->authmode = NULL; + pData->caCertFile = NULL; + pData->myCertFile = NULL; + pData->myPrivKeyFile = NULL; + pData->permittedPeers.nmemb = 0; ENDcreateInstance +BEGINcreateWrkrInstance +CODESTARTcreateWrkrInstance + pWrkrData->pRelpClt = NULL; + iRet = doCreateRelpClient(pWrkrData); +ENDcreateWrkrInstance + BEGINfreeInstance + int i; CODESTARTfreeInstance - if(pData->pRelpClt != NULL) - relpEngineCltDestruct(pRelpEngine, &pData->pRelpClt); free(pData->target); free(pData->port); free(pData->tplName); + free(pData->pristring); + free(pData->authmode); + free(pData->caCertFile); + free(pData->myCertFile); + free(pData->myPrivKeyFile); + for(i = 0 ; i < pData->permittedPeers.nmemb ; ++i) { + free(pData->permittedPeers.name[i]); + } ENDfreeInstance +BEGINfreeWrkrInstance +CODESTARTfreeWrkrInstance + if(pWrkrData->pRelpClt != NULL) + relpEngineCltDestruct(pRelpEngine, &pWrkrData->pRelpClt); +ENDfreeWrkrInstance + static inline void setInstParamDefaults(instanceData *pData) { @@ -139,12 +273,22 @@ setInstParamDefaults(instanceData *pData) pData->port = NULL; pData->tplName = NULL; pData->timeout = 90; + pData->sizeWindow = 0; + pData->rebindInterval = 0; + pData->bEnableTLS = DFLT_ENABLE_TLS; + pData->bEnableTLSZip = DFLT_ENABLE_TLSZIP; + pData->pristring = NULL; + pData->authmode = NULL; + pData->caCertFile = NULL; + pData->myCertFile = NULL; + pData->myPrivKeyFile = NULL; + pData->permittedPeers.nmemb = 0; } BEGINnewActInst struct cnfparamvals *pvals; - int i; + int i,j; CODESTARTnewActInst if((pvals = nvlstGetParams(lst, &actpblk, NULL)) == NULL) { ABORT_FINALIZE(RS_RET_MISSING_CNFPARAMS); @@ -164,6 +308,31 @@ CODESTARTnewActInst pData->tplName = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); } else if(!strcmp(actpblk.descr[i].name, "timeout")) { pData->timeout = (unsigned) pvals[i].val.d.n; + } else if(!strcmp(actpblk.descr[i].name, "rebindinterval")) { + pData->rebindInterval = (unsigned) pvals[i].val.d.n; + } else if(!strcmp(actpblk.descr[i].name, "windowsize")) { + pData->sizeWindow = (int) pvals[i].val.d.n; + } else if(!strcmp(actpblk.descr[i].name, "tls")) { + pData->bEnableTLS = (unsigned) pvals[i].val.d.n; + } else if(!strcmp(actpblk.descr[i].name, "tls.compression")) { + pData->bEnableTLSZip = (unsigned) pvals[i].val.d.n; + } else if(!strcmp(actpblk.descr[i].name, "tls.prioritystring")) { + pData->pristring = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + } else if(!strcmp(actpblk.descr[i].name, "tls.cacert")) { + pData->caCertFile = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + } else if(!strcmp(actpblk.descr[i].name, "tls.mycert")) { + pData->myCertFile = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + } else if(!strcmp(actpblk.descr[i].name, "tls.myprivkey")) { + pData->myPrivKeyFile = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + } else if(!strcmp(actpblk.descr[i].name, "tls.authmode")) { + pData->authmode = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + } else if(!strcmp(actpblk.descr[i].name, "tls.permittedpeer")) { + pData->permittedPeers.nmemb = pvals[i].val.d.ar->nmemb; + CHKmalloc(pData->permittedPeers.name = + malloc(sizeof(uchar*) * pData->permittedPeers.nmemb)); + for(j = 0 ; j < pvals[i].val.d.ar->nmemb ; ++j) { + pData->permittedPeers.name[j] = (uchar*)es_str2cstr(pvals[i].val.d.ar->arr[j], NULL); + } } else { dbgprintf("omrelp: program error, non-handled " "param '%s'\n", actpblk.descr[i].name); @@ -176,10 +345,9 @@ CODESTARTnewActInst "RSYSLOG_ForwardFormat" : (char*)pData->tplName), OMSR_NO_RQD_TPL_OPTS)); - CHKiRet(doCreateRelpClient(pData)); - CODE_STD_FINALIZERnewActInst - cnfparamvalsDestruct(pvals, &actpblk); + if(pvals != NULL) + cnfparamvalsDestruct(pvals, &actpblk); ENDnewActInst BEGINisCompatibleWithFeature @@ -204,65 +372,119 @@ ENDdbgPrintInstInfo /* try to connect to server * rgerhards, 2008-03-21 */ -static rsRetVal doConnect(instanceData *pData) +static rsRetVal doConnect(wrkrInstanceData_t *pWrkrData) { DEFiRet; - if(pData->bInitialConnect) { - iRet = relpCltConnect(pData->pRelpClt, glbl.GetDefPFFamily(), pData->port, pData->target); + if(pWrkrData->bInitialConnect) { + iRet = relpCltConnect(pWrkrData->pRelpClt, glbl.GetDefPFFamily(), + pWrkrData->pData->port, pWrkrData->pData->target); if(iRet == RELP_RET_OK) - pData->bInitialConnect = 0; + pWrkrData->bInitialConnect = 0; } else { - iRet = relpCltReconnect(pData->pRelpClt); + iRet = relpCltReconnect(pWrkrData->pRelpClt); } if(iRet == RELP_RET_OK) { - pData->bIsConnected = 1; + pWrkrData->bIsConnected = 1; + } else if(iRet == RELP_RET_ERR_NO_TLS) { + errmsg.LogError(0, RS_RET_RELP_NO_TLS, "Could not connect, librelp does NOT " + "does not support TLS (most probably GnuTLS lib " + "is too old)!"); + ABORT_FINALIZE(RS_RET_RELP_NO_TLS); } else { - pData->bIsConnected = 0; + pWrkrData->bIsConnected = 0; iRet = RS_RET_SUSPENDED; } +finalize_it: RETiRet; } BEGINtryResume CODESTARTtryResume - iRet = doConnect(pData); + if(pWrkrData->pData->bHadAuthFail) { + ABORT_FINALIZE(RS_RET_DISABLE_ACTION); + } + iRet = doConnect(pWrkrData); +finalize_it: ENDtryResume +static inline rsRetVal +doRebind(wrkrInstanceData_t *pWrkrData) +{ + DEFiRet; + DBGPRINTF("omrelp: destructing relp client due to rebindInterval\n"); + CHKiRet(relpEngineCltDestruct(pRelpEngine, &pWrkrData->pRelpClt)); + pWrkrData->bIsConnected = 0; + CHKiRet(doCreateRelpClient(pWrkrData)); +finalize_it: + RETiRet; +} + +BEGINbeginTransaction +CODESTARTbeginTransaction +dbgprintf("omrelp: beginTransaction\n"); + if(!pWrkrData->bIsConnected) { + CHKiRet(doConnect(pWrkrData)); + } + relpCltHintBurstBegin(pWrkrData->pRelpClt); +finalize_it: +ENDbeginTransaction BEGINdoAction uchar *pMsg; /* temporary buffering */ size_t lenMsg; relpRetVal ret; + instanceData *pData; CODESTARTdoAction + pData = pWrkrData->pData; dbgprintf(" %s:%s/RELP\n", pData->target, getRelpPt(pData)); - if(!pData->bIsConnected) { - CHKiRet(doConnect(pData)); + if(!pWrkrData->bIsConnected) { + CHKiRet(doConnect(pWrkrData)); } pMsg = ppString[0]; lenMsg = strlen((char*) pMsg); /* TODO: don't we get this? */ - /* TODO: think about handling oversize messages! */ + /* we need to truncate oversize msgs - no way around that... */ if((int) lenMsg > glbl.GetMaxLine()) lenMsg = glbl.GetMaxLine(); /* forward */ - ret = relpCltSendSyslog(pData->pRelpClt, (uchar*) pMsg, lenMsg); + ret = relpCltSendSyslog(pWrkrData->pRelpClt, (uchar*) pMsg, lenMsg); if(ret != RELP_RET_OK) { /* error! */ dbgprintf("error forwarding via relp, suspending\n"); - iRet = RS_RET_SUSPENDED; + ABORT_FINALIZE(RS_RET_SUSPENDED); } + if(pData->rebindInterval != 0 && + (++pWrkrData->nSent >= pData->rebindInterval)) { + doRebind(pWrkrData); + } finalize_it: + if(pData->bHadAuthFail) + iRet = RS_RET_DISABLE_ACTION; + if(iRet == RS_RET_OK) { + /* we mimic non-commit, as otherwise our endTransaction handler + * will not get called. While this is not 100% correct, the worst + * that can happen is some message duplication, something that + * rsyslog generally accepts and prefers over message loss. + */ + iRet = RS_RET_PREVIOUS_COMMITTED; + } ENDdoAction +BEGINendTransaction +CODESTARTendTransaction + dbgprintf("omrelp: endTransaction\n"); + relpCltHintBurstEnd(pWrkrData->pRelpClt); +ENDendTransaction + BEGINparseSelectorAct uchar *q; int i; @@ -279,62 +501,6 @@ CODE_STD_STRING_REQUESTparseSelectorAct(1) if((iRet = createInstance(&pData)) != RS_RET_OK) FINALIZE; - /* we are now after the protocol indicator. Now check if we should - * use compression. We begin to use a new option format for this: - * @(option,option)host:port - * The first option defined is "z[0..9]" where the digit indicates - * the compression level. If it is not given, 9 (best compression) is - * assumed. An example action statement might be: - * :omrelp:(z5,o)127.0.0.1:1400 - * Which means send via TCP with medium (5) compresion (z) to the local - * host on port 1400. The '0' option means that octet-couting (as in - * IETF I-D syslog-transport-tls) is to be used for framing (this option - * applies to TCP-based syslog only and is ignored when specified with UDP). - * That is not yet implemented. - * rgerhards, 2006-12-07 - * TODO: think of all this in spite of RELP -- rgerhards, 2008-03-13 - */ - if(*p == '(') { - /* at this position, it *must* be an option indicator */ - do { - ++p; /* eat '(' or ',' (depending on when called) */ - /* check options */ - if(*p == 'z') { /* compression */ -# ifdef USE_NETZIP - ++p; /* eat */ - if(isdigit((int) *p)) { - int iLevel; - iLevel = *p - '0'; - ++p; /* eat */ - pData->compressionLevel = iLevel; - } else { - errmsg.LogError(0, NO_ERRCODE, "Invalid compression level '%c' specified in " - "forwardig action - NOT turning on compression.", - *p); - } -# else - errmsg.LogError(0, NO_ERRCODE, "Compression requested, but rsyslogd is not compiled " - "with compression support - request ignored."); -# endif /* #ifdef USE_NETZIP */ - } else { /* invalid option! Just skip it... */ - errmsg.LogError(0, NO_ERRCODE, "Invalid option %c in forwarding action - ignoring.", *p); - ++p; /* eat invalid option */ - } - /* the option processing is done. We now do a generic skip - * to either the next option or the end of the option - * block. - */ - while(*p && *p != ')' && *p != ',') - ++p; /* just skip it */ - } while(*p && *p == ','); /* Attention: do.. while() */ - if(*p == ')') - ++p; /* eat terminator, on to next */ - else - /* we probably have end of string - leave it for the rest - * of the code to handle it (but warn the user) - */ - errmsg.LogError(0, NO_ERRCODE, "Option block not terminated in forwarding action."); - } /* extract the host first (we do a trick - we replace the ';' or ':' with a '\0') * now skip to port and then template name. rgerhards 2005-07-06 */ @@ -384,7 +550,6 @@ CODE_STD_STRING_REQUESTparseSelectorAct(1) ++p; } - /* TODO: make this if go away! */ if(*p == ';') { *p = '\0'; /* trick to obtain hostname (later)! */ CHKmalloc(pData->target = ustrdup(q)); @@ -396,8 +561,6 @@ CODE_STD_STRING_REQUESTparseSelectorAct(1) /* process template */ CHKiRet(cflineParseTemplateName(&p, *ppOMSR, 0, OMSR_NO_RQD_TPL_OPTS, (uchar*) "RSYSLOG_ForwardFormat")); - CHKiRet(doCreateRelpClient(pData)); - CODE_STD_FINALIZERparseSelectorAct ENDparseSelectorAct @@ -415,8 +578,10 @@ ENDmodExit BEGINqueryEtryPt CODESTARTqueryEtryPt CODEqueryEtryPt_STD_OMOD_QUERIES +CODEqueryEtryPt_STD_OMOD8_QUERIES CODEqueryEtryPt_STD_CONF2_CNFNAME_QUERIES CODEqueryEtryPt_STD_CONF2_OMOD_QUERIES +CODEqueryEtryPt_TXIF_OMOD_QUERIES CODEqueryEtryPt_SetShutdownImmdtPtr ENDqueryEtryPt @@ -429,12 +594,12 @@ CODEmodInit_QueryRegCFSLineHdlr /* create our relp engine */ CHKiRet(relpEngineConstruct(&pRelpEngine)); CHKiRet(relpEngineSetDbgprint(pRelpEngine, dbgprintf)); + CHKiRet(relpEngineSetOnAuthErr(pRelpEngine, onAuthErr)); + CHKiRet(relpEngineSetOnGenericErr(pRelpEngine, onGenericErr)); + CHKiRet(relpEngineSetOnErr(pRelpEngine, onErr)); CHKiRet(relpEngineSetEnableCmd(pRelpEngine, (uchar*) "syslog", eRelpCmdState_Required)); /* tell which objects we need */ CHKiRet(objUse(errmsg, CORE_COMPONENT)); CHKiRet(objUse(glbl, CORE_COMPONENT)); ENDmodInit - -/* vim:set ai: - */ diff --git a/plugins/omruleset/Makefile.in b/plugins/omruleset/Makefile.in index e48a786..81cb87a 100644 --- a/plugins/omruleset/Makefile.in +++ b/plugins/omruleset/Makefile.in @@ -153,7 +153,6 @@ GREP = @GREP@ GSS_LIBS = @GSS_LIBS@ GUARDTIME_CFLAGS = @GUARDTIME_CFLAGS@ GUARDTIME_LIBS = @GUARDTIME_LIBS@ -HAVE_LIBGCRYPT_CONFIG = @HAVE_LIBGCRYPT_CONFIG@ HAVE_MYSQL_CONFIG = @HAVE_MYSQL_CONFIG@ HAVE_ORACLE_CONFIG = @HAVE_ORACLE_CONFIG@ HAVE_PGSQL_CONFIG = @HAVE_PGSQL_CONFIG@ @@ -174,14 +173,15 @@ LEXLIB = @LEXLIB@ LEX_OUTPUT_ROOT = @LEX_OUTPUT_ROOT@ LIBDBI_CFLAGS = @LIBDBI_CFLAGS@ LIBDBI_LIBS = @LIBDBI_LIBS@ -LIBEE_CFLAGS = @LIBEE_CFLAGS@ -LIBEE_LIBS = @LIBEE_LIBS@ LIBESTR_CFLAGS = @LIBESTR_CFLAGS@ LIBESTR_LIBS = @LIBESTR_LIBS@ LIBGCRYPT_CFLAGS = @LIBGCRYPT_CFLAGS@ +LIBGCRYPT_CONFIG = @LIBGCRYPT_CONFIG@ LIBGCRYPT_LIBS = @LIBGCRYPT_LIBS@ LIBLOGGING_CFLAGS = @LIBLOGGING_CFLAGS@ LIBLOGGING_LIBS = @LIBLOGGING_LIBS@ +LIBLOGGING_STDLOG_CFLAGS = @LIBLOGGING_STDLOG_CFLAGS@ +LIBLOGGING_STDLOG_LIBS = @LIBLOGGING_STDLOG_LIBS@ LIBLOGNORM_CFLAGS = @LIBLOGNORM_CFLAGS@ LIBLOGNORM_LIBS = @LIBLOGNORM_LIBS@ LIBM = @LIBM@ @@ -206,6 +206,8 @@ NM = @NM@ NMEDIT = @NMEDIT@ OBJDUMP = @OBJDUMP@ OBJEXT = @OBJEXT@ +OPENSSL_CFLAGS = @OPENSSL_CFLAGS@ +OPENSSL_LIBS = @OPENSSL_LIBS@ ORACLE_CFLAGS = @ORACLE_CFLAGS@ ORACLE_LIBS = @ORACLE_LIBS@ OTOOL = @OTOOL@ @@ -387,22 +389,25 @@ distclean-compile: @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/omruleset_la-omruleset.Plo@am__quote@ .c.o: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(COMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ $< -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Po +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.o$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c $< +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ $< .c.obj: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(COMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ `$(CYGPATH_W) '$<'` -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Po +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.obj$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ `$(CYGPATH_W) '$<'` &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c `$(CYGPATH_W) '$<'` +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ `$(CYGPATH_W) '$<'` .c.lo: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(LTCOMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ $< -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Plo +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.lo$$||'`;\ +@am__fastdepCC_TRUE@ $(LTCOMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Plo @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=yes @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ @am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(LTCOMPILE) -c -o $@ $< diff --git a/plugins/omruleset/omruleset.c b/plugins/omruleset/omruleset.c index 1176550..7341991 100644 --- a/plugins/omruleset/omruleset.c +++ b/plugins/omruleset/omruleset.c @@ -10,7 +10,7 @@ * * File begun on 2009-11-02 by RGerhards * - * Copyright 2009 Rainer Gerhards and Adiscon GmbH. + * Copyright 2009-2013 Rainer Gerhards and Adiscon GmbH. * * This file is part of rsyslog. * @@ -70,6 +70,10 @@ typedef struct _instanceData { uchar *pszRulesetName; /* primarily for debugging/display purposes */ } instanceData; +typedef struct wrkrInstanceData { + instanceData *pData; +} wrkrInstanceData_t; + typedef struct configSettings_s { ruleset_t *pRuleset; /* ruleset to enqueue message to (NULL = Default, not recommended) */ uchar *pszRulesetName; @@ -87,11 +91,21 @@ CODESTARTcreateInstance ENDcreateInstance +BEGINcreateWrkrInstance +CODESTARTcreateWrkrInstance +ENDcreateWrkrInstance + + BEGINisCompatibleWithFeature CODESTARTisCompatibleWithFeature ENDisCompatibleWithFeature +BEGINfreeWrkrInstance +CODESTARTfreeWrkrInstance +ENDfreeWrkrInstance + + BEGINfreeInstance CODESTARTfreeInstance free(pData->pszRulesetName); @@ -117,9 +131,9 @@ BEGINdoAction CODESTARTdoAction CHKmalloc(pMsg = MsgDup((msg_t*) ppString[0])); DBGPRINTF(":omruleset: forwarding message %p to ruleset %s[%p]\n", pMsg, - (char*) pData->pszRulesetName, pData->pRuleset); + (char*) pWrkrData->pData->pszRulesetName, pWrkrData->pData->pRuleset); MsgSetFlowControlType(pMsg, eFLOWCTL_NO_DELAY); - MsgSetRuleset(pMsg, pData->pRuleset); + MsgSetRuleset(pMsg, pWrkrData->pData->pRuleset); /* Note: we intentionally use submitMsg2() here, as we process messages * that were already run through the rate-limiter. So it is (at least) * questionable if they were rate-limited again. @@ -199,6 +213,7 @@ ENDmodExit BEGINqueryEtryPt CODESTARTqueryEtryPt CODEqueryEtryPt_STD_OMOD_QUERIES +CODEqueryEtryPt_STD_OMOD8_QUERIES CODEqueryEtryPt_STD_CONF2_CNFNAME_QUERIES ENDqueryEtryPt diff --git a/plugins/omsnmp/Makefile.in b/plugins/omsnmp/Makefile.in index 477d4ff..8cd725e 100644 --- a/plugins/omsnmp/Makefile.in +++ b/plugins/omsnmp/Makefile.in @@ -154,7 +154,6 @@ GREP = @GREP@ GSS_LIBS = @GSS_LIBS@ GUARDTIME_CFLAGS = @GUARDTIME_CFLAGS@ GUARDTIME_LIBS = @GUARDTIME_LIBS@ -HAVE_LIBGCRYPT_CONFIG = @HAVE_LIBGCRYPT_CONFIG@ HAVE_MYSQL_CONFIG = @HAVE_MYSQL_CONFIG@ HAVE_ORACLE_CONFIG = @HAVE_ORACLE_CONFIG@ HAVE_PGSQL_CONFIG = @HAVE_PGSQL_CONFIG@ @@ -175,14 +174,15 @@ LEXLIB = @LEXLIB@ LEX_OUTPUT_ROOT = @LEX_OUTPUT_ROOT@ LIBDBI_CFLAGS = @LIBDBI_CFLAGS@ LIBDBI_LIBS = @LIBDBI_LIBS@ -LIBEE_CFLAGS = @LIBEE_CFLAGS@ -LIBEE_LIBS = @LIBEE_LIBS@ LIBESTR_CFLAGS = @LIBESTR_CFLAGS@ LIBESTR_LIBS = @LIBESTR_LIBS@ LIBGCRYPT_CFLAGS = @LIBGCRYPT_CFLAGS@ +LIBGCRYPT_CONFIG = @LIBGCRYPT_CONFIG@ LIBGCRYPT_LIBS = @LIBGCRYPT_LIBS@ LIBLOGGING_CFLAGS = @LIBLOGGING_CFLAGS@ LIBLOGGING_LIBS = @LIBLOGGING_LIBS@ +LIBLOGGING_STDLOG_CFLAGS = @LIBLOGGING_STDLOG_CFLAGS@ +LIBLOGGING_STDLOG_LIBS = @LIBLOGGING_STDLOG_LIBS@ LIBLOGNORM_CFLAGS = @LIBLOGNORM_CFLAGS@ LIBLOGNORM_LIBS = @LIBLOGNORM_LIBS@ LIBM = @LIBM@ @@ -207,6 +207,8 @@ NM = @NM@ NMEDIT = @NMEDIT@ OBJDUMP = @OBJDUMP@ OBJEXT = @OBJEXT@ +OPENSSL_CFLAGS = @OPENSSL_CFLAGS@ +OPENSSL_LIBS = @OPENSSL_LIBS@ ORACLE_CFLAGS = @ORACLE_CFLAGS@ ORACLE_LIBS = @ORACLE_LIBS@ OTOOL = @OTOOL@ @@ -387,22 +389,25 @@ distclean-compile: @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/omsnmp_la-omsnmp.Plo@am__quote@ .c.o: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(COMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ $< -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Po +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.o$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c $< +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ $< .c.obj: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(COMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ `$(CYGPATH_W) '$<'` -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Po +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.obj$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ `$(CYGPATH_W) '$<'` &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c `$(CYGPATH_W) '$<'` +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ `$(CYGPATH_W) '$<'` .c.lo: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(LTCOMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ $< -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Plo +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.lo$$||'`;\ +@am__fastdepCC_TRUE@ $(LTCOMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Plo @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=yes @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ @am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(LTCOMPILE) -c -o $@ $< diff --git a/plugins/omsnmp/omsnmp.c b/plugins/omsnmp/omsnmp.c index 79e555b..a957021 100644 --- a/plugins/omsnmp/omsnmp.c +++ b/plugins/omsnmp/omsnmp.c @@ -2,7 +2,7 @@ * * This module sends an snmp trap. * - * Copyright 2007-2012 Adiscon GmbH. + * Copyright 2007-2013 Adiscon GmbH. * * This file is part of rsyslog. * @@ -74,15 +74,19 @@ typedef struct _instanceData { * http://www.adiscon.org/download/ADISCON-MONITORWARE-MIB.txt * http://www.adiscon.org/download/ADISCON-MIB.txt */ - int iPort; /* Target Port */ - int iSNMPVersion; /* SNMP Version to use */ - int iTrapType; /* Snmp TrapType or GenericType */ - int iSpecificType; /* Snmp Specific Type */ + int iPort; /* Target Port */ + int iSNMPVersion; /* SNMP Version to use */ + int iTrapType; /* Snmp TrapType or GenericType */ + int iSpecificType; /* Snmp Specific Type */ - netsnmp_session *snmpsession; /* Holds to SNMP Session, NULL if not initialized */ - uchar *tplName; /* format template to use */ + uchar *tplName; /* format template to use */ } instanceData; +typedef struct wrkrInstanceData { + instanceData *pData; + netsnmp_session *snmpsession; /* Holds to SNMP Session, NULL if not initialized */ +} wrkrInstanceData_t; + typedef struct configSettings_s { uchar* pszTransport; /* default transport */ uchar* pszTarget; @@ -112,15 +116,15 @@ static configSettings_t cs; /* action (instance) parameters */ static struct cnfparamdescr actpdescr[] = { { "server", eCmdHdlrString, CNFPARAM_REQUIRED }, - { "port", eCmdHdlrInt, CNFPARAM_REQUIRED }, - { "transport", eCmdHdlrString, CNFPARAM_REQUIRED }, - { "version", eCmdHdlrInt, CNFPARAM_REQUIRED }, - { "community", eCmdHdlrString, CNFPARAM_REQUIRED }, - { "enterpriseoid", eCmdHdlrString, CNFPARAM_REQUIRED }, - { "trapoid", eCmdHdlrString, CNFPARAM_REQUIRED }, - { "messageoid", eCmdHdlrString, CNFPARAM_REQUIRED }, - { "traptype", eCmdHdlrInt, CNFPARAM_REQUIRED }, - { "specifictype", eCmdHdlrInt, CNFPARAM_REQUIRED }, + { "port", eCmdHdlrInt, 0 }, + { "transport", eCmdHdlrString, 0 }, + { "version", eCmdHdlrInt, 0 }, + { "community", eCmdHdlrString, 0 }, + { "enterpriseoid", eCmdHdlrString, 0 }, + { "trapoid", eCmdHdlrString, 0 }, + { "messageoid", eCmdHdlrString, 0 }, + { "traptype", eCmdHdlrInt, 0 }, + { "specifictype", eCmdHdlrInt, 0 }, { "template", eCmdHdlrGetWord, 0 } }; static struct cnfparamblk actpblk = @@ -147,6 +151,10 @@ BEGINcreateInstance CODESTARTcreateInstance ENDcreateInstance +BEGINcreateWrkrInstance +CODESTARTcreateWrkrInstance + pWrkrData->snmpsession = NULL; +ENDcreateWrkrInstance BEGINdbgPrintInstInfo CODESTARTdbgPrintInstInfo @@ -171,14 +179,16 @@ ENDisCompatibleWithFeature /* Exit SNMP Session * alorbach, 2008-02-12 */ -static rsRetVal omsnmp_exitSession(instanceData *pData) +static rsRetVal +omsnmp_exitSession(wrkrInstanceData_t *pWrkrData) { DEFiRet; - if(pData->snmpsession != NULL) { - dbgprintf( "omsnmp_exitSession: Clearing Session to '%s' on Port = '%d'\n", pData->szTarget, pData->iPort); - snmp_close(pData->snmpsession); - pData->snmpsession = NULL; + if(pWrkrData->snmpsession != NULL) { + DBGPRINTF("omsnmp_exitSession: Clearing Session to '%s' on Port = '%d'\n", + pWrkrData->pData->szTarget, pWrkrData->pData->iPort); + snmp_close(pWrkrData->snmpsession); + pWrkrData->snmpsession = NULL; } RETiRet; @@ -187,15 +197,19 @@ static rsRetVal omsnmp_exitSession(instanceData *pData) /* Init SNMP Session * alorbach, 2008-02-12 */ -static rsRetVal omsnmp_initSession(instanceData *pData) +static rsRetVal +omsnmp_initSession(wrkrInstanceData_t *pWrkrData) { netsnmp_session session; + instanceData *pData; char szTargetAndPort[MAXHOSTNAMELEN+128]; /* work buffer for specifying a full target and port string */ DEFiRet; /* should not happen, but if session is not cleared yet - we do it now! */ - if (pData->snmpsession != NULL) - omsnmp_exitSession(pData); + if (pWrkrData->snmpsession != NULL) + omsnmp_exitSession(pWrkrData); + + pData = pWrkrData->pData; snprintf((char*)szTargetAndPort, sizeof(szTargetAndPort), "%s:%s:%d", (pData->szTransport == NULL) ? "udp" : (char*)pData->szTransport, @@ -203,7 +217,8 @@ static rsRetVal omsnmp_initSession(instanceData *pData) dbgprintf( "omsnmp_initSession: ENTER - Target = '%s' on Port = '%d'\n", pData->szTarget, pData->iPort); - putenv(strdup("POSIXLY_CORRECT=1")); + if (setenv("POSIXLY_CORRECT", "1", 1) == -1) + ABORT_FINALIZE(RS_RET_ERR); snmp_sess_init(&session); session.version = pData->iSNMPVersion; @@ -217,17 +232,18 @@ static rsRetVal omsnmp_initSession(instanceData *pData) session.community_len = strlen((char*) session.community); } - pData->snmpsession = snmp_open(&session); - if (pData->snmpsession == NULL) { + pWrkrData->snmpsession = snmp_open(&session); + if (pWrkrData->snmpsession == NULL) { errmsg.LogError(0, RS_RET_SUSPENDED, "omsnmp_initSession: snmp_open to host '%s' on Port '%d' failed\n", pData->szTarget, pData->iPort); /* Stay suspended */ iRet = RS_RET_SUSPENDED; } +finalize_it: RETiRet; } -static rsRetVal omsnmp_sendsnmp(instanceData *pData, uchar *psz) +static rsRetVal omsnmp_sendsnmp(wrkrInstanceData_t *pWrkrData, uchar *psz) { DEFiRet; @@ -239,10 +255,12 @@ static rsRetVal omsnmp_sendsnmp(instanceData *pData, uchar *psz) int status; char *trap = NULL; const char *strErr = NULL; + instanceData *pData; + pData = pWrkrData->pData; /* Init SNMP Session if necessary */ - if (pData->snmpsession == NULL) { - CHKiRet(omsnmp_initSession(pData)); + if (pWrkrData->snmpsession == NULL) { + CHKiRet(omsnmp_initSession(pWrkrData)); } /* String should not be NULL */ @@ -250,7 +268,7 @@ static rsRetVal omsnmp_sendsnmp(instanceData *pData, uchar *psz) dbgprintf( "omsnmp_sendsnmp: ENTER - Syslogmessage = '%s'\n", (char*)psz); /* If SNMP Version1 is configured !*/ - if(pData->snmpsession->version == SNMP_VERSION_1) { + if(pWrkrData->snmpsession->version == SNMP_VERSION_1) { pdu = snmp_pdu_create(SNMP_MSG_TRAP); /* Set enterprise */ @@ -261,7 +279,7 @@ static rsRetVal omsnmp_sendsnmp(instanceData *pData, uchar *psz) "failed '%s' with error '%s' \n", pData->szSyslogMessageOID, strErr); ABORT_FINALIZE(RS_RET_DISABLE_ACTION); } - pdu->enterprise = (oid *) MALLOC(enterpriseoidlen * sizeof(oid)); + CHKmalloc(pdu->enterprise = (oid *) MALLOC(enterpriseoidlen * sizeof(oid))); memcpy(pdu->enterprise, enterpriseoid, enterpriseoidlen * sizeof(oid)); pdu->enterprise_length = enterpriseoidlen; @@ -275,7 +293,7 @@ static rsRetVal omsnmp_sendsnmp(instanceData *pData, uchar *psz) pdu->time = get_uptime(); } /* If SNMP Version2c is configured !*/ - else if (pData->snmpsession->version == SNMP_VERSION_2c) + else if (pWrkrData->snmpsession->version == SNMP_VERSION_2c) { long sysuptime; char csysuptime[20]; @@ -320,15 +338,15 @@ static rsRetVal omsnmp_sendsnmp(instanceData *pData, uchar *psz) } /* Send the TRAP */ - status = snmp_send(pData->snmpsession, pdu) == 0; + status = snmp_send(pWrkrData->snmpsession, pdu) == 0; if (status) { /* Debug Output! */ - int iErrorCode = pData->snmpsession->s_snmp_errno; + int iErrorCode = pWrkrData->snmpsession->s_snmp_errno; errmsg.LogError(0, RS_RET_SUSPENDED, "omsnmp_sendsnmp: snmp_send failed error '%d', Description='%s'\n", iErrorCode*(-1), api_errors[iErrorCode*(-1)]); /* Clear Session */ - omsnmp_exitSession(pData); + omsnmp_exitSession(pWrkrData); ABORT_FINALIZE(RS_RET_SUSPENDED); } @@ -347,7 +365,7 @@ finalize_it: BEGINtryResume CODESTARTtryResume - iRet = omsnmp_initSession(pData); + iRet = omsnmp_initSession(pWrkrData); ENDtryResume BEGINdoAction @@ -358,19 +376,20 @@ CODESTARTdoAction } /* This will generate and send the SNMP Trap */ - iRet = omsnmp_sendsnmp(pData, ppString[0]); + iRet = omsnmp_sendsnmp(pWrkrData, ppString[0]); finalize_it: ENDdoAction BEGINfreeInstance CODESTARTfreeInstance - /* free snmp Session here */ - omsnmp_exitSession(pData); - free(pData->tplName); free(pData->szTarget); ENDfreeInstance +BEGINfreeWrkrInstance +CODESTARTfreeWrkrInstance + omsnmp_exitSession(pWrkrData); +ENDfreeWrkrInstance static inline void setInstParamDefaults(instanceData *pData) @@ -429,14 +448,10 @@ CODESTARTnewActInst } } - if(pData->tplName == NULL) { - CHKiRet(OMSRsetEntry(*ppOMSR, 0, (uchar*) "RSYSLOG_FileFormat", - OMSR_NO_RQD_TPL_OPTS)); - } else { - CHKiRet(OMSRsetEntry(*ppOMSR, 0, - (uchar*) strdup((char*) pData->tplName), - OMSR_NO_RQD_TPL_OPTS)); - } + CHKiRet(OMSRsetEntry(*ppOMSR, 0, (uchar*)strdup((pData->tplName == NULL) ? + "RSYSLOG_FileFormat" : (char*)pData->tplName), + OMSR_NO_RQD_TPL_OPTS)); + CODE_STD_FINALIZERnewActInst cnfparamvalsDestruct(pvals, &actpblk); ENDnewActInst @@ -503,9 +518,6 @@ CODE_STD_STRING_REQUESTparseSelectorAct(1) /* Set some defaults in the NetSNMP library */ netsnmp_ds_set_int(NETSNMP_DS_LIBRARY_ID, NETSNMP_DS_LIB_DEFAULT_PORT, pData->iPort ); - - /* Init Session Pointer */ - pData->snmpsession = NULL; CODE_STD_FINALIZERparseSelectorAct ENDparseSelectorAct @@ -549,6 +561,7 @@ ENDmodExit BEGINqueryEtryPt CODESTARTqueryEtryPt CODEqueryEtryPt_STD_OMOD_QUERIES +CODEqueryEtryPt_STD_OMOD8_QUERIES CODEqueryEtryPt_STD_CONF2_CNFNAME_QUERIES CODEqueryEtryPt_STD_CONF2_OMOD_QUERIES ENDqueryEtryPt diff --git a/plugins/omstdout/Makefile.in b/plugins/omstdout/Makefile.in index c6bbf77..21ff40f 100644 --- a/plugins/omstdout/Makefile.in +++ b/plugins/omstdout/Makefile.in @@ -153,7 +153,6 @@ GREP = @GREP@ GSS_LIBS = @GSS_LIBS@ GUARDTIME_CFLAGS = @GUARDTIME_CFLAGS@ GUARDTIME_LIBS = @GUARDTIME_LIBS@ -HAVE_LIBGCRYPT_CONFIG = @HAVE_LIBGCRYPT_CONFIG@ HAVE_MYSQL_CONFIG = @HAVE_MYSQL_CONFIG@ HAVE_ORACLE_CONFIG = @HAVE_ORACLE_CONFIG@ HAVE_PGSQL_CONFIG = @HAVE_PGSQL_CONFIG@ @@ -174,14 +173,15 @@ LEXLIB = @LEXLIB@ LEX_OUTPUT_ROOT = @LEX_OUTPUT_ROOT@ LIBDBI_CFLAGS = @LIBDBI_CFLAGS@ LIBDBI_LIBS = @LIBDBI_LIBS@ -LIBEE_CFLAGS = @LIBEE_CFLAGS@ -LIBEE_LIBS = @LIBEE_LIBS@ LIBESTR_CFLAGS = @LIBESTR_CFLAGS@ LIBESTR_LIBS = @LIBESTR_LIBS@ LIBGCRYPT_CFLAGS = @LIBGCRYPT_CFLAGS@ +LIBGCRYPT_CONFIG = @LIBGCRYPT_CONFIG@ LIBGCRYPT_LIBS = @LIBGCRYPT_LIBS@ LIBLOGGING_CFLAGS = @LIBLOGGING_CFLAGS@ LIBLOGGING_LIBS = @LIBLOGGING_LIBS@ +LIBLOGGING_STDLOG_CFLAGS = @LIBLOGGING_STDLOG_CFLAGS@ +LIBLOGGING_STDLOG_LIBS = @LIBLOGGING_STDLOG_LIBS@ LIBLOGNORM_CFLAGS = @LIBLOGNORM_CFLAGS@ LIBLOGNORM_LIBS = @LIBLOGNORM_LIBS@ LIBM = @LIBM@ @@ -206,6 +206,8 @@ NM = @NM@ NMEDIT = @NMEDIT@ OBJDUMP = @OBJDUMP@ OBJEXT = @OBJEXT@ +OPENSSL_CFLAGS = @OPENSSL_CFLAGS@ +OPENSSL_LIBS = @OPENSSL_LIBS@ ORACLE_CFLAGS = @ORACLE_CFLAGS@ ORACLE_LIBS = @ORACLE_LIBS@ OTOOL = @OTOOL@ @@ -387,22 +389,25 @@ distclean-compile: @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/omstdout_la-omstdout.Plo@am__quote@ .c.o: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(COMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ $< -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Po +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.o$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c $< +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ $< .c.obj: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(COMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ `$(CYGPATH_W) '$<'` -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Po +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.obj$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ `$(CYGPATH_W) '$<'` &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c `$(CYGPATH_W) '$<'` +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ `$(CYGPATH_W) '$<'` .c.lo: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(LTCOMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ $< -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Plo +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.lo$$||'`;\ +@am__fastdepCC_TRUE@ $(LTCOMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Plo @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=yes @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ @am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(LTCOMPILE) -c -o $@ $< diff --git a/plugins/omstdout/omstdout.c b/plugins/omstdout/omstdout.c index a84a759..5e63ed7 100644 --- a/plugins/omstdout/omstdout.c +++ b/plugins/omstdout/omstdout.c @@ -6,7 +6,7 @@ * * File begun on 2009-03-19 by RGerhards * - * Copyright 2009-2012 Adiscon GmbH. + * Copyright 2009-2013 Adiscon GmbH. * * This file is part of rsyslog. * @@ -60,6 +60,10 @@ typedef struct _instanceData { int bEnsureLFEnding; /* ensure that a linefeed is written at the end of EACH record (test aid for nettester) */ } instanceData; +typedef struct wrkrInstanceData { + instanceData *pData; +} wrkrInstanceData_t; + typedef struct configSettings_s { int bUseArrayInterface; /* shall action use array instead of string template interface? */ int bEnsureLFEnding; /* shall action use array instead of string template interface? */ @@ -76,6 +80,11 @@ CODESTARTcreateInstance ENDcreateInstance +BEGINcreateWrkrInstance +CODESTARTcreateWrkrInstance +ENDcreateWrkrInstance + + BEGINisCompatibleWithFeature CODESTARTisCompatibleWithFeature if(eFeat == sFEATURERepeatedMsgReduction) @@ -88,6 +97,11 @@ CODESTARTfreeInstance ENDfreeInstance +BEGINfreeWrkrInstance +CODESTARTfreeWrkrInstance +ENDfreeWrkrInstance + + BEGINdbgPrintInstInfo CODESTARTdbgPrintInstInfo ENDdbgPrintInstInfo @@ -107,7 +121,7 @@ BEGINdoAction size_t len; int r; CODESTARTdoAction - if(pData->bUseArrayInterface) { + if(pWrkrData->pData->bUseArrayInterface) { /* if we use array passing, we need to put together a string * ourselves. At this point, please keep in mind that omstdout is * primarily a testing aid. Other modules may do different processing @@ -142,10 +156,10 @@ CODESTARTdoAction * needs to be more solid. -- rgerhards, 2012-11-28 */ if((r = write(1, toWrite, len)) != (int) len) { /* 1 is stdout! */ - DBGPRINTF("omstdout: error %d writing to stdout[%d]: %s\n", + DBGPRINTF("omstdout: error %d writing to stdout[%zd]: %s\n", r, len, toWrite); } - if(pData->bEnsureLFEnding && toWrite[len-1] != '\n') { + if(pWrkrData->pData->bEnsureLFEnding && toWrite[len-1] != '\n') { if((r = write(1, "\n", 1)) != 1) { /* write missing LF */ DBGPRINTF("omstdout: error %d writing \\n to stdout\n", r); @@ -186,6 +200,7 @@ ENDmodExit BEGINqueryEtryPt CODESTARTqueryEtryPt CODEqueryEtryPt_STD_OMOD_QUERIES +CODEqueryEtryPt_STD_OMOD8_QUERIES CODEqueryEtryPt_STD_CONF2_CNFNAME_QUERIES ENDqueryEtryPt diff --git a/plugins/omtesting/Makefile.in b/plugins/omtesting/Makefile.in index abbd972..61ae0d7 100644 --- a/plugins/omtesting/Makefile.in +++ b/plugins/omtesting/Makefile.in @@ -153,7 +153,6 @@ GREP = @GREP@ GSS_LIBS = @GSS_LIBS@ GUARDTIME_CFLAGS = @GUARDTIME_CFLAGS@ GUARDTIME_LIBS = @GUARDTIME_LIBS@ -HAVE_LIBGCRYPT_CONFIG = @HAVE_LIBGCRYPT_CONFIG@ HAVE_MYSQL_CONFIG = @HAVE_MYSQL_CONFIG@ HAVE_ORACLE_CONFIG = @HAVE_ORACLE_CONFIG@ HAVE_PGSQL_CONFIG = @HAVE_PGSQL_CONFIG@ @@ -174,14 +173,15 @@ LEXLIB = @LEXLIB@ LEX_OUTPUT_ROOT = @LEX_OUTPUT_ROOT@ LIBDBI_CFLAGS = @LIBDBI_CFLAGS@ LIBDBI_LIBS = @LIBDBI_LIBS@ -LIBEE_CFLAGS = @LIBEE_CFLAGS@ -LIBEE_LIBS = @LIBEE_LIBS@ LIBESTR_CFLAGS = @LIBESTR_CFLAGS@ LIBESTR_LIBS = @LIBESTR_LIBS@ LIBGCRYPT_CFLAGS = @LIBGCRYPT_CFLAGS@ +LIBGCRYPT_CONFIG = @LIBGCRYPT_CONFIG@ LIBGCRYPT_LIBS = @LIBGCRYPT_LIBS@ LIBLOGGING_CFLAGS = @LIBLOGGING_CFLAGS@ LIBLOGGING_LIBS = @LIBLOGGING_LIBS@ +LIBLOGGING_STDLOG_CFLAGS = @LIBLOGGING_STDLOG_CFLAGS@ +LIBLOGGING_STDLOG_LIBS = @LIBLOGGING_STDLOG_LIBS@ LIBLOGNORM_CFLAGS = @LIBLOGNORM_CFLAGS@ LIBLOGNORM_LIBS = @LIBLOGNORM_LIBS@ LIBM = @LIBM@ @@ -206,6 +206,8 @@ NM = @NM@ NMEDIT = @NMEDIT@ OBJDUMP = @OBJDUMP@ OBJEXT = @OBJEXT@ +OPENSSL_CFLAGS = @OPENSSL_CFLAGS@ +OPENSSL_LIBS = @OPENSSL_LIBS@ ORACLE_CFLAGS = @ORACLE_CFLAGS@ ORACLE_LIBS = @ORACLE_LIBS@ OTOOL = @OTOOL@ @@ -386,22 +388,25 @@ distclean-compile: @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/omtesting_la-omtesting.Plo@am__quote@ .c.o: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(COMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ $< -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Po +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.o$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c $< +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ $< .c.obj: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(COMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ `$(CYGPATH_W) '$<'` -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Po +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.obj$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ `$(CYGPATH_W) '$<'` &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c `$(CYGPATH_W) '$<'` +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ `$(CYGPATH_W) '$<'` .c.lo: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(LTCOMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ $< -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Plo +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.lo$$||'`;\ +@am__fastdepCC_TRUE@ $(LTCOMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Plo @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=yes @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ @am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(LTCOMPILE) -c -o $@ $< diff --git a/plugins/omtesting/omtesting.c b/plugins/omtesting/omtesting.c index c9f1e06..0df3063 100644 --- a/plugins/omtesting/omtesting.c +++ b/plugins/omtesting/omtesting.c @@ -22,7 +22,7 @@ * NOTE: read comments in module-template.h to understand how this file * works! * - * Copyright 2007-2012 Rainer Gerhards and Adiscon GmbH. + * Copyright 2007-2013 Rainer Gerhards and Adiscon GmbH. * * This file is part of rsyslog. * @@ -49,6 +49,7 @@ #include <string.h> #include <ctype.h> #include <assert.h> +#include <pthread.h> #include "dirty.h" #include "syslogd-types.h" #include "module-template.h" @@ -63,7 +64,6 @@ MODULE_CNFNAME("omtesting") */ DEF_OMOD_STATIC_DATA - typedef struct _instanceData { enum { MD_SLEEP, MD_FAIL, MD_RANDFAIL, MD_ALWAYS_SUSPEND } mode; @@ -74,8 +74,16 @@ typedef struct _instanceData { int iFailFrequency; int iResumeAfter; int iCurrRetries; + int bFailed; /* indicates if we are already in failed state - this is necessary + * to work properly together with multiple worker instances. + */ + pthread_mutex_t mut; } instanceData; +typedef struct wrkrInstanceData { + instanceData *pData; +} wrkrInstanceData_t; + typedef struct configSettings_s { int bEchoStdout; /* echo non-failed messages to stdout */ } configSettings_t; @@ -90,9 +98,15 @@ BEGINcreateInstance CODESTARTcreateInstance pData->iWaitSeconds = 1; pData->iWaitUSeconds = 0; + pthread_mutex_init(&pData->mut, NULL); ENDcreateInstance +BEGINcreateWrkrInstance +CODESTARTcreateWrkrInstance +ENDcreateWrkrInstance + + BEGINdbgPrintInstInfo CODESTARTdbgPrintInstInfo dbgprintf("Action delays rule by %d second(s) and %d millisecond(s)\n", @@ -115,6 +129,7 @@ static rsRetVal doFailOnResume(instanceData *pData) dbgprintf("fail retry curr %d, max %d\n", pData->iCurrRetries, pData->iResumeAfter); if(++pData->iCurrRetries == pData->iResumeAfter) { iRet = RS_RET_OK; + pData->bFailed = 0; } else { iRet = RS_RET_SUSPENDED; } @@ -128,12 +143,18 @@ static rsRetVal doFail(instanceData *pData) { DEFiRet; - dbgprintf("fail curr %d, frquency %d\n", pData->iCurrCallNbr, pData->iFailFrequency); - if(pData->iCurrCallNbr++ % pData->iFailFrequency == 0) { - pData->iCurrRetries = 0; - iRet = RS_RET_SUSPENDED; + dbgprintf("fail curr %d, frequency %d, bFailed %d\n", pData->iCurrCallNbr, + pData->iFailFrequency, pData->bFailed); + if(pData->bFailed) { + ABORT_FINALIZE(RS_RET_SUSPENDED); + } else { + if(pData->iCurrCallNbr++ % pData->iFailFrequency == 0) { + pData->iCurrRetries = 0; + pData->bFailed = 1; + iRet = RS_RET_SUSPENDED; + } } - +finalize_it: RETiRet; } @@ -170,11 +191,12 @@ static rsRetVal doRandFail(void) BEGINtryResume CODESTARTtryResume dbgprintf("omtesting tryResume() called\n"); - switch(pData->mode) { + pthread_mutex_lock(&pWrkrData->pData->mut); + switch(pWrkrData->pData->mode) { case MD_SLEEP: break; case MD_FAIL: - iRet = doFailOnResume(pData); + iRet = doFailOnResume(pWrkrData->pData); break; case MD_RANDFAIL: iRet = doRandFail(); @@ -182,13 +204,17 @@ CODESTARTtryResume case MD_ALWAYS_SUSPEND: iRet = RS_RET_SUSPENDED; } + pthread_mutex_unlock(&pWrkrData->pData->mut); dbgprintf("omtesting tryResume() returns iRet %d\n", iRet); ENDtryResume BEGINdoAction + instanceData *pData; CODESTARTdoAction dbgprintf("omtesting received msg '%s'\n", ppString[0]); + pData = pWrkrData->pData; + pthread_mutex_lock(&pData->mut); switch(pData->mode) { case MD_SLEEP: iRet = doSleep(pData); @@ -208,18 +234,22 @@ CODESTARTdoAction fprintf(stdout, "%s", ppString[0]); fflush(stdout); } + pthread_mutex_unlock(&pData->mut); dbgprintf(":omtesting: end doAction(), iRet %d\n", iRet); ENDdoAction BEGINfreeInstance CODESTARTfreeInstance - /* we do not have instance data, so we do not need to - * do anything here. -- rgerhards, 2007-07-25 - */ + pthread_mutex_destroy(&pData->mut); ENDfreeInstance +BEGINfreeWrkrInstance +CODESTARTfreeWrkrInstance +ENDfreeWrkrInstance + + BEGINparseSelectorAct int i; uchar szBuf[1024]; @@ -313,6 +343,7 @@ ENDmodExit BEGINqueryEtryPt CODESTARTqueryEtryPt CODEqueryEtryPt_STD_OMOD_QUERIES +CODEqueryEtryPt_STD_OMOD8_QUERIES CODEqueryEtryPt_STD_CONF2_CNFNAME_QUERIES ENDqueryEtryPt diff --git a/plugins/omudpspoof/Makefile.in b/plugins/omudpspoof/Makefile.in index e0fea4c..ee3492f 100644 --- a/plugins/omudpspoof/Makefile.in +++ b/plugins/omudpspoof/Makefile.in @@ -154,7 +154,6 @@ GREP = @GREP@ GSS_LIBS = @GSS_LIBS@ GUARDTIME_CFLAGS = @GUARDTIME_CFLAGS@ GUARDTIME_LIBS = @GUARDTIME_LIBS@ -HAVE_LIBGCRYPT_CONFIG = @HAVE_LIBGCRYPT_CONFIG@ HAVE_MYSQL_CONFIG = @HAVE_MYSQL_CONFIG@ HAVE_ORACLE_CONFIG = @HAVE_ORACLE_CONFIG@ HAVE_PGSQL_CONFIG = @HAVE_PGSQL_CONFIG@ @@ -175,14 +174,15 @@ LEXLIB = @LEXLIB@ LEX_OUTPUT_ROOT = @LEX_OUTPUT_ROOT@ LIBDBI_CFLAGS = @LIBDBI_CFLAGS@ LIBDBI_LIBS = @LIBDBI_LIBS@ -LIBEE_CFLAGS = @LIBEE_CFLAGS@ -LIBEE_LIBS = @LIBEE_LIBS@ LIBESTR_CFLAGS = @LIBESTR_CFLAGS@ LIBESTR_LIBS = @LIBESTR_LIBS@ LIBGCRYPT_CFLAGS = @LIBGCRYPT_CFLAGS@ +LIBGCRYPT_CONFIG = @LIBGCRYPT_CONFIG@ LIBGCRYPT_LIBS = @LIBGCRYPT_LIBS@ LIBLOGGING_CFLAGS = @LIBLOGGING_CFLAGS@ LIBLOGGING_LIBS = @LIBLOGGING_LIBS@ +LIBLOGGING_STDLOG_CFLAGS = @LIBLOGGING_STDLOG_CFLAGS@ +LIBLOGGING_STDLOG_LIBS = @LIBLOGGING_STDLOG_LIBS@ LIBLOGNORM_CFLAGS = @LIBLOGNORM_CFLAGS@ LIBLOGNORM_LIBS = @LIBLOGNORM_LIBS@ LIBM = @LIBM@ @@ -207,6 +207,8 @@ NM = @NM@ NMEDIT = @NMEDIT@ OBJDUMP = @OBJDUMP@ OBJEXT = @OBJEXT@ +OPENSSL_CFLAGS = @OPENSSL_CFLAGS@ +OPENSSL_LIBS = @OPENSSL_LIBS@ ORACLE_CFLAGS = @ORACLE_CFLAGS@ ORACLE_LIBS = @ORACLE_LIBS@ OTOOL = @OTOOL@ @@ -388,22 +390,25 @@ distclean-compile: @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/omudpspoof_la-omudpspoof.Plo@am__quote@ .c.o: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(COMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ $< -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Po +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.o$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c $< +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ $< .c.obj: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(COMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ `$(CYGPATH_W) '$<'` -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Po +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.obj$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ `$(CYGPATH_W) '$<'` &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c `$(CYGPATH_W) '$<'` +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ `$(CYGPATH_W) '$<'` .c.lo: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(LTCOMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ $< -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Plo +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.lo$$||'`;\ +@am__fastdepCC_TRUE@ $(LTCOMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Plo @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=yes @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ @am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(LTCOMPILE) -c -o $@ $< diff --git a/plugins/omudpspoof/omudpspoof.c b/plugins/omudpspoof/omudpspoof.c index c80f0e5..92fa48f 100644 --- a/plugins/omudpspoof/omudpspoof.c +++ b/plugins/omudpspoof/omudpspoof.c @@ -98,15 +98,19 @@ typedef struct _instanceData { uchar *port; uchar *sourceTpl; int mtu; - int *pSockArray; /* sockets to use for UDP */ - struct addrinfo *f_addr; - u_short sourcePort; u_short sourcePortStart; /* for sorce port iteration */ u_short sourcePortEnd; int bReportLibnetInitErr; /* help prevent multiple error messages on init err */ +} instanceData; + +typedef struct wrkrInstanceData { + instanceData *pData; libnet_t *libnet_handle; + u_short sourcePort; + int *pSockArray; /* sockets to use for UDP */ + struct addrinfo *f_addr; char errbuf[LIBNET_ERRBUF_SIZE]; -} instanceData; +} wrkrInstanceData_t; #define DFLT_SOURCE_PORT_START 32000 #define DFLT_SOURCE_PORT_END 42000 @@ -172,7 +176,7 @@ ENDinitConfVars pthread_mutex_t mutLibnet; /* forward definitions */ -static rsRetVal doTryResume(instanceData *pData); +static rsRetVal doTryResume(wrkrInstanceData_t *pWrkrData); /* this function gets the default template. It coordinates action between @@ -217,15 +221,14 @@ finalize_it: * rgerhards, 2009-05-29 */ static rsRetVal -closeUDPSockets(instanceData *pData) +closeUDPSockets(wrkrInstanceData_t *pWrkrData) { DEFiRet; - assert(pData != NULL); - if(pData->pSockArray != NULL) { - net.closeUDPListenSockets(pData->pSockArray); - pData->pSockArray = NULL; - freeaddrinfo(pData->f_addr); - pData->f_addr = NULL; + if(pWrkrData->pSockArray != NULL) { + net.closeUDPListenSockets(pWrkrData->pSockArray); + pWrkrData->pSockArray = NULL; + freeaddrinfo(pWrkrData->f_addr); + pWrkrData->f_addr = NULL; } RETiRet; } @@ -310,12 +313,17 @@ ENDfreeCnf BEGINcreateInstance CODESTARTcreateInstance - pData->libnet_handle = NULL; pData->mtu = 1500; pData->bReportLibnetInitErr = 1; ENDcreateInstance +BEGINcreateWrkrInstance +CODESTARTcreateWrkrInstance + pWrkrData->libnet_handle = NULL; + pWrkrData->sourcePort = pData->sourcePortStart; +ENDcreateWrkrInstance + BEGINisCompatibleWithFeature CODESTARTisCompatibleWithFeature if(eFeat == sFEATURERepeatedMsgReduction) @@ -326,15 +334,19 @@ ENDisCompatibleWithFeature BEGINfreeInstance CODESTARTfreeInstance /* final cleanup */ - closeUDPSockets(pData); free(pData->tplName); free(pData->port); free(pData->host); free(pData->sourceTpl); - if(pData->libnet_handle != NULL) - libnet_destroy(pData->libnet_handle); ENDfreeInstance +BEGINfreeWrkrInstance +CODESTARTfreeWrkrInstance + closeUDPSockets(pWrkrData); + if(pWrkrData->libnet_handle != NULL) + libnet_destroy(pWrkrData->libnet_handle); +ENDfreeWrkrInstance + BEGINdbgPrintInstInfo CODESTARTdbgPrintInstInfo @@ -348,11 +360,12 @@ ENDdbgPrintInstInfo * rgehards, 2007-12-20 */ static inline rsRetVal -UDPSend(instanceData *pData, uchar *pszSourcename, char *msg, size_t len) +UDPSend(wrkrInstanceData_t *pWrkrData, uchar *pszSourcename, char *msg, size_t len) { struct addrinfo *r; int lsent = 0; int bSendSuccess; + instanceData *pData; struct sockaddr_in *tempaddr,source_ip; libnet_ptag_t ip, ipo; libnet_ptag_t udp; @@ -363,19 +376,20 @@ UDPSend(instanceData *pData, uchar *pszSourcename, char *msg, size_t len) unsigned maxPktLen, pktLen; DEFiRet; - if(pData->pSockArray == NULL) { - CHKiRet(doTryResume(pData)); + if(pWrkrData->pSockArray == NULL) { + CHKiRet(doTryResume(pWrkrData)); } + pData = pWrkrData->pData; if(len > 65528) { DBGPRINTF("omudpspoof: msg with length %d truncated to 64k: '%.768s'\n", - len, msg); + (int) len, msg); len = 65528; } ip = ipo = udp = 0; - if(pData->sourcePort++ >= pData->sourcePortEnd){ - pData->sourcePort = pData->sourcePortStart; + if(pWrkrData->sourcePort++ >= pData->sourcePortEnd){ + pWrkrData->sourcePort = pData->sourcePortStart; } inet_pton(AF_INET, (char*)pszSourcename, &(source_ip.sin_addr)); @@ -383,7 +397,7 @@ UDPSend(instanceData *pData, uchar *pszSourcename, char *msg, size_t len) bSendSuccess = RSFALSE; d_pthread_mutex_lock(&mutLibnet); bNeedUnlock = 1; - for (r = pData->f_addr; r && bSendSuccess == RSFALSE ; r = r->ai_next) { + for (r = pWrkrData->f_addr; r && bSendSuccess == RSFALSE ; r = r->ai_next) { tempaddr = (struct sockaddr_in *)r->ai_addr; /* Getting max payload size (must be multiple of 8) */ maxPktLen = (pData->mtu - LIBNET_IPV4_H) & ~0x07; @@ -400,19 +414,19 @@ UDPSend(instanceData *pData, uchar *pszSourcename, char *msg, size_t len) } DBGPRINTF("omudpspoof: stage 1: MF:%d, hdrOffs %d, pktLen %d\n", (hdrOffs & IP_MF) >> 13, (hdrOffs & 0x1FFF) << 3, pktLen); - libnet_clear_packet(pData->libnet_handle); + libnet_clear_packet(pWrkrData->libnet_handle); /* note: libnet does need ports in host order NOT in network byte order! -- rgerhards, 2009-11-12 */ udp = libnet_build_udp( - ntohs(pData->sourcePort),/* source port */ + pWrkrData->sourcePort, /* source port */ ntohs(tempaddr->sin_port),/* destination port */ pktLen+LIBNET_UDP_H, /* packet length */ 0, /* checksum */ (u_char*)msg, /* payload */ pktLen, /* payload size */ - pData->libnet_handle, /* libnet handle */ + pWrkrData->libnet_handle, /* libnet handle */ udp); /* libnet id */ if (udp == -1) { - DBGPRINTF("omudpspoof: can't build UDP header: %s\n", libnet_geterror(pData->libnet_handle)); + DBGPRINTF("omudpspoof: can't build UDP header: %s\n", libnet_geterror(pWrkrData->libnet_handle)); } ip = libnet_build_ipv4( @@ -427,22 +441,22 @@ UDPSend(instanceData *pData, uchar *pszSourcename, char *msg, size_t len) tempaddr->sin_addr.s_addr, NULL, /* payload */ 0, /* payload size */ - pData->libnet_handle, /* libnet handle */ + pWrkrData->libnet_handle, /* libnet handle */ ip); /* libnet id */ if (ip == -1) { - DBGPRINTF("omudpspoof: can't build IP header: %s\n", libnet_geterror(pData->libnet_handle)); + DBGPRINTF("omudpspoof: can't build IP header: %s\n", libnet_geterror(pWrkrData->libnet_handle)); } /* Write it to the wire. */ - lsent = libnet_write(pData->libnet_handle); + lsent = libnet_write(pWrkrData->libnet_handle); if(lsent != (int) (LIBNET_IPV4_H+LIBNET_UDP_H+pktLen)) { /* note: access to fd is a libnet internal. If a newer version of libnet does * not expose that member, we should simply remove it. However, while it is there * it is useful for consolidating with strace output. */ DBGPRINTF("omudpspoof: write error (total len %d): pktLen %d, sent %d, fd %d: %s\n", - len, LIBNET_IPV4_H+LIBNET_UDP_H+pktLen, lsent, pData->libnet_handle->fd, - libnet_geterror(pData->libnet_handle)); + (int) len, LIBNET_IPV4_H+LIBNET_UDP_H+pktLen, lsent, pWrkrData->libnet_handle->fd, + libnet_geterror(pWrkrData->libnet_handle)); if(lsent != -1) { bSendSuccess = RSTRUE; } @@ -452,7 +466,7 @@ UDPSend(instanceData *pData, uchar *pszSourcename, char *msg, size_t len) msgOffs += pktLen; /* We need to get rid of the UDP header to build the other fragments */ - libnet_clear_packet(pData->libnet_handle); + libnet_clear_packet(pWrkrData->libnet_handle); ip = LIBNET_PTAG_INITIALIZER; while(len > msgOffs ) { /* loop until all payload is sent */ /* check if there will be more fragments */ @@ -481,16 +495,16 @@ UDPSend(instanceData *pData, uchar *pszSourcename, char *msg, size_t len) tempaddr->sin_addr.s_addr, (uint8_t*)(msg+msgOffs), /* payload */ pktLen, /* payload size */ - pData->libnet_handle, /* libnet handle */ + pWrkrData->libnet_handle, /* libnet handle */ ip); /* libnet id */ if (ip == -1) { - DBGPRINTF("omudpspoof: can't build IP fragment header: %s\n", libnet_geterror(pData->libnet_handle)); + DBGPRINTF("omudpspoof: can't build IP fragment header: %s\n", libnet_geterror(pWrkrData->libnet_handle)); } /* Write it to the wire. */ - lsent = libnet_write(pData->libnet_handle); + lsent = libnet_write(pWrkrData->libnet_handle); if(lsent != (int) (LIBNET_IPV4_H+pktLen)) { DBGPRINTF("omudpspoof: fragment write error len %d, sent %d: %s\n", - LIBNET_IPV4_H+LIBNET_UDP_H+len, lsent, libnet_geterror(pData->libnet_handle)); + (int) (LIBNET_IPV4_H+LIBNET_UDP_H+len), lsent, libnet_geterror(pWrkrData->libnet_handle)); bSendSuccess = RSFALSE; continue; } @@ -500,9 +514,9 @@ UDPSend(instanceData *pData, uchar *pszSourcename, char *msg, size_t len) finalize_it: if(iRet != RS_RET_OK) { - if(pData->libnet_handle != NULL) { - libnet_destroy(pData->libnet_handle); - pData->libnet_handle = NULL; + if(pWrkrData->libnet_handle != NULL) { + libnet_destroy(pWrkrData->libnet_handle); + pWrkrData->libnet_handle = NULL; } } if(bNeedUnlock) { @@ -515,29 +529,28 @@ finalize_it: /* try to resume connection if it is not ready * rgerhards, 2007-08-02 */ -static rsRetVal doTryResume(instanceData *pData) +static rsRetVal doTryResume(wrkrInstanceData_t *pWrkrData) { int iErr; struct addrinfo *res; struct addrinfo hints; + instanceData *pData; DEFiRet; - if(pData->pSockArray != NULL) + if(pWrkrData->pSockArray != NULL) FINALIZE; + pData = pWrkrData->pData; - if(pData->host == NULL) - ABORT_FINALIZE(RS_RET_DISABLE_ACTION); - - if(pData->libnet_handle == NULL) { + if(pWrkrData->libnet_handle == NULL) { /* Initialize the libnet library. Root priviledges are required. * this initializes a IPv4 socket to use for forging UDP packets. */ - pData->libnet_handle = libnet_init( + pWrkrData->libnet_handle = libnet_init( LIBNET_RAW4, /* injection type */ NULL, /* network interface */ - pData->errbuf); /* errbuf */ + pWrkrData->errbuf); /* errbuf */ - if(pData->libnet_handle == NULL) { + if(pWrkrData->libnet_handle == NULL) { if(pData->bReportLibnetInitErr) { errmsg.LogError(0, RS_RET_ERR_LIBNET_INIT, "omudpsoof: error " "initializing libnet - are you running as root?"); @@ -562,17 +575,16 @@ static rsRetVal doTryResume(instanceData *pData) ABORT_FINALIZE(RS_RET_SUSPENDED); } DBGPRINTF("%s found, resuming.\n", pData->host); - pData->f_addr = res; - pData->pSockArray = net.create_udp_socket((uchar*)pData->host, NULL, 0); + pWrkrData->f_addr = res; + pWrkrData->pSockArray = net.create_udp_socket((uchar*)pData->host, NULL, 0, 0); finalize_it: if(iRet != RS_RET_OK) { - if(pData->f_addr != NULL) { - freeaddrinfo(pData->f_addr); - pData->f_addr = NULL; + if(pWrkrData->f_addr != NULL) { + freeaddrinfo(pWrkrData->f_addr); + pWrkrData->f_addr = NULL; } - if(iRet != RS_RET_DISABLE_ACTION) - iRet = RS_RET_SUSPENDED; + iRet = RS_RET_SUSPENDED; } RETiRet; @@ -581,7 +593,7 @@ finalize_it: BEGINtryResume CODESTARTtryResume - iRet = doTryResume(pData); + iRet = doTryResume(pWrkrData); ENDtryResume BEGINdoAction @@ -589,10 +601,10 @@ BEGINdoAction unsigned l; int iMaxLine; CODESTARTdoAction - CHKiRet(doTryResume(pData)); + CHKiRet(doTryResume(pWrkrData)); - DBGPRINTF(" %s:%s/omudpspoof, src '%s', msg strt '%.256s'\n", pData->host, - getFwdPt(pData), ppString[1], ppString[0]); + DBGPRINTF(" %s:%s/omudpspoof, src '%s', msg strt '%.256s'\n", pWrkrData->pData->host, + getFwdPt(pWrkrData->pData), ppString[1], ppString[0]); iMaxLine = glbl.GetMaxLine(); psz = (char*) ppString[0]; @@ -600,7 +612,7 @@ CODESTARTdoAction if((int) l > iMaxLine) l = iMaxLine; - CHKiRet(UDPSend(pData, ppString[1], psz, l)); + CHKiRet(UDPSend(pWrkrData, ppString[1], psz, l)); finalize_it: ENDdoAction @@ -664,7 +676,6 @@ CODESTARTnewActInst } } CODE_STD_STRING_REQUESTnewActInst(2) - pData->sourcePort = pData->sourcePortStart; tplToUse = ustrdup((pData->tplName == NULL) ? getDfltTpl() : pData->tplName); CHKiRet(OMSRsetEntry(*ppOMSR, 0, tplToUse, OMSR_NO_RQD_TPL_OPTS)); @@ -703,7 +714,7 @@ CODE_STD_STRING_REQUESTparseSelectorAct(2) else CHKmalloc(pData->port = ustrdup(cs.pszTargetPort)); CHKiRet(OMSRsetEntry(*ppOMSR, 1, ustrdup(sourceTpl), OMSR_NO_RQD_TPL_OPTS)); - pData->sourcePort = pData->sourcePortStart = cs.iSourcePortStart; + pData->sourcePortStart = cs.iSourcePortStart; pData->sourcePortEnd = cs.iSourcePortEnd; /* process template */ @@ -744,6 +755,7 @@ ENDmodExit BEGINqueryEtryPt CODESTARTqueryEtryPt CODEqueryEtryPt_STD_OMOD_QUERIES +CODEqueryEtryPt_STD_OMOD8_QUERIES CODEqueryEtryPt_STD_CONF2_OMOD_QUERIES CODEqueryEtryPt_STD_CONF2_QUERIES CODEqueryEtryPt_STD_CONF2_setModCnf_QUERIES diff --git a/plugins/omuxsock/Makefile.in b/plugins/omuxsock/Makefile.in index 56ed12d..c9840db 100644 --- a/plugins/omuxsock/Makefile.in +++ b/plugins/omuxsock/Makefile.in @@ -153,7 +153,6 @@ GREP = @GREP@ GSS_LIBS = @GSS_LIBS@ GUARDTIME_CFLAGS = @GUARDTIME_CFLAGS@ GUARDTIME_LIBS = @GUARDTIME_LIBS@ -HAVE_LIBGCRYPT_CONFIG = @HAVE_LIBGCRYPT_CONFIG@ HAVE_MYSQL_CONFIG = @HAVE_MYSQL_CONFIG@ HAVE_ORACLE_CONFIG = @HAVE_ORACLE_CONFIG@ HAVE_PGSQL_CONFIG = @HAVE_PGSQL_CONFIG@ @@ -174,14 +173,15 @@ LEXLIB = @LEXLIB@ LEX_OUTPUT_ROOT = @LEX_OUTPUT_ROOT@ LIBDBI_CFLAGS = @LIBDBI_CFLAGS@ LIBDBI_LIBS = @LIBDBI_LIBS@ -LIBEE_CFLAGS = @LIBEE_CFLAGS@ -LIBEE_LIBS = @LIBEE_LIBS@ LIBESTR_CFLAGS = @LIBESTR_CFLAGS@ LIBESTR_LIBS = @LIBESTR_LIBS@ LIBGCRYPT_CFLAGS = @LIBGCRYPT_CFLAGS@ +LIBGCRYPT_CONFIG = @LIBGCRYPT_CONFIG@ LIBGCRYPT_LIBS = @LIBGCRYPT_LIBS@ LIBLOGGING_CFLAGS = @LIBLOGGING_CFLAGS@ LIBLOGGING_LIBS = @LIBLOGGING_LIBS@ +LIBLOGGING_STDLOG_CFLAGS = @LIBLOGGING_STDLOG_CFLAGS@ +LIBLOGGING_STDLOG_LIBS = @LIBLOGGING_STDLOG_LIBS@ LIBLOGNORM_CFLAGS = @LIBLOGNORM_CFLAGS@ LIBLOGNORM_LIBS = @LIBLOGNORM_LIBS@ LIBM = @LIBM@ @@ -206,6 +206,8 @@ NM = @NM@ NMEDIT = @NMEDIT@ OBJDUMP = @OBJDUMP@ OBJEXT = @OBJEXT@ +OPENSSL_CFLAGS = @OPENSSL_CFLAGS@ +OPENSSL_LIBS = @OPENSSL_LIBS@ ORACLE_CFLAGS = @ORACLE_CFLAGS@ ORACLE_LIBS = @ORACLE_LIBS@ OTOOL = @OTOOL@ @@ -387,22 +389,25 @@ distclean-compile: @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/omuxsock_la-omuxsock.Plo@am__quote@ .c.o: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(COMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ $< -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Po +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.o$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c $< +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ $< .c.obj: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(COMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ `$(CYGPATH_W) '$<'` -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Po +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.obj$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ `$(CYGPATH_W) '$<'` &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c `$(CYGPATH_W) '$<'` +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ `$(CYGPATH_W) '$<'` .c.lo: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(LTCOMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ $< -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Plo +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.lo$$||'`;\ +@am__fastdepCC_TRUE@ $(LTCOMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Plo @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=yes @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ @am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(LTCOMPILE) -c -o $@ $< diff --git a/plugins/omuxsock/omuxsock.c b/plugins/omuxsock/omuxsock.c index 583b9f9..da4e8e9 100644 --- a/plugins/omuxsock/omuxsock.c +++ b/plugins/omuxsock/omuxsock.c @@ -4,7 +4,7 @@ * NOTE: read comments in module-template.h to understand how this file * works! * - * Copyright 2010-2012 Adiscon GmbH. + * Copyright 2010-2013 Adiscon GmbH. * * This file is part of rsyslog. * @@ -60,10 +60,14 @@ typedef struct _instanceData { permittedPeers_t *pPermPeers; uchar *sockName; int sock; - int bIsConnected; /* are we connected to remote host? 0 - no, 1 - yes, UDP means addr resolved */ struct sockaddr_un addr; } instanceData; + +typedef struct wrkrInstanceData { + instanceData *pData; +} wrkrInstanceData_t; + /* config data */ typedef struct configSettings_s { uchar *tplName; /* name of the default template to use */ @@ -90,6 +94,7 @@ static modConfData_t *loadModConf = NULL;/* modConf ptr to use for the current l static modConfData_t *runModConf = NULL;/* modConf ptr to use for the current exec process */ +static pthread_mutex_t mutDoAct = PTHREAD_MUTEX_INITIALIZER; BEGINinitConfVars /* (re)set config variables to default values */ CODESTARTinitConfVars @@ -147,7 +152,6 @@ closeSocket(instanceData *pData) close(pData->sock); pData->sock = INVLD_SOCK; } -pData->bIsConnected = 0; // TODO: remove this variable altogether RETiRet; } @@ -224,6 +228,10 @@ CODESTARTcreateInstance pData->sock = INVLD_SOCK; ENDcreateInstance +BEGINcreateWrkrInstance +CODESTARTcreateWrkrInstance +ENDcreateWrkrInstance + BEGINisCompatibleWithFeature CODESTARTisCompatibleWithFeature @@ -239,6 +247,10 @@ CODESTARTfreeInstance free(pData->sockName); ENDfreeInstance +BEGINfreeWrkrInstance +CODESTARTfreeWrkrInstance +ENDfreeWrkrInstance + BEGINdbgPrintInstInfo CODESTARTdbgPrintInstInfo @@ -332,7 +344,7 @@ static rsRetVal doTryResume(instanceData *pData) BEGINtryResume CODESTARTtryResume - iRet = doTryResume(pData); + iRet = doTryResume(pWrkrData->pData); ENDtryResume BEGINdoAction @@ -340,20 +352,22 @@ BEGINdoAction register unsigned l; int iMaxLine; CODESTARTdoAction - CHKiRet(doTryResume(pData)); + pthread_mutex_lock(&mutDoAct); + CHKiRet(doTryResume(pWrkrData->pData)); iMaxLine = glbl.GetMaxLine(); - DBGPRINTF(" omuxsock:%s\n", pData->sockName); + DBGPRINTF(" omuxsock:%s\n", pWrkrData->pData->sockName); psz = (char*) ppString[0]; l = strlen((char*) psz); if((int) l > iMaxLine) l = iMaxLine; - CHKiRet(sendMsg(pData, psz, l)); + CHKiRet(sendMsg(pWrkrData->pData, psz, l)); finalize_it: + pthread_mutex_unlock(&mutDoAct); ENDdoAction @@ -413,6 +427,7 @@ ENDmodExit BEGINqueryEtryPt CODESTARTqueryEtryPt CODEqueryEtryPt_STD_OMOD_QUERIES +CODEqueryEtryPt_STD_OMOD8_QUERIES CODEqueryEtryPt_STD_CONF2_QUERIES CODEqueryEtryPt_STD_CONF2_setModCnf_QUERIES ENDqueryEtryPt diff --git a/plugins/omzmq3/Makefile.in b/plugins/omzmq3/Makefile.in index c83e201..9a866f2 100644 --- a/plugins/omzmq3/Makefile.in +++ b/plugins/omzmq3/Makefile.in @@ -154,7 +154,6 @@ GREP = @GREP@ GSS_LIBS = @GSS_LIBS@ GUARDTIME_CFLAGS = @GUARDTIME_CFLAGS@ GUARDTIME_LIBS = @GUARDTIME_LIBS@ -HAVE_LIBGCRYPT_CONFIG = @HAVE_LIBGCRYPT_CONFIG@ HAVE_MYSQL_CONFIG = @HAVE_MYSQL_CONFIG@ HAVE_ORACLE_CONFIG = @HAVE_ORACLE_CONFIG@ HAVE_PGSQL_CONFIG = @HAVE_PGSQL_CONFIG@ @@ -175,14 +174,15 @@ LEXLIB = @LEXLIB@ LEX_OUTPUT_ROOT = @LEX_OUTPUT_ROOT@ LIBDBI_CFLAGS = @LIBDBI_CFLAGS@ LIBDBI_LIBS = @LIBDBI_LIBS@ -LIBEE_CFLAGS = @LIBEE_CFLAGS@ -LIBEE_LIBS = @LIBEE_LIBS@ LIBESTR_CFLAGS = @LIBESTR_CFLAGS@ LIBESTR_LIBS = @LIBESTR_LIBS@ LIBGCRYPT_CFLAGS = @LIBGCRYPT_CFLAGS@ +LIBGCRYPT_CONFIG = @LIBGCRYPT_CONFIG@ LIBGCRYPT_LIBS = @LIBGCRYPT_LIBS@ LIBLOGGING_CFLAGS = @LIBLOGGING_CFLAGS@ LIBLOGGING_LIBS = @LIBLOGGING_LIBS@ +LIBLOGGING_STDLOG_CFLAGS = @LIBLOGGING_STDLOG_CFLAGS@ +LIBLOGGING_STDLOG_LIBS = @LIBLOGGING_STDLOG_LIBS@ LIBLOGNORM_CFLAGS = @LIBLOGNORM_CFLAGS@ LIBLOGNORM_LIBS = @LIBLOGNORM_LIBS@ LIBM = @LIBM@ @@ -207,6 +207,8 @@ NM = @NM@ NMEDIT = @NMEDIT@ OBJDUMP = @OBJDUMP@ OBJEXT = @OBJEXT@ +OPENSSL_CFLAGS = @OPENSSL_CFLAGS@ +OPENSSL_LIBS = @OPENSSL_LIBS@ ORACLE_CFLAGS = @ORACLE_CFLAGS@ ORACLE_LIBS = @ORACLE_LIBS@ OTOOL = @OTOOL@ @@ -388,22 +390,25 @@ distclean-compile: @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/omzmq3_la-omzmq3.Plo@am__quote@ .c.o: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(COMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ $< -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Po +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.o$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c $< +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ $< .c.obj: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(COMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ `$(CYGPATH_W) '$<'` -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Po +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.obj$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ `$(CYGPATH_W) '$<'` &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c `$(CYGPATH_W) '$<'` +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ `$(CYGPATH_W) '$<'` .c.lo: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(LTCOMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ $< -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Plo +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.lo$$||'`;\ +@am__fastdepCC_TRUE@ $(LTCOMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Plo @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=yes @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ @am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(LTCOMPILE) -c -o $@ $< diff --git a/plugins/omzmq3/README b/plugins/omzmq3/README index ccc96c7..c2a3355 100644 --- a/plugins/omzmq3/README +++ b/plugins/omzmq3/README @@ -1,16 +1,10 @@ ZeroMQ 3.x Output Plugin Building this plugin: -Requires libzmq and libczmq. First, install libzmq from the HEAD on github: -http://github.com/zeromq/libzmq. You can clone the repository, build, then -install it. The directions for doing so are there in the readme. Then, do -the same for libczmq: http://github.com/zeromq/czmq. At some point, the 3.1 -version of libzmq will be released, and a supporting version of libczmq. -At that time, you could simply download and install the tarballs instead of -using git to clone the repositories. Those tarballs (when available) can -be found at http://download.zeromq.org. As of this writing (5/31/2012), the -most recent version of czmq (1.1.0) and libzmq (3.1.0-beta) will not compile -properly. +Requires libzmq and libczmq. First, download the tarballs of both libzmq +and its supporting libczmq from http://download.zeromq.org. As of this +writing (04/23/2013), the most recent versions of libzmq and czmq are +3.2.2 and 1.3.2 respectively. Configure, build, and then install both libs. Omzmq3 allows you to push data out of rsyslog from a zeromq socket. The example below binds a PUB socket to port 7171, and any message fitting the criteria will diff --git a/plugins/omzmq3/omzmq3.c b/plugins/omzmq3/omzmq3.c index ee6756b..4eb4a37 100644 --- a/plugins/omzmq3/omzmq3.c +++ b/plugins/omzmq3/omzmq3.c @@ -2,26 +2,25 @@ * Copyright 2012 Talksum, Inc * Using the czmq interface to zeromq, we output * to a zmq socket. - - -* -* This program is free software: you can redistribute it and/or -* modify it under the terms of the GNU Lesser General Public License -* as published by the Free Software Foundation, either version 3 of -* the License, or (at your option) any later version. -* -* This program is distributed in the hope that it will be useful, but -* WITHOUT ANY WARRANTY; without even the implied warranty of -* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU -* Lesser General Public License for more details. -* -* You should have received a copy of the GNU Lesser General Public -* License along with this program. If not, see -* <http://www.gnu.org/licenses/>. -* -* Author: David Kelly -* <davidk@talksum.com> -*/ + * Copyright (C) 2014 Rainer Gerhards + * + * This program is free software: you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public License + * as published by the Free Software Foundation, either version 3 of + * the License, or (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, but + * WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this program. If not, see + * <http://www.gnu.org/licenses/>. + * + * Author: David Kelly + * <davidk@talksum.com> + */ #include "config.h" @@ -51,6 +50,8 @@ MODULE_CNFNAME("omzmq3") DEF_OMOD_STATIC_DATA DEFobjCurrIf(errmsg) +static pthread_mutex_t mutDoAct = PTHREAD_MUTEX_INITIALIZER; + /* convienent symbols to denote a socket we want to bind vs one we want to just connect to */ @@ -97,6 +98,10 @@ typedef struct _instanceData { uchar* tplName; } instanceData; +typedef struct wrkrInstanceData { + instanceData *pData; +} wrkrInstanceData_t; + /* ---------------------------------------------------------------------------- * Static definitions/initializations @@ -110,9 +115,10 @@ static zctx_t* s_context = NULL; static int s_workerThreads = -1; static struct socket_type types[] = { - {"PUB", ZMQ_PUB }, - {"PUSH", ZMQ_PUSH }, - {"XPUB", ZMQ_XPUB } + {"PUB", ZMQ_PUB }, + {"PUSH", ZMQ_PUSH }, + {"DEALER", ZMQ_DEALER }, + {"XPUB", ZMQ_XPUB } }; static struct socket_action actions[] = { @@ -201,17 +207,18 @@ static rsRetVal initZMQ(instanceData* pData) { /* create the context if necessary. */ if (NULL == s_context) { + zsys_handler_set(NULL); s_context = zctx_new(); if (s_workerThreads > 0) zctx_set_iothreads(s_context, s_workerThreads); } pData->socket = zsocket_new(s_context, pData->type); - - /* ALWAYS set the HWM as the zmq3 default is 1000 and we default - to 0 (infinity) */ - zsocket_set_rcvhwm(pData->socket, pData->rcvHWM); - zsocket_set_sndhwm(pData->socket, pData->sndHWM); - + if (NULL == pData->socket) { + errmsg.LogError(0, RS_RET_NO_ERRCODE, + "omzmq3: zsocket_new failed for %s: %s", + pData->description, zmq_strerror(errno)); + ABORT_FINALIZE(RS_RET_NO_ERRCODE); + } /* use czmq defaults for these, unless set to non-default values */ if(pData->identity) zsocket_set_identity(pData->socket, (char*)pData->identity); if(pData->sndBuf > -1) zsocket_set_sndbuf(pData->socket, pData->sndBuf); @@ -228,17 +235,26 @@ static rsRetVal initZMQ(instanceData* pData) { if(pData->reconnectIVLMax > -1) zsocket_set_reconnect_ivl_max(pData->socket, pData->reconnectIVLMax); if(pData->ipv4Only > -1) zsocket_set_ipv4only(pData->socket, pData->ipv4Only); if(pData->affinity != 1) zsocket_set_affinity(pData->socket, pData->affinity); - + if(pData->rcvHWM > -1) zsocket_set_rcvhwm(pData->socket, pData->rcvHWM); + if(pData->sndHWM > -1) zsocket_set_sndhwm(pData->socket, pData->sndHWM); + /* bind or connect to it */ if (pData->action == ACTION_BIND) { /* bind asserts, so no need to test return val here which isn't the greatest api -- oh well */ - zsocket_bind(pData->socket, (char*)pData->description); + if(-1 == zsocket_bind(pData->socket, (char*)pData->description)) { + errmsg.LogError(0, RS_RET_NO_ERRCODE, "omzmq3: bind failed for %s: %s", + pData->description, zmq_strerror(errno)); + ABORT_FINALIZE(RS_RET_NO_ERRCODE); + } + DBGPRINTF("omzmq3: bind to %s successful\n",pData->description); } else { - if(zsocket_connect(pData->socket, (char*)pData->description) == -1) { - errmsg.LogError(0, RS_RET_SUSPENDED, "omzmq3: connect failed!"); - ABORT_FINALIZE(RS_RET_SUSPENDED); + if(-1 == zsocket_connect(pData->socket, (char*)pData->description)) { + errmsg.LogError(0, RS_RET_NO_ERRCODE, "omzmq3: connect failed for %s: %s", + pData->description, zmq_strerror(errno)); + ABORT_FINALIZE(RS_RET_NO_ERRCODE); } + DBGPRINTF("omzmq3: connect to %s successful", pData->description); } finalize_it: RETiRet; @@ -256,7 +272,7 @@ rsRetVal writeZMQ(uchar* msg, instanceData* pData) { /* whine if things went wrong */ if (result == -1) { - errmsg.LogError(0, NO_ERRCODE, "omzmq3: send of %s failed with return %d", msg, result); + errmsg.LogError(0, NO_ERRCODE, "omzmq3: send of %s failed: %s", msg, zmq_strerror(errno)); ABORT_FINALIZE(RS_RET_ERR); } finalize_it: @@ -265,13 +281,13 @@ rsRetVal writeZMQ(uchar* msg, instanceData* pData) { static inline void setInstParamDefaults(instanceData* pData) { - pData->description = (uchar*)"tcp://*:7171"; + pData->description = NULL; pData->socket = NULL; pData->tplName = NULL; pData->type = ZMQ_PUB; pData->action = ACTION_BIND; - pData->sndHWM = 0; /*unlimited*/ - pData->rcvHWM = 0; /*unlimited*/ + pData->sndHWM = -1; + pData->rcvHWM = -1; pData->identity = NULL; pData->sndBuf = -1; pData->rcvBuf = -1; @@ -298,6 +314,11 @@ BEGINcreateInstance CODESTARTcreateInstance ENDcreateInstance + +BEGINcreateWrkrInstance +CODESTARTcreateWrkrInstance +ENDcreateWrkrInstance + BEGINisCompatibleWithFeature CODESTARTisCompatibleWithFeature if(eFeat == sFEATURERepeatedMsgReduction) @@ -314,136 +335,150 @@ CODESTARTfreeInstance closeZMQ(pData); free(pData->description); free(pData->tplName); + free(pData->identity); ENDfreeInstance + +BEGINfreeWrkrInstance +CODESTARTfreeWrkrInstance +ENDfreeWrkrInstance + + BEGINtryResume CODESTARTtryResume - if(NULL == pData->socket) - iRet = initZMQ(pData); + pthread_mutex_lock(&mutDoAct); + if(NULL == pWrkrData->pData->socket) + iRet = initZMQ(pWrkrData->pData); + pthread_mutex_unlock(&mutDoAct); ENDtryResume BEGINdoAction + instanceData *pData = pWrkrData->pData; CODESTARTdoAction -iRet = writeZMQ(ppString[0], pData); + pthread_mutex_lock(&mutDoAct); + iRet = writeZMQ(ppString[0], pData); + pthread_mutex_unlock(&mutDoAct); ENDdoAction BEGINnewActInst - struct cnfparamvals *pvals; - int i; + struct cnfparamvals *pvals; + int i; CODESTARTnewActInst -if((pvals = nvlstGetParams(lst, &actpblk, NULL)) == NULL) { - ABORT_FINALIZE(RS_RET_MISSING_CNFPARAMS); - } - -CHKiRet(createInstance(&pData)); -setInstParamDefaults(pData); - -CODE_STD_STRING_REQUESTnewActInst(1) -for(i = 0 ; i < actpblk.nParams ; ++i) { - if(!pvals[i].bUsed) - continue; - if(!strcmp(actpblk.descr[i].name, "description")) { - pData->description = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); - } else if(!strcmp(actpblk.descr[i].name, "template")) { - pData->tplName = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); - } else if(!strcmp(actpblk.descr[i].name, "sockType")){ - pData->type = getSocketType(es_str2cstr(pvals[i].val.d.estr, NULL)); - } else if(!strcmp(actpblk.descr[i].name, "action")){ - pData->action = getSocketAction(es_str2cstr(pvals[i].val.d.estr, NULL)); - } else if(!strcmp(actpblk.descr[i].name, "sndHWM")) { - pData->sndHWM = (int) pvals[i].val.d.n; - } else if(!strcmp(actpblk.descr[i].name, "rcvHWM")) { - pData->rcvHWM = (int) pvals[i].val.d.n; - } else if(!strcmp(actpblk.descr[i].name, "identity")){ - pData->identity = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); - } else if(!strcmp(actpblk.descr[i].name, "sndBuf")) { - pData->sndBuf = (int) pvals[i].val.d.n; - } else if(!strcmp(actpblk.descr[i].name, "rcvBuf")) { - pData->rcvBuf = (int) pvals[i].val.d.n; - } else if(!strcmp(actpblk.descr[i].name, "linger")) { - pData->linger = (int) pvals[i].val.d.n; - } else if(!strcmp(actpblk.descr[i].name, "backlog")) { - pData->backlog = (int) pvals[i].val.d.n; - } else if(!strcmp(actpblk.descr[i].name, "sndTimeout")) { - pData->sndTimeout = (int) pvals[i].val.d.n; - } else if(!strcmp(actpblk.descr[i].name, "rcvTimeout")) { - pData->rcvTimeout = (int) pvals[i].val.d.n; - } else if(!strcmp(actpblk.descr[i].name, "maxMsgSize")) { - pData->maxMsgSize = (int) pvals[i].val.d.n; - } else if(!strcmp(actpblk.descr[i].name, "rate")) { - pData->rate = (int) pvals[i].val.d.n; - } else if(!strcmp(actpblk.descr[i].name, "recoveryIVL")) { - pData->recoveryIVL = (int) pvals[i].val.d.n; - } else if(!strcmp(actpblk.descr[i].name, "multicastHops")) { - pData->multicastHops = (int) pvals[i].val.d.n; - } else if(!strcmp(actpblk.descr[i].name, "reconnectIVL")) { - pData->reconnectIVL = (int) pvals[i].val.d.n; - } else if(!strcmp(actpblk.descr[i].name, "reconnectIVLMax")) { - pData->reconnectIVLMax = (int) pvals[i].val.d.n; - } else if(!strcmp(actpblk.descr[i].name, "ipv4Only")) { - pData->ipv4Only = (int) pvals[i].val.d.n; - } else if(!strcmp(actpblk.descr[i].name, "affinity")) { - pData->affinity = (int) pvals[i].val.d.n; - } else if(!strcmp(actpblk.descr[i].name, "globalWorkerThreads")) { - s_workerThreads = (int) pvals[i].val.d.n; + if ((pvals = nvlstGetParams(lst, &actpblk, NULL)) == NULL) { + ABORT_FINALIZE(RS_RET_MISSING_CNFPARAMS); + } + + CHKiRet(createInstance(&pData)); + setInstParamDefaults(pData); + + CODE_STD_STRING_REQUESTnewActInst(1) + for (i = 0; i < actpblk.nParams; ++i) { + if (!pvals[i].bUsed) + continue; + if (!strcmp(actpblk.descr[i].name, "description")) { + pData->description = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + } else if (!strcmp(actpblk.descr[i].name, "template")) { + pData->tplName = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + } else if (!strcmp(actpblk.descr[i].name, "sockType")){ + pData->type = getSocketType(es_str2cstr(pvals[i].val.d.estr, NULL)); + } else if (!strcmp(actpblk.descr[i].name, "action")){ + pData->action = getSocketAction(es_str2cstr(pvals[i].val.d.estr, NULL)); + } else if (!strcmp(actpblk.descr[i].name, "sndHWM")) { + pData->sndHWM = (int) pvals[i].val.d.n; + } else if (!strcmp(actpblk.descr[i].name, "rcvHWM")) { + pData->rcvHWM = (int) pvals[i].val.d.n; + } else if (!strcmp(actpblk.descr[i].name, "identity")){ + pData->identity = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + } else if (!strcmp(actpblk.descr[i].name, "sndBuf")) { + pData->sndBuf = (int) pvals[i].val.d.n; + } else if (!strcmp(actpblk.descr[i].name, "rcvBuf")) { + pData->rcvBuf = (int) pvals[i].val.d.n; + } else if(!strcmp(actpblk.descr[i].name, "linger")) { + pData->linger = (int) pvals[i].val.d.n; + } else if (!strcmp(actpblk.descr[i].name, "backlog")) { + pData->backlog = (int) pvals[i].val.d.n; + } else if (!strcmp(actpblk.descr[i].name, "sndTimeout")) { + pData->sndTimeout = (int) pvals[i].val.d.n; + } else if (!strcmp(actpblk.descr[i].name, "rcvTimeout")) { + pData->rcvTimeout = (int) pvals[i].val.d.n; + } else if (!strcmp(actpblk.descr[i].name, "maxMsgSize")) { + pData->maxMsgSize = (int) pvals[i].val.d.n; + } else if (!strcmp(actpblk.descr[i].name, "rate")) { + pData->rate = (int) pvals[i].val.d.n; + } else if (!strcmp(actpblk.descr[i].name, "recoveryIVL")) { + pData->recoveryIVL = (int) pvals[i].val.d.n; + } else if (!strcmp(actpblk.descr[i].name, "multicastHops")) { + pData->multicastHops = (int) pvals[i].val.d.n; + } else if (!strcmp(actpblk.descr[i].name, "reconnectIVL")) { + pData->reconnectIVL = (int) pvals[i].val.d.n; + } else if (!strcmp(actpblk.descr[i].name, "reconnectIVLMax")) { + pData->reconnectIVLMax = (int) pvals[i].val.d.n; + } else if (!strcmp(actpblk.descr[i].name, "ipv4Only")) { + pData->ipv4Only = (int) pvals[i].val.d.n; + } else if (!strcmp(actpblk.descr[i].name, "affinity")) { + pData->affinity = (int) pvals[i].val.d.n; + } else if (!strcmp(actpblk.descr[i].name, "globalWorkerThreads")) { + s_workerThreads = (int) pvals[i].val.d.n; + } else { + errmsg.LogError(0, NO_ERRCODE, "omzmq3: program error, non-handled " + "param '%s'\n", actpblk.descr[i].name); + } + } + + if (pData->tplName == NULL) { + CHKiRet(OMSRsetEntry(*ppOMSR, 0, (uchar*)strdup("RSYSLOG_ForwardFormat"), OMSR_NO_RQD_TPL_OPTS)); } else { - errmsg.LogError(0, NO_ERRCODE, "omzmq3: program error, non-handled " - "param '%s'\n", actpblk.descr[i].name); + CHKiRet(OMSRsetEntry(*ppOMSR, 0, (uchar*)pData->tplName, OMSR_NO_RQD_TPL_OPTS)); + } + if (NULL == pData->description) { + errmsg.LogError(0, RS_RET_CONFIG_ERROR, "omzmq3: you didn't enter a description"); + ABORT_FINALIZE(RS_RET_CONFIG_ERROR); + } + if (pData->type == -1) { + errmsg.LogError(0, RS_RET_CONFIG_ERROR, "omzmq3: unknown socket type."); + ABORT_FINALIZE(RS_RET_CONFIG_ERROR); } - } - -if(pData->tplName == NULL) { - CHKiRet(OMSRsetEntry(*ppOMSR, 0, NULL, OMSR_TPL_AS_MSG)); - } else { - CHKiRet(OMSRsetEntry(*ppOMSR, 0, (uchar*)pData->tplName, OMSR_NO_RQD_TPL_OPTS)); - } - -if(pData->type == -1) { - errmsg.LogError(0, RS_RET_CONFIG_ERROR, "omzmq3: unknown socket type."); - ABORT_FINALIZE(RS_RET_CONFIG_ERROR); - } -if(pData->action == -1) { - errmsg.LogError(0, RS_RET_CONFIG_ERROR, "omzmq3: unknown socket action"); - ABORT_FINALIZE(RS_RET_CONFIG_ERROR); - } - - -CODE_STD_FINALIZERnewActInst - cnfparamvalsDestruct(pvals, &actpblk); + if (pData->action == -1) { + errmsg.LogError(0, RS_RET_CONFIG_ERROR, "omzmq3: unknown socket action"); + ABORT_FINALIZE(RS_RET_CONFIG_ERROR); + } + + CODE_STD_FINALIZERnewActInst + cnfparamvalsDestruct(pvals, &actpblk); ENDnewActInst BEGINparseSelectorAct CODESTARTparseSelectorAct - -/* tell the engine we only want one template string */ -CODE_STD_STRING_REQUESTparseSelectorAct(1) + /* tell the engine we only want one template string */ + CODE_STD_STRING_REQUESTparseSelectorAct(1) if(!strncmp((char*) p, ":omzmq3:", sizeof(":omzmq3:") - 1)) errmsg.LogError(0, RS_RET_LEGA_ACT_NOT_SUPPORTED, "omzmq3 supports only v6 config format, use: " "action(type=\"omzmq3\" serverport=...)"); ABORT_FINALIZE(RS_RET_CONFLINE_UNPROCESSED); -CODE_STD_FINALIZERparseSelectorAct + CODE_STD_FINALIZERparseSelectorAct ENDparseSelectorAct BEGINinitConfVars /* (re)set config variables to defaults */ CODESTARTinitConfVars -s_workerThreads = -1; + s_workerThreads = -1; ENDinitConfVars BEGINmodExit CODESTARTmodExit -if(NULL != s_context) { - zctx_destroy(&s_context); - s_context=NULL; - } + if (NULL != s_context) { + zctx_destroy(&s_context); + s_context=NULL; + } ENDmodExit BEGINqueryEtryPt CODESTARTqueryEtryPt -CODEqueryEtryPt_STD_OMOD_QUERIES -CODEqueryEtryPt_STD_CONF2_OMOD_QUERIES + CODEqueryEtryPt_STD_OMOD_QUERIES + CODEqueryEtryPt_STD_CONF2_OMOD_QUERIES + CODEqueryEtryPt_STD_OMOD8_QUERIES ENDqueryEtryPt BEGINmodInit() @@ -454,9 +489,6 @@ CODEmodInit_QueryRegCFSLineHdlr INITChkCoreFeature(bCoreSupportsBatching, CORE_FEATURE_BATCHING); DBGPRINTF("omzmq3: module compiled with rsyslog version %s.\n", VERSION); -INITLegCnfVars -CHKiRet(omsdRegCFSLineHdlr((uchar *)"omzmq3workerthreads", 0, eCmdHdlrInt, NULL, &s_workerThreads, STD_LOADABLE_MODULE_ID)); + INITLegCnfVars + CHKiRet(omsdRegCFSLineHdlr((uchar *)"omzmq3workerthreads", 0, eCmdHdlrInt, NULL, &s_workerThreads, STD_LOADABLE_MODULE_ID)); ENDmodInit - - - diff --git a/plugins/pmaixforwardedfrom/Makefile.in b/plugins/pmaixforwardedfrom/Makefile.in index 26d363c..94cb0d1 100644 --- a/plugins/pmaixforwardedfrom/Makefile.in +++ b/plugins/pmaixforwardedfrom/Makefile.in @@ -155,7 +155,6 @@ GREP = @GREP@ GSS_LIBS = @GSS_LIBS@ GUARDTIME_CFLAGS = @GUARDTIME_CFLAGS@ GUARDTIME_LIBS = @GUARDTIME_LIBS@ -HAVE_LIBGCRYPT_CONFIG = @HAVE_LIBGCRYPT_CONFIG@ HAVE_MYSQL_CONFIG = @HAVE_MYSQL_CONFIG@ HAVE_ORACLE_CONFIG = @HAVE_ORACLE_CONFIG@ HAVE_PGSQL_CONFIG = @HAVE_PGSQL_CONFIG@ @@ -176,14 +175,15 @@ LEXLIB = @LEXLIB@ LEX_OUTPUT_ROOT = @LEX_OUTPUT_ROOT@ LIBDBI_CFLAGS = @LIBDBI_CFLAGS@ LIBDBI_LIBS = @LIBDBI_LIBS@ -LIBEE_CFLAGS = @LIBEE_CFLAGS@ -LIBEE_LIBS = @LIBEE_LIBS@ LIBESTR_CFLAGS = @LIBESTR_CFLAGS@ LIBESTR_LIBS = @LIBESTR_LIBS@ LIBGCRYPT_CFLAGS = @LIBGCRYPT_CFLAGS@ +LIBGCRYPT_CONFIG = @LIBGCRYPT_CONFIG@ LIBGCRYPT_LIBS = @LIBGCRYPT_LIBS@ LIBLOGGING_CFLAGS = @LIBLOGGING_CFLAGS@ LIBLOGGING_LIBS = @LIBLOGGING_LIBS@ +LIBLOGGING_STDLOG_CFLAGS = @LIBLOGGING_STDLOG_CFLAGS@ +LIBLOGGING_STDLOG_LIBS = @LIBLOGGING_STDLOG_LIBS@ LIBLOGNORM_CFLAGS = @LIBLOGNORM_CFLAGS@ LIBLOGNORM_LIBS = @LIBLOGNORM_LIBS@ LIBM = @LIBM@ @@ -208,6 +208,8 @@ NM = @NM@ NMEDIT = @NMEDIT@ OBJDUMP = @OBJDUMP@ OBJEXT = @OBJEXT@ +OPENSSL_CFLAGS = @OPENSSL_CFLAGS@ +OPENSSL_LIBS = @OPENSSL_LIBS@ ORACLE_CFLAGS = @ORACLE_CFLAGS@ ORACLE_LIBS = @ORACLE_LIBS@ OTOOL = @OTOOL@ @@ -389,22 +391,25 @@ distclean-compile: @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/pmaixforwardedfrom_la-pmaixforwardedfrom.Plo@am__quote@ .c.o: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(COMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ $< -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Po +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.o$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c $< +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ $< .c.obj: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(COMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ `$(CYGPATH_W) '$<'` -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Po +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.obj$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ `$(CYGPATH_W) '$<'` &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c `$(CYGPATH_W) '$<'` +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ `$(CYGPATH_W) '$<'` .c.lo: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(LTCOMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ $< -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Plo +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.lo$$||'`;\ +@am__fastdepCC_TRUE@ $(LTCOMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Plo @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=yes @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ @am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(LTCOMPILE) -c -o $@ $< diff --git a/plugins/pmaixforwardedfrom/pmaixforwardedfrom.c b/plugins/pmaixforwardedfrom/pmaixforwardedfrom.c index 76198e9..30d46c2 100644 --- a/plugins/pmaixforwardedfrom/pmaixforwardedfrom.c +++ b/plugins/pmaixforwardedfrom/pmaixforwardedfrom.c @@ -69,7 +69,6 @@ ENDisCompatibleWithFeature BEGINparse uchar *p2parse; - uchar *opening; int lenMsg; #define OpeningText "Message forwarded from " CODESTARTparse diff --git a/plugins/pmciscoios/Makefile.am b/plugins/pmciscoios/Makefile.am new file mode 100644 index 0000000..23a08ba --- /dev/null +++ b/plugins/pmciscoios/Makefile.am @@ -0,0 +1,8 @@ +pkglib_LTLIBRARIES = pmciscoios.la + +pmciscoios_la_SOURCES = pmciscoios.c +pmciscoios_la_CPPFLAGS = $(RSRT_CFLAGS) $(PTHREADS_CFLAGS) -I ../../tools +pmciscoios_la_LDFLAGS = -module -avoid-version +pmciscoios_la_LIBADD = + +EXTRA_DIST = diff --git a/plugins/pmciscoios/Makefile.in b/plugins/pmciscoios/Makefile.in new file mode 100644 index 0000000..f2cfa5e --- /dev/null +++ b/plugins/pmciscoios/Makefile.in @@ -0,0 +1,637 @@ +# Makefile.in generated by automake 1.11.3 from Makefile.am. +# @configure_input@ + +# Copyright (C) 1994, 1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002, +# 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010, 2011 Free Software +# Foundation, Inc. +# This Makefile.in is free software; the Free Software Foundation +# gives unlimited permission to copy and/or distribute it, +# with or without modifications, as long as this notice is preserved. + +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY, to the extent permitted by law; without +# even the implied warranty of MERCHANTABILITY or FITNESS FOR A +# PARTICULAR PURPOSE. + +@SET_MAKE@ + +VPATH = @srcdir@ +pkgdatadir = $(datadir)/@PACKAGE@ +pkgincludedir = $(includedir)/@PACKAGE@ +pkglibdir = $(libdir)/@PACKAGE@ +pkglibexecdir = $(libexecdir)/@PACKAGE@ +am__cd = CDPATH="$${ZSH_VERSION+.}$(PATH_SEPARATOR)" && cd +install_sh_DATA = $(install_sh) -c -m 644 +install_sh_PROGRAM = $(install_sh) -c +install_sh_SCRIPT = $(install_sh) -c +INSTALL_HEADER = $(INSTALL_DATA) +transform = $(program_transform_name) +NORMAL_INSTALL = : +PRE_INSTALL = : +POST_INSTALL = : +NORMAL_UNINSTALL = : +PRE_UNINSTALL = : +POST_UNINSTALL = : +build_triplet = @build@ +host_triplet = @host@ +subdir = plugins/pmciscoios +DIST_COMMON = $(srcdir)/Makefile.am $(srcdir)/Makefile.in +ACLOCAL_M4 = $(top_srcdir)/aclocal.m4 +am__aclocal_m4_deps = $(top_srcdir)/m4/atomic_operations.m4 \ + $(top_srcdir)/m4/atomic_operations_64bit.m4 \ + $(top_srcdir)/m4/libtool.m4 $(top_srcdir)/m4/ltoptions.m4 \ + $(top_srcdir)/m4/ltsugar.m4 $(top_srcdir)/m4/ltversion.m4 \ + $(top_srcdir)/m4/lt~obsolete.m4 $(top_srcdir)/configure.ac +am__configure_deps = $(am__aclocal_m4_deps) $(CONFIGURE_DEPENDENCIES) \ + $(ACLOCAL_M4) +mkinstalldirs = $(install_sh) -d +CONFIG_HEADER = $(top_builddir)/config.h +CONFIG_CLEAN_FILES = +CONFIG_CLEAN_VPATH_FILES = +am__vpath_adj_setup = srcdirstrip=`echo "$(srcdir)" | sed 's|.|.|g'`; +am__vpath_adj = case $$p in \ + $(srcdir)/*) f=`echo "$$p" | sed "s|^$$srcdirstrip/||"`;; \ + *) f=$$p;; \ + esac; +am__strip_dir = f=`echo $$p | sed -e 's|^.*/||'`; +am__install_max = 40 +am__nobase_strip_setup = \ + srcdirstrip=`echo "$(srcdir)" | sed 's/[].[^$$\\*|]/\\\\&/g'` +am__nobase_strip = \ + for p in $$list; do echo "$$p"; done | sed -e "s|$$srcdirstrip/||" +am__nobase_list = $(am__nobase_strip_setup); \ + for p in $$list; do echo "$$p $$p"; done | \ + sed "s| $$srcdirstrip/| |;"' / .*\//!s/ .*/ ./; s,\( .*\)/[^/]*$$,\1,' | \ + $(AWK) 'BEGIN { files["."] = "" } { files[$$2] = files[$$2] " " $$1; \ + if (++n[$$2] == $(am__install_max)) \ + { print $$2, files[$$2]; n[$$2] = 0; files[$$2] = "" } } \ + END { for (dir in files) print dir, files[dir] }' +am__base_list = \ + sed '$$!N;$$!N;$$!N;$$!N;$$!N;$$!N;$$!N;s/\n/ /g' | \ + sed '$$!N;$$!N;$$!N;$$!N;s/\n/ /g' +am__uninstall_files_from_dir = { \ + test -z "$$files" \ + || { test ! -d "$$dir" && test ! -f "$$dir" && test ! -r "$$dir"; } \ + || { echo " ( cd '$$dir' && rm -f" $$files ")"; \ + $(am__cd) "$$dir" && rm -f $$files; }; \ + } +am__installdirs = "$(DESTDIR)$(pkglibdir)" +LTLIBRARIES = $(pkglib_LTLIBRARIES) +pmciscoios_la_DEPENDENCIES = +am_pmciscoios_la_OBJECTS = pmciscoios_la-pmciscoios.lo +pmciscoios_la_OBJECTS = $(am_pmciscoios_la_OBJECTS) +AM_V_lt = $(am__v_lt_@AM_V@) +am__v_lt_ = $(am__v_lt_@AM_DEFAULT_V@) +am__v_lt_0 = --silent +pmciscoios_la_LINK = $(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \ + $(LIBTOOLFLAGS) --mode=link $(CCLD) $(AM_CFLAGS) $(CFLAGS) \ + $(pmciscoios_la_LDFLAGS) $(LDFLAGS) -o $@ +DEFAULT_INCLUDES = -I.@am__isrc@ -I$(top_builddir) +depcomp = $(SHELL) $(top_srcdir)/depcomp +am__depfiles_maybe = depfiles +am__mv = mv -f +COMPILE = $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(AM_CPPFLAGS) \ + $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) +LTCOMPILE = $(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \ + $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) \ + $(DEFAULT_INCLUDES) $(INCLUDES) $(AM_CPPFLAGS) $(CPPFLAGS) \ + $(AM_CFLAGS) $(CFLAGS) +AM_V_CC = $(am__v_CC_@AM_V@) +am__v_CC_ = $(am__v_CC_@AM_DEFAULT_V@) +am__v_CC_0 = @echo " CC " $@; +AM_V_at = $(am__v_at_@AM_V@) +am__v_at_ = $(am__v_at_@AM_DEFAULT_V@) +am__v_at_0 = @ +CCLD = $(CC) +LINK = $(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \ + $(LIBTOOLFLAGS) --mode=link $(CCLD) $(AM_CFLAGS) $(CFLAGS) \ + $(AM_LDFLAGS) $(LDFLAGS) -o $@ +AM_V_CCLD = $(am__v_CCLD_@AM_V@) +am__v_CCLD_ = $(am__v_CCLD_@AM_DEFAULT_V@) +am__v_CCLD_0 = @echo " CCLD " $@; +AM_V_GEN = $(am__v_GEN_@AM_V@) +am__v_GEN_ = $(am__v_GEN_@AM_DEFAULT_V@) +am__v_GEN_0 = @echo " GEN " $@; +SOURCES = $(pmciscoios_la_SOURCES) +DIST_SOURCES = $(pmciscoios_la_SOURCES) +ETAGS = etags +CTAGS = ctags +DISTFILES = $(DIST_COMMON) $(DIST_SOURCES) $(TEXINFOS) $(EXTRA_DIST) +ACLOCAL = @ACLOCAL@ +AMTAR = @AMTAR@ +AM_DEFAULT_VERBOSITY = @AM_DEFAULT_VERBOSITY@ +AR = @AR@ +AUTOCONF = @AUTOCONF@ +AUTOHEADER = @AUTOHEADER@ +AUTOMAKE = @AUTOMAKE@ +AWK = @AWK@ +CC = @CC@ +CCDEPMODE = @CCDEPMODE@ +CFLAGS = @CFLAGS@ +CPP = @CPP@ +CPPFLAGS = @CPPFLAGS@ +CURL_CFLAGS = @CURL_CFLAGS@ +CURL_LIBS = @CURL_LIBS@ +CYGPATH_W = @CYGPATH_W@ +CZMQ_CFLAGS = @CZMQ_CFLAGS@ +CZMQ_LIBS = @CZMQ_LIBS@ +DEFS = @DEFS@ +DEPDIR = @DEPDIR@ +DLLTOOL = @DLLTOOL@ +DL_LIBS = @DL_LIBS@ +DSYMUTIL = @DSYMUTIL@ +DUMPBIN = @DUMPBIN@ +ECHO_C = @ECHO_C@ +ECHO_N = @ECHO_N@ +ECHO_T = @ECHO_T@ +EGREP = @EGREP@ +EXEEXT = @EXEEXT@ +FGREP = @FGREP@ +GNUTLS_CFLAGS = @GNUTLS_CFLAGS@ +GNUTLS_LIBS = @GNUTLS_LIBS@ +GREP = @GREP@ +GSS_LIBS = @GSS_LIBS@ +GUARDTIME_CFLAGS = @GUARDTIME_CFLAGS@ +GUARDTIME_LIBS = @GUARDTIME_LIBS@ +HAVE_MYSQL_CONFIG = @HAVE_MYSQL_CONFIG@ +HAVE_ORACLE_CONFIG = @HAVE_ORACLE_CONFIG@ +HAVE_PGSQL_CONFIG = @HAVE_PGSQL_CONFIG@ +HIREDIS_CFLAGS = @HIREDIS_CFLAGS@ +HIREDIS_LIBS = @HIREDIS_LIBS@ +IMUDP_LIBS = @IMUDP_LIBS@ +INSTALL = @INSTALL@ +INSTALL_DATA = @INSTALL_DATA@ +INSTALL_PROGRAM = @INSTALL_PROGRAM@ +INSTALL_SCRIPT = @INSTALL_SCRIPT@ +INSTALL_STRIP_PROGRAM = @INSTALL_STRIP_PROGRAM@ +JSON_C_CFLAGS = @JSON_C_CFLAGS@ +JSON_C_LIBS = @JSON_C_LIBS@ +LD = @LD@ +LDFLAGS = @LDFLAGS@ +LEX = @LEX@ +LEXLIB = @LEXLIB@ +LEX_OUTPUT_ROOT = @LEX_OUTPUT_ROOT@ +LIBDBI_CFLAGS = @LIBDBI_CFLAGS@ +LIBDBI_LIBS = @LIBDBI_LIBS@ +LIBESTR_CFLAGS = @LIBESTR_CFLAGS@ +LIBESTR_LIBS = @LIBESTR_LIBS@ +LIBGCRYPT_CFLAGS = @LIBGCRYPT_CFLAGS@ +LIBGCRYPT_CONFIG = @LIBGCRYPT_CONFIG@ +LIBGCRYPT_LIBS = @LIBGCRYPT_LIBS@ +LIBLOGGING_CFLAGS = @LIBLOGGING_CFLAGS@ +LIBLOGGING_LIBS = @LIBLOGGING_LIBS@ +LIBLOGGING_STDLOG_CFLAGS = @LIBLOGGING_STDLOG_CFLAGS@ +LIBLOGGING_STDLOG_LIBS = @LIBLOGGING_STDLOG_LIBS@ +LIBLOGNORM_CFLAGS = @LIBLOGNORM_CFLAGS@ +LIBLOGNORM_LIBS = @LIBLOGNORM_LIBS@ +LIBM = @LIBM@ +LIBMONGO_CLIENT_CFLAGS = @LIBMONGO_CLIENT_CFLAGS@ +LIBMONGO_CLIENT_LIBS = @LIBMONGO_CLIENT_LIBS@ +LIBOBJS = @LIBOBJS@ +LIBS = @LIBS@ +LIBSYSTEMD_JOURNAL_CFLAGS = @LIBSYSTEMD_JOURNAL_CFLAGS@ +LIBSYSTEMD_JOURNAL_LIBS = @LIBSYSTEMD_JOURNAL_LIBS@ +LIBTOOL = @LIBTOOL@ +LIBUUID_CFLAGS = @LIBUUID_CFLAGS@ +LIBUUID_LIBS = @LIBUUID_LIBS@ +LIPO = @LIPO@ +LN_S = @LN_S@ +LTLIBOBJS = @LTLIBOBJS@ +MAKEINFO = @MAKEINFO@ +MANIFEST_TOOL = @MANIFEST_TOOL@ +MKDIR_P = @MKDIR_P@ +MYSQL_CFLAGS = @MYSQL_CFLAGS@ +MYSQL_LIBS = @MYSQL_LIBS@ +NM = @NM@ +NMEDIT = @NMEDIT@ +OBJDUMP = @OBJDUMP@ +OBJEXT = @OBJEXT@ +OPENSSL_CFLAGS = @OPENSSL_CFLAGS@ +OPENSSL_LIBS = @OPENSSL_LIBS@ +ORACLE_CFLAGS = @ORACLE_CFLAGS@ +ORACLE_LIBS = @ORACLE_LIBS@ +OTOOL = @OTOOL@ +OTOOL64 = @OTOOL64@ +PACKAGE = @PACKAGE@ +PACKAGE_BUGREPORT = @PACKAGE_BUGREPORT@ +PACKAGE_NAME = @PACKAGE_NAME@ +PACKAGE_STRING = @PACKAGE_STRING@ +PACKAGE_TARNAME = @PACKAGE_TARNAME@ +PACKAGE_URL = @PACKAGE_URL@ +PACKAGE_VERSION = @PACKAGE_VERSION@ +PATH_SEPARATOR = @PATH_SEPARATOR@ +PGSQL_CFLAGS = @PGSQL_CFLAGS@ +PGSQL_LIBS = @PGSQL_LIBS@ +PKG_CONFIG = @PKG_CONFIG@ +PKG_CONFIG_LIBDIR = @PKG_CONFIG_LIBDIR@ +PKG_CONFIG_PATH = @PKG_CONFIG_PATH@ +PTHREADS_CFLAGS = @PTHREADS_CFLAGS@ +PTHREADS_LIBS = @PTHREADS_LIBS@ +RABBITMQ_CFLAGS = @RABBITMQ_CFLAGS@ +RABBITMQ_LIBS = @RABBITMQ_LIBS@ +RANLIB = @RANLIB@ +RELP_CFLAGS = @RELP_CFLAGS@ +RELP_LIBS = @RELP_LIBS@ +RSRT_CFLAGS = @RSRT_CFLAGS@ +RSRT_CFLAGS1 = @RSRT_CFLAGS1@ +RSRT_LIBS = @RSRT_LIBS@ +RSRT_LIBS1 = @RSRT_LIBS1@ +RST2MAN = @RST2MAN@ +RT_LIBS = @RT_LIBS@ +SED = @SED@ +SET_MAKE = @SET_MAKE@ +SHELL = @SHELL@ +SNMP_CFLAGS = @SNMP_CFLAGS@ +SNMP_LIBS = @SNMP_LIBS@ +SOL_LIBS = @SOL_LIBS@ +STRIP = @STRIP@ +UDPSPOOF_CFLAGS = @UDPSPOOF_CFLAGS@ +UDPSPOOF_LIBS = @UDPSPOOF_LIBS@ +VERSION = @VERSION@ +YACC = @YACC@ +YFLAGS = @YFLAGS@ +ZLIB_LIBS = @ZLIB_LIBS@ +abs_builddir = @abs_builddir@ +abs_srcdir = @abs_srcdir@ +abs_top_builddir = @abs_top_builddir@ +abs_top_srcdir = @abs_top_srcdir@ +ac_ct_AR = @ac_ct_AR@ +ac_ct_CC = @ac_ct_CC@ +ac_ct_DUMPBIN = @ac_ct_DUMPBIN@ +am__include = @am__include@ +am__leading_dot = @am__leading_dot@ +am__quote = @am__quote@ +am__tar = @am__tar@ +am__untar = @am__untar@ +bindir = @bindir@ +build = @build@ +build_alias = @build_alias@ +build_cpu = @build_cpu@ +build_os = @build_os@ +build_vendor = @build_vendor@ +builddir = @builddir@ +datadir = @datadir@ +datarootdir = @datarootdir@ +docdir = @docdir@ +dvidir = @dvidir@ +exec_prefix = @exec_prefix@ +have_valgrind = @have_valgrind@ +host = @host@ +host_alias = @host_alias@ +host_cpu = @host_cpu@ +host_os = @host_os@ +host_vendor = @host_vendor@ +htmldir = @htmldir@ +includedir = @includedir@ +infodir = @infodir@ +install_sh = @install_sh@ +libdir = @libdir@ +libexecdir = @libexecdir@ +localedir = @localedir@ +localstatedir = @localstatedir@ +mandir = @mandir@ +mkdir_p = @mkdir_p@ +moddirs = @moddirs@ +oldincludedir = @oldincludedir@ +pdfdir = @pdfdir@ +prefix = @prefix@ +program_transform_name = @program_transform_name@ +psdir = @psdir@ +sbindir = @sbindir@ +sharedstatedir = @sharedstatedir@ +srcdir = @srcdir@ +sysconfdir = @sysconfdir@ +systemdsystemunitdir = @systemdsystemunitdir@ +target_alias = @target_alias@ +top_build_prefix = @top_build_prefix@ +top_builddir = @top_builddir@ +top_srcdir = @top_srcdir@ +pkglib_LTLIBRARIES = pmciscoios.la +pmciscoios_la_SOURCES = pmciscoios.c +pmciscoios_la_CPPFLAGS = $(RSRT_CFLAGS) $(PTHREADS_CFLAGS) -I ../../tools +pmciscoios_la_LDFLAGS = -module -avoid-version +pmciscoios_la_LIBADD = +EXTRA_DIST = +all: all-am + +.SUFFIXES: +.SUFFIXES: .c .lo .o .obj +$(srcdir)/Makefile.in: $(srcdir)/Makefile.am $(am__configure_deps) + @for dep in $?; do \ + case '$(am__configure_deps)' in \ + *$$dep*) \ + ( cd $(top_builddir) && $(MAKE) $(AM_MAKEFLAGS) am--refresh ) \ + && { if test -f $@; then exit 0; else break; fi; }; \ + exit 1;; \ + esac; \ + done; \ + echo ' cd $(top_srcdir) && $(AUTOMAKE) --gnu plugins/pmciscoios/Makefile'; \ + $(am__cd) $(top_srcdir) && \ + $(AUTOMAKE) --gnu plugins/pmciscoios/Makefile +.PRECIOUS: Makefile +Makefile: $(srcdir)/Makefile.in $(top_builddir)/config.status + @case '$?' in \ + *config.status*) \ + cd $(top_builddir) && $(MAKE) $(AM_MAKEFLAGS) am--refresh;; \ + *) \ + echo ' cd $(top_builddir) && $(SHELL) ./config.status $(subdir)/$@ $(am__depfiles_maybe)'; \ + cd $(top_builddir) && $(SHELL) ./config.status $(subdir)/$@ $(am__depfiles_maybe);; \ + esac; + +$(top_builddir)/config.status: $(top_srcdir)/configure $(CONFIG_STATUS_DEPENDENCIES) + cd $(top_builddir) && $(MAKE) $(AM_MAKEFLAGS) am--refresh + +$(top_srcdir)/configure: $(am__configure_deps) + cd $(top_builddir) && $(MAKE) $(AM_MAKEFLAGS) am--refresh +$(ACLOCAL_M4): $(am__aclocal_m4_deps) + cd $(top_builddir) && $(MAKE) $(AM_MAKEFLAGS) am--refresh +$(am__aclocal_m4_deps): +install-pkglibLTLIBRARIES: $(pkglib_LTLIBRARIES) + @$(NORMAL_INSTALL) + test -z "$(pkglibdir)" || $(MKDIR_P) "$(DESTDIR)$(pkglibdir)" + @list='$(pkglib_LTLIBRARIES)'; test -n "$(pkglibdir)" || list=; \ + list2=; for p in $$list; do \ + if test -f $$p; then \ + list2="$$list2 $$p"; \ + else :; fi; \ + done; \ + test -z "$$list2" || { \ + echo " $(LIBTOOL) $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=install $(INSTALL) $(INSTALL_STRIP_FLAG) $$list2 '$(DESTDIR)$(pkglibdir)'"; \ + $(LIBTOOL) $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=install $(INSTALL) $(INSTALL_STRIP_FLAG) $$list2 "$(DESTDIR)$(pkglibdir)"; \ + } + +uninstall-pkglibLTLIBRARIES: + @$(NORMAL_UNINSTALL) + @list='$(pkglib_LTLIBRARIES)'; test -n "$(pkglibdir)" || list=; \ + for p in $$list; do \ + $(am__strip_dir) \ + echo " $(LIBTOOL) $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=uninstall rm -f '$(DESTDIR)$(pkglibdir)/$$f'"; \ + $(LIBTOOL) $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=uninstall rm -f "$(DESTDIR)$(pkglibdir)/$$f"; \ + done + +clean-pkglibLTLIBRARIES: + -test -z "$(pkglib_LTLIBRARIES)" || rm -f $(pkglib_LTLIBRARIES) + @list='$(pkglib_LTLIBRARIES)'; for p in $$list; do \ + dir="`echo $$p | sed -e 's|/[^/]*$$||'`"; \ + test "$$dir" != "$$p" || dir=.; \ + echo "rm -f \"$${dir}/so_locations\""; \ + rm -f "$${dir}/so_locations"; \ + done +pmciscoios.la: $(pmciscoios_la_OBJECTS) $(pmciscoios_la_DEPENDENCIES) $(EXTRA_pmciscoios_la_DEPENDENCIES) + $(AM_V_CCLD)$(pmciscoios_la_LINK) -rpath $(pkglibdir) $(pmciscoios_la_OBJECTS) $(pmciscoios_la_LIBADD) $(LIBS) + +mostlyclean-compile: + -rm -f *.$(OBJEXT) + +distclean-compile: + -rm -f *.tab.c + +@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/pmciscoios_la-pmciscoios.Plo@am__quote@ + +.c.o: +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.o$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ $< + +.c.obj: +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.obj$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ `$(CYGPATH_W) '$<'` &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ `$(CYGPATH_W) '$<'` + +.c.lo: +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.lo$$||'`;\ +@am__fastdepCC_TRUE@ $(LTCOMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Plo +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=yes @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(LTCOMPILE) -c -o $@ $< + +pmciscoios_la-pmciscoios.lo: pmciscoios.c +@am__fastdepCC_TRUE@ $(AM_V_CC)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(pmciscoios_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT pmciscoios_la-pmciscoios.lo -MD -MP -MF $(DEPDIR)/pmciscoios_la-pmciscoios.Tpo -c -o pmciscoios_la-pmciscoios.lo `test -f 'pmciscoios.c' || echo '$(srcdir)/'`pmciscoios.c +@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/pmciscoios_la-pmciscoios.Tpo $(DEPDIR)/pmciscoios_la-pmciscoios.Plo +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='pmciscoios.c' object='pmciscoios_la-pmciscoios.lo' libtool=yes @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(pmciscoios_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o pmciscoios_la-pmciscoios.lo `test -f 'pmciscoios.c' || echo '$(srcdir)/'`pmciscoios.c + +mostlyclean-libtool: + -rm -f *.lo + +clean-libtool: + -rm -rf .libs _libs + +ID: $(HEADERS) $(SOURCES) $(LISP) $(TAGS_FILES) + list='$(SOURCES) $(HEADERS) $(LISP) $(TAGS_FILES)'; \ + unique=`for i in $$list; do \ + if test -f "$$i"; then echo $$i; else echo $(srcdir)/$$i; fi; \ + done | \ + $(AWK) '{ files[$$0] = 1; nonempty = 1; } \ + END { if (nonempty) { for (i in files) print i; }; }'`; \ + mkid -fID $$unique +tags: TAGS + +TAGS: $(HEADERS) $(SOURCES) $(TAGS_DEPENDENCIES) \ + $(TAGS_FILES) $(LISP) + set x; \ + here=`pwd`; \ + list='$(SOURCES) $(HEADERS) $(LISP) $(TAGS_FILES)'; \ + unique=`for i in $$list; do \ + if test -f "$$i"; then echo $$i; else echo $(srcdir)/$$i; fi; \ + done | \ + $(AWK) '{ files[$$0] = 1; nonempty = 1; } \ + END { if (nonempty) { for (i in files) print i; }; }'`; \ + shift; \ + if test -z "$(ETAGS_ARGS)$$*$$unique"; then :; else \ + test -n "$$unique" || unique=$$empty_fix; \ + if test $$# -gt 0; then \ + $(ETAGS) $(ETAGSFLAGS) $(AM_ETAGSFLAGS) $(ETAGS_ARGS) \ + "$$@" $$unique; \ + else \ + $(ETAGS) $(ETAGSFLAGS) $(AM_ETAGSFLAGS) $(ETAGS_ARGS) \ + $$unique; \ + fi; \ + fi +ctags: CTAGS +CTAGS: $(HEADERS) $(SOURCES) $(TAGS_DEPENDENCIES) \ + $(TAGS_FILES) $(LISP) + list='$(SOURCES) $(HEADERS) $(LISP) $(TAGS_FILES)'; \ + unique=`for i in $$list; do \ + if test -f "$$i"; then echo $$i; else echo $(srcdir)/$$i; fi; \ + done | \ + $(AWK) '{ files[$$0] = 1; nonempty = 1; } \ + END { if (nonempty) { for (i in files) print i; }; }'`; \ + test -z "$(CTAGS_ARGS)$$unique" \ + || $(CTAGS) $(CTAGSFLAGS) $(AM_CTAGSFLAGS) $(CTAGS_ARGS) \ + $$unique + +GTAGS: + here=`$(am__cd) $(top_builddir) && pwd` \ + && $(am__cd) $(top_srcdir) \ + && gtags -i $(GTAGS_ARGS) "$$here" + +distclean-tags: + -rm -f TAGS ID GTAGS GRTAGS GSYMS GPATH tags + +distdir: $(DISTFILES) + @srcdirstrip=`echo "$(srcdir)" | sed 's/[].[^$$\\*]/\\\\&/g'`; \ + topsrcdirstrip=`echo "$(top_srcdir)" | sed 's/[].[^$$\\*]/\\\\&/g'`; \ + list='$(DISTFILES)'; \ + dist_files=`for file in $$list; do echo $$file; done | \ + sed -e "s|^$$srcdirstrip/||;t" \ + -e "s|^$$topsrcdirstrip/|$(top_builddir)/|;t"`; \ + case $$dist_files in \ + */*) $(MKDIR_P) `echo "$$dist_files" | \ + sed '/\//!d;s|^|$(distdir)/|;s,/[^/]*$$,,' | \ + sort -u` ;; \ + esac; \ + for file in $$dist_files; do \ + if test -f $$file || test -d $$file; then d=.; else d=$(srcdir); fi; \ + if test -d $$d/$$file; then \ + dir=`echo "/$$file" | sed -e 's,/[^/]*$$,,'`; \ + if test -d "$(distdir)/$$file"; then \ + find "$(distdir)/$$file" -type d ! -perm -700 -exec chmod u+rwx {} \;; \ + fi; \ + if test -d $(srcdir)/$$file && test $$d != $(srcdir); then \ + cp -fpR $(srcdir)/$$file "$(distdir)$$dir" || exit 1; \ + find "$(distdir)/$$file" -type d ! -perm -700 -exec chmod u+rwx {} \;; \ + fi; \ + cp -fpR $$d/$$file "$(distdir)$$dir" || exit 1; \ + else \ + test -f "$(distdir)/$$file" \ + || cp -p $$d/$$file "$(distdir)/$$file" \ + || exit 1; \ + fi; \ + done +check-am: all-am +check: check-am +all-am: Makefile $(LTLIBRARIES) +installdirs: + for dir in "$(DESTDIR)$(pkglibdir)"; do \ + test -z "$$dir" || $(MKDIR_P) "$$dir"; \ + done +install: install-am +install-exec: install-exec-am +install-data: install-data-am +uninstall: uninstall-am + +install-am: all-am + @$(MAKE) $(AM_MAKEFLAGS) install-exec-am install-data-am + +installcheck: installcheck-am +install-strip: + if test -z '$(STRIP)'; then \ + $(MAKE) $(AM_MAKEFLAGS) INSTALL_PROGRAM="$(INSTALL_STRIP_PROGRAM)" \ + install_sh_PROGRAM="$(INSTALL_STRIP_PROGRAM)" INSTALL_STRIP_FLAG=-s \ + install; \ + else \ + $(MAKE) $(AM_MAKEFLAGS) INSTALL_PROGRAM="$(INSTALL_STRIP_PROGRAM)" \ + install_sh_PROGRAM="$(INSTALL_STRIP_PROGRAM)" INSTALL_STRIP_FLAG=-s \ + "INSTALL_PROGRAM_ENV=STRIPPROG='$(STRIP)'" install; \ + fi +mostlyclean-generic: + +clean-generic: + +distclean-generic: + -test -z "$(CONFIG_CLEAN_FILES)" || rm -f $(CONFIG_CLEAN_FILES) + -test . = "$(srcdir)" || test -z "$(CONFIG_CLEAN_VPATH_FILES)" || rm -f $(CONFIG_CLEAN_VPATH_FILES) + +maintainer-clean-generic: + @echo "This command is intended for maintainers to use" + @echo "it deletes files that may require special tools to rebuild." +clean: clean-am + +clean-am: clean-generic clean-libtool clean-pkglibLTLIBRARIES \ + mostlyclean-am + +distclean: distclean-am + -rm -rf ./$(DEPDIR) + -rm -f Makefile +distclean-am: clean-am distclean-compile distclean-generic \ + distclean-tags + +dvi: dvi-am + +dvi-am: + +html: html-am + +html-am: + +info: info-am + +info-am: + +install-data-am: + +install-dvi: install-dvi-am + +install-dvi-am: + +install-exec-am: install-pkglibLTLIBRARIES + +install-html: install-html-am + +install-html-am: + +install-info: install-info-am + +install-info-am: + +install-man: + +install-pdf: install-pdf-am + +install-pdf-am: + +install-ps: install-ps-am + +install-ps-am: + +installcheck-am: + +maintainer-clean: maintainer-clean-am + -rm -rf ./$(DEPDIR) + -rm -f Makefile +maintainer-clean-am: distclean-am maintainer-clean-generic + +mostlyclean: mostlyclean-am + +mostlyclean-am: mostlyclean-compile mostlyclean-generic \ + mostlyclean-libtool + +pdf: pdf-am + +pdf-am: + +ps: ps-am + +ps-am: + +uninstall-am: uninstall-pkglibLTLIBRARIES + +.MAKE: install-am install-strip + +.PHONY: CTAGS GTAGS all all-am check check-am clean clean-generic \ + clean-libtool clean-pkglibLTLIBRARIES ctags distclean \ + distclean-compile distclean-generic distclean-libtool \ + distclean-tags distdir dvi dvi-am html html-am info info-am \ + install install-am install-data install-data-am install-dvi \ + install-dvi-am install-exec install-exec-am install-html \ + install-html-am install-info install-info-am install-man \ + install-pdf install-pdf-am install-pkglibLTLIBRARIES \ + install-ps install-ps-am install-strip installcheck \ + installcheck-am installdirs maintainer-clean \ + maintainer-clean-generic mostlyclean mostlyclean-compile \ + mostlyclean-generic mostlyclean-libtool pdf pdf-am ps ps-am \ + tags uninstall uninstall-am uninstall-pkglibLTLIBRARIES + + +# Tell versions [3.59,3.63) of GNU make to not export all variables. +# Otherwise a system limit (for SysV at least) may be exceeded. +.NOEXPORT: diff --git a/plugins/pmciscoios/pmciscoios.c b/plugins/pmciscoios/pmciscoios.c new file mode 100644 index 0000000..6dfb0fe --- /dev/null +++ b/plugins/pmciscoios/pmciscoios.c @@ -0,0 +1,255 @@ +/* pmrciscoios.c + * This is a parser module for CISCO IOS "syslog" format. + * + * File begun on 2014-07-07 by RGerhards + * + * Copyright 2014 Rainer Gerhards and Adiscon GmbH. + * + * This file is part of rsyslog. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * -or- + * see COPYING.ASL20 in the source distribution + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +#include "config.h" +#include "rsyslog.h" +#include <stdlib.h> +#include <string.h> +#include <assert.h> +#include <errno.h> +#include <ctype.h> +#include "syslogd.h" +#include "conf.h" +#include "syslogd-types.h" +#include "template.h" +#include "msg.h" +#include "module-template.h" +#include "glbl.h" +#include "errmsg.h" +#include "parser.h" +#include "datetime.h" +#include "unicode-helper.h" + +MODULE_TYPE_PARSER +MODULE_TYPE_NOKEEP +PARSER_NAME("rsyslog.ciscoios") +MODULE_CNFNAME("pmciscoios") + +/* internal structures */ +DEF_PMOD_STATIC_DATA +DEFobjCurrIf(errmsg) +DEFobjCurrIf(glbl) +DEFobjCurrIf(parser) +DEFobjCurrIf(datetime) + + +/* parser instance parameters */ +static struct cnfparamdescr parserpdescr[] = { + { "present.origin", eCmdHdlrBinary, 0 } +}; +static struct cnfparamblk parserpblk = + { CNFPARAMBLK_VERSION, + sizeof(parserpdescr)/sizeof(struct cnfparamdescr), + parserpdescr + }; + +struct instanceConf_s { + int bOriginPresent; /* is ORIGIN field present? */ +}; + +BEGINisCompatibleWithFeature +CODESTARTisCompatibleWithFeature + if(eFeat == sFEATUREAutomaticSanitazion) + iRet = RS_RET_OK; + if(eFeat == sFEATUREAutomaticPRIParsing) + iRet = RS_RET_OK; +ENDisCompatibleWithFeature + + +/* create input instance, set default paramters, and + * add it to the list of instances. + */ +static rsRetVal +createInstance(instanceConf_t **pinst) +{ + instanceConf_t *inst; + DEFiRet; + CHKmalloc(inst = MALLOC(sizeof(instanceConf_t))); + inst->bOriginPresent = 0; + *pinst = inst; +finalize_it: + RETiRet; +} + +BEGINnewParserInst + struct cnfparamvals *pvals; + int i; +CODESTARTnewParserInst + DBGPRINTF("newParserInst (pmciscoios)\n"); + + CHKiRet(createInstance(&inst)); + + if(lst == NULL) + FINALIZE; /* just set defaults, no param block! */ + + if((pvals = nvlstGetParams(lst, &parserpblk, NULL)) == NULL) { + ABORT_FINALIZE(RS_RET_MISSING_CNFPARAMS); + } + + if(Debug) { + dbgprintf("parser param blk in pmciscoios:\n"); + cnfparamsPrint(&parserpblk, pvals); + } + + for(i = 0 ; i < parserpblk.nParams ; ++i) { + if(!pvals[i].bUsed) + continue; + if(!strcmp(parserpblk.descr[i].name, "present.origin")) { + inst->bOriginPresent = (int) pvals[i].val.d.n; + } else { + dbgprintf("pmciscoios: program error, non-handled " + "param '%s'\n", parserpblk.descr[i].name); + } + } +finalize_it: +CODE_STD_FINALIZERnewParserInst + if(lst != NULL) + cnfparamvalsDestruct(pvals, &parserpblk); +ENDnewParserInst + + +BEGINfreeParserInst +CODESTARTfreeParserInst + dbgprintf("pmciscoios: free parser instance %p\n", pInst); +ENDfreeParserInst + + +BEGINparse2 + uchar *p2parse; + long long msgcounter; + int lenMsg; + int i; + int iHostname; + uchar bufParseTAG[512]; + uchar bufParseHOSTNAME[CONF_HOSTNAME_MAXSIZE]; /* used by origin */ +CODESTARTparse2 + DBGPRINTF("Message will now be parsed by pmciscoios\n"); + assert(pMsg != NULL); + assert(pMsg->pszRawMsg != NULL); + lenMsg = pMsg->iLenRawMsg - pMsg->offAfterPRI; /* note: offAfterPRI is already the number of PRI chars (do not add one!) */ + p2parse = pMsg->pszRawMsg + pMsg->offAfterPRI; /* point to start of text, after PRI */ + + /* first obtain the MESSAGE COUNTER. It must be numeric up until + * the ": " terminator sequence + */ + msgcounter = 0; + while(lenMsg > 0 && (*p2parse >= '0' && *p2parse <= '9') ) { + msgcounter = msgcounter * 10 + *p2parse - '0'; + ++p2parse, --lenMsg; + } + DBGPRINTF("pmciscoios: msgcntr %lld\n", msgcounter); + + /* delimiter check */ + if(lenMsg < 2 || *p2parse != ':' || *(p2parse+1) != ' ') { + DBGPRINTF("pmciscoios: fail after seqno: '%s'\n", p2parse); + ABORT_FINALIZE(RS_RET_COULD_NOT_PARSE); + } + p2parse += 2; + + /* ORIGIN (optional) */ + if(pInst->bOriginPresent) { + iHostname = 0; + while( lenMsg > 1 + && !(*p2parse == ':' && *(p2parse+1) == ' ') /* IPv6 is e.g. "::1" (loopback) */ + && iHostname < (int) sizeof(bufParseHOSTNAME) - 1 ) { + bufParseHOSTNAME[iHostname++] = *p2parse++; + --lenMsg; + } + bufParseHOSTNAME[iHostname] = '\0'; + /* delimiter check */ + if(lenMsg < 2 || *(p2parse+1) != ' ') { + DBGPRINTF("pmciscoios: fail after origin: '%s'\n", p2parse); + ABORT_FINALIZE(RS_RET_COULD_NOT_PARSE); + } + p2parse += 2; + } + + /* TIMESTAMP */ + if(datetime.ParseTIMESTAMP3164(&(pMsg->tTIMESTAMP), &p2parse, &lenMsg, PARSE3164_TZSTRING) == RS_RET_OK) { + if(pMsg->dfltTZ[0] != '\0') + applyDfltTZ(&pMsg->tTIMESTAMP, pMsg->dfltTZ); + } else { + DBGPRINTF("pmciscoios: fail at timestamp: '%s'\n", p2parse); + ABORT_FINALIZE(RS_RET_COULD_NOT_PARSE); + } + /* Note: date parser strips ": ", so we cannot do the delimiter check here */ + + /* parse SYSLOG TAG. must always start with '%', else we have a field mismatch */ + if(lenMsg < 1 || *p2parse != '%') { + DBGPRINTF("pmciscoios: fail at tag begin (no '%%'): '%s'\n", p2parse); + ABORT_FINALIZE(RS_RET_COULD_NOT_PARSE); + } + + i = 0; + while(lenMsg > 0 && *p2parse != ':' && *p2parse != ' ' && i < (int) sizeof(bufParseTAG) - 2) { + bufParseTAG[i++] = *p2parse++; + --lenMsg; + } + /* delimiter check */ + if(lenMsg < 2 || *p2parse != ':' || *(p2parse+1) != ' ') { + DBGPRINTF("pmciscoios: fail after tag: '%s'\n", p2parse); + ABORT_FINALIZE(RS_RET_COULD_NOT_PARSE); + } + + ++p2parse; + bufParseTAG[i++] = ':'; + bufParseTAG[i] = '\0'; /* terminate string */ + + /* if we reach this point, we have a wellformed message and can persist the values */ + MsgSetTAG(pMsg, bufParseTAG, i); + if(pInst->bOriginPresent) + MsgSetHOSTNAME(pMsg, bufParseHOSTNAME, iHostname); + MsgSetMSGoffs(pMsg, p2parse - pMsg->pszRawMsg); + setProtocolVersion(pMsg, MSG_LEGACY_PROTOCOL); +finalize_it: +ENDparse2 + + +BEGINmodExit +CODESTARTmodExit + /* release what we no longer need */ + objRelease(errmsg, CORE_COMPONENT); + objRelease(glbl, CORE_COMPONENT); + objRelease(parser, CORE_COMPONENT); + objRelease(datetime, CORE_COMPONENT); +ENDmodExit + + +BEGINqueryEtryPt +CODESTARTqueryEtryPt +CODEqueryEtryPt_STD_PMOD2_QUERIES +CODEqueryEtryPt_IsCompatibleWithFeature_IF_OMOD_QUERIES +ENDqueryEtryPt + + +BEGINmodInit() +CODESTARTmodInit + *ipIFVersProvided = CURR_MOD_IF_VERSION; /* we only support the current interface specification */ +CODEmodInit_QueryRegCFSLineHdlr + CHKiRet(objUse(glbl, CORE_COMPONENT)); + CHKiRet(objUse(errmsg, CORE_COMPONENT)); + CHKiRet(objUse(parser, CORE_COMPONENT)); + CHKiRet(objUse(datetime, CORE_COMPONENT)); + + DBGPRINTF("pmciscoios parser init called\n"); +ENDmodInit diff --git a/plugins/pmcisconames/Makefile.in b/plugins/pmcisconames/Makefile.in index 0ccd6ee..824e1fc 100644 --- a/plugins/pmcisconames/Makefile.in +++ b/plugins/pmcisconames/Makefile.in @@ -154,7 +154,6 @@ GREP = @GREP@ GSS_LIBS = @GSS_LIBS@ GUARDTIME_CFLAGS = @GUARDTIME_CFLAGS@ GUARDTIME_LIBS = @GUARDTIME_LIBS@ -HAVE_LIBGCRYPT_CONFIG = @HAVE_LIBGCRYPT_CONFIG@ HAVE_MYSQL_CONFIG = @HAVE_MYSQL_CONFIG@ HAVE_ORACLE_CONFIG = @HAVE_ORACLE_CONFIG@ HAVE_PGSQL_CONFIG = @HAVE_PGSQL_CONFIG@ @@ -175,14 +174,15 @@ LEXLIB = @LEXLIB@ LEX_OUTPUT_ROOT = @LEX_OUTPUT_ROOT@ LIBDBI_CFLAGS = @LIBDBI_CFLAGS@ LIBDBI_LIBS = @LIBDBI_LIBS@ -LIBEE_CFLAGS = @LIBEE_CFLAGS@ -LIBEE_LIBS = @LIBEE_LIBS@ LIBESTR_CFLAGS = @LIBESTR_CFLAGS@ LIBESTR_LIBS = @LIBESTR_LIBS@ LIBGCRYPT_CFLAGS = @LIBGCRYPT_CFLAGS@ +LIBGCRYPT_CONFIG = @LIBGCRYPT_CONFIG@ LIBGCRYPT_LIBS = @LIBGCRYPT_LIBS@ LIBLOGGING_CFLAGS = @LIBLOGGING_CFLAGS@ LIBLOGGING_LIBS = @LIBLOGGING_LIBS@ +LIBLOGGING_STDLOG_CFLAGS = @LIBLOGGING_STDLOG_CFLAGS@ +LIBLOGGING_STDLOG_LIBS = @LIBLOGGING_STDLOG_LIBS@ LIBLOGNORM_CFLAGS = @LIBLOGNORM_CFLAGS@ LIBLOGNORM_LIBS = @LIBLOGNORM_LIBS@ LIBM = @LIBM@ @@ -207,6 +207,8 @@ NM = @NM@ NMEDIT = @NMEDIT@ OBJDUMP = @OBJDUMP@ OBJEXT = @OBJEXT@ +OPENSSL_CFLAGS = @OPENSSL_CFLAGS@ +OPENSSL_LIBS = @OPENSSL_LIBS@ ORACLE_CFLAGS = @ORACLE_CFLAGS@ ORACLE_LIBS = @ORACLE_LIBS@ OTOOL = @OTOOL@ @@ -388,22 +390,25 @@ distclean-compile: @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/pmcisconames_la-pmcisconames.Plo@am__quote@ .c.o: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(COMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ $< -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Po +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.o$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c $< +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ $< .c.obj: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(COMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ `$(CYGPATH_W) '$<'` -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Po +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.obj$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ `$(CYGPATH_W) '$<'` &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c `$(CYGPATH_W) '$<'` +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ `$(CYGPATH_W) '$<'` .c.lo: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(LTCOMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ $< -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Plo +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.lo$$||'`;\ +@am__fastdepCC_TRUE@ $(LTCOMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Plo @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=yes @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ @am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(LTCOMPILE) -c -o $@ $< diff --git a/plugins/pmlastmsg/Makefile.in b/plugins/pmlastmsg/Makefile.in index 4f6b9e8..99e52c3 100644 --- a/plugins/pmlastmsg/Makefile.in +++ b/plugins/pmlastmsg/Makefile.in @@ -153,7 +153,6 @@ GREP = @GREP@ GSS_LIBS = @GSS_LIBS@ GUARDTIME_CFLAGS = @GUARDTIME_CFLAGS@ GUARDTIME_LIBS = @GUARDTIME_LIBS@ -HAVE_LIBGCRYPT_CONFIG = @HAVE_LIBGCRYPT_CONFIG@ HAVE_MYSQL_CONFIG = @HAVE_MYSQL_CONFIG@ HAVE_ORACLE_CONFIG = @HAVE_ORACLE_CONFIG@ HAVE_PGSQL_CONFIG = @HAVE_PGSQL_CONFIG@ @@ -174,14 +173,15 @@ LEXLIB = @LEXLIB@ LEX_OUTPUT_ROOT = @LEX_OUTPUT_ROOT@ LIBDBI_CFLAGS = @LIBDBI_CFLAGS@ LIBDBI_LIBS = @LIBDBI_LIBS@ -LIBEE_CFLAGS = @LIBEE_CFLAGS@ -LIBEE_LIBS = @LIBEE_LIBS@ LIBESTR_CFLAGS = @LIBESTR_CFLAGS@ LIBESTR_LIBS = @LIBESTR_LIBS@ LIBGCRYPT_CFLAGS = @LIBGCRYPT_CFLAGS@ +LIBGCRYPT_CONFIG = @LIBGCRYPT_CONFIG@ LIBGCRYPT_LIBS = @LIBGCRYPT_LIBS@ LIBLOGGING_CFLAGS = @LIBLOGGING_CFLAGS@ LIBLOGGING_LIBS = @LIBLOGGING_LIBS@ +LIBLOGGING_STDLOG_CFLAGS = @LIBLOGGING_STDLOG_CFLAGS@ +LIBLOGGING_STDLOG_LIBS = @LIBLOGGING_STDLOG_LIBS@ LIBLOGNORM_CFLAGS = @LIBLOGNORM_CFLAGS@ LIBLOGNORM_LIBS = @LIBLOGNORM_LIBS@ LIBM = @LIBM@ @@ -206,6 +206,8 @@ NM = @NM@ NMEDIT = @NMEDIT@ OBJDUMP = @OBJDUMP@ OBJEXT = @OBJEXT@ +OPENSSL_CFLAGS = @OPENSSL_CFLAGS@ +OPENSSL_LIBS = @OPENSSL_LIBS@ ORACLE_CFLAGS = @ORACLE_CFLAGS@ ORACLE_LIBS = @ORACLE_LIBS@ OTOOL = @OTOOL@ @@ -387,22 +389,25 @@ distclean-compile: @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/pmlastmsg_la-pmlastmsg.Plo@am__quote@ .c.o: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(COMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ $< -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Po +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.o$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c $< +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ $< .c.obj: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(COMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ `$(CYGPATH_W) '$<'` -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Po +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.obj$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ `$(CYGPATH_W) '$<'` &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c `$(CYGPATH_W) '$<'` +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ `$(CYGPATH_W) '$<'` .c.lo: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(LTCOMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ $< -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Plo +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.lo$$||'`;\ +@am__fastdepCC_TRUE@ $(LTCOMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Plo @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=yes @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ @am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(LTCOMPILE) -c -o $@ $< diff --git a/plugins/pmlastmsg/pmlastmsg.c b/plugins/pmlastmsg/pmlastmsg.c index a290c44..8c475fb 100644 --- a/plugins/pmlastmsg/pmlastmsg.c +++ b/plugins/pmlastmsg/pmlastmsg.c @@ -132,7 +132,7 @@ dbgprintf("wrong closing text!\n"); */ DBGPRINTF("pmlastmsg detected a \"last message repeated n times\" message\n"); - setProtocolVersion(pMsg, 0); + setProtocolVersion(pMsg, MSG_LEGACY_PROTOCOL); memcpy(&pMsg->tTIMESTAMP, &pMsg->tRcvdAt, sizeof(struct syslogTime)); MsgSetMSGoffs(pMsg, pMsg->offAfterPRI); /* we don't have a header! */ MsgSetTAG(pMsg, (uchar*)"", 0); diff --git a/plugins/pmrfc3164sd/Makefile.in b/plugins/pmrfc3164sd/Makefile.in index 1b54d3e..77aba41 100644 --- a/plugins/pmrfc3164sd/Makefile.in +++ b/plugins/pmrfc3164sd/Makefile.in @@ -154,7 +154,6 @@ GREP = @GREP@ GSS_LIBS = @GSS_LIBS@ GUARDTIME_CFLAGS = @GUARDTIME_CFLAGS@ GUARDTIME_LIBS = @GUARDTIME_LIBS@ -HAVE_LIBGCRYPT_CONFIG = @HAVE_LIBGCRYPT_CONFIG@ HAVE_MYSQL_CONFIG = @HAVE_MYSQL_CONFIG@ HAVE_ORACLE_CONFIG = @HAVE_ORACLE_CONFIG@ HAVE_PGSQL_CONFIG = @HAVE_PGSQL_CONFIG@ @@ -175,14 +174,15 @@ LEXLIB = @LEXLIB@ LEX_OUTPUT_ROOT = @LEX_OUTPUT_ROOT@ LIBDBI_CFLAGS = @LIBDBI_CFLAGS@ LIBDBI_LIBS = @LIBDBI_LIBS@ -LIBEE_CFLAGS = @LIBEE_CFLAGS@ -LIBEE_LIBS = @LIBEE_LIBS@ LIBESTR_CFLAGS = @LIBESTR_CFLAGS@ LIBESTR_LIBS = @LIBESTR_LIBS@ LIBGCRYPT_CFLAGS = @LIBGCRYPT_CFLAGS@ +LIBGCRYPT_CONFIG = @LIBGCRYPT_CONFIG@ LIBGCRYPT_LIBS = @LIBGCRYPT_LIBS@ LIBLOGGING_CFLAGS = @LIBLOGGING_CFLAGS@ LIBLOGGING_LIBS = @LIBLOGGING_LIBS@ +LIBLOGGING_STDLOG_CFLAGS = @LIBLOGGING_STDLOG_CFLAGS@ +LIBLOGGING_STDLOG_LIBS = @LIBLOGGING_STDLOG_LIBS@ LIBLOGNORM_CFLAGS = @LIBLOGNORM_CFLAGS@ LIBLOGNORM_LIBS = @LIBLOGNORM_LIBS@ LIBM = @LIBM@ @@ -207,6 +207,8 @@ NM = @NM@ NMEDIT = @NMEDIT@ OBJDUMP = @OBJDUMP@ OBJEXT = @OBJEXT@ +OPENSSL_CFLAGS = @OPENSSL_CFLAGS@ +OPENSSL_LIBS = @OPENSSL_LIBS@ ORACLE_CFLAGS = @ORACLE_CFLAGS@ ORACLE_LIBS = @ORACLE_LIBS@ OTOOL = @OTOOL@ @@ -388,22 +390,25 @@ distclean-compile: @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/pmrfc3164sd_la-pmrfc3164sd.Plo@am__quote@ .c.o: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(COMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ $< -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Po +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.o$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c $< +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ $< .c.obj: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(COMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ `$(CYGPATH_W) '$<'` -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Po +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.obj$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ `$(CYGPATH_W) '$<'` &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c `$(CYGPATH_W) '$<'` +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ `$(CYGPATH_W) '$<'` .c.lo: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(LTCOMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ $< -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Plo +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.lo$$||'`;\ +@am__fastdepCC_TRUE@ $(LTCOMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Plo @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=yes @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ @am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(LTCOMPILE) -c -o $@ $< diff --git a/plugins/pmrfc3164sd/pmrfc3164sd.c b/plugins/pmrfc3164sd/pmrfc3164sd.c index de5805b..0e69083 100644 --- a/plugins/pmrfc3164sd/pmrfc3164sd.c +++ b/plugins/pmrfc3164sd/pmrfc3164sd.c @@ -177,7 +177,7 @@ CODESTARTparse assert(pMsg->pszRawMsg != NULL); lenMsg = pMsg->iLenRawMsg - pMsg->offAfterPRI; /* note: offAfterPRI is already the number of PRI chars (do not add one!) */ p2parse = pMsg->pszRawMsg + pMsg->offAfterPRI; /* point to start of text, after PRI */ - setProtocolVersion(pMsg, 0); + setProtocolVersion(pMsg, MSG_LEGACY_PROTOCOL); /* Check to see if msg contains a timestamp. We start by assuming * that the message timestamp is the time of reception (which we @@ -187,12 +187,12 @@ CODESTARTparse */ if(datetime.ParseTIMESTAMP3339(&(pMsg->tTIMESTAMP), &p2parse, &lenMsg) == RS_RET_OK) { /* we are done - parse pointer is moved by ParseTIMESTAMP3339 */; - } else if(datetime.ParseTIMESTAMP3164(&(pMsg->tTIMESTAMP), &p2parse, &lenMsg) == RS_RET_OK) { + } else if(datetime.ParseTIMESTAMP3164(&(pMsg->tTIMESTAMP), &p2parse, &lenMsg, NO_PARSE3164_TZSTRING) == RS_RET_OK) { /* we are done - parse pointer is moved by ParseTIMESTAMP3164 */; } else if(*p2parse == ' ' && lenMsg > 1) { /* try to see if it is slighly malformed - HP procurve seems to do that sometimes */ ++p2parse; /* move over space */ --lenMsg; - if(datetime.ParseTIMESTAMP3164(&(pMsg->tTIMESTAMP), &p2parse, &lenMsg) == RS_RET_OK) { + if(datetime.ParseTIMESTAMP3164(&(pMsg->tTIMESTAMP), &p2parse, &lenMsg, NO_PARSE3164_TZSTRING) == RS_RET_OK) { /* indeed, we got it! */ /* we are done - parse pointer is moved by ParseTIMESTAMP3164 */; } else {/* parse pointer needs to be restored, as we moved it off-by-one diff --git a/plugins/pmsnare/Makefile.in b/plugins/pmsnare/Makefile.in index dc08469..a926a8f 100644 --- a/plugins/pmsnare/Makefile.in +++ b/plugins/pmsnare/Makefile.in @@ -153,7 +153,6 @@ GREP = @GREP@ GSS_LIBS = @GSS_LIBS@ GUARDTIME_CFLAGS = @GUARDTIME_CFLAGS@ GUARDTIME_LIBS = @GUARDTIME_LIBS@ -HAVE_LIBGCRYPT_CONFIG = @HAVE_LIBGCRYPT_CONFIG@ HAVE_MYSQL_CONFIG = @HAVE_MYSQL_CONFIG@ HAVE_ORACLE_CONFIG = @HAVE_ORACLE_CONFIG@ HAVE_PGSQL_CONFIG = @HAVE_PGSQL_CONFIG@ @@ -174,14 +173,15 @@ LEXLIB = @LEXLIB@ LEX_OUTPUT_ROOT = @LEX_OUTPUT_ROOT@ LIBDBI_CFLAGS = @LIBDBI_CFLAGS@ LIBDBI_LIBS = @LIBDBI_LIBS@ -LIBEE_CFLAGS = @LIBEE_CFLAGS@ -LIBEE_LIBS = @LIBEE_LIBS@ LIBESTR_CFLAGS = @LIBESTR_CFLAGS@ LIBESTR_LIBS = @LIBESTR_LIBS@ LIBGCRYPT_CFLAGS = @LIBGCRYPT_CFLAGS@ +LIBGCRYPT_CONFIG = @LIBGCRYPT_CONFIG@ LIBGCRYPT_LIBS = @LIBGCRYPT_LIBS@ LIBLOGGING_CFLAGS = @LIBLOGGING_CFLAGS@ LIBLOGGING_LIBS = @LIBLOGGING_LIBS@ +LIBLOGGING_STDLOG_CFLAGS = @LIBLOGGING_STDLOG_CFLAGS@ +LIBLOGGING_STDLOG_LIBS = @LIBLOGGING_STDLOG_LIBS@ LIBLOGNORM_CFLAGS = @LIBLOGNORM_CFLAGS@ LIBLOGNORM_LIBS = @LIBLOGNORM_LIBS@ LIBM = @LIBM@ @@ -206,6 +206,8 @@ NM = @NM@ NMEDIT = @NMEDIT@ OBJDUMP = @OBJDUMP@ OBJEXT = @OBJEXT@ +OPENSSL_CFLAGS = @OPENSSL_CFLAGS@ +OPENSSL_LIBS = @OPENSSL_LIBS@ ORACLE_CFLAGS = @ORACLE_CFLAGS@ ORACLE_LIBS = @ORACLE_LIBS@ OTOOL = @OTOOL@ @@ -387,22 +389,25 @@ distclean-compile: @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/pmsnare_la-pmsnare.Plo@am__quote@ .c.o: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(COMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ $< -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Po +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.o$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c $< +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ $< .c.obj: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(COMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ `$(CYGPATH_W) '$<'` -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Po +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.obj$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ `$(CYGPATH_W) '$<'` &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c `$(CYGPATH_W) '$<'` +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ `$(CYGPATH_W) '$<'` .c.lo: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(LTCOMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ $< -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Plo +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.lo$$||'`;\ +@am__fastdepCC_TRUE@ $(LTCOMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Plo @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=yes @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ @am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(LTCOMPILE) -c -o $@ $< diff --git a/plugins/sm_cust_bindcdr/Makefile.in b/plugins/sm_cust_bindcdr/Makefile.in index 8474410..a9e36aa 100644 --- a/plugins/sm_cust_bindcdr/Makefile.in +++ b/plugins/sm_cust_bindcdr/Makefile.in @@ -35,7 +35,7 @@ POST_UNINSTALL = : build_triplet = @build@ host_triplet = @host@ subdir = plugins/sm_cust_bindcdr -DIST_COMMON = $(srcdir)/Makefile.am $(srcdir)/Makefile.in +DIST_COMMON = README $(srcdir)/Makefile.am $(srcdir)/Makefile.in ACLOCAL_M4 = $(top_srcdir)/aclocal.m4 am__aclocal_m4_deps = $(top_srcdir)/m4/atomic_operations.m4 \ $(top_srcdir)/m4/atomic_operations_64bit.m4 \ @@ -154,7 +154,6 @@ GREP = @GREP@ GSS_LIBS = @GSS_LIBS@ GUARDTIME_CFLAGS = @GUARDTIME_CFLAGS@ GUARDTIME_LIBS = @GUARDTIME_LIBS@ -HAVE_LIBGCRYPT_CONFIG = @HAVE_LIBGCRYPT_CONFIG@ HAVE_MYSQL_CONFIG = @HAVE_MYSQL_CONFIG@ HAVE_ORACLE_CONFIG = @HAVE_ORACLE_CONFIG@ HAVE_PGSQL_CONFIG = @HAVE_PGSQL_CONFIG@ @@ -175,14 +174,15 @@ LEXLIB = @LEXLIB@ LEX_OUTPUT_ROOT = @LEX_OUTPUT_ROOT@ LIBDBI_CFLAGS = @LIBDBI_CFLAGS@ LIBDBI_LIBS = @LIBDBI_LIBS@ -LIBEE_CFLAGS = @LIBEE_CFLAGS@ -LIBEE_LIBS = @LIBEE_LIBS@ LIBESTR_CFLAGS = @LIBESTR_CFLAGS@ LIBESTR_LIBS = @LIBESTR_LIBS@ LIBGCRYPT_CFLAGS = @LIBGCRYPT_CFLAGS@ +LIBGCRYPT_CONFIG = @LIBGCRYPT_CONFIG@ LIBGCRYPT_LIBS = @LIBGCRYPT_LIBS@ LIBLOGGING_CFLAGS = @LIBLOGGING_CFLAGS@ LIBLOGGING_LIBS = @LIBLOGGING_LIBS@ +LIBLOGGING_STDLOG_CFLAGS = @LIBLOGGING_STDLOG_CFLAGS@ +LIBLOGGING_STDLOG_LIBS = @LIBLOGGING_STDLOG_LIBS@ LIBLOGNORM_CFLAGS = @LIBLOGNORM_CFLAGS@ LIBLOGNORM_LIBS = @LIBLOGNORM_LIBS@ LIBM = @LIBM@ @@ -207,6 +207,8 @@ NM = @NM@ NMEDIT = @NMEDIT@ OBJDUMP = @OBJDUMP@ OBJEXT = @OBJEXT@ +OPENSSL_CFLAGS = @OPENSSL_CFLAGS@ +OPENSSL_LIBS = @OPENSSL_LIBS@ ORACLE_CFLAGS = @ORACLE_CFLAGS@ ORACLE_LIBS = @ORACLE_LIBS@ OTOOL = @OTOOL@ @@ -387,22 +389,25 @@ distclean-compile: @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/sm_cust_bindcdr_la-sm_cust_bindcdr.Plo@am__quote@ .c.o: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(COMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ $< -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Po +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.o$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c $< +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ $< .c.obj: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(COMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ `$(CYGPATH_W) '$<'` -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Po +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.obj$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ `$(CYGPATH_W) '$<'` &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c `$(CYGPATH_W) '$<'` +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ `$(CYGPATH_W) '$<'` .c.lo: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(LTCOMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ $< -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Plo +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.lo$$||'`;\ +@am__fastdepCC_TRUE@ $(LTCOMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Plo @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=yes @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ @am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(LTCOMPILE) -c -o $@ $< diff --git a/plugins/sm_cust_bindcdr/README b/plugins/sm_cust_bindcdr/README new file mode 100644 index 0000000..b1d28c5 --- /dev/null +++ b/plugins/sm_cust_bindcdr/README @@ -0,0 +1,10 @@ +This module was created for a custom project with a very specific use +case. It was included into the source tree as a sample if someone +would like to create similar work. I strongly doubt it has any +value to the community other than serving as an example. Thus, +I do not intend to maintain this module, except if a sponsor +comes up that actually wants this to be done. I prefer to +leave it in the source tree even if it is broken starting with v8. +If that creates a problem, I would simply delete it. + +Rainer Gerhards diff --git a/rsyslog.service.in b/rsyslog.service.in index 08a4870..cb629ee 100644 --- a/rsyslog.service.in +++ b/rsyslog.service.in @@ -1,10 +1,14 @@ [Unit] Description=System Logging Service +Requires=syslog.socket +Documentation=man:rsyslogd(8) +Documentation=http://www.rsyslog.com/doc/ [Service] +Type=notify ExecStart=@sbindir@/rsyslogd -n -Sockets=syslog.socket StandardOutput=null +Restart=on-failure [Install] WantedBy=multi-user.target diff --git a/runtime/Makefile.am b/runtime/Makefile.am index fe4afb0..d07409e 100644 --- a/runtime/Makefile.am +++ b/runtime/Makefile.am @@ -25,6 +25,8 @@ librsyslog_la_SOURCES = \ unlimited_select.h \ conf.c \ conf.h \ + janitor.c \ + janitor.h \ rsconf.c \ rsconf.h \ parser.h \ @@ -69,6 +71,8 @@ librsyslog_la_SOURCES = \ prop.h \ ratelimit.c \ ratelimit.h \ + lookup.c \ + lookup.h \ cfsysline.c \ cfsysline.h \ sd-daemon.c \ @@ -97,12 +101,13 @@ librsyslog_la_SOURCES = \ # if WITH_MODDIRS -librsyslog_la_CPPFLAGS = -DSD_EXPORT_SYMBOLS -D_PATH_MODDIR=\"$(pkglibdir)/:$(moddirs)\" $(PTHREADS_CFLAGS) $(LIBEE_CFLAGS) $(LIBGCRYPT_CFLAGS) -I\$(top_srcdir)/tools +librsyslog_la_CPPFLAGS = -DSD_EXPORT_SYMBOLS -D_PATH_MODDIR=\"$(pkglibdir)/:$(moddirs)\" else -librsyslog_la_CPPFLAGS = -DSD_EXPORT_SYMBOLS -D_PATH_MODDIR=\"$(pkglibdir)/\" -I$(top_srcdir) $(PTHREADS_CFLAGS) $(LIBEE_CFLAGS) $(LIBGCRYPT_CFLAGS) -I\$(top_srcdir)/tools -I\$(top_srcdir)/grammar +librsyslog_la_CPPFLAGS = -DSD_EXPORT_SYMBOLS -D_PATH_MODDIR=\"$(pkglibdir)/\" -I\$(top_srcdir) -I\$(top_srcdir)/grammar endif #librsyslog_la_LDFLAGS = -module -avoid-version -librsyslog_la_LIBADD = $(DL_LIBS) $(RT_LIBS) $(LIBGCRYPT_LIBS) $(LIBEE_LIBS) +librsyslog_la_CPPFLAGS += $(PTHREADS_CFLAGS) $(LIBUUID_CFLAGS) $(JSON_C_CFLAGS) ${LIBLOGGING_STDLOG_CFLAGS} -I\$(top_srcdir)/tools +librsyslog_la_LIBADD = $(DL_LIBS) $(RT_LIBS) $(LIBUUID_LIBS) $(JSON_C_LIBS) # # regular expression support diff --git a/runtime/Makefile.in b/runtime/Makefile.in index 81ef760..a3339d8 100644 --- a/runtime/Makefile.in +++ b/runtime/Makefile.in @@ -133,24 +133,26 @@ am__DEPENDENCIES_1 = librsyslog_la_DEPENDENCIES = $(am__DEPENDENCIES_1) \ $(am__DEPENDENCIES_1) $(am__DEPENDENCIES_1) \ $(am__DEPENDENCIES_1) +am__dirstamp = $(am__leading_dot)dirstamp am_librsyslog_la_OBJECTS = librsyslog_la-rsyslog.lo \ librsyslog_la-dnscache.lo librsyslog_la-glbl.lo \ - librsyslog_la-conf.lo librsyslog_la-rsconf.lo \ - librsyslog_la-parser.lo librsyslog_la-strgen.lo \ - librsyslog_la-msg.lo librsyslog_la-linkedlist.lo \ - librsyslog_la-objomsr.lo librsyslog_la-stringbuf.lo \ - librsyslog_la-datetime.lo librsyslog_la-srutils.lo \ - librsyslog_la-errmsg.lo librsyslog_la-debug.lo \ - librsyslog_la-obj.lo librsyslog_la-modules.lo \ - librsyslog_la-statsobj.lo librsyslog_la-stream.lo \ - librsyslog_la-var.lo librsyslog_la-wtp.lo librsyslog_la-wti.lo \ + librsyslog_la-conf.lo librsyslog_la-janitor.lo \ + librsyslog_la-rsconf.lo librsyslog_la-parser.lo \ + librsyslog_la-strgen.lo librsyslog_la-msg.lo \ + librsyslog_la-linkedlist.lo librsyslog_la-objomsr.lo \ + librsyslog_la-stringbuf.lo librsyslog_la-datetime.lo \ + librsyslog_la-srutils.lo librsyslog_la-errmsg.lo \ + librsyslog_la-debug.lo librsyslog_la-obj.lo \ + librsyslog_la-modules.lo librsyslog_la-statsobj.lo \ + librsyslog_la-stream.lo librsyslog_la-var.lo \ + librsyslog_la-wtp.lo librsyslog_la-wti.lo \ librsyslog_la-queue.lo librsyslog_la-ruleset.lo \ librsyslog_la-prop.lo librsyslog_la-ratelimit.lo \ - librsyslog_la-cfsysline.lo librsyslog_la-sd-daemon.lo \ - librsyslog_la-action.lo librsyslog_la-threads.lo \ - librsyslog_la-parse.lo librsyslog_la-hashtable.lo \ - librsyslog_la-hashtable_itr.lo librsyslog_la-outchannel.lo \ - librsyslog_la-template.lo + librsyslog_la-lookup.lo librsyslog_la-cfsysline.lo \ + librsyslog_la-sd-daemon.lo ../librsyslog_la-action.lo \ + ../librsyslog_la-threads.lo ../librsyslog_la-parse.lo \ + librsyslog_la-hashtable.lo librsyslog_la-hashtable_itr.lo \ + ../librsyslog_la-outchannel.lo ../librsyslog_la-template.lo librsyslog_la_OBJECTS = $(am_librsyslog_la_OBJECTS) @ENABLE_LIBGCRYPT_TRUE@lmcry_gcry_la_DEPENDENCIES = libgcry.la \ @ENABLE_LIBGCRYPT_TRUE@ $(am__DEPENDENCIES_1) @@ -325,7 +327,6 @@ GREP = @GREP@ GSS_LIBS = @GSS_LIBS@ GUARDTIME_CFLAGS = @GUARDTIME_CFLAGS@ GUARDTIME_LIBS = @GUARDTIME_LIBS@ -HAVE_LIBGCRYPT_CONFIG = @HAVE_LIBGCRYPT_CONFIG@ HAVE_MYSQL_CONFIG = @HAVE_MYSQL_CONFIG@ HAVE_ORACLE_CONFIG = @HAVE_ORACLE_CONFIG@ HAVE_PGSQL_CONFIG = @HAVE_PGSQL_CONFIG@ @@ -346,14 +347,15 @@ LEXLIB = @LEXLIB@ LEX_OUTPUT_ROOT = @LEX_OUTPUT_ROOT@ LIBDBI_CFLAGS = @LIBDBI_CFLAGS@ LIBDBI_LIBS = @LIBDBI_LIBS@ -LIBEE_CFLAGS = @LIBEE_CFLAGS@ -LIBEE_LIBS = @LIBEE_LIBS@ LIBESTR_CFLAGS = @LIBESTR_CFLAGS@ LIBESTR_LIBS = @LIBESTR_LIBS@ LIBGCRYPT_CFLAGS = @LIBGCRYPT_CFLAGS@ +LIBGCRYPT_CONFIG = @LIBGCRYPT_CONFIG@ LIBGCRYPT_LIBS = @LIBGCRYPT_LIBS@ LIBLOGGING_CFLAGS = @LIBLOGGING_CFLAGS@ LIBLOGGING_LIBS = @LIBLOGGING_LIBS@ +LIBLOGGING_STDLOG_CFLAGS = @LIBLOGGING_STDLOG_CFLAGS@ +LIBLOGGING_STDLOG_LIBS = @LIBLOGGING_STDLOG_LIBS@ LIBLOGNORM_CFLAGS = @LIBLOGNORM_CFLAGS@ LIBLOGNORM_LIBS = @LIBLOGNORM_LIBS@ LIBM = @LIBM@ @@ -378,6 +380,8 @@ NM = @NM@ NMEDIT = @NMEDIT@ OBJDUMP = @OBJDUMP@ OBJEXT = @OBJEXT@ +OPENSSL_CFLAGS = @OPENSSL_CFLAGS@ +OPENSSL_LIBS = @OPENSSL_LIBS@ ORACLE_CFLAGS = @ORACLE_CFLAGS@ ORACLE_LIBS = @ORACLE_LIBS@ OTOOL = @OTOOL@ @@ -502,6 +506,8 @@ librsyslog_la_SOURCES = \ unlimited_select.h \ conf.c \ conf.h \ + janitor.c \ + janitor.h \ rsconf.c \ rsconf.h \ parser.h \ @@ -546,6 +552,8 @@ librsyslog_la_SOURCES = \ prop.h \ ratelimit.c \ ratelimit.h \ + lookup.c \ + lookup.h \ cfsysline.c \ cfsysline.h \ sd-daemon.c \ @@ -570,13 +578,25 @@ librsyslog_la_SOURCES = \ ../template.c \ ../template.h -@WITH_MODDIRS_FALSE@librsyslog_la_CPPFLAGS = -DSD_EXPORT_SYMBOLS -D_PATH_MODDIR=\"$(pkglibdir)/\" -I$(top_srcdir) $(PTHREADS_CFLAGS) $(LIBEE_CFLAGS) $(LIBGCRYPT_CFLAGS) -I\$(top_srcdir)/tools -I\$(top_srcdir)/grammar +#librsyslog_la_LDFLAGS = -module -avoid-version +@WITH_MODDIRS_FALSE@librsyslog_la_CPPFLAGS = -DSD_EXPORT_SYMBOLS \ +@WITH_MODDIRS_FALSE@ -D_PATH_MODDIR=\"$(pkglibdir)/\" \ +@WITH_MODDIRS_FALSE@ -I\$(top_srcdir) -I\$(top_srcdir)/grammar \ +@WITH_MODDIRS_FALSE@ $(PTHREADS_CFLAGS) $(LIBUUID_CFLAGS) \ +@WITH_MODDIRS_FALSE@ $(JSON_C_CFLAGS) \ +@WITH_MODDIRS_FALSE@ ${LIBLOGGING_STDLOG_CFLAGS} \ +@WITH_MODDIRS_FALSE@ -I\$(top_srcdir)/tools # the files with ../ we need to work on - so that they either become part of the # runtime or will no longer be needed. -- rgerhards, 2008-06-13 # -@WITH_MODDIRS_TRUE@librsyslog_la_CPPFLAGS = -DSD_EXPORT_SYMBOLS -D_PATH_MODDIR=\"$(pkglibdir)/:$(moddirs)\" $(PTHREADS_CFLAGS) $(LIBEE_CFLAGS) $(LIBGCRYPT_CFLAGS) -I\$(top_srcdir)/tools #librsyslog_la_LDFLAGS = -module -avoid-version -librsyslog_la_LIBADD = $(DL_LIBS) $(RT_LIBS) $(LIBGCRYPT_LIBS) $(LIBEE_LIBS) +@WITH_MODDIRS_TRUE@librsyslog_la_CPPFLAGS = -DSD_EXPORT_SYMBOLS \ +@WITH_MODDIRS_TRUE@ -D_PATH_MODDIR=\"$(pkglibdir)/:$(moddirs)\" \ +@WITH_MODDIRS_TRUE@ $(PTHREADS_CFLAGS) $(LIBUUID_CFLAGS) \ +@WITH_MODDIRS_TRUE@ $(JSON_C_CFLAGS) \ +@WITH_MODDIRS_TRUE@ ${LIBLOGGING_STDLOG_CFLAGS} \ +@WITH_MODDIRS_TRUE@ -I\$(top_srcdir)/tools +librsyslog_la_LIBADD = $(DL_LIBS) $(RT_LIBS) $(LIBUUID_LIBS) $(JSON_C_LIBS) @ENABLE_REGEXP_TRUE@lmregexp_la_SOURCES = regexp.c regexp.h @ENABLE_REGEXP_TRUE@lmregexp_la_CPPFLAGS = $(PTHREADS_CFLAGS) $(RSRT_CFLAGS) @ENABLE_REGEXP_TRUE@lmregexp_la_LDFLAGS = -module -avoid-version @@ -706,6 +726,22 @@ libgcry.la: $(libgcry_la_OBJECTS) $(libgcry_la_DEPENDENCIES) $(EXTRA_libgcry_la_ $(AM_V_CCLD)$(LINK) $(am_libgcry_la_rpath) $(libgcry_la_OBJECTS) $(libgcry_la_LIBADD) $(LIBS) librsgt.la: $(librsgt_la_OBJECTS) $(librsgt_la_DEPENDENCIES) $(EXTRA_librsgt_la_DEPENDENCIES) $(AM_V_CCLD)$(LINK) $(am_librsgt_la_rpath) $(librsgt_la_OBJECTS) $(librsgt_la_LIBADD) $(LIBS) +../$(am__dirstamp): + @$(MKDIR_P) .. + @: > ../$(am__dirstamp) +../$(DEPDIR)/$(am__dirstamp): + @$(MKDIR_P) ../$(DEPDIR) + @: > ../$(DEPDIR)/$(am__dirstamp) +../librsyslog_la-action.lo: ../$(am__dirstamp) \ + ../$(DEPDIR)/$(am__dirstamp) +../librsyslog_la-threads.lo: ../$(am__dirstamp) \ + ../$(DEPDIR)/$(am__dirstamp) +../librsyslog_la-parse.lo: ../$(am__dirstamp) \ + ../$(DEPDIR)/$(am__dirstamp) +../librsyslog_la-outchannel.lo: ../$(am__dirstamp) \ + ../$(DEPDIR)/$(am__dirstamp) +../librsyslog_la-template.lo: ../$(am__dirstamp) \ + ../$(DEPDIR)/$(am__dirstamp) librsyslog.la: $(librsyslog_la_OBJECTS) $(librsyslog_la_DEPENDENCIES) $(EXTRA_librsyslog_la_DEPENDENCIES) $(AM_V_CCLD)$(LINK) $(librsyslog_la_OBJECTS) $(librsyslog_la_LIBADD) $(LIBS) lmcry_gcry.la: $(lmcry_gcry_la_OBJECTS) $(lmcry_gcry_la_DEPENDENCIES) $(EXTRA_lmcry_gcry_la_DEPENDENCIES) @@ -772,15 +808,29 @@ clean-sbinPROGRAMS: mostlyclean-compile: -rm -f *.$(OBJEXT) + -rm -f ../librsyslog_la-action.$(OBJEXT) + -rm -f ../librsyslog_la-action.lo + -rm -f ../librsyslog_la-outchannel.$(OBJEXT) + -rm -f ../librsyslog_la-outchannel.lo + -rm -f ../librsyslog_la-parse.$(OBJEXT) + -rm -f ../librsyslog_la-parse.lo + -rm -f ../librsyslog_la-template.$(OBJEXT) + -rm -f ../librsyslog_la-template.lo + -rm -f ../librsyslog_la-threads.$(OBJEXT) + -rm -f ../librsyslog_la-threads.lo distclean-compile: -rm -f *.tab.c +@AMDEP_TRUE@@am__include@ @am__quote@../$(DEPDIR)/librsyslog_la-action.Plo@am__quote@ +@AMDEP_TRUE@@am__include@ @am__quote@../$(DEPDIR)/librsyslog_la-outchannel.Plo@am__quote@ +@AMDEP_TRUE@@am__include@ @am__quote@../$(DEPDIR)/librsyslog_la-parse.Plo@am__quote@ +@AMDEP_TRUE@@am__include@ @am__quote@../$(DEPDIR)/librsyslog_la-template.Plo@am__quote@ +@AMDEP_TRUE@@am__include@ @am__quote@../$(DEPDIR)/librsyslog_la-threads.Plo@am__quote@ @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/libgcry_la-libgcry.Plo@am__quote@ @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/libgcry_la-libgcry_common.Plo@am__quote@ @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/librsgt.Plo@am__quote@ @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/librsgt_read.Plo@am__quote@ -@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/librsyslog_la-action.Plo@am__quote@ @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/librsyslog_la-cfsysline.Plo@am__quote@ @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/librsyslog_la-conf.Plo@am__quote@ @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/librsyslog_la-datetime.Plo@am__quote@ @@ -790,13 +840,13 @@ distclean-compile: @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/librsyslog_la-glbl.Plo@am__quote@ @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/librsyslog_la-hashtable.Plo@am__quote@ @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/librsyslog_la-hashtable_itr.Plo@am__quote@ +@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/librsyslog_la-janitor.Plo@am__quote@ @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/librsyslog_la-linkedlist.Plo@am__quote@ +@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/librsyslog_la-lookup.Plo@am__quote@ @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/librsyslog_la-modules.Plo@am__quote@ @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/librsyslog_la-msg.Plo@am__quote@ @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/librsyslog_la-obj.Plo@am__quote@ @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/librsyslog_la-objomsr.Plo@am__quote@ -@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/librsyslog_la-outchannel.Plo@am__quote@ -@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/librsyslog_la-parse.Plo@am__quote@ @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/librsyslog_la-parser.Plo@am__quote@ @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/librsyslog_la-prop.Plo@am__quote@ @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/librsyslog_la-queue.Plo@am__quote@ @@ -810,8 +860,6 @@ distclean-compile: @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/librsyslog_la-stream.Plo@am__quote@ @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/librsyslog_la-strgen.Plo@am__quote@ @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/librsyslog_la-stringbuf.Plo@am__quote@ -@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/librsyslog_la-template.Plo@am__quote@ -@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/librsyslog_la-threads.Plo@am__quote@ @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/librsyslog_la-var.Plo@am__quote@ @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/librsyslog_la-wti.Plo@am__quote@ @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/librsyslog_la-wtp.Plo@am__quote@ @@ -833,22 +881,25 @@ distclean-compile: @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/lmzlibw_la-zlibw.Plo@am__quote@ .c.o: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(COMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ $< -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Po +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.o$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c $< +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ $< .c.obj: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(COMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ `$(CYGPATH_W) '$<'` -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Po +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.obj$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ `$(CYGPATH_W) '$<'` &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c `$(CYGPATH_W) '$<'` +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ `$(CYGPATH_W) '$<'` .c.lo: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(LTCOMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ $< -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Plo +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.lo$$||'`;\ +@am__fastdepCC_TRUE@ $(LTCOMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Plo @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=yes @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ @am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(LTCOMPILE) -c -o $@ $< @@ -895,6 +946,13 @@ librsyslog_la-conf.lo: conf.c @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ @am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(librsyslog_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o librsyslog_la-conf.lo `test -f 'conf.c' || echo '$(srcdir)/'`conf.c +librsyslog_la-janitor.lo: janitor.c +@am__fastdepCC_TRUE@ $(AM_V_CC)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(librsyslog_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT librsyslog_la-janitor.lo -MD -MP -MF $(DEPDIR)/librsyslog_la-janitor.Tpo -c -o librsyslog_la-janitor.lo `test -f 'janitor.c' || echo '$(srcdir)/'`janitor.c +@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/librsyslog_la-janitor.Tpo $(DEPDIR)/librsyslog_la-janitor.Plo +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='janitor.c' object='librsyslog_la-janitor.lo' libtool=yes @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(librsyslog_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o librsyslog_la-janitor.lo `test -f 'janitor.c' || echo '$(srcdir)/'`janitor.c + librsyslog_la-rsconf.lo: rsconf.c @am__fastdepCC_TRUE@ $(AM_V_CC)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(librsyslog_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT librsyslog_la-rsconf.lo -MD -MP -MF $(DEPDIR)/librsyslog_la-rsconf.Tpo -c -o librsyslog_la-rsconf.lo `test -f 'rsconf.c' || echo '$(srcdir)/'`rsconf.c @am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/librsyslog_la-rsconf.Tpo $(DEPDIR)/librsyslog_la-rsconf.Plo @@ -1049,6 +1107,13 @@ librsyslog_la-ratelimit.lo: ratelimit.c @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ @am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(librsyslog_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o librsyslog_la-ratelimit.lo `test -f 'ratelimit.c' || echo '$(srcdir)/'`ratelimit.c +librsyslog_la-lookup.lo: lookup.c +@am__fastdepCC_TRUE@ $(AM_V_CC)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(librsyslog_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT librsyslog_la-lookup.lo -MD -MP -MF $(DEPDIR)/librsyslog_la-lookup.Tpo -c -o librsyslog_la-lookup.lo `test -f 'lookup.c' || echo '$(srcdir)/'`lookup.c +@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/librsyslog_la-lookup.Tpo $(DEPDIR)/librsyslog_la-lookup.Plo +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='lookup.c' object='librsyslog_la-lookup.lo' libtool=yes @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(librsyslog_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o librsyslog_la-lookup.lo `test -f 'lookup.c' || echo '$(srcdir)/'`lookup.c + librsyslog_la-cfsysline.lo: cfsysline.c @am__fastdepCC_TRUE@ $(AM_V_CC)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(librsyslog_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT librsyslog_la-cfsysline.lo -MD -MP -MF $(DEPDIR)/librsyslog_la-cfsysline.Tpo -c -o librsyslog_la-cfsysline.lo `test -f 'cfsysline.c' || echo '$(srcdir)/'`cfsysline.c @am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/librsyslog_la-cfsysline.Tpo $(DEPDIR)/librsyslog_la-cfsysline.Plo @@ -1063,26 +1128,26 @@ librsyslog_la-sd-daemon.lo: sd-daemon.c @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ @am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(librsyslog_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o librsyslog_la-sd-daemon.lo `test -f 'sd-daemon.c' || echo '$(srcdir)/'`sd-daemon.c -librsyslog_la-action.lo: ../action.c -@am__fastdepCC_TRUE@ $(AM_V_CC)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(librsyslog_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT librsyslog_la-action.lo -MD -MP -MF $(DEPDIR)/librsyslog_la-action.Tpo -c -o librsyslog_la-action.lo `test -f '../action.c' || echo '$(srcdir)/'`../action.c -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/librsyslog_la-action.Tpo $(DEPDIR)/librsyslog_la-action.Plo -@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='../action.c' object='librsyslog_la-action.lo' libtool=yes @AMDEPBACKSLASH@ +../librsyslog_la-action.lo: ../action.c +@am__fastdepCC_TRUE@ $(AM_V_CC)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(librsyslog_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT ../librsyslog_la-action.lo -MD -MP -MF ../$(DEPDIR)/librsyslog_la-action.Tpo -c -o ../librsyslog_la-action.lo `test -f '../action.c' || echo '$(srcdir)/'`../action.c +@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) ../$(DEPDIR)/librsyslog_la-action.Tpo ../$(DEPDIR)/librsyslog_la-action.Plo +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='../action.c' object='../librsyslog_la-action.lo' libtool=yes @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(librsyslog_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o librsyslog_la-action.lo `test -f '../action.c' || echo '$(srcdir)/'`../action.c +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(librsyslog_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o ../librsyslog_la-action.lo `test -f '../action.c' || echo '$(srcdir)/'`../action.c -librsyslog_la-threads.lo: ../threads.c -@am__fastdepCC_TRUE@ $(AM_V_CC)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(librsyslog_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT librsyslog_la-threads.lo -MD -MP -MF $(DEPDIR)/librsyslog_la-threads.Tpo -c -o librsyslog_la-threads.lo `test -f '../threads.c' || echo '$(srcdir)/'`../threads.c -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/librsyslog_la-threads.Tpo $(DEPDIR)/librsyslog_la-threads.Plo -@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='../threads.c' object='librsyslog_la-threads.lo' libtool=yes @AMDEPBACKSLASH@ +../librsyslog_la-threads.lo: ../threads.c +@am__fastdepCC_TRUE@ $(AM_V_CC)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(librsyslog_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT ../librsyslog_la-threads.lo -MD -MP -MF ../$(DEPDIR)/librsyslog_la-threads.Tpo -c -o ../librsyslog_la-threads.lo `test -f '../threads.c' || echo '$(srcdir)/'`../threads.c +@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) ../$(DEPDIR)/librsyslog_la-threads.Tpo ../$(DEPDIR)/librsyslog_la-threads.Plo +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='../threads.c' object='../librsyslog_la-threads.lo' libtool=yes @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(librsyslog_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o librsyslog_la-threads.lo `test -f '../threads.c' || echo '$(srcdir)/'`../threads.c +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(librsyslog_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o ../librsyslog_la-threads.lo `test -f '../threads.c' || echo '$(srcdir)/'`../threads.c -librsyslog_la-parse.lo: ../parse.c -@am__fastdepCC_TRUE@ $(AM_V_CC)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(librsyslog_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT librsyslog_la-parse.lo -MD -MP -MF $(DEPDIR)/librsyslog_la-parse.Tpo -c -o librsyslog_la-parse.lo `test -f '../parse.c' || echo '$(srcdir)/'`../parse.c -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/librsyslog_la-parse.Tpo $(DEPDIR)/librsyslog_la-parse.Plo -@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='../parse.c' object='librsyslog_la-parse.lo' libtool=yes @AMDEPBACKSLASH@ +../librsyslog_la-parse.lo: ../parse.c +@am__fastdepCC_TRUE@ $(AM_V_CC)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(librsyslog_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT ../librsyslog_la-parse.lo -MD -MP -MF ../$(DEPDIR)/librsyslog_la-parse.Tpo -c -o ../librsyslog_la-parse.lo `test -f '../parse.c' || echo '$(srcdir)/'`../parse.c +@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) ../$(DEPDIR)/librsyslog_la-parse.Tpo ../$(DEPDIR)/librsyslog_la-parse.Plo +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='../parse.c' object='../librsyslog_la-parse.lo' libtool=yes @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(librsyslog_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o librsyslog_la-parse.lo `test -f '../parse.c' || echo '$(srcdir)/'`../parse.c +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(librsyslog_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o ../librsyslog_la-parse.lo `test -f '../parse.c' || echo '$(srcdir)/'`../parse.c librsyslog_la-hashtable.lo: hashtable.c @am__fastdepCC_TRUE@ $(AM_V_CC)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(librsyslog_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT librsyslog_la-hashtable.lo -MD -MP -MF $(DEPDIR)/librsyslog_la-hashtable.Tpo -c -o librsyslog_la-hashtable.lo `test -f 'hashtable.c' || echo '$(srcdir)/'`hashtable.c @@ -1098,19 +1163,19 @@ librsyslog_la-hashtable_itr.lo: hashtable_itr.c @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ @am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(librsyslog_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o librsyslog_la-hashtable_itr.lo `test -f 'hashtable_itr.c' || echo '$(srcdir)/'`hashtable_itr.c -librsyslog_la-outchannel.lo: ../outchannel.c -@am__fastdepCC_TRUE@ $(AM_V_CC)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(librsyslog_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT librsyslog_la-outchannel.lo -MD -MP -MF $(DEPDIR)/librsyslog_la-outchannel.Tpo -c -o librsyslog_la-outchannel.lo `test -f '../outchannel.c' || echo '$(srcdir)/'`../outchannel.c -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/librsyslog_la-outchannel.Tpo $(DEPDIR)/librsyslog_la-outchannel.Plo -@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='../outchannel.c' object='librsyslog_la-outchannel.lo' libtool=yes @AMDEPBACKSLASH@ +../librsyslog_la-outchannel.lo: ../outchannel.c +@am__fastdepCC_TRUE@ $(AM_V_CC)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(librsyslog_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT ../librsyslog_la-outchannel.lo -MD -MP -MF ../$(DEPDIR)/librsyslog_la-outchannel.Tpo -c -o ../librsyslog_la-outchannel.lo `test -f '../outchannel.c' || echo '$(srcdir)/'`../outchannel.c +@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) ../$(DEPDIR)/librsyslog_la-outchannel.Tpo ../$(DEPDIR)/librsyslog_la-outchannel.Plo +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='../outchannel.c' object='../librsyslog_la-outchannel.lo' libtool=yes @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(librsyslog_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o librsyslog_la-outchannel.lo `test -f '../outchannel.c' || echo '$(srcdir)/'`../outchannel.c +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(librsyslog_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o ../librsyslog_la-outchannel.lo `test -f '../outchannel.c' || echo '$(srcdir)/'`../outchannel.c -librsyslog_la-template.lo: ../template.c -@am__fastdepCC_TRUE@ $(AM_V_CC)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(librsyslog_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT librsyslog_la-template.lo -MD -MP -MF $(DEPDIR)/librsyslog_la-template.Tpo -c -o librsyslog_la-template.lo `test -f '../template.c' || echo '$(srcdir)/'`../template.c -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/librsyslog_la-template.Tpo $(DEPDIR)/librsyslog_la-template.Plo -@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='../template.c' object='librsyslog_la-template.lo' libtool=yes @AMDEPBACKSLASH@ +../librsyslog_la-template.lo: ../template.c +@am__fastdepCC_TRUE@ $(AM_V_CC)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(librsyslog_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT ../librsyslog_la-template.lo -MD -MP -MF ../$(DEPDIR)/librsyslog_la-template.Tpo -c -o ../librsyslog_la-template.lo `test -f '../template.c' || echo '$(srcdir)/'`../template.c +@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) ../$(DEPDIR)/librsyslog_la-template.Tpo ../$(DEPDIR)/librsyslog_la-template.Plo +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='../template.c' object='../librsyslog_la-template.lo' libtool=yes @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(librsyslog_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o librsyslog_la-template.lo `test -f '../template.c' || echo '$(srcdir)/'`../template.c +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(librsyslog_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o ../librsyslog_la-template.lo `test -f '../template.c' || echo '$(srcdir)/'`../template.c lmcry_gcry_la-lmcry_gcry.lo: lmcry_gcry.c @am__fastdepCC_TRUE@ $(AM_V_CC)$(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) $(LIBTOOLFLAGS) --mode=compile $(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(lmcry_gcry_la_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT lmcry_gcry_la-lmcry_gcry.lo -MD -MP -MF $(DEPDIR)/lmcry_gcry_la-lmcry_gcry.Tpo -c -o lmcry_gcry_la-lmcry_gcry.lo `test -f 'lmcry_gcry.c' || echo '$(srcdir)/'`lmcry_gcry.c @@ -1229,6 +1294,7 @@ mostlyclean-libtool: clean-libtool: -rm -rf .libs _libs + -rm -rf ../.libs ../_libs ID: $(HEADERS) $(SOURCES) $(LISP) $(TAGS_FILES) list='$(SOURCES) $(HEADERS) $(LISP) $(TAGS_FILES)'; \ @@ -1345,6 +1411,8 @@ clean-generic: distclean-generic: -test -z "$(CONFIG_CLEAN_FILES)" || rm -f $(CONFIG_CLEAN_FILES) -test . = "$(srcdir)" || test -z "$(CONFIG_CLEAN_VPATH_FILES)" || rm -f $(CONFIG_CLEAN_VPATH_FILES) + -rm -f ../$(DEPDIR)/$(am__dirstamp) + -rm -f ../$(am__dirstamp) maintainer-clean-generic: @echo "This command is intended for maintainers to use" @@ -1355,7 +1423,7 @@ clean-am: clean-generic clean-libtool clean-noinstLTLIBRARIES \ clean-pkglibLTLIBRARIES clean-sbinPROGRAMS mostlyclean-am distclean: distclean-am - -rm -rf ./$(DEPDIR) + -rm -rf ../$(DEPDIR) ./$(DEPDIR) -rm -f Makefile distclean-am: clean-am distclean-compile distclean-generic \ distclean-tags @@ -1401,7 +1469,7 @@ install-ps-am: installcheck-am: maintainer-clean: maintainer-clean-am - -rm -rf ./$(DEPDIR) + -rm -rf ../$(DEPDIR) ./$(DEPDIR) -rm -f Makefile maintainer-clean-am: distclean-am maintainer-clean-generic diff --git a/runtime/atomic.h b/runtime/atomic.h index 2a89558..d59858a 100644 --- a/runtime/atomic.h +++ b/runtime/atomic.h @@ -181,18 +181,18 @@ (*data) -= val; pthread_mutex_unlock(phlpmut); } -# define DEF_ATOMIC_HELPER_MUT(x) pthread_mutex_t x -# define INIT_ATOMIC_HELPER_MUT(x) pthread_mutex_init(&(x), NULL) -# define DESTROY_ATOMIC_HELPER_MUT(x) pthread_mutex_destroy(&(x)) +# define DEF_ATOMIC_HELPER_MUT(x) pthread_mutex_t x; +# define INIT_ATOMIC_HELPER_MUT(x) pthread_mutex_init(&(x), NULL); +# define DESTROY_ATOMIC_HELPER_MUT(x) pthread_mutex_destroy(&(x)); # define PREFER_ATOMIC_INC(data) ((void) ++data) #endif /* we need to handle 64bit atomics seperately as some platforms have - * 32 bit atomics, but not 64 biot ones... -- rgerhards, 2010-12-01 + * 32 bit atomics, but not 64 bit ones... -- rgerhards, 2010-12-01 */ -#ifdef HAVE_ATOMIC_BUILTINS_64BIT +#ifdef HAVE_ATOMIC_BUILTINS64 # define ATOMIC_INC_uint64(data, phlpmut) ((void) __sync_fetch_and_add(data, 1)) # define ATOMIC_DEC_unit64(data, phlpmut) ((void) __sync_sub_and_fetch(data, 1)) # define ATOMIC_INC_AND_FETCH_uint64(data, phlpmut) __sync_fetch_and_add(data, 1) @@ -221,9 +221,9 @@ return(val); } -# define DEF_ATOMIC_HELPER_MUT64(x) pthread_mutex_t x +# define DEF_ATOMIC_HELPER_MUT64(x) pthread_mutex_t x; # define INIT_ATOMIC_HELPER_MUT64(x) pthread_mutex_init(&(x), NULL) # define DESTROY_ATOMIC_HELPER_MUT64(x) pthread_mutex_destroy(&(x)) -#endif /* #ifdef HAVE_ATOMIC_BUILTINS_64BIT */ +#endif /* #ifdef HAVE_ATOMIC_BUILTINS64 */ #endif /* #ifndef INCLUDED_ATOMIC_H */ diff --git a/runtime/batch.h b/runtime/batch.h index 2ec0767..e3fa045 100644 --- a/runtime/batch.h +++ b/runtime/batch.h @@ -2,7 +2,7 @@ * I am not sure yet if this will become a full-blown object. For now, this header just * includes the object definition and is not accompanied by code. * - * Copyright 2009 by Rainer Gerhards and Adiscon GmbH. + * Copyright 2009-2013 by Rainer Gerhards and Adiscon GmbH. * * This file is part of the rsyslog runtime library. * @@ -46,17 +46,6 @@ typedef unsigned char batch_state_t; */ struct batch_obj_s { msg_t *pMsg; - /* work variables for action processing; these are reused for each action (or block of - * actions) - */ - sbool bPrevWasSuspended; - /* following are caches to save allocs if not absolutely necessary */ - uchar *staticActStrings[CONF_OMOD_NUMSTRINGS_MAXSIZE]; /**< for strings */ - /* a cache to save malloc(), if not absolutely necessary */ - void *staticActParams[CONF_OMOD_NUMSTRINGS_MAXSIZE]; /**< for anything else */ - size_t staticLenStrings[CONF_OMOD_NUMSTRINGS_MAXSIZE]; - /* and the same for the message length (if used) */ - /* end action work variables */ }; /* the batch @@ -77,11 +66,7 @@ struct batch_s { int maxElem; /* maximum number of elements that this batch supports */ int nElem; /* actual number of element in this entry */ int nElemDeq; /* actual number of elements dequeued (and thus to be deleted) - see comment above! */ - int iDoneUpTo; /* all messages below this index have state other than RDY */ qDeqID deqID; /* ID of dequeue operation that generated this batch */ - int *pbShutdownImmediate;/* end processing of this batch immediately if set to 1 */ - sbool *active; /* which messages are active for processing, NULL=all */ - sbool bSingleRuleset; /* do all msgs of this batch use a single ruleset? */ batch_obj_t *pElem; /* batch elements */ batch_state_t *eltState;/* state (array!) for individual objects. NOTE: we have moved this out of batch_obj_t because we @@ -93,27 +78,9 @@ struct batch_s { }; -/* some inline functions (we may move this off to an object .. or not) */ -static inline void -batchSetSingleRuleset(batch_t *pBatch, sbool val) { - pBatch->bSingleRuleset = val; -} - -/* get the batches ruleset (if we have a single ruleset) */ -static inline ruleset_t* -batchGetRuleset(batch_t *pBatch) { - return (pBatch->nElem > 0) ? pBatch->pElem[0].pMsg->pRuleset : NULL; -} - -/* get the ruleset of a specifc element of the batch (index not verified!) */ -static inline ruleset_t* -batchElemGetRuleset(batch_t *pBatch, int i) { - return pBatch->pElem[i].pMsg->pRuleset; -} - /* get number of msgs for this batch */ static inline int -batchNumMsgs(batch_t *pBatch) { +batchNumMsgs(const batch_t * const pBatch) { return pBatch->nElem; } @@ -123,7 +90,7 @@ batchNumMsgs(batch_t *pBatch) { * the state table. -- rgerhards, 2010-06-10 */ static inline void -batchSetElemState(batch_t *pBatch, int i, batch_state_t newState) { +batchSetElemState(batch_t * const pBatch, const int i, const batch_state_t newState) { if(pBatch->eltState[i] != BATCH_STATE_DISC) pBatch->eltState[i] = newState; } @@ -133,9 +100,8 @@ batchSetElemState(batch_t *pBatch, int i, batch_state_t newState) { * element index is valid. -- rgerhards, 2010-06-10 */ static inline int -batchIsValidElem(batch_t *pBatch, int i) { - return( (pBatch->eltState[i] != BATCH_STATE_DISC) - && (pBatch->active == NULL || pBatch->active[i])); +batchIsValidElem(const batch_t * const pBatch, const int i) { + return(pBatch->eltState[i] != BATCH_STATE_DISC); } @@ -144,17 +110,7 @@ batchIsValidElem(batch_t *pBatch, int i) { * object itself cannot be freed! -- rgerhards, 2010-06-15 */ static inline void -batchFree(batch_t *pBatch) { - int i; - int j; - for(i = 0 ; i < pBatch->maxElem ; ++i) { - for(j = 0 ; j < CONF_OMOD_NUMSTRINGS_MAXSIZE ; ++j) { - /* staticActParams MUST be freed immediately (if required), - * so we do not need to do that! - */ - free(pBatch->pElem[i].staticActStrings[j]); - } - } +batchFree(batch_t * const pBatch) { free(pBatch->pElem); free(pBatch->eltState); } @@ -165,13 +121,11 @@ batchFree(batch_t *pBatch) { * provided. -- rgerhards, 2010-06-15 */ static inline rsRetVal -batchInit(batch_t *pBatch, int maxElem) { +batchInit(batch_t *const pBatch, const int maxElem) { DEFiRet; - pBatch->iDoneUpTo = 0; pBatch->maxElem = maxElem; CHKmalloc(pBatch->pElem = calloc((size_t)maxElem, sizeof(batch_obj_t))); CHKmalloc(pBatch->eltState = calloc((size_t)maxElem, sizeof(batch_state_t))); - // TODO: replace calloc by inidividual writes? finalize_it: RETiRet; } @@ -179,7 +133,7 @@ finalize_it: /* primarily a helper for debug purposes, get human-readble name of state */ static inline char * -batchState2String(batch_state_t state) { +batchState2String(const batch_state_t state) { switch(state) { case BATCH_STATE_RDY: return "BATCH_STATE_RDY"; diff --git a/runtime/cfsysline.c b/runtime/cfsysline.c index a437b7f..2e7bd36 100644 --- a/runtime/cfsysline.c +++ b/runtime/cfsysline.c @@ -363,9 +363,12 @@ static rsRetVal doGetGID(uchar **pp, rsRetVal (*pSetHdlr)(void*, uid_t), void *p } do { + char *p; + /* Increase bufsize and try again.*/ bufSize *= 2; - CHKmalloc(stringBuf = realloc(stringBuf, bufSize)); + CHKmalloc(p = realloc(stringBuf, bufSize)); + stringBuf = p; err = getgrnam_r((char*)szName, &gBuf, stringBuf, bufSize, &pgBuf); } while((pgBuf == NULL) && (err == ERANGE)); @@ -522,15 +525,14 @@ finalize_it: static rsRetVal doGetWord(uchar **pp, rsRetVal (*pSetHdlr)(void*, uchar*), void *pVal) { DEFiRet; - cstr_t *pStrB; + cstr_t *pStrB = NULL; uchar *pNewVal; ASSERT(pp != NULL); ASSERT(*pp != NULL); CHKiRet(getWord(pp, &pStrB)); - CHKiRet(cstrConvSzStrAndDestruct(pStrB, &pNewVal, 0)); - pStrB = NULL; + CHKiRet(cstrConvSzStrAndDestruct(&pStrB, &pNewVal, 0)); DBGPRINTF("doGetWord: get newval '%s' (len %d), hdlr %p\n", pNewVal, (int) ustrlen(pNewVal), pSetHdlr); @@ -915,7 +917,7 @@ DEFFUNC_llExecFunc(unregHdlrsHeadExec) int iNumElts; /* first find element */ - iRet = llFindAndDelete(&(pListHdr->llCmdHdlrs), pParam); + CHKiRet(llFindAndDelete(&(pListHdr->llCmdHdlrs), pParam)); /* now go back and check how many elements are left */ CHKiRet(llGetNumElts(&(pListHdr->llCmdHdlrs), &iNumElts)); diff --git a/runtime/conf.c b/runtime/conf.c index c97391c..87600ad 100644 --- a/runtime/conf.c +++ b/runtime/conf.c @@ -129,6 +129,23 @@ finalize_it: } +/* remove leading spaces from name; this "fixes" some anomalies in + * getSubString(), but I was not brave enough to fix the former as + * it has many other callers... -- rgerhards, 2013-05-27 + */ +static inline void +ltrim(char *src) +{ + char *dst = src; + while(isspace(*src)) + ++src; /*SKIP*/; + if(dst != src) { + while(*src != '\0') + *dst++ = *src++; + *dst = '\0'; + } +} + /* parse and interpret a $-config line that starts with * a name (this is common code). It is parsed to the name * and then the proper sub-function is called to handle @@ -155,6 +172,7 @@ doNameLine(uchar **pp, void* pVal) errmsg.LogError(0, RS_RET_NOT_FOUND, "Invalid config line: could not extract name - line ignored"); ABORT_FINALIZE(RS_RET_NOT_FOUND); } + ltrim(szName); if(*p == ',') ++p; /* comma was eaten */ @@ -243,7 +261,7 @@ rsRetVal cflineParseTemplateName(uchar** pp, omodStringRequest_t *pOMSR, int iEn { uchar *p; uchar *tplName = NULL; - cstr_t *pStrB; + cstr_t *pStrB = NULL; DEFiRet; ASSERT(pp != NULL); @@ -276,14 +294,17 @@ rsRetVal cflineParseTemplateName(uchar** pp, omodStringRequest_t *pOMSR, int iEn ++p; } CHKiRet(cstrFinalize(pStrB)); - CHKiRet(cstrConvSzStrAndDestruct(pStrB, &tplName, 0)); + CHKiRet(cstrConvSzStrAndDestruct(&pStrB, &tplName, 0)); } CHKiRet(OMSRsetEntry(pOMSR, iEntry, tplName, iTplOpts)); finalize_it: - if(iRet != RS_RET_OK) + if(iRet != RS_RET_OK) { free(tplName); + if(pStrB != NULL) + cstrDestruct(&pStrB); + } *pp = p; @@ -468,110 +489,6 @@ rsRetVal DecodePRIFilter(uchar *pline, uchar pmask[]) } -/* Helper to cfline(). This function takes the filter part of a property - * based filter and decodes it. It processes the line up to the beginning - * of the action part. A pointer to that beginnig is passed back to the caller. - * rgerhards 2005-09-15 - */ -rsRetVal DecodePropFilter(uchar *pline, struct cnfstmt *stmt) -{ - rsParsObj *pPars; - cstr_t *pCSCompOp; - cstr_t *pCSPropName; - rsRetVal iRet; - int iOffset; /* for compare operations */ - - ASSERT(pline != NULL); - - dbgprintf("Decoding property-based filter '%s'\n", pline); - - /* create parser object starting with line string without leading colon */ - if((iRet = rsParsConstructFromSz(&pPars, pline+1)) != RS_RET_OK) { - errmsg.LogError(0, iRet, "Error %d constructing parser object - ignoring selector", iRet); - return(iRet); - } - - /* read property */ - iRet = parsDelimCStr(pPars, &pCSPropName, ',', 1, 1, 1); - if(iRet != RS_RET_OK) { - errmsg.LogError(0, iRet, "error %d parsing filter property - ignoring selector", iRet); - rsParsDestruct(pPars); - return(iRet); - } - iRet = propNameToID(pCSPropName, &stmt->d.s_propfilt.propID); - if(iRet != RS_RET_OK) { - errmsg.LogError(0, iRet, "error %d parsing filter property - ignoring selector", iRet); - rsParsDestruct(pPars); - return(iRet); - } - if(stmt->d.s_propfilt.propID == PROP_CEE) { - /* in CEE case, we need to preserve the actual property name */ - if((stmt->d.s_propfilt.propName = - es_newStrFromBuf((char*)cstrGetSzStrNoNULL(pCSPropName)+2, cstrLen(pCSPropName)-2)) == NULL) { - cstrDestruct(&pCSPropName); - return(RS_RET_ERR); - } - } - cstrDestruct(&pCSPropName); - - /* read operation */ - iRet = parsDelimCStr(pPars, &pCSCompOp, ',', 1, 1, 1); - if(iRet != RS_RET_OK) { - errmsg.LogError(0, iRet, "error %d compare operation property - ignoring selector", iRet); - rsParsDestruct(pPars); - return(iRet); - } - - /* we now first check if the condition is to be negated. To do so, we first - * must make sure we have at least one char in the param and then check the - * first one. - * rgerhards, 2005-09-26 - */ - if(rsCStrLen(pCSCompOp) > 0) { - if(*rsCStrGetBufBeg(pCSCompOp) == '!') { - stmt->d.s_propfilt.isNegated = 1; - iOffset = 1; /* ignore '!' */ - } else { - stmt->d.s_propfilt.isNegated = 0; - iOffset = 0; - } - } else { - stmt->d.s_propfilt.isNegated = 0; - iOffset = 0; - } - - if(!rsCStrOffsetSzStrCmp(pCSCompOp, iOffset, (uchar*) "contains", 8)) { - stmt->d.s_propfilt.operation = FIOP_CONTAINS; - } else if(!rsCStrOffsetSzStrCmp(pCSCompOp, iOffset, (uchar*) "isequal", 7)) { - stmt->d.s_propfilt.operation = FIOP_ISEQUAL; - } else if(!rsCStrOffsetSzStrCmp(pCSCompOp, iOffset, (uchar*) "isempty", 7)) { - stmt->d.s_propfilt.operation = FIOP_ISEMPTY; - } else if(!rsCStrOffsetSzStrCmp(pCSCompOp, iOffset, (uchar*) "startswith", 10)) { - stmt->d.s_propfilt.operation = FIOP_STARTSWITH; - } else if(!rsCStrOffsetSzStrCmp(pCSCompOp, iOffset, (unsigned char*) "regex", 5)) { - stmt->d.s_propfilt.operation = FIOP_REGEX; - } else if(!rsCStrOffsetSzStrCmp(pCSCompOp, iOffset, (unsigned char*) "ereregex", 8)) { - stmt->d.s_propfilt.operation = FIOP_EREREGEX; - } else { - errmsg.LogError(0, NO_ERRCODE, "error: invalid compare operation '%s' - ignoring selector", - (char*) rsCStrGetSzStrNoNULL(pCSCompOp)); - } - rsCStrDestruct(&pCSCompOp); /* no longer needed */ - - if(stmt->d.s_propfilt.operation != FIOP_ISEMPTY) { - /* read compare value */ - iRet = parsQuotedCStr(pPars, &stmt->d.s_propfilt.pCSCompValue); - if(iRet != RS_RET_OK) { - errmsg.LogError(0, iRet, "error %d compare value property - ignoring selector", iRet); - rsParsDestruct(pPars); - return(iRet); - } - } - - return rsParsDestruct(pPars); -} - - /* process the action part of a selector line * rgerhards, 2007-08-01 */ @@ -604,12 +521,10 @@ rsRetVal cflineDoAction(rsconf_t *conf, uchar **p, action_t **ppAction) bHadWarning = 1; iRet = RS_RET_OK; } - if(iRet == RS_RET_OK || iRet == RS_RET_SUSPENDED) { - if((iRet = addAction(&pAction, pMod, pModData, pOMSR, NULL, NULL, - (iRet == RS_RET_SUSPENDED)? 1 : 0)) == RS_RET_OK) { + if(iRet == RS_RET_OK) { + if((iRet = addAction(&pAction, pMod, pModData, pOMSR, NULL, NULL)) == RS_RET_OK) { /* here check if the module is compatible with select features * (currently, we have no such features!) */ - pAction->eState = ACT_STATE_RDY; /* action is enabled */ conf->actions.nbrActions++; /* one more active action! */ } break; diff --git a/runtime/conf.h b/runtime/conf.h index a1bb51a..cb95667 100644 --- a/runtime/conf.h +++ b/runtime/conf.h @@ -63,7 +63,6 @@ rsRetVal cflineParseTemplateName(uchar** pp, omodStringRequest_t *pOMSR, int iEn rsRetVal cflineParseFileName(uchar* p, uchar *pFileName, omodStringRequest_t *pOMSR, int iEntry, int iTplOpts, uchar *pszTpl); rsRetVal DecodePRIFilter(uchar *pline, uchar pmask[]); -rsRetVal DecodePropFilter(uchar *pline, struct cnfstmt *stmt); rsRetVal cflineDoAction(rsconf_t *conf, uchar **p, action_t **ppAction); extern EHostnameCmpMode eDfltHostnameCmpMode; extern cstr_t *pDfltHostnameCmp; diff --git a/runtime/cryprov.h b/runtime/cryprov.h index 8496b74..2742a4a 100644 --- a/runtime/cryprov.h +++ b/runtime/cryprov.h @@ -24,16 +24,25 @@ #ifndef INCLUDED_CRYPROV_H #define INCLUDED_CRYPROV_H -#include <gcrypt.h> +/* we unfortunately need to have two different param names depending on the + * context in which parameters are set. Other than (re/over)engineering the core + * interface, we just define some values to keep track of that. + */ +#define CRYPROV_PARAMTYPE_REGULAR 0 +#define CRYPROV_PARAMTYPE_DISK 1 /* interface */ BEGINinterface(cryprov) /* name must also be changed in ENDinterface macro! */ rsRetVal (*Construct)(void *ppThis); - rsRetVal (*SetCnfParam)(void *ppThis, struct nvlst *lst); + rsRetVal (*SetCnfParam)(void *ppThis, struct nvlst *lst, int paramType); rsRetVal (*Destruct)(void *ppThis); - rsRetVal (*OnFileOpen)(void *pThis, uchar *fn, void *pFileInstData); + rsRetVal (*OnFileOpen)(void *pThis, uchar *fn, void *pFileInstData, char openMode); rsRetVal (*Encrypt)(void *pFileInstData, uchar *buf, size_t *lenBuf); + rsRetVal (*Decrypt)(void *pFileInstData, uchar *buf, size_t *lenBuf); rsRetVal (*OnFileClose)(void *pFileInstData, off64_t offsLogfile); + rsRetVal (*DeleteStateFiles)(uchar *logfn); + rsRetVal (*GetBytesLeftInBlock)(void *pFileInstData, ssize_t *left); + void (*SetDeleteOnClose)(void *pFileInstData, int val); ENDinterface(cryprov) -#define cryprovCURR_IF_VERSION 1 /* increment whenever you change the interface structure! */ +#define cryprovCURR_IF_VERSION 3 /* increment whenever you change the interface structure! */ #endif /* #ifndef INCLUDED_CRYPROV_H */ diff --git a/runtime/datetime.c b/runtime/datetime.c index 841ff62..ba2ed35 100644 --- a/runtime/datetime.c +++ b/runtime/datetime.c @@ -5,7 +5,7 @@ * in a useful manner. It is still undecided if all functions will continue * to stay here or some will be moved into parser modules (once we have them). * - * Copyright 2008-2012 Rainer Gerhards and Adiscon GmbH. + * Copyright 2008-2014 Rainer Gerhards and Adiscon GmbH. * * This file is part of the rsyslog runtime library. * @@ -363,9 +363,19 @@ finalize_it: * If a *valid* timestamp is found, the string length is decremented * by the number of characters processed. If it is not a valid timestamp, * the length is kept unmodified. -- rgerhards, 2009-09-23 + * + * We support this format: + * [yyyy] Mon mm [yyyy] hh:mm:ss[.subsec][ TZSTRING:] + * Note that [yyyy] and [.subsec] are non-standard but frequently occur. + * Also [yyyy] can only occur once -- if it occurs twice, we flag the + * timestamp as invalid. if bParseTZ is true, we try to obtain a + * TZSTRING. Note that in this case it MUST be terminated by a colon + * (Cisco format). This option is a bit dangerous, as it could already + * by the tag. So it MUST only be enabled in specialised parsers. + * subsec, [yyyy] in front, TZSTRING was added in 2014-07-08 rgerhards */ static rsRetVal -ParseTIMESTAMP3164(struct syslogTime *pTime, uchar** ppszTS, int *pLenStr) +ParseTIMESTAMP3164(struct syslogTime *pTime, uchar** ppszTS, int *pLenStr, const int bParseTZ) { /* variables to temporarily hold time information while we parse */ int month; @@ -374,6 +384,12 @@ ParseTIMESTAMP3164(struct syslogTime *pTime, uchar** ppszTS, int *pLenStr) int hour; /* 24 hour clock */ int minute; int second; + int secfrac; /* fractional seconds (must be 32 bit!) */ + int secfracPrecision; + char tzstring[16]; + char OffsetMode = '\0'; /* UTC offset: \0 -> indicate no update */ + char OffsetHour; /* UTC offset in hours */ + int OffsetMinute; /* UTC offset in minutes */ /* end variables to temporarily hold time information while we parse */ int lenStr; uchar *pszTS; @@ -386,6 +402,21 @@ ParseTIMESTAMP3164(struct syslogTime *pTime, uchar** ppszTS, int *pLenStr) assert(pLenStr != NULL); lenStr = *pLenStr; + if(lenStr < 3) + ABORT_FINALIZE(RS_RET_INVLD_TIME); + + /* first check if we have a year in front of the timestamp. some devices (e.g. Brocade) + * do this. As it is pretty straightforward to detect and chance of misinterpretation + * is low, we try to parse it. + */ + if(*pszTS >= '0' && *pszTS <= '9') { + /* OK, either we have a prepended year or an invalid format! */ + year = srSLMGParseInt32(&pszTS, &lenStr); + if(year < 1970 || year > 2100 || *pszTS != ' ') + ABORT_FINALIZE(RS_RET_INVLD_TIME); + ++pszTS; /* skip SP */ + } + /* If we look at the month (Jan, Feb, Mar, Apr, May, Jun, Jul, Aug, Sep, Oct, Nov, Dec), * we may see the following character sequences occur: * @@ -407,9 +438,6 @@ ParseTIMESTAMP3164(struct syslogTime *pTime, uchar** ppszTS, int *pLenStr) * june, when it first manifested. This also lead to invalid parsing of the rest * of the message, as the time stamp was not detected to be correct. - rgerhards */ - if(lenStr < 3) - ABORT_FINALIZE(RS_RET_INVLD_TIME); - switch(*pszTS++) { case 'j': @@ -558,7 +586,7 @@ ParseTIMESTAMP3164(struct syslogTime *pTime, uchar** ppszTS, int *pLenStr) /* time part */ hour = srSLMGParseInt32(&pszTS, &lenStr); - if(hour > 1970 && hour < 2100) { + if(year == 0 && hour > 1970 && hour < 2100) { /* if so, we assume this actually is a year. This is a format found * e.g. in Cisco devices. * (if you read this 2100+ trying to fix a bug, congratulate me @@ -590,6 +618,41 @@ ParseTIMESTAMP3164(struct syslogTime *pTime, uchar** ppszTS, int *pLenStr) if(second < 0 || second > 60) ABORT_FINALIZE(RS_RET_INVLD_TIME); + /* as an extension e.g. found in CISCO IOS, we support sub-second resultion. + * It's presence is indicated by a dot immediately following the second. + */ + if(lenStr > 0 && *pszTS == '.') { + --lenStr; + uchar *pszStart = ++pszTS; + secfrac = srSLMGParseInt32(&pszTS, &lenStr); + secfracPrecision = (int) (pszTS - pszStart); + } else { + secfracPrecision = 0; + secfrac = 0; + } + + /* try to parse the TZSTRING if we are instructed to do so */ + if(bParseTZ && lenStr > 2 && *pszTS == ' ') { + int i; + for( ++pszTS, --lenStr, i = 0 + ; lenStr > 0 && i < (int) sizeof(tzstring) - 1 && *pszTS != ':' && *pszTS != ' ' + ; --lenStr) + tzstring[i++] = *pszTS++; + if(i > 0) { + /* found TZ, apply it */ + tzinfo_t* tzinfo; + tzstring[i] = '\0'; + if((tzinfo = glblFindTimezoneInfo((char*) tzstring)) == NULL) { + DBGPRINTF("ParseTIMESTAMP3164: invalid TZ string '%s' -- ignored\n", + tzstring); + } else { + OffsetMode = tzinfo->offsMode; + OffsetHour = tzinfo->offsHour; + OffsetMinute = tzinfo->offsMin; + } + } + } + /* we provide support for an extra ":" after the date. While this is an * invalid format, it occurs frequently enough (e.g. with Cisco devices) * to permit it as a valid case. -- rgerhards, 2008-09-12 @@ -618,14 +681,28 @@ ParseTIMESTAMP3164(struct syslogTime *pTime, uchar** ppszTS, int *pLenStr) pTime->hour = hour; pTime->minute = minute; pTime->second = second; - pTime->secfracPrecision = 0; - pTime->secfrac = 0; + pTime->secfrac = secfrac; + pTime->secfracPrecision = secfracPrecision; + if(OffsetMode != '\0') { /* need to update TZ info? */ + pTime->OffsetMode = OffsetMode; + pTime->OffsetHour = OffsetHour; + pTime->OffsetMinute = OffsetMinute; + } *pLenStr = lenStr; finalize_it: RETiRet; } +void +applyDfltTZ(struct syslogTime *pTime, char *tz) +{ + pTime->OffsetMode = tz[0]; + pTime->OffsetHour = (tz[1] - '0') * 10 + (tz[2] - '0'); + pTime->OffsetMinute = (tz[4] - '0') * 10 + (tz[5] - '0'); + +} + /******************************************************************* * END CODE-LIBLOGGING * *******************************************************************/ @@ -979,6 +1056,31 @@ int formatTimestampUnix(struct syslogTime *ts, char *pBuf) return 11; } +/* 0 - Sunday, 1, Monday, ... + * Note that we cannot use strftime() and helpers as they rely on the TZ + * variable (again, arghhhh). So we need to do it ourselves... + * Note: in the year 2100, this algorithm does not work correctly (due to + * leap time rules. To fix it then (*IF* this code really still exists then), + * just use 2100 as new anchor year and adapt the initial day number. + */ +int getWeekdayNbr(struct syslogTime *ts) +{ + int wday; + int g, f; + + g = ts->year; + if(ts->month < 3) { + g--; + f = ts->month + 13; + } else { + f = ts->month + 1; + } + wday = ((36525*g)/100) + ((306*f)/10) + ts->day - 621049; + wday %= 7; + return wday; +} + + /* queryInterface function * rgerhards, 2008-03-05 diff --git a/runtime/datetime.h b/runtime/datetime.h index 9f3611e..c96f7ee 100644 --- a/runtime/datetime.h +++ b/runtime/datetime.h @@ -34,7 +34,7 @@ typedef struct datetime_s { BEGINinterface(datetime) /* name must also be changed in ENDinterface macro! */ void (*getCurrTime)(struct syslogTime *t, time_t *ttSeconds); rsRetVal (*ParseTIMESTAMP3339)(struct syslogTime *pTime, uchar** ppszTS, int*); - rsRetVal (*ParseTIMESTAMP3164)(struct syslogTime *pTime, uchar** pszTS, int*); + rsRetVal (*ParseTIMESTAMP3164)(struct syslogTime *pTime, uchar** pszTS, int*, const int bParseTZ); int (*formatTimestampToMySQL)(struct syslogTime *ts, char* pDst); int (*formatTimestampToPgSQL)(struct syslogTime *ts, char *pDst); int (*formatTimestamp3339)(struct syslogTime *ts, char* pBuf); @@ -48,7 +48,7 @@ BEGINinterface(datetime) /* name must also be changed in ENDinterface macro! */ int (*formatTimestampUnix)(struct syslogTime *ts, char*pBuf); time_t (*syslogTime2time_t)(struct syslogTime *ts); ENDinterface(datetime) -#define datetimeCURR_IF_VERSION 7 /* increment whenever you change the interface structure! */ +#define datetimeCURR_IF_VERSION 8 /* increment whenever you change the interface structure! */ /* interface changes: * 1 - initial version * 2 - not compatible to 1 - bugfix required ParseTIMESTAMP3164 to accept char ** as @@ -58,9 +58,15 @@ ENDinterface(datetime) * 4 - formatTimestamp3164 takes a third int parameter * 5 - merge of versions 3 + 4 (2010-03-09) * 6 - see above + * 8 - ParseTIMESTAMP3164 has addtl parameter to permit TZ string parsing */ +#define PARSE3164_TZSTRING 1 +#define NO_PARSE3164_TZSTRING 0 + /* prototypes */ PROTOTYPEObj(datetime); +void applyDfltTZ(struct syslogTime *pTime, char *tz); +int getWeekdayNbr(struct syslogTime *ts); #endif /* #ifndef INCLUDED_DATETIME_H */ diff --git a/runtime/debug.c b/runtime/debug.c index 876f61d..bfdfa34 100644 --- a/runtime/debug.c +++ b/runtime/debug.c @@ -60,7 +60,7 @@ /* static data (some time to be replaced) */ DEFobjCurrIf(obj) -int Debug; /* debug flag - read-only after startup */ +int Debug = DEBUG_OFF; /* debug flag - read-only after startup */ int debugging_on = 0; /* read-only, except on sig USR1 */ static int bLogFuncFlow = 0; /* shall the function entry and exit be logged to the debug log? */ static int bLogAllocFree = 0; /* shall calls to (m/c)alloc and free be logged to the debug log? */ @@ -70,8 +70,8 @@ static int bPrintTime = 1; /* print a timestamp together with debug message */ static int bPrintAllDebugOnExit = 0; static int bAbortTrace = 1; /* print a trace after SIGABRT or SIGSEGV */ static int bOutputTidToStderr = 0;/* output TID to stderr on thread creation */ -static char *pszAltDbgFileName = NULL; /* if set, debug output is *also* sent to here */ -static int altdbg = -1; /* and the handle for alternate debug output */ +char *pszAltDbgFileName = NULL; /* if set, debug output is *also* sent to here */ +int altdbg = -1; /* and the handle for alternate debug output */ int stddbg = 1; /* the handle for regular debug output, set to stdout if not forking, -1 otherwise */ /* list of files/objects that should be printed */ @@ -695,10 +695,11 @@ static dbgThrdInfo_t *dbgGetThrdInfo(void) pthread_mutex_lock(&mutCallStack); if((pThrd = pthread_getspecific(keyCallStack)) == NULL) { /* construct object */ - pThrd = calloc(1, sizeof(dbgThrdInfo_t)); - pThrd->thrd = pthread_self(); - (void) pthread_setspecific(keyCallStack, pThrd); - DLL_Add(CallStack, pThrd); + if((pThrd = calloc(1, sizeof(dbgThrdInfo_t))) != NULL) { + pThrd->thrd = pthread_self(); + (void) pthread_setspecific(keyCallStack, pThrd); + DLL_Add(CallStack, pThrd); + } } pthread_mutex_unlock(&mutCallStack); return pThrd; @@ -738,25 +739,27 @@ static void dbgGetThrdName(char *pszBuf, size_t lenBuf, pthread_t thrd, int bInc snprintf(pszBuf, lenBuf, "%lx", (long) thrd); } else { if(bIncludeNumID) { - snprintf(pszBuf, lenBuf, "%s (%lx)", pThrd->pszThrdName, (long) thrd); + snprintf(pszBuf, lenBuf, "%-15s (%lx)", pThrd->pszThrdName, (long) thrd); } else { - snprintf(pszBuf, lenBuf, "%s", pThrd->pszThrdName); + snprintf(pszBuf, lenBuf, "%-15s", pThrd->pszThrdName); } } - } /* set a name for the current thread. The caller provided string is duplicated. + * Note: we must lock the "dbgprint" mutex, because dbgprint() uses the thread + * name and we could get a race (and abort) in cases where both are executed in + * parallel and we free or incompletely-copy the string. */ void dbgSetThrdName(uchar *pszName) { -return; - + pthread_mutex_lock(&mutdbgprint); dbgThrdInfo_t *pThrd = dbgGetThrdInfo(); if(pThrd->pszThrdName != NULL) free(pThrd->pszThrdName); pThrd->pszThrdName = strdup((char*)pszName); + pthread_mutex_unlock(&mutdbgprint); } @@ -1430,6 +1433,7 @@ dbgSetDebugFile(uchar *fn) rsRetVal dbgClassInit(void) { + pthread_mutexattr_t mutAttr; rsRetVal iRet; /* do not use DEFiRet, as this makes calls into the debug system! */ struct sigaction sigAct; @@ -1437,14 +1441,16 @@ rsRetVal dbgClassInit(void) (void) pthread_key_create(&keyCallStack, dbgCallStackDestruct); /* MUST be the first action done! */ - /* we initialize all Mutexes with code, as some platforms seem to have - * bugs in the static initializer macros. So better be on the safe side... - * rgerhards, 2008-03-06 + /* the mutexes must be recursive, because it may be called from within + * signal handlers, which can lead to a hang if the signal interrupted dbgprintf + * (yes, we have really seen that situation in practice!). -- rgerhards, 2013-05-17 */ - pthread_mutex_init(&mutFuncDBList, NULL); - pthread_mutex_init(&mutMutLog, NULL); - pthread_mutex_init(&mutCallStack, NULL); - pthread_mutex_init(&mutdbgprint, NULL); + pthread_mutexattr_init(&mutAttr); + pthread_mutexattr_settype(&mutAttr, PTHREAD_MUTEX_RECURSIVE); + pthread_mutex_init(&mutFuncDBList, &mutAttr); + pthread_mutex_init(&mutMutLog, &mutAttr); + pthread_mutex_init(&mutCallStack, &mutAttr); + pthread_mutex_init(&mutdbgprint, &mutAttr); /* while we try not to use any of the real rsyslog code (to avoid infinite loops), we * need to have the ability to query object names. Thus, we need to obtain a pointer to diff --git a/runtime/debug.h b/runtime/debug.h index f322609..6881ee7 100644 --- a/runtime/debug.h +++ b/runtime/debug.h @@ -109,6 +109,10 @@ void *dbgmalloc(size_t size); void dbgOutputTID(char* name); int dbgGetDbglogFd(void); +/* external data */ +extern char *pszAltDbgFileName; /* if set, debug output is *also* sent to here */ +extern int altdbg; /* and the handle for alternate debug output */ + /* macros */ #ifdef DEBUGLESS # define DBGPRINTF(...) {} diff --git a/runtime/errmsg.c b/runtime/errmsg.c index dcb5b18..b394143 100644 --- a/runtime/errmsg.c +++ b/runtime/errmsg.c @@ -7,7 +7,7 @@ * to take further case, as the code now boils to be either my own or, a few lines, * of the original BSD-licenses sysklogd code. rgerhards, 2012-01-16 * - * Copyright 2008-2012 Adiscon GmbH. + * Copyright 2008-2013 Adiscon GmbH. * * This file is part of the rsyslog runtime library. * @@ -56,51 +56,90 @@ DEFobjStaticHelpers * maps to a specific error event). * rgerhards, 2008-06-27 */ -static void __attribute__((format(printf, 3, 4))) -LogError(int iErrno, int iErrCode, char *fmt, ... ) +static void +doLogMsg(const int iErrno, const int iErrCode, const int severity, const char *msg) { - va_list ap; - char buf[1024]; - char msg[1024]; + char buf[2048]; char errStr[1024]; - size_t lenBuf; - - BEGINfunc - assert(fmt != NULL); - /* Format parameters */ - va_start(ap, fmt); - lenBuf = vsnprintf(buf, sizeof(buf), fmt, ap); - if(lenBuf >= sizeof(buf)) { - /* if our buffer was too small, we simply truncate. */ - lenBuf--; - } - va_end(ap); - /* Log the error now */ - buf[sizeof(buf)/sizeof(char) - 1] = '\0'; /* just to be on the safe side... */ - - dbgprintf("Called LogError, msg: %s\n", buf); + dbgprintf("Called LogMsg, msg: %s\n", msg); if(iErrno != 0) { rs_strerror_r(iErrno, errStr, sizeof(errStr)); if(iErrCode == NO_ERRCODE || iErrCode == RS_RET_ERR) { - snprintf(msg, sizeof(msg), "%s: %s", buf, errStr); + snprintf(buf, sizeof(buf), "%s: %s", msg, errStr); } else { - snprintf(msg, sizeof(msg), "%s: %s [try http://www.rsyslog.com/e/%d ]", buf, errStr, iErrCode * -1); + snprintf(buf, sizeof(buf), "%s: %s [try http://www.rsyslog.com/e/%d ]", msg, errStr, iErrCode * -1); } } else { if(iErrCode == NO_ERRCODE || iErrCode == RS_RET_ERR) { - snprintf(msg, sizeof(msg), "%s", buf); + snprintf(buf, sizeof(buf), "%s", msg); } else { - snprintf(msg, sizeof(msg), "%s [try http://www.rsyslog.com/e/%d ]", buf, iErrCode * -1); + snprintf(buf, sizeof(buf), "%s [try http://www.rsyslog.com/e/%d ]", msg, iErrCode * -1); } } - msg[sizeof(msg)/sizeof(char) - 1] = '\0'; /* just to be on the safe side... */ + buf[sizeof(buf)/sizeof(char) - 1] = '\0'; /* just to be on the safe side... */ errno = 0; - glblErrLogger(iErrCode, (uchar*)msg); + glblErrLogger(severity, iErrCode, (uchar*)buf); +} + +/* We now receive three parameters: one is the internal error code + * which will also become the error message number, the second is + * errno - if it is non-zero, the corresponding error message is included + * in the text and finally the message text itself. Note that it is not + * 100% clean to use the internal errcode, as it may be reached from + * multiple actual error causes. However, it is much better than having + * no error code at all (and in most cases, a single internal error code + * maps to a specific error event). + * rgerhards, 2008-06-27 + */ +static void __attribute__((format(printf, 3, 4))) +LogError(const int iErrno, const int iErrCode, const char *fmt, ... ) +{ + va_list ap; + char buf[2048]; + size_t lenBuf; + + va_start(ap, fmt); + lenBuf = vsnprintf(buf, sizeof(buf), fmt, ap); + if(lenBuf >= sizeof(buf)) { + /* if our buffer was too small, we simply truncate. */ + lenBuf--; + } + va_end(ap); + buf[sizeof(buf)/sizeof(char) - 1] = '\0'; /* just to be on the safe side... */ + + doLogMsg(iErrno, iErrCode, LOG_ERR, buf); +} - ENDfunc +/* We now receive three parameters: one is the internal error code + * which will also become the error message number, the second is + * errno - if it is non-zero, the corresponding error message is included + * in the text and finally the message text itself. Note that it is not + * 100% clean to use the internal errcode, as it may be reached from + * multiple actual error causes. However, it is much better than having + * no error code at all (and in most cases, a single internal error code + * maps to a specific error event). + * rgerhards, 2008-06-27 + */ +static void __attribute__((format(printf, 4, 5))) +LogMsg(const int iErrno, const int iErrCode, const int severity, const char *fmt, ... ) +{ + va_list ap; + char buf[2048]; + size_t lenBuf; + + va_start(ap, fmt); + lenBuf = vsnprintf(buf, sizeof(buf), fmt, ap); + if(lenBuf >= sizeof(buf)) { + /* if our buffer was too small, we simply truncate. */ + lenBuf--; + } + va_end(ap); + buf[sizeof(buf)/sizeof(char) - 1] = '\0'; /* just to be on the safe side... */ + + doLogMsg(iErrno, iErrCode, severity, buf); } @@ -119,6 +158,7 @@ CODESTARTobjQueryInterface(errmsg) * of course, also affects the "if" above). */ pIf->LogError = LogError; + pIf->LogMsg = LogMsg; finalize_it: ENDobjQueryInterface(errmsg) diff --git a/runtime/errmsg.h b/runtime/errmsg.h index dfa70c0..b1b4741 100644 --- a/runtime/errmsg.h +++ b/runtime/errmsg.h @@ -1,6 +1,6 @@ /* The errmsg object. It is used to emit error message inside rsyslog. * - * Copyright 2008-2012 Rainer Gerhards and Adiscon GmbH. + * Copyright 2008-2013 Rainer Gerhards and Adiscon GmbH. * * This file is part of the rsyslog runtime library. * @@ -23,7 +23,6 @@ #include "errmsg.h" -/* TODO: define error codes */ #define NO_ERRCODE -1 /* the errmsg object */ @@ -34,9 +33,11 @@ typedef struct errmsg_s { /* interfaces */ BEGINinterface(errmsg) /* name must also be changed in ENDinterface macro! */ - void __attribute__((format(printf, 3, 4))) (*LogError)(int iErrno, int iErrCode, char *pszErrFmt, ... ); + void __attribute__((format(printf, 3, 4))) (*LogError)(const int iErrno, const int iErrCode, const char *pszErrFmt, ... ); + /* v2, 2013-11-29 */ + void __attribute__((format(printf, 4, 5))) (*LogMsg)(const int iErrno, const int iErrCode, const int severity, const char *pszErrFmt, ... ); ENDinterface(errmsg) -#define errmsgCURR_IF_VERSION 1 /* increment whenever you change the interface structure! */ +#define errmsgCURR_IF_VERSION 2 /* increment whenever you change the interface structure! */ /* prototypes */ diff --git a/runtime/glbl.c b/runtime/glbl.c index b3fe3a1..1867410 100644 --- a/runtime/glbl.c +++ b/runtime/glbl.c @@ -7,7 +7,7 @@ * * Module begun 2008-04-16 by Rainer Gerhards * - * Copyright 2008-2013 Rainer Gerhards and Adiscon GmbH. + * Copyright 2008-2014 Rainer Gerhards and Adiscon GmbH. * * This file is part of the rsyslog runtime library. * @@ -31,8 +31,12 @@ #include <sys/socket.h> #include <sys/types.h> #include <sys/stat.h> +#include <fcntl.h> #include <unistd.h> +#include <pthread.h> +#include <ctype.h> #include <assert.h> +#include <stdint.h> #include "rsyslog.h" #include "obj.h" @@ -42,6 +46,8 @@ #include "prop.h" #include "atomic.h" #include "errmsg.h" +#include "action.h" +#include "parserif.h" #include "rainerscript.h" #include "net.h" @@ -60,7 +66,16 @@ DEFobjCurrIf(net) * For this object, these variables are obviously what makes the "meat" of the * class... */ +int glblDebugOnShutdown = 0; /* start debug log when we are shut down */ +stdlog_channel_t stdlog_hdl = NULL; /* handle to be used for stdlog */ + +static struct cnfobj *mainqCnfObj = NULL;/* main queue object, to be used later in startup sequence */ +int bProcessInternalMessages = 1; /* Should rsyslog itself process internal messages? + * 1 - yes + * 0 - send them to libstdlog (e.g. to push to journal) + */ static uchar *pszWorkDir = NULL; +static uchar *stdlog_chanspec = NULL; static int bOptimizeUniProc = 1; /* enable uniprocessor optimizations */ static int bParseHOSTNAMEandTAG = 1; /* parser modification (based on startup params!) */ static int bPreserveFQDN = 0; /* should FQDNs always be preserved? */ @@ -71,6 +86,7 @@ static int option_DisallowWarning = 1; /* complain if message from disallowed se static int bDisableDNS = 0; /* don't look up IP addresses of remote messages */ static prop_t *propLocalIPIF = NULL;/* IP address to report for the local host (default is 127.0.0.1) */ static prop_t *propLocalHostName = NULL;/* our hostname as FQDN - read-only after startup */ +static prop_t *propLocalHostNameToDelete = NULL;/* see GenerateLocalHostName function hdr comment! */ static uchar *LocalHostName = NULL;/* our hostname - read-only after startup, except HUP */ static uchar *LocalHostNameOverride = NULL;/* user-overridden hostname - read-only after startup */ static uchar *LocalFQDNName = NULL;/* our hostname as FQDN - read-only after startup, except HUP */ @@ -82,6 +98,15 @@ static uchar *pszDfltNetstrmDrvrCAF = NULL; /* default CA file for the netstrm d static uchar *pszDfltNetstrmDrvrKeyFile = NULL; /* default key file for the netstrm driver (server) */ static uchar *pszDfltNetstrmDrvrCertFile = NULL; /* default cert file for the netstrm driver (server) */ static int bTerminateInputs = 0; /* global switch that inputs shall terminate ASAP (1=> terminate) */ +static uchar cCCEscapeChar = '#'; /* character to be used to start an escape sequence for control chars */ +static int bDropTrailingLF = 1; /* drop trailing LF's on reception? */ +static int bEscapeCCOnRcv = 1; /* escape control characters on reception: 0 - no, 1 - yes */ +static int bSpaceLFOnRcv = 0; /* replace newlines with spaces on reception: 0 - no, 1 - yes */ +static int bEscape8BitChars = 0; /* escape characters > 127 on reception: 0 - no, 1 - yes */ +static int bEscapeTab = 1; /* escape tab control character when doing CC escapes: 0 - no, 1 - yes */ +static int bParserEscapeCCCStyle = 0; /* escape control characters in c style: 0 - no, 1 - yes */ +short janitorInterval = 10; /* interval (in minutes) at which the janitor runs */ + pid_t glbl_ourpid; #ifndef HAVE_ATOMIC_BUILTINS static DEF_ATOMIC_HELPER_MUT(mutTerminateInputs); @@ -89,7 +114,10 @@ static DEF_ATOMIC_HELPER_MUT(mutTerminateInputs); #ifdef USE_UNLIMITED_SELECT static int iFdSetSize = howmany(FD_SETSIZE, __NFDBITS) * sizeof (fd_mask); /* size of select() bitmask in bytes */ #endif +static uchar *SourceIPofLocalClient = NULL; /* [ar] Source IP for local client to be used on multihomed host */ +tzinfo_t *tzinfos = NULL; +static int ntzinfos; /* tables for interfacing with the v6 config system */ static struct cnfparamdescr cnfparamdescr[] = { @@ -97,10 +125,25 @@ static struct cnfparamdescr cnfparamdescr[] = { { "dropmsgswithmaliciousdnsptrrecords", eCmdHdlrBinary, 0 }, { "localhostname", eCmdHdlrGetWord, 0 }, { "preservefqdn", eCmdHdlrBinary, 0 }, + { "debug.onshutdown", eCmdHdlrBinary, 0 }, + { "debug.logfile", eCmdHdlrString, 0 }, { "defaultnetstreamdrivercafile", eCmdHdlrString, 0 }, { "defaultnetstreamdriverkeyfile", eCmdHdlrString, 0 }, + { "defaultnetstreamdrivercertfile", eCmdHdlrString, 0 }, { "defaultnetstreamdriver", eCmdHdlrString, 0 }, { "maxmessagesize", eCmdHdlrSize, 0 }, + { "action.reportsuspension", eCmdHdlrBinary, 0 }, + { "action.reportsuspensioncontinuation", eCmdHdlrBinary, 0 }, + { "parser.controlcharacterescapeprefix", eCmdHdlrGetChar, 0 }, + { "parser.droptrailinglfonreception", eCmdHdlrBinary, 0 }, + { "parser.escapecontrolcharactersonreceive", eCmdHdlrBinary, 0 }, + { "parser.spacelfonreceive", eCmdHdlrBinary, 0 }, + { "parser.escape8bitcharactersonreceive", eCmdHdlrBinary, 0}, + { "parser.escapecontrolcharactertab", eCmdHdlrBinary, 0}, + { "parser.escapecontrolcharacterscstyle", eCmdHdlrBinary, 0 }, + { "stdlog.channelspec", eCmdHdlrString, 0 }, + { "janitor.interval", eCmdHdlrPositiveInt, 0 }, + { "processinternalmessages", eCmdHdlrBinary, 0 } }; static struct cnfparamblk paramblk = { CNFPARAMBLK_VERSION, @@ -108,6 +151,16 @@ static struct cnfparamblk paramblk = cnfparamdescr }; +static struct cnfparamdescr timezonecnfparamdescr[] = { + { "id", eCmdHdlrString, 0 }, + { "offset", eCmdHdlrGetWord, 0 } +}; +static struct cnfparamblk timezonepblk = + { CNFPARAMBLK_VERSION, + sizeof(timezonecnfparamdescr)/sizeof(struct cnfparamdescr), + timezonecnfparamdescr + }; + static struct cnfparamvals *cnfparamvals = NULL; /* we need to support multiple calls into our param block, so we need * to persist the current settings. Note that this must be re-set @@ -136,6 +189,7 @@ static dataType Get##nameFunc(void) \ SIMP_PROP(ParseHOSTNAMEandTAG, bParseHOSTNAMEandTAG, int) SIMP_PROP(OptimizeUniProc, bOptimizeUniProc, int) SIMP_PROP(PreserveFQDN, bPreserveFQDN, int) +SIMP_PROP(mainqCnfObj, mainqCnfObj, struct cnfobj *) SIMP_PROP(MaxLine, iMaxLine, int) SIMP_PROP(DefPFFamily, iDefPFFamily, int) /* note that in the future we may check the family argument */ SIMP_PROP(DropMalPTRMsgs, bDropMalPTRMsgs, int) @@ -143,6 +197,13 @@ SIMP_PROP(Option_DisallowWarning, option_DisallowWarning, int) SIMP_PROP(DisableDNS, bDisableDNS, int) SIMP_PROP(StripDomains, StripDomains, char**) SIMP_PROP(LocalHosts, LocalHosts, char**) +SIMP_PROP(ParserControlCharacterEscapePrefix, cCCEscapeChar, uchar) +SIMP_PROP(ParserDropTrailingLFOnReception, bDropTrailingLF, int) +SIMP_PROP(ParserEscapeControlCharactersOnReceive, bEscapeCCOnRcv, int) +SIMP_PROP(ParserSpaceLFOnReceive, bSpaceLFOnRcv, int) +SIMP_PROP(ParserEscape8BitCharactersOnReceive, bEscape8BitChars, int) +SIMP_PROP(ParserEscapeControlCharacterTab, bEscapeTab, int) +SIMP_PROP(ParserEscapeControlCharactersCStyle, bParserEscapeCCCStyle, int) #ifdef USE_UNLIMITED_SELECT SIMP_PROP(FdSetSize, iFdSetSize, int) #endif @@ -379,17 +440,31 @@ GetLocalDomain(void) /* generate the local hostname property. This must be done after the hostname info * has been set as well as PreserveFQDN. * rgerhards, 2009-06-30 + * NOTE: This function tries to avoid locking by not destructing the previous value + * immediately. This is so that current readers can continue to use the previous name. + * Otherwise, we would need to use read/write locks to protect the update process. + * In order to do so, we save the previous value and delete it when we are called again + * the next time. Note that this in theory is racy and can lead to a double-free. + * In practice, however, the window of exposure to trigger this is extremely short + * and as this functions is very infrequently being called (on HUP), the trigger + * condition for this bug is so highly unlikely that it never occurs in practice. + * Probably if you HUP rsyslog every few milliseconds, but who does that... + * To further reduce risk potential, we do only update the property when there + * actually is a hostname change, which makes it even less likely. + * rgerhards, 2013-10-28 */ static rsRetVal GenerateLocalHostNameProperty(void) { - DEFiRet; + uchar *pszPrev; + int lenPrev; + prop_t *hostnameNew; uchar *pszName; + DEFiRet; - if(propLocalHostName != NULL) - prop.Destruct(&propLocalHostName); + if(propLocalHostNameToDelete != NULL) + prop.Destruct(&propLocalHostNameToDelete); - CHKiRet(prop.Construct(&propLocalHostName)); if(LocalHostNameOverride == NULL) { if(LocalHostName == NULL) pszName = (uchar*) "[localhost]"; @@ -403,8 +478,20 @@ GenerateLocalHostNameProperty(void) pszName = LocalHostNameOverride; } DBGPRINTF("GenerateLocalHostName uses '%s'\n", pszName); - CHKiRet(prop.SetString(propLocalHostName, pszName, ustrlen(pszName))); - CHKiRet(prop.ConstructFinalize(propLocalHostName)); + + if(propLocalHostName == NULL) + pszPrev = (uchar*)""; /* make sure strcmp() below does not match */ + else + prop.GetString(propLocalHostName, &pszPrev, &lenPrev); + + if(ustrcmp(pszPrev, pszName)) { + /* we need to update */ + CHKiRet(prop.Construct(&hostnameNew)); + CHKiRet(prop.SetString(hostnameNew, pszName, ustrlen(pszName))); + CHKiRet(prop.ConstructFinalize(hostnameNew)); + propLocalHostNameToDelete = propLocalHostName; + propLocalHostName = hostnameNew; + } finalize_it: RETiRet; @@ -445,6 +532,14 @@ GetWorkDir(void) return(pszWorkDir == NULL ? (uchar*) "" : pszWorkDir); } +/* return the "raw" working directory, which means + * NULL if unset. + */ +const uchar * +glblGetWorkDirRaw(void) +{ + return pszWorkDir; +} /* return the current default netstream driver */ static uchar* @@ -478,6 +573,23 @@ GetDfltNetstrmDrvrCertFile(void) } +/* [ar] Source IP for local client to be used on multihomed host */ +static rsRetVal +SetSourceIPofLocalClient(uchar *newname) +{ + if(SourceIPofLocalClient != NULL) { + free(SourceIPofLocalClient); } + SourceIPofLocalClient = newname; + return RS_RET_OK; +} + +static uchar* +GetSourceIPofLocalClient(void) +{ + return(SourceIPofLocalClient); +} + + /* queryInterface function * rgerhards, 2008-02-21 */ @@ -498,6 +610,8 @@ CODESTARTobjQueryInterface(glbl) pIf->GetLocalHostIP = GetLocalHostIP; pIf->SetGlobalInputTermination = SetGlobalInputTermination; pIf->GetGlobalInputTermState = GetGlobalInputTermState; + pIf->GetSourceIPofLocalClient = GetSourceIPofLocalClient; /* [ar] */ + pIf->SetSourceIPofLocalClient = SetSourceIPofLocalClient; /* [ar] */ #define SIMP_PROP(name) \ pIf->Get##name = Get##name; \ pIf->Set##name = Set##name; @@ -509,11 +623,19 @@ CODESTARTobjQueryInterface(glbl) SIMP_PROP(DropMalPTRMsgs); SIMP_PROP(Option_DisallowWarning); SIMP_PROP(DisableDNS); + SIMP_PROP(mainqCnfObj); SIMP_PROP(LocalFQDNName) SIMP_PROP(LocalHostName) SIMP_PROP(LocalDomain) SIMP_PROP(StripDomains) SIMP_PROP(LocalHosts) + SIMP_PROP(ParserControlCharacterEscapePrefix) + SIMP_PROP(ParserDropTrailingLFOnReception) + SIMP_PROP(ParserEscapeControlCharactersOnReceive) + SIMP_PROP(ParserSpaceLFOnReceive) + SIMP_PROP(ParserEscape8BitCharactersOnReceive) + SIMP_PROP(ParserEscapeControlCharacterTab) + SIMP_PROP(ParserEscapeControlCharactersCStyle) SIMP_PROP(DfltNetstrmDrvr) SIMP_PROP(DfltNetstrmDrvrCAF) SIMP_PROP(DfltNetstrmDrvrKeyFile) @@ -547,6 +669,13 @@ static rsRetVal resetConfigVariables(uchar __attribute__((unused)) *pp, void __a bOptimizeUniProc = 1; bPreserveFQDN = 0; iMaxLine = 8192; + cCCEscapeChar = '#'; + bDropTrailingLF = 1; + bEscapeCCOnRcv = 1; /* default is to escape control characters */ + bSpaceLFOnRcv = 0; + bEscape8BitChars = 0; /* default is not to escape control characters */ + bEscapeTab = 1; /* default is to escape tab characters */ + bParserEscapeCCCStyle = 0; #ifdef USE_UNLIMITED_SELECT iFdSetSize = howmany(FD_SETSIZE, __NFDBITS) * sizeof (fd_mask); #endif @@ -559,10 +688,126 @@ static rsRetVal resetConfigVariables(uchar __attribute__((unused)) *pp, void __a void glblPrepCnf(void) { + free(mainqCnfObj); + mainqCnfObj = NULL; free(cnfparamvals); cnfparamvals = NULL; } + +static void +freeTimezoneInfo(void) +{ + int i; + for(i = 0 ; i < ntzinfos ; ++i) + free(tzinfos[i].id); + free(tzinfos); + tzinfos = NULL; +} + +static void +displayTzinfos(void) +{ + int i; + if(!Debug) + return; + for(i = 0 ; i < ntzinfos ; ++i) + dbgprintf("tzinfo: '%s':%c%2.2d:%2.2d\n", + tzinfos[i].id, tzinfos[i].offsMode, + tzinfos[i].offsHour, tzinfos[i].offsMin); +} + + +/* Note: this function is NOT thread-safe! + * This is currently not needed as used only during + * initialization. + */ +static inline rsRetVal +addTimezoneInfo(uchar *tzid, char offsMode, int8_t offsHour, int8_t offsMin) +{ + DEFiRet; + tzinfo_t *newti; + CHKmalloc(newti = realloc(tzinfos, (ntzinfos+1)*sizeof(tzinfo_t))); + CHKmalloc(newti[ntzinfos].id = strdup((char*)tzid)); + newti[ntzinfos].offsMode = offsMode; + newti[ntzinfos].offsHour = offsHour; + newti[ntzinfos].offsMin = offsMin; + ++ntzinfos, tzinfos = newti; +finalize_it: + RETiRet; +} + + +static int +bs_arrcmp_tzinfo(const void *s1, const void *s2) +{ + return strcmp((char*)s1, (char*)((tzinfo_t*)s2)->id); +} +/* returns matching timezone info or NULL if no entry exists */ +tzinfo_t* +glblFindTimezoneInfo(char *id) +{ + return (tzinfo_t*) bsearch(id, tzinfos, ntzinfos, sizeof(tzinfo_t), bs_arrcmp_tzinfo); +} + +/* handle the timezone() object. Each incarnation adds one additional + * zone info to the global table of time zones. + */ +void +glblProcessTimezone(struct cnfobj *o) +{ + struct cnfparamvals *pvals; + uchar *id = NULL; + uchar *offset = NULL; + char offsMode; + int8_t offsHour; + int8_t offsMin; + int i; + + pvals = nvlstGetParams(o->nvlst, &timezonepblk, NULL); + dbgprintf("timezone param blk after glblProcessTimezone:\n"); + cnfparamsPrint(&timezonepblk, pvals); + + for(i = 0 ; i < timezonepblk.nParams ; ++i) { + if(!pvals[i].bUsed) + continue; + if(!strcmp(timezonepblk.descr[i].name, "id")) { + id = (uchar*) es_str2cstr(pvals[i].val.d.estr, NULL); + } else if(!strcmp(timezonepblk.descr[i].name, "offset")) { + offset = (uchar*) es_str2cstr(pvals[i].val.d.estr, NULL); + } else { + dbgprintf("glblProcessTimezone: program error, non-handled " + "param '%s'\n", timezonepblk.descr[i].name); + } + } + + if( strlen((char*)offset) != 6 + || !(offset[0] == '-' || offset[0] == '+') + || !(isdigit(offset[1]) && isdigit(offset[2])) + || offset[3] != ':' + || !(isdigit(offset[4]) && isdigit(offset[5])) + ) { + parser_errmsg("timezone offset has invalid format. Must be +/-hh:mm, e.g. \"-07:00\"."); + goto done; + } + + offsHour = (offset[1] - '0') * 10 + offset[2] - '0'; + offsMin = (offset[4] - '0') * 10 + offset[5] - '0'; + offsMode = offset[0]; + + if(offsHour > 12 || offsMin > 59) { + parser_errmsg("timezone offset outside of supported range (hours 0..12, minutes 0..59)"); + goto done; + } + + addTimezoneInfo(id, offsMode, offsHour, offsMin); + +done: + cnfparamvalsDestruct(pvals, &timezonepblk); + free(id); + free(offset); +} + /* handle a global config object. Note that multiple global config statements * are permitted (because of plugin support), so once we got a param block, * we need to hold to it. @@ -571,17 +816,80 @@ glblPrepCnf(void) void glblProcessCnf(struct cnfobj *o) { + int i; + cnfparamvals = nvlstGetParams(o->nvlst, ¶mblk, cnfparamvals); dbgprintf("glbl param blk after glblProcessCnf:\n"); cnfparamsPrint(¶mblk, cnfparamvals); + + /* The next thing is a bit hackish and should be changed in higher + * versions. There are a select few parameters which we need to + * act on immediately. These are processed here. + */ + for(i = 0 ; i < paramblk.nParams ; ++i) { + if(!cnfparamvals[i].bUsed) + continue; + if(!strcmp(paramblk.descr[i].name, "processinternalmessages")) { + bProcessInternalMessages = (int) cnfparamvals[i].val.d.n; + } else if(!strcmp(paramblk.descr[i].name, "stdlog.channelspec")) { + stdlog_chanspec = (uchar*) + es_str2cstr(cnfparamvals[i].val.d.estr, NULL); + stdlog_hdl = stdlog_open("rsyslogd", 0, STDLOG_SYSLOG, + (char*) stdlog_chanspec); + } + } } +/* Set mainq parameters. Note that when this is not called, we'll use the + * legacy parameter config. mainq parameters can only be set once. + */ +void +glblProcessMainQCnf(struct cnfobj *o) +{ + if(mainqCnfObj == NULL) { + mainqCnfObj = o; + } else { + errmsg.LogError(0, RS_RET_ERR, "main_queue() object can only be specified " + "once - all but first ignored\n"); + } +} + +/* destruct the main q cnf object after it is no longer needed. This is + * also used to do some final checks. + */ +void +glblDestructMainqCnfObj() +{ + /* Only destruct if not NULL! */ + if (mainqCnfObj != NULL) { + nvlstChkUnused(mainqCnfObj->nvlst); + } + cnfobjDestruct(mainqCnfObj); + mainqCnfObj = NULL; +} + +/* comparison function for qsort() and string array compare + * this is for the string lookup table type + */ +static int +qs_arrcmp_tzinfo(const void *s1, const void *s2) +{ + return strcmp(((tzinfo_t*)s1)->id, ((tzinfo_t*)s2)->id); +} + +/* This processes the "regular" parameters which are to be set after the + * config has been fully loaded. + */ void glblDoneLoadCnf(void) { int i; unsigned char *cstr; + qsort(tzinfos, ntzinfos, sizeof(tzinfo_t), qs_arrcmp_tzinfo); + DBGPRINTF("Timezone information table (%d entries):\n", ntzinfos); + displayTzinfos(); + if(cnfparamvals == NULL) goto finalize_it; @@ -599,6 +907,10 @@ glblDoneLoadCnf(void) free(pszDfltNetstrmDrvrKeyFile); pszDfltNetstrmDrvrKeyFile = (uchar*) es_str2cstr(cnfparamvals[i].val.d.estr, NULL); + } else if(!strcmp(paramblk.descr[i].name, "defaultnetstreamdrivercertfile")) { + free(pszDfltNetstrmDrvrCertFile); + pszDfltNetstrmDrvrCertFile = (uchar*) + es_str2cstr(cnfparamvals[i].val.d.estr, NULL); } else if(!strcmp(paramblk.descr[i].name, "defaultnetstreamdrivercafile")) { free(pszDfltNetstrmDrvrCAF); pszDfltNetstrmDrvrCAF = (uchar*) @@ -612,14 +924,51 @@ glblDoneLoadCnf(void) } else if(!strcmp(paramblk.descr[i].name, "dropmsgswithmaliciousdnsptrrecords")) { bDropMalPTRMsgs = (int) cnfparamvals[i].val.d.n; + } else if(!strcmp(paramblk.descr[i].name, "action.reportsuspension")) { + bActionReportSuspension = (int) cnfparamvals[i].val.d.n; + } else if(!strcmp(paramblk.descr[i].name, "action.reportsuspensioncontinuation")) { + bActionReportSuspensionCont = (int) cnfparamvals[i].val.d.n; } else if(!strcmp(paramblk.descr[i].name, "maxmessagesize")) { iMaxLine = (int) cnfparamvals[i].val.d.n; + } else if(!strcmp(paramblk.descr[i].name, "debug.onshutdown")) { + glblDebugOnShutdown = (int) cnfparamvals[i].val.d.n; + errmsg.LogError(0, RS_RET_OK, "debug: onShutdown set to %d", glblDebugOnShutdown); + } else if(!strcmp(paramblk.descr[i].name, "parser.controlcharacterescapeprefix")) { + cCCEscapeChar = (uchar) *es_str2cstr(cnfparamvals[i].val.d.estr, NULL); + } else if(!strcmp(paramblk.descr[i].name, "parser.droptrailinglfonreception")) { + bDropTrailingLF = (int) cnfparamvals[i].val.d.n; + } else if(!strcmp(paramblk.descr[i].name, "parser.escapecontrolcharactersonreceive")) { + bEscapeCCOnRcv = (int) cnfparamvals[i].val.d.n; + } else if(!strcmp(paramblk.descr[i].name, "parser.spacelfonreceive")) { + bSpaceLFOnRcv = (int) cnfparamvals[i].val.d.n; + } else if(!strcmp(paramblk.descr[i].name, "parser.escape8bitcharactersonreceive")) { + bEscape8BitChars = (int) cnfparamvals[i].val.d.n; + } else if(!strcmp(paramblk.descr[i].name, "parser.escapecontrolcharactertab")) { + bEscapeTab = (int) cnfparamvals[i].val.d.n; + } else if(!strcmp(paramblk.descr[i].name, "parser.escapecontrolcharacterscstyle")) { + bParserEscapeCCCStyle = (int) cnfparamvals[i].val.d.n; + } else if(!strcmp(paramblk.descr[i].name, "debug.logfile")) { + if(pszAltDbgFileName == NULL) { + pszAltDbgFileName = es_str2cstr(cnfparamvals[i].val.d.estr, NULL); + if((altdbg = open(pszAltDbgFileName, O_WRONLY|O_CREAT|O_TRUNC|O_NOCTTY|O_CLOEXEC, S_IRUSR|S_IWUSR)) == -1) { + errmsg.LogError(0, RS_RET_ERR, "debug log file '%s' could not be opened", pszAltDbgFileName); + } + } + errmsg.LogError(0, RS_RET_OK, "debug log file is '%s', fd %d", pszAltDbgFileName, altdbg); + } else if(!strcmp(paramblk.descr[i].name, "janitor.interval")) { + janitorInterval = (int) cnfparamvals[i].val.d.n; } else { dbgprintf("glblDoneLoadCnf: program error, non-handled " "param '%s'\n", paramblk.descr[i].name); } } -finalize_it: ; + + if(glblDebugOnShutdown && Debug != DEBUG_FULL) { + Debug = DEBUG_ONDEMAND; + stddbg = -1; + } + +finalize_it: return; } @@ -645,8 +994,16 @@ BEGINAbstractObjClassInit(glbl, 1, OBJ_IS_CORE_MODULE) /* class, version */ CHKiRet(regCfSysLineHdlr((uchar *)"localhostipif", 0, eCmdHdlrGetWord, setLocalHostIPIF, NULL, NULL)); CHKiRet(regCfSysLineHdlr((uchar *)"optimizeforuniprocessor", 0, eCmdHdlrBinary, NULL, &bOptimizeUniProc, NULL)); CHKiRet(regCfSysLineHdlr((uchar *)"preservefqdn", 0, eCmdHdlrBinary, NULL, &bPreserveFQDN, NULL)); - CHKiRet(regCfSysLineHdlr((uchar *)"maxmessagesize", 0, eCmdHdlrSize, - NULL, &iMaxLine, NULL)); + CHKiRet(regCfSysLineHdlr((uchar *)"maxmessagesize", 0, eCmdHdlrSize, NULL, &iMaxLine, NULL)); + + /* Deprecated parser config options */ + CHKiRet(regCfSysLineHdlr((uchar *)"controlcharacterescapeprefix", 0, eCmdHdlrGetChar, NULL, &cCCEscapeChar, NULL)); + CHKiRet(regCfSysLineHdlr((uchar *)"droptrailinglfonreception", 0, eCmdHdlrBinary, NULL, &bDropTrailingLF, NULL)); + CHKiRet(regCfSysLineHdlr((uchar *)"escapecontrolcharactersonreceive", 0, eCmdHdlrBinary, NULL, &bEscapeCCOnRcv, NULL)); + CHKiRet(regCfSysLineHdlr((uchar *)"spacelfonreceive", 0, eCmdHdlrBinary, NULL, &bSpaceLFOnRcv, NULL)); + CHKiRet(regCfSysLineHdlr((uchar *)"escape8bitcharactersonreceive", 0, eCmdHdlrBinary, NULL, &bEscape8BitChars, NULL)); + CHKiRet(regCfSysLineHdlr((uchar *)"escapecontrolcharactertab", 0, eCmdHdlrBinary, NULL, &bEscapeTab, NULL)); + CHKiRet(regCfSysLineHdlr((uchar *)"resetconfigvariables", 1, eCmdHdlrCustomHandler, resetConfigVariables, NULL, NULL)); INIT_ATOMIC_HELPER_MUT(mutTerminateInputs); @@ -666,7 +1023,10 @@ BEGINObjClassExit(glbl, OBJ_IS_CORE_MODULE) /* class, version */ free(LocalHostName); free(LocalHostNameOverride); free(LocalFQDNName); + freeTimezoneInfo(); objRelease(prop, CORE_COMPONENT); + if(propLocalHostNameToDelete != NULL) + prop.Destruct(&propLocalHostNameToDelete); DESTROY_ATOMIC_HELPER_MUT(mutTerminateInputs); ENDObjClassExit(glbl) diff --git a/runtime/glbl.h b/runtime/glbl.h index e95e48f..8afedc9 100644 --- a/runtime/glbl.h +++ b/runtime/glbl.h @@ -8,7 +8,7 @@ * Please note that there currently is no glbl.c file as we do not yet * have any implementations. * - * Copyright 2008-2012 Rainer Gerhards and Adiscon GmbH. + * Copyright 2008-2014 Rainer Gerhards and Adiscon GmbH. * * This file is part of the rsyslog runtime library. * @@ -31,12 +31,15 @@ #define GLBL_H_INCLUDED #include <sys/types.h> +#include <liblogging/stdlog.h> #include "rainerscript.h" #include "prop.h" #define glblGetIOBufSize() 4096 /* size of the IO buffer, e.g. for strm class */ extern pid_t glbl_ourpid; +extern int bProcessInternalMessages; +extern stdlog_channel_t stdlog_hdl; /* interfaces */ BEGINinterface(glbl) /* name must also be changed in ENDinterface macro! */ @@ -52,6 +55,7 @@ BEGINinterface(glbl) /* name must also be changed in ENDinterface macro! */ SIMP_PROP(Option_DisallowWarning, int) SIMP_PROP(DisableDNS, int) SIMP_PROP(LocalFQDNName, uchar*) + SIMP_PROP(mainqCnfObj, struct cnfobj*) SIMP_PROP(LocalHostName, uchar*) SIMP_PROP(LocalDomain, uchar*) SIMP_PROP(StripDomains, char**) @@ -60,6 +64,14 @@ BEGINinterface(glbl) /* name must also be changed in ENDinterface macro! */ SIMP_PROP(DfltNetstrmDrvrCAF, uchar*) SIMP_PROP(DfltNetstrmDrvrKeyFile, uchar*) SIMP_PROP(DfltNetstrmDrvrCertFile, uchar*) + SIMP_PROP(ParserControlCharacterEscapePrefix, uchar) + SIMP_PROP(ParserDropTrailingLFOnReception, int) + SIMP_PROP(ParserEscapeControlCharactersOnReceive, int) + SIMP_PROP(ParserSpaceLFOnReceive, int) + SIMP_PROP(ParserEscape8BitCharactersOnReceive, int) + SIMP_PROP(ParserEscapeControlCharacterTab, int) + SIMP_PROP(ParserEscapeControlCharactersCStyle, int) + /* added v3, 2009-06-30 */ rsRetVal (*GenerateLocalHostNameProperty)(void); prop_t* (*GetLocalHostNameProp)(void); @@ -81,6 +93,8 @@ BEGINinterface(glbl) /* name must also be changed in ENDinterface macro! */ /* next change is v9! */ /* v8 - 2012-03-21 */ prop_t* (*GetLocalHostIP)(void); + uchar* (*GetSourceIPofLocalClient)(void); /* [ar] */ + rsRetVal (*SetSourceIPofLocalClient)(uchar*); /* [ar] */ #undef SIMP_PROP ENDinterface(glbl) #define glblCURR_IF_VERSION 7 /* increment whenever you change the interface structure! */ @@ -89,11 +103,19 @@ ENDinterface(glbl) /* the remaining prototypes */ PROTOTYPEObj(glbl); +extern int glblDebugOnShutdown; /* start debug log when we are shut down */ +extern short janitorInterval; + static inline pid_t glblGetOurPid(void) { return glbl_ourpid; } static inline void glblSetOurPid(pid_t pid) { glbl_ourpid = pid; } void glblPrepCnf(void); void glblProcessCnf(struct cnfobj *o); +void glblProcessTimezone(struct cnfobj *o); +void glblProcessMainQCnf(struct cnfobj *o); +void glblDestructMainqCnfObj(); void glblDoneLoadCnf(void); +const uchar * glblGetWorkDirRaw(void); +tzinfo_t* glblFindTimezoneInfo(char *id); #endif /* #ifndef GLBL_H_INCLUDED */ diff --git a/runtime/janitor.c b/runtime/janitor.c new file mode 100644 index 0000000..b4471a8 --- /dev/null +++ b/runtime/janitor.c @@ -0,0 +1,101 @@ +/* janitor.c - rsyslog's janitor + * + * The rsyslog janitor can be used to periodically clean out + * resources. It was initially developed to close files that + * were not written to for some time (omfile plugin), but has + * a generic interface that can be used for all similar tasks. + * + * Module begun 2014-05-15 by Rainer Gerhards + * + * Copyright (C) 2014 by Rainer Gerhards and Adiscon GmbH. + * + * This file is part of the rsyslog runtime library. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * -or- + * see COPYING.ASL20 in the source distribution + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#include "config.h" +#include <stdlib.h> +#include <assert.h> +#include <string.h> +#include <pthread.h> + +#include "rsyslog.h" +#include "janitor.h" + +static struct janitorEtry *janitorRoot = NULL; /* TODO: move to runConf? */ +static pthread_mutex_t janitorMut = PTHREAD_MUTEX_INITIALIZER; + +rsRetVal +janitorAddEtry(void (*cb)(void*), const char *id, void *pUsr) +{ + struct janitorEtry *etry; + DEFiRet; + CHKmalloc(etry = malloc(sizeof(struct janitorEtry))); + CHKmalloc(etry->id = strdup(id)); + etry->pUsr = pUsr; + etry->cb = cb; + etry->next = janitorRoot; + pthread_mutex_lock(&janitorMut); + janitorRoot = etry; + pthread_mutex_unlock(&janitorMut); + DBGPRINTF("janitor: entry %p, id '%s' added\n", etry, id); +finalize_it: + RETiRet; +} + +rsRetVal +janitorDelEtry(const char *__restrict__ const id) +{ + struct janitorEtry *curr, *prev = NULL; + DEFiRet; + + pthread_mutex_lock(&janitorMut); + for(curr = janitorRoot ; curr != NULL ; curr = curr->next) { + if(!strcmp(curr->id, id)) { + if(prev == NULL) { + janitorRoot = curr->next; + } else { + prev->next = curr->next; + } + free(curr->id); + free(curr); + DBGPRINTF("janitor: deleted entry '%s'\n", id); + ABORT_FINALIZE(RS_RET_OK); + } + prev = curr; + } + DBGPRINTF("janitor: to be deleted entry '%s' not found\n", id); + iRet = RS_RET_NOT_FOUND; +finalize_it: + pthread_mutex_unlock(&janitorMut); + RETiRet; +} + +/* run the janitor; all entries are processed */ +void +janitorRun(void) +{ + struct janitorEtry *curr; + + dbgprintf("janitorRun() called\n"); + pthread_mutex_lock(&janitorMut); + for(curr = janitorRoot ; curr != NULL ; curr = curr->next) { + DBGPRINTF("janitor: processing entry %p, id '%s'\n", + curr, curr->id); + curr->cb(curr->pUsr); + } + pthread_mutex_unlock(&janitorMut); +} diff --git a/runtime/janitor.h b/runtime/janitor.h new file mode 100644 index 0000000..2585721 --- /dev/null +++ b/runtime/janitor.h @@ -0,0 +1,35 @@ +/* rsyslog's janitor + * + * Copyright (C) 2014 by Rainer Gerhards and Adiscon GmbH. + * + * This file is part of the rsyslog runtime library. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * -or- + * see COPYING.ASL20 in the source distribution + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +#ifndef INCLUDED_JANITOR_H +#define INCLUDED_JANITOR_H + +struct janitorEtry { + struct janitorEtry *next; + char *id; /* ID used to remove entry */ + void (*cb)(void *pUsr); + void *pUsr; /* user-settable pointer (passed to callback) */ +}; + +rsRetVal janitorAddEtry(void (*cb)(void*), const char *id, void *pUsr); +rsRetVal janitorDelEtry(const char *__restrict__ const id); +void janitorRun(void); + +#endif /* #ifndef INCLUDED_JANITOR_H */ diff --git a/runtime/libgcry.c b/runtime/libgcry.c index 51c10af..9dabf5f 100644 --- a/runtime/libgcry.c +++ b/runtime/libgcry.c @@ -49,8 +49,12 @@ #include <errno.h> #include "rsyslog.h" +#include "srUtils.h" #include "libgcry.h" +#define READBUF_SIZE 4096 /* size of the read buffer */ + +static rsRetVal rsgcryBlkBegin(gcryfile gf); static rsRetVal eiWriteRec(gcryfile gf, char *recHdr, size_t lenRecHdr, char *buf, size_t lenBuf) @@ -90,20 +94,67 @@ finalize_it: RETiRet; } +static rsRetVal +eiRead(gcryfile gf) +{ + ssize_t nRead; + DEFiRet; + + if(gf->readBuf == NULL) { + CHKmalloc(gf->readBuf = malloc(READBUF_SIZE)); + } + + nRead = read(gf->fd, gf->readBuf, READBUF_SIZE); + if(nRead <= 0) { /* TODO: provide specific EOF case? */ + ABORT_FINALIZE(RS_RET_ERR); + } + gf->readBufMaxIdx = (int16_t) nRead; + gf->readBufIdx = 0; + +finalize_it: + RETiRet; +} + + +/* returns EOF on any kind of error */ +static int +eiReadChar(gcryfile gf) +{ + int c; + + if(gf->readBufIdx >= gf->readBufMaxIdx) { + if(eiRead(gf) != RS_RET_OK) { + c = EOF; + goto finalize_it; + } + } + c = gf->readBuf[gf->readBufIdx++]; +finalize_it: + return c; +} + static rsRetVal eiCheckFiletype(gcryfile gf) { char hdrBuf[128]; size_t toRead, didRead; + sbool bNeedClose = 0; DEFiRet; - CHKiRet(eiOpenRead(gf)); + if(gf->fd == -1) { + bNeedClose = 1; + CHKiRet(eiOpenRead(gf)); + } + if(Debug) memset(hdrBuf, 0, sizeof(hdrBuf)); /* for dbgprintf below! */ toRead = sizeof("FILETYPE:")-1 + sizeof(RSGCRY_FILETYPE_NAME)-1 + 1; didRead = read(gf->fd, hdrBuf, toRead); - close(gf->fd); - DBGPRINTF("eiCheckFiletype read %d bytes: '%s'\n", didRead, hdrBuf); + if(bNeedClose) { + close(gf->fd); + gf->fd = -1; + } + DBGPRINTF("eiCheckFiletype read %zd bytes: '%s'\n", didRead, hdrBuf); if( didRead != toRead || strncmp(hdrBuf, "FILETYPE:" RSGCRY_FILETYPE_NAME "\n", toRead)) iRet = RS_RET_EI_INVLD_FILE; @@ -111,6 +162,98 @@ finalize_it: RETiRet; } +/* rectype/value must be EIF_MAX_*_LEN+1 long! + * returns 0 on success or something else on error/EOF + */ +static rsRetVal +eiGetRecord(gcryfile gf, char *rectype, char *value) +{ + unsigned short i, j; + int c; + DEFiRet; + + c = eiReadChar(gf); + if(c == EOF) { ABORT_FINALIZE(RS_RET_NO_DATA); } + for(i = 0 ; i < EIF_MAX_RECTYPE_LEN ; ++i) { + if(c == ':' || c == EOF) + break; + rectype[i] = c; + c = eiReadChar(gf); + } + if(c != ':') { ABORT_FINALIZE(RS_RET_ERR); } + rectype[i] = '\0'; + j = 0; + for(++i ; i < EIF_MAX_VALUE_LEN ; ++i, ++j) { + c = eiReadChar(gf); + if(c == '\n' || c == EOF) + break; + value[j] = c; + } + if(c != '\n') { ABORT_FINALIZE(RS_RET_ERR); } + value[j] = '\0'; +finalize_it: + RETiRet; +} + +static rsRetVal +eiGetIV(gcryfile gf, uchar *iv, size_t leniv) +{ + char rectype[EIF_MAX_RECTYPE_LEN+1]; + char value[EIF_MAX_VALUE_LEN+1]; + size_t valueLen; + unsigned short i, j; + unsigned char nibble; + DEFiRet; + + CHKiRet(eiGetRecord(gf, rectype, value)); + if(strcmp(rectype, "IV")) { + DBGPRINTF("no IV record found when expected, record type " + "seen is '%s'\n", rectype); + ABORT_FINALIZE(RS_RET_ERR); + } + valueLen = strlen(value); + if(valueLen/2 != leniv) { + DBGPRINTF("length of IV is %zd, expected %zd\n", + valueLen/2, leniv); + ABORT_FINALIZE(RS_RET_ERR); + } + + for(i = j = 0 ; i < valueLen ; ++i) { + if(value[i] >= '0' && value[i] <= '9') + nibble = value[i] - '0'; + else if(value[i] >= 'a' && value[i] <= 'f') + nibble = value[i] - 'a' + 10; + else { + DBGPRINTF("invalid IV '%s'\n", value); + ABORT_FINALIZE(RS_RET_ERR); + } + if(i % 2 == 0) + iv[j] = nibble << 4; + else + iv[j++] |= nibble; + } +finalize_it: + RETiRet; +} + +static rsRetVal +eiGetEND(gcryfile gf, off64_t *offs) +{ + char rectype[EIF_MAX_RECTYPE_LEN+1]; + char value[EIF_MAX_VALUE_LEN+1]; + DEFiRet; + + CHKiRet(eiGetRecord(gf, rectype, value)); + if(strcmp(rectype, "END")) { + DBGPRINTF("no END record found when expected, record type " + "seen is '%s'\n", rectype); + ABORT_FINALIZE(RS_RET_ERR); + } + *offs = atoll(value); +finalize_it: + RETiRet; +} + static rsRetVal eiOpenAppend(gcryfile gf) { @@ -177,13 +320,55 @@ eiClose(gcryfile gf, off64_t offsLogfile) size_t len; if(gf->fd == -1) return; - /* 2^64 is 20 digits, so the snprintf buffer is large enough */ - len = snprintf(offs, sizeof(offs), "%lld", offsLogfile); - eiWriteRec(gf, "END:", 4, offs, len); + if(gf->openMode == 'w') { + /* 2^64 is 20 digits, so the snprintf buffer is large enough */ + len = snprintf(offs, sizeof(offs), "%lld", (long long) offsLogfile); + eiWriteRec(gf, "END:", 4, offs, len); + } + gcry_cipher_close(gf->chd); + free(gf->readBuf); close(gf->fd); + gf->fd = -1; DBGPRINTF("encryption info file %s: closed\n", gf->eiName); } +/* this returns the number of bytes left inside the block or -1, if the block + * size is unbounded. The function automatically handles end-of-block and begins + * to read the next block in this case. + */ +rsRetVal +gcryfileGetBytesLeftInBlock(gcryfile gf, ssize_t *left) +{ + DEFiRet; + if(gf->bytesToBlkEnd == 0) { + DBGPRINTF("libgcry: end of current crypto block\n"); + gcry_cipher_close(gf->chd); + CHKiRet(rsgcryBlkBegin(gf)); + } + *left = gf->bytesToBlkEnd; +finalize_it: + // TODO: remove once this code is sufficiently well-proven + DBGPRINTF("gcryfileGetBytesLeftInBlock returns %lld, iRet %d\n", (long long) *left, iRet); + RETiRet; +} + +/* this is a special functon for use by the rsyslog disk queue subsystem. It + * needs to have the capability to delete state when a queue file is rolled + * over. This simply generates the file name and deletes it. It must take care + * of "all" state files, which currently happens to be a single one. + */ +rsRetVal +gcryfileDeleteState(uchar *logfn) +{ + char fn[MAXFNAME+1]; + DEFiRet; + snprintf(fn, sizeof(fn), "%s%s", logfn, ENCINFO_SUFFIX); + fn[MAXFNAME] = '\0'; /* be on save side */ + DBGPRINTF("crypto provider deletes state file '%s' on request\n", fn); + unlink(fn); + RETiRet; +} + static rsRetVal gcryfileConstruct(gcryctx ctx, gcryfile *pgf, uchar *logfn) { @@ -193,6 +378,7 @@ gcryfileConstruct(gcryctx ctx, gcryfile *pgf, uchar *logfn) CHKmalloc(gf = calloc(1, sizeof(struct gcryfile_s))); gf->ctx = ctx; + gf->fd = -1; snprintf(fn, sizeof(fn), "%s%s", logfn, ENCINFO_SUFFIX); fn[MAXFNAME] = '\0'; /* be on save side */ gf->eiName = (uchar*) strdup(fn); @@ -219,7 +405,12 @@ gcryfileDestruct(gcryfile gf, off64_t offsLogfile) if(gf == NULL) goto done; + DBGPRINTF("libgcry: close file %s\n", gf->eiName); eiClose(gf, offsLogfile); + if(gf->bDeleteOnClose) { + DBGPRINTF("unlink file '%s' due to bDeleteOnClose set\n", gf->eiName); + unlink((char*)gf->eiName); + } free(gf->eiName); free(gf); done: return r; @@ -238,7 +429,7 @@ addPadding(gcryfile pF, uchar *buf, size_t *plen) unsigned i; size_t nPad; nPad = (pF->blkLength - *plen % pF->blkLength) % pF->blkLength; - DBGPRINTF("libgcry: addPadding %d chars, blkLength %d, mod %d, pad %d\n", + DBGPRINTF("libgcry: addPadding %zd chars, blkLength %zd, mod %zd, pad %zd\n", *plen, pF->blkLength, *plen % pF->blkLength, nPad); for(i = 0 ; i < nPad ; ++i) buf[(*plen)+i] = 0x00; @@ -246,13 +437,13 @@ addPadding(gcryfile pF, uchar *buf, size_t *plen) } static inline void -removePadding(char *buf, size_t *plen) +removePadding(uchar *buf, size_t *plen) { unsigned len = (unsigned) *plen; unsigned iSrc, iDst; - char *frstNUL; + uchar *frstNUL; - frstNUL = strchr(buf, 0x00); + frstNUL = (uchar*)strchr((char*)buf, 0x00); if(frstNUL == NULL) goto done; iDst = iSrc = frstNUL - buf; @@ -343,53 +534,123 @@ seedIV(gcryfile gf, uchar **iv) } } -rsRetVal -rsgcryInitCrypt(gcryctx ctx, gcryfile *pgf, uchar *fname) +static inline rsRetVal +readIV(gcryfile gf, uchar **iv) { - gcry_error_t gcryError; - gcryfile gf = NULL; - uchar *iv = NULL; + rsRetVal localRet; DEFiRet; - CHKiRet(gcryfileConstruct(ctx, &gf, fname)); + if(gf->fd == -1) { + while(gf->fd == -1) { + localRet = eiOpenRead(gf); + if(localRet == RS_RET_EI_NO_EXISTS) { + /* wait until it is created */ + srSleep(0, 10000); + } else { + CHKiRet(localRet); + } + } + CHKiRet(eiCheckFiletype(gf)); + } + *iv = malloc(gf->blkLength); /* do NOT zero-out! */ + CHKiRet(eiGetIV(gf, *iv, (size_t) gf->blkLength)); +finalize_it: + RETiRet; +} + +/* this tries to read the END record. HOWEVER, no such record may be + * present, which is the case if we handle a currently-written to queue + * file. On the other hand, the queue file may contain multiple blocks. So + * what we do is try to see if there is a block end or not - and set the + * status accordingly. Note that once we found no end-of-block, we will never + * retry. This is because that case can never happen under current queue + * implementations. -- gerhards, 2013-05-16 + */ +static inline rsRetVal +readBlkEnd(gcryfile gf) +{ + off64_t blkEnd; + DEFiRet; + + iRet = eiGetEND(gf, &blkEnd); + if(iRet == RS_RET_OK) { + gf->bytesToBlkEnd = (ssize_t) blkEnd; + } else if(iRet == RS_RET_NO_DATA) { + gf->bytesToBlkEnd = -1; + } else { + FINALIZE; + } + +finalize_it: + RETiRet; +} - gf->blkLength = gcry_cipher_get_algo_blklen(ctx->algo); - gcryError = gcry_cipher_open(&gf->chd, ctx->algo, ctx->mode, 0); +/* Read the block begin metadata and set our state variables accordingly. Can also + * be used to init the first block in write case. + */ +static rsRetVal +rsgcryBlkBegin(gcryfile gf) +{ + gcry_error_t gcryError; + uchar *iv = NULL; + DEFiRet; + + gcryError = gcry_cipher_open(&gf->chd, gf->ctx->algo, gf->ctx->mode, 0); if (gcryError) { - dbgprintf("gcry_cipher_open failed: %s/%s\n", - gcry_strsource(gcryError), - gcry_strerror(gcryError)); + DBGPRINTF("gcry_cipher_open failed: %s/%s\n", + gcry_strsource(gcryError), gcry_strerror(gcryError)); ABORT_FINALIZE(RS_RET_ERR); } gcryError = gcry_cipher_setkey(gf->chd, gf->ctx->key, gf->ctx->keyLen); if (gcryError) { - dbgprintf("gcry_cipher_setkey failed: %s/%s\n", - gcry_strsource(gcryError), - gcry_strerror(gcryError)); + DBGPRINTF("gcry_cipher_setkey failed: %s/%s\n", + gcry_strsource(gcryError), gcry_strerror(gcryError)); ABORT_FINALIZE(RS_RET_ERR); } - seedIV(gf, &iv); + if(gf->openMode == 'r') { + readIV(gf, &iv); + readBlkEnd(gf); + } else { + seedIV(gf, &iv); + } + gcryError = gcry_cipher_setiv(gf->chd, iv, gf->blkLength); if (gcryError) { - dbgprintf("gcry_cipher_setiv failed: %s/%s\n", - gcry_strsource(gcryError), - gcry_strerror(gcryError)); + DBGPRINTF("gcry_cipher_setiv failed: %s/%s\n", + gcry_strsource(gcryError), gcry_strerror(gcryError)); ABORT_FINALIZE(RS_RET_ERR); } - CHKiRet(eiOpenAppend(gf)); - CHKiRet(eiWriteIV(gf, iv)); - *pgf = gf; + + if(gf->openMode == 'w') { + CHKiRet(eiOpenAppend(gf)); + CHKiRet(eiWriteIV(gf, iv)); + } finalize_it: free(iv); + RETiRet; +} + +rsRetVal +rsgcryInitCrypt(gcryctx ctx, gcryfile *pgf, uchar *fname, char openMode) +{ + gcryfile gf = NULL; + DEFiRet; + + CHKiRet(gcryfileConstruct(ctx, &gf, fname)); + gf->openMode = openMode; + gf->blkLength = gcry_cipher_get_algo_blklen(ctx->algo); + CHKiRet(rsgcryBlkBegin(gf)); + *pgf = gf; +finalize_it: if(iRet != RS_RET_OK && gf != NULL) gcryfileDestruct(gf, -1); RETiRet; } -int +rsRetVal rsgcryEncrypt(gcryfile pF, uchar *buf, size_t *len) { int gcryError; @@ -410,6 +671,35 @@ finalize_it: RETiRet; } +/* TODO: handle multiple blocks + * test-read END record; if present, store offset, else unbounded (current active block) + * when decrypting, check if bound is reached. If yes, split into two blocks, get new IV for + * second one. + */ +rsRetVal +rsgcryDecrypt(gcryfile pF, uchar *buf, size_t *len) +{ + gcry_error_t gcryError; + DEFiRet; + + if(pF->bytesToBlkEnd != -1) + pF->bytesToBlkEnd -= *len; + gcryError = gcry_cipher_decrypt(pF->chd, buf, *len, NULL, 0); + if(gcryError) { + DBGPRINTF("gcry_cipher_decrypt failed: %s/%s\n", + gcry_strsource(gcryError), + gcry_strerror(gcryError)); + ABORT_FINALIZE(RS_RET_ERR); + } + removePadding(buf, len); + // TODO: remove dbgprintf once things are sufficently stable -- rgerhards, 2013-05-16 + dbgprintf("libgcry: decrypted, bytesToBlkEnd %lld, buffer is now '%50.50s'\n", (long long) pF->bytesToBlkEnd, buf); + +finalize_it: + RETiRet; +} + + /* module-init dummy for potential later use */ int diff --git a/runtime/libgcry.h b/runtime/libgcry.h index b77b0f9..ae5a673 100644 --- a/runtime/libgcry.h +++ b/runtime/libgcry.h @@ -21,7 +21,7 @@ #ifndef INCLUDED_LIBGCRY_H #define INCLUDED_LIBGCRY_H #include <stdint.h> - +#include <gcrypt.h> struct gcryctx_s { uchar *key; @@ -38,7 +38,15 @@ struct gcryfile_s { size_t blkLength; /* size of low-level crypto block */ uchar *eiName; /* name of .encinfo file */ int fd; /* descriptor of .encinfo file (-1 if not open) */ + char openMode; /* 'r': read, 'w': write */ gcryctx ctx; + uchar *readBuf; + int16_t readBufIdx; + int16_t readBufMaxIdx; + int8_t bDeleteOnClose; /* for queue support, similar to stream subsys */ + ssize_t bytesToBlkEnd; /* number of bytes remaining in current crypto block + -1 means -> no end (still being writen to, queue files), + 0 means -> end of block, new one must be started. */ }; int gcryGetKeyFromFile(char *fn, char **key, unsigned *keylen); @@ -50,8 +58,12 @@ rsRetVal rsgcrySetAlgo(gcryctx ctx, uchar *modename); gcryctx gcryCtxNew(void); void rsgcryCtxDel(gcryctx ctx); int gcryfileDestruct(gcryfile gf, off64_t offsLogfile); -rsRetVal rsgcryInitCrypt(gcryctx ctx, gcryfile *pgf, uchar *fname); -int rsgcryEncrypt(gcryfile pF, uchar *buf, size_t *len); +rsRetVal rsgcryInitCrypt(gcryctx ctx, gcryfile *pgf, uchar *fname, char openMode); +rsRetVal rsgcryEncrypt(gcryfile pF, uchar *buf, size_t *len); +rsRetVal rsgcryDecrypt(gcryfile pF, uchar *buf, size_t *len); +int gcryGetKeyFromProg(char *cmd, char **key, unsigned *keylen); +rsRetVal gcryfileDeleteState(uchar *fn); +rsRetVal gcryfileGetBytesLeftInBlock(gcryfile gf, ssize_t *left); /* error states */ #define RSGCRYE_EI_OPEN 1 /* error opening .encinfo file */ @@ -62,6 +74,14 @@ int rsgcryEncrypt(gcryfile pF, uchar *buf, size_t *len); #define RSGCRY_FILETYPE_NAME "rsyslog-enrcyption-info" #define ENCINFO_SUFFIX ".encinfo" +/* Note: gf may validly be NULL, e.g. if file has not yet been opened! */ +static inline void +gcryfileSetDeleteOnClose(gcryfile gf, int val) +{ + if(gf != NULL) + gf->bDeleteOnClose = val; +} + static inline int rsgcryAlgoname2Algo(char *algoname) { if(!strcmp((char*)algoname, "3DES")) return GCRY_CIPHER_3DES; diff --git a/runtime/libgcry_common.c b/runtime/libgcry_common.c index 63b5e5d..07a524d 100644 --- a/runtime/libgcry_common.c +++ b/runtime/libgcry_common.c @@ -98,7 +98,7 @@ fprintf(stderr, "pre execve: %s\n", cmd); */ /* we should never reach this point, but if we do, we terminate */ -done: return; + return; } diff --git a/runtime/librsgt.c b/runtime/librsgt.c index ae0b0df..f824749 100644 --- a/runtime/librsgt.c +++ b/runtime/librsgt.c @@ -75,7 +75,7 @@ reportGTAPIErr(gtctx ctx, gtfile gf, char *apiname, int ecode) char errbuf[4096]; snprintf(errbuf, sizeof(errbuf), "%s[%s:%d]: %s", (gf == NULL) ? (uchar*)"" : gf->sigfilename, - apiname, ecode, GT_getErrorString(ecode)); + apiname, ecode, GTHTTP_getErrorString(ecode)); errbuf[sizeof(errbuf)-1] = '\0'; reportErr(ctx, errbuf); } @@ -285,7 +285,9 @@ int tlv8Write(gtfile gf, int flags, int tlvtype, int len) { int r; - r = tlvbufAddOctet(gf, (flags << 5)|tlvtype); + assert((flags & RSGT_TYPE_MASK) == 0); + assert((tlvtype & RSGT_TYPE_MASK) == tlvtype); + r = tlvbufAddOctet(gf, (flags & ~RSGT_FLAG_TLV16) | tlvtype); if(r != 0) goto done; r = tlvbufAddOctet(gf, len & 0xff); done: return r; @@ -296,7 +298,9 @@ tlv16Write(gtfile gf, int flags, int tlvtype, uint16_t len) { uint16_t typ; int r; - typ = ((flags|1) << 13)|tlvtype; + assert((flags & RSGT_TYPE_MASK) == 0); + assert((tlvtype >> 8 & RSGT_TYPE_MASK) == (tlvtype >> 8)); + typ = ((flags | RSGT_FLAG_TLV16) << 8) | tlvtype; r = tlvbufAddOctet(gf, typ >> 8); if(r != 0) goto done; r = tlvbufAddOctet(gf, typ & 0xff); @@ -402,6 +406,7 @@ readStateFile(gtfile gf) free(gf->blkStrtHash); goto err; } + close(fd); return; err: diff --git a/runtime/librsgt.h b/runtime/librsgt.h index bfcc462..bf9c9c3 100644 --- a/runtime/librsgt.h +++ b/runtime/librsgt.h @@ -151,7 +151,10 @@ struct rsgtstatefile { }; /* Flags and record types for TLV handling */ +#define RSGT_FLAG_NONCRIT 0x80 +#define RSGT_FLAG_FORWARD 0x40 #define RSGT_FLAG_TLV16 0x20 +#define RSGT_TYPE_MASK 0x1f /* error states */ #define RSGTE_IO 1 /* any kind of io error */ diff --git a/runtime/librsgt_read.c b/runtime/librsgt_read.c index 25c0db4..bcb0099 100644 --- a/runtime/librsgt_read.c +++ b/runtime/librsgt_read.c @@ -249,7 +249,7 @@ rsgt_tlvRecRead(FILE *fp, tlvrecord_t *rec) NEXTC; rec->hdr[0] = c; rec->tlvtype = c & 0x1f; - if(c & 0x20) { /* tlv16? */ + if(c & RSGT_FLAG_TLV16) { /* tlv16? */ rec->lenHdr = 4; NEXTC; rec->hdr[1] = c; @@ -267,7 +267,7 @@ rsgt_tlvRecRead(FILE *fp, tlvrecord_t *rec) rec->tlvlen = c; } if(fread(rec->data, (size_t) rec->tlvlen, 1, fp) != 1) { - r = RSGTE_IO; + r = feof(fp) ? RSGTE_EOF : RSGTE_IO; goto done; } @@ -290,7 +290,7 @@ rsgt_tlvDecodeSUBREC(tlvrecord_t *rec, uint16_t *stridx, tlvrecord_t *newrec) c = rec->data[(*stridx)++]; newrec->hdr[0] = c; newrec->tlvtype = c & 0x1f; - if(c & 0x20) { /* tlv16? */ + if(c & RSGT_FLAG_TLV16) { /* tlv16? */ newrec->lenHdr = 4; if(rec->tlvlen == *stridx) {r=RSGTE_LEN; goto done;} c = rec->data[(*stridx)++]; @@ -611,9 +611,9 @@ rsgt_printBLOCK_SIG(FILE *fp, block_sig_t *bs, uint8_t verbose) fprintf(fp, "\tIV............: "); outputHexBlob(fp, bs->iv, getIVLen(bs), verbose); fputc('\n', fp); - fprintf(fp, "\tRecord Count..: %llu\n", bs->recCount); + fprintf(fp, "\tRecord Count..: %llu\n", (long long unsigned) bs->recCount); fprintf(fp, "\tSignature Type: %s\n", sigTypeName(bs->sigID)); - fprintf(fp, "\tSignature Len.: %u\n", bs->sig.der.len); + fprintf(fp, "\tSignature Len.: %u\n", (unsigned) bs->sig.der.len); fprintf(fp, "\tSignature.....: "); outputHexBlob(fp, bs->sig.der.data, bs->sig.der.len, verbose); fputc('\n', fp); diff --git a/runtime/lmcry_gcry.c b/runtime/lmcry_gcry.c index 0a9b94b..9a0c007 100644 --- a/runtime/lmcry_gcry.c +++ b/runtime/lmcry_gcry.c @@ -43,17 +43,30 @@ DEFobjCurrIf(errmsg) DEFobjCurrIf(glbl) /* tables for interfacing with the v6 config system */ -static struct cnfparamdescr cnfpdescr[] = { +static struct cnfparamdescr cnfpdescrRegular[] = { { "cry.key", eCmdHdlrGetWord, 0 }, { "cry.keyfile", eCmdHdlrGetWord, 0 }, { "cry.keyprogram", eCmdHdlrGetWord, 0 }, { "cry.mode", eCmdHdlrGetWord, 0 }, /* CBC, ECB, etc */ { "cry.algo", eCmdHdlrGetWord, 0 } }; -static struct cnfparamblk pblk = +static struct cnfparamblk pblkRegular = { CNFPARAMBLK_VERSION, - sizeof(cnfpdescr)/sizeof(struct cnfparamdescr), - cnfpdescr + sizeof(cnfpdescrRegular)/sizeof(struct cnfparamdescr), + cnfpdescrRegular + }; + +static struct cnfparamdescr cnfpdescrQueue[] = { + { "queue.cry.key", eCmdHdlrGetWord, 0 }, + { "queue.cry.keyfile", eCmdHdlrGetWord, 0 }, + { "queue.cry.keyprogram", eCmdHdlrGetWord, 0 }, + { "queue.cry.mode", eCmdHdlrGetWord, 0 }, /* CBC, ECB, etc */ + { "queue.cry.algo", eCmdHdlrGetWord, 0 } +}; +static struct cnfparamblk pblkQueue = + { CNFPARAMBLK_VERSION, + sizeof(cnfpdescrQueue)/sizeof(struct cnfparamdescr), + cnfpdescrQueue }; @@ -85,7 +98,7 @@ ENDobjDestruct(lmcry_gcry) * Defaults are expected to have been set during construction. */ static rsRetVal -SetCnfParam(void *pT, struct nvlst *lst) +SetCnfParam(void *pT, struct nvlst *lst, int paramType) { lmcry_gcry_t *pThis = (lmcry_gcry_t*) pT; int i, r; @@ -97,34 +110,41 @@ SetCnfParam(void *pT, struct nvlst *lst) uchar *mode = NULL; int nKeys; /* number of keys (actually methods) specified */ struct cnfparamvals *pvals; + struct cnfparamblk *pblk; DEFiRet; + pblk = (paramType == CRYPROV_PARAMTYPE_REGULAR ) ? &pblkRegular : &pblkQueue; nKeys = 0; - pvals = nvlstGetParams(lst, &pblk, NULL); + pvals = nvlstGetParams(lst, pblk, NULL); if(Debug) { dbgprintf("param blk in lmcry_gcry:\n"); - cnfparamsPrint(&pblk, pvals); + cnfparamsPrint(pblk, pvals); } - for(i = 0 ; i < pblk.nParams ; ++i) { + for(i = 0 ; i < pblk->nParams ; ++i) { if(!pvals[i].bUsed) continue; - if(!strcmp(pblk.descr[i].name, "cry.key")) { + if(!strcmp(pblk->descr[i].name, "cry.key") || + !strcmp(pblk->descr[i].name, "queue.cry.key")) { key = (uchar*) es_str2cstr(pvals[i].val.d.estr, NULL); ++nKeys; - } else if(!strcmp(pblk.descr[i].name, "cry.keyfile")) { + } else if(!strcmp(pblk->descr[i].name, "cry.keyfile") || + !strcmp(pblk->descr[i].name, "queue.cry.keyfile")) { keyfile = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); ++nKeys; - } else if(!strcmp(pblk.descr[i].name, "cry.keyprogram")) { + } else if(!strcmp(pblk->descr[i].name, "cry.keyprogram") || + !strcmp(pblk->descr[i].name, "queue.cry.keyprogram")) { keyprogram = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); ++nKeys; - } else if(!strcmp(pblk.descr[i].name, "cry.mode")) { + } else if(!strcmp(pblk->descr[i].name, "cry.mode") || + !strcmp(pblk->descr[i].name, "queue.cry.mode")) { mode = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); - } else if(!strcmp(pblk.descr[i].name, "cry.algo")) { + } else if(!strcmp(pblk->descr[i].name, "cry.algo") || + !strcmp(pblk->descr[i].name, "queue.cry.algo")) { algo = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); } else { DBGPRINTF("lmcry_gcry: program error, non-handled " - "param '%s'\n", pblk.descr[i].name); + "param '%s'\n", pblk->descr[i].name); } } if(algo != NULL) { @@ -177,7 +197,7 @@ SetCnfParam(void *pT, struct nvlst *lst) ABORT_FINALIZE(RS_RET_INVALID_PARAMS); } - cnfparamvalsDestruct(pvals, &pblk); + cnfparamvalsDestruct(pvals, pblk); if(key != NULL) { memset(key, 0, strlen((char*)key)); free(key); @@ -189,15 +209,33 @@ finalize_it: RETiRet; } +static void +SetDeleteOnClose(void *pF, int val) +{ + gcryfileSetDeleteOnClose(pF, val); +} + +static rsRetVal +GetBytesLeftInBlock(void *pF, ssize_t *left) +{ + return gcryfileGetBytesLeftInBlock((gcryfile) pF, left); +} static rsRetVal -OnFileOpen(void *pT, uchar *fn, void *pGF) +DeleteStateFiles(uchar *logfn) +{ + return gcryfileDeleteState(logfn); +} + +static rsRetVal +OnFileOpen(void *pT, uchar *fn, void *pGF, char openMode) { lmcry_gcry_t *pThis = (lmcry_gcry_t*) pT; gcryfile *pgf = (gcryfile*) pGF; DEFiRet; + DBGPRINTF("lmcry_gcry: open file '%s', mode '%c'\n", fn, openMode); - CHKiRet(rsgcryInitCrypt(pThis->ctx, pgf, fn)); + CHKiRet(rsgcryInitCrypt(pThis->ctx, pgf, fn, openMode)); finalize_it: /* TODO: enable this error message (need to cleanup loop first ;)) errmsg.LogError(0, iRet, "Encryption Provider" @@ -207,6 +245,16 @@ finalize_it: } static rsRetVal +Decrypt(void *pF, uchar *rec, size_t *lenRec) +{ + DEFiRet; + iRet = rsgcryDecrypt(pF, rec, lenRec); + + RETiRet; +} + + +static rsRetVal Encrypt(void *pF, uchar *rec, size_t *lenRec) { DEFiRet; @@ -231,10 +279,14 @@ CODESTARTobjQueryInterface(lmcry_gcry) } pIf->Construct = (rsRetVal(*)(void*)) lmcry_gcryConstruct; pIf->SetCnfParam = SetCnfParam; + pIf->SetDeleteOnClose = SetDeleteOnClose; pIf->Destruct = (rsRetVal(*)(void*)) lmcry_gcryDestruct; pIf->OnFileOpen = OnFileOpen; pIf->Encrypt = Encrypt; + pIf->Decrypt = Decrypt; pIf->OnFileClose = OnFileClose; + pIf->DeleteStateFiles = DeleteStateFiles; + pIf->GetBytesLeftInBlock = GetBytesLeftInBlock; finalize_it: ENDobjQueryInterface(lmcry_gcry) diff --git a/runtime/lmsig_gt.c b/runtime/lmsig_gt.c index e9194c7..116a48d 100644 --- a/runtime/lmsig_gt.c +++ b/runtime/lmsig_gt.c @@ -89,7 +89,11 @@ SetCnfParam(void *pT, struct nvlst *lst) int i; uchar *cstr; struct cnfparamvals *pvals; + DEFiRet; pvals = nvlstGetParams(lst, &pblk, NULL); + if(pvals == NULL) { + ABORT_FINALIZE(RS_RET_MISSING_CNFPARAMS); + } if(Debug) { dbgprintf("sig param blk in lmsig_gt:\n"); cnfparamsPrint(&pblk, pvals); @@ -120,8 +124,10 @@ SetCnfParam(void *pT, struct nvlst *lst) "param '%s'\n", pblk.descr[i].name); } } - cnfparamvalsDestruct(pvals, &pblk); - return RS_RET_OK; +finalize_it: + if(pvals != NULL) + cnfparamvalsDestruct(pvals, &pblk); + RETiRet; } diff --git a/runtime/lookup.c b/runtime/lookup.c new file mode 100644 index 0000000..f7ed899 --- /dev/null +++ b/runtime/lookup.c @@ -0,0 +1,377 @@ +/* lookup.c + * Support for lookup tables in RainerScript. + * + * Copyright 2013 Adiscon GmbH. + * + * This file is part of the rsyslog runtime library. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * -or- + * see COPYING.ASL20 in the source distribution + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +#include "config.h" +#include <stdlib.h> +#include <string.h> +#include <errno.h> +#include <sys/types.h> +#include <sys/stat.h> +#include <fcntl.h> +#include <json.h> +#include <assert.h> + +#include "rsyslog.h" +#include "srUtils.h" +#include "errmsg.h" +#include "lookup.h" +#include "msg.h" +#include "rsconf.h" +#include "dirty.h" +#include "unicode-helper.h" + +/* definitions for objects we access */ +DEFobjStaticHelpers +DEFobjCurrIf(errmsg) +DEFobjCurrIf(glbl) + +/* forward definitions */ +static rsRetVal lookupReadFile(lookup_t *pThis); + +/* static data */ +/* tables for interfacing with the v6 config system (as far as we need to) */ +static struct cnfparamdescr modpdescr[] = { + { "name", eCmdHdlrString, CNFPARAM_REQUIRED }, + { "file", eCmdHdlrString, CNFPARAM_REQUIRED } +}; +static struct cnfparamblk modpblk = + { CNFPARAMBLK_VERSION, + sizeof(modpdescr)/sizeof(struct cnfparamdescr), + modpdescr + }; + + +/* create a new lookup table object AND include it in our list of + * lookup tables. + */ +rsRetVal +lookupNew(lookup_t **ppThis) +{ + lookup_t *pThis = NULL; + DEFiRet; + + CHKmalloc(pThis = malloc(sizeof(lookup_t))); + pthread_rwlock_init(&pThis->rwlock, NULL); + pThis->name = NULL; + + if(loadConf->lu_tabs.root == NULL) { + loadConf->lu_tabs.root = pThis; + pThis->next = NULL; + } else { + pThis->next = loadConf->lu_tabs.last; + } + loadConf->lu_tabs.last = pThis; + + *ppThis = pThis; +finalize_it: + if(iRet != RS_RET_OK) { + free(pThis); + } + RETiRet; +} +void +lookupDestruct(lookup_t *pThis) +{ + pthread_rwlock_destroy(&pThis->rwlock); + free(pThis->name); + free(pThis); +} + +void +lookupInitCnf(lookup_tables_t *lu_tabs) +{ + lu_tabs->root = NULL; + lu_tabs->last = NULL; +} + + +/* comparison function for qsort() and string array compare + * this is for the string lookup table type + */ +static int +qs_arrcmp_strtab(const void *s1, const void *s2) +{ + return ustrcmp(((lookup_string_tab_etry_t*)s1)->key, ((lookup_string_tab_etry_t*)s2)->key); +} +/* comparison function for bsearch() and string array compare + * this is for the string lookup table type + */ +static int +bs_arrcmp_strtab(const void *s1, const void *s2) +{ + return strcmp((char*)s1, (char*)((lookup_string_tab_etry_t*)s2)->key); +} + +rsRetVal +lookupBuildTable(lookup_t *pThis, struct json_object *jroot) +{ + //struct json_object *jversion, *jnomatch, *jtype, *jtab; + struct json_object *jtab; + struct json_object *jrow, *jindex, *jvalue; + uint32_t i; + uint32_t maxStrSize; + DEFiRet; + +#if 0 // enable when we continue to work on this module + jversion = json_object_object_get(jroot, "version"); + jnomatch = json_object_object_get(jroot, "nomatch"); + jtype = json_object_object_get(jroot, "type"); +#endif + jtab = json_object_object_get(jroot, "table"); + pThis->nmemb = json_object_array_length(jtab); + CHKmalloc(pThis->d.strtab = malloc(pThis->nmemb * sizeof(lookup_string_tab_etry_t))); + + maxStrSize = 0; + for(i = 0 ; i < pThis->nmemb ; ++i) { + jrow = json_object_array_get_idx(jtab, i); + jindex = json_object_object_get(jrow, "index"); + jvalue = json_object_object_get(jrow, "value"); + CHKmalloc(pThis->d.strtab[i].key = (uchar*) strdup(json_object_get_string(jindex))); + CHKmalloc(pThis->d.strtab[i].val = (uchar*) strdup(json_object_get_string(jvalue))); + maxStrSize += ustrlen(pThis->d.strtab[i].val); + } + + qsort(pThis->d.strtab, pThis->nmemb, sizeof(lookup_string_tab_etry_t), qs_arrcmp_strtab); +dbgprintf("DDDD: table loaded (max size %u):\n", maxStrSize); +for(i = 0 ; i < pThis->nmemb ; ++i) + dbgprintf("key: '%s', val: '%s'\n", pThis->d.strtab[i].key, pThis->d.strtab[i].val); + +finalize_it: + RETiRet; +} + + +/* find a lookup table. This is a naive O(n) algo, but this really + * doesn't matter as it is called only a few times during config + * load. The function returns either a pointer to the requested + * table or NULL, if not found. + */ +lookup_t * +lookupFindTable(uchar *name) +{ + lookup_t *curr; + + for(curr = loadConf->lu_tabs.root ; curr != NULL ; curr = curr->next) { + if(!ustrcmp(curr->name, name)) + break; + } + return curr; +} + + +/* this reloads a lookup table. This is done while the engine is running, + * as such the function must ensure proper locking and proper order of + * operations (so that nothing can interfere). If the table cannot be loaded, + * the old table is continued to be used. + */ +static rsRetVal +lookupReload(lookup_t *pThis) +{ + uint32_t i; + lookup_t newlu; /* dummy to be able to use support functions without + affecting current settings. */ + DEFiRet; + + DBGPRINTF("reload requested for lookup table '%s'\n", pThis->name); + memset(&newlu, 0, sizeof(newlu)); + CHKmalloc(newlu.name = ustrdup(pThis->name)); + CHKmalloc(newlu.filename = ustrdup(pThis->filename)); + CHKiRet(lookupReadFile(&newlu)); + /* all went well, copy over data members */ + pthread_rwlock_wrlock(&pThis->rwlock); + for(i = 0 ; i < pThis->nmemb ; ++i) { + free(pThis->d.strtab[i].key), /* we don't care about exec order of frees */ + free(pThis->d.strtab[i].val); + } + free(pThis->d.strtab); + pThis->d.strtab = newlu.d.strtab; /* hand table AND ALL STRINGS over! */ + pthread_rwlock_unlock(&pThis->rwlock); + errmsg.LogError(0, RS_RET_OK, "lookup table '%s' reloaded from file '%s'", + pThis->name, pThis->filename); +finalize_it: + free(newlu.name); + free(newlu.filename); + RETiRet; +} + + +/* reload all lookup tables on HUP */ +void +lookupDoHUP() +{ + lookup_t *lu; + for(lu = loadConf->lu_tabs.root ; lu != NULL ; lu = lu->next) { + lookupReload(lu); + } +} + + +/* returns either a pointer to the value (read only!) or NULL + * if either the key could not be found or an error occured. + * Note that an estr_t object is returned. The caller is + * responsible for freeing it. + */ +es_str_t * +lookupKey_estr(lookup_t *pThis, uchar *key) +{ + lookup_string_tab_etry_t *etry; + char *r; + es_str_t *estr; + + pthread_rwlock_rdlock(&pThis->rwlock); + etry = bsearch(key, pThis->d.strtab, pThis->nmemb, sizeof(lookup_string_tab_etry_t), bs_arrcmp_strtab); + if(etry == NULL) { + r = ""; // TODO: use set default + } else { + r = (char*)etry->val; + } + estr = es_newStrFromCStr(r, strlen(r)); + pthread_rwlock_unlock(&pThis->rwlock); + return estr; +} + + +/* note: widely-deployed json_c 0.9 does NOT support incremental + * parsing. In order to keep compatible with e.g. Ubuntu 12.04LTS, + * we read the file into one big memory buffer and parse it at once. + * While this is not very elegant, it will not pose any real issue + * for "reasonable" lookup tables (and "unreasonably" large ones + * will probably have other issues as well...). + */ +static rsRetVal +lookupReadFile(lookup_t *pThis) +{ + struct json_tokener *tokener = NULL; + struct json_object *json = NULL; + int eno = errno; + char errStr[1024]; + char *iobuf = NULL; + int fd; + ssize_t nread; + struct stat sb; + DEFiRet; + + + if(stat((char*)pThis->filename, &sb) == -1) { + eno = errno; + errmsg.LogError(0, RS_RET_FILE_NOT_FOUND, + "lookup table file '%s' stat failed: %s", + pThis->filename, rs_strerror_r(eno, errStr, sizeof(errStr))); + ABORT_FINALIZE(RS_RET_FILE_NOT_FOUND); + } + + CHKmalloc(iobuf = malloc(sb.st_size)); + + if((fd = open((const char*) pThis->filename, O_RDONLY)) == -1) { + eno = errno; + errmsg.LogError(0, RS_RET_FILE_NOT_FOUND, + "lookup table file '%s' could not be opened: %s", + pThis->filename, rs_strerror_r(eno, errStr, sizeof(errStr))); + ABORT_FINALIZE(RS_RET_FILE_NOT_FOUND); + } + + tokener = json_tokener_new(); + nread = read(fd, iobuf, sb.st_size); + if(nread != (ssize_t) sb.st_size) { + eno = errno; + errmsg.LogError(0, RS_RET_READ_ERR, + "lookup table file '%s' read error: %s", + pThis->filename, rs_strerror_r(eno, errStr, sizeof(errStr))); + ABORT_FINALIZE(RS_RET_READ_ERR); + } + + json = json_tokener_parse_ex(tokener, iobuf, sb.st_size); + if(json == NULL) { + errmsg.LogError(0, RS_RET_JSON_PARSE_ERR, + "lookup table file '%s' json parsing error", + pThis->filename); + ABORT_FINALIZE(RS_RET_JSON_PARSE_ERR); + } + free(iobuf); /* early free to sever resources*/ + iobuf = NULL; /* make sure no double-free */ + + /* got json object, now populate our own in-memory structure */ + CHKiRet(lookupBuildTable(pThis, json)); + +finalize_it: + free(iobuf); + if(tokener != NULL) + json_tokener_free(tokener); + if(json != NULL) + json_object_put(json); + RETiRet; +} + + +rsRetVal +lookupProcessCnf(struct cnfobj *o) +{ + struct cnfparamvals *pvals; + lookup_t *lu; + short i; + DEFiRet; + + pvals = nvlstGetParams(o->nvlst, &modpblk, NULL); + if(pvals == NULL) { + ABORT_FINALIZE(RS_RET_MISSING_CNFPARAMS); + } + DBGPRINTF("lookupProcessCnf params:\n"); + cnfparamsPrint(&modpblk, pvals); + + CHKiRet(lookupNew(&lu)); + + for(i = 0 ; i < modpblk.nParams ; ++i) { + if(!pvals[i].bUsed) + continue; + if(!strcmp(modpblk.descr[i].name, "file")) { + CHKmalloc(lu->filename = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL)); + } else if(!strcmp(modpblk.descr[i].name, "name")) { + CHKmalloc(lu->name = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL)); + } else { + dbgprintf("lookup_table: program error, non-handled " + "param '%s'\n", modpblk.descr[i].name); + } + } + CHKiRet(lookupReadFile(lu)); + DBGPRINTF("lookup table '%s' loaded from file '%s'\n", lu->name, lu->filename); + +finalize_it: + cnfparamvalsDestruct(pvals, &modpblk); + RETiRet; +} + +void +lookupClassExit(void) +{ + objRelease(glbl, CORE_COMPONENT); + objRelease(errmsg, CORE_COMPONENT); +} + +rsRetVal +lookupClassInit(void) +{ + DEFiRet; + CHKiRet(objGetObjInterface(&obj)); + CHKiRet(objUse(glbl, CORE_COMPONENT)); + CHKiRet(objUse(errmsg, CORE_COMPONENT)); +finalize_it: + RETiRet; +} diff --git a/runtime/lookup.h b/runtime/lookup.h new file mode 100644 index 0000000..c478d67 --- /dev/null +++ b/runtime/lookup.h @@ -0,0 +1,57 @@ +/* header for lookup.c + * + * Copyright 2013 Adiscon GmbH. + * + * This file is part of the rsyslog runtime library. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * -or- + * see COPYING.ASL20 in the source distribution + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +#ifndef INCLUDED_LOOKUP_H +#define INCLUDED_LOOKUP_H +#include <libestr.h> + +struct lookup_tables_s { + lookup_t *root; /* the root of the template list */ + lookup_t *last; /* points to the last element of the template list */ +}; + +struct lookup_string_tab_etry_s { + uchar *key; + uchar *val; +}; + +/* a single lookup table */ +struct lookup_s { + pthread_rwlock_t rwlock; /* protect us in case of dynamic reloads */ + uchar *name; + uchar *filename; + uint32_t nmemb; + union { + lookup_string_tab_etry_t *strtab; + } d; + lookup_t *next; +}; + +/* prototypes */ +void lookupInitCnf(lookup_tables_t *lu_tabs); +rsRetVal lookupProcessCnf(struct cnfobj *o); +lookup_t *lookupFindTable(uchar *name); +es_str_t * lookupKey_estr(lookup_t *pThis, uchar *key); +void lookupDestruct(lookup_t *pThis); +void lookupClassExit(void); +void lookupDoHUP(); +rsRetVal lookupClassInit(void); + +#endif /* #ifndef INCLUDED_LOOKUP_H */ diff --git a/runtime/module-template.h b/runtime/module-template.h index 8a958f9..9fadf63 100644 --- a/runtime/module-template.h +++ b/runtime/module-template.h @@ -4,25 +4,23 @@ * * File begun on 2007-07-25 by RGerhards * - * Copyright 2007-2012 Adiscon GmbH. This is Adiscon-exclusive code without any other - * contributions. *** GPLv3 *** + * Copyright 2007-2012 Adiscon GmbH. * * This file is part of the rsyslog runtime library. * - * Rsyslog is free software: you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation, either version 3 of the License, or - * (at your option) any later version. - * - * Rsyslog is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * - * You should have received a copy of the GNU General Public License - * along with Rsyslog. If not, see <http://www.gnu.org/licenses/>. - * - * A copy of the GPL can be found in the file "COPYING" in this distribution. + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * -or- + * see COPYING.ASL20 in the source distribution + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. */ #ifndef MODULE_TEMPLATE_H_INCLUDED #define MODULE_TEMPLATE_H_INCLUDED 1 @@ -175,6 +173,44 @@ static rsRetVal freeInstance(void* pModData)\ RETiRet;\ } +/* createWrkrInstance() + */ +#define BEGINcreateWrkrInstance \ +static rsRetVal createWrkrInstance(wrkrInstanceData_t **ppWrkrData, instanceData *pData)\ + {\ + DEFiRet; /* store error code here */\ + wrkrInstanceData_t *pWrkrData; /* use this to point to data elements */ + +#define CODESTARTcreateWrkrInstance \ + if((pWrkrData = calloc(1, sizeof(wrkrInstanceData_t))) == NULL) {\ + *ppWrkrData = NULL;\ + ENDfunc \ + return RS_RET_OUT_OF_MEMORY;\ + } \ + pWrkrData->pData = pData; + +#define ENDcreateWrkrInstance \ + *ppWrkrData = pWrkrData;\ + RETiRet;\ +} + +/* freeWrkrInstance */ +#define BEGINfreeWrkrInstance \ +static rsRetVal freeWrkrInstance(void* pd)\ +{\ + DEFiRet;\ + wrkrInstanceData_t *pWrkrData; + +#define CODESTARTfreeWrkrInstance \ + pWrkrData = (wrkrInstanceData_t*) pd; + +#define ENDfreeWrkrInstance \ + if(pWrkrData != NULL)\ + free(pWrkrData); /* we need to free this in any case */\ + RETiRet;\ +} + + /* isCompatibleWithFeature() */ #define BEGINisCompatibleWithFeature \ @@ -194,7 +230,7 @@ static rsRetVal isCompatibleWithFeature(syslogFeature __attribute__((unused)) eF * introduced in v4.3.3 -- rgerhards, 2009-04-27 */ #define BEGINbeginTransaction \ -static rsRetVal beginTransaction(instanceData __attribute__((unused)) *pData)\ +static rsRetVal beginTransaction(wrkrInstanceData_t __attribute__((unused)) *pWrkrData)\ {\ DEFiRet; @@ -205,11 +241,28 @@ static rsRetVal beginTransaction(instanceData __attribute__((unused)) *pData)\ } +/* commitTransaction() + * Commits a transaction. Note that beginTransaction() must have been + * called before this entry point. It receives the full batch of messages + * to be processed in pParam parameter. + * introduced in v8.1.3 -- rgerhards, 2013-12-04 + */ +#define BEGINcommitTransaction \ +static rsRetVal commitTransaction(wrkrInstanceData_t __attribute__((unused)) *const pWrkrData, actWrkrIParams_t *const pParams, const unsigned nParams)\ +{\ + DEFiRet; + +#define CODESTARTcommitTransaction /* currently empty, but may be extended */ + +#define ENDcommitTransaction \ + RETiRet;\ +} + /* endTransaction() * introduced in v4.3.3 -- rgerhards, 2009-04-27 */ #define BEGINendTransaction \ -static rsRetVal endTransaction(instanceData __attribute__((unused)) *pData)\ +static rsRetVal endTransaction(wrkrInstanceData_t __attribute__((unused)) *pWrkrData)\ {\ DEFiRet; @@ -223,7 +276,7 @@ static rsRetVal endTransaction(instanceData __attribute__((unused)) *pData)\ /* doAction() */ #define BEGINdoAction \ -static rsRetVal doAction(uchar __attribute__((unused)) **ppString, unsigned __attribute__((unused)) iMsgOpts, instanceData __attribute__((unused)) *pData)\ +static rsRetVal doAction(uchar __attribute__((unused)) **ppString, wrkrInstanceData_t __attribute__((unused)) *pWrkrData)\ {\ DEFiRet; @@ -372,6 +425,44 @@ static rsRetVal newInpInst(struct nvlst *lst)\ } + +/* newParserInst() + * This is basically the equivalent to newActInst() for creating parser + * module (listener) instances. + */ +#define BEGINnewParserInst \ +static rsRetVal newParserInst(struct nvlst *lst, void *pinst)\ +{\ + instanceConf_t *inst; \ + DEFiRet; + +#define CODESTARTnewParserInst \ + +#define CODE_STD_FINALIZERnewParserInst + +#define ENDnewParserInst \ + if(iRet == RS_RET_OK) \ + *((instanceConf_t**)pinst) = inst; \ + RETiRet;\ +} + + +/* freeParserInst */ +#define BEGINfreeParserInst \ +static rsRetVal freeParserInst(void* pi)\ +{\ + DEFiRet;\ + instanceConf_t *pInst; + +#define CODESTARTfreeParserInst\ + pInst = (instanceConf_t*) pi; + +#define ENDfreeParserInst\ + if(pInst != NULL)\ + free(pInst);\ + RETiRet;\ +} + /* tryResume() * This entry point is called to check if a module can resume operations. This * happens when a module requested that it be suspended. In suspended state, @@ -382,12 +473,12 @@ static rsRetVal newInpInst(struct nvlst *lst)\ * rgerhard, 2007-08-02 */ #define BEGINtryResume \ -static rsRetVal tryResume(instanceData __attribute__((unused)) *pData)\ +static rsRetVal tryResume(wrkrInstanceData_t __attribute__((unused)) *pWrkrData)\ {\ DEFiRet; #define CODESTARTtryResume \ - assert(pData != NULL); + assert(pWrkrData != NULL); #define ENDtryResume \ RETiRet;\ @@ -448,8 +539,7 @@ static rsRetVal queryEtryPt(uchar *name, rsRetVal (**pEtryPoint)())\ } /* the following definition is the standard block for queryEtryPt for output - * modules. This can be used if no specific handling (e.g. to cover version - * differences) is needed. + * modules WHICH DO NOT SUPPORT TRANSACTIONS. */ #define CODEqueryEtryPt_STD_OMOD_QUERIES \ CODEqueryEtryPt_STD_MOD_QUERIES \ @@ -467,6 +557,34 @@ static rsRetVal queryEtryPt(uchar *name, rsRetVal (**pEtryPoint)())\ *pEtryPoint = tryResume;\ } +/* the following definition is the standard block for queryEtryPt for output + * modules using the transaction interface. + */ +#define CODEqueryEtryPt_STD_OMODTX_QUERIES \ + CODEqueryEtryPt_STD_MOD_QUERIES \ + else if(!strcmp((char*) name, "beginTransaction")) {\ + *pEtryPoint = beginTransaction;\ + } else if(!strcmp((char*) name, "commitTransaction")) {\ + *pEtryPoint = commitTransaction;\ + } else if(!strcmp((char*) name, "dbgPrintInstInfo")) {\ + *pEtryPoint = dbgPrintInstInfo;\ + } else if(!strcmp((char*) name, "freeInstance")) {\ + *pEtryPoint = freeInstance;\ + } else if(!strcmp((char*) name, "parseSelectorAct")) {\ + *pEtryPoint = parseSelectorAct;\ + } else if(!strcmp((char*) name, "isCompatibleWithFeature")) {\ + *pEtryPoint = isCompatibleWithFeature;\ + } else if(!strcmp((char*) name, "tryResume")) {\ + *pEtryPoint = tryResume;\ + } + +/* standard queries for output module interface in rsyslog v8+ */ +#define CODEqueryEtryPt_STD_OMOD8_QUERIES \ + else if(!strcmp((char*) name, "createWrkrInstance")) {\ + *pEtryPoint = createWrkrInstance;\ + } else if(!strcmp((char*) name, "freeWrkrInstance")) {\ + *pEtryPoint = freeWrkrInstance;\ + } /* the following definition is queryEtryPt block that must be added * if an output module supports the transactional interface. @@ -587,6 +705,24 @@ static rsRetVal queryEtryPt(uchar *name, rsRetVal (**pEtryPoint)())\ *pEtryPoint = GetParserName;\ } +/* the following definition is the standard block for queryEtryPt for PARSER + * modules obeying the v2+ config interface. + */ +#define CODEqueryEtryPt_STD_PMOD2_QUERIES \ + CODEqueryEtryPt_STD_MOD_QUERIES \ + else if(!strcmp((char*) name, "parse2")) {\ + *pEtryPoint = parse2;\ + } else if(!strcmp((char*) name, "GetParserName")) {\ + *pEtryPoint = GetParserName;\ + } else if(!strcmp((char*) name, "newParserInst")) {\ + *pEtryPoint = newParserInst;\ + } else if(!strcmp((char*) name, "freeParserInst")) {\ + *pEtryPoint = freeParserInst;\ + } \ + CODEqueryEtryPt_STD_CONF2_CNFNAME_QUERIES + + + /* the following definition is the standard block for queryEtryPt for Strgen * modules. This can be used if no specific handling (e.g. to cover version * differences) is needed. @@ -960,7 +1096,7 @@ static rsRetVal SetShutdownImmdtPtr(instanceData __attribute__((unused)) *pData, } -/* parse() - main entry point of parser modules +/* parse() - main entry point of parser modules (v1 config interface) */ #define BEGINparse \ static rsRetVal parse(msg_t *pMsg)\ @@ -975,10 +1111,32 @@ static rsRetVal parse(msg_t *pMsg)\ } +/* parse2() - main entry point of parser modules (v2+ config interface) + */ +#define BEGINparse2 \ +static rsRetVal parse2(instanceConf_t *const pInst, msg_t *pMsg)\ +{\ + DEFiRet; + +#define CODESTARTparse2 \ + assert(pInst != NULL);\ + assert(pMsg != NULL); + +#define ENDparse2 \ + RETiRet;\ +} + + /* strgen() - main entry point of parser modules + * Note that we do NOT use size_t as this permits us to store the + * values directly into optimized heap structures. + * ppBuf is the buffer pointer + * pLenBuf is the current max size of this buffer + * pStrLen is an output parameter that MUST hold the length + * of the generated string on exit (this is cached) */ #define BEGINstrgen \ -static rsRetVal strgen(msg_t *pMsg, uchar **ppBuf, size_t *pLenBuf) \ +static rsRetVal strgen(msg_t *const pMsg, actWrkrIParams_t *const iparam) \ {\ DEFiRet; diff --git a/runtime/modules.c b/runtime/modules.c index 5660630..6c5583a 100644 --- a/runtime/modules.c +++ b/runtime/modules.c @@ -64,7 +64,6 @@ /* static data */ DEFobjStaticHelpers DEFobjCurrIf(errmsg) -DEFobjCurrIf(parser) DEFobjCurrIf(strgen) static modInfo_t *pLoadedModules = NULL; /* list of currently-loaded modules */ @@ -358,7 +357,7 @@ addModToGlblList(modInfo_t *pThis) rsRetVal readyModForCnf(modInfo_t *pThis, cfgmodules_etry_t **ppNew, cfgmodules_etry_t **ppLast) { - cfgmodules_etry_t *pNew; + cfgmodules_etry_t *pNew = NULL; cfgmodules_etry_t *pLast; DEFiRet; assert(pThis != NULL); @@ -403,6 +402,10 @@ readyModForCnf(modInfo_t *pThis, cfgmodules_etry_t **ppNew, cfgmodules_etry_t ** *ppLast = pLast; *ppNew = pNew; finalize_it: + if(iRet != RS_RET_OK) { + if(pNew != NULL) + free(pNew); + } RETiRet; } @@ -555,7 +558,6 @@ doModInit(rsRetVal (*modInit)(int, int*, rsRetVal(**)(), rsRetVal(*)(), modInfo_ rsRetVal localRet; modInfo_t *pNew = NULL; uchar *pName; - parser_t *pParser; /* used for parser modules */ strgen_t *pStrgen; /* used for strgen modules */ rsRetVal (*GetName)(uchar**); rsRetVal (*modGetType)(eModType_t *pType); @@ -641,7 +643,7 @@ doModInit(rsRetVal (*modInit)(int, int*, rsRetVal(**)(), rsRetVal(*)(), modInfo_ pNew->mod.im.bCanRun = 0; localRet = (*pNew->modQueryEtryPt)((uchar*)"newInpInst", &pNew->mod.im.newInpInst); if(localRet == RS_RET_MODULE_ENTRY_POINT_NOT_FOUND) { - pNew->mod.om.newActInst = NULL; + pNew->mod.im.newInpInst = NULL; } else if(localRet != RS_RET_OK) { ABORT_FINALIZE(localRet); } @@ -649,9 +651,11 @@ doModInit(rsRetVal (*modInit)(int, int*, rsRetVal(**)(), rsRetVal(*)(), modInfo_ case eMOD_OUT: CHKiRet((*pNew->modQueryEtryPt)((uchar*)"freeInstance", &pNew->freeInstance)); CHKiRet((*pNew->modQueryEtryPt)((uchar*)"dbgPrintInstInfo", &pNew->dbgPrintInstInfo)); - CHKiRet((*pNew->modQueryEtryPt)((uchar*)"doAction", &pNew->mod.om.doAction)); CHKiRet((*pNew->modQueryEtryPt)((uchar*)"parseSelectorAct", &pNew->mod.om.parseSelectorAct)); CHKiRet((*pNew->modQueryEtryPt)((uchar*)"tryResume", &pNew->tryResume)); + CHKiRet((*pNew->modQueryEtryPt)((uchar*)"createWrkrInstance", &pNew->mod.om.createWrkrInstance)); + CHKiRet((*pNew->modQueryEtryPt)((uchar*)"freeWrkrInstance", &pNew->mod.om.freeWrkrInstance)); + /* try load optional interfaces */ localRet = (*pNew->modQueryEtryPt)((uchar*)"doHUP", &pNew->doHUP); if(localRet != RS_RET_OK && localRet != RS_RET_MODULE_ENTRY_POINT_NOT_FOUND) @@ -661,11 +665,56 @@ doModInit(rsRetVal (*modInit)(int, int*, rsRetVal(**)(), rsRetVal(*)(), modInfo_ if(localRet != RS_RET_OK && localRet != RS_RET_MODULE_ENTRY_POINT_NOT_FOUND) ABORT_FINALIZE(localRet); + pNew->mod.om.supportsTX = 1; localRet = (*pNew->modQueryEtryPt)((uchar*)"beginTransaction", &pNew->mod.om.beginTransaction); - if(localRet == RS_RET_MODULE_ENTRY_POINT_NOT_FOUND) + if(localRet == RS_RET_MODULE_ENTRY_POINT_NOT_FOUND) { pNew->mod.om.beginTransaction = dummyBeginTransaction; - else if(localRet != RS_RET_OK) + pNew->mod.om.supportsTX = 0; + } else if(localRet != RS_RET_OK) { + ABORT_FINALIZE(localRet); + } + + localRet = (*pNew->modQueryEtryPt)((uchar*)"doAction", + &pNew->mod.om.doAction); + if(localRet == RS_RET_MODULE_ENTRY_POINT_NOT_FOUND) { + pNew->mod.om.doAction = NULL; + } else if(localRet != RS_RET_OK) { ABORT_FINALIZE(localRet); + } + + localRet = (*pNew->modQueryEtryPt)((uchar*)"commitTransaction", + &pNew->mod.om.commitTransaction); + if(localRet == RS_RET_MODULE_ENTRY_POINT_NOT_FOUND) { + pNew->mod.om.commitTransaction = NULL; + } else if(localRet != RS_RET_OK) { + ABORT_FINALIZE(localRet); + } + + if(pNew->mod.om.doAction == NULL && pNew->mod.om.commitTransaction == NULL) { + errmsg.LogError(0, RS_RET_INVLD_OMOD, + "module %s does neither provide doAction() " + "nor commitTransaction() interface - cannot " + "load", name); + ABORT_FINALIZE(RS_RET_INVLD_OMOD); + } + + if(pNew->mod.om.commitTransaction != NULL) { + if(pNew->mod.om.doAction != NULL){ + errmsg.LogError(0, RS_RET_INVLD_OMOD, + "module %s provides both doAction() " + "and commitTransaction() interface, using " + "commitTransaction()", name); + pNew->mod.om.doAction = NULL; + } + if(pNew->mod.om.beginTransaction == NULL){ + errmsg.LogError(0, RS_RET_INVLD_OMOD, + "module %s provides both commitTransaction() " + "but does not provide beginTransaction() - " + "cannot load", name); + ABORT_FINALIZE(RS_RET_INVLD_OMOD); + } + } + localRet = (*pNew->modQueryEtryPt)((uchar*)"endTransaction", &pNew->mod.om.endTransaction); @@ -685,31 +734,25 @@ doModInit(rsRetVal (*modInit)(int, int*, rsRetVal(**)(), rsRetVal(*)(), modInfo_ case eMOD_LIB: break; case eMOD_PARSER: - /* first, we need to obtain the parser object. We could not do that during - * init as that would have caused class bootstrap issues which are not - * absolutely necessary. Note that we can call objUse() multiple times, it - * handles that. - */ - CHKiRet(objUse(parser, CORE_COMPONENT)); - /* here, we create a new parser object */ - CHKiRet((*pNew->modQueryEtryPt)((uchar*)"parse", &pNew->mod.pm.parse)); + localRet = (*pNew->modQueryEtryPt)((uchar*)"parse2", + &pNew->mod.pm.parse2); + if(localRet == RS_RET_OK) { + pNew->mod.pm.parse = NULL; + CHKiRet((*pNew->modQueryEtryPt)((uchar*)"newParserInst", + &pNew->mod.pm.newParserInst)); + CHKiRet((*pNew->modQueryEtryPt)((uchar*)"freeParserInst", + &pNew->mod.pm.freeParserInst)); + } else if(localRet == RS_RET_MODULE_ENTRY_POINT_NOT_FOUND) { + pNew->mod.pm.parse2 = NULL; + pNew->mod.pm.newParserInst = NULL; + pNew->mod.pm.freeParserInst = NULL; + CHKiRet((*pNew->modQueryEtryPt)((uchar*)"parse", &pNew->mod.pm.parse)); + } else { + ABORT_FINALIZE(localRet); + } CHKiRet((*pNew->modQueryEtryPt)((uchar*)"GetParserName", &GetName)); CHKiRet(GetName(&pName)); - CHKiRet(parser.Construct(&pParser)); - - /* check some features */ - localRet = pNew->isCompatibleWithFeature(sFEATUREAutomaticSanitazion); - if(localRet == RS_RET_OK){ - CHKiRet(parser.SetDoSanitazion(pParser, RSTRUE)); - } - localRet = pNew->isCompatibleWithFeature(sFEATUREAutomaticPRIParsing); - if(localRet == RS_RET_OK){ - CHKiRet(parser.SetDoPRIParsing(pParser, RSTRUE)); - } - - CHKiRet(parser.SetName(pParser, pName)); - CHKiRet(parser.SetModPtr(pParser, pNew)); - CHKiRet(parser.ConstructFinalize(pParser)); + CHKiRet(parserConstructViaModAndName(pNew, pName, NULL)); break; case eMOD_STRGEN: /* first, we need to obtain the strgen object. We could not do that during @@ -718,7 +761,6 @@ doModInit(rsRetVal (*modInit)(int, int*, rsRetVal(**)(), rsRetVal(*)(), modInfo_ * handles that. */ CHKiRet(objUse(strgen, CORE_COMPONENT)); - /* here, we create a new parser object */ CHKiRet((*pNew->modQueryEtryPt)((uchar*)"strgen", &pNew->mod.sm.strgen)); CHKiRet((*pNew->modQueryEtryPt)((uchar*)"GetName", &GetName)); CHKiRet(GetName(&pName)); @@ -1083,7 +1125,7 @@ Load(uchar *pModName, sbool bConfLoad, struct nvlst *lst) free(pPathBuf); /* we always alloc enough memory for everything we potentiall need to add */ lenPathBuf = PATHBUF_OVERHEAD; - CHKmalloc(pPathBuf = malloc(sizeof(char)*lenPathBuf)); + CHKmalloc(pPathBuf = malloc(sizeof(uchar)*lenPathBuf)); } *pPathBuf = '\0'; /* we do not need to append the path - its already in the module name */ iPathLen = 0; @@ -1106,7 +1148,7 @@ Load(uchar *pModName, sbool bConfLoad, struct nvlst *lst) free(pPathBuf); /* we always alloc enough memory for everything we potentiall need to add */ lenPathBuf = iPathLen + PATHBUF_OVERHEAD; - CHKmalloc(pPathBuf = malloc(sizeof(char)*lenPathBuf)); + CHKmalloc(pPathBuf = malloc(sizeof(uchar)*lenPathBuf)); } memcpy((char *) pPathBuf, (char *)pModDirCurr, iPathLen); @@ -1130,7 +1172,6 @@ Load(uchar *pModName, sbool bConfLoad, struct nvlst *lst) * algo over time... -- rgerhards, 2008-03-05 */ strncat((char *) pPathBuf, ".so", lenPathBuf - strlen((char*) pPathBuf) - 1); - iPathLen += 3; } /* complete load path constructed, so ... GO! */ @@ -1326,7 +1367,6 @@ BEGINObjClassExit(module, OBJ_IS_LOADABLE_MODULE) /* CHANGE class also in END MA CODESTARTObjClassExit(module) /* release objects we no longer need */ objRelease(errmsg, CORE_COMPONENT); - objRelease(parser, CORE_COMPONENT); free(pModDir); # ifdef DEBUG modUsrPrintAll(); /* debug aid - TODO: integrate with debug.c, at least the settings! */ diff --git a/runtime/modules.h b/runtime/modules.h index 64644be..634d298 100644 --- a/runtime/modules.h +++ b/runtime/modules.h @@ -122,10 +122,6 @@ struct modInfo_s { rsRetVal (*activateCnf)(void*Cnf); /* make provided config the running conf */ rsRetVal (*freeCnf)(void*Cnf); /* end v2 config system specific */ - /* below: create an instance of this module. Most importantly the module - * can allocate instance memory in this call. - */ - rsRetVal (*createInstance)(); union { struct {/* data for input modules */ /* TODO: remove? */rsRetVal (*willRun)(void); /* check if the current config will be able to run*/ @@ -138,20 +134,27 @@ struct modInfo_s { /* below: perform the configured action */ rsRetVal (*beginTransaction)(void*); - rsRetVal (*doAction)(uchar**, unsigned, void*); + rsRetVal (*commitTransaction)(void *const, actWrkrIParams_t *const, const unsigned); + rsRetVal (*doAction)(uchar**, void*); rsRetVal (*endTransaction)(void*); rsRetVal (*parseSelectorAct)(uchar**, void**,omodStringRequest_t**); rsRetVal (*newActInst)(uchar *modName, struct nvlst *lst, void **, omodStringRequest_t **); rsRetVal (*SetShutdownImmdtPtr)(void *pData, void *pPtr); + rsRetVal (*createWrkrInstance)(void*ppWrkrData, void*pData); + rsRetVal (*freeWrkrInstance)(void*pWrkrData); + sbool supportsTX; /* set if the module supports transactions */ } om; struct { /* data for library modules */ char dummy; } lm; struct { /* data for parser modules */ + rsRetVal (*newParserInst)(struct nvlst *lst, void *pinst); + rsRetVal (*freeParserInst)(void *pinst); + rsRetVal (*parse2)(instanceConf_t *const, msg_t*); rsRetVal (*parse)(msg_t*); } pm; struct { /* data for strgen modules */ - rsRetVal (*strgen)(msg_t*, uchar**, size_t *); + rsRetVal (*strgen)(const msg_t*const, actWrkrIParams_t *const iparam); } sm; } mod; void *pModHdlr; /* handler to the dynamic library holding the module */ diff --git a/runtime/msg.c b/runtime/msg.c index a5c5281..d911b8b 100644 --- a/runtime/msg.c +++ b/runtime/msg.c @@ -7,7 +7,7 @@ * of the "old" message code without any modifications. However, it * helps to have things at the right place one we go to the meat of it. * - * Copyright 2007-2013 Rainer Gerhards and Adiscon GmbH. + * Copyright 2007-2014 Rainer Gerhards and Adiscon GmbH. * * This file is part of the rsyslog runtime library. * @@ -41,9 +41,9 @@ #endif #include <netdb.h> #include <libestr.h> -#include <json/json.h> +#include <json.h> /* For struct json_object_iter, should not be necessary in future versions */ -#include <json/json_object_private.h> +#include <json_object_private.h> #if HAVE_MALLOC_H # include <malloc.h> #endif @@ -65,6 +65,13 @@ #include "net.h" #include "var.h" #include "rsconf.h" +#include "parserif.h" + +/* TODO: move the global variable root to the config object - had no time to to it + * right now before vacation -- rgerhards, 2013-07-22 + */ +static pthread_rwlock_t glblVars_rwlock; +struct json_object *global_var_root = NULL; /* static data */ DEFobjStaticHelpers @@ -75,6 +82,8 @@ DEFobjCurrIf(prop) DEFobjCurrIf(net) DEFobjCurrIf(var) +static char *one_digit[10] = { "0", "1", "2", "3", "4", "5", "6", "7", "8", "9" }; + static char *two_digits[100] = { "00", "01", "02", "03", "04", "05", "06", "07", "08", "09", "10", "11", "12", "13", "14", "15", "16", "17", "18", "19", @@ -87,10 +96,36 @@ static char *two_digits[100] = { "80", "81", "82", "83", "84", "85", "86", "87", "88", "89", "90", "91", "92", "93", "94", "95", "96", "97", "98", "99"}; +static char *wdayNames[7] = { "Sun", "Mon", "Tue", "Wed", "Thu", "Fri", "Sat" }; + +/* The following is a table of supported years. This permits us + * to avoid dynamic memory allocation. Note that the time-based + * algos need to be upgraded after the year 2099 in any case. + * Quite honestly, I don't expect that this is a real problem ;) + */ +static char *years[] = { + "1967", "1968", "1969", "1970", "1971", "1972", "1973", "1974", + "1975", "1976", "1977", "1978", "1979", "1980", "1981", "1982", + "1983", "1984", "1985", "1986", "1987", "1988", "1989", "1990", + "1991", "1992", "1993", "1994", "1995", "1996", "1997", "1998", + "1999", "2000", "2001", "2002", "2003", "2004", "2005", "2006", + "2007", "2008", "2009", "2010", "2011", "2012", "2013", "2014", + "2015", "2016", "2017", "2018", "2019", "2020", "2021", "2022", + "2023", "2024", "2025", "2026", "2027", "2028", "2029", "2030", + "2031", "2032", "2033", "2034", "2035", "2036", "2037", "2038", + "2039", "2040", "2041", "2042", "2043", "2044", "2045", "2046", + "2047", "2048", "2049", "2050", "2051", "2052", "2053", "2054", + "2055", "2056", "2057", "2058", "2059", "2060", "2061", "2062", + "2063", "2064", "2065", "2066", "2067", "2068", "2069", "2070", + "2071", "2072", "2073", "2074", "2075", "2076", "2077", "2078", + "2079", "2080", "2081", "2082", "2083", "2084", "2085", "2086", + "2087", "2088", "2089", "2090", "2091", "2092", "2093", "2094", + "2095", "2096", "2097", "2098", "2099" }; + static struct { uchar *pszName; short lenName; -} syslog_pri_names[192] = { +} syslog_pri_names[200] = { { UCHAR_CONSTANT("0"), 3}, { UCHAR_CONSTANT("1"), 3}, { UCHAR_CONSTANT("2"), 3}, @@ -282,22 +317,30 @@ static struct { { UCHAR_CONSTANT("188"), 5}, { UCHAR_CONSTANT("189"), 5}, { UCHAR_CONSTANT("190"), 5}, - { UCHAR_CONSTANT("191"), 5} + { UCHAR_CONSTANT("191"), 5}, + { UCHAR_CONSTANT("192"), 5}, + { UCHAR_CONSTANT("193"), 5}, + { UCHAR_CONSTANT("194"), 5}, + { UCHAR_CONSTANT("195"), 5}, + { UCHAR_CONSTANT("196"), 5}, + { UCHAR_CONSTANT("197"), 5}, + { UCHAR_CONSTANT("198"), 5}, + { UCHAR_CONSTANT("199"), 5} }; static char hexdigit[16] = {'0', '1', '2', '3', '4', '5', '6', '7', '8', '9', 'A', 'B', 'C', 'D', 'E', 'F' }; /*syslog facility names (as of RFC5424) */ -static char *syslog_fac_names[24] = { "kern", "user", "mail", "daemon", "auth", "syslog", "lpr", +static char *syslog_fac_names[LOG_NFACILITIES] = { "kern", "user", "mail", "daemon", "auth", "syslog", "lpr", "news", "uucp", "cron", "authpriv", "ftp", "ntp", "audit", "alert", "clock", "local0", "local1", "local2", "local3", - "local4", "local5", "local6", "local7" }; + "local4", "local5", "local6", "local7", "invld" }; /* length of the facility names string (for optimizatiions) */ -static short len_syslog_fac_names[24] = { 4, 4, 4, 6, 4, 6, 3, +static short len_syslog_fac_names[LOG_NFACILITIES] = { 4, 4, 4, 6, 4, 6, 3, 4, 4, 4, 8, 3, 3, 5, 5, 5, 6, 6, 6, 6, - 6, 6, 6, 6 }; + 6, 6, 6, 6, 5 }; /* table of severity names (in numerical order)*/ static char *syslog_severity_names[8] = { "emerg", "alert", "crit", "err", "warning", "notice", "info", "debug" }; @@ -307,8 +350,8 @@ static short len_syslog_severity_names[8] = { 5, 5, 4, 3, 7, 6, 4, 5 }; * and facility values to a numerical string... -- rgerhars, 2009-06-17 */ -static char *syslog_number_names[24] = { "0", "1", "2", "3", "4", "5", "6", "7", "8", "9", "10", "11", "12", "13", "14", - "15", "16", "17", "18", "19", "20", "21", "22", "23" }; +static char *syslog_number_names[LOG_NFACILITIES] = { "0", "1", "2", "3", "4", "5", "6", "7", "8", "9", "10", "11", "12", "13", "14", + "15", "16", "17", "18", "19", "20", "21", "22", "23", "24" }; /* global variables */ #if defined(HAVE_MALLOC_TRIM) && !defined(HAVE_ATOMIC_BUILTINS) @@ -316,8 +359,8 @@ static pthread_mutex_t mutTrimCtr; /* mutex to handle malloc trim */ #endif /* some forward declarations */ -static int getAPPNAMELen(msg_t *pM, sbool bLockMutex); -static rsRetVal jsonPathFindParent(msg_t *pM, uchar *name, uchar *leaf, struct json_object **parent, int bCreate); +static int getAPPNAMELen(msg_t * const pM, sbool bLockMutex); +static rsRetVal jsonPathFindParent(struct json_object *jroot, uchar *name, uchar *leaf, struct json_object **parent, int bCreate); static uchar * jsonPathGetLeaf(uchar *name, int lenName); static struct json_object *jsonDeepCopy(struct json_object *src); @@ -372,23 +415,16 @@ void MsgSetRcvFromWithoutAddRef(msg_t *pThis, prop_t *new) * If ruleset cannot be found, no update is done. */ static void -MsgSetRulesetByName(msg_t *pMsg, cstr_t *rulesetName) +MsgSetRulesetByName(msg_t * const pMsg, cstr_t *rulesetName) { rulesetGetRuleset(runConf, &(pMsg->pRuleset), rsCStrGetSzStrNoNULL(rulesetName)); } - -static inline int getProtocolVersion(msg_t *pM) -{ - return(pM->iProtocolVersion); -} - - /* do a DNS reverse resolution, if not already done, reflect status * rgerhards, 2009-11-16 */ static inline rsRetVal -resolveDNS(msg_t *pMsg) { +resolveDNS(msg_t * const pMsg) { rsRetVal localRet; prop_t *propFromHost = NULL; prop_t *ip; @@ -419,7 +455,7 @@ finalize_it: static inline void -getInputName(msg_t *pM, uchar **ppsz, int *plen) +getInputName(msg_t * const pM, uchar **ppsz, int *plen) { BEGINfunc if(pM == NULL || pM->pInputName == NULL) { @@ -433,7 +469,7 @@ getInputName(msg_t *pM, uchar **ppsz, int *plen) static inline uchar* -getRcvFromIP(msg_t *pM) +getRcvFromIP(msg_t * const pM) { uchar *psz; int len; @@ -452,14 +488,12 @@ getRcvFromIP(msg_t *pM) } -/* map a property name (C string) to a property ID */ +/* map a property name (string) to a property ID */ rsRetVal -propNameStrToID(uchar *pName, propid_t *pPropID) +propNameToID(uchar *pName, propid_t *pPropID) { DEFiRet; - assert(pName != NULL); - /* sometimes there are aliases to the original MonitoWare * property names. These come after || in the ifs below. */ if(!strcmp((char*) pName, "msg")) { @@ -507,6 +541,8 @@ propNameStrToID(uchar *pName, propid_t *pPropID) *pPropID = PROP_PROCID; } else if(!strcmp((char*) pName, "msgid")) { *pPropID = PROP_MSGID; + } else if(!strcmp((char*) pName, "jsonmesg")) { + *pPropID = PROP_JSONMESG; } else if(!strcmp((char*) pName, "parsesuccess")) { *pPropID = PROP_PARSESUCCESS; #ifdef USE_LIBUUID @@ -534,13 +570,18 @@ propNameStrToID(uchar *pName, propid_t *pPropID) *pPropID = PROP_SYS_MYHOSTNAME; } else if(!strcmp((char*) pName, "$!all-json")) { *pPropID = PROP_CEE_ALL_JSON; - } else if(!strncmp((char*) pName, "$!", 2)) { - *pPropID = PROP_CEE; } else if(!strcmp((char*) pName, "$bom")) { *pPropID = PROP_SYS_BOM; } else if(!strcmp((char*) pName, "$uptime")) { *pPropID = PROP_SYS_UPTIME; + } else if(!strncmp((char*) pName, "$!", 2) || pName[0] == '!') { + *pPropID = PROP_CEE; + } else if(!strncmp((char*) pName, "$.", 2) || pName[0] == '.') { + *pPropID = PROP_LOCAL_VAR; + } else if(!strncmp((char*) pName, "$/", 2) || pName[0] == '/') { + *pPropID = PROP_GLOBAL_VAR; } else { + DBGPRINTF("PROP_INVALID for name '%s'\n", pName); *pPropID = PROP_INVALID; iRet = RS_RET_VAR_NOT_FOUND; } @@ -549,21 +590,6 @@ propNameStrToID(uchar *pName, propid_t *pPropID) } -/* map a property name (string) to a property ID */ -rsRetVal -propNameToID(cstr_t *pCSPropName, propid_t *pPropID) -{ - uchar *pName; - DEFiRet; - - assert(pCSPropName != NULL); - assert(pPropID != NULL); - pName = rsCStrGetSzStrNoNULL(pCSPropName); - iRet = propNameStrToID(pName, pPropID); - RETiRet; -} - - /* map a property ID to a name string (useful for displaying) */ uchar *propIDToName(propid_t propID) { @@ -612,6 +638,8 @@ uchar *propIDToName(propid_t propID) return UCHAR_CONSTANT("procid"); case PROP_MSGID: return UCHAR_CONSTANT("msgid"); + case PROP_JSONMESG: + return UCHAR_CONSTANT("jsonmesg"); case PROP_PARSESUCCESS: return UCHAR_CONSTANT("parsesuccess"); case PROP_SYS_NOW: @@ -634,6 +662,8 @@ uchar *propIDToName(propid_t propID) return UCHAR_CONSTANT("$MYHOSTNAME"); case PROP_CEE: return UCHAR_CONSTANT("*CEE-based property*"); + case PROP_LOCAL_VAR: + return UCHAR_CONSTANT("*LOCAL_VARIABLE*"); case PROP_CEE_ALL_JSON: return UCHAR_CONSTANT("$!all-json"); case PROP_SYS_BOM: @@ -678,8 +708,8 @@ static inline rsRetVal msgBaseConstruct(msg_t **ppThis) pM->flowCtlType = 0; pM->bParseSuccess = 0; pM->iRefCount = 1; - pM->iSeverity = -1; - pM->iFacility = -1; + pM->iSeverity = LOG_DEBUG; + pM->iFacility = LOG_INVLD; pM->iLenPROGNAME = -1; pM->offAfterPRI = 0; pM->offMSG = -1; @@ -699,7 +729,7 @@ static inline rsRetVal msgBaseConstruct(msg_t **ppThis) pM->pszTIMESTAMP3339 = NULL; pM->pszTIMESTAMP_MySQL = NULL; pM->pszTIMESTAMP_PgSQL = NULL; - pM->pCSStrucData = NULL; + pM->pszStrucData = NULL; pM->pCSAPPNAME = NULL; pM->pCSPROCID = NULL; pM->pCSMSGID = NULL; @@ -708,6 +738,8 @@ static inline rsRetVal msgBaseConstruct(msg_t **ppThis) pM->rcvFrom.pRcvFrom = NULL; pM->pRuleset = NULL; pM->json = NULL; + pM->localvars = NULL; + pM->dfltTZ[0] = '\0'; memset(&pM->tRcvdAt, 0, sizeof(pM->tRcvdAt)); memset(&pM->tTIMESTAMP, 0, sizeof(pM->tTIMESTAMP)); pM->TAG.pszTAG = NULL; @@ -841,10 +873,9 @@ CODESTARTobjDestruct(msg) free(pThis->pszRcvdAt_PgSQL); free(pThis->pszTIMESTAMP_MySQL); free(pThis->pszTIMESTAMP_PgSQL); + free(pThis->pszStrucData); if(pThis->iLenPROGNAME >= CONF_PROGNAME_BUFSIZE) free(pThis->PROGNAME.ptr); - if(pThis->pCSStrucData != NULL) - rsCStrDestruct(&pThis->pCSStrucData); if(pThis->pCSAPPNAME != NULL) rsCStrDestruct(&pThis->pCSAPPNAME); if(pThis->pCSPROCID != NULL) @@ -853,6 +884,8 @@ CODESTARTobjDestruct(msg) rsCStrDestruct(&pThis->pCSMSGID); if(pThis->json != NULL) json_object_put(pThis->json); + if(pThis->localvars != NULL) + json_object_put(pThis->localvars); if(pThis->pszUUID != NULL) free(pThis->pszUUID); # ifndef HAVE_ATOMIC_BUILTINS @@ -994,14 +1027,21 @@ msg_t* MsgDup(msg_t* pOld) tmpCOPYSZ(HOSTNAME); } } + if(pOld->pszStrucData == NULL) { + pNew->pszStrucData = NULL; + } else { + pNew->pszStrucData = (uchar*)strdup((char*)pOld->pszStrucData); + pNew->lenStrucData = pOld->lenStrucData; + } - tmpCOPYCSTR(StrucData); tmpCOPYCSTR(APPNAME); tmpCOPYCSTR(PROCID); tmpCOPYCSTR(MSGID); if(pOld->json != NULL) pNew->json = jsonDeepCopy(pOld->json); + if(pOld->localvars != NULL) + pNew->localvars = jsonDeepCopy(pOld->localvars); /* we do not copy all other cache properties, as we do not even know * if they are needed once again. So we let them re-create if needed. @@ -1056,12 +1096,17 @@ static rsRetVal MsgSerialize(msg_t *pThis, strm_t *pStrm) CHKiRet(obj.SerializeProp(pStrm, UCHAR_CONSTANT("pszRcvFrom"), PROPTYPE_PSZ, (void*) psz)); psz = getRcvFromIP(pThis); CHKiRet(obj.SerializeProp(pStrm, UCHAR_CONSTANT("pszRcvFromIP"), PROPTYPE_PSZ, (void*) psz)); + psz = pThis->pszStrucData; + CHKiRet(obj.SerializeProp(pStrm, UCHAR_CONSTANT("pszRcvStrucData"), PROPTYPE_PSZ, (void*) psz)); if(pThis->json != NULL) { psz = (uchar*) json_object_get_string(pThis->json); CHKiRet(obj.SerializeProp(pStrm, UCHAR_CONSTANT("json"), PROPTYPE_PSZ, (void*) psz)); } + if(pThis->localvars != NULL) { + psz = (uchar*) json_object_get_string(pThis->localvars); + CHKiRet(obj.SerializeProp(pStrm, UCHAR_CONSTANT("localvars"), PROPTYPE_PSZ, (void*) psz)); + } - objSerializePTR(pStrm, pCSStrucData, CSTR); objSerializePTR(pStrm, pCSAPPNAME, CSTR); objSerializePTR(pStrm, pCSPROCID, CSTR); objSerializePTR(pStrm, pCSMSGID, CSTR); @@ -1107,13 +1152,12 @@ reinitVar(var_t *pVar) */ #define isProp(name) !rsCStrSzStrCmp(pVar->pcsName, (uchar*) name, sizeof(name) - 1) rsRetVal -MsgDeserialize(msg_t *pMsg, strm_t *pStrm) +MsgDeserialize(msg_t * const pMsg, strm_t *pStrm) { prop_t *myProp; prop_t *propRcvFrom = NULL; prop_t *propRcvFromIP = NULL; struct json_tokener *tokener; - struct json_object *json; var_t *pVar = NULL; DEFiRet; @@ -1197,12 +1241,21 @@ MsgDeserialize(msg_t *pMsg, strm_t *pStrm) } if(isProp("json")) { tokener = json_tokener_new(); - json = json_tokener_parse_ex(tokener, (char*)rsCStrGetSzStrNoNULL(pVar->val.pStr), + pMsg->json = json_tokener_parse_ex(tokener, (char*)rsCStrGetSzStrNoNULL(pVar->val.pStr), cstrLen(pVar->val.pStr)); + json_tokener_free(tokener); reinitVar(pVar); CHKiRet(objDeserializeProperty(pVar, pStrm)); } - if(isProp("pCSStrucData")) { + if(isProp("localvars")) { + tokener = json_tokener_new(); + pMsg->localvars = json_tokener_parse_ex(tokener, (char*)rsCStrGetSzStrNoNULL(pVar->val.pStr), + cstrLen(pVar->val.pStr)); + json_tokener_free(tokener); + reinitVar(pVar); + CHKiRet(objDeserializeProperty(pVar, pStrm)); + } + if(isProp("pszStrucData")) { MsgSetStructuredData(pMsg, (char*) rsCStrGetSzStrNoNULL(pVar->val.pStr)); reinitVar(pVar); CHKiRet(objDeserializeProperty(pVar, pStrm)); @@ -1255,7 +1308,7 @@ finalize_it: * * pSecondMsgPointer = MsgAddRef(pOrgMsgPointer); */ -msg_t *MsgAddRef(msg_t *pM) +msg_t *MsgAddRef(msg_t * const pM) { assert(pM != NULL); # ifdef HAVE_ATOMIC_BUILTINS @@ -1281,7 +1334,7 @@ msg_t *MsgAddRef(msg_t *pM) * rgerhards, 2005-11-24 * THIS MUST be called with the message lock locked. */ -static rsRetVal aquirePROCIDFromTAG(msg_t *pM) +static rsRetVal aquirePROCIDFromTAG(msg_t * const pM) { register int i; uchar *pszTag; @@ -1292,7 +1345,7 @@ static rsRetVal aquirePROCIDFromTAG(msg_t *pM) if(pM->pCSPROCID != NULL) return RS_RET_OK; /* we are already done ;) */ - if(getProtocolVersion(pM) != 0) + if(msgGetProtocolVersion(pM) != 0) return RS_RET_OK; /* we can only emulate if we have legacy format */ pszTag = (uchar*) ((pM->iLenTAG < CONF_TAG_BUFSIZE) ? pM->TAG.szBuf : pM->TAG.pszTAG); @@ -1346,7 +1399,7 @@ finalize_it: * rgerhards, 2005-10-19 */ static inline rsRetVal -aquireProgramName(msg_t *pM) +aquireProgramName(msg_t * const pM) { int i; uchar *pszTag, *pszProgName; @@ -1376,7 +1429,7 @@ finalize_it: /* Access methods - dumb & easy, not a comment for each ;) */ -void setProtocolVersion(msg_t *pM, int iNewVersion) +void setProtocolVersion(msg_t * const pM, int iNewVersion) { assert(pM != NULL); if(iNewVersion != 0 && iNewVersion != 1) { @@ -1387,7 +1440,7 @@ void setProtocolVersion(msg_t *pM, int iNewVersion) } /* note: string is taken from constant pool, do NOT free */ -char *getProtocolVersionString(msg_t *pM) +char *getProtocolVersionString(msg_t * const pM) { assert(pM != NULL); return(pM->iProtocolVersion ? "1" : "0"); @@ -1397,7 +1450,7 @@ char *getProtocolVersionString(msg_t *pM) /* note: libuuid seems not to be thread-safe, so we need * to get some safeguards in place. */ -static void msgSetUUID(msg_t *pM) +static void msgSetUUID(msg_t * const pM) { size_t lenRes = sizeof(uuid_t) * 2 + 1; char hex_char [] = "0123456789ABCDEF"; @@ -1405,7 +1458,7 @@ static void msgSetUUID(msg_t *pM) uuid_t uuid; static pthread_mutex_t mutUUID = PTHREAD_MUTEX_INITIALIZER; - dbgprintf("[MsgSetUUID] START\n"); + dbgprintf("[MsgSetUUID] START, lenRes %llu\n", (long long unsigned) lenRes); assert(pM != NULL); if((pM->pszUUID = (uchar*) MALLOC(lenRes)) == NULL) { @@ -1419,13 +1472,13 @@ static void msgSetUUID(msg_t *pM) pM->pszUUID[byte_nbr * 2 + 1] = hex_char[uuid [byte_nbr] & 15]; } + pM->pszUUID[lenRes-1] = '\0'; dbgprintf("[MsgSetUUID] UUID : %s LEN: %d \n", pM->pszUUID, (int)lenRes); - pM->pszUUID[lenRes] = '\0'; } dbgprintf("[MsgSetUUID] END\n"); } -void getUUID(msg_t *pM, uchar **pBuf, int *piLen) +void getUUID(msg_t * const pM, uchar **pBuf, int *piLen) { dbgprintf("[getUUID] START\n"); if(pM == NULL) { @@ -1451,7 +1504,7 @@ void getUUID(msg_t *pM, uchar **pBuf, int *piLen) #endif void -getRawMsg(msg_t *pM, uchar **pBuf, int *piLen) +getRawMsg(msg_t * const pM, uchar **pBuf, int *piLen) { if(pM == NULL) { *pBuf= UCHAR_CONSTANT(""); @@ -1471,17 +1524,17 @@ getRawMsg(msg_t *pM, uchar **pBuf, int *piLen) /* note: setMSGLen() is only for friends who really know what they * do. Setting an invalid length can be desasterous! */ -void setMSGLen(msg_t *pM, int lenMsg) +void setMSGLen(msg_t * const pM, int lenMsg) { pM->iLenMSG = lenMsg; } -int getMSGLen(msg_t *pM) +int getMSGLen(msg_t * const pM) { return((pM == NULL) ? 0 : pM->iLenMSG); } -uchar *getMSG(msg_t *pM) +uchar *getMSG(msg_t * const pM) { uchar *ret; if(pM == NULL) @@ -1497,16 +1550,19 @@ uchar *getMSG(msg_t *pM) /* Get PRI value as integer */ -static int getPRIi(msg_t *pM) +static int getPRIi(msg_t * const pM) { - return (pM->iFacility << 3) + (pM->iSeverity); + int pri = (pM->iFacility << 3) + (pM->iSeverity); + if(pri > 191) + pri = LOG_PRI_INVLD; + return pri; } /* Get PRI value in text form */ char * -getPRI(msg_t *pM) +getPRI(msg_t * const pM) { /* PRI is a number in the range 0..191. Thus, we use a simple lookup table to obtain the * string value. It looks a bit clumpsy here in code ;) @@ -1522,7 +1578,7 @@ getPRI(msg_t *pM) char * -getTimeReported(msg_t *pM, enum tplFormatTypes eFmt) +getTimeReported(msg_t * const pM, enum tplFormatTypes eFmt) { BEGINfunc if(pM == NULL) @@ -1587,12 +1643,37 @@ getTimeReported(msg_t *pM, enum tplFormatTypes eFmt) MsgUnlock(pM); } return(pM->pszTIMESTAMP_SecFrac); + case tplFmtWDayName: + return wdayNames[getWeekdayNbr(&pM->tTIMESTAMP)]; + case tplFmtWDay: + return one_digit[getWeekdayNbr(&pM->tTIMESTAMP)]; + case tplFmtMonth: + return two_digits[(int)pM->tTIMESTAMP.month]; + case tplFmtYear: + if(pM->tTIMESTAMP.year >= 1967 && pM->tTIMESTAMP.year <= 2099) + return years[pM->tTIMESTAMP.year - 1967]; + else + return "YEAR OUT OF RANGE(1967-2099)"; + case tplFmtDay: + return two_digits[(int)pM->tTIMESTAMP.day]; + case tplFmtHour: + return two_digits[(int)pM->tTIMESTAMP.hour]; + case tplFmtMinute: + return two_digits[(int)pM->tTIMESTAMP.minute]; + case tplFmtSecond: + return two_digits[(int)pM->tTIMESTAMP.second]; + case tplFmtTZOffsHour: + return two_digits[(int)pM->tTIMESTAMP.OffsetHour]; + case tplFmtTZOffsMin: + return two_digits[(int)pM->tTIMESTAMP.OffsetMinute]; + case tplFmtTZOffsDirection: + return (pM->tTIMESTAMP.OffsetMode == '+')? "+" : "-"; } ENDfunc return "INVALID eFmt OPTION!"; } -static inline char *getTimeGenerated(msg_t *pM, enum tplFormatTypes eFmt) +static char *getTimeGenerated(msg_t * const pM, enum tplFormatTypes eFmt) { BEGINfunc if(pM == NULL) @@ -1673,13 +1754,38 @@ static inline char *getTimeGenerated(msg_t *pM, enum tplFormatTypes eFmt) MsgUnlock(pM); } return(pM->pszRcvdAt_SecFrac); + case tplFmtWDayName: + return wdayNames[getWeekdayNbr(&pM->tRcvdAt)]; + case tplFmtWDay: + return one_digit[getWeekdayNbr(&pM->tRcvdAt)]; + case tplFmtMonth: + return two_digits[(int)pM->tRcvdAt.month]; + case tplFmtYear: + if(pM->tRcvdAt.year >= 1967 && pM->tRcvdAt.year <= 2099) + return years[pM->tRcvdAt.year - 1967]; + else + return "YEAR OUT OF RANGE(1967-2099)"; + case tplFmtDay: + return two_digits[(int)pM->tRcvdAt.day]; + case tplFmtHour: + return two_digits[(int)pM->tRcvdAt.hour]; + case tplFmtMinute: + return two_digits[(int)pM->tRcvdAt.minute]; + case tplFmtSecond: + return two_digits[(int)pM->tRcvdAt.second]; + case tplFmtTZOffsHour: + return two_digits[(int)pM->tRcvdAt.OffsetHour]; + case tplFmtTZOffsMin: + return two_digits[(int)pM->tRcvdAt.OffsetMinute]; + case tplFmtTZOffsDirection: + return (pM->tRcvdAt.OffsetMode == '+')? "+" : "-"; } ENDfunc return "INVALID eFmt OPTION!"; } -static inline char *getSeverity(msg_t *pM) +static inline char *getSeverity(msg_t * const pM) { char *name = NULL; @@ -1696,7 +1802,7 @@ static inline char *getSeverity(msg_t *pM) } -static inline char *getSeverityStr(msg_t *pM) +static inline char *getSeverityStr(msg_t * const pM) { char *name = NULL; @@ -1712,7 +1818,7 @@ static inline char *getSeverityStr(msg_t *pM) return name; } -static inline char *getFacility(msg_t *pM) +static inline char *getFacility(msg_t * const pM) { char *name = NULL; @@ -1728,7 +1834,7 @@ static inline char *getFacility(msg_t *pM) return name; } -static inline char *getFacilityStr(msg_t *pM) +static inline char *getFacilityStr(msg_t * const pM) { char *name = NULL; @@ -1752,7 +1858,7 @@ static inline char *getFacilityStr(msg_t *pM) * rgerhards, 2008-03-14 */ rsRetVal -MsgSetFlowControlType(msg_t *pMsg, flowControl_t eFlowCtl) +MsgSetFlowControlType(msg_t * const pMsg, flowControl_t eFlowCtl) { DEFiRet; assert(pMsg != NULL); @@ -1767,7 +1873,7 @@ MsgSetFlowControlType(msg_t *pMsg, flowControl_t eFlowCtl) * rgerhards, 2009-06-16 */ rsRetVal -MsgSetAfterPRIOffs(msg_t *pMsg, short offs) +MsgSetAfterPRIOffs(msg_t * const pMsg, short offs) { assert(pMsg != NULL); pMsg->offAfterPRI = offs; @@ -1781,7 +1887,7 @@ MsgSetAfterPRIOffs(msg_t *pMsg, short offs) * which already obtained the lock. So in general, this function here must * only be called when it it safe to do so without it aquiring a lock. */ -rsRetVal MsgSetAPPNAME(msg_t *pMsg, char* pszAPPNAME) +rsRetVal MsgSetAPPNAME(msg_t *__restrict__ const pMsg, const char* pszAPPNAME) { DEFiRet; assert(pMsg != NULL); @@ -1799,7 +1905,7 @@ finalize_it: /* rgerhards 2004-11-24: set PROCID in msg object */ -rsRetVal MsgSetPROCID(msg_t *pMsg, char* pszPROCID) +rsRetVal MsgSetPROCID(msg_t *__restrict__ const pMsg, const char* pszPROCID) { DEFiRet; ISOBJ_TYPE_assert(pMsg, msg); @@ -1820,7 +1926,7 @@ finalize_it: * This must be called WITHOUT the message lock being held. * rgerhards, 2009-06-26 */ -static inline void preparePROCID(msg_t *pM, sbool bLockMutex) +static inline void preparePROCID(msg_t * const pM, sbool bLockMutex) { if(pM->pCSPROCID == NULL) { if(bLockMutex == LOCK_MUTEX) @@ -1848,7 +1954,7 @@ static inline int getPROCIDLen(msg_t *pM, sbool bLockMutex) /* rgerhards, 2005-11-24 */ -char *getPROCID(msg_t *pM, sbool bLockMutex) +char *getPROCID(msg_t * const pM, sbool bLockMutex) { uchar *pszRet; @@ -1868,7 +1974,7 @@ char *getPROCID(msg_t *pM, sbool bLockMutex) /* rgerhards 2004-11-24: set MSGID in msg object */ -rsRetVal MsgSetMSGID(msg_t *pMsg, char* pszMSGID) +rsRetVal MsgSetMSGID(msg_t * const pMsg, const char* pszMSGID) { DEFiRet; ISOBJ_TYPE_assert(pMsg, msg); @@ -1887,7 +1993,7 @@ finalize_it: /* Return state of last parser. If it had success, "OK" is returned, else * "FAIL". All from the constant pool. */ -static inline char *getParseSuccess(msg_t *pM) +static inline char *getParseSuccess(msg_t * const pM) { return (pM->bParseSuccess) ? "OK" : "FAIL"; } @@ -1895,7 +2001,7 @@ static inline char *getParseSuccess(msg_t *pM) /* al, 2011-07-26: LockMsg to avoid race conditions */ -static inline char *getMSGID(msg_t *pM) +static inline char *getMSGID(msg_t * const pM) { if (pM->pCSMSGID == NULL) { return "-"; @@ -1910,15 +2016,104 @@ static inline char *getMSGID(msg_t *pM) /* rgerhards 2012-03-15: set parser success (an integer, acutally bool) */ -void MsgSetParseSuccess(msg_t *pMsg, int bSuccess) +void MsgSetParseSuccess(msg_t * const pMsg, int bSuccess) { assert(pMsg != NULL); pMsg->bParseSuccess = bSuccess; } + +/* return full message as a json string */ +const uchar* +msgGetJSONMESG(msg_t *__restrict__ const pMsg) +{ + struct json_object *json; + struct json_object *jval; + uchar *pRes; /* result pointer */ + rs_size_t bufLen = -1; /* length of string or -1, if not known */ + + json = json_object_new_object(); + + jval = json_object_new_string((char*)getMSG(pMsg)); + json_object_object_add(json, "msg", jval); + + getRawMsg(pMsg, &pRes, &bufLen); + jval = json_object_new_string((char*)pRes); + json_object_object_add(json, "rawmsg", jval); + + pRes = (uchar*)getTimeReported(pMsg, tplFmtRFC3339Date); + jval = json_object_new_string((char*)pRes); + json_object_object_add(json, "timereported", jval); + + jval = json_object_new_string(getHOSTNAME(pMsg)); + json_object_object_add(json, "hostname", jval); + + getTAG(pMsg, &pRes, &bufLen); + jval = json_object_new_string((char*)pRes); + json_object_object_add(json, "syslogtag", jval); + + getInputName(pMsg, &pRes, &bufLen); + jval = json_object_new_string((char*)pRes); + json_object_object_add(json, "inputname", jval); + + jval = json_object_new_string((char*)getRcvFrom(pMsg)); + json_object_object_add(json, "fromhost", jval); + + jval = json_object_new_string((char*)getRcvFromIP(pMsg)); + json_object_object_add(json, "fromhost-ip", jval); + + jval = json_object_new_string(getPRI(pMsg)); + json_object_object_add(json, "pri", jval); + + jval = json_object_new_string(getFacility(pMsg)); + json_object_object_add(json, "syslogfacility", jval); + + jval = json_object_new_string(getSeverity(pMsg)); + json_object_object_add(json, "syslogseverity", jval); + + pRes = (uchar*)getTimeGenerated(pMsg, tplFmtRFC3339Date); + jval = json_object_new_string((char*)pRes); + json_object_object_add(json, "timegenerated", jval); + + jval = json_object_new_string((char*)getProgramName(pMsg, LOCK_MUTEX)); + json_object_object_add(json, "programname", jval); + + jval = json_object_new_string(getProtocolVersionString(pMsg)); + json_object_object_add(json, "protocol-version", jval); + + MsgGetStructuredData(pMsg, &pRes, &bufLen); + jval = json_object_new_string((char*)pRes); + json_object_object_add(json, "structured-data", jval); + + jval = json_object_new_string(getAPPNAME(pMsg, LOCK_MUTEX)); + json_object_object_add(json, "app-name", jval); + + jval = json_object_new_string(getPROCID(pMsg, LOCK_MUTEX)); + json_object_object_add(json, "procid", jval); + + jval = json_object_new_string(getMSGID(pMsg)); + json_object_object_add(json, "msgid", jval); + +#ifdef USE_LIBUUID + if(pMsg->pszUUID == NULL) { + jval = NULL; + } else { + getUUID(pMsg, &pRes, &bufLen); + jval = json_object_new_string((char*)pRes); + } + json_object_object_add(json, "uuid", jval); +#endif + + json_object_object_add(json, "$!", pMsg->json); + + pRes = (uchar*) strdup(json_object_get_string(json)); + json_object_put(json); + return pRes; +} + /* rgerhards 2009-06-12: set associated ruleset */ -void MsgSetRuleset(msg_t *pMsg, ruleset_t *pRuleset) +void MsgSetRuleset(msg_t * const pMsg, ruleset_t *pRuleset) { assert(pMsg != NULL); pMsg->pRuleset = pRuleset; @@ -1928,7 +2123,7 @@ void MsgSetRuleset(msg_t *pMsg, ruleset_t *pRuleset) /* set TAG in msg object * (rewritten 2009-06-18 rgerhards) */ -void MsgSetTAG(msg_t *pMsg, uchar* pszBuf, size_t lenBuf) +void MsgSetTAG(msg_t *__restrict__ const pMsg, const uchar* pszBuf, const size_t lenBuf) { uchar *pBuf; assert(pMsg != NULL); @@ -1961,7 +2156,7 @@ void MsgSetTAG(msg_t *pMsg, uchar* pszBuf, size_t lenBuf) * if there is a TAG and, if not, if it can emulate it. * rgerhards, 2005-11-24 */ -static inline void tryEmulateTAG(msg_t *pM, sbool bLockMutex) +static inline void tryEmulateTAG(msg_t * const pM, sbool bLockMutex) { size_t lenTAG; uchar bufTAG[CONF_TAG_MAXSIZE]; @@ -1975,7 +2170,7 @@ static inline void tryEmulateTAG(msg_t *pM, sbool bLockMutex) return; /* done, no need to emulate */ } - if(getProtocolVersion(pM) == 1) { + if(msgGetProtocolVersion(pM) == 1) { if(!strcmp(getPROCID(pM, MUTEX_ALREADY_LOCKED), "-")) { /* no process ID, use APP-NAME only */ MsgSetTAG(pM, (uchar*) getAPPNAME(pM, MUTEX_ALREADY_LOCKED), getAPPNAMELen(pM, MUTEX_ALREADY_LOCKED)); @@ -1993,7 +2188,7 @@ static inline void tryEmulateTAG(msg_t *pM, sbool bLockMutex) void -getTAG(msg_t *pM, uchar **ppBuf, int *piLen) +getTAG(msg_t * const pM, uchar **ppBuf, int *piLen) { if(pM == NULL) { *ppBuf = UCHAR_CONSTANT(""); @@ -2012,7 +2207,7 @@ getTAG(msg_t *pM, uchar **ppBuf, int *piLen) } -int getHOSTNAMELen(msg_t *pM) +int getHOSTNAMELen(msg_t * const pM) { if(pM == NULL) return 0; @@ -2028,7 +2223,7 @@ int getHOSTNAMELen(msg_t *pM) } -char *getHOSTNAME(msg_t *pM) +char *getHOSTNAME(msg_t * const pM) { if(pM == NULL) return ""; @@ -2049,7 +2244,7 @@ char *getHOSTNAME(msg_t *pM) } -uchar *getRcvFrom(msg_t *pM) +uchar *getRcvFrom(msg_t * const pM) { uchar *psz; int len; @@ -2071,52 +2266,37 @@ uchar *getRcvFrom(msg_t *pM) /* rgerhards 2004-11-24: set STRUCTURED DATA in msg object */ -rsRetVal MsgSetStructuredData(msg_t *pMsg, char* pszStrucData) +rsRetVal MsgSetStructuredData(msg_t * const pMsg, const char* pszStrucData) { DEFiRet; ISOBJ_TYPE_assert(pMsg, msg); - if(pMsg->pCSStrucData == NULL) { - /* we need to obtain the object first */ - CHKiRet(rsCStrConstruct(&pMsg->pCSStrucData)); - } - /* if we reach this point, we have the object */ - iRet = rsCStrSetSzStr(pMsg->pCSStrucData, (uchar*) pszStrucData); - + free(pMsg->pszStrucData); + CHKmalloc(pMsg->pszStrucData = (uchar*)strdup(pszStrucData)); + pMsg->lenStrucData = strlen(pszStrucData); finalize_it: RETiRet; } -/* get the length of the "STRUCTURED-DATA" sz string - * rgerhards, 2005-11-24 - */ -#if 0 /* This method is currently not called, be we like to preserve it */ -static int getStructuredDataLen(msg_t *pM) -{ - return (pM->pCSStrucData == NULL) ? 1 : rsCStrLen(pM->pCSStrucData); -} -#endif - -/* get the "STRUCTURED-DATA" as sz string - * rgerhards, 2005-11-24 - */ -static inline char *getStructuredData(msg_t *pM) +/* get the "STRUCTURED-DATA" as sz string, including length */ +void +MsgGetStructuredData(msg_t * const pM, uchar **pBuf, rs_size_t *len) { - uchar *pszRet; - MsgLock(pM); - if(pM->pCSStrucData == NULL) - pszRet = UCHAR_CONSTANT("-"); - else - pszRet = rsCStrGetSzStrNoNULL(pM->pCSStrucData); + if(pM->pszStrucData == NULL) { + *pBuf = UCHAR_CONSTANT("-"), + *len = 1; + } else { + *pBuf = pM->pszStrucData, + *len = pM->lenStrucData; + } MsgUnlock(pM); - return (char*) pszRet; } /* get the "programname" as sz string * rgerhards, 2005-10-19 */ -uchar *getProgramName(msg_t *pM, sbool bLockMutex) +uchar *getProgramName(msg_t * const pM, sbool bLockMutex) { if(pM->iLenPROGNAME == -1) { if(bLockMutex == LOCK_MUTEX) { @@ -2139,13 +2319,13 @@ uchar *getProgramName(msg_t *pM, sbool bLockMutex) * now would like to send out the same one via syslog-protocol. * MUST be called with the Msg Lock locked! */ -static void tryEmulateAPPNAME(msg_t *pM) +static void tryEmulateAPPNAME(msg_t * const pM) { assert(pM != NULL); if(pM->pCSAPPNAME != NULL) return; /* we are already done */ - if(getProtocolVersion(pM) == 0) { + if(msgGetProtocolVersion(pM) == 0) { /* only then it makes sense to emulate */ MsgSetAPPNAME(pM, (char*)getProgramName(pM, MUTEX_ALREADY_LOCKED)); } @@ -2157,7 +2337,7 @@ static void tryEmulateAPPNAME(msg_t *pM) * This must be called WITHOUT the message lock being held. * rgerhards, 2009-06-26 */ -static inline void prepareAPPNAME(msg_t *pM, sbool bLockMutex) +static inline void prepareAPPNAME(msg_t * const pM, sbool bLockMutex) { if(pM->pCSAPPNAME == NULL) { if(bLockMutex == LOCK_MUTEX) @@ -2174,7 +2354,7 @@ static inline void prepareAPPNAME(msg_t *pM, sbool bLockMutex) /* rgerhards, 2005-11-24 */ -char *getAPPNAME(msg_t *pM, sbool bLockMutex) +char *getAPPNAME(msg_t * const pM, sbool bLockMutex) { uchar *pszRet; @@ -2193,7 +2373,7 @@ char *getAPPNAME(msg_t *pM, sbool bLockMutex) /* rgerhards, 2005-11-24 */ -static int getAPPNAMELen(msg_t *pM, sbool bLockMutex) +static int getAPPNAMELen(msg_t * const pM, sbool bLockMutex) { assert(pM != NULL); prepareAPPNAME(pM, bLockMutex); @@ -2215,6 +2395,15 @@ void MsgSetInputName(msg_t *pThis, prop_t *inputName) pThis->pInputName = inputName; } +/* Set default TZ. Note that at most 7 chars are set, as we would + * otherwise overrun our buffer! + */ +void MsgSetDfltTZ(msg_t *pThis, char *tz) +{ + strncpy(pThis->dfltTZ, tz, 7); + pThis->dfltTZ[7] = '\0'; /* ensure 0-Term in case of overflow! */ +} + /* Set the pfrominet socket store, so that we can obtain the peer at some * later time. Note that we do not check if pRcvFrom is already set, so this @@ -2237,7 +2426,6 @@ finalize_it: RETiRet; } - /* rgerhards 2008-09-10: set RcvFrom name in msg object. This calls AddRef() * on the property, because this must be done in all current cases and there * is no case expected where this may not be necessary. @@ -2258,7 +2446,7 @@ void MsgSetRcvFrom(msg_t *pThis, prop_t *new) * name (but it works only for the immediate previous). * rgerhards, 2009-06-31 */ -void MsgSetRcvFromStr(msg_t *pThis, uchar *psz, int len, prop_t **ppProp) +void MsgSetRcvFromStr(msg_t * const pThis, const uchar *psz, const int len, prop_t **ppProp) { assert(pThis != NULL); assert(ppProp != NULL); @@ -2293,7 +2481,7 @@ rsRetVal MsgSetRcvFromIP(msg_t *pThis, prop_t *new) * name (but it works only for the immediate previous). * rgerhards, 2009-06-31 */ -rsRetVal MsgSetRcvFromIPStr(msg_t *pThis, uchar *psz, int len, prop_t **ppProp) +rsRetVal MsgSetRcvFromIPStr(msg_t *const pThis, const uchar *psz, const int len, prop_t **ppProp) { DEFiRet; assert(pThis != NULL); @@ -2316,7 +2504,7 @@ finalize_it: * we need it. The rest of the code already knows how to handle an * unset HOSTNAME. */ -void MsgSetHOSTNAME(msg_t *pThis, uchar* pszHOSTNAME, int lenHOSTNAME) +void MsgSetHOSTNAME(msg_t *pThis, const uchar* pszHOSTNAME, const int lenHOSTNAME) { assert(pThis != NULL); @@ -2342,7 +2530,7 @@ void MsgSetHOSTNAME(msg_t *pThis, uchar* pszHOSTNAME, int lenHOSTNAME) * (exactly by one). This can happen if we have a message that does not * contain any MSG part. */ -void MsgSetMSGoffs(msg_t *pMsg, short offs) +void MsgSetMSGoffs(msg_t * const pMsg, short offs) { ISOBJ_TYPE_assert(pMsg, msg); pMsg->offMSG = offs; @@ -2367,7 +2555,7 @@ void MsgSetMSGoffs(msg_t *pMsg, short offs) * the caller is responsible for freeing it. * rgerhards, 2009-06-23 */ -rsRetVal MsgReplaceMSG(msg_t *pThis, uchar* pszMSG, int lenMSG) +rsRetVal MsgReplaceMSG(msg_t *pThis, const uchar* pszMSG, int lenMSG) { int lenNew; uchar *bufNew; @@ -2400,12 +2588,14 @@ finalize_it: * terminated by '\0'. * rgerhards, 2009-06-16 */ -void MsgSetRawMsg(msg_t *pThis, char* pszRawMsg, size_t lenMsg) +void MsgSetRawMsg(msg_t *pThis, const char* pszRawMsg, size_t lenMsg) { + int deltaSize; assert(pThis != NULL); if(pThis->pszRawMsg != pThis->szRawMsg) free(pThis->pszRawMsg); + deltaSize = lenMsg - pThis->iLenRawMsg; pThis->iLenRawMsg = lenMsg; if(pThis->iLenRawMsg < CONF_RAWMSG_BUFSIZE) { /* small enough: use fixed buffer (faster!) */ @@ -2418,6 +2608,11 @@ void MsgSetRawMsg(msg_t *pThis, char* pszRawMsg, size_t lenMsg) memcpy(pThis->pszRawMsg, pszRawMsg, pThis->iLenRawMsg); pThis->pszRawMsg[pThis->iLenRawMsg] = '\0'; /* this also works with truncation! */ + /* correct other information */ + if(pThis->iLenRawMsg > pThis->offMSG) + pThis->iLenMSG += deltaSize; + else + pThis->iLenMSG = 0; } @@ -2426,7 +2621,7 @@ void MsgSetRawMsg(msg_t *pThis, char* pszRawMsg, size_t lenMsg) * try to remove it altogether). * rgerhards, 2009-06-16 */ -void MsgSetRawMsgWOSize(msg_t *pMsg, char* pszRawMsg) +void MsgSetRawMsgWOSize(msg_t * const pMsg, char* pszRawMsg) { MsgSetRawMsg(pMsg, pszRawMsg, strlen(pszRawMsg)); } @@ -2442,11 +2637,11 @@ void MsgSetRawMsgWOSize(msg_t *pMsg, char* pszRawMsg) char *textpri(char *pRes, int pri) { assert(pRes != NULL); - memcpy(pRes, syslog_fac_names[LOG_FAC(pri)], len_syslog_fac_names[LOG_FAC(pri)]); - pRes[len_syslog_fac_names[LOG_FAC(pri)]] = '.'; - memcpy(pRes+len_syslog_fac_names[LOG_FAC(pri)]+1, - syslog_severity_names[LOG_PRI(pri)], - len_syslog_severity_names[LOG_PRI(pri)]+1 /* for \0! */); + memcpy(pRes, syslog_fac_names[pri2fac(pri)], len_syslog_fac_names[pri2fac(pri)]); + pRes[len_syslog_fac_names[pri2fac(pri)]] = '.'; + memcpy(pRes+len_syslog_fac_names[pri2fac(pri)]+1, + syslog_severity_names[pri2sev(pri)], + len_syslog_severity_names[pri2sev(pri)]+1 /* for \0! */); return pRes; } @@ -2463,11 +2658,17 @@ typedef enum ENOWType { NOW_NOW, NOW_YEAR, NOW_MONTH, NOW_DAY, NOW_HOUR, NOW_HHO static uchar *getNOW(eNOWType eNow, struct syslogTime *t) { uchar *pBuf; + struct syslogTime tt; if((pBuf = (uchar*) MALLOC(sizeof(uchar) * tmpBUFSIZE)) == NULL) { return NULL; } + if(t == NULL) { /* can happen if called via script engine */ + datetime.getCurrTime(&tt, NULL); + t = &tt; + } + if(t->year == 0) { /* not yet set! */ datetime.getCurrTime(t, NULL); } @@ -2495,10 +2696,10 @@ static uchar *getNOW(eNOWType eNow, struct syslogTime *t) memcpy(pBuf, two_digits[(int)t->hour], 3); break; case NOW_HHOUR: - memcpy(pBuf, two_digits[t->hour/30], 3); + memcpy(pBuf, two_digits[t->minute/30], 3); break; case NOW_QHOUR: - memcpy(pBuf, two_digits[t->hour/15], 3); + memcpy(pBuf, two_digits[t->minute/15], 3); break; case NOW_MINUTE: memcpy(pBuf, two_digits[(int)t->minute], 3); @@ -2510,12 +2711,12 @@ static uchar *getNOW(eNOWType eNow, struct syslogTime *t) #undef tmpBUFSIZE /* clean up */ -/* Get a CEE-Property as string value*/ +/* Get a JSON-Property as string value (used for various types of JSON-based vars) */ rsRetVal -getCEEPropVal(msg_t *pM, es_str_t *propName, uchar **pRes, rs_size_t *buflen, unsigned short *pbMustBeFreed) +getJSONPropVal(msg_t * const pMsg, msgPropDescr_t *pProp, uchar **pRes, rs_size_t *buflen, unsigned short *pbMustBeFreed) { - uchar *name = NULL; uchar *leaf; + struct json_object *jroot; struct json_object *parent; struct json_object *field; DEFiRet; @@ -2523,15 +2724,26 @@ getCEEPropVal(msg_t *pM, es_str_t *propName, uchar **pRes, rs_size_t *buflen, un if(*pbMustBeFreed) free(*pRes); *pRes = NULL; - // TODO: mutex? - if(pM->json == NULL) goto finalize_it; - if(!es_strbufcmp(propName, (uchar*)"!", 1)) { - field = pM->json; + if(pProp->id == PROP_CEE) { + jroot = pMsg->json; + } else if(pProp->id == PROP_LOCAL_VAR) { + jroot = pMsg->localvars; + } else if(pProp->id == PROP_GLOBAL_VAR) { + pthread_rwlock_rdlock(&glblVars_rwlock); + jroot = global_var_root; } else { - name = (uchar*)es_str2cstr(propName, NULL); - leaf = jsonPathGetLeaf(name, ustrlen(name)); - CHKiRet(jsonPathFindParent(pM, name, leaf, &parent, 1)); + DBGPRINTF("msgGetJSONPropVal; invalid property id %d\n", + pProp->id); + ABORT_FINALIZE(RS_RET_NOT_FOUND); + } + if(jroot == NULL) goto finalize_it; + + if(!strcmp((char*)pProp->name, "!")) { + field = jroot; + } else { + leaf = jsonPathGetLeaf(pProp->name, pProp->nameLen); + CHKiRet(jsonPathFindParent(jroot, pProp->name, leaf, &parent, 1)); field = json_object_object_get(parent, (char*)leaf); } if(field != NULL) { @@ -2541,7 +2753,8 @@ getCEEPropVal(msg_t *pM, es_str_t *propName, uchar **pRes, rs_size_t *buflen, un } finalize_it: - free(name); + if(pProp->id == PROP_GLOBAL_VAR) + pthread_rwlock_unlock(&glblVars_rwlock); if(*pRes == NULL) { /* could not find any value, so set it to empty */ *pRes = (unsigned char*)""; @@ -2551,55 +2764,71 @@ finalize_it: } -/* Get a CEE-Property as native json object - */ +/* Get a JSON-based-variable as native json object */ rsRetVal -msgGetCEEPropJSON(msg_t *pM, es_str_t *propName, struct json_object **pjson) +msgGetJSONPropJSON(msg_t * const pMsg, msgPropDescr_t *pProp, struct json_object **pjson) { - uchar *name = NULL; + struct json_object *jroot; uchar *leaf; struct json_object *parent; DEFiRet; - // TODO: mutex? - if(pM->json == NULL) { + if(pProp->id == PROP_CEE) { + jroot = pMsg->json; + } else if(pProp->id == PROP_LOCAL_VAR) { + jroot = pMsg->localvars; + } else if(pProp->id == PROP_GLOBAL_VAR) { + pthread_rwlock_rdlock(&glblVars_rwlock); + jroot = global_var_root; + } else { + DBGPRINTF("msgGetJSONPropJSON; invalid property id %d\n", + pProp->id); + ABORT_FINALIZE(RS_RET_NOT_FOUND); + } + if(jroot == NULL) { + DBGPRINTF("msgGetJSONPropJSON; jroot empty for property %s\n", + pProp->name); ABORT_FINALIZE(RS_RET_NOT_FOUND); } - if(!es_strbufcmp(propName, (uchar*)"!", 1)) { - *pjson = pM->json; + if(!strcmp((char*)pProp->name, "!")) { + *pjson = jroot; FINALIZE; } - name = (uchar*)es_str2cstr(propName, NULL); - leaf = jsonPathGetLeaf(name, ustrlen(name)); - CHKiRet(jsonPathFindParent(pM, name, leaf, &parent, 1)); + leaf = jsonPathGetLeaf(pProp->name, pProp->nameLen); + CHKiRet(jsonPathFindParent(jroot, pProp->name, leaf, &parent, 1)); *pjson = json_object_object_get(parent, (char*)leaf); if(*pjson == NULL) { ABORT_FINALIZE(RS_RET_NOT_FOUND); } finalize_it: - free(name); + if(pProp->id == PROP_GLOBAL_VAR) + pthread_rwlock_unlock(&glblVars_rwlock); RETiRet; } /* Encode a JSON value and add it to provided string. Note that * the string object may be NULL. In this case, it is created - * if and only if escaping is needed. + * if and only if escaping is needed. if escapeAll is false, previously + * escaped strings are left as is */ static rsRetVal -jsonAddVal(uchar *pSrc, unsigned buflen, es_str_t **dst) +jsonAddVal(uchar *pSrc, unsigned buflen, es_str_t **dst, int escapeAll) { unsigned char c; es_size_t i; char numbuf[4]; + unsigned ni; + unsigned char nc; int j; DEFiRet; for(i = 0 ; i < buflen ; ++i) { c = pSrc[i]; - if( (c >= 0x23 && c <= 0x5b) + if( (c >= 0x23 && c <= 0x2e) + || (c >= 0x30 && c <= 0x5b) || (c >= 0x5d /* && c <= 0x10FFFF*/) || c == 0x20 || c == 0x21) { /* no need to escape */ @@ -2629,6 +2858,23 @@ jsonAddVal(uchar *pSrc, unsigned buflen, es_str_t **dst) es_addBuf(dst, "\\/", 2); break; case '\\': + if (escapeAll == RSFALSE) { + ni = i + 1; + if (ni <= buflen) { + nc = pSrc[ni]; + + /* Attempt to not double encode */ + if ( nc == '"' || nc == '/' || nc == '\\' || nc == 'b' || nc == 'f' + || nc == 'n' || nc == 'r' || nc == 't' || nc == 'u') { + + es_addChar(dst, c); + es_addChar(dst, nc); + i = ni; + break; + } + } + } + es_addBuf(dst, "\\\\", 2); break; case '\010': @@ -2674,7 +2920,7 @@ finalize_it: * rgerhards, 2012-03-16 */ static rsRetVal -jsonEncode(uchar **ppRes, unsigned short *pbMustBeFreed, int *pBufLen) +jsonEncode(uchar **ppRes, unsigned short *pbMustBeFreed, int *pBufLen, int escapeAll) { unsigned buflen; uchar *pSrc; @@ -2683,7 +2929,7 @@ jsonEncode(uchar **ppRes, unsigned short *pbMustBeFreed, int *pBufLen) pSrc = *ppRes; buflen = (*pBufLen == -1) ? ustrlen(pSrc) : *pBufLen; - CHKiRet(jsonAddVal(pSrc, buflen, &dst)); + CHKiRet(jsonAddVal(pSrc, buflen, &dst, escapeAll)); if(dst != NULL) { /* we updated the string and need to replace the @@ -2712,7 +2958,7 @@ finalize_it: * something to consider at a later stage. rgerhards, 2012-04-19 */ static rsRetVal -jsonField(struct templateEntry *pTpe, uchar **ppRes, unsigned short *pbMustBeFreed, int *pBufLen) +jsonField(struct templateEntry *pTpe, uchar **ppRes, unsigned short *pbMustBeFreed, int *pBufLen, int escapeAll) { unsigned buflen; uchar *pSrc; @@ -2726,7 +2972,7 @@ jsonField(struct templateEntry *pTpe, uchar **ppRes, unsigned short *pbMustBeFre es_addChar(&dst, '"'); es_addBuf(&dst, (char*)pTpe->fieldName, pTpe->lenFieldName); es_addBufConstcstr(&dst, "\":\""); - CHKiRet(jsonAddVal(pSrc, buflen, &dst)); + CHKiRet(jsonAddVal(pSrc, buflen, &dst, escapeAll)); es_addChar(&dst, '"'); if(*pbMustBeFreed) @@ -2783,9 +3029,9 @@ finalize_it: #define RET_OUT_OF_MEMORY { *pbMustBeFreed = 0;\ *pPropLen = sizeof("**OUT OF MEMORY**") - 1; \ return(UCHAR_CONSTANT("**OUT OF MEMORY**"));} -uchar *MsgGetProp(msg_t *pMsg, struct templateEntry *pTpe, - propid_t propid, es_str_t *propName, rs_size_t *pPropLen, - unsigned short *pbMustBeFreed, struct syslogTime *ttNow) +uchar *MsgGetProp(msg_t *__restrict__ const pMsg, struct templateEntry *__restrict__ const pTpe, + msgPropDescr_t *pProp, rs_size_t *__restrict__ const pPropLen, + unsigned short *__restrict__ const pbMustBeFreed, struct syslogTime * const ttNow) { uchar *pRes; /* result pointer */ rs_size_t bufLen = -1; /* length of string or -1, if not known */ @@ -2807,7 +3053,7 @@ uchar *MsgGetProp(msg_t *pMsg, struct templateEntry *pTpe, *pbMustBeFreed = 0; - switch(propid) { + switch(pProp->id) { case PROP_MSG: pRes = getMSG(pMsg); bufLen = getMSGLen(pMsg); @@ -2880,7 +3126,7 @@ uchar *MsgGetProp(msg_t *pMsg, struct templateEntry *pTpe, pRes = (uchar*)getProtocolVersionString(pMsg); break; case PROP_STRUCTURED_DATA: - pRes = (uchar*)getStructuredData(pMsg); + MsgGetStructuredData(pMsg, &pRes, &bufLen); break; case PROP_APP_NAME: pRes = (uchar*)getAPPNAME(pMsg, LOCK_MUTEX); @@ -2891,6 +3137,10 @@ uchar *MsgGetProp(msg_t *pMsg, struct templateEntry *pTpe, case PROP_MSGID: pRes = (uchar*)getMSGID(pMsg); break; + case PROP_JSONMESG: + pRes = (uchar*)msgGetJSONMESG(pMsg); + *pbMustBeFreed = 1; + break; #ifdef USE_LIBUUID case PROP_UUID: getUUID(pMsg, &pRes, &bufLen); @@ -2968,8 +3218,6 @@ uchar *MsgGetProp(msg_t *pMsg, struct templateEntry *pTpe, break; case PROP_CEE_ALL_JSON: if(pMsg->json == NULL) { - if(*pbMustBeFreed == 1) - free(pRes); pRes = (uchar*) "{}"; bufLen = 2; *pbMustBeFreed = 0; @@ -2979,35 +3227,59 @@ uchar *MsgGetProp(msg_t *pMsg, struct templateEntry *pTpe, } break; case PROP_CEE: - getCEEPropVal(pMsg, propName, &pRes, &bufLen, pbMustBeFreed); + case PROP_LOCAL_VAR: + case PROP_GLOBAL_VAR: + getJSONPropVal(pMsg, pProp, &pRes, &bufLen, pbMustBeFreed); break; case PROP_SYS_BOM: - if(*pbMustBeFreed == 1) - free(pRes); pRes = (uchar*) "\xEF\xBB\xBF"; *pbMustBeFreed = 0; break; case PROP_SYS_UPTIME: # ifndef HAVE_SYSINFO_UPTIME - /* An alternative on some systems (eg Solaris) is to scan - * /var/adm/utmpx for last boot time. - */ + /* An alternative on some systems (eg Solaris) is to scan + * /var/adm/utmpx for last boot time. + */ pRes = (uchar*) "UPTIME NOT available on this system"; *pbMustBeFreed = 0; + +# elif defined(__FreeBSD__) + + { + struct timespec tp; + + if((pRes = (uchar*) MALLOC(sizeof(uchar) * 32)) == NULL) { + RET_OUT_OF_MEMORY; + } + + if(clock_gettime(CLOCK_UPTIME, &tp) == -1) { + free(pRes); + *pPropLen = sizeof("**SYSCALL FAILED**") - 1; + return(UCHAR_CONSTANT("**SYSCALL FAILED**")); + } + + *pbMustBeFreed = 1; + + snprintf((char*) pRes, sizeof(uchar) * 32, "%ld", tp.tv_sec); + } + # else + { struct sysinfo s_info; if((pRes = (uchar*) MALLOC(sizeof(uchar) * 32)) == NULL) { RET_OUT_OF_MEMORY; } - *pbMustBeFreed = 1; if(sysinfo(&s_info) < 0) { + free(pRes); *pPropLen = sizeof("**SYSCALL FAILED**") - 1; return(UCHAR_CONSTANT("**SYSCALL FAILED**")); } + *pbMustBeFreed = 1; + snprintf((char*) pRes, sizeof(uchar) * 32, "%ld", s_info.uptime); } # endif @@ -3016,7 +3288,7 @@ uchar *MsgGetProp(msg_t *pMsg, struct templateEntry *pTpe, /* there is no point in continuing, we may even otherwise render the * error message unreadable. rgerhards, 2007-07-10 */ - dbgprintf("invalid property id: '%d'\n", propid); + dbgprintf("invalid property id: '%d'\n", pProp->id); *pbMustBeFreed = 0; *pPropLen = sizeof("**INVALID PROPERTY NAME**") - 1; return UCHAR_CONSTANT("**INVALID PROPERTY NAME**"); @@ -3075,7 +3347,7 @@ uchar *MsgGetProp(msg_t *pMsg, struct templateEntry *pTpe, /* we got our end pointer, now do the copy */ /* TODO: code copied from below, this is a candidate for a separate function */ iLen = pFldEnd - pFld + 1; /* the +1 is for an actual char, NOT \0! */ - pBufStart = pBuf = MALLOC((iLen + 1) * sizeof(char)); + pBufStart = pBuf = MALLOC((iLen + 1) * sizeof(uchar)); if(pBuf == NULL) { if(*pbMustBeFreed == 1) free(pRes); @@ -3253,7 +3525,7 @@ uchar *MsgGetProp(msg_t *pMsg, struct templateEntry *pTpe, if(iTo > bufLen) /* iTo is very large, if no to-position is set in the template! */ iTo = bufLen; iLen = iTo - iFrom + 1; /* the +1 is for an actual char, NOT \0! */ - pBufStart = pBuf = MALLOC((iLen + 1) * sizeof(char)); + pBufStart = pBuf = MALLOC((iLen + 1) * sizeof(uchar)); if(pBuf == NULL) { if(*pbMustBeFreed == 1) free(pRes); @@ -3307,7 +3579,7 @@ uchar *MsgGetProp(msg_t *pMsg, struct templateEntry *pTpe, uchar *pBStart; uchar *pB; uchar *pSrc; - pBStart = pB = MALLOC((bufLen + 1) * sizeof(char)); + pBStart = pB = MALLOC((bufLen + 1) * sizeof(uchar)); if(pB == NULL) { if(*pbMustBeFreed == 1) free(pRes); @@ -3612,9 +3884,13 @@ uchar *MsgGetProp(msg_t *pMsg, struct templateEntry *pTpe, bufLen = -1; *pbMustBeFreed = 1; } else if(pTpe->data.field.options.bJSON) { - jsonEncode(&pRes, pbMustBeFreed, &bufLen); + jsonEncode(&pRes, pbMustBeFreed, &bufLen, RSTRUE); } else if(pTpe->data.field.options.bJSONf) { - jsonField(pTpe, &pRes, pbMustBeFreed, &bufLen); + jsonField(pTpe, &pRes, pbMustBeFreed, &bufLen, RSTRUE); + } else if(pTpe->data.field.options.bJSONr) { + jsonEncode(&pRes, pbMustBeFreed, &bufLen, RSFALSE); + } else if(pTpe->data.field.options.bJSONfr) { + jsonField(pTpe, &pRes, pbMustBeFreed, &bufLen, RSFALSE); } *pPropLen = (bufLen == -1) ? ustrlen(pRes) : bufLen; @@ -3624,66 +3900,6 @@ uchar *MsgGetProp(msg_t *pMsg, struct templateEntry *pTpe, } -/* The function returns a cee variable suitable for use with RainerScript. - * Note: caller must free the returned string. - * Note that we need to do a lot of conversions between es_str_t and cstr -- this will go away once - * we have moved larger parts of rsyslog to es_str_t. Acceptable for the moment, especially as we intend - * to rewrite the script engine as well! - * rgerhards, 2010-12-03 - */ -es_str_t* -msgGetCEEVarNew(msg_t *pMsg, char *name) -{ - uchar *leaf; - char *val; - es_str_t *estr = NULL; - struct json_object *json, *parent; - - ISOBJ_TYPE_assert(pMsg, msg); - - if(pMsg->json == NULL) { - estr = es_newStr(1); - goto done; - } - leaf = jsonPathGetLeaf((uchar*)name, strlen(name)); - if(jsonPathFindParent(pMsg, (uchar*)name, leaf, &parent, 1) != RS_RET_OK) { - estr = es_newStr(1); - goto done; - } - json = json_object_object_get(parent, (char*)leaf); - val = (char*)json_object_get_string(json); - estr = es_newStrFromCStr(val, strlen(val)); -done: - return estr; -} - - -/* Return an es_str_t for given message property. - */ -es_str_t* -msgGetMsgVarNew(msg_t *pThis, uchar *name) -{ - rs_size_t propLen; - uchar *pszProp = NULL; - propid_t propid; - unsigned short bMustBeFreed = 0; - es_str_t *estr; - - ISOBJ_TYPE_assert(pThis, msg); - - /* always call MsgGetProp() without a template specifier */ - /* TODO: optimize propNameToID() call -- rgerhards, 2009-06-26 */ - propNameStrToID(name, &propid); - pszProp = (uchar*) MsgGetProp(pThis, NULL, propid, NULL, &propLen, &bMustBeFreed, NULL); - - estr = es_newStrFromCStr((char*)pszProp, propLen); - if(bMustBeFreed) - free(pszProp); - - return estr; -} - - /* This function can be used as a generic way to set properties. * We have to handle a lot of legacy, so our return value is not always * 100% correct (called functions do not always provide one, should @@ -3734,7 +3950,7 @@ rsRetVal MsgSetProperty(msg_t *pThis, var_t *pProp) prop.Destruct(&propRcvFrom); } else if(isProp("pszHOSTNAME")) { MsgSetHOSTNAME(pThis, rsCStrGetSzStrNoNULL(pProp->val.pStr), rsCStrLen(pProp->val.pStr)); - } else if(isProp("pCSStrucData")) { + } else if(isProp("pszStrucData")) { MsgSetStructuredData(pThis, (char*) rsCStrGetSzStrNoNULL(pProp->val.pStr)); } else if(isProp("pCSAPPNAME")) { MsgSetAPPNAME(pThis, (char*) rsCStrGetSzStrNoNULL(pProp->val.pStr)); @@ -3769,12 +3985,142 @@ finalize_it: #undef isProp +/* Set a single property based on the JSON object provided. The + * property name is extracted from the JSON object. + */ +static rsRetVal +msgSetPropViaJSON(msg_t *__restrict__ const pMsg, const char *name, struct json_object *json) +{ + const char *psz; + int val; + prop_t *propFromHost = NULL; + prop_t *propRcvFromIP = NULL; + DEFiRet; + + // TODO: think if we need to lock the message mutex. For some updates + // we probably need to! + + /* note: json_object_get_string() manages the memory of the returned + * string. So we MUST NOT free it! + */ + dbgprintf("DDDD: msgSetPropViaJSON key: '%s'\n", name); + if(!strcmp(name, "rawmsg")) { + psz = json_object_get_string(json); + MsgSetRawMsg(pMsg, psz, strlen(psz)); + } else if(!strcmp(name, "msg")) { + psz = json_object_get_string(json); + MsgReplaceMSG(pMsg, (const uchar*)psz, strlen(psz)); + } else if(!strcmp(name, "syslogtag")) { + psz = json_object_get_string(json); + MsgSetTAG(pMsg, (const uchar*)psz, strlen(psz)); + } else if(!strcmp(name, "syslogfacility")) { + val = json_object_get_int(json); + if(val >= 0 && val <= 24) + pMsg->iFacility = val; + else + DBGPRINTF("mmexternal: invalid fac %d requested -- ignored\n", val); + } else if(!strcmp(name, "syslogseverity")) { + val = json_object_get_int(json); + if(val >= 0 && val <= 7) + pMsg->iSeverity = val; + else + DBGPRINTF("mmexternal: invalid fac %d requested -- ignored\n", val); + } else if(!strcmp(name, "procid")) { + psz = json_object_get_string(json); + MsgSetPROCID(pMsg, psz); + } else if(!strcmp(name, "msgid")) { + psz = json_object_get_string(json); + MsgSetMSGID(pMsg, psz); + } else if(!strcmp(name, "structured-data")) { + psz = json_object_get_string(json); + MsgSetStructuredData(pMsg, psz); + } else if(!strcmp(name, "hostname") || !strcmp(name, "source")) { + psz = json_object_get_string(json); + MsgSetHOSTNAME(pMsg, (const uchar*)psz, strlen(psz)); + } else if(!strcmp(name, "fromhost")) { + psz = json_object_get_string(json); + MsgSetRcvFromStr(pMsg, (const uchar*) psz, 0, &propFromHost); + } else if(!strcmp(name, "fromhost-ip")) { + psz = json_object_get_string(json); + MsgSetRcvFromIPStr(pMsg, (const uchar*)psz, strlen(psz), &propRcvFromIP); + } else if(!strcmp(name, "$!")) { + msgAddJSON(pMsg, (uchar*)"!", json); + } else { + /* we ignore unknown properties */ + DBGPRINTF("msgSetPropViaJSON: unkonwn property ignored: %s\n", + name); + } + RETiRet; +} + + +/* set message properties based on JSON string. This function does it all, + * including parsing the JSON string. If an error is detected, the operation + * is aborted at the time of error. Any modifications made before the + * error ocurs are still PERSISTED. + * This function is meant to support the external message modifiction module + * interface. As such, replacing properties is expressively permited. Note that + * properties which were derived from the message during parsing are NOT + * updated if the underlying (raw)msg property is changed. + */ +rsRetVal +MsgSetPropsViaJSON(msg_t *__restrict__ const pMsg, const uchar *__restrict__ const jsonstr) +{ + struct json_tokener *tokener = NULL; + struct json_object *json; + const char *errMsg; + DEFiRet; + + DBGPRINTF("DDDDDD: JSON string for message mod: '%s'\n", jsonstr); + if(!strcmp((char*)jsonstr, "{}")) /* shortcut for a common case */ + FINALIZE; + + tokener = json_tokener_new(); + + json = json_tokener_parse_ex(tokener, (char*)jsonstr, ustrlen(jsonstr)); + if(Debug) { + errMsg = NULL; + if(json == NULL) { + enum json_tokener_error err; + + err = tokener->err; + if(err != json_tokener_continue) +# if HAVE_JSON_TOKENER_ERROR_DESC + errMsg = json_tokener_error_desc(err); +# else + errMsg = json_tokener_errors[err]; +# endif + else + errMsg = "Unterminated input"; + } else if(!json_object_is_type(json, json_type_object)) + errMsg = "JSON value is not an object"; + if(errMsg != NULL) { + DBGPRINTF("MsgSetPropsViaJSON: Error parsing JSON '%s': %s\n", + jsonstr, errMsg); + } + } + if(json == NULL || !json_object_is_type(json, json_type_object)) { + ABORT_FINALIZE(RS_RET_JSON_PARSE_ERR); + } + + json_object_object_foreach(json, name, val) { + msgSetPropViaJSON(pMsg, name, val); + } + json_object_put(json); + +finalize_it: + if(tokener != NULL) + json_tokener_free(tokener); + RETiRet; +} + + /* get the severity - this is an entry point that * satisfies the base object class getSeverity semantics. * rgerhards, 2008-01-14 */ rsRetVal -MsgGetSeverity(msg_t *pMsg, int *piSeverity) +MsgGetSeverity(msg_t * const pMsg, int *piSeverity) { *piSeverity = pMsg->iSeverity; return RS_RET_OK; @@ -3785,16 +4131,22 @@ static uchar * jsonPathGetLeaf(uchar *name, int lenName) { int i; - for(i = lenName ; name[i] != '!' && i >= 0 ; --i) - /* just skip */; - if(name[i] == '!') + for(i = lenName ; i >= 0 ; --i) + if(i == 0) { + if(name[0] == '!' || name[0] == '.' || name[0] == '/') + break; + } else { + if(name[i] == '!') + break; + } + if(name[i] == '!' || name[i] == '.' || name[i] == '/') ++i; return name + i; } static rsRetVal -jsonPathFindNext(struct json_object *root, uchar **name, uchar *leaf, +jsonPathFindNext(struct json_object *root, uchar *namestart, uchar **name, uchar *leaf, struct json_object **found, int bCreate) { uchar namebuf[1024]; @@ -3803,13 +4155,12 @@ jsonPathFindNext(struct json_object *root, uchar **name, uchar *leaf, uchar *p = *name; DEFiRet; - if(*p == '!') + if(*p == '!' || (*name == namestart && (*p == '.' || *p == '/'))) ++p; - for(i = 0 ; *p && *p != '!' && p != leaf && i < sizeof(namebuf)-1 ; ++i, ++p) + for(i = 0 ; *p && !(p == namestart && (*p == '.' || *p == '/')) && *p != '!' && p != leaf && i < sizeof(namebuf)-1 ; ++i, ++p) namebuf[i] = *p; if(i > 0) { namebuf[i] = '\0'; - dbgprintf("AAAA: next JSONPath elt: '%s'\n", namebuf); json = json_object_object_get(root, (char*)namebuf); } else json = root; @@ -3829,12 +4180,14 @@ finalize_it: } static rsRetVal -jsonPathFindParent(msg_t *pM, uchar *name, uchar *leaf, struct json_object **parent, int bCreate) +jsonPathFindParent(struct json_object *jroot, uchar *name, uchar *leaf, struct json_object **parent, int bCreate) { + uchar *namestart; DEFiRet; - *parent = pM->json; + namestart = name; + *parent = jroot; while(name < leaf-1) { - jsonPathFindNext(*parent, &name, leaf, parent, bCreate); + jsonPathFindNext(*parent, namestart, &name, leaf, parent, bCreate); } RETiRet; } @@ -3847,7 +4200,6 @@ jsonMerge(struct json_object *existing, struct json_object *json) struct json_object_iter it; json_object_object_foreachC(json, it) { -DBGPRINTF("AAAA jsonMerge adds '%s'\n", it.key); json_object_object_add(existing, it.key, json_object_get(it.val)); } @@ -3861,63 +4213,75 @@ DBGPRINTF("AAAA jsonMerge adds '%s'\n", it.key); /* find a JSON structure element (field or container doesn't matter). */ rsRetVal -jsonFind(msg_t *pM, es_str_t *propName, struct json_object **jsonres) +jsonFind(struct json_object *jroot, msgPropDescr_t *pProp, struct json_object **jsonres) { - uchar *name = NULL; uchar *leaf; struct json_object *parent; struct json_object *field; DEFiRet; - if(pM->json == NULL) { + if(jroot == NULL) { field = NULL; goto finalize_it; } - if(!es_strbufcmp(propName, (uchar*)"!", 1)) { - field = pM->json; + if(!strcmp((char*)pProp->name, "!")) { + field = jroot; } else { - name = (uchar*)es_str2cstr(propName, NULL); - leaf = jsonPathGetLeaf(name, ustrlen(name)); - CHKiRet(jsonPathFindParent(pM, name, leaf, &parent, 0)); + leaf = jsonPathGetLeaf(pProp->name, pProp->nameLen); + CHKiRet(jsonPathFindParent(jroot, pProp->name, leaf, &parent, 0)); field = json_object_object_get(parent, (char*)leaf); } *jsonres = field; finalize_it: - free(name); RETiRet; } rsRetVal -msgAddJSON(msg_t *pM, uchar *name, struct json_object *json) +msgAddJSON(msg_t * const pM, uchar *name, struct json_object *json) { /* TODO: error checks! This is a quick&dirty PoC! */ + struct json_object **pjroot; struct json_object *parent, *leafnode; uchar *leaf; DEFiRet; MsgLock(pM); - if(name[0] == '!' && name[1] == '\0') { - if(pM->json == NULL) - pM->json = json; + if(name[0] == '!') { + pjroot = &pM->json; + } else if(name[0] == '.') { + pjroot = &pM->localvars; + } else { /* globl var */ + pthread_rwlock_wrlock(&glblVars_rwlock); + pjroot = &global_var_root; + } + + if(name[1] == '\0') { /* full tree? */ + if(*pjroot == NULL) + *pjroot = json; else - CHKiRet(jsonMerge(pM->json, json)); + CHKiRet(jsonMerge(*pjroot, json)); } else { - if(pM->json == NULL) { + if(*pjroot == NULL) { /* now we need a root obj */ - pM->json = json_object_new_object(); + *pjroot = json_object_new_object(); } leaf = jsonPathGetLeaf(name, ustrlen(name)); - CHKiRet(jsonPathFindParent(pM, name, leaf, &parent, 1)); + CHKiRet(jsonPathFindParent(*pjroot, name, leaf, &parent, 1)); + if (json_object_get_type(parent) != json_type_object) { + DBGPRINTF("msgAddJSON: not a container in json path," + "name is '%s'\n", name); + json_object_put(json); + ABORT_FINALIZE(RS_RET_INVLD_SETOP); + } leafnode = json_object_object_get(parent, (char*)leaf); if(leafnode == NULL) { json_object_object_add(parent, (char*)leaf, json); } else { if(json_object_get_type(json) == json_type_object) { - CHKiRet(jsonMerge(pM->json, json)); + CHKiRet(jsonMerge(*pjroot, json)); } else { -//dbgprintf("AAAA: leafnode already exists, type is %d, update with %d\n", (int)json_object_get_type(leafnode), (int)json_object_get_type(json)); /* TODO: improve the code below, however, the current * state is not really bad */ if(json_object_get_type(leafnode) == json_type_object) { @@ -3941,35 +4305,52 @@ msgAddJSON(msg_t *pM, uchar *name, struct json_object *json) } finalize_it: + if(name[0] == '/') + pthread_rwlock_unlock(&glblVars_rwlock); MsgUnlock(pM); RETiRet; } + rsRetVal -msgDelJSON(msg_t *pM, uchar *name) +msgDelJSON(msg_t * const pM, uchar *name) { + struct json_object **jroot; struct json_object *parent, *leafnode; uchar *leaf; DEFiRet; -dbgprintf("AAAA: unset variable '%s'\n", name); MsgLock(pM); - if(name[0] == '!' && name[1] == '\0') { - /* strange, but I think we should permit this. After all, + + if(name[0] == '!') { + jroot = &pM->json; + } else if(name[0] == '.') { + jroot = &pM->localvars; + } else { /* globl var */ + pthread_rwlock_wrlock(&glblVars_rwlock); + jroot = &global_var_root; + } + if(jroot == NULL) { + DBGPRINTF("msgDelJSONVar; jroot empty in unset for property %s\n", + name); + FINALIZE; + } + + if(name[1] == '\0') { + /* full tree! Strange, but I think we should permit this. After all, * we trust rsyslog.conf to be written by the admin. */ DBGPRINTF("unsetting JSON root object\n"); - json_object_put(pM->json); - pM->json = NULL; + json_object_put(*jroot); + *jroot = NULL; } else { - if(pM->json == NULL) { + if(*jroot == NULL) { /* now we need a root obj */ - pM->json = json_object_new_object(); + *jroot = json_object_new_object(); } leaf = jsonPathGetLeaf(name, ustrlen(name)); - CHKiRet(jsonPathFindParent(pM, name, leaf, &parent, 1)); + CHKiRet(jsonPathFindParent(*jroot, name, leaf, &parent, 1)); leafnode = json_object_object_get(parent, (char*)leaf); -DBGPRINTF("AAAA: unset found JSON value path '%s', " "leaf '%s', leafnode %p\n", name, leaf, leafnode); if(leafnode == NULL) { DBGPRINTF("unset JSON: could not find '%s'\n", name); ABORT_FINALIZE(RS_RET_JNAME_NOTFOUND); @@ -3982,6 +4363,8 @@ DBGPRINTF("AAAA: unset found JSON value path '%s', " "leaf '%s', leafnode %p\n", } finalize_it: + if(name[0] == '/') + pthread_rwlock_unlock(&glblVars_rwlock); MsgUnlock(pM); RETiRet; } @@ -4003,7 +4386,11 @@ jsonDeepCopy(struct json_object *src) dst = json_object_new_double(json_object_get_double(src)); break; case json_type_int: +#ifdef HAVE_JSON_OBJECT_NEW_INT64 + dst = json_object_new_int64(json_object_get_int64(src)); +#else /* HAVE_JSON_OBJECT_NEW_INT64 */ dst = json_object_new_int(json_object_get_int(src)); +#endif /* HAVE_JSON_OBJECT_NEW_INT64 */ break; case json_type_string: dst = json_object_new_string(json_object_get_string(src)); @@ -4034,7 +4421,7 @@ done: return dst; rsRetVal -msgSetJSONFromVar(msg_t *pMsg, uchar *varname, struct var *v) +msgSetJSONFromVar(msg_t * const pMsg, uchar *varname, struct var *v) { struct json_object *json = NULL; char *cstr; @@ -4046,7 +4433,11 @@ msgSetJSONFromVar(msg_t *pMsg, uchar *varname, struct var *v) free(cstr); break; case 'N':/* number (integer) */ +#ifdef HAVE_JSON_OBJECT_NEW_INT64 + json = json_object_new_int64(v->d.n); +#else /* HAVE_JSON_OBJECT_NEW_INT64 */ json = json_object_new_int((int) v->d.n); +#endif /* HAVE_JSON_OBJECT_NEW_INT64 */ break; case 'J':/* native JSON */ json = jsonDeepCopy(v->d.json); @@ -4055,11 +4446,76 @@ msgSetJSONFromVar(msg_t *pMsg, uchar *varname, struct var *v) v->datatype); ABORT_FINALIZE(RS_RET_ERR); } - msgAddJSON(pMsg, varname+1, json); + + msgAddJSON(pMsg, varname, json); finalize_it: RETiRet; } +rsRetVal +MsgAddToStructuredData(msg_t * const pMsg, uchar *toadd, rs_size_t len) +{ + uchar *newptr; + rs_size_t newlen; + DEFiRet; + newlen = (pMsg->pszStrucData[0] == '-') ? len : pMsg->lenStrucData + len; + CHKmalloc(newptr = (uchar*) realloc(pMsg->pszStrucData, newlen+1)); + pMsg->pszStrucData = newptr; + if(pMsg->pszStrucData[0] == '-') { /* empty? */ + memcpy(pMsg->pszStrucData, toadd, len); + } else { + memcpy(pMsg->pszStrucData+pMsg->lenStrucData, toadd, len); + } + pMsg->pszStrucData[newlen] = '\0'; + pMsg->lenStrucData = newlen; +finalize_it: + RETiRet; +} + + +/* Fill a message propert description. Space must already be alloced + * by the caller. This is for efficiency, as we expect this to happen + * as part of a larger structure alloc. + * Note that CEE/LOCAL_VAR properties can come in either as + * "$!xx"/"$.xx" or "!xx"/".xx" - we will unify them here. + */ +rsRetVal +msgPropDescrFill(msgPropDescr_t *pProp, uchar *name, int nameLen) +{ + propid_t id; + int offs; + DEFiRet; + if(propNameToID(name, &id) != RS_RET_OK) { + parser_errmsg("invalid property '%s'", name); + ABORT_FINALIZE(RS_RET_INVLD_PROP); + } + if(id == PROP_CEE || id == PROP_LOCAL_VAR || id == PROP_GLOBAL_VAR) { + /* in these cases, we need the field name for later processing */ + /* normalize name: remove $ if present */ + offs = (name[0] == '$') ? 1 : 0; + pProp->name = ustrdup(name + offs); + pProp->nameLen = nameLen - offs; + /* we patch the root name, so that support functions do not need to + * check for different root chars. */ + pProp->name[0] = '!'; + } + pProp->id = id; +finalize_it: + RETiRet; +} + +void +msgPropDescrDestruct(msgPropDescr_t *pProp) +{ + if(pProp != NULL) { + if(pProp->id == PROP_CEE || + pProp->id == PROP_LOCAL_VAR || + pProp->id == PROP_GLOBAL_VAR) + free(pProp->name); + } +} + + /* dummy */ rsRetVal msgQueryInterface(void) { return RS_RET_NOT_IMPLEMENTED; } @@ -4068,6 +4524,8 @@ rsRetVal msgQueryInterface(void) { return RS_RET_NOT_IMPLEMENTED; } * rgerhards, 2008-01-04 */ BEGINObjClassInit(msg, 1, OBJ_IS_CORE_MODULE) + pthread_rwlock_init(&glblVars_rwlock, NULL); + /* request objects we use */ CHKiRet(objUse(datetime, CORE_COMPONENT)); CHKiRet(objUse(glbl, CORE_COMPONENT)); diff --git a/runtime/msg.h b/runtime/msg.h index edf5ed9..ec22545 100644 --- a/runtime/msg.h +++ b/runtime/msg.h @@ -3,7 +3,7 @@ * * File begun on 2007-07-13 by RGerhards (extracted from syslogd.c) * - * Copyright 2007-2012 Rainer Gerhards and Adiscon GmbH. + * Copyright 2007-2013 Rainer Gerhards and Adiscon GmbH. * * This file is part of the rsyslog runtime library. * @@ -30,13 +30,12 @@ #include <pthread.h> #include <libestr.h> -#include <json/json.h> +#include <stdint.h> +#include <json.h> #include "obj.h" #include "syslogd-types.h" #include "template.h" #include "atomic.h" -#include "libee/libee.h" - /* rgerhards 2004-11-08: The following structure represents a * syslog message. @@ -64,7 +63,6 @@ struct msg { once data has entered the queue, this property is no longer needed. */ pthread_mutex_t mut; int iRefCount; /* reference counter (0 = unused) */ - sbool bAlreadyFreed; /* aid to help detect a well-hidden bad bug -- TODO: remove when no longer needed */ sbool bParseSuccess; /* set to reflect state of last executed higher level parser */ short iSeverity; /* the severity 0..7 */ short iFacility; /* Facility code 0 .. 23*/ @@ -88,7 +86,8 @@ struct msg { char *pszTIMESTAMP3339; /* TIMESTAMP as RFC3339 formatted string (32 charcters at most) */ char *pszTIMESTAMP_MySQL;/* TIMESTAMP as MySQL formatted string (always 14 charcters) */ char *pszTIMESTAMP_PgSQL;/* TIMESTAMP as PgSQL formatted string (always 21 characters) */ - cstr_t *pCSStrucData; /* STRUCTURED-DATA */ + uchar *pszStrucData; /* STRUCTURED-DATA */ + uint16_t lenStrucData; /* (cached) length of STRUCTURED-DATA */ cstr_t *pCSAPPNAME; /* APP-NAME */ cstr_t *pCSPROCID; /* PROCID */ cstr_t *pCSMSGID; /* MSGID */ @@ -110,6 +109,7 @@ struct msg { struct syslogTime tRcvdAt;/* time the message entered this program */ struct syslogTime tTIMESTAMP;/* (parsed) value of the timestamp */ struct json_object *json; + struct json_object *localvars; /* some fixed-size buffers to save malloc()/free() for frequently used fields (from the default templates) */ uchar szRawMsg[CONF_RAWMSG_BUFSIZE]; /* most messages are small, and these are stored here (without malloc/free!) */ uchar szHOSTNAME[CONF_HOSTNAME_BUFSIZE]; @@ -127,7 +127,8 @@ struct msg { char pszRcvdAt_SecFrac[7]; /* same as above. Both are fractional seconds for their respective timestamp */ char pszTIMESTAMP_Unix[12]; /* almost as small as a pointer! */ char pszRcvdAt_Unix[12]; - uchar *pszUUID; /* The message's UUID */ + char dfltTZ[8]; /* 7 chars max, less overhead than ptr! */ + uchar *pszUUID; /* The message's UUID */ }; @@ -144,6 +145,9 @@ struct msg { #define NEEDS_ACLCHK_U 0x080 /* check UDP ACLs after DNS resolution has been done in main queue consumer */ #define NO_PRI_IN_RAW 0x100 /* rawmsg does not include a PRI (Solaris!), but PRI is already set correctly in the msg object */ +/* (syslog) protocol types */ +#define MSG_LEGACY_PROTOCOL 0 +#define MSG_RFC5424_PROTOCOL 1 /* function prototypes */ @@ -157,41 +161,40 @@ msg_t* MsgDup(msg_t* pOld); msg_t *MsgAddRef(msg_t *pM); void setProtocolVersion(msg_t *pM, int iNewVersion); void MsgSetInputName(msg_t *pMsg, prop_t*); -rsRetVal MsgSetAPPNAME(msg_t *pMsg, char* pszAPPNAME); -rsRetVal MsgSetPROCID(msg_t *pMsg, char* pszPROCID); -rsRetVal MsgSetMSGID(msg_t *pMsg, char* pszMSGID); +void MsgSetDfltTZ(msg_t *pThis, char *tz); +rsRetVal MsgSetAPPNAME(msg_t *pMsg, const char* pszAPPNAME); +rsRetVal MsgSetPROCID(msg_t *pMsg, const char* pszPROCID); +rsRetVal MsgSetMSGID(msg_t *pMsg, const char* pszMSGID); void MsgSetParseSuccess(msg_t *pMsg, int bSuccess); -void MsgSetTAG(msg_t *pMsg, uchar* pszBuf, size_t lenBuf); +void MsgSetTAG(msg_t *pMsg, const uchar* pszBuf, const size_t lenBuf); void MsgSetRuleset(msg_t *pMsg, ruleset_t*); rsRetVal MsgSetFlowControlType(msg_t *pMsg, flowControl_t eFlowCtl); -rsRetVal MsgSetStructuredData(msg_t *pMsg, char* pszStrucData); +rsRetVal MsgSetStructuredData(msg_t *const pMsg, const char* pszStrucData); +rsRetVal MsgAddToStructuredData(msg_t *pMsg, uchar *toadd, rs_size_t len); +void MsgGetStructuredData(msg_t *pM, uchar **pBuf, rs_size_t *len); rsRetVal msgSetFromSockinfo(msg_t *pThis, struct sockaddr_storage *sa); void MsgSetRcvFrom(msg_t *pMsg, prop_t*); -void MsgSetRcvFromStr(msg_t *pMsg, uchar* pszRcvFrom, int, prop_t **); +void MsgSetRcvFromStr(msg_t *const pMsg, const uchar* pszRcvFrom, const int, prop_t **); rsRetVal MsgSetRcvFromIP(msg_t *pMsg, prop_t*); -rsRetVal MsgSetRcvFromIPStr(msg_t *pThis, uchar *psz, int len, prop_t **ppProp); -void MsgSetHOSTNAME(msg_t *pMsg, uchar* pszHOSTNAME, int lenHOSTNAME); +rsRetVal MsgSetRcvFromIPStr(msg_t *const pThis, const uchar *psz, const int len, prop_t **ppProp); +void MsgSetHOSTNAME(msg_t *pMsg, const uchar* pszHOSTNAME, const int lenHOSTNAME); rsRetVal MsgSetAfterPRIOffs(msg_t *pMsg, short offs); void MsgSetMSGoffs(msg_t *pMsg, short offs); void MsgSetRawMsgWOSize(msg_t *pMsg, char* pszRawMsg); -void MsgSetRawMsg(msg_t *pMsg, char* pszRawMsg, size_t lenMsg); -rsRetVal MsgReplaceMSG(msg_t *pThis, uchar* pszMSG, int lenMSG); -uchar *MsgGetProp(msg_t *pMsg, struct templateEntry *pTpe, - propid_t propid, es_str_t *propName, +void MsgSetRawMsg(msg_t *pMsg, const char* pszRawMsg, size_t lenMsg); +rsRetVal MsgReplaceMSG(msg_t *pThis, const uchar* pszMSG, int lenMSG); +uchar *MsgGetProp(msg_t *pMsg, struct templateEntry *pTpe, msgPropDescr_t *pProp, rs_size_t *pPropLen, unsigned short *pbMustBeFreed, struct syslogTime *ttNow); -rsRetVal msgGetMsgVar(msg_t *pThis, cstr_t *pstrPropName, var_t **ppVar); -es_str_t* msgGetMsgVarNew(msg_t *pThis, uchar *name); uchar *getRcvFrom(msg_t *pM); void getTAG(msg_t *pM, uchar **ppBuf, int *piLen); char *getTimeReported(msg_t *pM, enum tplFormatTypes eFmt); char *getPRI(msg_t *pMsg); void getRawMsg(msg_t *pM, uchar **pBuf, int *piLen); -rsRetVal msgGetCEEVar(msg_t *pThis, cstr_t *propName, var_t **ppVar); -es_str_t* msgGetCEEVarNew(msg_t *pMsg, char *name); rsRetVal msgAddJSON(msg_t *pM, uchar *name, struct json_object *json); -rsRetVal getCEEPropVal(msg_t *pM, es_str_t *propName, uchar **pRes, rs_size_t *buflen, unsigned short *pbMustBeFreed); rsRetVal MsgGetSeverity(msg_t *pThis, int *piSeverity); rsRetVal MsgDeserialize(msg_t *pMsg, strm_t *pStrm); +rsRetVal MsgSetPropsViaJSON(msg_t *__restrict__ const pMsg, const uchar *__restrict__ const json); +const uchar* msgGetJSONMESG(msg_t *__restrict__ const pMsg); /* TODO: remove these five (so far used in action.c) */ uchar *getMSG(msg_t *pM); @@ -205,18 +208,29 @@ char *getHOSTNAME(msg_t *pM); int getHOSTNAMELen(msg_t *pM); uchar *getProgramName(msg_t *pM, sbool bLockMutex); uchar *getRcvFrom(msg_t *pM); -rsRetVal propNameToID(cstr_t *pCSPropName, propid_t *pPropID); +rsRetVal propNameToID(uchar *pName, propid_t *pPropID); uchar *propIDToName(propid_t propID); -rsRetVal msgGetCEEPropJSON(msg_t *pM, es_str_t *propName, struct json_object **pjson); +rsRetVal msgGetJSONPropJSON(msg_t *pMsg, msgPropDescr_t *pProp, struct json_object **pjson); +rsRetVal getJSONPropVal(msg_t *pMsg, msgPropDescr_t *pProp, uchar **pRes, rs_size_t *buflen, unsigned short *pbMustBeFreed); rsRetVal msgSetJSONFromVar(msg_t *pMsg, uchar *varname, struct var *var); rsRetVal msgDelJSON(msg_t *pMsg, uchar *varname); -rsRetVal jsonFind(msg_t *pM, es_str_t *propName, struct json_object **jsonres); +rsRetVal jsonFind(struct json_object *jroot, msgPropDescr_t *pProp, struct json_object **jsonres); + +rsRetVal msgPropDescrFill(msgPropDescr_t *pProp, uchar *name, int nameLen); +void msgPropDescrDestruct(msgPropDescr_t *pProp); -static inline rsRetVal -msgUnsetJSON(msg_t *pMsg, uchar *varname) { - return msgDelJSON(pMsg, varname+1); +static inline int +msgGetProtocolVersion(msg_t *pM) +{ + return(pM->iProtocolVersion); } +/* returns non-zero if the message has structured data, 0 otherwise */ +static inline sbool +MsgHasStructuredData(msg_t *pM) +{ + return (pM->pszStrucData == NULL) ? 0 : 1; +} /* ------------------------------ some inline functions ------------------------------ */ diff --git a/runtime/net.c b/runtime/net.c index b291213..45bc621 100644 --- a/runtime/net.c +++ b/runtime/net.c @@ -175,7 +175,7 @@ AddPermittedPeerWildcard(permittedPeers_t *pPeer, uchar* pszStr, size_t lenStr) assert(pPeer != NULL); assert(pszStr != NULL); - CHKmalloc(pNew = calloc(1, sizeof(permittedPeers_t))); + CHKmalloc(pNew = calloc(1, sizeof(*pNew))); if(lenStr == 0) { /* empty domain components are permitted */ pNew->wildcardType = PEER_WILDCARD_EMPTY_COMPONENT; @@ -232,6 +232,7 @@ finalize_it: /* enqueue the element */ if(pPeer->pWildcardRoot == NULL) { pPeer->pWildcardRoot = pNew; + pPeer->pWildcardLast = pNew; } else { pPeer->pWildcardLast->pNext = pNew; } @@ -712,8 +713,10 @@ static rsRetVal AddAllowedSender(struct AllowedSenders **ppRoot, struct AllowedS memcpy(allowIP.addr.NetAddr, res->ai_addr, res->ai_addrlen); if((iRet = AddAllowedSenderEntry(ppRoot, ppLast, &allowIP, iSignificantBits)) - != RS_RET_OK) + != RS_RET_OK) { + free(allowIP.addr.NetAddr); FINALIZE; + } break; case AF_INET6: /* IPv6 - but need to check if it is a v6-mapped IPv4 */ if(IN6_IS_ADDR_V4MAPPED (&SIN6(res->ai_addr)->sin6_addr)) { @@ -721,7 +724,7 @@ static rsRetVal AddAllowedSender(struct AllowedSenders **ppRoot, struct AllowedS iSignificantBits = 32; allowIP.flags = 0; - if((allowIP.addr.NetAddr = MALLOC(sizeof(struct sockaddr_in))) + if((allowIP.addr.NetAddr = (struct sockaddr *) MALLOC(sizeof(struct sockaddr_in))) == NULL) { ABORT_FINALIZE(RS_RET_OUT_OF_MEMORY); } @@ -736,8 +739,10 @@ static rsRetVal AddAllowedSender(struct AllowedSenders **ppRoot, struct AllowedS if((iRet = AddAllowedSenderEntry(ppRoot, ppLast, &allowIP, iSignificantBits)) - != RS_RET_OK) + != RS_RET_OK) { + free(allowIP.addr.NetAddr); FINALIZE; + } } else { /* finally add IPv6 */ @@ -750,8 +755,10 @@ static rsRetVal AddAllowedSender(struct AllowedSenders **ppRoot, struct AllowedS if((iRet = AddAllowedSenderEntry(ppRoot, ppLast, &allowIP, iSignificantBits)) - != RS_RET_OK) + != RS_RET_OK) { + free(allowIP.addr.NetAddr); FINALIZE; + } } break; } @@ -886,6 +893,7 @@ rsRetVal addAllowedSenderLine(char* pName, uchar** ppRestOfConfLine) errmsg.LogError(0, iRet, "Error %d adding allowed sender entry " "- terminating, nothing more will be added.", iRet); rsParsDestruct(pPars); + free(uIP); return(iRet); } } @@ -1082,22 +1090,18 @@ void debugListenInfo(int fd, char *type) { char *szFamily; int port; - struct sockaddr sa; - struct sockaddr_in *ipv4; - struct sockaddr_in6 *ipv6; + struct sockaddr_storage sa; socklen_t saLen = sizeof(sa); - if(getsockname(fd, &sa, &saLen) == 0) { - switch(sa.sa_family) { + if(getsockname(fd, (struct sockaddr *) &sa, &saLen) == 0) { + switch(sa.ss_family) { case PF_INET: szFamily = "IPv4"; - ipv4 = (struct sockaddr_in*)(void*) &sa; - port = ntohs(ipv4->sin_port); + port = ntohs(((struct sockaddr_in *) &sa)->sin_port); break; case PF_INET6: szFamily = "IPv6"; - ipv6 = (struct sockaddr_in6*)(void*) &sa; - port = ntohs(ipv6->sin6_port); + port = ntohs(((struct sockaddr_in6 *) &sa)->sin6_port); break; default: szFamily = "other"; @@ -1151,8 +1155,11 @@ getLocalHostname(uchar **ppName) buf_len = 128; /* Initial guess */ CHKmalloc(buf = MALLOC(buf_len)); } else { + uchar *p; + buf_len += buf_len; - CHKmalloc(buf = realloc (buf, buf_len)); + CHKmalloc(p = realloc (buf, buf_len)); + buf = p; } } while((gethostname((char*)buf, buf_len) == 0 && !memchr (buf, '\0', buf_len)) || errno == ENAMETOOLONG); @@ -1188,12 +1195,16 @@ void closeUDPListenSockets(int *pSockArr) * hostname and/or pszPort may be NULL, but not both! * bIsServer indicates if a server socket should be created * 1 - server, 0 - client + * param rcvbuf indicates desired rcvbuf size; 0 means OS default */ -int *create_udp_socket(uchar *hostname, uchar *pszPort, int bIsServer) +int *create_udp_socket(uchar *hostname, uchar *pszPort, int bIsServer, int rcvbuf) { struct addrinfo hints, *res, *r; int error, maxs, *s, *socks, on = 1; int sockflags; + int actrcvbuf; + socklen_t optlen; + char errStr[1024]; assert(!((pszPort == NULL) && (hostname == NULL))); memset(&hints, 0, sizeof(hints)); @@ -1296,6 +1307,35 @@ int *create_udp_socket(uchar *hostname, uchar *pszPort, int bIsServer) continue; } + if(rcvbuf != 0) { +# if defined(SO_RCVBUFFORCE) + if(setsockopt(*s, SOL_SOCKET, SO_RCVBUFFORCE, &rcvbuf, sizeof(rcvbuf)) < 0) +# endif + { + /* if we fail, try to do it the regular way. Experiments show that at + * least some platforms do not return an error here, but silently set + * it to the max permitted value. So we do our error check a bit + * differently by querying the size below. + */ + setsockopt(*s, SOL_SOCKET, SO_RCVBUF, &rcvbuf, sizeof(rcvbuf)); + } + } + + if(Debug || rcvbuf != 0) { + optlen = sizeof(actrcvbuf); + if(getsockopt(*s, SOL_SOCKET, SO_RCVBUF, &actrcvbuf, &optlen) == 0) { + dbgprintf("socket %d, actual os socket rcvbuf size %d\n", *s, actrcvbuf); + if(rcvbuf != 0 && actrcvbuf/2 != rcvbuf) { + errmsg.LogError(errno, NO_ERRCODE, + "cannot set os socket rcvbuf size %d for socket %d, value now is %d", + rcvbuf, *s, actrcvbuf/2); + } + } else { + dbgprintf("could not obtain os socket rcvbuf size for socket %d: %s\n", + *s, rs_strerror_r(errno, errStr, sizeof(errStr))); + } + } + if(bIsServer) { /* rgerhards, 2007-06-22: if we run on a kernel that does not support * the IPV6_V6ONLY socket option, we need to use a work-around. On such diff --git a/runtime/net.h b/runtime/net.h index b196116..d7a7b51 100644 --- a/runtime/net.h +++ b/runtime/net.h @@ -137,7 +137,7 @@ BEGINinterface(net) /* name must also be changed in ENDinterface macro! */ void (*PrintAllowedSenders)(int iListToPrint); void (*clearAllowedSenders)(uchar*); void (*debugListenInfo)(int fd, char *type); - int *(*create_udp_socket)(uchar *hostname, uchar *LogPort, int bIsServer); + int *(*create_udp_socket)(uchar *hostname, uchar *LogPort, int bIsServer, int rcvbuf); void (*closeUDPListenSockets)(int *finet); int (*isAllowedSender)(uchar *pszType, struct sockaddr *pFrom, const char *pszFromHost); /* deprecated! */ rsRetVal (*getLocalHostname)(uchar**); diff --git a/runtime/nsd_gtls.c b/runtime/nsd_gtls.c index 6ef4feb..b2ecabd 100644 --- a/runtime/nsd_gtls.c +++ b/runtime/nsd_gtls.c @@ -2,7 +2,7 @@ * * An implementation of the nsd interface for GnuTLS. * - * Copyright (C) 2007, 2008 Rainer Gerhards and Adiscon GmbH. + * Copyright (C) 2007-2014 Rainer Gerhards and Adiscon GmbH. * * This file is part of the rsyslog runtime library. * @@ -50,6 +50,7 @@ #include "nsd_ptcp.h" #include "nsdsel_gtls.h" #include "nsd_gtls.h" +#include "unicode-helper.h" /* things to move to some better place/functionality - TODO */ #define CRLFILE "crl.pem" @@ -120,14 +121,13 @@ readFile(uchar *pszFile, gnutls_datum_t *pBuf) if((fd = open((char*)pszFile, O_RDONLY)) == -1) { errmsg.LogError(0, RS_RET_FILE_NOT_FOUND, "can not read file '%s'", pszFile); ABORT_FINALIZE(RS_RET_FILE_NOT_FOUND); - } if(fstat(fd, &stat_st) == -1) { errmsg.LogError(0, RS_RET_FILE_NO_STAT, "can not stat file '%s'", pszFile); ABORT_FINALIZE(RS_RET_FILE_NO_STAT); } - + /* 1MB limit */ if(stat_st.st_size > 1024 * 1024) { errmsg.LogError(0, RS_RET_FILE_TOO_LARGE, "file '%s' too large, max 1MB", pszFile); @@ -141,9 +141,9 @@ readFile(uchar *pszFile, gnutls_datum_t *pBuf) ABORT_FINALIZE(RS_RET_IO_ERROR); } - close(fd); - finalize_it: + if(fd != -1) + close(fd); if(iRet != RS_RET_OK) { if(pBuf->data != NULL) { free(pBuf->data); @@ -288,12 +288,12 @@ gtlsGetCertInfo(nsd_gtls_t *pThis, cstr_t **ppStr) expiration_time = gnutls_x509_crt_get_expiration_time(cert); activation_time = gnutls_x509_crt_get_activation_time(cert); - ctime_r(&activation_time, szBuf); - szBuf[strlen(szBuf) - 1] = '\0'; /* strip linefeed */ - CHKiRet(rsCStrAppendStrf(pStr, (uchar*)"Certificate 1 info: " + ctime_r(&activation_time, (char*)szBuf); + szBuf[ustrlen(szBuf) - 1] = '\0'; /* strip linefeed */ + CHKiRet(rsCStrAppendStrf(pStr, "Certificate 1 info: " "certificate valid from %s ", szBuf)); - ctime_r(&expiration_time, szBuf); - szBuf[strlen(szBuf) - 1] = '\0'; /* strip linefeed */ + ctime_r(&expiration_time, (char*)szBuf); + szBuf[ustrlen(szBuf) - 1] = '\0'; /* strip linefeed */ CHKiRet(rsCStrAppendStrf(pStr, "to %s; ", szBuf)); /* Extract some of the public key algorithm's parameters */ @@ -303,20 +303,20 @@ gtlsGetCertInfo(nsd_gtls_t *pThis, cstr_t **ppStr) /* names */ tmp = szBufLen; - if(gnutls_x509_crt_get_dn(cert, szBuf, &tmp) + if(gnutls_x509_crt_get_dn(cert, (char*)szBuf, &tmp) == GNUTLS_E_SHORT_MEMORY_BUFFER) { szBufLen = tmp; szBuf = malloc(tmp); - gnutls_x509_crt_get_dn(cert, szBuf, &tmp); + gnutls_x509_crt_get_dn(cert, (char*)szBuf, &tmp); } CHKiRet(rsCStrAppendStrf(pStr, "DN: %s; ", szBuf)); tmp = szBufLen; - if(gnutls_x509_crt_get_issuer_dn(cert, szBuf, &tmp) + if(gnutls_x509_crt_get_issuer_dn(cert, (char*)szBuf, &tmp) == GNUTLS_E_SHORT_MEMORY_BUFFER) { szBufLen = tmp; szBuf = realloc((szBuf == szBufA) ? NULL : szBuf, tmp); - gnutls_x509_crt_get_issuer_dn(cert, szBuf, &tmp); + gnutls_x509_crt_get_issuer_dn(cert, (char*)szBuf, &tmp); } CHKiRet(rsCStrAppendStrf(pStr, "Issuer DN: %s; ", szBuf)); @@ -547,10 +547,20 @@ gtlsAddOurCert(void) keyFile = glbl.GetDfltNetstrmDrvrKeyFile(); dbgprintf("GTLS certificate file: '%s'\n", certFile); dbgprintf("GTLS key file: '%s'\n", keyFile); + if(certFile == NULL) { + errmsg.LogError(0, RS_RET_CERT_MISSING, "error: certificate file is not set, cannot " + "continue"); + ABORT_FINALIZE(RS_RET_CERT_MISSING); + } + if(keyFile == NULL) { + errmsg.LogError(0, RS_RET_CERTKEY_MISSING, "error: key file is not set, cannot " + "continue"); + ABORT_FINALIZE(RS_RET_CERTKEY_MISSING); + } CHKgnutls(gnutls_certificate_set_x509_key_file(xcred, (char*)certFile, (char*)keyFile, GNUTLS_X509_FMT_PEM)); finalize_it: - if(iRet != RS_RET_OK) { + if(iRet != RS_RET_OK && iRet != RS_RET_CERT_MISSING && iRet != RS_RET_CERTKEY_MISSING) { pGnuErr = gtlsStrerror(gnuRet); errno = 0; errmsg.LogError(0, iRet, "error adding our certificate. GnuTLS error %d, message: '%s', " @@ -580,6 +590,11 @@ gtlsGlblInit(void) /* sets the trusted cas file */ cafile = glbl.GetDfltNetstrmDrvrCAF(); + if(cafile == NULL) { + errmsg.LogError(0, RS_RET_CA_CERT_MISSING, "error: ca certificate is not set, cannot " + "continue"); + ABORT_FINALIZE(RS_RET_CA_CERT_MISSING); + } dbgprintf("GTLS CA file: '%s'\n", cafile); gnuRet = gnutls_certificate_set_x509_trust_file(xcred, (char*)cafile, GNUTLS_X509_FMT_PEM); if(gnuRet < 0) { @@ -1569,7 +1584,9 @@ finalize_it: static rsRetVal EnableKeepAlive(nsd_t *pNsd) { - return nsd_ptcp.EnableKeepAlive(pNsd); + nsd_gtls_t *pThis = (nsd_gtls_t*) pNsd; + ISOBJ_TYPE_assert(pThis, nsd_gtls); + return nsd_ptcp.EnableKeepAlive(pThis->pTcp); } diff --git a/runtime/nspoll.c b/runtime/nspoll.c index a936b25..94fba03 100644 --- a/runtime/nspoll.c +++ b/runtime/nspoll.c @@ -6,25 +6,21 @@ * * Work on this module begun 2009-11-18 by Rainer Gerhards. * - * Copyright 2009 Rainer Gerhards and Adiscon GmbH. + * Copyright 2009-2014 Rainer Gerhards and Adiscon GmbH. * - * This file is part of the rsyslog runtime library. - * - * The rsyslog runtime library is free software: you can redistribute it and/or modify - * it under the terms of the GNU Lesser General Public License as published by - * the Free Software Foundation, either version 3 of the License, or - * (at your option) any later version. - * - * The rsyslog runtime library is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU Lesser General Public License for more details. - * - * You should have received a copy of the GNU Lesser General Public License - * along with the rsyslog runtime library. If not, see <http://www.gnu.org/licenses/>. - * - * A copy of the GPL can be found in the file "COPYING" in this distribution. - * A copy of the LGPL can be found in the file "COPYING.LESSER" in this distribution. + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * -or- + * see COPYING.ASL20 in the source distribution + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. */ #include "config.h" @@ -106,6 +102,7 @@ CODESTARTobjDestruct(nspoll) * a driver name string as load indicator (because we also need that string * to release the driver */ + free(pThis->pBaseDrvrName); if(pThis->pDrvrName != NULL) { obj.ReleaseObj(__FILE__, pThis->pDrvrName+2, DONT_LOAD_LIB, (void*) &pThis->Drvr); free(pThis->pDrvrName); @@ -138,6 +135,29 @@ Wait(nspoll_t *pThis, int timeout, int *numEntries, nsd_epworkset_t workset[]) { } +/* set the base driver name. If the driver name + * is set to NULL, the previously set name is deleted but + * no name set again (which results in the system default being + * used)-- rgerhards, 2008-05-05 + */ +static rsRetVal +SetDrvrName(nspoll_t *pThis, uchar *pszName) +{ + DEFiRet; + ISOBJ_TYPE_assert(pThis, netstrms); + if(pThis->pBaseDrvrName != NULL) { + free(pThis->pBaseDrvrName); + pThis->pBaseDrvrName = NULL; + } + + if(pszName != NULL) { + CHKmalloc(pThis->pBaseDrvrName = (uchar*) strdup((char*) pszName)); + } +finalize_it: + RETiRet; +} + + /* semantics like the epoll_ctl() function, does the same thing. * rgerhards, 2009-11-18 */ @@ -164,6 +184,7 @@ CODESTARTobjQueryInterface(nspoll) */ pIf->Construct = nspollConstruct; pIf->ConstructFinalize = ConstructFinalize; + pIf->SetDrvrName = SetDrvrName; pIf->Destruct = nspollDestruct; pIf->Wait = Wait; pIf->Ctl = Ctl; diff --git a/runtime/nspoll.h b/runtime/nspoll.h index 037f6c3..a9e9eb0 100644 --- a/runtime/nspoll.h +++ b/runtime/nspoll.h @@ -4,21 +4,19 @@ * * This file is part of the rsyslog runtime library. * - * The rsyslog runtime library is free software: you can redistribute it and/or modify - * it under the terms of the GNU Lesser General Public License as published by - * the Free Software Foundation, either version 3 of the License, or - * (at your option) any later version. - * - * The rsyslog runtime library is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU Lesser General Public License for more details. - * - * You should have received a copy of the GNU Lesser General Public License - * along with the rsyslog runtime library. If not, see <http://www.gnu.org/licenses/>. - * - * A copy of the GPL can be found in the file "COPYING" in this distribution. - * A copy of the LGPL can be found in the file "COPYING.LESSER" in this distribution. + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * -or- + * see COPYING.ASL20 in the source distribution + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. */ #ifndef INCLUDED_NSPOLL_H @@ -53,8 +51,10 @@ BEGINinterface(nspoll) /* name must also be changed in ENDinterface macro! */ rsRetVal (*Wait)(nspoll_t *pNsdpoll, int timeout, int *numEntries, nsd_epworkset_t workset[]); rsRetVal (*Ctl)(nspoll_t *pNsdpoll, netstrm_t *pStrm, int id, void *pUsr, int mode, int op); rsRetVal (*IsEPollSupported)(void); /* static method */ + /* v3 - 2013-09-17 by rgerhards */ + rsRetVal (*SetDrvrName)(nspoll_t *pThis, uchar *name); ENDinterface(nspoll) -#define nspollCURR_IF_VERSION 2 /* increment whenever you change the interface structure! */ +#define nspollCURR_IF_VERSION 3 /* increment whenever you change the interface structure! */ /* interface change in v2 is that wait supports multiple return objects */ /* prototypes */ diff --git a/runtime/nssel.c b/runtime/nssel.c index 751dae9..533d92b 100644 --- a/runtime/nssel.c +++ b/runtime/nssel.c @@ -9,7 +9,7 @@ * * Work on this module begun 2008-04-22 by Rainer Gerhards. * - * Copyright 2008-2012 Adiscon GmbH. + * Copyright 2008-2014 Adiscon GmbH. * * This file is part of the rsyslog runtime library. * @@ -107,6 +107,7 @@ CODESTARTobjDestruct(nssel) * a driver name string as load indicator (because we also need that string * to release the driver */ + free(pThis->pBaseDrvrName); if(pThis->pDrvrName != NULL) { obj.ReleaseObj(__FILE__, pThis->pDrvrName+2, DONT_LOAD_LIB, (void*) &pThis->Drvr); free(pThis->pDrvrName); @@ -127,6 +128,29 @@ finalize_it: } +/* set the base driver name. If the driver name + * is set to NULL, the previously set name is deleted but + * no name set again (which results in the system default being + * used)-- rgerhards, 2008-05-05 + */ +static rsRetVal +SetDrvrName(nssel_t *pThis, uchar *pszName) +{ + DEFiRet; + ISOBJ_TYPE_assert(pThis, netstrms); + if(pThis->pBaseDrvrName != NULL) { + free(pThis->pBaseDrvrName); + pThis->pBaseDrvrName = NULL; + } + + if(pszName != NULL) { + CHKmalloc(pThis->pBaseDrvrName = (uchar*) strdup((char*) pszName)); + } +finalize_it: + RETiRet; +} + + /* Add a stream object to the current select() set. * Note that a single stream may have multiple "sockets" if * it is a listener. If so, all of them are begin added. @@ -195,6 +219,7 @@ CODESTARTobjQueryInterface(nssel) pIf->Construct = nsselConstruct; pIf->ConstructFinalize = ConstructFinalize; pIf->Destruct = nsselDestruct; + pIf->SetDrvrName = SetDrvrName; pIf->Add = Add; pIf->Wait = Wait; pIf->IsReady = IsReady; diff --git a/runtime/nssel.h b/runtime/nssel.h index d7f4fcd..6131d9b 100644 --- a/runtime/nssel.h +++ b/runtime/nssel.h @@ -42,8 +42,10 @@ BEGINinterface(nssel) /* name must also be changed in ENDinterface macro! */ rsRetVal (*Add)(nssel_t *pThis, netstrm_t *pStrm, nsdsel_waitOp_t waitOp); rsRetVal (*Wait)(nssel_t *pThis, int *pNumReady); rsRetVal (*IsReady)(nssel_t *pThis, netstrm_t *pStrm, nsdsel_waitOp_t waitOp, int *pbIsReady, int *piNumReady); + /* v2 - 2013-09-17 by rgerhards */ + rsRetVal (*SetDrvrName)(nssel_t *pThis, uchar *name); ENDinterface(nssel) -#define nsselCURR_IF_VERSION 1 /* increment whenever you change the interface structure! */ +#define nsselCURR_IF_VERSION 2 /* increment whenever you change the interface structure! */ /* prototypes */ PROTOTYPEObj(nssel); diff --git a/runtime/parser.c b/runtime/parser.c index 74b28f4..1baa502 100644 --- a/runtime/parser.c +++ b/runtime/parser.c @@ -57,13 +57,9 @@ DEFobjCurrIf(ruleset) /* static data */ -/* config data */ -static uchar cCCEscapeChar = '#';/* character to be used to start an escape sequence for control chars */ -static int bEscapeCCOnRcv = 1; /* escape control characters on reception: 0 - no, 1 - yes */ -static int bSpaceLFOnRcv = 0; /* replace newlines with spaces on reception: 0 - no, 1 - yes */ -static int bEscape8BitChars = 0; /* escape characters > 127 on reception: 0 - no, 1 - yes */ -static int bEscapeTab = 1; /* escape tab control character when doing CC escapes: 0 - no, 1 - yes */ -static int bDropTrailingLF = 1; /* drop trailing LF's on reception? */ +static char hexdigit[16] = + {'0', '1', '2', '3', '4', '5', '6', '7', '8', + '9', 'A', 'B', 'C', 'D', 'E', 'F' }; /* This is the list of all parsers known to us. * This is also used to unload all modules on shutdown. @@ -138,7 +134,7 @@ AddParserToList(parserList_t **ppListRoot, parser_t *pParser) /* add at tail */ pTail->pNext = pThis; } - +DBGPRINTF("DDDDD: added parser '%s' to list %p\n", pParser->pName, ppListRoot); finalize_it: RETiRet; } @@ -195,6 +191,68 @@ finalize_it: } +/* set the parser name - string is copied over, call can continue to use it, + * but must free it if desired. + */ +static rsRetVal +SetName(parser_t *pThis, uchar *name) +{ + DEFiRet; + + ISOBJ_TYPE_assert(pThis, parser); + assert(name != NULL); + + if(pThis->pName != NULL) { + free(pThis->pName); + pThis->pName = NULL; + } + + CHKmalloc(pThis->pName = ustrdup(name)); + +finalize_it: + RETiRet; +} + + +/* set a pointer to "our" module. Note that no module + * pointer must already be set. + */ +static rsRetVal +SetModPtr(parser_t *pThis, modInfo_t *pMod) +{ + ISOBJ_TYPE_assert(pThis, parser); + assert(pMod != NULL); + assert(pThis->pModule == NULL); + pThis->pModule = pMod; + return RS_RET_OK; +} + + +/* Specify if we should do standard message sanitazion before we pass the data + * down to the parser. + */ +static rsRetVal +SetDoSanitazion(parser_t *pThis, int bDoIt) +{ + ISOBJ_TYPE_assert(pThis, parser); + pThis->bDoSanitazion = bDoIt; + return RS_RET_OK; +} + + +/* Specify if we should do standard PRI parsing before we pass the data + * down to the parser module. + */ +static rsRetVal +SetDoPRIParsing(parser_t *pThis, int bDoIt) +{ + ISOBJ_TYPE_assert(pThis, parser); + pThis->bDoPRIParsing = bDoIt; + return RS_RET_OK; +} + + + BEGINobjConstruct(parser) /* be sure to specify the object type also in END macro! */ ENDobjConstruct(parser) @@ -215,9 +273,46 @@ finalize_it: RETiRet; } + +/* construct a parser object via a pointer to the parser module + * and the name. This is a separate function because we need it + * in multiple spots inside the code. + */ +rsRetVal +parserConstructViaModAndName(modInfo_t *__restrict__ pMod, uchar *const __restrict__ pName, void *pInst) +{ + rsRetVal localRet; + parser_t *pParser; + DEFiRet; + + if(pInst == NULL && pMod->mod.pm.newParserInst != NULL) { + /* this happens for the default instance on ModLoad time */ + CHKiRet(pMod->mod.pm.newParserInst(NULL, &pInst)); + } + CHKiRet(parserConstruct(&pParser)); + /* check some features */ + localRet = pMod->isCompatibleWithFeature(sFEATUREAutomaticSanitazion); + if(localRet == RS_RET_OK){ + CHKiRet(SetDoSanitazion(pParser, RSTRUE)); + } + localRet = pMod->isCompatibleWithFeature(sFEATUREAutomaticPRIParsing); + if(localRet == RS_RET_OK){ + CHKiRet(SetDoPRIParsing(pParser, RSTRUE)); + } + + CHKiRet(SetName(pParser, pName)); + CHKiRet(SetModPtr(pParser, pMod)); + pParser->pInst = pInst; + CHKiRet(parserConstructFinalize(pParser)); +finalize_it: + RETiRet; +} BEGINobjDestruct(parser) /* be sure to specify the object type also in END and CODESTART macros! */ CODESTARTobjDestruct(parser) DBGPRINTF("destructing parser '%s'\n", pThis->pName); + if(pThis->pInst != NULL) { + pThis->pModule->mod.pm.freeParserInst(pThis->pInst); + } free(pThis->pName); ENDobjDestruct(parser) @@ -319,6 +414,7 @@ SanitizeMsg(msg_t *pMsg) size_t iDst; size_t iMaxLine; size_t maxDest; + uchar pc; sbool bUpdatedLen = RSFALSE; uchar szSanBuf[32*1024]; /* buffer used for sanitizing a string */ @@ -343,7 +439,7 @@ SanitizeMsg(msg_t *pMsg) * compatible to recent IETF developments, we allow the user to * turn on/off this handling. rgerhards, 2007-07-23 */ - if(bDropTrailingLF && pszMsg[lenMsg-1] == '\n') { + if(glbl.GetParserDropTrailingLFOnReception() && pszMsg[lenMsg-1] == '\n') { DBGPRINTF("dropped LF at very end of message (DropTrailingLF is set)\n"); lenMsg--; pszMsg[lenMsg] = '\0'; @@ -363,14 +459,15 @@ SanitizeMsg(msg_t *pMsg) int bNeedSanitize = 0; for(iSrc = 0 ; iSrc < lenMsg ; iSrc++) { if(pszMsg[iSrc] < 32) { - if(bSpaceLFOnRcv && pszMsg[iSrc] == '\n') + if(glbl.GetParserSpaceLFOnReceive() && pszMsg[iSrc] == '\n') { pszMsg[iSrc] = ' '; - else if(pszMsg[iSrc] == '\0' || bEscapeCCOnRcv) { + } else if(pszMsg[iSrc] == '\0' || glbl.GetParserEscapeControlCharactersOnReceive()) { bNeedSanitize = 1; - if (!bSpaceLFOnRcv) + if (!glbl.GetParserSpaceLFOnReceive()) { break; + } } - } else if(pszMsg[iSrc] > 127 && bEscape8BitChars) { + } else if(pszMsg[iSrc] > 127 && glbl.GetParserEscape8BitCharactersOnReceive()) { bNeedSanitize = 1; break; } @@ -387,6 +484,7 @@ SanitizeMsg(msg_t *pMsg) */ iMaxLine = glbl.GetMaxLine(); maxDest = lenMsg * 4; /* message can grow at most four-fold */ + if(maxDest > iMaxLine) maxDest = iMaxLine; /* but not more than the max size! */ if(maxDest < sizeof(szSanBuf)) @@ -399,28 +497,82 @@ SanitizeMsg(msg_t *pMsg) } iDst = iSrc; while(iSrc < lenMsg && iDst < maxDest - 3) { /* leave some space if last char must be escaped */ - if((pszMsg[iSrc] < 32) && (pszMsg[iSrc] != '\t' || bEscapeTab)) { + if((pszMsg[iSrc] < 32) && (pszMsg[iSrc] != '\t' || glbl.GetParserEscapeControlCharacterTab())) { /* note: \0 must always be escaped, the rest of the code currently * can not handle it! -- rgerhards, 2009-08-26 */ - if(pszMsg[iSrc] == '\0' || bEscapeCCOnRcv) { + if(pszMsg[iSrc] == '\0' || glbl.GetParserEscapeControlCharactersOnReceive()) { /* we are configured to escape control characters. Please note * that this most probably break non-western character sets like * Japanese, Korean or Chinese. rgerhards, 2007-07-17 */ - pDst[iDst++] = cCCEscapeChar; + if (glbl.GetParserEscapeControlCharactersCStyle()) { + pDst[iDst++] = '\\'; + + switch (pszMsg[iSrc]) { + case '\0': + pDst[iDst++] = '0'; + break; + case '\a': + pDst[iDst++] = 'a'; + break; + case '\b': + pDst[iDst++] = 'b'; + break; + case '\e': + pDst[iDst++] = 'e'; + break; + case '\f': + pDst[iDst++] = 'f'; + break; + case '\n': + pDst[iDst++] = 'n'; + break; + case '\r': + pDst[iDst++] = 'r'; + break; + case '\t': + pDst[iDst++] = 't'; + break; + case '\v': + pDst[iDst++] = 'v'; + break; + default: + pDst[iDst++] = 'x'; + + pc = pszMsg[iSrc]; + pDst[iDst++] = hexdigit[(pc & 0xF0) >> 4]; + pDst[iDst++] = hexdigit[pc & 0xF]; + + break; + } + + } else { + pDst[iDst++] = glbl.GetParserControlCharacterEscapePrefix(); + pDst[iDst++] = '0' + ((pszMsg[iSrc] & 0300) >> 6); + pDst[iDst++] = '0' + ((pszMsg[iSrc] & 0070) >> 3); + pDst[iDst++] = '0' + ((pszMsg[iSrc] & 0007)); + } + } + + } else if(pszMsg[iSrc] > 127 && glbl.GetParserEscape8BitCharactersOnReceive()) { + if (glbl.GetParserEscapeControlCharactersCStyle()) { + pDst[iDst++] = '\\'; + pDst[iDst++] = 'x'; + + pc = pszMsg[iSrc]; + pDst[iDst++] = hexdigit[(pc & 0xF0) >> 4]; + pDst[iDst++] = hexdigit[pc & 0xF]; + + } else { + /* In this case, we also do the conversion. Note that this most + * probably breaks European languages. -- rgerhards, 2010-01-27 + */ + pDst[iDst++] = glbl.GetParserControlCharacterEscapePrefix(); pDst[iDst++] = '0' + ((pszMsg[iSrc] & 0300) >> 6); pDst[iDst++] = '0' + ((pszMsg[iSrc] & 0070) >> 3); pDst[iDst++] = '0' + ((pszMsg[iSrc] & 0007)); } - } else if(pszMsg[iSrc] > 127 && bEscape8BitChars) { - /* In this case, we also do the conversion. Note that this most - * probably breaks European languages. -- rgerhards, 2010-01-27 - */ - pDst[iDst++] = cCCEscapeChar; - pDst[iDst++] = '0' + ((pszMsg[iSrc] & 0300) >> 6); - pDst[iDst++] = '0' + ((pszMsg[iSrc] & 0070) >> 3); - pDst[iDst++] = '0' + ((pszMsg[iSrc] & 0007)); } else { pDst[iDst++] = pszMsg[iSrc]; } @@ -468,11 +620,11 @@ ParsePRI(msg_t *pMsg) } if(*msg == '>') ++msg; - if(pri & ~(LOG_FACMASK|LOG_PRIMASK)) - pri = DEFUPRI; + if(pri > LOG_MAXPRI) + pri = LOG_PRI_INVLD; } - pMsg->iFacility = LOG_FAC(pri); - pMsg->iSeverity = LOG_PRI(pri); + pMsg->iFacility = pri2fac(pri); + pMsg->iSeverity = pri2sev(pri); MsgSetAfterPRIOffs(pMsg, msg - pMsg->pszRawMsg); } RETiRet; @@ -533,7 +685,10 @@ ParseMsg(msg_t *pMsg) } bIsSanitized = RSTRUE; } - localRet = pParser->pModule->mod.pm.parse(pMsg); + if(pParser->pModule->mod.pm.parse2 == NULL) + localRet = pParser->pModule->mod.pm.parse(pMsg); + else + localRet = pParser->pModule->mod.pm.parse2(pParser->pInst, pMsg); DBGPRINTF("Parser '%s' returned %d\n", pParser->pName, localRet); if(localRet != RS_RET_COULD_NOT_PARSE) break; @@ -561,68 +716,6 @@ ParseMsg(msg_t *pMsg) finalize_it: RETiRet; } - -/* set the parser name - string is copied over, call can continue to use it, - * but must free it if desired. - */ -static rsRetVal -SetName(parser_t *pThis, uchar *name) -{ - DEFiRet; - - ISOBJ_TYPE_assert(pThis, parser); - assert(name != NULL); - - if(pThis->pName != NULL) { - free(pThis->pName); - pThis->pName = NULL; - } - - CHKmalloc(pThis->pName = ustrdup(name)); - -finalize_it: - RETiRet; -} - - -/* set a pointer to "our" module. Note that no module - * pointer must already be set. - */ -static rsRetVal -SetModPtr(parser_t *pThis, modInfo_t *pMod) -{ - ISOBJ_TYPE_assert(pThis, parser); - assert(pMod != NULL); - assert(pThis->pModule == NULL); - pThis->pModule = pMod; - return RS_RET_OK; -} - - -/* Specify if we should do standard message sanitazion before we pass the data - * down to the parser. - */ -static rsRetVal -SetDoSanitazion(parser_t *pThis, int bDoIt) -{ - ISOBJ_TYPE_assert(pThis, parser); - pThis->bDoSanitazion = bDoIt; - return RS_RET_OK; -} - - -/* Specify if we should do standard PRI parsing before we pass the data - * down to the parser module. - */ -static rsRetVal -SetDoPRIParsing(parser_t *pThis, int bDoIt) -{ - ISOBJ_TYPE_assert(pThis, parser); - pThis->bDoPRIParsing = bDoIt; - return RS_RET_OK; -} - - /* queryInterface function-- rgerhards, 2009-11-03 */ BEGINobjQueryInterface(parser) @@ -653,24 +746,6 @@ CODESTARTobjQueryInterface(parser) finalize_it: ENDobjQueryInterface(parser) - - -/* Reset config variables to default values. - * rgerhards, 2007-07-17 - */ -static rsRetVal -resetConfigVariables(uchar __attribute__((unused)) *pp, void __attribute__((unused)) *pVal) -{ - cCCEscapeChar = '#'; - bEscapeCCOnRcv = 1; /* default is to escape control characters */ - bSpaceLFOnRcv = 0; - bEscape8BitChars = 0; /* default is to escape control characters */ - bEscapeTab = 1; /* default is to escape control characters */ - bDropTrailingLF = 1; /* default is to drop trailing LF's on reception */ - - return RS_RET_OK; -} - /* This destroys the master parserlist and all of its parser entries. MUST only be * done when the module is shut down. Parser modules are NOT unloaded, rsyslog * does that at a later stage for all dynamically loaded modules. @@ -714,15 +789,6 @@ BEGINObjClassInit(parser, 1, OBJ_IS_CORE_MODULE) /* class, version */ CHKiRet(objUse(datetime, CORE_COMPONENT)); CHKiRet(objUse(ruleset, CORE_COMPONENT)); - CHKiRet(regCfSysLineHdlr((uchar *)"controlcharacterescapeprefix", 0, eCmdHdlrGetChar, NULL, &cCCEscapeChar, NULL)); - CHKiRet(regCfSysLineHdlr((uchar *)"droptrailinglfonreception", 0, eCmdHdlrBinary, NULL, &bDropTrailingLF, NULL)); - CHKiRet(regCfSysLineHdlr((uchar *)"escapecontrolcharactersonreceive", 0, eCmdHdlrBinary, NULL, &bEscapeCCOnRcv, NULL)); - CHKiRet(regCfSysLineHdlr((uchar *)"spacelfonreceive", 0, eCmdHdlrBinary, NULL, &bSpaceLFOnRcv, NULL)); - CHKiRet(regCfSysLineHdlr((uchar *)"escape8bitcharactersonreceive", 0, eCmdHdlrBinary, NULL, &bEscape8BitChars, NULL)); - CHKiRet(regCfSysLineHdlr((uchar *)"escapecontrolcharactertab", 0, eCmdHdlrBinary, NULL, &bEscapeTab, NULL)); - CHKiRet(regCfSysLineHdlr((uchar *)"resetconfigvariables", 1, eCmdHdlrCustomHandler, resetConfigVariables, NULL, NULL)); - InitParserList(&pParsLstRoot); InitParserList(&pDfltParsLst); ENDObjClassInit(parser) - diff --git a/runtime/parser.h b/runtime/parser.h index 87a6269..bd8a2a5 100644 --- a/runtime/parser.h +++ b/runtime/parser.h @@ -21,7 +21,6 @@ #ifndef INCLUDED_PARSER_H #define INCLUDED_PARSER_H - /* we create a small helper object, a list of parsers, that we can use to * build a chain of them whereever this is needed (initially thought to be * used in ruleset.c as well as ourselvs). @@ -37,6 +36,7 @@ struct parser_s { BEGINobjInstance; /* Data to implement generic object - MUST be the first data element! */ uchar *pName; /* name of this parser */ modInfo_t *pModule; /* pointer to parser's module */ + void *pInst; /* instance data for the parser (v2+ module interface) */ sbool bDoSanitazion; /* do standard message sanitazion before calling parser? */ sbool bDoPRIParsing; /* do standard PRI parsing before calling parser? */ }; @@ -66,6 +66,7 @@ void printParserList(parserList_t *pList); /* prototypes */ PROTOTYPEObj(parser); +rsRetVal parserConstructViaModAndName(modInfo_t *pMod, uchar *const pName, void *parserInst); #endif /* #ifndef INCLUDED_PARSER_H */ diff --git a/runtime/prop.c b/runtime/prop.c index cb89fac..0686b40 100644 --- a/runtime/prop.c +++ b/runtime/prop.c @@ -73,7 +73,7 @@ ENDobjDestruct(prop) /* set string, we make our own private copy! This MUST only be called BEFORE * ConstructFinalize()! */ -static rsRetVal SetString(prop_t *pThis, uchar *psz, int len) +static rsRetVal SetString(prop_t *pThis, const uchar *psz, const int len) { DEFiRet; ISOBJ_TYPE_assert(pThis, prop); @@ -139,15 +139,24 @@ static rsRetVal AddRef(prop_t *pThis) /* this is a "do it all in one shot" function that creates a new property, * assigns the provided string to it and finalizes the property. Among the - * convenience, it is alos (very, very) slightly faster. + * convenience, it is also (very, very) slightly faster. * rgerhards, 2009-07-01 */ -static rsRetVal CreateStringProp(prop_t **ppThis, uchar* psz, int len) +static rsRetVal CreateStringProp(prop_t **ppThis, const uchar* psz, const int len) { + prop_t *pThis = NULL; DEFiRet; - propConstruct(ppThis); - SetString(*ppThis, psz, len); - propConstructFinalize(*ppThis); + + CHKiRet(propConstruct(&pThis)); + CHKiRet(SetString(pThis, psz, len)); + CHKiRet(propConstructFinalize(pThis)); + *ppThis = pThis; +finalize_it: + if(iRet != RS_RET_OK) { + if(pThis != NULL) + propDestruct(&pThis); + } + RETiRet; } @@ -161,7 +170,7 @@ static rsRetVal CreateStringProp(prop_t **ppThis, uchar* psz, int len) * existing property). * rgerhards, 2009-07-01 */ -rsRetVal CreateOrReuseStringProp(prop_t **ppThis, uchar *psz, int len) +rsRetVal CreateOrReuseStringProp(prop_t **ppThis, const uchar *psz, const int len) { uchar *pszPrev; int lenPrev; diff --git a/runtime/prop.h b/runtime/prop.h index c7564e6..38d801d 100644 --- a/runtime/prop.h +++ b/runtime/prop.h @@ -33,7 +33,7 @@ struct prop_s { uchar sz[CONF_PROP_BUFSIZE]; } szVal; int len; /* we use int intentionally, otherwise we may get some troubles... */ - DEF_ATOMIC_HELPER_MUT(mutRefCount); + DEF_ATOMIC_HELPER_MUT(mutRefCount) }; /* interfaces */ @@ -42,12 +42,12 @@ BEGINinterface(prop) /* name must also be changed in ENDinterface macro! */ rsRetVal (*Construct)(prop_t **ppThis); rsRetVal (*ConstructFinalize)(prop_t *pThis); rsRetVal (*Destruct)(prop_t **ppThis); - rsRetVal (*SetString)(prop_t *pThis, uchar* psz, int len); + rsRetVal (*SetString)(prop_t *pThis, const uchar* psz, const int len); rsRetVal (*GetString)(prop_t *pThis, uchar** ppsz, int *plen); int (*GetStringLen)(prop_t *pThis); rsRetVal (*AddRef)(prop_t *pThis); - rsRetVal (*CreateStringProp)(prop_t **ppThis, uchar* psz, int len); - rsRetVal (*CreateOrReuseStringProp)(prop_t **ppThis, uchar *psz, int len); + rsRetVal (*CreateStringProp)(prop_t **ppThis, const uchar* psz, const int len); + rsRetVal (*CreateOrReuseStringProp)(prop_t **ppThis, const uchar *psz, const int len); ENDinterface(prop) #define propCURR_IF_VERSION 1 /* increment whenever you change the interface structure! */ diff --git a/runtime/queue.c b/runtime/queue.c index 600b568..8d10064 100644 --- a/runtime/queue.c +++ b/runtime/queue.c @@ -12,7 +12,7 @@ * function names - this makes it really hard to read and does not provide much * benefit, at least I (now) think so... * - * Copyright 2008-2011 Rainer Gerhards and Adiscon GmbH. + * Copyright 2008-2013 Rainer Gerhards and Adiscon GmbH. * * This file is part of the rsyslog runtime library. * @@ -59,6 +59,7 @@ #include "datetime.h" #include "unicode-helper.h" #include "statsobj.h" +#include "parserif.h" #ifdef OS_SOLARIS # include <sched.h> @@ -72,6 +73,11 @@ DEFobjCurrIf(errmsg) DEFobjCurrIf(datetime) DEFobjCurrIf(statsobj) + +#ifdef ENABLE_IMDIAG +unsigned int iOverallQueueSize = 0; +#endif + /* forward-definitions */ static inline rsRetVal doEnqSingleObj(qqueue_t *pThis, flowControl_t flowCtlType, msg_t *pMsg); static rsRetVal qqueueChkPersist(qqueue_t *pThis, int nUpdates); @@ -85,8 +91,8 @@ static rsRetVal qqueueMultiEnqObjDirect(qqueue_t *pThis, multi_submit_t *pMultiS static rsRetVal qAddDirect(qqueue_t *pThis, msg_t *pMsg); static rsRetVal qDestructDirect(qqueue_t __attribute__((unused)) *pThis); static rsRetVal qConstructDirect(qqueue_t __attribute__((unused)) *pThis); -static rsRetVal qDelDirect(qqueue_t __attribute__((unused)) *pThis); static rsRetVal qDestructDisk(qqueue_t *pThis); +rsRetVal qqueueSetSpoolDir(qqueue_t *pThis, uchar *pszSpoolDir, int lenSpoolDir); /* some constants for queuePersist () */ #define QUEUE_CHECKPOINT 1 @@ -95,6 +101,7 @@ static rsRetVal qDestructDisk(qqueue_t *pThis); /* tables for interfacing with the v6 config system */ static struct cnfparamdescr cnfpdescr[] = { { "queue.filename", eCmdHdlrGetWord, 0 }, + { "queue.spooldirectory", eCmdHdlrGetWord, 0 }, { "queue.size", eCmdHdlrSize, 0 }, { "queue.dequeuebatchsize", eCmdHdlrInt, 0 }, { "queue.maxdiskspace", eCmdHdlrSize, 0 }, @@ -118,6 +125,7 @@ static struct cnfparamdescr cnfpdescr[] = { { "queue.dequeueslowdown", eCmdHdlrInt, 0 }, { "queue.dequeuetimebegin", eCmdHdlrInt, 0 }, { "queue.dequeuetimeend", eCmdHdlrInt, 0 }, + { "queue.cry.provider", eCmdHdlrGetWord, 0 } }; static struct cnfparamblk pblk = { CNFPARAMBLK_VERSION, @@ -256,7 +264,7 @@ qqueueDbgPrint(qqueue_t *pThis) (pThis->pszFilePrefix == NULL) ? "[NONE]" : (char*)pThis->pszFilePrefix); dbgoprint((obj_t*) pThis, "queue.size: %d\n", pThis->iMaxQueueSize); dbgoprint((obj_t*) pThis, "queue.dequeuebatchsize: %d\n", pThis->iDeqBatchSize); - dbgoprint((obj_t*) pThis, "queue.maxdiskspace: %lld\n", pThis->iMaxFileSize); + dbgoprint((obj_t*) pThis, "queue.maxdiskspace: %lld\n", pThis->sizeOnDiskMax); dbgoprint((obj_t*) pThis, "queue.highwatermark: %d\n", pThis->iHighWtrMrk); dbgoprint((obj_t*) pThis, "queue.lowwatermark: %d\n", pThis->iLowWtrMrk); dbgoprint((obj_t*) pThis, "queue.fulldelaymark: %d\n", pThis->iFullDlyMrk); @@ -350,16 +358,15 @@ qqueueAdviseMaxWorkers(qqueue_t *pThis) if(pThis->bIsDA && getLogicalQueueSize(pThis) >= pThis->iHighWtrMrk) { DBGOPRINT((obj_t*) pThis, "(re)activating DA worker\n"); wtpAdviseMaxWorkers(pThis->pWtpDA, 1); /* disk queues have always one worker */ + } + if(getLogicalQueueSize(pThis) == 0) { + iMaxWorkers = 0; + } else if(pThis->qType == QUEUETYPE_DISK || pThis->iMinMsgsPerWrkr == 0) { + iMaxWorkers = 1; } else { - if(getLogicalQueueSize(pThis) == 0) { - iMaxWorkers = 0; - } else if(pThis->qType == QUEUETYPE_DISK || pThis->iMinMsgsPerWrkr == 0) { - iMaxWorkers = 1; - } else { - iMaxWorkers = getLogicalQueueSize(pThis) / pThis->iMinMsgsPerWrkr + 1; - } - wtpAdviseMaxWorkers(pThis->pWtpReg, iMaxWorkers); + iMaxWorkers = getLogicalQueueSize(pThis) / pThis->iMinMsgsPerWrkr + 1; } + wtpAdviseMaxWorkers(pThis->pWtpReg, iMaxWorkers); } RETiRet; @@ -418,6 +425,7 @@ StartDA(qqueue_t *pThis) CHKiRet(qqueueSetiDeqSlowdown(pThis->pqDA, pThis->iDeqSlowdown)); CHKiRet(qqueueSetMaxFileSize(pThis->pqDA, pThis->iMaxFileSize)); CHKiRet(qqueueSetFilePrefix(pThis->pqDA, pThis->pszFilePrefix, pThis->lenFilePrefix)); + CHKiRet(qqueueSetSpoolDir(pThis->pqDA, pThis->pszSpoolDir, pThis->lenSpoolDir)); CHKiRet(qqueueSetiPersistUpdCnt(pThis->pqDA, pThis->iPersistUpdCnt)); CHKiRet(qqueueSetbSyncQueueFiles(pThis->pqDA, pThis->bSyncQueueFiles)); CHKiRet(qqueueSettoActShutdown(pThis->pqDA, pThis->toActShutdown)); @@ -483,7 +491,6 @@ InitDA(qqueue_t *pThis, int bLockMutex) CHKiRet(wtpSetpfDoWork (pThis->pWtpDA, (rsRetVal (*)(void *pUsr, void *pWti)) ConsumerDA)); CHKiRet(wtpSetpfObjProcessed (pThis->pWtpDA, (rsRetVal (*)(void *pUsr, wti_t *pWti)) batchProcessed)); CHKiRet(wtpSetpmutUsr (pThis->pWtpDA, pThis->mut)); - CHKiRet(wtpSetpcondBusy (pThis->pWtpDA, &pThis->notEmpty)); CHKiRet(wtpSetiNumWorkerThreads (pThis->pWtpDA, 1)); CHKiRet(wtpSettoWrkShutdown (pThis->pWtpDA, pThis->toWrkShutdown)); CHKiRet(wtpSetpUsr (pThis->pWtpDA, pThis)); @@ -706,9 +713,13 @@ queueSwitchToEmergencyMode(qqueue_t *pThis, rsRetVal initiatingError) pThis->qType = QUEUETYPE_DIRECT; pThis->qConstruct = qConstructDirect; pThis->qDestruct = qDestructDirect; + /* these entry points shall not be used in direct mode + * To catch program errors, make us abort if that happens! + * rgerhards, 2013-11-05 + */ pThis->qAdd = qAddDirect; - pThis->qDel = qDelDirect; pThis->MultiEnq = qqueueMultiEnqObjDirect; + pThis->qDel = NULL; if(pThis->pqParent != NULL) { DBGOPRINT((obj_t*) pThis, "DA queue is in emergency mode, disabling DA in parent\n"); pThis->pqParent->bIsDA = 0; @@ -731,7 +742,7 @@ qqueueLoadPersStrmInfoFixup(strm_t *pStrm, qqueue_t __attribute__((unused)) *pTh DEFiRet; ISOBJ_TYPE_assert(pStrm, strm); ISOBJ_TYPE_assert(pThis, qqueue); - CHKiRet(strm.SetDir(pStrm, glbl.GetWorkDir(), strlen((char*)glbl.GetWorkDir()))); + CHKiRet(strm.SetDir(pStrm, pThis->pszSpoolDir, pThis->lenSpoolDir)); finalize_it: RETiRet; } @@ -776,11 +787,19 @@ qqueueTryLoadPersistedInfo(qqueue_t *pThis) (rsRetVal(*)(obj_t*,void*))qqueueLoadPersStrmInfoFixup, pThis)); CHKiRet(obj.Deserialize(&pThis->tVars.disk.pReadDel, (uchar*) "strm", psQIF, (rsRetVal(*)(obj_t*,void*))qqueueLoadPersStrmInfoFixup, pThis)); - /* create a duplicate for the read "pointer". */ CHKiRet(strm.Dup(pThis->tVars.disk.pReadDel, &pThis->tVars.disk.pReadDeq)); CHKiRet(strm.SetbDeleteOnClose(pThis->tVars.disk.pReadDeq, 0)); /* deq must NOT delete the files! */ CHKiRet(strm.ConstructFinalize(pThis->tVars.disk.pReadDeq)); + /* if we use a crypto provider, we need to amend the objects with it's info */ + if(pThis->useCryprov) { + CHKiRet(strm.Setcryprov(pThis->tVars.disk.pWrite, &pThis->cryprov)); + CHKiRet(strm.SetcryprovData(pThis->tVars.disk.pWrite, pThis->cryprovData)); + CHKiRet(strm.Setcryprov(pThis->tVars.disk.pReadDeq, &pThis->cryprov)); + CHKiRet(strm.SetcryprovData(pThis->tVars.disk.pReadDeq, pThis->cryprovData)); + CHKiRet(strm.Setcryprov(pThis->tVars.disk.pReadDel, &pThis->cryprov)); + CHKiRet(strm.SetcryprovData(pThis->tVars.disk.pReadDel, pThis->cryprovData)); + } CHKiRet(strm.SeekCurrOffs(pThis->tVars.disk.pWrite)); CHKiRet(strm.SeekCurrOffs(pThis->tVars.disk.pReadDel)); @@ -830,27 +849,39 @@ static rsRetVal qConstructDisk(qqueue_t *pThis) } else { CHKiRet(strm.Construct(&pThis->tVars.disk.pWrite)); CHKiRet(strm.SetbSync(pThis->tVars.disk.pWrite, pThis->bSyncQueueFiles)); - CHKiRet(strm.SetDir(pThis->tVars.disk.pWrite, glbl.GetWorkDir(), strlen((char*)glbl.GetWorkDir()))); + CHKiRet(strm.SetDir(pThis->tVars.disk.pWrite, pThis->pszSpoolDir, pThis->lenSpoolDir)); CHKiRet(strm.SetiMaxFiles(pThis->tVars.disk.pWrite, 10000000)); CHKiRet(strm.SettOperationsMode(pThis->tVars.disk.pWrite, STREAMMODE_WRITE)); CHKiRet(strm.SetsType(pThis->tVars.disk.pWrite, STREAMTYPE_FILE_CIRCULAR)); + if(pThis->useCryprov) { + CHKiRet(strm.Setcryprov(pThis->tVars.disk.pWrite, &pThis->cryprov)); + CHKiRet(strm.SetcryprovData(pThis->tVars.disk.pWrite, pThis->cryprovData)); + } CHKiRet(strm.ConstructFinalize(pThis->tVars.disk.pWrite)); CHKiRet(strm.Construct(&pThis->tVars.disk.pReadDeq)); CHKiRet(strm.SetbDeleteOnClose(pThis->tVars.disk.pReadDeq, 0)); - CHKiRet(strm.SetDir(pThis->tVars.disk.pReadDeq, glbl.GetWorkDir(), strlen((char*)glbl.GetWorkDir()))); + CHKiRet(strm.SetDir(pThis->tVars.disk.pReadDeq, pThis->pszSpoolDir, pThis->lenSpoolDir)); CHKiRet(strm.SetiMaxFiles(pThis->tVars.disk.pReadDeq, 10000000)); CHKiRet(strm.SettOperationsMode(pThis->tVars.disk.pReadDeq, STREAMMODE_READ)); CHKiRet(strm.SetsType(pThis->tVars.disk.pReadDeq, STREAMTYPE_FILE_CIRCULAR)); + if(pThis->useCryprov) { + CHKiRet(strm.Setcryprov(pThis->tVars.disk.pReadDeq, &pThis->cryprov)); + CHKiRet(strm.SetcryprovData(pThis->tVars.disk.pReadDeq, pThis->cryprovData)); + } CHKiRet(strm.ConstructFinalize(pThis->tVars.disk.pReadDeq)); CHKiRet(strm.Construct(&pThis->tVars.disk.pReadDel)); CHKiRet(strm.SetbSync(pThis->tVars.disk.pReadDel, pThis->bSyncQueueFiles)); CHKiRet(strm.SetbDeleteOnClose(pThis->tVars.disk.pReadDel, 1)); - CHKiRet(strm.SetDir(pThis->tVars.disk.pReadDel, glbl.GetWorkDir(), strlen((char*)glbl.GetWorkDir()))); + CHKiRet(strm.SetDir(pThis->tVars.disk.pReadDel, pThis->pszSpoolDir, pThis->lenSpoolDir)); CHKiRet(strm.SetiMaxFiles(pThis->tVars.disk.pReadDel, 10000000)); CHKiRet(strm.SettOperationsMode(pThis->tVars.disk.pReadDel, STREAMMODE_READ)); CHKiRet(strm.SetsType(pThis->tVars.disk.pReadDel, STREAMTYPE_FILE_CIRCULAR)); + if(pThis->useCryprov) { + CHKiRet(strm.Setcryprov(pThis->tVars.disk.pReadDel, &pThis->cryprov)); + CHKiRet(strm.SetcryprovData(pThis->tVars.disk.pReadDel, pThis->cryprovData)); + } CHKiRet(strm.ConstructFinalize(pThis->tVars.disk.pReadDel)); CHKiRet(strm.SetFName(pThis->tVars.disk.pWrite, pThis->pszFilePrefix, pThis->lenFilePrefix)); @@ -877,7 +908,8 @@ static rsRetVal qDestructDisk(qqueue_t *pThis) DEFiRet; ASSERT(pThis != NULL); - + + free(pThis->pszQIFNam); if(pThis->tVars.disk.pWrite != NULL) strm.Destruct(&pThis->tVars.disk.pWrite); if(pThis->tVars.disk.pReadDeq != NULL) @@ -937,13 +969,11 @@ static rsRetVal qDestructDirect(qqueue_t __attribute__((unused)) *pThis) return RS_RET_OK; } -static rsRetVal qAddDirect(qqueue_t *pThis, msg_t* pMsg) +static rsRetVal qAddDirectWithWti(qqueue_t *pThis, msg_t* pMsg, wti_t *pWti) { batch_t singleBatch; batch_obj_t batchObj; batch_state_t batchState = BATCH_STATE_RDY; - sbool active = 1; - int i; DEFiRet; //TODO: init batchObj (states _OK and new fields -- CHECK) @@ -963,46 +993,29 @@ static rsRetVal qAddDirect(qqueue_t *pThis, msg_t* pMsg) singleBatch.nElem = 1; /* there always is only one in direct mode */ singleBatch.pElem = &batchObj; singleBatch.eltState = &batchState; - singleBatch.active = &active; - iRet = pThis->pConsumer(pThis->pAction, &singleBatch, &pThis->bShutdownImmediate); - /* delete the batch string params: TODO: create its own "class" for this */ - for(i = 0 ; i < CONF_OMOD_NUMSTRINGS_MAXSIZE ; ++i) { - free(batchObj.staticActStrings[i]); - } + iRet = pThis->pConsumer(pThis->pAction, &singleBatch, pWti); msgDestruct(&pMsg); RETiRet; } -/* "enqueue" a batch in direct mode. This is a shortcut which saves all the overhead - * otherwise incured. -- rgerhards, ~2010-06-23 +/* this is called if we do not have a pWti. This currently only happens + * when we are called from a main queue in direct mode. If so, we need + * to obtain a dummy pWti. */ -rsRetVal qqueueEnqObjDirectBatch(qqueue_t *pThis, batch_t *pBatch) +static rsRetVal +qAddDirect(qqueue_t *pThis, msg_t* pMsg) { + wti_t *pWti; DEFiRet; - ASSERT(pThis != NULL); - - /* calling the consumer is quite different here than it is from a worker thread */ - /* we need to provide the consumer's return value back to the caller because in direct - * mode the consumer probably has a lot to convey (which get's lost in the other modes - * because they are asynchronous. But direct mode is deliberately synchronous. - * rgerhards, 2008-02-12 - * We use our knowledge about the batch_t structure below, but without that, we - * pay a too-large performance toll... -- rgerhards, 2009-04-22 - */ - iRet = pThis->pConsumer(pThis->pAction, pBatch, NULL); - + pWti = wtiGetDummy(); + pWti->pbShutdownImmediate = &pThis->bShutdownImmediate; + iRet = qAddDirectWithWti(pThis, pMsg, pWti); RETiRet; } -static rsRetVal qDelDirect(qqueue_t __attribute__((unused)) *pThis) -{ - return RS_RET_OK; -} - - /* --------------- end type-specific handlers -------------------- */ @@ -1022,7 +1035,15 @@ qqueueAdd(qqueue_t *pThis, msg_t *pMsg) if(pThis->qType != QUEUETYPE_DIRECT) { ATOMIC_INC(&pThis->iQueueSize, &pThis->mutQueueSize); - DBGOPRINT((obj_t*) pThis, "entry added, size now log %d, phys %d entries\n", +# ifdef ENABLE_IMDIAG +# ifdef HAVE_ATOMIC_BUILTINS + /* mutex is never used due to conditional compilation */ + ATOMIC_INC(&iOverallQueueSize, &NULL); +# else + ++iOverallQueueSize; /* racy, but we can't wait for a mutex! */ +# endif +# endif + DBGOPRINT((obj_t*) pThis, "qqueueAdd: entry added, size now log %d, phys %d entries\n", getLogicalQueueSize(pThis), getPhysicalQueueSize(pThis)); } @@ -1297,10 +1318,11 @@ finalize_it: * to modify some parameters before the queue is actually started. */ rsRetVal qqueueConstruct(qqueue_t **ppThis, queueType_t qType, int iWorkerThreads, - int iMaxQueueSize, rsRetVal (*pConsumer)(void*, batch_t*,int*)) + int iMaxQueueSize, rsRetVal (*pConsumer)(void*, batch_t*, wti_t*)) { DEFiRet; qqueue_t *pThis; + const uchar *const workDir = glblGetWorkDirRaw(); ASSERT(ppThis != NULL); ASSERT(pConsumer != NULL); @@ -1310,16 +1332,19 @@ rsRetVal qqueueConstruct(qqueue_t **ppThis, queueType_t qType, int iWorkerThread /* we have an object, so let's fill the properties */ objConstructSetObjInfo(pThis); - if((pThis->pszSpoolDir = (uchar*) strdup((char*)glbl.GetWorkDir())) == NULL) - ABORT_FINALIZE(RS_RET_OUT_OF_MEMORY); + if(workDir != NULL) { + if((pThis->pszSpoolDir = ustrdup(workDir)) == NULL) + ABORT_FINALIZE(RS_RET_OUT_OF_MEMORY); + pThis->lenSpoolDir = ustrlen(pThis->pszSpoolDir); + } /* set some water marks so that we have useful defaults if none are set specifically */ pThis->iFullDlyMrk = -1; pThis->iLightDlyMrk = -1; - pThis->lenSpoolDir = ustrlen(pThis->pszSpoolDir); pThis->iMaxFileSize = 1024 * 1024; /* default is 1 MiB */ pThis->iQueueSize = 0; pThis->nLogDeq = 0; + pThis->useCryprov = 0; pThis->iMaxQueueSize = iMaxQueueSize; pThis->pConsumer = pConsumer; pThis->iNumWorkerThreads = iWorkerThreads; @@ -1352,9 +1377,9 @@ qqueueSetDefaultsActionQueue(qqueue_t *pThis) pThis->qType = QUEUETYPE_DIRECT; /* type of the main message queue above */ pThis->iMaxQueueSize = 1000; /* size of the main message queue above */ pThis->iDeqBatchSize = 128; /* default batch size */ - pThis->iHighWtrMrk = 800; /* high water mark for disk-assisted queues */ - pThis->iLowWtrMrk = 200; /* low water mark for disk-assisted queues */ - pThis->iDiscardMrk = 980; /* begin to discard messages */ + pThis->iHighWtrMrk = -1; /* high water mark for disk-assisted queues */ + pThis->iLowWtrMrk = -1; /* low water mark for disk-assisted queues */ + pThis->iDiscardMrk = -1; /* begin to discard messages */ pThis->iDiscardSeverity = 8; /* turn off */ pThis->iNumWorkerThreads = 1; /* number of worker threads for the mm queue above */ pThis->iMaxFileSize = 1024*1024; @@ -1364,7 +1389,7 @@ qqueueSetDefaultsActionQueue(qqueue_t *pThis) pThis->toActShutdown = 1000; /* action shutdown (in phase 2) */ pThis->toEnq = 2000; /* timeout for queue enque */ pThis->toWrkShutdown = 60000; /* timeout for worker thread shutdown */ - pThis->iMinMsgsPerWrkr = 100; /* minimum messages per worker needed to start a new one */ + pThis->iMinMsgsPerWrkr = -1; /* minimum messages per worker needed to start a new one */ pThis->bSaveOnShutdown = 1; /* save queue on shutdown (when DA enabled)? */ pThis->sizeOnDiskMax = 0; /* unlimited */ pThis->iDeqSlowdown = 0; @@ -1382,9 +1407,9 @@ qqueueSetDefaultsRulesetQueue(qqueue_t *pThis) pThis->qType = QUEUETYPE_FIXED_ARRAY; /* type of the main message queue above */ pThis->iMaxQueueSize = 50000; /* size of the main message queue above */ pThis->iDeqBatchSize = 1024; /* default batch size */ - pThis->iHighWtrMrk = 45000; /* high water mark for disk-assisted queues */ - pThis->iLowWtrMrk = 20000; /* low water mark for disk-assisted queues */ - pThis->iDiscardMrk = 49500; /* begin to discard messages */ + pThis->iHighWtrMrk = -1; /* high water mark for disk-assisted queues */ + pThis->iLowWtrMrk = -1; /* low water mark for disk-assisted queues */ + pThis->iDiscardMrk = -1; /* begin to discard messages */ pThis->iDiscardSeverity = 8; /* turn off */ pThis->iNumWorkerThreads = 1; /* number of worker threads for the mm queue above */ pThis->iMaxFileSize = 16*1024*1024; @@ -1394,7 +1419,7 @@ qqueueSetDefaultsRulesetQueue(qqueue_t *pThis) pThis->toActShutdown = 1000; /* action shutdown (in phase 2) */ pThis->toEnq = 2000; /* timeout for queue enque */ pThis->toWrkShutdown = 60000; /* timeout for worker thread shutdown */ - pThis->iMinMsgsPerWrkr = 1000; /* minimum messages per worker needed to start a new one */ + pThis->iMinMsgsPerWrkr = -1; /* minimum messages per worker needed to start a new one */ pThis->bSaveOnShutdown = 1; /* save queue on shutdown (when DA enabled)? */ pThis->sizeOnDiskMax = 0; /* unlimited */ pThis->iDeqSlowdown = 0; @@ -1466,8 +1491,8 @@ DoDeleteBatchFromQStore(qqueue_t *pThis, int nElem) */ if(bytesDel != 0) { pThis->tVars.disk.sizeOnDisk -= bytesDel; - DBGOPRINT((obj_t*) pThis, "a %lld octet file has been deleted, now %lld octets disk " - "space used\n", bytesDel, pThis->tVars.disk.sizeOnDisk); + DBGOPRINT((obj_t*) pThis, "doDeleteBatch: a %lld octet file has been deleted, now %lld octets disk " + "space used\n", (long long) bytesDel, pThis->tVars.disk.sizeOnDisk); /* awake possibly waiting enq process */ pthread_cond_signal(&pThis->notFull); /* we hold the mutex while we are in here! */ } @@ -1479,8 +1504,16 @@ DoDeleteBatchFromQStore(qqueue_t *pThis, int nElem) /* iQueueSize is not decremented by qDel(), so we need to do it ourselves */ ATOMIC_SUB(&pThis->iQueueSize, nElem, &pThis->mutQueueSize); +# ifdef ENABLE_IMDIAG +# ifdef HAVE_ATOMIC_BUILTINS + /* mutex is never used due to conditional compilation */ + ATOMIC_SUB(&iOverallQueueSize, nElem, &NULL); +# else + iOverallQueueSize -= nElem; /* racy, but we can't wait for a mutex! */ +# endif +# endif ATOMIC_SUB(&pThis->nLogDeq, nElem, &pThis->mutLogDeq); - DBGPRINTF("delete batch from store, new sizes: log %d, phys %d\n", + DBGPRINTF("doDeleteBatch: delete batch from store, new sizes: log %d, phys %d\n", getLogicalQueueSize(pThis), getPhysicalQueueSize(pThis)); ++pThis->deqIDDel; /* one more batch dequeued */ @@ -1550,13 +1583,13 @@ DeleteProcessedBatch(qqueue_t *pThis, batch_t *pBatch) localRet = doEnqSingleObj(pThis, eFLOWCTL_NO_DELAY, MsgAddRef(pMsg)); ++nEnqueued; if(localRet != RS_RET_OK) { - DBGPRINTF("error %d re-enqueuing unprocessed data element - discarded\n", localRet); + DBGPRINTF("DeleteProcessedBatch: error %d re-enqueuing unprocessed data element - discarded\n", localRet); } } msgDestruct(&pMsg); } - DBGPRINTF("we deleted %d objects and enqueued %d objects\n", i-nEnqueued, nEnqueued); + DBGPRINTF("DeleteProcessedBatch: we deleted %d objects and enqueued %d objects\n", i-nEnqueued, nEnqueued); if(nEnqueued > 0) qqueueChkPersist(pThis, nEnqueued); @@ -1675,6 +1708,9 @@ DequeueConsumable(qqueue_t *pThis, wti_t *pWti) /* The rate limiter * + * IMPORTANT: the rate-limiter MUST unlock and re-lock the queue when + * it actually delays processing. Otherwise inputs are stalled. + * * Here we may wait if a dequeue time window is defined or if we are * rate-limited. TODO: If we do so, we should also look into the * way new worker threads are spawned. Obviously, it doesn't make much @@ -1760,8 +1796,10 @@ RateLimiter(qqueue_t *pThis) } if(iDelay > 0) { + pthread_mutex_unlock(pThis->mut); DBGOPRINT((obj_t*) pThis, "outside dequeue time window, delaying %d seconds\n", iDelay); srSleep(iDelay, 0); + pthread_mutex_lock(pThis->mut); } RETiRet; @@ -1855,7 +1893,8 @@ ConsumerReg(qqueue_t *pThis, wti_t *pWti) pthread_setcancelstate(PTHREAD_CANCEL_ENABLE, &iCancelStateSave); - CHKiRet(pThis->pConsumer(pThis->pAction, &pWti->batch, &pThis->bShutdownImmediate)); + pWti->pbShutdownImmediate = &pThis->bShutdownImmediate; + CHKiRet(pThis->pConsumer(pThis->pAction, &pWti->batch, pWti)); /* we now need to check if we should deliberately delay processing a bit * and, if so, do that. -- rgerhards, 2008-01-30 @@ -1913,8 +1952,16 @@ ConsumerDA(qqueue_t *pThis, wti_t *pWti) /* iterate over returned results and enqueue them in DA queue */ for(i = 0 ; i < pWti->batch.nElem && !pThis->bShutdownImmediate ; i++) { - CHKiRet(qqueueEnqMsg(pThis->pqDA, eFLOWCTL_NO_DELAY, - MsgAddRef(pWti->batch.pElem[i].pMsg))); + iRet = qqueueEnqMsg(pThis->pqDA, eFLOWCTL_NO_DELAY, MsgAddRef(pWti->batch.pElem[i].pMsg)); + if(iRet != RS_RET_OK) { + if(iRet == RS_RET_ERR_QUEUE_EMERGENCY) { + /* Queue emergency error occured */ + DBGOPRINT((obj_t*) pThis, "ConsumerDA:qqueueEnqMsg caught RS_RET_ERR_QUEUE_EMERGENCY, aborting loop.\n"); + FINALIZE; + } else { + DBGOPRINT((obj_t*) pThis, "ConsumerDA:qqueueEnqMsg item (%d) returned with error state: '%d'\n", i, iRet); + } + } pWti->batch.eltState[i] = BATCH_STATE_COMM; /* commited to other queue! */ } @@ -1922,10 +1969,38 @@ ConsumerDA(qqueue_t *pThis, wti_t *pWti) pthread_setcancelstate(iCancelStateSave, NULL); finalize_it: + /* Check the last return state of qqueueEnqMsg. If an error was returned, we acknowledge it only. + * Unless the error code is RS_RET_ERR_QUEUE_EMERGENCY, we reset the return state to RS_RET_OK. + * Otherwise the Caller functions would run into an infinite Loop trying to enqueue the + * same messages over and over again. + * + * However we do NOT overwrite positive return states like + * RS_RET_TERMINATE_NOW, + * RS_RET_NO_RUN, + * RS_RET_IDLE, + * RS_RET_TERMINATE_WHEN_IDLE + * These return states are important for Queue handling of the upper laying functions. + * RGer: Note that checking for iRet < 0 is a bit bold. In theory, positive iRet + * values are "OK" states, and things that the caller shall deal with. However, + * this has not been done so consistently. Andre convinced me that the current + * code is an elegant solution. However, if problems with queue workers and/or + * shutdown come up, this code here should be looked at suspiciously. In those + * cases it may work out to check all status codes explicitely, just to avoid + * a pitfall due to unexpected states being passed on to the caller. + */ + if( iRet != RS_RET_OK && + iRet != RS_RET_ERR_QUEUE_EMERGENCY && + iRet < 0) { + DBGOPRINT((obj_t*) pThis, "ConsumerDA:qqueueEnqMsg Resetting iRet from %d back to RS_RET_OK\n", iRet); + iRet = RS_RET_OK; + } else { + DBGOPRINT((obj_t*) pThis, "ConsumerDA:qqueueEnqMsg returns with iRet %d\n", iRet); + } + /* now we are done, but potentially need to re-aquire the mutex */ if(bNeedReLock) d_pthread_mutex_lock(pThis->mut); - DBGOPRINT((obj_t*) pThis, "DAConsumer returns with iRet %d\n", iRet); + RETiRet; } @@ -1997,11 +2072,22 @@ qqueueStart(qqueue_t *pThis) /* this is the ConstructionFinalizer */ uchar pszBuf[64]; uchar pszQIFNam[MAXFNAME]; int wrk; + int goodval; /* a "good value" to use for comparisons (different objects) */ uchar *qName; size_t lenBuf; ASSERT(pThis != NULL); + dbgoprint((obj_t*) pThis, "starting queue\n"); + + if(pThis->pszSpoolDir == NULL) { + /* note: we need to pick the path so late as we do not have + * the workdir during early config load + */ + if((pThis->pszSpoolDir = (uchar*) strdup((char*)glbl.GetWorkDir())) == NULL) + ABORT_FINALIZE(RS_RET_OUT_OF_MEMORY); + pThis->lenSpoolDir = ustrlen(pThis->pszSpoolDir); + } /* set type-specific handlers and other very type-specific things * (we can not totally hide it...) */ @@ -2033,7 +2119,7 @@ qqueueStart(qqueue_t *pThis) /* this is the ConstructionFinalizer */ pThis->iNumWorkerThreads = 1; /* we need exactly one worker */ /* pre-construct file name for .qi file */ pThis->lenQIFNam = snprintf((char*)pszQIFNam, sizeof(pszQIFNam) / sizeof(uchar), - "%s/%s.qi", (char*) glbl.GetWorkDir(), (char*)pThis->pszFilePrefix); + "%s/%s.qi", (char*) pThis->pszSpoolDir, (char*)pThis->pszFilePrefix); pThis->pszQIFNam = ustrdup(pszQIFNam); DBGOPRINT((obj_t*) pThis, ".qi file name is '%s', len %d\n", pThis->pszQIFNam, (int) pThis->lenQIFNam); @@ -2041,28 +2127,123 @@ qqueueStart(qqueue_t *pThis) /* this is the ConstructionFinalizer */ case QUEUETYPE_DIRECT: pThis->qConstruct = qConstructDirect; pThis->qDestruct = qDestructDirect; + /* these entry points shall not be used in direct mode + * To catch program errors, make us abort if that happens! + * rgerhards, 2013-11-05 + */ pThis->qAdd = qAddDirect; - pThis->qDel = qDelDirect; pThis->MultiEnq = qqueueMultiEnqObjDirect; + pThis->qDel = NULL; break; } - if(pThis->iFullDlyMrk == -1) - pThis->iFullDlyMrk = pThis->iMaxQueueSize - - (pThis->iMaxQueueSize / 100) * 3; /* default 97% */ - if(pThis->iLightDlyMrk == -1) - pThis->iLightDlyMrk = pThis->iMaxQueueSize - - (pThis->iMaxQueueSize / 100) * 30; /* default 70% */ + if(pThis->iMaxQueueSize < 100 + && (pThis->qType == QUEUETYPE_LINKEDLIST || pThis->qType == QUEUETYPE_FIXED_ARRAY)) { + errmsg.LogError(0, RS_RET_OK_WARN, "Note: queue.size=\"%d\" is very " + "low and can lead to unpredictable results. See also " + "http://www.rsyslog.com/lower-bound-for-queue-sizes/", + pThis->iMaxQueueSize); + } /* we need to do a quick check if our water marks are set plausible. If not, - * we correct the most important shortcomings. TODO: do that!!!! -- rgerhards, 2008-03-14 + * we correct the most important shortcomings. */ + goodval = (pThis->iMaxQueueSize / 100) * 60; + if(pThis->iHighWtrMrk != -1 && pThis->iHighWtrMrk < goodval) { + errmsg.LogError(0, RS_RET_CONF_PARSE_WARNING, "queue \"%s\": high water mark " + "is set quite low at %d. You should only set it below " + "60%% (%d) if you have a good reason for this.", + obj.GetName((obj_t*) pThis), pThis->iHighWtrMrk, goodval); + } + + if(pThis->iNumWorkerThreads > 1) { + goodval = (pThis->iMaxQueueSize / 100) * 10; + if(pThis->iMinMsgsPerWrkr != -1 && pThis->iMinMsgsPerWrkr < goodval) { + errmsg.LogError(0, RS_RET_CONF_PARSE_WARNING, "queue \"%s\": " + "queue.workerThreadMinimumMessage " + "is set quite low at %d. You should only set it below " + "10%% (%d) if you have a good reason for this.", + obj.GetName((obj_t*) pThis), pThis->iMinMsgsPerWrkr, goodval); + } + } + + if(pThis->iDiscardMrk > pThis->iMaxQueueSize) { + errmsg.LogError(0, RS_RET_CONF_PARSE_WARNING, "queue \"%s\": " + "queue.discardMark %d is set larger than queue.size", + obj.GetName((obj_t*) pThis), pThis->iDiscardMrk); + } + + goodval = (pThis->iMaxQueueSize / 100) * 80; + if(pThis->iDiscardMrk != -1 && pThis->iDiscardMrk < goodval) { + errmsg.LogError(0, RS_RET_CONF_PARSE_WARNING, "queue \"%s\": queue.discardMark " + "is set quite low at %d. You should only set it below " + "80%% (%d) if you have a good reason for this.", + obj.GetName((obj_t*) pThis), pThis->iDiscardMrk, goodval); + } + + if(pThis->pszFilePrefix != NULL) { /* This means we have a potential DA queue */ + if(pThis->iFullDlyMrk != -1 && pThis->iFullDlyMrk < pThis->iHighWtrMrk) { + errmsg.LogError(0, RS_RET_CONF_WRN_FULLDLY_BELOW_HIGHWTR, + "queue \"%s\": queue.fullDelayMark " + "is set below high water mark. This will result in DA mode " + " NOT being activated for full delayable messages", + obj.GetName((obj_t*) pThis)); + } + } + + /* now come parameter corrections and defaults */ + if(pThis->iHighWtrMrk < 2 || pThis->iHighWtrMrk > pThis->iMaxQueueSize) { + pThis->iHighWtrMrk = (pThis->iMaxQueueSize / 100) * 90; + if(pThis->iHighWtrMrk == 0) { /* guard against very low max queue sizes! */ + pThis->iHighWtrMrk = pThis->iMaxQueueSize; + } + } + if( pThis->iLowWtrMrk < 2 + || pThis->iLowWtrMrk > pThis->iMaxQueueSize + || pThis->iLowWtrMrk > pThis->iHighWtrMrk ) { + pThis->iLowWtrMrk = (pThis->iMaxQueueSize / 100) * 70; + if(pThis->iLowWtrMrk == 0) { + pThis->iLowWtrMrk = 1; + } + } + + if( pThis->iMinMsgsPerWrkr < 1 + || pThis->iMinMsgsPerWrkr > pThis->iMaxQueueSize ) { + pThis->iMinMsgsPerWrkr = pThis->iMaxQueueSize / pThis->iNumWorkerThreads; + } + + if(pThis->iFullDlyMrk == -1 || pThis->iFullDlyMrk > pThis->iMaxQueueSize) { + pThis->iFullDlyMrk = (pThis->iMaxQueueSize / 100) * 97; + if(pThis->iFullDlyMrk == 0) { + pThis->iFullDlyMrk = + (pThis->iMaxQueueSize == 1) ? 1 : pThis->iMaxQueueSize - 1; + } + } + if(pThis->iLightDlyMrk == -1 || pThis->iLightDlyMrk > pThis->iMaxQueueSize) { + pThis->iLightDlyMrk = (pThis->iMaxQueueSize / 100) * 70; + if(pThis->iLightDlyMrk == 0) { + pThis->iLightDlyMrk = + (pThis->iMaxQueueSize == 1) ? 1 : pThis->iMaxQueueSize - 1; + } + } + + if(pThis->iDiscardMrk < 1 || pThis->iDiscardMrk > pThis->iMaxQueueSize) { + pThis->iDiscardMrk = (pThis->iMaxQueueSize / 100) * 98; + if(pThis->iDiscardMrk == 0) { + /* for very small queues, we disable this by default */ + pThis->iDiscardMrk = pThis->iMaxQueueSize; + } + } + + if(pThis->iMaxQueueSize > 0 && pThis->iDeqBatchSize > pThis->iMaxQueueSize) { + pThis->iDeqBatchSize = pThis->iMaxQueueSize; + } /* finalize some initializations that could not yet be done because it is * influenced by properties which might have been set after queueConstruct () */ if(pThis->pqParent == NULL) { - pThis->mut = (pthread_mutex_t *) MALLOC (sizeof (pthread_mutex_t)); + CHKmalloc(pThis->mut = (pthread_mutex_t *) MALLOC (sizeof (pthread_mutex_t))); pthread_mutex_init(pThis->mut, NULL); } else { /* child queue, we need to use parent's mutex */ @@ -2072,7 +2253,6 @@ qqueueStart(qqueue_t *pThis) /* this is the ConstructionFinalizer */ pthread_mutex_init(&pThis->mutThrdMgmt, NULL); pthread_cond_init (&pThis->notFull, NULL); - pthread_cond_init (&pThis->notEmpty, NULL); pthread_cond_init (&pThis->belowFullDlyWtrMrk, NULL); pthread_cond_init (&pThis->belowLightDlyWtrMrk, NULL); @@ -2089,12 +2269,16 @@ qqueueStart(qqueue_t *pThis) /* this is the ConstructionFinalizer */ pThis->iFullDlyMrk = wrk; } - DBGOPRINT((obj_t*) pThis, "type %d, enq-only %d, disk assisted %d, maxFileSz %lld, lqsize %d, pqsize %d, child %d, " - "full delay %d, light delay %d, deq batch size %d starting\n", - pThis->qType, pThis->bEnqOnly, pThis->bIsDA, pThis->iMaxFileSize, + DBGOPRINT((obj_t*) pThis, "params: type %d, enq-only %d, disk assisted %d, spoolDir '%s', maxFileSz %lld, " + "maxQSize %d, lqsize %d, pqsize %d, child %d, full delay %d, " + "light delay %d, deq batch size %d, high wtrmrk %d, low wtrmrk %d, " + "discardmrk %d, max wrkr %d, min msgs f. wrkr %d\n", + pThis->qType, pThis->bEnqOnly, pThis->bIsDA, pThis->pszSpoolDir, + pThis->iMaxFileSize, pThis->iMaxQueueSize, getLogicalQueueSize(pThis), getPhysicalQueueSize(pThis), pThis->pqParent == NULL ? 0 : 1, pThis->iFullDlyMrk, pThis->iLightDlyMrk, - pThis->iDeqBatchSize); + pThis->iDeqBatchSize, pThis->iHighWtrMrk, pThis->iLowWtrMrk, + pThis->iDiscardMrk, pThis->iNumWorkerThreads, pThis->iMinMsgsPerWrkr); pThis->bQueueStarted = 1; if(pThis->qType == QUEUETYPE_DIRECT) @@ -2111,7 +2295,6 @@ qqueueStart(qqueue_t *pThis) /* this is the ConstructionFinalizer */ CHKiRet(wtpSetpfDoWork (pThis->pWtpReg, (rsRetVal (*)(void *pUsr, void *pWti)) ConsumerReg)); CHKiRet(wtpSetpfObjProcessed (pThis->pWtpReg, (rsRetVal (*)(void *pUsr, wti_t *pWti)) batchProcessed)); CHKiRet(wtpSetpmutUsr (pThis->pWtpReg, pThis->mut)); - CHKiRet(wtpSetpcondBusy (pThis->pWtpReg, &pThis->notEmpty)); CHKiRet(wtpSetiNumWorkerThreads (pThis->pWtpReg, pThis->iNumWorkerThreads)); CHKiRet(wtpSettoWrkShutdown (pThis->pWtpReg, pThis->toWrkShutdown)); CHKiRet(wtpSetpUsr (pThis->pWtpReg, pThis)); @@ -2135,30 +2318,35 @@ qqueueStart(qqueue_t *pThis) /* this is the ConstructionFinalizer */ /* we need to save the queue size, as the stats module initializes it to 0! */ /* iQueueSize is a dual-use counter: no init, no mutex! */ CHKiRet(statsobj.AddCounter(pThis->statsobj, UCHAR_CONSTANT("size"), - ctrType_Int, &pThis->iQueueSize)); + ctrType_Int, CTR_FLAG_NONE, &pThis->iQueueSize)); STATSCOUNTER_INIT(pThis->ctrEnqueued, pThis->mutCtrEnqueued); CHKiRet(statsobj.AddCounter(pThis->statsobj, UCHAR_CONSTANT("enqueued"), - ctrType_IntCtr, &pThis->ctrEnqueued)); + ctrType_IntCtr, CTR_FLAG_RESETTABLE, &pThis->ctrEnqueued)); STATSCOUNTER_INIT(pThis->ctrFull, pThis->mutCtrFull); CHKiRet(statsobj.AddCounter(pThis->statsobj, UCHAR_CONSTANT("full"), - ctrType_IntCtr, &pThis->ctrFull)); + ctrType_IntCtr, CTR_FLAG_RESETTABLE, &pThis->ctrFull)); STATSCOUNTER_INIT(pThis->ctrFDscrd, pThis->mutCtrFDscrd); CHKiRet(statsobj.AddCounter(pThis->statsobj, UCHAR_CONSTANT("discarded.full"), - ctrType_IntCtr, &pThis->ctrFDscrd)); + ctrType_IntCtr, CTR_FLAG_RESETTABLE, &pThis->ctrFDscrd)); STATSCOUNTER_INIT(pThis->ctrNFDscrd, pThis->mutCtrNFDscrd); CHKiRet(statsobj.AddCounter(pThis->statsobj, UCHAR_CONSTANT("discarded.nf"), - ctrType_IntCtr, &pThis->ctrNFDscrd)); + ctrType_IntCtr, CTR_FLAG_RESETTABLE, &pThis->ctrNFDscrd)); pThis->ctrMaxqsize = 0; /* no mutex needed, thus no init call */ CHKiRet(statsobj.AddCounter(pThis->statsobj, UCHAR_CONSTANT("maxqsize"), - ctrType_Int, &pThis->ctrMaxqsize)); + ctrType_Int, CTR_FLAG_NONE, &pThis->ctrMaxqsize)); CHKiRet(statsobj.ConstructFinalize(pThis->statsobj)); finalize_it: + if(iRet != RS_RET_OK) { + /* note: a child uses it's parent mutex, so do not delete it! */ + if(pThis->pqParent == NULL && pThis->mut != NULL) + free(pThis->mut); + } RETiRet; } @@ -2376,7 +2564,6 @@ CODESTARTobjDestruct(qqueue) } pthread_mutex_destroy(&pThis->mutThrdMgmt); pthread_cond_destroy(&pThis->notFull); - pthread_cond_destroy(&pThis->notEmpty); pthread_cond_destroy(&pThis->belowFullDlyWtrMrk); pthread_cond_destroy(&pThis->belowLightDlyWtrMrk); @@ -2389,6 +2576,13 @@ CODESTARTobjDestruct(qqueue) free(pThis->pszFilePrefix); free(pThis->pszSpoolDir); + if(pThis->useCryprov) { + pThis->cryprov.Destruct(&pThis->cryprovData); + obj.ReleaseObj(__FILE__, pThis->cryprovNameFull+2, pThis->cryprovNameFull, + (void*) &pThis->cryprov); + free(pThis->cryprovName); + free(pThis->cryprovNameFull); + } /* some queues do not provide stats and thus have no statsobj! */ if(pThis->statsobj != NULL) @@ -2396,6 +2590,24 @@ CODESTARTobjDestruct(qqueue) ENDobjDestruct(qqueue) +/* set the queue's spool directory. The directory MUST NOT be NULL. + * The passed-in string is duplicated. So if the caller does not need + * it any longer, it must free it. + */ +rsRetVal +qqueueSetSpoolDir(qqueue_t *pThis, uchar *pszSpoolDir, int lenSpoolDir) +{ + DEFiRet; + + free(pThis->pszSpoolDir); + CHKmalloc(pThis->pszSpoolDir = ustrdup(pszSpoolDir)); + pThis->lenSpoolDir = lenSpoolDir; + +finalize_it: + RETiRet; +} + + /* set the queue's file prefix * The passed-in string is duplicated. So if the caller does not need * it any longer, it must free it. @@ -2491,7 +2703,7 @@ doEnqSingleObj(qqueue_t *pThis, flowControl_t flowCtlType, msg_t *pMsg) * In any case, this was the old code (if we do the TODO): * pthread_cond_wait(&pThis->belowFullDlyWtrMrk, pThis->mut); */ - DBGOPRINT((obj_t*) pThis, "enqueueMsg: FullDelay mark reached for full delayable message " + DBGOPRINT((obj_t*) pThis, "doEnqSingleObject: FullDelay mark reached for full delayable message " "- blocking, queue size is %d.\n", pThis->iQueueSize); timeoutComp(&t, 1000); err = pthread_cond_timedwait(&pThis->belowLightDlyWtrMrk, pThis->mut, &t); @@ -2508,7 +2720,7 @@ doEnqSingleObj(qqueue_t *pThis, flowControl_t flowCtlType, msg_t *pMsg) } } else if(flowCtlType == eFLOWCTL_LIGHT_DELAY && !glbl.GetGlobalInputTermState()) { if(pThis->iQueueSize >= pThis->iLightDlyMrk) { - DBGOPRINT((obj_t*) pThis, "enqueueMsg: LightDelay mark reached for light " + DBGOPRINT((obj_t*) pThis, "doEnqSingleObject: LightDelay mark reached for light " "delayable message - blocking a bit.\n"); timeoutComp(&t, 1000); /* 1000 millisconds = 1 second TODO: make configurable */ err = pthread_cond_timedwait(&pThis->belowLightDlyWtrMrk, pThis->mut, &t); @@ -2527,28 +2739,30 @@ doEnqSingleObj(qqueue_t *pThis, flowControl_t flowCtlType, msg_t *pMsg) * the queue to become ready or drop the new message. -- rgerhards, 2008-03-14 */ while( (pThis->iMaxQueueSize > 0 && pThis->iQueueSize >= pThis->iMaxQueueSize) - || (pThis->qType == QUEUETYPE_DISK && pThis->sizeOnDiskMax != 0 + || ((pThis->qType == QUEUETYPE_DISK || pThis->bIsDA) && pThis->sizeOnDiskMax != 0 && pThis->tVars.disk.sizeOnDisk > pThis->sizeOnDiskMax)) { STATSCOUNTER_INC(pThis->ctrFull, pThis->mutCtrFull); if(pThis->toEnq == 0 || pThis->bEnqOnly) { - DBGOPRINT((obj_t*) pThis, "enqueueMsg: queue FULL - configured for immediate discarding.\n"); + DBGOPRINT((obj_t*) pThis, "doEnqSingleObject: queue FULL - configured for immediate discarding QueueSize=%d " + "MaxQueueSize=%d sizeOnDisk=%lld sizeOnDiskMax=%lld\n", pThis->iQueueSize, pThis->iMaxQueueSize, + pThis->tVars.disk.sizeOnDisk, pThis->sizeOnDiskMax); STATSCOUNTER_INC(pThis->ctrFDscrd, pThis->mutCtrFDscrd); msgDestruct(&pMsg); ABORT_FINALIZE(RS_RET_QUEUE_FULL); } else { - DBGOPRINT((obj_t*) pThis, "enqueueMsg: queue FULL - waiting %dms to drain.\n", pThis->toEnq); + DBGOPRINT((obj_t*) pThis, "doEnqSingleObject: queue FULL - waiting %dms to drain.\n", pThis->toEnq); if(glbl.GetGlobalInputTermState()) { - DBGOPRINT((obj_t*) pThis, "enqueueMsg: queue FULL, discard due to FORCE_TERM.\n"); + DBGOPRINT((obj_t*) pThis, "doEnqSingleObject: queue FULL, discard due to FORCE_TERM.\n"); ABORT_FINALIZE(RS_RET_FORCE_TERM); } timeoutComp(&t, pThis->toEnq); if(pthread_cond_timedwait(&pThis->notFull, pThis->mut, &t) != 0) { - DBGOPRINT((obj_t*) pThis, "enqueueMsg: cond timeout, dropping message!\n"); + DBGOPRINT((obj_t*) pThis, "doEnqSingleObject: cond timeout, dropping message!\n"); STATSCOUNTER_INC(pThis->ctrFDscrd, pThis->mutCtrFDscrd); msgDestruct(&pMsg); ABORT_FINALIZE(RS_RET_QUEUE_FULL); } - dbgoprint((obj_t*) pThis, "enqueueMsg: wait solved queue full condition, enqueing\n"); + dbgoprint((obj_t*) pThis, "doEnqSingleObject: wait solved queue full condition, enqueing\n"); } } @@ -2608,13 +2822,14 @@ static rsRetVal qqueueMultiEnqObjDirect(qqueue_t *pThis, multi_submit_t *pMultiSub) { int i; + wti_t *pWti; DEFiRet; - ISOBJ_TYPE_assert(pThis, qqueue); - assert(pMultiSub != NULL); + pWti = wtiGetDummy(); + pWti->pbShutdownImmediate = &pThis->bShutdownImmediate; for(i = 0 ; i < pMultiSub->nElem ; ++i) { - CHKiRet(qAddDirect(pThis, (void*)pMultiSub->ppMsgs[i])); + CHKiRet(qAddDirectWithWti(pThis, (void*)pMultiSub->ppMsgs[i], pWti)); } finalize_it: @@ -2623,22 +2838,7 @@ finalize_it: /* ------------------------------ END multi-enqueue functions ------------------------------ */ -/* enqueue a new user data element in direct mode - * NOTE/TODO: This is a TESTER/EXPERIEMENTAL, to be changed to better - * code later on (like multi submit!) 2010-06-10 - * Enqueues the new element and awakes worker thread. - */ -rsRetVal -qqueueEnqMsgDirect(qqueue_t *pThis, msg_t *pMsg) -{ - DEFiRet; - ISOBJ_TYPE_assert(pThis, qqueue); - iRet = qAddDirect(pThis, pMsg); - RETiRet; -} - - -/* enqueue a new user data element +/* enqueue a new user data element * Enqueues the new element and awakes worker thread. */ rsRetVal @@ -2672,27 +2872,67 @@ finalize_it: } -/* take v6 config list and extract the queue params out of it. Hand the - * param values back to the caller. Caller is responsible for destructing - * them when no longer needed. Caller can use this param block to configure - * all parameters for a newly created queue with one call to qqueueSetParams(). - * rgerhards, 2011-07-22 +/* are any queue params set at all? 1 - yes, 0 - no + * We need to evaluate the param block for this function, which is somewhat + * inefficient. HOWEVER, this is only done during config load, so we really + * don't care... -- rgerhards, 2013-05-10 */ -rsRetVal -qqueueDoCnfParams(struct nvlst *lst, struct cnfparamvals **ppvals) +int +queueCnfParamsSet(struct nvlst *lst) { - *ppvals = nvlstGetParams(lst, &pblk, NULL); - return RS_RET_OK; + int r; + struct cnfparamvals *pvals; + + pvals = nvlstGetParams(lst, &pblk, NULL); + r = cnfparamvalsIsSet(&pblk, pvals); + cnfparamvalsDestruct(pvals, &pblk); + return r; } -/* are any queue params set at all? 1 - yes, 0 - no */ -int -queueCnfParamsSet(struct cnfparamvals *pvals) +static inline rsRetVal +initCryprov(qqueue_t *pThis, struct nvlst *lst) { - return cnfparamvalsIsSet(&pblk, pvals); -} + uchar szDrvrName[1024]; + DEFiRet; + + if(snprintf((char*)szDrvrName, sizeof(szDrvrName), "lmcry_%s", pThis->cryprovName) + == sizeof(szDrvrName)) { + errmsg.LogError(0, RS_RET_ERR, "queue: crypto provider " + "name is too long: '%s' - encryption disabled", + pThis->cryprovName); + ABORT_FINALIZE(RS_RET_ERR); + } + pThis->cryprovNameFull = ustrdup(szDrvrName); + + pThis->cryprov.ifVersion = cryprovCURR_IF_VERSION; + /* The pDrvrName+2 below is a hack to obtain the object name. It + * safes us to have yet another variable with the name without "lm" in + * front of it. If we change the module load interface, we may re-think + * about this hack, but for the time being it is efficient and clean enough. + */ + if(obj.UseObj(__FILE__, szDrvrName, szDrvrName, (void*) &pThis->cryprov) + != RS_RET_OK) { + errmsg.LogError(0, RS_RET_LOAD_ERROR, "queue: could not load " + "crypto provider '%s' - encryption disabled", + szDrvrName); + ABORT_FINALIZE(RS_RET_CRYPROV_ERR); + } + if(pThis->cryprov.Construct(&pThis->cryprovData) != RS_RET_OK) { + errmsg.LogError(0, RS_RET_CRYPROV_ERR, "queue: error constructing " + "crypto provider %s dataset - encryption disabled", + szDrvrName); + ABORT_FINALIZE(RS_RET_CRYPROV_ERR); + } + CHKiRet(pThis->cryprov.SetCnfParam(pThis->cryprovData, lst, CRYPROV_PARAMTYPE_DISK)); + + dbgprintf("loaded crypto provider %s, data instance at %p\n", + szDrvrName, pThis->cryprovData); + pThis->useCryprov = 1; +finalize_it: + RETiRet; +} /* apply all params from param block to queue. Must be called before * finalizing. This supports the v6 config system. Defaults were already @@ -2700,21 +2940,40 @@ queueCnfParamsSet(struct cnfparamvals *pvals) * function. */ rsRetVal -qqueueApplyCnfParam(qqueue_t *pThis, struct cnfparamvals *pvals) +qqueueApplyCnfParam(qqueue_t *pThis, struct nvlst *lst) { int i; + struct cnfparamvals *pvals; + + pvals = nvlstGetParams(lst, &pblk, NULL); + if(Debug) { + dbgprintf("queue param blk:\n"); + cnfparamsPrint(&pblk, pvals); + } for(i = 0 ; i < pblk.nParams ; ++i) { if(!pvals[i].bUsed) continue; if(!strcmp(pblk.descr[i].name, "queue.filename")) { pThis->pszFilePrefix = (uchar*) es_str2cstr(pvals[i].val.d.estr, NULL); pThis->lenFilePrefix = es_strlen(pvals[i].val.d.estr); + } else if(!strcmp(pblk.descr[i].name, "queue.cry.provider")) { + pThis->cryprovName = (uchar*) es_str2cstr(pvals[i].val.d.estr, NULL); + } else if(!strcmp(pblk.descr[i].name, "queue.spooldirectory")) { + free(pThis->pszSpoolDir); + pThis->pszSpoolDir = (uchar*) es_str2cstr(pvals[i].val.d.estr, NULL); + pThis->lenSpoolDir = es_strlen(pvals[i].val.d.estr); + if(pThis->pszSpoolDir[pThis->lenSpoolDir-1] == '/') { + pThis->pszSpoolDir[pThis->lenSpoolDir-1] = '\0'; + --pThis->lenSpoolDir; + parser_errmsg("queue.spooldirectory must not end with '/', " + "corrected to '%s'", pThis->pszSpoolDir); + } } else if(!strcmp(pblk.descr[i].name, "queue.size")) { pThis->iMaxQueueSize = pvals[i].val.d.n; } else if(!strcmp(pblk.descr[i].name, "queue.dequeuebatchsize")) { pThis->iDeqBatchSize = pvals[i].val.d.n; } else if(!strcmp(pblk.descr[i].name, "queue.maxdiskspace")) { - pThis->iMaxFileSize = pvals[i].val.d.n; + pThis->sizeOnDiskMax = pvals[i].val.d.n; } else if(!strcmp(pblk.descr[i].name, "queue.highwatermark")) { pThis->iHighWtrMrk = pvals[i].val.d.n; } else if(!strcmp(pblk.descr[i].name, "queue.lowwatermark")) { @@ -2760,6 +3019,27 @@ qqueueApplyCnfParam(qqueue_t *pThis, struct cnfparamvals *pvals) "param '%s'\n", pblk.descr[i].name); } } + if(pThis->qType == QUEUETYPE_DISK) { + if(pThis->pszFilePrefix == NULL) { + errmsg.LogError(0, RS_RET_QUEUE_DISK_NO_FN, "error on queue '%s', disk mode selected, but " + "no queue file name given; queue type changed to 'linkedList'", + obj.GetName((obj_t*) pThis)); + pThis->qType = QUEUETYPE_LINKEDLIST; + } + } + + if(pThis->pszFilePrefix == NULL && pThis->cryprovName != NULL) { + errmsg.LogError(0, RS_RET_QUEUE_CRY_DISK_ONLY, "error on queue '%s', crypto provider can " + "only be set for disk or disk assisted queue - ignored", + obj.GetName((obj_t*) pThis)); + free(pThis->cryprovName); + pThis->cryprovName = NULL; + } + + if(pThis->cryprovName != NULL) { + initCryprov(pThis, lst); + } + cnfparamvalsDestruct(pvals, &pblk); return RS_RET_OK; } @@ -2781,6 +3061,7 @@ DEFpropSetMeth(qqueue, iFullDlyMrk, int) DEFpropSetMeth(qqueue, iDiscardSeverity, int) DEFpropSetMeth(qqueue, iLightDlyMrk, int) DEFpropSetMeth(qqueue, bIsDA, int) +DEFpropSetMeth(qqueue, iNumWorkerThreads, int) DEFpropSetMeth(qqueue, iMinMsgsPerWrkr, int) DEFpropSetMeth(qqueue, bSaveOnShutdown, int) DEFpropSetMeth(qqueue, pAction, action_t*) @@ -2804,6 +3085,9 @@ static rsRetVal qqueueSetProperty(qqueue_t *pThis, var_t *pProp) if(isProp("iQueueSize")) { pThis->iQueueSize = pProp->val.num; +# ifdef ENABLE_IMDIAG + iOverallQueueSize += pThis->iQueueSize; +# endif } else if(isProp("tVars.disk.sizeOnDisk")) { pThis->tVars.disk.sizeOnDisk = pProp->val.num; } else if(isProp("qType")) { diff --git a/runtime/queue.h b/runtime/queue.h index 886fac8..902c3d9 100644 --- a/runtime/queue.h +++ b/runtime/queue.h @@ -30,6 +30,7 @@ #include "batch.h" #include "stream.h" #include "statsobj.h" +#include "cryprov.h" /* support for the toDelete list */ typedef struct toDeleteLst_s toDeleteLst_t; @@ -102,11 +103,10 @@ struct queue_s { * the user really wanted...). -- rgerhards, 2008-04-02 */ /* end dequeue time window */ - rsRetVal (*pConsumer)(void *,batch_t*,int*); /* user-supplied consumer function for dequeued messages */ + rsRetVal (*pConsumer)(void *,batch_t*, wti_t*); /* user-supplied consumer function for dequeued messages */ /* calling interface for pConsumer: arg1 is the global user pointer from this structure, arg2 is the * user pointer array that was dequeued (actual sample: for actions, arg1 is the pAction and arg2 - * is pointer to an array of message message pointers), arg3 is a pointer to an interger which is zero - * during normal operations and one if the consumer must urgently shut down. + * is pointer to an array of message message pointers) */ /* type-specific handlers (set during construction) */ rsRetVal (*qConstruct)(struct queue_s *pThis); @@ -121,7 +121,7 @@ struct queue_s { /* synchronization variables */ pthread_mutex_t mutThrdMgmt; /* mutex for the queue's thread management */ pthread_mutex_t *mut; /* mutex for enqueing and dequeueing messages */ - pthread_cond_t notFull, notEmpty; + pthread_cond_t notFull; pthread_cond_t belowFullDlyWtrMrk; /* below eFLOWCTL_FULL_DELAY watermark */ pthread_cond_t belowLightDlyWtrMrk; /* below eFLOWCTL_FULL_DELAY watermark */ int bThrdStateChanged; /* at least one thread state has changed if 1 */ @@ -168,14 +168,19 @@ struct queue_s { strm_t *pReadDel; /* current file for deleting */ } disk; } tVars; - DEF_ATOMIC_HELPER_MUT(mutQueueSize); - DEF_ATOMIC_HELPER_MUT(mutLogDeq); + sbool useCryprov; /* quicker than checkig ptr (1 vs 8 bytes!) */ + uchar *cryprovName; /* crypto provider to use */ + cryprov_if_t cryprov; /* ptr to crypto provider interface */ + void *cryprovData; /* opaque data ptr for provider use */ + uchar *cryprovNameFull;/* full internal crypto provider name */ + DEF_ATOMIC_HELPER_MUT(mutQueueSize) + DEF_ATOMIC_HELPER_MUT(mutLogDeq) /* for statistics subsystem */ statsobj_t *statsobj; - STATSCOUNTER_DEF(ctrEnqueued, mutCtrEnqueued); - STATSCOUNTER_DEF(ctrFull, mutCtrFull); - STATSCOUNTER_DEF(ctrFDscrd, mutCtrFDscrd); - STATSCOUNTER_DEF(ctrNFDscrd, mutCtrNFDscrd); + STATSCOUNTER_DEF(ctrEnqueued, mutCtrEnqueued) + STATSCOUNTER_DEF(ctrFull, mutCtrFull) + STATSCOUNTER_DEF(ctrFDscrd, mutCtrFDscrd) + STATSCOUNTER_DEF(ctrNFDscrd, mutCtrNFDscrd) int ctrMaxqsize; /* NOT guarded by a mutex */ }; @@ -189,17 +194,14 @@ struct queue_s { /* prototypes */ rsRetVal qqueueDestruct(qqueue_t **ppThis); -rsRetVal qqueueEnqMsgDirect(qqueue_t *pThis, msg_t *pMsg); rsRetVal qqueueEnqMsg(qqueue_t *pThis, flowControl_t flwCtlType, msg_t *pMsg); rsRetVal qqueueStart(qqueue_t *pThis); rsRetVal qqueueSetMaxFileSize(qqueue_t *pThis, size_t iMaxFileSize); rsRetVal qqueueSetFilePrefix(qqueue_t *pThis, uchar *pszPrefix, size_t iLenPrefix); rsRetVal qqueueConstruct(qqueue_t **ppThis, queueType_t qType, int iWorkerThreads, - int iMaxQueueSize, rsRetVal (*pConsumer)(void*,batch_t*, int*)); -rsRetVal qqueueEnqObjDirectBatch(qqueue_t *pThis, batch_t *pBatch); -rsRetVal qqueueDoCnfParams(struct nvlst *lst, struct cnfparamvals **ppvals); -int queueCnfParamsSet(struct cnfparamvals *pvals); -rsRetVal qqueueApplyCnfParam(qqueue_t *pThis, struct cnfparamvals *pvals); + int iMaxQueueSize, rsRetVal (*pConsumer)(void*,batch_t*, wti_t *)); +int queueCnfParamsSet(struct nvlst *lst); +rsRetVal qqueueApplyCnfParam(qqueue_t *pThis, struct nvlst *lst); void qqueueSetDefaultsRulesetQueue(qqueue_t *pThis); void qqueueSetDefaultsActionQueue(qqueue_t *pThis); void qqueueDbgPrint(qqueue_t *pThis); @@ -219,6 +221,7 @@ PROTOTYPEpropSetMeth(qqueue, iLowWtrMrk, int); PROTOTYPEpropSetMeth(qqueue, iDiscardMrk, int); PROTOTYPEpropSetMeth(qqueue, iDiscardSeverity, int); PROTOTYPEpropSetMeth(qqueue, iMinMsgsPerWrkr, int); +PROTOTYPEpropSetMeth(qqueue, iNumWorkerThreads, int); PROTOTYPEpropSetMeth(qqueue, bSaveOnShutdown, int); PROTOTYPEpropSetMeth(qqueue, pAction, action_t*); PROTOTYPEpropSetMeth(qqueue, iDeqSlowdown, int); @@ -226,4 +229,8 @@ PROTOTYPEpropSetMeth(qqueue, sizeOnDiskMax, int64); PROTOTYPEpropSetMeth(qqueue, iDeqBatchSize, int); #define qqueueGetID(pThis) ((unsigned long) pThis) +#ifdef ENABLE_IMDIAG +extern unsigned int iOverallQueueSize; +#endif + #endif /* #ifndef QUEUE_H_INCLUDED */ diff --git a/runtime/ratelimit.c b/runtime/ratelimit.c index d83da2d..23605b1 100644 --- a/runtime/ratelimit.c +++ b/runtime/ratelimit.c @@ -73,16 +73,8 @@ static inline rsRetVal doLastMessageRepeatedNTimes(ratelimit_t *ratelimit, msg_t *pMsg, msg_t **ppRepMsg) { int bNeedUnlockMutex = 0; - rsRetVal localRet; DEFiRet; - if((pMsg->msgFlags & NEEDS_PARSING) != 0) { - if((localRet = parser.ParseMsg(pMsg)) != RS_RET_OK) { - DBGPRINTF("Message discarded, parsing error %d\n", localRet); - ABORT_FINALIZE(RS_RET_DISCARDMSG); - } - } - if(ratelimit->bThreadSafe) { pthread_mutex_lock(&ratelimit->mut); bNeedUnlockMutex = 1; @@ -128,8 +120,8 @@ tellLostCnt(ratelimit_t *ratelimit) snprintf((char*)msgbuf, sizeof(msgbuf), "%s: %u messages lost due to rate-limiting", ratelimit->name, ratelimit->missed); - logmsgInternal(RS_RET_RATE_LIMITED, LOG_SYSLOG|LOG_INFO, msgbuf, 0); ratelimit->missed = 0; + logmsgInternal(RS_RET_RATE_LIMITED, LOG_SYSLOG|LOG_INFO, msgbuf, 0); } } @@ -150,16 +142,25 @@ withinRatelimit(ratelimit_t *ratelimit, time_t tt) goto finalize_it; } + /* we primarily need "NoTimeCache" mode for imjournal, as it + * sets the message generation time to the journal timestamp. + * As such, we do not get a proper indication of the actual + * message rate. To prevent this, we need to query local + * system time ourselvs. + */ + if(ratelimit->bNoTimeCache) + tt = time(NULL); + assert(ratelimit->burst != 0); if(ratelimit->begin == 0) ratelimit->begin = tt; - /* resume if we go out of out time window */ + /* resume if we go out of time window */ if(tt > ratelimit->begin + ratelimit->interval) { - tellLostCnt(ratelimit); ratelimit->begin = 0; ratelimit->done = 0; + tellLostCnt(ratelimit); } /* do actual limit check */ @@ -167,13 +168,13 @@ withinRatelimit(ratelimit_t *ratelimit, time_t tt) ratelimit->done++; ret = 1; } else { - if(ratelimit->missed == 0) { + ratelimit->missed++; + if(ratelimit->missed == 1) { snprintf((char*)msgbuf, sizeof(msgbuf), "%s: begin to drop messages due to rate-limiting", ratelimit->name); logmsgInternal(RS_RET_RATE_LIMITED, LOG_SYSLOG|LOG_INFO, msgbuf, 0); } - ratelimit->missed++; ret = 0; } @@ -200,8 +201,17 @@ rsRetVal ratelimitMsg(ratelimit_t *ratelimit, msg_t *pMsg, msg_t **ppRepMsg) { DEFiRet; + rsRetVal localRet; *ppRepMsg = NULL; + + if((pMsg->msgFlags & NEEDS_PARSING) != 0) { + if((localRet = parser.ParseMsg(pMsg)) != RS_RET_OK) { + DBGPRINTF("Message discarded, parsing error %d\n", localRet); + ABORT_FINALIZE(RS_RET_DISCARDMSG); + } + } + /* Only the messages having severity level at or below the * treshold (the value is >=) are subject to ratelimiting. */ if(ratelimit->interval && (pMsg->iSeverity >= ratelimit->severity)) { @@ -214,6 +224,10 @@ ratelimitMsg(ratelimit_t *ratelimit, msg_t *pMsg, msg_t **ppRepMsg) CHKiRet(doLastMessageRepeatedNTimes(ratelimit, pMsg, ppRepMsg)); } finalize_it: + if(Debug) { + if(iRet == RS_RET_DISCARDMSG) + dbgprintf("message discarded by ratelimiting\n"); + } RETiRet; } @@ -288,6 +302,8 @@ ratelimitNew(ratelimit_t **ppThis, char *modname, char *dynname) } /* pThis->severity == 0 - all messages are ratelimited */ pThis->bReduceRepeatMsgs = loadConf->globals.bReduceRepeatMsgs; + DBGPRINTF("ratelimit:%s:new ratelimiter:bReduceRepeatMsgs %d\n", + pThis->name, pThis->bReduceRepeatMsgs); *ppThis = pThis; finalize_it: RETiRet; @@ -318,6 +334,12 @@ ratelimitSetThreadSafe(ratelimit_t *ratelimit) ratelimit->bThreadSafe = 1; pthread_mutex_init(&ratelimit->mut, NULL); } +void +ratelimitSetNoTimeCache(ratelimit_t *ratelimit) +{ + ratelimit->bNoTimeCache = 1; + pthread_mutex_init(&ratelimit->mut, NULL); +} /* Severity level determines which messages are subject to * ratelimiting. Default (no value set) is all messages. @@ -368,4 +390,3 @@ ratelimitModInit(void) finalize_it: RETiRet; } - diff --git a/runtime/ratelimit.h b/runtime/ratelimit.h index a058b06..563777f 100644 --- a/runtime/ratelimit.h +++ b/runtime/ratelimit.h @@ -35,6 +35,7 @@ struct ratelimit_s { unsigned nsupp; /**< nbr of msgs suppressed */ msg_t *pMsg; sbool bThreadSafe; /**< do we need to operate in Thread-Safe mode? */ + sbool bNoTimeCache; /**< if we shall not used cached reception time */ pthread_mutex_t mut; /**< mutex if thread-safe operation desired */ }; @@ -42,6 +43,7 @@ struct ratelimit_s { rsRetVal ratelimitNew(ratelimit_t **ppThis, char *modname, char *dynname); void ratelimitSetThreadSafe(ratelimit_t *ratelimit); void ratelimitSetLinuxLike(ratelimit_t *ratelimit, unsigned short interval, unsigned short burst); +void ratelimitSetNoTimeCache(ratelimit_t *ratelimit); void ratelimitSetSeverity(ratelimit_t *ratelimit, intTiny severity); rsRetVal ratelimitMsg(ratelimit_t *ratelimit, msg_t *pMsg, msg_t **ppRep); rsRetVal ratelimitAddMsg(ratelimit_t *ratelimit, multi_submit_t *pMultiSub, msg_t *pMsg); diff --git a/runtime/rsconf.c b/runtime/rsconf.c index d8b81f1..729911d 100644 --- a/runtime/rsconf.c +++ b/runtime/rsconf.c @@ -2,7 +2,7 @@ * * Module begun 2011-04-19 by Rainer Gerhards * - * Copyright 2011-2012 Adiscon GmbH. + * Copyright 2011-2013 Adiscon GmbH. * * This file is part of the rsyslog runtime library. * @@ -83,7 +83,7 @@ rsconf_t *runConf = NULL;/* the currently running config */ rsconf_t *loadConf = NULL;/* the config currently being loaded (no concurrent config load supported!) */ /* hardcoded standard templates (used for defaults) */ -static uchar template_DebugFormat[] = "\"Debug line with all properties:\nFROMHOST: '%FROMHOST%', fromhost-ip: '%fromhost-ip%', HOSTNAME: '%HOSTNAME%', PRI: %PRI%,\nsyslogtag '%syslogtag%', programname: '%programname%', APP-NAME: '%APP-NAME%', PROCID: '%PROCID%', MSGID: '%MSGID%',\nTIMESTAMP: '%TIMESTAMP%', STRUCTURED-DATA: '%STRUCTURED-DATA%',\nmsg: '%msg%'\nescaped msg: '%msg:::drop-cc%'\ninputname: %inputname% rawmsg: '%rawmsg%'\n\n\""; +static uchar template_DebugFormat[] = "\"Debug line with all properties:\nFROMHOST: '%FROMHOST%', fromhost-ip: '%fromhost-ip%', HOSTNAME: '%HOSTNAME%', PRI: %PRI%,\nsyslogtag '%syslogtag%', programname: '%programname%', APP-NAME: '%APP-NAME%', PROCID: '%PROCID%', MSGID: '%MSGID%',\nTIMESTAMP: '%TIMESTAMP%', STRUCTURED-DATA: '%STRUCTURED-DATA%',\nmsg: '%msg%'\nescaped msg: '%msg:::drop-cc%'\ninputname: %inputname% rawmsg: '%rawmsg%'\n$!:%$!%\n$.:%$.%\n$/:%$/%\n\n\""; static uchar template_SyslogProtocol23Format[] = "\"<%PRI%>1 %TIMESTAMP:::date-rfc3339% %HOSTNAME% %APP-NAME% %PROCID% %MSGID% %STRUCTURED-DATA% %msg%\n\""; static uchar template_TraditionalFileFormat[] = "=RSYSLOG_TraditionalFileFormat"; static uchar template_FileFormat[] = "=RSYSLOG_FileFormat"; @@ -108,6 +108,16 @@ static struct cnfparamblk inppblk = inppdescr }; +static struct cnfparamdescr parserpdescr[] = { + { "type", eCmdHdlrString, CNFPARAM_REQUIRED }, + { "name", eCmdHdlrString, CNFPARAM_REQUIRED } +}; +static struct cnfparamblk parserpblk = + { CNFPARAMBLK_VERSION, + sizeof(parserpdescr)/sizeof(struct cnfparamdescr), + parserpdescr + }; + /* forward-definitions */ void cnfDoCfsysline(char *ln); @@ -124,15 +134,16 @@ BEGINobjConstruct(rsconf) /* be sure to specify the object type also in END macr pThis->templates.last = NULL; pThis->templates.lastStatic = NULL; pThis->actions.nbrActions = 0; + lookupInitCnf(&pThis->lu_tabs); CHKiRet(llInit(&pThis->rulesets.llRulesets, rulesetDestructForLinkedList, rulesetKeyDestruct, strcasecmp)); /* queue params */ - pThis->globals.mainQ.iMainMsgQueueSize = 10000; - pThis->globals.mainQ.iMainMsgQHighWtrMark = 8000; - pThis->globals.mainQ.iMainMsgQLowWtrMark = 2000; - pThis->globals.mainQ.iMainMsgQDiscardMark = 9800; + pThis->globals.mainQ.iMainMsgQueueSize = 100000; + pThis->globals.mainQ.iMainMsgQHighWtrMark = 80000; + pThis->globals.mainQ.iMainMsgQLowWtrMark = 20000; + pThis->globals.mainQ.iMainMsgQDiscardMark = 98000; pThis->globals.mainQ.iMainMsgQDiscardSeverity = 8; - pThis->globals.mainQ.iMainMsgQueueNumWorkers = 1; + pThis->globals.mainQ.iMainMsgQueueNumWorkers = 2; pThis->globals.mainQ.MainMsgQueType = QUEUETYPE_FIXED_ARRAY; pThis->globals.mainQ.pszMainMsgQFName = NULL; pThis->globals.mainQ.iMainMsgQueMaxFileSize = 1024*1024; @@ -142,10 +153,10 @@ BEGINobjConstruct(rsconf) /* be sure to specify the object type also in END macr pThis->globals.mainQ.iMainMsgQtoActShutdown = 1000; pThis->globals.mainQ.iMainMsgQtoEnq = 2000; pThis->globals.mainQ.iMainMsgQtoWrkShutdown = 60000; - pThis->globals.mainQ.iMainMsgQWrkMinMsgs = 100; + pThis->globals.mainQ.iMainMsgQWrkMinMsgs = 40000; pThis->globals.mainQ.iMainMsgQDeqSlowdown = 0; pThis->globals.mainQ.iMainMsgQueMaxDiskSpace = 0; - pThis->globals.mainQ.iMainMsgQueDeqBatchSize = 32; + pThis->globals.mainQ.iMainMsgQueDeqBatchSize = 256; pThis->globals.mainQ.bMainMsgQSaveOnShutdown = 1; pThis->globals.mainQ.iMainMsgQueueDeqtWinFromHr = 0; pThis->globals.mainQ.iMainMsgQueueDeqtWinToHr = 25; @@ -253,89 +264,52 @@ CODESTARTobjDebugPrint(rsconf) ENDobjDebugPrint(rsconf) -/* This function returns the current date in different - * variants. It is used to construct the $NOW series of - * system properties. The returned buffer must be freed - * by the caller when no longer needed. If the function - * can not allocate memory, it returns a NULL pointer. - * TODO: this was taken from msg.c and we should consolidate it with the code - * there. This is especially important when we increase the number of system - * variables (what we definitely want to do). - */ -typedef enum ENOWType { NOW_NOW, NOW_YEAR, NOW_MONTH, NOW_DAY, NOW_HOUR, NOW_MINUTE } eNOWType; -static rsRetVal -getNOW(eNOWType eNow, es_str_t **estr) +rsRetVal +parserProcessCnf(struct cnfobj *o) { + struct cnfparamvals *pvals; + modInfo_t *pMod; + uchar *cnfModName = NULL; + uchar *parserName = NULL; + int paramIdx; + void *parserInst; + parser_t *myparser; DEFiRet; - uchar szBuf[16]; - struct syslogTime t; - es_size_t len; - - datetime.getCurrTime(&t, NULL); - switch(eNow) { - case NOW_NOW: - len = snprintf((char*) szBuf, sizeof(szBuf)/sizeof(uchar), - "%4.4d-%2.2d-%2.2d", t.year, t.month, t.day); - break; - case NOW_YEAR: - len = snprintf((char*) szBuf, sizeof(szBuf)/sizeof(uchar), "%4.4d", t.year); - break; - case NOW_MONTH: - len = snprintf((char*) szBuf, sizeof(szBuf)/sizeof(uchar), "%2.2d", t.month); - break; - case NOW_DAY: - len = snprintf((char*) szBuf, sizeof(szBuf)/sizeof(uchar), "%2.2d", t.day); - break; - case NOW_HOUR: - len = snprintf((char*) szBuf, sizeof(szBuf)/sizeof(uchar), "%2.2d", t.hour); - break; - case NOW_MINUTE: - len = snprintf((char*) szBuf, sizeof(szBuf)/sizeof(uchar), "%2.2d", t.minute); - break; - default: - len = snprintf((char*) szBuf, sizeof(szBuf)/sizeof(uchar), "*invld eNow*"); - break; - } - - /* now create a string object out of it and hand that over to the var */ - *estr = es_newStrFromCStr((char*)szBuf, len); - - RETiRet; -} - + pvals = nvlstGetParams(o->nvlst, &parserpblk, NULL); + if(pvals == NULL) { + ABORT_FINALIZE(RS_RET_CONFIG_ERROR); + } + DBGPRINTF("input param blk after parserProcessCnf:\n"); + cnfparamsPrint(&parserpblk, pvals); + paramIdx = cnfparamGetIdx(&parserpblk, "name"); + parserName = (uchar*)es_str2cstr(pvals[paramIdx].val.d.estr, NULL); + if(parser.FindParser(&myparser, parserName) != RS_RET_PARSER_NOT_FOUND) { + errmsg.LogError(0, RS_RET_PARSER_NAME_EXISTS, + "parser module name '%s' already exists", cnfModName); + ABORT_FINALIZE(RS_RET_PARSER_NAME_EXISTS); + } -static inline es_str_t * -getSysVar(char *name) -{ - es_str_t *estr = NULL; - rsRetVal iRet = RS_RET_OK; - - if(!strcmp(name, "now")) { - CHKiRet(getNOW(NOW_NOW, &estr)); - } else if(!strcmp(name, "year")) { - CHKiRet(getNOW(NOW_YEAR, &estr)); - } else if(!strcmp(name, "month")) { - CHKiRet(getNOW(NOW_MONTH, &estr)); - } else if(!strcmp(name, "day")) { - CHKiRet(getNOW(NOW_DAY, &estr)); - } else if(!strcmp(name, "hour")) { - CHKiRet(getNOW(NOW_HOUR, &estr)); - } else if(!strcmp(name, "minute")) { - CHKiRet(getNOW(NOW_MINUTE, &estr)); - } else if(!strcmp(name, "myhostname")) { - char *hn = (char*)glbl.GetLocalHostName(); - estr = es_newStrFromCStr(hn, strlen(hn)); - } else { - ABORT_FINALIZE(RS_RET_SYSVAR_NOT_FOUND); + paramIdx = cnfparamGetIdx(&parserpblk, "type"); + cnfModName = (uchar*)es_str2cstr(pvals[paramIdx].val.d.estr, NULL); + if((pMod = module.FindWithCnfName(loadConf, cnfModName, eMOD_PARSER)) == NULL) { + errmsg.LogError(0, RS_RET_MOD_UNKNOWN, "parser module name '%s' is unknown", cnfModName); + ABORT_FINALIZE(RS_RET_MOD_UNKNOWN); } -finalize_it: - if(iRet != RS_RET_OK) { - dbgprintf("getSysVar error iRet %d\n", iRet); - if(estr == NULL) - estr = es_newStrFromCStr("*ERROR*", sizeof("*ERROR*") - 1); + if(pMod->mod.pm.newParserInst == NULL) { + errmsg.LogError(0, RS_RET_MOD_NO_PARSER_STMT, + "parser module '%s' does not support parser() statement", cnfModName); + ABORT_FINALIZE(RS_RET_MOD_NO_INPUT_STMT); } - return estr; + CHKiRet(pMod->mod.pm.newParserInst(o->nvlst, &parserInst)); + + /* all well, so let's (try) to add parser to config */ + CHKiRet(parserConstructViaModAndName(pMod, parserName, parserInst)); +finalize_it: + free(cnfModName); + free(parserName); + cnfparamvalsDestruct(pvals, &parserpblk); + RETiRet; } @@ -377,6 +351,21 @@ finalize_it: extern int yylineno; void +parser_warnmsg(char *fmt, ...) +{ + va_list ap; + char errBuf[1024]; + + va_start(ap, fmt); + if(vsnprintf(errBuf, sizeof(errBuf), fmt, ap) == sizeof(errBuf)) + errBuf[sizeof(errBuf)-1] = '\0'; + errmsg.LogError(0, RS_RET_CONF_PARSE_WARNING, + "warning during parsing file %s, on or before line %d: %s", + cnfcurrfn, yylineno, errBuf); + va_end(ap); +} + +void parser_errmsg(char *fmt, ...) { va_list ap; @@ -399,6 +388,7 @@ yyerror(char *s) } void cnfDoObj(struct cnfobj *o) { + int bDestructObj = 1; int bChkUnuse = 1; dbgprintf("cnf:global:obj: "); @@ -407,12 +397,25 @@ void cnfDoObj(struct cnfobj *o) case CNFOBJ_GLOBAL: glblProcessCnf(o); break; + case CNFOBJ_TIMEZONE: + glblProcessTimezone(o); + break; + case CNFOBJ_MAINQ: + glblProcessMainQCnf(o); + bDestructObj = 0; + break; case CNFOBJ_MODULE: modulesProcessCnf(o); break; case CNFOBJ_INPUT: inputProcessCnf(o); break; + case CNFOBJ_LOOKUP_TABLE: + lookupProcessCnf(o); + break; + case CNFOBJ_PARSER: + parserProcessCnf(o); + break; case CNFOBJ_TPL: if(tplProcessCnf(o) != RS_RET_OK) parser_errmsg("error processing template object"); @@ -430,9 +433,11 @@ void cnfDoObj(struct cnfobj *o) o->objType); break; } - if(bChkUnuse) - nvlstChkUnused(o->nvlst); - cnfobjDestruct(o); + if(bDestructObj) { + if(bChkUnuse) + nvlstChkUnused(o->nvlst); + cnfobjDestruct(o); + } } void cnfDoScript(struct cnfstmt *script) @@ -468,30 +473,6 @@ void cnfDoBSDHost(char *ln) "solution (Block '%s')", ln); free(ln); } - -es_str_t* -cnfGetVar(char *name, void *usrptr) -{ - es_str_t *estr; - if(name[0] == '$') { - if(name[1] == '$') - estr = getSysVar(name+2); - else if(name[1] == '!') - estr = msgGetCEEVarNew((msg_t*) usrptr, name+2); - else - estr = msgGetMsgVarNew((msg_t*) usrptr, (uchar*)name+1); - } else { /* if this happens, we have a program logic error */ - estr = es_newStrFromCStr("err: var must start with $", - strlen("err: var must start with $")); - } - if(Debug) { - char *s; - s = es_str2cstr(estr, NULL); - dbgprintf("rainerscript: var '%s': '%s'\n", name, s); - free(s); - } - return estr; -} /*------------------------------ end interface to flex/bison parser ------------------------------*/ @@ -585,6 +566,7 @@ dropPrivileges(rsconf_t *cnf) static inline void tellCoreConfigLoadDone(void) { + DBGPRINTF("telling rsyslog core that config load for %p is done\n", loadConf); glblDoneLoadCnf(); } @@ -757,9 +739,18 @@ startInputModules(void) static inline rsRetVal activateMainQueue() { + struct cnfobj *mainqCnfObj; DEFiRet; + + mainqCnfObj = glbl.GetmainqCnfObj(); + DBGPRINTF("activateMainQueue: mainq cnf obj ptr is %p\n", mainqCnfObj); /* create message queue */ - CHKiRet_Hdlr(createMainQueue(&pMsgQueue, UCHAR_CONSTANT("main Q"), NULL)) { + iRet = createMainQueue(&pMsgQueue, UCHAR_CONSTANT("main Q"), + (mainqCnfObj == NULL) ? NULL : mainqCnfObj->nvlst); + if(iRet == RS_RET_OK) { + iRet = startMainQueue(pMsgQueue); + } + if(iRet != RS_RET_OK) { /* no queue is fatal, we need to give up in that case... */ fprintf(stderr, "fatal error %d: could not create message queue - rsyslogd can not run!\n", iRet); FINALIZE; @@ -768,6 +759,7 @@ activateMainQueue() bHaveMainQueue = (ourConf->globals.mainQ.MainMsgQueType == QUEUETYPE_DIRECT) ? 0 : 1; DBGPRINTF("Main processing queue is initialized and running\n"); finalize_it: + glblDestructMainqCnfObj(); RETiRet; } @@ -821,6 +813,7 @@ activate(rsconf_t *cnf) tellModulesActivateConfig(); startInputModules(); CHKiRet(activateActions()); + CHKiRet(activateRulesetQueues()); CHKiRet(activateMainQueue()); /* finally let the inputs run... */ runInputModules(); @@ -1298,6 +1291,7 @@ ourConf = loadConf; // TODO: remove, once ourConf is gone! ABORT_FINALIZE(RS_RET_NO_ACTIONS); } tellLexEndParsing(); + DBGPRINTF("Number of actions in this configuration: %d\n", iActionNbr); rulesetOptimizeAll(loadConf); tellCoreConfigLoadDone(); diff --git a/runtime/rsconf.h b/runtime/rsconf.h index 484fec8..894c0d1 100644 --- a/runtime/rsconf.h +++ b/runtime/rsconf.h @@ -25,6 +25,7 @@ #include "linkedlist.h" #include "queue.h" +#include "lookup.h" /* --- configuration objects (the plan is to have ALL upper layers in this file) --- */ @@ -143,6 +144,7 @@ struct rsconf_s { globals_t globals; defaults_t defaults; templates_t templates; + lookup_tables_t lu_tabs; outchannels_t och; actions_t actions; rulesets_t rulesets; diff --git a/runtime/rsyslog.c b/runtime/rsyslog.c index 047dfa9..7f020d3 100644 --- a/runtime/rsyslog.c +++ b/runtime/rsyslog.c @@ -35,7 +35,7 @@ * * Module begun 2008-04-16 by Rainer Gerhards * - * Copyright 2008 Rainer Gerhards and Adiscon GmbH. + * Copyright 2008-2014 Rainer Gerhards and Adiscon GmbH. * * This file is part of the rsyslog runtime library. * @@ -59,6 +59,7 @@ #include <stdio.h> #include <stdlib.h> #include <assert.h> +#include <liblogging/stdlog.h> #include "rsyslog.h" #include "obj.h" @@ -74,6 +75,7 @@ #include "prop.h" #include "ruleset.h" #include "parser.h" +#include "lookup.h" #include "strgen.h" #include "statsobj.h" #include "atomic.h" @@ -85,11 +87,11 @@ int default_thr_sched_policy; #endif /* forward definitions */ -static rsRetVal dfltErrLogger(int, uchar *errMsg); +static void dfltErrLogger(const int, const int, const uchar *errMsg); /* globally visible static data - see comment in rsyslog.h for details */ uchar *glblModPath; /* module load path */ -rsRetVal (*glblErrLogger)(int, uchar*) = dfltErrLogger; /* the error logger to use by the errmsg module */ +void (*glblErrLogger)(const int, const int, const uchar*) = dfltErrLogger; /* the error logger to use by the errmsg module */ /* static data */ static int iRefCount = 0; /* our refcount - it MUST exist only once inside a process (not thread) @@ -101,24 +103,21 @@ static int iRefCount = 0; /* our refcount - it MUST exist only once inside a pro * default so that we can log errors during the intial phase, most importantly * during initialization. -- rgerhards. 2008-04-17 */ -static rsRetVal dfltErrLogger(int iErr, uchar *errMsg) +static void +dfltErrLogger(const int severity, const int iErr, const uchar *errMsg) { - DEFiRet; - fprintf(stderr, "rsyslog runtime error(%d): %s\n", iErr, errMsg); - RETiRet; + fprintf(stderr, "rsyslog runtime error(%d,%d): %s\n", severity, iErr, errMsg); } /* set the error log function * rgerhards, 2008-04-18 */ -rsRetVal -rsrtSetErrLogger(rsRetVal (*errLogger)(int, uchar*)) +void +rsrtSetErrLogger(void (*errLogger)(const int, const int, const uchar*)) { - DEFiRet; assert(errLogger != NULL); glblErrLogger = errLogger; - RETiRet; } @@ -138,6 +137,8 @@ rsrtInit(char **ppErrObj, obj_if_t *pObjIF) if(iRefCount == 0) { /* init runtime only if not yet done */ + stdlog_init(0); + stdlog_hdl = NULL; #ifdef HAVE_PTHREAD_SETSCHEDPARAM CHKiRet(pthread_getschedparam(pthread_self(), &default_thr_sched_policy, @@ -186,6 +187,8 @@ rsrtInit(char **ppErrObj, obj_if_t *pObjIF) CHKiRet(strgenClassInit(NULL)); if(ppErrObj != NULL) *ppErrObj = "rsconf"; CHKiRet(rsconfClassInit(NULL)); + if(ppErrObj != NULL) *ppErrObj = "lookup"; + CHKiRet(lookupClassInit()); /* dummy "classes" */ if(ppErrObj != NULL) *ppErrObj = "str"; diff --git a/runtime/rsyslog.h b/runtime/rsyslog.h index a901d2e..f7d5b8a 100644 --- a/runtime/rsyslog.h +++ b/runtime/rsyslog.h @@ -3,7 +3,7 @@ * * Begun 2005-09-15 RGerhards * - * Copyright (C) 2005-2008 by Rainer Gerhards and Adiscon GmbH + * Copyright (C) 2005-2014 by Rainer Gerhards and Adiscon GmbH * * This file is part of the rsyslog runtime library. * @@ -49,6 +49,7 @@ #define CONF_PROGNAME_BUFSIZE 16 #define CONF_HOSTNAME_BUFSIZE 32 #define CONF_PROP_BUFSIZE 16 /* should be close to sizeof(ptr) or lighly above it */ +#define CONF_IPARAMS_BUFSIZE 16 /* initial size of iparams array in wti (is automatically extended) */ #define CONF_MIN_SIZE_FOR_COMPRESS 60 /* config param: minimum message size to try compression. The smaller * the message, the less likely is any compression gain. We check for * gain before we submit the message. But to do so we still need to @@ -76,19 +77,58 @@ * # End Config Settings # * * ############################################################# */ -/* portability: not all platforms have these defines, so we - * define them here if they are missing. -- rgerhards, 2008-03-04 +/* make sure we uses consistent macros, no matter what the + * platform gives us. */ -#ifndef LOG_MAKEPRI -# define LOG_MAKEPRI(fac, pri) (((fac) << 3) | (pri)) -#endif -#ifndef LOG_PRI -# define LOG_PRI(p) ((p) & LOG_PRIMASK) -#endif -#ifndef LOG_FAC -# define LOG_FAC(p) (((p) & LOG_FACMASK) >> 3) -#endif - +#define LOG_NFACILITIES 24+1 /* plus one for our special "invld" facility! */ +#define LOG_MAXPRI 191 /* highest supported valid PRI value --> RFC3164, RFC5424 */ +#undef LOG_MAKEPRI +#define LOG_PRI_INVLD LOG_INVLD|LOG_DEBUG /* PRI is invalid --> special "invld.=debug" PRI code (rsyslog-specific) */ + +#define LOG_EMERG 0 /* system is unusable */ +#define LOG_ALERT 1 /* action must be taken immediately */ +#define LOG_CRIT 2 /* critical conditions */ +#define LOG_ERR 3 /* error conditions */ +#define LOG_WARNING 4 /* warning conditions */ +#define LOG_NOTICE 5 /* normal but significant condition */ +#define LOG_INFO 6 /* informational */ +#define LOG_DEBUG 7 /* debug-level messages */ + +#define LOG_KERN (0<<3) /* kernel messages */ +#define LOG_USER (1<<3) /* random user-level messages */ +#define LOG_MAIL (2<<3) /* mail system */ +#define LOG_DAEMON (3<<3) /* system daemons */ +#define LOG_AUTH (4<<3) /* security/authorization messages */ +#define LOG_SYSLOG (5<<3) /* messages generated internally by syslogd */ +#define LOG_LPR (6<<3) /* line printer subsystem */ +#define LOG_NEWS (7<<3) /* network news subsystem */ +#define LOG_UUCP (8<<3) /* UUCP subsystem */ +#define LOG_CRON (9<<3) /* clock daemon */ +#define LOG_AUTHPRIV (10<<3) /* security/authorization messages (private) */ +#define LOG_FTP (11<<3) /* ftp daemon */ +#define LOG_LOCAL0 (16<<3) /* reserved for local use */ +#define LOG_LOCAL1 (17<<3) /* reserved for local use */ +#define LOG_LOCAL2 (18<<3) /* reserved for local use */ +#define LOG_LOCAL3 (19<<3) /* reserved for local use */ +#define LOG_LOCAL4 (20<<3) /* reserved for local use */ +#define LOG_LOCAL5 (21<<3) /* reserved for local use */ +#define LOG_LOCAL6 (22<<3) /* reserved for local use */ +#define LOG_LOCAL7 (23<<3) /* reserved for local use */ +#define LOG_FAC_INVLD 24 +#define LOG_INVLD (LOG_FAC_INVLD<<3) /* invalid facility/PRI code */ + +/* we need to use a function to avoid side-effects. This MUST guard + * against invalid facility values. rgerhards, 2014-09-16 + */ +static inline int pri2fac(const int pri) +{ + int fac = pri >> 3; + return (fac > 23) ? LOG_FAC_INVLD : fac; +} +static inline int pri2sev(const int pri) +{ + return pri & 0x07; +} /* the rsyslog core provides information about present feature to plugins * asking it. Below are feature-test macros which must be used to query @@ -102,50 +142,6 @@ #define _PATH_CONSOLE "/dev/console" #endif -/* properties are now encoded as (tiny) integers. I do not use an enum as I would like - * to keep the memory footprint small (and thus cache hits high). - * rgerhards, 2009-06-26 - */ -typedef uintTiny propid_t; -#define PROP_INVALID 0 -#define PROP_MSG 1 -#define PROP_TIMESTAMP 2 -#define PROP_HOSTNAME 3 -#define PROP_SYSLOGTAG 4 -#define PROP_RAWMSG 5 -#define PROP_INPUTNAME 6 -#define PROP_FROMHOST 7 -#define PROP_FROMHOST_IP 8 -#define PROP_PRI 9 -#define PROP_PRI_TEXT 10 -#define PROP_IUT 11 -#define PROP_SYSLOGFACILITY 12 -#define PROP_SYSLOGFACILITY_TEXT 13 -#define PROP_SYSLOGSEVERITY 14 -#define PROP_SYSLOGSEVERITY_TEXT 15 -#define PROP_TIMEGENERATED 16 -#define PROP_PROGRAMNAME 17 -#define PROP_PROTOCOL_VERSION 18 -#define PROP_STRUCTURED_DATA 19 -#define PROP_APP_NAME 20 -#define PROP_PROCID 21 -#define PROP_MSGID 22 -#define PROP_PARSESUCCESS 23 -#define PROP_SYS_NOW 150 -#define PROP_SYS_YEAR 151 -#define PROP_SYS_MONTH 152 -#define PROP_SYS_DAY 153 -#define PROP_SYS_HOUR 154 -#define PROP_SYS_HHOUR 155 -#define PROP_SYS_QHOUR 156 -#define PROP_SYS_MINUTE 157 -#define PROP_SYS_MYHOSTNAME 158 -#define PROP_CEE 200 -#define PROP_CEE_ALL_JSON 201 -#define PROP_SYS_BOM 159 -#define PROP_SYS_UPTIME 160 -#define PROP_UUID 161 - /* The error codes below are orginally "borrowed" from * liblogging. As such, we reserve values up to -2999 @@ -399,7 +395,7 @@ enum rsRetVal_ /** return value. All methods return this if not specified oth RS_RET_RULESET_EXISTS = -2306,/**< ruleset already exists */ RS_RET_DEPRECATED = -2307,/**< deprecated functionality is used */ RS_RET_DS_PROP_SEQ_ERR = -2308,/**< property sequence error deserializing object */ - RS_RET_TPL_INVLD_PROP = -2309,/**< property name error in template (unknown name) */ + RS_RET_INVLD_PROP = -2309,/**< property name error (unknown name) */ RS_RET_NO_RULEBASE = -2310,/**< mmnormalize: rulebase can not be found or otherwise invalid */ RS_RET_INVLD_MODE = -2311,/**< invalid mode specified in configuration */ RS_RET_INVLD_ANON_BITS = -2312,/**< mmanon: invalid number of bits to anonymize specified */ @@ -412,6 +408,30 @@ enum rsRetVal_ /** return value. All methods return this if not specified oth RS_RET_EI_INVLD_FILE = -2325,/**< header indicates the file is no .encinfo file */ RS_RET_CRY_INVLD_ALGO = -2326,/**< user specified invalid (unkonwn) crypto algorithm */ RS_RET_CRY_INVLD_MODE = -2327,/**< user specified invalid (unkonwn) crypto mode */ + RS_RET_QUEUE_DISK_NO_FN = -2328,/**< disk queue configured, but filename not set */ + RS_RET_CA_CERT_MISSING = -2329,/**< a CA cert is missing where one is required (e.g. TLS) */ + RS_RET_CERT_MISSING = -2330,/**< a cert is missing where one is required (e.g. TLS) */ + RS_RET_CERTKEY_MISSING = -2331,/**< a cert (private) key is missing where one is required (e.g. TLS) */ + RS_RET_STRUC_DATA_INVLD = -2349,/**< structured data is malformed */ + + /* up to 2350 reserved for 7.4 */ + RS_RET_QUEUE_CRY_DISK_ONLY = -2351,/**< crypto provider only supported for disk-associated queues */ + RS_RET_NO_DATA = -2352,/**< file has no data; more a state than a real error */ + RS_RET_RELP_AUTH_FAIL = -2353,/**< RELP peer authentication failed */ + RS_RET_ERR_UDPSEND = -2354,/**< sending msg via UDP failed */ + RS_RET_LAST_ERRREPORT = -2355,/**< module does not emit more error messages as limit is reached */ + RS_RET_READ_ERR = -2356,/**< read error occured (file i/o) */ + RS_RET_CONF_PARSE_WARNING = -2357,/**< warning parsing config file */ + RS_RET_CONF_WRN_FULLDLY_BELOW_HIGHWTR = -2358,/**< warning queue full delay mark below high wtr mark */ + RS_RET_RESUMED = -2359,/**< status: action was resumed (used for reporting) */ + RS_RET_RELP_NO_TLS = -2360,/**< librel does not support TLS (but TLS requested) */ + RS_RET_STATEFILE_WRONG_FNAME = -2361,/**< state file is for wrong file */ + + /* up to 2400 reserved for 7.5 & 7.6 */ + RS_RET_INVLD_OMOD = -2400, /**< invalid output module, does not provide proper interfaces */ + RS_RET_INVLD_INTERFACE_INPUT = -2401, /**< invalid value for "interface.input" parameter (ext progs) */ + RS_RET_PARSER_NAME_EXISTS = -2402, /**< parser name already exists */ + RS_RET_MOD_NO_PARSER_STMT = -2403, /**< (parser) module does not support parser() statement */ /* RainerScript error messages (range 1000.. 1999) */ RS_RET_SYSVAR_NOT_FOUND = 1001, /**< system variable could not be found (maybe misspelled) */ @@ -430,7 +450,12 @@ enum rsRetVal_ /** return value. All methods return this if not specified oth * Be sure to call the to-be-returned variable always "iRet" and * the function finalizer always "finalize_it". */ -#define CHKiRet(code) if((iRet = code) != RS_RET_OK) goto finalize_it +#if HAVE_BUILTIN_EXCEPT +# define CHKiRet(code) if(__builtin_expect(((iRet = code) != RS_RET_OK), 0)) goto finalize_it +#else +# define CHKiRet(code) if((iRet = code) != RS_RET_OK) goto finalize_it +#endif + /* macro below is to be used if we need our own handling, eg for cleanup */ #define CHKiRet_Hdlr(code) if((iRet = code) != RS_RET_OK) /* macro below is to handle failing malloc/calloc/strdup... which we almost always handle in the same way... */ @@ -495,6 +520,37 @@ extern pthread_attr_t default_thread_attr; extern int default_thr_sched_policy; #endif +/* The following structure defines immutable parameters which need to + * be passed as action parameters. + * + * Note that output plugins may request multiple templates. Let's say + * an output requests n templates. Than the overall table must hold + * n*nbrMsgs records, and each messages begins on a n-boundary. There + * is a macro defined below to access the proper element. + * + * WARNING: THIS STRUCTURE IS PART OF THE ***OUTPUT MODULE INTERFACE*** + * It is passed into the doCommit() function. Do NOT modify it until + * absolutely necessary - all output plugins need to be changed! + * + * If a change is "just" for internal working, consider adding a + * separate paramter outside of this structure. Of course, it is + * best to avoid this as well ;-) + * rgerhards, 2013-12-04 + */ +struct __attribute__ ((__packed__)) actWrkrIParams { + uchar *param; + uint32_t lenBuf; /* length of string buffer (if string ptr) */ + uint32_t lenStr; /* length of current string (if string ptr) */ +}; + +/* macro to access actWrkrIParams base object: + * param is ptr to base address + * nActTpls is the number of templates the action has requested + * iMsg is the message index + * iTpl is the template index + * This macro can be used for read and write access. + */ +#define actParam(param, nActTpls, iMsg, iTpl) (param[(iMsg*nActTpls)+iTpl]) /* for the time being, we do our own portability handling here. It * looks like autotools either does not yet support checks for it, or @@ -530,13 +586,13 @@ void dbgprintf(char *, ...) __attribute__((format(printf, 1, 2))); * add them. -- rgerhards, 2008-04-17 */ extern uchar *glblModPath; /* module load path */ -extern rsRetVal (*glblErrLogger)(int, uchar*); +extern void (*glblErrLogger)(const int, const int, const uchar*); /* some runtime prototypes */ rsRetVal rsrtInit(char **ppErrObj, obj_if_t *pObjIF); rsRetVal rsrtExit(void); int rsrtIsInit(void); -rsRetVal rsrtSetErrLogger(rsRetVal (*errLogger)(int, uchar*)); +void rsrtSetErrLogger(void (*errLogger)(const int, const int, const uchar*)); /* this define below is (later) intended to be used to implement empty * structs. TODO: check if compilers supports this and, if not, define diff --git a/runtime/ruleset.c b/runtime/ruleset.c index e334893..8de7225 100644 --- a/runtime/ruleset.c +++ b/runtime/ruleset.c @@ -48,6 +48,7 @@ #include "rainerscript.h" #include "srUtils.h" #include "modules.h" +#include "wti.h" #include "dirty.h" /* for main ruleset queue creation */ /* static data */ @@ -67,8 +68,8 @@ static struct cnfparamblk rspblk = }; /* forward definitions */ -static rsRetVal processBatch(batch_t *pBatch); -static rsRetVal scriptExec(struct cnfstmt *root, batch_t *pBatch, sbool *active); +static rsRetVal processBatch(batch_t *pBatch, wti_t *pWti); +static rsRetVal scriptExec(struct cnfstmt *root, msg_t *pMsg, wti_t *pWti); /* ---------- linked-list key handling functions (ruleset) ---------- */ @@ -160,226 +161,130 @@ finalize_it: RETiRet; } - -/* This function is similar to processBatch(), but works on a batch that - * contains rules from multiple rulesets. In this case, we can not push - * the whole batch through the ruleset. Instead, we examine it and - * partition it into sub-rulesets which we then push through the system. - * rgerhards, 2010-06-15 - */ -static inline rsRetVal -processBatchMultiRuleset(batch_t *pBatch) +/* driver to iterate over all rulesets */ +DEFFUNC_llExecFunc(doActivateRulesetQueues) { - ruleset_t *currRuleset; - batch_t snglRuleBatch; - int i; - int iStart; /* start index of partial batch */ - int iNew; /* index for new (temporary) batch */ - int bHaveUnprocessed; /* do we (still) have unprocessed entries? (loop term predicate) */ DEFiRet; - - do { - bHaveUnprocessed = 0; - /* search for first unprocessed element */ - for(iStart = 0 ; iStart < pBatch->nElem && pBatch->eltState[iStart] == BATCH_STATE_DISC ; ++iStart) - /* just search, no action */; - if(iStart == pBatch->nElem) - break; /* everything processed */ - - /* prepare temporary batch */ - CHKiRet(batchInit(&snglRuleBatch, pBatch->nElem)); - snglRuleBatch.pbShutdownImmediate = pBatch->pbShutdownImmediate; - currRuleset = batchElemGetRuleset(pBatch, iStart); - iNew = 0; - for(i = iStart ; i < pBatch->nElem ; ++i) { - if(batchElemGetRuleset(pBatch, i) == currRuleset) { - /* for performance reasons, we copy only those members that we actually need */ - snglRuleBatch.pElem[iNew].pMsg = pBatch->pElem[i].pMsg; - snglRuleBatch.eltState[iNew] = pBatch->eltState[i]; - ++iNew; - /* We indicate the element also as done, so it will not be processed again */ - pBatch->eltState[i] = BATCH_STATE_DISC; - } else { - bHaveUnprocessed = 1; - } - } - snglRuleBatch.nElem = iNew; /* was left just right by the for loop */ - batchSetSingleRuleset(&snglRuleBatch, 1); - /* process temp batch */ - processBatch(&snglRuleBatch); - batchFree(&snglRuleBatch); - } while(bHaveUnprocessed == 1); - -finalize_it: + ruleset_t* pThis = (ruleset_t*) pData; + dbgprintf("Activating Ruleset Queue[%p] for Ruleset %s\n", + pThis->pQueue, pThis->pszName); + if(pThis->pQueue != NULL) + startMainQueue(pThis->pQueue); RETiRet; } - -/* return a new "active" structure for the batch. Free with freeActive(). */ -static inline sbool *newActive(batch_t *pBatch) +/* activate all ruleset queues */ +rsRetVal +activateRulesetQueues() { - return malloc(sizeof(sbool) * batchNumMsgs(pBatch)); - + DEFiRet; + + llExecFunc(&(runConf->rulesets.llRulesets), doActivateRulesetQueues, NULL); + + RETiRet; } -static inline void freeActive(sbool *active) { free(active); } -/* for details, see scriptExec() header comment! */ -/* call action for all messages with filter on */ static rsRetVal -execAct(struct cnfstmt *stmt, batch_t *pBatch, sbool *active) +execAct(struct cnfstmt *stmt, msg_t *pMsg, wti_t *pWti) { DEFiRet; -dbgprintf("RRRR: execAct [%s]: batch of %d elements, active %p\n", modGetName(stmt->d.act->pMod), batchNumMsgs(pBatch), active); - pBatch->active = active; - stmt->d.act->submitToActQ(stmt->d.act, pBatch); + if(stmt->d.act->bDisabled) { + DBGPRINTF("action %d died, do NOT execute\n", stmt->d.act->iActionNbr); + FINALIZE; + } + + DBGPRINTF("executing action %d\n", stmt->d.act->iActionNbr); + stmt->d.act->submitToActQ(stmt->d.act, pWti, pMsg); + if(iRet != RS_RET_DISCARDMSG) { + /* note: we ignore the error code here, as we do NEVER want to + * stop script execution due to action return code + */ + iRet = RS_RET_OK; + } +finalize_it: RETiRet; } static rsRetVal -execSet(struct cnfstmt *stmt, batch_t *pBatch, sbool *active) +execSet(struct cnfstmt *stmt, msg_t *pMsg) { - int i; struct var result; DEFiRet; - for(i = 0 ; i < batchNumMsgs(pBatch) && !*(pBatch->pbShutdownImmediate) ; ++i) { - if( pBatch->eltState[i] != BATCH_STATE_DISC - && (active == NULL || active[i])) { - cnfexprEval(stmt->d.s_set.expr, &result, pBatch->pElem[i].pMsg); - msgSetJSONFromVar(pBatch->pElem[i].pMsg, stmt->d.s_set.varname, - &result); - varDelete(&result); - } - } + cnfexprEval(stmt->d.s_set.expr, &result, pMsg); + msgSetJSONFromVar(pMsg, stmt->d.s_set.varname, &result); + varDelete(&result); RETiRet; } static rsRetVal -execUnset(struct cnfstmt *stmt, batch_t *pBatch, sbool *active) +execUnset(struct cnfstmt *stmt, msg_t *pMsg) { - int i; DEFiRet; - for(i = 0 ; i < batchNumMsgs(pBatch) && !*(pBatch->pbShutdownImmediate) ; ++i) { - if( pBatch->eltState[i] != BATCH_STATE_DISC - && (active == NULL || active[i])) { - msgUnsetJSON(pBatch->pElem[i].pMsg, stmt->d.s_unset.varname); - } - } + msgDelJSON(pMsg, stmt->d.s_unset.varname); RETiRet; } -/* for details, see scriptExec() header comment! */ -/* "stop" simply discards the filtered items - it's just a (hopefully more intuitive - * shortcut for users. - */ static rsRetVal -execStop(batch_t *pBatch, sbool *active) +execCall(struct cnfstmt *stmt, msg_t *pMsg, wti_t *pWti) { - int i; DEFiRet; - for(i = 0 ; i < batchNumMsgs(pBatch) && !*(pBatch->pbShutdownImmediate) ; ++i) { - if( pBatch->eltState[i] != BATCH_STATE_DISC - && (active == NULL || active[i])) { - pBatch->eltState[i] = BATCH_STATE_DISC; - } + if(stmt->d.s_call.ruleset == NULL) { + CHKiRet(scriptExec(stmt->d.s_call.stmt, pMsg, pWti)); + } else { + CHKmalloc(pMsg = MsgDup((msg_t*) pMsg)); + DBGPRINTF("CALL: forwarding message to async ruleset %p\n", + stmt->d.s_call.ruleset->pQueue); + MsgSetFlowControlType(pMsg, eFLOWCTL_NO_DELAY); + MsgSetRuleset(pMsg, stmt->d.s_call.ruleset); + /* Note: we intentionally use submitMsg2() here, as we process messages + * that were already run through the rate-limiter. + */ + submitMsg2(pMsg); } +finalize_it: RETiRet; } -/* for details, see scriptExec() header comment! */ -// save current filter, evaluate new one -// perform then (if any message) -// if ELSE given: -// set new filter, inverted -// perform else (if any messages) static rsRetVal -execIf(struct cnfstmt *stmt, batch_t *pBatch, sbool *active) +execIf(struct cnfstmt *stmt, msg_t *pMsg, wti_t *pWti) { - sbool *newAct; - int i; sbool bRet; - sbool allInactive = 1; DEFiRet; - newAct = newActive(pBatch); - for(i = 0 ; i < batchNumMsgs(pBatch) ; ++i) { - if(*(pBatch->pbShutdownImmediate)) - FINALIZE; - if(pBatch->eltState[i] == BATCH_STATE_DISC) - continue; /* will be ignored in any case */ - if(active == NULL || active[i]) { - bRet = cnfexprEvalBool(stmt->d.s_if.expr, pBatch->pElem[i].pMsg); - allInactive = 0; - } else - bRet = 0; - newAct[i] = bRet; - DBGPRINTF("batch: item %d: expr eval: %d\n", i, bRet); - } - - if(allInactive) { - DBGPRINTF("execIf: all batch elements are inactive, holding execution\n"); - freeActive(newAct); - FINALIZE; - } - - if(stmt->d.s_if.t_then != NULL) { - scriptExec(stmt->d.s_if.t_then, pBatch, newAct); - } - if(stmt->d.s_if.t_else != NULL) { - for(i = 0 ; i < batchNumMsgs(pBatch) ; ++i) { - if(*(pBatch->pbShutdownImmediate)) - FINALIZE; - if(pBatch->eltState[i] != BATCH_STATE_DISC - && (active == NULL || active[i])) - newAct[i] = !newAct[i]; - } - scriptExec(stmt->d.s_if.t_else, pBatch, newAct); + bRet = cnfexprEvalBool(stmt->d.s_if.expr, pMsg); + DBGPRINTF("if condition result is %d\n", bRet); + if(bRet) { + if(stmt->d.s_if.t_then != NULL) + CHKiRet(scriptExec(stmt->d.s_if.t_then, pMsg, pWti)); + } else { + if(stmt->d.s_if.t_else != NULL) + CHKiRet(scriptExec(stmt->d.s_if.t_else, pMsg, pWti)); } - freeActive(newAct); finalize_it: RETiRet; } -/* for details, see scriptExec() header comment! */ -static void -execPRIFILT(struct cnfstmt *stmt, batch_t *pBatch, sbool *active) +static rsRetVal +execPRIFILT(struct cnfstmt *stmt, msg_t *pMsg, wti_t *pWti) { - sbool *newAct; - msg_t *pMsg; int bRet; - int i; - newAct = newActive(pBatch); - for(i = 0 ; i < batchNumMsgs(pBatch) ; ++i) { - if(*(pBatch->pbShutdownImmediate)) - return; - if(pBatch->eltState[i] == BATCH_STATE_DISC) - continue; /* will be ignored in any case */ - pMsg = pBatch->pElem[i].pMsg; - if(active == NULL || active[i]) { - if( (stmt->d.s_prifilt.pmask[pMsg->iFacility] == TABLE_NOPRI) || - ((stmt->d.s_prifilt.pmask[pMsg->iFacility] - & (1<<pMsg->iSeverity)) == 0) ) - bRet = 0; - else - bRet = 1; - } else - bRet = 0; - newAct[i] = bRet; - DBGPRINTF("batch: item %d PRIFILT %d\n", i, newAct[i]); - } - - if(stmt->d.s_prifilt.t_then != NULL) { - scriptExec(stmt->d.s_prifilt.t_then, pBatch, newAct); - } - if(stmt->d.s_prifilt.t_else != NULL) { - for(i = 0 ; i < batchNumMsgs(pBatch) ; ++i) { - if(*(pBatch->pbShutdownImmediate)) - return; - if(pBatch->eltState[i] != BATCH_STATE_DISC - && (active == NULL || active[i])) - newAct[i] = !newAct[i]; - } - scriptExec(stmt->d.s_prifilt.t_else, pBatch, newAct); + DEFiRet; + if( (stmt->d.s_prifilt.pmask[pMsg->iFacility] == TABLE_NOPRI) || + ((stmt->d.s_prifilt.pmask[pMsg->iFacility] + & (1<<pMsg->iSeverity)) == 0) ) + bRet = 0; + else + bRet = 1; + + DBGPRINTF("PRIFILT condition result is %d\n", bRet); + if(bRet) { + if(stmt->d.s_prifilt.t_then != NULL) + CHKiRet(scriptExec(stmt->d.s_prifilt.t_then, pMsg, pWti)); + } else { + if(stmt->d.s_prifilt.t_else != NULL) + CHKiRet(scriptExec(stmt->d.s_prifilt.t_else, pMsg, pWti)); } - freeActive(newAct); +finalize_it: + RETiRet; } @@ -392,12 +297,11 @@ evalPROPFILT(struct cnfstmt *stmt, msg_t *pMsg) int bRet = 0; rs_size_t propLen; - if(stmt->d.s_propfilt.propID == PROP_INVALID) + if(stmt->d.s_propfilt.prop.id == PROP_INVALID) goto done; - pszPropVal = MsgGetProp(pMsg, NULL, stmt->d.s_propfilt.propID, - stmt->d.s_propfilt.propName, &propLen, - &pbMustBeFreed, NULL); + pszPropVal = MsgGetProp(pMsg, NULL, &stmt->d.s_propfilt.prop, + &propLen, &pbMustBeFreed, NULL); /* Now do the compares (short list currently ;)) */ switch(stmt->d.s_propfilt.operation ) { @@ -441,15 +345,18 @@ evalPROPFILT(struct cnfstmt *stmt, msg_t *pMsg) bRet = (bRet == 1) ? 0 : 1; if(Debug) { - char *cstr; - if(stmt->d.s_propfilt.propID == PROP_CEE) { - cstr = es_str2cstr(stmt->d.s_propfilt.propName, NULL); + if(stmt->d.s_propfilt.prop.id == PROP_CEE) { DBGPRINTF("Filter: check for CEE property '%s' (value '%s') ", - cstr, pszPropVal); - free(cstr); + stmt->d.s_propfilt.prop.name, pszPropVal); + } else if(stmt->d.s_propfilt.prop.id == PROP_LOCAL_VAR) { + DBGPRINTF("Filter: check for local var '%s' (value '%s') ", + stmt->d.s_propfilt.prop.name, pszPropVal); + } else if(stmt->d.s_propfilt.prop.id == PROP_GLOBAL_VAR) { + DBGPRINTF("Filter: check for global var '%s' (value '%s') ", + stmt->d.s_propfilt.prop.name, pszPropVal); } else { DBGPRINTF("Filter: check for property '%s' (value '%s') ", - propIDToName(stmt->d.s_propfilt.propID), pszPropVal); + propIDToName(stmt->d.s_propfilt.prop.id), pszPropVal); } if(stmt->d.s_propfilt.isNegated) DBGPRINTF("NOT "); @@ -472,79 +379,67 @@ done: return bRet; } -/* for details, see scriptExec() header comment! */ -static void -execPROPFILT(struct cnfstmt *stmt, batch_t *pBatch, sbool *active) +static rsRetVal +execPROPFILT(struct cnfstmt *stmt, msg_t *pMsg, wti_t *pWti) { - sbool *thenAct; sbool bRet; - int i; - thenAct = newActive(pBatch); - for(i = 0 ; i < batchNumMsgs(pBatch) ; ++i) { - if(*(pBatch->pbShutdownImmediate)) - return; - if(pBatch->eltState[i] == BATCH_STATE_DISC) - continue; /* will be ignored in any case */ - if(active == NULL || active[i]) { - bRet = evalPROPFILT(stmt, pBatch->pElem[i].pMsg); - } else - bRet = 0; - thenAct[i] = bRet; - DBGPRINTF("batch: item %d PROPFILT %d\n", i, thenAct[i]); - } + DEFiRet; - scriptExec(stmt->d.s_propfilt.t_then, pBatch, thenAct); - freeActive(thenAct); + bRet = evalPROPFILT(stmt, pMsg); + DBGPRINTF("PROPFILT condition result is %d\n", bRet); + if(bRet) + CHKiRet(scriptExec(stmt->d.s_propfilt.t_then, pMsg, pWti)); +finalize_it: + RETiRet; } /* The rainerscript execution engine. It is debatable if that would be better * contained in grammer/rainerscript.c, HOWEVER, that file focusses primarily * on the parsing and object creation part. So as an actual executor, it is * better suited here. - * param active: if NULL, all messages are active (to be processed), if non-null - * this is an array of the same size as the batch. If 1, the message - * is to be processed, otherwise not. - * NOTE: this function must receive batches which contain a single ruleset ONLY! * rgerhards, 2012-09-04 */ static rsRetVal -scriptExec(struct cnfstmt *root, batch_t *pBatch, sbool *active) +scriptExec(struct cnfstmt *root, msg_t *pMsg, wti_t *pWti) { - DEFiRet; struct cnfstmt *stmt; + DEFiRet; for(stmt = root ; stmt != NULL ; stmt = stmt->next) { + if(*pWti->pbShutdownImmediate) { + DBGPRINTF("scriptExec: ShutdownImmediate set, " + "force terminating\n"); + ABORT_FINALIZE(RS_RET_FORCE_TERM); + } if(Debug) { - dbgprintf("scriptExec: batch of %d elements, active %p, active[0]:%d\n", - batchNumMsgs(pBatch), active, (active == NULL ? 1 : active[0])); cnfstmtPrintOnly(stmt, 2, 0); } switch(stmt->nodetype) { case S_NOP: break; case S_STOP: - execStop(pBatch, active); + ABORT_FINALIZE(RS_RET_DISCARDMSG); break; case S_ACT: - execAct(stmt, pBatch, active); + CHKiRet(execAct(stmt, pMsg, pWti)); break; case S_SET: - execSet(stmt, pBatch, active); + CHKiRet(execSet(stmt, pMsg)); break; case S_UNSET: - execUnset(stmt, pBatch, active); + CHKiRet(execUnset(stmt, pMsg)); break; case S_CALL: - scriptExec(stmt->d.s_call.stmt, pBatch, active); + CHKiRet(execCall(stmt, pMsg, pWti)); break; case S_IF: - execIf(stmt, pBatch, active); + CHKiRet(execIf(stmt, pMsg, pWti)); break; case S_PRIFILT: - execPRIFILT(stmt, pBatch, active); + CHKiRet(execPRIFILT(stmt, pMsg, pWti)); break; case S_PROPFILT: - execPROPFILT(stmt, pBatch, active); + CHKiRet(execPROPFILT(stmt, pMsg, pWti)); break; default: dbgprintf("error: unknown stmt type %u during exec\n", @@ -552,36 +447,43 @@ scriptExec(struct cnfstmt *root, batch_t *pBatch, sbool *active) break; } } +finalize_it: RETiRet; } /* Process (consume) a batch of messages. Calls the actions configured. - * If the whole batch uses a singel ruleset, we can process the batch as - * a whole. Otherwise, we need to process it slower, on a message-by-message - * basis (what can be optimized to a per-ruleset basis) - * rgerhards, 2005-10-13 + * This is called by MAIN queues. */ static rsRetVal -processBatch(batch_t *pBatch) +processBatch(batch_t *pBatch, wti_t *pWti) { - ruleset_t *pThis; + int i; + msg_t *pMsg; + ruleset_t *pRuleset; DEFiRet; - assert(pBatch != NULL); - - DBGPRINTF("processBatch: batch of %d elements must be processed\n", pBatch->nElem); - if(pBatch->bSingleRuleset) { - pThis = batchGetRuleset(pBatch); - if(pThis == NULL) - pThis = ourConf->rulesets.pDflt; - ISOBJ_TYPE_assert(pThis, ruleset); - CHKiRet(scriptExec(pThis->root, pBatch, NULL)); - } else { - CHKiRet(processBatchMultiRuleset(pBatch)); + + DBGPRINTF("processBATCH: batch of %d elements must be processed\n", pBatch->nElem); + + wtiResetExecState(pWti, pBatch); + + /* execution phase */ + for(i = 0 ; i < batchNumMsgs(pBatch) && !*(pWti->pbShutdownImmediate) ; ++i) { + pMsg = pBatch->pElem[i].pMsg; + DBGPRINTF("processBATCH: next msg %d: %.128s\n", i, pMsg->pszRawMsg); + pRuleset = (pMsg->pRuleset == NULL) ? ourConf->rulesets.pDflt : pMsg->pRuleset; + scriptExec(pRuleset->root, pMsg, pWti); + // TODO: think if we need a return state of scriptExec - most probably + // the answer is "no", as we need to process the batch in any case! + // TODO: we must refactor this! flag messages as committed + batchSetElemState(pBatch, i, BATCH_STATE_COMM); } -finalize_it: - DBGPRINTF("ruleset.ProcessMsg() returns %d\n", iRet); + /* commit phase */ + dbgprintf("END batch execution phase, entering to commit phase\n"); + actionCommitAllDirect(pWti); + + DBGPRINTF("processBATCH: batch of %d elements has been processed\n", pBatch->nElem); RETiRet; } @@ -742,7 +644,7 @@ CODESTARTobjDestruct(ruleset) parser.DestructParserList(&pThis->pParserLst); } free(pThis->pszName); - cnfstmtDestruct(pThis->root); + cnfstmtDestructLst(pThis->root); ENDobjDestruct(ruleset) @@ -928,7 +830,6 @@ rsRetVal rulesetProcessCnf(struct cnfobj *o) { struct cnfparamvals *pvals; - struct cnfparamvals *queueParams; rsRetVal localRet; uchar *rsName = NULL; uchar *parserName; @@ -952,14 +853,21 @@ rulesetProcessCnf(struct cnfobj *o) errmsg.LogError(0, RS_RET_RULESET_EXISTS, "error: ruleset '%s' specified more than once", rsName); - cnfstmtDestruct(o->script); + cnfstmtDestructLst(o->script); ABORT_FINALIZE(RS_RET_RULESET_EXISTS); } else if(localRet != RS_RET_NOT_FOUND) { ABORT_FINALIZE(localRet); } + CHKiRet(rulesetConstruct(&pRuleset)); - CHKiRet(rulesetSetName(pRuleset, rsName)); - CHKiRet(rulesetConstructFinalize(loadConf, pRuleset)); + if((localRet = rulesetSetName(pRuleset, rsName)) != RS_RET_OK) { + rulesetDestruct(&pRuleset); + ABORT_FINALIZE(localRet); + } + if((localRet = rulesetConstructFinalize(loadConf, pRuleset)) != RS_RET_OK) { + rulesetDestruct(&pRuleset); + ABORT_FINALIZE(localRet); + } addScript(pRuleset, o->script); /* we have only two params, so we do NOT do the usual param loop */ @@ -969,16 +877,15 @@ rulesetProcessCnf(struct cnfobj *o) for(i = 0 ; i < ar->nmemb ; ++i) { parserName = (uchar*)es_str2cstr(ar->arr[i], NULL); doRulesetAddParser(pRuleset, parserName); - free(parserName); + /* note parserName is freed in doRulesetAddParser()! */ } } /* pick up ruleset queue parameters */ - qqueueDoCnfParams(o->nvlst, &queueParams); - if(queueCnfParamsSet(queueParams)) { + if(queueCnfParamsSet(o->nvlst)) { rsname = (pRuleset->pszName == NULL) ? (uchar*) "[ruleset]" : pRuleset->pszName; DBGPRINTF("adding a ruleset-specific \"main\" queue for ruleset '%s'\n", rsname); - CHKiRet(createMainQueue(&pRuleset->pQueue, rsname, queueParams)); + CHKiRet(createMainQueue(&pRuleset->pQueue, rsname, o->nvlst)); } finalize_it: diff --git a/runtime/ruleset.h b/runtime/ruleset.h index cbf8243..d3dfd66 100644 --- a/runtime/ruleset.h +++ b/runtime/ruleset.h @@ -2,7 +2,7 @@ * * This implements rulesets within rsyslog. * - * Copyright 2009-2012 Rainer Gerhards and Adiscon GmbH. + * Copyright 2009-2013 Rainer Gerhards and Adiscon GmbH. * * This file is part of the rsyslog runtime library. * @@ -46,7 +46,7 @@ BEGINinterface(ruleset) /* name must also be changed in ENDinterface macro! */ rsRetVal (*Destruct)(ruleset_t **ppThis); rsRetVal (*DestructAllActions)(rsconf_t *conf); rsRetVal (*SetName)(ruleset_t *pThis, uchar *pszName); - rsRetVal (*ProcessBatch)(batch_t*); + rsRetVal (*ProcessBatch)(batch_t*, wti_t *); rsRetVal (*GetRuleset)(rsconf_t *conf, ruleset_t **ppThis, uchar*); rsRetVal (*SetDefaultRuleset)(rsconf_t *conf, uchar*); rsRetVal (*SetCurrRuleset)(rsconf_t *conf, uchar*); @@ -64,8 +64,9 @@ BEGINinterface(ruleset) /* name must also be changed in ENDinterface macro! */ /* AddRule() removed */ /*TODO:REMOVE*/rsRetVal (*IterateAllActions)(rsconf_t *conf, rsRetVal (*pFunc)(void*, void*), void* pParam); void (*AddScript)(ruleset_t *pThis, struct cnfstmt *script); + /* v8: changed processBatch interface */ ENDinterface(ruleset) -#define rulesetCURR_IF_VERSION 7 /* increment whenever you change the interface structure! */ +#define rulesetCURR_IF_VERSION 8 /* increment whenever you change the interface structure! */ /* prototypes */ @@ -90,6 +91,13 @@ rulesetGetName(ruleset_t *pRuleset) return pRuleset->pszName; } +/* returns 1 if the ruleset has a queue associtated, 0 if not */ +static inline int +rulesetHasQueue(ruleset_t *pRuleset) +{ + return pRuleset->pQueue == NULL ? 0 : 1; +} + /* we will most probably convert this module back to traditional C * calling sequence, so here we go... @@ -97,6 +105,7 @@ rulesetGetName(ruleset_t *pRuleset) rsRetVal rulesetGetRuleset(rsconf_t *conf, ruleset_t **ppRuleset, uchar *pszName); rsRetVal rulesetOptimizeAll(rsconf_t *conf); rsRetVal rulesetProcessCnf(struct cnfobj *o); +rsRetVal activateRulesetQueues(void); /* Set a current rule set to already-known pointer */ static inline void diff --git a/runtime/srUtils.h b/runtime/srUtils.h index 8626a4b..1dd2506 100644 --- a/runtime/srUtils.h +++ b/runtime/srUtils.h @@ -77,7 +77,7 @@ unsigned char *srUtilStrDup(unsigned char *pOld, size_t len); * for it. * added 2007-07-17 by rgerhards */ -int makeFileParentDirs(uchar *szFile, size_t lenFile, mode_t mode, uid_t uid, gid_t gid, int bFailOnChown); +int makeFileParentDirs(const uchar *const szFile, size_t lenFile, mode_t mode, uid_t uid, gid_t gid, int bFailOnChown); int execProg(uchar *program, int bWait, uchar *arg); void skipWhiteSpace(uchar **pp); rsRetVal genFileName(uchar **ppName, uchar *pDirName, size_t lenDirName, uchar *pFName, diff --git a/runtime/srutils.c b/runtime/srutils.c index 6a509b4..d7e71ed 100644 --- a/runtime/srutils.c +++ b/runtime/srutils.c @@ -86,6 +86,7 @@ syslogName_t syslogFacNames[] = { {"mark", LOG_MARK}, /* INTERNAL */ {"news", LOG_NEWS}, {"security", LOG_AUTH}, /* DEPRECATED */ + {"bsd_security", (13<<3) }, /* BSD-specific, unfortunatly with duplicate name... */ {"syslog", LOG_SYSLOG}, {"user", LOG_USER}, {"uucp", LOG_UUCP}, @@ -95,6 +96,7 @@ syslogName_t syslogFacNames[] = { #if defined(LOG_AUDIT) {"audit", LOG_AUDIT}, #endif + {"console", (14 << 3)}, /* BSD-specific priority */ {"local0", LOG_LOCAL0}, {"local1", LOG_LOCAL1}, {"local2", LOG_LOCAL2}, @@ -103,6 +105,7 @@ syslogName_t syslogFacNames[] = { {"local5", LOG_LOCAL5}, {"local6", LOG_LOCAL6}, {"local7", LOG_LOCAL7}, + {"invld", LOG_INVLD}, {NULL, -1}, }; @@ -189,7 +192,7 @@ uchar *srUtilStrDup(uchar *pOld, size_t len) * try because otherwise we would potentially run into an endless loop. * loop. -- rgerhards, 2010-03-25 */ -int makeFileParentDirs(uchar *szFile, size_t lenFile, mode_t mode, +int makeFileParentDirs(const uchar *const szFile, size_t lenFile, mode_t mode, uid_t uid, gid_t gid, int bFailOnChownFail) { uchar *p; diff --git a/runtime/statsobj.c b/runtime/statsobj.c index 2527561..edac7d4 100644 --- a/runtime/statsobj.c +++ b/runtime/statsobj.c @@ -142,7 +142,7 @@ finalize_it: * is called. */ static rsRetVal -addCounter(statsobj_t *pThis, uchar *ctrName, statsCtrType_t ctrType, void *pCtr) +addCounter(statsobj_t *pThis, uchar *ctrName, statsCtrType_t ctrType, int8_t flags, void *pCtr) { ctr_t *ctr; DEFiRet; @@ -151,6 +151,7 @@ addCounter(statsobj_t *pThis, uchar *ctrName, statsCtrType_t ctrType, void *pCtr ctr->next = NULL; ctr->prev = NULL; CHKmalloc(ctr->name = ustrdup(ctrName)); + ctr->flags = flags; ctr->ctrType = ctrType; switch(ctrType) { case ctrType_IntCtr: @@ -166,9 +167,24 @@ finalize_it: RETiRet; } +static inline void +resetResettableCtr(ctr_t *pCtr, int8_t bResetCtrs) +{ + if(bResetCtrs && (pCtr->flags & CTR_FLAG_RESETTABLE)) { + switch(pCtr->ctrType) { + case ctrType_IntCtr: + *(pCtr->val.pIntCtr) = 0; + break; + case ctrType_Int: + *(pCtr->val.pInt) = 0; + break; + } + } +} + /* get all the object's countes together as CEE. */ static rsRetVal -getStatsLineCEE(statsobj_t *pThis, cstr_t **ppcstr, int cee_cookie) +getStatsLineCEE(statsobj_t *pThis, cstr_t **ppcstr, int cee_cookie, int8_t bResetCtrs) { cstr_t *pcstr; ctr_t *pCtr; @@ -209,7 +225,7 @@ getStatsLineCEE(statsobj_t *pThis, cstr_t **ppcstr, int cee_cookie) } else { cstrAppendChar(pcstr, '}'); } - + resetResettableCtr(pCtr, bResetCtrs); } pthread_mutex_unlock(&pThis->mutCtr); @@ -223,7 +239,7 @@ finalize_it: /* get all the object's countes together with object name as one line. */ static rsRetVal -getStatsLine(statsobj_t *pThis, cstr_t **ppcstr) +getStatsLine(statsobj_t *pThis, cstr_t **ppcstr, int8_t bResetCtrs) { cstr_t *pcstr; ctr_t *pCtr; @@ -247,6 +263,7 @@ getStatsLine(statsobj_t *pThis, cstr_t **ppcstr) break; } cstrAppendChar(pcstr, ' '); + resetResettableCtr(pCtr, bResetCtrs); } pthread_mutex_unlock(&pThis->mutCtr); @@ -265,7 +282,7 @@ finalize_it: * line. If the callback reports an error, processing is stopped. */ static rsRetVal -getAllStatsLines(rsRetVal(*cb)(void*, cstr_t*), void *usrptr, statsFmtType_t fmt) +getAllStatsLines(rsRetVal(*cb)(void*, cstr_t*), void *usrptr, statsFmtType_t fmt, int8_t bResetCtrs) { statsobj_t *o; cstr_t *cstr; @@ -274,13 +291,13 @@ getAllStatsLines(rsRetVal(*cb)(void*, cstr_t*), void *usrptr, statsFmtType_t fmt for(o = objRoot ; o != NULL ; o = o->next) { switch(fmt) { case statsFmt_Legacy: - CHKiRet(getStatsLine(o, &cstr)); + CHKiRet(getStatsLine(o, &cstr, bResetCtrs)); break; case statsFmt_CEE: - CHKiRet(getStatsLineCEE(o, &cstr, 1)); + CHKiRet(getStatsLineCEE(o, &cstr, 1, bResetCtrs)); break; case statsFmt_JSON: - CHKiRet(getStatsLineCEE(o, &cstr, 0)); + CHKiRet(getStatsLineCEE(o, &cstr, 0, bResetCtrs)); break; } CHKiRet(cb(usrptr, cstr)); @@ -348,7 +365,7 @@ CODESTARTobjQueryInterface(statsobj) pIf->Destruct = statsobjDestruct; pIf->DebugPrint = statsobjDebugPrint; pIf->SetName = setName; - pIf->GetStatsLine = getStatsLine; + //pIf->GetStatsLine = getStatsLine; pIf->GetAllStatsLines = getAllStatsLines; pIf->AddCounter = addCounter; pIf->EnableStats = enableStats; diff --git a/runtime/statsobj.h b/runtime/statsobj.h index 14b3321..347f639 100644 --- a/runtime/statsobj.h +++ b/runtime/statsobj.h @@ -50,6 +50,9 @@ typedef enum statsFmtType_e { statsFmt_CEE } statsFmtType_t; +/* counter flags */ +#define CTR_FLAG_NONE 0 +#define CTR_FLAG_RESETTABLE 1 /* helper entity, the counter */ typedef struct ctr_s { @@ -59,6 +62,7 @@ typedef struct ctr_s { intctr_t *pIntCtr; int *pInt; } val; + int8_t flags; struct ctr_s *next, *prev; } ctr_t; @@ -82,15 +86,17 @@ BEGINinterface(statsobj) /* name must also be changed in ENDinterface macro! */ rsRetVal (*ConstructFinalize)(statsobj_t *pThis); rsRetVal (*Destruct)(statsobj_t **ppThis); rsRetVal (*SetName)(statsobj_t *pThis, uchar *name); - rsRetVal (*GetStatsLine)(statsobj_t *pThis, cstr_t **ppcstr); - rsRetVal (*GetAllStatsLines)(rsRetVal(*cb)(void*, cstr_t*), void *usrptr, statsFmtType_t fmt); - rsRetVal (*AddCounter)(statsobj_t *pThis, uchar *ctrName, statsCtrType_t ctrType, void *pCtr); + //rsRetVal (*GetStatsLine)(statsobj_t *pThis, cstr_t **ppcstr); + rsRetVal (*GetAllStatsLines)(rsRetVal(*cb)(void*, cstr_t*), void *usrptr, statsFmtType_t fmt, int8_t bResetCtr); + rsRetVal (*AddCounter)(statsobj_t *pThis, uchar *ctrName, statsCtrType_t ctrType, int8_t flags, void *pCtr); rsRetVal (*EnableStats)(void); ENDinterface(statsobj) -#define statsobjCURR_IF_VERSION 10 /* increment whenever you change the interface structure! */ +#define statsobjCURR_IF_VERSION 11 /* increment whenever you change the interface structure! */ /* Changes * v2-v9 rserved for future use in "older" version branches * v10, 2012-04-01: GetAllStatsLines got fmt parameter + * v11, 2013-09-07: - add "flags" to AddCounter API + * - GetAllStatsLines got parameter telling if ctrs shall be reset */ @@ -133,7 +139,7 @@ PROTOTYPEObj(statsobj); */ #define STATSCOUNTER_DEF(ctr, mut) \ intctr_t ctr; \ - DEF_ATOMIC_HELPER_MUT64(mut); + DEF_ATOMIC_HELPER_MUT64(mut) #define STATSCOUNTER_INIT(ctr, mut) \ INIT_ATOMIC_HELPER_MUT64(mut); \ diff --git a/runtime/stream.c b/runtime/stream.c index b781324..1bf1c47 100644 --- a/runtime/stream.c +++ b/runtime/stream.c @@ -67,7 +67,6 @@ # define O_LARGEFILE 0 #endif #ifndef HAVE_LSEEK64 - typedef off_t off64_t; # define lseek64(fd, offset, whence) lseek(fd, offset, whence) #endif @@ -77,7 +76,7 @@ DEFobjCurrIf(zlibw) /* forward definitions */ static rsRetVal strmFlushInternal(strm_t *pThis, int bFlushZip); -static rsRetVal strmWrite(strm_t *pThis, uchar *pBuf, size_t lenBuf); +static rsRetVal strmWrite(strm_t *__restrict__ const pThis, const uchar *__restrict__ const pBuf, const size_t lenBuf); static rsRetVal strmCloseFile(strm_t *pThis); static void *asyncWriterThread(void *pPtr); static rsRetVal doZipWrite(strm_t *pThis, uchar *pBuf, size_t lenBuf, int bFlush); @@ -257,7 +256,9 @@ doPhysOpen(strm_t *pThis) if(pThis->cryprov != NULL) { CHKiRet(pThis->cryprov->OnFileOpen(pThis->cryprovData, - pThis->pszCurrFName, &pThis->cryprovFileData)); + pThis->pszCurrFName, &pThis->cryprovFileData, + (pThis->tOperationsMode == STREAMMODE_READ) ? 'r' : 'w')); + pThis->cryprov->SetDeleteOnClose(pThis->cryprovFileData, pThis->bDeleteOnClose); } finalize_it: RETiRet; @@ -300,9 +301,11 @@ CheckFileChange(strm_t *pThis) if(stat((char*) pThis->pszCurrFName, &statName) == -1) ABORT_FINALIZE(RS_RET_IO_ERROR); DBGPRINTF("stream/after deserialize checking for file change on '%s', " - "inode %u/%u, size/currOffs %llu/%llu\n", - pThis->pszCurrFName, (unsigned) pThis->inode, - (unsigned) statName.st_ino, statName.st_size, pThis->iCurrOffs); + "inode %u/%u, size/currOffs %llu/%llu\n", + pThis->pszCurrFName, (unsigned) pThis->inode, + (unsigned) statName.st_ino, + (long long unsigned) statName.st_size, + (long long unsigned) pThis->iCurrOffs); if(pThis->inode != statName.st_ino || statName.st_size < pThis->iCurrOffs) { DBGPRINTF("stream: file %s has changed\n", pThis->pszCurrFName); pThis->iCurrOffs = 0; @@ -405,6 +408,12 @@ static rsRetVal strmCloseFile(strm_t *pThis) } } + /* if we have a signature provider, we must make sure that the crypto + * state files are opened and proper close processing happens. */ + if(pThis->cryprov != NULL && pThis->fd == -1) { + strmOpenFile(pThis); + } + /* the file may already be closed (or never have opened), so guard * against this. -- rgerhards, 2010-03-19 */ @@ -551,11 +560,14 @@ finalize_it: * rgerhards, 2008-02-13 */ static rsRetVal -strmReadBuf(strm_t *pThis) +strmReadBuf(strm_t *pThis, int *padBytes) { DEFiRet; int bRun; long iLenRead; + size_t actualDataLen; + size_t toRead; + ssize_t bytesLeft; ISOBJ_TYPE_assert(pThis, strm); /* We need to try read at least twice because we may run into EOF and need to switch files. */ @@ -566,13 +578,35 @@ strmReadBuf(strm_t *pThis) * rgerhards, 2008-02-13 */ CHKiRet(strmOpenFile(pThis)); - iLenRead = read(pThis->fd, pThis->pIOBuf, pThis->sIOBufSize); + if(pThis->cryprov == NULL) { + toRead = pThis->sIOBufSize; + } else { + CHKiRet(pThis->cryprov->GetBytesLeftInBlock(pThis->cryprovFileData, &bytesLeft)); + if(bytesLeft == -1 || bytesLeft > (ssize_t) pThis->sIOBufSize) { + toRead = pThis->sIOBufSize; + } else { + toRead = (size_t) bytesLeft; + } + } + iLenRead = read(pThis->fd, pThis->pIOBuf, toRead); DBGOPRINT((obj_t*) pThis, "file %d read %ld bytes\n", pThis->fd, iLenRead); + /* end crypto */ if(iLenRead == 0) { CHKiRet(strmHandleEOF(pThis)); } else if(iLenRead < 0) ABORT_FINALIZE(RS_RET_IO_ERROR); else { /* good read */ + /* here we place our crypto interface */ + if(pThis->cryprov != NULL) { + actualDataLen = iLenRead; + pThis->cryprov->Decrypt(pThis->cryprovFileData, pThis->pIOBuf, &actualDataLen); + *padBytes = iLenRead - actualDataLen; + iLenRead = actualDataLen; + DBGOPRINT((obj_t*) pThis, "encrypted file %d pad bytes %d, actual " + "data %ld\n", pThis->fd, *padBytes, iLenRead); + } else { + *padBytes = 0; + } pThis->iBufPtrMax = iLenRead; bRun = 0; /* exit loop */ } @@ -594,6 +628,7 @@ finalize_it: */ static rsRetVal strmReadChar(strm_t *pThis, uchar *pC) { + int padBytes = 0; /* in crypto mode, we may have some padding (non-data) bytes */ DEFiRet; ASSERT(pThis != NULL); @@ -609,8 +644,9 @@ static rsRetVal strmReadChar(strm_t *pThis, uchar *pC) /* do we need to obtain a new buffer? */ if(pThis->iBufPtr >= pThis->iBufPtrMax) { - CHKiRet(strmReadBuf(pThis)); + CHKiRet(strmReadBuf(pThis, &padBytes)); } + pThis->iCurrOffs += padBytes; /* if we reach this point, we have data available in the buffer */ @@ -646,7 +682,7 @@ static rsRetVal strmUnreadChar(strm_t *pThis, uchar c) * destruction of the returned CStr object! -- dlang 2010-12-13 */ static rsRetVal -strmReadLine(strm_t *pThis, cstr_t **ppCStr, int mode) +strmReadLine(strm_t *pThis, cstr_t **ppCStr, uint8_t mode, sbool bEscapeLF) { /* mode = 0 single line mode (equivalent to ReadLine) * mode = 1 LFLF mode (paragraph, blank line between entries) @@ -656,6 +692,7 @@ strmReadLine(strm_t *pThis, cstr_t **ppCStr, int mode) uchar c; uchar finished; rsRetVal readCharRet; + sbool bPrevWasNL; DEFiRet; ASSERT(pThis != NULL); @@ -681,18 +718,25 @@ strmReadLine(strm_t *pThis, cstr_t **ppCStr, int mode) CHKiRet(cstrFinalize(*ppCStr)); } else if(mode == 1) { finished=0; + bPrevWasNL = 0; while(finished == 0){ if(c != '\n') { CHKiRet(cstrAppendChar(*ppCStr, c)); CHKiRet(strmReadChar(pThis, &c)); + bPrevWasNL = 0; } else { if ((((*ppCStr)->iStrLen) > 0) ){ - if ((*ppCStr)->pBuf[(*ppCStr)->iStrLen -1 ] == '\n'){ - rsCStrTruncate(*ppCStr,1); /* remove the prior newline */ + if(bPrevWasNL) { + rsCStrTruncate(*ppCStr, (bEscapeLF) ? 4 : 1); /* remove the prior newline */ finished=1; } else { - CHKiRet(cstrAppendChar(*ppCStr, c)); + if(bEscapeLF) { + CHKiRet(rsCStrAppendStrWithLen(*ppCStr, (uchar*)"#012", sizeof("#012")-1)); + } else { + CHKiRet(cstrAppendChar(*ppCStr, c)); + } CHKiRet(strmReadChar(pThis, &c)); + bPrevWasNL = 1; } } else { finished=1; /* this is a blank line, a \n with nothing since the last complete record */ @@ -703,6 +747,7 @@ strmReadLine(strm_t *pThis, cstr_t **ppCStr, int mode) } else if(mode == 2) { /* indented follow-up lines */ finished=0; + bPrevWasNL = 0; while(finished == 0){ if ((*ppCStr)->iStrLen == 0){ if(c != '\n') { @@ -713,22 +758,31 @@ strmReadLine(strm_t *pThis, cstr_t **ppCStr, int mode) finished=1; /* this is a blank line, a \n with nothing since the last complete record */ } } else { - if ((*ppCStr)->pBuf[(*ppCStr)->iStrLen -1 ] != '\n'){ - /* not the first character after a newline, add it to the buffer */ - CHKiRet(cstrAppendChar(*ppCStr, c)); - CHKiRet(strmReadChar(pThis, &c)); - } else { + if(bPrevWasNL) { if ((c == ' ') || (c == '\t')){ CHKiRet(cstrAppendChar(*ppCStr, c)); CHKiRet(strmReadChar(pThis, &c)); + bPrevWasNL = 0; } else { /* clean things up by putting the character we just read back into * the input buffer and removing the LF character that is currently at the * end of the output string */ CHKiRet(strmUnreadChar(pThis, c)); - rsCStrTruncate(*ppCStr,1); + rsCStrTruncate(*ppCStr, (bEscapeLF) ? 4 : 1); finished=1; } + } else { /* not the first character after a newline, add it to the buffer */ + if(c == '\n') { + bPrevWasNL = 1; + if(bEscapeLF) { + CHKiRet(rsCStrAppendStrWithLen(*ppCStr, (uchar*)"#012", sizeof("#012")-1)); + } else { + CHKiRet(cstrAppendChar(*ppCStr, c)); + } + } else { + CHKiRet(cstrAppendChar(*ppCStr, c)); + } + CHKiRet(strmReadChar(pThis, &c)); } } } @@ -1408,7 +1462,7 @@ static rsRetVal strmSeek(strm_t *pThis, off64_t offs) DBGOPRINT((obj_t*) pThis, "file %d seek, pos %llu\n", pThis->fd, (long long unsigned) offs); i = lseek64(pThis->fd, offs, SEEK_SET); if(i != offs) { - DBGPRINTF("strmSeek: error %lld seeking to offset %lld\n", i, offs); + DBGPRINTF("strmSeek: error %lld seeking to offset %lld\n", i, (long long) offs); ABORT_FINALIZE(RS_RET_IO_ERROR); } pThis->iCurrOffs = offs; /* we are now at *this* offset */ @@ -1455,6 +1509,8 @@ strmMultiFileSeek(strm_t *pThis, int FNum, off64_t offs, off64_t *bytesDel) "deleting '%s' (%lld bytes)\n", pThis->iCurrFNum, FNum, pThis->pszCurrFName, (long long) *bytesDel); unlink((char*)pThis->pszCurrFName); + if(pThis->cryprov != NULL) + pThis->cryprov->DeleteStateFiles(pThis->pszCurrFName); free(pThis->pszCurrFName); pThis->pszCurrFName = NULL; pThis->iCurrFNum = FNum; @@ -1468,24 +1524,38 @@ finalize_it: } - /* seek to current offset. This is primarily a helper to readjust the OS file * pointer after a strm object has been deserialized. */ static rsRetVal strmSeekCurrOffs(strm_t *pThis) { + off64_t targetOffs; + uchar c; DEFiRet; ISOBJ_TYPE_assert(pThis, strm); - iRet = strmSeek(pThis, pThis->iCurrOffs); + if(pThis->cryprov == NULL || pThis->tOperationsMode != STREAMMODE_READ) { + iRet = strmSeek(pThis, pThis->iCurrOffs); + FINALIZE; + } + + /* As the cryprov may use CBC or similiar things, we need to read skip data */ + targetOffs = pThis->iCurrOffs; + pThis->iCurrOffs = 0; + DBGOPRINT((obj_t*) pThis, "encrypted, doing skip read of %lld bytes\n", + (long long) targetOffs); + while(targetOffs != pThis->iCurrOffs) { + CHKiRet(strmReadChar(pThis, &c)); + } +finalize_it: RETiRet; } /* write a *single* character to a stream object -- rgerhards, 2008-01-10 */ -static rsRetVal strmWriteChar(strm_t *pThis, uchar c) +static rsRetVal strmWriteChar(strm_t *__restrict__ const pThis, const uchar c) { DEFiRet; @@ -1518,7 +1588,7 @@ finalize_it: * strmWrite(), which does the lock (aka: we must not lock it, else we * would run into a recursive lock, resulting in a deadlock!) */ -static rsRetVal strmWriteLong(strm_t *pThis, long i) +static rsRetVal strmWriteLong(strm_t *__restrict__ const pThis, const long i) { DEFiRet; uchar szBuf[32]; @@ -1550,7 +1620,7 @@ finalize_it: * worth nothing. -- rgerhards, 2010-03-10 */ static rsRetVal -strmWrite(strm_t *pThis, uchar *pBuf, size_t lenBuf) +strmWrite(strm_t *__restrict__ const pThis, const uchar *__restrict__ const pBuf, size_t lenBuf) { DEFiRet; size_t iWrite; @@ -1605,8 +1675,7 @@ finalize_it: /* property set methods */ /* simple ones first */ -DEFpropSetMeth(strm, bDeleteOnClose, int) -DEFpropSetMeth(strm, iMaxFileSize, int) +DEFpropSetMeth(strm, iMaxFileSize, int64) DEFpropSetMeth(strm, iFileNumDigits, int) DEFpropSetMeth(strm, tOperationsMode, int) DEFpropSetMeth(strm, tOpenMode, mode_t) @@ -1621,6 +1690,15 @@ DEFpropSetMeth(strm, pszSizeLimitCmd, uchar*) DEFpropSetMeth(strm, cryprov, cryprov_if_t*) DEFpropSetMeth(strm, cryprovData, void*) +static rsRetVal strmSetbDeleteOnClose(strm_t *pThis, int val) +{ + pThis->bDeleteOnClose = val; + if(pThis->cryprov != NULL) { + pThis->cryprov->SetDeleteOnClose(pThis->cryprovFileData, pThis->bDeleteOnClose); + } + return RS_RET_OK; +} + static rsRetVal strmSetiMaxFiles(strm_t *pThis, int iNewVal) { pThis->iMaxFiles = iNewVal; diff --git a/runtime/stream.h b/runtime/stream.h index 61d5ede..8f26bdc 100644 --- a/runtime/stream.h +++ b/runtime/stream.h @@ -66,6 +66,7 @@ #define STREAM_H_INCLUDED #include <pthread.h> +#include <stdint.h> #include "obj-types.h" #include "glbl.h" #include "stream.h" @@ -158,12 +159,11 @@ BEGINinterface(strm) /* name must also be changed in ENDinterface macro! */ rsRetVal (*Construct)(strm_t **ppThis); rsRetVal (*ConstructFinalize)(strm_t *pThis); rsRetVal (*Destruct)(strm_t **ppThis); - rsRetVal (*SetMaxFileSize)(strm_t *pThis, int64 iMaxFileSize); rsRetVal (*SetFileName)(strm_t *pThis, uchar *pszName, size_t iLenName); rsRetVal (*ReadChar)(strm_t *pThis, uchar *pC); rsRetVal (*UnreadChar)(strm_t *pThis, uchar c); rsRetVal (*SeekCurrOffs)(strm_t *pThis); - rsRetVal (*Write)(strm_t *pThis, uchar *pBuf, size_t lenBuf); + rsRetVal (*Write)(strm_t *const pThis, const uchar *const pBuf, size_t lenBuf); rsRetVal (*WriteChar)(strm_t *pThis, uchar c); rsRetVal (*WriteLong)(strm_t *pThis, long i); rsRetVal (*SetFName)(strm_t *pThis, uchar *pszPrefix, size_t iLenPrefix); @@ -176,7 +176,7 @@ BEGINinterface(strm) /* name must also be changed in ENDinterface macro! */ rsRetVal (*SetWCntr)(strm_t *pThis, number_t *pWCnt); rsRetVal (*Dup)(strm_t *pThis, strm_t **ppNew); INTERFACEpropSetMeth(strm, bDeleteOnClose, int); - INTERFACEpropSetMeth(strm, iMaxFileSize, int); + INTERFACEpropSetMeth(strm, iMaxFileSize, int64); INTERFACEpropSetMeth(strm, iMaxFiles, int); INTERFACEpropSetMeth(strm, iFileNumDigits, int); INTERFACEpropSetMeth(strm, tOperationsMode, int); @@ -189,7 +189,7 @@ BEGINinterface(strm) /* name must also be changed in ENDinterface macro! */ INTERFACEpropSetMeth(strm, iFlushInterval, int); INTERFACEpropSetMeth(strm, pszSizeLimitCmd, uchar*); /* v6 added */ - rsRetVal (*ReadLine)(strm_t *pThis, cstr_t **ppCStr, int mode); + rsRetVal (*ReadLine)(strm_t *pThis, cstr_t **ppCStr, uint8_t mode, sbool bEscapeLF); /* v7 added 2012-09-14 */ INTERFACEpropSetMeth(strm, bVeryReliableZip, int); /* v8 added 2013-03-21 */ @@ -198,7 +198,8 @@ BEGINinterface(strm) /* name must also be changed in ENDinterface macro! */ INTERFACEpropSetMeth(strm, cryprov, cryprov_if_t*); INTERFACEpropSetMeth(strm, cryprovData, void*); ENDinterface(strm) -#define strmCURR_IF_VERSION 9 /* increment whenever you change the interface structure! */ +#define strmCURR_IF_VERSION 10 /* increment whenever you change the interface structure! */ +/* V10, 2013-09-10: added new parameter bEscapeLF, changed mode to uint8_t (rgerhards) */ static inline int strmGetCurrFileNum(strm_t *pStrm) { diff --git a/runtime/stringbuf.c b/runtime/stringbuf.c index 75d2eac..430e3b8 100644 --- a/runtime/stringbuf.c +++ b/runtime/stringbuf.c @@ -107,18 +107,18 @@ finalize_it: /* a helper function for rsCStr*Strf() */ -static rsRetVal rsCStrConstructFromszStrv(cstr_t **ppThis, uchar *fmt, va_list ap) +static rsRetVal rsCStrConstructFromszStrv(cstr_t **ppThis, char *fmt, va_list ap) __attribute__((format(gnu_printf,2, 0))); +static rsRetVal rsCStrConstructFromszStrv(cstr_t **ppThis, char *fmt, va_list ap) { DEFiRet; cstr_t *pThis; va_list ap2; - uchar *sz; int len; assert(ppThis != NULL); va_copy(ap2, ap); - len = vsnprintf(NULL, 0, fmt, ap2); + len = vsnprintf(NULL, 0, (char*)fmt, ap2); va_end(ap2); if(len < 0) @@ -133,7 +133,7 @@ static rsRetVal rsCStrConstructFromszStrv(cstr_t **ppThis, uchar *fmt, va_list a ABORT_FINALIZE(RS_RET_OUT_OF_MEMORY); } - vsnprintf(pThis->pBuf, len, fmt, ap); + vsnprintf((char*)pThis->pBuf, len, (char*)fmt, ap); *ppThis = pThis; finalize_it: RETiRet; @@ -142,7 +142,7 @@ finalize_it: /* construct from a printf-style formated string */ -rsRetVal rsCStrConstructFromszStrf(cstr_t **ppThis, uchar *fmt, ...) +rsRetVal rsCStrConstructFromszStrf(cstr_t **ppThis, char *fmt, ...) { DEFiRet; va_list ap; @@ -309,20 +309,20 @@ rsRetVal cstrAppendCStr(cstr_t *pThis, cstr_t *pstrAppend) /* append a printf-style formated string */ -rsRetVal rsCStrAppendStrf(cstr_t *pThis, uchar *fmt, ...) +rsRetVal rsCStrAppendStrf(cstr_t *pThis, char *fmt, ...) { DEFiRet; va_list ap; - cstr_t *pStr; + cstr_t *pStr = NULL; va_start(ap, fmt); - iRet = rsCStrConstructFromszStrv(&pStr, fmt, ap); + iRet = rsCStrConstructFromszStrv(&pStr, (char*)fmt, ap); va_end(ap); CHKiRet(iRet); iRet = cstrAppendCStr(pThis, pStr); - rsCStrDestruct(pStr); + rsCStrDestruct(&pStr); finalize_it: RETiRet; } @@ -457,11 +457,14 @@ uchar* rsCStrGetSzStr(cstr_t *pThis) * PLEASE NOTE: the caller must free the memory returned in ppSz in any case * (except, of course, if it is NULL). */ -rsRetVal cstrConvSzStrAndDestruct(cstr_t *pThis, uchar **ppSz, int bRetNULL) +rsRetVal cstrConvSzStrAndDestruct(cstr_t **ppThis, uchar **ppSz, int bRetNULL) { DEFiRet; uchar* pRetBuf; + cstr_t *pThis; + assert(ppThis != NULL); + pThis = *ppThis; rsCHECKVALIDOBJECT(pThis, OIDrsCStr); assert(ppSz != NULL); assert(bRetNULL == 0 || bRetNULL == 1); @@ -475,7 +478,7 @@ rsRetVal cstrConvSzStrAndDestruct(cstr_t *pThis, uchar **ppSz, int bRetNULL) } } else pRetBuf = pThis->pBuf; - + *ppSz = pRetBuf; finalize_it: @@ -484,6 +487,8 @@ finalize_it: * also free the sz String buffer, which we pass on to the user. */ RSFREEOBJ(pThis); + *ppThis = NULL; + RETiRet; } @@ -528,26 +533,6 @@ rsRetVal rsCStrTruncate(cstr_t *pThis, size_t nTrunc) /* Trim trailing whitespace from a given string */ -rsRetVal rsCStrTrimTrailingWhiteSpace(cstr_t *pThis) -{ - register int i; - register uchar *pC; - rsCHECKVALIDOBJECT(pThis, OIDrsCStr); - - i = pThis->iStrLen; - pC = pThis->pBuf + i - 1; - while(i > 0 && isspace((int)*pC)) { - --pC; - --i; - } - /* i now is the new string length! */ - pThis->iStrLen = i; - - return RS_RET_OK; -} - -/* Trim trailing whitespace from a given string - */ rsRetVal cstrTrimTrailingWhiteSpace(cstr_t *pThis) { register int i; @@ -563,8 +548,10 @@ rsRetVal cstrTrimTrailingWhiteSpace(cstr_t *pThis) --i; } /* i now is the new string length! */ - pThis->iStrLen = i; - pThis->pBuf[pThis->iStrLen] = '0'; /* we always have this space */ + if(i != (int) pThis->iStrLen) { + pThis->iStrLen = i; + pThis->pBuf[pThis->iStrLen] = '\0'; /* we always have this space */ + } done: return RS_RET_OK; } diff --git a/runtime/stringbuf.h b/runtime/stringbuf.h index b301f4b..51a1c3a 100644 --- a/runtime/stringbuf.h +++ b/runtime/stringbuf.h @@ -58,7 +58,7 @@ rsRetVal cstrConstruct(cstr_t **ppThis); rsRetVal cstrConstructFromESStr(cstr_t **ppThis, es_str_t *str); rsRetVal rsCStrConstructFromszStr(cstr_t **ppThis, uchar *sz); rsRetVal rsCStrConstructFromCStr(cstr_t **ppThis, cstr_t *pFrom); -rsRetVal rsCStrConstructFromszStrf(cstr_t **ppThis, uchar *fmt, ...); +rsRetVal rsCStrConstructFromszStrf(cstr_t **ppThis, char *fmt, ...) __attribute__((format(printf,2, 3))); /** * Destruct the string buffer object. @@ -155,7 +155,6 @@ static inline uchar* cstrGetSzStrNoNULL(cstr_t *pThis) */ rsRetVal rsCStrTruncate(cstr_t *pThis, size_t nTrunc); -rsRetVal rsCStrTrimTrailingWhiteSpace(cstr_t *pThis); rsRetVal cstrTrimTrailingWhiteSpace(cstr_t *pThis); /** @@ -179,7 +178,7 @@ rsRetVal rsCStrAppendStrWithLen(cstr_t *pThis, uchar* psz, size_t iStrLen); * * \param fmt pointer to the format string (see man 3 printf for details). Must not be NULL. */ -rsRetVal rsCStrAppendStrf(cstr_t *pThis, uchar *fmt, ...); +rsRetVal rsCStrAppendStrf(cstr_t *pThis, char *fmt, ...) __attribute__((format(printf,2, 3))); /** * Append an integer to the string. No special formatting is @@ -211,7 +210,7 @@ rsRetVal rsCStrConvertToBool(cstr_t *pStr, number_t *pBool); /* new calling interface */ rsRetVal cstrFinalize(cstr_t *pThis); -rsRetVal cstrConvSzStrAndDestruct(cstr_t *pThis, uchar **ppSz, int bRetNULL); +rsRetVal cstrConvSzStrAndDestruct(cstr_t **pThis, uchar **ppSz, int bRetNULL); rsRetVal cstrAppendCStr(cstr_t *pThis, cstr_t *pstrAppend); /* now come inline-like functions */ diff --git a/runtime/strmsrv.h b/runtime/strmsrv.h index 9ef28e4..f3d56d1 100644 --- a/runtime/strmsrv.h +++ b/runtime/strmsrv.h @@ -43,7 +43,7 @@ struct strmsrv_s { uchar *pszInputName; /**< value to be used as input name */ permittedPeers_t *pPermPeers;/**< driver's permitted peers */ int iLstnMax; /**< max nbr of listeners currently supported */ - netstrm_t **ppLstn; /**< our netstream listners */ + netstrm_t **ppLstn; /**< our netstream listeners */ strmLstnPortList_t **ppLstnPort; /**< pointer to relevant listen port description */ int iSessMax; /**< max number of sessions supported */ strmLstnPortList_t *pLstnPorts; /**< head pointer for listen ports */ diff --git a/runtime/syslogd-types.h b/runtime/syslogd-types.h index 6947a11..fe9dfa9 100644 --- a/runtime/syslogd-types.h +++ b/runtime/syslogd-types.h @@ -4,7 +4,7 @@ * * File begun on 2007-07-13 by RGerhards (extracted from syslogd.c) * - * Copyright 2007-2012 Adiscon GmbH. + * Copyright 2007-2014 Adiscon GmbH. * * This file is part of the rsyslog runtime library. * @@ -27,9 +27,6 @@ #include "stringbuf.h" #include <sys/param.h> -#if HAVE_SYSLOG_H -#include <syslog.h> -#endif /* we use RSTRUE/FALSE to prevent name claches with other packages */ #define RSFALSE 0 @@ -106,6 +103,17 @@ struct syslogTime { }; typedef struct syslogTime syslogTime_t; +struct tzinfo { + char *id; + char offsMode; + int8_t offsHour; + int8_t offsMin; +}; +typedef struct tzinfo tzinfo_t; + +typedef enum { ACT_STRING_PASSING = 0, ACT_ARRAY_PASSING = 1, ACT_MSG_PASSING = 2, + ACT_JSON_PASSING = 3} paramPassing_t; + #endif /* #ifndef SYSLOGD_TYPES_INCLUDED */ /* vi:set ai: */ diff --git a/runtime/typedefs.h b/runtime/typedefs.h index 5cc24e4..bf92021 100644 --- a/runtime/typedefs.h +++ b/runtime/typedefs.h @@ -3,7 +3,7 @@ * * Begun 2010-11-25 RGerhards * - * Copyright (C) 2005-2008 by Rainer Gerhards and Adiscon GmbH + * Copyright (C) 2005-2014 by Rainer Gerhards and Adiscon GmbH * * This file is part of the rsyslog runtime library. * @@ -25,6 +25,10 @@ */ #ifndef INCLUDED_TYPEDEFS_H #define INCLUDED_TYPEDEFS_H +#include <stdint.h> +#if defined(__FreeBSD__) || !defined(HAVE_LSEEK64) +#include <sys/types.h> +#endif /* some universal fixed size integer defines ... */ typedef long long int64; @@ -56,6 +60,7 @@ typedef struct nsdsel_ptcp_s nsdsel_ptcp_t; typedef struct nsdsel_gtls_s nsdsel_gtls_t; typedef struct nsdpoll_ptcp_s nsdpoll_ptcp_t; typedef struct wti_s wti_t; +typedef struct msgPropDescr_s msgPropDescr_t; typedef struct msg msg_t; typedef struct queue_s qqueue_t; typedef struct prop_s prop_t; @@ -93,6 +98,9 @@ typedef struct outchannels_s outchannels_t; typedef struct modConfData_s modConfData_t; typedef struct instanceConf_s instanceConf_t; typedef struct ratelimit_s ratelimit_t; +typedef struct lookup_string_tab_etry_s lookup_string_tab_etry_t; +typedef struct lookup_tables_s lookup_tables_t; +typedef struct lookup_s lookup_t; typedef struct action_s action_t; typedef int rs_size_t; /* we do never need more than 2Gig strings, signed permits to * use -1 as a special flag. */ @@ -101,6 +109,7 @@ typedef uint64 qDeqID; /* queue Dequeue order ID. 32 bits is considered dangerou typedef struct tcpLstnPortList_s tcpLstnPortList_t; // TODO: rename? typedef struct strmLstnPortList_s strmLstnPortList_t; // TODO: rename? +typedef struct actWrkrIParams actWrkrIParams_t; /* under Solaris (actually only SPARC), we need to redefine some types * to be void, so that we get void* pointers. Otherwise, we will see @@ -148,6 +157,60 @@ typedef enum { FIOP_ISEMPTY = 6 /* string empty <=> strlen(s) == 0 ?*/ } fiop_t; +#ifndef HAVE_LSEEK64 +# ifndef HAVE_OFF64_T + typedef off_t off64_t; +# endif +#endif + + +/* properties are now encoded as (tiny) integers. I do not use an enum as I would like + * to keep the memory footprint small (and thus cache hits high). + * rgerhards, 2009-06-26 + */ +typedef uintTiny propid_t; +#define PROP_INVALID 0 +#define PROP_MSG 1 +#define PROP_TIMESTAMP 2 +#define PROP_HOSTNAME 3 +#define PROP_SYSLOGTAG 4 +#define PROP_RAWMSG 5 +#define PROP_INPUTNAME 6 +#define PROP_FROMHOST 7 +#define PROP_FROMHOST_IP 8 +#define PROP_PRI 9 +#define PROP_PRI_TEXT 10 +#define PROP_IUT 11 +#define PROP_SYSLOGFACILITY 12 +#define PROP_SYSLOGFACILITY_TEXT 13 +#define PROP_SYSLOGSEVERITY 14 +#define PROP_SYSLOGSEVERITY_TEXT 15 +#define PROP_TIMEGENERATED 16 +#define PROP_PROGRAMNAME 17 +#define PROP_PROTOCOL_VERSION 18 +#define PROP_STRUCTURED_DATA 19 +#define PROP_APP_NAME 20 +#define PROP_PROCID 21 +#define PROP_MSGID 22 +#define PROP_PARSESUCCESS 23 +#define PROP_JSONMESG 24 +#define PROP_SYS_NOW 150 +#define PROP_SYS_YEAR 151 +#define PROP_SYS_MONTH 152 +#define PROP_SYS_DAY 153 +#define PROP_SYS_HOUR 154 +#define PROP_SYS_HHOUR 155 +#define PROP_SYS_QHOUR 156 +#define PROP_SYS_MINUTE 157 +#define PROP_SYS_MYHOSTNAME 158 +#define PROP_SYS_BOM 159 +#define PROP_SYS_UPTIME 160 +#define PROP_UUID 161 +#define PROP_CEE 200 +#define PROP_CEE_ALL_JSON 201 +#define PROP_LOCAL_VAR 202 +#define PROP_GLOBAL_VAR 203 + /* types of configuration handlers */ typedef enum cslCmdHdlrType { @@ -198,6 +261,15 @@ struct multi_submit_s { msg_t **ppMsgs; }; +/* the following structure is a helper to describe a message property */ +struct msgPropDescr_s { + propid_t id; + uchar *name; /* name and lenName are only set for dynamic */ + int nameLen; /* properties (JSON) */ +}; + +/* some forward-definitions from the grammar */ +struct nvlst; +struct cnfobj; + #endif /* multi-include protection */ -/* vim:set ai: - */ diff --git a/runtime/unicode-helper.h b/runtime/unicode-helper.h index b7db276..6337701 100644 --- a/runtime/unicode-helper.h +++ b/runtime/unicode-helper.h @@ -9,7 +9,7 @@ * * Begun 2009-05-21 RGerhards * - * Copyright (C) 2009-2012 by Rainer Gerhards and Adiscon GmbH + * Copyright (C) 2009-2014 by Rainer Gerhards and Adiscon GmbH * * This file is part of the rsyslog runtime library. * @@ -36,26 +36,26 @@ # define ustrncpy(psz1, psz2, len) strncpy((char*)(psz1), (char*)(psz2), (len)) # define ustrdup(psz) (uchar*)strdup((char*)(psz)) #else - static inline uchar* ustrncpy(uchar *psz1, uchar *psz2, size_t len) + static inline uchar* ustrncpy(uchar *psz1, const uchar *psz2, size_t len) { - return (uchar*) strncpy((char*) psz1, (char*) psz2, len); + return (uchar*) strncpy((char*) psz1, (const char*) psz2, len); } - static inline uchar* ustrdup(uchar *psz) + static inline uchar* ustrdup(const uchar *psz) { - return (uchar*) strdup((char*)psz); + return (uchar*) strdup((const char*)psz); } #endif /* #ifdef DEBUG */ -static inline int ustrcmp(uchar *psz1, uchar *psz2) +static inline int ustrcmp(const uchar *psz1, const uchar *psz2) { - return strcmp((char*) psz1, (char*) psz2); + return strcmp((const char*) psz1, (const char*) psz2); } -static inline int ustrlen(uchar *psz) +static inline int ustrlen(const uchar *psz) { - return strlen((char*) psz); + return strlen((const char*) psz); } diff --git a/runtime/wti.c b/runtime/wti.c index f91fb5a..3e0554a 100644 --- a/runtime/wti.c +++ b/runtime/wti.c @@ -44,12 +44,15 @@ #include "wti.h" #include "obj.h" #include "glbl.h" +#include "action.h" #include "atomic.h" /* static data */ DEFobjStaticHelpers DEFobjCurrIf(glbl) +pthread_key_t thrd_wti_key; + /* forward-definitions */ /* methods */ @@ -171,8 +174,9 @@ BEGINobjDestruct(wti) /* be sure to specify the object type also in END and CODE CODESTARTobjDestruct(wti) /* actual destruction */ batchFree(&pThis->batch); + free(pThis->actWrkrInfo); + pthread_cond_destroy(&pThis->pcondBusy); DESTROY_ATOMIC_HELPER_MUT(pThis->mutIsRunning); - free(pThis->pszDbgHdr); ENDobjDestruct(wti) @@ -181,6 +185,7 @@ ENDobjDestruct(wti) */ BEGINobjConstruct(wti) /* be sure to specify the object type also in END macro! */ INIT_ATOMIC_HELPER_MUT(pThis->mutIsRunning); + pthread_cond_init(&pThis->pcondBusy, NULL); ENDobjConstruct(wti) @@ -195,11 +200,20 @@ wtiConstructFinalize(wti_t *pThis) ISOBJ_TYPE_assert(pThis, wti); - DBGPRINTF("%s: finalizing construction of worker instance data\n", wtiGetDbgHdr(pThis)); + DBGPRINTF("%s: finalizing construction of worker instance data (for %d actions)\n", + wtiGetDbgHdr(pThis), iActionNbr); /* initialize our thread instance descriptor (no concurrency here) */ pThis->bIsRunning = RSFALSE; + /* must use calloc as we need zero-init */ + CHKmalloc(pThis->actWrkrInfo = calloc(iActionNbr, sizeof(actWrkrInfo_t))); + + if(pThis->pWtp == NULL) { + dbgprintf("wtiConstructFinalize: pWtp not set, this may be intentional\n"); + FINALIZE; + } + /* we now alloc the array for user pointers. We obtain the max from the queue itself. */ CHKiRet(pThis->pWtp->pfGetDeqBatchSize(pThis->pWtp->pUsr, &iDeqBatchSize)); CHKiRet(batchInit(&pThis->batch, iDeqBatchSize)); @@ -249,10 +263,10 @@ doIdleProcessing(wti_t *pThis, wtp_t *pWtp, int *pbInactivityTOOccured) if(pThis->bAlwaysRunning) { /* never shut down any started worker */ - d_pthread_cond_wait(pWtp->pcondBusy, pWtp->pmutUsr); + d_pthread_cond_wait(&pThis->pcondBusy, pWtp->pmutUsr); } else { timeoutComp(&t, pWtp->toWrkShutdown);/* get absolute timeout */ - if(d_pthread_cond_timedwait(pWtp->pcondBusy, pWtp->pmutUsr, &t) != 0) { + if(d_pthread_cond_timedwait(&pThis->pcondBusy, pWtp->pmutUsr, &t) != 0) { DBGPRINTF("%s: inactivity timeout, worker terminating...\n", wtiGetDbgHdr(pThis)); *pbInactivityTOOccured = 1; /* indicate we had a timeout */ } @@ -270,71 +284,100 @@ doIdleProcessing(wti_t *pThis, wtp_t *pWtp, int *pbInactivityTOOccured) */ #pragma GCC diagnostic ignored "-Wempty-body" rsRetVal -wtiWorker(wti_t *pThis) +wtiWorker(wti_t *__restrict__ const pThis) { - wtp_t *pWtp; /* our worker thread pool */ + wtp_t *__restrict__ const pWtp = pThis->pWtp; /* our worker thread pool -- shortcut */ + const action_t *__restrict__ pAction; int bInactivityTOOccured = 0; rsRetVal localRet; rsRetVal terminateRet; + actWrkrInfo_t *__restrict__ wrkrInfo; int iCancelStateSave; + int i, j, k; DEFiRet; - ISOBJ_TYPE_assert(pThis, wti); - pWtp = pThis->pWtp; /* shortcut */ - ISOBJ_TYPE_assert(pWtp, wtp); - dbgSetThrdName(pThis->pszDbgHdr); pthread_cleanup_push(wtiWorkerCancelCleanup, pThis); pthread_setcancelstate(PTHREAD_CANCEL_DISABLE, &iCancelStateSave); - + DBGPRINTF("wti %p: worker starting\n", pThis); /* now we have our identity, on to real processing */ - while(1) { /* loop will be broken below - need to do mutex locks */ + + /* note: in this loop, the mutex is "never" unlocked. Of course, + * this is not true: it actually is unlocked when the actual processing + * is done, as part of pWtp->pfDoWork() processing. Note that this + * function is required to re-lock it when done. We cannot do the + * lock/unlock here ourselfs, as pfDoWork() needs to access queue + * structures itself. + * The same goes for pfRateLimiter(). While we could unlock/lock when + * we call it, in practice the function is often called without any + * ratelimiting actually done. Only the rate limiter itself knows + * that. As such, it needs to bear the burden of doing the locking + * when required. -- rgerhards, 2013-11-20 + */ + d_pthread_mutex_lock(pWtp->pmutUsr); + while(1) { /* loop will be broken below */ if(pWtp->pfRateLimiter != NULL) { /* call rate-limiter, if defined */ pWtp->pfRateLimiter(pWtp->pUsr); } - d_pthread_mutex_lock(pWtp->pmutUsr); - /* first check if we are in shutdown process (but evaluate a bit later) */ terminateRet = wtpChkStopWrkr(pWtp, MUTEX_ALREADY_LOCKED); if(terminateRet == RS_RET_TERMINATE_NOW) { /* we now need to free the old batch */ localRet = pWtp->pfObjProcessed(pWtp->pUsr, pThis); - DBGOPRINT((obj_t*) pThis, "terminating worker because of TERMINATE_NOW mode, del iRet %d\n", - localRet); - d_pthread_mutex_unlock(pWtp->pmutUsr); + DBGOPRINT((obj_t*) pThis, "terminating worker because of " + "TERMINATE_NOW mode, del iRet %d\n", localRet); break; } /* try to execute and process whatever we have */ - /* Note that this function releases and re-aquires the mutex. The returned - * information on idle state must be processed before releasing the mutex again. - */ localRet = pWtp->pfDoWork(pWtp->pUsr, pThis); if(localRet == RS_RET_ERR_QUEUE_EMERGENCY) { - d_pthread_mutex_unlock(pWtp->pmutUsr); break; /* end of loop */ } else if(localRet == RS_RET_IDLE) { if(terminateRet == RS_RET_TERMINATE_WHEN_IDLE || bInactivityTOOccured) { - d_pthread_mutex_unlock(pWtp->pmutUsr); DBGOPRINT((obj_t*) pThis, "terminating worker terminateRet=%d, bInactivityTOOccured=%d\n", terminateRet, bInactivityTOOccured); break; /* end of loop */ } doIdleProcessing(pThis, pWtp, &bInactivityTOOccured); - d_pthread_mutex_unlock(pWtp->pmutUsr); continue; /* request next iteration */ } - d_pthread_mutex_unlock(pWtp->pmutUsr); - bInactivityTOOccured = 0; /* reset for next run */ } + d_pthread_mutex_unlock(pWtp->pmutUsr); + + DBGPRINTF("DDDD: wti %p: worker cleanup action instances\n", pThis); + for(i = 0 ; i < iActionNbr ; ++i) { + wrkrInfo = &(pThis->actWrkrInfo[i]); + dbgprintf("wti %p, action %d, ptr %p\n", pThis, i, wrkrInfo->actWrkrData); + if(wrkrInfo->actWrkrData != NULL) { + pAction = wrkrInfo->pAction; + pAction->pMod->mod.om.freeWrkrInstance(wrkrInfo->actWrkrData); + if(pAction->isTransactional) { + /* free iparam "cache" - we need to go through to max! */ + for(j = 0 ; j < wrkrInfo->p.tx.maxIParams ; ++j) { + for(k = 0 ; k < pAction->iNumTpls ; ++k) { + free(actParam(wrkrInfo->p.tx.iparams, + pAction->iNumTpls, j, k).param); + } + } + free(wrkrInfo->p.tx.iparams); + wrkrInfo->p.tx.iparams = NULL; + wrkrInfo->p.tx.currIParam = 0; + wrkrInfo->p.tx.maxIParams = 0; + } + wrkrInfo->actWrkrData = NULL; /* re-init for next activation */ + } + } + /* indicate termination */ pthread_cleanup_pop(0); /* remove cleanup handler */ pthread_setcancelstate(iCancelStateSave, NULL); + dbgprintf("wti %p: worker exiting\n", pThis); RETiRet; } @@ -374,6 +417,33 @@ finalize_it: } +/* This function returns (and creates if necessary) a dummy wti suitable + * for use by the rule engine. It is intended to be used for direct-mode + * main queues (folks, don't do that!). Once created, data is stored in + * thread-specific storage. + * Note: we do NOT do error checking -- if this functions fails, all the + * rest will fail as well... (also, it will only fail under OOM, so...). + * Memleak: we leak pWti's when run in direct mode. However, this is only + * a cosmetic leak, as we need them until all inputs are terminated, + * what means essentially until rsyslog itself is terminated. So we + * don't care -- it's just not nice in valgrind, but that's it. + */ +wti_t * +wtiGetDummy(void) +{ + wti_t *pWti; + + pWti = (wti_t*) pthread_getspecific(thrd_wti_key); + if(pWti == NULL) { + wtiConstruct(&pWti); + wtiConstructFinalize(pWti); + if(pthread_setspecific(thrd_wti_key, pWti) != 0) { + DBGPRINTF("wtiGetDummy: error setspecific thrd_wti_key\n"); + } + } + return pWti; +} + /* dummy */ rsRetVal wtiQueryInterface(void) { return RS_RET_NOT_IMPLEMENTED; } @@ -383,6 +453,7 @@ BEGINObjClassExit(wti, OBJ_IS_CORE_MODULE) /* CHANGE class also in END MACRO! */ CODESTARTObjClassExit(nsdsel_gtls) /* release objects we no longer need */ objRelease(glbl, CORE_COMPONENT); + pthread_key_delete(thrd_wti_key); ENDObjClassExit(wti) @@ -391,8 +462,14 @@ ENDObjClassExit(wti) * rgerhards, 2008-01-09 */ BEGINObjClassInit(wti, 1, OBJ_IS_CORE_MODULE) /* one is the object version (most important for persisting) */ + int r; /* request objects we use */ CHKiRet(objUse(glbl, CORE_COMPONENT)); + r = pthread_key_create(&thrd_wti_key, NULL); + if(r != 0) { + dbgprintf("wti.c: pthread_key_create failed\n"); + iRet = RS_RET_ERR; + } ENDObjClassInit(wti) /* vi:set ai: diff --git a/runtime/wti.h b/runtime/wti.h index 014251f..496ef23 100644 --- a/runtime/wti.h +++ b/runtime/wti.h @@ -1,6 +1,6 @@ /* Definition of the worker thread instance (wti) class. * - * Copyright 2008-2012 Adiscon GmbH. + * Copyright 2008-2013 Adiscon GmbH. * * This file is part of the rsyslog runtime library. * @@ -26,34 +26,183 @@ #include "wtp.h" #include "obj.h" #include "batch.h" +#include "action.h" +#define ACT_STATE_RDY 0 /* action ready, waiting for new transaction */ +#define ACT_STATE_ITX 1 /* transaction active, waiting for new data or commit */ +#define ACT_STATE_COMM 2 /* transaction finished (a transient state) */ +#define ACT_STATE_RTRY 3 /* failure occured, trying to restablish ready state */ +#define ACT_STATE_SUSP 4 /* suspended due to failure (return fail until timeout expired) */ +/* note: 3 bit bit field --> highest value is 7! */ + +typedef struct actWrkrInfo { + action_t *pAction; + void *actWrkrData; + uint16_t uResumeOKinRow;/* number of times in a row that resume said OK with an + immediate failure following */ + int iNbrResRtry; /* number of retries since last suspend */ + struct { + unsigned actState : 3; + unsigned bJustResumed : 1; + } flags; + union { + struct { + actWrkrIParams_t *iparams;/* dynamically sized array for transactional outputs */ + int currIParam; + int maxIParams; /* current max */ + } tx; + struct { + actWrkrIParams_t actParams[CONF_OMOD_NUMSTRINGS_MAXSIZE]; + } nontx; + } p; /* short name for "parameters" */ +} actWrkrInfo_t; + /* the worker thread instance class */ struct wti_s { BEGINobjInstance; pthread_t thrdID; /* thread ID */ int bIsRunning; /* is this thread currently running? (must be int for atomic op!) */ sbool bAlwaysRunning; /* should this thread always run? */ + int *pbShutdownImmediate;/* end processing of this batch immediately if set to 1 */ wtp_t *pWtp; /* my worker thread pool (important if only the work thread instance is passed! */ - batch_t batch; /* pointer to an object array meaningful for current user pointer (e.g. queue pUsr data elemt) */ + batch_t batch; /* pointer to an object array meaningful for current user + pointer (e.g. queue pUsr data elemt) */ uchar *pszDbgHdr; /* header string for debug messages */ - DEF_ATOMIC_HELPER_MUT(mutIsRunning); + actWrkrInfo_t *actWrkrInfo; /* *array* of action wrkr infos for all actions + (sized for max nbr of actions in config!) */ + pthread_cond_t pcondBusy; /* condition to wake up the worker, protected by pmutUsr in wtp */ + DEF_ATOMIC_HELPER_MUT(mutIsRunning) + struct { + uint8_t bPrevWasSuspended; + uint8_t bDoAutoCommit; /* do a commit after each message + * this is usually set for batches with 0 element, but may + * also be added as a user-selectable option (not implemented yet) + */ + } execState; /* state for the execution engine */ }; /* prototypes */ rsRetVal wtiConstruct(wti_t **ppThis); -rsRetVal wtiConstructFinalize(wti_t *pThis); +rsRetVal wtiConstructFinalize(wti_t * const pThis); rsRetVal wtiDestruct(wti_t **ppThis); -rsRetVal wtiWorker(wti_t *pThis); -rsRetVal wtiSetDbgHdr(wti_t *pThis, uchar *pszMsg, size_t lenMsg); -rsRetVal wtiCancelThrd(wti_t *pThis); -rsRetVal wtiSetAlwaysRunning(wti_t *pThis); -rsRetVal wtiSetState(wti_t *pThis, sbool bNew); -rsRetVal wtiWakeupThrd(wti_t *pThis); -sbool wtiGetState(wti_t *pThis); +rsRetVal wtiWorker(wti_t * const pThis); +rsRetVal wtiSetDbgHdr(wti_t * const pThis, uchar *pszMsg, size_t lenMsg); +rsRetVal wtiCancelThrd(wti_t * const pThis); +rsRetVal wtiSetAlwaysRunning(wti_t * const pThis); +rsRetVal wtiSetState(wti_t * const pThis, sbool bNew); +rsRetVal wtiWakeupThrd(wti_t * const pThis); +sbool wtiGetState(wti_t * const pThis); +wti_t *wtiGetDummy(void); PROTOTYPEObjClassInit(wti); PROTOTYPEpropSetMeth(wti, pszDbgHdr, uchar*); PROTOTYPEpropSetMeth(wti, pWtp, wtp_t*); +static inline uint8_t +getActionStateByNbr(wti_t * const pWti, const int iActNbr) +{ + return((uint8_t) pWti->actWrkrInfo[iActNbr].flags.actState); +} + +static inline uint8_t +getActionState(wti_t * const pWti, action_t * const pAction) +{ + return((uint8_t) pWti->actWrkrInfo[pAction->iActionNbr].flags.actState); +} + +static inline void +setActionState(wti_t * const pWti, action_t * const pAction, uint8_t newState) +{ + pWti->actWrkrInfo[pAction->iActionNbr].flags.actState = newState; +} + +static inline int +getActionJustResumed(wti_t * const pWti, action_t * const pAction) +{ + return(pWti->actWrkrInfo[pAction->iActionNbr].flags.bJustResumed); +} + +static inline void +setActionJustResumed(wti_t * const pWti, action_t * const pAction, int val) +{ + pWti->actWrkrInfo[pAction->iActionNbr].flags.bJustResumed = val; +} + + +static inline uint16_t +getActionResumeInRow(wti_t * const pWti, action_t * const pAction) +{ + return(pWti->actWrkrInfo[pAction->iActionNbr].uResumeOKinRow); +} + +static inline void +setActionResumeInRow(wti_t * const pWti, action_t * const pAction, uint16_t val) +{ + pWti->actWrkrInfo[pAction->iActionNbr].uResumeOKinRow = val; +} + +static inline void +incActionResumeInRow(wti_t * const pWti, action_t * const pAction) +{ + pWti->actWrkrInfo[pAction->iActionNbr].uResumeOKinRow++; +} + +static inline int +getActionNbrResRtry(wti_t * const pWti, action_t * const pAction) +{ + return(pWti->actWrkrInfo[pAction->iActionNbr].iNbrResRtry); +} + +static inline void +setActionNbrResRtry(wti_t * const pWti, action_t * const pAction, const uint16_t val) +{ + pWti->actWrkrInfo[pAction->iActionNbr].iNbrResRtry = val; +} + +static inline void +incActionNbrResRtry(wti_t * const pWti, action_t * const pAction) +{ + pWti->actWrkrInfo[pAction->iActionNbr].iNbrResRtry++; +} + +/* note: this function is only called once in action.c */ +static inline rsRetVal +wtiNewIParam(wti_t *const pWti, action_t *const pAction, actWrkrIParams_t **piparams) +{ + actWrkrInfo_t *const wrkrInfo = &(pWti->actWrkrInfo[pAction->iActionNbr]); + actWrkrIParams_t *iparams; + int newMax; + DEFiRet; + + if(wrkrInfo->p.tx.currIParam == wrkrInfo->p.tx.maxIParams) { + /* we need to extend */ + newMax = (wrkrInfo->p.tx.maxIParams == 0) ? CONF_IPARAMS_BUFSIZE + : 2 * wrkrInfo->p.tx.maxIParams; + CHKmalloc(iparams = realloc(wrkrInfo->p.tx.iparams, + sizeof(actWrkrIParams_t) * pAction->iNumTpls * newMax)); + memset(iparams + (wrkrInfo->p.tx.currIParam * pAction->iNumTpls), 0, + sizeof(actWrkrIParams_t) * pAction->iNumTpls * (newMax - wrkrInfo->p.tx.maxIParams)); + wrkrInfo->p.tx.iparams = iparams; + wrkrInfo->p.tx.maxIParams = newMax; + } + *piparams = wrkrInfo->p.tx.iparams + wrkrInfo->p.tx.currIParam * pAction->iNumTpls; + ++wrkrInfo->p.tx.currIParam; + +finalize_it: + RETiRet; +} + +static inline void +wtiInitIParam(actWrkrIParams_t *piparams) +{ + memset(piparams, 0, sizeof(actWrkrIParams_t)); +} + +static inline void +wtiResetExecState(wti_t * const pWti, batch_t * const pBatch) +{ + pWti->execState.bPrevWasSuspended = 0; + pWti->execState.bDoAutoCommit = (batchNumMsgs(pBatch) == 1); +} #endif /* #ifndef WTI_H_INCLUDED */ diff --git a/runtime/wtp.c b/runtime/wtp.c index 19151e7..66942e6 100644 --- a/runtime/wtp.c +++ b/runtime/wtp.c @@ -8,7 +8,7 @@ * (and in the web doc set on http://www.rsyslog.com/doc). Be sure to read it * if you are getting aquainted to the object. * - * Copyright 2008,2009 Rainer Gerhards and Adiscon GmbH. + * Copyright 2008-2013 Rainer Gerhards and Adiscon GmbH. * * This file is part of the rsyslog runtime library. * @@ -91,6 +91,7 @@ BEGINobjConstruct(wtp) /* be sure to specify the object type also in END macro! pthread_cond_init(&pThis->condThrdTrm, NULL); pthread_attr_init(&pThis->attrThrd); /* Set thread scheduling policy to default */ +#warning do we need this any longer? I think it was a cure for an already fixed bug.. #ifdef HAVE_PTHREAD_SETSCHEDPARAM pthread_attr_setschedpolicy(&pThis->attrThrd, default_thr_sched_policy); pthread_attr_setschedparam(&pThis->attrThrd, &default_sched_param); @@ -121,7 +122,8 @@ wtpConstructFinalize(wtp_t *pThis) ISOBJ_TYPE_assert(pThis, wtp); - DBGPRINTF("%s: finalizing construction of worker thread pool\n", wtpGetDbgHdr(pThis)); + DBGPRINTF("%s: finalizing construction of worker thread pool (numworkerThreads %d)\n", + wtpGetDbgHdr(pThis), pThis->iNumWorkerThreads); /* alloc and construct workers - this can only be done in finalizer as we previously do * not know the max number of workers */ @@ -233,9 +235,9 @@ wtpShutdownAll(wtp_t *pThis, wtpState_t tShutdownCmd, struct timespec *ptTimeout /* lock mutex to prevent races (may otherwise happen during idle processing and such...) */ d_pthread_mutex_lock(pThis->pmutUsr); wtpSetState(pThis, tShutdownCmd); - pthread_cond_broadcast(pThis->pcondBusy); /* wake up all workers */ /* awake workers in retry loop */ for(i = 0 ; i < pThis->iNumWorkerThreads ; ++i) { + pthread_cond_signal(&pThis->pWrkr[i]->pcondBusy); wtiWakeupThrd(pThis->pWrkr[i]); } d_pthread_mutex_unlock(pThis->pmutUsr); @@ -455,7 +457,7 @@ wtpAdviseMaxWorkers(wtp_t *pThis, int nMaxWrkr) { DEFiRet; int nMissing; /* number workers missing to run */ - int i; + int i, nRunning; ISOBJ_TYPE_assert(pThis, wtp); @@ -475,7 +477,13 @@ wtpAdviseMaxWorkers(wtp_t *pThis, int nMaxWrkr) CHKiRet(wtpStartWrkr(pThis)); } } else { - pthread_cond_signal(pThis->pcondBusy); + /* we have needed number of workers, but they may be sleeping */ + for(i = 0, nRunning = 0; i < pThis->iNumWorkerThreads && nRunning < nMaxWrkr; ++i) { + if (wtiGetState(pThis->pWrkr[i]) != WRKTHRD_STOPPED) { + pthread_cond_signal(&pThis->pWrkr[i]->pcondBusy); + nRunning++; + } + } } @@ -490,7 +498,6 @@ DEFpropSetMeth(wtp, wtpState, wtpState_t) DEFpropSetMeth(wtp, iNumWorkerThreads, int) DEFpropSetMeth(wtp, pUsr, void*) DEFpropSetMethPTR(wtp, pmutUsr, pthread_mutex_t) -DEFpropSetMethPTR(wtp, pcondBusy, pthread_cond_t) DEFpropSetMethFP(wtp, pfChkStopWrkr, rsRetVal(*pVal)(void*, int)) DEFpropSetMethFP(wtp, pfRateLimiter, rsRetVal(*pVal)(void*)) DEFpropSetMethFP(wtp, pfGetDeqBatchSize, rsRetVal(*pVal)(void*, int*)) diff --git a/runtime/wtp.h b/runtime/wtp.h index 25992f7..531355f 100644 --- a/runtime/wtp.h +++ b/runtime/wtp.h @@ -56,7 +56,6 @@ struct wtp_s { void *pUsr; /* pointer to user object (in this case, the queue the wtp belongs to) */ pthread_attr_t attrThrd;/* attribute for new threads (created just once and cached here) */ pthread_mutex_t *pmutUsr; - pthread_cond_t *pcondBusy; /* condition the user will signal "busy again, keep runing" on (awakes worker) */ rsRetVal (*pfChkStopWrkr)(void *pUsr, int); rsRetVal (*pfGetDeqBatchSize)(void *pUsr, int*); /* obtains max dequeue count from queue config */ rsRetVal (*pfObjProcessed)(void *pUsr, wti_t *pWti); /* indicate user object is processed */ @@ -64,8 +63,8 @@ struct wtp_s { rsRetVal (*pfDoWork)(void *pUsr, void *pWti); /* end user objects */ uchar *pszDbgHdr; /* header string for debug messages */ - DEF_ATOMIC_HELPER_MUT(mutCurNumWrkThrd); - DEF_ATOMIC_HELPER_MUT(mutWtpState); + DEF_ATOMIC_HELPER_MUT(mutCurNumWrkThrd) + DEF_ATOMIC_HELPER_MUT(mutWtpState) }; /* some symbolic constants for easier reference */ @@ -95,6 +94,5 @@ PROTOTYPEpropSetMeth(wtp, iMaxWorkerThreads, int); PROTOTYPEpropSetMeth(wtp, pUsr, void*); PROTOTYPEpropSetMeth(wtp, iNumWorkerThreads, int); PROTOTYPEpropSetMethPTR(wtp, pmutUsr, pthread_mutex_t); -PROTOTYPEpropSetMethPTR(wtp, pcondBusy, pthread_cond_t); #endif /* #ifndef WTP_H_INCLUDED */ diff --git a/tcps_sess.c b/tcps_sess.c index 5821e44..4d18b01 100644 --- a/tcps_sess.c +++ b/tcps_sess.c @@ -59,8 +59,6 @@ DEFobjCurrIf(netstrm) DEFobjCurrIf(prop) DEFobjCurrIf(datetime) -static int iMaxLine; /* maximum size of a single message */ - /* forward definitions */ static rsRetVal Close(tcps_sess_t *pThis); @@ -72,7 +70,7 @@ BEGINobjConstruct(tcps_sess) /* be sure to specify the object type also in END m pThis->bAtStrtOfFram = 1; /* indicate frame header expected */ pThis->eFraming = TCP_FRAMING_OCTET_STUFFING; /* just make sure... */ /* now allocate the message reception buffer */ - CHKmalloc(pThis->pMsg = (uchar*) MALLOC(sizeof(uchar) * iMaxLine + 1)); + CHKmalloc(pThis->pMsg = (uchar*) MALLOC(sizeof(uchar) * glbl.GetMaxLine() + 1)); finalize_it: ENDobjConstruct(tcps_sess) @@ -253,6 +251,8 @@ defaultDoSubmitMessage(tcps_sess_t *pThis, struct syslogTime *stTime, time_t ttG CHKiRet(msgConstructWithTime(&pMsg, stTime, ttGenTime)); MsgSetRawMsg(pMsg, (char*)pThis->pMsg, pThis->iMsg); MsgSetInputName(pMsg, pThis->pLstnInfo->pInputName); + if(pThis->pLstnInfo->dfltTZ != NULL) + MsgSetDfltTZ(pMsg, (char*) pThis->pLstnInfo->dfltTZ); MsgSetFlowControlType(pMsg, pThis->pSrv->bUseFlowControl ? eFLOWCTL_LIGHT_DELAY : eFLOWCTL_NO_DELAY); pMsg->msgFlags = NEEDS_PARSING | PARSE_HOSTNAME; @@ -307,7 +307,7 @@ PrepareClose(tcps_sess_t *pThis) * generate an error message and discard the frame. */ errmsg.LogError(0, NO_ERRCODE, "Incomplete frame at end of stream in session %p - " - "ignoring extra data (a message may be lost).\n", pThis->pStrm); + "ignoring extra data (a message may be lost).", pThis->pStrm); /* nothing more to do */ } else { /* here, we have traditional framing. Missing LF at the end * of message may occur. As such, we process the message in @@ -356,6 +356,7 @@ processDataRcvd(tcps_sess_t *pThis, char c, struct syslogTime *stTime, time_t tt { DEFiRet; ISOBJ_TYPE_assert(pThis, tcps_sess); + int iMaxLine = glbl.GetMaxLine(); if(pThis->inputState == eAtStrtFram) { if(pThis->bSuppOctetFram && c >= '0' && c <= '9') { @@ -375,13 +376,13 @@ processDataRcvd(tcps_sess_t *pThis, char c, struct syslogTime *stTime, time_t tt DBGPRINTF("TCP Message with octet-counter, size %d.\n", pThis->iOctetsRemain); if(c != ' ') { errmsg.LogError(0, NO_ERRCODE, "Framing Error in received TCP message: " - "delimiter is not SP but has ASCII value %d.\n", c); + "delimiter is not SP but has ASCII value %d.", c); } if(pThis->iOctetsRemain < 1) { /* TODO: handle the case where the octet count is 0! */ DBGPRINTF("Framing Error: invalid octet count\n"); errmsg.LogError(0, NO_ERRCODE, "Framing Error in received TCP message: " - "invalid octet count %d.\n", pThis->iOctetsRemain); + "invalid octet count %d.", pThis->iOctetsRemain); } else if(pThis->iOctetsRemain > iMaxLine) { /* while we can not do anything against it, we can at least log an indication * that something went wrong) -- rgerhards, 2008-03-14 @@ -389,7 +390,7 @@ processDataRcvd(tcps_sess_t *pThis, char c, struct syslogTime *stTime, time_t tt DBGPRINTF("truncating message with %d octets - max msg size is %d\n", pThis->iOctetsRemain, iMaxLine); errmsg.LogError(0, NO_ERRCODE, "received oversize message: size is %d bytes, " - "max msg size is %d, truncating...\n", pThis->iOctetsRemain, iMaxLine); + "max msg size is %d, truncating...", pThis->iOctetsRemain, iMaxLine); } pThis->inputState = eInMsg; } @@ -545,7 +546,6 @@ BEGINObjClassInit(tcps_sess, 1, OBJ_IS_CORE_MODULE) /* class, version - CHANGE c CHKiRet(objUse(prop, CORE_COMPONENT)); CHKiRet(objUse(glbl, CORE_COMPONENT)); - iMaxLine = glbl.GetMaxLine(); /* get maximum size we currently support */ objRelease(glbl, CORE_COMPONENT); /* set our own handlers */ @@ -134,6 +134,7 @@ addNewLstnPort(tcpsrv_t *pThis, uchar *pszPort, int bSuppOctetFram) /* create entry */ CHKmalloc(pEntry = MALLOC(sizeof(tcpLstnPortList_t))); CHKmalloc(pEntry->pszPort = ustrdup(pszPort)); + strcpy((char*)pEntry->dfltTZ, (char*)pThis->dfltTZ); pEntry->pSrv = pThis; pEntry->pRuleset = pThis->pRuleset; pEntry->bSuppOctetFram = bSuppOctetFram; @@ -157,7 +158,7 @@ addNewLstnPort(tcpsrv_t *pThis, uchar *pszPort, int bSuppOctetFram) ratelimitSetThreadSafe(pEntry->ratelimiter); STATSCOUNTER_INIT(pEntry->ctrSubmit, pEntry->mutCtrSubmit); CHKiRet(statsobj.AddCounter(pEntry->stats, UCHAR_CONSTANT("submitted"), - ctrType_IntCtr, &(pEntry->ctrSubmit))); + ctrType_IntCtr, CTR_FLAG_RESETTABLE, &(pEntry->ctrSubmit))); CHKiRet(statsobj.ConstructFinalize(pEntry->stats)); finalize_it: @@ -743,7 +744,8 @@ RunSelect(tcpsrv_t *pThis, nsd_epworkset_t workset[], size_t sizeWorkset) pthread_cleanup_push(RunCancelCleanup, (void*) &pSel); while(1) { CHKiRet(nssel.Construct(&pSel)); - // TODO: set driver + if(pThis->pszDrvrName != NULL) + CHKiRet(nssel.SetDrvrName(pSel, pThis->pszDrvrName)); CHKiRet(nssel.ConstructFinalize(pSel)); /* Add the TCP listen sockets to the list of read descriptors. */ @@ -859,7 +861,8 @@ Run(tcpsrv_t *pThis) * to prevent us from leaking anything. -- rgerhards, 20080-04-24 */ if((localRet = nspoll.Construct(&pPoll)) == RS_RET_OK) { - // TODO: set driver + if(pThis->pszDrvrName != NULL) + CHKiRet(nspoll.SetDrvrName(pPoll, pThis->pszDrvrName)); localRet = nspoll.ConstructFinalize(pPoll); } if(localRet != RS_RET_OK) { @@ -916,9 +919,11 @@ BEGINobjConstruct(tcpsrv) /* be sure to specify the object type also in END macr pThis->addtlFrameDelim = TCPSRV_NO_ADDTL_DELIMITER; pThis->bDisableLFDelim = 0; pThis->OnMsgReceive = NULL; + pThis->dfltTZ[0] = '\0'; pThis->ratelimitInterval = 0; pThis->ratelimitBurst = 10000; pThis->bUseFlowControl = 1; + pThis->pszDrvrName = NULL; ENDobjConstruct(tcpsrv) @@ -931,12 +936,13 @@ tcpsrvConstructFinalize(tcpsrv_t *pThis) /* prepare network stream subsystem */ CHKiRet(netstrms.Construct(&pThis->pNS)); + if(pThis->pszDrvrName != NULL) + CHKiRet(netstrms.SetDrvrName(pThis->pNS, pThis->pszDrvrName)); CHKiRet(netstrms.SetDrvrMode(pThis->pNS, pThis->iDrvrMode)); if(pThis->pszDrvrAuthMode != NULL) CHKiRet(netstrms.SetDrvrAuthMode(pThis->pNS, pThis->pszDrvrAuthMode)); if(pThis->pPermPeers != NULL) CHKiRet(netstrms.SetDrvrPermPeers(pThis->pNS, pThis->pPermPeers)); - // TODO: set driver! CHKiRet(netstrms.ConstructFinalize(pThis->pNS)); /* set up listeners */ @@ -948,6 +954,8 @@ finalize_it: if(iRet != RS_RET_OK) { if(pThis->pNS != NULL) netstrms.Destruct(&pThis->pNS); + errmsg.LogError(0, iRet, "tcpsrv could not create listener (inputname: '%s')", + (pThis->pszInputName == NULL) ? (uchar*)"*UNSET*" : pThis->pszInputName); } RETiRet; } @@ -963,6 +971,7 @@ CODESTARTobjDestruct(tcpsrv) if(pThis->pNS != NULL) netstrms.Destruct(&pThis->pNS); + free(pThis->pszDrvrName); free(pThis->pszDrvrAuthMode); free(pThis->ppLstn); free(pThis->ppLstnPort); @@ -1107,6 +1116,15 @@ SetAddtlFrameDelim(tcpsrv_t *pThis, int iDelim) } +static rsRetVal +SetDfltTZ(tcpsrv_t *pThis, uchar *tz) +{ + DEFiRet; + ISOBJ_TYPE_assert(pThis, tcpsrv); + strcpy((char*)pThis->dfltTZ, (char*)tz); + RETiRet; +} + /* Set the input name to use -- rgerhards, 2008-12-10 */ static rsRetVal SetInputName(tcpsrv_t *pThis, uchar *name) @@ -1171,6 +1189,16 @@ SetDrvrMode(tcpsrv_t *pThis, int iMode) RETiRet; } +static rsRetVal +SetDrvrName(tcpsrv_t *pThis, uchar *name) +{ + DEFiRet; + ISOBJ_TYPE_assert(pThis, tcpsrv); + free(pThis->pszDrvrName); + CHKmalloc(pThis->pszDrvrName = ustrdup(name)); +finalize_it: + RETiRet; +} /* set the driver authentication mode -- rgerhards, 2008-05-19 */ static rsRetVal @@ -1266,6 +1294,7 @@ CODESTARTobjQueryInterface(tcpsrv) pIf->SetKeepAlive = SetKeepAlive; pIf->SetUsrP = SetUsrP; pIf->SetInputName = SetInputName; + pIf->SetDfltTZ = SetDfltTZ; pIf->SetAddtlFrameDelim = SetAddtlFrameDelim; pIf->SetbDisableLFDelim = SetbDisableLFDelim; pIf->SetSessMax = SetSessMax; @@ -1273,6 +1302,7 @@ CODESTARTobjQueryInterface(tcpsrv) pIf->SetLstnMax = SetLstnMax; pIf->SetDrvrMode = SetDrvrMode; pIf->SetDrvrAuthMode = SetDrvrAuthMode; + pIf->SetDrvrName = SetDrvrName; pIf->SetDrvrPermPeers = SetDrvrPermPeers; pIf->SetCBIsPermittedHost = SetCBIsPermittedHost; pIf->SetCBOpenLstnSocks = SetCBOpenLstnSocks; @@ -1384,8 +1414,6 @@ stopWorkerPool(void) pthread_cond_destroy(&wrkrInfo[i].run); } pthread_cond_destroy(&wrkrIdle); - pthread_mutex_destroy(&wrkrMut); - } @@ -1393,10 +1421,14 @@ stopWorkerPool(void) BEGINmodExit CODESTARTmodExit - stopWorkerPool(); + if(bWrkrRunning) { + stopWorkerPool(); + bWrkrRunning = 0; + } /* de-init in reverse order! */ tcpsrvClassExit(); tcps_sessClassExit(); + pthread_mutex_destroy(&wrkrMut); ENDmodExit @@ -43,6 +43,7 @@ struct tcpLstnPortList_s { statsobj_t *stats; /**< associated stats object */ sbool bSuppOctetFram; /**< do we support octect-counted framing? (if no->legay only!)*/ ratelimit_t *ratelimiter; + uchar dfltTZ[8]; /**< default TZ if none in timestamp; '\0' =No Default */ STATSCOUNTER_DEF(ctrSubmit, mutCtrSubmit) tcpLstnPortList_t *pNext; /**< next port or NULL */ }; @@ -56,6 +57,7 @@ struct tcpsrv_s { netstrms_t *pNS; /**< pointer to network stream subsystem */ int iDrvrMode; /**< mode of the stream driver to use */ uchar *pszDrvrAuthMode; /**< auth mode of the stream driver to use */ + uchar *pszDrvrName; /**< name of stream driver to use */ uchar *pszInputName; /**< value to be used as input name */ ruleset_t *pRuleset; /**< ruleset to bind to */ permittedPeers_t *pPermPeers;/**< driver's permitted peers */ @@ -63,10 +65,11 @@ struct tcpsrv_s { sbool bUsingEPoll; /**< are we in epoll mode (means we do not need to keep track of sessions!) */ sbool bUseFlowControl; /**< use flow control (make light delayable) */ int iLstnCurr; /**< max nbr of listeners currently supported */ - netstrm_t **ppLstn; /**< our netstream listners */ + netstrm_t **ppLstn; /**< our netstream listeners */ tcpLstnPortList_t **ppLstnPort; /**< pointer to relevant listen port description */ - int iLstnMax; /**< max number of listners supported */ + int iLstnMax; /**< max number of listeners supported */ int iSessMax; /**< max number of sessions supported */ + uchar dfltTZ[8]; /**< default TZ if none in timestamp; '\0' =No Default */ tcpLstnPortList_t *pLstnPorts; /**< head pointer for listen ports */ int addtlFrameDelim; /**< additional frame delimiter for plain TCP syslog framing (e.g. to handle NetScreen) */ @@ -110,7 +113,6 @@ BEGINinterface(tcpsrv) /* name must also be changed in ENDinterface macro! */ rsRetVal (*ConstructFinalize)(tcpsrv_t __attribute__((unused)) *pThis); rsRetVal (*Destruct)(tcpsrv_t **ppThis); rsRetVal (*configureTCPListen)(tcpsrv_t*, uchar *pszPort, int bSuppOctetFram); - //rsRetVal (*SessAccept)(tcpsrv_t *pThis, tcpLstnPortList_t*, tcps_sess_t **ppSess, netstrm_t *pStrm); rsRetVal (*create_tcp_socket)(tcpsrv_t *pThis); rsRetVal (*Run)(tcpsrv_t *pThis); /* set methods */ @@ -147,8 +149,12 @@ BEGINinterface(tcpsrv) /* name must also be changed in ENDinterface macro! */ rsRetVal (*SetKeepAlive)(tcpsrv_t*, int); /* added v13 -- rgerhards, 2012-10-15 */ rsRetVal (*SetLinuxLikeRatelimiters)(tcpsrv_t *pThis, int interval, int burst); + /* added v14 -- rgerhards, 2013-07-28 */ + rsRetVal (*SetDfltTZ)(tcpsrv_t *pThis, uchar *dfltTZ); + /* added v15 -- rgerhards, 2013-09-17 */ + rsRetVal (*SetDrvrName)(tcpsrv_t *pThis, uchar *pszName); ENDinterface(tcpsrv) -#define tcpsrvCURR_IF_VERSION 13 /* increment whenever you change the interface structure! */ +#define tcpsrvCURR_IF_VERSION 15 /* increment whenever you change the interface structure! */ /* change for v4: * - SetAddtlFrameDelim() added -- rgerhards, 2008-12-10 * - SetInputName() added -- rgerhards, 2008-12-10 @@ -1,7 +1,7 @@ /* This is the template processing code of rsyslog. * begun 2004-11-17 rgerhards * - * Copyright 2004-2012 Rainer Gerhards and Adiscon + * Copyright 2004-2014 Rainer Gerhards and Adiscon * * This file is part of rsyslog. * @@ -34,7 +34,7 @@ #include <string.h> #include <ctype.h> #include <assert.h> -#include <json/json.h> +#include <json.h> #include "stringbuf.h" #include "syslogd-types.h" #include "template.h" @@ -45,6 +45,7 @@ #include "strgen.h" #include "rsconf.h" #include "msg.h" +#include "parserif.h" #include "unicode-helper.h" /* static data */ @@ -116,16 +117,16 @@ static int bFirstRegexpErrmsg = 1; /**< did we already do a "can't load regexp" /* helper to tplToString and strgen's, extends buffer */ #define ALLOC_INC 128 rsRetVal -ExtendBuf(uchar **pBuf, size_t *pLenBuf, size_t iMinSize) +ExtendBuf(actWrkrIParams_t *__restrict__ const iparam, const size_t iMinSize) { uchar *pNewBuf; size_t iNewSize; DEFiRet; iNewSize = (iMinSize / ALLOC_INC + 1) * ALLOC_INC; - CHKmalloc(pNewBuf = (uchar*) realloc(*pBuf, iNewSize)); - *pBuf = pNewBuf; - *pLenBuf = iNewSize; + CHKmalloc(pNewBuf = (uchar*) realloc(iparam->param, iNewSize)); + iparam->param = pNewBuf; + iparam->lenBuf = iNewSize; finalize_it: RETiRet; @@ -137,42 +138,39 @@ finalize_it: * The function takes a pointer to a template and a pointer to a msg object * as well as a pointer to an output buffer and its size. Note that the output * buffer pointer may be NULL, size 0, in which case a new one is allocated. - * The outpub buffer is grown as required. It is the caller's duty to free the + * The output buffer is grown as required. It is the caller's duty to free the * buffer when it is done. Note that it is advisable to reuse memory, as this * offers big performance improvements. * rewritten 2009-06-19 rgerhards */ rsRetVal -tplToString(struct template *pTpl, msg_t *pMsg, uchar **ppBuf, size_t *pLenBuf, - struct syslogTime *ttNow) +tplToString(struct template *__restrict__ const pTpl, + msg_t *__restrict__ const pMsg, + actWrkrIParams_t *__restrict const iparam, + struct syslogTime *const ttNow) { DEFiRet; - struct templateEntry *pTpe; + struct templateEntry *__restrict__ pTpe; size_t iBuf; unsigned short bMustBeFreed = 0; uchar *pVal; rs_size_t iLenVal = 0; - assert(pTpl != NULL); - assert(pMsg != NULL); - assert(ppBuf != NULL); - assert(pLenBuf != NULL); - if(pTpl->pStrgen != NULL) { - CHKiRet(pTpl->pStrgen(pMsg, ppBuf, pLenBuf)); + CHKiRet(pTpl->pStrgen(pMsg, iparam)); FINALIZE; } - if(pTpl->subtree != NULL) { + if(pTpl->bHaveSubtree) { /* only a single CEE subtree must be provided */ /* note: we could optimize the code below, however, this is * not worth the effort, as this passing mode is not expected * in subtree mode and so most probably only used for debug & test. */ - getCEEPropVal(pMsg, pTpl->subtree, &pVal, &iLenVal, &bMustBeFreed); - if(iLenVal >= (rs_size_t)*pLenBuf) /* we reserve one char for the final \0! */ - CHKiRet(ExtendBuf(ppBuf, pLenBuf, iLenVal + 1)); - memcpy(*ppBuf, pVal, iLenVal+1); + getJSONPropVal(pMsg, &pTpl->subtree, &pVal, &iLenVal, &bMustBeFreed); + if(iLenVal >= (rs_size_t)iparam->lenBuf) /* we reserve one char for the final \0! */ + CHKiRet(ExtendBuf(iparam, iLenVal + 1)); + memcpy(iparam->param, pVal, iLenVal+1); if(bMustBeFreed) free(pVal); FINALIZE; @@ -193,9 +191,8 @@ tplToString(struct template *pTpl, msg_t *pMsg, uchar **ppBuf, size_t *pLenBuf, iLenVal = pTpe->data.constant.iLenConstant; bMustBeFreed = 0; } else if(pTpe->eEntryType == FIELD) { - pVal = (uchar*) MsgGetProp(pMsg, pTpe, pTpe->data.field.propid, - pTpe->data.field.propName, &iLenVal, - &bMustBeFreed, ttNow); + pVal = (uchar*) MsgGetProp(pMsg, pTpe, &pTpe->data.field.msgProp, + &iLenVal, &bMustBeFreed, ttNow); /* we now need to check if we should use SQL option. In this case, * we must go over the generated string and escape '\'' characters. * rgerhards, 2005-09-22: the option values below look somewhat misplaced, @@ -212,10 +209,10 @@ tplToString(struct template *pTpl, msg_t *pMsg, uchar **ppBuf, size_t *pLenBuf, /* got source, now copy over */ if(iLenVal > 0) { /* may be zero depending on property */ /* first, make sure buffer fits */ - if(iBuf + iLenVal >= *pLenBuf) /* we reserve one char for the final \0! */ - CHKiRet(ExtendBuf(ppBuf, pLenBuf, iBuf + iLenVal + 1)); + if(iBuf + iLenVal >= iparam->lenBuf) /* we reserve one char for the final \0! */ + CHKiRet(ExtendBuf(iparam, iBuf + iLenVal + 1)); - memcpy(*ppBuf + iBuf, pVal, iLenVal); + memcpy(iparam->param + iBuf, pVal, iLenVal); iBuf += iLenVal; } @@ -225,15 +222,16 @@ tplToString(struct template *pTpl, msg_t *pMsg, uchar **ppBuf, size_t *pLenBuf, pTpe = pTpe->pNext; } - if(iBuf == *pLenBuf) { + if(iBuf == iparam->lenBuf) { /* in the weired case of an *empty* template, this can happen. * it is debatable if we should really fix it here or simply * forbid that case. However, performance toll is minimal, so - * I tend to permit it. -- 201011-05 rgerhards + * I tend to permit it. -- 2010-11-05 rgerhards */ - CHKiRet(ExtendBuf(ppBuf, pLenBuf, iBuf + 1)); + CHKiRet(ExtendBuf(iparam, iBuf + 1)); } - (*ppBuf)[iBuf] = '\0'; + iparam->param[iBuf] = '\0'; + iparam->lenStr = iBuf; finalize_it: RETiRet; @@ -264,12 +262,12 @@ tplToArray(struct template *pTpl, msg_t *pMsg, uchar*** ppArr, struct syslogTime assert(pMsg != NULL); assert(ppArr != NULL); - if(pTpl->subtree) { + if(pTpl->bHaveSubtree) { /* Note: this mode is untested, as there is no official plugin * using array passing, so I simply could not test it. */ CHKmalloc(pArr = calloc(2, sizeof(uchar*))); - getCEEPropVal(pMsg, pTpl->subtree, &pVal, &propLen, &bMustBeFreed); + getJSONPropVal(pMsg, &pTpl->subtree, &pVal, &propLen, &bMustBeFreed); if(bMustBeFreed) { /* if it must be freed, it is our own private copy... */ pArr[0] = pVal; /* ... so we can use it! */ } else { @@ -290,9 +288,8 @@ tplToArray(struct template *pTpl, msg_t *pMsg, uchar*** ppArr, struct syslogTime if(pTpe->eEntryType == CONSTANT) { CHKmalloc(pArr[iArr] = (uchar*)strdup((char*) pTpe->data.constant.pConstant)); } else if(pTpe->eEntryType == FIELD) { - pVal = (uchar*) MsgGetProp(pMsg, pTpe, pTpe->data.field.propid, - pTpe->data.field.propName, &propLen, - &bMustBeFreed, ttNow); + pVal = (uchar*) MsgGetProp(pMsg, pTpe, &pTpe->data.field.msgProp, + &propLen, &bMustBeFreed, ttNow); if(bMustBeFreed) { /* if it must be freed, it is our own private copy... */ pArr[iArr] = pVal; /* ... so we can use it! */ } else { @@ -326,8 +323,8 @@ tplToJSON(struct template *pTpl, msg_t *pMsg, struct json_object **pjson, struct rsRetVal localRet; DEFiRet; - if(pTpl->subtree != NULL){ - localRet = jsonFind(pMsg, pTpl->subtree, pjson); + if(pTpl->bHaveSubtree){ + localRet = jsonFind(pMsg->json, &pTpl->subtree, pjson); if(*pjson == NULL) { /* we need to have a root object! */ *pjson = json_object_new_object(); @@ -345,23 +342,24 @@ tplToJSON(struct template *pTpl, msg_t *pMsg, struct json_object **pjson, struct jsonf = json_object_new_string((char*) pTpe->data.constant.pConstant); json_object_object_add(json, (char*)pTpe->fieldName, jsonf); } else if(pTpe->eEntryType == FIELD) { - if(pTpe->data.field.propid == PROP_CEE) { - localRet = msgGetCEEPropJSON(pMsg, pTpe->data.field.propName, &jsonf); + if(pTpe->data.field.msgProp.id == PROP_CEE || + pTpe->data.field.msgProp.id == PROP_LOCAL_VAR || + pTpe->data.field.msgProp.id == PROP_GLOBAL_VAR ) { + localRet = msgGetJSONPropJSON(pMsg, &pTpe->data.field.msgProp, &jsonf); if(localRet == RS_RET_OK) { json_object_object_add(json, (char*)pTpe->fieldName, json_object_get(jsonf)); } else { - DBGPRINTF("tplToJSON: error %d looking up property\n", - localRet); + DBGPRINTF("tplToJSON: error %d looking up property %s\n", + localRet, pTpe->fieldName); if(pTpe->data.field.options.bMandatory) { json_object_object_add(json, (char*)pTpe->fieldName, NULL); } } } else { - pVal = (uchar*) MsgGetProp(pMsg, pTpe, pTpe->data.field.propid, - pTpe->data.field.propName, &propLen, - &bMustBeFreed, ttNow); + pVal = (uchar*) MsgGetProp(pMsg, pTpe, &pTpe->data.field.msgProp, + &propLen, &bMustBeFreed, ttNow); if(pTpe->data.field.options.bMandatory || propLen > 0) { - jsonf = json_object_new_string_len((char*)pVal, propLen); + jsonf = json_object_new_string_len((char*)pVal, propLen+1); json_object_object_add(json, (char*)pTpe->fieldName, jsonf); } if(bMustBeFreed) { /* json-c makes its own private copy! */ @@ -477,7 +475,7 @@ doEscape(uchar **pp, rs_size_t *pLen, unsigned short *pbMustBeFreed, int mode) ++p; } CHKiRet(cstrFinalize(pStrB)); - CHKiRet(cstrConvSzStrAndDestruct(pStrB, &pszGenerated, 0)); + CHKiRet(cstrConvSzStrAndDestruct(&pStrB, &pszGenerated, 0)); if(*pbMustBeFreed) free(*pp); /* discard previous value */ @@ -638,7 +636,7 @@ do_Constant(unsigned char **pp, struct template *pTpl, int bDoEscapes) * 2005-09-09 rgerhards */ pTpe->data.constant.iLenConstant = rsCStrLen(pStrB); - CHKiRet(cstrConvSzStrAndDestruct(pStrB, &pTpe->data.constant.pConstant, 0)); + CHKiRet(cstrConvSzStrAndDestruct(&pStrB, &pTpe->data.constant.pConstant, 0)); *pp = p; @@ -646,6 +644,17 @@ finalize_it: RETiRet; } +/* Helper that checks to see if a property already has a format + * type defined + */ +static int hasFormat(struct templateEntry *pTpe) { + return ( + pTpe->data.field.options.bCSV || + pTpe->data.field.options.bJSON || + pTpe->data.field.options.bJSONf || + pTpe->data.field.options.bJSONr + ); +} /* Helper to do_Parameter(). This parses the formatting options * specified in a template variable. It returns the passed-in pointer @@ -695,6 +704,28 @@ static void doOptions(unsigned char **pp, struct templateEntry *pTpe) pTpe->data.field.eDateFormat = tplFmtUnixDate; } else if(!strcmp((char*)Buf, "date-subseconds")) { pTpe->data.field.eDateFormat = tplFmtSecFrac; + } else if(!strcmp((char*)Buf, "date-wdayname")) { + pTpe->data.field.eDateFormat = tplFmtWDayName; + } else if(!strcmp((char*)Buf, "date-wday")) { + pTpe->data.field.eDateFormat = tplFmtWDay; + } else if(!strcmp((char*)Buf, "date-year")) { + pTpe->data.field.eDateFormat = tplFmtYear; + } else if(!strcmp((char*)Buf, "date-month")) { + pTpe->data.field.eDateFormat = tplFmtMonth; + } else if(!strcmp((char*)Buf, "date-day")) { + pTpe->data.field.eDateFormat = tplFmtDay; + } else if(!strcmp((char*)Buf, "date-hour")) { + pTpe->data.field.eDateFormat = tplFmtHour; + } else if(!strcmp((char*)Buf, "date-minute")) { + pTpe->data.field.eDateFormat = tplFmtMinute; + } else if(!strcmp((char*)Buf, "date-second")) { + pTpe->data.field.eDateFormat = tplFmtSecond; + } else if(!strcmp((char*)Buf, "date-tzoffshour")) { + pTpe->data.field.eDateFormat = tplFmtTZOffsHour; + } else if(!strcmp((char*)Buf, "date-tzoffsmin")) { + pTpe->data.field.eDateFormat = tplFmtTZOffsMin; + } else if(!strcmp((char*)Buf, "date-tzoffsdirection")) { + pTpe->data.field.eDateFormat = tplFmtTZOffsDirection; } else if(!strcmp((char*)Buf, "lowercase")) { pTpe->data.field.eCaseConv = tplCaseConvLower; } else if(!strcmp((char*)Buf, "uppercase")) { @@ -716,26 +747,40 @@ static void doOptions(unsigned char **pp, struct templateEntry *pTpe) } else if(!strcmp((char*)Buf, "pos-end-relative")) { pTpe->data.field.options.bFromPosEndRelative = 1; } else if(!strcmp((char*)Buf, "csv")) { - if(pTpe->data.field.options.bJSON || pTpe->data.field.options.bJSONf) { + if(hasFormat(pTpe)) { errmsg.LogError(0, NO_ERRCODE, "error: can only specify " - "one option out of (json, jsonf, csv) - csv ignored"); + "one option out of (json, jsonf, jsonr, jsonfr, csv) - csv ignored"); } else { pTpe->data.field.options.bCSV = 1; } } else if(!strcmp((char*)Buf, "json")) { - if(pTpe->data.field.options.bCSV || pTpe->data.field.options.bJSON) { + if(hasFormat(pTpe)) { errmsg.LogError(0, NO_ERRCODE, "error: can only specify " - "one option out of (json, jsonf, csv) - json ignored"); + "one option out of (json, jsonf, jsonr, jsonfr, csv) - json ignored"); } else { pTpe->data.field.options.bJSON = 1; } } else if(!strcmp((char*)Buf, "jsonf")) { - if(pTpe->data.field.options.bCSV || pTpe->data.field.options.bJSON) { + if(hasFormat(pTpe)) { errmsg.LogError(0, NO_ERRCODE, "error: can only specify " - "one option out of (json, jsonf, csv) - jsonf ignored"); + "one option out of (json, jsonf, jsonr, jsonfr, csv) - jsonf ignored"); } else { pTpe->data.field.options.bJSONf = 1; } + } else if(!strcmp((char*)Buf, "jsonr")) { + if(hasFormat(pTpe)) { + errmsg.LogError(0, NO_ERRCODE, "error: can only specify " + "one option out of (json, jsonf, jsonr, jsonfr, csv) - jsonr ignored"); + } else { + pTpe->data.field.options.bJSONr = 1; + } + } else if(!strcmp((char*)Buf, "jsonfr")) { + if(hasFormat(pTpe)) { + errmsg.LogError(0, NO_ERRCODE, "error: can only specify " + "one option out of (json, jsonf, jsonr, jsonfr, csv) - jsonfr ignored"); + } else { + pTpe->data.field.options.bJSONfr = 1; + } } else if(!strcmp((char*)Buf, "mandatory-field")) { pTpe->data.field.options.bMandatory = 1; } else { @@ -747,7 +792,6 @@ static void doOptions(unsigned char **pp, struct templateEntry *pTpe) *pp = p; } - /* helper to tplAddLine. Parses a parameter and generates * the necessary structure. */ @@ -755,7 +799,7 @@ static rsRetVal do_Parameter(uchar **pp, struct template *pTpl) { uchar *p; - cstr_t *pStrProp; + cstr_t *pStrProp = NULL; cstr_t *pStrField = NULL; struct templateEntry *pTpe; int iNum; /* to compute numbers */ @@ -785,19 +829,8 @@ do_Parameter(uchar **pp, struct template *pTpl) /* got the name */ cstrFinalize(pStrProp); - if(propNameToID(pStrProp, &pTpe->data.field.propid) != RS_RET_OK) { - errmsg.LogError(0, RS_RET_TPL_INVLD_PROP, "template '%s': invalid parameter '%s'", - pTpl->pszName, cstrGetSzStrNoNULL(pStrProp)); - cstrDestruct(&pStrProp); - ABORT_FINALIZE(RS_RET_TPL_INVLD_PROP); - } - if(pTpe->data.field.propid == PROP_CEE) { - /* in CEE case, we need to preserve the actual property name */ - if((pTpe->data.field.propName = es_newStrFromCStr((char*)cstrGetSzStrNoNULL(pStrProp)+1, cstrLen(pStrProp)-1)) == NULL) { - cstrDestruct(&pStrProp); - ABORT_FINALIZE(RS_RET_OUT_OF_MEMORY); - } - } + CHKiRet(msgPropDescrFill(&pTpe->data.field.msgProp, cstrGetSzStrNoNULL(pStrProp), + cstrLen(pStrProp))); /* Check frompos, if it has an R, then topos should be a regex */ if(*p == ':') { @@ -921,7 +954,7 @@ do_Parameter(uchar **pp, struct template *pTpl) if(iNum < 0 || iNum > 255) { errmsg.LogError(0, NO_ERRCODE, "error: non-USASCII delimiter character value %d in template - using 9 (HT) as substitute", iNum); pTpe->data.field.field_delim = 9; - } else { + } else { pTpe->data.field.field_delim = iNum; # ifdef STRICT_GPLV3 if (*p == '+') { @@ -935,8 +968,12 @@ do_Parameter(uchar **pp, struct template *pTpl) while(isdigit((int)*p)) iNum = iNum * 10 + *p++ - '0'; pTpe->data.field.iFromPos = iNum; + } else if(*p != ':') { + parser_errmsg("error: invalid character '%c' in frompos after \"F,\", property: '%s' " + "be sure to use DECIMAL character codes!", *p, (char*) *pp); + ABORT_FINALIZE(RS_RET_SYNTAX_ERROR); } - } + } } } else { /* invalid character after F, so we need to reject @@ -1094,8 +1131,8 @@ do_Parameter(uchar **pp, struct template *pTpl) /* save field name - if none was given, use the property name instead */ if(pStrField == NULL) { - if(pTpe->data.field.propid == PROP_CEE) { - /* in CEE case, we remove "$!" from the fieldname - it's just our indicator */ + if(pTpe->data.field.msgProp.id == PROP_CEE || pTpe->data.field.msgProp.id == PROP_LOCAL_VAR) { + /* in CEE case, we remove "$!"/"$." from the fieldname - it's just our indicator */ pTpe->fieldName = ustrdup(cstrGetSzStrNoNULL(pStrProp)+2); pTpe->lenFieldName = cstrLen(pStrProp)-2; } else { @@ -1111,10 +1148,11 @@ do_Parameter(uchar **pp, struct template *pTpl) DBGPRINTF("template/do_Parameter: fieldName is NULL!\n"); ABORT_FINALIZE(RS_RET_OUT_OF_MEMORY); } - cstrDestruct(&pStrProp); if(*p) ++p; /* eat '%' */ *pp = p; finalize_it: + if(pStrProp != NULL) + cstrDestruct(&pStrProp); RETiRet; } @@ -1184,6 +1222,7 @@ struct template *tplAddLine(rsconf_t *conf, char* pName, uchar** ppRestOfConfLin if((pTpl = tplConstruct(conf)) == NULL) return NULL; + DBGPRINTF("tplAddLine processing template '%s'\n", pName); pTpl->iLenName = strlen(pName); pTpl->pszName = (char*) MALLOC(sizeof(char) * (pTpl->iLenName + 1)); if(pTpl->pszName == NULL) { @@ -1371,7 +1410,7 @@ createPropertyTpe(struct template *pTpl, struct cnfobj *o) int bPosRelativeToEnd = 0; char *re_expr = NULL; struct cnfparamvals *pvals = NULL; - enum {F_NONE, F_CSV, F_JSON, F_JSONF} formatType = F_NONE; + enum {F_NONE, F_CSV, F_JSON, F_JSONF, F_JSONR, F_JSONFR} formatType = F_NONE; enum {CC_NONE, CC_ESCAPE, CC_SPACE, CC_DROP} controlchr = CC_NONE; enum {SP_NONE, SP_DROP, SP_REPLACE} secpath = SP_NONE; enum tplFormatCaseConvTypes caseconv = tplCaseConvNo; @@ -1463,6 +1502,10 @@ createPropertyTpe(struct template *pTpl, struct cnfobj *o) formatType = F_JSON; } else if(!es_strbufcmp(pvals[i].val.d.estr, (uchar*)"jsonf", sizeof("jsonf")-1)) { formatType = F_JSONF; + } else if(!es_strbufcmp(pvals[i].val.d.estr, (uchar*)"jsonr", sizeof("jsonr")-1)) { + formatType = F_JSONR; + } else if(!es_strbufcmp(pvals[i].val.d.estr, (uchar*)"jsonfr", sizeof("jsonfr")-1)) { + formatType = F_JSONFR; } else { uchar *typeStr = (uchar*) es_str2cstr(pvals[i].val.d.estr, NULL); errmsg.LogError(0, RS_RET_ERR, "invalid format type '%s' for property", @@ -1526,6 +1569,28 @@ createPropertyTpe(struct template *pTpl, struct cnfobj *o) datefmt = tplFmtUnixDate; } else if(!es_strbufcmp(pvals[i].val.d.estr, (uchar*)"subseconds", sizeof("subseconds")-1)) { datefmt = tplFmtSecFrac; + } else if(!es_strbufcmp(pvals[i].val.d.estr, (uchar*)"wdayname", sizeof("wdayname")-1)) { + datefmt = tplFmtWDayName; + } else if(!es_strbufcmp(pvals[i].val.d.estr, (uchar*)"wday", sizeof("wday")-1)) { + datefmt = tplFmtWDay; + } else if(!es_strbufcmp(pvals[i].val.d.estr, (uchar*)"year", sizeof("year")-1)) { + datefmt = tplFmtYear; + } else if(!es_strbufcmp(pvals[i].val.d.estr, (uchar*)"month", sizeof("month")-1)) { + datefmt = tplFmtMonth; + } else if(!es_strbufcmp(pvals[i].val.d.estr, (uchar*)"day", sizeof("day")-1)) { + datefmt = tplFmtDay; + } else if(!es_strbufcmp(pvals[i].val.d.estr, (uchar*)"hour", sizeof("hour")-1)) { + datefmt = tplFmtHour; + } else if(!es_strbufcmp(pvals[i].val.d.estr, (uchar*)"minute", sizeof("minute")-1)) { + datefmt = tplFmtMinute; + } else if(!es_strbufcmp(pvals[i].val.d.estr, (uchar*)"second", sizeof("second")-1)) { + datefmt = tplFmtSecond; + } else if(!es_strbufcmp(pvals[i].val.d.estr, (uchar*)"tzoffshour", sizeof("tzoffshour")-1)) { + datefmt = tplFmtTZOffsHour; + } else if(!es_strbufcmp(pvals[i].val.d.estr, (uchar*)"tzoffsmin", sizeof("tzoffsmin")-1)) { + datefmt = tplFmtTZOffsMin; + } else if(!es_strbufcmp(pvals[i].val.d.estr, (uchar*)"tzoffsdirection", sizeof("tzoffsdirection")-1)) { + datefmt = tplFmtTZOffsDirection; } else { uchar *typeStr = (uchar*) es_str2cstr(pvals[i].val.d.estr, NULL); errmsg.LogError(0, RS_RET_ERR, "invalid date format '%s' for property", @@ -1574,12 +1639,8 @@ createPropertyTpe(struct template *pTpl, struct cnfobj *o) /* apply */ CHKmalloc(pTpe = tpeConstruct(pTpl)); pTpe->eEntryType = FIELD; - CHKiRet(propNameToID(name, &pTpe->data.field.propid)); - if(pTpe->data.field.propid == PROP_CEE) { - /* in CEE case, we need to preserve the actual property name */ - pTpe->data.field.propName = es_newStrFromCStr((char*)cstrGetSzStrNoNULL(name)+1, - cstrLen(name)-1); - } + CHKiRet(msgPropDescrFill(&pTpe->data.field.msgProp, cstrGetSzStrNoNULL(name), + cstrLen(name))); pTpe->data.field.options.bDropLastLF = droplastlf; pTpe->data.field.options.bSPIffNo1stSP = spifno1stsp; pTpe->data.field.options.bMandatory = mandatory; @@ -1597,6 +1658,12 @@ createPropertyTpe(struct template *pTpl, struct cnfobj *o) case F_JSONF: pTpe->data.field.options.bJSONf = 1; break; + case F_JSONR: + pTpe->data.field.options.bJSONr = 1; + break; + case F_JSONFR: + pTpe->data.field.options.bJSONfr = 1; + break; } switch(controlchr) { case CC_NONE: @@ -1712,8 +1779,9 @@ tplProcessCnf(struct cnfobj *o) char *name = NULL; uchar *tplStr = NULL; uchar *plugin = NULL; - es_str_t *subtree = NULL; uchar *p; + msgPropDescr_t subtree; + sbool bHaveSubtree = 0; enum { T_STRING, T_PLUGIN, T_LIST, T_SUBTREE } tplType = T_STRING; /* init just to keep compiler happy: mandatory parameter */ int i; @@ -1760,10 +1828,11 @@ tplProcessCnf(struct cnfobj *o) free(name); /* overall assigned */ ABORT_FINALIZE(RS_RET_ERR); } else { - /* TODO: unify strings! */ - char *cstr = es_str2cstr(pvals[i].val.d.estr, NULL); - subtree = es_newStrFromBuf(cstr+1, es_strlen(pvals[i].val.d.estr)-1); + uchar *cstr; + cstr = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + CHKiRet(msgPropDescrFill(&subtree, cstr, ustrlen(cstr))); free(cstr); + bHaveSubtree = 1; } } else if(!strcmp(pblk.descr[i].name, "plugin")) { plugin = (uchar*) es_str2cstr(pvals[i].val.d.estr, NULL); @@ -1806,7 +1875,7 @@ tplProcessCnf(struct cnfobj *o) } } - if(subtree == NULL) { + if(!bHaveSubtree) { if(tplType == T_SUBTREE) { errmsg.LogError(0, RS_RET_ERR, "template '%s' of type subtree needs " "subtree parameter", name); @@ -1876,7 +1945,8 @@ tplProcessCnf(struct cnfobj *o) break; case T_LIST: createListTpl(pTpl, o); break; - case T_SUBTREE: pTpl->subtree = subtree; + case T_SUBTREE: memcpy(&pTpl->subtree, &subtree, sizeof(msgPropDescr_t)); + pTpl->bHaveSubtree = 1; break; } @@ -1968,9 +2038,8 @@ void tplDeleteAll(rsconf_t *conf) regexp.regfree(&(pTpeDel->data.field.re)); } } - if(pTpeDel->data.field.propName != NULL) - es_deleteStr(pTpeDel->data.field.propName); #endif + msgPropDescrDestruct(&pTpeDel->data.field.msgProp); break; } free(pTpeDel->fieldName); @@ -1980,8 +2049,8 @@ void tplDeleteAll(rsconf_t *conf) pTplDel = pTpl; pTpl = pTpl->pNext; free(pTplDel->pszName); - if(pTplDel->subtree != NULL) - es_deleteStr(pTplDel->subtree); + if(pTplDel->bHaveSubtree) + msgPropDescrDestruct(&pTplDel->subtree); free(pTplDel); } ENDfunc @@ -2028,9 +2097,8 @@ void tplDeleteNew(rsconf_t *conf) regexp.regfree(&(pTpeDel->data.field.re)); } } - if(pTpeDel->data.field.propName != NULL) - es_deleteStr(pTpeDel->data.field.propName); #endif + msgPropDescrDestruct(&pTpeDel->data.field.msgProp); break; } /*dbgprintf("\n");*/ @@ -2039,8 +2107,8 @@ void tplDeleteNew(rsconf_t *conf) pTplDel = pTpl; pTpl = pTpl->pNext; free(pTplDel->pszName); - if(pTplDel->subtree != NULL) - es_deleteStr(pTplDel->subtree); + if(pTplDel->bHaveSubtree) + msgPropDescrDestruct(&pTplDel->subtree); free(pTplDel); } ENDfunc @@ -2082,11 +2150,13 @@ void tplPrintList(rsconf_t *conf) pTpe->data.constant.pConstant); break; case FIELD: - dbgprintf("(FIELD), value: '%d' ", pTpe->data.field.propid); - if(pTpe->data.field.propid == PROP_CEE) { - char *cstr = es_str2cstr(pTpe->data.field.propName, NULL); - dbgprintf("[EE-Property: '%s'] ", cstr); - free(cstr); + dbgprintf("(FIELD), value: '%d' ", pTpe->data.field.msgProp.id); + if(pTpe->data.field.msgProp.id == PROP_CEE) { + dbgprintf("[EE-Property: '%s'] ", pTpe->data.field.msgProp.name); + } else if(pTpe->data.field.msgProp.id == PROP_LOCAL_VAR) { + dbgprintf("[Local Var: '%s'] ", pTpe->data.field.msgProp.name); + //} else if(pTpe->data.field.propid == PROP_GLOBAL_VAR) { + // dbgprintf("[Global Var: '%s'] ", pTpe->data.field.propName); } switch(pTpe->data.field.eDateFormat) { case tplFmtDefault: @@ -2152,6 +2222,12 @@ void tplPrintList(rsconf_t *conf) if(pTpe->data.field.options.bJSONf) { dbgprintf("[format as JSON field] "); } + if(pTpe->data.field.options.bJSONr) { + dbgprintf("[format as JSON without re-escaping] "); + } + if(pTpe->data.field.options.bJSONfr) { + dbgprintf("[format as JSON field without re-escaping] "); + } if(pTpe->data.field.options.bMandatory) { dbgprintf("[mandatory field] "); } @@ -1,7 +1,7 @@ /* This is the header for template processing code of rsyslog. * begun 2004-11-17 rgerhards * - * Copyright (C) 2004-2012 by Rainer Gerhards and Adiscon GmbH + * Copyright (C) 2004-2013 by Rainer Gerhards and Adiscon GmbH * * This file is part of rsyslog. * @@ -30,7 +30,7 @@ #ifndef TEMPLATE_H_INCLUDED #define TEMPLATE_H_INCLUDED 1 -#include <json/json.h> +#include <json.h> #include <libestr.h> #include "regexp.h" #include "stringbuf.h" @@ -39,8 +39,9 @@ struct template { struct template *pNext; char *pszName; int iLenName; - rsRetVal (*pStrgen)(msg_t*, uchar**, size_t *); - es_str_t *subtree; /* subtree name for subtree-type templates */ + rsRetVal (*pStrgen)(const msg_t*const, actWrkrIParams_t *const iparam); + sbool bHaveSubtree; + msgPropDescr_t subtree; /* subtree property name for subtree-type templates */ int tpenElements; /* number of elements in templateEntry list */ struct templateEntry *pEntryRoot; struct templateEntry *pEntryLast; @@ -58,7 +59,11 @@ struct template { enum EntryTypes { UNDEFINED = 0, CONSTANT = 1, FIELD = 2 }; enum tplFormatTypes { tplFmtDefault = 0, tplFmtMySQLDate = 1, tplFmtRFC3164Date = 2, tplFmtRFC3339Date = 3, tplFmtPgSQLDate = 4, - tplFmtSecFrac = 5, tplFmtRFC3164BuggyDate = 6, tplFmtUnixDate}; + tplFmtSecFrac = 5, tplFmtRFC3164BuggyDate = 6, tplFmtUnixDate = 7, + tplFmtWDayName = 8, tplFmtYear = 9, tplFmtMonth = 10, tplFmtDay = 11, + tplFmtHour = 12, tplFmtMinute = 13, tplFmtSecond = 14, + tplFmtTZOffsHour = 15, tplFmtTZOffsMin = 16, tplFmtTZOffsDirection = 17, + tplFmtWDay = 18}; enum tplFormatCaseConvTypes { tplCaseConvNo = 0, tplCaseConvUpper = 1, tplCaseConvLower = 2 }; enum tplRegexType { TPL_REGEX_BRE = 0, /* posix BRE */ TPL_REGEX_ERE = 1 /* posix ERE */ @@ -79,7 +84,7 @@ struct templateEntry { int iLenConstant; /* its length */ } constant; struct { - propid_t propid; /* property to be used */ + msgPropDescr_t msgProp; /* property to be used */ unsigned iFromPos; /* for partial strings only chars from this position ... */ unsigned iToPos; /* up to that one... */ unsigned iFieldNr; /* for field extraction: field to extract */ @@ -103,7 +108,6 @@ struct templateEntry { int field_expand; /* use multiple instances of the field delimiter as a single one? */ #endif - es_str_t *propName; /**< property name (currently being used for CEE only) */ enum tplFormatTypes eDateFormat; enum tplFormatCaseConvTypes eCaseConv; @@ -118,6 +122,8 @@ struct templateEntry { unsigned bCSV: 1; /* format field in CSV (RFC 4180) format */ unsigned bJSON: 1; /* format field JSON escaped */ unsigned bJSONf: 1; /* format field JSON *field* (n/v pair) */ + unsigned bJSONr: 1; /* format field JSON non escaped */ + unsigned bJSONfr: 1; /* format field JSON *field* non escaped (n/v pair) */ unsigned bMandatory: 1; /* mandatory field - emit even if empty */ unsigned bFromPosEndRelative: 1;/* is From/To-Pos relative to end of string? */ } options; /* options as bit fields */ @@ -143,7 +149,7 @@ void tplDeleteAll(rsconf_t *conf); void tplDeleteNew(rsconf_t *conf); void tplPrintList(rsconf_t *conf); void tplLastStaticInit(rsconf_t *conf, struct template *tpl); -rsRetVal ExtendBuf(uchar **pBuf, size_t *pLenBuf, size_t iMinSize); +rsRetVal ExtendBuf(actWrkrIParams_t *const iparam, const size_t iMinSize); int tplRequiresDateCall(struct template *pTpl); /* note: if a compiler warning for undefined type tells you to look at this * code line below, the actual cause is that you currently MUST include template.h @@ -151,9 +157,13 @@ int tplRequiresDateCall(struct template *pTpl); * rgerhards, 2007-08-06 */ rsRetVal tplToArray(struct template *pTpl, msg_t *pMsg, uchar*** ppArr, struct syslogTime *ttNow); -rsRetVal tplToString(struct template *pTpl, msg_t *pMsg, uchar** ppSz, size_t *, struct syslogTime *ttNow); rsRetVal tplToJSON(struct template *pTpl, msg_t *pMsg, struct json_object **, struct syslogTime *ttNow); rsRetVal doEscape(uchar **pp, rs_size_t *pLen, unsigned short *pbMustBeFreed, int escapeMode); +rsRetVal +tplToString(struct template *__restrict__ const pTpl, + msg_t *__restrict__ const pMsg, + actWrkrIParams_t *__restrict const iparam, + struct syslogTime *const ttNow); rsRetVal templateInit(); rsRetVal tplProcessCnf(struct cnfobj *o); diff --git a/tests/Makefile.am b/tests/Makefile.am index b339e79..0fa5a36 100644 --- a/tests/Makefile.am +++ b/tests/Makefile.am @@ -6,7 +6,16 @@ TESTS = $(TESTRUNS) if ENABLE_IMDIAG TESTS += \ + stop-localvar.sh \ + stop-msgvar.sh \ + fac_authpriv.sh \ + fac_local0.sh \ + fac_mail.sh \ + fac_news.sh \ + fac_uucp.sh \ + rfc5424parser.sh \ arrayqueue.sh \ + global_vars.sh \ da-mainmsg-q.sh \ validation-run.sh \ imtcp-multiport.sh \ @@ -14,6 +23,7 @@ TESTS += \ diskqueue.sh \ diskqueue-fsync.sh \ rulesetmultiqueue.sh \ + rulesetmultiqueue-v6.sh \ manytcp.sh \ rsf_getenv.sh \ imtcp_conndrop.sh \ @@ -45,6 +55,7 @@ TESTS += \ execonlywhenprevsuspended2.sh \ execonlywhenprevsuspended3.sh \ execonlywhenprevsuspended4.sh \ + execonlywhenprevsuspended_multiwrkr.sh \ pipe_noreader.sh \ dircreate_dflt.sh \ dircreate_off.sh \ @@ -53,10 +64,11 @@ TESTS += \ imuxsock_ccmiddle_root.sh \ udp-msgreduc-vg.sh \ udp-msgreduc-orgmsg-vg.sh \ - queue-persist.sh + queue-persist.sh \ discard-rptdmsg.sh \ discard-allmark.sh \ discard.sh \ + stop.sh \ failover-async.sh \ failover-double.sh \ failover-basic.sh \ @@ -71,6 +83,7 @@ TESTS += \ rscript_prifilt.sh \ rscript_optimizer1.sh \ rscript_ruleset_call.sh \ + rs_optimizer_pri.sh \ cee_simple.sh \ cee_diskqueue.sh \ incltest.sh \ @@ -96,7 +109,9 @@ if ENABLE_MYSQL_TESTS TESTS += \ mysql-basic.sh \ mysql-basic-cnf6.sh \ - mysql-asyn.sh + mysql-asyn.sh \ + mysql-actq-mt.sh \ + mysql-actq-mt-withpause.sh if ENABLE_OMLIBDBI TESTS += \ libdbi-basic.sh \ @@ -105,7 +120,8 @@ endif if HAVE_VALGRIND TESTS += \ mysql-basic-vg.sh \ - mysql-asyn-vg.sh + mysql-asyn-vg.sh \ + mysql-actq-mt-withpause-vg.sh endif endif @@ -117,6 +133,27 @@ TESTS += \ imptcp_conndrop.sh endif +if ENABLE_ELASTICSEARCH +TESTS += \ + elasticsearch-basic.sh \ + elasticsearch-basic-bulk.sh \ + elasticsearch-basic-errorfile-empty.sh \ + elasticsearch-basic-errorfile-populated.sh \ + elasticsearch-bulk-errorfile-empty.sh \ + elasticsearch-bulk-errorfile-populated.sh +endif + +if ENABLE_MMPSTRUCDATA +TESTS += \ + mmpstrucdata.sh +endif + +if ENABLE_MMJSONPARSE +TESTS += \ + mmjsonparse_simple.sh \ + mmjsonparse_cim.sh +endif + if ENABLE_GNUTLS # TODO: re-enable in newer version #TESTS += \ @@ -134,6 +171,10 @@ if ENABLE_OMUXSOCK TESTS += uxsock_simple.sh endif +if ENABLE_RELP +TESTS += sndrcv_relp.sh +endif + if ENABLE_OMUDPSPOOF TESTS += sndrcv_omudpspoof.sh \ sndrcv_omudpspoof_nonstdpt.sh @@ -289,6 +330,40 @@ EXTRA_DIST= 1.rstest 2.rstest 3.rstest err1.rstest \ testsuites/rscript_stop.conf \ rscript_stop2.sh \ testsuites/rscript_stop2.conf \ + stop.sh \ + testsuites/stop.conf \ + rscript_le.sh \ + testsuites/rscript_le.conf \ + rscript_ge.sh \ + testsuites/rscript_ge.conf \ + rscript_lt.sh \ + testsuites/rscript_lt.conf \ + rscript_gt.sh \ + testsuites/rscript_gt.conf \ + rscript_ne.sh \ + testsuites/rscript_ne.conf \ + rscript_eq.sh \ + testsuites/rscript_eq.conf \ + stop-localvar.sh \ + testsuites/stop-localvar.conf \ + stop-msgvar.sh \ + testsuites/stop-msgvar.conf \ + global_vars.sh \ + testsuites/global_vars.conf \ + rfc5424parser.sh \ + testsuites/rfc5424parser.conf \ + fac_authpriv.sh \ + testsuites/fac_authpriv.conf \ + fac_local0.sh \ + testsuites/fac_local0.conf \ + fac_mail.sh \ + testsuites/fac_mail.conf \ + fac_news.sh \ + testsuites/fac_news.conf \ + fac_uucp.sh \ + testsuites/fac_uucp.conf \ + rs_optimizer_pri.sh \ + testsuites/rs_optimizer_pri.conf \ rscript_prifilt.sh \ testsuites/rscript_prifilt.conf \ rscript_optimizer1.sh \ @@ -299,6 +374,10 @@ EXTRA_DIST= 1.rstest 2.rstest 3.rstest err1.rstest \ testsuites/cee_simple.conf \ cee_diskqueue.sh \ testsuites/cee_diskqueue.conf \ + mmjsonparse_simple.sh \ + testsuites/mmjsonparse_simple.conf \ + mmjsonparse_cim.sh \ + testsuites/mmjsonparse_cim.conf \ incltest.sh \ testsuites/incltest.conf \ incltest_dir.sh \ @@ -308,6 +387,18 @@ EXTRA_DIST= 1.rstest 2.rstest 3.rstest err1.rstest \ incltest_dir_wildcard.sh \ testsuites/incltest_dir_wildcard.conf \ testsuites/incltest.d/include.conf \ + elasticsearch-basic.sh \ + testsuites/elasticsearch-basic.conf \ + elasticsearch-basic-bulk.sh \ + testsuites/elasticsearch-basic-bulk.conf \ + elasticsearch-basic-errorfile-empty.sh \ + testsuites/elasticsearch-basic-errorfile-empty.conf \ + elasticsearch-basic-errorfile-populated.sh \ + testsuites/elasticsearch-basic-errorfile-populated.conf \ + elasticsearch-bulk-errorfile-empty.sh \ + testsuites/elasticsearch-bulk-errorfile-empty.conf \ + elasticsearch-bulk-errorfile-populated.sh \ + testsuites/elasticsearch-bulk-errorfile-populated.conf \ linkedlistqueue.sh \ testsuites/linkedlistqueue.conf \ da-mainmsg-q.sh \ @@ -397,6 +488,9 @@ EXTRA_DIST= 1.rstest 2.rstest 3.rstest err1.rstest \ sndrcv.sh \ testsuites/sndrcv_sender.conf \ testsuites/sndrcv_rcvr.conf \ + sndrcv_relp.sh \ + testsuites/sndrcv_relp_sender.conf \ + testsuites/sndrcv_relp_rcvr.conf \ sndrcv_udp.sh \ testsuites/sndrcv_udp_sender.conf \ testsuites/sndrcv_udp_rcvr.conf \ @@ -460,6 +554,8 @@ EXTRA_DIST= 1.rstest 2.rstest 3.rstest err1.rstest \ testsuites/master.nolimittag \ rulesetmultiqueue.sh \ testsuites/rulesetmultiqueue.conf \ + rulesetmultiqueue-v6.sh \ + testsuites/rulesetmultiqueue-v6.conf \ omruleset.sh \ testsuites/omruleset.conf \ omruleset-queue.sh \ @@ -478,6 +574,8 @@ EXTRA_DIST= 1.rstest 2.rstest 3.rstest err1.rstest \ testsuites/execonlywhenprevsuspended3.conf \ execonlywhenprevsuspended4.sh \ testsuites/execonlywhenprevsuspended4.conf \ + execonlywhenprevsuspended_multiwrkr.sh \ + testsuites/execonlywhenprevsuspended_multiwrkr.conf \ tabescape_dflt.sh \ testsuites/tabescape_dflt.conf \ testsuites/1.tabescape_dflt \ @@ -511,6 +609,12 @@ EXTRA_DIST= 1.rstest 2.rstest 3.rstest err1.rstest \ mysql-asyn.sh \ mysql-asyn-vg.sh \ testsuites/mysql-asyn.conf \ + mysql-actq-mt.sh \ + mysql-actq-mt-withpause.sh \ + mysql-actq-mt-withpause-vg.sh \ + testsuites/mysql-actq-mt.conf \ + mmpstrucdata.sh \ + testsuites/mmpstrucdata.conf \ cfg.sh # TODO: re-enable diff --git a/tests/Makefile.in b/tests/Makefile.in index c8c959e..6524d86 100644 --- a/tests/Makefile.in +++ b/tests/Makefile.in @@ -48,10 +48,21 @@ host_triplet = @host@ @ENABLE_TESTBENCH_TRUE@ $(am__append_7) $(am__append_8) \ @ENABLE_TESTBENCH_TRUE@ $(am__append_9) $(am__append_10) \ @ENABLE_TESTBENCH_TRUE@ $(am__append_11) $(am__append_12) \ -@ENABLE_TESTBENCH_TRUE@ $(am__append_13) +@ENABLE_TESTBENCH_TRUE@ $(am__append_13) $(am__append_14) \ +@ENABLE_TESTBENCH_TRUE@ $(am__append_15) $(am__append_16) \ +@ENABLE_TESTBENCH_TRUE@ $(am__append_17) #TESTS = $(TESTRUNS) cfg.sh @ENABLE_IMDIAG_TRUE@@ENABLE_TESTBENCH_TRUE@am__append_1 = \ +@ENABLE_IMDIAG_TRUE@@ENABLE_TESTBENCH_TRUE@ stop-localvar.sh \ +@ENABLE_IMDIAG_TRUE@@ENABLE_TESTBENCH_TRUE@ stop-msgvar.sh \ +@ENABLE_IMDIAG_TRUE@@ENABLE_TESTBENCH_TRUE@ fac_authpriv.sh \ +@ENABLE_IMDIAG_TRUE@@ENABLE_TESTBENCH_TRUE@ fac_local0.sh \ +@ENABLE_IMDIAG_TRUE@@ENABLE_TESTBENCH_TRUE@ fac_mail.sh \ +@ENABLE_IMDIAG_TRUE@@ENABLE_TESTBENCH_TRUE@ fac_news.sh \ +@ENABLE_IMDIAG_TRUE@@ENABLE_TESTBENCH_TRUE@ fac_uucp.sh \ +@ENABLE_IMDIAG_TRUE@@ENABLE_TESTBENCH_TRUE@ rfc5424parser.sh \ @ENABLE_IMDIAG_TRUE@@ENABLE_TESTBENCH_TRUE@ arrayqueue.sh \ +@ENABLE_IMDIAG_TRUE@@ENABLE_TESTBENCH_TRUE@ global_vars.sh \ @ENABLE_IMDIAG_TRUE@@ENABLE_TESTBENCH_TRUE@ da-mainmsg-q.sh \ @ENABLE_IMDIAG_TRUE@@ENABLE_TESTBENCH_TRUE@ validation-run.sh \ @ENABLE_IMDIAG_TRUE@@ENABLE_TESTBENCH_TRUE@ imtcp-multiport.sh \ @@ -59,6 +70,7 @@ host_triplet = @host@ @ENABLE_IMDIAG_TRUE@@ENABLE_TESTBENCH_TRUE@ diskqueue.sh \ @ENABLE_IMDIAG_TRUE@@ENABLE_TESTBENCH_TRUE@ diskqueue-fsync.sh \ @ENABLE_IMDIAG_TRUE@@ENABLE_TESTBENCH_TRUE@ rulesetmultiqueue.sh \ +@ENABLE_IMDIAG_TRUE@@ENABLE_TESTBENCH_TRUE@ rulesetmultiqueue-v6.sh \ @ENABLE_IMDIAG_TRUE@@ENABLE_TESTBENCH_TRUE@ manytcp.sh \ @ENABLE_IMDIAG_TRUE@@ENABLE_TESTBENCH_TRUE@ rsf_getenv.sh \ @ENABLE_IMDIAG_TRUE@@ENABLE_TESTBENCH_TRUE@ imtcp_conndrop.sh \ @@ -90,6 +102,7 @@ host_triplet = @host@ @ENABLE_IMDIAG_TRUE@@ENABLE_TESTBENCH_TRUE@ execonlywhenprevsuspended2.sh \ @ENABLE_IMDIAG_TRUE@@ENABLE_TESTBENCH_TRUE@ execonlywhenprevsuspended3.sh \ @ENABLE_IMDIAG_TRUE@@ENABLE_TESTBENCH_TRUE@ execonlywhenprevsuspended4.sh \ +@ENABLE_IMDIAG_TRUE@@ENABLE_TESTBENCH_TRUE@ execonlywhenprevsuspended_multiwrkr.sh \ @ENABLE_IMDIAG_TRUE@@ENABLE_TESTBENCH_TRUE@ pipe_noreader.sh \ @ENABLE_IMDIAG_TRUE@@ENABLE_TESTBENCH_TRUE@ dircreate_dflt.sh \ @ENABLE_IMDIAG_TRUE@@ENABLE_TESTBENCH_TRUE@ dircreate_off.sh \ @@ -98,7 +111,33 @@ host_triplet = @host@ @ENABLE_IMDIAG_TRUE@@ENABLE_TESTBENCH_TRUE@ imuxsock_ccmiddle_root.sh \ @ENABLE_IMDIAG_TRUE@@ENABLE_TESTBENCH_TRUE@ udp-msgreduc-vg.sh \ @ENABLE_IMDIAG_TRUE@@ENABLE_TESTBENCH_TRUE@ udp-msgreduc-orgmsg-vg.sh \ -@ENABLE_IMDIAG_TRUE@@ENABLE_TESTBENCH_TRUE@ queue-persist.sh +@ENABLE_IMDIAG_TRUE@@ENABLE_TESTBENCH_TRUE@ queue-persist.sh \ +@ENABLE_IMDIAG_TRUE@@ENABLE_TESTBENCH_TRUE@ discard-rptdmsg.sh \ +@ENABLE_IMDIAG_TRUE@@ENABLE_TESTBENCH_TRUE@ discard-allmark.sh \ +@ENABLE_IMDIAG_TRUE@@ENABLE_TESTBENCH_TRUE@ discard.sh \ +@ENABLE_IMDIAG_TRUE@@ENABLE_TESTBENCH_TRUE@ stop.sh \ +@ENABLE_IMDIAG_TRUE@@ENABLE_TESTBENCH_TRUE@ failover-async.sh \ +@ENABLE_IMDIAG_TRUE@@ENABLE_TESTBENCH_TRUE@ failover-double.sh \ +@ENABLE_IMDIAG_TRUE@@ENABLE_TESTBENCH_TRUE@ failover-basic.sh \ +@ENABLE_IMDIAG_TRUE@@ENABLE_TESTBENCH_TRUE@ failover-rptd.sh \ +@ENABLE_IMDIAG_TRUE@@ENABLE_TESTBENCH_TRUE@ failover-no-rptd.sh \ +@ENABLE_IMDIAG_TRUE@@ENABLE_TESTBENCH_TRUE@ failover-no-basic.sh \ +@ENABLE_IMDIAG_TRUE@@ENABLE_TESTBENCH_TRUE@ rcvr_fail_restore.sh \ +@ENABLE_IMDIAG_TRUE@@ENABLE_TESTBENCH_TRUE@ rscript_contains.sh \ +@ENABLE_IMDIAG_TRUE@@ENABLE_TESTBENCH_TRUE@ rscript_field.sh \ +@ENABLE_IMDIAG_TRUE@@ENABLE_TESTBENCH_TRUE@ rscript_stop.sh \ +@ENABLE_IMDIAG_TRUE@@ENABLE_TESTBENCH_TRUE@ rscript_stop2.sh \ +@ENABLE_IMDIAG_TRUE@@ENABLE_TESTBENCH_TRUE@ rscript_prifilt.sh \ +@ENABLE_IMDIAG_TRUE@@ENABLE_TESTBENCH_TRUE@ rscript_optimizer1.sh \ +@ENABLE_IMDIAG_TRUE@@ENABLE_TESTBENCH_TRUE@ rscript_ruleset_call.sh \ +@ENABLE_IMDIAG_TRUE@@ENABLE_TESTBENCH_TRUE@ rs_optimizer_pri.sh \ +@ENABLE_IMDIAG_TRUE@@ENABLE_TESTBENCH_TRUE@ cee_simple.sh \ +@ENABLE_IMDIAG_TRUE@@ENABLE_TESTBENCH_TRUE@ cee_diskqueue.sh \ +@ENABLE_IMDIAG_TRUE@@ENABLE_TESTBENCH_TRUE@ incltest.sh \ +@ENABLE_IMDIAG_TRUE@@ENABLE_TESTBENCH_TRUE@ incltest_dir.sh \ +@ENABLE_IMDIAG_TRUE@@ENABLE_TESTBENCH_TRUE@ incltest_dir_wildcard.sh \ +@ENABLE_IMDIAG_TRUE@@ENABLE_TESTBENCH_TRUE@ incltest_dir_empty_wildcard.sh \ +@ENABLE_IMDIAG_TRUE@@ENABLE_TESTBENCH_TRUE@ linkedlistqueue.sh @ENABLE_IMDIAG_TRUE@@ENABLE_TESTBENCH_TRUE@@HAVE_VALGRIND_TRUE@am__append_2 = \ @ENABLE_IMDIAG_TRUE@@ENABLE_TESTBENCH_TRUE@@HAVE_VALGRIND_TRUE@ discard-rptdmsg-vg.sh \ @@ -112,7 +151,9 @@ host_triplet = @host@ @ENABLE_MYSQL_TESTS_TRUE@@ENABLE_TESTBENCH_TRUE@am__append_3 = \ @ENABLE_MYSQL_TESTS_TRUE@@ENABLE_TESTBENCH_TRUE@ mysql-basic.sh \ @ENABLE_MYSQL_TESTS_TRUE@@ENABLE_TESTBENCH_TRUE@ mysql-basic-cnf6.sh \ -@ENABLE_MYSQL_TESTS_TRUE@@ENABLE_TESTBENCH_TRUE@ mysql-asyn.sh +@ENABLE_MYSQL_TESTS_TRUE@@ENABLE_TESTBENCH_TRUE@ mysql-asyn.sh \ +@ENABLE_MYSQL_TESTS_TRUE@@ENABLE_TESTBENCH_TRUE@ mysql-actq-mt.sh \ +@ENABLE_MYSQL_TESTS_TRUE@@ENABLE_TESTBENCH_TRUE@ mysql-actq-mt-withpause.sh @ENABLE_MYSQL_TESTS_TRUE@@ENABLE_OMLIBDBI_TRUE@@ENABLE_TESTBENCH_TRUE@am__append_4 = \ @ENABLE_MYSQL_TESTS_TRUE@@ENABLE_OMLIBDBI_TRUE@@ENABLE_TESTBENCH_TRUE@ libdbi-basic.sh \ @@ -120,7 +161,8 @@ host_triplet = @host@ @ENABLE_MYSQL_TESTS_TRUE@@ENABLE_TESTBENCH_TRUE@@HAVE_VALGRIND_TRUE@am__append_5 = \ @ENABLE_MYSQL_TESTS_TRUE@@ENABLE_TESTBENCH_TRUE@@HAVE_VALGRIND_TRUE@ mysql-basic-vg.sh \ -@ENABLE_MYSQL_TESTS_TRUE@@ENABLE_TESTBENCH_TRUE@@HAVE_VALGRIND_TRUE@ mysql-asyn-vg.sh +@ENABLE_MYSQL_TESTS_TRUE@@ENABLE_TESTBENCH_TRUE@@HAVE_VALGRIND_TRUE@ mysql-asyn-vg.sh \ +@ENABLE_MYSQL_TESTS_TRUE@@ENABLE_TESTBENCH_TRUE@@HAVE_VALGRIND_TRUE@ mysql-actq-mt-withpause-vg.sh @ENABLE_IMPTCP_TRUE@@ENABLE_TESTBENCH_TRUE@am__append_6 = \ @ENABLE_IMPTCP_TRUE@@ENABLE_TESTBENCH_TRUE@ manyptcp.sh \ @@ -128,20 +170,36 @@ host_triplet = @host@ @ENABLE_IMPTCP_TRUE@@ENABLE_TESTBENCH_TRUE@ imptcp_addtlframedelim.sh \ @ENABLE_IMPTCP_TRUE@@ENABLE_TESTBENCH_TRUE@ imptcp_conndrop.sh +@ENABLE_ELASTICSEARCH_TRUE@@ENABLE_TESTBENCH_TRUE@am__append_7 = \ +@ENABLE_ELASTICSEARCH_TRUE@@ENABLE_TESTBENCH_TRUE@ elasticsearch-basic.sh \ +@ENABLE_ELASTICSEARCH_TRUE@@ENABLE_TESTBENCH_TRUE@ elasticsearch-basic-bulk.sh \ +@ENABLE_ELASTICSEARCH_TRUE@@ENABLE_TESTBENCH_TRUE@ elasticsearch-basic-errorfile-empty.sh \ +@ENABLE_ELASTICSEARCH_TRUE@@ENABLE_TESTBENCH_TRUE@ elasticsearch-basic-errorfile-populated.sh \ +@ENABLE_ELASTICSEARCH_TRUE@@ENABLE_TESTBENCH_TRUE@ elasticsearch-bulk-errorfile-empty.sh \ +@ENABLE_ELASTICSEARCH_TRUE@@ENABLE_TESTBENCH_TRUE@ elasticsearch-bulk-errorfile-populated.sh + +@ENABLE_MMPSTRUCDATA_TRUE@@ENABLE_TESTBENCH_TRUE@am__append_8 = \ +@ENABLE_MMPSTRUCDATA_TRUE@@ENABLE_TESTBENCH_TRUE@ mmpstrucdata.sh + +@ENABLE_MMJSONPARSE_TRUE@@ENABLE_TESTBENCH_TRUE@am__append_9 = \ +@ENABLE_MMJSONPARSE_TRUE@@ENABLE_TESTBENCH_TRUE@ mmjsonparse_simple.sh \ +@ENABLE_MMJSONPARSE_TRUE@@ENABLE_TESTBENCH_TRUE@ mmjsonparse_cim.sh + # TODO: re-enable in newer version #TESTS += \ # #sndrcv_tls_anon.sh \ # #sndrcv_tls_anon_rebind.sh \ # #imtcp-tls-basic.sh -@ENABLE_GNUTLS_TRUE@@ENABLE_TESTBENCH_TRUE@@HAVE_VALGRIND_TRUE@am__append_7 = imtcp-tls-basic-vg.sh \ +@ENABLE_GNUTLS_TRUE@@ENABLE_TESTBENCH_TRUE@@HAVE_VALGRIND_TRUE@am__append_10 = imtcp-tls-basic-vg.sh \ @ENABLE_GNUTLS_TRUE@@ENABLE_TESTBENCH_TRUE@@HAVE_VALGRIND_TRUE@ imtcp_conndrop_tls-vg.sh -@ENABLE_OMUXSOCK_TRUE@@ENABLE_TESTBENCH_TRUE@am__append_8 = uxsock_simple.sh -@ENABLE_OMUDPSPOOF_TRUE@@ENABLE_TESTBENCH_TRUE@am__append_9 = sndrcv_omudpspoof.sh \ +@ENABLE_OMUXSOCK_TRUE@@ENABLE_TESTBENCH_TRUE@am__append_11 = uxsock_simple.sh +@ENABLE_RELP_TRUE@@ENABLE_TESTBENCH_TRUE@am__append_12 = sndrcv_relp.sh +@ENABLE_OMUDPSPOOF_TRUE@@ENABLE_TESTBENCH_TRUE@am__append_13 = sndrcv_omudpspoof.sh \ @ENABLE_OMUDPSPOOF_TRUE@@ENABLE_TESTBENCH_TRUE@ sndrcv_omudpspoof_nonstdpt.sh -@ENABLE_OMSTDOUT_TRUE@@ENABLE_TESTBENCH_TRUE@am__append_10 = omod-if-array.sh \ +@ENABLE_OMSTDOUT_TRUE@@ENABLE_TESTBENCH_TRUE@am__append_14 = omod-if-array.sh \ @ENABLE_OMSTDOUT_TRUE@@ENABLE_TESTBENCH_TRUE@ proprepltest.sh \ @ENABLE_OMSTDOUT_TRUE@@ENABLE_TESTBENCH_TRUE@ parsertest.sh \ @ENABLE_OMSTDOUT_TRUE@@ENABLE_TESTBENCH_TRUE@ timestamp.sh \ @@ -153,16 +211,16 @@ host_triplet = @host@ @ENABLE_OMSTDOUT_TRUE@@ENABLE_TESTBENCH_TRUE@ tabescape_off.sh \ @ENABLE_OMSTDOUT_TRUE@@ENABLE_TESTBENCH_TRUE@ fieldtest.sh -@ENABLE_IMDIAG_TRUE@@ENABLE_OMRULESET_TRUE@@ENABLE_TESTBENCH_TRUE@am__append_11 = omruleset.sh \ +@ENABLE_IMDIAG_TRUE@@ENABLE_OMRULESET_TRUE@@ENABLE_TESTBENCH_TRUE@am__append_15 = omruleset.sh \ @ENABLE_IMDIAG_TRUE@@ENABLE_OMRULESET_TRUE@@ENABLE_TESTBENCH_TRUE@ omruleset-queue.sh # random.sh is temporarily disabled as it needs some work # to rsyslog core to complete in reasonable time #TESTS += random.sh -@ENABLE_IMFILE_TRUE@@ENABLE_TESTBENCH_TRUE@am__append_12 = imfile-basic.sh -@ENABLE_IMFILE_TRUE@@ENABLE_TESTBENCH_TRUE@@HAVE_VALGRIND_TRUE@am__append_13 = imfile-basic-vg.sh -@ENABLE_GNUTLS_TRUE@am__append_14 = -lgcrypt +@ENABLE_IMFILE_TRUE@@ENABLE_TESTBENCH_TRUE@am__append_16 = imfile-basic.sh +@ENABLE_IMFILE_TRUE@@ENABLE_TESTBENCH_TRUE@@HAVE_VALGRIND_TRUE@am__append_17 = imfile-basic-vg.sh +@ENABLE_GNUTLS_TRUE@am__append_18 = -lgcrypt subdir = tests DIST_COMMON = README $(srcdir)/Makefile.am $(srcdir)/Makefile.in ACLOCAL_M4 = $(top_srcdir)/aclocal.m4 @@ -297,7 +355,6 @@ GREP = @GREP@ GSS_LIBS = @GSS_LIBS@ GUARDTIME_CFLAGS = @GUARDTIME_CFLAGS@ GUARDTIME_LIBS = @GUARDTIME_LIBS@ -HAVE_LIBGCRYPT_CONFIG = @HAVE_LIBGCRYPT_CONFIG@ HAVE_MYSQL_CONFIG = @HAVE_MYSQL_CONFIG@ HAVE_ORACLE_CONFIG = @HAVE_ORACLE_CONFIG@ HAVE_PGSQL_CONFIG = @HAVE_PGSQL_CONFIG@ @@ -318,14 +375,15 @@ LEXLIB = @LEXLIB@ LEX_OUTPUT_ROOT = @LEX_OUTPUT_ROOT@ LIBDBI_CFLAGS = @LIBDBI_CFLAGS@ LIBDBI_LIBS = @LIBDBI_LIBS@ -LIBEE_CFLAGS = @LIBEE_CFLAGS@ -LIBEE_LIBS = @LIBEE_LIBS@ LIBESTR_CFLAGS = @LIBESTR_CFLAGS@ LIBESTR_LIBS = @LIBESTR_LIBS@ LIBGCRYPT_CFLAGS = @LIBGCRYPT_CFLAGS@ +LIBGCRYPT_CONFIG = @LIBGCRYPT_CONFIG@ LIBGCRYPT_LIBS = @LIBGCRYPT_LIBS@ LIBLOGGING_CFLAGS = @LIBLOGGING_CFLAGS@ LIBLOGGING_LIBS = @LIBLOGGING_LIBS@ +LIBLOGGING_STDLOG_CFLAGS = @LIBLOGGING_STDLOG_CFLAGS@ +LIBLOGGING_STDLOG_LIBS = @LIBLOGGING_STDLOG_LIBS@ LIBLOGNORM_CFLAGS = @LIBLOGNORM_CFLAGS@ LIBLOGNORM_LIBS = @LIBLOGNORM_LIBS@ LIBM = @LIBM@ @@ -350,6 +408,8 @@ NM = @NM@ NMEDIT = @NMEDIT@ OBJDUMP = @OBJDUMP@ OBJEXT = @OBJEXT@ +OPENSSL_CFLAGS = @OPENSSL_CFLAGS@ +OPENSSL_LIBS = @OPENSSL_LIBS@ ORACLE_CFLAGS = @ORACLE_CFLAGS@ ORACLE_LIBS = @ORACLE_LIBS@ OTOOL = @OTOOL@ @@ -561,6 +621,40 @@ EXTRA_DIST = 1.rstest 2.rstest 3.rstest err1.rstest \ testsuites/rscript_stop.conf \ rscript_stop2.sh \ testsuites/rscript_stop2.conf \ + stop.sh \ + testsuites/stop.conf \ + rscript_le.sh \ + testsuites/rscript_le.conf \ + rscript_ge.sh \ + testsuites/rscript_ge.conf \ + rscript_lt.sh \ + testsuites/rscript_lt.conf \ + rscript_gt.sh \ + testsuites/rscript_gt.conf \ + rscript_ne.sh \ + testsuites/rscript_ne.conf \ + rscript_eq.sh \ + testsuites/rscript_eq.conf \ + stop-localvar.sh \ + testsuites/stop-localvar.conf \ + stop-msgvar.sh \ + testsuites/stop-msgvar.conf \ + global_vars.sh \ + testsuites/global_vars.conf \ + rfc5424parser.sh \ + testsuites/rfc5424parser.conf \ + fac_authpriv.sh \ + testsuites/fac_authpriv.conf \ + fac_local0.sh \ + testsuites/fac_local0.conf \ + fac_mail.sh \ + testsuites/fac_mail.conf \ + fac_news.sh \ + testsuites/fac_news.conf \ + fac_uucp.sh \ + testsuites/fac_uucp.conf \ + rs_optimizer_pri.sh \ + testsuites/rs_optimizer_pri.conf \ rscript_prifilt.sh \ testsuites/rscript_prifilt.conf \ rscript_optimizer1.sh \ @@ -571,6 +665,10 @@ EXTRA_DIST = 1.rstest 2.rstest 3.rstest err1.rstest \ testsuites/cee_simple.conf \ cee_diskqueue.sh \ testsuites/cee_diskqueue.conf \ + mmjsonparse_simple.sh \ + testsuites/mmjsonparse_simple.conf \ + mmjsonparse_cim.sh \ + testsuites/mmjsonparse_cim.conf \ incltest.sh \ testsuites/incltest.conf \ incltest_dir.sh \ @@ -580,6 +678,18 @@ EXTRA_DIST = 1.rstest 2.rstest 3.rstest err1.rstest \ incltest_dir_wildcard.sh \ testsuites/incltest_dir_wildcard.conf \ testsuites/incltest.d/include.conf \ + elasticsearch-basic.sh \ + testsuites/elasticsearch-basic.conf \ + elasticsearch-basic-bulk.sh \ + testsuites/elasticsearch-basic-bulk.conf \ + elasticsearch-basic-errorfile-empty.sh \ + testsuites/elasticsearch-basic-errorfile-empty.conf \ + elasticsearch-basic-errorfile-populated.sh \ + testsuites/elasticsearch-basic-errorfile-populated.conf \ + elasticsearch-bulk-errorfile-empty.sh \ + testsuites/elasticsearch-bulk-errorfile-empty.conf \ + elasticsearch-bulk-errorfile-populated.sh \ + testsuites/elasticsearch-bulk-errorfile-populated.conf \ linkedlistqueue.sh \ testsuites/linkedlistqueue.conf \ da-mainmsg-q.sh \ @@ -669,6 +779,9 @@ EXTRA_DIST = 1.rstest 2.rstest 3.rstest err1.rstest \ sndrcv.sh \ testsuites/sndrcv_sender.conf \ testsuites/sndrcv_rcvr.conf \ + sndrcv_relp.sh \ + testsuites/sndrcv_relp_sender.conf \ + testsuites/sndrcv_relp_rcvr.conf \ sndrcv_udp.sh \ testsuites/sndrcv_udp_sender.conf \ testsuites/sndrcv_udp_rcvr.conf \ @@ -732,6 +845,8 @@ EXTRA_DIST = 1.rstest 2.rstest 3.rstest err1.rstest \ testsuites/master.nolimittag \ rulesetmultiqueue.sh \ testsuites/rulesetmultiqueue.conf \ + rulesetmultiqueue-v6.sh \ + testsuites/rulesetmultiqueue-v6.conf \ omruleset.sh \ testsuites/omruleset.conf \ omruleset-queue.sh \ @@ -750,6 +865,8 @@ EXTRA_DIST = 1.rstest 2.rstest 3.rstest err1.rstest \ testsuites/execonlywhenprevsuspended3.conf \ execonlywhenprevsuspended4.sh \ testsuites/execonlywhenprevsuspended4.conf \ + execonlywhenprevsuspended_multiwrkr.sh \ + testsuites/execonlywhenprevsuspended_multiwrkr.conf \ tabescape_dflt.sh \ testsuites/tabescape_dflt.conf \ testsuites/1.tabescape_dflt \ @@ -783,6 +900,12 @@ EXTRA_DIST = 1.rstest 2.rstest 3.rstest err1.rstest \ mysql-asyn.sh \ mysql-asyn-vg.sh \ testsuites/mysql-asyn.conf \ + mysql-actq-mt.sh \ + mysql-actq-mt-withpause.sh \ + mysql-actq-mt-withpause-vg.sh \ + testsuites/mysql-actq-mt.conf \ + mmpstrucdata.sh \ + testsuites/mmpstrucdata.conf \ cfg.sh @@ -802,7 +925,7 @@ uxsockrcvr_LDADD = $(SOL_LIBS) tcpflood_SOURCES = tcpflood.c tcpflood_CPPFLAGS = $(PTHREADS_CFLAGS) $(GNUTLS_CFLAGS) tcpflood_LDADD = $(SOL_LIBS) $(PTHREADS_LIBS) $(GNUTLS_LIBS) \ - $(am__append_14) + $(am__append_18) minitcpsrv_SOURCES = minitcpsrvr.c minitcpsrv_LDADD = $(SOL_LIBS) syslog_caller_SOURCES = syslog_caller.c @@ -918,22 +1041,25 @@ distclean-compile: @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/uxsockrcvr.Po@am__quote@ .c.o: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(COMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ $< -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Po +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.o$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c $< +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ $< .c.obj: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(COMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ `$(CYGPATH_W) '$<'` -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Po +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.obj$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ `$(CYGPATH_W) '$<'` &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c `$(CYGPATH_W) '$<'` +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ `$(CYGPATH_W) '$<'` .c.lo: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(LTCOMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ $< -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Plo +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.lo$$||'`;\ +@am__fastdepCC_TRUE@ $(LTCOMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Plo @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=yes @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ @am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(LTCOMPILE) -c -o $@ $< @@ -1257,30 +1383,6 @@ uninstall-am: mostlyclean-generic mostlyclean-libtool pdf pdf-am ps ps-am \ tags uninstall uninstall-am -@ENABLE_IMDIAG_TRUE@@ENABLE_TESTBENCH_TRUE@ discard-rptdmsg.sh \ -@ENABLE_IMDIAG_TRUE@@ENABLE_TESTBENCH_TRUE@ discard-allmark.sh \ -@ENABLE_IMDIAG_TRUE@@ENABLE_TESTBENCH_TRUE@ discard.sh \ -@ENABLE_IMDIAG_TRUE@@ENABLE_TESTBENCH_TRUE@ failover-async.sh \ -@ENABLE_IMDIAG_TRUE@@ENABLE_TESTBENCH_TRUE@ failover-double.sh \ -@ENABLE_IMDIAG_TRUE@@ENABLE_TESTBENCH_TRUE@ failover-basic.sh \ -@ENABLE_IMDIAG_TRUE@@ENABLE_TESTBENCH_TRUE@ failover-rptd.sh \ -@ENABLE_IMDIAG_TRUE@@ENABLE_TESTBENCH_TRUE@ failover-no-rptd.sh \ -@ENABLE_IMDIAG_TRUE@@ENABLE_TESTBENCH_TRUE@ failover-no-basic.sh \ -@ENABLE_IMDIAG_TRUE@@ENABLE_TESTBENCH_TRUE@ rcvr_fail_restore.sh \ -@ENABLE_IMDIAG_TRUE@@ENABLE_TESTBENCH_TRUE@ rscript_contains.sh \ -@ENABLE_IMDIAG_TRUE@@ENABLE_TESTBENCH_TRUE@ rscript_field.sh \ -@ENABLE_IMDIAG_TRUE@@ENABLE_TESTBENCH_TRUE@ rscript_stop.sh \ -@ENABLE_IMDIAG_TRUE@@ENABLE_TESTBENCH_TRUE@ rscript_stop2.sh \ -@ENABLE_IMDIAG_TRUE@@ENABLE_TESTBENCH_TRUE@ rscript_prifilt.sh \ -@ENABLE_IMDIAG_TRUE@@ENABLE_TESTBENCH_TRUE@ rscript_optimizer1.sh \ -@ENABLE_IMDIAG_TRUE@@ENABLE_TESTBENCH_TRUE@ rscript_ruleset_call.sh \ -@ENABLE_IMDIAG_TRUE@@ENABLE_TESTBENCH_TRUE@ cee_simple.sh \ -@ENABLE_IMDIAG_TRUE@@ENABLE_TESTBENCH_TRUE@ cee_diskqueue.sh \ -@ENABLE_IMDIAG_TRUE@@ENABLE_TESTBENCH_TRUE@ incltest.sh \ -@ENABLE_IMDIAG_TRUE@@ENABLE_TESTBENCH_TRUE@ incltest_dir.sh \ -@ENABLE_IMDIAG_TRUE@@ENABLE_TESTBENCH_TRUE@ incltest_dir_wildcard.sh \ -@ENABLE_IMDIAG_TRUE@@ENABLE_TESTBENCH_TRUE@ incltest_dir_empty_wildcard.sh \ -@ENABLE_IMDIAG_TRUE@@ENABLE_TESTBENCH_TRUE@ linkedlistqueue.sh @ENABLE_GNUTLS_TRUE@@ENABLE_TESTBENCH_TRUE@@HAVE_VALGRIND_TRUE@ manytcp-too-few-tls-vg.sh # rtinit tests disabled for the moment - also questionable if they diff --git a/tests/chkseq.c b/tests/chkseq.c index b22c899..596d8dc 100644 --- a/tests/chkseq.c +++ b/tests/chkseq.c @@ -7,10 +7,14 @@ * -s<starting number> -e<ending number> * default for s is 0. -e should be given (else it is also 0) * -d may be specified, in which case duplicate messages are permitted. + * -m number of messages permitted to be missing without triggering a + * failure. This is necessary for some failover tests, where it is + * impossible to totally guard against messagt loss. By default, NO + * message is permitted to be lost. * * Part of the testbench for rsyslog. * - * Copyright 2009 Rainer Gerhards and Adiscon GmbH. + * Copyright 2009-2014 Rainer Gerhards and Adiscon GmbH. * * This file is part of rsyslog. * @@ -47,12 +51,15 @@ int main(int argc, char *argv[]) int dupsPermitted = 0; int start = 0, end = 0; int opt; + int lostok = 0; /* how many messages are OK to be lost? */ int nDups = 0; + int reachedEOF; int edLen; /* length of extra data */ static char edBuf[500*1024]; /* buffer for extra data (pretty large to be on the save side...) */ + static char ioBuf[sizeof(edBuf)+1024]; char *file = NULL; - while((opt = getopt(argc, argv, "e:f:ds:vE")) != EOF) { + while((opt = getopt(argc, argv, "e:f:ds:vm:E")) != EOF) { switch((char)opt) { case 'f': file = optarg; @@ -69,6 +76,9 @@ int main(int argc, char *argv[]) case 'v': ++verbose; break; + case 'm': + lostok = atoi(optarg); + break; case 'E': bHaveExtraData = 1; break; @@ -102,19 +112,32 @@ int main(int argc, char *argv[]) for(i = start ; i < end+1 ; ++i) { if(bHaveExtraData) { - scanfOK = fscanf(fp, "%d,%d,%s\n", &val, &edLen, edBuf) == 3 ? 1 : 0; + if(fgets(ioBuf, sizeof(ioBuf), fp) == NULL) { + scanfOK = 0; + } else { + scanfOK = sscanf(ioBuf, "%d,%d,%s\n", &val, &edLen, edBuf) == 3 ? 1 : 0; + } if(edLen != (int) strlen(edBuf)) { printf("extra data length specified %d, but actually is %ld in record %d\n", edLen, (long) strlen(edBuf), i); exit(1); } } else { - scanfOK = fscanf(fp, "%d\n", &val) == 1 ? 1 : 0; + if(fgets(ioBuf, sizeof(ioBuf), fp) == NULL) { + scanfOK = 0; + } else { + scanfOK = sscanf(ioBuf, "%d\n", &val) == 1 ? 1 : 0; + } } if(!scanfOK) { printf("scanf error in index i=%d\n", i); exit(1); } + while(val > i && lostok > 0) { + --lostok; + printf("message %d missing (ok due to -m [now %d])\n", i, lostok); + ++i; + } if(val != i) { if(val == i - 1 && dupsPermitted) { --i; @@ -126,15 +149,57 @@ int main(int argc, char *argv[]) } } - if(nDups != 0) - printf("info: had %d duplicates (this is no error)\n", nDups); - if(i - 1 != end) { printf("only %d records in file, expected %d\n", i - 1, end); exit(1); } - if(!feof(fp)) { + int c = getc(fp); + if(c == EOF) { + reachedEOF = 1; + } else { + ungetc(c, fp); + /* if duplicates are permitted, we need to do a final check if we have duplicates at the + * end of file. + */ + if(dupsPermitted) { + i = end; + while(!feof(fp)) { + if(bHaveExtraData) { + if(fgets(ioBuf, sizeof(ioBuf), fp) == NULL) { + scanfOK = 0; + } else { + scanfOK = sscanf(ioBuf, "%d,%d,%s\n", &val, &edLen, edBuf) == 3 ? 1 : 0; + } + if(edLen != (int) strlen(edBuf)) { + printf("extra data length specified %d, but actually is %ld in record %d\n", + edLen, (long) strlen(edBuf), i); + exit(1); + } + } else { + if(fgets(ioBuf, sizeof(ioBuf), fp) == NULL) { + scanfOK = 0; + } else { + scanfOK = sscanf(ioBuf, "%d\n", &val) == 1 ? 1 : 0; + } + } + + if(val != i) { + reachedEOF = 0; + goto breakIF; + } + } + reachedEOF = feof(fp) ? 1 : 0; + } else { + reachedEOF = 0; + } + } + +breakIF: + if(nDups != 0) + printf("info: had %d duplicates (this is no error)\n", nDups); + + if(!reachedEOF) { printf("end of processing, but NOT end of file!\n"); exit(1); } diff --git a/tests/daqueue-persist.sh b/tests/daqueue-persist.sh index feb2a34..0781a7d 100755 --- a/tests/daqueue-persist.sh +++ b/tests/daqueue-persist.sh @@ -2,6 +2,7 @@ # to carry out multiple tests with different queue modes # added 2009-05-27 by Rgerhards # This file is part of the rsyslog project, released under GPLv3 +echo =============================================================================== echo \[daqueue-persist.sh\]: test data persisting at shutdown source $srcdir/daqueue-persist-drvr.sh LinkedList source $srcdir/daqueue-persist-drvr.sh FixedArray diff --git a/tests/diag.sh b/tests/diag.sh index 2fdcbfb..fd294ce 100755 --- a/tests/diag.sh +++ b/tests/diag.sh @@ -22,6 +22,7 @@ case $1 in rm -rf test-spool test-logdir stat-file1 rm -f rsyslog.out.*.log work-presort rsyslog.pipe rm -f rsyslog.input rsyslog.empty + rm -f rsyslog.errorfile rm -f core.* vgcore.* # Note: rsyslog.action.*.include must NOT be deleted, as it # is used to setup some parameters BEFORE calling init. This @@ -35,8 +36,20 @@ case $1 in rm -rf test-spool test-logdir stat-file1 rm -f rsyslog.out.*.log rsyslog.random.data work-presort rsyslog.pipe rm -f rsyslog.input rsyslog.conf.tlscert stat-file1 rsyslog.empty + rm -f rsyslog.errorfile echo ------------------------------------------------------------------------------- ;; + 'es-init') # initialize local Elasticsearch *testbench* instance for the next + # test. NOTE: do NOT put anything useful on that instance! + curl -XDELETE localhost:9200/rsyslog_testbench + ;; + 'es-getdata') # read data from ES to a local file so that we can process + # it with out regular tooling. + # Note: param 2 MUST be number of records to read (ES does + # not return the full set unless you tell it explicitely). + curl localhost:9200/rsyslog_testbench/_search?size=$2 > work + python $srcdir/es_response_get_msgnum.py > rsyslog.out.log + ;; 'startup') # start rsyslogd with default params. $2 is the config file name to use # returns only after successful startup, $3 is the instance (blank or 2!) $valgrind ../tools/rsyslogd -u2 -n -irsyslog$3.pid -M../runtime/.libs:../.libs -f$srcdir/testsuites/$2 & @@ -109,6 +122,7 @@ case $1 in echo Shutting down instance 2 fi $srcdir/diag.sh wait-queueempty $2 + ./msleep 100 # wait 100 milliseconds kill `cat rsyslog$2.pid` # note: we do not wait for the actual termination! ;; @@ -141,7 +155,7 @@ case $1 in 'seq-check') # do the usual sequence check to see if everything was properly received. $2 is the instance. rm -f work cp rsyslog.out.log work-presort - sort < rsyslog.out.log > work + sort -g < rsyslog.out.log > work # $4... are just to have the abilit to pass in more options... # add -v to chkseq if you need more verbose output ./chkseq -fwork -s$2 -e$3 $4 $5 $6 $7 @@ -154,7 +168,7 @@ case $1 in # a duplicateof seq-check, but we could not change its calling conventions without # breaking a lot of exitings test cases, so we preferred to duplicate the code here. rm -f work2 - sort < rsyslog2.out.log > work2 + sort -g < rsyslog2.out.log > work2 # $4... are just to have the abilit to pass in more options... # add -v to chkseq if you need more verbose output ./chkseq -fwork2 -s$2 -e$3 $4 $5 $6 $7 @@ -167,7 +181,7 @@ case $1 in 'gzip-seq-check') # do the usual sequence check, but for gzip files rm -f work ls -l rsyslog.out.log - gunzip < rsyslog.out.log | sort > work + gunzip < rsyslog.out.log | sort -g > work ls -l work # $4... are just to have the abilit to pass in more options... ./chkseq -fwork -v -s$2 -e$3 $4 $5 $6 $7 @@ -190,5 +204,12 @@ case $1 in ZCAT=zcat fi ;; + 'generate-HOSTNAME') # generate the HOSTNAME file + source $srcdir/diag.sh startup gethostname.conf + source $srcdir/diag.sh tcpflood -m1 -M "<128>" + ./msleep 100 + source $srcdir/diag.sh shutdown-when-empty # shut down rsyslogd when done processing messages + source $srcdir/diag.sh wait-shutdown # we need to wait until rsyslogd is finished! + ;; *) echo "invalid argument" $1 esac diff --git a/tests/diskqueue.sh b/tests/diskqueue.sh index b871e9e..853a836 100755 --- a/tests/diskqueue.sh +++ b/tests/diskqueue.sh @@ -5,6 +5,7 @@ # added 2009-04-17 by Rgerhards # This file is part of the rsyslog project, released under GPLv3 # uncomment for debugging support: +echo =============================================================================== echo \[diskqueue.sh\]: testing queue disk-only mode # uncomment for debugging support: #export RSYSLOG_DEBUG="debug nostdout noprintmutexaction" diff --git a/tests/elasticsearch-basic-bulk.sh b/tests/elasticsearch-basic-bulk.sh new file mode 100755 index 0000000..f62db16 --- /dev/null +++ b/tests/elasticsearch-basic-bulk.sh @@ -0,0 +1,12 @@ +# This file is part of the rsyslog project, released under ASL 2.0 +echo =============================================================================== +echo \[elasticsearch-basic-bulk.sh\]: basic test for elasticsearch functionality +source $srcdir/diag.sh init +source $srcdir/diag.sh es-init +source $srcdir/diag.sh startup elasticsearch-basic-bulk.conf +source $srcdir/diag.sh injectmsg 0 10000 +source $srcdir/diag.sh shutdown-when-empty +source $srcdir/diag.sh wait-shutdown +source $srcdir/diag.sh es-getdata 10000 +source $srcdir/diag.sh seq-check 0 9999 +source $srcdir/diag.sh exit diff --git a/tests/elasticsearch-basic-errorfile-empty.sh b/tests/elasticsearch-basic-errorfile-empty.sh new file mode 100755 index 0000000..6dbcde9 --- /dev/null +++ b/tests/elasticsearch-basic-errorfile-empty.sh @@ -0,0 +1,17 @@ +# This file is part of the rsyslog project, released under ASL 2.0 +echo =============================================================================== +echo \[elasticsearch-basic-errorfile-empty\]: basic test for elasticsearch functionality +source $srcdir/diag.sh init +source $srcdir/diag.sh es-init +source $srcdir/diag.sh startup elasticsearch-basic-errorfile-empty.conf +source $srcdir/diag.sh injectmsg 0 10000 +source $srcdir/diag.sh shutdown-when-empty +source $srcdir/diag.sh wait-shutdown +source $srcdir/diag.sh es-getdata 10000 +if [ -f rsyslog.errorfile ] +then + echo "error: error file exists!" + exit 1 +fi +source $srcdir/diag.sh seq-check 0 9999 +source $srcdir/diag.sh exit diff --git a/tests/elasticsearch-basic-errorfile-populated.sh b/tests/elasticsearch-basic-errorfile-populated.sh new file mode 100755 index 0000000..7239a59 --- /dev/null +++ b/tests/elasticsearch-basic-errorfile-populated.sh @@ -0,0 +1,26 @@ +# This file is part of the rsyslog project, released under ASL 2.0 +echo =============================================================================== +echo \[elasticsearch-basic-errorfile-populated\]: basic test for elasticsearch functionality +source $srcdir/diag.sh init +source $srcdir/diag.sh es-init +curl -XPUT localhost:9200/rsyslog_testbench/ -d '{ + "mappings": { + "test-type": { + "properties": { + "msgnum": { + "type": "integer" + } + } + } + } +}' +source $srcdir/diag.sh startup elasticsearch-basic-errorfile-populated.conf +source $srcdir/diag.sh injectmsg 0 1000 +source $srcdir/diag.sh shutdown-when-empty +source $srcdir/diag.sh wait-shutdown +if [ ! -f rsyslog.errorfile ] +then + echo "error: error file does not exist!" + exit 1 +fi +source $srcdir/diag.sh exit diff --git a/tests/elasticsearch-basic.sh b/tests/elasticsearch-basic.sh new file mode 100755 index 0000000..18c847c --- /dev/null +++ b/tests/elasticsearch-basic.sh @@ -0,0 +1,12 @@ +# This file is part of the rsyslog project, released under ASL 2.0 +echo =============================================================================== +echo \[elasticsearch-basic.sh\]: basic test for elasticsearch functionality +source $srcdir/diag.sh init +source $srcdir/diag.sh es-init +source $srcdir/diag.sh startup elasticsearch-basic.conf +source $srcdir/diag.sh injectmsg 0 10000 +source $srcdir/diag.sh shutdown-when-empty +source $srcdir/diag.sh wait-shutdown +source $srcdir/diag.sh es-getdata 10000 +source $srcdir/diag.sh seq-check 0 9999 +source $srcdir/diag.sh exit diff --git a/tests/elasticsearch-bulk-errorfile-empty.sh b/tests/elasticsearch-bulk-errorfile-empty.sh new file mode 100755 index 0000000..a370e72 --- /dev/null +++ b/tests/elasticsearch-bulk-errorfile-empty.sh @@ -0,0 +1,17 @@ +# This file is part of the rsyslog project, released under ASL 2.0 +echo =============================================================================== +echo \[elasticsearch-bulk-errorfile-empty\]: basic test for elasticsearch functionality +source $srcdir/diag.sh init +source $srcdir/diag.sh es-init +source $srcdir/diag.sh startup elasticsearch-bulk-errorfile-empty.conf +source $srcdir/diag.sh injectmsg 0 10000 +source $srcdir/diag.sh shutdown-when-empty +source $srcdir/diag.sh wait-shutdown +source $srcdir/diag.sh es-getdata 10000 +if [ -f rsyslog.errorfile ] +then + echo "error: error file exists!" + exit 1 +fi +source $srcdir/diag.sh seq-check 0 9999 +source $srcdir/diag.sh exit diff --git a/tests/elasticsearch-bulk-errorfile-populated.sh b/tests/elasticsearch-bulk-errorfile-populated.sh new file mode 100755 index 0000000..169fa1c --- /dev/null +++ b/tests/elasticsearch-bulk-errorfile-populated.sh @@ -0,0 +1,26 @@ +# This file is part of the rsyslog project, released under ASL 2.0 +echo =============================================================================== +echo \[elasticsearch-bulk-errorfile-populated\]: basic test for elasticsearch functionality +source $srcdir/diag.sh init +source $srcdir/diag.sh es-init +curl -XPUT localhost:9200/rsyslog_testbench/ -d '{ + "mappings": { + "test-type": { + "properties": { + "msgnum": { + "type": "integer" + } + } + } + } +}' +source $srcdir/diag.sh startup elasticsearch-bulk-errorfile-populated.conf +source $srcdir/diag.sh injectmsg 0 1000 +source $srcdir/diag.sh shutdown-when-empty +source $srcdir/diag.sh wait-shutdown +if [ ! -f rsyslog.errorfile ] +then + echo "error: error file does not exist!" + exit 1 +fi +source $srcdir/diag.sh exit diff --git a/tests/execonlywhenprevsuspended_multiwrkr.sh b/tests/execonlywhenprevsuspended_multiwrkr.sh new file mode 100755 index 0000000..5d960b0 --- /dev/null +++ b/tests/execonlywhenprevsuspended_multiwrkr.sh @@ -0,0 +1,10 @@ +# rgerhards, 2013-12-05 +echo ===================================================================================== +echo \[execonlywhenprevsuspended_multiwrkr.sh\]: test execonly...suspended functionality multiworker case +source $srcdir/diag.sh init +source $srcdir/diag.sh startup execonlywhenprevsuspended_multiwrkr.conf +source $srcdir/diag.sh injectmsg 0 1000 +source $srcdir/diag.sh shutdown-when-empty # shut down rsyslogd when done processing messages +source $srcdir/diag.sh wait-shutdown +source $srcdir/diag.sh seq-check 1 999 +source $srcdir/diag.sh exit diff --git a/tests/fac_authpriv.sh b/tests/fac_authpriv.sh new file mode 100755 index 0000000..f2c8544 --- /dev/null +++ b/tests/fac_authpriv.sh @@ -0,0 +1,11 @@ +# This tests proper processing of the authpriv facility. +# added 2014-09-16 by Rgerhards + +# This file is part of the rsyslog project, released under ASL 2.0 +source $srcdir/diag.sh init +source $srcdir/diag.sh startup fac_authpriv.conf +source $srcdir/diag.sh tcpflood -m1000 -P 81 +source $srcdir/diag.sh shutdown-when-empty # shut down rsyslogd when done processing messages +source $srcdir/diag.sh wait-shutdown # and wait for it to terminate +source $srcdir/diag.sh seq-check 0 999 +source $srcdir/diag.sh exit diff --git a/tests/fac_local0.sh b/tests/fac_local0.sh new file mode 100755 index 0000000..516d3c0 --- /dev/null +++ b/tests/fac_local0.sh @@ -0,0 +1,10 @@ +# added 2014-09-17 by Rgerhards + +# This file is part of the rsyslog project, released under ASL 2.0 +source $srcdir/diag.sh init +source $srcdir/diag.sh startup fac_local0.conf +source $srcdir/diag.sh tcpflood -m1000 -P 129 +source $srcdir/diag.sh shutdown-when-empty # shut down rsyslogd when done processing messages +source $srcdir/diag.sh wait-shutdown # and wait for it to terminate +source $srcdir/diag.sh seq-check 0 999 +source $srcdir/diag.sh exit diff --git a/tests/fac_mail.sh b/tests/fac_mail.sh new file mode 100755 index 0000000..e8873da --- /dev/null +++ b/tests/fac_mail.sh @@ -0,0 +1,10 @@ +# added 2014-09-17 by Rgerhards + +# This file is part of the rsyslog project, released under ASL 2.0 +source $srcdir/diag.sh init +source $srcdir/diag.sh startup fac_mail.conf +source $srcdir/diag.sh tcpflood -m1000 -P 17 +source $srcdir/diag.sh shutdown-when-empty # shut down rsyslogd when done processing messages +source $srcdir/diag.sh wait-shutdown # and wait for it to terminate +source $srcdir/diag.sh seq-check 0 999 +source $srcdir/diag.sh exit diff --git a/tests/fac_news.sh b/tests/fac_news.sh new file mode 100755 index 0000000..85ded82 --- /dev/null +++ b/tests/fac_news.sh @@ -0,0 +1,10 @@ +# added 2014-09-17 by Rgerhards + +# This file is part of the rsyslog project, released under ASL 2.0 +source $srcdir/diag.sh init +source $srcdir/diag.sh startup fac_news.conf +source $srcdir/diag.sh tcpflood -m1000 -P 57 +source $srcdir/diag.sh shutdown-when-empty # shut down rsyslogd when done processing messages +source $srcdir/diag.sh wait-shutdown # and wait for it to terminate +source $srcdir/diag.sh seq-check 0 999 +source $srcdir/diag.sh exit diff --git a/tests/fac_uucp.sh b/tests/fac_uucp.sh new file mode 100755 index 0000000..6c5c06b --- /dev/null +++ b/tests/fac_uucp.sh @@ -0,0 +1,10 @@ +# added 2014-09-17 by Rgerhards + +# This file is part of the rsyslog project, released under ASL 2.0 +source $srcdir/diag.sh init +source $srcdir/diag.sh startup fac_uucp.conf +source $srcdir/diag.sh tcpflood -m1000 -P 65 +source $srcdir/diag.sh shutdown-when-empty # shut down rsyslogd when done processing messages +source $srcdir/diag.sh wait-shutdown # and wait for it to terminate +source $srcdir/diag.sh seq-check 0 999 +source $srcdir/diag.sh exit diff --git a/tests/fieldtest.sh b/tests/fieldtest.sh index 9875fda..10541ef 100755 --- a/tests/fieldtest.sh +++ b/tests/fieldtest.sh @@ -1,5 +1,6 @@ echo \[fieldtest.sh\]: test fieldtest via udp $srcdir/killrsyslog.sh # kill rsyslogd if it runs for some reason +source $srcdir/diag.sh generate-HOSTNAME ./nettester -tfield1 -iudp if [ "$?" -ne "0" ]; then diff --git a/tests/global_vars.sh b/tests/global_vars.sh new file mode 100755 index 0000000..5c6c579 --- /dev/null +++ b/tests/global_vars.sh @@ -0,0 +1,15 @@ +# Test for global variables +# added 2013-11-18 by rgerhards +# This file is part of the rsyslog project, released under ASL 2.0 +echo =============================================================================== +echo \[global_vars.sh\]: testing global variable support +source $srcdir/diag.sh init +source $srcdir/diag.sh startup global_vars.conf + +# 40000 messages should be enough +source $srcdir/diag.sh injectmsg 0 40000 + +source $srcdir/diag.sh shutdown-when-empty # shut down rsyslogd when done processing messages +source $srcdir/diag.sh wait-shutdown +source $srcdir/diag.sh seq-check 0 39999 +source $srcdir/diag.sh exit diff --git a/tests/inputname.sh b/tests/inputname.sh index 71f11c1..f89285b 100755 --- a/tests/inputname.sh +++ b/tests/inputname.sh @@ -1,5 +1,6 @@ echo \[inputname.sh\]: testing $InputTCPServerInputName directive $srcdir/killrsyslog.sh # kill rsyslogd if it runs for some reason +source $srcdir/diag.sh generate-HOSTNAME echo port 12514 ./nettester -tinputname_imtcp_12514 -cinputname_imtcp -itcp -p12514 diff --git a/tests/killrsyslog.sh b/tests/killrsyslog.sh index aac2490..9edf773 100755 --- a/tests/killrsyslog.sh +++ b/tests/killrsyslog.sh @@ -4,10 +4,12 @@ then echo rsyslog.pid exists, trying to shut down rsyslogd process `cat rsyslog.pid`. kill -9 `cat rsyslog.pid` sleep 1 + rm rsyslog.pid fi if [ -e "rsyslog2.pid" ] then echo rsyslog2.pid exists, trying to shut down rsyslogd process `cat rsyslog2.pid`. kill -9 `cat rsyslog2.pid` sleep 1 + rm rsyslog2.pid fi diff --git a/tests/manytcp.sh b/tests/manytcp.sh index ec8f245..f52c4df 100755 --- a/tests/manytcp.sh +++ b/tests/manytcp.sh @@ -3,7 +3,7 @@ echo \[manytcp.sh\]: test concurrent tcp connections source $srcdir/diag.sh init source $srcdir/diag.sh startup manytcp.conf # the config file specifies exactly 1100 connections -source $srcdir/diag.sh tcpflood -c1000 -m40000 +source $srcdir/diag.sh tcpflood -c-1100 -m40000 # the sleep below is needed to prevent too-early termination of the tcp listener sleep 1 source $srcdir/diag.sh shutdown-when-empty # shut down rsyslogd when done processing messages diff --git a/tests/mmjsonparse_cim.sh b/tests/mmjsonparse_cim.sh new file mode 100755 index 0000000..68beeba --- /dev/null +++ b/tests/mmjsonparse_cim.sh @@ -0,0 +1,13 @@ +# added 2014-07-15 by rgerhards +# This file is part of the rsyslog project, released under ASL 2.0 +echo =============================================================================== +echo \[mmjsonparse_cim.sh\]: basic test for mmjsonparse module with "cim" cookie +source $srcdir/diag.sh init +source $srcdir/diag.sh startup mmjsonparse_cim.conf +./tcpflood -m 5000 -j "@cim: " +echo doing shutdown +source $srcdir/diag.sh shutdown-when-empty +echo wait on shutdown +source $srcdir/diag.sh wait-shutdown +source $srcdir/diag.sh seq-check 0 4999 +source $srcdir/diag.sh exit diff --git a/tests/mmjsonparse_simple.sh b/tests/mmjsonparse_simple.sh new file mode 100755 index 0000000..62ff8f9 --- /dev/null +++ b/tests/mmjsonparse_simple.sh @@ -0,0 +1,13 @@ +# added 2014-07-15 by rgerhards +# This file is part of the rsyslog project, released under ASL 2.0 +echo =============================================================================== +echo \[mmjsonparse_simple.sh\]: basic test for mmjsonparse module with defaults +source $srcdir/diag.sh init +source $srcdir/diag.sh startup mmjsonparse_simple.conf +./tcpflood -m 5000 -j "@cee: " +echo doing shutdown +source $srcdir/diag.sh shutdown-when-empty +echo wait on shutdown +source $srcdir/diag.sh wait-shutdown +source $srcdir/diag.sh seq-check 0 4999 +source $srcdir/diag.sh exit diff --git a/tests/mmpstrucdata.sh b/tests/mmpstrucdata.sh new file mode 100755 index 0000000..62b6ba9 --- /dev/null +++ b/tests/mmpstrucdata.sh @@ -0,0 +1,12 @@ +# This file is part of the rsyslog project, released under ASL 2.0 +# rgerhards, 2013-11-22 +echo =============================================================================== +echo \[mmpstrucdata.sh\]: testing mmpstrucdata +source $srcdir/diag.sh init +source $srcdir/diag.sh startup mmpstrucdata.conf +sleep 1 +source $srcdir/diag.sh tcpflood -m100 -y +source $srcdir/diag.sh shutdown-when-empty # shut down rsyslogd when done processing messages +source $srcdir/diag.sh wait-shutdown +source $srcdir/diag.sh seq-check 0 99 +source $srcdir/diag.sh exit diff --git a/tests/mysql-actq-mt-withpause-vg.sh b/tests/mysql-actq-mt-withpause-vg.sh new file mode 100755 index 0000000..ab76a02 --- /dev/null +++ b/tests/mysql-actq-mt-withpause-vg.sh @@ -0,0 +1,22 @@ +# This file is part of the rsyslog project, released under ASL 2.0 +echo =============================================================================== +echo \[mysql-act-mt.sh\]: test for mysql with multithread actionq +source $srcdir/diag.sh init +mysql --user=rsyslog --password=testbench < testsuites/mysql-truncate.sql +source $srcdir/diag.sh startup-vg mysql-actq-mt.conf +source $srcdir/diag.sh injectmsg 0 50000 +source $srcdir/diag.sh wait-queueempty +echo waiting for worker threads to timeout +./msleep 3000 +source $srcdir/diag.sh injectmsg 50000 50000 +source $srcdir/diag.sh wait-queueempty +echo waiting for worker threads to timeout +./msleep 2000 +source $srcdir/diag.sh injectmsg 100000 50000 +source $srcdir/diag.sh shutdown-when-empty +source $srcdir/diag.sh wait-shutdown-vg +source $srcdir/diag.sh check-exit-vg +# note "-s" is requried to suppress the select "field header" +mysql -s --user=rsyslog --password=testbench < testsuites/mysql-select-msg.sql > rsyslog.out.log +source $srcdir/diag.sh seq-check 0 149999 +source $srcdir/diag.sh exit diff --git a/tests/mysql-actq-mt-withpause.sh b/tests/mysql-actq-mt-withpause.sh new file mode 100755 index 0000000..d680ddf --- /dev/null +++ b/tests/mysql-actq-mt-withpause.sh @@ -0,0 +1,21 @@ +# This file is part of the rsyslog project, released under ASL 2.0 +echo =============================================================================== +echo \[mysql-act-mt.sh\]: test for mysql with multithread actionq +source $srcdir/diag.sh init +mysql --user=rsyslog --password=testbench < testsuites/mysql-truncate.sql +source $srcdir/diag.sh startup mysql-actq-mt.conf +source $srcdir/diag.sh injectmsg 0 50000 +source $srcdir/diag.sh wait-queueempty +echo waiting for worker threads to timeout +./msleep 3000 +source $srcdir/diag.sh injectmsg 50000 50000 +source $srcdir/diag.sh wait-queueempty +echo waiting for worker threads to timeout +./msleep 2000 +source $srcdir/diag.sh injectmsg 100000 50000 +source $srcdir/diag.sh shutdown-when-empty +source $srcdir/diag.sh wait-shutdown +# note "-s" is requried to suppress the select "field header" +mysql -s --user=rsyslog --password=testbench < testsuites/mysql-select-msg.sql > rsyslog.out.log +source $srcdir/diag.sh seq-check 0 149999 +source $srcdir/diag.sh exit diff --git a/tests/mysql-actq-mt.sh b/tests/mysql-actq-mt.sh new file mode 100755 index 0000000..ae96cde --- /dev/null +++ b/tests/mysql-actq-mt.sh @@ -0,0 +1,13 @@ +# This file is part of the rsyslog project, released under ASL 2.0 +echo =============================================================================== +echo \[mysql-act-mt.sh\]: test for mysql with multithread actionq +source $srcdir/diag.sh init +mysql --user=rsyslog --password=testbench < testsuites/mysql-truncate.sql +source $srcdir/diag.sh startup mysql-actq-mt.conf +source $srcdir/diag.sh injectmsg 0 150000 +source $srcdir/diag.sh shutdown-when-empty +source $srcdir/diag.sh wait-shutdown +# note "-s" is requried to suppress the select "field header" +mysql -s --user=rsyslog --password=testbench < testsuites/mysql-select-msg.sql > rsyslog.out.log +source $srcdir/diag.sh seq-check 0 149999 +source $srcdir/diag.sh exit diff --git a/tests/nettester.c b/tests/nettester.c index 4e4fe55..82f3013 100644 --- a/tests/nettester.c +++ b/tests/nettester.c @@ -12,7 +12,7 @@ * * Part of the testbench for rsyslog. * - * Copyright 2009 Rainer Gerhards and Adiscon GmbH. + * Copyright 2009-2014 Rainer Gerhards and Adiscon GmbH. * * This file is part of rsyslog. * @@ -49,6 +49,7 @@ #include <getopt.h> #include <errno.h> #include <ctype.h> +#include <netdb.h> #define EXIT_FAILURE 1 #define INVALID_SOCKET -1 @@ -65,6 +66,7 @@ static char* pszCustomConf = NULL; /* custom config file, use -c conf to specify static int verbose = 0; /* verbose output? -v option */ static int IPv4Only = 0; /* use only IPv4 in rsyslogd call? */ static char **ourEnvp; +static char *ourHostName; /* these two are quick hacks... */ int iFailed = 0; @@ -339,6 +341,44 @@ void unescapeTestdata(char *testdata) } +/* expand variables in expected string. Here we use tilde (~) as expension + * character, because the more natural % is very common in syslog messages + * (and most importantly in the samples we currently have. + * Currently supported are: + * ~H - our hostname + * Note: yes, there are vulns in this code. Doesn't matter, as it is a + * quick and dirty test program, NOT intended to be used in any production! + */ +static void +doVarsInExpected(char **pe) +{ + char *n, *newBase; + char *e = *pe; + n = newBase = malloc(strlen(e) + 1024); /* we simply say "sufficient" */ + while(*e) { + if(*e == '~') { + ++e; + if(*e == 'H') { + ++e; + char *hn = ourHostName; + while(*hn) + *n++ = *hn++; + } else { + *n++ = '?'; + ++e; + } + } else if(*e == '\\') { + ++e; /* skip */ + *n++ = *e++; + } else { + *n++ = *e++; + } + } + *n = '\0'; + free(*pe); + *pe = newBase; +} + /* Process a specific test case. File name is provided. * Needs to return 0 if all is OK, something else otherwise. */ @@ -391,9 +431,9 @@ processTestFile(int fd, char *pszFileName) */ getline(&expected, &lenLn, fp); expected[strlen(expected)-1] = '\0'; /* remove \n */ + doVarsInExpected(&expected); /* pull response from server and then check if it meets our expectation */ -//printf("try pull pipe...\n"); readLine(fd, buf); if(strlen(buf) == 0) { printf("something went wrong - read a zero-length string from rsyslogd\n"); @@ -493,6 +533,23 @@ void doAtExit(void) unlink(NETTEST_INPUT_CONF_FILE); } + +/* Note: the HOSTNAME file must have been pre-generated */ +static void +getHostname(void) +{ + size_t dummy; + FILE *fp; + if((fp = fopen("HOSTNAME", "r")) == NULL) { + perror("HOSTNAME"); + printf("error opening HOSTNAME configuration file\n"); + exit(1); + } + getline(&ourHostName, &dummy, fp); + fclose(fp); +} + + /* Run the test suite. This must be called with exactly one parameter, the * name of the test suite. For details, see file header comment at the top * of this file. @@ -508,6 +565,8 @@ int main(int argc, char *argv[], char *envp[]) char testcases[4096]; ourEnvp = envp; + getHostname(); + while((opt = getopt(argc, argv, "4c:i:p:t:v")) != EOF) { switch((char)opt) { case '4': diff --git a/tests/omod-if-array.sh b/tests/omod-if-array.sh index 4e916f1..3e0409e 100755 --- a/tests/omod-if-array.sh +++ b/tests/omod-if-array.sh @@ -1,4 +1,8 @@ echo \[omod-if-array.sh\]: test omod-if-array via udp +echo NOTE: the interface checked with this test is currently NOT +echo supported. We may support it again in the future. So for now\, +echo we just skip this test and do not remove it. +exit 77 $srcdir/killrsyslog.sh # kill rsyslogd if it runs for some reason ./nettester -tomod-if-array -iudp -p4711 diff --git a/tests/parsertest.sh b/tests/parsertest.sh index 9f9c2f7..eda098e 100755 --- a/tests/parsertest.sh +++ b/tests/parsertest.sh @@ -1,5 +1,15 @@ echo TEST: \[parsertest.sh\]: various parser tests source $srcdir/diag.sh init + +# first we need to obtain the hostname as rsyslog sees it +rm -f HOSTNAME +source $srcdir/diag.sh startup gethostname.conf +source $srcdir/diag.sh tcpflood -m1 -M "<128>" +./msleep 100 +source $srcdir/diag.sh shutdown-when-empty # shut down rsyslogd when done processing messages +source $srcdir/diag.sh wait-shutdown # we need to wait until rsyslogd is finished! + +# now start the real tests source $srcdir/diag.sh nettester parse1 udp source $srcdir/diag.sh nettester parse1 tcp source $srcdir/diag.sh nettester parse2 udp @@ -38,4 +48,5 @@ source $srcdir/diag.sh nettester parse-nodate tcp -4 source $srcdir/diag.sh nettester snare_ccoff_udp udp -4 source $srcdir/diag.sh nettester snare_ccoff_udp2 udp -4 +rm -f HOSTNAME source $srcdir/diag.sh exit diff --git a/tests/proprepltest.sh b/tests/proprepltest.sh index 2e59a31..fcc0b18 100755 --- a/tests/proprepltest.sh +++ b/tests/proprepltest.sh @@ -1,5 +1,6 @@ echo \[proprepltest.sh\]: various tests for the property replacer source $srcdir/diag.sh init +source $srcdir/diag.sh generate-HOSTNAME source $srcdir/diag.sh nettester rfctag udp source $srcdir/diag.sh nettester rfctag tcp source $srcdir/diag.sh nettester nolimittag udp diff --git a/tests/rcvr_fail_restore.sh b/tests/rcvr_fail_restore.sh index 79486f1..0822f57 100755 --- a/tests/rcvr_fail_restore.sh +++ b/tests/rcvr_fail_restore.sh @@ -11,9 +11,12 @@ source $srcdir/diag.sh init # set instance-specific debugging parameters! #export RSYSLOG_DEBUG="debug nostdout" #export RSYSLOG_DEBUGLOG="log2" +echo starting receiver source $srcdir/diag.sh startup rcvr_fail_restore_rcvr.conf 2 +#export RSYSLOG_DEBUG="debug nostdout" #export RSYSLOG_DEBUGLOG="log" #valgrind="valgrind" +echo starting sender source $srcdir/diag.sh startup rcvr_fail_restore_sender.conf # re-set params so that new instances do not thrash it... #unset RSYSLOG_DEBUG @@ -29,6 +32,7 @@ source $srcdir/diag.sh wait-queueempty # Step 2: shutdown receiver, then send some more data, which then # needs to go into the queue. # +echo step 2 source $srcdir/diag.sh shutdown-when-empty 2 source $srcdir/diag.sh wait-shutdown 2 @@ -41,6 +45,7 @@ ls -l test-spool # # Step 3: restart receiver, wait that the sender drains its queue # +echo step 3 #export RSYSLOG_DEBUGLOG="log2" source $srcdir/diag.sh startup rcvr_fail_restore_rcvr.conf 2 echo waiting for sender to drain queue [may need a short while] @@ -54,6 +59,7 @@ echo file size to expect is $OLDFILESIZE # Step 4: send new data. Queue files are not permitted to grow now # (but one file continous to exist). # +echo step 4 source $srcdir/diag.sh injectmsg 11001 10 source $srcdir/diag.sh wait-queueempty @@ -82,6 +88,7 @@ fi # Step 5: stop receiver again, then send some more data, which then # needs to go into the queue. # +echo step 5 echo "*** done primary test *** now checking if DA can be restarted" source $srcdir/diag.sh shutdown-when-empty 2 source $srcdir/diag.sh wait-shutdown 2 @@ -95,6 +102,7 @@ ls -l test-spool # # Step 6: restart receiver, wait that the sender drains its queue # +echo step 6 source $srcdir/diag.sh startup rcvr_fail_restore_rcvr.conf 2 echo waiting for sender to drain queue [may need a short while] source $srcdir/diag.sh wait-queueempty @@ -118,5 +126,5 @@ then exit 1 fi # do the final check -source $srcdir/diag.sh seq-check 1 21010 +source $srcdir/diag.sh seq-check 1 21010 -m 100 source $srcdir/diag.sh exit diff --git a/tests/rfc5424parser.sh b/tests/rfc5424parser.sh new file mode 100755 index 0000000..3f5be49 --- /dev/null +++ b/tests/rfc5424parser.sh @@ -0,0 +1,12 @@ +# This file is part of the rsyslog project, released under ASL 2.0 +# rgerhards, 2013-11-22 +echo =============================================================================== +echo \[rfc5424parser.sh\]: testing mmpstrucdata +source $srcdir/diag.sh init +source $srcdir/diag.sh startup rfc5424parser.conf +sleep 1 +source $srcdir/diag.sh tcpflood -m100 -y +source $srcdir/diag.sh shutdown-when-empty # shut down rsyslogd when done processing messages +source $srcdir/diag.sh wait-shutdown +source $srcdir/diag.sh seq-check 0 99 +source $srcdir/diag.sh exit diff --git a/tests/rs_optimizer_pri.sh b/tests/rs_optimizer_pri.sh new file mode 100755 index 0000000..4d6e463 --- /dev/null +++ b/tests/rs_optimizer_pri.sh @@ -0,0 +1,17 @@ +# Test for the RainerScript optimizer, folding of +# syslogfacility/priority-text to prifilt. Unfortunately, we cannot yet +# automatically detect if the optimizer does not correctly fold, but we +# can at least detect if it segfaults or otherwise creates incorrect code. +# This file is part of the rsyslog project, released under ASL 2.0 +# rgerhards, 2013-11-20 +echo =============================================================================== +echo \[rs_optimizer_pri.sh\]: testing RainerScript PRI optimizer +source $srcdir/diag.sh init +source $srcdir/diag.sh startup rs_optimizer_pri.conf +sleep 1 +source $srcdir/diag.sh tcpflood -m100 # correct facility +source $srcdir/diag.sh tcpflood -m100 -P175 # incorrect facility --> must be ignored +source $srcdir/diag.sh shutdown-when-empty # shut down rsyslogd when done processing messages +source $srcdir/diag.sh wait-shutdown +source $srcdir/diag.sh seq-check 0 99 +source $srcdir/diag.sh exit diff --git a/tests/rscript_eq.sh b/tests/rscript_eq.sh new file mode 100755 index 0000000..5668d4b --- /dev/null +++ b/tests/rscript_eq.sh @@ -0,0 +1,13 @@ +# added 2014-01-17 by rgerhards +# This file is part of the rsyslog project, released under ASL 2.0 +echo =============================================================================== +echo \[rscript_eq.sh\]: testing rainerscript EQ statement +source $srcdir/diag.sh init +source $srcdir/diag.sh startup rscript_eq.conf +source $srcdir/diag.sh injectmsg 0 8000 +echo doing shutdown +source $srcdir/diag.sh shutdown-when-empty +echo wait on shutdown +source $srcdir/diag.sh wait-shutdown +source $srcdir/diag.sh seq-check 5000 5002 +source $srcdir/diag.sh exit diff --git a/tests/rscript_ge.sh b/tests/rscript_ge.sh new file mode 100755 index 0000000..798e944 --- /dev/null +++ b/tests/rscript_ge.sh @@ -0,0 +1,13 @@ +# added 2014-01-17 by rgerhards +# This file is part of the rsyslog project, released under ASL 2.0 +echo =============================================================================== +echo \[rscript_ge.sh\]: testing rainerscript GE statement +source $srcdir/diag.sh init +source $srcdir/diag.sh startup rscript_ge.conf +source $srcdir/diag.sh injectmsg 0 8000 +echo doing shutdown +source $srcdir/diag.sh shutdown-when-empty +echo wait on shutdown +source $srcdir/diag.sh wait-shutdown +source $srcdir/diag.sh seq-check 0 4999 +source $srcdir/diag.sh exit diff --git a/tests/rscript_gt.sh b/tests/rscript_gt.sh new file mode 100755 index 0000000..157f6f3 --- /dev/null +++ b/tests/rscript_gt.sh @@ -0,0 +1,13 @@ +# added 2014-01-17 by rgerhards +# This file is part of the rsyslog project, released under ASL 2.0 +echo =============================================================================== +echo \[rscript_gt.sh\]: testing rainerscript GT statement +source $srcdir/diag.sh init +source $srcdir/diag.sh startup rscript_gt.conf +source $srcdir/diag.sh injectmsg 0 8000 +echo doing shutdown +source $srcdir/diag.sh shutdown-when-empty +echo wait on shutdown +source $srcdir/diag.sh wait-shutdown +source $srcdir/diag.sh seq-check 5000 7999 +source $srcdir/diag.sh exit diff --git a/tests/rscript_le.sh b/tests/rscript_le.sh new file mode 100755 index 0000000..426f207 --- /dev/null +++ b/tests/rscript_le.sh @@ -0,0 +1,13 @@ +# added 2014-01-17 by rgerhards +# This file is part of the rsyslog project, released under ASL 2.0 +echo =============================================================================== +echo \[rscript_le.sh\]: testing rainerscript LE statement +source $srcdir/diag.sh init +source $srcdir/diag.sh startup rscript_le.conf +source $srcdir/diag.sh injectmsg 0 8000 +echo doing shutdown +source $srcdir/diag.sh shutdown-when-empty +echo wait on shutdown +source $srcdir/diag.sh wait-shutdown +source $srcdir/diag.sh seq-check 0 5000 +source $srcdir/diag.sh exit diff --git a/tests/rscript_lt.sh b/tests/rscript_lt.sh new file mode 100755 index 0000000..5d35164 --- /dev/null +++ b/tests/rscript_lt.sh @@ -0,0 +1,13 @@ +# added 2014-01-17 by rgerhards +# This file is part of the rsyslog project, released under ASL 2.0 +echo =============================================================================== +echo \[rscript_lt.sh\]: testing rainerscript LT statement +source $srcdir/diag.sh init +source $srcdir/diag.sh startup rscript_lt.conf +source $srcdir/diag.sh injectmsg 0 8000 +echo doing shutdown +source $srcdir/diag.sh shutdown-when-empty +echo wait on shutdown +source $srcdir/diag.sh wait-shutdown +source $srcdir/diag.sh seq-check 0 4999 +source $srcdir/diag.sh exit diff --git a/tests/rscript_ne.sh b/tests/rscript_ne.sh new file mode 100755 index 0000000..9459ddd --- /dev/null +++ b/tests/rscript_ne.sh @@ -0,0 +1,13 @@ +# added 2014-01-17 by rgerhards +# This file is part of the rsyslog project, released under ASL 2.0 +echo =============================================================================== +echo \[rscript_ne.sh\]: testing rainerscript NE statement +source $srcdir/diag.sh init +source $srcdir/diag.sh startup rscript_ne.conf +source $srcdir/diag.sh injectmsg 0 8000 +echo doing shutdown +source $srcdir/diag.sh shutdown-when-empty +echo wait on shutdown +source $srcdir/diag.sh wait-shutdown +source $srcdir/diag.sh seq-check 5000 5002 +source $srcdir/diag.sh exit diff --git a/tests/rulesetmultiqueue-v6.sh b/tests/rulesetmultiqueue-v6.sh new file mode 100755 index 0000000..21166fe --- /dev/null +++ b/tests/rulesetmultiqueue-v6.sh @@ -0,0 +1,33 @@ +# Test for disk-only queue mode with v6+ config +# This tests defines three rulesets, each one with its own queue. Then, it +# sends data to them and checks the outcome. Note that we do need to +# use some custom code as the test driver framework does not (yet?) +# support multi-output-file operations. +# added 2013-11-14 by Rgerhards +# This file is part of the rsyslog project, released under GPLv3 +echo =============================================================================== +echo \[rulesetmultiqueu.sh\]: testing multiple queues via rulesets +source $srcdir/diag.sh init +rm -f rsyslog.out1.log rsyslog.out2.log rsyslog.out3.log +source $srcdir/diag.sh startup rulesetmultiqueue-v6.conf +source $srcdir/diag.sh wait-startup +# now fill the three files (a bit sequentially, but they should +# still get their share of concurrency - to increase the chance +# we use three connections per set). +source $srcdir/diag.sh tcpflood -c3 -p13514 -m20000 -i0 +source $srcdir/diag.sh tcpflood -c3 -p13515 -m20000 -i20000 +source $srcdir/diag.sh tcpflood -c3 -p13516 -m20000 -i40000 + +# in this version of the imdiag, we do not have the capability to poll +# all queues for emptyness. So we do a sleep in the hopes that this will +# sufficiently drain the queues. This is race, but the best we currently +# can do... - rgerhards, 2009-11-05 +sleep 2 +source $srcdir/diag.sh shutdown-when-empty # shut down rsyslogd when done processing messages +source $srcdir/diag.sh wait-shutdown +# now consolidate all logs into a single one so that we can use the +# regular check logic +cat rsyslog.out1.log rsyslog.out2.log rsyslog.out3.log > rsyslog.out.log +source $srcdir/diag.sh seq-check 0 59999 +rm -f rsyslog.out1.log rsyslog.out2.log rsyslog.out3.log +source $srcdir/diag.sh exit diff --git a/tests/sndrcv_relp.sh b/tests/sndrcv_relp.sh new file mode 100755 index 0000000..e679651 --- /dev/null +++ b/tests/sndrcv_relp.sh @@ -0,0 +1,5 @@ +# added 2013-12-10 by Rgerhards +# This file is part of the rsyslog project, released under ASL 2.0 +echo =============================================================================== +echo \[sndrcv_relp.sh\]: testing sending and receiving via relp +source $srcdir/sndrcv_drvr.sh sndrcv_relp 50000 diff --git a/tests/stop-localvar.sh b/tests/stop-localvar.sh new file mode 100755 index 0000000..9157301 --- /dev/null +++ b/tests/stop-localvar.sh @@ -0,0 +1,12 @@ +# Test for "stop" statement +# This file is part of the rsyslog project, released under ASL 2.0 +echo =============================================================================== +echo \[stop-localvar.sh\]: testing stop statement together with local variables +source $srcdir/diag.sh init +source $srcdir/diag.sh startup stop-localvar.conf +sleep 1 +source $srcdir/diag.sh tcpflood -m2000 -i1 +source $srcdir/diag.sh shutdown-when-empty # shut down rsyslogd when done processing messages +source $srcdir/diag.sh wait-shutdown +source $srcdir/diag.sh seq-check 100 999 +source $srcdir/diag.sh exit diff --git a/tests/stop-msgvar.sh b/tests/stop-msgvar.sh new file mode 100755 index 0000000..d8902da --- /dev/null +++ b/tests/stop-msgvar.sh @@ -0,0 +1,12 @@ +# Test for "stop" statement +# This file is part of the rsyslog project, released under ASL 2.0 +echo =============================================================================== +echo \[stop-msgvar.sh\]: testing stop statement together with message variables +source $srcdir/diag.sh init +source $srcdir/diag.sh startup stop-msgvar.conf +sleep 1 +source $srcdir/diag.sh tcpflood -m2000 -i1 +source $srcdir/diag.sh shutdown-when-empty # shut down rsyslogd when done processing messages +source $srcdir/diag.sh wait-shutdown +source $srcdir/diag.sh seq-check 100 999 +source $srcdir/diag.sh exit diff --git a/tests/stop.sh b/tests/stop.sh new file mode 100755 index 0000000..f3dcf99 --- /dev/null +++ b/tests/stop.sh @@ -0,0 +1,12 @@ +# Test for "stop" statement +# This file is part of the rsyslog project, released under ASL 2.0 +echo =============================================================================== +echo \[stop.sh\]: testing stop statement +source $srcdir/diag.sh init +source $srcdir/diag.sh startup stop.conf +sleep 1 +source $srcdir/diag.sh tcpflood -m10 -i1 +source $srcdir/diag.sh shutdown-when-empty # shut down rsyslogd when done processing messages +source $srcdir/diag.sh wait-shutdown +source $srcdir/diag.sh seq-check 2 10 +source $srcdir/diag.sh exit diff --git a/tests/tabescape_dflt.sh b/tests/tabescape_dflt.sh index d0e13ec..6a47949 100755 --- a/tests/tabescape_dflt.sh +++ b/tests/tabescape_dflt.sh @@ -1,6 +1,7 @@ echo =============================================================================== echo \[tabescape_dflt.sh\]: test for default tab escaping -$srcdir/killrsyslog.sh # kill rsyslogd if it runs for some reason +source $srcdir/diag.sh init +source $srcdir/diag.sh generate-HOSTNAME ./nettester -ttabescape_dflt -iudp if [ "$?" -ne "0" ]; then diff --git a/tests/tabescape_off.sh b/tests/tabescape_off.sh index 71ede7c..90f4792 100755 --- a/tests/tabescape_off.sh +++ b/tests/tabescape_off.sh @@ -1,6 +1,7 @@ echo =============================================================================== echo \[tabescape_off.sh\]: test for tab escaping off -$srcdir/killrsyslog.sh # kill rsyslogd if it runs for some reason +source $srcdir/diag.sh init +source $srcdir/diag.sh generate-HOSTNAME ./nettester -ttabescape_off -iudp if [ "$?" -ne "0" ]; then diff --git a/tests/tcpflood.c b/tests/tcpflood.c index b3cef2e..1c60aa4 100644 --- a/tests/tcpflood.c +++ b/tests/tcpflood.c @@ -6,7 +6,12 @@ * -p target port (default 13514) * -n number of target ports (targets are in range -p..(-p+-n-1) * Note -c must also be set to at LEAST the number of -n! - * -c number of connections (default 1) + * -c number of connections (default 1), use negative number + * to set a "soft limit": if tcpflood cannot open the + * requested number of connections, gracefully degrade to + * whatever number could be opened. This is useful in environments + * where system config constraints cannot be overriden (e.g. + * vservers, non-admin users, ...) * -m number of messages to send (connection is random) * -i initial message number (optional) * -P PRI to be used for generated messages (default is 167). @@ -48,13 +53,15 @@ * -b number of messages within a batch (default: 100,000,000 millions) * -Y use multiple threads, one per connection (which means 1 if one only connection * is configured!) + * -y use RFC5424 style test message * -z private key file for TLS mode * -Z cert (public key) file for TLS mode * -L loglevel to use for GnuTLS troubleshooting (0-off to 10-all, 0 default) + * -j format message in json, parameter is JSON cookie * * Part of the testbench for rsyslog. * - * Copyright 2009, 2010 Rainer Gerhards and Adiscon GmbH. + * Copyright 2009, 2013 Rainer Gerhards and Adiscon GmbH. * * This file is part of rsyslog. * @@ -111,9 +118,11 @@ static int targetPort = 13514; static int numTargetPorts = 1; static int dynFileIDs = 0; static int extraDataLen = 0; /* amount of extra data to add to message */ +static int useRFC5424Format = 0; /* should the test message be in RFC5424 format? */ static int bRandomizeExtraData = 0; /* randomize amount of extra data added */ static int numMsgsToSend; /* number of messages to send */ -static unsigned numConnections = 1; /* number of connections to create */ +static int numConnections = 1; /* number of connections to create */ +static int softLimitConnections = 0; /* soft connection limit, see -c option description */ static int *sockArray; /* array of sockets to use */ static int msgNum = 0; /* initial message number to start with */ static int bShowProgress = 1; /* show progress messages */ @@ -137,6 +146,7 @@ static int numThrds = 1; /* number of threads to use */ static char *tlsCertFile = NULL; static char *tlsKeyFile = NULL; static int tlsLogLevel = 0; +static char *jsonCookie = NULL; /* if non-NULL, use JSON format with this cookie */ #ifdef ENABLE_GNUTLS static gnutls_session_t *sessArray; /* array of TLS sessions to use */ @@ -208,7 +218,7 @@ int openConn(int *fd) int rnd; if((sock=socket(AF_INET, SOCK_STREAM, 0))==-1) { - perror("socket()"); + perror("\nsocket()"); return(1); } @@ -250,7 +260,7 @@ int openConn(int *fd) */ int openConnections(void) { - unsigned i; + int i; char msgBuf[128]; size_t lenMsg; @@ -270,6 +280,12 @@ int openConnections(void) } if(openConn(&(sockArray[i])) != 0) { printf("error in trying to open connection i=%d\n", i); + if(softLimitConnections) { + numConnections = i - 1; + printf("Connection limit is soft, continuing with only %d " + "connections.\n", numConnections); + break; + } return 1; } if(transport == TP_TLS) { @@ -294,7 +310,7 @@ int openConnections(void) */ void closeConnections(void) { - unsigned i; + int i; size_t lenMsg; struct linger ling; char msgBuf[128]; @@ -358,13 +374,28 @@ genMsg(char *buf, size_t maxBuf, int *pLenBuf, struct instdata *inst) } } } while(!done); /* Attention: do..while()! */ + } else if(jsonCookie != NULL) { + if(useRFC5424Format) { + *pLenBuf = snprintf(buf, maxBuf, "<%s>1 2003-03-01T01:00:00.000Z mymachine.example.com tcpflood " + "- tag [tcpflood@32473 MSGNUM=\"%8.8d\"] %s{\"msgnum\":%d}%c", + msgPRI, msgNum, jsonCookie, msgNum, frameDelim); + } else { + *pLenBuf = snprintf(buf, maxBuf, "<%s>Mar 1 01:00:00 172.20.245.8 tag %s{\"msgnum\":%d}%c", + msgPRI, jsonCookie, msgNum, frameDelim); + } } else if(MsgToSend == NULL) { if(dynFileIDs > 0) { snprintf(dynFileIDBuf, sizeof(dynFileIDBuf), "%d:", rand() % dynFileIDs); } if(extraDataLen == 0) { - *pLenBuf = snprintf(buf, maxBuf, "<%s>Mar 1 01:00:00 172.20.245.8 tag msgnum:%s%8.8d:%c", - msgPRI, dynFileIDBuf, msgNum, frameDelim); + if(useRFC5424Format) { + *pLenBuf = snprintf(buf, maxBuf, "<%s>1 2003-03-01T01:00:00.000Z mymachine.example.com tcpflood " + "- tag [tcpflood@32473 MSGNUM=\"%8.8d\"] msgnum:%s%8.8d:%c", + msgPRI, msgNum, dynFileIDBuf, msgNum, frameDelim); + } else { + *pLenBuf = snprintf(buf, maxBuf, "<%s>Mar 1 01:00:00 172.20.245.8 tag msgnum:%s%8.8d:%c", + msgPRI, dynFileIDBuf, msgNum, frameDelim); + } } else { if(bRandomizeExtraData) edLen = ((long) rand() + extraDataLen) % extraDataLen + 1; @@ -372,8 +403,14 @@ genMsg(char *buf, size_t maxBuf, int *pLenBuf, struct instdata *inst) edLen = extraDataLen; memset(extraData, 'X', edLen); extraData[edLen] = '\0'; - *pLenBuf = snprintf(buf, maxBuf, "<%s>Mar 1 01:00:00 172.20.245.8 tag msgnum:%s%8.8d:%d:%s%c", - msgPRI, dynFileIDBuf, msgNum, edLen, extraData, frameDelim); + if(useRFC5424Format) { + *pLenBuf = snprintf(buf, maxBuf, "<%s>1 2003-03-01T01:00:00.000Z mymachine.example.com tcpflood " + "- tag [tcpflood@32473 MSGNUM=\"%8.8d\"] msgnum:%s%8.8d:%c", + msgPRI, msgNum, dynFileIDBuf, msgNum, frameDelim); + } else { + *pLenBuf = snprintf(buf, maxBuf, "<%s>Mar 1 01:00:00 172.20.245.8 tag msgnum:%s%8.8d:%d:%s%c", + msgPRI, dynFileIDBuf, msgNum, edLen, extraData, frameDelim); + } } } else { /* use fixed message format from command line */ @@ -419,7 +456,7 @@ int sendMessages(struct instdata *inst) if(runMultithreaded) { socknum = inst->idx; } else { - if(i < numConnections) + if((int) i < numConnections) socknum = i; else if(i >= inst->numMsgs - numConnections) { socknum = i - (inst->numMsgs - numConnections); @@ -830,7 +867,7 @@ int main(int argc, char *argv[]) setvbuf(stdout, buf, _IONBF, 48); - while((opt = getopt(argc, argv, "b:ef:F:t:p:c:C:m:i:I:P:d:Dn:L:M:rsBR:S:T:XW:Yz:Z:")) != -1) { + while((opt = getopt(argc, argv, "b:ef:F:t:p:c:C:m:i:I:P:d:Dn:L:M:rsBR:S:T:XW:yYz:Z:j:")) != -1) { switch (opt) { case 'b': batchsize = atoll(optarg); break; @@ -840,7 +877,11 @@ int main(int argc, char *argv[]) break; case 'n': numTargetPorts = atoi(optarg); break; - case 'c': numConnections = (unsigned) atoi(optarg); + case 'c': numConnections = atoi(optarg); + if(numConnections < 0) { + numConnections *= -1; + softLimitConnections = 1; + } break; case 'C': numFileIterations = atoi(optarg); break; @@ -850,6 +891,8 @@ int main(int argc, char *argv[]) break; case 'P': msgPRI = optarg; break; + case 'j': jsonCookie = optarg; + break; case 'd': extraDataLen = atoi(optarg); if(extraDataLen > MAX_EXTRADATA_LEN) { fprintf(stderr, "-d max is %d!\n", @@ -908,6 +951,8 @@ int main(int argc, char *argv[]) break; case 'Y': runMultithreaded = 1; break; + case 'y': useRFC5424Format = 1; + break; case 'z': tlsKeyFile = optarg; break; case 'Z': tlsCertFile = optarg; diff --git a/tests/testsuites/elasticsearch-basic-bulk.conf b/tests/testsuites/elasticsearch-basic-bulk.conf new file mode 100644 index 0000000..69a0495 --- /dev/null +++ b/tests/testsuites/elasticsearch-basic-bulk.conf @@ -0,0 +1,10 @@ +$IncludeConfig diag-common.conf + +template(name="tpl" type="string" + string="{\"msgnum\":\"%msg:F,58:2%\"}") + +module(load="../plugins/omelasticsearch/.libs/omelasticsearch") +:msg, contains, "msgnum:" action(type="omelasticsearch" + template="tpl" + searchIndex="rsyslog_testbench" + bulkmode="on") diff --git a/tests/testsuites/elasticsearch-basic-errorfile-empty.conf b/tests/testsuites/elasticsearch-basic-errorfile-empty.conf new file mode 100644 index 0000000..6b0371c --- /dev/null +++ b/tests/testsuites/elasticsearch-basic-errorfile-empty.conf @@ -0,0 +1,10 @@ +$IncludeConfig diag-common.conf + +template(name="tpl" type="string" + string="{\"msgnum\":\"%msg:F,58:2%\"}") + +module(load="../plugins/omelasticsearch/.libs/omelasticsearch") +:msg, contains, "msgnum:" action(type="omelasticsearch" + template="tpl" + searchIndex="rsyslog_testbench" + errorFile="./rsyslog.errorfile") diff --git a/tests/testsuites/elasticsearch-basic-errorfile-populated.conf b/tests/testsuites/elasticsearch-basic-errorfile-populated.conf new file mode 100644 index 0000000..d13b712 --- /dev/null +++ b/tests/testsuites/elasticsearch-basic-errorfile-populated.conf @@ -0,0 +1,15 @@ +$IncludeConfig diag-common.conf + +# Note: we must mess up with the template, because we can not +# instruct ES to put further constraints on the data type and +# values. So we require integer and make sure it is none. +template(name="tpl" type="string" + string="{\"msgnum\":\"x%msg:F,58:2%\"}") + +module(load="../plugins/omelasticsearch/.libs/omelasticsearch") +:msg, contains, "msgnum:" action(type="omelasticsearch" + template="tpl" + searchIndex="rsyslog_testbench" + searchType="test-type" + bulkmode="off" + errorFile="./rsyslog.errorfile") diff --git a/tests/testsuites/elasticsearch-basic.conf b/tests/testsuites/elasticsearch-basic.conf new file mode 100644 index 0000000..627bacc --- /dev/null +++ b/tests/testsuites/elasticsearch-basic.conf @@ -0,0 +1,9 @@ +$IncludeConfig diag-common.conf + +template(name="tpl" type="string" + string="{\"msgnum\":\"%msg:F,58:2%\"}") + +module(load="../plugins/omelasticsearch/.libs/omelasticsearch") +:msg, contains, "msgnum:" action(type="omelasticsearch" + template="tpl" + searchIndex="rsyslog_testbench") diff --git a/tests/testsuites/elasticsearch-bulk-errorfile-empty.conf b/tests/testsuites/elasticsearch-bulk-errorfile-empty.conf new file mode 100644 index 0000000..46e7cce --- /dev/null +++ b/tests/testsuites/elasticsearch-bulk-errorfile-empty.conf @@ -0,0 +1,11 @@ +$IncludeConfig diag-common.conf + +template(name="tpl" type="string" + string="{\"msgnum\":\"%msg:F,58:2%\"}") + +module(load="../plugins/omelasticsearch/.libs/omelasticsearch") +:msg, contains, "msgnum:" action(type="omelasticsearch" + template="tpl" + searchIndex="rsyslog_testbench" + bulkmode="on" + errorFile="./rsyslog.errorfile") diff --git a/tests/testsuites/elasticsearch-bulk-errorfile-populated.conf b/tests/testsuites/elasticsearch-bulk-errorfile-populated.conf new file mode 100644 index 0000000..21e7ddd --- /dev/null +++ b/tests/testsuites/elasticsearch-bulk-errorfile-populated.conf @@ -0,0 +1,15 @@ +$IncludeConfig diag-common.conf + +# Note: we must mess up with the template, because we can not +# instruct ES to put further constraints on the data type and +# values. So we require integer and make sure it is none. +template(name="tpl" type="string" + string="{\"msgnum\":\"x%msg:F,58:2%\"}") + +module(load="../plugins/omelasticsearch/.libs/omelasticsearch") +:msg, contains, "msgnum:" action(type="omelasticsearch" + template="tpl" + searchIndex="rsyslog_testbench" + searchType="test-type" + bulkmode="on" + errorFile="./rsyslog.errorfile") diff --git a/tests/testsuites/execonlywhenprevsuspended.conf b/tests/testsuites/execonlywhenprevsuspended.conf index 04dc6b5..8f9263d 100644 --- a/tests/testsuites/execonlywhenprevsuspended.conf +++ b/tests/testsuites/execonlywhenprevsuspended.conf @@ -1,5 +1,6 @@ # See main .sh file for info # rgerhards, 2010-06-23 +main_queue(queue.workerthreads="1") $IncludeConfig diag-common.conf # omtesting provides the ability to cause "SUSPENDED" action state diff --git a/tests/testsuites/execonlywhenprevsuspended_multiwrkr.conf b/tests/testsuites/execonlywhenprevsuspended_multiwrkr.conf new file mode 100644 index 0000000..7af2536 --- /dev/null +++ b/tests/testsuites/execonlywhenprevsuspended_multiwrkr.conf @@ -0,0 +1,12 @@ +main_queue(queue.dequeueBatchSize="10" queue.workerthreads="3" queue.workerthreadminimummessages="100") +$IncludeConfig diag-common.conf + +# omtesting provides the ability to cause "SUSPENDED" action state +$ModLoad ../plugins/omtesting/.libs/omtesting + +$MainMsgQueueTimeoutShutdown 100000 +$template outfmt,"%msg:F,58:2%\n" + +:msg, contains, "msgnum:" :omtesting:fail 2 0 +$ActionExecOnlyWhenPreviousIsSuspended on +& ./rsyslog.out.log;outfmt diff --git a/tests/testsuites/fac_authpriv.conf b/tests/testsuites/fac_authpriv.conf new file mode 100644 index 0000000..6a6484b --- /dev/null +++ b/tests/testsuites/fac_authpriv.conf @@ -0,0 +1,7 @@ +$IncludeConfig diag-common.conf + +$ModLoad ../plugins/imtcp/.libs/imtcp +$InputTCPServerRun 13514 + +$template outfmt,"%msg:F,58:2%,%msg:F,58:3%,%msg:F,58:4%\n" +authpriv.* ./rsyslog.out.log;outfmt diff --git a/tests/testsuites/fac_local0.conf b/tests/testsuites/fac_local0.conf new file mode 100644 index 0000000..2357e63 --- /dev/null +++ b/tests/testsuites/fac_local0.conf @@ -0,0 +1,8 @@ +$IncludeConfig diag-common.conf + +module(load="../plugins/imtcp/.libs/imtcp") +input(type="imtcp" port="13514") + +template(type="string" name="outfmt" string="%msg:F,58:2%,%msg:F,58:3%,%msg:F,58:4%\n") +if $syslogfacility-text == "local0" then + action(type="omfile" file="rsyslog.out.log" template="outfmt") diff --git a/tests/testsuites/fac_mail.conf b/tests/testsuites/fac_mail.conf new file mode 100644 index 0000000..ffe0dfd --- /dev/null +++ b/tests/testsuites/fac_mail.conf @@ -0,0 +1,7 @@ +$IncludeConfig diag-common.conf + +module(load="../plugins/imtcp/.libs/imtcp") +input(type="imtcp" port="13514") + +template(type="string" name="outfmt" string="%msg:F,58:2%,%msg:F,58:3%,%msg:F,58:4%\n") +mail.* action(type="omfile" file="rsyslog.out.log" template="outfmt") diff --git a/tests/testsuites/fac_news.conf b/tests/testsuites/fac_news.conf new file mode 100644 index 0000000..efc6ba0 --- /dev/null +++ b/tests/testsuites/fac_news.conf @@ -0,0 +1,8 @@ +$IncludeConfig diag-common.conf + +module(load="../plugins/imtcp/.libs/imtcp") +input(type="imtcp" port="13514") + +template(type="string" name="outfmt" string="%msg:F,58:2%,%msg:F,58:3%,%msg:F,58:4%\n") +if prifilt("news.*") then + action(type="omfile" file="rsyslog.out.log" template="outfmt") diff --git a/tests/testsuites/fac_uucp.conf b/tests/testsuites/fac_uucp.conf new file mode 100644 index 0000000..90d9646 --- /dev/null +++ b/tests/testsuites/fac_uucp.conf @@ -0,0 +1,7 @@ +$IncludeConfig diag-common.conf + +module(load="../plugins/imtcp/.libs/imtcp") +input(type="imtcp" port="13514") + +template(type="string" name="outfmt" string="%msg:F,58:2%,%msg:F,58:3%,%msg:F,58:4%\n") +uucp.* action(type="omfile" file="rsyslog.out.log" template="outfmt") diff --git a/tests/testsuites/global_vars.conf b/tests/testsuites/global_vars.conf new file mode 100644 index 0000000..0d1a3cb --- /dev/null +++ b/tests/testsuites/global_vars.conf @@ -0,0 +1,17 @@ +$IncludeConfig diag-common.conf + +$MainMsgQueueTimeoutShutdown 10000 + +module(load="../plugins/imtcp/.libs/imtcp") +input(type="imtcp" port="13514") + +template(name="outfmt" type="string" string="%$/msgnum%\n") +template(name="dynfile" type="string" string="rsyslog.out.log") /* trick to use relative path names! */ + +if $/msgnum == "" then + set $/msgnum = 0; + +if $msg contains "msgnum:" then { + action(type="omfile" dynaFile="dynfile" template="outfmt") + set $/msgnum = $/msgnum + 1; +} diff --git a/tests/testsuites/mmjsonparse_cim.conf b/tests/testsuites/mmjsonparse_cim.conf new file mode 100644 index 0000000..47cfd7d --- /dev/null +++ b/tests/testsuites/mmjsonparse_cim.conf @@ -0,0 +1,11 @@ +$IncludeConfig diag-common.conf +template(name="outfmt" type="string" string="%$!cim!msgnum%\n") + +module(load="../plugins/mmjsonparse/.libs/mmjsonparse") +module(load="../plugins/imptcp/.libs/imptcp") +input(type="imptcp" port="13514") + +action(type="mmjsonparse" cookie="@cim:" container="!cim") +if $parsesuccess == "OK" then { + action(type="omfile" file="./rsyslog.out.log" template="outfmt") +} diff --git a/tests/testsuites/mmjsonparse_simple.conf b/tests/testsuites/mmjsonparse_simple.conf new file mode 100644 index 0000000..f298a57 --- /dev/null +++ b/tests/testsuites/mmjsonparse_simple.conf @@ -0,0 +1,11 @@ +$IncludeConfig diag-common.conf +template(name="outfmt" type="string" string="%$!msgnum%\n") + +module(load="../plugins/mmjsonparse/.libs/mmjsonparse") +module(load="../plugins/imptcp/.libs/imptcp") +input(type="imptcp" port="13514") + +action(type="mmjsonparse") +if $parsesuccess == "OK" then { + action(type="omfile" file="./rsyslog.out.log" template="outfmt") +} diff --git a/tests/testsuites/mmpstrucdata.conf b/tests/testsuites/mmpstrucdata.conf new file mode 100644 index 0000000..fd18fd9 --- /dev/null +++ b/tests/testsuites/mmpstrucdata.conf @@ -0,0 +1,12 @@ +$IncludeConfig diag-common.conf + +module(load="../plugins/mmpstrucdata/.libs/mmpstrucdata") +module(load="../plugins/imtcp/.libs/imtcp") + +template(name="outfmt" type="string" string="%$!rfc5424-sd!tcpflood@32473!msgnum%\n") + +input(type="imtcp" port="13514") + +action(type="mmpstrucdata") +if $msg contains "msgnum" then + action(type="omfile" template="outfmt" file="rsyslog.out.log") diff --git a/tests/testsuites/mysql-actq-mt.conf b/tests/testsuites/mysql-actq-mt.conf new file mode 100644 index 0000000..e12b77b --- /dev/null +++ b/tests/testsuites/mysql-actq-mt.conf @@ -0,0 +1,14 @@ +$IncludeConfig diag-common.conf + +module(load="../plugins/ommysql/.libs/ommysql") + +:msg, contains, "msgnum:" { + action(type="ommysql" server="127.0.0.1" + db="Syslog" uid="rsyslog" pwd="testbench" + queue.size="10000" queue.type="linkedList" + queue.workerthreads="5" + queue.workerthreadMinimumMessages="500" + queue.timeoutWorkerthreadShutdown="1000" + queue.timeoutEnqueue="10000" + ) +} diff --git a/tests/testsuites/mysql-asyn.conf b/tests/testsuites/mysql-asyn.conf index acdf9bb..44b151a 100644 --- a/tests/testsuites/mysql-asyn.conf +++ b/tests/testsuites/mysql-asyn.conf @@ -2,4 +2,5 @@ $IncludeConfig diag-common.conf $ModLoad ../plugins/ommysql/.libs/ommysql $ActionQueueType LinkedList +$ActionQueueTimeoutEnqueue 10000 # 10 second to make sure we do not loose due to action q full :msg, contains, "msgnum:" :ommysql:127.0.0.1,Syslog,rsyslog,testbench; diff --git a/tests/testsuites/rcvr_fail_restore_sender.conf b/tests/testsuites/rcvr_fail_restore_sender.conf index 6b11aa4..d5c34ef 100644 --- a/tests/testsuites/rcvr_fail_restore_sender.conf +++ b/tests/testsuites/rcvr_fail_restore_sender.conf @@ -5,11 +5,17 @@ $ModLoad ../plugins/imtcp/.libs/imtcp $InputTCPServerRun 13514 $WorkDirectory test-spool +$MainMsgQueueSize 2000 +$MainMsgQueueLowWaterMark 800 +$MainMsgQueueHighWaterMark 1000 +$MainMsgQueueDequeueBatchSize 1 $MainMsgQueueMaxFileSize 1g +$MainMsgQueueWorkerThreads 1 $MainMsgQueueFileName mainq # we use the shortest resume interval a) to let the test not run too long # and b) make sure some retries happen before the reconnect $ActionResumeInterval 1 +$ActionSendResendLastMsgOnReconnect on $ActionResumeRetryCount -1 *.* @@127.0.0.1:13515 diff --git a/tests/testsuites/rfc5424parser.conf b/tests/testsuites/rfc5424parser.conf new file mode 100644 index 0000000..cd90d12 --- /dev/null +++ b/tests/testsuites/rfc5424parser.conf @@ -0,0 +1,10 @@ +$IncludeConfig diag-common.conf + +module(load="../plugins/imtcp/.libs/imtcp") + +template(name="outfmt" type="string" string="%msg:F,58:2%\n") + +input(type="imtcp" port="13514") + +if $msg contains "msgnum" then + action(type="omfile" template="outfmt" file="rsyslog.out.log") diff --git a/tests/testsuites/rs_optimizer_pri.conf b/tests/testsuites/rs_optimizer_pri.conf new file mode 100644 index 0000000..9ff27dc --- /dev/null +++ b/tests/testsuites/rs_optimizer_pri.conf @@ -0,0 +1,8 @@ +$IncludeConfig diag-common.conf +template(name="outfmt" type="string" string="%msg:F,58:2%\n") + +module(load="../plugins/imtcp/.libs/imtcp") +input(type="imtcp" port="13514") + +if $syslogfacility-text == "local4" then + action(type="omfile" template="outfmt" file="rsyslog.out.log") diff --git a/tests/testsuites/rscript_eq.conf b/tests/testsuites/rscript_eq.conf new file mode 100644 index 0000000..0130aed --- /dev/null +++ b/tests/testsuites/rscript_eq.conf @@ -0,0 +1,14 @@ +$IncludeConfig diag-common.conf + +template(name="outfmt" type="list") { + property(name="$!usr!msgnum") + constant(value="\n") +} + +if $msg contains 'msgnum' then { + set $!usr!msgnum = field($msg, 58, 2); + if $!usr!msgnum == "00005000" or + $!usr!msgnum == "00005001" or + $!usr!msgnum == "00005002" then + action(type="omfile" file="./rsyslog.out.log" template="outfmt") +} diff --git a/tests/testsuites/rscript_ge.conf b/tests/testsuites/rscript_ge.conf new file mode 100644 index 0000000..c2ff86a --- /dev/null +++ b/tests/testsuites/rscript_ge.conf @@ -0,0 +1,13 @@ +$IncludeConfig diag-common.conf + +template(name="outfmt" type="list") { + property(name="$!usr!msgnum") + constant(value="\n") +} + +if $msg contains 'msgnum' then { + set $!usr!msgnum = field($msg, 58, 2); + if $!usr!msgnum >= "00005000" then + stop + action(type="omfile" file="./rsyslog.out.log" template="outfmt") +} diff --git a/tests/testsuites/rscript_gt.conf b/tests/testsuites/rscript_gt.conf new file mode 100644 index 0000000..9a651ad --- /dev/null +++ b/tests/testsuites/rscript_gt.conf @@ -0,0 +1,12 @@ +$IncludeConfig diag-common.conf + +template(name="outfmt" type="list") { + property(name="$!usr!msgnum") + constant(value="\n") +} + +if $msg contains 'msgnum' then { + set $!usr!msgnum = field($msg, 58, 2); + if $!usr!msgnum > "00004999" then + action(type="omfile" file="./rsyslog.out.log" template="outfmt") +} diff --git a/tests/testsuites/rscript_le.conf b/tests/testsuites/rscript_le.conf new file mode 100644 index 0000000..1963cc5 --- /dev/null +++ b/tests/testsuites/rscript_le.conf @@ -0,0 +1,12 @@ +$IncludeConfig diag-common.conf + +template(name="outfmt" type="list") { + property(name="$!usr!msgnum") + constant(value="\n") +} + +if $msg contains 'msgnum' then { + set $!usr!msgnum = field($msg, 58, 2); + if $!usr!msgnum <= "00005000" then + action(type="omfile" file="./rsyslog.out.log" template="outfmt") +} diff --git a/tests/testsuites/rscript_lt.conf b/tests/testsuites/rscript_lt.conf new file mode 100644 index 0000000..1f455ec --- /dev/null +++ b/tests/testsuites/rscript_lt.conf @@ -0,0 +1,12 @@ +$IncludeConfig diag-common.conf + +template(name="outfmt" type="list") { + property(name="$!usr!msgnum") + constant(value="\n") +} + +if $msg contains 'msgnum' then { + set $!usr!msgnum = field($msg, 58, 2); + if $!usr!msgnum < "00005000" then + action(type="omfile" file="./rsyslog.out.log" template="outfmt") +} diff --git a/tests/testsuites/rscript_ne.conf b/tests/testsuites/rscript_ne.conf new file mode 100644 index 0000000..43148ca --- /dev/null +++ b/tests/testsuites/rscript_ne.conf @@ -0,0 +1,18 @@ +$IncludeConfig diag-common.conf + +template(name="outfmt" type="list") { + property(name="$!usr!msgnum") + constant(value="\n") +} + +if $msg contains 'msgnum' then { + set $!usr!msgnum = field($msg, 58, 2); + if $!usr!msgnum != "00005000" and + $!usr!msgnum != "00005001" and + $!usr!msgnum != "00005002" then + set $!usr!write = 0; + else + set $!usr!write = 1; + if $!usr!write == 1 then + action(type="omfile" file="./rsyslog.out.log" template="outfmt") +} diff --git a/tests/testsuites/rulesetmultiqueue-v6.conf b/tests/testsuites/rulesetmultiqueue-v6.conf new file mode 100644 index 0000000..3aeaa33 --- /dev/null +++ b/tests/testsuites/rulesetmultiqueue-v6.conf @@ -0,0 +1,34 @@ +# Test for multiple ruleset queues (see .sh file for details) +# rgerhards, 2009-10-30 +$IncludeConfig diag-common.conf +$ModLoad ../plugins/imtcp/.libs/imtcp +$MainMsgQueueTimeoutShutdown 10000 + +# general definition +$template outfmt,"%msg:F,58:2%\n" + +# create the individual rulesets +$template dynfile1,"rsyslog.out1.log" # trick to use relative path names! +ruleset(name="file1" queue.type="linkedList") { + :msg, contains, "msgnum:" ?dynfile1;outfmt +} + +$template dynfile2,"rsyslog.out2.log" # trick to use relative path names! +ruleset(name="file2" queue.type="linkedList") { + :msg, contains, "msgnum:" ?dynfile2;outfmt +} + +$template dynfile3,"rsyslog.out3.log" # trick to use relative path names! +ruleset(name="file3" queue.type="linkedList") { + :msg, contains, "msgnum:" ?dynfile3;outfmt +} + +# start listeners and bind them to rulesets +$InputTCPServerBindRuleset file1 +$InputTCPServerRun 13514 + +$InputTCPServerBindRuleset file2 +$InputTCPServerRun 13515 + +$InputTCPServerBindRuleset file3 +$InputTCPServerRun 13516 diff --git a/tests/testsuites/samples.parse-nodate b/tests/testsuites/samples.parse-nodate index 5432bca..720bdda 100644 --- a/tests/testsuites/samples.parse-nodate +++ b/tests/testsuites/samples.parse-nodate @@ -1,5 +1,5 @@ <27>xapi: [error|xen3|15|Guest liveness monitor D:bca30ab3f1c1|master_connection] Connection to master died. I will continue to retry indefinitely (supressing future logging of this message) -27,daemon,err,localhost.localdomain,xapi,xapi:, [error|xen3|15|Guest liveness monitor D:bca30ab3f1c1|master_connection] Connection to master died. I will continue to retry indefinitely (supressing future logging of this message) +27,daemon,err,~H,xapi,xapi:, [error|xen3|15|Guest liveness monitor D:bca30ab3f1c1|master_connection] Connection to master died. I will continue to retry indefinitely (supressing future logging of this message) # a message with just text (as permitted by rfc 3164) # it is questionable if the current sample result is really correct as of 3164! This is a message! diff --git a/tests/testsuites/samples.snare_ccoff_udp b/tests/testsuites/samples.snare_ccoff_udp index 1ae7e8b..3a7f5e5 100644 --- a/tests/testsuites/samples.snare_ccoff_udp +++ b/tests/testsuites/samples.snare_ccoff_udp @@ -3,12 +3,12 @@ # *real* cases (just mangled to anonymize them...) # Sample 1 - note the absence of PRI! windowsserver MSWinEventLog 1 Security 1167 Fri Mar 19 15:33:30 2010 540 Security SYSTEM User Success Audit WINDOWSSERVER Logon/Logoff Successful Network Logon: User Name: WINDOWSSERVER$ Domain: DOMX Logon ID: (0x0,0xF88396) Logon Type: 3 Logon Process: Kerberos Authentication Package: Kerberos Workstation Name: Logon GUID: {79b6eb79-7bcc-8a2e-7dad-953c51dc00fd} Caller User Name: - Caller Domain: - Caller Logon ID: - Caller Process ID: - Transited Services: - Source Network Address: 10.11.11.3 Source Port: 3306 733\n -13,user,notice,localhost.localdomain,windowsserver,windowsserver MSWinEventLog 1 Security 1167 Fri, Mar 19 15:33:30 2010 540 Security SYSTEM User Success Audit WINDOWSSERVER Logon/Logoff Successful Network Logon: User Name: WINDOWSSERVER$ Domain: DOMX Logon ID: (0x0,0xF88396) Logon Type: 3 Logon Process: Kerberos Authentication Package: Kerberos Workstation Name: Logon GUID: {79b6eb79-7bcc-8a2e-7dad-953c51dc00fd} Caller User Name: - Caller Domain: - Caller Logon ID: - Caller Process ID: - Transited Services: - Source Network Address: 10.11.11.3 Source Port: 3306 733 +13,user,notice,~H,windowsserver,windowsserver MSWinEventLog 1 Security 1167 Fri, Mar 19 15:33:30 2010 540 Security SYSTEM User Success Audit WINDOWSSERVER Logon/Logoff Successful Network Logon: User Name: WINDOWSSERVER$ Domain: DOMX Logon ID: (0x0,0xF88396) Logon Type: 3 Logon Process: Kerberos Authentication Package: Kerberos Workstation Name: Logon GUID: {79b6eb79-7bcc-8a2e-7dad-953c51dc00fd} Caller User Name: - Caller Domain: - Caller Logon ID: - Caller Process ID: - Transited Services: - Source Network Address: 10.11.11.3 Source Port: 3306 733 # Sample 2 # the samples below need to be disabled for the "workaround patch" for the message # parser to work. They need to be re-enabled once a final solution has been crafted #windowsserver MSWinEventLog 1 Security 1166 Fri Mar 19 15:33:30 2010 576 Security SYSTEM User Success Audit WINDOWSSERVER Logon/Logoff Special privileges assigned to new logon: User Name: WINDOWSSERVER$ Domain: DOMX Logon ID: (0x0,0xF88396) Privileges: SeSecurityPrivilege SeBackupPrivilege SeRestorePrivilege SeTakeOwnershipPrivilege SeDebugPrivilege SeSystemEnvironmentPrivilege SeLoadDriverPrivilege SeImpersonatePrivilege SeEnableDelegationPrivilege 732\n -#13,user,notice,localhost,windowsserver,windowsserver MSWinEventLog 1 Security 1166 Fri, Mar 19 15:33:30 2010 576 Security SYSTEM User Success Audit WINDOWSSERVER Logon/Logoff Special privileges assigned to new logon: User Name: WINDOWSSERVER$ Domain: DOMX Logon ID: (0x0,0xF88396) Privileges: SeSecurityPrivilege SeBackupPrivilege SeRestorePrivilege SeTakeOwnershipPrivilege SeDebugPrivilege SeSystemEnvironmentPrivilege SeLoadDriverPrivilege SeImpersonatePrivilege SeEnableDelegationPrivilege 732 +#13,user,notice,~H,windowsserver,windowsserver MSWinEventLog 1 Security 1166 Fri, Mar 19 15:33:30 2010 576 Security SYSTEM User Success Audit WINDOWSSERVER Logon/Logoff Special privileges assigned to new logon: User Name: WINDOWSSERVER$ Domain: DOMX Logon ID: (0x0,0xF88396) Privileges: SeSecurityPrivilege SeBackupPrivilege SeRestorePrivilege SeTakeOwnershipPrivilege SeDebugPrivilege SeSystemEnvironmentPrivilege SeLoadDriverPrivilege SeImpersonatePrivilege SeEnableDelegationPrivilege 732 # Sample 3 #windowsserver MSWinEventLog 1 Security 1165 Fri Mar 19 15:33:30 2010 538 Security SYSTEM User Success Audit WINDOWSSERVER Logon/Logoff User Logoff: User Name: WINDOWSSERVER$ Domain: DOMX Logon ID: (0x0,0xF8830B) Logon Type: 3 731\n -#13,user,notice,localhost,windowsserver,windowsserver MSWinEventLog 1 Security 1165 Fri, Mar 19 15:33:30 2010 538 Security SYSTEM User Success Audit WINDOWSSERVER Logon/Logoff User Logoff: User Name: WINDOWSSERVER$ Domain: DOMX Logon ID: (0x0,0xF8830B) Logon Type: 3 731 +#13,user,notice,~H,windowsserver,windowsserver MSWinEventLog 1 Security 1165 Fri, Mar 19 15:33:30 2010 538 Security SYSTEM User Success Audit WINDOWSSERVER Logon/Logoff User Logoff: User Name: WINDOWSSERVER$ Domain: DOMX Logon ID: (0x0,0xF8830B) Logon Type: 3 731 diff --git a/tests/testsuites/samples.snare_ccoff_udp2 b/tests/testsuites/samples.snare_ccoff_udp2 index da3a2b1..05233e7 100644 --- a/tests/testsuites/samples.snare_ccoff_udp2 +++ b/tests/testsuites/samples.snare_ccoff_udp2 @@ -14,13 +14,13 @@ test insert into windows (Message, Facility,FromHost, Priority, DeviceReportedTime, ReceivedAt, InfoUnitID, SysLogTag) values ('', 1, 'test',5, '20100321185328', '20100321185328', 1, '') # and yet another one we have seen in practice UX=Abcd-efg-hij-klmno; XXXXX=1111111111, Z123=192.12.231.245:11111, S1234=123456789, XXXXXX=111111111 -insert into windows (Message, Facility,FromHost, Priority, DeviceReportedTime, ReceivedAt, InfoUnitID, SysLogTag) values (' XXXXX=1111111111, Z123=192.12.231.245:11111, S1234=123456789, XXXXXX=111111111', 1, 'localhost.localdomain',5, '20100321185328', '20100321185328', 1, 'UX=Abcd-efg-hij-klmno;') +insert into windows (Message, Facility,FromHost, Priority, DeviceReportedTime, ReceivedAt, InfoUnitID, SysLogTag) values (' XXXXX=1111111111, Z123=192.12.231.245:11111, S1234=123456789, XXXXXX=111111111', 1, '~H',5, '20100321185328', '20100321185328', 1, 'UX=Abcd-efg-hij-klmno;') # Sample 1 - note the absence of PRI! windowsserver MSWinEventLog 1 Security 1167 Fri Mar 19 15:33:30 2010 540 Security SYSTEM User Success Audit WINDOWSSERVER Logon/Logoff Successful Network Logon: User Name: WINDOWSSERVER$ Domain: DOMX Logon ID: (0x0,0xF88396) Logon Type: 3 Logon Process: Kerberos Authentication Package: Kerberos Workstation Name: Logon GUID: {79b6eb79-7bcc-8a2e-7dad-953c51dc00fd} Caller User Name: - Caller Domain: - Caller Logon ID: - Caller Process ID: - Transited Services: - Source Network Address: 10.11.11.3 Source Port: 3306 733\n -insert into windows (Message, Facility,FromHost, Priority, DeviceReportedTime, ReceivedAt, InfoUnitID, SysLogTag) values (' Mar 19 15:33:30 2010 540 Security SYSTEM User Success Audit WINDOWSSERVER Logon/Logoff Successful Network Logon: User Name: WINDOWSSERVER$ Domain: DOMX Logon ID: (0x0,0xF88396) Logon Type: 3 Logon Process: Kerberos Authentication Package: Kerberos Workstation Name: Logon GUID: {79b6eb79-7bcc-8a2e-7dad-953c51dc00fd} Caller User Name: - Caller Domain: - Caller Logon ID: - Caller Process ID: - Transited Services: - Source Network Address: 10.11.11.3 Source Port: 3306 733', 1, 'localhost.localdomain',5, '20100321185328', '20100321185328', 1, 'windowsserver MSWinEventLog 1 Security 1167 Fri') +insert into windows (Message, Facility,FromHost, Priority, DeviceReportedTime, ReceivedAt, InfoUnitID, SysLogTag) values (' Mar 19 15:33:30 2010 540 Security SYSTEM User Success Audit WINDOWSSERVER Logon/Logoff Successful Network Logon: User Name: WINDOWSSERVER$ Domain: DOMX Logon ID: (0x0,0xF88396) Logon Type: 3 Logon Process: Kerberos Authentication Package: Kerberos Workstation Name: Logon GUID: {79b6eb79-7bcc-8a2e-7dad-953c51dc00fd} Caller User Name: - Caller Domain: - Caller Logon ID: - Caller Process ID: - Transited Services: - Source Network Address: 10.11.11.3 Source Port: 3306 733', 1, '~H',5, '20100321185328', '20100321185328', 1, 'windowsserver MSWinEventLog 1 Security 1167 Fri') # Sample 2 windowsserver MSWinEventLog 1 Security 1166 Fri Mar 19 15:33:30 2010 576 Security SYSTEM User Success Audit WINDOWSSERVER Logon/Logoff Special privileges assigned to new logon: User Name: WINDOWSSERVER$ Domain: DOMX Logon ID: (0x0,0xF88396) Privileges: SeSecurityPrivilege SeBackupPrivilege SeRestorePrivilege SeTakeOwnershipPrivilege SeDebugPrivilege SeSystemEnvironmentPrivilege SeLoadDriverPrivilege SeImpersonatePrivilege SeEnableDelegationPrivilege 732\n -insert into windows (Message, Facility,FromHost, Priority, DeviceReportedTime, ReceivedAt, InfoUnitID, SysLogTag) values (' Mar 19 15:33:30 2010 576 Security SYSTEM User Success Audit WINDOWSSERVER Logon/Logoff Special privileges assigned to new logon: User Name: WINDOWSSERVER$ Domain: DOMX Logon ID: (0x0,0xF88396) Privileges: SeSecurityPrivilege SeBackupPrivilege SeRestorePrivilege SeTakeOwnershipPrivilege SeDebugPrivilege SeSystemEnvironmentPrivilege SeLoadDriverPrivilege SeImpersonatePrivilege SeEnableDelegationPrivilege 732', 1, 'localhost.localdomain',5, '20100321185328', '20100321185328', 1, 'windowsserver MSWinEventLog 1 Security 1166 Fri') +insert into windows (Message, Facility,FromHost, Priority, DeviceReportedTime, ReceivedAt, InfoUnitID, SysLogTag) values (' Mar 19 15:33:30 2010 576 Security SYSTEM User Success Audit WINDOWSSERVER Logon/Logoff Special privileges assigned to new logon: User Name: WINDOWSSERVER$ Domain: DOMX Logon ID: (0x0,0xF88396) Privileges: SeSecurityPrivilege SeBackupPrivilege SeRestorePrivilege SeTakeOwnershipPrivilege SeDebugPrivilege SeSystemEnvironmentPrivilege SeLoadDriverPrivilege SeImpersonatePrivilege SeEnableDelegationPrivilege 732', 1, '~H',5, '20100321185328', '20100321185328', 1, 'windowsserver MSWinEventLog 1 Security 1166 Fri') # Sample 3 windowsserver MSWinEventLog 1 Security 1165 Fri Mar 19 15:33:30 2010 538 Security SYSTEM User Success Audit WINDOWSSERVER Logon/Logoff User Logoff: User Name: WINDOWSSERVER$ Domain: DOMX Logon ID: (0x0,0xF8830B) Logon Type: 3 731\n -insert into windows (Message, Facility,FromHost, Priority, DeviceReportedTime, ReceivedAt, InfoUnitID, SysLogTag) values (' Mar 19 15:33:30 2010 538 Security SYSTEM User Success Audit WINDOWSSERVER Logon/Logoff User Logoff: User Name: WINDOWSSERVER$ Domain: DOMX Logon ID: (0x0,0xF8830B) Logon Type: 3 731', 1, 'localhost.localdomain',5, '20100321185328', '20100321185328', 1, 'windowsserver MSWinEventLog 1 Security 1165 Fri') +insert into windows (Message, Facility,FromHost, Priority, DeviceReportedTime, ReceivedAt, InfoUnitID, SysLogTag) values (' Mar 19 15:33:30 2010 538 Security SYSTEM User Success Audit WINDOWSSERVER Logon/Logoff User Logoff: User Name: WINDOWSSERVER$ Domain: DOMX Logon ID: (0x0,0xF8830B) Logon Type: 3 731', 1, '~H',5, '20100321185328', '20100321185328', 1, 'windowsserver MSWinEventLog 1 Security 1165 Fri') diff --git a/tests/testsuites/sndrcv_relp_rcvr.conf b/tests/testsuites/sndrcv_relp_rcvr.conf new file mode 100644 index 0000000..d79f714 --- /dev/null +++ b/tests/testsuites/sndrcv_relp_rcvr.conf @@ -0,0 +1,9 @@ +# rgerhards, 2013-12-10 +$IncludeConfig diag-common.conf + +module(load="../plugins/imrelp/.libs/imrelp") +# then SENDER sends to this port (not tcpflood!) +input(type="imrelp" port="13515") + +$template outfmt,"%msg:F,58:2%\n" +:msg, contains, "msgnum:" action(type="omfile" file="rsyslog.out.log" template="outfmt") diff --git a/tests/testsuites/sndrcv_relp_sender.conf b/tests/testsuites/sndrcv_relp_sender.conf new file mode 100644 index 0000000..7706622 --- /dev/null +++ b/tests/testsuites/sndrcv_relp_sender.conf @@ -0,0 +1,8 @@ +# rgerhards, 2013-12-10 +$IncludeConfig diag-common2.conf + +module(load="../plugins/omrelp/.libs/omrelp") +module(load="../plugins/imptcp/.libs/imptcp") +input(type="imptcp" port="13514") /* this port for tcpflood! */ + +action(type="omrelp" protocol="tcp" target="127.0.0.1" port="13515") diff --git a/tests/testsuites/stop-localvar.conf b/tests/testsuites/stop-localvar.conf new file mode 100644 index 0000000..63df650 --- /dev/null +++ b/tests/testsuites/stop-localvar.conf @@ -0,0 +1,21 @@ +/* note: variables are strings (at least in v7), so we need to convert + * to a number when we check the conditon. + * Even if we change the variable representation at some later point, + * we should NOT change this test here, but better add a new one. + * rgerhards, 2013-11-19 + */ +$IncludeConfig diag-common.conf +template(name="outfmt" type="string" string="%$.nbr%\n") + +module(load="../plugins/imtcp/.libs/imtcp") +input(type="imtcp" port="13514") + +if $msg contains "msgnum:" then { + set $.nbr = field($msg, 58, 2); + if cnum($.nbr) < 100 then + stop + /* check is intentionally more complex than needed! */ + else if not (cnum($.nbr) > 999) then { + action(type="omfile" file="rsyslog.out.log" template="outfmt") + } +} diff --git a/tests/testsuites/stop-msgvar.conf b/tests/testsuites/stop-msgvar.conf new file mode 100644 index 0000000..020ebd8 --- /dev/null +++ b/tests/testsuites/stop-msgvar.conf @@ -0,0 +1,21 @@ +/* note: variables are strings (at least in v7), so we need to convert + * to a number when we check the conditon. + * Even if we change the variable representation at some later point, + * we should NOT change this test here, but better add a new one. + * rgerhards, 2013-11-19 + */ +$IncludeConfig diag-common.conf +template(name="outfmt" type="string" string="%$!nbr%\n") + +module(load="../plugins/imtcp/.libs/imtcp") +input(type="imtcp" port="13514") + +if $msg contains "msgnum:" then { + set $!nbr = field($msg, 58, 2); + if cnum($!nbr) < 100 then + stop + /* check is intentionally more complex than needed! */ + else if not (cnum($!nbr) > 999) then { + action(type="omfile" file="rsyslog.out.log" template="outfmt") + } +} diff --git a/tests/testsuites/stop.conf b/tests/testsuites/stop.conf new file mode 100644 index 0000000..84beab8 --- /dev/null +++ b/tests/testsuites/stop.conf @@ -0,0 +1,11 @@ +$IncludeConfig diag-common.conf + +module(load="../plugins/imtcp/.libs/imtcp") +input(type="imtcp" port="13514") + +if $msg contains "00000001" then + stop + +template(name="outfmt" type="string" string="%msg:F,58:2%\n") +if $msg contains "msgnum:" then + action(type="omfile" file="rsyslog.out.log" template="outfmt") diff --git a/tests/testsuites/weird.parse1 b/tests/testsuites/weird.parse1 index 907198a..c24d983 100644 --- a/tests/testsuites/weird.parse1 +++ b/tests/testsuites/weird.parse1 @@ -11,10 +11,10 @@ 14,user,info,Aug 30 23:00:05,X4711,,, # there is a SP at the end of the line <14>Aug 30 23:00:05 -14,user,info,Aug 30 23:00:05,localhost.localdomain,,, +14,user,info,Aug 30 23:00:05,~H,,, # and here is no SP at the end of the line <14>Aug 30 23:00:05 -14,user,info,Aug 30 23:00:05,localhost.localdomain,,, +14,user,info,Aug 30 23:00:05,~H,,, # unfortunately, I can not test missing dates with this test suite, because # we would have the current date in the response, which we can not check against # @@ -31,7 +31,7 @@ 14,user,info,Aug 30 23:00:05,X4711,,, # there is a SP at the end of the line <14>2010-08-30T23:00:05Z -14,user,info,Aug 30 23:00:05,localhost.localdomain,,, +14,user,info,Aug 30 23:00:05,~H,,, # and here is no SP at the end of the line <14>2010-08-30T23:00:05Z -14,user,info,Aug 30 23:00:05,localhost.localdomain,,, +14,user,info,Aug 30 23:00:05,~H,,, diff --git a/tests/timestamp.sh b/tests/timestamp.sh index 71416c3..e18a98b 100755 --- a/tests/timestamp.sh +++ b/tests/timestamp.sh @@ -1,5 +1,6 @@ echo \[timestamp.sh\]: various timestamp tests source $srcdir/diag.sh init +source $srcdir/diag.sh generate-HOSTNAME source $srcdir/diag.sh nettester ts3164 udp source $srcdir/diag.sh nettester ts3164 tcp source $srcdir/diag.sh nettester ts3339 udp diff --git a/tests/validation-run.sh b/tests/validation-run.sh index a68ee8a..2250e12 100755 --- a/tests/validation-run.sh +++ b/tests/validation-run.sh @@ -23,19 +23,19 @@ #set -x echo \[validation-run.sh\]: testing configuraton validation echo "testing a failed configuration verification run" -../tools/rsyslogd -dn -u2 -c4 -N1 -f$srcdir/testsuites/invalid.conf -M../runtime/.libs:../.libs +../tools/rsyslogd -u2 -N1 -f$srcdir/testsuites/invalid.conf -M../runtime/.libs:../.libs if [ $? -ne 1 ]; then echo "after test 1: return code ne 1" exit 1 fi echo testing a valid config verification run -../tools/rsyslogd -u2 -c4 -N1 -f$srcdir/testsuites/valid.conf -M../runtime/.libs:../.libs +../tools/rsyslogd -u2 -N1 -f$srcdir/testsuites/valid.conf -M../runtime/.libs:../.libs if [ $? -ne 0 ]; then echo "after test 2: return code ne 0" exit 1 fi echo testing empty config file -../tools/rsyslogd -u2 -c4 -N1 -f/dev/null -M../runtime/.libs:../.libs +../tools/rsyslogd -u2 -N1 -f/dev/null -M../runtime/.libs:../.libs if [ $? -ne 1 ]; then echo "after test 3: return code ne 1" exit 1 diff --git a/tools/Makefile.am b/tools/Makefile.am index 9a1497c..639dbf2 100644 --- a/tools/Makefile.am +++ b/tools/Makefile.am @@ -7,6 +7,7 @@ man_MANS = rsyslogd.8 rsyslog.conf.5 sbin_PROGRAMS += rsyslogd rsyslogd_SOURCES = \ syslogd.c \ + rsyslogd.c \ syslogd.h \ omshell.c \ omshell.h \ @@ -42,7 +43,7 @@ rsyslogd_CPPFLAGS = $(PTHREADS_CFLAGS) $(RSRT_CFLAGS) # note: it looks like librsyslog.la must be explicitely given on LDDADD, # otherwise dependencies are not properly calculated (resulting in a # potentially incomplete build, a problem we had several times...) -rsyslogd_LDADD = ../grammar/libgrammar.la ../runtime/librsyslog.la $(ZLIB_LIBS) $(PTHREADS_LIBS) $(RSRT_LIBS) $(SOL_LIBS) $(LIBEE_LIBS) $(LIBLOGNORM_LIBS) $(LIBUUID_LIBS) $(LIBGCRYPT_LIBS) +rsyslogd_LDADD = ../grammar/libgrammar.la ../runtime/librsyslog.la $(ZLIB_LIBS) $(PTHREADS_LIBS) $(RSRT_LIBS) $(SOL_LIBS) $(LIBUUID_LIBS) $(LIBLOGGING_STDLOG_LIBS) rsyslogd_LDFLAGS = -export-dynamic EXTRA_DIST = $(man_MANS) \ diff --git a/tools/Makefile.in b/tools/Makefile.in index f335183..bc87b45 100644 --- a/tools/Makefile.in +++ b/tools/Makefile.in @@ -96,19 +96,19 @@ am__rsyslog_diag_hostname_SOURCES_DIST = gethostn.c rsyslog_diag_hostname_OBJECTS = $(am_rsyslog_diag_hostname_OBJECTS) rsyslog_diag_hostname_LDADD = $(LDADD) am_rsyslogd_OBJECTS = rsyslogd-syslogd.$(OBJEXT) \ - rsyslogd-omshell.$(OBJEXT) rsyslogd-omusrmsg.$(OBJEXT) \ - rsyslogd-omfwd.$(OBJEXT) rsyslogd-omfile.$(OBJEXT) \ - rsyslogd-ompipe.$(OBJEXT) rsyslogd-omdiscard.$(OBJEXT) \ - rsyslogd-pmrfc5424.$(OBJEXT) rsyslogd-pmrfc3164.$(OBJEXT) \ - rsyslogd-smtradfile.$(OBJEXT) rsyslogd-smfile.$(OBJEXT) \ - rsyslogd-smfwd.$(OBJEXT) rsyslogd-smtradfwd.$(OBJEXT) \ - rsyslogd-iminternal.$(OBJEXT) rsyslogd-pidfile.$(OBJEXT) + rsyslogd-rsyslogd.$(OBJEXT) rsyslogd-omshell.$(OBJEXT) \ + rsyslogd-omusrmsg.$(OBJEXT) rsyslogd-omfwd.$(OBJEXT) \ + rsyslogd-omfile.$(OBJEXT) rsyslogd-ompipe.$(OBJEXT) \ + rsyslogd-omdiscard.$(OBJEXT) rsyslogd-pmrfc5424.$(OBJEXT) \ + rsyslogd-pmrfc3164.$(OBJEXT) rsyslogd-smtradfile.$(OBJEXT) \ + rsyslogd-smfile.$(OBJEXT) rsyslogd-smfwd.$(OBJEXT) \ + rsyslogd-smtradfwd.$(OBJEXT) rsyslogd-iminternal.$(OBJEXT) \ + rsyslogd-pidfile.$(OBJEXT) rsyslogd_OBJECTS = $(am_rsyslogd_OBJECTS) rsyslogd_DEPENDENCIES = ../grammar/libgrammar.la \ ../runtime/librsyslog.la $(am__DEPENDENCIES_1) \ $(am__DEPENDENCIES_1) $(am__DEPENDENCIES_1) \ $(am__DEPENDENCIES_1) $(am__DEPENDENCIES_1) \ - $(am__DEPENDENCIES_1) $(am__DEPENDENCIES_1) \ $(am__DEPENDENCIES_1) rsyslogd_LINK = $(LIBTOOL) $(AM_V_lt) --tag=CC $(AM_LIBTOOLFLAGS) \ $(LIBTOOLFLAGS) --mode=link $(CCLD) $(AM_CFLAGS) $(CFLAGS) \ @@ -221,7 +221,6 @@ GREP = @GREP@ GSS_LIBS = @GSS_LIBS@ GUARDTIME_CFLAGS = @GUARDTIME_CFLAGS@ GUARDTIME_LIBS = @GUARDTIME_LIBS@ -HAVE_LIBGCRYPT_CONFIG = @HAVE_LIBGCRYPT_CONFIG@ HAVE_MYSQL_CONFIG = @HAVE_MYSQL_CONFIG@ HAVE_ORACLE_CONFIG = @HAVE_ORACLE_CONFIG@ HAVE_PGSQL_CONFIG = @HAVE_PGSQL_CONFIG@ @@ -242,14 +241,15 @@ LEXLIB = @LEXLIB@ LEX_OUTPUT_ROOT = @LEX_OUTPUT_ROOT@ LIBDBI_CFLAGS = @LIBDBI_CFLAGS@ LIBDBI_LIBS = @LIBDBI_LIBS@ -LIBEE_CFLAGS = @LIBEE_CFLAGS@ -LIBEE_LIBS = @LIBEE_LIBS@ LIBESTR_CFLAGS = @LIBESTR_CFLAGS@ LIBESTR_LIBS = @LIBESTR_LIBS@ LIBGCRYPT_CFLAGS = @LIBGCRYPT_CFLAGS@ +LIBGCRYPT_CONFIG = @LIBGCRYPT_CONFIG@ LIBGCRYPT_LIBS = @LIBGCRYPT_LIBS@ LIBLOGGING_CFLAGS = @LIBLOGGING_CFLAGS@ LIBLOGGING_LIBS = @LIBLOGGING_LIBS@ +LIBLOGGING_STDLOG_CFLAGS = @LIBLOGGING_STDLOG_CFLAGS@ +LIBLOGGING_STDLOG_LIBS = @LIBLOGGING_STDLOG_LIBS@ LIBLOGNORM_CFLAGS = @LIBLOGNORM_CFLAGS@ LIBLOGNORM_LIBS = @LIBLOGNORM_LIBS@ LIBM = @LIBM@ @@ -274,6 +274,8 @@ NM = @NM@ NMEDIT = @NMEDIT@ OBJDUMP = @OBJDUMP@ OBJEXT = @OBJEXT@ +OPENSSL_CFLAGS = @OPENSSL_CFLAGS@ +OPENSSL_LIBS = @OPENSSL_LIBS@ ORACLE_CFLAGS = @ORACLE_CFLAGS@ ORACLE_LIBS = @ORACLE_LIBS@ OTOOL = @OTOOL@ @@ -377,6 +379,7 @@ man1_MANS = $(am__append_4) $(am__append_8) man_MANS = rsyslogd.8 rsyslog.conf.5 rsyslogd_SOURCES = \ syslogd.c \ + rsyslogd.c \ syslogd.h \ omshell.c \ omshell.h \ @@ -413,7 +416,7 @@ rsyslogd_CPPFLAGS = $(PTHREADS_CFLAGS) $(RSRT_CFLAGS) # note: it looks like librsyslog.la must be explicitely given on LDDADD, # otherwise dependencies are not properly calculated (resulting in a # potentially incomplete build, a problem we had several times...) -rsyslogd_LDADD = ../grammar/libgrammar.la ../runtime/librsyslog.la $(ZLIB_LIBS) $(PTHREADS_LIBS) $(RSRT_LIBS) $(SOL_LIBS) $(LIBEE_LIBS) $(LIBLOGNORM_LIBS) $(LIBUUID_LIBS) $(LIBGCRYPT_LIBS) +rsyslogd_LDADD = ../grammar/libgrammar.la ../runtime/librsyslog.la $(ZLIB_LIBS) $(PTHREADS_LIBS) $(RSRT_LIBS) $(SOL_LIBS) $(LIBUUID_LIBS) $(LIBLOGGING_STDLOG_LIBS) rsyslogd_LDFLAGS = -export-dynamic EXTRA_DIST = $(man_MANS) rsgtutil.rst rscryutil.rst recover_qi.pl \ $(am__append_6) $(am__append_10) @@ -593,6 +596,7 @@ distclean-compile: @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/rsyslogd-pidfile.Po@am__quote@ @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/rsyslogd-pmrfc3164.Po@am__quote@ @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/rsyslogd-pmrfc5424.Po@am__quote@ +@AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/rsyslogd-rsyslogd.Po@am__quote@ @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/rsyslogd-smfile.Po@am__quote@ @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/rsyslogd-smfwd.Po@am__quote@ @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/rsyslogd-smtradfile.Po@am__quote@ @@ -601,22 +605,25 @@ distclean-compile: @AMDEP_TRUE@@am__include@ @am__quote@./$(DEPDIR)/zpipe.Po@am__quote@ .c.o: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(COMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ $< -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Po +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.o$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c $< +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ $< .c.obj: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(COMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ `$(CYGPATH_W) '$<'` -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Po +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.obj$$||'`;\ +@am__fastdepCC_TRUE@ $(COMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ `$(CYGPATH_W) '$<'` &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Po @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=no @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ -@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c `$(CYGPATH_W) '$<'` +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(COMPILE) -c -o $@ `$(CYGPATH_W) '$<'` .c.lo: -@am__fastdepCC_TRUE@ $(AM_V_CC)$(LTCOMPILE) -MT $@ -MD -MP -MF $(DEPDIR)/$*.Tpo -c -o $@ $< -@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/$*.Tpo $(DEPDIR)/$*.Plo +@am__fastdepCC_TRUE@ $(AM_V_CC)depbase=`echo $@ | sed 's|[^/]*$$|$(DEPDIR)/&|;s|\.lo$$||'`;\ +@am__fastdepCC_TRUE@ $(LTCOMPILE) -MT $@ -MD -MP -MF $$depbase.Tpo -c -o $@ $< &&\ +@am__fastdepCC_TRUE@ $(am__mv) $$depbase.Tpo $$depbase.Plo @AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='$<' object='$@' libtool=yes @AMDEPBACKSLASH@ @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ @am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(LTCOMPILE) -c -o $@ $< @@ -677,6 +684,20 @@ rsyslogd-syslogd.obj: syslogd.c @AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ @am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(rsyslogd_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o rsyslogd-syslogd.obj `if test -f 'syslogd.c'; then $(CYGPATH_W) 'syslogd.c'; else $(CYGPATH_W) '$(srcdir)/syslogd.c'; fi` +rsyslogd-rsyslogd.o: rsyslogd.c +@am__fastdepCC_TRUE@ $(AM_V_CC)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(rsyslogd_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT rsyslogd-rsyslogd.o -MD -MP -MF $(DEPDIR)/rsyslogd-rsyslogd.Tpo -c -o rsyslogd-rsyslogd.o `test -f 'rsyslogd.c' || echo '$(srcdir)/'`rsyslogd.c +@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/rsyslogd-rsyslogd.Tpo $(DEPDIR)/rsyslogd-rsyslogd.Po +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='rsyslogd.c' object='rsyslogd-rsyslogd.o' libtool=no @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(rsyslogd_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o rsyslogd-rsyslogd.o `test -f 'rsyslogd.c' || echo '$(srcdir)/'`rsyslogd.c + +rsyslogd-rsyslogd.obj: rsyslogd.c +@am__fastdepCC_TRUE@ $(AM_V_CC)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(rsyslogd_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT rsyslogd-rsyslogd.obj -MD -MP -MF $(DEPDIR)/rsyslogd-rsyslogd.Tpo -c -o rsyslogd-rsyslogd.obj `if test -f 'rsyslogd.c'; then $(CYGPATH_W) 'rsyslogd.c'; else $(CYGPATH_W) '$(srcdir)/rsyslogd.c'; fi` +@am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/rsyslogd-rsyslogd.Tpo $(DEPDIR)/rsyslogd-rsyslogd.Po +@AMDEP_TRUE@@am__fastdepCC_FALSE@ $(AM_V_CC)source='rsyslogd.c' object='rsyslogd-rsyslogd.obj' libtool=no @AMDEPBACKSLASH@ +@AMDEP_TRUE@@am__fastdepCC_FALSE@ DEPDIR=$(DEPDIR) $(CCDEPMODE) $(depcomp) @AMDEPBACKSLASH@ +@am__fastdepCC_FALSE@ $(AM_V_CC@am__nodep@)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(rsyslogd_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -c -o rsyslogd-rsyslogd.obj `if test -f 'rsyslogd.c'; then $(CYGPATH_W) 'rsyslogd.c'; else $(CYGPATH_W) '$(srcdir)/rsyslogd.c'; fi` + rsyslogd-omshell.o: omshell.c @am__fastdepCC_TRUE@ $(AM_V_CC)$(CC) $(DEFS) $(DEFAULT_INCLUDES) $(INCLUDES) $(rsyslogd_CPPFLAGS) $(CPPFLAGS) $(AM_CFLAGS) $(CFLAGS) -MT rsyslogd-omshell.o -MD -MP -MF $(DEPDIR)/rsyslogd-omshell.Tpo -c -o rsyslogd-omshell.o `test -f 'omshell.c' || echo '$(srcdir)/'`omshell.c @am__fastdepCC_TRUE@ $(AM_V_at)$(am__mv) $(DEPDIR)/rsyslogd-omshell.Tpo $(DEPDIR)/rsyslogd-omshell.Po diff --git a/tools/gethostn.c b/tools/gethostn.c index df7ce38..be7f678 100644 --- a/tools/gethostn.c +++ b/tools/gethostn.c @@ -8,20 +8,19 @@ * * This file is part of rsyslog. * - * Rsyslog is free software: you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation, either version 3 of the License, or - * (at your option) any later version. - * - * Rsyslog is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * - * You should have received a copy of the GNU General Public License - * along with Rsyslog. If not, see <http://www.gnu.org/licenses/>. - * - * A copy of the GPL can be found in the file "COPYING" in this distribution. + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * -or- + * see COPYING.ASL20 in the source distribution + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. */ #include "config.h" diff --git a/tools/iminternal.c b/tools/iminternal.c index 167e2b2..ebfd8f5 100644 --- a/tools/iminternal.c +++ b/tools/iminternal.c @@ -10,20 +10,19 @@ * * This file is part of rsyslog. * - * Rsyslog is free software: you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation, either version 3 of the License, or - * (at your option) any later version. - * - * Rsyslog is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * - * You should have received a copy of the GNU General Public License - * along with Rsyslog. If not, see <http://www.gnu.org/licenses/>. - * - * A copy of the GPL can be found in the file "COPYING" in this distribution. + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * -or- + * see COPYING.ASL20 in the source distribution + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. */ #include "config.h" #include "rsyslog.h" diff --git a/tools/iminternal.h b/tools/iminternal.h index 8a9e250..fac48df 100644 --- a/tools/iminternal.h +++ b/tools/iminternal.h @@ -8,20 +8,19 @@ * * This file is part of rsyslog. * - * Rsyslog is free software: you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation, either version 3 of the License, or - * (at your option) any later version. - * - * Rsyslog is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * - * You should have received a copy of the GNU General Public License - * along with Rsyslog. If not, see <http://www.gnu.org/licenses/>. - * - * A copy of the GPL can be found in the file "COPYING" in this distribution. + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * -or- + * see COPYING.ASL20 in the source distribution + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. */ #ifndef IMINTERNAL_H_INCLUDED diff --git a/tools/logctl.c b/tools/logctl.c index 1ab8ead..b583c68 100644 --- a/tools/logctl.c +++ b/tools/logctl.c @@ -26,20 +26,19 @@ * * This file is part of rsyslog. * - * Rsyslog is free software: you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation, either version 3 of the License, or - * (at your option) any later version. - * - * Rsyslog is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * - * You should have received a copy of the GNU General Public License - * along with Rsyslog. If not, see <http://www.gnu.org/licenses/>. - * - * A copy of the GPL can be found in the file "COPYING" in this distribution. + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * -or- + * see COPYING.ASL20 in the source distribution + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. */ #include "config.h" #include <stdio.h> diff --git a/tools/omdiscard.c b/tools/omdiscard.c index 15c6ea8..a76bcc3 100644 --- a/tools/omdiscard.c +++ b/tools/omdiscard.c @@ -6,7 +6,7 @@ * * File begun on 2007-07-24 by RGerhards * - * Copyright 2007-2012 Adiscon GmbH. + * Copyright 2007-2013 Adiscon GmbH. * * This file is part of rsyslog. * @@ -49,6 +49,10 @@ typedef struct _instanceData { EMPTY_STRUCT } instanceData; +typedef struct wrkrInstanceData { + instanceData *pData; +} wrkrInstanceData_t; + /* we do not need a createInstance()! BEGINcreateInstance CODESTARTcreateInstance @@ -56,6 +60,11 @@ ENDcreateInstance */ +BEGINcreateWrkrInstance +CODESTARTcreateWrkrInstance +ENDcreateWrkrInstance + + BEGINdbgPrintInstInfo CODESTARTdbgPrintInstInfo /* do nothing */ @@ -87,6 +96,11 @@ CODESTARTfreeInstance ENDfreeInstance +BEGINfreeWrkrInstance +CODESTARTfreeWrkrInstance +ENDfreeWrkrInstance + + BEGINparseSelectorAct CODESTARTparseSelectorAct CODE_STD_STRING_REQUESTparseSelectorAct(0) @@ -114,6 +128,7 @@ ENDmodExit BEGINqueryEtryPt CODESTARTqueryEtryPt CODEqueryEtryPt_STD_OMOD_QUERIES +CODEqueryEtryPt_STD_OMOD8_QUERIES ENDqueryEtryPt diff --git a/tools/omfile.c b/tools/omfile.c index ba9f7f7..39c0173 100644 --- a/tools/omfile.c +++ b/tools/omfile.c @@ -17,7 +17,7 @@ * pipes. These have been moved to ompipe, to reduced the entanglement * between the two different functionalities. -- rgerhards * - * Copyright 2007-2013 Adiscon GmbH. + * Copyright 2007-2014 Adiscon GmbH. * * This file is part of rsyslog. * @@ -71,6 +71,7 @@ #include "statsobj.h" #include "sigprov.h" #include "cryprov.h" +#include "janitor.h" MODULE_TYPE_OUTPUT MODULE_TYPE_NOKEEP @@ -95,7 +96,7 @@ DEFobjCurrIf(statsobj) * That should be sufficient (and even than, there would no really bad effect ;)). * The variable below is the global counter/clock. */ -#if HAVE_ATOMIC_BUILTINS_64BIT +#if HAVE_ATOMIC_BUILTINS64 static uint64 clockFileAccess = 0; #else static unsigned clockFileAccess = 0; @@ -107,7 +108,7 @@ static pthread_mutex_t mutClock; static inline uint64 getClockFileAccess(void) { -#if HAVE_ATOMIC_BUILTINS_64BIT +#if HAVE_ATOMIC_BUILTINS64 return ATOMIC_INC_AND_FETCH_uint64(&clockFileAccess, &mutClock); #else return ATOMIC_INC_AND_FETCH_unsigned(&clockFileAccess, &mutClock); @@ -122,6 +123,7 @@ struct s_dynaFileCacheEntry { strm_t *pStrm; /* our output stream */ void *sigprovFileData; /* opaque data ptr for provider use */ uint64 clkTickAccessed;/* for LRU - based on clockFileAccess */ + short nInactive; /* number of minutes not writen - for close timeout */ }; typedef struct s_dynaFileCacheEntry dynaFileCacheEntry; @@ -133,14 +135,17 @@ typedef struct s_dynaFileCacheEntry dynaFileCacheEntry; typedef struct _instanceData { - uchar *f_fname; /* file or template name (display only) */ + pthread_mutex_t mutWrite; /* guard against multiple instances writing to single file */ + uchar *fname; /* file or template name (display only) */ uchar *tplName; /* name of assigned template */ strm_t *pStrm; /* our output stream */ + short nInactive; /* number of minutes not writen (STATIC files only) */ char bDynamicName; /* 0 - static name, 1 - dynamic name (with properties) */ int fCreateMode; /* file creation mode for open() */ int fDirCreateMode; /* creation mode for mkdir() */ int bCreateDirs; /* auto-create directories? */ int bSyncFile; /* should the file by sync()'ed? 1- yes, 0- no */ + uint8_t iNumTpls; /* number of tpls we use */ uid_t fileUID; /* IDs for creation */ uid_t dirUID; gid_t fileGID; @@ -155,7 +160,6 @@ typedef struct _instanceData { uchar *cryprovName; /* crypto provider */ uchar *cryprovNameFull;/* full internal crypto provider name */ void *cryprovData; /* opaque data ptr for provider use */ - void *cryprovFileData;/* opaque data ptr for file instance */ cryprov_if_t cryprov; /* ptr to crypto provider interface */ sbool useCryprov; /* quicker than checkig ptr (1 vs 8 bytes!) */ int iCurrElt; /* currently active cache element (-1 = none) */ @@ -171,6 +175,7 @@ typedef struct _instanceData { int iZipLevel; /* zip mode to use for this selector */ int iIOBufSize; /* size of associated io buffer */ int iFlushInterval; /* how fast flush buffer on inactivity? */ + short iCloseTimeout; /* after how many *minutes* shall the file be closed if inactive? */ sbool bFlushOnTXEnd; /* flush write buffers when transaction has ended? */ sbool bUseAsyncWriter; /* use async stream writer? */ sbool bVeryRobustZip; @@ -180,9 +185,16 @@ typedef struct _instanceData { STATSCOUNTER_DEF(ctrEvict, mutCtrEvict); STATSCOUNTER_DEF(ctrMiss, mutCtrMiss); STATSCOUNTER_DEF(ctrMax, mutCtrMax); + STATSCOUNTER_DEF(ctrCloseTimeouts, mutCtrCloseTimeouts); + char janitorID[128]; /* holds ID for janitor calls */ } instanceData; +typedef struct wrkrInstanceData { + instanceData *pData; +} wrkrInstanceData_t; + + typedef struct configSettings_s { int iDynaFileCacheSize; /* max cache for dynamic files */ int fCreateMode; /* mode to use when creating files */ @@ -207,6 +219,12 @@ uchar *pszFileDfltTplName; /* name of the default template to use */ struct modConfData_s { rsconf_t *pConf; /* our overall config object */ uchar *tplName; /* default template */ + int fCreateMode; /* default mode to use when creating files */ + int fDirCreateMode; /* default mode to use when creating files */ + uid_t fileUID; /* default IDs for creation */ + uid_t dirUID; + gid_t fileGID; + gid_t dirGID; }; static modConfData_t *loadModConf = NULL;/* modConf ptr to use for the current load process */ @@ -216,6 +234,16 @@ static modConfData_t *runModConf = NULL;/* modConf ptr to use for the current ex /* module-global parameters */ static struct cnfparamdescr modpdescr[] = { { "template", eCmdHdlrGetWord, 0 }, + { "dircreatemode", eCmdHdlrFileCreateMode, 0 }, + { "filecreatemode", eCmdHdlrFileCreateMode, 0 }, + { "dirowner", eCmdHdlrUID, 0 }, + { "dirownernum", eCmdHdlrInt, 0 }, + { "dirgroup", eCmdHdlrGID, 0 }, + { "dirgroupnum", eCmdHdlrInt, 0 }, + { "fileowner", eCmdHdlrUID, 0 }, + { "fileownernum", eCmdHdlrInt, 0 }, + { "filegroup", eCmdHdlrGID, 0 }, + { "filegroupnum", eCmdHdlrInt, 0 }, }; static struct cnfparamblk modpblk = { CNFPARAMBLK_VERSION, @@ -233,9 +261,13 @@ static struct cnfparamdescr actpdescr[] = { { "flushontxend", eCmdHdlrBinary, 0 }, /* legacy: omfileflushontxend */ { "iobuffersize", eCmdHdlrSize, 0 }, /* legacy: omfileiobuffersize */ { "dirowner", eCmdHdlrUID, 0 }, /* legacy: dirowner */ + { "dirownernum", eCmdHdlrInt, 0 }, /* legacy: dirownernum */ { "dirgroup", eCmdHdlrGID, 0 }, /* legacy: dirgroup */ + { "dirgroupnum", eCmdHdlrInt, 0 }, /* legacy: dirgroupnum */ { "fileowner", eCmdHdlrUID, 0 }, /* legacy: fileowner */ + { "fileownernum", eCmdHdlrInt, 0 }, /* legacy: fileownernum */ { "filegroup", eCmdHdlrGID, 0 }, /* legacy: filegroup */ + { "filegroupnum", eCmdHdlrInt, 0 }, /* legacy: filegroupnum */ { "dircreatemode", eCmdHdlrFileCreateMode, 0 }, /* legacy: dircreatemode */ { "filecreatemode", eCmdHdlrFileCreateMode, 0 }, /* legacy: filecreatemode */ { "failonchownfailure", eCmdHdlrBinary, 0 }, /* legacy: failonchownfailure */ @@ -245,6 +277,7 @@ static struct cnfparamdescr actpdescr[] = { { "dynafile", eCmdHdlrString, 0 }, /* "dynafile" MUST be present */ { "sig.provider", eCmdHdlrGetWord, 0 }, { "cry.provider", eCmdHdlrGetWord, 0 }, + { "closetimeout", eCmdHdlrPositiveInt, 0 }, { "template", eCmdHdlrGetWord, 0 } }; static struct cnfparamblk actpblk = @@ -287,11 +320,11 @@ CODESTARTdbgPrintInstInfo if(pData->bDynamicName) { dbgprintf("[dynamic]\n"); } else { /* regular file */ - dbgprintf("%s%s\n", pData->f_fname, + dbgprintf("%s%s\n", pData->fname, (pData->pStrm == NULL) ? " (closed)" : ""); } - dbgprintf("\ttemplate='%s'\n", pData->f_fname); + dbgprintf("\ttemplate='%s'\n", pData->fname); dbgprintf("\tuse async writer=%d\n", pData->bUseAsyncWriter); dbgprintf("\tflush on TX end=%d\n", pData->bFlushOnTXEnd); dbgprintf("\tflush interval=%d\n", pData->iFlushInterval); @@ -411,7 +444,7 @@ static rsRetVal cflineParseOutchannel(instanceData *pData, uchar* p, omodStringR } /* OK, we finally got a correct template. So let's use it... */ - pData->f_fname = ustrdup(pOch->pszFileTemplate); + pData->fname = ustrdup(pOch->pszFileTemplate); pData->iSizeLimit = pOch->uSizeLimit; /* WARNING: It is dangerous "just" to pass the pointer. As we * never rebuild the output channel description, this is acceptable here. @@ -433,7 +466,7 @@ finalize_it: * if the entry should be d_free()ed and 0 if not. */ static rsRetVal -dynaFileDelCacheEntry(instanceData *pData, int iEntry, int bFreeEntry) +dynaFileDelCacheEntry(instanceData *__restrict__ const pData, const int iEntry, const int bFreeEntry) { dynaFileCacheEntry **pCache = pData->dynCache; DEFiRet; @@ -473,7 +506,7 @@ finalize_it: * rgerhards, 2008-10-23 */ static inline void -dynaFileFreeCacheEntries(instanceData *pData) +dynaFileFreeCacheEntries(instanceData *__restrict__ const pData) { register int i; ASSERT(pData != NULL); @@ -489,7 +522,7 @@ dynaFileFreeCacheEntries(instanceData *pData) /* This function frees the dynamic file name cache. */ -static void dynaFileFreeCache(instanceData *pData) +static void dynaFileFreeCache(instanceData *__restrict__ const pData) { ASSERT(pData != NULL); @@ -503,7 +536,7 @@ static void dynaFileFreeCache(instanceData *pData) /* close current file */ static rsRetVal -closeFile(instanceData *pData) +closeFile(instanceData *__restrict__ const pData) { DEFiRet; if(pData->useSigprov) { @@ -517,7 +550,7 @@ closeFile(instanceData *pData) /* This prepares the signature provider to process a file */ static rsRetVal -sigprovPrepare(instanceData *pData, uchar *fn) +sigprovPrepare(instanceData *__restrict__ const pData, uchar *__restrict__ const fn) { DEFiRet; pData->sigprov.OnFileOpen(pData->sigprovData, fn, &pData->sigprovFileData); @@ -531,7 +564,7 @@ sigprovPrepare(instanceData *pData, uchar *fn) * changed to iRet interface - 2009-03-19 */ static rsRetVal -prepareFile(instanceData *pData, uchar *newFileName) +prepareFile(instanceData *__restrict__ const pData, const uchar *__restrict__ const newFileName) { int fd; DEFiRet; @@ -633,7 +666,7 @@ finalize_it: * This is a helper to writeFile(). rgerhards, 2007-07-03 */ static inline rsRetVal -prepareDynFile(instanceData *pData, uchar *newFileName, unsigned iMsgOpts) +prepareDynFile(instanceData *__restrict__ const pData, const uchar *__restrict__ const newFileName) { uint64 ctOldest; /* "timestamp" of oldest element */ int iOldest; @@ -727,16 +760,10 @@ prepareDynFile(instanceData *pData, uchar *newFileName, unsigned iMsgOpts) /* check if we had an error */ if(localRet != RS_RET_OK) { - /* do not report anything if the message is an internally-generated - * message. Otherwise, we could run into a never-ending loop. The bad - * news is that we also lose errors on startup messages, but so it is. + /* We do no longer care about internal messages. The errmsg rate limiter + * will take care of too-frequent error messages. */ - if(iMsgOpts & INTERNAL_MSG) { - DBGPRINTF("Could not open dynaFile '%s', state %d, discarding message\n", - newFileName, localRet); - } else { - errmsg.LogError(0, localRet, "Could not open dynamic file '%s' [state %d] - discarding message", newFileName, localRet); - } + errmsg.LogError(0, localRet, "Could not open dynamic file '%s' [state %d] - discarding message", newFileName, localRet); ABORT_FINALIZE(localRet); } @@ -752,6 +779,7 @@ prepareDynFile(instanceData *pData, uchar *newFileName, unsigned iMsgOpts) DBGPRINTF("Added new entry %d for file cache, file '%s'.\n", iFirstFree, newFileName); finalize_it: + pCache[pData->iCurrElt]->nInactive = 0; RETiRet; } @@ -762,13 +790,14 @@ finalize_it: * rgerhards, 2009-06-03 */ static rsRetVal -doWrite(instanceData *pData, uchar *pszBuf, int lenBuf) +doWrite(instanceData *__restrict__ const pData, uchar *__restrict__ const pszBuf, const int lenBuf) { DEFiRet; ASSERT(pData != NULL); ASSERT(pszBuf != NULL); - DBGPRINTF("write to stream, pData->pStrm %p, lenBuf %d\n", pData->pStrm, lenBuf); + DBGPRINTF("omfile: write to stream, pData->pStrm %p, lenBuf %d, strt data %.128s\n", + pData->pStrm, lenBuf, pszBuf); if(pData->pStrm != NULL){ CHKiRet(strm.Write(pData->pStrm, pszBuf, lenBuf)); if(pData->useSigprov) { @@ -783,27 +812,34 @@ finalize_it: /* rgerhards 2004-11-11: write to a file output. */ static rsRetVal -writeFile(uchar **ppString, unsigned iMsgOpts, instanceData *pData) +writeFile(instanceData *__restrict__ const pData, + const actWrkrIParams_t *__restrict__ const pParam, + const int iMsg) { DEFiRet; - ASSERT(pData != NULL); - + STATSCOUNTER_INC(pData->ctrRequests, pData->mutCtrRequests); /* first check if we have a dynamic file name and, if so, * check if it still is ok or a new file needs to be created */ if(pData->bDynamicName) { - CHKiRet(prepareDynFile(pData, ppString[1], iMsgOpts)); + DBGPRINTF("omfile: file to log to: %s\n", + actParam(pParam, pData->iNumTpls, iMsg, 1).param); + CHKiRet(prepareDynFile(pData, actParam(pParam, pData->iNumTpls, iMsg, 1).param)); } else { /* "regular", non-dynafile */ if(pData->pStrm == NULL) { - CHKiRet(prepareFile(pData, pData->f_fname)); + CHKiRet(prepareFile(pData, pData->fname)); if(pData->pStrm == NULL) { - errmsg.LogError(0, RS_RET_NO_FILE_ACCESS, "Could no open output file '%s'", pData->f_fname); + errmsg.LogError(0, RS_RET_NO_FILE_ACCESS, + "Could not open output file '%s'", pData->fname); } } + pData->nInactive = 0; } - CHKiRet(doWrite(pData, ppString[0], strlen(CHAR_CONVERT(ppString[0])))); + CHKiRet(doWrite(pData, + actParam(pParam, pData->iNumTpls, iMsg, 0).param, + actParam(pParam, pData->iNumTpls, iMsg, 0).lenStr)); finalize_it: RETiRet; @@ -815,6 +851,12 @@ CODESTARTbeginCnfLoad loadModConf = pModConf; pModConf->pConf = pConf; pModConf->tplName = NULL; + pModConf->fCreateMode = 0644; + pModConf->fDirCreateMode = 0700; + pModConf->fileUID = -1; + pModConf->dirUID = -1; + pModConf->fileGID = -1; + pModConf->dirGID = -1; ENDbeginCnfLoad BEGINsetModCnf @@ -834,8 +876,10 @@ CODESTARTsetModCnf } for(i = 0 ; i < modpblk.nParams ; ++i) { - if(!pvals[i].bUsed) + if(!pvals[i].bUsed) { continue; + } + if(!strcmp(modpblk.descr[i].name, "template")) { loadModConf->tplName = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); if(pszFileDfltTplName != NULL) { @@ -843,6 +887,26 @@ CODESTARTsetModCnf "was already set via legacy directive - may lead to inconsistent " "results."); } + } else if(!strcmp(modpblk.descr[i].name, "dircreatemode")) { + loadModConf->fDirCreateMode = (int) pvals[i].val.d.n; + } else if(!strcmp(modpblk.descr[i].name, "filecreatemode")) { + loadModConf->fCreateMode = (int) pvals[i].val.d.n; + } else if(!strcmp(modpblk.descr[i].name, "dirowner")) { + loadModConf->dirUID = (int) pvals[i].val.d.n; + } else if(!strcmp(modpblk.descr[i].name, "dirownernum")) { + loadModConf->dirUID = (int) pvals[i].val.d.n; + } else if(!strcmp(modpblk.descr[i].name, "dirgroup")) { + loadModConf->dirGID = (int) pvals[i].val.d.n; + } else if(!strcmp(modpblk.descr[i].name, "dirgroupnum")) { + loadModConf->dirGID = (int) pvals[i].val.d.n; + } else if(!strcmp(modpblk.descr[i].name, "fileowner")) { + loadModConf->fileUID = (int) pvals[i].val.d.n; + } else if(!strcmp(modpblk.descr[i].name, "fileownernum")) { + loadModConf->fileUID = (int) pvals[i].val.d.n; + } else if(!strcmp(modpblk.descr[i].name, "filegroup")) { + loadModConf->fileGID = (int) pvals[i].val.d.n; + } else if(!strcmp(modpblk.descr[i].name, "filegroupnum")) { + loadModConf->fileGID = (int) pvals[i].val.d.n; } else { dbgprintf("omfile: program error, non-handled " "param '%s' in beginCnfLoad\n", modpblk.descr[i].name); @@ -853,6 +917,54 @@ finalize_it: cnfparamvalsDestruct(pvals, &modpblk); ENDsetModCnf +/* This function checks dynafile cache for janitor action */ +static inline void +janitorChkDynaFiles(instanceData *__restrict__ const pData) +{ + int i; + dynaFileCacheEntry **pCache = pData->dynCache; + + for(i = 0 ; i < pData->iCurrCacheSize ; ++i) { + if(pCache[i] == NULL) + continue; + DBGPRINTF("omfile janitor: checking dynafile %d:%s, inactive since %d\n", i, + pCache[i]->pName == NULL ? UCHAR_CONSTANT("[OPEN FAILED]") : pCache[i]->pName, + (int) pCache[i]->nInactive); + if(pCache[i]->nInactive >= pData->iCloseTimeout) { + STATSCOUNTER_INC(pData->ctrCloseTimeouts, pData->mutCtrCloseTimeouts); + dynaFileDelCacheEntry(pData, i, 1); + if(pData->iCurrElt == i) + pData->iCurrElt = -1; /* no longer available! */ + } else { + pCache[i]->nInactive += janitorInterval; + } + } +} + +/* callback for the janitor. This cleans out files (if so configured) */ +void +janitorCB(void *pUsr) +{ + instanceData *__restrict__ const pData = (instanceData *) pUsr; + pthread_mutex_lock(&pData->mutWrite); + if(pData->bDynamicName) { + janitorChkDynaFiles(pData); + } else { + if(pData->pStrm != NULL) { + DBGPRINTF("omfile janitor: checking file %s, inactive since %d\n", + pData->fname, pData->nInactive); + if(pData->nInactive >= pData->iCloseTimeout) { + STATSCOUNTER_INC(pData->ctrCloseTimeouts, pData->mutCtrCloseTimeouts); + closeFile(pData); + } else { + pData->nInactive += janitorInterval; + } + } + } + pthread_mutex_unlock(&pData->mutWrite); +} + + BEGINendCnfLoad CODESTARTendCnfLoad loadModConf = NULL; /* done loading */ @@ -879,13 +991,21 @@ ENDfreeCnf BEGINcreateInstance CODESTARTcreateInstance pData->pStrm = NULL; + pthread_mutex_init(&pData->mutWrite, NULL); ENDcreateInstance +BEGINcreateWrkrInstance +CODESTARTcreateWrkrInstance +ENDcreateWrkrInstance + + BEGINfreeInstance CODESTARTfreeInstance free(pData->tplName); - free(pData->f_fname); + free(pData->fname); + if(pData->iCloseTimeout > 0) + janitorDelEtry(pData->janitorID); if(pData->bDynamicName) { dynaFileFreeCache(pData); } else if(pData->pStrm != NULL) @@ -904,9 +1024,15 @@ CODESTARTfreeInstance free(pData->cryprovName); free(pData->cryprovNameFull); } + pthread_mutex_destroy(&pData->mutWrite); ENDfreeInstance +BEGINfreeWrkrInstance +CODESTARTfreeWrkrInstance +ENDfreeWrkrInstance + + BEGINtryResume CODESTARTtryResume ENDtryResume @@ -917,8 +1043,15 @@ CODESTARTbeginTransaction ENDbeginTransaction -BEGINendTransaction -CODESTARTendTransaction +BEGINcommitTransaction + instanceData *__restrict__ const pData = pWrkrData->pData; + unsigned i; +CODESTARTcommitTransaction + pthread_mutex_lock(&pData->mutWrite); + + for(i = 0 ; i < nParams ; ++i) { + writeFile(pData, pParams, i); + } /* Note: pStrm may be NULL if there was an error opening the stream */ if(pData->bFlushOnTXEnd && pData->pStrm != NULL) { /* if we have an async writer, it controls the flush via @@ -929,38 +1062,23 @@ CODESTARTendTransaction CHKiRet(strm.Flush(pData->pStrm)); } finalize_it: -ENDendTransaction - - -BEGINdoAction -CODESTARTdoAction - DBGPRINTF("file to log to: %s\n", - (pData->bDynamicName) ? ppString[1] : pData->f_fname); - DBGPRINTF("omfile: start of data: '%.128s'\n", ppString[0]); - STATSCOUNTER_INC(pData->ctrRequests, pData->mutCtrRequests); - CHKiRet(writeFile(ppString, iMsgOpts, pData)); - if(!bCoreSupportsBatching && pData->bFlushOnTXEnd) { - CHKiRet(strm.Flush(pData->pStrm)); - } -finalize_it: - if(iRet == RS_RET_OK) - iRet = RS_RET_DEFER_COMMIT; -ENDdoAction + pthread_mutex_unlock(&pData->mutWrite); +ENDcommitTransaction static inline void -setInstParamDefaults(instanceData *pData) +setInstParamDefaults(instanceData *__restrict__ const pData) { - pData->f_fname = NULL; + pData->fname = NULL; pData->tplName = NULL; - pData->fileUID = -1; - pData->fileGID = -1; - pData->dirUID = -1; - pData->dirGID = -1; + pData->fileUID = loadModConf->fileUID; + pData->fileGID = loadModConf->fileGID; + pData->dirUID = loadModConf->dirUID; + pData->dirGID = loadModConf->dirGID; pData->bFailOnChown = 1; pData->iDynaFileCacheSize = 10; - pData->fCreateMode = 0644; - pData->fDirCreateMode = 0700; + pData->fCreateMode = loadModConf->fCreateMode; + pData->fDirCreateMode = loadModConf->fDirCreateMode; pData->bCreateDirs = 1; pData->bSyncFile = 0; pData->iZipLevel = 0; @@ -973,11 +1091,12 @@ setInstParamDefaults(instanceData *pData) pData->cryprovName = NULL; pData->useSigprov = 0; pData->useCryprov = 0; + pData->iCloseTimeout = -1; } static rsRetVal -setupInstStatsCtrs(instanceData *pData) +setupInstStatsCtrs(instanceData *__restrict__ const pData) { uchar ctrName[512]; DEFiRet; @@ -987,25 +1106,28 @@ setupInstStatsCtrs(instanceData *pData) } /* support statistics gathering */ - snprintf((char*)ctrName, sizeof(ctrName), "dynafile cache %s", pData->f_fname); + snprintf((char*)ctrName, sizeof(ctrName), "dynafile cache %s", pData->fname); ctrName[sizeof(ctrName)-1] = '\0'; /* be on the save side */ CHKiRet(statsobj.Construct(&(pData->stats))); CHKiRet(statsobj.SetName(pData->stats, ctrName)); STATSCOUNTER_INIT(pData->ctrRequests, pData->mutCtrRequests); CHKiRet(statsobj.AddCounter(pData->stats, UCHAR_CONSTANT("requests"), - ctrType_IntCtr, &(pData->ctrRequests))); + ctrType_IntCtr, CTR_FLAG_RESETTABLE, &(pData->ctrRequests))); STATSCOUNTER_INIT(pData->ctrLevel0, pData->mutCtrLevel0); CHKiRet(statsobj.AddCounter(pData->stats, UCHAR_CONSTANT("level0"), - ctrType_IntCtr, &(pData->ctrLevel0))); + ctrType_IntCtr, CTR_FLAG_RESETTABLE, &(pData->ctrLevel0))); STATSCOUNTER_INIT(pData->ctrMiss, pData->mutCtrMiss); CHKiRet(statsobj.AddCounter(pData->stats, UCHAR_CONSTANT("missed"), - ctrType_IntCtr, &(pData->ctrMiss))); + ctrType_IntCtr, CTR_FLAG_RESETTABLE, &(pData->ctrMiss))); STATSCOUNTER_INIT(pData->ctrEvict, pData->mutCtrEvict); CHKiRet(statsobj.AddCounter(pData->stats, UCHAR_CONSTANT("evicted"), - ctrType_IntCtr, &(pData->ctrEvict))); + ctrType_IntCtr, CTR_FLAG_RESETTABLE, &(pData->ctrEvict))); STATSCOUNTER_INIT(pData->ctrMax, pData->mutCtrMax); CHKiRet(statsobj.AddCounter(pData->stats, UCHAR_CONSTANT("maxused"), - ctrType_IntCtr, &(pData->ctrMax))); + ctrType_IntCtr, CTR_FLAG_RESETTABLE, &(pData->ctrMax))); + STATSCOUNTER_INIT(pData->ctrCloseTimeouts, pData->mutCtrCloseTimeouts); + CHKiRet(statsobj.AddCounter(pData->stats, UCHAR_CONSTANT("closetimeouts"), + ctrType_IntCtr, CTR_FLAG_RESETTABLE, &(pData->ctrCloseTimeouts))); CHKiRet(statsobj.ConstructFinalize(pData->stats)); finalize_it: @@ -1013,7 +1135,7 @@ finalize_it: } static inline void -initSigprov(instanceData *pData, struct nvlst *lst) +initSigprov(instanceData *__restrict__ const pData, struct nvlst *lst) { uchar szDrvrName[1024]; @@ -1055,7 +1177,7 @@ done: return; } static inline rsRetVal -initCryprov(instanceData *pData, struct nvlst *lst) +initCryprov(instanceData *__restrict__ const pData, struct nvlst *lst) { uchar szDrvrName[1024]; DEFiRet; @@ -1089,7 +1211,7 @@ initCryprov(instanceData *pData, struct nvlst *lst) szDrvrName); ABORT_FINALIZE(RS_RET_CRYPROV_ERR); } - CHKiRet(pData->cryprov.SetCnfParam(pData->cryprovData, lst)); + CHKiRet(pData->cryprov.SetCnfParam(pData->cryprovData, lst, CRYPROV_PARAMTYPE_REGULAR)); dbgprintf("loaded crypto provider %s, data instance at %p\n", szDrvrName, pData->cryprovData); @@ -1139,12 +1261,20 @@ CODESTARTnewActInst pData->iIOBufSize = (int) pvals[i].val.d.n; } else if(!strcmp(actpblk.descr[i].name, "dirowner")) { pData->dirUID = (int) pvals[i].val.d.n; + } else if(!strcmp(actpblk.descr[i].name, "dirownernum")) { + pData->dirUID = (int) pvals[i].val.d.n; } else if(!strcmp(actpblk.descr[i].name, "dirgroup")) { pData->dirGID = (int) pvals[i].val.d.n; + } else if(!strcmp(actpblk.descr[i].name, "dirgroupnum")) { + pData->dirGID = (int) pvals[i].val.d.n; } else if(!strcmp(actpblk.descr[i].name, "fileowner")) { pData->fileUID = (int) pvals[i].val.d.n; + } else if(!strcmp(actpblk.descr[i].name, "fileownernum")) { + pData->fileUID = (int) pvals[i].val.d.n; } else if(!strcmp(actpblk.descr[i].name, "filegroup")) { pData->fileGID = (int) pvals[i].val.d.n; + } else if(!strcmp(actpblk.descr[i].name, "filegroupnum")) { + pData->fileGID = (int) pvals[i].val.d.n; } else if(!strcmp(actpblk.descr[i].name, "dircreatemode")) { pData->fDirCreateMode = (int) pvals[i].val.d.n; } else if(!strcmp(actpblk.descr[i].name, "filecreatemode")) { @@ -1156,11 +1286,11 @@ CODESTARTnewActInst } else if(!strcmp(actpblk.descr[i].name, "createdirs")) { pData->bCreateDirs = (int) pvals[i].val.d.n; } else if(!strcmp(actpblk.descr[i].name, "file")) { - pData->f_fname = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + pData->fname = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); CODE_STD_STRING_REQUESTnewActInst(1) pData->bDynamicName = 0; } else if(!strcmp(actpblk.descr[i].name, "dynafile")) { - pData->f_fname = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + pData->fname = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); CODE_STD_STRING_REQUESTnewActInst(2) pData->bDynamicName = 1; } else if(!strcmp(actpblk.descr[i].name, "template")) { @@ -1169,13 +1299,15 @@ CODESTARTnewActInst pData->sigprovName = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); } else if(!strcmp(actpblk.descr[i].name, "cry.provider")) { pData->cryprovName = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + } else if(!strcmp(actpblk.descr[i].name, "closetimeout")) { + pData->iCloseTimeout = (int) pvals[i].val.d.n; } else { dbgprintf("omfile: program error, non-handled " "param '%s'\n", actpblk.descr[i].name); } } - if(pData->f_fname == NULL) { + if(pData->fname == NULL) { errmsg.LogError(0, RS_RET_MISSING_CNFPARAMS, "omfile: either the \"file\" or " "\"dynfile\" parameter must be given"); ABORT_FINALIZE(RS_RET_MISSING_CNFPARAMS); @@ -1191,12 +1323,14 @@ CODESTARTnewActInst tplToUse = ustrdup((pData->tplName == NULL) ? getDfltTpl() : pData->tplName); CHKiRet(OMSRsetEntry(*ppOMSR, 0, tplToUse, OMSR_NO_RQD_TPL_OPTS)); + pData->iNumTpls = 1; if(pData->bDynamicName) { /* "filename" is actually a template name, we need this as string 1. So let's add it * to the pOMSR. -- rgerhards, 2007-07-27 */ - CHKiRet(OMSRsetEntry(*ppOMSR, 1, ustrdup(pData->f_fname), OMSR_NO_RQD_TPL_OPTS)); + CHKiRet(OMSRsetEntry(*ppOMSR, 1, ustrdup(pData->fname), OMSR_NO_RQD_TPL_OPTS)); + pData->iNumTpls = 2; // TODO: create unified code for this (legacy+v6 system) /* we now allocate the cache table */ CHKmalloc(pData->dynCache = (dynaFileCacheEntry**) @@ -1206,6 +1340,17 @@ CODESTARTnewActInst // TODO: add pData->iSizeLimit = 0; /* default value, use outchannels to configure! */ setupInstStatsCtrs(pData); + if(pData->iCloseTimeout == -1) { /* unset? */ + pData->iCloseTimeout = (pData->bDynamicName) ? 10 : 0; + } + + snprintf(pData->janitorID, sizeof(pData->janitorID), "omfile:%sfile:%s:%p", + (pData->bDynamicName) ? "dyna" : "", pData->fname, pData); + pData->janitorID[sizeof(pData->janitorID)-1] = '\0'; /* just in case... */ + + if(pData->iCloseTimeout > 0) + janitorAddEtry(janitorCB, pData->janitorID, pData); + CODE_STD_FINALIZERnewActInst cnfparamvalsDestruct(pvals, &actpblk); ENDnewActInst @@ -1238,6 +1383,7 @@ CODESTARTparseSelectorAct switch(*p) { case '$': CODE_STD_STRING_REQUESTparseSelectorAct(1) + pData->iNumTpls = 1; /* rgerhards 2005-06-21: this is a special setting for output-channel * definitions. In the long term, this setting will probably replace * anything else, but for the time being we must co-exist with the @@ -1253,15 +1399,16 @@ CODESTARTparseSelectorAct * a template name. rgerhards, 2007-07-03 */ CODE_STD_STRING_REQUESTparseSelectorAct(2) + pData->iNumTpls = 2; ++p; /* eat '?' */ CHKiRet(cflineParseFileName(p, fname, *ppOMSR, 0, OMSR_NO_RQD_TPL_OPTS, getDfltTpl())); - pData->f_fname = ustrdup(fname); + pData->fname = ustrdup(fname); pData->bDynamicName = 1; pData->iCurrElt = -1; /* no current element */ /* "filename" is actually a template name, we need this as string 1. So let's add it * to the pOMSR. -- rgerhards, 2007-07-27 */ - CHKiRet(OMSRsetEntry(*ppOMSR, 1, ustrdup(pData->f_fname), OMSR_NO_RQD_TPL_OPTS)); + CHKiRet(OMSRsetEntry(*ppOMSR, 1, ustrdup(pData->fname), OMSR_NO_RQD_TPL_OPTS)); /* we now allocate the cache table */ CHKmalloc(pData->dynCache = (dynaFileCacheEntry**) calloc(cs.iDynaFileCacheSize, sizeof(dynaFileCacheEntry*))); @@ -1270,8 +1417,9 @@ CODESTARTparseSelectorAct case '/': case '.': CODE_STD_STRING_REQUESTparseSelectorAct(1) + pData->iNumTpls = 1; CHKiRet(cflineParseFileName(p, fname, *ppOMSR, 0, OMSR_NO_RQD_TPL_OPTS, getDfltTpl())); - pData->f_fname = ustrdup(fname); + pData->fname = ustrdup(fname); pData->bDynamicName = 0; break; default: @@ -1294,6 +1442,7 @@ CODESTARTparseSelectorAct pData->iFlushInterval = cs.iFlushInterval; pData->bUseAsyncWriter = cs.bUseAsyncWriter; pData->bVeryRobustZip = 0; /* cannot be specified via legacy conf */ + pData->iCloseTimeout = 0; /* cannot be specified via legacy conf */ setupInstStatsCtrs(pData); CODE_STD_FINALIZERparseSelectorAct ENDparseSelectorAct @@ -1327,6 +1476,7 @@ static rsRetVal resetConfigVariables(uchar __attribute__((unused)) *pp, void __a BEGINdoHUP CODESTARTdoHUP + pthread_mutex_lock(&pData->mutWrite); if(pData->bDynamicName) { dynaFileFreeCacheEntries(pData); } else { @@ -1334,6 +1484,7 @@ CODESTARTdoHUP closeFile(pData); } } + pthread_mutex_unlock(&pData->mutWrite); ENDdoHUP @@ -1348,11 +1499,11 @@ ENDmodExit BEGINqueryEtryPt CODESTARTqueryEtryPt -CODEqueryEtryPt_STD_OMOD_QUERIES +CODEqueryEtryPt_STD_OMODTX_QUERIES +CODEqueryEtryPt_STD_OMOD8_QUERIES CODEqueryEtryPt_STD_CONF2_QUERIES CODEqueryEtryPt_STD_CONF2_setModCnf_QUERIES CODEqueryEtryPt_STD_CONF2_OMOD_QUERIES -CODEqueryEtryPt_TXIF_OMOD_QUERIES /* we support the transactional interface! */ CODEqueryEtryPt_doHUP ENDqueryEtryPt @@ -1377,9 +1528,13 @@ INITLegCnfVars CHKiRet(omsdRegCFSLineHdlr((uchar *)"omfileflushontxend", 0, eCmdHdlrBinary, NULL, &cs.bFlushOnTXEnd, STD_LOADABLE_MODULE_ID)); CHKiRet(omsdRegCFSLineHdlr((uchar *)"omfileiobuffersize", 0, eCmdHdlrSize, NULL, &cs.iIOBufSize, STD_LOADABLE_MODULE_ID)); CHKiRet(omsdRegCFSLineHdlr((uchar *)"dirowner", 0, eCmdHdlrUID, NULL, &cs.dirUID, STD_LOADABLE_MODULE_ID)); + CHKiRet(omsdRegCFSLineHdlr((uchar *)"dirownernum", 0, eCmdHdlrInt, NULL, &cs.dirUID, STD_LOADABLE_MODULE_ID)); CHKiRet(omsdRegCFSLineHdlr((uchar *)"dirgroup", 0, eCmdHdlrGID, NULL, &cs.dirGID, STD_LOADABLE_MODULE_ID)); + CHKiRet(omsdRegCFSLineHdlr((uchar *)"dirgroupnum", 0, eCmdHdlrInt, NULL, &cs.dirGID, STD_LOADABLE_MODULE_ID)); CHKiRet(omsdRegCFSLineHdlr((uchar *)"fileowner", 0, eCmdHdlrUID, NULL, &cs.fileUID, STD_LOADABLE_MODULE_ID)); + CHKiRet(omsdRegCFSLineHdlr((uchar *)"fileownernum", 0, eCmdHdlrInt, NULL, &cs.fileUID, STD_LOADABLE_MODULE_ID)); CHKiRet(omsdRegCFSLineHdlr((uchar *)"filegroup", 0, eCmdHdlrGID, NULL, &cs.fileGID, STD_LOADABLE_MODULE_ID)); + CHKiRet(omsdRegCFSLineHdlr((uchar *)"filegroupnum", 0, eCmdHdlrInt, NULL, &cs.fileGID, STD_LOADABLE_MODULE_ID)); CHKiRet(omsdRegCFSLineHdlr((uchar *)"dircreatemode", 0, eCmdHdlrFileCreateMode, NULL, &cs.fDirCreateMode, STD_LOADABLE_MODULE_ID)); CHKiRet(omsdRegCFSLineHdlr((uchar *)"filecreatemode", 0, eCmdHdlrFileCreateMode, NULL, &cs.fCreateMode, STD_LOADABLE_MODULE_ID)); CHKiRet(omsdRegCFSLineHdlr((uchar *)"createdirs", 0, eCmdHdlrBinary, NULL, &cs.bCreateDirs, STD_LOADABLE_MODULE_ID)); diff --git a/tools/omfwd.c b/tools/omfwd.c index 129392d..538281d 100644 --- a/tools/omfwd.c +++ b/tools/omfwd.c @@ -4,7 +4,7 @@ * NOTE: read comments in module-template.h to understand how this file * works! * - * Copyright 2007-2012 Adiscon GmbH. + * Copyright 2007-2013 Adiscon GmbH. * * This file is part of rsyslog. * @@ -21,9 +21,6 @@ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. * See the License for the specific language governing permissions and * limitations under the License. - * - * TODO v6 config: - * - permitted peer *list* */ #include "config.h" #include "rsyslog.h" @@ -39,6 +36,7 @@ #include <errno.h> #include <ctype.h> #include <unistd.h> +#include <stdint.h> #ifdef USE_NETZIP #include <zlib.h> #endif @@ -74,32 +72,51 @@ DEFobjCurrIf(netstrms) DEFobjCurrIf(netstrm) DEFobjCurrIf(tcpclt) + +/* some local constants (just) for better readybility */ +#define IS_FLUSH 1 +#define NO_FLUSH 0 + typedef struct _instanceData { uchar *tplName; /* name of assigned template */ - netstrms_t *pNS; /* netstream subsystem */ - netstrm_t *pNetstrm; /* our output netstream */ uchar *pszStrmDrvr; uchar *pszStrmDrvrAuthMode; permittedPeers_t *pPermPeers; int iStrmDrvrMode; char *target; - int *pSockArray; /* sockets to use for UDP */ - int bIsConnected; /* are we connected to remote host? 0 - no, 1 - yes, UDP means addr resolved */ - struct addrinfo *f_addr; int compressionLevel; /* 0 - no compression, else level for zlib */ char *port; int protocol; int iRebindInterval; /* rebind interval */ - int nXmit; /* number of transmissions since last (re-)bind */ # define FORW_UDP 0 # define FORW_TCP 1 /* following fields for TCP-based delivery */ TCPFRAMINGMODE tcp_framing; int bResendLastOnRecon; /* should the last message be re-sent on a successful reconnect? */ +# define COMPRESS_NEVER 0 +# define COMPRESS_SINGLE_MSG 1 /* old, single-message compression */ + /* all other settings are for stream-compression */ +# define COMPRESS_STREAM_ALWAYS 2 + uint8_t compressionMode; + int errsToReport; /* max number of errors to report (per instance) */ + sbool strmCompFlushOnTxEnd; /* flush stream compression on transaction end? */ +} instanceData; + +typedef struct wrkrInstanceData { + instanceData *pData; + netstrms_t *pNS; /* netstream subsystem */ + netstrm_t *pNetstrm; /* our output netstream */ + struct addrinfo *f_addr; + int *pSockArray; /* sockets to use for UDP */ + int bIsConnected; /* are we connected to remote host? 0 - no, 1 - yes, UDP means addr resolved */ + int nXmit; /* number of transmissions since last (re-)bind */ tcpclt_t *pTCPClt; /* our tcpclt object */ + sbool bzInitDone; /* did we do an init of zstrm already? */ + z_stream zstrm; /* zip stream to use for tcp compression */ uchar sndBuf[16*1024]; /* this is intensionally fixed -- see no good reason to make configurable */ unsigned offsSndBuf; /* next free spot in send buffer */ -} instanceData; + int errsToReport; /* (remaining) number of errors to report */ +} wrkrInstanceData_t; /* config data */ typedef struct configSettings_s { @@ -132,6 +149,9 @@ static struct cnfparamdescr actpdescr[] = { { "protocol", eCmdHdlrGetWord, 0 }, { "tcp_framing", eCmdHdlrGetWord, 0 }, { "ziplevel", eCmdHdlrInt, 0 }, + { "compression.mode", eCmdHdlrGetWord, 0 }, + { "compression.stream.flushontxend", eCmdHdlrBinary, 0 }, + { "maxerrormessages", eCmdHdlrInt, 0 }, { "rebindinterval", eCmdHdlrInt, 0 }, { "streamdriver", eCmdHdlrGetWord, 0 }, { "streamdrivermode", eCmdHdlrInt, 0 }, @@ -155,6 +175,9 @@ static modConfData_t *loadModConf = NULL;/* modConf ptr to use for the current l static modConfData_t *runModConf = NULL;/* modConf ptr to use for the current exec process */ +static rsRetVal initTCP(wrkrInstanceData_t *pWrkrData); + + BEGINinitConfVars /* (re)set config variables to default values */ CODESTARTinitConfVars cs.pszTplName = NULL; /* name of the default template to use */ @@ -168,7 +191,8 @@ CODESTARTinitConfVars ENDinitConfVars -static rsRetVal doTryResume(instanceData *pData); +static rsRetVal doTryResume(wrkrInstanceData_t *); +static rsRetVal doZipFinish(wrkrInstanceData_t *); /* this function gets the default template. It coordinates action between * old-style and new-style configuration parts. @@ -212,17 +236,16 @@ finalize_it: * rgerhards, 2009-05-29 */ static rsRetVal -closeUDPSockets(instanceData *pData) +closeUDPSockets(wrkrInstanceData_t *pWrkrData) { DEFiRet; - assert(pData != NULL); - if(pData->pSockArray != NULL) { - net.closeUDPListenSockets(pData->pSockArray); - pData->pSockArray = NULL; - freeaddrinfo(pData->f_addr); - pData->f_addr = NULL; + if(pWrkrData->pSockArray != NULL) { + net.closeUDPListenSockets(pWrkrData->pSockArray); + pWrkrData->pSockArray = NULL; + freeaddrinfo(pWrkrData->f_addr); + pWrkrData->f_addr = NULL; } -pData->bIsConnected = 0; // TODO: remove this variable altogether +pWrkrData->bIsConnected = 0; // TODO: remove this variable altogether RETiRet; } @@ -233,17 +256,17 @@ pData->bIsConnected = 0; // TODO: remove this variable altogether * rgerhards, 2008-06-04 * Note that we DO NOT discard the current buffer contents * (if any). This permits us to save data between sessions. In - * the wort case, some duplication occurs, but we do not + * the worst case, some duplication occurs, but we do not * loose data. */ static inline void -DestructTCPInstanceData(instanceData *pData) +DestructTCPInstanceData(wrkrInstanceData_t *pWrkrData) { - assert(pData != NULL); - if(pData->pNetstrm != NULL) - netstrm.Destruct(&pData->pNetstrm); - if(pData->pNS != NULL) - netstrms.Destruct(&pData->pNS); + doZipFinish(pWrkrData); + if(pWrkrData->pNetstrm != NULL) + netstrm.Destruct(&pWrkrData->pNetstrm); + if(pWrkrData->pNS != NULL) + netstrms.Destruct(&pWrkrData->pNS); } @@ -314,10 +337,25 @@ ENDfreeCnf BEGINcreateInstance CODESTARTcreateInstance - pData->offsSndBuf = 0; + pData->errsToReport = 5; + if(cs.pszStrmDrvr != NULL) + CHKmalloc(pData->pszStrmDrvr = (uchar*)strdup((char*)cs.pszStrmDrvr)); + if(cs.pszStrmDrvrAuthMode != NULL) + CHKmalloc(pData->pszStrmDrvrAuthMode = + (uchar*)strdup((char*)cs.pszStrmDrvrAuthMode)); +finalize_it: ENDcreateInstance +BEGINcreateWrkrInstance +CODESTARTcreateWrkrInstance + dbgprintf("DDDD: createWrkrInstance: pWrkrData %p\n", pWrkrData); + pWrkrData->offsSndBuf = 0; + pWrkrData->errsToReport = pData->errsToReport; + iRet = initTCP(pWrkrData); +ENDcreateWrkrInstance + + BEGINisCompatibleWithFeature CODESTARTisCompatibleWithFeature if(eFeat == sFEATURERepeatedMsgReduction) @@ -327,22 +365,25 @@ ENDisCompatibleWithFeature BEGINfreeInstance CODESTARTfreeInstance - /* final cleanup */ - DestructTCPInstanceData(pData); - closeUDPSockets(pData); - - if(pData->protocol == FORW_TCP) { - tcpclt.Destruct(&pData->pTCPClt); - } - - free(pData->port); - free(pData->target); free(pData->pszStrmDrvr); free(pData->pszStrmDrvrAuthMode); + free(pData->port); + free(pData->target); net.DestructPermittedPeers(&pData->pPermPeers); ENDfreeInstance +BEGINfreeWrkrInstance +CODESTARTfreeWrkrInstance + DestructTCPInstanceData(pWrkrData); + closeUDPSockets(pWrkrData); + + if(pWrkrData->pData->protocol == FORW_TCP) { + tcpclt.Destruct(&pWrkrData->pTCPClt); + } +ENDfreeWrkrInstance + + BEGINdbgPrintInstInfo CODESTARTdbgPrintInstInfo dbgprintf("%s", pData->target); @@ -352,25 +393,29 @@ ENDdbgPrintInstInfo /* Send a message via UDP * rgehards, 2007-12-20 */ -static rsRetVal UDPSend(instanceData *pData, char *msg, size_t len) +static rsRetVal UDPSend(wrkrInstanceData_t *__restrict__ const pWrkrData, + uchar *__restrict__ const msg, + const size_t len) { DEFiRet; struct addrinfo *r; int i; unsigned lsent = 0; - int bSendSuccess; + sbool bSendSuccess; + int lasterrno = ENOENT; + char errStr[1024]; - if(pData->iRebindInterval && (pData->nXmit++ % pData->iRebindInterval == 0)) { + if(pWrkrData->pData->iRebindInterval && (pWrkrData->nXmit++ % pWrkrData->pData->iRebindInterval == 0)) { dbgprintf("omfwd dropping UDP 'connection' (as configured)\n"); - pData->nXmit = 1; /* else we have an addtl wrap at 2^31-1 */ - CHKiRet(closeUDPSockets(pData)); + pWrkrData->nXmit = 1; /* else we have an addtl wrap at 2^31-1 */ + CHKiRet(closeUDPSockets(pWrkrData)); } - if(pData->pSockArray == NULL) { - CHKiRet(doTryResume(pData)); + if(pWrkrData->pSockArray == NULL) { + CHKiRet(doTryResume(pWrkrData)); } - if(pData->pSockArray != NULL) { + if(pWrkrData->pSockArray != NULL) { /* we need to track if we have success sending to the remote * peer. Success is indicated by at least one sendto() call * succeeding. We track this be bSendSuccess. We can not simply @@ -379,25 +424,37 @@ static rsRetVal UDPSend(instanceData *pData, char *msg, size_t len) * the sendto() succeeded. -- rgerhards, 2007-06-22 */ bSendSuccess = RSFALSE; - for (r = pData->f_addr; r; r = r->ai_next) { - for (i = 0; i < *pData->pSockArray; i++) { - lsent = sendto(pData->pSockArray[i+1], msg, len, 0, r->ai_addr, r->ai_addrlen); + for (r = pWrkrData->f_addr; r; r = r->ai_next) { + for (i = 0; i < *pWrkrData->pSockArray; i++) { + lsent = sendto(pWrkrData->pSockArray[i+1], msg, len, 0, r->ai_addr, r->ai_addrlen); if (lsent == len) { bSendSuccess = RSTRUE; break; } else { - int eno = errno; - char errStr[1024]; - dbgprintf("sendto() error: %d = %s.\n", - eno, rs_strerror_r(eno, errStr, sizeof(errStr))); + lasterrno = errno; + DBGPRINTF("sendto() error: %d = %s.\n", + lasterrno, + rs_strerror_r(lasterrno, errStr, sizeof(errStr))); } } if (lsent == len && !send_to_all) break; } /* finished looping */ - if (bSendSuccess == RSFALSE) { + if(bSendSuccess == RSFALSE) { dbgprintf("error forwarding via udp, suspending\n"); + if(pWrkrData->errsToReport > 0) { + rs_strerror_r(lasterrno, errStr, sizeof(errStr)); + errmsg.LogError(0, RS_RET_ERR_UDPSEND, "omfwd: error sending " + "via udp: %s", errStr); + if(pWrkrData->errsToReport == 1) { + errmsg.LogError(0, RS_RET_LAST_ERRREPORT, "omfwd: " + "max number of error message emitted " + "- further messages will be " + "suppressed"); + } + --pWrkrData->errsToReport; + } iRet = RS_RET_SUSPENDED; } } @@ -423,24 +480,19 @@ finalize_it: /* CODE FOR SENDING TCP MESSAGES */ - -/* Send a buffer via TCP. Usually, this is used to send the current - * send buffer, but if a message is larger than the buffer, we need to - * have the capability to send the message buffer directly. - * rgerhards, 2011-04-04 - */ static rsRetVal -TCPSendBuf(instanceData *pData, uchar *buf, unsigned len) +TCPSendBufUncompressed(wrkrInstanceData_t *pWrkrData, uchar *buf, unsigned len) { DEFiRet; unsigned alreadySent; ssize_t lenSend; alreadySent = 0; - CHKiRet(netstrm.CheckConnection(pData->pNetstrm)); /* hack for plain tcp syslog - see ptcp driver for details */ + CHKiRet(netstrm.CheckConnection(pWrkrData->pNetstrm)); /* hack for plain tcp syslog - see ptcp driver for details */ + while(alreadySent != len) { lenSend = len - alreadySent; - CHKiRet(netstrm.Send(pData->pNetstrm, buf+alreadySent, &lenSend)); + CHKiRet(netstrm.Send(pWrkrData->pNetstrm, buf+alreadySent, &lenSend)); DBGPRINTF("omfwd: TCP sent %ld bytes, requested %u\n", (long) lenSend, len - alreadySent); alreadySent += lenSend; } @@ -449,38 +501,136 @@ finalize_it: if(iRet != RS_RET_OK) { /* error! */ dbgprintf("TCPSendBuf error %d, destruct TCP Connection!\n", iRet); - DestructTCPInstanceData(pData); + DestructTCPInstanceData(pWrkrData); iRet = RS_RET_SUSPENDED; } RETiRet; } +static rsRetVal +TCPSendBufCompressed(wrkrInstanceData_t *pWrkrData, uchar *buf, unsigned len, sbool bIsFlush) +{ + int zRet; /* zlib return state */ + unsigned outavail; + uchar zipBuf[32*1024]; + int op; + DEFiRet; + + if(!pWrkrData->bzInitDone) { + /* allocate deflate state */ + pWrkrData->zstrm.zalloc = Z_NULL; + pWrkrData->zstrm.zfree = Z_NULL; + pWrkrData->zstrm.opaque = Z_NULL; + /* see note in file header for the params we use with deflateInit2() */ + zRet = deflateInit(&pWrkrData->zstrm, 9); + if(zRet != Z_OK) { + DBGPRINTF("error %d returned from zlib/deflateInit()\n", zRet); + ABORT_FINALIZE(RS_RET_ZLIB_ERR); + } + pWrkrData->bzInitDone = RSTRUE; + } + + /* now doing the compression */ + pWrkrData->zstrm.next_in = (Bytef*) buf; + pWrkrData->zstrm.avail_in = len; + if(pWrkrData->pData->strmCompFlushOnTxEnd && bIsFlush) + op = Z_SYNC_FLUSH; + else + op = Z_NO_FLUSH; + /* run deflate() on buffer until everything has been compressed */ + do { + DBGPRINTF("omfwd: in deflate() loop, avail_in %d, total_in %ld, isFlush %d\n", pWrkrData->zstrm.avail_in, pWrkrData->zstrm.total_in, bIsFlush); + pWrkrData->zstrm.avail_out = sizeof(zipBuf); + pWrkrData->zstrm.next_out = zipBuf; + zRet = deflate(&pWrkrData->zstrm, op); /* no bad return value */ + DBGPRINTF("after deflate, ret %d, avail_out %d\n", zRet, pWrkrData->zstrm.avail_out); + outavail = sizeof(zipBuf) - pWrkrData->zstrm.avail_out; + if(outavail != 0) { + CHKiRet(TCPSendBufUncompressed(pWrkrData, zipBuf, outavail)); + } + } while (pWrkrData->zstrm.avail_out == 0); + +finalize_it: + RETiRet; +} + +static rsRetVal +TCPSendBuf(wrkrInstanceData_t *pWrkrData, uchar *buf, unsigned len, sbool bIsFlush) +{ + DEFiRet; + if(pWrkrData->pData->compressionMode >= COMPRESS_STREAM_ALWAYS) + iRet = TCPSendBufCompressed(pWrkrData, buf, len, bIsFlush); + else + iRet = TCPSendBufUncompressed(pWrkrData, buf, len); + RETiRet; +} + +/* finish zlib buffer, to be called before closing the ZIP file (if + * running in stream mode). + */ +static rsRetVal +doZipFinish(wrkrInstanceData_t *pWrkrData) +{ + int zRet; /* zlib return state */ + DEFiRet; + unsigned outavail; + uchar zipBuf[32*1024]; + + if(!pWrkrData->bzInitDone) + goto done; + +// TODO: can we get this into a single common function? +dbgprintf("DDDD: in doZipFinish()\n"); + pWrkrData->zstrm.avail_in = 0; + /* run deflate() on buffer until everything has been compressed */ + do { + DBGPRINTF("in deflate() loop, avail_in %d, total_in %ld\n", pWrkrData->zstrm.avail_in, pWrkrData->zstrm.total_in); + pWrkrData->zstrm.avail_out = sizeof(zipBuf); + pWrkrData->zstrm.next_out = zipBuf; + zRet = deflate(&pWrkrData->zstrm, Z_FINISH); /* no bad return value */ + DBGPRINTF("after deflate, ret %d, avail_out %d\n", zRet, pWrkrData->zstrm.avail_out); + outavail = sizeof(zipBuf) - pWrkrData->zstrm.avail_out; + if(outavail != 0) { + CHKiRet(TCPSendBufUncompressed(pWrkrData, zipBuf, outavail)); + } + } while (pWrkrData->zstrm.avail_out == 0); + +finalize_it: + zRet = deflateEnd(&pWrkrData->zstrm); + if(zRet != Z_OK) { + DBGPRINTF("error %d returned from zlib/deflateEnd()\n", zRet); + } + + pWrkrData->bzInitDone = 0; +done: RETiRet; +} + /* Add frame to send buffer (or send, if requried) */ static rsRetVal TCPSendFrame(void *pvData, char *msg, size_t len) { DEFiRet; - instanceData *pData = (instanceData *) pvData; + wrkrInstanceData_t *pWrkrData = (wrkrInstanceData_t *) pvData; DBGPRINTF("omfwd: add %u bytes to send buffer (curr offs %u)\n", - (unsigned) len, pData->offsSndBuf); - if(pData->offsSndBuf != 0 && pData->offsSndBuf + len >= sizeof(pData->sndBuf)) { + (unsigned) len, pWrkrData->offsSndBuf); + if(pWrkrData->offsSndBuf != 0 && pWrkrData->offsSndBuf + len >= sizeof(pWrkrData->sndBuf)) { /* no buffer space left, need to commit previous records */ - CHKiRet(TCPSendBuf(pData, pData->sndBuf, pData->offsSndBuf)); - pData->offsSndBuf = 0; + CHKiRet(TCPSendBuf(pWrkrData, pWrkrData->sndBuf, pWrkrData->offsSndBuf, NO_FLUSH)); + pWrkrData->offsSndBuf = 0; iRet = RS_RET_PREVIOUS_COMMITTED; } /* check if the message is too large to fit into buffer */ - if(len > sizeof(pData->sndBuf)) { - CHKiRet(TCPSendBuf(pData, (uchar*)msg, len)); + if(len > sizeof(pWrkrData->sndBuf)) { + CHKiRet(TCPSendBuf(pWrkrData, (uchar*)msg, len, NO_FLUSH)); ABORT_FINALIZE(RS_RET_OK); /* committed everything so far */ } /* we now know the buffer has enough free space */ - memcpy(pData->sndBuf + pData->offsSndBuf, msg, len); - pData->offsSndBuf += len; + memcpy(pWrkrData->sndBuf + pWrkrData->offsSndBuf, msg, len); + pWrkrData->offsSndBuf += len; iRet = RS_RET_DEFER_COMMIT; finalize_it: @@ -495,11 +645,10 @@ finalize_it: static rsRetVal TCPSendPrepRetry(void *pvData) { DEFiRet; - instanceData *pData = (instanceData *) pvData; -dbgprintf("TCPSendPrepRetry performs a DestructTCPInstanceData\n"); + wrkrInstanceData_t *pWrkrData = (wrkrInstanceData_t *) pvData; - assert(pData != NULL); - DestructTCPInstanceData(pData); + assert(pWrkrData != NULL); + DestructTCPInstanceData(pWrkrData); RETiRet; } @@ -510,36 +659,39 @@ dbgprintf("TCPSendPrepRetry performs a DestructTCPInstanceData\n"); static rsRetVal TCPSendInit(void *pvData) { DEFiRet; - instanceData *pData = (instanceData *) pvData; + wrkrInstanceData_t *pWrkrData = (wrkrInstanceData_t *) pvData; + instanceData *pData; - assert(pData != NULL); - if(pData->pNetstrm == NULL) { + assert(pWrkrData != NULL); + pData = pWrkrData->pData; + + if(pWrkrData->pNetstrm == NULL) { dbgprintf("TCPSendInit CREATE\n"); - CHKiRet(netstrms.Construct(&pData->pNS)); + CHKiRet(netstrms.Construct(&pWrkrData->pNS)); /* the stream driver must be set before the object is finalized! */ - CHKiRet(netstrms.SetDrvrName(pData->pNS, pData->pszStrmDrvr)); - CHKiRet(netstrms.ConstructFinalize(pData->pNS)); + CHKiRet(netstrms.SetDrvrName(pWrkrData->pNS, pData->pszStrmDrvr)); + CHKiRet(netstrms.ConstructFinalize(pWrkrData->pNS)); /* now create the actual stream and connect to the server */ - CHKiRet(netstrms.CreateStrm(pData->pNS, &pData->pNetstrm)); - CHKiRet(netstrm.ConstructFinalize(pData->pNetstrm)); - CHKiRet(netstrm.SetDrvrMode(pData->pNetstrm, pData->iStrmDrvrMode)); + CHKiRet(netstrms.CreateStrm(pWrkrData->pNS, &pWrkrData->pNetstrm)); + CHKiRet(netstrm.ConstructFinalize(pWrkrData->pNetstrm)); + CHKiRet(netstrm.SetDrvrMode(pWrkrData->pNetstrm, pData->iStrmDrvrMode)); /* now set optional params, but only if they were actually configured */ if(pData->pszStrmDrvrAuthMode != NULL) { - CHKiRet(netstrm.SetDrvrAuthMode(pData->pNetstrm, pData->pszStrmDrvrAuthMode)); + CHKiRet(netstrm.SetDrvrAuthMode(pWrkrData->pNetstrm, pData->pszStrmDrvrAuthMode)); } if(pData->pPermPeers != NULL) { - CHKiRet(netstrm.SetDrvrPermPeers(pData->pNetstrm, pData->pPermPeers)); + CHKiRet(netstrm.SetDrvrPermPeers(pWrkrData->pNetstrm, pData->pPermPeers)); } /* params set, now connect */ - CHKiRet(netstrm.Connect(pData->pNetstrm, glbl.GetDefPFFamily(), + CHKiRet(netstrm.Connect(pWrkrData->pNetstrm, glbl.GetDefPFFamily(), (uchar*)pData->port, (uchar*)pData->target)); } finalize_it: if(iRet != RS_RET_OK) { dbgprintf("TCPSendInit FAILED with %d.\n", iRet); - DestructTCPInstanceData(pData); + DestructTCPInstanceData(pWrkrData); } RETiRet; @@ -549,15 +701,17 @@ finalize_it: /* try to resume connection if it is not ready * rgerhards, 2007-08-02 */ -static rsRetVal doTryResume(instanceData *pData) +static rsRetVal doTryResume(wrkrInstanceData_t *pWrkrData) { int iErr; struct addrinfo *res; struct addrinfo hints; + instanceData *pData; DEFiRet; - if(pData->bIsConnected) + if(pWrkrData->bIsConnected) FINALIZE; + pData = pWrkrData->pData; /* The remote address is not yet known and needs to be obtained */ dbgprintf(" %s\n", pData->target); @@ -573,20 +727,20 @@ static rsRetVal doTryResume(instanceData *pData) ABORT_FINALIZE(RS_RET_SUSPENDED); } dbgprintf("%s found, resuming.\n", pData->target); - pData->f_addr = res; - pData->bIsConnected = 1; - if(pData->pSockArray == NULL) { - pData->pSockArray = net.create_udp_socket((uchar*)pData->target, NULL, 0); + pWrkrData->f_addr = res; + pWrkrData->bIsConnected = 1; + if(pWrkrData->pSockArray == NULL) { + pWrkrData->pSockArray = net.create_udp_socket((uchar*)pData->target, NULL, 0, 0); } } else { - CHKiRet(TCPSendInit((void*)pData)); + CHKiRet(TCPSendInit((void*)pWrkrData)); } finalize_it: if(iRet != RS_RET_OK) { - if(pData->f_addr != NULL) { - freeaddrinfo(pData->f_addr); - pData->f_addr = NULL; + if(pWrkrData->f_addr != NULL) { + freeaddrinfo(pWrkrData->f_addr); + pWrkrData->f_addr = NULL; } iRet = RS_RET_SUSPENDED; } @@ -597,33 +751,35 @@ finalize_it: BEGINtryResume CODESTARTtryResume - iRet = doTryResume(pData); + dbgprintf("DDDD: tryResume: pWrkrData %p\n", pWrkrData); + iRet = doTryResume(pWrkrData); ENDtryResume BEGINbeginTransaction CODESTARTbeginTransaction dbgprintf("omfwd: beginTransaction\n"); + iRet = doTryResume(pWrkrData); ENDbeginTransaction -BEGINdoAction - char *psz; /* temporary buffering */ +static rsRetVal +processMsg(wrkrInstanceData_t *__restrict__ const pWrkrData, + actWrkrIParams_t *__restrict__ const iparam) +{ + uchar *psz; /* temporary buffering */ register unsigned l; int iMaxLine; # ifdef USE_NETZIP Bytef *out = NULL; /* for compression */ # endif -CODESTARTdoAction - CHKiRet(doTryResume(pData)); + instanceData *__restrict__ const pData = pWrkrData->pData; + DEFiRet; iMaxLine = glbl.GetMaxLine(); - dbgprintf(" %s:%s/%s\n", pData->target, pData->port, - pData->protocol == FORW_UDP ? "udp" : "tcp"); - - psz = (char*) ppString[0]; - l = strlen((char*) psz); + psz = iparam->param; + l = iparam->lenStr; if((int) l > iMaxLine) l = iMaxLine; @@ -636,11 +792,10 @@ CODESTARTdoAction * hard-coded but this may be changed to a config parameter. * rgerhards, 2006-11-30 */ - if(pData->compressionLevel && (l > CONF_MIN_SIZE_FOR_COMPRESS)) { + if(pData->compressionMode == COMPRESS_SINGLE_MSG && (l > CONF_MIN_SIZE_FOR_COMPRESS)) { uLongf destLen = iMaxLine + iMaxLine/100 +12; /* recommended value from zlib doc */ uLong srcLen = l; int ret; - /* TODO: optimize malloc sequence? -- rgerhards, 2008-09-02 */ CHKmalloc(out = (Bytef*) MALLOC(destLen)); out[0] = 'z'; out[1] = '\0'; @@ -660,7 +815,7 @@ CODESTARTdoAction } else if(destLen+1 < l) { /* only use compression if there is a gain in using it! */ dbgprintf("there is gain in compression, so we do it\n"); - psz = (char*) out; + psz = out; l = destLen + 1; /* take care for the "z" at message start! */ } ++destLen; @@ -669,14 +824,14 @@ CODESTARTdoAction if(pData->protocol == FORW_UDP) { /* forward via UDP */ - CHKiRet(UDPSend(pData, psz, l)); + CHKiRet(UDPSend(pWrkrData, psz, l)); } else { /* forward via TCP */ - iRet = tcpclt.Send(pData->pTCPClt, pData, psz, l); + iRet = tcpclt.Send(pWrkrData->pTCPClt, pWrkrData, (char *)psz, l); if(iRet != RS_RET_OK && iRet != RS_RET_DEFER_COMMIT && iRet != RS_RET_PREVIOUS_COMMITTED) { /* error! */ dbgprintf("error forwarding via tcp, suspending\n"); - DestructTCPInstanceData(pData); + DestructTCPInstanceData(pWrkrData); iRet = RS_RET_SUSPENDED; } } @@ -684,18 +839,30 @@ finalize_it: # ifdef USE_NETZIP free(out); /* is NULL if it was never used... */ # endif -ENDdoAction + RETiRet; +} + +BEGINcommitTransaction + unsigned i; +CODESTARTcommitTransaction + CHKiRet(doTryResume(pWrkrData)); + dbgprintf(" %s:%s/%s\n", pWrkrData->pData->target, pWrkrData->pData->port, + pWrkrData->pData->protocol == FORW_UDP ? "udp" : "tcp"); -BEGINendTransaction -CODESTARTendTransaction -dbgprintf("omfwd: endTransaction, offsSndBuf %u\n", pData->offsSndBuf); - if(pData->offsSndBuf != 0) { - iRet = TCPSendBuf(pData, pData->sndBuf, pData->offsSndBuf); - pData->offsSndBuf = 0; + for(i = 0 ; i < nParams ; ++i) { + iRet = processMsg(pWrkrData, &actParam(pParams, 1, i, 0)); + if(iRet != RS_RET_OK && iRet != RS_RET_DEFER_COMMIT && iRet != RS_RET_PREVIOUS_COMMITTED) + FINALIZE; } -ENDendTransaction +dbgprintf("omfwd: endTransaction, offsSndBuf %u, iRet %d\n", pWrkrData->offsSndBuf, iRet); + if(pWrkrData->offsSndBuf != 0) { + iRet = TCPSendBuf(pWrkrData, pWrkrData->sndBuf, pWrkrData->offsSndBuf, IS_FLUSH); + pWrkrData->offsSndBuf = 0; + } +finalize_it: +ENDcommitTransaction /* This function loads TCP support, if not already loaded. It will be called @@ -719,25 +886,22 @@ finalize_it: * created. */ static rsRetVal -initTCP(instanceData *pData) +initTCP(wrkrInstanceData_t *pWrkrData) { + instanceData *pData; DEFiRet; + + pData = pWrkrData->pData; if(pData->protocol == FORW_TCP) { /* create our tcpclt */ - CHKiRet(tcpclt.Construct(&pData->pTCPClt)); - CHKiRet(tcpclt.SetResendLastOnRecon(pData->pTCPClt, pData->bResendLastOnRecon)); + CHKiRet(tcpclt.Construct(&pWrkrData->pTCPClt)); + CHKiRet(tcpclt.SetResendLastOnRecon(pWrkrData->pTCPClt, pData->bResendLastOnRecon)); /* and set callbacks */ - CHKiRet(tcpclt.SetSendInit(pData->pTCPClt, TCPSendInit)); - CHKiRet(tcpclt.SetSendFrame(pData->pTCPClt, TCPSendFrame)); - CHKiRet(tcpclt.SetSendPrepRetry(pData->pTCPClt, TCPSendPrepRetry)); - CHKiRet(tcpclt.SetFraming(pData->pTCPClt, pData->tcp_framing)); - CHKiRet(tcpclt.SetRebindInterval(pData->pTCPClt, pData->iRebindInterval)); - pData->iStrmDrvrMode = cs.iStrmDrvrMode; - if(cs.pszStrmDrvr != NULL) - CHKmalloc(pData->pszStrmDrvr = (uchar*)strdup((char*)cs.pszStrmDrvr)); - if(cs.pszStrmDrvrAuthMode != NULL) - CHKmalloc(pData->pszStrmDrvrAuthMode = - (uchar*)strdup((char*)cs.pszStrmDrvrAuthMode)); + CHKiRet(tcpclt.SetSendInit(pWrkrData->pTCPClt, TCPSendInit)); + CHKiRet(tcpclt.SetSendFrame(pWrkrData->pTCPClt, TCPSendFrame)); + CHKiRet(tcpclt.SetSendPrepRetry(pWrkrData->pTCPClt, TCPSendPrepRetry)); + CHKiRet(tcpclt.SetFraming(pWrkrData->pTCPClt, pData->tcp_framing)); + CHKiRet(tcpclt.SetRebindInterval(pWrkrData->pTCPClt, pData->iRebindInterval)); } finalize_it: RETiRet; @@ -756,14 +920,19 @@ setInstParamDefaults(instanceData *pData) pData->iRebindInterval = 0; pData->bResendLastOnRecon = 0; pData->pPermPeers = NULL; - pData->compressionLevel = 0; + pData->compressionLevel = 9; + pData->strmCompFlushOnTxEnd = 1; + pData->compressionMode = COMPRESS_NEVER; + pData->errsToReport = 5; } BEGINnewActInst struct cnfparamvals *pvals; uchar *tplToUse; + char *cstr; int i; rsRetVal localRet; + int complevel = -1; CODESTARTnewActInst DBGPRINTF("newActInst (omfwd)\n"); @@ -860,9 +1029,10 @@ CODESTARTnewActInst free(str); } else if(!strcmp(actpblk.descr[i].name, "ziplevel")) { # ifdef USE_NETZIP - int complevel = pvals[i].val.d.n; + complevel = pvals[i].val.d.n; if(complevel >= 0 && complevel <= 10) { pData->compressionLevel = complevel; + pData->compressionMode = COMPRESS_SINGLE_MSG; } else { errmsg.LogError(0, NO_ERRCODE, "Invalid ziplevel %d specified in " "forwardig action - NOT turning on compression.", @@ -872,21 +1042,50 @@ CODESTARTnewActInst errmsg.LogError(0, NO_ERRCODE, "Compression requested, but rsyslogd is not compiled " "with compression support - request ignored."); # endif /* #ifdef USE_NETZIP */ + } else if(!strcmp(actpblk.descr[i].name, "maxerrormessages")) { + pData->errsToReport = (int) pvals[i].val.d.n; } else if(!strcmp(actpblk.descr[i].name, "resendlastmsgonreconnect")) { pData->bResendLastOnRecon = (int) pvals[i].val.d.n; } else if(!strcmp(actpblk.descr[i].name, "template")) { pData->tplName = (uchar*)es_str2cstr(pvals[i].val.d.estr, NULL); + } else if(!strcmp(actpblk.descr[i].name, "compression.stream.flushontxend")) { + pData->strmCompFlushOnTxEnd = (sbool) pvals[i].val.d.n; + } else if(!strcmp(actpblk.descr[i].name, "compression.mode")) { + cstr = es_str2cstr(pvals[i].val.d.estr, NULL); + if(!strcasecmp(cstr, "stream:always")) { + pData->compressionMode = COMPRESS_STREAM_ALWAYS; + } else if(!strcasecmp(cstr, "none")) { + pData->compressionMode = COMPRESS_NEVER; + } else if(!strcasecmp(cstr, "single")) { + pData->compressionMode = COMPRESS_SINGLE_MSG; + } else { + errmsg.LogError(0, RS_RET_PARAM_ERROR, "omfwd: invalid value for 'compression.mode' " + "parameter (given is '%s')", cstr); + free(cstr); + ABORT_FINALIZE(RS_RET_PARAM_ERROR); + } + free(cstr); } else { DBGPRINTF("omfwd: program error, non-handled " "param '%s'\n", actpblk.descr[i].name); } } + + if(complevel != -1) { + pData->compressionLevel = complevel; + if(pData->compressionMode == COMPRESS_NEVER) { + /* to keep compatible with pre-7.3.11, only setting the + * compresion level means old-style single-message mode. + */ + pData->compressionMode = COMPRESS_SINGLE_MSG; + } + } + CODE_STD_STRING_REQUESTnewActInst(1) tplToUse = ustrdup((pData->tplName == NULL) ? getDfltTpl() : pData->tplName); CHKiRet(OMSRsetEntry(*ppOMSR, 0, tplToUse, OMSR_NO_RQD_TPL_OPTS)); - CHKiRet(initTCP(pData)); CODE_STD_FINALIZERnewActInst cnfparamvalsDestruct(pvals, &actpblk); ENDnewActInst @@ -948,6 +1147,7 @@ CODE_STD_STRING_REQUESTparseSelectorAct(1) iLevel = *p - '0'; ++p; /* eat */ pData->compressionLevel = iLevel; + pData->compressionMode = COMPRESS_SINGLE_MSG; } else { errmsg.LogError(0, NO_ERRCODE, "Invalid compression level '%c' specified in " "forwardig action - NOT turning on compression.", @@ -1025,7 +1225,6 @@ CODE_STD_STRING_REQUESTparseSelectorAct(1) while(*p && *p != ';' && *p != '#' && !isspace((int) *p)) ++p; /*JUST SKIP*/ - /* TODO: make this if go away! */ if(*p == ';' || *p == '#' || isspace(*p)) { uchar cTmp = *p; *p = '\0'; /* trick to obtain hostname (later)! */ @@ -1055,7 +1254,6 @@ CODE_STD_STRING_REQUESTparseSelectorAct(1) cs.pPermPeers = NULL; } } - CHKiRet(initTCP(pData)); CODE_STD_FINALIZERparseSelectorAct ENDparseSelectorAct @@ -1090,11 +1288,11 @@ ENDmodExit BEGINqueryEtryPt CODESTARTqueryEtryPt -CODEqueryEtryPt_STD_OMOD_QUERIES +CODEqueryEtryPt_STD_OMODTX_QUERIES +CODEqueryEtryPt_STD_OMOD8_QUERIES CODEqueryEtryPt_STD_CONF2_QUERIES CODEqueryEtryPt_STD_CONF2_setModCnf_QUERIES CODEqueryEtryPt_STD_CONF2_OMOD_QUERIES -CODEqueryEtryPt_TXIF_OMOD_QUERIES /* we support the transactional interface! */ ENDqueryEtryPt diff --git a/tools/ompipe.c b/tools/ompipe.c index 420e2b1..c94568b 100644 --- a/tools/ompipe.c +++ b/tools/ompipe.c @@ -12,7 +12,7 @@ * NOTE: read comments in module-template.h to understand how this pipe * works! * - * Copyright 2007-2012 Rainer Gerhards and Adiscon GmbH. + * Copyright 2007-2014 Rainer Gerhards and Adiscon GmbH. * * This file is part of rsyslog. * @@ -69,9 +69,14 @@ typedef struct _instanceData { uchar *pipe; /* pipe or template name (display only) */ uchar *tplName; /* format template to use */ short fd; /* pipe descriptor for (current) pipe */ + pthread_mutex_t mutWrite; /* guard against multiple instances writing to same pipe */ sbool bHadError; /* did we already have/report an error on this pipe? */ } instanceData; +typedef struct wrkrInstanceData { + instanceData *pData; +} wrkrInstanceData_t; + typedef struct configSettings_s { EMPTY_STRUCT } configSettings_t; @@ -154,7 +159,7 @@ preparePipe(instanceData *pData) if(!pData->bHadError) { char errStr[1024]; rs_strerror_r(errno, errStr, sizeof(errStr)); - errmsg.LogError(0, RS_RET_NO_FILE_ACCESS, "Could no open output pipe '%s': %s", + errmsg.LogError(0, RS_RET_NO_FILE_ACCESS, "Could not open output pipe '%s': %s", pData->pipe, errStr); pData->bHadError = 1; } @@ -276,25 +281,66 @@ CODESTARTcreateInstance pData->pipe = NULL; pData->fd = -1; pData->bHadError = 0; + pthread_mutex_init(&pData->mutWrite, NULL); ENDcreateInstance +BEGINcreateWrkrInstance +CODESTARTcreateWrkrInstance +ENDcreateWrkrInstance + + BEGINfreeInstance CODESTARTfreeInstance + pthread_mutex_destroy(&pData->mutWrite); free(pData->pipe); if(pData->fd != -1) close(pData->fd); ENDfreeInstance +BEGINfreeWrkrInstance +CODESTARTfreeWrkrInstance +ENDfreeWrkrInstance + + BEGINtryResume + instanceData *__restrict__ const pData = pWrkrData->pData; + fd_set wrds; + struct timeval tv; + int ready; CODESTARTtryResume + if(pData->fd == -1) { + rsRetVal iRetLocal; + iRetLocal = preparePipe(pData); + if((iRetLocal != RS_RET_OK) || (pData->fd == -1)) + ABORT_FINALIZE(RS_RET_SUSPENDED); + } else { + /* we can reach this if the pipe is full, so we need + * to check if we can write again. /dev/xconsole is the + * ugly example of why this is necessary. + */ + FD_ZERO(&wrds); + FD_SET(pData->fd, &wrds); + tv.tv_sec = 0; + tv.tv_usec = 0; + ready = select(pData->fd+1, NULL, &wrds, NULL, &tv); + DBGPRINTF("ompipe: tryResume: ready to write fd %d: %d\n", pData->fd, ready); + if(ready != 1) + ABORT_FINALIZE(RS_RET_SUSPENDED); + } +finalize_it: ENDtryResume BEGINdoAction + instanceData *pData; CODESTARTdoAction - DBGPRINTF(" (%s)\n", pData->pipe); + pData = pWrkrData->pData; + DBGPRINTF("ompipe: writing to %s\n", pData->pipe); + /* this module is single-threaded by nature */ + pthread_mutex_lock(&pData->mutWrite); iRet = writePipe(ppString, pData); + pthread_mutex_unlock(&pData->mutWrite); ENDdoAction @@ -329,14 +375,9 @@ CODESTARTnewActInst } } - if(pData->tplName == NULL) { - CHKiRet(OMSRsetEntry(*ppOMSR, 0, (uchar*) "RSYSLOG_FileFormat", - OMSR_NO_RQD_TPL_OPTS)); - } else { - CHKiRet(OMSRsetEntry(*ppOMSR, 0, - (uchar*) strdup((char*) pData->tplName), - OMSR_NO_RQD_TPL_OPTS)); - } + CHKiRet(OMSRsetEntry(*ppOMSR, 0, (uchar*)strdup((pData->tplName == NULL) ? + "RSYSLOG_FileFormat" : (char*)pData->tplName), + OMSR_NO_RQD_TPL_OPTS)); CODE_STD_FINALIZERnewActInst cnfparamvalsDestruct(pvals, &actpblk); ENDnewActInst @@ -386,6 +427,7 @@ ENDmodExit BEGINqueryEtryPt CODESTARTqueryEtryPt CODEqueryEtryPt_STD_OMOD_QUERIES +CODEqueryEtryPt_STD_OMOD8_QUERIES CODEqueryEtryPt_doHUP CODEqueryEtryPt_STD_CONF2_QUERIES CODEqueryEtryPt_STD_CONF2_CNFNAME_QUERIES diff --git a/tools/omshell.c b/tools/omshell.c index ac62fa6..ad6e979 100644 --- a/tools/omshell.c +++ b/tools/omshell.c @@ -19,7 +19,7 @@ * of the "old" message code without any modifications. However, it * helps to have things at the right place one we go to the meat of it. * - * Copyright 2007-2012 Adiscon GmbH. + * Copyright 2007-2013 Adiscon GmbH. * * This file is part of rsyslog. * @@ -63,11 +63,20 @@ typedef struct _instanceData { uchar progName[MAXFNAME]; /* program to execute */ } instanceData; +typedef struct wrkrInstanceData { + instanceData *pData; +} wrkrInstanceData_t; + BEGINcreateInstance CODESTARTcreateInstance ENDcreateInstance +BEGINcreateWrkrInstance +CODESTARTcreateWrkrInstance +ENDcreateWrkrInstance + + BEGINisCompatibleWithFeature CODESTARTisCompatibleWithFeature if(eFeat == sFEATURERepeatedMsgReduction) @@ -80,6 +89,11 @@ CODESTARTfreeInstance ENDfreeInstance +BEGINfreeWrkrInstance +CODESTARTfreeWrkrInstance +ENDfreeWrkrInstance + + BEGINdbgPrintInstInfo CODESTARTdbgPrintInstInfo printf("%s", pData->progName); @@ -92,13 +106,9 @@ ENDtryResume BEGINdoAction CODESTARTdoAction - /* TODO: using pData->progName is not clean from the point of - * modularization. We'll change that as we go ahead with modularization. - * rgerhards, 2007-07-20 - */ dbgprintf("\n"); - if(execProg((uchar*) pData->progName, 1, ppString[0]) == 0) - errmsg.LogError(0, NO_ERRCODE, "Executing program '%s' failed", (char*)pData->progName); + if(execProg((uchar*) pWrkrData->pData->progName, 1, ppString[0]) == 0) + errmsg.LogError(0, NO_ERRCODE, "Executing program '%s' failed", (char*)pWrkrData->pData->progName); ENDdoAction @@ -139,6 +149,7 @@ ENDmodExit BEGINqueryEtryPt CODESTARTqueryEtryPt CODEqueryEtryPt_STD_OMOD_QUERIES +CODEqueryEtryPt_STD_OMOD8_QUERIES ENDqueryEtryPt diff --git a/tools/omusrmsg.c b/tools/omusrmsg.c index f4cc409..5d0b088 100644 --- a/tools/omusrmsg.c +++ b/tools/omusrmsg.c @@ -8,7 +8,7 @@ * File begun on 2007-07-20 by RGerhards (extracted from syslogd.c, which at the * time of the fork from sysklogd was under BSD license) * - * Copyright 2007-2012 Adiscon GmbH. + * Copyright 2007-2013 Adiscon GmbH. * * This file is part of rsyslog. * @@ -87,6 +87,10 @@ typedef struct _instanceData { uchar *tplName; } instanceData; +typedef struct wrkrInstanceData { + instanceData *pData; +} wrkrInstanceData_t; + typedef struct configSettings_s { EMPTY_STRUCT } configSettings_t; @@ -115,6 +119,11 @@ CODESTARTcreateInstance ENDcreateInstance +BEGINcreateWrkrInstance +CODESTARTcreateWrkrInstance +ENDcreateWrkrInstance + + BEGINisCompatibleWithFeature CODESTARTisCompatibleWithFeature if(eFeat == sFEATURERepeatedMsgReduction) @@ -128,6 +137,11 @@ CODESTARTfreeInstance ENDfreeInstance +BEGINfreeWrkrInstance +CODESTARTfreeWrkrInstance +ENDfreeWrkrInstance + + BEGINdbgPrintInstInfo register int i; CODESTARTdbgPrintInstInfo @@ -276,7 +290,7 @@ ENDtryResume BEGINdoAction CODESTARTdoAction dbgprintf("\n"); - iRet = wallmsg(ppString[0], pData); + iRet = wallmsg(ppString[0], pWrkrData->pData); ENDdoAction @@ -435,6 +449,7 @@ ENDmodExit BEGINqueryEtryPt CODESTARTqueryEtryPt CODEqueryEtryPt_STD_OMOD_QUERIES +CODEqueryEtryPt_STD_OMOD8_QUERIES CODEqueryEtryPt_STD_CONF2_OMOD_QUERIES ENDqueryEtryPt diff --git a/tools/pmrfc3164.c b/tools/pmrfc3164.c index 5dfa74f..7ffae43 100644 --- a/tools/pmrfc3164.c +++ b/tools/pmrfc3164.c @@ -6,24 +6,23 @@ * * File begun on 2009-11-04 by RGerhards * - * Copyright 2007, 2009 Rainer Gerhards and Adiscon GmbH. + * Copyright 2007-2014 Rainer Gerhards and Adiscon GmbH. * * This file is part of rsyslog. * - * Rsyslog is free software: you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation, either version 3 of the License, or - * (at your option) any later version. - * - * Rsyslog is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * - * You should have received a copy of the GNU General Public License - * along with Rsyslog. If not, see <http://www.gnu.org/licenses/>. - * - * A copy of the GPL can be found in the file "COPYING" in this distribution. + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * -or- + * see COPYING.ASL20 in the source distribution + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. */ #include "config.h" #include "rsyslog.h" @@ -84,7 +83,7 @@ CODESTARTparse assert(pMsg->pszRawMsg != NULL); lenMsg = pMsg->iLenRawMsg - pMsg->offAfterPRI; /* note: offAfterPRI is already the number of PRI chars (do not add one!) */ p2parse = pMsg->pszRawMsg + pMsg->offAfterPRI; /* point to start of text, after PRI */ - setProtocolVersion(pMsg, 0); + setProtocolVersion(pMsg, MSG_LEGACY_PROTOCOL); /* Check to see if msg contains a timestamp. We start by assuming * that the message timestamp is the time of reception (which we @@ -94,12 +93,14 @@ CODESTARTparse */ if(datetime.ParseTIMESTAMP3339(&(pMsg->tTIMESTAMP), &p2parse, &lenMsg) == RS_RET_OK) { /* we are done - parse pointer is moved by ParseTIMESTAMP3339 */; - } else if(datetime.ParseTIMESTAMP3164(&(pMsg->tTIMESTAMP), &p2parse, &lenMsg) == RS_RET_OK) { + } else if(datetime.ParseTIMESTAMP3164(&(pMsg->tTIMESTAMP), &p2parse, &lenMsg, NO_PARSE3164_TZSTRING) == RS_RET_OK) { + if(pMsg->dfltTZ[0] != '\0') + applyDfltTZ(&pMsg->tTIMESTAMP, pMsg->dfltTZ); /* we are done - parse pointer is moved by ParseTIMESTAMP3164 */; } else if(*p2parse == ' ' && lenMsg > 1) { /* try to see if it is slighly malformed - HP procurve seems to do that sometimes */ ++p2parse; /* move over space */ --lenMsg; - if(datetime.ParseTIMESTAMP3164(&(pMsg->tTIMESTAMP), &p2parse, &lenMsg) == RS_RET_OK) { + if(datetime.ParseTIMESTAMP3164(&(pMsg->tTIMESTAMP), &p2parse, &lenMsg, NO_PARSE3164_TZSTRING) == RS_RET_OK) { /* indeed, we got it! */ /* we are done - parse pointer is moved by ParseTIMESTAMP3164 */; } else {/* parse pointer needs to be restored, as we moved it off-by-one diff --git a/tools/pmrfc3164.h b/tools/pmrfc3164.h index 2430409..81987a4 100644 --- a/tools/pmrfc3164.h +++ b/tools/pmrfc3164.h @@ -7,20 +7,19 @@ * * This file is part of rsyslog. * - * Rsyslog is free software: you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation, either version 3 of the License, or - * (at your option) any later version. - * - * Rsyslog is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * - * You should have received a copy of the GNU General Public License - * along with Rsyslog. If not, see <http://www.gnu.org/licenses/>. - * - * A copy of the GPL can be found in the file "COPYING" in this distribution. + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * -or- + * see COPYING.ASL20 in the source distribution + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. */ #ifndef PMRFC3164_H_INCLUDED #define PMRFC3164_H_INCLUDED 1 diff --git a/tools/pmrfc5424.c b/tools/pmrfc5424.c index 9b5c616..18dc846 100644 --- a/tools/pmrfc5424.c +++ b/tools/pmrfc5424.c @@ -6,24 +6,23 @@ * * File begun on 2009-11-03 by RGerhards * - * Copyright 2007, 2009 Rainer Gerhards and Adiscon GmbH. + * Copyright 2007-2014 Rainer Gerhards and Adiscon GmbH. * * This file is part of rsyslog. * - * Rsyslog is free software: you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation, either version 3 of the License, or - * (at your option) any later version. - * - * Rsyslog is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * - * You should have received a copy of the GNU General Public License - * along with Rsyslog. If not, see <http://www.gnu.org/licenses/>. - * - * A copy of the GPL can be found in the file "COPYING" in this distribution. + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * -or- + * see COPYING.ASL20 in the source distribution + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. */ #include "config.h" #include "rsyslog.h" @@ -227,7 +226,7 @@ CODESTARTparse ABORT_FINALIZE(RS_RET_COULD_NOT_PARSE); } DBGPRINTF("Message has RFC5424/syslog-protocol format.\n"); - setProtocolVersion(pMsg, 1); + setProtocolVersion(pMsg, MSG_RFC5424_PROTOCOL); p2parse += 2; lenMsg -= 2; diff --git a/tools/pmrfc5424.h b/tools/pmrfc5424.h index df2a1c8..036fd30 100644 --- a/tools/pmrfc5424.h +++ b/tools/pmrfc5424.h @@ -3,24 +3,23 @@ * * File begun on 2009-11-03 by RGerhards * - * Copyright 2009 Rainer Gerhards and Adiscon GmbH. + * Copyright 2009-2014 Rainer Gerhards and Adiscon GmbH. * * This file is part of rsyslog. * - * Rsyslog is free software: you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation, either version 3 of the License, or - * (at your option) any later version. - * - * Rsyslog is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * - * You should have received a copy of the GNU General Public License - * along with Rsyslog. If not, see <http://www.gnu.org/licenses/>. - * - * A copy of the GPL can be found in the file "COPYING" in this distribution. + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * -or- + * see COPYING.ASL20 in the source distribution + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. */ #ifndef PMRFC54254_H_INCLUDED #define PMRFC54254_H_INCLUDED 1 diff --git a/tools/recover_qi.pl b/tools/recover_qi.pl index 4e2cf9d..eb8de55 100755 --- a/tools/recover_qi.pl +++ b/tools/recover_qi.pl @@ -1,207 +1,207 @@ -#!/usr/bin/perl -w
-# recover rsyslog disk queue index (.qi) from queue files (.nnnnnnnn).
-#
-# See:
-# runtime/queue.c: qqueuePersist()
-# runtime/queue.c: qqueueTryLoadPersistedInfo()
-#
-# kaiwang.chen@gmail.com 2012-03-14
-#
-use strict;
-use Getopt::Long;
-
-my %opt = ();
-GetOptions(\%opt,"spool|w=s","basename|f=s","digits|d=i","help!");
-if ($opt{help}) {
- print "Usage:
-\t$0 -w WorkDirectory -f QueueFileName -d 8 > QueueFileName.qi
-";
- exit;
-}
-
-# runtime/queue.c: qConstructDisk()
-my $iMaxFiles = 10000000; # 0+"1".( "0"x($opt{digits} - 1));
-
-# get the list of queue files, spool directory excluded
-my $re = qr/^\Q$opt{basename}\E\.\d{$opt{digits}}$/;
-opendir(DIR, $opt{spool}) or die "can’t open spool: $!";
-my @qf = grep { /$re/ && -f "$opt{spool}/$_" } readdir(DIR);
-closedir DIR;
-
-# ensure order and continuity
-@qf = sort @qf;
-my ($head) = ($qf[0] =~ /(\d+)$/);
-my ($tail) = ($qf[-1] =~ /(\d+)$/);
-$head += 0;
-$tail += 0;
-if ($tail-$head+1 != @qf || $tail > $iMaxFiles) {
- die "broken queue: missing file(s) or wrong tail\n";
-}
-
-# collect some counters about the queue, assuming all are unprocessed entries.
-my $sizeOnDisk = 0;
-my $iQueueSize = 0;
-chdir($opt{spool}) or die "can't chdir to spool: $!";
-print STDERR "traversing ". @qf ." files, please wait...\n";
-for (@qf) {
- open FH, "<", $_ or die "can't read queue file $_\n";
- $sizeOnDisk += (stat FH)[7];
- while (<FH>) {
- $iQueueSize++ if /^<Obj/; # runtime/msg.c: MsgSerialize()
- }
- close FH;
-}
-# happen to reuse last stat
-my $iCurrOffs_Write = (stat(_))[7];
-
-# runtime/queue.c: qqueuePersist()
-my $qqueue = Rsyslog::OPB->new("qqueue",1);
-$qqueue->property("iQueueSize", "INT", $iQueueSize);
-$qqueue->property("tVars.disk.sizeOnDisk", "INT64", $sizeOnDisk);
-$qqueue->property("tVars.disk.bytesRead", "INT64", 0);
-
-# runtime/stream.h: strmType_t
-my $STREAMTYPE_FILE_CIRCULAR = 1;
-# runtime/stream.h: strmMode_t
-my $STREAMMODE_READ = 1;
-my $STREAMMODE_WRITE_APPEND = 4;
-
-# runtime/stream.c: strmSerialize()
-# write to end
-my $strm_Write = Rsyslog::Obj->new("strm",1);
-$strm_Write->property( "iCurrFNum", "INT", $tail);
-$strm_Write->property( "pszFName", "PSZ", $opt{basename});
-$strm_Write->property( "iMaxFiles", "INT", $iMaxFiles);
-$strm_Write->property( "bDeleteOnClose", "INT", 0);
-$strm_Write->property( "sType", "INT", $STREAMTYPE_FILE_CIRCULAR);
-$strm_Write->property("tOperationsMode", "INT", $STREAMMODE_WRITE_APPEND);
-$strm_Write->property( "tOpenMode", "INT", 0600);
-$strm_Write->property( "iCurrOffs","INT64", $iCurrOffs_Write);
-# read from head
-my $strm_ReadDel = Rsyslog::Obj->new("strm",1);
-$strm_ReadDel->property( "iCurrFNum", "INT", $head);
-$strm_ReadDel->property( "pszFName", "PSZ", $opt{basename});
-$strm_ReadDel->property( "iMaxFiles", "INT", $iMaxFiles);
-$strm_ReadDel->property( "bDeleteOnClose", "INT", 1);
-$strm_ReadDel->property( "sType", "INT", $STREAMTYPE_FILE_CIRCULAR);
-$strm_ReadDel->property("tOperationsMode", "INT", $STREAMMODE_READ);
-$strm_ReadDel->property( "tOpenMode", "INT", 0600);
-$strm_ReadDel->property( "iCurrOffs","INT64", 0);
-
-# .qi
-print $qqueue->serialize();
-print $strm_Write->serialize();
-print $strm_ReadDel->serialize();
-
-exit;
-#-----------------------------------------------------------------------------
-
-package Rsyslog::Serializable;
-# runtime/obj.c
-sub COOKIE_OBJLINE { '<' }
-sub COOKIE_PROPLINE { '+' }
-sub COOKIE_ENDLINE { '>' }
-sub COOKIE_BLANKLINE { '.' }
-# VARTYPE(short_ptype)
-sub VARTYPE {
- my ($t) = @_;
- # runtime/obj-types.h: propType_t
- my $ptype = "PROPTYPE_".$t;
- # runtime/var.h: varType_t
- my %vm = (
- VARTYPE_NONE => 0,
- VARTYPE_STR => 1,
- VARTYPE_NUMBER => 2,
- VARTYPE_SYSLOGTIME => 3,
- );
- # runtime/obj.c: SerializeProp()
- my %p2v = (
- #PROPTYPE_NONE => "",
- PROPTYPE_PSZ => "VARTYPE_STR",
- PROPTYPE_SHORT => "VARTYPE_NUMBER",
- PROPTYPE_INT => "VARTYPE_NUMBER",
- PROPTYPE_LONG => "VARTYPE_NUMBER",
- PROPTYPE_INT64 => "VARTYPE_NUMBER",
- PROPTYPE_CSTR => "VARTYPE_STR",
- #PROPTYPE_SYSLOGTIME => "VARTYPE_SYSLOGTIME",
- );
- my $vtype = $p2v{$ptype};
- unless ($vtype) {
- die "property type $t is not supported!\n";
- }
- return $vm{$vtype};
-}
-sub serialize {
- my $self = shift;
- # runtime/obj.c: objSerializeHeader()
- my $x = COOKIE_OBJLINE();
- $x .= join(":", $self->type(), $self->cver(), $self->id(), $self->version());
- $x .= ":\n";
- for ( values %{$self->{props}} ) {
- # runtime/obj.c: SerializeProp()
- $x .= COOKIE_PROPLINE();
- $x .= join(":",
- $_->{name},
- VARTYPE($_->{type}),
- length($_->{value}),
- $_->{value});
- $x .= ":\n";
- }
- # runtime/obj.c: EndSerialize()
- $x .= COOKIE_ENDLINE() . "End\n";
- $x .= COOKIE_BLANKLINE() . "\n";
-}
-# constructor: new(id,version)
-sub new {
- my ($class, $id, $version) = @_;
- $class = ref $class if ref $class;
- bless {
- id => $id,
- version => $version,
- props => {},
- }, $class;
-}
-sub id {
- my $self = shift;
- if (@_) {
- my $x = $self->{id};
- $self->{id} = shift;
- return $x;
- }
- return $self->{id};
-}
-sub version {
- my $self = shift;
- if (@_) {
- my $x = $self->{version};
- $self->{version} = shift;
- return $x;
- }
- return $self->{version};
-}
-# property(name, type, value)
-sub property {
- my $self = shift;
- my $name = shift;
- if (@_) {
- my $x = $self->{props}{$name};
- $self->{props}{$name}{name} = $name;
- $self->{props}{$name}{type} = shift;
- $self->{props}{$name}{value} = shift;
- return $x;
- }
- return $self->{props}{$name};
-}
-1;
-package Rsyslog::OPB;
-use base qw(Rsyslog::Serializable);
-sub type { 'OPB' }
-sub cver { 1 }
-sub new { shift->SUPER::new(@_) }
-1;
-package Rsyslog::Obj;
-use base qw(Rsyslog::Serializable);
-sub type { 'Obj' }
-sub cver { 1 }
-sub new { shift->SUPER::new(@_) }
-1;
+#!/usr/bin/perl -w +# recover rsyslog disk queue index (.qi) from queue files (.nnnnnnnn). +# +# See: +# runtime/queue.c: qqueuePersist() +# runtime/queue.c: qqueueTryLoadPersistedInfo() +# +# kaiwang.chen@gmail.com 2012-03-14 +# +use strict; +use Getopt::Long; + +my %opt = (); +GetOptions(\%opt,"spool|w=s","basename|f=s","digits|d=i","help!"); +if ($opt{help}) { + print "Usage: +\t$0 -w WorkDirectory -f QueueFileName -d 8 > QueueFileName.qi +"; + exit; +} + +# runtime/queue.c: qConstructDisk() +my $iMaxFiles = 10000000; # 0+"1".( "0"x($opt{digits} - 1)); + +# get the list of queue files, spool directory excluded +my $re = qr/^\Q$opt{basename}\E\.\d{$opt{digits}}$/; +opendir(DIR, $opt{spool}) or die "can’t open spool: $!"; +my @qf = grep { /$re/ && -f "$opt{spool}/$_" } readdir(DIR); +closedir DIR; + +# ensure order and continuity +@qf = sort @qf; +my ($head) = ($qf[0] =~ /(\d+)$/); +my ($tail) = ($qf[-1] =~ /(\d+)$/); +$head += 0; +$tail += 0; +if ($tail-$head+1 != @qf || $tail > $iMaxFiles) { + die "broken queue: missing file(s) or wrong tail\n"; +} + +# collect some counters about the queue, assuming all are unprocessed entries. +my $sizeOnDisk = 0; +my $iQueueSize = 0; +chdir($opt{spool}) or die "can't chdir to spool: $!"; +print STDERR "traversing ". @qf ." files, please wait...\n"; +for (@qf) { + open FH, "<", $_ or die "can't read queue file $_\n"; + $sizeOnDisk += (stat FH)[7]; + while (<FH>) { + $iQueueSize++ if /^<Obj/; # runtime/msg.c: MsgSerialize() + } + close FH; +} +# happen to reuse last stat +my $iCurrOffs_Write = (stat(_))[7]; + +# runtime/queue.c: qqueuePersist() +my $qqueue = Rsyslog::OPB->new("qqueue",1); +$qqueue->property("iQueueSize", "INT", $iQueueSize); +$qqueue->property("tVars.disk.sizeOnDisk", "INT64", $sizeOnDisk); +$qqueue->property("tVars.disk.bytesRead", "INT64", 0); + +# runtime/stream.h: strmType_t +my $STREAMTYPE_FILE_CIRCULAR = 1; +# runtime/stream.h: strmMode_t +my $STREAMMODE_READ = 1; +my $STREAMMODE_WRITE_APPEND = 4; + +# runtime/stream.c: strmSerialize() +# write to end +my $strm_Write = Rsyslog::Obj->new("strm",1); +$strm_Write->property( "iCurrFNum", "INT", $tail); +$strm_Write->property( "pszFName", "PSZ", $opt{basename}); +$strm_Write->property( "iMaxFiles", "INT", $iMaxFiles); +$strm_Write->property( "bDeleteOnClose", "INT", 0); +$strm_Write->property( "sType", "INT", $STREAMTYPE_FILE_CIRCULAR); +$strm_Write->property("tOperationsMode", "INT", $STREAMMODE_WRITE_APPEND); +$strm_Write->property( "tOpenMode", "INT", 0600); +$strm_Write->property( "iCurrOffs","INT64", $iCurrOffs_Write); +# read from head +my $strm_ReadDel = Rsyslog::Obj->new("strm",1); +$strm_ReadDel->property( "iCurrFNum", "INT", $head); +$strm_ReadDel->property( "pszFName", "PSZ", $opt{basename}); +$strm_ReadDel->property( "iMaxFiles", "INT", $iMaxFiles); +$strm_ReadDel->property( "bDeleteOnClose", "INT", 1); +$strm_ReadDel->property( "sType", "INT", $STREAMTYPE_FILE_CIRCULAR); +$strm_ReadDel->property("tOperationsMode", "INT", $STREAMMODE_READ); +$strm_ReadDel->property( "tOpenMode", "INT", 0600); +$strm_ReadDel->property( "iCurrOffs","INT64", 0); + +# .qi +print $qqueue->serialize(); +print $strm_Write->serialize(); +print $strm_ReadDel->serialize(); + +exit; +#----------------------------------------------------------------------------- + +package Rsyslog::Serializable; +# runtime/obj.c +sub COOKIE_OBJLINE { '<' } +sub COOKIE_PROPLINE { '+' } +sub COOKIE_ENDLINE { '>' } +sub COOKIE_BLANKLINE { '.' } +# VARTYPE(short_ptype) +sub VARTYPE { + my ($t) = @_; + # runtime/obj-types.h: propType_t + my $ptype = "PROPTYPE_".$t; + # runtime/var.h: varType_t + my %vm = ( + VARTYPE_NONE => 0, + VARTYPE_STR => 1, + VARTYPE_NUMBER => 2, + VARTYPE_SYSLOGTIME => 3, + ); + # runtime/obj.c: SerializeProp() + my %p2v = ( + #PROPTYPE_NONE => "", + PROPTYPE_PSZ => "VARTYPE_STR", + PROPTYPE_SHORT => "VARTYPE_NUMBER", + PROPTYPE_INT => "VARTYPE_NUMBER", + PROPTYPE_LONG => "VARTYPE_NUMBER", + PROPTYPE_INT64 => "VARTYPE_NUMBER", + PROPTYPE_CSTR => "VARTYPE_STR", + #PROPTYPE_SYSLOGTIME => "VARTYPE_SYSLOGTIME", + ); + my $vtype = $p2v{$ptype}; + unless ($vtype) { + die "property type $t is not supported!\n"; + } + return $vm{$vtype}; +} +sub serialize { + my $self = shift; + # runtime/obj.c: objSerializeHeader() + my $x = COOKIE_OBJLINE(); + $x .= join(":", $self->type(), $self->cver(), $self->id(), $self->version()); + $x .= ":\n"; + for ( values %{$self->{props}} ) { + # runtime/obj.c: SerializeProp() + $x .= COOKIE_PROPLINE(); + $x .= join(":", + $_->{name}, + VARTYPE($_->{type}), + length($_->{value}), + $_->{value}); + $x .= ":\n"; + } + # runtime/obj.c: EndSerialize() + $x .= COOKIE_ENDLINE() . "End\n"; + $x .= COOKIE_BLANKLINE() . "\n"; +} +# constructor: new(id,version) +sub new { + my ($class, $id, $version) = @_; + $class = ref $class if ref $class; + bless { + id => $id, + version => $version, + props => {}, + }, $class; +} +sub id { + my $self = shift; + if (@_) { + my $x = $self->{id}; + $self->{id} = shift; + return $x; + } + return $self->{id}; +} +sub version { + my $self = shift; + if (@_) { + my $x = $self->{version}; + $self->{version} = shift; + return $x; + } + return $self->{version}; +} +# property(name, type, value) +sub property { + my $self = shift; + my $name = shift; + if (@_) { + my $x = $self->{props}{$name}; + $self->{props}{$name}{name} = $name; + $self->{props}{$name}{type} = shift; + $self->{props}{$name}{value} = shift; + return $x; + } + return $self->{props}{$name}; +} +1; +package Rsyslog::OPB; +use base qw(Rsyslog::Serializable); +sub type { 'OPB' } +sub cver { 1 } +sub new { shift->SUPER::new(@_) } +1; +package Rsyslog::Obj; +use base qw(Rsyslog::Serializable); +sub type { 'Obj' } +sub cver { 1 } +sub new { shift->SUPER::new(@_) } +1; diff --git a/tools/rscryutil.1 b/tools/rscryutil.1 new file mode 100644 index 0000000..cd6dc9d --- /dev/null +++ b/tools/rscryutil.1 @@ -0,0 +1,202 @@ +.\" Man page generated from reStructeredText. +. +.TH RSCRYUTIL 1 "2013-04-15" "" "" +.SH NAME +rscryutil \- Manage Encrypted Log Files +. +.nr rst2man-indent-level 0 +. +.de1 rstReportMargin +\\$1 \\n[an-margin] +level \\n[rst2man-indent-level] +level margin: \\n[rst2man-indent\\n[rst2man-indent-level]] +- +\\n[rst2man-indent0] +\\n[rst2man-indent1] +\\n[rst2man-indent2] +.. +.de1 INDENT +.\" .rstReportMargin pre: +. RS \\$1 +. nr rst2man-indent\\n[rst2man-indent-level] \\n[an-margin] +. nr rst2man-indent-level +1 +.\" .rstReportMargin post: +.. +.de UNINDENT +. RE +.\" indent \\n[an-margin] +.\" old: \\n[rst2man-indent\\n[rst2man-indent-level]] +.nr rst2man-indent-level -1 +.\" new: \\n[rst2man-indent\\n[rst2man-indent-level]] +.in \\n[rst2man-indent\\n[rst2man-indent-level]]u +.. +.SH SYNOPSIS +.sp +.nf +.ft C +rscryutil [OPTIONS] [FILE] ... +.ft P +.fi +.SH DESCRIPTION +.sp +This tool performs various operations on encrypted log files. +Most importantly, it provides the ability to decrypt them. +.SH OPTIONS +.INDENT 0.0 +.TP +.B \-d, \-\-decrypt +Select decryption mode. This is the default mode. +.TP +.BI \-W, \-\-write\-keyfile \ <file> +Utility function to write a key to a keyfile. The key can be obtained +via any method. +.TP +.B \-v, \-\-verbose +Select verbose mode. +.TP +.B \-f, \-\-force +Forces operations that otherwise would fail. +.TP +.BI \-k, \-\-keyfile \ <file> +Reads the key from <file>. File _must_ contain the key, only, no headers +or other meta information. Keyfiles can be generated via the +\fI\-\-write\-keyfile\fP option. +.TP +.BI \-p, \-\-key\-program \ <path\-to\-program> +In this mode, the key is provided by a so\-called "key program". This program +is executed and must return the key to (as well as some meta information) +via stdout. The core idea of key programs is that using this interface the +user can implement as complex (and secure) method to obtain keys as +desired, all without the need to make modifications to rsyslog. +.TP +.BI \-K, \-\-key \ <KEY> +TESTING AID, NOT FOR PRODUCTION USE. This uses the KEY specified +on the command line. This is the actual key, and as such this mode +is highly insecure. However, it can be useful for intial testing +steps. This option may be removed in the future. +.TP +.BI \-a, \-\-algo \ <algo> +Sets the encryption algorightm (cipher) to be used. See below +for supported algorithms. The default is "AES128". +.TP +.BI \-m, \-\-mode \ <mode> +Sets the ciphermode to be used. See below for supported modes. +The default is "CBC". +.TP +.BI \-r, \-\-generate\-random\-key \ <bytes> +Generates a random key of length <bytes>. This option is +meant to be used together with \fI\-\-write\-keyfile\fP (and it is hard +to envision any other valid use for it). +.UNINDENT +.SH OPERATION MODES +.sp +The operation mode specifies what exactly the tool does with the provided +files. The default operation mode is "dump", but this may change in the future. +Thus, it is recommended to always set the operations mode explicitely. If +multiple operations mode are set on the command line, results are +unpredictable. +.SS decrypt +.sp +The provided log files are decrypted. Note that the \fI.encinfo\fP side files +must exist and be accessible in order for decryption to to work. +.SS write\-keyfile +.sp +In this mode no log files are processed; thus it is an error to specify +any on the command line. The specified keyfile is written. The key itself +is obtained via the usual key commands. If \fI\-\-keyfile\fP is used, that +file is effectively copied. +.sp +For security reasons, existing key files are _not_ overwritten. To permit +this, specify the \fI\-\-force\fP option. When doing so, keep in mind that lost +keys cannot be recovered and data encrypted with them may also be considered +lost. +.sp +Keyfiles are always created with 0400 permission, that is read access for only +the user. An exception is when an existing file is overwritten via the +\fI\-\-force\fP option, in which case the former permissions still apply. +.SH EXIT CODES +.sp +The command returns an exit code of 0 if everything went fine, and some +other code in case of failures. +.SH SUPPORTED ALGORITHMS +.sp +We basically support what libgcrypt supports. This is: +.INDENT 0.0 +.INDENT 3.5 +3DES +CAST5 +BLOWFISH +AES128 +AES192 +AES256 +TWOFISH +TWOFISH128 +ARCFOUR +DES +SERPENT128 +SERPENT192 +SERPENT256 +RFC2268_40 +SEED +CAMELLIA128 +CAMELLIA192 +CAMELLIA256 +.UNINDENT +.UNINDENT +.SH SUPPORTED CIPHER MODES +.sp +We basically support what libgcrypt supports. This is: +.INDENT 0.0 +.INDENT 3.5 +ECB +CFB +CBC +STREAM +OFB +CTR +AESWRAP +.UNINDENT +.UNINDENT +.SH EXAMPLES +.sp +\fBrscryutil logfile\fP +.sp +Decrypts "logfile" and sends data to stdout. +.sp +\fBrscryutil \-\-generate\-random\-key 16 \-\-keyfile /some/secured/path/keyfile\fP +.sp +Generates random key and stores it in the specified keyfile. +.SH LOG SIGNATURES +.sp +Encrypted log files can be used together with signing. To verify such a file, +it must be decrypted first, and the verification tool \fBrsgtutil(1)\fP must be +run on the decrypted file. +.SH SECURITY CONSIDERATIONS +.sp +Specifying keys directly on the command line (\fI\-\-key\fP option) is very +insecure and should +not be done, except for testing purposes with test keys. Even then it is +recommended to use keyfiles, which are also easy to handle during testing. +Keep in mind that command history is usally be kept by bash and can also +easily be monitored. +.sp +Local keyfiles are also a security risk. At a minimum, they should be +used with very restrictive file permissions. For this reason, +the \fIrscryutil\fP tool creates them with read permissions for the user, +only, no matter what umask is set to. +.sp +When selecting cipher algorithms and modes, care needs to be taken. The +defaults should be reasonable safe to use, but this tends to change over +time. Keep up with the most current crypto recommendations. +.SH SEE ALSO +.sp +\fBrsgtutil(1)\fP, \fBrsyslogd(8)\fP +.SH COPYRIGHT +.sp +This page is part of the \fIrsyslog\fP project, and is available under +LGPLv2. +.SH AUTHOR +Rainer Gerhards <rgerhards@adiscon.com> +.\" Generated by docutils manpage writer. +.\" +. diff --git a/tools/rscryutil.c b/tools/rscryutil.c index 2591b2c..701bd1a 100644 --- a/tools/rscryutil.c +++ b/tools/rscryutil.c @@ -120,8 +120,8 @@ eiGetIV(FILE *eifp, char *iv, size_t leniv) } valueLen = strlen(value); if(valueLen/2 != leniv) { - fprintf(stderr, "length of IV is %d, expected %d\n", - valueLen/2, leniv); + fprintf(stderr, "length of IV is %lld, expected %lld\n", + (long long) valueLen/2, (long long) leniv); r = 1; goto done; } @@ -170,8 +170,8 @@ initCrypt(FILE *eifp) blkLength = gcry_cipher_get_algo_blklen(cry_algo); if(blkLength > sizeof(iv)) { - fprintf(stderr, "internal error[%s:%d]: block length %d too large for " - "iv buffer\n", __FILE__, __LINE__, blkLength); + fprintf(stderr, "internal error[%s:%d]: block length %lld too large for " + "iv buffer\n", __FILE__, __LINE__, (long long) blkLength); r = 1; goto done; } if((r = eiGetIV(eifp, iv, blkLength)) != 0) goto done; @@ -179,8 +179,8 @@ initCrypt(FILE *eifp) size_t keyLength = gcry_cipher_get_algo_keylen(cry_algo); if(strlen(cry_key) != keyLength) { fprintf(stderr, "invalid key length; key is %u characters, but " - "exactly %u characters are required\n", cry_keylen, - keyLength); + "exactly %llu characters are required\n", cry_keylen, + (long long unsigned) keyLength); r = 1; goto done; } diff --git a/tools/rsgtutil.1 b/tools/rsgtutil.1 new file mode 100644 index 0000000..5543a11 --- /dev/null +++ b/tools/rsgtutil.1 @@ -0,0 +1,179 @@ +.\" Man page generated from reStructeredText. +. +.TH RSGTUTIL 1 "2013-03-25" "" "" +.SH NAME +rsgtutil \- Manage (GuardTime) Signed Log Files +. +.nr rst2man-indent-level 0 +. +.de1 rstReportMargin +\\$1 \\n[an-margin] +level \\n[rst2man-indent-level] +level margin: \\n[rst2man-indent\\n[rst2man-indent-level]] +- +\\n[rst2man-indent0] +\\n[rst2man-indent1] +\\n[rst2man-indent2] +.. +.de1 INDENT +.\" .rstReportMargin pre: +. RS \\$1 +. nr rst2man-indent\\n[rst2man-indent-level] \\n[an-margin] +. nr rst2man-indent-level +1 +.\" .rstReportMargin post: +.. +.de UNINDENT +. RE +.\" indent \\n[an-margin] +.\" old: \\n[rst2man-indent\\n[rst2man-indent-level]] +.nr rst2man-indent-level -1 +.\" new: \\n[rst2man-indent\\n[rst2man-indent-level]] +.in \\n[rst2man-indent\\n[rst2man-indent-level]]u +.. +.SH SYNOPSIS +.sp +.nf +.ft C +rsgtutil [OPTIONS] [FILE] ... +.ft P +.fi +.SH DESCRIPTION +.sp +This tool performs various maintenance operations on signed log files. +It specifically supports the GuardTime signature provider. +.sp +The \fIrsgtutil\fP tool is the primary tool to verify log file signatures, +dump signature file contents and carry out other maintenance operations. +The tool offers different operation modes, which are selected via +command line options. +.sp +The processing of multiple files is permitted. Depending on operation +mode, either the signature file or the base log file must be specified. +Within a single call, only a single operations mode is permitted. To +use different modes on different files, multiple calles, one for each +mode, must be made. +.sp +If no file is specified on the command line, stdin is used instead. Note +that not all operation modes support stdin. +.SH OPTIONS +.INDENT 0.0 +.TP +.B \-D, \-\-dump +Select "dump" operations mode. +.TP +.B \-t, \-\-verify +Select "verify" operations mode. +.TP +.B \-T, \-\-detect\-file\-type +Select "detect\-file\-type" operations mode. +.TP +.B \-B, \-\-show\-sigblock\-params +Select "show\-sigblock\-params" operations mode. +.TP +.B \-s, \-\-show\-verified +Prints out information about correctly verified blocks (by default, only +errors are printed). +.TP +.B \-v, \-\-verbose +Select verbose mode. Most importantly, hashes and signatures are printed +in full length (can be \fBvery\fP lengthy) rather than the usual abbreviation. +.TP +.B \-e, \-\-extend +Select extend mode. This extends the RFC3161 signatures. Note that this +mode also implies a full verification. If there are verify errors, extending +will also fail. +.TP +.BI \-P \ <URL>, \ \-\-publications\-server \ <URL> +Sets the publications server. If not set but required by the operation a +default server is used. The default server is not necessarily optimal +in regard to performance and reliability. +.UNINDENT +.SH OPERATION MODES +.sp +The operation mode specifies what exactly the tool does with the provided +files. The default operation mode is "dump", but this may change in the future. +Thus, it is recommended to always set the operations mode explicitely. If +multiple operations mode are set on the command line, results are +unpredictable. +.SS dump +.sp +The provided \fIsignature\fP files are dumped. For each top\-level record, the*u +type code is printed as well as q short description. If there is additional +information available, it will be printed in tab\-indented lines below the +main record dump. The actual \fIlog\fP files need not to be present. +.SS verify +.sp +This mode does not work with stdin. On the command line, the \fIlog\fP file names +are specified. The corresponding \fIsignature\fP files (ending on ".gtsig") must also +be preset at the same location as the log file. In verify mode, both the log +and signature file is read and the validity of the log file checked. If verification +errors are detected these are printed and processing of the file aborted. By default, +each file is verified individually, without taking cross\-file hash chains into +account (so the order of files on the command line does not matter). +.sp +Note that the actual amount of what can be verified depends on the parameters with +which the signature file was written. If record and tree hashes are present, they +will be verified and thus fine\-granular error reporting is possible. If they are +not present, only the block signature itself is verified. +.sp +By default, only errors are printed. To also print successful verifications, use the +\fB\-\-show\-verified\fP option. +.SS extend +.sp +This extends the RFC3161 signatures. This includes a full verification +of the file. If there are verification errors, extending will also fail. +Note that a signature can only be extended when the required hash has been +published. Currently, these hashes are created at the 15th of each month at +0:00hrs UTC. It takes another few days to get them finally published. As such, +it can be assumed that extending is only possible after this happend (which +means it may take slightly above a month). +.sp +To prevent data corruption, a copy of the signature file is created during +extension. So there must be enough disk space available for both files, +otherwise the operation will fail. If the log file is named logfile, the +signature file is logfile.gtsig and the temporary work file is named +logfile.gtsig.new. When extending finished successfully, the original +signature file (logfile.gtsig in our example) is renamed with the .old +postfix (logfile.gtsig.old) and the temporary file written under the +original name. The .old file can be deleted. It is just kept as a +precaution to prevent signature loss. Note that any already existing +.old or .new files are overwritten by these operations. +.SS detect\-file\-type +.sp +This mode is used to detect the type of some well\-know files used inside the +signature system. The detection is based on the file header. This mode is +primarily a debug aid. +.SS show\-sigblock\-params +.sp +This mode is used to print signature block parameters. It is similar to \fIdump\fP +mode, but will ignore everything except signature blocks. Also, some additional +meta information is printed. This mode is primarily a debug aid. +.SH EXIT CODES +.sp +The command returns an exit code of 0 if everything went fine, and some +other code in case of failures. +.SH EXAMPLES +.sp +\fBrsgtutil \-\-verify logfile\fP +.sp +This verifies the file "logfile" via its associated signature file +"logfile.gtsig". If errors are detected, these are reported to stderr. +Otherwise, rsgtutil terminates without messages. +.sp +\fBrsgtutil \-\-dump logfile.gtsig\fP +.sp +This dumps the content of the signature file "logfile.gtsig". The +actual log file is not being processed and does not even need to be +present. +.SH SEE ALSO +.sp +\fBrsyslogd(8)\fP +.SH COPYRIGHT +.sp +This page is part of the \fIrsyslog\fP project, and is available under +LGPLv2. +.SH AUTHOR +Rainer Gerhards <rgerhards@adiscon.com> +.\" Generated by docutils manpage writer. +.\" +. diff --git a/tools/rsgtutil.c b/tools/rsgtutil.c index 095b806..9808268 100644 --- a/tools/rsgtutil.c +++ b/tools/rsgtutil.c @@ -74,7 +74,7 @@ dumpFile(char *name) if(fp != stdin) fclose(fp); return; -err: fprintf(stderr, "error %d processing file %s\n", r, name); +err: fprintf(stderr, "error %d (%s) processing file %s\n", r, RSGTE2String(r), name); } static void @@ -103,7 +103,7 @@ showSigblkParams(char *name) ++blkCnt; rsgt_printBLOCK_SIG(stdout, bs, verbose); printf("\t***META INFORMATION:\n"); - printf("\tBlock Nbr in File...: %llu\n", blkCnt); + printf("\tBlock Nbr in File...: %llu\n", (long long unsigned) blkCnt); printf("\tHas Record Hashes...: %d\n", bHasRecHashes); printf("\tHas Tree Hashes.....: %d\n", bHasIntermedHashes); } @@ -113,7 +113,7 @@ showSigblkParams(char *name) return; err: if(r != RSGTE_EOF) - fprintf(stderr, "error %d processing file %s\n", r, name); + fprintf(stderr, "error %d (%s) processing file %s\n", r, RSGTE2String(r), name); } static void @@ -145,7 +145,7 @@ detectFileType(char *name) if(fp != stdin) fclose(fp); return; -err: fprintf(stderr, "error %d processing file %s\n", r, name); +err: fprintf(stderr, "error %d (%s) processing file %s\n", r, RSGTE2String(r), name); } static inline int @@ -259,8 +259,17 @@ verify(char *name) if(bs != NULL) rsgt_objfree(0x0902, bs); if((r = rsgt_getBlockParams(sigfp, 1, &bs, &bHasRecHashes, - &bHasIntermedHashes)) != 0) + &bHasIntermedHashes)) != 0) { + if(ectx.blkNum == 0) { + fprintf(stderr, "EOF before finding any signature block - " + "is the file still open and being written to?\n"); + } else { + if(verbose) + fprintf(stderr, "EOF after signature block %lld\n", + (long long unsigned) ectx.blkNum); + } goto done; + } rsgt_vrfyBlkInit(gf, bs, bHasRecHashes, bHasIntermedHashes); ectx.recNum = 0; ++ectx.blkNum; @@ -327,7 +336,7 @@ done: return; err: - fprintf(stderr, "error %d processing file %s\n", r, name); + fprintf(stderr, "error %d (%s) processing file %s\n", r, RSGTE2String(r), name); if(logfp != NULL) fclose(logfp); if(sigfp != NULL) diff --git a/tools/rsyslogd.8 b/tools/rsyslogd.8 index 620006f..6d295a5 100644 --- a/tools/rsyslogd.8 +++ b/tools/rsyslogd.8 @@ -1,7 +1,7 @@ .\" Copyright 2004-2008 Rainer Gerhards and Adiscon for the rsyslog modifications .\" May be distributed under the GNU General Public License .\" -.TH RSYSLOGD 8 "16 October 2012" "Version 6.4.3" "Linux System Administration" +.TH RSYSLOGD 8 "27 May 2014" "Version 8.3.3" "Linux System Administration" .SH NAME rsyslogd \- reliable and extended syslogd .SH SYNOPSIS @@ -191,6 +191,10 @@ is specified and the host logging resolves to satu.infodrom.north.de no domain would be cut, you will have to specify two domains like: .BR "\-s north.de:infodrom.north.de" . .TP +.BI "\-S ip_address" "local client source IP" +rsyslogd uses ip_address as local client address while connecting +to remote logserver. Currently used by omrelp only and only with tcp. +.TP .BI "\-u " "userlevel" This is a "catch all" option for some very seldomly-used user settings. The "userlevel" variable selects multiple things. Add the specific values @@ -235,16 +239,6 @@ kill -HUP $(cat /var/run/rsyslogd.pid) This lets .B rsyslogd perform close all open files. -Also, in v3 a full restart will be done in order to read changed configuration files. -Note that this means a full rsyslogd restart is done. This has, among others, -the consequence that TCP and other connections are torn down. Also, if any -queues are not running in disk assisted mode or are not set to persist data -on shutdown, queue data is lost. HUPing rsyslogd is an extremely expensive -operation and should only be done when actually necessary. Actually, it is -a rsyslgod stop immediately followed by a restart. Future versions will remove -this restart functionality of HUP (it will go away in v5). So it is advised to use -HUP only for closing files, and a "real restart" (e.g. /etc/rc.d/rsyslogd restart) -to activate configuration changes. .TP .B TERM ", " INT ", " QUIT .B Rsyslogd diff --git a/tools/rsyslogd.c b/tools/rsyslogd.c new file mode 100644 index 0000000..a0fe02c --- /dev/null +++ b/tools/rsyslogd.c @@ -0,0 +1,1324 @@ +/* This is the main rsyslogd file. + * It contains code * that is known to be validly under ASL 2.0, + * because it was either written from scratch by me (rgerhards) or + * contributors who agreed to ASL 2.0. + * + * Copyright 2004-2014 Rainer Gerhards and Adiscon + * + * This file is part of rsyslog. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * -or- + * see COPYING.ASL20 in the source distribution + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +#include "config.h" +#include "rsyslog.h" + +#include <signal.h> +#include <liblogging/stdlog.h> +#ifdef OS_SOLARIS +# include <errno.h> +#else +# include <sys/errno.h> +#endif +#include "sd-daemon.h" + +#include "wti.h" +#include "ratelimit.h" +#include "parser.h" +#include "linkedlist.h" +#include "ruleset.h" +#include "action.h" +#include "iminternal.h" +#include "errmsg.h" +#include "threads.h" +#include "dnscache.h" +#include "prop.h" +#include "unicode-helper.h" +#include "net.h" +#include "errmsg.h" +#include "glbl.h" +#include "debug.h" +#include "srUtils.h" +#include "rsconf.h" +#include "cfsysline.h" +#include "datetime.h" +#include "dirty.h" +#include "janitor.h" + +DEFobjCurrIf(obj) +DEFobjCurrIf(prop) +DEFobjCurrIf(parser) +DEFobjCurrIf(ruleset) +DEFobjCurrIf(net) +DEFobjCurrIf(errmsg) +DEFobjCurrIf(rsconf) +DEFobjCurrIf(module) +DEFobjCurrIf(datetime) +DEFobjCurrIf(glbl) + +/* imports from syslogd.c, these should go away over time (as we + * migrate/replace more and more code to ASL 2.0). + */ +extern int bHadHUP; +extern int bFinished; +extern int doFork; +extern pid_t ppid; +extern char *PidFile; + +extern int realMain(int argc, char **argv); +extern rsRetVal queryLocalHostname(void); +void syslogdInit(void); +void syslogd_die(void); +void syslogd_releaseClassPointers(void); +void syslogd_sighup_handler(); +char **syslogd_crunch_list(char *list); +rsRetVal syslogd_doGlblProcessInit(void); +rsRetVal syslogd_obtainClassPointers(void); +/* end syslogd.c imports */ +extern int yydebug; /* interface to flex */ + + +/* forward definitions */ +void rsyslogd_submitErrMsg(const int severity, const int iErr, const uchar *msg); + + +/* global data items */ +rsconf_t *ourConf = NULL; /* our config object */ +int MarkInterval = 20 * 60; /* interval between marks in seconds - read-only after startup */ +ratelimit_t *dflt_ratelimiter = NULL; /* ratelimiter for submits without explicit one */ +uchar *ConfFile = (uchar*) "/etc/rsyslog.conf"; +int bHaveMainQueue = 0;/* set to 1 if the main queue - in queueing mode - is available + * If the main queue is either not yet ready or not running in + * queueing mode (mode DIRECT!), then this is set to 0. + */ +qqueue_t *pMsgQueue = NULL; /* default main message queue */ +prop_t *pInternalInputName = NULL; /* there is only one global inputName for all internally-generated messages */ +ratelimit_t *internalMsg_ratelimiter = NULL; /* ratelimiter for rsyslog-own messages */ +int send_to_all = 0; /* send message to all IPv4/IPv6 addresses */ + +static struct queuefilenames_s { + struct queuefilenames_s *next; + uchar *name; +} *queuefilenames = NULL; + + +void +rsyslogd_usage(void) +{ + fprintf(stderr, "usage: rsyslogd [options]\n" + "use \"man rsyslogd\" for details. To run rsyslog " + "interactively, use \"rsyslogd -n\"" + "to run it in debug mode use \"rsyslogd -dn\"\n" + "For further information see http://www.rsyslog.com/doc\n"); + exit(1); /* "good" exit - done to terminate usage() */ +} + + +/* print version and compile-time setting information */ +static void +printVersion(void) +{ + printf("rsyslogd %s, ", VERSION); + printf("compiled with:\n"); +#ifdef FEATURE_REGEXP + printf("\tFEATURE_REGEXP:\t\t\t\tYes\n"); +#else + printf("\tFEATURE_REGEXP:\t\t\t\tNo\n"); +#endif +#if defined(SYSLOG_INET) && defined(USE_GSSAPI) + printf("\tGSSAPI Kerberos 5 support:\t\tYes\n"); +#else + printf("\tGSSAPI Kerberos 5 support:\t\tNo\n"); +#endif +#ifndef NDEBUG + printf("\tFEATURE_DEBUG (debug build, slow code):\tYes\n"); +#else + printf("\tFEATURE_DEBUG (debug build, slow code):\tNo\n"); +#endif +#ifdef HAVE_ATOMIC_BUILTINS + printf("\t32bit Atomic operations supported:\tYes\n"); +#else + printf("\t32bit Atomic operations supported:\tNo\n"); +#endif +#ifdef HAVE_ATOMIC_BUILTINS64 + printf("\t64bit Atomic operations supported:\tYes\n"); +#else + printf("\t64bit Atomic operations supported:\tNo\n"); +#endif +#ifdef HAVE_JEMALLOC + printf("\tmemory allocator:\t\t\tjemalloc\n"); +#else + printf("\tmemory allocator:\t\t\tsystem default\n"); +#endif +#ifdef RTINST + printf("\tRuntime Instrumentation (slow code):\tYes\n"); +#else + printf("\tRuntime Instrumentation (slow code):\tNo\n"); +#endif +#ifdef USE_LIBUUID + printf("\tuuid support:\t\t\t\tYes\n"); +#else + printf("\tuuid support:\t\t\t\tNo\n"); +#endif +#ifdef HAVE_JSON_OBJECT_NEW_INT64 + printf("\tNumber of Bits in RainerScript integers: 64\n"); +#else + printf("\tNumber of Bits in RainerScript integers: 32 (due to too-old json-c lib)\n"); +#endif + printf("\nSee http://www.rsyslog.com for more information.\n"); +} + + + +void +rsyslogd_sigttin_handler() +{ + /* this is just a dummy to care for our sigttin input + * module cancel interface. The important point is that + * it actually does *nothing*. + */ +} + +rsRetVal +rsyslogd_InitStdRatelimiters(void) +{ + DEFiRet; + CHKiRet(ratelimitNew(&dflt_ratelimiter, "rsyslogd", "dflt")); + /* TODO: add linux-type limiting capability */ + CHKiRet(ratelimitNew(&internalMsg_ratelimiter, "rsyslogd", "internal_messages")); + ratelimitSetLinuxLike(internalMsg_ratelimiter, 5, 500); + /* TODO: make internalMsg ratelimit settings configurable */ +finalize_it: + RETiRet; +} + + +/* Method to initialize all global classes and use the objects that we need. + * rgerhards, 2008-01-04 + * rgerhards, 2008-04-16: the actual initialization is now carried out by the runtime + */ +rsRetVal +rsyslogd_InitGlobalClasses(void) +{ + DEFiRet; + char *pErrObj; /* tells us which object failed if that happens (useful for troubleshooting!) */ + + /* Intialize the runtime system */ + pErrObj = "rsyslog runtime"; /* set in case the runtime errors before setting an object */ + CHKiRet(rsrtInit(&pErrObj, &obj)); + rsrtSetErrLogger(rsyslogd_submitErrMsg); + + /* Now tell the system which classes we need ourselfs */ + pErrObj = "glbl"; + CHKiRet(objUse(glbl, CORE_COMPONENT)); + pErrObj = "errmsg"; + CHKiRet(objUse(errmsg, CORE_COMPONENT)); + pErrObj = "module"; + CHKiRet(objUse(module, CORE_COMPONENT)); + pErrObj = "datetime"; + CHKiRet(objUse(datetime, CORE_COMPONENT)); + pErrObj = "ruleset"; + CHKiRet(objUse(ruleset, CORE_COMPONENT)); + /*pErrObj = "conf"; + CHKiRet(objUse(conf, CORE_COMPONENT));*/ + pErrObj = "prop"; + CHKiRet(objUse(prop, CORE_COMPONENT)); + pErrObj = "parser"; + CHKiRet(objUse(parser, CORE_COMPONENT)); + pErrObj = "rsconf"; + CHKiRet(objUse(rsconf, CORE_COMPONENT)); + + /* intialize some dummy classes that are not part of the runtime */ + pErrObj = "action"; + CHKiRet(actionClassInit()); + pErrObj = "template"; + CHKiRet(templateInit()); + + /* TODO: the dependency on net shall go away! -- rgerhards, 2008-03-07 */ + pErrObj = "net"; + CHKiRet(objUse(net, LM_NET_FILENAME)); + + dnscacheInit(); + initRainerscript(); + ratelimitModInit(); + + /* we need to create the inputName property (only once during our lifetime) */ + CHKiRet(prop.Construct(&pInternalInputName)); + CHKiRet(prop.SetString(pInternalInputName, UCHAR_CONSTANT("rsyslogd"), sizeof("rsyslogd") - 1)); + CHKiRet(prop.ConstructFinalize(pInternalInputName)); + +finalize_it: + if(iRet != RS_RET_OK) { + /* we know we are inside the init sequence, so we can safely emit + * messages to stderr. -- rgerhards, 2008-04-02 + */ + fprintf(stderr, "Error during class init for object '%s' - failing...\n", pErrObj); + fprintf(stderr, "rsyslogd initializiation failed - global classes could not be initialized.\n" + "Did you do a \"make install\"?\n" + "Suggested action: run rsyslogd with -d -n options to see what exactly " + "fails.\n"); + } + + RETiRet; +} + +/* preprocess a batch of messages, that is ready them for actual processing. This is done + * as a first stage and totally in parallel to any other worker active in the system. So + * it helps us keep up the overall concurrency level. + * rgerhards, 2010-06-09 + */ +static inline rsRetVal +preprocessBatch(batch_t *pBatch, int *pbShutdownImmediate) { + prop_t *ip; + prop_t *fqdn; + prop_t *localName; + prop_t *propFromHost = NULL; + prop_t *propFromHostIP = NULL; + int bIsPermitted; + msg_t *pMsg; + int i; + rsRetVal localRet; + DEFiRet; + + for(i = 0 ; i < pBatch->nElem && !*pbShutdownImmediate ; i++) { + pMsg = pBatch->pElem[i].pMsg; + if((pMsg->msgFlags & NEEDS_ACLCHK_U) != 0) { + DBGPRINTF("msgConsumer: UDP ACL must be checked for message (hostname-based)\n"); + if(net.cvthname(pMsg->rcvFrom.pfrominet, &localName, &fqdn, &ip) != RS_RET_OK) + continue; + bIsPermitted = net.isAllowedSender2((uchar*)"UDP", + (struct sockaddr *)pMsg->rcvFrom.pfrominet, (char*)propGetSzStr(fqdn), 1); + if(!bIsPermitted) { + DBGPRINTF("Message from '%s' discarded, not a permitted sender host\n", + propGetSzStr(fqdn)); + pBatch->eltState[i] = BATCH_STATE_DISC; + } else { + /* save some of the info we obtained */ + MsgSetRcvFrom(pMsg, localName); + CHKiRet(MsgSetRcvFromIP(pMsg, ip)); + pMsg->msgFlags &= ~NEEDS_ACLCHK_U; + } + } + if((pMsg->msgFlags & NEEDS_PARSING) != 0) { + if((localRet = parser.ParseMsg(pMsg)) != RS_RET_OK) { + DBGPRINTF("Message discarded, parsing error %d\n", localRet); + pBatch->eltState[i] = BATCH_STATE_DISC; + } + } + } + +finalize_it: + if(propFromHost != NULL) + prop.Destruct(&propFromHost); + if(propFromHostIP != NULL) + prop.Destruct(&propFromHostIP); + RETiRet; +} + + +/* The consumer of dequeued messages. This function is called by the + * queue engine on dequeueing of a message. It runs on a SEPARATE + * THREAD. It receives an array of pointers, which it must iterate + * over. We do not do any further batching, as this is of no benefit + * for the main queue. + */ +static rsRetVal +msgConsumer(void __attribute__((unused)) *notNeeded, batch_t *pBatch, wti_t *pWti) +{ + DEFiRet; + assert(pBatch != NULL); + preprocessBatch(pBatch, pWti->pbShutdownImmediate); + ruleset.ProcessBatch(pBatch, pWti); +//TODO: the BATCH_STATE_COMM must be set somewhere down the road, but we +//do not have this yet and so we emulate -- 2010-06-10 +int i; + for(i = 0 ; i < pBatch->nElem && !*pWti->pbShutdownImmediate ; i++) { + pBatch->eltState[i] = BATCH_STATE_COMM; + } + RETiRet; +} + + +/* create a main message queue, now also used for ruleset queues. This function + * needs to be moved to some other module, but it is considered acceptable for + * the time being (remember that we want to restructure config processing at large!). + * rgerhards, 2009-10-27 + */ +rsRetVal createMainQueue(qqueue_t **ppQueue, uchar *pszQueueName, struct nvlst *lst) +{ + struct queuefilenames_s *qfn; + uchar *qfname = NULL; + static int qfn_renamenum = 0; + uchar qfrenamebuf[1024]; + DEFiRet; + + /* create message queue */ + CHKiRet_Hdlr(qqueueConstruct(ppQueue, ourConf->globals.mainQ.MainMsgQueType, ourConf->globals.mainQ.iMainMsgQueueNumWorkers, ourConf->globals.mainQ.iMainMsgQueueSize, msgConsumer)) { + /* no queue is fatal, we need to give up in that case... */ + errmsg.LogError(0, iRet, "could not create (ruleset) main message queue"); \ + } + /* name our main queue object (it's not fatal if it fails...) */ + obj.SetName((obj_t*) (*ppQueue), pszQueueName); + + if(lst == NULL) { /* use legacy parameters? */ + /* ... set some properties ... */ + # define setQPROP(func, directive, data) \ + CHKiRet_Hdlr(func(*ppQueue, data)) { \ + errmsg.LogError(0, NO_ERRCODE, "Invalid " #directive ", error %d. Ignored, running with default setting", iRet); \ + } + # define setQPROPstr(func, directive, data) \ + CHKiRet_Hdlr(func(*ppQueue, data, (data == NULL)? 0 : strlen((char*) data))) { \ + errmsg.LogError(0, NO_ERRCODE, "Invalid " #directive ", error %d. Ignored, running with default setting", iRet); \ + } + + if(ourConf->globals.mainQ.pszMainMsgQFName != NULL) { + /* check if the queue file name is unique, else emit an error */ + for(qfn = queuefilenames ; qfn != NULL ; qfn = qfn->next) { + dbgprintf("check queue file name '%s' vs '%s'\n", qfn->name, ourConf->globals.mainQ.pszMainMsgQFName ); + if(!ustrcmp(qfn->name, ourConf->globals.mainQ.pszMainMsgQFName)) { + snprintf((char*)qfrenamebuf, sizeof(qfrenamebuf), "%d-%s-%s", + ++qfn_renamenum, ourConf->globals.mainQ.pszMainMsgQFName, + (pszQueueName == NULL) ? "NONAME" : (char*)pszQueueName); + qfname = ustrdup(qfrenamebuf); + errmsg.LogError(0, NO_ERRCODE, "Error: queue file name '%s' already in use " + " - using '%s' instead", ourConf->globals.mainQ.pszMainMsgQFName, qfname); + break; + } + } + if(qfname == NULL) + qfname = ustrdup(ourConf->globals.mainQ.pszMainMsgQFName); + qfn = malloc(sizeof(struct queuefilenames_s)); + qfn->name = qfname; + qfn->next = queuefilenames; + queuefilenames = qfn; + } + + setQPROP(qqueueSetMaxFileSize, "$MainMsgQueueFileSize", ourConf->globals.mainQ.iMainMsgQueMaxFileSize); + setQPROP(qqueueSetsizeOnDiskMax, "$MainMsgQueueMaxDiskSpace", ourConf->globals.mainQ.iMainMsgQueMaxDiskSpace); + setQPROP(qqueueSetiDeqBatchSize, "$MainMsgQueueDequeueBatchSize", ourConf->globals.mainQ.iMainMsgQueDeqBatchSize); + setQPROPstr(qqueueSetFilePrefix, "$MainMsgQueueFileName", qfname); + setQPROP(qqueueSetiPersistUpdCnt, "$MainMsgQueueCheckpointInterval", ourConf->globals.mainQ.iMainMsgQPersistUpdCnt); + setQPROP(qqueueSetbSyncQueueFiles, "$MainMsgQueueSyncQueueFiles", ourConf->globals.mainQ.bMainMsgQSyncQeueFiles); + setQPROP(qqueueSettoQShutdown, "$MainMsgQueueTimeoutShutdown", ourConf->globals.mainQ.iMainMsgQtoQShutdown ); + setQPROP(qqueueSettoActShutdown, "$MainMsgQueueTimeoutActionCompletion", ourConf->globals.mainQ.iMainMsgQtoActShutdown); + setQPROP(qqueueSettoWrkShutdown, "$MainMsgQueueWorkerTimeoutThreadShutdown", ourConf->globals.mainQ.iMainMsgQtoWrkShutdown); + setQPROP(qqueueSettoEnq, "$MainMsgQueueTimeoutEnqueue", ourConf->globals.mainQ.iMainMsgQtoEnq); + setQPROP(qqueueSetiHighWtrMrk, "$MainMsgQueueHighWaterMark", ourConf->globals.mainQ.iMainMsgQHighWtrMark); + setQPROP(qqueueSetiLowWtrMrk, "$MainMsgQueueLowWaterMark", ourConf->globals.mainQ.iMainMsgQLowWtrMark); + setQPROP(qqueueSetiDiscardMrk, "$MainMsgQueueDiscardMark", ourConf->globals.mainQ.iMainMsgQDiscardMark); + setQPROP(qqueueSetiDiscardSeverity, "$MainMsgQueueDiscardSeverity", ourConf->globals.mainQ.iMainMsgQDiscardSeverity); + setQPROP(qqueueSetiMinMsgsPerWrkr, "$MainMsgQueueWorkerThreadMinimumMessages", ourConf->globals.mainQ.iMainMsgQWrkMinMsgs); + setQPROP(qqueueSetbSaveOnShutdown, "$MainMsgQueueSaveOnShutdown", ourConf->globals.mainQ.bMainMsgQSaveOnShutdown); + setQPROP(qqueueSetiDeqSlowdown, "$MainMsgQueueDequeueSlowdown", ourConf->globals.mainQ.iMainMsgQDeqSlowdown); + setQPROP(qqueueSetiDeqtWinFromHr, "$MainMsgQueueDequeueTimeBegin", ourConf->globals.mainQ.iMainMsgQueueDeqtWinFromHr); + setQPROP(qqueueSetiDeqtWinToHr, "$MainMsgQueueDequeueTimeEnd", ourConf->globals.mainQ.iMainMsgQueueDeqtWinToHr); + + # undef setQPROP + # undef setQPROPstr + } else { /* use new style config! */ + qqueueSetDefaultsRulesetQueue(*ppQueue); + qqueueApplyCnfParam(*ppQueue, lst); + } + RETiRet; +} + +rsRetVal +startMainQueue(qqueue_t *pQueue) +{ + DEFiRet; + CHKiRet_Hdlr(qqueueStart(pQueue)) { + /* no queue is fatal, we need to give up in that case... */ + errmsg.LogError(0, iRet, "could not start (ruleset) main message queue"); \ + } + RETiRet; +} + + +/* this is a special function used to submit an error message. This + * function is also passed to the runtime library as the generic error + * message handler. -- rgerhards, 2008-04-17 + */ +void +rsyslogd_submitErrMsg(const int severity, const int iErr, const uchar *msg) +{ + logmsgInternal(iErr, LOG_SYSLOG|(severity & 0x07), msg, 0); +} + +static inline rsRetVal +submitMsgWithDfltRatelimiter(msg_t *pMsg) +{ + return ratelimitAddMsg(dflt_ratelimiter, NULL, pMsg); +} + + +/* This function logs a message to rsyslog itself, using its own + * internal structures. This means external programs (like the + * system journal) will never see this message. + */ +static rsRetVal +logmsgInternalSelf(const int iErr, const int pri, const size_t lenMsg, + const char *__restrict__ const msg, int flags) +{ + uchar pszTag[33]; + msg_t *pMsg; + DEFiRet; + + CHKiRet(msgConstruct(&pMsg)); + MsgSetInputName(pMsg, pInternalInputName); + MsgSetRawMsg(pMsg, (char*)msg, lenMsg); + MsgSetHOSTNAME(pMsg, glbl.GetLocalHostName(), ustrlen(glbl.GetLocalHostName())); + MsgSetRcvFrom(pMsg, glbl.GetLocalHostNameProp()); + MsgSetRcvFromIP(pMsg, glbl.GetLocalHostIP()); + MsgSetMSGoffs(pMsg, 0); + /* check if we have an error code associated and, if so, + * adjust the tag. -- rgerhards, 2008-06-27 + */ + if(iErr == NO_ERRCODE) { + MsgSetTAG(pMsg, UCHAR_CONSTANT("rsyslogd:"), sizeof("rsyslogd:") - 1); + } else { + size_t len = snprintf((char*)pszTag, sizeof(pszTag), "rsyslogd%d:", iErr); + pszTag[32] = '\0'; /* just to make sure... */ + MsgSetTAG(pMsg, pszTag, len); + } + pMsg->iFacility = pri2fac(pri); + pMsg->iSeverity = pri2sev(pri); + flags |= INTERNAL_MSG; + pMsg->msgFlags = flags; + + if(bHaveMainQueue == 0) { /* not yet in queued mode */ + iminternalAddMsg(pMsg); + } else { + /* we have the queue, so we can simply provide the + * message to the queue engine. + */ + ratelimitAddMsg(internalMsg_ratelimiter, NULL, pMsg); + } +finalize_it: + RETiRet; +} + + + +/* rgerhards 2004-11-09: the following is a function that can be used + * to log a message orginating from the syslogd itself. + */ +rsRetVal +logmsgInternal(int iErr, int pri, const uchar *const msg, int flags) +{ + size_t lenMsg; + unsigned i; + char *bufModMsg = NULL; /* buffer for modified message, should we need to modify */ + DEFiRet; + + /* we first do a path the remove control characters that may have accidently + * introduced (program error!). This costs performance, but we do not expect + * to be called very frequently in any case ;) -- rgerhards, 2013-12-19. + */ + lenMsg = ustrlen(msg); + for(i = 0 ; i < lenMsg ; ++i) { + if(msg[i] < 0x20 || msg[i] == 0x7f) { + if(bufModMsg == NULL) { + CHKmalloc(bufModMsg = strdup((char*) msg)); + } + bufModMsg[i] = ' '; + } + } + + if(bProcessInternalMessages) { + CHKiRet(logmsgInternalSelf(iErr, pri, lenMsg, + (bufModMsg == NULL) ? (char*)msg : bufModMsg, + flags)); + } else { + stdlog_log(stdlog_hdl, pri2sev(pri), "%s", + (bufModMsg == NULL) ? (char*)msg : bufModMsg); + } + + /* we now check if we should print internal messages out to stderr. This was + * suggested by HKS as a way to help people troubleshoot rsyslog configuration + * (by running it interactively. This makes an awful lot of sense, so I add + * it here. -- rgerhards, 2008-07-28 + * Note that error messages can not be disabled during a config verify. This + * permits us to process unmodified config files which otherwise contain a + * supressor statement. + */ + if(((Debug == DEBUG_FULL || !doFork) && ourConf->globals.bErrMsgToStderr) || iConfigVerify) { + if(pri2sev(pri) == LOG_ERR) + fprintf(stderr, "rsyslogd: %s\n", (bufModMsg == NULL) ? (char*)msg : bufModMsg); + } + +finalize_it: + free(bufModMsg); + RETiRet; +} + +rsRetVal +submitMsg(msg_t *pMsg) +{ + return submitMsgWithDfltRatelimiter(pMsg); +} + +/* submit a message to the main message queue. This is primarily + * a hook to prevent the need for callers to know about the main message queue + * rgerhards, 2008-02-13 + */ +rsRetVal +submitMsg2(msg_t *pMsg) +{ + qqueue_t *pQueue; + ruleset_t *pRuleset; + DEFiRet; + + ISOBJ_TYPE_assert(pMsg, msg); + + pRuleset = MsgGetRuleset(pMsg); + pQueue = (pRuleset == NULL) ? pMsgQueue : ruleset.GetRulesetQueue(pRuleset); + + /* if a plugin logs a message during shutdown, the queue may no longer exist */ + if(pQueue == NULL) { + DBGPRINTF("submitMsg2() could not submit message - " + "queue does (no longer?) exist - ignored\n"); + FINALIZE; + } + + qqueueEnqMsg(pQueue, pMsg->flowCtlType, pMsg); + +finalize_it: + RETiRet; +} + +/* submit multiple messages at once, very similar to submitMsg, just + * for multi_submit_t. All messages need to go into the SAME queue! + * rgerhards, 2009-06-16 + */ +rsRetVal +multiSubmitMsg2(multi_submit_t *pMultiSub) +{ + qqueue_t *pQueue; + ruleset_t *pRuleset; + DEFiRet; + assert(pMultiSub != NULL); + + if(pMultiSub->nElem == 0) + FINALIZE; + + pRuleset = MsgGetRuleset(pMultiSub->ppMsgs[0]); + pQueue = (pRuleset == NULL) ? pMsgQueue : ruleset.GetRulesetQueue(pRuleset); + + /* if a plugin logs a message during shutdown, the queue may no longer exist */ + if(pQueue == NULL) { + DBGPRINTF("multiSubmitMsg() could not submit message - " + "queue does (no longer?) exist - ignored\n"); + FINALIZE; + } + + iRet = pQueue->MultiEnq(pQueue, pMultiSub); + pMultiSub->nElem = 0; + +finalize_it: + RETiRet; +} +rsRetVal +multiSubmitMsg(multi_submit_t *pMultiSub) /* backward compat. level */ +{ + return multiSubmitMsg2(pMultiSub); +} + + +/* flush multiSubmit, e.g. at end of read records */ +rsRetVal +multiSubmitFlush(multi_submit_t *pMultiSub) +{ + DEFiRet; + if(pMultiSub->nElem > 0) { + iRet = multiSubmitMsg2(pMultiSub); + } + RETiRet; +} + + +/* some support for command line option parsing. Any non-trivial options must be + * buffered until the complete command line has been parsed. This is necessary to + * prevent dependencies between the options. That, in turn, means we need to have + * something that is capable of buffering options and there values. The follwing + * functions handle that. + * rgerhards, 2008-04-04 + */ +typedef struct bufOpt { + struct bufOpt *pNext; + char optchar; + char *arg; +} bufOpt_t; +static bufOpt_t *bufOptRoot = NULL; +static bufOpt_t *bufOptLast = NULL; + +/* add option buffer */ +static rsRetVal +bufOptAdd(char opt, char *arg) +{ + DEFiRet; + bufOpt_t *pBuf; + + if((pBuf = MALLOC(sizeof(bufOpt_t))) == NULL) + ABORT_FINALIZE(RS_RET_OUT_OF_MEMORY); + + pBuf->optchar = opt; + pBuf->arg = arg; + pBuf->pNext = NULL; + + if(bufOptLast == NULL) { + bufOptRoot = pBuf; /* then there is also no root! */ + } else { + bufOptLast->pNext = pBuf; + } + bufOptLast = pBuf; + +finalize_it: + RETiRet; +} + + +/* remove option buffer from top of list, return values and destruct buffer itself. + * returns RS_RET_END_OF_LINKEDLIST when no more options are present. + * (we use int *opt instead of char *opt to keep consistent with getopt()) + */ +static rsRetVal +bufOptRemove(int *opt, char **arg) +{ + DEFiRet; + bufOpt_t *pBuf; + + if(bufOptRoot == NULL) + ABORT_FINALIZE(RS_RET_END_OF_LINKEDLIST); + pBuf = bufOptRoot; + + *opt = pBuf->optchar; + *arg = pBuf->arg; + + bufOptRoot = pBuf->pNext; + free(pBuf); + +finalize_it: + RETiRet; +} + + +rsRetVal +rsyslogdInit(void) +{ + char bufStartUpMsg[512]; + struct sigaction sigAct; + DEFiRet; + + memset(&sigAct, 0, sizeof (sigAct)); + sigemptyset(&sigAct.sa_mask); + sigAct.sa_handler = syslogd_sighup_handler; + sigaction(SIGHUP, &sigAct, NULL); + + CHKiRet(rsconf.Activate(ourConf)); + DBGPRINTF(" started.\n"); + + if(ourConf->globals.bLogStatusMsgs) { + snprintf(bufStartUpMsg, sizeof(bufStartUpMsg)/sizeof(char), + " [origin software=\"rsyslogd\" " "swVersion=\"" VERSION \ + "\" x-pid=\"%d\" x-info=\"http://www.rsyslog.com\"] start", + (int) glblGetOurPid()); + logmsgInternal(NO_ERRCODE, LOG_SYSLOG|LOG_INFO, (uchar*)bufStartUpMsg, 0); + } + +finalize_it: + RETiRet; +} + + +/* This is the main entry point into rsyslogd. Over time, we should try to + * modularize it a bit more... + */ +void +initAll(int argc, char **argv) +{ + rsRetVal localRet; + int ch; + extern int optind; + extern char *optarg; + int bEOptionWasGiven = 0; + int iHelperUOpt; + int bChDirRoot = 1; /* change the current working directory to "/"? */ + char *arg; /* for command line option processing */ + char cwdbuf[128]; /* buffer to obtain/display current working directory */ + DEFiRet; + + /* first, parse the command line options. We do not carry out any actual work, just + * see what we should do. This relieves us from certain anomalies and we can process + * the parameters down below in the correct order. For example, we must know the + * value of -M before we can do the init, but at the same time we need to have + * the base classes init before we can process most of the options. Now, with the + * split of functionality, this is no longer a problem. Thanks to varmofekoj for + * suggesting this algo. + * Note: where we just need to set some flags and can do so without knowledge + * of other options, we do this during the inital option processing. + * rgerhards, 2008-04-04 + */ + while((ch = getopt(argc, argv, "46a:Ac:dDef:g:hi:l:m:M:nN:op:qQr::s:S:t:T:u:vwx")) != EOF) { + switch((char)ch) { + case '4': + case '6': + case 'A': + case 'f': /* configuration file */ + case 'i': /* pid file name */ + case 'l': + case 'n': /* don't fork */ + case 'N': /* enable config verify mode */ + case 'q': /* add hostname if DNS resolving has failed */ + case 'Q': /* dont resolve hostnames in ACL to IPs */ + case 's': + case 'S': /* Source IP for local client to be used on multihomed host */ + case 'T': /* chroot on startup (primarily for testing) */ + case 'u': /* misc user settings */ + case 'w': /* disable disallowed host warnings */ + case 'x': /* disable dns for remote messages */ + CHKiRet(bufOptAdd(ch, optarg)); + break; + case 'd': /* debug - must be handled now, so that debug is active during init! */ + debugging_on = 1; + Debug = 1; + yydebug = 1; + break; + case 'D': /* BISON debug */ + yydebug = 1; + break; + case 'e': /* log every message (no repeat message supression) */ + bEOptionWasGiven = 1; + break; + case 'M': /* default module load path -- this MUST be carried out immediately! */ + glblModPath = (uchar*) optarg; + break; + case 'v': /* MUST be carried out immediately! */ + printVersion(); + exit(0); /* exit for -v option - so this is a "good one" */ + case '?': + default: + rsyslogd_usage(); + } + } + + if(argc - optind) + rsyslogd_usage(); + + DBGPRINTF("rsyslogd %s startup, module path '%s', cwd:%s\n", + VERSION, glblModPath == NULL ? "" : (char*)glblModPath, + getcwd(cwdbuf, sizeof(cwdbuf))); + + /* we are done with the initial option parsing and processing. Now we init the system. */ + + ppid = getpid(); + + CHKiRet(rsyslogd_InitGlobalClasses()); + CHKiRet(syslogd_obtainClassPointers()); + + /* doing some core initializations */ + + /* get our host and domain names - we need to do this early as we may emit + * error log messages, which need the correct hostname. -- rgerhards, 2008-04-04 + */ + queryLocalHostname(); + + /* initialize the objects */ + if((iRet = modInitIminternal()) != RS_RET_OK) { + fprintf(stderr, "fatal error: could not initialize errbuf object (error code %d).\n", + iRet); + exit(1); /* "good" exit, leaving at init for fatal error */ + } + + + /* END core initializations - we now come back to carrying out command line options*/ + + while((iRet = bufOptRemove(&ch, &arg)) == RS_RET_OK) { + DBGPRINTF("deque option %c, optarg '%s'\n", ch, (arg == NULL) ? "" : arg); + switch((char)ch) { + case '4': + glbl.SetDefPFFamily(PF_INET); + break; + case '6': + glbl.SetDefPFFamily(PF_INET6); + break; + case 'A': + send_to_all++; + break; + case 'S': /* Source IP for local client to be used on multihomed host */ + if(glbl.GetSourceIPofLocalClient() != NULL) { + fprintf (stderr, "rsyslogd: Only one -S argument allowed, the first one is taken.\n"); + } else { + glbl.SetSourceIPofLocalClient((uchar*)arg); + } + break; + case 'f': /* configuration file */ + ConfFile = (uchar*) arg; + break; + case 'i': /* pid file name */ + PidFile = arg; + break; + case 'l': + if(glbl.GetLocalHosts() != NULL) { + fprintf (stderr, "rsyslogd: Only one -l argument allowed, the first one is taken.\n"); + } else { + glbl.SetLocalHosts(syslogd_crunch_list(arg)); + } + break; + case 'n': /* don't fork */ + doFork = 0; + break; + case 'N': /* enable config verify mode */ + iConfigVerify = atoi(arg); + break; + case 'q': /* add hostname if DNS resolving has failed */ + *(net.pACLAddHostnameOnFail) = 1; + break; + case 'Q': /* dont resolve hostnames in ACL to IPs */ + *(net.pACLDontResolve) = 1; + break; + case 's': + if(glbl.GetStripDomains() != NULL) { + fprintf (stderr, "rsyslogd: Only one -s argument allowed, the first one is taken.\n"); + } else { + glbl.SetStripDomains(syslogd_crunch_list(arg)); + } + break; + case 'T':/* chroot() immediately at program startup, but only for testing, NOT security yet */ + if(chroot(arg) != 0) { + perror("chroot"); + exit(1); + } + break; + case 'u': /* misc user settings */ + iHelperUOpt = atoi(arg); + if(iHelperUOpt & 0x01) + glbl.SetParseHOSTNAMEandTAG(0); + if(iHelperUOpt & 0x02) + bChDirRoot = 0; + break; + case 'w': /* disable disallowed host warnigs */ + glbl.SetOption_DisallowWarning(0); + break; + case 'x': /* disable dns for remote messages */ + glbl.SetDisableDNS(1); + break; + case '?': + default: + rsyslogd_usage(); + } + } + + if(iRet != RS_RET_END_OF_LINKEDLIST) + FINALIZE; + + if(iConfigVerify) { + fprintf(stderr, "rsyslogd: version %s, config validation run (level %d), master config %s\n", + VERSION, iConfigVerify, ConfFile); + } + + localRet = rsconf.Load(&ourConf, ConfFile); + + syslogdInit(); + + if(localRet == RS_RET_NONFATAL_CONFIG_ERR) { + if(loadConf->globals.bAbortOnUncleanConfig) { + fprintf(stderr, "rsyslogd: $AbortOnUncleanConfig is set, and config is not clean.\n" + "Check error log for details, fix errors and restart. As a last\n" + "resort, you may want to remove $AbortOnUncleanConfig to permit a\n" + "startup with a dirty config.\n"); + exit(2); + } + if(iConfigVerify) { + /* a bit dirty, but useful... */ + exit(1); + } + localRet = RS_RET_OK; + } + CHKiRet(localRet); + + CHKiRet(rsyslogd_InitStdRatelimiters()); + + if(bChDirRoot) { + if(chdir("/") != 0) + fprintf(stderr, "Can not do 'cd /' - still trying to run\n"); + } + + /* process compatibility mode settings */ + if(bEOptionWasGiven) { + errmsg.LogError(0, NO_ERRCODE, "WARNING: \"message repeated n times\" feature MUST be turned on in " + "rsyslog.conf - CURRENTLY EVERY MESSAGE WILL BE LOGGED. Visit " + "http://www.rsyslog.com/rptdmsgreduction to learn " + "more and cast your vote if you want us to keep this feature."); + } + + if(!iConfigVerify) + CHKiRet(syslogd_doGlblProcessInit()); + + /* Send a signal to the parent so it can terminate. */ + if(glblGetOurPid() != ppid) + kill(ppid, SIGTERM); + + CHKiRet(rsyslogdInit()); + + if(Debug && debugging_on) { + dbgprintf("Debugging enabled, SIGUSR1 to turn off debugging.\n"); + } + + /* END OF INTIALIZATION */ + DBGPRINTF("initialization completed, transitioning to regular run mode\n"); + + /* close stderr and stdout if they are kept open during a fork. Note that this + * may introduce subtle security issues: if we are in a jail, one may break out of + * it via these descriptors. But if I close them earlier, error messages will (once + * again) not be emitted to the user that starts the daemon. As root jail support + * is still in its infancy (and not really done), we currently accept this issue. + * rgerhards, 2009-06-29 + */ + if(doFork) { + close(1); + close(2); + ourConf->globals.bErrMsgToStderr = 0; + } + +finalize_it: + if(iRet == RS_RET_VALIDATION_RUN) { + fprintf(stderr, "rsyslogd: End of config validation run. Bye.\n"); + exit(0); + } else if(iRet != RS_RET_OK) { + fprintf(stderr, "rsyslogd: run failed with error %d (see rsyslog.h " + "or try http://www.rsyslog.com/e/%d to learn what that number means)\n", iRet, iRet*-1); + exit(1); + } + + ENDfunc +} + +void +rsyslogdDebugSwitch() +{ + time_t tTime; + struct tm tp; + struct sigaction sigAct; + + datetime.GetTime(&tTime); + localtime_r(&tTime, &tp); + if(debugging_on == 0) { + debugging_on = 1; + dbgprintf("\n"); + dbgprintf("\n"); + dbgprintf("********************************************************************************\n"); + dbgprintf("Switching debugging_on to true at %2.2d:%2.2d:%2.2d\n", + tp.tm_hour, tp.tm_min, tp.tm_sec); + dbgprintf("********************************************************************************\n"); + } else { + dbgprintf("********************************************************************************\n"); + dbgprintf("Switching debugging_on to false at %2.2d:%2.2d:%2.2d\n", + tp.tm_hour, tp.tm_min, tp.tm_sec); + dbgprintf("********************************************************************************\n"); + dbgprintf("\n"); + dbgprintf("\n"); + debugging_on = 0; + } + + memset(&sigAct, 0, sizeof (sigAct)); + sigemptyset(&sigAct.sa_mask); + sigAct.sa_handler = rsyslogdDebugSwitch; + sigaction(SIGUSR1, &sigAct, NULL); +} + + +/* this function pulls all internal messages from the buffer + * and puts them into the processing engine. + * We can only do limited error handling, as this would not + * really help us. TODO: add error messages? + * rgerhards, 2007-08-03 + */ +static inline void processImInternal(void) +{ + msg_t *pMsg; + + while(iminternalRemoveMsg(&pMsg) == RS_RET_OK) { + submitMsgWithDfltRatelimiter(pMsg); + } +} + + +/* This takes a received message that must be decoded and submits it to + * the main message queue. This is a legacy function which is being provided + * to aid older input plugins that do not support message creation via + * the new interfaces themselves. It is not recommended to use this + * function for new plugins. -- rgerhards, 2009-10-12 + */ +rsRetVal +parseAndSubmitMessage(uchar *hname, uchar *hnameIP, uchar *msg, int len, int flags, flowControl_t flowCtlType, + prop_t *pInputName, struct syslogTime *stTime, time_t ttGenTime, ruleset_t *pRuleset) +{ + prop_t *pProp = NULL; + msg_t *pMsg; + DEFiRet; + + /* we now create our own message object and submit it to the queue */ + if(stTime == NULL) { + CHKiRet(msgConstruct(&pMsg)); + } else { + CHKiRet(msgConstructWithTime(&pMsg, stTime, ttGenTime)); + } + if(pInputName != NULL) + MsgSetInputName(pMsg, pInputName); + MsgSetRawMsg(pMsg, (char*)msg, len); + MsgSetFlowControlType(pMsg, flowCtlType); + MsgSetRuleset(pMsg, pRuleset); + pMsg->msgFlags = flags | NEEDS_PARSING; + + MsgSetRcvFromStr(pMsg, hname, ustrlen(hname), &pProp); + CHKiRet(prop.Destruct(&pProp)); + CHKiRet(MsgSetRcvFromIPStr(pMsg, hnameIP, ustrlen(hnameIP), &pProp)); + CHKiRet(prop.Destruct(&pProp)); + CHKiRet(submitMsg2(pMsg)); + +finalize_it: + RETiRet; +} + + +/* helper to doHUP(), this "HUPs" each action. The necessary locking + * is done inside the action class and nothing we need to take care of. + * rgerhards, 2008-10-22 + */ +DEFFUNC_llExecFunc(doHUPActions) +{ + BEGINfunc + actionCallHUPHdlr((action_t*) pData); + ENDfunc + return RS_RET_OK; /* we ignore errors, we can not do anything either way */ +} + + +/* This function processes a HUP after one has been detected. Note that this + * is *NOT* the sighup handler. The signal is recorded by the handler, that record + * detected inside the mainloop and then this function is called to do the + * real work. -- rgerhards, 2008-10-22 + * Note: there is a VERY slim chance of a data race when the hostname is reset. + * We prefer to take this risk rather than sync all accesses, because to the best + * of my analysis it can not really hurt (the actual property is reference-counted) + * but the sync would require some extra CPU for *each* message processed. + * rgerhards, 2012-04-11 + */ +static inline void +doHUP(void) +{ + char buf[512]; + + if(ourConf->globals.bLogStatusMsgs) { + snprintf(buf, sizeof(buf) / sizeof(char), + " [origin software=\"rsyslogd\" " "swVersion=\"" VERSION + "\" x-pid=\"%d\" x-info=\"http://www.rsyslog.com\"] rsyslogd was HUPed", + (int) glblGetOurPid()); + errno = 0; + logmsgInternal(NO_ERRCODE, LOG_SYSLOG|LOG_INFO, (uchar*)buf, 0); + } + + queryLocalHostname(); /* re-read our name */ + ruleset.IterateAllActions(ourConf, doHUPActions, NULL); + lookupDoHUP(); +} + +/* rsyslogdDoDie() is a signal handler. If called, it sets the bFinished variable + * to indicate the program should terminate. However, it does not terminate + * it itself, because that causes issues with multi-threading. The actual + * termination is then done on the main thread. This solution might introduce + * a minimal delay, but it is much cleaner than the approach of doing everything + * inside the signal handler. + * rgerhards, 2005-10-26 + * Note: + * - we do not call DBGPRINTF() as this may cause us to block in case something + * with the threading is wrong. + * - we do not really care about the return state of write(), but we need this + * strange check we do to silence compiler warnings (thanks, Ubuntu!) + */ +void +rsyslogdDoDie(int sig) +{ +# define MSG1 "DoDie called.\n" +# define MSG2 "DoDie called 5 times - unconditional exit\n" + static int iRetries = 0; /* debug aid */ + dbgprintf(MSG1); + if(Debug == DEBUG_FULL) { + if(write(1, MSG1, sizeof(MSG1) - 1)) {} + } + if(iRetries++ == 4) { + if(Debug == DEBUG_FULL) { + if(write(1, MSG2, sizeof(MSG2) - 1)) {} + } + abort(); + } + bFinished = sig; + if(glblDebugOnShutdown) { + /* kind of hackish - set to 0, so that debug_swith will enable + * and AND emit the "start debug log" message. + */ + debugging_on = 0; + rsyslogdDebugSwitch(); + } +# undef MSG1 +# undef MSG2 +} + + +/* This is the main processing loop. It is called after successful initialization. + * When it returns, the syslogd terminates. + * Its sole function is to provide some housekeeping things. The real work is done + * by the other threads spawned. + */ +static void +mainloop(void) +{ + struct timeval tvSelectTimeout; + + BEGINfunc + /* first check if we have any internal messages queued and spit them out. */ + processImInternal(); + + while(!bFinished){ + tvSelectTimeout.tv_sec = janitorInterval * 60; /* interval is in minutes! */ + tvSelectTimeout.tv_usec = 0; + select(1, NULL, NULL, NULL, &tvSelectTimeout); + if(bFinished) + break; /* exit as quickly as possible */ + + janitorRun(); + + if(bHadHUP) { + doHUP(); + bHadHUP = 0; + continue; + } + } + ENDfunc +} + +/* Finalize and destruct all actions. + */ +void +rsyslogd_destructAllActions(void) +{ + ruleset.DestructAllActions(runConf); + bHaveMainQueue = 0; /* flag that internal messages need to be temporarily stored */ +} + + +/* de-initialize everything, make ready for termination */ +static void +deinitAll(void) +{ + char buf[256]; + + DBGPRINTF("exiting on signal %d\n", bFinished); + + /* IMPORTANT: we should close the inputs first, and THEN send our termination + * message. If we do it the other way around, logmsgInternal() may block on + * a full queue and the inputs still fill up that queue. Depending on the + * scheduling order, we may end up with logmsgInternal being held for a quite + * long time. When the inputs are terminated first, that should not happen + * because the queue is drained in parallel. The situation could only become + * an issue with extremely long running actions in a queue full environment. + * However, such actions are at least considered poorly written, if not + * outright wrong. So we do not care about this very remote problem. + * rgerhards, 2008-01-11 + */ + + /* close the inputs */ + DBGPRINTF("Terminating input threads...\n"); + glbl.SetGlobalInputTermination(); + thrdTerminateAll(); + + /* and THEN send the termination log message (see long comment above) */ + if(bFinished && runConf->globals.bLogStatusMsgs) { + (void) snprintf(buf, sizeof(buf) / sizeof(char), + " [origin software=\"rsyslogd\" " "swVersion=\"" VERSION \ + "\" x-pid=\"%d\" x-info=\"http://www.rsyslog.com\"]" " exiting on signal %d.", + (int) glblGetOurPid(), bFinished); + errno = 0; + logmsgInternal(NO_ERRCODE, LOG_SYSLOG|LOG_INFO, (uchar*)buf, 0); + } + /* we sleep for 50ms to give the queue a chance to pick up the exit message; + * otherwise we have seen cases where the message did not make it to log + * files, even on idle systems. + */ + srSleep(0, 50); + + /* drain queue (if configured so) and stop main queue worker thread pool */ + DBGPRINTF("Terminating main queue...\n"); + qqueueDestruct(&pMsgQueue); + pMsgQueue = NULL; + + /* Free ressources and close connections. This includes flushing any remaining + * repeated msgs. + */ + DBGPRINTF("Terminating outputs...\n"); + rsyslogd_destructAllActions(); + + DBGPRINTF("all primary multi-thread sources have been terminated - now doing aux cleanup...\n"); + + DBGPRINTF("destructing current config...\n"); + rsconf.Destruct(&runConf); + + modExitIminternal(); + + if(pInternalInputName != NULL) + prop.Destruct(&pInternalInputName); + + /* the following line cleans up CfSysLineHandlers that were not based on loadable + * modules. As such, they are not yet cleared. */ + unregCfSysLineHdlrs(); + + /*dbgPrintAllDebugInfo(); / * this is the last spot where this can be done - below output modules are unloaded! */ + + syslogd_releaseClassPointers(); + + parserClassExit(); + rsconfClassExit(); + strExit(); + ratelimitModExit(); + dnscacheDeinit(); + thrdExit(); + + module.UnloadAndDestructAll(eMOD_LINK_ALL); + + rsrtExit(); /* runtime MUST always be deinitialized LAST (except for debug system) */ + DBGPRINTF("Clean shutdown completed, bye\n"); + + /* dbgClassExit MUST be the last one, because it de-inits the debug system */ + dbgClassExit(); + + /* NO CODE HERE - dbgClassExit() must be the last thing before exit()! */ + syslogd_die(); +} + +/* This is the main entry point into rsyslogd. This must be a function in its own + * right in order to intialize the debug system in a portable way (otherwise we would + * need to have a statement before variable definitions. + * rgerhards, 20080-01-28 + */ +int +main(int argc, char **argv) +{ + dbgClassInit(); + initAll(argc, argv); + sd_notify(0, "READY=1"); + + mainloop(); + deinitAll(); + return 0; +} diff --git a/tools/smfile.c b/tools/smfile.c index 1e0bf09..71f3d8b 100644 --- a/tools/smfile.c +++ b/tools/smfile.c @@ -12,24 +12,23 @@ * * File begun on 2010-06-01 by RGerhards * - * Copyright 2010 Rainer Gerhards and Adiscon GmbH. + * Copyright 2010-2014 Rainer Gerhards and Adiscon GmbH. * * This file is part of rsyslog. * - * Rsyslog is free software: you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation, either version 3 of the License, or - * (at your option) any later version. - * - * Rsyslog is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * - * You should have received a copy of the GNU General Public License - * along with Rsyslog. If not, see <http://www.gnu.org/licenses/>. - * - * A copy of the GPL can be found in the file "COPYING" in this distribution. + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * -or- + * see COPYING.ASL20 in the source distribution + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. */ #include "config.h" #include "rsyslog.h" @@ -88,29 +87,31 @@ CODESTARTstrgen ++lenTotal; /* then we need to introduce one additional space */ /* now make sure buffer is large enough */ - if(lenTotal >= *pLenBuf) - CHKiRet(ExtendBuf(ppBuf, pLenBuf, lenTotal)); + if(lenTotal >= iparam->lenBuf) + CHKiRet(ExtendBuf(iparam, lenTotal)); /* and concatenate the resulting string */ - memcpy(*ppBuf, pTimeStamp, lenTimeStamp); + memcpy(iparam->param, pTimeStamp, lenTimeStamp); iBuf = lenTimeStamp; - *(*ppBuf + iBuf++) = ' '; + iparam->param[iBuf++] = ' '; - memcpy(*ppBuf + iBuf, pHOSTNAME, lenHOSTNAME); + memcpy(iparam->param + iBuf, pHOSTNAME, lenHOSTNAME); iBuf += lenHOSTNAME; - *(*ppBuf + iBuf++) = ' '; + iparam->param[iBuf++] = ' '; - memcpy(*ppBuf + iBuf, pTAG, lenTAG); + memcpy(iparam->param + iBuf, pTAG, lenTAG); iBuf += lenTAG; if(pMSG[0] != ' ') - *(*ppBuf + iBuf++) = ' '; - memcpy(*ppBuf + iBuf, pMSG, lenMSG); + iparam->param[iBuf++] = ' '; + memcpy(iparam->param + iBuf, pMSG, lenMSG); iBuf += lenMSG; /* trailer */ - *(*ppBuf + iBuf++) = '\n'; - *(*ppBuf + iBuf) = '\0'; + iparam->param[iBuf++] = '\n'; + iparam->param[iBuf] = '\0'; + + iparam->lenStr = lenTotal - 1; /* do not count \0! */ finalize_it: ENDstrgen diff --git a/tools/smfile.h b/tools/smfile.h index 10946db..893a951 100644 --- a/tools/smfile.h +++ b/tools/smfile.h @@ -3,24 +3,23 @@ * * File begun on 2010-06-04 by RGerhards * - * Copyright 2010 Rainer Gerhards and Adiscon GmbH. + * Copyright 2010-2014 Rainer Gerhards and Adiscon GmbH. * * This file is part of rsyslog. * - * Rsyslog is free software: you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation, either version 3 of the License, or - * (at your option) any later version. - * - * Rsyslog is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * - * You should have received a copy of the GNU General Public License - * along with Rsyslog. If not, see <http://www.gnu.org/licenses/>. - * - * A copy of the GPL can be found in the file "COPYING" in this distribution. + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * -or- + * see COPYING.ASL20 in the source distribution + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. */ #ifndef SMFILE_H_INCLUDED #define SMFILE_H_INCLUDED 1 diff --git a/tools/smfwd.c b/tools/smfwd.c index 60fe94a..62bf7fe 100644 --- a/tools/smfwd.c +++ b/tools/smfwd.c @@ -9,24 +9,23 @@ * * File begun on 2010-06-01 by RGerhards * - * Copyright 2010 Rainer Gerhards and Adiscon GmbH. + * Copyright 2010-2014 Rainer Gerhards and Adiscon GmbH. * * This file is part of rsyslog. * - * Rsyslog is free software: you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation, either version 3 of the License, or - * (at your option) any later version. - * - * Rsyslog is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * - * You should have received a copy of the GNU General Public License - * along with Rsyslog. If not, see <http://www.gnu.org/licenses/>. - * - * A copy of the GPL can be found in the file "COPYING" in this distribution. + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * -or- + * see COPYING.ASL20 in the source distribution + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. */ #include "config.h" #include "rsyslog.h" @@ -91,33 +90,35 @@ CODESTARTstrgen ++lenTotal; /* then we need to introduce one additional space */ /* now make sure buffer is large enough */ - if(lenTotal >= *pLenBuf) - CHKiRet(ExtendBuf(ppBuf, pLenBuf, lenTotal)); + if(lenTotal >= iparam->lenBuf) + CHKiRet(ExtendBuf(iparam, lenTotal)); /* and concatenate the resulting string */ - **ppBuf = '<'; - memcpy(*ppBuf + 1, pPRI, lenPRI); + iparam->param[0] = '<'; + memcpy(iparam->param + 1, pPRI, lenPRI); iBuf = lenPRI + 1; - *(*ppBuf + iBuf++) = '>'; + iparam->param[iBuf++] = '>'; - memcpy(*ppBuf + iBuf, pTimeStamp, lenTimeStamp); + memcpy(iparam->param + iBuf, pTimeStamp, lenTimeStamp); iBuf += lenTimeStamp; - *(*ppBuf + iBuf++) = ' '; + iparam->param[iBuf++] = ' '; - memcpy(*ppBuf + iBuf, pHOSTNAME, lenHOSTNAME); + memcpy(iparam->param + iBuf, pHOSTNAME, lenHOSTNAME); iBuf += lenHOSTNAME; - *(*ppBuf + iBuf++) = ' '; + iparam->param[iBuf++] = ' '; - memcpy(*ppBuf + iBuf, pTAG, lenTAG); + memcpy(iparam->param + iBuf, pTAG, lenTAG); iBuf += lenTAG; if(pMSG[0] != ' ') - *(*ppBuf + iBuf++) = ' '; - memcpy(*ppBuf + iBuf, pMSG, lenMSG); + iparam->param[iBuf++] = ' '; + memcpy(iparam->param + iBuf, pMSG, lenMSG); iBuf += lenMSG; /* string terminator */ - *(*ppBuf + iBuf) = '\0'; + iparam->param[iBuf] = '\0'; + + iparam->lenStr = lenTotal - 1; /* do not count \0! */ finalize_it: ENDstrgen diff --git a/tools/smfwd.h b/tools/smfwd.h index 191a6bf..78bb1e3 100644 --- a/tools/smfwd.h +++ b/tools/smfwd.h @@ -2,24 +2,23 @@ * * File begun on 2010-06-04 by RGerhards * - * Copyright 2010 Rainer Gerhards and Adiscon GmbH. + * Copyright 2010-2014 Rainer Gerhards and Adiscon GmbH. * * This file is part of rsyslog. * - * Rsyslog is free software: you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation, either version 3 of the License, or - * (at your option) any later version. - * - * Rsyslog is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * - * You should have received a copy of the GNU General Public License - * along with Rsyslog. If not, see <http://www.gnu.org/licenses/>. - * - * A copy of the GPL can be found in the file "COPYING" in this distribution. + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * -or- + * see COPYING.ASL20 in the source distribution + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. */ #ifndef SMFWD_H_INCLUDED #define SMFWD_H_INCLUDED 1 diff --git a/tools/smtradfile.c b/tools/smtradfile.c index 5484f7b..35072aa 100644 --- a/tools/smtradfile.c +++ b/tools/smtradfile.c @@ -9,24 +9,23 @@ * * File begun on 2010-06-01 by RGerhards * - * Copyright 2010 Rainer Gerhards and Adiscon GmbH. + * Copyright 2010-2014 Rainer Gerhards and Adiscon GmbH. * * This file is part of rsyslog. * - * Rsyslog is free software: you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation, either version 3 of the License, or - * (at your option) any later version. - * - * Rsyslog is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * - * You should have received a copy of the GNU General Public License - * along with Rsyslog. If not, see <http://www.gnu.org/licenses/>. - * - * A copy of the GPL can be found in the file "COPYING" in this distribution. + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * -or- + * see COPYING.ASL20 in the source distribution + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. */ #include "config.h" #include "rsyslog.h" @@ -83,28 +82,30 @@ CODESTARTstrgen ++lenTotal; /* then we need to introduce one additional space */ /* now make sure buffer is large enough */ - if(lenTotal >= *pLenBuf) - CHKiRet(ExtendBuf(ppBuf, pLenBuf, lenTotal)); + if(lenTotal >= iparam->lenBuf) + CHKiRet(ExtendBuf(iparam, lenTotal)); /* and concatenate the resulting string */ - memcpy(*ppBuf, pTimeStamp, CONST_LEN_TIMESTAMP_3164); - *(*ppBuf + CONST_LEN_TIMESTAMP_3164) = ' '; + memcpy(iparam->param, pTimeStamp, CONST_LEN_TIMESTAMP_3164); + iparam->param[CONST_LEN_TIMESTAMP_3164] = ' '; - memcpy(*ppBuf + CONST_LEN_TIMESTAMP_3164 + 1, pHOSTNAME, lenHOSTNAME); + memcpy(iparam->param + CONST_LEN_TIMESTAMP_3164 + 1, pHOSTNAME, lenHOSTNAME); iBuf = CONST_LEN_TIMESTAMP_3164 + 1 + lenHOSTNAME; - *(*ppBuf + iBuf++) = ' '; + iparam->param[iBuf++] = ' '; - memcpy(*ppBuf + iBuf, pTAG, lenTAG); + memcpy(iparam->param + iBuf, pTAG, lenTAG); iBuf += lenTAG; if(pMSG[0] != ' ') - *(*ppBuf + iBuf++) = ' '; - memcpy(*ppBuf + iBuf, pMSG, lenMSG); + iparam->param[iBuf++] = ' '; + memcpy(iparam->param + iBuf, pMSG, lenMSG); iBuf += lenMSG; /* trailer */ - *(*ppBuf + iBuf++) = '\n'; - *(*ppBuf + iBuf) = '\0'; + iparam->param[iBuf++] = '\n'; + iparam->param[iBuf] = '\0'; + + iparam->lenStr = lenTotal - 1; /* do not count \0! */ finalize_it: ENDstrgen diff --git a/tools/smtradfile.h b/tools/smtradfile.h index afc737e..7f61fdb 100644 --- a/tools/smtradfile.h +++ b/tools/smtradfile.h @@ -3,24 +3,23 @@ * * File begun on 2010-06-01 by RGerhards * - * Copyright 2010 Rainer Gerhards and Adiscon GmbH. + * Copyright 2010-2014 Rainer Gerhards and Adiscon GmbH. * * This file is part of rsyslog. * - * Rsyslog is free software: you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation, either version 3 of the License, or - * (at your option) any later version. - * - * Rsyslog is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * - * You should have received a copy of the GNU General Public License - * along with Rsyslog. If not, see <http://www.gnu.org/licenses/>. - * - * A copy of the GPL can be found in the file "COPYING" in this distribution. + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * -or- + * see COPYING.ASL20 in the source distribution + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. */ #ifndef SMTRADFILE_H_INCLUDED #define SMTRADFILE_H_INCLUDED 1 diff --git a/tools/smtradfwd.c b/tools/smtradfwd.c index 3771743..6ffab9b 100644 --- a/tools/smtradfwd.c +++ b/tools/smtradfwd.c @@ -9,24 +9,23 @@ * * File begun on 2010-06-01 by RGerhards * - * Copyright 2010 Rainer Gerhards and Adiscon GmbH. + * Copyright 2010-2014 Rainer Gerhards and Adiscon GmbH. * * This file is part of rsyslog. * - * Rsyslog is free software: you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation, either version 3 of the License, or - * (at your option) any later version. - * - * Rsyslog is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * - * You should have received a copy of the GNU General Public License - * along with Rsyslog. If not, see <http://www.gnu.org/licenses/>. - * - * A copy of the GPL can be found in the file "COPYING" in this distribution. + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * -or- + * see COPYING.ASL20 in the source distribution + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. */ #include "config.h" #include "rsyslog.h" @@ -89,33 +88,35 @@ CODESTARTstrgen ++lenTotal; /* then we need to introduce one additional space */ /* now make sure buffer is large enough */ - if(lenTotal >= *pLenBuf) - CHKiRet(ExtendBuf(ppBuf, pLenBuf, lenTotal)); + if(lenTotal >= iparam->lenBuf) + CHKiRet(ExtendBuf(iparam, lenTotal)); /* and concatenate the resulting string */ - **ppBuf = '<'; - memcpy(*ppBuf + 1, pPRI, lenPRI); + iparam->param[0] = '<'; + memcpy(iparam->param + 1, pPRI, lenPRI); iBuf = lenPRI + 1; - *(*ppBuf + iBuf++) = '>'; + iparam->param[iBuf++] = '>'; - memcpy(*ppBuf + iBuf, pTimeStamp, CONST_LEN_TIMESTAMP_3164); + memcpy(iparam->param + iBuf, pTimeStamp, CONST_LEN_TIMESTAMP_3164); iBuf += CONST_LEN_TIMESTAMP_3164; - *(*ppBuf + iBuf++) = ' '; + iparam->param[iBuf++] = ' '; - memcpy(*ppBuf + iBuf, pHOSTNAME, lenHOSTNAME); + memcpy(iparam->param + iBuf, pHOSTNAME, lenHOSTNAME); iBuf += lenHOSTNAME; - *(*ppBuf + iBuf++) = ' '; + iparam->param[iBuf++] = ' '; - memcpy(*ppBuf + iBuf, pTAG, lenTAG); + memcpy(iparam->param + iBuf, pTAG, lenTAG); iBuf += lenTAG; if(pMSG[0] != ' ') - *(*ppBuf + iBuf++) = ' '; - memcpy(*ppBuf + iBuf, pMSG, lenMSG); + iparam->param[iBuf++] = ' '; + memcpy(iparam->param + iBuf, pMSG, lenMSG); iBuf += lenMSG; /* string terminator */ - *(*ppBuf + iBuf) = '\0'; + iparam->param[iBuf] = '\0'; + + iparam->lenStr = lenTotal - 1; /* do not count \0! */ finalize_it: ENDstrgen diff --git a/tools/smtradfwd.h b/tools/smtradfwd.h index 9ff0ab5..7ac6dea 100644 --- a/tools/smtradfwd.h +++ b/tools/smtradfwd.h @@ -2,24 +2,23 @@ * * File begun on 2010-06-04 by RGerhards * - * Copyright 2010 Rainer Gerhards and Adiscon GmbH. + * Copyright 2010-2014 Rainer Gerhards and Adiscon GmbH. * * This file is part of rsyslog. * - * Rsyslog is free software: you can redistribute it and/or modify - * it under the terms of the GNU General Public License as published by - * the Free Software Foundation, either version 3 of the License, or - * (at your option) any later version. - * - * Rsyslog is distributed in the hope that it will be useful, - * but WITHOUT ANY WARRANTY; without even the implied warranty of - * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - * GNU General Public License for more details. - * - * You should have received a copy of the GNU General Public License - * along with Rsyslog. If not, see <http://www.gnu.org/licenses/>. - * - * A copy of the GPL can be found in the file "COPYING" in this distribution. + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * -or- + * see COPYING.ASL20 in the source distribution + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. */ #ifndef SMTRADFWD_H_INCLUDED #define SMTRADFWD_H_INCLUDED 1 diff --git a/tools/syslogd.c b/tools/syslogd.c index 1b38bf9..ac45775 100644 --- a/tools/syslogd.c +++ b/tools/syslogd.c @@ -1,10 +1,16 @@ /** - * \brief This is the main file of the rsyslogd daemon. + * main rsyslog file with GPLv3 content. * - * Please visit the rsyslog project at + * *********************** NOTE ************************ + * * Do no longer patch this file. If there is hard * + * * need to, talk to Rainer as to how we can make any * + * * patch be licensed under ASL 2.0. * + * * THIS FILE WILL GO AWAY. The new main file is * + * * rsyslogd.c. * + * ***************************************************** * + * Please visit the rsyslog project at * http://www.rsyslog.com - * * to learn more about it and discuss any questions you may have. * * rsyslog had initially been forked from the sysklogd project. @@ -18,10 +24,8 @@ * This Project was intiated and is maintained by * Rainer Gerhards <rgerhards@hq.adiscon.com>. * - * For further information, please see http://www.rsyslog.com - * * rsyslog - An Enhanced syslogd Replacement. - * Copyright 2003-2012 Rainer Gerhards and Adiscon GmbH. + * Copyright 2003-2014 Rainer Gerhards and Adiscon GmbH. * * This file is part of rsyslog. * @@ -43,15 +47,12 @@ #include "config.h" #include "rsyslog.h" -#define DEFUPRI (LOG_USER|LOG_NOTICE) - #include <unistd.h> #include <stdlib.h> #include <stdio.h> #include <stddef.h> #include <ctype.h> #include <limits.h> -#define GNU_SOURCE #include <string.h> #include <stdarg.h> #include <time.h> @@ -88,12 +89,6 @@ #include <paths.h> #endif -#ifdef USE_NETZIP -#include <zlib.h> -#endif - -extern int yydebug; /* interface to flex */ - #include <netdb.h> #include "pidfile.h" @@ -105,179 +100,77 @@ extern int yydebug; /* interface to flex */ #include "syslogd.h" #include "msg.h" -#include "modules.h" -#include "action.h" #include "iminternal.h" -#include "cfsysline.h" #include "threads.h" -#include "wti.h" -#include "queue.h" -#include "stream.h" -#include "conf.h" -#include "errmsg.h" -#include "datetime.h" #include "parser.h" -#include "batch.h" #include "unicode-helper.h" -#include "ruleset.h" #include "net.h" -#include "prop.h" -#include "rsconf.h" #include "dnscache.h" #include "sd-daemon.h" -#include "rainerscript.h" #include "ratelimit.h" /* definitions for objects we access */ DEFobjCurrIf(obj) DEFobjCurrIf(glbl) -DEFobjCurrIf(datetime) /* TODO: make go away! */ -DEFobjCurrIf(conf) -DEFobjCurrIf(module) -DEFobjCurrIf(errmsg) -DEFobjCurrIf(ruleset) -DEFobjCurrIf(prop) -DEFobjCurrIf(parser) -DEFobjCurrIf(rsconf) DEFobjCurrIf(net) /* TODO: make go away! */ /* forward definitions */ -static rsRetVal GlobalClassExit(void); -static rsRetVal queryLocalHostname(void); - - -#ifndef _PATH_LOGCONF -#define _PATH_LOGCONF "/etc/rsyslog.conf" -#endif - -#ifndef _PATH_MODDIR -# if defined(__FreeBSD__) -# define _PATH_MODDIR "/usr/local/lib/rsyslog/" -# else -# define _PATH_MODDIR "/lib/rsyslog/" -# endif -#endif - -#if defined(SYSLOGD_PIDNAME) -# undef _PATH_LOGPID -# if defined(FSSTND) -# ifdef OS_BSD -# define _PATH_VARRUN "/var/run/" -# endif -# if defined(__sun) || defined(__hpux) -# define _PATH_VARRUN "/var/run/" -# endif -# define _PATH_LOGPID _PATH_VARRUN SYSLOGD_PIDNAME -# else -# define _PATH_LOGPID "/etc/" SYSLOGD_PIDNAME -# endif -#else -# ifndef _PATH_LOGPID -# if defined(__sun) || defined(__hpux) -# define _PATH_VARRUN "/var/run/" -# endif -# if defined(FSSTND) -# define _PATH_LOGPID _PATH_VARRUN "rsyslogd.pid" -# else -# define _PATH_LOGPID "/etc/rsyslogd.pid" -# endif -# endif +rsRetVal queryLocalHostname(void); + +/* forward defintions from rsyslogd.c (ASL 2.0 code) */ +extern ratelimit_t *internalMsg_ratelimiter; +extern uchar *ConfFile; +extern ratelimit_t *dflt_ratelimiter; +extern void rsyslogd_usage(void); +extern rsRetVal rsyslogdInit(void); +extern void rsyslogd_destructAllActions(void); +extern void rsyslogd_sigttin_handler(); +void rsyslogd_submitErrMsg(const int severity, const int iErr, const uchar *msg); +rsRetVal rsyslogd_InitGlobalClasses(void); +rsRetVal rsyslogd_InitStdRatelimiters(void); +rsRetVal rsyslogdInit(void); +void rsyslogdDebugSwitch(); +void rsyslogdDoDie(int sig); + + +#ifndef _PATH_LOGPID +# define _PATH_LOGPID "/var/run/rsyslogd.pid" #endif #ifndef _PATH_TTY # define _PATH_TTY "/dev/tty" #endif +char *PidFile = _PATH_LOGPID; /* read-only after startup */ -rsconf_t *ourConf; /* our config object */ - -static prop_t *pInternalInputName = NULL; /* there is only one global inputName for all internally-generated messages */ -static uchar *ConfFile = (uchar*) _PATH_LOGCONF; /* read-only after startup */ -static char *PidFile = _PATH_LOGPID; /* read-only after startup */ - -/* mypid is read-only after the initial fork() */ -static int bHadHUP = 0; /* did we have a HUP? */ - -static int bFinished = 0; /* used by termination signal handler, read-only except there - * is either 0 or the number of the signal that requested the - * termination. - */ +int bHadHUP = 0; /* did we have a HUP? */ +int bFinished = 0; /* used by termination signal handler, read-only except there + * is either 0 or the number of the signal that requested the + * termination. + */ int iConfigVerify = 0; /* is this just a config verify run? */ - -#define LIST_DELIMITER ':' /* delimiter between two hosts */ - -static pid_t ppid; /* This is a quick and dirty hack used for spliting main/startup thread */ - -struct queuefilenames_s { - struct queuefilenames_s *next; - uchar *name; -} *queuefilenames = NULL; - - -static ratelimit_t *dflt_ratelimiter = NULL; /* ratelimiter for submits without explicit one */ -static ratelimit_t *internalMsg_ratelimiter = NULL; /* ratelimiter for rsyslog-own messages */ -int MarkInterval = 20 * 60; /* interval between marks in seconds - read-only after startup */ -int send_to_all = 0; /* send message to all IPv4/IPv6 addresses */ -static int doFork = 1; /* fork - run in daemon mode - read-only after startup */ -int bHaveMainQueue = 0;/* set to 1 if the main queue - in queueing mode - is available - * If the main queue is either not yet ready or not running in - * queueing mode (mode DIRECT!), then this is set to 0. - */ - -extern int errno; - -/* main message queue and its configuration parameters */ -qqueue_t *pMsgQueue = NULL; /* the main message queue */ +pid_t ppid; /* This is a quick and dirty hack used for spliting main/startup thread */ +int doFork = 1; /* fork - run in daemon mode - read-only after startup */ /* up to the next comment, prototypes that should be removed by reordering */ /* Function prototypes. */ -static char **crunch_list(char *list); static void reapchild(); -static void debug_switch(); -static void sighup_handler(); - - -static int usage(void) -{ - fprintf(stderr, "usage: rsyslogd [-46AdnqQvwx] [-l<hostlist>] [-s<domainlist>]\n" - " [-f<conffile>] [-i<pidfile>] [-N<level>] [-M<module load path>]\n" - " [-u<number>]\n" - "For further information see http://www.rsyslog.com/doc\n"); - exit(1); /* "good" exit - done to terminate usage() */ -} - - -/* ------------------------------ some support functions for imdiag ------------------------------ * - * This is a bit dirty, but the only way to do it, at least with reasonable effort. - * rgerhards, 2009-05-25 - */ - -/* return back the approximate current number of messages in the main message queue - * This number includes the messages that reside in an associated DA queue (if - * it exists) -- rgerhards, 2009-10-14 - */ -rsRetVal -diagGetMainMsgQSize(int *piSize) -{ - DEFiRet; - assert(piSize != NULL); - *piSize = (pMsgQueue->pqDA != NULL) ? pMsgQueue->pqDA->iQueueSize : 0; - *piSize += pMsgQueue->iQueueSize; - RETiRet; -} - - -/* ------------------------------ end support functions for imdiag ------------------------------ */ +#define LIST_DELIMITER ':' /* delimiter between two hosts */ /* rgerhards, 2005-10-24: crunch_list is called only during option processing. So * it is never called once rsyslogd is running. This code * contains some exits, but they are considered safe because they only happen * during startup. Anyhow, when we review the code here, we might want to * reconsider the exit()s. + * Note: this stems back to sysklogd, so we cannot put it under ASL 2.0. But + * we may want to check if the code inside the BSD sources is exactly the same + * (remember that sysklogd forked the BSD sources). If so, the BSD license applies + * and permits us to move to ASL 2.0 (but we need to check the fine details). + * Probably it is best just to rewrite this code. */ -static char **crunch_list(char *list) +char **syslogd_crunch_list(char *list) { int count, i; char *p, *q; @@ -330,15 +223,11 @@ static char **crunch_list(char *list) strcpy(result[count],p); result[++count] = NULL; -#if 0 - count=0; - while (result[count]) - DBGPRINTF("#%d: %s\n", count, StripDomains[count++]); -#endif return result; } +/* also stems back to sysklogd in whole */ void untty(void) #ifdef HAVE_SETSID { @@ -353,11 +242,13 @@ void untty(void) pid_t pid; if(!Debug) { + /* Peng Haitao <penght@cn.fujitsu.com> contribution */ pid = getpid(); if (setpgid(pid, pid) < 0) { perror("setpgid"); exit(1); } + /* end Peng Haitao <penght@cn.fujitsu.com> contribution */ i = open(_PATH_TTY, O_RDWR|O_CLOEXEC); if (i >= 0) { @@ -375,295 +266,7 @@ void untty(void) } #endif - -/* This takes a received message that must be decoded and submits it to - * the main message queue. This is a legacy function which is being provided - * to aid older input plugins that do not support message creation via - * the new interfaces themselves. It is not recommended to use this - * function for new plugins. -- rgerhards, 2009-10-12 - */ -rsRetVal -parseAndSubmitMessage(uchar *hname, uchar *hnameIP, uchar *msg, int len, int flags, flowControl_t flowCtlType, - prop_t *pInputName, struct syslogTime *stTime, time_t ttGenTime, ruleset_t *pRuleset) -{ - prop_t *pProp = NULL; - msg_t *pMsg; - DEFiRet; - - /* we now create our own message object and submit it to the queue */ - if(stTime == NULL) { - CHKiRet(msgConstruct(&pMsg)); - } else { - CHKiRet(msgConstructWithTime(&pMsg, stTime, ttGenTime)); - } - if(pInputName != NULL) - MsgSetInputName(pMsg, pInputName); - MsgSetRawMsg(pMsg, (char*)msg, len); - MsgSetFlowControlType(pMsg, flowCtlType); - MsgSetRuleset(pMsg, pRuleset); - pMsg->msgFlags = flags | NEEDS_PARSING; - - MsgSetRcvFromStr(pMsg, hname, ustrlen(hname), &pProp); - CHKiRet(prop.Destruct(&pProp)); - CHKiRet(MsgSetRcvFromIPStr(pMsg, hnameIP, ustrlen(hnameIP), &pProp)); - CHKiRet(prop.Destruct(&pProp)); - CHKiRet(submitMsg2(pMsg)); - -finalize_it: - RETiRet; -} - - -/* this is a special function used to submit an error message. This - * function is also passed to the runtime library as the generic error - * message handler. -- rgerhards, 2008-04-17 - */ -rsRetVal -submitErrMsg(int iErr, uchar *msg) -{ - DEFiRet; - iRet = logmsgInternal(iErr, LOG_SYSLOG|LOG_ERR, msg, 0); - RETiRet; -} - - -static inline rsRetVal -submitMsgWithDfltRatelimiter(msg_t *pMsg) -{ - return ratelimitAddMsg(dflt_ratelimiter, NULL, pMsg); -} - -/* rgerhards 2004-11-09: the following is a function that can be used - * to log a message orginating from the syslogd itself. - */ -rsRetVal -logmsgInternal(int iErr, int pri, uchar *msg, int flags) -{ - uchar pszTag[33]; - msg_t *pMsg; - DEFiRet; - - CHKiRet(msgConstruct(&pMsg)); - MsgSetInputName(pMsg, pInternalInputName); - MsgSetRawMsgWOSize(pMsg, (char*)msg); - MsgSetHOSTNAME(pMsg, glbl.GetLocalHostName(), ustrlen(glbl.GetLocalHostName())); - MsgSetRcvFrom(pMsg, glbl.GetLocalHostNameProp()); - MsgSetRcvFromIP(pMsg, glbl.GetLocalHostIP()); - MsgSetMSGoffs(pMsg, 0); - /* check if we have an error code associated and, if so, - * adjust the tag. -- rgerhards, 2008-06-27 - */ - if(iErr == NO_ERRCODE) { - MsgSetTAG(pMsg, UCHAR_CONSTANT("rsyslogd:"), sizeof("rsyslogd:") - 1); - } else { - size_t len = snprintf((char*)pszTag, sizeof(pszTag), "rsyslogd%d:", iErr); - pszTag[32] = '\0'; /* just to make sure... */ - MsgSetTAG(pMsg, pszTag, len); - } - pMsg->iFacility = LOG_FAC(pri); - pMsg->iSeverity = LOG_PRI(pri); - flags |= INTERNAL_MSG; - pMsg->msgFlags = flags; - - /* we now check if we should print internal messages out to stderr. This was - * suggested by HKS as a way to help people troubleshoot rsyslog configuration - * (by running it interactively. This makes an awful lot of sense, so I add - * it here. -- rgerhards, 2008-07-28 - * Note that error messages can not be disable during a config verify. This - * permits us to process unmodified config files which otherwise contain a - * supressor statement. - */ - if(((Debug == DEBUG_FULL || !doFork) && ourConf->globals.bErrMsgToStderr) || iConfigVerify) { - if(LOG_PRI(pri) == LOG_ERR) - fprintf(stderr, "rsyslogd: %s\n", msg); - } - - if(bHaveMainQueue == 0) { /* not yet in queued mode */ - iminternalAddMsg(pMsg); - } else { - /* we have the queue, so we can simply provide the - * message to the queue engine. - */ - ratelimitAddMsg(internalMsg_ratelimiter, NULL, pMsg); - //submitMsgWithDfltRatelimiter(pMsg); - } -finalize_it: - RETiRet; -} - - -/* preprocess a batch of messages, that is ready them for actual processing. This is done - * as a first stage and totally in parallel to any other worker active in the system. So - * it helps us keep up the overall concurrency level. - * rgerhards, 2010-06-09 - */ -static inline rsRetVal -preprocessBatch(batch_t *pBatch) { - prop_t *ip; - prop_t *fqdn; - prop_t *localName; - prop_t *propFromHost = NULL; - prop_t *propFromHostIP = NULL; - int bSingleRuleset; - ruleset_t *batchRuleset; /* the ruleset used for all message inside the batch, if there is a single one */ - int bIsPermitted; - msg_t *pMsg; - int i; - rsRetVal localRet; - DEFiRet; - - bSingleRuleset = 1; - batchRuleset = (pBatch->nElem > 0) ? pBatch->pElem[0].pMsg->pRuleset : NULL; - - for(i = 0 ; i < pBatch->nElem && !*(pBatch->pbShutdownImmediate) ; i++) { - pMsg = pBatch->pElem[i].pMsg; - if((pMsg->msgFlags & NEEDS_ACLCHK_U) != 0) { - DBGPRINTF("msgConsumer: UDP ACL must be checked for message (hostname-based)\n"); - if(net.cvthname(pMsg->rcvFrom.pfrominet, &localName, &fqdn, &ip) != RS_RET_OK) - continue; - bIsPermitted = net.isAllowedSender2((uchar*)"UDP", - (struct sockaddr *)pMsg->rcvFrom.pfrominet, (char*)propGetSzStr(fqdn), 1); - if(!bIsPermitted) { - DBGPRINTF("Message from '%s' discarded, not a permitted sender host\n", - propGetSzStr(fqdn)); - pBatch->eltState[i] = BATCH_STATE_DISC; - } else { - /* save some of the info we obtained */ - MsgSetRcvFrom(pMsg, localName); - CHKiRet(MsgSetRcvFromIP(pMsg, ip)); - pMsg->msgFlags &= ~NEEDS_ACLCHK_U; - } - } - if((pMsg->msgFlags & NEEDS_PARSING) != 0) { - if((localRet = parser.ParseMsg(pMsg)) != RS_RET_OK) { - DBGPRINTF("Message discarded, parsing error %d\n", localRet); - pBatch->eltState[i] = BATCH_STATE_DISC; - } - } - if(pMsg->pRuleset != batchRuleset) - bSingleRuleset = 0; - } - - batchSetSingleRuleset(pBatch, bSingleRuleset); - -finalize_it: - if(propFromHost != NULL) - prop.Destruct(&propFromHost); - if(propFromHostIP != NULL) - prop.Destruct(&propFromHostIP); - RETiRet; -} - -/* The consumer of dequeued messages. This function is called by the - * queue engine on dequeueing of a message. It runs on a SEPARATE - * THREAD. It receives an array of pointers, which it must iterate - * over. We do not do any further batching, as this is of no benefit - * for the main queue. - */ -static rsRetVal -msgConsumer(void __attribute__((unused)) *notNeeded, batch_t *pBatch, int *pbShutdownImmediate) -{ - DEFiRet; - assert(pBatch != NULL); - pBatch->pbShutdownImmediate = pbShutdownImmediate; /* TODO: move this to batch creation! */ - preprocessBatch(pBatch); - ruleset.ProcessBatch(pBatch); -//TODO: the BATCH_STATE_COMM must be set somewhere down the road, but we -//do not have this yet and so we emulate -- 2010-06-10 -int i; - for(i = 0 ; i < pBatch->nElem && !*pbShutdownImmediate ; i++) { - pBatch->eltState[i] = BATCH_STATE_COMM; - } - RETiRet; -} - - -/* submit a message to the main message queue. This is primarily - * a hook to prevent the need for callers to know about the main message queue - * rgerhards, 2008-02-13 - */ -rsRetVal -submitMsg2(msg_t *pMsg) -{ - qqueue_t *pQueue; - ruleset_t *pRuleset; - DEFiRet; - - ISOBJ_TYPE_assert(pMsg, msg); - - pRuleset = MsgGetRuleset(pMsg); - pQueue = (pRuleset == NULL) ? pMsgQueue : ruleset.GetRulesetQueue(pRuleset); - - /* if a plugin logs a message during shutdown, the queue may no longer exist */ - if(pQueue == NULL) { - DBGPRINTF("submitMsg2() could not submit message - " - "queue does (no longer?) exist - ignored\n"); - FINALIZE; - } - - qqueueEnqMsg(pQueue, pMsg->flowCtlType, pMsg); - -finalize_it: - RETiRet; -} - -rsRetVal -submitMsg(msg_t *pMsg) -{ - return submitMsgWithDfltRatelimiter(pMsg); -} - - -/* submit multiple messages at once, very similar to submitMsg, just - * for multi_submit_t. All messages need to go into the SAME queue! - * rgerhards, 2009-06-16 - */ -rsRetVal -multiSubmitMsg2(multi_submit_t *pMultiSub) -{ - qqueue_t *pQueue; - ruleset_t *pRuleset; - DEFiRet; - assert(pMultiSub != NULL); - - if(pMultiSub->nElem == 0) - FINALIZE; - - pRuleset = MsgGetRuleset(pMultiSub->ppMsgs[0]); - pQueue = (pRuleset == NULL) ? pMsgQueue : ruleset.GetRulesetQueue(pRuleset); - - /* if a plugin logs a message during shutdown, the queue may no longer exist */ - if(pQueue == NULL) { - DBGPRINTF("multiSubmitMsg() could not submit message - " - "queue does (no longer?) exist - ignored\n"); - FINALIZE; - } - - iRet = pQueue->MultiEnq(pQueue, pMultiSub); - pMultiSub->nElem = 0; - -finalize_it: - RETiRet; -} -rsRetVal -multiSubmitMsg(multi_submit_t *pMultiSub) /* backward compat. level */ -{ - return multiSubmitMsg2(pMultiSub); -} - - -/* flush multiSubmit, e.g. at end of read records */ -rsRetVal -multiSubmitFlush(multi_submit_t *pMultiSub) -{ - DEFiRet; - if(pMultiSub->nElem > 0) { - iRet = multiSubmitMsg2(pMultiSub); - } - RETiRet; -} - - +/* function stems back to sysklogd */ static void reapchild() { @@ -680,178 +283,12 @@ reapchild() } -static void debug_switch() -{ - time_t tTime; - struct tm tp; - struct sigaction sigAct; - - datetime.GetTime(&tTime); - localtime_r(&tTime, &tp); - if(debugging_on == 0) { - debugging_on = 1; - dbgprintf("\n"); - dbgprintf("\n"); - dbgprintf("********************************************************************************\n"); - dbgprintf("Switching debugging_on to true at %2.2d:%2.2d:%2.2d\n", - tp.tm_hour, tp.tm_min, tp.tm_sec); - dbgprintf("********************************************************************************\n"); - } else { - dbgprintf("********************************************************************************\n"); - dbgprintf("Switching debugging_on to false at %2.2d:%2.2d:%2.2d\n", - tp.tm_hour, tp.tm_min, tp.tm_sec); - dbgprintf("********************************************************************************\n"); - dbgprintf("\n"); - dbgprintf("\n"); - debugging_on = 0; - } - - memset(&sigAct, 0, sizeof (sigAct)); - sigemptyset(&sigAct.sa_mask); - sigAct.sa_handler = debug_switch; - sigaction(SIGUSR1, &sigAct, NULL); -} - - -/* doDie() is a signal handler. If called, it sets the bFinished variable - * to indicate the program should terminate. However, it does not terminate - * it itself, because that causes issues with multi-threading. The actual - * termination is then done on the main thread. This solution might introduce - * a minimal delay, but it is much cleaner than the approach of doing everything - * inside the signal handler. - * rgerhards, 2005-10-26 - * Note: - * - we do not call DBGPRINTF() as this may cause us to block in case something - * with the threading is wrong. - * - we do not really care about the return state of write(), but we need this - * strange check we do to silence compiler warnings (thanks, Ubuntu!) - */ -static void doDie(int sig) -{ -# define MSG1 "DoDie called.\n" -# define MSG2 "DoDie called 5 times - unconditional exit\n" - static int iRetries = 0; /* debug aid */ - dbgprintf(MSG1); - if(Debug == DEBUG_FULL) { - if(write(1, MSG1, sizeof(MSG1) - 1)) {} - } - if(iRetries++ == 4) { - if(Debug == DEBUG_FULL) { - if(write(1, MSG2, sizeof(MSG2) - 1)) {} - } - abort(); - } - bFinished = sig; -# undef MSG1 -# undef MSG2 -} - - -/* Finalize and destruct all actions. - */ -static inline void -destructAllActions(void) -{ - ruleset.DestructAllActions(runConf); - bHaveMainQueue = 0; // flag that internal messages need to be temporarily stored -} - -/* die() is called when the program shall end. This typically only occurs - * during sigterm or during the initialization. - * As die() is intended to shutdown rsyslogd, it is - * safe to call exit() here. Just make sure that die() itself is not called - * at inapropriate places. As a general rule of thumb, it is a bad idea to add - * any calls to die() in new code! - * rgerhards, 2005-10-24 - */ -static void -die(int sig) +/* GPL code - maybe check BSD sources? */ +void +syslogd_die(void) { - char buf[256]; - - DBGPRINTF("exiting on signal %d\n", sig); - - /* IMPORTANT: we should close the inputs first, and THEN send our termination - * message. If we do it the other way around, logmsgInternal() may block on - * a full queue and the inputs still fill up that queue. Depending on the - * scheduling order, we may end up with logmsgInternal being held for a quite - * long time. When the inputs are terminated first, that should not happen - * because the queue is drained in parallel. The situation could only become - * an issue with extremely long running actions in a queue full environment. - * However, such actions are at least considered poorly written, if not - * outright wrong. So we do not care about this very remote problem. - * rgerhards, 2008-01-11 - */ - - /* close the inputs */ - DBGPRINTF("Terminating input threads...\n"); - glbl.SetGlobalInputTermination(); - thrdTerminateAll(); - - /* and THEN send the termination log message (see long comment above) */ - if(sig && runConf->globals.bLogStatusMsgs) { - (void) snprintf(buf, sizeof(buf) / sizeof(char), - " [origin software=\"rsyslogd\" " "swVersion=\"" VERSION \ - "\" x-pid=\"%d\" x-info=\"http://www.rsyslog.com\"]" " exiting on signal %d.", - (int) glblGetOurPid(), sig); - errno = 0; - logmsgInternal(NO_ERRCODE, LOG_SYSLOG|LOG_INFO, (uchar*)buf, 0); - } - /* we sleep for 50ms to give the queue a chance to pick up the exit message; - * otherwise we have seen cases where the message did not make it to log - * files, even on idle systems. - */ - srSleep(0, 50); - - /* drain queue (if configured so) and stop main queue worker thread pool */ - DBGPRINTF("Terminating main queue...\n"); - qqueueDestruct(&pMsgQueue); - pMsgQueue = NULL; - - /* Free ressources and close connections. This includes flushing any remaining - * repeated msgs. - */ - DBGPRINTF("Terminating outputs...\n"); - destructAllActions(); - - DBGPRINTF("all primary multi-thread sources have been terminated - now doing aux cleanup...\n"); - - DBGPRINTF("destructing current config...\n"); - rsconf.Destruct(&runConf); - - /* rger 2005-02-22 - * now clean up the in-memory structures. OK, the OS - * would also take care of that, but if we do it - * ourselfs, this makes finding memory leaks a lot - * easier. - */ - /* de-init some modules */ - modExitIminternal(); - - /*dbgPrintAllDebugInfo(); / * this is the last spot where this can be done - below output modules are unloaded! */ - - /* the following line cleans up CfSysLineHandlers that were not based on loadable - * modules. As such, they are not yet cleared. - */ - unregCfSysLineHdlrs(); - - /* destruct our global properties */ - if(pInternalInputName != NULL) - prop.Destruct(&pInternalInputName); - - /* terminate the remaining classes */ - GlobalClassExit(); - - module.UnloadAndDestructAll(eMOD_LINK_ALL); - - DBGPRINTF("Clean shutdown completed, bye\n"); - /* dbgClassExit MUST be the last one, because it de-inits the debug system */ - dbgClassExit(); - - /* NO CODE HERE - dbgClassExit() must be the last thing before exit()! */ remove_pid(PidFile); - exit(0); /* "good" exit, this is the terminator function for rsyslog [die()] */ } /* @@ -864,318 +301,6 @@ static void doexit() exit(0); /* "good" exit, only during child-creation */ } -#if 0 /* TODO: re-enable, currently not used */ -/* helper to generateConfigDAG, to print out all actions via - * the llExecFunc() facility. - * rgerhards, 2007-08-02 - */ -struct dag_info { - FILE *fp; /* output file */ - int iActUnit; /* current action unit number */ - int iAct; /* current action in unit */ - int bDiscarded; /* message discarded (config error) */ - }; -DEFFUNC_llExecFunc(generateConfigDAGAction) -{ - action_t *pAction; - uchar *pszModName; - uchar *pszVertexName; - struct dag_info *pDagInfo; - DEFiRet; - - pDagInfo = (struct dag_info*) pParam; - pAction = (action_t*) pData; - - pszModName = module.GetStateName(pAction->pMod); - - /* vertex */ - if(pAction->pszName == NULL) { - if(!strcmp((char*)pszModName, "builtin-discard")) - pszVertexName = (uchar*)"discard"; - else - pszVertexName = pszModName; - } else { - pszVertexName = pAction->pszName; - } - - fprintf(pDagInfo->fp, "\tact%d_%d\t\t[label=\"%s\"%s%s]\n", - pDagInfo->iActUnit, pDagInfo->iAct, pszVertexName, - pDagInfo->bDiscarded ? " style=dotted color=red" : "", - (pAction->pQueue->qType == QUEUETYPE_DIRECT) ? "" : " shape=hexagon" - ); - - /* edge */ - if(pDagInfo->iAct == 0) { - } else { - fprintf(pDagInfo->fp, "\tact%d_%d -> act%d_%d[%s%s]\n", - pDagInfo->iActUnit, pDagInfo->iAct - 1, - pDagInfo->iActUnit, pDagInfo->iAct, - pDagInfo->bDiscarded ? " style=dotted color=red" : "", - pAction->bExecWhenPrevSusp ? " label=\"only if\\nsuspended\"" : "" ); - } - - /* check for discard */ - if(!strcmp((char*) pszModName, "builtin-discard")) { - fprintf(pDagInfo->fp, "\tact%d_%d\t\t[shape=box]\n", - pDagInfo->iActUnit, pDagInfo->iAct); - pDagInfo->bDiscarded = 1; - } - - - ++pDagInfo->iAct; - - RETiRet; -} - - -/* create config DAG - * This functions takes a rsyslog config and produces a .dot file for use - * with graphviz (http://www.graphviz.org). This is done in an effort to - * document, and also potentially troubleshoot, configurations. Plus, I - * consider it a nice feature to explain some concepts. Note that the - * current version only produces a graph with relatively little information. - * This is a foundation that may be later expanded (if it turns out to be - * useful enough). - * rgerhards, 2009-05-11 - */ -static rsRetVal -generateConfigDAG(uchar *pszDAGFile) -{ - //rule_t *f; - FILE *fp; - int iActUnit = 1; - //int bHasFilter = 0; /* filter associated with this action unit? */ - //int bHadFilter; - //int i; - struct dag_info dagInfo; - //char *pszFilterName; - char szConnectingNode[64]; - DEFiRet; - - assert(pszDAGFile != NULL); - - logmsgInternal(NO_ERRCODE, LOG_SYSLOG|LOG_INFO, (uchar*) - "Configuration graph generation is unfortunately disabled " - "in the current code base.", 0); - ABORT_FINALIZE(RS_RET_FILENAME_INVALID); - - if((fp = fopen((char*) pszDAGFile, "w")) == NULL) { - logmsgInternal(NO_ERRCODE, LOG_SYSLOG|LOG_INFO, (uchar*) - "configuraton graph output file could not be opened, none generated", 0); - ABORT_FINALIZE(RS_RET_FILENAME_INVALID); - } - - dagInfo.fp = fp; - - /* from here on, we assume writes go well. This here is a really - * unimportant utility function and if something goes wrong, it has - * almost no effect. So let's not overdo this... - */ - fprintf(fp, "# graph created by rsyslog " VERSION "\n\n" - "# use the dot tool from http://www.graphviz.org to visualize!\n" - "digraph rsyslogConfig {\n" - "\tinputs [shape=tripleoctagon]\n" - "\tinputs -> act0_0\n" - "\tact0_0 [label=\"main\\nqueue\" shape=hexagon]\n" - /*"\tmainq -> act1_0\n"*/ - ); - strcpy(szConnectingNode, "act0_0"); - dagInfo.bDiscarded = 0; - -/* TODO: re-enable! */ -#if 0 - for(f = Files; f != NULL ; f = f->f_next) { - /* BSD-Style filters are currently ignored */ - bHadFilter = bHasFilter; - if(f->f_filter_type == FILTER_PRI) { - bHasFilter = 0; - for (i = 0; i <= LOG_NFACILITIES; i++) - if (f->f_filterData.f_pmask[i] != 0xff) { - bHasFilter = 1; - break; - } - } else { - bHasFilter = 1; - } - - /* we know we have a filter, so it can be false */ - switch(f->f_filter_type) { - case FILTER_PRI: - pszFilterName = "pri filter"; - break; - case FILTER_PROP: - pszFilterName = "property filter"; - break; - case FILTER_EXPR: - pszFilterName = "script filter"; - break; - } - - /* write action unit node */ - if(bHasFilter) { - fprintf(fp, "\t%s -> act%d_end\t[label=\"%s:\\nfalse\"]\n", - szConnectingNode, iActUnit, pszFilterName); - fprintf(fp, "\t%s -> act%d_0\t[label=\"%s:\\ntrue\"]\n", - szConnectingNode, iActUnit, pszFilterName); - fprintf(fp, "\tact%d_end\t\t\t\t[shape=point]\n", iActUnit); - snprintf(szConnectingNode, sizeof(szConnectingNode), "act%d_end", iActUnit); - } else { - fprintf(fp, "\t%s -> act%d_0\t[label=\"no filter\"]\n", - szConnectingNode, iActUnit); - snprintf(szConnectingNode, sizeof(szConnectingNode), "act%d_0", iActUnit); - } - - /* draw individual nodes */ - dagInfo.iActUnit = iActUnit; - dagInfo.iAct = 0; - dagInfo.bDiscarded = 0; - llExecFunc(&f->llActList, generateConfigDAGAction, &dagInfo); /* actions */ - - /* finish up */ - if(bHasFilter && !dagInfo.bDiscarded) { - fprintf(fp, "\tact%d_%d -> %s\n", - iActUnit, dagInfo.iAct - 1, szConnectingNode); - } - - ++iActUnit; - } -#endif - - fprintf(fp, "\t%s -> act%d_0\n", szConnectingNode, iActUnit); - fprintf(fp, "\tact%d_0\t\t[label=discard shape=box]\n" - "}\n", iActUnit); - fclose(fp); - -finalize_it: - RETiRet; -} -#endif - - -/* create a main message queue, now also used for ruleset queues. This function - * needs to be moved to some other module, but it is considered acceptable for - * the time being (remember that we want to restructure config processing at large!). - * rgerhards, 2009-10-27 - */ -rsRetVal createMainQueue(qqueue_t **ppQueue, uchar *pszQueueName, struct cnfparamvals *queueParams) -{ - struct queuefilenames_s *qfn; - uchar *qfname = NULL; - static int qfn_renamenum = 0; - uchar qfrenamebuf[1024]; - DEFiRet; - - /* create message queue */ - CHKiRet_Hdlr(qqueueConstruct(ppQueue, ourConf->globals.mainQ.MainMsgQueType, ourConf->globals.mainQ.iMainMsgQueueNumWorkers, ourConf->globals.mainQ.iMainMsgQueueSize, msgConsumer)) { - /* no queue is fatal, we need to give up in that case... */ - errmsg.LogError(0, iRet, "could not create (ruleset) main message queue"); \ - } - /* name our main queue object (it's not fatal if it fails...) */ - obj.SetName((obj_t*) (*ppQueue), pszQueueName); - - if(queueParams == NULL) { /* use legacy parameters? */ - /* ... set some properties ... */ - # define setQPROP(func, directive, data) \ - CHKiRet_Hdlr(func(*ppQueue, data)) { \ - errmsg.LogError(0, NO_ERRCODE, "Invalid " #directive ", error %d. Ignored, running with default setting", iRet); \ - } - # define setQPROPstr(func, directive, data) \ - CHKiRet_Hdlr(func(*ppQueue, data, (data == NULL)? 0 : strlen((char*) data))) { \ - errmsg.LogError(0, NO_ERRCODE, "Invalid " #directive ", error %d. Ignored, running with default setting", iRet); \ - } - - if(ourConf->globals.mainQ.pszMainMsgQFName != NULL) { - /* check if the queue file name is unique, else emit an error */ - for(qfn = queuefilenames ; qfn != NULL ; qfn = qfn->next) { - dbgprintf("check queue file name '%s' vs '%s'\n", qfn->name, ourConf->globals.mainQ.pszMainMsgQFName ); - if(!ustrcmp(qfn->name, ourConf->globals.mainQ.pszMainMsgQFName)) { - snprintf((char*)qfrenamebuf, sizeof(qfrenamebuf), "%d-%s-%s", - ++qfn_renamenum, ourConf->globals.mainQ.pszMainMsgQFName, - (pszQueueName == NULL) ? "NONAME" : (char*)pszQueueName); - qfname = ustrdup(qfrenamebuf); - errmsg.LogError(0, NO_ERRCODE, "Error: queue file name '%s' already in use " - " - using '%s' instead", ourConf->globals.mainQ.pszMainMsgQFName, qfname); - break; - } - } - if(qfname == NULL) - qfname = ustrdup(ourConf->globals.mainQ.pszMainMsgQFName); - qfn = malloc(sizeof(struct queuefilenames_s)); - qfn->name = qfname; - qfn->next = queuefilenames; - queuefilenames = qfn; - } - - setQPROP(qqueueSetMaxFileSize, "$MainMsgQueueFileSize", ourConf->globals.mainQ.iMainMsgQueMaxFileSize); - setQPROP(qqueueSetsizeOnDiskMax, "$MainMsgQueueMaxDiskSpace", ourConf->globals.mainQ.iMainMsgQueMaxDiskSpace); - setQPROP(qqueueSetiDeqBatchSize, "$MainMsgQueueDequeueBatchSize", ourConf->globals.mainQ.iMainMsgQueDeqBatchSize); - setQPROPstr(qqueueSetFilePrefix, "$MainMsgQueueFileName", qfname); - setQPROP(qqueueSetiPersistUpdCnt, "$MainMsgQueueCheckpointInterval", ourConf->globals.mainQ.iMainMsgQPersistUpdCnt); - setQPROP(qqueueSetbSyncQueueFiles, "$MainMsgQueueSyncQueueFiles", ourConf->globals.mainQ.bMainMsgQSyncQeueFiles); - setQPROP(qqueueSettoQShutdown, "$MainMsgQueueTimeoutShutdown", ourConf->globals.mainQ.iMainMsgQtoQShutdown ); - setQPROP(qqueueSettoActShutdown, "$MainMsgQueueTimeoutActionCompletion", ourConf->globals.mainQ.iMainMsgQtoActShutdown); - setQPROP(qqueueSettoWrkShutdown, "$MainMsgQueueWorkerTimeoutThreadShutdown", ourConf->globals.mainQ.iMainMsgQtoWrkShutdown); - setQPROP(qqueueSettoEnq, "$MainMsgQueueTimeoutEnqueue", ourConf->globals.mainQ.iMainMsgQtoEnq); - setQPROP(qqueueSetiHighWtrMrk, "$MainMsgQueueHighWaterMark", ourConf->globals.mainQ.iMainMsgQHighWtrMark); - setQPROP(qqueueSetiLowWtrMrk, "$MainMsgQueueLowWaterMark", ourConf->globals.mainQ.iMainMsgQLowWtrMark); - setQPROP(qqueueSetiDiscardMrk, "$MainMsgQueueDiscardMark", ourConf->globals.mainQ.iMainMsgQDiscardMark); - setQPROP(qqueueSetiDiscardSeverity, "$MainMsgQueueDiscardSeverity", ourConf->globals.mainQ.iMainMsgQDiscardSeverity); - setQPROP(qqueueSetiMinMsgsPerWrkr, "$MainMsgQueueWorkerThreadMinimumMessages", ourConf->globals.mainQ.iMainMsgQWrkMinMsgs); - setQPROP(qqueueSetbSaveOnShutdown, "$MainMsgQueueSaveOnShutdown", ourConf->globals.mainQ.bMainMsgQSaveOnShutdown); - setQPROP(qqueueSetiDeqSlowdown, "$MainMsgQueueDequeueSlowdown", ourConf->globals.mainQ.iMainMsgQDeqSlowdown); - setQPROP(qqueueSetiDeqtWinFromHr, "$MainMsgQueueDequeueTimeBegin", ourConf->globals.mainQ.iMainMsgQueueDeqtWinFromHr); - setQPROP(qqueueSetiDeqtWinToHr, "$MainMsgQueueDequeueTimeEnd", ourConf->globals.mainQ.iMainMsgQueueDeqtWinToHr); - - # undef setQPROP - # undef setQPROPstr - } else { /* use new style config! */ - qqueueSetDefaultsRulesetQueue(*ppQueue); - qqueueApplyCnfParam(*ppQueue, queueParams); - } - - /* ... and finally start the queue! */ - CHKiRet_Hdlr(qqueueStart(*ppQueue)) { - /* no queue is fatal, we need to give up in that case... */ - errmsg.LogError(0, iRet, "could not start (ruleset) main message queue"); \ - } - RETiRet; -} - - -/* INIT -- Initialize syslogd - * Note that if iConfigVerify is set, only the config file is verified but nothing - * else happens. -- rgerhards, 2008-07-28 - */ -static rsRetVal -init(void) -{ - char bufStartUpMsg[512]; - struct sigaction sigAct; - DEFiRet; - - memset(&sigAct, 0, sizeof (sigAct)); - sigemptyset(&sigAct.sa_mask); - sigAct.sa_handler = sighup_handler; - sigaction(SIGHUP, &sigAct, NULL); - - CHKiRet(rsconf.Activate(ourConf)); - DBGPRINTF(" started.\n"); - - /* we now generate the startup message. It now includes everything to - * identify this instance. -- rgerhards, 2005-08-17 - */ - if(ourConf->globals.bLogStatusMsgs) { - snprintf(bufStartUpMsg, sizeof(bufStartUpMsg)/sizeof(char), - " [origin software=\"rsyslogd\" " "swVersion=\"" VERSION \ - "\" x-pid=\"%d\" x-info=\"http://www.rsyslog.com\"] start", - (int) glblGetOurPid()); - logmsgInternal(NO_ERRCODE, LOG_SYSLOG|LOG_INFO, (uchar*)bufStartUpMsg, 0); - } - -finalize_it: - RETiRet; -} - /* * The following function is resposible for handling a SIGHUP signal. Since @@ -1183,7 +308,7 @@ finalize_it: * doing this during a signal handler. Instead this function simply sets * a flag variable which will tells the main loop to do "the right thing". */ -void sighup_handler() +void syslogd_sighup_handler() { struct sigaction sigAct; @@ -1191,266 +316,50 @@ void sighup_handler() memset(&sigAct, 0, sizeof (sigAct)); sigemptyset(&sigAct.sa_mask); - sigAct.sa_handler = sighup_handler; + sigAct.sa_handler = syslogd_sighup_handler; sigaction(SIGHUP, &sigAct, NULL); } -void sigttin_handler() -{ -} - -/* this function pulls all internal messages from the buffer - * and puts them into the processing engine. - * We can only do limited error handling, as this would not - * really help us. TODO: add error messages? - * rgerhards, 2007-08-03 - */ -static inline void processImInternal(void) -{ - msg_t *pMsg; - - while(iminternalRemoveMsg(&pMsg) == RS_RET_OK) { - submitMsgWithDfltRatelimiter(pMsg); - } -} - - -/* helper to doHUP(), this "HUPs" each action. The necessary locking - * is done inside the action class and nothing we need to take care of. - * rgerhards, 2008-10-22 - */ -DEFFUNC_llExecFunc(doHUPActions) -{ - BEGINfunc - actionCallHUPHdlr((action_t*) pData); - ENDfunc - return RS_RET_OK; /* we ignore errors, we can not do anything either way */ -} - - -/* This function processes a HUP after one has been detected. Note that this - * is *NOT* the sighup handler. The signal is recorded by the handler, that record - * detected inside the mainloop and then this function is called to do the - * real work. -- rgerhards, 2008-10-22 - * Note: there is a VERY slim chance of a data race when the hostname is reset. - * We prefer to take this risk rather than sync all accesses, because to the best - * of my analysis it can not really hurt (the actual property is reference-counted) - * but the sync would require some extra CPU for *each* message processed. - * rgerhards, 2012-04-11 - */ -static inline void -doHUP(void) -{ - char buf[512]; - - if(ourConf->globals.bLogStatusMsgs) { - snprintf(buf, sizeof(buf) / sizeof(char), - " [origin software=\"rsyslogd\" " "swVersion=\"" VERSION - "\" x-pid=\"%d\" x-info=\"http://www.rsyslog.com\"] rsyslogd was HUPed", - (int) glblGetOurPid()); - errno = 0; - logmsgInternal(NO_ERRCODE, LOG_SYSLOG|LOG_INFO, (uchar*)buf, 0); - } - - queryLocalHostname(); /* re-read our name */ - ruleset.IterateAllActions(ourConf, doHUPActions, NULL); -} - - -/* This is the main processing loop. It is called after successful initialization. - * When it returns, the syslogd terminates. - * Its sole function is to provide some housekeeping things. The real work is done - * by the other threads spawned. - */ -static void -mainloop(void) -{ - struct timeval tvSelectTimeout; - - BEGINfunc - /* first check if we have any internal messages queued and spit them out. We used - * to do that on any loop iteration, but that is no longer necessry. The reason - * is that once we reach this point here, we always run on multiple threads and - * thus the main queue is properly initialized. -- rgerhards, 2008-06-09 - */ - processImInternal(); - - while(!bFinished){ - /* this is now just a wait - please note that we do use a near-"eternal" - * timeout of 1 day. This enables us to help safe the environment - * by not unnecessarily awaking rsyslog on a regular tick (just think - * powertop, for example). In that case, we primarily wait for a signal, - * but a once-a-day wakeup should be quite acceptable. -- rgerhards, 2008-06-09 - */ - tvSelectTimeout.tv_sec = 86400 /*1 day*/; - tvSelectTimeout.tv_usec = 0; - select(1, NULL, NULL, NULL, &tvSelectTimeout); - if(bFinished) - break; /* exit as quickly as possible */ - - if(bHadHUP) { - doHUP(); - bHadHUP = 0; - continue; - } - } - ENDfunc -} - -/* print version and compile-time setting information. - */ -static void printVersion(void) -{ - printf("rsyslogd %s, ", VERSION); - printf("compiled with:\n"); -#ifdef FEATURE_REGEXP - printf("\tFEATURE_REGEXP:\t\t\t\tYes\n"); -#else - printf("\tFEATURE_REGEXP:\t\t\t\tNo\n"); -#endif -#if defined(_LARGE_FILES) || (defined (_FILE_OFFSET_BITS) && _FILE_OFFSET_BITS >= 64) - printf("\tFEATURE_LARGEFILE:\t\t\tYes\n"); -#else - printf("\tFEATURE_LARGEFILE:\t\t\tNo\n"); -#endif -#if defined(SYSLOG_INET) && defined(USE_GSSAPI) - printf("\tGSSAPI Kerberos 5 support:\t\tYes\n"); -#else - printf("\tGSSAPI Kerberos 5 support:\t\tNo\n"); -#endif -#ifndef NDEBUG - printf("\tFEATURE_DEBUG (debug build, slow code):\tYes\n"); -#else - printf("\tFEATURE_DEBUG (debug build, slow code):\tNo\n"); -#endif -#ifdef HAVE_ATOMIC_BUILTINS - printf("\t32bit Atomic operations supported:\tYes\n"); -#else - printf("\t32bit Atomic operations supported:\tNo\n"); -#endif -#ifdef HAVE_ATOMIC_BUILTINS_64BIT - printf("\t64bit Atomic operations supported:\tYes\n"); -#else - printf("\t64bit Atomic operations supported:\tNo\n"); -#endif -#ifdef RTINST - printf("\tRuntime Instrumentation (slow code):\tYes\n"); -#else - printf("\tRuntime Instrumentation (slow code):\tNo\n"); -#endif -#ifdef USE_LIBUUID - printf("\tuuid support:\t\t\t\tYes\n"); -#else - printf("\tuuid support:\t\t\t\tNo\n"); -#endif - printf("\nSee http://www.rsyslog.com for more information.\n"); -} - - -/* Method to initialize all global classes and use the objects that we need. - * rgerhards, 2008-01-04 - * rgerhards, 2008-04-16: the actual initialization is now carried out by the runtime - */ -static rsRetVal -InitGlobalClasses(void) +/* obtain ptrs to all clases we need. */ +rsRetVal +syslogd_obtainClassPointers(void) { DEFiRet; char *pErrObj; /* tells us which object failed if that happens (useful for troubleshooting!) */ - /* Intialize the runtime system */ - pErrObj = "rsyslog runtime"; /* set in case the runtime errors before setting an object */ - CHKiRet(rsrtInit(&pErrObj, &obj)); - CHKiRet(rsrtSetErrLogger(submitErrMsg)); /* set out error handler */ + CHKiRet(objGetObjInterface(&obj)); /* this provides the root pointer for all other queries */ /* Now tell the system which classes we need ourselfs */ pErrObj = "glbl"; CHKiRet(objUse(glbl, CORE_COMPONENT)); - pErrObj = "errmsg"; - CHKiRet(objUse(errmsg, CORE_COMPONENT)); - pErrObj = "module"; - CHKiRet(objUse(module, CORE_COMPONENT)); - pErrObj = "datetime"; - CHKiRet(objUse(datetime, CORE_COMPONENT)); - pErrObj = "ruleset"; - CHKiRet(objUse(ruleset, CORE_COMPONENT)); - pErrObj = "conf"; - CHKiRet(objUse(conf, CORE_COMPONENT)); - pErrObj = "prop"; - CHKiRet(objUse(prop, CORE_COMPONENT)); - pErrObj = "parser"; - CHKiRet(objUse(parser, CORE_COMPONENT)); - pErrObj = "rsconf"; - CHKiRet(objUse(rsconf, CORE_COMPONENT)); - - /* intialize some dummy classes that are not part of the runtime */ - pErrObj = "action"; - CHKiRet(actionClassInit()); - pErrObj = "template"; - CHKiRet(templateInit()); /* TODO: the dependency on net shall go away! -- rgerhards, 2008-03-07 */ pErrObj = "net"; CHKiRet(objUse(net, LM_NET_FILENAME)); - dnscacheInit(); - initRainerscript(); - ratelimitModInit(); finalize_it: if(iRet != RS_RET_OK) { /* we know we are inside the init sequence, so we can safely emit * messages to stderr. -- rgerhards, 2008-04-02 */ - fprintf(stderr, "Error during class init for object '%s' - failing...\n", pErrObj); + fprintf(stderr, "Error obtaining object '%s' - failing...\n", pErrObj); } RETiRet; } -/* Method to exit all global classes. We do not do any error checking here, - * because that wouldn't help us at all. So better try to deinit blindly - * as much as succeeds (which usually means everything will). We just must - * be careful to do the de-init in the opposite order of the init, because - * of the dependencies. However, its not as important this time, because - * we have reference counting. - * rgerhards, 2008-03-10 - */ -static rsRetVal -GlobalClassExit(void) +void +syslogd_releaseClassPointers(void) { - DEFiRet; - - /* first, release everything we used ourself */ objRelease(net, LM_NET_FILENAME);/* TODO: the dependency on net shall go away! -- rgerhards, 2008-03-07 */ - objRelease(prop, CORE_COMPONENT); - objRelease(conf, CORE_COMPONENT); - objRelease(ruleset, CORE_COMPONENT); - parserClassExit(); /* this is hack, currently core_modules do not get this automatically called */ - rsconfClassExit(); /* this is hack, currently core_modules do not get this automatically called */ - objRelease(datetime, CORE_COMPONENT); - - /* TODO: implement the rest of the deinit */ - /* dummy "classes */ - strExit(); - ratelimitModExit(); - -#if 0 - CHKiRet(objGetObjInterface(&obj)); /* this provides the root pointer for all other queries */ - /* the following classes were intialized by objClassInit() */ - CHKiRet(objUse(errmsg, CORE_COMPONENT)); - CHKiRet(objUse(module, CORE_COMPONENT)); -#endif - dnscacheDeinit(); - rsrtExit(); /* *THIS* *MUST/SHOULD?* always be the first class initilizer being called (except debug)! */ - - RETiRet; } /* query our host and domain names - we need to do this early as we may emit * rgerhards, 2012-04-11 */ -static rsRetVal +rsRetVal queryLocalHostname(void) { uchar *LocalHostName; @@ -1515,8 +424,10 @@ queryLocalHostname(void) } } + /* Marius Tomaschewski <mt@suse.com> contribution */ /* LocalDomain is "" or part of LocalHostName, allocate a new string */ CHKmalloc(LocalDomain = (uchar*)strdup((char*)LocalDomain)); + /* Marius Tomaschewski <mt@suse.com> contribution */ /* Convert to lower case to recognize the correct domain laterly */ for(p = LocalDomain ; *p ; p++) @@ -1529,90 +440,24 @@ queryLocalHostname(void) glbl.SetLocalHostName(LocalHostName); glbl.SetLocalDomain(LocalDomain); + /* Canonical contribution - ASL 2.0 fine (email exchange 2014-05-27) */ if ( strlen((char*)LocalDomain) ) { CHKmalloc(LocalFQDNName = (uchar*)malloc(strlen((char*)LocalDomain)+strlen((char*)LocalHostName)+2));/* one for dot, one for NUL! */ if ( sprintf((char*)LocalFQDNName,"%s.%s",(char*)LocalHostName,(char*)LocalDomain) ) glbl.SetLocalFQDNName(LocalFQDNName); } + /* end canonical contrib */ glbl.GenerateLocalHostNameProperty(); /* must be redone after conf processing, FQDN setting may have changed */ finalize_it: RETiRet; } - -/* some support for command line option parsing. Any non-trivial options must be - * buffered until the complete command line has been parsed. This is necessary to - * prevent dependencies between the options. That, in turn, means we need to have - * something that is capable of buffering options and there values. The follwing - * functions handle that. - * rgerhards, 2008-04-04 - */ -typedef struct bufOpt { - struct bufOpt *pNext; - char optchar; - char *arg; -} bufOpt_t; -static bufOpt_t *bufOptRoot = NULL; -static bufOpt_t *bufOptLast = NULL; - -/* add option buffer */ -static rsRetVal -bufOptAdd(char opt, char *arg) -{ - DEFiRet; - bufOpt_t *pBuf; - - if((pBuf = MALLOC(sizeof(bufOpt_t))) == NULL) - ABORT_FINALIZE(RS_RET_OUT_OF_MEMORY); - - pBuf->optchar = opt; - pBuf->arg = arg; - pBuf->pNext = NULL; - - if(bufOptLast == NULL) { - bufOptRoot = pBuf; /* then there is also no root! */ - } else { - bufOptLast->pNext = pBuf; - } - bufOptLast = pBuf; - -finalize_it: - RETiRet; -} - - - -/* remove option buffer from top of list, return values and destruct buffer itself. - * returns RS_RET_END_OF_LINKEDLIST when no more options are present. - * (we use int *opt instead of char *opt to keep consistent with getopt()) - */ -static rsRetVal -bufOptRemove(int *opt, char **arg) -{ - DEFiRet; - bufOpt_t *pBuf; - - if(bufOptRoot == NULL) - ABORT_FINALIZE(RS_RET_END_OF_LINKEDLIST); - pBuf = bufOptRoot; - - *opt = pBuf->optchar; - *arg = pBuf->arg; - - bufOptRoot = pBuf->pNext; - free(pBuf); - -finalize_it: - RETiRet; -} - - /* global initialization, to be done only once and before the mainloop is started. * rgerhards, 2008-07-28 (extracted from realMain()) */ -static rsRetVal -doGlblProcessInit(void) +rsRetVal +syslogd_doGlblProcessInit(void) { struct sigaction sigAct; int num_fds; @@ -1630,8 +475,10 @@ doGlblProcessInit(void) sigAct.sa_handler = doexit; sigaction(SIGTERM, &sigAct, NULL); + /* RH contribution */ /* stop writing debug messages to stdout (if debugging is on) */ stddbg = -1; + /* end RH contribution */ dbgprintf("ready for forking\n"); if (fork()) { @@ -1715,7 +562,8 @@ doGlblProcessInit(void) } else { - fputs("Pidfile (and pid) already exist.\n", stderr); + fprintf(stderr, "rsyslogd: pidfile '%s' and pid %d already exist.\n", + PidFile, getpid()); exit(1); /* exit during startup - questionable */ } glblSetOurPid(getpid()); @@ -1727,16 +575,16 @@ doGlblProcessInit(void) sigaction(SIGSEGV, &sigAct, NULL); sigAct.sa_handler = sigsegvHdlr; sigaction(SIGABRT, &sigAct, NULL); - sigAct.sa_handler = doDie; + sigAct.sa_handler = rsyslogdDoDie; sigaction(SIGTERM, &sigAct, NULL); - sigAct.sa_handler = Debug ? doDie : SIG_IGN; + sigAct.sa_handler = Debug ? rsyslogdDoDie : SIG_IGN; sigaction(SIGINT, &sigAct, NULL); sigaction(SIGQUIT, &sigAct, NULL); sigAct.sa_handler = reapchild; sigaction(SIGCHLD, &sigAct, NULL); - sigAct.sa_handler = Debug ? debug_switch : SIG_IGN; + sigAct.sa_handler = Debug ? rsyslogdDebugSwitch : SIG_IGN; sigaction(SIGUSR1, &sigAct, NULL); - sigAct.sa_handler = sigttin_handler; + sigAct.sa_handler = rsyslogd_sigttin_handler; sigaction(SIGTTIN, &sigAct, NULL); /* (ab)used to interrupt input threads */ sigAct.sa_handler = SIG_IGN; sigaction(SIGPIPE, &sigAct, NULL); @@ -1746,325 +594,10 @@ doGlblProcessInit(void) } -/* This is the main entry point into rsyslogd. Over time, we should try to - * modularize it a bit more... - */ -int realMain(int argc, char **argv) +void +syslogdInit(void) { - rsRetVal localRet; - int ch; - extern int optind; - extern char *optarg; - int bEOptionWasGiven = 0; - int iHelperUOpt; - int bChDirRoot = 1; /* change the current working directory to "/"? */ - char *arg; /* for command line option processing */ - char cwdbuf[128]; /* buffer to obtain/display current working directory */ - DEFiRet; - - /* first, parse the command line options. We do not carry out any actual work, just - * see what we should do. This relieves us from certain anomalies and we can process - * the parameters down below in the correct order. For example, we must know the - * value of -M before we can do the init, but at the same time we need to have - * the base classes init before we can process most of the options. Now, with the - * split of functionality, this is no longer a problem. Thanks to varmofekoj for - * suggesting this algo. - * Note: where we just need to set some flags and can do so without knowledge - * of other options, we do this during the inital option processing. - * rgerhards, 2008-04-04 - */ - while((ch = getopt(argc, argv, "46a:Ac:dDef:g:hi:l:m:M:nN:op:qQr::s:t:T:u:vwx")) != EOF) { - switch((char)ch) { - case '4': - case '6': - case 'A': - case 'a': - case 'f': /* configuration file */ - case 'h': - case 'i': /* pid file name */ - case 'l': - case 'm': /* mark interval */ - case 'n': /* don't fork */ - case 'N': /* enable config verify mode */ - case 'o': - case 'p': - case 'q': /* add hostname if DNS resolving has failed */ - case 'Q': /* dont resolve hostnames in ACL to IPs */ - case 's': - case 'T': /* chroot on startup (primarily for testing) */ - case 'u': /* misc user settings */ - case 'w': /* disable disallowed host warnings */ - case 'x': /* disable dns for remote messages */ - case 'g': /* enable tcp gssapi logging */ - case 'r': /* accept remote messages */ - case 't': /* enable tcp logging */ - CHKiRet(bufOptAdd(ch, optarg)); - break; - case 'c': /* compatibility mode */ - fprintf(stderr, "rsyslogd: error: option -c is no longer supported - ignored\n"); - break; - case 'd': /* debug - must be handled now, so that debug is active during init! */ - debugging_on = 1; - Debug = 1; - yydebug = 1; - break; - case 'D': /* BISON debug */ - yydebug = 1; - break; - case 'e': /* log every message (no repeat message supression) */ - bEOptionWasGiven = 1; - break; - case 'M': /* default module load path -- this MUST be carried out immediately! */ - glblModPath = (uchar*) optarg; - break; - case 'v': /* MUST be carried out immediately! */ - printVersion(); - exit(0); /* exit for -v option - so this is a "good one" */ - case '?': - default: - usage(); - } - } - - if(argc - optind) - usage(); - - DBGPRINTF("rsyslogd %s startup, module path '%s', cwd:%s\n", - VERSION, glblModPath == NULL ? "" : (char*)glblModPath, - getcwd(cwdbuf, sizeof(cwdbuf))); - - /* we are done with the initial option parsing and processing. Now we init the system. */ - - ppid = getpid(); - - CHKiRet_Hdlr(InitGlobalClasses()) { - fprintf(stderr, "rsyslogd initializiation failed - global classes could not be initialized.\n" - "Did you do a \"make install\"?\n" - "Suggested action: run rsyslogd with -d -n options to see what exactly " - "fails.\n"); - FINALIZE; - } - - /* doing some core initializations */ - - /* we need to create the inputName property (only once during our lifetime) */ - CHKiRet(prop.Construct(&pInternalInputName)); - CHKiRet(prop.SetString(pInternalInputName, UCHAR_CONSTANT("rsyslogd"), sizeof("rsyslogd") - 1)); - CHKiRet(prop.ConstructFinalize(pInternalInputName)); - - /* get our host and domain names - we need to do this early as we may emit - * error log messages, which need the correct hostname. -- rgerhards, 2008-04-04 - */ - queryLocalHostname(); - - /* initialize the objects */ - if((iRet = modInitIminternal()) != RS_RET_OK) { - fprintf(stderr, "fatal error: could not initialize errbuf object (error code %d).\n", - iRet); - exit(1); /* "good" exit, leaving at init for fatal error */ - } - - - /* END core initializations - we now come back to carrying out command line options*/ - - while((iRet = bufOptRemove(&ch, &arg)) == RS_RET_OK) { - DBGPRINTF("deque option %c, optarg '%s'\n", ch, (arg == NULL) ? "" : arg); - switch((char)ch) { - case '4': - glbl.SetDefPFFamily(PF_INET); - break; - case '6': - glbl.SetDefPFFamily(PF_INET6); - break; - case 'A': - send_to_all++; - break; - case 'a': - fprintf(stderr, "rsyslogd: error -a is no longer supported, use module imuxsock instead"); - break; - case 'f': /* configuration file */ - ConfFile = (uchar*) arg; - break; - case 'g': /* enable tcp gssapi logging */ - fprintf(stderr, "rsyslogd: -g option no longer supported - ignored\n"); - case 'h': - fprintf(stderr, "rsyslogd: error -h is no longer supported - ignored"); - break; - case 'i': /* pid file name */ - PidFile = arg; - break; - case 'l': - if(glbl.GetLocalHosts() != NULL) { - fprintf (stderr, "rsyslogd: Only one -l argument allowed, the first one is taken.\n"); - } else { - glbl.SetLocalHosts(crunch_list(arg)); - } - break; - case 'm': /* mark interval */ - fprintf(stderr, "rsyslogd: error -m is no longer supported - use immark instead"); - break; - case 'n': /* don't fork */ - doFork = 0; - break; - case 'N': /* enable config verify mode */ - iConfigVerify = atoi(arg); - break; - case 'o': - fprintf(stderr, "error -o is no longer supported, use module imuxsock instead"); - break; - case 'p': - fprintf(stderr, "error -p is no longer supported, use module imuxsock instead"); - break; - case 'q': /* add hostname if DNS resolving has failed */ - *(net.pACLAddHostnameOnFail) = 1; - break; - case 'Q': /* dont resolve hostnames in ACL to IPs */ - *(net.pACLDontResolve) = 1; - break; - case 'r': /* accept remote messages */ - fprintf(stderr, "rsyslogd: error option -r is no longer supported - ignored"); - break; - case 's': - if(glbl.GetStripDomains() != NULL) { - fprintf (stderr, "rsyslogd: Only one -s argument allowed, the first one is taken.\n"); - } else { - glbl.SetStripDomains(crunch_list(arg)); - } - break; - case 't': /* enable tcp logging */ - fprintf(stderr, "rsyslogd: error option -t is no longer supported - ignored"); - break; - case 'T':/* chroot() immediately at program startup, but only for testing, NOT security yet */ - if(chroot(arg) != 0) { - perror("chroot"); - exit(1); - } - break; - case 'u': /* misc user settings */ - iHelperUOpt = atoi(arg); - if(iHelperUOpt & 0x01) - glbl.SetParseHOSTNAMEandTAG(0); - if(iHelperUOpt & 0x02) - bChDirRoot = 0; - break; - case 'w': /* disable disallowed host warnigs */ - glbl.SetOption_DisallowWarning(0); - break; - case 'x': /* disable dns for remote messages */ - glbl.SetDisableDNS(1); - break; - case '?': - default: - usage(); - } - } - - if(iRet != RS_RET_END_OF_LINKEDLIST) - FINALIZE; - - if(iConfigVerify) { - fprintf(stderr, "rsyslogd: version %s, config validation run (level %d), master config %s\n", - VERSION, iConfigVerify, ConfFile); - } - - localRet = rsconf.Load(&ourConf, ConfFile); + /* oxpa <iippolitov@gmail.com> contribution, need to check ASL 2.0 */ queryLocalHostname(); /* need to re-query to pick up a changed hostname due to config */ - - if(localRet == RS_RET_NONFATAL_CONFIG_ERR) { - if(loadConf->globals.bAbortOnUncleanConfig) { - fprintf(stderr, "rsyslogd: $AbortOnUncleanConfig is set, and config is not clean.\n" - "Check error log for details, fix errors and restart. As a last\n" - "resort, you may want to remove $AbortOnUncleanConfig to permit a\n" - "startup with a dirty config.\n"); - exit(2); - } - if(iConfigVerify) { - /* a bit dirty, but useful... */ - exit(1); - } - localRet = RS_RET_OK; - } - CHKiRet(localRet); - - CHKiRet(ratelimitNew(&dflt_ratelimiter, "rsyslogd", "dflt")); - /* TODO: add linux-type limiting capability */ - CHKiRet(ratelimitNew(&internalMsg_ratelimiter, "rsyslogd", "internal_messages")); - ratelimitSetLinuxLike(internalMsg_ratelimiter, 5, 500); - /* TODO: make internalMsg ratelimit settings configurable */ - - if(bChDirRoot) { - if(chdir("/") != 0) - fprintf(stderr, "Can not do 'cd /' - still trying to run\n"); - } - - /* process compatibility mode settings */ - if(bEOptionWasGiven) { - errmsg.LogError(0, NO_ERRCODE, "WARNING: \"message repeated n times\" feature MUST be turned on in " - "rsyslog.conf - CURRENTLY EVERY MESSAGE WILL BE LOGGED. Visit " - "http://www.rsyslog.com/rptdmsgreduction to learn " - "more and cast your vote if you want us to keep this feature."); - } - - if(!iConfigVerify) - CHKiRet(doGlblProcessInit()); - - /* Send a signal to the parent so it can terminate. */ - if(glblGetOurPid() != ppid) - kill(ppid, SIGTERM); - - CHKiRet(init()); - - if(Debug && debugging_on) { - dbgprintf("Debugging enabled, SIGUSR1 to turn off debugging.\n"); - } - - /* END OF INTIALIZATION */ - DBGPRINTF("initialization completed, transitioning to regular run mode\n"); - - /* close stderr and stdout if they are kept open during a fork. Note that this - * may introduce subtle security issues: if we are in a jail, one may break out of - * it via these descriptors. But if I close them earlier, error messages will (once - * again) not be emitted to the user that starts the daemon. As root jail support - * is still in its infancy (and not really done), we currently accept this issue. - * rgerhards, 2009-06-29 - */ - if(doFork) { - close(1); - close(2); - ourConf->globals.bErrMsgToStderr = 0; - } - - mainloop(); - - /* do any de-init's that need to be done AFTER this comment */ - - die(bFinished); - - thrdExit(); - -finalize_it: - if(iRet == RS_RET_VALIDATION_RUN) { - fprintf(stderr, "rsyslogd: End of config validation run. Bye.\n"); - } else if(iRet != RS_RET_OK) { - fprintf(stderr, "rsyslogd: run failed with error %d (see rsyslog.h " - "or try http://www.rsyslog.com/e/%d to learn what that number means)\n", iRet, iRet*-1); - exit(1); - } - - ENDfunc - return 0; + /* end oxpa */ } - - -/* This is the main entry point into rsyslogd. This must be a function in its own - * right in order to intialize the debug system in a portable way (otherwise we would - * need to have a statement before variable definitions. - * rgerhards, 20080-01-28 - */ -int main(int argc, char **argv) -{ - dbgClassInit(); - return realMain(argc, argv); -} -/* vim:set ai: - */ |