summaryrefslogtreecommitdiff
path: root/mail/rspamd
diff options
context:
space:
mode:
authorwiedi <wiedi@pkgsrc.org>2015-01-17 17:34:07 +0000
committerwiedi <wiedi@pkgsrc.org>2015-01-17 17:34:07 +0000
commitfacd65db9019fda22b821537be50e7ad74059db6 (patch)
tree0b0bd7a48ccd82968dbac5957930c035728deea3 /mail/rspamd
parent64763a4336378ac9854959697cfd32d348e49f7d (diff)
downloadpkgsrc-facd65db9019fda22b821537be50e7ad74059db6.tar.gz
Update to 0.8.0
0.8.0: * New fuzzy check logic: - use shingles algorithm for fuzzy matching - use blake2 instead of md5 for larger output space - combine fuzzy and strict matching - allow to organize private storages by means of keys - preserve compatibility with previous versions * New fuzzy storage: - use sqlite instead of own memory based hash tables - rework commands interface - add conversion from the old format - add fuzzy match by shignles - support old rspamd versions * Add lemmatizing for words used in fuzzy hashes that allows to improve match quality by using of the first forms of all words * Rework language detection * Fix several critical bugs, memory leaks and deadlocks: - memory leak in HTML nodes parsing - deadlock in logger code - deadlock in signals processing - crashes in fuzzy_storage - crashes in tokenizers if the input was empty * Import new libucl with several bugfixes and improvements * Support listening on ipv6 addresses only * Fix macro expansion in SPF module * Several bugfixes in DKIM module * Add load headers support for mime parts to the lua API * Add documentation for: - workers in general - fuzzy_storage worker - fuzzy_check plugin - mimepart and textpart lua API modules 0.7.6: * Apply boundary fix for dkim simple canonization * Fix ping command * Return nil if header was not found in lua_task * Fix hang in upstreams revive logic * Decode entitles when normalizing HTML parts * Fix logic of finding URLs in HTML parts * Do not include \0 into length of text when performing conversion to utf8 * Fix raw vs parsed reperesentations Raw parts are now: - decoded b64/qp, but *NOT* converted to utf-8 Processed parts are now: - converted to UTF-8 - normalized if needed (e.g. HTML tags are stripped) * Rework DKIM canonization to line based * Fix fuzzy hashes addding * Use more specific hash function for fuzzy * Fix leaking of iconv descriptors * Fix PTR resolving in lua resolver * Rework spf module. - Copy data to memory pool as cached record might be destroyed causing freed memory being passed to the protocol output (use after free) - Allow SPF_NEUTRAL policy to be handled separately - Add R_SPF_NEUTRAL to the default config * Rework `register_symbols` function * Allow to disable components of hfilter 0.7.5: * Fix owner when creating folder /run/rspamd (by @sfirmery) * Fix IP validity checks * Decode URLs obtained from HTML tags * Fix crash with unweighted upstreams * Stop processing headers in parts * Set sockaddr.sa_family properly when connectig to upstreams * Fix reload issues in surbl and fuzzy_check (reported by @citrin) * Fix timeouts in redirector * Improve lua errors reporting * Fix lua closures processing in libucl * Rework calling of lua functions from regexp module * Choose raw regexp for raw headers * Rework conversion to utf since glib one is broken * Ignore SGML style tags in html * Fix old bug with non-capturing https urls * Fix memory corruption on fuzzy reload (reported by @citrin) * Fix percents display in rspamc * Fix buffer update for DKIM * Do not validate utf for raw headers 0.7.4: * Fix build under *BSD * Detect HAN unicode script * Implement language detection heuristic for text parts * Fix time output in history * Improve piechart coloring * Fix \r\n conversion in DKIM module (reported by @citrin) * Try to detect systems with no IPv6 support * Fix multiple/single values in use settings (reported by @citrin) * Rework IP addresses in upstreams: - Select ipv4/unix addresses if they exist and use ipv6 for ipv6 only upstreams (since the support of ipv6 is poor in many OSes and environments) - Free IP list on upstream destruction - Add test cases for addresses selection - Allow adding of free form IP addresses to upstreams * Fix endiannes in lua_radix search (reported by @citrin) * Soft shutdown should also set wanna_die flag (reported by @citrin) * Stop use-after-free in event loop termination * Fix processing of very short messages in DKIM (reported by @citrin) * Detect systems without shared mutexes * Fix issues with PTR and MX elements in SPF parser (reported by @citrin) 0.7.3: * New upstreams code: - simplify upstreams API; - unify strings parsing in upstreams definition; - add configuration options for the upstreams; - for failed upstreams re-resolve their addresses; - use all resolved addresses for an upstream (round-robin); - implement stable hashing and use it by default for upstreams; - add unit test for upstreams module. * Rework signals processing in all rspamd workers: - signals are now processed in the event loop; - implement the most common signal handlers for all workers; - add callbacks for workers specific signal handlers * Fix critical issue with fuzzy storage: Fuzzy stroage could not save any hashes on termination due to bugged signals handling * Fix roll history IP storage * Rework ipv4/ipv6 handling in parsing addresses: - turn off support of IPV6_V6ONLY socket option; - create ipv6 socket prior to ipv4 one to handle systems with v6/v4 sockets enabled (Linux) * Remove CBL as it's wholly included in Spamhaus XBL (by @fatalbanana) * Remove nszones.com fake RBL (by @citrin) * Fix upstreams interaction for fuzzy_check * Verify spf PTR records (reported by @citrin) * Fix spf MX records parsing * Add compatibility for old libevent (by @yellowman) * Sync bugfixes from libucl 0.7.2: * Convert all maps to the compressed radix trie * Allow IPv6 addresses in IP maps * Remove dynamic items support from symbols cache * Allow hex encoded output of strings * Fix bug with control connections count * Process fuzzy weight correctly (reported by @fatalbanana) * Remove extra reference retain of http connection on error * Remove deprecated options from the default config * Add `one_shot` attr to metric's symbols * Doc: add documentation for metrics * Add Upstart job to debian packaging (by @CameronNemo) * Config: improve SURBL symbols descriptions (by @citrin) * Config: reflect SURBL changes (by @citrin): - Outblaze removed, malware moved to separate list: http://www.surbl.org/news/internal/MW-malware-sublist-added-to-multi * Fix C modules initialization on restart * Treat single IP as a single IP in radix lists (reported by @citrin) * Do not touch file and core limits if not asked explicitly (reported by @citrin) * Improve logging for fuzzy errors * Block SIGPIPE for HTTP writing * Doc: update manual pages * Fix HTTP connection termination * Reduce default number of parallel requests to 8 * Sync with libucl include features 0.7.1: * Fix typo in stat output. * Fix issues with includes crossing with the system includes * Restore testing framework * Add radix trie test suite * Implement new path-compressed radix trie. - The performance benefit over the old algorithm is about 1.5 times. - Memory usage is significantly lower as well. - Now radix trie can accept any IPv4/IPv6 values * Various improvements to the memory pools code * Fix writing reply to a client when no filters are defined * Write base32 encoded fuzzy * Fix 'soft reject' action * Fix rspamd reload and modules reconfiguration * Fix subject rewriting for the default subject * Fix states for processing task and pre-filters * Fix issues with connection closing * Fix crashes in rdns * Fix ratelimit pre-filter * Update exim patch. - Update to the recent exim version - Strip extra leading src/ from the patch - Remove sendfile since it was broken - Fix rspamd spam report for exim * Improve documentation 0.7.0: * Use HTTP protocol for all operatiosn * Webui worker is now removed and controller works as webui * Allow to serve static files via controller by option `static_dir` * Rspamd interface is now a part of rspamd * Rspamc client has been rewritten to use HTTP and non-blocking mode allowing to start multiple operations simultaneously (see `-n` option) * Lua API was completely reworked to satisfy modern standards of LUA: * Module `lua-message` was removed * Reduced number of superglobals registered by rspamd * Many functions has been redesigned * Symbols registration is now more convenient * Users settings has been rewritten as lua plugin * Reworked headers system as gmime's based one misses many headers and is very slow to get headers values * Reorganized code and removed embedded jannsson by using UCL for all json parsing * Migrated to `librdns` for DNS resolving that improves concurrency for DNS requests significantly * Fixed tonns of bugs in MIME processing * Improved metrcis and default symbol's weights * Added new RBL's * Fixed a number of issues in the modules * Removed several memory leaks found * Fix unicode processing * Fix fuzzy checking for unicode parts * Significantly improve documentation and especially LUA API docs * See migration notes at https://rspamd.com/doc/migration.html
Diffstat (limited to 'mail/rspamd')
-rw-r--r--mail/rspamd/Makefile12
-rw-r--r--mail/rspamd/PLIST21
-rw-r--r--mail/rspamd/distinfo11
-rw-r--r--mail/rspamd/patches/patch-src_client_rspamc.c32
-rw-r--r--mail/rspamd/patches/patch-src_lua_lua__task.c15
5 files changed, 72 insertions, 19 deletions
diff --git a/mail/rspamd/Makefile b/mail/rspamd/Makefile
index d625e7f3ddb..e05bcb58763 100644
--- a/mail/rspamd/Makefile
+++ b/mail/rspamd/Makefile
@@ -1,9 +1,9 @@
-# $NetBSD: Makefile,v 1.8 2014/10/19 22:27:46 alnsn Exp $
+# $NetBSD: Makefile,v 1.9 2015/01/17 17:34:07 wiedi Exp $
-DISTNAME= rspamd-0.6.9
-PKGREVISION= 2
+DISTNAME= rspamd-0.8.0
CATEGORIES= mail
-MASTER_SITES= https://rspamd.com/downloads/
+MASTER_SITES= http://www.rspamd.com/downloads/
+EXTRACT_SUFX= .tar.xz
MAINTAINER= wiedi@frubar.net
HOMEPAGE= https://www.rspamd.com/
@@ -12,7 +12,8 @@ LICENSE= 2-clause-bsd
USE_CMAKE= yes
USE_TOOLS+= perl pkg-config
-
+USE_LANGUAGES= c99
+CPPFLAGS.SunOS+= -D__EXTENSIONS__
LDFLAGS+= ${COMPILER_RPATH_FLAG}${PREFIX}/lib/rspamd
PKG_SYSCONFSUBDIR= rspamd
@@ -49,5 +50,6 @@ post-extract:
.include "../../devel/libevent/buildlink3.mk"
.include "../../devel/glib2/buildlink3.mk"
.include "../../mail/gmime/buildlink3.mk"
+.include "../../databases/sqlite3/buildlink3.mk"
.include "../../lang/lua/buildlink3.mk"
.include "../../mk/bsd.pkg.mk"
diff --git a/mail/rspamd/PLIST b/mail/rspamd/PLIST
index a53a9aa0a07..62f9a8d1671 100644
--- a/mail/rspamd/PLIST
+++ b/mail/rspamd/PLIST
@@ -1,28 +1,21 @@
-@comment $NetBSD: PLIST,v 1.1 2014/02/07 14:39:19 wiedi Exp $
+@comment $NetBSD: PLIST,v 1.2 2015/01/17 17:34:07 wiedi Exp $
bin/rspamc
bin/rspamc-${PKGVERSION}
bin/rspamd
bin/${PKGNAME}
-include/rspamd/librspamdclient.h
lib/rspamd/librspamd-cdb.so
lib/rspamd/librspamd-cdb.so.${PKGVERSION}
lib/rspamd/librspamd-http-parser.so
lib/rspamd/librspamd-http-parser.so.${PKGVERSION}
-lib/rspamd/librspamd-json.so
-lib/rspamd/librspamd-json.so.${PKGVERSION}
lib/rspamd/librspamd-lua.so
lib/rspamd/librspamd-lua.so.${PKGVERSION}
lib/rspamd/librspamd-mime.so
lib/rspamd/librspamd-mime.so.${PKGVERSION}
lib/rspamd/librspamd-server.so
lib/rspamd/librspamd-server.so.${PKGVERSION}
-lib/rspamd/librspamd-ucl.so
-lib/rspamd/librspamd-ucl.so.${PKGVERSION}
lib/rspamd/librspamd-util.so
lib/rspamd/librspamd-util.so.${PKGVERSION}
lib/rspamd/librspamdclient.so
-lib/rspamd/librspamdclient.so.${PKGVERSION}
-lib/rspamd/librspamdclient_static.a
man/man1/rspamc.1
man/man8/rspamd.8
share/examples/rspamd/2tld.inc
@@ -51,5 +44,15 @@ share/rspamd/lua/once_received.lua
share/rspamd/lua/phishing.lua
share/rspamd/lua/ratelimit.lua
share/rspamd/lua/rbl.lua
+share/rspamd/lua/settings.lua
share/rspamd/lua/trie.lua
-share/rspamd/lua/whitelist.lua
+share/rspamd/www/README.md
+share/rspamd/www/css/rspamd.css
+share/rspamd/www/img/asc.png
+share/rspamd/www/img/desc.png
+share/rspamd/www/img/spinner.gif
+share/rspamd/www/img/spinner.png
+share/rspamd/www/index.html
+share/rspamd/www/js/jquery.paginatetable.js
+share/rspamd/www/js/rspamd.js
+share/rspamd/www/plugins.txt
diff --git a/mail/rspamd/distinfo b/mail/rspamd/distinfo
index d434f802aab..04ae1c54256 100644
--- a/mail/rspamd/distinfo
+++ b/mail/rspamd/distinfo
@@ -1,6 +1,7 @@
-$NetBSD: distinfo,v 1.2 2014/05/14 16:07:06 wiedi Exp $
+$NetBSD: distinfo,v 1.3 2015/01/17 17:34:07 wiedi Exp $
-SHA1 (rspamd-0.6.9.tar.gz) = aed555b278509c99020624f38d2a77c38a3fa0f6
-RMD160 (rspamd-0.6.9.tar.gz) = 233d75efd3af370e3b504d9f13f7f282482a0cbe
-Size (rspamd-0.6.9.tar.gz) = 1088703 bytes
-SHA1 (patch-CMakeLists.txt) = e6d86ceea7351276937c8e6bc25d1497f3ad06da
+SHA1 (rspamd-0.8.0.tar.xz) = b3d704c0cf3ed92271f3f4f640f6406c7ec267b2
+RMD160 (rspamd-0.8.0.tar.xz) = 02b4889a3e5d3d89357be50a05bf4a6b15e2b738
+Size (rspamd-0.8.0.tar.xz) = 754636 bytes
+SHA1 (patch-src_client_rspamc.c) = cd4001655198b93fed898a01b5adbc206a25c604
+SHA1 (patch-src_lua_lua__task.c) = 88dfa4f7ee98aad5d91b4aac4a40324ea1339598
diff --git a/mail/rspamd/patches/patch-src_client_rspamc.c b/mail/rspamd/patches/patch-src_client_rspamc.c
new file mode 100644
index 00000000000..6515a555b3f
--- /dev/null
+++ b/mail/rspamd/patches/patch-src_client_rspamc.c
@@ -0,0 +1,32 @@
+$NetBSD: patch-src_client_rspamc.c,v 1.1 2015/01/17 17:34:07 wiedi Exp $
+the struct dirent d_type is not available on SunOS
+
+--- src/client/rspamc.c.orig 2014-12-22 17:57:39.000000000 +0000
++++ src/client/rspamc.c
+@@ -853,6 +853,9 @@ rspamc_process_dir (struct event_base *e
+ DIR *d;
+ gint cur_req = 0;
+ struct dirent *ent;
++#if defined(__sun)
++ struct stat sb;
++#endif
+ FILE *in;
+ char filebuf[PATH_MAX];
+
+@@ -860,9 +863,14 @@ rspamc_process_dir (struct event_base *e
+
+ if (d != NULL) {
+ while ((ent = readdir (d))) {
++ rspamd_snprintf (filebuf, sizeof (filebuf), "%s%c%s",
++ name, G_DIR_SEPARATOR, ent->d_name);
++#if defined(__sun)
++ if (stat (filebuf, &sb)) continue;
++ if (S_ISREG (sb.st_mode)) {
++#else
+ if (ent->d_type == DT_REG || ent->d_type == DT_UNKNOWN) {
+- rspamd_snprintf (filebuf, sizeof (filebuf), "%s%c%s",
+- name, G_DIR_SEPARATOR, ent->d_name);
++#endif
+ if (access (filebuf, R_OK) != -1) {
+ in = fopen (filebuf, "r");
+ if (in == NULL) {
diff --git a/mail/rspamd/patches/patch-src_lua_lua__task.c b/mail/rspamd/patches/patch-src_lua_lua__task.c
new file mode 100644
index 00000000000..a2ab23644eb
--- /dev/null
+++ b/mail/rspamd/patches/patch-src_lua_lua__task.c
@@ -0,0 +1,15 @@
+$NetBSD: patch-src_lua_lua__task.c,v 1.1 2015/01/17 17:34:07 wiedi Exp $
+
+tm_gmtoff is not a POSIX field
+--- src/lua/lua_task.c.orig 2015-01-02 11:07:18.000000000 +0000
++++ src/lua/lua_task.c
+@@ -1765,7 +1765,9 @@ lua_task_get_date (lua_State *L)
+
+ tt = tim;
+ localtime_r (&tt, &t);
++#if !defined(__sun)
+ t.tm_gmtoff = 0;
++#endif
+ t.tm_isdst = 0;
+ tim = mktime (&t);
+ }