How to setup?

[HansVanEijsden]

First of all, thank you for the Razor and Pyzor plugins.
I started with Pyzor. I installed pyzor with pip3 and I installed also pyzorsocket.
I copied pyzor.lua to /usr/share/rspamd/lua (I hope that's the right location, other plugins like for example antivirus.lua, dcc.lua and whitelist.lua are there too).
I reloaded rspamd and checked the log file. I discovered this error:

2018-04-05 13:44:51 #54881(main) <if4q5g>; cfg; rspamd_config_is_module_enabled: lua module pyzor is enabled but has not been configured
2018-04-05 13:44:51 #54881(main) <if4q5g>; cfg; rspamd_config_is_module_enabled: pyzor disabling unconfigured lua module

It's my first time working with external plugins. I searched the documentation but couldn't find any clear steps on how to install and how to configure.
Wha's the next step I have to take?

I have tried to create /etc/rspamd/local.d/pyzor.conf with enabled = true; in it, but nothing. I also tried to include pyzor in filters = "chartable,dkim,spf,surbl,regexp,fuzzy_check"; (at options.inc), but nothing.

# uname -a
Linux vps 4.14.0-0.bpo.3-amd64 #1 SMP Debian 4.14.13-1~bpo9+1 (2018-01-14) x86_64 GNU/Linux

# rspamd --version
Rspamd daemon version 1.7.2

Thanks!

[cgt]

Pyzor

I think your problem is that nothing reads your /etc/rspamd/local.d/pyzor.conf file. local.d/ conf files are supposed to be included by the corresponding conf file in modules.d/, and all modules.d/ conf files are included by /etc/rspamd/modules.conf, but of course there is no modules.d/pyzor.conf. You could make one yourself that includes your conf file, but I would suggest just include an empty pyzor config block in your /etc/rspamd/rspamd.conf.local, that's what I do. Just put pyzor { } in that file, enabled = true is unnecessary.

There are two settings for the pyzor module: host and port. You can configure them like so:

pyzor {
    host = 127.0.0.1;
    port = 5953;
}

If you just put the empty block pyzor { } and don't set host and port explicitly, it will use the default values shown above. That should take care of the rspamd configuration.

You will need to start pyzorsocket yourself, rspamd can't do this for you. I use the following systemd service:

# /etc/systemd/system/pyzorsocket.service
[Unit]
Description=pyzorsocket

[Service]
Type=simple
User=pyzorsocket
Group=pyzorsocket
ExecStart=/opt/pyzorsocket/bin/pyzorsocket 127.0.0.1 5953
StandardOutput=syslog
StandardError=syslog
Restart=always

[Install]
WantedBy=multi-user.target

You will need to change the user/group, unless you create a pyzorsocket user like I have, and the location of the pyzorsocket script (I installed it in a virtualenv under /opt/pyzorsocket/).

That should take care of Pyzor.

Razor

The Razor plugin you can either install manually or with a Debian package.

To install it manually, you'll have to install the razorsocket systemd service and the razor.lua rspamd module. The service files are razor.socket and [email protected] in rspamd-plugins/razor/debian/, which you can just put in /etc/systemd/system/ like pyzorsocket.service and do systemctl reload-daemon and enable/start them. You'll also have to put the razorsocket script somewhere and change the service user/group and script path as needed.

To configure it in rspamd (assuming you don't change the default settings for razorsocket and razor.lua), just put razor { } in your rspamd.conf.local. If you put razor.lua in /usr/share/rspamd/lua, you shouldn't need further configuration.

If you'd like, you can install it from a Debian package instead. Either build the package yourself (all the necessary metadata is in the repository) or use the one I have built: https://cgt.name/files/rspamd-razor_1_all.deb (for Debian Stretch).

The package installs the systemd service, /usr/bin/razorsocket, and /usr/share/rspamd-plugins/razor.lua. It also creates a user "razorsocket". In addition to adding the module config razor { } to your rspamd config, you will also have to add a line to your modules config in /etc/rspamd/common.conf so that rspamd knows where to find razor.lua.

modules {
    path = "$PLUGINSDIR/lua/",
    path = "/usr/share/rspamd-plugins/razor.lua",
}

Of course, this last part is not necessary if you manually install razor.lua to /usr/share/rspamd/lua/.

Hope this helps. Feel free to ask if you have more questions.

[HansVanEijsden]

Hi @cgt, wow that's a great response, thank you very much for taking the time.

I followed the instructions and installed both plugins successfully (and Razor manually). So nice to have!

2018-04-05 18:34:41 #35679(main) <75z4zz>; cfg; rspamd_init_lua_filters: init lua module pyzor
2018-04-05 18:34:41 #35679(main) <75z4zz>; cfg; rspamd_init_lua_filters: init lua module razor

Both up & running without errors.
There's only one thing: the scores of PYZOR and RAZOR are always 0.00.

Pyzor (anonymized to example.com):
2018-04-05 19:10:15 #35690(rspamd_proxy) lua; pyzor.lua:34: count=355 wl=2
[...]
2018-04-05 19:10:17 #35690(rspamd_proxy) <de0df8>; proxy; rspamd_task_write_log: id: <[email protected]>, qid: <14194840007>, ip: 42.231.160.242, from: <[email protected]>, (default: T (reject): [15.00/15.00] [URIBL_BLACK(7.50){str8creative.to.multi.uribl.com;},IP_SCORE(4.76){ip: (9.91), ipnet: 42.224.0.0/12(7.93), asn: 4837(4.96), country: CN(0.98);},RSPAMD_URIBL(4.50){str8creative.to;},MIME_BAD_ATTACHMENT(4.00){html;},RBL_SPAMHAUS_XBL(4.00){242.160.231.42.zen.spamhaus.org : 127.0.0.4;},URL_IN_SUBJECT(4.00){0.25;example.com;},FORGED_MUA_THUNDERBIRD_MSGID_UNKNOWN(2.50){},DCC_BULK(2.00){bulk Body=1 Fuz1=1 Fuz2=many;},RBL_MAILSPIKE_WORST(2.00){242.160.231.42.rep.mailspike.net : 127.0.0.10;},RBL_SENDERSCORE(2.00){242.160.231.42.bl.score.senderscore.com;},RBL_SPAMHAUS_CSS(2.00){242.160.231.42.zen.spamhaus.org : 127.0.0.3;},RBL_SPAMHAUS_PBL(2.00){242.160.231.42.zen.spamhaus.org : 127.0.0.11;},BROKEN_CONTENT_TYPE(1.50){},FAKE_REPLY(1.00){},ABUSE_SURBL(0.50){str8creative.to.multi.surbl.org;},MX_INVALID(0.50){cached;},R_SPF_ALLOW(-0.20){+ip4:42.231.160.242;},MIME_GOOD(-0.10){multipart/mixed;text/plain;},ARC_NA(0.00){},ASN(0.00){asn:4837, ipnet:42.224.0.0/12, country:CN;},CLAM_VIRUS(0.00){SecuriteInfo.com.JS.Redir.Gen-3.UNOFFICIAL;},DBL_SPAM(0.00){str8creative.to.dbl.spamhaus.org;},DMARC_NA(0.00){str8creative.to;},FROM_EQ_ENVFROM(0.00){},FROM_HAS_DN(0.00){},HAS_ATTACHMENT(0.00){},MID_RHS_MATCH_FROM(0.00){},PYZOR(0.00){count=353 wl=2;},RCPT_COUNT_ONE(0.00){1;},RCVD_COUNT_TWO(0.00){2;},RCVD_NO_TLS_LAST(0.00){},R_DKIM_NA(0.00){},TO_DN_NONE(0.00){},TO_MATCH_ENVRCPT_ALL(0.00){}]), len: 1892, time: 1676.838ms real, 8.306ms virtual, dns req: 37, digest: <6c1ec99145747a09d0bcaa90b711650e>, rcpts: <[email protected]>, mime_rcpt: <example.com>

Razor (also anonymized to example.com):
2018-04-05 22:25:10 #35690(rspamd_proxy) <0fed16>; proxy; rspamd_task_write_log: id: <[email protected]>, qid: <DDF6B3A028>, ip: 45.40.119.99, from: <[email protected]>, (default: T (reject): [23.86/15.00] [DBL_SPAM(6.50){nothingbutsew.com.dbl.spamhaus.org;},URIBL_SBL_CSS(6.00){nothingbutsew.com;},BAYES_SPAM(4.00){100.00%;},FUZZY_DENIED(3.61){1:4bb2eadd69:0.94:txt;},RBL_SPAMHAUS_CSS(2.00){99.119.40.45.zen.spamhaus.org : 127.0.0.3;},IP_SCORE(1.00){asn: 36352(4.96), country: US(0.06);},CT_EXTRA_SEMI(1.00){},HTML_SHORT_LINK_IMG_3(0.50){},DMARC_POLICY_ALLOW(-0.25){nothingbutsew.com;none;},R_DKIM_ALLOW(-0.20){nothingbutsew.com;},R_SPF_ALLOW(-0.20){+ip4:45.40.119.96/30;},MIME_GOOD(-0.10){multipart/alternative;text/plain;},MX_GOOD(-0.01){vmt1.nothingbutsew.com;},ARC_NA(0.00){},ASN(0.00){asn:36352, ipnet:45.40.112.0/20, country:US;},DKIM_TRACE(0.00){nothingbutsew.com:+;},FROM_EQ_ENVFROM(0.00){},FROM_HAS_DN(0.00){},MID_RHS_MATCH_FROM(0.00){},PRECEDENCE_BULK(0.00){},PYZOR(0.00){count=6 wl=0;},RAZOR(0.00){},RCPT_COUNT_ONE(0.00){1;},RCVD_COUNT_TWO(0.00){2;},RCVD_NO_TLS_LAST(0.00){},TO_DN_NONE(0.00){},TO_MATCH_ENVRCPT_ALL(0.00){}]), len: 13996, time: 4606.599ms real, 22.830ms virtual, dns req: 38, digest: <dc91ff450dfe1f5837566deaa6d1e345>, rcpts: <[email protected]>, mime_rcpt: <[email protected]>

Everywhere it's score/weight 0.00. In the lua plugins the symbols are already specified. Do I need to specify those symbol weights somewhere else too?

[cgt]

Try setting scores for the symbols in local.d/metrics.conf. Just add:

symbol "PYZOR" {
    weight = 2.0;
    description = "Detected as spam by pyzor";
}
symbol "RAZOR" {
    weight = 2.0;
    description = "Detected as spam by Vipul's Razor";
}

(Doesn't have to be in a group block.)

You can change the scores if you like. Keep in mind that the Pyzor score is dynamic. The weight you set will be multiplied by a score set by the script depending on the number of spam/ham reports in Pyzor. You can see the algorithm used here: https://github.com/cgt/rspamd-plugins/blob/master/pyzor/pyzor.lua#L43 I didn't put much thought into the algorithm, I just picked some numbers.

[HansVanEijsden]

Razor and Pyzor are working great now.
I took your values as starting values and I wil adjust them on all the servers during the next weeks if needed.
I will close this ticket, I'm sure it will help more people. Thanks again!

[echodreamz]

How exactly do you install the pyzorsocket?

Thanks!

[cgt]

@echodreamz You install it like any other Python program. I have it installed in a Python virtualenv in /opt/pyzorsocket.

In case you are not familiar with Python/virtualenv, the process is roughly this:

# python3 -m venv /opt/pyzorsocket
# . /opt/pyzorsocket/bin/activate

Then cd to the directory where you have downloaded the pyzorsocket code and run

# pip install wheel
# pip install .
# deactivate

Now you should have pyzorsocket installed at /opt/pyzorsocket/bin/pyzorsocket.

[echodreamz]

Thank you! I was able to figure it out, was just having a bit of a dumb moment, was a hella day that day was lol.