Niall's Blog

I've used Mutt as my mail reader (aka MUA) for years. My personal mail goes through OpenBSD's greylister, spamd(8) which cuts out a very large portion of spam. However, my work email account, and also any personal account subscribed to mailing lists, still get a fair bit of spam. So some additional filtering is needed. Enter OSBF-Lua. OSBF-Lua is a port of the orthogonal sparse bigrams with confidence factor classifier from the CRM114 project. It was recommended to me around a year ago by Pedro Martelletto, a talented systems hacker who is also a very nice guy. Thanks Pedro! Anyways, OSBF-Lua is very easy to set up - assuming you already have procmail or something similar installed and hooked up to your MTA. I'm going to assume you are installing this on OpenBSD, but the instructions should not differ much for any modern UNIX system. Installing OSBF-Lua First, install the package on your machine. Any moderately recent version of OpenBSD should have a binary package for it, and its dependency, the Lua programming language:

$ sudo pkg_add -i osbf-lua
lua-5.1.4: complete
osbf-lua-2.0.4p1: complete

On OpenBSD, the important files will be installed to /usr/local/share/osbf-lua. Now follow the official OSBF-Lua install instructions, which I'm reproducing below with the OpenBSD-specific paths. Configuring OSBF-Lua to filter your mail

# Do the following steps under your account, not as root

#1) Create your local osbf-lua dir:
    mkdir $HOME/osbf-lua
# 2) Create your log and cache dirs:
    mkdir $HOME/osbf-lua/log
    mkdir $HOME/osbf-lua/cache
# Note: Old messages in the cache dir should be deleted
# regularly, typically from a cron job, to preserve disk space. 

# 3) Copy the spamfilter config file to your dir:
    cp /usr/local/share/osbf-lua/spamfilter_config.lua \
        $HOME/osbf-lua

# 4) Edit spamfilter_config.lua to set your password
    $EDITOR $HOME/osbf-lua/spamfilter_config.lua

# 5) Change to the current dir to your osbf-lua dir and 
# and create the spamfilter databases (spam.cfc, nonspam.cfc)
     cd $HOME/osbf-lua
     lua /usr/local/share/osbf-lua/create_databases.lua
# 6) Add the following lines to your .procmailrc:

# set OSBF_LUA_DIR to where spamfilter.lua, 
# spamfilter_command.lua, etc were installed

OSBF_LUA_DIR=/usr/local/share/osbf-lua
OSBF_LUA_USER_DIR=$HOME/osbf-lua

:0fw: .msgid.lock
* < 350000 # don't check messages greater than 350000 bytes
| $OSBF_LUA_DIR/spamfilter.lua --udir $OSBF_LUA_USER_DIR

Training OSBF-Lua from inside Mutt You should now have OSBF-Lua hooked up to your mail pipeline. However, it will only work with the email gateway control method. That is, OSBF-Lua will notice certain commands in the Subject header and respond to those. While nifty and occasionally useful, it is tedious to train the filter by sending emails to yourself. A much easier method is to set up a Mutt macro which invokes the filter. I have mine bound to shift-s (spam) and shift-h (ham). The additions to your $HOME/.muttrc file are trivial:

macro index,pager H "~/learn-nonspam.sh"
macro index,pager S "~/learn-spam.sh"

Now for the simple shell scripts: $HOME/learn-spam.sh

#!/bin/sh
UDIR=$HOME/osbf-lua
LEARN=/usr/local/share/osbf-lua/spamfilter.lua
$LEARN --learn=spam --udir=$UDIR

$HOME/learn-nonspam.sh

#!/bin/sh
UDIR=$HOME/osbf-lua
LEARN=/usr/local/share/osbf-lua/spamfilter.lua
$LEARN --learn=nonspam --udir=$UDIR

Now you can train the filter from inside Mutt, with a single keypress! That should save a lot of repetitive and time-consuming fiddling with the e-mail gateway method.

Niall O'Higgins is an author and software developer. He wrote the O'Reilly book MongoDB and Python. He also develops Strider Open Source Continuous Deployment and offers full-stack consulting services at FrozenRidge.co.

Tweet