<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0"
     xmlns:content="http://purl.org/rss/1.0/modules/content/"
     xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
     xmlns:atom="http://www.w3.org/2005/Atom"
     xmlns:dc="http://purl.org/dc/elements/1.1/"
     xmlns:wfw="http://wellformedweb.org/CommentAPI/"
     >
  <channel>
    <title>Niall O'Higgins</title>
    <link>http://niallohiggins.com/</link>
    <description></description>
    <pubDate>Tue, 03 Apr 2012 18:02:46 GMT</pubDate>
    <generator>Blogofile</generator>
    <sy:updatePeriod>hourly</sy:updatePeriod>
    <sy:updateFrequency>1</sy:updateFrequency>
    <item>
      <title>Using OpenBSD's OpenSMTPd for Email</title>
      <link>http://niallohiggins.com/2009/10/31/using-openbsds-opensmtpd-for-email/</link>
      <pubDate>Sat, 31 Oct 2009 13:52:00 PDT</pubDate>
      <category><![CDATA[Technical]]></category>
      <category><![CDATA[UNIX]]></category>
      <guid>http://niallohiggins.com/?p=635</guid>
      <description>Using OpenBSD's OpenSMTPd for Email</description>
      <content:encoded><![CDATA[
As many readers may be aware, the venerable <a href="http://www.freebase.com/view/en/sendmail">Sendmail</a> has been the default mail daemon in OpenBSD for years.  This is largely because it is the only reasonable BSD-licensed mail server around.  Personally, I have never trusted Sendmail enough to use it on any of my hosts - despite the fact that it has been audited by the OpenBSD team.  It has a Byzantine configuration which I could never figure out, and perhaps more importantly has a terrible security record, owing at least partly to its monolithic single-process design.  So I've always used either <a href="http://www.qmail.org/top.html">Qmail</a> or more recently <a href="http://www.postfix.org/start.html'>Postfix</a>.  Both Qmail and Postfix have a multi-process design with clean points of separation and privilege.  This approach greatly reduces the possible attack surface for privilege escalation.

<b>Qmail, Postfix</b>

<div id="fbtb-8579730708116563167" itemid="http://www.freebase.com/id/en/postfix" itemscope=""><div class="fbtb-frame"><div class="fbtb-title"><a href="http://www.freebase.com/url?c=w&id=/en/postfix&mode=i&url=http://www.freebase.com/view/en/postfix&wid=topicblocks&wurl=http://www.freebase.com/widget/topic%3Fid%3D/en/postfix%26mode%3Di%26panes%3Dimage,article_props" target="_top">Postfix</a></div><div class="fbtb-content"><div class="fbtb-pane fbtb-image-pane "><a href="http://www.freebase.com/url?c=w&id=/en/postfix&mode=i&url=http://www.freebase.com/view/en/postfix&wid=topicblocks&wurl=http://www.freebase.com/widget/topic%3Fid%3D/en/postfix%26mode%3Di%26panes%3Dimage,article_props" target="_top"><img class="fbtb-img-150" src="http://img.freebase.com/api/trans/image_thumb/en/postfix?errorid=%2Ffreebase%2Fno_image_png&maxheight=150&mode=fillcropmid&maxwidth=150" title="Postfix"></img></a></div><div class="fbtb-pane fbtb-articleprops-pane fbtb-pane-last"><div class="fbtb-description">Postfix is a free and open source mail transfer agent (MTA), a computer program for the routing and delivery of email. It is intended as...<div class="fbtb-more"><a href="http://www.freebase.com/url?c=w&id=/en/postfix&mode=i&url=http://en.wikipedia.org/wiki/index.html%3Fcurid%3D1265916&wid=topicblocks&wurl=http://www.freebase.com/widget/topic%3Fid%3D/en/postfix%26mode%3Di%26panes%3Dimage,article_props" target="_top">more &raquo;</a></div></div><div class="fbtb-properties"><div class="fbtb-property"><span class="fbtb-label">License: </span><a class="pv" href="http://www.freebase.com/url?c=w&id=/en/postfix&mode=i&url=http://www.freebase.com/view/en/ibm_public_license&wid=topicblocks&wurl=http://www.freebase.com/widget/topic%3Fid%3D/en/postfix%26mode%3Di%26panes%3Dimage,article_props" target="_top" title="IBM Public License">IBM Public License</a></div></div></div></div><div class="fbtb-get"><a href="http://www.freebase.com/url?c=w&id=/en/postfix&mode=i&url=http://www.freebase.com/topicblocks%3Fid%3D/en/postfix%26mode%3Di%26panes%3Dimage,article_props&wid=topicblocks&wurl=http://www.freebase.com/widget/topic%3Fid%3D/en/postfix%26mode%3Di%26panes%3Dimage,article_props" target="_top">Get one for any topic!</a></div></div></div><style type="text/css">#fbtb-8579730708116563167{position:relative;color:#666}#fbtb-8579730708116563167 * div, #fbtb-8579730708116563167 * img{text-align:left;vertical-align:baseline;font-family:"Helvetica Neue", Arial, Helvetica, sans-serif;font-size:11px;font-weight:normal;font-style:normal;line-height:1.3;border:0;outline:0;padding:0;margin:0}#fbtb-8579730708116563167 a{color:#17b;text-decoration:none;border:0;outline:0;padding:0;margin:0}#fbtb-8579730708116563167 a:hover{text-decoration:underline}#fbtb-8579730708116563167 .fbtb-frame{width:432px;height:265px;background:#eee;-moz-border-radius:5px;-webkit-border-radius:5px}#fbtb-8579730708116563167 .fbtb-title{padding:5px 10px;font-size:13px;font-weight:bold;line-height:1.6}#fbtb-8579730708116563167 .fbtb-content{float:left;border:1px solid #ccc;margin-left:5px;height:210px}#fbtb-8579730708116563167 .fbtb-pane{float:left;width:210px;height:210px;overflow:auto;border-right:1px solid #ccc;background-color:#e5e5e5}#fbtb-8579730708116563167 .fbtb-pane-last{border:0px}#fbtb-8579730708116563167 .fbtb-get{clear:both;padding:5px 8px;line-height:1.1}#fbtb-8579730708116563167 .fbtb-get a{color:#888;display:block;text-align:right;background:url(http://res.freebase.com/s/1f27668d55bcdfe2bbc22f25b96dce13eea4eb6a99ad92b3d3fcbe4976e70045/resources/images/freebase-widget-attribution.png) no-repeat center left}#fbtb-8579730708116563167 .fbtb-get a:hover{text-decoration:none;color:#f71;background:url(http://res.freebase.com/s/1f27668d55bcdfe2bbc22f25b96dce13eea4eb6a99ad92b3d3fcbe4976e70045/resources/images/freebase-widget-attribution-over.png) no-repeat center left}#fbtb-8579730708116563167 img{border:1px solid #fff}#fbtb-8579730708116563167 .fbtb-more{text-align:right;padding:4px 0 0 0}#fbtb-8579730708116563167 .fbtb-properties{border-top:1px dotted #ccc;clear:both}#fbtb-8579730708116563167 .fbtb-property{border-bottom:1px dotted #ccc;padding:4px 6px}#fbtb-8579730708116563167 .fbtb-label{font-size:9px;font-weight:bold;color:#444;padding-right:4px}#fbtb-8579730708116563167 .fbtb-image-pane{text-align:center}#fbtb-8579730708116563167 .fbtb-img-150{width:150px;height:150px;margin:28px 0 0 0}#fbtb-8579730708116563167 .fbtb-articleprops-pane{background:#f8f8f8}#fbtb-8579730708116563167 .fbtb-articleprops-pane .fbtb-description{padding:6px}</style><img src="http://www.freebase.com/private/beacon?c=w&id=/en/postfix&mode=i&wid=topicblocks&wurl=http://www.freebase.com/widget/topic%3Fid%3D/en/postfix%26mode%3Di%26panes%3Dimage,article_props" style="position:absolute;top:0;left:0;border:0;outline:0;padding:0;margin:0;">

Qmail has <a href="http://marc.info/?l=openbsd-ports&m=99947716330334&w=2">a very strange license</a> which prevents it even being in the OpenBSD ports system.  Postfix is not BSD-licensed, and so <a href="http://www.openbsd.org/goals.html">cannot be included in the base system</a>.  This means that running Postfix can be a little bit of extra work, since you have to deal with installing and upgrading packages.

Wouldn't it be nice if there was a modern, simple, secure SMTP daemon in base?  Now there is.  New in <a href="http://www.openbsd.org/46.html">OpenBSD 4.6</a> is the latest secure SMTP daemon on the block, <a href="http://www.openbsd.org/cgi-bin/man.cgi?query=smtpd&apropos=0&sektion=0&manpath=OpenBSD+Current&arch=i386&format=html">OpenSMTPd</a>. 

<b>Turning on OpenSMTPd</b>

Sendmail is still the default MTA in base.  You must follow these instructions to enable OpenSMTPd on your system:

<blockquote>
smtpd is not enabled by default.  In order to use it as the system mailer, ensure the mail queue is empty, then stop sendmail(8):

           # pkill sendmail

     Modify the current mailwrapper(8) settings by editing /etc/mailer.conf:

           sendmail        /usr/sbin/smtpctl
           send-mail       /usr/sbin/smtpctl
           mailq           /usr/sbin/smtpctl
           makemap         /usr/libexec/smtpd/makemap
           newaliases      /usr/libexec/smtpd/makemap

     Rebuild the aliases database, and enable the daemon:

           # newaliases
           # echo "sendmail_flags=NO" >> /etc/rc.conf.local
           # echo "smtpd_flags=" >> /etc/rc.conf.local
           # smtpd
</blockquote>

Note that while debugging your setup, you might find running smtpd in verbose foreground mode via `smtpd -dv' useful.

<b>OpenSMTPd configuration</b>

I'm very impressed at how simple and clean the OpenSMTPd configuration is.  Check out the docs <a href="http://www.openbsd.org/cgi-bin/man.cgi?query=smtpd.conf&apropos=0&sektion=0&manpath=OpenBSD+Current&arch=i386&format=html">here</a>.  There are some more docs and example configs at <a href="https://calomel.org/opensmtpd.html">Calomel.org</a>. 

It still took me a little while to figure out a few things, so I thought I'd post my configurations to help others.

<b>Using OpenSMTPd as a Backup MX</b>

I've been using Postfix as a backup MX for unworkable.org.  I decided to try OpenSMTPd in this role instead.  

<pre lang="python">
listen on lo0
listen on bnx0

map "aliases" { source db "/etc/mail/aliases.db" }

accept from all for local deliver to mbox
accept for all relay

accept from all for domain "unworkable.org" relay
</pre>

The configuration is pretty straight forward once you are aware that the default 'from' is 'local' - that is why its necessary to add `accept from all' to accept mail from the outside world.

<b>Relaying mail to another SMTP server for delivery (nullmailer) with SSL</b>

I use <a href="http://www.mutt.org">Mutt</a> as my MUA.  Mutt assumes you have a local MTA to deliver mail.  This means you need to use something like nullmailer or msmtp.  Until now.  My ISP (sonic.net) doesn't let me use port 25, so I have to relay to their SMTP server to send mail,

<pre lang="python">
listen on lo0

map aliases { source db "/etc/mail/aliases.db" }
map secrets { source db "/etc/mail/secrets.db" }

accept for local deliver to maildir
accept for all relay via smtp.sonic.net ssl enable auth
</pre>

The /etc/mail/secrets.db file is generated from a map, /etc/mail/secrets. This file includes your username and password - check out the <a href="http://www.openbsd.org/cgi-bin/man.cgi?query=smtpd.conf&apropos=0&sektion=0&manpath=OpenBSD+Current&arch=i386&format=html">smtpd.conf manual page</a> for details.
]]></content:encoded>
    </item>
    <item>
      <title>Read a file line by line in C - secure fgets idiom</title>
      <link>http://niallohiggins.com/2009/10/03/read-a-file-line-by-line-in-c-secure-fgets-idiom/</link>
      <pubDate>Sat, 03 Oct 2009 14:57:40 PDT</pubDate>
      <category><![CDATA[Technical]]></category>
      <category><![CDATA[C]]></category>
      <category><![CDATA[UNIX]]></category>
      <guid>http://niallohiggins.com/?p=590</guid>
      <description>Read a file line by line in C - secure fgets idiom</description>
      <content:encoded><![CDATA[
A pretty common thing to do in any program is read a file line-by-line.  In other interpreted or managed languages this is trivial, the standard libraries will make it super easy for you.  Just look at how simple it is to do this in Python or Perl or even Shell.

In C its a little more complicated, because you have to think about how much memory you need up front, and also the standard library is kind of crufty.  You always have to worry about overflowing buffers or dereferencing a NULL pointer.

<a href="http://niallohiggins.com/wp-content/uploads/2009/10/buffer-overflow.jpg"><img src="http://niallohiggins.com/wp-content/uploads/2009/10/buffer-overflow.jpg" alt="buffer-overflow" title="buffer-overflow" width="155" height="160" class="aligncenter size-full wp-image-601" /></a>

However, there is a nice libc function available in BSD-derived platforms (including Mac OS X) - <a href="http://www.openbsd.org/cgi-bin/man.cgi?query=fgetln&apropos=0&sektion=0&manpath=OpenBSD+Current&arch=i386&format=html">fgetln(3)</a>.  This function makes it nice and easy to read arbitrary-length lines from a file in a safe way.  Unfortunately, its not available in GNU libc - that is to say, if you use this function, your program won't compile on Linux.  Its not a trivial libc function to port - unlike say strlcpy - since it relies on private details of the FILE structure.  These private details don't happen to be the same in glibc, so it doesn't work out of the box.

While GNU libc doesn't provide fgetln(3), it does provide its own similar function, <a href="http://www.gnu.org/s/libc/manual/html_node/Line-Input.html">getline(3)</a>.  Of course, if you use this function - which is a bit uglier than fgetln(3) in my opinion - your program won't work on BSD libc systems.

So basically, neither of these functions are usable if you want your program to be reasonably portable.  Pretty much everything I write in C I want to work on at least Linux, the BSDs, and Mac OS X.

You could write your own line reading function on top of ANSI C.  Or you might be able to get away with using the existing ANSI function, <a href="http://www.openbsd.org/cgi-bin/man.cgi?query=fgets&apropos=0&sektion=0&manpath=OpenBSD+Current&arch=i386&format=html">fgets(3)</a>.  You need to be careful with fgets, however.  You can easily introduce bugs if you aren't careful to cover all the error cases.

The other big problem with fgets is that you need to know the maximum length of lines you are going to read in advance, otherwise you'll end up with truncation.  In most applications, you can get away with a kilobyte or two on the stack for each line and be ok. In some places, it could be a deal killer though.

Anyway, here is a good idiom for using fgets:

<pre lang="c">
char buf[MAXLINELEN];
while (fgets(buf, MAXLINELEN, ifp) != NULL) {
    buf[strcspn(buf, "\n")] = '\0';
    if (buf[0] == '\0')
        continue;
}
</pre>

The explanation of why you use strcspn() can be found in the <a href="http://www.openbsd.org/cgi-bin/man.cgi?query=fgets&apropos=0&sektion=0&manpath=OpenBSD+Current&arch=i386&format=html">OpenBSD manual page</a>.
]]></content:encoded>
    </item>
    <item>
      <title>Py Web SF: The San Francisco Python & Web Technology Meet-up</title>
      <link>http://niallohiggins.com/2009/07/24/py-web-sf-the-san-francisco-python-web-technology-meet-up/</link>
      <pubDate>Fri, 24 Jul 2009 19:29:47 PDT</pubDate>
      <category><![CDATA[Technical]]></category>
      <category><![CDATA[Python]]></category>
      <category><![CDATA[UNIX]]></category>
      <category><![CDATA[JavaScript]]></category>
      <category><![CDATA[Windows]]></category>
      <guid>http://niallohiggins.com/?p=563</guid>
      <description>Py Web SF: The San Francisco Python & Web Technology Meet-up</description>
      <content:encoded><![CDATA[
Last month I started <a href="http://pywebsf.org">Py Web SF</a>, the San Francisco Python & Web Technology meet-up.  The idea is 1-2 conversation-style presentations of about 30 minutes with a group of 10-20 people.  My hope is to have a more intimate group than the very good <a href="http://www.baypiggies.net/">Bay Piggies</a> (which I highly recommend).  With a small group, it is possible to have more interaction, discussion and collaboration.  In a typical lecture/audience format, people unfortunately tend to switch into "passive listener" mode.

<p>
<a href="http://pywebsf.org"><img src="http://niallohiggins.com/wp-content/uploads/2009/07/pywebsf.png" alt="pywebsf" title="pywebsf" width="321" height="156" class="aligncenter size-full wp-image-566" /></a>
</p>

<p>
<strong>June meet-up</strong>
<br/>

Anyway, the first meet-up went extremely well - we had 15 people show up, which was a perfect number for the space.  <a href="http://jjinux.blogspot.com/">Shannon -jj Behrens</a> gave an excellent talk on <A href="http://www.pywebsf.org/2009/06/24/shannon-jj-behrens-techniques-for-building-third-party-restful-web-services/">building RESTful Web services</a> and <a href="http://monkey.org/~marius">Marius Eriksen</a> - in fact a colleague from the <a href="http://www.openbsd.org">OpenBSD project</a> - gave an <a href="http://www.pywebsf.org/2009/06/24/marius-a-eriksen-geodjango-a-world-class-geographic-web-framework/">awesome talk on GeoDjango</a>.  Slides for both talks are online, of course.
</p>

<p>
<strong>July meet-up</strong>
<br/>

<a href="http://www.pywebsf.org/2009/07/21/py-web-sf-2-july-28th-6pm-sf-main-public-library%e2%80%99s-stong-room/'>This month's meet</a> should be equally good.  We have two great talks lined up - Aseem Mohanty, a colleague of mine from <a href="http://www.metaweb.com">Metaweb Technologies</a> presenting a comparison of Django and Pylons.  Then we have Alec Flett, another Metaweb'er, speaking about all the issues involved in scaling Python web applications.
</p>

<p>
<strong>Check it out</strong>
<br/>

If you are interested in checking out the event, its <b>July 28th, 6pm @ SF Main Public Libraryâ€™s Stong Room</b>.  Full details can be found <a href=" http://www.pywebsf.org/2009/07/21/py-web-sf-2-july-28th-6pm-sf-main-public-library%E2%80%99s-stong-room/">at pywebsf.org</a>. Or if you are interested in giving a talk, <a href="mailto:niallo@pywebsf.org">just let me know</a>.
</p>
]]></content:encoded>
    </item>
    <item>
      <title>tmux, a BSD alternative to GNU Screen</title>
      <link>http://niallohiggins.com/2009/06/04/tmux-a-bsd-alternative-to-gnu-screen/</link>
      <pubDate>Thu, 04 Jun 2009 20:25:13 PDT</pubDate>
      <category><![CDATA[Technical]]></category>
      <category><![CDATA[C]]></category>
      <category><![CDATA[UNIX]]></category>
      <guid>http://niallohiggins.com/?p=543</guid>
      <description>tmux, a BSD alternative to GNU Screen</description>
      <content:encoded><![CDATA[
<img src="http://tmux.sourceforge.net/tmux1.png" width="320" height="200" />

<p>
I started using <a href="http://tmux.sourceforge.net/">tmux</a> today.  It's a terminal multiplexer / task switcher for UNIX-likes, very much in the same vein as  <a href="http://en.wikipedia.org/wiki/GNU_Screen">GNU Screen</a>.  However, it's a from-scratch implementation, designed to be clean, sane and easy to configure.  The more liberal 3-clause BSD license is a plus also, since it means that <a href="http://www.openbsd.org">OpenBSD</a> has been able to <A href="http://archives.neohapsis.com/archives/openbsd/cvs/2009-06/0050.html">integrate it into the source tree</a>, so that it's available out of the box.
</p>

<p>
<b>Comparison with GNU Screen</b>
<br/>
I've been a heavy screen user for many years - almost all my work is done on remote screen sessions.  However, screen configuration has always been essentially black magic to me.  For this reason, tmux and its nice manual page is a breath of fresh air.  `tmux list-commands' is very straight forward and easy to grok.  Furthermore, I like that everything in tmux is scriptable from the command line - you can run commands like `tmux resize-pane-up -t comms' to resize the pane on a session called 'comms'.
</p>

<p>
The other thing I really like about tmux is its default status bar.  Some people might hate this, but I find it very useful to have a clock and a list of windows along with the process executing in them.  This took quite some work to set up to my liking in GNU screen, but the default in tmux is great.
</p>

<p>
<b>My config</b>
<br/>
One thing I don't much like is the default of C-b as the 'prefix' command.  I suppose this makes some sense, since the author doesn't want to clobber GNU screen key bindings.  Perhaps he will consider changing it to C-a, like in GNU screen, in the future.  In any case, this isn't hard to change.  Also, I am constantly using C-a C-a to switch back to the previous window - the default for this action in tmux is C-b l.  Much less friendly in my opinion - of course, it's also easy to change!
</p>

<p>
So here are the contents of my $HOME/.tmux.conf:
</p>

<pre lang="bash">
set -g prefix C-a
bind-key C-a last-window
</pre>

<p>
<b>Getting tmux</b>
<br/>

I'm sure that packages exist for most operating systems.  You can grab the source from <a href="http://tmux.sourceforge.net/">http://tmux.sourceforge.net/</a>.  On OpenBSD, you can simply run `pkg_add -i tmux' to get the binary on your system.
</p>

<p>
<b>UPDATE</b>
<br/>
Since OpenBSD 4.6, tmux is part of the base system.  This means that if you are running OpenBSD 4.6 or later, you don't need to install any packages in order to get tmux.
</p>
]]></content:encoded>
    </item>
    <item>
      <title>Easy private DNS - authoritative and recursive - with Unbound</title>
      <link>http://niallohiggins.com/2009/05/19/easy-private-dns-authoritative-and-recursive-with-unbound/</link>
      <pubDate>Tue, 19 May 2009 18:30:32 PDT</pubDate>
      <category><![CDATA[Technical]]></category>
      <category><![CDATA[UNIX]]></category>
      <category><![CDATA[Bicycle]]></category>
      <guid>http://niallohiggins.com/?p=484</guid>
      <description>Easy private DNS - authoritative and recursive - with Unbound</description>
      <content:encoded><![CDATA[
Lots of people have a small home network.  Usually you have a combo box which acts as a router/firewall/file server.  Then you have a couple of other machines hooked up, and you share the Internet using NAT.  A private DNS server is helpful in this kind of scenario for two reasons:
<ul>
	<li>Recursive resolver cache can speed up common DNS lookups.</li>
	<li>Private authoritative resolver lets you easily refer to machine in your home by name, instead of remembering IP addresses.</li>
</ul>

<b>The DNS Dichotomy</b>

For many years there has been a dichotomy in the DNS server implementation world, pretty much between the <a href="http://en.wikipedia.org/wiki/BIND">ISC's BIND</a> and just about everything else.  The essence of this dichotomy is that BIND integrates both <a href="http://en.wikipedia.org/wiki/Domain_Name_System'>recursive and authoritative</a> resolvers into a single, monolithic daemon process, while practically everything else splits these two functions explicitly.  For example, the well-known alternative DNS package <a href="http://en.wikipedia.org/wiki/Djbdns">djbdns</a>, has one tool - tinydns - for the authoritative portion while another - dnscache - implements the caching recursive resolver functionality.

<b>Convenience costs you security</b>

The monolithic BIND approach has certain limited benefits - mainly that it is convenient to configure and install a private DNS server which acts both as a cache and as an authority for the private domain.  Unfortunately, this design has severe implications for the robustness of the software.  It serves both to increase complexity within a single process while ignoring the <a href="http://en.wikipedia.org/wiki/Principle_of_least_privilege">principle of least privilege</a>.  Essentially, BIND is a horribly complicated beast, with <a href="http://oldwww.isc.org/index.pl?/sw/bind/bind-security.php#matrix">serious security vulnerabilities being found pretty often</a> - and even the smallest security flaw can result in major problems due to the single process design.

<b>Alternative approaches</b>

[caption id="attachment_487" align="aligncenter" width="250" caption="Unbound: A modern, secure DNS server"]<a href="http://niallohiggins.com/wp-content/uploads/2009/05/unbound-250.png"><img src="http://niallohiggins.com/wp-content/uploads/2009/05/unbound-250.png" alt="Unbound" title="unbound" width="250" height="125" class="size-full wp-image-487" /></a>[/caption]

While djbdns might be one of the better-known BIND alternatives, I recently came across <a href="http://www.unbound.net/">Unbound</a>, a BSD licensed recursive resolver.  One of the authors of Unbound is also an <a href="http://www.openbsd.org">OpenBSD</a> developer, which inspires confidence in the security of the software.

<b>Unbound also does simple authoritative resolution</b>

One of the nifty features of Unbound is that you can very simply configure it to act as an authority for your private domains.  Due to this feature, you can have a single daemon on your home network router acting as both a cache and server for your local domain.  This is very nice.  In fact, I have found the Unbound configuration format to be considerably nicer to deal with than that of BIND.

<b>Setup under OpenBSD</b>

This describes how I set up Unbound on my OpenBSD machine - it should be a pretty similar procedure on most other operating systems.

<pre lang="bash">
# install the package
$ sudo pkg_add -i unbound
</pre>

Now you have the binaries on disk, you can edit the configuration to set up your private domain.  Unbound runs as a recursive resolver out of the box, so this is just about all the configuration you'll need to do.

<pre lang="bash">
# edit the config
$ sudo vi /var/unbound/etc/unbound.conf
</pre>

For a single machine, add the following under 'server', replacing 'inet' with the desired name of your local domain, and 'joust' with the name of your machine:

<pre lang="config">
    local-zone: "inet." static
    local-data: "joust.inet. IN A 192.168.1.1"
</pre>

Since you want the DNS server to be accessible from other machines, you probably want it to listen on 0.0.0.0 (all available interfaces).  Make sure you have some kind of firewall in place before you do this, though - you don't want to let random Internet hosts query your DNS server:

<pre lang="config">
    interface: 0.0.0.0
    # Make sure you have a packet filter to block queries from the Internet.
    # Alternatively, set this only for your local network.
    access-control: 0.0.0.0/0 allow
</pre>

Now you can start up Unbound:

<pre lang="bash">
$ sudo /usr/local/sbin/unbound
</pre>

And of course you probably want it to come up on boot, so follow these instructions:

<pre lang="bash">
$ pkg_info -D unbound
Information for inst:unbound-1.2.1p0

Install notice:
You should add:

    syslogd_flags="${syslogd_flags} -a /var/unbound/dev/log"

to /etc/rc.conf.local to create a syslog socket in the unbound chroot.

You may also want to add the following to /etc/rc.local to start unbound
at boot:

        if [ -x /usr/local/sbin/unbound ]; then
                echo -n ' unbound'; /usr/local/sbin/unbound 
        fi
</pre>]]></content:encoded>
    </item>
    <item>
      <title>Mount remote filesystems via SSH on Windows with free software</title>
      <link>http://niallohiggins.com/2009/05/12/mount-remote-filesystems-via-ssh-on-windows-with-free-software/</link>
      <pubDate>Tue, 12 May 2009 20:06:44 PDT</pubDate>
      <category><![CDATA[Technical]]></category>
      <category><![CDATA[Windows]]></category>
      <category><![CDATA[UNIX]]></category>
      <guid>http://niallohiggins.com/?p=444</guid>
      <description>Mount remote filesystems via SSH on Windows with free software</description>
      <content:encoded><![CDATA[
I often use Windows as a terminal to my various UNIX systems.  Sometimes its helpful to run proprietary software - and I don't have time/inclination to mess around with half-baked emulators/ports/binary blobs/whatevers under Linux.  I either run a completely open system like <a href="http://www.openbsd.org">OpenBSD</a> or I run Windows.

<a href="http://dokan-dev.net/en/"><img src="http://dokan-dev.net/en/wp-content/uploads/2007/11/dokan.png" /></a>

Anyway, I never use Windows to do any real work.  I always shell into a remote system to actually get things done.  Either <a href="http://www.chiark.greenend.org.uk/~sgtatham/putty/">PuTTY</a> or - if you prefer real <a href="http://www.openssh.com">OpenSSH</a> like me - <a href="http://www.cygwin.com/">OpenSSH via Cygwin/X</a> work fine for getting a terminal.  <a href="http://winscp.net/eng/index.php">WinSCP</a> or Cygwin's OpenSSH for scp(1) are good for file-transfer under most circumstances.  However, one of the nice things about Windows is the Explorer shell.  It - and its <a href="http://www.kde.org">KDE knock-off</a> - are useful for certain file management operations.  Why not leverage it?

So I started looking for a way to mount remote filesystems via SSH, so that they appear as native Windows volumes.  And I found a way to do it, for free.

<b>Dokan user mode filesystem for Windows</b>

<a href="http://dokan-dev.net/en/">Dokan</a> is basically <a href="http://fuse.sourceforge.net/">FUSE</a> for Windows.  That's all dandy and there are <a href="http://dustfs.zekjur.net/">plenty of useful FUSE filesystems out there, like this one which uses my BitTorrent implementation</a>.  Whats cool about Dokan is they also do an <a ref="http://dokan-dev.net/en/docs/dokan-sshfs-readme/">SSH FS implementation</a>.

<a href="http://dustfs.zekjur.net/"><img src="http://niallohiggins.com/wp-content/uploads/2009/05/logo3.png" /></a>

<b>Is it hard to set up</b>

I figured this thing was surely going to be a PITA and probably not work to boot.  In fact, you just install three things - some Microsoft runtime library, the main Dokan library, and Dokan SSHFS - and there you go.  There is simple GUI app to set up remote mounts that supports all the things you'd expect, saving sessions, both password and public key authentication.

<b>Does it actually work</b>

Yes, although it doesn't seem to support symlinks.  A symlink to a directory on a remote system appears as a file under Dokan.  So no $HOME/public_html for you - oh well.

<b>Final thoughts</b>

Its fun to look at your horribly un-organised UNIX home directory in Windows, and see just how messy it is.  Almost makes me want to start cleaning things up.  But then I remember I know how to use locate(1) and find(1).]]></content:encoded>
    </item>
    <item>
      <title>Importing a CVS repository to Google Code Subversion</title>
      <link>http://niallohiggins.com/2009/05/06/importing-a-cvs-repository-to-google-code-subversion/</link>
      <pubDate>Wed, 06 May 2009 21:11:17 PDT</pubDate>
      <category><![CDATA[Technical]]></category>
      <category><![CDATA[C]]></category>
      <category><![CDATA[UNIX]]></category>
      <category><![CDATA[BitTorrent]]></category>
      <guid>http://niallohiggins.com/?p=407</guid>
      <description>Importing a CVS repository to Google Code Subversion</description>
      <content:encoded><![CDATA[
My <a href="http://p2presearch.com/unworkable/">C BitTorrent implementation</a>, Unworkable, used to be hosted on an anonymous CVS repository I had running on my server at home.  This was fine, until I reinstalled the machine from scratch and didn't feel like setting up the whole anonymous CVS access again.  Its a pretty painful process, unfortunately, although there is this <a href="http://www.openbsd.org/anoncvs.shar">guide to setting up anonymous CVS</a>.

<b>No public VCS, bad for an Open Source project</b>

So, for a while, there was no public source control for Unworkable, which sucked.  Its difficult and cumbersome for other developers to write diffs and track changes without one.  While I generally like to maintain full control of the source code hosting, I've had good experiences with <a href="http://code.google.com">Google Code</a> before.  They don't show ads, and their interface is clean and to the point, unlike say SourceForge, which is covered in ads and various nonsense.

<img src="http://www.gstatic.com/codesite/ph/images/code_sm.png" />

Anyway, I had a CVS repository and I first wanted to convert it to Subversion, including all the history, then I wanted to import that into Google Code.

<b>Converting from CVS to Subversion</b>

The first thing to do was to convert my existing CVS repository to Subversion.  There is a nice tool specifically for this, <a href="http://cvs2svn.tigris.org/">cvs2svn</a>.  It is in fact very easy to use, at least in my basic case - I only work with HEAD or in SVN terminology, trunk.  I simply ran:

<pre lang="bash">
cvs2svn --trunk-only --svnrepos ~/unworkable-svn ~/unworkable-cvs
</pre>

Et voila, I have a shiny new Subversion repository in ~/unworkable-svn, with full history.

<b>Importing Subversion repository to Google Code</b>

Google Code lets you import an existing Subversion repository pretty easily, as long as you have an empty project.  When your Google Code project is created, it will be set to revision 1.  In Subversion-land, revision 0 is sort of magic, and so you will need to overwrite it to properly import your existing repository.  Google give you a place to do this, but its slightly confusing because they don't put it under 'Administer'.

In order to <b>reset your repository</b> you must:

<ul>
<li>Log into your Google Code project with administrator privileges.</li>
<li>Browse to the 'Source' page (either 'Checkout' or 'Browse' but not 'Changes').</li>
<li>Scroll to the bottom of the page, and click the 'reset this repository' link, which is sort of hidden.</li>
<li>Choose the option "Did you just start this project and do you want to 'svnsync' content from an existing repository into this project?"</li>
<li>Click the big "Reset Repository" button which has a big red warning label beside it.</li>
</ul>

Now you are ready to import your repository.  You will use the 'svnsync' tool included in Subversion 1.4 and up to do this.  There are two commands, one which takes a path to both the Google Code repository and your repository, and another which takes just a path to the Google Code repository.  The first one will run quite quickly, the second one can take a while as it imports each individual revision,

<pre lang="bash">
# This command takes both the path to your Google Code repository
# and the path to the repository you want to import
svnsync init --username YOURUSERNAME 
    https://YOURPROJECT.googlecode.com/svn file:///path/to/localrepos
# This command takes just the path to the Google Code repository.
# It will take a while to complete.
svnsync sync --username YOURUSERNAME https://YOURPROJECT.googlecode.com/svn
</pre>

Once you've done that, your code is imported.  Enjoy!]]></content:encoded>
    </item>
    <item>
      <title>OpenBSD 4.5 is out, solid release, but some package bugs</title>
      <link>http://niallohiggins.com/2009/05/04/openbsd-45-is-out-some-package-bugs/</link>
      <pubDate>Mon, 04 May 2009 23:32:54 PDT</pubDate>
      <category><![CDATA[Technical]]></category>
      <category><![CDATA[UNIX]]></category>
      <guid>http://niallohiggins.com/?p=392</guid>
      <description>OpenBSD 4.5 is out, solid release, but some package bugs</description>
      <content:encoded><![CDATA[
<a href="http://www.openbsd.org/45.html">OpenBSD 4.5</a> was released the other day.  I upgraded one of my servers and workstations to the new release, from 4.4-current and 4.4-release respectively.  Mostly, things have gone pretty smoothly, as is usually the case with OpenBSD.  The new release has plenty of incremental improvements, with the developers gradually polishing and refining things such that the operating system as a whole gets better and better.

<img src="http://www.openbsd.org/images/Pufftron.jpg" />

Some of the highlights for me include the new <a href="http://undeadly.org/cgi?action=article&sid=20081120152718">multi-plexing, re-sampling, low-latency audio server aucat(1)</a>, the <a href="http://undeadly.org/cgi?action=article&sid=20081123034225">new even-stricter malloc(3)</a> which can catch buffer overflows of even a single byte.  Additionally, the Atheros wireless driver, <a href="http://www.openbsd.org/cgi-bin/man.cgi?query=ath&sektion=4">ath(4)</a>, which I have in a couple of my laptops, now supports <a href="http://undeadly.org/cgi?action=article&sid=20081115170501">WPA</a>.

All in all, lots of nice improvements which make OpenBSD even more of a pleasure to use.  I did however come across one nasty bug after upgrading one of my servers.  We use the <a href="http://www.xs4all.nl/~wpd/symon/">symon system monitor</a> on all our servers to log and graph all the sorts of system and network metrics you'd expect - cpu usage, disk, memory, network io, etc.  This is very useful for capacity planning and also for spotting bugs or mis-configurations.  Especially on a shared, multi-user system you want to keep an eye on resource utilisation over time.  Unfortunately, the 4.5-release symon package for amd64 would exit after startup.  When I ran it in debug mode, it gave me a 'Bus error' - then when I twiddled the symon.conf a bit more to remove some sensors, it would exit with a segfault, and finally I could get it to run by disabling some stuff but it would spew warning messages to standard IO.  Specifically, it had a problem with the <b>if(lo0)</b> and <b>if(bnx)</b> sensors.  If I disabled those, it would run, but spit out warnings.  However, these sensors were pretty useful to me, so not having them was very annoying.

I noticed some recent commits to the -current symon port, so I decided to give that a shot.  Its always pretty hairy building -current ports on -release, but in this case I didn't have much choice.  Fortunately for me, the -current symon port built and ran fine, completely eliminating the issues I'd had with the 4.5 -release packages.  So, this was a mild annoyance, although it does highlight the sad demise of the -stable ports tree.

Overall, 4.5 is a solid release with plenty of new features, just be warned that the symon package might not work out of the box for you, if you rely on it.]]></content:encoded>
    </item>
    <item>
      <title>Apache mod_rewrite RewriteRule with query string</title>
      <link>http://niallohiggins.com/2009/05/01/apache-mod_rewrite-rewriterule-with-query-string/</link>
      <pubDate>Fri, 01 May 2009 21:35:52 PDT</pubDate>
      <category><![CDATA[Technical]]></category>
      <category><![CDATA[UNIX]]></category>
      <guid>http://niallohiggins.com/?p=387</guid>
      <description>Apache mod_rewrite RewriteRule with query string</description>
      <content:encoded><![CDATA[
I was converting some mod_rewrite rules from the <a href="http://lighttpd.net">Lighttpd</a> webserver to <a href="http://httpd.apache.org">Apache</a> today.

<img src="http://upload.wikimedia.org/wikipedia/commons/thumb/c/cd/ASF-logo.svg/160px-ASF-logo.svg.png" />

While Lighttpd and Apache both have request rewriting modules with pretty equivalent functionality, there are some significant differences nonetheless.  Specifically, I was trying to rewrite a URL of the form:

<pre>
/script?key=123abcxyz
</pre>

to a file on the local disk:

<pre>
/abc/123/123abcxyz
</pre>

In Lighttpd, I had a single rewrite rule handling both the URL and its query string:

<pre>
"^/script?key=(([0-9a-f]{3})([0-9a-f]{3}).*)" => "/$2/$3/$1"
</pre>

The regular expression might be a little confusing at first, but its reasonably straight forward.

My first thought was that I could do pretty much exactly the same thing with an Apache <a href="http://httpd.apache.org/docs/2.2/mod/mod_rewrite.html#rewriterule">RewriteRule</a>, like so:

<pre>
RewriteRule ^/script?key=(([0-9a-f]{3})([0-9a-f]{3}).*) /$2/$3/$1
</pre>

Unfortunately this won't work - the RewriteRule will only be passed the URL - that is, /script without the query string (?key=foo).

So how do you make the RewriteRule aware of the value of the query string to rewrite to the local on-disk file correctly?  You must have a <a href="http://httpd.apache.org/docs/2.2/mod/mod_rewrite.html#rewritecond">RewriteCond</a> directive preceeding the RewriteRule.  RewriteCond can run a grouping regular expression over the query string, and RewriteRule can pull those groups out of the previous RewriteCond with a special syntax:

<pre>
RewriteCond %{QUERY_STRING} key=(([0-9a-f]{3})([0-9a-f]{3}).*)
RewriteRule ^/script /%2/%3/%1
</pre>

So there you are - how to have an Apache RewriteRule operate on parts of the query string as well as on the URL.  The solution ends up being a little more convoluted under Apache than under Lighttpd, but is still manageable.]]></content:encoded>
    </item>
    <item>
      <title>Good spam filtering with OSBF-Lua and Mutt</title>
      <link>http://niallohiggins.com/2009/01/17/good-spam-filtering-with-osbf-lua-and-mutt/</link>
      <pubDate>Sat, 17 Jan 2009 00:11:24 PST</pubDate>
      <category><![CDATA[Technical]]></category>
      <category><![CDATA[UNIX]]></category>
      <guid>http://niallohiggins.com/?p=290</guid>
      <description>Good spam filtering with OSBF-Lua and Mutt</description>
      <content:encoded><![CDATA[
I've used <a href="http://www.mutt.org">Mutt</a> as my mail reader (aka MUA) for years.  My personal mail goes through OpenBSD's greylister, <a href="http://www.openbsd.org/spamd/">spamd(8)</a> which cuts out a very large portion of spam.  However, my work email account, and also any personal account subscribed to mailing lists, still get a fair bit of spam.  So some additional filtering is needed.

Enter <a href="http://osbf-lua.luaforge.net/">OSBF-Lua</a>.  OSBF-Lua is a port of the orthogonal sparse bigrams with confidence factor classifier from the <a href="http://crm114.sourceforge.net/">CRM114</a> project.  It was recommended to me around a year ago by <a href="http://ambientworks.net/~pedro/">Pedro Martelletto</a>, a talented systems hacker who is also a very nice guy.  Thanks Pedro!

Anyways, OSBF-Lua is very easy to set up - assuming you already have <a href="http://www.procmail.org/">procmail</a> or something similar installed and hooked up to your MTA.  I'm going to assume you are installing this on <a href="http://www.openbsd.org">OpenBSD</a>, but the instructions should not differ much for any modern UNIX system.

<b>Installing OSBF-Lua</b>

First, install the package on your machine.  Any moderately recent version of OpenBSD should have a binary package for it, and its dependency, the <a href="http://www.lua.org">Lua</a> programming language:
<pre lang="bash" line="1">
$ sudo pkg_add -i osbf-lua
lua-5.1.4: complete
osbf-lua-2.0.4p1: complete
</pre>

On OpenBSD, the important files will be installed to <b>/usr/local/share/osbf-lua</b>.  Now follow the <a href="http://osbf-lua.luaforge.net/#installation">official OSBF-Lua install instructions</a>, which I'm reproducing below with the OpenBSD-specific paths.

<b>Configuring OSBF-Lua to filter your mail</b>
 
<pre lang="bash" line="0">
# Do the following steps under your account, not as root

#1) Create your local osbf-lua dir:
    mkdir $HOME/osbf-lua
# 2) Create your log and cache dirs:
    mkdir $HOME/osbf-lua/log
    mkdir $HOME/osbf-lua/cache
# Note: Old messages in the cache dir should be deleted
# regularly, typically from a cron job, to preserve disk space. 

# 3) Copy the spamfilter config file to your dir:
    cp /usr/local/share/osbf-lua/spamfilter_config.lua \
        $HOME/osbf-lua

# 4) Edit spamfilter_config.lua to set your password
    $EDITOR $HOME/osbf-lua/spamfilter_config.lua

# 5) Change to the current dir to your osbf-lua dir and 
# and create the spamfilter databases (spam.cfc, nonspam.cfc)
     cd $HOME/osbf-lua
     lua /usr/local/share/osbf-lua/create_databases.lua
# 6) Add the following lines to your .procmailrc:

# set OSBF_LUA_DIR to where spamfilter.lua, 
# spamfilter_command.lua, etc were installed

OSBF_LUA_DIR=/usr/local/share/osbf-lua
OSBF_LUA_USER_DIR=$HOME/osbf-lua

:0fw: .msgid.lock
* < 350000 # don't check messages greater than 350000 bytes
| $OSBF_LUA_DIR/spamfilter.lua --udir $OSBF_LUA_USER_DIR
</pre>

<b>Training OSBF-Lua from inside Mutt</b>

You should now have OSBF-Lua hooked up to your mail pipeline.  However, it will only work with the email gateway control method.  That is, OSBF-Lua will notice certain commands in the Subject header and respond to those.  While nifty and occasionally useful, it is tedious to train the filter by sending emails to yourself.  A much easier method is to set up a Mutt macro which invokes the filter.  I have mine bound to shift-s (spam) and shift-h (ham).  The additions to your $HOME/.muttrc file are trivial:

<pre lang="ini" line="1">
macro index,pager H "<pipe-message>~/learn-nonspam.sh<enter>"
macro index,pager S "<pipe-message>~/learn-spam.sh<enter>"
</pre>

Now for the simple shell scripts:

$HOME/learn-spam.sh
<pre lang="bash" line="1">
#!/bin/sh
UDIR=$HOME/osbf-lua
LEARN=/usr/local/share/osbf-lua/spamfilter.lua
$LEARN --learn=spam --udir=$UDIR
</pre>

$HOME/learn-nonspam.sh
<pre lang="bash" line="1">
#!/bin/sh
UDIR=$HOME/osbf-lua
LEARN=/usr/local/share/osbf-lua/spamfilter.lua
$LEARN --learn=nonspam --udir=$UDIR
</pre>


Now you can train the filter from inside Mutt, with a single keypress!  That should save a lot of repetitive and time-consuming fiddling with the e-mail gateway method.
]]></content:encoded>
    </item>
  </channel>
</rss>

