So the old server has always crashed quite a bit and been a bit slow for no obvious reason otherwise. So I spent some time trying to optimize the server with various things but then it occurred to me to consider this message on login:
### ZFS Tuning ###
ZFS is NOT tuned for instances with less than 1GB RAM.
For tuning 256MB and 512MB instances, the following link is recommended reading
Now the image I’m using has less than 1GB RAM so I went to that wiki and tried to do some stuff and totally messed up the performance. At least that somewhat proved to me that it was ZFS and low memory causing the random hangs. The wiki notes the ‘bursting’ behaviour of the ZFS filesystem sometimes. That’s what the server had been doing! Trying to tweak it a bit I just made it worse. There was nothing in swap and any disk based activity was REALLY SLOW. Just tarring things up after my tweaks freezes the system… Darn fancy filesystems. Combine that with I was starting to get some package issues with a mixture of compiled packages and trying ‘pkgng’ I thought I’d given Debian a try again.
Debian has no slowness issues! I’m 99% sure it was the ZFS and low memory. So as much as I like FreeBSD this I’ll use Debian. I do like apt better than pkgng anyhow, much more refined. I was almost getting into a version of RPM hell with FreeBSD there… (At least it’d never ruin the whole system with BSD though)
Anyhow switching to Debian worked fine, I can’t get the SMTP server working for some reason but everything else was easy to port. And there is no more random hanging. Also no swap space is used at all for the same configuration, so either it very likely was ZFS that was the issue on FreeBSD.