So looking at XKCD you’d think ‘How could they possibly let this bug go?’ That’s so obvious:
Looking at this other blog that purports to show the code causing the issue (and I do look at various forms of computer code daily) you’d say ‘How the heck would they ever know this is a problem?’ Perhaps we shouldn’t be allowed to code in C. The fix:
The most important part of the fix was this:
/* Read type and payload length first */
if (1 + 2 + 16 > s->s3->rrec.length)
return 0; /* silently discard */
hbtype = *p++;
if (1 + 2 + payload + 16 > s->s3->rrec.length)
return 0; /* silently discard per RFC 6520 sec. 4 */
pl = p;
This does two things: the first check stops zero-length heartbeats. The second check checks to make sure that the actual record length is sufficiently long. That’s it.
OK, maybe I see somewhere there a related fix they descibe. I suppose if one gets over the ‘s->s3->rrec.length’ it makes sense as long as they are doing that (I assume) pointer *p++ right too. And pl=p, that’s totally obvious!
If you look here it wasn’t looking to request lengths at all before the fix. Some QA! If the technology is to allow you to request a number of letters and you don’t test that you can get more than you should that is still sort of bad.
Thank goodness for SQL, Java, and Python! Even thank goodness for Mumps (the programming language). At least (not in Mumps) then you could make the request an object of the requested length and drop everything else.
This site was secure I found thanks to it using forward secrecy (at least since the conversion to Debian – seems like the old version of FreeBSD was unaffected as well). No idea how it was using that but phew! Since ran some SSL check and we’re an even more tight ship now!
The one bit of switching to Debian that was difficult was setting up Exim and SMTP. I tried using the configuration off of FreeBSD but Debian just has it’s own special setup. For whatever reason the FreeBSD configuration resulted in SMTP reject errors.
This is a helpful command for debugging the exim configuration – it really helps to know what Exim4 is actually doing with all those config files!
So I tried the Debian approach and that worked ok for receiving mail (which the FreeBSD also did). However I still couldn’t connect with the mail client. Looking at it again if I’d read that wiki closely and followed all the instructions it would have saved me a couple of hours… I just needed to uncomment the SASL bit in /etc/exim4/exim4.conf.template. So always follow the instructions closely I suppose!
So the old server has always crashed quite a bit and been a bit slow for no obvious reason otherwise. So I spent some time trying to optimize the server with various things but then it occurred to me to consider this message on login:
### ZFS Tuning ###
ZFS is NOT tuned for instances with less than 1GB RAM.
For tuning 256MB and 512MB instances, the following link is recommended reading
Now the image I’m using has less than 1GB RAM so I went to that wiki and tried to do some stuff and totally messed up the performance. At least that somewhat proved to me that it was ZFS and low memory causing the random hangs. The wiki notes the ‘bursting’ behaviour of the ZFS filesystem sometimes. That’s what the server had been doing! Trying to tweak it a bit I just made it worse. There was nothing in swap and any disk based activity was REALLY SLOW. Just tarring things up after my tweaks freezes the system… Darn fancy filesystems. Combine that with I was starting to get some package issues with a mixture of compiled packages and trying ‘pkgng’ I thought I’d given Debian a try again.
Debian has no slowness issues! I’m 99% sure it was the ZFS and low memory. So as much as I like FreeBSD this I’ll use Debian. I do like apt better than pkgng anyhow, much more refined. I was almost getting into a version of RPM hell with FreeBSD there… (At least it’d never ruin the whole system with BSD though)
Anyhow switching to Debian worked fine, I can’t get the SMTP server working for some reason but everything else was easy to port. And there is no more random hanging. Also no swap space is used at all for the same configuration, so either it very likely was ZFS that was the issue on FreeBSD.