Categories
BSD/Linux Computer Tech

(Forced) Move from zfsonlinux to Ubuntu ZFS

I upgraded the server from 14.04 to 16.04 and slowly discovered everything was not working perfectly with the ZFS functionality. Researching it appears zfsonlinux doesn’t support 16.04 since ZFS is a part of 16.04. However I couldn’t get the Ubuntu version to load and got this error. It gave me the pointer that the old trusty (14.04) version was the issue.

The following packages have unmet dependencies:
 zfsutils-linux : Depends: zfs-doc (= 0.6.5.6-0ubuntu15) but 0.6.5.7-1~trusty is to be installed
                  Depends: libnvpair1linux but it is not going to be installed
                  Depends: libuutil1linux but it is not going to be installed
                  Depends: libzfs2linux but it is not going to be installed
                  Depends: libzpool2linux but it is not going to be installed
                  Recommends: zfs-zed but it is not going to be installed
E: Unable to correct problems, you have held broken packages.

So had to purge the non Ubuntu trusty packages and start over. This will find the old installed zfs packages:

dpkg -l|grep zfs

Then you can purge the packages old trusty (14.04) to remove them:

sudo apt-get purge zfs-doc

Make sure nothing odd is in /etc/apt. Then install the native Ubuntu version with the normal:

sudo apt install zfsutils-linux
Categories
BSD/Linux Security Tech

The wonders of Let’s Encrypt and Certbot in relation to StartSSL/StartCom issues

All websites should be encrypted. Definitely don’t do anything secure over an unencrypted connection.

To support that the site has always had SSL setup. However I just discovered that the SSL certs the site uses by StartSSL have been revoked by all major browsers it appears!

A little digging for a new free SSL cert site came up with Let’s Encrypt. That in combo with Certbot is SUPER EASY to get certs and keep them up to date. All for free! I’ll probably even donate it’s so easy now!

The trick for certbot for me was to use the standalone check so I didn’t have to mess with the NGINX server’s folder security.

$ certbot certonly --standalone -d example.com -d www.example.com

Then you just point NGINX to the PEM files it creates and you’re set!

ssl_certificate /etc/letsencrypt/live/example.com/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/example.com/privkey.pem;

EASY PEASY! Now Chrome and Safari work again!

Then to renew all the certs at once you can run this

certbot renew --pre-hook "service nginx stop" --post-hook "service nginx start"
Categories
BSD/Linux Tech

Duplicity restore

As a part of restoring all the posts from the old Drupal server I finally had to use Duplicity to restore a backup. Always wondered how it’d actually work. Twas easy! This restores a backup from 9/1/2014 to /tmp/restore.

duplicity restore –time 2014-09-01 s3+http://rack_deb/drupal /tmp/restore2

Categories
BSD/Linux Tech

ZFS notes

Ignore this post if you have no idea what ZFS is.

ZFS related notes for posterity. I’ve got it up and running as well as BTRFS. ZFS seems better. So that’s settled.

Categories
BSD/Linux DrupalRecover Tech

piSpy

Here’s a quick python script to use the camera you can attach to a Raspberry Pi and take a picture whenever it detects motion. This is pretty cool, there are all sorts of silly things you can do if you can do motion detection. And to do it with the Raspberry Pi and attached camera is relatively inexpensive (less than $100) to boot! Next time someone knocks our mailbox down we’ll have ’em!

I started from the script here and modified it to just take one picture to both test whether anything changed as well as save the picture if something changed. Seems a bit better than the original taking two pictures – at least this doesn’t cause the Pi to freeze. That means the Pi have to scan a bit larger image for a change but skipping pixels seems to keep it under control. It also takes better pictures as takes some time (the -t 500 in the raspistill call) to calibrate the camera.

#!/usr/bin/env python

import StringIO
import subprocess
import os
import time
from datetime import datetime
from PIL import Image

# Motion detection settings:
# Threshold (how much a pixel has to change by to be marked as "changed")
# Sensitivity (how many changed pixels before capturing an image)
# ForceCapture (whether to force an image to be captured every forceCaptureTime seconds)
threshold = 10
sensitivity = 2000
forceCapture = True
forceCaptureTime = 60 * 60 # Once an hour

# File settings
saveWidth = 1280
saveHeight = 960
diskSpaceToReserve = 500 * 1024 * 1024 # Keep 500 mb free on disk


# Capture a small test image (for motion detection)
def captureTestImage():
    command = "raspistill -w %s -h %s -t 500 -e bmp -o -" % (1280, 960)
    imageData = StringIO.StringIO()
    imageData.write(subprocess.check_output(command, shell=True))
    imageData.seek(0)
    im = Image.open(imageData)
    buffer = im.load()
    imageData.close()
    return im, buffer

# Keep free space above given level
def keepDiskSpaceFree(bytesToReserve):
    if (getFreeSpace() < bytesToReserve):
        for filename in sorted(os.listdir(".")):
            if filename.startswith("capture") and filename.endswith(".jpg"):
                os.remove(filename)
                print "Deleted %s to avoid filling disk" % filename
                if (getFreeSpace() > bytesToReserve):
                    return

# Get available disk space
def getFreeSpace():
    st = os.statvfs(".")
    du = st.f_bavail * st.f_frsize
    return du

# Get first image
image1, buffer1 = captureTestImage()

# Reset last capture time
lastCapture = time.time()

while (True):
    # Get comparison image
    image2, buffer2 = captureTestImage()

    # Count changed pixels
    changedPixels = 0

    for x in xrange(0, 1280, 4):
        for y in xrange(0, 960, 4):
            # Just check green channel as it's the highest quality channel
            pixdiff = abs(buffer1[x,y][1] - buffer2[x,y][1])
            if pixdiff > threshold:
                changedPixels += 1

    # Check force capture
    if forceCapture:
        if time.time() - lastCapture > forceCaptureTime:
            changedPixels = sensitivity + 1

    # Save an image if pixels changed
    if changedPixels > sensitivity:
        lastCapture = time.time()
        timeN = datetime.now()
        print "Save jpg"
        filename = "capture-%04d%02d%02d-%02d%02d%02d.jpg" % (timeN.year, timeN.month, timeN.day, timeN.hour, timeN.minute, timeN.second)
        image2.save(filename, "JPEG")
        keepDiskSpaceFree(bytesToReserve)
    # Swap comparison buffers
    image1 = image2
    buffer1 = buffer2
    time.sleep(2)
    print "Done waiting"
Categories
BSD/Linux DrupalRecover Security Tech

Heartbleed

So looking at XKCD you’d think ‘How could they possibly let this bug go?’ That’s so obvious:

Looking at this other blog that purports to show the code causing the issue (and I do look at various forms of computer code daily) you’d say ‘How the heck would they ever know this is a problem?’ Perhaps we shouldn’t be allowed to code in C. The fix:

The fix

The most important part of the fix was this:

/* Read type and payload length first */
if (1 + 2 + 16 > s->s3->rrec.length)
    return 0; /* silently discard */
hbtype = *p++;
n2s(p, payload);
if (1 + 2 + payload + 16 > s->s3->rrec.length)
    return 0; /* silently discard per RFC 6520 sec. 4 */
pl = p;

This does two things: the first check stops zero-length heartbeats. The second check checks to make sure that the actual record length is sufficiently long. That’s it.

OK, maybe I see somewhere there a related fix they descibe. I suppose if one gets over the ‘s->s3->rrec.length’ it makes sense as long as they are doing that (I assume) pointer *p++ right too. And pl=p, that’s totally obvious!

If you look here it wasn’t looking to request lengths at all before the fix. Some QA! If the technology is to allow you to request a number of letters and you don’t test that you can get more than you should that is still sort of bad.

Thank goodness for SQL, Java, and Python! Even thank goodness for Mumps (the programming language). At least (not in Mumps) then you could make the request an object of the requested length and drop everything else.

This site was secure I found thanks to it using forward secrecy (at least since the conversion to Debian – seems like the old version of FreeBSD was unaffected as well). No idea how it was using that but phew! Since ran some SSL check and we’re an even more tight ship now!

Categories
BSD/Linux DrupalRecover Tech

Debian and Exim4

The one bit of switching to Debian that was difficult was setting up Exim and SMTP. I tried using the configuration off of FreeBSD but Debian just has it’s own special setup. For whatever reason the FreeBSD configuration resulted in SMTP reject errors.

This is a helpful command for debugging the exim configuration – it really helps to know what Exim4 is actually doing with all those config files!

exim -bP

So I tried the Debian approach and that worked ok for receiving mail (which the FreeBSD also did). However I still couldn’t connect with the mail client. Looking at it again if I’d read that wiki closely and followed all the instructions it would have saved me a couple of hours… I just needed to uncomment the SASL bit in /etc/exim4/exim4.conf.template. So always follow the instructions closely I suppose!

Categories
BSD/Linux DrupalRecover Tech

FreeBSD, ZFS, Rackspace, and to Debian

So the old server has always crashed quite a bit and been a bit slow for no obvious reason otherwise. So I spent some time trying to optimize the server with various things but then it occurred to me to consider this message on login:

### ZFS Tuning ###

ZFS is NOT tuned for instances with less than 1GB RAM.

For tuning 256MB and 512MB instances, the following link is recommended reading

http://wiki.freebsd.org/ZFSTuningGuide

Now the image I’m using has less than 1GB RAM so I went to that wiki and tried to do some stuff and totally messed up the performance. At least that somewhat proved to me that it was ZFS and low memory causing the random hangs. The wiki notes the ‘bursting’ behaviour of the ZFS filesystem sometimes. That’s what the server had been doing! Trying to tweak it a bit I just made it worse. There was nothing in swap and any disk based activity was REALLY SLOW. Just tarring things up after my tweaks freezes the system… Darn fancy filesystems. Combine that with I was starting to get some package issues with a mixture of compiled packages and trying ‘pkgng’ I thought I’d given Debian a try again.

Debian has no slowness issues! I’m 99% sure it was the ZFS and low memory. So as much as I like FreeBSD this I’ll use Debian. I do like apt better than pkgng anyhow, much more refined. I was almost getting into a version of RPM hell with FreeBSD there… (At least it’d never ruin the whole system with BSD though)

Anyhow switching to Debian worked fine, I can’t get the SMTP server working for some reason but everything else was easy to port. And there is no more random hanging. Also  no swap space is used at all for the same configuration, so either it very likely was ZFS that was the issue on FreeBSD.

Categories
BSD/Linux DrupalRecover Tech

Fetchmail and SSL certificates

I have to search for this periodically being sites like to change their SSL certificates every once and a while. So here’s another place to look for it!

If you don’t know what Fetchmail is its a little program (maybe its gigatic, I don’t know) that fetches email for you from many different email servers. This allows me to have one place where I store all my email across all my email addresses and then I can use an IMAP email server to view it all in one place. Very nice if you’ve many different email address.

In order to do this magic you have to tell fetchmail how to login to your email accounts and its always better to do this securely over SSL. To do it over SSL you have to tell fetchmail the site’s SSL certificiate (or you could weaken fetchmail to not care to validate the certificate but that seems silly). And to do that you have to tell it in hex or other computer sprach.

So first you need to get the SSL certificate in computer sprach:

openssl s_client -connect pop3.live.com:995 -showcerts | openssl x509 -fingerprint -noout -md5

Running this will get you the certificate computer sprach entry itself that’ll look something like this:

91:63:CF:6F:DF:03:3C:B9:3E:19:B3:1C:FF:EB:3C:25

Second you need to tell fecthmail to use this id when it connects to the server in the fetchmailrc file:

poll pop3.live.com uidl protocol POP3 user “xxxyyy@live.com” there with password “ohSuchASecurePassword!@#$” is xxxx@tttt here ssl sslfingerprint ‘91:63:CF:6F:DF:03:3C:B9:3E:19:B3:1C:FF:EB:3C:25

There you have it! That’s almost a good memorable, secure password I just made up there.

Categories
BSD/Linux DrupalRecover Tech

Port forwarding VNC connection

One sad day our old modem failed. The phone company gave us a new DSL modem/router. I had figured out how to setup portforwarding and such on the old modem. I can’t get into the new one at all! After briefly attempting to get info out of the phone company about how to access the new modem’s admin interface to no success I gave up on all my nice old port forwards.