Archive, compress, encrypt and checksum in one pass

I got very bored with my existing (read: crap) backup scripts for my colo server, and hence decided to do something about it.

Previously I’d been using tar, pbzip, openssl and sha256sum in a script doing discrete operations, but this was inefficient in both disk IOPS, temporary space usage and processor time.

With a bit of tinkering, the whole thing can be done in one pass with a bit of BASH-fu:

tar -cf - $FILESPEC 2>/dev/null \
|pbzip2 \
|openssl enc -aes-256-cbc -salt -pass file:$SECRET \
|tee >(sha256sum |sed "s/\-/$OUTFILE/g" > $LOCALPATH/$OUTFILE.sha256sum)\
|cat - > $LOCALPATH/$OUTFILE

This will use tar to archive the contents of $FILESPEC  and send it to pbzip (parallel bzip compression – scales over as many cores as you have in your box – very useful!). This is then piped into OpenSSL which applies a 256-bit AES cyclical block cipher, locked with a password read from the file specified by $SECRET.

Finally, tee is then used to split the output of OpenSSL – one stream goes to sha256sum to generate the checksum, while the other is sent to cat which writes it to disk. The sha256sum looks a bit more complex than expected as, due to the fact that sha256sum is receiving data on STDIN, it will use “-” as the filename in the checksum file.

This is not ideal as using sha256sum’s “-c” option to subsequently verify the file will result in a file not found error. The sed command takes the output of sha256sum, looks for the occurrence of “-” and replaces it with the filename, suffixed by “.sha256sum”.

And, surprisingly, it works 🙂

Speedometers and speedo converters (Celica ST205 JDM)

Having been chatting with a couple of people about speedo converters, speed limiters and soforth on JDM cars (specifically the ST205 Celica GT-Four in this case), I put together a little video detailing my experiments and findings. Hope it helps someone!

I have an ST185 RC to play with now!

In addition to the ST205, I’ve managed to acquire an ST185 to play with in return for helping a friend with his XK8. Yet another time-sink / money pit 🙂

It’s in a bit of a state seeing as some little tw@ decided to stuff a brick through the rear window, but all of the important bits are present…

Adding an external SSD to a mid-2011 iMac

OK, so I was a cheapskate and now I’m paying the price for not having stumped up the quite frankly preposterous number of sheckles required to have a factory-fit Solid State Drive (SSD) in my iMac. This little rambling post should allow you to get an idea for how I’ve gone about performing this upgrade. It’s actually not that difficult and anyone who’s swapped a drive out in a PC before now should be more than capable of doing this.

It’s also worth underlining that you do not need to open your iShiny during this procedure – it’s purely external. Having had to open a friend’s iMac the other day to replace his hard drive (yes, you can do that without the fans going mad – more here) and the hassle involved in keeping everything dust- and fingerprint-free, I was more than willing to investigate the external options afforded by my until-recently unused Thunderbolt ports. Continue reading

Don’t panic! VoLTE will get better…

A colleague cited some random exclamatory industry press piece today that made me get all nostalgic about 3G and, specifically, about how shit it was.

They were getting all excited about the fact that – shock horror – a device using VoLTE to conduct a voice call consumed approximately twice the amount of energy as it did when performing a voice call of identical duration on 2G.

Of course it will!

Wind the clock back ten years and exactly the same was true of voice calls on the first 3G handsets: My old Motorola barge of a 3G “smart” phone – the A920 (pictured left) – couldn’t last through an hour-long conversation on a fresh charge when on 3G, yet on 2G it could go for four or five hours.

As the silicon gets optimised (reducing power consumed to drive the processors) and the networks improve (more cells meaning better coverage meaning less power consumed to drive the radio transmitter), it’ll soon achieve parity with 3G. As takeup is also expected to be much more rapid than 3G, this process shouldn’t take as long this time round as it’s in everyone’s interests for it to get better faster.

Interestingly, the latest generation of 3G phones now exceed the energy efficiency of their 2G counterparts as companies are continuing to optimise the 3G chipsets whereas the 2G stuff is essentially considered old hat…

Corporal Jones is right – don’t panic!

Braindamaged design Jaguar XK8 oil / temperature fail

A friend of mine has an XK8 that’s just started making abominable noises from its engine’s nether regions and I’ve said that I’ll take a look at it for him.

It turns out that XK8s dropping oil pressure which causes them to chew up their big-end and main bearings is actually a common problem – the oil pump gives up and the first indication you have that there’s something wrong is the evil scrapey scrawpy clonky bangy noise from under the bonnet.

The reason the very undesirable audible warning appears before the car gives you any other indication is because the oil pressure gauge is actually just an elaborate light! Continue reading

Mid 2011 iMac hard drive upgrade

Well, having read loads of things about the problems involved with upgrading the hard drive in a mid-2011 iMac, I recently had to deal with the issue when working on a friend’s machine whose drive had exploded just after the warranty had expired.

If you don’t want or need an SSD installed along side your hard drive, you can commandeer the SSD’s SATA port to power a standard mechanical drive without a temperature sensor.

To defeat the fan problems, you can trick the Mac in to ignoring the sensor in hardware by adding part number 922-9877 to the original mechanical drive SATA port. It’s essentially a plug with pins 1 and 6 shorted, which presumably instructs the hardware management processor that the drive is not present and hence it should ignore invalid (or missing) temperature readings.

This seems to have worked, and the machine is now blissfully unaware that it has a mechanical drive fitted. Obviously you need to keep an eye on temperatures, and one of the free fan management apps can manually ramp the mechanical bay fan up accordingly, but this seems to be a reasonably good workaround. I’m going to try and dissect the dead drive to see from where the temperature data appears and if it’s easy enough to retrofit an auxiliary temperature sensor in to the SATA cable, but that’ll have to wait for another day!