Portable Apps - Such a good idea after all?

Like many people, leaving all my personal information Out There is something that goes against my nature.  After all my years in IT, if there's one thing I've learned it's that Out There isn't entirely as safe and secure as many would have you believe.  I much prefer to have my personal details and data In Here, with me and, preferably, as much as possible in the "everything online" World, offline whenever not in use.

It's a principle I apply to backing up all my files.  I use the three-layer approach.  There's the data I have in use every day.  That sits on the machine I use every day.  Then there's the first-level backup, changes automatically copied to network attached storage (NAS) regularly throughout the day.  Then there's the third-level which consists of a portable hard-drive, which is only attached and switched on when data is being copied to it.  After that it's secured away from prying eyes.

Now, you can call me paranoid, but that doesn't mean they're not out to get you, to paraphrase Joseph Heller.  So it was with this in mind that I started looking at my email accounts.  Like many, I have a lot of them.  Trying to keep track of each individually, using webmail, is not an attractive proposition and so I have, for many years, been using various email clients.

Like most I suppose, I first started using MS Outlook Express.  I mainly used this because of  Microsoft's intransigence over allowing users to access Hotmail accounts with non-Microsoft tools. However, when my use of Hotmail dropped to virtually zero, once I'd discovered other email providers, I started looking around for other alternatives to OE.

Eventually, after trying a few other initially promising applications, I happened upon Mozilla Thunderbird.  And there I stayed.  I liked the idea of being able to gather all of my email from various sources into one collection and to delete it from Out There.

But, of course, it would still be on my main machine.  What would be even better would be for me to be able to carry all my email around with me and to be able to access it wherever a computer was available.  That set me thinking: there were probably many other applications to which that could also apply.  I have my preferences but they are not shared with everyone.  What if my Irfanview, ArsClip, BitmeterII, TrueCrypt or 7-Zip wasn't installed on the machine I was using?

What I needed was a USB data stick and some portable versions of my favourite apps.  My first was a tiny 64MB one: useful for experimenting but not really of much use in practice so, spotting an 8GB Sandisk Titanium Cruzer in PCWorld for £15 I decided to go for it seriously.

I first tried the U3 platform that came with the device but it soon became very obvious that U3 is somewhat less than well supported, with many portable versions of applications being well out-of-date compared with the "normal" version.   It wasn't long before I abandoned it in favour of John T. Haller's PortableApps platform (http://tinyurl.com/fxozf ) instead.  Versions of most applications which have been made portable for the PortableApps platform seem to be kept slap-bang up-to-date.

I tried a few small apps which would not cause me any trouble if they didn't work out and these seemed to suggest that this would be a viable platform for what I wanted to do.  So, it was time to get a bit more adventurous.  Time to try Portable Thunderbird.

Download and install was a doddle: the portable versions come with their own installer, a .paf file, but essentially all that happens is that everything is installed in its own folder on the device, under the PortableApps platform folder.  Nothing gets stored on the host computer; that's the whole point.  You should be able to plug your flash drive into any computer and use your applications there without leaving any trace behind when you remove it.  So, nothing in the Windows Registry; nothing in the Appdata folders of C: drive User Profile.  All that was needed was to transfer across my original Thunderbird profile to the portable version, including all of the email folders, and then delete Thunderbird from the computer.

All seemed to work just fine but I did notice that in use, Thunderbird on the flash drive was significantly slower than it had been on the computer.  The Cruzer has a light that flashes quickly to indicate data activity and slower when idle.  Whenever I was using Thunderbird the light was flashing like mad most of the time and not just when email was being downloaded from my ISPs.

On investigation I realised the reason: Thunderbird's email storage is all based on folders.  Each email folder and sub-folder you set up to store and organise your email results in a physical folder being created in the email file system hierarchy.  So, deleting an email, for instance, results in the email file being read out of where it exists, say your Inbox, deleted from there and and then written to the Trash folder, a two-way data movement across the USB port.

Now, where we mostly are today, with USB 2.0 as the most common standard, and its at best 80MBps data transfer speeds, this is never going to be as fast as native data speeds on most computers, between the processor and the hard drives.  In addition, data speeds reading and writing from and to the memory chips on the Cruzer are, at best 5.6MBps and 1.7MBps respectively (PCPro: http://tinyurl.com/2v5zw6s  - I'm on WindowsXP so Vista's ReadyBoost doesn't come as installed).

Consequently, moving data about as Thunderbird does is never going to be a good idea on a portable platform such as this, if at all.  Coming from the world of "real" IT, as I do, I'm used to data stores where all the information is kept in one place and where locating that data is all done by index pointers.  Want to move data from one "folder" to another?  Just change the pointer; don't move the data.  Thunderbird (and, it's fair to say, in this it is not unique) doesn't work that way.

So, compromises have to be accepted for accessibility, so I carried on using Portable Thunderbird.  The size of my email folders grew, as they do: you always seem to keep more than you delete.  I had automatic compacting set on and when this kicked into action you really noticed how slow the flash drive was.  I also backed up the entire Thunderbird profile on a regular basis, just to ensure that I didn't lose anything if I lost the flash drive.

It was during one of these backup exercises that I noticed something a lot more disturbing.  I came to check a backup and the log indicated the backup had failed.  It was then that I noticed that the light on the Cruzer had gone out, just like as if I had Ejected it but not yet unplugged it!  I tried the backup again and this time, half-way through the light suddenly went bright and not pulsing.  Shortly after that I got a message indicating a critical problem with the flash drive.

Strange thing, though, was that once everything was restarted, Thunderbird carried on working perfectly OK, I just didn't seem to be able to backup the data!  I tried avoiding using my backup software of preference (Synchredible - http://tinyurl.com/35z89rk - Highly recommended) and simply do a direct copy of all of the folders.  I stil got the same outcome, so the problem clearly wasn't the backup software.

I then tried the device in other computers and with different operating systems but every time the same outcome: the copy would fail part way through.  I then resorted to copying folders and even individual files one-by-one.  Finally I discovered that the problem affected certain files but not others.  However, try as I may, I could find any common factors that would cause the problem.  Indeed, a copy that failed one time might work another!  I hadn't tried to move the data so it presumably still occupied exactly the same memory location on the Cruzer.

Now, I can't figure out why this problem exists.  Let's face it, the Cruzer flash drive has no moving parts; it's all electronic.  Do memory chips "wear out"?  Why should they?  Is there any way of checking if they have?  Well, it turns out that the type of memory used in flash drives can wear out.  Erasing data from flash drive memory chips causes eventual deterioration (Wikipedia suggest, at worst, after around 100K program-erase cycles - http://tinyurl.com/b549s).  So, maybe this is what has happened, exacerbated by Thunderbird's email management methodology.

Eventually, however, I did manage to get a clean backup of all of my Thunderbird profile.  By now I was starting to wonder if I could trust this platform any more.  After all, if you can't back up your email and then lose it all you really are in the brown, sticky stuff.

So, I decided to abandon my flirtation with portable applications, at least in so far as concerns my use of Thunderbird.  I have reinstalled the standard Thunderbird application on my hard drive and transferred over by eventually successfully backed up profile.  That is what I am now using today.

Have I entirely abandoned portable applications?  No.  However, I have restricted what I have installed on the Cruzer flash drive to just those applications whose data I could afford to lose or which don't actually store much if any data locally or what they do store remains mostly static and unchanging.  Mostly they consist of utilities that make my life so much simpler.

Will I go back to Portable Thunderbird?  Unlikely, so long as the Thunderbird developers do nothing to change their email storage methodology.  I really can live without that sinking feeling every time a backup fails.  As for flash drives in general, we can only hope that the reliability of memory chips improves in the future.  Speeds as well.  The gradual introduction of the far faster data speeds that come with the new USB 3.0 standard may also go a long way to making flash drives a viable working platform.

Until then, I'm sticking with what I can trust.