Silent, fast, long-living USB storage devices for use as NSLU2 hard drive?

As I recently bought a NSLU2 ("slug") for 24/7 server usage and random ARM-based development (more on that later), I'm looking for a suitable storage device to use as the root filesystem for a complete Debian system.

The requirements are:

  • Large enough (>= 1 GB).
  • Silent. Hard disks are not an option (no IDE, no SATA, no USB, neither 2.5" nor 3.5")
  • Low power consumption.
  • As fast as possible.
  • USB-attached (the slug only has 2 USB connectors).
  • Should live as long as possible.

The last item is the most important.

The obvious choice is a USB memory stick, but unfortunately those are flash-based and only survive a certain number of write cycles. Thus I'm looking for something which at least survives enough write cycles to make it usable for a few years...

I do know several ways to reduce the number of writes via software, that's not what I'm currently interested in. I'd like to know which storage types will survive the longest amount of time (because of their hardware properties).

Some options:

  • USB memory stick, obviously. Usually quite slow, but there are quite fast ones (read 33 MB/s, write 22 MB/s), too. Some seem to survive 100.000 or even 1.000.000 write cycles, but that may not be enough. Are there any sticks with more possible write cycles?
  • Any other USB-attached (via external USB card-reader if necessary) silent storage type: SD, MMC, Transflash, Memory Stick, Compact Flash, whatever. Does any of those survive more write cycles?
  • Any other viable option?

Has somebody else done something like this before and can share some experiences as to which storage type is best suited for such a scenario? I suspect my slug will be mostly idle, but there might also be phases where it runs on 100% CPU and heavy disk I/O for multiple days in a row...

Comments

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

10 year waranty

I saw this flash disc has 10 year waranty:
CORSAIR FLASH VOYAGER 8GB USB 2.0

And it cost only 760 SEK (about 100 USD)

I assume it can take alot of write cycles if they
dare give it 10 year waranty.

I am tempted to try myself, would be lovely as
web and mail server...

/Henrik

NFS

How about NFS root?

NFS

Hm, that would only move the "problem" somewhere else, right? I need some storage somewhere which I can mount via NFS...

Of course, I could probably use the slug as sort of a "thin client" to get the root filesystem (and everything else) from some remote server or so, but I'd prefer to have everything in one place.

JFFS2

And JFFS2?.

There are some embedded oriented filesystems as JFFS2, squashfs (read-only AFAIK) and maybe others.

Maybe this could be useful for you.

JFFS2

Yes, JFFS2 might be an option. I don't know if the Debian installer (for the slug) supports this out of the box (I'll check later). Maybe you can do it manually... I'll need to read more about JFFS2 before I decide to use it, but it sounds like a good choice for a filesystem on flash devices.

But the question still remains which type (or brand) of flash device is best suited...

JFFS2

i looked into using JFFS2 myself for such things but it's not for use as a filesystem. it's more for distributing wear when you have a flash chip embedded right on a board. if you're using anything like a CF/SD card, that self-regulates bad blocks, etc.

All the card types you

All the card types you mentioned are flash-based. So beside a ram-based fs there's no alternative for that.

But I don't know anyone who has managed to produce block failures on a flash device. Even in a test from the guys from ct', where a several hundred thousand times the same logical block was written, they had to abort the test after a few days, as no block failure was detected. So you should try it out. It's definitely the cheapest and easiest way.

Flash

Yeah, maybe I'll just do some stress tests on a cheapo USB stick (over a long period of time). I intend to use this setup for several years, so just a few days of testing will probably not suffice...

Solid State Disks

You want solid state disks, i.e. RAM+battery.

You didn't mention it has to be cheap and it won't be but it will be *really* fast and will last as long as memory does.

Don't know if they are already packed as USB devices, I know for sure they come in PCI format.

Forget CF, MMC, SD, etc. they are all the same: flash memory in different packages. Flash comes in commercial grade and industrial grade. Industrial grade lives longer but costs a lot more.

Regards,

L

Solid State Disks

Solid State Disks sounds good. I'll see if I can find some for USB and for a reasonable price.

Do you know how many (anticipated) write cycles "industrial grade" flash devices can survive?

Is USB 2.5" or 1.8" HD really too loud?

If you don't want HDs, there is no option but a flash based drive. However, I'm using an Hitachi MicroDrive in my PCEngines WRAP based router, and I never heard it, not even with an open case, so perhaps a USB<->CF adapter of some sort and a microdrive is what you should actually look for. But this isn't reeally fast, about usually fast enough (approx 10MB/sec IIRC). Pretty much unlimited writes.

But I also had some SanDisk MicroCruizer USB flash drive (quite small form factor and pretty fast (20MB write and about 25-30MB read IIRC). I did put one of them on a stresstest to get an idea how many writes it actually survived, and it was at least in the 500k write cycles range, when I aborted the test without a single failure. I _should_ have let it run until I got a failure, but calculating a maximum of about 500 writes per day on a specific part of the flash, I still get at least 3 years of use out of it. But I replaced every flash drive I had within 3 years if not faster, so it didn't matter to me.

Disks are too loud, yes.

Disks are too loud, yes. Maybe I'll use a 2.5" disk one day when I really needs lots of storage, but for now I'd like to stay with absolutely noise-less devices.

Microdrives might be an options, although they seem to be not as robust as CF, and probably make noise (they have spinning disks, AFAIK)...

Don't forget Microdrive 1"

Don't forget Microdrive 1" cf sized ones like those used in some mp3 portables. The hitachi 4gb (with usb card reader) costs 65$ in pricewatch.

Perhaps you should ask a friend with an mp3 microdrive based portable like creative's zeon to let you "listen" to it. I highly doubt you'll hear anything like a spinning disc at all.

I don't recommend you go with ram+battery, the battery will die before any of the flash ram discs.

Actually permit me to correct the above statement.

(no, i'm not the same "anonymous")

Anyways, the battery will only "die" if it is being used while the drive is disconnected from the mains (system power,, etc). The battery only serves to keep the RAM alive when no mains power is coming.

You can always hack it and connect it to a LiIon battery or NiMh or somesuch. They are becoming quite small. You want it to live forever, that's your clue. SSD drive, which you mod to contain a rechargeable battery. You can also, depending on the voltage of the ram on the drive, hook it up to SureFire CR123 batteries, (SF also makes some damn fine flashlights that use those batteries, but not cheap, sold mostly to military and people who value literally bulletproof gear), or you can use their rechargeable batteries sold as "energizer T2 or T10" (in the USA, not sure what they're sold as in Germany...

USB Pocket HD

Hi

I bought a NSLU2, too.
I knew that flash memory has it's problems with lifetime regarding write cycles.

So I decided to use a 5 GB Seagate Pocket HD
http://www.seagate.com/www/en-us/products/portable/pocket_drives/

I can't say anything about the performance nor anything else, because I've never managed to get Debian nor Gentoo running on the nslu2.
The last try with the Debian Installer RC2 failed, so I decided to give the nslu2 away, but I keep the pocket HD ;-)

Debian on flash > 4GB

>So I decided to use a 5 GB Seagate Pocket HD
>...
>The last try with the Debian Installer RC2 failed, so I decided to give the nslu2 away, but I keep the pocket HD ;-)

BTW: Just found out with an 8GB memory stick that apparently the slug doesn't like storage access to places > 4GB. Maybe that was the reason for your problem. Did it crash while formatting, killing the SSH connection?

Just leave the last GB blank.

NSLU2 Debian install

Hi,

if it failed during the (ext3) formatting, this is a known problem. You're running out of memory. Format the USB stick manually on a PC, then choose "Don't format, keep old data" (or so) in the Debian installer...

That worked for me.

the Problem is, that I can't

the Problem is, that I can't access the nslu2 after flashing the debian installier on the nslu2. I never get access nor after 20min of waiting.

ssh?

Which IP do you try to connect to via ssh? If your NSLU2 had the default configuration before the install, you need ssh installer@192.178.1.77. If it used DHCP before, the installer will use DHCP, too, so you need to find out which IP address it got via DHCP...

yes but...

I tried with the IP I configured for gentoo installation no luck.
I flashed the linksys firmware after that I flashed the debian installer and tried with the default ip, no luck.
And finally I tried with a DHCP Server, no luck.

Serial port

As I understand you have already given up, but adding a serial port
probably would have told you what was wrong:
http://www.nslu2-linux.org/wiki/HowTo/AddASerialPort

When I first installed NSLU2 it hung waiting because it had no DNS server (seems safer to use DHCP), added DNS via console (serial port). Another time it hung at "setting the system clock again". I removed the battery and reinserted it after that it booted normally.

I am very satisfied with them. I run two NSLU2 at different locations with one mirroring the other with rsync over ssh every night, see:
http://www.eit.se/hb/misc/projects/slug/2/

I know, "no hard drives"

but I don't know if you're aware of the CompactFlash form-factor nor the pocket hard drives. I've not personally experienced them so I can't vouch for their quietness (or lack thereof), but they are probably what you're looking for if you don't want a flash-based solution:

http://www.seagate.com/www/en-us/products/portable/compact_flash/
http://www.seagate.com/www/en-us/products/portable/pocket_drives/

OTOH, I've heard that Flash is overcoming its write-limiting problem, and intelligent algos should minimize the impact, so flash isn't a so bad an option either.

[Disclaimer: I have interned and will again intern for Seagate]

Pocket hard drives

I'm not sure about pocket hard drives. Is there any reliable data on how long they last (under heavy load, 24/7 usage, over several month at least)?

Yep, flash devices nowadays have some mechanisms in hardware and their firmware to minimize the impact, but is that enough to make them usable for at least a few years in 24/7 mode? That's my main concern. I'm pretty sure they'll do just fine for accasional usage, but how do they perform in a server scenario?

Seagate Pocket Drives

Well, today broke my second seagate 8Gb Pocketdrive.
It works for exactly 2 months 24/7 on the slug, then it'll die. :)

DON'T BUY THEM! :)

Pocket drives

Hi Uwe.

I can't speak for the pocket drives (especially not seagate ones), but as mentioned before, I can't hear the microdrive I'm using (which should be about the same as the seagate CF-format HDs/pocket drives).
I previously had a "normal" compact flash on my WRAP board (a SanDisk Ultra type 2GB CF card), which worked nicely for 2 years with syslog running, which meant at least one write every 5 minutes to /var/log (and I didn't have noatime set on the mounts, accidently). No single write or read failure.
As for the microdrive: They are heavily utilized in the professional digital photo area, and a friend of mine uses his since approx. 3 years ago, daily for at least 2-3 hours of taking photos and loading them to his computer. I do suspect that this is actually more and especially heavier use than your disk would see.

Hope this helped a bit.

Sven

I'm using 2.5" notebook

I'm using 2.5" notebook drives in external USB enclosures.

If you install sg3-utils ("apt-get install sg3-utils") then you can spin down the drives when not in use.
This makes then absolutely silent, and saves power too.

Issue the command "sg-start /dev/sda 0" to spin down (or /dev/sdb or sdc, etc..) Disks will spin up automatically on disk access, so you can put this command in cron to run every half hour or so to re-spin down.