Uwe Hermann | A slightly paranoid Debian developer

I recently wanted to buy some MP3 files from Amazon (a whole album in my case, but you can also just buy single MP3 files if you want). Digital music downloads from Amazon are often much cheaper than buying the physical CD (from Amazon), and you can also instantly get the stuff within seconds, without having to wait for the physical CD to be shipped to your place.

The good thing about Amazon's MP3 downloads is that the files are not infested with any DRM-crap (if that were the case I wouldn't spend a single penny on such useless junk, of course). This allows you to burn the MP3 files on CDs and/or play them on any device you like (MP3 player of choice, laptop, hifi-system, car, e-book reader with MP3 playback support, etc. etc).

Granted, you can not re-sell the digital files on eBay later, this is the one little drawback you have when compared to physical CDs, but I guess most people can usually live with that. Also, it would be great if Amazon would provide Ogg Vorbis files instead (or in addition to) MP3 files, of course.

Anyway, in order to download the MP3 files you buy from Amazon, they suggest to install the Amazon MP3 Downloader, which (surprisingly) is even available in a Mac and Linux version (only 32-bit though), but is (unsurprisingly) closed-source. This is no-go, of course, but luckily there is an alternative.

The clamz tool (GPL, version 3 or later) allows you to easily download single Amazon MP3 files, or whole albums. First, you need to login to your Amazon account and then visit a certain Amazon page (which sets a special "congratulations, the Amazon MP3 Downloader has been successfully installed" cookie in your browser). See the clamz website for the respective URL for your country. For Germany, use this URL.

The clamz installation is easy enough on Debian:

  $ apt-get install clamz

IMPORTANT: It seems you need at least version 0.5 for recent Amazon files as they apparently changed something, see #647043. Current Debian unstable as of today already has 0.5, though.

After that is done, the rest is easy: In Amazon, click on "Buy MP3" or "Buy MP3 album", which will download a special AmazonMP3-1234567890.amz file. You can then let clamz download all the MP3s by typing:

  $ clamz AmazonMP3-1234567890.amz

Wait a few minutes, and you'll have a bunch of non-DRM MP3 files in your current directory. Easy.

See the manpage for a bunch of options which let you configure clamz to your preferences.

Yep, so I bought a new laptop recently, my IBM/Lenovo Thinkpad T40p was slowly getting really unbearably sloooow (Celeron 1.5 GHz, 2 GB RAM max). After comparing some models I set out to buy a certain laptop in a local store, which they didn't have in stock, so I spontaneously got another model, the HP Pavilion dv7-3127eg (HP product number VY554EA).

Why this one? Well, the killer feature for me was that it has two SATA disks, hence allows me to run a RAID-1 in my laptop. This allows me to sleep better at night, knowing that the next dying disk will not necessarily lead to data loss (yes, I do still perform regular backups, of course).

Other pros: Much faster than the old notebook, this one is an AMD Turion II Dual-Core Mobile M520 at 2.3 GHz per core, it has 4 GB RAM (8 GB max), and uses an AMD RS780 / SB700 chipset which is supported by the Free-Software / Open-Source BIOS / firmware project coreboot, so this might make the laptop a good coreboot-target on the long run. I'll probably start working on that when I'm willing to open / dissect it or when the warranty expires, whichever happens first.

Anyway, I set up a page at randomprojects.org which contains lots more details about using Linux on this laptop:

http://randomprojects.org/wiki/HP_Pavilion_dv7-3127eg

Most of the hardware is supported out of the box, though I haven't yet tested everything. There may be issues with suspend-to-disk / suspend-to-RAM, sometimes it seems to hang (may be just a simple config change is needed in /etc/hibernate/disk.cfg).

Cons: Pretty big and heavy (but that's OK, I use it mostly as "semi-mobile desktop replacement"), glossy screen, loud fans (probably due to the two disks).

For reference, here's an lspci of the box:

  $ lspci -tvnn
  -[0000:00]-+-00.0  Advanced Micro Devices [AMD] RS780 Host Bridge Alternate [1022:9601]
           +-02.0-[01]--+-00.0  ATI Technologies Inc M96 [Mobility Radeon HD 4650] [1002:9480]
           |            \-00.1  ATI Technologies Inc RV710/730 [1002:aa38]
           +-04.0-[02-07]--
           +-05.0-[08]----00.0  Atheros Communications Inc. AR9285 Wireless Network Adapter (PCI-Express) [168c:002b]
           +-06.0-[09]----00.0  Realtek Semiconductor Co., Ltd. RTL8111/8168B PCI Express Gigabit Ethernet controller [10ec:8168]
           +-0a.0-[0a]--
           +-11.0  ATI Technologies Inc SB700/SB800 SATA Controller [AHCI mode] [1002:4391]
           +-12.0  ATI Technologies Inc SB700/SB800 USB OHCI0 Controller [1002:4397]
           +-12.1  ATI Technologies Inc SB700 USB OHCI1 Controller [1002:4398]
           +-12.2  ATI Technologies Inc SB700/SB800 USB EHCI Controller [1002:4396]
           +-13.0  ATI Technologies Inc SB700/SB800 USB OHCI0 Controller [1002:4397]
           +-13.1  ATI Technologies Inc SB700 USB OHCI1 Controller [1002:4398]
           +-13.2  ATI Technologies Inc SB700/SB800 USB EHCI Controller [1002:4396]
           +-14.0  ATI Technologies Inc SBx00 SMBus Controller [1002:4385]
           +-14.2  ATI Technologies Inc SBx00 Azalia (Intel HDA) [1002:4383]
           +-14.3  ATI Technologies Inc SB700/SB800 LPC host controller [1002:439d]
           +-14.4-[0b]--
           +-18.0  Advanced Micro Devices [AMD] K10 [Opteron, Athlon64, Sempron] HyperTransport Configuration [1022:1200]
           +-18.1  Advanced Micro Devices [AMD] K10 [Opteron, Athlon64, Sempron] Address Map [1022:1201]
           +-18.2  Advanced Micro Devices [AMD] K10 [Opteron, Athlon64, Sempron] DRAM Controller [1022:1202]
           +-18.3  Advanced Micro Devices [AMD] K10 [Opteron, Athlon64, Sempron] Miscellaneous Control [1022:1203]
           \-18.4  Advanced Micro Devices [AMD] K10 [Opteron, Athlon64, Sempron] Link Control [1022:1204]

Full lspci -vvvxxxxnnn, lsusb -vvv, and a much more detailed list of tested hardware components is available in the wiki.

I've been buying quite a lot of (usually cheapo) gadgets recently, which I'll probably introduce / review in various blog posts sooner or later. Let me start with a fun little gadget, a digital USB-based microscope. I found out about it via this thread over at lostscrews.com.

You can get this (or a very similar device) e.g. on eBay for roughly 50 Euros. Mine seems to be from a company called Oasis (though they're probably just the reseller, not sure). The device doesn't seem to have a nice name, but I can see UMO19 MCU003 on the microscope, so I guess that's the name or model number.

It can focus on magnifications of 20x or 400x. The image resolution is said to be a max. of 1600x1200, but in practice most of my images are 640x480, maybe I have to change some settings and/or the resolution depends on the magnification factor and lighting conditions.

The device acts as a simple UVC webcam when attached to USB, so you can view the images easily via any compatible webcam software, e.g. luvcview and also save screenshots of the magnified areas (see images).

First three from left to right: SMD LED (400x), clothes/jacket (400x), random PCB (20x). The other two below: A via on a PCB (400x), and the "pixels" of a TFT screen (400x).

It worked out of the box on Linux for me, the uvcvideo kernel driver was loaded automatically.

 $ lsusb
 Bus 001 Device 013: ID 0ac8:3610 Z-Star Microelectronics Corp.

I set up a wiki page for more details (including full lsusb -vvv) and sample images at:

   http://randomprojects.org/wiki/Oasis_UMO19_MCU003_USB_microscope

I will also post some more images there over the next few days.

This is a really fun device for having a look at stuff you'd normally not see (or not well enough), and also useful for e.g. checking PCB solder joints, checking all kinds of electronics for errors or missing/misaligned parts, finding the chip name / model number of very tiny chips etc. etc. I can also imagine it's quite nice for biological use-cases, e.g. for studying insects, tissue, plants, and so on.

Anyway, definately a nice toy for relatively low price, I can highly recommend a device like this. Check eBay (search for e.g. "usb mikroskop 400") and various online shops for similar devices, there seem to be a large number of them with different names and from different vendors. Just make sure it has at least 400x magnification, there are also some with only 80x or 200x which is not as useful as 400x, of course.

This is what I set up for backups recently using a cheap USB-enclosure which can house 2 SATA disks and shows them as 2 USB mass-storage devices to my system (using only one USB cable). Without any further introduction, here goes the HOWTO:

First, create one big partition on each of the two disks (/dev/sdc and /dev/sdd in my case) of the exact same size. The cfdisk details are omitted here.

  $ cfdisk /dev/sdc
  $ cfdisk /dev/sdd

Then, create a new RAID array using the mdadm utility:

  $ mdadm --create /dev/md0 --level=1 --raid-devices=2 /dev/sdc1 /dev/sdd1

The array is named md0, consists of the two devices (--raid-devices=2) /dev/sdc1 and /dev/sdd1, and it's a RAID-1 array, i.e. data is simply mirrored on both disks so if one of them fails you don't lose data (--level=1). After this has been done the array will be synchronized so that both disks contain the same data (this process will take a long time). You can watch the current status via:

  $ cat /proc/mdstat
  Personalities : [raid1]
  md0 : active raid1 sdd1[1] sdc1[0]
        1465135869 blocks super 1.1 [2/2] [UU]
        [>....................]  resync =  0.0% (70016/1465135869) finish=2440.6min speed=10002K/sec
  unused devices: 

Some more info is also available from mdadm:

  $ mdadm --detail --scan
  ARRAY /dev/md0 metadata=1.01 name=foobar:0 UUID=1234578:1234578:1234578:1234578

  $ mdadm --detail /dev/md0
  /dev/md0:
          Version : 1.01
    Creation Time : Sat Feb  6 23:58:51 2010
       Raid Level : raid1
       Array Size : 1465135869 (1397.26 GiB 1500.30 GB)
    Used Dev Size : 1465135869 (1397.26 GiB 1500.30 GB)
     Raid Devices : 2
    Total Devices : 2
      Persistence : Superblock is persistent
      Update Time : Sun Feb  7 00:03:21 2010
            State : active, resyncing
   Active Devices : 2
  Working Devices : 2
   Failed Devices : 0
    Spare Devices : 0
   Rebuild Status : 0% complete
             Name : foobar:0  (local to host foobar)
             UUID : 1234578:1234578:1234578:1234578
           Events : 1
      Number   Major   Minor   RaidDevice State
         0       8       33        0      active sync   /dev/sdc1
         1       8       49        1      active sync   /dev/sdd1

Next, you'll want to create a big partition on the RAID device (cfdisk details omitted)...

  $ cfdisk /dev/md0

...and then encrypt all the (future) data on the device using dm-crypt+LUKS and cryptsetup:

  $ cryptsetup --verbose --verify-passphrase luksFormat /dev/md0p1
  Enter your desired pasphrase here (twice)
  $ cryptsetup luksOpen /dev/md0p1 myraid

After opening the encrypted container with cryptsetup luksOpen you can create a filesystem on it (ext3 in my case):

  $ mkfs.ext3 -j -m 0 /dev/mapper/myraid

That's about it. In future you can access the RAID data by using the steps below.

Starting the RAID and mouting the drive:

  $ mdadm --assemble /dev/md0 /dev/sdc1 /dev/sdd1
  $ cryptsetup luksOpen /dev/md0p1 myraid
  $ mount -t ext3 /dev/mapper/myraid /mnt

Shutting down the RAID:

  $ umount /mnt
  $ cryptsetup luksClose myraid
  $ mdadm --stop /dev/md0

That's all. Performance is shitty due to all the data being shoved out over one USB cable (and USB itself being too slow for these amounts of data), but I don't care too much about that as this setup is meant for backups, not performance-critical stuff.

Update 04/2011: Thanks to Bohdan Zograf there's a Belorussian translation of this article now!

I've been using CRM114 as spam filter for a while now, and I'm quite happy with it. Due to bug #529720 though (incompatible upstream file format changes) I decided to start my setup from scratch with a recent CRM114 version from unstable. Here's a short HOWTO, hope it's useful for others.

First you need to install crm114 and set up a few files in your $HOME directory.

  $ sudo apt-get install crm114
  $ mkdir ~/.crm114
  $ cd ~/.crm114
  $ cp /usr/share/doc/crm114/examples/mailfilter.cf.gz .
  $ gunzip mailfilter.cf.gz
  $ cp /usr/share/crm114/mailtrainer.crm .
  $ touch rewrites.mfp priolist.mfp

Edit ~/.crm114/mailfilter.cf and set the following variables (some are optional, but that's what I currently use):

  :spw: /mypassword/
  :add_verbose_stats: /no/
  :add_extra_stuff: /no/
  :rewrites_enabled: /no/
  :spam_flag_subject_string: //
  :unsure_flag_subject_string: //
  :log_to_allmail.txt: /no/

The :log_to_allmail.txt: /no/ option should probably stay at "yes" for the first few days until you have tested your setup and everything works OK. The ~/.crm114/allmail.txt file will contain all your mails, in case something goes wrong.

Now set up empty spam and nonspam files like this:

  $ cssutil -b -r spam.css
  $ cssutil -b -r nonspam.css

Test the setup by invoking mailreaver.crm as follows, typing some test text and then pressing CTRL+d:

  $ /usr/share/crm114/mailreaver.crm -u ~/.crm114
  test
  [CTRL-d]
  ** ACCEPT: CRM114 PASS osb unique microgroom Matcher **
  CLASSIFY fails; success probability: 0.5000  pR: 0.0000
  Best match to file #0 (nonspam.css) prob: 0.5000  pR: 0.0000
  Total features in input file: 8
  #0 (nonspam.css): features: 1, hits: 0, prob: 5.00e-01, pR:   0.00
  #1 (spam.css): features: 1, hits: 0, prob: 5.00e-01, pR:   0.00
  X-CRM114-Version: 200904023-BlameSteveJobs ( TRE 0.7.6 (BSD) ) MF-35EB8B9A [pR: 0.0000]
  X-CRM114-CacheID: sfid-20090920_151224_574131_D290E589
  X-CRM114-Status: UNSURE (0.0000) This message is 'unsure'; please train it!

The output should look similar to the above. If there are errors instead, you should check your settings in ~/.crm114/mailfilter.cf.

Now you have to setup a procmail rule for crm114:

  :0fw: crm114.lock
  | /usr/share/crm114/mailreaver.crm -u /home/uwe/.crm114

  :0:
  * ^X-CRM114-Status: SPAM.*
  IN.spam-crm114

In my case this rule is also followed by a spamassassin rule, so all my mail goes through two different spam filters (will look into dspam and bogofilter also I guess, the more the better).

Finally, in .muttrc I have the following configs so I can press SHIFT+x to mark a mail as spam, and SHIFT+h to mark it as non-spam (ham).

macro index X '| formail -I X-CRM114-Status -I X-CRM114-Action -I X-CRM114-Version | /usr/share/crm114/mailreaver.crm -u /home/uwe/.crm114/ --spam'
macro index H '| formail -I X-CRM114-Status -I X-CRM114-Action -I X-CRM114-Version | /usr/share/crm114/mailreaver.crm -u /home/uwe/.crm114/ --good'
macro pager X '| formail -I X-CRM114-Status -I X-CRM114-Action -I X-CRM114-Version | /usr/share/crm114/mailreaver.crm -u /home/uwe/.crm114/ --spam'
macro pager H '| formail -I X-CRM114-Status -I X-CRM114-Action -I X-CRM114-Version | /usr/share/crm114/mailreaver.crm -u /home/uwe/.crm114/ --good'

Important: crm114 is most effective if you start with empty CSS files (as shown above) and only train it by marking mails as spam/ham when it gets them wrong. The process will take a few hours or maybe a day (depending on how many mails per day you get), then the misclassification rate gets very low...

Update 2009-09-23: Changed --spam/--nonspam to the correct options for mailreaver/mailtrainer, --spam/--good.