TSM and removable USB disks

 

IBMs Tivoli Storage Manager is one of the best and worst backup systems I have ever used. It’s kinda of like using a chainsaw to cut butter – wrong but just soo good. The good side is that you can make TSM do just about everything. The bad side is that you can make TSM do just about everything!

A recent experiment I did was to use USB hard drives as removable media instead of traditional tapes – meaning many gigabytes or storage with fast access times for a low cost. The downside is that you need to swap media in and out manually.

In this guide, I’m not going to go into the exact details on how to configure TSM from scratch. I’ll skip the part of installing the server and get straight into the non-obvious parts of using the USB storage devices as removable media.

So first, what we have to do is set up a library for our USB HDDs to sit in:

define library usbdrives libtype=manual autolabel=yes

We’ve now defined the library and called it ‘usbdrives’. We want it to be manually operated – as I’m yet to see an autochanger for hard disks! 🙂

We now need to define a drive in the library for TSM to write to:
define drive usbdrives usbdrive

Now we need a path between the library and the drive:
define path SERVER1 usbdrive srctype=server desttype=drive library=usbdrives device=/mnt/usb

This tells TSM that we will mount our USB device at /mnt/usb. You can change this for what you want.

Then TSM needs a device class to know how to handle the USB drives:
define devclass usbdrives devtype=removablefile library=usbdrives mountwait=2880 mountretention=1

This is getting a little more in depth, so I’ll explain a little more. We want to define the device class to be part of the usbdrives library. The device type needs to be a removable file. The mountwait value tells TSM how long to wait for a volume to be available – I make this 2880 minutes – which is 2 days. If you don’t get the drive mounted in 2 days, TSM will abort the process. I also set mountretention to 1 minute. This means that after 1 minute of TSM finishing with the USB drive, it will automatically unmount it.

We’re almost there now as far as TSM is concerned.

We now need to configure a storage pool to group together our handywork. I assume at this point that you have 5 USB hard disks you want to use – substitute this for how many you actually have. If you don’t know, set 1 or 2 – as it can always be updated later on.

define stgpool usbdrives usbdrives maxscratch=5

TSM is now configured – but we still have to prepare the USB drives!

TSM is quite clever when it comes to using drives as storage media. Insert your USB drive and let it spin up and be detected by your OS. In my case here, I’ll use Linux as an example. If you’re using a different OS, you’ll need to substitute the following into how you’d do it on your OS. That being said, this has been untested on Windows.

Create a single partition on your new drive using fdisk. In my case, the USB HDD is sdk.
fdisk /dev/sdk

Save and write the new partition. You now should see something like this:

# fdisk -l /dev/sdk
 
Disk /dev/sdk: 1000.2 GB, 1000204886016 bytes
255 heads, 63 sectors/track, 121601 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
 
Device Boot Start End Blocks Id System
/dev/sdk1 1 121601 976760001 83 Linux

We now need to create a file system. IBM state that the file system MUST be ext2 – however I have tested this with ext3 and can verify that it works. I am yet to try it with a different file system like XFS. One thing to note here is that TSM will use the LABEL of the disk to identify it. It also requires that there are no spaces or periods in the label.

mkfs.ext3 -L USBVOL1 /dev/sdk1

If we continue to mount the new partition to /mnt/usb, and tell TSM to start a migration, you will notice that a file in /mnt/usb will appear with the same name as the file system label we created earlier.

All in all, the manual work is a little annoying – however the flexibility that you now get by being able to use cheap USB drives as bulk storage is great. If all is configured correctly, after a migration, you should get something like this:

tsm: SERVER1>q vol
 
Volume Name Storage Device Estimated Pct Volume
Pool Name Class Name Capacity Util Status
------------------------ ----------- ---------- --------- ----- --------
/mnt/raid/tsm/disk01.dsm DISKBACK DISK 10.2 G 100.0 On-Line
/mnt/raid/tsm/disk02.dsm DISKBACK DISK 10.2 G 100.0 On-Line
/mnt/raid/tsm/disk03.dsm DISKBACK DISK 10.2 G 99.8 On-Line
/mnt/raid/tsm/disk04.dsm DISKBACK DISK 10.2 G 100.0 On-Line
5QF7TH53 USBDRIVES USBDRIVES 267.2 G 48.1 Filling
9QF57E67 USBDRIVES USBDRIVES 285.0 G 4.7 Filling

  4 Responses to “TSM and removable USB disks”

  1. Great write up….straight to the point… i am trying to do this on Windows. My only problem is Windows see this as a regular drive. I am trying to figure out how to define the library/drive path to it the external USB drive.

    I already have the library via remote site using the VTl

  2. I always get the message
    Mount REMOVABLEFILE volume SCRTCH R/W in drive USBDRIVE (/tsmdevices/sdc1) of library USBDRIVES within 2862 minutes

    the drive is known by the os as removable

  3. User running TSM server instance needs not only RW rights to the /mnt/usb filesystem, but also RW access to the device, i.e. /dev/sdk1

  4. i ended up by writing a udev rule which launches a script to mount the drive, creates a filedev, or put a filedev r/w, based on the label name.
    a remove rule, or script puts the filedev offline

 Leave a Reply

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>

(required)

(required)