Increasing the Capacity of a Single Disk System

A single disk system usually has two or more partitions on its only disk (at the very least, a swap partition and a system partition). Expanding the storage capacity of the video storage partition (or the system partition, if the videos are stored along with the system files on the system partition) is basically like expanding a single partition volume, except that there are a couple of extra steps before you get started, and you must carry out the appropriate type of copy for each of the partitions, not just a single partition.

The extra steps are necessitated by the fact that the single disk system must have a bootable disk and the system also needs some swap space on the disk.

To illustrate the complete steps necessary to expand a single disk system, let's take the example of a system that has three partitions, one for /boot, one for swap space, and one for root. Furthermore, let's assume that the /boot partition is formatted using ext3 and the root partition is formatted using XFS.

As we noted in the previous section, the duration of the copy operation that is used to expand a volume depends on the amount of data on it (duh) so you may wish to clean up all of the old stuff that you were thinking of deleting, before you proceed. On the other hand, if you could do that, you probably wouldn't need a bigger disk in the first place....

Again, we begin by getting ourselves a copy of Knoppix (the CD works just swell but later versions will only fit on a DVD):

     http://knopper.net/knoppix/index-en.html

And burning the ISO image onto a CD (or DVD, if you took that route).

Next, configure the single disk machine who's disk is to be expanded with the old disk and new disk attached and a CD/DVD drive added, if necessary. In this example, we're presuming that the original drive is on SATA0 and the new drive is on SATA1, which should map to /dev/sda and /dev/sdb respectively, when Knoppix is booted.

Put the Knoppix CD/DVD into the CD/DVD drive and boot it. Once Knoppix comes up (you're booting from a CD/DVD, remember, so be patient), open up a console window and become super user. To do this, simply type the "su" command. There's no password (how refreshing).

Before proceeding, you should check that the original and new drives are installed where you expect them to be. You can do this by typing:

     /sbin/fdisk -lu

The first drive should show a valid partition table that looks something like this:

     Disk /dev/sda: 640.1 GB, 640135028736 bytes
     255 heads, 63 sectors/track, 77825 cylinders, total 1250263728 sectors
     Units = sectors of 1 * 512 = 512 bytes
     Disk identifier: 0x63c6c105
     Device Boot      Start         End      Blocks   Id  System
  /dev/sda1   *          63      996029      497983+  83  Linux
  /dev/sda2          996030     4996214     2000092+  82  Linux swap / Solaris
  /dev/sda3         4996215  1250263728   622633756   83  Linux

Be sure that the original drive's size and partition information is correct to ensure that it is mounted where you think it is. Also verify that the new drive has nothing on it (or has the expected layout, if you're reusing it). Since, we're going to be wiping out all data on the new drive in a minute, it pays to make sure you've got the right target in your sights.

You can also check the various partitions to see what type of file systems are on each of them. In our example, we'd do:

     /bin/lsblk -f

Alternately, blkid will also tell you which type of file system a block device has mounted, so you can use:

     /sbin/blkid /dev/sda1
     /sbin/blkid /dev/sda3

If you are checking file system types on mounted file systems (e.g. on a system that is in service), you can use:

     df -T /dev/sda1
     df -T /dev/sda3

Make a note of each partition's file system type. We're expecting to see an ext3 file system on /dev/sda1 and an XFS file system on /dev/sda3.

If everything looks good, you can avoid all the trouble of setting up the boot sector with the boot loader by just copying the first few tracks of the source disk directly with dd:

     dd bs=512 count=2048 if=/dev/sda of=/dev/sdb

This will also copy the partition table, which probably won't be correct for the new disk. However, since we are expanding the disk size, usually the only required change to the copied partition table is that the last partition be expanded to fit the new disk. This can be done by deleting the last partition and adding a new partition that uses all of the available space:

     su
     /sbin/fdisk /dev/sdb
       u                         (All sizes are given in sectors, not
                                   cylinders)
       d                         (Delete a partition)
       3                         (Partition 3)
       n                         (To create a new partition)
       p                         (As a primary partition)
       3                         (Partition 3)
       <cr>                      (Accept previous end sector + 1 as the start)
       <cr>                      (To accept the last sector as the end)
       p                         (To verify the partition table)
       w                         (Write the partition table to the disk)

Be sure that the appropriate partition is marked as the boot partition, if it hasn't already been marked that way (it should be).

Alternately, for the advanced format hard drives (e.g. 1.5, 2 TB, or higher drives), one can likely no longer continue in the same old manner, as described above (unless you are copying from one advanced format drive to another, in which case have at it).

When copying from a regular format hard drive (with 512 byte physical sectors) to an advanced format hard drive (with 4096 byte physical secors), you will probably need to create the partition table from scratch because, if the partitions on the drive are not laid out properly and/or the file systems not constructed carefully, you will suffer horrible performance problems. This all but mandates that you lay the partitions out by hand, set the boot loader up manually, set the file systems up ahead of time, and copy all of the files manually. No use of dd or another direct partition copy program to speed up the process is possible.

Note that, with the later versions of Knoppix that we have used (e.g. 7.2), fdisk knows about advanced format hard drives and handles alignment of the partitions on the proper boundaries, if you let it. It should start the first partition on sector 2048 and, if you use end sector numbers like +512M or +4G, calculate the proper end sectors so that the next partition will always begin on the proper boundary. We now begin by creating the partition table as follows:

     su
     /sbin/fdisk /dev/sdb
       u                         (All sizes are given in sectors, not
                                   cylinders)
       n                         (To create a new partition)
       p                         (As a primary partition)
       1                         (Partition 1)
       2048                      (Start the partition on a 4K boundary [i.e.
                                   0 mod 8])
       +512M                     (To define a 512 MB partition)
       n                         (To create a new partition)
       p                         (As a primary partition)
       2                         (Partition 2)
       <cr>                      (To accept the next block as the start)
       +2G                       (To define a 2 GB partition [or use +4G])
       t                         (To set the partition type)
       2                         (Partition 2)
       82                        (Partition type 82, swap space)
       n                         (To create a new partition)
       p                         (As a primary partition)
       3                         (Partition 3)
       <cr>                      (To accept the next block as the start)
       <cr>                      (To accept the last cylinder as the end)
       w                         (Write the partition table to the disk)

This creates a new partition layout that has approximately 500MB for the boot partition, 2GB for swap space, and the remaining disk space for the root partition. Just by way of example, if you were interested in upping the swap space to 4GB from 2GB, you might use +4G instead of +2G for the second partition.

After you've created the new partitions, you can check your work with:

     /sbin/fdisk -lu /dev/sdb

You should see something like this (in this example, we used +4G for the swap space):

     Disk /dev/sdb: 2000.0 GB, 1999988850688 bytes
     255 heads, 63 sectors/track, 243151 cylinders, total 3906228224 sectors
     Units = sectors of 1 * 512 = 512 bytes
     Sector size (logical/physical): 512 bytes / 4096 bytes
     I/O size (minimum/optimal): 512 bytes / 4096 bytes
     Disk identifier: 0x0000144d
     Device Boot      Start         End      Blocks   Id  System
  /dev/sdb1   *        2048     1050623      524288   83  Linux
  /dev/sdb2         1050624     9439231     4194304   82  Linux swap / Solaris
  /dev/sdb3         9439232  3906228223  1948394496   83  Linux

Since we laid out the partition table using sector numbers instead of cylinders, you'll probably see some whining from fdisk about the partitions not starting on a cylinder boundary. Since all of the cylinder/sector numbers have been fake for many years, and since all sector addressing is done using LBAs, it doesn't much matter where the partitions start/end, as long as they all start on a sector whose address is 0 mod 8. So, ignore the warnings from fdisk about cylinder alignment.

Incidentally, if the new disk is bigger than 2TB, you must use parted to format it because parted is the only partition editor that supports disks larger than 2TB. Additionally, if you have a disk that is larger than 2TB, it is guaranteed to be an advanced format disk, so you must make sure to choose a starting sector for each partition that observes the 4K block boundaries, to avoid severe performance hits. Also, to ensure compatibility with boot loaders that require more than 62 sectors following the partition table, it is smart to always start the first partition at sector 2048. Since parted has a parameter that automatically figures out what is the optimal alignment for a partition, based on disk geometry, we can just pick 2048 by default and it will tell us if that number won't work. So, run it like this:

     su
     /sbin/parted -a optimal /dev/sdc
       mklabel gpt
       mkpart primary ext2 2048s 512MiB
       set 1 boot on
       mkpart primary linux-swap 512MiB 4.5GiB
       mkpart primary ext2 4.5GiB 100%
       print
       quit

The print command should show something like this:

     Model: ATA WDC WD4003FZEX-00Z (scsi)
     Disk /dev/sdc: 4001GB
     Sector size (logical/physical): 512B/4096B
     Partition Table: gpt
     Number  Start   End     Size    File system     Name     Flags
      1      1049kB  512MB   511MB                   primary  boot
      2      512MB   4607MB  4096MB  linux-swap(v1)  primary  
      3      4607MB  4001GB  3996GB                  primary

We used the "s" after "2048" to tell parted that the first number for the first partition was a starting sector number, not a size in megabytes. We used the "%" after "100" in the last partition to tell parted that the ending sector number was 100% of the disk, in other words the last physical sector.

Once all of the partitions have been created, you must install the boot loader (Mythbuntu uses GRUB) on the disk. If you have a disk that is 2TB or less and you are not using a GNU Partition Table (i.e. you set the disk up with fdisk), you can avoid the trouble of setting up the boot sector with the boot loader by just copying the first few tracks of the source disk directly with dd.

If you copied the first sectors of an already-bootable disk (which you whould have done if you used "dd bs=512 count=2048 ..." in the first method shown above), you are all set, since you've already copied the loader.

If you had to set up a new partion table from scratch, in order to accomodate an advanced format hard drive, you can still copy the relevant MBR information from the original disk but, since we've already set up the partition table, we must do the copy very carefully to avoid wiping it out:

     dd bs=446 count=1 if=/dev/sda of=/dev/sdb
     dd bs=2 skip=255 seek=255 count=1 if=/dev/sda of=/dev/sdb
     dd bs=512 skip=1 seek=1 count=2047 if=/dev/sda of=/dev/sdb

This copies the first 446 bytes of the first sector (a.k.a. the MBR), which contains the boot loader code. It skips over the next 64 bytes, which contain the partition table. Then, it copies the two-byte magic cookie that defines it as the MBR. Finally, it copies the next 2047 sectors, which contain the GRUB Stage 1.5 code. Once that is done, you should have a bootable disk that is partitioned the way you want it.

You might want to test that the system can boot the disk (admittedly, it won't get far but you should at least see GRUB Stage 1.5 running), before you proceed any further, because, if it didn't work, now is the time to go with Plan B, before you copy any data to the new disk. Note that, if you used the third method outlined above and created a GNU Partition Table, you will nearly always have to use Plan B.

To recap, Plan B is used if you are using a GNU Partition Table (i.e. you set the disk up with parted), dd wiped out the partition table, or GRUB wouldn't boot into Stage 1.5. Plan B consists of installing GRUB from scratch, which is best done using the original OS installation disk, in rescue mode. To use Plan B, you must first copy the files to the /boot partition, as outlined below.

With the partition table defined the way you want it (and regardless of whether the disk is bootable), you can tell Knoppix to reread all of the partition information and define mount points for those partitions that it finds with this command:

     partprobe

The next step is to set up the boot partition. Hopefully, you have a separate boot partition on both the old and the new disks (it makes life so much easier). If so, the following works well to format the boot partition on the new disk with the right file system type and copy all of its data:

     dd bs=64M if=/dev/sda1 of=/dev/sdb1

For bonus points, this also copies the partition label from the old disk so that the GRUB configuration file need not be altered in order to get the system to boot. This is by far and away the easiest way to set up the boot partition. However, it depends on the two partitions being the same size or the new one being slightly larger. If that's not the case, you will need to format the new partition using the appropriate mkfs and then copy all of the old files over to it, after mounting the two partitions (old and new), using the copy command.

All of the ext2/ext3 file systems, as noted in the orginal disk's partitions, that were not set up with dd, must be made in the new disk's corresponding partitions with mkfs. Similarly, swap space must be initialized with mkswap. However, any XFS file systems need not be created ahead of time as their copy operation will take care of that for us. If the direct copy of the /boot partition wasn't used, here's an example of how we'd set up one ext2 or ext3 file system and one swap space, first using mkfs.ext2:

     mkfs.ext2 -L /boot -T news /dev/sdb1
     mkswap -v1 /dev/sdb2

or mkfs.ext3:

     mkfs.ext3 -j -L /boot -T news /dev/sdb1
     mkswap -v1 /dev/sdb2

Once you've done this, copy any files needed for the OS boot sequence to the boot partition. Knoppix thoughtfully provides predefined mount points in the "/media" directory for each of the attached devices that it finds (and even adds mount points for any new partitions created using fdisk when you use partprobe) so you can do:

     mount -r /dev/sda1 /media/sda1
     mount /dev/sdb1 /media/sdb1
     cp -R --preserve=all /media/sda1/* /media/sdb1
     umount /media/sda1
     umount /media/sdb1

Now is the time, if you are going with GRUB Plan B, to set up the boot loader (as noted above, if you set up the boot sector, etc. using one of the direct copy methods, you can omit this step). There are so many ways of doing this that it is beyond the scope of these notes. Hopefully, you won't have to do it this way but, if you do, there are plenty of notes on the Internet that give the details. Find them and read them first. Just to make sure that you do it correctly, the basic steps should look something like this:

  1. Boot with your system's Installation CD and get into rescue mode.
  2. If you are asked if you want to look for an existing Linux installation, you can skip this step. This will get you to the console prompt a lot quicker.
  3. Once the console prompt appears, type "grub".
  4. Set GRUB's root device to the partition containing the boot directory by typing "root (hd1,0)". In our case, this would be /dev/sdb1. But, you may not want to live dangerously and trust that GRUB (which uses a truly strange numbering scheme for boot devices) will pick the right device to whack. In that case, you could uncable your original disk and leave only the new disk cabled. Then, you'd set GRUB's root device using "root (hd0,0)".
  5. Write GRUB to the hard drive with "setup (hd1)" or "setup (hd0)".
  6. Bail out by typing "quit".

Note that you may be able to do all of this with Knoppix and skip the rescue disk, depending on how compatible your system is with the Knoppix version of GRUB.

The data on an original XFS partition should now be copied to the new drive. This is done with:

     xfs_copy -d /dev/sda3 /dev/sdb3

The copy will take quite a while, depending on the size of the original partition and how full it is (if you were smart and deleted all of the junk shows off the original partition, it will be faster). Once the copy completes, the new disk will contain an exact copy of the original disk and the new XFS file system will be same size as the original (bummer).

The next step requires that you mount the new file system, using the mount points that Knoppix provides in the "/media" directory:

     mount /dev/sdb3 /media/sdb3

You can check that the mounted file system looks like the original with:

     ls -l /media/sdb3

If you're happy with the new file system, you can increase its size to fill the entire partition (nice) with:

     xfs_growfs /media/sdb3

If, instead of an XFS partition, you have an ext2/ext3 partition, you can copy it with dd as follows:

     dd bs=64M if=/dev/sda3 of=/dev/sdb3

We know that we didn't discuss the direct copying of the entire old disk with dd but some people might try it, since it frequently works. If you did this and just copied the whole hard disk with dd, and the ext2/ext3 partition that you're expanding is the last on the drive, you can expand it, after the copy, like this:

     fdisk /dev/sdb
       u                                   (All sizes are given in sectors, not
                                             cylinders)
       d                                   (Delete a partition)
       3                                   (Partition 3)
       n                                   (To create a new partition)
       p                                   (As a primary partition)
       3                                   (Partition 3)
       <cr>                                (Accept prev end sector + 1 as start)
       <cr>                                (Accept last sector as the end)
       p                                   (To verify the partition table)
       w                                   (Write partition table to the disk)
     partprobe

Since none of the data in the old partition is touched when the partition is deleted or put back, it is all still there when the new partition is created.

If you wish, you can check your work:

     mount /dev/sdb3 /media/sdb3
     ls                                    (look around to see if it is OK)
     umount /dev/sdb3 /media/sdb3

All that remains is to expand the size of the file system to fill the new, bigger partition:

     fsck -n /dev/sdb3                     (only for ext3
     tune2fs -O^has_journal /dev/sdb3        file systems)
     e2fsck -f /dev/sdb3                   (nothing works without it)
     resize2fs /dev/sdb3                   (resize the file system to fit the
                                             partition)
     fsck -n /dev/sdb3                     (again? yes again)
     tune2fs -j /dev/sdb3                  (only for ext3 file systems)
     tune2fs -c 0 -i 0 /dev/sdb3           (turn off auto fsck)
     sync

If GRUB was set up to boot using the UUID of the boot partition, you may need to alter the GRUB configuration file on the /boot partition and then reinstall GRUB, if you created a new /boot partition, since its UUID will have changed. See the Recovering GRUB section below.

If your system is set up to mount partitions using UUIDs, you may have to alter /etc/fstab and change the UUID of the file system to mount on "/", depending on how you constructed and copied it.

You can find the UUID of the partition in question, in either case, using:

     /sbin/blkid /dev/sdb3

You can use vim under Knoppix, or it also has a visual editor called leaf. After mounting the appropriate partition, you can run leaf from a root console window, so that you can edit all files, like this:

     leaf &

Once that's done, unmount the new file system:

     umount /media/sdb3

Shut down Knoppix and recable the new disk to replace the original. Boot the machine with the new disk installed and see how it works. If your system will not boot, probably because the UUID of the new disk does not match the UUID in GRUB's boot configuration, see the Recovering GRUB section for help with how to recover a bootable system.