Shared Video Server

You can store videos on any network-attached server that has lots of room, a high-speed network connection and a processor that will be able to keep up with the load of serving packets to the frontends. Typical DVDs average between 4 and 8 gigabytes so you can figure how big the server's disks need to be to hold your collection. For a 2TB drive, you can expect between 200 and 440 DVDs. A nicely-sized video server can be built from four 2TB drives. Even better would be a video server with four 4TB drives, which will let you hold all of your DVDs plus all the episodes of your favorite television shows.

You need to run a recent version of Samba on the server so if you don't have a copy, install it now. Older versions of Samba only support SMBFS, whereas newer versions support CIFS, which is much better. So, although you can mount the older Samba share on Mythbuntu, using smbfs, you may get unexpected I/O errors and other such problems that can cause you grief. Using a newer version of Samba and CIFS should help.

A little farther along in this section, we'll illustrate how to set up a video directory (or high-level directory with mount points for multiple disks) on your video server. The goal is to have a single, top level directory that can be exposed to the outside world as one mount point. You can have multiple directories below it, that will be handled by the MythVideo setup.

In the meantime, let's deal with adding disks to your video server. If you are starting from scratch, or have run out of space and want to add another drive, install the drive in the server and then, if it is 2TB or smaller, format it using fdisk:

     su
     /sbin/fdisk -u /dev/sdc
       n                         (To create a new partition)
       p                         (As a primary partition)
       1                         (Partition 1)
       <cr>                      (To accept the first sector as the start)
       <cr>                      (To accept the last sector as the end)
       w                         (Write the partition table to the disk)

You can check your work with:

     /sbin/fdisk -lu /dev/sdc

You should see something like this:

     Disk /dev/sdc: 2000.4 GB, 2000398934016 bytes
     255 heads, 63 sectors/track, 243201 cylinders, total 3907029168 sectors
     Units = sectors of 1 * 512 = 512 bytes
     Device Boot      Start         End      Blocks   Id  System
  /dev/sdc1              64  3907029167  1953514552   83  Linux

If you have one of the advanced format hard drives (e.g. the newer 1.5 or 2 TB drives from Western Digital or Samsung), you need to pay particular attention to how you create the partitions on the drive so that they are always aligned on a 4K boundary. If you don't do this, performance will suffer heavily.

When creating a single partition, earlier versions of fdisk defaulted to creating the partition starting at sector 63 (they kept the first 63 sectors, from 0-62, for the partiton table and boot information). In the case of advanced format hard drives, this is a very unfortunate choice. Luckily for us, for an advanced format hard drive, later versions of fdisk default to creating the partition starting at sector 2048 (it keeps the first 2047 sectors, from 0-2047, for the partiton table and expanded boot information). But, if you'd like to override the default choice (we don't advise it), you can partition the disk something like this:

     su
     /sbin/fdisk -u /dev/sdc
       n                         (To create a new partition)
       p                         (As a primary partition)
       1                         (Partition 1)
       64                        (Start the partition on a 4K boundary [i.e. 0
                                   mod 8].
       <cr>                      (To accept the last sector as the end)
       w                         (Write the partition table to the disk)

As we said, later versions of fdisk start the first partition at sector 2048, to allow room for the larger boot loaders that are installed by GRUB. If that's the case for your version of fdisk, there is no need to override the starting sector number, just take the default.

You can check your work with:

     /sbin/fdisk -lu /dev/sdc

You should see something like this:

     Disk /dev/sdc: 2000.4 GB, 2000398934016 bytes
     255 heads, 63 sectors/track, 243201 cylinders, total 3907029168 sectors
     Units = sectors of 1 * 512 = 512 bytes
     Device Boot      Start         End      Blocks   Id  System
  /dev/sdc1              64  3907029167  1953514552   83  Linux

If the new disk is bigger than 2TB, you must use parted to format the new disk because it is the only partition editor that is able to support disks larger than 2TB. Additionally, if you have a disk that is larger than 2TB, it is guaranteed to be an advanced format disk, so you must make sure to choose a starting sector for the partition that observes the 4K block boundary, to avoid severe performance hits. Also, to ensure compatibility with boot loaders that require more than 62 sectors following the partition table, it is smart to always start the first partition at sector 2048. Since parted has a parameter that automatically figures out what is the optimal alignment for a partition, based on disk geometry, we can just pick 2048 by default and it will tell us if that number won't work. So, run it like this:

     su
     /sbin/parted -a optimal /dev/sdc
       mklabel gpt
       mkpart primary ext2 2048s 100%
       print
       quit

The print command should show something like this:

     Model: ATA WDC WD40EZRZ-00G (scsi)
     Disk /dev/sdc: 4001GB
     Sector size (logical/physical): 512B/4096B
     Partition Table: gpt
     Number  Start   End     Size    File system  Name     Flags
      1      1049kB  4001GB  4001GB               primary

We used the "s" after "2048" to tell parted that the first number was a starting sector number, not a size in megabytes. We used the "%" after "100" to tell parted that the ending number was 100% of the disk, in other words the last physical sector.

The next step is to make the file system that you wish to use for the partition, with mkfs. Usually for a video server, you should pick an ext3 file system and use mkfs.ext3. Note that for advanced format drives, it is extremely important, regardless of which file system you choose, that the block size chosen for the file system is a multiple of 4096. If not, there will be a severe performance hit. Since we are going to be using this drive to store a few large video files, there is no real penalty to always use 4096-byte blocks so why not just get into the habit of always using them. Thus, for mkfs.ext3, we'll always use:

     /sbin/mkfs.ext3 -b 4096 -j -L / -T news /dev/sdc1

In this example, we labeled the file system "/". You may wish to use some other label, chosen to match what will be on the drive. Be especially careful about this if your system boots with the "LABEL=" parameter in the GRUB configuration and/or mounts partitions using the "LABEL=" parameter in /etc/fstab.

Since the file system will only hold static data, there's no real reason to have fsck check it at boot time. This is especially relevant because for the large file systems that are found on 1 or 2TB disks, checking the file system could take a very, very long time (i.e. hours and hours). If you have several large disks on your video server and all of the file systems happen to get coincidentally checked, booting your system could easily take half a day. You will most certainly want to turn off this dubious feature with:

     /sbin/tune2fs -c 0 -i 0 /dev/sdc1

If you are into housekeeping, you may want to delete the superfluous lost+found directory:

     rm -rf /DVD/sdc/lost+found

To hook the new disk up to your video server, add another subdirectory to the high-level directory that contains the mount points for the multiple disks on the server:

     mkdir /DVD/sdc

In simpler times, it was merely a matter of editing /etc/fstab and adding a line to mount /dev/sdc to the /DVD/sdc mount point. For example:

     /dev/sdc1  /DVD/sdc  ext3  0  0

However, it has now become a case of the inmates truely being in charge of the asylum.

If you are using an HBA to attach disks (which is a common approach for video servers), as well as the ATA ports on the system's motherboard, it can lead to the names assigned to the disks being inconsistent from one boot to the next. This makes it particularly fun when it comes to mounting the disks. And, better yet, there's no way to fix this stupidity. You just have to take what they give you.

Instead of using actual device names, the fstab needs to mount the disks by UUID. The UUID in question is assigned when the partitions are built so you need to obtain the UUID after the disk is partitioned and formatted. Assuming that your video server has more than one or two disks (it is a video server, after all), you can figure out what the UUIDs of all of the disks are with:

     su
     /sbin/blkid /dev/sdxx1
     /sbin/blkid /dev/sdxx2
     /sbin/blkid /dev/sdxx3
          .
          .
          .

But, before you do this, make sure that the disk you are asking the UUID for is really the disk that you mean to mount. You should copy down the physical serial number from the disk's external label, when you install the disk, and then look at the disk information for the device that you suspect is the disk in question. If the serial number you copied down is the same, you can proceed. Get the disk info this way:

     su
     /usr/bin/udisks --show-info /dev/sdxx

And, speaking of the inmates running the asylum, if your video server is running under CentOS 7.x, the udisks command has been removed and you must now use the following command, provided by the udisks2 package:

     su
     /usr/bin/udisksctl info -b /dev/sdxx

However, in their infinite wisdom, the smartest people in the world have decided to leave out such identifying information as the device serial numbers from the output produced by udisksctl, so you may have to resort to:

     su
     /usr/sbin/smartcl -a /dev/sdxx

It may be of assistance to you to note that the disks plugged into HBA ports are assigned disk names first (e.g. /dev/sda, /dev/sdb, etc.), before the disks plugged into the motherboard ATA ports. Also, it appears that for most HBAs (which implement SAS/SATA ports with a 4-port controller chip), the disk names are assigned in ascending order in groups of four, to the disks plugged into Port 3, Port 2, Port 1, Port 0, then Port 7, Port 6, Port 5, and Port 4, etc. So, Port 3 gets the name /dev/sda, Port 2 gets /dev/sdb, ..., Port 7 gets /dev/sde, Port 6 gets /dev/sdf, etc. But, only if there is something actually plugged into the port.

Now, we create a shared directory, which is then exported by Samba so that it can be mounted by any system wishing to see the videos. In this example we will call the directory /dvdlib:

     su
     mkdir /dvdlib
     mkdir /dvdlib/posters
     mkdir /dvdlib/DVD
     mkdir /dvdlib/DVD/Movies
     mkdir /dvdlib/mnt
     chgrp joeblow /dvdlib/
     chmod ug=rwx,o=rx /dvdlib/
     chgrp joeblow /dvdlib/DVD/
     chmod ug=rwx,o=rx /dvdlib/DVD/

Continuing with this example, the /dvdlib/DVD/Movies subdirectory is actually on the boot disk (third partition) and is used to hold any videos (i.e. movies) that are stored on the boot disk itself. Similarly, the posters directory is used to hold poster artwork for the movies and TV shows served by the video server.

The partitions on the boot disk, (either plugged into a motherboard ATA port or connected to a RAID controller), are mounted in the file system in the usual spots (i.e. "/" and "/boot"). But, for all of the other disks plugged into the motherboard, as well as any disks on an HBA, mount points must be created somewhere in the file system for them. Since they must be accessible by any system that wishes to access the videos, they must be exported as part of the /dvdlib tree so we'll hang them off the /dvdlib/mnt subdirectory. We'll give them names that indicate their port numbers:

     su
     mkdir /dvdlib/mnt/hba0p0
     mkdir /dvdlib/mnt/hba0p1
     mkdir /dvdlib/mnt/hba0p2
     mkdir /dvdlib/mnt/hba0p3
     mkdir /dvdlib/mnt/hba0p4

Once you have the UUIDs for all of the disks/partitions that you want to mount, you can update /etc/fstab, which should look something like this:

     #
     # /etc/fstab
     #
     # Originally updated by J. Blow on Fri Jul 4 2014, 11:22:03
     # Updated for this install by J. Blow on Tue Nov 3 2015, 08:30:45
     #
     # Accessible filesystems, by reference, are maintained under '/dev/disk'
     # See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info
     #
     # The first SATA disk, on the motherboard.  This is the boot disk.
     UUID=fdbc0fb2-ff6a-f1c1-bbcc-8a0287b2ccc3 /               ext3 defaults 1 1
     UUID=820dc26e-f1fe-fcfc-b9ae-2fd222b8a922 /boot           ext3 defaults 1 2
     UUID=f0ec0fcc-cc3a-f038-a03d-9ac7dbc8d803 swap            swap defaults 0 0
     # The SAS/SATA disks, on HBA0 (in port order).
     UUID=29c02f99-9967-f730-affe-c00bff997c71 /dvdlib/mnt/hba0p0 ext3 defaults 0 2
     UUID=39f6c03c-ec70-f7b8-88ff-ef92e1ac20a0 /dvdlib/mnt/hba0p1 ext3 defaults 0 2
     UUID=2d0f1f77-2db9-f32c-aecc-b2dcba6aa1cf /dvdlib/mnt/hba0p2 ext3 defaults 0 2
     UUID=cbacbffd-2d08-fd3e-8f02-7b7f08cbe2da /dvdlib/mnt/hba0p3 ext3 defaults 0 2
     UUID=c2eb2097-09c2-fa02-926d-a222cbdf0c1d /dvdlib/mnt/hba0p4 ext3 defaults 0 2
     # Temporary file systems, system file systems, etc.
     tmpfs                   /dev/shm                tmpfs   defaults        0 0
     devpts                  /dev/pts                devpts  gid=5,mode=620  0 0
     sysfs                   /sys                    sysfs   defaults        0 0
     proc                    /proc                   proc    defaults        0 0

If you add a new disk or replace an existing disk with a different one, you will need to go through all of the steps described above to find the UUID of the disk and update /etc/fstab with it.

Once fstab is updated, you can either reboot the system or, we prefer, try auto-mounting everything first (this way, if fstab is broken, your system will still be up and running and you'll have a chance to fix things):

     su
     mount -a -v

After all the file systems are mounted, you should give them any non-root permissions that you want them to have. Doing this before they are mounted will only modify the permissions of the directory entries in /mnt which are overridden when the file systems are mounted:

     su
     chgrp joeblow /dvdlib/mnt/hba
     chmod ug=rwx,o=rx /dvdlib/mnt/hba

The top level index of the videos taxonomy is now created on the video server using symlinks. All of the symlinks must be soft links that are relative to the /dvdlib subdirectory so that they will work on the systems that mount the exported subtree. Here is an example of a suggested taxonomy:

     su
     cd /dvdlib/DVD
     ln -s ../mnt/hba0p0 Action
     ln -s ../mnt/hba0p4/Cartoons Cartoons
     ln -s ../mnt/hba0p4/Comedy Comedy
     ln -s ../mnt/hba0p4/Cooking Cooking
     ln -s ../mnt/hba0p2/"Cops and Robbers" "Cops and Robbers"
     ln -s ../mnt/hba0p2/Detectives Detectives
     ln -s ../mnt/hba0p4/Documentaries Documentaries
     ln -s ../mnt/hba0p1/Drama Drama
     ln -s ../mnt/hba0p3/Lawyers Lawyers
     ln -s ../mnt/hba0p3/Medicine Medicine
     ln -s ../mnt/hba0p4/Music Music
     ln -s ../mnt/hba0p1/Politics Politics
     ln -s ../mnt/hba0p1/"Science Fiction" "Science Fiction"
     ln -s ../mnt/hba0p4/Transportation Transportation

You will note that some index entries (e.g. Action) point directly to a full disk. All of the subdirectories and files on this disk will appear under the single heading (i.e. Action). Other index entries (e.g. Comedy and Cooking) point to a specific subtree on a single disk. Some of the videos on that disk are in the Comedy top level index and some are in the Cooking top level index. You are pretty much free to arrange the videos directories as you like. Just make sure that all of the symlinks are located at the top of the tree.

Creating the top level index in this manner allows it to be maintained from a single location on the video server itself. You do not have to change any links on the client machines.

Note that the appropriate directory structure on each of the individual file systems should be set up according to the index found in the Myth videometadata table.

Now is the time to copy your video files to your video storage disk(s).

Also, you'll recall that, earlier in this section, we set up a posters directory to hold all of the DVD posters or covers that are downloaded from the Internet or made up by you. This is necessary because the only copy of the database will point to this directory, so it must be accessible to all of the Mythbuntu machines that will access the DVD library. If you've already begin collecting posters, now is the time to copy them to the posters directory.

Next, we'll adjust the Samba config file to allow the high-level video directory to be mounted by guests (i.e. Mythbuntu clients).

     /etc/samba/smb.conf:
     Basically, all you have to do is add a share point for the DVD library:
      [dvdlib]
         comment = DVD Library
         path = /dvdlib
         browseable = yes
         read only = yes
         guest ok = yes
     You do not need to monkey with any of the other security settings in the
     Samba config file.
     Restart Samba to get it to export the new share.

Then, proceed to the steps outlined in Mounting Shares Under Mythbuntu to set up shared video directories under Mythbuntu.

Finally, if you need to expand a disk on you video server, you should proceed, as described in the Increasing Storage Capacity section, to partition, format, and populate the new disk via direct file copy or via lftp. In all probability, you should follow the steps that detail how to set up an Ext3 partition. That section also has tips on how to copy the video onto the new system.