Guide data is loaded into the MythTV database by a program called mythfilldatabase. This program expects that the guide data will be in a format specified by the XMLTV standard, a description of which is found here:
http://xmltv.cvs.sourceforge.net/checkout/xmltv/xmltv/xmltv.dtd
Additional descriptive information can be found in the XMLTV wiki here:
http://wiki.xmltv.org/index.php/XMLTVFormat
The program that obtains guide data (from whichever source is appropriate) is called a grabber in XMLTV parlance. For many countries, a grabber has already been written and is included in the XMLTV package. Check the MythTV wiki for a list of what's available:
http://www.mythtv.org/wiki/XMLTV
If there is no grabber available for your country or your goal is simply to create your own guide data (for whatever reasons), from some source of your own, you'll need to generate XMLTV files containing the data, probably on a daily basis. You can use a support module from the Perl XMLTV package, if you're inclined to write your code in Perl, to generate guide data files in the correct format, which can be fed into mythfilldatabase.
Begin by downloading the latest version of the complete XMLTV package from sourceforge:
http://sourceforge.net/projects/xmltv/files/
Unzip the distribution file:
tar -xvjf xmltv-0.5.56.tar.bz2
Now, generate the package's makefile, like so:
cd xmltv-0.5.56
perl Makefile.PL
This will ask you a lot of questions about which optional components and grabbers you want. Answer no to all of them. All we really want is the XMLTV.pm module itself, so that we can use it to write XMLTV files.
If need be, install any missing prerequisites (but not any of the optional modules). For example, install Slurp and the XML writer like this:
perl -MCPAN -e shell
install File::Slurp
install XML::Writer
Once you've installed any missing prerequisites, you can build XMLTV:
make
Then, find the XMLTV.pm module in the lib directory and copy it somewhere that Perl programs can find it. For example:
su
cp lib/XMLTV.pm /usr/lib/perl5/site_perl/5.8.8
When you build your guide data, make sure to generate channel data that looks like this:
<channel id="nn">
<display-name>CALLSIGN</display-name>
<display-name>nn</display-name>
</channel>
The "CALLSIGN" value is whatever call sign is used for that channel (e.g. "WGBH"). This is typically not the name used to display the channel by MythTV but, rather, the official call letters for an over-the-air broadcaster or short name for a cable broadcaster. The important thing is that "nn", the actual channel number that is used by MythTV in the channel's xmltvid field of the channel table (see below), is the second display name. It is imperative that every channel that is described in the XMLTV data has a second display name that matches the xmltvid field of some channel in the channel table or mythfilldatabase will make a channel up, which could have quite unpleasant results.
This Perl code should work to generate the correct XML:
@ChanList = ();
push(@ChanList, [$CallSign]);
push @ChanList, [$Channel];
$XMLTVHand->write_channel({
id => $Channel,
"display-name" => \@ChanList });
Program detail slots are generated for each channel. Each slot gives the start and stop time for the program, as well as the channel it is broadcast on:
<programme start="yyyymmddhhmmss" stop="yyyymmddhhmmss" channel="nn">
<title lang="en">The National Parks: America's Best Idea</title>
<sub-title lang="en">The Scripture of Nature (1851-1890)</sub-title>
<episode-num system="xmltv_ns">.1.</episode-num>
</programme>
When you generate your program data, forget about the timezone offset on the start and stop stamps and the story about GMT in the XMLTV DTD. Its local time all the way, no matter what you do. Also, be aware that "nn" must match the previously-written channel entry. Here is some sample Perl that should do the trick:
%Program = ();
$Program{channel} = $Channel;
($Sec, $Min, $Hour, $MDay, $Mon, $Year) = localtime($StartTime);
$Program{start} = sprintf("%04d%02d%02d%02d%02d%02d",
1900+$Year, 1+$Mon, $MDay, $Hour, $Min, $Sec);
($Sec, $Min, $Hour, $MDay, $Mon, $Year) = localtime($SlotStamp+1);
$Program{stop} = sprintf("%04d%02d%02d%02d%02d%02d",
1900+$Year, 1+$Mon, $MDay, $Hour, $Min, $Sec);
$Program{"episode-num"} = [[sprintf(".%d.", $EpisodeNum), "xmltv_ns"]];
$Program->{title} = [["en", "The National Parks: America's Best Idea"]];
$Program->{"sub-title"} = [["en", "The Scripture of Nature (1851-1890)"]];
You'll probably want to set episode numbers with code like this:
$Program{"episode-num"} = [[sprintf(".%d.", $EpiNum), "xmltv_ns"]];
You can read the story about the episode numbering scheme employed by XMLTV in the DTD but, essentially, you just want to set the middle component of the episode number to the actual, sequential episode number, which you do by generating a number like ".11.".
If you have them available, run times for programs or movies need to be converted into seconds. Normally, run times are the actual time of the program, without commercials (e.g. for a movie, it would be the running time in the movie theatre or on the DVD). You can add them with code like this:
$Program{length} = $RunTime * 60;
The genre groups that MythTV uses are the common ones such as "news", "sports", "comedy" and "drama". If you had, for example, a set of simple genre tags you might use some code like this, inside of a loop, to convert them to genres that MythTV can use, in the XMLTV file:
@GenreGroups = ();
foreach $ProgType (@ProgTypes)
{
if ($ProgType eq "com") { push(@GenreGroups, ["comedy", "en"]); }
elsif ($ProgType eq "dra") { push(@GenreGroups, ["drama", "en"]); }
elsif ($ProgType eq "kid") { push(@GenreGroups, ["children", "en"]); }
elsif ($ProgType eq "mov") { push(@GenreGroups, ["movie", "en"]); }
elsif ($ProgType eq "new") { push(@GenreGroups, ["news", "en"]); }
elsif ($ProgType eq "sci")
{ push(@GenreGroups, ["science fiction", "en"]); }
elsif ($ProgType eq "sop") { push(@GenreGroups, ["daytime", "en"]); }
elsif ($ProgType eq "spo") { push(@GenreGroups, ["sports", "en"]); }
}
if (scalar(@GenreGroups) > 0) { $Program{category} = [@GenreGroups]; }
For data that repeats, such as crew members, you can push the items onto an array and then generate the XMLTV data from the arrays like this:
@Actors = split(/,\s/, $ActorNames);
@Directors = split(/,\s/, $DirectorNames);
@Producers = split(/,\s/, $ProducerNames);
@Writers = split(/,\s/, $WriterNames);
if (scalar(@Actors) > 0) { $Program{credits}{actor} = \@Actors; }
if (scalar(@Directors) > 0)
{ $Program{credits}{director} = \@Directors; }
if (scalar(@Producers) > 0)
{ $Program{credits}{producer} = \@Producers; }
if (scalar(@Writers) > 0) { $Program{credits}{writer} = \@Writers; }
For any data that has no special requirements, you can use a table that looks something like this to indicate how it is processed:
my %ProgramTags = ( # Tags for program data
"DE", "desc|en", # Description
"DT", "date", # Release date (year)
"ET", "sub-title|en", # Episode title
"MP", "rating|MPAA", # MPAA rating
"TI", "title|en", # Program title
"ZZ", "Junk"); # Placeholder
if (exists($ProgramTags{$FieldKey}))
{
if ($ProgramTags{$FieldKey} =~ /\|/)
{
@XtraTags = split(/\|/, $ProgramTags{$FieldKey});
$FieldTag = shift(@XtraTags); unshift(@XtraTags, $InfoItem);
$Program{$FieldTag} = [[@XtraTags]];
}
else { $Program{$ProgramTags{$FieldKey}} = $InfoItem; }
}
When all is said and done, the program data should be written to the XMLTV file:
$XMLTVHand->write_programme(\%Program);
Before you attempt to load any guide data into MythTV, be sure that you set the xmltvid field of each channel in the channel table to the same value as was set for the second display-name in your generated data. You can update the field like this:
update channel set xmltvid='nn' where chanid=mmmm;
Or, you can go back and reload your channel table with the original data that you used to populate it, setting the xmltvid in that data, like so:
-- Add the only real analog channels from the PSIP section
insert into channel set chanid=3002, channum='2', freqid=2, sourceid=3,
xmltvid='2', callsign='WGBH', name='WGBH-2 (PBS, Boston)';
insert into channel set chanid=3003, channum='3', freqid=3, sourceid=3,
xmltvid='3', callsign='HSN', name='HSN (Shopping)', visible=0;
.
.
.
-- Add the analog virtual channels from the PSIP section
insert into channel set chanid=3023, channum='23', freqid=23, sourceid=3,
xmltvid='23', callsign='DAYSTAR', name='Daystar (Religious)', visible=0;
insert into channel set chanid=3024, channum='24', freqid=24, sourceid=3,
xmltvid='24', callsign='DISNEY', name='Disney';
.
.
.
Once you've generated your XMLTV guide data files and made them available to the MythTV master backend, you can load the generated data with:
mythfilldatabase --file n /path/to/my/guidedata.xml
Where "n" is the source number (e.g. cable, antenna) that is to be updated.
We suggest that you try this on your test system first, in case you muck it up (its easy to do). To begin with, you can trim your generated data to only include one channel and a day's worth of listings. Once you get the listings to show up as the right channel, at the right times, you can switch to your production system.
A good way to check the listings is with MythWeb. The TV/Program Listing page will show you pretty quickly whether the data is good or not. If you do muck it up and want to retry, these deletes will clear the database:
delete from channel where chanid=bogus;
delete from program where chanid=bogus;
delete from programrating where chanid=bogus;
or, for a range of guide data:
delete from channel where chanid>=bogus-low and chanid<=bogus-hi;
delete from program where chanid>=bogus-low and chanid<=bogus-hi;
delete from programrating where chanid>=bogus-low and chanid<=bogus-hi;
Note that the oldprogram table may hold borked program descriptions but its probably best not to delete things from it, since it is used by recorded programs, etc. Besides, who cares if SpongeBob mistakenly investigates a murder in a hotel in New York, anyway?
After you've got your code working and you are successfully generating XMLTV guidedata files, you'll need to load them into MythTV. We use a script that is run by cron at regular intervals. Here it is:
guidedata-update:
#!/bin/sh
#
# Script to update MythTV guide data using XMLTV files that were built
# elsewhere, on a remote server.
#
# This script will mirror the remote server's XMLTV directory for the chosen
# zipcode, using lftp, and then load any files, using mythfilldatabase, that
# are newer than the time of the last load. Note that the mirror directory
# on the local host and the remote server's XMLTV directory must have the
# same path name.
#
#
# Should be run at regular intervals from cron, any time after the XMLTV
# files are built on the remote server. Use the following:
#
# guidedata-update zipcode source
#
# zipcode - The zipcode or Canadian postal code to update data for.
#
# source - The source number (from the mythconverg database) whose guide
# data is to be updated. More than one source can be updated at
# once by specifying a comma-separated list. The default, if
# omitted, is 1.
#
#
# Where things are found.
#
RemoteHost=192.168.1.1
RemoteUser=guidedata
RemotePass=asecretpass
XMLTVDir=/var/guidedata/XMLTVData
Owner=root
Group=mythtvuser
#
# Stupidity checking and parameter processing.
#
if test x"$1" = x; then
echo "Zipcode omitted, defaulting to 12345"
ZipCode="12345"
else
ZipCode=$1
fi
XMLTVPath="${XMLTVDir}/${ZipCode}"
if test x"$2" = x; then
echo "Source omitted, defaulting to 1"
Source="1"
else
Source=$2
fi
Source=${Source},
#
# Mirror the remote server's XMLTV directory here.
#
# Note that we must include the path to echo in the mirror command, otherwise
# it fails under cron.
#
/bin/echo -e "mirror -e ${XMLTVPath}/ ${XMLTVPath}" \
| /usr/bin/lftp -u ${RemoteUser},${RemotePass} ${RemoteHost}
chown ${Owner}:${Group} ${XMLTVPath}/
chmod ug=rw,o=r ${XMLTVPath}/
#
# For each source that we were asked to update, troll through the mirrored
# directory looking for any guide data files that are newer than the time of
# the last update.
#
while echo $Source | grep \, 2>&1 >/dev/null; do
#
# Grab the next source out of the list. Then, remove it from the list.
ProgSrc=${Source%%\,}
Source=${Source\,}
#
# Process all of the files that we haven't done yet.
#
if [ ! -f ${XMLTVDir}/${ZipCode}_${ProgSrc}previousupdate ]; then
touch -t 197001010500 \
-f ${XMLTVDir}/${ZipCode}_${ProgSrc}previousupdate
fi
/usr/bin/find ${XMLTVPath} -name *.xml -type f \
-newer ${XMLTVDir}/${ZipCode}_${ProgSrc}previousupdate \
-exec /usr/bin/mythfilldatabase --file ${ProgSrc} \{\} \; \
>/dev/null 2>&1
# -exec echo "/usr/bin/mythfilldatabase --file ${ProgSrc} {}" \;
# -exec /usr/bin/mythfilldatabase --file ${ProgSrc} \{\} \;
touch -f ${XMLTVDir}/${ZipCode}_${ProgSrc}previousupdate
done