Toppy.org.uk ~ View topic

Author

Message

< Computer connections ~ xml2mei

Page 8 of 9

Goto page Previous 1, 2, 3, 4, 5, 6, 7, 8, 9 Next

TopTapper

Posted: Fri Dec 02, 2016 11:54 am

Frequent contributor Joined: 29 Jan 2006 Posts: 234

Just in case anyone has given up on free microsoft legacy service -- after two or so weeks of "no new data available" it sprang into life three days ago ...

Code:

... xmltv.xml : last output 01/12/2016 11:53
... mc2xml.chl: 1-6, 9-10, 14-15, 21, 24, 28, 30-32, 47
... mc2xml.ren: [file not found]

Loading ..... : mc2xml (c) <mc2xml@gmail.com> (v1.6)
Reminder .... : Unauthorized redistribution prohibited.
Reminder .... : If this software is useful, please donate!
Reading ..... : mc2xml.dat
Downloading . : microsoft.com
Downloaded .. : 918320 bytes in 4s
Source ...... : BDS
Headend ..... : 120020
Version ..... : 2949510313
Start time .. : 2016-12-02T00:00:00 UTC
End time .... : 2016-12-16T23:58:00 UTC
Parsing ..... : Freeview - Crystal Palace
Reading ..... : mc2xml.chl
Writing ..... : xmltv.xml
Writing ..... : mc2xml.dat
Completed ... : 4925610 bytes in 5s, 17 channels, 5943 programs scheduled.

... xmltv.xml : has been updated
xml2mei.php version X8a started: 02/12/2016 10:22:18

(I haven't checked the data).

leenoble

Posted: Thu Jan 05, 2017 10:52 pm

Frequent contributor Joined: 09 Dec 2005 Posts: 291 Location: Cheshire, UK

Hi again,

Following last year's Atlas feed fiasco I've since moved house and haven't had an opportunity to revisit the EPG population problem until now. I've been relying on the freeview scan, which is less than ideal given all the duplicate episodes I keep recording. I read over the last couple of pages on the as2mei thread again and came over here, but I can't see what the current state of play is for getting data onto my Mac and then onto the Toppy.

It looks like you're getting the data form the media center source, is that right? But then I only see links to a .exe file to convert to xml before piping in to this xml2mei script. Did I miss something?

Cheers,
L

_________________
Toppy: TF5800PVR 250Gb; Firmware: 5.13.65T; Tx: Winter Hill;
Autostart TAPs: MyStuff 6.0, eit2mei, SuperPiP;
Other Taps: HDFW [Af|Br|De|E2|Eg|Em|Fs|I|R3|Re|Sl|Vb|Vy|Wf|Xp], Channel Remover;
EPGrt2mei 1.1 & rtsearcher plugin by me for MS description searching.

EMJB

Posted: Fri Jan 06, 2017 10:19 am

Frequent contributor Joined: 08 Jul 2005 Posts: 3645 Location: Maldon Essex

I am using schedules direct as the xml source, and have no complaints (other than paying a few £! a year). I am now using an updated version of xml2mei which now seems to be identifying repeats very reliably even when BBC2 programmes are repeated on BBC4 or v.v. If anyone is keen to use this facility I will make an effort to publish my latest version, but note that the necessary data may not be in other feeds.

EMJB

_________________
Silver 5800, Board Rev 1.2, MS recomm F/W, Sudbury Tx, varying TAP config

EMJB

Posted: Sat Jan 07, 2017 11:27 am

Frequent contributor Joined: 08 Jul 2005 Posts: 3645 Location: Maldon Essex

Have just uploaded version X9, with the improved repeat recognition and some changes suggested by bizman.

EMJB

_________________
Silver 5800, Board Rev 1.2, MS recomm F/W, Sudbury Tx, varying TAP config

leenoble

Posted: Sat Jan 07, 2017 11:14 pm

Frequent contributor Joined: 09 Dec 2005 Posts: 291 Location: Cheshire, UK

I've signed up for the free trial of Schedules Direct. I'm a bit confused about "choosing a lineup". Do I have to do that? Anything I type in just comes up with a list of stuff based in Canada. I'll download the latest version you've kindly uploaded and might get a chance to play with it tomorrow. Cheers.

EMJB

Posted: Sun Jan 08, 2017 10:06 am

Frequent contributor Joined: 08 Jul 2005 Posts: 3645 Location: Maldon Essex

leenoble wrote:

I'm a bit confused about "choosing a lineup". Do I have to do that? Anything I type in just comes up with a list of stuff based in Canada.

Sorry, I can't remember the details of how I selected UK & my region, but it was in there somewhere. You might find more information if you trawl through the early part of this thread and the preceding threads discussing alternative data sources.

EMJB

_________________
Silver 5800, Board Rev 1.2, MS recomm F/W, Sudbury Tx, varying TAP config

Bizman

Posted: Sun Jan 08, 2017 10:18 am

Frequent contributor Joined: 06 Apr 2010 Posts: 1707

For the channel line-up you will need to create a file mc2mei.chl to look something like this:

Code:

# include channels 1 to 21 (inclusive)
1-21
24
25
28
30-32
37
41-42
46
48
60-61
#exclusions
-13 # +1
-16 # omit QVC
-18 #

according to your choice of channels.

_________________
TF5810, F/W: 5.15.09T 4/9/2009 -FmTe+EzHsVb
TAPs: EIT Sub (Game) v0.6; SecCache (UK) v0.4; MHEG On/Off A3; MPDisplayLITE V1.2; MyInfo B5.6; PruneEPG 1.0; EPG2MEI v0.96; QuickJump 1.72; Extend v1.7; (fsSave 1.1);
Sig generated by MyInfo on 3/8/17
Other TAPs : FsCheck, Channel Organiser. PSU Caps 7/2/2011 Tx: Mendip. TV: Samsung (32"), NSLU2, STB: Labgear FV300. PC Acer Aspire One with Win7 Starter

leenoble

Posted: Tue Jan 10, 2017 12:37 am

Frequent contributor Joined: 09 Dec 2005 Posts: 291 Location: Cheshire, UK

Thanks for the help. I've managed to get the data from Schedules Direct. The key was to use the json interface rather than the xmltv one.

I struggled to get the mei filesize down below 4Mb where the Toppy can cope with it more easily. Had to chop a few +1 channels and reduce to about 8 days data to get there though.

Something odd happens with the xml2mei script though. If I use the -sl option to get the series link data, MyStuff sets a timer for what seems to be pretty much every single programme in the EPG!

After deleting all the timers, none of the series link timers in MyStuff work any more so I think there's something different or missing wrt the sl data. I can cope without those provided there's a backup way to identify episodes. The OTA EPG data seems to put e.g. S7 Episode 15 at the end of the descriptions as I've been using that information to remove duplicates and re-records for the last few months. Any change you could add an option to stick the data that's in the xml at the end of the description so I can do it that way if required?

I'll probably take a look at doing it myself as PHP is the thing I get paid for doing in my day job, (github project - pull request?) but you might want to take on the job Wink

EMJB

Posted: Tue Jan 10, 2017 10:06 am

Frequent contributor Joined: 08 Jul 2005 Posts: 3645 Location: Maldon Essex

The series link data is NOT the same as that published (as I thought I explained in the PDF), but should be self-consistent. Thus MyStuff series setup with one either source should work OK while you stay on that source, but will cease to work as soon as you change source.

With regards to the episode data, I don't remember seeing that in the xml data when I last looked at it, but that was months ago. Are you suggesting that it is there somewhere?? The enhanced "new" & "repeat" logic should be very effective at preventing repeat recordings, but won't cope with recording episodes that were not recorded due to clashes etc.

EMJB

_________________
Silver 5800, Board Rev 1.2, MS recomm F/W, Sudbury Tx, varying TAP config

leenoble

Posted: Tue Jan 10, 2017 10:18 am

Frequent contributor Joined: 09 Dec 2005 Posts: 291 Location: Cheshire, UK

There appear to be different episode numbering schemed in the xml feed. Just looking at a few random ones in my latest download.

Code:

<programme start="20170110171500 +0000" stop="20170110180000 +0000" channel="I1.21441.schedulesdirect.org">
   <title lang="en">Pointless</title>

   <episode-num system="dd_progid">EP01320765.0866</episode-num>
   <episode-num system="xmltv_ns">15.31.</episode-num>
   <new />
</programme>

Looks like series 15 ep31 in the xmltv_ns.

Code:

<programme start="20170114093000 +0000" stop="20170114100000 +0000" channel="I7.17155.schedulesdirect.org">
   <title lang="en">The Big Bang Theory</title>
   <sub-title lang="en">The Prom Equivalency</sub-title>
   <episode-num system="dd_progid">EP01270873.0170</episode-num>
   <episode-num system="xmltv_ns">7.7/24.</episode-num>
</programme>

Season 7 episode 7 of 24

I presume the dd_progid is a reference to some other service, which if access was possible would grant more detailed information about the episode number. I only mention it since it looks like most programmes have that if they don't have the xmltv_ns attribute, but recognise that it's probably way beyond scope to dig into that.

An example in the Freeview.mei file will have (S7 Ep7/24) at the end of the programme description.

I confess to not having read a PDF but will do. I'm aware it's not your problem to fix issues with MyStuff but thanks for the info about lack of connection between data sources. It's been years since I looked at this stuff and it's one of those things that gets fiddled with so infrequently that there's absolutely no retention going in in my head.

It's a genuine offer if you would like to farm out any of the work. I previously wrote a plugin for my own use which I used with rt2mei which used a list of episode titles to prefix Rpt to the start of episodes I did not wish to record (this was before series link data).

Edit: Here's the alleged spec on the dd_progid http://forums.schedulesdirect.org/viewtopic.php?f=8&t=575 but it doesn't seem to make much sense. Looking at an episode of The Waltons it says EP01269161.001 and 0.3/25 in the xmltv_ns tag.

The episiode is titled The Hunt, which when I check on episode guides is episode 4 of season 1 (possibly episode 3 if the first one was a pilot but I don't think so). So for the xmltv_ns it looks like it's using an index starting at zero (series 0 episode 3 starting from 0) but I'm pretty sure that's not all series start with counts from 0 which might make things challenging to say the least. But then the dd_progid says it's episode 0001! I've registered on their forum to ask but awaiting confirmation of my account.

Edit II: Having looked again, it does look like the xmltv_ns episode numbers are 0 indexed. Big Bang Theory example I gave above is Season 8 episode 8 of 24

EMJB

Posted: Tue Jan 10, 2017 2:28 pm

Frequent contributor Joined: 08 Jul 2005 Posts: 3645 Location: Maldon Essex

@leenoble

The "dd_progid" information is that which I use as the series link identification. I don't remember finding anything that easily relates to season and episode info there, but probably didn't look very hard.

Edit: Previous problem reported was my mistake.

EMJB

Last edited by EMJB on Sat Jan 14, 2017 8:47 pm; edited 1 time in total
_________________
Silver 5800, Board Rev 1.2, MS recomm F/W, Sudbury Tx, varying TAP config

leenoble

Posted: Tue Jan 10, 2017 2:33 pm

Frequent contributor Joined: 09 Dec 2005 Posts: 291 Location: Cheshire, UK

EMJB wrote:

@leenoble

Have knocked together something to add the episode data to the end of the description. However quite frequently the episode number appears to exceed the episode count, such as:

Code:

E=12, C=8 extracted from <episode-num system="xmltv_ns">11.8.</episode-num>
# 11.8 should be season 12 episode 9
E=17, C=8 extracted from <episode-num system="xmltv_ns">16.8.</episode-num>
# likewise season 17 episode 9. At least that's how it seems in the limited examples I've looked at
<snip>

EMJB

Thanks for looking at it. I might have a bash later on myself.

Bizman

Posted: Thu Jan 12, 2017 8:46 am

Frequent contributor Joined: 06 Apr 2010 Posts: 1707

A bit off-topic, but Shedules Direct has now added LCN83 'Blaze' to its portfolio, not that the American programme content is anything to get exited about.. Note that for Freeview it is off air from 04: 00-05:00 (06:00 on Wednesdays).

EMJB

Posted: Sat Jan 14, 2017 8:48 pm

Frequent contributor Joined: 08 Jul 2005 Posts: 3645 Location: Maldon Essex

Have just uploaded version X10 and associated pdf, with episode information now generated apparently successfully.

EMJB

_________________
Silver 5800, Board Rev 1.2, MS recomm F/W, Sudbury Tx, varying TAP config

Bizman

Posted: Sun Jan 15, 2017 8:19 pm

Frequent contributor Joined: 06 Apr 2010 Posts: 1707

EMJB wrote:

Have just uploaded version X10

Many thanks for that, series & episode information has been greatly missed (by me) until now. I have updated my 'Pick of the Week' software to identify any given first episodes of series.

All times are GMT + 1 Hour

View next topic View previous topic Page 8 of 9 Goto page Previous 1, 2, 3, 4, 5, 6, 7, 8, 9 Next Toppy.org.uk Forum Index ~ Computer connections

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum