Click here to go to the forum index Click here for the home page
 
Author Message

<  MyStuff  ~  Proposed update to MEI Format

Page 1 of 4
Goto page 1, 2, 3, 4  Next
Bawbagg
Posted: Fri Jan 13, 2006 2:44 am Reply with quote
MyStuff Team Joined: 11 Aug 2005 Posts: 1122
When I originally devised the MEI format, I didn't expect anything like the take up that we've seen Very Happy. I knew I was omitting certain fields from it, but I thought I'd be able to just slip them in at a later point in time without upsetting anyone. Now, there are at 2 taps currently using MEI format as an EPG source, and at least 4 authors of MEI generating software I know of. I wonder how many more folks are up to things in quiet Surprised?? I therefore guess any change to the format needs to be done using a democratic process Smile.

MEI, in its current incarnation is missing some key fields that are available. Most notably, those that are missing, but are in the XMLTV DTD are:
    * credits
    * language
    * orig-language
    * icon
    * url
    * country
    * audio

Now, as not many folks in here are remotely bothered about the XMLTV DTD, the one that really is of interest to us is/are credits - these are currently available in the radiotimes.com download.

We have 2 reasonable options open to us:
    1. Include credit information in the description field (which is the easy way out)
    2. Tag new field(s) on the back of MEI with credit information, and any others we fancy

If we take option 2 then we have a new format - which needs to be separated from MEI, and TAP authors will need to modify their code to use it. The beauty of option 1 is that we could just slip it in without anyone noticing!! However I like the idea of option 2, because I'd like to change the name of MEI to disassociate it from MyStuff not because there's anything wrong with MyStuff... but because it can (and is) be used by other taps. I guess I'm therefore proposing that MEI format be superceeded by TEI format (Toppy_Extended_Info).

I should say that the heavy hitters in taking any decision as to what does and doesn't go in TEI (and indeed, if it happens at all) are BobD and rwg Very Happy.

Let the debate begin Laughing.

BB

_________________
TAPs: MyStuff Something or other + whatever CW recommends
MEI readme and latest version at http://my.opera.com/bawbagg
Current MyStuff Known Bugs http://www.BobDsMyStuff.co.uk/Bugs.shtml
View user's profile Send private message Visit poster's website
charley
Posted: Fri Jan 13, 2006 3:09 am Reply with quote
Frequent contributor Joined: 13 Jul 2005 Posts: 1563 Location: Belfast
I suggest that you go with option 2 and bite the bullit now as although option 1 is simpler it will become increasingly difficult to change if there is a wider take up over time. My 2 cents worth.

Its a great asset either way with MyStuff and MEISearch, or will it become TEISearch Smile
Thanks.

_________________
regards Charley
Toppy: TF5800PVR250GB &300GB;Firmware: ; Remote: Harmony 655; Tx:Divis; Autostart TAPs: MEISearch 1.35, MyStuff 6, AutoReboot V2.2, epg2mei, Power Manager v2.0, TAP Commander 1.2, TF5000 Display v1.50, MHEG_State; Other:
View user's profile Send private message
jimbo
Posted: Fri Jan 13, 2006 9:11 am Reply with quote
Regular contributor Joined: 19 Sep 2005 Posts: 82 Location: North Yorkshire, Bilsdale Transmitter.
Option 2
But as you say its upto to you, bob and rwg as you'll be doing the work.

_________________
Firmware: 5.12.88
TAPS: MyStuff 5.21a, TF5000 Display, Remote Extender, Description Extender, eit2mei
View user's profile Send private message Visit poster's website
CeeGee
Posted: Fri Jan 13, 2006 9:22 am Reply with quote
Frequent contributor Joined: 13 Apr 2005 Posts: 390
Option 2 - before it is too late to easily switch formats, and affects too many users of current and future TAPS.
How did we live before the days of MEI/TEI? A great asset, and should be made as future proof as possible.
However, I'm not the one that does all the hard work devising the TAPs.

Colin

_________________
Model:TF5810PVR. Transmitter: Crystal Palace. TAPS: My Stuff, OneList, MHEG Control, Remote Extender, Description Extender, MP3Jukebox. Remote: Harmony 525

All I want to do is watch some television!
View user's profile Send private message
Mike
Posted: Fri Jan 13, 2006 10:01 am Reply with quote
Regular contributor Joined: 26 Mar 2005 Posts: 43
BB,

I am not a Tap writer, but have a fair bit of data management background. The MEI format looks nice and simple, and also has (I guess deliberately) been kept small by avoiding the space-wasting xml-tag style of doing things.

Is there some simple way of making the format self-describing so that extra fields can be added as things evolve,

This could be a simple as:

First row must contain a version no - version no defines fields present in that version, when new fields are added the version no is incremented, and new fileds must be added at the end of each row. Then if an app is reading a version no greater than it was written to handle it is expected to read to the next CRLF - or whatever row delimiter is used ignoring columns after the number it understands how to handle - This would allow new columns to be added, but not to be taken away again.

Alternatively add a new first row with column names for each column contained in the file, this would allow columns to be added and removed at will provioded they were not critical to the app, but I suspect would make processing the file somewhat more complex.

Just some thoghts, but when sharing data some form of self-describing format often helps, but one can often achieve some of the benefits without being as space wasteful as most xml structures are.

Mike
View user's profile Send private message
EMJB
Posted: Fri Jan 13, 2006 10:11 am Reply with quote
Frequent contributor Joined: 08 Jul 2005 Posts: 3638 Location: Maldon Essex
Option 2 gets my vote, but I am just one of the lucky users rather than hard working writers.

How about adding a few unused fields for further EPG data should it become available while you are about it, as suggested in another thread?

If anyone was to ever try to merge broadcast EPG data to cope with late changes, a field that they could write association data into could be useful. I know Nigel thinks association would be difficult, but we have some very resourceful people around on this forum, and an ability to put a warning up when programme times have changed would be a useful start.

One approach to provide future flexibility that I have used (between PC applications) is to define a basic data structure followed by an indeterminate number of text fields each of which includes its own identifier such as beginning with (an abbreviation of) "First shown in=". One would then need unique identifiers for the ends of such fields (e.g. tab) and end of line.It means the data size is not fixed, but gives considerable flexibility for enhancements while retaining backward compatibility for any TAPs designed to cope with this structure.

Hope these comments help provoke some even better ideas.
View user's profile Send private message
RobTuck
Posted: Fri Jan 13, 2006 10:43 am Reply with quote
Frequent contributor Joined: 20 May 2005 Posts: 251 Location: Cardiff, Wales, UK
I'd also go with option 2.

As someone developing applications that use the MEI format, I think it would be more beneficial in the long run and only minor code modifications should be needed by the other developers to accommodate it.

_________________
Topfield TF5800, MyStuff 5.62c, rt2mei on Debian 5.0 Linux, MWI 0.50a.
View user's profile Send private message
DX
Posted: Fri Jan 13, 2006 11:59 am Reply with quote
Frequent contributor Joined: 06 Apr 2005 Posts: 2695
Another vote for option 2.

I agree with Mike's suggestion of adding any new fields to the end of the line, and having taps read up to CR/LF. While it does mean you can't delete existing fields the overhead for an unused one is small. (I'll declare a vested interest - I've already coded for that Smile )

If we are looking for fields to be added I'd like director and actors fields (I think I've seen talk of that for MyStuff). For "Season Pass" style functionality provision of a series id and episode id would be useful (although I'm not sure if the RT site can currently provide this info).
View user's profile Send private message
Mark_B
Posted: Fri Jan 13, 2006 12:39 pm Reply with quote
MyStuff Team Joined: 12 Aug 2005 Posts: 254
My thoughts:

In changing the format is it worth adding something as a header that is used to say the mei date of extract rather than reply on the file date - this seems to cause people problems from time to time and would menas no matter how you got the file to the toppy it should still work. More and more means to automate distribution abound. Taps could then show this as the file start date and we'd know when the number of days were up in taps for its usage (Mystuff)

In terms of options for extra info - I think we should consider the following points carefully and it may need to be a hybrid solution....

1. Memory usage - are we to add the credits just for films or for all programs - will it add a lot to size?

2. I want to use the data to search on and set timers in the future. Is it easier to have one long description to search on for words or will tap developers be programming word search against lots of different fields in the file.

3. For layout having fields separately defined is good - as its easier to show them on windows with nice labels against and harder if its all in a string.....

So I agree its up to taps but suggest the answer should be:

Option 2 - add the new data; director can be multiple as can actor - so we need to decide how we lay these out as a string with commas in or otherwise

Plus - we need to consider the search aspect in the taps and decide if it is worthwhile for tap developers in the short term to have this data also appended to the end of the long description to support timer searches as this comes about.

As this would be extra code and some taps may choose not to do it then the option to add to the description at generate mei/tei a second copy of this data for FILMS for support this feature. This can currently be done in Radio Times for film dates where you can add Date:1975 to the end of the long description as one of the settings. So maybe we could allow the same for these and if all the taps exploit directly then we could remove this later by unsetting the flag. Would people be happy just for films or would other like to find all say comedy with Ricky Gervais in?

Only disadvantage here would not be to bawbagg as this would allow him to complete this for now and create a std tap but might create more memory usage across the multiple taps that use tei/mei. If this was for films alone then it wouldnt be too high hopefully. is its for everything then I wonder how big this would get.

If we didnt do this second set of appending then the tap developers would need to implement multi searches in side their taps of the title description director, actor etc so adding to the confusion of word searching potentially.......

Also if we go down this route of both - then I suppose we could end up with - appending other data - like is the film dubbed from orginal language, is it in black and white,....etc....

Last point is - do we expect other +1 channels in the future and as such should we add in a feature to dealt with this now likes Birtles 3 inside the extended code to keep functionality on 2.2 or do we all need to make the move to Birtles 3....

Sticking with Birtles 2.2 - can we rely on Radio Times to keep their word and add MORE4+1 shortly or should we get this feature in for use now and future proofing....
View user's profile Send private message
wooders
Posted: Fri Jan 13, 2006 2:12 pm Reply with quote
Frequent contributor Joined: 25 Aug 2005 Posts: 428 Location: The National Forest
I will post my thoughts on file format tonight when I can just copy and paste what I have already sent to BB.

On the subject of a name for the new format, it seems to me that we need 3 elements
  1. Toppy or Topfield
  2. EPG, because that is the main data content
  3. External or Imported or Alternative or similar because this data is sourced from somewhere other than the broadcast datastream. I, personally, don't think that the adjective 'extended' is particularly appropriate to a new general format.
Just my 2d (old currency). Out of the above you can get TEE or TIE or TEA as TLAs. If TEA is favoured then my new scripts will probably be called rt2cuppa Very Happy

Wooders

_________________
TF5800, TS On, F/W: MS6 Recommended F/W 12/9/2009 -Sy+Pe
TAPs: EPG2MEI v0.96; Extend v1.7; Font Manager 1.0d; MyInfo B5.5; MyStuff 6.4; WSSkiller V2.12d; SecCache (UK) v0.4; EIT Sub (Game) v0.6; MHEG On/Off A3;
Sig generated by MyInfo on 30/3/12


Let rt2mei 1.1a feed your Toppy with EPG data from the Radio Times website.
Download rt2mei from http://www.wooders.co.uk/rt2mei
View user's profile Send private message Visit poster's website
CeeGee
Posted: Fri Jan 13, 2006 3:50 pm Reply with quote
Frequent contributor Joined: 13 Apr 2005 Posts: 390
wooders wrote:
Out of the above you can get TEE or TIE or TEA as TLAs. If TEA is favoured then my new scripts will probably be called rt2cuppa Very Happy


And then you will get prople asking why they are not getting a hot brew from the CAM slots Very Happy

Colin

ps. Shouldn't that be TEI - not TIE, you are getting your letters knotted up.

_________________
Model:TF5810PVR. Transmitter: Crystal Palace. TAPS: My Stuff, OneList, MHEG Control, Remote Extender, Description Extender, MP3Jukebox. Remote: Harmony 525

All I want to do is watch some television!
View user's profile Send private message
nwhitfield
Posted: Fri Jan 13, 2006 3:59 pm Reply with quote
Site Admin Joined: 20 Mar 2005 Posts: 9579
TIE (Topfield Imported EPG or something like that) appeals from a quirky point of view to me, because it's the reverse of EIT which is where the standard Toppy usually gets its information from.

Nigel.

_________________
Support this site - make a donation to our running costs
View user's profile Send private message Visit poster's website
Mark_B
Posted: Fri Jan 13, 2006 4:00 pm Reply with quote
MyStuff Team Joined: 12 Aug 2005 Posts: 254
We have some good brands already with mei and mystuff - I wonder if we need to disassociate?

Are we looking to brand for:

- a radiotimes data format?
- an epg standard format for the toppy from any source?

Would we be adding stuff on the end for future growth?

Suppose depends if the format is compatible with old or a new one that taps would have to move to. More we alter rather than extend the bigger the hit on taps and others I suppose....

Blank sheet.....

RadioTimes_EPG.dat

seems most applicable (to me Very Happy )
View user's profile Send private message
nwhitfield
Posted: Fri Jan 13, 2006 5:03 pm Reply with quote
Site Admin Joined: 20 Mar 2005 Posts: 9579
I think the key things are:

a) the data is sourced externally
b) it uses its own mechanism for display and storage on the Toppy, rather than the system EIT mechanism

Those are the differentiators; while the data source is the Radio Times at the moment, it wouldn't specifically mention that in a product title since a) RT data is intended only for personal use, and pinching their branding isn't a wise idea and b) there's no reason why you have to use the RT data to create an MEI file.

People in other countries might well find data sources of their own to generate MEI data, just as they've done to harvest data for the existing TGD format.

Nigel.

_________________
Support this site - make a donation to our running costs
View user's profile Send private message Visit poster's website
Mike
Posted: Fri Jan 13, 2006 5:31 pm Reply with quote
Regular contributor Joined: 26 Mar 2005 Posts: 43
Now the ideal would be a full ANSI complaint SQL implemenatation on the Toppy, with distributed database support via the USB to a Windows or Linux implementation thus allowing full integration of any data on the Toppy with data sources of the users choosing. Think of the possibilities!
This would also mean anyone could add columns as required and the existing apps could continue in blissful ignorance!

How much memory is usable by Taps??

(Just in case anyone thinks I am serious with this, I am not but who knows in a few years!)

Mike
View user's profile Send private message

Display posts from previous:  

All times are GMT + 1 Hour
Page 1 of 4
Goto page 1, 2, 3, 4  Next

Jump to:  

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum