Click here to go to the forum index Click here for the home page
 
Author Message

<  TAP and patch development  ~  DescExt firmware patch (like DescriptionExtender)

Page 1 of 8
Goto page 1, 2, 3, 4, 5, 6, 7, 8  Next
R2-D2
Posted: Thu Sep 06, 2007 9:38 pm Reply with quote
Frequent contributor Joined: 18 Dec 2006 Posts: 12149
The DescExt firmware patch reorganises the data returned in TYPE_TapEvent structures by TAP_GetEvent() and TAP_GetCurrentEvent(). The eventName and description fields are combined, which will (in normal usage) have the effect of allowing more of the description to be passed to the TAP. The genre is also added. TAPs can easily detect whether the patch is running, in order to adjust their interpretation of the data. TAPs that are unaware of the patch will (safely) get less description.

The new structure is:
Code:
typedef struct
{
  word  evtId;
  dword startTime;
  dword   endTime;
  word  duration;
  byte  runningStatus;
  char  eventNameDesc[253];  // Replaces eventName and description
  byte  genre;       // New element
  byte  genreSub;    // New element
  byte  descOffset;  // New element - usually zero without patch
  byte  parentalRating;
  byte  satIdx;
  word  svcId;
  word  orgNetId;
  word  tsId;
} TYPE_TapEventDE;
descOffset is the position of the start of the description in the combined eventNameDesc field. TAPs can detect whether the patch is running by using PatchIsInstalled((dword*)0x80000000, "De") from FireBirdLib, or slightly less safely by examining descOffset (if it is zero the patch is not active and so the description will actually be at offset 128). When the patch is active, descOffset will be equal to the length of the event name plus 1 (for the terminating byte), i.e. strlen(eventNameDesc)+1 (since the offset naturally will include the zero termination byte). The genre is a value from 0 to 15, and this indicates the coarse genre. The sub-genre is another value from 0 to 15 which provides another level of detail (but does not seem to be used in the UK). The Toppy itself only provides a description for the first 11 values of genre (and does not actually make use of the sub-genre).

HDFW enables users to temporarily patch their firmware, which can be helpful for testing purposes.

The example TAPs in quickEPG.zip show code before (quickepg1.c) and after (quickepgDE.c), to illustrate the changes needed to utilise the patch (the code in quickepgDE will cope if the patch is not detected). The example is based on Topfield's example quickEPG TAP but adds two extra lines in the display: the start and end of the short description (to show where it cuts off). There is also data about the number of characters of event name and description, and the genre information in brackets. Here are the key bits:
Code:
// Initialise in TAP_Main() probably...
HasDE = PatchIsInstalled((dword*)0x80000000, "De");

// In the code retrieving events...
if(!HasDE)
{
  offset = 128;
  genre = 0;
  genreSub = 0;
}
else
{
  offset = _evtInfo[idx].descOffset;
  genre = _evtInfo[idx].genre;
  genreSub = _evtInfo[idx].genreSub;
}
nlen = strlen(_evtInfo[idx].eventNameDesc);
desc = _evtInfo[idx].eventNameDesc+offset;
dlen = strlen(desc);
The included file genres.h gives lookup tables for genre and genreSub (according the information I can find).

Note: latest version (V6) can currently be found here.


Last edited by R2-D2 on Thu Nov 20, 2008 7:27 am; edited 5 times in total
View user's profile Send private message Visit poster's website
janilxx
Posted: Fri Sep 07, 2007 5:03 am Reply with quote
Frequent contributor Joined: 17 Aug 2005 Posts: 724
R2-D2 wrote:
As far as I know, genres are:
Code:
0x00 = Unclassfied
0x01 = Movie/Drama
0x02 = News
0x03 = Show/Game
0x04 = Sports
0x05 = Children
0x06 = Music/Dance
0x07 = Arts/Culture
0x08 = Social
0x09 = Education
0x0a = Leisure
0x0b = UNKNOWN
0x0c = UNKNOWN
0x0d = UNKNOWN
0x0e = UNKNOWN
0x0f = user(??) defined


Specification allows two level deep genres. You have here first level.
For example 0x01/0x03 = Movie/Drama - science fiction/fantasy/horror

See http://webapp.etsi.org/action/OP/OP20060428/en_300468v010701o.pdf
Page 38, table 28.

Is it so that Toppy itself supports only the first level? Stupid Topfield coders Cool
View user's profile Send private message
janilxx
Posted: Fri Sep 07, 2007 5:21 am Reply with quote
Frequent contributor Joined: 17 Aug 2005 Posts: 724
How would this patch work together with DescriptionExtender TAP?

I mean there will be many TAPs that supports DE and won't be updated anymore and longer descriptions should still be available for them (via DE).

It is quite unlikely that for example Improbox or Jag's EPG will be updated anymore. And how about e.g. mei2archive and UKAS and TAPs like that.
View user's profile Send private message
R2-D2
Posted: Fri Sep 07, 2007 6:49 am Reply with quote
Frequent contributor Joined: 18 Dec 2006 Posts: 12149
janilxx wrote:
Is it so that Toppy itself supports only the first level?
No, it's that the Toppy only seems to use the first level (shown in the EPG). Actually, I think the sub-genre could be added, with a slight modification to the patch -- it might lose another byte of the description, though (space for extra code is very tight on the patch for some firmwares).
View user's profile Send private message Visit poster's website
R2-D2
Posted: Fri Sep 07, 2007 7:01 am Reply with quote
Frequent contributor Joined: 18 Dec 2006 Posts: 12149
janilxx wrote:
How would this patch work together with DescriptionExtender TAP?
The current DescriptionExtender is (of course) unaware of the patch, but I think it will still deliver the short description in the extended info (via TAP_EPG_GetExtInfo()). There may be issues in an unaware TAP using this info, since it will get shortened (short) descriptions, so it may not detect DE is running (resulting in some duplication of descriptions?). But if this patch gains acceptance then it might be possible for DE to reduce the effect on unaware TAPs. I think. Smile
View user's profile Send private message Visit poster's website
R2-D2
Posted: Fri Sep 07, 2007 7:54 am Reply with quote
Frequent contributor Joined: 18 Dec 2006 Posts: 12149
OK -- I'm going to add the sub-genre. It's a conceptually small change to make (the byte alignment is coincidentally just right!), but has a knock-on of lots of other small changes to the patches so it will take a while. Once finished, the genre will be moved to eventName[253] and the sub-genre will be at eventName[254]. Ideally the two nibbles could be recombined to occupy the same byte, but there isn't room to add the extra instructions. I'll also add the full list of genres to the example TAP.
View user's profile Send private message Visit poster's website
R2-D2
Posted: Fri Sep 07, 2007 2:14 pm Reply with quote
Frequent contributor Joined: 18 Dec 2006 Posts: 12149
Sub-genre added -- patch, example TAPs and code updated. The structure will remain static now.
View user's profile Send private message Visit poster's website
dvbgeek
Posted: Fri Sep 07, 2007 2:40 pm Reply with quote
Frequent contributor Joined: 16 Apr 2007 Posts: 168 Location: Sweden
R2-D2 wrote:
janilxx wrote:
Is it so that Toppy itself supports only the first level?
No, it's that the Toppy only seems to use the first level (shown in the EPG). Actually, I think the sub-genre could be added, with a slight modification to the patch -- it might lose another byte of the description, though (space for extra code is very tight on the patch for some firmwares).

FYI, when harvesting EPG with eit2mei/DE (through the native EPG) on the TF5700 HDMI, it sometimes gets the first-level genre for a programme (ex. Movie/Drama) and sometimes the second-level (ex. science fiction/fantasy/horror) but never both levels for the same programme as far as I can see.

_________________
Model: TF5700PVRt HDMI (ID 13426) / Firmware: 2.84 with recommended patches

Autoload TAPs: Power Down / TF5000Display / eit2mei - mei2archive - DescriptionExtender / MyStuff / TAP Commander (always the latest version of everything)
View user's profile Send private message
R2-D2
Posted: Fri Sep 07, 2007 2:45 pm Reply with quote
Frequent contributor Joined: 18 Dec 2006 Posts: 12149
dvbgeek wrote:
but never both levels for the same programme as far as I can see.
I'm not sure what you're seeing, but the sub-genre is meaningless without the genre. However, the two parts can be combined as a single byte (the firmware unpacks them to two bytes, and it would take some manipulation to pack them back again).
View user's profile Send private message Visit poster's website
Toni
Posted: Sat Sep 08, 2007 10:04 am Reply with quote
Regular contributor Joined: 09 Mar 2007 Posts: 40 Location: Finland
Did I understand correctly, that this patch doesn't actually display as much data from event description as DescriptionExtender does? With this patch event name and description are both stored into one variable that can hold up to 251 characters? Otherwise it seems like a great idea to do a patch instead of a tap.
View user's profile Send private message
R2-D2
Posted: Sat Sep 08, 2007 10:40 am Reply with quote
Frequent contributor Joined: 18 Dec 2006 Posts: 12149
Toni wrote:
Did I understand correctly, that this patch doesn't actually display as much data from event description as DescriptionExtender does?
Potentially, I guess. Technically the name and description could both be 256 characters. But in reality, the TAP API's (normal) event name buffer is massively oversized at 127 characters, and the description buffer is often slightly undersized at the same limit. So, it's quite common for the short description to be more than 127 characters, but the name+description will be less than 251. If the broadcasters want to go overboard on description they would use the extended event description data, which is untouched by this patch -- I've not seen Freeview data making use of this, though.

The demo TAP shows the character usage on the displayed EPG data, if you're interested. Run it without the patch and see how many descriptions get clipped to 127 bytes.
View user's profile Send private message Visit poster's website
DX
Posted: Sat Sep 08, 2007 9:11 pm Reply with quote
Frequent contributor Joined: 06 Apr 2005 Posts: 2672
R2-D2 wrote:
TAPs can detect whether the patch is running by examining descOffset -- if it is zero the patch is not active (and so the description will actually be at offset 128).

I'm having trouble with this bit. On my normal unpatched system descOffset seems to contain garbage values if the description length is less than 127 chars. Checking the value against strlen(eventNameDesc)+1 helps but it still occasionally goes wrong.

I'm running 5.13.40 with various taps (Description Extender, crid, eit2mei, mei2archive, Surfer and MyStuff), but nothing out of the ordinary.

Any suggestions? Or will support need to be an ini file option?
View user's profile Send private message
R2-D2
Posted: Sat Sep 08, 2007 9:54 pm Reply with quote
Frequent contributor Joined: 18 Dec 2006 Posts: 12149
DX wrote:
I'm running 5.13.40 with various taps
Darn, that's what I've been using, too. I'll have to stare at it a bit more...
View user's profile Send private message Visit poster's website
R2-D2
Posted: Sun Sep 09, 2007 10:22 am Reply with quote
Frequent contributor Joined: 18 Dec 2006 Posts: 12149
DX wrote:
Any suggestions? Or will support need to be an ini file option?
Topfield's paranoid extra termination of the event name and description (which is entirely unnecessary) only happens when the strings are longer than 127 bytes. Darn. You could test the genre and sub-genre and if either are greater than 15 then the patch isn't applied. But again, there's still a possibility of a false positive. Maybe this is something that would be best solved reliably by Patch IDs?
View user's profile Send private message Visit poster's website
DX
Posted: Sun Sep 09, 2007 11:17 am Reply with quote
Frequent contributor Joined: 06 Apr 2005 Posts: 2672
I already test genre and genreSub - great minds and all that Laughing

My current attempt at auto-detection also checks all the entries returned by TAP_GetEvent before I start processing the data and if any of them fails I know there is no patch. This further reduces the chances of false positives.

In tests using all the above I no longer see false positives, but although the odds are long the possibility still exists. It's probably more likely outside the UK where short descriptions under 127 characters are more common, and where only now/next is broadcast reducing the number of entries checked.

My other thought (not implemented) was if I detect Description Extender is loaded assume the patch isn't. I don't know if this check is worth it.

My inclination at the moment is to go with an in file option as it's easy for me and anyone patching the firmware shouldn't have problems updating an ini file by adding "AutoDetectDEP=1". I'll build and upload new test versions of eit2mei, mei2archive, and SeriesLink with this included in the next day or two.
View user's profile Send private message

Display posts from previous:  

All times are GMT
Page 1 of 8
Goto page 1, 2, 3, 4, 5, 6, 7, 8  Next

Jump to:  

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum