Click here to go to the forum index Click here for the home page
 
Author Message

<  TAP and patch development  ~  Up-to-date Topfield rec file header docs anywhere?

Page 1 of 1
Bob_Swede
Posted: Mon Jan 25, 2010 12:10 am Reply with quote
Frequent contributor Joined: 22 Oct 2007 Posts: 1066 Location: Sweden
I have tried to locate a document outlining the format of the Topfield REC file header.
So far I have found a discussion here: Topfield rec file format documentation where there is embedded information (but no doc file).
But it seems like it is not updated or incomplete. Sad

There are also a few tools linked in this thread:
TFRecEdit.exe is purportedly an editor for the header, but as soon as I try loading any rec file it complains that the date format is invalid, which it of course is not....

TFHead.exe is a header viewer, which can actually load my rec files, but cannot display the program description. It displays other things though. Smile

Since these are quite old (thread and sample applications) I believe they may not have been updated along with changes to the toppies.

So can anyone here point me to a valid document describing the current rec file header format? I want to write my own application to check the file headers (and possibly modify them)....

TIA Cool

_________________
Bo B (Sweden)
5700HDMIPVRt, TS On, F/W: patches (3.17 aka 5.14.77)
TAPs: TF5000 Display v1.53; ChannelChangeButtonHack 1.1; MyStuff 6.4; TAP Cmder 1.34; MyInfo B5.4;
(EPG: XMLTV2Mei for Win + MyStuff)
(Toppy access: ASUS WL-500g-P FTP installed using chunkywizard's guide)
Sig partly gen. by MyInfo on 12/4/2013
View user's profile Send private message
R2-D2
Posted: Mon Jan 25, 2010 3:12 pm Reply with quote
Frequent contributor Joined: 18 Dec 2006 Posts: 12148
Same place as normal: FireBird's Doc. Although there is a bit missing about the TF5810 (or TF5800 5.14.X) format's small differences.

_________________
Troubleshooting -- User Manual -- Dark Side of the Matrix: Firmwares and Patches
View user's profile Send private message Visit poster's website
Bob_Swede
Posted: Fri Jan 29, 2010 11:13 am Reply with quote
Frequent contributor Joined: 22 Oct 2007 Posts: 1066 Location: Sweden
R2-D2 wrote:
Same place as normal: FireBird's Doc. Although there is a bit missing about the TF5810 (or TF5800 5.14.X) format's small differences.

I have studied what I could get hold of and I must say that the TF_Internal_Info.pdf document is really confusing....
The last pages contain the recording file header info, but I can't make any sense out of it. For example it says that the struct tRECServiceInfo5000; holds 38 bytes of data. But if I cound all of the constituent fields I get 48 bytes (2 bytes + 11 words + 1 char[24] array = 48 bytes).
It also says that SatIndex is DVB-s only, but does not show a struct member for DVB-t...

I am a Pascal programmer so i am trying to convert these structs into packed records and I have a problem with the syntax, maybe that's where the size diff is coming from?
What does it mean when there is a field identifier that ends with a : plus a number? Like:
word TPIdx:10;
word TunerNum:2;
word DelFlag:1;
word CASFlag:1;
word LockFlag:1;
word SkipFlag:1;
I have never seen that before, does it mean that these are excluded? But then there are too many excludes to match the difference between 48 and 38...
Confused Shocked

_________________
Bo B (Sweden)
5700HDMIPVRt, TS On, F/W: patches (3.17 aka 5.14.77)
TAPs: TF5000 Display v1.53; ChannelChangeButtonHack 1.1; MyStuff 6.4; TAP Cmder 1.34; MyInfo B5.4;
(EPG: XMLTV2Mei for Win + MyStuff)
(Toppy access: ASUS WL-500g-P FTP installed using chunkywizard's guide)
Sig partly gen. by MyInfo on 12/4/2013
View user's profile Send private message
R2-D2
Posted: Fri Jan 29, 2010 12:13 pm Reply with quote
Frequent contributor Joined: 18 Dec 2006 Posts: 12148
Bob_Swede wrote:
What does it mean when there is a field identifier that ends with a : plus a number?
It means that field uses that number of bits from the storage type, and adjacent ones from the same type are merged into one. In the specific example, those fields comprise the 16 bits of a single word entry in the struct.

_________________
Troubleshooting -- User Manual -- Dark Side of the Matrix: Firmwares and Patches
View user's profile Send private message Visit poster's website
Bob_Swede
Posted: Fri Jan 29, 2010 2:09 pm Reply with quote
Frequent contributor Joined: 22 Oct 2007 Posts: 1066 Location: Sweden
R2-D2 wrote:
Bob_Swede wrote:
What does it mean when there is a field identifier that ends with a : plus a number?
It means that field uses that number of bits from the storage type, and adjacent ones from the same type are merged into one. In the specific example, those fields comprise the 16 bits of a single word entry in the struct.

Thanks, then I understand, except not the bit order. Is it like this:
D15-D6=TPIdx
D5-D4=TunerNum
D3=DelFlag
D2=CASFlag
D1=LockFlag
D0=SkipFlag

or the other way around:

D0-D9=TPIdx
D10-D11=TunerNum
D12=DelFlag
D13=CASFlag
D14=LockFlag
D15=SkipFlag

_________________
Bo B (Sweden)
5700HDMIPVRt, TS On, F/W: patches (3.17 aka 5.14.77)
TAPs: TF5000 Display v1.53; ChannelChangeButtonHack 1.1; MyStuff 6.4; TAP Cmder 1.34; MyInfo B5.4;
(EPG: XMLTV2Mei for Win + MyStuff)
(Toppy access: ASUS WL-500g-P FTP installed using chunkywizard's guide)
Sig partly gen. by MyInfo on 12/4/2013
View user's profile Send private message
R2-D2
Posted: Fri Jan 29, 2010 3:18 pm Reply with quote
Frequent contributor Joined: 18 Dec 2006 Posts: 12148
Bob_Swede wrote:
Is it like this:
D15-D6=TPIdx
Yes, the Toppy is big endian. But be careful because your Windows machine is most likely little endian. Bit fields are a pain.

_________________
Troubleshooting -- User Manual -- Dark Side of the Matrix: Firmwares and Patches
View user's profile Send private message Visit poster's website
Bob_Swede
Posted: Fri Jan 29, 2010 4:17 pm Reply with quote
Frequent contributor Joined: 22 Oct 2007 Posts: 1066 Location: Sweden
Right now I am trraversing a recording file using HxD as editor and trying to match the header info to Firebird's document....
When that is done I will know how the TF5700 rec header is built up. It now looks like it uses the TF5000 service info record but has a longer (by 8 bytes) TPInfoTer record.
I am looking for the items that identify the recording files versus channel they came from so I can change them to what would be the new values after the Swedish channel reorganization on Feb 1.
If I don't do this I fear I will lose ALL of my extended info for the 150 or so recordings now on disk. Sad

As far as I have managed to deduce the way MyStuff matches recordings to the stored info is based on:
1) The service name (which these clowns will change around)
2) The timestamp of the recording file (not so good, would have been better to use the internal info from the recording header on the start time)
3) Something else like the channel number (the LCN is nowhere to be seen in the header)?
4) Then I guess MyStuff tries to match the information against its data on available channels from the toppy to find the LCN and therefore also the icon to display. Failing a match the information data seems to be immediately erased from the mei file so this is a one-off chance. If one misses out here all is lost!

Now, since they will change the service name as well as the channel for many existing services it will be quite some challenge to fix it without losing the descriptions.

I was hoping that it would be possible to just change the file where MyStuff keeps this extended data, but it seems like I am out of luck there.
So changing the header of existing rec files seems to be also required, unfortunately. Sad
Or maybe stuffing the right extended info into the header from the MyStuff file dat file? A problem when the MyStuff data is lost is that MyStuff displays the data inside the rec file, which in many cases is very limited and sometimes outright wrong.

_________________
Bo B (Sweden)
5700HDMIPVRt, TS On, F/W: patches (3.17 aka 5.14.77)
TAPs: TF5000 Display v1.53; ChannelChangeButtonHack 1.1; MyStuff 6.4; TAP Cmder 1.34; MyInfo B5.4;
(EPG: XMLTV2Mei for Win + MyStuff)
(Toppy access: ASUS WL-500g-P FTP installed using chunkywizard's guide)
Sig partly gen. by MyInfo on 12/4/2013
View user's profile Send private message
R2-D2
Posted: Fri Jan 29, 2010 5:54 pm Reply with quote
Frequent contributor Joined: 18 Dec 2006 Posts: 12148
Bob_Swede wrote:
It now looks like it uses the TF5000 service info record but has a longer (by 8 bytes) TPInfoTer record.
That's noted in the doc (Note 3), and follow the link in the latest item in the History (on the first page) to see what the (now known) extra entries are.

_________________
Troubleshooting -- User Manual -- Dark Side of the Matrix: Firmwares and Patches
View user's profile Send private message Visit poster's website
Bob_Swede
Posted: Sat Jan 30, 2010 12:08 am Reply with quote
Frequent contributor Joined: 22 Oct 2007 Posts: 1066 Location: Sweden
R2-D2 wrote:
Bob_Swede wrote:
It now looks like it uses the TF5000 service info record but has a longer (by 8 bytes) TPInfoTer record.
That's noted in the doc (Note 3), and follow the link in the latest item in the History (on the first page) to see what the (now known) extra entries are.

Yes, I noted the Note 3, but I did not realize that more info was available on an earlier page...
Anyway I am now well on my way to a program to decode the rec file header fully. So far only for the TF5700 of course since I don't have rec files for anything else.

_________________
Bo B (Sweden)
5700HDMIPVRt, TS On, F/W: patches (3.17 aka 5.14.77)
TAPs: TF5000 Display v1.53; ChannelChangeButtonHack 1.1; MyStuff 6.4; TAP Cmder 1.34; MyInfo B5.4;
(EPG: XMLTV2Mei for Win + MyStuff)
(Toppy access: ASUS WL-500g-P FTP installed using chunkywizard's guide)
Sig partly gen. by MyInfo on 12/4/2013
View user's profile Send private message
Bob_Swede
Posted: Sun Jan 31, 2010 5:39 pm Reply with quote
Frequent contributor Joined: 22 Oct 2007 Posts: 1066 Location: Sweden
Field value duplication?
I have noted when looking at a fair number of rec files that there is duplication in some header data and I wonder if this is a general rule or just a coincidence?

I found that:
ServiceID = PMTPID
PCRPID = VideoPID

I have not yet seen one rec file where this is not so, so I wonder if they are the same by "design"?
If I edit the rec file header and change the ServiceID, must I then also change the PMTPID etc?

_________________
Bo B (Sweden)
5700HDMIPVRt, TS On, F/W: patches (3.17 aka 5.14.77)
TAPs: TF5000 Display v1.53; ChannelChangeButtonHack 1.1; MyStuff 6.4; TAP Cmder 1.34; MyInfo B5.4;
(EPG: XMLTV2Mei for Win + MyStuff)
(Toppy access: ASUS WL-500g-P FTP installed using chunkywizard's guide)
Sig partly gen. by MyInfo on 12/4/2013
View user's profile Send private message
R2-D2
Posted: Sun Jan 31, 2010 5:53 pm Reply with quote
Frequent contributor Joined: 18 Dec 2006 Posts: 12148
Bob_Swede wrote:
if this is a general rule or just a coincidence?
It is not a rule (they are completely separate IDs) but a number of channels in the UK also do the same thing. And plenty do not.

_________________
Troubleshooting -- User Manual -- Dark Side of the Matrix: Firmwares and Patches
View user's profile Send private message Visit poster's website
iron
Posted: Mon Feb 01, 2010 11:57 pm Reply with quote
Regular contributor Joined: 03 Jun 2005 Posts: 91 Location: Tunbridge Wells, UK
Bob_Swede wrote:
If I edit the rec file header and change the ServiceID, must I then also change the PMTPID etc?

The PIDs reflect the values in the transport stream and to change them in the header would achieve nothing apart from perhaps making the file unplayable. It happens that the two you have mentioned are critical and should not be changed. It may interest you to know just how little of the header data is required by the PVR. For example if you were to zero everything except the ServiceType, ServiceID and the PMTPID the file would still play perfectly.

Examples
Original header up to the programme title
Code:
Offset(h) 00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F

00000000  54 46 72 63 50 10 00 00 00 1E 00 1A 00 00 00 00  TFrcP...........
00000010  15 30 12 3F 12 3F 02 6C 02 6C 02 6D 43 42 42 43  .0.?.?.l.l.mCBBC
00000020  20 43 68 61 6E 6E 65 6C 00 00 00 00 00 00 00 00   Channel........
00000030  00 00 00 00 00 00 00 00 00 37 08 00 00 0B 62 10  .........7....b.
00000040  10 0C 00 00 23 3A 30 11 80 00 00 1E 00 00 EB 60  ....#:0.?.....`
00000050  D7 B2 0A 00 D7 B2 0A 1E 04 0D 00 47 69 6D 6D 65  ײ..ײ.....Gimme
00000060  20 61 20 42 72 65 61 6B 50 65 61 72 73 6F 6E 20   a BreakPearson
<Remainder not shown>

Edited to leave just ServiceId(123F)and PMTPID(123F), ServiceType is 0 for TV.
Code:
Offset(h) 00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F

00000000  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
00000010  00 00 12 3F 12 3F 00 00 00 00 00 00 00 00 00 00  ...?.?..........
00000020  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
00000030  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
00000040  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
00000050  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
00000060  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
<zeroed to offset 0x0EB0>

iron.

_________________
Model: TF5800PVR - Silver Firmware: Recommended_5800; TAPS: UK Timer 1.2, USB Accelerator 1.0, eit_sub, seccacheUK; cables: 10m TF to PC only, 2 metres both ways!
View user's profile Send private message
idheath
Posted: Sun Feb 12, 2012 10:24 pm Reply with quote
Joined: 12 Jun 2005 Posts: 19
Just wondering if there's been any update on this activity (since I'm trying to find a Mac OS X application to do this).
View user's profile Send private message

Display posts from previous:  

All times are GMT + 1 Hour
Page 1 of 1

Jump to:  

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum