Click here to go to the forum index Click here for the home page
 
Author Message

<  MyStuff  ~  EPG2MEI -- new event API?

Page 2 of 39
Goto page Previous  1, 2, 3, ... 37, 38, 39  Next
R2-D2
Posted: Thu Jan 08, 2009 11:13 am Reply with quote
Frequent contributor Joined: 18 Dec 2006 Posts: 12148
EPG2MEI now deletes any duplicate event it finds in the same slot (i.e. same starting time). It can also adjust end times based on the assumption that these are not spotted by the firmware. However, something that can't easily be detected is when an existing pair of events is replaced by a single event. In this case, the firmware will keep an overlapping pair of events (or all three!) but EPG2MEI will shorten the (new) first event so they don't overlap (and discard the old first event).

Some firmwares do actually spot changes to event durations (notably the Freeview+ ones), so they've obviously realised this bug at some point.

_________________
Troubleshooting -- User Manual -- Dark Side of the Matrix: Firmwares and Patches
View user's profile Send private message Visit poster's website
R2-D2
Posted: Mon Jan 12, 2009 2:17 pm Reply with quote
Frequent contributor Joined: 18 Dec 2006 Posts: 12148
EPG2MEI seemed to cope a bit with the Darts overrunning on BBC2 (I assume it overran... I was just watching the serial log spotting Top Gear being altered). But it was far from perfect. I think therefore that (together with other issues) any sort of fiddling with EIT events after the fact (i.e. by EPG2MEI during a scan) is not going to work properly. The EIT collector's basic problem of not spotting changes to the end time of an event needs fixing. As does it's very bad habit of adding new events to overlap old ones. (I have a feeling that [X] might help with the description changes...)

On a related note, I've finally figured out why some classes of events go AWOL. It seems that the firmware sets up the Other P/f and Schedule handlers pretty much once, but the Actual P/f and Schedule handlers will tend to follow the mux used for the main service (well, the main Path actually). The Other handlers only get changed when you exit certain menus, so there is a strong likelihood that they will become out of sync with the Actual handlers, resulting in incomplete coverage. This will usually manifest itself a while after the Toppy has booted, once it's done things to jig about the Paths like make recordings. That's part of the reason why it's been hard to spot, as well as the fact that when they are out of sync the difference (well, the critical one) is missing P/f events for a single mux -- which makes it look like the Actual P/f has gone wrong when it hasn't (it's just doubling up the events already being collected as Other P/f on a different mux [Path]).

One approach may be to make a version of [Pf] that uses a totally static path allocation (PathA would work fine) and disables lots of competing (and unnecessary) EIT collectors -- taking care not to disturb [He]. I would be leaning more towards dumping the native EIT collectors entirely if it weren't for the fact that the TF5810 doesn't like that at all...

_________________
Troubleshooting -- User Manual -- Dark Side of the Matrix: Firmwares and Patches
View user's profile Send private message Visit poster's website
chris_s
Posted: Mon Jan 12, 2009 2:40 pm Reply with quote
Frequent contributor Joined: 23 May 2008 Posts: 403
I was just about to post a new thread about still missing the odd recording, and it seeming to be related to EIT collection during playback (drop out of playback and the recording(s) start up there and then). I have been getting a little confused reading and trying to understand the [Pf] thread, and wondering whether I should try [Pf] on my 5810, and here you are talking about modifying [Pf], so shall I just wait to see what comes out of these latest investigations?

_________________
TF5810, F/W: MS6 Recommended F/W 12/9/2009
TAPs: QuickJump 1.72; MPDisplayLITE V1.2; QuickBlank 0.2; SecCache (UK) v0.4; EIT Sub (Game) v0.6; MyInfo B5.6; EPG2MEI v0.96; MyStuff 6.5; Font Manager 1.0d; Extend v1.7; MHEG On/Off A3; (Start Channel LCN);
Sig generated by MyInfo on 14/3/12
,
TV=Panasonic TX-32DT2 IDTV (32" CRT via CVBS-SCART), Tx: Sutton Coldfield. "My other PVR is a TiVo".
View user's profile Send private message
Tactilis
Posted: Mon Jan 12, 2009 3:34 pm Reply with quote
Regular contributor Joined: 31 Dec 2006 Posts: 53
Chris

Your signature is very long and it forces the message area to be wider than the available screen width in my browser.

This means that I have to repeatedly scroll the window right and left to read each line of your text.

I suspect that other people have the same difficulty too depending on the resolution of their screens.

Would it be possible to break the patch line of your signature into two lines to avoid this problem?


Martin
View user's profile Send private message
juwlz
Posted: Mon Jan 12, 2009 3:52 pm Reply with quote
MyStuff Team Joined: 12 Aug 2005 Posts: 10801 Location: Wokingham, Berkshire (Hannington transmitter)
Tactilis wrote:
Chris

Would it be possible to break the patch line of your signature into two lines to avoid this problem?
Or just lose all the commas and underscores in the patch list?

Or have a link to SimonC's decoder instead?

Julie

_________________
5800, TS On, F/W: MS6 Recommended F/W 12/9/2009 -Sy+BmC0CfCtFsIMPePsR3UUuUxZ
5810, TS On, F/W: F/W: MS6 Recommended F/W 12/9/2009 +BmCfCtFsR3Z
MyStuff info; Help!; Reference
Harmony 885 remote. Sig date 8 April 2012
View user's profile Send private message Visit poster's website
chris_s
Posted: Mon Jan 12, 2009 3:55 pm Reply with quote
Frequent contributor Joined: 23 May 2008 Posts: 403
Tactilis wrote:
I suspect that other people have the same difficulty too depending on the resolution of their screens.

Would it be possible to break the patch line of your signature into two lines to avoid this problem?


I have noticed that other peoples sigs can either fit Ok in my browser, or appear in quite small fonts - don't really understand it myself, but I can put a break in for now.

I will have a go at juwlz's suggestions later.

Chris.

_________________
TF5810, F/W: MS6 Recommended F/W 12/9/2009
TAPs: QuickJump 1.72; MPDisplayLITE V1.2; QuickBlank 0.2; SecCache (UK) v0.4; EIT Sub (Game) v0.6; MyInfo B5.6; EPG2MEI v0.96; MyStuff 6.5; Font Manager 1.0d; Extend v1.7; MHEG On/Off A3; (Start Channel LCN);
Sig generated by MyInfo on 14/3/12
,
TV=Panasonic TX-32DT2 IDTV (32" CRT via CVBS-SCART), Tx: Sutton Coldfield. "My other PVR is a TiVo".
View user's profile Send private message
R2-D2
Posted: Mon Jan 12, 2009 4:29 pm Reply with quote
Frequent contributor Joined: 18 Dec 2006 Posts: 12148
chris_s wrote:
wondering whether I should try [Pf] on my 5810
On the TF5810 it's less important, since the main reason for considering [Pf] (V1) at present is if you want EIT collection to work during playback, which it should already do on 5.15.X.

_________________
Troubleshooting -- User Manual -- Dark Side of the Matrix: Firmwares and Patches
View user's profile Send private message Visit poster's website
R2-D2
Posted: Fri Jan 16, 2009 7:22 pm Reply with quote
Frequent contributor Joined: 18 Dec 2006 Posts: 12148
Finally got fed up trying to cajole the native mess of EIT collectors into something usable, and have now embarked on the alternative which dumps the lot, while keeping the [He] mechanism.

Another step forward has been made in making a patch to fix the TF5800 firmwares to also check the end time of previously added events -- I was getting distracted by the myriad variations across all firmwares, and then distracted by a subtle omission in the replacement code.

That step has solved a lot of issues, and has now allowed EPG2MEI to tidy up the native EPG. I was considering a patch to extend existing link scans to mop up duplicate/old events, but in the end it seems sensible to leave that up to EPG2MEI, during its own scans (since that's triggered by detected changes).

_________________
Troubleshooting -- User Manual -- Dark Side of the Matrix: Firmwares and Patches
View user's profile Send private message Visit poster's website
R2-D2
Posted: Wed Jan 21, 2009 11:02 pm Reply with quote
Frequent contributor Joined: 18 Dec 2006 Posts: 12148
More observations:
  • The firmware is constantly flip-flopping between adding an event and deleting it (with SecCache this sort of activity will tend to be limited to when changing channels).
  • The main reason for this is because the events are "old", which means they ended more than 1 hour ago.
  • Events that are start more than 8 days in the future are also added then immediately deleted.
  • Whenever a new event becomes marked as "running", the old event is unceremoniously deleted. It'll probably be added again later (especially on channel change)... then deleted again when it's more than 1 hour old.
  • Currently, there are 11,752 events in broadcast EIT, and the firmware can hold 14,000. This covers (at best) the previous day's worth of data and the next 8 days.
I think adjusting the bounds to fit the broadcast data better won't cause any (more) issues, and will prevent the flip-flopping.

_________________
Troubleshooting -- User Manual -- Dark Side of the Matrix: Firmwares and Patches
View user's profile Send private message Visit poster's website
R2-D2
Posted: Fri Jan 23, 2009 7:40 pm Reply with quote
Frequent contributor Joined: 18 Dec 2006 Posts: 12148
With a handful of new patches, EPG2MEI seemed to cope quite well with the changes yesterday lunchtime on BBC2 due to the overrunning tennis. I've got a log that shows Daily Politics and Working Lunch moving around quite a bit. (In the end they decided to record Working Lunch and not do it live.)

Today was probably a bit more normal. Since this morning, EPG2MEI has done the following:
Code:
0900, initial scan, 11766 events (plus 916 old events, to go back 24hrs)
          15 duplicate events on 46 (smileTV 2) deleted
0922, regular update due to 727 (Absolute Radio) changing every event_id
          69 duplicate events deleted (on 727)
1044, update due to a handful of changes on 7/21/81/728
          9 duplicate events deleted (mainly on 728)
1200, update due to a handful of changes on ditto plus a few others
          3 duplicate events deleted
1747, update due to changes on lots of channels, now 11769 events
          103 duplicate events deleted (mainly on 22 and 97)
1800, update due mainly to 16 (QVC) changing schedule, now 11770 events
          19 duplicate events deleted (on 16)
Obviously there are a lot of changes (mainly additions) overnight, but I'm quite encouraged that once I've added a way to exclude certain channels there will be very few updates in a normal day, even with the silly things the broadcasters do (and the updates look like they will be timely enough for things like MyStuff to react).

I've made a patch to enable the internal UTC time to continue to be updated from the broadcast signals during playback, because it's used by EPG2MEI to judge whether a change is significant enough to prompt a scan. But I've seen that sometimes the collector just dies -- I think this is the same issue as when the collector used by [He]/[Ab] dies, so you lose bookmarks. I think it's most likely to be buffer overflows in one (or some) of the BIOS call mechanism, the inter-task message passing, or the main task event queue. [Pe] may be preventing this leaking memory but it clearly isn't helping enough to keep certain bits alive. In essence, this area is thoroughly broken -- I think the individual parts are OK, but it's the way they've been put together that is broken.

_________________
Troubleshooting -- User Manual -- Dark Side of the Matrix: Firmwares and Patches
View user's profile Send private message Visit poster's website
R2-D2
Posted: Fri Jan 23, 2009 11:41 pm Reply with quote
Frequent contributor Joined: 18 Dec 2006 Posts: 12148
Bah! Just been stressing the Toppy during recording and managed to trip the stuck EIT collector (so Extend kicked in due to lost end signal). During this time I had monitors on all the bits I thought might be failing... but alas no, nothing popped up. The EIT sections with the Now&Next info had been collected and were sat waiting to be delivered -- [Pe] should have caused these to be deallocated if the queues were full (or getting that way), but the memory was kosher. Sad

_________________
Troubleshooting -- User Manual -- Dark Side of the Matrix: Firmwares and Patches
View user's profile Send private message Visit poster's website
R2-D2
Posted: Sat Jan 24, 2009 11:32 am Reply with quote
Frequent contributor Joined: 18 Dec 2006 Posts: 12148
Hmm... false alarm, I think. I changed (improved) the monitoring and removed all the patches which might be masking the problem, and it's only the last step in the event prioritising/mangling which causes the issue (as discovered previously, and which [Pe] attempts to alleviate). So now it's probably just a matter of trying to adjust [Pe] to work better in this regard.

_________________
Troubleshooting -- User Manual -- Dark Side of the Matrix: Firmwares and Patches
View user's profile Send private message Visit poster's website
R2-D2
Posted: Sat Jan 24, 2009 9:04 pm Reply with quote
Frequent contributor Joined: 18 Dec 2006 Posts: 12148
99.9% of the messages which are actually causing the issue end up (eventually) as NOPs, thanks to SecCache. So it seemed sensible to try to plug SecCache in at the lowest level (rather than it sitting at the top level) and reject EIT sections before they get passed from the DVB section handler task to the main task. It's maybe a bit naughty since it slows down that task (due to the necessary CRC calculations and checking the cache), but it's not an interrupt handler so it feels like it might be OK. Indeed, it does appear to solve the issue, even without a helping hand from the sort of thing that [Pe] does.

(This is very tiring.)

_________________
Troubleshooting -- User Manual -- Dark Side of the Matrix: Firmwares and Patches
View user's profile Send private message Visit poster's website
andyrogers
Posted: Sat Jan 24, 2009 9:10 pm Reply with quote
Frequent contributor Joined: 07 Dec 2005 Posts: 814
Hi R2

Does it sound like you are nearing a public Alpha/Beta release of epg2mei?

It sounds like it is progressing very well.

Andy

_________________

Firmware: 5.14.09 Patched
AutoStart Taps: MyStuff v5.52b, eit2mei beta 7.8m7, , Power Manager v1.2, Tap Commander v1.32, TF5000 Display v1.53a, Discription Extender v2.3, SDS 1.3b, QuickJump 1.72
Other Taps: Sudoka, mei2archive, mei2eit, snake, meisearch
View user's profile Send private message
R2-D2
Posted: Sat Jan 24, 2009 10:46 pm Reply with quote
Frequent contributor Joined: 18 Dec 2006 Posts: 12148
I might deliberately stop it working on 5.14.X, though. Smile

_________________
Troubleshooting -- User Manual -- Dark Side of the Matrix: Firmwares and Patches
View user's profile Send private message Visit poster's website

Display posts from previous:  

All times are GMT + 1 Hour
Page 2 of 39
Goto page Previous  1, 2, 3, ... 37, 38, 39  Next

Jump to:  

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum