Click here to go to the forum index Click here for the home page
 
Author Message

<  TAP and patch development  ~  Another speed up -- [Es] EIT Section Cache

Page 1 of 4
Goto page 1, 2, 3, 4  Next
R2-D2
Posted: Sat May 17, 2008 2:29 pm Reply with quote
Frequent contributor Joined: 18 Dec 2006 Posts: 12148
The TF58x0 models also have a EIT Section Cache which is supposed to stop the Toppy repeatedly processing the EIT data. On unpatched firmwares it doesn't really achieve this, but the [Ec] patch fixes the intended behaviour.

However, the cache is rubbish. Smile It stores the CRCs of up to 14,000 sections (or 16,000 on 5.14.X and 5.15.X), and linearly searches through them to find matching ones. I have a feeling that this sort of processing is (once it's fairly full) almost as bad as processing the section anyway.

First thought was to speed up the access by keeping it sorted or something. But then I realised that it was keeping CRCs for old sections with different version_numbers so just reorganising the layout would maybe work better, keeping a simple array of version_number and CRC for each of the 256 possible section_numbers for each service. Using this strategy allows up to 130 different services (or 149 on 5.14/5.X), which is more than adequate for Freeview (79 now?).

A quick profiling shows that this is normally the 4th most expensive routine (after MyStuff, the Idle mechanism, and lots of things releasing interrupts) and ended up 52nd (and insignificant) when optimised. (The %age of (profilable) CPU usage dropped for this routine from 3.1% to 0.1% -- we're now getting to the stage where idle TAPs and all the hidden routines [which disable interrupts] need to be investigated!)

So [Es] EIT Section Cache enables this optimisation. I'd say it was a little bit experimental, since I've made an assumption about the version_number thing. But the data seem to whiz in. And I can't really do longer term tests with the Toppy left on since I'm forever switching it off for one reason or another. Smile

Edit: use in conjunction with [Ec] V3, which is the same as V2 but elides the fix that [Es] removes the need for.


Last edited by R2-D2 on Tue May 20, 2008 10:05 pm; edited 1 time in total

_________________
Troubleshooting -- User Manual -- Dark Side of the Matrix: Firmwares and Patches
View user's profile Send private message Visit poster's website
chunkywizard
Posted: Sat May 17, 2008 3:28 pm Reply with quote
MyStuff Team Joined: 13 May 2005 Posts: 9319 Location: Sindlesham, Berkshire Tx: Hannington
Hi R2

This isn't another of your 1340 specific firmwares is it? Wink it says not intended for my firmware.

CW

_________________
Toppy:TF5800 F/W: 1365PaTcHeD
TAPs: MyStuff | EPG2MEI | FontManager | SecCache UK | Extend | FastScanGUI | EIT_sub_game | MyInfo | ScreenCapture with OSD | MHEG On/Off
MyStuff Manual & Logos | Known Bugs | Recommended F/W
View user's profile Send private message Visit poster's website
glenmcfar
Posted: Sat May 17, 2008 3:29 pm Reply with quote
Frequent contributor Joined: 07 Dec 2006 Posts: 4519 Location: Dundonald, Ayrshire, Scotland
I'm getting 404 not found when i click on the link.

Glen.

_________________
H/W: TF5800 | URC-7555 | Asus | Best Firmware Ever!
A/S: SecCacheUK, EitSub, EPG2MEI, Display, Extend, QuickJump, FontManager, TapLauncher, MyStuff
T/L: TapCommander, Surfer, MeiSearch, MediaManager | HDFW, CutAds, Sudoku
View user's profile Send private message
R2-D2
Posted: Sat May 17, 2008 3:32 pm Reply with quote
Frequent contributor Joined: 18 Dec 2006 Posts: 12148
chunkywizard wrote:
it says not intended for my firmware.
You can't apply it dynamically, and it won't apply to a firmware which has the [Ec] patch applied (because they patch the same area).

Anyway, I've taken it offline for a bit because there was a small de-optimisation in there. Smile

_________________
Troubleshooting -- User Manual -- Dark Side of the Matrix: Firmwares and Patches
View user's profile Send private message Visit poster's website
glenmcfar
Posted: Sat May 17, 2008 3:34 pm Reply with quote
Frequent contributor Joined: 07 Dec 2006 Posts: 4519 Location: Dundonald, Ayrshire, Scotland
R2-D2 wrote:
You can't apply it dynamically, and it won't apply to a firmware which has the [Ec] patch applied (because they patch the same area).


CW - part of the fun is having to guess what he doesn't tell you Twisted Evil

Glen.

_________________
H/W: TF5800 | URC-7555 | Asus | Best Firmware Ever!
A/S: SecCacheUK, EitSub, EPG2MEI, Display, Extend, QuickJump, FontManager, TapLauncher, MyStuff
T/L: TapCommander, Surfer, MeiSearch, MediaManager | HDFW, CutAds, Sudoku
View user's profile Send private message
chunkywizard
Posted: Sat May 17, 2008 3:34 pm Reply with quote
MyStuff Team Joined: 13 May 2005 Posts: 9319 Location: Sindlesham, Berkshire Tx: Hannington
I tried applying it to a bare 1365 and it still says not intended for this firmware.

CW

[edit] erm perhaps I'm talking rubbish. Just tried again and it now seems fine. So does the Es patch include the Ec fix as well?

_________________
Toppy:TF5800 F/W: 1365PaTcHeD
TAPs: MyStuff | EPG2MEI | FontManager | SecCache UK | Extend | FastScanGUI | EIT_sub_game | MyInfo | ScreenCapture with OSD | MHEG On/Off
MyStuff Manual & Logos | Known Bugs | Recommended F/W
View user's profile Send private message Visit poster's website
R2-D2
Posted: Sat May 17, 2008 3:49 pm Reply with quote
Frequent contributor Joined: 18 Dec 2006 Posts: 12148
chunkywizard wrote:
So does the Es patch include the Ec fix as well?
Well, yes, I didn't replicate their bug in the new code, if that's what you mean. Smile

Modified version is now up, which is fully optimised (Laughing). So it's worth re-downloading.

_________________
Troubleshooting -- User Manual -- Dark Side of the Matrix: Firmwares and Patches
View user's profile Send private message Visit poster's website
glenmcfar
Posted: Sat May 17, 2008 4:10 pm Reply with quote
Frequent contributor Joined: 07 Dec 2006 Posts: 4519 Location: Dundonald, Ayrshire, Scotland
How much memory will the EIT data use with all these EIT patches applied? Same as before, or less? I remember that there is 3.5MB reserved but this can be altered with your E0/1/2 patches. Might be handy to save a meg or two, especially when MeiSearch is running Laughing

Glen.

_________________
H/W: TF5800 | URC-7555 | Asus | Best Firmware Ever!
A/S: SecCacheUK, EitSub, EPG2MEI, Display, Extend, QuickJump, FontManager, TapLauncher, MyStuff
T/L: TapCommander, Surfer, MeiSearch, MediaManager | HDFW, CutAds, Sudoku
View user's profile Send private message
chunkywizard
Posted: Sat May 17, 2008 4:20 pm Reply with quote
MyStuff Team Joined: 13 May 2005 Posts: 9319 Location: Sindlesham, Berkshire Tx: Hannington
All loaded up and my Toppy still seems to work Wink any tests you want me to try?

CW

_________________
Toppy:TF5800 F/W: 1365PaTcHeD
TAPs: MyStuff | EPG2MEI | FontManager | SecCache UK | Extend | FastScanGUI | EIT_sub_game | MyInfo | ScreenCapture with OSD | MHEG On/Off
MyStuff Manual & Logos | Known Bugs | Recommended F/W
View user's profile Send private message Visit poster's website
R2-D2
Posted: Sat May 17, 2008 4:20 pm Reply with quote
Frequent contributor Joined: 18 Dec 2006 Posts: 12148
glenmcfar wrote:
How much memory will the EIT data use with all these EIT patches applied?
These caches are quite small, and they're also statically allocated so cannot really be reused. This new patch just uses the same memory (well, slightly less than all of it) but with the data structure completely reorganised. (The EPG memory patches alter the text buffers for storing the long descriptions.)

The EIT data seems a bit strange to me: they seem to reliably be using section_numbers 0,1,8,16,24,...,248 (with the 1 being out of sequence, and gaps that I wasn't expecting). Confused Odd.

_________________
Troubleshooting -- User Manual -- Dark Side of the Matrix: Firmwares and Patches
View user's profile Send private message Visit poster's website
R2-D2
Posted: Sat May 17, 2008 4:34 pm Reply with quote
Frequent contributor Joined: 18 Dec 2006 Posts: 12148
chunkywizard wrote:
any tests you want me to try?
The only significant things that occur to me are whether it still catches the update in the early AM (I'm sure it will) and whether it gets a bit sluggish on changing channels [muxes]. I think they aren't consistent over version_numbers across muxes so it's most likely that this will cause a small overhead (which may not be noticeable!). In many ways I think this will be preferable, leaving the other parts of the system to identify new or changed events (which it can do properly with the other EIT fixes), rather than wasting a huge amount of time with the stupidly slow cache (what's the antonym?).

_________________
Troubleshooting -- User Manual -- Dark Side of the Matrix: Firmwares and Patches
View user's profile Send private message Visit poster's website
andyrogers
Posted: Sat May 17, 2008 5:04 pm Reply with quote
Frequent contributor Joined: 07 Dec 2005 Posts: 814
R2

Does this [Es] patch technically replace the [Ec] patch, but includes the improvemetns of the EC patch in the Es patch as well?

Thanks

Andy

_________________

Firmware: 5.14.09 Patched
AutoStart Taps: MyStuff v5.52b, eit2mei beta 7.8m7, , Power Manager v1.2, Tap Commander v1.32, TF5000 Display v1.53a, Discription Extender v2.3, SDS 1.3b, QuickJump 1.72
Other Taps: Sudoka, mei2archive, mei2eit, snake, meisearch
View user's profile Send private message
R2-D2
Posted: Sat May 17, 2008 5:12 pm Reply with quote
Frequent contributor Joined: 18 Dec 2006 Posts: 12148
andyrogers wrote:
Does this [Es] patch technically replace the [Ec] patch
Yes. The [Ec] patch fixed a bug in Topfield's version of the cache. [Es] implements a new way of doing the caching, and so replaces Topfield's code, making [Ec] redundant.

_________________
Troubleshooting -- User Manual -- Dark Side of the Matrix: Firmwares and Patches
View user's profile Send private message Visit poster's website
aramil
Posted: Sat May 17, 2008 11:01 pm Reply with quote
Frequent contributor Joined: 20 Jun 2006 Posts: 649 Location: Worcester, UK ; TX:Malvern Central
Ok uploaded and applied i will let you know if there are any issues but as always nice and smooth Smile

_________________
Toppy: TF5810; 500Gb spinpoint HDD (Lycom IDE-Sata);Fan Cooled; AI=Off, TS=On;
Firmware: MS6 Recommended F/W 12/9/2009 +CbCfCtDFsRtScStTaUUcUuUxVbXpZ;
PC OS: Fedora 14-64bit / Windows 8 Pro-64bit; TAPs: SecCache (UK) v0.4; EIT Sub (Game) v0.6; EPG2MEI v0.96; Font Manager 1.0d; MyStuff 6.4; Extend v1.7; MyInfo 5.6RC1; MHEG On/Off A3.
View user's profile Send private message
mike.hinson
Posted: Sat May 17, 2008 11:38 pm Reply with quote
Frequent contributor Joined: 18 Nov 2005 Posts: 1735 Location: Bordon, Hants UK
I am Es instead of Ec now Smile

edit: this is wrong information.. both are needed.


Last edited by mike.hinson on Mon Jun 09, 2008 9:32 pm; edited 1 time in total
View user's profile Send private message

Display posts from previous:  

All times are GMT + 1 Hour
Page 1 of 4
Goto page 1, 2, 3, 4  Next

Jump to:  

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum