For discussion of firmware updates, enhancement requests, and other upgrades to your Toppy, but not TAPs or computer-related

Moderators: Technical, Oz mods

Post Reply
DeadBeef
Frequent contributor
Posts: 264
Joined: Mon Jan 09, 2006 7:28 pm

USB Accelerator

Post by DeadBeef »

USB Accelerator

The TAP and the patch accelerate USB transfers by suppressing the calculation of the CRC16 checksum for data blocks being transferred from PVR to PC.

Results measured with a 1617870848-byte file transferred from a TF5000PVR and a 2-year old PC (Athlon) running Windows XP:
  • w/o modification: 725265 msec => 2.2 MB/s
  • with modification: 479344 msec => 3.37 MB/s
It corresponds to a gain of approximately 50%. (This may vary depending on the firmware version, PVR model and the PC performance.)

In order to benefit from the USB Accelerator you should either install the patch or run the TAP once before starting to transfer the data after restart.
Both do basically the same thing. The patch modifies the firmware permanently. The TAP modifies the firmware in the RAM temporarily (i.e. until next restart).

The modified firmware can only be deployed with tools tolerating the protocol deviation mentioned above. Currently the following tools can be used with modified firmware: The original TfDll cannot be used because it always expects a correct CRC16 checksum and would abort the transfer.

The CRC16 calculation is dispensable because USB chips ensure the data integrity by deploying the CRC16.

The patched TfDll also accepts data packets with valid CRC16.

Limitations

The modification is only applied to transfers from PVR to PC. The opposite direction remains as is.

Disclaimer

The use of the software is done at your own discretion and risk and with agreement that you will be solely responsible for any damage to your computer system or loss of data that results from such activities. The author assumes no responsibility for errors.
Even though many tests have shown that the transferred data is not corrupted it is still possible that the transferred data may be unusable.

Download

USB Accelerator 1.0
Last edited by DeadBeef on Mon Jun 04, 2007 6:29 pm, edited 2 times in total.
R2-D2
Frequent contributor
Posts: 12148
Joined: Mon Dec 18, 2006 11:15 am
Contact:

Re: USB Accelerator

Post by R2-D2 »

Another stunner from DeadBeef!
DeadBeef wrote:The CRC16 calculation is dispensable because USB chips ensure the data integrity by deploying the CRC16.
They really don't know what they're doing, do they? :roll: I suppose this is something brought over from the serial comms.

Thank you for your hard work on this -- I can't wait to try it out!
HydeTheDarkerSide
Frequent contributor
Posts: 5956
Joined: Wed May 11, 2005 10:17 pm
Location: Hannington Transmitter : Sony KDL 40Z5800
Contact:

Post by HydeTheDarkerSide »

Downloaded and ready to be copied over to the Toppy. Fantastic work indeed :D 8)

Many thanks

Before I go and test this myself, just wondering if you've already covered off my next question...

Is this performance increase the same %'ish in both Turbo and non-Turbo modes?
Hyde.
[size=10:da1fd20a33][b:da1fd20a33]2x TF5800 All controlled with Harmony Ultimate [/b:da1fd20a33], TS On, F/W: MS6 Recommended F/W 12/9/2009 -RSy+BmC0CbCfCtDsEgEmEvFsGIMPePsR0ScUUuWfXZ
TAPs: PcControl B1.3; EPG2MEI v0.96; Font Manager 1.0d; Extend v1.7; SecCache (UK) v0.4; EIT Sub (Game) v0.6; MyInfo B5.6; MyStuff 6.5 RC2;
[color=blue:da1fd20a33]MyStuff Links: http://www.toppy.org.uk/~mystuff/index.shtml[/color:da1fd20a33]
Sig generated by MyInfo on 11/10/13[/size:da1fd20a33]
EMJB
Frequent contributor
Posts: 3645
Joined: Fri Jul 08, 2005 9:43 am
Location: Maldon Essex

Post by EMJB »

HydeTheDarkerSide wrote:Is this performance increase the same %'ish in both Turbo and non-Turbo modes?
Somewhat of a red herring, but I find non-turbo mode leads to crashes on large file with 5.13.55, so never use it for anything other than TAP loading, and transferring ini files, screen dumps, etc. However I suppose there is a remote chance this will fix that crashing too!

EMJB
glenmcfar
Frequent contributor
Posts: 4519
Joined: Thu Dec 07, 2006 1:38 am
Location: Dundonald, Ayrshire, Scotland

Post by glenmcfar »

Can I just ask if this is patched from Aldarin's Vista-friendly dll, or the original Topfield Vista-nasty dll?
H/W: TF5800 | URC-7555 | Asus | Best Firmware Ever!
A/S: SecCacheUK, EitSub, EPG2MEI, Display, Extend, QuickJump, FontManager, TapLauncher, MyStuff
T/L: TapCommander, Surfer, MeiSearch, MediaManager | HDFW, CutAds, Sudoku
DeadBeef
Frequent contributor
Posts: 264
Joined: Mon Jan 09, 2006 7:28 pm

Post by DeadBeef »

HydeTheDarkerSide wrote:Is this performance increase the same %'ish in both Turbo and non-Turbo modes?
I don't think so. Beta testers reported a variety of numbers in different modes and with different equipment.
glenmcfar wrote:Can I just ask if this is patched from Aldarin's Vista-friendly dll, or the original Topfield Vista-nasty dll?
I patched the original Topfield DLL. I assume Aldarin is going to release required changes soon.
R2-D2
Frequent contributor
Posts: 12148
Joined: Mon Dec 18, 2006 11:15 am
Contact:

Post by R2-D2 »

A quick test and it works great! (I'm wondering now if there's even more than could be pushed out of the firmware -- although I thought the EMMA2 chipset had a limit on the USB at 1.2Mbps...)

Major things that need an update to cope with this are puppy and ftpd-topfield.

Oh, and is it my imagination or did replacing the TfDll.dll give an improvement all by itself? :)
jumbo
Frequent contributor
Posts: 4731
Joined: Mon Apr 11, 2005 5:25 pm

Post by jumbo »

Since CRC16 is not performed does it mean that it is possible that transferred files may contain incorrect data? Or is it not needed in this case anyway.
DeadBeef
Frequent contributor
Posts: 264
Joined: Mon Jan 09, 2006 7:28 pm

Post by DeadBeef »

R2-D2 wrote:A quick test and it works great! (I'm wondering now if there's even more than could be pushed out of the firmware -- although I thought the EMMA2 chipset had a limit on the USB at 1.2Mbps...)
PLX (NetChip) specified in an application note for uPD61130 a maximum transfer rate of 11 MB/s. We are at approximately 25% of it.

In my opinion the major bottleneck in the Topfield's USB protocol is the acknowledgement handling. I assume the transfer rate could be twice as high if the ACKs could be omited or being handled similar to the moving window like in TCP.
R2-D2 wrote: Oh, and is it my imagination or did replacing the TfDll.dll give an improvement all by itself? :)
It depends on the version you used before.
Since CRC16 is not performed does it mean that it is possible that transferred files may contain incorrect data? Or is it not needed in this case anyway.
Theoretically transferred files may contain invalid data due to omission of the CRC16 check in the software. Since the data integrity is guaranteed by the USB specification and USB hardware implementation it is very improbable.
R2-D2
Frequent contributor
Posts: 12148
Joined: Mon Dec 18, 2006 11:15 am
Contact:

Post by R2-D2 »

jumbo wrote:Since CRC16 is not performed does it mean that it is possible that transferred files may contain incorrect data?
No, the USB protocol includes CRC checks, so it's completely unnecessary. Like I mentioned above, I think it's just been brought over from the serial comms code. (Where's the DOH! emoticon?)
danb77
Frequent contributor
Posts: 763
Joined: Sun Nov 20, 2005 7:08 pm

Re: USB Accelerator

Post by danb77 »

DeadBeef wrote: [*]MacTF (with slightly changed settings)
What are the changes which need to be made?
TF5800, IA On, TS On, F/W: 13.65AbBfBmBqC0CeCkCpCwCyDeEcEeEfEpErEsEvFFlFmFsGmHHeIKtMMhNfOtPPcPePfPsRRaReRhRpRsSScSdSlSrStSyT2TaTdTeTfTpTsUUuUxVrWfXpXwZXl
TAPs: SDS V1.3d; Extend v1.7; TF5000 Display v1.53; QuickJump 1.69; MyStuff 6.0; Tap Launcher 3.10; Font Manager 1.0d; TAP Commander 1.34; WSSkiller V2.12d; Power Manager v1.2; UK Subtitle 1.8; SecCache (UK) v0.4; EIT Sub v0.6; MyInfo B4.0; EPG2MEI v0.93e;
Sig generated by MyInfo TAP on 21/9/09
malc
Frequent contributor
Posts: 1380
Joined: Sun May 08, 2005 9:12 am
Contact:

Post by malc »

If the USB chips include CRC then how come the humax USB transfer is so error prone? It is meant to be a weak point, so much so that someone has written a program to compare 2 downloaded files and make one good one. If usb is inherrently error free why isn't it for the humax?
TF5800, F/W: 5.13.65T 14/4/2009 -SyTeXp+CwEmMOtR2ReScTdUUuXZXl
TAPs: Archive v1.0a; TAP Commander 1.34; UK Auto Scheduler v0.73; UK Timers v1.2a; QuickJump 1.72; Matrix Screensaver V1.7; TF5000 Display v1.53; PruneEPG 1.0; MyInfo B5.6;
Sig generated by MyInfo on 10/4/19
DeadBeef
Frequent contributor
Posts: 264
Joined: Mon Jan 09, 2006 7:28 pm

Post by DeadBeef »

malc wrote:If the USB chips include CRC then how come the humax USB transfer is so error prone? It is meant to be a weak point, so much so that someone has written a program to compare 2 downloaded files and make one good one. If usb is inherrently error free why isn't it for the humax?
The physical link is not the only error source. The data might be lost or corrupted in the software (e.g. firmware or Windows device driver). For more information please refer to USB in a NutShell, section Data Packets.
malc
Frequent contributor
Posts: 1380
Joined: Sun May 08, 2005 9:12 am
Contact:

Post by malc »

Useful link. I can well imagine that the h/w works out the crc. I assume this flags any errors and the driver software resends rather than the h/w being responsible for resend errors. Especially since the data packet can be 1k.

The humax problem might be a poor driver. But if that is the case are you sure that the toppy driver resends bytes in error or does it ONLY look at the CRC it includes in the data block. If so then you could end up with something as unreliable as the humax usb.
TF5800, F/W: 5.13.65T 14/4/2009 -SyTeXp+CwEmMOtR2ReScTdUUuXZXl
TAPs: Archive v1.0a; TAP Commander 1.34; UK Auto Scheduler v0.73; UK Timers v1.2a; QuickJump 1.72; Matrix Screensaver V1.7; TF5000 Display v1.53; PruneEPG 1.0; MyInfo B5.6;
Sig generated by MyInfo on 10/4/19
DeadBeef
Frequent contributor
Posts: 264
Joined: Mon Jan 09, 2006 7:28 pm

Post by DeadBeef »

malc wrote:Useful link. I can well imagine that the h/w works out the crc. I assume this flags any errors and the driver software resends rather than the h/w being responsible for resend errors. Especially since the data packet can be 1k.

The humax problem might be a poor driver. But if that is the case are you sure that the toppy driver resends bytes in error or does it ONLY look at the CRC it includes in the data block. If so then you could end up with something as unreliable as the humax usb.
According to the USB spec the data is not passed to the software in either case if the receiving hardware detected a CRC error (with or without the CRC computed by the transmitting software). So the handling of both cases must be basically the same. The firmware waits 5 seconds for an ACK. If no ACK/NACK/CANCEL is received within 5 seconds the firmware retransmits the packet. If the firmware still does not receive any response the transfer is aborted.

Humax might also have a hardware problem in the USB interface. When the TF5000 PVR was introduced it had a very instable USB connectivity. IIRC it was a hardware problem.

There are lots of volunteers testing the modification. Let's sit and wait for the data loss complaints. :wink:
Post Reply