Click here to go to the forum index Click here for the home page
 
Author Message

<  TAP and patch development  ~  How to save or access mheg5/mhp DSMCC files ?

Page 2 of 4
Goto page Previous  1, 2, 3, 4  Next
bdb
Posted: Thu Dec 10, 2009 10:53 pm Reply with quote
Frequent contributor Joined: 18 Oct 2005 Posts: 499
Laughing
View user's profile Send private message
fblasot
Posted: Fri Dec 11, 2009 11:47 am Reply with quote
Regular contributor Joined: 18 Mar 2008 Posts: 99 Location: Turin, Italy
Hi,

I'm developing my new tap...

When I press Guide on rc I wish to change to a specific channel (LCN 5) and then start the capture of dsmcc data.
I try to use TAP_Channel_Start, the toppy react with a blank screen...
May be the param that I set (as fixed value read with TAP_Channel_GetCurrent with my channel active and mainSub=1) are wrong...
Also, is there a way to change to a channel using LCN numbering or with ServiceID ?

@bdb:
you said, after set the hook, to change a channel to wake up the process.
Is there a way to force the start only after the channel was changed ?
This to avoid collecting of data from other channel since yesterday doing test several times as I apply the patch 0xF0 the process start with collecting (indeed because I already use the tap but I can think that in normal use in the future my new tap could be run many times...)

_________________
TF5800 Firmware 5.13.65B Patch
Taps: TAPCommander v1.34, QuickJump v1.72, RecCopy v4.5a, SDS 1.3e, HDFW v2.4, ChkEncryptedRecs v0.1 (my 1st TAP !), FixEITPremium v0.1 (my 2nd TAP !)
View user's profile Send private message
simonc
Posted: Fri Dec 11, 2009 2:39 pm Reply with quote
Frequent contributor Joined: 12 Apr 2005 Posts: 5639 Location: Cheltenham
If you want to use LCN you'll have to convert it to service number using TAP_Channel_GetInfo. I think there's a #define for main/sub somewhere in tap.h
View user's profile Send private message Visit poster's website
fblasot
Posted: Fri Dec 11, 2009 3:35 pm Reply with quote
Regular contributor Joined: 18 Mar 2008 Posts: 99 Location: Turin, Italy
Hi simonc,
I know that using TAP_Channel_GetInfo I can get LCN and serviceId.

The question is how to change to a specific chanell using LCN or svcId: I understood that it cannot be done with TAP_Channel_Start...

_________________
TF5800 Firmware 5.13.65B Patch
Taps: TAPCommander v1.34, QuickJump v1.72, RecCopy v4.5a, SDS 1.3e, HDFW v2.4, ChkEncryptedRecs v0.1 (my 1st TAP !), FixEITPremium v0.1 (my 2nd TAP !)
View user's profile Send private message
bdb
Posted: Fri Dec 11, 2009 4:54 pm Reply with quote
Frequent contributor Joined: 18 Oct 2005 Posts: 499
Quote:
you said, after set the hook, to change a channel to wake up the process.

you will have to figure out if/when you need to this - the issue is that the code to setup the section filters is called when you change channel. If the dsmcc filters are disabled, simply activating the hook is insufficient. If however the section filters are already enabled, then the hook can just be enabled/disabled to allow you to grab the data. There is some code somewhere (no idea where ... somewhere in one of the mheg/interactive services discussions) that provides a means to trigger the enabling of dsmcc without changing channels.

bdb
View user's profile Send private message
simonc
Posted: Fri Dec 11, 2009 7:00 pm Reply with quote
Frequent contributor Joined: 12 Apr 2005 Posts: 5639 Location: Cheltenham
Here's an approximation of what I do in QuickBlank
Code:

// convert LCN to Topfield channel number
int TAP_Channel_GetInfoLCN(int svcType, int LCN, TYPE_TapChInfo* chInfo)
{
   int tvCount = 0, radioCount = 0, count;
   int svcNum;

   TAP_Channel_GetTotalNum(&tvCount, &radioCount);
   count = svcType == SVC_TYPE_Tv ? tvCount : radioCount;
   for (svcNum = 0; svcNum < count; ++svcNum)
   {
      TAP_Channel_GetInfo(svcType, svcNum, chInfo);
      // TAP_Print("%d==%d\n", info.logicalChNum, LCN);
      if (chInfo->logicalChNum == (word)LCN)
         return svcNum;
   }

   return -1;
}

TYPE_TapChInfo chInfo;
int svcNum = TAP_Channel_GetInfoLCN(SVC_TYPE_Tv, 4, &chInfo);
if (svcNum > -1)
   TAP_Channel_Start(CHANNEL_Main, SVC_TYPE_Tv, svcNum);
View user's profile Send private message Visit poster's website
fblasot
Posted: Fri Dec 11, 2009 10:18 pm Reply with quote
Regular contributor Joined: 18 Mar 2008 Posts: 99 Location: Turin, Italy
Thanks simonc,
it works rock !
(I have no doubt...)

_________________
TF5800 Firmware 5.13.65B Patch
Taps: TAPCommander v1.34, QuickJump v1.72, RecCopy v4.5a, SDS 1.3e, HDFW v2.4, ChkEncryptedRecs v0.1 (my 1st TAP !), FixEITPremium v0.1 (my 2nd TAP !)
View user's profile Send private message
fblasot
Posted: Sun Oct 10, 2010 4:47 pm Reply with quote
Regular contributor Joined: 18 Mar 2008 Posts: 99 Location: Turin, Italy
Hi at all !

After a long pause I started again on my project to get epg data transmitted using MHP DSMCC carousel and, thanks to your helps, finally I got it !

I'm using the last LIBDSMCC files included in dvbdata-0.6 (by Richard Palmer if I'm not wrong) without make any modification to those files, just some #define (and other small issue) in my source tap before include the libdsmcc files.

What I get are n zipped files (d0.zip for today data, d1.zip for tomorrow, and so on up to d6.zip for a complete week).
Of course these files are correctly unzipped on my pc but

how I can unzip them directly on the toppy using a tap ?

I hope that an already zip library is available for toppy else I fear that I have to start from some linux source...

For who is interested this is my beta.
Your suggestions are welcome !

Code:

/********************************************************************************************************************/
/* Name :   GetDSMCC                                                                                                */
/* For  :   Topfield TF5xxxPVR                                                                                      */
/* Author:  FBlasot, with thanks to toppy community                                                                 */
/* Descr.:  Test carousel DSMCC of Canale5 Mediaset                                                                 */
/* Licence: This TAP is freeware. You may use this code in your own                                                 */
/*          projects free of charge.                                                                                */
/*          Use at your own risk!                                                                                   */
/*                                                                                                                  */
/********************************************************************************************************************/

#include                <tap.h>
#include                <string.h>
#include                "libFireBird.h"

#define PROGRAM_NAME    "GETDSMCC"
#define VERSION         "V0.1"
#define LOGNAME         "GETDSMCC.log"

// to let the tap run directly from user when in "/ProgramFiles"
// disable the define when it will run from dir "/ProgramFiles/Auto Start" using a timer (something like RecCopy)
#define _LIVETEST_


TAP_ID                  (0x8E0A42FD);
TAP_PROGRAM_NAME        (PROGRAM_NAME " " VERSION);
TAP_AUTHOR_NAME         ("FBlasot");
TAP_DESCRIPTION         ("Test carousel DSMCC of Canale5 Mediaset");
TAP_ETCINFO             (__DATE__);

#include                "TSRCommander.inc"

dword   Carousel_Id=0x44c, Pid_Id=0x1900;   // founded with DVBStreamExplorer
int      LCN=5;   // LCN channel that transmit data
int      DownloadSecs=4500;   // 4500 secs = 1,5 hour (it need around 1 hour to get all files)
bool   SaveDSMCCData=TRUE;
bool   SaveDSMCCLog=FALSE;
char   TimerRecName[]="CANALE 5.rec";
int      DebugLevel=3;   //0=niente, 1=console, 2=file, 3=console+file

#define fnLog   PROGRAM_NAME ".log"
TYPE_File *fpLog = 0;
bool fpLogAppend=TRUE;

void MyLog(char *buffer)
{
char *TS;
char CRLF[] = {'\r', '\n'};
byte Sec;
   
   if (DebugLevel==0) return;
   
    TS = TimeFormat(Now (&Sec), Sec, TIMESTAMP_HMS);
     if (TS [0]) strcat (TS, " ");
     
   if (DebugLevel>=2 && !fpLog)
   {
      if (!fpLogAppend)
         if (TAP_Hdd_Exist(fnLog)) TAP_Hdd_Delete(fnLog);
         
      TAP_Hdd_Create(fnLog, ATTR_NORMAL);      
      fpLog = TAP_Hdd_Fopen(fnLog);      
      if (!fpLog)
      {
          TAP_Print(TS);
         TAP_Print("%s: ", PROGRAM_NAME);
          TAP_Print("ERROR: cannot create file %s !", fnLog);
         TAP_Print("\n");         
          return;
      } 
      TAP_Hdd_Fseek(fpLog, 0, 2 /*SEEK_END*/);
   }
      
     if (DebugLevel==1)   //console
     {
       TAP_Print(TS);
      TAP_Print("%s: ", PROGRAM_NAME);
      TAP_Print("%s", buffer);
      TAP_Print("\n");        
     }
     else if (DebugLevel==2)   // file
     {
       if (fpLog)
      {
         TAP_Hdd_Fwrite (TS, strlen (TS), 1, fpLog);
         TAP_Hdd_Fwrite (buffer, strlen (buffer), 1, fpLog);
         TAP_Hdd_Fwrite (CRLF, 2, 1, fpLog);
      }        
     }
     else if (DebugLevel==3)   // console+file
     {
       TAP_Print(TS);
      TAP_Print("%s: ", PROGRAM_NAME);
      TAP_Print("%s", buffer);
      TAP_Print("\n");
      
       if (fpLog)
      {
         TAP_Hdd_Fwrite (TS, strlen (TS), 1, fpLog);
         TAP_Hdd_Fwrite (buffer, strlen (buffer), 1, fpLog);
         TAP_Hdd_Fwrite (CRLF, 2, 1, fpLog);
      }        
     }
     
}


char                    puffer[512];
//#define DEBUG(...)      {sprintf(puffer, __VA_ARGS__); LogEntry (LOGNAME, PROGRAM_NAME, TRUE, TIMESTAMP_HMS, puffer);}   // file + seriale
//#define DEBUG(...)      {sprintf(puffer, __VA_ARGS__); LogEntry (NULL, "", TRUE, TIMESTAMP_HMS, puffer);}   // solo seriale
//#define DEBUG(...)      {sprintf(puffer, __VA_ARGS__); LogEntry (LOGNAME, PROGRAM_NAME, FALSE, TIMESTAMP_HMS, puffer);}   // solo file
//#define DEBUG(...)      {sprintf(puffer, __VA_ARGS__); LogEntry (NULL, PROGRAM_NAME, FALSE, TIMESTAMP_HMS, puffer);}      // niente ???
//#define DEBUG(...)   {}

#define DEBUG(...)      {sprintf(puffer, __VA_ARGS__); MyLog(puffer);}


#define fnDSMCCData   "dsmcc.data"
TYPE_File *fpDSMCCData = 0;

dword CheckTime=0;
bool _InUpdateEPG = FALSE;

#define MAX_SVCNUMLCN   501      //500 + 1, to start from 1 instead of 0
int svcNumByLCN[MAX_SVCNUMLCN];

dword HookIndex;

// for make libdsmcc compatible with TAP...
#define   _STDIO_H_   // to avoid include stdio.h
#define _STDLIB_H_   // and stdlib.h
#define   _SYS_LOG_H   // and syslog.h

//#define fprintf(x, ...)   {sprintf(puffer, __VA_ARGS__); TAP_Hdd_Fwrite(puffer, 1, sizeof(puffer), fpDSMCCLog);}
//#define fprintf(x, ...)      {sprintf(puffer, __VA_ARGS__); LogEntry (NULL, "", TRUE, TIMESTAMP_HMS, puffer);}
#define fprintf(x, ...)      {sprintf(puffer, __VA_ARGS__); MyLog(puffer);}

//#define syslog(x, ...)   {sprintf(puffer, __VA_ARGS__); TAP_Hdd_Fwrite(puffer, 1, sizeof(puffer), fpDSMCCLog);} 
//#define syslog(x, ...)      {sprintf(puffer, __VA_ARGS__); LogEntry (NULL, "", TRUE, TIMESTAMP_HMS, puffer);} 
#define syslog(x, ...)      {sprintf(puffer, __VA_ARGS__); MyLog(puffer);} 

#define fwrite(Data, Dim, Length, fp)   TAP_Hdd_Fwrite(Data, Dim, Length, fp);
#define fclose(fp)   TAP_Hdd_Fclose(fp);

// I cannot got to work with zlib so this is dummy, compression stream is not used in DSMCC (for the moment !)
#define uncompress(x, ...) Z_DATA_ERROR

// substitute for use FILE for TAP
// used in libdsmcc files as standard I/O
typedef TYPE_File FILE;

FILE *fopen(const char *path, const char *mode)
{
char *ptr, D [FBLIB_DIR_SIZE], F[50];
FILE *fRet = 0;

  ptr = strrchr(path, '/');
  strncpy (D, path, ptr-path+1);
  strcpy(F, ptr+1);

  ptr = strrchr(D, '/');
  *ptr = '\0';

  HDD_ChangeDir(D);

  if ((TAP_Hdd_Exist(F)) && (mode[0]=='a'))
  {
   fRet = TAP_Hdd_Fopen(F);
   TAP_Hdd_Fseek( fRet, 0, 2 /*SEEK_END*/ );
  }
  else
  {
   TAP_Hdd_Delete(F);
    TAP_Hdd_Create(F, ATTR_NORMAL);
   fRet = TAP_Hdd_Fopen(F);
  }

  return fRet;
}


#include   "libdsmcc.h"
#include    "dsmcc.c"
#include    "dsmcc-receiver.c"
#include    "dsmcc-cache.c"
#include    "dsmcc-biop.c"
#include   "dsmcc-descriptor.c"
#include   "dsmcc-util.c"

// must be here, after libdsmcc include
int mkdir(const char *pathname, mode_t mode)
{
char   D [FBLIB_DIR_SIZE];
char   *d, *p;

  InitTAPAPIFix();

  strncpy (D, pathname, sizeof(D) - 1);  // we need 1 char for a terminating / later
  D[sizeof(D) - 2] = '\0';

  if (*D == '/')
  {
    if (!ChangeDirRoot()) return -1;
    strcpy (D, D + 1);
  }

  strcat (D, "/");
  d = D;

  while ((p = strchr (d, '/')))
  {
    *p = '\0';
    if (*d)
    {
      TAP_Hdd_Create(d, ATTR_FOLDER);
      if (TAP_Hdd_ChangeDir(d) == 0) return -1;
    }
    d = p + 1;
  }

  return 0;
}
// end for libdsmcc


struct dsmcc_status *status;
struct stream *s;
struct pid_buffer *pbuf;


bool TSRCommanderExitTAP (void)
{
  DEBUG("GETDSMCC: Terminating...");

  return TRUE;
}

void TSRCommanderConfigDialog (void)
{
}


void Hook_Process_DSMCC(dword HookIndex, tCPURegs *CPURegs)
{
  byte *buf;
  int section_length;
  int off, elab;
  unsigned long car_id = 0, type_id_len = 0;
  unsigned short mess_id = 0;
      

  //DEBUG("GETDSMCC: in Hook_Process_DSMCC");

  if ((CPURegs->a0 > 0x80000000) && (CPURegs->a0 < 0x84000000))
  {
        buf = (byte *)(CPURegs->a0);
        section_length = 3 + (((buf[1]<<8) + buf[2]) & 0xfff);
       
      //ignore if not dsmcc
       if ((buf[0] != 0x3b) && (buf[0] != 0x3c) && (buf[0] != 0x3d))
       {
           DEBUG("GETDSMCC: section_process_DSMCC(), invalid table_id = %02x", buf[0]);
           return;
       }

        //DEBUG("GETDSMCC: in Hook_Process_DSMCC point 2");

       //save everything
       if (fpDSMCCData)
       {
           TAP_Hdd_Fwrite((byte *)&buf[0], section_length, 1, fpDSMCCData);
       }

      // extract carousel id from stream
      // to get only files from specific carousel id
      switch(buf[0]) {
         case DSMCC_SECTION_INDICATION:

            off=DSMCC_MSGHDR_OFFSET;
            mess_id=(buf[off+2] << 8) | buf[off+3];          
         
            if (mess_id == 0x1006) {
               off=DSMCC_DSI_OFFSET+DSMCC_BIOP_OFFSET;
               type_id_len = (buf[off] << 24) | (buf[off+1] << 16) |
                              (buf[off+2] << 8)  | (buf[off+3]);
               off=off+4+type_id_len+4+4+4+1+4+2;
                  car_id = (buf[off] << 24) | (buf[off+1] << 16) |
                           (buf[off+2] << 8)  | buf[off+3];                  
            }
            else if (mess_id == 0x1002) {
               off=DSMCC_DII_OFFSET;
               car_id = (buf[off] << 24) | (buf[off+1] << 16) |
                                (buf[off+2] << 8)  | (buf[off+3]);                  
            }
            
            break;
         case DSMCC_SECTION_DATA:
            off=DSMCC_DATAHDR_OFFSET;
            car_id = (buf[off+4] << 24) | (buf[off+5] << 16) |
                        (buf[off+6] << 8) | buf[off+7];
                        
            break;
         case DSMCC_SECTION_DESCR:
            car_id = 0;
            break;
         default:
            break;
      }         
      
      elab=(car_id == Carousel_Id);
                     
      //pass to dsmcc
      if (elab)
            dsmcc_process_section(status, buf, section_length, s->pid);
  }
}


bool EPGFilesExist(void)
{
bool ret=FALSE;

   DEBUG("GETDSMCC: checking complete EPG files...");
   
   HDD_TAP_PushDir();
   HDD_ChangeDir("/tmp/cache/Canale5");
   
   ret=(TAP_Hdd_Exist("d0.zip") &&
      TAP_Hdd_Exist("d1.zip")   &&
      TAP_Hdd_Exist("d2.zip")   &&
      TAP_Hdd_Exist("d3.zip")   &&
      TAP_Hdd_Exist("d4.zip")   &&
      TAP_Hdd_Exist("d5.zip")   &&
      TAP_Hdd_Exist("d6.zip"));
   
   if (ret) DEBUG("GETDSMCC: found complete EPG files !");

   HDD_TAP_PopDir();
   return ret;
}

void UpdateEPG(void)
{
int CursvcType, CursvcNum;
dword Count, CountChkFiles, TickNow;
TYPE_PlayInfo playinfo;

  DEBUG("GETDMSCC: Entering in UpdateEPG");

  // some test before process
  if (TAP_Hdd_GetPlayInfo(&playinfo))
  {
   if (playinfo.playMode!=PLAYMODE_None)
        {
           DEBUG("GETDSMCC: not in TV Service (Playing)");
           return;
     }
  }

  // save current channel 
  TAP_Channel_GetCurrent( &CursvcType, &CursvcNum );
  //DEBUG("GETDSMCC: Saved Current Channel Type=%d Num=%d", CursvcType, CursvcNum );

  if ( CursvcType == SVC_TYPE_Radio)
  {
        DEBUG("GETDSMCC: not in TV Service (Radio)");
        return;
  }

  if (svcNumByLCN[LCN] == -1)
  {
        DEBUG("GETDSMCC: ERROR: not found LCN %d", LCN);
        return;
  } 

  if (TAP_Channel_IsPlayable( CHANNEL_Main, SVC_TYPE_Tv, svcNumByLCN[LCN]) != 0)
  {
        DEBUG("GETDSMCC: ERROR: cannot play LCN %d", LCN);
        return;
  }
  // end tests


  if (SaveDSMCCData)
  {
     HDD_ChangeDir("/ProgramFiles/EPG");

     if (TAP_Hdd_Exist(fnDSMCCData)) TAP_Hdd_Delete(fnDSMCCData);
     TAP_Hdd_Create(fnDSMCCData, ATTR_NORMAL);
     fpDSMCCData = TAP_Hdd_Fopen(fnDSMCCData);
     if (!fpDSMCCData)
     {
           DEBUG("GETDSMCC: ERROR: cannot create file %s !", fnDSMCCData);
           return;
     } 
  }

  // files downloaded are saved on toppy in dir /tmp/cache/Canale5
  status = dsmcc_open("Canale5", (SaveDSMCCLog) ? fpLog : NULL);

  s = malloc(sizeof(struct stream));
  s->pid = Pid_Id; // on freq. 594Mhz
  s->assoc_tag = 0x64;
  status->carousels[0].streams = s;
  status->carousels[0].id = Carousel_Id;

  pbuf = malloc(sizeof(struct pid_buffer));
  pbuf->pid = s->pid;
  pbuf->pointer_field = 0;
  pbuf->in_section = 0;
  pbuf->cont = -1;
  pbuf->next = NULL;

  status->buffers = pbuf;

  // set fw Hook for Process_DSMCC
  DEBUG("GETDSMCC: setting Hook...");

  // 0x80157e74 eit
  // 0x801b67fc dsmcc

  HookIndex = HookSet ((dword *) 0x801b67fc, (dword *) Hook_Process_DSMCC);
  HookEnable (HookIndex, TRUE);

  //DEBUG("GETDSMCC: HookSet returned %d", HookIndex);

  // patch filter to handle MHP DSMCC, thanks bdp
  *(unsigned long *)0x801B58DC = 0x241900f0; //was 0x24190106

  // hide InfoBox
  TAP_SysOsdControl( SYSOSD_InfoBox, FALSE );

  // if already on C5 I switch first to channel - 1
  // for trigger the dsmcc process
  if (CursvcNum == svcNumByLCN[LCN]) TAP_Channel_Start(CHANNEL_Main, SVC_TYPE_Tv, svcNumByLCN[LCN-1]);  // go to Rete4

  // Interactive Service On
  DEBUG("GETDSMCC: setting Interactive Service ON...");
  InteractiveSetStatus(TRUE);

  // switch to C5
  TAP_Channel_Start(CHANNEL_Main, SVC_TYPE_Tv, svcNumByLCN[LCN]);
  TAP_Channel_Scale(CHANNEL_Main, 0, 0, 360, 288, TRUE );

  DEBUG("GETDSMCC: start delay of %d sec", DownloadSecs);

  // you can end before time or all files download using rc EXIT
  Count=TAP_GetTick();
  CountChkFiles=Count;
  while (((TickNow=TAP_GetTick())<=(Count+DownloadSecs*1000/10)) && _InUpdateEPG)
  {

     // check each 30 secs if all files are presents
     if (TickNow>(CountChkFiles+3000))   // every 30 secs
     {
      CountChkFiles=TickNow;
        if (EPGFilesExist())
           break;   
   }
   
     TAP_SystemProc();   
  }

  // Interactive Service Off
  DEBUG("GETDSMCC: setting Interactive Service OFF...");
  InteractiveSetStatus(FALSE);

  // normal screen
  DEBUG("GETDSMCC: restore saved channel");
  TAP_Channel_Scale(CHANNEL_Main, 0, 0, 720, 576, TRUE );
  TAP_Channel_Start(CHANNEL_Main, CursvcType, CursvcNum);
 
  // restore original code
  *(unsigned long *)0x801B58DC = 0x24190106;

  DEBUG("GETDSMCC: Disabling Hook...");
  HookExit();

  // reshow InfoBox
  TAP_SysOsdControl( SYSOSD_InfoBox, TRUE );

  dsmcc_close(status);

  TAP_Hdd_Fclose(fpDSMCCData);
}

// done in this way since if running directly from main tap doesn't work
// may be becouse it is still in window Archive ?
dword TAP_EventHandler(word event, dword param1, dword param2)
{
  dword mainState, subState;
  (void) param2;

  if (event == EVT_IDLE)
  {
       TSRCommanderWork();

        if (CheckTime>0 && TAP_GetTick()>CheckTime)
        {   
         // run   
         CheckTime=0;

         _InUpdateEPG = TRUE;
   
         UpdateEPG();
   
         if ( ! _InUpdateEPG)
         {   // exit pressed
            DEBUG("GETDSMCC: exit before completation !");
               TSRCommanderErase();
               if (fpLog) TAP_Hdd_Fclose(fpLog);
              TAP_Exit();            
            return 0;
         }
   
         _InUpdateEPG = FALSE;
   
         // exit
         DEBUG("GETDSMCC: exit and shutdown !");
            TSRCommanderErase();
            if (fpLog) TAP_Hdd_Fclose(fpLog);
#ifndef _LIVETEST_
            Shutdown(TaskPower);
#endif
           TAP_Exit();         
        }   
  }
  else if ((event == EVT_KEY) && (param1 == RKEY_Exit))
  {
   TAP_GetState( &mainState, &subState );
   if( mainState == STATE_Normal )
   {
      if ( _InUpdateEPG)
      {
         _InUpdateEPG = FALSE;
         return 0;
      }
               
   }
  } 
 
  return param1;
}

int FindTimer(void)
{
  int j;
  TYPE_TimerInfo TimerInfo;
  word MJD;
  byte Hour, Minute, Second, iHtimer, iMtimer;
   
  TAP_GetTime(&MJD, &Hour, &Minute, &Second);
  DEBUG("GETDSMCC: now is %d:%d", Hour, Minute);
 
  for (j = TAP_Timer_GetTotalNum() - 1; j >= 0; j--)
    if (TAP_Timer_GetInfo(j, &TimerInfo))
    {
      iHtimer=(TimerInfo.startTime >> 8) & 0xff;
      iMtimer=TimerInfo.startTime & 0xff;
      
      DEBUG("GETDSMCC: timer %d:%d filename=%s", iHtimer, iMtimer, TimerInfo.fileName);
      
      if (iHtimer==Hour && iMtimer==Minute &&
           strcmp(TimerInfo.fileName, TimerRecName) == 0 &&
              TimerInfo.isRec==0 /*&& TimerInfo.reservationType == RESERVE_TYPE_Everyday */ )
      return j;
      
    }

  return -1;
}

void RemoveDir(char *path)
{
dword files;
bool flRoot;
TYPE_File f;   
         
   HDD_TAP_PushDir();
   
   if (path[0]=='/')
   {
      TAP_Hdd_ChangeDir("/");
      
      path=path+1;
      flRoot=TRUE;      
   }
   else
      flRoot=FALSE;

   TAP_Hdd_ChangeDir(path);
   
   files = TAP_Hdd_FindFirst(&f);
    while (files--)
     {
      if (f.attr==ATTR_FOLDER && strcmp(f.name, ".")!=0 && strcmp(f.name, "..")!=0)
         RemoveDir(f.name);
      else if (f.attr==ATTR_NORMAL)
      {
         DEBUG("delete file %s", f.name);
         TAP_Hdd_Delete(f.name);   
      }
   
       TAP_Hdd_FindNext(&f);
     }

   if (flRoot)
      TAP_Hdd_ChangeDir("/");
      
   DEBUG("delete dir %s", path);
   TAP_Hdd_Delete(path);

   HDD_TAP_PopDir();
}

int TAP_Main(void)
{
int tvCount = 0, radioCount = 0, svcNum;
TYPE_TapChInfo chInfo;
char text[128];

  InitTAPAPIFix();

  DEBUG("GETDSMCC: started...");
 
#ifndef _LIVETEST_
  if (!(TAP_GetTick() < 6000 && HDD_TAP_GetCurrentDir(text) == 0 && strcmp(text, "/ProgramFiles/Auto Start") == 0))
  {
     DEBUG("GETDSMCC: running by User, exiting...");
   if (fpLog) TAP_Hdd_Fclose(fpLog);         
   return 0;   
  }
 
  // check if running by our timer   
  if (FindTimer()!=-1)
  {
    DEBUG("GETDSMCC: processing our timer");

    TAP_GenerateEvent(EVT_KEY, RKEY_NoUse, 0);   // prevent the receiver from switching off
    CheckTime = TAP_GetTick() + 3000;
  }
  else
  {
     DEBUG("GETDSMCC: ending since not running by Our Timer...");
     if (fpLog) TAP_Hdd_Fclose(fpLog);
   return 0;   
  }
#else
   CheckTime = TAP_GetTick() + 3000;
#endif

  HDD_ChangeDir("/ProgramFiles");
  if (!TAP_Hdd_Exist("EPG"))
  {
   DEBUG("GETDSMCC: creating EPG folder...");
   TAP_Hdd_Create("EPG", ATTR_FOLDER);
  }

  // load global array svcNumByLCN, index is LCN, data is ServiceNumber
  // used for update the EPG by LCN position
  // to be simple array position = LCN position
  for (svcNum = 0; svcNum < MAX_SVCNUMLCN; ++svcNum)
  {
   svcNumByLCN[svcNum]=-1;
  }   
  TAP_Channel_GetTotalNum(&tvCount, &radioCount); 
  for (svcNum = 0; svcNum < tvCount; ++svcNum)
  {
     TAP_Channel_GetInfo(SVC_TYPE_Tv, svcNum, &chInfo);
         DEBUG("GETDSMCC: svcNum=%d LCN=%d", svcNum, chInfo.logicalChNum);
   if (chInfo.logicalChNum<MAX_SVCNUMLCN)
      svcNumByLCN[chInfo.logicalChNum]=svcNum;
   //else
   //   DEBUG("GETDSMCC: Out of svcNumByLCN array ! svcNum=%d LCN=%d", svcNum, chInfo.logicalChNum);
  }   

  DEBUG("GETDMSCC: remove dir /tmp");
  RemoveDir("/tmp");
 
  DEBUG("GETDSMCC: Terminate Stay Resident to waiting my time to run...");

  TSRCommanderInit(0, TRUE);
 
  return 1;
}

_________________
TF5800 Firmware 5.13.65B Patch
Taps: TAPCommander v1.34, QuickJump v1.72, RecCopy v4.5a, SDS 1.3e, HDFW v2.4, ChkEncryptedRecs v0.1 (my 1st TAP !), FixEITPremium v0.1 (my 2nd TAP !)
View user's profile Send private message
R2-D2
Posted: Sun Oct 10, 2010 6:59 pm Reply with quote
Frequent contributor Joined: 18 Dec 2006 Posts: 12148
Cool!

It may not be very useful to you (or anyone) but the UK firmware does contain zlib stuff, because it contains PNG decoding stuff. I've not persevered to see exactly what is (or isn't included), but I imagine that if the Toppy needs to decode PNGs from the UK MHEG stuff it'd be fairly complete (if not easy to use). However, I'm reminded that our MHEG stuff is one of the biggest sources of instability on the Toppy!

_________________
Troubleshooting -- User Manual -- Dark Side of the Matrix: Firmwares and Patches
View user's profile Send private message Visit poster's website
fblasot
Posted: Sun Oct 10, 2010 7:18 pm Reply with quote
Regular contributor Joined: 18 Mar 2008 Posts: 99 Location: Turin, Italy
Since the zlib entry functions are not declared in tap.h I think it is not easy to use them...

I know about instability with mheg...may be the problem is not the "downloading" of the mheg files (and about them, are them saved on the hdd or are all in memory ?) but the video rendering and navigation...

In my case the downloading to hdd seems work well even if to download the files it took about 1 hour while the same operation with DVBStreamExplorer took 5 minutes on my pc...

Could be that toppy loose packets ? Maybe bdb could have some point of view...

_________________
TF5800 Firmware 5.13.65B Patch
Taps: TAPCommander v1.34, QuickJump v1.72, RecCopy v4.5a, SDS 1.3e, HDFW v2.4, ChkEncryptedRecs v0.1 (my 1st TAP !), FixEITPremium v0.1 (my 2nd TAP !)
View user's profile Send private message
R2-D2
Posted: Mon Oct 11, 2010 8:12 am Reply with quote
Frequent contributor Joined: 18 Dec 2006 Posts: 12148
fblasot wrote:
In my case the downloading to hdd seems work well even if to download the files it took about 1 hour while the same operation with DVBStreamExplorer took 5 minutes on my pc...

The Toppy is generally very busy handling more important sections, and the largest waste of time is processing the EIT tables. If you're not running (or not able to run) SecCache then you'll have to put up with the Toppy's weak cache. I imagine another major issue with MHEG turned on will be the limited buffers filling up, and that's probably where missed packets arise, so you'll have to wait until they appear again. I think there are some optimisations possible in the hardware only collecting complete sections, but my memory on the specifics is very hazy. Confused

_________________
Troubleshooting -- User Manual -- Dark Side of the Matrix: Firmwares and Patches
View user's profile Send private message Visit poster's website
bdb
Posted: Mon Oct 11, 2010 10:40 pm Reply with quote
Frequent contributor Joined: 18 Oct 2005 Posts: 499
Downloading the files should be achievable at the same rate as they are transmitted.
In the UK, the DSMCC carousels only take a few minutes to go round, and unless the reception is poor can be collected in a single pass.
The firmware internal buffer space is limited, so if packets are not removed quick enough, they may get thrown away.
The section filters also have the ability to perform hardware filtering based on version number changes - but I don't think the firmware implements this.
I've always including my own filtering to avoid heavy processing of duplicate/unchanged packets - maybe you could try separating the collection from the processing.

Whilst zlib is in the firmware, it is pretty os neutral, and can be recompiled on any platform without much trouble - not sure what else you need to do to handle .zip format; all the uk stuff can be extracted just by calling 'inflate'.


-don't forget that the toppy firmware also uses libdsmcc ...
bdb
View user's profile Send private message
fblasot
Posted: Tue Oct 12, 2010 5:41 pm Reply with quote
Regular contributor Joined: 18 Mar 2008 Posts: 99 Location: Turin, Italy
Hi bdb

Quote:
...unless the reception is poor...

that is not a problem, quality 90%

Quote:
...so if packets are not removed quick enough...

but the removal of packet is something that I should do ? I believe that it is automatic !
Or maybe the problem can be the slow process in my tap (of course togheter with the libdsmcc files...) ?
Since I haven't modified the libdsmcc files, is there a way to call the original present in the fw ? Maybe the process is more quick ?

Quote:
...I've always including my own filtering to avoid heavy processing of duplicate/unchanged packets...

That can be helpfull, is there a your source that I can study how to create my own filtering ?

Quote:
...by calling 'inflate'...

As I remeber 'inflate' works on compressed stream data; starting from d0.zip I need first extract the single compressed stream data of each files included in the zip and then, I'm pretty sure use the inflate...

I have founded some source of a .zip decompressor (something like miniunzip...) that uses zlib and I'm trying to compile a tap...

EDIT: supposing to use the embedded libdsmcc of the toppy fw...where the downloaded files will be saved ?

_________________
TF5800 Firmware 5.13.65B Patch
Taps: TAPCommander v1.34, QuickJump v1.72, RecCopy v4.5a, SDS 1.3e, HDFW v2.4, ChkEncryptedRecs v0.1 (my 1st TAP !), FixEITPremium v0.1 (my 2nd TAP !)
View user's profile Send private message
bdb
Posted: Wed Oct 13, 2010 9:21 pm Reply with quote
Frequent contributor Joined: 18 Oct 2005 Posts: 499
I think the process goes something like this:
- dsmcc packet arrives in the DVB stream.
- the hardware section filter adds it to a buffer + performs the section reassembly.
- once the section is complete, an interrupt is generated, informing the firmware that the buffer has been updated
- the firmware then adds a task to the assosciated task queue
all this happens under interrupt control, and is lightweight

there are 23 tasks that handle all the very low level functions, such as setting emma registers. These cycle round every ?? ms, waking up if they have anything to do
- the main firmware loop (that call the taps) is task 0.
- some of the tasks handle the dvb section data; they will wake up whenever their input buffer is not empty. They pull out the dsmcc traffic from the hardware buffer, and add it to another queue. This is signalled to the main firmware by adding an event to the event queue.

the main firmware task essentially sits and polls the event queue.
- it will get events from all the low level tasks - e.g. hdd/usb/remote control/front panel/i2c/etc.
- whenever it gets an event, it dispatches the relevent handler (this may be done through another level of queue).
- if there is no handler, it throws away the event.
- if a handler runs for too long, without emptying its queue, calling the dispatcher, or allowing the other handlers to run [a tap can do this via TAP_SystemProc()], then the queues may become full and events will be lost. The loss of r/c keypresses is often due to the input queue overflowing. Similarly incomplete epgs can be caused by incoming packets getting discarded [r2 is the expert on this].

You have hooked the dsmcc handler - 'process_dsmcc()'. This handler is passed a pointer to the dsmcc section data. When the handler completes, the buffer will be freed.

By passing the section directly to libdsmcc, you are really in the lap of the gods as to how long it will take to process the packet. I suspect it will be quite variable. The dsmcc code is very malloc and file intensive - so some sections may trigger a large chain of activity.
If you spend too long in there you are likely to lose lots if incoming data.
It may be better to just use the hook to save the raw data, then process this at your leisure from the tap, so that the processing does not interfere with the collection.

As the data is frequently repeated, it is better to filter it so that you do not repeatedly re-process data.
After an initial flurry of activity, the amount of new data is quite low [well it is for our MHEG/EPG streams - does your epg data update live, or once per day?]
I have dug out some code for you - http://www.toppy.org.uk/~offdigital/dsmcc.c

There are 2 ways to hook a function. You can either insert your code to run before the firmware function runs, or you can get your function to replace the firmware function.
I don't know which type libFirebird uses; but I suspect the former. If so, then after you have called libdsmcc, the firmware will also call its libdsmcc code ... this will potentially double the processing load - although with your data stream, it may bail out fairly quickly.

some suggestions:
- have the hook just save the raw dsmcc data to a file.
- run 'dsmcc_process_section()' from the tap, taking data from the saved file, rather than in the hook.
- replace firmware 'process_dsmcc()', rather then insert 'Hook_Process_DSMCC()'.
- prevent the dsmcc data from being processed more than once by adding a table filter.
- consider a faster malloc() - there were some threads on dlmalloc...


So far I have your code:
- compiled, including zlib
- it collects a dsmcc.data file
- I only left it for ~1.5 minute, and it saved a 5MB file containing the entire carousel.
This was with the wrong Carousel_Id - so no calls to dsmcc_process_section() were made
- using the correct Carousel_Id, then I only received ~150KB of data into dsmcc.data in about 1 miunte, but lots of uncompressed files on my disk!!

If I compile the code on windows, and feed it the dsmcc.data file, then it correctly extracts+uncompresses lots of files to tmp/cache. It looks like the complete set.

- this suggests that decoupling the collection from the processing will be necessary to ensure data is not dropped.


Quote:
supposing to use the embedded libdsmcc of the toppy fw...where the downloaded files will be saved ?
I don't know ... The firmware veers off from the libdsmcc code when it starts saving the data files. I spent some time trying to figure out the data structures, but everything is dynamic - lots of chains of malloced data buffers.

bdb
View user's profile Send private message
fblasot
Posted: Thu Oct 14, 2010 1:07 pm Reply with quote
Regular contributor Joined: 18 Mar 2008 Posts: 99 Location: Turin, Italy
Hi bdb,

first, thanks for your suggestion and help !

Quote:
- compiled, including zlib

I also tryed to include zlib but neves success (with cygwin)...for this reason there is the #define uncompress(x, ...) Z_DATA_ERROR
How did you do ?

Quote:
- using the correct Carousel_Id, then I only received ~150KB of data into dsmcc.data in about 1 miunte, but lots of uncompressed files on my disk!!

If I compile the code on windows, and feed it the dsmcc.data file, then it correctly extracts+uncompresses lots of files to tmp/cache. It looks like the complete set.


I don't understand the difference between the two situation...(sorry for my english...)
Do you mean that with the dsmcc process done in toppy you gets only few files while using windows, with the saved dsmcc.data, you gets all the files ?

Anyway I will try to save dsmcc.data for 15 minutes and after pass the saved files to dmscc_process as you suggest.

Thanks again.

_________________
TF5800 Firmware 5.13.65B Patch
Taps: TAPCommander v1.34, QuickJump v1.72, RecCopy v4.5a, SDS 1.3e, HDFW v2.4, ChkEncryptedRecs v0.1 (my 1st TAP !), FixEITPremium v0.1 (my 2nd TAP !)
View user's profile Send private message

Display posts from previous:  

All times are GMT + 1 Hour
Page 2 of 4
Goto page Previous  1, 2, 3, 4  Next

Jump to:  

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum