Difference between revisions of "/shared2/wc24/nwc24dl.bin"

From WiiBrew
< /shared2‎ | wc24
Jump to: navigation, search
(Added max_errors field, added reference to libwc4 wc24.h.)
Line 75: Line 75:
 
         u16 group_id;
 
         u16 group_id;
 
         u16 unk16;
 
         u16 unk16;
         u16 max_errors;//Rough but not exact, number of errors until WC24 starts downloading the next task with the same entry ID? It's unknown what happens when this is reached when WC24 reaches the last entry for the ID, or when this number is reached with only one entry for the ID.
+
         u16 max_errors;//Rough but not exact, number of errors until WC24 starts downloading the next task with the same entry ID? It's unknown what happens when this is reached when WC24 reaches the last entry for the ID, or when this number is reached with only one entry for the ID. Seems to be decreased by KD, maybe when it switches to downloading the next entry with same ID, unknown.
 
         u16 total_errors;//Zero for no error for the last dl. Unknown for subTasks.
 
         u16 total_errors;//Zero for no error for the last dl. Unknown for subTasks.
 
         u16 dl_freq_perday;//Download frequency in minutes per day.
 
         u16 dl_freq_perday;//Download frequency in minutes per day.

Revision as of 19:13, 26 September 2010

This is the "download list" for WC24. One of the primary functions of WiiConnect24 is that various titles can request that it download files over HTTP(S) and store them in NAND -- while the PPC is doing other tasks, or even while it is off! The order of the entries downloaded is determined by finding the first entry with the last dl timestamp plus the entry's (dl_freq_perday field * 60) compared with the current UTC time, and the UTC time must be less than the other compared value in order for the entry to be downloaded next. The dl frequency in days is determined by the dl_freq_days field, but this is normally set for daily. The actual time when entries are downloaded can drift between 0-10 minutes. When entries are downloaded successfully, the next time the entry is downloaded it sends the If-Modified-Since header in the HTTP request with a converted timestamp of the record's last_modified timestamp. When the record's next_dl field is zero, it's unknown when WC24 downloads the entry if ever. When a entry is initially installed/written to nwc24dl.bin, in general ioctl KD_Download probably should be used so that KD starts downloading at the intervals specified by the dl_freq fields. The dl freq checks are done every two minutes minutes, thus the lowest dl freq is two minutes and a dl freq of every minute can be used to make sure WC24 downloads the content every 2 minutes.

The file begins with an 0x80-byte header:

0000000: 5763446c 00000001 00000000 00000000  WcDl............ (WcDl is magic, 1 might be version number)
0000010: 00200008 00780000 00000000 00000000  . ...x.......... (0x78 is number of slots)
[...] 

There is then a list of 16 records, each 0x10 bytes long:

0000080: 48414541 01354081 0135400b 80000000  HAEA.5@..5@.....
0000090: 48414541 01354083 0135400b a0000000  HAEA.5@..5@.....
00000a0: 00000000 00000000 00000000 00000000  ................
00000b0: 00000000 00000000 00000000 00000000  ................
00000c0: 00000000 00000000 00000000 00000000  ................
00000d0: 48414541 013540c2 0135400b c8000000  HAEA.5@..5@.....
00000e0: 48414541 013540cb 0135400b f0000000  HAEA.5@..5@.....
00000f0: 00000000 00000000 00000000 00000000  ................
0000100: 524d4345 0135417a 01353fea c0000000  RMCE.5Az.5?.....
0000110: 48414241 013542ff 01353ef8 0a000000  HABA.5B..5>.....
0000120: 524d4345 01354480 01353fea dc000000  RMCE.5D..5?.....
0000130: 00000000 00000000 00000000 00000000  ................
0000140: 00000000 00000000 00000000 00000000  ................
0000150: 48415445 01354040 01353efb c0000000  HATE.5@@.5>.....
0000160: 48415445 0135447a 01353efb c0000000  HATE.5Dz.5>.....
0000170: 48415445 0135449b 01353efb c0000000  HATE.5D..5>.....

Each record is the lower 4 bytes of the TitleID that made a request. The next two words are timestamps. The timestamps are stored in minutes. When converted to seconds, it's a standard timestamp. There is one byte of flags, and 3 bytes of padding. The first timestamp in a record is the timestamp for the next dl time. The second timestamp in a record is the server last modified timestamp.

Beginning at 0x800, there are 0x200-byte "slots". The first two bytes are the slot number (0000). The next byte is the "slot" type. 3 for "normal" content downloaded to the title's data dir, 2 for Wii Message Board Announcements. The next byte is the flags byte from the record header list. The next u32 at +0x4 is flags. Bits 1 - 3 are used for security keys and RSA verification disabling. For bits 1 and 3, clear to use hard-coded KD key, set to use key from title's wc24pubk.mod under the data directory. Bit 1 is for RSA public key. When bit 2 is set, RSA signature verification is disabled.(It is unknown if the signature +0x40 unknown bytes for raw content is still needed.) Bit 3 is for AES key. The IV is contained in the content. If wc24pubk.mod doesn't exist in the title's data directory when WC24 attempts to open it, the task is deleted.(See below) When flags bit 4 is set, the entry is only downloaded in idle mode. At offset +0x25 is a subTask byte related field. This must be non-zero to use subTasks. At +0x26 is u16 subTask flags? When bit 1 is set, in memory ".xx" is appended to the URL, where xx is the 2-digit decimal subTask ID. At +0xb4, there is a URL. At +0x1a0, is the filename inside the VFF. The downloaded content is written inside wc24dl.vff in the title's data dir. The title must create wc24dl.vff, WC24 does not create it. If wc24dl.vff does not exist when content is downloaded, the task is deleted.(See below. The internal VFF filename is appended with ".xx", if +0x26 bit 1 is set. xx is subTask ID.) At +0x30 is a timestamp, if the total errors u16 at +0x2e is not zero. At +0x20 is the negative WC24 error code, if any. Beginning at offset 0x34 is a u32 array for subTask parameters, for all 32 subTask IDs. These parameters may be timestamps for the last download time?(Or, they could be next dl time.) At +0x28 is a u32 for subTask ID bitmask enable.(Only subTask IDs with matching set bit numbers can be used.) At +0x1fd is a byte for the NHTTP Root CA. When a task is deleted, the record is cleared, the entry is cleared, and the entry type is set to 0xff.

The format of the WC24 content header is available here.

The below is from libwc24 wc24.h.

 #define WC24_TYPE_UNK 1
 #define WC24_TYPE_MSGBOARD 2//E-Mail downloaded to msg board.(Plus MIME data.)
 #define WC24_TYPE_TITLEDATA 3//Content is is downloaded to title's data dir.
 #define WC24_TYPE_EMPTY 0xff
 
 #define WC24_FLAGS_RSA_WC24PUBKMOD BIT(1)//Use RSA public key from wc24pubk.mod in title's data dir instead of default key, if content is encrypted.
 #define WC24_FLAGS_RSA_VERIFY_DISABLE BIT(2)//Skip RSA signature verification.
 #define WC24_FLAGS_AES_WC24PUBKMOD BIT(3)//Use AES key from wc24pubk.mod in title's data dir instead of default key.
 #define WC24_FLAGS_IDLEONLY BIT(4)//Only download this entry in idle mode.
 #define WC24_FLAGS_MAIL_DLFREQHDR_RANGESKIP BIT(30)//For mail entries, when the X-Wii-Download-Interval header in mail is used when nwc24dl_header.reserved_mailnum <= ent.index, skip the range check on the header value frequency. If this block fails, the header parser function returns -4: if(input_freq - 180 <= 9900). 

 #define WC24_SUBTASK_FLAGS_ENABLE BIT(0)//Use subTasks
 
 typedef struct _nwc24dl_header
 {
        u32 magic;//"WcDl" 0x5763446c
        u32 unk4;//0x1
        u32 filler[2];
        u16 unk10;//0x20
        u16 unk12;//0x8
        u16 max_entries;//Max num of entries/records that can be stored in nwc24dl.bin, always 0x78.
        u8 reserved[0x6a];
 } __attribute__((packed)) nwc24dl_header;
 
 typedef struct _nwc24dl_record
 {
       u32 ID;//Titleid low, except for sysmenu. Zero for empty.
       u32 next_dl;//Timestamp for next time to download.
       u32 last_modified;//Last dl timestamp. Zero when download failed, with error_code less than zero, except for error [[WiiConnect24#WC24 Errors|107305]].(HTTP 304) This timestamp is the last dl timestamp for when the download succeeded without any error, with HTTP 200 OK.
       u8  record_flags;//?
       u8  filler[3];
 } __attribute__((packed)) nwc24dl_record;
 
 typedef struct _nwc24dl_entry
 { 
        u16 index;//Zero-based.
        u8 type;
        u8 record_flags;//?
        u32 flags;
        u32 ID;//Titleid low, except for sysmenu.
        u64 titleid;
        u16 group_id;
        u16 unk16;
        u16 max_errors;//Rough but not exact, number of errors until WC24 starts downloading the next task with the same entry ID? It's unknown what happens when this is reached when WC24 reaches the last entry for the ID, or when this number is reached with only one entry for the ID. Seems to be decreased by KD, maybe when it switches to downloading the next entry with same ID, unknown.
        u16 total_errors;//Zero for no error for the last dl. Unknown for subTasks.
        u16 dl_freq_perday;//Download frequency in minutes per day.
        u16 dl_freq_days;//Dl frequency in minutes, for when the next day of downloading begins.(Starting at midnight of dl day. Usually 0x5a0 for daily.)
        s32 error_code;//Zero for no error, negative WC24 error code otherwise.
        u8 unk24SubTask;
        u8 unk25SubTask;//Must be non zero to use subTasks?
        u8 subTaskFlags;
        u8 unk27SubTask;
        u32 subTaskBitmask;//Bitmask subTask ID enable. Only subTasks with IDs/numbers with matching set bits are used.
        u16 unk2cSubTask;
        u16 unk2eSubTask;//0x5a0 usually for entries with subTasks? Might be related to dl time?
        u32 dl_timestamp;//Last dl timestamp. Zero when download failed, with error_code less than zero.
        u32 subTaskParams[32];//Timestamps of last dl time for each subTask?
        char url[0xec];
        char filename[0x40];//Filename inside wc24dl.vff, without a leading root directory slash. This path can probably can be a path with sub-directories.
        u8 unk1e0[0x1d];
        u8 NHTTP_RootCA;//Usually zero.
        u16 unk1fe;
 } __attribute__((packed)) nwc24dl_entry;
0000800: 00000080 00000000 48414541 00000001  ........HAEA....
0000810: 00000002 00010000 00010000 00f005a0  ................
0000820: 00000000 00000000 00000000 00000000  ................
0000830: 013484bf 00000000 00000000 00000000  .4..............
0000840: 00000000 00000000 00000000 00000000  ................
*
00008b0: 00000000 68747470 3a2f2f63 66682e77  ....http://cfh.w
00008c0: 6170702e 7769692e 636f6d2f 616e6e6f  app.wii.com/anno
00008d0: 756e6365 2f303439 2f312f31 2e62696e  unce/049/1/1.bin
00008e0: 0062696e 2e736967 6e656400 00000000  .bin.signed.....
00008f0: 00000000 00000000 00000000 00000000  ................
*
0000a00: 000100a0 00000000 48414541 00000001  ........HAEA....
0000a10: 00000002 00010000 00010000 00f005a0  ................
0000a20: 00000000 00000000 00000000 00000000  ................
0000a30: 01338d3f 00000000 00000000 00000000  .3.?............
0000a40: 00000000 00000000 00000000 00000000  ................
*
0000ab4: http://cfh.wapp.wii.com/announce/049/1/2.bin.bin.signed
0000c00: 0002ff00 00000000 00000000 00000000  ................
0000c10: 00000000 00000000 00000000 00000000  ................
*
0xff in the third byte indicates an unused slot.
0000e00: 0003ff00 00000000 00000000 00000000  ................
0001000: 0004ff00 00000000 00000000 00000000  ................
0001200: 000502c8 00000000 48414541 00000001  ........HAEA....
0001210: 00000002 00010000 00010000 00f005a0  ................
0001220: 00000000 00000000 00000000 00000000  ................
0001230: 01350634 00000000 00000000 00000000  .5.4............
0001240: 00000000 00000000 00000000 00000000  ................
*
00012b4: http://cfh.wapp.wii.com/announce/049/1/3.bin
0001400: 000602f0 00000000 48414541 00000001  ........HAEA....
0001410: 00000002 00010000 00010000 00f005a0  ................
0001420: 00000000 00000000 00000000 00000000  ................
0001430: 01350634 00000000 00000000 00000000  .5.4............
0001440: 00000000 00000000 00000000 00000000  ................
*
00014b4: 00000000 http://cfh.wapp.wii.com/announce/049/1/4.bin
0001600: 0007ff00 00000000 00000000 00000000  ................

0001800: 000803c0 00000004 524d4345 00010004  ........RMCE....
0001810: 524d4345 30310000 00630000 016805a0  RMCE01...c...h..
0001820: 00000000 00000000 00000000 00000000  ................
0001830: 01354012 00000000 00000000 00000000  .5@.............
0001840: 00000000 00000000 00000000 00000000  ................
*
00018b4: https://mariokartwii.race.gs.nintendowifi.net/raceservice/maindl_eu_en.ashx
*
00019a0: distmap.bin
0001a00: 0009030a 0000000a 48414241 00010002  ........HABA....
0001a10: 48414241 30310000 00640000 05a005a0  HABA01...d......
0001a20: 00000000 00000000 00000000 000005a0  ................
0001a30: 01351a98 00000000 00000000 00000000  .5..............
0001a40: 00000000 00000000 00000000 00000000  ................
*
0001ab4: http://ccs.cdn.shop.wii.com/ccs/download/0001000248414241/dynamicBanner_en_US
*
0001ba0: RcmdData.bin
0001c00: 000a03dc 00000004 524d4345 00010004  ........RMCE....
0001c10: 524d4350 30310000 00640000 05a005a0  RMCP01...d......
0001c20: 00000000 00000000 00000000 00000000  ................
0001c30: 01353ee0 00000000 00000000 00000000  .5>.............
0001c40: 00000000 00000000 00000000 00000000  ................
*
0001cb4: https://mariokartwii.sake.gsnintendowifi.net/SakeFileServer/ghostdownload.aspx?gameid=1687&region=0&p0=134364160&c0=18&t0=167378&p1=149728745&c1=22&t1=171334
*
0001da0: ghost.bin
*
0001df0: 00000000 00000000 00000001 00010000  ................
0001e00: 000bff00 00000000 00000000 00000000  ................
0001e10: 00000000 00000000 00000000 00000000  ................
*
0002000: 000cff00 00000000 00000000 00000000  ................
0002010: 00000000 00000000 00000000 00000000  ................
*
0002200: 000d02c0 0000000a 48415445 00010001  ........HATE....
0002210: 48415445 30310000 00640000 016805a0  HATE01...d...h..
0002220: 00000000 00000000 00000000 00000000  ................
0002230: 0134a410 00000000 00000000 00000000  .4..............
0002240: 00000000 00000000 00000000 00000000  ................
*
00022b4: 00000000 http://entu.wapp.wii.com/0/US/en/message1
00023f0: 00000000 00000000 00000000 01000000  ................
0002400: 000e02c0 0000000a 48415445 00010001  ........HATE....
0002410: 48415445 30310000 00640000 05a005a0  HATE01...d......
0002420: 00000000 00000000 00000000 00000000  ................
*
00024b4: http://entu.wapp.wii.com/0/US/en/message2

0002600: 000f03c0 0000000a 48415445 00010001  ........HATE....
0002610: 48415445 30310000 00180000 05a005a0  HATE01..........
0002620: 00000000 00000000 00000000 00000000  ................
0002630: 01353e42 00000000 00000000 00000000  .5>B............
0002640: 00000000 00000000 00000000 00000000  ................
*
00026b4: http://entu.wapp.wii.com/0/US/en/wc24data.LZ
*
00027a0: WC24Data.bin
*
000f600: 0077ff00 00000000 00000000 00000000  .w..............


WhoDares' Findings

This file contains a list of URLs that WiiConnect24 downloads data from, and can be found in /shared2/wc24

File header looks something like this

struct WC24File
{
  char magic_word[4], // 'WcDl'
  int unknown_value_1,
  int unknown_value_2,
  int unknown_value_3,
  short unknown_value_4,
  short wc24_header_file_offset,
  short wc24_record_count,
  short unknown_value_5
}

wc24_header_file_offset is big endian, and needs to be shifted left 4 bits to get the true address. The space between file header and wc24 record headers is padded with 0x00 bytes


At offset 00000080 there is the first of a batch of header records. The structure of these is something like

struct WC24Header
{
  char game_title[4],
  u32 memory_address_1,
  u32 memory_address_2,
  u8 record_pointer,
  u8 padding[3]
}

My first record has the following data { 'HAEA', 0x0135F25A, 0x0135F209, 0x80, { 0x00, 0x00, 0x00 } }

If you shift the record_pointer 4 bits to the right, you get 0x00000800, which is a point inside the file which contains a more detailed record

Interestingly, in my list of headers, I get two HAEA records, then a blank record, then two HAFP records, followed by another two HAEA records, then another blank, then RMCP, HABA, then two final RMCP records

struct WC24Record
{
  short record_index,
  short unknown_value_1,
  int unknown_value_2,
  char game_title[4],
  int unknown_value_3,
  int unknown_value_4,
  int unknown_value_5,
  int unknown_value_6,
  int unknown_value_7,
  int unknown_value_8,
  int unknown_value_9,
  int unknown_value_10,
  int unknown_value_11,
  int unknown_value_12,
  char unknown_value_13[128],
  char wc24_main_url[236],
  char wc24_sub_url_files[96],
}

record_index starts with the value of 0x0000, and counts up each record

unknown_value_1 = 0xFF00 if the record is empty (big endian)

unknown_value_2 always seems to be 0

unknown_value_4 seems to be a some sort of application id, sometimes the same as game_title, but on the HAEA entries, they are 0x00000002 (big endian)

unknown_value_5 could actually be a part of unknown_value_4. When my game title is HAFP, unknown_value_4 and unknown_value_5 combine to create a text string of 'HAFP01', followed by two 0x00's to fill the space. Might just be coincidental though.

unknown_value_7 looks like a relative memory address (big endian), could be wrong, but they do stay consistant between records with the same game ID

unknown_value_8 to unknown_value_11 are always 0x00000000

unknown_value_12 looks like another memory address

unknown_value_13 looks like string space, but is always blank in all of my records

wc24_main_url seems an odd length, so there maybe another smaller string tucked in at the end

wc24_sub_url_files is a bit confusing, it might be an array of filesnames to use with the wc24_main_url. However, judging from the data, it might actually be for a single filename, just keeps getting overwritten with different values


I know there's a lot of unknowns here, but there might be some useful information. I'll update as I find out...

Note: Previous Info was taken from the Wiibrew Forums