Difference between revisions of "RST0 (File Format)"
Gamebuster (Talk | contribs) (→Compression) |
Gamebuster (Talk | contribs) (→Compression) |
||
| Line 230: | Line 230: | ||
| 0x4 | | 0x4 | ||
| u32 | | u32 | ||
| − | | | + | | '''CRC''' |
| A CRC value. For Excitebots, the algorithm used can be found here [https://github.com/TheGameCommunity/ExciteBot/blob/4157071b2a0265fe37da2a8fb3585b79ceb593f3/src/main/java/com/gamebuster19901/excite/game/propritary/crc/CRCTester.java#L1]. Other games probably use a similar but different algorithm. | | A CRC value. For Excitebots, the algorithm used can be found here [https://github.com/TheGameCommunity/ExciteBot/blob/4157071b2a0265fe37da2a8fb3585b79ceb593f3/src/main/java/com/gamebuster19901/excite/game/propritary/crc/CRCTester.java#L1]. Other games probably use a similar but different algorithm. | ||
|- | |- | ||
Revision as of 18:19, 17 October 2023
The RST0 format is a proprietary archive format created by Monster Games. This format can be found in Donkey Kong Country Returns 3D, Excite Truck, and Excitebots: Trick Racing. It appears under a few different file extensions: .lvl, .res, and .trk; all are the same format though, with the extension just describing the contents of the file rather than the format. RST0 is a little-endian format; all fourCCs and magic values that appear in the file appear reversed.
|
This file format needs more research There are a lot of unknowns left |
Contents
Format
RST0 files contain a header, which is a static size of 0x44 bytes, then are padded until 0x80, where a table of contents is present, listing every resource in the file. Some RST0 files are compressed; on compressed files, the header will be uncompressed, and most of the rest of the file (including the ToC and most of the resources in the file) will be compressed. There are some RST0 files that contain uncompressed resource data following the end of the compressed data, though.
Header
| Offset | Type | Name | Notes | |
|---|---|---|---|---|
| 0x0 | char[4] | Magic | RST0
| |
| 0x4 | u32 | Header Size | Always 0x44 in Donkey Kong Country Returns. In Excitebots: Trick Racing, the value is always 0x40 (Header size divided by 2). | |
| 0x8 | u32 | Unknown | Always 0x12 in Donkey Kong Country Returns? In Excitebots: Trick Racing, the value is always 0xE | |
| 0xC | u32 | Unknown | Compression-related. In DKCR: Always 0 on uncompressed files and 2 on compressed files. No idea what the number means though; maybe it's a "compression type" enum? In Excitebots, this value is either 1 or 3. | |
| 0x10 | u32 | File Size | Includes the header, but doesn't include the padding at the end of the file. | |
| 0x14 | i32 | Unix timestamp | Corresponds to the last edit of the file contents by the developers. For games with a separate TOC file, this must match the timestamp in the TOC file or the game will intentionally crash. | |
| 0x18 | u32 | Table of Contents Offset | Always 0x80 | |
| 0x1C | u32 | Resource count? | In Donkey Kong Country Returns, this is the amount of resources. In Excitebots: Trick Racing, this value is always 0x0. | |
| 0x20 | u32 | Resource Count | The amount of resources. Appears to be a duplicate in Donkey Kong. | |
| 0x24 | u32 | Decompressed Size | Matches the decompressed sizes in the compressed section header. Equal to the file size (minus the header) on uncompressed files. | |
| 0x28 | u32 | Compressed Size | Size of the entire compressed section. Equal to the file size (minus the header) on uncompressed files. | |
| 0x2C | u32 | Unknown | ||
| 0x30 | u32 | Unknown | ||
| 0x34 | u32 | String Table Size | Size of the string table that contains the resource file names | |
| 0x38 | u32 | Resource ID Table Size | Size of the table of resource IDs after the string table | |
| 0x3C | u32 | Unknown | Some sort of hash? | |
| 0x40 | u32 | Unknown | Some sort of hash? | |
| 0x44 | End of header; padding to 0x80 | |||
Table of Contents
The ToC contains three arrays:
| Type | Name |
|---|---|
| Resources | Resource Definitions |
| string | Filename Table |
| Padding; the size of the name table is always a multiple of 0x10 | |
| Resource IDs | Resource ID Table |
| End of ToC; pad to 0x80 then file data starts | |
Resources
| Offset | Type | Name | Notes |
|---|---|---|---|
| 0x0 | u32 | Filename Offset | Relative to the start of the name table |
| 0x4 | char[4] | Resource Type | Since the file is little-endian, the fourCC is backwards; eg, LDOM is actually MODL, XET is actually TEX , etc
|
| 0x8 | u32 | Unknown | Flags? 0x10000000 seems to be set on uncompressed resources in compressed archives. |
| 0xC | u32 | Resource Size | |
| 0x10 | u32 | Resource Offset | Relative to the start of the ToC |
| 0x14 | u32 | Unknown | Some sort of hash? |
| 0x18 | u32 | Resource ID | Likely a hash of the resource name. This ID appears again later in the ID table and is used by other files to reference this resource. |
| 0x1C | u32 | Unknown | |
| 0x20 | u32 | Unknown | |
| 0x24 | u32 | Unknown | |
| 0x28 | u32 | Unknown | |
| 0x2C | End of resource definition | ||
Resource IDs
| Offset | Type | Name |
|---|---|---|
| 0x0 | u32 | Resource ID |
| 0x4 | u32 | Resource Index |
| 0x8 | End of resource ID | |
Compression
RST0 files use a custom run-length encoding compression algorithm. The compressed data begins with a 0x19-byte header before the actual compressed data begins. The header is actually two separate headers - one for the entire compressed section, comprising the first 0x10 bytes, and then one for the compressed data, comprising the last 0x9 bytes, which is why there's some redundant values included.
| Offset | Type | Name | Notes |
|---|---|---|---|
| 0x0 | char[4] | Magic | rCMP
|
| 0x4 | u32 | CRC | A CRC value. For Excitebots, the algorithm used can be found here [1]. Other games probably use a similar but different algorithm. |
| 0x8 | u32 | Compressed data size | This is 0x10 smaller than the compressed size from the RST0 header because it doesn't include the initial compressed section header. |
| 0xC | u32 | Decompressed data size | This will match the decompressed size from the RST0 header. |
| 0x10 | u8 | Unknown | Always 0x4F? |
| 0x11 | u32 | Compressed data size | Matches the other compressed data size value. |
| 0x15 | u32 | Decompressed data size | Matches the other decompressed data size value as well as the one from the RST0 header. |
| 0x19 | End of header; compressed data begins | ||
The compressed data is made up of multiple chunks. Each chunk has a 32-bit value as a header, where each bit tells you how to read the next part of the data. After reading in the header value, check each bit separately, starting at the least significant bit (0x1) and going up to the 31st bit (0x40000000). Note that although the header value is 32-bit, the top bit (0x80000000) is always set and doesn't actually correspond to anything in the file, so it should be ignored. After checking all 31 bits, read the next header byte and repeat.
For each bit, do the following depending on the value of the bit:
- 0: Read a single byte directly out of the compressed data stream
- 1: Seek backwards in the decompressed data buffer and copy bytes out of it back to the front
0 is straightforward and doesn't need much further explanation. 1 is where things get a little bit complicated. The next bytes in the compressed stream are an indicator value that will tell you how far back to seek in the decompressed data buffer and how much data to read out of them. The value will be a different size depending on how much data is needed at any given moment. Read one byte, and check the bottom two bits (0x1 and 0x2); they will tell you how large the indicator value is, and how to use the data in it. It's also important to note that the indicator value is little-endian, even though it has to be read one byte at a time; so the first byte you read is the bottom byte, and the last one is the top byte.
Using these definitions:
| A | Number of bytes to seek back in the decompressed buffer |
| B | Number of bytes to read back from the decompressed buffer |
| C | Indicator size/type flags |
| D | These bits will be 0 to indicate a 32-bit indicator value. If non-zero, then the indicator is 24-bit. |
Then depending on the value of the bottom two bits, this is the structure of the indicator value:
| Value | Size | Structure |
|---|---|---|
| 00 | 8-bit | AAAAAACC
The byte count is always 3. |
| 01 | 16-bit | AAAAAAAAAAAAAACC
The byte count is always 3. |
| 10 | 16-bit | AAAAAAAAAABBBBCC
Add 3 to the byte count. |
| 11 | 24- or 32-bit | This one is different depending on the contents of the bottom byte. Check 5 of the middle bits (the D bits):
If any of the D bits are non-zero, then the value is 24-bit: (add 2 to the byte count)
Otherwise, the value is 32-bit: (add 3 to the byte count)
|
Tools
- RSTExtract by Aruki will unpack and decompress Donkey Kong Country Returns 3D RST0 archives,
