Savegames

Revision as of 07:42, 17 August 2016 by Wwylele (talk | contribs) (→‎The SAVE partition: Correct entry fields based on research of some system savegames.)

This page describes the format and encryption of savegames contained in gamecards, SD/NAND, and SD/NAND extdata. You can find savegames from various 3DS games on the Games page.


Savegame Encryption

Repeating CTR Fail

On the 3DS savegames are stored much like on the DS, that is on a FLASH chip in the gamecart. On the DS these savegames were stored in plain-text but on the 3DS a layer of encryption was added. This is AES-CTR, as the contents of several savegames exhibit the odd behavior that xor-ing certain parts of the savegame together will result in the plain-text appearing.

The reason this works is because the stream cipher used has a period of 512 bytes. That is to say, it will repeat the same keystream after 512 bytes. The way you encrypt with a stream cipher is you XOR your data with the keystream as it is produced. Unfortunately, if your streamcipher repeats and you are encrypting a known plain-text (in our case, zeros) you are basically giving away your valuable keystream.

So how do you use this to decrypt a savegame on a 3DS? First off, you chunk up the savegame into 512 byte chunks. Then, you bin these chunks by their contents, discarding any that contain only FF. Now look for the most common chunk. This is your keystream. Now XOR the keystream with your original savegame and you should have a fully decrypted savegame. XOR with the keystream again to produce an encrypted savegame.

Savegame keyY

All gamecard and SD savegames are encrypted with AES-CTR. The base CTR for gamecard savegames is all-zero. The gamecard savegame keyslots' keyY(these savegame keyslots use the hardware key-generator) is unique for each region and for each game. The NCSD partition flags determine the method used to generate this keyY. When the save NCSD flags checked by the running NATIVE_FIRM are all-zero, the system will use the repeating CTR, otherwise a proper CTR which never repeats within the image is used.

The AES-CMAC (which uses a hardware key-generator keyslot, as mentioned above) at the the beginning of the savegame must match the calculated CMAC using the DISA/DIFF data, otherwise the savegame is considered corrupted(see below).

When all of the flags checked by the running NATIVE_FIRM are clear, the keyY(original keyY method used with saves where the CTR repeats within the image) is the following:

Offset Size Description
0x0 0x8 First 8-bytes from the plaintext CXI accessdesc signature.
0x8 0x4 u32 CardID0 from gamecard plaintext-mode command 0x90, Process9 reads this with the NTRCARD hw. The actual cmdID used by Process9 is different since Process9 reads it with the gamecard in encrypted-mode.
0xC 0x4 u32 CardID1 from gamecard plaintext-mode command 0xA0, Process9 reads this with the NTRCARD hw. The actual cmdID used by Process9 is different since Process9 reads it with the gamecard in encrypted-mode.
2.0.0-2 Hashed keyY and 2.2.0-4 Savegame Encryption

When certain NCSD partition flags are set, a SHA-256 hash is calculated over the data from the CXI(same data used with the original plain keyY), and the 0x40-bytes read from a gamecard command(this 0x40-byte data is also read by GetRomId, which is the gamecard-uniqueID). The first 0x10-bytes from this hash is used for the keyY. When flag[7] is set, the CTR will never repeat within the save image, unlike the original CTR-method. All games which had the retail NCSD image finalized after the 2.2.0-4 update(and contain 2.2.0-4+ in the System update partition), use this encryption method.

This keyY generation method was implemented with 2.0.0-2 via NCSD partition flag[3], however the proper CTR wasn't implemented for flag[7] until 2.2.0-4. The hashed keyY flag[3] implemented with 2.0.0-2 was likely never used with retail gamecards.

6.0.0-11 Savegame keyY

6.0.0-11 implemented support for generating the savegame keyY with a new method, this method is much more complex than previous keyY methods. This is enabled via new NCSD partition flags, all retail games which have the NCSD image finalized after the 6.0.0-11 release(and 6.0.0-11+ in the system update partition) will have these flags set for using this new method.

A SHA-256 hash is calculated over the same data used with the above hashed keyY method, after hashing the above data the following data is hashed: the CXI programID, and the ExeFS:/.code hash from the decrypted ExeFS header. An AES-CMAC (the keyslot used for this uses the hardware key-scrambler) is then calculated over this hash, the output CMAC is used for the savegame keyY.

The keyY used for calculating this AES-CMAC is initialized while NATIVE_FIRM is loading, this keyY is generated via the RSA engine. The RSA slot used here is slot0(key-data for slot0 is initialized by bootrom), this RSA slot0 key-data is overwritten during system boot. This RSA slot0 key-data gets overwritten with the RSA key-data used for verifying RSA signatures, every time Process9 verifies any RSA signatures except for NCCH accessdesc signatures. Starting with 7.0.0-13 this key-init function used at boot is also used to initialize a separate keyslot used for the new NCCH encryption method.

This Process9 key-init function first checks if a certain 0x10-byte block in the 0x01FF8000 region is all-zero. When all-zero it immediately returns, otherwise it clears that block then continues to do the key generation. This is likely for supporting launching a v6.0+ NATIVE_FIRM under this FIRM.

Wear leveling

The 3DS employs a wear leveling scheme on the savegame FLASH chips(only used for CARD1 gamecards). This is done through the usage of blockmaps and a journal. The blockmap is located at offset 0 of the flash chip, and is immediately followed by the journal. The initial state is dictated by the blockmap, and the journal is then applied to that.

First, there are 8 bytes whose purposes are currently unknown. Then comes the actual blockmap. The blockmap structure is simple:

struct header_entry {
        uint8_t phys_sec; // when bit7 is set, block has checksums, otherwise checksums are all zero
        uint8_t alloc_cnt;
        uint8_t chksums[8];
} __attribute__((__packed__));

There's one entry per sector, counting from physical sector 1 (sector 0 contains the blockmap/journal).

The 2 bytes that follow the blockmap are the CRC16 (with starting value 0xFFFF (like modbus)) of the first 8 bytes and the blockmap.

Then comes the journal. The journal structure is as follows:

struct sector_entry {
        uint8_t virt_sec;       // Mapped to sector
        uint8_t prev_virt_sec;  // Physical sector previously mapped to
        uint8_t phys_sec;       // Mapped from sector
        uint8_t prev_phys_sec;  // Virtual sector previously mapped to
        uint8_t phys_realloc_cnt;       // Amount of times physical sector has been remapped
        uint8_t virt_realloc_cnt;       // Amount of times virtual sector has been remapped
        uint8_t chksums[8];
} __attribute__((__packed__));

struct long_sector_entry{
        struct sector_entry sector;
        struct sector_entry dupe;
        uint32_t magic;
}__attribute__((__packed__));

With magic being a constant 0x080d6ce0.

The checksums in the blockmap/journal entries work as follows:

  • each byte is the checksum of an encrypted 0x200 bytes large block
  • to calculate the checksum, a CRC16 of the block (with starting value 0xFFFF) is calculated, and the two bytes of the CRC16 are XORed together to produce the 8bit checksum

AES CMAC header

Image offset Length Description
0x00 0x10 AES-CMAC over a 0x20-byte SHA256 hash(the keyslot used here uses the hardware key-scrambler).
0x10 0xF0 Zero padding

This AES CMAC is used to "sign" the DISA/DIFF header. Each time the savegame is updated the hash stored in the DISA/DIFF is updated, therefore the CMAC must be updated each time the save is modified as well. SHA256_Update() is used to calculate the hash with the blocks described below.

Savegame Types

Type Description
CTR-EXT0 SD/NAND Extdata
CTR-SYS0 System SaveData
CTR-NOR0 Gamecard Savegames
CTR-SAV0 Savegames
CTR-SIGN SD Savegames
CTR-9DB0 Title database extdata images

Extdata SHA256 Blocks

Block Size Description
0x8 Savegame type
0x8 u64 extdataID
0x4 0 for Quota.dat, 1 otherwise
0x8 u64 imageID: First word is the hex ID from image filename, second word is the hex ID of the sub-dir under the <ExtdataIDLow> directory (all-zero for Quota.dat)
0x100 DIFF header

System SaveData SHA256 Blocks

Block Size Description
0x8 Savegame type
0x8 SaveID
0x100 DISA header

CTR-NOR0 SHA256 Blocks

Block Size Description
0x8 Savegame type
0x100 DISA header

CTR-SAV0 SHA256 Blocks

Block Size Description
0x8 Savegame type
Input data, for gamecard savegames this is the output SHA-256 hash from CTR-NOR0.

For gamecard savegames the output hash from this is used with the CMAC. This save-type is also used for SD savegames, for SD saves the input data is the 0x100-byte DISA header. For SD savegames, the calculated output hash is used with CTR-SIGN.

CTR-SIGN SHA256 Blocks

Block Size Description
0x8 Savegame type
0x8 ProgramID/SaveID
0x20 SHA-256 hash from CTR-SAV0

This is used for SD savegames, the calculated hash from this is used with the CMAC.

CTR-9DB0 SHA256 Blocks

Block Size Description
0x8 Savegame type
0x4 ID, each .db image has a separate ID.
0x100 DIFF header

This is used for the SD/NAND /dbs .db extdata images, see Title_Database regarding AES-CMAC keyslots used here.

Partitions

There can be multiple partitions in the image. The partitions are represented by tables of DIFI blobs inside a DISA/DIFF structure. The order of the DIFI blobs is the order of the partitions in the image.

DISA

  • This is located @ 0x100 in the image, following the CMAC header.
  • If the uint32 @ 0x68 in the DISA(the low 8-bits) is non-zero, then the secondary table is is used, otherwise the primary table is used.
  • If the table has more then 1 DIFI then the uint32 @ 0x168 is the offset from the DATA partition to the file base (masked with 0xFFFFFFFE).
Start Length Description
0x00 4 Magic ("DISA")
0x04 4 Magic Number (0x40000)
0x08 8 Total partition entries in a table
0x10 8 Offset to secondary partition table
0x18 8 Offset to primary partition table
0x20 8 Partition table size
0x28 8 SAVE Partition entry offset in the partition table
0x30 8 SAVE Partition entry length in the partition table
0x38 8 DATA Partition entry offset in the partition table
0x40 8 DATA Partition entry length in the partition table
0x48 8 SAVE Partition offset
0x50 8 SAVE Partition length
0x58 8 DATA Partition offset
0x60 8 DATA Partition length
0x68 4 Active table (and the offset to the filebase)
0x6C 0x20 Hash from active table
0x8C 0x74 Reserved
  • The hash in the DISA hashes the Active Table (starting from tables's offset to tables's offset + table length) with SHA256.
  • The partition offsets are absolute offsets in the image.
  • The SAVE partition offset is usually 0x1000. The SAVE/DATA partitions begins with the DPFS partitions, the relative offset for the IVFC partition data is specified by the DPFS header.

The DIFIs table at offset 0x200 in the image has 2 DIFIs when the DATA partition isn't used, 4 DIFIs otherwise. Each partition table contains the SAVE DIFI entry and optionally the DATA entry. The secondary partition table is located at offset 0x200 in the image, and the primary table follows the secondary table.

The non-active table is for backup.

DIFF

  • This is the extdata equivalent of DISA, for extdata which use FS. DIFF is only used for extdata.
Start Length Description
0x00 4 Magic ("DIFF")
0x04 4 Magic Number (0x30000)
0x08 8 Secondary partition table offset
0x10 8 Primary partition table offset
0x18 8 Partition table length
0x20 8 File Base Offset
0x28 8 File Base Size
0x30 4 Active Partition Table (0 = Primary, 1 = Secondary)
0x34 0x20 Hash of the Active Partition Table
0x54 0x8 Unique Extdata Identifier
0x5C 0xA4 Reserved1
  • The Unique Extdata Identifier is a randomly generated series of bytes which is used to identify a specific extdata image. It's currently known to be used with the VSXE FST, if the ID doesn't match the one stored in the file's file entry, the extdata image isn't mounted in the FS.

DIFI

These 0x12C-byte blobs describe the partitions. Following each partition is an unused 0xFFFFFFFF cleartext word in the raw image. Every DIFI blob describes a partition. Partitions are catted together, so after the end of one partition is the beginning of the next.

For most games there's only 1 partition (The SAVE partition) and some (like Asphalt 3D, Steel Diver & Lego Star Wars III) has 2 partitions.

  • 2 Partitions means that the files inside the SAVE partition is on the DATA partition.
  • The DISA/DIFF headers support a maximum of 2 partitions.
Start Length Description
0x00 4 Magic ("DIFI")
0x04 4 Magic Number (0x10000)
0x08 8 Offset to "IVFC" blob in DIFI (Always 0x44)
0x10 8 Size of "IVFC" blob
0x18 8 Offset to "DPFS" blob in DIFI (Always 0xBC)
0x20 8 Size of "DPFS" blob
0x28 8 Offset to the hash in DIFI (Always 0x10C)
0x30 8 Size of this hash
0x38 4 Flags (when this byte is non-zero, this is a DATA partition)
0x3C 8 File base offset (for DATA partitions)

IVFC

Start Length Description
0x00 4 Magic ("IVFC")
0x04 4 Magic Number (0x20000)
0x08 0x8 Master hash size
0x10 0x8 Level 1 relative offset
0x18 0x8 Level 1 hashdata size
0x20 0x4 Level 1 block size, in log2
0x24 0x4 Reserved
0x28 0x8 Level 2 relative offset
0x30 0x8 Level 2 hashdata size
0x38 0x4 Level 2 block size, in log2.
0x3C 0x4 Reserved
0x40 0x8 Level 3 relative offset
0x48 0x8 Level 3 hashdata size
0x50 0x4 Level 3 block size, in log2.
0x54 0x4 Reserved
0x58 8 Level 4 filesystem relative offset
0x60 8 Level 4 filesystem size
0x68 8 Level 4 filesystem block size, in log2.
0x70 8 Unknown (usually 0x78=120)
  • This savegame IVFC is almost identical to the RomFS IVFC, except for the additional filesystem level. Exactly like RomFS, each level except level4 is a hash-table where each hash entry hashes the data in the next level, padded to the log2 block size.

DPFS

Start Length Description
0x00 4 Magic ("DPFS")
0x04 4 Magic Number (0x10000)
0x08 8 Offset to first table
0x10 8 First table length
0x18 8 First table block size (1<<value)
0x20 8 Offset to second table
0x28 8 Second table length
0x30 8 Second table block size (1<<value)
0x38 8 IVFC partition offset
0x40 8 IVFC partition size
0x48 8 IVFC partition block size (1<<value)
  • Every block this table point to is written twice (concatenated). You can see that the offset to the next block is twice the length (except the data which always begin after 0x1000).
  • The offsets contained in the DPFS and IVFC are relative to the partition offset in the DISA/DIFF. The offsets from the IVFC are additionally added with the IVFC partition offset from the DPFS.

The first partition's data usually starts at 0x2000. First comes the hashtable (usually start @ 0x40 into the partition) and then the filesystem.

The hashtable entries' size is 2^x where x is the 'Filesystem block size' from the IVFC block.

DIFI Hash

The last 0x20-bytes of the partition following the DIFI, IVFC and DPFS is a SHA256 hash. The offset to this hash is stored in the DIFI. This hashes the IVFC level 1, with the buffer which is hashed aligned to the IVFC level 1 log2 block-size.

Summary Drawing

 

The SAVE partition

  • The SAVE filesystem works with a backup. There are two SAVE blocks inside the partition concatenated. Which SAVE block is the updated one is unknown yet.. (I'm guessing from experience that (image[0x100B] & 0x20) == 0x20 --> 1st SAVE --Elisherer 01:30, 18 October 2011 (CEST))

Finding the folders table:

  • If DATA partition exists: At folder table exact offset from the SAVE struct (from the beginning of the struct).
  • Otherwise: The 'folder table offset' * 'folder table media' (=0x200) from the 'filestore offset'. (usually 0 from filebase)

Finding the files table:

  • If DATA partition exists: At file table exact offset from the SAVE struct (from the beginning of the struct).
  • Otherwise: The 'file table offset' * 'file table media' (=0x200) from the 'filestore offset'.

Detemining the filestore base:

  • If DATA partition exists: At file base from the DATA's DIFI struct into the DATA partition.
  • Otherwise: At the 'filestore offset' from the beginning of the SAVE struct.

Folder's entry structure:

 struct folder_entry {
     u32 parent_folder_index;     // stores entry count for dummy folders; 0 for the root folder (entry[1])
     u8  filename[0x10];          // "b" for dummy folders; "" for the root folder (entry[1])
     u32 next_folder_index;       // 0 if the last folder, or a dummy folder
     u32 first_sub_folder_index;  // 0 if has no sub folder, or a dummy folder
     u32 first_sub_file_index;    // 0 if has no sub file(?), or a dummy folder
     u32 unk;                     // flags? always 0?
     u32 next_dummy_folder_index; // 0 if the last dummy folder; unknown (often 0) if not a dummy folder
 }

File's entry structure:

 struct file_entry {
     u32 parent_folder_index;   // stores entry count for dummy files
     u8  filename[0x10];        // "\x01\x15" for dummy files
     u32 next_file_index;       // 0 if the last file, or a dummy file
     u32 unk1;                  // looks like time stamp? 0 if a dummy file
     u32 block_offset;          // 0 if a dummy file
     u64 file_size;             // 0 if a dummy file
     u32 unk2;                  // flags?
     u32 next_dummy_file_index; // 0 if the last dummy file, unknown (always zero?) if not a dummy file
 }

Both table contains some "dummy" entries, which are probably left there when deleting files/folders, reserved for future use. The first entry of the two tables are always a dummy entry.

Reading the files out is as simple as taking the file base offset and adding (block_offset * 0x200) to it.

Here's a follow-up example from the Legend of Zelda: Ocarina of Time 3D:

//FST entry = SAVE base + File base + (FST offset * 0x200) + (FST entry # * 0x30)
//0x2600    = 0x2000    + 0x400     + (0x1        * 0x200) + (0x0         * 0x30)

00002600: 03000000 09000000 00000000 00000000  ................
00002610: 00000000 00000000 00000000 00000000  ................
00002620: 00000000 00000000 00000000 00000000  ................
00002630: 01000000 73797374 656D2E64 61740000  ....system.dat..
00002640: 00000000 00000000 D57B1100 02000000  ........Õ{......
00002650: 22000000 00000000 E8121500 00000000  ".......è.......
00002660: 01000000 73617665 30302E62 696E0000  ....save00.bin..
00002670: 00000000 01000000 69921100 03000000  ........i’......
00002680: DC140000 00000000 04000000 00000000  Ü...............
Start Length Description
0x00 4 Magic ("SAVE")
0x04 4 Magic Number (0x40000)
0x08 8 Offset to data in this SAVE header(normally 0x20)
0x10 8 Partition Size [medias]
0x18 4 Partition Media Size
0x1C 8 Unknown
0x24 4 Media-size for the below sections
0x28 8 FolderMap Offset
0x30 4 FolderMap Size
0x34 4 Unknown, FolderMap size-related
0x38 8 FileMap Offset
0x40 4 FileMap Size
0x44 4 Unknown, FileMap size-related
0x48 8 BlockMap Offset
0x50 4 BlockMap Size
0x54 4 Uknown, BlockMap size-related
0x58 8 File store offset (from SAVE)
0x60 4 File store length [medias]
0x64 4 Unknown, File store size-related
0x68 4/8 Folders Table offset (8 bytes in DATA)
0x6C 4 Folders Table Length (medias) (Only in no DATA)
0x70 4 Folders Table unknown
0x74 4 Unknown, Folders Table size-related
0x78 4/8 Files Table offset (8 bytes in DATA)
0x7C 4 Files Table Length (medias) (Only in no DATA)
0x80 4 Files Table unknown
0x84 4 Unknown, Files Table size-related
  • The FolderMap and FileMap still unknown. They are tables of uint32.
  • The BlockMap is a map of the blocks in the filestore. An entry in the BlockMap is 2 uint32: {uint32 start_block; uint32 end_block; }. This is still being researched. (You can use 3DSExplorer to see those maps.

Summary Drawing

 

Initialization

When a save FLASH contains all xFFFF blocks it's assumed uninitialized by the game cartridges and it initializes default data in place, without prompting the user. The 0xFFFFFFFF blocks are uninitialized data. When creating a non-gamecard savegame and other images/files, it's initially all 0xFFFFFFFF until it's formatted where some of the blocks are overwritten with encrypted data.

I got a new game SplinterCell3D-Pal and I downloaded the save and it was 128KB of 0xFF, except the first 0x10 bytes which were the letter 'Z' (uppercase) --Elisherer 22:41, 15 October 2011 (CEST)

Fun Facts

If you have facts that you found out by looking at the binary files please share them here:

  • From one save to another the game backups the last files that were in the partition and the entire image header in "random" locations.. --Elisherer 22:41, 15 October 2011 (CEST)

Tools

  • 3dsfuse supports reading and modifying savegames. In the mounted FUSE filesystem, the /output.sav is the raw FLASH save-image. When the save was modified, a separate tool to update the CMAC must be used with /clean.sav, prior to writing output.sav to a gamecard.
  • 3DSExplorer supports reading of savegames, it doesn't support reading the new encrypted savegames and maybe in the future it will support modifying (some of the modyfing code is already implemented).

Japanese