Encryption
On the 3DS savegames are stored much like on the DS, that is on a FLASH chip in the gamecart. On the DS these savegames were stored in plaintext but on the 3DS a layer of encryption was added. This is highly likely a streamcipher, as the contents of several savegames exhibit the odd behaviour that xor-ing certain parts of the savegame together will result in the plaintext appearing.
The reason this works is because the streamcipher used has a period of 512 bytes. That is to say, it will repeat the same keystream after 512 bytes. The way you encrypt with a streamcipher is you XOR your data with the keystream as it is produced. Unfortunately, if your streamcipher repeats and you are encrypting a known plaintext (in our case, zeroes) you are basically giving away your valuable keystream.
So how do you use this to decrypt a savegame on a 3DS? First off, you chunk up the savegame into 512 byte chunks. Then, you bin these chunks by their contents, discarding any that contain only FF. Now look for the most common chunk. This is your keystream. Now XOR the keystream with your original savegame and you should have a fully decrypted savegame. XOR with the keystream again to produce an encrypted savegame.
Wearleveling
The 3DS employs a wearleveling scheme on the savegame FLASH chips. This is done trough blockmaps. Depending on the size of the flashchip, these are located somewhere at the beginning of the flashchip, in the first sector. The structure is as follows:
struct sector_entry { uint8_t virt_sec; // Mapped to sector uint8_t prev_virt_sec; // Physical sector previously mapped to uint8_t phys_sec; // Mapped from sector uint8_t prev_phys_sec; // Virtual sector previously mapped to uint8_t phys_realloc_cnt;// Amount of times virtual sector has been remapped uint8_t virt_realloc_cnt;// Amount of times physical sector has been remapped uint8_t chksums[8]; } __attribute__((packed)); struct long_sector_entry { struct sector_entry sector; struct sector_entry dupe; uint32_t magic; };
With magic being a constant 0x080d6ce0.
Filesystem
Savefiles stored on the FLASH are using a custom FS.
It seems the file entries are stored somewhere in the third block.
The first entry is the root directory, stored with a filename of '!'.
struct fs_entry { u32 nodes; u8 filename[0x10]; u32 index; u32 unk1; // magic? u32 block_offset; u32 file_size; u32 unk2; u32 unk3; // flags and/or date? u32 unk4; }
Example from Super MonkeyBall 3D:
0003800: 04000000 21000000 00000000 00000000 ....!........... 0003810: 00000000 00000000 00000000 00000000 ................ 0003820: 00000000 00000000 00000000 00000000 ................ 0003830: 01000000 736d6233 64732e64 61740000 ....smb3ds.dat.. 0003840: 00000000 00000000 d57b1100 05000000 .........{...... 0003850: e4060000 00000000 c8cf0008 00000000 ................ 0003860: 01000000 6d677265 706c6179 30302e64 ....mgreplay00.d 0003870: 61740000 01000000 d57b1100 09000000 at.......{...... 0003880: 1c210000 00000000 cd331000 00000000 .!.......3...... 0003890: 01000000 6d677265 706c6179 30312e64 ....mgreplay01.d 00038a0: 61740000 02000000 d57b1100 1a000000 at.......{...... 00038b0: 1c210000 00000000 00000000 00000000 .!..............