|4-0||Write FIFO count (0-16)|
|9-5||Read FIFO count (0-16)|
|10||Flush write FIFO (1=Clear write FIFO)|
|11||Flush read fifo (1=Clear read FIFO)|
|18-16||MAC size (encoding = (maclen-2)/2)|
|19||? (MAC related)|
|20||MAC input control (0 = read MAC from FIFO, 1 = read from MAC register)|
|21||MAC status (0 = invalid, 1 = verified)|
|22||Output endianness (1=Big endian, 0=Little endian)|
|23||Input endianness (1=Big endian, 0=Little endian)|
|24||Output word order (1=Normal order, 0=Reversed order)|
|25||Input word order (1=Normal order, 0=Reversed order)|
|26||Update keyslot (set to 1 when the keyslot is changed)|
|29-27||Mode (0=CCM decrypt, 1=CCM encrypt, 2=CTR, 3=CTR, 4=CBC decrypt, 5=CBC encrypt, 6=ECB decrypt, 7=ECB encrypt)|
|30||Interrupt enable (1=enable, 0=disable)|
|31||Start (1=enable/busy, 0=idle)|
TODO: Explain what this does.
Up to 128 bytes of input data can be buffered.
|6||Hardware key-generator type: 0 = 3DS, 1 = DSi.|
|7||This normally has value 1 written here when updating keys. 0 = disable key FIFO flush, 1 = enable key FIFO flush.|
Bit6 is only used when keyslots >=4 are used, value1 has the same affect as doing key-init with the TWL keyslots. Bit6 is checked when a keyY was written, for when the final-normalkey needs updated via the key-generator.
This register specifies the counter (CTR mode), nonce (CCM mode) or the initialization vector (CBC mode) depending on the mode of operation. For CBC and CTR mode this register takes up the full 16 bytes, but for CCM mode the nonce is only the first 12 bytes.
This register specifies the message authentication code (MAC) for use in CCM mode.
Endianness and word order
When writing to the REG_AESCTR or REG_AESMAC register, the hardware will process the written data according to the current input endianness specified in AESCNT. However, the current specified input word order will not be honored for this register, and always defaults to reversed word order. Therefore, for normal word order, the reversal must be carried out manually if required.
There are 0x40 keyslots, each of which stores three keys called keyX, keyY and normalkey. All keys can be set explicitly, but the normalkey can optionally be generated using a hardware key scrambler instead (see below). There is no way to read the contents of a keyslot.
|0x00-0x03||These are the TWL keyslots, the key-data for these can be set via the REG_AESKEY0-REG_AESKEY3 registers. These keyslots are initialized by NATIVE_FIRM. The console-unique portion of two of these keyslots are only initialized by NATIVE_FIRM during initial hard-boot.|
|0x0D||See EncryptDecryptAes, this uses the hardware key-scrambler.|
|0x11||This is used for general normal-key crypto, where the normal-key is set by FIRM. This keyslot is also used by the New3DS FIRM arm9 binary loader.|
|0x14||Starting with 5.0.0-11, NATIVE_FIRM Process9 now sets the keyY for this to the same one it uses for initializing 3 of the keyslots' keyYs from here.|
|0x20..0x23||All of these keyslots(initialized by bootrom) are set to the same key-data. These seem to be set to a regular normal-key?|
|0x25||The keyX and keyY initialized by bootrom for this keyslot are console-unique. This keyslot is used for the v7.0 NCCH encryption, the keyX is initialized during NATIVE_FIRM boot. The keyY/CTR used for this keyslot is the same as keyslot 0x2C.|
|0x2C||Used to decrypt NCCH, the keyY is set by Process9(see here regarding the keyY). Keyslots 0x2C..0x2F all use the same keyX, set by bootrom.|
|0x2D||See EncryptDecryptAes, this uses the hardware key-scrambler.|
|0x2E||See EncryptDecryptAes. This keyY is set by NATIVE_FIRM.|
|0x2F||Initially this keyslot has the same keyY as keyslot 0x2D, initialized by bootrom. This keyY is initialized during NATIVE_FIRM boot. This is the keyslot used for calculating v6.0 gamecard savegames' keyYs.|
|0x31||See EncryptDecryptAes, this uses the hardware key-scrambler. NATIVE_FIRM sets this keyY to the same one used for keyslot 0x2E.|
|0x32||See EncryptDecryptAes. This keyslot keyX is the same keyX used for keyslot 0x31.|
|0x34-0x37||All four of these keyslots use the same keyX. Keyslots 0x35/0x36 use the same keyY, see EncryptDecryptAes for keyslot 0x36.|
|0x38||See EncryptDecryptAes, this uses the hardware key-scrambler.|
|0x39||See EncryptDecryptAes. This keyslot keyX is the same keyX used for keyslot 0x38. NATIVE_FIRM sets this keyY to the same one used for keyslot 0x2E.|
|0x3D||This keyslot uses keyY. Used to decrypt title keys in Ticket. Used by Gateway.|
|0x3E||This keyslot uses an unique keyX/keyY.|
|0x3F||This keyslot uses an unique keyX/keyY.|
The contents of the keyslot specified in REG_AESKEYCNT can be updated by consecutively writing four words to REG_AESKEYXFIFO (keyX), REG_AESKEYYFIFO(keyY), or REG_AESKEYFIFO (normalkey).
After writing to a keyslot, the keyslot in REG_AESKEYCNT must be set again, even when writing to the same keyslot.
Writing to the key FIFOs with byte writes results in the AES engine converting the byte to a word for setting the key word, with this: word = (byteval) | (byteval<<8) | (byteval<<16) | (byteval<<24). The result is the same regardless of which FIFO register byte was written to.
The TWL keyslots 0x00-0x03 can be set directly by writing to the REG_AESKEY0-REG_AESKEY3 registers.
The ARM9 bootrom initializes the keyX for each 3DS keyslot, the ARM9 bootrom also initializes the keyY for the keyslots where NATIVE_FIRM doesn't set the keyY. In certain cases Process9 may also set the keyX.
Hardware key generator
A dedicated hardware key generator can be used to generate a keyslot's normalkey from its keyX and keyY by enabling the corresponding bit in REG_AESKEYCNT. If enabled, the hardware key generator is triggered by writing the keyY (which also seems to be the only way to trigger it). The algorithm used for key generation is unknown.
Unless noted otherwise, all keyslots on retail units use the hardware key-scrambler.
FIRM-launch key clearing
Starting with 9.0.0-20 the Process9 FIRM-launch code now "clears" the following AES keyslots, with certain keydata by writing the normal-key: 0x15 and 0x18-0x20. These are the keyslots used by the New3DS FIRM arm9bin loader(minus keyslot 0x11), so the New3DS Process9 presumably does this too.