Difference between revisions of "DISA and DIFF"

From 3dbrew
Jump to navigation Jump to search
 
(9 intermediate revisions by the same user not shown)
Line 1: Line 1:
−
This is a page under construction. The purpose is to refactor some non-savegame specific stuff out of the [[Savegames]] page and correct mistakes.
+
This page describes DISA and DIFF format as the underlying container of [[Savegames]], [[Extdata]] and [[Title Database]]. For further format specification of the inner data, please refer to their own page.
 +
 
 +
All data in this page is little-endian. All "unused / padding" fields can contain uninitialized data unless otherwise specified.
  
 
== Overview ==
 
== Overview ==
  
−
DISA and DIFF are two container formats. They are very similar and are used for various purpose in 3DS. Here is a summary table of their usage, the CMAC type and key slot used (the meaning of these is explained in the next section):
+
DISA and DIFF are two container formats. They are very similar and are used for various purposes in 3DS. Here is a summary table of their usage, the CMAC type and the AES key slot used (the meaning of these is explained in the next section):
  
 
{| class="wikitable" border="1"
 
{| class="wikitable" border="1"
Line 15: Line 17:
 
| Gamecard
 
| Gamecard
 
| DISA
 
| DISA
−
| [[#CTR-SAV0|CTR-SAV0]]
+
| [[#CTR-NOR0|CTR-NOR0]]
−
| 0x19
+
| 0x19 / 0x33
 
|-
 
|-
 
| [[Savegames]]
 
| [[Savegames]]
Line 24: Line 26:
 
| 0x30
 
| 0x30
 
|-
 
|-
−
| [[Savegames|System SaveData]]
+
| [[System SaveData]]
 
| [[Flash Filesystem|NAND]]
 
| [[Flash Filesystem|NAND]]
 
| DISA
 
| DISA
Line 54: Line 56:
 
| 0x0B
 
| 0x0B
 
|}
 
|}
−
 
−
This page only describe the container formats. The inner format for each usage is described in the corresponding page.
 
  
 
== Encryption ==
 
== Encryption ==
  
−
DISA and DIFF formats don't have their own encryption specification. They follows the encryption method defined by their media:
+
DISA and DIFF formats don't have their own encryption specification. They follow the encryption method defined by their media:
  
−
* Gamecard savegames has special wear leveling + encryption layers. See [[Savegames]] for detail.
+
* Gamecard savegames have special wear leveling + encryption layers. See [[Savegames]] for detail.
−
* Files on SD follows [[SD Filesystem|the general SD filesystem encryption rule]].
+
* Files on SD follow [[SD Filesystem|the general SD filesystem encryption rule]].
−
* Files on NAND are in cleartext, but encrypted by [[Flash Filesystem|the NAND partition encryption]].
+
* Files on NAND are in cleartext, after decrypting [[Flash Filesystem|the NAND partition encryption]].
  
 
== Format ==
 
== Format ==
  
−
A DISA / DIFF file consists of following components:
+
A DISA / DIFF file consists of the following components:
  
 
* 0x100-byte AES CMAC
 
* 0x100-byte AES CMAC
Line 81: Line 81:
 
== AES CMAC ==
 
== AES CMAC ==
  
−
The AES CMAC is located at the beginning of the DISA / DIFF image, and it is 0x10 long. the rest 0xF0 bytes before the header are unused.
+
The AES CMAC is located at the beginning of the DISA / DIFF image, and it is 0x10 long. The rest 0xF0 bytes before the header are unused.
  
−
The key used for the AES CMAC is generated by the hardware key scrambler. See the keyslot it uses in the table above.
+
The key used for the AES CMAC is generated by the [[AES Registers|hardware key engine]]. See the keyslot it uses in the table above.
  
−
The data being authenticated by the AES CMAC is a 0x20-byte SHA-256 hash of a data block. The data block has different content formats among CMAC types. Effectively, all types of data block contain a copy or a hash of the header, which is the start of the the rest of the verification chain, so the AES CMAC authenticates the whole save image. Each type of data block is explained below.
+
The data being authenticated by the AES CMAC is a 0x20-byte SHA-256 hash of a data block. The data block has different content formats among CMAC types. All types of data block contain a copy or a hash of the header, which is the start of the the rest of the verification chain, so the AES CMAC effectively authenticates the whole save image. Each type of data block is explained below.
  
−
=== CTR-SAV0 ===
+
=== CTR-NOR0 ===
  
 
This CMAC type is used for gamecard savegames. It is 0x28-byte long.
 
This CMAC type is used for gamecard savegames. It is 0x28-byte long.
Line 98: Line 98:
 
| 0x00
 
| 0x00
 
| 8
 
| 8
−
| Magic "CTR-SAV0"
+
| Magic "CTR-NOR0"
 
|-
 
|-
 
| 0x8
 
| 0x8
 
| 0x20
 
| 0x20
 
| SHA-256 of the following 0x108-byte block
 
| SHA-256 of the following 0x108-byte block
−
|-
+
{| class="wikitable" border="1"
−
|
+
! Offset
−
 
+
! Length
−
|
+
! Description
−
 
 
−
|
 
−
 
 
 
|-
 
|-
 
| 0x00
 
| 0x00
 
| 8
 
| 8
−
| Magic "CTR-NOR0"
+
| Magic "CTR-SAV0"
 
|-
 
|-
 
| 0x08
 
| 0x08
 
| 0x100
 
| 0x100
 
| Copy of the DISA header
 
| Copy of the DISA header
 +
|}
 
|}
 
|}
  
Line 140: Line 138:
 
| 0x20
 
| 0x20
 
| SHA-256 of the following 0x108-byte block
 
| SHA-256 of the following 0x108-byte block
−
|-
+
{| class="wikitable" border="1"
−
|
+
! Offset
−
 
+
! Length
−
|
+
! Description
−
 
 
−
|
 
−
 
 
 
|-
 
|-
 
| 0x00
 
| 0x00
Line 156: Line 151:
 
| Copy of the DISA header
 
| Copy of the DISA header
 
|}
 
|}
 +
|}
 +
  
 
=== CTR-SYS0 ===
 
=== CTR-SYS0 ===
Line 237: Line 234:
 
== Header ==
 
== Header ==
  
−
The header defines the rest components of the file (partitions and their tables). All offsets in the header are relative to the beginning of the DISA/DIFF file, except for partition descriptor offsets, which are relative to the beginning of the (active) partition table. DISA and DIFF have different header format.
+
The header located at offset 0x100 defines the rest components of the file (partitions and their tables). All offsets in the header are relative to the beginning of the DISA/DIFF file, except for partition descriptor offsets, which are relative to the beginning of the (active) partition table. DISA and DIFF have different header format.
  
 
=== DISA header ===
 
=== DISA header ===
Line 325: Line 322:
 
Note:
 
Note:
  
−
* When partition count is 1, there is no partition B and all of its related fields are zero.
+
* When the partition count is 1, there is no partition B and all of its related fields are zero.
  
 
=== DIFF header ===
 
=== DIFF header ===
Line 386: Line 383:
 
== Partition table & partition descriptor ==
 
== Partition table & partition descriptor ==
  
−
Their are two partition tables, but only one of them is active. When operating on a DISA / DIFF file, 3DS FS alternately activate one of the two tables, presumably for data backup or atomic file writing. A newly created DISA / DIFF file may have invalid data in its the inactive partition table.
+
There are two partition tables, but only one of them is active. When operating on a DISA / DIFF file, 3DS FS alternately activate one of the two tables, presumably for data backup or atomic file writing. A newly created DISA / DIFF file may have entirely uninitialized data in the inactive partition table.
  
−
One partition table contains one or two partition descriptors , each of which describe the layout of one partition. A partition descriptor contains the following components:
+
One partition table contains one or two partition descriptors , each of which describes the layout of one partition. A partition descriptor contains the following components:
  
 
* DIFI header
 
* DIFI header
Line 397: Line 394:
 
=== DIFI header ===
 
=== DIFI header ===
  
−
The DIFI header locates at the beginning of a partition descriptor. This header defines the rest components of the partition descriptor (IVFC descriptor, DPFS descriptor and partition hash). All offsets are relative to the beginning of the partition descriptor, except for <code>External IVFC level 4 offset</code>, which is related to the beginning of the partition.
+
The DIFI header locates at the beginning of a partition descriptor. This header defines the rest components of the partition descriptor (IVFC descriptor, DPFS descriptor and partition master hash). All offsets are relative to the beginning of the partition descriptor, except for <code>External IVFC level 4 offset</code>, which is relative to the beginning of the partition.
  
 
{| class="wikitable" border="1"
 
{| class="wikitable" border="1"
Line 450: Line 447:
 
| 0x3C
 
| 0x3C
 
| 8
 
| 8
−
| External IVFC level 4 offset
+
| External IVFC level 4 offset (zero if external IVFC level 4 disabled)
 
|}
 
|}
  
Line 459: Line 456:
 
=== IVFC descriptor ===
 
=== IVFC descriptor ===
  
−
This header defines each level of IVFC tree (explained in the section [[#Partition]]). All the offsets are relative to the beginning of DPFS level 3.
+
This header defines each level of IVFC tree (explained in the section [[#Partition]]). All offsets are relative to the beginning of DPFS level 3.
  
 
{| class="wikitable" border="1"
 
{| class="wikitable" border="1"
Line 528: Line 525:
 
| 0x58
 
| 0x58
 
| 8
 
| 8
−
| IVFC level 4 offset (unused for external IVFC level 4?)
+
| IVFC level 4 offset (unused if external IVFC level 4 enabled)
 
|-
 
|-
 
| 0x60
 
| 0x60
Line 535: Line 532:
 
|-
 
|-
 
| 0x68
 
| 0x68
−
| 8
+
| 4
 
| IVFC level 4 block size in log2
 
| IVFC level 4 block size in log2
 +
|-
 +
| 0x6C
 +
| 4
 +
| Padding
 
|-
 
|-
 
| 0x70
 
| 0x70
 
| 8
 
| 8
−
| IVFC descriptor size? usually 0x78
+
| IVFC descriptor size? The value is usually 0x78
 
|}
 
|}
  
 
=== DPFS descriptor ===
 
=== DPFS descriptor ===
  
−
This header defines each level of DPFS tree (explained in the section [[#Partition]]). All the offsets are relative to the beginning of the partition.
+
This header defines each level of DPFS tree (explained in the section [[#Partition]]). All offsets are relative to the beginning of the partition.
  
 
{| class="wikitable" border="1"
 
{| class="wikitable" border="1"
Line 639: Line 640:
 
=== DPFS tree ===
 
=== DPFS tree ===
  
−
Everything inside the DPFS tree comes in pairs, and at one time only one of a pair is active. The tree is probably designed for atomic writing: for a file writing operation, it writes to the inactive part, then commit the data by switching a bit to activate it.
+
Everything inside the DPFS tree comes in pairs, and at one time only one of a pair is active. The tree is probably designed for atomic writing: for a file writing operation, it writes to the inactive part, then commits the data by switching a bit to activate it.
  
 
Each level of DPFS tree consists of a pair of chunks. The size of one chunk is defined as it in the DPFS descriptor, so the total size of a level is actually twice as large as the size recorded in the descriptor. For level 1 and 2, each chunk is a bit array, in which each bit corresponds to a block in the next level (the block size of the next level is also defined in the DPFS descriptor). This bit indicates which one of the pair in the next level is active for this block: 0 means the first one and 1 means the second one. The active chunk of level 1 is selected by <code>DPFS tree level 1 selector</code> in the DIFI header. The bit array is encoded in u32 array, with MSB as the first bit of each 32 bits.
 
Each level of DPFS tree consists of a pair of chunks. The size of one chunk is defined as it in the DPFS descriptor, so the total size of a level is actually twice as large as the size recorded in the descriptor. For level 1 and 2, each chunk is a bit array, in which each bit corresponds to a block in the next level (the block size of the next level is also defined in the DPFS descriptor). This bit indicates which one of the pair in the next level is active for this block: 0 means the first one and 1 means the second one. The active chunk of level 1 is selected by <code>DPFS tree level 1 selector</code> in the DIFI header. The bit array is encoded in u32 array, with MSB as the first bit of each 32 bits.
  
−
To access data in level 3, one need to check the bits in level 1 and level 2 to know which chunk of level 3 is active for the accessing location. For example, for a following configuration:
+
To access data in level 3, one needs to check the bits in level 1 and level 2 to know which chunk of level 3 is active for the accessed location. For example, for a following configuration:
  
 
* Level 1: size = 4 bytes = 32 bits
 
* Level 1: size = 4 bytes = 32 bits
Line 649: Line 650:
 
* Level 3: size = 0x1B7F000, block size = 0x1000, block size = 0x1000 bytes
 
* Level 3: size = 0x1B7F000, block size = 0x1000, block size = 0x1000 bytes
  
−
and one want to read byte at 0x1234567 of level 3, the following calculation is performed:
+
if one want to read byte at 0x1234567 of level 3, the following calculation is performed:
  
 
* get level 2 bit index <code>0x1234567 / 0x1000 = 0x1234</code>, and its byte location <code>0x1234 / 8 = 0x246</code>
 
* get level 2 bit index <code>0x1234567 / 0x1000 = 0x1234</code>, and its byte location <code>0x1234 / 8 = 0x246</code>
Line 663: Line 664:
 
=== IVFC tree ===
 
=== IVFC tree ===
  
−
The IVFC tree is used for data verification. It is very similar to the IVFC tree in RomFS*, except that it has an additional level in DISA / DIFF. For level 1, 2 and 3, each level is a list of SHA-256 hash, of which each corresponds to a block of the next level, padded to block size (the block size of the next level is defined in the IVFC descriptor).
+
The IVFC tree is used for data verification. It is very similar to the IVFC tree in [[RomFS]], except that it has an additional level here. For level 1, 2 and 3, each level is a list of SHA-256 hash, of which each corresponds to a block of the next level which is zero-padded to align the block size (the block size of the next level is defined in the IVFC descriptor).
  
−
The partition master hash in the partition descriptor can be seen as IVFC level 0, which hashes level 1 following the same rule. The partition hash is usually 0x20 long, consisting only one hash. This is because most DISA / DIFF files are not large enough to have multiple hashes on the top level, which isn't the case for some title database files.
+
The partition master hash in the partition descriptor can be seen as IVFC level 0, which hashes level 1 following the same rule. The master hash is usually 0x20 long, consisting only one hash. This is because most DISA / DIFF files are not large enough to have multiple hashes on the top level, which isn't the case for some title database files.
  
−
However, not all data are hashed - only ranges that have been written with valid data are properly hashed.
+
However, not all data is hashed - only ranges that have been written with valid data are properly hashed.
  
 
Level 4 is the actual content of the partition, which is what the container format essentially contains.
 
Level 4 is the actual content of the partition, which is what the container format essentially contains.
  
−
== Extract content from a DISA / DIFF container ==
+
== Extracting content from a DISA / DIFF container ==
  
 
* Find the active partition table and the partition(s).
 
* Find the active partition table and the partition(s).
Line 678: Line 679:
 
* The IVFC level 4 is the inner content of the file. The format of it varies among different usage. Refer their own page for further extraction.
 
* The IVFC level 4 is the inner content of the file. The format of it varies among different usage. Refer their own page for further extraction.
  
−
== Chain of Trust ==
+
== Chain of trust ==
  
 
* AES CMAC verifies the header.
 
* AES CMAC verifies the header.
Line 684: Line 685:
 
* In the partition table, each descriptor verifies level 1 of its IVFC tree via the master hash.
 
* In the partition table, each descriptor verifies level 1 of its IVFC tree via the master hash.
 
* Each IVFC level verifies the next level, until the level 4, which is the inner content.
 
* Each IVFC level verifies the next level, until the level 4, which is the inner content.
 +
 +
== Summary diagram ==
 +
 +
[[File:Disa-diff.png]]

Latest revision as of 07:21, 2 January 2020

This page describes DISA and DIFF format as the underlying container of Savegames, Extdata and Title Database. For further format specification of the inner data, please refer to their own page.

All data in this page is little-endian. All "unused / padding" fields can contain uninitialized data unless otherwise specified.

Overview[edit]

DISA and DIFF are two container formats. They are very similar and are used for various purposes in 3DS. Here is a summary table of their usage, the CMAC type and the AES key slot used (the meaning of these is explained in the next section):

Usage Media Container format CMAC type CMAC Keyslot
Savegames Gamecard DISA CTR-NOR0 0x19 / 0x33
Savegames SD DISA CTR-SIGN 0x30
System SaveData NAND DISA CTR-SYS0 0x30
Private Extdata SD DIFF CTR-EXT0 0x30
Shared Extdata NAND DIFF CTR-EXT0 0x30
Title Database SD DIFF CTR-9DB0 0x30
Title Database NAND DIFF CTR-9DB0 0x0B

Encryption[edit]

DISA and DIFF formats don't have their own encryption specification. They follow the encryption method defined by their media:

Format[edit]

A DISA / DIFF file consists of the following components:

  • 0x100-byte AES CMAC
  • 0x100-byte Header
  • Secondary partition table
    • Contains one or two partition descriptors, depending on the number of partitions.
  • Primary partition table
    • Same layout as the secondary one
  • Partition A
  • Partition B (optional)
    • can only exist for DISA.

AES CMAC[edit]

The AES CMAC is located at the beginning of the DISA / DIFF image, and it is 0x10 long. The rest 0xF0 bytes before the header are unused.

The key used for the AES CMAC is generated by the hardware key engine. See the keyslot it uses in the table above.

The data being authenticated by the AES CMAC is a 0x20-byte SHA-256 hash of a data block. The data block has different content formats among CMAC types. All types of data block contain a copy or a hash of the header, which is the start of the the rest of the verification chain, so the AES CMAC effectively authenticates the whole save image. Each type of data block is explained below.

CTR-NOR0[edit]

This CMAC type is used for gamecard savegames. It is 0x28-byte long.

Offset Length Description
0x00 8 Magic "CTR-NOR0"
0x8 0x20 SHA-256 of the following 0x108-byte block
Offset Length Description
0x00 8 Magic "CTR-SAV0"
0x08 0x100 Copy of the DISA header

CTR-SIGN[edit]

This CMAC type is used for SD savegames. It is 0x30-byte long.

Offset Length Description
0x00 8 Magic "CTR-SIGN"
0x08 8 Title ID
0x10 0x20 SHA-256 of the following 0x108-byte block
Offset Length Description
0x00 8 Magic "CTR-SAV0"
0x08 0x100 Copy of the DISA header


CTR-SYS0[edit]

This CMAC type is used for NAND system save. It is 0x110-byte long.

Offset Length Description
0x00 8 Magic "CTR-SYS0"
0x08 8 Save ID. The higher word is always zero
0x10 0x100 Copy of the DISA header

CTR-EXT0[edit]

This CMAC type is used for extdata. It is 0x11C-byte long.

Offset Length Description
0x00 8 Magic "CTR-EXT0"
0x08 8 Extdata ID
0x10 4 0 for Quota.dat, 1 otherwise
0x14 4 ID in the device file name. 0 for Quota.dat
0x18 4 ID in the device directory name that the file is in. 0 for Quota.dat
0x1C 0x100 Copy of the DIFF header

CTR-9DB0[edit]

This CMAC type is used for title database. It is 0x10C-byte long.

Offset Length Description
0x00 8 Magic "CTR-9DB0"
0x08 4 Database ID. Each .db file has its own ID
0x0C 0x100 Copy of the DIFF header

Header[edit]

The header located at offset 0x100 defines the rest components of the file (partitions and their tables). All offsets in the header are relative to the beginning of the DISA/DIFF file, except for partition descriptor offsets, which are relative to the beginning of the (active) partition table. DISA and DIFF have different header format.

DISA header[edit]

Offset Length Description
0x00 4 Magic "DISA"
0x04 4 Magic 0x40000
0x08 4 Partition count, 1 or 2
0x0C 4 Padding
0x10 8 Secondary partition table offset
0x18 8 Primary partition table offset
0x20 8 Partition table size
0x28 8 Partition A descriptor offset in the partition table
0x30 8 Partition A descriptor size
0x38 8 Partition B descriptor offset in the partition table
0x40 8 Partition B descriptor size
0x48 8 Partition A offset
0x50 8 Partition A size
0x58 8 Partition B offset
0x60 8 Partition B size
0x68 1 Active partition table, 0 = primary, 1 = secondary
0x69 3 Padding
0x6C 0x20 SHA-256 over the active partition table
0x8C 0x74 Unused

Note:

  • When the partition count is 1, there is no partition B and all of its related fields are zero.

DIFF header[edit]

Offset Length Description
0x00 4 Magic "DIFF"
0x04 4 Magic 0x30000
0x08 8 Secondary partition table/descriptor offset
0x10 8 Primary partition table/descriptor offset
0x18 8 Partition table/descriptor size
0x20 8 Partition (A) offset
0x28 8 Partition (A) size
0x30 4 Active partition descriptor, 0 = primary, 1 = secondary
0x34 0x20 SHA-256 over the active partition table/descriptor
0x54 8 Unique identifier
0x5C 0xA4 Unused, might contain leftover data

Note:

  • Since DIFF can only contain one partition, a partition table can only have one partition descriptor, so they become the same concept here.
  • See Extdata for its usage of the unique identifier field. For title database files, this field is zero.

Partition table & partition descriptor[edit]

There are two partition tables, but only one of them is active. When operating on a DISA / DIFF file, 3DS FS alternately activate one of the two tables, presumably for data backup or atomic file writing. A newly created DISA / DIFF file may have entirely uninitialized data in the inactive partition table.

One partition table contains one or two partition descriptors , each of which describes the layout of one partition. A partition descriptor contains the following components:

  • DIFI header
  • IVFC descriptor
  • DPFS descriptor
  • Partition master hash

DIFI header[edit]

The DIFI header locates at the beginning of a partition descriptor. This header defines the rest components of the partition descriptor (IVFC descriptor, DPFS descriptor and partition master hash). All offsets are relative to the beginning of the partition descriptor, except for External IVFC level 4 offset, which is relative to the beginning of the partition.

Offset Length Description
0x00 4 Magic "DIFI"
0x04 4 Magic 0x10000
0x08 8 IVFC descriptor offset
0x10 8 IVFC descriptor size
0x18 8 DPFS descriptor offset
0x20 8 DPFS descriptor size
0x28 8 Partition hash offset
0x30 8 Partition hash size
0x38 1 If none zero, enable external IVFC level 4.
0x39 1 DPFS tree level 1 selector
0x3A 2 Padding
0x3C 8 External IVFC level 4 offset (zero if external IVFC level 4 disabled)

Note:

  • The meaning of fields after 0x38 are explained in the section #Partition

IVFC descriptor[edit]

This header defines each level of IVFC tree (explained in the section #Partition). All offsets are relative to the beginning of DPFS level 3.

Offset Length Description
0x00 4 Magic "IVFC"
0x04 4 Magic 0x20000
0x08 8 Master hash size = partition master hash size in DIFI header
0x10 8 IVFC level 1 offset
0x18 8 IVFC level 1 size
0x20 4 IVFC level 1 block size in log2
0x24 4 Padding
0x28 8 IVFC level 2 offset
0x30 8 IVFC level 2 size
0x38 4 IVFC level 2 block size in log2
0x3C 4 Padding
0x40 8 IVFC level 3 offset
0x48 8 IVFC level 3 size
0x50 4 IVFC level 3 block size in log2
0x54 4 Padding
0x58 8 IVFC level 4 offset (unused if external IVFC level 4 enabled)
0x60 8 IVFC level 4 size
0x68 4 IVFC level 4 block size in log2
0x6C 4 Padding
0x70 8 IVFC descriptor size? The value is usually 0x78

DPFS descriptor[edit]

This header defines each level of DPFS tree (explained in the section #Partition). All offsets are relative to the beginning of the partition.

Offset Length Description
0x00 4 Magic "DPFS"
0x04 4 Magic 0x10000
0x08 8 DPFS level 1 offset
0x10 8 DPFS level 1 size
0x18 4 DPFS level 1 block size in log2 (unused?)
0x1C 4 Padding
0x20 8 DPFS level 2 offset
0x28 8 DPFS level 2 size
0x30 4 DPFS level 2 block size in log2
0x34 4 Padding
0x38 8 DPFS level 3 offset
0x40 8 DPFS level 3 size
0x48 4 DPFS level 3 block size in log2
0x4C 4 Padding

Partition master hash[edit]

This is a SHA-256 hash list over IVFC level 1. See #IVFC tree for explanation.

Partition[edit]

A partition can have two types of layout. This is determined by the field DIFF + 0x38 (Enable external IVFC level 4).

The layout type 0 (external IVFC level 4 disabled) contains

  • DPFS level 1
  • DPFS level 2
  • DPFS level 3, and inside
    • IVFC level 1
    • IVFC level 2
    • IVFC level 3
    • IVFC level 4 (the actual content data)

The layout type 1 (external IVFC level 4 enabled) contains

  • DPFS level 1
  • DPFS level 2
  • DPFS level 3, and inside
    • IVFC level 1
    • IVFC level 2
    • IVFC level 3
  • IVFC level 4 (the actual content data, note that this is out side DPFS level 3)

DPFS tree[edit]

Everything inside the DPFS tree comes in pairs, and at one time only one of a pair is active. The tree is probably designed for atomic writing: for a file writing operation, it writes to the inactive part, then commits the data by switching a bit to activate it.

Each level of DPFS tree consists of a pair of chunks. The size of one chunk is defined as it in the DPFS descriptor, so the total size of a level is actually twice as large as the size recorded in the descriptor. For level 1 and 2, each chunk is a bit array, in which each bit corresponds to a block in the next level (the block size of the next level is also defined in the DPFS descriptor). This bit indicates which one of the pair in the next level is active for this block: 0 means the first one and 1 means the second one. The active chunk of level 1 is selected by DPFS tree level 1 selector in the DIFI header. The bit array is encoded in u32 array, with MSB as the first bit of each 32 bits.

To access data in level 3, one needs to check the bits in level 1 and level 2 to know which chunk of level 3 is active for the accessed location. For example, for a following configuration:

  • Level 1: size = 4 bytes = 32 bits
  • Level 2: size = 0x380 bytes = 0x1C00 bits, block size = 0x80 bytes
  • Level 3: size = 0x1B7F000, block size = 0x1000, block size = 0x1000 bytes

if one want to read byte at 0x1234567 of level 3, the following calculation is performed:

  • get level 2 bit index 0x1234567 / 0x1000 = 0x1234, and its byte location 0x1234 / 8 = 0x246
  • get level 1 bit index 0x246 / 0x80 = 4
  • get level1_selector from DIFI header
  • read level2_selector = Level1[level1_selector].bits[4];
  • read level3_selector = Level2[level2_selector].bits[0x1234];
  • read data = Level3[level3_selector].bytes[0x1234567] as the final data
  • in the code above Levelx[k] means the k-th chunk in level x, where k = 0, 1. .bits[n] is expanded to (.u32_array[n / 32] >> (31 - n % 32)) & 1 as the bit array is encoded in u32 array.

Effectively, the active data is scattered among the two level 3 chunk. One can assemble the whole active level 3 image following the same rule.

IVFC tree[edit]

The IVFC tree is used for data verification. It is very similar to the IVFC tree in RomFS, except that it has an additional level here. For level 1, 2 and 3, each level is a list of SHA-256 hash, of which each corresponds to a block of the next level which is zero-padded to align the block size (the block size of the next level is defined in the IVFC descriptor).

The partition master hash in the partition descriptor can be seen as IVFC level 0, which hashes level 1 following the same rule. The master hash is usually 0x20 long, consisting only one hash. This is because most DISA / DIFF files are not large enough to have multiple hashes on the top level, which isn't the case for some title database files.

However, not all data is hashed - only ranges that have been written with valid data are properly hashed.

Level 4 is the actual content of the partition, which is what the container format essentially contains.

Extracting content from a DISA / DIFF container[edit]

  • Find the active partition table and the partition(s).
  • Unwrap DPFS tree of partition(s) by reconstructing active data.
  • Unwrap IVFC tree. Either take out level 4 directly, or, better, verify all the hashes and poison the data that is not properly hashed.
  • The IVFC level 4 is the inner content of the file. The format of it varies among different usage. Refer their own page for further extraction.

Chain of trust[edit]

  • AES CMAC verifies the header.
  • The header verifies the active partition table via the table hash.
  • In the partition table, each descriptor verifies level 1 of its IVFC tree via the master hash.
  • Each IVFC level verifies the next level, until the level 4, which is the inner content.

Summary diagram[edit]

Disa-diff.png

Retrieved from "https://www.3dbrew.org/w/index.php?title=DISA_and_DIFF&oldid=21144"