Hardware calibration
Jump to navigation
Jump to search
File format
The file consists out of a 0x200 big header (though the actual header is only 0x30 bytes, the rest is zerofilled), plus data whose size is indicated in the header.
Header
| Offset | Size | Description |
|---|---|---|
| 0x0 | 0x4 | Magic "CCAL" |
| 0x4 | 0x4 | Version |
| 0x8 | 0x4 | Data size, always 0x7D0 |
| 0xC | 0x1 | Model version (?) |
| 0xD | 0x1 | CAL revision (incremented each time the CAL file is updated) |
| 0xE | 0x2 | Bitmask of successful Aging tests |
| 0x10 | 0x20 | Signature of the data section.
HMACSHA256 is used always except in the below cases where SHA256 is used: - devunits - PARTNER-DEBUGGER - PARTNER-CAPTURE - the SNAKE counterparts of the above - SNAKE-IS-DEBUGGER |
| 0x30 | 0x1D0 | Zerofilled, padding for the 512byte block size |
Aging masks
| Bit | Description |
|---|---|
| 0 | FCRAM |
| 1 | LCD flicker (always successful) |
| 2 | Camera |
| 3 | Touch panel (always successful) |
| 4 | Circle pad (analog stick) |
| 5 | Codec |
| 6 | Gyroscope |
| 7 | RTC |
| 8 | Accelerometer |
| 9 | Surround |
| A | Power saving mode (ABL) |
| B | 3D screen (ULCD) |
| C | Backlight PWM |
| D | Analog stick A (???) |
| E | Camera extensions |
| F | Power saving mode (ABL) in legacy (DSi/GBA) mode |
Data blocks
| Offset | Size | ConfigInfoBlk | Description |
|---|---|---|---|
| 0x00 | 0x10 | 00040000 | Touch panel |
| 0x14 | 0x08 | ??? | Circle pad |
| 0x20 | 2* | 00050000 | Screen flicker |
| 0x24 | 1* | ??? | RTC |
| 0x28 | 1* | ??? | DSPRAM related |
| 0x30 | 0x8A | ??? | Camera position |
| 0xBC | 0x12 | ??? | Gyroscope |
| 0xD0 | 0xC | ??? | Accelerometer |
| 0xE0 | 0x134 | ??? | Codec |
| 0x218 | 0x06 | ??? | Programmable Infrared Transmitter (PIT) |
| 0x220 | 0x214 | ??? | 3D filters |
| 0x440 | 0x20 | ??? | Power saving mode |
| 0x470 | 0x20 | ??? | ??? |
| 0x4A0 | 0x38 | 0x00050002 | Backlight PWM |
| 0x4E0 | 0x18 | ??? | Circle pad extra |
| 0x500 | 0xC | ??? | ??? |
| 0x510 | 0x20 | ??? | ??? |
| 0x540 | 0x08 | ??? | MCU |
| 0x550 | 0x02 | ??? | 3D screen (ULCD) delay |
| 0x560 | 0x08 | ??? | Microphone echo cancellation |
| 0x570 | 0x10C | ??? | Power saving mode (ABL) extra |
| 0x680 | 0x08 | ??? | CStick (Right stick) |
| 0x690 | 0x18 | ??? | Quad Tracking Module (QTM) |
Data block formats
Touch
| Offset | Description |
|---|---|
| 0x00 | s16 RawX0 |
| 0x02 | s16 RawY0 |
| 0x04 | s16 PointX0 |
| 0x06 | s16 PointY0 |
| 0x08 | s16 RawX1 |
| 0x0A | s16 RawY1 |
| 0x0C | s16 PointX1 |
| 0x0E | s16 PointY1 |
Circle pad
| Offset | Description |
|---|---|
| 0x00 | s16 CenterX |
| 0x02 | s16 CenterY |
Screen flicker
| Offset | Description |
|---|---|
| 0 | u8 FlickerTop |
| 1 | u8 FlickerBottom |
| 2* | Checksum low byte, NOT THIS[0] |
| 3* | Checksum high byte, THIS[1] |
RTC
| Offset | Description |
|---|---|
| 0 | u8 CompensationValue (???) |
| 1* | Checksum byte, NOT THIS[0] |
Camera position
| Offset | Description |
|---|---|
| 0x00 | u32 flags |
| 0x04 | float scale |
| 0x08 | float RotationZ |
| 0x0C | float TranslationX |
| 0x10 | float TranslationY |
| 0x14 | float RotationX |
| 0x18 | float RotationY |
| 0x1C | float ViewAngleRight |
| 0x20 | float ViewAngleLeft |
| 0x24 | float ChartDistance(???) |
| 0x28 | float CameraDistance |
| 0x2C | s16 ImageWidth |
| 0x2E | s16 ImageHeight |
| 0x30 | u8 reserved[0x10] |
| 0x40 | u8 ???[0x40] |
| 0x80 | s16 aeBaseTarget(???) |
| 0x82 | s16 kRL |
| 0x84 | s16 kGL |
| 0x86 | s16 kBL |
| 0x88 | s16 ccmPosition |
Gyro
| Offset | Description |
|---|---|
| 0x00 | s16 ZeroX |
| 0x02 | s16 PlusX |
| 0x04 | s16 MinusX |
| 0x06 | s16 ZeroY |
| 0x08 | s16 PlusY |
| 0x0A | s16 MinusY |
| 0x0C | s16 ZeroZ |
| 0x0E | s16 PlusZ |
| 0x10 | s16 MinusZ |
Accel
| Offset | Description |
|---|---|
| 0x00 | s16 OffsetX |
| 0x02 | s16 ScaleX |
| 0x04 | s16 OffsetY |
| 0x06 | s16 ScaleY |
| 0x08 | s16 OffsetZ |
| 0x0A | s16 ScaleZ |
CDC
| Offset | Description |
|---|---|
| 0x00 | u8 DriverGainHP |
| 0x01 | u8 DriverGainSP |
| 0x02 | u8 AnalogVolumeHP |
| 0x03 | u8 AnalogVolumeSP |
| 0x04 | s8 ShutterVolume[2] |
| 0x06 | u8 MicrophoneBias |
| 0x07 | u8 QuickCharge (???) |
| 0x08 | u8 PGA_GAIN (microphone gain) |
| 0x09 | u8 reserved[3] |
| 0x0C | s16 FilterHP32[3*5] |
| 0x2A | s16 FilterHP47[3*5] |
| 0x48 | s16 FilterSP32[3*5] |
| 0x66 | s16 FilterSP47[3*5] |
| 0x84 | s16 FilterMic32[(1+2)+((1+4)*5)] |
| 0xBC | s16 FilterMic47[(1+2)+((1+4)*5)] |
| 0xF4 | s16 FilterFree[(1+2)+((1+4)*5)] |
| 0x12C | u8 AnalogInterval |
| 0x12D | u8 AnalogStabilize |
| 0x12E | u8 AnalogPrecharge |
| 0x12F | u8 AnalogSense |
| 0x130 | u8 AnalogDebounce |
| 0x131 | u8 Analog_XP_Pullup |
| 0x132 | u8 YM_Driver |
| 0x133 | u8 reserved |
PIT
| Offset | Description |
|---|---|
| 0x00 | u16 VisibleFactor |
| 0x02 | u16 IRFactor |
3D filters
| Offset | Description |
|---|---|
| 0x00 | u16 SpecialFilter[0x100] |
| 0x200 | u32 IIRSurroundFilter[5] |
ABL
| Offset | Description |
|---|---|
| 0x00 | u32 DitherPattern |
| 0x04 | s16 StartX |
| 0x06 | s16 StartY |
| 0x08 | u16 SizeX |
| 0x0A | u16 SizeY |
| 0x0C | s16 GTH_Ratio |
| 0x0E | u8 DitherMode |
| 0x0F | u8 MinRS |
| 0x10 | u8 MaxRS |
| 0x11 | u8 MinGTH |
| 0x12 | u8 MinMax (???) |
| 0x13 | u8 ExMax (???) |
| 0x14 | u8 inertia |
| 0x15 | u8 LutListRS[9] |
| 0x1E | u8 reserved[2] |
BLPWM
| Offset | Description |
|---|---|
| 0x00 | float coefficient[3][3] |
| 0x24 | u8 NumLevels |
| 0x25 | u8 padding |
| 0x26 | u16 brightnesses[7]; |
| 0x34 | u16 BaseDivisor |
| 0x36 | u16 MinimumBrightnessHw |
Circle pad extra
| Offset | Description |
|---|---|
| 0x00 | float ScaleX |
| 0x04 | float ScaleY |
| 0x08 | s16 MaxX |
| 0x0A | s16 MinX |
| 0x0C | s16 MaxY |
| 0x0E | s16 MinY |
| 0x10 | s16 type |
| 0x12 | u8 unknown_padding[6] |
MCU
| Offset | Description |
|---|---|
| 0x00 | s16 SVR2_Min (???) |
| 0x02 | s16 SVR2_Max (???) |
| 0x04 | s16 VolumeSliderMin |
| 0x06 | s16 VolumeSliderMax |
ULCD delay
| Offset | Description |
|---|---|
| 0x00 | u8 To2D |
| 0x01 | u8 To3D |
Microphone echo cancel
| Offset | Description |
|---|---|
| 0x00 | s8 params[8] |
ABL extra
| Offset | Description |
|---|---|
| 0x00 | u8 MaxInertia |
| 0x01 | u8 pad |
| 0x02 | u16 PWM_CNT_EX |
| 0x04 | u32 Histogram1 |
| 0x08 | u32 Histogram2 |
| 0x0C | u32 adjust[0x40] |
CStick
| Offset | Description |
|---|---|
| 0x00 | u8 ThinningCountX(???) |
| 0x01 | u8 ThinningCountY(???) |
| 0x02 | u16 reserved[3] |
QTM
| Offset | Description |
|---|---|
| 0x00 | float DivisorAtZero (???) |
| 0x04 | float TranslationX |
| 0x08 | float TranslationY |
| 0x0C | float RotationZ |
| 0x10 | float HorizontalAngle |
| 0x14 | float OptimalDistance |
Reading
If 0x1FF81006 is 3 or 4 or 7 or 8 or 9 then the callibration data is read from the EEPROM using the i2c:EEP service command 0x001000C0, using offset 0x000 for HWCAL0, and offset 0x800 for HWCAL1.
Otherwise attempt is made to read CTRNAND:/ro/sys/HWCAL(0|1).dat instead.