Hardware calibration

Revision as of 02:40, 24 September 2017 by MarcusD (talk | contribs)

File format

The file consists out of a 0x200 big header (though the actual header is only 0x30 bytes, the rest is zerofilled), plus data whose size is indicated in the header.

Header

Offset Size Description
0x0 0x4 Magic "CCAL"
0x4 0x4 Version
0x8 0x4 Data size, always 0x7D0
0xC 0x2? ???
0xE 0x2 Bitmask of successful Aging tests
0x10 0x20 On devunits and if 0x1FF81006 is 3 or 4 or 7 or 8 or 9 then SHA256,

otherwise HMACSHA256 signature of the data section

0x30 0x1D0 Zerofilled

Aging masks

Bit Description
0 FCRAM
1 LCD flicker (always successful)
2 Camera
3 Touch panel (always successful)
4 Circle pad (analog stick)
5 Codec
6 Gyroscope
7 RTC
8 Accelerometer
9 Surround
A Power saving mode (ABL)
B 3D screen (ULCD)
C Backlight PWM
D Analog stick A (???)
E Camera extensions
F Power saving mode (ABL) in legacy (DSi/GBA) mode

Data blocks

Offset Size Description
0x14 0x08 Circle pad
0x30 0x8C Camera position
0x218 0x06 Programmable Infrared Transmitter (PIT)
0x220 0x214 3D filters
0x4E0 0x18 Circle pad extra
0x540 0x0A MCU
0x550 0x04 3D screen (ULCD) delay
0x560 0x0A Microphone echo cancellation
0x570 0x10E Power saving mode (ABL) extra
0x680 0x0A CStick (Right stick)
0x690 0x1A Q Tracking Module (QTM)

Data block formats

Circle pad

Offset Description
0x00 s16 CenterX
0x02 s16 CenterY

Camera position

Offset Description
0x00 u32 flags
0x04 float scale
0x08 float RotationZ
0x0C float TranslationX
0x10 float TranslationY
0x14 float RotationX
0x18 float RotationY
0x1C float ViewAngleRight
0x20 float ViewAngleLeft
0x24 float ChartDistance(???)
0x28 float CameraDistance
0x2C s16 ImageWidth
0x2E s16 ImageHeight
0x30 u8 reserved[0x10]
0x40 u8 ???[0x40]
0x80 s16 aeBaseTarget(???)
0x82 s16 kRL
0x84 s16 kGL
0x86 s16 kBL
0x88 s16 ccmPosition

3D filters

Offset Description
0x00 u16 SpecialFilter[0x100]
0x200 u32 IIRSurroundFilter[5]

Circle pad extra

Offset Description
0x00 float ScaleX
0x04 float ScaleY
0x08 s16 MaxX
0x0A s16 MinX
0x0C s16 MaxY
0x0E s16 MinY
0x10 s16 type
0x12 u8 unknown_padding[6]

ULCD delay

Offset Description
0x00 u8 To2D
0x01 u8 To3D

Microphone echo cancel

Offset Description
0x00 s8 params[8]

ABL

Offset Description
0x00 u8 MaxInertia
0x01 u8 pad
0x02 u16 PWM_CNT_EX
0x04 u32 Histogram1
0x08 u32 Histogram2
0x0C u32 adjust[0x40]

Reading

If 0x1FF81006 is 3 or 4 or 7 or 8 or 9 then the callibration data is read from the EEPROM using the i2c:EEP service command 0x001000C0, using offset 0x000 for HWCAL0, and offset 0x800 for HWCAL1. Otherwise attempt is made to read CTRNAND:/ro/sys/HWCAL(0|1).dat instead.