Difference between revisions of "Hardware calibration"
(Clean up some of the non-descriptive and/or wrong variable names) |
(added MCU slider cfgblk ID) |
||
| Line 196: | Line 196: | ||
| 0x540 | | 0x540 | ||
| 0x08 | | 0x08 | ||
| − | | | + | | 0x00120000 |
| [[#MCU|MCU]] | | [[#MCU|MCU]] | ||
|- | |- | ||
Revision as of 03:52, 17 April 2021
File format
The file consists out of a 0x200 big header (though the actual header is only 0x30 bytes, the rest is zerofilled), plus data whose size is indicated in the header.
Header
| Offset | Size | Description |
|---|---|---|
| 0x0 | 0x4 | Magic "CCAL" |
| 0x4 | 0x4 | Version |
| 0x8 | 0x4 | Data size, always 0x7D0 |
| 0xC | 0x1 | Model version (?) |
| 0xD | 0x1 | CAL revision (incremented each time the CAL file is updated) |
| 0xE | 0x2 | Bitmask of successful Aging tests |
| 0x10 | 0x20 | Signature of the data section.
HMACSHA256 is used always except in the below cases where SHA256 is used: - devunits - PARTNER-DEBUGGER - PARTNER-CAPTURE - the SNAKE counterparts of the above - SNAKE-IS-DEBUGGER |
| 0x30 | 0x1D0 | Zerofilled, padding for the 512byte block size |
Aging masks
| Bit | Description |
|---|---|
| 0 | FCRAM |
| 1 | LCD "flicker"/contrast (always successful) |
| 2 | Camera |
| 3 | Touch panel (always successful) |
| 4 | Circle pad (analog stick) |
| 5 | Codec |
| 6 | Gyroscope |
| 7 | RTC |
| 8 | Accelerometer |
| 9 | Surround |
| A | Adaptive BackLight (ABL) |
| B | 3D screen (ULCD) |
| C | Backlight PWM |
| D | Analog stick A (???) |
| E | Camera extensions |
| F | Adaptive BackLight (ABL) in legacy (DSi/GBA) mode |
Data blocks
| Offset | Size | ConfigInfoBlk | Description |
|---|---|---|---|
| 0x00 | 0x10 | 00040000 | Touch panel |
| 0x14 | 0x08 | ??? | Circle pad |
| 0x20 | 2* | 00050000 | Display panel contrast |
| 0x24 | 1* | ??? | RTC |
| 0x28 | 1* | ??? | DSPRAM related |
| 0x30 | 0x8A | ??? | Camera position |
| 0xBC | 0x12 | ??? | Gyroscope |
| 0xD0 | 0xC | ??? | Accelerometer |
| 0xE0 | 0x134 | ??? | Codec |
| 0x218 | 0x06 | ??? | Programmable Infrared Transmitter (PIT) |
| 0x220 | 0x214 | ??? | 3D filters |
| 0x440 | 0x20 | ??? | Adaptive BackLight / Power saving mode |
| 0x470 | 0x20 | ??? | ??? |
| 0x4A0 | 0x38 | 0x00050002 | Backlight PWM |
| 0x4E0 | 0x18 | ??? | Circle pad extra |
| 0x500 | 0xC | ??? | ??? |
| 0x510 | 0x20 | ??? | ??? |
| 0x540 | 0x08 | 0x00120000 | MCU |
| 0x550 | 0x02 | ??? | 3D screen (ULCD) delay |
| 0x560 | 0x08 | ??? | Microphone echo cancellation |
| 0x570 | 0x10C | ??? | Power saving mode (ABL) extra |
| 0x680 | 0x08 | ??? | CStick (Right stick) |
| 0x690 | 0x18 | ??? | Quad Tracking Module (QTM) |
Data block formats
Touch
Used for mapping touch ADC values to display pixel co-ordinates.
[4096, 4096] --> [320, 240]
[RawX, RawY] --> [PointX, PointY]
Usually [PointX0, PointY0] is placed around 25% from the top-left corner, and the same for [PointX1, PointY1] except 25% from the bottom-right corner. This offsetting is needed because the touch film starts to distort outside of that rectangle, which would skew the touch results near the center of the screen.
| Offset | Declaration |
|---|---|
| 0x00 | s16 RawX0 |
| 0x02 | s16 RawY0 |
| 0x04 | s16 PointX0 |
| 0x06 | s16 PointY0 |
| 0x08 | s16 RawX1 |
| 0x0A | s16 RawY1 |
| 0x0C | s16 PointX1 |
| 0x0E | s16 PointY1 |
Circle pad
Contains the centering position of the circle pad. For other circle pad settings, see circle pad extra.
| Offset | Declaration | Description |
|---|---|---|
| 0x00 | s16 CenterX | Raw analog values corresponding to zero input position |
| 0x02 | s16 CenterY |
Screen flicker
These values are written to MCU register 0x03 and 0x04 respectively. They both set the display contrast voltage.
| Offset | Declaration | Description |
|---|---|---|
| 0 | u8 FlickerTop | Contrast voltage |
| 1 | u8 FlickerBottom | |
| 2* | Inline checksum | Checksum low byte, NOT THIS[0] |
| 3* | Checksum high byte, THIS[1] |
RTC
| Offset | Declaration | Description |
|---|---|---|
| 0 | u8 CompensationValue | (???) |
| 1* | Checksum byte, NOT THIS[0] |
Camera position
| Offset | Declaration |
|---|---|
| 0x00 | u32 flags |
| 0x04 | float scale |
| 0x08 | float RotationZ |
| 0x0C | float TranslationX |
| 0x10 | float TranslationY |
| 0x14 | float RotationX |
| 0x18 | float RotationY |
| 0x1C | float ViewAngleRight |
| 0x20 | float ViewAngleLeft |
| 0x24 | float ChartDistance(???) |
| 0x28 | float CameraDistance |
| 0x2C | s16 ImageWidth |
| 0x2E | s16 ImageHeight |
| 0x30 | u8 reserved[0x10] |
| 0x40 | u8 ???[0x40] |
| 0x80 | s16 aeBaseTarget(???) |
| 0x82 | s16 kRL |
| 0x84 | s16 kGL |
| 0x86 | s16 kBL |
| 0x88 | s16 ccmPosition |
Gyro
| Offset | Declaration |
|---|---|
| 0x00 | s16 ZeroX |
| 0x02 | s16 PlusX |
| 0x04 | s16 MinusX |
| 0x06 | s16 ZeroY |
| 0x08 | s16 PlusY |
| 0x0A | s16 MinusY |
| 0x0C | s16 ZeroZ |
| 0x0E | s16 PlusZ |
| 0x10 | s16 MinusZ |
Accel
| Offset | Declaration |
|---|---|
| 0x00 | s16 OffsetX |
| 0x02 | s16 ScaleX |
| 0x04 | s16 OffsetY |
| 0x06 | s16 ScaleY |
| 0x08 | s16 OffsetZ |
| 0x0A | s16 ScaleZ |
CDC
| Offset | Declaration | Description |
|---|---|---|
| 0x00 | u8 DriverGainHP | Headphone gain |
| 0x01 | u8 DriverGainSP | Speaker gain |
| 0x02 | u8 AnalogVolumeHP | |
| 0x03 | u8 AnalogVolumeSP | |
| 0x04 | s8 ShutterVolume[2] | |
| 0x06 | u8 MicrophoneBias | Capacitive microphone bias voltage |
| 0x07 | u8 QuickCharge | (???) |
| 0x08 | u8 PGA_GAIN | ??? (microphone gain) |
| 0x09 | u8 reserved[3] | |
| 0x0C | s16 FilterHP32[3*5] | Headphone filter for 32728.49Hz sampling rate |
| 0x2A | s16 FilterHP47[3*5] | Headphone filter for 47605Hz sampling rate |
| 0x48 | s16 FilterSP32[3*5] | Speaker filter for 32728.49Hz sampling rate |
| 0x66 | s16 FilterSP47[3*5] | Speaker filter for 47605Hz sampling rate |
| 0x84 | s16 FilterMic32[(1+2)+((1+4)*5)] | Microphone filter for 32728.49Hz sampling rate |
| 0xBC | s16 FilterMic47[(1+2)+((1+4)*5)] | Microphone filter for 47605Hz sampling rate |
| 0xF4 | s16 FilterFree[(1+2)+((1+4)*5)] | Unknown |
| 0x12C | u8 AnalogInterval | |
| 0x12D | u8 AnalogStabilize | |
| 0x12E | u8 AnalogPrecharge | |
| 0x12F | u8 AnalogSense | |
| 0x130 | u8 AnalogDebounce | |
| 0x131 | u8 Analog_XP_Pullup | |
| 0x132 | u8 YM_Driver | ??? (circle-pad or touch panel related?) |
| 0x133 | u8 reserved |
PIT
| Offset | Declaration |
|---|---|
| 0x00 | u16 VisibleFactor |
| 0x02 | u16 IRFactor |
3D filters
| Offset | Declaration |
|---|---|
| 0x00 | u16 SpecialFilter[0x100] |
| 0x200 | u32 IIRSurroundFilter[5] |
ABL
| Offset | Declaration |
|---|---|
| 0x00 | u32 DitherPattern |
| 0x04 | s16 StartX |
| 0x06 | s16 StartY |
| 0x08 | u16 SizeX |
| 0x0A | u16 SizeY |
| 0x0C | s16 GTH_Ratio |
| 0x0E | u8 DitherMode |
| 0x0F | u8 MinRS |
| 0x10 | u8 MaxRS |
| 0x11 | u8 MinGTH |
| 0x12 | u8 MinMax (???) |
| 0x13 | u8 ExMax (???) |
| 0x14 | u8 inertia |
| 0x15 | u8 LutListRS[9] |
| 0x1E | u8 reserved[2] |
BLPWM
| Offset | Declaration |
|---|---|
| 0x00 | float coefficient[3][3] |
| 0x24 | u8 NumLevels |
| 0x25 | u8 padding |
| 0x26 | u16 brightnesses[7]; |
| 0x34 | u16 BaseDivisor |
| 0x36 | u16 MinimumBrightnessHw |
Circle pad extra
| Offset | Declaration |
|---|---|
| 0x00 | float ScaleX |
| 0x04 | float ScaleY |
| 0x08 | s16 MaxX |
| 0x0A | s16 MinX |
| 0x0C | s16 MaxY |
| 0x0E | s16 MinY |
| 0x10 | s16 type |
| 0x12 | u8 unknown_padding[6] |
MCU
Somewhat misleading, these values are actually used for clamping the MCU's raw slider readings to comprehensible values.
| Offset | Declaration | Description |
|---|---|---|
| 0x00 | s16 SVR2_Min | Raw 3D volume slider values <= this map to 3D slider value 0.0 |
| 0x02 | s16 SVR2_Max | Raw 3D volume slider values >= this map to 3D slider value 1.0 |
| 0x04 | s16 VolumeSliderMin | Written to MCU reg 0x58. Volume slider values <= this map to volume value 0x00 |
| 0x06 | s16 VolumeSliderMax | Written to MCU reg 0x59. Volume slider values >= this map to volume value 0x3F |
ULCD delay
There is a delay between switching the parallax barrier, and adjusting the backlight. These delay values determine how many VBlank events to wait on before switching the backlight curves to the appropriate mode.
This is needed only to prevent epillepsy from analog jitter causing unwanted mode switches, and both values are usually always set to 1 or 2.
| Offset | Declaration |
|---|---|
| 0x00 | u8 To2D |
| 0x01 | u8 To3D |
Microphone echo cancel
| Offset | Declaration |
|---|---|
| 0x00 | s8 params[8] |
ABL extra
| Offset | Declaration |
|---|---|
| 0x00 | u8 MaxInertia |
| 0x01 | u8 pad |
| 0x02 | u16 PWM_CNT_EX |
| 0x04 | u32 Histogram1 |
| 0x08 | u32 Histogram2 |
| 0x0C | u32 adjust[0x40] |
CStick
| Offset | Declaration |
|---|---|
| 0x00 | u8 ThinningCountX(???) |
| 0x01 | u8 ThinningCountY(???) |
| 0x02 | u16 reserved[3] |
QTM
| Offset | Declaration |
|---|---|
| 0x00 | float DivisorAtZero (???) |
| 0x04 | float TranslationX |
| 0x08 | float TranslationY |
| 0x0C | float RotationZ |
| 0x10 | float HorizontalAngle |
| 0x14 | float OptimalDistance |
Reading
If 0x1FF81006 is 3 or 4 or 7 or 8 or 9 then the callibration data is read from the EEPROM using the i2c:EEP service command 0x001000C0, using offset 0x000 for HWCAL0, and offset 0x800 for HWCAL1.
Otherwise attempt is made to read CTRNAND:/ro/sys/HWCAL(0|1).dat instead.