GPU/Pitfalls - Revision history

Wyatt: Document L4-format texture crash

2025-11-01T21:39:06Z

Document L4-format texture crash

← Older revision		Revision as of 00:39, 2 November 2025
Line 52:		Line 52:
	\|(unused)		\|(unused)
	\|}		\|}

			=== TEXUNIT1 Corruption and Instability ===

			If an L4-formatted texture (or likely any 4-bit texture format) located within VRAM is bound to TEXUNIT1, the PICA can become unstable, usually freezing after a seemingly random amount of time. If that texture is 8x16 or larger, the GPU will also display corrupt visuals until the console is rebooted; this will even persist in other software. Unbinding the texture makes the console stable again, but doesn't fix the visuals, though the severity of the corruption is slightly reduced because the exact results also depend on the texture data. Notably, this does not seem to occur with other TEXUNITs or with textures located in FCRAM. This could use more investigation.

			Writer's Note: I struggled for days to reproduce this. I happened across it in a real application, and swapping between L4 and L8 affected the issue with 100% consistency. I then went to make this write-up months later and made a test program to reproduce the bug, which worked as expected. Like a doofus I rewrote the program a bit to test a bit more thoroughly, somehow caused the bug to stop occurring, and haven't been able to reproduce it since. Needs further investigation.

	== Shaders ==		== Shaders ==

JayFoxRox at 19:52, 14 March 2016

2016-03-14T18:52:38Z

← Older revision		Revision as of 21:52, 14 March 2016
Line 20:		Line 20:
	=== Output mapping in GPUREG_SH_OUTMAP_MASK ===		=== Output mapping in GPUREG_SH_OUTMAP_MASK ===

	The output masking in [[GPU/Internal_Registers#GPUREG_SH_OUTMAP_MASK\|GPUREG_SH_OUTMAP_MASK]] influences how the registers starting at [[GPU/Internal_Registers#GPUREG_SH_OUTMAP_Oi\|GPUREG_SH_OUTMAP_Oi]] ~~will~~ map to outputs in the shader.		The output masking in [[GPU/Internal_Registers#GPUREG_SH_OUTMAP_MASK\|GPUREG_SH_OUTMAP_MASK]] influences how the registers starting at [[GPU/Internal_Registers#GPUREG_SH_OUTMAP_Oi\|GPUREG_SH_OUTMAP_Oi]] map to outputs in the shader.

	If an output is disabled in [[GPU/Internal_Registers#GPUREG_SH_OUTMAP_MASK\|GPUREG_SH_OUTMAP_MASK]] it means that no slot in the [[GPU/Internal_Registers#GPUREG_SH_OUTMAP_Oi\|GPUREG_SH_OUTMAP_Oi]] registers is consumed.		If an output is disabled in [[GPU/Internal_Registers#GPUREG_SH_OUTMAP_MASK\|GPUREG_SH_OUTMAP_MASK]] it means that no slot in the [[GPU/Internal_Registers#GPUREG_SH_OUTMAP_Oi\|GPUREG_SH_OUTMAP_Oi]] registers is consumed.

JayFoxRox: Wording to make things more clear

2016-03-14T18:51:27Z

Wording to make things more clear

← Older revision		Revision as of 21:51, 14 March 2016
Line 7:		Line 7:
	=== Vertex attribute alignment ===		=== Vertex attribute alignment ===

	Vertex components which are defined through [[GPU/Internal_Registers#GPUREG_ATTRIBBUFFERi_CONFIG1\|GPUREG_ATTRIBBUFFERi_CONFIG1]] will be aligned.		Vertex components which are defined through [[GPU/Internal_Registers#GPUREG_ATTRIBBUFFERi_CONFIG1\|GPUREG_ATTRIBBUFFERi_CONFIG1]] will be accessed aligned by the GPU.
	* Vertex attributes will be aligned to their component element size.		* Vertex attributes will be aligned to their component element size.
	* Padding attributes (Component type > 11) will always aligned to 4 byte offets into the buffer.		* Padding attributes (Component type > 11) will always aligned to 4 byte offets into the buffer.
	* The stride should ~~remain~~ passed unaligned.		* The stride which is passed to the GPU should be passed unaligned.

	=== Vertex stride in GPUREG_ATTRIBBUFFERi_CONFIG2 ===		=== Vertex stride in GPUREG_ATTRIBBUFFERi_CONFIG2 ===

JayFoxRox: Some extra notes about alignment

2016-03-14T15:16:52Z

Some extra notes about alignment

← Older revision		Revision as of 18:16, 14 March 2016
Line 10:		Line 10:
	* Vertex attributes will be aligned to their component element size.		* Vertex attributes will be aligned to their component element size.
	* Padding attributes (Component type > 11) will always aligned to 4 byte offets into the buffer.		* Padding attributes (Component type > 11) will always aligned to 4 byte offets into the buffer.
			* The stride should remain passed unaligned.

	=== Vertex stride in GPUREG_ATTRIBBUFFERi_CONFIG2 ===		=== Vertex stride in GPUREG_ATTRIBBUFFERi_CONFIG2 ===
Line 54:		Line 55:
	== Shaders ==		== Shaders ==

	=== ~~Write output~~ components exactly once ===		=== Configued Output components must be written exactly once ===

	Each configured output component has to be written exactly once or the PICA freezes.		Each configured output component has to be written exactly once or the PICA freezes.

JayFoxRox: style

2016-03-14T10:35:30Z

style

← Older revision		Revision as of 13:35, 14 March 2016
Line 19:		Line 19:
	=== Output mapping in GPUREG_SH_OUTMAP_MASK ===		=== Output mapping in GPUREG_SH_OUTMAP_MASK ===

	The output ~~mapping~~ in [[GPU/Internal_Registers#GPUREG_SH_OUTMAP_MASK\|GPUREG_SH_OUTMAP_MASK]] ~~configures~~ how the registers starting at [[GPU/Internal_Registers#GPUREG_SH_OUTMAP_Oi\|GPUREG_SH_OUTMAP_Oi]] will map ~~the~~ outputs in the shader.		The output masking in [[GPU/Internal_Registers#GPUREG_SH_OUTMAP_MASK\|GPUREG_SH_OUTMAP_MASK]] influences how the registers starting at [[GPU/Internal_Registers#GPUREG_SH_OUTMAP_Oi\|GPUREG_SH_OUTMAP_Oi]] will map to outputs in the shader.

	If an output is disabled in [[GPU/Internal_Registers#GPUREG_SH_OUTMAP_MASK\|GPUREG_SH_OUTMAP_MASK]] it means that no slot in the [[GPU/Internal_Registers#GPUREG_SH_OUTMAP_Oi\|GPUREG_SH_OUTMAP_Oi]] registers is consumed.		If an output is disabled in [[GPU/Internal_Registers#GPUREG_SH_OUTMAP_MASK\|GPUREG_SH_OUTMAP_MASK]] it means that no slot in the [[GPU/Internal_Registers#GPUREG_SH_OUTMAP_Oi\|GPUREG_SH_OUTMAP_Oi]] registers is consumed.
	[[GPU/Internal_Registers#GPUREG_SH_OUTMAP_TOTAL\|GPUREG_SH_OUTMAP_TOTAL]] configures the number of used consecutive slots in the outmap.		[[GPU/Internal_Registers#GPUREG_SH_OUTMAP_TOTAL\|GPUREG_SH_OUTMAP_TOTAL]] configures the number of used consecutive slots in the outmap.

JayFoxRox: Mostly based on findings by yuriks

2016-03-14T10:29:36Z

Mostly based on findings by yuriks

New page

This page collects some oddities and pitfalls of the PICA GPU which is used in the 3DS.

[[Category:GPU]]

== Internal Registers ==

=== Vertex attribute alignment ===

Vertex components which are defined through [[GPU/Internal_Registers#GPUREG_ATTRIBBUFFERi_CONFIG1|GPUREG_ATTRIBBUFFERi_CONFIG1]] will be aligned.
* Vertex attributes will be aligned to their component element size.
* Padding attributes (Component type > 11) will always aligned to 4 byte offets into the buffer.

=== Vertex stride in GPUREG_ATTRIBBUFFERi_CONFIG2 ===

The vertex stride set in [[GPU/Internal_Registers#GPUREG_ATTRIBBUFFERi_CONFIG2|GPUREG_ATTRIBBUFFERi_CONFIG2]] must match the actual size of the vertex contained in the buffer or the PICA will freeze or it won't draw anything.

If you want to use a different stride you have to pad the data accordingly with padding attributes.

=== Output mapping in GPUREG_SH_OUTMAP_MASK ===

The output mapping in [[GPU/Internal_Registers#GPUREG_SH_OUTMAP_MASK|GPUREG_SH_OUTMAP_MASK]] configures how the registers starting at [[GPU/Internal_Registers#GPUREG_SH_OUTMAP_Oi|GPUREG_SH_OUTMAP_Oi]] will map the outputs in the shader.
If an output is disabled in [[GPU/Internal_Registers#GPUREG_SH_OUTMAP_MASK|GPUREG_SH_OUTMAP_MASK]] it means that no slot in the [[GPU/Internal_Registers#GPUREG_SH_OUTMAP_Oi|GPUREG_SH_OUTMAP_Oi]] registers is consumed.
[[GPU/Internal_Registers#GPUREG_SH_OUTMAP_TOTAL|GPUREG_SH_OUTMAP_TOTAL]] configures the number of used consecutive slots in the outmap.

Example:

{| class="wikitable" border="1"
! Register
! Value
! Meaning
|-
|GPUREG_SH_OUTMAP_TOTAL
|0x00000002
|2 outputs enabled
|-
|GPUREG_SH_OUTMAP_MASK
|0x00000011
|o0 enabled, o4 enabled
|-
|GPUREG_SH_OUTMAP_O0
|0x03020100
|o0 = pos.xyzw
|-
|GPUREG_SH_OUTMAP_O1
|0x0B0A0908
|o4 = color.rgba
|-
|GPUREG_SH_OUTMAP_O2
|...
|(unused)
|}

== Shaders ==

=== Write output components exactly once ===

Each configured output component has to be written exactly once or the PICA freezes.

=== MOVA instructions can't be adjacent ===

Having 2 consecutive MOVA instructions will freeze the PICA. This can be relaxed by placing a NOP between 2 MOVAs or by rearranging the code.