The-Three-Generations-of-GPU-Texture-Distribution

This content was automatically converted from the project's wiki Markdown to HTML. See the Basis Universal GitHub wiki for the latest content.

At a Glance

Gen	Ship	Process at Load	Effective Shipped bpp Relative to BC7
1	Raw GPU blocks	None	~8 bpp
2	Distorted GPU blocks	LZ Decompress or CRN transcode	~4–7 bpp
3	Portable latent	Codec Transcode + SSD Cache	~1.5–3 bpp (0.35–2 at larger block sizes)

Gen 1 — Ship GPU Blocks

Source → ASTC / BCn → Ship → GPU

1998-1999 (S3TC/DXT1): Dark Ages: Distribution format = Execution format.
Disk size ≈ GPU memory size.
Optional: LZ, but low savings: ~10%.
Format or vendor specific tools, each with their own quirks and bugs.
Large downloads, no install-time processing.
ASTC, due to its complexity, has been stuck here until Basis Universal v2.0 released (XUASTC LDR).

Assumption: GPU formats are opaque blocks, most assume it's "impossible" to compress further

Gen 2 — RDO + LZ or Clusterization

Source → GPU blocks
       → RDO / clustering
       → LZ compressor / CRN transcode
       → Decompress → GPU blocks

Dec. 2011: crunch (now here, also see SIGGRAPH reference) library introduced both .CRN (VQ clustering+entropy coding of the BC1-5 latent) and RDO+LZ. Later: bc7enc_rdo.
crunch library deployed at planetary scale.
Opaque block stream made more compressible by crudely distorting content (unstructured distortion: GPU block structure reuse).
GPU block latent compression introduced (CRN for specifically BC1-5).
General-purpose entropy coding (RDO+LZ) or custom entropy coding (CRN).
Still tied to GPU block layout, not portable.
No, or minimal, psychovisual optimization.
Huge downloads: 100's of gigabytes and growing (unsustainable).
Increasingly misaligned with hardware trends: Gen 4/5 SSD throughput now exceeds LZ decode speed, and network bandwidth is the dominant bottleneck.

Assumption: Keep GPU blocks, distort them to increase redundancy; most still assuming no more compression is possible

Gen 3 — Compiled Texture Model

Source → Portable latent
       → Weight Grid DCT Transform + Quantize
       → Arithmetic coding (CABAC-style)
       → Ship
       → Install-time / Streaming transcode / Deblocking
           (parallel on modern 8+ core CPUs)
       → Cache GPU blocks (NVMe / Gen5 SSD: ~8–14 GB/s)
       → GPU

2019: Basis Universal ETC1S, UASTC LDR/HDR, formalizes portable texture intermediates. Standardized by Khronos.
2026: Basis Universal v2.0 introduces XUASTC LDR codec.
Distribution format ≠ Execution format. Signal → Texture IR → GPU blocks.
GPU blocks treated as derived artifacts.
Transform-domain parameter modeling (structured/perceptually lossless distortion via DCT). Psychovisual optimization.
Clean rate scaling (block size + quantization).
CPU or GPU deblocking introduced (GPU texture compression-aware shading)
Download time reduced to 1/3 to 1/2 size vs. Gen 2 (or less).
Install-time transcode to GPU blocks cached on SSD (~1 gigatexel/sec. with 8 threads, or much faster with compute/hardware decoders).
Alternatives: Streaming transcode with caching to SSD. Compute shader transcoders. Hardware transcoders.
Aligned with hardware trends: ultra-fast SSDs and CPUs with dozens of cores that otherwise sit idle during download/install.

Assumption: Compress and distribute textures as real signals, not opaque distorted blocks. Apply modern psychovisual methods. GPU formats are execution formats, not distribution formats. Streaming or install-time transcode to modern SSDs.

Core Insight

Textures should be treated like compiled shaders:

Ship a compact intermediate. Compile locally. Cache the execution format.

Paradigm Transition

Era	Mental Model	Technique	What's Being Compressed
Gen 1	GPU blocks are final	None (optional container-level LZ)	Opaque, high-entropy block bytes
Gen 2	GPU blocks are noisy data	RDO + entropy coding, clusterization	Distorted block byte streams
Gen 3	GPU textures are signals	Transform coding + psychovisual modeling	Structured latent parameter fields

In early 2026 Basis Universal is a Gen 3 system living in a mostly Gen 1/2 world.

When Gen 3 Matters

Texture Payload	Gen 2 Download	Gen 3 Download	Impact
2 GB	~1.5 GB	~500 MB	Modest
20 GB	~12 GB	~4 GB	Significant
100 GB	~60 GB	~20 GB	Critical
Planetary scale	Petabytes/yr	~1/3 PB/yr	Infrastructure-level

The larger the texture payload, the more Gen 3 matters.

For small projects, any generation is viable. At large scale, Gen 2 becomes increasingly difficult to sustain.

What We Expect

XUASTC LDR covers ASTC (and BC7 with latent to latent transcoding or bc7f), and we doubt many (if any) developers will consider writing their own system for all 14 block sizes. The sheer complexity of ASTC itself is a big barrier.

However, latent domain BC7 with DCT is likely inevitable now. (Who will do "XBC7" first?) BC7 has weight grids, too. It's essentially a stripped-down ASTC. Binomial could do it, but we already have XUASTC LDR, and ASTC is now the dominant format with billions of shipped hardware decoders.