Textures that don't stall

A texture has to be decoded, have its mipmaps made, and then be uploaded. Do all of that on the drawing thread for a big model and the app freezes. The Bistro exterior has 405 textures, and before this work it froze for 1.1–4.4 s while they arrived.

Every texture now goes through one queue, whether it belongs to a material, a sprite or a glTF model. Decoding happens off the drawing thread, on every core but two, and uploads are spread across frames with the smallest mip levels going first. A texture on its way shows a blurrier copy of itself, or transparent black, but never garbage. A model is drawn once all its textures have memory.

That’s automatic. What you choose is what format the textures arrive in.

Cook them

PNG, JPEG and Basis all have to be turned into something a GPU can sample every time they load. A cooked set skips that work:

• wall.ktx2: UASTC, which is transcoded as it loads, for any device
• wall.astc.ktx2: ASTC
• wall.bc.ktx2: BC7, BC5 or BC4
• wall.etc2.ktx2: ETC2 or EAC

Each file has a full mip chain, and each level is compressed with zstd. Name wall.ktx2 and the renderer loads the first sibling holding a format this device really samples. It checks the format in the file’s header, not its name, and falls back to wall.ktx2 itself.

./tool/cook_textures.sh models/lamp/textures --gltf models/lamp/lamp.gltf

That writes models/lamp/textures.cooked/, or wherever --into says. It builds the cooker the first time. The other flags are --targets (any of astc, bc, etc2 and basis, all four by default), --threads, --max-size and --lossless.

A file can’t tell you whether it’s a colour or a normal map, so the cook needs to be told:

• With --gltf, the scene decides. normalTexture is a normal map, renormalised at every level with all three channels kept. The base colour of a MASK material is a cut-out, with its coverage kept at every level so foliage doesn’t thin out with distance. Base colour, emissive, sheen and specular colour are sRGB, and everything else is linear.
• Without it, file names decide. *Normal* is a normal map, and *BaseColor*, *Albedo*, *Diffuse* and *Emissive* are sRGB. There are no cut-outs, since nothing in a name says where the alpha test is.

--lossless is for sprites. It writes RGBA8 in wall.ktx2 alone, with no mips and no siblings, and the pixels come out exactly as they went in.

A cook can be interrupted and resumed. Each file goes in under another name and is renamed into place, and a file that’s newer than its source and was cooked with the same flags is skipped. The same input cooks to the same bytes, whatever the thread count.

The script never makes single-channel textures or two-channel normal maps, because it can’t know they’re safe: glTF often packs occlusion and metal-roughness into one image, and no Orblit material rebuilds a normal’s third channel. orblit_texture_cook has --single-channel and --two-channel-normals for when you know better.

A glTF that already names .ktx2 files, like the Bistro’s, can be pointed at the cooked folder as it is. The renderer only looks for siblings of a name ending in .ktx2, so a scene that names PNGs gets PNGs.

What it’s worth

Loading the Bistro exterior on an M4 Pro, three runs each:

	Arrives in	Longest frame
Before, Basis	2.0–5.1 s	1.1–4.4 s
Now, Basis	2.1–2.5 s	94–183 ms
Now, cooked	0.83–1.03 s	78–92 ms

That last longest frame is the model’s first draw, not texture work. In the gallery, the cooked Bistro arrived in 1.35–1.65 s, against 3.3–3.7 s for Basis before. Basis on a busy machine can now be slower than it was, at 3.8–7.8 s, because its work is spread out rather than taken in one frozen frame. Cook them.

Budgets

How large textures load and how much uploads per frame both start from the device’s tier:

Tier	Largest side	Upload per frame
Low	1024	4 MB
Medium	2048	16 MB
High	4096	32 MB

The largest side is also capped at what the device supports. Set either yourself on the pipeline:

final pipeline = OrblitPipeline(
  textures: OrblitTextureLimits(maxSize: 1024, uploadKilobytes: 8192),
);
return pipeline;

A texture larger than maxSize drops its largest mip levels and never uploads them. A picture without mips is halved as it decodes. The cooked Bistro takes 1195 MB at full size and 299 MB at 1024. Textures already loaded keep their size.

Left null, uploadKilobytes isn’t fixed. It starts at the tier’s figure and follows what frames actually cost while textures arrive. A number you set is held exactly. A frame always uploads at least one level, however large, or a level bigger than the budget would never go.

The tiers are starting points: no phone has measured them yet.

In a browser

A browser has no files to look beside, so ask the device which ones are worth fetching, and provide the first that exists:

import 'dart:typed_data';

import 'package:orblit_filament/orblit_filament.dart';

Future<OrblitMaterial> cookedWall(
  OrblitDeviceProfile device,
  Future<Uint8List?> Function(String path) fetch,
) async {
  for (final candidate in device.textureCandidates('textures/wall.ktx2')) {
    final bytes = await fetch(candidate);
    if (bytes == null) continue;
    await OrblitResources.provide(OrblitResources.nameFor(candidate), bytes);
    break;
  }
  final name = OrblitResources.nameFor('textures/wall.ktx2');
  return OrblitMaterial(key: 1, baseColourMap: OrblitTexture(name));
}

textureCandidates lists the siblings in the renderer’s order, with the universal file last. Splats shows how to get the profile. Decoding in a browser runs on Web Workers, and no page task went over 50 ms while twelve 2048² textures loaded, against 118–206 ms before.

Environments load the same way. OrblitEnvironment.fromImage takes an .hdr or .exr and filters it while the scene runs, which Lighting covers.

Limits

• Apple devices take no sRGB ASTC. Filament’s Metal backend doesn’t sample it, so colour comes from the BC file on a Mac and the ETC2 file on iOS.
• ETC2 is ETC1 quality for now. The cooker makes its ETC2 files from UASTC, which only reaches the ETC1 subset: 23.6 dB on the Bistro’s cobblestone normal map. iOS uses it for colour, and so does any device that has neither ASTC nor BC.
• The Bistro’s own normal maps store two channels. Nothing rebuilds the third, so it draws with bent normals, cooked or not.
• A Basis file with no mips can’t show a smaller level while it waits.
• Seen on macOS, Chrome with WebGL 2, the iOS simulator and the Android emulator. Not run on Linux, Windows, Android on OpenGL ES, phones, Safari or Firefox.