Models out of files

A model file is more than its geometry. A character has clips and a skeleton, a product comes in several finishes, and a lamp carries its own light. The renderer reads all of that as it loads the file and reports it back, so you can ask for things by the names the file gives them rather than by numbers you typed in.

What a file holds

OrblitView.onAssetInfo hands you an OrblitAssetInfo whenever the scene builds something new out of a file. It describes the model as the renderer has it, so an FBX that became glTF on the way in is described as the glTF.

• clips: each clip’s name and length in seconds.
• skins: joint names, which joint each hangs from, and where each one rests.
• variants and materials: names.
• lights and cameras: what the file carries. A camera gives its vertical field of view in degrees, or its view height if it’s orthographic.
• boundsMin and boundsMax: the box the geometry fills, in the file’s own units.
• unsupported: glTF extensions the renderer doesn’t draw. The parts that need them are drawn without them, and the scene notes say so too. Clear coat and sheen draw; anisotropy doesn’t, and turns up here.

It isn’t sent once per file. A view that arrives later hears about the file the next time an object is made of it, so keep what you’re given in a map keyed by info.path.

Clips

import 'package:flutter/widgets.dart';
import 'package:orblit_filament/orblit_filament.dart';
import 'package:vector_math/vector_math_64.dart';

final fox = OrblitResources.nameFor('fox.glb');
final models = <String, OrblitAssetInfo>{};

// Running, faded in from walking. fade goes from 0 (all walk) to 1 (all run).
OrblitObject foxAt(Matrix4 placement, double seconds, double fade) {
  final info = models[fox];
  final walk = info?.clipNamed('Walk');
  final run = info?.clipNamed('Run');
  return OrblitObject(
    key: 1,
    transform: placement,
    colour: Vector3.all(0.8),
    mesh: fox,
    animation: walk == null || run == null
        ? null
        : OrblitAnimation(
            clip: run,
            seconds: seconds,
            from: OrblitAnimation(clip: walk, seconds: seconds),
            fade: fade,
          ),
  );
}

Widget foxView(OrblitScene scene) => OrblitView(
  scene: scene,
  onAssetInfo: (info) => models[info.path] = info,
);

clip is a position in info.clips, and clipNamed finds it. A clip the file doesn’t have becomes a scene note, and nothing plays. seconds is where the clip is when the scene is sent, and speed is how many of its seconds pass per second after that. A speed of 0 holds it, which is how you scrub. loop decides whether it wraps or stops on its last frame.

The renderer samples the clip as each frame is drawn, so motion stays smooth between sends. A host with a held clock gets exactly seconds every frame: the running fox came out byte-identical across two runs of the app on macOS. A clip runs on for at most a quarter of a second after the last scene, then holds.

A fade is between two clips, not a chain: the from clip’s own from isn’t read. Set animation to null and the model goes back to the pose the file rests in.

Joints

OrblitObject.joints takes a list of OrblitJointPoses, each naming a skin, a joint and a transform relative to the joint’s parent. It replaces that joint’s local transform after any clip, so a hand-set joint always wins. info.jointNamed('Head') finds the skin and joint by name.

To drive a skin from an orblit_rig armature, bind them. Bones find joints by name:

import 'package:orblit_filament/orblit_filament.dart';
import 'package:orblit_rig/orblit_rig.dart';
import 'package:orblit_stage/orblit_stage.dart';
import 'package:vector_math/vector_math_64.dart';

List<OrblitJointPose> nodding(OrblitAssetInfo info, String bone, double angle) {
  final skin = info.skins.first;
  final armature = armatureOfSkin(skin);
  final binding = OrblitSkinBinding(armature, skin, index: 0);

  final pose = Pose(binding.armature);
  pose[bone].rotation = Quaternion.axisAngle(Vector3(1, 0, 0), angle);
  pose.evaluate();
  return binding.jointsFor(pose);
}

Build the binding once and keep it. That’s only inside one function here to keep the example short. armatureOfSkin makes one bone per joint. An unnamed joint becomes joint 3, and a repeated name gets .001, the way Blender does it. boneNamesOfSkin lists the names. An armature you built yourself works too, as long as the names match and it’s in the model’s own space.

jointsFor returns every joint, including those at rest, because a joint left out would stay wherever it was last put. It doesn’t evaluate the pose for you. Evaluate once per frame, since inverse kinematics writes back into the pose and a second pass can move it again.

A skinned model’s bounding box follows the pose, so a character that’s walked away from where it was bound isn’t culled. In the check, a skin moved 10 m still drew 3590 pixels, and none with that fitting switched off.

Variants

variant is a position in info.variants, so info.variants.indexOf('beach') picks a finish by name. Null keeps the file’s own materials. An Orblit material set on the object still overrides every variant.

A file’s lights

The renderer takes a file’s lights out of the model rather than drawing them itself. As the file made them, they’d cast no shadows and count against nothing, and a directional one would fight your sun for the one slot there is. Put them in the scene as ordinary lights instead:

import 'package:orblit_filament/orblit_filament.dart';
import 'package:vector_math/vector_math_64.dart';

OrblitScene lampScene(OrblitAssetInfo info, OrblitCamera camera) {
  final placement = Matrix4.translationValues(2, 0, 0);
  return OrblitScene(
    camera: camera,
    objects: [
      OrblitObject(
        key: 1,
        transform: placement,
        colour: Vector3.all(0.8),
        mesh: info.path,
      ),
    ],
    lights: [...info.lightsFor(placement, keyOf: (i) => 100 + i)],
  );
}

Give lightsFor the object’s own placement. keyOf gives each light its scene key, which has to stay clear of every other key in the scene. The lights arrive as the file stated them, in lumens or lux. The bulb in Khronos’s punctual-lights lamp is about 20 lumens, like a real one, and at a daylight exposure that’s black. The example has a night camera for it, so don’t take a dark lamp for a bug.

FBX and OBJ

A .fbx or .obj is converted to GLB by ufbx 0.23.0 when it’s first named, then kept for the life of the process. The conversion runs off the drawing thread natively, and in place in a browser. An OBJ’s .mtl and pictures are looked for beside it, so if you’re handing over bytes, provide them under names beside the OBJ’s own. The GLB is Y-up, right-handed and in metres, and an OBJ is taken to be in metres already. Files over 512 MB are refused.

What the file had that glTF can’t hold is listed in the scene notes. That includes cameras, lights, curves and NURBS, constraints and vertex caches, as well as separate metalness, roughness, glossiness, specular and opacity textures (the factors are kept), bump maps, UV transforms, procedural textures, all but the first layer of a layered texture, and pictures that aren’t PNG, JPEG or KTX2.

To convert ahead of time, native/headless/build.sh in orblit_filament builds orblit_import. That script is written for a Mac.

orblit_import "Samba Dancing.fbx" dancer.glb

It runs the same function as load time, so the bytes match. It prints what it wrote, how long it took, the model’s size in metres (a character 180 m tall shows up here first), and then the losses. It doesn’t read textures: an embedded one goes into the GLB, and a referenced one keeps a path relative to the input, so write the .glb next to the original. Twenty of its outputs pass Khronos’s glTF-Validator with no errors, the same input gives the same bytes, and 3,000 mutated inputs ran clean under AddressSanitizer and UndefinedBehaviorSanitizer.

Where it’s been checked, and what’s missing

The Imported models example has the fox, a walking man, a shoe in three finishes, the lamp, clear coat, a sheen chair, the barn lamp it can’t fully draw, an FBX dancer and an OBJ. The files aren’t committed: tool/fetch_import_samples.sh fetches about 25 MB. With the clock held, frames were checked on macOS, in Chrome on WebGL 2, in the iOS simulator and in the Android emulator, and the dancer strikes the same pose on all four. No frames have been checked on phones, Linux, Windows, Safari or Firefox.

• Scene files can’t play clips or pick variants. No component does it, and orblit_stage doesn’t resolve clip names.
• The editor won’t open .fbx or .obj. It labels them as unopenable.
• Nothing exports FBX. That’s a decision rather than a gap: FBX comes in, and doesn’t go out.
• Switching clips can leave channels behind. If the new clip doesn’t animate a node that the old one did, the node stays where the old clip left it. That’s how gltfio behaves.
• Morph weights a clip animated aren’t reset when the object stops being posed.