README.md

Khronos glTF Sample Renderer

This is the official Khronos glTF 2.0 Sample Renderer using WebGL.

Try out the glTF Sample Viewer which uses this renderer in combination with a custom frontend.

Table of Contents

• Khronos glTF Sample Renderer
  • Table of Contents
  • Credits
  • Features
  • API
    • GltfView
    • GltfState
      • GraphController
      • AnimationTimer
      • PhysicsController
      • Dirty flags
        • AnimatableProperty dirty flags
        • Node transform dirty flags
        • Resetting dirty flags
    • ResourceLoader
  • Render Fidelity Tools
  • Development
    • Formatting
      • Visual Studio Code
    • Testing

Credits

Developed and refactored by UX3D. Supported by the Khronos Group and by Google for the glTF Draco mesh compression import. Formerly hosted together with the example frontend at the glTF Sample Viewer repository. Original code based on the concluded glTF-WebGL-PBR project. Previously supported by Facebook for animations, skinning and morphing. For KHR_interactivity, the behavior engine of the glTF-InteractivityGraph-AuthoringTool is used.

Features

• glTF 2.0
• KHR_accessor_float64
  • Animations
  • KHR_animation_pointer
  • Mesh Attributes not supported since WebGL2 only supports 32 bit
  • Skins not supported since WebGL2 only supports 32 bit
• KHR_animation_pointer
• KHR_draco_mesh_compression
• KHR_implicit_shapes
• KHR_interactivity
• KHR_lights_punctual
• KHR_materials_anisotropy
• KHR_materials_clearcoat
• KHR_materials_diffuse_transmission
• KHR_materials_dispersion
• KHR_materials_emissive_strength
• KHR_materials_ior
• KHR_materials_iridescence
• KHR_materials_pbrSpecularGlossiness
• KHR_materials_sheen
• KHR_materials_specular
• KHR_materials_transmission
• KHR_materials_unlit
• KHR_materials_variants
• KHR_materials_volume
• KHR_materials_volume_scatter
  • For dense volumes using KHR_materials_diffuse_transmission
  • For sparse volumes using KHR_materials_transmission
• KHR_mesh_quantization
• KHR_node_hoverability
• KHR_node_selectability
• KHR_node_visibility
• KHR_physics_rigid_bodies
  Supported Engines:
  • NVIDIA PhysX (limitations)
• KHR_texture_basisu
• KHR_texture_transform
• KHR_xmp_json_ld
• EXT_mesh_gpu_instancing
• EXT_texture_webp

API

glTF Sample Renderer is made to be integrated into web applications (see glTF Sample Viewer) or to be used for automated testing (see Render Fidelity Tools).

The API consists of several components that in combination allow flexible configuration of the glTF viewer.

More detailed information about the API is listed in the api documentation.

GltfView

The GltfView component is associated with one WebGL2 context. In practice this means it will be associated with one HTML5 Canvas. This component manages the interaction between the canvas and the GL context. It therefore specifies the viewport, the swapchain and can be used to schedule frame renders.

const view = new GltfView(webGl2Context);

The view is also used to render frames, either on every window repaint event or on demand, e.g. when taking a frame capture.

const update = () => {
    view.renderFrame(state, canvas.width, canvas.height);
    window.requestAnimationFrame(update);
};
window.requestAnimationFrame(update);

The GltfView handles the order of execution for animations, interactivity and physics:

1. Animations are applied
2. Any playing interactivity graph is executed
3. The transform hierarchy is computed
4. Any playing physics engine is updated and applied
5. The scene is rendered
6. If physics is used, all dirty flags are reset

GltfState

The GltfState encapsulates the state of the content of a GltfView. As currently some WebGL resources are stored directly in the Gltf objects, the state cannot be shared between views.

const state = view.createState();
state.sceneIndex = 0;
state.animationIndices = [0, 1, 2];
state.animationTimer.start();

The state is passed to the view.renderFrame function to specify the content that should be rendered.

GraphController

The GltfState contains an instance of the GraphController which can be used to load and execute KHR_interactivity graphs. One can also send custom events to the graph or subscribe to custom event via callbacks.

In the GltfState you can define an array of selection and hover points. Each element of the array represents one controller. If triggerSelection is set to true, the renderer will return the picking result of the clicked position via selectionCallback. The interactivity engine will be notified as well, if KHR_node_selectability is used in the current glTF.

If enableHover is set to true, the renderer will return the picking result of the hovered position via hoverCallback. The interactivity engine receives hover results independent of enableHover based on the hoverPositions array. enableHover enables the use of custom hover handling independent of KHR_interactivity and is set to false by default.

To make sure that KHR_interactivity always behaves correctly together with KHR_node_selectability and KHR_node_hoverability, update the values in the hoverPositions and selectionPositions arrays and trigger selections via triggerSelection. Currently, only one controller is supported. All entries except the first one of each array are ignored. Arrays are used to enable multiple controllers in the future without breaking the API.

AnimationTimer

The GltfState contains an instance of the AnimationTimer, which is used to play, pause and reset animations. It needs to be started to enable animations. The KHR_interactivity extension controls animations if present. Therefore, the GraphController uses the time of the AnimationTimer to control animations. The GraphController is paused and resumed independently from the AnimationTimer, thus if an interactivity graph is paused, currently playing animations will continue playing if the AnimationTimer is not paused as well.

PhysicsController

The GltfState contains an instance of the PhysicsController, which manages rigid-body physics simulation for glTF scenes that use the KHR_physics_rigid_bodies and KHR_implicit_shapes extensions. The controller is available on the state as state.physicsController.

Before loading any scene, the physics engine must be initialized. Currently only "NvidiaPhysX" is supported:

await state.physicsController.initializeEngine("NvidiaPhysX");

After a glTF has been loaded, call loadScene to build the physics actors for the active scene:

state.physicsController.loadScene(state, state.sceneIndex);

The simulation is advanced by calling simulateStep each frame inside GltfView. The controller uses a fixed-step accumulator and only advances the simulation when enough time (simulationStepTime, default 1/60 s) has elapsed.

Playback can be paused and resumed independently of other state:

state.physicsController.pauseSimulation();
state.physicsController.resumeSimulation();

The playing property reflects whether the simulation is currently running, and enabled indicates whether the loaded scene contains any physics data.

Debug visualization of colliders and joints can be toggled at runtime:

state.physicsController.enableDebugColliders(true);
state.physicsController.enableDebugJoints(true);

The following runtime physics operations are available, and are also called internally by the KHR_interactivity engine:

// Apply a linear and/or angular impulse to a node
state.physicsController.applyImpulse(nodeIndex, linearImpulse, angularImpulse);

// Apply an impulse at a specific world-space position on a node
state.physicsController.applyPointImpulse(nodeIndex, impulse, position);

// Cast a ray and return the first hit
const hit = state.physicsController.rayCast(rayStart, rayEnd);

Dirty flags

Dirty flags are per-property boolean markers used to propagate change information through the scene graph without re-evaluating the entire hierarchy every frame. They are the primary mechanism by which animations, KHR_interactivity, and the physics simulation communicate what has changed.

AnimatableProperty dirty flags

Every animatable property defined by the glTF Object Model (e.g. translation, rotation, scale, mass, linearVelocity) is backed by an AnimatableProperty instance that carries a dirty boolean. A property is marked dirty whenever its value is written — either by the animation system or by the interactivity graph. It does not matter, if the written value is the same as the old value, since one can animate an e.g. a node transform to stay still. If the dirty flag would not be set in this case, the physics engine might apply e.g. gravitational forces to the an animated body.

Node transform dirty flags

Each gltfNode carries two additional boolean flags that are computed during scene.applyTransformHierarchy():

Flag	Set when
node.dirtyTransform	This node's local transform or any ancestor's transform changed since the last reset.
node.dirtyScale	This node's scale or any ancestor's scale changed since the last reset.

Resetting dirty flags

glTF.resetAllDirtyFlags() performs a full reset in one call.

GltfView.renderFrame() calls this automatically at the end of each frame — but only when the physics simulation is either disabled or actively playing. When the simulation is paused, dirty flags are intentionally not cleared so that any changes made while paused (e.g. via KHR_interactivity) are still visible to the physics engine once playback resumes.

If you manually advance the simulation (e.g. via a single-step button) you must reset dirty flags yourself afterwards:

state.physicsController.simulateStep(state, 1 / 60);
state.gltf.resetAllDirtyFlags();

ResourceLoader

The ResourceLoader can be used to load external resources and make them available to the renderer.

state.gltf = await resourceLoader.loadGltf("path/to/some.gltf");

Render Fidelity Tools

The glTF Sample Renderer is integrated into Google's render fidelity tools. The render fidelity tools allow the comparison of different renderers. To run the project follow the instructions here and here. For information on how the glTF Sample Renderer was integrated see the pull request on Github.

Development

After cloning this repository, run

npm install

to install all dependencies. To test and view your changes on a canvas, it is recommended to clone glTF Sample Viewer which uses this renderer as a submodule.

npm run build will build the npm package and put the bundled code into the dist directory.

npm run build_docs will regenerate the API documentation.

Formatting

This repository uses Prettier for code formatting and ESLint for linting.

To check the formatting and linting of the code run:

npm run format

Prettier and ESLint are configured to only run on files in the "source" folder and ignore files in "source/libs".

Visual Studio Code

There are extensions for both Prettier and ESLint in Visual Studio Code. They can be found in the marketplace:

• Prettier - Code formatter
• ESLint

You are encouraged to run Prettier and ESLint on your code before committing.

Testing

glTF-Sample-Render uses Playwright for testing.
Currently, only KHR_interactivity tests are implemented.

To run the tests run

npm run test

Playwright creates a new browser instance for each test. It can run on Chrome, Safari, Firefox and emulated mobile browsers. After all tests were run, a browser window with a summary will open. The tests/testApp directory contains a minimal frontend to be able to start a testing server. The server is started automatically. For debugging the test server you can also start it manually by running

npm run testApp

Tests are defined in the tests directory by files with the .spec.ts ending.
The interactivity tests download all test assets from the glTF-Test-Assets-Interactivity repository, loads each test file and listens on the test/onStart, test/onSuccess and test/onFailed events to determine if an interactivity test passes or not. test/onStart returns the needed execution time in seconds.

You can also run more complex Playwright commands such as npx playwright test --ui or npx playwright test --project chromium. For more information check https://playwright.dev/docs/running-tests

One can also use the Playwright extension for Visual Studio Code to run the test more easily with advanced parameters, run tests only selectively or debug tests by adding breakpoints.