---
name: vibexr
description: Use when the user wants a WebXR, VR, AR or passthrough experience - a headset demo, an interactive 3D scene, a "world", or something they can point at, grab, pinch or poke with hands or controllers in VITURE glasses, PICO, Quest, Vision Pro or Galaxy XR (or test with a mouse on the desktop). Builds it as a VibeXR world - one plain object added to the single-file VibeXR framework (three.js r186 via import map, no build step) - and delivers one HTML file that runs in the headset browser after tapping "Enter XR".
---

# VibeXR worlds

VibeXR is a single-file WebXR interaction framework. Input (hands, controllers, gaze / transient pointers, mouse and keyboard on the desktop and in XR), hover / select / drag / two-hand logic, UI, the hand menu, passthrough and the session are already solved. Your job is only to write a **world**: an object with `init(ctx)` that builds content under `ctx.root` and makes things interactive with `ctx.add(...)`.

Reference (read when you need detail):

- API: https://docs.vibexr.io/docs/api.md
- Rules and the lessons behind them: https://docs.vibexr.io/docs/rules.md
- Examples and recipes: https://docs.vibexr.io/docs/examples.md

## The red line

**Never edit framework code.** Do not change anything inside the framework's `<script type="module">`: no new defaults, no patched classes, no tweaks to input, rendering, menu or session code. Add a world, and nothing else. If a behaviour seems wrong, it is almost always a deliberate default with a recorded reason (see rules.md); adapt the world, or tell the user, rather than patching the framework. Runtime tuning via `VXR.ICFG` is allowed only when the user asks for different feel.


## UI: conform to the framework, exactly

Every UI detail is already defined by the framework: buttons, toggles, sliders, panels, labels, the hand menu, hover and press states, colours, line widths, corner radii, type. **Use them; never restyle or re-invent them.** Consistency across worlds is the point: a user who learned one world's menu must recognise every other one.

- Build UI only with `ctx.button` / `ctx.slider` / `ctx.panel` / `ctx.label` and the hand menu (`ctx.app.menu`). No hand-drawn buttons, custom pill shapes, other fonts, glows, colours or hover effects.
- Where something truly can't use those classes (a page without the framework, a canvas you draw yourself), reproduce the framework's definitions to the pixel; the values are listed in rules.md, "House UI guidelines" U9. If the framework changes them, follow the new version.
- Before you ship, compare a screenshot of your UI side by side with a framework hand menu: same weights, same outlines, same states.

## Important: OST or VST? Check the display first

Not every headset shows the same thing. **VST** headsets (PICO, Quest, Vision Pro, Galaxy XR) film the room and draw over it: ~90-110 deg field of view, black is black. Nothing special is needed there; use the normal layout. **OST** glasses (VITURE, XREAL, Rokid, RayNeo, Spectacles, HoloLens, Magic Leap) add light over the real world: the field of view is small (often 30-55 deg diagonal, ~25-30 deg vertical), **black is drawn as nothing**, dark colours are faint, and whatever is outside that small window simply isn't seen.

The framework file ships a display add-on: `ctx.app.display` (`kind`: `"desktop" | "vr" | "vst" | "ost"`, `fov: {h, v}` measured per eye, `fitDistance(w, h, margin)`, `inView(point)`, `onChange(fn)`). It is final by the calibration reload (~10 frames into the session), so read it in `init`. Test OST layouts on a Quest with `?ost=46x26` (masks everything outside a 46 x 26 deg window) or force a kind with `?display=ost`.

When `ctx.app.display.ost`:

1. **Fit the main thing into the window**: place it at `display.fitDistance(width, height, 0.8)` (or scale it down), centred at eye level. A panel 30 deg below the eyes is outside a 26 deg tall window.
2. **Keep UI inside the view**: follow or recentre it, compact; nothing important more than ~15-20 deg off centre. Add a cue at the window edge for things outside it.
3. **Light on nothing**: no dark backdrop or dark fills that carry meaning (they vanish); bright text and outlines, thicker lines, text ~1.5x the VST size. Avoid large bright full-view surfaces (a virtual floor, a sky) - they flood the small window; hide them or make them optional.
4. **Input may be thinner**: many glasses have no hand tracking; make every action reachable by ray / gaze / controller / mouse, not only by poke.

The add-on already turns off the backdrop and dark hands on OST. On VST and VR, change nothing.

Eye tracking is TBD: WebXR exposes no continuous gaze; "gaze" in these docs means Vision Pro's look-and-pinch, which arrives as a transient pointer only when the user pinches. Don't design gaze-hover interactions.

## Workflow

1. **Get the framework.** Download `https://docs.vibexr.io/framework/vibexr.html` and save it as the deliverable, e.g. `my-world.html`. Check `window.VXR.VERSION` / the changelog at the top of the file; this skill matches v0.9.5.
2. **Write the world** as a plain object (contract below).
3. **Register it and make it the startup world** by adding one module script after the framework's closing `</script>` (just before `</body>`; the file is a complete HTML document):
   ```html
   <script type="module">
   const { app } = window.VXR;
   const MyWorld = { id: "my-world", title: "My World", init(ctx) { /* ... */ } };
   app.worlds.register(MyWorld);
   await app.worlds.load("my-world");
   </script>
   ```
   (The framework itself registers the built-in Playground and runs `await app.worlds.load("playground")`; your script then replaces it. From then on every session start, calibration and session end reloads your world.)
4. **Serve over local HTTP**: `python3 -m http.server 8000` in the file's folder. Never open it as `file://` (modules and WebXR need http(s)).
5. **Self-test on the desktop** (loop below) until every assert passes and the console is clean.
6. **Then the headset**: open the same page on the headset browser (HTTPS or a forwarded localhost; WebXR needs a secure context) and tap **Enter XR**. Report what you verified on desktop and what still needs a headset check (hand tracking feel, poke, passthrough).

## World contract

```js
const MyWorld = {
  id: "my-world",            // unique
  title: "My World",         // shown in the log
  init(ctx) { },             // build everything here; may be async
  enter(ctx) { },            // optional, after init
  update(dt, t, ctx) { },    // optional, every frame; dt in s (<= 1/30), t in s
  exit(ctx) { },             // optional, before automatic cleanup
  claimKeys: "wasd",         // optional: keys the world reads itself (ctx.app.keys); navigation leaves them alone
  background: 0.6,           // optional 0..1: an AR session starts with the Background raised this far
};
```

`ctx` members: `app`, `THREE`, `root` (an un-rotated Group already in the scene), `world`, `add(obj, opts) -> Interactable`, `button(label, opts) -> UIButton`, `slider(label, opts) -> UISlider`, `panel(title, opts) -> UIPanel` (+ `layout(items, opts)`), `label(text, {height, color}) -> Mesh`, `log(...args)`, `session` (`null` | `"ar"` | `"vr"`).

There is no `ctx.box`, `ctx.interactable`, `ctx.on`, `ctx.reload`, `ctx.info` or `ctx.placeHome` (the last two are not in v0.9.5). Build meshes with `new ctx.THREE.Mesh(...)`, make them interactive with `ctx.add(mesh, opts)`, listen with `interactable.on("tap", fn)`, reload with `ctx.app.worlds.reload()`.

Interactable options: `near`, `far` (default true), `poke` (`null` | `"surface"` | `"volume"`), `draggable`, `twoHand`, `multi`, `translateOnly`, `faceViewer` (stages: carried level, swings to face you), `constrain(t, it)`, `precise`, `box`, `highlight` (default true), `outlinePush`, `longPress`, `onPokeDepth(d)`, `dragThreshold`.

Events (`it.on(name, e => ...)`, every payload has `type`, `target`, most have `interactor`): `hoverenter`, `hoverexit`, `statechange {level}`, `press {point}`, `dragstart`, `drag {point, delta}`, `dragend`, `longpress`, `tap`, `release {cancelled}`, `transformstart`, `scale {ratio, delta}`, `rotate {quaternion, angle}`, `transformend`.

Buttons: `ctx.button("Label", { w, h, toggle, value, onClick(value), fontSize, swatch, capsule })`; you add `button.root` to the scene yourself, as with sliders, panels and labels.

App extras a world may use: `ctx.app.setBackdrop([top, horizon, bottom] | null)` (the world's own sky; reset on unload), `ctx.app.keys` (held keys, lower case), `ctx.app.input.head` (desktop: the camera position; `input.headQuat` is only live in XR, use `ctx.app.camera.quaternion` on the desktop).

Automatic on unload: the world's sky reset to black, owned interactables removed, `ctx.root` removed and every geometry / material / texture under it disposed. Not automatic: anything added to `app.scene` directly, `app.im.add` interactables, timers, DOM listeners.

## The 13 rules that matter most

1. **Parent everything to `ctx.root`; create interactables only through `ctx.add` / `ctx.button` / `ctx.slider`.** That is what makes reload and unload clean.
2. **`init` must be idempotent.** It runs on page load, on session start, ~10 frames later (real head pose) and on session end. Read `ctx.session` and `ctx.app.input.head` fresh each time; rebuild, don't accumulate.
3. **Place from the live head, in `init`.** There is no layout helper in v0.9.5: position things from `ctx.app.input.head` (e.g. the main object ~0.8-1.6 m ahead, slightly below eye height; a control panel at ~0.55 m, 30 deg down so it can be poked), or copy the `inFront(ctx, obj, { yaw, dist, pitch })` helper from examples.md. Nothing follows the viewer unless your `update` moves it.
4. **No throw, no inertia, no physics by default.** Released objects stay where they were let go. Clamp with `constrain({position, quaternion, scale})` (world space). The two-hand scale is always clamped to 0.25-4x first; `constrain` can narrow that, not widen it.
5. **`precise: true` for anything not box-shaped** (spheres, models); the default far hover uses the bounding box. A parent interactable with controls riding on it needs an explicit `box` (just the part you grab): the nearest box on the ray wins.
6. **Poke surfaces face local +Z.** Use `ctx.button` / `ctx.slider` rather than custom buttons: they click on release, support poke, pinch, ray and mouse, and play the framework's click. Within 6 cm in front of a control the fingertip pokes it; a pinch there does not select it.
7. **Never set `renderOrder` on UI.** It is global: a far panel's text would paint over a near panel. Offset layers physically in z.
8. **UI is opaque with an alpha cutout** (`alphaTest: 0.5, alphaToCoverage: true`) so passthrough never shows through it; text-only planes use `transparent: true, depthWrite: false`.
9. **Hide a collider's material, not the object** (`mesh.material.visible = false`). An invisible object, or one with an invisible ancestor, is not hit-testable.
10. **Don't add your own rays, cursors, pinch balls or glow.** Hover is the framework's white outline; ray vs direct is decided by hover alone. Mark decorative meshes `userData.noCollider` / `noHighlight`. Don't overwrite `Interactable.onAny` (sound + event log use it).
11. **React through Interactable events, not gamepad buttons.** Gaze and transient pointers have no gamepad; their select arrives through the framework's events like everything else.
12. **Don't touch `scene.background`; detach shared framework objects in `exit`; keep `update` from throwing** (one exception disables it for the rest of the page). For a sky call `ctx.app.setBackdrop([...])` in `init`; to start AR dimmed set `world.background`; use `ctx.session` / `ctx.app.vst.opacity` to adapt to passthrough. If the world reads keys, declare them in `claimKeys` (W A S D / Q E otherwise fly the desktop camera and walk the viewer in XR; F frames).
13. **Check `ctx.app.display` before laying out.** OST glasses: fit the content into `display.fov` (`fitDistance`), keep UI centred, no meaning in black, no big bright floors / skies. VST and VR: the normal layout (see "OST or VST?" above).

## House UI guidelines

Follow these on top of the rules (details and reasons: rules.md, "House UI guidelines"):

- **One "Enter XR" button**, passthrough first (`immersive-ar`, else `immersive-vr`). The 0.9.5 template's button says "Enter": relabel it **Enter XR**. Never separate VR / AR buttons.
- **Hand menu, two pages**: your world's own controls (`ctx.button`, hung on `ctx.app.menu.root`) on page one with a full-width **Settings ›**; the framework's shared switches, Background and **Exit XR** on Settings. On vibexr.io the hub glue does this paging; in a standalone file keep your own controls few.
- **Windows**: a strict grid, max **4 columns**, equal-height header pills, a scroll column + drag-to-scroll when rows overflow, every canvas at one pixel density, open **pinned (6DoF)** with follow as an opt-in.
- **Style**: black board, white lines; hover = heavier line; on = inverted. Colours in one palette object.
- **Credits**: card ends with `by <author> · WebGPU | WebGL 2`; every outside asset gets author, licence and source.
- **Feeds**: a tap on content never navigates; only an Enter button goes in.
- **Text size by visual angle** (rules U10): titles ≥ ~2°, body ≥ ~1.2°, small print ≥ ~1° at the reading distance (at 0.7 m: 25 / 15 / 12 mm). When in doubt, bigger.
- **What you see stops the ray** (rules U15): panel boards, labels and displayed models are inert interactables (`highlight: false`) so the ray and cursor stop on them; a grabbable frame or tool answers the ray only on its drawn edges and handle (`precise: true`), never on an invisible pane.
- **Background music by default** (rules U14): unless the user says otherwise, every world ships with music that fits its theme (a Greek temple: a lyre in the Dorian mode; a sky garden: soft bells and wind), generated live with Web Audio (no samples unless you can credit and license them). Keep it light and calm: a handful of audio nodes, sparse notes, no convolution reverb (one short feedback delay at most) - a full score only where music is the point (a rhythm game). On by default at a low level, fading in once the audio context runs (after Enter / the first click), a **Music ●/○** toggle on hand-menu page one, stopped in `exit`. Credit it on the card as "Music generated live".



Do not judge by eyeballing screenshots. Drive the real input path with synthetic pointer events, step frames by hand, and assert numbers. On the desktop, the mouse is a full far-interaction source (left-click = ray pinch, wheel while dragging = push / pull), so taps, drags, button clicks and long presses are all testable. Navigation uses the other inputs (right-drag look, right + W A S D / Q E fly, middle pan, Alt + left orbit, wheel zoom, F frame), so plain left-button events never move the camera. Keys are testable too: `window.dispatchEvent(new KeyboardEvent("keydown", { key: "w" }))` puts `"w"` in `app.keys` until the matching `keyup`.

Run this in the served page (browser automation `evaluate`, the devtools console, or an extra test `<script type="module">` appended to a throwaway copy of the file, then read the result from the DOM):

```js
const { app, THREE: T } = window.VXR;
app.renderer.setAnimationLoop(null);                 // stop the RAF loop: we step frames ourselves
const dom = app.renderer.domElement;
const step = (n = 1) => { for (let i = 0; i < n; i++) { app.last = performance.now() - 1000 / 60; app._loop(0, null); } };
const toClient = (obj) => {                          // world position -> client pixel
  const v = obj.getWorldPosition(new T.Vector3()).project(app.camera), r = dom.getBoundingClientRect();
  return { clientX: r.left + (v.x + 1) / 2 * r.width, clientY: r.top + (1 - v.y) / 2 * r.height };
};
const ev = (type, p, target = dom) => target.dispatchEvent(new PointerEvent(type, { ...p, button: 0, bubbles: true }));
const assert = (name, cond, info = "") => console.log((cond ? "PASS " : "FAIL ") + name, info);

const root = app.worlds.ctx.root;
const cube = root.getObjectByName("hello-cube");     // name your objects so tests can find them
let p = toClient(cube);
ev("pointermove", p); step(2);
assert("hover", cube.userData.interactable.hoverers.size === 1);

const c0 = cube.material.color.getHex();
ev("pointerdown", p); step(1); ev("pointerup", p, window); step(1);   // pointerup goes to window
assert("tap recolours", cube.material.color.getHex() !== c0);
assert("tap logged", app.events.some(e => e.type === "tap" && e.name === "hello-cube"));

const w0 = cube.getWorldPosition(new T.Vector3());
ev("pointerdown", p); step(1);
for (let k = 1; k <= 10; k++) { ev("pointermove", { clientX: p.clientX + 12 * k, clientY: p.clientY }); step(1); }
ev("pointerup", { clientX: p.clientX + 120, clientY: p.clientY }, window); step(1);
assert("drag moved it", cube.getWorldPosition(new T.Vector3()).distanceTo(w0) > 0.05);
```

What to assert:

- **Positions / distances**: `obj.getWorldPosition(v)`, distance from `app.input.head` (desktop: the camera position), yaw from the view direction (`app.camera.quaternion` on the desktop).
- **Angles**: `obj.getWorldQuaternion(q).angleTo(q0)`, or the angle between an object's +Z and the direction to the head (is a panel facing the viewer?).
- **Sizes / overlap**: `new T.Box3().setFromObject(obj)`; check panels do not intersect, labels fit inside their panel, objects sit above a table (`box.min.y`).
- **Visibility on screen**: `toClient(obj)` inside the canvas rect.
- **Events**: `app.events` (`{type, name, via, t}`, `name` = `object.name`) and your own state.
- **Errors**: `app.worlds.current === MyWorld`, `MyWorld.update !== null` after stepping (an exception disables update), no console errors, the page's `#error` element hidden.

Notes: tap requires press-to-release under 550 ms of real time and long press needs 650 ms of real time (timers use `performance.now()`), so wait real time for long presses. `step()` gives `dt = 1/60`. Worlds reload on `app.worlds.reload()`; re-run the asserts after a reload to prove `init` is idempotent. Headless Chrome works (`--headless=new --use-angle=swiftshader --enable-unsafe-swiftshader`, network needed for the three.js CDN). The page's render loop never goes idle, so `--dump-dom` with `--virtual-time-budget` or `--timeout` may never return: have the test script `fetch()` its results to a small local server (or read them over the devtools protocol) and kill Chrome afterwards. The framework's `<link rel="icon">` points to `../Resources/common/favicon-64.png`; that 404 is harmless and is not a console error from your world.

## Multiplayer: people, hands and voice

When a world has more than one person (a shared table, a game, a chat room), use the shared modules on vibexr.io instead of writing networking or avatars yourself. They are plain ES modules, importable from any page:

```js
const { createNet } = await import("https://vibexr.io/net.js");                    // rooms, data, voice
const { createRemoteHand } = await import("https://vibexr.io/hands.js");            // other people's real hands
```

- **Room**: `createNet({ app: "<world id>", room, dev, onMessage, onPeers, onStatus })`. Signaling and fallback relay go over public MQTT brokers (they work on headset Wi-Fi in China); each pair then upgrades to a direct WebRTC channel. `net.send(data, to?)` uses the direct channel when it is open and the brokers otherwise. `net.peers()` lists `{id, dev, p2p}`.
- **Pose, every message**: head position + orientation, and per hand the grip pose. With hand tracking, also the **25 joints** of each hand as one flat array `[x, y, z, qx, qy, qz, qw] x 25` in WebXR joint order (wrist, thumb x4, index x5, middle x5, ring x5, pinky x5), rounded to 1 mm / 0.001.
- **Rate**: 30 messages a second while every peer has a direct channel; while anyone is still relayed through the brokers, send the joints only every 4th message (the brokers are shared and slower). A message without joints means "unchanged": the receiver keeps the last set. `null` means the hand is gone or is holding a controller.
- **Frame**: send in a frame both sides share. Same room and origin: world space. A game table / arena: that object's local frame (`arena.worldToLocal`, orientation premultiplied by the arena's inverse). Seats around a table, each person at their own origin: rotate every joint position about the table centre and every joint orientation by the same yaw (their seat angle minus yours).
- **Draw**: `createRemoteHand("left" | "right", parent, { fill })` gives the same outlined hand as your own (the WebXR generic hand, 2 mm white outline). Call `.set(joints | null)` when a message arrives and `.update(k)` every frame. `parent` is the group the joints are expressed in (the scene, or the arena). `fill: "mask"` in framework worlds (see-through fill, like the framework's own hands); `"black"` on fully black scenes drawn late (the lobby). A person on controllers gets the framework's small outlined ball per hand instead.
- **Smoothing**: ease toward the newest pose with `k = min(1, dt * 30)`: about two network frames, smooth without adding visible lag. No extrapolation.
- **Voice, on by default**: `net.voice.enable()` from the Enter XR click (it asks for the microphone once); everyone in the room hears everyone. Each frame call `net.voice.place(peerId, x, y, z)` with their head position and `net.voice.listener(headPos, forward, up)` with yours, so voices come from where people are. Put a `Mic ●/○` toggle on the hand menu (page one) and remember the choice (`localStorage "vibexr.voice"`). Studio does not hold worlds back for using the microphone.

## Delivery format

- **One HTML file**: the framework file unchanged, plus your world in one appended `<script type="module">`.
- three.js r186 comes from the import map already in the file. Inside the world use `ctx.THREE`; if you need a three.js addon, import it in your script via the same map (`import { GLTFLoader } from "three/examples/jsm/loaders/GLTFLoader.js";`).
- No build step, no bundler, no npm, no extra framework.
- Give each important object a `name`; write the info card text in plain words (what to do with hands, controller and mouse).
- Tell the user: the URL to open, how to enter (tap "Enter"; it starts passthrough where supported, VR otherwise), and what was verified on the desktop.

A text-to-world generator ("say it, wear it" in about 30 seconds) is a stated goal and is coming; today, worlds are written by hand or by an agent following this skill.