Camera & picking

Step 3 of the 3D track. The scene is built, the fox runs — but you can only look at all of it from wherever you parked the camera, and there is nothing to touch. This step is about the two things that turn a picture into an app: the finger drives the camera, and the finger hits objects.

Both come down to the same facts: the camera is an ordinary node, and 3D touch events work exactly like the UI ones, with a single new idea — the ray.

Orbiting the scene with a finger

The classic orbit camera: it sits on a sphere around a target, and the finger turns two angles — yaw (around the vertical) and pitch (height above the horizon). Catch the drag with touchstart + ev.track() — the same API as dragging in the UI track:

TypeScript
const scene = new Scene({ skybox: "#1b2233" })
scene.add(Light.sun({ direction: [-1, -2, 0.7], intensity: 90000, shadowsQuality: 2 }))
scene.add(Mesh.plane({ material: Material.lit({ color: "#2f4a35", roughness: 1 }), normal: [0, 1, 0], scale: 24, receiveShadows: true }))
const wood = Material.lit({ color: "#b5854b", roughness: 0.8 })
scene.add(
  Mesh.box({ material: wood, size: 0.8, position: [0.6, 0.4, 0.4], castShadows: true }),
  Mesh.box({ material: wood, size: 0.55, position: [0.6, 1.075, 0.4], eulerAngles: [0, 30, 0], castShadows: true }),
  Mesh.box({ material: wood, size: 0.6, position: [-0.7, 0.3, 1.1], eulerAngles: [0, -20, 0], castShadows: true }),
)

// Orbit: the camera sits on a sphere around the target. yaw turns around the vertical,
// pitch is the height above the horizon, radius is the distance. Angles here are radians — plain math
const target = [0, 0.5, 0]
let yaw = 0.6, pitch = 0.5, radius = 6
function placeCamera() {
  scene.camera.position = [
    target[0] + radius * Math.cos(pitch) * Math.sin(yaw),
    target[1] + radius * Math.sin(pitch),
    target[2] + radius * Math.cos(pitch) * Math.cos(yaw),
  ]
  scene.camera.lookAt(target)
}
placeCamera()

// A finger lands on the scene: capture the angles at touch-down and add the finger's offset to them
scene.addEventListener("touchstart", ev => {
  const yaw0 = yaw, pitch0 = pitch
  ev.track({
    onMove: p => {
      yaw = yaw0 - (p.clientX - ev.clientX) * 0.01
      pitch = Mathf.clamp(pitch0 + (p.clientY - ev.clientY) * 0.01, 0.1, 1.4)
      placeCamera()
    },
  })
})

// Zoom in / out — plain HUD buttons that change radius
const zoom = (label: string, step: number) =>
  UIButton(UIText(label).style({ color: "#fff", fontSize: 18, fontWeight: 700 }))
    .style({ bgColor: "#ffffff22", width: 40, height: 40, borderRadius: 12, alignItems: "center", justifyContent: "center", $pressed: { opacity: 0.6 } })
    .onClick(() => { radius = Mathf.clamp(radius + step, 3, 12); placeCamera() })
const hud = UIWidget(
  UIText("Drag to orbit").style({ fontSize: 13, color: "#c7ccd6" }),
  UIRow(zoom("−", 1), zoom("+", -1)).style({ gap: 8 }),
).style({ top: "comfort-top", left: 16, gap: 10 })
scene.widgets(hud)
scene.open()
  • The camera is a node, not a director. scene.camera.position and lookAt(point) are all you need; there is no "camera controller" in the SDK and none is wanted. Set the position yourself — from an orbit, from the hero's position, from anything.
  • Compute the position, don't spin the camera. The temptation to write camera.eulerAngles.y += … ends in drift and gimbal lock. Keep the state in variables (yaw / pitch / radius) and recompute the position from scratch every time — easier to debug, and it saves to disk in one line.
  • A drag is touchstart + ev.track(), exactly as in UI. The angles captured at touch-down (yaw0, pitch0) plus the offset from the touch point (p.clientX - ev.clientX) — don't confuse that with p.deltaX, which is the shift between consecutive events, not since the gesture began.
  • A scene has no one to fight over the finger. Every pointer the UI over it didn't take is the scene's, so its track() needs no claim (that option is for drags inside scrolling UI, where a list wants the same finger) — the HUD buttons keep their taps, the rest of the screen orbits.
  • Mathf.clamp(pitch, 0.1, 1.4) keeps the camera from diving under the floor or flipping at the zenith — a clamp is almost always what you want over a "free" camera.
  • 3D touches arrive in logical px — the same coordinates as ev.clientX in UI. That matters one section down.

There's a lens too: camera.fov (vertical angle in degrees, 60 by default), camera.near / camera.far for the view range. Full list in the Camera reference.

Hitting an object with a finger

A mesh on its own is transparent to the finger: the engine draws it but doesn't know where its bounds are. To make an object tappable it needs a Shape aspect — a body for hit-testing. {} fits a box straight to the mesh:

TypeScript
const scene = new Scene({ skybox: "#1b2233" })
scene.add(Light.sun({ direction: [-1, -2, 0.7], intensity: 90000, shadowsQuality: 2 }))
scene.add(Mesh.plane({ material: Material.lit({ color: "#2f4a35", roughness: 1 }), normal: [0, 1, 0], scale: 24, receiveShadows: true }))
scene.camera.position = [0, 3.5, 5]
scene.camera.lookAt([0, 0.4, 0])

const wood = Material.lit({ color: "#b5854b", roughness: 0.8 })
const picked = Material.lit({ color: "#FF4032", roughness: 0.8 })

// Shape is what makes a node tangible to the finger — without it a tap goes straight through the mesh.
// {} — a box the size of the mesh itself
const crates: Mesh[] = [[-1.4, 0.3], [0.2, -0.6], [1.5, 0.5]].map(([x, z], i) =>
  Mesh.box({ material: wood, size: 0.8, position: [x, 0.4, z], eulerAngles: [0, x * 25, 0], castShadows: true, name: `crate ${i + 1}` })
    .aspect(Shape, {}),
)
scene.add(...crates)

const label = UIText("Tap a crate").style({ fontSize: 13, color: "#c7ccd6" })
const hud = UIWidget(label).style({ top: "comfort-top", left: 16, right: 16 })
scene.widgets(hud)
scene.open()

// A listener on the node — when the reaction belongs to the object itself
const taps = crates.map(() => 0)
crates.forEach((crate, i) => crate.addEventListener("click", () => taps[i]++))

// A listener on the scene sees every tap: ev.target is the Shape-carrying node under the finger, or null.
// Both fire: the node first, then the scene
let current: Mesh | null = null
scene.addEventListener("click", ev => {
  current?.setMaterial(wood)
  current = ev.target as Mesh | null
  current?.setMaterial(picked)
  label.text = current ? `picked ${current.name} · taps on it: ${taps[crates.indexOf(current)]}` : "missed — selection cleared"
})
  • No Shape, no hits. This is the number-one reason "the click doesn't work": the node draws, but it has no pick body, so the tap falls through to the scene with ev.target === null.
  • Two places to listen. node.addEventListener("click", …) — the reaction belongs to the object; scene.addEventListener("click", …) — sees every tap, misses included, and knows what was hit (ev.target). A miss is only visible to the scene — that's the whole difference.
  • A bare Shape follows position, not the simulation. Its pick body is static: a write to node.position moves it along, but motion applied any other way (a tween, physics) leaves it behind. An object the simulation moves needs a real body — next step.
  • {} is a box from the mesh, but you can be precise: { sphere: 0.4 }, { capsule: { halfHeight: 0.5, radius: 0.3 } }, { box: [0.4, 0.4, 0.4] }. Boxes take half-extents: box: [0.5, 0.5, 0.5] around a 1×1×1 cube. And they're world units — the node's scale doesn't touch them.
  • raycast: false in Shape removes a node from picking while keeping its physical behavior — that's how decoration that shouldn't steal taps is marked.
  • touchstart works on nodes just the same and can ev.track() — that's "grab an object and drag it", ready-made.

The ray and the plane

A tap gives a point on the screen, and the world is three-dimensional: one screen point corresponds to a whole ray running into the scene. camera.getRay(x, y) builds that ray, and from there you have two ways to ask what it hit: Physics.raycast against bodies, Plane.intersectRay against a mathematical plane (a floor, a wall, a table) that has neither a mesh nor a body:

TypeScript
const scene = new Scene({ skybox: "#1b2233" })
scene.add(Light.sun({ direction: [-1, -2, 0.7], intensity: 90000, shadowsQuality: 2 }))
scene.add(Mesh.plane({ material: Material.lit({ color: "#2f4a35", roughness: 1 }), normal: [0, 1, 0], scale: 24, receiveShadows: true }))
scene.camera.position = [0, 4.5, 6]
scene.camera.lookAt([0, 0, -0.5])
const wood = Material.lit({ color: "#b5854b", roughness: 0.8 })

const label = UIText("Tap the ground to place a crate, tap a crate to remove it").style({ fontSize: 13, color: "#c7ccd6" })
const hud = UIWidget(label).style({ top: "comfort-top", left: 16, right: 16 })
scene.widgets(hud)
scene.open()

// The floor is a mathematical plane at y = 0 (no body, no mesh): intersect the camera ray with it
const ground = new Plane([0, 1, 0], [0, 0, 0])
let count = 0

scene.addEventListener("click", ev => {
  // A ray from the camera through a screen point — logical px, same as ev.clientX / ev.clientY
  const ray = scene.camera.getRay(ev.clientX, ev.clientY)

  // 1. Ask physics: did the ray hit an existing crate? Remove it
  const hit = Physics.raycast(ray.origin, ray.dir, 100)
  if (hit?.node) {
    scene.remove(hit.node)
    hit.node.destroy()
    label.text = `crates: ${--count}`
    return
  }

  // 2. Otherwise intersect the floor: t is the parameter along the ray, getPoint(t) gives the point
  const t = ground.intersectRay(ray)
  if (t === null) return
  const p = ray.getPoint(t)
  scene.add(
    Mesh.box({ material: wood, size: 0.6, position: [p.x, 0.3, p.z], eulerAngles: [0, Math.random() * 90, 0], castShadows: true })
      .aspect(Shape, {}),
  )
  label.text = `crates: ${++count}`
})

The order inside the handler isn't accidental: first "did we hit an object?", and only if not, "where on the floor?". That way one tap does different things depending on what's under the finger — the standard shape of "place / select / remove".

  • Plane is math, not a scene object. new Plane([0, 1, 0], [0, 0, 0]) is an infinite plane through the origin with its normal up. It draws nothing, collides with nothing and costs nothing: perfect for "where on the floor did they point".
  • intersectRay returns t, not a point. t is the parameter along the ray; ray.getPoint(t) gives the point. null means "the ray is parallel to the plane or pointing away from it".
  • Physics.raycast(origin, dir, maxDist) finds the closest body with a Shape (bare pick bodies included) and returns { node, point, normal, fraction } or null. normal is the surface normal at the hit point: that's how decals are placed, objects aligned to a wall, projectiles bounced.
  • ev.target versus raycast — the same hit by two routes. The event is handier for "something got tapped"; an explicit raycast is for rays that don't come from a tap (the hero's gaze, an aim, "can the enemy see the player?") or when you need point and normal.
  • destroy() is the only way to remove a node for good; the node is unusable afterwards. scene.remove(node) only takes it out of the scene.
Note

raycast's distance limit is the third argument (1000 by default). fraction is measured against it, so the hit distance is fraction × maxDist × |dir|; normalize dir if you want meters.

A camera that follows

A camera rigidly bolted to the hero jitters and makes people seasick. The trick that fixes it in one line is smooth chasing: every frame the camera moves not to the desired point but a fraction of the way there:

TypeScript
const scene = new Scene({ skybox: "#1b2233" })
scene.add(Light.sun({ direction: [-1, -2, 0.7], intensity: 90000, shadowsQuality: 2 }))
scene.add(Mesh.plane({ material: Material.lit({ color: "#2f4a35", roughness: 1 }), normal: [0, 1, 0], scale: 40, receiveShadows: true }))
const wood = Material.lit({ color: "#b5854b", roughness: 0.8 })
for (let i = 0; i < 8; i++) {
  const a = (i / 8) * Mathf.TAU
  scene.add(Mesh.box({ material: wood, size: 0.6, position: [Math.cos(a) * 5, 0.3, Math.sin(a) * 5], eulerAngles: [0, i * 20, 0], castShadows: true }))
}
scene.camera.position = [0, 4, 8]
scene.camera.lookAt([0, 0, 0])

const follow = signal(true)
const chip = UIButton(UIText(() => follow.value ? "camera: following" : "camera: parked").style({ color: "#fff", fontSize: 13, fontWeight: 700 }))
  .style({ bgColor: "#ffffff22", borderRadius: 999, px: 12, py: 7, $pressed: { opacity: 0.6 } })
  .onClick(() => follow.value = !follow.value)
const hud = UIWidget(chip).style({ top: "comfort-top", left: 16 })
scene.widgets(hud)
scene.open()

// The fox runs a circle of radius 3 — her own aspect
class Run extends Aspect<"run"> {
  t = 0
  update(dt: number) {
    this.t += dt * 0.7
    this.node.position = [Math.cos(this.t) * 3, 0, Math.sin(this.t) * 3]
    this.node.eulerAngles = [0, -this.t * RAD2DEG, 0]
  }
}

// The camera's aspect: the desired point is behind the fox and above her. The camera doesn't jump there,
// it chases — a fraction of the way per frame that doesn't depend on FPS. It runs after Run, every frame
class Chase extends Aspect<"chase"> {
  static after = [Run]
  target: Node | null = null
  update(dt: number) {
    if (!follow.value || !this.target) return
    const behind = this.target.position.add(this.target.forward.scale(3)).add([0, 2, 0])
    this.node.position = this.node.position.lerp(behind, 1 - Math.pow(0.02, dt))
    this.node.lookAt(this.target.position.add([0, 0.4, 0]))
  }
}

async function main() {
  const fox = await Model.load("https://le.codes/guide-assets/fox-anim.glb")
  fox.castShadows = true
  scene.add(fox.aspect(Run))
  fox.anim.playLoop("run")
  scene.camera.aspect(Chase, { target: fox })
}
main()
  • 1 - Math.pow(k, dt) is the frame-rate-correct lerp. A naive lerp(target, 0.1) every frame moves at different speeds at 60 and 120 FPS; in this form k is the fraction of the remaining distance left after one second (0.02 ≈ "close the gap 50× per second"), and the result looks the same on any hardware.
  • The position chases, the gaze doesn't. lookAt is computed from the already-smoothed position — that's enough; smoothing the look-at point too is only worth it for slow cinematic cameras.
  • Vectors are values. fox.position.add([...]) returns a new Vec3 and leaves the original alone — which is why computing "position plus offset" inline is so convenient. A node only changes when you assign back (node.position = …) or set a scalar (node.x = …).
  • "Behind" comes from the fox's facing. Every node has forward, its −Z axis in world space. Our fox model looks along its own +Z, so behind her is position + forward × 3; for a model that looks along −Z it's position − forward × 3. Check which way a model faces once, by eye.
  • Two aspects, one order. Run moves the fox, Chase reads where she is: static after = [Run] puts the camera after her in every frame, so it never trails a frame behind. Both use update(dt) — the late phase, right before the frame draws.
  • The same trick is the basis for "camera in a car", "over-the-shoulder camera" and "camera that eases back to its default once released".

The whole screen

Putting it together: the yard, the fox, a finger orbit, tap the ground for the fox's destination, tap a crate to select it. Meanwhile the camera softly keeps the fox centered — orbiting and following at the same time:

TypeScript
// The yard + the fox: drag to orbit around her, tap the ground and she runs there, tap a crate to highlight it.
const scene = new Scene({ skybox: "#1b2233" })
scene.add(Light.sun({ direction: [-1, -2, 0.7], intensity: 90000, shadowsQuality: 2 }))

const lit = (color: string, roughness = 1) => Material.lit({ color, roughness })
scene.add(Mesh.plane({ material: lit("#2f4a35"), normal: [0, 1, 0], scale: 30, receiveShadows: true }))

function Tree(x: number, z: number, height = 1.6) {
  const tree = new Node()
  tree.position = [x, 0, z]
  const crown = Geometry.cylinder({ radiusTop: 0, radiusBottom: 0.55, edges: 8, smooth: false })
    .scale(1, height, 1).translate(0, height / 2 + 0.6, 0)
  tree.add(
    Mesh.cylinder({ material: lit("#6b4a2b"), radius: 0.12, scale: [1, 0.7, 1], position: [0, 0.35, 0], castShadows: true }),
    Mesh.from(crown, { material: lit("#3d8b4f"), castShadows: true }),
  )
  return tree
}
const wood = lit("#b5854b", 0.8), picked = lit("#FF4032", 0.8)
const Crate = (x: number, z: number) =>
  Mesh.box({ material: wood, size: 0.8, position: [x, 0.4, z], eulerAngles: [0, x * 30, 0], castShadows: true }).aspect(Shape, {})
scene.add(Tree(-3.2, -1.4, 2), Tree(3, -2, 1.6), Tree(-2.6, 2.6, 1.3), Tree(3.2, 2.2, 2.2), Crate(2, 1), Crate(-2.5, -0.8), Crate(1, -2.5))

const hint = UIText("Loading the fox…").style({ fontSize: 13, color: "#c7ccd6" })
const hud = UIWidget(hint).style({ top: "comfort-top", left: 16, right: 16 })
scene.widgets(hud)

// The fox's aspect: run to the destination a tap on the ground set, then rest
let goal: Vec3 | null = null
class GoTo extends Aspect<"goTo", Model> {
  running = false
  update(dt: number) {
    if (!goal) return
    const to = goal.sub(this.node.position)
    if (to.length() < 0.15) { goal = null; this.running = false; this.node.anim.playLoop("idle", { fade: 0.2 }); return }
    if (!this.running) { this.running = true; this.node.anim.playLoop("run", { fade: 0.15 }) }
    this.node.eulerAngles = [0, Math.atan2(to.x, to.z) * RAD2DEG, 0]   // the model faces along its own +Z
    this.node.position = this.node.position.add(to.normalize().scale(3 * dt))
  }
}

// The camera's aspect: an orbit around a target that eases along after the fox — after she moved
class Orbit extends Aspect<"orbit"> {
  static after = [GoTo]
  yaw = 0.6
  pitch = 0.5
  target = Vec3.from([0, 0.5, 0])
  follow: Node | null = null
  update(dt: number) {
    if (this.follow) this.target = this.target.lerp(this.follow.position.add([0, 0.5, 0]), 1 - Math.pow(0.01, dt))
    this.node.position = this.target.add([7 * Math.cos(this.pitch) * Math.sin(this.yaw), 7 * Math.sin(this.pitch), 7 * Math.cos(this.pitch) * Math.cos(this.yaw)])
    this.node.lookAt(this.target)
  }
}
const camera = scene.camera.aspect(Orbit)
scene.addEventListener("touchstart", ev => {
  const yaw0 = camera.orbit.yaw, pitch0 = camera.orbit.pitch
  ev.track({ onMove: p => {
    camera.orbit.yaw = yaw0 - (p.clientX - ev.clientX) * 0.01
    camera.orbit.pitch = Mathf.clamp(pitch0 + (p.clientY - ev.clientY) * 0.01, 0.15, 1.4)
  } })
})
scene.open()

// A tap: on a crate — select it, on the ground — a destination for the fox
const ground = new Plane([0, 1, 0], [0, 0, 0])
let selected: Mesh | null = null
scene.addEventListener("click", ev => {
  if (ev.target) { selected?.setMaterial(wood); selected = ev.target as Mesh; selected.setMaterial(picked); return }
  const ray = scene.camera.getRay(ev.clientX, ev.clientY)
  const t = ground.intersectRay(ray)
  if (t !== null) { const p = ray.getPoint(t); goal = Vec3.from([p.x, 0, p.z]) }
})

async function main() {
  const fox = await Model.load("https://le.codes/guide-assets/fox-anim.glb")
  fox.castShadows = true
  fox.eulerAngles = [0, 90, 0]
  scene.add(fox.aspect(GoTo))
  fox.anim.playLoop("idle")
  camera.orbit.follow = fox
  hint.text = "Tap the ground and the fox runs. Drag to orbit"
}
main()

Three independent pieces of state — the orbit angles, the camera target and the fox's destination — each kept where it's used: the camera's Orbit holds the angles and the target, the fox's GoTo runs her to the destination, and static after puts the camera after the fox in every frame. The finger only writes camera.orbit.yaw / pitch — an aspect's fields are reachable through its accessor (camera.orbit), named after the class's first generic. That's how 3D scenes grow: not one big update function, but small behaviors on the nodes they move.

The fox's facing comes from Math.atan2(to.x, to.z): atan2 returns radians, RAD2DEG converts to the degrees eulerAngles expects. The argument order depends on which axis the model treats as forward: ours faces +Z, hence (x, z); a model facing +X would need (-z, x). You check that once, by eye.

What you learned

  • The camera is a node: position + lookAt; keep the state in variables (yaw / pitch / radius) and recompute the position instead of spinning the camera.
  • Dragging over a scene is scene.addEventListener("touchstart") + ev.track({ onMove }) — no claim on a scene; measure the offset from the touch point — delta* is between events.
  • Without a Shape aspect a node is invisible to the finger; a node listener is about that object, a scene listener is about every tap, misses included (ev.target === null).
  • camera.getRay(x, y) → Physics.raycast(...) for bodies, or plane.intersectRay(ray) + ray.getPoint(t) for a floor.
  • A smooth camera is a lerp with an FPS-independent fraction, 1 - Math.pow(k, dt) — in an aspect on the camera, ordered after what it follows with static after = [Other].

Reference: Camera · Ray, Plane & Noise · 3D physics & shapes · Pointer events · Node.

Next: Physics — crates start falling and the fox starts walking on her own legs.