Camera & picking
Step 3 of the 3D track. The scene is built, the fox runs — but you can only look at all of it from wherever you parked the camera, and there is nothing to touch. This step is about the two things that turn a picture into an app: the finger drives the camera, and the finger hits objects.
Both come down to the same facts: the camera is an ordinary node, and 3D touch events work exactly like the UI ones, with a single new idea — the ray.
Orbiting the scene with a finger
The classic orbit camera: it sits on a sphere around a target, and the finger turns two angles — yaw (around the vertical) and pitch (height above the horizon). Catch the drag with touchstart + ev.track() — the same API as dragging in the UI track:
const scene = new Scene({ skybox: "#1b2233" })
scene.add(Light.sun({ direction: [-1, -2, 0.7], intensity: 90000, shadowsQuality: 2 }))
scene.add(Mesh.plane({ material: Material.lit({ color: "#2f4a35", roughness: 1 }), normal: [0, 1, 0], scale: 24, receiveShadows: true }))
const wood = Material.lit({ color: "#b5854b", roughness: 0.8 })
scene.add(
Mesh.box({ material: wood, size: 0.8, position: [0.6, 0.4, 0.4], castShadows: true }),
Mesh.box({ material: wood, size: 0.55, position: [0.6, 1.075, 0.4], eulerAngles: [0, 30, 0], castShadows: true }),
Mesh.box({ material: wood, size: 0.6, position: [-0.7, 0.3, 1.1], eulerAngles: [0, -20, 0], castShadows: true }),
)
// Orbit: the camera sits on a sphere around the target. yaw turns around the vertical,
// pitch is the height above the horizon, radius is the distance. Angles here are radians — plain math
const target = [0, 0.5, 0]
let yaw = 0.6, pitch = 0.5, radius = 6
function placeCamera() {
scene.camera.position = [
target[0] + radius * Math.cos(pitch) * Math.sin(yaw),
target[1] + radius * Math.sin(pitch),
target[2] + radius * Math.cos(pitch) * Math.cos(yaw),
]
scene.camera.lookAt(target)
}
placeCamera()
// A finger lands on the scene: capture the angles at touch-down and add the finger's offset to them
scene.addEventListener("touchstart", ev => {
const yaw0 = yaw, pitch0 = pitch
ev.track({
onMove: p => {
yaw = yaw0 - (p.clientX - ev.clientX) * 0.01
pitch = Mathf.clamp(pitch0 + (p.clientY - ev.clientY) * 0.01, 0.1, 1.4)
placeCamera()
},
})
})
// Zoom in / out — plain HUD buttons that change radius
const zoom = (label: string, step: number) =>
UIButton(UIText(label).style({ color: "#fff", fontSize: 18, fontWeight: 700 }))
.style({ bgColor: "#ffffff22", width: 40, height: 40, borderRadius: 12, alignItems: "center", justifyContent: "center", $pressed: { opacity: 0.6 } })
.onClick(() => { radius = Mathf.clamp(radius + step, 3, 12); placeCamera() })
const hud = UIWidget(
UIText("Drag to orbit").style({ fontSize: 13, color: "#c7ccd6" }),
UIRow(zoom("−", 1), zoom("+", -1)).style({ gap: 8 }),
).style({ top: "comfort-top", left: 16, gap: 10 })
scene.widgets(hud)
scene.open()- The camera is a node, not a director.
scene.camera.positionandlookAt(point)are all you need; there is no "camera controller" in the SDK and none is wanted. Set the position yourself — from an orbit, from the hero's position, from anything. - Compute the position, don't spin the camera. The temptation to write
camera.eulerAngles.y += …ends in drift and gimbal lock. Keep the state in variables (yaw / pitch / radius) and recompute the position from scratch every time — easier to debug, and it saves to disk in one line. - A drag is
touchstart+ev.track(), exactly as in UI. The angles captured at touch-down (yaw0,pitch0) plus the offset from the touch point (p.clientX - ev.clientX) — don't confuse that withp.deltaX, which is the shift between consecutive events, not since the gesture began. - A scene has no one to fight over the finger. Every pointer the UI over it didn't take is the scene's, so its
track()needs noclaim(that option is for drags inside scrolling UI, where a list wants the same finger) — the HUD buttons keep their taps, the rest of the screen orbits. Mathf.clamp(pitch, 0.1, 1.4)keeps the camera from diving under the floor or flipping at the zenith — a clamp is almost always what you want over a "free" camera.- 3D touches arrive in logical px — the same coordinates as
ev.clientXin UI. That matters one section down.
There's a lens too: camera.fov (vertical angle in degrees, 60 by default), camera.near / camera.far for the view range. Full list in the Camera reference.
Hitting an object with a finger
A mesh on its own is transparent to the finger: the engine draws it but doesn't know where its bounds are. To make an object tappable it needs a Shape aspect — a body for hit-testing. {} fits a box straight to the mesh:
const scene = new Scene({ skybox: "#1b2233" })
scene.add(Light.sun({ direction: [-1, -2, 0.7], intensity: 90000, shadowsQuality: 2 }))
scene.add(Mesh.plane({ material: Material.lit({ color: "#2f4a35", roughness: 1 }), normal: [0, 1, 0], scale: 24, receiveShadows: true }))
scene.camera.position = [0, 3.5, 5]
scene.camera.lookAt([0, 0.4, 0])
const wood = Material.lit({ color: "#b5854b", roughness: 0.8 })
const picked = Material.lit({ color: "#FF4032", roughness: 0.8 })
// Shape is what makes a node tangible to the finger — without it a tap goes straight through the mesh.
// {} — a box the size of the mesh itself
const crates: Mesh[] = [[-1.4, 0.3], [0.2, -0.6], [1.5, 0.5]].map(([x, z], i) =>
Mesh.box({ material: wood, size: 0.8, position: [x, 0.4, z], eulerAngles: [0, x * 25, 0], castShadows: true, name: `crate ${i + 1}` })
.aspect(Shape, {}),
)
scene.add(...crates)
const label = UIText("Tap a crate").style({ fontSize: 13, color: "#c7ccd6" })
const hud = UIWidget(label).style({ top: "comfort-top", left: 16, right: 16 })
scene.widgets(hud)
scene.open()
// A listener on the node — when the reaction belongs to the object itself
const taps = crates.map(() => 0)
crates.forEach((crate, i) => crate.addEventListener("click", () => taps[i]++))
// A listener on the scene sees every tap: ev.target is the Shape-carrying node under the finger, or null.
// Both fire: the node first, then the scene
let current: Mesh | null = null
scene.addEventListener("click", ev => {
current?.setMaterial(wood)
current = ev.target as Mesh | null
current?.setMaterial(picked)
label.text = current ? `picked ${current.name} · taps on it: ${taps[crates.indexOf(current)]}` : "missed — selection cleared"
})- No
Shape, no hits. This is the number-one reason "the click doesn't work": the node draws, but it has no pick body, so the tap falls through to the scene withev.target === null. - Two places to listen.
node.addEventListener("click", …)— the reaction belongs to the object;scene.addEventListener("click", …)— sees every tap, misses included, and knows what was hit (ev.target). A miss is only visible to the scene — that's the whole difference. - A bare
Shapefollowsposition, not the simulation. Its pick body is static: a write tonode.positionmoves it along, but motion applied any other way (a tween, physics) leaves it behind. An object the simulation moves needs a real body — next step. {}is a box from the mesh, but you can be precise:{ sphere: 0.4 },{ capsule: { halfHeight: 0.5, radius: 0.3 } },{ box: [0.4, 0.4, 0.4] }. Boxes take half-extents:box: [0.5, 0.5, 0.5]around a 1×1×1 cube. And they're world units — the node'sscaledoesn't touch them.raycast: falseinShaperemoves a node from picking while keeping its physical behavior — that's how decoration that shouldn't steal taps is marked.touchstartworks on nodes just the same and canev.track()— that's "grab an object and drag it", ready-made.
The ray and the plane
A tap gives a point on the screen, and the world is three-dimensional: one screen point corresponds to a whole ray running into the scene. camera.getRay(x, y) builds that ray, and from there you have two ways to ask what it hit: Physics.raycast against bodies, Plane.intersectRay against a mathematical plane (a floor, a wall, a table) that has neither a mesh nor a body:
const scene = new Scene({ skybox: "#1b2233" })
scene.add(Light.sun({ direction: [-1, -2, 0.7], intensity: 90000, shadowsQuality: 2 }))
scene.add(Mesh.plane({ material: Material.lit({ color: "#2f4a35", roughness: 1 }), normal: [0, 1, 0], scale: 24, receiveShadows: true }))
scene.camera.position = [0, 4.5, 6]
scene.camera.lookAt([0, 0, -0.5])
const wood = Material.lit({ color: "#b5854b", roughness: 0.8 })
const label = UIText("Tap the ground to place a crate, tap a crate to remove it").style({ fontSize: 13, color: "#c7ccd6" })
const hud = UIWidget(label).style({ top: "comfort-top", left: 16, right: 16 })
scene.widgets(hud)
scene.open()
// The floor is a mathematical plane at y = 0 (no body, no mesh): intersect the camera ray with it
const ground = new Plane([0, 1, 0], [0, 0, 0])
let count = 0
scene.addEventListener("click", ev => {
// A ray from the camera through a screen point — logical px, same as ev.clientX / ev.clientY
const ray = scene.camera.getRay(ev.clientX, ev.clientY)
// 1. Ask physics: did the ray hit an existing crate? Remove it
const hit = Physics.raycast(ray.origin, ray.dir, 100)
if (hit?.node) {
scene.remove(hit.node)
hit.node.destroy()
label.text = `crates: ${--count}`
return
}
// 2. Otherwise intersect the floor: t is the parameter along the ray, getPoint(t) gives the point
const t = ground.intersectRay(ray)
if (t === null) return
const p = ray.getPoint(t)
scene.add(
Mesh.box({ material: wood, size: 0.6, position: [p.x, 0.3, p.z], eulerAngles: [0, Math.random() * 90, 0], castShadows: true })
.aspect(Shape, {}),
)
label.text = `crates: ${++count}`
})The order inside the handler isn't accidental: first "did we hit an object?", and only if not, "where on the floor?". That way one tap does different things depending on what's under the finger — the standard shape of "place / select / remove".
Planeis math, not a scene object.new Plane([0, 1, 0], [0, 0, 0])is an infinite plane through the origin with its normal up. It draws nothing, collides with nothing and costs nothing: perfect for "where on the floor did they point".intersectRayreturnst, not a point.tis the parameter along the ray;ray.getPoint(t)gives the point.nullmeans "the ray is parallel to the plane or pointing away from it".Physics.raycast(origin, dir, maxDist)finds the closest body with aShape(bare pick bodies included) and returns{ node, point, normal, fraction }ornull.normalis the surface normal at the hit point: that's how decals are placed, objects aligned to a wall, projectiles bounced.ev.targetversusraycast— the same hit by two routes. The event is handier for "something got tapped"; an explicit raycast is for rays that don't come from a tap (the hero's gaze, an aim, "can the enemy see the player?") or when you needpointandnormal.destroy()is the only way to remove a node for good; the node is unusable afterwards.scene.remove(node)only takes it out of the scene.
raycast's distance limit is the third argument (1000 by default). fraction is measured against it, so the hit distance is fraction × maxDist × |dir|; normalize dir if you want meters.
A camera that follows
A camera rigidly bolted to the hero jitters and makes people seasick. The trick that fixes it in one line is smooth chasing: every frame the camera moves not to the desired point but a fraction of the way there:
const scene = new Scene({ skybox: "#1b2233" })
scene.add(Light.sun({ direction: [-1, -2, 0.7], intensity: 90000, shadowsQuality: 2 }))
scene.add(Mesh.plane({ material: Material.lit({ color: "#2f4a35", roughness: 1 }), normal: [0, 1, 0], scale: 40, receiveShadows: true }))
const wood = Material.lit({ color: "#b5854b", roughness: 0.8 })
for (let i = 0; i < 8; i++) {
const a = (i / 8) * Mathf.TAU
scene.add(Mesh.box({ material: wood, size: 0.6, position: [Math.cos(a) * 5, 0.3, Math.sin(a) * 5], eulerAngles: [0, i * 20, 0], castShadows: true }))
}
scene.camera.position = [0, 4, 8]
scene.camera.lookAt([0, 0, 0])
const follow = signal(true)
const chip = UIButton(UIText(() => follow.value ? "camera: following" : "camera: parked").style({ color: "#fff", fontSize: 13, fontWeight: 700 }))
.style({ bgColor: "#ffffff22", borderRadius: 999, px: 12, py: 7, $pressed: { opacity: 0.6 } })
.onClick(() => follow.value = !follow.value)
const hud = UIWidget(chip).style({ top: "comfort-top", left: 16 })
scene.widgets(hud)
scene.open()
// The fox runs a circle of radius 3 — her own aspect
class Run extends Aspect<"run"> {
t = 0
update(dt: number) {
this.t += dt * 0.7
this.node.position = [Math.cos(this.t) * 3, 0, Math.sin(this.t) * 3]
this.node.eulerAngles = [0, -this.t * RAD2DEG, 0]
}
}
// The camera's aspect: the desired point is behind the fox and above her. The camera doesn't jump there,
// it chases — a fraction of the way per frame that doesn't depend on FPS. It runs after Run, every frame
class Chase extends Aspect<"chase"> {
static after = [Run]
target: Node | null = null
update(dt: number) {
if (!follow.value || !this.target) return
const behind = this.target.position.add(this.target.forward.scale(3)).add([0, 2, 0])
this.node.position = this.node.position.lerp(behind, 1 - Math.pow(0.02, dt))
this.node.lookAt(this.target.position.add([0, 0.4, 0]))
}
}
async function main() {
const fox = await Model.load("https://le.codes/guide-assets/fox-anim.glb")
fox.castShadows = true
scene.add(fox.aspect(Run))
fox.anim.playLoop("run")
scene.camera.aspect(Chase, { target: fox })
}
main()1 - Math.pow(k, dt)is the frame-rate-correctlerp. A naivelerp(target, 0.1)every frame moves at different speeds at 60 and 120 FPS; in this formkis the fraction of the remaining distance left after one second (0.02 ≈ "close the gap 50× per second"), and the result looks the same on any hardware.- The position chases, the gaze doesn't.
lookAtis computed from the already-smoothed position — that's enough; smoothing the look-at point too is only worth it for slow cinematic cameras. - Vectors are values.
fox.position.add([...])returns a newVec3and leaves the original alone — which is why computing "position plus offset" inline is so convenient. A node only changes when you assign back (node.position = …) or set a scalar (node.x = …). - "Behind" comes from the fox's facing. Every node has
forward, its −Z axis in world space. Our fox model looks along its own +Z, so behind her isposition + forward × 3; for a model that looks along −Z it'sposition − forward × 3. Check which way a model faces once, by eye. - Two aspects, one order.
Runmoves the fox,Chasereads where she is:static after = [Run]puts the camera after her in every frame, so it never trails a frame behind. Both useupdate(dt)— the late phase, right before the frame draws. - The same trick is the basis for "camera in a car", "over-the-shoulder camera" and "camera that eases back to its default once released".
The whole screen
Putting it together: the yard, the fox, a finger orbit, tap the ground for the fox's destination, tap a crate to select it. Meanwhile the camera softly keeps the fox centered — orbiting and following at the same time:
// The yard + the fox: drag to orbit around her, tap the ground and she runs there, tap a crate to highlight it.
const scene = new Scene({ skybox: "#1b2233" })
scene.add(Light.sun({ direction: [-1, -2, 0.7], intensity: 90000, shadowsQuality: 2 }))
const lit = (color: string, roughness = 1) => Material.lit({ color, roughness })
scene.add(Mesh.plane({ material: lit("#2f4a35"), normal: [0, 1, 0], scale: 30, receiveShadows: true }))
function Tree(x: number, z: number, height = 1.6) {
const tree = new Node()
tree.position = [x, 0, z]
const crown = Geometry.cylinder({ radiusTop: 0, radiusBottom: 0.55, edges: 8, smooth: false })
.scale(1, height, 1).translate(0, height / 2 + 0.6, 0)
tree.add(
Mesh.cylinder({ material: lit("#6b4a2b"), radius: 0.12, scale: [1, 0.7, 1], position: [0, 0.35, 0], castShadows: true }),
Mesh.from(crown, { material: lit("#3d8b4f"), castShadows: true }),
)
return tree
}
const wood = lit("#b5854b", 0.8), picked = lit("#FF4032", 0.8)
const Crate = (x: number, z: number) =>
Mesh.box({ material: wood, size: 0.8, position: [x, 0.4, z], eulerAngles: [0, x * 30, 0], castShadows: true }).aspect(Shape, {})
scene.add(Tree(-3.2, -1.4, 2), Tree(3, -2, 1.6), Tree(-2.6, 2.6, 1.3), Tree(3.2, 2.2, 2.2), Crate(2, 1), Crate(-2.5, -0.8), Crate(1, -2.5))
const hint = UIText("Loading the fox…").style({ fontSize: 13, color: "#c7ccd6" })
const hud = UIWidget(hint).style({ top: "comfort-top", left: 16, right: 16 })
scene.widgets(hud)
// The fox's aspect: run to the destination a tap on the ground set, then rest
let goal: Vec3 | null = null
class GoTo extends Aspect<"goTo", Model> {
running = false
update(dt: number) {
if (!goal) return
const to = goal.sub(this.node.position)
if (to.length() < 0.15) { goal = null; this.running = false; this.node.anim.playLoop("idle", { fade: 0.2 }); return }
if (!this.running) { this.running = true; this.node.anim.playLoop("run", { fade: 0.15 }) }
this.node.eulerAngles = [0, Math.atan2(to.x, to.z) * RAD2DEG, 0] // the model faces along its own +Z
this.node.position = this.node.position.add(to.normalize().scale(3 * dt))
}
}
// The camera's aspect: an orbit around a target that eases along after the fox — after she moved
class Orbit extends Aspect<"orbit"> {
static after = [GoTo]
yaw = 0.6
pitch = 0.5
target = Vec3.from([0, 0.5, 0])
follow: Node | null = null
update(dt: number) {
if (this.follow) this.target = this.target.lerp(this.follow.position.add([0, 0.5, 0]), 1 - Math.pow(0.01, dt))
this.node.position = this.target.add([7 * Math.cos(this.pitch) * Math.sin(this.yaw), 7 * Math.sin(this.pitch), 7 * Math.cos(this.pitch) * Math.cos(this.yaw)])
this.node.lookAt(this.target)
}
}
const camera = scene.camera.aspect(Orbit)
scene.addEventListener("touchstart", ev => {
const yaw0 = camera.orbit.yaw, pitch0 = camera.orbit.pitch
ev.track({ onMove: p => {
camera.orbit.yaw = yaw0 - (p.clientX - ev.clientX) * 0.01
camera.orbit.pitch = Mathf.clamp(pitch0 + (p.clientY - ev.clientY) * 0.01, 0.15, 1.4)
} })
})
scene.open()
// A tap: on a crate — select it, on the ground — a destination for the fox
const ground = new Plane([0, 1, 0], [0, 0, 0])
let selected: Mesh | null = null
scene.addEventListener("click", ev => {
if (ev.target) { selected?.setMaterial(wood); selected = ev.target as Mesh; selected.setMaterial(picked); return }
const ray = scene.camera.getRay(ev.clientX, ev.clientY)
const t = ground.intersectRay(ray)
if (t !== null) { const p = ray.getPoint(t); goal = Vec3.from([p.x, 0, p.z]) }
})
async function main() {
const fox = await Model.load("https://le.codes/guide-assets/fox-anim.glb")
fox.castShadows = true
fox.eulerAngles = [0, 90, 0]
scene.add(fox.aspect(GoTo))
fox.anim.playLoop("idle")
camera.orbit.follow = fox
hint.text = "Tap the ground and the fox runs. Drag to orbit"
}
main()Three independent pieces of state — the orbit angles, the camera target and the fox's destination — each kept where it's used: the camera's Orbit holds the angles and the target, the fox's GoTo runs her to the destination, and static after puts the camera after the fox in every frame. The finger only writes camera.orbit.yaw / pitch — an aspect's fields are reachable through its accessor (camera.orbit), named after the class's first generic. That's how 3D scenes grow: not one big update function, but small behaviors on the nodes they move.
The fox's facing comes from Math.atan2(to.x, to.z): atan2 returns radians, RAD2DEG converts to the degrees eulerAngles expects. The argument order depends on which axis the model treats as forward: ours faces +Z, hence (x, z); a model facing +X would need (-z, x). You check that once, by eye.
What you learned
- The camera is a node:
position+lookAt; keep the state in variables (yaw / pitch / radius) and recompute the position instead of spinning the camera. - Dragging over a scene is
scene.addEventListener("touchstart")+ev.track({ onMove })— noclaimon a scene; measure the offset from the touch point —delta*is between events. - Without a
Shapeaspect a node is invisible to the finger; a node listener is about that object, a scene listener is about every tap, misses included (ev.target === null). camera.getRay(x, y)→Physics.raycast(...)for bodies, orplane.intersectRay(ray)+ray.getPoint(t)for a floor.- A smooth camera is a
lerpwith an FPS-independent fraction,1 - Math.pow(k, dt)— in an aspect on the camera, ordered after what it follows withstatic after = [Other].
Reference: Camera · Ray, Plane & Noise · 3D physics & shapes · Pointer events · Node.
Next: Physics — crates start falling and the fox starts walking on her own legs.