RUStart

Pointer events & gestures

Taps, clicks, long presses and drags on UI elements, 3D nodes and 2D sprites arrive as the same events: click, touchstart (its ev.track() follows a drag), a button's long press and the mouse hover. Keyboard keys, mouse buttons and gamepads are Input; a plain button tap is UIButton.onClick (interactive elements).

  • drag an element around the screen with the finger (ev.track)
  • tap a 3D object or a 2D sprite in a scene (addEventListener("click"))
  • know that a tap in a scene hit nothing
  • press and hold an item, then drag it (onLongPress)
  • swipe a row sideways inside a scrolling list (claim)
  • highlight a button or show a hint under the mouse cursor (onMouseEnter)
  • read where the finger is: the screen point, the finger, the 2D world point

Platforms: every host; a drag's claim works by direction on iOS and Android only (on the web any claim stops the scroll both ways; desktop and the web player ignore it); the hover needs a mouse (desktop, the web, the web player), a finger never hovers.

At a glance

TypeScript
let left = 40, top = 40                               // the card's place on the screen, logical px

const card = UIButton(UIText("Drag me").style({ color: "white" }))
  .style({ position: "absolute", left, top, width: 140, height: 60, bgColor: "#4f7cff", borderRadius: 12 })
  .onTouchStart(ev => {
    const fromLeft = left, fromTop = top
    const place = () => card.style({ left, top })
    ev.track({
      claim: true,                                    // the drag is ours: a scroll around it does not take it
      onMove: p => { left = fromLeft + p.clientX - ev.clientX; top = fromTop + p.clientY - ev.clientY; place() },
      onEnd: () => console.log("dropped at", left, top),
      onCancel: () => { left = fromLeft; top = fromTop; place() },   // taken away: back where it was
    })
  })

UIScreen(card).style({ bgColor: "#101114" }).open()

What can be tapped: buttons, screens, 3D nodes, 2D sprites

Every surface delivers the same two events — click (pointer-up over the target) and touchstart (pointer-down, where a drag starts) — but each becomes a target in its own way:

Surface How a target becomes hittable Listen via
3D Node has a Shape aspect (a pick body) node.addEventListener('click' | 'touchstart', cb)
3D Scene the open scene (target = the hit node or null) scene.addEventListener(…)
2D Node2D has a Shape2D + a body to hit-test (Physics2D, Trigger2D, CharacterController2D) node.addEventListener(…)
2D Scene2D Scene2D.active (target = the hit node or null) scene.addEventListener(…), or scene.pick(worldPoint) for manual hit-tests
UI UIButton; UIScreen and UIWidget for touch-down only button: .onClick / .onTouchStart / .onLongPress / .onMouseEnter; screen, widget: .onTouchStart

To make a card, a row or an icon tappable, wrap it in a UIButton: no other element has onClick.

Tap a 3D object or a sprite in a scene: addEventListener("click")

To know which 3D object or sprite the user tapped — to select it, open it, pick it up — listen for click on the node; it is hit only with a Shape (3D) or a 2D body (Scene2D § pointer events). The scene's listeners hear every tap, with ev.target the hit node or null when the tap hit nothing:

TypeScript
const scene = new Scene()
const crate = Mesh.box({ name: "crate", material: Material.lit({ color: "#4f7cff" }) })
  .aspect(Shape, {})                                   // the Shape makes the node tappable
scene.add(Light.sun(), crate)
scene.camera.position = [0, 0, 4]
scene.camera.lookAt([0, 0, 0])
scene.open()

crate.addEventListener("click", ev => console.log("tapped", ev.target.name, "at", ev.clientX, ev.clientY))
scene.addEventListener("click", ev => {
  if (ev.target === null) console.log("tapped empty space")
})

The hit node's own listener runs first. Every click then reaches the scene's listeners; a touchstart reaches them only while nobody has called ev.track() yet — the node, or the first scene listener, that tracks takes the gesture. A tap inside a loaded model reaches the model's node.

Note

a scene's click fires at every release, at the release point, whatever the track did: on iOS, Android, desktop and the web player a drag also ends in a click on whatever is under the finger; on the web a press that moved more than 10 px does not click.

Note

on the web the hit is the drawn mesh's box, so a node with no Shape is hit there too — give it a Shape anyway: iOS, Android and desktop hit only shapes. In headless tests (lecodes render / lecodes test) no 3D node is hit: the scene's listeners get target null.

Tap position and target: ClickEvent, TouchStartEvent, LongPressEvent

TypeScript
ev.type                   // "click" | "touchstart" | "longpress"
ev.clientX, ev.clientY    // pointer position in logical px (screen space)
ev.pointerId              // stable id for this finger
ev.target                 // the hit Node / Node2D (null: nothing hit, scene listeners); the element in UI
ev.worldX, ev.worldY      // 2D scenes only: the hit point in world space (undefined in 3D)

The three events carry the same fields; touchstart and longpress also have track() (below). A 3D scene's touches are in the same logical units as the UI's on every host (points on iOS, dp on Android, CSS px on the web) — the units of camera.displaySize and camera.getRay.

Drag an element with the finger: ev.track

To drag, call ev.track() in a touchstart (or a long-press) listener: every later move of that finger goes to the handler until it lifts. There is no drag and drop between apps or from the desktop — a drag is always inside the app, followed this way.

TypeScript
ev.track({
  onMove(pos)   { },   // pos: { clientX, clientY, deltaX, deltaY }
  onEnd(pos)    { },   // finger lifted normally
  onCancel()    { },   // gesture taken away (system gesture, teardown) — always clean up here too
  claim: 'pan-y',      // see below
})

The At a glance card is the complete form: keep where the drag started, move by pos.clientX - ev.clientX, put things back in onCancel.

Note

move an element that stays where it is dropped with left / top (position: "absolute"), not a transform: desktop, the web player and headless tests find what a finger touches by the laid-out box, so a translated element is grabbed at its old place. A transform suits what springs back when the finger lifts (the swipe below).

Note

deltaX / deltaY are per-move deltas — movement since the previous move — on iOS, Android, desktop and the web player, but on the web they are measured from the touch-down. The offset from where the finger went down, pos.clientX - ev.clientX, reads the same on every host.

Swipe or drag inside a scrolling list: claim

A row swiped sideways in a list (swipe to delete), a slider or a joystick inside a UIScrollable or a UIVirtualizedList competes with the list's own scroll, which can take the gesture away — the track then gets onCancel. The track's claim says which movement is yours, so the row keeps its sideways swipe while an up-and-down drag still scrolls the list:

TypeScript
claim: true         // take the pointer unconditionally (free 2D drag: joystick, canvas painting)
claim: 'pan-x'      // claim horizontal movement, let vertical start a scroll
claim: 'pan-y'      // claim vertical movement (bottom sheets)
claim: 'pan-up' | 'pan-down' | 'pan-left' | 'pan-right'   // one direction only

A message row that is deleted by a swipe to the left:

TypeScript
const row = (label: string) => {
  const el: UIButton = UIButton(UIText(label))
    .style({ height: 56, px: 16, justifyContent: "flex-start", bgColor: "white" })
    .onTouchStart(ev => {
      const slide = (dx: number) => el.style({ transform: `translate(${dx}px, 0px)` })
      ev.track({
        claim: "pan-x",                              // sideways is the row's, up and down scroll the list
        onMove: p => slide(p.clientX - ev.clientX),
        onEnd: p => { if (p.clientX - ev.clientX < -120) console.log("delete", label); slide(0) },
        onCancel: () => slide(0),
      })
    })
  return el
}

UIScreen(UIScrollable(Array.from({ length: 30 }, (_, i) => row(`Message ${i + 1}`))).style({ flex: 1 })).open()
Note

the claim works by direction on iOS and Android. On the web any claim keeps the browser from scrolling for the whole gesture, so a vertical drag that starts on a pan-x row does not scroll the list. The desktop host and the web player ignore the claim: a drag inside a scrollable goes to the scroll once it passes the scroll threshold, the track gets no more moves, then onEnd at the release — which is why the row decides at onEnd, from the release point.

Press and hold, then drag: onLongPress

UIButton.onLongPress fires when a finger is held still on the button for 0.5 s (Android: the system's long-press timeout). A handled long press swallows the click that would follow the release, so a tap and a hold do different things on one button. Its LongPressEvent can track() like a touchstart, so a hold flows straight into a drag:

TypeScript
let left = 40, top = 40

const tile = UIButton(UIText("Hold, then drag").style({ color: "white" }))
  .style({ position: "absolute", left, top, width: 160, height: 60, bgColor: "#2a9d8f", borderRadius: 12,
           $pressed: { opacity: 0.7 } })
  .onClick(() => console.log("opened"))                // a short tap
  .onLongPress(ev => {                                  // held: no click follows
    console.log("picked up")
    device.vibrate("medium")
    const fromLeft = left, fromTop = top
    const place = () => tile.style({ left, top })
    ev.track({
      claim: true,
      onMove: p => { left = fromLeft + p.clientX - ev.clientX; top = fromTop + p.clientY - ev.clientY; place() },
      onEnd: () => console.log("moved to", left, top),
      onCancel: () => { left = fromLeft; top = fromTop; place() },
    })
  })

UIScreen(tile).open()

Mouse hover: onMouseEnter

UIButton.onMouseEnter fires when a mouse cursor with no button down enters the button (desktop, the web, the web player; a finger never hovers). Its MouseEnterEvent carries type ("mouseenter"), clientX, clientY, target — no pointerId, a cursor is one. There is no leave event: the hover is one span, and ev.track({ onMove?, onEnd?, onCancel? }) follows it — onMove while the cursor moves inside (delivered only when present), onEnd when it leaves (the last position), onCancel when the element went away underneath. No claim: a hover competes with nothing. The highlight itself is the reserved $hovered class (style classes), which needs no listener; the query is isHovered.

TypeScript
const hint = signal("")

const save = UIButton(UIText("Save").style({ color: "white" }))
  .style({ height: 44, px: 20, bgColor: "#4f7cff", borderRadius: 8, $hovered: { bgColor: "#6b93ff" } })
  .onClick(() => console.log("saved"))
  .onMouseEnter(ev => {
    hint.value = "Saves the draft"
    ev.track({ onEnd: () => { hint.value = "" } })      // the cursor left the button
  })

UIScreen(UIColumn(save, UIText(() => hint.value).style({ color: "#888" })).style({ gap: 8, padding: 40 })).open()

Pitfalls

TypeScript
// ✗ reading deltaX as the distance dragged — on iOS, Android and desktop it is one move's step
onMove: p => knob.style({ transform: `translate(${p.deltaX}px, 0)` })   // jitters near zero
// ✓ the offset from the touch-down, the same on every host
onMove: p => knob.style({ transform: `translate(${p.clientX - ev.clientX}px, 0)` })

// ✗ cleanup only in onEnd — a claimed system gesture skips it
ev.track({ onEnd: stop })
// ✓ onCancel fires instead of onEnd when the gesture is taken away
ev.track({ onEnd: stop, onCancel: stop })

A button dragged from its onTouchStart that also has onClick clicks at the end of every drag released over it — on every host but the web, where a press that moved more than 10 px does not click. For an item that opens on a tap and moves on a drag, drag it from onLongPress: the hold swallows the click.

See also

← BackInputNext →fetch · WebSocket