Any audio source — synth, pattern, mic, effect chain — can be treated as a live signal that drives shaders, draw calls, window properties, or anything else. Three abstractions, one idea: audio is a data bus, not just sound.

source accepts: any Tone instrument/node, Tone.Analyser, Web Audio AnalyserNode, or 'mic' (harness mic — enable with toolbar toggle).

The inverse of the audio signal bus — sample any canvas or camera feed as a live signal to drive audio params, effects, or anything numeric. Camera, viz canvas, layer canvas — all valid sources.

opts: x, y = normalized center of sample region (0–1), radius = region size (0–1), fps = sample rate.
source: 'camera' (toolbar camera), any HTMLCanvasElement (getCanvas(0), viz.canvas, mirror canvas).
Pass an existing video.signal() object as source to onMotion/onBrightness to share one sampling loop across multiple triggers.

// Mouse → shader custom uniform
const ms = sensors.mouse();
const s = new Shader(({ uv, custom }) => {
  const d = length(uv - vec2(custom.x, custom.y));
  return [1.0 - smoothstep(0.0, 0.1, d), uv.y, uv.x, 1.0];
});
ms.stream(m => s.set([m.x, m.y, 0, 0])); s.start();

// Gamepad left stick → draw
sensors.gamepad(0).stream(g => {
  draw.circle(400 + g.axis(0) * 200, 225 + g.axis(1) * 200, 20, 'white');
});

// Device tilt → camera shader (mobile)
const mot = sensors.motion();
const s2 = new Shader(({ uv, col, custom }) => {
  const shift = vec2(custom.x * 0.05, custom.y * 0.05);
  return [col.r, col.g, col.b, 1.0]; // shift uv by tilt
}, { video: cam });
mot.stream(m => s2.set([m.gamma / 90, m.beta / 180, 0, 0])); s2.start();

// iOS motion permission (call from button click)
btn.onclick = async () => { const mot = await sensors.requestMotion(); }

return [r, g, b, a] — use an array literal; automatically becomes vec4f.

Math.* → all map to WGSL equivalents: Math.sin, Math.abs, Math.pow, Math.floor, etc.

vec2(x,y) / vec3(r,g,b) / vec4(r,g,b,a) — vector constructors available globally.

WGSL built-ins available as-is: mix, clamp, smoothstep, step, fract, length, normalize, dot, cross, reflect.

Ternary cond ? a : b — works; compiles to WGSL select(b, a, cond).

if / else, for, while — all supported.

Helper functions — declare inside the arrow fn body; hoisted to WGSL. Type params with default values: r = 0.0 → f32, center = vec2(0,0) → vec2f.

// Generative — no video needed
new Shader(({ uv, time }) => {
  const r = Math.sin(uv.x * 10 + time) * 0.5 + 0.5;
  return [r, uv.y, 0.5, 1.0];
}).start();

// Camera filter — col = pixel at current uv
new Shader(({ uv, col, time }) => {
  const grey = col.r * 0.299 + col.g * 0.587 + col.b * 0.114;
  return [grey, grey * Math.abs(Math.sin(time)), grey, 1.0];
}, { video: cam }).start();

// Audio-reactive — .bind() fills custom.x=rms, .y=bass, .z=mid, .w=high
const s = new Shader(({ uv, time, custom }) => {
  const r = Math.sin(uv.x * 20 * (1.0 + custom.x * 5.0) + time) * 0.5 + 0.5;
  return [r * custom.x, r * 0.3, 1.0 - r, 1.0];
});
s.bind(audio.signal(synth)).start();

// Helper function — default values are type hints
new Shader(({ uv, time }) => {
  function circle(center = vec2(0,0), p = vec2(0,0), r = 0.0) {
    return 1.0 - smoothstep(r - 0.01, r + 0.01, length(p - center));
  }
  const c = circle(vec2(0.5, 0.5), uv, 0.25 + Math.sin(time) * 0.1);
  return [c, c * 0.4, c * 0.9, 1.0];
}).start();

// Apply shader directly to any window
wm.applyShader('win-camera', ({ uv, col, time }) => {
  return [1.0 - col.r, 1.0 - col.g, 1.0 - col.b, 1.0];
});

Beat as state machine: Each pat() callback fires at precise musical times — use it to advance a CA, flip a palette, emit a particle burst. Audio is the clock, not the picture.

Amplitude as physics force: sig.value or sig.bass → feed to gravity, repulsion, blur strength, shader uniform. Read any frame with no setup.

Note value as index: The note string in a callback can select palette index, drawing mode, CA rule — anything discrete.

Shared parameter: The same variable that drives synth detune can drive visual distortion — change one number, both domains shift.