Projector Scenes for Apple TV

No. 015 & 014 · tvOS app · started Nov 2026in use

Why is there a projector pointed at the foyer wall?

The projector went up for Everett’s birthday. It hangs upside down from a stand on the upstairs balcony, corner-pinned to the wall above the dining room opening, with a mask cut around the French door trim. Once all that was done, turning it off for eleven months a year seemed wasteful.

So it got scenes: a night sky, aurora, the Earth, a planetarium, an aquarium, the holidays. The first version was a web app on a laptop. This one is a native Apple TV app, so the laptop can stay closed.

Version one: a browser on a laptop

Each scene is a small definition: draw a frame, list some options, map up to nine number keys to actions. The setup panel and a phone remote build themselves from that. A WebGL pass adds bloom, grain and a vignette, dithers the dark gradients so they don’t band, and applies color gains tuned for cream paint.

The planetarium uses astronomy-engine for 3,200 stars down to magnitude 5.6, the planets, and the Moon’s real phase. One preset opens on Austin at 8 PM on December 9, 2022, facing southeast: a nearly full Moon, Mars at opposition, Jupiter and Saturn up.

It worked. It also meant a laptop on the hall table every night, and browser fullscreen on macOS claims a whole Space, which took the laptop’s own screen with it. I wrote a small Swift app with a borderless window that covered only the projector. Once there was Swift in the project anyway, the browser started to look like the wrong host.

Version two: an Apple TV

tvOS has no web view, so nothing could be wrapped. Everything was ported. What made that bearable is Canvas.swift, a canvas-2D-shaped API on top of Metal. The web drawing code ports almost line by line. One final render pass does the corner pin, bloom, gains, masks and dither.

Setup happens on a phone. The Apple TV runs a small web server on port 8080 and serves its own setup page: drag the four corners, tune the gains, pick a scene. A second page, /play, turns phones into gamepads over a WebSocket. A Cloudflare Worker relays them when guests are on cellular instead of the Wi-Fi. The TV only ever connects out, and a phone can only steer.

It also runs the room. The Hue cans go through the bridge’s local API at about ten changes a second, and the 600-light Twinkly string on the stair rail gets raw frames over UDP at 25 fps. Nothing goes through anyone’s cloud. When a family checks in at the party, the cans run a yellow runway chase from the front door to the wall.

What fought back

  • Seasick footage. Some hand-held stock clips swayed more than 100 pixels on the wall, and looping them forward and back made it worse. A script measures the drift with sub-pixel frame matching and counter-shifts the crop. Fifteen clips are now within 10 px.
  • The night sky at 66 ms a frame. On the Apple TV’s A12, per-pixel stars took 51 ms of that. They became a GPU buffer of point stars sorted by angle. It holds 60 fps now, and the same change went into the aurora, Earth and solar system scenes.
  • Abyssal Garden. This is a port of Noniv’s MIT-licensed WebGPU underwater world. The A12 is limited by triangles, not pixels, so it got distance culling, a seabed drawn to 170 m instead of 240, and an automatic resolution loop. The loop’s step-up rule kept bouncing, because queued frames finish in a burst right after each change and look fast. The brown blocks in open water were a light-shaft distance overflowing a half float. It averages 60 fps at 960 × 528.
  • The setup page died overnight. tvOS takes back a suspended app’s sockets. The server now reopens and rechecks its port when the app wakes up.
  • Andromeda went black. The media list was cached for six hours, and new images went up inside that window. It refetches on wake and on any cache miss now.

Numbers

Wall 218 × 120 in
Scenes 18 on the web, 22 on the Apple TV
Swift 93 files, about 19,500 lines, 58 commits in ten days
Largest image Hubble’s 665-megapixel Triangulum, in 217 tiles
Footage about 400 MB, pulled from R2 by manifest, so new clips don’t need an app update
Phones up to 12 per party code, 6 player colors

Small things

  • Shake a DualSense like a snow globe in the Christmas scene and the snow swirls.
  • The DualSense’s right trigger stiffens as the fireworks finale builds.
  • Every sound effect is generated in code. There are no audio files.
  • “Red-eye flight” turns the room deep blue, with a red wingtip light by the door and a green one by the wall.
  • The New Year’s scene starts its countdown in the last minute of December 31 without being asked.

Next

Test the helicopter game on the real wall before the 12th. Leave the Twinkly firmware alone until after the 12th. Then: a sunset schedule, and a sky that matches the actual weather outside.

Animated scenes for a 218-inch foyer wall, ported from a laptop browser to a native Apple TV app that also runs the room's lights. Status: in use.

Open the next build?