Skip to content

Media catalog

Every video and photo the app can show is registered in one place: apps/mobile/src/features/media/catalog.ts. Adding a family memory is one entry there plus one reference from the feature that uses it.

Every asset belongs to a senior

Files live in a numbered folder per person, and each catalog entry carries the matching ownerId:

apps/mobile/assets/
  1/photos/  1/videos/     Tom
  2/photos/  2/videos/     Nyna

The number is the senior's id in src/features/seniors/roster.ts. Screens read through videosFor(seniorId) and photosFor(seniorId) rather than the whole catalog, so one senior never sees another's memories. See Multiple seniors on one device.

Adding a video

  1. Drop the file in apps/mobile/assets/<seniorId>/videos/, or get a public URL for it. Bundled files ship in the binary (fine for short clips); URLs stream (better for anything large). A streaming URL must serve raw video bytes with HTTP range support. Google Drive share and uc?export=download links do neither (every media request gets redirected to a one-time URL), so the native players show a black stage; a catalog test rejects them. Bundle the file or serve it from the backend instead.
  2. Add an entry to videoAssets:
{
  id: 'summer-picnic',
  kind: 'video',
  ownerId: '1',
  source: require('@/assets/1/videos/summer-picnic.mp4'),
  title: 'The picnic at the lake',
  description: 'A family spreads a checked blanket under a tree...',
  size: { width: 1280, height: 720 },
  credit: 'Summer 1994, camcorder',
}

size is the video as displayed. Read it with ffprobe -v error -select_streams v:0 -show_entries stream=width,height,sample_aspect_ratio -of csv=p=0 file.mp4, and widen the width by the sample aspect ratio when it is not 1:1. tests/video-shape.test.ts reads the file's own header and fails when the two disagree.

The description is read aloud as the audio description before playback, so write it for someone who cannot see the video. Write the require path out in full: Metro resolves it at build time, so a path assembled from a senior id at runtime does not bundle.

  1. Add a quiz in src/features/video-quiz/questions.ts, pulling the clip and its audio description from the catalog. Its seniorId must match the clip's ownerId, which tests/media-catalog.test.ts checks:
{
  id: 'vid-summer-picnic',
  seniorId: '1',
  videoUri: requireVideo('summer-picnic').source,
  audioDescription: { text: requireVideo('summer-picnic').description },
  captions: [ /* ... */ ],
  question: '...',
  // ...
}

Captions stay with the quiz because they are timed to that clip. A quiz without both an audio description and captions is not publishable and the screen refuses to play it (canPublish, src/features/video-quiz/validation.ts).

A video memory

A Stories memory can name a video by videoId. Tom's skydiving memory was the first: it names skydiving (assets/1/videos/skydiving.mp4, re-encoded at 720p with the audio track dropped). A memory's video plays muted and on a loop, so the memory's song stays the one sound.

A memory may name a video before the file exists. Until that id is in videoAssets, the memory shows without a video in a senior's build, and development builds show a note in its place asking for the file in the repo's inputs/ folder. Register the file as above to finish it.

A landscape video (wider than 4:3, like 16:9) is small on an upright phone. While one shows, the app unlocks rotation and shows "Turn your phone sideways to see it bigger." Turned sideways, the video fills the screen and a tap shows only Restart and Next. Everywhere else the app stays locked upright. A portrait (9:16) or near-square video fits upright and asks for nothing. The shape comes from the catalog's size (videoShape).

Adding a photo

  1. Drop the file in apps/mobile/assets/<seniorId>/photos/, or get a public URL.
  2. Add an entry to photoAssets with that ownerId and the same required description, plus an optional caption shown visibly under the photo.
  3. Set mediaId on a question in src/features/stories/questions.ts:
{ id: 'q-wedding', mediaId: 'wedding-1959', prompt: 'Where were you married?', ... }

The Stories screen shows the photo above the prompt, using the description as its accessible label.

The rules

  • A description is required. validateMediaAsset rejects a blank one, and canShow is checked before anything renders, so an undescribed photo is never displayed rather than displayed unlabeled. This is the same Born Accessible gate the video quiz applies.
  • Ids are unique across the whole catalog, not just within one senior. validateCatalog reports duplicates.
  • Every asset has an owner. A senior only ever sees their own media, and a quiz's seniorId must match its clip's ownerId.
  • requireVideo throws on a typo. Quizzes are authored at module load, so a bad id fails immediately in development and in tests instead of rendering an empty player.
  • tests/media-catalog.test.ts enforces all of the above on the shipped catalog, including that every registered video is actually used by a quiz.
  • docs/multiple-seniors.md — how ownerId scopes the catalog to one person.
  • docs/video-quiz.md — the Born Accessible video quiz feature.
  • docs/photo-questions.md — a separate, deferred EXIF-driven photo quiz. That feature would read location and date from a photo's own metadata; the catalog here is about photos attached to authored Stories questions, and the two can coexist.
  • docs/accessibility.md — the project-wide accessibility requirements.