Put video in your libdragon game. Palette-indexed delta tiles, decoded on the VR4300, with a soundtrack the video clocks itself off. No RDP, no RSP video ucode, no external dependencies.
Built and measured at Elyan Labs while putting a 98-second music video on a cartridge; extracted into a library so you can do it in ten lines.
# 1. encode (needs ffmpeg, numpy, pillow, and libdragon's audioconv64)
python3 tools/fdvenc.py myclip.mp4 assets/clip --fps 12
# -> assets/clip.fdv, assets/clip.wav64, and a CHURN number: read it.
# 2. put both in your DFS, add src/fdv64.c to your build, then:fdv_config_t cfg = fdv_config_default();
cfg.video_path = "rom:/clip.fdv";
cfg.audio_path = "rom:/clip.wav64"; /* MONO. see below. */
fdv_player_t *v; fdv_open(&v, &cfg);
while (!fdv_finished(v)) {
surface_t *fb = display_get();
fdv_poll(v, fb, x, y); /* decodes if due, draws if it did */
display_show(fb); /* you present */
fdv_audio_pump(v); /* you pump */
}
fdv_close(v);That is the whole integration. example/ is a complete ROM doing exactly this.
fdv64 is a guest in your game loop: no global state, no interrupt handlers,
it never calls display_show(), and it touches exactly one mixer channel — the
one you name.
You must have called these before fdv_open():
| call | why |
|---|---|
display_init(...) |
fdv draws into the surface you hand it |
dfs_init(...) |
fdv reads its clip through DragonFS |
audio_init(hz, bufs) |
hz must match your .wav64 |
mixer_init(n) |
fdv uses one channel of it |
timer_init() |
libdragon's mixer needs it |
rspq_init() |
the mixer runs on the RSP. If you call rdpq_init() you already have this; a pure-CPU game does not, and without it the first mixer_poll() halts the RSP |
The one non-obvious thing about this codec, and the thing that decides whether your clip fits:
Grain is what costs, not motion.
FDV1 only stores 8×8 tiles that changed, so a fixed-camera shot should be nearly free. In practice a first encode of real footage usually reports 80–90 % of tiles changing every frame — a delta coder doing nothing. That is almost never the subject moving. It is film grain and compression noise: after quantisation to 256 colours a pixel that wobbles by one level flips its index, and a tile counts as changed if one of its 64 pixels differs.
fdvenc.py prints the churn and tells you what to do about it:
CHURN 85.6% of tiles per frame
^ HIGH. That is almost certainly grain, not motion. Try --denoise heavy
before lowering --fps.
On the reference clip, temporal denoise plus a 10-pixel change threshold took 28.1 KB/frame at 85.6 % churn down to 16.05 KB/frame at 48.8 % — a third of the file from denoising alone. On this codec, denoise is not a quality filter that happens to help. It is the compression. Reach for it before you reach for a lower frame rate: dropping fps costs motion you wanted, denoising costs grain you did not.
Encode your soundtrack mono. fdvenc.py always does; fdv_open() refuses
a stereo .wav64 and returns FDV_ERR_AUDIO_CHANNELS.
This is not a style preference. A 2-channel VADPCM .wav64 halts the RSP
inside the first mixer_poll() with:
RSP ASSERTION FAILED (0xff01) - Invalid overlay
Overlay 0x1 not registered
and the symptom is that your ROM prints its init line, draws one frame, and then goes quiet forever — which looks exactly like a slow video decoder and is not. One bit of the asset header separates working from RSP-down. It cost an afternoon to find, so fdv64 checks the header at open and fails loudly with a message that tells you the fix.
We isolated the variable to the channel count and stopped there; whether the
fault is libdragon's, audioconv64's, or the asset's is not established.
See FINDINGS.md.
From the reference clip — 98 seconds, 136×240 pillarboxed into 320×240, 12 fps, mono 22,050 Hz VADPCM — played end to end under ares:
| frames presented | 1,175 / 1,175, none skipped |
| frame rate (vblank-derived) | 12.04 fps average, 11.41 minimum |
| A/V drift over 98 s | 84 ms — one frame period, the floor of the measurement |
| decode | 5.83 ms/frame |
| blit | 4.20 ms/frame |
| CPU idle | ~88 % at 12 fps |
| size | 16.05 KB/frame video, 19.7 MB ROM |
The example/ ROM (320×240 full screen, 121 frames) re-measures the same
shape: 121/121 presented, 0 skipped, 83 ms drift, 3/3 frame CRCs matching the
host decoder.
The CPU is not the limit. Decode plus blit is ~10 ms of an 83 ms budget. What limits you is cartridge size and framebuffer height.
Bytes per frame scale with area and churn; ROM size scales with that times frame rate times duration. Measured points in bold, the rest scaled from them at ~50 % churn:
| resolution | KB/frame | 10 s @ 12 fps | 60 s @ 12 fps | 60 s @ 15 fps | 60 s @ 24 fps |
|---|---|---|---|---|---|
| 136×240 (portrait, pillarboxed) | 16.0 | 1.9 MB | 11.3 MB | 14.1 MB | 22.5 MB |
| 160×120 | 8.0 | 0.9 MB | 5.6 MB | 7.0 MB | 11.3 MB |
| 256×192 | 21.0 | 2.5 MB | 14.8 MB | 18.5 MB | 29.6 MB |
| 320×240 (full screen) | 34.6 | 4.2 MB | 24.4 MB | 30.5 MB | 48.8 MB |
Pick a target ROM size, then trade resolution against frame rate. A 64 MB flashcart takes a full-screen minute at 24 fps; 32 MB takes a full-screen minute at 15.
python3 tools/check_fdv.py assets/clip.fdv --png out_%d.png # see the frames
cd example && make check # ROM vs hostcheck_fdv.py is an independent decoder — it shares no code with
src/fdv64.c on purpose. It hashes the decoded frame, not the file, so a
player that reads bytes correctly and decodes them wrongly fails it. Have your
ROM print FDVCRC frame=<n> fnv=<hex> from fdv_frame_crc() and
FDVSYNC frames=<n> fps=<n> samples=<n> hz=<n>, and check_fdv.py grades both
the pixels and the sync.
src/fdv64.h the API, and the integration contract in its comments
src/fdv64.c the player (~340 lines)
tools/fdvenc.py encoder: video -> .fdv + mono .wav64, prints the churn
tools/check_fdv.py independent host decoder + receipt grader
tools/run_rom.py headless ares runner used by `make check`
example/ a complete ROM, buildable with `make N64_INST=/path/to/libdragon`
FORMAT.md the FDV1 wire format
FINDINGS.md what was measured, what was inherited, what is still unknown
cd example
make N64_INST=/path/to/libdragon # the dir containing n64.mk
make assets SRC_VIDEO=/path/to/your.mp4 # optional: re-encodeThe Makefile checks that N64_INST actually contains an n64.mk and says so
loudly if not, because a stale N64_INST in the environment is the most
common way this fails.
The bundled clip is the Elyan Labs logo animation — Scott's own work, freely shareable, included so a stranger can build and run this without sourcing a video first.
MPL-2.0 — chosen so this is genuinely usable in your game, including a closed-source commercial one.
In plain terms, and this is the question every ROM developer asks first:
- Your game stays yours. MPL is file-scoped, not project-scoped. Including
fdv64.h, compilingfdv64.c, and statically linking it into your ROM does not make your game MPL. Your own source files stay under whatever licence you like, closed or commercial. - Static linking is explicitly fine. MPL-2.0 §3.2 covers distributing the larger work under your own terms, and unlike LGPL it asks nothing about relinkability — which is what makes it workable for a cartridge image where there is no such thing as a shared library.
- If you change fdv64's own files, publish those files' source under MPL-2.0. Fix a bug in the decoder, improve the blit, extend the format — those changes come back. Anything you write in your own files does not.
So: use it, ship it, sell it. Just don't fork the codec privately.
Full text in LICENSE.