All work38 / 44
Story Shorts
Pipeline that writes, voices and renders vertical story videos: Claude writes the scenes, edge-tts reads them, Remotion renders.
- Year
- 2026
- Status
- Prototype
- Stack
- Remotion, TypeScript, ffmpeg
A pipeline for vertical story videos: give it a topic, and Claude writes the script and the scenes as Remotion components, edge-tts voices it, ffmpeg mixes the music under the voice, Remotion renders, and a QC step checks the result.
The hard part was timing. The speech engine's word timestamps run up to 135 ms ahead of the actual sound depending on the speaking rate, so the pipeline calibrates them per episode from the silences in the audio.