Prosody

Word size follows the delivery, so drawn-out words land large and rushed words stay small, and the rhythm of the voice is visible with the sound off

Installation

$ pnpm dlx shadcn@latest add @remocn/caption-prosody

Usage

CaptionProsody sizes every word by how long the speaker spent on each of its letters, measured against the rest of the page. A word held twice as long as the page's typical pace grows to maxScale, one said twice as fast shrinks to minScale, and everything in between scales smoothly. Sizes are fixed when a page starts, so the line never reflows: words then appear at the moment they are spoken, and the word being spoken takes activeColor.

The effect comes straight from the transcript timing, so it is only as expressive as the timestamps are precise — word-level timestamps from whisper.cpp or the OpenAI API work best. The component renders only the text block; see Positioning and Get a transcript.

tsx

import { AbsoluteFill } from "remotion";
import { CaptionProsody } from "@/components/remocn/caption-prosody";
import captions from "./captions.json";
 
export const MyScene = () => (
  <AbsoluteFill
    style={{
      justifyContent: "flex-end",
      alignItems: "center",
      paddingBottom: 108,
      paddingInline: 192,
    }}
  >
    <CaptionProsody captions={captions} />
  </AbsoluteFill>
);

Props11

Prop	Type	Default
captions	Caption[]— The timed transcript, in the `Caption` shape from `@remotion/captions`. Times are relative to the start of the surrounding `Sequence`.
minScale	number0.75 Size of a word said twice as fast as the page's typical pace, relative to `fontSize`.
maxScale	number1.6 Size of a word held twice as long as the page's typical pace, relative to `fontSize`.
combineTokensWithinMilliseconds	number1200 How much speech one page may span before the next word starts a new page. Pages also break at the end of a sentence and on any pause of at least `holdMs` (300 ms minimum).
holdMs	number600 How long a page stays after its last word when the next page is not already due. It fades out at the end of the hold.
fontSize	number64 Base font size in pixels; a word at the typical pace renders at this size.
fontWeight	number800 Font weight. The font itself is inherited.
color	string"#ffffff" Colour of every word that is not being spoken right now.
activeColor	string"#facc15" Colour of the word being spoken.
shadow	booleantrue A tight text shadow that keeps the caption legible over bright footage.
className	string— Classes for the text block — width, font family, `uppercase`.

captionsCaption[]
The timed transcript, in the `Caption` shape from `@remotion/captions`. Times are relative to the start of the surrounding `Sequence`.
Default—
minScalenumber
Size of a word said twice as fast as the page's typical pace, relative to `fontSize`.
Default0.75
maxScalenumber
Size of a word held twice as long as the page's typical pace, relative to `fontSize`.
Default1.6
combineTokensWithinMillisecondsnumber
How much speech one page may span before the next word starts a new page. Pages also break at the end of a sentence and on any pause of at least `holdMs` (300 ms minimum).
Default1200
holdMsnumber
How long a page stays after its last word when the next page is not already due. It fades out at the end of the hold.
Default600
fontSizenumber
Base font size in pixels; a word at the typical pace renders at this size.
Default64
fontWeightnumber
Font weight. The font itself is inherited.
Default800
colorstring
Colour of every word that is not being spoken right now.
Default"#ffffff"
activeColorstring
Colour of the word being spoken.
Default"#facc15"
shadowboolean
A tight text shadow that keeps the caption legible over bright footage.
Defaulttrue
classNamestring
Classes for the text block — width, font family, `uppercase`.
Default—