Async & Data Fetching Patterns

In Remotion-based projects, handling asynchronous operations and data fetching requires a shift from standard web application patterns. Because video rendering must be deterministic and frame-accurate, all data must be resolved or properly synchronized with the rendering engine to avoid flickering or missing frames.

The following patterns describe how to manage data fetching, media metadata resolution, and asset loading within the Agent Board.

Pre-rendering Data with calculateMetadata

The most robust pattern for data fetching in Remotion is the calculateMetadata function. This function allows you to perform asynchronous tasks—such as fetching API data or reading file headers—before the composition begins rendering.

Fetching Dynamic Props

Use calculateMetadata to resolve props from an external API. This ensures the component receives all necessary data at the moment of mount.

import { Composition, CalculateMetadataFunction } from "remotion";

const calculateMetadata: CalculateMetadataFunction<Props> = async ({ props, abortSignal }) => {
  const response = await fetch(`https://api.example.com/data/${props.id}`, {
    signal: abortSignal, // Handles cancellation if props change in Studio
  });
  const data = await response.json();

  return {
    props: {
      ...props,
      fetchedData: data,
    },
    // You can also dynamically set duration based on the data
    durationInFrames: data.duration * 30, 
  };
};

export const Root = () => (
  <Composition
    id="DynamicComp"
    component={MyComponent}
    durationInFrames={300} // Default placeholder
    fps={30}
    width={1920}
    height={1080}
    calculateMetadata={calculateMetadata}
  />
);

Media Metadata Resolution

When working with video or audio assets, you often need their duration or dimensions to configure your composition. Use the getMediaMetadata pattern (via Mediabunny) within calculateMetadata to ensure the composition perfectly matches the source material.

import { CalculateMetadataFunction } from 'remotion';
import { getMediaMetadata } from './utils/media';

const calculateMetadata: CalculateMetadataFunction<Props> = async ({ props }) => {
  const { durationInSeconds, dimensions } = await getMediaMetadata(props.videoSrc);

  return {
    durationInFrames: Math.ceil(durationInSeconds * 30),
    width: dimensions?.width ?? 1920,
    height: dimensions?.height ?? 1080,
  };
};

Asset Loading Patterns

Local Assets with staticFile()

To ensure assets are correctly resolved across different environments (local development, Lambda, or sub-path deployments), always wrap local paths in staticFile(). This function handles the necessary URL encoding and path resolution.

import { Img, staticFile } from 'remotion';

export const Logo = () => {
  // Path is relative to the 'public/' folder
  return <Img src={staticFile('brand/logo.png')} />;
};

Remote Assets

Remote URLs (HTTPS) can be used directly. Remotion’s core components (<Img>, <Video>, <Audio>) internally handle the "blocking" mechanism, ensuring the asset is buffered/loaded before the frame is captured during the render process.

Async Media Processing

For complex workflows, such as generating filmstrips or validating uploads, use the following utility patterns.

Validation with canDecode

Before attempting to render a composition with a user-provided video, verify the browser's ability to decode the codec.

import { canDecode } from './utils/decode';

const validateSource = async (url: string) => {
  const isCompatible = await canDecode(url);
  if (!isCompatible) {
    console.error("The video format is not supported by the current environment.");
  }
};

Frame Extraction

To extract frames asynchronously (e.g., for thumbnails or AI analysis), use the extractFrames pattern. This pattern uses AbortSignal to prevent memory leaks and unnecessary processing.

import { extractFrames } from './utils/frames';

const generateThumbnail = async (src: string) => {
  const controller = new AbortController();
  
  await extractFrames({
    src,
    timestampsInSeconds: [0], // Extract first frame
    onVideoSample: (sample) => {
      // Process the VideoSample (draw to canvas, etc.)
      console.log("Frame extracted at:", sample.timestamp);
    },
    signal: controller.signal,
  });
};

Critical Constraints

To maintain deterministic rendering, you must adhere to these rules regarding asynchronous logic:

1. No useEffect Fetching: Do not fetch data inside a useEffect hook. By the time the data arrives, Remotion may have already attempted to render several frames, leading to "popping" or empty states in the final video.
2. No Time-Based Async: Avoid setTimeout, setInterval, or requestAnimationFrame for logic that affects the visual state. All "async" visual changes must be mapped to the useCurrentFrame() hook.
3. Handle Abort Signals: When performing fetches in calculateMetadata or utility functions, always pass the abortSignal. This ensures that when a user rapidly changes props in the Remotion Studio, stale network requests are cancelled.
4. Deterministic 3D: When using @remotion/three, ensure all 3D asset loading is completed before the render begins. Do not use standard useFrame() from R3F; instead, drive object properties directly using values derived from useCurrentFrame().