3D & Complex Animations

Integrating 3D with React Three Fiber

Agent Board leverages @remotion/three to bring 3D capabilities into the video timeline. This allows you to use Three.js and React Three Fiber (R3F) while ensuring perfectly synchronized, frame-accurate renders.

Prerequisites

To enable 3D support, ensure the @remotion/three package is installed in your project:

npx remotion add @remotion/three

The ThreeCanvas Component

All 3D content must be wrapped in the <ThreeCanvas> component. Unlike a standard R3F <Canvas>, <ThreeCanvas> is designed to bridge the Remotion timeline with the Three.js renderer.

You must provide explicit width and height props, typically sourced from useVideoConfig().

import { ThreeCanvas } from "@remotion/three";
import { useVideoConfig } from "remotion";

const My3DScene = () => {
  const { width, height } = useVideoConfig();

  return (
    <ThreeCanvas width={width} height={height}>
      <ambientLight intensity={0.5} />
      <pointLight position={[10, 10, 10]} />
      <mesh>
        <boxGeometry args={[1, 1, 1]} />
        <meshStandardMaterial color="royalblue" />
      </mesh>
    </ThreeCanvas>
  );
};

Animation Constraints and Rules

To prevent flickering and ensure deterministic rendering (where every frame looks identical every time it is rendered), follow these strict animation rules:

1. Frame-Driven Animations Only

Animations must be driven by the useCurrentFrame() hook. Using useFrame() from @react-three/fiber is strictly forbidden because it relies on the browser's requestAnimationFrame clock rather than Remotion's frame-stepper.

Correct Implementation:

import { useCurrentFrame } from "remotion";

const RotatingBox = () => {
  const frame = useCurrentFrame();
  const rotation = frame * 0.05;

  return (
    <mesh rotation={[0, rotation, 0]}>
      <boxGeometry />
      <meshStandardMaterial color="orange" />
    </mesh>
  );
};

2. No Self-Animating Shaders or Models

Shaders, physics engines, or GLTF models with internal animation loops must have their internal "time" or "progress" uniform mapped to Remotion's frame count.

3. Sequences inside ThreeCanvas

When using <Sequence> within a <ThreeCanvas>, you must set the layout prop to "none". This prevents Remotion from attempting to wrap 3D objects in <div> containers, which would break the Three.js scene graph.

<ThreeCanvas width={width} height={height}>
  <Sequence from={30} layout="none">
    <mesh>{/* This mesh only appears after frame 30 */}</mesh>
  </Sequence>
</ThreeCanvas>

Complex Animation Patterns

For cinematic movement, avoid linear math. Use Remotion’s built-in interpolation and spring primitives to drive Three.js properties.

Spring Physics

Springs provide natural-feeling motion for camera moves or object entrances.

import { spring, useCurrentFrame, useVideoConfig } from "remotion";

const FallingObject = () => {
  const frame = useCurrentFrame();
  const { fps } = useVideoConfig();

  const drop = spring({
    frame,
    fps,
    config: {
      damping: 12,
      stiffness: 100,
    },
  });

  // Animate Y position from 10 to 0
  const yPos = 10 - drop * 10;

  return (
    <mesh position={[0, yPos, 0]}>
      <sphereGeometry />
    </mesh>
  );
};

Interpolation Curves

Use interpolate to map the current frame to complex values like opacity, scale, or rotation with specific easing functions.

import { interpolate, easing } from "remotion";

const OpacityAnimation = () => {
  const frame = useCurrentFrame();
  
  const opacity = interpolate(
    frame,
    [0, 60], // From frame 0 to 60
    [0, 1],  // From 0% to 100% opacity
    {
      easing: easing.bezier(0.42, 0, 0.58, 1),
      extrapolateRight: "clamp",
    }
  );

  return (
    <meshStandardMaterial transparent opacity={opacity} color="red" />
  );
};

Technical Best Practices

| Rule | Reason | | :--- | :--- | | Avoid CSS Transitions | CSS animations are not synchronized with the video encoder and will cause frames to drop or skip. | | Use staticFile() | When loading textures or 3D models from the public/ folder, always wrap paths in staticFile() to ensure compatibility across different deployment environments. | | Multiply by FPS | When defining durations in seconds, always multiply by the fps value from useVideoConfig() to maintain timing consistency across different composition settings. | | Lighting | Always include at least one <ambientLight /> in your <ThreeCanvas> to ensure objects are visible. |