`analyze_footage`

Analyzes raw video clips using Whisper transcription, silence detection, and adaptive Claude Vision to classify each clip's quality, speaker presence, and suggested usage.

When to use

Use as the first step in an auto-assembly pipeline (see Footage Director guide). The output feeds footage_assembler and assemble_transcript.

In the editor

Add it from the Library (AI group), or with + add step and a search for analyze_footage, as the first step of an auto-assembly template.

Under Inputs, add clips as a file input naming the footage. This one usually points at a whole set of files rather than a single one, since the point of the step is to sift through everything you shot. analysis_mode decides how closely it looks: audio_only listens, keyframe looks at frames spread through the clip, and adaptive (the default) listens first and only looks closely at the parts that seem worth it. Set language if you know it, or leave it empty to let the transcription work it out.

This step needs your own OpenAI key for transcription. Add it in the workspace's Variables & Service API Keys, reachable from the editor's Settings; a run that needs a key you have not set stops and asks for it before it does any work.

Nothing appears on the timeline. What you get is a read on every clip: how good it is, whether someone is speaking, what happens in it, and whether it is better used as your main footage, as b-roll, or not at all. That report is what footage_assembler and assemble_transcript read, so wire this step into them as an input.

YAML Example

- id: analyze_raw
  processor: analyze_footage
  input:
    - id: clips
      file: raw-footage        # points to a glob matching multiple files
  params:
    analysis_mode: adaptive
    language: "en"
  cache: true

Inputs

ID	Source	Description
`clips`	file (glob)	One or more raw video clips to analyze

Params

Param	Type	Default	Description
`analysis_mode`	select: `audio_only`, `keyframe`, `adaptive`	`"adaptive"`	`"audio_only"` (transcript + silence only), `"keyframe"` (sparse frames), `"adaptive"` (audio first, then adds vision for interesting moments)
`frame_interval_sec`	duration (s)	`5`	Interval for sparse frame extraction in keyframe/adaptive mode
`silence_threshold_db`	number	`-40`	dB threshold for silence detection
`silence_min_duration_sec`	duration (s)	`0.5`	Minimum duration to count as a silence
`dense_context_sec`	duration (s)	`0`	Context window around interesting timecodes for adaptive dense sampling
`language`	string	none	ISO 639-1 language code for Whisper. Empty = auto-detect.

Output

Analysis containing FootageAnalysis with per-clip metadata: transcript, silence ranges, visual summary, quality score, key moments, suggested usage (backbone/broll/skip).

Notes

Uses Puffin Ship's built-in AI (no API key required); requires an OPENAI_API_KEY (set in your workspace's Service API Keys) for transcription.
adaptive mode is recommended: it starts with cheap audio analysis and only adds vision for clips with interesting content.
Always use cache: true: analysis is expensive and deterministic.