sound_fx

Places social-media-style sound effects as non-destructive audio clips on a dedicated timeline level, using a two-phase pipeline: LLM moment detection + deterministic rules-based matching.

Overview

sound_fx runs two phases:

1. Moment detection (Phase 1): a single LLM completion reads your transcript and cut points, returning a structured list of narrative moments (hook, punchline, twist, etc.) with timestamps and energy scores.

2. Rules-based matching (Phase 2): a deterministic Go lookup table maps (moment_type, energy, sfx_style) to specific SFX clips and volumes. No agentic loop, no randomness.

Sound effects are prepared the first time they are needed and reused after that. Where a CC0-licensed recording is available it is used; otherwise the effect is synthesized.

In the editor

Add it from the Library (Audio group), or with + add step and a search for sound_fx, after your cut and your transcript.

Under Inputs, add source, either a file input or a step input, whose length sets the bounds it works within. Add transcript as a step input as well: without it the step falls back to placing effects at the busiest edit points, which is much blunter than placing them where something is actually said. timeline as a step input tells it where your cuts are.

sfx_intensity is the setting to think about: surgical puts one or two effects on the real punchlines, balanced gives four to six, hyped covers everything. sfx_style picks the palette. hook_sfx puts a small sound at the very start if nothing else lands there. Point sfx_dir at a folder of your own sounds to override or extend the built-in ones.

Set Output Level to a level of its own, such as sfx. The effects arrive as their own clips there in the run timeline, leaving your original audio untouched, so anything you dislike can simply be deleted.

Example

- id: sfx
  processor: sound_fx
  input:
    - id: source
      step: recorder
    - id: transcript
      step: transcript
    - id: timeline
      step: main_edit
  output:
    timeline: sfx
  params:
    sfx_intensity: balanced
    sfx_style: tiktok

Inputs

Parameters

Intensity Tiers

Controls which moment types are eligible:

Special Behaviors

• Riser auto-pairing: balanced/hyped automatically places a riser before every punchline/twist
• No-repeat rule: each SFX name used at most once per video
• Hook guarantee: if hook_sfx: true and no moment before 2s, inserts pop_soft at T=0
• Duration scaling: short clips (<20s) capped at 3 SFX; long clips (>45s) get a 1.5× ceiling

No-Transcript Fallback

When no transcript input is wired, Phase 1 is skipped. Phase 2 clusters edit cut points and places transition SFX at the 2–3 most prominent clusters.

SFX Library

30 built-in categories, synthesized or sourced from CC0-licensed audio:

whoosh, whoosh_down, whoosh_short, impact, impact_deep, impact_metal, riser, riser_short, riser_dark, ding, ding_double, coin, pop, pop_soft, glitch, glitch_long, swoosh_left, swoosh_right, zoom_in, zoom_out, rewind, typing, error, success, suspense, crowd_cheer, heartbeat, sparkle, bass_drop, record_scratch

User SFX Override

Point sfx_dir at a directory of .wav/.mp3/.aac files. Files named after a built-in SFX (e.g. whoosh.wav) replace the built-in version; unknown filenames are added as extras for the rules table to use.

Output

EditDecisions on the configured level. Each SFX clip:

• Type: "audio"
• Placement.Position: timeline position from rules-based matching
• Transforms.Volume: energy-scaled volume (0–1 linear)

Notes

• Phase 1 uses Puffin Ship's built-in AI (no API key required). Phase 2 is pure Go with no API calls.
• Original audio is unchanged: SFX clips sit on a separate lane.
• Each sound effect is prepared once and reused afterwards, so the step is fast after its first run.