Puffin Ship Docs

Rating runs and model training

Puffin Ship makes editorial judgements for you: which take to keep, which clip belongs where, how a script should read. Rating those judgements is how they get better. This page covers what rating does, and how to turn the training use off if you would rather not take part.

Rating a finished run

When a run finishes, the run view offers How was this run? Give it one to five stars, add a note if something specific was good or bad, and save. You can skip it; it will not ask again for that run.

Notes are worth writing. A one-star rating says something went wrong; a note says what.

What ratings are used for

Ratings do two jobs.

The first is yours: they are a record of which attempt was good, which is useful when a run group has half a dozen attempts in it.

The second is ours. Rated runs may be used to train and improve the models behind Puffin Ship's editorial decisions, so the parts you rate highly become more likely and the parts you rate poorly become less so.

The run rating has an Include in training data checkbox, ticked by default. Untick it to rate a run for your own reference without contributing it.

Turning training use off

The workspace-wide control is in Workspace Settings → General → Model Training: Help improve Puffin Ship's models. It is on by default and only the workspace owner can change it.

Turning it off stops any future use of that workspace's rated runs. Content that has already been used in training cannot be removed from a model that was already trained on it, which is why the setting stops future use rather than promising erasure.

The Privacy Policy is the authoritative statement of what is collected and how it is used.

Note the two controls are different in scope: the checkbox on a rating covers that one run, the workspace setting covers everything in the workspace from that point on.

Comparing model output

Some runs are executed twice, once by the main model and once by an alternative, so the two can be compared. When a run has a comparison available, a Compare button appears in the run's timeline view, with a count of the steps where the two disagreed.

The comparison shows each step's decisions side by side, highlighting the differences, and lets you say which side you preferred. That preference is a rating like any other, and it is the most useful signal we get.

Next