# AI roto: clicks, tracking, job progress and what you get

> How AIRoto turns clicks into tracked mattes, the track toolbar and shortcuts, what the job state line and Details log tell you, the mask options, and what to do with the result (alpha, layers, Cryptomatte, Convert to Roto).

[AIRoto](/docs/nodes/airoto) makes mattes from clicks. Open its panel, then work in the Viewer.

## Clicks

| In the Viewer | Does |
|---|---|
| click | adds a point to the current object (a new object if there is none) |
| Shift + click or Alt + click | adds a "not this" point |
| drag | draws a box around the object |

The point shows at once and the mask follows a moment later (about 0.1 s once the model is loaded). While the panel is open, the frame you are on is prepared in the background, so the first click on a frame is fast too. The first click of a session loads the model (a few seconds).

**Pre-compute while idle** (a checkbox in the Jobs panel, on by default): after a minute without edits, with the GPU free and enough video memory, the AI worker prepares the AIRoto nodes' frames (the current frame, then the next ones) so the first click gives its mask in well under a tenth of a second instead of waiting for the frame to be prepared. It stops the moment you edit or change frame.

## Tracking

The toolbar works like the Tracker's, from the frame the Viewer is on:

| Button | Key (pointer over the Viewer) | Does |
|---|---|---|
| ◀◀ | Z | track backward to the in point |
| ◀\| | X | track one frame backward |
| ■ | | stop; the frames already tracked are kept |
| \|▶ | C | track one frame forward |
| ▶▶ | V | track forward to the out point |
| ▶ 10 | | track the next 10 frames |

- **in / out** limit the track range (empty = the project range). ⇤ / ⇥ set them to the current frame.
- **Fixing a frame:** click on the frame that went wrong, then track from there. Only the frames from that point on are tracked again; every other frame keeps its matte exactly.
- The strip under the toolbar and a line on the timeline show the tracked frames as they finish; green ticks are the frames with clicks.
- **follow** moves the timeline with the frame being tracked.

## What the AI is doing

The line under the toolbar is the job's real state, reported by the AI worker:

| State | Meaning |
|---|---|
| Starting… / Installing… | the AI worker is starting (the first time, it installs its runtime) |
| Waiting for the GPU (held by …) | another heavy GPU job runs; this one starts when it ends |
| Loading model … | the model is loading (the log says cached or cold, and how long) |
| Tracking frame N (k of m) | frames come in; each shows as it finishes |
| Cancelling… | stop was pressed; the worker stops within one frame |
| Cancelled at frame N (k frames kept) | the worker confirmed the stop |
| Tracked N frames in … s | done, with speed and peak GPU memory |
| Error: … | the real error message |

If a stop takes longer than a few seconds, the line says why (for example, the model is still loading).

**Details** (under the panel; it remembers whether it is open) is a timestamped log of the current or last job: its settings, GPU waits, model load, each frame's time, GPU memory, the stop request and its confirmation, warnings and errors. **Copy log** copies it.

**Workspace > Jobs** opens a panel next to Background Renders with one line per background job (the AI nodes AIRoto, AIMatteRefine, AICleanPlate, …); the All / AI / Renders / Imports row filters it. Click a line to open that node's panel.

## Mask options

| Knob | Does |
|---|---|
| grow / shrink | grows (positive) or shrinks (negative) every matte by that many pixels |
| edge smooth | removes jagged and boiling edges without moving the edge |
| feather | softens the edge |
| deduplicate held frames | finds frames that repeat the one before (animation on 2s, held frames) and gives them the same matte instead of tracking them again, so the edge does not boil |
| held frame threshold | how different a frame may be and still count as a repeat |
| masks / opacity (panel) | the tinted masks in the Viewer and how strong they are |

The mask options change the output at once; nothing is tracked again.

## Playback and scrubbing

The tinted masks in the Viewer play and scrub with the picture, like a Roto overlay in Nuke. As
frames are tracked (and when a track ends), every tracked frame's mask is loaded into memory in the
background, so each frame's mask is drawn together with its image, with no wait. A fix on one frame
(new clicks, a re-track) reloads that frame only. Viewing the AIRoto output itself (alpha or a layer)
plays from the Viewer's frame cache like any other node.

## What you get / next steps

| Output | What it is | Use it |
|---|---|---|
| alpha | all enabled objects together (`alpha`: replace, multiply or keep the input alpha) | Premult, Merge, or the mask input of a Grade, Blur, ColorCorrect… |
| `airoto_<name>` | one layer per object, its matte in r, g, b and a | the Viewer's layer menu, or a node's mask channel, to work on one object only |
| `CryptoAIRoto00` | Cryptomatte with an exact ID per object | a Cryptomatte node: Ctrl+click objects to pick them by name |
| **Convert to Roto** (panel button) | a new Roto node on the plate with one animated bezier shape per enabled object, named and coloured like the object | edit the splines like any Roto shape, then refine them with SmartRoto |

**Convert to Roto** fits each object's matte with MaskToSpline (the Roto panel's *Import from mask*),
with the same point count on every frame. Its options sit next to the button: *adaptive keys* (keys
only where the shape changes, within *key tol* pixels) or *key every frame*, the fit tolerance *tol*
in pixels, *points* (the most points per shape, 128 by default: AI mattes have detailed edges), and *smooth* (temporal smoothing in frames; 0 by default, since the mattes are tracked already: raise it to calm a boiling edge, at the cost of accuracy). The new node and its shapes are one undo
step; the state line shows the progress and **Details** logs each shape's points, keys and fit error.
A typical workflow: click, track, check the masks by playing, fix frames, then Convert to Roto and
hand-finish the edges in the Roto node, or keep the AIRoto output as the matte.

## Models

| Model | Use |
|---|---|
| Auto (default) | SAM 3.1 when an object has a text prompt; otherwise SAM 2.1 Large, or EfficientTAM-S when the GPU has little free memory (under 3 GB free, or a card under 8 GB). The Details log says what Auto picked and why. |
| SAM 2.1 Large | best edges |
| EfficientTAM-S | light: about 1.5x faster per frame and a third less GPU memory than SAM 2.1, slightly softer edges |
| EfficientTAM-Ti | lightest; can pick a smaller part of the object from the same clicks |
| EdgeTAM | fast preview |
| SAM 3 / SAM 3.1 | text prompts ("person", "car"); SAM License, accepted in the panel first |

SAM 2.1, EfficientTAM and EdgeTAM are Apache-2.0. See [AI tools and licences](/docs/concepts/ai).


---
Sources: opencomp_core/nodes/ai/airoto.py, opencomp_core/ai/roto.py, opencomp_core/ai/roto_convert.py, opencomp_core/ai/jobstate.py, opencomp_electron/src/components/ai/AIRotoPanel.tsx, opencomp_electron/src/components/ai/AiJobDetails.tsx, opencomp_electron/src/components/ai/AiJobsPanel.tsx, opencomp_ai/ops/sam2_video.py, opencomp_ai/ops/efficienttam_video.py