# Localization

> Local copies of files that live on a slow network share: policies, preferences, the Cache menu and the Read node's bar.

Reading plates over the network can be slow and uneven. Localization keeps a copy of the files a
comp reads on a local disk and reads that copy instead. It works
the same way as in Nuke: the same preferences, the same knob on the Read node, the same Cache menu
and the same coloured bar on the node.

The comp is rescanned for Reads to copy only when a Read's file, range or policy, or the current frame, changes, so an idle comp costs nothing.

The copying happens in the background. It never makes the interface or a render wait: until a
frame's copy is there, the Read reads the source file.

## Turn it on for a Read

Every Read (and DeepRead, and GeoImport for Alembic files) has a **localization policy** knob under
its file name:

| Policy | What happens |
|---|---|
| on | The files are copied, wherever they are. |
| from auto-localize path | The files are copied when they are under one of the *auto-localize from* folders in the Preferences. This is the default. |
| on demand | The files are copied only when you click **Update** on the node (or use Force Update). |
| off | The files are never copied. |

Only the frames in the Read's frame range are copied, starting at the current frame and working
outwards (image sequences before movie files). The **Update** button, on the same row as the policy menu, copies the
files now, and copies again any file whose source has changed.

New Reads take their policy from Preferences › Performance › Localization › localization policy.
The policy is saved in the comp and in exported `.nk` scripts as Nuke's `localizationPolicy` knob,
and `.nk` scripts that use it (or the older `cacheLocal` knob) load with it.

## The bar on the node

A bar at the top of the Read's thumbnail shows where its files are:

- **amber**, filling from the left: copying (a movie file shows no bar until it is complete);
- **green**: every frame in the range is local and up to date;
- **red**: a source file changed after it was copied;
- **striped red**: out of date, and the Read is reading the source files until you update it.
  *hide out of date progress bar* in the Preferences hides this one.

The colours are theme colours (`--oc-graph-loc-*`).

## When a source file changes

A local copy counts as up to date while its size and modification time match the source's. Each
file is checked again after *check for updated files every* minutes (30 by default), or at once with
Cache › Localization › Force Update.

- With *read source files when localized files are out of date* on (the default here), a Read with
  any out-of-date frame reads all its frames from the source files until they are copied again.
  Turned off, it keeps reading the old copies.
- With mode **on**, Reads whose policy is *on* or *from auto-localize path* are copied again by
  themselves. *On demand* Reads stay red until you click **Update**.
- With mode **manual**, nothing is copied by itself; out-of-date files are shown red and you update
  them with **Update** or Force Update.

## Preferences › Performance › Localization

| Preference | What it does |
|---|---|
| mode | **on**: copies Reads set to *on* or *from auto-localize path* by itself and checks every localized file for changes. **manual**: checks for changes but only copies when you ask. **off**: nothing is copied and every Read reads its source files. The status bar shows the mode at the bottom right. |
| read source files when localized files are out of date | See above. |
| hide out of date progress bar | Hides the striped bar of Reads reading their source files. |
| pause localization on script/project open | Opening a comp pauses localization, so it opens faster. Resume it with Cache › Localization › Pause. |
| localization policy | The policy new Reads get. |
| auto-localize from | The folders whose files are copied by *from auto-localize path* Reads, for example `/mnt/jobs`. Separate several folders with `;` (Nuke takes one). Empty: those Reads are not copied. |
| localize to | Where the copies go. Empty uses `/cache/<app>/localized` when the machine has a writable `/cache` disk, else `localized` in your user cache folder (`~/.cache/<app>/localized`). A file keeps its full path inside it, with the leading `/` turned into `_` (as in Nuke): `/mnt/jobs/a.exr` becomes `<localize to>/_mnt/jobs/a.exr`. |
| limit to (GB) | The most space the copies may use; 0 means no limit, and a negative number keeps that much disk space free. When a new copy would go over it, the copies no open comp uses are deleted, least recently used first. If that is not enough, localization pauses and a *Failed to Localize File* dialog says so. The dialog shows how much is in use and free; free space is measured on the disk that holds the *localize to* folder (or its nearest existing parent folder). |
| check for updated files every (minutes) | How long a copy is trusted before its source is checked again. |
| never check for updated files | Never checks by itself (Force Update still does). |
| limit copy speed (MB/s) | Keeps the copying from filling the network link (ours; 0 = no limit). |

## Cache › Localization

- **Pause**: stops copying (and starts it again). The status bar shows *(paused)*. Force Update still works while paused.
- **Force Update › All / Selected / On Demand Only**: checks the sources now and copies what is missing or out of date, for every Read, the selected ones, or the Reads set to *on demand*.
- **Clear Unused Local Files...**: lists the local files no Read in the open comp uses. **Continue** deletes them, **Cancel** keeps them.

Copying stops while the Viewer plays, so playback has the disk and network to itself, and goes on
when it stops.

## How the copying works

- One background thread in the backend copies one file at a time.
- An interrupted copy (Pause, playback, quitting) is kept as a partial file and continues from where
  it stopped, as long as the source has not changed.
- A finished copy is checked by size and gets the source's modification time.
- The source files are only ever read, never changed.
- Each Read's set of files has a manifest in `<localize to>/.manifests/`, so the copies are known
  again in the next session.

## Rendering from the command line and the farm

`render.sh` and the LAN farm workers read the source files by default. Add `--localize` (Nuke's
`--use-local-file-cache` works too) to read the local copies that match their source instead:

```
./render.sh --localize -F 1001-1100 shot.nk
./render.sh --worker --localize
```

Nothing is copied during a render; a copy that no longer matches its source is skipped and the
source is read.

## Differences from Nuke

- *auto-localize from* takes several folders.
- *limit copy speed* is ours.
- Least recently used copies that no open comp needs are removed when the limit is reached; Nuke
  only asks you to clear space.
- Nuke's `NUKE_LOCALIZATION_NUMWATCHERS` (threads that check files) has no counterpart: one thread
  checks and copies.


---
Sources: opencomp_core/localization/engine.py, opencomp_core/localization/prefs.py, opencomp_core/localization/knobs.py, opencomp_server/localization_commands.py, opencomp_electron/src/components/localization/LocalizationPrefs.tsx, opencomp_electron/src/components/Toolbar.tsx, opencomp_electron/src/components/graph/renderer.ts, opencomp_server/render_cli.py, opencomp_server/farm_worker.py