Compressors
Production databases are usually too large and too sensitive to hand to developers. A Compressor is a scheduled job that takes a production dump and turns it into something smaller and safe to work with — emptied of bulk data, anonymised, and distributed to wherever development instances can pick it up.
Compressors have their own top-level menu.
What a compressor run does
Each run takes an input dump and applies these steps in order. Every step is optional and controlled by a field on the compressor.
- Empty DB (
cleardb) — strips bulk data out of the database. - Truncate lines — runs your own SQL statements against the database, in sequence. See below.
- Anonymize — replaces personal data.
- Remove and regenerate web assets — clears the asset bundles that came with the dump and rebuilds them, so a restored instance does not serve stale CSS and JS.
The result is written out to one or more target volumes.
Choosing the input
There are two ways to tell a compressor what to compress.
| Field | Behaviour |
|---|---|
| Source Volume + Regex | Scans the volume and picks the matching dump. The regex defaults to \.dump\.gz$ |
| Use this dump | Uses one specific dump and ignores the volume and regex entirely |
Use this dump is the iteration lever: when you are tuning truncate rules and do not want the input changing under you, pin a dump and re-run.
A compressor must have either a specific dump, or both a source volume and a regex — it will not save otherwise.
Configuration
| Field | Purpose |
|---|---|
| Branch | The branch whose instance is used to perform the work |
| Anonymize | Run the anonymisation step. Required field — decide deliberately |
| Empty DB | Run cleardb to strip bulk data. Default: off |
| Exclude Tables | Comma-separated list of tables to leave alone |
| Timeout Hours | Abort the run after this long. Default: 24 |
| Settings | Extra runtime settings for the instance doing the work |
| Remove Webassets | Remove and regenerate asset bundles |
| Cronjob | The schedule the compressor runs on |
| Active | Whether the compressor runs at all |
Truncate lines
Truncate lines are your own SQL, run against the database between the
cleardb and anonymise steps. Use them for project-specific bulk data that
cleardb does not know about — a large log table, an integration queue, an
audit trail.
| Field | Purpose |
|---|---|
| Sequence | Order the statements run in |
| Enabled | Skip a line without deleting it |
| Content | The SQL to execute |
| Last Run / Last Duration | Recorded per line, so you can see which statement is slow |
A disabled line is recorded as skipped rather than silently ignored, so the run log reflects what actually happened.
These statements run against a copy of production data with no confirmation step. Test a new truncate line against a pinned dump before putting it on a schedule.
Outputs
A compressor writes its result to one or more outputs, each of which is a filename on a target volume. This is how one compression run can feed several machines — a dev server and a laptop sync target, for example.
Monitoring a compressor
The form records enough to tell whether the job is still doing its job:
| Field | Meaning |
|---|---|
| Date Last Success | When it last completed |
| Last Input Size / Last Output Size | Human-readable, so the ratio is obvious |
| Performance | Computed compression ratio |
| Last Log | What the most recent run did, step by step |
| Logs | Historical run logs |
A compressor whose output size creeps up over releases usually means new bulk tables have appeared that no truncate line covers yet.
Schedule now triggers a run immediately without waiting for the cron.
How the output gets used
Point a repository's Dump for new instance at a compressor output and set New Instance Mode to Restore Dump. Every new branch then starts life with a realistic, anonymised database instead of an empty one.
See Repositories and Instance Lifecycle.
Troubleshooting
The run failed during cleardb. The compressor raises on a non-zero exit and records the output in the log. Check that the branch's instance is healthy first — the work happens inside it.
Anonymisation failed. Same pattern: the log holds the process output. A common cause is a module that the dump expects but the branch does not have installed.
Assets steps failed. Asset removal and regeneration are treated as non-fatal: the failure is appended to the log and the run continues. A restored instance may then need Remove Web Assets run by hand.
Output is barely smaller than input. Either Empty DB is off, or the bulk data lives in tables no truncate line covers. Compare Last Input Size with Last Output Size over several runs to spot the drift.