Test Runs
A Test Run is one execution of a branch's tests against a specific commit. Test runs live under the Test Runs menu (Developer role and above) and also appear on the branch they belong to.
zCICD can run three kinds of test in the same run — unit tests, robot tests and migration tests — and it is deliberate about not running tests it does not need to.
How work is skipped
This is the part of zCICD that saves the most time, and it is worth understanding before you configure anything.
zCICD does not re-run every test on every commit. For each Odoo module it asks
zodoo for a hash covering that module and its dependencies (odoo list-deps).
Before running a module's tests it looks for a previous test run — anywhere in
the same repository — that already passed with the identical combination of:
- module
- dependency hash
- test tags
- Python version
If such a run exists, the line is marked Reused and nothing is executed.
Two consequences follow from this:
- Changing one module does not invalidate unrelated modules, because their dependency hashes have not moved. Zebroo's experience is that this removes up to 95% of a test run's work compared with a pipeline that always tests everything.
- Reuse is shared across branches in the same repository. A module already proven green on another branch at the same hash will not be tested again.
To force a full run, tick No Reuse on the test run. Every line then executes regardless of history.
Reuse keys on the dependency hash, not on which files the commit touched. Adding a dependency to a module changes its hash and correctly re-tests it, even if the module's own code is untouched.
Test run states
| State | Meaning |
|---|---|
| Ready To Test | Created, waiting to start |
| Running | Executing |
| Success | Everything passed |
| Failed | At least one line failed, or the run was aborted |
| Omitted | Superseded — see below |
Individual lines have their own state: Open, Running, Success, Failed or Abort. The run's Done Rate and Success Rate are computed from its lines.
When test runs are created
A branch with Test at new commit enabled gets a test run automatically each time a new commit arrives.
If several test runs pile up for one branch and the branch does not have Test at new commit set, zCICD keeps the newest and marks the rest Omitted, so the queue does not fill with stale runs.
The three test types
Each type is configured as a test setting on the branch (or inherited from the repository). A branch can hold several settings of the same type — for example two unit-test settings with different tag filters — and each produces its own lines.
Unit tests
Odoo's own test framework.
| Setting | Default | Purpose |
|---|---|---|
| Filter to tags | at_install/{module},post_install/{module},standard/{module} | Which Odoo test tags to run. {module} is substituted per module |
| Regex | .* | Only test modules whose name matches |
| Skip Odoo Enterprise and OCA modules | on | Do not test third-party code |
| Pre-Calculate Hashes | off | Compute all module hashes up front, so skipping is decided before any container starts |
| Worker Batch | 8 | Lines bundled into one background job |
Each module runs in an isolated environment: containers are brought down with
their volumes, Postgres is started fresh, the database is reset, and only then is
the module installed and tested. Queue jobs and cron are switched off, and
SERVER_WIDE_MODULES is reduced to base,web so the run stays minimal.
Robot tests
Browser-driven acceptance tests.
| Setting | Default | Purpose |
|---|---|---|
| Filter to tags | (empty) | Restrict which robot tests run |
| Regex | .* | Restrict by name |
| In Parallel | 1,2,5,10,20,50 | Comma-separated thread counts. One separate test run is started per number |
In Parallel is a load-testing feature, not a speed setting. The default runs
the suite six times over — single-threaded, then with 2, 5, 10, 20 and 50
parallel threads — to find out where the system breaks under concurrency. If you
only want a functional check, set it to 1.
Unlike unit tests, robot tests run with queue jobs and cron enabled, because they exercise the system as a user would.
Migration tests
Verifies that an existing database survives the update.
The only specific setting is Dump — the database to migrate. Leave it empty to use the branch's normal source. The dump must live on the same machine as the test run.
Settings shared by all three
| Setting | Default | Purpose |
|---|---|---|
| Timeout Seconds | 3600 | Per-line time limit |
| Retries | 3 | Attempts before a line is considered failed |
| Machine | (from repository) | Where the tests run |
| Python Version | 3.10.14 | Part of the reuse key — changing it re-tests everything |
| Use Snapshots | off | Use filesystem snapshots to speed up environment setup |
| Settings | — | Extra runtime settings appended to the instance configuration |
Settings cascade downwards: what is configured on the repository is applied to its branches, and a branch's settings are copied onto each test run it creates. Editing a test run's settings therefore affects only that run.
Running and managing a run
| Action | Effect |
|---|---|
| Rerun | Deletes the run's queue jobs and lines, resets it to Ready To Test, and lets the scheduler pick it up again |
| Abort | Marks pending queue jobs done, sets open lines to Abort, and fails the run |
| Force Success | Set on an individual line. The line counts as passed even though it failed |
Force Success is the escape hatch for a known-bad test that should not block a release. It is recorded on the line, so it stays visible rather than silently disappearing.
A test run containing a forced-success line still reports overall success. Use it deliberately, and prefer fixing or excluding the test via tags where you can.
Watching a run
- The Test Runs list shows state, done rate and duration.
- Opening a run shows its lines grouped by type, each with its own log, exception info, duration and try count.
- Test TV on the branch (Developer role) shows tests executing live.
- Each line records the machine it ran on and the path to its log file.
Troubleshooting
A run failed during preparation. If zCICD could not check out the commit or reload the source, the whole run fails before any test executes, and the test setting is flagged Preparation failed. The error is posted to the run's message log. This is usually a source or settings problem, not a test failure.
Everything is Reused and nothing ran. That is the intended behaviour when nothing relevant changed. If you need proof, set No Reuse and rerun.
Lines fail intermittently. Check Try Count on the line — a line that failed all its retries is a real failure; one that passed on retry may indicate a timing-sensitive test. Raise Timeout Seconds if lines are being cut off.