Error Handling

When a step fails on a record, zSYNC by default fails the batch: the error is stored on the batch, the retry pattern applies, and depending on Abort on Error the pipeline stops. That is still the default. On top of it every step can react to errors per record — and every error, whatever it comes from, is described with the same fields.

The error fields

Every error gets three fields, next to the usual message (str) and the exception type (type):

Field	Meaning	Examples
error_source	where the error comes from	http, network, ftp, sftp, filesystem, sql, odoo, python
error_code	the code of that source	HTTP status 404, SQLSTATE '23505', FTP reply '550', 'ENOENT', 'ValidationError'
error_details	what the other side said	response body, message of the database, FTP reply

Source	Set by	error_code	error_details
http	Web Worker	status code (integer)	response body
network	Web Worker, any socket error	e.g. ConnectTimeout, ECONNREFUSED	message
ftp	FTP Worker	reply code ('550')	server reply
sftp	SFTP connection	e.g. AuthenticationException	message
filesystem	file workers, missing files over SFTP	errno name ('ENOENT', 'EACCES')	message and file name
sql	SQL Worker, SQL errors of any kind	SQLSTATE ('23505' unique, '23503' foreign key, '22012' division by zero)	message of the database, incl. the constraint name
odoo	Odoo errors	ValidationError, UserError, AccessError, ...	message
python	everything else	—	—

You find the fields in the error of a failed batch (Batches tab of the instance) and — with an error action, see below — on the record itself.

The details are cleaned before they are stored: NUL bytes are removed, values that look like credentials (password=…, "client_secret": "…", Bearer …) are masked, and the credentials the request was sent with (Authorization and API key headers, cookies, token parameters) are masked wherever the other side mirrors them back. error_details is cut at 4096 characters.

Reacting to errors

The settings live on the step, tab Error Handling:

• On Record Error
  • Fail batch — default, the behaviour so far.
  • Continue with error — the record goes on to the next step and carries the error under the key ___zbs_error.
  • Skip record — the record is dropped, the batch goes on with the others.
• Error Condition — optional Python expression that decides which errors On Record Error applies to. Available: error (the fields above), record (the input record that failed) and index.
  • truthy: On Record Error applies
  • falsy: the error is treated as if nothing was configured
  • 'fail', 'continue' or 'skip': this action, whatever On Record Error says

Examples:

# only "not known yet" answers of a web service go on
error.error_source == 'http' and error.error_code == 404

# per error: 410 Gone means "deleted" -> drop, 404 -> go on, anything else fails
'skip' if error.error_code == 410 else 'continue' if error.error_code == 404 else 'fail'

# duplicate rows in the target database are fine
error.error_source == 'sql' and error.error_code == '23505'

If a condition does not apply, Skip Record On Error keeps working as before. An explicit 'fail' always fails the batch.

Reading the error in the following steps

A record that went on with an error carries ___zbs_error with error_source, error_code, error_details, str, type, worker, worker_model and line. The key starts with ___, so it is internal: the Odoo dumper ignores it, the Web Worker does not send it, and it is not part of record.keys().

record.get() hands out the stored dictionary, so read the fields with brackets (in the Error Condition, error.error_code works as well):

error = record.get('___zbs_error')
if error and error['error_code'] == 404:
    # order is not known to the service yet - try again tomorrow
    ...

Use the Output Filter of the failing step, or the Input Filter of a later step, to let only good or only failed records through, e.g. not record.get('___zbs_error').

Workers that process a whole batch at once

Some workers do not handle records one by one — the SQL dumper writes the whole batch in one statement, the Odoo dumper creates and updates all records of the batch in one go. If such a worker fails, part of the batch may already be written. So by default the error rules do not apply to these workers: the batch fails as before, whatever is configured.

The SQL dumper is the exception, as long as its connection uses a transaction (Use Transaction, the default): the external transaction is rolled back, so nothing of a failed batch is written. The error rules are then asked for every record of the batch with the same error. Continue passes every input record on with the error — note that the following step then gets the input records, not the dumper's usual create/update result. Use Batch Size 1 if you need the decision per record.

Streaming workers

Workers that hand out their records lazily (e.g. streamed CSV reading) produce them only while the output of the step is being written. An error there is handled by the same rules, per input record:

• the rows read before the error stay in the output,
• Continue with error: the input record (e.g. the file) follows with the error attached, the rest of its rows is not read,
• Skip record (or Skip Record on Error): the rest of its rows is left out,
• otherwise the batch fails as before.

What is never handled per record

These errors always fail the batch, whatever is configured:

• retry signals (Retryable Job Error, waiting for the next time frame)
• serialization conflicts and aborted transactions of the Odoo database — after such an error no further statement would work
• errors in the Error Condition itself — a broken condition must not let records pass or disappear

Errors of external connections (SQL Worker against another database) do not touch the Odoo transaction and can be handled.

:::warning Writes into the Odoo database If an Odoo write fails half way (e.g. a constraint after the record was inserted) and the error is handled per record, what was written so far stays in the transaction. This is the same as with Skip Record On Error. Prefer error rules on steps that talk to other systems. :::

For developers: adding a new error source

A worker that knows more than the bare exception attaches it before raising:

from odoo.addons.zbsync.exceptions import annotate_error

raise annotate_error(
    MyApiError(...),
    source="myapi",
    code=reply.code,
    details=reply.text,
    secrets={self.api_key},  # masked wherever it shows up
)

A worker that processes the whole batch at once may let the error rules handle a failed batch by returning True from _batch_error_rules_apply() — only if a failure leaves nothing behind (see the SQL dumper).

With Continue with error the failed input record goes into the worker's output. A worker with an after_commit hook (e.g. deleting the files it read) must leave such records alone: check self._has_record_error(record), as the file grabbers do.

For exceptions of a library the module uses everywhere, extend zbs.exception.formatter._classify_exception instead (see zbsync_web for requests or zbsync_ftp for ftplib). Attached details win over the classification and are also found along the cause chain, so a worker may re-raise a neutral exception from the original one.