Purpose and Use Case
The zbsync_ftp module adds a Grabber and Dumper pair for exchanging files with a remote FTP, FTPS, or SFTP server:
| Worker | Technical name | Role |
|---|
| FTP Grabber | zbs.grabber.file.ftp | Lists and downloads files from a remote server, making their content available to the pipeline. |
| FTP Dumper | zbs.dumper.file.ftp | Uploads a record's content to a path on a remote server. |
Both workers share their path-pattern and file-handling fields with the local File Grabber / Dumper (they reuse the same underlying mixin), but connect over the network via an FTP Connection instead of reading/writing the local filesystem directly.
FTP Connection
Before using either worker, create an FTP Connection (Connections → FTP) with:
| Field | Description |
|---|
| Host / Port | Server address. Default port is 21. |
| Type | FTP, SFTP, or FTPS. |
| Username / Password | Login credentials. |
| Private SSH Key | SFTP only — alternative to username/password authentication. |
| Timeout | Connection timeout in seconds (default 10). |
FTP Grabber
Configuration
| Field | Description |
|---|
| FTP Host | The FTP Connection to use. |
| Filepath | Search path for files to fetch. Supports the same wildcard/record-attribute patterns as the File Grabber (e.g. record.filename). |
| Search Subfolders (Recursive) | Recursively lists files in subfolders under the search path. |
| Filter Type | Regex or Wildcard — how Regex Filter / Wildcard Filter below are interpreted. |
| Regex Filter | Regular expression the file path must match. Used when Filter Type is Regex. |
| Wildcard Filter | Glob-style pattern (* expanded to .*) the file path must match. Used when Filter Type is Wildcard. |
| Limit Files | Maximum number of files to fetch per matched path (0 = unlimited). |
| Encoding as text | If set, downloaded file content is decoded to text using this encoding; otherwise content is returned as binary. |
| Action After Read | Move or Delete the file on the FTP server after it has been read (same semantics as the File Grabber). |
| Destination Path | Required when Action After Read is Move. |
How It Works
- Connects using the configured FTP Connection, lists matching files at the resolved path(s), and applies the regex/wildcard filter and file limit.
- Downloads each file to a temporary local path before reading its content — large files above the configured streaming threshold are kept as attachments rather than loaded fully into memory.
- After a successful pipeline commit, if Action After Read is
Move or Delete, the worker performs that action against the original files on the server.
FTP Dumper
Configuration
| Field | Description |
|---|
| FTP Host | The FTP Connection to use. |
| Filepath | Destination path expression, e.g. /tmp/a/b/{record.filename}.csv. Supports {guid} and any key of the incoming record. |
| Encoding as text | Required if the record's content is text (a ValidationError is raised otherwise) — used to encode the text to bytes before upload. |
How It Works
- Evaluates the Filepath expression against the incoming record, creates any missing parent directories on the server, and uploads the content.
Usage and Best Practices
Common Pitfalls
- SFTP without credentials or key: SFTP connections need either a username/password or a Private SSH Key — leaving both blank fails at connect time.
- Text content without an encoding: the FTP Dumper raises a validation error if it receives text content but no Encoding as text is configured.
- Wildcard vs. Regex confusion: make sure Filter Type matches the filter field you actually filled in — a wildcard pattern typed into Regex Filter (or vice versa) will silently match nothing or too much.
- Use Limit Files to cap how many files a single run downloads, especially on first-time syncs against large remote directories.
- Prefer narrow, specific Filepath patterns over recursive searches from a high-level directory to reduce the number of remote listing calls.