Skip to main content

Zebroo Sets

Data is passed as Zebroo Sets between workers. There are several ways to then access the Zebroo Set data within a worker.

Record​

Available for workers which receive a list of data as an input from preceding pipeline steps. record references the currently processed data object.

record.display_name = f"{record.first_name} - {record.last_name}"
record #return record

Data​

Available for workers which receive a list of data as an input from preceding pipeline steps. data references the input data of the current batch. That is the complete input only if the line has Batch Size 0 (the default); with Batch Size 1 data holds just the current record. Logic over all records (deduplication, counting) needs Batch Size 0 - see Batches.

all_display_names = []

for rec in data:  # Iterate through all records of the batch
    all_display_names.append(f"{rec.first_name} - {rec.last_name}")

all_display_names  # Return all_display_names

Tip: if data is a list with a single element, here's a shortcut for accessing the first list element: data.attribute1 can be used in place of of data[0].attribute1.

Index​

Available for workers which receive a list of data as an input from preceding pipeline steps. index references the index number of the currently processed record within the current batch (always 0 with Batch Size 1).

Indexing starts at 0.

Type: Integer

record.position = index
record #return record

HTTP Response​

Available for workers that make HTTP/S requests. response references the response content of the request triggered by the worker.

value = response.value

# When receiving headers is enabled on the worker
headers = response.headers

...
project.com/records/start_index={response.last_index + 1}&pagination_size={response.page_size}

See zSYNC Code Fields for the rest of the variables available in code fields (env, instance_env, vars, and more).