Airflow Xcom Exclusive !!link!! Link

Airflow XCom exclusivity: Overview & key behaviors

• What XComs are: small pickled JSON-like messages Airflow tasks use to exchange data.
• Exclusivity concern: “exclusive” means whether an XCom written by one task run (task instance) can collide with or be overwritten by XComs from other task runs, and how to ensure one writer/reader relationship per logical key.

1. The Standard XCom: Utility and Pitfalls

By default, Airflow tasks push and pull XComs via the metadata database (usually PostgreSQL or MySQL). A simple pattern is:

When not to use exclusive XComs

• When multiple consumers should independently read the same value (e.g., config flags).
• When you need full Airflow-native observability without custom backends and the occasional duplicate processing is acceptable.
• For very large payloads — better to store artifacts externally.

@task def consumer_1(data): # data is automatically pulled only to this task print(data) airflow xcom exclusive

Task B: Pull

def pull_task(**context): pulled = context["ti"].xcom_pull(task_ids="push_task") print(pulled["data"]) Airflow XCom exclusivity: Overview & key behaviors