Data Lineage Specification Template (OpenLineage)

Data Lineage Specification: [Organisation or platform name]

Specification for capturing, storing and using data lineage across a data platform: the questions lineage must answer, capture scope and granularity, the OpenLineage event model (specification 2-0-2) with required facets, dataset and job naming rules, producers and integration points, the lineage store and its consumers (impact analysis, incident triage, catalogue, privacy and audit), quality checks for lineage itself, and rollout. Replace every [bracketed] value and every row marked Example. Worked examples use Corvid Commerce, a fictional SaaS retailer with Airflow orchestration, Spark and dbt on an Apache Iceberg lakehouse and a Snowflake warehouse for finance.

Document control

Approvals

Revision history

Contents

• 1. Purpose and scope
• 2. Lineage use cases and questions
• 3. Capture scope and granularity
• 4. Event model
• 5. Required facets
• 6. Naming conventions
• 7. Producers and integration points
• 8. Lineage store and architecture
• 9. Consumers and access
• 10. Lineage quality and monitoring
• 11. Security and privacy
• 12. Rollout plan
• 13. Glossary and references
• Standard and licence note

1. Purpose and scope

[Two to four sentences on the problem and the outcome.]

Example: When the Orders data product published wrong totals in [month], it took two days to find which dashboards and finance extracts had read the bad partition. Corvid Commerce will capture lineage at run time from the orchestrator and every engine that writes shared datasets, so that the owner of a broken dataset can list affected consumers within minutes, and governance can show where personal data flows.

2. Lineage use cases and questions

3. Capture scope and granularity

Rules:

1. Run-time lineage wins over manual lineage for the same edge. Manual edges show a "manual" marker and the date they were last confirmed.
2. Every shared dataset in the catalogue has at least one producing job in the lineage graph, or an approved exception.
3. Development and test runs are emitted to a separate namespace and are not shown to consumers.

4. Event model

OpenLineage defines three kinds of event:

Core entities and identity:

Run event types in specification 2-0-2 are START, RUNNING, COMPLETE, ABORT, FAIL and OTHER. A run normally sends START and then exactly one of COMPLETE, FAIL or ABORT.

4.1 Example run event

Example: the dbt model that builds the Orders mart completes. Values are illustrative.

{
  "eventType": "COMPLETE",
  "eventTime": "2026-10-30T06:12:44.120Z",
  "producer": "https://github.com/example-org/corvid-lineage-producer/v1.4.0",
  "schemaURL": "https://openlineage.io/spec/2-0-2/OpenLineage.json#/$defs/RunEvent",
  "run": {
    "runId": "0192d5a8-7c3e-7b21-9f40-3a1c2e5d8b77",
    "facets": {
      "parent": {
        "_producer": "https://github.com/example-org/corvid-lineage-producer/v1.4.0",
        "_schemaURL": "https://openlineage.io/spec/facets/1-2-0/ParentRunFacet.json#/$defs/ParentRunFacet",
        "run": {"runId": "0192d5a1-0b2f-7e55-8c10-4f7a9d3e2c11"},
        "job": {"namespace": "airflow-prod", "name": "orders_daily.dbt_build"}
      }
    }
  },
  "job": {
    "namespace": "dbt-prod",
    "name": "sales.marts.fct_orders"
  },
  "inputs": [
    {"namespace": "iceberg://lake-prod", "name": "sales.staging.stg_orders"},
    {"namespace": "iceberg://lake-prod", "name": "sales.staging.stg_fx_rates"}
  ],
  "outputs": [
    {"namespace": "iceberg://lake-prod", "name": "sales.marts.fct_orders"}
  ]
}

The facet _schemaURL versions above are illustrative; record the versions your client library actually emits in section 5.

5. Required facets

Every facet carries _producer and _schemaURL. This platform requires the following:

5.1 Column-level lineage transformation types

The column lineage facet records, for each output field, the input fields and how they were used. The specification distinguishes:

A masking flag marks inputs that were hashed or otherwise obfuscated on the way to the output; LUC-004 uses it to show where personal data is pseudonymised.

Example: fct_orders.order_total_eur has a DIRECT TRANSFORMATION from stg_orders.order_total and stg_fx_rates.rate, and an INDIRECT JOIN from stg_fx_rates.currency_code.

6. Naming conventions

6.1 Datasets

The Snowflake row follows the organisation and account name form that the OpenLineage naming page recommends over the legacy account locator; switching later breaks existing graph links. Where the specification does not list a store (the Iceberg row), record a local rule here and attach a symlinks facet that maps the table to its storage path. Check the current naming page before adding a store.

6.2 Jobs

Rules: one namespace per producer and environment; never reuse a job name for a different job; renames are announced and recorded as a symlink or alias in the backend.

7. Producers and integration points

Producers send events over HTTP to the collector endpoint with an API key per producer, or to a Kafka topic when the network does not allow direct HTTP.

8. Lineage store and architecture

9. Consumers and access

The impact check in pull requests works as follows:

10. Lineage quality and monitoring

Example: in week 2026-W43 Corvid had 1,240 shared datasets, of which 1,221 had a producer in the graph, giving LQ-001 coverage of 98.5%.

11. Security and privacy

Lineage is processing metadata, but job names, SQL and error messages can still contain personal data. Treat the lineage store at the highest classification of the metadata it may hold.

12. Rollout plan

13. Glossary and references

13.1 Glossary

13.2 References

Standard and licence note

Based on the OpenLineage specification 2-0-2 and its published facet and naming documentation (OpenLineage, an LF AI and Data Foundation project, Apache License 2.0). Event types, field names, facet names and naming patterns are used as identifiers; no specification text is reproduced. The use cases, requirements, quality checks, local naming rules and examples are original, and Corvid Commerce is fictional. This template is original text written for this collection. Check the current OpenLineage specification and the documentation of your integrations before use. Template licence: MIT.