Overview#

A medallion lakehouse pattern progressively curates data through raw, cleansed, and business-ready layers while keeping the data in OneLake as Delta/Parquet tables [S1] [S2]. Fabric supports this pattern because a lakehouse stores tables in OneLake, Spark can write and transform those tables, and SQL or Direct Lake consumers can read curated outputs without a separate platform copy [S1] [S3] [S4].

Original diagram: bronze, silver, and gold zones as layered Delta tables in one OneLake - ingestion engines land to bronze, Spark promotes bronze to silver to gold, and gold is served read-only via the SQL analytics endpoint and Direct Lake

Core concepts#

The bronze layer is the landing point for source data, the silver layer applies cleansing and conformance, and the gold layer contains curated tables for BI and SQL consumption. That bronze/silver/gold naming is a design convention, so the layer names are inference; the mechanics are grounded in Fabric lakehouse, Delta table, Spark, SQL endpoint, and Direct Lake behavior [S1] [S2] [S3] [S4].

Managed lakehouse tables belong in the Tables area and use Delta format so they can be discovered by the SQL analytics endpoint and used by downstream engines [S1] [S3]. Delta supplies transaction-log semantics over immutable Parquet files, enabling repeatable table versions and recovery patterns that are important for multi-stage curation [S2].

How it works and best practices#

Use Data Factory Copy activity or shortcuts to land source data, Spark notebooks or job definitions for complex transformations, and the lakehouse SQL analytics endpoint or Direct Lake semantic models for serving curated tables [S1] [S3] [S4] [S5]. Use Dataflow Gen2 when low-code Power Query preparation is the right authoring model for a specific source [S6].

File layout is part of the design. Optimized Write and compaction help control small files, while V-Order can improve read performance for read-heavy curated tables at the cost of extra write work [S7] [S8]. Gold tables that serve BI should therefore be treated as read-optimized products, not just the final folder in a pipeline [S4] [S7].

Original diagram: decision guide for Pipeline Copy activity vs Dataflow Gen2 vs Spark vs OneLake shortcut across a medallion pipeline, plus the V-Order-per-zone file-layout rule

Implementation example#

The medallion implementation is a sequence of Delta table writes, not a sequence of separate storage systems. Bronze captures source-shaped data, silver applies quality and conformance rules, and gold publishes business-ready tables for SQL and BI serving [S1] [S2] [S3] [S4].

python
# Illustrative bronze -> silver -> gold flow in Spark.
bronze = spark.read.table("bronze_orders")

silver = (
    bronze.dropDuplicates(["OrderId"])
    .filter("OrderDate IS NOT NULL")
    .filter("CustomerId IS NOT NULL")
)
silver.write.format("delta").mode("overwrite").saveAsTable("silver_orders")

gold = silver.groupBy("OrderDate", "CustomerId").sum("SalesAmount")
gold.write.format("delta").mode("overwrite").saveAsTable("gold_customer_sales_daily")

Inference: the transformation logic is illustrative. The sourced mechanics are lakehouse Delta tables in OneLake, Spark as an engineering path, SQL endpoint serving, Direct Lake consumption, and Delta transaction-log behavior [S1] [S2] [S3] [S4].

yaml
table_maintenance:
  bronze:
    priority: replayability
  silver:
    priority: quality_and_conformance
  gold:
    priority: serving_performance
    controls:
      - compaction_review
      - v_order_decision
      - direct_lake_file_size_review

The maintenance controls are grounded in file-layout sources: V-Order, compaction, optimized write, and Direct Lake storage behavior all affect downstream read performance [S4] [S7] [S8].

What goes wrong#

A common failure is calling a table gold while leaving it with raw file layout, weak ownership, or no security model. Small files, over-partitioning, and unmanaged Delta maintenance can slow both metadata sync and query execution [S3] [S7] [S8]. Another failure is relying only on SQL endpoint permissions when Spark or OneLake access paths can still reach the underlying data [S3] [S9].

Internals#

Architecture & design#

A Fabric medallion implementation is physically a set of Delta tables in OneLake, with each layer represented by tables and folders inside a lakehouse [S1] [S2]. The SQL analytics endpoint exposes eligible managed Delta tables as a read-only T-SQL surface [S3].

How it works internally#

Delta stores table state in a transaction log, with immutable Parquet files referenced by log actions [S2]. SQL endpoint metadata sync reads Delta metadata from table folders so SQL consumers can see table changes without a separate copy operation [S3].

Performance characteristics#

Performance is dominated by file size, file count, partitioning, V-Order, and Spark execution behavior [S7] [S8]. The native Spark execution engine can accelerate eligible Spark workloads, while compaction and optimized write reduce downstream file-count penalties [S8] [S10].

Worked example#

A sales platform can land order extracts into bronze with Copy activity, transform customer and product conformance in silver using Spark, publish aggregated gold Delta tables, expose those tables through the lakehouse SQL endpoint, and build a Direct Lake semantic model for Power BI [S1] [S3] [S4] [S5].