Fabric Codexfor Microsoft Fabric

Advisor Help Sign in

Spark / Data Engineering

Notebooks, Spark job definitions, and the runtime that powers engineering workloads.

Content

Lesson
Spark SQL in Microsoft Fabric — Intermediate: Catalyst, AQE, and choosing your SQL engine
Lesson
Spark SQL in Microsoft Fabric — Expert: Native Execution Engine, shuffle, and read-path internals
Lesson
Spark SQL in Microsoft Fabric — Beginner: querying the lakehouse from notebooks
Lesson
Architecting Spark Workloads in Microsoft Fabric
Lesson
Spark Engine Internals in Microsoft Fabric: Native Execution, Compute Topology, and the Write Path
Lesson
The Spark engine in Microsoft Fabric — Beginner
Article
Apache Spark in Microsoft Fabric: A Practitioner's Guide to Compute, Delta Writes, and the Native Execution Engine
How Fabric's fully managed Spark platform actually behaves: starter vs. custom pool startup mechanics and billing, node/executor architecture, autoscale internals, Delta write optimization (Optimized Write, Auto Compaction, V-Order), the C++ Native Execution Engine and its correctness edge cases, Efficient Scaledown's remote-shuffle resilience, operations tooling for diagnosing failures and reading history, runtime release channels, Synapse migration, and Spark Job Definition / DataFrameWriterV2 / cluster-concurrency patterns for production pipelines — every claim sourced and cited.