Let’s face it. Nobody wakes up excited to manually track data changes across systems. That’s why Microsoft Fabric’s Data Factory has introduced a new best friend for data engineers: Change Data Capture (CDC) in Copy Job (Preview). It’s like having a super-efficient intern who never sleeps, never complains, and always knows exactly which rows were inserted, updated, or deleted.
This feature is designed to keep your destination data fresh and synchronized with minimal effort. Whether you're wrangling Azure SQL DB, SQL Server, or Fabric Lakehouse tables, CDC in Copy Job helps you move data smarter, not harder.
CDC in Copy Job is a native capability in Fabric’s Data Factory that detects and replicates changes in source data such as insertions, updates, and deletions without requiring you to babysit your pipeline.
It supports two flavors of incremental copy:
This flexibility means you can tailor your data movement strategy to match your source system’s personality, whether it’s CDC-savvy or still living in the watermark era.
Zero Manual Intervention
No more setting reminders to update your pipeline logic. CDC handles it all behind the scenes.
Automatic Replication
Your destination data stays in sync like a well-rehearsed dance routine with no missed steps.
Flexible Incremental Copy Options
Choose your method per table. CDC or watermark? You decide. It’s like a buffet, but for data movement.
Optimized Performance
Only changed data gets processed. That means faster jobs and less strain on your source systems. Your database will thank you.
Smarter Data Movement
Copy Job remembers where it left off, so even if something crashes, it picks up right where it stopped. Reliable, like your favorite coffee mug.
Using CDC in Copy Job is refreshingly simple:
Supported sources include Azure SQL DB, SQL Server (on-prem and managed), and Fabric Lakehouse. Destinations include Azure SQL DB, SQL Server, Fabric SQL Database, and Snowflake.
Now for the fine print. Microsoft notes that Slowly Changing Dimension Type 2 (SCD2) is not yet supported for CDC-enabled source datastores. That’s a bummer for folks who love tracking historical changes in dimensional models. But the fact that they called it out means it’s probably on the roadmap. Fingers crossed, SCD2 support might just be around the corner.
CDC in Copy Job is like upgrading from a flip phone to a smartphone for your data pipelines. It’s smarter, faster, and way less needy. Whether you’re modernizing legacy systems or building new ones in Fabric, this feature is a must-have for any data engineer who values efficiency and sanity.
And when SCD2 support finally arrives? That’ll be the cherry on top. If you have any questions, feel free to reach out to me on Linkedin!