Cole Tramp's Microsoft Insights

Overview

Let’s say you are running Spark workloads in Microsoft Fabric. You have committed capacity (CUs) to handle your usual work like BI, data transformation, and dashboards. But Spark jobs can spike unpredictably. If you tie Spark to that shared capacity, those spikes can starve your regular workloads or force you to overprovision capacity, which wastes money.

Autoscale Billing for Spark solves this problem by letting Spark operate independently. Once enabled, serverless Spark does not consume your assigned Fabric capacity units. Instead, each Spark job spins up on its own dedicated resources. You are billed per use, and your other workloads are unaffected.

In short: turn it on, forget about it, and let Spark run without disrupting everything else.

How Serverless Spark Works Without Using Fabric Capacity Units

Here is what happens under the hood:

Read More

 

Overview

Every organization today is swimming in data. The challenge isn’t collecting it, it’s making sure the right people can actually find, use, and trust it. That’s where Microsoft Fabric Domains come in. Think of them as a way to bring order to the chaos. Instead of one central IT team trying to juggle everything, Fabric Domains shift ownership to individual business areas like Sales, HR, or Marketing. Each domain gathers together the workspaces that belong to that team, while subdomains allow you to get even more specific. The real magic is that domains also support governance: settings can be delegated, admins and contributors assigned, and default domains set up so things stay consistent.

Why Microsoft Fabric Domains Help

Read More

Overview

Microsoft Fabric continues to evolve, and the latest release of Customer Managed Keys (CMKs) is a big leap forward. For organizations watching closely for Fabric’s eventual arrival in Azure Government, this update is more than just a security checkbox. It is proof that Fabric is steadily aligning with compliance-heavy environments. Microsoft has made it clear through rep conversations and roadmaps that Fabric is expected to land in Azure Gov by late 2026. With CMKs now available, along with previously released features like Block Public Internet Access and Private Link Configuration, Fabric is building a compliance-ready foundation.

For Azure Government customers who must meet strict standards such as CMMC compliance, these new capabilities are essential. The message is clear: now is the time to start exploring Fabric in commercial tenants to prepare for the eventual Azure Gov rollout.

Why CMKs Matter

Read More

Many organizations are not moving to new data platforms because their existing pipelines are broken. The real challenge is that valuable data is often trapped in silos, spread across multiple systems, departments, and tools, making it hard to share, govern, and use effectively. In a world where data is the foundation for innovation, especially in the era of generative AI, the opportunity lies in bringing that data together, securing it, and making it ready for whatever comes next.

Microsoft Fabric offers exactly that. By unifying storage, governance, and analytics in a single, integrated platform, Fabric allows organizations to break down silos, strengthen security, and enable future-ready data strategies without disrupting the systems they already rely on.

OneLake: Your Single Source of Truth

Read More

In today’s fast-paced business landscape, organizations rely on consistent, secure, and easily maintainable configurations across their data workflows. For enterprises working with Microsoft Fabric, managing these configurations in a way that scales across teams and environments can be challenging. The Fabric Variable Library, currently in preview, aims to solve this challenge by offering a centralized, secure, and flexible way to store and manage variables for use across pipelines, notebooks, and other Fabric artifacts.

This article explores the business use case for adopting the Fabric Variable Library, how it fits into CI/CD processes, and the value it brings to teams that need consistency and governance in their data and analytics operations.

Centralizing Configuration for Consistency

Read More

Delta Lake is an open-source storage framework that enables ACID transactions, schema enforcement, time travel, and unified batch and streaming processing on top of Parquet-based data lakes. Microsoft Fabric natively integrates Delta Lake into its ecosystem, ensuring that all data in OneLake is Delta Lake-compatible by default.

This design means all Fabric workloads, including Data Engineering, Data Science, Real-Time Analytics, and Power BI, interact with a consistent, versioned, and transactionally safe data foundation.

Why Delta Tables Matter in Microsoft Fabric

Read More

As organizations modernize their data platforms, Microsoft Fabric offers two powerful options for managing data workloads: the lakehouse and the data warehouse. Both are built on Delta Lake and integrate seamlessly with OneLake, Fabric’s unified data lake. While they share a common foundation, they are optimized for distinct personas, workloads, and data lifecycles.

Read More

 

In today’s data-driven economy, organizations are expected to leverage their data assets not only for strategic decision-making but also for regulatory compliance, security, and operational agility. With the exponential growth of data across services, platforms, and business units, maintaining visibility and control over an enterprise data estate is becoming more complex and more critical.

Enter Microsoft Purview in Microsoft Fabric: a unified solution that brings governance, compliance, and data insights into the center of your Microsoft ecosystem. With the integration of Microsoft Purview directly into Microsoft Fabric via the Purview Hub, businesses now have a single pane of glass to manage, monitor, and secure their data across the enterprise.

Centralized Data Governance for Fabric-Driven Organizations

The Microsoft Purview Hub, now available in preview, is a governance command center within Microsoft Fabric. Designed for Fabric administrators, it provides a comprehensive view of an organization's data landscape, offering insights into:

Read More

Overview: What Is the Fabric Data Agent?

The Microsoft Fabric Data Agent is a built-in capability within Microsoft Fabric that allows organizations to create conversational, AI-powered Q&A systems over structured data stored in OneLake. It empowers users to ask plain-English questions and receive accurate, relevant answers directly from trusted datasets, without requiring coding, query languages, or advanced analytics knowledge.

By integrating generative AI with structured data sources, the Data Agent reduces friction between people and insights. This enables faster, smarter, and more inclusive decision-making across the business.

Key Business Use Cases

1. Empower Non-Technical Users

Read More

 

As more organizations seek greater flexibility and performance in their data pipelines, Microsoft's Dataflow Gen2, now part of Microsoft Fabric, offers compelling improvements over the legacy Dataflow Gen1 experience. If you’ve been using Gen1 for Power BI transformations, it may be time to consider moving your workloads.

Let’s explore the differences, the advantages of Gen2, and how to migrate efficiently using the import/export method.

Understanding the Evolution: Dataflow Gen1 vs Gen2

Dataflow Gen1 was introduced as part of Power BI to help users perform ETL (Extract, Transform, Load) operations with ease using Power Query. It allowed for cloud-based data prep at scale - but it came with several limitations that began to restrict enterprise use cases as demand grew.

Read More