RelatedWhat I read
before I decide.

Battle Drones - a vibe coded RTS with audio-based multiplayer

MemCurrent: Designing Persistent Memory for Production Agents

AI tools have made it easy to vibe code agents. Building and maintaining an enterprise...

Stop asking whether your pharma company should "go agile"

One pharma company works in four very different kinds of environment at the same time. So the useful question isn't whether to adopt agile. It's agile to solve what, how much, and what kind. Over the years, I've been asked some version of the same question countless times: "Shoul…

Continuous axis for any Date table in Power BI

How to get a continuous X-axis at the month level in Power BI with any Date table, like the one we get using the auto date/time feature.

Investment screening reshapes rather than shrinks foreign investment

Driven by national security concerns, a growing number of countries screen foreign investment in an expanding range of sectors, yet evidence on the economic effects of such screening is scarce. This column uses new sector-level data for 43 OECD and EU countries to show that scree…

Bank liquidity regulation and the growth of private credit

We study the role of bank liquidity regulation in the growth of private credit. Following the introduction of the liquidity coverage ratio (LCR), private credit increases significantly more in U.S. counties with a larger footprint of banks subject to the LCR or of LCR banks with…

How Claude designed protein binders against scorpion neurotoxins

We asked an internal research Claude model to design minibinders (small proteins that attach tightly to a target) against scorpion neurotoxins.

Introducing Agent Memory on Databricks

Today we're introducing agent memory on Agent Bricks, Databricks' agent developer...

The Visibility You Think You Have Isn't the Visibility You Have.

Image In Digital.ai's 2025 State of Agile, 55% of organizations said they have complete visibility into what they're building. In the same survey, 63% can't reliably deliver it, up 12 points in a year. Those two findings describe the same companies. Sit with that. Visibility went…

Continuous axis for any Date table in Power BI

Power BI enables the continuous X-axis on the auto date/time hierarchy even at the month level. We can get the same behavior on any Date table by setting the right metadata in TMDL. In a previous article, Creating a simpler…

Lower capital requirements are no shortcut to European competitiveness

The European banking sector is known for high regulatory complexity, market fragmentation, and uneven implementation of international standards. However, this column argues that across-the-board reductions in capital requirements are not the solution. European requirements are no…

Geoeconomic pressure

The increasing use of geoeconomic pressure is reshaping the global economy.

Franchisee reconciles cash in 94% less time with ChatGPT Work

An independent Schlotzsky’s and Cinnabon franchisee cut menu repricing time by 95% and built a workflow that could free up $100K in cash flow.

How Hamlet uses Databricks Lakebase Postgres to test changes against production data

Hamlet, an AI-powered platform that makes local government decision-making accessible...

Business Requirements: From Business Decisions to Clear Action

Image Business requirements are about more than choosing the right words. A requirement can sound clear and still create problems if the team has not fully understood the business need or made the decisions required to move forward. Strong business requirements start with the wor…

Monitoring The Impact Of Full-Text Indexing On Semantic Model Refresh In Power BI

The September release of Power BI saw the introduction of a powerful new feature: full text indexing, which makes searching and filtering on text columns much faster and enables the use of the new TextContains() and TextSimilarity() DAX functions. You need to enable it on a per-c…

What happens to innovation after Big Tech buys a startup

Concerns abound that ‘killer acquisitions’ by tech giants could deter startup entry and stifle innovation. This column analyses patent data before and after 1,200 acquisitions by eight technology giants between 1995 and 2020. On average, the ‘Big Eight’ firms appear to invest in…

Central banks and macro financial stability in a fragmented global economy

The 25th BIS Annual Conference took place in Basel, Switzerland on 26 June 2026. The event brought together a distinguished group of central bank Governors, leading academics and former public officials to exchange views on the theme “Central banks and macro-financial stability i…

Terminal X matches senior investor judgment 88.6% with GPT-6.1

Terminal X’s Eval Agent grades outputs against a 54-criterion rubric, catching 25% more deviances from a fund’s mandate with GPT-6.1 Sol.

How HP Is Building an AI-Ready CDP with Databricks CustomerLake

At Data + AI Summit 2026, Luis Alonso, Global Head of Data Strategy & Engineering...

Pragmatism Crushes Dogma in the Wild

Originally published on Engineering Leadership. In my journey of delivering an immersive Product Development Mentor Program over the last eight weeks, a compelling narrative unfolded that beautifully illustrates the essence and true strength of Scrum. This story, rooted in the pr…

ApproximateDistinctCount In Power BI Is Sometimes Slower Than DistinctCount

As a follow-on from last week’s post about the new ApproximateDistinctCount function in DAX, I want to share something I found out from my recent testing for this function: ApproximateDistinctCount is sometimes slower than DistinctCount. This sounds counter-intuitive, right? Afte…

Taming original sin redux: The role of benchmark-driven flows

Emerging market capital flows remain highly sensitive to global shocks. This column analyses foreign investors in emerging market local-currency sovereign bonds depending on whether they are benchmarked to J.P. Morgan’s main local-currency government bond index or not. Benchmarke…

Inflation after the Iran war

An unexpected increase in energy costs raises inflation directly through flexibly-set prices. Whether that shock amplifies to a larger and persistent rise in inflation depends on its transmission through the economy.

OpenAI acquires Orchestra Bio

OpenAI is acquiring Orchestra Bio to bring AI-powered operations to life sciences R&D, helping biopharma teams accelerate drug development and advance new treatments.

CustomerLake enters Beta: Leading enterprises going live with the Agentic CDP

Today, we’re excited to share a major CustomerLake milestone: Campaign Agents is...

IA e Product Owner: decidere sulla base delle evidenze

L'intelligenza artificiale permette di analizzare informazioni e sviluppare nuove proposte in tempi sempre più brevi. Per il Product Owner diventa ancora più importante comprendere quali informazioni siano affidabili e come utilizzarle per prendere decisioni sul prodotto. Image I…

Understanding DAX contracts when using AI

Defining the right contract is a key activity for a DAX developer using AI. Here we introduce the concept of a contract through simple examples.

Here, there, and everywhere: Remote work and the employment of disabled and older workers in the UK

Disabled and older adults have lower employment rates than other workers. Policymakers increasingly point to remote work as a way to keep these groups attached to the labour market. This column uses the pandemic-driven rise in working from home in the UK as a quasi-experiment to…

Competing rails for crossborder payments: banks, fintechs, and stablecoins

Cross-border payments remain slow and expensive despite significant improvements in digital financial infrastructure over the past several decades.

Advancing coding in low‑resource programming languages

Learn how OpenAI is improving ChatGPT across low-resource programming languages, with new evaluations for software engineering, SQLite, and COBOL migration.

What makes a catalog open? Unity Catalog Advances the Open Lakehouse

While almost every platform now supports open formats, what makes a catalog truly...

Your Transformation is Suboptimal Because You Keep Your Dependencies

You trained the teams. You renamed the roles. AI tools, Sprints, backlogs, planning events, all in place. Teams got a lot faster but end-to-end delivery not so. Why? Well, likely you changed how people work but you didn't change how the work depends on itself. And dependencies de…

Understanding DAX contracts when using AI

Defining the right contract is a key activity for a DAX developer using AI. In this article, we introduce the concept of a contract through simple examples. Contract. As a DAX developer, you will use this term frequently; indeed, when…

Forward guidance and the limits of public information

When nominal interest rates hit the effective lower bound, central banks turned to unconventional measures, with forward guidance emerging as a cornerstone of monetary policy. Drawing on economic literature and data on interest rate forecasts, this column assesses the impact of f…

Offshore forwards and exchange rate transmission in emerging markets

We study how offshore non-deliverable forward (NDF) markets transmit exchange-rate pressures to domestic currency markets. Using unique transaction-level data on the universe of centrally cleared NDFs for ten emerging-market currencies over 2013–2025, we show that offshore NDF or…

How Itaú is building generative AI for its entire customer base

With ia.i, a conversational interface, and the IARA governance platform, Itaú is applying its AI First strategy to customers and employees — in one of Brazil’s most regulated sectors.

How to get the most out of running dbt on Databricks

Tens of thousands of organizations rely on dbt for data transformation. Where you...

How To Create A Team Wall For Your Scrum Team

Image Transparency is the backbone of empiricism, which is the foundation of Scrum. Without transparency, inspection and adaptation are pointless. The higher the transparency you have in your team, the better. Bear in mind that all Scrum Team members are responsible for keeping t…

FabCon Europe 2026: 4 announcements that may change how we work with data and AI

Barcelona is done. Nearly 5,000 people, 130+ sessions, and a very long list of announcements. This is my cut of the ones that matter, why they matter, and what you should do about them. TL;DR Everything you build in Fabric now reaches Microsoft Copilot and agents directly. Fabric…

Letting AI agents write econometric code

Large language models increasingly write the code behind empirical research. This column reports a benchmark of 40 econometric tasks in Stata, R, and Python. For each task, two leading models write a script, as a chatbot that never runs it and as an agent that runs and revises it…

Chokepoint disruptions and their macroeconomic consequences

Rising geopolitical tensions, including trade restrictions and conflicts, are increasingly disrupting global supply chains, posing significant risks at critical chokepoints. Strong substitution effects and the use of inventories have helped to mitigate adverse output effects of c…

Grounding AI agents in biological reality: Building blocks for autonomous wet labs

General Science

Databricks Is Doubling Down On New York City As A Premier AI Talent Hub

When Databricks opened its New York R&D hub at 5 Bryant Park earlier this year, we...

AI-Assisted Product Discovery: When Code Is Cheap, Judgment Is the Job

TL; DR: Deciding What to Build Is the New Job, Now that Code Is No Longer a Constraint Agentic coding has removed the cost gate that used to stop product teams from chasing every idea at once, and the constraint has moved to product judgment: the ability to tell what is worth bui…

The Problem With Approximate Distinct Counts In Power BI – And How To Solve It

As you’ve probably seen in the blog post for the September 2026 release of Power BI, Import mode and Direct Lake mode semantic models now support the ApproximateDistinctCount() DAX function. I tested its performance on a Direct Lake semantic model with a 1.4 billion row fact tabl…

Fiscal consolidation: Temporary costs versus lasting scars

Public debt is at or near record highs across the OECD, increasing the pressure on governments to restore fiscal sustainability over time. This column draws on new cross-country evidence showing that the output costs of fiscal consolidation are modest and short-lived on average,…

SS2/26 – High loan to income lending

Supervisory statement 2/26

Designing more efficient quantum circuits with Claude

We gave Claude plain-language specifications for eight quantum circuits, along with an automated evaluator to check and score its designs. Working largely on its own, Claude improved on the best published circuits, in some cases by an order of magnitude or more, which could bring…

Data Engineering Weekly #292

The Weekly Data Engineering Newsletter

AI and the Product Owner's Accountability

Image It's Monday morning. Over the weekend, an AI system read every support ticket from the last quarter, every sales-call transcript and every competitor's release notes. It rewrote sixty Product Backlog items, re-ordered the top twenty and drafted a Product Goal, with a short…

FabCon and SQLCon 2026 in Barcelona: Building the data foundation for Microsoft Copilot and agents

Microsoft Fabric and SQL innovations announced at FabCon and SQLCon Barcelona 2026 help organizations build trusted data foundations for AI. The post FabCon and SQLCon 2026 in Barcelona: Building the data foundation for Microsoft Copilot and agents appeared first on Microsoft Fab…

Mortgage Lock-in: A Review of the Literature

Federal Reserve Bank of Atlanta Working Papers by Kristopher Gerardi, David Hao Zhang and Franklin Qian

PS19/26 – High loan to income lending

PRA policy statement 19/26 | FCA policy statement 26/20

Claude reveals ECDSA curve origins

The decades-old mystery of the origins of one of the most popular cryptographic standards has been explained.

DuckDB Ecosystem Newsletter : October 2026

The October 2026 DuckDB Ecosystem Newsletter: Jev typed AI answers in DuckDB and MotherDuck SQL, DuckDB 2.0's PEG parser and async S3 reads, DuckLake maintenance, analysis-ready OpenStreetMap, OneLake in the browser, a native Mac app, plus a community spotlight on Mark Litwintsch…

Strategy Thinks in Bets. Tactics Execute Them.

Image Something has been spreading through the product teams I work with since around 2022, and it has the same shape as every good idea that becomes a religion. Teams are describing everything as a bet. The instinct behind it is a sound one. Outcomes, the real measure of product…

Improvements To Power Query Integration In Power BI Report Builder

There are some nice improvements to Power Query integration in the latest version of Power BI Report Builder that, while they are documented, didn’t get an official blog post but which I think deserve to be highlighted. First of all the “Get Data” button has been renamed “Power Q…

The transmission of shocks across sectors and the dynamics of sectoral prices

European Central Bank Working Papers by Francesca Monti and Leïla Van Keirsbilck

Money and the architecture of trust

Money is one of the most consequential social institutions humanity has built. Starting from the most elementary monetary act, a payment, this paper provides an account of how money works, grounded in the tradition of credit theories of money.

The AI expectations gap

How will AI reshape the economy in the years ahead? What is the core source of disagreement about AI's future impact on the economy? This note presents new survey evidence on these questions.

Transactions are Generally Available, advancing warehousing on the Lakehouse

Multi-Statement Transactions are Generally Available, enabling organizations to run...

Stop Guessing, Start Forecasting: Why Scrum Teams Should Embrace Monte Carlo

Image How confident are you when someone asks, "When will it be done?" If you've worked with a Scrum Team for any length of time, you'll know this question well. And you'll probably recognise the uncomfortable silence that sometimes follows. We estimate. We plan. We look at veloc…

Skills for Report Authoring in Power BI: A Conversation with Harleen Kaur (Fabric Insider Ep. 15)

We’ve already covered how AI can build and modify a semantic model using the Power BI Modeling MCP server. But building the semantic model is only half the job, someone still has to build the actual report on top of it. In this episode of Fabric Insider, I sit down with Harleen K…

Commercial real estate debt financing in Europe: evidence from a network analysis

European Systemic Risk Board Working Papers by Frauke Skudelny, Mihai Aliman, Martin Regnér, Cecilia Sarchi and Matteo Valetto

CP13/26 – The Prudential Regulation Authority’s automatic thresholds indexation framework

Consultation paper 13/26

How employment is changing at Anthropic

How has AI restructured work and employment at Anthropic? As a company at the frontier of AI adoption, Anthropic could be a leading indicator of how AI will shift work organization, hiring, and labor demand across the economy.

Closing the Binary Blind Spot: Domain-Knowledge Injection for LLM Malware Triage in npm Packages

IntroductionSupply chain attacks on package registries have grown in volume and sophistication [1]...

See the System: When a Team Impediment Is Really a Design Problem

Organizational problems rarely arrive labelled as organization-design problems. They show up in everyday work. A team waits for a decision. Another starts before its inputs are final and has to redo the work. Someone escalates to protect a launch date. A new control gets added af…

Fabric CLI Explained: Install, Use, and Connect It to GitHub Copilot

There are multiple ways you can connect AI to Power BI and Microsoft Fabric. You could use MCP servers, you could use Fabric Skills, but there’s also another way: Fabric CLI, the command line interface. It’s not used as much as the other methods, but it’s quite powerful. In this…

Structural Estimation with Unstructured Data

Federal Reserve Bank of Dallas Working Papers by Jesús Fernández-Villaverde, Minchul Shin, Stephen Hansen, Ryohei Oishi and Sara Casella

PRA Regulatory Digest – September 2026

The PRA Regulatory Digest is for people working in the UK financial services industry and highlights key regulatory news and publications delivered for the month.

Working at the frontier shapes expectations for the future

We’re sharing two reports: the first covers how AI is changing employment inside Anthropic, and the second looks at the gap between how the public and Anthropic employees expect AI to affect work and the economy.

From JSON Strings to Typed Columns

DuckDB's read_json turns JSON files into tables. It reads through part of the data to work out a schema, then gives you typed columns. However, a lot of JSON does not come as files. It comes as a text column: event payloads in a Postgres table, API responses stored by an ingestio…

How to Perform a Gap Analysis

Image Knowing where you are today and where you want to go is important, but you also need to understand what stands between the two. Learning how to perform a gap analysis helps teams identify what needs to change before they can reach their desired future state. In business ana…

Start using DAX with AI

The time has come: in 2026, using AI to write DAX has started to be effective. Latest news and the new courses are at https://www.sqlbi.com/ai/ We also created three new video courses that should not become obsolete in two weeks!…

How Firms Form Beliefs and the Implications for Inflation

Board of Governors of the Federal Reserve System Finance and Economics Discussion Series by Robert Minton and Hugo Monnery

Letter from David Bailey 'Thematic feedback on accounting for IFRS 9 expected credit losses (ECL)'

Letter to chief financial officers of selected PRA-regulated deposit-takers which provides thematic feedback from the PRA’s review of written auditor reports received in 2026 covering IFRS 9 expected credit loss accounting (ECL) and accounting for climate risk.

Stack Overflow for Agents triples signups with ChatGPT plugin

Two engineers shipped Stack Overflow for Agents in a few weeks with Codex, and its ChatGPT plugin helped drive up to 10× growth in agent creation.

Benchmarking DuckDB v2.0-dev on NVIDIA Vera

We love running benchmarks on all sorts of new hardware. This often includes somewhat exotic hardware – where we sometimes need to compile a new DuckDB binary. This time, in collaboration with the NVIDIA team, we got our hands on their new Vera chip, aimed at agentic workloads. N…

When Software Meets Hardware

When software meets hardware, engineering principles remain largely the same, but the sensible defaults often change. Kuldeep Singh describes our experience modernizing a large IoT fleet of battery-powered sensors. Our familiar principles remained important; what changed was how…

The hidden complexity of a daily average in DAX

A simple daily average can hide complexities and return different values. We explore several edge-case scenarios to build the desired calculation using AI to accelerate the development.

Credit Surfaces and Economic Uncertainty

Board of Governors of the Federal Reserve System Finance and Economics Discussion Series by David E. Rappoport and John Geanakoplos

The PRA held a captive insurance industry roundtable

On 17 September 2026, the PRA hosted a roundtable in relation to CP11/26 – A tailored regime for captive insurance.

Benefit Street Partners answers questions faster with Codex

Anant Kumar builds tools to compare loan valuations and analyze credit agreements, using Codex for code review and faster follow-ups.

prompt_jev() now works on images

prompt_jev() now takes images: store them as BLOBs in MotherDuck, ask closed questions in SQL, and get typed answers with a confidence score you can join with the rest of your data.

The First 90 Days Field Guide For New Scrum Masters

Discover the practical steps you need to take within your first 3 months on the job to confidently solidify your position in your Scrum Team. Image Imagine your company has recently asked you to take over the Scrum Master role within your Scrum Team, or maybe you've personally ch…

PBIP, PBIR, and Making Power BI AI-Addressable: A Conversation with Rui Romano (Fabric Insider Ep. 14)

PBIX isn’t going away, but the code behind it is finally becoming something you can version, script, and hand to an AI agent without things blowing up. In this episode of Fabric Insider, I sit down with Rui Romano, Product Manager on the Power BI team within Fabric, the person be…

Inference Based on Scale, Label, and Economic Restrictions

Federal Reserve Bank of Philadelphia Working Papers by Daniel F. Waggoner, Juan Francisco Rubio-Ramírez and Jonas E. Arias

Bringing GPT-Rosalind to academic research labs

Introducing the Rosalind Academic Plan, offering eligible academic labs sponsored access to GPT-Rosalind and tools to advance life sciences research.

MotherDuck is coming to Azure

MotherDuck is coming to Azure. Pick your region on the waitlist; regions open in order of demand. Your Copilot agents can connect to MotherDuck today.

Build your slide template

Building a slide template starts with three foundational choices — colours, fonts and layouts. Sumeet Gayathri Moghe covers the basics of color theory and how to build a reusable palette, the essentials of typography, including serif, sans-serif, handwritten and monospace fonts,…

Where we stand on AI and DAX in 2026

The time has come: in 2026, using AI to write DAX has started to be effective. Let me recap what has changed between 2023 and 2026, and what we are doing with our three new courses to best assist you…

Prices and Monetary Policy: The Role of Financial Constraints

Federal Reserve Bank of San Francisco Working Papers by Michael D. Bauer, Alexander Czarnota and Mathias Klein

Accelerating the search for antimicrobial peptides with Google Antigravity

Accelerating the search for novel antimicrobial peptides using Google Antigravity for Science.

A Chief of Staff's Guide to Context Layers

Kaitlyn, MotherDuck’s Chief of Staff, on why self-serve analytics with AI agents fragments your metrics, and how non-technical leaders can get everyone on one definition of metrics.

Scrum: Delivery Method or AI Governance?

The same Scrum framework sits behind organisations that get real value from AI and those that don't. What separates them is where Scrum is placed in their operating model. For many organisations, Scrum is a delivery method inside a project: the business case fixes scope, budget,…

The hidden complexity of a daily average in DAX

A simple daily average masks a level of complexity that may not be immediately apparent. In this article, we use AI to create a daily average, and then to analyze several edge-case scenarios to build a sound calculation. Computing an…

Putting Earth AI to work: Geospatial agents for our planet's biggest challenges

Earth AI

The Self-Organizing Data Stack

When agents answer the questions, they can maintain the data model. Here is what that demands from the orchestration engine.

Your Sprint Review isn't inspection. It's theatre.

Image The demo works. The feature does exactly what the slide said it would. Someone says "nice." A couple of people do the little typing-clap in the chat. "Any questions?" A pause, the good kind, the kind that means we're done. "Great, thanks everyone." And the review is over. N…

Find Unused Objects In Power BI Semantic Models With Semantic Link Labs

A few weeks ago I wrote about how Semantic Link Labs now has tools with interactive UIs and showed how you could use this to view lineage and so Vertipaq-Analyzer-stuff in a notebook. I didn’t show what I think is the coolest new feature though: the ability to find the tables, co…

Analyzing technical documents with Deep Review

General Science

What Is Inside an Iceberg Table?

An Apache Iceberg table looks like a regular table when you query it, but underneath it is a small tree of files: one JSON file at the top, a few layers of Avro files in the middle, and Parquet files at the bottom. When something is slow or looks wrong, it helps to know what is i…

7 Rules for Agile Teamwork Without Slowing Everyone Down

Seven practical rules for increasing collaboration, shared ownership, and team autonomy without involving everyone in every decision.

Fabric Core MCP Server: Scan and Configure OneLake Security with AI

With the appearance of AI tools, LLMs, MCP servers, and skills, everyone these days is interested in what tool can do what. I’ve already explained the Power BI Modeling MCP server and Fabric Skills in previous videos. In this one, I’ll talk about the Fabric Core MCP server, an MC…

Research@ 2026: 10 ways we're amplifying human ingenuity

General Science

Data Engineering Weekly #291

The Weekly Data Engineering Newsletter

The Craft of Practitioner Voice

A couple of months ago Jim Highsmith explained how practitioners writing for each other forms a distinct style of writing. He now adds a section to the article with more detail on the craft of how we write in this style. How the writer shows up in the prose, its four principles,…

DAX UDFs, Arrow REST API, and Outbound Access Protection: A Conversation with Kay Unkroth (Fabric Insider Ep. 13)

DAX UDFs just went GA, there’s a new REST API for running DAX queries at scale, and a network security feature most people haven’t even heard of yet is quietly changing how enterprises govern their semantic models. In this episode of Fabric Insider, I sit down with Kay Unkroth, P…

Google DeepMind and Singapore Government collaborate to build fine-tuned AI model for scam detection

Learn how Google DeepMind and GovTech Singapore uses lightweight on-device Gemma models to intercept conversational scams in private chats with zero cloud data egress.

Agent Mode in the DuckDB CLI

Let's Ask Claude We asked Claude to measure the effect of agent mode. In the experiment, Claude Code answered the 22 TPC-H questions on a scale factor 100 dataset. The questions were phrased in plain English, without any SQL. Each question was answered three times with agent mode…

Are Harnesses Worth the Cost?

Naresh Jain compares coding agents building the same API service with and without a behavioral harness. A frontier model, GPT-6 Sol, passed every external evaluation either way. But a cost-optimized model, GPT-6 Luna, succeeded in only 3 of 10 unharnessed runs and all 10 with the…

The query finished…Why is my Fabric SQL database still consuming CUs?

Two minutes of database activity can mean 17 minutes of compute billing. And a CPU chart doesn't tell the whole story. Let's work through SQL database in Fabric billing with examples from application workloads, development databases, and everyday troubleshooting. The post The que…

Developing Gemini 4 Argon responsibly and safely

Data Engineering Is the Cache Invalidation Problem ft. Josh Wills

Josh Wills, creator of dbt-duckdb and former head of data engineering at Slack, on why the model barely matters, why agents still write ugly and expensive data pipelines, and why data engineering is still the cache invalidation problem.

You Need Tighter WIP Limits With AI

In a recent class, we were discussing how teams are using AI and finishing coding faster but still missing Sprint Goals. How can these things both be true? That question gets to the heart of a shift many Scrum Teams are experiencing right now. AI has changed the bottleneck. Codin…

OneLake Security: Setting Up Row Level Security From Lakehouse to Report

If you’ve set up row level security in Power BI before, you know the usual drill: open the semantic model, add a role, write a DAX filter. But once your data lives inside a Microsoft Fabric OneLake structure, there’s a better way to do this. You can implement it directly in OneLa…

Gemini 4 Argon for Robotics

A DuckDB Database with No Data in It

A DuckDB database file does not have to contain any data. It can store nothing but view definitions that point at files stored somewhere else, such as Parquet on object storage. The file stays a few hundred kilobytes no matter how large the data it describes, and anyone who attac…

Cognitive Trap: Accountability Diffusion

Accountability Diffusion Trap Most mistakes in Scrum aren’t because people don’t understand the framework—they come from applying reasonable thinking in the wrong context. Cognitive traps happen when decisions favor efficiency, control, or comfort over transparency, inspection, a…

Find What Changed And When In Your Power BI Semantic Model

Here’s a situation I’ve found myself in many times. You’re called to help someone whose Power BI report is suddenly very slow. You ask whether any changes were made to the semantic model around the time performance got worse and the answer is no, absolutely not. You then spend an…

Gemini 4 Argon: Built to solve your hardest problems

5 Jev use cases for analytics, with real queries and datasets

Five prompt_jev() use cases on real data, with SQL you can rerun: AI agent traces, Hacker News sentiment, sales calls, cloud job postings and a regex replacement.

To Switch, or Not to Switch the AI Model

TL;DR: The Hidden Cost of Switching AI Models A new AI model arrives that is cheaper per token, scores better on benchmarks, and seems to require nothing more than a setting change. However, the token bill shows only part of the cost of switching AI models. There is also the effo…

Writing DAX at the correct granularity

A DAX calculation can produce convincing values at the detail level and fail at the total. The problem is often that the business rule was evaluated at the wrong granularity. This video introduces four grains, or levels of granularity, to…

Hello Python 3.15, goodbye 3.10, and now I can use 3.11

Data Engineering Weekly #290

The Weekly Data Engineering Newsletter

Fragments: October 4

In response to my last fragments (probably the bit about us worrying if LLMs have consciousness when we when we should be wondering why they don’t have a conscience) “Metalanguage” replied: we shipped the id and forgot the superego. classic software lifecycle. I don’t know what w…

Writing DAX at the correct granularity

A DAX calculation can produce convincing values at the detail level and still fail at the total. The problem is often not the total itself: the business rule was evaluated at the wrong granularity. This article introduces four grains, or…

A new feature for my blog, built using my voice

Importing Data using Java Table Functions

Guest blog post by a DuckDB user. The Analytics Screen Problem At the company where I work, we store vulnerability data for our clients as a collection of documents in MongoDB. Initially, our analytics screen performed well, but as our data volume increased, response times began…

Principles for effective slides

Not every presentation needs slides, but when they earn their place, they work as a visual channel that complements the speaker rather than replacing them. Sumeet Gayathri Moghe sets out the principles that follow from that idea — control over the rate of knowledge exposition, ti…

Choosing A Power BI Storage Mode For Reporting On Data From Azure Databricks

In case you haven’t already seen the blog post on the Power BI blog or the discussion on LinkedIn, we at Microsoft published a new white paper last week to help you decide which storage mode to use when you’re using Power BI to create semantic models and reports on data stored in…

Claude Haiku 5.5

Build It Yourself: What You'll Make at the Data Outpost Workshops

Data Outpost kicks off on November 4, 2026 in San Francisco with 11 hands-on workshops from MotherDuck, Vercel, Sigma, Prefect, Estuary, Airbyte, Reducto, Braintrust, and Together AI. Here's what you'll build in each one: your own agents, the data pipelines they need, and the ski…

AI Changes What Makes a Product Backlog Valuable

AI makes detailed backlog items and working software easier to produce. Learn why that favors more dynamic backlogs and makes early validation more important.

OneLake Updates: A Conversation with Josh Caplan (Fabric Insider Ep 12)

In this episode of Fabric Insider, I sat down with Josh Caplan, Director of Product for OneLake at Microsoft. Josh has been around this product family for a long time, starting as an intern on the Analysis Services team back in 2007, working on Project Gemini (which became Power…

Does better work always mean better workers?

Faster String Aggregations with Dimension Tables

Analytical workloads are full of repeated strings: product names, country names, station names, user agents, category labels. Take DuckDB's public train services dataset, which has one row for every stop a Dutch railway train makes. The data comes from the open datasets published…

Bliki: Sensible Default

A Sensible Default is a practice that, absent some overriding context, should be used when carrying out a certain kind of task. In software development such sensible defaults might include things like “use version control”, “separate UI logic from domain logic”, “automate deploym…

Implementing SCD Type 2 in Microsoft Fabric – The Definitive Guide!

Slowly Changing Dimensions (SCD) are one of the fundamental concepts in dimensional modeling. If you are not sure what dimensional modeling is, I suggest you first check the series of articles I wrote on data modeling some time ago. And, when it comes to SCD in particular, SCD Ty…

Unlocking Earth AI’s planetary geospatial foundation models for global public health

Earth AI

AI Needs Data, Data Needs AI: What You'll Learn at Data Outpost

Data Outpost is an AI and data conference in San Francisco. Its main day, November 5, 2026, has 13 talks and panels from teams at Canva, Stripe, DoorDash, Posit, dbt and more. Here's what each session covers, from agents that write back to context layers, SQL harnesses, and AI fo…

Fragments: September 29

Simon Willison: The more time I spend working with coding agents, the more convinced I am that they make software engineering even harder We can do amazing things with them, but unlocking their full potential requires extraordinary discipline and knowledge This has been a constan…

Teach Claude Your Power BI Standards, Custom Skills for the Power BI Modeling MCP Server

Can you teach Claude, or any LLM, to follow your standards and your guidelines, so that when it connects to a Power BI model, when it wants to build a semantic model or a report, it does it exactly as you do, following all the instructions that you would normally provide? Yes, it…

We're going to need default hard budget caps on pretty much everything

Thinking Like a Data Architect

A Field guide for data leaders

Fragments: September 24

Rob Bowley is “flipping tables in his head” with anger at the current media coverage of the danger of AI killing us all The risk I’m worried about isn’t a future machine deciding to wipe us out. It’s today’s AI, being wired into everything, carelessly and fast. Cyber attacks have…

Microsoft Fabric in GCC High: Building the data foundation for AI

Government organizations are ready to move beyond using AI to answer questions and toward intelligent systems and agents that can take action. The post Microsoft Fabric in GCC High: Building the data foundation for AI appeared first on Microsoft Fabric Blog.

OpenAI DevDay 2026 live blog

Jev and DuckDB: Plain-English Conditions in SQL

Say you have a Parquet file of 50,000 support tickets and want to know which customers are angry. A LIKE pattern won't find that. The usual options are to label data and train a classifier, or to send each row to a chat LLM and parse the text it returns. The first takes work up f…

Healthy Feedback

Human collaboration, like most things, improves with feedback. But it's not obvious how to make feedback effective. Anuja Karnik and Sumeet Gayathri Moghe pass on a bevy of tips for healthy feedback, based on the principle that both praise and criticism should be seen as positive…

Matrix Totals - The Whiteboard #13

Let's talk about matrix totals in this episode of The Whiteboard! Special guest: the SQLBI Whiteboard! Learn abstract DAX concepts in a more interactive way with "The Whiteboard" series. Read more: https://www.sqlbi.com/blog/marco/2022/07/14/the-whiteboard-video-series-on-sqlbi-y…

2026 in LLMs (so far)

Jev for analytics: turn text columns into gold, in plain SQL

Turn a free-text column into typed SQL columns with prompt_jev(): 100,000 consumer complaints classified in 82 seconds, at about 1% of a frontier LLM's cost, plus how to check the labels before you trust them.

I don't like LLMs

I have a lot of mixed feelings about AI and LLM technology. I’m fascinated by its effect on our profession, excited by the potential gains in productivity - and thus the products we could rapidly build. On the other hand, I’m fearful of the damage AI might cause: agent swarms tak…

Referencing Images And Other Files In Power BI Reports Using OneLake URLs

Several years ago I wrote a very popular blog post about how to store images for your reports inside your Power BI semantic model. It solved the problem of how you could use display images (for example of products) inside your reports without making those images available via a p…

Claude Opus 5.5, GPT-6 Sol, GPT-6 Luna, and a new price war

Data Engineering Weekly #289

The Weekly Data Engineering Newsletter

Fragments: September 16

Reports of agentic hacking continue, in this case it happened back in May and it seems OpenAI did not disclose that they were responsible. Simon Willison sees two options: After the Hugging Face and Wiki attacks OpenAI were still unable to review their previous logs and determine…

Fabric Apps & Rayfin Deep Dive: A Conversation with Sachin Patney | Fabric Insider Ep. 11

I have written and recorded about Rayfin before, what it is, how the Data App template works, and the cost story that makes it genuinely significant. But this conversation goes much deeper. I sat down with Sachin Patney, who leads Fabric App Development at Microsoft, and by exten…

Announcing DuckDB 1.5.6

In this blog post, we highlight a few important fixes in DuckDB v1.5.6, the sixth patch release in DuckDB's 1.5 (Variegata) line. The release ships bugfixes, performance improvements and security patches. You can find the full release notes on GitHub. To install the new version,…

Nail the Narrative

Sumeet Gayathri Moghe finds many folks building presentations get tangled in building slides without a coherent narrative. He advises distilling the big idea, visualizing the audience, and building a structured storyline. more…

SQLBI Whiteboard

SQLBI Whiteboard is a free, open-source whiteboard for Windows 11 designed for teaching DAX, SQL, and data modeling, with pen ink, live application capture, and syntax-highlighted code on the same canvas. Pen-first ink — Low-latency, pressure-aware strokes with palm rejection,…

DuckDB and Hugging Face: Querying Datasets Directly

Hugging Face is where much of the machine learning community publishes and finds its datasets, while DuckDB is the in-process analytical database that queries files like CSV and Parquet directly, with no server or warehouse to install or run. Did you know that, since DuckDB v0.10…

AI Can Write Backlog Items. It Can’t Create Shared Understanding

AI can generate polished user stories and acceptance criteria in seconds. Learn how to use it without skipping product judgment, user evidence, or team conversations.

No-Code Power BI Report Authoring — GitHub Copilot + Fabric Skills + Power BI Desktop Bridge

AI can perform a lot of interesting tasks these days. It can check your email. It can draft an email response. And it can do many actions within the Power BI environment — including creating a complete Power BI report entirely from a prompt window, without touching anything insid…

Querying the entire internet (100 billion rows!) with MotherDuck

Common Crawl publishes petabytes of web crawl data on S3. With DuckDB and MotherDuck you can query the Common Crawl dataset directly, no cluster and no download, and measure how fast the vibe-coded web is growing.

Social Media Engagement: summer 2026

A quick survey of recent engagement of my posts on social media, indicating which service has by far the most engagement, and which service has seen a precipitous decline since early 2025. more…

Testing DAX measures by using AI

How to use AI to identify relevant test cases in a semantic model and create repeatable DAX tests for a measure.

Data Engineering Weekly #288

The Weekly Data Engineering Newsletter

Fragments: September 8

Christian Catalini says we’re in a situation where we are vastly reducing the cost of generating things, but not the cost of verifying them:. This explains why the first major AI products appeared in chat, image generation, and code assistance. Not because these were the hardest…

Testing DAX measures by using AI

This article describes how to use AI to identify relevant test cases in a semantic model and create repeatable DAX tests for a measure. A DAX measure is correct only when it produces the expected result in every situation of…

dbt Charts on MotherDuck: Declarative Dashboards

Define your dashboards in YAML with dbt Charts, then the same dbt build that runs your pipeline deploys them to MotherDuck Dives. Just 2 config changes! Your Dives will refresh live whenever your stakeholders take a look. See how we constructed the dbt package as well.

Do you even need a presentation?

Like me, Sumeet Gayathri Moghe is tired of poor presentations with bad slide decks. He's started to write a series of posts on how to avoid these calamities, beginning with a post that questions whether a presentation is needed at all. more…

Tools With Interactive UIs In Fabric Notebooks With Semantic Link Labs

There’s so much going on in the Fabric community that it can be hard to keep up with it all. Semantic Link Labs is a great example: in the six months or so since I last had a proper look at it my colleague Michael Kovalsky has done a whole load of cool things and … Continue readi…

Introducing prompt_jev(): bringing Jev to Motherduck SQL

Classify text in SQL with prompt_jev(), powered by TypeSafe's Jev. 100,000 rows labeled in 40 seconds for $0.50 at frontier-LLM accuracy. Live on paid plans.

Bliki: Paracelsus Maxim

The difference between a medicine and a poison is dosage. Often we talk about certain habits, in programming or life, are good or bad. But few things are simple binaries. Some vary with context: reading a book is a good thing sitting in my garden, but not while driving my car. Bu…

Displaying The Output Of Detail Rows Expressions In Power BI Reports Using The Paginated Report Visual

In last week’s post I mentioned that while Power BI reports (unlike Excel PivotTables) do not support the Detail Rows Expression feature, it is possible to partially work around this limitation by using the paginated report visual. In this post I’ll show you how I was able to do…

DuckDB Ecosystem Newsletter : September 2026

The September 2026 DuckDB Ecosystem Newsletter: DuckLabs joins AWS and MotherDuck acquires Tower, DuckDB v2.0-alpha lands, table functions in pure Java, bulk loads into SQL Server, Zarr stores as SQL tables, an infinite canvas for your data, plus a community spotlight on Vladimir…

Fostering Agency in Daily Leadership

This is the third and final part of the series on agency. Part one explained what agency actually is, and part two looked at what a lack of agency looks like in practice. This part could almost be short. Most of it is already captured in the fifth principle of the Agile Manifesto…

Creating DAX functions with AI to remove duplicated code

How to leverage AI to refactor existing DAX code, remove duplicated parts, and centralize the common code in user-defined functions.

They All Write, Store, and Read Data: How to Pick The Right Database

SQLite, Postgres, DuckDB and 20 others: they all write, store and read data. The questions you ask every day tell you which one you need.

An Accidental Blackboard

Giles Edwards-Alexander reports that during an experiment to see how productive a team could be using fully agentic engineering practices, the team accidentally prompted the agents into creating a blackboard coordination system inside the git repository. more…

Creating DAX functions with AI to remove duplicated code

Code written by AI can contain many duplicate parts. User-defined functions are a great tool to simplify complex measures; this article shows how to leverage AI to build better DAX code. AI is getting better and better at generating DAX…

Data Engineering Weekly #287

The Weekly Data Engineering Newsletter

Maybe We Shouldn't Be Reviewing All This Code

TL;DR Or, perhaps the problem isn't that AI has broken code review, maybe it’s that we've been using code review to solve the wrong problems I was on a panel recently with Brian Houck from DX at Code Remix, hosted by Moderne. It was one of the more interesting panels I’ve done, l…

Restoring alternating rows in Power BI matrices with Fluent 2

Microsoft recently introduced a new default visual design style for Power BI named Fluent 2. I personally disagree with a few choices in the new defaults, including the removal of alternating row colors from Table and Matrix visuals. However, the…

Your DuckDB Flock Deserves A Mother: MotherDuck Now Offers Open Source DuckDB Support

We're launching commercial support for open-source DuckDB, so every organization can get the power of a modern data engine, with direct access to database experts.

Fragments: September 1

Like many readers, I’m wary of AI generated prose. Simon Wilison has written an LLM cliché highlighter - paste in some text, or a URL, and it will flag various patterns common to LLMs. It references a wikipedia page of signs of AI writing. That page points out that: Humans are no…

How to check table sizes in a Microsoft Fabric warehouse

If you've tried to check how much storage your Fabric warehouse tables consume, you've probably discovered that the usual SQL Server approaches don't work. This post explains why, what else doesn't work, and the one approach that does The post How to check table sizes in a Micros…

The Data Plane Is Yours: Dagster+ Hybrid on Kubernetes and Azure

This article draws on a Dagster+ Hybrid deployment on Kubernetes and Azure at a very large enterprise. It covers the work outside the Helm chart: workload ownership, isolation, networking, identity, sizing, and capacity.

What Leaders Do That Keep Scrum Teams Working Like Individuals

Scrum teams often work like individuals because the organization staffs, measures, interrupts, and rewards them that way. Leaders can change the conditions that make real teamwork possible.

(Don’t) Stop materializing your silver layer!

Why is your silver layer a set of tables? Challenging the fully materialized medallion pattern - with benchmark numbers and reproducible demo scripts. The post (Don’t) Stop materializing your silver layer! appeared first on Data Mozart.

Data Engineering Weekly #286

The Weekly Data Engineering Newsletter

Why Your Scrum Team Still Works Like Individuals

Learn how handoffs, individual task ownership, and status-report daily scrums keep Scrum teams from collaborating—and what to change next sprint.

Analyzing the performance impact of visual calculations

Visual calculations can either improve or degrade the performance of a report. Let's see how we ensure visual calculations have a positive impact on performance.

Data Engineering Weekly #285

The Weekly Data Engineering Newsletter

Lack of Agency – When the system swallows the human

In my previous article, I explored the concept of agency — the capacity and freedom of individuals to act independently, make choices, and meaningfully influence their circumstances. But what does it look like when agency is entirely absent? I learned the hard way when I tried to…

Analyzing the performance impact of visual calculations

Visual calculations can either improve or degrade the performance of a report. In this article, we outline how we ensure visual calculations have a positive impact on performance. The goal of visual calculations is to simplify some reports and calculations,…

Building an Operational Ontology: An E-Commerce Walkthrough

The write side of the ontology conversation: named actions, business rules, and write-back to the systems of record — a pattern already running at enterprise scale.

Leading Agile Initiatives: How Leaders Help Agile Change Succeed

An agile initiative is not mainly a one-time rollout of Scrum, Kanban, SAFe, Jira, new job titles, or a different meeting calendar.

Dynamic formatting by hierarchy level with ISINSCOPE and ISATLEVEL

How to apply a different formatting rule at each level of a hierarchy using ISINSCOPE in a measure or ISATLEVEL in a visual calculation.

Data Engineering Weekly #284

The Weekly Data Engineering Newsletter

Focusing Where We Can Have the Most Impact

Mountain Goat Software is shifting its training and coaching focus to private client work, where teams and leaders can apply agile to their real challenges.

Liar, liar! What to do when AI writes your pipelines confidently wrong?

Should we let AI write our pipelines? YES...But not the way you probably think. The post Liar, liar! What to do when AI writes your pipelines confidently wrong? appeared first on Data Mozart.

Data Engineering Weekly #283

The Weekly Data Engineering Newsletter

We Tried Agile and It Didn’t Work

“We tried agile and it didn’t work” is one of the most revealing things a leader can hear.

One-minute trap! How new Fabric Warehouse CU metering affect your workloads

Microsoft is moving Fabric Warehouse CU metering from per-query CPU time to per-workspace virtual-node time. That brings Fabric closer to Snowflake and Databricks, but it also changes the cost equation: dense workloads may benefit, while sparse, chatty workloads could get a surpr…

What an Ontology for AI Agents Actually Needs

How to think about the semantic system that makes an agent coherent, governable, and useful.

Why Scrum Isn’t Working Even Though You’re Doing Scrum

Story splitting helps teams turn large user stories into smaller, valuable pieces they can finish within a sprint without turning them into tasks.

Why data teams are emerging as leaders in AI agent adoption

New research from Microsoft and MIT Technology Review Insights reveals how organizations are building trust in AI agents—starting with structured data workflows and scaling through shared context and real-time intelligence. The post Why data teams are emerging as leaders in AI ag…

Data Engineering Weekly #282

The Weekly Data Engineering Newsletter

MobAI: From Copilot to Team Pilot

AI can make individuals faster. For leaders, the bigger question is whether it helps teams build shared context, judgment, and better decisions. AI can make individuals faster. It can also make teams more fragmented. That is one of the risks I see in how many organizations approa…

FabCon Europe 2026: The sessions we’re most excited to bring to Barcelona

Get a first look at FabCon Europe 2026 in Barcelona, with curated session picks across Microsoft Fabric, SQL, AI, real-time intelligence, and data engineering—plus insights into the innovations shaping the future of data. The post FabCon Europe 2026: The sessions we’re most excit…

On Benchmarking

Why a throughput number is not an architecture decision.

Beyond “Taking Responsibility”: Why Leading for Agency is the Future of Modern Management

In recent years, a new word has steadily climbed the ranks of leadership literature and corporate strategy sessions: Agency. While many managers intuitively grasp that it has something to do with employees taking ownership, the concept goes far deeper than simply “taking responsi…

Variable libraries in Microsoft Fabric: One value to rule them all!

Hardcoded values are the cockroaches of a data platform: you think you got them all, then Test starts reading from Prod and nobody notices for three days. Variable libraries in Microsoft Fabric put your configuration in one versioned place and let each stage resolve its own value…

Orchestration is More than Scheduling: Declarative Automation in Dagster

Define the outcome, not the orchestration. Declarative Automation lets you express your desired asset state while Dagster continuously handles the work needed to achieve it.

Catalyst Leadership: You Cannot Optimize the Whole System if Your Inner System Optimizes for Safety

Most leaders are not held back by a lack of competence. They are held back by an inner system that was built for a different kind of world. According to Bob Anderson and Bill Adams at Leadership Circle, their research suggests that 70-80% of leaders lead reactively. That does not…

Data, AI, and the case for continuous learning

Microsoft Data Days brings together training, certification pathways, and more—helping you build practical skills and stay ahead in a rapidly evolving field. The post Data, AI, and the case for continuous learning appeared first on Microsoft Fabric Blog.

Data Engineering Weekly #281

The Weekly Data Engineering Newsletter

When Planning Should Become A Shared Problem

Turn planning pressure into a shared conversation about options and tradeoffs.

Microsoft Build 2026: Building agentic apps with Microsoft Fabric and Microsoft Databases

Microsoft Build 2026 highlights advancements in app development with Microsoft Fabric and Microsoft Databases, emphasizing a unified data and AI platform for scalable, agentic applications. The post Microsoft Build 2026: Building agentic apps with Microsoft Fabric and Microsoft D…

Data Engineering Weekly #280

The Weekly Data Engineering Newsletter

Leading Complexity 2025 — Just Two Weeks Away!

We’re just two weeks away from the launch of Leading Complexity 2025, and we couldn’t be more excited to welcome the amazing group of participants who’ve signed up from around the world. This year marks the fourth season of the program — and the theme feels more urgent and releva…

CLUSTER BY in Fabric Warehouse: when the magic shows up and when it doesn’t!

Microsoft's announcement post for data clustering in Fabric Warehouse promised 24x speedups. So I built a 100-million-row clickstream table and tested it myself. The result is more nuanced than the marketing - and more useful for anyone deciding whether to actually adopt this fea…

Data Engineering Weekly #279

The Weekly Data Engineering Newsletter

How to Lead—and Not Be Led—by AI in Complexity

A conversation with long-time Crisp collaborator Jonathan ReamsSummary (Abstract) AI is accelerating our tools—and our context. In this Leading Complexity Podcast episode, Jonathan Reams argues that leaders must grow their inner capacity at least as fast as AI evolves, or risk be…

Hot, Cold, Gone: Caching and Retention in Microsoft Fabric Eventhouse

Two policies, three storage states, one rule connecting them. A newcomer's guide to caching, retention, and tiered storage in Microsoft Fabric Eventhouse. The post Hot, Cold, Gone: Caching and Retention in Microsoft Fabric Eventhouse appeared first on Data Mozart.

Classifying a Million Snowflake Columns in 9 Days, Solo, with Dagster

We’re consolidating a decade of legacy systems into Snowflake, which leaves us with over a million columns that all need to be classified and tagged for data governance. I built the system that does it: a tiered classification engine, a human-in-the-loop override workflow, and th…

Trying Out Lovable – A New Way to Build Websites with AI

Recently, I explored a new tool called Lovable. It’s an AI-powered platform that helps you build websites and web apps just by describing what you want. You write a short description (called a “prompt”), and Lovable builds the code and design for you. As someone who works with Ag…

Highlights from FabCon and SQLCon 2026: Where databases and Fabric come together

From technical sessions to hallway conversations, FabCon and SQLCon 2026 showcased the momentum behind the Fabric and SQL ecosystems. The post Highlights from FabCon and SQLCon 2026: Where databases and Fabric come together appeared first on Microsoft Fabric Blog.

Data Engineering Weekly #278

The Weekly Data Engineering Newsletter

Leveling Up Leadership: A 5-Year Journey Boosting Engagement at a Game Studio

How a long-term leadership program helped build stronger leaders, increase engagement, and boost collaboration across teams. The Challenge: When Engagement Slipped, Leadership Stepped Up When the executive team at a large game studio noticed a dip in employee engagement in their…

FabCon and SQLCon 2026: Unifying databases and Fabric on a single data platform

We're bring attendees together to share real experiences and solve challenges side-by-side. Only together can we move into meaningful results. The post FabCon and SQLCon 2026: Unifying databases and Fabric on a single data platform appeared first on Microsoft Fabric Blog.

Prefect is Acquiring Dagster

A letter from Nick Schrock, founder and creator of Dagster

Leading Complexity – Year 4: Navigating the Age of AI (and Everything Else)

Three years ago, we launched the Leading Complexity Program with a simple but urgent question: How do we lead when the world is changing at an exponential pace? Organizations everywhere were feeling the pressure of complexity—messy systems, rapid change, and no playbook for how t…

Advancing agentic AI with Microsoft databases across a unified data estate

Built on a consistent Microsoft SQL foundation from on premises to the cloud, Azure SQL brings AI capabilities directly into your database experience. The post Advancing agentic AI with Microsoft databases across a unified data estate appeared first on Microsoft Fabric Blog.

Community Showcase Part 2

Some of the most interesting Dagster projects come from the community. This post highlights creative community-built applications.

Mob Programming with AI: Inside a High-Performing Team’s Journey

Introduction What happens when you combine a passionate development team, a dash of Tesla-inspired thinking, and the principles of Mob Programming, with AI as an active participant? At Traton-Scania, the AI-Enablement team is doing just that, transforming how they build software…

Unpacking your top questions on agentic AI: The Shift podcast

We're launching a new podcast called "The Shift," which does exactly that. Grounded in questions we heard from you after Ignite news, we're releasing eight episodes that bring engineering, product, and strategy perspectives together. The post Unpacking your top questions on agent…

Operationalizing Data Orchestration: Best Practices for DevOps, Infra, and Code Locations

A complete guide with all the insights, tips, and some predictions for the data platform engineer, just like an Almanack provides, with practical information for daily life.

Three Strategies for Leading in Complexity

Summary of my talk from Agile Sverige 2025 A few years ago, I heard the quote: That felt true then—and it’s even more true now. After co-hosting the LeadingComplexity.com program for several years with over 30 world-class thought leaders on the topic of leading in complex environ…

How to Make the Architectural Case for Dagster

Mature orchestration environments often work operationally while still leaving critical data dependencies implicit. This post introduces the Orchestration Maturity Model, explains the architectural ceiling of job-centric systems, and shows how Dagster’s asset-aware approach helps…

How we deploy confidently with Canary Deployment

When you push to production multiple times a day, the stakes are high — and manual rollbacks slow. Here’s my experiences from deploying services using Canary Deployments for an important backend service for a large global streaming service. Why Canary Deployments? We adopted Cana…

Community Showcase Part 1

Some of the most interesting Dagster projects come from the community. This post highlights creative community-built applications.

1-2-4-All and Meetings People Actually Want to Attend

Have you ever found yourself stuck in a meeting where one person does all the talking… and everyone else just tunes out? Or where it’s so open-ended that nothing really gets done? Even with smart people in the room, the best ideas often never get heard. That’s precisely the probl…

How Dagster Compass Powers Brooklyn Data’s Self-Service Analytics

Text-to-analytics promises self-service access to data, but adoption depends on usability, governance, and trust. In this guest post, Brooklyn Data explains how it evaluated Compass, deployed it on top of Snowflake, and enabled teams to answer operational questions directly in Sl…

Unlock Your Team’s Full Potential: Dive into a Liberating Structures Immersion Workshop

Are you ready to revolutionize the way your team collaborates, innovates, and solves problems? Welcome to the world of Liberating Structures, where conventional meeting formats are transformed into dynamic, inclusive, and highly productive sessions. ## What Are Liberating Structu…

Snowflake Runs Your Data: Dagster Runs Everything Else

Snowflake increasingly handles transformation and data freshness internally through features like Dynamic Tables and Cortex. Dagster complements Snowflake by providing orchestration, lineage, automation, and cost visibility across your broader data platform from SQL-defined asset…

Align Your Strategies

In the past few weeks, I’ve received similar questions from two different companies, which I’d like to reflect on here. Both organizations were in the process of crafting strategies for critical areas: one for AI and the other for cybersecurity. Their question to me was whether i…

We Tried ty for Performance. It Found Real Bugs

We adopted Astral’s new Python type checker, ty, to speed up type checking in the Dagster monorepo. The performance gains were dramatic, but the bigger surprise was that ty caught real runtime bugs Pyright missed. Here’s what we learned migrating a large Python codebase increment…

Why your teams aren’t really empowered (and how to fix It)

What actually empowers a team? Despite all the talk about empowered teams in tech, there's often confusion about what real empowerment looks like. At its core, an empowered product team is given significant customer or business problems to solve, rather than being handed features…

The Dagster Almanack: From Complexity to Composability

A complete guide with all the insights, tips, and some predictions for the data platform engineer, just like an Almanack provides, with practical information for daily life.

Announcing the Dagster+ Terraform Provider

The Dagster+ Terraform provider lets platform teams manage deployments, access controls, alerting, and more as code. Define entire environments declaratively, review changes through pull requests, and integrate Dagster+ into your existing infrastructure workflows.

The Missing Half of the Enterprise Context Layer

AI agents that only understand business definitions without knowing whether the underlying pipeline actually succeeded are confidently wrong and operational context from the orchestrator is the missing piece.

How to Orchestrate Across Multiple Databricks Workspaces Without Losing Your Mind

Once your pipelines span multiple Databricks workspaces, you're no longer orchestrating a single system you're coordinating a distributed one.

Dagster 1.13: Octopus's Garden

Dagster skills, partitioned asset checks, state backed components, virtual assets, and stronger integrations.

Monorepos, the hub-and-spoke model, and Copybara

How we configure Copybara for bi-directional syncing to enable a hub-and-spoke model for Git repositories

Making Dagster Easier to Contribute to in an AI-Driven World

AI has made contributing to open source easier but reviewing contributions is still hard. At Dagster, we’re improving the contributor experience with smarter review tooling, clearer guidelines, and a focus on contributions that are easier to evaluate, merge, and maintain.

DataOps with Dagster: A Practical Guide to Building a Reliable Data Platform

DataOps is about building a system that provides visibility into what's happening and control over how it behaves

Unlocking the Full Value of Your Databricks

Standardizing on Databricks is a smart strategic move, but consolidation alone does not create a working operating model across teams, tools, and downstream systems. By pairing Databricks and Unity Catalog with Dagster, enterprises can add the coordination layer needed for depend…