Microsoft Ignite Debrief for Data & AI

Microsoft Ignite Debrief for Data & AI

Intro & Context

Microsoft Ignite 2024, Microsoft’s flagship annual event, took place a few weeks ago, at the end of November. Having had time to digest the various sessions and announcements, I wanted to share my highlights and key takeaways related to data and AI. I won’t be covering all announcements as I’m keen to share some personal views on 5 or so areas and a list of what I consider to be the more interesting things announced but, for those interested, the Microsoft book of news is the one-stop-shop for all announcements. Fabric-specific announcements can be seen here.

It’s worth mentioning that most of the announcements below, other than those around general availability, aren’t usually available immediately. Some minor updates are but otherwise you can expect most features to be coming in 2025. 

My Key Takeaways

Fabric Databases enabling PBI writeback
  • SQL Databases in Microsoft Fabric - SQL Server workloads made available through Fabric, extending analytical capability to include a Fabric data store for transactional workloads. While this is interesting on its own merit in the sense that it offers a way for organisations to consolidate transactional and analytical databases, the real reason I’ve included this as one of my favourites was down to the additional announcement that this will enable functionality in 2025 for Power BI writeback where users can input data through Power BI and write to a backend database - awesome. 
  • Purview Investment - there is a whole section in the book of news on Purview that’s worth digesting - a couple of specific call outs include renaming Purview Data Catalog to Purview Unified Catalog, extending data quality support for Fabric, Databricks, Snowflake and more, extending DLP capability, and Microsoft Purview Analytics in OneLake. In my opinion, though, the key takeaway here was that Microsoft are clearly investing heavily in Purview. I would argue that’s somewhat overdue, but it will be an interesting space to keep a close eye on over the next 6-18 months
  • Azure AI - Azure AI Foundry is a rebranding and extension of functionality for the existing Azure AI Studio, AI Foundry is also going to provide prebuilt templates and an AI Agent service. Other AI announcements include AI reports covering impact assessments covering project details such as model cards, model versions, content safety filter configurations and evaluation metrics, an AI scenario in the Cloud Adoption Framework, and an Azure Well Architected AI scenario.
  • Copilot - I’m reasonably confident that Copilot will be part of most, if not all, Microsoft events for the foreseeable future. Broader announcements included things like Agents in SharePoint and Project Manager Agent (that I’m interested to try), but I think my takeaway from Ignite was that there was a focus on customising, integrating, and extending purpose built Copilots. I felt this was slightly more targeted than the usual Copilot off the shelf type of conversation. From the Data and AI angle, the integration between Copilot Studio and AI foundry is interesting as it unlocks the capability to utilise AI Search as well as bringing your own models (or using industry specific models). Since Ignite wrapped up, Microsoft also announced GitHub Copilot free for VSCode.
  • OneLake Catalog - “a complete solution to explore, manage, and govern your entire Fabric data estate.“ I see this as bridging the gap between existing management functionality (e.g. lineage) and what sits in Purview, and I’m sure that many users who use OneLake Catalog extensively will be attracted to further data quality and security functionality of Purview. For me, two key improvements OneLake Catalog brings are the ability to see metadata for all OneLake assets - not just lakehouses but pipelines, notebooks, etc. - such as tags, lineage, and the ability to view previous runs, and the new “govern” tab that gives a view of your whole data estate including sensitivity label coverage
  • General Availability of sustainability data solutions in Microsoft Fabric - it’s always refreshing to see some Fabric functionality get the GA label, but alongside that, I think the current ESG solutions through Microsoft are either the Sustainability Manager ($4k per month), CSRD reporting templates through Purview (limited scope), or to build your own. This could be a very interesting way to meet ESG data and reporting requirements through Microsoft Fabric. There was a spokesperson from an existing customer who mentioned using this capability of implementing new use cases in short sprints within two to six weeks - I can’t wait to get my hands on it
  • Fabric AutoML (or AI Functions in Fabric) - I got a preview of this during a couple of sessions at FabCon Europe in September, but Ignite marks the preview availability of auto ML, enabling implementations of various algorithms and AI functions in just one or two lines of code. This also ties in to a study Microsoft funded that, unsurprisingly, indicated the biggest challenge with AI adoption is skills. It’s always a fine line in balancing low-code capability and maintaining the quality of deeply technical products, but I personally welcome the AutoML functionality to support some rapid prototyping work

Other Announcements

  • Tenant switcher - for many people, this will seem inconsequential, but as someone that works across different tenancies in the nature of my role, this is such a quality of life update to be able to switch tenant without having to log out and log in
  • The general availability of external data sharing allows you to directly share OneLake tables and folders with other Fabric tenants in an easy, quick, and secure manner
  • Fabric Billing Update - organizations with multiple capacities can now direct Copilot in Fabric consumption and billing to a specific capacity, no matter where the Copilot in Fabric usage actually takes place. It’s worth noting this means you would still need an F64+ capacity, but Copilot could be triggered from an F4 for example. In an ideal world, this would be further improved so having 2xF32 capacities (i.e. anything totalling F64+) would also be able to utilise copilot, but it’s good progress
  • Fabric Surge Protection - helps protect capacities from unexpected surges in background workload consumption. Admins can use surge protection to set a limit on background activity consumption, which will prevent background jobs from starting when reached
  • SQL Server 2025 - now available in preview
  • Fabric General availability announcements - the most notable is probably Fabric Real Time Intelligence, but this also includes sustainability data solutions mentioned above, the API for GraphQL, Azure SQL DB Mirroring, the Workload Development Kit, and external data sharing. All great to see
  • Fabric Workspace monitoring - detailed diagnostic logs for workspaces to troubleshoot performance issues, capacity performance, and data downtime
  • Fabric integration to Esri ArcGIS - preview of integration with Esri ArcGIS for advanced spatial analytics
  • Fabric Open Mirroring - a feature that allows any application or data provider to write change data directly into a mirrored database within Fabric, released to preview
  • Power BI core visuals - okay, this isn’t quite available yet, but it’s interesting to see that Microsoft are sharing a bit more about upcoming visualisations for Power BI, and also that it looks so aesthetically pleasing. The core visuals vision board is available here