My Hopes for Feature Announcements at FabCon ‘24

My Hopes for Feature Announcements at FabCon ‘24

What is FabCon 24?

Europe’s first Microsoft Fabric community conference, labelled FabCon, is kicking off on September 24th, following the first global Fabric conference and preceding the 2025 event planned in Las Vegas next March/April (2025). As detailed on the conference website, a number of Microsoft’s Data & AI experts, including some of the Fabric product leadership will be in attendance.

Alongside a number of interesting speaker sessions, there is a planned session focusing on the Microsoft Fabric roadmap. On some online community forums, there has been talk around some announcements being “held back” over the last 1-2 months with the intention of providing some announcements during the conference this week.

My hopes for announcements

I thought it would be helpful to shortlist the top N announcements I’d like to see, but given the number of ongoing development activities and items covered both in the roadmap and Fabric ideas, I thought it helpful to break it down to the top 3 hopes related to known roadmap developments, top 3 unrelated to known roadmap developments, and top 3 other or more out there hoped-for announcements.

  • Version Control Updates - improvement in item support, especially for Dataflow Gen2, and ideally the ability to see the code that sits behind Dataflow Gen2 items to validate and version control
  • Data Exfiltration Protection - though it’s unlikely to be a killer feature for all customers, data exfiltration protection for notebooks would be important for the more security conscious users and not covered by OneSecurity (planned) or Purview
  • T-SQL improvements (e.g. MERGE, CTEs) - I think the current T-SQL functionality meets most core needs, but does provide some challenge in specific needs not being met (e.g. MERGE). I also see some ongoing questions around the pattern of looking to move on-premise data to Azure SQL before feeding in to Fabric vs. Ingesting to Fabric immediately, and given where Azure SQL functionality is, there’s a compelling argument to move there first
  • While I tried to stick with top 3, I have to add that the incremental load support in dataflows is another roadmap item that I would love to see more about this week

Unrelated to roadmap hopes

  • Workload Management (queues, prioritisation, adjustable burst behaviours) - I think there are lots of options as to how this is implemented, but ultimately I believe there would be a lot of value to being able to prioritise certain jobs or workspaces, and it would encourage use of larger capacities rather than splitting all non-production and production workloads at the capacity level only, which is currently the only way to facilitate workload segregation
  • Low-code UPSERT for data pipelines and dataflows - similar to the utility around implementing incremental load support above, being able to UPSERT via data pipelines would be a great quality of life improvement, especially considering it’s already possible in dataverse dataflows
  • Parameters / environment variable support (for notebooks and between experiences) - I have seen a number of examples around metadata driven pipelines already, and made use of functionality to utilise parameters for dataflows, but I think that being able to do this within notebooks and across experiences (between data pipelines and dataflows) as well as for deployment pipelines would be great

Others

  • Preview Features going GA - Fabric is maturing rapidly, and also adding lots of new features regularly, but it would be great to see a few bits of information around things becoming generally available. First of all, for longer-standing features, It would be good to see some features move from preview to GA. I’d also like to understand what the cadence or path from preview to GA looks like for upcoming features, as well as get more visibility of when things are actually happening and what the impact is. We have the roadmap for new features, but it’s more difficult to get a clear view on how this impacts existing features or environments
  • Native mirroring for on-prem SQL Server - currently, mirroring on-prem data sources somewhat relies on setup via an AzureSQL instance. In theory, this doesn’t provide much of a blocker, but in practice, native integration to onprem SQL Servers for mirroring would provide a much better user experience
  • Asset Tagging - a few months ago, Microsoft added Folder support to help organise Workspace assets, which do their job, but the addition of tags could be more user-friendly by extending the current “certified” and “favourite” options in OneLake and hopefully provide additional options for security and governance (e.g. attaching RBAC to tags)