Enable Azure blob storage (delta lake) as a source

[basland]

Today, more and more data platforms are setup on Azure delta lake instead of SQL Databases. If the Data API Builder would support exposing delta lake data over the API, that would open up a whole new set of use cases.

[MedAnd]

Interesting idea @basland! Imagine if we could use the data-api-builder in a Data Product / Data Mesh pattern, allowing access to both transactional and analytical data.

@aaronpowell - wondering if this would require some sort of compute layer though, in between delta lake storage and data-api-builder, something like serverless or dedicated pools for sql / data bricks, or data explorer?

[basland]

If I were to build such API from scratch I'd use Azure Functions with Python to "query" the blob files and return data in JSON form. Something like Databricks would do the same. Perhaps Functions are cheaper to run than Databricks?

[diallonortv]

You kind of alreay able to do that using Synapse serverless, Synapse serverless is an on demande engine capable of understanding delta lake https://learn.microsoft.com/en-us/azure/synapse-analytics/spark/apache-spark-what-is-delta-lake then you can plug data api builder on top of the sql engine of Synapse.
Regards

Vincent