Parameterized queries with PySpark
PySpark has always provided wonderful SQL and Python APIs for querying data. As of Databricks Runtime 12.1 and Apache Spark 3.4, parameterized queries...
Page 1
Named Arguments for SQL Functions
Today, we introduce the new availability of named arguments for SQL functions. With this feature, you can invoke functions in more flexible ways...
Introducing Python User-Defined Table Functions (UDTFs)
Apache Spark™ 3.5 and Databricks Runtime 14.0 have brought an exciting feature to the table: Python user-defined table functions (UDTFs). In this blog...
Apache Spark ❤️ Apache DataSketches: New Sketch-Based Approximate Distinct Counting
Introduction In this blog post, we'll explore a set of advanced SQL functions available within Apache Spark that leverage the HyperLogLog algorithm, enabling...
Introducing Apache Spark™ 3.5
Today, we are happy to announce the availability of Apache Spark™ 3.5 on Databricks as part of Databricks Runtime 14.0. We extend our...
Introducing Lakehouse Federation Capabilities in Unity Catalog
Lakehouse Federation is now in public preview! Data teams face many challenges to quickly access the right data primarily due to data fragmentation...
Introducing Apache Spark™ 3.4 for Databricks Runtime 13.0
Today, we are happy to announce the availability of Apache Spark™ 3.4 on Databricks as part of Databricks Runtime 13.0 . We extend...
New Built-in Functions for Databricks SQL
Built-in functions extend the power of SQL with specific transformations of values for common needs and use cases. For example, the LOG10 function...
What’s New With SQL User-Defined Functions
Since their initial release , SQL user-defined functions have become hugely popular among both Databricks Runtime and Databricks SQL customers. This simple yet...