Skip to main content
Takuya Ueshin

Takuya Ueshin

Takuya Ueshin's posts

PySpark in 2023: A Year in Review

Data Engineering

March 25, 2024/5 min read

PySpark in 2023: A Year in Review

pyhton

Data Engineering

November 7, 2023/5 min read

Introducing Python User-Defined Table Functions (UDTFs)

Arrow-optimized Python UDFs in Apache Spark™ 3.5

Data Engineering

November 6, 2023/4 min read

Arrow-optimized Python UDFs in Apache Spark™ 3.5

Memory Profiling in PySpark

Data Engineering

November 29, 2022/4 min read

Memory Profiling in PySpark

How to Profile PySpark

Open Source

October 6, 2022/4 min read

How to Profile PySpark

dbt-blog-og-3

Partners

December 6, 2021/2 min read

Deploying dbt on Databricks Just Got Even Simpler

python-autocomplete-og

Open Source

December 15, 2020/4 min read

Python Autocomplete Improvements for Databricks Notebooks

Interoperability between Apache Spark and Koalas

Solutions

August 11, 2020/16 min read

Interoperability between Koalas and Apache Spark

Data Intelligence Platforms

Product

June 24, 2020/6 min read

Introducing Koalas 1.0

Learn how easy it is to convert code written in pandas to Koalas on Apache Spark, includes best practices and how to use Koalas-specific APIs to improve productivity.

Solutions

March 30, 2020/16 min read

10 Minutes from pandas to Koalas on Apache Spark

Data Intelligence Platforms

Open Source

November 16, 2018/4 min read

Introducing New Built-in and Higher-Order Functions for Complex Data Types in Apache Spark 2.4