Skip to main content
Login
Why Databricks
Discover
For Executives
For Startups
Lakehouse Architecture
Mosaic Research
Customers
Featured Stories
See All Customers
Partners
Cloud Providers
Databricks on AWS, Azure, and GCP
Consulting & System Integrators
Experts to build, deploy and migrate to Databricks
Technology Partners
Connect your existing tools to your Lakehouse
C&SI Partner Program
Build, deploy or migrate to the Lakehouse
Data Partners
Access the ecosystem of data consumers
Partner Solutions
Find custom industry and migration solutions
Built on Databricks
Build, market and grow your business
Product
Databricks Platform
Platform Overview
A unified platform for data, analytics and AI
Data Management
Data reliability, security and performance
Sharing
An open, secure, zero-copy sharing for all data
Data Warehousing
Serverless data warehouse for SQL analytics
Governance
Unified governance for all data, analytics and AI assets
Real-Time Analytics
Real-time analytics, AI and applications made simple
Artificial Intelligence
Build and deploy ML and GenAI applications
Data Engineering
ETL and orchestration for batch and streaming data
Business Intelligence
Intelligent analytics for real-world data
Data Science
Collaborative data science at scale
Integrations and Data
Marketplace
Open marketplace for data, analytics and AI
IDE Integrations
Build on the Lakehouse in your favorite IDE
Partner Connect
Discover and integrate with the Databricks ecosystem
Pricing
Databricks Pricing
Explore product pricing, DBUs and more
Cost Calculator
Estimate your compute costs on any cloud
Open Source
Open Source Technologies
Learn more about the innovations behind the platform
Solutions
Databricks for Industries
Communications
Media and Entertainment
Financial Services
Public Sector
Healthcare & Life Sciences
Retail
Manufacturing
See All Industries
Cross Industry Solutions
Customer Data Platform
Cyber Security
Migration & Deployment
Data Migration
Professional Services
Solution Accelerators
Explore Accelerators
Move faster toward outcomes that matter
Resources
Training and Certification
Learning Overview
Hub for training, certification, events and more
Training Overview
Discover curriculum tailored to your needs
Databricks Academy
Sign in to the Databricks learning platform
Certification
Gain recognition and differentiation
University Alliance
Want to teach Databricks? See how.
Events
Data + AI Summit
Data + AI World Tour
Data Intelligence Days
Event Calendar
Blog and Podcasts
Databricks Blog
Explore news, product announcements, and more
Databricks Mosaic Research Blog
Discover the latest in our Gen AI research
Data Brew Podcast
Let’s talk data!
Champions of Data + AI Podcast
Insights from data leaders powering innovation
Get Help
Customer Support
Documentation
Community
Dive Deep
Resource Center
Demo Center
About
Company
Who We Are
Our Team
Databricks Ventures
Contact Us
Careers
Working at Databricks
Open Jobs
Press
Awards and Recognition
Newsroom
Security and Trust
Security and Trust
Get a Demo
Login
Contact Us
Try Databricks
Blog
/
Ruifeng Zheng
Ruifeng Zheng
Ruifeng Zheng's posts
Data Engineering
March 25, 2024
/
5 min read
PySpark in 2023: A Year in Review
Data Engineering
November 13, 2023
/
4 min read
Python Dependency Management in Spark Connect
