My First 90 Days as a Databricks Engineering Leader

I recently joined Databricks as Site Lead for the Seattle office and lead engineering for the Partner Platform team. This career move builds off of my 18+ years in the data and platforms space. I started my career at Microsoft on the Excel team and most recently was VP of Engineering & Product for the Data Management team at Tableau. From working on pivot tables to visual data prep, I have enjoyed helping customers make better decisions through data.

So, why did I join Databricks? As we ramp up and expand our presence in the Seattle area, I spend a lot of time talking to future Bricksters (candidates) – this is one of the questions I hear the most. The short answer – Databricks has the right ingredients to truly democratize data and AI. Databricks has a strong history of innovation and execution, has attracted top talent in the field and has the customer reach to realize this vision. There is a lot of excitement and positive momentum, so I thought I’d explain how I approached my decision to join Databricks and reflect on the experience of my first 90 days here.

Inspiring mission to simplify and democratize data

Databricks is on a mission to simplify and democratize data and AI, helping data teams solve the hardest enterprise data challenges. With the Databricks Lakehouse Platform, customers can unify all their data, analytics and AI workloads on one simple platform. This is so powerful! In my short time here, I’ve learned about incredible customer stories like Regeneron and Shell, and the impact these customers are having on the world is truly inspirational.

With Databricks, Regeneron has accelerated drug target identification, increasing overall productivity and getting new drugs out faster and cheaper. If this pandemic has taught us anything, it’s that rapid medical innovation is vital, so helping companies like Regeneron process data faster can ultimately save lives. Shell is using Databricks to help deliver innovative energy solutions for a cleaner world. As a mom of two young boys, protecting our environment for future generations is so important.

These customer stories are not relegated to a specific industry or sector. We have the opportunity to help customers across various industries – from life sciences to non-profits, finance to technology and everything in between. Working at a company where you believe in the mission and get to be part of solving some of the larger problems affecting society is inspiring and unifying.

Collaborative, humble-smart co-workers

Working to solve challenging problems is way more fun and effective when you like who you work with. This brings me to one of the best parts of Databricks: the people! All of my colleagues and teammates are really knowledgeable about their areas and passionate about our customers and the opportunity for impact. It’s been really exciting to work with top talent across our various business functions. Our senior leaders truly care, many of them are co-founders and have shepherded the company from its infancy to a period of rapid growth. Whether it’s my manager Reynold (one of the co-founders of Dataricks who is the top contributor to Apache Spark), Vinod our SVP of engineering, or my engineering peers including:

• Lei engineering leader for the visualization team,
• Michalis engineering leader for the query processing and storage team,
• Sridhar engineering leader for the SQL Gateway team,
• Arik founder of Redash and architect
• Shant architect for Databricks SQL

This is a talented team I enjoy working with and learning from every day. I often tell my colleagues and teams that I spend more of my awake hours with my team than my two kids, so I want to make sure I work with collaborative, friendly, and smart colleagues.

Culture and speed of innovation

A great mission, great people and a proven culture of innovation is the secret sauce to success. Databricks has a history of innovation. It was started by the founders of Apache Spark, but has not stopped there. The company is constantly looking at ways to out innovate themselves. In a short time, the company has created MLflow, Delta Tables, Delta Sharing, Unity Catalog, Databricks SQL…the list goes on. Our teams are solving hard problems every day. Whether it’s launching and monitoring millions of virtual machines per day that process exabytes of data or building user experiences to simplify the complexity of data science, there’s opportunity here to sink your teeth into solving hard technical problems across the software stack.

In the 90 days I’ve been here, we launched the Seattle site, launched Partner Connect, set the official data warehousing performance record, announced GA of Databricks SQL and were named a leader in the Gartner Magic Quadrant for DBMS and DSML – making Databricks the only cloud-native vendor to achieve this. Wow – it’s been so humbling to be part of these incredible achievements. If this is what the first 3 months looks like – am eager to see what we accomplish as a team in 2022!

In my time here, I’ve also been impressed with our strong engineering culture: well thought-through design docs, investment in dev tooling and productivity, and strong end-to-end ownership across teams. Of course, fast growth doesn’t come without growing pains. For instance, our onboarding needs to constantly be revamped because we’re growing so quickly! What worked 6 months ago doesn’t work for an engineering team that has now more than doubled in size. Even with a great document culture, there’s still tribal knowledge, and it takes time to figure out who knows what. There’s so much opportunity to be part of growing the team, the product and the culture on this rocket ship.

We are hiring!

And yes - we’re hiring =) Our Seattle site is very strategic to our growth and we have core efforts being developed out of Seattle. You can read more in our blog post here.

Despite the challenges of COVID, we are developing an exciting culture. Before the holidays, our Seattle Bricksters got together to build gingerbread houses and enjoyed boba with members of the senior leadership team. Last month we had our first industry networking event in Bellevue, hosting over 50 engineering leaders who got to learn more about Databricks from our CTO Matei and SVP of engineering Vinod. Databricks and our leadership is committed to growing our presence locally.

I can’t wait to look back on this blog post of my first 90 days and see what else we will achieve down the road!