Software Engineer - Billing Infrastructure - Databricks

Software Engineer – Billing Infrastructure

Mission

The goal of the Billing Infrastructure team is to give all customers pricing that emulates the value of the Databricks platform, visibility into their Databricks bills, and cost controls to help manage their accounts. To achieve this, we build data reporting pipelines that support the underlying pricing infrastructure supporting tens to hundreds of millions of DBUs (Databricks Units) across multiple clouds and regions, UIs that allow Databricks administrators to view and manage their bill, and APIs and integrations to downstream processors to handle payments for all customers.

Outcomes

  • Develop and extend the Databricks product. This implies, among others, writing software in Scala, Python, and Javascript, building data pipelines (Apache Spark, Apache Kafka), integrating with third-party applications, and interacting with cloud APIs (AWS, Azure, CloudFormation, Terraform).
  • Be responsible for full software development lifecycle – design, development, testing, operating in production.
  • Architect solutions to achieve a high level of reliability, scalability and security.
  • Communicate effectively with other engineers in the same team, with other teams and with various other stakeholders such as product managers.
  • Operate in an Agile development environment.

Competencies

  • 4+ years of production level experience in one of: Java, Scala, C++, or Python.
  • Experience in architecting, developing, deploying, and operating large scale distributed systems.
  • Experience with distributed data processing systems (Apache Spark, Apache Kafka).
  • Experience with cloud APIs (e.g. a public cloud such as AWS, Azure, GCP, or an advanced private cloud such as Google, Facebook).
  • Experience working on a SaaS platform or with Service-Oriented Architectures.
  • Experience with API development.
  • Good knowledge of SQL.
  • Experience with software security and systems that handle sensitive data.
  • Exposure to container technologies, such as Kubernetes, Docker.

About Databricks

Databricks’ mission is to accelerate innovation for its customers by unifying Data Science, Engineering and Business. Founded by the original creators of Apache Spark™, Databricks provides a Unified Analytics Platform for data science teams to collaborate with data engineering and lines of business to build data products. Users achieve faster time-to-value with Databricks by creating analytic workflows that go from ETL and interactive exploration to production. The company also makes it easier for its users to focus on their data by providing a fully managed, scalable, and secure cloud infrastructure that reduces operational complexity and total cost of ownership. Databricks, venture-backed by Andreessen Horowitz, NEA and Battery Ventures, among others, has a global customer base that includes Viacom, Shell, and HP.  For more information, visit www.databricks.com.

Apache, Apache Spark and Spark are trademarks of the Apache Software Foundation.

APPLY