Applying Natural Language Processing to Healthcare Text at Scale
This is a co-authored post written in collaboration with Moritz Steller, AI Evangelist, at John Snow Labs. Don't miss our virtual workshop, Extract Real-World Data with NLP, on July 15 to learn about our new NLP solutions. In 2015, HIMSS estimated that the healthcare industry in the U.S. produced 1.2 billion clinical documents. That’s a...
How to Build a Scalable Wide and Deep Product Recommender
Download the notebooks referenced throughout this article. I have a favorite coffee shop I’ve been visiting for years. When I walk in, the barista knows me by name and asks if I’d like my usual drink. Most of the time, the answer is “yes”, but every now and then, I see they have some seasonal...
Machine Learning-based Item Matching for Retailers and Brands
Item matching is a core function in online marketplaces. To ensure an optimized customer experience, retailers compare new and updated product information against existing listings to ensure consistency and avoid duplication. Online retailers may also compare their listings with those of their competitors to identify differences in price and inventory. Suppliers making products available across...
Data-driven Software: Towards the Future of Programming in Data Science
This is a guest authored post by Tim Hunter, data scientist, and Rocío Ventura Abreu, data scientist, of ABN AMRO Bank N.V. Data science is now placed at the center of business decision making thanks to the tremendous success of data-driven analytics. However, more stringent expectations around data quality control, reproducibility, auditability and ease of...
Attack of the Delta Clones (Against Disaster Recovery Availability Complexity)
Notebook: Using Deep Clone for Disaster Recovery with Delta Lake on Databricks For most businesses, the creation of a business continuity plan is crucial to ensure vital services, such as data stores, remain online in the event of a disaster, emergency or other issue. For many, it is mission critical that data teams can still...
7 Reasons to Learn PyTorch on Databricks
What expedites the process of learning new concepts, languages or systems? When learning a new task, do you look for analogs from skills you already possess? Across all learning endeavors, three favorable characteristics stand out: familiarity, clarity and simplicity. Familiarity eases the transition because of a recognizable link between the old and new ways of...
Fine-Grained Time Series Forecasting at Scale With Facebook Prophet and Apache Spark: Updated for Spark 3
Advances in time series forecasting are enabling retailers to generate more reliable demand forecasts. The challenge now is to produce these forecasts in a timely manner and at a level of granularity that allows the business to make precise adjustments to product inventories. Leveraging Apache Spark™ and Facebook Prophet, more and more enterprises facing these...
Upgrade Production Workloads to Be Safer, Easier, and Faster With Databricks Runtime 7.3 LTS
What a difference a year makes. One year ago, Databricks Runtime version (DBR) 6.4 was released -- followed by 8 more DBR releases. But now it’s time to plan for an upgrade to 7.3 for Long-Term Support (LTS) and compatibility, as support for DBR 6.4 will end on April 1, 2021. (Note that a new...
Analyzing Algorand Blockchain Data With Databricks Delta (Part 2)
This post was written in collaboration betweeen Eric Gieseke, principal software engineer at Algorand, and Anindita Mahapatra, solutions architect, Databricks. Algorand is a public, decentralized blockchain system that uses a proof of stake consensus protocol. It is fast and energy efficient, with a transaction commit time under five seconds and a throughput of one...
Accelerating ML Experimentation in MLflow
This fall, I interned with the ML team, which is responsible for building the tools and services that make it easy to do machine learning on Databricks. During my internship, I implemented several ease-of-use features in MLflow, an open-source machine learning lifecycle management project, and made enhancements to the Reproduce Run capability on the Databricks...
