Research Survey: Productivity benefits from Databricks Assistant
In the fast-paced landscape of data science and engineering, integrating Artificial Intelligence (AI) has become integral for enhancing productivity. We’ve seen many tools...
Over the past few months, we’ve been gathering your feedback and focusing on both the quality of Databricks Assistant’s responses and the overall user experience. Today, we're excited to showcase a more advanced Databricks Assistant, packed with powerful new features all designed to simplify prompt engineering and accelerate your workflows.
Key enhancements include:
- @-mentioning tables: Users can now @-mention tables within Assistant prompts, ensuring the Assistant references the correct table for more accurate suggestions.
- Inline improvements: Users can highlight specific lines within cells and initialize the inline Assistant, focusing suggestions on selected lines or query sections.
- Assistant Usage logs: A new system table (system.access.assistant_events) has been introduced to help admins and managers track Assistant usage and engagement.
- Diagnosing job errors: Diagnose job errors directly from the Workflows/Jobs pages.
- AI-generated filters: Create filters in the new results table using only natural language.
- Optimize SQL Queries: Leverage the Assistant to optimize potential inefficient SQL queries.
- Assistant Quick Fix: For certain cell errors, the Assistant will automatically suggest single-line fixes within 1-3 seconds.
@-Tables
Users can now directly @-mention tables in Assistant prompts to improve direct responses. Upon hitting @, users will be provided with a list of tables sorted in order of relevancy. Users can also @-mention multiple tables in prompts. On the right-hand side, users can see full directory/catalog paths.
Inline Improvements
Previously, the inline Assistant could only operate on entire cells or queries, often regenerating the entire block of code, even when only minor adjustments were needed. Now, we’ve introduced the ability to highlight specific lines within cells or queries, enabling you to target exactly where the Assistant should make changes. This enhancement gives you more control and speeds up response generation.
Assistant Usage Logs
Admins and managers can now track Assistant adoption and engagement with the newly introduced Assistant system table (system.access.assistant_events). Each row in this table logs user interactions with the side panel or inline chat.
We’ve also created a custom sample dashboard allowing you to quickly visualize information like:
- Active users by day and month
- Active users per workspace
- Top users overall
- Submissions data: per workspace and total
Diagnosing Job Failures
Users can now use the Assistant directly within broken jobs to better understand and correct errors.
AI-Generated Filters
Use the Assistant to quickly filter result outputs using natural language. Filters do not require statement re-execution and can be chained together to drill into specific insights.
Improve SQL Efficiency
Leverage syntax highlight warnings and the /optimize command to improve inefficient SQL queries. Optimizations pop up in real-time, helping you quickly identify inefficiencies before execution.
Assistant Quick Fix
Quick Fix is a powerful new feature that automatically resolves common cell errors in SQL or Python we have high confidence can be addressed by an LLM. This includes errors like trailing commas, typos, syntax mistakes, and more. Designed for speed, Quick Fix delivers suggestions in just 1-3 seconds and is seamlessly integrated with keyboard shortcuts.
Try Databricks Assistant Today!
To see Databricks Assistant in action check out our demo video to see how you can use Assistant to build data pipelines, SQL queries, and data visualizations. Learn other ways to use the Databricks Assistant to increase your developer productivity by checking out our blog on Tips and Tricks on using the Databricks Assistant
Try Databricks for free
