Security Operations on the Data Lakehouse: Hunters SOC Platform is now available for Databricks customers

Cybersecurity is a big data problem: The growing volume and complexity of data flowing in and out of enterprises have created new cybersecurity challenges. Existing SIEM solutions cannot scale with the velocity of data growth without taxing security budgets and draining existing resources.

Today, cybersecurity company Hunters is announcing the availability of its SOC Platform for Databricks customers. For the first time, Databricks customers will be able to attain an end-to-end, security operations platform on their own Databricks Lakehouse Platform deployments, while keeping the flexibility of owning all the data and having the power to build their own additional security analytics on the Lakehouse.

Hunters SOC Platform is a modern SIEM alternative that ingests, normalizes and analyzes data from all security data sources of an organization, including endpoint telemetry, network traffic, identity management, and cloud infrastructure. Not only does Hunters provide a wider variety of security-related data integrations, but the platform also identifies threats in real-time across the attack surface and provides security teams prioritized incidents to handle, reducing the time needed to contain and remediate threats to the organization.

What can Databricks customers do with Hunters to create even more value?

Build a Security Data Lake

One of the biggest burdens of security teams today is managing the ingestion of terabytes of data from dozens of security products. Hunters eases this process with a state-of-the-art engine that provides scalable ingestion, monitoring and optimization. Moreover, it comes prebuilt with a large library of off-the-shelf integrations that can be set up in minutes.

Hunters SOC Platform ingests and performs the ETL of all security-related data into the customer's Databricks Lakehouse using the customer's cloud storage: the customer retains ownership of all the security data. The Hunters ETL follows the Databricks' Medallion Architecture model storing the raw data and also normalizing the data into a unified schema that will facilitate further analysis. While Hunters already provides a rich set of analytical capabilities, customers with advanced cybersecurity analytics teams can augment the Hunters capabilities by leveraging Databricks Data Science and Machine Learning capabilities and the partner technologies in the Databricks ecosystem. For example, many customers have AI/ML models for detecting threats that are highly specific and customized to their specific organizational context (eg. insider threats). Such detections are so specific that it does not make sense for a vendor like Hunters to build into their product. Hunters provides the flexibility for customers to leverage the Databricks lakehouse for such use cases.

Detect and Investigate Incidents

Hunters provides a library of hundreds of built-in detection rules that cover the majority of the threat landscape, mapped onto a common industry framework (the MITRE ATT&CK). This allows customers to visualize coverage and understand their security gaps. All detection rules are pre-verified on real-world customer data to minimize false positives and excessive alerting. The detection rules are deployed directly to all customer tenants without requiring any action or tweaking, thereby automatically reducing the cybersecurity risk with little operational overhead.

Each alert also passes through an investigation engine, where it is automatically enriched with contextual information from various sources, and sophisticated dynamic scoring is applied to it to reduce alert fatigue. Not all signals from the same detection logic require the same urgency. For example, alerts that involve sensitive assets (e.g., C-level, domain servers, etc.) are prioritized, and risk for known benign behaviors is lowered (e.g., an executable IoC signed by Microsoft). Addressing the priority of alerts or incidents with dynamic scoring helps security teams manage their SOC workloads more efficiently.

When the SOC analyst gets to an alert, all contextual information is provided in a single pane of glass to expedite triage and investigation. The contextual information goes beyond 'simple' enrichment of IP addresses with threat intelligence feeds, to deep correlation such as linking the user name in a CrowdStrike EDR alert with login records from the Okta authentication logs. Hunters' deep correlation capability is powered by a graph correlation engine: Alerts across entities and attack surfaces are automatically correlated on a graph. This graph correlation capability enables Hunters to highlight high-fidelity threat activity and provides analysts the flexibility to leverage low-fidelity signals that are often overlooked without generating more noise.

For incidents requiring investigations from multiple organizations including third party service providers and/or government agencies, Databricks provides cleanrooms where collaborating investigators can jointly investigate an incident using the relevant subsets of data and the custom analytics that can be proprietary to different organizations.

Search & Incident Response

Having all of your security data stored in a modern data lake has great advantages for incident responders, and anyone who wants to gain insights on massive amounts of data.

Using Hunters and Databricks, customers can not only store petabytes of data, but also make use of them in their day-to-day investigations and in their most critical incidents. Some capabilities that support this are the following:

• IOC Search: Purpose-built search capability, allows responders to search all organizational data ingested by Hunters that resides on the Lakehouse for IOCs (IP, domain, hash) in seconds within the SOC Platform itself.
• Entity Search: Makes it easy to see all information about an entity in the environment in a centralized place. For example, from one suspicious login alert, customers can easily pivot to see the latest logins of the user in question across all endpoints, cloud infrastructure, and SaaS providers. In the same user interface, a responder can observe which alerts a user in question was involved in, and what is their role in the organization. Entity related views create huge efficiencies and productivity for security teams.
• Raw data access: all of your security data is available for you to dive into as you see fit, both from within the Hunters console, and from your familiar Databricks interface. You can run queries on months of data to find that needle in a haystack, create operational dashboards that help expedite investigations, and run your own AI/ML models.

The openness of the Hunters and Databricks integration encourages security teams to innovate in their fight against cyber criminals. The Hunters SOC Platform not only helps security teams to do their day-to-day job more efficiently and effectively, but also provides all the data in a Databricks lakehouse where they can experiment, create, and test their own security analytics and AI/ML models and contribute these back to the cybersecurity community at large. Cybersecurity is a team sport. Let a thousand flowers bloom.

If you want to try Hunters out on your Databricks Lakehouse, please request a demo!