Threat Hunting: Do lower search costs produce better results?
SIEM
Intellyx BrainBlog for Securonix by Jason English
Companies now expect to acquire security tools on a pay-as-you go, OpEx basis, and vendors who didn’t start out as managed service providers are now trying to adapt their pricing models to more flexible ones for Hybrid IT environments. Maintaining premium pricing is difficult in today’s competitive vendor market, and customers inevitably want to realize the cost savings promise of a multi-tenant cloud service for security.
Even with more flexible vendor licensing options, many enterprises are not only failing to save money, they are starting to watch cloud storage fees balloon exponentially once a newer service-based SIEM starts operating and threat hunters get to work.
In most of our careers in IT, we usually bargain for cost-cutting measures to result in reduced performance levels and inferior quality of service. However, in the area of reduced search and data storage costs, we may be able to turn that maxim on its head.
As it turns out, if search performance levels remain good, and SIEM cost-of-entry and cloud storage prices are much lower, the incentives are aligned to make SOC analysis and threat hunting practices far more frequent and productive at the same time.
Time and space in the cloud is money
IT services vendors frequently recommend and co-manage a suite of security tools for small and mid-sized companies, whereas larger enterprises usually have a dedicated SecOps team who purchases and operates a SIEM platform in conjunction with other tools.
The average cost of entry for a SIEM platform was historically quite high — tens-or-hundreds-of-thousands of dollars for the largest companies, depending on how servers and teams were configured and licensed, but the initial cost of entry is just the start of today’s cost concerns.
The value and risk accounting for security investments is changing, and therefore, the old on-premises focused way of licensing by facility, or server, or by geo-located team, or by SOC, must also change.
The rest of the enterprise’s software portfolio is already moving to cloud-based hosting, SaaS-based services and microservices-based applications, available with elastic scaling of compute and capacity. The proliferation of momentary API connections to external services, and ephemeral container-based servers that can appear and disappear in seconds makes the old licensing paradigms untenable.
Where can we keep all the data we need?
“To get an idea of how important a SIEM is, consider the scale of the security incidents and data involved. A large enterprise may generate more than 25,000 events per second (EPS) and require 50 TB or more of data storage.” — Paul Shread, eSecurity Planet, July 2020
Data is the lifeblood of the enterprise. The dual ability to live search the wide user and system data flowing through all the nodes of an enterprise, while searching deeply against long-term data trends and anomalies, comprises the majority of any threat reduction effort.
Gartner recently predicted (rather safely, I’d say) that storage requirements for any SIEM environment can be expected to double every year. Unfortunately, while some storage costs are dropping, the on-demand storage required for a SecOps team environment may not fall as fast. This means data storage costs could quickly outstrip other security IT investments by a wide margin.
One reasonable approach to this problem was separating system event data into live short-term storage (or ‘hot’ storage) that is ready in an instant for more current or critical security queries, and long-term storage (‘cool’ or ‘cold’ storage) for historical data that is stored more cheaply in some form of archive or data lake, but is less readily accessible and much slower to search.
Where to set that threshold though, if threat hunters want access to as much relevant data as they can get?
Constrained resources demand multi-tenancy
In addition to data costs, enterprise search can also hammer compute resources, especially in a collaborative environment, where threat hunters are rightfully empowered.
Teams may want to use a SIEM to access huge volumes of logs — or run a ‘virtual SNORT’ –across the whole IDS plane for signs of attackers or vulnerabilities.
These searches may involve looking for specific text strings, correlation with UEBA data collections, software configuration update info, local network settings and global community-gathered hazards, all of which require heavy processing to correlate for relevant actions. If multiple teams dip into the enterprise data lake and try to share the work, it significantly drains allocated cloud resources.
While leading cloud providers already offered burst capacity for certain transactional workloads, recently AWS gave more data and content oriented S3 buckets and CDNs a real upgrade to support the processing side of the equation for SIEMs as well.
Managed container orchestration allows enterprise search compute workloads and data to be loaded in on-demand very quickly from ‘hot’ sources like AWS EMR, but also rather quickly from lower-cost ‘warm’ sources like the Athena serverless platform.
Securonix is one leading SIEM vendor taking full advantage of this new paradigm with its new multi-tenant Securonix SearchMore offering.
A confluence of updates to their SaaS SIEM platform, including the decoupling of compute and storage, the discovery of active threats in raw data, and the ability to work as a community-powered threat hunting team can reduce the mean time to detect threats, while lowering the costs of long-term search, and improving threat hunting.
Enriching, parsing and indexing data to a security data lake is a well-automated process, but recent improvements mean that searches can even take place against raw data to identify threats in real-time. Think of it as ‘live intrusion detection for everything,’
On the long-term data search side, historically moving security data to offline or cheaper storage has reduced baseline analytics capability, and increased search times. Securonix’s embrace of serverless compute can reduce such storage costs to 1/3 of historical norms, while keeping baselines running exponentially, and returning search results in operationally acceptable timeframes.
Multiple SecOps teams can concurrently operate their own consoles, to conduct real-time searches, long-term searches and threat hunting activities across massive incoming content feeds from live apps and long-term storage, correlated with community resources like MITRE ATT&CK and Sigma Community, and Securonix threat research, as well as sharing queries with other teams in the organization to promote future successes.
The Intellyx Take
As microservices patterns and globally distributed cloud application delivery rapidly grow to prominence, our threat detection and enterprise search capabilities must also scale to meet the needs of this much larger theater of operations.
Threat hunting has always been an integral part of a complete SecOps practice. We want threat hunters to have free range — to collect logs, research global vulnerabilities, predict upcoming threat patterns, and catch incoming attacks — before they can make an impact on our application environments, and end users.
We started the migration of our applications to the cloud for the undeniable advantages of elastic scaling to meet customer demand, pay-as-you go cost efficiency and the ability to leverage the exact right services for the job.
Rather than accepting this modern theater of operations as an expanded threat surface, why not put all the advantages of cloud to work for our security teams – and set them free to set their search sights wider, enriched with a deeper historical range of data?
Let’s never disincentivize threat hunting efforts again. If we can finally solve for enterprise search capacity and cost constraints in the cloud, then the success of threat detection, avoidance and mitigation efforts depend only upon the intelligence and commitment of the organization.
