A Growing Symphony of Security Analytics Tools Needs Careful Orchestration

Dave McClure, Chief Strategist, Coalfire Federal

Security analytics toolsavailable to companies are increasing rapidly. However, cyber incident and vulnerability prevention, detection, response, and recovery times remain significant challenges as the types of attacks and attack vectors increase.  Newer cyber analytics using machine learning are of primary interest because rule-based or signature-based prevention tools struggle to detect or stop advanced cybersecurity threats.  CIOs and CISOs find that they often need to integrate or “orchestrate” existing cyber analytical tools, processes, and data into repeatable, automated workflows to fully support solid security operations activities.  Concurrently, architectural challenges flourish as cloud services, mobile usage and IoT devices rapidly generate increasing amounts of data, new systems endpoints, and network traffic flows. 

Our experience with providing consulting and engineering support for IT continuous monitoring and continuous diagnostics and mitigation and SIEM as well as vulnerability risk assessments and technical security assessments in the commercial market has led to the following discoveries:

Security analytics is not a single solution; it requires careful orchestration.

There are a wide range of commercial products and open-source tools available to perform cybersecurity analytics, but companies should not fool themselves.  The full value of enterprise security analytics cannot be gained by installing hardware or network appliances only.  The retail, financial, healthcare, payments, and hospitality sectors are building systems that ingest terabytes of security data, but experienced analysts can only read triaged data at a few events per minute.  Even with superb visualization tools, analysts will only be able to mentally process tens of events per minute.

AI has spawned machine learning and algorithmic tools that can help pare vulnerability datasets to a smaller size, but in-house analysts also must know what questions to ask for common use cases (e.g., cyberthreats, insider threats, data exfiltration, and user account access abuse or misuse).  As an old saying goes, “a fool with a tool is still a fool.”  Moreover, companies often have several security tools that are deployed in independent silos and many of them invoke duplicative capabilities from different suppliers, sometimes on the same system. Security analytics will need to connect these silos and automate processes and investigations across these tools, evolving to the point where they function as a “force multiplier” for better threat detection.

Accurate inventories of networks, systems, and endpoints are required.

Shadow IT – a combination of unauthorized and unidentified solutions -- is a growing problem across all market sectors.  In the federal government, for example, audits by the General Accountability Office and Inspectors’ Generals demonstrate the lack of complete knowledge about what resides on and interacts with government networks (hardware, software, mobile, IoT, etc.).  You simply can’t protect what you don’t know about. Our advanced testing teams and CDM teams often encounter much the same outcome in commercial firms.   Fortunately, new market tools are much better at tracing networks and detecting/identifying devices.  Security analytics correctly applied to network traffic helps to shine light into the shadows.

Trained people knowledgeable in advanced security analytical tools are essential.

CxOs and their CIOs must hire and retain the right team for security analytics and treat it as an ongoing investment.  This hiring is no easy task.  Security analysts need to be curious, explore the high value anomaly data they collect, trace unusual patterns, and follow the trail of an investigation wherever it leads. Looking at individual events or correlated events are not sufficient anymore.  This is being augmented by a rise in “offensive hunting” where highly trained analysts emulate bad actors (hired hacktivists, rogue nation-states, insider threats) and their tactics to penetrate networks, devices, application and system weaknesses. 

Advanced security analytics requires the bridging of two new professional domains -- hacking and data science. Most experts specialize in one or the other, not both.  Accordingly, both data scientists and security experts need to closely work together to enable the adequate use cases essential to good security diagnostics and continuous monitoring.  Companies still need competent security analysts to tweak models, confirm “good” versus “bad” anomalies and analyze critical outputs of agentless appliances.  While Red Teams and hunt teams are a separate role for cyber defense in larger organizations, skills and experience are in short supply and often an outsourced capability.  These capabilities combined with effective incident response processes are critical for continuous security improvements and a full understanding of different types of attacks, including commonly used phishing and other social engineering techniques.

More importantly, companies need to set up cyber analytics to fit their risk profile and threat vectors.  It is not uncommon to drop sophisticated tools into company networks without using skilled staff to do the required “tuning” to optimize detection and protection for specific and unique threat ecosystems.  One size does not fit all, but going with default settings is creating such an environment (which is easy to breach and design zero-day exploits with far ranging impacts).

Continuous monitoring must include continuous data sharing internally and externally.

We find that critical data for cyber analytics may not always be owned by the security departments.  It’s the business lines that are the data owners and access controllers.  Well-defined cyber and data governance and stakeholder management is needed to tackle this complication.  Additionally, proper processes and technology are key to collecting and delivering the right data.

Going forward, security officials should re-set current approaches by clearly defining what they want to achieve with cyber analytics followed by consideration of three key initial steps:

  • Firstly, if your company is relatively new to the cyber analytics space, it may be worthwhile to explore the open source versions of tools or, alternatively, pursue time-bound pilots or “proofs of value” that can be up and running with demonstrated results in days or weeks before jumping to advanced tool acquisitions.  This can help with up-front investment buy-in and gain you some experience with how the tools can work for your security needs.

  • Secondly, you should follow a strategy and roadmap that embrace “no single tool; no single database; and, no single approach to solving a cyber threat problem.”  Getting locked into a single vendor/solution given the evolution of the market could be premature, particularly if they are not tuned to changing threat environments.  Your security analytics can and should complement diverse strengths rather than compete against each other.​

  • Finally, setbacks should not be unexpected.  Security analytics will not be able to successfully solve all detection problems or pinpoint every threat.  They can, however, reduce alert volumes and false-positives, surface previously unknown threats, and uncover abusive insider threats.  CXOs must learn from each journey down thorny cybersecurity pathways…. from their own internal mistakes and those of others.  Continuous adaptation, learning and adjustment is simply a reality in such a complex and ever changing cyber threat environment.

1 According to Gartner, security analytics tools include Security Incident and Event Management (SIEM), User and Entity Behavior Analytics (UEBA), Intrusion Protection Systems (IPS), Network Traffic Analytics (NTA), Endpoint Protection Platforms (EPP), Endpoint Detection and Response (EDR), Data Loss Prevention (DLP), Data Exfiltration Analytics, and Identity and Access Management (IAM) analytics.

Dave McClure


Dave McClure — Chief Strategist, Coalfire Federal

Recent Posts

Post Topics



Accounting Agency AICPA Assessment assessments ASV audit AWS AWS Certified Cloud Practitioner AWS Certs AWS Summit bitcoin Black Hat Black Hat 2017 blockchain Blueborne Breach BSides BSidesLV Burp BYOD California Consumer Privacy Act careers CCPA Chertoff CISO cloud CMMC CoalfireOne Compliance Covid-19 credit cards C-Store Culture Cyber cyber attacks Cyber Engineering cyber incident Cyber Risk cyber threats cyberchrime cyberinsurance cybersecurity danger Dangers Data DDoS DevOps DevSecOps DFARS DFARS 7012 diacap diarmf Digital Forensics DoD DRG DSS e-banking Education encryption engineering ePHI Equifax Europe EU-US Privacy Shield federal FedRAMP financial services FISMA Foglight forensics Gartner Report GDPR Google Cloud NEXT '18 government GRC hack hacker hacking Halloween Health Healthcare heartbleed Higher Education HIMSS HIPAA HITECH HITRUST HITRUST CSF Horror Incident Response interview IoT ISO IT JAB JSON keylogging Kubernetes Vulnerability labs LAN law firms leadership legal legislation merchant mobile NESA News NH-ISAC NIST NIST 800-171 NIST SP 800-171 NotPetya NRF NYCCR O365 OCR of P2PE PA DSS PA-DSS password passwords Payments PCI PCI DSS penetration Penetration Testing pentesting Petya/NotPetya PHI Phishing Phising policy POODLE PowerShell Presidential Executive Order Privacy program Ransomware Retail Risk RSA RSA 2019 Safe Harbor Scanning Scans scary security security. SOC SOC 2 social social engineering Spectre Splunk Spooky Spraying Attack SSAE State Stories Story test Testing theft Virtualization Visa vulnerability Vulnerability management web Wifi women XSS