A Cyber Engineering Primer: Automated Tools for Compliance Auditing

Kyle Snavely, Senior Consultant, Cyber Engineering, Coalfire

Part Two in a Series
Other Post in the Series:

Cybersecurity practitioners sometimes forget to define and explain the terms we use during the course of our work. Thus, my colleagues and I have embarked on a series of posts that provide a primer on some of the most important cyber engineering practices. In this post, we will focus on configuration management (CM).

A large part of the CM task is to ensure your systems stay aligned to the baselines you have deployed in the environment. Baselines are the expected base values an organization has determined each configured system should meet. A baseline for a typical Windows server might include password complexity, audited events, and Windows Firewall settings. Maintaining and assessing configuration management can be extremely time consuming even if you are operating in a small environment. Because of this, many vendors have developed automated tools to audit configuration settings within enterprise environments.

Automated compliance scans authenticate to target systems to enumerate values such as entries in a configuration file or Windows Registry. The ‘actual values’ that are pulled from the target system are compared to the ‘expected value’ provided in the check definitions. Often, when there is not a perfect match between actual and expected, the check will result in a “fail.” This does not necessarily mean that the target system is out of compliance; it could be that the ‘expected value’ or the directory/registry path for the setting within the check needs to be updated with an organizationally defined value.

Configuration management becomes more challenging when your organization needs to step outside the default benchmarks and checks due to operational requirements. Additionally, some out-of-the-box audit configuration files may create false positives, false negatives, or not cover everything needed in your environment and will require modification. In this post, we’ll look at modifying configuration compliance checks and discuss configuration management tools, which can be leveraged to detect changes and remediate out-of-compliance configurations.

The process for monitoring compliance is straightforward. Security analysts configure tools with a defined baseline and then schedule the tools to check the hosts on a regular basis. Following the scans, analysts review the reports for out-of-compliance hosts, and the analyst creates tickets for remediation action to bring those systems back into compliance.

In most environments, a very large number of tickets are generated, so most organizations seek to automate remediation actions. Doing so enforces the baseline and forces administrators with privileged access to utilize the configuration management process while reducing on-the-fly changes to production systems. Remediation action also reduces the risk of a bad actor modifying settings without going unnoticed.

Customizing Compliance Checks

During the process of deploying a hardened configuration, documentation is created to capture changes and record justifications for deviations from the chosen baseline. Since most tools provide standard checks for a multitude of baselines, some customization is needed to accurately audit settings that were modified from defaults during the deployment phase. This in turn will create clean reports and reduce/eliminate post-reporting justification of failed checks.

There are several popular tools for automating compliance checks. Below, we will highlight how the process works for three popular vendors: Tenable Nessus, Qualys, and Rapid 7 Nexpose. These tools are the most common, come pre-packaged with popular checks for most operating systems and baselines, and allow for customization to organizationally defined values.

Nessus Audit
Tenable Nessus is shipped with a large library of configuration baseline checks including best practices, Center for Internet Security (CIS), and Defense Information Security Agency (DISA). These are great starting points to build from when customizing compliance checks to meet your unique environment requirements. Tenable provides thorough documentation that defines use syntax and keywords, which can be applied when creating custom audit files (https://support.tenable.com/support-center/nessus_compliance_reference.pdf). Nessus relies heavily on Regular Expressions (RegEx) for pattern matching when searching configurations for settings and comparing to those that are expected by the audit/baseline. Given the flexibility of RegEx, the native support for Windows Registry, and flat configuration files, powerful multistep checks can be written. It’s important to note, creating your own compliance checks with Nessus is largely a manual endeavor via your favorite text editor.

Example Syntax for .audit file check

The above snippet is from the Ubuntu CIS level 1 compliance audit file. In the case of this check, it is verifying Chrony is installed by querying the package manager (dpkg) and checking the output for a status showing the package is installed. Configuration files can be checked for perinate lines, and files can be checked for the appropriate permissions, ownership and so forth. Tenable provides quite a lot of flexibility with their compliance checks.

Qualys (Policy Compliance)
Policy Compliance is an add-on module from Qualys that provides configuration compliance scanning functionality with the ability to create and customize checks. Similar to Tenable Nessus, this module provides a library of common compliance benchmarks out of the box. But unlike Nessus, all custom editing is completed via a graphical user interface, which may provide a straightforward method of modification for auditors and administrators.

Source: https://www.qualys.com/docs/qualys-policy-compliance-guide.pdf

Source: https://www.qualys.com/docs/qualys-policy-compliance-guide.pdf

Rapid7 Nexpose also provides out-of-the-box compliance policies with Policy Manager, which can be copied and modified much like the two previous scanning tools. Rapid7 does require the purchase of the Policy Editor module to enable custom checks and modification to the bundled baselines. These settings can be modified using the graphical interface.

Source: https://nexpose.help.rapid7.com/docs/creating-a-custom-policy

Automating Baseline Compliance and Remediation with Alternative Tools
In addition to the popular compliance scanning tools noted above, configuration management tools can be utilized to verify and remediate out-of-compliance assets. Remediation works by comparing the current running configuration to a baseline that has been defined in the tool. Below, I’ve described several popular tools: Puppet, Chef, Powershell, and Microsoft’s SCCM, each of which allow for highly customizable configuration management and auditing.

Puppet monitors properties and triggers events when the desired state is out of sync with properties on the target. Sync issues with the desired state are broken down into two categories: corrective and intentional. A corrective event occurs when Puppet finds a property that is out of sync with the desired state. An intentional event occurs when the desired state is updated and is applied to a target property. In the case of configuration remediation, Puppet will make corrective changes to target properties ensuring at each runtime the target state is returned to the desired state. Alternatively, Puppet can be run in a no-op state where it reports on would-be changes but does not implement remediation automatically.

Chef Automate
Chef Automate’s compliance module contains multiple baseline profiles out of the box including CIS and some standard hardening checks. Chef also provides the ability to create custom profiles to match the needs of your environment by using the InSpec CLI.

Example of Chef Compliance Check Syntax. Source: https://learn.chef.io/modules/create-a-custom-profile#
After creating a new profile, it can be uploaded and run against the nodes in your system. Lastly, Chef Automate provides a graphical interface to review compliance and the ability to write cookbooks for automatic remediation of out-of-compliance settings.

System Center Configuration Manager (SCCM)
Microsoft provides provisions within SCCM for compliance monitoring and reporting. The tools are located within the assets and compliance portion of the management console, which allow you to define your own benchmark or apply pre-defined vendor settings. Customization options include configuration items and software updates. Each configuration setting can be specified as required, optional, or prohibited during creation in the graphical user interface. As of this writing, there is a pre-release feature to utilize SCAP extensions to import SCAP content into SCCM for compliance monitoring, which is a welcome ability for administrators working in the Federal space. As with most other CM tools, SCCM also provides the capability for remediation of non-compliant settings.

Powershell DSC Environment Analyzer
Microsoft’s Powershell Desired State Configuration (DSC) is a module within Powershell that allows administrators to compare the configuration of a target to the intended configuration created in a MOF file. DSC can be used in a push or pull configuration, meaning that the targets themselves can poll the configuration server for updates or they can be pushed out. DSC can be implemented to check the running configuration for compliance and remediate anything that does not match the description in the MOF files. Microsoft also created an additional module to report on compliance called DSC Environment Analyzer. The Environment Analyzer module can take the DSC data and churn out reporting to show administrators which targets and settings are not in alignment with the MOF defined baseline. The data can also be integrated into Power BI to generate high-quality reporting.

Configuration management is a critical component of an information system’s security posture. A properly maintained baseline allows an organization to satisfy many best-practice and regulatory requirements such as identification and authentication, auditing, and system communications. Quickly identifying out-of-compliance systems and remediating these misconfigurations is critical to maintaining a secure system. As systems grow in size and complexity, a mature automated method for assessing system configuration management is needed.

Kyle Snavely


Kyle Snavely — Senior Consultant, Cyber Engineering, Coalfire

Recent Posts

Post Topics



Accounting Agency AICPA Assessment assessments ASV audit AWS AWS Certified Cloud Practitioner AWS Certs AWS Summit bitcoin Black Hat Black Hat 2017 blockchain Blueborne Breach BSides BSidesLV Burp BYOD California Consumer Privacy Act careers CCPA Chertoff CISO cloud CMMC CoalfireOne Compliance Covid-19 credit cards C-Store Culture Cyber cyber attacks Cyber Engineering cyber incident Cyber Risk cyber threats cyberchrime cyberinsurance cybersecurity danger Dangers Data DDoS DevOps DevSecOps DFARS DFARS 7012 diacap diarmf Digital Forensics DoD DRG DSS e-banking Education encryption engineering ePHI Equifax Europe EU-US Privacy Shield federal FedRAMP financial services FISMA Foglight forensics Gartner Report GDPR Google Cloud NEXT '18 government GRC hack hacker hacking Halloween Health Healthcare heartbleed Higher Education HIMSS HIPAA HITECH HITRUST HITRUST CSF Horror Incident Response interview IoT ISO IT JAB JSON keylogging Kubernetes Vulnerability labs LAN law firms leadership legal legislation merchant mobile NESA News NH-ISAC NIST NIST 800-171 NIST SP 800-171 NotPetya NRF NYCCR O365 OCR of P2PE PA DSS PA-DSS password passwords Payments PCI PCI DSS penetration Penetration Testing pentesting Petya/NotPetya PHI Phishing Phising policy POODLE PowerShell Presidential Executive Order Privacy program Ransomware Retail Risk RSA RSA 2019 Safe Harbor Scanning Scans scary security security. SOC SOC 2 social social engineering Spectre Splunk Spooky Spraying Attack SSAE State Stories Story test Testing theft Virtualization Visa vulnerability Vulnerability management web Wifi women XSS