Social Engineering- Beyond the Baseline

Brandon Edmunds, Senior Security Consultant, Coalfire Labs

Coalfire Labs does a lot of Social Engineering testing. Traditional Social Engineering testing involves a mundane process of taking a sample of a population and then attacking those “targets” with some pretext calls or a phishing email in order to obtain credentials. Metrics are recorded and then reported back in some form of a deliverable, usually a report. As an example, in a standard Social Engineering engagement, we had a Pretext Calling campaign that included a target selection of 10 users. We made 10 phone calls and talked three of the targeted people out of their passwords.  Along with these calls, we delivered a phishing email attack as well, directed to a larger group of 100 users.  In response to the phishing email, 12 users submitted their credentials. In our engagement report, we noted a baseline for that engagement was a 30% fail rate in pretext calls and a 9% fail rate on the phishing campaign (or “success rate” if you’re looking at this the way our pentesters do).  These numbers quantitatively indicate that social engineering is a risk to the tested environment. However, does this provide a true baseline for measuring risk?  Even if repeated annually?

If it does, then it begs the question, will this be indicative of how the company will truly respond in a real attack situation? I suggest the answer is no, for several reasons. The quality of campaign could vary from year-to-year, the person or persons responding to the threat could just be having an off day, recent events may have influenced the response or any number of other variations could come into play and interfere with the results, skewing the baseline. Beyond that, the baseline failure percentage rate likely means little to executives – 9% failure on the phishing campaign is 91% success, which on the surface sounds like pretty good performance. These baseline results show that the client is somewhat susceptible to a Social Engineering attack, but does it show impact to business assets? Does this testing prove the Incident Response team can do their job effectively?     

The weakest link
More recently than the example above, Coalfire Labs conducted a Social Engineering campaign against a large bank. Coalfire Labs sent out 160 emails and made 30 pretext calls and from all of that, solicited a single successful response, which was an email. From the traditional, quantitative perspective, that bank has a less than 1% fail rate. Combine those results with, being well protected by firewalls, intrusion-detection-systems, corporate antivirus, a strong patch policy, web content filtering and email SPAM protection (all of which Coalfire later discovered were indeed in place), and one would construe that the less-than-one-percent failure rate supports the conclusion that this bank has a solid Information Security program.  Fortunately, this phishing campaign in which Coalfire Labs launched was in support of going beyond the baseline.  Our client understood the aforementioned statistical concern – that success is still failure – and had asked us to go as far as we could with any access gained.  So we did.

1% failure = 100% success...for the attacker
The phishing campaign included sending malware, so that when the single user clicked the link we immediately had access to their system. This shell was launched in memory, undetected by many defenses layers and connected directly into a server in our datacenter. One click and the game was on.  With this shell access, we were able to find and ‘steal’ intellectual property, enumerate sensitive company procedures and exploit additional vulnerabilities on other systems inside their network, ultimately compromising a Domain Administrator’s password.  Had we been this bank’s actual real-world adversary, we could have easily made this access persistent and gone on undetected with unauthorized remote access for an extended period of time.  Sound familiar?  That’s such a common attack pattern that the industry has termed this the “APT”, or Advanced Persistent Threat.

Prepare for real-world attacks!
Simply stated: by going beyond the baseline, Coalfire Labs was able to identify a true weakness in a seemingly locked down environment. These results would have gone overlooked in a traditional-style Social Engineering campaign. The bank would have most likely saw the 1% fail rate and considered that an acceptable risk. Instead, Coalfire Labs was able to demonstrate the weakness and tie that to a real world threat, which provides that client with ammunition for future training and improvements for implementing stronger security across their architecture. A baseline Social Engineering test where failures are recorded and metrics are reported is not enough for preparing your environment for today’s attacks. This has been true for recent big name attacks that have been in the news lately. Successful attacks by real attackers are not going to take a “cookie-cutter” approach, so testing your environment for resilience against such an attack should not use one either. It is time to go beyond the baseline and utilize real scenarios and tie Social Engineering testing back to impact against the assets that are at risk.

Brandon Edmunds


Brandon Edmunds — Senior Security Consultant, Coalfire Labs

Recent Posts

Post Topics