In my last blog post “The Existential Crisis of a WAF,” I talked through the consequences of an attack getting through either by a rule not matching, a device misconfiguration, or traffic obfuscation. The latter includes the inability to decrypt and parse the traffic, which was the case with the Equifax breach. I also discussed Trusted Execution™, a new technology that provides huge value over the rule-based network security devices, including:
In this second installment, I thought I would explore the true cost in man-hours of a rule-based network security device, such as the WAF, NG Firewall, IDS/IPS. I am not going to put a number to the below scenario, but instead I want you to think about what the real cost is to you and your organization.
But first, I want to take a stroll down Memory Lane (no overflows, please…). When I was a young buck in the Air Force, I became an expert at the various IDS and firewall devices that were deployed at the time. One of the firewalls I was managing had several protocol “proxies,” including HTTP that made it capable of understanding the upper layers of the OSI stack web traffic and enforcing a negative security model for that protocol. In this sense, you can say that this firewall was the predecessor to the WAF. This firewall protected the base’s web page, and several employee portals, which was cutting-edge at the time. Every day I would come in and sift through the firewall logs to make sure no big red flags were present. I thought I was so cool, being the only one on base to be able to read the mountains of log entries.
That all changed when I was selected to go to a DOD Unix Security Course, where I was first introduced to the Buffer Overflow exploitations in the Unix finger and sendmail services, which were the exploitations that allowed the famed Morris Worm to be so successful in 1988 (https://en.wikipedia.org/wiki/Morris_worm).
I learned how to think like a hacker: carefully performing target selection, reconnaissance, enumeration, exploitation, and finally ways to cover my tracks. I also learned methods to meticulously lock down the system in order to defend against hacking. To conclude the course, everyone in our training competed in a “Capture the Flag” exercise, which was both exhilarating and enlightening. In the end, I was completely pumped about cybersecurity. Who knew learning hacking techniques would open up new possibilities and turn my professional world upside down?! Sure, in the past I dabbled in hacking, but that was more with services like SMTP and FTP, and rudimentary password cracking. Slackware Linux was just gaining popularity as a free Unix-ish Operating System you can use for hacking, so serious hacking was restricted to licensed Unix and Solaris systems, which was cost-prohibitive to the novice.
Knowing more also had its challenges. When I got back into my daily job reviewing firewall logs, I started to ask the tough “What If” questions: What if I messed up the firewall and exposed us? What if an attack got through? What if there is a hacker in our systems RIGHT NOW??? I could get court-martialed for incompetence! Shortly after my Unix Security training, the Air Force decided to classify all Information Systems as Weapon Systems. This meant that now we had to perform an extensive security audit, classification, and readiness accreditation of every IT system on base. I had to draft the contingency plans to cover the possibility of a compromised system. That led to an extremely important question – perhaps the most important question a security guy can ask:
“If we were compromised, how would I know?”
Fast-forward to today, and guess what? That question is still a very valid one – How do you know if you are compromised? If you manage ANY security product, the one fact you have to accept is that you don’t know what you don’t know. Meaning, if an attack is not recognized by your security device for any of the reasons mentioned at the beginning of this post, can you prove that you were (or weren’t) compromised?
Consider this scenario: If I create a script to scan for all 34 documented CVEs that are related to Nginx web server https://www.cvedetails.com/vendor/10048/Nginx.html (assuming each of them can be tested from the network), and I slip in one or two Zero-Day tests that actually get through (Yay Me!), for a single IP address targeted, what count of WAF alerts will you have? IDS alerts? Firewall alerts?
Thankfully, you may have a SIEM that gives you a grand total of 153 alerts (a completely arbitrary number) for the total count of devices in the path, including that shiny new RASP you just installed on the targeted server. Now what? What is the contingency plan to make sure every security device did its job and blocked the 34 attacks? Or is it 36? Or is it 100? Do you really know???
It just so happens that attack number 35 was a file-less buffer-overflow that was not yet released to the public, and I now own the box with at least the privileges of the Nginx process. (Inquire about our Nginx Attack Demo.) I am now “living off the land” until I am discovered. This period is called Dwell Time, and the clock is ticking until I am discovered, so I have to work fast to secure the beachhead and cover my tracks. I am thinking to myself that I generated enough “noise” with the other attacks, I am keeping you (the Security Administrator) busy trying to figure out what just happened. Chances are good you won’t find out I’m in your network for awhile.
So, now what? What is your course of action if you think an attack may have happened but you’re not sure? Without the sufficient evidence that the breach attempt was successful, can you request that the server be quarantined? How long will it take to sift through all the alerts and document the attack to give enough compelling evidence to justify disrupting business? Can the server just be replaced with a known-good image? What is the real cost of reacting to an attack? Or of not reacting? AND will you always react this way EVERY SINGLE TIME you see a flutter of alerts coming from an opportunistic web scanner?
I now have to ask another “What If” question: What if you had a tool in your toolbox that showed exactly what did, or did not happen to that Nginx web server? What if you could open up the Virsec UI and see that an attack got through the network, but Virsec did its job and killed the offensive spawned process? Virsec’s Trusted Execution technology detected the buffer overflow, saw the Nginx process jump from an authorized memory location to an unauthorized memory location, then triggered a Micro-Protection action that killed the offending Process ID, returning the application to smooth operation. No guesswork, no forensics, no quarantining, just a copasetic server humming along, serving out its share of inconvenienced electrons in the form of web pages.
In the example below, the Virsec system is in monitoring mode so we can see the entire Kill Chain of the attack. Starting from the bottom line, working up, it starts with a Buffer Overflow , where the system detects a jump to a memory location that is not intended by the application. We then see hostname executed, followed by a wget that creates a file, modifies it, then deletes it.
Usually Buffer Overflow attacks are extremely hard to detect with conventional security products, but Virsec would have killed the offending process at the first alert, thereby stopping attackers in their tracks.
The result? You cut costs in the form of staff-hours, prove value in defending the network, and safeguard your customer’s data from being stolen. You become the hero and they place a bronze statue of you down in the lobby. Your next stop? CISO!
I hope you enjoyed this post, and for those of you who are maintaining industry certifications, don’t forget to put in for your CPEs when you read any of our blog posts. Cheers!
Further resources:
The Existential Crisis of a WAF