It’s not always easy to understand the trust value of a penetration testing until a critical security incident happens to you. Last year at Codemotion Rome, Giovanni Mellini is Head of Information, systems, and network security at ENAV S.pA., the Italian air navigation service provider, shared a demonstration based on a remote code execution critical incident he had faced in his work.
Giovanni is also the founder and president of Cyber Saiyan – a not for profit organization founded to promote social initiatives to spread cybersecurity and ethical hacking culture. Cyber Saiyan organizes RomHack, a free cybersecurity conference held yearly in Rome.
Giovanni demonstrated how we can detect remote code execution attacks and how can we mitigate the risk. This included how you can respond and what kind of countermeasures can be used to respond and react.
We’re sharing some of the central points of his presentation here, but you’ll want to view the video of his talk below to gain an appreciation of the live coding in detail.
The problem with Remote Code Execution attacks
In Remote Code Execution (RCE), an attacker is able to run code of their choosing with system-level privileges on a server that possesses the appropriate weakness. Once sufficiently compromised the attacker may be able to access any and all information on a server such as databases containing information that unsuspecting clients provided.
Giovanni shared “You can execute everything on the system just by uploading something. You can download software, do pivoting, moving to another server, shut down the system, by rebooting the serve. You have to fix this now.”
From penetration testing to bug detection
Giovanni began by sharing that a problem was unearthed during penetration testing: “We had a security check on a production system which revealed a critical security problem.”
He notes the importance of security by design, “When you do a project, a lot of people that have to work together and interact. In the ideal world, everyone is happy. However, it’s a huge effort for everyone to react to do something. After we deploy something it can be quite hard to fix, and go back and fix again. This is why we talk about the value of security by design. From my experience, if you don’t have security since the beginning of the project, probably, you will have security issues at the end.”
If you are fortunate enough to have an effective security process somewhere in your company, there is a chance you will intercept a security problem before it goes live. However, according to Giovanni “For us, this was not the case. We found out about the problem in the web application only after the IT department deployed it and it was being actively used.”
How to recover from Remote Code Execution
There are two options:
1. Shutdown your system until fixed
2. Mitigate the risk and keep the system online
Giovanni notes that that shutting down is the easiest solution, however, it takes time, which may not be appropriate for mission-critical applications: “You have to shut down, do a new fix, do a test again and come back when you’re done. It typically takes at least a month and during this month, this system is not available to uses.”
Or you can mitigate the risk and system while remaining online:
“Like a defender, you have to think about how can you detect this kind of vulnerability in the live system, so you can intercept this kind of behaviour. And after you intercept this, you can have an alert and with your security team, you can react and say okay, we found that the vulnerability is exploited, used by someone, and now we investigate.”
How we can mitigate this risk of the remote code execution attack and keep the system online?
Giovanni suggests you need to understand your scenario and collect relevant logs: to mitigate the risk first as a defender, you have to know this system and you have to know of the system works and you have to collect the data that can help you to find the vulnerability.
Specifically, the Context is IIS Web series +ASPX on Windows:
- Collect the logs
- Send to a central log collector
- Analyze
- Find a pattern
Giovanni showed a live demo of how to do this using Splunk.
“First of all, we have to find to the device and get these files in the put to the central see so useful. In this case, I use Splunk. You have to extract from the software, you have to think just about doing this operation.
This is in real-time so you have to work with adjuncts that are real-time. So you have to keep the logs while written. So the agents are listening to the device and when something is brought to the file they keep and send. So this is important for doing security – keep a copy of the logs.
Also for forensic activity, imagine that this remote system is compromised It is a good question if this is compromised, so we cannot rely for forensic analysis to the logs because the system is not secure for us. It’s something that we don’t trust anymore.”
Write your own detection rule:
- Identify pattern unique fields
- Lower the false positives
- Schedule a search for the pattern
- Create an alert
- Risk mitigation
Giovanni warns, “You have to keep in mind that you need to be very precise and lower the false positive because imagine that you work in a system with a lot of logs, if you have a false positive, this means that we are not working well. You have to be precise.”
Automation is everything
However, you cannot just put people in front of a screen looking at the logs, you need to automate. In this case, Giovanni’s team utilized a functionality of Splunk but he suggests that elastic search can also provide these capabilities.
“After we detect, we launch the script and so we can create a ticket on a service desk platform so other people can work on this and check for events.”
Giovanni stresses that when it comes to security, there is no magic potion and that often security problems can be attributed to a chain of missing controls/configurations.