False Positives, False Negatives and Tooling
Beware of False prophets:
Something we have encountered with our clients when using MSSP’s (Managed Security Service Providers) is in relation to tools and validation. Tools are necessary to discover security weaknesses across the fullstack which is nothing new. Fullstack visibility of security controls is key when operating a robust vulnerability management operation but there are some pitfalls…..
We’ve seen false positives being promoted to “real issues” resulting in service tickets being raised and assigned to SoC (Security Operations Centre) operations staff. After some time it is discovered that this issue is not real and the FP is simply a result of the tool not interpreting a result of a test correctly. The negative impact is simply wasting staff time, from developers to operations staff and all in-between in order to close-off the false alarm. This detracts from the SoC efficiency and time that should be spent on real issues when following a risk based approach to cyber security.
Even worse we have seen Real issues being marked as false positives resulting in a false negative. So a real issue is ignored. This is obviously much worse, a cyber security issue has just been closed off and ignored but why?…..
A little knowledge is a dangerous thing…
We found that in some cases the SoC operations staff simply did not understand the vulnerability or they couldn’t reproduce it or validate the issue properly. This is not uncommon due to the fact that the SoC operations staff are not experienced enough to validate complex or non standard issues
Jack of all trades, master of none:
A weakness of a generic SoC is they simply don’t have the experienced staff to manage, understand and validate complex vulnerabilities.
There is a lack of bespoke tooling for validation also and have reliance on generic security tools. – Protection of an enterprise takes time and experience, there are many areas for automation which reduces the workload but some aspects of cyber security take human intelligence. Simple “Vanilla”
Most SoC operations staff, frankly dont stay for long in that position as it is relatively low paid and provides minimal experience and exposure.
Attackers dont generally rely on rooms full of screens, generic out-of-a-box tools and have significant skill.
This is something to consider, food for thought?