When you understand the difference between deterministic and probabilistic threat detection, you can better choose the right mix of processes and tools that will keep your data, systems and users most secure.
Here is a spoiler, though: As you compare probabilistic and deterministic methods, you will likely conclude that both approaches are needed to some degree. That means you’re on the right track. When you employ both, you can use the strengths of each approach while mitigating their respective weaknesses. In other words, these methods are different but complementary.
To help you figure out when to use each method, we put together this overview. In each section, we start by defining terms, and then we delve into the pros and cons of using the approach to detect threats.
What is probabilistic threat detection?
Probabilistic threat detection involves the use of probability-based analytic methods to identify potential security threats or malicious activities within a system. This approach doesn’t rely on fixed (deterministic) rules or signatures alone. Instead, it relies on the likelihood—or probability—that certain behaviors or patterns may indicate the presence of a security threat.
Tools for probabilistic threat detection analyze various factors and assign weights to different indicators. That helps cybersecurity systems—and security teams—to prioritize and respond to potential threats based on their perceived risk.
This approach to threat detection presents advantages as well as challenges. Here’s a look at some of the pros and cons of using probabilistic and deterministic detections.
Pros
Let’s start with the pros of probabilistic threat detection.
- Adaptability to new threats. Probabilistic threat detection can help you identify new and evolving threats that may not have definitive signatures. Machine learning and behavioral analysis can adapt to changing attack tactics. Slight pivots in attacker tools and techniques won’t necessarily fake out these detection techniques.
- Reduced false positives to unknown threats. Probabilistic methods may result in fewer false negatives for threats that have not been seen before. That’s because these methods don’t require a perfect match to a known signature to send an alert. Probabilistic methods are inherently non-binary.
- Behavioral analysis. This is often part of probabilistic threat detection. It typically uses a baseline of normal system behavior. That, in turn, makes it easier to detect deviations that may indicate a security threat.
- Continuous learning. Machine learning models for probabilistic threat detection can continuously learn, incorporate feedback from security analysts, and adapt to changes in the threat landscape. That means their accuracy is not static and can improve over time.
Cons
Now, here is a rundown of some cons.
- False positives. Probabilistic methods will produce false positives. They rely on statistical models that might interpret unusual but benign behavior as a potential threat. That can lead to alerts on activities that aren’t malicious. Taken to extremes this can waste security analysts’ time. But making the models less sensitive can lead to false negatives. That’s why tuning is part of ongoing maintenance.
- Complexity and resource intensiveness. Implementing and maintaining probabilistic threat detection systems can be complex and demand a lot of resources. That is especially true when it comes to systems that use machine learning because they require a great deal of computing power and expertise to operate.
- Cost issues. Probabilistic methods and tools deal with uncertainty, which is a key design principle. So they may not be as cost effective as deterministic approaches for detecting well-known threats.
- Difficulty in interpreting results. It can be a challenge to understand the output of probabilistic models. You may have difficulty discerning why a particular activity is flagged as a potential threat, as the rationale is deep within the model. To interpret the results, you may need a deeper understanding of the underlying algorithms.
- Data quality and bias. The quality of training data has a direct impact on how effective probabilistic models are at threat detection. If the training data is biased, or if it isn’t representative of real-world scenarios, the model’s performance may suffer as a result. Garbage in, garbage out.
What is deterministic threat detection?
Deterministic threat detection assumes that analyses have definite outcomes. It also assumes that the likelihood of those outcomes can be determined with a binary level of precision. It follows that deterministic threat detection relies on fixed rules to identify threats. Like probabilistic threat detection, it has its own set of advantages and challenges. Here are some pros and cons to keep in mind.
Pros
Once again, we will lead off with the pros.
- High accuracy for known threats. Deterministic methods for threat detection can identify known threats with a high level of accuracy. These are threats that have clear signatures or patterns. So, deterministic methods are effective for flagging well-established malware and attack techniques.
- Low false positive rate. Deterministic threat detection tends to have a lower false positive rate for known threats. That’s because it looks for exact matches with known signatures. A hit is nearly certain to be a true positive.
- Clear actionable alerts. When a deterministic system identifies a threat based on a known signature, it sends a clear and actionable alert. That means security teams can respond with confidence. They have valuable insight into the nature of the identified threat because they are told what it is.
- Simplicity and efficiency. Tools that rely on deterministic methods are often easier to implement than probabilistic ones. They tend to be more resource efficient, too, as the answer is either yes or no.
Cons
Now, here’s is a list of the cons of deterministic threat detection.
- Ineffectiveness against unknown threats. Deterministic tools can struggle to identify threats that lack known patterns or elements. So they won’t be able to protect you against zero-day attacks or attacks where tools or techniques have been slightly morphed.
- Limited adaptability. The lack of adaptability to new or evolving threats means that deterministic systems may become outdated fast. To make sure that deterministic methods can keep up with the threat landscape, you must constantly update signature databases. And these databases can only be updated when unknown bad threats are converted into known bad threats.
- False negatives for polymorphic threats. Polymorphic malware can constantly change its code to evade signature-based detection. Deterministic methods won’t be able to identify these threats effectively. This often leads to false negatives.
- Dependency on regular updates. As noted earlier, deterministic systems rely heavily on signature databases, so they must be kept up to date. If updates are delayed or there’s a gap in coverage, the system may not be able to detect new threats. And typically someone needs to be patient zero before the signature can be updated. You can only hope that it isn’t you!
- Limited insight into behavioral patterns. Deterministic methods typically focus on specific elements or the detection of specific versions of malicious software. So, they may not provide insights into broader behavioral patterns. This can limit your ability to detect sophisticated attacks or incursions that have already progressed beyond the specific artifact that the signature detected.
Deterministic vs. probabilistic threat detection? The best choice is to use both
As you can see, there are many benefits and challenges with using deterministic and probabilistic threat detection individually. The reality is that you need to use both approaches in cybersecurity because your business continuously faces both known and unknown risks.
Probabilistic methods can help you to address emerging and evolving security threats. And deterministic methods can help you more cost effectively stop known threats. When you use both, you can create a more comprehensive and adaptive security posture for your business.
One thing is certain: To keep pace with today’s cyber risks, your security team needs both an advanced and reliable approach to detect known threats and stop attackers in their tracks. So, when you put together the processes and tools to address both probabilistic and deterministic risks, consider a solution like Proofpoint Shadow, which is a part of Proofpoint Identity Threat Defense.
Proofpoint Shadow provides a deterministic approach to threat detection, but one that is very unlike most traditional approaches. It uses widely distributed deceptions to lure attackers into engaging with them and setting off alerts to their presence. It effectively turns determinism on its head—instead of searching the environment for the attacker, it lets the attacker come to the detection system. These deceptions are hidden deeply within all of your company’s endpoints and are impossible to discern from legitimate resources.
They look like real files, file shares, RDP sessions, credentials, database connections, emails, chats, scripts and more—they’re just what attackers are looking for when they invade your environment and are attempting to escalate privilege and move laterally. Just when the attacker thinks they are safely past your organization’s security detections they are literally setting off alarms that only the good guys can hear. What’s more, deception-based detections have incredibly low false positives and negatives. When they are tripped you can be quite certain you have a real threat actor.
When an attacker engages with them, Proofpoint sends a real-time alert with detailed forensics to your security team. They can then use this information to make intelligent choices to stop the attack. While attackers can be innovative and crafty, they tend to take similar steps once they initially compromise an environment on their way to your organization’s IT crown jewels. Proofpoint understands this and our platform uses that insight to help you shut down known threats, fast.
To learn more about Proofpoint Shadow and how it uses a deterministic, deception-based approach, see this solution brief.