Question

How to use probability to detect email phishing

Answer 1

Phishing is the act of sending an email to a user falsely claiming to be an established legitimate enterprise in an attempt to scam the user into surrendering private information that will be used for identity theft.

Phishing email will typically direct the user to visit a website where they are asked to update personal information, such as a password, credit card, social security, or bank account numbers, that the legitimate organization already has. The website, however, is bogus and will capture and steal any information the user enters on the page When it comes to phishing emails designed to steal credentials, researchers at Proofpoint found that one in four are targeting Apple IDs. Microsoft Outlook credentials are the second-most targeted, with Google Drive credentials coming in third.While large phishing campaigns designed to steal Apple account credentials may be the most common, the most effective in terms of click rate are emails targeting Dropbox credentials.Click rates for smaller, more customized phishing campaigns are significantly higher. Not only is that incentivizing attackers to make their campaigns more targeted and convincing, it's also making the task of protecting organizations with phishing awareness programs alone increasingly dubious.

We research. You level up.

    

• Products
• Support
• Pricing
• Why Us
• Labs
• More

For business

Explained: Bayesian spam filtering

Posted: February 17, 2017 by Pieter Arntz

Bayesian spam filtering is based on Bayes rule, a statistical theorem that gives you the probability of an event. In Bayesian filtering it is used to give you the probability that a certain email is spam.

The name

Named after the statistician Rev. Thomas Bayes who provided an equation that basically allows new information to update the outcome of a probability calculation. The rule is also called the Bayes-Price rule after the mathematician Richard Price, as he recognized the importance of the theorem, made some corrections to Bayes’ work and put the rule to use.

Spam

When dealing with spam the theorem is used to calculate a probability whether a certain message is spam based on words in the title and message, learning from messages that were identified as spam and messages that were identified as not being spam (sometimes called ham).

False positives

The objective of the learning ability is to reduce the number of false positives. As annoying it might be to receive a spam message, it is worse to not receive a message from a customer just because he used a word that triggered the filter.

Scoring

Other methods often use simple scoring filters. If a message contains specific words a few points are added to that messages’ score and when it exceeds a certain score, the message is regarded as spam. Not only is this a very arbitrary method, it’s also a given that this will result in spammers changing their wording.

We research. You level up.

    

• Products
• Support
• Pricing
• Why Us
• Labs
• More

For business

Explained: Bayesian spam filtering

Posted: February 17, 2017 by Pieter Arntz

Bayesian spam filtering is based on Bayes rule, a statistical theorem that gives you the probability of an event. In Bayesian filtering it is used to give you the probability that a certain email is spam.

The name

Named after the statistician Rev. Thomas Bayes who provided an equation that basically allows new information to update the outcome of a probability calculation. The rule is also called the Bayes-Price rule after the mathematician Richard Price, as he recognized the importance of the theorem, made some corrections to Bayes’ work and put the rule to use.

Spam

When dealing with spam the theorem is used to calculate a probability whether a certain message is spam based on words in the title and message, learning from messages that were identified as spam and messages that were identified as not being spam (sometimes called ham).

False positives

The objective of the learning ability is to reduce the number of false positives. As annoying it might be to receive a spam message, it is worse to not receive a message from a customer just because he used a word that triggered the filter.

Scoring

Other methods often use simple scoring filters. If a message contains specific words a few points are added to that messages’ score and when it exceeds a certain score, the message is regarded as spam. Not only is this a very arbitrary method, it’s also a given that this will result in spammers changing their wording. Take for example “Viagra” which is a word that will surely give you a high score. As soon as spammers found that out they switched to variations like “V!agra” and so on. A cat and mouse game that will keep you busy creating new rules.

Learning

If the filtering is allowed for individual input the precision can be enhanced on a per-user base. Different users may attract specific forms of spam based on their online activities. Or what is spam to one person is a “must-read” newsletter to the next. Every time the user confirms or denies that a message is spam, the filtering process can calculate a more refined probability for the next occasion.

Poisoning

A downside of Bayesian filtering in cases of more or less targeted spam is that spammers will start using words or whole pieces of text that will lower the score. During prolonged use, these words might get associated with spam, which is called poisoning.

Bypasses

A few methods to bypass “bad word” filtering.

• The use of images to replace words that are known to raise the score

• Deliberate misspelling, as mentioned earlier.
• Using homograph letters, which are characters from other character-sets that look similar to letters in the messages’ character set. For example the Omicron from the Greek which looks exactly the same as an “O”, but has a different character encoding.

Conclusion

Bayesian filtering is a method of spam filtering that has a learning ability, although limited. Knowing how spam filters work will make it more clear how some messages get through and how you can make your own mails less prone to get caught in a spam filter.