Spam: Not every e-mail is desired.
How providers block the spam flood
The good news first: The number of spam mails in the worldwide mail volume has been steadily declining for years. While in 2008 an unbelievable 92.6 percent of the e-mails were still largely unrelated advertising, this share fell to 54.1 percent by 2017. According to the first figures for 2017, this trend seems to continue. But this is not a reason to sit back and look at the problem as done, on the contrary. Because another evaluation gives rise to the issue of spam even increased attention. According to data from Securelist, a Kaspersky Lab-powered security portal, the share of spam messages with malicious software in the appendix rose sharply last year.
How to avoid spam filters
The company's analysts are evaluating the feedback from their in-house anti-virus software, which in the third quarter of 2017 struck 73,066,751 times. In the first quarter, there were 41,088,551 announcements. Most of the mailings, namely 6.5 per cent, went to the account of the botnet Necurs, which among other things the Ransomware Locky spreads. In other words, users' mailboxes are less spam than they were a few years ago, but they should be handled with the utmost care.
Spam filter for client computer
The fact that the number of junk mails has declined significantly in recent years is, above all, a merit of the e-mail providers. They have installed filtering programs on their servers, all of which are clear as spam, and can be used to delete identifiable mails without further inquiry. The most effective weapon has been the Bayesian filter, a learning algorithm, which recognizes up to 90 percent of all commercial e-mails
The filter was named after the English mathematician Thomas Bayes, who in the 18th century dealt with probability calculation and formulated the sentence of Bayes, which serves as a mathematical basis for today's filter programs. The anti-spam software that works with a Bayesian filter estimates probabilities. It calculates the likelihood of an e-mail containing one or more defined words to be spam.
The Bayesian filter must be trained by the user before he can be used. To do this, the software is presented with numerous spam, but also normal mails and allows the texts to be analyzed. Each mail is marked as desired or undesirable by the user. The program now gradually aggregates lists of words, which typically occur in junk and in desired messages. These lists use it as the basis for its spam detection.
For example, if the terms Potency and Viagra appear in a message, the probability of a spam mail is quite high. On the other hand, it could, of course, also be the e-mail of a urologist to his patient. In order not to delete such serious messages inadvertently, the filter searches, even for words that occur in desired mails, in this example, for example, according to prescription or practice. If it finds it, it reduces the probability that it is a spam mail, and the message is delivered
Virtually every e-mail provider and any anti-spam software today uses a Bayesian filter. Since the providers daily millions of mails durchleuusen and the filters are further improved, the recognition rates are also increasing. However, the e-mail providers do not rely solely on a filter, but combine Bayes with other methods. These include, for example, Vipul's Razor, a network based on user feedback. You report junk e-mails to you to a distributed spam catalog, which is constantly updated due to feedback. For example, other detection methods search for URLs that appear in spam blacklists, or check whether the mails contain a digital signature that confirms the authenticity of the mail address (DomainKeys Identified Mail Signature, DKIM). Since many spammers send their messages now as image files to bypass the Bayesian filter, the anti-spam programs also check the ratio of text to image in a message.
For the advertising industry, spam filters are, of course, an annoyance, because there is always the danger that they also make serious offers undeliverable. On the Internet, therefore, you will find various offers of companies, which check before the shipping, whether the advertising, pardon: product information, the usual spam filter happens. For testing, they often use the Spamihilator software, a widely used filter for use on client computers. Parallel to this, tips for e-mail marketing are offered. One of the most effective measures is registering with a Whitelist, the reputable provider listed and queried by the e-mail providers. Popular services of this kind are for example The DNS Whitelist or The Spamhaus Whitelist.
Both lists check the reputation of the reported sender addresses, which means that a company can also be banished from these lists. Other frequently-read tips include avoiding repetitions for multiple mailings or using a CSA-certified newsletter software. There are also warnings about the use of words in upper-case letters, the setting of several exclamation marks in a row, texts relating to credits or loans, phrases such as "now access" or "just today" or the use of purchased address lists >
In addition to the mentioned whitelists, there are also blacklists that collect the IP addresses and / or the domain names of known spam senders and make them available for polling spam filters. One of the most popular and influential lists of this type is the spam service, SpamCop or URIBL.
Blacklists have the disadvantage that partly also serious providers land on them. This happened, for example, to the US retailer Gap in 2012. In the US, customers can also have the invoice for their purchases sent by e-mail. In a branch, however, an address was entered incorrectly into the database so that Gap sent a mail with a typing error. This was enough for The Spamhaus to put Gap's e-mail server on its blacklist. Subsequently, all mailings of the company were blocked by numerous providers. Since the incident occurred during the pre-Christmas period, the company suffered a delicate loss of sales.
In order to react quickly to such errors, a whole series of websites has been established, which check IP addresses to see if they appear in the various spam blacklists. Sites like blacklistalert.org, multirbl.valli.org or online mailserver Blacklist Checker match the addresses with dozens of spam lists and show if and where they are listed. Serious vendors should then contact their provider and at the same time request a de-listing at the Blacklist. However, a few days may pass before the entry is actually deleted.
Larger e-mail programs such as Outlook or Thunderbird bring their own spam filters. Microsoft is already pre-configured with its mail client, the user can only select the protection level and set mail addresses to a black or whitelist. Thunderbird, on the other hand, has a Bayesian filter that wants to be trained accordingly. And some anti-virus programs also contain their own spam filter, some of which are directly integrated into the user's mail client.
In addition, there are a number of special anti-spam programs that act as a proxy server for the mail clients and intercept advertising messages from POP3 or IMAP accounts. Which client is used by the user does not matter. However, the tools can not filter spam from Webmail services such as Gmail. Only Superspamkiller and Spamfighter can also access Livemail and Yahoo accounts.
The methodology of the tools is hardly different from that of the professional filters for the providers. They all use a Bayesian filter, which is initially fed and trained with examples, the longer the better. Most programs combine it with other methods and, for example, respond to certain keywords in the header and in the subject line of a mail. In addition, some candidates also access lists of junk mail signatures in the Internet. The most popular and, according to various tests, the best program, this is the open-source Spamihilator software
Because of the much larger data sets, the Bayes filters are often finer and more up-to-date than the filters of the private anti-spam tools. On the other hand, the programs offer the possibility to individually set them up for e-mails, which carry serious content but are nevertheless undesirable. The best thing to do is try using a tool like Spamihilator to reduce the amount of spam in your inbox.
No comments:
Post a Comment