A Different Perspective

9 Responses to “Free email services lose 1 – 2% of your e-mails”

1. I actually wonder where those statistics come from, and how other alternatives compare. My assumption is that this is based on reports of mail sent but never delivered.

   I host my own email server. I have no way of knowing, unless people report a problem, that mail is being lost. So it could be that figure would be accurate for me as well. I just don’t have a large enough body of subscribers to gather the data like the “big boys.”

   Dwayne
   March 11th, 2006

2. Uh, hold the phone. In my non-pseudonomynous life I own a good stake in a regional ISP and do this stuff daily. Before you go postal on 1-2% of missing email, here are some realities to consider:
   1) Google up some spam stats and you’ll see 50-85% of all email are spam, meaning that we could be perhaps talking about .5 to .05% of *valid* mail is being lost.
   2) Speaking of spam, .5% as a false positive result on a spam filter is believable. The result is undelivered mail; this factor coupled with the above one means that the vast majority of undelivered email is going to be actual or suspected spam. Valid emails lost are very small.
   3) As a business owner, I’m fully aware that the number of times that people tell me their cheque must have gotten lost in the mail exceeds by far the number of times it actually happened – but it does happen. Don’t have any stats on the loss of postal mail, but .5 to 1%? I’d believe that; some of it gets “returned” to sender but not all.b
   4) Misaddressed mail. Is it “missing” or just in the wrong place with no report of its waywardness? Applies to postal and email.
   5) “The web will always be a little bit broken.” This from the designers and operators of the Internet. You have to understand how the web works and understand that some amount missing packets – perhaps an email – is inevitable. It’s a tradeoff, we can have something that works 99.9% of the time or something that works not at all. Easy choice.
   6) Smaller hosts that promise 100% performance of a service like email delivery either don’t understand the issues (see #5) or are letting their marketing voice overspeak their technical voice. If packets route off of your network, you can’t guarantee the performance – period. If they don’t though, you don’t have the Internet. We have clients call and say they sent an email 10 minutes ago and it hasn’t arrived yet. My answer? We aren’t sending out a search party… although email appears instantaneous most of the time, there are occasions where a bad route will occur or some similar issue will delay delivery of the email. As long as it’s not a permenant error, it will be resent and most times will eventually find its way through after a bit of a delay. This goes back to understanding how the whole setup actually works (see #5 again).
   7) I would venture to guess that this claimed 1-2% of emails is not just the big free hosts, it’s universal – but it really isn’t affecting your life the way it’s made to sound. For the most part, it’s also not fixable, because an ISP or mail host can’t just scour the ether looking for something that it doesn’t know it was supposed to receive. (See all of the foregoing.)

   Dwayne, this probably fleshes out your comment a bit more… and fwiw, data from subscribers insisting they sent it but it never arrived is worthless. Don’t get me started on helpdesk stories!

   Alan, I think your offer is a good one, but honestly I don’t think it makes any difference. Owning your own domain is irrellevant, it’s all about the mail host. I receive hundreds of emails daily, and without a spam filter my inbox would be basically unusable. If that means losing the odd email, then that’s what it costs. Practically, I can’t even recall the last time someone sent something I didn’t receive… which would be a small fraction of a percent. As I calculated .05% above, that’d be 1 email in 10,000. And odds are, it was some kind of spam. I’m honestly not that concerned about it.

   I’ll avoid for now the discussion of the mail agent and the inevitable slanderous comments I would make about MS-Exchange skewing the stats by dropping most of the mail that gets dropped and how people should really use better software to – uh, oh, I said I wouldn’t get into this part.

   I do want to agree wholeheartedly with your WordPress recommendation though… running something like that on a proper hosting service really is not much more effort than using one of the bloghosts like blogger or typepad and so forth… and at least doing it this way, you have control of your own domain and content.

   At least that’s my 2 cents.

   Brother Maynard
   March 11th, 2006

3. Dwayne -

   I know that my server peforms better than 1-2%, which is what I reacted to. The .5% suggested by Brother Maynard may be possible, but I think is still a little high.

   Brother Maynard -

   You make some great points, though I think you do overstate in a couple of areas. The original post linked to written by Craig of Craig’s list states it’s not a spam filter issue, though the volume of spam could certainly skew the numbers as you’ve suggested.

   Also, spam filters are not necessary to nearly eliminate spam… yeah, you heard me right I use BoxTrapper, which requires a human element some do not like, but which is more effective and only results in lost email based on inaction by the sender. The first time, and only the first time, a sender sends from an email address to my email address, they must confirm they are human by replying to an auto-response. I have found this much more effective, and it is a service to my knowledge the free email providers do not offer. They force you to rely on their spam filters which do not always do the best job and lose email (which is evidently in addition to the 1-2% listed by Craig). I recently sent an email to the new spiritual-formation.com newsletter mailing list, and I noticed only 45% opened, even though all subscriptions are fairly new. The server uses the right protocols for sending to such a group. Now a few of them have images blocked, and thus tracking will be negated, so it’s probably closer to 70-80% opened. Most of the emails in the unopened list are either free email services or AOL (a whole nuther can of worms I should make another whole post on). Spam filters will inevitably block email you want and have even signed up for. BoxTrapper may allow through an occasional spam email (if someone has a virus and their Outlook is sending email from their email address, it will get through if it is on your whitelist).

   And I would note that owning your own domain and using that email may not make as much a difference as previously implied, it does make a difference in that and other areas. Controlling what, if any, spam filter or spam protection you use rather than what you are forced to use in the service can enhance reception rates.

   Also your numbers were not figured accurately. A much broader range is suggested by your spam numbers, both lower and higher. .0015 (if 1% is lost and 85% spam) on the low end and a full percent on the high end (if 2% is lost and 50% spam). Also .5% would be 1 in 2,000 and not ten. And as Dwayne pointed out in a comment on Jordon’s post, statistics don’t work that way any way. You’d still be losing 1-2% of legitimate email (thanks Dwayne, I should’ve caught that but didn’t!).

   Also, the advantage I stress most has nothing to do with those statistics. You are not bound to the free email services for eternity and subject to their changes. You can take your email set up through your domain hosting account to another hosting account if you’d like. You don’t have to change email to change service providers. That is HUGE, and probably the most important part of having your own domain name.

   And by the way, the last time I did not receive something was sent to my gmail account. It was a change of shoot days for an acting gig I booked. And it did not end up in the spam filter.

   Alan
   March 11th, 2006

4. Oh, I should mention your description of the way the internet works was excellent, and I’d never promote 100% deliverablility, though I do think even .5% seems unaccaptable if you are excluding spam filters. Including spam filters, there’s not much you can do because a lot of people are using some pretty awful filters.

   Alan
   March 11th, 2006

5. Alan,

   Some good discussion points there. I confess I didn’t really read the details that Craig gave, so if that’s thrown me off, it’s my fault for not reading thoroughly. I also haven’t read Jordon’s post on it. That said…

   - I’m not familiar with BoxTrapper, but my general response to anyone who makes me jump through hoops in order to email them is simply not to email them. Most of these involve visiting a website to confirm rather than replying via email which is better, but I generally just don’t. I get very annoyed when I post to a list and get autoresponses from mail plugins like that… anyone using such an application should take care to configure it properly. Since I’m in an Internet-related business, I don’t think I’ll ever use something like that because I get a lot of first contacts for sales inquiries via email, and I want to make it easier to get these sales leads, not harder.

   - I’m one of those people who decline to send any received or read receipts and deny images being displayed in my email. I don’t know how many others out there are like me on these points, but there should probably be more. Read receipts are requested by Outlook too easily (by default?) and most people that request simply have the feature turned on for all mail don’t know how to turn it off and don’t even see the receipts that come back, so why add unnecessary traffic? Images displayed in email are a privacy issue.

   - On the subject of spam filters, SpamBayes and SpamAssassin do a very credible job, assuming proper configuration and training. Commercial email hosts that do any serious level of mail hosting simply *must* provide some level of spam filtering, and this has been our experience. Not only is it a market expectation, but one of the only ways to really clamp down on the spam problem is for everyone that does mail hosting, from providers to private boxes, to take the problem seriously enough to start bouncing spam and dealing with it proactively. It can actually cut your bandwidth requirement. Spam filters generally will score each email as a percentage probability that it’s spam; on our system, x% score gets it bounced, and between x% and %x gets it flagged :SPAM: in the subject line but delivered anyway. Our attitude is that it’s better to get a small amount spam to sort through if it means you get all the ham. Of even the flagged ones, I’ve had one false positive (flagged but delivered) in a year, and it was a newsletter, so I could see why it got flagged.

   - You’re quite correct in the advantage that a domain name gives you… freedom. On its own it will do nothing about spam, but it does allow you to freely move your mail hosting to a provider that handles spam the way you want it done. I do recommend people own their domain, I’ve just never done so for anti-spam reasons. It makes sense though in the ability to select a mail host on that basis.

   - There are errors in my numbers. I was calculating .05% so it would be 1 in 20,000. On the other hand, these are just statistics without a detailed description of their origin and meaning, and I heard someplace that 80% of all statistics are just made up on the spot to prove a particular point ;^)

   - I agree that .5% of missing ham is too high, if the statistic is a percentage of *properly formed* emails *that arrive at the mail server.* There are many potential reasons for an email not to be delivered or deliverable, over which the mail host has no control; given those caveats, I’m sure that your mail server and ours both perform much better than 99.5%. I’m assuming that these reasons are included in the claimed 1-2% and are not spam-related, so our .05% or .1% (or whatever) are no longer the same statistic as the 1-2% one since we’ve changed the definition.

   Good discussion.

   Brother Maynard
   March 11th, 2006

6. BM -

   BoxTrapper is of the email reply variety and not the visit a webpage, so it takes just a few seconds to veryify one time. I think because of its effectiveness, it should not be a burden to people to reply.

   I do not use it on my business emails, for the reasons you have stated. But it can also be configured to allow for certain subject lines. So the contact form on this website, for example, will get through to me without an auto-reply even though it is the first email from them.

   If I know an email is comingn from someone, I’ll just add them to the white list. It only takes a few seconds of my time, and like I’ve said… Box Trapper is very effective at removing spam.

   I also offer SpamAssassin on my server. I use it on my business emails, but instead of activating the spam box feature I have the subject line tagged [SPAM] because I’ve found it to give a lot of false-positives, even when configured for maximum results. Though I must confess I’ve seen a drop in false-positives, so perhaps the software’s getting better (most of my server-side software gets updated automatically to keep everything on the front edge).

   And an element to this which adds to the original issue of the free email services (and lumping in AOL) is the email tax they’re charging companies to insure delivery (currently just AOL and Yahoo!). I know some have blown it out of proportion, but I feel the incentive to keep proper spam controls will dip severely once there is a financial incentive for the companies to gain paying customers who want to see emails get through.

   Stats can be twisted and manipulated… but it’s still 1 in 2,000 one percent would be 1 in 100 right? A .5 percent would be 1 in 200? so a .05% would be 1 in 2,000. Just wanted to show off my beautiful math chops…

   And of course Dwayne’s point about stastics not working that way is valid… if it is 1-2% of all emails not getting through, it is still 1-2% of all valid non-spam emails.

   I’ve found most of my customers don’t know the huge benefits of the hosting account, so I’m rewriting my welcome email so this spam information gets through to them.

   Alan
   March 11th, 2006

7. BoxTrapper sounds far less obnoxious than the others I’ve seen. SpamAssassin is one of the leading solutions; we’re using it as well. I believe when we set it up, we trained it with some tens of thousands of emails we’d collected, confirmed spam – we then ran it for a while internally before putting all of our clients on it. We do bounce spam over a threshold, and deliver flagged versions of the rest; we were very concerned about ham-preservation, but it’s been extremely accurate for us.

   I think we’re in agreement on the stats and their significance… now you’ve made me actually think about the math, your last post has the correct .05=1/2000 (mine were wrong and your prior was wrong at .5=1/2000).

   Now I think we’re basically agreed but down to wishing we could quantify the % of dropped emails that are not delivered through the host’s fault and the % not delivered through no fault of theirs. The latter should be universal, with the former being the real issue that could be compared or complained about between mail hosts.

   One thing that does occur to me is the way in which gmail supposedly has great spam filtering but afaik doesn’t accommodate the maybe-spam category the way that Spam-Assassin can be config’d to do. I could see wayward mail there. Of course, I don’t really know… I went and got myself a gmail account and then couldn’t figure where or when I would really use it, since I’ve got my own domain(s) and can access mail for them via webmail, mail client, or even PINE if the urge strikes me ;^) …anyway, the less maybe-spam you get in your free email account, the more likely you are to be losing some ham.

   Ah well. Maybe if we start sending 2 copies of everything, we’ll compensate ;^)

   -Bro.M.

   Brother Maynard
   March 11th, 2006

8. BM -

   The interaction on this post has been great. I hope other people find it as interesting as we do!

   I was particularly interested in Jordon’s post because I know Google is rolling out gmail for domains. It is currently in beta mode. I would not want to recommend that service if that 1-2% number is accurate and reflects email that is properly formatted and such as you have pointed out.

   Unfortunately, the original post seems to be hearsay where we are supposed to trust Craig’s word because of guys he knows at the major free email services. While I actually think Craig to be a trustworthy guy (from what I can tell of his business dealings and online persona), I’m not willing to blanketly trust his numbers. His post, though, does raise concern for me, as I do not think he would have posted what he did had he not felt there were legitimate concerns that should be aired. So I think there’s probably something going on, but I’m not sure we can trust it is as extreme as his speculative 1-2%

   Alan
   March 11th, 2006

9. Alan,

   The propellerheads in the crowd always stand out as the ones people get tired of listening to – but they’re always fascinated with the conversation. I dunno, maybe we lost everyone but I’ve enjoyed it!

   You’re right about the gmail domains. I liked Google when they were eating into Microsoft’s market, but not so much when they start snacking on mine.

   Cheers,

   Bro.M.

   Brother Maynard
   March 12th, 2006