Sep 292008
The false positive problem with Gmail continues, in the last few days the following have wrongly gone into the spam bucket
- notification of my bank statement (never used to go into spam, so this is a new problem)
- notifications about my toddler’s music class
- my son’s class list
- Bob Park’s “What’s New” newsletter
- a reply to email to the public W3C DOM mailing list (the original email made it through ok)
- a request from someone to use one of my photos
Few of them match the problems to which Gmail proffers solutions. I’m going to have to rethink what I do for travel and weekends and stop using Gmail as a pass-through spam filter.
When this happened to me, the cause was inappropriate forwarding rules on my mail server. I had a .procmailrc that was routing the e‑mail through gmail; procmail added headers in such a way that gmail was treating my server as the source for all that e‑mail, instead of ascribing it to the original sender. The result was that gmail received lots of e‑mail from my host, almost all of it spam (since almost all e‑mail sent to me is spam), and decided after a few weeks that my host was a spam source. (It didn’t help that there were multiple users of that mail server who were using gmail as a pass-through spam filter via procmail.)
I fixed the problem by replacing my .procmailrc with a .forward file that had my MTA do the routing itstelf; the MTA is smarter about this sort of thing than procmail is, and modified the headers in a more appropriate way; once I did that, things got much much better.
If this sounds like it might be a match for what you’re doing, feel free to contact me for more details.
Hi David, it sounds like most or all of the email you were forwarding was marked as spam; this isn’t the case for me (I should have made that clearer). Most of the real email does get through (albeit after a certain period of training Gmail), and most of the spam doesn’t (ditto for the training). So it doesn’t sound to me like I have the same problem as you had. I would guess it’s Gmail raising the bar in the endless circle of spam-fighting, and raising it a little too high.
Yeah, I was forwarding all of my e‑mail, so of course 95% or 99% or whatever was correctly marked as spam; if that’s not what you’re doing, then it does sound like a different situation.
Yes, I’m forwarding all of my email, and almost all the spam mail is correctly marked as spam. Most of the non-spam is also correctly marked as not spam; the number of false positives is just high enough to be annoying. I believe my ISP (which has an option for forwarding mail) is doing the right thing because lots of real mail does get through; I understood your first comment to mean none of your real mail got through as Gmail thought your ISP only sent spam.
This is getting as complicated as figuring out what’s spam and what isn’t; I think I need more caffeine 😉
No, my real mail went through; it’s just that 5% or so of it got flagged as spam. I could be wrong, but I think that, when I was using the bad configuration, the problem was that the mailer doing the forwarding inserted a Received: header containing an envelope-from that mentioned the e‑mail address I was forwarding from. Whereas, in the good situation, the forwarding MTA inserts a Received: header, but with no envelope-from. (It’s the Received: header right after the Authentication-Results: header that Gmail inserts.)
I don’t know enough about mail headers to know what’s going on here; it is the case that somebody else using my mail server ran into the same problem, talked to a coworker of his at Google (convenient, that!), and learned that Gmail was classifying too much of the mail he was forwarding as spam because of that, and that when we changed how we were forwarding e‑mail, the problem went away for both of us.
So I would look at your mail headers; if your address at your ISP is mentioned in an envelope-from in that Received: header, it’s probably the same thing, otherwise not. Another way to tell is to look at the Authentication-Results: header (or the Received-SPF: header): — if the address mentioned after the words “domain of” is your e‑mail address that you’re forwarding of, then that’s the problem, if it’s the e‑mail address of the original sender, then something else is the problem.