In March 2009, major improvements to all three lists were completed. The most substantial improvements were made to ivmSIP/24. So I'll cover that last. First, I'll briefly address improvements to ivmSIP and ivmURI.
ivmSIP IMPROVEMENTS: During recent work on ivmSIP/24, a “bug” was discovered in the ivmSIP portion of the engine which was causing needless False Negatives. This had been there since the beginning of the invaluement lists! Fixing this should lead to additional ivmSIP effectiveness. (I about fell out of my seat when I spotted the bug!) So whatever you've thought/read/experienced about ivmSIP... it is now even better!
ivmURI IMPROVEMENTS: Large improvements to the ivmURI engine were made. For example, most good DNSBLs will employ some type of False Positives-prevention filters which then lead to unintended False Negatives (items that really should have been blacklisted). ivmURI is no different and those unintended FNs produced by False Positives-prevention filters are a fact of life. But I'm happy to report that the ivmSIP FP-prevention filters have been greatly improved so as to continue preventing FPs just as well as ever, but those unintended FNs have now been greatly trimmed. This should boost ivmURI’s “catch rates” noticeably, if not substantially.
ivmSIP/24 IMPROVEMENTS: This is where the vast majority of improvements occurred. ivmSIP/24 is almost like a whole new list.
The invaluement 'engine' now has an automated system which enumerates through all 256 possible IPs for each /24 block and individually evaluates each IP for possible exclusion from ivmSIP/24. The improvements made are substantial because ivmSIP/24 now does NOT have listed many subranges allocated to innocent senders in those /24 blocks which were split between innocent senders and spammers. Yet, at the same time, such subranges allocated to spammers remain listed on ivmSIP/24, with many IPs preemtively listed.
Additionally, we now have the ability to manually 'whitelist' individual IPs from ivmSIP/24 without having to delist the whole block. Even better, when that happens, the IP is NOT in our regular whitelist and, therefore, has just as good a chance to get listed in the regular ivmSIP list as any other IP. So this should probably be called an ivmSIP/24 “exemption list”, not a whitelist. Should that IP ever get caught sending spam, then all bets are off and its existence on the ivmSIP/24 “exemption list” is then completely ignored. This helps prevent spammers from lying about IPs and adding IPs to the “exemption list” where they really just haven't yet used that IP for spamming yet.
In the past, there have been many gut-wrenching situations where an innocent sender is using an IP located on a /24 block where the vast majority of IPs on that block are used by egregious spammers. In many cases, we’ve had to ignore the removal request for those 'innocent' senders. In other cases, we’ve had to remove the whole /24 block from ivmSIP/24 on account of the innocent senders on that block, even where most of the block was used for spamming.
Along the way, a common theme is that the innocent sender will complain (in their ivmSIP/24 removal request) that their outbound e-mail is getting blocked by [dedacted ISP-01], [dedacted ISP-02], [dedacted ISP-03], [dedacted ISP-04], [dedacted ISP-05], [dedacted ISP-06], [dedacted ISP-07], [dedacted ISP-08]. (Typically, prior to these new improvements to ivmSIP/24, the 'innocent' sender thinks that ivmSIP/24 is at fault for deliverability problems to these ISPs, because ivmSIP/24 was often the only list to show up on public lookup forms as having this IP blacklisted.) Some such complaints are probably “chaff” and might not be indicators of /24 blocking by some of these ISPs. To be extra clear, I've very unsure if all of these ISPs I mentioned do this. These are just names that have come up from what were really innocent senders. But [dedacted ISP-01] comes up often and consistently. So I feel very confident that [dedacted ISP-01] aggressively blocks /24 blocks in a manner similar to using (the old!) ivmSIP/24 for outright blocking (but with their own internal /24 blacklist). I'm also fairly sure that [dedacted ISP-02], [dedacted ISP-06], [dedacted ISP-05] does this.
Sorry, I couldn't name names in public. But all of these are 'household name' ISPs. The bottom line is that many of these ISPs use /24 blocking similar to the old ivmSIP/24. And the new ivmSIP/24 technology has now leapfrogged their technology. I dare say that ivmSIP/24 is now a whole generation ahead of whatever they are doing internally. The “nuts and bolts” of the matter is that there will be many innocent senders continuing to get blocked by some of those ISP's internal /24 lists... but where most of those same innocent senders are NOT listed on the newly improved ivmSIP/24... but where the actual spammers on each of those /24 blocks are still preemptively blacklisted on ivmSIP/24.
Also, some of our subscribers who are not particularly concerned about FPs (and this is a bit atypical for our subscribers) have taken the news about ivmSIP/24 and 'yawned'. But I should remind everyone that many /24 blocks which are crawling with snowshoe spammers, but have been previously removed from ivmSIP/24 to protect innocent senders... but these blocks are now re-listed on the new ivmSIP/24 list (but with the innocent senders’ IPs not listed). Since these are now added back to ivmSIP/24, then that means that the new and improved ivmSIP/24 will reduce both FPs AND FNs!
QUESTION: Does this mean that ivmSIP/24 is now safe to use for outright blocking?
I can safely say that the new ivmSIP/24 is now order of magnitudes safer for outright blocking than the old ivmSIP/24. But subscribers, and especially larger ISPs, should still use caution. And, in general, everyone should outright block on any dnsbl “at their own risk”, and only after evaluating the track record for that dnsbl.
At the least, in a scoring system, I'd at least add a point or two to ivmSIP/24. But I wouldn't recommend scoring it as high or higher than ivmSIP.
And if you haven't ever implemented ivmSIP/24, now is a great time to start!
