Update on Cox blocking of Craigslist

The original claim of a Cox "blacklist" originated from a statement by Tom Foremski at Silicon Valley Watcher. Foremski originally wrote:

Back on February 23rd Authentium acknowledged that their software is blocking Craigslist but it still hasn't fixed the problem, more than three months later. That's a heck of long time to delete some text from their blacklist.

Now, he says (quoted by George Ou at ZDNet):

I assumed there was a blacklist - I have no idea how Craigslist is being blocked

In fact, we know now that it's a combination of a bug in a firewall driver produced by Authentium software and unusual (but not incorrect) behavior by the Craigslist webserver setting the initial TCP window size to 0. The facts of the problem came out (at least between Craigslist, Cox, and Authentium) at the time the problem was first reported, was fixed in a beta release within weeks, and has only affected Cox customers who use Authentium's security suite.

BTW, I disagree with Richard Bennett and George Ou's remarks which attribute the problem entirely or largely to Craigslist--the behavior of the server is not contrary to the RFC. The initial SYN packet from the client to Craigslist is responded to by Craigslist with a SYN-ACK packet with window size of zero, which means don't send me any data, only an ACK. The client then sends an ACK (completing the three-way TCP handshake), at which point Craigslist sends an ACK packet with a larger window size which the pre-fix version of the Authentium software fails to process. The initial response of the Authentium software to slow down is a reasonable and apparently desired response by Craigslist--they want new clients to hold off transmitting data (an HTTP request) until they give the OK. Authentium took full responsibility for the problem, and they were right to do so.

The story from Foremski was uncritically repeated by Matt Stoller at MyDD, Timothy Karr at Save the Internet (and a couple of other blogs), and now in a Wall Street Journal op-ed piece by Sen. Ron Wyden (D-OR), in a lapse from his normally good judgment about Internet-related matters (e.g., the Cox/Wyden Internet Freedom Act of 1995 and the Cox/Wyden Internet Tax Freedom Act of 1998).

Stoller and Karr went on to repeat the "blacklist" claim even after having the full story, and I don't believe either of them has retracted the claim that this issue is relevant to the network neutrality debate.

Craig Newmark complains that he didn't get good responsiveness from Authentium, which Authentium disputes, but he has indicated satisfaction with Cox.

The story has been picked up by George Ou at ZDNet (here and here) and by Glenn Harlan Reynolds at Instapundit (here, here, and here).

This issue was a user software application issue that had no more to do with network neutrality than a browser incompatibility issue, a webserver disk failure, or a fiber cut. Each of these things can prevent a user from reaching some specific content, but none is imposed by the network provider or remedied by act of Congress or the FCC. Those who continue to treat it otherwise even after knowing the details are demonstrating questionable judgment and integrity.

UPDATE: Craig Newmark has now stated that there was no deliberate blocking here and the Authentium explanation is correct. I've exchanged a few emails with him asking whether the behavior of the Craigslist.org webserver is specifically intended to regulate the rate of new HTTP connections (and whether the behavior is coming from something like an application-layer switch negotiating the TCP handshake); he said he's passed that on to his technical team and I'll report here if I get confirmation or refutation on that point.

One puzzling paragraph of his latest blog post is this one:

One good outcome of this is that we flushed out a swiftboater (in the generic sense), and this helps me understand the way disinformation gangs operate. Unfortunately, in some blogs, a good guy has been linked with the swiftboater, which isn't fair, and hopefully, we can do something about that.

I'm not sure who he's calling a swiftboater, who he's calling a good guy, and who he's calling a disinformation gang. So far as I can see, the disinformation gang in this incident has been the "Save the Internet" crowd, who still have yet to admit the clear facts of the matter. I asked for clarification, but Craig declined to identify who he's referring to (except that he's not referring to Matt Stoller or Timothy Karr).

UPDATE: July 12, 2006: The Craigslist.org webserver has changed its behavior and no longer sends a SYN-ACK packet with a window size of 0; it now gives a window size of 4380. This change by Craigslist.org works as a fix to the Authentium issue. I wonder why they only made the change now.