Blocking Network Packets – The Comcast Fiasco

Sometimes you just can’t be neutral. That, apparently, is the reasoning behind Comcast’s recently revealed actions regarding several peer-to-peer applications, including BitTorrent.

In case you haven’t been following the story, here’s a quick rundown.

It started five days ago when the Associated Press ran a story called “Comcast Blocks Some Internet Traffic.” In this story, the reporter, Peter Svensson, points out that Comcast was blocking P2P traffic by interfering with certain packet streams.

Each PC gets a message invisible to the user that looks like it comes from the other computer, telling it to stop communicating. But neither message originated from the other computer — it comes from Comcast. If it were a telephone conversation, it would be like the operator breaking into the conversation, telling each talker in the voice of the other: “Sorry, I have to hang up. Good bye.”

This may have never been major news except for two things: First, back in August, Comcast denied that it was doing this exact thing, Second, Ubuntu 7.10, Gutsy Gibbon, was released six days ago. And when the #1 Linux distribution in America is released, you can bet your sweet bippy that the FTP mirrors for it will get hammered, leading people to consider using BitTorrent to get it. Comcast users had problems getting Gutsy.

 Svensson would go on to explain exactly how the tests were conducted and how Comcast was specifically blocking the traffic in a follow-up article. They used a copy of the (public domain) King James Bible and tried to download it to, and from, Comcast subscribers via the BitTorrent protocol.

In two out of three tries, the transfer was blocked. In the third, the transfer started only after a 10-minute delay.

According to Svensson, what was happening was that Comcast was inserting RST (Reset) packets into the communication – but forging the address to make it look like it was coming from either the sender’s or recipient’s computer. Packet capture and analysis showed that neither computer was sending these RST packets. It was originating from the middle – from Comcast. Furthermore, they tested via a Time Warner Cable connection and found that Comcast was sending these reset packets to subscribers to other networks as well.

Peter Eckersley of the Electronic Frontier Foundation then got into the action with an investigation of his own, showing that the same thing happens when you use a Gnutella P2P node, or, even, as Kevin Kanarski found out, when you use IBM’s Lotus Notes with Domino.

As you can see from these traces, the Notes client saw the RST packets coming from the Domino server IP and the Domino server saw the RST packets coming from the Notes client PC. However the trace doesn’t show either one of them sending the RST packets which means something on the network in between was sending them.

At this point, the story blows up and is all around the Web – with many different angles to it.

The first angle I want to look at is the angle of quality of service and the end-user experience.

In a prepared statement, Comcast representative Charlie Douglas said that: “Comcast does not block access to any Web sites or online applications, including peer-to-peer activity like BitTorrent.” The AP test seems to prove that statement inaccurate – or at least, imply that Comcast is playing a very slick semantic game with the meaning of “block.” Some BT data is being blocked. Some BT data isn’t. What are accurate things to say is that Comcast is interfering in access to BitTorrent. It would also be accurate to say that Comcast is intentionally degrading the quality of service to BitTorrent.

This is important because all defenses of Comcast begin and end with the idea that if Comcast does degrade BitTorrent traffic, it does so out of altruistic goals to serve all of its subscribers. Or, as Comcast put it in a statement, “We have a responsibility to provide all of our customers with a good Internet experience and we use the latest technologies to manage our network so that they can continue to enjoy these applications.”

But clearly, for BitTorrent (and IBM Lotus Notes users) the end-user experience is being worsened. Does that mean everyone else’s access is being improved? Not very likely.

See, BitTorrent gets lumped in with Napster as a peer-to-peer tool, but it does so unfairly, in my opinion. BitTorrent is, ultimately a tool used to improve the quality of service and, in its own way, improve network performance. On the client end, it improves the quality of service through faster downloads. On the server end, it improves the quality of service through reduced bandwidth requirements.

But Comcast, according to the AP, degraded the performance of BitTorrent and then expressly said that they weren’t. That can cause end-users to think that BitTorrent is an unreliable application. The end-user then prefers more traditional methods of downloading large files – FTP, IRC, UseNet, etc. These methods take up more overall bandwidth than BitTorrent would have. In such a manner, Comcast’s blocking of BitTorrent degrades quality of service for the Internet as a whole.

Here’s another angle: If the claims made by the Associated Press are true, Comcast is not only sending these reset packets without disclosing so to customers, but also that these RST packets look like they’re coming from “spoofed” or “forged” IP addresses – the sender thinks these packets are coming from the receiver, and the receiver thinks these packets are coming from the sender. However, as studies by the AP and others have shown, they’re coming from neither – or more bluntly, they’re coming from Comcast.

These forged IP addresses can complicate the troubleshooting and delay mean time to repair of networking problems. Quite frankly, because Comcast is allegedly sending out false information, the engineers don’t have the information they need to solve problems. You can only imagine the frustration of a network engineer, called up by an irate C-level executive, who wants to know why Lotus Notes isn’t responding when he tries to access the server from home. The engineer tests the packets and finds that the executive’s computer is sending out excessive reset packets. The engineer then goes out to check on the executive’s computer, only to find that the server is now sending out the packets.

Now the engineer can call in IBM, but they won’t know what’s going on, only that some users are reporting excessive reset packets. The company might go so far as to dump Lotus Notes and go with a different solution – which means IBM has lost money.

It gets worse. Forged IP addresses are essentially “impersonating” another person’s computer. If Comcast did this while denying it was doing so, it can also be argued that this “impersonation” was an attempt to “defraud.” Even if Comcast gets explicit permission to do exactly this in Comcast subscribers’ terms of service, they do not get the permission of people on other networks, accessing a Comcast subscriber’s computer in a P2P session.

Further complicating this is the fact that BitTorrent, Inc., the company whose founder invented BitTorrent, has recently gotten distribution deals which allow BitTorrent, Inc., to sell legal downloads of television shows online. These are the same television shows that are also available via Comcast’s cable television service. That may make Comcast’s alleged throttling of BitTorrent traffic at best a conflict of interest, at worst, “anti-competitive behavior.”

We’re not lawyers, but we’re pretty sure these issues could put Comcast in legal hot water. Even if not, it is a major customer service problem from a company notorious for bad customer service. Comcast is the cable company in the YouTube video where the cable installer spent so much time on hold that he ended up falling asleep on the customer’s couch. Comcast also, as far back as 2003, was threatening to disconnect – and followed through on disconnecting – customers who used “too much” bandwidth on an Unlimited Bandwidth plan, while refusing to disclose where, exactly, the bandwidth cap lies, and gallingly insisting that there was no bandwidth cap.

In fact, a 75 year old woman walked into a Comcast branch office, started smashing stuff with a claw hammer, and the Washington Post began the story with the lede:

“Sometimes truly American virtues arise in outlaws who — by dint of heroic but questionable endeavors — display the mettle of the national character.”

This customer service problem is so great that Advertising Age said that Comcast should spend less money on advertising in order to improve its customer service. When the trade magazine for the advertising industry tells you that you need to do less advertising, and one of its major columnists declares a “Consumer Jihad” and titles the article, “Comcast Must Die” you’ve got a major problem.

Journalistic ethics also demand that a reporter try to get all sides of the story, and you’ll have noticed thus far that we’ve not shown Comcast’s side. However, the Comcast representative I spoke to was only willing to answer questions “on background.” This is how someone at Comcast spoke to the New York Times, and apparently, the New York Times has no problem with that arrangement.

But, I don’t believe that Comcast reps should be speaking on background. “On background,” I reserve for people who could lose jobs or face physical harm for whistleblowing and talking about something that is of vital information. I couldn’t think of any reason a Comcast representative couldn’t talk about Comcast’s policies on the record. That’s why there’s nobody from Comcast telling their side of the story in this article.

Sometimes you just can’t be neutral.