{"id":138,"date":"2014-11-18T21:05:15","date_gmt":"2014-11-18T21:05:15","guid":{"rendered":"http:\/\/www.pseudonymity.net\/?p=138"},"modified":"2016-07-21T00:42:14","modified_gmt":"2016-07-20T23:42:14","slug":"a-quick-investigation-of-edgecast-cdn-blocking-in-china","status":"publish","type":"post","link":"https:\/\/www.pseudonymity.net\/blog\/index.php\/2014\/11\/18\/a-quick-investigation-of-edgecast-cdn-blocking-in-china\/","title":{"rendered":"A Quick Investigation of EdgeCast CDN Blocking in China"},"content":{"rendered":"

\"arbyreed-blacksmith_forge_heating_iron\"<\/a><\/p>\n

This morning, GreatFire.org<\/a> published a story<\/a> stating that EdgeCast CDN, one of the more popular content distribution networks that handles content for a number of large websites, has been blocked by the Chinese national filter. As a result, a friend emailed me asking what I thought, and pointed out that all we have are a few reports and a link to a status update from EdgeCast themselves.<\/p>\n

As usual, my attempt to write a short email failed, and I ended up carrying out an impromptu investigation into this. With minor edits, I’ve reproduced my email detailing how I looked into this below. For reference, this was carried out from an internet connection based in Oxford, UK.<\/p>\n

Based on prior knowledge we have evidence that China will man-in-the-middle (UDP) DNS requests for blocked sites, but ignore genuine ones. So first, let’s pick a Chinese IP address almost at random:<\/p>\n

\n$ ping baidu.cn
\nPING baidu.cn (220.181.111.86) 56(84) bytes of data.
\n<\/code>\n<\/p><\/blockquote>\n

Check that it’s actually in China, using the MaxMind GeoIP database:<\/p>\n

\n$ geoiplookup 220.181.111.86
\nGeoIP Country Edition: CN, China
\nGeoIP City Edition, Rev 1: CN, 22, Beijing, Beijing, N\/A, 39.928902,
\n116.388298, 0, 0
\n<\/code>\n<\/p><\/blockquote>\n

Check that it’s not a DNS server:<\/p>\n

\n$ dig @220.181.111.86 baidu.cn <\/p>\n

; <<>> DiG 9.9.2-P2 <<>> @220.181.111.86 baidu.cn
\n; (1 server found)
\n;; global options: +cmd
\n;; connection timed out; no servers could be reached
\n<\/code>\n<\/p><\/blockquote>\n

Excellent. No response. Now, perform a DNS lookup from our (presumably uncensored) connection in the UK to an edgecastcdn.net host:<\/p>\n

\n$ dig edgecastcdn.net<\/p>\n

; <<>> DiG 9.9.2-P2 <<>> edgecastcdn.net
\n;; global options: +cmd
\n;; Got answer:
\n;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 635\n;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 2, ADDITIONAL: 5\n\n;; OPT PSEUDOSECTION:\n; EDNS: version: 0, flags:; udp: 4096\n;; QUESTION SECTION:\n;edgecastcdn.net.\t\tIN\tA\n\n;; ANSWER SECTION:\nedgecastcdn.net.\t3370\tIN\tA\t93.184.221.133\n
\n<\/code>\n<\/p><\/blockquote>\n

Now let’s see what happens if we look up edgecastcdn.net at the baidu.cn IP, recalling that it is not actually a DNS server:<\/p>\n

\n$ dig @220.181.111.86 edgecastcdn.net<\/p>\n

; <<>> DiG 9.9.2-P2 <<>> @220.181.111.86 edgecastcdn.net
\n; (1 server found)
\n;; global options: +cmd
\n;; Got answer:
\n;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 63357\n;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0\n\n;; QUESTION SECTION:\n;edgecastcdn.net.\t\tIN\tA\n\n;; ANSWER SECTION:\nedgecastcdn.net.\t30640\tIN\tA\t203.98.7.65\n\n;; Query time: 397 msec\n;; SERVER: 220.181.111.86#53(220.181.111.86)\n;; WHEN: Tue Nov 18 12:02:53 2014\n;; MSG SIZE rcvd: 64\n<\/code>\n<\/p><\/blockquote>\n

Interesting. We get a response, which took 397 milliseconds. Let's look up the returned IP:<\/p>\n

\n$ dig -x 203.98.7.65<\/p>\n

; <<>> DiG 9.9.2-P2 <<>> -x 203.98.7.65
\n;; global options: +cmd
\n;; Got answer:
\n;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 7947\n;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1\n\n;; OPT PSEUDOSECTION:\n; EDNS: version: 0, flags:; udp: 4096\n;; QUESTION SECTION:\n;65.7.98.203.in-addr.arpa.\tIN\tPTR\n\n;; AUTHORITY SECTION:\n7.98.203.in-addr.arpa.\t3600\tIN\tSOA\tns1.netlink.co.nz.\nsoa.netlink.co.nz. 2008110600 7200 1200 1728000 172800\n\n;; Query time: 767 msec\n;; SERVER: 129.67.1.180#53(129.67.1.180)\n;; WHEN: Tue Nov 18 12:11:43 2014\n;; MSG SIZE rcvd: 110\n<\/code>\n<\/p><\/blockquote>\n

That doesn't look like a genuine response! A quick WHOIS:<\/p>\n

\n$ whois netlink.co.nz
\n% New Zealand Domain Name Registry Limited
\n% Users confirm on submission their agreement to all published Terms
\n%
\nversion: 5.00
\nquery_datetime: 2014-11-19T01:12:03+13:00
\ndomain_name: netlink.co.nz
\nquery_status: 200 Active
\ndomain_dateregistered: 1997-03-24T00:00:00+12:00
\ndomain_datebilleduntil: 2014-12-01T00:00:00+13:00
\ndomain_datelastmodified: 2014-11-01T23:37:30+13:00
\ndomain_delegaterequested: yes
\ndomain_signed: no
\n%
\nregistrar_name: Vodafone New Zealand Limited (Clear)
\nregistrar_address1: Private Bag 92161
\nregistrar_city: Auckland
\nregistrar_country: NZ (NEW ZEALAND)
\nregistrar_phone: +64 508 888 800
\nregistrar_email: registry@clear.net.nz
\n%
\nregistrant_contact_name: NetLink
\nregistrant_contact_address1: PO Box 5358
\nregistrant_contact_city: Wellington
\nregistrant_contact_country: NZ (NEW ZEALAND)
\nregistrant_contact_phone: +64 4 9228499
\nregistrant_contact_fax: +64 4 9228401
\nregistrant_contact_email: dns@netlink.co.nz
\n%
\nadmin_contact_name: Netlink Operations Centre
\nadmin_contact_address1: PO Box 5358
\nadmin_contact_city: Wellington
\nadmin_contact_country: NZ (NEW ZEALAND)
\nadmin_contact_phone: +64 4 922 8499
\nadmin_contact_fax: +64 4 922 8401
\nadmin_contact_email: dns@netlink.co.nz
\n%
\ntechnical_contact_name: Netlink Operations Centre
\ntechnical_contact_address1: PO Box 1762
\ntechnical_contact_address2: Wellington, New Zealand
\ntechnical_contact_phone: +64 4 495 5021
\ntechnical_contact_fax: +64 4 495 5197
\ntechnical_contact_email: dns@netlink.co.nz
\n%
\nns_ip4_01: 202.20.93.10
\nns_name_01: ns1.netlink.co.nz
\nns_ip4_02: 203.96.152.12
\nns_name_02: ns2.netlink.co.nz
\n<\/code>\n<\/p><\/blockquote>\n

There does appear to be evidence of interference at the network level -- the request to edgecastcdn.net appears to be redirected to a Vodafone-operated host in New Zealand. I'm interested by the DNS response time of 397 msecs for the lookup, and I have a feeling that that could reveal interesting things about the possible location of the man-in-the-middle attack. A quick whois on baidu.cn gives their genuine DNS server as being located at 202.108.22.220, so we can use that as our test IP instead:<\/p>\n

\n$ dig @202.108.22.220 baidu.cn
\n
\n;; Query time: 313 msec<\/p>\n

$ dig @202.108.22.220 edgecastcdn.net
\n
\n;; Query time: 281 msec
\n<\/code>\n<\/p><\/blockquote>\n

With one data point it seems that a reply for a censored domain is a few tens of milliseconds quicker than an uncensored one, but one data point does not science make. Here's 100 requests to each:<\/p>\n

\n$ for i in $(seq 1 100); do
\n dig @202.108.22.220 edgecastcdn.net | grep \"Query time\" | sed -e \"s\/.*: (.*) msec\/1\/\" >> edgecastresults.txt;
\ndone<\/p>\n

$ for i in $(seq 1 100); do
\n dig @202.108.22.220 baidu.cn | grep \"Query time\" | sed -e \"s\/.*: (.*) msec\/1\/\" >> baiduresults.txt;
\ndone
\n<\/code>\n<\/p><\/blockquote>\n

That gives us two set of statistics that we can summarise quickly in R:<\/p>\n

\n$ R
\n> baidu <- read.csv('baiduresults.txt')\n> edgecast <- read.csv('edgecastresults.txt')\n> summary( baidu )<\/p>\n

X323
\n Min. :286.0
\n 1st Qu.:303.0
\n Median :306.0
\n Mean :323.2
\n 3rd Qu.:309.0
\n Max. :652.0 <\/p>\n

> summary( edgecast )<\/p>\n

X324
\n Min. :249.0
\n 1st Qu.:293.5
\n Median :306.0
\n Mean :314.7
\n 3rd Qu.:309.0
\n Max. :622.0
\n<\/code>\n<\/p><\/blockquote>\n

The timing doesn't seem to be that different overall. I suspect that, with the Baidu DNS being in Beijing, there is a good chance of the man-in-the-middle attacks being sufficiently geographically close that it makes little difference to the output. Maybe we could pick a reasonable DNS server that isn't in Beijing and test against that.<\/p>\n

A quick Google for 'china dns server' gives this page: https:\/\/sites.google.com\/site\/kiwi78\/public-dns-servers<\/a>, and we randomly pick one that claims to be in Chengdu.<\/p>\n

\n$ dig @61.139.54.66 baidu.cn<\/p>\n

; <<>> DiG 9.9.2-P2 <<>> @61.139.54.66 baidu.cn
\n; (1 server found)
\n;; global options: +cmd
\n;; Got answer:
\n;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 7210\n;; flags: qr rd ra; QUERY: 1, ANSWER: 3, AUTHORITY: 4, ADDITIONAL: 5\n\n;; OPT PSEUDOSECTION:\n; EDNS: version: 0, flags:; udp: 4096\n;; QUESTION SECTION:\n;baidu.cn.\t\t\tIN\tA\n\n;; ANSWER SECTION:\nbaidu.cn.\t\t120\tIN\tA\t220.181.111.85\nbaidu.cn.\t\t120\tIN\tA\t123.125.114.144\nbaidu.cn.\t\t120\tIN\tA\t220.181.111.86\n\n;; AUTHORITY SECTION:\nbaidu.cn.\t\t120\tIN\tNS\tns2.baidu.com.\nbaidu.cn.\t\t120\tIN\tNS\tns3.baidu.com.\nbaidu.cn.\t\t120\tIN\tNS\tns4.baidu.com.\nbaidu.cn.\t\t120\tIN\tNS\tns1.baidu.com.\n\n;; ADDITIONAL SECTION:\nns1.baidu.com.\t\t75287\tIN\tA\t202.108.22.220\nns2.baidu.com.\t\t74049\tIN\tA\t61.135.165.235\nns3.baidu.com.\t\t74049\tIN\tA\t220.181.37.10\nns4.baidu.com.\t\t74049\tIN\tA\t220.181.38.10\n\n;; Query time: 426 msec\n;; SERVER: 61.139.54.66#53(61.139.54.66)\n;; WHEN: Tue Nov 18 12:29:19 2014\n;; MSG SIZE rcvd: 230\n\njoss@kafka:~\/tmp\/exp$ geoiplookup 61.139.54.66\nGeoIP Country Edition: CN, China\nGeoIP City Edition, Rev 1: CN, 32, Sichuan, Chengdu, N\/A, 30.666700,\n104.066704, 0, 0\n<\/code>\n<\/p><\/blockquote>\n

That responds to our innocuous query, and seems to be in Chengdu according to our geoip database. Let's try the 100 requests trick on it. (I won't repeat the code as it's illustrated above.):<\/p>\n

\n$ summary(baidu)
\n X376
\n Min. :275.0
\n 1st Qu.:305.0
\n Median :321.0
\n Mean :359.8
\n 3rd Qu.:408.8
\n Max. :868.0 <\/p>\n

$ summary(edgecast)
\n X338
\n Min. :247
\n 1st Qu.:307
\n Median :317
\n Mean :335
\n 3rd Qu.:335
\n Max. :628
\n<\/code>\n<\/p><\/blockquote>\n

There still isn't anything particularly damning based on the timing. Of course, there are lots of issues with DNS caching to think about, and my simple shell scripts didn't check for things like timeouts or no replies, so this might not be as simple as it looks, but we've still got some interesting data with which to play. At the very least, I can support GreatFire's claim that China are doing things to the edgecastcdn domain.<\/p>\n

The first improvement that I'll make, which will probably run tonight, is to add a `sleep 90`<\/code> call in the for loop that runs the repeated requests. That should hopefully avoid the most obvious form of rate limiting.<\/p>\n

This wasn't intended to be a full and detailed investigation, but certainly threw up some interesting findings and a confirmation of the key points of GreatFire.org's story. In the wider sense, the idea of blocking a content distribution network rather than a website itself is a bold step and has significant collateral implications, as GreatFire point out at length. <\/p>\n

For me, looking forward, the most interesting aspects of these stories are to derive the logic and intentions behind the behaviour itself, which will require much more significant analytical and theoretical tools than simply probing what was blocked where and how. Why are these particular networks or sites chosen for filtering, and what can that tell us? How quickly are sites blocked, and unblocked, in relation to political or social events? Filtering provides a fascinating lens into the motivations and through processes of those carrying it out. As blocking policies develop, with governments and populations become more comfortable with using the internet in all aspects of life, the ability to draw inferences from such overt interference in the network will be an incredibly rich seam to mine.<\/p>\n","protected":false},"excerpt":{"rendered":"

This morning, GreatFire.org published a story stating that EdgeCast CDN, one of the more popular content distribution networks that handles content for a number of large websites, has been blocked by the Chinese national filter. As a result, a friend emailed me asking what I thought, and pointed out that all we have are a […]<\/p>\n","protected":false},"author":2,"featured_media":169,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[3,4],"tags":[14],"class_list":["post-138","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-censorship","category-china","tag-oii"],"_links":{"self":[{"href":"https:\/\/www.pseudonymity.net\/blog\/index.php\/wp-json\/wp\/v2\/posts\/138","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.pseudonymity.net\/blog\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.pseudonymity.net\/blog\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.pseudonymity.net\/blog\/index.php\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.pseudonymity.net\/blog\/index.php\/wp-json\/wp\/v2\/comments?post=138"}],"version-history":[{"count":12,"href":"https:\/\/www.pseudonymity.net\/blog\/index.php\/wp-json\/wp\/v2\/posts\/138\/revisions"}],"predecessor-version":[{"id":193,"href":"https:\/\/www.pseudonymity.net\/blog\/index.php\/wp-json\/wp\/v2\/posts\/138\/revisions\/193"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.pseudonymity.net\/blog\/index.php\/wp-json\/wp\/v2\/media\/169"}],"wp:attachment":[{"href":"https:\/\/www.pseudonymity.net\/blog\/index.php\/wp-json\/wp\/v2\/media?parent=138"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.pseudonymity.net\/blog\/index.php\/wp-json\/wp\/v2\/categories?post=138"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.pseudonymity.net\/blog\/index.php\/wp-json\/wp\/v2\/tags?post=138"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}