2013 DNS census MX records Updated 2025-01-29 +Created 1970-01-01
View more
Let' see if there's anything in records/mx.xz.
mx.csv is 21GB.
They do have " in the files to escape commas so:
mx.py
import csv
import sys
writer = csv.writer(sys.stdout)
with open('mx.csv', 'r') as f:
    reader = csv.reader(f)
    for row in reader:
        writer.writerow([row[0], row[3]])
Would have been better with csvkit: stackoverflow.com/questions/36287982/bash-parse-csv-with-quotes-commas-and-newlines
then:
# uniq not amazing as there are often two or three slightly different records repeated on multiple timestamps, but down to 11 GB
python3 mx.py | uniq > mx-uniq.csv
sqlite3 mx.sqlite 'create table t(d text, m text)'
# 13 GB
time sqlite3 mx.sqlite ".import --csv --skip 1 'mx-uniq.csv' t"

# 41 GB
time sqlite3 mx.sqlite 'create index td on t(d)'
time sqlite3 mx.sqlite 'create index tm on t(m)'
time sqlite3 mx.sqlite 'create index tdm on t(d, m)'

# Remove dupes.
# Rows: 150m
time sqlite3 mx.sqlite <<EOF
delete from t
where rowid not in (
  select min(rowid)
  from t
  group by d, m
)
EOF

# 15 GB
time sqlite3 mx.sqlite vacuum
Let's see what the hits use:
awk -F, 'NR>1{ print $2 }' ../media/cia-2010-covert-communication-websites/hits.csv | xargs -I{} sqlite3 mx.sqlite "select distinct * from t where d = '{}'"
At around 267 total hits, only 84 have MX records, and from those that do, almost all of them have exactly:
smtp.secureserver.net
mailstore1.secureserver.net
with only three exceptions:
dailynewsandsports.com|dailynewsandsports.com
inews-today.com|mail.inews-today.com
just-kidding-news.com|just-kidding-news.com
We need to count out of the totals!
sqlite3 mx.sqlite "select count(*) from t where m = 'mailstore1.secureserver.net'"
which gives, ~18M, so nope, it is too much by itself...
Let's try to use that to reduce av.sqlite from 2013 DNS Census virtual host cleanup a bit further:
time sqlite3 mx.sqlite '.mode csv' "attach 'aiddcu.sqlite' as 'av'" '.load ./ip' "select ipi2s(av.t.i), av.t.d from av.t inner join t as mx on av.t.d = mx.d and mx.m = 'mailstore1.secureserver.net' order by av.t.i asc" > avm.csv
where avm stands for av with mx pruning. This leaves us with only ~500k entries left. With one more figerprint we could do a Wayback Machine CDX scanning scan.
Let's check that we still have most our hits in there:
grep -f <(awk -F, 'NR>1{print $2}' /home/ciro/bak/git/media/cia-2010-covert-communication-websites/hits.csv) avm.csv
At 267 hits we got 81, so all are still present.
secureserver is a hosting provider, we can see their blank page e.g. at: web.archive.org/web/20110128152204/http://emmano.com/. security.stackexchange.com/questions/12610/why-did-secureserver-net-godaddy-access-my-gmail-account/12616#12616 comments:
secureserver.net is the name GoDaddy use as the reverse DNS for IP addresses used for dedicated/virtual server hosting
Read the full article
2013 DNS census secureserver.net MX records intersection 2013 DNS Census virtual host cleanup Updated 2025-01-29 +Created 2023-07-19
View more
We intersect 2013 DNS Census virtual host cleanup with 2013 DNS census MX records and that leaves 460k hits. We did lose a third on the the MX records as of 260 hits since secureserver.net is only used in 1/3 of sites, but we also concentrate 9x, so it may be worth it.
Then we Wayback Machine CDX scanning. it takes about 5 days, but it is manageale.
We did a full Wayback Machine CDX scanning for JAR, SWF and cgi-bin in those, but only found a single new hit:
Read the full article
Hits with nearby IP hits Updated 2025-01-29 +Created 1970-01-01
View more
alljohnny.com: one of the Reuters websites.
62.22.60.49: telecom-headlines.com. Found with: visual inspection of full 2013 DNS Census virtual host cleanup list just before worldnewsnetworking.com. Tested viewdns.info range: 62.22.60.34 - 62.22.60.66
  • 62.22.60.33: newsperk.com. Unclear. Stylistically perfect, but no comms not found. 2011. English. Egypt. news.
  • 62.22.60.34: freeslideshow.net. Legit? Attempting to open any HTML archives leads to an infinite page load loop, e.g. 2010. A subpage however exists: web.archive.org/web/20101230001640/http://freeslideshow.net/index_files/a.htm and appears legit.
  • 62.22.60.40: travel-passage.com. Unclear. No archives of toplevel, only subpage: 2009. No clear comms. Chinese.
  • 62.22.60.42: newsupdatesite.com. Hit.
  • 62.22.60.46: flyingtimeline.com. Hit.
  • 62.22.60.47: globalemergenceadvisorsbkserver.com. Legit.
  • 62.22.60.48: currentcommunique.com. Hit.
  • 62.22.60.49: telecom-headlines.com. Hit.
  • 62.22.60.52: collectedmedias.com. Hit.
  • 62.22.60.54: romulusactualites.com. No archives.
  • 62.22.60.55: thefilmcentre.com. Hit.
  • 62.22.60.56: traveltimenews.com. Hit.
62.22.61.206 worldnewsnetworking.com. Found with: 2013 DNS Census virtual host cleanup heuristic keyword searches. Tested viewdns.info range: 62.22.61.188 - 62.22.61.224
63.131.229.12 cyberreportagenews.com. Tested viewdns.info range: 63.131.228.248 - 63.131.229.30
  • 63.131.229.2: fightskillsresource.com. Hit
  • 63.131.229.4: unitedterritorynews.com. Hit
  • 63.131.229.9: show-dustry.com. Hit
  • 63.131.229.10: afghanpoetry.net. Hit. Also at 74.254.12.166 in another range.
  • 63.131.229.11: mythriftytrip.com. Hit
  • 63.131.229.12: cyberreportagenews.com. Hit.
  • 63.131.229.13: sunrise-news.com. Hit.
  • 63.131.229.15: cricketnewsforindia.com. Archive quite broken, likely hit.
  • 63.131.229.16:
    • nutricion-saludable.info. No archives.
    • nutricion-saludable.net. Hit.
  • 63.131.229.18: itnl-xchange.com. Hit.
  • 63.131.229.20:
    • fixashion.net. Hit.
    • a few others
63.130.160.50 theglobalheadlines.com. Found with: 2013 DNS census secureserver.net MX records intersection 2013 DNS Census virtual host cleanup. Tested viewdns.info range: 63.130.160.35 - 63.130.160.75
  • 63.130.160.50: theglobalheadlines.com. Hit.
  • 63.130.160.51:
    • hai-pow.com. Hit.
    • secudenetworksecurity.com. No archives.
  • 63.130.160.53: echessnews.com. Hit.
  • 63.130.160.59: technologiewissen.com. No archives from the time. Would be Technology knowledge in German, so another likely German hit. Shame.
  • 63.130.160.60: boxingstop.net. Hit.
  • 63.130.160.61: bookmarksthis.com. No archives.
  • 63.130.160.62: azerinews.org. Hit.
64.16.204.55 holein1news.com. Found with: 2013 DNS Census virtual host cleanup heuristic keyword searches. Tested viewdns.info range: 64.16.204.50 - 64.16.204.63. With did Wayback Machine have so few archives here? TODO stopping viewdns.info exploration a bit short due to that.
  • 64.16.204.35: ironcityfootball.com. Legit/broke.
  • 64.16.204.51: africannewsandsports.com. No archives. rdns source: viewdns.info
  • 64.16.204.53: bosniakbusinessnews.com. No archives. A Bosniak is someone from an ethnicity from Bosnia.
  • 64.16.204.54: affairesdumonde.com. No archives. rdns source: viewdns.info
  • 64.16.204.55: holein1news.com. Hit.
  • 64.16.204.56: fightorgohome.com. No archives. rdns source: viewdns.info
  • 64.16.204.58: tech-topix.com. Hit.
  • 64.16.204.60: pakpoldaily.com. No archives. rdns source: viewdns.info. TODO meaning? Might be Indonesian, maybe linked to police: www.facebook.com/watch/?v=880204266271955
65.61.127.163 capture-nature.com. whois.arin.net/rest/net/NET-65-61-96-0-1/pft?s=65.61.127.163: Net Range: 65.61.96.0 - 65.61.127.255. Organization. Name: TierPoint, LLC. Tested viewdns.info range: 65.61.127.149 -
66.45.179.205 noticiasporjanua.com. Found with: 2013 DNS Census virtual host cleanup. Tested viewdns.info range: 66.45.179.187 - 66.45.179.223
  • 66.45.179.187: mail03.gatesfoundation.org. Legit.
  • 66.45.179.192: thegraceofislam.com. Hit.
  • 66.45.179.193: arabicnewsunfiltered.com. Hit.
  • 66.45.179.194: raulsonsglobalnews.com. Hit.
  • 66.45.179.195: aryannews.net. Hit.
  • 66.45.179.199: attivitaestremi.com. Hit.
  • 66.45.179.200: foodwineandsuch.com. No archives.
  • 66.45.179.201: hitthepavementnow.com. Hit.
  • 66.45.179.203: noticiascontinental.com. Hit.
  • 66.45.179.205: noticiasporjanua.com. Hit.
  • 66.45.179.206: podisticamondiale.com. Hit.
  • 66.45.179.207: reflectordenoticias.com. Hit.
  • 66.45.179.208: havenofgamerz.com. Hit.
  • 66.45.179.209: vejaaeuropa.com. web.archive.org/web/20130810131440/http://www.vejaaeuropa.com/: Welcome to the US Petabox. Shame, could be another Brazil hit since "veja" (look in Brazilian Portuguese) would be "mira" in Spanish, not "veja".
  • 66.45.179.210: sa-michigan.com. Hit.
  • 66.45.179.211: absolutebearing.net. Hit.
  • 66.45.179.212: grandretirement.net. No archives.
  • 66.45.179.213: myportaltonews.com. Hit.
  • 66.45.179.214: investmentintellect.com. Hit.
  • 66.45.179.215: nigeriastar.net 2012-03-12. Hit.
66.104.169.184 bcenews.com. Found with: 2013 DNS Census virtual host cleanup heuristic keyword searches. Tested viewdns.info range: 66.104.169.158 - 66.104.169.189
  • 66.104.169.162: bestsportsnews.net. Archive broken.
  • 66.104.169.163: doctorsoncallsite.com. Hit.
  • 66.104.169.164: lightandshadowonline.com. Hit.
  • 66.104.169.168: plugged-into-news.net. Hit.
  • 66.104.169.169: worldsportsite.com. Likely hit, but comms not found. 2011. Arabic. . sports. has some apparently unrelated archives from 2008.
  • 66.104.169.171: golf-on-holiday.com. Hit.
  • 66.104.169.172: perspectiva-noticias.com. Hit.
  • 66.104.169.175: aquaswimming.com. Hit.
  • 66.104.169.177: dojo-temple.com. Hit.
  • 66.104.169.179: neighbour-news.com. Hit.
  • 66.104.169.180: medicatechinfo.com. Hit.
    • 205.178.189.131: securitytrails.com 2009-06-25 - 2009-07-02 Network Solutions, LLC., "ip_count": 726755. Moved to new one 2009-07-02 - 2010-11-03
  • 66.104.169.181: brickmanfinancialnews.com. Hit.
  • 66.104.169.182: casanewsnow.com. Hit.
  • 66.104.169.183: aworldofnews.com. No archives.
  • 66.104.169.184: bcenews.com. Hit.
  • 66.104.169.197: teamshula.com. Legit.
66.104.173.186 myworldlymusic.com. Found with: 2013 DNS Census virtual host cleanup heuristic keyword searches. Tested viewdns.info range: 66.104.173.158 - 66.104.173.194
  • 66.104.173.161: fanatic-pc-gamers.com. 2013: Welcome to the US Petabox
  • 66.104.173.163: runakonews.com. Hit.
  • 66.104.173.164: shoppingadventure.net. Hit.
  • 66.104.173.165: entertaining-ly.com. Hit.
  • 66.104.173.166: zubeenews.com. Hit.
  • 66.104.173.169: smart-financeology.com. Hit.
  • 66.104.173.173: remarkably has two potential hits, both shown in viewdns.info, and one of them was also in the 2013 DNS Census.
    • worldfeedstoday.com. No main page archives. Subpage archive: 2011. English. news.
    • world-newsfeeds.com. No archives.
  • 66.104.173.175: media-coverage-now.com. Hit.
  • 66.104.173.176: jbc-online-news.com. Hit.
  • 66.104.173.177: webscooper.com. Hit.
  • 66.104.173.178: dk-dcinvestment.com. Hit.
  • 66.104.173.179: newsforthetech.com. Welcome to the US Petabox.
  • 66.104.173.180: stara-turistick.com. Hit.
  • 66.104.173.181: playbackpolitics.com. Hit.
  • 66.104.173.182: snapnewsfront.net. Hit.
  • 66.104.173.183: ingenuitytrendz.com. Hit.
  • 66.104.173.184: armashoy.com. Hit.
  • 66.104.173.185: baocontact.com. Hit.
  • 66.104.173.186: myworldlymusic.com. Hit.
  • 66.104.173.189: hitpoint-gaming.com. Hit.
66.104.175.40 beyondnetworknews.com. whois.arin.net/rest/net/NET-66-104-0-0-1/pft?s=66.104.175.40. Net Range:66.104.0.0 - 66.107.255.255. 2012 Internet Census puts most/all hits in this range under ip66-104-175-34.z175-104-66.customer.algx.net, algx.net redirects to verizon.com as of 2023. Related: superuser.com/questions/956568/why-are-my-pings-going-to-customer-algx-net. Tested viewdns.info range: 66.104.175.24 - unknown
  • 66.104.175.34: itwebtoday.com. Hit.
  • 66.104.175.35: drglobalnews.com. Hit.
  • 66.104.175.36: adilnews.net. Hit.
  • 66.104.175.37: technewstogo.com. web.archive.org/web/20110201205946/http://technewstogo.com/ "UNDER CONSTRUCTION"
  • 66.104.175.40: beyondnetworknews.com. Hit.
  • 66.104.175.41: grubbersworldrugbynews.com. Hit.
  • 66.104.175.44: yourtripfinder.net. Hit.
  • 66.104.175.45: rollinsnetwork.com. Hit.
  • 66.104.175.46: infosharenews.com. Hit.
  • 66.104.175.47: southasiaheadlines.com. Hit.
  • 66.104.175.48: worlddispatch.net. Hit.
  • 66.104.175.49: webworldsports.com. Hit.
  • 66.104.175.50: fly-bybirdies.com. Hit.
  • 66.104.175.51: businessexchangetoday.com. Hit.
  • 66.104.175.52: mensajeradenoticias.com. Hit.
  • 66.104.175.53: info-ology.net. Hit.
  • 66.104.175.54: marketflows.net. Hit.
  • 66.104.175.57: metanewsdaily.com. Hit.
  • 66.104.175.218: remote.taxconsultantsgroup.com. No archives.
66.175.106.148 activegaminginfo.com. whois.arin.net/rest/net/NET-66-175-106-128-1/pft?s=66.175.106.148: Net Range: 66.175.106.128 - 66.175.106.159. Customer Name: DIAMOND-COLESON. Tested viewdns.info range: 66.175.106.131 - 66.175.106.178
  • 66.175.106.10: nationalchecktrust.com. Legit?
  • 66.175.106.134: paddlescoop.com. Hit.
  • 66.175.106.137: kessingerssportsnews.com. Hit.
  • 66.175.106.138: factorforcenews.com. Hit.
  • 66.175.106.140: aroundthemiddleeast.com. No Wayback Machine hits. Last resolved: 2012-06-29.
  • 66.175.106.142: kanata-news.com. Hit.
  • 66.175.106.143: thecricketfan.com. Hit.
  • 66.175.106.146: inews-today.com. Initially found with 2013 DNS Census virtual host cleanup heuristic keyword searches which gave IP address 193.203.49.212. But that has no nearby hits. 66.175.106.146 was later found on viewdns.info, and slotted into this other existing IP range.
    • 193.203.49.211 datingso.com: legit? Russian dating website
    • 193.203.49.212 inews-today.com. Hit.
    • 193.203.49.223 zatysi.net: legit
    • 193.203.49.226 kinotopik.com: legit? Russian
    • 193.203.49.229 rotor-volgograd.com. Legit.
    • 193.203.49.233 ordercytotec.com. Broken.
  • 66.175.106.147: starwarsweb.net. Hit.
  • 66.175.106.149: feedsdemexicoyelmundo.com. Hit.
  • 66.175.106.150: noticiasmusica.net. Hit.
  • 66.175.106.155: atomworldnews.com. Hit.
  • 66.175.106.158: nouvellesetdesrapports.com. Hit.
  • 66.175.106.166: exchange.katzbarron.com. Legit. Reverse IP source: 2012 Internet Census
  • 66.175.106.183: mail.lfdatacenter.com. No archives.
66.237.236.247 comunidaddenoticias.com. Tested viewdns.info range: 66.237.236.222 - 66.237.236.254
  • 66.237.236.227: newsandmusicminute.com. Hit.
  • 66.237.236.229: pearls-playlist.com 2011-11-13. Hit.
  • 66.237.236.230: beyondthefringe.info 2013-01-02. Hit.
  • 66.237.236.231: primetimemovies.net 2011-06-22. Hit.
  • 66.237.236.235: persephneintl.com. Hit.
  • 66.237.236.236: directoalgrano.net 2012-01-23. Hit.
  • 66.237.236.240: actualizaciondebeisbol.com. Hit.
  • 66.237.236.243: mygadgettech.com. Hit.
  • 66.237.236.247: comunidaddenoticias.com. Hit.
  • 66.237.236.249: sumerjaseahora.com. Hit.
69.84.156.90 stickshiftnews.com. Found with: 2013 DNS Census virtual host cleanup heuristic keyword searches. Tested viewdns.info range: 69.84.156.64 - 69.84.156.95
  • 69.84.156.69: al-ashak-news-me.com. Hit.
  • 69.84.156.70: theventurenews.info. No archives. business.
  • 69.84.156.71: worldfinancetoday.net. Hit.
  • 69.84.156.72: autonewsarabia.com. Hit.
  • 69.84.156.74: blue-moon-news.com. Hit.
  • 69.84.156.75: theoutergreen.com. No archives. Might have been another golf hit.
  • 69.84.156.76: tnc-urdu.com. Hit.
  • 69.84.156.79: jassimnews.com. No archives/broken.
  • 69.84.156.80: noticiasdenuestromundo.com. No archives. Spanish. news.
  • 69.84.156.82: arabicnewsonline.com. Hit.
  • 69.84.156.83: unganadormundial.com. Hit.
  • 69.84.156.84: focusonbokeh.com. No archives/broken. Only a "Sony" logo remains: web.archive.org/web/20110207222330/http://focusonbokeh.com/images/logo_014.jpg
  • 69.84.156.85: classic-rocktopia.com. No archives. Presumably rock climbing.
  • 69.84.156.87: i7diver.com. No archives.
  • 69.84.156.88: diariodeelmundo.com. Hit.
  • 69.84.156.89: todaysarabnews.com. Hit.
  • 69.84.156.90: stickshiftnews.com. Hit.
  • 69.84.156.91: theinternationalgoal.com. Hit.
74.116.72.236 techtopnews.com. Found with: 2013 DNS Census virtual host cleanup heuristic keyword searches. Tested viewdns.info range: 74.116.72.215 - 74.116.72.254
  • 74.116.72.199: newsungraphics.com. Legit.
  • 74.116.72.209: newsung.com. Legit/broken.
  • 74.116.72.214: ofinancialinc.com. Legit.
  • 74.116.72.219: stockpromoters.com. Legit.
  • 74.116.72.227: dayenews.com. hit.
  • 74.116.72.229: guide-daventure.com. Hit.
  • 74.116.72.230: spaceage-exchange.com. No archives.
  • 74.116.72.231: bleachersfootballnews.com. Hit.
  • 74.116.72.232: indirectfreekick.com. Hit.
  • 74.116.72.233: wwiichronicles.net. Hit.
  • 74.116.72.234: petroleumagenews.com. Hit.
  • 74.116.72.235: the-open-book-online.com. Hit.
  • 74.116.72.236: techtopnews.com. Hit.
  • 74.116.72.237: noticiasdiariasdedeportes.com. No archives. Sad, another potential Brazil hit.
  • 74.116.72.238: pohandakhbar.com. No archives. TODO meaning. "akhbar" is news in Arabic. But what is "Poh"? Sounds like a South Asian name.
  • 74.116.72.239: crickettoday.info. Hit.
  • 74.116.72.240: zafernews.com. Hit.
  • 74.116.72.241: itechnewstoday.com. Broken/GoDaddy takeover
  • 74.116.72.242: gdgtsource.com. Hit.
  • 74.116.72.243: waronfilmonline.com. No archives.
  • 74.116.72.244: arborstribune.org. No archives.
  • 74.116.72.245: wineenthusiastonline.com. Welcome to the US Petabox.
  • 74.116.72.246: vuvuzelanews.com. Hit.
  • 74.116.72.247: ballbatstumpsandbails.com. Hit.
  • 74.116.72.248: kioni-sailing.com. No archives.
  • 74.116.72.249: round-trip-travel.com. Hit.
  • 74.116.72.250: arabicnewsource.com. Hit.
74.254.12.168 non-stop-news.net. Found with: 2013 DNS Census virtual host cleanup heuristic keyword searches. Tested viewdns.info range: 74.254.12.158 - 74.254.12.195. This domain exceptionally also has a second IP also with multihits: 207.239.196.230. The fact that the range has rdns sources with hits from both 2013 DNS Census and viewdns.info suggests this range is correct.
  • 74.254.12.163: half-court.net. Hit.
  • 74.254.12.163: dailywellnessnews.com. Hit.
  • 74.254.12.165: dylandon.net. Hit. rdns source: viewdns.info.
  • 74.254.12.166: afghanpoetry.net. Hit.
  • 74.254.12.168: non-stop-news.net. Hit.
  • 74.254.12.169: soldiersofsouthasia.com. Hit.
  • 74.254.12.170: greek-news.info. 2013. Welcome to the US Petabox. rdns source: viewdns.info
  • 74.254.12.171: autism-news.org. Hit.
  • 74.254.12.172: thesportsguidebook.com. rdns source: 2013 DNS Census. Only has archive of one subpage: 2009. English. sports.
  • 74.254.12.174: reliefline.info. web.archive.org/web/20090416064302/http://www.reliefline.info:80/ Archive too broken.
  • 74.254.12.176: pakcricketgrd.com. Hit.
  • 74.254.12.177: networkofnews.com. Hit.
  • 74.254.12.179: wineconnaisseur.net. Hit.
  • 74.254.12.180: helpinghandssite.com. Hit.
  • 74.254.12.185: newskwest.com. No archives.
  • 74.254.12.187: efiinvestment.com. No archives.
  • 74.254.12.188: first-tee-golf.com. Hit.
  • 74.254.12.189: fabu-foto.com. Hit.
  • 74.254.12.190: viptravelabroad.com. Hit.
199.85.212.118 just-kidding-news.com
  • 199.85.212.118 rdns source: 2013 DNS Census virtual host cleanup heuristic keyword searches, dnshistory.org (2009-09-23 -> 2011-01-25) and viewdns.info: "location": "United States", "owner": "VIMRO, LLC", "lastseen": "2012-01-11". Tested viewdns.info range: 199.85.212.95 - 199.85.212.128. Not sure worth it given the many 2013 DNS Census misses surrounding.
    • 199.85.212.98: colorsxpress.com. Legit
    • 199.85.212.104:
      • jobindons.com 2013-10-19.
      • piogroup.org 2012-12-29.
    • 199.85.212.105: mide-news.com. Hit.
    • 199.85.212.109: game2be.com. Infinite load loop: web.archive.org/web/20080102074404/http://www.game2be.com/
    • 199.85.212.111:
      • newsandsportscentral.com. Hit.
      • and many many others, not bothering with it
    • 199.85.212.115: veryperi.com. Legit? 2011. Style is similar.
    • 199.85.212.116: approselect.com. Legit?
    • 199.85.212.117: innovative-software-solutions.com. broken/legit
    • 199.85.212.118: just-kidding-news.com. Hit.
    • 199.85.212.119: invisus.com. Legit
    • 199.85.212.120: allurebyjustine.com. Legit?
    • 199.85.212.121: stockprouniversity.com
    • 199.85.212.122: stjosephswoodshop.com Legit?
    • 199.85.212.125: time-spacer.net. Welcome to the US Petabox.
    • 199.85.212.132: qualitytrans.net. Legit?
    • 199.85.212.134: mywellnessminder.com. Legit?
    • 199.85.212.138: crystalglassinc.com
    • 199.85.212.140: davistech-llc.com
  • 68.178.232.100: see rastadirect.net. rdns source: viewdns.info: "location": "United States", "owner": "GoDaddy.com, LLC", "lastseen": "2012-06-29"
  • 209.85.45.84. Tested viewdns.info range: 209.85.45.74 - 209.85.45.94.
    • 209.85.45.2: dz8.dailyrazor.com
    • 209.85.45.2: jr4consulting.com
    • 209.85.45.41: guitarzza.com. No archives of time.
    • 209.85.45.46: evergraindecking.com. No archives of time.
    • 209.85.45.114: mauritiuspropertyconsultant.com. Legit/ broken.
    • 209.85.45.160: bieltvedt.net. No archives of time.
    • 209.85.45.160: golfstats.dk. No archives.
    • 209.85.45.225: infokus.ca
    • 209.85.45.225: mail.tomlatham.net
    • 209.85.45.225: mail.tomlatham.org
    • 209.85.45.239: flavacationcenter.com
204.176.38.143 noticiassofisticadas.com. Found with: 2013 DNS Census virtual host cleanup. Tested viewdns.info range: 204.176.38.125 - 204.176.38.154
  • 204.176.38.130: i-pressnews.com. Hit.
  • 204.176.38.132: turkishnewslinks.com. Hit.
  • 204.176.38.134: photographyarecord.com. Hit.
  • 204.176.38.135: breakingthewicket.com. Hit.
  • 204.176.38.136: politicalworldtoday.com. Hit.
  • 204.176.38.137: hi-tech-today.com. Hit.
  • 204.176.38.138: continental-business-news.com. TODO. 2011. Cannot find comms. Also header and footer are not limited width which is unusual. Further HTML similarity reversing would be needed.
  • 204.176.38.139: bigscreenbattles.com. Hit.
  • 204.176.38.141: rakotafootball.com. Hit.
  • 204.176.38.142: senderosdemontana.com. Hit.
  • 204.176.38.143: noticiassofisticadas.com. Hit.
  • 204.176.38.144: techno-today.com. Hit.
  • 204.176.38.145: tickettonews.com. Hit.
  • 204.176.38.146: dps-digitalphotosharing.com. Hit.
  • 204.176.38.147: theputtingreen.com. Hit.
  • 204.176.38.149: sportsnewstodayar.com. Hit.
  • 204.176.38.150: kairuafricanews.com. Hit.
204.176.39.115 globalprovincesnews.com. Tested viewdns.info range: 204.176.39.93 - 204.176.39.124
  • 204.176.39.97: beamingnews.com. Hit.
  • 204.176.39.98: cubriendonoticias.com. Hit.
  • 204.176.39.100: rowleyworldpost.com. Hit.
  • 204.176.39.101: noticiastopicas.com. No archives.
  • 204.176.39.103: economicnewsbuzz.com. Hit.
  • 204.176.39.104: spectranewsonline.com. Hit.
  • 204.176.39.105: entertainmentnewscompany.com. Hit.
  • 204.176.39.107: guidetoelectronics.net. Uncertain. 2010. English. tech, electronics. Possible CGI comms variant.
  • 204.176.39.110: arabnewsatdawn.com. Hit.
  • 204.176.39.114: messengergalaxy.com. Uncertain. 2011. Would be the first example of something more commercial/service offering we've seen so far. Possible CGI comms variant.
  • 204.176.39.115: globalprovincesnews.com. Hit.
  • 204.176.39.116: mahparah-news.com. Hit.
  • 204.176.39.119: commercialspacedesign.com. Hit.
207.210.250.132 aeronet-news.com. Found with: 2013 DNS Census virtual host cleanup heuristic keyword searches. Tested viewdns.info range: 207.210.250.126 - 207.210.250.157
  • 207.210.250.131: starrynightnews.com. Hit.
  • 207.210.250.132: aeronet-news.com. Hit.
  • 207.210.250.133: bakaribulletin.com. Hit.
  • 207.210.250.134: deprensaenlarevisiondehoy.com. Hit.
  • 207.210.250.135: icwb-news.com. Hit.
  • 207.210.250.136: sportsreelhighlights.com. Hit.
  • 207.210.250.137: fashionforward.info. No archives.
  • 207.210.250.138: inquiry-human-past.com. Hit.
  • 207.210.250.139: thefairwaysaregreen.com. Hit.
  • 207.210.250.142: russiaupdate.com 2011-11-13. No archives of the time, only older unrelated archives: web.archive.org/web/20010429003443/http://russiaupdate.com/.
  • 207.210.250.143: archaeologyreview.net. Hit.
  • 207.210.250.144: highspeed-news.com. No archives.
  • 207.210.250.146: noticias-caracas.com. Hit.
  • 207.210.250.147: bailandstump.com. Hit.
  • 207.210.250.148: classicalmusic4arab.com. No archives.
  • 207.210.250.149: globalventurestat.com. Hit.
  • 207.210.250.152: al-rashidrealestate.com. Hit.
  • 207.210.250.153: newsintheworld-ru.com. Hit.
  • 207.210.250.154: news-unlimited.info. No archives. Shame, as perfect theme, and has per ipinf.ru/domains/207.210.250.154/
208.254.40.117 worldnewsandent.com. whois.arin.net/rest/net/NET-208-192-0-0-1/pft?s=208.254.40.117: Net Range 208.192.0.0 - 208.255.255.255. Tested viewdns.info range: 208.254.40.92 - 208.254.40.135
  • 208.254.40.96: sixty2media.com. Hit.
  • 208.254.40.99: newspoliticssource.com. Hit.
  • 208.254.40.110 musical-fortune.net. Hit.
  • 208.254.40.113: ashoka-gemstones.com. Hit.
  • 208.254.40.117: worldnewsandent.com. Hit.
  • 208.254.40.124: riskandrewardnews.com. Hit.
  • 208.254.40.129: mailb.casella.com. Legit.
208.254.42.205 driversinternationalgolf.com. Not too far from 208.254.40.117 right? Tested viewdns.info range: 208.254.42.178 - 208.254.42.233.
210.80.75.55 philippinenewsonline.net. Tested viewdns.info range: 210.80.75.30 - 210.80.75.67
  • 210.80.75.35: aroundtheworldnews.net. No archives. ipinf.ru/domains/210.80.75.33/ disagrees and places it at .33.
  • 210.80.75.36: e-commodities.net. Hit.
  • 210.80.75.37: trekkingtoday.com. Hit.
  • 210.80.75.41: multinews-33.com. Hit.
  • 210.80.75.42: movimientodenticias.com. No archives.
  • 210.80.75.43: gulfandmiddleeastnews.com. Hit.
  • 210.80.75.44: whirlybirdinflight.com. Hit.
  • 210.80.75.45: kings-game.net. Hit.
  • 210.80.75.46: topglobalnewsdaily.com. Hit.
  • 210.80.75.49: recipe-dujour.com. Hit.
  • 210.80.75.53: sportsman-elite.com. No archives.
  • 210.80.75.55: philippinenewsonline.net. Hit.
  • 210.80.75.56: technewsforme.com. Hit.
  • 210.80.75.59: goldeportesnoticias.com. No archives.
  • 210.80.75.68: gigabyte-usa.com. Legit.
212.4.16.232 mynewscheck.com. Found with: 2013 DNS Census virtual host cleanup heuristic keyword searches. Tested viewdns.info range: 212.4.16.214 - 212.4.17.10.
Other hits:
  • 208.91.197.132. rdns source: viewdns.info: "location" : "British Virgin Islands", "owner" : "Confluence Networks Inc", "lastseen" : "2013-09-26". So this is after the previous one, unlikely to be correct.
  • 205.178.189.131. source: securitytrails.com
212.4.17.38 fightwithoutrules.com. whois.arin.net/rest/net/NET-208-192-0-0-1/pft?s=208.254.40.117. Net Range: 208.192.0.0 - 208.255.255.255. Organization: Name: Verizon Business. Tested viewdns.info range: 212.4.17.8 - 212.4.17.79
  • 212.4.17.41: newtechfrontier.com. Hit.
  • 212.4.17.43: smart-travel-consultant.com. Hit.
  • 212.4.17.46: atentlaloc.com. Hit.
  • 212.4.17.53: newsresolution.net. Hit.
  • 212.4.17.56: lesummumdelafinance.com. Hit.
  • 212.4.17.56: thepinnacleoffinance.com. No Wayback machine archives.
  • 212.4.17.61: tech-stop.org. Archive: 2011. Feels likely. No commons found. .org hit? Has subdomain "gear.tech-stop.org" according to 2013 DNS Census, which suggests CGI comms, but no links to it
  • 212.4.17.98: topbillingsite.com. Hit.
  • 212.4.17.122: b2bworldglobal.com. Hit.
There were also some other reverse IP hits for fightwithoutrules.com, but no CIA websites there:
  • 204.11.56.25 - British Virgin Islands - Confluence Networks Inc - 2013-09-26. Many domains.
  • 208.91.197.19 - British Virgin Islands - Confluence Networks Inc - 2013-05-20. Many domains.
212.4.18.129 sightseeingnews.com. Found with: 2013 DNS Census virtual host cleanup heuristic keyword searches. Tested viewdns.info range: 212.4.18.115 - 212.4.18.148. TODO expand. Interesting wide/sparse range? Or perhaps it's two separate ranges?
212.209.74.105 globalbaseballnews.com. Tested viewdns.info range: 212.209.74.100 - 212.209.74.132. Found with: 2013 DNS Census virtual host cleanup heuristic keyword searches
  • 212.209.74.105: globalbaseballnews.com. Hit.
  • 212.209.74.106: football-de-luxe.com. Hit.
  • 212.209.74.111: worldconcerns.info. No archives.
  • 212.209.74.112: developmental-league.com. Unclear. CGI comms variant? 2010. English. CGI. American football.
  • 212.209.74.115: mediocampodefutbol.com. Hit.
  • 212.209.74.117: myengineeringaffinity.com. Hit.
  • 212.209.74.122: atthemovies.biz. Archive very broken. Has link to unarchived JAR: web.archive.org/web/20110809232811oe_/http://www.atthemovies.biz/movieslides.jar. Would have been the fist .biz hit found: Non .com .net TLDs
  • 212.209.74.123: worldfinancialexchangenews.com. Hit.
  • 212.209.74.124: urouttahere.com. No archives. Meaning presumably "you're out of here"? One wonders what the theme would have been!
  • 212.209.74.125: avoilurefixe.com. Hit.
  • 212.209.74.126: headlines2day.com. Hit.
    • 118.139.174.11. Reverse IP source: viewdns.info
      • 118.139.174.11: 712 domain hits on it
      • 118.139.174.21: theargentineanwineco.com 2013-09-26. No Wayback machine archive.
      • nothing else on the +-20 range
    • 184.168.221.91. Reverse IP source: 2013 DNS Census
  • 212.209.74.127: construction-zones.com. Unclear. CGI comms variant? 2009. No known comms found. English. construction. Has a login page: web.archive.org/web/20091130144158/http://construction-zones.com/login.html so maybe CGI comms variant
212.209.79.40 hydradraco.com. Found with: visual inspection of full 2013 DNS Census virtual host cleanup list just after globalbaseballnews.com. Tested viewdns.info range: 212.209.79.35 - 212.209.79.63
  • 212.209.79.34: fgnl.net. Hit. securitytrails.com provides IP history:
    • 212.209.79.34: 2008-09-01 - 2010-04-19.
    • 212.4.18.133: 2010-04-19 - 2019-06-19. Tested viewdns.info range: 212.4.18.122 - 212.4.18.148
    both under MCI Communications Services, Inc. d/b/a Verizon Business.
  • 212.209.79.37: fitness-sources.com. Hit.
  • 212.209.79.40: hydradraco.com. Hit.
  • 212.209.79.41: noticiasdelmundolatino.com. Hit.
  • 212.209.79.42: suparakuvi.com. Hit.
  • 212.209.79.44: myigadgets.net. Unclear. 2010. tech. Contains some helpers to: iGoogle. This page is very interesting. and quite different from the others, as it contains highly specialized functionality. No known comms found. The choice of homepage languages is also very suspicious: Arabic, Farsi, French, Chinese and Spanish.
  • 212.209.79.46: cetusdelph.com. Hit.
  • 212.209.79.47: willtoworship.com. Hit.
  • 212.209.79.48: themvconnection.com. Hit.
  • 212.209.79.51: pi-resources.net. Hit.
  • 212.209.79.52: newel-adserver.com. Redirects to newel.com which is legit.
  • 212.209.79.53: ourscubaworld.com. Hit.
  • 212.209.79.58: tech-love-home.com. Hit.
  • 212.209.79.60: first-solo-aviation.com. Hit.
  • 212.209.79.61: china-destinations.org. Hit.
212.209.90.84 thenewseditor.com. Found with: 2013 DNS Census virtual host cleanup heuristic keyword searches. Tested viewdns.info range: 212.209.90.64 - 212.209.90.99
  • 212.209.90.69: worldedgenews.com. Hit.
  • 212.209.90.72: talkingpointnews.info. No archives.
  • 212.209.90.75: prebitinvestment.com. No archives.
  • 212.209.90.77: energy-bulb.com 2011. English. energy. Comms not found, but has unarchived link to: web.archive.org/web/20110128182345/https://webmail.energy-bulb.com/login.html. CGI comms variant?
  • 212.209.90.79: freeblink.com. No archives for timerange, then legit.
  • 212.209.90.80: nsmovies.net. Hit.
  • 212.209.90.82: middleeastjournal.net. Hit.
  • 212.209.90.84: thenewseditor.com. Hit.
  • 212.209.90.87: newsandweathersource.com. Hit.
  • 212.209.90.89: pakisports.com. Hit.
  • 212.209.90.90: vriha-aesthetics.com. Hit.
  • 212.209.90.92: amishkanews.com. Hit.
  • 212.209.90.93: theentertainbiz.com. Hit.
  • 212.209.90.94: eurosportssummary.com. Hit.
  • 212.209.91.14: teracom.net. Legit
216.105.98.152: modernarabicnews.com. Found with: 2013 DNS Census virtual host cleanup heuristic keyword searches. Tested viewdns.info range: 216.105.98.125 - 216.105.98.167
  • 216.105.98.118:
    • estudashboard.com: broken
    • fintrade.us: legit
  • 216.105.98.132: europeantravelcafe.com. Likely a hit, but comms not found. 2010. English. Europe. travel. Marked copyright 2009. There's a currency converter at: web.archive.org/web/20100724024644/http://www.europeantravelcafe.com/tools.html which could be suspicious.
  • 216.105.98.134: fuenteneta.com. No archives.
  • 216.105.98.135: ilat-news.com. No archives.
  • 216.105.98.136: etherealinspirations.net. No archives.
  • 216.105.98.137: the-news-zone.com. Archive very broken: web.archive.org/web/20130814194744/http://the-news-zone.com/
  • 216.105.98.138: photozoomnews.com. No archives.
  • 216.105.98.139: cultura-digital.net. Hit.
  • 216.105.98.140: uaeshoppingspree.com. Hit.
  • 216.105.98.141: jabarifootball.com. No archives. "Jabari" is a Swahili/Arabic name[ref]
  • 216.105.98.142: globalreview-ar.com. No archives. Shame, could have been our first Argentinian site.
  • 216.105.98.144: garanziadellasicurezza.com. Archives quite broken: web.archive.org/web/20110424044637/http://www.garanziadellasicurezza.com:80/ Unarchived JAR: /web/20110424044637oe_/http://www.garanziadellasicurezza.com/garanzia.jar Would be another precious Italy hit...
  • 216.105.98.145: montanismoaventura.com. Hit.
  • 216.105.98.146: large-format-news.com. No archives.
  • 216.105.98.147: nepalnewsbrief.com. Hit. dnshistory.org marks it as having IP 2010-03-10 -> 2010-08-15 216.169.148.94 [ref]. This range does feel a bit different from the others, too many broken archives, and relatively early ones too. Explored viewdns.info range: 216.169.148.84 - 216.169.148.104, empty for period.
  • 216.105.98.148: teclafinance.com. No archives. One wonders what "tecla" would have stood for. It is Portuguese for "keyboard key", but finance is English so.
  • 216.105.98.149: entreman.com: legit? web.archive.org/web/20110128212738/http://entreman.com/
  • 216.105.98.152: modernarabicnews.com. Hit.
  • 216.105.98.153: global-headlines.com. No archives of the period, then was a legitimate WordPress website for a while.
  • 216.105.98.154: everythingcricket.org. Hit.
  • 216.105.98.156: familyhealthonline.net. Hit.
  • 216.105.98.157: delacorne.com. No archives.
  • 216.105.98.158: econfutures.com. No archives.
  • 216.105.98.161: kstcloud.com. No archives.
219.90.61.123 journeystravelled.com Tested viewdns.info range: 219.90.61.100 - 219.90.61.133
  • 219.90.61.100: pressstory.com: "Under construction". web.archive.org/web/20110128124548/http://pressstory.com/
  • 219.90.61.103: bet2plays.com. "Under construction". Unlikely thematic, too spicy.
  • 219.90.61.110: surya-brahma.com. Hit
  • 219.90.61.111: classicalmusicboxonline.com. Hit.
  • 219.90.61.116: athletepro.net. Hit.
  • 219.90.61.117: lajornadanow.com. Hit.
  • 219.90.61.119: aviation-navigation.com. No archives.
  • 219.90.61.120: theinternationalworld.com. Hit.
  • 219.90.61.121: thepyramidnews.com. Hit.
  • 219.90.61.122: iran-newslink-today.com. Hit.
  • 219.90.61.123: journeystravelled.com. Hit.
219.90.62.243 fitness-dawg.com. whois.arin.net/rest/net/NET-219-0-0-0-1/pft?s=219.90.62.243. Net Type: Allocated to APNIC. Tested viewdns.info range: unknown - 219.90.62.255
  • 219.90.62.173:
    • dominatingduos.com: 2013-08-12T17:53:09. No archive
    • has other domains
  • 219.90.62.193: centralnewsreleasers.com. Only a 2018 of the robots.txt: web.archive.org/web/*/http://centralnewsreleasers.com/* so likely not a hit
  • 219.90.62.209: penniesbythemillions.com. No archives.
  • 219.90.62.229: information-junky.com. Hit.
  • 219.90.62.231: todosperuahora.com. Hit.
  • 219.90.62.232: race26point2.com. Hit. No archives, but has subdomain: secure.race26point2.com, so likely CGI comms.
  • 219.90.62.233: theworld-news.net. Hit.
  • 219.90.62.234: recuerdosdeviajeonline.com. Hit
  • 219.90.62.235: ordenpolicial.com. No Wayback Machine archives. Last resolved: 2012-01-11.
  • 219.90.62.237: elcorreodenoticias.com. Hit.
  • 219.90.62.238: freshtechonline.com. Hit.
  • 219.90.62.240: cityworldnewsnow.com. Hit. No archives but has subdomain: secure.cityworldnewsnow.com so likely CGI comms.
  • 219.90.62.241: newscentertoday.com. Hit.
  • 219.90.62.242: ride-captain.com. Hit.
  • 219.90.62.244: easytraveleurope.com. Hit.
  • 219.90.62.245: world-news-now.net. Hit.
  • 219.90.62.246: negativeaperture.com. Hit.
  • 219.90.62.247: conquermstoday.com. Hit
  • 219.90.62.249: forensic-exchange.com. 2013 archive: web.archive.org/web/20130714094026/http://forensic-exchange.com/. Appears to be a buggy Wayback Machine archive somehow, so inconclusive.
Read the full article
Wayback Machine CDX scanning with Tor parallelization Updated 2025-01-29 +Created 1970-01-01
View more
Dire times require dire methods: cia-2010-covert-communication-websites/cdx-tor.sh.
First we must start the tor servers with the tor-army command from: stackoverflow.com/questions/14321214/how-to-run-multiple-tor-processes-at-once-with-different-exit-ips/76749983#76749983
tor-army 100
and then use it on a newline separated domain name list to check;
./cdx-tor.sh infile.txt
This creates a directory infile.txt.cdx/ containing:
  • infile.txt.cdx/out00, out01, etc.: the suspected CDX lines from domains from each tor instance based on the simple criteria that the CDX can handle directly. We split the input domains into 100 piles, and give one selected pile per tor instance.
  • infile.txt.cdx/out: the final combined CDX output of out00, out01, ...
  • infile.txt.cdx/out.post: the final output containing only domain names that match further CLI criteria that cannot be easily encoded on the CDX query. This is the cleanest domain name list you should look into at the end basically.
Since archive is so abysmal in its data access, e.g. a Google BigQuery would solve our issues in seconds, we have to come up with creative ways of getting around their IP throttling.
The CIA doesn't play fair. They're actually the exact opposite of fair. So neither shall we.
This should allow a full sweep of the 4.5M records in 2013 DNS Census virtual host cleanup in a reasonable amount of time. After JAR/SWF/CGI filtering we obtained 5.8k domains, so a reduction factor of about 1 million with likely very few losses. Not bad.
5.8k is still a bit annoying to fully go over however, so we can also try to count CDX hits to the domains and remove anything with too many hits, since the CIA websites basically have very few archives:
cd 2013-dns-census-a-novirt-domains.txt.cdx
./cdx-tor.sh -d out.post domain-list.txt
cd out.post.cdx
cut -d' ' -f1 out | uniq -c | sort -k1 -n | awk 'match($2, /([^,]+),([^)]+)/, a) {printf("%s.%s %d\n", a[2], a[1], $1)}' > out.count
This gives us something like:
12654montana.com 1
aeronet-news.com 1
atohms.com 1
av3net.com 1
beechstreetas400.com 1
sorted by increasing hit counts, so we can go down as far as patience allows for!
New results from a full CDX scan of 2013-dns-census-a-novirt.csv:
  • 219.90.61.123 journeystravelled.com
Read the full article