When you Google most of the hit domains, many of them show up on "expired domain trackers", and above all Chinese expired domain trackers for some reason, notably e.g.:
This suggests that scraping these lists might be a good starting point to obtaining "all expired domains ever".
Data comparison:
We've made the following pipelines for hupo.com + webmasterhome.cn merging:
./hupo.sh &
./webmastercn.sh &
./justdropped.sh &
wait
./justdropped-post.sh
./hupo-merge.sh
# Export as small Google indexable files in a Git repository.
./hupo-repo.sh
# Export as per year zips for Internet Archive.
./hupo-zip.sh
# Obtain count statistics:
./hupo-wc.sh
Count unique domains in the repos:
( echo */*/*/* | xargs cat ) | sort -u | wc
The extracted data is present at:Soon after uploading, these repos started getting some interesting traffic, presumably started by security trackers going "bling bling" on certain malicious domain names in their databases:
  • GitHub trackers:
    • admin-monitor.shiyue.com
    • anquan.didichuxing.com
    • app.cloudsek.com
    • app.flare.io
    • app.rainforest.tech
    • app.shadowmap.com
    • bo.serenety.xmco.fr 8 1
    • bts.linecorp.com
    • burn2give.vercel.app
    • cbs.ctm360.com 17 2
    • code6.d1m.cn
    • code6-ops.juzifenqi.com
    • codefend.devops.cndatacom.com
    • dlp-code.airudder.com
    • easm.atrust.sangfor.com
    • ec2-34-248-93-242.eu-west-1.compute.amazonaws.com
    • ecall.beygoo.me 2 1
    • eos.vip.vip.com 1 1
    • foradar.baimaohui.net 2 1
    • fty.beygoo.me
    • hive.telefonica.com.br 2 1
    • hulrud.tistory.com
    • kartos.enthec.com
    • soc.futuoa.com
    • lullar-com-3.appspot.com
    • penetration.houtai.io 2 1
    • platform.sec.corp.qihoo.net
    • plus.k8s.onemt.co 4 1
    • pmp.beygoo.me 2 1
    • portal.protectorg.com
    • qa-boss.amh-group.com
    • saicmotor.saas.cubesec.cn
    • scan.huoban.com
    • sec.welab-inc.com
    • security.ctrip.com 10 3
    • siem-gs.int.black-unique.com 2 1
    • soc-github.daojia-inc.com
    • spigotmc.org 2 1
    • tcallzgroup.blueliv.com
    • tcthreatcompass05.blueliv.com 4 1
    • tix.testsite.woa.com 2 1
    • toucan.belcy.com 1 1
    • turbo.gwmdevops.com 18 2
    • urlscan.watcherlab.com
    • zelenka.guru. Looks like a Russian hacker forum.
  • LinkedIn profile views:
Check for overlap of the merge:
grep -Fx -f <( jq -r '.[].host' ../media/cia-2010-covert-communication-websites/hits.json ) cia-2010-covert-communication-websites/tmp/merge/*
Next, we can start searching by keyword with Wayback Machine CDX scanning with Tor parallelization with out helper ../cia-2010-covert-communication-websites/hupo-cdx-tor.sh, e.g. to check domains that contain the term "news":
./hupo-cdx-tor.sh mydir 'news|global' 2011 2019
produces per-year results for the regex term news|global between the years under:
tmp/hupo-cdx-tor/mydir/2011
tmp/hupo-cdx-tor/mydir/2012
OK lets:
./hupo-cdx-tor.sh out 'news|headline|internationali|mondo|mundo|mondi|iran|today'
Other searches that are not dense enough for our patience:
world|global|[^.]info
OMG news search might be producing some golden, golden new hits!!! Going full into this. Hits:
  • thepyramidnews.com
  • echessnews.com
  • tickettonews.com
  • airuafricanews.com
  • vuvuzelanews.com
  • dayenews.com
  • newsupdatesite.com
  • arabicnewsonline.com
  • arabicnewsunfiltered.com
  • newsandsportscentral.com
  • networkofnews.com
  • trekkingtoday.com
  • financial-crisis-news.com
and a few more. It's amazing.
TODO what does this Chinese forum track? New registrations? Their focus seems to be domain name speculation
Some of the threads contain domain dumps. We haven't yet seen a scrapable URL pattern, but their data goes way back and did have various hits. The forum seems to have started in 2006: club.domain.cn/forum.php?mod=forumdisplay&fid=41&page=10127
club.domain.cn/forum.php?mod=viewthread&tid=241704 "【国际域名拟删除列表】2007年06月16日" is the earliest list we could find. It is an expired domain list.
Some hits:
pastebin.com/CTXnhjeS dated mega early on Sep 30th, 2012 by CYBERTAZIEX.
This source was found by Oleg Shakirov.
Holy fuck the type of data source that we get in this area of work!
This pastebin contained a few new hits, in addition to some pre-existing ones. Most of the hits them seem to be linked to the IP 72.34.53.174, which presumably is a major part of the fingerprint found by CYBERTAZIEX, though unsurprisingly methodology is unclear. As documented, the domains appear to be linked to a "Condor hosting" provider, but it is hard to find any information about it online.
From the title, it would seem that someone hacked into Condor and defaced all of its sites, including unknowingly some CIA ones which is LOL.
Ciro Santilli checked every single non-subdomain domain in the list.
Other files under the same account: pastebin.com/u/cybertaziex did not seem of interest.
The author's real name appears to be Deni Suwandi: twitter.com/denz_999 from Indonesia, but all accounts appear to be inactive, otherwise we'd ping him to ask for more info about the list.
www.zone-h.com lists some of the domains. They also seem to have intended to have snapshots of the defaces but we can't see them which is sad:
alljohnny.com had a hit: ipinf.ru/domains/alljohnny.com/, and so Ciro started looking around... and a good number of other things have hits.
Not all of them, definitely less data than viewdns.info.
But they do reverse IP, and they show which nearby reverse IPs have hits on the same page, for free, which is great!
Shame their ordering is purely alphabetical, doesn't properly order the IPs so it is a bit of a pain, but we can handle it.
OMG, Russians!!!
The data here had a little bit of non-overlap from other sources. 4 new confirmed hits were found, plus 4 possible others that were left as candidates.
In this section we document the outcomes of more detailed inspection of both the communication mechanisms (JavaScript, JAR, swf) and HTML that might help to better fingerprint the websites.
There are four main types of communication mechanisms found:
  • There is also one known instance where a .zip extension was used! web.archive.org/web/20131101104829*/http://plugged-into-news.net/weatherbug.zip as:
    <applet codebase="/web/20101229222144oe_/http://plugged-into-news.net/" archive="/web/20101229222144oe_/http://plugged-into-news.net/weatherbug.zip"
    JAR is the most common comms, and one of the most distinctive, making it a great fingerprint.
    Several of the JAR files are named something like either:
    as if to pose as Internet speed testing tools? The wonderful subtleties of the late 2000s Internet are a bit over our heads.
    All JARs are directly under root, not in subdirectories, and the basename usually consist of one word, though sometimes two camel cased.
  • JavaScript file. There are two subtypes:
    • JavaScript with SHAs. Rare. Likely older. Way more fingerprintable.
    • JavaScript without SHAs. They have all been obfuscated slightly different and compressed. But the file sizes are all very similar from 8kB to 10kB, and they all look similar, so visually it is very easy to detect a match with good likelyhood.
  • Adobe Flash swf file. In all instances found so far, the name of the SWF matches the name of the second level domain exactly, e.g.:
    http://tee-shot.net/tee-shot.swf
    While this is somewhat of a fingerprint, it is worth noting that is was a relatively commonly used pattern. But it is also the rarest of the mechanisms. This is a at a dissonance with the rest of the web, which circa 2010 already had way more SWF than JAR apparently.
    Some of the SWF websites have archives for empty /servlet pages:
    ./bailsnboots.com/20110201234509/servlet/teammate/index.html
    ./currentcommunique.com/20110130162713/servlet/summer/index.html
    ./mynepalnews.com/20110204095758/servlet/SnoopServlet/index.html
    ./mynepalnews.com/20110204095403/servlet/release/index.html
    ./www.hassannews.net/20101230175421/servlet/jordan/index.html
    ./zerosandonesnews.com/20110209084339/servlet/technews/index.html
    which makes us think that it is a part of the SWF system.
  • CGI comms
These have short single word names with some meaning linked to their website.
Because the communication mechanisms are so crucial, they tend to be less varied, and serve as very good fingerprints. It is not ludicrous, e.g. identical files, but one look at a few and you will know the others.
We've come across a few shallow and stylistically similar websites on suspicious ranges with this pattern.
No JS/JAR/SWF comms, but rather a subdomain, and an HTTPS page with .cgi extension that leads to a login page. Some names seen for this subdomain:
  • secure.: most common
  • ssl.: also common
  • various other more creative ones linked to the website theme itself, e.g.:
    • musical-fortune.net has a backstage.musical-fortune.net
The question is, is this part of some legitimate tooling that created such patterns? And if so which? Or are they actual hits with a new comms mechanism not previously seen?
The fact that:
  • hits of this type are so dense in the suspicious ranges
  • they are so stylistically similar between on another
  • citizenlabs specifically mentioned a "CGI" comms method
suggests to Ciro that they are an actual hit.
In particular, the secure and ssl ones are overused, and together with some heuristics allowed us to find our first two non Reuters ranges! Section "secure subdomain search on 2013 DNS Census"
The CGI comms websites contain the only occurrence of HTTPS, so it might open up the door for a certificate fingerprint as proposed by user joelcollinsdc at: news.ycombinator.com/item?id=36280801!
crt.sh appears to be a good way to look into this:
They all appear to use either of:
Let's try another one for secure.altworldnews.com: search.censys.io/certificates/e88f8db87414401fd00728db39a7698d874dbe1ae9d88b01c675105fabf69b94. Nope, no direct mega hits here either.
There are two types of JavaScript found so far. The ones with SHA and the ones without. There are only 2 examples of JS with SHA:Both files start with precisely the same string:
var ms="\u062F\u0631\u064A\u0627\u0641\u062A\u06CC",lc="\u062A\u0647\u064A\u0647 \u0645\u062A\u0646",mn="\u0628\u0631\u062F\u0627\u0632\u0634 \u062F\u0631 \u062C\u0631\u064A\u0627\u0646 \u0627\u0633\u062A...\u0644\u0637\u0641\u0627 \u0635\u0628\u0631 \u0643\u0646\u064A\u062F",lt="\u062A\u0647\u064A\u0647 \u0645\u062A\u0646",ne="\u067E\u0627\u0633\u062E",kf="\u062E\u0631\u0648\u062C",mb="\u062D\u0630\u0641",mv="\u062F\u0631\u064A\u0627\u0641\u062A\u06CC",nt="\u0627\u0631\u0633\u0627\u0644",ig="\u062B\u0628\u062A \u063A\u0644\u0637. \u062C\u0647\u062A \u062A\u062C\u062F\u064A\u062F \u062B\u0628\u062A \u0635\u0641\u062D\u0647 \u0631\u0627 \u0628\u0627\u0632\u0622\u0648\u0631\u06CC \u06A9\u0646\u064A\u062F",hs="\u063A\u064A\u0631 \u0642\u0627\u0628\u0644 \u0627\u062C\u0631\u0627. \u062E\u0637\u0627 \u062F\u0631 \u0627\u062A\u0651\u0635\u0627\u0644",ji="\u063A\u064A\u0631 \u0642\u0627\u0628\u0644 \u0627\u062C\u0631\u0627. \u062E\u0637\u0627 \u062F\u0631 \u0627\u062A\u0651\u0635\u0627\u0644",ie="\u063A\u064A\u0631 \u0642\u0627\u0628\u0644 \u0627\u062C\u0631\u0627. \u062E\u0637\u0627 \u062F\u0631 \u0627\u062A\u0651\u0635\u0627\u0644",gc="\u0633\u0648\u0627\u0631 \u06A9\u0631\u062F\u0646 \u062A\u06A9\u0645\u064A\u0644 \u0634\u062F",gz="\u0645\u0637\u0645\u0626\u0646\u064A\u062F \u06A9\u0647 \u0645\u064A\u062E\u0648\u0627\u0647\u064A\u062F \u067E\u064A\u0627\u0645 \u0631\u0627 \u062D\u0630\u0641 \u06A9\u0646\u064A\u062F\u061F"
Good fingerprint present in all of them:
throw new Error("B64 D.1");};if(at[1]==-1){throw new Error("B64 D.2");};if(at[2]==-1){if(f<ay.length){throw new Error("B64 D.3");};dg=2;}else if(at[3]==-1){if(f<ay.length){throw new Error("B64 D.4")
Googling most domains gives only very few results, and most of them are just useless lists of expired domains. Skipping those for now.
Googling "dedrickonline.com" has a git at www.webwiki.de/dedrickonline.com# Furthermore, it also contains the IP address "65.61.127.174" under the "Technik" tab!
Unfortunately that website appears to be split by language? E.g. the English version does not contain it: www.webwiki.com/dedrickonline.com, which would make searching a bit harder, but still doable.
But if we can Google search those IPs there, we might just hit gold.
IP search did work! www.webwiki.de/65.61.127.174
But doesn't often/ever work unfortunately for others.
Searching on github.com: github.com/DrWhax/cia-website-comms by Jurre van Bergen from September 2022 contains some of the links to some of the ones reported by Reuters including some of their JARs, presumably for reversing purposees. Pinged him at: github.com/DrWhax/cia-website-comms/issues/1
Grepping the 2013 DNS Census first by overused CGI comms subdomains secure. and ssl. leaves 200k lines. Grepping for the overused "news" led to hits:
  • secure.worldnewsandent.com,2012-02-13T21:28:15,208.254.40.117
  • ssl.beyondnetworknews.com,2012-02-13T20:10:13,66.104.175.40
Also tried but failed:
OK, after the initial successes in secure., we went a bit more data intensive:
New results: only one...
  • 208.254.42.205 secure.driversinternationalgolf.com,2012-02-13T10:42:20,
After 2013 DNS Census virtual host cleanup heuristic keyword searches we later understood why there were so few hits here: the 2013 DNS Census didn't capture the secure. subdomains of many domains it had for some reason. Shame, because if it had, this method would have yielded many more results.

Pinned article: Introduction to the OurBigBook Project

Welcome to the OurBigBook Project! Our goal is to create the perfect publishing platform for STEM subjects, and get university-level students to write the best free STEM tutorials ever.
Everyone is welcome to create an account and play with the site: ourbigbook.com/go/register. We belive that students themselves can write amazing tutorials, but teachers are welcome too. You can write about anything you want, it doesn't have to be STEM or even educational. Silly test content is very welcome and you won't be penalized in any way. Just keep it legal!
We have two killer features:
  1. topics: topics group articles by different users with the same title, e.g. here is the topic for the "Fundamental Theorem of Calculus" ourbigbook.com/go/topic/fundamental-theorem-of-calculus
    Articles of different users are sorted by upvote within each article page. This feature is a bit like:
    • a Wikipedia where each user can have their own version of each article
    • a Q&A website like Stack Overflow, where multiple people can give their views on a given topic, and the best ones are sorted by upvote. Except you don't need to wait for someone to ask first, and any topic goes, no matter how narrow or broad
    This feature makes it possible for readers to find better explanations of any topic created by other writers. And it allows writers to create an explanation in a place that readers might actually find it.
    Figure 1.
    Screenshot of the "Derivative" topic page
    . View it live at: ourbigbook.com/go/topic/derivative
  2. local editing: you can store all your personal knowledge base content locally in a plaintext markup format that can be edited locally and published either:
    This way you can be sure that even if OurBigBook.com were to go down one day (which we have no plans to do as it is quite cheap to host!), your content will still be perfectly readable as a static site.
    Figure 2.
    You can publish local OurBigBook lightweight markup files to either https://OurBigBook.com or as a static website
    .
    Figure 3.
    Visual Studio Code extension installation
    .
    Figure 4.
    Visual Studio Code extension tree navigation
    .
    Figure 5.
    Web editor
    . You can also edit articles on the Web editor without installing anything locally.
    Video 3.
    Edit locally and publish demo
    . Source. This shows editing OurBigBook Markup and publishing it using the Visual Studio Code extension.
    Video 4.
    OurBigBook Visual Studio Code extension editing and navigation demo
    . Source.
  3. https://raw.githubusercontent.com/ourbigbook/ourbigbook-media/master/feature/x/hilbert-space-arrow.png
  4. Infinitely deep tables of contents:
    Figure 6.
    Dynamic article tree with infinitely deep table of contents
    .
    Descendant pages can also show up as toplevel e.g.: ourbigbook.com/cirosantilli/chordate-subclade
All our software is open source and hosted at: github.com/ourbigbook/ourbigbook
Further documentation can be found at: docs.ourbigbook.com
Feel free to reach our to us for any help or suggestions: docs.ourbigbook.com/#contact