There are two types of people:
- those who are autodidacts
- those who didn't really learn
Some possible definitions:
- learning without a gun pointed at your head
- learning from an e-book or video rather than from a talking head 5 rows of chairs in front of youHow that is different from a video, you tell me.
Thus OurBigBook.com.
Run output is placed under
out/:Some of the output data is stored as
.cpickle files. To observe those files, you need the original Python classes, and therefore you have to be inside Docker, from the host it won't work.We can list all the plots that have been produced under Plots are also available in SVG and PDF formats, e.g.:
out/ withfind -name '*.png'The output directory has a hierarchical structure of type:where:
./out/manual/wildtype_000000/000000/generation_000000/000000/wildtype_000000: variant conditions.wildtypeis a human readable label, and000000is an index amongst the possiblewildtypeconditions. For example, we can have different simulations with different nutrients, or different DNA sequences. An example of this is shown at run variants.000000: initial random seed for the initial cell, likely fed to NumPy'snp.random.seedgenereation_000000: this will increase with generations if we simulate multiple cells, which is supported by the model000000: this will presumably contain the cell index within a generation
We also understand that some of the top level directories contain summaries over all cells, e.g. the
massFractionSummary.pdf plot exists at several levels of the hierarchy:./out/manual/plotOut/massFractionSummary.pdf
./out/manual/wildtype_000000/plotOut/massFractionSummary.pdf
./out/manual/wildtype_000000/000000/plotOut/massFractionSummary.pdf
./out/manual/wildtype_000000/000000/generation_000000/000000/plotOut/massFractionSummary.pdfEach of thoes four levels of
plotOut is generated by a different one of the analysis scripts:./out/manual/plotOut: generated bypython runscripts/manual/analysisVariant.py. Contains comparisons of different variant conditions. We confirm this by looking at the results of run variants../out/manual/wildtype_000000/plotOut: generated bypython runscripts/manual/analysisCohort.py --variant_index 0. TODO not sure how to differentiate between two different labels e.g.wildtype_000000andsomethingElse_000000. If-vis not given, a it just picks the first one alphabetically. TODO not sure how to automatically generate all of those plots without inspecting the directories../out/manual/wildtype_000000/000000/plotOut: generated bypython runscripts/manual/analysisMultigen.py --variant_index 0 --seed 0./out/manual/wildtype_000000/000000/generation_000000/000000/plotOut: generated bypython runscripts/manual/analysisSingle.py --variant_index 0 --seed 0 --generation 0 --daughter 0. Contains information about a single specific cell.
Spin like mad between:
- standards
- high level generators. We use the assembler
asand linkerld. - hexdumps
- file decompilers. We use
readelf. It makes it faster to read the ELF file by turning it into human readable output. But you must have seen one byte-by-byte example first, and think howreadelfoutput maps to the standard. - low-level generators: stand-alone libraries that let you control every field of the ELF files you generated. github.com/BR903/ELFkickers, github.com/sqall01/ZwoELF and many more on GitHub.
- consumer: the
execsystem call of the Linux kernel can parse ELF files to starts processes: github.com/torvalds/linux/blob/v4.11/fs/binfmt_elf.c, stackoverflow.com/questions/8352535/how-does-kernel-get-an-executable-binary-file-running-under-linux/31394861#31394861
This was the first full scale nuclear reactor in the world, and was brought up slowly to test it out.
Hanford B Reactor tour by Studio McGraw
. Source. 2016.- youtu.be/8rlVHEY7BF0?t=335 good description of the fuel element. It uses uranium metal, not Uranium dioxide
- youtu.be/8rlVHEY7BF0?t=652 N Reactor and F Reactor were identical, and came up 2 months later, but much faster because of what they learned on the B
Quantum entanglement is often called spooky/surprising/unintuitive, but they key question is to understand why.
To understand that, you have to understand why it is fundamentally impossible for the entangled particle pair be in a predefined state according to experiments done e.g. where one is deterministically yes and the other deterministically down.
In other words, why local hidden-variable theory is not valid.
How to generate entangled particles:
- particle decay, notably pair production
- for photons, notably: spontaneous parametric down-conversion, e.g.: www.youtube.com/watch?v=tn1sEaw1K2k "Shanni Prutchi Construction of an Entangled Photon Source" by HACKADAY (2015). Estimatd price: 5000 USD.
Bell's Theorem: The Quantum Venn Diagram Paradox by minutephysics (2017)
Source. Contains the clearest Bell test experiment description seen so far.
It clearly describes the photon-based 22.5, 45 degree/85%/15% probability photon polarization experiment and its result conceptually.
It does not mention spontaneous parametric down-conversion but that's what they likely hint at.
Done in Collaboration with 3Blue1Brown.
Question asking further clarification on why the 100/85/50 thing is surprising: physics.stackexchange.com/questions/357039/why-is-the-quantum-venn-diagram-paradox-considered-a-paradox/597982#597982
Bell's Inequality I by ViaScience (2014)
Source. Quantum Entanglement & Spooky Action at a Distance by Veritasium (2015)
Source. Gives a clear explanation of a thought Bell test experiments with electron spin of electron pairs from photon decay with three 120-degree separated slits. The downside is that he does not clearly describe an experimental setup, it is quite generic.Quantum Mechanics: Animation explaining quantum physics by Physics Videos by Eugene Khutoryansky (2013)
Source. Usual Eugene, good animations, and not too precise explanations :-) youtu.be/iVpXrbZ4bnU?t=922 describes a conceptual spin entangled electron-positron pair production Stern-Gerlach experiment as a Bell test experiments. The 85% is mentioned, but not explained at all.Quantum Entanglement: Spooky Action at a Distance by Don Lincoln (2020)
Source. This only has two merits compared to Video 3. "Quantum Entanglement & Spooky Action at a Distance by Veritasium (2015)": it mentions the Aspect et al. (1982) Bell test experiment, and it shows the continuous curve similar to en.wikipedia.org/wiki/File:Bell.svg. But it just does not clearly explain the bell test.
Quantum Entanglement Lab by Scientific American (2013)
Source. The hosts interview Professor Enrique Galvez of Colgate University who shows briefly the optical table setup without great details, and then moves to a whiteboard explanation. Treats the audience as stupid, doesn't say the keywords spontaneous parametric down-conversion and Bell's theorem which they clearly allude to. You can even them showing a two second footage of the professor explaining the rotation experiments and the data for it, but that's all you get.Follows the "certified teacher only" approach which is in Ciro Santilli's opinion a fatal flaw of most elearning systems out there, OurBigBook.com won't suffer from that!
But that is a very, very good project.
All notes appear to have been extracted from existing notes, as noted on the bottom of each page.
Appears to have mixed licenses. E.g.:
- phys.libretexts.org/Bookshelves/University_Physics/Book%3A_University_Physics_(OpenStax)/Book%3A_University_Physics_III_-_Optics_and_Modern_Physics_(OpenStax)/06%3A_Photons_and_Matter_Waves/6.06%3A_De_Broglies_Matter_Waves is CC BY
- but we had seen another one that was CC BY-NC-SA
- phys.libretexts.org/Courses/University_of_California_Davis/UCD%3A_Physics_9HE_-_Modern_Physics/06%3A_Emission_and_Absorption_of_Photons/6.1%3A_Transitions_Between_Stationary_States CC BY-SA
- chem.libretexts.org/Bookshelves/Introductory_Chemistry/Introductory_Chemistry_(CK-12) uses the custom "CK-12 license" which seems a bit like CC BY-NC-SA
- some don't even have a free license, e.g.: phys.libretexts.org/Bookshelves/Quantum_Mechanics/Quantum_Mechanics_(Fowler)/00%3A_Front_Matter/04%3A_Licensing
TODO how does it work exactly? Do they ask for permission from authors in every case, including when the content has open license? Or when it has open license, do they just do it? In some cases, the notes have no license, so they must have asked.
TODO what is the source code that authors write? LaTeX or something else? LaTeX feels extremely likely given that it is what most original materials were already written in.
They are attempting a "model up this entire university" thing: phys.libretexts.org/Courses which is good. E.g. they have a bunch of "quantum mechanics ones under: phys.libretexts.org/Bookshelves/Quantum_Mechanics
Appears to be UC Davies-based mostly.
They claim to use this closed source backend: www.nice.com/resources/cxone-expert-knowledge-management? Seriously? For a publicly funded project with low-tech requirements?? It is mind blowing.
Some issues:
- the internal cross references are somewhat broken as of 2022.
- their URLs are HUGE! All components of every ancestor are in it. E.g. check this out: phys.libretexts.org/Bookshelves/Quantum_Mechanics/Introductory_Quantum_Mechanics_(Fitzpatrick)/12%3A_Time-Dependent_Perturbation_Theory/12.13%3A_Forbidden_Transitions Insane.
OK let's database it:
The general result from eigendecomposition of a matrix:becomes:where is an orthogonal matrix, and therefore has .
Let' see if there's anything in records/mx.xz.
mx.csv is 21GB.
They do have
" in the files to escape commas so:mx.pyWould have been better with csvkit: stackoverflow.com/questions/36287982/bash-parse-csv-with-quotes-commas-and-newlines
import csv
import sys
writer = csv.writer(sys.stdout)
with open('mx.csv', 'r') as f:
reader = csv.reader(f)
for row in reader:
writer.writerow([row[0], row[3]])then:
# uniq not amazing as there are often two or three slightly different records repeated on multiple timestamps, but down to 11 GB
python3 mx.py | uniq > mx-uniq.csv
sqlite3 mx.sqlite 'create table t(d text, m text)'
# 13 GB
time sqlite3 mx.sqlite ".import --csv --skip 1 'mx-uniq.csv' t"
# 41 GB
time sqlite3 mx.sqlite 'create index td on t(d)'
time sqlite3 mx.sqlite 'create index tm on t(m)'
time sqlite3 mx.sqlite 'create index tdm on t(d, m)'
# Remove dupes.
# Rows: 150m
time sqlite3 mx.sqlite <<EOF
delete from t
where rowid not in (
select min(rowid)
from t
group by d, m
)
EOF
# 15 GB
time sqlite3 mx.sqlite vacuumLet's see what the hits use:
awk -F, 'NR>1{ print $2 }' ../media/cia-2010-covert-communication-websites/hits.csv | xargs -I{} sqlite3 mx.sqlite "select distinct * from t where d = '{}'"At around 267 total hits, only 84 have MX records, and from those that do, almost all of them have exactly:with only three exceptions:We need to count out of the totals!which gives, ~18M, so nope, it is too much by itself...
smtp.secureserver.net
mailstore1.secureserver.netdailynewsandsports.com|dailynewsandsports.com
inews-today.com|mail.inews-today.com
just-kidding-news.com|just-kidding-news.comsqlite3 mx.sqlite "select count(*) from t where m = 'mailstore1.secureserver.net'"Let's try to use that to reduce where
av.sqlite from 2013 DNS Census virtual host cleanup a bit further:time sqlite3 mx.sqlite '.mode csv' "attach 'aiddcu.sqlite' as 'av'" '.load ./ip' "select ipi2s(av.t.i), av.t.d from av.t inner join t as mx on av.t.d = mx.d and mx.m = 'mailstore1.secureserver.net' order by av.t.i asc" > avm.csvavm stands for av with mx pruning. This leaves us with only ~500k entries left. With one more figerprint we could do a Wayback Machine CDX scanning scan.Let's check that we still have most our hits in there:At 267 hits we got 81, so all are still present.
grep -f <(awk -F, 'NR>1{print $2}' /home/ciro/bak/git/media/cia-2010-covert-communication-websites/hits.csv) avm.csvsecureserver is a hosting provider, we can see their blank page e.g. at: web.archive.org/web/20110128152204/http://emmano.com/. security.stackexchange.com/questions/12610/why-did-secureserver-net-godaddy-access-my-gmail-account/12616#12616 comments:
secureserver.net is the name GoDaddy use as the reverse DNS for IP addresses used for dedicated/virtual server hosting
Some amazing people have put book source codes on GitHub. This is a list of such repos.
Mentioned at: aws.amazon.com/ec2/instance-types/g4/
TODO meaning of "nd"? "n" presumably means Nvidia, but what is the "d"? Compare it g4ad.xlarge which has AMD GPUs. aws.amazon.com/ec2/instance-types/g4/ mentions:
G4 instances are available with a choice of NVIDIA GPUs (G4dn) or AMD GPUs (G4ad).
Price:
- 2025-03-10: 0.526 USD / Hour
Unlisted articles are being shown, click here to show only listed articles.