Funding:
- 2023: 1.1m pounds www.uktech.news/deep-tech/lumai-grant-20230215
2009 Nobel Prize lecture
. Poor Charles was too debilitated by Alzheimer's disease to give the talk himself! But if you've got a pulse, you can get the prize, so all good.An example where
SELECT FOR UPDATE is a good solution to an use case can be seen at: nodejs/sequelize/raw/parallel_select_and_update.js.SELECT FOR UPDATE vs/together with the SQL transaction isolation level is commented at: stackoverflow.com/questions/10935850/when-to-use-select-for-update.- foreign keys are capitalized:
- you must give
foreignKeywhen using aliases, otherwise it fails subtely. That would be derived automatically. - stackoverflow.com/questions/41502699/return-flat-object-from-sequelize-with-association can't auto-flatten to reuse the database's
ORDER limitandoffsetdon't work withoutsubQuery: falsewhen doing includes! It is just too buggy. Examples of this can be found e.g. under nodejs/sequelize/many_to_many_same_model.js.- stackoverflow.com/questions/34059081/how-do-i-reference-an-association-when-creating-a-row-in-sequelize-without-assum hard to not duplicate foreign keys values everywhere
- stack traces permanently broken or requiring non-obvious configs:
- does not automatically update fields on hooks: github.com/sequelize/sequelize/issues/8586#issuecomment-422877555
- cannot change columns when other columns have constraints due to the backup table?
- you have to use
.get()forattributealiased fields, why? stackoverflow.com/questions/32649218/how-do-i-select-a-column-using-an-alias/69890944#69890944 .idgets added toSELECTno matter what, breakingGROUP BYunless you do horrible workarounds:- no simple built-in mechanism for transaction retries: Sequelize transaction retries
- impossible to do subqueries in general. Docs just tell you to use literals. This in particular prevents single query deletes with join as done at nodejs/sequelize/raw/many_to_many.js:Also, you can't get query strings either: github.com/sequelize/sequelize/issues/2325
- sequelize.org/master/manual/sub-queries.html: the docs actually just tell you to use literals, lol
- stackoverflow.com/questions/45354001/nodejs-sequelize-delete-with-nested-select-query
- migrations. Generally speaking, anything but the simplest migrations are exceedingly hard to get right, as you have to go very low level when doing migrations. Syntax can be very different from regular DB operations.
- no way to do (non-raw) queries during migrations, e.g. to update fields based on other fields in a complex way?
- github.com/sequelize/cli/issues/862
- stackoverflow.com/questions/18742962/add-data-in-sequelize-migration-script
- stackoverflow.com/questions/38671483/sequelize-migration-update-model-after-updating-column-attributes
- stackoverflow.com/questions/38998397/can-i-use-sequelize-models-in-migration-scripts
- stackoverflow.com/questions/45286429/custom-query-on-sequelize-seeder`queryInterface.sequelize.models` contains only
SequelizeMeta. Not sure why they have this limitation.
Edit: actually things will likely just work if immediately after making table changes you just instantiate a new sequelize and do any data changes. - stackoverflow.com/questions/56043246/node-js-sequelize-no-primary-keys-when-migrating/56046101#56046101
- SQLite
changeColumnmigrations do on delete cascades of other tables. SQLite does not have change column statements, so they have to drop and recreate tables, but they don't temporarily remove cascades, so you lose data: stackoverflow.com/questions/62667269/sequelize-js-how-do-we-change-column-type-in-migration/70486686#70486686 - associations require full explicit index construction: stackoverflow.com/questions/39651853/how-to-create-join-table-with-foreign-keys-with-sequelize-or-sequelize-cli
- ability to iterate over a large result without blowing up memory and without using limit + offset (which is inneficient e.g. when looping over recursive queries). This is also known as cursor or streaming interfaces:E.g. the Python SQLite interface supports this just fine: stackoverflow.com/questions/29582736/python3-is-there-a-way-to-iterate-row-by-row-over-a-very-large-sqlite-table-wi
- stack overflow
- stackoverflow.com/questions/28787889/how-can-i-set-up-sequelize-js-to-stream-data-instead-of-a-promise-callback
- stackoverflow.com/questions/43964067/how-to-implement-cursor-pagination-using-sequelize
- stackoverflow.com/questions/57164242/perform-sequelize-findall-in-a-huge-array
- stackoverflow.com/questions/55191891/how-to-loop-through-result-in-sequelize generic loop
- issue tracker
- stack overflow
- empty
attributes: []breaks some nested queries: github.com/sequelize/sequelize/issues/16436 - does not expose a iteration API that supports large arrays?E.g. Python SQLite does: stackoverflow.com/questions/29582736/python3-is-there-a-way-to-iterate-row-by-row-over-a-very-large-sqlite-table-wi
Our WIP script: wikipedia/import-categories.sh.
Related:
- opendata.stackexchange.com/questions/1533/download-wikipedia-articles-from-a-specific-category
- webapps.stackexchange.com/questions/16359/is-there-a-way-to-download-a-list-of-all-wikipedia-categories/172480#172480
- stackoverflow.com/questions/40119322/how-to-download-all-pages-inside-a-category-in-wikipedia
- category tree on Stack Overflow
- stackoverflow.com/questions/17432254/wikipedia-category-hierarchy-from-dumps/77313490#77313490 Canon but no good answers.
- stackoverflow.com/questions/12227134/how-to-fetch-category-tree-of-wiki
- stackoverflow.com/questions/21782410/finding-subcategories-of-a-wikipedia-category-using-category-and-categorylinks-t. Actually explains it: stackoverflow.com/questions/21782410/finding-subcategories-of-a-wikipedia-category-using-category-and-categorylinks-t/21798259#21798259
- stackoverflow.com/questions/27279649/how-to-build-wikipedia-category-hierarchy
- mdkzaman.com/knowledge-graph-from-wikipedia-category-hierarchy/
Consider:
Jewish_physicists
Let's observe them in MySQL:outputs:
mysql enwiki -e "select page_id, page_namespace, page_title, page_is_redirect from page where page_namespace in (0, 14) and page_title in ('Computer_storage_devices', 'Computer_data_storage')"+----------+----------------+--------------------------+------------------+
| page_id | page_namespace | page_title | page_is_redirect |
+----------+----------------+--------------------------+------------------+
| 5300 | 0 | Computer_data_storage | 0 |
| 42371130 | 0 | Computer_storage_devices | 1 |
| 711721 | 14 | Computer_data_storage | 0 |
| 895945 | 14 | Computer_storage_devices | 0 |
+----------+----------------+--------------------------+------------------+mysql enwiki -e "select cl_from, cl_to from categorylinks where cl_from in (5300, 711721, 895945, 42371130)"+----------+-----------------------------------------------------------------------+
| cl_from | cl_to |
+----------+-----------------------------------------------------------------------+
| 5300 | All_articles_containing_potentially_dated_statements |
| 5300 | Articles_containing_potentially_dated_statements_from_2009 |
| 5300 | Articles_containing_potentially_dated_statements_from_2011 |
| 5300 | Articles_with_GND_identifiers |
| 5300 | Articles_with_NKC_identifiers |
| 5300 | Articles_with_short_description |
| 5300 | Computer_architecture |
| 5300 | Computer_data_storage |
| 5300 | Short_description_matches_Wikidata |
| 5300 | Use_dmy_dates_from_June_2020 |
| 5300 | Wikipedia_articles_incorporating_text_from_the_Federal_Standard_1037C |
| 711721 | Computer_architecture |
| 711721 | Computer_data |
| 711721 | Computer_hardware_by_type |
| 711721 | Data_storage |
| 895945 | Computer_data_storage |
| 895945 | Computer_peripherals |
| 895945 | Recording_devices |
| 42371130 | Redirects_from_alternative_names |
+----------+-----------------------------------------------------------------------+So we see that
cl_from encodes the parent categories:- parent categories of categories:
- en.wikipedia.org/wiki/Category:Computer_data_storage, which has ID
711721, has parent categories: "Computer hardware by type", "Computer data", "Data storage", "Computer architecture". This matches exactly on the database. These are all encoded on the source code of the page:{{DEFAULTSORT:Storage}} [[Category:Computer hardware by type]] [[Category:Computer data|Storage]] [[Category:Data storage|Computer]] [[Category:Computer architecture]] - en.wikipedia.org/wiki/Category:Computer_storage_devices has parent categories: "Computer data storage", "Recording devices", "Computer peripherals". This matches exactly on the database.
- en.wikipedia.org/wiki/Category:Computer_data_storage, which has ID
- parent categories of pages:
- en.wikipedia.org/wiki/Computer_storage_devices whish is a redirect gets the magic category "Redirects_from_alternative_names", a humongous placeholder with many thousands of pages: en.wikipedia.org/wiki/Category:Redirects_from_alternative_names
- en.wikipedia.org/wiki/Computer_data_storage shows only two categories onthe web UI: "Computer data storage" and "Computer architecture". Both of these are present on the database and at the end of the source code:The others appear to be more magic. Two of them we can guess from the templates:
{{DEFAULTSORT:Computer Data Storage}} [[Category:Computer data storage| ]] [[Category:Computer architecture]]are likely{{short description|Storage of digital data readable by computers}} {{Use dmy dates|date=June 2020}}Use_dmy_dates_from_June_2020andArticles_with_short_descriptionbut the rest is more magic and not necessarily present in-source.
So to find all articls and categories under a given category title, say en.wikipedia.org/wiki/Category:Mathematics we can run:
mariadb enwiki -e "select cl_from, cl_to, page_namespace, page_title from categorylinks inner join page on page_namespace in (0, 14) and cl_from = page_id and cl_to = 'Mathematics'"- against all odds, the experiment worked and we got DNA out of the water, despite a bunch of non-bio newbs actively messing with random parts of the experiment
- PuntSeq and Biomakespace people, and all those tho do scientific outreach, are awesome!
- biology is hard
- creating insanely media rich articles like this is also hard, but the following helped enormously:
- Wikimedia Commons to store large media files out of Git
- Asciidoctor extensions to easily include those media files. The lessons learnt in this article were then an important motivation for Ciro's OurBigBook Markup, to which this article was later migrated.
- Nomacs to give Google Photos photos meaningful names and to edit people's faces out of pictures ;-)
- some scientific Wikipedia pages may or may not have been edited with better pictures during the course of writing this article
Ciro Santilli had to see this in a few separate places, until he underestood: that little pictur emust be a thing! Examples:
- mojim watermarks: mojim.com/twy105509x7x2.htm
- some Japanese website: kotobank.jp/word/%E5%A4%A7%E7%96%91-556655
Really weird and obscure company, good coverage: thequantuminsider.com/2020/02/06/quantum-computing-incorporated-the-first-publicly-traded-quantum-computing-stock/
Publicly traded in 2007, but only pivoted to quantum computing much later.
Protocols are the biologist term for "recipe".
I found that a lot of biology comes down to this: get the right recipe, follow it well even though you don't understand all the proprietary details, and pray.
Start by looking at: Maxwell-Boltzmann vs Bose-Einstein vs Fermi-Dirac statistics.
Revolutionary for its time, and a big part of Ciro's Enlightenment.
But too insane, and did not keep up with internet age, and so Ciro wants to kill it now.
Unknown real developer name, claims to be from Canada on YouTube channel about: www.youtube.com/@TheBibitesDigitalLife/about, likely because he's a software developer and wants to keep his employer's claws away from his side project.
Appears to be closed source unfortunately, so not suitable for research.
Video 1. "What will happen after 100h of evolution? by The Bibites (2022)" mentions it was started five years ago, so circa 2017.
Appears to be Unity-based, if you download and extract for Linux you get files named
UnityPlayer.so.Author is named Leo Caussan in game credits at startup: www.linkedin.com/in/l%C3%A9o-caussan-560350136/, a Canadian software engineer.
Was not very Linux compatible: www.reddit.com/r/TheBibites/comments/vqk6ac/program_stalls_at_a_blue_screen/ Trying to run 0.5.0 leads to a blank screen after you click "start simulation".
What will happen after 100h of evolution? by The Bibites (2022)
Source. The summary from www.geeksforgeeks.org/tree-traversals-inorder-preorder-and-postorder/ is a winner:
1
/ \
2 3
/ \
4 5- inorder DFS: 4 2 5 1 3
- preorder DFS: 1 2 4 5 3
- postorder DFS: 4 5 2 3 1
- breadth-first search: 1 2 3 4 5
In principle one could talk about tree traversal of unordered trees as a number of possible traversals without a fixed order. But we won't consider that under this section, only deterministic ordered tree traversals.
Unlisted articles are being shown, click here to show only listed articles.