Ciro's Edict #5 Link to an image or video in another file that has an
\x on title from another file Updated 2025-05-23 +Created 1970-01-01Issue report at: github.com/ourbigbook/ourbigbook/issues/198 Suppose you had:
programming-language.ciro
= Programming language
\Image[https://raw.githubusercontent.com/cirosantilli/media/master/python-logo.jpg]
{title=The \x[python-programming-language] logo}
== Python
{c}
{disambiguate=programming-language}= Logos I like
\x[image-the-python-logo]Now, when rendering
\x[image-the-python-logo], we would need to fetch two IDs:image-the-python-logofor theTheandlogopartpython-programming-languageitself, to know that\x[python-programming-language]should render asPython
But after group all SQL queries together was done, there was no way to know that rendering
image-the-python-logo would imply also fetching python-programming-language.This was solved by adding a new database entry type,
REFS_TABLE_X_TITLE_TITLE to the existing References table, which tracks dependencies between IDs.- upload all of cirosantilli.com to ourbigbook.com. I will do this by implementing an import from filesystem functionality based on the OurBigBook CLI. This will also require implementing slit headeres on the server to work well, I'll need to create one
Articlefor every header on render. - get
\xand\Includeworking on the live web preview editor. This will require creating a new simple API, currently the editor jus shows broken references, but final render works because it goes through the database backend - implement email verification signup. Finally! Maybe add some notifications too, e.g. on new comments or likes.
Ciro's Edict #5 Skip ID extraction and rendering based on database timestamps Updated 2025-05-23 +Created 1970-01-01
This means while developing a website locally with the OurBigBook CLI, if you have a bunch of files with an error in one of them, your first run will run slowly until the error:but further runs will blast through the files that worked, skipping all files that have sucessfully converted:so you can fix file by file and move on quickly.
extract_ids README.ciro
extract_ids README.ciro finished in 73.82836899906397 ms
extract_ids art.ciro
extract_ids art.ciro finished in 671.1738419979811 ms
extract_ids ciro-santilli.ciro
extract_ids ciro-santilli.ciro finished in 1009.6256089992821 ms
extract_ids science.ciro
error: science.ciro:13686:1: named argument "parent" given multiple times
extract_ids science.ciro finished in 1649.6193730011582 msextract_ids README.ciro
extract_ids README.ciro skipped by timestamp
extract_ids art.ciro
extract_ids art.ciro skipped by timestamp
extract_ids ciro-santilli.ciro
extract_ids ciro-santilli.ciro skipped by timestamp
extract_ids science.ciroMore details at: cirosantilli.com/ourbigbook#no-render-timestamp
This was not fully trivial to implement because we had to rework how duplicate IDs are checked. Previously, we just nuked the DB every time on a directory conversion, and then repopulated everything. If a duplicated showed up on a file, it was a duplicate.
But now that we are not necessarily extracing IDs from every file, we can't just nuke the database anymore, otherwise we'd lose the information. Therefore, what we have to do is to convert every file, and only at the end check the duplicates.
Managed to do that with a single query as documented at: stackoverflow.com/questions/71235548/how-to-find-all-rows-that-have-certain-columns-duplicated-in-sequelize/71235550#71235550
Added
ourbigbook --format-source automatic code formatting. I implemented it for the following reasons:- I want to do certain automatic modifications to source code on web, e.g.:
- later on, much later, this will allow WYSIWYG export to plaintext
This also ended up having one unexpected benefit: whenever a new feature is added that deprecates an old feature, by converting the large corpus from github.com/cirosantilli/cirosantilli.github.io to the new feature I can test the new preferred feature very well.
For example, converting
\x[blue cat] en masse to the new insane syntax <blue cat> found several bugs with the new insane syntax.This seemed somewhat easy at first, so I started it as a way of procrastinating more urgent Web features (web scares me, you know), but it ended being insanely hard to implement, because there are many edge cases. Also, most bugs are not acceptable, as they would corrupt your precious source code and potentially output.
But well, it is done!
To run examples on a specific database:All examples can be tested on all databases with:
./index.jsor./index.js l: SQLite./index.js p: PostgreSQL. You must manually create a database calledtmpand ensure that peer authentication works for it
cd sequelize
./testOverview of the examples:
- nodejs/sequelize/index.js: a bunch of basic examples
- nodejs/sequelize/update.js: This file is also where we are storing our expression-foo for now, e.g. how to do stuff like
col1 + col2. Such knowledge can however be used basically anywhere else however, e.g. inASorWHEREclauses, not just inUPDATE. - nodejs/sequelize/count.js: a simplified single-table count example. In practice, this will be usually done together with JOIN queries across multiple tables. Answers: stackoverflow.com/questions/22627258/how-does-group-by-works-in-sequelize/69896449#69896449
- nodejs/sequelize/date.js: automatic date typecasts
- nodejs/sequelize/like.js: LIKE
- nodejs/sequelize/camel_case.js: trying to get everything in the database camel cased, columns starting with lowercase, and tables starting with uppercase. The defaults documented on getting started documentation do uppercase foreign keys, and lowercase non-foreign keys. It's a mess.
- nodejs/sequelize/ignore_duplicates.js: ignore query on unique violation with
ignoreDuplicates: truewhich does SQLiteINSERT OR IGNORE INTOor PostgreSQLON CONFLICT DO NOTHING. Closely related Upsert versions:- Upsert
- nodejs/sequelize/upsert.js:
.upsertselects the conflict column automatically by unique columns. Both SQLite and PostgreSQL doINSERT INTO ON CONFLICT. PostgreSQL usesRETURNING, which was added too recently to SQLite: www.sqlite.org/lang_returning.html - nodejs/sequelize/update_on_duplicate.js:
.bulkCreate({}, { updateOnDuplicate: ['col1', col2'] }. Produces queries analogous to.upsert. This method is cool because it can upsert multiple columns at once. But it is annoying that you have to specify all fields to be updated one by one manually. - stackoverflow.com/questions/29063232/how-to-get-the-id-of-an-inserted-or-updated-record-in-sequelize-upsert/72092277#72092277
- stackoverflow.com/questions/55531860/sequelize-bulkcreate-updateonduplicate-for-postgresql
- Upsert
- nodejs/sequelize/inc.js: demonstrate the
incrementmethod. In SQLite, it produces a statement of type:UPDATE `IntegerNames` SET `value`=`value`+ 1,`updatedAt`='2021-11-03 10:23:45.409 +00:00' WHERE `id` = 3 - nodejs/sequelize/sync_alter.js: illustrates
Model.sync({alter: true})to modify a table definition, answers: stackoverflow.com/questions/54898994/bulkupdate-in-sequelize-orm/69044138#69044138 - nodejs/sequelize/truncate_key.js
- nodejs/sequelize/validation.js: is handled by a third-party library: github.com/validatorjs/validator.js. They then add a few extra validators on top of that.The
args: truething is explained at: stackoverflow.com/questions/58522387/unhandled-rejection-sequelizevalidationerror-validation-error-cannot-create-pr/70263032#70263032 - nodejs/sequelize/composite_index.js: stackoverflow.com/questions/34664853/sequelize-composite-unique-constraint
- nodejs/sequelize/indent_log.js: stackoverflow.com/questions/34664853/sequelize-composite-unique-constraint
- association examples:
- nodejs/sequelize/one_to_many.js: basic one-to-many examples.
- nodejs/sequelize/many_to_many.js: basic many-to-many examples, each user can like multiple posts. Answers: stackoverflow.com/questions/22958683/how-to-implement-many-to-many-association-in-sequelize/67973948#67973948
- ORDER BY include:
- nodejs/sequelize/many_to_many_custom_table.js: many-to-many example, but where we craft our own table which can hold extra data. In our case, users can like posts, but likes have a integer weight associated with them. Related threads:
- nodejs/sequelize/many_to_many_same_model.js: association between a model and itself: users can follow other users. Related:
- nodejs/sequelize/many_to_many_same_model_super.js
- nodejs/sequelize/many_to_many_super.js: "Super many to many": sequelize.org/master/manual/advanced-many-to-many.html This should not exist and shows how bad this library is for associations, you need all that boilerplate in order to expose certain relationships that aren't otherwise exposed by a direct
hasManywith implicit join table.
- nested includes to produce queries with multiple JOIN:
- nodejs/sequelize/nested_include.js: find all posts by users that a given user follows. Answers: stackoverflow.com/questions/42632943/sequelize-multiple-where-clause/68018083#68018083
- nodejs/sequelize/nested_include_super.js: like nodejs/sequelize/nested_include.js but with a super many to many. We should move this to nodejs/sequelize/many_to_many_super.js.
- two relationships between two specific tables: we need to use
as:to disambiguate them- nodejs/sequelize/many_to_many_double.js: users can both follow and like posts
- nodejs/sequelize/one_to_many_double.js: posts have the author and a mandatory reviewer
- hooks
- internals:
- nodejs/sequelize/common.js: common utilities used across examples, most notably:
- to easily setup different DBRM
- nodejs/sequelize/min_nocommon.js: to copy paste to Stack Overflow
- nodejs/sequelize/min.js: template for new exapmles in the folder
- nodejs/sequelize/common.js: common utilities used across examples, most notably:
Ciro Santilli is extremely passionate about this issue, partly due to Ciro Santilli's self perceived compassionate personality.
One of Ciro's most direct experiences with social inequality is described at: São Remo, the favela next to USP.
We have to be careful not to make everyone poorer when trying to reduce inequality.
But as things stand as of 2020, increasing taxes on the very richest, and notably wealth tax, and investing it in free gifted education, seems like a safe bet to achieve any meaningful level of equal opportunity and meritocracy.
The new default homepage for a logged out user how shows a list of the topics with the most articles.
This is a reasonable choice for default homepage, and it immediately exposes users to this central feature of the website: the topic system.
Doing this required in particular calculating the best title for a topic, since it is possible to have different titles with the same ID, the most common way being with capitalization changes, e.g.:would both have topic ID
JavaScript
Javascriptjavascript.The algorithm chosen is to pick the top 10 most upvoted topics, and select the most common title from amongst them. This should make topic title vandalism quite hard. This was made in a single SQL query, and became the most complext SQL query Ciro Santilli has ever written so far: twitter.com/cirosantilli2/status/1549721815832043522
Screenshot showing the list of topics
. The page is: ourbigbook.com for the logged out user, ourbigbook.com/go/topics for the logged in user.
Screenshot showing a topic page
. The page is: ourbigbook.com/go/topic/vector-space. Before this sprint, we didn't have the "Vector Space" at the top, as it wasn't necessarily trivial to determine what the preferred title would be.Five votes:
All but president are per state. Official list seems to be e.g. for Sao Paulo: divulgacandcontas.tse.jus.br/divulga/#/estados/2022/2040602022/SP/candidatos
Every invertible matrix can be written as:where:Note therefore that this decomposition is unique up to swapping the order of eigenvectors. We could fix a canonical form by sorting eigenvectors from smallest to largest in the case of a real number.
- is a diagonal matrix containing the eigenvalues of
- columns of are eigenvectors of
Intuitively, Note that this is just the change of basis formula, and so:
- changes basis to align to the eigenvectors
- multiplies eigenvectors simply by eigenvalues
- changes back to the original basis
Every article now has a (very basic) GitHub-like issue tracker. Comments now go under issues, and issues go under articles. Issues themselves are very similar to articles, with a title and a body.
This was part of 1.0, but not the first priority, but I did it now anyways because I'm trying to do all the database changes ASAP as I'm not in the mood to write database migrations.
Here's an example:
- ourbigbook.com/go/issue/2/donald-trump/atomic-orbital a specific issue about the article "Atomic Orbital" by Donald Trump. Note the comments possibly by other users at the bottom.
- ourbigbook.com/go/issues/1/donald-trump/atomic-orbital list of issues about the article "Atomic Orbital" by Donald Trump
You can then detect one of the photons, and when you do you know that the other one is there as well and ready to be used. two photon interference experiment comes to mind, which is the basis of photonic quantum computer, where you need two photons to be produced at the exact same time to produce quantum entanglement.
Features Alan Migdall of the National Institute of Standards and Technology. Produced by the Joint Quantum Institute (JQI).
Mentions that this phenomena is useful to determine the efficiency of a single photon detector, as you have the second photon of the pair as a control.
Also briefly describes how the input energy and momentum must balance out the output energy and momentum of the two photons coming out (determined by the output frequency and angle).
Shows the crystal close up of the crystal branded "Cleveland Crystals Inc.". Mentions that only one in a billion photon gets scattered.
Also shows a photomultiplier tube.
Then shows their actual optical table setup, with two tunnels of adjustable angle to get photons with different properties.
Very short whiteboard video by Peter Mosley from the University of Bath, but it's worth it for newbs. Basically describes spontaneous parametric down-conversion.
One interesting thing he mentions is that you could get single photons by making your sunglasses thicker and thicker to reduce how many photons pass, but one big downside problem is that then you don't know when the photon is going to come through, that becomes essentially random, and then you can't use this technique if you need two photons at the same time, which is often the case, see also: two photon interference experiment.
Unlisted articles are being shown, click here to show only listed articles.