Ciro Santilli publishes videos of this not-so-common visual programming experiments on his YouTube channel occasionally: www.youtube.com/c/CiroSantilli. Ciro should however not be lazy and also upload each video produced to Wikimedia Commons, since YouTube does not offer a download option even for videos marked with a Creative Commons license: www.quora.com/Can-I-download-Creative-Commons-licensed-YouTube-videos-to-edit-them-and-use-them/answer/Tarmo-Toikkanen!
This is also where Ciro's downtime converged to in his early 30's, since he long lost patience for stupid video games and television series.
Ciro developed one interesting technique: while scrolling through YouTube's useless recommendations, when he understands what a channel is about, he either immediately:This helps to keep this feed clean of boring stuff he already knows about. There is unfortunately an infinite amount of useless videos out there however on the topics of:and no matter how much you say you don't want to hear about them, YouTube juts keeps on sending more.
- subscribes if it is amazing and then "Don't recommend channel"
- otherwise just "Don't recommend channel" immediately
- sports
- music, mostly idiotic top of the charts
- news and political commentary
- food
- programming tutorials. Meh, got Stack Overflow.
- stuff that is not in English, and notably languages that Ciro does not even speak!
- motorcycles
- ASMR
- cute animals
- gaming and movie commentary. Ciro is interested only in a very specific number of video games
- nature life, e.g. hiking, cycling, or living in isolation, this Ciro enjoys
- science for kids (popular science)
Things Ciro hates about YouTube:
- you can't follow or ignore a subject, only indirectly tell the algorithm about that. Once you click a popular cat video, you will be forced to watch cat videos for all eternity.
Likely FFmpeg is the backend of YouTube.
Bought by Google in 2006.
YouTube: From Concept to Hypergrowth Jawed Karim (2006)
Source. YouTube co-founder explains that the key enabling technology for YouTube was the addition of video capabilities to Macromedia Flash 7.It is incomprehensible how it works.
If there's anything remotely offensive, that's understandable, that the Google advertising censorship machine would take it down. But even perfecly polite comments are taken down.
And even as a channel admin, I cannot find the comment anywhere to approve it. They seem to be actually fully removed.
Also YouTube notifies you of some comments, but randomly does not notify you of others. I've checked Held comments and "Comments that may be offensive have been hidden".
Everything is very blurred and you can't be certain of anything.
One thing YouTube seems to not like is if you comment multiple times. Even when replying to the channel admin each time. Presumably to prevent spammers from adding a lot of replies.
And once it removes one of your comments from a thread, it starts to remove every single future comment from tha thread, that seems to be deterministic.
One thing to try with known users that make good comments often is "Mark user as approved user". But that feature is very hard to find on the web UI and its usage must be minimal.
Also the "I haven't respoded" tab simply doesn't work, and leaves dozens of comments unresponded not shown.
YouTube not usable as a commenting platform. It's just a huge noisy mess. Perhaps this is a reflection of our Internet world in general.
- www.youtube.com/channel/UCDyR_C_QVjZR24ze0fl5S_Q Goat-on-a-Stick channel
Afraid of Technology by adarkenedroom (2008)
Source. TODO source show, appears "Brass Eye", TODO episode www.reddit.com/r/videos/comments/jpyfi/technology_scares_the_crap_out_of_me/This thing dowloads YouTube videos. The thing downloads Twitter videos. The thing downloads BBC videos. It is just Godlike.
- www.youtube.com/channel/UCM2YmsRUeIbRkqjgNm0eTGQ Journeyman Pictures. Basically a VICE-like, focused on fucked up things happening in poor countries or regions.
- Mediocre Amateur
Ciro Santilli wants to rule this with OurBigBook.com.
This was the pre-Internet precursor of wikis. This program was likely venerable, shame it predates Ciro Santilli's era.
But the thing was much more bloated it seems, and also included visual programming elements, and WYSISYG UI creation.
No open signup it seems. TODO CV of owner.
They are making a proof assistant to integrate into the website: github.com/bookofproofs/fpl/, reminds Ciro Santilli of website front-end for a mathematical formal proof system.
Originally by Springer, but later moved to the European mathematical society.
Written mostly by Eric W. Weisstein.
Ciro once saw a printed version of the CRC "concise" encyclopedia of mathematics. It is about 12 cm thick. Imagine if it wasn't concise!!!
Infinite Napkin is the one-person open source replacemente we needed for it! And OurBigBook.com will be the final multi-person replacement.
Ahh, this dude is just like Ciro Santilli, trying to create the ultimate natural sciences encyclopedia!
____
In 1995, Weisstein converted a Microsoft Word document of over 200 pages to hypertext format and uploaded it to his webspace at Caltech under the title Eric's Treasure Trove of Sciences.
____
In 1995, Weisstein converted a Microsoft Word document of over 200 pages to hypertext format and uploaded it to his webspace at Caltech under the title Eric's Treasure Trove of Sciences.
____
Decent encyclopedia of mathematics. Not much motivation, mostly statements though.
Created by:
- John Baez
- David Corfield
- Urs Schreiber
Unlike Wikipedia, they have a more sane forum commenting system, e.g. a page/forum pair:
Based on GitHub pull requests: github.com/planetmath
Joe Corneli, of of the contributors, mentions this in a cool-sounding "Peeragogy" context at metameso.org/~joe/:
I earned my doctorate at The Open University in Milton Keynes, with a thesis focused on peer produced support for peer learning in the mathematics domain. The main case study was planetmath.org; the ideas also informed the development of “Peeragogy”.
A wiki that gathers mathematical proofs.
MediaWiki-based.
This appears to be the creator: github.com/externl "Joe George".
Cosmopedia is a dataset of synthetic textbooks, blogposts, stories, posts and WikiHow articles generated by Mixtral-8x7B-Instruct-v0.1.The dataset contains over 30 million files and 25 billion tokens, making it the largest open synthetic dataset to date.
App-only as of 2023, i.e. for children.
Humans make the table of contents, and then AI fills it. Ciro was thinking about doint the exact same thing at some point, maybe starting from Wikipedia categories.
Funding:
This section is about wikis that are hosted on a blockchain of some sort.
Appears to be a Wikipedia clone but with much lower/no notability requirements guidelines, which overcomes one of Wikipedia's main issues: deletionism.
They do have the interesting idea of importing deleted Wikipedia pages as a source of content, which leads to some epic "most viewed pages" such as en.everybodywiki.com/List_of_erotic_and_sex_workers_with_unnatural_death which currently reads:
Stop Being Pervs, Go Watch Lichfaop/Faoplich Instead and you can also visit MR Info 24 for more details.
We can for example see Ciro Santilli's deleted entry PsiQuantum at: en.everybodywiki.com/PsiQuantum, Wikipedia deletion page: en.wikipedia.org/wiki/Wikipedia:Articles_for_deletion/PsiQuantum. Their attribution is atrocious however, e.g. it does not seem possible to find any mention of "Ciro Santilli" on the edit history, which just points to the delete article which is not visible anymore. They could really get into trouble for this one day.
Their main use case, as suggested by the website itself, if for people/brands to create pages about themselves.
This combined with the lack of "one version of each page per person" seems like an explosive invitation for unsolvable edit wars.
The website is backed by a French startup: jobs.stationf.co/companies/wiki-valley.
Website: golden.com
April 2024: merged with some fraud protection thing, is it sill a Wiki? Unclear, seem sto have lost that aspect: twitter.com/judegomila/status/1783028847983956430
Social media:
techcrunch.com/2019/04/30/golden-launch/Exactly! Deletionism on Wikipedia is so sad, and especially for companies. In particular e.g. Ciro Santilli tried to create a page for PsiQuantum, and it got reverted... and now golden has one of the largest Google hits for it: golden.com/wiki/PsiQuantum-PBDGXRA
Quote 1.
Golden wiki vs Deletionism on Wikipedia
. To state the obvious: Wikipedia is an incredibly useful website, but Gomila pointed out that notable companies and technologies like SV Angel, Benchling, Lisk and Urbit don’t currently have entries. Part of the problem is what he called Wikipedia’s “arbitrary notability threshold,” where pages are deleted for not being notable enough. (This is also what happened years ago to the Wikipedia page about yours truly — which I swear I didn’t write myself.)
TODO how do they do moderation?
As of April 2024Asked at: twitter.com/cirosantilli/status/1777250258235302233 Their last tweets were from August 2023, so maybe they just silently shutdown? Their name is too generic and hard to search for efficiently...
Login is currently disabled.
They do have knowledge graph built-in which is cool.
WikiAlpha is an alternative to Wikipedia, where the main difference is that our deletion policy is far more lenient with regard to notability requirements. Basically, WikiAlpha is a near-indiscriminate collection of information in the form of articles on any topic: you can create an article about the band you just started, your pet dog, yourself, your house - as long as your content does not fall under our speedy deletion policy, it will likely remain on the site forever!
10k GitHub Stars by BookStack (2022)
Source. Answering to an AMA unfortunately :-) But some OK small bits of information trickled through.As of 2022 visible at: www.nature.com/scitable
Apparently they had a separate URL as just scitable.com, so they were somewhat serious about it before shutting it down.
As of 2022 marked:RIP.
This page has been archived and is no longer updated
www.nature.com/scitable/blog/student-voices/ has last entry 2015, so presumably that's the shutdown year.
Self description:so quite related to OurBigBook.com.
Using our platform, you can customize your own eBooks for your students. Create an online classroom. Contribute and share content and connect with networks of colleagues.
9-Year Hustle to Achieve a Single Goal by EO
. Source. Interview with Akshay Kothari and Ivan Zhao.- Ivan Zhao www.linkedin.com/in/ivanhzhao/
- Chris Prucha
- Jessica Lam
- Simon Last
- Toby Schachman: x.com/mandy3284. Left the company at some point.
- Akshay Kothari: Stanford
Is there an offline mode or is there not?
Possible to publish pages: www.notion.so/help/public-pages-and-web-publishing
But non-paid plan currently disables "Search engine indexing" of that sharing, so it's useless. There's an "Allow duplicate as template" button though which is nice.
URLs are horrendous however, e.g.: lofty-flower-be4.notion.site/aa-2274c59a06124d5b974b781a67340670 Only the
aa
in that came from us. They don't even have the guts for a fixed subdomain.Also it does not work without JavaScript, no SSR, everything is dynamic.
They don't show multiple input pages on the same render, e.g.: lofty-flower-be4.notion.site/aa-2274c59a06124d5b974b781a67340670 does not contain the child lofty-flower-be4.notion.site/bb-45df7212a2e14e04b3f9604035c7acf4 as already implemented on OurBigBook Web Dynamic Article Tree.
Cross page links to work fine. But you don't link to explicit IDs, only internal hidden IDs. This can be even slightly confusing to users as multiple identical options can show up when you start creating a link. They do try to disambiguate with the parent page however.
So this is a reasonable single-person publishing platform for your notes.
Someone made and sold a helper for it:
Originally at github.com/zadam/trilium, then after development stopped the community took it up at: github.com/TriliumNext/Notes.
Tree based organization at last. Infinitely deep.
Amazing WYSIWYG, including maths and tables, plus insane plugins like canvas mode, and specific file formats like code/mermaid diagrams/drawing mode.
Intentionally or not, they've basically made an open source Notion, with the possible exception that Notion historically started on web and moved to the desktop, while Trillium went the other way round.
Version history with automatic snapshots at intervals. TODO how is it implemented? Do they just ZIP multiple versions?
No multiuser features. Except for that, could have been a good starting point of an online multiuser thing such as OurBigBook.com!
With Book Notes it is possible possible to see more than one page at a time on the output, which his a major feature of OurBigBook. But does it show on HTML export as well?
You can static HTML export any subtree by right clicking on it in the navigation tree.
Is there a CLI to export to HTML? github.com/zadam/trilium/issues/3012
HTML export keeps all data as HTML is their native format. This may be inherited from CKEditor. The files are mostly visible, but there is some CSS missing, it is not 100% like editor, notably math is broken. There is also a hosted way of exposing: github.com/zadam/trilium/wiki/Sharing.
trilium.rocks however has a very good export, it is just a question of how much they had to hacked things, source at: github.com/zerebos/trilium.rocks
The default tHTML export uses frame navigation, with a toc fixed on the left frame. Efficient, but not of this century.
There is no concept of user created unique text IDs: you can have the same headers in the same folders in the UI. It's not even a matter of scopes. On exports they are differentiated as
1_name
, 2_name
, etc../Trilium Demo/Books/To read/1_HR.md
./Trilium Demo/Books/To read/2_HR.md
./Trilium Demo/Books/To read/HR.md
Markdown export warns:
this preserves most of the formatting
Architecture: runs on local SQLite database via better-sqlite3. Data apparently stored in SQLite database at
~/.local/share/trilium-data
, no raw files.Markup is stored as HTML as seen from:
sqlite3 document.db 'SELECT * from note_contents'
. HTML is their native storage format, quite interesting. But this means it is not source centric, so any source editing would have to go via import/export. It can be done apparently: github.com/zadam/trilium/wiki/Markdown but involves shoving a ZIP around.WYSIWYG based on ckeditor.com/ which is a dependency. It is kind of cool that the view in which you view the output is exactly the same as the one you edit in, and there is no intermediate format, just the HTML.
Math is KaTeX based.
It also runs on the browser via a server: github.com/zadam/trilium/wiki/Server-installation. And they have a paid service for it at: trilium.cc/. Quite impressive.
They have server to from desktop sync: github.com/zadam/trilium/wiki/synchronization. There is no conflict resolution, one of them wins randomly. But they have revision history, and anything lost will be in the revision history. They have so many features it is mind blowing.
Maintainer announced he would be slowing down development since January 2024: github.com/zadam/trilium/issues/4620?ref=selfh.st
Why Wikipedia sucks: Section "Wikipedia".
Best languages:
- latin
- esperanto. Other constructed languages: en.wikipedia.org/wiki/Wikipedia:List_of_constructed_languages_with_Wikipedias
The most important page of Wikipedia is undoubtedly: en.wikipedia.org/wiki/Wikipedia:Reliable_sources/Perennial_sources which lists the accepted and non accepted sources. Basically, the decision of what is true in this world.
Wikipedia is incredibly picky about copyright. E.g.: en.wikipedia.org/wiki/Wikipedia:Deletion_of_all_fair_use_images_of_living_people because "such portrait could be created". Yes, with a time machine, no problem! This does more harm than good... excessive!
Citing in Wikipedia is painful. Partly because of they have a billion different templates that you have to navigate. They should really have a system where you can easily reuse existing sources across articles! Section "How to use a single source multiple times in a Wikipedia article?"
What Happened To Wikipedia's Founders?
Source. - youtu.be/_Rt0eAPLDkM?t=113 encyclopedia correction stickers. OMG!
- youtu.be/_Rt0eAPLDkM?t=201 Jimmy was a moderator on MUD games
Inside the Wikimedia Foundation offices by Wikimedia Foundation (2008)
Source. WikiFauna refers to a classification of different Wiki contributor stereotypes. Some of them originate from the venerable C2 wiki.
- What motivates the WikiGnomes?A: ObSouthPark:
- Clean up zillions of WikiPages.
- ???
- PROFIT!!
- en.wikipedia.org/wiki/Wikipedia:WikiGnome
Some examples by Ciro Santilli follow.
Of the tutorial-subjectivity type:
- This edit perfectly summarizes how Ciro feels about Wikipedia (no particular hate towards that user, he was a teacher at the prestigious Pierre and Marie Curie University and actually as a wiki page about him):which removed the only diagram that was actually understandable to non-Mathematicians, which Ciro Santilli had created, and received many upvotes at: math.stackexchange.com/questions/776039/intuition-behind-normal-subgroups/3732426#3732426. The removal does not generate any notifications to you unless you follow the page which would lead to infinite noise, and is extremely difficult to find out how to contact the other person. The removal justification is even somewhat ad hominem: how does he know Ciro Santilli is also not a professional Mathematician? :-) Maybe it is obvious because Ciro explains in a way that is understandable. Also removal makes no effort to contact original author. Of course, this is caused by the fact that there must also have been a bunch of useless edits not done by Ciro, and there is no reputation system to see if you should ignore a person or not immediately, so removal author has no patience anymore. This is what makes it impossible to contribute to Wikipedia: your stuff gets deleted at any time, and you don't know how to appeal it. Ciro is going to regret having written this rant after Daniel replies and shows the diagram is crap. But that would be better than not getting a reply and not learning that the diagram is crap.
rm a cryptic diagram (not understandable by a professional mathematician, without further explanations
- en.wikipedia.org/w/index.php?title=Finite_field&type=revision&diff=1044934168&oldid=1044905041 on finite fields with edit comment "Obviously: X ≡ α". Discussion at en.wikipedia.org/wiki/Talk:Finite_field#Concrete_simple_worked_out_example Some people simply don't know how to explain things to beginners, or don't think Wikipedia is where it should be done. One simply can't waste time fighting off those people, writing good tutorials is hard enough in itself without that fight.
- en.wikipedia.org/w/index.php?title=Discrete_Fourier_transform&diff=1193622235&oldid=1193529573 by user Bob K. removed Ciro Santilli's awesome simple image of the Discrete Fourier transform as seen at en.wikipedia.org/w/index.php?title=Discrete_Fourier_transform&oldid=1176616763:with message:
Hello. I am a retired electrical engineer, living near Washington,DC. Most of my contributions are in the area of DSP, where I have about 40 years of experience in applications on many different processors and architectures.
Thank you so much!!remove non-helpful image
Maybe it is a common thread that these old "experts" keep removing anything that is actually intelligible by beginners? Section "There is value in tutorials written by beginners"Also ranted at: x.com/cirosantilli/status/1808862417566290252Figure 1. Ciro Santilli's awesome graph removed by Bob K. from the Discrete Fourier transform page.Source at: numpy/fft_plot.py. - when Ciro Santilli created Scott Hassan's page, he originally included mentions of his saucy divorce: en.wikipedia.org/w/index.php?title=Scott_Hassan&oldid=1091706391 These were reverted by Scott's puppets three times, and Ciro and two other editors fought back. Finally, Ciro understood that Hassan's puppets were likely right about the removal because you can't talk about private matters of someone who is low profile:even if it is published in well known and reliable publications like the bloody New York Times. In this case, it is clear that most people wanted to see this information summarized on Wikipedia since others fought back Hassan's puppet. This is therefore a failure of Wikipedia to show what the people actually want to read about.This case is similar to the PsiQuantum one. Something is extremely well known in an important niche, and many people want to read about it. But because the average person does not know about this important subject, and you are limited about what you can write about it or not, thus hurting the people who want to know about it.
Notability constraints, which are are way too strict:There are even a Wikis that were created to remove notability constraints: Wiki without notability requirements.
- even information about important companies can be disputed. E.g. once Ciro Santilli tried to create a page for PsiQuantum, a startup with $650m in funding, and there was a deletion proposal because it did not contain verifiable sources not linked directly to information provided by the company itself: en.wikipedia.org/wiki/Wikipedia:Articles_for_deletion/PsiQuantum Although this argument is correct, it is also true about 90% of everything that is on Wikipedia about any company. Where else can you get any information about a B2B company? Their clients are not going to say anything. Lawsuits and scandals are kind of the only possible source... In that case, the page was deleted with 2 votes against vs 3 votes for deletion.is very similar to Stack Exchange's own Stack Overflow content deletion issues. Ain't Nobody Got Time For That. "Ain't Nobody Got Time for That" actually has a Wiki page: en.wikipedia.org/wiki/Ain%27t_Nobody_Got_Time_for_That. That's notable. Unlike a $600M+ company of course.
should we delete this extremely likely useful/correct content or not according to this extremely complex system of guidelines"
In December 2023 the page was re-created, and seemed to stick: en.wikipedia.org/wiki/Talk:PsiQuantum#Secondary_sources It's just a random going back and forth. Author Ctjk has an interesting background:I am a legal official at a major government antitrust agency. The only plausible connection is we regulate tech firms
For these reasons reason why Ciro basically only contributes images to Wikipedia: because they are either all in or all out, and you can determine which one of them it is. And this allows images to be more attributable, so people can actually see that it was Ciro that created a given amazing image, thus overcoming Wikipedia's lack of reputation system a little bit as well.
Wikipedia is perfect for things like biographies, geography, or history, which have a much more defined and subjective expository order. But when it comes to "tutorials of how to actually do stuff", which is what mathematics and physics are basically about, Wikipedia has a very hard time to go beyond dry definitions which are only useful for people who already half know the stuff. But to learn from zero, newbies need tutorials with intuition and examples.
Bibliography:
- gwern.net/inclusionism from gwern.net:
Iron Law of Bureaucracy: the downwards deletionism spiral discourages contribution and is how Wikipedia will die.
- Quote "Golden wiki vs Deletionism on Wikipedia"
Per-table dumps created with mysqldump and listed at: dumps.wikimedia.org/. Most notably, for the English Wikipedia: dumps.wikimedia.org/enwiki/latest/
A few of the files are not actual tables but derived data, notably dumps.wikimedia.org/enwiki/latest/enwiki-latest-all-titles-in-ns0.gz from Download titles of all Wikipedia articles
The tables are "documented" under: www.mediawiki.org/wiki/Manual:Database_layout, e.g. the central "page" table: www.mediawiki.org/wiki/Manual:Page_table. But in many cases it is impossible to deduce what fields are from those docs.
dumps.wikimedia.org/enwiki/latest/enwiki-latest-category.sql.gz contains a list of categories. It only contains the categories and some counts, but it doesn't contain the subcategories and pages under each category, so it is a bit pointless.
The schema is listed at: www.mediawiki.org/wiki/Manual:Category_table
The SQL first defines the table:followed by a few humongous inserts:which we can see at: en.wikipedia.org/wiki/Category:Computer_storage_devices
CREATE TABLE `category` (
`cat_id` int(10) unsigned NOT NULL AUTO_INCREMENT,
`cat_title` varbinary(255) NOT NULL DEFAULT '',
`cat_pages` int(11) NOT NULL DEFAULT 0,
`cat_subcats` int(11) NOT NULL DEFAULT 0,
`cat_files` int(11) NOT NULL DEFAULT 0,
PRIMARY KEY (`cat_id`),
UNIQUE KEY `cat_title` (`cat_title`),
KEY `cat_pages` (`cat_pages`)
) ENGINE=InnoDB AUTO_INCREMENT=249228235 DEFAULT CHARSET=binary ROW_FORMAT=COMPRESSED;
INSERT INTO `category` VALUES (2,'Unprintworthy_redirects',1597224,20,0),(3,'Computer_storage_devices',88,11,0)
Se see that en.wikipedia.org/wiki/Category:Computer_storage_devices_by_companyso it contains only categories.
- en.wikipedia.org/wiki/Category:Computer_storage_devices is a subcategory of that category and it appears in that file.
- en.wikipedia.org/wiki/Acronis_Secure_Zone is a page of the category, and it does not appear
We can check this with:and it shows:There doesn't seem to be any interlink between the categories, only page and subcategory counts therefore.
sed -s 's/),/\n/g' enwiki-latest-category.sql | grep Computer_storage_devices
(3,'Computer_storage_devices',88,11,0
(521773,'Computer_storage_devices_by_company',6,6,0
The schema is listed at: www.mediawiki.org/wiki/Manual:Categorylinks_table
On the SQL:
CREATE TABLE `categorylinks` (
`cl_from` int(8) unsigned NOT NULL DEFAULT 0,
`cl_to` varbinary(255) NOT NULL DEFAULT '',
`cl_sortkey` varbinary(230) NOT NULL DEFAULT '',
`cl_timestamp` timestamp NOT NULL DEFAULT current_timestamp() ON UPDATE current_timestamp(),
`cl_sortkey_prefix` varbinary(255) NOT NULL DEFAULT '',
`cl_collation` varbinary(32) NOT NULL DEFAULT '',
`cl_type` enum('page','subcat','file') NOT NULL DEFAULT 'page',
PRIMARY KEY (`cl_from`,`cl_to`),
KEY `cl_timestamp` (`cl_to`,`cl_timestamp`),
KEY `cl_sortkey` (`cl_to`,`cl_type`,`cl_sortkey`,`cl_from`),
KEY `cl_collation_ext` (`cl_collation`,`cl_to`,`cl_type`,`cl_from`)
) ENGINE=InnoDB DEFAULT CHARSET=binary ROW_FORMAT=COMPRESSED;
TODO what is
cl_from
? We've tried:page_id
: nope, there is notpage_id
of 3
cl_to
appears to always be a category string name.The format appears to be described at: www.mediawiki.org/wiki/Manual:Categorylinks_table
A sample INSERT entry is:
(3,'Computer_storage_devices',88,11,0)
dumps.wikimedia.org/enwiki/latest/enwiki-latest-all-titles-in-ns0.gz Characterization:
- contains redirects, e.g. en.wikipedia.org/wiki/"Ampere_North" redirects to en.wikipedia.org/wiki/Ampere_North,_New_Jersey and both are present. Noted in this comment: stackoverflow.com/questions/24474288/how-to-obtain-a-list-of-titles-of-all-wikipedia-articles#comment136016773_24474476
Our WIP script: wikipedia/import-categories.sh.
Related:
- opendata.stackexchange.com/questions/1533/download-wikipedia-articles-from-a-specific-category
- webapps.stackexchange.com/questions/16359/is-there-a-way-to-download-a-list-of-all-wikipedia-categories/172480#172480
- stackoverflow.com/questions/40119322/how-to-download-all-pages-inside-a-category-in-wikipedia
- category tree on Stack Overflow
- stackoverflow.com/questions/17432254/wikipedia-category-hierarchy-from-dumps/77313490#77313490 Canon but no good answers.
- stackoverflow.com/questions/12227134/how-to-fetch-category-tree-of-wiki
- stackoverflow.com/questions/21782410/finding-subcategories-of-a-wikipedia-category-using-category-and-categorylinks-t. Actually explains it: stackoverflow.com/questions/21782410/finding-subcategories-of-a-wikipedia-category-using-category-and-categorylinks-t/21798259#21798259
- stackoverflow.com/questions/27279649/how-to-build-wikipedia-category-hierarchy
- mdkzaman.com/knowledge-graph-from-wikipedia-category-hierarchy/
Consider:
Jewish_physicists
Let's observe them in MySQL:outputs:
mysql enwiki -e "select page_id, page_namespace, page_title, page_is_redirect from page where page_namespace in (0, 14) and page_title in ('Computer_storage_devices', 'Computer_data_storage')"
+----------+----------------+--------------------------+------------------+
| page_id | page_namespace | page_title | page_is_redirect |
+----------+----------------+--------------------------+------------------+
| 5300 | 0 | Computer_data_storage | 0 |
| 42371130 | 0 | Computer_storage_devices | 1 |
| 711721 | 14 | Computer_data_storage | 0 |
| 895945 | 14 | Computer_storage_devices | 0 |
+----------+----------------+--------------------------+------------------+
mysql enwiki -e "select cl_from, cl_to from categorylinks where cl_from in (5300, 711721, 895945, 42371130)"
+----------+-----------------------------------------------------------------------+
| cl_from | cl_to |
+----------+-----------------------------------------------------------------------+
| 5300 | All_articles_containing_potentially_dated_statements |
| 5300 | Articles_containing_potentially_dated_statements_from_2009 |
| 5300 | Articles_containing_potentially_dated_statements_from_2011 |
| 5300 | Articles_with_GND_identifiers |
| 5300 | Articles_with_NKC_identifiers |
| 5300 | Articles_with_short_description |
| 5300 | Computer_architecture |
| 5300 | Computer_data_storage |
| 5300 | Short_description_matches_Wikidata |
| 5300 | Use_dmy_dates_from_June_2020 |
| 5300 | Wikipedia_articles_incorporating_text_from_the_Federal_Standard_1037C |
| 711721 | Computer_architecture |
| 711721 | Computer_data |
| 711721 | Computer_hardware_by_type |
| 711721 | Data_storage |
| 895945 | Computer_data_storage |
| 895945 | Computer_peripherals |
| 895945 | Recording_devices |
| 42371130 | Redirects_from_alternative_names |
+----------+-----------------------------------------------------------------------+
So we see that
cl_from
encodes the parent categories:- parent categories of categories:
- en.wikipedia.org/wiki/Category:Computer_data_storage, which has ID
711721
, has parent categories: "Computer hardware by type", "Computer data", "Data storage", "Computer architecture". This matches exactly on the database. These are all encoded on the source code of the page:{{DEFAULTSORT:Storage}} [[Category:Computer hardware by type]] [[Category:Computer data|Storage]] [[Category:Data storage|Computer]] [[Category:Computer architecture]]
- en.wikipedia.org/wiki/Category:Computer_storage_devices has parent categories: "Computer data storage", "Recording devices", "Computer peripherals". This matches exactly on the database.
- en.wikipedia.org/wiki/Category:Computer_data_storage, which has ID
- parent categories of pages:
- en.wikipedia.org/wiki/Computer_storage_devices whish is a redirect gets the magic category "Redirects_from_alternative_names", a humongous placeholder with many thousands of pages: en.wikipedia.org/wiki/Category:Redirects_from_alternative_names
- en.wikipedia.org/wiki/Computer_data_storage shows only two categories onthe web UI: "Computer data storage" and "Computer architecture". Both of these are present on the database and at the end of the source code:The others appear to be more magic. Two of them we can guess from the templates:
{{DEFAULTSORT:Computer Data Storage}} [[Category:Computer data storage| ]] [[Category:Computer architecture]]
are likely{{short description|Storage of digital data readable by computers}} {{Use dmy dates|date=June 2020}}
Use_dmy_dates_from_June_2020
andArticles_with_short_description
but the rest is more magic and not necessarily present in-source.
So to find all articls and categories under a given category title, say en.wikipedia.org/wiki/Category:Mathematics we can run:
mariadb enwiki -e "select cl_from, cl_to, page_namespace, page_title from categorylinks inner join page on page_namespace in (0, 14) and cl_from = page_id and cl_to = 'Mathematics'"
en.wikipedia.org/wiki/Help:Footnotes#Footnotes:_using_a_source_more_than_once gives the following method:
Definition, anywhere on article, likely ideally as the first usage:
<ref name="myname">{{cite web ...}}</ref>
And then you can use it later on as:which automatically expands the exact same thing, or using the shortcut:
<ref name="myname" />
{{r|myname}}
To cite multiple pages of a book: en.wikipedia.org/wiki/Wikipedia:Citing_sources#Citing_multiple_pages_of_the_same_source, the best method is to define and use the reference without adding the Do not set the page in or for multiple pages:
p
or location
in cite
as:<ref name="googleStory">{{cite book |title=The Google Story}}</ref>{{rp|p=123}}
cite
, otherwise it shows up on the references. Instead we use the {{rp}}
template. And then use the reference with the {{r}}
template as:{{r|googleStory|p=456}}
{{r|googleStory|pp=123, 156-158}}
To avoid duplication when citing multiple pages: Section "How to use a single source multiple times in a Wikipedia article?"
A good big sample definition:There is also
<ref name="googleStory">{{cite book |last1=Vise |first1=David |author-link1=David A. Vise |last2=Malseed |first2=Mark |author-link2=Mark Malseed |title=The Google Story |date=2008 |publisher=Delacorte Press |url=https://archive.org/details/isbn_9780385342728}}</ref>
title-link
to link to a wiki page. But it is incompatible with url=
for Internet Archive Open Library links which is a shame.So, it turns out that Wikipedia does have a (ultra obscure as usual) mechanism for pull requests. You learn a new one every day.
OMG they have that. Slightly slightly overlap with OurBigBook.com.
A 2022 clone of phabricator.wikimedia.org/source/mediawiki.git gives first commits from 2003 by:
- Lee Daniel Crocker: en.wikipedia.org/wiki/Lee_Daniel_Crockerso that gives a good notion of the last major rewrite.
He is best known for rewriting the software upon which Wikipedia runs, to address scalability problems.
- Brion Vibber
TODO when was wikipedia open sourced from Nupedia? The early days of Wikipedia are quite obscure due to its transition from Nupedia.
One day, once WebGraph exposes a PageRank CLI, we will be able to do it fully from the CLI, it will be beautiful.
Cool tool that allows you to graphically visualize page view counts of specific pages. It offers somewhat similar insights to Google Trends.
Homepage: pageviews.wmcloud.org/
Documentation: meta.wikimedia.org/wiki/Pageviews_Analysis#Massviews
The homepage shows views of selected pages, e.g. when Google had their 25th birthday: pageviews.wmcloud.org/?project=en.wikipedia.org&platform=all-access&agent=user&redirects=0&start=2023-09-11&end=2023-10-01&pages=Cat|Dog|Larry_Page Larry Page briefly beat "Cat" and "Dog".
/topviews
shows the most viewed pages for a given month: pageviews.wmcloud.org/topviews/?project=en.wikipedia.org&platform=all-access&date=2023-08&excludes= It is extremelly epic that XXX: Return of Xander Cage, a 2017 film, is on the top ten of the August 2023 month. The page was around 8th place on a Google search for "xxx": archive.ph/wip/giRY8 at the time. XXXX (beer) was also on the top 20, followed by Sex on 21.The Wikidata/Abstract Wikipedia guy.
Because of edit wars and encyclopedic tone requirements. See also: Wikipedia.
Thus OurBigBook.com.
One thing to note is that Jimmy was a finance worker before starting wikipdia, e.g. he had capital to hire Larry Sanger.
Maybe that's the way to go about it, make money first, and later on change the world.
Starting just after the beginning of the Internet can't hurt either. Though tooling must have been insane back then.
Open source software engine created for and used by Wikipedia.
Their reference markup is incredibly overengineered, convoluted, and underdocumented, it is unbelivable!
Use the reference:
This is a fact.{{sfn|Schweber|1994|p=487}}
Define the reference:
===Sources===
{{refbegin|2|indent=yes}}
*{{Cite book|author-link=Silvan S. Schweber |title=QED and the Men Who Made It: Dyson, Feynman, Schwinger, and Tomonaga|last=Schweber|first=Silvan S.|location=Princeton|publisher=University Press|year=1994 |isbn=978-0-691-03327-3 |url=https://archive.org/details/qedmenwhomadeitd0000schw/page/492 |url-access=registration}}
{{refend}}
sfn
is magic and matches the the author last name and date from the Cite
, it is documented at: en.wikipedia.org/wiki/Template:SfnUnforutunately, if there are multiple duplicate
Cite
s inline in the article, it will complain that there are multiple definitions, and you have to first factor out the article by replacing all those existing Cite
with sfn
, and keeping just one Cite
at the bottom. What a pain...You can also link to a specific page of the book, e.g. if it is a book is on Internet Archive Open Library with:
{{sfn|Murray|1997|p=[https://archive.org/details/supermenstory00murr/page/86 86]}}
For multiple pages should use
pp=
instead of p=
. Does not seem to make much difference on the rendered output besides showing p.
vs pp.
, but so be it:{{sfn|Murray|1997|pp=[https://archive.org/details/supermenstory00murr/page/86 86-87]}}
Let's see how long they last:
- Julian Schwinger: en.wikipedia.org/w/index.php?title=Julian_Schwinger&oldid=1039812272 greatly expanded the Early life and career with information from the book QED and the men who made it: Dyson, Feynman, Schwinger, and Tomonaga by Silvan Schweber (1994)
A really good option to store educational media such as images and video!
Shame that like the rest of Wikimedia, their interface is so clunky and lacking obvious features.
This is basically what Jimmy Wales had originally set out to make Wikipedia, a peer reviewed thing.
But then he noticed the entry barrier was too high while inviding an economist to review an article he wrote, and just made the more open thing instead.
The venerable first wiki.
The pre-Eternal September feeling is palpable.
People could freely comment their thoughts and sign below, making it much closer to what Ciro Santilli wants OurBigBook.com to be. But with upvotes ;-)
Nothing can better encapsulate the nostalgia of early day Internet. Genius at times, banal at others, you will be forever in our hearts!
This is good, and very close competitor to OurBigBook.com.
But they killed local build, so they are going to die.
Generally, if something is labelled as "e-learning", it's not a good sign, as it implies that it adheres to the "teacher"/"student" separation which Ciro Santilli much despises: E-learning websites must allow students to create learning content.
Charging for certification is fine. Creating exams and preventing cheating has a cost.
Another thing that is fine charging for is dedicated 1-to-1 tutor time. This is something Udacity is doing as of 2022.
www.investopedia.com/articles/investing/042815/how-coursera-works-makes-money.asp has a good mention:and it links to: www.freecodecamp.org/news/massive-open-online-courses-started-out-completely-free-but-where-are-they-now-1dd1020f59/, very good article!
MOOCs were first created by people with utopian visions for the internet. This means the idea for platforms like Coursera was likely conceived without a business plan in mind. Nonetheless, Coursera has managed to monetize its platform. It is worth noting, however, that monetization has lead to the effective elimination of the original MOOC idea, which is predicated on ideals like free and open access, as well as the building of online communities.Coursera users must pay to engage with the material in a meaningful way and take courses for individualistic purposes. This has been a consistent trend among all major online education platforms.
That is a fundamental guiding principle of OurBigBook.com. The educational content must be licensed CC BY-SA!
Perhaps the most reliable way of reaching this state is E-learning websites must allow students to create learning content.
Bibliography:
- academia.stackexchange.com/questions/86179/is-it-financially-worth-it-to-teach-a-mooc-e-g-coursera Is it financially worth it to teach a MOOC (e.g. Coursera)?
- www.classcentral.com/about amazing, they can make money just from ads! I wouldn't expect that they could scale like TripAdvisor, because travelling means very local knowledge, I would expect there to be much fewer MOOCs and for them to be more easily findable on Google. Good thing though, this website.
This is a key philosophy of OurBigBook.com!
MOOCs are a bad idea. We don't want to simply map the pre-computer classroom to the Internet. The Internet allows, and requires, fundamentally new ways to do things. More like Stack Overflow/Wikipedia. More like OurBigBook.com.
A more specific type of E-learning website generally run by a specific organization.
A website, usually hosted by an university, that takes what is done in class, and pastes it online. It is already much more rational and efficient, and opens up the way for potential sharing outside of the institution (or by default paywalling as the University of Oxford did.
The fundametnal problem with VLEs is that they tend to not have enough incentives for students to contribute at all to the content. This is basically the major motivation behind OurBigBook.com.
Some courses at least allow you to see material for free, e.g.: www.coursera.org/learn/quantum-optics-single-photon/lecture/UYjLu/1-1-canonical-quantization. Lots of video focus as usual for MOOCs.
Some are paywalled: www.coursera.org/learn/theory-of-angular-momentum?specialization=quantum-mechanics-for-engineers
It is extremely hard to find the course materials without enrolling, even if enrolling for free! By trying to make money, they make their website shit.
The comment section does have a lot of activity: www.coursera.org/learn/statistical-mechanics/discussions/weeks/2! Nice. And works like a proper issue tracker. But it is also very hidden.
November 2023 topics:
- quantum field theory: no
- condensed matter: 1 by Rahul Nandkishore from Colorado Boulder: www.coursera.org/specializations/the-physics-of-emergence-introduction-to-condensed-matter
Harvard University + MIT combo.
As of 2022:Fuck that.
- can't see course material before start date. Once archived, you can see it but requires login...
- on free mode, limited course access
Also, they have an ICP.
November 2023 course search:
- Condensed matter: 4 hits, so not too bad
- quantum field theory: no hits
Articles were limited to the first 100 out of 169 total. Click here to view all children of Website genre.
Articles by others on the same topic
There are currently no matching articles.