List of personal knowledge base software Updated 2025-03-28 +Created 1970-01-01
View more
TODO look into those more:
Major downsides that most of those personal knowledge databases have:
  • very little/no focus on public publishing, which is the primary focus of OurBigBook.com
  • either limited or no multiuser features, e.g. edit protection and cross user topics
  • graph based instead of tree based. For books we need a single clear ordering of a tree. Graph should come as a secondary thing through tags.
Closed source dump:
Read the full article
Obsidian (software) Updated 2025-03-28 +Created 1970-01-01
View more
Good:
Bad:
Figure 1.
Obsidian demo
. Source.
Read the full article
OpenStax Updated 2025-03-28 +Created 1970-01-01
View more
These people have good intentions.
The problem is that they don't manage to go critical because there's to way for students to create content, everything is manually curated.
You can't even publicly comment on the textbooks. Or at least Ciro Santilli hasn't found a way to do so. There is just a "submit suggestion" box.
This massive lost opportunity is even shown graphically at: cnx.org/about (archive) where there is a clear separation between:
  • "authors", who can create content
  • "students", who can consume content
Maybe this wasn't the case in their legacy website, legacy.cnx.org/content?legacy=true, but not sure, and they are retiring that now.
Thus, OurBigBook.com. License: CC BY! So we could re-use their stuff!
TODO what are the books written in?
Video 1.
Richard Baraniuk on open-source learning by TED (2006)
Source.
Read the full article
Alternatives Updated 2025-03-28 +Created 1970-01-01
View more
These are websites that offer somewhat overlapping services, many of which served inspirations, and why we think something different is needed to achieve our goals.
Notably, OurBigBook is the result of Ciro Santilli's experiences with:
OurBigBook could be seen as a cross between those three websites.
Quick mentions:
Static website-only alternatives:
Conceptual:
Read the full article
OurBigBook.com Updated 2025-03-28 +Created 1970-01-01
View more
Mission: to live in a world where you can learn university-level mathematics, physics, chemistry, biology and engineering from perfect free books that anyone can write to get famous.
Live website: ourbigbook.com
Further information and rationale: Section "OurBigBook.com"
The project's mission is one of, or perhaps the most important, life objective of Ciro Santilli. Reproductive goals aside. These two types of goal are incommensurable. This is one of the great challenges of life.
This is ongoing project.
Ciro's goals in advertising this half done project are is partly to obtain some feedback, and partly to give the idea to someone else who might help push it further, be it in this stack or not.
Better editing support is a must, likely WYSIWYG.
But besides that, it is already in broad strokes the best approach Ciro Santilli can come up with to try and reach the mission statement only with technical advances, i.e. without large amounts of money or political influence which Ciro Santilli does not have.
Maybe that website isn't enough of a technical advance to reach its mission. Maybe there is some further not yet imagined technical insight that would push it into viability. Maybe not. But one must try. Only God can know the answer to these questions.
As of 2022, Ciro has spent about 2.5 years full time working on this project. First he spent about 1 year in 2014 on the first iteration: github.com/booktree/booktree, a GitLab fork, but then decided it was not the way to go.
Then around 2021 he put in some more 1.5 year of full time work, now with a possibly overly complicated (or perhaps just insane/immature) Next.js/Sequelize from scratch website stack.
It makes Ciro a bit ashamed to see that "so little user visible stuff was achieved in so much time". It is partly because he and many people underestimate the difficulty of web development. Perhaps there were some bad stack/useless feature choices issues. And a good dose of indulging in studying the natural sciences to bootstrap content and have fun. But really trying is the only way to learn.
Read the full article
--format-source Updated 2025-03-28 +Created 1970-01-01
View more
Added ourbigbook --format-source automatic code formatting. I implemented it for the following reasons:
  • I want to do certain automatic modifications to source code on web, e.g.:
    • allow users to select the parent article of a new article on the web UI, but that is currently doable only with \Include macros
    • allow users to edit the source only for a specific header
  • later on, much later, this will allow WYSIWYG export to plaintext
This also ended up having one unexpected benefit: whenever a new feature is added that deprecates an old feature, by converting the large corpus from github.com/cirosantilli/cirosantilli.github.io to the new feature I can test the new preferred feature very well.
For example, converting \x[blue cat] en masse to the new insane syntax <blue cat> found several bugs with the new insane syntax.
This seemed somewhat easy at first, so I started it as a way of procrastinating more urgent Web features (web scares me, you know), but it ended being insanely hard to implement, because there are many edge cases. Also, most bugs are not acceptable, as they would corrupt your precious source code and potentially output.
But well, it is done!
Read the full article
Next steps Updated 2025-03-28 +Created 1970-01-01
View more
Editor. As last time. And the one before. But now it is for real.
I guess ended up doing all the "how things should look like" features because they clarify what the website is supposed to do, and I already have my own content to bring it alive via ourbigbook --web upload.
But now I honestly feel that all the major elements of "how things should look like" have fallen into place.
And yeah, nobody else is never going to contribute as things are! WYSIWYG is a must.
I was really impressed by Trillium Notes. I should have checked it long ago. The UI is amazing, and being all Js-based, could potentially be reused for our purposes. The project itself is a single-person/full trust notetaking only for now however, so not a direct replacement to OurBigBook.
Read the full article
Trillium Notes Updated 2025-03-28 +Created 1970-01-01
View more
Originally at github.com/zadam/trilium, then after development stopped the community took it up at: github.com/TriliumNext/Notes.
Tree based organization at last. Infinitely deep.
Amazing WYSIWYG, including maths and tables, plus insane plugins like canvas mode, and specific file formats like code/mermaid diagrams/drawing mode.
Intentionally or not, they've basically made an open source Notion, with the possible exception that Notion historically started on web and moved to the desktop, while Trillium went the other way round.
Version history with automatic snapshots at intervals. TODO how is it implemented? Do they just ZIP multiple versions?
No multiuser features. Except for that, could have been a good starting point of an online multiuser thing such as OurBigBook.com!
With Book Notes it is possible possible to see more than one page at a time on the output, which his a major feature of OurBigBook. But does it show on HTML export as well?
You can static HTML export any subtree by right clicking on it in the navigation tree.
Is there a CLI to export to HTML? github.com/zadam/trilium/issues/3012
HTML export keeps all data as HTML is their native format. This may be inherited from CKEditor. The files are mostly visible, but there is some CSS missing, it is not 100% like editor, notably math is broken. There is also a hosted way of exposing: github.com/zadam/trilium/wiki/Sharing.
trilium.rocks however has a very good export, it is just a question of how much they had to hacked things, source at: github.com/zerebos/trilium.rocks
The default tHTML export uses frame navigation, with a toc fixed on the left frame. Efficient, but not of this century.
There is no concept of user created unique text IDs: you can have the same headers in the same folders in the UI. It's not even a matter of scopes. On exports they are differentiated as 1_name, 2_name, etc.
./Trilium Demo/Books/To read/1_HR.md
./Trilium Demo/Books/To read/2_HR.md
./Trilium Demo/Books/To read/HR.md
Markdown export warns:
this preserves most of the formatting
Architecture: runs on local SQLite database via better-sqlite3. Data apparently stored in SQLite database at ~/.local/share/trilium-data, no raw files.
Markup is stored as HTML as seen from: sqlite3 document.db 'SELECT * from note_contents'. HTML is their native storage format, quite interesting. But this means it is not source centric, so any source editing would have to go via import/export. It can be done apparently: github.com/zadam/trilium/wiki/Markdown but involves shoving a ZIP around.
WYSIWYG based on ckeditor.com/ which is a dependency. It is kind of cool that the view in which you view the output is exactly the same as the one you edit in, and there is no intermediate format, just the HTML.
Math is KaTeX based.
It also runs on the browser via a server: github.com/zadam/trilium/wiki/Server-installation. And they have a paid service for it at: trilium.cc/. Quite impressive.
They have server to from desktop sync: github.com/zadam/trilium/wiki/synchronization. There is no conflict resolution, one of them wins randomly. But they have revision history, and anything lost will be in the revision history. They have so many features it is mind blowing.
Maintainer announced he would be slowing down development since January 2024: github.com/zadam/trilium/issues/4620?ref=selfh.st
Read the full article
How the tech improved Updated 2025-03-28 +Created 2025-03-08
View more
In any case, the outcome of that is that the tech has improved. And I have done a relatively good job of clearly publishing any "more user visible" improvements to docs.ourbigbook.com/news and social media such as
though it is important to note that there have been more than one "fix a hard bug" weeks that were not published because they would just bore readers.
During this period the main focus has been on improving OurBigBook Web, i.e. the dynamic website that powers OurBigBook.com. There are two reasons for that:
  • Web is what has the OurBigBook topics feature for mind-melding, which is the killer feature of OurBigBook compared to other note taking apps and therefore deserves the highest levels of priority
    Static website generation is an indispensable escape valve that ensures that your content can be published forever even if OurBigBook.com goes down one day, which it won't as long as I live. But the innovation is Web.
  • static website generation was closer to good enough, but web was much further and is fundamentally harder.
    I'm extremely satisfied with OurBigBook static website generation and haven't touched it as much. It wasn't easy to reach this state, but I'm there.
    But Web is a different and much more complex beast.
    Making CLI software that will run on a person's local computer under full trust and building a bunch of HTML from lightweight markup in bulk is one thing.
    But making a public dynamic website that has to continuously maintain a coherent database state on granular updates, while giving users some trust but not enough for them to blow everything up is on a totally different level. See e.g. the recent SPAM attack we've had to fend off.
    Figure 1.
    Screenshot showing voting manipulated SPAM as the most highly upvoted article on OurBigBook.com
    . Source.
    And then there's also the issue of front-end being mega-hard to get right.
As a result, Web is now way less buggy and much more usable.
If you look through the list of Web updates, there is nothing specifically mind blowing. The core ideas have largely crystallized, and we are just trying to making them click. I have a few more punches up my sleeve, but the core is decided.
Figure 3.
OurBigBook Web article announcement
. Source. Another cute new feature, you can send an email to your followers about a new amazing article you created.
Web process has been somewhat slower than what I'd like. Of course, it is the case of any project that things are easily said than done. But there are two other main structural factors that have played into it:
  • I have my first baby now, and we're learning how to deal with that on the fly.
    For example, we could have put him on childcare a bit earlier, but due to inexperience we've kept him a bit longer than we maybe should have.
    Things are well sorted out now, but not matter how good your support system is, at the end of the day, and more often night, it is you the parents that have to deal with a lot of inevitable baby issues. Unless you want them to turn into psychopaths and drug addicts that is, which I don't. I've reached the point of semi failure middle age that the baby feels like my best moonshot.
    All of this sets a fundamental limit on how many hours you can work per week.
    But at least with the donations I was able to work on OurBigBook at all. Because if it weren't for that, I would have to focus entirely on the generic job instead and OurBigBook would have been put on hold.
  • the choice of Web stack. I was allured by Next.js. I can see the beauty and usefulness of a Node.js render front-end that also runs on backend and hydration. That is awesome.
    But:
    • React is insanely hard to learn and understand. Furthermore, it is also hard to understand the performance problem that it solves, and actually have a benchmark where this problem is solved faster than just delivering some HTML files with ad-hoc Js on top.
    • the lack (or perhaps excess of shitty) actual web framework like Ruby on Rails and Django means that I have to rediscover the wheel many times over for all the essential support activities like testing, login and so one
    At this point a rewrite is out of the question. I've managed to master things well enough to get a decent result, and given up on the few things that I couldn't for the life of me achieve, after documenting them very well for posterity of course.
Aside from Web, there was only one thing that received a significant improvement, and that was the OurBigBook VS Code extension. The extension is not perfect, and it is not the "final UI", which has to be some WYSIWYG implementation, and there are some fundamental limitations that cannot be overcome without patching VS Code itself. However, the extension is already extremely usable, and I'm writing this on it right now. Basics like syntax highlighting, jump to definition and autocomplete are very useful and usable.
Figure 4.
Tree navigation in the OurBigBook Visual Studio Code extension
.
Read the full article
I ended up doing tech rather than content as usual Updated 2025-03-28 +Created 2025-03-08
View more
At first I had intended to create a lot more content for the world class university located where I lived, but I ended up not doing that and just improving the project tech instead.
There are a few reasons for this, good or bad:
  • as a tech nerd, my natural tendency is to first sit down by myself and code to solve big general problems rather than go out and try to solve specific people's specific problems to obtain money and users
  • at one point I got the feeling that helping students with a bunch of small courses might be useful, and that instead I might get more impact by instead by focusing on creating content for a next big thing area such as:
    because many of the courses are fundamentally useless by design due to misalignment between university and reality.
    I'm still not sure what to do about that, but I do think I'll try to do a bit of course solving at least and see how it goes.
    One thing I've learned first hand through Ciro Santilli's Stack Overflow contributions and Linux Kernel Module Cheat is that the barrier to make money from a useful open source learning project that benefits a large number of people a little bit is huge, perhaps infinite, and that it might be better to instead focus more intensely on fewer users. This insight pushes me more towards going for solving local courses.
    Another consideration that supports going for courses is that being close to students is perhaps my only unfair advantage. There is likely no one else in the world in the same position that I'm at, with some "free time" to chill with undergrads and help them with 100% of my undivided attention and passion.
    A point that pulls me towards the big tutorials however is that my time is almost up, and focusing on them would increase the chances that I will be work in those fields afterwards. This feeling may go against the best interests of the project, but it is perhaps an inevitable self preservation consideration unless someone decides to free me from that forever with the 2M :-)
  • the entry barrier to help students of a top university is rather high. The students are already extremely busy and pressured (this is pe), and if it is in the slightest hard to explain their problems to you because you are not fluent enough in their subject, they will find a faster way to obtain the knowledge and never come to you.
  • I also did a bit of procrastinating with a few quick few exploration into cute programming projects. Nothing too crazy long however, just the usual. It's in my nature to have broad interests, and perhaps only such a person can make a OurBigBook.com. I'm not a fast worker. But I never stop. Once something is in my "this must be done or learnt list", I just keep coming back to it again and again until it happens.
The downsides of going for tech first are severe:
  • you risk being misaligned with what users want and spend enormous amounts of time on useless features
  • it is also rather demotivating that you are working hard on a really cool feature but you know that there are no users yet so no one will benefit from it, and that this feature alone is not enough to attract the users anyways
There are however counterpoints to these as for anything else:
  • I'm a user and I'm always improving it for myself. If there are other people like me out there, they will love it. If there aren't, perhaps I'll never be able to do anything that caters for them well enough anyways.
  • as the two users made me understand, once someone touches your thing, they expect it to be perfect, and their standards are extremely high. This is understandable in part given the large number of note taking apps in existence, and notably WYSIWYG ones. As such, there is some rationale for improving tech.
Read the full article
Metrics and rationales Updated 2025-03-28 +Created 2025-03-08
View more
Long story short, the project is so far a complete failure on the most important metric: number of regular users, which current sits at exactly one: myself.
There were notable users who found the project online and who actually tried to use the website for some content and provided extremely valuable feedback:
Unfortunately after the period of a few weeks they stopped using it to follow their other priorities instead. Which is of course totally fine, however sad.
I still believe that the OurBigBook Web feature is a significant tech innovation that could make the website go big.
I also believe that the project gets many fundamentals of braindumping right, notably the infinitely deep table of contents without forced scoping, e.g.:
- Mathematics
  - Calculus
does not make Calculus have an ID orr URL of mathematics/calculus, rather it's just calculus.
But there is a fundamental difficulty in reaching critical mass to that self-sustaining point, as people don't seem to be convinced by these logical "my system is better" argument alone, as opposed to having them Google into stuff they need now and then understand that the project is awesome.
A closely related critical mass issue is that existing big multiuser knowledge base websites such as Stack Overflow and Wikipedia have a tremendous advantage on PageRank. No matter how useless a Wikipedia article about something is, it will always be on top of Google within a week of creation for title hits. And since the main goal of publishing your stuff is to get it seen, it makes much more sense for writers to publish on such existing websites whenever possible, because anywhere else it is way way less likely to be seen by anybody.
Even I end up writing way more on Stack Overflow than on OurBigBook as a programmer. But I still believe that there is a value to OurBigBook, for the usual reasons of:
  • it allows you to organize a more global view of a subject, i.e. a book. Even I write answers on Stack Overflow, I also tend to organize links to these answers in a structured ways here, see e.g. big topics such as SQL
  • deletionism and overly narrowness of allowed topics/style
Perhaps what saddens me the most is that even on GitHub stars/Twitter/Hacker news terms there is almost no interest in the project despite the fact that I consider that it has innovations, while many other note taking apps as well in the thousands of stars. Maybe I'm just delusional and all the tech that I'm doing is completely useless?
Part of the issue is probably linked to the fact that most other note taking apps focus on "help me organize my ideas so I can make more money" and often completely ignore "I want to publish my knowledge", and stuff that helps you make money is always easier to sell and promote.
OurBigBook on the other hand a huge focus on "I want to publish me knowledge". It aims almost single mindedly in being the best tool ever for that. However this doesn't make money for people, and therefore there are going to be way less potential users.
I do believe strongly that all it takes is a few users for the project to snowball. For some people, once you start braindumping, it is very addictive, and you never want to stop basically. So with only a few of those we can open large parts of undergrad knowledge to the world. But these people are few, and so far I haven't been able to find even a single one like me, and on top of that convince them that I have created the ultimate system for their knowledge publishing desires.
Another general lesson is that I should perhaps aimed for greater compatibility with existing systems such as Obsidian. Taking something that many people already know and use can have a huge impact on acceptance. E.g. anything that touches Obsidian can reach thousands of stars: github.com/KosmosisDire/obsidian-webpage-export. Note taking apps that aim for "markdown" compatibility also tend to fare better, even if in the end you inevitably have to extend the Markdown for some of your features. And WYSIWYG, which I want but don't have, is perhaps the ultimate familiarity.
Another issue compared to other platforms is that OurBigBook just came out late. Obsidian launched in 2020. Roam Research and Trillium Notes also came earlier. And it is hard to fight the advantage already gained by those on the "I'm going to take some personal notes" area. I do believe however that there a strong separation between "these are my personal notes" and "I want to publish these". Once you decide to publish your knowledge, you immediately start to write in a different way, and it is very hard to convert pre-existing "private" notes into ones suitable for public consumption.
Read the full article
What might be next Updated 2025-03-28 +Created 2025-03-08
View more
OK, I need to do content. I know :-) At the university I'm at, the only department that is open is the mathematics one. Both:
  • physically, I'm sitting next to some students right now, though they don't yet know that their saviour is just next to them.
  • in terms of publishing the course materials online. Many of them even have solution
All other courses extremely closed, notably Physics, which is the other course I'd consider. There are upsides and downsides for going for Mathematics:
  • upside:
    • maths doesn't change with time
    • maths doesn't require experiments
  • downside: most of it is useless compared to Physics
If I were free to choose, I might go for Physics instead. But maths isn't hard, and I think I'll just go with the hand I'm dealt this time to start with.
Tech wise, the big things are the following ones to which I have given different levels of architectural consideration (i.e. read: I'm afraid they'll be fucking hard and that I'll spend a month on yet another useless feature that won't help get a single user). I don't think I'll do those before at least a little bit of content, we'll see:
  • WYSIWYG: this is not a question of if, but when and how. Even I miss it when dealing with images. I was particularly impressed by Trillium Notes, and might consider forking it or reusing some of its components
  • perfect two way sync from web to local: github.com/ourbigbook/ourbigbook/issues/326
    Currently, after much effort, publishing from local to web is extremely good.
    But pulling back changes that you make on web UI locally is not really possible. A basic version can be made easily, but a great version requires some thought.
    In particular, preventing accidental rewrite on simultaneous local + web edits require edit history to be in place.
    The rationale here is that users would start editing on Web with a low entry barrier. And as they become more committed to the project, they would eventually transition to having editing most of their content locally from a desktop, with the exception of a few minor edits on the go when they are on a cell phone, and which we want to very easily and automatically be pulled back to local as soon as they open an editor on their laptop.
    I.e. we want to add a downwards arrow to the following diagram:
    https://raw.githubusercontent.com/ourbigbook/ourbigbook-media/master/feature/local-editing/bigb-publish-to-web-or-static-editor-logos.svg
Smaller cute tech that I might do before content "real quick" include:
Another thing I really want to do before time is up is to create a video summarizing my philosophy of education. I want it to be as fun and funny and sad as possible, with silly moving animated images and slides, not just me talking to the camera. Although all of the points I intend to talk about have undoubtedly been covered by others, it is something that I feel so strongly about that I would like to tell others about it more personally. If I start it it will likely take a few days to get done, and I'm not sure wha the final quality would be. It is a bit sad to not do "project work", but I think I'll end up doing it regardless. Class it under "fundraising" if you will, as it may help to find other like minded but rich people.
Read the full article
Zettlr Updated 2025-03-28 +Created 2024-08-10
View more
Interesting "gradual" WYSIWYG. You get inline previews for for things like images, maths and links. And if you click to edit the thing, the preview mostly goes away and becomes the corresponding source code instead.
Read the full article
Zim Updated 2025-03-28 +Created 2024-11-18
View more
Local only.
WYSIWYG:
  • bold
  • images
  • lists. But it is either hard or impossible to have a paragraph inside a list item.
Figure 1.
Zim
.
Mathematics requires a plugin and a full LaTeX install: zim-wiki.org/manual/Plugins/Equation_Editor.html They have a bunch of plugins: zim-wiki.org/manual/Plugins.html
Can only link to toplevel of each source, not subheaders? And subpages get forced scope. github.com/zim-desktop-wiki/zim-desktop-wiki
Publishing to static HTML can be done with:
zim --export Notes -o out
The output does not contain any table of contents? There is a plugin however: zim-wiki.org/manual/Plugins/Table_Of_Contents.html
It is unclear if their markup is compatible with an existing language of if it was made up from scratch. Wikipedia says:
In Zim, text is written and saved in a lightweight mark-up that is a hybrid of DokuWiki and Markdown.
You can't determine the ordering or pages at the same level, alphabetical ordering of force. The poplevel is encoded in notebook.zim:
[Notebook]
home=Home
Feature request: github.com/zim-desktop-wiki/zim-desktop-wiki/issues/32. It's not usable as a publishing system!
Read the full article