to get these local conversion updates and a few others I've added recently.
Thanks a lot once again for this report! Local to web upload is hard to get right, and I've had many issues myself, but I try to fix this with high priority, and new users stress it in different ways which is great.
A tip, during web upload, ourbigbook first splits the headers to separate files under out/web, so looking there can help understand what went wrong. But not something I want users to ever have to worry about of course, the goal is that if it converts locally without error, it should upload without error.
Your files have to be really huge before being in a single file starts becoming a problem, so don't worry too much about it.
Previously cirosantilli.com was one humongous README.adoc, but it reached a point where the HTML was too slow to load lol.
That was part of the motivation of ourbigbook. But when you reach that point, ourbigbook makes the transition to multifile will be as seamless as it can be.
BTW you might want to use the {parent= syntax rather than header levels.
= My h1
= My h2
{parent=My h1}
= My h3
{parent=My h2}
This allows you to move chunks around very easily (within the same file, or to other files), without having to change a billion header levels each time.
Hmm can you give a bit more of the error logs and perhaps local source? Also double check that every file is reachable from the README.bigb via \\Include (I'm going to add a local check for this soon).
OK, I understood the problem now by inspecting the server DB. The root problem is that you uploaded something like (by accident or due to another ourbigbook bug):
= asdf $$
$$ qwer
which the server incorrectly accepted, such syntax should not be allowed.
When, the web_delete things is what happens when you delete an ID locally, but the server still contains it, so it deletes it from the server, which means: 1) mark as unlisted 2) make the body empty (remove all sub-ids like images) thus leaving you with:
= asdf $$
which is an unclosed string literal thus the error message.
I've now tried to clean that up from the DB, and I'll make sure that the offending syntax is forbidden both locally and on server (I've been adding several such checks in recent patches and I'll add this in as well, I was not aware of it).
Can you try ourbigbook --web again and see if it got fixed?
Some logs for my own reference:
path
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
@pioyi/claus-processcreated-by-the-german-chemist-carl-friedrich-claus-it-makes-it-possible-to-convert-hydrogen-sulfide-from-natural-gas-to-elemental-sulfur-the-overall-reaction-can-be-descibed-as-follows-2-h-2s-plus-so-2-3s-plus-2h-20.bigb
(1 row)
ourbigbook::DATABASE=> select "titleSource" from "File" where "path" LIKE '@pioyi/claus-processcreated-by-the-german-chemist-carl-friedrich-claus-it-makes-it-possible-to-convert-hydrogen-sulfide-from-natural-gas-to-elemental%';
titleSource
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Claus processCreated by the german chemist Carl Friedrich Claus, it makes it possible to convert hydrogen sulfide (from natural gas) to elemental sulfur.The overall reaction can be descibed as follows: $$
(1 row)
ourbigbook::DATABASE=> select "bodySource" from "File" where "path" LIKE '@pioyi/claus-processcreated-by-the-german-chemist-carl-friedrich-claus-it-makes-it-possible-to-convert-hydrogen-sulfide-from-natural-gas-to-elemental%';
bodySource
-----------------------------------------------------------------------------------------------------------------------------------------------------
2 H_2S + SO_2 -> 3S + 2H_20. +
$$ +
+
The Claus' process served as a better replacement of the Frasch process, which obtained elemental sulfur from naturally found deposits underground.+
(1 row)
ourbigbook::DATABASE=>
And how I tried to fix it:
ourbigbook::DATABASE=> update "File" set "bodySource" = '' where "path" LIKE '@pioyi/claus-processcreated-by-the-german-chemist-carl-friedrich-claus-it-makes-it-possible-to-convert-hydrogen-sulfide-from-natural-gas-to-elemental%';
UPDATE 1
ourbigbook::DATABASE=> update "File" set "titleSource" = 'claus-processcreated-by-the-german-chemist-carl-friedrich-claus-it-makes-it-possible-to-convert-hydrogen-sulfide-from-natural-gas-to-elemental-sulfur-the-overall-reaction-can-be-descibed-as-follows-2-h-2s-plus-so-2-3s-plus-2h-20' where "path" LIKE '@pioyi/claus-processcreated-by-the-german-chemist-carl-friedrich-claus-it-makes-it-possible-to-convert-hydrogen-sulfide-from-natural-gas-to-elemental%';
UPDATE 1
ourbigbook::DATABASE=>
These are things that would be really cool to have as a more structured metadat like Wikipedia does via Wikidata, but I'm staying off the structured data for now, though tempted.
Awesome! Good reminder that I need to add comment editing as well 😂
And as usual, feel free to write about anything you want and at any quality level, including quick drafts or self notes. There is no moderation and nor algorithmic downside for crap content of any kind.
BTW I've added the following to docs.ourbigbook.com/ourbigbook-com-content-license I hope this is a reasonable clause given that I'm putting years of my life into this project. I understand it could put some people off, and I don't intend to ever relicense anything lightly, as it would be an instant reputational hit. This had been on the back of my mind for a while, but after seeing your contributions I've decided to do it now before more content gets added. It does not affect any of the pre-existing content.
Starting from August 22 2024, users also automatically grant to the OurBigBook Project a non-exclusive license to relicense their content. This could be used for example to:
sell the content to companies that do not wish to comply with the CC By-SA license, e.g. for LLM training. We will try to avoid ever doing this as much as possible since it goes against the vision of the project for open knowledge. But it could one day be the difference between life and death of the project, so we'd like to keep that door open just in case.
add a new license to content on the website which we feel might better serve all users
Any such relicensing does not affect the original CC BY-SA 4.0 license nor your ownership of the content. It only adds new licenses on top of it. This way the content remains free no matter what.
A person from the Philippines. I just Googled your username and it says it's a surname in the Philippines haha Feel free to write about anything you want, there are no penalties or interference of any kind.
BTW, I'm curious at which level/which university/course you are at if you'd like to share that here or privately: ourbigbook.com/cirosantilli/contact This is totally optional of course. Philipino University undergrad is my guess (?) :-)
Some more things that I encountered as I tried to add a second article: Whenever I publish something new from my terminal, I am immediately logged off and I have to keep logging back in to make small changes to the articles that I have written.
One important thing though: there is currently no way to sync server -> local, only local -> server.
I really want to do this, but it is potentially hard because of how to handle merge conflicts. Which in turn requires some kind of history implementation for us to know who is outdated, local, server or both.
So for now, once you start doing things locally, don't ever edit on server, or else the next local push will just erase everything you edited on server.
Also I couldn't publish a second article correctly. It seemed like name clashes were present, as I had made a second "references" section for the second article. But why is that? Shouldn't it all be allowed as the two "references" sections have different parents and thus are not identical.
and so you get a conflict on references and it blows up.
I understand this is not how systems most people use work and it can be a bit confusing, but I strongly believe that this is a desirable feature and one of the great "innovations" of OurBigBook.
Already in this example we can see that /hydrogen-sulfide/book-1-by-john-smith is not a very good ID, we would rather want just /book-1-by-john-smith for it.
I.e., when you start scoping, how do you unscope things?
If we automatically added each ancestor ID to each ID, we would have IDs that don't even fit in allowed URLs anymore.
In order to get topics hits across multiple users, we need short IDs, and not auto scoping by ancestors by default is the best bet.
I remember that I published my first article with a hack that I have already forgotten. And this of course is not the standard way that you wanted. I find the publication process a bit too complex for a beginner, especially for one who has no knowledge of computer science! (thankfully I have some, but even this was not enough in my case. I even watched some videos on how to publish projects and yet I couldn't find a way that made me comfortable).
Again, I apologize for my misuse of the website. I am surely not reading the docs enough.
These are perfectly valid points, do criticize freely, and perhaps inevitable killer blows to the success of this project.
Ultimately, full user convenience can only come when WYSIWYG is done on Web UI and local, and perfect sync happens between them. Which I want to do. It's just a question of should I do content first to try and get regular users or after these potentially big tech steps (because as it may be obvious, I've done far too much tech, with far too few users besides myself to this point).
The scoping issue above is inevitable I believe: a "small" paradigm shift that makes our key innovation work (topics).
But regardless, things are definitely not forgiving now, and or not well explained well enough on intuitive UIs/error messages, and I want to get as close as possible to perfection in those points as I can.
Things are in this state for a mixture of reasons, good and bad.
A big part is of course my blind spots due to lack of other users, which makes feedback like yours extremely valuable.
But there is also a pragmatic feeling that at least initially, we are more likely to get a small number of huge users than a large number of small users, because small users only come from Google to web edit, and Google won't ever see us unless some big users put amazing content.
In any case, I will try to get as close as perfection in in-app explanations as I can and any ideas toward that are welcome. I want users to be able to discover how to do things as much as possible without reading the docs, definitely.
Is there a way to not have a new line every time I put some latex in the text?
Do you mean like inline vs block maths with $ vs $$? As in:
My inline $\sqrt{2}$ is nice.
My block:
$$
\sqrt{2}
$$
is also nice.
You can do code blocks with triple backticks like in markdown btw (though we are better and also support double backticks only :-)): docs.ourbigbook.com/#code-block
``
= Bleach
{scope}
== Whatever
``
which renders as:
= Bleach
{scope}
== Whatever
That should work. Could you show me all files in your local directory?
Notably, do you have the above in a file called "bleach.big"? Or is it just inside your README.bigb?
If separate file, you need to include as:
README.bigb
= Index
\Include[bleach]
If same file as in:
= Index
= Bleach
== Whatever
then it doesn't work for web upload, because everything has to be a child of = Index to go into web tree (all web goes under a single root). So instead you need:
= Index
== Bleach
=== Whatever
I should, and will add a error message for that case btw if its getting silently ignored. I had tried to be more general from before Web existed, and something need to be slightly restricted to work on web.
This is one of those "needs a better error message on my blindspot" cases.
Later on let me know how your local files look for --web upload. In principle, everything should work even if the articles already exist on server, it should just restructure everything to look exactly like your local output tree.
I'll check the homepage 500 issue. The more bugs found the better!
Also maybe some things to check (not that they are 100% bugs): 1) When viewing parent/children articles sometimes the dropdown menu doesn't work when clicked (the upside down triangle).
2) My idea is to one day allow storing undeletable post history as well, so that when you link to someone's permalink, it NEVER ever goes down. This is similar to what Wikipedia and Stack Overflow do. If this ever becomes a problem we can reconsider, but generally speaking storage is cheap and CPU-hours are relatively expensive.
My new article seemed awesome when I previewed it via the index.html file but when I uploaded it (with --web) the inheritance was broken. Maybe that's because the article already existed and I tried to push it again! Yes, that makes sense.
I'll have a look at what you mean when I can restore your homepage :-) Also, if you manage to push your .bigb files into a github repo, that might help debug/understand issues (even though ultimately I should be able to figure it out from db).
I feel kind of bad for all the requests that I am making, I should preview my stuff before committing it.
Not at all, if anything comes up feel free to immediately comment without too much thinking. Because I have basically no users besides myself, any feedback is extremely valuable in guiding what I should do next.
Yes, give it a try and let me know if you face any issues. I want both to be amazing, but perhaps cli is a bit better now for those who can use it. Give the Visual Code Extension a try.
The equals sign space: = would generate a header, but you can't create headers directly on the web editor. You should instead create new sections by just creating new articles with the New button. Then you set the parent to the other article as shown at: www.youtube.com/watch?v=yQRjLyqILl0 Note that the UI changed slightly from that older video, the parent selection is now under the "metadata" tab in the editor.
If you find any other issues or have any doubts do let me know and I'll try to fix/reply ASAP. This serious editor bug was there because I use mostly local editing, but I want the editor to work perfectly as well.
out/web, so looking there can help understand what went wrong. But not something I want users to ever have to worry about of course, the goal is that if it converts locally without error, it should upload without error.{parent=syntax rather than header levels.README.bigbvia\\Include(I'm going to add a local check for this soon).web_deletethings is what happens when you delete an ID locally, but the server still contains it, so it deletes it from the server, which means: 1) mark as unlisted 2) make the body empty (remove all sub-ids like images) thus leaving you with:ourbigbook --webagain and see if it got fixed?title2like: ourbigbook.com/cirosantilli/list-of-chemical-elements#helium with source: ourbigbook.com/go/source/cirosantilli/helium{scope}in both, this is what I was trying to say at: ourbigbook.com/go/discussion/1/pioyi/oveview and related docs: docs.ourbigbook.com/#h-scope-argumentreferencesand it blows up./hydrogen-sulfide/book-1-by-john-smithis not a very good ID, we would rather want just/book-1-by-john-smithfor it.$vs$$? As in:= Indexto go into web tree (all web goes under a single root). So instead you need:=would generate a header, but you can't create headers directly on the web editor. You should instead create new sections by just creating new articles with the New button. Then you set the parent to the other article as shown at: www.youtube.com/watch?v=yQRjLyqILl0 Note that the UI changed slightly from that older video, the parent selection is now under the "metadata" tab in the editor.