Folding@home by Ciro Santilli 35 Updated +Created
Is fog computing more efficient than cloud computing? by Ciro Santilli 35 Updated +Created
Advantages of fog: there is only one, reusing hardware that would be otherwise idle.
Disadvantages:
  • in cloud, you can put your datacenter on the location with the cheapest possible power. On fog you can't.
  • on fog there is some waste due to network communication.
  • you will likely optimize code less well because you might be targeting a wide array of different types of hardware, so more power (and time) wastage. Furthermore, some of the hardware used will not not be optimal for the task, e.g. CPU instead of GPU.
All of this makes Ciro Santilli doubtful if it wouldn't be more efficient for volunteers simply to donate money rather than inefficient power usage.
Bibliography:
Amazon Web Services by Ciro Santilli 35 Updated +Created
AWS service by Ciro Santilli 35 Updated +Created
Git tips / git rebase moves commits one by one by Ciro Santilli 35 Updated +Created
In order to solve conflicts, you just have to understand what commit you are trying to move where.
E.g. if from:
5 master
|
4 7 my-feature HEAD
| |
3 6
|/
2
|
1
we do:
git rebase master
what happens step by step is first 6 is moved on top of 5:
6on5 HEAD
|
5 master
|
4                 7 my-feature
|                 |
3                 6
|                 |
2-----------------+
|
1
and then 7 is moved on top of the new 6:
7on5 HEAD
|
6on5
|
5 master
|
4                 7 my-feature
|                 |
3                 6
|                 |
2-----------------+
|
1
All good? so OK, let's move the my-feature to the new 7:
7on5 my-feature HEAD
|
6on5
|
5 master
|
4
|
3
|
2
|
1
Git tips / How to modify the commit tree by Ciro Santilli 35 Updated +Created
Option 1) git commit. Doh!!!
Option 2) git rebase. Basically allows you to do arbitrary modifications to the tree. The most important ones are:
AWS Deep Learning Base GPU AMI (Ubuntu 20.04) by Ciro Santilli 35 Updated +Created
These come with pre-installed drivers, so e.g. nvidia-smi just works on them out of the box, tested on g5.xlarge which has an Nvidia A10G GPU. Good choice as a starting point for deep learning experiments.
EC2 instance store volume by Ciro Santilli 35 Updated +Created
Large but ephemeral storage for EC2 instances. Predetermined by the EC2 instance type. Stays in the local server disk. Not automatically mounted.
EC2 instance type by Ciro Santilli 35 Updated +Created
Amazon's informtion about their own intances is so bad and non-public that this was created: instances.vantage.sh/
Git tips / How to visualize the commit tree by Ciro Santilli 35 Updated +Created
Generate a minimal test repo. You should get in the habit of doing this to test stuff out.
#!/usr/bin/env bash

mkdir git-tips
cd git-tips
git init

for i in 1 2 3 4 5; do
  echo $i > f
  git add f
  git commit -m $i
done

git checkout HEAD~2
git checkout -b my-feature

for i in 6 7; do
  echo $i > f
  git add f
  git commit -m $i
done
Google Cloud Platform by Ciro Santilli 35 Updated +Created
Infrastructure as a service by Ciro Santilli 35 Updated +Created
You SSH into a an OS like Ubuntu and do whatever you want from there. E.g. Amazon EC2.
The OS is usually virualized, and you get only a certain share of the CPU by default.
High performance computing by Ciro Santilli 35 Updated +Created
bsub get job stdout and stderr by Ciro Santilli 35 Updated +Created
By default, LSF only sends you an email with the stdout and stderr included in it, and does not show or store anything locally.
One option to store things locally is to use:
bsub -oo stdout.log -eo stderr.log 'echo myout; echo myerr 1>&2'
as documented at:Or to use files with the job id in them:
bsub -oo %J.out -eo %J.err 'echo myout; echo myerr 1>&2'
By default bsub -oo:
  • also contains the LSF metadata in addition to the actual submitted process stdout
  • prevents the completion email from being sent
To get just the stdout to the file, use bsub -N -oo which:
  • stores only stdout on the file
  • re-enables the completion email
as mentioned at:
Another option is to run with the bsub -I option:
bsub -I 'echo a;sleep 1;echo b;sleep 1;echo c'
This immediately prints stdout and stderr to the terminal.
Git tips / Linear history vs branching by Ciro Santilli 35 Updated +Created
There are two ways to organize a project:
Some people like merges, but they are ugly and stupid. Rebase instead and keep linear history.
Linear history:
5 master
|
4
|
3
|
2
|
1 first commit
Branched history:
7   master
|\
| \
6  \
|\  \
| |  |
3 4  5
| |  |
| /  /
|/  /
2  /
| /
1/  first commit
Here commits 6 and 7 are the so called "merge commits":
  • they have multiple parents:
    • 6 has parents 3 and 4
    • 7 has parents 5 and 6
  • they are useless and don't contain any real information
Which type of tree do you think will be easier to understand and maintain?
????
????????????
You may disconnect now if you still like branched history.
Git tips / Merge two or more commits into one by Ciro Santilli 35 Updated +Created
Before
7 my-feature HEAD
|
6
|
5 master
|
4
|
3
|
2
|
1
Oh, commit 6 was just a temporary step, should be put together with commit 7:
git rebase -i HEAD~2
Mark 6 to be squashed.
After:
67 my-feature HEAD
|
5 master
|
4
|
3
|
2
|
1
Better now, ready to push.
Git tips / The key to solve conflicts: see the two conflicting diffs by Ciro Santilli 35 Updated +Created
The key to solve conflicts is:
You have to understand what are the two commits that touched a given line (one from master, one from features), and then combine them somehow.
Or in other words, at every rebase conflict we have something like:
master-commit    feature-commit
|                |
|                |
base-commit------+
|
|
Therefore there are 2 diffs that you have to understand and reconcile:
  • base-commit to master-commit
  • base-commit to feature-commit
Git tips / Understand the commit tree by Ciro Santilli 35 Updated +Created
This is the most important thing to understand Git!
You must:
  • be able to visualize the commit tree
  • understand how each git command modifies the commit DAG
Big Bang by Ciro Santilli 35 Updated +Created

Pinned article: ourbigbook/introduction-to-the-ourbigbook-project

Welcome to the OurBigBook Project! Our goal is to create the perfect publishing platform for STEM subjects, and get university-level students to write the best free STEM tutorials ever.
Everyone is welcome to create an account and play with the site: ourbigbook.com/go/register. We belive that students themselves can write amazing tutorials, but teachers are welcome too. You can write about anything you want, it doesn't have to be STEM or even educational. Silly test content is very welcome and you won't be penalized in any way. Just keep it legal!
We have two killer features:
  1. topics: topics group articles by different users with the same title, e.g. here is the topic for the "Fundamental Theorem of Calculus" ourbigbook.com/go/topic/fundamental-theorem-of-calculus
    Articles of different users are sorted by upvote within each article page. This feature is a bit like:
    • a Wikipedia where each user can have their own version of each article
    • a Q&A website like Stack Overflow, where multiple people can give their views on a given topic, and the best ones are sorted by upvote. Except you don't need to wait for someone to ask first, and any topic goes, no matter how narrow or broad
    This feature makes it possible for readers to find better explanations of any topic created by other writers. And it allows writers to create an explanation in a place that readers might actually find it.
    Figure 1.
    Screenshot of the "Derivative" topic page
    . View it live at: ourbigbook.com/go/topic/derivative
  2. local editing: you can store all your personal knowledge base content locally in a plaintext markup format that can be edited locally and published either:
    This way you can be sure that even if OurBigBook.com were to go down one day (which we have no plans to do as it is quite cheap to host!), your content will still be perfectly readable as a static site.
    Figure 5. . You can also edit articles on the Web editor without installing anything locally.
    Video 3.
    Edit locally and publish demo
    . Source. This shows editing OurBigBook Markup and publishing it using the Visual Studio Code extension.
  3. https://raw.githubusercontent.com/ourbigbook/ourbigbook-media/master/feature/x/hilbert-space-arrow.png
  4. Infinitely deep tables of contents:
    Figure 6.
    Dynamic article tree with infinitely deep table of contents
    .
    Descendant pages can also show up as toplevel e.g.: ourbigbook.com/cirosantilli/chordate-subclade
All our software is open source and hosted at: github.com/ourbigbook/ourbigbook
Further documentation can be found at: docs.ourbigbook.com
Feel free to reach our to us for any help or suggestions: docs.ourbigbook.com/#contact