Where to store images

Top articles Latest articles New article in topic
Where to store images by Ciro Santilli 34 Updated 2024-11-19 Created 1970-01-01
Since images are large, they bring the following challenges:
  • keeping images in the main Git repository with text content makes the repository huge and slow to clone, and should not be done
  • storing and serving images could cost us, which we want to avoid
To solve those problems, the following alternatives appear to be stable enough and should be used decreasing preference:
  • for all images, use the separate GitHub repository: github.com/cirosantilli/media
    This way, the entire website is relies on a single third party: GitHub, so we have a simple single point of failure.
    We are at the mercy of GitHub's 1GB size policy: help.github.com/en/articles/what-is-my-disk-quota, but it will take a while to hit that.
    GitLab however has a 10Gb maximum size: about.gitlab.com/2015/04/08/gitlab-dot-com-storage-limit-raised-to-10gb-per-repo/ so we could move there is we ever blow up 1Gb on GitHub.
    Both GitLab and GitHub allow uploading files through the web UI, so downloading a large repo is never needed to contribute.
    GitHub does not serve videos like it does images however as of 2019.
  • Wikimedia Commons for videos if the following conditions are met:
    • in scope: "educational material in a broad sense", but not e.g. "Private image collections, e.g. private party photos, photos of yourself and your friends, your collection of holiday snaps and so on.". I don't think they will be too picky even with low quality photos.
    • allowed format, e.g. images or videos, but not ZIPs
    • allowed license: CC BY SA, but no fair use
    Since Wikimedia Commons has a higher level of curation and is an educational not-for-profit, it is the method most likely to remain available for the longest time.
    For this reason, we highly recommend uploading any acceptable files there as well as an additional backup.
    The downside is that its tooling is not as good, e.g. there are a bunch of messy unofficial tools for batch operations, and upload takes more effort.
    Another downside of Wikimedia Commons is that while we can choose the basename of files, it also adds some extra SHA crap to the beginning of URLs, making them harder to predict.
    Another serious downside is that they randomly rename images without redirects... e.g. they renamed upload.wikimedia.org/wikipedia/en/0/03/STJ_SVG_file.svg to upload.wikimedia.org/wikipedia/commons/8/81/Superconducting_tunnel_junction.svg
    Another "downside" is that they are extremely strict about copyright compliance. This is good because you can be pretty sure that they are correct in general, but it also means that they are very conservative, and delete things where fair use would be OK. And if those fair uses have no Wikipedia page, they won't show up anywhere.
  • archive.org for anything else, e.g. videos that Wikimedia commons does not accept.
    All content will be tracked under the cirosantilli collection: archive.org/details/cirosantilli
    archive.org has a very convenient upload and lax requirements. The generated URLs are predictable (single SHA prefix for the entire collection).
    Never trust a website that is not on GitHub Pages, for-profit companies will take down everything immediately as soon as it stops making them money.
    Every external link to non-GitHub pages must be archived. And GitHub links must be forked.
    We should also backup images that Wikimedia Commons does not accept here in addition to the github.com/cirosantilli/media repository.
The following do have direct links:
Read the full article