Ciro Santilli @cirosantilli 37

Setting: you are sending bits through a communication channel, each bit has a random probability of getting flipped, and so you use some error correction code to achieve some minimal error, at the expense of longer messages.

This theorem sets an upper bound on how efficient you can be in your encoding, for any encoding.

The next big question, which the theorem does not cover is how to construct codes that reach or approach the limit. Important such codes include:

But besides this, there is also the practical consideration of if you can encode/decode fast enough to keep up with the coded bandwidth given your hardware capabilities.

news.mit.edu/2010/gallager-codes-0121 explains how turbo codes were first reached without a very good mathematical proof behind them, but were still revolutionary in experimental performance, e.g. turbo codes were used in 3G/4G.

But this motivated researchers to find other such algorithms that they would be able to prove things about, and so they rediscovered the much earlier low-density parity-check code, which had been published in the 60's but was forgotten, partially because it was computationally expensive.

 Read the full article

Heat equation by

Ciro Santilli 37  Updated 2025-06-17  +Created 1970-01-01

 View more

Besides being useful in engineering, it was very important historically from a "development of mathematics point of view", e.g. it was the initial motivation for the Fourier series.

Some interesting properties:

TODO confirm: for a fixed boundary condition that does not depend on time, the solutions always approaches one specific equilibrium function.
This is in contrast notably with the wave equation, which can oscillate forever.
TODO: for a given point, can the temperature go down and then up, or is it always monotonic with time?
information propagates instantly to infinitely far. Again in contrast to the wave equation, where information propagates at wave speed.

Sample numerical solutions:

with FreeFem:
- heat-dirichlet.1d.freefem
- heat-dirichlet-2d-freefem

 Read the full article

Isometry group by

Ciro Santilli 37  Updated 2025-06-17  +Created 1970-01-01

 View more

The group of all transformations that preserve some bilinear form, notable examples:

orthogonal group preserves the inner product
unitary group preserves a Hermitian form
Lorentz group preserves the Minkowski inner product

 Read the full article

The direct product of two cyclic groups of coprime order is another cyclic group by

Ciro Santilli 37  Updated 2025-06-17  +Created 1970-01-01

 View more

You just map the value (1, 1)

C_{m} \times C_{n}

to the value 1 of

C_{mn}

, and it works out. E.g. for

C_{2} \times C_{3}

, the group generated by of (1, 1) is:

0 = (0, 0)
1 = (1, 1)
2 = (0, 2)
3 = (1, 0)
4 = (0, 1)
5 = (1, 2)
6 = (0, 0) = 0

 Read the full article

Klein four-group by

Ciro Santilli 37  Updated 2025-06-17  +Created 1970-01-01

 View more

C_{2} \times C_{2}

 Read the full article

The lower level you go into a computer, the harder it is to observe things by

Ciro Santilli 37  Updated 2025-06-17  +Created 1970-01-01

 View more

This is a general principle of software/hardware design that Ciro feels holds wide applicability.

The most extreme case of this is of course the integrated circuit itself, in which it is essentially impossible (?) to observe the specific value of some indidual wire at some point.

Somewhat on the other extreme, we have high level programming languages running on top of an operating system: at this point, you can just GDB step debug your program, print the value of any variable/memory location, and fully understand anything that you want. Provided that you manage to easily reach that point of interest.

And for anything in between we have various intermediate levels of complication. The most notable perhaps being developing the operating system itself. At this level, you can't so easily step debug (although techniques do exist). For early boot or bootloaders for example, you might want to use JTAG for example on real hardware.

In parallel to this, there is also another very important pair of closely linked tradeoffs:

the lower level at which something is implemented, the faster it runs
emulation gives you observability back, at the cost of slower runtime

Emulation also has another potential downside: unless you are very careful at implementing things correctly, your model might not be representative of the real thing. Also, there may be important tradeoffs between how much the model looks like the real thing, and how fast it runs. For example, QEMU's use of binary translation allows it to run orders of magnitude faster than gem5. However, you are unable to make any predictions about system performance with QEMU, since you are not modelling key elements like the cache or CPU pipeline.

Instrumentation is another technique that has can be considered to achieve greater observability.

 Read the full article

ENIAC by