Suppose we specify:
  • a .dat file
  • the offset in bytes within that file
The question then is, which transaction is encoded at that position of the file?
This would allow us to index inscriptions in the .dat files directly with fast C tools, and then retrive the transaction ID to get cleaner data and metadata.
It should be possible if we managed to take the information from bitcoindev.network/understanding-the-data/ and dump into an indexed SQLite database.
I tried to start things off with LevelDBDumper:
LevelDBDumper -d ~/snap/bitcoin-core/common/.bitcoin/indexes/txindex -f btc.csv -q -o . -t csv
but that consumed all 64 GB of RAM on P51... github.com/mdawsonuk/LevelDBDumper/issues/15
But OK, nevermind that repo, it can be done easily with the LevelDB API of any language: bitcoin.stackexchange.com/questions/121888/what-is-the-data-format-layout-for-txindex-leveldb-values. Just the data seems wrong and we don't know why.