Running Bitcoin Core v0.7 and Earlier
The following is a result of my research into the historical syncing performance of all Bitcoin Core releases and the subsequent challenges of building really old releases for which it's difficult to find precompiled binaries.
Note that if you're planning on syncing these versions of Bitcoin Core, you're going to have a bad time. I'm not the first person to look into this; during my research I found that Sjors Provoost ran similar experiments in 2017 and wrote about it.
Investigating Syncing Failures
None of the versions prior to v0.8.0 will be able to complete a sync to chain tip for various reasons.
v0.3: stopped syncing after block 124,275 with this error:
ERROR: ConnectInputs() : fb0a1d8d34 VerifySignature failed InvalidChainFound: invalid block=0000000000004939267f height=124276 work=6613870563198902508
At first glance this doesn't look like a particularly weird transaction. However, if we start looking into the history of whomever was spending funds locked by that Bitcoin address, we can see that they crafted several odd transactions with hundreds of "0 value" outputs. It wouldn't be a stretch to assume this was a fairly technical person who was trying to break the network.
Eventually, something that jumped out at me was the size of this input's signature. 75 bytes is on the extreme upper bound whereas most P2PKH spends are 71 - 73 bytes.
You can read an excellent deep dive into bitcoin transaction signature history here; it notes that if you try to validate the early chain with a newer version of OpenSSL (1.0.0p / 1.0.1k) then you'll get failures because the DER validation is stricter and will reject certain types of encodings.
The workaround is to either build Bitcoin Core with an older version of OpenSSL or to manually apply this code patch before building. I was confused at first because gitian builder uses Ubuntu 10.04 virtual machines as build environments, which I figured should be using the older (compatible) version of OpenSSL... however, Andrew Chow pointed out that they backported this OpenSSL patch in 2015.
v0.4 and v0.5 both stopped syncing after block 258,354 with error
EXCEPTION: 11DbException Db::put: Cannot allocate memory bitcoin in ProcessMessage() ProcessMessage(block, 901212 bytes) FAILED received block 0000000000000023e872 REORGANIZE
This is notable because it looks like block 258355 is the first block that is 900KB in size; before this height it looks like almost all mined blocks were hitting the default "soft limit" of 250KB at the time.
v0.6 stopped syncing after block 364,670 with error
EXCEPTION: 11DbException Db::put: Cannot allocate memory bitcoin in ProcessMessages() ProcessMessage(block, 999787 bytes) FAILED received block 000000000000000001d3
This is similarly notable because it looks like block 364,671 is the first block that hit 1MB in size.
v0.7 failed at the same block but with a different error logged.
received block 00000000000000000221 ERROR: ConnectBlock() : UpdateTxIndex failed InvalidChainFound: invalid block=00000000000000000221 height=364671 ERROR: SetBestChain() : SetBestChainInner failed ERROR: AcceptBlock() : AddToBlockIndex failed ERROR: ProcessBlock() : AcceptBlock FAILED
I suspect that the v0.4 - v0.7 stalls are all due to the same issue, but what is it? Is this a result of the "BDB locks issue" that caused the unintentional chain split in 2013? I tried creating a file ~/.bitcoin/DB_CONFIG as per these instructions:
set_lg_dir database
set_lk_max_locks 537000But each node version still stalls at the same block height as before. It turns out that you ALSO need to configure set_lk_max_objects AND the original recommended value of 537000 isn't high enough if you want to get all the way to block 700,000. The following ~/.bitcoin/DB_CONFIG values worked for me:
set_lg_dir database set_lk_max_locks 1000000 set_lk_max_objects 1000000
Performance Results
You can find my raw syncing data here.
Though you can hardly even see the data for v0.3 since it's nearly the same as v0.4 and v0.5 and ends abruptly at height 124,000. Here's a log scale version to zoom in a bit more.
It is noteworthy that there's an inflection point near block 190,000 at which point v0.4 starts performing better than v0.5. My best guess is that this is a result of checkpoints. Bitcoin 0.3.2 introduced a mechanism called checkpoints to prevent denial-of-service attacks during initial block download by ensuring that new full nodes couldn’t be tricked into spending excessive amounts of effort validating alternative blockchains that were different than the best-known chain at certain points in time.
Bitcoin 0.5.0 built upon those checkpoints to speed up syncing by skipping verification of signatures in blocks that were earlier in the block chain than the most recent checkpoint. "But wait," you might be thinking, "I thought you forced the nodes to validate all historical signatures by setting assumevalid=0 in the config?" I sure do, but that setting doesn't have any effect for Bitcoin Core versions before v0.14 when it was first introduced.
As such, I'd say that v0.5 is generally slower than v0.4 and the early syncing performance is actually cheating / an unfair comparison.
Legacy vs Modern Performance
How much more performant is the latest v22 release compared to these old releases?
Syncing to block 124,000:
- v22 is 17X faster than v0.3
These blocks are all empty, which is why the performance difference is far less than for nodes that sync more of the blockchain.
Syncing to block 258,000:
- v22 is 77X faster than v0.4
- v22 is 83X faster than v0.5
Syncing to block 364,000:
- v22 is 40X faster than v0.6
- v22 is 38X faster than v0.7
Forward Compatibility is Hard
You may often hear claims that one can run the very first version of the Bitcoin software and it would be compatible with the current network. Naturally, the true answer is much more complicated and nuanced. In order to get really old versions of Bitcoin software to sync to chain tip, some changes are needed. It's additionally tricky because we can see that not all consensus critical code is necessarily code written by Bitcoin developers - sometimes it's in third party libraries that have changed over time, thus making the software build process itself a potential point of consensus failure!
