Metal Bitcoin Seed Storage Stress Test (Round III)

Note: This is a recurring series; here you can read the full results of the entire series here.

For this round of testing I’m repeating the same procedures but with the addition of another realistic stressor - accelerated cooling after being heated. In a house fire, it's unlikely that the fire will burn itself out naturally - it will most likely be put out by firefighters who dump a ton of water on it. As such, at the end of the heat test I submerge the device into a bucket of water.

The Contenders

Ballet Crypto: To be clear, this device is not advertised as being resistant to disasters, I just threw it in the mix because I had several laying around and was curious. Ballet explicitly says to "avoid common hazards such as fire and corrosive chemicals, or anything else that can cause damage to the top sticker or the wallet itself." My impression is that it's designed to be "offline storage" for every day use, not an indestructible backup. It's also worth noting that because the private keys are generated by the company that creates the device, you have to trust that the company is not keeping a copy and that no supply chain attacks have occurred.

BitHD Frozen Armor: Claims "Indestructible Robustness" and fireproof:
304 stainless steel: 2624℉.
Aviation aluminum alloy: 1220℉.
Claims "Corrosion Resistant" - Saline-alkaline tolerant; acid & rust proof.

Bunkeroid: Claims "BUNKEROID can not be damaged by a common accident or natural disaster. Stainless steel has more durability than paper or any other deposit."

Bunkeroid is unique in that you tell them which seed words you want on your tiles (plus a bunch of other words you won't use in order to reduce your trust in them) and they have a special "no data stored" system.

When the transaction is confirmed, we use the TXID of your transaction to create a  'NoDataStored - system ID' number. We use the 'HMAC' function to create a code, an encrypted text, which is created from the text of your payment combined with our secret password. This HMAC is subsequently engraved on the Bunkeroid. This code gives us an indication that you are our customer without knowing who you are or where you are from. With the 'NoDataStored - system ID' we can easily identify you.

Cobo Tablet: Crafted from 304-grade stainless steel, the Cobo Tablet has a fireproof threshold ranging from 1399 to 1455 ℃ / 2550 to 2651 °F.

Cryptosteel Capsule: Claims to be extremely durable - fireproof up to 1400C/2500F. Shockproof. Waterproof. Stainless.

Crypto Keys: Doesn't appear to make any claims about its durability.

Cypherwheel: I received a pre-production version to test. Claims to be:

• Fireproof up to 2000°F
• Rust proof
• Waterproof

Ellipal Metal Mnemonic: Claims to be IP65 rated meaning it is dust proof and water jet proof. Fire proof - Water proof - Insect proof - for life time keeping.

Hodlinox: Claims to be fire proof, water proof, corrosion resistant and unaffected by various acids: this 316 stainless steel plate is virtually indestructible.

Quadrat Register: Made of stainless 304 grade german steel; as far as claims I only see the web site state that it's secure against x-ray spying.

Safe Seed: Makes no durability claims.

The Steely: Claims to be constructed from 304 Stainless Steel. Fireproof up to 1200 ºC / 2200 ºF, Waterproof. Corrosion Resistant & Rustproof.

Untested Devices

Cypherplate: Not shipping yet.

Cryptoetch: Learned about it after I had completed this round of tests. Queued up for next round.

Ledger Eternity: not released yet; few details are known other than this patent filing. Asked a Ledger representative about it at a conference booth and they wouldn’t disclose any information.

Device Setup / Assembly

Ballet Crypto: No setup required; the private key is already printed onto the device.

BitHD Frozen Armor: This has the tiniest screws I've seen yet - it's quite frustrating to handle them. The tiles are nice and thick though, I found it easy to push them out of the template in which they are shipped.

Bunkeroid: Easy enough to assemble since the tiles are pre-engraved with the full words.

Cobo Tablet: While I like that it's easier to insert the letters due to how the Cobo Tablet opens, I'm not a fan of using small screws as they could create weak points. The screws are so delicate that I'm able to strip the heads by hand with the tiny screwdriver that is provided. I do think that the packaging of the letter tiles is a nice touch (sandwiched between two sticky plastic sheets) and keeps them organized without being a pain to remove like most full metal sheets of tiles.

Cryptosteel Capsule: This was pretty easy to assemble other than the final retention pin, though unlike pretty much every other style of device it's fairly annoying to read back due to the orientation and spacing of the tiles. I also noted that it comes with a very large assortment of tiles, which makes it more versatile for potential use cases.

Crypto Keys: Pretty simple - a dozen pieces of steel, but from experience running these tests simple is good. Major bonus points that they ship with a 3D printed stamp holder. FYI it's not advisable to try to stamp the pieces while they're still on the key ring; they need to be completely flat on the stamping surface.

Cypherwheel: This is a unique device for sure; you can see a video of the setup process here. While you have to slide the tiles along rails, the rail system is very different from the cryptosteel style. The tolerances on Cypherwheel are much lower, which is a good thing in my opinion as it means tiles are less likely to move around. The downside is that it takes a lot more effort to get them into place. Expect 1 to 2 minutes per 4 letter word, meaning it can easily take you an hour to fill the whole wheel. One neat feature is that each word has its own one-way "lock" that can't be undone once you've pushed it into place. One thing I don't think is necessary is the supplied set of tweezers, as they are too large to help slide the tiles to the left - I very quickly switched to using the center punch to push the tiles.

Ellipal Metal Mnemonic: This is the first metal device I've seen with a hinge that flips open like a briefcase. Once again, I'm not a fan of the tiny screws. I also found the tiles frustrating because they take some force to pop out of the sheet and I had several fly across the room.

Hodlinox: Setup was straightforward; the one annoyance I had was that the back plate is not a flat surface due to the threading for the screw, thus it was harder to keep secure when stamping letters.

Quadrat Register: This one's an oddball, as it came with no actual assembly instructions. There were several pages of instructions about setting up the "two factor" encoding scheme, but none about actually putting the pieces of the device together. Thankfully it was fairly intuitive; I figured out pretty quickly that each tile was notched so that you could keep them all on the rail in the correct orientation. However, I was confused as to why there were only 6 spacer tiles; I guess you could just use a blank (unpunched) tile as a spacer. But then as I neared the end I realized that if you want to store a 24 word seed phrase with the first 4 characters of each word as a separate tile, you won't have room to use spacers and you won't be able to read the tiles without removing them from the rail. However, this problem could be solved by just assigning an entire BIP39 word to a given tile encoding. Given such a setup you could potentially store 4 complete 24 word seed phrases in this one device.

Safe Seed: Not much to say; it's a copper sheet. This was the cheapest of all the devices at $16 so it will be interesting to see how it performs.

The Steely: I appreciate that the tiles come already separated into individual plastic containers. Assembly was straightforward and required no tools. The sliding shutter feature is unique though we'll see if it's actually functional.

Heat Tests

I applied ~2000°F heat for 10 minutes or until failure occurred, whichever came first. Then the device was submerged in a bucket of water to simulate a firefighter putting out a structure fire.

Ballet Crypto: As expected, all the important info on the paper burned off.

BitHD Frozen Armor: Seemed to be doing fine for 6 minutes and then it started collapsing so I immersed it in water. Catastrophic failure. Upon looking back at their web site, apparently only the tiles are stainless steel - the casing is aluminum, which melts at common house fire temperatures!

Bunkeroid: The test seemed to be going fine for the first 5 minutes, then one end of the device collapsed and a bunch of slag fell to the ground. Upon further inspection, the interior of the device had also collapsed. This is a catastrophic failure. Bunkeroid's web site claims that "every part of Bunkeroid is made from stainless steel" but I'm quite sure that the main portion of the case is actually constructed of aluminum based upon how it melted. To my knowledge there are no stainless steel alloys that melt below 2400°F, plus the main body of the device did not change color after it was heated as steel tends to do.

After doing some digging, it looks like this is one of the designers saying that the body is, in fact, made of aluminum.

Cobo Tablet: From viewing it externally it seemed like it held up pretty well, however, upon opening the device it was clear that the rails warped and tiles were moved around. Chalking this up as a catastrophic loss.

I spoke about this with the head of Hardware at Cobo and he told me that they discovered this flaw after running their own experiment and that they are improving the design by adding two more screws in the middle of the plate to prevent the deforming. If they're happy with the performance of this new version, hopefully I'll re-test Cobo in Round IV of this series.

Cryptosteel Capsule: Performed admirably - I didn't even have any trouble reopening the capsule by hand.

Crypto Keys: No data loss.

Cypherwheel: No visible damage, however the covers are stuck and don't rotate, making it impossible to read the seed. Cyphersafe told me ahead of time that if I encountered problems with the cover, it would be fairly easy to pry it off. I'm going to wait until after the other tests are completed to do that, as I have no reason to believe that the seed phrase has been compromised. When shaking the device I couldn't hear any tiles rattling around, so I don't think they came loose.

Ellipal Metal Mnemonic: Seemed to be doing fine but then suddenly turned to slag after 6 minutes, much like the BitHD and Bunkeroid. One thing I've noticed is that the devices that don't change color and turn red hot appear to be the ones that fail suddenly. Perhaps because they're absorbing all of the heat rather than dissipating it.

Hodlinox: It was slightly challenging to reopen the Hodlinox because the plates bowed slightly (about 1 mm) such that the clamp on the bottom plate was holding the top plate in place. Just had to pry the pieces apart a bit and hammer the top plate off to the side.

We can see that the etched words are still legible, though faint, while the stamped words are quite easy to read. Note that I didn't fill the whole device with words in order to minimize my time spent doing so.

Quadrat Register: The tube opened effortlessly and none of the tiles were out of place or had lost any data.

Safe Seed: While there was some discoloration of the device, both engraved and stamped words are still legible. Note that I didn't fill the whole device with words in order to minimize my time spent doing so.

The Steely:  It took less than 2 minutes of applied heat before a dozen tiles fell out. The fancy looking shutters on the front didn't help with the structural integrity; the cover and rails expanded sufficiently that the tiles fell right through the middle of the device. Upon being immersed in the water bucket, another 3 dozen tiles fell out. Catastrophic failure.

Corrosion Tests

Normally I submerge the devices in muriatic acid for 12 hours and them pull them out once the bubbling stops. However, this time the acid had a MUCH stronger reaction than I'm used to seeing - it was billowing smoke for a good 20 minutes. I checked back after 3 hours and it looked like the reactions were over, so I pulled the devices out. This was the first time that the reactions ceased because the acid got completely used up!

Ballet Crypto: The rest of the info that was etched onto the card has disappeared.

BitHD Frozen Armor:  Still sizzling - I'm sure that if the acid hadn't been used up, it would have completely disintegrated.

Bunkeroid: Still sizzling; I'm sure that the rest of the aluminum body would have completely disintegrated if I'd added more acid.

Cobo Tablet: While the acid burned off the printed text on the body of the device, there was no additional data loss from after the heat test.

Cryptosteel Capsule: No acid entered the capsule. Even if it had, I doubt any data loss would have occurred.

Crypto Keys: No data loss.

Cypherwheel: This one yielded another interesting result. You may recall that after the heat test I couldn't rotate the wheels. Well, the acid appears to have dissolved the hub around which they rotated, so both of the wheels came off. Cyphersafe believes that the corrosion resistance of the hub could be improved by switching from 416 to 304 stainless steel.

Oddly enough, while there was some data loss it was somewhat minimal and followed a pattern. About half of the words lost their last letter, but ONLY the last letter. So while this is partial data loss, I wouldn't consider it catastrophic - sufficient data remains that you could write some software to brute force the seed phrase. Cyphersafe also believes that this issue can be resolved by moving the locking divot further away from the tile and switching to 304 stainless steel which will result in the "wall" it creates being thicker and harder to corrode.

Ellipal Metal Mnemonic: There's no photo of this device because there was literally nothing left. It appears that this device reacted the worst to the acid and depleted most of it. Needless to say, catastrophic failure. I've never had a device disintegrate like this before.

Hodlinox: Doesn't appear to have become any less legible in comparison to the heat test.

Quadrat Register:  No acid entered the tube; even if it had I would expect no data loss.

Safe Seed: No data loss.

The Steely:  Hard to tell if there was additional data loss due to acid, but doesn't look like it.

Crush Tests

I use a 20 ton hydraulic press to deform the devices and simulate something like a large building collapsing on top of the device.

Ballet Crypto: If only the private key data was actually stamped onto the metal card, it would be fine. But unfortunately it's printed on paper that's glued to the metal. It's likely that any crushing force would shear and destroy the paper.

BitHD Frozen Armor:  There wasn't enough of the device left to bother crushing it, though we know that aluminum is a fairly malleable metal so I wouldn't have expected it to survive.

Bunkeroid: There wasn't enough of the device left to bother with crushing it, though we know that aluminum is a fairly malleable metal so I wouldn't have expected it to survive.

Cobo Tablet: Like the vast majority of tile-in-rail based devices, it tends not to do well with crushing because the rails get warped, space is created, and tiles fall out.

Cryptosteel Capsule: This was an interesting result. I gave it a 50/50 chance of bending and it held up pretty well to several tons of pressure, but eventually gave in a bit to my 20 ton press. The reason the result is interesting is that I didn't anticipate the consequences of bending the tube. Obviously the data inside is still intact, but how are we going to read it? Because the end of the rod is permanently affixed to the screw-on cap, it's now impossible to unscrew the device.

I don't have the tools on hand to attempt opening the device, but I expect you'd want to somehow cut through the tube just below where the threads of the cap end. But that's not all! Because there is only a retention clip on the opposite end of the rod, you'd then have to be extremely careful while pulling the rod out of the tube to ensure that the tiles don't fall off, causing data loss. Because this is clearly a suboptimal Schrödinger's cat type of situation with the potential for data loss, I feel the need to deduct a few points.

Crypto Keys: It was no surprise that the pieces bent under pressure - you can actually bend them by hand if you try hard enough. However, the steel pieces would need to be completely torn apart in order to cause data loss.

Cypherwheel: I was able to get the wheel to bend slightly, a millimeter or so, but not enough to cause additional data loss.

Ellipal Metal Mnemonic: Nothing was left to crush. Unlikely it would have survived anyway, since aluminum is fairly malleable.

Hodlinox: Performed as expected given that it's just two sheets of steel; no data loss.

Quadrat Register: Performed admirably - the only result from 20 tons of force being applied was a tiny mark you can see in the bottom photo. This is the best result by any device with this crush test in the several years I've been performing them.

Safe Seed: It's hard to tell from the photo but the engraved words are still faintly legible. No data loss.

The Steely: Another catastrophic failure for rail-based devices.

Final Scores

Remember that these aren't highly scientific experiments and my grading system is arbitrary - my goal is to get a sense of which devices are the most robust relative to the rest of the market. If I'm unable to cause any data loss then a device should get an "A" rating, while a device that is quite easy to destroy gets an "F" - anything in between is a grey area.

Final Thoughts

I can't believe that companies are manufacturing devices out of aluminum; I'm sure it's easier to manufacture but you're sacrificing robustness against the most common disaster.

Also, in general the KISS continues to apply. Two of the top performing devices - Crypto Keys and Safe Seed, were also two of the cheapest!

If cost is of no concern, I'd choose the Quadrat Register (365 €) from this batch - the device is built like a tank and I suspect could survive getting run over by one!