npub10pβ¦mn2l2 on Nostr: Weβve got a good #DecodingBitcoin post for you today. The topic? ...
Weβve got a good #DecodingBitcoin post for you today. The topic? ππ§ππ’ππ§π§ππ¬π¬, AKA the order of bytes in a computerβs memory.
When we put it like that it sounds a little boring, but thereβs something interesting, and dare we say πππππ¦πππ, about how bitcoin handles endiannessβ¦
Before we get to that, letβs better understand what endianness is.
Imagine reading directions in different languages: while English is written and read from left to right, Arabic text flows from right to left.
Similarly, computers have two ways to store data:
1. ππ’π -ππ§ππ’ππ§ (BE): Most significant byte first
2. ππ’πππ₯π-ππ§ππ’ππ§ (LE): Least significant byte first
When computers with different byte orders try to communicate, they can misread each other. Itβs like two people reading numbers from opposite directions.
ππ’π -ππ§ππ’ππ§ π¬ππ¨π«ππ¬ ππ‘π π¦π¨π¬π π¬π’π π§π’ππ’πππ§π ππ²ππ ππ’π«π¬π. This is similar to how humans read numbers and Hex in most cases: starting with the most important information.
Suppose we want to store the number 12345678 (hexadecimal: 0x00BC614E) in memory. In big-endian, the bytes are stored in this order:
00 BC 61 4E
Observe that:
- The π¦π¨π¬π π¬π’π π§π’ππ’πππ§π ππ²ππ (00) is stored at the π₯π¨π°ππ¬π π¦ππ¦π¨π«π² ππππ«ππ¬π¬ (00).
- The π₯πππ¬π π¬π’π π§π’ππ’πππ§π ππ²ππ (4E) is stored at the π‘π’π π‘ππ¬π ππππ«ππ¬π¬ (03).
Big-endian is considered more "human-readable" because the data is stored in the order we naturally read it.
ππ’πππ₯π-ππ§ππ’ππ§ π¬ππ¨π«ππ¬ ππ‘π π₯πππ¬π π¬π’π π§π’ππ’πππ§π ππ²ππ ππ’π«π¬π. This might feel counter intuitive to humans but is more efficient for modern processors.
Using the same number 12345678 (0x00BC614E), here's how it looks in little-endian:
4E 61 BC 00
This time, the π₯πππ¬π π¬π’π π§π’ππ’πππ§π ππ²ππ (4E) is stored at the π₯π¨π°ππ¬π π¦ππ¦π¨π«π² ππππ«ππ¬π¬ (00).
The π¦π¨π¬π π¬π’π π§π’ππ’πππ§π ππ²ππ (00) is stored at the π‘π’π π‘ππ¬π ππππ«ππ¬π¬ (03).
This "reversal" of bytes is common in the Bitcoin Core codebase.
In bitcoin, most data like transaction IDs, block headers, and amounts are all in little-endian format or with the bytes reversed.
πππ‘π: πΈππππππππ π ππππ¦ πππππππ π‘π πππ‘πππππ . πΌπ‘ ππ πππ‘ πππππππ‘ π‘π π ππ¦ π βππ β ππ πππ‘π‘ππ-ππππππ. πππππ π‘βπππβπ ππ ππππππππ π‘πππ πππ π‘βππ π€π π ππ¦ βππ¦π‘π π π€πππππβ ππ βπππ£πππ π ππ¦π‘ππ β.
For readability, the bytes are swapped back to the style of big-endian when this data is displayed to humans. A block explorer is one example of where you can see this.
Bitcoin Coreβs JSON-RPC interface was the first time block hashes were printed for human consumption. That was when someone decided to reverse the ordering of hash so that it looked like a human readable integer.
It turns out the real block hash, the actual sha256 value you get if you compute the hash yourself, is:
e4b1d56439d46d9070e58c4368cccc97596fa908daf101000000000000000000
The zeros are actually on the right! At first glance it looks like this is a very large number, but we know the integer value of a block hash actually gets smaller as the difficulty increases.
It's clear that the bytes are reversed and in the style of little-endian. But why? We can thank Satoshi for that. Satoshi decided to interpret the block hash as a little-endian integer. The more zeroes there are on the right side, the smaller the (little-endian) integer.
Since most modern CPUs are little-endian, bitcoin uses it to optimize performance.
However, network protocols typically use big-endian, creating a mismatch π
Big-endian is used for network communication (network byte order). Little-endian is used for bitcoinβs internal storage.
This duality requires developers to frequently, and sometimes frustratingly, convert between the two formats when working with bitcoin data.
Have you been the victim of an endianness oversight when writing bitcoin code? It's a common source of pain for developers new to bitcoin (and even the seasoned ones!)
As covered by the transaction ID example earlier, byte order confusion can be common.
Another gotcha is length specification. When converting to little-endian, always specify the correct byte length:
Hope you learned something new about endianness today. If you enjoyed this, share it with a friend and donβt forget to follow us, nprofile1qyt8wumn8ghj7etyv4hzumn0wd68ytnvv9hxgtcppemhxue69uhkummn9ekx7mp0qqs8sccnw8c4ns0xjen9mgdgks25d46k2kcssk60uxs635zmngxh5ns3quvtq (nprofileβ¦uvtq) !
This material is from Decoding Bitcoin, your go-to resource for understanding #bitcoin, privacy, and decentralization.
Visit https://bitcoindevs.xyz/decoding for the full lesson, and more free, interactive content.
Thanks for reading!
When we put it like that it sounds a little boring, but thereβs something interesting, and dare we say πππππ¦πππ, about how bitcoin handles endiannessβ¦
Before we get to that, letβs better understand what endianness is.
Imagine reading directions in different languages: while English is written and read from left to right, Arabic text flows from right to left.
Similarly, computers have two ways to store data:
1. ππ’π -ππ§ππ’ππ§ (BE): Most significant byte first
2. ππ’πππ₯π-ππ§ππ’ππ§ (LE): Least significant byte first
When computers with different byte orders try to communicate, they can misread each other. Itβs like two people reading numbers from opposite directions.
ππ’π -ππ§ππ’ππ§ π¬ππ¨π«ππ¬ ππ‘π π¦π¨π¬π π¬π’π π§π’ππ’πππ§π ππ²ππ ππ’π«π¬π. This is similar to how humans read numbers and Hex in most cases: starting with the most important information.
Suppose we want to store the number 12345678 (hexadecimal: 0x00BC614E) in memory. In big-endian, the bytes are stored in this order:
00 BC 61 4E
Observe that:
- The π¦π¨π¬π π¬π’π π§π’ππ’πππ§π ππ²ππ (00) is stored at the π₯π¨π°ππ¬π π¦ππ¦π¨π«π² ππππ«ππ¬π¬ (00).
- The π₯πππ¬π π¬π’π π§π’ππ’πππ§π ππ²ππ (4E) is stored at the π‘π’π π‘ππ¬π ππππ«ππ¬π¬ (03).
Big-endian is considered more "human-readable" because the data is stored in the order we naturally read it.
ππ’πππ₯π-ππ§ππ’ππ§ π¬ππ¨π«ππ¬ ππ‘π π₯πππ¬π π¬π’π π§π’ππ’πππ§π ππ²ππ ππ’π«π¬π. This might feel counter intuitive to humans but is more efficient for modern processors.
Using the same number 12345678 (0x00BC614E), here's how it looks in little-endian:
4E 61 BC 00
This time, the π₯πππ¬π π¬π’π π§π’ππ’πππ§π ππ²ππ (4E) is stored at the π₯π¨π°ππ¬π π¦ππ¦π¨π«π² ππππ«ππ¬π¬ (00).
The π¦π¨π¬π π¬π’π π§π’ππ’πππ§π ππ²ππ (00) is stored at the π‘π’π π‘ππ¬π ππππ«ππ¬π¬ (03).
This "reversal" of bytes is common in the Bitcoin Core codebase.
In bitcoin, most data like transaction IDs, block headers, and amounts are all in little-endian format or with the bytes reversed.
πππ‘π: πΈππππππππ π ππππ¦ πππππππ π‘π πππ‘πππππ . πΌπ‘ ππ πππ‘ πππππππ‘ π‘π π ππ¦ π βππ β ππ πππ‘π‘ππ-ππππππ. πππππ π‘βπππβπ ππ ππππππππ π‘πππ πππ π‘βππ π€π π ππ¦ βππ¦π‘π π π€πππππβ ππ βπππ£πππ π ππ¦π‘ππ β.
For readability, the bytes are swapped back to the style of big-endian when this data is displayed to humans. A block explorer is one example of where you can see this.
Bitcoin Coreβs JSON-RPC interface was the first time block hashes were printed for human consumption. That was when someone decided to reverse the ordering of hash so that it looked like a human readable integer.
It turns out the real block hash, the actual sha256 value you get if you compute the hash yourself, is:
e4b1d56439d46d9070e58c4368cccc97596fa908daf101000000000000000000
The zeros are actually on the right! At first glance it looks like this is a very large number, but we know the integer value of a block hash actually gets smaller as the difficulty increases.
It's clear that the bytes are reversed and in the style of little-endian. But why? We can thank Satoshi for that. Satoshi decided to interpret the block hash as a little-endian integer. The more zeroes there are on the right side, the smaller the (little-endian) integer.
Since most modern CPUs are little-endian, bitcoin uses it to optimize performance.
However, network protocols typically use big-endian, creating a mismatch π
Big-endian is used for network communication (network byte order). Little-endian is used for bitcoinβs internal storage.
This duality requires developers to frequently, and sometimes frustratingly, convert between the two formats when working with bitcoin data.
Have you been the victim of an endianness oversight when writing bitcoin code? It's a common source of pain for developers new to bitcoin (and even the seasoned ones!)
As covered by the transaction ID example earlier, byte order confusion can be common.
Another gotcha is length specification. When converting to little-endian, always specify the correct byte length:
Hope you learned something new about endianness today. If you enjoyed this, share it with a friend and donβt forget to follow us, nprofile1qyt8wumn8ghj7etyv4hzumn0wd68ytnvv9hxgtcppemhxue69uhkummn9ekx7mp0qqs8sccnw8c4ns0xjen9mgdgks25d46k2kcssk60uxs635zmngxh5ns3quvtq (nprofileβ¦uvtq) !
This material is from Decoding Bitcoin, your go-to resource for understanding #bitcoin, privacy, and decentralization.
Visit https://bitcoindevs.xyz/decoding for the full lesson, and more free, interactive content.
Thanks for reading!