jb55 on Nostr: “the data structures make a difference. LMDB relies solely on OS page caches ...
“the data structures make a difference. LMDB relies solely on OS page caches because only the underlying virtual memory manager really knows what's going on in the system, and also, B+trees are inherently the most cache friendly. Your app-level data access patterns just aren't going to outperform them. The tree structure inherently optimizes for LRU cache mgmt; the tree root is always hit on every access, so it is always most-often-and-recently-used. As accesses navigate down the tree to the leaf nodes, each node is successively less frequently used. The net result is that the root and interior branches naturally stay hot in a cache, and the leaf nodes naturally age out fastest, resulting in optimal cache hit rates for any access pattern.”
Howard Chu @ Symas (npub1tt4…x9ud) #lmdb #nostrdb #strfry
Published at
2023-08-27 22:49:49Event JSON
{
"id": "4c0e6ff0b5ba666a77e34306daf3d104ca201c2e7d4f08e14592a647a2953b11",
"pubkey": "32e1827635450ebb3c5a7d12c1f8e7b2b514439ac10a67eef3d9fd9c5c68e245",
"created_at": 1693176589,
"kind": 1,
"tags": [
[
"p",
"5aeb250b3075a12bd05e16c8a3c40da91a553fa92164a39915a3a0615fe51864"
],
[
"t",
"lmdb"
],
[
"t",
"nostrdb"
],
[
"t",
"strfry"
]
],
"content": "“the data structures make a difference. LMDB relies solely on OS page caches because only the underlying virtual memory manager really knows what's going on in the system, and also, B+trees are inherently the most cache friendly. Your app-level data access patterns just aren't going to outperform them. The tree structure inherently optimizes for LRU cache mgmt; the tree root is always hit on every access, so it is always most-often-and-recently-used. As accesses navigate down the tree to the leaf nodes, each node is successively less frequently used. The net result is that the root and interior branches naturally stay hot in a cache, and the leaf nodes naturally age out fastest, resulting in optimal cache hit rates for any access pattern.”\n\nnostr:npub1tt4j2zeswksjh5z7zmy283qd4yd920afy9j28xg45wsxzhl9rpjqrex9ud #lmdb #nostrdb #strfry",
"sig": "7d75dd6663af8823e5a7b3f200effb0811f72fa07fb1dd7322326863499f4a7b05c188c80acc99d4a0c1958e06e41b25bf1dac06fa411e70702c2b2945b60bf2"
}