Hailey on Nostr: I remain unconvinced that docker layers are a good abstraction. Building an image ...
I remain unconvinced that docker layers are a good abstraction.
Building an image that uses another image as a base? Sure, that makes sense, but keeping all those layers around and exposing them to the user as a domain concept does not.
There's an ongoing runtime perf cost to supporting them, and they're just not that effective when it comes to deduplicating image contents. Consider your `bundle install` layer, yeah you can reuse it between app versions that don't bump any gems, but the moment you bump even one gem, you're paying hundreds of MB if not close to a GB for that bump.
I keep thinking about this paper (https://arxiv.org/pdf/2305.13162.pdf) out of the AWS Lambda team where they mention flattening layers into a single ext4 with a deterministic extent layout. Deduplicating 512 KiB chunks of this image turns out to be a lot more effective for them than layer-based deduplication ever was, plus it enables image lazy loading in a way that layers simply can't achieve.
Building an image that uses another image as a base? Sure, that makes sense, but keeping all those layers around and exposing them to the user as a domain concept does not.
There's an ongoing runtime perf cost to supporting them, and they're just not that effective when it comes to deduplicating image contents. Consider your `bundle install` layer, yeah you can reuse it between app versions that don't bump any gems, but the moment you bump even one gem, you're paying hundreds of MB if not close to a GB for that bump.
I keep thinking about this paper (https://arxiv.org/pdf/2305.13162.pdf) out of the AWS Lambda team where they mention flattening layers into a single ext4 with a deterministic extent layout. Deduplicating 512 KiB chunks of this image turns out to be a lot more effective for them than layer-based deduplication ever was, plus it enables image lazy loading in a way that layers simply can't achieve.