jsm on Nostr: Interesting ideas here from Meta. Tokenization has a lot of problems with how we ...

Interesting ideas here from Meta. Tokenization has a lot of problems with how we represent text. You can get weird glitches etc. Working with “dynamic patches” could allow more reasoning where necessary from the model and avoid tokenization glitches and oddities.

https://ai.meta.com/research/publications/byte-latent-transformer-patches-scale-better-than-tokens/