Simon Willison on Nostr: I built a little tool for playing with the GPT-3 tokenizer using an Observable ...

I built a little tool for playing with the GPT-3 tokenizer using an Observable notebook

You can convert a string of text into numeric tokens, convert tokens back to text and search the full list of tokens

https://observablehq.com/@simonw/gpt-3-token-encoder-decoder