Kay Ohtie on Nostr: I'm a little out of my element admittedly in this Python I need to write. I'm looking ...

I'm a little out of my element admittedly in this Python I need to write. I'm looking for a way to compare a dataset from a very large (hundreds of thousands, millions even, of records) table against a very large list of entries. This is _not_ something Python is inherently very good at; it's rather slow, and I know modules like Pandas may work, but seem their own beast?

Ultimately I need to exclude one list of IDs from the result of an entire table, basically like, "I already have these, I don't need to duplicate this work".

This is for my emoji re-downloader script. I've got the basic premise for the operation but am struggling because of the large dataset domain