david on Nostr: Best way to load a lot of data into neo4j is with a csv file. So alongside LMDB ...
Best way to load a lot of data into neo4j is with a csv file. So alongside LMDB perhaps I should maintain blocks of csv files with eventId,created_at,author,kind as the header.
I’ll maintain a websocket connection to strfry and load new events into neo4j one at a time as they come in. But events will get missed, so I’ll use systemd to set up a service which will routinely take the latest batch of events as csv and make sure they’ve been loaded into neo4j. Older cvs files can get deleted once I’ve ensured they’re in neo4j.
Then have separate scripts to tend to the relationships: eventA is a reaction to eventB, eventC is a reply to eventD, etc.
I’ll maintain a websocket connection to strfry and load new events into neo4j one at a time as they come in. But events will get missed, so I’ll use systemd to set up a service which will routinely take the latest batch of events as csv and make sure they’ve been loaded into neo4j. Older cvs files can get deleted once I’ve ensured they’re in neo4j.
Then have separate scripts to tend to the relationships: eventA is a reaction to eventB, eventC is a reply to eventD, etc.