Mehrad :kdenew: :emacs: 🇪🇺 on Nostr: nprofile1q…l9g62 I work with some terabytes of data, but we are just three people ...
nprofile1qy2hwumn8ghj7un9d3shjtnddaehgu3wwp6kyqpqdc9aqnp985efx2a0k3a6vzjwexcy95shlgnuqhs724nj85z397zq4l9g62 (nprofile…9g62)
I work with some terabytes of data, but we are just three people working on it. So we did it the primitive way: having one person having write access to the raw data, everyone store the intermediary files on their own computer, code and "final" data files and reports are synced with a local storage (Seafile), and in each person's folder. If we need to use the output of someone's analysis as input of another pipeline or code, we store the sha256sum in a log file for posterity.
I work with some terabytes of data, but we are just three people working on it. So we did it the primitive way: having one person having write access to the raw data, everyone store the intermediary files on their own computer, code and "final" data files and reports are synced with a local storage (Seafile), and in each person's folder. If we need to use the output of someone's analysis as input of another pipeline or code, we store the sha256sum in a log file for posterity.