fishcake on Nostr: My 2024 review and high-level report. Let's start with some stats for the ...
My 2024 review and high-level report. Let's start with some stats for the nostr.build:
- 720 Million media requests served
- 650 TiB of bandwidth consumed
- 8 Million unique visitors
- 85% average cache hit rate
- Improved median CPU time per request from 200 down to 2 milliseconds per request
- 20 requests per second are served on average
- 99.994% uptime for the hosted media
- 98% uptime for the website and uploading servers
- 2.5K daily free uploads on average
Some improvements for the free uploads:
- Increased free upload size from 10MiB to 50MiB
- Increased image resolution from 1980px to 3840px
- Increased per server RAM from 1GiB to 4GiB
- Added full mirror of all uploaded media to the additional vendor (backup)
- Implemented moderation headers (can be used to hide NSFW content if desired)
Now some of the stats of my dev work for the nostr.build:
- 1,333 git commits
- 42 repositories with my contributions
- 10k+ lines of code written, or changed
Some features and completed projects:
- Rewritten nostr.build account page (AI, stats, nostr share, drag-n-drop, variaty of sizes, poster for video, etc.)
- Added AI Studio, onboarded multiple models from SD and Flux.
- Added media statistics
- Rewritten media serving front-end (dropping CPU utilization by 100x) and improving TTFB as well as cache hit rate (from 50%+ to 85%+ on average)
- Implemented variaty of the spam, and other illegal media upload prevention mechanisms (automated AI media scanner, PhotoDNA scanner, etc.)
- Implemented AI Moderation API that is shareable with other nostr media hosts (e.g., nostrcheck.me)
- Implemented upload of pdf, zip, etc., with full virus scanning.
- Various improvements to the PHP code, bug fixes and optimizations.
- Implemented e.nostr.build to allow easy embedding and sharing of the video/audio files for the account holders
- Referal links and account credits
- Onboarded more nostr clients for uploads (e.g., YakiHonne)
Some of the prototyping I've done for nostr.build:
- Semantic search of the media (ML based), multilingual
- Basic search for media using AI Vision
- Video transcoding using serverless and server-based setups
- Video transcoding to DASH/HLS (CMAF)
- Multi-party audio/video chat
Some of the in-flight projects that I'll have to complete in 2025:
- Seamless implementation of the Blossom protocol (70% done)
- Rewrite of the post-upload scanning to utilize workflows (50% done)
- New video upload and transcoding experience, HLS, DASH, VBR video/audio (10% done)
- New NoSQL based data-plane backend with the new API (50% done)
- Full rewrite of the Accounts page (5% done, blocked by new API and data-plane)
- Additional models for the AI Studio (90% done)
- <Secret projects with new experiences and features> (25% done)
I am sure I am forgetting a bunch, and this is only done from my memory.
- 720 Million media requests served
- 650 TiB of bandwidth consumed
- 8 Million unique visitors
- 85% average cache hit rate
- Improved median CPU time per request from 200 down to 2 milliseconds per request
- 20 requests per second are served on average
- 99.994% uptime for the hosted media
- 98% uptime for the website and uploading servers
- 2.5K daily free uploads on average
Some improvements for the free uploads:
- Increased free upload size from 10MiB to 50MiB
- Increased image resolution from 1980px to 3840px
- Increased per server RAM from 1GiB to 4GiB
- Added full mirror of all uploaded media to the additional vendor (backup)
- Implemented moderation headers (can be used to hide NSFW content if desired)
Now some of the stats of my dev work for the nostr.build:
- 1,333 git commits
- 42 repositories with my contributions
- 10k+ lines of code written, or changed
Some features and completed projects:
- Rewritten nostr.build account page (AI, stats, nostr share, drag-n-drop, variaty of sizes, poster for video, etc.)
- Added AI Studio, onboarded multiple models from SD and Flux.
- Added media statistics
- Rewritten media serving front-end (dropping CPU utilization by 100x) and improving TTFB as well as cache hit rate (from 50%+ to 85%+ on average)
- Implemented variaty of the spam, and other illegal media upload prevention mechanisms (automated AI media scanner, PhotoDNA scanner, etc.)
- Implemented AI Moderation API that is shareable with other nostr media hosts (e.g., nostrcheck.me)
- Implemented upload of pdf, zip, etc., with full virus scanning.
- Various improvements to the PHP code, bug fixes and optimizations.
- Implemented e.nostr.build to allow easy embedding and sharing of the video/audio files for the account holders
- Referal links and account credits
- Onboarded more nostr clients for uploads (e.g., YakiHonne)
Some of the prototyping I've done for nostr.build:
- Semantic search of the media (ML based), multilingual
- Basic search for media using AI Vision
- Video transcoding using serverless and server-based setups
- Video transcoding to DASH/HLS (CMAF)
- Multi-party audio/video chat
Some of the in-flight projects that I'll have to complete in 2025:
- Seamless implementation of the Blossom protocol (70% done)
- Rewrite of the post-upload scanning to utilize workflows (50% done)
- New video upload and transcoding experience, HLS, DASH, VBR video/audio (10% done)
- New NoSQL based data-plane backend with the new API (50% done)
- Full rewrite of the Accounts page (5% done, blocked by new API and data-plane)
- Additional models for the AI Studio (90% done)
- <Secret projects with new experiences and features> (25% done)
I am sure I am forgetting a bunch, and this is only done from my memory.