ADIL 🦂 丰 ₿ ⚡ on Nostr: How to compile website content for LLMs study? 1️⃣ llmstxt - will compile any ...
How to compile website content for LLMs study?
1️⃣ llmstxt - will compile any website into a single text file that can be passed to any LLM. https://llmstxt.firecrawl.dev/
2️⃣ Or get the copied text of any web page by entering https://r.jina.ai at the beginning of the web link
These two methods will help you with web content. Instead of complex and cluttered HTML code, it takes only the necessary text from the site, removing everything unnecessary: markup, scripts and styles, which simplifies AI training, improves the quality of answers.
1️⃣ llmstxt - will compile any website into a single text file that can be passed to any LLM. https://llmstxt.firecrawl.dev/
2️⃣ Or get the copied text of any web page by entering https://r.jina.ai at the beginning of the web link
These two methods will help you with web content. Instead of complex and cluttered HTML code, it takes only the necessary text from the site, removing everything unnecessary: markup, scripts and styles, which simplifies AI training, improves the quality of answers.