https://polgrabia.me/tags/search/Recent content in Search on Tomasz PółgrabiaHugoen-USSun, 04 Oct 2026 19:00:00 +0200https://polgrabia.me/posts/20261004-how-to-crawl-the-world-do-we-need-google/Sun, 04 Oct 2026 19:00:00 +0200https://polgrabia.me/posts/20261004-how-to-crawl-the-world-do-we-need-google/<h1 id="pre-word">Pre-word</h1> <p>This is a rather tricky question. Or, to put it better - a question worth thinking about: how much do we actually depend on search engines?</p> <p>In the age of AI we are being charged pretty much for every query or MCP call. Every web search made by an agent costs tokens, API credits, or a subscription tier. And it adds up quickly, because agents search a <em>lot</em>. But does it need to be this way?</p>