Apparently the Claude Code fetch tool uses an SLM for website summarization to not dump the entire website in context. Fascinating. There's currently no commercial API for this. 100 likes and I'll build one.

Thanks to the resurgence of RL, LLMs are finally able to reliably coordinate tools and reasoning to do high-precision retrieval. Companies like Happenstance, Clado, and Mintlify have already shifted to agentic search, and it's only a matter of time until anything less feels broken to users.

At my last job our documentation was terrible, so we decided to revamp it. This was a process that took dozens of people including multiple PMs, countless meetings, over a thousand hours of engineering/design time and, if I had to estimate, ten thousand people-hours. I begged everyone involved to use @mintlify throughout the process, which included weeks to identify a suitable vendor. Nobody listened to me. After this entire revamp was completed, the docs were: >aesthetically unappealing >difficult to maintain, >had terrible search >and no AI chat >inconsistent branding >no llms.txt >weird flashes when navigating pages >completely unoptimized for SEO >beyond difficult to index by LLMs And worst of all: any large changes still require multiple stakeholders and weeks to implement. To be fair, this company had some of the most technical and sprawling documentation in the world, spanning dozens of products. But switching to Mintlify would likely have provided tens of millions in value over multiple years for such a developer facing product. That's why I'm so bullish on Mintlify. Sure, they make it easy for companies to build out documentation. That's great. But the real value is that as these companies grow and their documentation invariably becomes more sophisticated, they'll need a solid foundation to build on. And there's no more solid foundation for your docs than Mintlify. (This is not a sponsored post. I just use them. They make my life easier as someone who spends a lot of time in docs so they deserve a shoutout. Everything here is true and not exaggerated.)