Felix Becker /

The tool to build LLM tools

Google was not the first search company, but it was the first to introduce page rank. This made all the difference. Since the advent of GPT-4, I've been anticipating the emergence of a new Google. However, I'm skeptical that Perplexity or You.com represent the next evolution of search.

The new internet search engine will transform websites into services, akin to LLM tools. Consider the following scenarios:

  1. You want to book a restaurant in city X. The process might involve using Google Maps to generate a list, Yelp to check menus, parsing the menu PDFs, and then utilizing several booking SaaS platforms to confirm availability.
  2. You could use LinkedIn and Crunchbase to construct complex nested queries to identify potential contacts.

These use cases involve multi-step processes and require deep integrations into services. The big question is, who will build these integrations? Unlike the 'Zapierfication' of the internet, many of these services are resistant to programmatic data access.

Websites can be divided into two types: server-side rendered content and client-side rendered content. Although they may appear identical to the end-user, the latter relies on private APIs to interact with the backend.

There are three approaches to convert websites into LLM digestable services/ tools:

  1. Vision-based action models, such as Adept.
  2. Scrapers combined with HTML minifiers, like TinyFish.
  3. API tools, for example, Multi On and Anon.com.

I believe that information retrieval will involve a combination of the first two approaches, but to truly advance web interactivity, we must address the third challenge.

There are two types of tasks to consider:

  • Public APIs: These will likely be addressed with OpenAPI configurations and tools within the next two years.
  • Private APIs: These present several challenges, including lack of documentation, rapid changes leading to brittleness, rate limiting, and authentication issues like cookie handling and rotation.

I am curious about who is developing the tools to construct LLM tools from private APIs. Anon.com is a promising first step, but the question remains: can this process be automated?