A large language model is a machine learning model that has been trained with text documents (such as books, internet articles, patents) to predict how any given text continues. For example, such a model might predict that a text starting with the words “On a hot summer day I like to eat ” might continue “ice cream.” You can think of a language model as an auto-complete for any text.

The most prominent use case for LLMs is chatting with an AI assistant. For this purpose some language models (including ChatGPT and Claude) have been expressly trained to produce conversational text between a user and a helpful and honest assistant. Given a text such as “User: What is the capital of Norway?” such a model is more likely to continue with “Assistant: Oslo” than with e.g. “Assistant: None of your business.” or “User2: What is the capital of Sweden? User3: What is the capital of Finland?”

Claude is a conversational large language model operated by Anthropic. Claude is available as a web application at claude.ai. IPRally makes use of Claude through Anthropic API.

There are two reasons. First, the models available for self-hosting are not as good as Claude in answering questions.

Second, we want you to get answers to your questions promptly. The number of simultaneous questions we receive may vary a lot over the day – occasionally we need to answer a large number of simultaneous questions and sometimes we do not have any questions to answer. Handling this sort of load efficiently and quickly can be very challenging and expensive. An online LLM service such as Anthropic’s Claude spreads the load between all of its users, and thus helps us answer your questions consistently quickly.

While there are many large language models offered as services, there are currently two which answer questions better than the others. These are ChatGPT from OpenAI and Claude from Anthropic. We want to give answers of highest quality so our choice was between ChatGPT and Claude.

The primary benefit of using Claude is its long context window, the length of the text that the model can be prompted with. Due to the length of its context window, we can be sure that we can answer questions about any single patent without needing to break down the text to parts.

Claude 3 Haiku. In our tests it offered the best combination of high quality answers, speed of answering, and low cost, allowing us to offer the initial Patent Q&A feature to you and every other user at no extra cost to you.