What is a large language model?
A large language model is a machine learning model trained with text documents (such as books, internet articles, and patents) to predict how any given text continues. For example, such a model might predict that a text starting with the words “On a hot summer day I like to eat ” might continue “ice cream.” You can think of a language model as an auto-complete for any text.
The most prominent use case for LLMs is chatting with an AI assistant. For this purpose, some language models have been expressly trained to produce conversational text between a user and a helpful and honest assistant. Given a text such as “What is the capital of Norway?” such a model is more likely to continue it with “Oslo” than with e.g. “None of your business.” or “she had often pondered”.
Why do you use online LLM services rather than hosting an LLM of your own?
There are two reasons. First, the online LLM services produce more accurate answers than the models available for self-hosting.
Second, we want to generate answers to our users’ questions promptly. The number of simultaneous questions we receive may vary significantly over the day. Occasionally, we need to answer a large number of simultaneous questions, and sometimes, we do not have any questions to answer. Handling this load efficiently and quickly can be challenging and expensive. An online LLM service spreads the load between all of its users and thus helps us answer your questions consistently and quickly.
What exact versions of LLM services does IPRally use?
For Ask AI, we use the Gemini model developed by Google. We use Gemini Flash version 2.0.
For Image search, we use Claude 3.5 Sonnet. In our tests, it offered the best combination of high-quality answers and speed of answering at a competitive cost.