Ollama

Ollama (Github) is the easiest way to run LLM models like LLama-2, Mistral, and many more from a single installable application. Ollama also allows you to run a server to programmatically interact with your loaded model, which is how AnythingLLM and Ollama integrate.

OllamaOllama_ai

You are responsible for running and maintaining your instance of Ollama so that AnythingLLM can chat with it and use it for generative responses!

Like other LLM providers, the Chat Model Selection dropdown will automatically populate when your endpoint is entered. All models in your installation will be available to you for chatting.