Ollama Tutorial

Running Models Locally Ollama is a free, open-source application that downloads open-weights language models and runs them on your own machine. No cloud account, no API key, and ...

Install on macOS and Linux The script installs the ollama command and registers a service so models load on demand. Install on Windows Download the installer from ollama.com and ...

The Model Library Models live in the Ollama library under names with tag suffixes. Common families include llama3.2, qwen2.5, mistral and phi3. Pull a Model Downloading separately ...

Step 1 - Fit the Hardware A model needs roughly 1 GB of RAM or VRAM per billion parameters at full precision, less when quantized. Start with a small model, then move up until ...

Everyday Commands Running and Inspecting ollama show reveals the context length a model expects, total size and quantization, useful before tuning. Inside the Chat REPL The ...

Why a Modelfile A Modelfile is a small text file that wraps a base model with your own system prompt, defaults and parameters. You get a model tuned to your task without any ...

The Local HTTP API Ollama listens on port 11434 by default. Any language can send a generate request over HTTP. OpenAI-Compatible Endpoint Point existing OpenAI clients at Ollama ...

Why Embeddings Embedding models turn text into vectors for similarity search and RAG. Ollama runs embedding models locally, so your indexing stays private and free. Pull an ...

Why Local Coding Coding agents and editors normally call hosted models. Pointing them at Ollama keeps your code and prompts in-house, removes token costs, and works fully offline. ...

Expose the API Ollama listens only on localhost by default. Set OLLAMA_HOST to reach it from other machines. Clients can now call http://your-server:11434 from the local network. ...

Where Speed Comes From Token speed depends on the model size, the device it runs on, the context length and the number of concurrent requests. A GPU beats CPU by a wide margin for ...

Best Practices Checklist Pin model tags in your scripts; tags change. Test every new model on your real prompts. Use q4 quantization unless quality demands more. Keep context at ...