Best Practices and Interview Q&A

Best Practices Checklist

• Pin model tags in your scripts; tags change.
• Test every new model on your real prompts.
• Use q4 quantization unless quality demands more.
• Keep context at the smallest useful size.
• Run versions under test in a separate model name.

Interview Q&A

Q: How does Ollama run models locally?

A: It pulls quantized GGUF weights from the model library, manages them on disk, and runs them with a local engine exposed through CLI and HTTP APIs.

Q: Quantization tradeoff?

A: Fewer bits per weight means less memory and faster inference at a small quality cost. q4 is a solid default; use q8 or full precision when quality matters and memory allows.

Q: How do you make Ollama models OpenAI-compatible?

A: No conversion needed, the /v1 endpoint speaks the OpenAI chat format; point any client at http://localhost:11434/v1.

Q: How do you speed up slow generation?

A: Use a smaller or more quantized model, run on GPU, shorten context, and tune OLLAMA_NUM_PARALLEL against actual load.

Key Points

• Pin tags and re-test models as the library evolves.
• Quantization and context control the memory story.
• The OpenAI-compatible endpoint makes integration trivial.
• Fit models to hardware; measure, then tune.