Docker x Ollama x Hugging Face
By Rohan Kataria · Mar 22, 2024
Key points
My Dockerfile runs an Ollama server on Hugging Face Spaces with LLAMA2, CODELLAMA and NOMIC-EMBED-TEXT.
Ollama lets me try models and embeddings for free on my own system, including my own GGUF models.
The server API powers two of my Hugging Face Spaces projects, built with LangChain's ChatOllama and OLLAMAEmbedding modules.
The demo server runs on a T4 Small for $0.6/hr.