Ollama

How to create model files

Create a file called mixtral.model

FROM /Users/gavi/.cache/lm-studio/models/TheBloke/Mixtral-8x7B-Instruct-v0.1-GGUF/mixtral-8x7b-instruct-v0.1.Q5_K_M.gguf
TEMPLATE "[INST] {{ .Prompt }} [/INST]"%  

Then you can import it via the bash command

 ollama create mixtral -f mixtral.model

You can list the models via ollama list command

ollama list                
NAME            	ID          	SIZE  	MODIFIED   
mistral7b:latest	76826b687112	5.9 GB	8 days ago	
mixtral:latest  	df2ff018b933	32 GB 	8 days ago

You can run these models via

ollama run mixtral

The open-ai API is running at port

http://localhost:11434

I could not find a way to change the port number

You can run multiple models and they show up when you call them individually