Replace OpenAI API with local models: Ollama + LiteLLM, text gen webui, google colab

Описание

I walk through my 3 favourite methods for running an OpenAI compatible api powered by local models: Ollama + Litellm, Text Generation WebUI and google colab

https://ollama.ai/
https://github.com/BerriAI/litellm
https://github.com/oobabooga/text-generation-webui
https://colab.research.google.com/drive/1YX80sq6t727S7A_6ZnbBxxx2Jp1hHsu6?usp=sharin #scrollTo=LGQ8BiMuXMDG
https://github.com/Troyanovsky/Local-LLM-Comparison-Colab-UI

You can sign up for the next intake of the course I mentioned at
https://devacademy.co.nz/practical-ai-for-developers/

00:00 Introduction
00:29 A simple script
01:02 Ollama + LiteLLM
03:57 textgen-webui
09:31 Google colab + textgen-webui