Tiny Text + Vision Models - Fine tuning and API Setup

Опубликовано: 22 Апрель 2024
на канале: Trelis Research

5,378

191

➡️ Get Life-time Access to the Complete Scripts (and future improvements): https://Trelis.com/ADVANCED-vision/
➡️ One-click fine-tuning and LLM templates: https://github.com/TrelisResearch/one...
➡️ Trelis Livestreams: Thursdays 5 pm Irish time on YouTube and X.
➡️ Newsletter: https://blog.Trelis.com
➡️ Resources/Support/Discord: https://Trelis.com/About

VIDEO RESOURCES:
Slides: https://docs.google.com/presentation/...
Moondream model: https://huggingface.co/vikhyatk/moond...
Moondream Github: https://github.com/vikhyat/moondream
Chess Dataset: https://huggingface.co/datasets/Treli...

TIMESTAMPS:
0:00 Fine-tuning tiny multi-modal models
0:11 Moondream server demo
1:41 Video Overview
4:22 Multi-modal model architecture
6:33 Moondream architecture
6:50 Moondream vision encoder (SigLIP)
13:05 Moondream MLP (visionprojection)
15:50 Moondream Language Model (Phi)
17:15 Applying LoRA adapters to a multi-modal model
18:59 Fine-tuning notebook demo
37:58 Deploying a custom API for multi-modal models
42:00 vLLM
42:49 Training a multi-modal model from scratch
43:44 Multi-modal datasets
44:00 Video resources