Harnessing Open Source Solutions in the Generative AI Landscape

Presentation Overview
This session will feature a comprehensive demonstration about how to finetune and host generative AI models (e.g. Qwen2.5, llama3.2) using open-sourced solutions. We will show attendees how to finetune open-source models (e.g. Qwen2.5, Llama3.2) using the LLaMA-Factory library and how to host these finetuned models using the open-sourced vLLM library.

Problem Statement
The generative AI landscape has largely dominated by proprietary solutions with closed models. This has fostered a mindset that generative AI is beyond the reach of practitioners and solely controlled OpenAI, Anthropic, etc.

Proposed Solution
Open-source frameworks such as LLaMA-Factory and vLLM provide practitioners with full control over their AI models. These tools not only enable advanced fine-tuning through diverse training approaches—such as supervised fine-tuning and reinforcement learning routines (PPO, DPO) but also facilitate hosting on your own hardware, whether it be GPUs, TPUs, or XPUs.

Benefits

1. Ownership and Control: Maintain autonomy over your AI models, ensuring consistent performance aligned with your specific needs.
2. Cost Efficiency: Hosting models independently can be extremely cost-effective, particularly in scenarios with high-scale usage.
3. Model Customization: Benefit from various training methods to refine your models exactly as desired.
4. Community Support: Take advantage of an open-source community that continually enhances these robust frameworks.

Conclusion
We will demonstrate the practical application of open-source frameworks like LLaMA-Factory and vLLM for fine-tuning and hosting generative AI models. By utilizing these tools, practitioners can gain greater control over their models, reduce operational costs, and tailor their AI solutions to specific needs. Additionally, the open-source community provides ongoing support and enhancements, making these frameworks a viable alternative to proprietary solutions in the generative AI landscape.

Resources:

• https://github.com/vllm-project/vllm
• https://github.com/hiyouga/LLaMA-Factory