What is reft.cpp¶
-
A high-performance and easy-to-use LLM/LM serving tools for inference, training. All of Ops/Ops-Fusion/Ops-Optimization, serving framework and training tools in C++ without Python/PyTorch, inspired by llm.c of Andrej Karpathy.
-
An excutable file of "reft" with model weights is all you need to run the reft-supported model on your GPU(s) with the better performance.
-
Our deliveralbes are for enterprises, institutes, individuals, GPU/NPU chipset vendors, AIDC, who are seeking for the better performance, cost-efficient, easy-to-use of LLM/LMs.
Supported models¶
- : Done
- : To-Do
| Models | Nvidia GPU | AMD GPU | Hexagon NPU | Moore Threads GPU | Meta GPU |
|---|---|---|---|---|---|
| Qwen2.5-0.5/1.5/3/7/14/32/72B(-Instruct) | |||||
| Qwen2.5-Math-1.5/7/72B(-Instruct) | |||||
| Qwen2.5-Coder-0.5/1.5/3/7/14/32B(-Instruct) | |||||
| DeepSeek-R1 | |||||
| DeepSeek-V3 | |||||
| DeepSeek-R1-Distill-Qwen-1.5/7/14/32B |
| Models | Nvidia GPU | AMD GPU | Hexagon NPU | Moore Threads GPU | Meta GPU |
|---|---|---|---|---|---|
| Whisper | |||||
| OpenVoice | |||||
| MeloTTS-English/... |
License Statement
- This software is provided "as is" for non-commercial use only. You may distribute the executable files freely for personal or evaluation purposes. Commercial use, including but not limited to integration into products, services, or resale, requires explicit written permission from the author. Contact: [ai@reft-ai.com] for licensing inquiries.
- No reverse engineering, modification, or decompilation is permitted.