deepgeek/MiniCPM-o
2
0
Fork 0
mirror of https://github.com/OpenBMB/MiniCPM-V.git synced 2026-02-05 18:29:18 +08:00
Code Issues Packages Projects Releases Wiki Activity

按照要求修改了标题，术语，章节描述

Browse Source
This commit is contained in:
root
2024-07-18 15:33:38 +08:00
parent 96bd6b9a13
commit 3e3c6299b3
1 changed files with 42 additions and 31 deletions
Download Patch File Download Diff File Expand all files Collapse all files

73
docs/xinference_infer.md
View File

@@ -1,21 +1,26 @@
## about xinference ## Xinference Infer
xinference is a unified inference platform that provides a unified interface for different inference engines. It supports LLM, text generation, image generation, and more.but it's not bigger than swift too much. Xinference is a unified inference platform that provides a unified interface for different inference engines. It supports LLM, text generation, image generation, and more.but it's not bigger than Swift too much.
## xinference install
### Xinference install
Xinference can be installed simply by using the following easy bash code:
```shell ```shell
pip install "xinference[all]" pip install "xinference[all]"
``` ```
## quick start ### Quick start
1. start xinference The initial steps for conducting inference with Xinference involve downloading the model during the first launch.
1. Start xinference in the terminal:
```shell ```shell
xinference xinference
``` ```
2. start the web ui. 2. Start the web ui.
3. Search for "MiniCPM-Llama3-V-2_5" in the search box. 3. Search for "MiniCPM-Llama3-V-2_5" in the search box.
[alt text](../assets/xinferenc_demo_image/xinference_search_box.png)
4. find and click the MiniCPM-Llama3-V-2_5 button. ![alt text](../assets/xinferenc_demo_image/xinference_search_box.png)
5. follow the config and launch the model.
4. Find and click the MiniCPM-Llama3-V-2_5 button.
5. Follow the config and launch the model.
```plaintext ```plaintext
Model engine : Transformers Model engine : Transformers
model format : pytorch model format : pytorch
@@ -24,33 +29,39 @@ quantization : none
N-GPU : auto N-GPU : auto
Replica : 1 Replica : 1
``` ```
6. after first click the launch button,xinference will download the model from huggingface. we should click the webui button. 6. After first click the launch button,xinference will download the model from huggingface. We should click the webui button.
![alt text](../assets/xinferenc_demo_image/xinference_webui_button.png)
7. upload the image and chatting with the MiniCPM-Llama3-V-2_5
## local MiniCPM-Llama3-V-2_5 launch ![alt text](../assets/xinferenc_demo_image/xinference_webui_button.png)
1. start xinference
7. Upload the image and chatting with the MiniCPM-Llama3-V-2_5
### Local MiniCPM-Llama3-V-2_5 Launch
If you have already downloaded the MiniCPM-Llama3-V-2_5 model locally, you can proceed with Xinference inference following these steps:
1. Start xinference
```shell ```shell
xinference xinference
``` ```
2. start the web ui. 2. Start the web ui.
3. To register a new model, follow these steps: the settings highlighted in red are fixed and cannot be changed, whereas others are customizable according to your needs. Complete the process by clicking the 'Register Model' button. 3. To register a new model, follow these steps: the settings highlighted in red are fixed and cannot be changed, whereas others are customizable according to your needs. Complete the process by clicking the 'Register Model' button.
![alt text](../assets/xinferenc_demo_image/xinference_register_model1.png) ![alt text](../assets/xinferenc_demo_image/xinference_register_model1.png)
![alt text](../assets/xinferenc_demo_image/xinference_register_model2.png) ![alt text](../assets/xinferenc_demo_image/xinference_register_model2.png)
4. After completing the model registration, proceed to 'Custom Models' and locate the model you just registered.
5. follow the config and launch the model.
```plaintext
Model engine : Transformers
model format : pytorch
Model size : 8
quantization : none
N-GPU : auto
Replica : 1
```
6. after first click the launch button,xinference will download the model from huggingface. we should click the chat button.
![alt text](../assets/xinferenc_demo_image/xinference_webui_button.png)
7. upload the image and chatting with the MiniCPM-Llama3-V-2_5
## FAQ 4. After completing the model registration, proceed to 'Custom Models' and locate the model you just registered.
5. Follow the config and launch the model.
```plaintext
Model engine : Transformers
model format : pytorch
Model size : 8
quantization : none
N-GPU : auto
Replica : 1
```
6. After first click the launch button,Xinference will download the model from Huggingface. we should click the chat button.
![alt text](../assets/xinferenc_demo_image/xinference_webui_button.png)
7. Upload the image and chatting with the MiniCPM-Llama3-V-2_5
### FAQ
1. Why can't the sixth step open the WebUI? 1. Why can't the sixth step open the WebUI?
maybe your firewall or mac os to prevent the web to open.
Maybe your firewall or mac os to prevent the web to open.