From 3896a6fea6edb9dcf95755bf6214a9a410143fd1 Mon Sep 17 00:00:00 2001 From: yiranyyu <2606375857@qq.com> Date: Thu, 23 May 2024 16:33:08 +0800 Subject: [PATCH] update readme --- README.md | 2 +- README_zh.md | 2 +- docs/compare_with_phi-3_vision.md | 24 +++++++++++++++++++++--- 3 files changed, 23 insertions(+), 5 deletions(-) diff --git a/README.md b/README.md index 57bb5bc..2588191 100644 --- a/README.md +++ b/README.md @@ -25,7 +25,7 @@ ## News -* [2024.05.23] We've released a comprehensive comparison between Phi-3-vision-128k-instruct and MiniCPM-Llama3-V 2.5, including benchmarks evaluations, and multilingual capabilities 🌟📊🌍. Click [here](#evaluation) to view more details. +* [2024.05.23] We've released a comprehensive comparison between Phi-3-vision-128k-instruct and MiniCPM-Llama3-V 2.5, including benchmarks evaluations, and multilingual capabilities 🌟📊🌍. Click [here](./docs/compare_with_phi-3_vision.md) to view more details. * [2024.05.20] We open-soure MiniCPM-Llama3-V 2.5, it has improved OCR capability and supports 30+ languages, representing the first edge-side MLLM achieving GPT-4V level performance! We provide [efficient inference](#deployment-on-mobile-phone) and [simple fine-tuning](./finetune/readme.md). Try it now! * [2024.04.23] MiniCPM-V-2.0 supports vLLM now! Click [here](#vllm) to view more details. * [2024.04.18] We create a HuggingFace Space to host the demo of MiniCPM-V 2.0 at [here](https://huggingface.co/spaces/openbmb/MiniCPM-V-2)! diff --git a/README_zh.md b/README_zh.md index bd038cb..0eff044 100644 --- a/README_zh.md +++ b/README_zh.md @@ -28,7 +28,7 @@ ## 更新日志 -* [2024.05.23] 我们添加了Phi-3-vision-128k-instruct与MiniCPM-Llama3-V 2.5的全面对比，包括基准测试评估和多语言能力 🌟📊🌍。点击[这里](#性能评估)查看详细信息。 +* [2024.05.23] 我们添加了Phi-3-vision-128k-instruct与MiniCPM-Llama3-V 2.5的全面对比，包括基准测试评估和多语言能力 🌟📊🌍。点击[这里](./docs/compare_with_phi-3_vision.md)查看详细信息。 * [2024.05.20] 我们开源了 MiniCPM-Llama3-V 2.5，增强了 OCR 能力，支持 30 多种语言，并首次在端侧实现了 GPT-4V 级的多模态能力！我们提供了[高效推理](#手机端部署)和[简易微调](./finetune/readme.md)的支持，欢迎试用！ * [2024.04.23] 我们增加了对 [vLLM](#vllm) 的支持，欢迎体验！ diff --git a/docs/compare_with_phi-3_vision.md b/docs/compare_with_phi-3_vision.md index b6b9d58..1a025e4 100644 --- a/docs/compare_with_phi-3_vision.md +++ b/docs/compare_with_phi-3_vision.md @@ -1,8 +1,8 @@ ## Phi-3-vision-128K-Instruct vs MiniCPM-Llama3-V 2.5 -Comparison results of Phi-3-vision-128K-Instruct and MiniCPM-Llama3-V 2.5, regarding the model size, hardware requirements, and performances on multiple popular benchmarks. +Comparison results of Phi-3-vision-128K-Instruct and MiniCPM-Llama3-V 2.5, regarding the model size, hardware requirements, and performances. -我们提供了从模型参数、硬件需求、全面性能指标等方面对比 Phi-3-vision-128K-Instruct 和 MiniCPM-Llama3-V 2.5 的结果。 +我们提供了从模型参数、硬件需求、性能指标等方面对比 Phi-3-vision-128K-Instruct 和 MiniCPM-Llama3-V 2.5 的结果。 ## Hardeware Requirements （硬件需求） @@ -18,7 +18,9 @@ With in4 quantization, MiniCPM-Llama3-V 2.5 delivers smooth inference with only ## Model Size and Peformance （模型参数和性能） +In most benchmarks, MiniCPM-Llama3-V 2.5 achieves **better performance** compared with Phi-3-vision-128K-Instruct. +在大多数评测集上， MiniCPM-Llama3-V 2.5 相比于 Phi-3-vision-128K-Instruct 都展现出了**更优的性能表现**. | | Phi-3-vision-128K-Instruct | MiniCPM-Llama3-V 2.5| |:-|:----------:|:-------------------:| @@ -28,4 +30,20 @@ With in4 quantization, MiniCPM-Llama3-V 2.5 delivers smooth inference with only | RealworldQA | 58.8 | **63.5**| | TextVQA | 72.2 | **76.6** | | ScienceQA| **90.8** | 89.0 | -| POPE | 83.4 | **87.2** | \ No newline at end of file +| POPE | 83.4 | **87.2** | + + +## Multilingual Capabilities + + +MiniCPM-Llama3-V 2.5 exhibits **stronger multilingual** capabilities compared with Phi-3-vision-128K-Instruct on LLaVA Bench. + +MiniCPM-Llama3-V 2.5 在对话和推理评测榜单 LLaVA Bench 上展现出了比 Phi-3-vision-128K-Instruct **更强的多语言的性能**。 + +
+ +
+ Evaluation results of LLaVABench in multiple languages +
+ 多语言LLaVA Bench评测结果 +