diff --git a/README.md b/README.md index 7078c61..3adb0e6 100644 --- a/README.md +++ b/README.md @@ -23,19 +23,29 @@ [中文文档](./README_zh.md) ## Contents +- [Contents](#contents) - [OmniLMM-12B](#omnilmm-12b) + - [Evaluation](#evaluation) + - [Examples](#examples) - [OmniLMM-3B](#omnilmm-3b) + - [Evaluation](#evaluation-1) + - [Examples](#examples-1) - [Demo](#demo) - [Install](#install) - [Inference](#inference) -- [Model Zoo](#model-zoo) + - [Model Zoo](#model-zoo) + - [Multi-turn Conversation](#multi-turn-conversation) +- [✅ TODO](#-todo) +- [Model License](#model-license) +- [Statement](#statement) +- [🏫 Institutions](#-institutions) ## OmniLMM-12B **OmniLMM-12B** is the most capable version. The model is built based on EVA02-5B and Zephyr-7B-β, connected with a perceiver resampler layer, and trained on multimodal data in a curriculum fashion. The model has three notable features: - 🔥 **Strong Performance.** - OmniLMM-12B achieves **leading performance** among models with comparable sizes, surpassing established LMMs on multiple benchmarks (including MME, MMBench, SEED-Bench, etc). The model endows **rich multimodal world knowledge**. + OmniLMM-12B achieves **leading performance** among models with comparable sizes, surpassing established LMMs on multiple benchmarks (including MME, MMBench, SEED-Bench, etc). - 🏆 **Trustworthy Behavior.** @@ -43,9 +53,15 @@ - 🕹 **Real-time Multimodal Interaction.** - We combine the OmniLMM-12B and GPT-3.5 into a **real-time multimodal interactive assistant**. The assistant accepts video streams from the camera and speech streams from the microphone and emits speech output. While still primary, we find the model can **replicate some of the fun cases shown in the Gemini Demo video, without any video edition**. + We combine the OmniLMM-12B and GPT-3.5 (text-only) into a **real-time multimodal interactive assistant**. The assistant accepts video streams from the camera and speech streams from the microphone and emits speech output. While still primary, we find the model can **replicate some of the fun cases shown in the Gemini Demo video, without any video edition**. + ### Evaluation +
+