- Replaced manual index calculation with `np.linspace` for improved efficiency and readability.
- Reduced computation overhead by utilizing NumPy's vectorized operations for generating evenly spaced frame indices.
Compared with the streamlit implementation of 2.5, this code implementation can better play the new multi-modal capabilities of 2.6:
1. The application supports the upload and processing of text, single image, multiple images and videos, and can process different types of input according to the mode selected by the user.
2. Video frame extraction and encoding: In video mode, frames are extracted from the uploaded video through the decord library and uniformly sampled so that the model can process and generate responses. More detailed and clear variables and annotations. Convenient for learning and use
3. File upload and processing: Support users to upload pictures and videos, and perform corresponding processing according to different modes, such as displaying pictures in single picture mode, displaying multiple pictures in multi-picture mode, and processing video frames in video mode. You can switch back and forth between different media.
4. Tip: You can use the command `streamlit run ./web_demo_streamlit-minicpmv2_6.py --server.maxUploadSize 1024`
to adjust the maximum upload size to 1024MB or larger files. The default 200MB limit of Streamlit's file_uploader component might be insufficient for video-based interactions. Adjust the size based on your GPU memory usage.
* Implement chatbot functionality using Streamlit
This commit adds the implementation of a chatbot using Streamlit, a Python library for building interactive web applications. The chatbot allows users to interact with an AI assistant, asking questions and receiving responses in real-time.
Features include:
- Integration with the MiniCPM-V-2.0 model for generating responses.
- User-friendly interface with text input for questions and options for uploading images.
- Sidebar settings for adjusting parameters such as max_length, top_p, and temperature.
- Ability to clear chat history to start a new conversation.
The chat history and session state are managed using Streamlit's session_state functionality, ensuring a seamless user experience across interactions.
This implementation provides a simple and intuitive way for users to engage with the chatbot, making it accessible for various use cases.
* update MiniCPM-Llama3-V-2_5 streamlit demo
* Update web_demo_streamlit-2_5.py
This update, based on the May 25, 2024 version of modeling_minicpmv.py, includes the following enhancements:
1. Introduction of repetition_penalty and top_k parameters to the st.sidebar, enabling users to adjust these model parameters dynamically.
2. Default support for stream=True in the model.chat method to facilitate real-time streaming responses.
JamePeng