Issue on page /internvl2.5/evaluation

[Andrew-Koulogeorge]

Hi
I am trying to reproduce your results from the InternVL2.5 family not he MMMU dataset. When running InternVL2.5 family with VLMEvalKit framework for sizes larger then 2Bn on the MMMU dataset, there are some examples with many images (and thus many tokens) and this is giving me cuda out of memory errors (The hardware I am using has 48GB)

When you evaluated this model did you have more RAM? Did you implement model parallelism yourself?

[yuecao0119]

Hi,

1. For different GPU occupancy of the model, you can refer https://internvl.readthedocs.io/en/latest/internvl2.5/deployment.html#memory-usage-testing
2. For model splitting, please refer https://github.com/open-compass/VLMEvalKit/blob/22cc2d35980fb15dbe9146309e163f7b2800adc3/docs/en/Quickstart.md?plain=1#L150