Cannot quantize Qwen2-VL-72B-Instruct on a single gpu card

[WeiweiZhang1]

when running below shell cmd, the device-related error is reported, and the quantization of the 72B model is expected to be done by a single gpu card.

CUDA_VISIBLE_DEVICES=$device
python3 -m auto_round --mllm
--model ${model_dir}/Qwen2-VL-72B-Instruct
--group_size 64
--bits 2
--iters 2000
--nsample 1024
--low_gpu_mem_usage
--seqlen 2048
--model_dtype "float16"
--format 'auto_gptq,auto_round' \

[wenhuach21]

@n1ck-guo following llm, disable auto-mapping for single card

[wenhuach21]

Besides, please add a gpu ut for 70B models

[wenhuach21]

#395