Error: Cannot perform a '--user' install. User site-packages are not visible in the virtual environment. CM error : portable CM script failed (name = build-mlperf-inference-server-nvidia, return code = 256)

[shyambansal17]

please help me out with this error.

[arjunsuresh]

Hi @shyambansal17 which command did you run here?

[shyambansal17]

Hi @arjunsuresh i got rid of the that error but now facing another error that is and i think it needs modification in the generation engine file.

[2024-12-12 21:35:10,654 builder.py:250 INFO] Using FP16 network
[12/12/2024-21:35:13] [TRT] [W] ITensor::setType(Half) was called on non I/O tensor: embln_output. This will have no effect unless this tensor is marked as an output.
[12/12/2024-21:35:13] [TRT] [I] Using default for use_int8_scale_max: true
[12/12/2024-21:35:13] [TRT] [F] Assertion failed: (mSM == kSM_90 || mSM == kSM_87 || mSM == kSM_86 || mSM == kSM_89 || mSM == kSM_80 || mSM == kSM_75 || mSM == kSM_72) && (type == DataType::kINT8 || type == DataType::kHALF) && "requesting maxSeqlen not compatible with GPU arch"
plugin/bertQKVToContextPlugin/qkvToContextPlugin.cpp:620
Aborting...

---

PyCUDA ERROR: The context stack was not empty upon module cleanup.

A context was still active when the context stack was being
cleaned up. At this point in our execution, CUDA may already
have been deinitialized, so there is no way we can finish
cleanly. The program will be aborted now.
Use Context.pop() to avoid this problem.

Traceback (most recent call last):
File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "/home/cmuser/CM/repos/local/cache/29aa009c9f3f4d5e/repo/closed/NVIDIA/code/main.py", line 231, in
main(main_args, DETECTED_SYSTEM)
File "/home/cmuser/CM/repos/local/cache/29aa009c9f3f4d5e/repo/closed/NVIDIA/code/main.py", line 144, in main
dispatch_action(main_args, config_dict, workload_setting)
File "/home/cmuser/CM/repos/local/cache/29aa009c9f3f4d5e/repo/closed/NVIDIA/code/main.py", line 202, in dispatch_action
handler.run()
File "/home/cmuser/CM/repos/local/cache/29aa009c9f3f4d5e/repo/closed/NVIDIA/code/actionhandler/base.py", line 82, in run
self.handle_failure()
File "/home/cmuser/CM/repos/local/cache/29aa009c9f3f4d5e/repo/closed/NVIDIA/code/actionhandler/base.py", line 186, in handle_failure
self.action_handler.handle_failure()
File "/home/cmuser/CM/repos/local/cache/29aa009c9f3f4d5e/repo/closed/NVIDIA/code/actionhandler/generate_engines.py", line 186, in handle_failure
raise RuntimeError("Building engines failed!")
RuntimeError: Building engines failed!
make: *** [Makefile:37: generate_engines] Error 1

CM error: Portable CM script failed (name = app-mlperf-inference-nvidia, return code = 256)

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

[arjunsuresh]

Which Nvidia GPU are you running on?

[shyambansal17]

NVIDIA V100

[shyambansal17]

@arjunsuresh ??

[arjunsuresh]

Unfortunately I don't think Nvidia implementation support V100 or any GPUs in the Turing generation.

[shyambansal17]

@arjunsuresh deos it support RTX 3060 or 3050 ??

[arjunsuresh]

Officially no. But most of the smaller mlperf inference models should run fine there.

[shyambansal17]

@arjunsuresh can you please list out all the smaller mlperf inference models i can run on my system ?