Issue Type
Inaccurate or outdated content
Location
README.md
Description
Hi,
I followed the Ascend setup tutorial, but python offline.py fails during LLM(...) initialization.
Environment:
uv pip list|grep -E "vllm|fl|torch"
Using Python 3.11.11 environment at: /root/code/.venv
flag-gems 5.0.1rc0 /root/code/FlagGems
flagtree 0.5.0+ascend3.2
torch 2.9.1
torch-npu 2.9.0
torchaudio 2.11.0
torchvision 0.24.1
vllm 0.13.0+empty /root/code/vllm
Reproduction:
- Set up the environment according to the tutorial
- Run:
from vllm import LLM, SamplingParams
import torch
from vllm.config.compilation import CompilationConfig
if __name__ == '__main__':
prompts = [
"Hello, my name is",
]
# Create a sampling params object.
sampling_params = SamplingParams(max_tokens=10, temperature=0.0)
# Create an LLM.
llm = LLM(model="/data2/models/Qwen3-4B", max_num_batched_tokens=16384, max_num_seqs=2048)
# Generate texts from the prompts.
outputs = llm.generate(prompts, sampling_params)
for output in outputs:
prompt = output.prompt
generated_text = output.outputs[0].text
print(f"Prompt: {prompt!r}, Generated text: {generated_text!r}")
Observed error:
python offline.py
INFO 04-07 17:17:40 [utils.py:253] non-default args: {'max_num_batched_tokens': 16384, 'max_num_seqs': 2048, 'disable_log_stats': True, 'model': '/data2/models/Qwen3-4B'}
Traceback (most recent call last):
File "/root/code/files/offline.py", line 13, in <module>
llm = LLM(model="/data2/models/Qwen3-4B", max_num_batched_tokens=16384, max_num_seqs=2048)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/code/vllm/vllm/entrypoints/llm.py", line 351, in __init__
self.llm_engine = LLMEngine.from_engine_args(
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/code/vllm/vllm/v1/engine/llm_engine.py", line 175, in from_engine_args
vllm_config = engine_args.create_engine_config(usage_context)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/code/vllm/vllm/engine/arg_utils.py", line 1314, in create_engine_config
device_config = DeviceConfig(device=cast(Device, current_platform.device_type))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/code/.venv/lib/python3.11/site-packages/pydantic/_internal/_dataclasses.py", line 121, in __init__
s.__pydantic_validator__.validate_python(ArgsKwargs(args, kwargs), self_instance=s)
File "/root/code/vllm/vllm/config/device.py", line 75, in __post_init__
self.device = torch.device(self.device_type)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: Device string must not be empty
[ERROR] 2026-04-07-17:17:40 (PID:1172960, Device:-1, RankID:-1) ERR99999 UNKNOWN applicaiton exception
Questions:
- Is
vllm-ascend required for Ascend support, instead of only installing the main vllm+empty package?
- Are there additional environment variables or plugin-registration steps needed so the Ascend platform plugin is discovered correctly?
Thanks.
Suggested Fix
No response
Issue Type
Inaccurate or outdated content
Location
README.md
Description
Hi,
I followed the Ascend setup tutorial, but
python offline.pyfails duringLLM(...)initialization.Environment:
Reproduction:
Observed error:
Questions:
vllm-ascendrequired for Ascend support, instead of only installing the mainvllm+emptypackage?Thanks.
Suggested Fix
No response