Versions 0.9.0 through 0.18.0 of LMDeploy can be tricked into executing operating‑system commands when an attacker sends specially crafted data to the.

Meta Description: Compare vLLM, SGLang, TensorRT-LLM, LMDeploy, and llama.cpp to choose the right inference server for production LLMs and AI agents.

How to choose an AI inference engine for production: compatibility, latency metrics, benchmark pitfalls, and where vLLM, SGLang, LMDeploy and llama.cpp fit.

LMDeploy’s DistServe feature can be tricked into connecting to an attacker‑controlled server.

The LMdeploy software includes an RPC server that accepts messages and deserializes them without checking their content.

LMDeploy is a toolkit for compressing, deploying, and serving large language models. Starting in version 0.9.1 and prior to version 0.10.2, the LMdeploy implements an rpc server (AsyncRPCServer in zmq_rpc.py) for supporting the RPC communications. In its core functionality call_and_response(), I found it will directly use the pickles.loads() to deserialize …

LMDeploy's disaggregated serving feature allows a malicious peer to inject code that can be executed by the server, potentially leading to unauthorized.

InternLM lmdeploy's handle_zmq_recv coroutine deserializes peer-to-peer cache-free requests using pickle.loads() before verifying the object's type. The source of the serialized data is a ZeroMQ PULL socket connected to an endpoint specifie

LMDeploy deserializes disaggregated-serving peer messages with pickle. The handle_zmq_recv coroutine in lmdeploy/pytorch/disagg/conn/engine_conn.py reads peer-to-peer cache-free requests with recv_pyobj(), which deserializes the received bytes with pickle.loads(), and the isinstance check against DistServeCacheFreeRequest runs only after deserialization has already completed. The peer that supplies those bytes is caller-controlled: p2p_connect passes remote_engine_endpoint_info.zmq_address from …

Production LLM serving is now a systems problem, not a generate() loop. For real workloads, the choice of inference stack drives your tokens per second, tail latency, and ultimately cost per million tokens on a given GPU fleet. This comparison focuses on 4 widely used stacks: vLLM NVIDIA TensorRT-LLM Hugging Face Text Generation Inference (TGI v3) LMDeploy 1. vLLM, PagedAttention as the open baseline…

The lmdeploy API server has a security weakness that allows unauthorized access to internal services and sensitive information.