LiteLLM Proxy is a component used for model serving that integrates OpenAI, Anthropic, Azure, Bedrock, and local LLM endpoints. Employed by organizations requiring large language model capabilities, LiteLLM optimizes and routes AI requests efficiently. However, its configuration is crucial to prevent unauthorized access because it widely disseminates its API for distributed use. From startups to large-scale enterprises, LiteLLM facilitates AI model deployment but requires secure setup practices. Usage involves defining virtual keys and configuring routing parameters adequately. With increasing adoption, securing this technology against exposure threats is paramount.
LiteLLM Proxy Exposure involves revealing sensitive configuration details through its API endpoints due to missing or disabled master keys. This exposure can disclose the model listing API, making it possible to determine what models are configured. Attackers can access /v1/models and /model/info to retrieve information about upstream configurations. Misconfigurations may allow unrestricted access to sensitive endpoints without authentication. This vulnerability presents a significant risk of data misuse and unauthorized interaction. Therefore, detecting such exposures ensures the security of integrated models and continued privacy.
The vulnerability is evident when the LiteLLM proxy does not enforce `general_settings.master_key` for its API endpoints. Without a valid key, identifiers and model configurations can be queried, exposing critical architectural details. Vital endpoints include /model/info and /chat/completions, commonly left open. Attackers operating remotely can explore these paths without particular credentials, potentially leading to unauthorized operations. The template identifies whether the status code shows unauthorized (200) responses and examines JSON responses for critical indicators such as 'data', 'litellm_params', and 'model_info'. These technical details confirm uncontrolled access to deployed AI models.
The effects of exploiting this vulnerability include unauthorized model use, leading to increased operational costs as attackers utilize resources freely. It may escalate to data exfiltration from cached prompt logs, comprising privacy and potentially sensitive information. Worse, it could provide access for internal network operations, making lateral movements possible within the enterprise. These actions can translate into both financial and reputational damage if not mitigated. Proactive detection and resolution are crucial in securing shared AI infrastructure from such adverse impacts.
REFERENCES
- Ensure that the `general_settings.master_key` is configured correctly in the LiteLLM proxy settings.
- Implement per-user virtual keys for accessing the LiteLLM API securely.
- Position the proxy behind an authenticated reverse proxy or within a secure service mesh.
- Regularly audit and update access controls to prevent unauthorized data leaks.
- Perform routine security checks and vulnerability assessments to identify potential weaknesses in configurations.
Get AI-powered remediation steps tailored to your asset.
Try AI Solutions →