Setting up the Virtana AI Chatbot

Copilot reads its connectivity settings from six environment variables. The three inference variables are required for every deployment; the three embedding variables are optional and inherit the inference values when you leave them unset.

The following table describes each variable:

Virtana recommends keeping embeddings on OpenAI regardless of which provider handles inference. The Copilot document search store is indexed with the OpenAI text-embedding-3-large model at 1024 dimensions, and any other embedding model produces vectors that the existing store can't match.

Connect directly to a public provider when your Global View deployment can reach the provider's API, and you don't need centralized credential management or request logging. Copilot resolves its model aliases to that provider's model IDs and calls the provider's default endpoint, so you only set the provider and the key.

The following example configures a direct OpenAI connection:

export VIRTANA_AI_MODEL_PROVIDER="OPENAI"
export VIRTANA_AI_API_KEY="<openai_api_key>"

To use Gemini instead, set VIRTANA_AI_MODEL_PROVIDER to GEMINI and supply a Gemini API key.

Set VIRTANA_AI_MODEL_PROVIDER to LITELLM when you want your proxy configuration, rather than Copilot, to decide which upstream model serves each workload. Copilot sends its internal aliases as literal model names, so your proxy holds the complete alias-to-model mapping, and you can change any mapping without touching Global View. Virtana recommends this pattern for proxy deployments.

The following example configures Copilot to send aliases to a LiteLLM proxy while keeping embeddings on OpenAI:

export VIRTANA_AI_MODEL_PROVIDER="LITELLM"
export VIRTANA_AI_API_KEY="<litellm_proxy_key>"
export VIRTANA_AI_INFERENCE_ENDPOINT="http://<litellm_host>:4000/v1"

export VIRTANA_AI_EMBEDDING_PROVIDER="OPENAI"
export VIRTANA_AI_EMBEDDING_API_KEY="<openai_api_key>"
export VIRTANA_AI_EMBEDDING_ENDPOINT="https://api.openai.com/v1"

Your litellm_config.yaml file must define a model_name entry for every alias listed in model aliases and their resolved model IDs. A request for an alias that the proxy doesn't define returns a 4xx "model not found" error, and the Copilot feature that uses that alias stops working.

Set VIRTANA_AI_MODEL_PROVIDER to OPENAI or GEMINI and point VIRTANA_AI_INFERENCE_ENDPOINT at your LiteLLM proxy when the proxy already exposes standard provider model names. Copilot resolves each alias to a provider model ID before it sends the request, so the proxy sees names such as gpt-4o-mini rather than VIRTANA_MDL_MINI.

The following example sends OpenAI model IDs to a LiteLLM proxy:

export VIRTANA_AI_MODEL_PROVIDER="OPENAI"
export VIRTANA_AI_API_KEY="<litellm_proxy_key>"
export VIRTANA_AI_INFERENCE_ENDPOINT="http://<litellm_host>:4000/v1"

Set VIRTANA_AI_API_KEY to the LiteLLM proxy key in this pattern, not to your OpenAI or Gemini key. The proxy authenticates Copilot with its own key and uses the upstream provider key from its own configuration.

Copilot refers to models by alias throughout the application, and resolves each alias at runtime based on VIRTANA_AI_MODEL_PROVIDER. Aliases are grouped by workload: MINI aliases handle fast, low-cost tasks, REASON aliases handle analysis and generation, NANO aliases handle identifier extraction, and VIRTANA_MDL_EMBEDDING handles document indexing and search.

The following table lists every alias, the model each provider resolves it to, and the model_name value your LiteLLM proxy must expose when VIRTANA_AI_MODEL_PROVIDER is LITELLM:

You can override Copilot's default model choices with environment variables, which is useful when you want one workload to use a stronger or cheaper model than the rest. Two levels of override are available, and the variables you set apply on top of whatever provider mapping is already in effect.

Choose the level that matches what you want to change:

Model parameters such as temperature and token limits live in the chatbot_modules/chatbot.conf file, organized into one section per workload. Each section names a model, either as a VIRTANA_MDL_* alias or as a concrete provider model ID, along with the parameters Copilot sends with each request to that model.

The following parameters are supported in a chatbot.conf section:

Copilot resolves its configuration in two independent layers, and knowing which layer wins tells you where to make a change. Connectivity decides where requests go and which key authenticates them; configuration decides which model name and parameters each request carries.

The following rules describe which source takes precedence in each layer: