LLM Configuration

Using Custom or Unsupported Models

NEX is developed, tested, and optimized against a specific set of supported models and configurations. Direct access to these models is provided because their behavior, capabilities, and limitations are well understood and known to work reliably with NEX.

When you configure a custom, third-party, quantized, experimental, or obsolete model, you do so at your own risk. Such models may lack required capabilities, behave inconsistently, or fail to follow instructions in ways that can partially or completely break NEX functionality.

By using a custom model, you assume full responsibility for its behavior and compatibility. Issues caused by unsupported models are not considered bugs in NEX, and support or guarantees of functionality cannot be provided in these cases.

Enterprise Customers using Locally Deployed LLMs

Enterprise deployments that rely on locally hosted, self-managed, or privately deployed LLMs must be configured, secured, and maintained by the customer’s internal IT or infrastructure teams.

ChipNexus does not provide troubleshooting, setup, networking, security, or performance support for customer-managed LLM environments by default. This includes (but is not limited to) issues related to authentication, connectivity, hardware, scaling, model performance, or inference reliability.

Support for these scenarios is only available when explicitly covered by a separate enterprise or professional services agreement.

NEX allows you to connect your own Large Language Model (LLM) provider instead of using the default gateway.

All providers share these common fields:

Variable	Description
`LLM_PROVIDER`	The backend provider name (e.g. `openai`, `google`, `bedrock`, `ollama`)
`LLM_API_KEY`	Your API key for the provider (use `any` for local models)
`LLM_MODEL`	The model identifier or name
`LLM_DISPLAY_NAME`	A friendly name shown in the NEX UI
`LLM_BASE_URL`	(Optional) Custom API base URL (mainly for local/self-hosted providers)
`LLM_PROVIDER_REGION`	(Optional) Cloud region for providers like AWS Bedrock

Info

Only one provider can be active at a time. Make sure you define a single LLM_PROVIDER block in your config.

Example Configurations

Below are ready-to-use examples for the most common providers.

Google (Gemini)

LLM_PROVIDER=google
LLM_API_KEY=your-api-key-here
LLM_MODEL=gemini-3-pro-preview
LLM_DISPLAY_NAME="Gemini 3 Pro"

OpenAI

LLM_PROVIDER=openai
LLM_API_KEY=your-api-key-here
LLM_MODEL=gpt5.2
LLM_DISPLAY_NAME="GPT 5.2"

AWS Bedrock (Anthropic Claude)

Warning

Make sure your AWS credentials and Bedrock permissions are properly configured in your environment before starting NEX.

LLM_PROVIDER=bedrock
LLM_API_KEY=your-api-key-here
LLM_MODEL=arn:aws:bedrock:us-east-1:456510618647:inference-profile/global.anthropic.claude-sonnet-4-6
LLM_PROVIDER_REGION=us-east-1
LLM_DISPLAY_NAME="Claude Sonnet 4.6"

Ollama and Other Local LLM Providers

Tip

Ensure your local LLM server is running before starting NEX.

LLM_PROVIDER=ollama
LLM_MODEL=gpt-oss:latest
LLM_BASE_URL=http://localhost:11434/v1
LLM_DISPLAY_NAME="GPT-OSS"
LLM_API_KEY=any

LLM Runtime Limits

NEX also supports three optional settings that control output size, internal reasoning budget, and the context window used for usage reporting.

Variable	Description
`LLM_MAX_TOKENS`	Maximum number of output tokens the model is allowed to generate for a response. This is useful for keeping answers shorter, reducing cost, or enforcing predictable response sizes.
`LLM_THINKING_BUDGET`	Maximum reasoning budget for providers and models that support extended thinking or reasoning traces. Higher values can improve complex reasoning, but may increase latency and token usage.
`LLM_THINKING_EFFORT`	Controls the qualitative reasoning effort level for models that support this parameter via an OpenAI-compatible API. Accepted values are `none`, `minimal`, `low`, `medium`, or `high`. This setting is only applicable to supported models and providers.
`LLM_CONTEXT_LENGTH`	Maximum context window of the configured model. This allows NEX to correctly report context usage in the interface. If not set, NEX defaults to `1_000_000`.

Notes

LLM_MAX_TOKENS applies as a general cross-provider response limit.
LLM_CONTEXT_LENGTH helps NEX track how much of the model's context window is being used and display it correctly in the UI.
LLM_THINKING_BUDGET is only used by providers that expose configurable thinking or reasoning budgets.
Not all models use the full configured thinking budget. Some models dynamically decide how much reasoning to use.
Higher thinking budgets may improve performance on difficult coding or analysis tasks, but usually increase cost and response time.
LLM_THINKING_EFFORT is only supported by certain models exposed through OpenAI-compatible APIs and may be ignored by providers that do not implement this parameter.

Configure Custom TLS Certificates

If your LLM endpoint uses a private CA, a self-signed certificate, or mutual TLS (mTLS), NEX can be configured to trust a custom certificate chain when connecting to your server.

The following TLS-related settings are supported:

Variable	Description
`TLS_VERIFY`	Enables or disables TLS certificate verification. Leave this enabled unless you are working in a temporary local test environment.
`TLS_CA_CERT_PATH`	Path to a custom CA bundle or PEM file used to verify the server certificate.
`TLS_CLIENT_CERT_PATH`	Path to a client certificate file for mTLS authentication.
`TLS_CLIENT_KEY_PATH`	Path to the private key that matches `TLS_CLIENT_CERT_PATH`.

Example: Trust a Custom CA Certificate

Use this when your server certificate is signed by an internal or private certificate authority:

LLM_PROVIDER=openai
LLM_API_KEY=your-api-key-here
LLM_MODEL=your-model-name
LLM_BASE_URL=https://your-llm-server.example.com/v1
TLS_CA_CERT_PATH=/absolute/path/to/ca-cert.pem

Example: Mutual TLS (mTLS)

Use this when your server requires both a trusted CA and a client certificate:

LLM_PROVIDER=openai
LLM_API_KEY=your-api-key-here
LLM_MODEL=your-model-name
LLM_BASE_URL=https://your-llm-server.example.com/v1
TLS_CA_CERT_PATH=/absolute/path/to/ca-cert.pem
TLS_CLIENT_CERT_PATH=/absolute/path/to/client-cert.pem
TLS_CLIENT_KEY_PATH=/absolute/path/to/client-key.pem

Disable TLS Verification

Warning

TLS verification should normally remain enabled. Disabling verification reduces connection security and should only be used for short-lived local testing.

Info

Custom TLS settings are applied when using OpenAI-compatible endpoints configured through LLM_BASE_URL. If TLS_CLIENT_CERT_PATH is provided without TLS_CLIENT_KEY_PATH, NEX will only succeed if the certificate file also contains the private key.

Only use this for local development or troubleshooting:

TLS_VERIFY=false