LLM Model Configuration Management

This document describes the centralized configuration management system for LLM client models.

Overview

The LLM configuration system provides a centralized, type-safe way to manage all LLM provider models, their costs, capabilities, and default parameters using YAML configuration files.

Features

Centralized Configuration: All model settings in one YAML file
Type Safety: Pydantic-based validation ensures configuration integrity
Hot Reloading: Support for reloading configuration without restarting
Flexible Model Management: Easy to add/modify models without code changes
Cost Tracking: Automatic cost estimation based on configured pricing
Model Capabilities: Track features like vision, streaming, function calling

Architecture

Configuration Files

Main Configuration

Path: aiecs/config/llm_models.yaml
Format: YAML
Environment Variable: LLM_MODELS_CONFIG (optional override)

Configuration Schema

providers:
  <provider_name>:
    provider_name: string
    default_model: string
    model_mappings:  # Optional: for model aliases
      <alias>: <actual_model_name>
    models:
      - name: string
        display_name: string  # Optional
        description: string   # Optional
        costs:
          input: float   # USD per 1K tokens
          output: float  # USD per 1K tokens
        capabilities:
          streaming: boolean
          vision: boolean
          function_calling: boolean
          max_tokens: integer
          context_window: integer
        default_params:
          temperature: float (0.0-2.0)
          max_tokens: integer
          top_p: float (0.0-1.0)
          top_k: integer

Usage

Loading Configuration

Configuration is automatically loaded on first use:

from aiecs.llm.config_loader import get_llm_config_loader

# Get the loader instance
loader = get_llm_config_loader()

# Get full configuration
config = loader.get_config()

# Get specific provider configuration
provider_config = loader.get_provider_config("OpenAI")

# Get specific model configuration
model_config = loader.get_model_config("OpenAI", "gpt-4-turbo")

# Get default model for a provider
default_model = loader.get_default_model("Vertex")

Hot Reloading Configuration

from aiecs.llm.config_loader import reload_llm_config
from aiecs.llm.client_factory import LLMClientFactory

# Reload configuration from file
config = reload_llm_config()

# Or reload via client factory
LLMClientFactory.reload_config()

Using Configured Models

The LLM clients automatically use the configuration:

from aiecs.llm.client_factory import LLMClientFactory

# Get a client - it will use configured defaults
client = LLMClientFactory.get_client("OpenAI")

# Generate text - uses default model from config
response = await client.generate_text(
    messages=[{"role": "user", "content": "Hello"}]
)

# Or specify a model explicitly
response = await client.generate_text(
    messages=[{"role": "user", "content": "Hello"}],
    model="gpt-4o"
)

Validating Configuration

# Run the validation script
poetry run python -m aiecs.llm.validate_config

This will:

Load and validate the configuration file
Check for errors and warnings
Display a summary of all configured providers and models

Configuration Management

Adding a New Model

Open aiecs/config/llm_models.yaml
Find the appropriate provider section
Add the model to the models list:

providers:
  openai:
    models:
      - name: "gpt-5"  # New model
        display_name: "GPT-5"
        description: "Next generation model"
        costs:
          input: 0.001
          output: 0.003
        capabilities:
          streaming: true
          vision: true
          function_calling: true
          max_tokens: 16384
          context_window: 256000
        default_params:
          temperature: 0.7
          max_tokens: 8192
          top_p: 1.0
          top_k: 0

Validate the configuration:

poetry run python -m aiecs.llm.validate_config

Reload configuration (optional, for hot-reloading):

from aiecs.llm.client_factory import LLMClientFactory
LLMClientFactory.reload_config()

Adding Model Aliases

For providers with model name variations (like xAI/Grok):

providers:
  xai:
    model_mappings:
      "Grok 4": "grok-4"
      "Grok 4 Normal": "grok-4"
    models:
      - name: "grok-4"
        # ... configuration

Updating Model Costs

When pricing changes, simply update the YAML file:

costs:
  input: 0.002  # Updated price
  output: 0.006  # Updated price

No code changes required!

Configuration Validation

The system validates:

Required Fields: All required fields must be present
Cost Values: Must be non-negative
Model Names: Must be unique within a provider
Default Model: Must exist in the models list
Model Mappings: Aliases must point to valid models
Parameters: Must be within valid ranges (e.g., temperature 0.0-2.0)

Warnings are issued for:

Zero costs (may be intentional for free tiers)
Default max_tokens exceeding capability max_tokens

Environment Variables

LLM_MODELS_CONFIG

Override the default configuration file path:

export LLM_MODELS_CONFIG=/path/to/custom/llm_models.yaml

In .env file

LLM_MODELS_CONFIG=/path/to/custom/llm_models.yaml

API Reference

LLMConfigLoader

Methods:

get_config(): Get full configuration
reload_config(): Reload from file
get_provider_config(provider_name): Get provider configuration
get_model_config(provider_name, model_name): Get model configuration
get_default_model(provider_name): Get default model name
is_loaded(): Check if configuration is loaded
get_config_path(): Get path to current config file

Configuration Models

LLMModelsConfig

Root configuration containing all providers

ProviderConfig

Configuration for a single provider
Contains models list, default model, and mappings

ModelConfig

Complete configuration for a single model
Contains costs, capabilities, and default parameters

ModelCostConfig

Token cost information (input/output)

ModelCapabilities

Model features and limits

ModelDefaultParams

Default inference parameters

Best Practices

Validation: Always run validation after modifying configuration
Version Control: Keep llm_models.yaml in version control
Cost Updates: Regularly update pricing information
Documentation: Add descriptions to new models
Testing: Test new models before deploying to production
Hot Reload: Use hot reload in development, restart in production

Troubleshooting

Configuration Not Found

Error: FileNotFoundError: LLM models configuration file not found

Solutions:

Ensure aiecs/config/llm_models.yaml exists
Set LLM_MODELS_CONFIG environment variable
Check file permissions

Validation Errors

Error: ConfigValidationError: ...

Solutions:

Run validation script to see detailed errors
Check YAML syntax (indentation, quotes)
Verify all required fields are present
Ensure costs are non-negative numbers

Model Not Found

Error: Model configuration not found for specific model

Solutions:

Check model name spelling
Verify model exists in YAML configuration
Check if model alias is correctly mapped
Reload configuration if recently added

Migration Guide

If you have existing hardcoded model configurations, they will continue to work as fallbacks. However, we recommend migrating to the new configuration system:

All model costs are now in the YAML file
Default models are configured per provider
Model aliases are centrally managed
Custom parameters can be set in configuration

The old token_costs dictionaries in client classes are deprecated but still functional for backward compatibility.

Examples

Complete Provider Configuration

providers:
  openai:
    provider_name: "OpenAI"
    default_model: "gpt-4-turbo"
    models:
      - name: "gpt-4-turbo"
        display_name: "GPT-4 Turbo"
        description: "Fast and capable GPT-4 variant"
        costs:
          input: 0.01
          output: 0.03
        capabilities:
          streaming: true
          vision: true
          function_calling: true
          max_tokens: 4096
          context_window: 128000
        default_params:
          temperature: 0.7
          max_tokens: 4096
          top_p: 1.0
          top_k: 0

Programmatic Access

from aiecs.llm.config_loader import get_llm_config_loader

loader = get_llm_config_loader()
config = loader.get_config()

# Iterate through all providers
for provider_name, provider_config in config.providers.items():
    print(f"Provider: {provider_name}")
    print(f"  Default: {provider_config.default_model}")
    
    # Iterate through models
    for model in provider_config.models:
        print(f"  Model: {model.name}")
        print(f"    Cost: ${model.costs.input} in / ${model.costs.output} out")
        print(f"    Max Tokens: {model.capabilities.max_tokens}")

Support

For issues or questions:

Check this documentation
Run the validation script
Review error logs
Check YAML syntax