Skip to content

    Inference that lets you ship a feature

    Run open models serverless or on Dedicated GPUs, through one OpenAI-compatible API.

    curl https://api.flex.ai/v1/chat/completions \
      -H "Authorization: Bearer $FLEXAI_API_KEY" \
      -H "Content-Type: application/json" \
      -d '{
        "model": "Meta-Llama-3.1-8B-Instruct-FP8",
        "messages": [{"role": "user", "content": "Hello from FlexAI"}]
      }'

    Built for agent loops

    The inference API gives your agent the primitives; AgentOS adds the run-level controls.

    From the inference API

    • ✓ Tool calls
    • ✓ Structured output
    • ✓ Streaming
    • ✓ Model fallback

    With AgentOS In trial

    • ✓ Eval harnesses
    • ✓ Approval logic
    • ✓ Per-run observability
    Explore AgentOS →

    One API shape, serverless or dedicated

    Start on shared per-token endpoints. When steady volume justifies your own GPUs, move to Dedicated GPUs on the same OpenAI-compatible API: point your client at your endpoint URL and key.

    import os
    from openai import OpenAI
    
    client = OpenAI(
        base_url="https://api.flex.ai/v1",
        api_key=os.environ["FLEXAI_API_KEY"],
    )
    
    response = client.chat.completions.create(
        model="Meta-Llama-3.1-8B-Instruct-FP8",
        messages=[{"role": "user", "content": "Hello"}],
    )

    Production visibility without guesswork

    Live usage metrics
    Latency, tokens, GPU, region, cache. In one view.
    Autoscaling
    Scale replicas between your min and max, down to zero.
    FlexAI Console: workload management dashboard

    Start with one endpoint

    Sizing a workload? Compare costs in the GPU savings calculator or see Dedicated GPUs for dedicated capacity.

    Frequently Asked Questions

    FlexAI is optimized for low-latency streaming inference. Actual latency depends on model, context length, hardware, region, and traffic pattern.

    FlexAI automatically scales GPU endpoints based on request volume. Endpoints scale between the min and max replicas you set, down to zero. You pay per minute for running replicas, and nothing when they scale to zero.

    Any model that runs on vLLM, SGLang, Triton or a custom container, including your own fine-tunes.

    More managed AI services: fine-tuning, training, and the platform overview.