Skip to content
All workCase study 02 / 05

AI platform · Engineering teams · 2026

Agentic AI that engineers trust

Engineers lost weeks to repetitive tasks and scattered documentation. I designed and led an agentic AI platform on private GPUs, with the guardrails that made people trust it. Its adoption led to a dedicated team.

vLLM logo with the platform diagram, the stack and the key figures around it
Role
Platform architect and project lead
Built for
The engineering teams
Duration
2026
Stack
Kubernetes · NVIDIA GPU Operator · vLLM

  • 1 team

    created to run and grow it, after its adoption

  • Private GPUs

    open models served in-house, no data sent to a third-party API

  • Traceable

    access, costs and usage, by design

01 · The need

Engineering teams lost time on repetitive, low-value tasks, some of which dragged on for months, and the information they needed was scattered across many tools. AI could take that work off their hands, but only if they trusted it.

02 · Constraints and decision

Constraints
  • Data · nothing leaves the company; models run on internal GPUs.
  • Trust · engineers use agents they can control: who has access, what it costs, what it did.
  • Fit · the platform plugs into the tools teams already use.
Options
  • A hosted AI assistant · quick to start, but data leaves the company and costs stay opaque.
  • A private agentic platform · more to build, but data, costs and usage stay under control.
Decision
  • Models · open models served with vLLM on Kubernetes, on internal GPUs with the NVIDIA GPU Operator.
  • Agents · reusable skills and agent workflows with OpenCode and LangGraph, for precise engineering tasks.
  • Knowledge · a RAG with RAGFlow over the scattered documentation, so answers come from one place.
  • Trust · access control with Keycloak, visible costs, traceable usage and a few simple rules.

03 · Architecture

The platform

Engineers reach the platform over HTTPS through Keycloak; agents and skills built with OpenCode and LangGraph send prompts to open models served by vLLM on internal GPUs with the NVIDIA GPU Operator, and get context from a RAG built with RAGFlow over the documentation; GPU metrics feed a costs and usage view
Access control at the door, open models on private GPUs, one place for answers, and every call traced. Click to enlarge.
  • Kubernetes
  • NVIDIA GPU Operator
  • vLLM
  • LangGraph
  • OpenCode
  • RAGFlow
  • Keycloak

04 · Result

  • Used daily by several engineering teams.
  • Its adoption led to a dedicated team to run and grow the platform.
  • Repetitive tasks handed to agents, and answers found in one place instead of many.

05 · What I learned

  • The hard part of agentic AI is not the model, it is trust: people adopt agents they can control, measure and explain.

Next case study

Alfa-Safety · 2024 – 2025Collecting and analyzing 2 TB of logs a week
Elasticsearch logo with the platform diagram, the stack and the key figures around it
Open to AI Platform, DevOps and Forward Deployed Engineer rolesLet’s build what runs next.contact@stevenoumi.com