Agentic AI Runtime Server

1System architecture

Complete stack deployed on a single EC2 g4dn.2xlarge instance with GPU acceleration.

Important system paths

# Main runtime
/opt/agentic-runtime/
  app/                    # FastAPI code
    main.py
    config.py
    routers/              # health, agents, crews
    graphs/               # LangGraph state machines
    crews/                # CrewAI crews
  venv/                   # Python 3.12 with deps
  data/                   # SQLite checkpoints
  logs/                   # api.log, api-error.log
  docker-compose.yml      # Open WebUI

systemd services

agentic-api.service   -> uvicorn app.main:app on :8000
open-webui.service    -> docker compose up (Open WebUI)
ollama.service        -> LLM runtime on :11434

2API Endpoints

REST API documented with OpenAPI. Swagger UI available at /docs.

Request schema

# POST /agents/run
{
  "query": "string",        # Question or task
  "thread_id": "string"     # Session ID (for checkpoints)
}

# POST /crews/run
{
  "query": "string"         # Task for the crew
}

3curl Examples

Healthcheck

curl -s http://localhost:8000/health

# Response:
{"status":"ok","service":"agentic-ai-runtime-server","version":"1.0.0"}

List models

curl -s http://localhost:8000/models | jq

Run LangGraph agent

curl -X POST http://localhost:8000/agents/run \
  -H "Content-Type: application/json" \
  -d '{
    "query": "Explain how LangGraph works",
    "thread_id": "demo-001"
  }' | jq

Run CrewAI crew

curl -X POST http://localhost:8000/crews/run \
  -H "Content-Type: application/json" \
  -d '{"query": "Research Kubernetes"}' | jq

4systemd service management

Basic commands

# Status
sudo systemctl status agentic-api open-webui ollama

# Restart
sudo systemctl restart agentic-api
sudo systemctl restart open-webui

# Stop
sudo systemctl stop agentic-api

# Check if it starts at boot
sudo systemctl is-enabled agentic-api open-webui ollama

agentic-api service structure

# /etc/systemd/system/agentic-api.service
[Unit]
Description=Agentic AI Runtime Server API
After=network.target ollama.service
Wants=ollama.service

[Service]
Type=simple
User=ubuntu
WorkingDirectory=/opt/agentic-runtime
EnvironmentFile=/opt/agentic-runtime/.env
ExecStart=/opt/agentic-runtime/venv/bin/uvicorn \
  app.main:app --host 0.0.0.0 --port 8000
Restart=always
RestartSec=10

[Install]
WantedBy=multi-user.target

5Logs and monitoring

API Logs

# Real-time via journald
sudo journalctl -u agentic-api -f

# Last 100 lines
sudo journalctl -u agentic-api -n 100 --no-pager

# Errors today
sudo journalctl -u agentic-api --since today -p err

# Logs to file
tail -f /opt/agentic-runtime/logs/api.log
tail -f /opt/agentic-runtime/logs/api-error.log

Open WebUI Logs (Docker)

sudo docker logs -f open-webui
sudo docker logs --tail 100 open-webui

Ollama Logs

sudo journalctl -u ollama -f

GPU monitoring

# Snapshot
nvidia-smi

# Watch every 2 seconds
watch -n 2 nvidia-smi

# Processes using GPU
nvidia-smi --query-compute-apps=pid,process_name,used_memory --format=csv

System resources

htop                        # CPU and RAM
df -h                       # Disk
sudo ss -tulpn              # Open ports
sudo docker stats           # Container resources

6Ollama model management

List installed models

ollama list

Download new model

# Recommended models for T4 (16 GB VRAM)
ollama pull qwen2.5:14b         # ~9 GB - complex reasoning
ollama pull deepseek-coder:6.7b # ~4 GB - code
ollama pull llama3.2-vision:11b # ~8 GB - vision (multimodal)

# Small models (fast)
ollama pull phi3:mini
ollama pull gemma2:2b

Check available VRAM

nvidia-smi --query-gpu=memory.used,memory.free,memory.total --format=csv

Remove model

ollama rm model-name

Quick inference test

ollama run llama3.1:8b "Respond only: OK"
# Should respond in 1-3 seconds

7Customize agents

Modify LangGraph graph

Edit /opt/agentic-runtime/app/graphs/research_graph.py. Current nodes are planner, researcher, writer. You can add more or change prompts.

# Example: add a "critic" node before the writer
def critic_node(state: ResearchState):
    prompt = f"Critique this research:\n{state['research'][-1]}"
    response = llm.invoke(prompt)
    return {"research": [response.content]}

g.add_node("critic", critic_node)
g.add_edge("researcher", "critic")
g.add_edge("critic", "writer")

Modify CrewAI crew

Edit /opt/agentic-runtime/app/crews/research_crew.py. Change role, goal, backstory and description of each Task.

Adjust the model used

# Edit .env
sudo nano /opt/agentic-runtime/.env

# Change models
ROUTER_MODEL=llama3.2:3b
MAIN_MODEL=qwen2.5:14b        # Change this
EMBED_MODEL=nomic-embed-text

# Restart to apply
sudo systemctl restart agentic-api

Add a new endpoint

# Create /opt/agentic-runtime/app/routers/custom.py
from fastapi import APIRouter

router = APIRouter(prefix="/custom", tags=["custom"])

@router.get("/ping")
async def ping():
    return {"pong": True}

# Register in main.py
from app.routers import custom
app.include_router(custom.router)

# Restart
sudo systemctl restart agentic-api

8Troubleshooting

Quick diagnostic

# 1. Active services
systemctl is-active agentic-api open-webui ollama

# 2. Listening ports
sudo ss -tulpn | grep -E ':(3000|8000|11434)'

# 3. GPU accessible
nvidia-smi

# 4. Docker with GPU
sudo docker run --rm --gpus all nvidia/cuda:12.1.1-base-ubuntu22.04 nvidia-smi

# 5. API responds
curl -s http://localhost:8000/health | jq

# 6. Ollama responds
curl -s http://localhost:11434/api/tags | jq

9Backup and restore

What to back up

• /opt/agentic-runtime/app/ - your customized code
• /opt/agentic-runtime/.env - configuration
• /opt/agentic-runtime/data/ - LangGraph SQLite checkpoints
• ollama list - model names (models will re-download)
• Docker volume open-webui-data - chat history

Full backup

# Create tarball
sudo tar -czf /tmp/agentic-backup-$(date +%Y%m%d).tar.gz \
  /opt/agentic-runtime/app \
  /opt/agentic-runtime/.env \
  /opt/agentic-runtime/data

# Backup Docker volume
sudo docker run --rm \
  -v open-webui-data:/data \
  -v /tmp:/backup \
  alpine tar czf /backup/open-webui-data.tar.gz -C /data .

# Download locally
scp ubuntu@your-ip:/tmp/agentic-backup-*.tar.gz ./

Restore

sudo tar -xzf agentic-backup-YYYYMMDD.tar.gz -C /
sudo systemctl restart agentic-api