A
Azul AI
End-to-end fine-tuning & inference platform
Private beta

Private Local AI,
built end to end

Fine-tune, evaluate, and serve your own LLMs on your own GPUs. Datasets, training, RAG, and inference in one unified platform. Your data and your models never leave your machines.

SFT + RAFT + DPO · LoRA / QLoRA · RAG over pgvector

AI model visualization

The Platform

Not a wrapper around an API. Purpose-built for private, local LLM development.

Train models bigger than your GPU

A deterministic placement planner pools VRAM across mismatched GPUs, so 7B–14B LoRA and QLoRA jobs fit on the hardware you already own. No NVLink required.

The full fine-tuning toolkit

Supervised fine-tuning and Direct Preference Optimization with LoRA / QLoRA adapters and GPU-aware quantization, with every batch's loss retained uncapped for post-run analysis.

Runs that survive the night

Automatic checkpointing, heartbeats, and exact-state resume mean a multi-hour run picks up right where it left off instead of restarting from epoch one.

Know your model actually got better

Held-out probe suites graded by exact, numeric, and LLM-as-judge methods with median-of-3 sampling to cut judge variance, then charted as a pass-rate curve across a training campaign's stages.

Serve it the moment it's trained

Fine-tuned and base models served over OpenAI-compatible vLLM endpoints, with a control-plane arbiter that keeps training and inference from ever colliding on the same GPU.

Ground models in your own documents

Knowledge bases ingest your documents, chunk them, and store embeddings in Postgres pgvector for retrieval-augmented generation alongside your fine-tuned models.

Synthetic data
dataset generation
Generate persona and custom SFT / DPO training sets, no hand-labeling
Hugging Face
model export
Push fine-tuned checkpoints straight to the Hub
A/B compare
inference comparison
Fine-tuned vs base completions on the same prompt, side by side

Real runs, real numbers

Live figures from the training and evaluation datasets behind an actual fine-tuning campaign run on the platform. No mockups.

Held-out evaluation suite
180 contamination-free probes, stratified across 12 capabilities, graded by an LLM-as-judge with median-of-3 sampling.
DPO preference pairs by axis
573 chosen/rejected pairs, voice-balanced, targeting eight alignment behaviors.
  • Voice42.9%
  • Grounding25.3%
  • Reconciliation8.7%
  • Identity8%
  • Answerability4.9%
  • Privacy3.8%
  • Embellishment3.5%
  • Abstention2.8%
Adversarial stress suite
108 harder probes across seven attack categories: multi-hop inference, contradiction defense, and attribution boundaries.

Cloud convenience. On-prem privacy.

A control plane that runs anywhere with no GPU, and a data plane that runs on your own workstation or datacenter. They communicate only over a queue and HTTP. This means your hardware can live in a homelab, behind a tunnel, or in your own VPC.

Control plane

No GPU
  • Next.js 15 web
    Dashboard, training campaigns, eval grader, playground
  • Go REST API
    Auth, multi-tenant projects, job orchestration
  • Inference arbiter
    Pauses/resumes vLLM around dual-GPU jobs
HTTP

Data plane

GPU workstation
  • Python GPU worker
    PyTorch + Transformers training pipeline
  • vLLM servers
    OpenAI-compatible serving, fine-tuned + base
  • Container manager
    Swaps served checkpoints, restarts vLLM

Shared backbone

PostgreSQL + pgvector
State, metrics, 384-dim embeddings
Runner protocol
Job manifest + progress callbacks any GPU box speaks
AWS S3
Datasets & model artifacts over presigned URLs

Start with one workstation and enroll more GPU boxes as you grow. The same control plane drives the whole fleet.

SB
Sander Blue
Builder of Azul AI

A whole platform, one codebase

Azul AI is a solo project currently in private beta. The entire stack, from the GPU scheduler to the dashboard, lives in one codebase maintained by one person. That keeps the platform coherent and quick to fix, and it means your feedback goes straight to the person who wrote the code.

  • Every layer in one place: GPU scheduling, Go API, Python training worker, vLLM serving, and the dashboard
  • Built for real hardware: heterogeneous dual-GPU model parallelism, GPU-GPU communication, RNG-exact checkpoint resume, and GPU arbitration between training and serving
  • The full ML loop, end to end: dataset design, SFT + DPO fine-tuning, and LLM-as-judge evaluation