Hardware & studio

Understanding the machine, down to the silicon

Quality software is born from a fine understanding of what runs it: silicon, memory, thermal flows. That hardware culture informs every architecture decision I make.

Scroll

01 · Acceleration

Hardware for AI

GPU, VRAM, inference, training: knowing where every teraflop goes to size machines that actually serve the models.

GPU

The massively parallel compute engine

  • Ada Lovelace, Blackwell, RDNA architectures: what actually changes from one generation to the next.
  • CUDA cores for general-purpose compute, Tensor cores for the matrix multiplications behind neural nets.
  • FP16, BF16 and FP8 precision: trading raw speed against numerical stability.
  • PCIe 5.0 and NVLink: interconnect bandwidth dictates how well multi-GPU scales.
FP16 throughput (dense Tensor cores)
RTX 4090≈ 330 TFLOPS
RTX 5090≈ 420 TFLOPS
H100 SXM≈ 990 TFLOPS

VRAM

The number-one constraint for local models

  • Model size dictates everything: weights, activations and KV cache must fit in memory together.
  • Q4/Q8 quantization (GGUF, AWQ): cutting the memory footprint by three with controlled quality loss.
  • The KV cache grows with context: 128k tokens cost extra gigabytes of memory.
  • CPU/NVMe offloading: possible, but every offloaded layer is paid for in latency.
VRAM required for inference (Q4)
7B model≈ 5 GB
13B model≈ 9 GB
70B model≈ 40 GB

Inference

Serving fast, serving right

  • First-token latency vs sustained throughput: two different optimizations, two architectures.
  • Continuous batching (vLLM, TGI): sharing the GPU across requests without blowing up latency.
  • Local servers: llama.cpp and Ollama make private inference viable on a workstation.
  • Memory bandwidth, not compute, is what caps throughput in tokens per second.
Local ballpark figures
7B Q4 · RTX 4090≈ 120 tokens/s
70B Q4 · 2 × 24 GB≈ 15 tokens/s
Target first token< 300 ms

Training

Where hardware becomes a system

  • Multi-GPU: data, tensor or pipeline parallelism depending on model size and memory budget.
  • NVLink and InfiniBand interconnects: feeding the GPUs faster than they can compute.
  • Cooling and power delivery: at 700 W per card, thermals become a design decision.
  • Checkpointing to NVMe: saving often without strangling the training loop.
Fine-tuning: typical VRAM
7B · LoRA1 × 24 GB
13B · QLoRA1 × 24 GB
70B · LoRA4 × 80 GB

02 · Anatomy

Anatomy of a machine

Seven components, one balance: every part is chosen for the role it plays in the chain — not for its spec sheet.

The conductor: it sequences everything the machine executes. Clock speed, core count and L3 cache determine system responsiveness and compile times.

  • Performance vs efficiency cores
  • L3 cache: what compilation feeds on
  • Real TDP under sustained load

01 / 07

03 · The studio

The studio

The home workstation: built for focus, calibrated for color, silent under load.

Studio points of interest

  1. Main display

    27-inch 4K panel probe-calibrated every month: the pixel I see is the pixel the client will see.

    27" · 4K · 99% DCI-P3

  2. Workstation

    Main machine, hand-built: silent at idle, stable under prolonged inference load.

    Ryzen 9 · 64 GB DDR5 · RTX 4090 24 GB

  3. Peripherals

    Low-profile mechanical keyboard and vertical trackball: the ergonomics of a gesture repeated eight hours a day.

    Silent switches · vertical trackball

  4. Audio

    Nearfield monitoring and a dynamic mic for calls: clean sound with zero post-processing.

    5" monitors · XLR dynamic mic

  5. Lighting

    Bias lighting behind the display with adjustable temperature: less strain, better contrast perception.

    2700–6500 K · CRI > 95

  6. Local server

    A silent node under the desk: local CI, encrypted backups and open-source models served over the network.

    Proxmox · 32 GB ECC · 2 × 2 TB NVMe

A well-designed machine, well-delivered software

Let's talk about your product — the infrastructure will follow.

Let's work together

© 2020–2026 Davy Nana — All rights reserved

Legal noticePrivacyClient space
--:--