· The journal

Notes from the road

Experiments, lessons, and half-formed thoughts from the workshop.

Adventures with PCIe speeds, topology and RCCL for LLM inference

I recently had the privilege to upgrade my main workstation with two more GPUs. With this, I had to figure out how to squeeze them in the limited space of my workstation chassis.

Read note ↗

Optimizing CPU-only inference for better performance

I recently came upon a use case where I needed a small LLM model to generate summaries and parse some messaging data. It was time to explore CPU only inference.

Read note ↗

Welcome to the hearth

A small introduction to this new corner of the internet, and the kinds of things I hope to leave here.

Read note ↗