/ Recent content on tim's thoughts Hugo en-us © timothyckl. All rights reserved. Sun, 07 Jun 2026 10:53:28 +0800 /posts/building-a-tiny-gpt-from-scratch/ Sun, 07 Jun 2026 10:53:28 +0800 /posts/building-a-tiny-gpt-from-scratch/ <p>In this series, I&rsquo;ll build a small GPT-2-style language model from scratch, train it on a subset of the <a href="https://huggingface.co/datasets/HuggingFaceFW/fineweb-edu">HuggingFaceFW/fineweb-edu</a> dataset, and use it as a baseline for experiments in training, inference optimisation, architecture changes, and mechanistic interpretability.</p> <p>The aim is to understand the full language modelling pipeline from end-to-end: data, tokenisation, model architecture, training, generation , evaluation, inspection.</p> <h2 id="who-show-read-this">Who show read this</h2> <p>This series is for developers with some programming and machine learning experience who want to understand decoder-only Transformers by building, training, modifying, and inspecting a small GPT-2- style language model.</p> /posts/writing-latex/ Fri, 05 Sep 2025 13:15:40 +0800 /posts/writing-latex/ <h2 id="background">Background</h2> <p>As a university student in the field of computer science, my workflow largely revolves around note-taking, programming, and writing math equations. I heavily rely on Neovim and its plugins as my primary development environment due to its versatility and efficiency.</p> <p>Currently, I write my math notes and tutorials on an iPad, but I would prefer documents that look more professional. After some googling I discovered LaTeX and it&rsquo;s superior formatting and ability to handle equations. However, one gap in my workflow was handing LaTeX documents directly within Neovim without switching to external software.</p>