Timeline
Every published field note in order—what I was learning, building, and trying to explain at the time.
2026
12 entries
AresSim - Mars Survival Simulation & RL Environment
AresSim is a 32×32 Mars survival grid I built from scratch to test RL algorithms and LLM agents: a rover explores terrain, extracts ice, and keeps a 5×5 colony pad alive under weather, battery drain, in seed-reproducible worlds.
Trisynapse Memory - An Evidence-First Memory Engine for Agents
Trisynapse is a local-first memory engine for agents that keeps source evidence, retrieves the useful parts, and answers with citations.
Planning and Learning with Tabular Methods
A deep dive into planning and learning with tabular methods for Reinforcement Learning, focusing on value iteration and policy iteration.
TD Learning - First step to Unification of Monte Carlo and DP
A deep dive into Temporal Difference (TD) Learning for Reinforcement Learning, an idea that forms basis for modern deep reinforcement learning algorithms
Monte Carlo Methods - Learning from Experience
A deep dive into Monte Carlo methods for Reinforcement Learning, focusing on learning optimal strategies from raw experience without a model.
DotLM-165M: How I trained a 165M parameter language model from scratch
DotLM is a 165M parameter reasoning-capable SLM trained for all four stages of language modeling: Pretraining, Instruction Tuning, Alignment, and Reasoning using synthetically generated STE dataset.
Dynamic Programming - Planning with a Perfect Model
Introduction to Dynamic Programming: Policy Evaluation, Policy Iteration, Value Iteration and Generalized Policy Iteration
Finite Markov Decision Processes - Sequential Decision Making
Introduction to sequential decision-making in reinforcement learning, including agent–environment interaction, rewards, returns, value functions, and Bellman equations with intuitive example.
From Zero to 250M: How I Built the STE Dataset to Train a Tiny LLM from Scratch
The story of building a Simple Thoughts Experiments (STE) dataset from scratch.
Multi-armed Bandits : Introducing evaluative aspect to RL
Understanding the multi-armed bandit problem and how to solve it using reinforcement learning alongwith interactive demo. Chapter 3 of Reinforcement Learning: An Introduction by Sutton and Barto
From Minimax to Reinforcement Learning : Why RL wins at Tic-Tac-Toe ?
Understanding how different algorithms plays Tic-Tac-Toe and why RL wins at it. Interactive demo included!
Introduction to Reinforcement Learning
A comprehensive introduction to reinforcement learning from the book : Reinforcement Learning: An Introduction by Sutton and Barto