Planning and Learning with Tabular Methods

A visualization of planning and learning in Reinforcement Learning

This post captures my takeaways from chapter 8 of the book Reinforcement Learning: An Introduction by Sutton and Barto, the bible of RL.

Outline

Models and Planning

Dyna - Planning + Action + Learning

Prioritized Sweeping

Expected vs Sample Updates

Trajectory Sampling

Real-time DP (RTDP)

Planning at Decision Time

Heuristic Search

Rollout Algorithms

Monte Carlo Tree Search (MCTS)

References

Sutton, R. S., & Barto, A. G. (2018). Reinforcement Learning: An Introduction.

Tools

Image Generation: ChatGPT
Editor: Antigravity
Interactive Demo: Google AI Studio

Citation

Cited as:

Shanmukha Sainath. "Planning and Learning with Tabular Methods". TensorWrites (Jul 2026). https://www.tensorwrites.com/posts/plan-learn-with-tabular-methods

BibTeX:

@article{planlearnwithtabularmethods2026,
  title   = "Planning and Learning with Tabular Methods",
  author  = "Shanmukha Sainath",
  journal = "TensorWrites",
  year    = "2026",
  month   = "Jul",
  url     = "https://www.tensorwrites.com/posts/plan-learn-with-tabular-methods"
}