Uppercase the ratio term R_t in PPO objective (#201)

zafstojano · web-flow · commit b4a47aaa98b3 · 2026-01-05T11:52:00.000-08:00
diff --git a/chapters/11-policy-gradients.md b/chapters/11-policy-gradients.md
@@ -287,7 +287,7 @@ $$
 J(\theta)
 =
 \mathbb{E}_{t}\left[
-\min\left(r_t(\theta)A_t,\ \text{clip}(r_t(\theta),1-\varepsilon,1+\varepsilon)A_t\right)
+\min\left(R_t(\theta)A_t,\ \text{clip}(R_t(\theta),1-\varepsilon,1+\varepsilon)A_t\right)
 \right],
 \qquad
 R_t(\theta)=\frac{\pi_\theta(a_t\mid s_t)}{\pi_{\theta_{\text{old}}}(a_t\mid s_t)}.

Original file line number	Diff line number	Diff line change
`@@ -287,7 +287,7 @@ $$`
`287`	`287`	`J(\theta)`
`288`	`288`	`=`
`289`	`289`	`\mathbb{E}_{t}\left[`
`290`		`-\min\left(r_t(\theta)A_t,\ \text{clip}(r_t(\theta),1-\varepsilon,1+\varepsilon)A_t\right)`
	`290`	`+\min\left(R_t(\theta)A_t,\ \text{clip}(R_t(\theta),1-\varepsilon,1+\varepsilon)A_t\right)`
`291`	`291`	`\right],`
`292`	`292`	`\qquad`
`293`	`293`	`R_t(\theta)=\frac{\pi_\theta(a_t\mid s_t)}{\pi_{\theta_{\text{old}}}(a_t\mid s_t)}.`