The anchor is March 31, 33 AD — a known Passover. Every other Passover is a whole number of lunar months away from that date. The math gives you two candidates each year — one before March 31 and one after, roughly 15 days either side.
But Passover by biblical law must fall in the month of Abib — the spring month. Spring begins at the vernal equinox, which the calendar fixes at March 21.
So the rule is: if a candidate date falls before March 21, spring hasn't arrived yet — which means it can't be Abib — which means it can't be Passover. It gets eliminated.
The math doesn't prevent dates before March 21. The definition of spring does.