That website almost had the fix but it gave me enough info to find the actual problem.
There are two registry keys involved.
The website had me set HKEY_CURRENT_USER\Control Panel\Keyboard\InitialKeyboardIndicators to 2 to turn it on but mine was already 0 (which is supposed to be off) so that wasn’t it.
The Windows Annoyance website also listed a second key HKEY_USERS\.DEFAULT\Control Panel\Keyboard\InitialKeyboardIndicators and it mine a value of 2 in it. That was my problem. I set it to zero and all is well.
Here is a list of valid values:
Change the value to any of the following:
0 - all indicators off
1 - Caps Lock on
2 - Num Lock on
4 - Scroll Lock on
Or, combine them by adding the corresponding values:
3 - Caps Lock and Num Lock on
5 - Caps Lock and Scroll Lock on
6 - Num Lock and Scroll Lock on
7 - Caps Lock, Num Lock, and Scroll Lock on