Battery light stays on after changing starter

Most likely a wire at the starter was overlooked and was not reconnected. GM likes to use the large "battery" terminal on the starter solenoid as a convenient tie point for other circuits. In this case, the generator's fat output wire is connected to that large stud on the starter solenoid.

To verify this, measure the voltage on the large output terminal on the back of the generator. You should find full battery voltage there all the time. If it is missing, search around the starter for the wire that did not get reconnected.