Does a lighted extension cord tell me if it's safe to use?

No. The light only shows that the outlet has power and the cord is making a connection. It does not mean the cord is safe, undamaged, or rated for the load. A frayed cord can still light up until it shorts out. Check for cuts, kinks, and proper gauge separately.