Is it normal for an extension cord to feel warm?

Yes, that’s normal. A 12-gauge, 50-foot cord running 1800 watts for hours will get slightly warm due to electrical resistance. If it’s warm like a mug of tea you can hold comfortably, you’re fine. If it’s hot enough that you can’t keep your hand on it, or the insulation feels soft or smells weird, replace the cord.