Will upgrading to 32GB of RAM slow my boot times?

No. Boot time is almost entirely a function of your storage and CPU, not how much RAM you have. Modern SSDs load your OS in seconds, and any extra initialization time from more RAM is negligible. If you’re worried about boot times, upgrade to an NVMe SSD instead.