Key changes working only boils down to paying attention to how to resolve, and you're right, it's not that difficult.Key changes are minor. Tempo is key. I just posted a medley we did and the songs were in different keys. If it's that difficult for your (or any) band to do, then don't do them. Maybe it's just me but I don't see the difficulty.
Tempo can be more awkward, but then that's where a lot of the fun comes from!
If it's just not working, then figure it out until it does.