Just like with large cartridge rifles, it is all in the follow-through. If you lock your wrists and have a death-grip on the handgun, you are in for a bad day. Also, similar to rifle stocks, the design of the stock will help. Center-grips are easier to control than rear-grips.
Interestingly, it was my love of hunting with handguns that lead me to smaller chamberings for rifles. If I could effectively kill deer at 250 yards with a 14" barreled 7TCU, then surely my 7-08 was effective to 500, so why was I using a 7RM?