I find its very much a setup and design thing. My backcountry ti in 280ai is so pleasant to shoot in its current form it has become my primary rifle for anything 500yds and under unless Im out popping prarie dogs or punching paper.
That said a buddy of mine has a 7mm rem mag ruger that is about a pound and a half heavier and it is...down right punishing to fire even with a limbsaver pad.
I can certainly see how someone would have major issues developing a flinch firing that ruger. Sometimes I wonder if people having trouble would be well served adding a few ounces to the rifle in the form of a brake. It doesnt even have to be threaded, take some measurements with a caliper and send them in to grizzly gunworks and they will make you a precision machined clamp on brake though I really believe threading is a better option.
I would also agree that proper fitment is critical, if you aren't comfortable when its shouldered, its not like a 22, you just aren't going to shoot well.
I also believe practice builds confidence in the rifle and the overall setup. I believe confidence is a major part in landing that critical first shot when you are already hopped up on buck fever and worn down from hiking. You can have all the physics on a piece of paper in front of you, range finder says x distance, x drop at the distance, x drift from wind, dial this much, or hold over this much. The math all says it works, but if you don't have confidence, you second guess yourself and that shot isn't as good as it could have been or misses all together.
All that to say, get out and practice with that rifle. Find somewhere to stretch out beyond your comfort distance, and suddenly the distances you are planning to hunt at seem much shorter. When I was just starting out hunting years ago, anything beyond about 150yds seemed like no mans land. But that was back in Missouri where the average whitetail shot was 50-75yds. Then I stretched out to 200, then 250, then 300, 400, and 600 on coyotes in fields. Moved out west and started stretching out beyond that to 1000 all the way to 1700 and beyond shooting at steel and prairie dogs. Now anything under 500 yards seems paltry with one of the rifles I trust.
It can be done, but the more punishing the rifle the more practice it will take.