That’s just unarguable physics. Any argument to the contrary is simply logic-defying word salad of a person in denial on the subject.
In your own words, please describe what "unarguable physics" are at play here.
Regarding medium sized game, is the recoil of a .264 caliber rifle so much more than a .243 caliber or even a .223 caliber rifle that it significantly affects accuracy? Are most competent big game hunters significantly affected that much more by the recoil of a 6.5 Creedmoor over a 6mm Creedmoor or .223? So much so that their accuracy suffers in a meaningful way?
IMHO? No.
What is your definition of "meaningful"? A 1MOA difference in accuracy due to recoil, plus a 1 MOA rifle system, plus a 2MOA shift in accuracy due to poor field positioning and/or wind means that you could be off by as much as 8 inches at 200 yards. That shrinks to 6 inches if you remove the recoil factor. That is the difference between hitting the liver/back of the lungs or hitting the guts. Seems like a pretty big deal to me.
Are the benefits of more accuracy so important that they outweigh the benefits of larger calibers on big and/or dangerous game like moose,elk, and grizzlies? Does an honest cost/benefits analysis really shake out in favor of, say, using a 6.5 Creedmoor over a .300 or .338 Win Mag (or even a .270 win or 7mm Rem Mag) for grizzly? Really?
Again, IMHO? No.
What is the downside of using said 6.5 Creedmoor on a moose, elk, or bear? How hard do you think they are to kill?
What about the “tracking the shot” argument?
My take? The importance of the first shot can hardly be overstated. Because any experienced hunter knows that the vast majority of the the time all bets are off after the first shot. Most of the time the animal is on the move after the first shot. If the game stays put, you are lucky. Even then, being human, most rush follow up shots. All the “planning” done in the comfort of book study or at the range then goes out the window. Making a caliber choice on this basis is planning for failure. And planning for failure is never good. Ever.
Being able to see your shot has a ton of benefits beyond "planning for failure". When I see the bullet enter the vital-V, I know I can pack up my stuff and call for the meat wagon. It is just a matter of getting over to where the animal was and find it. If I see it hit a bit back because I flubbed the wind call, I know to make a very solid mental note (or pin it on OnX) and back out for a while before I go busting in to find the animal, potentially sending it into the next county when it could have just walked 100 yards and bedded down.
On the whole "planning for failure" subject, if you don't plan for when things go wrong, you are a fool. You train for success, you plan for failure. Otherwise you will not know what to do when things go wrong. And if you haven't had anything go wrong when shooting an animal, you haven't shot enough animals.
My point: beware of “fads” because often they dispense with a HONEST cost/benefits analysis - overemphasizing some factors and downplaying others to reach a desired conclusion. It was the same years ago when giant magnums were all the rage for the “flat-shooting wind bucking” abilities of 200 plus grain bullets. Back then, concerns about recoil were scoffed at as “unmanly” and “sissified”.
And my point is a lot of things were labeled "fads" by people who refused to change and those things are now accepted as common.
Automobiles for example. Most people at the time couldn't conceive of anyone needing to travel faster than a good horse team could pull you in a day.
What you are labeling as a "fad" is simply people showing empirical evidence that something works.
It’s been that way for over 100 years now for good reason.
It hasn't been that way for 100 years. 100 years ago, people were killing elephants with 7mm Mausers, and elk with 30-30s, and big ol' whitetails with 25-35s. The popularity of the 30-06 started in after WWII due to the massive amount of surplus firearms and ammunition. It wasn't until the late 50's/early 60's that the "magnum" craze started taking hold, predominantly driven by 1) gun companies, 2) gun writers, and 3) more disposable income in American households.
Also, your statement really only applies to America (and Americans hunting abroad). The Scandinavians have been content to kill moose with 6.5X55s for a long time and don't seem to need a 7RM or 300RM to get it done. Russians have been killing big ol brown bears for a century using the 7.62X54.
So, no, there really isn't a "good" reason why a lot of hunters have that opinion.