I've taken a lot of my game with a 264 win mag. The rifle was a 300 win mag. I shot everything (rocks, dirt clods, crows, coyotes, game, etc) with that rifle for several years and developed a terrible flinch doing it. I ended up re-barreling it to 264 win mag and have loved it ever since. I used the Nosler 140gr Accubond loaded to 3000fps for years with great results. I switched in 2012 to the 140gr Berger VLD as I increased my comfortable range and found a slower but very consistent load at 2914fps and have taken several animals with it. None have gone far after a hit with the Berger but few have exited. The bullets perform very differently but do the job.
I've never shot an elk with the 264 win mag but have with a 270 win and a 150gr Partition and later a 140gr Accubond. I would not be afraid to elk hunt with the 6.5mm 140 Accubond for sure. That said I currently use a bigger rifle.
My 264 weighs 10.5lbs with scope, ammo, sling, ready to hunt. I use it for antelope, whitetail, and all deer in flatter country. My other main rifle is a 7.25lb ready to hunt 300 win mag. I got one built with a better stock and muzzle brake that doesn't beat me. I will use it on mountain muleys, elk, and a moose hunt I have lined up for next year. I shoot 180gr Accubonds from it at 2956fps. Places where tough shot angles and bigger, heavier bones may be encountered I trust the Accubond more from my years of using them. Places where wind drift and long range consistency are more important than horsepower I like the 264 with the Berger.