Practice your shooting as much as you can. I like to be comfortable with 1/2 elk size vitals at 600y. But, I've never shot an elk more than 300y away. So I'd make rifle a priority but whatever you get comfortable with is what your range is, if your range is max of 250y that won't kill your elk hunt.
Physical conditioning will kill a hunt. I work out EVERY DAY year round. I live where I hunt and hike all summer. That said, it doesn't take a lot of time. I do these 20-30 min strength and cardio workouts (via the peloton app for $13 a month). Bodyweight only or with some dumbells. I recommend fitting working out into your daily routine...at least 4 days a week 20-30 min.
As an aside, I moved to Wyoming weighing 195lbs. I stopped drinking, started eating better, and working out 30 min a day. I'm down to 165lbs and feel wonderful. It's not a huge amount of time or work each day but it pays off amazingly. Anyway, I recommend daily exercise to everyone, but especially to those silly enough to chase elk at 7000 - 11,000 ft! No need to go nuts, just some exercise, think some about your food, and drink less...should pay off in so many areas of life.