I use a camera holster for the dslr body with main lense on, and a pouch for the second lense strapped to it, always in the backpack, on top close to the back, not much waste space and easy to get the camera out.
I usually only take it out if i'm near animals, around some probably interesting stuff or when the light is sonn going to be perfect. Then i just put it around my neck, front or side depending the terrain, with a hand on it.
I never found stuff carried outside the backpack to be practical or perfectly safe, unless maybe you're on a perfectly flat and large path.