I view it from a Maslow's Basic needs, at least the first 3, 1). food and water, 2). shelter, 3). security. If they get pushed out of their bedding area often enough, they simply will stop bedding there, and may or may not come back to it. So in short, their bedding area must provide a sense of security for them.
Generally young deer, including young bucks are not as disciplined as mature bucks. What I mean here is a mature buck in an area of pressure, does not water during the day, unless it get very hot, generally in the triple digits (there are of course exceptions for everything). The young deer with does, no matter how old the doe is, simply cannot control nor ignore it's need for food and water, so we very very often see them watering during the day.
So lets go back to my first paragraph (please reread it). Does with youngsters, are significantly less likely to leave an area due to human presence, as their youngsters needs often dictate the locations they chose. The youngsters simply must feed and water during the day (and at night). Mature bucks on the other hand, have learned that their survival depends on their behaviors, such as forgoing water during the day, they often bed in marginal food areas, and at times, pretty much locations with nonexistent food. For example, Decades ago I hunted this area with some extraordinary bucks (as in very large antlered). After passing a few good bucks the first few days, I started have significant difficulty finding mature bucks. So, I found myself hunting a fairly steep slope with a very very large bolder field. I mean boulders the size of cars and trucks, lose boulders. In short, the bucks were bedded, spread out in this boulder field. In another area, totally different area and hunt, I found them in the lava fields, very broken lava fields. Both areas were very difficult to hunt, because you'd almost have to step on them, but the ground under you (in this case rocks and boulders) was not stable. Honestly, I spooked more bucks than I got the opportunity to shoot.
As has been mentioned, many factors come into play as to where deer spend their days, Mature bucks are often more affected. But as the age old saying goes, the bucks are where you find them.
I hope that helps!