I have not found this to be a problem. In fact, my teaching schedule means that I have more time for shorter (3-4 night) scouting trips than most of my friends. What I do is schedule classes for Wednesdays and Thrusdays, Thursdays and Fridays, or (worst case) Tuesdays and Thursdays. That usually gives me largish chunks of time on non-teaching days to scout in the early fall. I then go on a longer solo hunt over thanksgiving. If you're teaching Thursday and Friday, your classes before thanksgiving are cancelled automatically. With the other teaching schedules, I have found students fairly open to rescheduling the Tuesday or Wednesday before thanksgiving. Oftentimes, I will give a longish research assignment over the holiday or presentations thereafter to rationalize the extra time away from class. If your classes are clustered toward the end of the week, this means you can hunt, eg, from the Friday before thanksgiving to the Tuesday or Wednesday thereafter. Not many people can afford to take almost two weeks off work to hunt without effecting vacation time. My colleagues often cancel a class or two to go on extended trips when giving papers. I have never cancelled class to go on a hunt--in fact, I have cancelled class only once in the last eleven years--but I do not see much difference between doing so and taking the excuse of an invited lecture to spend a few days running around out of town. What I would not do is talk about this with any of my colleagues. Academics generally do not understand or value being outdoors, much less hunting, and a compelling argument could be made for taking such time off for research.
P.s., the arrangement above applies perfectly to the northern zone hunting season in NY, which encompasses the Adirondack park, the largest park and the largest state-level protected area in the contiguous United States, and the largest National Historic Landmark. You can easily hike in three miles, drop off trail and hunt bucks that have never laid eyes on a human being--stay there for ten days or even three months and not see another person yourself.