I'd go. Hines Ward played his entire NFL career without a L ACL. There are plenty of people that cope just fine. Worst case scenario it doesn't go well and you end up with surgery anyway. You obviously have a higher possibility of meniscus involvement if something happens, but only god knows if something will happen.
Best case scenario you find out for your lifestyle you can live without it, other end of the spectrum if it's feeling unstable and giving out get it done after your elk hunt. I've rehabbed plenty of surgical and non-surgical ACL's. Surgical generally do better, but that doesn't mean non-surgical can't have good outcomes either. High impact cutting sports are very difficult without an ACL, but people that avoid those generally get along fine.