I had KCT Kydex make me a PALS compatible holder for my G20 that attaches right to my hip belt of my Kifaru pack. It was a simple and elegant solution to this problem for me.
I think in situations as serious as self protection in fast paced encounters like you are preparing for the KISS system should be considered. If you have carried/do carry a firearm on a regular basis on your hip/waist I wouldn't personally want to switch to a chest draw ... especially when stress gets involved while having to present the firearm. Do you really want to drill the new chest draw enough to get proficient under duress? Paddle holster on the waist belt of my pack while on, then EASY transition to pants when the pack is off. No interference with Bins, same place (roughly depending on your standard cary location) where it has always been. Extra points go to the active retention of serpa/safariland type holsters when moving in uneven, treacherous, unknown, and unpredictable terrain/vegetation. Last thing I want is a dislodged firearm if I take a tumble. Some might say that running active retention might hinder the draw, but I think the extra level of security in that terrain is worth the learning curve. Plus a serpa is like $25.
Edit/addition:
Also, with a chest holster you are messing with that rig if you layer up or layer down (not a huge deal if you already have a bino harness it is attached to) ...
But probably the most flagrant downfall of a chest rig to me is ... A significant number of firearm accidents happen during the re-holster. Zippers pulls get caught in trigger guards, poor finger discipline, fabric not clear of holster and/or angle of reintroduction. A chest holster is especially dangerous in this situation as the muzzle passes right over the majority of your vitals during the re-holster. A leg wound from a waist holster in the event of this rare but bad situation is a lot better than a chest ... especially considering the location where this might happen, miles from help.