When I use my pro trail, my pad and quilt stay in it. I have an oversize waterproof stuff sack. I get the thing into an 18" tall, 12" wide burrito, drop in stuff sack. I put it in meat shelf of pack, and compress as tight as possible. If I have meat to pack, I just transfer that burrito to the front of my pack. With the silex, I use an ee recon bivy. Same concept, but the Silex usually goes in a separate stuff sack in pack, but the bivy/pad/quilt all roll up burrito style in the meat shelf.
I can be ready to do my "camping" in less than 15 minutes with either setup. I can be ready to hit the trail in less than 10 with either. This is a leisurely pace - I could have either end done in 5-7 minutes if forced to.
My thought is taking each individual item and packing and compressing separately is where a lot of the time suck is. Sometimes you don't have a choice if you're packing the kitchen sink and space is at a premium. I just stare at my empty meat shelf, and far superior compression capabilities of the straps on my pack, and think that's a 2 for 1 over individual compression sacks.
All other aspects of camping versus going home(or truck, hotel, cabin, whatever) are the same - you're cooking and eating and changing/hanging clothes and sleeping just the same. You may lose another 2-5 minutes with things you could leave out at a base camp.
All told, I'm looking at 30 minutes total each day I wouldn't be spending if I weren't packing camp. On my hunt last year, I didn't have a walk to the area I hunted or back to truck or base camp that was less than 1.5 hours. On a 7 day hunt, that's almost 18 hours that could be spent hunting, sleeping, glassing, etc.
There is a tradeoff though - You're packing 6-25lbs of extra weight depending on how light your gear is, and how much food you have to carry. That adds up to extra calories burned on the time you are spending out there.
To me the middle ground is being prepared for 1-2 nights in the field. If I find elk, I camp. If not, I still have the option of staying put or heading out to try somewhere else. If I prepared for 10 days in the field, and packed 25lbs of extra crap a few miles in, I ain't coming out, elk or not. The animals are going to dictate what you do. I like having the option to stay in long term or come out with me in the truck, and the option to stay in 1-2 nights in my pack no matter what.