I run pretty hot and have those same surplus puffy pants. Being as I run hot my preferred setup when carrying camp on my back is the Kuiu zip off light merino baselayers, a light pant and full side zip rain pants. When I stop to glass and need warmth I find that throwing on the rain pants retains a lot of heat, especially if you're losing heat to wind, in this case the rain pants are really perfect.
I only consider carrying the puffy pants when day hunting and weight isn't such an issue. Out of 8 days in the snow and cold rain this past Nov in N Idaho I only wore the puffy pants once for a few hours sit and I probably did carry them every day. This was all in 25-35 deg temps, overcast, storming etc. Other than that I had all three of the above layers on when sitting and was fine. I did use the puffy pants to supplement my insanely light sleep system (I had a 40 deg quilt sleeping on the snow, had to loan my 20 deg bag to my dad who'd forgot to pack his bag) and they were a life saver.
Same on our elk hunt in Nov '16 I probably carried those pants every day in temps from 10-35 but that trip was dry and not so windy and I never once put the pants on.
With those being 2 lbs there are much better ways to supplement your sleep system. For backpack hunts I take the above mentioned 40 deg Apex quilt with a poncho hole and either drape it over myself while glassing or wear it like a poncho and drape it over my 20 deg down quilt as a great supplement for warmth at night. Have been fine down into the teens with this while sleeping in a floorless on an xTherm. I feel this is the most weight-efficient way to bundle up while glassing and help keep warm in camp whether sitting around or sleeping.