EdP
WKR
I actually hate cold and I don't deal with it well. I have less than 10% body fat and consequently, I easily get cold. But, those fleece-lined water-resistant synthetic/cotton blend pants, coupled with wool/synthetic blend long johns, keep my legs warm when sitting for 10 to 12 hours at a stretch.
As a small guy (5'8" & 150 lbs), I get cold easy when I'm not moving. My legs don't get to feeling cold but the heat they lose is significant and affects my whole body. So you may think fleece lined pants and long johns are adequate because your legs don't feel cold, but I bet you are loosing a lot of heat from them. That's where puffy pants can help, plus they are not bulky and heavy like fleece lined cotton pants.