as others have said the ease of body heat retention via putting a hood up is unbeatable but I also have many layers which are hoodless. Stuff used mostly as mid layers for sure. You can also get a down balaclava and stick it in a pocket...I have one and it is very good especially for sleeping on cold nights. Mine is from goosefeet gear and you can get is with overfill....my favorite piece is peloton 97 w hood.
I've always been a no hood guy except on my rain jacket. Over the winter I picked up a lightweight fleece layering 1/4 zip with hood. Worn it on a couple bear hunts........ never came off.
If it's warm I've got three hoods available. Sitka Lightweight hoody, Sitka Heavy Weight hoody, and rain gear. The lightweight hood is barely noticeable when stacking other layers.
When it gets cold I switch out the heavy weight hoody with a mid layer that doesn't have a hood and then add the Jetstream (hooded).
Midlayers and out for me because sometimes a vest and midlayer and a shell is enough if the midlayer has a hood. I wish the mfgs would make the hoods zip off to address the bulk problem when wearing outer layers.
If I'm moving a lot, layers will be taken on and off, and I will opt for no hoods except rain jacket. For glassing, camping, travel, etc. I like to have a hood on my insulating layer to be cozy.
Mine changes every trip out depending on the conditions and layers I choose but id say generally:
Hoodless Base
Hooded Mid
Hoodless vest or second mid (colder seasons)
Hooded Insulation
Hooded Rain
Obviously subject to change if its 90 degrees and sunny in august/early September. Then I'm only in one hooded base. But never less than 2 with me and never more than 3 worn at a time.
2 hoods at a time is my rule, generally both on items that are likely to be outer layers. In warm wx that might be a sun hoody/base layer and a light puffy. In cold weather where I live its never a base layer, its usually a puffy and a rain shell. I agree for cold wx the base layer hood is warm, I just get the same effect with better versatility and lower bulk by using a light balaclava and/or neck gaiter/buff.