So there's plenty of time to put away your optics, tripod, seat/stool, change upper torso clothing to accommodate a stalk, etc. but there is insufficient time to take off and pack puffy pants?
The only real issue with wearing a puffy skirt is wind blowing up it. You have to reheat that cold air over and over. While you could potentially save a couple of seconds with choosing a skirt over pants, any tangible time savings would be mostly gone with a few minutes of practice.
Recommended Answer: Practice setting up to glass and to pack up with the focus on speed and quietness. It is free and can dramatically reduce your time to get ready; coming and going.
While wearing a skirt it not my cup of tea, others can do what they want. There is another thread on this site talking about the benefits of wearing fishnet stockings while in the field. That too is not necessarily my cup of tea. However, this could be a great opportunity for some WKRs that want to be on the fore-front of the next major hunting clothes fad.