I have the same bivy, with a 20 degree REI sub kilo down bag, xLite Pad and use a polycro ground sheet. The first two trips I did with the Ptarmigan were in Jan and Feb of this year. One was a multi-day upland trip sleeping in a river canyon with water close by every night, it was getting down to about 25-30 at night and very windy. I slept bivy, no tarp and was fine. Humidity was low. The other trip was a two day climbing trip with the same gear in the desert but miles from any water source , 40-45 degrees at night. First night was insanely windy, was fine, second night the wind died off, I warmed up a lot in my bag and woke up to bad condensation. I was already sleeping no hood with my shoulder out of the bag, I ended up taking my bag out of the bivy because the wind was gone and I didn't feel I needed it. I never had the bivy zipped all the way up on either trip.
I sleep hot and I think my issue was getting too warm in my bag and starting to sweat, causing moisture buildup.
I think it's a lot like rain gear, if you're sitting for hours in a cold rain storm, internal condensation won't be a worry with any rain gear, hike uphill for a few hours in the same rain gear and you're going to heat up and have some condensation to deal with. Same with bivy sacks, they're made to repel external moisture sources but have a hard time dealing with moisture created within the shell so minimize that moisture.
I'd look at how warm you're getting at night, a lot of people say a bivy can add 10 degrees to your bag rating, you already have a 15 degree bag, it's 40 degrees out, how hot are you getting in your bag?