In my opinion, sleeping bag and clothes at a minimum should be kept in a dry bag. When leaving camp in the mornings, I like to stuff my sleeping bag into it during the day as well.
I've found over the years, if I am bringing a dry bag, I might as well bring a big one and use it to keep everything I can fit in it dry. I've found 1 Sea to Summit eVent 30L compression dry bag with my clothes, tent (minus poles), sleeping bag, sleep pad, etc... is much easier to pack then multiple compression bags. You get multiple compression sacks in your pack, they tend to be difficult to pack items around them considering they are somewhat spherical without side compression straps.