I’ve struggled with this very issue since we had our daughter almost 2 years ago. I don’t care at all about camo, but I do need to find clothes for her that will keep her warm and dry (and dry quickly if they get wet, which happens often in toddlerhood). Obviously your son is older, but some of the same principles could be useful. I try to avoid cotton when at all possible, and opt for synthetic fabrics and fleece. I have found a lot of good options at our local thrift store, and Walmart carries “rash guard” style swim shirts are great for inexpensive baselayer tops and T-shirt’s that dry quickly. I’ve found great deals on quality raingear and boots (which is where i’d Be willing to spend the most money if required) shopping on Sierra trading post and amazon.
We did a 4 night 5 day backpacking trip over Chilkoot Pass from Alaska to Canada last July, and our daughter (then 17 months) did it with only a single set of layered clothing that dried quickly. I did opt to carry an extra fleece suit and spare baselayer for her, but we never needed them.