I would be cautious about making your rifle lighter to some degree. .300's will generally shoot better when of moderate to heavier gross weight. This keeps the felt recoil down and allows you back on your scope quicker. If you lighten it up a lot, I would look into a heavier bullet to compensate for what you took out of it. That heavier bullet will assist with keeping the recoil more manageable with the lighter gross weight.
If you like how your rifle and bullet combination shoots right now and I wouldn't touch it. Changing things up will ultimately change how it performs. It may be better, but it may be worse also. It is pretty close to hunting season for rifle guys and I would conduct this experiment after the season. I would try and cut a pound out of your other gear and stick with what you got if it performs well for you.
Just my two cents though....
I love what Rosin says here. Makes perfect sense.