Like Newtosavage and I said, the factory Rem barrel is probably the issue from what you've written. His approach seems similar to mine. I actually bought a cheap used 7mm-08 Savage expecting to throw the barrel away and install a pre-fit. However, after shortening and fire lapping the barrel, bedding, and lapping the lugs it shoots <0.5 MOA 3-shot groups with 162 gr A-Max/ELD-M over 39.5 gr of Varget loaded to 2.805", so I kept it. That load will smoke whitetails past any range I will shoot. BTW the 7mm-08 like the 308 has a long free-bore so no need to chase the lands, your magazine length should work for this caliber.
If you don't want to give up on your barrel, the most common issues I come across with factory barrels are 1. many are rough out of the box and foul quickly. See previous comment for fix. and/or 2. the interior barrel dimensions. The bore should be slightly wider at the breech tapering slightly to the muzzle, any tight or loose spots degrade accuracy. You can check this by slugging the barrel (tapping a very soft lead fishing weight down it, Beartooth Bullets has instructions). I've had good results with Beartooth's low speed fire lapping as it doesn't wear down the lands like full power lapping. 3.5 gr of Trail Boss behind 1/2 sheet of Scott toilet paper and their 140 gr lapping bullet will get you about 550 fps and sort out dimensional issues if you follow their instructions.