Have you bedded the action, and lug? After doing that, mine doesn't seem to care; no POI shift taking the barreled action of the stock and reinstalling, pretty much regardless of exact torque. Before there was stock flexing on tightening and it probably mattered more.
If you don't bed the trigger guard, it might be a good idea to leave the front TG screw a little loose. My TG will add flex when tightening if the front screw is snug (doesn't fit squarely in the TG pocket and I've never bothered bedding the TG).