It used to be (in the pre-parallel limb days) you would lose both poundage and draw stroke by backing out the limb bolts, so you were in essence changing 2 different variables at the same time. Part of the speed decrease was related to poundage and part to the change in brace height/stroke. I cannot say whether the same is true with the geometry of today's bows, but a better comparison might be to compare 60# and 70# peak weight bows (all else equal).
That begs the question of whether a 70# bow backed off to 60# would generate the same speed as a 60# bow maxed out (all else equaly)? I would doubt it, but have no data to support that either.