Analyze My Groups

[Jordan Smith]

I will restate my original bold claim in a more accurate way: I have not seen any evidence, in my testing or any others, that small changes in seating depth or powder charge have an influence on precision. Instead, the null hypothesis, which is that there is no difference and random noise accounts for any observed differences, has been proven over and over again.

I keep hearing people say, in the same breath, that no test could ever be designed to conclusively answer whether tuning works, and that they are sure tuning works because benchrest shooters who shoot small groups like to do it. This is so irrational it makes my head hurt. I'm still waiting for ONE SINGLE REFERENCE to a test of small seating depth or powder charge tweaks done with >10rds per sample that demonstrates a statistically meaningful difference.

The null hypothesis has not been proven. Full stop. The data has failed to reject the null hypothesis. That's a big difference. You would need a LOT of data to even begin to think about the null hypothesis being proven. Talking about the null hypothesis being proven by a data set is very poor statistical practice.

Where do you keep hearing that? Certainly not from me. If you think that's what I've said, then I encourage you to go back and re-read my posts.

I don't know how much more clear I can be. I am not claiming that small differences make a difference, nor that they don't. I don't believe that it is feasible to definitively demonstrate either assertion within the practical limitations of shooting in real-world conditions with all the variables that go along with that, including the sample sizes needed for statistical significance, the very finite lives of CF rifle barrels, and the difference between barrels, calibers, and chamberings.

 

[solarshooter]

Well, the goal posts are moving with 10+-round groups now being a stipulation, but I have done that myself several times. Why did you choose 10-round groups in this quote? Of course 10-round groups are better than 3- or 5-round groups, but my point is that 10 still isn't even close to enough if we're after statistical power, whether they show a difference between charge weight and seating depth tweaks, or not.

Seriously though, can you show me a statistically robust test that shows that changes in powder charge or seating depth have an effect on precision?

So, you are making a blanket statement that tuning doesn’t work at all and it can’t be done? Doesn’t matter the rifle or load? I have a hard time with blanket statements.

Edit: what are you now considering small tweaks?

The goalposts are not moving. I continue to ask for one shred of evidence where anything resembling a statistically robust test shows that tuning makes a difference. I put the number 10 shots to it, which is still too low as we all acknowledge, but I was trying to be generous. Small tweaks, large tweaks, you name it. I think it's reasonable to apply bounds of safe max and min pressures, and rounds still feeding, not jamming like crazy into lands or seated with the bearing surface way below the neck or something wacky. Even then, give just one reference!

 

[Schmo]

Not trying to be rude, but maybe you guys should take this 6 page discussion to a new thread? Only about 1 out of the 6 pages directly pertains to why I started this thread.

 

[solarshooter]

The data has failed to reject the null hypothesis.

You are correct, this is the most correct way to state it. So we agree on this at least. The null hypothesis being that there is no clear mechanism showing that tuning via changing powder charge, seating depth, or tuner adjustment has any impact on precision.

I am not claiming that small differences make a difference, nor that they don't.

Which makes this statement logically flawed. There is actually no need to "prove" the null hypothesis. It's just that, it's the baseline truth position. Otherwise I could come up with theory after theory, and unless all are "disproven", one could claim your position. This is Russell's Teapot.

 

[Harvey_NW]

Isn’t that considered tuning a load?

If you want to consider it tuning, sure. But I don't, because IIRC Miles said "almost universally" if there's a possibility to shrink dispersion, it's by backing the powder charge off. I remember him saying something about some powders (can't remember if he specified double base) can "get a little squirrely" and open up near pressure. But in many cases with stable powders there isn't a measurable difference in the dispersion from different powder charges in large sample testing.

 

[Jordan Smith]

That true if the samples are similar . If they are very different, even smaller samples sizes can still be high confidence

More shots is better instead of arguing about what sample size is needed it’s best to just run a t test this takes into consideration the sample size.

Agreed. I'm using 30 samples as a roughly generalized minimum for statistical significance.

 

[Jordan Smith]

Which makes this statement logically flawed. There is actually no need to "prove" the null hypothesis. It's just that, it's the baseline truth position. Otherwise I could come up with theory after theory, and unless all are "disproven", one could claim your position. This is Russell's Teapot.

False. That's not how the physical universe works. Coming up with theory after theory, and testing each one, is exactly how we came to our collective current understanding of physics. Assuming that the non-existence of a cause-effect relationship between variables is the baseline truth position is completely unscientific and incorrect. Why do you think that testing the null hypothesis is common statistical practice, if there's no need to either reject it or not? Failing to reject the null hypothesis does NOT mean that the null hypothesis is true. It's not a truth position. It's a baseline statement that can be tested and then rejected if the data warrants it. Rejecting the null hypothesis or not is not based on a discrete quantity, either, but is based on a test result that is typically a value on a continuous spectrum of real numbers between 0 and 1. It's somewhat subjective, and depends on your choice of confidence level.

Your approach comes across as being influenced by confirmation bias. An extremely strong opinion based on very limited sets of data. A scientific approach is to look at the relationships between variables objectively, come up with a well-reasoned hypothesis (in this case, one that is consistent with physical principles such as wave theory and oscillatory motion) and only draw conclusions as strong as the data supports.

Neither claim here is unfalsifiable, but both require significant data and control of variables to come to definitive conclusions.

Let's put it simply. How many tests, and how large of a sample size in each test, would it take to say with conviction that small changes to powder charge and seating depth are not resolvable among the other variables in the equation of POI dispersion?

Similarly, how many tests and how large of a sample size in each test would it take to say confidently that there is a correlation between those load tweaks and POI dispersion?

I'll illustrate with a simple example, and then I'm done debating the statistics. You're welcome to believe whatever you want, but a course or textbook on statistical methods may be helpful.

Let's say I have a weighted coin. It's heavier on the "heads" side, so we would expect the coin to land tails up more often than heads up (the heavier side settles at the bottom). If I flip the coin twice and get heads both times, I could easily say that I have done two tests that both fail to reject the null hypothesis, so therefore, the null hypothesis is true and there is no physical correlation between the weight on the coin and the side it lands on - the results are purely caused by random variation. But this would be incorrect, wouldn't it? Further testing of 1000 coin flips then show 30% heads and 70% tails, and the null hypothesis is rejected.

 

[Jordan Smith]

If you want to consider it tuning, sure. But I don't, because IIRC Miles said "almost universally" if there's a possibility to shrink dispersion, it's by backing the powder charge off. I remember him saying something about some powders (can't remember if he specified double base) can "get a little squirrely" and open up near pressure. But in many cases with stable powders there isn't a measurable difference in the dispersion from different powder charges in large sample testing.

The opposite is also true in some cases. Many double-based powders are designed to burn most efficiently and consistently near SAAMI max pressures in modern chamberings.

 

[stan_wa]

How many tests, and how large of a sample size in each test, would it take to say with conviction that small changes to powder charge and seating depth are not resolvable among the other variables in the

I rest my case, have fun trying to shoot large enough groups with small changes to produce two outputs that are different when measured with a t test.

Let me know when you find it .

 

[solarshooter]

An extremely strong opinion based on very limited sets of data.

and only draw conclusions as strong as the data supports.

You've shown or linked to 0 data. I'm also done.

 

[stan_wa]

Only about 1 out of the 6 pages directly pertains to why I started this thread.

Sorry to contribute to what seems like detailing your post.
I did think it was relevant to your initial point because In regards to your first post I don’t think your groups are meaningful different from one another. I understand that can be difficult to accept, but I was trying to provide context that would say if you’re not changing components I also wouldn’t expect them to be different.

I’d say keep shooting once you have a proven good load and don’t worry too much.

 

[Schmo]

Sorry to contribute to what seems like detailing your post.
I did think it was relevant to your initial point because In regards to your first post I don’t think your groups are meaningful different from one another. I understand that can be difficult to accept, but I was trying to provide context that would say if you’re not changing components I also wouldn’t expect them to be different.

I’d say keep shooting once you have a proven good load and don’t worry too much.

Yeah man, you’re good. The early part did pertain, but then it spiraled into 6 pages of debating statistical validity