Drop Test Pass/Fail

[WestTN2288]

Is there a running list of tested scopes and whether they passed or failed?

I’ve read through a lot of the evals, but I’m just talking about a list of what passed and failed? Would be really helpful when looking for new scopes.

 

[JFK]

Agreed. One step further, it would be good if there was a single document that had pass/fail but also a simplified matrix to see how they failed. Some scopes that fail in the tests do so incredibly fast, while others tolerate some level of abuse but ultimately fail on the repeated taller drops. That seems like relevant information to include.

 

[Marbles]

Sounds like it is time for you two to get to work and compile one.

 

[JFK]

I’d be willing to do it. Something like an Excel sheet. It would be better for it to live with an Admin so it could be updated but I’m happy to get it going.

Make
Model
SFP or FFP
Pass or fail
If it failed, how.

Anything else?

 

[WestTN2288]

I’d be willing to do it. Something like an Excel sheet. It would be better for it to live with an Admin so it could be updated but I’m happy to get it going.

Make
Model
SFP or FFP
Pass or fail
If it failed, how.

Anything else?

I’d be willing to help you put it together.

Only thing I would add is MSRP price. I realize that could get confusing, but it’s a big part of most peoples decision

 

[JFK]

This is what I was describing. If there is any interest in it then I can send it to whomever wants to post it and manage it.

It’s not perfect. Some of the reviews were one and done while others described issues in subsequent use. “Partial pass” or “partial fail” is a fail on this sheet. Tried to pull out the key failure point but translating pages of narrative to a spreadsheet isn’t always the easiest thing to do. My thought was that it would serve as a quick reference that was updated as more scopes are evaluated.

 

[WestTN2288]

This is what I was describing. If there is any interest in it then I can send it to whomever wants to post it and manage it.

It’s not perfect. Some of the reviews were one and done while others described issues in subsequent use. “Partial pass” or “partial fail” is a fail on this sheet. Tried to pull out the key failure point but translating pages of narrative to a spreadsheet isn’t always the easiest thing to do. My thought was that it would serve as a quick reference that was updated as more scopes are evaluated.

I think that is perfect. Pretty much exactly what I had in mind… now to keep it live

 

[Macintosh]

Agree something like this is helpful, and probably should be integrated directly into the eval for each scope—believe I heard mention they were working on a database already so it may be in the works.
However, I think it needs more detail to be really illustrative of the actual trial. Add a column for each element of the trial so you can see what each scope did well and what it didnt do. You will see even among the failed scopes that some passed everything except the 3x36”, while others failed on the first 18”, failed to track, etc. To the extent that a sample of one or two scopes shows anything, that shows differing degrees of durability—its been claimed several times that some scope designs such as 1” tubes likely wont ever pass, so this seems relevant for a lot of people. MOST hunters use 1” non-dialing scopes so that seems a critical element of the test to make apparent.

 

[Alder_]

Is this up to date?

 

[WestTN2288]

I have not updated it…

 

[stan_wa]

I’d be willing to help you put it together.

Only thing I would add is MSRP price. I realize that could get confusing, but it’s a big part of most peoples decision

Common street price abs best sale price seems would be great , for example has anyone ever bought a nf for more that 10% off? I just saw the tenmile on sale for 1050 and msrp was like 1500$

 

[fwafwow]

Agree something like this is helpful, and probably should be integrated directly into the eval for each scope—believe I heard mention they were working on a database already so it may be in the works.
However, I think it needs more detail to be really illustrative of the actual trial. Add a column for each element of the trial so you can see what each scope did well and what it didnt do. You will see even among the failed scopes that some passed everything except the 3x36”, while others failed on the first 18”, failed to track, etc. To the extent that a sample of one or two scopes shows anything, that shows differing degrees of durability—its been claimed several times that some scope designs such as 1” tubes likely wont ever pass, so this seems relevant for a lot of people. MOST hunters use 1” non-dialing scopes so that seems a critical element of the test to make apparent.

For some of the additional detail, how about just including a link to each of the evaluations and the Q&A?

 

[Macintosh]

For some of the additional detail, how about just including a link to each of the evaluations and the Q&A?

I dont think that works nearly as well. To me, the additional detail is stuff like differentiating between "passed all tests excepted shifted 1.25moa on 3x36" drops" versus "failed all drops, failed at holding zero". Both of these are a "fail" I believe, but there is a wide range of difference between them that is missed without the nuance...but including that in a chart is perhaps 4 or 5 extra columns allowing you to quickly and easily compare all scopes tested at a time (and possibly filter by weight or magnification, price, etc), versus a 100-page read-a-thon to ferret out information where you dont even know what you are looking for.
The other thing I think a "form" like this sheet does (see what I did there?), is it forces a consistent format and an answer to a specific question. When I read some of the evals, as thorough as they are, they were written over a multi-year time period without a consistent order, and in text form--even re-reading some I have a hard time saying definitively in a couple cases if its a pass or a fail of a certain part of the test. I think having a form like this actually improves the consistency and rigor of the test itself.

 

[fwafwow]

I think anything that makes information more readily available to people is a great idea, and I support some sort of executive summary or this chart.

I am not being argumentative, I'm just sensitive to a couple of points. First, any summary is going to be inadequate in some respects, as it is less than all of the information. Second, a summary that is too exhaustive can become cumbersome. (If you have read The Checklist Manifesto, a similar point was made therein - checklists must be short enough to be helpful.) Third, this chart (or whatever) involves additional work (but if you or someone else will do it, this is a moot point.)

I dont think that works nearly as well. To me, the additional detail is stuff like differentiating between "passed all tests excepted shifted 1.25moa on 3x36" drops" versus "failed all drops, failed at holding zero". Both of these are a "fail" I believe, but there is a wide range of difference between them that is missed without the nuance...but including that in a chart is perhaps 4 or 5 extra columns allowing you to quickly and easily compare all scopes tested at a time (and possibly filter by weight or magnification, price, etc), versus a 100-page read-a-thon to ferret out information where you dont even know what you are looking for.

The above are all good points. But those additional 4-5 columns may be useful to you, but not everyone. To me, for example, I don't care about the nuance of any failures. I do care about the details of a pass, and in some instances would want the information you describe. In such an instance, I would prefer to read that test, and only that test.

My recollection of most of the tests that I have read is that they are pretty short when you skip over the recitation of the methodology. The Q&A can, however, run long. (A Vortex thread is at 29 pages.) I discount much of that anyways, at least in comparison to the test.

The other thing I think a "form" like this sheet does (see what I did there?), is it forces a consistent format and an answer to a specific question. When I read some of the evals, as thorough as they are, they were written over a multi-year time period without a consistent order, and in text form--even re-reading some I have a hard time saying definitively in a couple cases if its a pass or a fail of a certain part of the test. I think having a form like this actually improves the consistency and rigor of the test itself.

Nice pun.

I'm with you on all of these points. The number of tests has increased, as well as (I believe) the number of people reading them. I am fine with coming up with a method (or methods) to try to provide and summarize the information in a more uniform manner. I expect that Form and Ryan may have thoughts on how they think the information is best presented. And maybe the presentation can help them in making some aspects of the test easier to administer or record.

I don't think we are talking about mutually exclusive summary content. Maybe it's like a hierarchy of weblinks, with it being something like:

1. Super Dumbed Down for fwafwow - list of manufacturers, and maybe scope models, and just "pass/fail"
2. Linked to any of the above is more detail - and not necessarily the full test link, but some of the items you reference above (and more nested therein).
3. Repeat.

 

[fwafwow]

I have not updated it…

What's the date of the information - 9/24/23? Might be good to include in the table. I'm happy to help out on this. Let me know and we can update as needed.

 

[fwafwow]

Common street price abs best sale price seems would be great , for example has anyone ever bought a nf for more that 10% off? I just saw the tenmile on sale for 1050 and msrp was like 1500$

IMHO a can of worms and time that could be better spent on something else. If I was adding price info, I would give a range, denote whether that includes used prices, but probably lean on telling people how to find pricing for themselves.

The more I think about the summary, the more I like it. And I bet @robby denning or @Ryan Avery could figure out how to present it in a way that leads to more hits, and advertisers who would like those hits. Unfortunately not many scope manufacturers would benefit - at least currently.

 

[Macintosh]

@fwafwow , I hear what you are saying, I guess its the ability to quickly compare different aspects as a tool to identify what to look for in the evals that is helpful. I know for a fact a lot of people dont read that much--I do, but even I get lost in it sometimes, get distracted, etc. Example--the credo 3-9. This scope may have been a fail (I dont recall what is a fail vs a "partial pass")--but looking at this it sure would be a shame if the credo 3-9 and the leupold vx3 were both simply recorded as "fail"? Is this format too complicated? I personally think it provides additional clarity, allows comparisons at a glance, helps target what to look into deeper, but still offers a very quick, easy to digest format.

 

[fwafwow]

I like the chart below.

@fwafwow , I hear what you are saying, I guess its the ability to quickly compare different aspects as a tool to identify what to look for in the evals that is helpful.

Agree 100%

I know for a fact a lot of people dont read that much--I do, but even I get lost in it sometimes, get distracted, etc.

I read more than I care to admit, including my share of rabbit holes. I'm just trying to prevent involuntary rabbit holes. I want to read as much as possible - but only what I want to read. (I almost typed "need" but that's ironic given how much time we are spending on this.)

Example--the credo 3-9. This scope may have been a fail (I dont recall what is a fail vs a "partial pass")--but looking at this it sure would be a shame if the credo 3-9 and the leupold vx3 were both simply recorded as "fail"?

Agree.

Is this format too complicated? I personally think it provides additional clarity, allows comparisons at a glance, helps target what to look into deeper, but still offers a very quick, easy to digest format.

It definitely provides a clear comparison at a glance and helps narrow what to look at deeper. I don't think it's too complicated, depending on a few things. What happens with cells A2-5 - do they become additional columns in the table below? (My opinion is that the width should never take up a screen, and ideally not even a screen on a phone. I hate scrolling right/left to see info. But maybe that's an unusual idiosyncrasy. Also I personally don't care about any of those four items - unless it's about a passing scope.)

But maybe someone would like to filter for those items, and or any of the others? Like if I am set on 1" tube scopes, then maybe I only want to initially see what scopes with that dimension have passed. One could filter for other info - like manufacturer, focal plane - whatever. And in combination.

View attachment 639033

I think it's taking shape, at least in my mind. Which is a problem - it's often easy to poke holes in an example, and to say (or type) "this would be really good" but without having the expertise to make that happen.

 

[Ryan Avery]

@JFK @fwafwow Sorry I didn’t see this. I keep meaning to do something like this. But it keeps getting put on the back burner and the forum is not a great place to store it. Maybe a live version on the front end or on the shoot2hunt website

 

[cmahoney]

This is what I was describing. If there is any interest in it then I can send it to whomever wants to post it and manage it.

It’s not perfect. Some of the reviews were one and done while others described issues in subsequent use. “Partial pass” or “partial fail” is a fail on this sheet. Tried to pull out the key failure point but translating pages of narrative to a spreadsheet isn’t always the easiest thing to do. My thought was that it would serve as a quick reference that was updated as more scopes are evaluated.

Awesome


Sent from my iPhone using Tapatalk