Quote:
Originally Posted by jason.1969
There would be very, very few authenticated autos out there if this were the criteria…which by the way I fully support. If you don’t know, just say you don’t know.
|
The problem is that most people don't think probabilistically. They want a thumbs up or a thumbs down. They can't wrap their heads around the confusion matrix output that yields an 83% probability of a signature being authentic. The authenticator can make two different types of errors: they can reject a valid signature, and they can authenticate a forged one. But there's a trade off between these, and where you choose to operate on the ROC curve has consequences in either direction that increase or decrease the likelihood of getting it wrong one way or the other. The truth is, even the best of the best authenticators would be expected to make mistakes at a rate that almost no one in this hobby would find acceptable unless they were a mathematician or a statistician who knows what sort of outcomes to expect and who knows how to see the world through the lens of probability. If I were to say that I would expect the best experts in the world to be wrong something like 15% of the time, most people would probably tell me I'm an idiot, but the ones who don't call me an idiot would almost certainly tell me I'm an idiot when I tell them that they'd probably be wrong something like 30 to 40% of the time. But that's the ballpark of what I would expect to see if we were to set up a test with a sample set of signatures using a mix of authentic and forged examples with no provenance to accompany them. It's just not something that humans are going to be "good" at no matter how much one studies it. This is why provenance is so important. It can drastically increase the likelihood of authenticity.
Imagine if every LOA or COA came with the truth printed on it. "JSA is 91% confident that this baseball has been signed by Mickey Mantle", or "PSA/DNA estimates that there is a 74% probability that this photo has been signed by Willie Mays. However, this does not meet our confidence threshold of 85%, so we are unable to authenticate it at this time." Nobody would pay for their service if this was the end product. So they just give us the thumbs up or thumbs down instead. But the reality is, they're going to get this stuff wrong far more often than most people would expect, and certainly more often than they would be comfortable with if they knew the truth. But the reality is, PSA and JSA are both probably a hell of lot better at it than any of us are.