Quote:
Originally Posted by sphere and ash
There are statements made above that the probability of a match is so high that one may be “100% convinced,” or that the match shown for all six subjects may be as high as “99.9999%” or even “84-97%.” All of these estimates reflect a misunderstanding of probability, which I will attempt to explain below. But let me declare my bias from the outset: I am not convinced the stereoview depicts “six learned gents,” let alone the Knickerbocker Club.
All probabilities have a margin of error. Most people are aware of this when they see political polling: when one candidate leads in the polls 51-49, but the polling organization discloses a 3% margin of error, it is understood that the race is a statistical tie.
What we need to know is the margin of error for the facial recognition software used. The problem is that the software maker determines a margin of error using the same photographic process and type (say, a mug shot or passport photograph), similar lighting, contemporaneous images, etc. And what we have here are different photographic processes (salt, albumen, and, I believe, a silver gelatin copy photograph), with very different lighting (outdoor versus studio), taken many years apart, with limited visual information (these are group photographs taken from a distance where the ears are not visible, etc.), and where the original poster has altered the shadows in the photographs using another software program prior to analysis.
To give you some idea of how high the margin of error may be in this case, consider that a Google search shows estimates for facial recognition for African-American women may be higher than 35%. And that is with all the commonalities and without the difficulties cited above. I would be stunned if the margin of error here were not much higher. One can’t speak of meaningful probabilities in the presence of such a high margin of error. You’re asking the software to do something for which it was not designed and not tested.
|
I think you've misunderstood my statement. I am not claiming that there is a 99.9999% chance that this is a photo of the Knickerbockers based on the results of the software matches. If you read my statement carefully, you'll notice there are a couple of key qualifiers in it. Here's what I said:
"if the probability of each person being a "match" is 90% [Note added: this is NOT the same thing as the matching software outputting a "90% match"], then the probability of the group being the Knickerbockers is equivalent to the 1 - (0.1^6) = 0.999999 or 99.9999% chance that this is the Knickerbockers. However, this is based on the assumption that a "90% match" actually means the individuals in two photos are 90% likely to be the same person. I don't know if this assumption holds true, and wouldn't be surprised at all if it didn't."
The last part above highlighted in bold is important. While I don't know how their software is coded, thus I don't know enough about their specific outputs, I do write the same type of algorithms for work, so I have an idea of how I would go about writing my own code for such a task (I'm a data scientist, and facial recognition software is the same field of work). I'm not sure exactly what their "90% match" means in the real world, but I would wager money that it probably does not mean that there is a 90% likelihood of the two people being the same person (which is the mathematical assumption that my above calculation was based upon). I think I chose a poor example to convey my point. My point wasn't that this is a 99.9999% probability of being the Knickerbockers photo. My point was simply to demonstrate that the likelihood of it being a Knickerbockers photo
increases as a result of each individual having such high match percentages. This is Bayesian statistics 101 stuff.
As far as having a "misunderstanding of probability" is concerned, I assure you, I do not have a misunderstanding of probability theory. Perhaps I worded my post poorly, but if you read it carefully, paying attention to the qualifiers, you'll find I'm not saying what some people here seem to think I am.
Also, you wrote "all probabilities have a margin of error." This is not true. Probabilities have no such property. The probability of rolling a 2 on a fair die is 1/6. There is no margin of error associated with it. The probability of drawing the Ace of spades from a randomized deck of cards is 1/52. Again, there is no margin of error. Perhaps you meant to say that predictions or estimates have margins of error, not probabilities? That would be true, and if so, I would agree with your point that any actual calculation about the probability of this photo being a Knickerbockers photo would have to be based on the real-world implications of the facial recognition matching model's output. Hence I stated above in my original post that I wouldn't be surprised if a 90% match didn't actually mean a 90% probability of two photos being the same person. Every time I upload a family photo to Facebook, it asks if I would like to tag my wife as her sister. They are not twins. So, I'm guessing the real-world confidence we might have from facial matches is actually quite a bit lower than something like 90%.
09-04-2021, 02:34 PM
Similar Threads
Threaded Mode