Tactics Training as Probabilities

Tonight, I want to discuss a probabilistic perspective on the problem solving session. For this analysis, I'm going to model a tactical session as a binomial sampling phenomenon.

First, what is a binomial sampling? This question is best answered using the real-world example of a series of coin tosses. Were we to toss a coin several times and record the result at each toss, we would have a binomial sampling of heads and tails--its that simple. Tactical sessions follow the same model because each problem attempted by the tactician has one of two outcomes: pass or fail. Fortunately, the mathematics behind the probabilities of a binomial distribution are well established, so I won't review them here. But I will calculate probabilities using the binomial probabilities calculator available at VassarStats.

The following table shows the Probability that a Tactician with given a accuracy will solve at least a given number of problems Correct in a given number of Tries.

Tactician With Accuracy	Tries	Number Correct	Probability
98%	100	98 (98%)	0.67669
98%	100	99 (99%)	0.40327
99%	100	98 (98%)	0.92063
99%	100	99 (99%)	0.73576
98%	200	196 (98%)	0.62884
98%	200	198 (99%)	0.23515
99%	200	196 (98%)	0.94825
99%	200	198 (99%)	0.67668
98%	1000	990 (99%)	0.01023
99%	1000	990 (99%)	0.58304

Lets first consider the likelihood for a 98% tactician to "accidentally" get 99 problems out of 100 on some arbitrary set of 100 problems he might encounter in a normal training session: 40% (0.40327). Now, consider the likelihood that this 98% tactician will get 198 right out of 200: 23.5%. So the 98% tactician will look like a 99% tactician on over 40% of his stretches of 100 problems and more than 23% of his stretches of 200 problems!

Lets compare a 198/200 performance to our expectations for a true 99% tactician: 67.7%. So a true 99% tactician will look like a 99% tactician on more than 2/3 of his stretches of 200 problems.

So what do all of these numbers mean? In short, these probabilities say that 200 problems are insufficient to confidently distinguish a 99% tactician from a 98% tactician. Only at very high numbers of problems can these two tacticians be distinguished with good certainty. In the example above, a 98% tactician has only about a 1% chance to get at least 990 of 1000 problems correct while a 99% tactician has significantly better than an even chance (58.3%).

So, what is the bottom line of this analysis when thinking of our own accuracy statistics? Well, one must do a lot of problems of at least a given accuracy before he can be certain that he is actually a tactician of that accuracy!

With this in mind, I present my performance today at the CTS. I wish I would have warmed up--three misses before number 30.

4p-1f-12p-1f-8p-1f-73p
97% @ 1395 ± 90 ; 1383 final