Statistics tell you when to stop practicing

It turns out that given certain kind of exams, statistics will tell you. For example, take the Life in the UK test. This is an exam with 24 multiple choice questions. You need to answer at least 18 questions correctly in order to pass. There are mock exams available online. For example, there is lifeintheuktests.co.uk which offers 45 mock exams along with a key showing the answers. Here we have the perfect prerequisites for applying some basic statistics in order to find out how much time to spend practicing.

Let’s look at some probability theory. If you study enough to pass the exam with 95% probability on a single attempt, then you will pass after maximum two attemps with probability of 1-0.05*0.05 = 0.9975, i.e. 99.75%. If you study enough to pass with 80% probability on a single attempt, then you will pass after maximum two attempts with probability of 1-0.2*0.2=0.96, i.e. 96%.

You could even go further and calculate the expected costs given a probability of passing the exam on a single attempt. The Life in the UK test costs currently £50 (you can take multiple attempts), and the expected number of attempts until first success is the inverse (1/p) of the probability p of success on a single attempt. If you are a genius, your probability of passing is of course p=100%, and the expected number of attempts is 1/p=1/1=1. If you’re less genius and your probability of passing in a single attempt is p=50%, then you can expect 1/p=1/0.5=2 attempts. Multiply this by the cost of each exam, and you have expected costs of £100.

What probability p of success on a single attempt to aim for─that’s a more difficult question. Statistics won’t tell you unless you also put down an estimate on the costs of practicing and the costs of taking the exam, and have a way to relate that cost to the financial cost of taking the exam. I’ll leave that as an exercise for the reader :-).

So let’s assume that you’re aiming at passing the exam with 95% on the first attempt, which makes it quite unlikely that you have to go to the exam a second time and pay more. How much do you need to practice? Let’s assume that the questions in the mock exams cover the same material as the questions in the actual exam. All you need is an estimate of the probability p of passing the exam on the first attempt. Once your estimate p>95% you stop practicing! So how to get this estimate?

Well, here’s how. You pass the exam if you answer at least 18 questions correctly from 24 questions. We can model the number of questions you answer correctly as a random variable, distributed according to the Binomial distribution (https://en.wikipedia.org/wiki/Binomial_distribution). With a bit of trial and error, an online calculator (https://stattrek.com/online-calculator/binomial.aspx, there are many) will tell you that the probability of at least 18 “successes” (correct answers) among 24 “trials” (questions) is 95% if you get each answer right with a probability of 85%.

Thus, what you can do is to go through mock exams and estimate whether your probability of getting questions right is 85%─however that can be a bit tedious to calculate. Alternatively─and that might be more practical─you could also take many mock exams, and if you get 19/20 of the most recent 20 mock exams right (even better if you can look at most recent 40 mock exams, and see whether you got 38 right), then you have good reasons to believe that your probability of passing the actual exam is 95%, and you can might stop practicing. Yet another way is to take the mean of the number of correct answers you got in recent mock exams─if the mean is over 20.5 correct answers per mock exam, then this is an indicator that you are getting 85% of answers right (20.5/24 > 85%).

The above maths are crude here and there, and I have taken a few shortcuts (one of the shortcomings is that we have not looked how good the estimate actually is; but rather how to get an estimate in the first place). The calculations are based on a few assumptions. The most important assumption is that the questions in the mock exams test the same material (or are even the same questions) as the questions in the actual exam. Otherwise, above method would not work at all because your training material does not match the exam material. For the Life in the UK test and the cited website for practicing, I can say that this assumption holds─from my experience at least.