Data-Driven Answers to Questions about GMAT – Part 3

This article is the third part in a series of three articles. You can find the first part on this link and the second part on this link.

Are the first few questions more important than the remaining ones?

While the common understanding is that the first 10 questions set the tone of the exam and are thus more important, GMAC has, on various occasions, refuted this understanding.

Who is correct?

The data supports that there is indeed some truth in the common understanding.

How did I find so?

By assigning weights to questions in the different sets. Let’s understand.

From the ESRs, I extracted the data that how many questions each person got wrong in each of the four sets. I also calculated the correlation between the total number of incorrect questions in a section and the sectional score (Verbal or Quant). Obviously, the more questions a person gets wrong, the lower is his sectional score. So, there is a negative correlation between the two variables here. However, let me just focus on the magnitude of the correlation that tells us how strongly these two variables are related.

Verbal

In the case of Verbal, when I assigned equal weights to the number of incorrect questions in the four sets, I saw a correlation of 78%. However, when I assigned different weights to the four sets such that I assigned the greatest weight to the first set of questions, equal second highest weight to the second and third set of questions, and lowest weight to the fourth set of questions, the correlation jumped to 85%. (For the curious ones, I assigned weights of 4, 2, 2, and 1 to the first, second, third, and fourth sets)

Interestingly, when I assigned the opposite weights i.e. highest weight to the fourth set and lowest weight to the first set, the correlation went down significantly to 65%.

Quant

In the case of Quant, when I assigned equal weights to the number of incorrect questions in the four sets, I saw a correlation of 62%, much lower than that in Verbal. Does it indicate that in Quant, it’s less about the number of questions you get wrong and more about the kind of questions you get wrong than in Verbal?

Probably yes*.

In Quant, when I assigned different weights to the four sets such that I assigned the greatest weight to the first set of questions, equal second highest weight to the second and third set of questions, and lowest weight to the fourth set of questions, the correlation jumped significantly to 79%.

When I assigned the opposite weights i.e. highest weight to the fourth set and lowest weight to the first set, the correlation went down significantly to 30%.

Clearly, the data indicates that the first few questions are, at least to some extent, more important than the others, more so in Quant than in Verbal.

However, I do not recommend that you devote ‘considerably’ more time to the initial few questions. You may spend 10-15 seconds more per question for the initial few questions, if you so need. However, spending 30-45 seconds more per question on the initial few questions may be counterproductive since the test may start throwing very difficult questions at you that you cannot handle, and then towards the end, by which you might have marked a number of such overly difficult questions wrong, you’d not even have time to get questions of manageable difficulty correct.

Also, we’ll see towards the end of the article that there are cases in which people who got more questions incorrect in the beginning were able to score higher than others who got more questions correct in the beginning.

Besides, the answer to the next question will make it clear that ‘always’ spending more time on the initial questions is not a good strategy.

*(I’m not sure because I have a data of only 75 ESRs, which are almost entirely of Indians who are consistently better in Quant than in Verbal (Median Verbal score: 31; Median Quant Score: 48). Once we have hundreds or thousands of ESRs of a mix of non-natives and natives, we’ll be much surer of the conclusions that we draw from the data.)

Does the GMAT algorithm follow a fixed pattern i.e. gives similar difficulty level of questions for similar performance?

The short answer is No.

In other words, if we have two people who each have gotten initial four questions correct, the test may start giving high difficulty level questions to one while it may continue giving medium difficulty questions to another.

Here are the difficulty levels of the Quant questions of the people who each got ‘zero’ questions incorrect in the first set of questions:

First SetSecond SetThird SetFourth Set
Medium HighMedium HighMedium HighMedium High
Medium HighMedium HighMedium HighMedium High
MediumMedium HighMedium HighMedium High
MediumMedium HighMedium HighHigh
MediumMedium HighMedium HighMedium High
MediumMedium HighMedium HighMedium High
Medium HighMedium HighMedium HighMedium High
Medium HighMedium HighMedium HighMedium High
Medium HighMedium HighMedium HighMedium High
MediumMedium HighMedium HighMedium High
MediumMedium HighMedium HighMedium High
MediumMedium HighMedium HighMedium High
MediumMedium HighMedium HighMedium High
Medium HighMedium HighMedium HighMedium High
MediumMedium HighMedium HighMedium High
MediumMedium HighMedium HighMedium High
Medium HighMedium HighMedium HighMedium High
Medium HighMedium HighMedium HighMedium High

As we can see that while the average question difficulty level for some was ‘Medium’, for others, it was ‘Medium-High’. Since each one of them got zero questions wrong in the first quarter, there is absolutely no reason for the algorithm to throw questions of different average difficulty levels to different people, unless the algorithm can follow different ‘paths’ even for people performing exactly the same.

At this stage, somebody may wonder that ‘timing’ could be a factor here i.e. probably the people who were shown ‘medium’ average difficulty questions took more time to solve those questions and that is why the test gave them only or mainly medium level questions. However, I don’t think so. The reason is that in my last attempt, I also got medium average difficulty level questions in Quant in the first quarter even though my average time per question in the first quarter was 1 min 21 sec, well below the available average time per question.

Even in the above 18 cases, for people who got ‘Medium’ average difficulty level questions in the first quarter, the time per question in the first quarter varied from 1 min 20 sec to 2 min 32 sec, and for people who got ‘Medium-High’ average difficulty level questions in the first quarter, the time per question in the first quarter varied from 1 min 24 sec to 2 min 49 sec.

Clearly, the time per question parameter cannot explain the difference in the average difficulty levels of the questions people faced.

Thus, from the data, it seems quite clear that the GMAT algorithm can follow different paths for different people performing the same.

What does this mean for you?

  1. I see that a lot of people try to gauge how they are performing on the test by judging the difficulty level of the questions. If they see a difficult question (or, as many people say, a bold-face question), they think they are doing well. On the other hand, if they see a number of easy-medium questions, they think they are not doing well. Frankly, even before the above understanding, gauging one’s performance while taking the test never made any sense to me. I don’t think anybody gains from such an exercise. If a person thinks he is doing well, he may become excited and may make some silly mistakes. On the other hand, if a person thinks he is not doing well, he may become disappointed and lost interest.

    In addition, there is always a possibility of encountering experimental questions during the test. The difficulty level of those questions may not depend on how one is performing. Thus, all the analysis of how one is performing may be entirely wrong.

    The above understanding that the test may follow different paths makes gauging one’s performance while taking the test even more non-sensical. Thus, while taking the test, focus your energies on solving the question on the screen and not on judging your performance. Let the GMAT algorithm take care of the latter part. Besides, taking care of the former part is the best you can do for your score.

  2. The idea of spending more time per question on the initial 10 questions also comes into doubt. In your case, the test may end up giving you only (or mainly) easier questions in the first 10 questions. If you end up spending more than average time per question on those questions, you seriously jeopardize your chances of scoring very high on the section since you don’t leave yourself enough time to tackle more difficult questions that you are going to encounter later in the test. You may refer to my ESR analysis on this link. You can see that the difficulty level of the quant questions I faced peaked in the last quarter. However, I had enough time (on an average, 2.5 minutes) to solve those hard questions towards the end since I saved time in the first quarter, in which I took just 1 minute 21 seconds per question.

    Thus, you should be fine taking less than average time per question on the first few questions if the situation so warrants. You may try to cross-check your answers a bit more on the initial few questions. However, you should not just ‘sit’ on the initial questions, thinking that you are solving them too quickly. If the questions are on the easier side, you can choose to save time from those questions to use in later questions that will likely be more difficult.

That’s it for this article. We have also reached the end of this article series. Hope you enjoyed it!

Leave a comment

Your email address will not be published. Required fields are marked *

fourteen + 3 =