Because of Ceiling Effects, Is the Practical Ceiling on the Mega Test an IQ of 177+ (Raw Score = 43+)?
[Author's Note: I originally wrote this up as an
idle speculation intended for a handful of readers. However, some of my pages
are surfacing high on the hit lists of Google searches, and this is motivating
me to review what I've said, with an eye toward improving accuracy and
I don't mean to imply that the IQs of the highest scorers aren't as high as the Mega Test indicates. I just don't think that all of them can be at those levels. What I'm suggesting is that however intelligent they may actually be, their scores on the Mega Test set a lower bound on their IQs that's lower than what the current raw-score-to-IQ conversion table would indicate. My concern is that
(1) it seems to me that there are too many 1-in-110,000,000 and 1-in-30,000,000 scores to be reasonable for the population from which these high scorers are drawn, and
(2) ceiling effects might be expected to limit the absolute accuracy of the Mega Test's highest scores. A rough figure of merit that I've seen for ceiling effects is that they become serious once the raw score on a test exceeds about 90% of the total score. I'm suggesting that above a raw score of about 43, the Mega Test may not discriminate very well among its test subjects.
This is not to impugn the Mega Test in any way. I'm exceedingly impressed with it, and with its creator, Dr. Ron Hoeflin, as well as with the other Hoeflin tests.
The Problem: Too
Many Too High
A score of 47 on the Mega Test is equated with a deviation IQ of 190+ (standard deviation of 16), corresponding to an expected frequency of occurrence of 1 in 110,000,000+. I know of three individuals, all U. S citizens, who earned scores of 47 on the Mega Test, so they would constitute the quota for 330,000,000+ U. S. citizens. (There may have been additional scores of 47 on the Mega Test.) A score of 46 on the Mega Test is equated with a deviation IQ of 186, corresponding to an expected frequency of occurrence of 1 in 30,000,000+. I know of four U. S citizens who earned scores of 46 on the Mega Test. They would account for an additional 120,000,000+ U. S. citizens, for a total of 450,000,000+ U. S. citizens. Since there are only about 200,000,000 adult U. S. citizens, one wonders if these scores might be a bit high.
Possible Remedy: Reducing Our Topmost Claims
If we back them off one Mega-Test point, replacing a 47 with a 46, and a 46 with a 45, we would be assuming that the Mega Test has identified the seven brightest out of 126,000,000-or-more Americans. I'm also skeptical about that.
Dropping back two points reduces our quota to 41,000,000+ U. S. adults, with the three who scored 47 earning IQ's of 183 and the four who scored 46 assigned an IQ of 180+.. And dropping back three points brings us down to a quota of 14,500,000+ adults, and a maximum deviation IQ of 180+ and 177+, respectively.
. The last set of numbers sounds more plausible to me than the preceding three. It would seem surprising to me if more than a modest fraction of the smartest people in the country heard about, and took the Mega Test. For example, I suspect that many of them wouldn't have been willing to invest the time.
One of the interesting considerations about the Mega Test is that two of the three people who scored 46 on the Mega Test were women, one of whom was Marilyn vos Savant. Considering that the standard deviation for women's IQs is of the order of 5/6ths of the standard deviation for male IQ's, the maximum reasonable deviation IQ for the other woman who scored 46 on the Mega Test would be, perhaps, 174, corresponding to 1 in 30,000,000+ women. (I'm citing the other woman who scored a 46 because, while I might justify an anomalously high score for Marilyn vos Savant, I can't peg it that high for two women. Of course, this is predicated upon the notion that a standard deviation of about 13.6 is applicable to women-in-general.) To assign a higher value to a score of 46 on the Mega Test would seem to require that there are some women who fall completely off the frequency distribution for women. That would seem to equate a score of 46 on the Mega Test with an IQ of no more than 174+ (13.6/16 X 86 + 100).
I don't feel comfortable with this conclusion. It could be that the standard deviation for women is larger than 13.6. It could also be that some extraordinary women do, in fact, "fall off the curve". I'm going to assume that a score of 46 or higher on the Mega Test represents an IQ score of 177+. So what about scores of 43, 44, and 45? One way to go is to prorate them between 38 and 45, and the other might be to assign an IQ of 177+ to a 43 just as is currently assumed, and then treat higher scores the way Dr. Hoeflin has dealt with a perfect score of 48. I'm going to suggest that scores higher than 43 can't be assigned a very reliable IQ value.
It's worth noting that the analysis conducted by the Prometheus Society's Membership Committee showed extreme unreliability for raw scores above about 39. The Mega 27 was somewhat better, exhibiting reasonable reliability for scores up to 24, which might represent, perhaps, an IQ of 177+... the threshold for the Mega Society.
Interestingly, although scores above 39 on the Mega48 appear to become quite inaccurate, an analysis of the item scores of those who scored 43 or higher on the Mega48 indicates that all but one of them would have scored 24 or higher on the Mega24, and two who scored a 42 on the Mega48 had scores of 21 on the Mega21. Thus, a score of 43 on the Mega48 appears to have been a suitable choice for the one-in-a-million threshold for the Mega Society.
At What Score Do Ceiling Effects Limit the Reliabilities of Topmost Scores?
A ceiling effect would seem to apply for everyone who got a score of 43 or above. So everyone who made a Mega 48 score of 44 or above wouldn't have been measured on the Mega Test. All it could have provided would have been a floor score for them.
What IQ Does a Raw Score of 36 Portend?
The question arises: if a score of 46 only represents a deviation IQ of 177+, what does a score of 36 portend? I don't know that it necessarily follows that a raw score of 36 isn't consonant with a deviation IQ of 164+. As mentioned above, scores become quite unreliable above a raw score of 39, corresponding to a Mega-derived IQ of about 168+. In the Prometheus Society's Membership Committee Report, in the section reviewing the Concept Mastery Test, the results are shown for the Termites (IQ 140 and above), members of the Triple Nine Society (IQ 150 and up), and Members of the Xenophon Society (IQ 160 and up). The median raw score for the adult Termites on this test was 141. The median score for Triple-Nine members was 160. To get an idea of what this means, we can use statistics for the Termites to give us a clue. Out of the 1,004 Termites who took the CMT-A, 231 scored 160 or above on the CMT-A. I'm going to assume that this group of Termites was drawn from a potential group of 300-to-350 individuals who were at this level among the 258,000 children who were screened for the study. This corresponds to an expected frequency of occurrence among the 258,000 children of one in 740 to one in 860. This would correspond to a three-sigma deviation from the mean, and an IQ of 148-149. The median score for the Xenophon members was 180, which was very close to a perfect score of 190. Twenty of the Termites, or one in 13,000 earned scores this high, corresponding to a median IQ in the neighborhood of 160 This suggests that the IQs of Prometheans are in a very rarified range, and would support the contention that a score of 36 on the Mega Test does indeed indicate intelligence in the 4-sigma range.
Roles of Persistence, Rechecking, and Access to Library Resources in Generating Mega Test Scores
One problem with the Mega Test and with other take-as-long-as-you-want, you-mustn't-draw-upon-unauthorized-resources-for-help types of IQ tests is that they probably don't precisely measure IQs. Persistence, and the ability to utilize library resources must play a role in the scores someone earns on these tests. The spread of correct answers to different questions by Mega Test subjects who, for example, answered only one of its question correctly, or two or three of them shows that a group of people with approximately the same IQ can answer many more questions than their common IQ would suggest. (This would seem to me to have important implications for what's happening today in the scientific community. Perhaps, many people with IQs of 130 can, match the problem-solving ability of one person with an IQ of 150.)
Cross-Checks Upon the Capabilities of Mega Test Members
One way to assess the effectiveness of the Mega Test is to consider the intellects of those who have gotten the highest scores. In my experience over the past three+ years, everyone I've known who made a 36 or higher on the Mega Test has been phenomenally brilliant. Of course, it's hard to assign IQs to people on the basis of infrequent contact. It's also difficult to assess individuals who are far smarter than those we normally meet, so it would be unreliable for me to try to decide whether one of them has an IQ at the one-in-eleven-thousand level (IQ = 160) or at the one-in-thirty-one-thousand level (IQ = 164). one-in-thirty-one-thousand). Still, taken all-in-all, I think that the Mega Test has done a good job of measuring IQs that are too lofty to benchmark on virtually all conventional IQ tests. One feature about it that I find particularly attractive is that its non-verbal questions seem to me to be similar to those that you face in the real world of mathematical and physical problem-solving.
I hold no briefs for the conclusions I'm suggesting here--only observing that they would seem to me to be uncomfortable consequences of a reality check of the derived IQ scores. What do you think?
Using the Concept Mastery Tests (CMT-A and CMT-T) to Measure the Termites' Adult IQs
The bar charts below, taken from the Prometheus Society's Membership Committee Report, show the raw scores earned by the adult "Termites" in 1952 on the Concept Mastery Test-T (CMT-T). The average of the adult "Termite" scores on this test was 136.7 (Genetic Studies of Genius: Volume V, "The Gifted Group at Mid-Life", Thirty-Five Years' Follow-Up of the Superior Child, Lewis M. Terman and Melissa H. Oden, Stanford University Press, Stanford, California, 1959, pgs. 52-63 ).
The CMT-T was an updated and less difficult version of the CMT-A adult IQ test used in 1940 to assess the intelligence of the adult "Termites". As may be seen from the bar charts, there were severe ceiling effects for the CMT-T test scores made by all the groups taking the test. Sixteen out of 551 adult males who took the CMT-T test in 1950-52 fell in the 180-190 raw score range. To get an idea about what IQ level a CMT-T raw score of 180 portends, we may observe that on the previous CMT-A test, 21 of the adult "Termites" scored at or above 150 (raw score) out of a possible 190), corresponding to an estimated (by me) IQ of 160 or higher. Consequently, we might equate a raw score of 180 on the CMT-T with an adult deviation IQ of 160, and might set the ceiling of the CMT-T at 164. However, there is one other proviso to be considered. The researchers who administered the CMT-T test in 1950-52 estimated, using a line of equivalences" that the "Termites" would have scored about 15 points of raw score higher on the CMT-A IQ test if they had taken it at the age of 50-ish than they did in 1940 at the age of "40-ish". In other words, as measured by the CM tests, they got smarter by about 15 points of raw score or 6 points of IQ over the 10 years between taking the CMT-A in 1940 and taking the CMT-T in 1950-52. If that's the case, then a score of 180 on the CMT-T might imply an adult deviation IQ of 166, while the ceiling of the test might lie at IQ 170.