| Home |
Because of Ceiling Effects, Is the Practical Ceiling on the Mega Test an IQ of 177+ (Raw Score = 43+)?
[Author's Note: I originally wrote this up as an
idle speculation intended for a handful of readers. However, some of my pages
are surfacing high on the hit lists of Google searches, and this is motivating
me to review what I've said, with an eye toward improving accuracy and
restraint.
I don't
mean to imply that the IQs of the highest scorers aren't as high as the Mega
Test indicates. I just don't think that all of them can be at those levels. What
I'm suggesting is that however intelligent they may actually be, their scores on the Mega
Test set a lower bound on their IQs that's lower than what the current raw-score-to-IQ
conversion table would indicate. My concern is that
(1) it seems to me that there are too many 1-in-110,000,000 and 1-in-30,000,000
scores to be reasonable for the population from which these high scorers are
drawn, and
(2) ceiling effects might be expected to limit the absolute accuracy of the Mega
Test's highest scores. A rough figure of merit that I've seen for ceiling
effects is that they become serious once the raw score on a test exceeds about
90% of the total score. I'm suggesting that above a raw score of about 43, the
Mega Test may not discriminate very well among its test subjects.
This is not to impugn the Mega Test in any way. I'm
exceedingly impressed with it, and with its creator, Dr. Ron Hoeflin, as well as
with the other Hoeflin tests.
The Problem: Too
Many Too High
A score of 47 on the
Mega Test is equated with a deviation IQ of 190+ (standard deviation of 16), corresponding to
an expected frequency of occurrence of 1 in 110,000,000+. I know of three
individuals, all U. S citizens, who earned scores of 47 on the Mega Test, so they would constitute the quota for 330,000,000+ U. S.
citizens. (There may have been additional scores of 47 on the Mega Test.) A score of 46 on the Mega Test is equated with a
deviation IQ of 186, corresponding to an expected frequency of
occurrence of 1 in 30,000,000+. I know of four U. S citizens who
earned scores of 46 on the Mega Test. They would account for an
additional 120,000,000+ U. S. citizens, for a total of 450,000,000+
U. S. citizens. Since
there are only about 200,000,000 adult U. S. citizens, one
wonders if these scores might be a bit high.
Possible
Remedy: Reducing Our Topmost Claims
If we back them
off one Mega-Test point, replacing a 47 with a 46, and a 46 with
a 45, we would be assuming that the Mega Test has identified the seven brightest out of
126,000,000-or-more Americans. I'm also skeptical about that.
Dropping back two points reduces our quota to 41,000,000+
U. S. adults, with the three who scored 47 earning IQ's of 183 and the four who scored 46 assigned an IQ of
180+.. And dropping
back three points brings us down to a quota of 14,500,000+ adults,
and a maximum deviation IQ of 180+ and 177+, respectively.
. The last set of numbers sounds
more plausible to me than the preceding three. It would seem surprising to me if more than a modest fraction of the smartest people in the
country heard about, and took the Mega Test. For example, I suspect that many of
them wouldn't have been willing to invest the time.
One of the interesting considerations about
the Mega Test is that two of the three people who scored 46 on
the Mega Test were women, one of whom was Marilyn vos Savant.
Considering that the standard deviation for women's IQs is of the
order of 5/6ths of the standard deviation for male IQ's, the
maximum reasonable deviation IQ for the other woman who
scored 46 on the Mega Test would be, perhaps, 174, corresponding
to 1 in 30,000,000+ women. (I'm citing the other woman who scored
a 46 because, while I might justify an anomalously high score for
Marilyn vos Savant, I can't peg it that high for two women. Of
course, this is predicated upon the notion that a standard
deviation of about 13.6 is applicable to women-in-general.) To
assign a higher value to a score of 46 on the Mega Test would
seem to require that there are some women who fall completely off
the frequency distribution for women. That would seem to equate a score of 46 on the Mega Test
with an IQ of no more than 174+ (13.6/16 X 86 + 100).
I don't feel comfortable with this conclusion. It could be
that the standard deviation for women is larger than 13.6. It could also be that
some extraordinary women do, in fact, "fall off the curve". I'm going
to assume that a score of 46 or higher on the Mega Test represents an IQ score
of 177+. So what about scores of 43, 44, and 45? One way to go is to prorate them
between 38 and 45, and the other might be to assign an IQ of 177+ to a 43 just as
is currently assumed, and then treat higher scores the way Dr. Hoeflin has dealt
with a perfect score of 48. I'm going to suggest that scores higher than 43
can't be assigned a very reliable IQ value.
It's worth noting that the analysis
conducted by the Prometheus Society's Membership Committee showed
extreme unreliability for raw scores above about 39. The Mega 27
was somewhat better, exhibiting reasonable reliability for scores
up to 24, which might represent, perhaps, an IQ of 177+... the threshold for
the Mega Society.
Interestingly, although scores above 39 on the Mega48 appear
to become quite inaccurate, an analysis of the item scores of those who scored
43 or higher on the Mega48 indicates that all but one of them would have scored
24 or higher on the Mega24, and two who scored a 42 on the Mega48 had scores of
21 on the Mega21. Thus, a score of 43 on the Mega48 appears to have been a
suitable choice for the one-in-a-million threshold for the Mega Society.
At What Score
Do Ceiling Effects Limit the Reliabilities of Topmost Scores?
A ceiling effect would seem to apply for
everyone who got a score of 43 or above. So everyone who made a
Mega 48 score of 44 or above wouldn't have been measured on the
Mega Test. All it could have provided would have been a floor
score for them.
What IQ Does a
Raw Score of 36 Portend?
The question arises: if a score of 46
only represents a deviation IQ of 177+, what does a score of 36
portend? I don't know that it necessarily follows that a raw
score of 36 isn't consonant with a deviation IQ of 164+. As
mentioned above, scores become quite unreliable above a raw score
of 39, corresponding to a Mega-derived IQ of about 168+. In the
Prometheus Society's Membership Committee Report, in the section
reviewing the Concept Mastery Test, the results are shown for the
Termites (IQ 140 and above), members of the Triple Nine Society
(IQ 150 and up), and Members of the Xenophon Society (IQ 160 and
up). The median raw score for the adult Termites on this test was 141. The median score for Triple-Nine members was
160. To get an idea of what this means, we can use statistics for the Termites
to give us a clue. Out of the 1,004 Termites who took the CMT-A, 231 scored 160
or above on the CMT-A. I'm going to assume that this group of Termites was drawn
from a potential group of 300-to-350 individuals who were at this level among
the 258,000 children who were screened for the study. This corresponds to an
expected frequency of occurrence among the 258,000 children of one in 740 to one
in 860. This would correspond to a three-sigma deviation from the mean,
and an IQ of 148-149. The median
score for the Xenophon members was 180, which was very close to a
perfect score of 190. Twenty of the Termites, or one in 13,000 earned scores
this high, corresponding to a median IQ in the neighborhood of 160 This suggests that the IQs of Prometheans
are in a very rarified range, and would support the contention
that a score of 36 on the Mega Test does indeed indicate
intelligence in the 4-sigma range.
Roles of
Persistence, Rechecking, and Access to Library Resources in Generating Mega Test
Scores
One problem with the Mega Test and with other
take-as-long-as-you-want, you-mustn't-draw-upon-unauthorized-resources-for-help
types of IQ tests is that they probably don't precisely measure IQs.
Persistence, and the ability to utilize library resources must play a role in
the scores someone earns on these tests. The spread of correct answers to
different questions by Mega Test subjects who, for example, answered only one
of its question correctly, or two or three of them shows that a group of
people with approximately the same IQ can answer many more questions than their
common IQ would suggest. (This would seem to me to have important implications
for what's happening today in the scientific community. Perhaps, many people
with IQs of 130 can, match the problem-solving ability of one person with an IQ
of 150.)
Cross-Checks
Upon the Capabilities of Mega Test Members
One way to assess the effectiveness of the Mega Test is to
consider the intellects of those who have gotten the highest scores. In my
experience over the past three+ years, everyone I've known who made a 36 or
higher on the Mega Test has been phenomenally brilliant. Of course, it's hard to
assign IQs to people on the basis of infrequent contact. It's also difficult to
assess individuals who are far smarter than those we normally meet, so it would
be unreliable for me to try to decide whether one of them has an IQ at the
one-in-eleven-thousand level (IQ = 160) or at the one-in-thirty-one-thousand
level (IQ = 164). one-in-thirty-one-thousand). Still, taken all-in-all, I think
that the Mega Test has done a good job of measuring IQs that are too lofty to
benchmark on virtually all conventional IQ tests. One feature about it that I
find particularly attractive is that its non-verbal questions seem to me to be
similar to those that you face in the real world of mathematical and physical
problem-solving.
I hold no briefs for the conclusions I'm
suggesting here--only observing that they would seem to me to be
uncomfortable consequences of a reality check of the derived IQ
scores. What do you think?
Using the
Concept Mastery Tests (CMT-A and CMT-T) to Measure the Termites' Adult IQs
The bar charts below, taken from the Prometheus Society's Membership
Committee Report, show the raw scores earned by the adult
"Termites" in 1952 on the Concept Mastery Test-T (CMT-T). The average
of the adult "Termite" scores on this test was 136.7 (Genetic
Studies of Genius: Volume V, "The Gifted Group at Mid-Life",
Thirty-Five Years' Follow-Up of the Superior Child, Lewis M. Terman and
Melissa H. Oden, Stanford University Press, Stanford, California, 1959, pgs.
52-63 ).
The CMT-T was an updated and less difficult version of the CMT-A
adult IQ test used in 1940 to assess the intelligence of the adult
"Termites". As may be seen from the bar charts, there were severe
ceiling effects for the CMT-T test scores made by all the groups taking the
test. Sixteen out of 551 adult males who took the CMT-T test in 1950-52 fell in
the 180-190 raw score range. To get an idea about what IQ level a CMT-T raw
score of 180 portends, we may observe that on the previous CMT-A test, 21 of the
adult "Termites" scored
at or above 150 (raw score) out of a possible 190), corresponding to an
estimated (by me) IQ of 160 or higher. Consequently, we might equate a raw score
of 180 on the CMT-T with an adult deviation IQ of 160, and might set the ceiling
of the CMT-T at 164. However, there is one other proviso to be considered. The
researchers who administered the CMT-T test in 1950-52 estimated, using a line
of equivalences" that the "Termites" would have scored about 15
points of raw score higher on the CMT-A IQ test if they had taken it at the age
of 50-ish than they did in 1940 at the age of "40-ish". In other
words, as measured by the CM tests, they got smarter by about 15 points of raw
score or 6 points of IQ over the 10 years between taking the CMT-A in 1940 and
taking the CMT-T in 1950-52. If that's the case, then a score of 180 on the CMT-T
might imply an adult deviation IQ of 166, while the ceiling of the test might
lie at IQ 170.
