Spearman’s g and what cross-culture tells us

Saw this today. There exist many similar studies. They seem to say a lot. They seem to say nothing. I’ll comment on some moving parts below.

Spearman’s g Found in 31 Non-Western Nations: Strong Evidence That g Is a Universal Phenomenon
Abstract
Spearman’s g is the name for the shared variance across a set of intercorrelating cognitive tasks. For some—but not all—theorists, g is defined as general intelligence. While g is robustly observed in Western populations, it is questionable whether g is manifested in cognitive data from other cultural groups. To test whether g is a cross-cultural phenomenon, we searched for correlation matrices or data files containing cognitive variables collected from individuals in non-Western, nonindustrialized nations. We subjected these data to exploratory factor analysis (EFA) using promax rotation and 2 modern methods of selecting the number of factors. Samples that produced more than 1 factor were then subjected to a second-order EFA using the same procedures and a Schmid-Leiman solution. Across 97 samples from 31 countries totaling 52,340 individuals, we found that a single factor emerged unambiguously from 71 samples (73.2%) and that 23 of the remaining 26 samples (88.5%) produced a single second-order factor. The first factor in the initial EFA explained an average of 45.9% of observed variable variance (SD = 12.9%), which is similar to what is seen in Western samples. One sample that produced multiple second-order factors only did so with 1 method of selecting the number of factors in the initial EFA; the alternate method of selecting the number of factors produced a single higher-order factor. Factor extraction in a higher-order EFA was not possible in 2 samples. These results show that g appears in many cultures and is likely a universal phenomenon in humans.
Warne, R. T., & Burningham, C. (2019). Spearman’s g found in 31 non-Western nations: Strong evidence that g is a universal phenomenon. Psychological Bulletin, 145(3), 237-272. http://dx.doi.org/10.1037/bul0000184

Spearman’s g

… stands for “general factor” in intelligence, “found” by the inventor of factor analysis, Charles Spearman. Basically, there existed a number of cognitive tests all aimed at measuring intelligence. (On whether they, in fact, do measure intelligence, I recommend the great book by Stephen Jay Gould, The Mismeasure of Man, as supplementary reading.) The correlations among different answers one person gives to a number of tests were calculated (by Spearman, by hand, which was pretty remarkable). He found that, because the test scores were highly correlated amongst themselves (that is, if one person scores high on some tests, they’ll likely score high on others; this phenomenon is called the positive manifold), if he posited a latent variable (“latent” because it’s not explicitly measured by any particular test), then the variation of that variable can explain a large amount of variation in all the test scores. He calls this variable general intelligence, or g.

“Explain/capture the variation”

The variance, or “amount of variation”, refers to how much my score on a test varies from yours and other people’s. Suppose we have 10,000 test takers answering the same 2 questions, then their answers on each question will differ from each other. That’s variation. If people’s scores differ a lot from one another, the variation is large. If there are many tests, the variation is hard to interpret. However, I can assert that people’s score on test 1 will always be identical with their score on test 2. That’s a latent variable. My assertion will be true for some people, but false for others. Even for people whose scores are identical, there will still be individual differences. I can also calculate the extent to which people’s scores deviate from this assertion, and that’s residual variation. The residual variation will always be smaller than the original variation because part of the original variation is “captured” or “explained” by the variation of the latent variable. If the difference between residual and original variation is large enough, we deem it worthwhile to investigate the latent variable further as an explanatorily useful theoretical entity.

Thurstone’s alternative factors

Spearman’s technique starts by finding the latent variable that captures the most variance, and then see if the remaining variance can be captured by a secondary factor. He found that there wasn’t much to be gained by positing a secondary factor that wasn’t just residual. He then concluded that there existed a single intelligence factor g.

Using a different technique, L.L. Thurstone was able to extract multiple factors from the same positive manifold simultaneously. An important consequence, in philosophical terms, was that the data (of positive manifold) underdetermines Spearman’s theory. This insight was lost, however, in the inter-continental turf war. The two techniques were later combined by Spearman’s student, Cyril Burt, who was also known for an immensely influential but fabricated twin study on the genetic basis of intelligence (“The Burt Affair”).

In any case, the point here is that Spearman’s g is not necessitated by the positive manifold.

A more realistic look at “cross-cultural g“

What does g measure, exactly? In crude terms, it measures: whether or not scores from a number of different tests all constructed to measure intelligence highly correlate among themselves. The answer seems to be “yes”, even cross-culturally.

Here’s what it does not tell us: whether the correlation has anything to do with the human psyche.

There exists much skepticism over whether the positive manifold is innate or a function of education and SES. I will not write on that here because the topic is too big.

Instead, I’ll only raise a small concern: your GRE score probably highly correlate with your SAT score, which probably highly correlate with your GPA. Why? You might be smart. You might come from a family that values education. So on and so forth… but the GRE and the SAT might just be the same test.

Suppose, again, we have 10,000 people doing two test, TEST 1 and TEST 2, and we find that their scores are highly correlated. The response from factor analysis is to say that “this is because there is a single latent variable causally contributing to these tests.” Now, suppose I tell you that TEST 1 is a GRE taken a month ago; TEST 2 is the same GRE taken yesterday. Of course the scores will be the same — it’s the same test! Now, the response then becomes “there is a single latent variable causally contributing to this one test”. That might be true; that might not be. But it’s not something factor analysis can help you decide.

To be fair, I’m not suggesting that this is what’s happening in intelligence research. If anything, the sheer quantity of research out there makes it unlikely that all tests are the same. However, I still think that this is a non-negligible problem. What factor analysis tells us is whether the scores of different tests are similar. This is only revealing if we thought the tests were different. The more alternative explanations we have about why the tests are similar, the less impressive the positive manifold is.

Author
Recent Posts

Kino

Kino specializes in the philosophy of statistics and its application in the social sciences. She looks at the methodology of social sciences in general but psychology in particular through the lens of data analysis. Kino posts under the banner "Scattered Plot".