Assess thyself

[CarpCharacin]

I wish I had saved the Form A files. I think I'm a cognitive cripple, even though we both scored above the putative ceiling of Form B at 15-18.

 

[Sapphire Sebright]

This is.... interesting.
I pasted the answers into a google doc and then assigned a raw score point for correct answers and 0 points for incorrect answers:

Spoiler: Scoring

View attachment 2470825

The test has an interesting scoring system. If one gets 5+ answers correct in a row, their previous mistakes are disregarded, so their raw score is derived by taking the item number of the last item they got correct in their most recent 5+ correct streak and then the number of subsequent correct answers is added to that. So I believe the final raw score would be 51/55 raw, which is admittedly higher than I expected. I think my final raw score was 50. Some of the items are admittedly kind of ambiguous but I'm still surprised, lol.
I only have Spanish norms for Form B:
View attachment 2470842
"PD" is the raw score, "CI" is the corresponding standard score, and "PC" is the percentile rank. They are kinda questionable, since the norms are more generous at 15-18 than at 11-14, and then they get a lot harsher at 19+, but the American Form A norms still yield a standard score above 130 for 44 raw in the 15-18 age range, so I suppose it's possible that the disparity in ability between the 15-18 age range and the 19-25 age range really is considerable.

Interesting... you have a really good grasp on this stuff. It's cool.

Also, Happy New Year, Carp!

 

[Sapphire Sebright]

I wish I had saved the Form A files. I think I'm a cognitive cripple, even though we both scored above the putative ceiling of Form B at 15-18.

So... despite evidence that you did excellent in something, you still think that you're crippled?
No offense, but I think that the issue is less that you're a cripple, and more that you have no self-esteem whatsoever. Your good scores aren't a fluke -- you are an intelligent young man, and there's no reason to not believe that, despite what some people may say.

 

[CarpCharacin]

So... despite evidence that you did excellent in something, you still think that you're crippled?
No offense, but I think that the issue is less that you're a cripple, and more that you have no self-esteem whatsoever. Your good scores aren't a fluke -- you are an intelligent young man, and there's no reason to not believe that, despite what some people may say.

My good scores are probably due to my prior familiarity with stuff! The exception being early childhood testing (which I flunked the nonverbal part of, and that part is similar to the TONI, but got >99th percentile verbal) however the heritability of cognitive ability is low at that age. Out of all the TONI-2 norms, I trust the American Form A ones the most, but I'm still dubious.

 

[Sapphire Sebright]

My good scores are probably due to my prior familiarity with stuff! The exception being early childhood testing (which I flunked the nonverbal part of, and that part is similar to the TONI, but got >99th percentile verbal) however the heritability of cognitive ability is low at that age. Out of all the TONI-2 norms, I trust the American Form A ones the most, but I'm still dubious.

Very few people are naturally good at tests. We all learn things -- nothing is a natural skill to us. We all improve by increasing our familiarity with individual subjects.
Some things, like patterns and missing parts of patterns, follow a logical route -- but logic isn't straight-cut. Different people approach the same situation and images with different logic. Grains in a picture of straight-up static might arrange themselves "logically" in one person's mind to form a flower, but another person might see a woven pattern.

I am genuinely curious, by the way -- how did you come to be exposed to these kinds of tests? Did your parents or school test you, or did you come across them yourself?

 

[CarpCharacin]

Very few people are naturally good at tests. We all learn things -- nothing is a natural skill to us. We all improve by increasing our familiarity with individual subjects.

It depends on the type of test. Tests that are designed to not depend on previous knowledge (like the TONI) will yield an inflated score if one trains for it, yet some studies apparently say that scores on the TONI don't go up a whole lot with practice. Publishers of cognitive tests fight ardently to keep the test content private, yet old versions often end up getting leaked online, usually the foreign versions, and that's why we were able to take the TONI without paying a psychologist big bucks.

Some things, like patterns and missing parts of patterns, follow a logical route -- but logic isn't straight-cut. Different people approach the same situation and images with different logic. Grains in a picture of straight-up static might arrange themselves "logically" in one person's mind to form a flower, but another person might see a woven pattern.

That's why it's important to avoid ambiguity when designing items, but that becomes more difficult when designing harder items. People do very well on tests of pattern recognition also tend to do well on tests which measure other cognitive abilities, like verbal ability.

I am genuinely curious, by the way -- how did you come to be exposed to these kinds of tests? Did your parents or school test you, or did you come across them yourself?

Like I said, I have been professionally-tested, but haven't always performed all that well. Large discrepancies between cognitive test scores in individuals are probably more common than some would like to admit, but one's level of ability is still largely an inborn trait. If one trains for a specific test, their score will go up, but there will be a concomitant drop in how much that score correlates with scores yielded by other tests designed to estimate the same latent variable. Cognitive ability is a natural trait, and it's what determines how hard one has to work to understand and learn things, and some things will always be out of the grasp of some people, regardless if how hard they work, due to limited cognitive ability.

I've heard that false positives are rarer than false negatives when testing for extreme cognitive ability, but that might not always be the case, if the norms of the test are inherently messed up, or if the examinee has a lot more experience doing the tasks that are used as a proxy for underlying ability than those in the normative sample / reference group.

 

[Sapphire Sebright]

I've heard that false positives are rarer than false negatives when testing for extreme cognitive ability, but that might not always be the case, if the norms of the test are inherently messed up, or if the examinee has a lot more experience doing the tasks that are used as a proxy for underlying ability than those in the normative sample / reference group.

If that's true, then that means that my PSAT scores had a chance of being off. Maybe I didn't get high enough to go the the college I'm aiming for.

Like I said, I have been professionally-tested, but haven't always performed all that well.

And I've done not-so-good on tests before, too. It isn't always consistent. Some people might not have as high a cognitive function as others, but this doesn't mean that they're screwed over for life -- they can still live and work and learn what they want.

I've noticed that you seem to be extremely pessimistic. Can I ask you to try something?
Take a few different tests, and do them. Find ones that measure the same thing -- but don't pick tests that measure something that you know that you're bad at, that'll just make you feel worse (but we're all good at some things and really sucky at others). When you're done, add all of your scores together -- bad ones, good ones, all of them -- and divide by the number of tests.
How well did you do?
Even if you did really badly on some of them, there are good runs, too. It isn't necessarily that you're bad at the material -- it could just be a bad run.
Whatever you get, don't tell yourself that it was a fluke. Don't let yourself think that. Come right back here and tell me what you got, and how it compares to the scores in the test. Don't start thinking bad thoughts, just get back here and tell me.

You still have skills. You're still intelligent. I could never do what you're doing with these tests -- this is all you. The data interpretation, the grasping of the concepts, it's all you.

 

[Sapphire Sebright]

If that's true, then that means that my PSAT scores had a chance of being off. Maybe I didn't get high enough to go the the college I'm aiming for.

And I've done not-so-good on tests before, too. It isn't always consistent. Some people might not have as high a cognitive function as others, but this doesn't mean that they're screwed over for life -- they can still live and work and learn what they want.

I've noticed that you seem to be extremely pessimistic. Can I ask you to try something?
Take a few different tests, and do them. Find ones that measure the same thing -- but don't pick tests that measure something that you know that you're bad at, that'll just make you feel worse (but we're all good at some things and really sucky at others). When you're done, add all of your scores together -- bad ones, good ones, all of them -- and divide by the number of tests.
How well did you do?
Even if you did really badly on some of them, there are good runs, too. It isn't necessarily that you're bad at the material -- it could just be a bad run.
Whatever you get, don't tell yourself that it was a fluke. Don't let yourself think that. Come right back here and tell me what you got, and how it compares to the scores in the test. Don't start thinking bad thoughts, just get back here and tell me.

You still have skills. You're still intelligent. I could never do what you're doing with these tests -- this is all you. The data interpretation, the grasping of the concepts, it's all you.

And I am really bad at pep talks, I am sorry. I'm out of practice.

 

[CarpCharacin]

If that's true, then that means that my PSAT scores had a chance of being off. Maybe I didn't get high enough to go the the college I'm aiming for.

Academic achievement tests are a different beast, lol. I probably shouldn't beat myself up too much because I didn't score too far above average on the GED, because I probably did have significantly less exposure to the reference group, as the reference group was a group of recent high school graduates, and to pass the test, one needs to score better than 15% of the reference group, meaning that theoretically, about 15% of recent HS graduates would fail the test. My lowest score was on math, which was like 60th percentile when compared that reference group, and I was probably at least 2ish years younger than the reference group at the time, and only formally completed one quarter of 9th grade math. I still feel not-so-great for not doing better, but while I was taking the test, I actually thought that I'd undoubtedly fail to score above the 15th percentile cutoff, and ended the math test with 35-40 minutes remaining, so I guess the outcome could have been worse.

Anyway, there are tables which can be used to predict academic achievement scores using data from cognitive testing, tables which are generated using data from a sample of people who have taken both tests, and schools tend to diagnose learning disabilities if one scores much lower in an area on the academic achievement test than what would be predicted by their measured cognitive ability. However if that were to happen in my case, it'd be hard to distinguish an actual disability from a mere consequence of my unusual academic background.

And I've done not-so-good on tests before, too. It isn't always consistent. Some people might not have as high a cognitive function as others, but this doesn't mean that they're screwed over for life -- they can still live and work and learn what they want.

Though it also depends on what they want to do, lol. It of course doesn't take a whole lot of cognitive ability to do a simple job, but it takes a lot more ability to do something more complex, like being a researcher who discovers new things, and hopefully I have enough ability for that.

Take a few different tests, and do them. Find ones that measure the same thing -- but don't pick tests that measure something that you know that you're bad at, that'll just make you feel worse (but we're all good at some things and really sucky at others). When you're done, add all of your scores together -- bad ones, good ones, all of them -- and divide by the number of tests.
How well did you do?
Even if you did really badly on some of them, there are good runs, too. It isn't necessarily that you're bad at the material -- it could just be a bad run.
Whatever you get, don't tell yourself that it was a fluke. Don't let yourself think that. Come right back here and tell me what you got, and how it compares to the scores in the test. Don't start thinking bad thoughts, just get back here and tell me.

Hopefully I already have enough data to suggest that my high scores aren't a fluke, despite my bad performances, but I'm not sure. Composites of different tests (especially composites of tests which each measure different cognitive abilities) are also more than a simple average. Because the correlations between different tests are imperfect (even though they may be strong), it's rarer to do well across the board than the mean of the two tests would suggest. So for example, a Pearson correlation of .62 was reported between the TONI, and the verbal part of another test. If one got 130 (98th percentile) on both tests, a composite could be derived by using a formula, like this one.
15^2+15^2+2(0.62)(15)(15) is 729. The square root of 729 is 27, so if you take the sum of the two scores (260) subtract the 200 (since the mean of each score is 100) and divide the remaining 60 by 27, you get 2.22222222222, which is a z-score. You can turn it back into a standard score by multiplying it by 15 and adding back 100. So that leaves a composite of 133ish, if I'm using the formula correctly. Hopefully I am, but I ain't extremely confident, lol. I think the composite effects become more pronounced as the scores get higher, and they also of course become more pronounced if the correlations between the tests being composited are lower. More comprehensive tests frequently composite scores from 7-10 different "subtests", however if one has a large discrepancy between 2 or more of the subtests within a composite, the composite is deemed "uninterpretable". All cognitive tests typically measure a general factor, at least in the general population, but there are clusters of "broad abilities" which fall under that general factor (like fluid reasoning ability, long-term recall, etc.) and the correlations between tests which measure the same broad ability tend to be stronger than than those between tests which measure different broad abilities, even though all of the broad abilities of course fall under general ability.

 

[CarpCharacin]

Oops, I noticed some typos in my last post. Oh well, it's late.