Explanation: Reliability v's validity

As promised:

Reliability and validity can be confusing. I will try to clarify.

Any kind of test or measure - IQ tests, exams, langauge tests - should be both reliable and valid.

Being reliable means that it gets consistent results. Two people with the same ability should get the same result. The same person tested on more than one occasion should get the same result.

The problem with making an IQ test reliable is that it is hard to know someone's intelligence level without testing them, and if you then test them again on the same test, they will do better because of practice. Test can be made more reliable by reducing elements of chance or guesswork.

Being valid means that it measures what it aims to measure. Surprisingly often, this is not the case. For example, you might fail a memory test because you ran out of time. However, it was supposed to be a test of memory, not a test of speed. An IQ test is supposed to measure intelligence. You could not give someone a science test and say that it measured their intelligence, because knowledge of science is (in principle) separate from intelligence.

The problem with making an IQ test valid is that nobody seems to agree what exactly intelligence is. This makes it hard to know what kind of questions to include. For example, some people would include verbal questions such as:

What is the odd one out - hat, coat, scarf, shelf

Other people might say that language ability is separate from intelligence. Also, the question above could be accused of being culturally biased, as not all cultures wear hats etc.