| Human resource professionals are unlikely to need | | | | test itself (such as ambiguous questions), the |
| any convincing that the use of psychometric tests as | | | | respondent (such as mood or exposure to tests) and |
| an aid to employee selection and development is | | | | the testing environment (such as noise and heat) can |
| probably at an all time high. | | | | all impact upon the reliability of a test. |
| The increase in the use of aptitude and personality | | | | The website cited above stated that, as with validity, |
| tests in the workplace is a positive thing provided the | | | | reliability is best assessed by having somebody you |
| tests are chosen and used properly. | | | | know complete the test! In fact, reliability is typically |
| This article discusses what decision-makers should | | | | assessed by using the results of a sizable group of |
| look for in order to be confident they are making the | | | | people, not just one or two people. |
| right test choice. | | | | As with validity, there are a number of forms of |
| The Hong Kong website of an employee testing | | | | reliability. Among them, internal consistency assesses |
| system that is marketed worldwide claims: | | | | the extent to which each question in the test is |
| "Really, what is the most effective way to evaluate | | | | related to the overall scale score, whilst test-retest |
| the reliability and validity of any assessment tests so | | | | assesses the consistency of test scores overtime. |
| to help us to know exactly how to find the right | | | | Reliability information should also be found in the |
| productive people with certainty and predictability | | | | publisher's manual. If it is missing or inadequate, it |
| without any catastrophe in hiring any wrong people | | | | raises serious doubts about the integrity of the test. |
| who simply look good?" | | | | Most psychometric tests that are used in selection |
| "The most workable and effective answer of the | | | | require the comparison of the candidate's results to a |
| above questions is simply to TEST THE PEOPLE YOU | | | | group of similar others. This is how the score is made |
| KNOW VERY WELL; then you know which | | | | meaningful. |
| assessment test can be valid and reliable to use!" | | | | If I told you I scored 7/20 on extraversion, this |
| This perspective is fundamentally flawed. It appears | | | | would mean very little to you. You might ask me |
| that the person who wrote it has no more | | | | how other people who took the test scored. |
| understanding of tests than the average HR | | | | It is therefore important that a test has been |
| executive who wants to understand more and is | | | | standardised on suitable groups of people - often |
| looking to this article to assist them! | | | | referred to as a norm group. |
| Many laypersons might assume that you can assess | | | | One of our clients reported to us that she contacted |
| the validity of a test by self-completing it and/or | | | | the publisher making the claims above by email. She |
| asking somebody they know well to do likewise. | | | | asked about reliability and validity of the test as well |
| Obviously, the assumption is that we know ourselves | | | | as whether or not local norms were available. She |
| well and so if the test report provides an accurate | | | | never received a reply, despite 3 reminder emails! |
| reflection of the self that we know, it "must" be | | | | Additionally, if a test developed in one country is |
| valid. | | | | taken to another, it must go through a lengthy |
| However, research shows us that individuals make | | | | process of translation, validation and reliability checks. |
| flawed assessments of test reports. | | | | Many people do not realise this and assume that a |
| In one research study, human resource professionals | | | | test can easily be transported from one country to |
| attending a conference were asked to complete a | | | | another just by taking it to a professional translator. |
| personality test. Following this, they were given a | | | | This is not true! |
| randomly generated narrative report. They were | | | | As the use of psychometric tests in selection and |
| NOT told that it had been randomly generated and | | | | development continues to soar, the human resource |
| wereasked to evaluate its accuracy. | | | | professional will need an understanding of how to |
| 90% of the respondents agreed that the report was | | | | evaluate tests. |
| either amazingly accurate or very accurate | | | | In Asia in particular, we are noticing an influx of test |
| (remember, the report was randomly generated). | | | | publishers and distributors. |
| It's partly for reasons such as the above that various | | | | However they are not always reputable and many |
| worldwide psychological societies and academics have | | | | do not have psychologists in the business at all! We |
| suggested that we assess at least 4 types of validity | | | | have even heard of interested parties being told by |
| when evaluating tests. | | | | publishers that validity information is protected and |
| Validity refers to whether or not the test is fit for | | | | not available. |
| purpose (i.e., does it measure what it is supposed to | | | | Worse still, one provider who has fairly recently |
| measure or can it predict something that is | | | | headquartered in Hong Kong and is expanding |
| meaningful, such as performance?). | | | | throughout Asia claims that the founder has a PhD |
| Let's have a look at these important aspects of test | | | | from a US university, yet when a client of ours |
| validity: | | | | contacted the university, he was informed that they |
| Face Validity: Here, we simply ask if the questions in | | | | have no record of the founder's PhD! |
| the test look like they are measuring what the test | | | | The publisher referred to at the top of this article, |
| purports to measure. | | | | states: |
| If I claim my test assesses numerical reasoning and | | | | "Instead of using "years" to really know the person |
| you don't see any numerical data in the questions, | | | | followed with all kinds of risk, you can depend on our |
| you would doubt it has face validity. | | | | test to instantly know the person." |
| Assessing this type of validity is somewhat | | | | Whilst well designed, tested and validated |
| subjective and so it is considered to be the lowest | | | | assessments do provide extensive information on |
| level of validity. | | | | respondents that cannot be reliably and validly |
| Content Validity: We need to know whether the test | | | | obtained using other less scientific methods, no |
| questions are sufficiently representative of all of the | | | | reputable test publisher or distributor will claim their |
| possible questions that could assess the construct | | | | test can assist you to "instantly know the person". |
| we are interested in. | | | | In fact, it is imperative to schedule a feedback |
| For example, if we want to assess conscientiousness, | | | | session with your respondent following personality |
| but my test simply asks questions that relate to your | | | | testing to ensure that the profile you have for your |
| preference for following rules (only one aspect of | | | | respondent is valid. These sessions aim to elicit |
| conscientiousness), my test is unlikely to have | | | | behavioural evidence from the respondent. |
| content validity. | | | | Even after this exercise, the test user does not |
| Construct Validity: You may have decided that your | | | | "know the person". They will rather have a good |
| face-to-face salesperson must have a high level of | | | | understanding of the individual which will assist them |
| self-confidence. If you are considering using an | | | | in their selection and placement decision. |
| assessment of self-confidence to assist in your hiring | | | | Psychometric tests are thus useful tools in selection |
| decision, you'll need to evaluate whether the test | | | | and development. They have their limitations |
| really does assess the construct of self-confidence | | | | however. |
| that it claims to measure. | | | | With the growth of the industry and the adoption of |
| The best way to do this is to look in the publisher's | | | | test businesses by non-psychologists, it is in the |
| manual for the test and find evidence that the | | | | interest of the test purchaser/HR professional to |
| publisher has correlated scores on this test with | | | | ensure they are suitably keyed up on how to |
| scores on established tests of the same construct. | | | | evaluate the tests being marketed to them rather |
| This aspect of validity is cited as one of the two | | | | than blindly accepting strongly marketed but |
| most important. It is however somewhat technical as | | | | non-supported claims about tests. |
| numbers are involved. It is better understood | | | | For those in Asia interested in finding out more about |
| following training in the test or psychometric | | | | the above, PsyAsia International will be running a |
| assessment generally. | | | | two-hour seminar in Hong Kong and Singapore in |
| Criterion Validity: This evidence is less easy to obtain | | | | January 2008. Please visit the PsyAsia website (see |
| than construct validity evidence, however it is also | | | | below) and click on "HR Seminar Series" for further |
| cited as one of the two most important areas of | | | | details and registration. |
| validity. | | | | Note on author's authority in this area: |
| Here we need to link scores on our test with | | | | The author is a doctoral-level registered organisational |
| performance. So, to take the above example again, | | | | psychologist (Australia and Hong Kong). He has been |
| one would expect scores on self-confidence to | | | | delivering training in psychometric assessment that |
| predict face-to-face sales performance. If they do, | | | | leads to the British Psychological Society's Certificates |
| our test has criterion-validity. | | | | of Competence in Occupational Testing for over 10 |
| Again, the HR professional would look to the | | | | years. |
| publisher's manual for evidence rather than carrying | | | | His research in psychometric test validity has been |
| out the study themselves. | | | | recognised by the British Psychological Society with |
| So, in terms of validity at least, evaluating and | | | | an award for Scientific Contribution to Occupational |
| choosing the right test is a lot more complex than | | | | Psychology. |
| simply completing the test yourself or handing it to | | | | He has published his psychometric validation research |
| your colleague! Now we turn to reliability. | | | | in peer-reviewed international journals and he has |
| Reliability refers to the consistency with which a test | | | | reviewed related papers for the Society for Industrial |
| assesses the construct of interest. | | | | & Organizational Psychology (USA). |
| Simply put, if I were to test you today and you | | | | His MSc research studied the link between personality |
| scored 6 and tomorrow you scored 12, ignoring | | | | and well-being/stress disorders in ambulance crews, |
| practice effects, we might suggest there is | | | | whilst his PhD research investigated the validity of |
| something wrong with the test! | | | | personality theory and questionnaires for the |
| A more practical example would be the faith that | | | | prediction of work performance in the People's |
| you might place in a tape measure...if you measure | | | | Republic of China, Hong Kong SAR, Singapore, |
| the length of a table today and tomorrow and get | | | | Australia, New Zealand and the United Kingdom. |
| differentresults, you know something is wrong. The | | | | He has significant experience in testing, working with |
| measurement is inconsistent and so it is not reliable. | | | | multi-national companies worldwide as well as working |
| Reliability is vital for a test because if a test lacks | | | | with the Governments of Hong Kong SAR, Macau |
| consistency of measurement it can never be valid! | | | | SAR, |
| No test is 100% reliable, just like no method of | | | | Malaysia, Singapore and the UAE. |
| assessment is 100% reliable. Factors related to the | | | | |