Job Knowledge Tests

Job knowledge tests, sometimes referred to as achievement or mastery tests, typically consist of questions designed to assess technical or professional expertise in specific knowledge areas. Job knowledge tests evaluate what a person knows at the time of taking the test. Unlike cognitive ability tests, there is no attempt to assess the applicant's learning potential. In other words, a job knowledge test can be used to inform employers what an applicant currently knows, but not whether the individual can be relied on to master new material in a timely manner. Job knowledge tests are not appropriate when applicants will be trained after selection in the critical knowledge areas needed for the job.

Job knowledge tests are used in situations where applicants must already possess a body of learned information prior to being hired. They are particularly useful for jobs requiring specialized or technical knowledge that can only be acquired over an extended period of time. Examples of job knowledge tests include tests of basic accounting principles, computer programming, financial management, and knowledge of contract law. Job knowledge tests are often constructed on the basis of an analysis of the tasks that make up the job. While the most typical format for a job knowledge test is a multiple choice question format, other formats include written essays and fill-in-the-blank questions.

Licensing exams, agency certification, and/or professional certification programs are also job knowledge tests. Licensure and certification are both types of credentialing — the process of granting a designation that indicates competence in a subject or area. Licensure is more restrictive than certification and typically refers to the mandatory Governmental requirement necessary to practice in a particular profession or occupation. A passing score on a job knowledge test is typically a core requirement to obtain a professional license. Licensure implies practice and title protection. This means only individuals who hold a license are permitted to practice and use a particular title. For example, to practice law, a law school graduate must apply for admission into a state bar association that requires passing the bar licensure examination. Certification is usually a voluntary process instituted within a nongovernmental or single Governmental agency in which individuals are recognized for advanced knowledge and skill. As with licensure, certification typically requires a passing score on a job knowledge exam.

Considerations:

• Validity — Knowledge areas tested are very representative of those required to perform the job (i.e., high degree of content validity); Performance on job knowledge tests relates highly to performance on the job (i.e., high degree of criterion-related validity); Can add a substantial amount of incremental validity above and beyond the validity provided by general cognitive ability tests; Customized job knowledge tests have been shown to have slightly higher validity than off-the-shelf tests
• Face Validity/Applicant Reactions — Applicants often perceive job knowledge tests as being very fair (i.e., as having a high degree of face validity) because such tests are typically designed to measure knowledge directly applied to performance of the job
• Administration Method — Can be administered via paper and pencil or electronically
• Subgroup Differences — Tend to produce race and ethnic group differences larger than other valid predictors of job performance (e.g., work sample tests, personality tests)
• Development Costs — Typically expensive and time consuming to develop; Frequent updates to the test content and validation may be needed to keep up with changes in the job; Cost of purchasing an off-the-shelf job knowledge test is typically less expensive than developing a customized test
• Administration Costs — Generally inexpensive and requires few resources for administration
• Utility/ROI — High return on investment if you need applicants who possess technical expertise in specific job knowledge areas; Utility is lower when the job knowledge test contributes little to the prediction of job performance above and beyond inexpensive and readily available cognitive ability tests
• Common Uses — Best used for jobs requiring specific job knowledge on the first day of the job (i.e., where the knowledge is needed upon entry to the position)

References:

(See Section VI for a summary of each article)

Dubois, D., Shalin, V. L., Levi, K. R., & Borman, W. C. (1993). Job knowledge test design: A cognitively-oriented approach. U.S. Office of Naval Research Report, Institute Report 241, i-47.

Dye, D. A., Reck, M., & McDaniel, M. A. (1993). The validity of job knowledge measures. International Journal of Selection and Assessment, 1, 153-157.

Ree, M. J., Carretta, T. R., & Teachout, M. S. (1995). Role of ability and prior job knowledge in complex training performance. Journal of Applied Psychology, 80(6), 721-730.

Roth, P. L., Huffcutt, A. I., & Bobko, P. (2003). Ethnic group differences in measures of job performance: A new meta-analysis. Journal of Applied Psychology, 88(4), 694-706.

Sapitula, L., & Shartzer, M. C. (2001). Predicting the job performance of maintenance workers using a job knowledge test and a mechanical aptitude test. Applied H.R.M. Research, 6(1-2), 71-74.

NOTE: The following Society for Industrial and Organizational Psychology (SIOP) website contains information on Job Knowledge Tests:

http://www.siop.org/workplace/employment%20testing/employment_testing_toc.aspx