Improve Your Hiring By Asking the Right Questions
Posted on May 21, 2020 by P.K. Maric
Do you have trouble coming up with questions for a job skill test or an interview? Are you sure you are even asking the right questions? In our last post, we explained how to create a good screening test. Now let’s zoom into the question level. Depending on which skill level you’re testing for, your questions should fit somewhere in Bloom’s Taxonomy.
Bloom’s Taxonomy chart
If you are unfamiliar with Bloom’s Taxonomy, here’s a brief rundown using foreign language learning as an example:
Level 1: Remembering
This level is simply about recalling or recognizing pieces of information. You might be able to remember that the French word for dog is chien, but this doesn’t make you proficient in the language.
Level 2: Understanding
Understanding text or audio in a foreign language is a step above merely remembering specific words. But this is still a passive skill that’s relatively easy to learn.
Level 3: Applying
Using language to have a conversation or write an email requires both remembering words and understanding context to apply that knowledge practically.
Level 4: Analyzing
Analyzing requires the ability to break down information into essential parts, often to solve a problem. For example, figuring out the intention or meaning behind a piece of text to translate it accurately.
Level 5: Evaluating
Evaluating requires the ability to analyze something to determine its value, like discussing the literary value of a book. This means you need to be able to judge and criticize information and draw conclusions.
Level 6: Creating
Creating requires all the previous levels of knowledge to make something new, such as writing an article, essay, or fictional story.
Which Level of Knowledge Should You Target?
Don’t ask level 1 or 2 questions. Questions that require remembering or a basic understanding of a topic are trivial, and candidates can easily Google the answers. Even if a candidate remembers and understands some information, that doesn’t tell you much about their proficiency in the subject.
Level 3 (applying) is the minimum level of questions that you should ask. In many cases, you don’t even need to ask questions that are more difficult than this. For example, If you’re hiring a non-native English speaker for a customer support job, you need to make sure that they speak English well enough to communicate with customers. They don’t need to analyze or evaluate literature or write poetry, so testing English proficiency above Level 3 is unnecessary.
You can use Level 4 (analyzing) questions to screen translators, editors, or proofreaders, for example. They need to be able to scrutinize information, pay close attention to detail, and have a deep understanding of the language and the specific context it’s used in. Another example would be asking programmers to fix bugs in a piece of code.
You should avoid Level 5 (evaluating) questions simply because it’s difficult to score the answers to them, and often totally subjective. If someone wrote a long argument about why Firefox is a better web browser than Chrome, would you give them a good or bad score? It would likely depend on your own feelings about each browser, or how strong of an argument the person presented. But this doesn’t necessarily mean that the person is right, just that they are convincing. Someone displaying this level of competence may be very knowledgeable about the subject, but it’s nearly impossible to objectively judge their performance using level 5 questions. This makes evaluating-level questions useless for screening, despite their high level in Bloom’s Taxonomy.
If you are hiring a journalist, content writer, or programmer, then you need to test their ability to create new things, so level 6 questions are appropriate. Unlike level 5 questions, level 6 questions are easy to score. All you need to do is give candidates a work-sample task to see how well they perform. For journalists, this can be writing a short news article. For programmers, it can be creating a simple web application. You can specify criteria to see how each candidate performed the task, and compare them objectively.
No matter what level of question you ask, avoid any questions that are not relevant to the job. It’s well documented that puzzles and brain teasers are useless and have no correlation with job performance.
Personal questions can help you get to know a candidate better, but the place for them is in the interview, not the screening test. A good screening test should focus on skills required for the job, which ideally means questions that correspond to levels 3, 4, and 6 in Bloom’s Taxonomy.
TestDome Custom Question Types
There are several types of custom questions you can create on TestDome:
Live Coding Question (CODE)
This type of question is intended for programming tests, which allows you to create a coding task in various programming languages and add different test cases to validate candidate solutions. You can also add alternate solutions to give candidates more possibilities for solving the problem. This type of question can be used for both levels 4 and 6, for example, fixing bugs (level 4), or implementing a function (level 6).
CODE question example
Multiple Choice Question (MCQ)
Questions that have multiple answer options, where only one answer is correct. The answer options can also be shown in random order if you prefer. Suitable for level 3 or 4 questions.
MCQ question example
Multiple Correct Answers Question (MCA)
Questions that have multiple answer options where more than one answer can be correct. Like with the MCQ questions, the answer options can also be shown in random order. Suitable for level 3 or 4 questions.
MCA question example
Multiple MCQ Question (MMCQ)
A combination of MCQ and MCA questions, where multiple questions are split into different groups. Questions can be of either type, and answers can be shown in random order. Suitable for level 3 or 4 questions.
MMCQ question example
Text Answer Question (TEXT)
Free text input for open-ended questions. You can add an optional answer template to give the candidates an idea of how to construct their answers. This type of question needs to be manually scored. You can use this type of question to make level 3, 4, or 6 questions, depending on your requirements.
TEXT question example
Number Picker Question (NUM)
Questions where the answer is a number. You can set options such as minimum and maximum value, default value, rounding for decimal numbers, etc. This type of question requires only the final answer as an input, so it’s well suited to level 3 or 4 questions.
NUM question example
Fill the Blanks Question (FILL)
Questions where the solution requires typing in missing pieces of a given text. You can set different options such as case sensitive input, or insert a number picker or multiple choice selection instead of text input. This type of question is best suited for level 3 or 4 questions.
FILL question example
File Upload Question (FILE)
Allow candidates to upload a solution or any document, such as a résumé, certificate, or diploma scan. This type of question needs to be manually scored. You can make Level 3, 4, or 6 questions with this type of question.
FILE question example
File Upload with Validator Question (FILEV)
Allow candidates to upload their solution in a file, which will be automatically evaluated. Add test cases to validate answers automatically. You can also add an optional starting template file. This type of question works the same way as the FILE type question, with the additional benefit of automatically validating the solution. Suitable for level 3, 4, or 6 questions.
FILEV question example
MCQ, MCA, and MMCQ type questions are often perceived as relatively easy since possible answers are suggested. However, they can still require applying knowledge or analyzing context and be quite hard, depending on the question and the level of knowledge it requires.
TEXT and FILE type questions need to be scored manually. Be sure to have specific scoring criteria in place so that you can objectively grade and compare candidates.
Conclusion
Asking the appropriate questions is crucial for effective hiring. If you ask questions that are above the level of knowledge and skill needed for the job, you might inadvertently disqualify candidates that would otherwise be good hires. Or you might hire overqualified candidates that will be bored with the work and quit soon. On the other hand, if the questions are too easy, then they won’t represent the difficulties of the job well. Your new hires may struggle to do the required work and probably won’t last long before they quit in frustration, or you realize you need to fire them. Don’t let that happen, ask the right questions.
