All Collections
Technical Recruiters and Hiring Managers
Why did my candidate fail their test?
Why did my candidate fail their test?

Filtered uses test cases to score each challenge. This article explains how they work.

Oliver Weng avatar
Written by Oliver Weng
Updated over a week ago

TL;DR

If a candidate failed a test (or, perhaps, scored a 0%), it means that the candidate's code/analysis did not provide the result that was expected. In layman's terms: they got the wrong answer, or didn't answer at all. 

Filtered questions have many test cases, so even if a candidate didn't provide the perfectly correct solution, they can still score higher than an 0% if other parts of their code/analysis is correct.
​

How Filtered applies a score using test cases

Test cases help in validating candidates' code. Test cases form the basis for automated evaluation of a candidate’s code. A test case consists of an input to the code and an expected output. Once candidates submit the code, it is run against all the test cases. The output from the candidate’s code is compared with the expected output to see whether the test case has passed or failed.

A sample test case with an explanation can also aid in explaining the problem better to the candidates. Depending on the question, there can be up to 15 test cases. The test cases are of different sizes that cover different levels of complexity.

Types of test cases

There are two types of test cases for each problem.

  1. Sample test cases: Sample test cases are the test cases that the candidates can view and use it to validate their code. They see both input and expected output values for these test cases.

  2. Hidden test cases: All other test cases are hidden from the candidates. Candidates cannot see the input and expected output values for these test cases. They cannot see these test cases at any point.

The hidden test cases often include corner cases to ensure that the code submitted by the candidates also account for these. The hidden test cases also ensure that candidates write a logical code to solve the problem and not focus on writing a code that generates the expected output, if all test cases are exposed.

For example:

  • A test case where only a single space is present. 

  • A test case where only special characters are present in the string

Scoring

Not all test cases have the same difficulty level. More points are assigned to the difficult test cases as compared to the easy ones. This leads to a better distinction between outstanding, good, and average programmers. The sum of scores of all test cases is the total score assigned to a particular question. The overall score for a coding question is the sum of the scores of all the test cases which are successfully passed. 

Did this answer your question?