test_correct¶
-
test_correct
(check, diagnose, state=None)¶ Allows feedback from a diagnostic SCT, only if a check SCT fails.
A wrapper function around test_or()
, test_correct()
allows you to add logic to your SCT. Normally, your SCT is simply a script with subsequent pythonwhat
function calls, all of which have to pass. test_correct()
allows you to bypass this: you can specify a “sub-SCT” in the check
part, that should pass. If these tests pass, the “sub-SCT” in diagnose
is not executed. If the tests don’t pass, the “sub-SCT” in diagnose
is run, typically to dive deeper into what the error might be and give more specific feedback.
To accomplish this, the SCT in check
is executed silently, so that failure will not cause the SCT to stop and generate a feedback message. If the execution passes, all is good and test_correct()
is abandoned. If it fails, diagnose
is executed, not silently. If the diagnose
part fails, the feedback message that it generates is presented to the student. If it passes, the check
part is executed again, this time not silently, to make sure that a test_correct()
that contains a failing check
part leads to a failing SCT.
Example 1¶
As an example, suppose you want the student to calculate the mean of a Numpy array arr
and store it in res
. A possible solution could be:
*** =solution
```{python}
# Import numpy and create array
import numpy as np
arr = np.array([1, 2, 3, 4, 5, 6])
# Calculate result
result = np.mean(arr)
```
You want the SCT to pass when the student manages to store the correct value in the object result
. How result
was calculated, does not matter to you: as long as result
is correct, the SCT should accept the submission. If something about result
is not correct, you want to dig a little deeper and see if the student used the np.mean()
function correctly. The following SCT will do just that:
*** =sct
```{python}
test_correct(test_object('result'),
test_function('numpy.mean'))
success_msg("You own numpy!")
```
Let’s go over what happens when the student submits different pieces of code:
- The student submits
result = np.mean(arr)
, exactly the same as the solution.test_correct()
runstest_object('result')
. This test passes, sotest_correct()
stops. The SCT passes. - The student submits
result = np.sum(arr) / arr.size
, which also leads to the correct value inresult
.test_correct()
runstest_object('result')
. This test passes, sotest_correct()
stops before runningtest_function()
. The entire SCT passes even thoughnp.mean()
was not used. - The student submits
result = np.mean(arr + 1)
.test_correct()
runstest_object('result')
. This test fails, sotest_correct()
continues with ‘diagnose’, runningtest_function('numpy.mean')
. This function fails, since the argument passed tonumpy.mean()
in the student submission does not correspond to the argument passed in the solution. A meaningful, specific feedback message is presented to the student: you did not correctly specify the arguments insidenp.mean()
. - The student submits
result = np.mean(arr) + 1
.test_correct()
runstest_object('result')
. This test fails, sotest_correct()
continues with’diagnose’, runningtest_function('numpy.mean'). This function passes, because
np.mean()is called in exactly the same way in the student code as in the solution. Because there is something wrong -
resultis not correct - the 'check' SCT,
test_object(‘result’)is executed again, and this time its feedback on failure is presented to the student. The student gets the message that
result` does not contain the correct value.
Multiple functions in diagnose
and check
¶
You can also use test_correct()
with entire ‘sub-SCTs’ that are composed of several SCT calls. In this case, you may put multiple tests inside multi()
, as below..
*** =sct
```{python}
Ex().test_correct(
multi(test_object('a'), test_object('b')), # multiple check SCTs
test_function('numpy.mean')
)
```
Why to use test_correct()
¶
You will find that test_correct()
is an extremely powerful function to allow for different ways of solving the same problem. You can use test_correct()
to check the end result of a calculation. If the end result is correct, you can go ahead and accept the entire exercise. If the end result is incorrect, you can use the diagnose
part of test_correct()
to dig a little deeper.
It is also perfectly possible to use test_correct()
inside another test_correct()
.
Wrapper around test_or()
¶
test_correct()
is a wrapper around test_or()
. test_correct(diagnose, check)
is equivalent with:
def diagnose_and_check()
diagnose()
check()
test_or(diagnose_and_check, check)
Note that in each of the test_or
cases here, the submission has to pass the SCTs specified in check
.