News
Another flaw in assignment "Metamorphic Testing"
Written on 10.07.2023 09:30 by Dominic Steinhöfel
Dear seminar participants,
Stanimir discovered another problem in the current assignment. In the function "test_tiny_c_interpreter_with_wrong_dict," the "wrong_dict" was initialized only once. However, every call to "interpret_tiny_c" changes the dictionary, resulting in the interpreter being called with a potentially different dictionary every time. A correct implementation of the function looks like this:
def test_tiny_c_interpreter_with_wrong_dict():
"""
This function tests the Tiny-C interpreter with the metamorphic
oracle provided by `get_transformation_pair`. It completes normally
in the case of success and raises an `AssertionError` otherwise.
This version of `test_tiny_c_interpreter_with_wrong_dict` passes
a non-trivial initial dictionary to the interpreter to test the
error-finding capabilities of the metamorphic tester.
:return: Nothing.
"""
wrong_dict = tuple({var: 1 for var in string.ascii_lowercase}.items())
for _ in range(20):
generator = generate_transformation_pair(
DEAD_CODE_INSERTION_GRAMMAR_2, TINY_C_GRAMMAR, constraint
)
for pair in itertools.islice(generator, 5):
left = str(pair.filter(lambda node: node.value == "<left>")[0][1])
right = str(pair.filter(lambda node: node.value == "<right>")[0][1])
result_left = interpret_tiny_c(left, dict(wrong_dict), gas=20)
result_right = interpret_tiny_c(right, dict(wrong_dict), gas=20)
assert result_left == result_right
Mutable data structures are a constant source of trouble in programming. For this reason, I mostly work with immutable types in my research prototypes. Unfortunately, I was not that disciplined when implementing the current assignment.
This could be a nice lesson to be learned. However, you were affected by this problem and might have lost time when trying to work around it. For this, I am very sorry. We will consider this context when grading this part of the assignment.
All the best,
Dominic