Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Check homework - testing lesson 3 #425

Merged
merged 12 commits into from
Feb 4, 2021
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
# Variables and Data Types - HomeWork

## Strings

Before answering the questions:
1. Create a new directory Homework3
2. In the Homework3 directory create a new file for each question
2. In the Homework3 directory create a new file for each question
*Please submit just python files that you ran !!!*

### names.py
Expand All @@ -22,16 +22,16 @@ Create a file *names.py*

Create a file *times.py*

Receive from the user the values for X and T and prints a nice line like:
"You have to spend X*T minutes this week to complete ROSE homework"
X - The number of questions
Receive from the user the values for X and T and prints a nice line like:
"You have to spend X*T minutes this week to complete ROSE homework"
X - The number of questions
T - Time to complete each question

### letter.py

Create a file *letter.py*

Write a python code that accepts date, name, address1, and address2 (as we learned in the last lesson)
Write a python code that accepts date, name, address1, and address2 (as we learned in the last lesson)
The code should print the following output:

<date>
Expand All @@ -57,5 +57,5 @@ Declare the following variable :
Perform the following functions:
1. Use the “title()” function on the string variable and print it.
2. Print the length of the string by len() function.
3. Use the “replace(...,...)” function and replace the first word with the last word.
3. Use the “replace(... , ...)” function and replace the first word with the last word.
4. Use the “find()” function to know where is the ‘w’ word in the string.
Original file line number Diff line number Diff line change
Expand Up @@ -10,10 +10,10 @@ Before answering the questions:

Create a file *variables_1.py*

2. Declare a variable **x** with the value 9
3. Declare a variable **y** with the value 7
4. Declare a variable **z** with the sum of **x** and **y** variables.
5. Print out the contents of the **z** variable.
1. Declare a variable **x** with the value 9
2. Declare a variable **y** with the value 7
3. Declare a variable **z** with the sum of **x** and **y** variables.
4. Print out the contents of the **z** variable.


### variables_2.py
Expand All @@ -38,7 +38,5 @@ Finally, give a short answer to each question in *calculation.txt* file
2. Given (x = 1), what will be its value of after we run (x += 2)?
3. What is the result of float(1)?
4. What is the result of 10 == “10”?
5. Print the result of the following variable:
Number = ((((13 * 8 - 4) * 2 + 50) * 4 ) % 127 ) *5


5. Print the result of the following variable:
Number = ((((13 * 8 - 4) * 2 + 50) * 4 ) % 127 ) *5
Original file line number Diff line number Diff line change
@@ -0,0 +1,97 @@
"""
Automatically check student's homework
----------------------------------------
The tests will check:
1. The content of the exercise file
2. The execution and output of the exercise file
3. The output with different inputs

Guidelines:
- You have to specify the dir Student's exercise files should be placed
in the same dir as this test.
- Test names are in the form: test_<exercise>.py
- Before each test should be the appropriate fixture: @pytest.mark.<topic>
"""
import pytest


@pytest.mark.strings
def test_names(helpers):
helpers.student_file = helpers.home_dir + 'names.py'
AnnaDinaburgVulikh marked this conversation as resolved.
Show resolved Hide resolved
helpers.expected_pycode = [
# Create a string variable with your name, call it my_name
r'''\bmy_name\s*=\s*['"]\w+["']''',
# Create a string variable with your family name,
# call it my_family_name
r'''\bmy_family_name\s*=\s*['"]\w+["']''',
# Create a string variable called my_full_name which is composed from
# the 2 variables my_name and my_family_name.
r'''\bmy_full_name\s*=.*\bmy_name\b.*\bmy_family_name\b''',
# Create a variable with your city name: call it my_city_name
r'''\bmy_city_name\s*=\s*['"]\w+['"]''',
# Create a variable msg with "My name is X and I’m from Y" using
# the variables you created above
r'''\bmsg\s*=.*[f'My name is {]my_(full_)?name''' +
r'''[} and I'm from {my_city_name}]'''
]

helpers.test_assignment()


@pytest.mark.strings
def test_times(helpers):
helpers.student_file = helpers.home_dir + 'times.py'
expected_msg = ('You have to spend {} minutes this week to ' +
'complete ROSE homework')
helpers.input = [['2', '3'], ['11', '6'], ['10', '2']]
helpers.expected_stdout = [
[expected_msg.format(6)],
[expected_msg.format(66)],
[expected_msg.format(20)],
]

helpers.test_assignment()


@pytest.mark.strings
def test_letter(helpers):
helpers.student_file = helpers.home_dir + 'letter.py'
helpers.expected_pycode = [
r'''\b(print\(f'{date}\\n\\tFor\\n\\t{name}).*''',
]
expected_msg = ('{}\n\tFor\n\t{}\n\t{}\n' +
'\nDear Mr./Mrs. {}\n' +
'Please visit our office as soon as possible to arrange ' +
'your payments.\n' +
'We can\'t wait until it’s all done ...\n\n' +
'Sincerely\n\tKoogle Inc.\n\t{}')
helpers.input = [
['3.3.20', 'Anna', 'Rasbery, US', 'Hotel california, US'],
['5.4.21', 'Yael', 'Herzel 5, Haifa', 'Yam 1, Tel Aviv']
]
helpers.expected_stdout = [
[r'.*' + expected_msg.format(helpers.input[0][0],
helpers.input[0][1],
helpers.input[0][2],
helpers.input[0][1],
helpers.input[0][3])],
[r'.*' + expected_msg.format(helpers.input[1][0],
helpers.input[1][1],
helpers.input[1][2],
helpers.input[1][1],
helpers.input[1][3])]
]

helpers.test_assignment()


@pytest.mark.strings
def test_manipulations(helpers):
helpers.student_file = helpers.home_dir + 'manipulations.py'
helpers.expected_pycode = [
# declare the varible s: "...very long line..."
r'''\b[s]\s*=\s*[\'\'\'].*''',
r'''.*[\'\'\']'''
]

helpers.test_assignment()
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
"""
Automatically check student's homework
----------------------------------------
The tests will check:
1. The content of the exercise file
2. The execution and output of the exercise file
3. The output with different inputs

Guidelines:
- You have to specify the dir Student's exercise files should be placed
in the same dir as this test.
- Test names are in the form: test_<exercise>.py
- Before each test should be the appropriate fixture: @pytest.mark.<topic>
"""
import pytest


@pytest.mark.variables
def test_variables_1(helpers):
helpers.student_file = helpers.home_dir + 'variables_1.py'
helpers.expected_pycode = [
r'^.*x *= *9.*',
r'^.*y *= *7.*',
r'^.*z *= *x *\+ *y.*',
r'^.*print\(.*z.*\).*',
]
helpers.expected_stdout = [r'.*16']

helpers.test_assignment()


@pytest.mark.variables
def test_variables_2(helpers):
helpers.student_file = helpers.home_dir + 'variables_2.py'
helpers.expected_pycode = [
r'^.*\*.*',
r'^.*-.*',
r'^.*/.*',
r'^.*%.*',
]
helpers.input = [['10', '3'], ['5', '6']]
helpers.expected_stdout = [[r'.*30', r'.*7', r'.*' + str(10/3), r'.*1'],
[r'.*30', r'.*-1', r'.*' + str(5/6), r'.*5']]

helpers.test_assignment()


@pytest.mark.variables
def test_calculations(helpers):
helpers.student_file = helpers.home_dir + 'calculations.txt'
helpers.expected_pycode = [
# What is the result of 10 ** 3?
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

did you change something here? after the commit I'm getting failures :
[ WARNING]
For 10 ** 3 the expected result is 1000
for x=1; x+=2 the expected result is 3
For float(1) the expected result is 1.0
For Number = ((((13 * 8 - 4) * 2 + 50) * 4 ) % 127 )*5
the expected result is 555
FAILED

for all the students work

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I changed the regex, it worked for the plain answers as '555'. Can you give an example for the answers that fail?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @shiramax, sorry for earlier. Fixed the regex for the calculations exercise.

'1000',
# Given (x = 1), what will be the value of after we run (x += 2)?
'3',
# What is the result of float(1)?
r'1\.0',
# What is the result of 10 == “10”?
'False',
# Print the result of the following variable:
# Number = ((((13 * 8 - 4) * 2 + 50) * 4 ) % 127 ) *5
'555',
]

helpers.test_assignment()
62 changes: 56 additions & 6 deletions docs/course_materials/exercises/test_exercises/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,20 +6,28 @@ First, make sure you are using python 3.7 or later.

`pipenv --python /usr/local/bin/python3.7 shell`

For checking an exercise, you should use the following:
For checking a linux exercise, you should use the following:

`rose_check lesson/check_homework_1`
`rose_check <lesson>/<check_homework_#>`

If you need to run on a directory different from home use:
For checking other exercises, you should use the following:

`rose-check lesson/check_homework_1 -s your_home_dir`
`rose_check -t <mark_of_exercise>`

For the list of all available check files use:
If you need to run on a directory different from HOME use:

`rose-check <lesson>/<check_homework_#> -s <your_home_dir>`

or

`rose_check -t <mark_of_exercise> -s <your_home_dir>`
AnnaDinaburgVulikh marked this conversation as resolved.
Show resolved Hide resolved

For the list of all available check files use:

`rose_check --help`
AnnaDinaburgVulikh marked this conversation as resolved.
Show resolved Hide resolved


## Adding check_exercise files:
## Adding check_exercise files(linux lesson):
AnnaDinaburgVulikh marked this conversation as resolved.
Show resolved Hide resolved

To add a check file use the convention `check_` followed by `homework` or `class_exercise`, and exercise number.

Expand All @@ -39,3 +47,45 @@ The check_exercise file uses 3 lists for tests:
Should include all the paths we expect to be absent, since we want the student to delete or change them during the exercise.
All paths are relative to home directory.
For example: `hw1/other` or `hw1/file.txt`.

## Adding test files(not linux lesson):

To add a test file:
1. You should name it according to the `exercise/homework` theme.
2. For every test function, add an appropriate decorator (aka `mark` in pytest). Use the same `mark('@pytest.mark.<topic>')` for tests you would like to run together. For example: `@pytest.mark.variables` for variables homework check.
3. In order for `pytest` to recognize the new marks, you need to add them to the `pytest.ini` file.
4. Each function in the test file should be named: `test_exercise_#` and should invoke `helpers` as input. For example: `def test_variables_1(helpers)`
5. The `helpers` is defined by `pytest fixture` and creates a `Helpers class` that stores our check values and the functions for the testing.

## Variables in the files:

The tests uses 3 lists for tests:
helpers.student_file = helpers.home_dir + 'variables_1.py'

helpers.expected_stdout
1. **helpers.student_file**:
Stores the path for student file for testing.
Includes the HOME directory or the directory we supplied and the name of the student file according to the homework instructions.

2. **helpers.expected_pycode**:
Should include code/text we expect in the student answer file. All values shold be supplied as string or in regex format.
For example: `helpers.expected_pycode = [r'^.*x *= *9.*', r'^.*print\(.*z.*\).*']` for checking code.

3. **helpers.input**
Should include all the inputs we want to check.
For example: `helpers.input = [['10', '3'], ['5', '6']]`, will check 2 pairs of input.

4. **helpers.expected_stdout**:
Should include the output we expect to get.
For example: `helpers.expected_stdout = [[r'.*30', r'.*7', r'.*' + str(10/3), r'.*1']]` for 4 printouts tht can be prefixed with text.

## Under the hood (using pytest):

- rose_check.py calls pytest when `-t <mark>` is invoked
- pytest.ini sets the parameters for the pytest run:
- addopts: runs pytest using cmdline arguments
- --tb=no to hide long trace output
- -v for verbose, shows PASSED/FAILED
- --no-header, --no-summary - hides those parts of outputs
- markers - all our marks(decorators) should be registered here for pytest to recognize them.
- log_cle - specify which log to show on live log. Use `LOGGER.error(message)` to show output to the user (also available `LOGGER.info/warning`).
Loading