- Introduction
- Learning
- System Commands
- Using Pyenv and Pipenv
- Expected Output
- Automated Checks with GatorGrader
- Updates
- Travis
- System Requirements
- Reporting Problems
- Receiving Assistance
- Project Assessment
- Project Feedback
Designed for use with GitHub Classroom and GatorGrader, this repository contains the solution for a laboratory assignment in a software engineering course class that uses the Python programming language. The Travis CI builds for this repository will not pass, as evidenced by a red ✗ instead of a green ✔ appearing in the commit logs. An instructor would use this repository to create a "starter" repository with purposefully omitted features that a student would then need to add in order to achieve the stated learning objectives. Please bear in mind that much of the content in this document is written in its current form under the assumption that it will also accompany the "starter" repository that an instructor shares with a student through the use of GitHub Classroom.
This assignment requires a programmer to use three different types of test cases
to find and fix a defect in a Python program called program called
termfrequency/compute_tf_pipeline.py
. Please refer to the Preface and Chapters
1 through 5 in "Exercises in Programming Style" to learn more about this
program's input, output, and behavior. You can also review Chapters 1 through 3
in "Think Python" to learn more about how to program in Python and run Python
scripts. You also will refer to the first two chapters in the "Pytest" book. In
addition to the work that you did for the previous practical assignment —
exploring different command-line options for Pytest, installing the pytest-cov
code coverage calculation package, perform code coverage analysis, and
incrementally increasing the coverage of your test suite — you will gain
experience with refactoring Python source code so that it meets well-established
standards for source code formatting.
When you use the git commit
command to transfer your source code to your
GitHub repository, Travis CI will initialize a build
of your assignment, checking to see if it meets all of the requirements. If both
your source code and writing meet all of the established requirements, then you
will see a green ✔ in the listing of commits in GitHub. If your
submission does not meet the requirements, a red ✗ will appear instead.
The instructor will reduce a programmer's grade for this assignment if the red
✗ appears on the last commit in GitHub immediately before the
assignment's due date.
If you have not done so already, please read all of the relevant GitHub Guides that explain how to use many of the features that GitHub provides. In particular, please make sure that you have read the following GitHub guides: Mastering Markdown, Hello World, and Documenting Your Projects on GitHub. Each of these guides will help you to understand how to use both GitHub and GitHub Classroom.
To do well on this assignment, you should also read Chapters 1 and 2 in "Think Python", paying particularly close attention to the content about variables, expressions, and statements. You should also read the Preface and Chapters 1 through 5 in the "Exercises in Programming Style" book. Finally, you should read online articles about coverage testing with Pytest, Coverage.py, and Hypothesis for Python. Please see the course instructor or one of the student technical leaders or tutors if you have questions about any of these reading assignments.
This project invites students to enter system commands into a terminal window.
This assignment uses Docker to deliver programs, such
as gradle
and the source code and packages needed to run
GatorGrader, to a students'
computer, thereby eliminating the need for a programmer to install them on their
development workstation. Along with using Docker for automated grading and
assessment on your laptop, students are also asked to setup a full-fledged
Python development environment that leverages
Pyenv to download and manage versions of
Python and Pipenv to install and manage Python
packages.
Once you have installed Docker
Desktop, with MacOS and Linux
you can use the following docker run
command to start gradle grade
as a
containerized application, using the
DockaGator Docker image available
on
DockerHub.
docker run --rm --name dockagator \
-v "$(pwd)":/project \
-v "$HOME/.dockagator":/root/.local/share \
gatoreducator/dockagator
The aforementioned command will use "$(pwd)"
(i.e., the current working
directory) as the project directory and "$HOME/.dockagator"
as the cached
GatorGrader directory. Please note that both of these directories must exist,
although only the project directory must contain something. Generally, the
project directory should contain the source code and technical writing for this
assignment, as provided to a student by the instructor through GitHub.
Additionally, the cache directory should not contain anything other than
directories and programs created by DockaGator, thus ensuring that they are not
otherwise overwritten during the completion of the assignment. To ensure that
the previous command will work correctly, you should create the cache directory
by running the command mkdir $HOME/.dockagator
on the MacOS and Linux
operating systems. However, if you are using the Windows operating system then
you will instead need to type the command mkdir %HomeDrive%%HomePath%/.dockagator
. Finally, if the above docker run
command
does not work correctly on the Windows operating system, you may need to instead
run the following command to adapt to the differences in the cmd
terminal
window:
docker run --rm --name dockagator \
-v "%cd%:/project" \
-v "%HomeDrive%%HomePath%/.dockagator:/root/.local/share" \
gatoreducator/dockagator
Here are some additional commands that you may need to run when using Docker:
docker info
: display information about how Docker runs on your workstationdocker images
: show the Docker images installed on your workstationdocker container list
: list the active images running on your workstationdocker system prune
: remove many types of "dangling" components from your workstationdocker image prune
: remove all "dangling" docker images from your workstationdocker container prune
: remove all stopped docker containers from your workstationdocker rmi $(docker images -q) --force
: remove all docker images from your workstation
Since the above docker run
command uses a Docker images that, by default, runs
gradle grade
and then exits the Docker container, you may want to instead run
the following command so that you enter an "interactive terminal" that will
allow you to repeatedly run commands within the Docker container. Don't forget
that, if you are using the Windows operating system, then you will need to use a
different command to run Docker, as explained previously in this document.
docker run -it --rm --name dockagator \
-v "$(pwd)":/project \
-v "$HOME/.dockagator":/root/.local/share \
gatoreducator/dockagator /bin/bash
Once you have typed this command, you can use the GatorGrader
tool in the Docker container by
typing the command gradle grade
in your terminal. Running this command will
produce a lot of output that you should carefully inspect. If GatorGrader's
output shows that there are no mistakes in the assignment, then your source code
and writing are passing all of the automated baseline checks. However, if the
output indicates that there are mistakes, then you will need to understand what
they are and then try to fix them.
You can also complete several important Java programming tasks by using the
gradle
tool. For instance, you can compile (i.e., create bytecode from the
program's source code if it is correct) the program using the command gradle build
. Here are some other commands that you can type:
gradle grade
: run the GatorGrader tool to check your workgradle tasks
: display details about the Gradle system
To run one of these commands, you must be in the main (i.e., "home base")
directory for this assignment where the build.gradle
file is located.
Assuming that you will use Pyenv to download and manage your installation of Python, this practical assignment also invites you to use Pipenv to create a virtual environment, install and manage development packages, and to run Python commands. Here is a sample of the Pipenv commands that you will need to run during this assignment.
- Install and upgrade the
pipenv
command:pip install pipenv --user
- Install the development dependencies:
pipenv
command:pipenv install --dev
- Run the linters and the formatter to check your Python source code:
pipenv run lint --check
- Run the test suite to check your Python source code:
pipenv run test
- Run the test suite and calculate test suite coverage:
pipenv run cover
- Run the program with
pipenv
andpython
and a small input:pipenv run python termfrequency/compute_tf_pipeline.py inputs/input.txt
- Run the program with
pipenv
andpython
and a realistic input:pipenv run python termfrequency/compute_tf_pipeline.py inputs/pride-and-prejudice.txt
To run one of these commands, you must be in the main directory for this
assignment where the configuration files are located. Then, you can type these
commands in the terminal and study the output. Please note that the commands
that run a script in the scripts/
directory (e.g., pipenv run test
) will
only work if you are inside of your Docker container or running the commands on
a Linux workstation. With that said, you can run the majority of the commands in
these scripts on Windows, MacOS, and Linux.
Since the provided test suite also uses Pytest's markers, then if you want to exclude the test cases that use Hypothesis you can type:
pipenv run pytest -m "not hypothesisworks"
Alternatively, if you want to include the test cases that use Hypothesis you can type:
pipenv run pytest -m "hypothesisworks"
Running the program with the small input should produce the following output:
live - 2
mostly - 2
white - 1
tigers - 1
india - 1
wild - 1
lions - 1
africa - 1
Running the program with the realistic input should produce the following output:
mr - 786
elizabeth - 635
very - 488
darcy - 418
such - 395
mrs - 343
much - 329
more - 327
bennet - 323
bingley - 306
jane - 295
miss - 283
one - 275
know - 239
before - 229
herself - 227
though - 226
well - 224
never - 220
sister - 218
soon - 216
think - 211
now - 209
time - 203
good - 201
Running the Pytest test suite that calls the compute_tf_pipeline.py
and
checks its output should produce the following output:
filtered-normalized /",("" to make the string:
Trying example: test_filter_normalize_correctly_automated_unbounded(
input_string='\U0008cfaa/\U000f985b\x07",\x1d(\uf522""\U00037d63\U000e32cd\U000a8050\U000ac6ab\x1d\U000f8031',
)
filtered-normalized /",("" to make the string:
Trying example: test_filter_normalize_correctly_automated_unbounded(
input_string='𘈱',
)
filtered-normalized 𘈱 to make the string: 𘈱
Trying example: test_filter_normalize_correctly_automated_unbounded(
input_string='𘈱/',
)
filtered-normalized 𘈱/ to make the string: 𘈱
tests/test_compute_tf_pipeline.py ✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓ 100% ██████████
Results (1.36s):
21 passed
Please note that this is only an excerpt of the output from running the test suite with the inputs that were automatically generated by Hypothesis. It is also important to point out that the execution of the test suite will produce different output each time it is run. Finally, please be aware of the fact that Hypothesis could create Unicode strings that corrupt the output buffer in your terminal window. The easiest way to overcome that problem is to create a new terminal window and then run the test suite again.
In addition to meeting all of the requirements outlined in the assignment sheet, your submission must pass the following checks that GatorGrader automatically assesses:
- The compute_tf_pipeline.py in termfrequency has at least 1 of the
for w in word_list
fragment - The compute_tf_pipeline.py in termfrequency has at least 1 of the
\\W
fragment - The compute_tf_pipeline.py in termfrequency has at least 1 of the
with open
fragment - The compute_tf_pipeline.py in termfrequency has at least 6 single-line Python comment(s)
- The compute_tf_pipeline.py in termfrequency has at least 7 multiple-line Python comment(s)
- The compute_tf_pipeline.py in termfrequency has exactly 0 of the
TODO
fragment - The conftest.py in tests has exactly 0 of the
TODO
fragment - The cover.sh in scripts has exactly 0 of the
TODO
fragment - The file compute_tf_pipeline.py exists in the termfrequency directory
- The file conftest.py exists in the tests directory
- The file cover.sh exists in the scripts directory
- The file init.py exists in the tests directory
- The file lint.sh exists in the scripts directory!
- The file reflection.md exists in the writing directory
- The file test_compute_tf_pipeline.py exists in the tests directory
- The file test.sh exists in the scripts directory
- The init.py in tests has exactly 0 of the
TODO
fragment - The lint.sh in scripts has exactly 0 of the
TODO
fragment - The reflection.md in writing has at least 600 word(s) in total
- The reflection.md in writing has exactly 0 of the
Add Your Name Here
fragment - The reflection.md in writing has exactly 0 of the
TODO
fragment - The reflection.md in writing has exactly 10 of the
heading
tag - The reflection.md in writing has exactly 2 of the
code_block
tag - The repository has at least 10 commit(s)
- The test_compute_tf_pipeline.py in tests has at least 0 single-line Python comment(s)
- The test_compute_tf_pipeline.py in tests has at least 2 of the
@pytest.mark.parametrize
fragment - The test_compute_tf_pipeline.py in tests has at least 4 of the
@pytest.mark.hypothesisworks
fragment - The test_compute_tf_pipeline.py in tests has at least 5 multiple-line Python comment(s)
- The test_compute_tf_pipeline.py in tests has at least 5 of the
assert
fragment - The test_compute_tf_pipeline.py in tests has at least 5 of the
test_
fragment - The test_compute_tf_pipeline.py in tests has exactly 0 of the
TODO
fragment - The test.sh in scripts has exactly 0 of the
TODO
fragment
If GatorGrader's automated
checks pass correctly, the tool will produce the output like the following in
addition to returning a zero exit code (which you can access by typing the
command echo $?
).
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Passed 32/32 (100%) of checks for cmpsc-203-spring-2020-practical6! ┃
┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┛
If the course instructor updates the provided material for this assignment and you would like to receive these updates, then you can type this command in the main directory for this assignment:
git remote add download git@github.com:Allegheny-Computer-Science-203-S2020/cs203-S2020-practical6-starter.git
You should only need to type this command once; typing the command additional times may yield an error message but will not negatively influence the state of your repository. Now, you are ready to download the updates provided by the course instructor by typing:
git pull download master
This second command can be run whenever the course instructor needs to provide you with new source code for this assignment. However, please note that, if you have edited the files that the course instructor updated, running the previous command may lead to Git merge conflicts. If this happens, you may need to manually resolve them with the help of the instructor or a teaching assistant.
This assignment uses Travis CI to automatically run the checking programs every time you commit to your GitHub repository. The checking will start as soon as you have accepted the assignment, thus creating your own private repository, and the course instructor enables Travis for it. If you are using Travis for the first time, you will need to authorize Travis CI to access the private repositories that you created on GitHub.
We developed this assignment to work with the following software and versions:
- Docker Desktop
- Operating Systems
- Linux
- MacOS
- Windows 10 Pro
- Windows 10 Enterprise
- Programming Language Tools
- Gradle 5.4
- MDL 0.5.0
- OpenJDK 11.0.4
- Proselint 0.10.2
- Python 3.6 or 3.7
If you have found a problem with this assignment's provided source code, then you can go to the Computer Science 203 Practical 5 Starter repository and create an issue by clicking the "Issues" tab and then clicking the green "New Issue" button. If you have found a problem with the GatorGrader tool and the way that it checks you assignment, then you can follow the aforementioned steps to create an issue in its repository. To ensure that your issue is properly resolved, please provide as many details as is possible about the problem that you experienced. If you discover a problem with the laboratory assignment sheet, then please raise an issue in the cs203-S2020-sheets repository and mention this assignment.
Students who find — and use the appropriate GitHub issue tracker to correctly document — a mistake in any aspect of this laboratory assignment will receive free GitHub stickers and extra credit towards their grade for it.
If you are having trouble completing any part of this project, then please talk with either the course instructor or a student technical during the practical session. Alternatively, you may ask questions in the Slack workspace for this course. Finally, you can schedule a meeting during the course instructor's office hours.
Taking inspiration from the principles of specification-based grading, the grade that a student receives on this assignment will have the following components:
-
Percentage of Correct GatorGrader Checks: Students are encouraged to repeatedly try to implement a Java program that passes all of GatorGrader's checks by, for instance, creating a program that produces the correct output. Students should also repeatedly revise their technical writing to ensure that it also passes all of GatorGrader's checks about, for instance, the length of its content and its appropriate use of Markdown.
-
Travis CI Build Status: Since additional checks on the source code and/or technical writing may be encoded in Travis CI's actions and, moreover, all of the GatorGrader checks are also run in Travis CI, students will receive a checkmark grade if their last before-the-deadline build passes and a green ✔ appears in their GitHub commit log instead of a red ✗. As with the previous grading component, students are encouraged to repeatedly revise their source code and technical writing in an attempt to get their Travis CI build to pass.
-
Mastery of Technical Writing: Students will also receive a checkmark grade when the responses to the technical writing questions presented in the
writing/reflection.md
reveal a mastery of both writing skills and technical knowledge. To receive a checkmark grade, the submitted writing should have correct spelling, grammar, and punctuation in addition to following the rules of Markdown and providing technically accurate answers. Students are encouraged to ask the course instructor or a student technical leader to use the GitHub issue tracker to provide feedback on their mastery of technical writing skills. -
Mastery of Technical Knowledge and Skills: Students will receive also receive a checkmark grade when their GitHub repository reveals that they have mastered all the technical knowledge and skills developed during the completion of this project. As a part of this grade, the instructor will assess aspects of the project including, but not limited to, the inclusion of effective source code comments, creation of Git commit messages, correct use of Docker, and correct installation and use of a full-fledge Python development environment. Students are encouraged to ask the course instructor or a student technical leader to use the GitHub issue tracker to provide feedback on how well their work demonstrates mastery of the assignment's technical knowledge and skills.
All grades for this project will be reported through a student's GitHub repository using either messages in the GitHub commit log or issues raised in the issue tracker. Students should ask questions about their grade for this project in GitHub so as to facilitate an effective conversation about the submitted deliverables.
Students who wish to receive feedback on their work for this practical assignment should first open an issue on the issue tracker for this GitHub repository, giving an appropriate title and description for the type of feedback that you would like the course instructor to provide. After creating this issue, you will see that GitHub has created a unique web site that references this issue. To alert the course instructor to the fact that the issue was created and that you want feedback on your work, please send it to him by a Slack direct message at least 24 hours in advance of the project's due date. After the instructor responds to the issue, please resolve all of the stated concerns and participate in the discussion until the issue is closed.