HW4 ― Videos

In this assignment, you will learn to perform both trimmed action recognition and temporal action segmentation in full-length videos.

For more details, please click this link to view the slides of HW4.

Usage

To start working on this assignment, you should clone this repository into your local machine by using the following command.

git clone https://github.com/dlcv-spring-2019/hw4-<username>.git

Note that you should replace <username> with your own GitHub username.

Dataset

In the starter code of this repository, we have provided a shell script for downloading and extracting the dataset for this assignment. For Linux users, simply use the following command.

bash ./get_dataset.sh

The shell script will automatically download the dataset and store the data in a folder called hw4_data. Note that this command by default only works on Linux. If you are using other operating systems, you should download the dataset from this link and unzip the compressed file manually.

⚠️ IMPORTANT NOTE ⚠️
You should keep a copy of the dataset only in your local machine. DO NOT upload the dataset to this remote repository. If you extract the dataset manually, be sure to put them in a folder called hw4_data under the root directory of your local repository so that it will be included in the default .gitignore file.

For this dataset, the action labels are defined as below:

Action	Label
Other	0
Inspect/Read	1
Open	2
Take	3
Cut	4
Put	5
Close	6
Move Around	7
Divide/Pull Apart	8
Pour	9
Transfer	10

Utility

We have also provided a Python script for reading video files and retrieving labeled videos as a dictionary. For more information, please read the comments in reader.py.

Submission Rules

Deadline

108/06/05 (Wed.) 01:00 AM

Late Submission Policy

You have a five-day delay quota for the whole semester. Once you have exceeded your quota, the credit of any late submission will be deducted by 30% each day.

Note that while it is possible to continue your work in this repository after the deadline, we will by default grade your last commit before the deadline specified above. If you wish to use your quota or submit an earlier version of your repository, please contact the TAs and let them know which commit to grade. For more information, please check out this post.

Academic Honesty

Taking any unfair advantages over other class members (or letting anyone do so) is strictly prohibited. Violating university policy would result in an F grade for this course (NOT negotiable).
If you refer to some parts of the public code, you are required to specify the references in your report (e.g. URL to GitHub repositories).
You are encouraged to discuss homework assignments with your fellow class members, but you must complete the assignment by yourself. TAs will compare the similarity of everyone’s submission. Any form of cheating or plagiarism will not be tolerated and will also result in an F grade for students with such misconduct.

Submission Format

Aside from your own Python scripts and model files, you should make sure that your submission includes at least the following files in the root directory of this repository:

hw4_<StudentID>.pdf
The report of your homework assignment. Refer to the "Grading" section in the slides for what you should include in the report. Note that you should replace <StudentID> with your student ID, NOT your GitHub username.
hw4_p1.sh
The shell script file for data preprocessing. This script takes as input two folders: the first one contains the video data, and the second one is where you should output the label file named p1_valid.txt.
hw4_p2.sh
The shell script file for trimmed action recognition. This script takes as input two folders: the first one contains the video data, and the second one is where you should output the label file named p2_result.txt.
hw4_p3.sh
The shell script file for temporal action segmentation. This script takes as input two folders: the first one contains the video data, and the second one is where you should output the label files named <video_category>.txt. Note that you should replace <video_category> accordingly, and a total of 7 files should be generated in this script.

We will run your code in the following manner:

Problem 1

bash ./hw4_p1.sh $1 $2 $3

$1 is the folder containing the trimmed validation videos (e.g. TrimmedVideos/video/valid/).
$2 is the path to the ground truth label file for the videos (e.g. TrimmedVideos/label/gt_valid.csv).
$3 is the folder to which you should output your predicted labels (e.g. ./output/).

Problem 2

bash ./hw4_p2.sh $1 $2 $3

$1 is the folder containing the trimmed validation/test videos.
$2 is the path to the ground truth label file for the videos (e.g. TrimmedVideos/label/gt_valid.csv or TrimmedVideos/label/gt_test.csv).
$3 is the folder to which you should output your predicted labels (e.g. ./output/).

Problem 3

bash ./hw4_p3.sh $1 $2

$1 is the folder containing the full-length validation videos.
$2 is the folder to which you should output your predicted labels (e.g. ./output/).

NOTE
For the sake of conformity, please use the python3 command to call your .py files in all your shell scripts. Do not use python or other aliases, otherwise your commands may fail in our autograding scripts.

Packages

Below is a list of packages you are allowed to import in this assignment:

python: 3.5+
tensorflow: 1.13
keras: 2.2+
torch: 1.0
h5py: 2.9.0
numpy: 1.16.2
pandas: 0.24.0
torchvision: 0.2.2
cv2, matplotlib, skimage, Pillow, scipy, imageio
scikit-video: 1.1.11
The Python Standard Library

Note that using packages with different versions will very likely lead to compatibility issues, so make sure that you install the correct version if one is specified above. E-mail or ask the TAs first if you want to import other packages.

Remarks

If your model is larger than GitHub’s maximum capacity (100MB), you can upload your model to another cloud service (e.g. Dropbox). However, your shell script files should be able to download the model automatically. For a tutorial on how to do this using Dropbox, please click this link.
DO NOT hard code any path in your file or script, and the execution time of your testing code should not exceed an allowed maximum of 10 minutes.
If we fail to run your code due to not following the submission rules, you will receive 0 credit for this assignment.

Q&A

If you have any problems related to HW4, you may

Use TA hours (please check course website for time/location)
Contact TAs by e-mail (ntudlcvta2019@gmail.com)
Post your question in the comment section of this post

lknight8631/hw4-ntudlcvta2019