Incorrect cropping for some videos
Closed this issue · 9 comments
Hi! Thank you for the dataset, it's really cool! While downloading, I found 2 issues with cropping:
- Some clips do not contain any cropping information.
- Some videos contain cropping for a 720p version of a video when a 1080p version is available on Youtube. And there is no way to decide, which version should I download, because cropping for other videos is designed for the 1080p version.
For the first issue, here is the list of videos that do not contain any cropping in their corresponding *_crop_wh.txt files:
- RD/Radio6_0.mp4
- RD/Radio15_0.mp4
- WDA/DeanPhillips_0.mp4
- WDA/JimMcGovern1_0.mp4
- WDA/JimMcGovern_0.mp4
- WDA/JeffMerkley0_0.mp4
- WDA/LindaSnchez_0.mp4
- WDA/ByronDorgan0_0.mp4
- WRA/BillCassidy_0.mp4
- WRA/MittRomney_1.mp4
- WRA/RudyGiuliani_0.mp4
- WRA/JaimeHerreraBeutler_0.mp4
- WRA/DuncanHunter_1.mp4
- WRA/KevinBrady0_0.mp4
- WRA/CandiceMiller_0.mp4
For the second issue, here is an example of a video (but there are others!), which I downloaded in 1080p (best available quality), cropped it and the resulted crop turned out to be inconsistent with such a high resolution (but was appropriate for 720p).
- RD/Radio28
Here is the comparison between cropping a 1080p resolution (left) or 720p (right) version of the video:
I download the video the following way:
youtube-dl https://youtube.com/watch?v=VrFpdKk1UJc --quiet -f bestvideo[ext=mp4][height=1080] --output /tmp/hdtf_check/_videos_raw/RD_Radio28.mp4 --no-continue
And then I cut and crop it the following way:
ffmpeg -i /tmp/hdtf_check/_videos_raw/RD_Radio28.mp4 -strict -2 -loglevel quiet -qscale 0 -y -ss 00:05 -to 01:35 -filter:v "crop=530:530:372:0" /tmp/hdtf_check/RD_Radio28_000.mp4
So right now, it is impossible to infer in which resolution (720p or 1080p) should I download each particular video. If I download each video in its best possible resolution and crop, then a lot of videos will have incorrect cropping:
Could you please provide missing crops and also the necessary resolution for each video?
Hi, great thanks for pointing out this problem!
1.For the first issue, your list videos contains undesired factors, including hand movements(01:07 /WDA/ByronDorgan0_0.mp4), dynamic background(/WDA/MikeDoyle_0.mp4), large camera motion (/WDA/LindaSnchez_0.mp4) and so on. In training stage, our method discard these samples, so their cropping information is missing. We update the xx_crop_wh.txt with some missing samples.
- For the second issue, We utilize you-get tool to download the videos, but it seems that the downloaded videos are not always the best resolution, so we provide two solutions.
-1. We count the resolution information of videos and upload in xx_resolution.txt . The resolution is paired with xx_crop_wh.txt, but the videos may not be the best resolution.
-2. We provide the cropping method in readme and the zoom scale of the cropped window in xx_crop_ratio.txt, with this you are available to crop face in the best resolution videos.
Hi, I'm so sorry that the information of missing videos in issue 1 is discarded and I can not find these information on my computer. You can crop the face with method 2 in issue 2 if necessary.
Hi! Thank you for providing video resolutions! I redownloaded the videos in the given resolution and it allowed me to obtain the correct cropping:
As to the videos with undesired factors, I think it is fine that they are discarded, since they seem like outliers/noise
@universome @MRzzm Due to permissions and network reasons, the hd video data cannot be downloaded. Could you please share a downloaded data set? Thank you.
Hi @fuaiguo ! I think it's not allowed to distribute the videos directly, but you can try to use the downloading script which I used to download the dataset (it also cuts and crops them): https://github.com/universome/HDTF
@universome Thank you very much for your prompt reply. I will try the scheme as soon as possible.
@universome , how to download the audio as well?
Hi,
I used ffmpeg to cut the raw videos according to the given annotation time. But I found there is always some pause at the beginning of the cut clips. (vary from 1-3 seconds). The command I used is like ffmpeg -i [input_raw_video] -ss [start] -to [end] -c copy -y -qscale 0 [output].
Did you guys find this problem?
UPDATE:
Removing the -c copy option seems to work well.