michel-kraemer/gradle-download-task

Improve log output for parallel downloads

sschuberth opened this issue · 4 comments

sschuberth commented

With parallel downloads, it seems the log lists all (in may case > 3000) URLs at once as they are queued, but not when the actual download happens (as I don't assume all 3000 URLs are being downloaded from at the same time).

So, it would be nice if the log output would reflect which URLs are currently being downloaded from a time.

michel-kraemer commented

How are you executing the task? Are you on an interactive terminal (where you should get a progress bar for each file currently being downloaded) or are you running Gradle on some kind of CI server where you only see the log messages and no progress bar?

sschuberth commented

I'm using Git Bash for Windows, running in mintty as a terminal. But I also tried the new Windows Terminal app from Windows 11. In any case running this task looks like:

C:\Users\Sebastian\Development\GitHub\oss-review-toolkit\ort>gradlew.bat importScanCodeLicenseTexts
Configuration on demand is an incubating feature.

> Configure project :
Building ORT version 2255268ced-dirty.
Fetching directory listing from https://api.github.com/repos/nexB/scancode-toolkit/git/trees/develop...

> Task :utils:spdx-utils:importScanCodeLicenseTexts
Download https://raw.githubusercontent.com/nexB/scancode-toolkit/develop/src/licensedcode/data/licenses/389-exception.LICENSE
Download https://raw.githubusercontent.com/nexB/scancode-toolkit/develop/src/licensedcode/data/licenses/389-exception.yml
Download https://raw.githubusercontent.com/nexB/scancode-toolkit/develop/src/licensedcode/data/licenses/3com-microcode.LICENSE
Download https://raw.githubusercontent.com/nexB/scancode-toolkit/develop/src/licensedcode/data/licenses/3com-microcode.yml
<thousands of more URLs are printed to the console at once>
michel-kraemer commented

Please test the new version 5.1.3

sschuberth commented

Please test the new version 5.1.3

That seems to do the trick, thanks!

  • 👍1