This fork is an adaption of the fork of Destreamer for University of Pisa that implements the autofill for UNIPA login interface. The 99% of the code is equal to UNIPI version, so you can easily follow the instructions below. Thank them for their nice work!
Enjoy!
This fork allows people studying at the University of Palermo to download files and allow students to enter their private university profile directly, without inserting their personal data every time.
During the first run of the program, it will ask for your access credentials, and if you want to store them to automatically load for the next times.
If you agree, the access credentials to the UniPA authentication module are saved in the program's directory in a "credentials.txt" file following this format (1 string per line):
yourUserWithout@you.unipa.it
yourMagicPassword
You are storing plaintext-credentials, therefore pay attention! You can find the original work here, if you find any issues, please submit them to him.
If you have to download lots of files, you can create a python script to keep working after crash without doing any action. Of course you have to manually stop it with ctrl+z, or it will run forever!
#!/usr/bin/env python3
import os
import sys
string = ""
filename = os.path.basename(__file__)
for arg in sys.argv:
if (arg != "python" and arg!="python3" and arg!= "./"+filename and arg!= filename):
string+= (arg+" ")
print('''\n\nRunning forever-plugin for destreamer!
I will keep working after crash!
You are running with this arguments: {}
Remember to close with ctrl+z, or i will run forever!\n\n'''.format(string))
while True:
os.system("./destreamer.sh "+string)Create a file.py with the name you want (suggested forever.py....) and run the script with the desired arguments like the main program.
In addiction i recommend to use:
./destreamer.sh -f list.txt -O ./desidered/folder --format mp4 --skipor
python3 ./forever.py -f list.txt -O ./desidered/folder --format mp4 --skip
destreamer v3.0 is just around the corner. Download speed improvement is astonishing and we have a never before seen photo from the design sessions:
Help us pick a codename for the new release:
Comment in this thread: snobu#223
(Alternative artwork proposals are welcome! Submit one through an Issue.)
This release would not have been possible without the code and time contributed by two distinguished developers: @lukaarma and @kylon. Thank you!
- Politecnico di Milano: fork over at https://github.com/SamanFekri/destreamer
- Università di Pisa: fork over at https://github.com/Guray00/destreamer-unipi
- Università della Calabria: fork over at https://github.com/peppelongo96/UnicalDown
- Università degli Studi di Parma: fork over at https://github.com/vRuslan/destreamer-unipr
- Added title template
- Major code refactoring (all credits to @lukaarma)
- Destreamer is now able to refresh the session's access token. Use this with
-k(keep cookies) and tick "Remember Me" on login. - We added support for closed captions (see
--closedCaptionsbelow)
Hopefully this doesn't break the end user agreement for Microsoft Stream. Since we're simply saving the HLS stream to disk as if we were a browser, this does not abuse the streaming endpoints. However i take no responsibility if either Microsoft or your Office 365 admins request a chat with you in a small white room.
- Node.js: You'll need Node.js version 8.0 or higher. A GitHub Action runs tests on all major Node versions on every commit. One caveat for Node 8, if you get a
Parse Errorwithcode: HPE_HEADER_OVERFLOWyou're out of luck and you'll need to upgrade to Node 10+. - npm: usually comes with Node.js, type
npmin your terminal to check for its presence - ffmpeg: a recent version (year 2019 or above), in
$PATHor in the same directory as this README file (project root). - git: one or more npm dependencies require git.
Destreamer takes a honeybadger approach towards the OS it's running on. We've successfully tested it on Windows, macOS and Linux.
Make sure you use the right script (.sh, .ps1 or .cmd) and escape char (if using line breaks) for your shell.
PowerShell uses a backtick [ ` ] and cmd.exe uses a caret [ ^ ].
Note that destreamer won't run in an elevated (Administrator/root) shell. Running inside Cygwin/MinGW/MSYS may also fail, please use cmd.exe or PowerShell if you're on Windows.
WSL (Windows Subsystem for Linux) is not supported as it can't easily pop up a browser window. It may work by installing an X Window server (like Xming) and exporting the default display to it (export DISPLAY=:0) before running destreamer. See this issue for more on WSL v1 and v2.
Yes, yes you can. This may be useful if your main browser has some authentication plugins that are required for you to logon to your Microsoft Stream tenant.
To use your own browser for the authentication part, locate the following snippet in src/destreamer.ts:
const browser: puppeteer.Browser = await puppeteer.launch({
executablePath: getPuppeteerChromiumPath(),
headless: false,
userDataDir: (argv.keepLoginCookies) ? chromeCacheFolder : undefined,
args: [
'--disable-dev-shm-usage',
'--fast-start',
'--no-sandbox'
]
});Now, change executablePath to reflect the path to your browser and profile (i.e. to use Microsoft Edge on Windows):
executablePath: 'C:\Program Files (x86)\Microsoft\Edge\Application\msedge.exe',Note that for Mac/Linux the path will look a little different but no other changes are necessary.
You need to rebuild (npm run build) every time you change this configuration.
To build destreamer clone this repository, install dependencies and run the build script -
$ git clone https://github.com/Bixio999/destreamer-unipa
$ cd ./destreamer-unipa
$ npm install
$ npm run build$ ./destreamer.sh
Options:
--help Show help [boolean]
--version Show version number [boolean]
--username, -u The username used to log into Microsoft Stream (enabling this will fill in the email field for
you). [string]
--videoUrls, -i List of urls to videos or Microsoft Stream groups. [array]
--inputFile, -f Path to text file containing URLs and optionally outDirs. See the README for more on outDirs.
[string]
--outputDirectory, -o The directory where destreamer will save your downloads. [string] [default: "videos"]
--outputTemplate, -t The template for the title. See the README for more info.
[string] [default: "{title} - {publishDate} {uniqueId}"]
--keepLoginCookies, -k Let Chromium cache identity provider cookies so you can use "Remember me" during login.
Must be used every subsequent time you launch Destreamer if you want to log in automatically.
[boolean] [default: false]
--noExperiments, -x Do not attempt to render video thumbnails in the console. [boolean] [default: false]
--simulate, -s Disable video download and print metadata information to the console.
[boolean] [default: false]
--verbose, -v Print additional information to the console (use this before opening an issue on GitHub).
[boolean] [default: false]
--closedCaptions, --cc Check if closed captions are available and let the user choose which one to download (will not
ask if only one available). [boolean] [default: false]
--noCleanup, --nc Do not delete the downloaded video file when an FFmpeg error occurs.[boolean] [default: false]
--vcodec Re-encode video track. Specify FFmpeg codec (e.g. libx265) or set to "none" to disable video.
[string] [default: "copy"]
--acodec Re-encode audio track. Specify FFmpeg codec (e.g. libopus) or set to "none" to disable audio.
[string] [default: "copy"]
--format Output container format (mkv, mp4, mov, anything that FFmpeg supports).
[string] [default: "mkv"]
--skip Skip download if file already exists. [boolean] [default: false]
-
both --videoUrls and --inputFile also accept Microsoft Teams Groups url so if your Organization placed the videos you are interested in a group you can copy the link and Destreamer will download all the videos it can inside it! A group url looks like this https://web.microsoftstream.com/group/XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX
-
Passing
--usernameis optional. It's there to make logging in faster (the username field will be populated automatically on the login form). -
You can use an absolute path for
-o(output directory), for example/mnt/videos. -
We default to
.mkvfor the output container. If you prefer something else (likemp4), pass--format mp4.
Download a video -
$ ./destreamer.sh -i "https://web.microsoftstream.com/video/VIDEO-1"Download a video and re-encode with HEVC (libx265) -
$ ./destreamer.sh -i "https://web.microsoftstream.com/video/VIDEO-1" --vcodec libx265Download a video and speed up the interactive login by automagically filling in the username -
$ ./destreamer.sh -u user@example.com -i "https://web.microsoftstream.com/video/VIDEO-1"Download a video to a custom path -
$ ./destreamer.sh -i "https://web.microsoftstream.com/video/VIDEO-1" -o /Users/hacker/DownloadsDownload two or more videos -
$ ./destreamer.sh -i "https://web.microsoftstream.com/video/VIDEO-1" \
"https://web.microsoftstream.com/video/VIDEO-2"Download many videos but read URLs from a file -
$ ./destreamer.sh -f list.txtYou can create a .txt file containing your video URLs, one video per line. The text file can have any name, followed by the .txt extension.
Additionally you can have destreamer download each video in the input list to a separate directory.
These optional lines must start with white space(s).
Usage -
https://web.microsoftstream.com/video/xxxxxxxx-aaaa-xxxx-xxxx-xxxxxxxxxxxx
-dir=videos/lessons/week1
https://web.microsoftstream.com/video/xxxxxxxx-aaaa-xxxx-xxxx-xxxxxxxxxxxx
-dir=videos/lessons/week2"
The -t option allows users to input a template string for the output file names.
You can use one or more of the following magic sequence which will get substituted at runtime. The magic sequence must be surrounded by curly brackets like this: {title} {publishDate}
title: Video titleduration: Video duration in HH:MM:SS formatpublishDate: The date when the video was published in YYYY-MM-DD formatpublishTime: The time the video was published in HH:MM:SS formatauthor: Name of video publisherauthorEmail: E-mail of video publisheruniqueId: An unique-enough ID generated from the video metadata
Example -
Input:
-t '{title} - {duration} - {publishDate} - {publishTime} - {author} - {authorEmail} - {uniqueId}'
Expected filename:
This is an example - 0:16:18 - 2020-07-30 - 10:30:13 - lukaarma - example@domain.org - #3c6ca929.mkv
Windows Terminal -
iTerm2 on a Mac -
By default, downloads are saved under project root Destreamer/videos/ ( Not the system media Videos folder ), unless specified by -o (output directory).
Contributions are welcome. Open an issue first before sending in a pull request. All pull requests require at least one code review before they are merged to master.
Please open an issue and we'll look into it.