svt/encore

Installation Issues

Closed this issue · 10 comments

guacharo commented

Hello,

I am attempting a fresh install on a system that meets the minimum requirements.

cat /etc/redhat-release
CentOS Linux release 7.9.2009 (Core)

#Java
java --version
openjdk 11.0.12 2021-07-20 LTS
OpenJDK Runtime Environment 18.9 (build 11.0.12+7-LTS)
OpenJDK 64-Bit Server VM 18.9 (build 11.0.12+7-LTS, mixed mode, sharing)

#ffmpeg
ffmpeg --hide_banner --version
ffmpeg version 4.4.1-static https://johnvansickle.com/ffmpeg/ Copyright (c) 2000-2021 the FFmpeg developers

#Redis
redis-cli
127.0.0.1:6379> ping
PONG

After running ./gradlew clean check I see the following:

Task :Verify ffmpeg is in path, required for integration tests
Hyper fast Audio and Video encoder
usage: ffmpeg [options] [[infile options] -i infile]... {[outfile options] outfile}...

Use -h to get full help or, even better, run 'man ffmpeg'

Task :Verify ffprobe is in path, required for integration tests
Simple multimedia streams analyzer
usage: ffprobe [OPTIONS] [INPUT_FILE]

You have to specify one input file.
Use -h to get full help or, even better, run 'man ffprobe'.

Task :Verify mediainfo is in path, required for integration tests
Usage: "mediainfo [-Options...] FileName1 [Filename2...]"
"mediainfo --Help" for displaying more information

Task :compileTestJava
Note: Some input files use or override a deprecated API.
Note: Recompile with -Xlint:deprecation for details.

Task :test
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.springframework.util.ReflectionUtils (file:/home/tux/.gradle/caches/modules-2/files-2.1/org.springframework/spring-core/5.3.9/cfef19d1dfa41d56f8de66238dc015334997d573/spring-core-5.3.9.jar) to constructor java.net.URI()
WARNING: Please consider reporting this to the maintainers of org.springframework.util.ReflectionUtils
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
OpenJDK 64-Bit Server VM warning: Sharing is only supported for boot loader classes because bootstrap classpath has been appended

LocalEncodeIntegrationTest > jobIsSuccessfulAndNoAudioPresets(File) FAILED
java.lang.AssertionError at LocalEncodeIntegrationTest.kt:17

EncoreIntegrationTest > jobIsSuccessfulStereo(File) FAILED
java.lang.AssertionError at EncoreIntegrationTest.kt:95

EncoreIntegrationTest > jobIsSuccessfulSurround(File) FAILED
java.lang.AssertionError at EncoreIntegrationTest.kt:31

EncoreIntegrationTest > multipleInputsWithSeekAndDuration(File) FAILED
java.lang.AssertionError at EncoreIntegrationTest.kt:78

127 tests completed, 4 failed

Task :test FAILED

FAILURE: Build failed with an exception.

  • What went wrong:
    Execution failed for task ':test'.

There were failing tests. See the report at: file:///home/tux/encore/build/reports/tests/test/index.html

  • Try:
    Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

  • Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 8m 38s
11 actionable tasks: 10 executed, 1 up-to-date

Please let me know what is missing for a clean install.
Thanks in advance.

grusell commented

It appears some of the integration tests failed during the build, but without seeing the test report it is hard to say why. Could you check the test-report in /home/tux/encore/build/reports/tests/test/index.html and see what it says?

janderssonse commented

Hi, guacharo, maybe you could try with the ffmpeg and mediainfo versions available from the https://github.com/svt/homebrew-avtools and see if that works for you. That is versions we have tested ourselves. Please keep us updated.( I also had problems when using another mediainfo version than the one provide in avtools-repo, will have to pinpoint the problem at some point.) We are currently also discussing how we can improve testing out encore in general, in the team.

guacharo commented

Hello,

I moved my test environment from CentOS 7 to Ubuntu (Homebrew required a lot of dependencies not easily added to CentOS 7).

On Ubuntu the build process still failed on 2 tests.
EncoreIntegrationTest > jobIsCancelled(File) FAILED
java.lang.AssertionError at EncoreIntegrationTest.kt:154

LocalEncodeIntegrationTest > jobIsSuccessful

AndNoAudioPresets(File) FAILED
feign.RetryableException at FeignException.java:249
Caused by: java.net.SocketTimeoutException at Okio.java:232
Caused by: java.net.SocketTimeoutException at SocketInputStream.java:-2

I have attached the extended details of the failed tests in a txt file.

Please let me know if you need anything else.

failed_tests.txt

janderssonse commented

Could you also post the versions used: do

  • java --version,
  • ffmpeg -v
  • mediainfo --version
  • and for ubuntu version: . cat /etc/issue
guacharo commented

Here you go:

openjdk 11.0.11 2021-04-20
OpenJDK Runtime Environment (build 11.0.11+9-Ubuntu-0ubuntu2.18.04)
OpenJDK 64-Bit Server VM (build 11.0.11+9-Ubuntu-0ubuntu2.18.04, mixed mode, sharing)

ffmpeg version 4.3.1 Copyright (c) 2000-2020 the FFmpeg developers
built with gcc 7 (Ubuntu 7.5.0-3ubuntu1~18.04)
configuration: --prefix=/home/linuxbrew/.linuxbrew/Cellar/ffmpeg-encore/4.3.1_1 --enable-shared --enable-pthreads --enable-version3 --enable-hardcoded-tables --enable-avresample --cc=gcc-7 --host-cflags= --host-ldflags= --enable-gpl --enable-libaom --enable-libmp3lame --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libx265 --enable-lzma --enable-libass --enable-libfontconfig --enable-libfreetype --disable-libjack --disable-indev=jack --enable-libaom --enable-openssl --enable-libssh --enable-libvmaf --enable-nonfree --enable-libfdk-aac

MediaInfo Command line,
MediaInfoLib - v21.09

Ubuntu 18.04.4 LTS

janderssonse commented

Built here fine with same versions, (except i run on ubuntu 20 but that should not matter at all). I tested with --parallel flag to ./gradlew clean build --parallel to see if parallel builds could be a thing messing up things but that also worked fine. It seems from a quick glance at the log you sent that the mockserver serving the integrations tests are running might become unavailable, saw a lot of sockettimeouts for the requests there in the log, but I don't really know.

guacharo commented

Hello,

I was finally able to get a clean build. The main issue on both of my previous dev VMs I believe was resource related.
Initially I was testing on a 2vCPU system with 4G of RAM.

As soon as I bumped by test VM to 4vCPU with 8G of RAM, I was able to successfully build.

$ ./gradlew clean check
Starting a Gradle Daemon, 1 busy Daemon could not be reused, use --status for details

Task :Verify ffmpeg is in path, required for integration tests
Hyper fast Audio and Video encoder
usage: ffmpeg [options] [[infile options] -i infile]... {[outfile options] outfile}...

Use -h to get full help or, even better, run 'man ffmpeg'

Task :Verify ffprobe is in path, required for integration tests
Simple multimedia streams analyzer
usage: ffprobe [OPTIONS] [INPUT_FILE]

You have to specify one input file.
Use -h to get full help or, even better, run 'man ffprobe'.

Task :Verify mediainfo is in path, required for integration tests
Usage: "mediainfo [-Options...] FileName1 [Filename2...]"
"mediainfo --Help" for displaying more information

Task :compileTestJava
Note: Some input files use or override a deprecated API.
Note: Recompile with -Xlint:deprecation for details.

Task :test
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.springframework.util.ReflectionUtils (file:/home/tux/.gradle/caches/modules-2/files-2.1/org.springframework/spring-core/5.3.9/cfef19d1dfa41d56f8de66238dc015334997d573/spring-core-5.3.9.jar) to constructor java.net.URI()
WARNING: Please consider reporting this to the maintainers of org.springframework.util.ReflectionUtils
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
OpenJDK 64-Bit Server VM warning: Sharing is only supported for boot loader classes because bootstrap classpath has been appended

Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings

BUILD SUCCESSFUL in 14m 9s
15 actionable tasks: 14 executed, 1 up-to-date

However, I still don't see the OpenAPI3 Swagger Gui on port 8080 or 80.

$ curl http://localhost:8080/swagger-ui.html
curl: (7) Failed to connect to localhost port 8080 after 0 ms: Connection refused
$ curl http://localhost/swagger-ui.html
curl: (7) Failed to connect to localhost port 80 after 0 ms: Connection refused

What else am I missing?

grusell commented

The documentation is maybe a bit unclear on this, but the gradle clean check command doesnt actually build or start encore it just runs the tests. Since the tests passed you should be able to start encore by running
SPRING_PROFILES_ACTIVE=local ./gradlew bootRun

Alternatively, you could build an executable jar by running
./gradlew build
And then start encore by running
SPRING_PROFILES_ACTIVE=local java -jar build/libs/encore.jar

guacharo commented

Hello,

I was able to start encore and query the job queue.

I will continue to study the documentation on how to submit an encode job,

curl -v http://localhost:8080/encoreJobs

  • Trying ::1:8080...
  • TCP_NODELAY set
  • Connected to localhost (::1) port 8080 (#0)

GET /encoreJobs HTTP/1.1
Host: localhost:8080
User-Agent: curl/7.68.0
Accept: /

  • Mark bundle as not supporting multiuse
    < HTTP/1.1 200
    < Vary: Origin
    < Vary: Access-Control-Request-Method
    < Vary: Access-Control-Request-Headers
    < Content-Type: application/hal+json
    < Transfer-Encoding: chunked
    < Date: Fri, 05 Nov 2021 23:19:53 GMT
    <
    {
    "_embedded" : {
    "encoreJobs" : [ ]
    },
    "_links" : {
    "self" : {
    "href" : "http://localhost:8080/encoreJobs"
    },
    "profile" : {
    "href" : "http://localhost:8080/profile/encoreJobs"
    },
    "search" : {
    "href" : "http://localhost:8080/encoreJobs/search"
    }
    },
    "page" : {
    "size" : 20,
    "totalElements" : 0,
    "totalPages" : 0,
    "number" : 0
    }
  • Connection #0 to host localhost left intact

Thank you for the assistance.

janderssonse commented

Glad you got it further, closing this issue for now. We will try to improve the documentation in the future, with more examples etc.