kaltura/platform-install-packages

Installation fail - Ubuntu Xenial [16.04] all-in-one Kaltura server installation

Closed this issue · 9 comments

pkExec commented

Bug report

Please provide steps to reproduce, including errors from /opt/kaltura/log/kaltura_api_v3.log, /opt/kaltura/log/kaltura_apache_errors*log and /opt/kaltura/log/kaltura_prod.log:

  1. Start a new Amazon Lightsail instance with ubuntu 16.04
  2. Modify hosts file for your domain
  3. Install Percona as per the instructions https://github.com/kaltura/platform-install-packages/blob/Orion-15.11.0/doc/install-percona-deb.md
  4. Follow the instructions here https://github.com/kaltura/platform-install-packages/blob/Orion-15.11.0/doc/install-kaltura-xenial.md

Expected outcome:
Installed platform

Actual outcome:
Failed on the installing kaltura-db step.

Cause:
No databases are created

Root cause:
/opt/kaltura/bin/db_actions.rc has variable OP_DBS, while script expects DBS

Is this an all in one instance [single server] or a cluster?

  • All in one
  • Cluster
jessp01 commented

Hello @pkExec,

Thanks for reporting. This was already fixed and the updated kaltura-db package is available in the nightly repo. It will become the stable repo tomorrow.

For the time being, you can edit /etc/apt/sources.list.d/kaltura.list
So that it reads:

deb [arch=amd64] http://installrepo.origin.kaltura.org/repo/apt/xenial-nightly orion main

Then invoke:

# apt update
# apt purge kaltura-db

and re-run the deployment script.

Cheers,

pkExec commented

Thanks. Indeed this step was fixed. Now it fails on the "Waiting for the ElasticSearch daemon to start..." step.
It displays "We've waited a minute and the ES daemon is still unreachable, we're giving up: ("
sed: can't read /configurations/base.ini: No such file or directory

thecaribbean commented

I've received the Elastic Search error same as @pkExec

Setting up kaltura-elasticsearch (1.0.0-1+b6) ...
-> Downloading analysis-icu from elastic
[=================================================] 100%  
-> Installed analysis-icu
Waiting for the ElasticSearch daemon to start...
We've waited a minute and the ES daemon is still unreachable, we're giving up:(
Processing triggers for ureadahead (0.100.0-19.1) ...
Processing triggers for systemd (229-4ubuntu21.23) ...
sed: can't read /configurations/base.ini: No such file or directory

used a Fresh Install of Ubuntu 16.04 minimal iso and MySQL 5.5 Percona deb package only other package installed wis openssh-server

lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 16.04.6 LTS
Release: 16.04
Codename: xenial

mysql -V
mysql Ver 14.14 Distrib 5.5.62-38.14, for debian-linux-gnu (x86_64) using readline 5.1

jessp01 commented

Hello @pkExec, @thecaribbean,

Kaltura CE 15.14.0 was just released.
Kindly try to upgrade. If the issue persists, please edit /var/lib/dpkg/info/kaltura-elasticsearch.postinst
so that:

#!/bin/bash

becomes:

#!/bin/bash -x

then run:

# apt install -f

or, depending on the package's current state:

# dpkg-reconfigure kaltura-elasticsearch

And paste the output here.

Thanks,

thecaribbean commented

Started another fresh install and got up to:

Setting up kaltura-elasticsearch (1.0.0-1+b7) ...
-> Downloading analysis-icu from elastic
[=================================================] 100%  
-> Installed analysis-icu
Waiting for the ElasticSearch daemon to start...
We've waited a minute and the ES daemon is still unreachable, we're giving up:(
Processing triggers for ureadahead (0.100.0-19.1) ...
Processing triggers for systemd (229-4ubuntu21.23) ...
sed: can't read /configurations/base.ini: No such file or directory

edited /var/lib/dpkg/info/kaltura-elasticsearch.postinst

then run:

apt install -f
Reading package lists... Done
Building dependency tree
Reading state information... Done
0 upgraded, 0 newly installed, 0 to remove and 1 not upgraded.

Then ran:

root@video:~# dpkg-reconfigure kaltura-elasticsearch

  • KALTURA_FUNCTIONS_RC=/opt/kaltura/bin/kaltura-functions.rc
  • '[' '!' -r /opt/kaltura/bin/kaltura-functions.rc ']'
  • . /opt/kaltura/bin/kaltura-functions.rc
    ++ RC_FILE=/etc/kaltura.d/system.ini
    ++ '[' -r /etc/kaltura.d/system.ini ']'
    ++ . /etc/kaltura.d/system.ini
    +++ WEB_DIR=/opt/kaltura/web
    +++ LOG_DIR=/opt/kaltura/log
    +++ APP_DIR=/opt/kaltura/app
    +++ BASE_DIR=/opt/kaltura
    +++ PHP_BIN=/usr/bin/php
    +++ OS_KALTURA_USER=kaltura
    +++ MAILTO=myemail@example.com
    +++ ADMIN_CONSOLE_ADMIN_MAIL=myemail@example.com
    +++ DB1_HOST=127.0.0.1
    +++ DB1_NAME=kaltura
    +++ DB1_USER=kaltura
    +++ DB1_PASS=
    +++ DWH_HOST=127.0.0.1
    +++ DWH_PORT=3306
    +++ DWH_USER=etl
    +++ DWH_PASS=
    +++ KAVA_DB_HOST=@KAVA_DB_HOST@
    +++ KAVA_DB_PORT=@KAVA_DB_PORT@
    +++ KAVA_DB_USER=@KAVA_DB_USER@
    +++ KAVA_DB_PASS=@KAVA_DB_PASS@
    +++ SERVICE_URL=http://video.example.com
    +++ SPHINX_HOST=127.0.0.1
    +++ DB1_PORT=3306
    +++ SUPER_USER=root
    +++ SUPER_USER_PASSWD=
    +++ KALTURA_VIRTUAL_HOST_NAME=video.example.com
    +++ RED5_HOST=
    +++ dirname /var/lib/dpkg/info/kaltura-elasticsearch.postinst
    ++ COLORS_RC=/var/lib/dpkg/info/colors.sh
    ++ '[' -r /var/lib/dpkg/info/colors.sh ']'
    ++ '[' -r /etc/sysconfig/clock ']'
    ++ '[' -z '' ']'
    +++ get_tz
    +++ '[' -r /etc/sysconfig/clock ']'
    +++ '[' -n '' ']'
    ++++ awk -F ' ' '{print $1}'
    ++++ md5sum /etc/localtime
    +++ LOCALTIME_MD5=6a95f4e0602e0869a03a18a7501c6675
    ++++ sed s@/usr/share/zoneinfo/@@g
    ++++ tail -1
    ++++ awk -F ' ' '{print $2}'
    ++++ grep 6a95f4e0602e0869a03a18a7501c6675
    ++++ xargs md5sum
    ++++ find /usr/share/zoneinfo -type f
    +++ TZGUESS=America/St_Thomas
    +++ echo America/St_Thomas
    ++ ZONE=America/St_Thomas
    ++ export ZONE
    ++ '[' -r /opt/kaltura/app/configurations/consent_msgs ']'
    ++ CONSENT_FILE=/opt/kaltura/bin/contact.rc
    ++ SPHINX_DAEMON=kaltura-sphinx
    ++ BATCH_DAEMON=kaltura-batch
    ++ MEMCACHE_DAEMON=memcached
    +++ lsb_release -i -s
    ++ DISTRO=Ubuntu
    ++ '[' Ubuntu = Ubuntu -o Ubuntu = Debian ']'
    ++ APACHE_DAEMON=apache2
    ++ MYSQL_DAEMON=mysql
    ++ MONIT_DAEMON=monit
    ++ ALL_DAEMONS='apache2 kaltura-batch monit memcached'
    ++ INIT_CONFIG_UTIL=sysv-rc-conf
    ++ PACKAGE_TYPE=deb
    ++ QUERY_COMMAND='dpkg -l'
    ++ MAIL_LOG=/var/log/mail.log
    ++ POST_INST_MAIL_SUBJECT='Your new Kaltura server is up! and 6 Quick Tips For Smooth Kaltura Maintenance'
    ++ POST_INST_MAIL_TMPL=/opt/kaltura/app/configurations/post_inst_mail
    ++ ANALYTICS_SERVER_URI=https://installytics.kaltura.org/report/add_rec.php
    ++ NEWSLETTER_URL=https://agentcontribs.kaltura.org/newsletter_subscribe.php
    ++ NEWSLETTER_MSG='
    To keep up with the bi-weekly Kaltura releases, and stay up to date with
    the latest news, security and bug updates, and connect with the global
    Kaltura community - sign up for our regular Kaltura Community updates.
    Please enter your email to sign up, or enter NO to pass.
    '
  • '[' configure = configure ']'
  • update-rc.d kaltura-elastic-populate defaults
    ++ date +%Y_%m
  • TIME_STAMP=2020_01
  • ES_HOST=127.0.0.1
  • ES_PORT=9200
  • set -e
  • mv /etc/elasticsearch/elasticsearch.yml /etc/elasticsearch/elasticsearch.yml.orig
    ++ hostname
  • sed -e s#@BASE_DIR@#/opt/kaltura#g -e s#@LOG_DIR@#/opt/kaltura/log#g -e s#@hostname@#video#g /opt/kaltura/app/configurations/elastic/elasticsearch.template.yml
  • chown -R elasticsearch /opt/kaltura/log/elasticsearch /opt/kaltura/var/lib/elasticsearch
  • invoke-rc.d elasticsearch restart
  • '[' '!' -d /usr/share/elasticsearch/plugins/analysis-icu ']'
  • mkdir -p /opt/kaltura/app/configurations/elastic/populate
  • chown kaltura /opt/kaltura/app/configurations/elastic/populate
  • set +e
  • echo 'Waiting for the ElasticSearch daemon to start...'
    Waiting for the ElasticSearch daemon to start...
  • ATTEMPT=0
  • curl -f 'http://127.0.0.1:9200/_cluster/health?wait_for_status=yellow&timeout=10s'
  • sleep 5
  • '[' 0 -eq 22 ']'
    ++ expr 0 + 1
  • ATTEMPT=1
  • curl -f 'http://127.0.0.1:9200/_cluster/health?wait_for_status=yellow&timeout=10s'
  • sleep 5
  • '[' 1 -eq 22 ']'
    ++ expr 1 + 1
  • ATTEMPT=2
  • curl -f 'http://127.0.0.1:9200/_cluster/health?wait_for_status=yellow&timeout=10s'
  • sleep 5
  • '[' 2 -eq 22 ']'
    ++ expr 2 + 1

========truncated-output==============

after attempt 20. Ctl-D to kill

thecaribbean commented

I noticed:
sed: can't read /configurations/base.ini: No such file or directory

so ran

find / -name base.ini

and got it at this location

/opt/kaltura/app/configurations/base.ini

thecaribbean commented

Looking through the installation script

vi install_kaltura_all_in_1.sh

I note the last line is

for an all in one deployment, this needs to be set to true

sed -i 's@(exec_elastic\s*=\s*)false@\1true@' $APP_DIR/configurations/base.ini

so despite the error Elastic Search is installed and I'm able to log into KMC and upload/server media. Not sure how Elastic Search error will impact operation.

pkExec commented

Tried too, same errors as @thecaribbean on #659 (comment)

The first error (elasticsearch not starting) is because:
a) $JAVA_HOME is not set for root user.
b) Permissions are wrong as per https://stackoverflow.com/a/48390311/1543677
After following the instructions on stackoverflow and manually setting $JAVA_HOME, elasticsearch does start.

The second error (with the sed command in the install script), is because for some reason the $APP_DIR variable is not set. After replacing it with the installation path (/opt/kaltura/app), it finishes without errors.

  • 👍1
jessp01 commented

Hi @pkExec, all,

These ENV vars are defined in /etc/profile.d/kaltura-base.sh but of course, they will only be auto sourced when a new session is established, therefore, see this commit: 7d36b82
which should take care of that.

With regards to

Not sure how Elastic Search error will impact operation.

ES is actually only utilised by the esearch service (https://developer.kaltura.com/api-docs/service/eSearch) and, at present time, KMC and KMCng's code do not make use of it so it isn't a critical component for Kaltura CE.

Thanks all, for your reports,