3.0.1 - token in .cif.yml does not match sqlite tokens entry.
Closed this issue · 11 comments
Hey Wes,
(Erasing the template since I think my issue is very simple and I can describe it in 2-3 sentences)
Doing a brand new install of 3.0.1 - btw, noticed that the docker override now passes! - I was trying to build the latest version.
Everything installs/sets up. I start, and it seems that the token in ~/cif.yml does not match the entry token in the sqlite3 tokens table.
Not sure what would cause this. I can confirm that as soon as I set the sqlite3 token in the file, it starts working.
Adding some real info - this is after a brand new install (ubuntu 16.04):
cif@1ea428ac4827:~$ cat .cif.yml
token: 5646bdd5ba247423d92d2731e38f8298f86a1c7a3ff4bdbf6159343ce4598bb7b1f1f26447e320a8
$ cif -p
Traceback (most recent call last):
File "/usr/local/bin/cif", line 11, in <module>
load_entry_point('cifsdk==3.0.3', 'console_scripts', 'cif')()
File "/usr/local/lib/python2.7/dist-packages/cifsdk/client/client.py", line 147, in main
ret = cli.ping()
File "/usr/local/lib/python2.7/dist-packages/cifsdk/client/http.py", line 229, in ping
rv = self._get(uri)
File "/usr/local/lib/python2.7/dist-packages/cifsdk/client/http.py", line 97, in _get
raise e
cifsdk.exceptions.AuthError: unauthorized
$ sqlite3 /var/lib/cif/cif.sqlite
SQLite version 3.11.0 2016-02-15 17:29:24
Enter ".help" for usage hints.
sqlite> select * from tokens;
1|admin|9bb9a1f81fd191314d963a653c4cee828d878a1edb278c6380218d1c7902f66151d487b979ef175a||1|1|||1|
echo 'token: 9bb9a1f81fd191314d963a653c4cee828d878a1edb278c6380218d1c7902f66151d487b979ef175a' > .cif.yml
$ cif -p
roundtrip: 0.0238049030304 ms
roundtrip: 0.0273458957672 ms
roundtrip: 0.0282199382782 ms
roundtrip: 0.0267820358276 ms
@wesyoung Ping - anything on this? I believe it's the last thing preventing from launching the latest version in a docker container.
@wesyoung Ping - anything on this? I believe it's the last thing preventing from launching the latest version in a docker container.
Let me take a look at this @ventz. Wes is focusing on v4/v5 and isn't paying too much attention to v3 at the moment.
This is only when installing via the deployment kit?
I just ran through the 3.0.3 install a couple of times and it seems to be fine for me.
cif@cifv3:/usr/local/src/bearded-avenger-deploymentkit-3.0.3$ cif-tokens +-------------+----------+------------------+-------+------+-------+------+---------+----------------------------------------------------------------------------------+ | username | groups | last_activity_at | admin | read | write | acl | expires | token | +-------------+----------+------------------+-------+------+-------+------+---------+----------------------------------------------------------------------------------+ | admin | everyone | None | True | True | True | None | None | fdb13100dc5666abf309559a01d22ebb462dda27deb90ad33365c09dc11506d65455bfad07b2314b | | csirtg-smrt | everyone | None | None | None | True | None | None | 8f670e96dde69e23aa0b3530dad9548cd22e67278227006c918ff9db9e807b8525c55446e44e0fa8 | | hunter | everyone | None | None | None | True | None | None | 5e27d5fee9c8e24f0d936c981cb36432b93358fa21be744ad7a3d1d6cf12f4e37830dc64bff05265 | +-------------+----------+------------------+-------+------+-------+------+---------+----------------------------------------------------------------------------------+ cif@cifv3:/usr/local/src/bearded-avenger-deploymentkit-3.0.3$ cat /home/cif/.cif.yml token: fdb13100dc5666abf309559a01d22ebb462dda27deb90ad33365c09dc11506d65455bfad07b2314b
Can you try with 3.0.3 instead of 3.0.1?
@sfinlon - v 3.0.3 is not building for me in Docker:
Preparing wheel metadata: finished with status 'error'"]}
PLAY RECAP *********************************************************************
localhost : ok=18 changed=13 unreachable=0 failed=1
Removing intermediate container ff328effe753
The command '/bin/sh -c git checkout tags/$CIF_VERSION ; cp -f test.sh /root/test.sh ; chmod 755 /root/test.sh ; chmod 755 /entrypoint ; cd ubuntu16 && bash bootstrap.sh' returned a non-zero code: 2
Version 3.0.1 built without issues.
I am doing all this to build a docker version of this :)
@sfinlon Absolutely:
1.) mkdir 'container'
Place these files in it:
Dockerfile
FROM ubuntu:16.04
LABEL developer="Wes Young <wes@csirtgadgets.org>"
LABEL docker_maintainer="Ventz Petkov (@ventz) and Dylan Jacob (@dylanjacob)"
EXPOSE 443 5000
ENV DOCKER_BUILD=yes
ENV CIF_VERSION 3.0.3
ENV CIF_RUNTIME_PATH /var/lib/cif
ENV SUDO_USER root
ENV DEBIAN_FRONTEND=noninteractive
COPY supervisord.conf /usr/local/etc/supervisord.conf
COPY entrypoint /
RUN echo "resolvconf resolvconf/linkify-resolvconf boolean false" | debconf-set-selections ; \
mkdir -p /etc/resolvconf/resolv.conf.d ; \
mkdir -p /var/log/cif ; \
useradd cif
RUN apt-get update && apt-get install -y git sudo resolvconf supervisor curl; \
cd /tmp && git clone https://github.com/csirtgadgets/bearded-avenger-deploymentkit
WORKDIR /tmp/bearded-avenger-deploymentkit
# Override for Docker - don't need anything related to systemd; \
RUN git checkout tags/$CIF_VERSION ; \
cp -f test.sh /root/test.sh ; \
chmod 755 /root/test.sh ; \
chmod 755 /entrypoint ; \
cd ubuntu16 && bash bootstrap.sh
WORKDIR /tmp
RUN rm -Rf /tmp/bearded-avenger*
# This has to be last/post volume dir work.
# See NOTE at: https://docs.docker.com/engine/reference/builder/#volume
#VOLUME /etc/cif
#VOLUME /var/log/cif
#VOLUME /var/lib/cif
WORKDIR /home/cif
#RUN ln -s /home/cif/.cif.yml /root/.cif.yml ; \
#ln -s /home/cif/.cifrc /root/.cifrc
ENTRYPOINT ["/entrypoint", "-n"]
Entrypoint
#!/bin/bash
set -e
chown -R cif:cif /var/lib/cif
chown -R cif:cif /etc/cif
chown -R cif:cif /var/log/cif
/usr/bin/supervisord -n -c /usr/local/etc/supervisord.conf
supervisord.conf
[supervisord]
nodaemon = true
loglevel = DEBUG
[program:cif-router]
user=cif
command = cif-router --config /etc/cif/cif-router.yml --runtime-path=/var/lib/cif --hunter-threads 2 --gatherer-threads 2
autostart = true
stderr_logfile = /var/log/cif/cif-router.log
stopsignal = INT
[program:cif-httpd]
user=cif
command = cif-httpd --runtime-path=/var/lib/cif --listen 0.0.0.0
autostart = true
stderr_logfile = /var/log/cif/cif-httpd.log
stopsignal = INT
[program:csirtg-smrt]
user=cif
command = csirtg-smrt --client cif --config /etc/cif/csirtg-smrt.yml --runtime-path=/var/lib/cif --rule /etc/cif/rules/default --service --remember --delay 10
autostart = true
stderr_logfile = /var/log/cif/csirtg-smrt.log
stopsignal = INT
2.) build your container:
docker build --rm=true --force-rm=true -t ventz/docker-cif container
3.) run it:
#!/bin/bash
C=$(docker run --name cifv3 -it -d -p 443:443 -p 5000:5000 ventz/docker-cif)
echo "Getting a shell into the container..."
docker exec -it $C /bin/bash
This was the last working version:
https://github.com/ventz/docker-cif
Since then it broke, it went from problems to fixes, to patches -- eventually Wes merged some of the key "Docker related" things into the main code, which was awesome, so I could clean up (to above), and finally v 3.0.1 looked very promising, with the exception of the token link issue -- the token in the sqlite DB did not match the one in the config.
Ok, I fixed the build so it builds/runs, and it's replicating the token error. I'll have to look at the config to see where, but it looks like it's generating the admin token twice and either overriding or not updating the yml file.
@ventz i rewrote the majority of the build process and configurations...
I threw it in a repo on my github here:
https://github.com/sfinlon/cif-docker
In your setup, I didn't troubleshoot to figure out exactly where but it looks like it was running the ansible setup twice, once with the env vars set and once without. Because of the env var change it was creating /var/lib/cif/cif.sqlite and /var/lib/cif/cif.db and the yml pointed to sqlite but cif was running against db. Instead of just running the bootstrap, this builds cif and then installs the ansible role, I don't know if it's cleaner/easier but it's more straight forward.
Feel free to take it and rip it apart use what parts you want and throw it in your repo.
@sfinlon The goal all along was to push this into the official project. I would love it if you can take over it actually! (I would fix a couple of things in the Dockerfile to make the more "dynamic/variable"), but you are welcome to take whatever you want from my stuff.
I've been pinged by a lot of people about CIF + docker, and updating the container I have (>50 people in the past year), so it would be great to have it in the official github repo.