Stuck at "attaching to airflow_init_1"
rab241 opened this issue · 20 comments
Thank you for the help! I figured out on my own some issues and also cant find updated instructions on setting up the environments. Here is a screenshot of my issue. I also am getting that the username and password for airflow is wrong. I also dont understand where and when to enter that. I would appreciate the help!
Hi @rab241 It looks like the postgres container is not running. Can you
- tear down infra with
./tear_down_infra.sh {your-bucket-name}and then - Increase docker size as shown here apache/airflow#14365 (comment), restart docker and
./setup_infra.sh {your-bucket-name}
If the issue still exists can you copy paste the logs ?
Thank you that seemed to work! However it is still stuck at attaching to airflow-init_1 and nothing seems to happen
@rab241 It takes a few min(1-2), can you lmk how long you waited ?
@rab241 I had this same issue when rerunning ./setup_infra.sh. Clearing the Docker cache and rerunning the script solved it for me: docker system prune --volumes
Alternatively, maybe adding the --force-recreate flag to the docker compose command will fix this but I haven't tried it out yet.
@josephmachado I was able to recreate the issue by running the ./setup_infra.sh script, canceling it after the image builds, and then re-running the script. Modifying docker compose up airflow-init --force-recreate with this flag fixed the issue without having to manually clear my docker cache. If this solves the problem, I can submit a PR if you'd like.
Hi, thanks for all the help! However, this is the issue that comes up when I ran the airflow recreate
Thank you. That would be amazing @dennisgsmith
@rab241 Think the Postgres container is still running(hence the 5432 port in use already), can you restart the docker container and try again using this #7 (comment)
I had done that and redid it and the same 5432 port in use is still the error
@rab241 If you execute lsof -n -i:5432 | grep LISTEN you can see whatever is taking up that port. If you have a system version of Postgres using the port will need to shut it down.
nothing shows up when I run that command
@rab241
What if you run as sudo? sudo lsof -i :5432
If there is anything running, you should be able to kill it by specifying the PID
sudo kill -9 PID
dwyl/learn-postgresql#60 (comment)
You could also try killing it by specifying by postgres user without a PID
sudo pkill -u postgres
PostgresApp/PostgresApp#197 (comment)
@rab241 could you run docker ps and paste the output here please.
So while I was setting up again, it got stuck again at attaching to airflow. I went into the docker and manually ran the different containers which started to run the airflow init and then reached to points where my s3 bucket needed more permissions. I created the permissions and now it is looping on this error: An error occurred (ClusterNotFound) when calling the DescribeClusters operation: Cluster sde-batch-de-project not found.
it keeps sleeping for 60 seconds and checking
I cancelled and tore down the infrastructure and this error occurred: An error occurred (NoSuchEntity) when calling the DetachRolePolicy operation: The role with name sde-spectrum-redshift cannot be found.
Dissociating AWSGlueConsoleFullAccess policy from sde-spectrum-redshift role
with manually running airflow-init, and postgres this happens before what I just mentioned
It successfully set up containers and airflow after completely trying again
Then I ran this and the airflow password worked but then I dont know what to do now
pgcli -h localhost -p 5432 -U airflow
I realized it is working now lol! Thanks for all your help!