consumer: Cannot connect to amqp://user:**#rabbit1:5672//: failed to resolve broker hostname - python-3.x

Question

I have perused through very similar stackoverflow questions and changed my code to what it is now. I am still getting the cannot resolve broker hostname error. If I just use amqp:user:mypass#locahost:5672 I get the a connection refused
# docker-compoose.yml
version: "3.8"
services:
broker:
image: rabbitmq:3-management-alpine
hostname: rabbit1
environment: &env
RABBITMQ_ERLANG_COOKIE: "SWQOKODSQALRPCLNMEQG"
RABBITMQ_DEFAULT_USER: "user"
RABBITMQ_DEFAULT_PASS: "password"
RABBITMQ_DEFAULT_VHOST: "/"
CELERY_BROKER_URL: "amqp://user:password#rabbit1:5672"
ports:
- "5672:5672"
- "15672:15672"
expose:
- "5672"
networks:
- webnet
volumes:
- .:/home/user
worker:
build:
context: .
dockerfile: ./worker/Dockerfile
command: ["celery", "worker", "--app=worker.tasks.app", "--loglevel=INFO"]
environment:
<<: *env
links:
- broker
depends_on:
- broker
networks:
- webnet
volumes:
- .:/home/user
networks:
webnet:
# Dockerfile
FROM python:3.7-slim
RUN groupadd user && useradd --create-home --home-dir /home/user -g user user
ENV CELERY_USER = user
WORKDIR /home/user
COPY . .
RUN pip install --no-cache-dir -r requirements.txt
RUN rm requirements.txt
I have tried just starting the rabbitmq broker through docker-compose and then manually doing a celery worker --app=tasks.app and that will work after setting the CELERY_BROKER_URL=amqp://user:mypass#localhost:5672
It throws the error when I have the worker and broker in the same yaml. I've tried it without the network, manually setting the CELERY_BROKER_URL in the Dockerfile also. If anyone has ideas what I'm doing wrong, I would really appreciate it.

Answer 1

There are a few things going on here. The first, addressed in comments, is that from the perspective of your worker container, your rabbitmq service is available on host broker, so your CELERY_BROKER_URL variable should point at amqp://user:password#broker:5672.
The second issue is that rabbitmq takes a while to start up. This means you'll initially see errors from celery worker when it first starts up, because it will attempt to connect before rabbitmq is ready. However, it will retry until it is successful.
I was able to get everything to work correctly with the following docker-compose.yml:
version: "3.3"
services:
broker:
image: rabbitmq:3-management-alpine
hostname: rabbit1
environment: &env
RABBITMQ_ERLANG_COOKIE: "SWQOKODSQALRPCLNMEQG"
RABBITMQ_DEFAULT_USER: "user"
RABBITMQ_DEFAULT_PASS: "password"
RABBITMQ_DEFAULT_VHOST: "/"
CELERY_BROKER_URL: "amqp://user:password#broker:5672"
ports:
- "5672:5672"
- "15672:15672"
expose:
- "5672"
volumes:
- .:/home/user
worker:
image: celery
command: ["celery", "worker", "-l" "INFO"]
environment:
<<: *env
volumes:
- .:/home/user
That's very similar to yours, but (a) I'm using the official celery image, since you didn't show us your Dockerfile, and (b) I've removed the links: key, because links has been deprecated in favor of the DNS-based service discovery that is now the default. Oh, and (c) I got rid of the custom network, since if you're creating a single network you might as well just use the default that docker-compose creates for you.
When I bring this up, I eventually see:
broker_1 | 2020-05-11 21:46:10.098 [info] <0.634.0> connection <0.634.0> (172.26.0.3:39372 -> 172.26.0.2:5672): user 'user' authenticated and granted access to vhost '/'
broker_1 | 2020-05-11 21:46:10.111 [info] <0.639.0> accepting AMQP connection <0.639.0> (172.26.0.3:39374 -> 172.26.0.2:5672)
worker_1 | [2020-05-11 21:46:10,100: INFO/MainProcess] Connected to amqp://user:**#broker:5672//
worker_1 | [2020-05-11 21:46:10,115: INFO/MainProcess] mingle: searching for neighbors
worker_1 | [2020-05-11 21:46:11,140: INFO/MainProcess] mingle: all alone