I'm running an Azure Web app (containers) with custom container images. I've followed the steps to enable ssh into a container image and it works great when I'm only running a single container. But when I run the app as a multi-container app (with docker-compose file) with more than one container image I get the error below. For additional context this is a small python web app that using nginx and redis hence the need for more than one container. Only one of my custom images has ssh enabled and running and exposing port 2222.
Is this even possible? If not then I'm not sure how feasible it is to run a web app multi-container if I have no way to access a container for support purposes.
az webapp remote-connection create -g GROUPNAME -n APPNAME -p 2222 --verbose
Configured default 'GROUPNAME' for arg resource_group_name
remote-connection is deprecated and moving to cli-core, use `webapp create-remote-connection`
Port 2222 is open
Creating a socket on port: 2222
Setting socket options
Binding to socket on local address and port
Finished initialization
Status response message: FAILURE:2222:Unable to connect to WebApp
WARNING - Remote debugging may not be setup properly. Reponse content: FAILURE:2222:Unable to connect to WebApp
SSH is available { username: root, password: Docker! }
Start your favorite client and connect to port 2222
I also tried the create-remote-connection command but got similar results.
az webapp create-remote-connection -n APPNAME -g GROUPNAME --verbose &
Error I receive is:
Auto-selecting port: 52661
Finished initialization
Status response message: FAILURE:2222:Unable to connect to WebApp
WARNING - Remote debugging may not be setup properly. Reponse content: FAILURE:2222:Unable to connect to WebApp
Connection is not ready yet, please wait
.
Status response message: FAILURE:2222:Unable to connect to WebApp
WARNING - Remote debugging may not be setup properly. Reponse content: FAILURE:2222:Unable to connect to WebApp
Looks like its not supported :(
see...
How to SSH in to different containers in Multi Container Azure App Service
and...
Support SSH to specific container in multi-container setup
Related
I am trying to run Julia code in azure container app, but it fails for some reason.
docker file is pretty simple:
FROM julia:1.7-bullseye
COPY . /
EXPOSE 8080
RUN julia --project='/' -e 'using Pkg; Pkg.build()'
ENTRYPOINT julia --project='/' main.jl "0.0.0.0" "8080"
"0.0.0.0" and "8080" there is the host and port on which HTTP is served:
HTTP.serve(GENERAL_ROUTER, host, port_restful_api)
The app supposed to run a simple HTTP server that is served over port 8080, GET to / is the health check that returns simple "Up"
When I run that same image locally with docker run -p 8080:8080 my-test:latest it works fine and hitting localhost:8080 gives me my expected response
But when I have it in container app - revision fails to be provisioned (and no extra errors provided). In the ingress I added availability to accept traffic from anywhere and target port is 8080...
What I am missing?
UPDATE
At the same time in log analytics I can see my output to console that should indicate that the app is supposedly running fine, e.g.
[ Info: API handler service: Running on thread 1
In Julia those are done with
#info "API Server: Running on thread $(Threads.threadid())"
But in logs those are in stderr stream. Could it be that the ACA treats any output to stderr as startup error?
UPDATE 2:
The very same code works without any issues or changes on Azure Container Instance and in Azure Kubernetes Service...
So for now I stopped investigating and decided to use ACI.
You should verify if any process inside the julia image needs root privileges as ACA do not support running privileged containers based on the docs:
Azure Container Apps can't run privileged containers. If your program
attempts to run a process that requires root access, the application
inside the container experiences a runtime error.
Did you specify 8080 as your target port when configuring the HTTP ingress of your Container App? https://learn.microsoft.com/en-us/azure/container-apps/ingress?tabs=bash
The platform will try to ping your app at the specified port due to the default health probes. If it receives no response, it will restart your app: https://learn.microsoft.com/en-us/azure/container-apps/health-probes?tabs=arm-template
The same Azure Redis cache is getting connected from local machince.
The port 6380, on which cache is running is open in firewall of both inbound and outbound in the VM.
I tried in both NodeJs and Java. Both are connecting to remote Azure Redis from local and the exact same code for NodeJS and Java is not connceting to Azure Redis cache from VM.
Java config:
spring.redis.host=my-cache.redis.cache.windows.net
spring.redis.password=<password>
spring.redis.port=6380
spring.redis.ssl=true
NodeJS config:
const client = redis.createClient(6380,
'my-cache.redis.cache.windows.net',
{
auth_pass: <password>,
tls: { servername: 'my-cache.redis.cache.windows.net' }
});
well, the other end must accept connection as well, so you mist allow connections from the VM if you have any firewall rules at all:
https://learn.microsoft.com/en-us/azure/azure-cache-for-redis/cache-configure#firewall
This is solved by the following on my Windows Desktop VM:
Click Start.
Enter cmd in the Start menu search text box.
Right-click Command Prompt and select Run as Administrator.
Run the following command: ipconfig /flushdns
Run the following command: ipconfig /registerdns
I'm running into an annoying issue where my containerized app runs fine locally, but fails when deployed to azure, despite the ports being exposed properly.
The details:
azure web app for container with Linux OS on a Basic B1 machine.
nodejs v12 LTS with express server listening on HTTPS port 443. Includes an HTTP 404 (Not Found) error handler.
database db Mongodb Atlas on M0 Sandbox (free tier). Local and Azure IPs whitelisted.
docker container using Dockerfile. no Kubernetes, no yml config file.
docker image hosted on private Docker Hub repo, pulled by azure on build
Running locally on Windows 10 x64:
without docker:
node app.js
runs fine via https://localhost:443
with docker:
docker build -t myapp:latest . then
docker run -p 49160:443 -d myapp:latest
runs fine via https://localhost:49160
In both cases I get the expected output from my nodejs app:
Running in production on x64
Express HTTPS server listening on port 443
mongo :: connected to database mydb at mydb-tot7b.azure.mongodb.net
(obvously, myapp and mydb are not the actual names. Used here for convenience.)
Running in azure:
Under Configuration -> Application settings:
PORT=443
WEBSITES_PORT=443
DOCKER_ENABLE_CI=true
Pipeline set up pulling docker image myrepo/myapp:latest from private Docker Hub repo on build. Image is pulled correctly.
Fails via https://myapp.azurewebsites.net/ with error 503 Service Temporarily Unavailable.
Azure log:
2020-04-10 18:02:21.040 INFO - Pulling image: myrepo/myapp:latest
2020-04-10 18:02:22.128 INFO - latest Pulling from myrepo/myapp
2020-04-10 18:02:22.129 INFO - Digest: sha256:982[...]
2020-04-10 18:02:22.129 INFO - Status: Image is up to date for myrepo/myapp:latest
2020-04-10 18:02:22.131 INFO - Pull Image successful, Time taken: 0 Minutes and 1 Seconds
2020-04-10 18:02:22.143 INFO - Starting container for site
2020-04-10 18:02:22.144 INFO - docker run -d -p 7909:443 --name myapp_0_138f197c -e PORT=80 -e WEBSITES_ENABLE_APP_SERVICE_STORAGE=false -e WEBSITES_PORT=443 -e WEBSITE_SITE_NAME=myapp -e WEBSITE_AUTH_ENABLED=False -e WEBSITE_ROLE_INSTANCE_ID=0 -e WEBSITE_HOSTNAME=myapp.azurewebsites.net -e WEBSITE_INSTANCE_ID=6fc3[...] myrepo/myapp:latest
2020-04-10 18:02:22.144 INFO - Logging is not enabled for this container.Please use https://aka.ms/linux-diagnostics to enable logging to see container logs here.
2020-04-10 18:02:23.473 INFO - Initiating warmup request to container myapp_0_138f197c for site myapp
2020-04-10 18:02:45.304 INFO - Waiting for response to warmup request for container myapp_0_138f197c. Elapsed time = 21.8308149 sec
[...]
2020-04-10 18:06:08.252 INFO - Waiting for response to warmup request for container myapp_0_138f197c. Elapsed time = 224.7791547 sec
2020-04-10T18:02:24.893958779Z Running in production on x64
2020-04-10T18:02:26.275376640Z Express HTTPS server listening on port 80
2020-04-10T18:02:26.601261213Z mongo :: connected to database mydb at mydb-tot7b.azure.mongodb.net
2020-04-10 18:06:14.346 ERROR - Container myapp_0_138f197c for site myapp did not start within expected time limit. Elapsed time = 230.8732099 sec
2020-04-10 18:06:14.365 ERROR - Container myapp_0_138f197c didn't respond to HTTP pings on port: 443, failing site start. See container logs for debugging.
2020-04-10 18:06:14.427 INFO - Stoping site myapp because it failed during startup.
Why the containerized app seems to start correctly, as seen in the azure log, but the container fails to respond to HTTP ping on 443, which is exposed both in dockerfile and in azure with PORT and WEBSITES_PORT?
Do I need to make my HTTPS server explicitly respond to HTTP PING and if so, how do I do it?
Dockerfile:
FROM node:12
WORKDIR /usr/src/myapp
COPY package*.json ./
ARG NODE_ENV=production
ENV NODE_ENV=${NODE_ENV}
RUN npm install
COPY . .
EXPOSE 443
CMD [ "node", "app.js" ]
This issue might be considered a duplicate, but I provide a lot more relevant information, compared to most of the issues below. Also, I spent significant time in troubleshooting and none of these relevant resources seemed to help:
Docker never runs on Azure - Waiting for response to warmup request for container
Azure Web App on Linux: "Error: Container didn't respond to HTTP pings on port: 8080" - when using: "start": "pm2 start server.js"
How do you expose port 3000 using an Azure Web App Container?
https://superuser.com/questions/1479521/i-cannot-create-a-node-js-web-app-to-be-deployed-on-azurer
https://learn.microsoft.com/en-us/archive/blogs/waws/things-you-should-know-web-apps-and-linux#troubleshootingr
https://learn.microsoft.com/en-us/azure/app-service/containers/app-service-linux-intro#troubleshootingr
https://learn.microsoft.com/en-us/azure/app-service/containers/configure-language-nodejsr
https://github.com/MicrosoftDocs/azure-docs/issues/46401r
https://github.com/MicrosoftDocs/azure-docs/issues/34451r
https://omgdebugging.com/2017/12/22/azure-web-app-for-container-failing-site-start/r
https://devops.stackexchange.com/questions/4543/how-to-debug-a-docker-on-web-app-for-containers-in-azure-which-does-not-startr
https://learn.microsoft.com/en-us/azure/app-service/containers/app-service-linux-faq#custom-containersr
Running a docker container in Azure web app: didn't respond to HTTP pings on port
Docker Container fails to start in an Azure App Service
https://learn.microsoft.com/en-us/azure/container-instances/container-instances-troubleshootingr
https://serverfault.com/questions/1003418/azure-docker-app-error-site-did-not-start-within-expected-time-limit-and-cor
https://medium.com/#sergiibielskyi/azure-event-hub-listener-in-the-container-dd6f8d053814r
When enforcing HTTPS at the App Service level, TLS termination will happen and traffic will be routed to your container port so there's no need to enforce TLS from your container.
If your container listens on another port (ex: 8081), simply set the WEBSITES_PORT application setting to that port number. App Service will listen on port 80 and forward traffic to your container port.
My own app had been doing app.UseHttpsRedirection() in Startup.Configure. That was the cause of this error for me. As soon as I stopped doing that, the problem went away.
I think I am also facing similar issue and in my case I can't changed the source code for application to run on http.
We have a very basic requirement, wherein there is an API build on ASP.NET Core running on specific port (say 11000), and served only on HTTPS, i.e http traffic is disabled, and the API is rendered on custom domain, (on local machine it is achieved through host file entries).
Working on POC to deploy this API on Docker container on Azure App service. Running this setup on local system and it works perfectly fine.
Steps followed.
Compose build
Create Image
Tag this Image
Push this image to Azure Container Register
Set the correct values in Deployment Center
Add the Application Settings for WEBSITES_PORT and set its value to the port, 11000.
While starting the container in Azure it is throwing error as
28/06/2022 18:40:04.370 INFO - Site: XXXXXXXXXXXXXX-webapp - [
some-alpha-numeric-identifier] - Waiting for container to be ready.
Container Id: some-alpha-numeric-identifier
And the I know the reason for the error, while starting the container is connecting to http://127.0.0.1:11000 and the base application only listens on https, I need to know how to overcome this error, can we for the app service to connect to the base docker image only over https.
I create an image from cf ic command line expose 4000 then install nodejs,npm via apt-get and deploy a nodejs app on the image listen on port 4000.
And then I create a container with this image ,assign a public_ip to this container and run it.
But I found that I can not access the nodejs app with port [ http://public_ip4000 ].
When I login into container with command line cf ic exec -it container_id bash,I found that the nodejs app is running and I can access nodejs app by curl -GET http://localhost:4000/
Error Message is :net::ERR_CONNECTION_TIMED_OUT
Q: How can I access my nodejs app outside container?
Port 4000 is not exposed to the IBM Containers firewall.
There is a limited set of ports exposed, so I suggest you try a different port like 3000 or 5000. The complete list is not published for security reasons.
Alternatively you can create a Container Group with a single container. In that case you can define a route (domain) for your container that will automatically route all requests internally to your container port (like 4000).
You can find more details about creating container groups in Bluemix documentation:
https://console.ng.bluemix.net/docs/containers/container_creating_ov.html#container_group_ov
I'm trying to move some existing servers to be housed within docker containers. I have two: an app server and an api server but developed with node.js. I have them both working within an ubuntu vm and can hit both apps from outside the vm which is great.
Each server has it's own domain. The app server uses the domain app and the api server uses the api domain, clever I know. Locally I added both domains to my hosts file to point to the ip assigned to the ubuntu vm.
The only issue I'm having is there is a request sent from the app server that needs to be routed to the api server. Tried editing the hosts file of both the app server container (via the Dockerfile) and the ubuntu vm however the request fails.
Is there a simple way to get that request to not go out and try to resolve the api domain but get it to point to the api container?
A typical solution to this would be to use Docker's --link option to link the containers. That is, if you do:
docker run -d --name api myapi
docker run -d --name app --link api:api myapp
Then within the app container, the hostname api will map to the api container. You will also have a set of environment variables available that describe the exposed ports on the linked container. E.g., if your "api" container exposed port 80, the variable would look like:
API_PORT_80_TCP=tcp://172.17.0.10:80
API_PORT_80_TCP_PORT=80
API_PORT_80_TCP_PROTO=tcp
API_PORT=tcp://172.17.0.10:80
API_NAME=/app/api
API_PORT_80_TCP_ADDR=172.17.0.10
There are some disadvantages to the link option:
This only works for containers hosted on the same physical host
If you restart the "api" container, you have to restart the "app" container, too.
Both of these particular problems can probably be resolved by the orchestration tool of your choice if you are operating in a multi-host environment.
The linking feature (--link) is a legacy feature.
You should always prefer using Docker network drivers over linking.
Example: running a Redis container with Redis binding to localhost then running the redis-cli command and connecting to the Redis server over the localhost interface.
$ docker run -d --name redis example/redis --bind 127.0.0.1
$ # use the redis container's network stack to access localhost
$ docker run --rm -it --network container:redis example/redis-cli -h 127.0.0.1
See the docs for details.
https://docs.docker.com/compose/link-env-deprecated/
https://docs.docker.com/engine/reference/run/#network-settings