I have an azure function, I am using azure container registry image for the deployment but getting Azure runtime is unreachable. I have checked the docker logs from Kudu, it is throwing -
Pull Image successful, Time taken: 0 Minutes and 1 Seconds
2022-12-13T07:40:49.911Z INFO - Starting container for site
2022-12-13T07:40:49.912Z INFO - docker run -d --expose=80 --name <function_app>_0_e2779bae -e DOCKER_CUSTOM_IMAGE_NAME=/: -e WEBSITES_ENABLE_APP_SERVICE_STORAGE=false -e WEBSITE_SITE_NAME=<function_app> -e WEBSITE_AUTH_ENABLED=False -e PORT=80 -e WEBSITE_ROLE_INSTANCE_ID=0 -e WEBSITE_HOSTNAME=<function_app>.azurewebsites.net -e WEBSITE_INSTANCE_ID= -e WEBSITE_USE_DIAGNOSTIC_SERVER=False <image_path>
2022-12-13T07:40:49.913Z INFO - Logging is not enabled for this container.
Please use https://aka.ms/linux-diagnostics to enable logging to see container logs here.
2022-12-13T07:40:50.683Z INFO - Initiating warmup request to container <function_app>_0_e2779bae_msiProxy for site <function_app>
2022-12-13T07:40:50.692Z INFO - Container <function_app>_0_e2779bae_msiProxy for site <function_app> initialized successfully and is ready to serve requests.
2022-12-13T07:40:50.693Z INFO - Initiating warmup request to container <function_app>_0_e2779bae for site <function_app>
2022-12-13T07:40:50.706Z ERROR - Container <function_app>_0_e2779bae for site <function_app> has exited, failing site start
2022-12-13T07:40:50.712Z ERROR - Container <function_app>_0_e2779bae didn't respond to HTTP pings on port: 80, failing site start. See container logs for debugging.
2022-12-13T07:40:50.718Z INFO - Stopping site <function_app> because it failed during startup.
I have added docker url, username and password attributes in the app-settings and are correct. Can someone please help me out with the same?
Container <function_app>_0_e2779bae didn’t respond to HTTP pings on port: 80, failing site start.
There could be several reasons for the above error such as:
Allow the Port 8080 or 80 by adding the app setting as WEBSITES_PORT .
If it is the custom container, you have to give certain port number for routing the requests.
In Docker File, expose the above-mentioned port.
Refer to this MS Q&A Issue No 750525 regarding the similar error on Azure Functions Container Context and also many similar issues reported on GitHub Repositories such as 46401, 34451 for the solutions given by MSFT Azure Team.
I have tried several ways but got to know the reason at last.
The issue was with the dockerfile, but the error logs were not showing the actual cause.
Previously in the dockerfile we were using entrypoint and the code was not getting published to wwwroot
WORKDIR /app
COPY --from=publish /app/publish .
ENTRYPOINT ["dotnet", "someapplication.dll"]
and then we modified the file to publish it to wwwroot
WORKDIR /home/site/wwwroot
COPY --from=publish /app/publish .
ENV AzureWebJobsScriptRoot=/home/site/wwwroot
This resolved my problem.
Related
I built and deployed a docker image to ACR and App service. I exposed the port in the docker file and set WEBSITES_PORT in the application settings. But I kept getting "didn't respond to HTTP pings" error.
In my Dockerfile:
EXPOSE 6000
WORKDIR /webapp/
ENTRYPOINT ["python", "server/main.py"]
Here are the logs where site startup failed:
2022-12-13T19:44:04.647Z INFO - Waiting for response to warmup request for container xxxx. Elapsed time = 1651.7761754 sec
2022-12-13T19:44:20.040Z INFO - Waiting for response to warmup request for container xxxx. Elapsed time = 1667.1697173 sec
2022-12-13T19:44:22.963Z ERROR - Container xxxx for site yyyy is not running, failing site start
2022-12-13T19:44:22.984Z ERROR - Container xxxx didn't respond to HTTP pings on port: 6000, failing site start. See container logs for debugging.
2022-12-13T19:44:23.002Z INFO - Stopping site yyyy because it failed during startup.
/home/LogFiles/2022_12_14_ln1mdlwk000058_default_docker.log (https://yyyy.scm.azurewebsites.net/api/vfs/LogFiles/2022_12_14_aaaa_default_docker.log)
2022-12-14T05:32:00.645431054Z [2022-12-14 05:32:00 +0000] [1] [INFO] Running on http://127.0.0.1:6000 (CTRL + C to quit)
2022-12-14T05:50:20.981722886Z [2022-12-14 05:50:20 +0000] [1] [INFO] Running on http://127.0.0.1:6000 (CTRL + C to quit)
Here are the logs where docker runs:
2022-12-14T06:33:02.853Z INFO - Pulling image: cccc.azurecr.io/bbbb:tag
2022-12-14T06:33:03.642Z INFO - 94 Pulling from bbbb
2022-12-14T06:33:03.643Z INFO - Digest: sha256:123456
2022-12-14T06:33:03.643Z INFO - Status: Image is up to date for cccc.azurecr.io/bbbb:tag
2022-12-14T06:33:03.646Z INFO - Pull Image successful, Time taken: 0 Minutes and 0 Seconds
2022-12-14T06:33:03.656Z INFO - Starting container for site
2022-12-14T06:33:03.657Z INFO - docker run -d --expose=6000
--name xxxx_0_d7bfb097
-e DOCKER_CUSTOM_IMAGE_NAME=cccc.azurecr.io/bbbb:tag
-e PORT=6000
-e WEBSITES_ENABLE_APP_SERVICE_STORAGE=false
-e WEBSITES_PORT=6000
-e WEBSITE_SITE_NAME=xxxx
-e WEBSITE_AUTH_ENABLED=False
-e WEBSITE_ROLE_INSTANCE_ID=0
-e WEBSITE_HOSTNAME=xxxx.azurewebsites.net
-e WEBSITE_INSTANCE_ID=7210fbf7a062fd0ff739205809c9182b72493680f0a70d46e83abcbde795374a
-e HTTP_LOGGING_ENABLED=1
-e NODE_OPTIONS=--require /agents/node/build/src/Loader.js
-e JAVA_TOOL_OPTIONS=-javaagent:/agents/java/applicationinsights-agent-codeless.jar
-e StartupBootstrapper=Microsoft.ApplicationInsights.StartupBootstrapper
-e DOTNET_SHARED_STORE=/agents/core/store/lin
-e DOTNET_ADDITIONAL_DEPS=/agents/core/additionalDeps
-e WEBSITE_USE_DIAGNOSTIC_SERVER=False
cccc.azurecr.io/bbbb:tag
Btw I'm not sure where the variables like NODE_OPTIONS, StartupBootstrapper, and DOTNET_SHARED_STORE come from...
I have gone through the solutions on these posts:
Docker never runs on Azure - Waiting for response to warmup request for container
[Bug] Azure App Service Container exposes and pings different ports This post does not -p flag in docker run and it still works.
Azure web app for container - failed during startup - didn't respond to HTTP pings.
My understanding: it looks like the app already runs in the docker, since it has Running on http://127.0.0.1:6000. But the port it's not published to the host machine. The docker run command only has --expose flag, no -p. That's why it keeps telling me Container xxxx didn't respond to HTTP pings on port. I don’t know why it picked up the exposed port but not published it.
I have tried:
set PORT = 6000 only
set WEBSITES_PORT = 6000 only
set PORT = 6000 + WEBSITES_PORT = 6000
upgrade app service plan
SSH into app service. It shows SSH CONN CLOSE
But none of them works.
Since I can’t edit docer run command, what can I do to fix this?
Is my understanding of the current situation wrong?
Any help is appreciated. Thank you~
Figured it out! I should set the host to 0.0.0.0, instead of using the default one 127.0.0.0.
Another observation: after I deployed the image to app service, it will automatically add an environment variable PORT with the value of EXPOSE in Dockerfile. Without defining WEBSITES_PORT, it somehow works with this command docker run --expose=6000 -PORT=6000.
I've set up an outlook add-in via the yeoman office add-in generator as detailed in the tutorial here: https://learn.microsoft.com/en-us/outlook/add-ins/addin-tutorial
This is essentially a Node app which utilizes webpack. I am attempting to host this app on Azure App Service, however every time I deploy the app, I get the error shown below.
The app is set to run out of port 3000, which I have exposed in the app settings. However still no luck. Any thoughts on what the issue might be?
2019-04-15 04:26:53.591 ERROR - Container testaddin43984_0 for
site testaddin43984 did not start within expected time limit.
Elapsed time = 400.2035387 sec
2019-04-15 04:26:53.611 ERROR - Container testaddin43984_0 didn't
respond to HTTP pings on port: 8080, failing site start. See
container logs for debugging.
2019-04-15 04:35:39.506 INFO - Starting container for site
2019-04-15 04:35:39.507 INFO - docker run -d -p 31557:8080 --name
testaddin43984_0 -e WEBSITE_NODE_DEFAULT_VERSION=10.14 -e
APPSETTING_WEBSITE_NODE_DEFAULT_VERSION=10.14 -e
WEBSITES_PORT=3000 -e WEBSITE_SITE_NAME=testaddin43984 -e
WEBSITE_AUTH_ENABLED=False -e WEBSITE_ROLE_INSTANCE_ID=0 -e WEBSITE_INSTANCE_ID=801899fe3bb51451491b9664dc3eb742fd693df437561cc840139a9f5c7dc0bb -e HTTP_LOGGING_ENABLED=1 appsvc/node:10.14_1903081107
As the error states, the issue is with the start script. The script is taking more time and it eventually times-out.
2019-04-15 04:26:53.591 ERROR - Container testaddin43984_0 for
site testaddin43984 did not start within expected time limit.
Try to reduce the time taken by the npm start script in your project.
I use concurrently to solve these kind of problems
This is what happens to run the container. I wonder if there is a way to start the web app for containers with a custom DNS.
I have 5 microservices in my ILB-ASE
they need to be able to call each other using my custom DNS server in the VNet. When I check the resolv.conf i see 127.0.0.11. I need that to be set to my own custom dns server.
how can we inject my custom DNS value here?
Should we use the appsettings if so what are the values in the web app for containers?
So I can use the --dns option
The mystery part that Azure runs it. Some values are coming up from the appsettings.
2018-08-23 14:12:56.100 INFO - docker run -d -p 13940:5001 --name xxx
-e DOCKER_CUSTOM_IMAGE_NAME=xxx.azurecr.io/xxx:558 -e WEBSITES_ENABLE_APP_SERVICE_STORAGE=false -e WEBSITES_PORT=5001 -e
WEBSITE_SITE_NAME=xxx -e WEBSITE_AUTH_ENABLED=False -e
WEBSITE_ROLE_INSTANCE_ID=0 -e WEBSITE_INSTANCE_ID=xxx -e
HTTP_LOGGING_ENABLED=1 xxx.azurecr.io/xxx:558
=====DOCKER LOG=========
2018_08_23_RD0003FF2D0408_default_docker.log:
2018-08-23T14:12:49.755843301Z [40m[1m[33mwarn[39m[22m[49m:
Microsoft.AspNetCore.DataProtection.KeyManagement.XmlKeyManager[35]
2018-08-23T14:12:49.755897801Z No XML encryptor configured. Key
{xxx-xxx-xxx-xxx-xxx} may be persisted to storage in unencrypted form.
2018-08-23T14:12:54.761216323Z [40m[1m[33mwarn[39m[22m[49m:
Microsoft.AspNetCore.Server.Kestrel[0]
2018-08-23T14:12:54.761251623Z Overriding address(es) 'http://+:80'.
Binding to endpoints defined in UseKestrel() instead.
2018-08-23T14:12:54.908189021Z Hosting environment: Production
2018-08-23T14:12:54.908386123Z Content root path: /app
2018-08-23T14:12:54.908961927Z Now listening on: http://0.0.0.0:5001
2018-08-23T14:12:54.909256229Z Application started. Press Ctrl+C to
shut down.
2018_08_23_RD0003FF2D0408_docker.log:
2018-08-23 14:12:44.125 INFO - Recycling container because of
AppFrameworkVersionChange and appFrameworkVersion = xxx.xxx.io/xxx:558
2018-08-23 14:12:45.900 INFO - Starting container for site
2018-08-23 14:12:45.900 INFO - docker run -d -p 30464:5001 --name xxx
-e DOCKER_CUSTOM_IMAGE_NAME=xxx.azurecr.io/xxx:549 -e WEBSITES_ENABLE_APP_SERVICE_STORAGE=false -e WEBSITES_PORT=5001 -e
WEBSITE_SITE_NAME=xxx -e WEBSITE_AUTH_ENABLED=False -e
WEBSITE_ROLE_INSTANCE_ID=0 -e WEBSITE_INSTANCE_ID=xxx -e
HTTP_LOGGING_ENABLED=1 xxx.xxx.io/xxx:558
2018-08-23 14:12:55.972 INFO - Container xxx for site xxx initialized
successfully.
2018-08-23 14:12:55.976 INFO - Recycling container because of
AppSettingsChange and isMainSite = True
2018-08-23 14:12:56.099 INFO - Starting container for site
2018-08-23 14:12:56.100 INFO - docker run -d -p 13940:5001 --name xxx
-e DOCKER_CUSTOM_IMAGE_NAME=xxx.azurecr.io/xxx:558 -e WEBSITES_ENABLE_APP_SERVICE_STORAGE=false -e WEBSITES_PORT=5001 -e
WEBSITE_SITE_NAME=xxx -e WEBSITE_AUTH_ENABLED=False -e
WEBSITE_ROLE_INSTANCE_ID=0 -e WEBSITE_INSTANCE_ID=xxx -e
HTTP_LOGGING_ENABLED=1 xxx.azurecr.io/xxx:558
2018-08-23 14:13:05.385 INFO - Container xxx for site xxx initialized
successfully.
we responded to your question on Github and Reddit. Re-posting our response here for visibility.
"Currently, there is a workaround for this: you should modify the default resolv.conf to the custom DNS IP and then add your custom resolv.conf on docker build by adding a COPY command in your entrypoint script and pointing a custom resolv.conf to /etc.
However, we are investigating a better solution for this, so that manually updating the resolv.conf wouldn’t be necessary, so stay tuned."
You shouldn't use DNS to communicate with microservices, instead, you should make use of service registry.
Check this Microsoft paper talking about this:
Each microservice has a unique name (URL) that is used to resolve its
location. Your microservice needs to be addressable wherever it is
running. If you have to think about which computer is running a
particular microservice, things can go bad quickly. In the same way
that DNS resolves a URL to a particular computer, your microservice
needs to have a unique name so that its current location is
discoverable. Microservices need addressable names that make them
independent from the infrastructure that they are running on. This
implies that there is an interaction between how your service is
deployed and how it is discovered, because there needs to be a service
registry. In the same vein, when a computer fails, the registry
service must be able to indicate where the service is now running.
As you can see, the best solution will depend on your deployment model. Check this note about containers:
In some microservice deployment environments (called clusters, to be
covered in a later section), service discovery is built-in. For
example, within an Azure Container Service environment, Kubernetes and
DC/OS with Marathon can handle service instance registration and
deregistration. They also run a proxy on each cluster host that plays
the role of server-side discovery router. Another example is Azure
Service Fabric, which also provides a service registry through its
out-of-the-box Naming Service.
Hope it helps!
I have a 3rd party Tomcat server image/container running in LinuxVM on Azure. The LinxVM actually started as 2 images(NGINX loadbalancer) running via a docker-compose script, but to test this on a webapp I've wittled down to just the single tomcat image. Now, the compose script uses the key:
ports:
- 80:8090
- 8445:8443
In the VM I can run the docker-compose script and hit http://mypage:80 and it works just fine. I can also run:
docker run -d -p <somePort>:8090 --name tomcat_1 <myrepo/myimage>
I can then access my site with http://mypage:<somePort> regardless of which port I want to map to the container. This all works great.
Now, with the Azure Web App, I'm using an Azure Web App for Containers --> Docker Compose (Preview). My compose script looks something like:
version: "3.0"
services:
pdfd-tomcat:
image: <myrepo/myimage>
build:
context: .
args:
INCLUDE_DEMO: 'true'
LIBRE_SUPPORT: 'false'
HTML_SUPPORT: 'false'
container_name: Blackbox
environment:
TRN_PDFNET_KEY:
TRN_DB_TYPE: SQLITE
TRN_DB_LINK:
TRN_BALANCER_COOKIE_NAME: HAPROXID
TRN_DEBUG_MODE: 'false'
ports:
- 80:8090
- 8445:8443
I've exposed 80:8090 because I've read that Azure Web Apps only expose port 80 and 443 by default. However, I cannot access this site from any port once the web app is spun up. I've verified running this same compose script works in a VM. Now, when I hit the web app logs, I see this:
Status: Image is up to date for <myrepo/myimage>
2018-06-17 05:38:41.298 INFO - Starting container for site
2018-06-17 05:38:41.298 INFO - docker run -d -p 18455:8090 --name tomcat_1 -e WEBSITE_SITE_NAME=<mywebsite> -e WEBSITE_AUTH_ENABLED=False -e WEBSITE_ROLE_INSTANCE_ID=0 -e WEBSITE_INSTANCE_ID=<stuff goes here>
2018-06-17 05:38:41.298 INFO - Logging is not enabled for this container.
Please use https://aka.ms/linux-diagnostics to enable logging to see container logs here.
2018-06-17 05:38:56.994 INFO - Started multi-container app
2018-06-17 05:38:57.008 INFO - Container tomcat_1 for site <mywebsite> initialized successfully.
So, it seems that it's trying to map external port: 18455 to my internal 8090 port. Why? Also, if I try to hit my site via that port, I can't. Each time the app deploys/restarts it maps a different external port.
Also, if I retroactively go to 'Application Settings' and use the key/value: WEBSITES_PORT:<current externally mapped port> it has literally no effect. Then if the app gets restarted/redeployed, I can see that WEBSITES_PORT:<port> is what the previous port was mapped with, but since that changes every deployment, the current external port and the WEBSITES_PORT values never match. I don't even know if it works to begin with.
How the heck do get this working in a deterministic manner? I can supply other material if needed.
This boiled down to a permissions issue when using Tomcat 9.0+ with Docker.
Permission problem while running tomcat as a non-root user
The Dockerfile would create a new usergroup and add a user, then give that user permissions in the folders where Tomcat existed. If you jumped into the container via docker exec /bin/bash and checked permissions, they all seemed perfectly fine. However, logs would show that Tomcat couldn't gain access to those folders.
Once I implemented the fix as described by Empinator in the link everything worked (using root also worked).
I have a Linux App Service deployed within Azure, using an image from a private Azure Container Registry.
The service was up and running fine, and despite there being no changes to the image (no new pushes), the App Service was redeployed. I understand this to be an expected part of how Azure handles containers, swapping instances in and out with as it sees fit.
However, this image is no longer able to deploy successfully, here are the logs from Kudu (names changed):
2018-04-09 07:28:11.817 INFO - Starting container for site
2018-04-09 07:28:11.818 INFO - docker run -d -p 36165:8080 --name my-service_0 -e WEBSITES_ENABLE_APP_SERVICE_STORAGE=false -e WEBSITES_PORT=8080 -e WEBSITE_SITE_NAME=my-service -e WEBSITE_AUTH_ENABLED=False -e WEBSITE_ROLE_INSTANCE_ID=0 -e WEBSITE_INSTANCE_ID=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX -e HTTP_LOGGING_ENABLED=1 myacr.azurecr.io/images/my-image:latest
2018-04-09 07:28:14.200 INFO - Issuing docker pull myacr.azurecr.io/images/my-image:latest
2018-04-09 07:28:14.327 ERROR - docker pull returned STDERR>> Error response from daemon: Get https://myacr.azurecr.io/v2/images/my-image/manifests/latest: unauthorized: authentication required
Here are the results of pulling the image locally:
docker pull myacr.azurecr.io/images/my-image:latest
latest: Pulling from images/my-image
53478ce18e19: Already exists
d1c225ed7c34: Already exists
c6724ba0c09a: Already exists
377ae380bb14: Pulling fs layer
ab6e212b3598: Pulling fs layer
377ae380bb14: Downloading
ab6e212b3598: Verifying Checksum
ab6e212b3598: Download complete
unknown blob
I'm fairly sure that the image was successfully deployed 3 days ago and that there has been no pushes to the tag since then.
Is there anything I can do to recover this image?
Is there any way I could have known this deployment had started failing?
It seems your Web App for Containers instance is unable to authenticate with Azure Container Registry.
The Azure Container Registry to Linux Web App connection is a bit problematic at times. Using the "private registry" tab on the container settings page instead of ACR usually solves the problem for me.
Just provide your ACR repo credentials as below instead of selecting your repo on the ACR tab.