App Service Linux: Error in get or Add Endpoint: pod.fixupIpTables - azure-web-app-service

I have this error since a few days, someone else happens?, I need some help to recover the app service.
2022-12-17T09:58:05.475Z INFO - Status: Downloaded newer image for 10.0.3.138:13209/appsvc/php:8.1-fpm_20221208.1.tuxprod
2022-12-17T09:58:05.479Z INFO - Pull Image successful, Time taken: 3 Minutes and 7 Seconds
2022-12-17T09:58:05.520Z INFO - Starting container for site
2022-12-17T09:58:05.520Z INFO - docker run -d --expose=8080 --name mysite_1_d62e790c -e WEBSITE_SITE_NAME=mysite -e WEBSITE_AUTH_ENABLED=False -e PORT=8080 -e WEBSITE_ROLE_INSTANCE_ID=0 -e WEBSITE_HOSTNAME=mysite.azurewebsites.net -e WEBSITE_INSTANCE_ID=67e61faf16b7de2d5d297e6f2795d58b970924c4839c107793fe67be65ec83bd -e WEBSITE_USE_DIAGNOSTIC_SERVER=False appsvc/php:8.1-fpm_20221208.1.tuxprod
2022-12-17T09:58:05.520Z INFO - Logging is not enabled for this container.
Please use https://aka.ms/linux-diagnostics to enable logging to see container logs here.
2022-12-17T10:00:35.871Z ERROR - Container create failed for mysite_1_d62e790c with System.Exception, Error in get or Add Endpoint: pod.fixupIpTables error in FixupIPTables: Failed to host iptable rules: Failed to populate the following iptables:
Err on iptables -t nat -A PREROUTING -m addrtype --dst-type LOCAL -j ANTARES : Failed to add the rule for iptables -t nat -A PREROUTING -m addrtype --dst-type LOCAL -j ANTARES: exit status 2
Err on iptables -t nat -I OUTPUT -m addrtype --dst-type LOCAL -j ANTARES : Failed to add the rule for iptables -t nat -I OUTPUT -m addrtype --dst-type LOCAL -j ANTARES: exit status 2

If you are experiencing problem with site, you may check the docker log, this is helpful in troubleshooting your site when it doesn't start.
Different ways to access Docker logs is below:
Docker logs appear on the Container Settings page in the portal.
You can find the Docker log in the /LogFiles directory. You can access this via the Kudu (Advanced Tools) Bash console or by using
an FTP client to access it.
You can use our API to download the current logs.
Investigate on if your container is throwing any exceptions, If the container starts but does not respond to a ping, this will eventually log an event in the Docker log saying that it didn't start. Then its suggested to increase the 230 second wait time up to a limit of 1800 seconds.
To configure that, add an app setting called WEBSITES_CONTAINER_START_TIME_LIMIT and set it to the number of seconds you would like for us to wait for your container to start.
You may refer to the blog Things You Should Know: Web Apps and Linux for more details on this.

I eventually created a new app service plan. I migrated my app service to that new app service plan and the problem is gone.

Related

Docker: Error starting userland proxy: Bind for 0.0.0.0:50000: unexpected error Permission denied on Azure VM

I'm new to Docker so please be kind but I am testing it out on a Windows 10 image on Azure (I know I could run it directly but I wanted to try it in a VM first).
I have a fresh Windows 10 image that I have installed Docker for Windows 2.0.0 on.
Note: I did not tick the option to use Windows containers instead of linux containers.
Once it installed (and rebooted) I was prompted to install Hyper-V and Containers features (causing restarts).
Once it was all installed I open an Administrative PowerShell window to download Jenkins:
docker run -p 8080:8080 -p 50000:50000 -v jenkins_home:/var/jenkins_home jenkins/jenkins:lts
This gave me the error:
C:\Program Files\Docker\Docker\Resources\bin\docker.exe: Error response from daemon: driver failed programming external connectivity on endpoint goofy_lederberg (deaba2deeea0486c92ba8a1a32740295f03859b1b5829d39e39eff0b24613ebf): Error starting userland proxy: Bind for 0.0.0.0:50000: unexpected error Permission denied.
I thought this was strange as 50000 wasn't a port that I expected to be in use, changing this to different ports (50001) produced the same error.
Running:
netstat -a -n -o
Showed that the port was not in use.
If I remove -p 50000:50000 from the command it can bind and start Jenkins but I assume it needs this port mapping to work correctly.
Previous posts have suggested stopping the World Wide Web Publishing service but that isn't installed.
There are no other running Docker containers.
I assume the port is in use or something is stopping the port mapping.
Assuming a user has permission to create a port binding from their terminal are there any other techniques beside netstat to determine if something is bound to a port - either something internal to docker's own checking process or something at the host OS level?
Rather embarrassingly this worked this morning with no changes other than the VM was shutdown over the weekend.
Maybe all it needed was a reboot?

How to set the dns option in Azure web app for containers

This is what happens to run the container. I wonder if there is a way to start the web app for containers with a custom DNS.
I have 5 microservices in my ILB-ASE
they need to be able to call each other using my custom DNS server in the VNet. When I check the resolv.conf i see 127.0.0.11. I need that to be set to my own custom dns server.
how can we inject my custom DNS value here?
Should we use the appsettings if so what are the values in the web app for containers?
So I can use the --dns option
The mystery part that Azure runs it. Some values are coming up from the appsettings.
2018-08-23 14:12:56.100 INFO - docker run -d -p 13940:5001 --name xxx
-e DOCKER_CUSTOM_IMAGE_NAME=xxx.azurecr.io/xxx:558 -e WEBSITES_ENABLE_APP_SERVICE_STORAGE=false -e WEBSITES_PORT=5001 -e
WEBSITE_SITE_NAME=xxx -e WEBSITE_AUTH_ENABLED=False -e
WEBSITE_ROLE_INSTANCE_ID=0 -e WEBSITE_INSTANCE_ID=xxx -e
HTTP_LOGGING_ENABLED=1 xxx.azurecr.io/xxx:558
=====DOCKER LOG=========
2018_08_23_RD0003FF2D0408_default_docker.log:
​
2018-08-23T14:12:49.755843301Z [40m[1m[33mwarn[39m[22m[49m:
Microsoft.AspNetCore.DataProtection.KeyManagement.XmlKeyManager[35]
2018-08-23T14:12:49.755897801Z No XML encryptor configured. Key
{xxx-xxx-xxx-xxx-xxx} may be persisted to storage in unencrypted form.
2018-08-23T14:12:54.761216323Z [40m[1m[33mwarn[39m[22m[49m:
Microsoft.AspNetCore.Server.Kestrel[0]
2018-08-23T14:12:54.761251623Z Overriding address(es) 'http://+:80'.
Binding to endpoints defined in UseKestrel() instead.
2018-08-23T14:12:54.908189021Z Hosting environment: Production
2018-08-23T14:12:54.908386123Z Content root path: /app
2018-08-23T14:12:54.908961927Z Now listening on: http://0.0.0.0:5001
2018-08-23T14:12:54.909256229Z Application started. Press Ctrl+C to
shut down.
​
2018_08_23_RD0003FF2D0408_docker.log:
2018-08-23 14:12:44.125 INFO - Recycling container because of
AppFrameworkVersionChange and appFrameworkVersion = xxx.xxx.io/xxx:558
2018-08-23 14:12:45.900 INFO - Starting container for site
2018-08-23 14:12:45.900 INFO - docker run -d -p 30464:5001 --name xxx
-e DOCKER_CUSTOM_IMAGE_NAME=xxx.azurecr.io/xxx:549 -e WEBSITES_ENABLE_APP_SERVICE_STORAGE=false -e WEBSITES_PORT=5001 -e
WEBSITE_SITE_NAME=xxx -e WEBSITE_AUTH_ENABLED=False -e
WEBSITE_ROLE_INSTANCE_ID=0 -e WEBSITE_INSTANCE_ID=xxx -e
HTTP_LOGGING_ENABLED=1 xxx.xxx.io/xxx:558
​
2018-08-23 14:12:55.972 INFO - Container xxx for site xxx initialized
successfully.
2018-08-23 14:12:55.976 INFO - Recycling container because of
AppSettingsChange and isMainSite = True
2018-08-23 14:12:56.099 INFO - Starting container for site
2018-08-23 14:12:56.100 INFO - docker run -d -p 13940:5001 --name xxx
-e DOCKER_CUSTOM_IMAGE_NAME=xxx.azurecr.io/xxx:558 -e WEBSITES_ENABLE_APP_SERVICE_STORAGE=false -e WEBSITES_PORT=5001 -e
WEBSITE_SITE_NAME=xxx -e WEBSITE_AUTH_ENABLED=False -e
WEBSITE_ROLE_INSTANCE_ID=0 -e WEBSITE_INSTANCE_ID=xxx -e
HTTP_LOGGING_ENABLED=1 xxx.azurecr.io/xxx:558
2018-08-23 14:13:05.385 INFO - Container xxx for site xxx initialized
successfully.
​
we responded to your question on Github and Reddit. Re-posting our response here for visibility.
"Currently, there is a workaround for this: you should modify the default resolv.conf to the custom DNS IP and then add your custom resolv.conf on docker build by adding a COPY command in your entrypoint script and pointing a custom resolv.conf to /etc.
However, we are investigating a better solution for this, so that manually updating the resolv.conf wouldn’t be necessary, so stay tuned."
You shouldn't use DNS to communicate with microservices, instead, you should make use of service registry.
Check this Microsoft paper talking about this:
Each microservice has a unique name (URL) that is used to resolve its
location. Your microservice needs to be addressable wherever it is
running. If you have to think about which computer is running a
particular microservice, things can go bad quickly. In the same way
that DNS resolves a URL to a particular computer, your microservice
needs to have a unique name so that its current location is
discoverable. Microservices need addressable names that make them
independent from the infrastructure that they are running on. This
implies that there is an interaction between how your service is
deployed and how it is discovered, because there needs to be a service
registry. In the same vein, when a computer fails, the registry
service must be able to indicate where the service is now running.
As you can see, the best solution will depend on your deployment model. Check this note about containers:
In some microservice deployment environments (called clusters, to be
covered in a later section), service discovery is built-in. For
example, within an Azure Container Service environment, Kubernetes and
DC/OS with Marathon can handle service instance registration and
deregistration. They also run a proxy on each cluster host that plays
the role of server-side discovery router. Another example is Azure
Service Fabric, which also provides a service registry through its
out-of-the-box Naming Service.
Hope it helps!

Docker: "driver failed programming external connectivity on endpoint testcontainer"

I am getting the below error while creating a python3 container in manjaro VMware:
docker: Error response from daemon: driver failed programming external connectivity on endpoint testcontainer (c55fc0dd481c36765fcd968118c3fbf5c7fa686cdfc625c485f963109b0f89e3): (iptables failed: iptables --wait -t nat -A DOCKER -p tcp -d 0/0 --dport 5000 -j DNAT --to-destination 172.17.0.2:80 ! -i docker0: iptables: No chain/target/match by that name.
(exit status 1))`
i cannot understand what is the problem?
dockerfile:
FROM python:3.7-alpine
RUN adduser -D test`
WORKDIR /home/testapp`
ADD ./webapp/requirements.txt requirements.txt`
RUN pip3 install --upgrade pip
RUN pip3 install -r requirements.txt`
RUN pip3 install gunicorn
ADD ./webapp webapp/`
ENV FLASK_APP app.py
USER test
EXPOSE 5000
ENTRYPOINT ["./app.py"]
For clarity, the answer that worked for me from the thread linked in the comment section is:
# Enter below command, it will clear all chains.
$ sudo iptables -t filter -F
$ iptables -t filter -X
# Then restart Docker Service using below comamnd
$ systemctl restart docker
https://github.com/moby/moby/issues/16816#issuecomment-327074574
I bumped into this recently. The firewall wasn't running as the error suggested.
The solution:
systemctl start firewalld
After restarting the firewall, was able to raise the container up normally without error.
Might not be the only cause of this error, but the firewall being down produced this error in my case using Ubuntu 20.04 LTS (ARM64).

Access service running in docker container from inside another docker container

At the moment I'm running a node.js application inside a docker container which needs to connect to camunda, which runs in another container.
I start the containers with the following command
docker run -d --restart=always --name camunda -p 8000:8080 camunda/camunda-bpm-platform:tomcat-7.4.0
docker run -d --name app -p 3000:3000 app
Both applications are now running and I can access camunda by navigating to my host's IP on port 8000, and running wget http://localhost:8000 -q -O - also returns the camunda page. When I login to my app container with docker exec -it app sh and type wget http://localhost:8000 -q -O -, I cannot access camunda. Instead I get the following error:
wget: can't connect to remote host (127.0.0.1): Connection refused
When I link my app container to the camunda container with --link camunda:camunda, and type wget http://camunda:8000 -q -O - in my app container, I get the following error:
wget: can't connect to remote host (172.17.0.4): Connection refused`
I've seen this option, so I started my app container with --add-host camunda:my_hosts_ip and tried wget again, resulting in:
wget: can't connect to remote host (149.210.227.191): Operation timed out
When running wget http://149.210.227.191:5001 -q -O - on my host machine however, I get a correct response immediately.
Ideally I would like to just start my app container without the need to supply the external IP in any way, and let the app container just use the camunda service via the localhost or by linking the camunda container tot my app container. What would be the easiest way to achieve this?
Why does it not work?
Containers and host do not share their local IP stack. Thus, when you are within a container and try anything localhost:port the anything command will try to connect to the container-specific local IP stack, not the other container nor the host.
How to make it work?
Hard way: you either need to know the IP address of the other container and connect to this IP address..
Easier and cleaner way: .. either link your containers.
--link=[]
Add link to another container in the form of <name or id>:alias or just <name or id> in which case the alias will match the name
So you'll need to perform, assuming the camunda container is named camunda:
docker run -d --name app -p 3000:3000 --link camunda app
Then, once you docker-exec-ed into the container app you will be able to execute wget http://camunda:8080 -q -O - without error.
Note that while the linked containers graph cannot loop, e.g., camunda cannot be linked to app as you need to start a container to be able to link it, you actually do whatever you want/need playing with IP addresses.
Note also that you can specify the IP address of a container using the --ip option (though it can only be used in conjunction with --net for user-defined networks).
Original answer below. Note that link has been deprecated and the recommended replacement is network. That is explained in the answer to this question: docker-compose: difference between network and link
--
Use the --link camunda:camunda option for your app container. Then you can access camunda via http://camunda:8080/.... The link option adds a entry to the /etc/hosts file of the app container with the IP address of the camunda container. This also means you have to restart your app container if you restart the camunda container.

Can't access from outside process running in a Docker container

I'm trying to run a gameserver inside a docker container on my server but I'm having troubles connecting to it.
I created my container and started my gameserver (which is using port 7777) inside it.
I'm running the container with this command:
docker run -p 7777:7777 -v /home/gameserver/:/home -c=1024 -m=1024m -d --name my_gameserver game
I published the ports 7777 with the -p parameter but I can't connect to my gameserver, even though logs show that it is started.
I think I should bind my IP in some way but I have no idea what to do.
What I found so far is that docker inspect my_gameserver | grep IPAddress returns 172.17.0.24.
The problem was coming from the fact that I didn't expose the UDP port.
Correct command was:
docker run -p 7777:7777 -p 7777:7777/udp -v -d --name my_gameserver game

Resources