Dapr: Failed to proxy request: proxy not initialized. daprd startup may be incomplete - dapr

I,m performing some tests with Dapr and Service Invocation using gRPC and have run in to an issue. I'm runnig Dapr in standalone mode and starts two applications like this:
dapr run --app-id App1 --app-port 10078 --dapr-grpc-port 11078 .\App1.exe
dapr run --app-id App2 --app-port 10114 --dapr-grpc-port 11114 .\App2.exe
When App2 tries to make a gRPC call to localhost:11114 to reach App1 i get an error
'Grpc.Core.RpcException: Status(StatusCode="Unknown", Detail="failed to proxy request: proxy not initialized. daprd startup may be incomplete.", DebugException="Grpc.Core.Internal.CoreErrorDetailException: {"created":"#1650461046.340000000","description":"Error received from peer ipv4:10.16.100.25:11114","file":"..\..\..\src\core\lib\surface\call.cc","file_line":904,"grpc_message":"failed to proxy request: proxy not initialized. daprd startup may be incomplete.","grpc_status":2}")
App1 hosts a gRPC service on port 10078 and App2 host a gRPC service on port 10114. Both applications show up in the dashboard and the ports look correct.
When the app starts the log says "API gRPC server is running on port 11114" for App2 which seams correct. How can i diagnose this further?

Related

Spring Boot App deployed on Azure App Service port 8080

I'm trying to deploy a Spring Boot Rest Api (Gradle) to an Azure app service. There is an executable jar in the /home/site/wwwroot folder and a startup command (java -jar /home/site/wwwroot/app.jar). When I execute this script, the spring api starts but throws the following error: "The Tomcat connector configured to listen on port 80 failed to start. The port may already be in use or the connector may be misconfigured."
I can't run the app on another port because the Azure App Service only allows inbound HTTP traffic on port 80 and 443 for SSL. I tried killing the process that is running on port 80 but that kills my ssh terminal.
Is there a solution to 'clear' the proces running on port 80 and actually run my Spring application on it?
"The Tomcat connector configured to listen on port 80 failed to start.The port may already be in use or the connector may be misconfigured."
The error is because another process is running on the same port. To solve this, you have two options.
Try to run the application on a port other than 8080.
Identify and stop the process running on that specific port.
Check this post on how to change the default tomcat port number.
See: how the spring boot configuration works.
Check this blog and SO thread might be helpful. Tomcat Server Error - Port 8080 already in use

Azure container App fails to start with Julia image

I am trying to run Julia code in azure container app, but it fails for some reason.
docker file is pretty simple:
FROM julia:1.7-bullseye
COPY . /
EXPOSE 8080
RUN julia --project='/' -e 'using Pkg; Pkg.build()'
ENTRYPOINT julia --project='/' main.jl "0.0.0.0" "8080"
"0.0.0.0" and "8080" there is the host and port on which HTTP is served:
HTTP.serve(GENERAL_ROUTER, host, port_restful_api)
The app supposed to run a simple HTTP server that is served over port 8080, GET to / is the health check that returns simple "Up"
When I run that same image locally with docker run -p 8080:8080 my-test:latest it works fine and hitting localhost:8080 gives me my expected response
But when I have it in container app - revision fails to be provisioned (and no extra errors provided). In the ingress I added availability to accept traffic from anywhere and target port is 8080...
What I am missing?
UPDATE
At the same time in log analytics I can see my output to console that should indicate that the app is supposedly running fine, e.g.
[ Info: API handler service: Running on thread 1
In Julia those are done with
#info "API Server: Running on thread $(Threads.threadid())"
But in logs those are in stderr stream. Could it be that the ACA treats any output to stderr as startup error?
UPDATE 2:
The very same code works without any issues or changes on Azure Container Instance and in Azure Kubernetes Service...
So for now I stopped investigating and decided to use ACI.
You should verify if any process inside the julia image needs root privileges as ACA do not support running privileged containers based on the docs:
Azure Container Apps can't run privileged containers. If your program
attempts to run a process that requires root access, the application
inside the container experiences a runtime error.
Did you specify 8080 as your target port when configuring the HTTP ingress of your Container App? https://learn.microsoft.com/en-us/azure/container-apps/ingress?tabs=bash
The platform will try to ping your app at the specified port due to the default health probes. If it receives no response, it will restart your app: https://learn.microsoft.com/en-us/azure/container-apps/health-probes?tabs=arm-template

Running Logux on Google Cloud Run doesn't find the running port

While trying to get Logux running in Cloud Run, I get this error:
Container failed to start. Failed to start and then listen on the port defined by the PORT environment variable.
Even though the server is running an the Dockerfile is exposing the correct port (which is mapped in Cloud run as well).
The Dockerfile is located here: https://github.com/knownasilya/battle-chess/blob/main/Dockerfile
Note that the server runs https and ws on port 31337.
As mentioned by the Cloud Run Troubleshooting docs, if we get the message:
Container failed to start. Failed to start and then listen on the port defined by the PORT environment variable.
And we know our code/container is working as expected, we should check that the port that that Cloud Run will be checking correlates with the one used in our app and like in this case, that the container listens on 0.0.0.0 (all the net interfaces)
As well this is mentioned in the Cloud Run Requirements doc:
The container must listen for requests on 0.0.0.0 on the port to which requests are sent. By default, requests are sent to 8080, but you can configure Cloud Run to send requests to the port of your choice. Cloud Run injects the PORT environment variable into the container. Inside Cloud Run container instances, the value of the PORT environment variable always reflects the port to which requests are sent. It defaults to 8080.

Does Azure Web App for containers support ssh access when running a multi-container app?

I'm running an Azure Web app (containers) with custom container images. I've followed the steps to enable ssh into a container image and it works great when I'm only running a single container. But when I run the app as a multi-container app (with docker-compose file) with more than one container image I get the error below. For additional context this is a small python web app that using nginx and redis hence the need for more than one container. Only one of my custom images has ssh enabled and running and exposing port 2222.
Is this even possible? If not then I'm not sure how feasible it is to run a web app multi-container if I have no way to access a container for support purposes.
az webapp remote-connection create -g GROUPNAME -n APPNAME -p 2222 --verbose
Configured default 'GROUPNAME' for arg resource_group_name
remote-connection is deprecated and moving to cli-core, use `webapp create-remote-connection`
Port 2222 is open
Creating a socket on port: 2222
Setting socket options
Binding to socket on local address and port
Finished initialization
Status response message: FAILURE:2222:Unable to connect to WebApp
WARNING - Remote debugging may not be setup properly. Reponse content: FAILURE:2222:Unable to connect to WebApp
SSH is available { username: root, password: Docker! }
Start your favorite client and connect to port 2222
I also tried the create-remote-connection command but got similar results.
az webapp create-remote-connection -n APPNAME -g GROUPNAME --verbose &
Error I receive is:
Auto-selecting port: 52661
Finished initialization
Status response message: FAILURE:2222:Unable to connect to WebApp
WARNING - Remote debugging may not be setup properly. Reponse content: FAILURE:2222:Unable to connect to WebApp
Connection is not ready yet, please wait
.
Status response message: FAILURE:2222:Unable to connect to WebApp
WARNING - Remote debugging may not be setup properly. Reponse content: FAILURE:2222:Unable to connect to WebApp
Looks like its not supported :(
see...
How to SSH in to different containers in Multi Container Azure App Service
and...
Support SSH to specific container in multi-container setup

Telegraf daemon not receiving metrics from app deployed with Dokku

I've already monitored NodeJS apps in the past with Telegraf+InfluxDB+Grafana, but it's the first time I try to monitor a Dokku app, without success so far.
I have the following setup
app server monitoring serveur
- telegraf daemon (listening on udp 8125) ------> InfluxDB + Grafana
- dokku
- myapp : sending metrics on localhost:udp:8125
I have the issue that metrics sent from myapp on udp://localhost:8125 are not received by telegraf.
The connectivity from telegraf to influxdb and grafana is ok, because I ran the following command on the app server:
echo "foobar:1|c" | nc -u -w0 127.0.0.1 8125
And I could visualize inside Grafana the foobar counter increment.
So I'm guessing the issue is due to myapp or how I configured Dokku.
In myapp (a node JS app), I'm using the hot-shots package to send metrics (also tried with lynx without success).
This is the code I use to instanciate the statsd client.
var StatsD = require('hot-shots');
var client = new StatsD();
// Attaching an error handler to client's socket
client.socket.on('error', function(error) {
console.error('StatsD client error in socket: ', error);
});
I increment a counter on app startup to signal a restart. This is done with the following code:
metrics.increment('server_restart', function(err, bytes) {
if (err) {
console.log(err);
} else {
console.log('StatsD sent : ', bytes);
}
});
Inside the app server, I ran dokku logs myapp after performing a deployment.
I get the following output
$ dokku logs myapp
2017-05-06T14:51:19.977938845Z app[web.1]:
2017-05-06T14:51:19.977996938Z app[web.1]: > myapp#1.0.0 start /app
2017-05-06T14:51:19.978002950Z app[web.1]: > node .
2017-05-06T14:51:19.978005766Z app[web.1]:
2017-05-06T14:51:21.849086537Z app[web.1]: StatsD sent : 18
$
So it seems that the metrics payload is properly sent on the right port, but it never reaches telegraf.
Is there any configuration to do in Dokku to authorize external UDP requests to localhost ?
What can I do to debug this further ?
As the author requested in another thread the solution seems to be:
By default, can a docker container call host's localhost UDP?
Opening ports is only needed when you want to Listen for the requests not sending. By default Docker provides the necessary network namespace for your container to communicate to the host or outside world.
So, you could do it in two ways:
use --net host in your docker run and send requests to localhost:8125 in this case you containerized app is effectively sharing the host's network stack. So localhost points to the daemon that's already running in your host.
talk to the container network gateway (which is usually 172.17.0.1) or your host's hostname from your container. Then your are able to send packets to your daemon in your host.

Resources