We have a Prometheus Pushgateway running and listening to metrics push from our AWS Lambda function. However, the URL to the Pushgateway is accessible by the public, which might impose some security issues. We were wondering if there is any way we could add a layer of protection to the Pushgateway so that it is not publicly accessible?
I found this Github thread that may answered this question:
https://github.com/prometheus/pushgateway/issues/281
It proposed to set up a reverse proxy in front of the pushgateway. However, I am still confused on how that may actually work? We are currently using Kubernetes to deploy the Prometheus.
You can include authentication in your ingress controller by using a TLS secret as an ingress rule.
Here's an example that shows how to generate basic auth for your ingress:
https://kubernetes.github.io/ingress-nginx/examples/auth/basic/
Also, don't forget to include the Python handler function in your client to set the auth header as pointed out here:
https://github.com/prometheus/client_python#handlers-for-authentication
A suggestion here will be to make the URL of the Pushgateway Internal by using an AWS Internal Load Balancer, create an AWS Private Hosted Zone attach your VPC to this zone after this the next step will be to deploy the lambda in the same VPC.
This should solve the security issue.
You are right, you need reverse proxy here. I also faced with the same issue, so you need nginx in front of your prometheus/pushgateway.
First, install nginx using this article (you can start from Step 8 — Securing Prometheus if you already configured prometheus):
My nginx config :
events { }
http {
upstream prometheus {
server 127.0.0.1:9090;
keepalive 64;
}
upstream pushgateway {
server 127.0.0.1:9091;
keepalive 64;
}
server {
root /var/www/example;
listen 0.0.0.0:80;
server_name __;
location / {
auth_basic "Prometheus server authentication2";
auth_basic_user_file /etc/nginx/.htpasswd;
proxy_pass http://prometheus;
}
}
server {
root /var/www/example;
listen 0.0.0.0:3001;
server_name __;
location / {
auth_basic "Pushgateway server authentication";
auth_basic_user_file /etc/nginx/.htpasswd;
proxy_pass http://pushgateway;
}
}
}
my pushgateway.service file :
[Unit]
Description=Pushgateway
Wants=network-online.target
After=network-online.target
[Service]
User=pushgateway
Group=pushgateway
Type=simple
ExecStart=/usr/local/bin/pushgateway --web.listen-address="127.0.0.1:9091" --web.telemetry-path="/metrics" --persistence.file="/tmp/metric.store" --persistence.interval=5m --log.level="info" --log.format="logger:stdout?json=true"
[Install]
WantedBy=multi-user.target
It is important to set : --web.listen-address="127.0.0.1:9091", not ":9091" - so it will be exposed only to localhost.
Through the nginx pushgateway will be accessible on port 3001, port 9091 will be not public. Base authentication will be required to have access or push metrics.
About how to test it using Postman you can find here
Related
I'm still new to nginx and I want to accomplish this.
I have two servers (server1 and server2), with an sftp server (bitvise) on server1.
And on server2 I have an nginx docker container running.
I want to configure nginx so when trafic comes to server2 (the one with nginx) on port 22 , it get redirected to server1, where my sftp sever is present.
I have an dns "transfer.test.com" mapped to my server2 public IP (tested).
This is the configuration I have added to nginx conf file.
server {
listen 22;
server_name transfer.test.com;
return 301 https://google.com;
location / {
set $sftp server1-private-ip:22;
proxy_pass $sftp;
}
}
server1-private-ip is the private IP of server1 (the one with sftp).
but till now its not working.
I can connect to sftp using filezile using the private IP of server1 BUT
I can't connect to sftp using filezila using the private IP of server2, means the trafic is not getting redirected.
Thank you for the help.
If you want to use nginx as a proxy to non-HTTP protocols like SSH or SFTP, you should define your server in a stream context rather than http one. Typical main configuration file (usually /etc/nginx/nginx.conf) looks like
user <username>;
worker_processes <number>;
...
events {
worker_connections <number>;
}
http {
include /etc/nginx/mime.types;
... # other global http directives here
include /etc/nginx/conf.d/*.conf;
}
As you can see, configuration files for individual servers (or server groups) are being included within the http context. You should add stream block to your main configuration file:
user <username>;
worker_processes <number>;
...
events {
worker_connections <number>;
}
http {
...
}
stream {
server {
listen 22;
proxy_pass <server1_private_ip>:22;
}
}
Directives like server_name or location are meaningless in the server blocks defined under the stream context. Please note that for using above configuration nginx should be compliled with ngx_stream_core_module and ngx_stream_proxy_module modules.
I have media server running on port 1935 .
and I have a subdomain: "streaming.foo.com"
I need to configure Nginx to access rtmp://streaming.foo.com
I tried the RTMP Nginx plugin with this blog :
https://www.nginx.com/blog/video-streaming-for-remote-learning-with-nginx/
and configured the /etc/nginx/nginx.conf file.
rtmp {
server {
listen 1935;
application live {
live on;
dash on;
dash_path /tmp/dash; // what is this config??
dash_fragment 15s;
}
}
}
but I get this error from nginx :
nginx[176987]: nginx: [emerg] bind() to 0.0.0.0:1935 failed (98:
Address already in use)
It is RTMP, not HTTP. Nginx is a web server, which can proxy and balance HTTP only. It is not a general proxy server.
Use a general Proxy or Load balancer, like HA Proxy.
You can't have two services listening on the same port.
The common solution is to configure the real service (your media service) on another port (let's say 19350) than configure a reverse proxy on nginix to forward requests from the exposed port (1935 in this case) to the back-end service (19350 in our example).
Sorry but I don't know nginix (I use this config with apache) so I can't help you on how to configure a reverse proxy on it.
I'm try launch Nodejs app on alibabacloud ecs with using nginx.
my app is in /var/www/html/ar_exp folder
When I runnode app then app listen localhost:8080
my nginx default config:
`
server {
listen 80;
server_name mydomain.com;
location / {
proxy_pass http://localhost:8080;
}
}
`
When I check from my ecs server via curl mydomain.com or via wget mydomain.com response is correct from nodejs app. But when i check it in browser then: ERR_connection_timed_out.
For all files in nodejs app folder () i set the following settings:
chown -R www-data:www-data /var/www/
and 755 for all files
what else would I check?
Website inaccessibility from the ECS can be due to following reasons.
OS firewall: Make sure that the application OS firewall(ufw) settings are properly configured to allow communication to the application port.
VPC Security Group: Make sure that the security group setting is applied and required communication port is configured.Details to configure security group
I have a published site that uses HTTPS. The site needs to communicates with a HTTP node Express API. The API is run on my local machine. Everything worked fine until I switched the client application to use HTTPS. Now I receive mixed content warnings. I have been reading about reverse proxys and wonder if this could be the solution to my problem. Is it possible to proxy a request to my localhost? Or will localhost point to the server the proxy is on?
I have been looking at using nginx as the reverse proxy server but I have zero experience with proxys and not positive how to go about it.
I am mainly wondering if it is possible or not before I dig any deeper.
Yes, this is a pretty standard use case for using nginx (or any other reverse proxy). You would configure the location prefixes, etc that need to go to your backend application and proxy (via proxy_pass directive) to them. Any static content can be served directly from nginx. All of this can then behind nginx.
Assuming that your application is never issuing absolute urls which make use of "http://" this should resolve your mixed content warnings.
You will probably want to read some tutorials but the basics of your configuration would be:
server {
listen 443 ssl; # you can also add http2
server_name hostnames that you listen for;
ssl_certificate_key /path/to/cert.key;
ssl_certificate /path/to/cert.pem;
root /var/www/sites/foo.com;
location /path/handled/by/application {
proxy_pass http://localhost:8000; # or whatever port is
}
}
I created a VM, port 80 is open and installed NGINX on it.
I created 2 App Services which can be accessed via x1.azurewebsites.net and x2.azurewebsites.net
I configured the VM to act as an load balancer but when redirecting the traffic I get the following: https://i.gyazo.com/b94bed9c90d3b0f0c400c83f762f0544.png
I am not using my own domain. Does someone know what the issue could be?
I got the following configurations:
upstream backend {
server xx.azurewebsites.net;
server xxx.azurewebsites.net;
}
server {
listen 80 default_server;
listen [::]:80 default_server;
root /var/www/html;
server_name_;
location / {
proxy_pass http://backend;
}
}
Azure App Service uses cookies for ARR (Application Request Routing). You have to make sure that your NGinx reverse proxy configuration pass the correct cookie / header to your web app.
The other possibility (to make sure the behavior comes from ARR) is to disable it: https://azure.microsoft.com/en-us/blog/disabling-arrs-instance-affinity-in-windows-azure-web-sites/