Create proxies using Azure for a bot that i have developed.
I'm creating a bot which utilizes proxies to buy merchandise of sites from back-end and i wanted to generate some proxies using Azure to mask my local IP. There are various services that offer the proxies (data center and residential proxies) for a little amount.
However, i would like to generate proxies by myself using Azure. How can i create proxies and use it in my local application so websites believe that the request is coming from the proxy server?
I create few virtual VM's (two linux and two windows based). I tries using squid proxy to convert the VM into a proxy but since i do not have much knowledge on linux, i am facing multiple issues. Also, i am not understanding on how to proceed on the windows machine.
You can use Azure Functions to call proxies for App Services and OnPremise Servers:
Announcement:
https://azure.microsoft.com/en-us/updates/announcing-azure-functions-proxies-in-public-preview/
How To, Sample:
https://learn.microsoft.com/en-us/azure/azure-functions/functions-proxies
Related
I have an Api Rest developed with entity framework core 3.1 in C #, I need to deploy the application in a virtual machine in Azure, but it does not work, most of the tutorials that I have taken talk about how to create the virtual machine and publish a web application simple, any guide, help or tutorial?
Generally the error is 500 (internal server error), and problems with the web config
You need to make sure that external requests can land and be processed by the Web Server (typically IIS) running inside the VM. For that you need to open firewall ports to allow inbound traffic within the VM as well as through the network interface (found on the Networking tab) of the VM within the portal.
An API is technically deployed as part of a web application. Hence the following links would help.
Link 1
Link 2 (Note: Video has no voice)
That being said, deploying your API as a App Service in Azure (PaaS) is a much better approach rather than using VMs (unless your API has specific requirements that it needs to be deployed in a VM). App Services also makes setting up other associated services e.g. Logging and monitoring, authentication, etc. much easier.
My Azure setup involves two web apps and a PostgreSQL server. One of the web apps is a Node frontend, which should be available to the public. The other is a Python backend, which receives requests from the Node app and communicates with the Postgres database. The Python app contains HTTP endpoints that should not be available for anyone to access.
What is the recommended approach to protecting this Python app from unwanted traffic? Should I be blocking traffic outright through some sort of Azure configuration, or simply authenticating my HTTP requests?
I've tried only allowing the outbound IPs of the Node app to communicate with the Python app, via the Azure configuration. However, this seems to have left the Python app unable to communicate with the database, and additionally I can't even SSH or view its logs with this configuration.
It really depends on what your requirements are. If cost is no option, one way to protect the python app is to put it on an App Service Environment (ASE). This is an isolated instance of Azure Web Apps that you can protect behind an Internal Load Balancer. This solution will give you more security as you can enable a Network Security Group to block out Internet traffic and you could setup your Node App to communicate with your VNet with a VPN. This approach is also one of the more expensive approaches for a PaaS Web App.
Other options include setting up your "back-end" python as IaaS (but then you have to manage the updates), or you can use an App Gateway or 3rd party WAF device like KEMP (they have a 200 MBps device that is free) to protect you app.
Finally you can look at a scalability design where you put a queue or some other intermediary between your two web apps. This will allow for independent scaling and give you the opportunity to lock-down the Python app to only accept messages from the queue, not your front-end. A sample arch can be seen here (you can sub the function in this arch for your python app)
As developers we wrote microservices on Azure Service Fabric and we can run them in Azure in some sort of PaaS concept for many customers. But some of our customers do not want to run in the cloud, as databases are on-premises and not going to be available from the outside, not even through a DMZ. It's ok, we promised to support it as Azure Service Fabric can be installed as a cluster on-premises.
We have an API-gateway microservice running inside the cluster on every virtual machine, which uses the name resolver, and requests are routed and distributed accordingly, but the API that the API gateway microservice provides is the entrance for another piece of client software which our customers use, that software runs outside of the cluster and have to send requests to the API.
I suggested to use an Load Balancer like HA-Proxy or Nginx on a seperate machine (or machines) where the client software send their requests to and then the reverse proxy would forward it to an available machine inside the cluster.
It seems that is not what our customer want, another machine as load balancer is not an option. They suggest: make the client software smarter to figure out which host to go to, in other words: we should write our own fail-over/load balancer inside the client software.
What other options do we have?
Install Network Load Balancer Feature on each of the virtual machine to give the cluster a single IP address, is this even possible? Something like https://www.poweradmin.com/blog/configuring-network-load-balancing-in-windows-server/
Suggest an API gateway outside the cluster, like KONG https://getkong.org/
Something else ?
PS: The client applications do not send many requests per second, maybe a few per minute.
Very similar problem, we have a many services and Service Fabric Cluster that runs on-premises. When it's time to use the load balancer we install IIS on the same machine where Service Fabric cluster runs. As the IIS is a good load balancer we use IIS as a reverse proxy only for API Gateway. Kestrel hosting is using for other services that communicate by HTTP. The API gateway microservice is the single entry point for all clients and has always static URI inside SF, we used that URI to configure IIS
If you do not have possibility to use IIS then look at Using nginx as HTTP load balancer
You don't need another machine just for HTTP forwarding. Just use/run it as a service on the cluster.
Did you consider using the built in Reverse Proxy of Service Fabric? This runs on all nodes, and it will forward http calls to services inside the cluster.
You can also run nginx as a guest executable or inside a Container on the cluster.
We have also faced the same situation when started working with service fabric cluster. We configured Application Gateway as Proxy but it would not provide the function like HTTP to HTTPS redirection.
For that, we configured Nginx Instead of Azure Application Gateway as Proxy to Service Fabric Application.
Here´s what we try to do:
We try to run several App Servers with Meteor and Mongo DB Servers on Azure VMs. We have them bundled in one cloud service with Endpoint sets that balance the load.
We setting them up via the Management Console (Ubuntu VMs) and then deploy Meteor and the content via Meteor Up.
Now we want to do HTTPS. Initially we thought that is handled by the LoadBalancer by mapping an external port 443 to the internal port 80 as there are options to upload Certificates.
It seems there is no option to configure that this way except for deployments using Visual Studio (and those seem to have to use at least some Web roles).
Here is where we struggle:
Using HTTPS seems to be tied to deploying an App developed in visual studio and/or on Windows VMs?
That´s the question:
Is it possible to use load balanced HTTPS on szure with Linux VMs?
PS: This is the Article that made me think the Load Balancer might does SSL encryption by itself: https://msdn.microsoft.com/en-us/library/azure/ff795779.aspx
AFAIK, the load balancer of Azure would only spread the traffic to the instances internal endpoint, and do not encrypt the traffic in SSL. (see: https://azure.microsoft.com/en-us/documentation/articles/load-balancer-overview/)
Even for the deployment using Visual Studio, I believe the mechanism behind is also the IIS on each web role encrypting the traffic, but not the load balancer job.
So, you should add SSL on the web server of each VMs such as using nginx.
I'm new to Azure and have set up an Asure web site which will connect to a SOLR instance on an Azure VM. The web site cannot communicate with SOLR (port 8080) unless I create an endpoint. In order to create an endpoint I need to provide an internal and external port, however I do not want to allow external/remote access to SOLR, other than from the web site itself. Can anyone tell me how I should allow the web site to communicate with my VM, without making the VM publicly accessible?
Thank you,
JP
As #hhaggan stated, you cannot create a virtual network connection, or secure an endpoint, from Web Sites to a Virtual Machine. One option to consider is the use of SSL+certificate to secure the endpoint.
If you instead deploy your web application to a web role (in a cloud service) or a virtual machine, you can then take advantage of a brand new feature called Endpoint ACL. In essence, you can whitelist / blacklist a set of IP address ranges that may access your SOLR external endpoint. You can read about ACL'd endpoints, and how to configure them, here.
Now: You might be asking yourself why this doesn't work with Web Sites. That's because Web Sites runs on a server farm, and sites don't get unique IP addresses (unlike Cloud Services).
if you are talking about Windows Azure Websites, I am not sure if this can be done but if you are talking about Web roles in Windows Azure Cloud service, here is something that I hope it can be beneficial.
Connect ASP.NET application to SQL Server in Windows Azure via Virtual Network http://msdn.microsoft.com/en-us/library/windowsazure/jj823135.aspx
you might also would like to have a look on this http://www.windowsazure.com/en-us/services/virtual-network/ I don't think you will need it but I am just sharing it with you, it is for WIndows Azure Virtual Network.
let me know if this solved your problems or if you need anything else.