I'd like to setup Jenkins with a mix of Azure and on-premise agents. Ideally I would like the Jenkins master to be in Azure and have on-premise agents connect to that master, however the on-premise agents will not be publicly exposed but will be able to access the Jenkins master on Azure.
Is it possible to have a mix of Azure and on-premise agents? Is it possible to have the on-premise agents talk to a Jenkins master on Azure? If so, how would I configure this and what would I need to know?
Yes, it's possible. You just need to create an Azure VPN and then the on-premise agent could connect the Azure master agent through its private IP. In this way, you do not need to expose the on-premise agent to the public.
yes, this is possible. jenkins doesnt care where agents are located. for jenkins agent to talk to master on Azure they'd need to be able to communicate. Normally via ssh, I'm not quite sure if master talks to slaves or vice versa, but either way its possible to achieve using proper networking on Azure\on-premise.
If you'd need to connect to Azure master from agents, you'd need to assign public ip address to the master, in the reverse scenario you'd need to expose some port on your infrastructure that can be used to talk to your agents.
Another alternative is using site-to-site VPN\express route, to establish direct connection to on-premise stuff (this is a bit harder to achieve, but might be beneficial in the long run).
Related
I've got a Azure Build pipeline, which runs on Microsoft Hosted Build Agent.
CI build contains integration tests that need to connect to a SQL Server, which is hosted on a Azure Virtual Machine, however that SQL Server is accessible only from private network.
Is there a way for CI build to connect to that SQL Server in order to run integration tests?
Is there a way for CI build to connect to that SQL Server in order to run integration tests?
Indeed, just like Poney said, we could white list the IP address used by the Microsoft Azure Datacenter IP Ranges in the list mentioned:
https://www.microsoft.com/en-nz/download/details.aspx?id=41653
But this list gets updated every week, so please be mindful of this during the deployment planning.
As workaround, you could set start/stop IP range 0.0.0.0, but this option configures the firewall to allow all connections from Azure including connections from the subscriptions of other customers. When selecting this option, make sure your login and user permissions limit access to only authorized users.
Check the documents Azure SQL Database and SQL Data Warehouse IP firewall rules and Deploy .dacpc to Azure DB via VS Team Services Release–Using Hosted Agents for some more details.
This way you don't need to update the IP list according to the XML downloaded weekly.
Hope this helps.
I'am using AzureDevOps to build and pack my docker application.
The goal is to execute docker-compose commands in Azure VM, which is behind the firewall and can be access only thru vpn connection or standard web-browsing ports.
You can use deployment groups to achieve that. Reason this will work, because it is a one way communication (from agent to Azure Devops), so you dont really need to open ports for the VM, the VM has to only be able to reach Azure Devops endspoints (more on this).
TLDR. Using agents will work, because its an outgoing connection from the agent, not from Azure Devops to the agent.
I am working on Azure Poc . Is it possible to connect to on premise Sybase database from Azure cloud ?
Can I read and write data into Sybase from Azure cloud ?
Yes you can. Look at Azure Relay Hybrid Connections for a code-first approach or Site-to-Site VPN for an infrastructure one. Also since you're on a Linux VM, why not just SSH tunnel your traffic over? - probably the simplest solution to your conundrum.
There's also ExpressRoute which you may want to take a look at if we're talking serious business that needs connectivity, bandwidth and latency guarantees.
We have an application running in Azure that consists of the following:
A Web App front end, which talks to…
A WebApi running as a Web App as well, which can (as well as a couple other services) talk to…
A Cloud Service load balanced set of VMs which Are hosting an Elasticsearch cluster.
Additionally we have the scenario were dev’s whitelist their IPs so that their localhost version of the API can hit the VMs as well.
We have locked down our Elasticsearch VM’s by adding ACLs to the exposed end point. I whitelisted the outbound IPs that were listed on my App Services. I was under the mistaken impression that these were unique to my Api. It turns out that these are shared across the scale unit in Azure. Other services running in the same scale unit, could, if they knew the endpoint, access the data exposed on the endpoint in my cluster. I need to lock this down, and I am trying to find the easiest way. These are the things I am looking at, and I would appreciate advice and/or redirection.
Elastic Shield: Not being considered. This is a product by Elastic
that is designed to secure ES. This is ideal, but at the moment it
is out of scope (due to the cost and overhead)
List item
Elastic plugins: Not being considered. The main plugins (such as
Jetty) appear to be abandoned.
Azure VPN. I originally tried to set this up, but ran into too many
difficulties. The ACLs seemed to give me what I need without much
difficulty. I am not sure if I can set this up now. The things I
don’t know are:
I don’t think I can move existing VMs into a new VPN.
I think you have to recreate the VMs in that VPN from the get go
Could I move my Web App into the VPN? How does that work?
This would prob break my developer scenario as the localhost API
would not be able to access the VPN, right?
Add a certificate to requests: It would be ideal if I could have
requests require a cert or a header token. I assume to do this I
would need to create a proxy that would run on the VMs and do the
validation before forwarding the request on to my Elasticsearch.
Anything else? Is there another option I have not thought of?
Thanks!
~john
You can create a VPN point-to-site connecting your Web App with your IaaS VMs. This is the best solution because you will be able to use just internal IPs on your IaaS.
The easiest way to do that using Azure Portal is create a Web App and, create a new VPN and VNet using "setup" option at "Your Web App" -> Settings -> Networking -> VNET Integration -> Setup -> Create New Virtual Network.
After that, create your IaaS inside this new VNet.
You also can create a ARM template to create Web App, IaaS, VPN and everything that you need. Take a look at my ARM template to create PHP+MySQL using Web App and MariaDB Cluster connected by VPN: https://github.com/juliosene/azure-webapp-php-mariadb
I have a azure cloud service (a server) where i host a Redis database. I also have a web site hosted in azure web sites. I want the web site to be able to talk to the Redis DB on port 6379. I know I can configure a public endpoint for that port on my server but that would open it for whole Internet. I want it opened only for azure web sites (or even better, only for my web site). How can i do this?
Windows Azure Web Sites is in an isolation bubble separate from your Cloud Services and there's no way to bridge that gap. Ideally you'd do this by connecting the web site machine to other Azure services via a Virtual Network, but this FAQ confirms you can't do that right now:
Can I use Windows Azure websites with Virtual Network?
No. We do not support websites with virtual networks.
Opening Redis up over the internet shouldn't even be considered as it doesn't have the kind of security you'd want out of the box to be opening up its port publicly as it is meant to be co-located with your application, so you really wouldn't want to do that. Never mind the added network overhead which will eat into the performance you expect to get by leveraging something like Redis anyway.
I believe your best bet given your current configuration is to add a Web Role that's part of the same Azure Cloud Service and run your web based application out of that so that it can communicate with worker role. It only requires a little bit of configuration to get this going (i.e. adding an InternalEndpoint to the Redis Worker Role). While I realize Web Roles don't offer as frictionless a development model as Web Sites, you have to choose the right tool for the job.
Another option, if you want to setup your Redis on a VM instead of tying it to the Cloud Service directly, is that you can setup a Virtual Network, put the Redis VM on the virtual network and then configure the Cloud Service so that it's part of the same affinity group and add the NetworkConfiguration/VirtualNetworkSite configuration section to the Cloud Service's .cscfg.
Which approach makes more sense all depends on how you leverage your Redis instance, but the main benefit of the latter approach is that the Redis instance is not recreated each time you deploy your Cloud Service and, so, any data that's in it will stay available between deployments. Another benefit is if you want to build and leverage a Redis cluster across multiple Cloud Services this enables you to do that.