I have a resource group with a virtual network in Azure. The virtual network has subnets, but they are not appearing in the list, nor are they returned by the following cloud shell line:
az network vnet subnet list --vnet-name [vnet] --resource-group [resource group]
You would think that this is because they do not exist. However, one of these subnets has a load balancer, and two more have running containers in them. These resources are all still working. I can still access the containers through the load balancer, and the containers still have the same private IP addresses as before.
Furthermore, I can create create a new subnet with the same address space and Azure does not raise any errors or objections, then delete that subnet, and the load balancer and containers are still running just fine.
I think the problem may have been created when I created a new virtual machine in a fourth subnet of this network that had already been Delegated to ContainerInstance. You would think that Azure would throw an error and not let me do that, for everything else it complains about, but here we are.
Is there some kind of orphan virtual network that these subnets are in now? How do I troubleshoot this?
Thank you.
I tested in my environment working fine for me and able to get details of all of my subnet under a specific VNET even though for few of my subnet is delegated to particuler service and few of them attached to resources.
Furthermore, I can create create a new subnet with the same address
space and Azure does not raise any errors or objections, then delete
that subnet, and the load balancer and containers are still running
just fine
Not sure how this can be happened with you that should not be expected. I would suggest you to troubleshoot this kind of situation please reach out to Azure Support team.
Is there some kind of orphan virtual network that these subnets are in
now? How do I troubleshoot this?
No if there already some of your resources uses that VNEt and their subnet so this can not be terated as Orphan.
Related
While migrating a cluster we moved the vnet used by the AKS from one resource group (the one with the nodepool created by the AKS) to a different RG where we created the AKS cluster. This however, resulted in an unexpected state. The original vnet in the nodepool resource group stayed as is while it copied the vnet in to the AKS RG with the same ID. So now we have to vnet with the same name but in two different resource groups. Afterwards when we tried to create a new nodepool we received the following error:
Code="VMScaleSetMustBelongToSameVnetAsLB" Message="VM scale set
references virtual network
/subscriptions/12345/resourceGroups/project-test-k8s-mc-rg/providers/Microsoft.Network/virtualNetworks/AKS-VNET-931
which is different than load balancer virtual network
/subscriptions/12345/resourceGroups/project-test-k8s-rg/providers/Microsoft.Network/virtualNetworks/AKS-VNET-931. VM scale set and load balancer must belong to same virtual network."
The cluster was created with a managed vnet.
We tried searching for ways to change the load balancer created by AKS to use a different vnet, we do not see any options. We cannot afford to recreate the cluster at this stage. So do we have any other options to fix this issue?
There was no direct option to change the load balancer created by AKS to use a different VNet. If the load balancer uses an IP address in a different subnet, ensure the AKS cluster identity also has read access to that subnet. The VM scale set and load balancer must always belong to the same virtual network.
We can modify only address space and subnet only. Found one blog by "Ajay Kumar" refer tutorial for more information.
In my Azure subscription I have 2 peered VNETs. VNET1 has address space 10.16.0.0/16 and VNET2 has 10.250.21.0/24. I have chosen the space addresses so that they were completely different. Peering works given that a VM in VNET1 can ping a VM in VNET2.
However, from VNET 1 I can't access an App Service deployed in VNET2. I tried to access the App Service using the IP address and the private DNS name.
Network Security Groups in both VNETs seem to be fine. I'm able to change them so that basic network diagnostics work (for instance, ping).
Any suggestions please?
EDIT 1 + SOLUTION
I can say that it's solved. After peering the virtual networks, I had to "link the private DNS zones". I had a few issues with the address spaces, but nothing that a terraform destroy/apply couldn't solve.
I tried to reproduce the same in my environment and got the results like below:
I have created a vnet peering with two different space address like below:
Created a app service with premium p1v2 and added outbound traffic vnet integration and added a private endpoint like below:
Then, I created a vm and connect through bastion:
When I try to verify in command prompt using nslookup got result successfully like below:
We have managed AKS Cluster and it has a few applications PODS. In the same subscription, we have a few servers in the different Resource Group and different VNET. We have a requirement to happen a communication between these two VNET's. I have configured vnet peering between two VNET's but we can see that the communication is not happening.
When I add a rule like "Allow port 443 from all networks" on to the NSG of Virtual machines then everything works fine.
Troubleshooting steps are done.
VNET Peering
Got an API Server IP Address from the "kubeconfig" file and added in the NSG of VM's in a diff RG.
But did not resolve an issue. Could you please help me to fix the issue.
AKS Resources are behind the Internal Load Balancer, so peering did not help. I had to use the Public IP Address provisioned during the AKS Creation process in the NSG. After adding PIP(Available in MC_rg-*** resource group) everything started working.
I would suggest to try connecting the VNET's through VPN gateways .
From an Azure virtual network, connecting to another virtual network is essentially the same as connecting to an on premises network via site-to-site (S2S) VPN.
You will need to go through the below listed steps :
Create VNetA and VNetB and the Corresponding Local Networks.
Create the Dynamic Routing VPN Gateways for each virtual network.
Connect the VPN Gateways.
Please find the referred document for implementing the same solution I have mentioned above .
For more information on difference of vnet peering and vnet gateway you can refer this document.
I already have a resource group in azure that has a virtual machine and it is associated with a network interface to connect to a vnet in that group.
resource-group-1
-> virtual machine
-> network interface
-> vnet
-> other things ...
But now i created a second resource group which will have some resources and a vnet. I want to use for now, the same virtual machine that exists on resource-group-1, so i created a network interface there:
resource-group-2
-> network interface 2
-> vnet 2
Now i want to connect the network interface 2 of resource-group-1 to the virtual machine in resource-group-2.
I already tried, like the documentation says to stop the VM and going to Settings, Networking and Attach network interface, but i don't seem to be able to select anything useful there.
It seems that some time ago, this changes could only be done through powershell.
Another thing i saw is that we cannot change the vnet when adding a new nic to a VM. This might not be possible at all?
Currently, i guess the following options might make this possible
Find a way to change the VNET of the VM, which will make it stop working with the old VNET.
Create an equal VM with a snapshot for the new VNET.
Connect the two VNETs and probably it will magically work?
It's impossible to add a new network interface that exists in the different VNet from the network interface currently attached to the VM to an existing VM. You can not add two NICs in different VNets to the same Azure VM. Refer to this.
If you want to move your azure VM and its NIC to another resource group, a simple way is to move a VM to another resource group. Read supported moving resources.
In addition, there is a Step-by-Step PowerShell Guide: How to Move Azure VM to a different Resource Group?
I am just practicing the azure. And I created a virtual network and subnets.
Now I am not able to delete the subnet. I have disconnected all connected devices from the virtual network.
Here is the error getting while deleting subnet
Please help me to clean up the subnet and virtual network.
Thanks
Additional details
All resources in my account
And here is the subnets tab and error getting while deletion:
Subnet configuration:
As the error shows the subnet PrivateSubnet should be in use. And you can see it's delegated to the service Microsoft.Web/serverfarms. So you need to remove the delegated service first. Do it follow the steps here. When the delegated service list is empty, then you can delete the subnet as you want.
Update:
According to my experience, your subnet is in use network/PrivateSubnet/AppLink, it means there is a Web App integrate with your subnet at least. And I didn't see the Web App in your resource group. So I guess there someone else uses your subnet and his Web App still exists. You need to check it yourself.