I need to get a list of VMs that have unrestricted SSH.
I've been browsing the Azure SDK for Python documentation. There is an SshConfiguration class in the compute module, but it only has info about the public keys. There is a different SshConfiguration class in the batch AI module that can be used to get a list of public IPs that are allowed to connect, which is what I want. But I'm not using batch AI.
How can I get the information I want programmatically?
You'll have to take a detour for this as there is no direct method in the Compute module that gives this information directly.
Using methods from both the Compute and Network modules, I coded up the following script to list all VMs in the Subscription with unrestricted SSH access, i.e., list all the Inbound rules allowing access to the VM over port 22 from the Internet.
# Imports
from azure.common.credentials import ServicePrincipalCredentials
from azure.mgmt.network import NetworkManagementClient
from azure.mgmt.compute import ComputeManagementClient
# Set subscription ID
SUBSCRIPTION_ID = 'xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx'
def get_credentials():
credentials = ServicePrincipalCredentials(
client_id='xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx',
secret='xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx',
tenant='xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx'
)
return credentials
# Get credentials
credentials = get_credentials()
# Initialize management client
network_client = NetworkManagementClient(
credentials,
SUBSCRIPTION_ID
)
# Initialize management client
compute_client = ComputeManagementClient(
credentials,
SUBSCRIPTION_ID
)
def get_unrestricted_ssh_rules():
print('\nListing all VMs in Subscription with unrestricted SSH access:')
for vm in compute_client.virtual_machines.list_all():
# Get the VM Resource Group name
vm_rg_name = vm.id.split('/')[4]
# Loop through NICs
for nic in vm.network_profile.network_interfaces:
# Get the NIC name and Resource Group
nic_name = nic.id.split('/')[-1]
nic_rg_name = nic.id.split('/')[4]
# Get the associated NSG and its Resource Group
nsg = network_client.network_interfaces.get(
nic_rg_name, nic_name).network_security_group
nsg_name = nsg.id.split('/')[-1]
nsg_rg_name = nsg.id.split('/')[4]
# Get the associated Security Rules
for rule in network_client.security_rules.list(nsg_rg_name, nsg_name):
# Conditions:
# Rule direction: Inbound
# Rule Access: Allow
# Port: 22
# Source Address Prefix: 'Internet' or '*'
unrestricted = (rule.direction == 'Inbound' and rule.access == 'Allow' and ('22' in (rule.destination_port_ranges, rule.destination_port_range)
or rule.destination_port_range == '*') and (rule.source_address_prefix == '*' or rule.source_address_prefix == 'Internet'))
# Print all the Inbound rules allowing access to the VM over port 22 from the Internet
if unrestricted:
print "\nVM Name: ", vm.name, "\nResource Group Name: ", vm_rg_name, "\nRule Name: ", rule.name
# List all VMs in the Subscription with unrestricted SSH access
get_unrestricted_ssh_rules()
You may repeat the same with Subnets as well if the NSG is associated with a Subnet instead of a NIC.
References:
azure-mgmt-compute
VirtualMachinesOperations class
azure-mgmt-network
SecurityRulesOperations class
NetworkInterfacesOperations class
Hope this helps!
Related
I am a beginner with Python and with AzureML.
Currently, my task is to list all the running VMs (or Compute Instances) with status and (if running) for how long they ran.
I managed to connect to AzureML and list Subscriptions, Resource Groups and Workspaces, but I'm stuck on how to list running VMs now.
Here's the code that I have currently:
# get subscriptions list using credentials
subscription_client = SubscriptionClient(credentials)
sub_list = subscription_client.subscriptions.list()
print("Subscription ID".ljust(column_width) + "Display name")
print(separator)
for group in list(sub_list):
print(f'{group.subscription_id:<{column_width}}{group.display_name}')
subscription_id = group.subscription_id
resource_client = ResourceManagementClient(credentials, subscription_id)
group_list = resource_client.resource_groups.list()
print(" Resource Groups:")
for group in list(group_list):
print(f" {group.name}{group.location}")
print(" Workspaces:")
my_ml_client = Workspace.list(subscription_id, credentials, group.name)
for ws in list(my_ml_client):
try:
print(f" {ws}")
if ws:
compute = ComputeTarget(workspace=ws, name=group.name)
print('Found existing compute: ' + group.name)
except:()
Please note that this is more or less a learning exercise and it's not the final shape of the code, I will refactor once I get it to work.
Edit: I found an easy way to do this:
workspace = Workspace(
subscription_id=subscription_id,
resource_group=group.name,
workspace_name=ws,
)
print(workspace.compute_targets)
Edit2: If anyone stumbles on this question and is just beginning to understand Python+Azure just like I do, all this information is from official documentation (which is just hard to follow as a beginner).
The result from 'workspace.compute_targets' will contain both Compute Instances and AML Instances.
If you need to retrieve only the VMs (like I do) you need to take an extra step to filter the result like this:
if type(compute_list[vm]) == ComputeInstance:
As per my requirement using Azure Python SDK I want to get Azure VM family details if the existing VM belongs to N servies family or something else.
Below is the code which I am using.
from azure.mgmt.compute import ComputeManagementClient
from azure.common.credentials import ServicePrincipalCredentials
client_id = "sp appId"
secret = "sp password"
tenant = "sp tenant"
credentials = ServicePrincipalCredentials(
client_id = client_id,
secret = secret,
tenant = tenant
)
Subscription_Id = ''
compute_client =ComputeManagementClient(credentials,Subscription_Id)
resource_group_name='Networking-WebApp-AppGW-V1-E2ESSL'
virtual_machine_scale_set_name='VMSS'
results = compute_client.resource_skus.list(raw=True)
resourceSkusList = [result.as_dict() for result in results]
r=json.dumps(resourceSkusList)
print(r)
But here above code gives me details of all available resource SKUs and I wanted the resource SKU on the basis of a VM names as I want to check if a particular VM belongs to which family.
Could anyone help me in achieving this?
The SkuOperations class contains a list() method starting with azure-mgmt-storage==16.0.0. You can use a set to get rid of duplicate SKU names since the method will return numerous SKUs for each area and storage type it supports.
from azure.mgmt.storage import StorageManagementClient
from azure.identity import DefaultAzureCredential
storage_client = StorageManagementClient(
credential=DefaultAzureCredential(),
subscription_id="xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
)
skus = {sku.name for sku in storage_client.skus.list()}
for sku in skus:
print(sku)
Iterate the set for good format:
for sku in skus:
print(sku)
Which outputs the SKUs on newlines like so:
Standard_LRS
Standard_RAGRS
Standard_RAGZRS
Standard_ZRS
Premium_LRS
Standard_GRS
Standard_GZRS
Premium_ZRS
I am looking to create approx. 15 virtual machines with Terraform, of which all have their own Size in Azure such as B2S, B2MS etc. They also have different sized disks. I understand that you can use Copy Index to loop through arrays, however I'm not sure of the best way to do this with VM's that have lots of properties that are different.
Is there a way to create a map object for each VM specification and then just loop through it with a Virtual Machine creation in the TF file? At the moment the only way I can see of doing it is creating a separate virtual machine resource in the main file and referencing each individual map file..
Create a map as you mention with the vm prefix as the key and size as the value:
variable "vms" {
type = "map"
default = {
vm1 = "Standard_DS1_v2"
vm2 = "Standard_ES2_v2"
}
}
Create your VMS:
# Network Interfaces for each one of the VMs
resource "azurerm_network_interface" "main" {
# looping to create a resource for each entry in the map
for_each = var.vms
# Accessing keys in the map by each.key
name = "${each.key}-nic"
...
}
resource "azurerm_virtual_machine" "main" {
# Looping to create a VM per entry in the map
for_each = var.vms
# Accessing names of map entries
name = "vm-${each.key}-we"
# Here we make sure we access the corrrect
network_interface_ids = [azurerm_network_interface.main[each.key]]
vm_size = each.value
...
os_profile {
# Accessing names of map entries again
computer_name = "vm-${each.key}-we"
...
}
...
}
For brevity I didn't write down the whole example of creating azure vms.
There's many attributes you'll have to fill in as you need them.
Docs about how to create Azure VMS: https://www.terraform.io/docs/providers/azurerm/r/virtual_machine.html
Docs about resources and "looping" them: https://www.terraform.io/docs/configuration/resources.html
Terraform has seriously the best docs out there IMO.
I have a python list of Security Groups and I need to find to which EC2/RDS instance or ELB they are associated. What's the easiest way to do this in Boto3?
Also, to my best understanding one security group can be attached to several instances and one EC2 instance can have several security groups attached, so I need to find a way to identify those relationships to better clean it up. What I have right is a python list of security group objects.
This is my current code:
import boto3
import json
# regions = ["us-east-1","ap-southeast-1","ap-southeast-2","ap-northeast-1","eu-central-1","eu-west-1"]
regions = ["us-east-1"]
uncompliant_security_groups = []
for region in regions:
ec2 = boto3.resource('ec2', region_name=region)
sgs = list(ec2.security_groups.all())
for sg in sgs:
for rule in sg.ip_permissions:
# Check if list of IpRanges is not empty, source ip meets conditions
if len(rule.get('IpRanges')) > 0 and rule.get('IpRanges')[0]['CidrIp'] == '0.0.0.0/0':
if rule.get('FromPort') == None:
uncompliant_security_groups.append(sg)
if rule.get('FromPort') != None and rule.get('FromPort') < 1024 and rule.get('FromPort') != 80 and rule.get('FromPort') != 443:
uncompliant_security_groups.append(sg)
print(uncompliant_security_groups)
print(len(uncompliant_security_groups))
for sec_group in uncompliant_security_groups:
If you enable AWS Config Aggregator in the account (granted you have to pay for it):
account_id = '0123456789'
region = 'us-east-2'
sg_id = 'sg-0123456789'
relationship_data = CONFIG_CLIENT.get_aggregate_resource_config(
ConfigurationAggregatorName='agg_name',
ResourceIdentifier={
'SourceAccountId': account_id,
'SourceRegion': region,
'ResourceId': sg_id,
'ResourceType': 'AWS::EC2::SecurityGroup'
}
)]
relationship_data = relationship_data['ConfigurationItem']['relationships']
print(relationship_data)
Which should return some data like:
[
{'resourceType': 'AWS::EC2::NetworkInterface', 'resourceId': 'eni-0123456789', 'relationshipName': 'Is associated with NetworkInterface'},
{'resourceType': 'AWS::EC2::Instance', 'resourceId': 'i-0123456789', 'relationshipName': 'Is associated with Instance'},
{'resourceType': 'AWS::EC2::VPC', 'resourceId': 'vpc-0123456789', 'relationshipName': 'Is contained in Vpc'}
]
NOTE: This appears to ONLY work with AWS CONFIG AGGREGATORS! I have NO idea why this is, or if the data can be obtained from aws config by itself. However my org uses aws config so this enables this type of data for me.
Boto3 config docs:
https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/config.html
def get_instance_id_from_pip(self, pip):
subscription_id="69ff3a41-a66a-4d31-8c7d-9a1ef44595c3"
compute_client = ComputeManagementClient(self.credentials, subscription_id)
network_client = NetworkManagementClient(self.credentials, subscription_id)
print("Get all public IP")
for public_ip in network_client.public_ip_addresses.list_all():
if public_ip.ip_address == pip:
print(public_ip)
# Get id
pip_id= public_ip.id.split('/')
print("pip id : {}".format(pip_id))
rg_from_pip = pip_id[4].lower()
print("RG : {}".format(rg_from_pip))
pip_name = pip_id[-1]
print("pip name : {}".format(pip_name))
for vm in compute_client.virtual_machines.list_all():
vm_id = vm.id.split('/')
#print("vm ref id: {}".format(vm_id))
rg_from_vm = vm_id[4].lower()
if rg_from_pip == rg_from_vm:
## this is the VM in the same rg as pip
for ni_reference in vm.network_profile.network_interfaces:
ni_reference = ni_reference.id.split('/')
ni_name = ni_reference[8]
print("ni reference: {}".format(ni_reference))
net_interface = network_client.network_interfaces.get(rg_from_pip, ni_name)
print("net interface ref {}".format(net_interface))
public_ip_reference = net_interface.ip_configurations[0].public_ip_address
if public_ip_reference:
public_ip_reference = public_ip_reference.id.split('/')
ip_group = public_ip_reference[4]
ip_name = public_ip_reference[8]
print("IP group {}, IP name {}".format(ip_group, ip_name))
if ip_name == pip_name:
print("Thank god. Finallly !!!!")
print("VM ID :-> {}".format(vm.id))
return vm.id
I have above code to get the VM instance ID from Public ip but its not working. What is real surprising is that for all instances, I am getting x.public_ip_address.ip_address as None value.
I had multiple readthedocs references for python SDK of Azure. but, some how all links not working. Good job Azure :)
Some of them:
https://azure-sdk-for-python.readthedocs.io/en/v1.0.3/resourcemanagementcomputenetwork.html
https://azure-storage.readthedocs.io/en/latest/ref/azure.storage.file.fileservice.html
Second edit:
I got the answer to this and above code will return vm id given the public ip address. Though, you can see it is not absolutely optimized answer. Hence, better answers are welcome. Thanks!
Docs have moved here:
https://learn.microsoft.com/python/azure/
We made some redirection, but unfortunately it's not possible to go global redirection on RTD, so some pages are 404 :/
About your trouble, I would try to use the publicip operation group directly:
https://learn.microsoft.com/en-us/python/api/azure.mgmt.network.v2017_11_01.operations.publicipaddressesoperations?view=azure-python
You get this one in the Network client, as client.public_ip_addresses