`Hello everyone,
I am facing issue with dynamic inventory for azure
Getting following error:
ansible-inventory -i test.azure_rm.yaml --graph -vvv
ansible-inventory [core 2.13.7]
config file = /etc/ansible/ansible.cfg
configured module search path = ['/root/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
ansible python module location = /usr/lib/python3/dist-packages/ansible
ansible collection location = /root/.ansible/collections:/usr/share/ansible/collections
executable location = /usr/bin/ansible-inventory
python version = 3.10.6 (main, Nov 14 2022, 16:10:14) [GCC 11.3.0]
jinja version = 3.0.3
libyaml = True
Using /etc/ansible/ansible.cfg as config file
host_list declined parsing /root/test.azure_rm.yaml as it did not pass its verify_file() method
script declined parsing /root/test.azure_rm.yaml as it did not pass its verify_file() method
Using inventory plugin 'ansible_collections.azu`your text`re.azcollection.plugins.inventory.azure_rm' to process inventory source '/root/test.azure_rm.yaml'
toml declined parsing /root/test.azure_rm.yaml as it did not pass its verify_file() method
[WARNING]: * Failed to parse /root/test.azure_rm.yaml with auto plugin: Failed to get credentials. Either pass as
parameters, set environment variables, define a profile in ~/.azure/credentials, or install Azure CLI and log in (az login).
File "/usr/lib/python3/dist-packages/ansible/inventory/manager.py", line 290, in parse_source
plugin.parse(self._inventory, self._loader, source, cache=cache)
File "/usr/lib/python3/dist-packages/ansible/plugins/inventory/auto.py", line 59, in parse
plugin.parse(inventory, loader, path, cache=cache)
File "/root/.ansible/collections/ansible_collections/azure/azcollection/plugins/inventory/azure_rm.py", line 220, in parse
self._credential_setup()
File "/root/.ansible/collections/ansible_collections/azure/azcollection/plugins/inventory/azure_rm.py", line 241, in _credential_setup
self.azure_auth = AzureRMAuth(**auth_options)
File "/root/.ansible/collections/ansible_collections/azure/azcollection/plugins/module_utils/azure_rm_common.py", line 1479, in init
self.fail("Failed to get credentials. Either pass as parameters, set environment variables, "
File "/root/.ansible/collections/ansible_collections/azure/azcollection/plugins/module_utils/azure_rm_common.py", line 1605, in fail
self._fail_impl(msg)
File "/root/.ansible/collections/ansible_collections/azure/azcollection/plugins/module_utils/azure_rm_common.py", line 1608, in _default_fail_impl
raise AzureRMAuthException(msg)
[WARNING]: * Failed to parse /root/test.azure_rm.yaml with yaml plugin: Plugin configuration YAML file, not YAML inventory
File "/usr/lib/python3/dist-packages/ansible/inventory/manager.py", line 290, in parse_source
plugin.parse(self._inventory, self._loader, source, cache=cache)
File "/usr/lib/python3/dist-packages/ansible/plugins/inventory/yaml.py", line 114, in parse
raise AnsibleParserError('Plugin configuration YAML file, not YAML inventory')
[WARNING]: * Failed to parse /root/test.azure_rm.yaml with ini plugin: Invalid host pattern 'plugin:' supplied, ending in
is not allowed, this character is reserved to provide a port.
File "/usr/lib/python3/dist-packages/ansible/inventory/manager.py", line 290, in parse_source
plugin.parse(self._inventory, self._loader, source, cache=cache)
File "/usr/lib/python3/dist-packages/ansible/plugins/inventory/ini.py", line 136, in parse
raise AnsibleParserError(e)
[WARNING]: Unable to parse /root/test.azure_rm.yaml as an inventory source
[WARNING]: No inventory was parsed, only implicit localhost is available
> #all:
> |--#ungrouped:
Able to do az vm list
az vm list | wc -l
1341
My azure inventory yaml is below
*cat test.azure_rm.yam`your text`l*
plugin: azure.azcollection.azure_rm
include_vm_resource_groups:
- '*'
auth_source: auto
I've configured the credentials But I can't list the inventory using the dynamic inventory plugin. I have azure_rm.py in the same directory.`
Related
I'm unable to create an Ansible dynamic inventory for Azure. I get the following error:
bash-5.1# ansible-inventory -i inventory_azure_rm.yaml --graph -vvv
ansible-inventory [core 2.12.2]
config file = /playbook/ansible.cfg
configured module search path = ['/root/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
ansible python module location = /usr/lib/python3.9/site-packages/ansible
ansible collection location = /root/.ansible/collections:/usr/share/ansible/collections
executable location = /usr/bin/ansible-inventory
python version = 3.9.7 (default, Nov 24 2021, 21:15:59) [GCC 10.3.1 20211027]
jinja version = 3.0.3
libyaml = False
Using /playbook/ansible.cfg as config file
host_list declined parsing /playbook/inventory_azure_rm.yaml as it did not pass its verify_file() method
toml declined parsing /playbook/inventory_azure_rm.yaml as it did not pass its verify_file() method
[WARNING]: * Failed to parse /playbook/inventory_azure_rm.yaml with script plugin: problem running /playbook/inventory_azure_rm.yaml --list ([Errno 13] Permission denied:
'/playbook/inventory_azure_rm.yaml')
File "/usr/lib/python3.9/site-packages/ansible/inventory/manager.py", line 290, in parse_source
plugin.parse(self._inventory, self._loader, source, cache=cache)
File "/usr/lib/python3.9/site-packages/ansible/plugins/inventory/script.py", line 150, in parse
raise AnsibleParserError(to_native(e))
[WARNING]: * Failed to parse /playbook/inventory_azure_rm.yaml with auto plugin: name 'client_secret' is not defined
File "/usr/lib/python3.9/site-packages/ansible/inventory/manager.py", line 290, in parse_source
plugin.parse(self._inventory, self._loader, source, cache=cache)
File "/usr/lib/python3.9/site-packages/ansible/plugins/inventory/auto.py", line 58, in parse
plugin.parse(inventory, loader, path, cache=cache)
File "/root/.ansible/collections/ansible_collections/azure/azcollection/plugins/inventory/azure_rm.py", line 219, in parse
self._credential_setup()
File "/root/.ansible/collections/ansible_collections/azure/azcollection/plugins/inventory/azure_rm.py", line 240, in _credential_setup
self.azure_auth = AzureRMAuth(**auth_options)
File "/root/.ansible/collections/ansible_collections/azure/azcollection/plugins/module_utils/azure_rm_common.py", line 1522, in __init__
self.azure_credential_track2 = client_secret.ClientSecretCredential(client_id=self.credentials['client_id'],
[WARNING]: * Failed to parse /playbook/inventory_azure_rm.yaml with yaml plugin: Plugin configuration YAML file, not YAML inventory
File "/usr/lib/python3.9/site-packages/ansible/inventory/manager.py", line 290, in parse_source
plugin.parse(self._inventory, self._loader, source, cache=cache)
File "/usr/lib/python3.9/site-packages/ansible/plugins/inventory/yaml.py", line 112, in parse
raise AnsibleParserError('Plugin configuration YAML file, not YAML inventory')
[WARNING]: * Failed to parse /playbook/inventory_azure_rm.yaml with ini plugin: Invalid host pattern 'plugin:' supplied, ending in ':' is not allowed, this character is reserved to provide a port.
File "/usr/lib/python3.9/site-packages/ansible/inventory/manager.py", line 290, in parse_source
plugin.parse(self._inventory, self._loader, source, cache=cache)
File "/usr/lib/python3.9/site-packages/ansible/plugins/inventory/ini.py", line 136, in parse
raise AnsibleParserError(e)
[WARNING]: Unable to parse /playbook/inventory_azure_rm.yaml as an inventory source
[WARNING]: No inventory was parsed, only implicit localhost is available
#all:
|--#ungrouped:
The inventory_azure_rm.yaml file is:
plugin: azure.azcollection.azure_rm
auth_source: credential_file
plain_host_names: yes
include_vm_resource_groups:
- <redacted>
keyed_groups:
- key: tags.applicationRole
separator: ""
The ansible.cfg file is:
[defaults]
inventory = inventory_azure_rm.yaml
[inventory]
enable_plugins = host_list, script, auto, yaml, ini, toml
Ansible Azure collection version
bash-5.1# ansible-galaxy collection list
# /root/.ansible/collections/ansible_collections
Collection Version
------------------ -------
azure.azcollection 1.11.0
I would appreciate any help on trying to solve this.
Thank you.
Update:
Fixed inventory_azure_rm.yaml file permissions.
bash-5.1# ls -la inventory_azure_rm.yaml
-rw-r--r-- 1 root root 200 Feb 24 17:27 inventory_azure_rm.yaml
Updated the error stacktrace on the problem description running the command again.
Update2:
The Azure credentials file looks like this:
bash-5.1# cat ~/.azure/credentials
[default]
subscription_id=<redacted>
client_id=<redacted>
secret=<redacted>
tenant=<redacted>
cloud_environment=AzureCloud
I finally managed to fix the problem on parsing the dynamic inventory. I was doing the following:
pip install -r https://raw.githubusercontent.com/ansible-collections/azure/dev/requirements-azure.txt && \
ansible-galaxy collection install azure.azcollection:1.11.0
I've changed 2 things:
Invert the order on installing the collection and its dependencies. First I need to install the azure.azcollection and after that, its dependencies.
Install the azure.azcollection dependencies from the requirements.txt coming with the collection itself instead of doing it from Github.
This is the code working:
ansible-galaxy collection install azure.azcollection:1.11.0 && \
pip install -r ~/.ansible/collections/ansible_collections/azure/azcollection/requirements-azure.txt
The difference between the requirements.txt file from GitHub at https://raw.githubusercontent.com/ansible-collections/azure/dev/requirements-azure.txt and the local requirements.txt file at ~/.ansible/collections/ansible_collections/azure/azcollection/requirements-azure.txt is on azure-mgmt-network package version. The online version is 19.1.0 and the local (working) version is 12.0.0.
bash-5.1# diff -w requirements-azure.txt ~/.ansible/collections/ansible_collections/azure/azcollection/requirements-azure.txt
--- requirements-azure.txt
+++ /root/.ansible/collections/ansible_collections/azure/azcollection/requirements-azure.txt
## -19,7 +19,7 ##
azure-mgmt-monitor==3.0.0
azure-mgmt-managedservices==1.0.0
azure-mgmt-managementgroups==0.2.0
-azure-mgmt-network==19.1.0
+azure-mgmt-network==12.0.0
azure-mgmt-nspkg==2.0.0
azure-mgmt-privatedns==0.1.0
azure-mgmt-redis==5.0.0
This is pertaining to jfrog artifactory. pypi-public is our virtual repo and our internal pypi-internal is associated to pypi-public. I can see the package vapi_common on the web UI.
The below command is able to search the package
pip search vapi_common --index=https://<username>:<apikey>#company.jfrog.io/artifactory/api/pypi/pypi-public/simple
However, if I use the same index-url in ~/.pip/pip.conf
[global]
index-url = https://<username>:<apikey>#company.jfrog.io.jfrog.io/artifactory/api/pypi/pypi-public/simple
and then use the below command
pip search vapi_common -vvv -> fails the below error. As you can see, it is trying to reach pypi.org and is not honoring the index url given in pip.conf
pip search vapi_common -vvv
Starting new HTTPS connection (1): pypi.org:443
https://pypi.org:443 "POST /pypi HTTP/1.1" 200 419
ERROR: Exception:
Traceback (most recent call last):
File "/home/varmour/.pyenv/versions/3.8.8/lib/python3.8/site-packages/pip/_internal/cli/base_command.py", line 228, in _main
status = self.run(options, args)
File "/home/varmour/.pyenv/versions/3.8.8/lib/python3.8/site-packages/pip/_internal/commands/search.py", line 60, in run
pypi_hits = self.search(query, options)
File "/home/varmour/.pyenv/versions/3.8.8/lib/python3.8/site-packages/pip/_internal/commands/search.py", line 80, in search
hits = pypi.search({'name': query, 'summary': query}, 'or')
File "/home/varmour/.pyenv/versions/3.8.8/lib/python3.8/xmlrpc/client.py", line 1109, in __call__
return self.__send(self.__name, args)
File "/home/varmour/.pyenv/versions/3.8.8/lib/python3.8/xmlrpc/client.py", line 1450, in __request
response = self.__transport.request(
File "/home/varmour/.pyenv/versions/3.8.8/lib/python3.8/site-packages/pip/_internal/network/xmlrpc.py", line 45, in request
return self.parse_response(response.raw)
File "/home/varmour/.pyenv/versions/3.8.8/lib/python3.8/xmlrpc/client.py", line 1341, in parse_response
return u.close()
File "/home/varmour/.pyenv/versions/3.8.8/lib/python3.8/xmlrpc/client.py", line 655, in close
raise Fault(**self._stack[0])
xmlrpc.client.Fault: <Fault -32500: "RuntimeError: PyPI's XMLRPC API is currently disabled due to unmanageable load and will be deprecated in the near future. See https://status.python.org/ for more information.">
Please note you yourself use pip search --index=…. That is, you should use option index in pip.conf, not index-url. index is for pip search, index-url is for pip download/install.
See the docs at https://pip.pypa.io/en/stable/reference/pip_search/#options
Fix config:
pip config set global.index https://:#company.jfrog.io.jfrog.io/artifactory/api/pypi/pypi-public/simple
Perhaps even
pip config set global.index `pip config get global.index-url`
Trying to set up Azure CLI on my Kali Linux. This is the version of kali.
kali#kali:~$ uname -a
Linux kali 5.5.0-kali2-amd64 #1 SMP Debian
5.5.17-1kali1 (2020-04-21) x86_64 GNU/Linux
After installation, when I run az login, it takes me to the browser where I enter required credentials that are accepted, and the browser then shows following message.
You have logged into Microsoft Azure!
However, in the shell window, it shows the following error.
kali#kali:~$ az login
You have logged in. Now let us find all the subscriptions to which you have access...
The command failed with an unexpected error. Here is the traceback:
'SubscriptionClient' object has no attribute 'config'
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/knack/cli.py", line 215, in invoke
cmd_result = self.invocation.execute(args)
File "/usr/lib/python3/dist-packages/azure/cli/core/commands/_init_.py", line 654, in execute
raise ex
File "/usr/lib/python3/dist-packages/azure/cli/core/commands/_init_.py", line 718, in _run_jobs_serially
results.append(self._run_job(expanded_arg, cmd_copy))
File "/usr/lib/python3/dist-packages/azure/cli/core/commands/_init_.py", line 711, in _run_job
six.reraise(*sys.exc_info())
File "/usr/lib/python3/dist-packages/six.py", line 703, in reraise
raise value
File "/usr/lib/python3/dist-packages/azure/cli/core/commands/_init_.py", line 688, in _run_job
result = cmd_copy(params)
File "/usr/lib/python3/dist-packages/azure/cli/core/commands/_init.py", line 325, in __call_
return self.handler(*args, **kwargs)
File "/usr/lib/python3/dist-packages/azure/cli/core/_init_.py", line 782, in default_command_handler
return op(**command_args)
File "/usr/lib/python3/dist-packages/azure/cli/command_modules/profile/custom.py", line 152, in login
subscriptions = profile.find_subscriptions_on_login(
File "/usr/lib/python3/dist-packages/azure/cli/core/_profile.py", line 195, in find_subscriptions_on_login
subscriptions = subscription_finder.find_through_authorization_code_flow(
File "/usr/lib/python3/dist-packages/azure/cli/core/_profile.py", line 849, in find_through_authorization_code_flow
result = self._find_using_common_tenant(token_entry[_ACCESS_TOKEN], resource)
File "/usr/lib/python3/dist-packages/azure/cli/core/_profile.py", line 893, in _find_using_common_tenant
client = self._arm_client_factory(token_credential)
File "/usr/lib/python3/dist-packages/azure/cli/core/_profile.py", line 812, in create_arm_client_factory
configure_common_settings(cli_ctx, client)
File "/usr/lib/python3/dist-packages/azure/cli/core/commands/client_factory.py", line 79, in configure_common_settings
client.config.enable_http_logger = True
AttributeError: 'SubscriptionClient' object has no attribute 'config'
To open an issue, please run: 'az feedback'
What does this error mean and how can this be addressed?
honestly this looks like a version mismatch on the libraries you are using? I'd probably remove az cli completely and remove all the python modules that are related to azure and reinstall it, alternatively, just use a docker image with az cli provided by microsoft:
docker run -it mcr.microsoft.com/azure-cli
I need to use the google cloud api to write my Dataflow jobs.
As I understand it, I can't use pip install google-cloud-dataflow since Apache Beam wont' work on Python 3, so I've been using googleapiclient.discovery. However, when I issue the build() command, it bombs out citing the error:
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1045)
Background notes:
I'm sitting behind a Corporate Proxy, with HTTP(S)_PROXY set at the environment level
I also have CA_BUNDLE and REQUESTS_CA_BUNDLE set to my custom certs
I've installed certifi, but no love
I've attempted to run /Applications/Python\ 3.6/Install\
Certificates.command but couldn't find the .command in my virtualenv. Also, would prefer not to go down this path as it will make my Prod deployment a nightmare
Here's my code:
from oauth2client.client import GoogleCredentials
from googleapiclient.discovery import build
credentials = GoogleCredentials.get_application_default()
dataflow = build('dataflow', 'v1b3', credentials=credentials)
Result:
Traceback (most recent call last):
File "test_dataflow_creds.py", line 6, in
dataflow = build('dataflow', 'v1b3', credentials=credentials)
File "/Users/user/.pyenv/versions/unit-test-3.7/lib/python3.7/site-packages/googleapiclient/_helpers.py", line 130, in positional_wrapper
return wrapped(*args, **kwargs)
File "/Users/user/.pyenv/versions/unit-test-3.7/lib/python3.7/site-packages/googleapiclient/discovery.py", line 222, in build
requested_url, discovery_http, cache_discovery, cache)
File "/Users/user/.pyenv/versions/unit-test-3.7/lib/python3.7/site-packages/googleapiclient/discovery.py", line 269, in _retrieve_discovery_doc
resp, content = http.request(actual_url)
File "/Users/user/.pyenv/versions/unit-test-3.7/lib/python3.7/site-packages/httplib2/init.py", line 1924, in request
cachekey,
File "/Users/user/.pyenv/versions/unit-test-3.7/lib/python3.7/site-packages/httplib2/init.py", line 1595, in _request
conn, request_uri, method, body, headers
File "/Users/user/.pyenv/versions/unit-test-3.7/lib/python3.7/site-packages/httplib2/init.py", line 1501, in _conn_request
conn.connect()
File "/Users/user/.pyenv/versions/unit-test-3.7/lib/python3.7/site-packages/httplib2/init.py", line 1291, in connect
self.sock = self._context.wrap_socket(sock, server_hostname=self.host)
File "/Users/user/.pyenv/versions/3.7.0/lib/python3.7/ssl.py", line 412, in wrap_socket
session=session
File "/Users/user/.pyenv/versions/3.7.0/lib/python3.7/ssl.py", line 850, in _create
self.do_handshake()
File "/Users/user/.pyenv/versions/3.7.0/lib/python3.7/ssl.py", line 1108, in do_handshake
self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1045)
tl;dr: got it working by exporting all Certs to a common file, then appending to the Cert file in the path as specified by Certifi
steps:
In Firefox > Preferences > View Certificates > Your Certificates, export all the required ones.
Concatenate all of the above .crt files into one big bundle.
In bash, run python -m requests.certs to get the certs file python is using.
Append the bundled certs from step 2 above to the file from step 3.
Done
When attempting to build with Pants, I am seeing the following error:
File "build/bdist.macosx-10.10-intel/egg/pants/contrib/go/tasks/go_fetch.py", line 154, in _transitive_download_remote_libs
all_known_addresses)
File "build/bdist.macosx-10.10-intel/egg/pants/contrib/go/tasks/go_fetch.py", line 105, in _transitive_download_remote_libs
fetcher.fetch(go_remote_lib.import_path, dest=tmp_fetch_root, rev=go_remote_lib.rev)
File "build/bdist.macosx-10.10-intel/egg/pants/contrib/go/subsystems/fetchers.py", line 437, in fetch
github_root, github_rev = self._map_import_path(import_path, rev)
File "/Users/chad/.cache/pants/setup/bootstrap/pants.mbFDa8/install/lib/python2.7/site-packages/pants/util/memo.py", line 95, in memoize
result = func(*args, **kwargs)
File "build/bdist.macosx-10.10-intel/egg/pants/contrib/go/subsystems/fetchers.py", line 454, in _map_import_path
raise self.FetchError('Invalid gopkg.in package and rev in: {}'.format(import_path))
Exception message: Invalid gopkg.in package and rev in: gopkg.in/amz.v1/aws
Here is the contents of my BUILD file:
# Auto-generated by pants!
# To re-generate run: `pants buildgen.go --materialize --remote`
go_remote_library(rev='v1')
Looking into the code, I see that the error comes from a failure to match a regex in fetchers.py, on line 453.
I am running Pants version 0.0.59 on Mac OS X 10.10 (Yosemite)
Noting that #Huckphin stumbled on a bug here in pantsbuild.pants<=0.0.59. He filed an issue and now things are fixed up for handling gopkg.in remote import paths that point to sub-packages in the remote repo. The fix will be released with the regular Friday release on 11/20/2015 in 0.0.60.