How to pass dictionary or JSON to Invoke-AzVMRunCommand - Parameter?

How to pass dictionary or JSON to Invoke-AzVMRunCommand - Parameter? - linux

I have a Input file which I read in a env variable InputConf.
[ { "name": "administrators", "description": "Default group for DSS administrators", "sourceType": "LOCAL", "admin": true, "mayManageUDM": false, "mayCreateProjects": false, "mayCreateProjectsFromMacros": false, "mayCreateProjectsFromTemplates": false, "mayCreateProjectsFromDataikuApps": false, "mayWriteUnsafeCode": false, "mayWriteSafeCode": false, "mayCreateAuthenticatedConnections": false, "mayCreateCodeEnvs": false, "mayCreateClusters": false, "mayDevelopPlugins": false, "mayEditLibFolders": false, "mayManageCodeEnvs": false, "mayManageClusters": false, "mayViewIndexedHiveConnections": false, "mayCreatePublishedAPIServices": false, "mayWriteInRootProjectFolder": false, "mayCreateActiveWebContent": false, "canObtainAPITicketFromCookiesForGroupsRegex": "" }, { "name": "dss-se-npr", "description": "ElevatedSupport", "ldapGroupNames": "dss-se-npr", "sourceType": "LDAP", "admin": true, "mayManageUDM": true, "mayCreateProjects": true, "mayCreateProjectsFromMacros": true, "mayCreateProjectsFromTemplates": true, "mayCreateProjectsFromDataikuApps": true, "mayWriteUnsafeCode": true, "mayWriteSafeCode": true, "mayCreateAuthenticatedConnections": true, "mayCreateCodeEnvs": true, "mayCreateClusters": false, "mayDevelopPlugins": true, "mayEditLibFolders": true, "mayManageCodeEnvs": true, "mayManageClusters": false, "mayViewIndexedHiveConnections": false, "mayCreatePublishedAPIServices": true, "mayWriteInRootProjectFolder": true, "mayCreateActiveWebContent": true, "canObtainAPITicketFromCookiesForGroupsRegex": "" }, { "name": "dss-au-npr", "description": "Auditor", "ldapGroupNames": "dss-au-npr", "sourceType": "LDAP", "admin": false, "mayManageUDM": false, "mayCreateProjects": false, "mayCreateProjectsFromMacros": false, "mayCreateProjectsFromTemplates": false, "mayCreateProjectsFromDataikuApps": false, "mayWriteUnsafeCode": false, "mayWriteSafeCode": false, "mayCreateAuthenticatedConnections": false, "mayCreateCodeEnvs": false, "mayCreateClusters": false, "mayDevelopPlugins": false, "mayEditLibFolders": false, "mayManageCodeEnvs": false, "mayManageClusters": false, "mayViewIndexedHiveConnections": false, "mayCreatePublishedAPIServices": false, "mayWriteInRootProjectFolder": false, "mayCreateActiveWebContent": false, "canObtainAPITicketFromCookiesForGroupsRegex": "" }, { "name": "dss-ba-npr", "description": "BusinessAnalyst", "ldapGroupNames": "dss-ba-npr", "sourceType": "LDAP", "admin": false, "mayManageUDM": false, "mayCreateProjects": false, "mayCreateProjectsFromMacros": false, "mayCreateProjectsFromTemplates": false, "mayCreateProjectsFromDataikuApps": false, "mayWriteUnsafeCode": false, "mayWriteSafeCode": false, "mayCreateAuthenticatedConnections": false, "mayCreateCodeEnvs": false, "mayCreateClusters": false, "mayDevelopPlugins": false, "mayEditLibFolders": false, "mayManageCodeEnvs": false, "mayManageClusters": false, "mayViewIndexedHiveConnections": false, "mayCreatePublishedAPIServices": false, "mayWriteInRootProjectFolder": false, "mayCreateActiveWebContent": false, "canObtainAPITicketFromCookiesForGroupsRegex": "" }, { "name": "dss-ds-npr", "description": "DataScientist", "sourceType": "LDAP", "ldapGroupNames": "dss-ds-npr", "admin": false, "mayManageUDM": false, "mayCreateProjects": false, "mayCreateProjectsFromMacros": false, "mayCreateProjectsFromTemplates": false, "mayCreateProjectsFromDataikuApps": false, "mayWriteUnsafeCode": false, "mayWriteSafeCode": true, "mayCreateAuthenticatedConnections": false, "mayCreateCodeEnvs": false, "mayCreateClusters": false, "mayDevelopPlugins": false, "mayEditLibFolders": false, "mayManageCodeEnvs": false, "mayManageClusters": false, "mayViewIndexedHiveConnections": false, "mayCreatePublishedAPIServices": false, "mayWriteInRootProjectFolder": false, "mayCreateActiveWebContent": false, "canObtainAPITicketFromCookiesForGroupsRegex": "" }, { "name": "dss-dsfe-npr", "description": "DataScientistFeatureEnhancement", "ldapGroupNames": "dss-dsfe-npr", "sourceType": "LDAP", "admin": false, "mayManageUDM": false, "mayCreateProjects": false, "mayCreateProjectsFromMacros": false, "mayCreateProjectsFromTemplates": false, "mayCreateProjectsFromDataikuApps": false, "mayWriteUnsafeCode": false, "mayWriteSafeCode": true, "mayCreateAuthenticatedConnections": false, "mayCreateCodeEnvs": false, "mayCreateClusters": false, "mayDevelopPlugins": true, "mayEditLibFolders": false, "mayManageCodeEnvs": false, "mayManageClusters": false, "mayViewIndexedHiveConnections": false, "mayCreatePublishedAPIServices": false, "mayWriteInRootProjectFolder": false, "mayCreateActiveWebContent": false, "canObtainAPITicketFromCookiesForGroupsRegex": "" }, { "name": "dss-ls-npr", "description": "LeadScientist", "ldapGroupNames": "dss-ls-npr", "sourceType": "LDAP", "admin": false, "mayManageUDM": true, "mayCreateProjects": false, "mayCreateProjectsFromMacros": false, "mayCreateProjectsFromTemplates": false, "mayCreateProjectsFromDataikuApps": false, "mayWriteUnsafeCode": false, "mayWriteSafeCode": true, "mayCreateAuthenticatedConnections": false, "mayCreateCodeEnvs": false, "mayCreateClusters": false, "mayDevelopPlugins": false, "mayEditLibFolders": false, "mayManageCodeEnvs": false, "mayManageClusters": false, "mayViewIndexedHiveConnections": false, "mayCreatePublishedAPIServices": false, "mayWriteInRootProjectFolder": false, "mayCreateActiveWebContent": false, "canObtainAPITicketFromCookiesForGroupsRegex": "" }, { "name": "dss-sp-npr", "description": "Support", "ldapGroupNames": "dss-sp-npr", "sourceType": "LDAP", "admin": false, "mayManageUDM": true, "mayCreateProjects": true, "mayCreateProjectsFromMacros": true, "mayCreateProjectsFromTemplates": true, "mayCreateProjectsFromDataikuApps": true, "mayWriteUnsafeCode": false, "mayWriteSafeCode": true, "mayCreateAuthenticatedConnections": false, "mayCreateCodeEnvs": true, "mayCreateClusters": false, "mayDevelopPlugins": true, "mayEditLibFolders": true, "mayManageCodeEnvs": true, "mayManageClusters": false, "mayViewIndexedHiveConnections": false, "mayCreatePublishedAPIServices": false, "mayWriteInRootProjectFolder": false, "mayCreateActiveWebContent": false, "canObtainAPITicketFromCookiesForGroupsRegex": "" }, { "name": "dss-rm-npr", "description": "ReleaseManager", "ldapGroupNames": "dss-rm-npr", "sourceType": "LDAP", "admin": false, "mayManageUDM": false, "mayCreateProjects": false, "mayCreateProjectsFromMacros": false, "mayCreateProjectsFromTemplates": false, "mayCreateProjectsFromDataikuApps": false, "mayWriteUnsafeCode": false, "mayWriteSafeCode": false, "mayCreateAuthenticatedConnections": false, "mayCreateCodeEnvs": false, "mayCreateClusters": false, "mayDevelopPlugins": false, "mayEditLibFolders": false, "mayManageCodeEnvs": false, "mayManageClusters": false, "mayViewIndexedHiveConnections": false, "mayCreatePublishedAPIServices": false, "mayWriteInRootProjectFolder": false, "mayCreateActiveWebContent": false, "canObtainAPITicketFromCookiesForGroupsRegex": "" }, { "name": "dss-te-npr", "description": "Tester", "ldapGroupNames": "dss-te-npr", "sourceType": "LDAP", "admin": false, "mayManageUDM": false, "mayCreateProjects": false, "mayCreateProjectsFromMacros": false, "mayCreateProjectsFromTemplates": false, "mayCreateProjectsFromDataikuApps": false, "mayWriteUnsafeCode": false, "mayWriteSafeCode": false, "mayCreateAuthenticatedConnections": false, "mayCreateCodeEnvs": false, "mayCreateClusters": false, "mayDevelopPlugins": false, "mayEditLibFolders": false, "mayManageCodeEnvs": false, "mayManageClusters": false, "mayViewIndexedHiveConnections": false, "mayCreatePublishedAPIServices": false, "mayWriteInRootProjectFolder": false, "mayCreateActiveWebContent": false, "canObtainAPITicketFromCookiesForGroupsRegex": "" } ]
I need the variable to be copied to the Linux VM - /dss/data/groups.txt
I call the script using Invoke-AzVMRunCommand using RunShellScript
Invoke-AzVMRunCommand -ResourceGroupName $ResourceGroupName -VMName $VirtualMachineName -CommandId RunShellScript -ScriptPath "$($PSScriptRoot)/scripts/$PatchScript" -Parameter #{InputConfParam=$InputConf}
In the script, I do the below
echo $InputConf > /dss/data/groups.txt
However I do not get the entire string, but only get the first character of the InputString.
This is because of the spaces/tabs in the variable.
Is there a way I can get the entire JSON passed to the shell script? so I can copy the variable to the file on the server?
Thanks

Just did a very crude workaround or replacing the newline chars and spaces and sending the parameter to the shell script and replacing it back in the script.
Blob Storage was a good option - will surely try the approach.

Related

adding analyzers to Azure Search Index using REST API not saving

Having trouble getting the analyzers to save / update on the index. When creating, everything else (the tokenFilters, tokenizers, fields) saves fine, but the analyzers array is always empty?
await client.createOrUpdateIndex(index, { allowIndexDowntime: true });
Creating a new index:
let index = {
name: "test-index",
tokenizers: [{
"odatatype": "#Microsoft.Azure.Search.StandardTokenizerV2",
"name": "test_standard_v2",
"maxTokenLength": 255
}],
fields: [{
"name": "metadata_storage_path",
"type": "Edm.String",
"facetable": false,
"filterable": false,
"key": true,
"retrievable": true,
"searchable": false,
"sortable": false,
"analyzer": null,
"indexAnalyzer": null,
"searchAnalyzer": null,
"synonymMaps": [],
"fields": []
}, {
'name': 'metadata_storage_name',
'type': 'Edm.String',
'facetable': false,
'filterable': false,
'key': false,
'retrievable': true,
'searchable': true,
'sortable': false,
'synonymMaps': [],
'fields': [],
},
{
"name": "partialName",
"type": "Edm.String",
"retrievable": false,
"searchable": true,
"filterable": false,
"sortable": false,
"facetable": false,
"key": false,
"searchAnalyzer": "standardCmAnalyzer",
"indexAnalyzer": "filename_analyzer"
}],
tokenFilters: [{
"name": "nGramCmTokenFilter",
"odatatype": "#Microsoft.Azure.Search.NGramTokenFilterV2",
"minGram": 3,
"maxGram": 20
}],
analyzers: [{
"name": "standardCmAnalyzer",
"odatatype": "#Microsoft.Azure.Search.CustomAnalyzer",
"tokenizer": "test_standard_v2",
"tokenFilters": ["lowercase", "asciifolding"]
},
{
"name": "filename_analyzer",
"odatatype": "#Microsoft.Azure.Search.CustomAnalyzer",
"tokenizer": "test_standard_v2",
"tokenFilters": [
"nGramCmTokenFilter"
]
}],
};
Then creating it:
await client.createOrUpdateIndex(index, { allowIndexDowntime: true });
I noticed no error messages being returned.
EDIT:
Using the sdk #azure/search-documents ^11.1.0

Job Template in Ansible Tower using API Endpoint

How to create a job template in ansible tower using Api Endpoint in nodejs script.

I'm afraid I am not familiar with node.js. But as far as the Ansible part goes, navigate your browser to https://my-ansible-tower/api/v2/job_templates. At the bottom of the page, you will see the options you can set:
{
"name": "",
"description": "",
"job_type": "run",
"inventory": null,
"project": null,
"playbook": "",
"forks": 0,
"limit": "",
"verbosity": 0,
"extra_vars": "",
"job_tags": "",
"force_handlers": false,
"skip_tags": "",
"start_at_task": "",
"timeout": 0,
"use_fact_cache": false,
"host_config_key": "",
"ask_diff_mode_on_launch": false,
"ask_variables_on_launch": false,
"ask_limit_on_launch": false,
"ask_tags_on_launch": false,
"ask_skip_tags_on_launch": false,
"ask_job_type_on_launch": false,
"ask_verbosity_on_launch": false,
"ask_inventory_on_launch": false,
"ask_credential_on_launch": false,
"survey_enabled": false,
"become_enabled": false,
"diff_mode": false,
"allow_simultaneous": false,
"custom_virtualenv": null,
"job_slice_count": 1,
"credential": null,
"vault_credential": null
}
Set those in your node.js script and pust to that URL. That should work.
Good luck!

Unexpected search results from Azure Cognitive Search

I recently developed an index on Azure. I have the following index structure:
{"name": "my_index",
"fields":
[
{"name": "id", "type": "Edm.String", "filterable": true, "key": true, "searchable": true, "sortable": true, "facetable": false},
{"name": "metadata_storage_path", "type": "Edm.String", "searchable": false, "filterable": false, "retrievable": true, "sortable": false, "facetable": false},
{"name": "Name", "type": "Edm.String", "searchable": true, "retrievable": true, "filterable": true, "sortable": false, "facetable": true, "analyzer": "en.microsoft"},
{"name": "Description", "type": "Edm.String", "searchable": true, "retrievable": true, "filterable": false, "sortable": false, "facetable": false, "analyzer": "en.microsoft"},
{"name": "Content", "type": "Edm.String", "searchable": true, "retrievable": true, "filterable": false, "sortable": false, "facetable": false, "analyzer": "en.microsoft"}
}
When I try to search an entire phrase, for example, "cloud platform", I get some top results without any mention of "cloud platform" which is a bit strange. When I then look at the search.score, even the top results have very low score like 0.07. However, I could see the phrases appearing in the documents and I expect to have enough documents containing the phrase.
Does anyone know why that might be the case? Is it because I used the wrong analyzer?
Any potential tests I can try would also be hugely appreciated.

are you querying using REST or SDK, in both cases an example request will help to understand your issue better.
If I were doing this using REST it will be like this
https://<yourserviceName>.search.windows.net/indexes/<yourIndexName>/docs?api-version=2020-06-30&search=*&%24filter=description%20eq%20'cloud platform'
Note to make sure the exact match happens I am using filter instead of search.

Azure search: how to create a search index of complex type for blobs

I have a blob storage that has a number of folders, each folder has a number of pdf documents. I now want to create an azure search index which indexes the data by folder level, but includes a complex type structure (Collection(edm.ComplexType) that allows me to include all the documents. So the index looks like this:
{"name": "index",
"fields":
[
{"name": "id", "type": "Edm.String", "filterable": true, "key": true, "searchable": true, "sortable": true, "facetable": false},
{"name": "folderName", "type": "Collection(Edm.String)", "searchable": true, "retrievable": true, "filterable": true, "sortable": false, "facetable": true, "analyzer": "en.microsoft"},
{"name": "documents", "type": "Collection(Edm.ComplexType)",
"fields": [
{"name": "documentName", "type": "Edm.String", "searchable": true, "retrievable": true, "filterable": true, "sortable": false, "facetable": true, "analyzer": "en.microsoft"},
{"name": "content", "type": "Edm.String", "searchable": true, "retrievable": true, "filterable": false, "sortable": false, "facetable": false, "analyzer": "en.microsoft", "synonymMaps": ["synonymsmap"]},
{"name": "documentType", "type": "Collection(Edm.String)", "searchable": true, "retrievable": true, "filterable": true, "sortable": false, "facetable": true, "analyzer": "en.microsoft"},
{"name": "language", "type": "Collection(Edm.String)", "searchable": true, "retrievable": true, "filterable": true, "sortable": false, "facetable": true, "analyzer": "en.microsoft"}
]
}
]
}
Does anyone know how I should approach this? I have been creating and populating indexes using rest api.
I am thinking maybe I need to create a folder-level index structure and populate the folder-level details from some sql-table before populating the sub-fields with the blobs through skillset and indexer etc?
EDITS:
Maybe my ideas above are completely off-track. What I want to do is to search a term and return folder names based on the aggregate relevancy of documents within folders. Not sure if this is achievable in search or have to be processed afterwards. Any pointers?

Does anyone know how I should approach this? I have been creating and populating indexes using rest api.
A: If you want this structure, then you're right. You'll need to create your index and push data by yourself (rest api is one of the options)
I am thinking maybe I need to create a folder-level index structure and populate the folder-level details from some sql-table before populating the sub-fields with the blobs through skillset and indexer etc?
A: This is not a good idea, when searching using a particular term, you'll need to query all the possible indexes and do the ordering by yourself.
I personally would create a simple structure, which no complex types:
{
"name": "index",
"fields":
[
{"name": "id", "type": "Edm.String", "filterable": true, "key": true, "searchable": true, "sortable": true, "facetable": false},
{"name": "folderName", "type": "Collection(Edm.String)", "searchable": true, "retrievable": true, "filterable": true, "sortable": false, "facetable": true, "analyzer": "en.microsoft"},
{"name": "documentName", "type": "Edm.String", "searchable": true, "retrievable": true, "filterable": true, "sortable": false, "facetable": true, "analyzer": "en.microsoft"},
{"name": "content", "type": "Edm.String", "searchable": true, "retrievable": true, "filterable": false, "sortable": false, "facetable": false, "analyzer": "en.microsoft", "synonymMaps": ["synonymsmap"]},
{"name": "documentType", "type": "Collection(Edm.String)", "searchable": true, "retrievable": true, "filterable": true, "sortable": false, "facetable": true, "analyzer": "en.microsoft"},
{"name": "language", "type": "Collection(Edm.String)", "searchable": true, "retrievable": true, "filterable": true, "sortable": false, "facetable": true, "analyzer": "en.microsoft"}
]
}
want to retrieve all documents by a particular folder?
search=*&$filter=folderName eq 'abc'
want to retrieve a particular documents in a particular folder?
search=*&$filter=folderName eq 'abc' and documentName eq 'x.docx'
want to all documents that contain a particular term?
search=mickey mouse&$orderBy=folderName
simple and effective

JSHint and issue with underscore '_'

Getting the following error when running jshint;
line 4 col 5 Redefinition of '_'.
the code is complaining out is;
var _ = require('lodash');
jshint in project
{
"node": true,
"esnext": true,
"bitwise": true,
"eqeqeq": true,
"immed": true,
"latedef": "nofunc",
"newcap": true,
"noarg": true,
"regexp": true,
"undef": true,
"smarttabs": true,
"asi": true,
"debug": true,
"globals": {
"angular": false,
"_": false
}
}

Updated the .jshintrc to the following which seems to have resolved the problem.
{
"node": true,
"browser": true,
"esnext": true,
"bitwise": false,
"camelcase": false,
"eqeqeq": true,
"immed": true,
"indent": 4,
"latedef": true,
"newcap": true,
"noarg": true,
"quotmark": "single",
"regexp": true,
"undef": true,
"unused": false,
"strict": true,
"trailing": true,
"smarttabs": true,
"globals": {
"jQuery": true,
"angular": true,
"console": true,
"AppConfig": true,
"$": true,
"_": true,
"moment": true,
"module": true,
"inject": true,
"browser": true,
"element": true,
"describe": true,
"before": true,
"beforeEach": true,
"after": true,
"afterEach": true,
"expect": true,
"it": true,
"by": true,
"chai": true
}
}
Also created a .jshintrc-spec
{
"extends": ".jshintrc",
"globals": {
"describe": true,
"it": true,
"before": true,
"beforeEach": true,
"after": true,
"afterEach": true,
"chai": true,
"inject" : true
}
}

A better (and commonly used) way to resolve this is to use double underscores __ while requiring either underscore or the lodash library. No need to make a change to the jshintrc file just for this.

Develop Reference

node.js excel linux python-3.x azure haskell apache-spark rust .htaccess string

How to pass dictionary or JSON to Invoke-AzVMRunCommand - Parameter? - linux

Just did a very crude workaround or replacing the newline chars and spaces and sending the parameter to the shell script and replacing it back in the script. Blob Storage was a good option - will surely try the approach.

Related

adding analyzers to Azure Search Index using REST API not saving

Job Template in Ansible Tower using API Endpoint

Unexpected search results from Azure Cognitive Search

Azure search: how to create a search index of complex type for blobs

JSHint and issue with underscore '_'

Categories

Resources