I am trying to figure out how I can use Facter facts inside a Ruby template erb file configured with Puppet.
Sample Ruby template variable
zk.l.conn=
Expected config file output from Puppet
zk.l.conn=ip-xx-31-xx-xxx.ec2.internal:2181,ip-xxx-31-xx- xxx.ec2.internal:2181,ip-172-xxx-xxx-xx.ec2.internal:2181
Facter fact data:
"zk-internal": [
{
"host": "ip-xx-31-xx-xxx.ec2.internal",
"port": 2181,
"priority": 2,
"weight": 10
},
{
"host": "ip-xxx-31-xx-xxx.ec2.internal",
"port": 2181,
"priority": 3,
"weight": 10
},
{
"host": "ip-172-xxx-xxx-xx.ec2.internal",
"port": 2181,
"priority": 1,
"weight": 10
}
],
In short:
zk.1.conn=<%= #facts['zk-internal'].map { |h| "#{h['host']}:#{h['port']}" }.join(',') %>
#facts['zk-internal'] lets you access the structured fact value, used because #zk-internal wouldn't be a valid variable name due to the hyphen.
.map { |h| "#{h['host']}:#{h['port']}" } iterates over every element and returns new strings containing "host:port" from each element, so you have an array of hosts/ports returned.
.join(',') returns a string from the array with each element comma-separated.
This outputs:
zk.1.conn=ip-xx-31-xx-xxx.ec2.internal:2181,ip-xxx-31-xx-xxx.ec2.internal:2181,ip-172-xxx-xxx-xx.ec2.internal:2181
(Tested on Puppet 4.7)
Related
I'm trying to create a discussion note on a merge request on a certain line of a file with the GitLab api using this endpoint: https://docs.gitlab.com/ee/api/discussions.html#create-new-merge-request-thread
Part of the payload asks for a line_code
Attribute
Type
Required
Description
position[line_range][start][line_code]
string
yes
Line code for the start line
When I issue a POST I get a response with:
"message": "400 (Bad request) \"Note {:line_code=>[\"can't be blank\", \"must be a valid line code\"], :position=>[\"is incomplete\"]}\" not given"
What is this line_code? Is it some kind of calculated value? The documentation is rather vague here.
When I issue a GET for all the current notes on a merge_request I can see some notes have this line_code (see below). I'm trying to figure out how to create that value for new notes.
{
"id": 89,
"type": "DiffNote",
"body": "4",
"attachment": null,
"author": {
"id": 6,
"name": "brian c",
"username": "bc",
"state": "active",
"avatar_url": "https://www.gravatar.com/avatar/f590a9cf57136732dd0cb5z9b1563390?s=80&d=identicon",
"web_url": "http://gitlab.mycompany.us/thisIsMe"
},
"created_at": "2021-01-11T21:46:23.861Z",
"updated_at": "2021-01-11T21:46:23.861Z",
"system": false,
"noteable_id": 21,
"noteable_type": "MergeRequest",
"position": {
"base_sha": "3bf8094f0d54fc70a66698bd582f25c77243de3b",
"start_sha": "3bf8094f0d54fc70a66698bd582f25c77243de3b",
"head_sha": "a10e73cf84eae38286df56f4b58fa221d7eefc44",
"old_path": "b.txt",
"new_path": "b.txt",
"position_type": "text",
"old_line": null,
"new_line": 4,
"line_range": {
"start": {
"line_code": "aceba96ffdf13ce4cd4171c0248420cc03108ef0_0_4",
"type": "new",
"old_line": null,
"new_line": 4
},
"end": {
"line_code": "aceba96ffdf13ce4cd4171c0248420cc03108ef0_0_4",
"type": "new",
"old_line": null,
"new_line": 4
}
}
},
"resolvable": true,
"resolved": false,
"resolved_by": null,
"confidential": false,
"noteable_iid": 3,
"commands_changes": {}
},
Line code is hash of the file name + underscore + old line number + underscore + new line number
The documentation is wrong. line_code is required only if you are using position.line_range which is only required for adding diff note spanning multiple lines of diff. You don't need to deal with line_code for single-line diff notes. So it is not a required parameter. You can just use position.old_line or position.new_line.
That represents the line in the file you want the comment to appear on. For Merge Requests, comments can either be on the code or general discussions (though the API names seem to be backwards).
To add a general discussion comment, you can use the Notes API: https://docs.gitlab.com/ee/api/notes.html#create-new-merge-request-note. This will look like the comment here: https://gitlab.com/gitlab-org/gitlab/-/merge_requests/52673#note_495396729
To add a comment to the changed code in a Merge Request, you can use the Discussions API here: https://docs.gitlab.com/ee/api/discussions.html#create-new-merge-request-thread. This operation has options to set the file path and line number a comment should start on, a range that a comment applies to, etc. This is an example of a comment on the code itself: https://gitlab.com/gitlab-org/gitlab/-/merge_requests/52673/diffs#2eda52c44979de93f257b305ada778372eacba0b_6_5
It's not a bug, I meet this scene before.
maybe your "old_line" is not really exist.
just let set old_line and old_path to null
it will work normally.
I'm trying to best index contents of e-mail messages, subjects and email addresses. E-mails can contain both text and HTML representation. They can be in any language so I can't use language specific analysers unfortunately.
As I am new to this I have many questions:
First I used Standard Lucene analyser but after some testing and
checking what each analyser does I switched to using "simple"
analyser. Standard one didn't allow me to search by domain in
user#domain.com (It sees user and domain.com as tokens). Is "simple" the best I can use in my case?
How can I handle HTML contents of e-mail? I thought this should be
possible to do it in Azure Search but right now I think I would need
to strip HTML tags myself.
My users aren't tech savvy and I assumed "simple" query type will be
enough for them. I expect them to type word or two and find messages
containing this word/containing words starting with this word. From my tests it looks I need to append * to their queries to get "starting with" to work?
It would help if you included an example of your data and how you index and query. What happened, and what did you expect?
The standard Lucene analyzer will work with your user#domain.com example. It is correct that it produces the tokens user and domain.com. But the same happens when you query, and you will get records with the tokens user and domain.com.
CREATE INDEX
"fields": [
{"name": "Id", "type": "Edm.String", "searchable": false, "filterable": true, "retrievable": true, "sortable": true, "facetable": false, "key": true, "indexAnalyzer": null, "searchAnalyzer": null, "analyzer": null, "synonymMaps": [] },
{"name": "Email", "type": "Edm.String", "filterable": true, "sortable": true, "facetable": false, "searchable": true, "analyzer": "standard"}
]
UPLOAD
{
"value": [
{
"#search.action": "mergeOrUpload",
"Id": "1",
"Email": "user#domain.com"
},
{
"#search.action": "mergeOrUpload",
"Id": "2",
"Email": "some.user#some-domain.com"
},
{
"#search.action": "mergeOrUpload",
"Id": "3",
"Email": "another#another.com"
}
]
}
QUERY
Query, using full and all.
https://{{SEARCH_SVC}}.{{DNS_SUFFIX}}/indexes/{{INDEX_NAME}}/docs?search=user#domain.com&$count=true&$select=Id,Email&searchMode=all&queryType=full&api-version={{API-VERSION}}
Which produces results as expected (all records containing user and domain.com):
{
"#odata.context": "https://<your-search-env>.search.windows.net/indexes('dg-test-65392234')/$metadata#docs(*)",
"#odata.count": 2,
"value": [
{
"#search.score": 0.51623213,
"Id": "1",
"Email": "user#domain.com"
},
{
"#search.score": 0.25316024,
"Id": "2",
"Email": "some.user#some-domain.com"
}
]
}
If your expected result is to only get the record above where the email matches completely, you could instead use a phrase search. I.e. replace the search parameter above with search="user#domain.com" and you would get:
{
"#search.score": 0.51623213,
"Id": "1",
"Email": "user#domain.com"
}
Alternatively, you could use the keyword analyzer.
ANALYZE
You can compare the different analyzers directly via REST. Using the keyword analyzer on the Email property will produce a single token.
{
"text": "some-user#some-domain.com",
"analyzer": "keyword"
}
Results in the following tokens:
"tokens": [
{
"token": "some-user#some-domain.com",
"startOffset": 0,
"endOffset": 25,
"position": 0
}
]
Compared to the standard tokenizer, which does a decent job for most types of unstructured content.
{
"text": "some-user#some-domain.com",
"analyzer": "standard"
}
Which produces reasonable results for cases where the email address was part of some generic text.
"tokens": [
{
"token": "some",
"startOffset": 0,
"endOffset": 4,
"position": 0
},
{
"token": "user",
"startOffset": 5,
"endOffset": 9,
"position": 1
},
{
"token": "some",
"startOffset": 10,
"endOffset": 14,
"position": 2
},
{
"token": "domain.com",
"startOffset": 15,
"endOffset": 25,
"position": 3
}
]
SUMMARY
This is a long answer already, so I won't cover your other two questions in detail. I would suggest splitting them to separate questions so it can benefit others.
HTML content: You can use a built-in HTML analyzer that strips HTML tags. Or you can strip the HTML yourself using custom code. I typically use Beautiful Soup for cases like these or simple regular expressions for simpler cases.
Wildcard search: Usually, users don't expect automatic wildcards appended. The only application that does this is the Outlook client, which destroys precision. When I search for "Jan" (a common name), I annoyingly get all emails sent in January(!). And a search for Dan (again, a name), I also get all emails from Danmark (Denmark).
Everything in search is a trade-off between precision and recall. In your first example with the email address, your expectation was heavily geared toward precision. But, in your last wildcard question, you seem to prefer extreme recall with wildcards on everything. It all comes down to your expectations.
Not sure if this functionality exists. I'm trying to transform a list of comma separated IP addresses from the Azure DevOps build parameters into an array of objects. So far it's only splitting a comma separated list into an array of strings, but the template needs an array of objects.
The parameter value is a comma separated list of IP Addresses.
e.g. "192.168.0.1,192.168.0.2/32,127.0.0.1"
The ARM template would look like:
"variables": {
"ipaddresses": "[split(parameters('ipaddresses'), ',')]"
},
"resources": [
...
"ipRestrictions": "[stringArrToObjArr(variables('ipaddresses'))]" <--
...
]
And ideally function with the arrow above would yield a value for ipRestictions would be something like:
[
{
"ipAddress": "192.168.0.1"
},
{
"ipAddress": "192.168.0.2/32"
},
{
"ipAddress": "127.0.0.1"
},
]
you can use copy() function to do that:
"variables": {
"ipaddresses": "[split(parameters('ipaddresses'), ',')]"
"copy": [
{
"name": "myVariable",
"count": "[length(variables('ipaddresses'))]",
"input": {
"ipAddress": "[variables('ipaddresses')[copyIndex('myVariable')]]"
}
}
]
},
this would return the desired object into a variable called myVariable. if you want to rename it >> don't forget to rename it inside copyIndex() as well
I am not geeting output and error
------Exception-------
Class: Kitchen::ActionFailed
Message: 1 actions failed."
cookbook/test/integration/nodes
Json file
{
"id": "hive server",
"chef_type": "node",
"environment": "dev",
"json_class": "Chef::Node",
"run_list": [],
"automatic": {
"hostname": "test.net",
"fqdn": "127.0.0.1",
"name": "test.net",
"ipaddress": "127.0.0.1",
"node_zone": "green",
"roles": []
},
"attributes": {
"hiveserver": "true"
}
}
Recipe
hiveNodes = search(:node, "hiveserver:true AND environment:node.environment AND node_color:node["node_color"])
# hiveserverList = ""
# hiveNodes.each |hnode| do
# hiveserverList += hnode
#end
#file '/tmp/test.txt' do
# content '#{hiveserverList}'
#end
I think you mean to be using "hiveserver:true AND chef_environment:#{node.chef_environment} AND node_color:#{node["node_color"]}" as your search string. The #{} syntax is how you embed a Ruby expression value in to a string. Also for reasons of complex backwards compat, the environment on a node is called chef_environment.
[updated 17:15 on 28/09]
I'm manipulating json data of type:
[
{
"id": 1,
"title": "Sun",
"seeAlso": [
{
"id": 2,
"title": "Rain"
},
{
"id": 3,
"title": "Cloud"
}
]
},
{
"id": 2,
"title": "Rain",
"seeAlso": [
{
"id": 3,
"title": "Cloud"
}
]
},
{
"id": 3,
"title": "Cloud",
"seeAlso": [
{
"id": 1,
"title": "Sun"
}
]
},
];
After inclusion in the database, a node.js search using
db.documents.query(
q.where(
q.collection('test films'),
q.value('title','Sun')
).withOptions({categories: 'none'})
)
.result( function(results) {
console.log(JSON.stringify(results, null,2));
});
will return both the film titled 'Sun' and the films which have a seeAlso/title property (forgive the xpath syntax) = 'Sun'.
I need to find 1/ films with title = 'Sun' 2/ films with seeAlso/title = 'Sun'.
I tried a container query using q.scope() with no success; I don't find how to scope the root object node (first case) and for the second case,
q.where(q.scope(q.property('seeAlso'), q.value('title','Sun')))
returns as first result an item which matches all text inside the root object node
{
"index": 1,
"uri": "/1.json",
"path": "fn:doc(\"/1.json\")",
"score": 137216,
"confidence": 0.6202662,
"fitness": 0.6701325,
"href": "/v1/documents?uri=%2F1.json&database=Documents",
"mimetype": "application/json",
"format": "json",
"matches": [
{
"path": "fn:doc(\"/1.json\")/object-node()",
"match-text": [
"Sun Rain Cloud"
]
}
]
},
which seems crazy.
Any idea about how doing such searches on denormalized json data?
Laurent:
XPaths on JSON are supported by MarkLogic.
In particular, you might consider setting up a path range index to match /title at the root:
http://docs.marklogic.com/guide/admin/range_index#id_54948
Scoped property matching required either filtering or indexed positions to be accurate. An alternative is to set up another path range index on /seeAlso/title
For the match issue it would be useful to know the MarkLogic version and to see the entire query.
Hoping that helps,