I have the following paragraph that arrives in an email and I want to take only those that are by CI
CI: CI14710 CI14727 CI14732 CI14741 OC14735 CI14744 Ends\n\n
I am trying to use the following regular expression
CI: (?<CI>[A-Za-z0-9_-]{1,})
but I only take the first
I tried with this other expression but the logstash fails me
CI: (?<CI>[A-Za-z0-9_-]{1,}.*)\\n\\n
it is not seen but they go dobele \ n
I tried the below pattern in Grok debugger and able to collect all the words starting with CI.
CI: (?<CI>[A-Za-z0-9_-]{1,}.*) %{GREEDYDATA:data}
Output screenshot:
You can drop the field data using the drop filter
Related
I'm wondering if you can use wildcard characters with tags to get all tagged scenarios/features that match a certain pattern.
For example, I've used 17 unique tags on many scenarios throughout many of my feature files. The pattern is "#jira=CIS-" followed by 4 numbers, like #jira=CIS-1234 and #jira=CIS-5678.
I'm hoping I can use a wildcard character or something that will find all of the matches for me.
I want to be able to exclude them from being run, when I run all of my features/scenarios.
I've tried the follow:
--tags ~#jira
--tags ~#jira*
--tags ~#jira=*
--tags ~#jira=
Unfortunately none have given my the results I wanted. I was only able to exclude them when I used the exact tag, ex. ~#jira=CIS-1234. It's not a good solution to have to add each single one (of the 17 different tags) to the command line. These tags can change frequently, with new ones being added and old ones being removed, plus it would make for one real long command.
Yes. First read this - there is this un-documented expression-language (based on JS) for advanced tag selction based on the #key=val1,val2 form: https://stackoverflow.com/a/67219165/143475
So you should be able to do this:
valuesFor('#jira').isPresent
And even (here s will be a string, on which you can even do JS regex if you know how):
valuesFor('#jira').isEach(s => s.startsWith('CIS-'))
Would be great to get your confirmation and then this thread itself can help others and we can add it to the docs at some point.
I want to run pipeline specific branches in GitLab like the branch names are sprint_100, Sprint-1,SPRINT-202.
How should I give commands in rules?
(/^SPRINT_[0-9]+\.[0-9]+$/)||(/^Sprint_[0-9]+\.[0-9]+$/)||(/^Sprint_[0-9]+\.[0-9]+$/)
I used this, but it doesn't work. Kindly help me to sort it out this. Thank you!
This rules clause should work for your example to run a pipeline only branch names like sprint_100, Sprint-1 or SPRINT-202.
The regex will match case-insensitive on branch names starting with 'sprint' followed by either '-' or '_' and an unlimited amount of digits.
I am no regex expert so this regex can likely be improved.
rules:
- if: '$CI_COMMIT_BRANCH =~ /^SPRINT[-_][0-9]+/i'
I am using the downloading latest artifact feature.
For me it is not clear, how the job name I need to pass is created: my job name contains e.g. spaces, equal signs and brackets:
build win: [USE_PYTHON=ON]
I know that spaces are replaced by +-signs but what about the others characters?
Changing the job name is not an option because I use the matrix-feature and it creates names like these.
Thanks a lot for your help!
Example ci yaml:
build win:
...
parallel:
matrix:
- USE_PYTHON: ["USE_PYTHON=ON", "USE_PYTHON=OFF"]
You can use ASCII encoding like for space %20.
Find them here
https://www.w3schools.com/tags/ref_urlencode.ASP
I have two log lines like this:
[2020-04-01][14:57:31]E: Step 8/13: Main workflow (Python) (8m:48s)
[2020-04-01][15:14:02]W: Cannot find Latest build with tag: 'ArtifactSizeBaseline' to calculate metric 'total artifacts size'.
and a matching string like this
%{DATE:EventDate}\]\[%{TIME:EventTime}\](\s+)?%{WORD:Loglevel}:(\s+)?%{DATA:Step}:(\s+)%{GREEDYDATA:EventMessage}
My output should look like this for statement one:
{'EventDate':'2020-04-01', 'EventTime':'14:57:31', 'LogLevel':'E', 'Step':'Step 8/13', 'EventMessage':'Main workflow (Python) (8m:48s)'}
Ideally, the second logline does not contain a step. So, the output should look like
{'EventDate':'2020-04-01', 'EventTime':'15:14:02', 'LogLevel':'W', 'Step':'', 'EventMessage':'Cannot find Latest build with tag: 'ArtifactSizeBaseline' to calculate metric 'total artifacts size'.'}
But what I get is this
{'EventDate':'2020-04-01', 'EventTime':'15:14:02', 'LogLevel':'W', 'Step':'Cannot find Latest build with tag: ', 'EventMessage':''ArtifactSizeBaseline' to calculate metric 'total artifacts size'.'}
Is there a way for the match string to differentiate between these two log lines?
This regex matches both lines:
%{DATE:EventDate}\]\[%{TIME:EventTime}\](\s+)?%{WORD:Loglevel}:\s+((?=Step\s\b)%{DATA:Step}:)?\s?%{GREEDYDATA:EventMessage}
It uses a positive lookahead as well as the optional DATA-extraction if the word "Step" is found followed by a whitespace and a number.
Tested for both lines on this site:
https://grokconstructor.appspot.com/do/match
I hope I could help you.
I am using Logstash (with Kibana as the UI). I would like to extract some fields from my logs so that I can filter by them on the LHS of the UI.
A sample line from my log looks like this:
2013-07-04 00:27:16.341 -0700 [Comp40_db40_3720_18_25] client_login=C-316fff97-5a19-44f1-9d87-003ae0e36ac9 ip_address=192.168.4.1
In my logstash conf file, I put this:
filter {
grok {
type => "mylog"
pattern => "(?<CLIENT_NAME>Comp\d+_db\d+_\d+_\d+_\d+)"
}
}
Ideally, I would like to extract Comp40_db40_3720_18_25 (the number of digits can vary, but will always be at least 1 in each section separated by _) and client_login (can also be client_logout). Then, I can search for CLIENT_NAME=Comp40... CLIENT_NAME=Comp55, etc.
Am I missing something in my config to make this a field that I can use in Kibana?
Thanks!
If you are having any difficulty getting the pattern to match correctly, using the Grok Debugger is a great solution.
For your given problem you could just separate out your search data into another variable, and save the additional varying digits in another (trash) variable.
For example:
(?<SEARCH_FIELD>Comp\d+)%{GREEDYDATA:trash_variable}]
(Please use the Grok Debugger on the above pattern)