I am trying to formnulate a regex to get the ids from the below two strings examples:
/drugs/2/drug-19904-5106/magnesium-oxide-tablet/details
/drugs/2/drug-19906/magnesium-moxide-tablet/details
In the first case, I should get 19904-5106 and in the second case 19906.
So far I tried several, the closes I could get is [drugs/2/drug]-.*\d but would return g-19904-5106 and g-19907.
Please any help to get ride of the "g-"?
Thank you in advance.
When writing a regex expression, consider the patterns you see so that you can align it correctly. For example, if you know that your desired IDs always appear in something resembling ABCD-1234-5678 where 1234-5678 is the ID you want, then you can use that. If you also know that your IDs are always digits, then you can refine the search even more
For your example, using a regex string like
.+?-(\d+(?:-\d+)*)
should do the trick. In a python script that would look something like the following:
match = re.search(r'.+?-(\d+(?:-\d+)*)', my_string)
if match:
my_id = match.group(1)
The pattern may vary depending on the depth and complexity of your examples, but that works for both of the ones you provided
This is the closest I could find: \d+|.\d+-.\d+
I have several strings that I just want to get a subset of like so:
my-bucket-customer-staging-ie-app-data
my-bucket-customer2-longname-prod-uk-app-data
and I just need to get the customers name from the string, so with the above examples I'd be left with
customer
customer2-longname
There's probably a simple way of doing this with regex although I've failed miserably in my attempts.
I'm able to strip the first part of the string using
trimprefix("my-bucket-customer-staging-ie-app-data", "my-bucket-")
trimprefix("my-bucket-customer-longname-prod-uk-app-data", "my-bucket-")
resulting in
customer-staging-ie-app-data
customer-longname-prod-uk-app-data
However Terraform's trimsuffix won't work as there can be several different regions/environments used.
What I'd like to do is slice the string and ignore the last 4 elements, which should then return the customer name regardless of whether it contains an additional delimiter in it.
Something like this captures the customer, however fails for long customer names:
element(split("-",trimprefix("my-bucket-customer-staging-uk-app-data", "my-bucket-")), length(split("-",trimprefix("my-bucket-customer-staging-uk-app-data", "my-bucket-")))-5)
customer
and is also quite messy.
Is there a more obvious solution I'm missing
I believe it's what regex is for.
> try(one(regex("\\w*-\\w*-(\\w*(?:-\\w*)*)-\\w*-\\w*-\\w*-\\w*","my-bucket-customer-staging-ie-app-data")),"")
"customer"
> try(one(regex("\\w*-\\w*-(\\w*(?:-\\w*)*)-\\w*-\\w*-\\w*-\\w*","my-bucket-customer2-longname-prod-uk-app-data")),"")
"customer2-longname"
> try(one(regex("\\w*-\\w*-(\\w*(?:-\\w*)*)-\\w*-\\w*-\\w*-\\w*","my-bucket-customer2-longname-even-longer-prod-uk-app-data")),"")
"customer2-longname-even-longer"
Reference: https://www.terraform.io/language/functions/regex
This question already has an answer here:
How to match a sentence in Lua
(1 answer)
Closed 1 year ago.
Been stuck on this for over a day.
I'm trying to use gsub to extract a portion of an input string. The exact pattern of the input varies in different cases, so I'm trying to use a variable to represent that pattern, so that the same routine - which is otherwise identical - can be used in all cases, rather than separately coding each.
So, I have something along the lines of:
newstring , n = oldstring:gsub(matchstring[i],"%1");
where matchstring[] is an indexed table of the different possible pattern matches, set up so that "%1" will match the target sequence in each matchstring[].
For instance, matchstring[1] might be
"\[User\] <code:%w*>([^<]*)<\\code>.*" -- extract user name from within the <code>...<\code>
while matchstring[2] could be
"\[World\] (%w)* .*" -- extract user name as first word after prefix '[World] '
and matchstring[3] could be
"<code:%w*>([^<]*)<\\code>.*" -- extract username from within <code>...<\code> at start
This does not work.
Yet when, debugging one of the cases, I replace matchstring[i] with the exact same string -- only now passed as a string literal rather than saved in a variable -- it works.
So.. I'm guessing there must be some 'processing' of the string - stripping out special characters or something - when it's sent as a variable rather than a string literal ... but for the life of me I can't figure out how to adjust the matchstring[] entries to compensate!
Help much appreciated...
FACEPALM
Thankyou, Piglet, you got me on the right track.
Given how this particular platform processes & passes strings, anything within <...> needed the escape character \ for downstream use, but of course - duh - for the lua gsub's processing itself it needed the standard %
much obliged
I get the list of domains on a system and I need to replace only the patterns which contain "domain\username" with '*'.
As of now I am able to do mask the domain names with * using string.gsub() but What pattern should I add to make sure any presence of domain\username is replaced with *
Example:
If on the system there are 2 domains test.com and work-user.com and users as admin and guest a file has the following details:
User tried to login from TEST\admin; but should have logged in from work-user\user1, No logs present for testing\guest, account.
The domain test.com and WORK-USER.org are active and TESTING domain in inactive.
Then the output should look like this:
User tried to login from *********; but should have logged in from ********\user1, No logs present for testing\*****, account.
The domain ****.com and *********.org are active and TESTING domain in inactive.
Since Testing and user1 are not the domain and username on that system, they should not get replaced.
I have the logic to replace the username and domain name independently in any given format, but when it is the format of domain\username I am not able to replace it.
I have to add some logic\pattern after I get the domain name so it matches the above requirement.
Can you please let me know how to proceed?
I tried the below code:
test_string="User tried to login from TEST\\admin; but should have logged in from work-user\\user1, No logs present for testing\\guest, account. The domain test.com and WORK-USER.org are active and TESTING domain in inactive"
s= "test"
t=( string.gsub(s.."$DNname", "%$(%w+)", {DNname="\\([%w_]+)"}) )
n=( string.gsub(s.."$DNname", "%$(%w+)", {DNname="\\([%a%d]+)([%;%,%.%s]?)"}) ) print (t)
print(n)
r=string.match(test_string,t)
res=string.match(test_string,n)
print(r)
print(res)
It is printing nil, and is not able to match any pattern
First let's talk about why your code doesn't work.
For one thing, your patterns both have a backslash in them, so you are right away missing anything without a backslash:
print(t) -- test\([%w_]+)
print(n) -- test\([%a%d]+)([%;%,%.%s]?)
But there is also another problem. The only thing with a backslash that ought to match in your test message is TEST\admin. But here TEST is all uppercase, and pattern matching is case sensitive, so you will not find it.
The first part of the answer, then, is to make a case-insensitive pattern. This can be done as follows:
s= "[Tt][Ee][Ss][Tt]"
Here I have replaced each letter with the character class that will match either the uppercase or lowercase letter.
What happens if we look for this pattern in the original message, though? We will have an unfortunate problem: we will find testing and TESTING. It looks like you may have already encountered this problem as you wrote "([%;%,%.%s]?)".
The better way to do this is the frontier pattern. (Note that the frontier pattern is an undocumented feature in Lua 5.1. I'm not sure if it is in Lua 5.0 or not. It became a documented feature in Lua 5.2.)
The frontier pattern takes a character set and will only match spaces between characters where the previous character is not in the set and the next character is in the set. It sounds complicated, but basically it lets you find the beginnings or endings of words.
To use the frontier pattern, we need to figure out what a domain or username might look like. We may not be able to do this perfectly, but, in practice, being overly greedy should be fine.
s = "%f[%w-][Tt][Ee][Ss][Tt]%f[^%w-]"
This new pattern will match "TEST" and "test", but will not match "TESTING" or "testing".
Before proceeding, let's look at a problem that might occur with a domain like your "work-user". The character "-" has a special meaning in patterns, so we must escape it. All special characters can be escaped by adding a "%" in front. So, our work-user pattern would look like:
s = "%f[%w-][Ww][Oo][Rr][Kk]%-[Uu][Ss][Ee][Rr]%f[^%w-]"
Well, these kind of patterns are sort of awful to write out, so let's try to write a function to do it for us:
function string_to_pattern(str, frontier_set, ci)
-- escape magic characters
str = str:gsub("[][^$()%%.*+-?]", "%%%0")
if ci then
-- make the resulting pattern case-insensitive
str = str:gsub("%a", function(letter)
return "["..letter:upper()..letter:lower().."]"
end)
end
if frontier_set then
str = "%f["..frontier_set.."]"..str.."%f[^"..frontier_set.."]"
end
return str
end
print(string_to_pattern("work-user", "%w-", true))
-- %f[%w-][Ww][Oo][Rr][Kk]%-[Uu][Ss][Ee][Rr]%f[^%w-]
I'll go ahead a mention the corner case now: this pattern will not match "-work-user" or "work-user-". This may be okay or not depending on what kind of messages get generated. You could take "-" out of frontier set, but then you would match e.g. "my-work-user". You can decide if this matters, but I haven't thought how to solve it with Lua's pattern matching language.
Now, how do we replace a match with *'s? This part is pretty easy. The built-in string.gsub function will allow us to replace matches of our patterns with other strings. We just need to generate a replacement string that consists of as many *'s as characters.
function string_to_stars(str)
return ("*"):rep(str:len())
end
local pattern = string_to_pattern("test", "%w-", true)
print( (test_string:gsub(pattern, string_to_stars)) )
Now, there's a final problem. We can match users in the same we match domains. For example:
-- note that different frontier_set here
-- I don't know what the parameters for your usernames are,
-- but this matches your code
local pattern = string_to_pattern("admin", "%w_", true)
print( (test_string:gsub(pattern, string_to_stars)) )
However, even if we replace all the domains and usernames separately, the backslash between "TEST" and "admin" in "TEST\admin" will not be replaced. We could do a hack like this:
test_string:gsub("%*\\%*","***")
This would replace "**" with "***" in the final output. However, this is not quite robust because it could replace a "**" that was in the original message and not a result of our processing. To do things properly, we would have to iterate over all domain+user pairs and do something like this:
test_string:gsub(domain_pattern .. "\\" .. user_pattern, string_to_stars)
Note that this must be done before any other replacements, as otherwise the domain and username will have already been replaced, and can no longer be matched.
Now that the problem is solved in that way, let me suggest an alternative approach that reflects something more like what I would write from scratch. I think it is probably simpler and more readable. Instead of using pattern matching to find our domains and usernames exactly, let's instead just match tokens that could be domains or usernames and then check if they match exactly later.
local message = -- broken into multiple lines only for
-- formatting reasons
"User tried to login from TEST\\admin; but should "
.."have logged in from work-user\\user1, No logs present "
.."for testing\\guest, account. The domain test.com and "
.."WORK-USER.org are active and TESTING domain in inactive"
-- too greedy, but may not matter in your case
local domain_pattern = "%w[%w-]*"
-- again, not sure
local user_pattern = "[%w_]+"
-- for case-insensitivity, call :lower before inserting into the set
local domains = {["test"]=true, ["work-user"]=true}
local users = {["admin"]=true, ["guest"]=true}
local pattern = "(("..domain_pattern..")\\("..user_pattern.."))"
message = message:gsub(pattern, function(whole, domain, user)
-- only call lower if case-insensitive
if domains[domain:lower()] and users[user:lower()] then
return string_to_stars(whole)
else
return whole
end
end)
local function replace_set(message, pattern, set, ci)
return (message:gsub(pattern, function(str)
if ci then str = str:lower() end
if set[str] then
return string_to_stars(str)
else
return str
end
end))
end
message = replace_set(message, domain_pattern, domains, true)
message = replace_set(message, user_pattern, users, true)
print(message)
Notice how simple the patterns are in this example. We no longer need case-insensitive character classes like "[Tt]" because the case-insensitivity is checked after the matching by forcing both strings to be lowercase with string.lower (which may not be maximally efficient, but, hey, this is Lua). We no longer need to use the frontier pattern because we are guaranteed to get full words because of greedy matching. The backslash case is still weird, but I've handled it in the same "robust" way as I suggested above.
A final note: I don't know exactly why your doing this, but I can maybe guess that it is to prevent someone from seeing domains or usernames. Replacing them with *'s is not necessarily the best way to go. First, doing matching in these ways could be problematic if your messages are (for example) delimited with letters. This seems unlikely for user-friendly messages, but I don't know whether that's something you should count on when security is at stake. Another thing is that you are not hiding the lengths of the domains or usernames. This can also be a major source of insecurity. For example, a user might reasonably guess that ***** is "admin".
I'm getting a garbled JSON string from a HTTP request, so I'm looking for a temp solution to select the JSON string only.
The request.params() returns this:
[{"insured_initials":"Tt","insured_surname":"Test"}=, _=1329793147757,
callback=jQuery1707229194729661704_1329793018352
I would like everything from the start of the '{' to the end of the '}'.
I found lots of examples of doing similar things with other languages, but the purpose of this is not to only solve the problem, but also to learn Scala. Will someone please show me how to select that {....} part?
Regexps should do the trick:
"\\{.*\\}".r.findFirstIn("your json string here")
As Jens said, a regular expression usually suffices for this. However, the syntax is a bit different:
"""\{.*\}""".r
creates an object of scala.util.matching.Regex, which provides the typical query methods you may want to do on a regular expression.
In your case, you are simply interested in the first occurrence in a sequence, which is done via findFirstIn:
scala> """\{.*\}""".r.findFirstIn("""[{"insured_initials":"Tt","insured_surname":"Test"}=, _=1329793147757,callback=jQuery1707229194729661704_1329793018352""")
res1: Option[String] = Some({"insured_initials":"Tt","insured_surname":"Test"})
Note that it returns on Option type, which you can easily use in a match to find out if the regexp was found successfully or not.
Edit: A final point to watch out for is that the regular expressions normally do not match over linebreaks, so if your JSON is not fully contained in the first line, you may want to think about eliminating the linebreaks first.