Python regex search until specific word and exclude everything behind it - python-3.x
I have a script that always have the "get the" and the "get" in a string.
The "ONE TWO THREE" can vary, like it also can be "THIRTEEN FORTY" or "SIX". After these variations there will always be a 2nd "get".
I have the following code:
variable = 'get the ONE TWO THREE get FOUR FIVE'
myVariable = re.compile(r'(?<=get the) .*')
myVariableSearch = myVariable.search(variable)
mySearchGroup = myVariableSearch.group()
print(mySearchGroup)
#prints ONE TWO THREE get FOUR FIVE
I want my script to exclude the 2nd "get" and everything behind it. My desired result is to be just the "ONE TWO THREE".
How do I exclude this? Any help would be appreciated!
You can use
\bget\s+the\s+(.*?)(?=\s*\bget\b|$)
See the regex demo.
Details
\bget\s+the\s+ - whole word get, 1+ whitespaces, the, 1+ whitespaces
(.*?) - Group 1:
(?=\s*\bget\b|$) - a positive lookahead that requires 0+ whitespaces and then a whole word get, or end of string immediately on the right of the current location.
See the Python demo:
import re
variable = 'get the ONE TWO THREE get FOUR FIVE'
myVariableSearch = re.search(r'\bget\s+the\s+(.*?)(?=\s*\bget\b|$)', variable)
mySearchGroup = ''
if myVariableSearch:
mySearchGroup = myVariableSearch.group(1)
print(mySearchGroup)
# => ONE TWO THREE
Related
Need guidance with Regular Expression in Python
I need help with one of my current tasks wherein i am trying to pick only the table names from the query via Python So basically lets say a query looks like this Create table a.dummy_table1 as select a.dummycolumn1,a.dummycolumn2,a.dummycolumn3 from dual Now i am passing this query into Python using STRINGIO and then reading only the strings where it starts with "a" and has "_" in it like below table_list = set(re.findall(r'\ba\.\w+', str(data))) Here data is the dataframe in which i have parsed the query using StringIO now in table_list i am getting the below output a.dummy_table1 a.dummycolumn1 a.dummycolumn2 whereas the Expected output should have been like a.dummy_table1 <Let me know how we can get this done , have tried the above regular expression but that is not working properly> Any help on same would be highly appreciated
Your current regex string r"\ba.\w+" simply matches any string which: Begins with "a" (the "\ba" part) Followed by a period (the "." part) Followed by 1 or more alphanumeric characters (the "\w+" part). If I've understood your problem correctly, you are looking to extract from str(data) any string fragments which match this pattern instead: Begins with "a" Followed by a period Followed by 1 or more alphanumeric characters Followed by an underscore Followed by 1 or more alphanumeric characters Thus, the regular expression should have "_\w+" added to the end to match criteria 4 and 5: table_list = set(re.findall(r"\ba\.\w+_\w+", str(data)))
Strip characters to the left of a specific character in a pandas column
I have the following data: key German 0 0:- Profile 1 1 1:- Archetype Realist*in 2 2:- RIASEC Code: R- Realistic 3 3:- Subline Deine Stärke? Du bleibst dir selber treu. 4 4:- Copy Dein Erfolg basiert auf deiner praktischen Ver... In the "Key" column I would like to remove the numbers and colon dash which follows. This order is always the same (from the left). So for the first row I would like to remove "0:- ", and just leave "Profile 1". I am struggling to find the correct regex expression to do what I want. Originally I tried the following: df_json['key'] = df_json['key'].map(lambda x: x.strip(':- ')[1]) However, this approach is too restrictive since there can be multiple words in the field. I would like to use pd.Series.str.replace(), but I cant figure out the correct regex expression to achieve the desired results. Any help would be greatly appreciated.
With your shown samples, please try following. Using replace function of Pandas here. Simple explanation would be, apply replace function of Pandas to German column of dataframe and then use regex ^[0-9]+:-\s+ to replace values with NULL. df['German'].replace('(^[0-9]+:-\s+)','', regex=True) Explanation: ^[0-9]+: match starting digits followed by colon here. :-\s+: Match colon, followed by - followed by 1 or more space occurrences.
What about just using pandas.Series.str.partition instead of regular expressions: df['German'] = df['German'].str.partition()[2] This would split the series on the 1st space only and grab the trailing part. Alternatively to partition you could also just split: df['German'] = df['German'].str.split(' ', 1).str[1] If regex is a must for you, maybe use a lazy quantifier to match upto the 1st space character: df['German'] = df['German'].replace('^.*? +','', regex=True) Where: ^ - Start line anchor. .*? - Any 0+ (lazy) characters other than newline upto; + - 1+ literal space characters. Here is an online demo
You need df_json['key'] = df_json['key'].str.replace(r'^\d+:-\s*', '', regex=True) See the regex demo and the regex graph: Details: ^ - start of string \d+ - one or more digits : - a colon - - a hyphen \s* - zero or more whitespaces
Extract any non white Space \S and Non Digits \D which are immediately to the left of unwanted characters df['GermanFiltered']=df['German'].str.extract("((?<=^\d\:\-\s)\S+\D+)")
Using REGEX to grab the information after the match
I ran a PDF through a series of processes to extra the text from it. I was successful in that regard. However, now I want to extract specific text from documents. The document is set up as a multi lined string (I believe. when I paste it into Word the paragraph character is at the end of each line): Send Unit: COMPLETE NOA Selection: 20-0429.07 #for some reason, in this editor, despite the next line having > infront of it, the following line (Pni/Trk) keeps wrapping up to the line above. This doesn't exist in the actual doc. Pni/Trk: 3 Panel / 3 Track Panel Stack: STD Width: 142.0000 The information is want to extract are the numbers following "NOA Selection:". I know I can do a regex something to the effect of: pattern = re.compile(r'NOA\sSelection:\s\d*-\d*\.\d*) but I only want the numbers after the NOA selection, especially because NOA Selection will always be the same but the format of the numbers/letters/./-/etc. can vary pretty wildly. This looked promising but it is in Java and I haven't had much luck recreating it in Python. I think I need to use (?<=...), but haven't been able to implement it. Also, several of the examples show the string stored in the python file as a variable, but I'm trying to access it from a .txt file, so I might be going wrong there. This is what I have so far. with open('export1.txt', 'r') as d: contents = d.read() p = re.compile('(?<=NOA)') s = re.search(p, contents) print(s.group()) Thank you for any help you can provide.
With your shown samples, you could try following too. For sample 20-0429.07 I have kept .07 part optional in regex in case you have values 20-0429 only it should work for those also. import re val = """Send Unit: COMPLETE NOA Selection: 20-0429.07""" matches = re.findall(r'NOA\s+Selection:\s+(\d+-\d+(?:\.\d+)?)', val) print(matches) ['20-0429.07'] Explanation: Adding detailed explanation(only for explanation purposes). NOA\s+Selection:\s+ ##matching NOA spaces(1 or more occurrences) Selection: spaces(1 or more occurrences) (\d+-\d+(?:\.\d+)?) ##Creating capturing group matching(1 or more occurrences) digits-digits(1 or more occurrences) ##and in a non-capturing group matching dot followed by digits keeping it optional.
Keeping it simple, you could use re.findall here: inp = """Send Unit: COMPLETE NOA Selection: 20-0429.07""" matches = re.findall(r'\bNOA Selection: (\S+)', inp) print(matches) # ['20-0429.07']
FuzzyWuzzy for very similar records in Python
I have a dataset with which I want to find the closest string match. For that purpose I'm using FuzzyWuzzy in this way sol=process.extract(t,dev2,scorer=fuzz.token_sort_ratio) Where t is the string and dev2 is the list to compare to. My problem is that sometimes it has very similar records and options provided by FuzzyWuzzy seems to be lacking. And I've tested with token_sort, token_set, partial_token sort and set, ratio, partial_ratio, and WRatio. For example, the string Italy - Serie A gives me the following 2 closest matches. Token_sort_ratio: (92, 'Italy - Serie D');(86, 'Italian - Serie A') The one wanted is obviously the second one, but character by character is closer the first one, which is a different league. This happens as well with teams. If, let's say I have a string Buchtholz I would obtains Buchtholz II before I get TSV Buchtholz. My main guess now would be to try and weight the presence and absence of several characters more heavily, like single capital letters at the end of the string, so if there is a difference in the letter or an absence it is weighted as less close. Or for () and special characters. I don't know if there is a way to take this into account or you guys have a better approach to get the string that really matches.
Similarity matches often require knowledge of the data being analysed. i.e. it is not just a blind single round of matching. I recommend that you pass your results through more steps of matching, starting with inclusive/optimistic approaches (like token_set_ratio) with low cut off scores and working toward more exclusive/pessimistic approaches with higher cut off scores until you have a clear winner. If you know more about the text you're analyzing, you can even modify the strings as you progress. In a case I worked on, I did similarity matches of goods movement descriptions. In the descriptions the numbers sequences were more important than the text. e.g. when looking for a match for "SLURRY VALVE 250MM RAGMAX 2000" the 250 and 2000 part of the string are important, otherwise I get a "SLURRY VALVE 50MM RAGMAX 2000" as the best match instead of "VALVE B/F 250MM,RAGMAX 250RAG2000 RAGON" which is a better result. I put the similarity match process through two steps: 1. Get a bunch of similar matches using an optimistic matching scorer (token_set_ratio) 2. get the number sequences of these results and pass them through another round of matching with a more strict scorer (token_sort_ratio). Doing this gave me the better result in the example I showed above. Below is some blocks of code that could be of assistance: here's a function to get numbers from the sequence. (In your case you might use this to exclude numbers from your string instead?) def get_numbers_from_string(description): numbers = ''.join((ch if ch in '0123456789.-' else ' ') for ch in description) numbers = ' '.join([nr for nr in numbers.split()]) return numbers and here is a portion of the code I used to put the description match through two rounds: try: # get close match from goods move that has material numbers df_material = pd.DataFrame(process.extract(description, corpus_material, scorer=fuzz.token_set_ratio), columns=['Similar Text','Score'] ) if df_material['Score'][df_material['Score']>=cut_off_accuracy_materials].count()>=1: similar_text = df_material['Similar Text'].iloc[0] score = df_material['Score'].iloc[0] if nr_description_numbers>4: # if there are multiple matches found, then get best number combination match df_material = df_material[df_material['Score']>=cut_off_accuracy_materials] new_corpus = list(df_material['Similar Text']) new_corpus = np.vectorize(get_numbers_from_string)(new_corpus) df_material['numbers'] = new_corpus df_numbers = pd.DataFrame(process.extract(description_numbers, new_corpus, scorer=fuzz.token_sort_ratio), columns=['numbers','Score'] ) similar_text = df_material['Similar Text'][df_material['numbers']==df_numbers['numbers'].iloc[0]].iloc[0] nr_score = df_numbers['Score'].iloc[0] hope it helps, and good luck
Lua: Search a specific string
Hi all tried all the string pattrens and library arguments but still stuck. i want to get the name of the director from the following string i have tried the string.matcH but it matches the from the first character it finD from the string the string is... fixstrdirector = {id:39254,cast:[{id:15250,name:Hope Davis,character:Aunt Debra,order:5,cast_id:10,profile_path:/aIHF11Ss8P0A8JUfiWf8OHPVhOs.jpg},{id:53650,name:Anthony Mackie,character:Finn,order:3,cast_id:11,profile_path:/5VGGJ0Co8SC94iiedWb2o3C36T.jpg},{id:19034,name:Evangeline Lilly,character:Bailey Tallet,order:2,cast_id:12,profile_path:/oAOpJKgKEdW49jXrjvUcPcEQJb3.jpg},{id:6968,name:Hugh Jackman,character:Charlie Kenton,order:0,cast_id:13,profile_path:/wnl7esRbP3paALKn4bCr0k8qaFu.jpg},{id:79072,name:Kevin Durand,character:Ricky,order:4,cast_id:14,profile_path:/c95tTUjx5T0D0ROqTcINojpH6nB.jpg},{id:234479,name:Dakota Goyo,character:Max Kenton,order:1,cast_id:15,profile_path:/7PU6n4fhDuFwuwcYVyRNVEZE7ct.jpg},{id:8986,name:James Rebhorn,character:Marvin,order:6,cast_id:16,profile_path:/ezETMv0YM0Rg6YhKpu4vHuIY37D.jpg},{id:930729,name:Marco Ruggeri,character:Cliff,order:7,cast_id:17,profile_path:/1Ox63ukTd2yfOf1LVJOMXwmeQjO.jpg},{id:19860,name:Karl Yune,character:Tak Mashido,order:8,cast_id:18,profile_path:/qK315vPObCNdywdRN66971FtFez.jpg},{id:111206,name:Olga Fonda,character:Farra Lemkova,order:9,cast_id:19,profile_path:/j1qabOHf3Pf82f1lFpUmdF5XvSp.jpg},{id:53176,name:John Gatins,character:Kingpin,order:10,cast_id:41,profile_path:/A2MqnSKVzOuBf8MVfNyve2h2LxJ.jpg},{id:1126350,name:Sophie Levy,character:Big Sister,order:11,cast_id:42,profile_path:null},{id:1126351,name:Tess Levy,character:Little Sister,order:12,cast_id:43,profile_path:null},{id:1126352,name:Charlie Levy,character:Littlest Sister,order:13,cast_id:44,profile_path:null},{id:187983,name:Gregory Sims,character:Bill Panner,order:14,cast_id:45,profile_path:null}],crew:[{id:58726,name:Leslie Bohem,department:Writing,job:Screenplay,profile_path:null},{id:53176,name:John Gatins,department:Writing,job:Screenplay,profile_path:/A2MqnSKVzOuBf8MVfNyve2h2LxJ.jpg},{id:17825,name:Shawn Levy,department:Directing,job:Director,profile_path:/7f2f8EXdlWsPYN0HPGcIlG21xU.jpg},{id:12415,name:Richard Matheson,department:Writing,job:Story,profile_path:null},{id:57113,name:Dan Gilroy,department:Writing,job:Story,profile_path:null},{id:25210,name:Jeremy Leven,department:Writing,job:Story,profile_path:null},{id:17825,name:Shawn Levy,department:Production,job:Producer,profile_path:/7f2f8EXdlWsPYN0HPGcIlG21xU.jpg},{id:34970,name:Susan Montford,department:Production,job:Producer,profile_path:/1XJt51Y9ciPhkHrAYE0j6Jsmgji.jpg},{id:3183,name:Don Murphy,department:Production,job:Producer,profile_path:null},{id:34967,name:Rick Benattar,department:Production,job:Producer,profile_path:null},{id:1126348,name:Eric Hedayat,department:Production,job:Producer,profile_path:null},{id:186721,name:Ron Ames,department:Production,job:Producer,profile_path:null},{id:10956,name:Josh McLaglen,department:Production,job:Executive Producer,profile_path:null},{id:57634,name:Mary McLaglen,department:Production,job:Executive Producer,profile_path:null},{id:23779,name:Jack Rapke,department:Production,job:Executive Producer,profile_path:null},{id:488,name:Steven Spielberg,department:Production,job:Executive Producer,profile_path:/cuIYdFbEe89PHpoiOS9tmo84ED2.jpg},{id:30,name:Steve Starkey,department:Production,job:Executive Producer,profile_path:null},{id:24,name:Robert Zemeckis,department:Production,job:Executive Producer,profile_path:/isCuZ9PWIOyXzdf3ihodXzjIumL.jpg},{id:531,name:Danny Elfman,department:Sound,job:Original Music Composer,profile_path:/pWacZpYPos8io22nEiim7d3wp2j.jpg},{id:18265,name:Mauro Fiore,department:Crew,job:Cinematography,profile_path:null},{id:54271,name:Dean Zimmerman,department:Editing,job:Editor,profile_path:null},{id:25365,name:Richard Hicks,department:Production,job:Casting,profile_path:null},{id:5490,name:David Rubin,department:Production,job:Casting,profile_path:null},{id:52088,name:Tom Meyer,department:Art,job:Production Design,profile_path:null}]} i have tried string.match(fixstrdirector,"name:(.+),department:Directing") but it gives me the from the first occurace it find the name to the end of thr string output: Hope Davis,character:Aunt Debra,order:5,cast_id:10,profile_path:/aIHF11Ss8P0A8JUfiWf8OHPVhOs.jpg},{id:53650,name:Anthony Mackie,character:Finn,order:3,cast_id:11,profile_path:/5VGGJ0Co8SC94iiedWb2o3C36T.jpg},{id:19034,name:Evangeline Lilly,character:Bailey Tallet,order:2,cast_id:12,profile_path:/oAOpJKgKEdW49jXrjvUcPcEQJb3.jpg},{id:6968,name:Hugh Jackman,character:Charlie Kenton,order:0,cast_id:13,profile_path:/wnl7esRbP3paALKn4bCr0k8qaFu.jpg},{id:79072,name:Kevin Durand,character:Ricky,order:4,cast_id:14,profile_path:/c95tTUjx5T0D0ROqTcINojpH6nB.jpg},{id:234479,name:Dakota Goyo,character:Max Kenton,order:1,cast_id:15,profile_path:/7PU6n4fhDuFwuwcYVyRNVEZE7ct.jpg},{id:8986,name:James Rebhorn,character:Marvin,order:6,cast_id:16,profile_path:/ezETMv0YM0Rg6YhKpu4vHuIY37D.jpg},{id:930729,name:Marco Ruggeri,character:Cliff,order:7,cast_id:17,profile_path:/1Ox63ukTd2yfOf1LVJOMXwmeQjO.jpg},{id:19860,name:Karl Yune,character:Tak Mashido,order:8,cast_id:18,profile_path:/qK315vPObCNdywdRN66971FtFez.jpg},{id:111206,name:Olga Fonda,character:Farra Lemkova,order:9,cast_id:19,profile_path:/j1qabOHf3Pf82f1lFpUmdF5XvSp.jpg},{id:53176,name:John Gatins,character:Kingpin,order:10,cast_id:41,profile_path:/A2MqnSKVzOuBf8MVfNyve2h2LxJ.jpg},{id:1126350,name:Sophie Levy,character:Big Sister,order:11,cast_id:42,profile_path:null},{id:1126351,name:Tess Levy,character:Little Sister,order:12,cast_id:43,profile_path:null},{id:1126352,name:Charlie Levy,character:Littlest Sister,order:13,cast_id:44,profile_path:null},{id:187983,name:Gregory Sims,character:Bill Panner,order:14,cast_id:45,profile_path:null}],crew:[{id:58726,name:Leslie Bohem,department:Writing,job:Screenplay,profile_path:null},{id:53176,name:John Gatins,department:Writing,job:Screenplay,profile_path:/A2MqnSKVzOuBf8MVfNyve2h2LxJ.jpg},{id:17825,name:Shawn Levy
You're searching from the first occurrence of "name:" until the "department:Directing" with everything in between. Instead, you need to restrict what can be between the two strings. Here for example I'm saying that the characters that make up the name can only be alphanumeric or a space: string.match(fixstrdirector,"name:([%w ]+),department:Directing") Alternatively, given that there's a comma separating the parameters, a better approach would be to search for "name:" followed by any characters other than a comma, followed by "department:Directing": string.match(fixstrdirector,"name:([^,]+),department:Directing") Of course that wouldn't work if the name had a comma it in!
Lua patterns provides - modifier for tasks as you have above. As stated on PiL - Section 20.2: The + modifier matches one or more characters of the original class. It will always get the longest sequence that matches the pattern. Like *, the modifier - also matches zero or more occurrences of characters of the original class. However, instead of matching the longest sequence, it matches the shortest one. Next, when you are using . to match, it'll find any and all characters satisfying the pattern. Therefore, you'll get the result from first occurence of name until the ,department:Directing is found. Since you know that it is a JSON data, you can try to match for [^,]; that is, non-comma characters. So, for your case try: local tAllNames = {} for sName in fixstrdirector:gmatch( "name:([^,]-),department:Directing" ) do tAllNames[ #tAllNames + 1 ] = sName end and all your required names will be stored in the table tAllNames. An example of the above can be seen at codepad.