I want to get an array of all the lines which start by text: (till the first asset_performance_label)
I saw this post, but wasn't sure how to apply it.
Should I convert the proto to string, as I have tried?
text = extract_text_from_proto(r"(\w+)text:(\w+)asset_performance_label:", '''[pinned_field: HEADLINE_1
text: "5 Best Products"
asset_performance_label: PENDING
policy_summary_info
{
review_status: REVIEWED
approval_status: APPROVED
}
, pinned_field: HEADLINE_1
text: "10 Best Products 2021"
asset_performance_label: PENDING
policy_summary_info
{
review_status: REVIEWED
approval_status: APPROVED
}''')
def extract_text_from_proto(regex, proto_string):
regex = re.escape(regex)
result_array = [m.group() for m in re.finditer(regex, proto_string)]
return result_array
# return [extract_text(each_item, regex) for each_item in proto],
def extract_text(regex, item):
m = re.match(regex, str(item))
if m is None:
# text = "MISSING TEXT"
raise Exception("Ad is missing text")
else:
text = m.group(2)
return text
Expected result: ["5 Best Products","10 Best Products 2021"]
What if I want to match (optional) pinned_field: (word)? so the result could be: [HEADLINE_1: 5 Best Products', 'HEADLINE_1:10 Best Products 2021', 'some_text_without_pinned_field']` ?
You can use a single capture group, and match assert_performance_label in the next line. Use re.findall to return the group values.
\btext:\s*"([^"]+)"\n\s*asset_performance_label\b
The pattern matches
\btext:\s*" Match text: predeced by a word boundary \b to prevent a partial match
([^"]+) Capture group 1, match 1+ chars other than a double quote
"\n\s* Match a newline an optional whitespace chars
asset_performance_label\b Match `asset_performance_label followed by a word boundary
For example
import re
def extract_text_from_proto(regex, proto_string):
return re.findall(regex, proto_string)
text = extract_text_from_proto(r'\btext:\s*"([^"]+)"\n\s*asset_performance_label\b', '''[pinned_field: HEADLINE_1
text: "5 Best Products"
asset_performance_label: PENDING
policy_summary_info
{
review_status: REVIEWED
approval_status: APPROVED
}
, pinned_field: HEADLINE_1
text: "10 Best Products 2021"
asset_performance_label: PENDING
policy_summary_info
{
review_status: REVIEWED
approval_status: APPROVED
}''')
print(text)
Output
['5 Best Products', '10 Best Products 2021']
I need to scan a log file for users that ran certain SQL statements (DROP,CREATE,etc.), and return an array with the users and which SQL ddl's they tried(drop,create,etc.).
I have log files where each line normally looks like this:
'2019-01-14T-19:23:50Z UTC' [ db=dev user=joeschmoe pid=123 userid=1 xid=1234]' Log: Select *
however, sometimes the select statement will span multiple lines like this:
'2019-01-14T19:23:50Z UTC [ db=dev user=rb pid=16 userid=1 xid=8 ]' LOG: SELECT SUM (num_queries) num_all_queries
,SUM (CASE WHEN lalala is not null THEN num_queries ELSE 0 END) num_b
,SUM (CASE WHEN lalala is null THEN num_queries ELSE 0 END) num_non_b
,SUM (total_queue_time_min) total_queue_time_min
,SUM (CASE WHEN lalala is not null THEN total_queue_time_min ELSE 0 END) b_total_queue_time_min
,SUM (CASE WHEN lalala is null THEN total_queue_time_min ELSE 0 END) non_b_total_queue_time_min
,SUM (CASE WHEN lalala is not null THEN duration_s ELSE 0 END)/60.0 total_burst_usage_min
,SUM (CASE WHEN lalala is not null THEN 1 ELSE 0 END) num_lalalas
,MIN(firsttime) mintime
,MAX(lasttime) maxtime
,DATEDIFF (seconds, mintime, maxtime) workload_duration_s
,wration_s/60.0 workload_duration_min
LEFT JOIN (SELECT b FROM STfdaf LIMIT 1) sq ON sq.but_reon < 100
;
I am combing these logs for certain keywords in the SQL statements. I can write the regex to handle that but I need help getting this log in a format I can work with. I was originally using a for loop and a regex
for line in input:
user_match = re.search("DROP", line, re.IGNORECASE)
This wouldn't be accurate because when a sql statement spans multiple lines I wouldn't be able to tie the "DROP" back to the "USER" if the DROP occurred many lines after the initial line.
I'm not sure how to go about doing this. Whether its turning this text file into a list in python and programmatically combining multiple lines into one or whatever other options there may be.
I solved this issue by storing the user value in a variable that only updates if a match is found using RegEx. So if a keyword is found, the user value last stored in the variable would be returned along with the keyword.
for line in input:
error = []
user_match = re.search("USER=b[0-9]{6}", line, re.IGNORECASE)
serviceuser = re.search("USER=[a-z0-9]*", line, re.IGNORECASE)
if serviceuser:
user = (serviceuser.group().split("=")[1])
elif user_match:
user = (user_match.group().split("=")[1])
ddl = re.search(
"LINK|.DELETE.|INSERT|TRIGGER|TRUNCATE|UPDATE|WRITE", line, re.IGNORECASE)
if ddl:
error.append(user)
error.append(ddl.group())
I would like to do update a whole bunch of 'rev rec plan' records. Updating using mass update isn't available nor is an import possible.
So the only way I think I could do this is via script (probably scheduled), but I am not sure how to retrieve the 3 fields in my excel (internal id, start date and end date). I need the script to get the date fields from my file and using the internal id update the rec rec dates.
Thanks
You should be able to get started with the following code. Just get the internal id of the file you want to parse.
function parseCSVFromFile(fileId){
var csvFile = nlapiLoadFile(fileId);
var csv = csvFile.getValue();
var parsedCSV = CSVParser().parse(csv);
var data= parsedCSV.data;
return data;
}
function CSVParser(){function e(r,n){if(Array.isArray(r)){var i=[];return r.forEach(function(t){"object"==typeof t?i.push(e(t.file,t.config)):i.push(e(t,n))}),i}var i={data:[],errors:[]};if(!/(\.csv|\.txt)$/.test(r))return i.errors.push({type:"",code:"",message:"Unsupported file type.",row:""}),i;try{var a=fs.readFileSync(r).toString();return t(a,n)}catch(s){return i.errors.push(s),i}}function t(e,t){var r=a(t),i=new n(r),s=i.parse(e);return f(r.complete)&&r.complete(s),s}function r(e,t){function r(){"object"==typeof t&&("string"==typeof t.delimiter&&1==t.delimiter.length&&-1==l.BAD_DELIMITERS.indexOf(t.delimiter)&&(o=t.delimiter),("boolean"==typeof t.quotes||t.quotes instanceof Array)&&(f=t.quotes),"string"==typeof t.newline&&(d=t.newline))}function n(e){if("object"!=typeof e)return[];var t=[];for(var r in e)t.push(r);return t}function i(e,t){var r="";"string"==typeof e&&(e=JSON.parse(e)),"string"==typeof t&&(t=JSON.parse(t));var n=e instanceof Array&&e.length>0,i=!(t[0]instanceof Array);if(n){for(var s=0;s<e.length;s++)s>0&&(r+=o),r+=a(e[s],s);t.length>0&&(r+=d)}for(var f=0;f<t.length;f++){for(var l=n?e.length:t[f].length,u=0;l>u;u++){u>0&&(r+=o);var p=n&&i?e[u]:u;r+=a(t[f][p],u)}f<t.length-1&&(r+=d)}return r}function a(e,t){if("undefined"==typeof e||null===e)return"";e=e.toString().replace(/"/g,'""');var r="boolean"==typeof f&&f||f instanceof Array&&f[t]||s(e,l.BAD_DELIMITERS)||e.indexOf(o)>-1||" "==e.charAt(0)||" "==e.charAt(e.length-1);return r?'"'+e+'"':e}function s(e,t){for(var r=0;r<t.length;r++)if(e.indexOf(t[r])>-1)return!0;return!1}var f=!1,o=",",d="\r\n";if(r(),"string"==typeof e&&(e=JSON.parse(e)),e instanceof Array){if(!e.length||e[0]instanceof Array)return i(null,e);if("object"==typeof e[0])return i(n(e[0]),e)}else if("object"==typeof e)return"string"==typeof e.data&&(e.data=JSON.parse(e.data)),e.data instanceof Array&&(e.fields||(e.fields=e.data[0]instanceof Array?e.fields:n(e.data[0])),e.data[0]instanceof Array||"object"==typeof e.data[0]||(e.data=[e.data])),i(e.fields||[],e.data||[]);throw"exception: Unable to serialize unrecognized input"}function n(e){function t(){if(E&&m&&(p("Delimiter","UndetectableDelimiter","Unable to auto-detect delimiting character; defaulted to '"+l.DefaultDelimiter+"'"),m=!1),e.skipEmptyLines)for(var t=0;t<E.data.length;t++)1==E.data[t].length&&""==E.data[t][0]&&E.data.splice(t--,1);return r()&&n(),a()}function r(){return e.header&&0==w.length}function n(){if(E){for(var e=0;r()&&e<E.data.length;e++)for(var t=0;t<E.data[e].length;t++)w.push(E.data[e][t]);E.data.splice(0,1)}}function a(){if(!E||!e.header&&!e.dynamicTyping)return E;for(var t=0;t<E.data.length;t++){for(var r={},n=0;n<E.data[t].length;n++){if(e.dynamicTyping){var i=E.data[t][n];"true"==i||"TRUE"===i?E.data[t][n]=!0:"false"==i||"FALSE"===i?E.data[t][n]=!1:E.data[t][n]=u(i)}e.header&&(n>=w.length?(r.__parsed_extra||(r.__parsed_extra=[]),r.__parsed_extra.push(E.data[t][n])):r[w[n]]=E.data[t][n])}e.header&&(E.data[t]=r,n>w.length?p("FieldMismatch","TooManyFields","Too many fields: expected "+w.length+" fields but parsed "+n,t):n<w.length&&p("FieldMismatch","TooFewFields","Too few fields: expected "+w.length+" fields but parsed "+n,t))}return e.header&&E.meta&&(E.meta.fields=w),E}function o(t){for(var r,n,a,s=[","," ","|",";",l.RECORD_SEP,l.UNIT_SEP],f=0;f<s.length;f++){var o=s[f],d=0,u=0;a=void 0;for(var p=new i({delimiter:o,preview:10}).parse(t),c=0;c<p.data.length;c++){var h=p.data[c].length;u+=h,"undefined"!=typeof a?h>1&&(d+=Math.abs(h-a),a=h):a=h}u/=p.data.length,("undefined"==typeof n||n>d)&&u>1.99&&(n=d,r=o)}return e.delimiter=r,{successful:!!r,bestDelimiter:r}}function d(e){e=e.substr(0,1048576);var t=e.split("\r");if(1==t.length)return"\n";for(var r=0,n=0;n<t.length;n++)"\n"==t[n][0]&&r++;return r>=t.length/2?"\r\n":"\r"}function u(e){var t=g.test(e);return t?parseFloat(e):e}function p(e,t,r,n){E.errors.push({type:e,code:t,message:r,row:n})}var c,h,m,g=/^\s*-?(\d*\.?\d+|\d+\.?\d*)(e[-+]?\d+)?\s*$/i,y=this,v=0,b=!1,w=[],E={data:[],errors:[],meta:{}};if(f(e.step)){var x=e.step;e.step=function(n){if(E=n,r())t();else{if(t(),0==E.data.length)return;v+=n.data.length,e.preview&&v>e.preview?h.abort():x(E,y)}}}this.parse=function(r){if(e.newline||(e.newline=d(r)),m=!1,!e.delimiter){var n=o(r);n.successful?e.delimiter=n.bestDelimiter:(m=!0,e.delimiter=l.DefaultDelimiter),E.meta.delimiter=e.delimiter}var a=s(e);return e.preview&&e.header&&a.preview++,c=r,h=new i(a),E=h.parse(c),t(),!f(e.complete)||b||y.streamer&&!y.streamer.finished()||e.complete(E),b?{meta:{paused:!0}}:E||{meta:{paused:!1}}},this.pause=function(){b=!0,h.abort(),c=c.substr(h.getCharIndex())},this.resume=function(){b=!1,h=new i(e),h.parse(c),b||(y.streamer&&!y.streamer.finished()?y.streamer.resume():f(e.complete)&&e.complete(E))},this.abort=function(){h.abort(),f(e.complete)&&e.complete(E),c=""}}function i(e){e=e||{};var t=e.delimiter,r=e.newline,n=e.comments,i=e.step,a=e.preview,s=e.fastMode;if(("string"!=typeof t||1!=t.length||l.BAD_DELIMITERS.indexOf(t)>-1)&&(t=","),n===t)throw"Comment character same as delimiter";n===!0?n="#":("string"!=typeof n||l.BAD_DELIMITERS.indexOf(n)>-1)&&(n=!1),"\n"!=r&&"\r"!=r&&"\r\n"!=r&&(r="\n");var f=0,o=!1;this.parse=function(e){function l(){return w.push(e.substr(f)),v.push(w),f=c,y&&p(),u()}function d(t){v.push(w),w=[],f=t,O=e.indexOf(r,f)}function u(e){return{data:v,errors:b,meta:{delimiter:t,linebreak:r,aborted:o,truncated:!!e}}}function p(){i(u()),v=[],b=[]}if("string"!=typeof e)throw"Input must be a string";var c=e.length,h=t.length,m=r.length,g=n.length,y="function"==typeof i;f=0;var v=[],b=[],w=[];if(!e)return u();if(s){for(var E=e.split(r),x=0;x<E.length;x++)if(!n||E[x].substr(0,g)!=n){if(y){if(v=[E[x].split(t)],p(),o)return u()}else v.push(E[x].split(t));if(a&&x>=a)return v=v.slice(0,a),u(!0)}return u()}for(var D=e.indexOf(t,f),O=e.indexOf(r,f);;)if('"'!=e[f])if(n&&0===w.length&&e.substr(f,g)===n){if(-1==O)return u();f=O+m,O=e.indexOf(r,f),D=e.indexOf(t,f)}else if(-1!==D&&(O>D||-1===O))w.push(e.substring(f,D)),f=D+h,D=e.indexOf(t,f);else{if(-1===O)break;if(w.push(e.substring(f,O)),d(O+m),y&&(p(),o))return u();if(a&&v.length>=a)return u(!0)}else{var A=f;for(f++;;){var A=e.indexOf('"',A+1);if(-1===A)return b.push({type:"Quotes",code:"MissingQuotes",message:"Quoted field unterminated",row:v.length,index:f}),l();if(A===c-1)return w.push(e.substring(f,A).replace(/""/g,'"')),v.push(w),y&&p(),u();if('"'!=e[A+1]){if(e[A+1]==t){w.push(e.substring(f,A).replace(/""/g,'"')),f=A+1+h,D=e.indexOf(t,f),O=e.indexOf(r,f);break}if(e.substr(A+1,m)===r){if(w.push(e.substring(f,A).replace(/""/g,'"')),d(A+1+m),D=e.indexOf(t,f),y&&(p(),o))return u();if(a&&v.length>=a)return u(!0);break}}else A++}}return l()},this.abort=function(){o=!0},this.getCharIndex=function(){return f}}function a(e){"object"!=typeof e&&(e={});var t=s(e);return("string"!=typeof t.delimiter||1!=t.delimiter.length||l.BAD_DELIMITERS.indexOf(t.delimiter)>-1)&&(t.delimiter=o.delimiter),"\n"!=t.newline&&"\r"!=t.newline&&"\r\n"!=t.newline&&(t.newline=o.newline),"boolean"!=typeof t.header&&(t.header=o.header),"boolean"!=typeof t.dynamicTyping&&(t.dynamicTyping=o.dynamicTyping),"number"!=typeof t.preview&&(t.preview=o.preview),"function"!=typeof t.step&&(t.step=o.step),"function"!=typeof t.complete&&(t.complete=o.complete),"boolean"!=typeof t.skipEmptyLines&&(t.skipEmptyLines=o.skipEmptyLines),"boolean"!=typeof t.fastMode&&(t.fastMode=o.fastMode),t}function s(e){if("object"!=typeof e)return e;var t=e instanceof Array?[]:{};for(var r in e)t[r]=s(e[r]);return t}function f(e){return"function"==typeof e}var o={delimiter:"",newline:"",header:!1,dynamicTyping:!1,preview:0,step:void 0,comments:!1,complete:void 0,skipEmptyLines:!1,fastMode:!1},l={};return l.parse=t,l.parseFiles=e,l.unparse=r,l.RECORD_SEP=String.fromCharCode(30),l.UNIT_SEP=String.fromCharCode(31),l.BYTE_ORDER_MARK="\ufeff",l.BAD_DELIMITERS=["\r","\n",'"',l.BYTE_ORDER_MARK],l.DefaultDelimiter=",",l.Parser=i,l.ParserHandle=n,l}
Example:
var parsedCSV = parseCSVFromFile(123);
nlapiLogExecution('DEBUG', "parsedCSV ", JSON.stringify(parsedCSV));
I am new to Groovy and I could not get around this issue. I appreciate any help.
I want to read a file from Groovy. While I am reading the content, for each line I want to substitute the string '${random_id}' and '${entryAuthor}' with different string values.
protected def doPost(String url, URL bodyFile, Map headers = new HashMap() ) {
StringBuffer sb = new StringBuffer()
def randomId = getRandomId()
bodyFile.eachLine { line ->
sb.append( line.replace("\u0024\u007Brandom_id\u007D", randomId)
.replace("\u0024\u007BentryAuthor\u007D", entryAuthor) )
sb.append("\n")
}
return doPost(url, sb.toString())
}
But I got the following error:
groovy.lang.MissingPropertyException:
No such property: random_id for class: tests.SimplePostTest
Possible solutions: randomId
at foo.test.framework.FooTest.doPost_closure1(FooTest.groovy:85)
at groovy.lang.Closure.call(Closure.java:411)
at groovy.lang.Closure.call(Closure.java:427)
at foo.test.framework.FooTest.doPost(FooTest.groovy:83)
at foo.test.framework.FooTest.doPost(FooTest.groovy:80)
at tests.SimplePostTest.Post & check Entry ID(SimplePostTest.groovy:42)
Why would it complain about a property, when I am not doing anything? I also tried "\$\{random_id\}", which works in Java String.replace(), but not in Groovy.
You are doing it the hard way. Just evaluate your file's contents with Groovy's SimpleTemplateEngine.
import groovy.text.SimpleTemplateEngine
def text = 'Dear "$firstname $lastname",\nSo nice to meet you in <% print city %>.\nSee you in ${month},\n${signed}'
def binding = ["firstname":"Sam", "lastname":"Pullara", "city":"San Francisco", "month":"December", "signed":"Groovy-Dev"]
def engine = new SimpleTemplateEngine()
template = engine.createTemplate(text).make(binding)
def result = 'Dear "Sam Pullara",\nSo nice to meet you in San Francisco.\nSee you in December,\nGroovy-Dev'
assert result == template.toString()
you better use groovy.text.SimpleTemplateEngine class; check this for more details http://groovy.codehaus.org/Groovy+Templates
The issue here is that Groovy Strings will evaluate "${x}" by substituting the value of 'x', and we don't want that behaviour in this case. The trick is to use single-quotes which denote plain old Java Strings.
Using a data file like this:
${random_id} 1 ${entryAuthor}
${random_id} 2 ${entryAuthor}
${random_id} 3 ${entryAuthor}
Consider this code, which is analogous to the original:
// spoof HTTP POST body
def bodyFile = new File("body.txt").getText()
StringBuffer sb = new StringBuffer()
def randomId = "257" // TODO: use getRandomId()
def entryAuthor = "Bruce Eckel"
// use ' here because we don't want Groovy Strings, which would try to
// evaluate e.g. ${random_id}
String randomIdToken = '${random_id}'
String entryAuthorToken = '${entryAuthor}'
bodyFile.eachLine { def line ->
sb.append( line.replace(randomIdToken, randomId)
.replace(entryAuthorToken, entryAuthor) )
sb.append("\n")
}
println sb.toString()
The output is:
257 1 Bruce Eckel
257 2 Bruce Eckel
257 3 Bruce Eckel