First, I wish to extract the last word and first word for the Description column (this column contains at least 3 words) into a newly created column firstword and lastword. However, the word() function is not applied to all the rows. As such, there are many rows with empty lastword, though these rows actually have a last word (as you can see from the Description column). This is shown in the first two lines of codes.
Second, I am also trying to get the third line of code to replace the lastword with firstword, if lastword is empty. However it isn't working.
Is there a way to rectify this?
c1$lastword = word(c1$Description,start=-1) #extract last word
c1$firstword = word(c1$Description,start=1) #extract first word
c1$lastword=ifelse(c1$lastword == " ", c1$firstword, c1$lastword)
I realise that there is white space at the beginning of some of the rows of the Description variable, which isn't shown when viewed in R.
Removing the whitespace using stri_trim() solved the issue.
c1$Description = stri_trim(c1$Description, "left") #remove whitespace
Related
I have a very very large TSV file. The first line is headers. The following lines contain data followed by tabs or double-tabs if a field was blank otherwise the fields can contain alphanumerics or alphanumerics plus punctuation marks.
for example:
Field1<tab>Field2<tab>FieldN<newline>
The fields may contain spaces, punctuation or alphanumerics. The only thing(s) that remains true are:
each field is followed by a tab except the last one
the last field is followed by a newline
blank fields are filled with a tab. Like all other fields they are followed by a tab. This makes them double-tab.
I've tried many combinations of pattern matching in lua and never get it quite right. Typically the fields with punctuation (time and date fields) are the ones that get me.
I need the blank fields (the ones with double-tab) preserved so that the rest of the fields are always at the same index value.
Thanks in Advance!
Try the code below:
function test(s)
local n=0
s=s..'\t'
for w in s:gmatch("(.-)\t") do
n=n+1
print(n,"["..w.."]")
end
end
test("10\t20\t30\t\t50")
test("100\t200\t300\t\t500\t")
It adds a tab to the end of the string so that all fields are follow by a tab, even the last one.
Rows and columns are separated:
local filename = "big_tables.tsv" -- tab separated values
-- local filename = "big_tables.csv" -- comma separated values
local lines = io.lines(filename) -- open file as lines
local tables = {} -- new table with columns and rows as tables[n_column][n_row]=value
for line in lines do -- row iterator
local i = 1 -- first column
for value in (string.gmatch(line, "[^%s]+")) do -- tab separated values
-- for value in (string.gmatch(line, '%d[%d.]*')) do -- comma separated values
tables[i]=tables[i]or{} -- if not column then create new one
tables[i][#tables[i]+1]=tonumber(value) -- adding row value
i=i+1 -- column iterator
end
end
I extracted a pdf table using tabula.read_pdf but some of the data entries a) show a whitespace between the values and b) includes two sets of values into one column as shown one columns "Sports 2019/2018" and "Total 2019/2018": https://imgur.com/a/MviV6N9
In order for me to use df_1=df1["Sprots 2019/2018"].str.split(expand=True) to split the two values which are separated by a space, I need to remove the FIRST space shown in the first value so that it doesn't split into three columns.
I've tried df1["Sports 2019/2018"] = df1["Sports 2019/2018"].str.replace(" ", "") but this removes all the spaces, which would then combine the two values.
Is there a way to remove the first whitespace on column "Sports 2019/2018 so that it resembles the values on "Internet 2019/2018'?
df1["Sports 2019/2018"] = df1["Sports 2019/2018"].str.replace(" ", "", n = 1)
n=1 is an argument that will only replace the first character that will find.
I have Excel sheet which contains data similar to
Addresses
xyz,abc,olk
opn,opk,prt
we-ylj,tyf,uyfas
oiui,ytfy,tydry - We also work in bla,bla,bla
ytfyt,tyfyt,ghfyt
i-hgsd,gsdf-hgd,sdgh,- We also work in xxx,yy,zzz
ytsfgh,gfasdg,tydsfyt
I want to remove all substring which is next to the character "-" only if it's in the last position.
Result should be like
xyz,abc,olk
opn,opk,prt
we-ylj,tyf,uyfas
oiui,ytfy,tydry
ytfyt,tyfyt,ghfyt i-hgsd,gsdf-hgd,sdgh
ytsfgh,gfasdg,tydsfyt
I tried with =Substitute function but unable to replace data because of the last substring separated from "-" is not similar.
Going by your specifications, I would use two columns just so it's not a very long formula:
In B1:
=IFERROR(FIND(CHAR(1),SUBSTITUTE(A1,"-",CHAR(1),LEN(A1)-LEN(SUBSTITUTE(A1,"-",""))))-1,LEN(A1))
This gets the position of the last - or the full text length.
Then in C1:
=LEFT(A1,IF(FIND(",",A1)<B1,B1,LEN(A1)))
This checks if there's a , before the last -. If there is no ,, then the full text is taken.
EDIT: I only now noticed your edited comment. If it's just everything after - We, then I would use this:
=TRIM(LEFT(A1,IFERROR(FIND("- We",A1)-2,LEN(A1))))
I'm using python 3.x and openpyxl to parse an excel .xlsx file.
For each row, I check a column (C) to see if any of those keywords match.
If so, I add them to a separate list variable and also determine how many keywords were matched.
I then want to add the actual keywords into the next cell, and the total of keywords into the cell after. This is where I am having trouble, actually writing the results.
contents of the keywords.txt and results.xlsx file
here
import openpyxl
# Here I read a keywords.txt file and input them into a keywords variable
# I throwaway the first line to prevent a mismatch due to the unicode BOM
with open("keywords.txt") as f:
f.readline()
keywords = [line.rstrip("\n") for line in f]
# Load the workbook
wb = openpyxl.load_workbook("results.xlsx")
ws = wb.get_sheet_by_name("Sheet")
# Iterate through every row, only looking in column C for the keyword match.
for row in ws.iter_rows("C{}:E{}".format(ws.min_row, ws.max_row)):
# if there's a match, add to the keywords_found list
keywords_found = [key for key in keywords if key in row[0].value]
# if any keywords found, enter the keywords in column D
# and how many keywords into column E
if len(keywords_found):
row[1].value = keywords_found
row[2].value = len(keywords_found)
Now, I understand where I'm going wrong, in that ws.iter_rows(..) returns a tuple, which can't be modified. I figure I could two for loops, one for each row, and another for the columns in each row, but this test is a small example of a real-world scenario, where the amount of rows are in the tens of thousands.
I'm not quite sure which is the best way to go about this. Thankyou in advance for any help that you can provide.
Use the ws['C'] and then the offset() method of the relevant cell.
Thanks Charlie for the offset() tip. I modified the code slightly and now it works a treat.
for row in ws.iter_rows("C{}:C{}"...)
for cell in row:
....
if len(keywords_found):
cell.offset(0,1).value = str(keywords_found)
cell.offset(0,2).value = str(len(keywords_found))
Simple question here: I am trying to reference two columns and then divide them, but the column titles contain spaces.
Title 1: First Word Sum
Title 2: First Word Clicks
When I try to do something like this, it doesn't work:
cvr = (final.First Word Sum / final.First Word Clicks)
How do I rectify this? Thank you
You cannot do it the way you are trying to. However the following would work:
cvr = final['First Word Sum'] / final['First Word Clicks']