I have a text file where data is comma delimited with a litral \n character in between, i would like to insert the data into newline just after getting the \n character.
text file sample:
'what,is,your,name\n','my,name,is,david.hough\n','i,am,a,software,prof\n','what,is,your,name\n','my,name,is,eric.knot\n','i,am,a,software,prof\n','what,is,your,name\n','my,name,is,fisher.cold\n','i,am,a,software,prof\n',..
expected:
I need the output in the below form.
'what,is,your,name',
'my,name,is,david.hough',
'i,am,a,software,prof',
Tried:
file1 = open("test.text", "r")
Lines = file1.readlines()
for line in Lines:
print(line)
result:
'what,is,your,name\n','my,name,is,david.hough\n','i,am,a,software,prof\n','what,is,your,name\n','my,name,is,eric.knot\n','i,am,a,software,prof\n','what,is,your,name\n','my,name,is,fisher.cold\n','i,am,a,software,prof\n',..
well my comment does exactly what you asked, break lines at \n. your data is structured quite weirdly, but if you want the expected result that badly you can use regex
import re
file1 = open("test.text","r")
Lines = re.findall(r'\'.*?\',',file1.read().replace("\\n",""))
for line in Lines:
print(line)
Well you don't need push data to the other line manually. The \n does that work when you run the code.
I guess the problem is that you used quotes very frequently, try using a single pair of quotes and use \n after the first sentence and yeah without white space
'what,is,your,name\nmy,name,is,david.hough\ni,am,a,software,prof'
This is the first time I am trying to do python execution in GNU parallel.
I have the below python script. I am trying to run it in parallel with a text.txt document loading the variables. The text document has the variables one on each line.
I execute the below script with this code:
parallel --bar -a PairNames.txt python3 CreateDataTablePythonScriptv2.py
Here is the python script being executed:
import sqlite3
import sys
PairName = sys.argv[1]
print(PairName)
DTBLocation = '//mnt//c//Users//Jonathan//OneDrive - Mazars in Oman//Trading//Systems//FibMatrix//Testing Trade Analysis//SQLite//Trade Analysis.db
connection = sqlite3.connect(DTBLocation)
cursor = connection.cursor()
TableName = PairName+'_DATA'
print(TableName)
cursor.execute("""CREATE TABLE IF NOT EXISTS {}
(
Date_Time INTEGER,
Open REAL,
Max_60m_Box REAL
)""".format(TableName))
connection.commit()
connection.close()
It executes correctly the first variable just fine. But the remainder of the variables do print correctly from the print command for the PairName, but for print(TableName) I get the below displays:
GBPUSD
_DATAD
USDCHF
_DATAF
NZDJPY
_DATAY
Its weird to me that it prints the PairName just fine and correctly, but then the PairName does not show up when concating the TableName.
Also, its weird that an extra letter gets added to the end of DATA for each one. It appears that the extra letter at the end of the DATA is the last letter of the input variable. I don't know why its choping the 5 letters off and how it puts it at the end of the DATA.
I printed the tablename.
I watched this video at https://www.youtube.com/watch?v=OpaiGYxkSuQ&ab_channel=OleTange[^]
I tried moving the TableName concat to right under the PairName
I printed the type of the PairName, and it is a string
I tried seperating the varibales in the txt document by tabs and commas instead of next line
I tried assigning the "_DATA" to a variable and then concating the two objects. But it had same result:
TableEnd = '_DATA'
TableName = PairName + TableEnd
If I remove the concat of PairName+'_DATA' and just use PairName only as the TableName, then it works correctly.
Sorry if this is a simple answer, but I cannot figure it out and especially since there is not too much documentation / tutorials for a newbie on GNU Parallel in this situation. Thanks for the help!
The input file is not in DOS format (i.e. ends in a CRLF rather than just an LF)? I checked this using the FILE command:
$ file test.txt
test.txt: ASCII text, with CRLF line terminators
$
Since it was CRLF (DOS format), I converted it using tr:
Copy Codetr -d '\r' < input.file > output.file```
sample input
in parsing a text file .txt = ["'blah.txt'", "'blah1.txt'", "'blah2.txt'" ]
the expected output in another text file out_path.txt
blah.txt
blah1.txt
blah2.txt
Code that I tried, this just appends "[]" to the input file. While I also tried perl one liner replacing double and single quotes.
read_out_fh = open('out_path.txt',"r")
for line in read_out_fh:
for word in line.split():
curr_line = re.findall(r'"(\[^"]*)"', '\n')
print(curr_line)
this happens because while you reading a file it will be taken as string and not as a list even if u kept the formatting of a list. thats why you getting [] while doing re.for line in read_in_fh: here you are taking each letters in the string thats why you are not getting the desired output. so iwrote something first to transform the string into a list. while doing that i also eliminated "" and '' as you mensioned. then wrote it in to a new file example.txt.
Note: change the file name according to your files
read_out_fh = open('file.txt',"r")
for line in read_out_fh:
line=line.strip("[]").replace('"','').replace("'",'').split(", ")
with open("example.txt", "w") as output:
for word in line:
#print(word)
output.write(word+'\n')
example.txt(outputfile)
blah.txt
blah1.txt
blah2.txt
The code below works out for your example you gave in the question:
# Content of textfile.txt:
asdasdasd=["'blah.txt'", "'blah1.txt'", "'blah2.txt'"]asdasdasd
# Code:
import re
read_in_fh = open('textfile.txt',"r")
write_out_fh = open('out_path.txt', "w")
for line in read_in_fh:
find_list = re.findall(r'\[(".*?"*)\]', line)
for element in find_list[0].split(","):
element_formatted = element.replace('"','').replace("'","").strip()
write_out_fh.write(element_formatted + "\n")
write_out_fh.close()
# open file in read mode
f=open(text_file,'r')
# iterate over the file object
for line in f.read():
print(line)
# close the file
f.close()
the content of file is "Congratulations you have successfully opened the file"! when i try to run this code the output comes in following form:
c (newline) o (newline) n (newline) g.................
...... that is each character is printed individually on a new line because i used read()! but with readline it gives the answer in a single line! why is it so?
r.read() returns one string will all characters (the full file content).
Iterating a string iterates it character wise.
Use
for line in f: # no read()
instead to iterate line wise.
f.read() returns the whole file in a string. for i in iterates something. For a string, it iterates over its characters.
For readline(), it should not print the line. It would read the first line of the file, then print it character by character, like read. Is it possible that you used readlines(), which returns the lines as a list.
One more thing: there is with which takes a "closable" object and auto-closes it at the end of scope. And you can iterate over a file object. So, your code can be improved like this:
with open(text_file, 'r') as f:
for i in f:
print(i)
I have a csv file that is being fed to my Python script via stdin.
This is a comma separated file with quotations as text delimiter.
Here is an example line:
457,"Last,First",NYC
My script so far, splits each line by looking for commas, but how do I make it aware of the text delimiter quotes?
My current script:
for line in sys.stdin:
line = line.strip()
line.split(',')
print line
The code splits the name into two since it does not recognize the quotations enclosing that text field. I need the name to remain as a single element.
If it matters, the data is being fed through stdin within a hadoop-streaming program.
Thanks!
Well, you could do it more manually, with something like this:
row = []
enclosed = False
word = ''
for character in sys.stdin:
if character == '"':
enclosed = not enclosed
elif character = ',' and not enclosed:
row.append(word)
word = ''
else:
word += character
Haven't tested nor thought about it for too long but seems to me it could work. Probably someone more into Pythonist sintax could fine something better for doing the trick although ;)
Attempting to answer my own question. If I read right, it may be possible to send a streaming input into csv reader like so:
for line in csv.reader(sys.stdin):
print line