Concat multiple CSV rows into 1 in python - python-3.x

I am trying to contact the CSV rows. I tried to convert the CSV rows to list by pandas but it gets 'nan' values appended as some files are empty.
Also, I tried using zip but it concats column values.
with open(i) as f:
lines = f.readlines()
res = ""
for i, j in zip(lines[0].strip().split(','), lines[1].strip().split(',')):
res += "{} {},".format(i, j)
print(res.rstrip(','))
for line in lines[2:]:
print(line)
I have data as below,
Input data:-
Input CSV Data
Expected Output:-
Output CSV Data
The number of rows are more than 3,only sample is given here.
Suggest a way which will achieve the above task without creating a new file. Please point to any specific function or sample code.

This assumes your first line contains the correct amount of columns. It will read the whole file, ignore empty data ( ",,,,,," ) and accumulate enough data points to fill one row, then switch to the next row:
Write test file:
with open ("f.txt","w")as f:
f.write("""Circle,Year,1,2,3,4,5,6,7,8,9,10,11,12
abc,2018,,,,,,,,,,,,
2.2,8.0,6.5,9,88,,,,,,,,,,
55,66,77,88,,,,,,,,,,
5,3.2,7
def,2017,,,,,,,,,,,,
2.2,8.0,6.5,9,88,,,,,,,,,,
55,66,77,88,,,,,,,,,,
5,3.2,7
""")
Process test file:
data = [] # all data
temp = [] # data storage until enough found , then put into data
with open("f.txt","r") as r:
# get header and its lenght
title = r.readline().rstrip().split(",")
lenTitel = len(title)
data.append(title)
# process all remaining lines of the file
for l in r:
t = l.rstrip().split(",") # read one lines data
temp.extend( (x for x in t if x) ) # this eliminates all empty ,, pieces even in between
# if enough data accumulated, put as sublist into data, keep rest
if len (temp) > lenTitel:
data.append( temp[:lenTitel] )
temp = temp [lenTitel:]
if temp:
data.append(temp)
print(data)
Output:
[['Circle', 'Year', '1', '2', '3', '4', '5', '6', '7', '8', '9', '10', '11', '12'],
['abc', '2018', '2.2', '8.0', '6.5', '9', '88', '55', '66', '77', '88', '5', '3.2', '7'],
['def', '2017', '2.2', '8.0', '6.5', '9', '88', '55', '66', '77', '88', '5', '3.2', '7']]
Remarks:
your file cant have leading newlines, else the size of the title is incorrect.
newlines in between do not harm
you cannot have "empty" cells - they get eliminated

As long as nothing weird is going on in the files, something like this should work:
with open(i) as f:
result = []
for line in f:
result += line.strip().split(',')
print(result)

Related

How to iterate a list, to get combination of all next values

Problem: Obtain the combination of all next values of the array.
Explanation: All the values should be formed by contiguous elements in the array.
Posible solution(not optimal):
l_number= list("24256")
result= []
while l_number:
n = ""
for x in l_number:
n += x
result.append(str(n))
l_number.pop(0)
print(result)
The output:
['2', '24', '242', '2425', '24256', '4', '42', '425', '4256', '2', '25', '256', '5', '56', '6']
I got this, but it is not very efficient.
Is there any optimized way to do this?
Since you will be outputting all contiguous sublists, the size of the output will be O(N^2) where N is the length of the list, and time complexity will be at least quadratic no matter what (You could achieve linear space complexity by using a generator). Here is one way to compute all contiguous substrings with a list comprehension:
s = "24256"
n = len(s)
res = [s[i:j] for i in range(n) for j in range(i+1,n+1)]
print(res)
# ['2', '24', '242', '2425', '24256', '4', '42', '425', '4256', '2', '25', '256', '5', '56', '6']
As an aside, it is usually undesirable to build up lists with str += ... since a new copy of str is created on each iteration. One workaround is to create a list of components and call .join once all components have been found.

Why list remains unchanged Python

After iterating through a list to change each value to an integer, the list remains unchanged with all the values still being strings.
As a result, sorting does not get applied either
a = ['14', '22', '4', '52', '54', '59']
for ea in a:
ea = int(ea)
a.sort()
print (a)
Output: '14', '22', '4', '52', '54', '59'
Should be : 4, 14, 22, 52, 54, 59
Your code is not changing the list itself. You are creating a new variable, converting it to an int, and throwing it away.
Use this instead
a = ['14', '22', '4', '52', '54', '59']
a = list(map(int, a)) #this converts the strings into integers and assigns the new list to a
a.sort() #this sorts it
print (a)
ea = int(ea) is not changing the element within the list. So as you do not change the list (which can be seen if you print the list before sorting it), the sort operation is doing it's job correctly because it is sorting strings here, not integer values.
You could change your loop to provide the index and modify the original entries in the list by using the enumerate function as follows:
a = ['14', '22', '4', '52', '54', '59']
for index, ea in enumerate(a):
a[index] = int(ea)
a.sort()
print(a)

How with open works?

Today I was trying to open a file and build some list from the data of the file. I am using the with statement. But I have the following doubt:
If I write the following code:
def Preset_wheel_filler(self):
"""
Complete the Preset_wheel and also apply preset values when one
preset is selected.
"""
with open('Preset.txt', 'r') as PresetFile:
Presets = [line.split()[1:] for line in PresetFile if 'name:'
in line.split()]
with open('Preset.txt', 'r') as PresetFile:
channel_list = ['1', '2', '3', '4', '5', '6', '7', '8']
Preset_values = [line.split() for line in PresetFile for
y in channel_list if y in line.split()]
print(len(Preset_values))
The length of the last list created is 16. (Which is correct)
Now, if I rearrange the code like this:
def Preset_wheel_filler(self):
"""
Complete the Preset_wheel and also apply preset values when one
preset is selected.
"""
with open('Preset.txt', 'r') as PresetFile:
Presets = [line.split()[1:] for line in PresetFile if 'name:'
in line.split()]
channel_list = ['1', '2', '3', '4', '5', '6', '7', '8']
Preset_values = [line.split() for line in PresetFile for
y in channel_list if y in line.split()]
print(len(Preset_values))
The printed length is 0.
My question is: Why should I write the with open statement twice?
Thanks in advance.
Files are stream based. Once you read all the data in the file its position pointer is at the end and nothing can be read anymore. You "could" seek to 0 to fix that, but - you have the full line in the first go, simply parse both things from it.
PresetFile.seek(0) # goes back to the beginning of the file.
See https://docs.python.org/3/tutorial/inputoutput.html#methods-of-file-objects
for details on how to use seek and maybee peek into https://docs.python.org/3/library/functions.html#open
again.

Python CSV - Writing different lists in the same CSV-file

SEE UPDATE BELOW!
For my Python program I need to write 3 different lists to a csv file, each in a different column. Each lists has a different size.
l1 = ['1', '2', '3', '4', '5']
l2 = ['11', '22', '33', '44']
l3 = ['111', '222', '333']
f = 'test.csv'
outputFile = open(f, 'w', newline='')
outputWriter = csv.writer(resultFile, delimiter=';')
outputWriter.writerow(headerNames)
for r in l3:
resultFile.write(';' + ';' + r + '\n')
for r in l2:
resultFile.write(';' + r + '\n')
for r in l1:
resultFile.write(r + '\n')
resultFile.close()
Unfortunately this doesn't work. The values of the lists are written below each other list in the column to the right. I would prefer to have the list values written beside one another just like this:
1;11;111
2;22;222
etc.
I am sure there is an easy way to get this done, but after hours of trying I still cannot figure it out.
UPDATE:
I tried the following. It is progress, but I am still not there yet.
f = input('filename: ')
l1 = ['1', '2', '3', '4', '5']
l2 = ['11', '22', '33', '44']
l3 = ['111', '222', '333']
headerNames = ['Name1', 'Name2', 'Name3']
rows = zip(l1, l2, l3)
with open(f, 'w', newline='') as resultFile:
resultWriter = csv.writer(resultFile, delimiter=';')
resultWriter.writerow(headerNames)
for row in rows:
resultWriter.writerow(row)
It write the data in the format I would like, however the values 4, 5 and 44 are not writen.
Your first attempt is not using the csv module properly, nor transposing the rows like your second attempt does.
Now zipping the rows will stop as soon as the shortest row ends. You want itertools.ziplongest instead (with a fill value of 0 for instance)
import itertools,csv
f = "out.csv"
l1 = ['1', '2', '3', '4', '5']
l2 = ['11', '22', '33', '44']
l3 = ['111', '222', '333']
headerNames = ['Name1', 'Name2', 'Name3']
rows = itertools.zip_longest(l1, l2, l3, fillvalue=0)
with open(f, 'w', newline='') as resultFile:
resultWriter = csv.writer(resultFile, delimiter=';')
resultWriter.writerow(headerNames)
resultWriter.writerows(rows) # write all rows in a row :)
output file contains:
Name1;Name2;Name3
1;11;111
2;22;222
3;33;333
4;44;0
5;0;0

Matrix input from a text file(python 3)

Im trying to find a way to be able to input a matrix from a text file;
for example, a text file would contain
1 2 3
4 5 6
7 8 9
And it would make a matrix with those numbers and put it in matrix = [[1,2,3],[4,5,6],[7,8,9]]
And then this has to be compatible with the way I print the matrix:
print('\n'.join([' '.join(map(str, row)) for row in matrix]))
So far,I tried this
chemin = input('entrez le chemin du fichier')
path = input('enter file location')
f = open ( path , 'r')
matrix = [ map(int,line.split(','))) for line in f if line.strip() != "" ]
All it does is return me a map object and return an error when I try to print the matrix.
What am I doing wrong? Matrix should contain the matrix read from the text file and not map object,and I dont want to use external library such as numpy
Thanks
You can use list comprehension as such:
myfile.txt:
1 2 3
4 5 6
7 8 9
>>> matrix = open('myfile.txt').read()
>>> matrix = [item.split() for item in matrix.split('\n')[:-1]]
>>> matrix
[['1', '2', '3'], ['4', '5', '6'], ['7', '8', '9']]
>>>
You can also create a function for this:
>>> def matrix(file):
... contents = open(file).read()
... return [item.split() for item in contents.split('\n')[:-1]]
...
>>> matrix('myfile.txt')
[['1', '2', '3'], ['4', '5', '6'], ['7', '8', '9']]
>>>
is working with both python2(e.g. Python 2.7.10) and python3(e.g. Python 3.6.4)
rows=3
cols=3
with open('in.txt') as f:
data = []
for i in range(0, rows):
data.append(list(map(int, f.readline().split()[:cols])))
print (data)

Resources