I have developed the following code and fetched the matched output using a for loop.I need to print these output elements in separate column using python.
excel file name - Sample_data.xlsx
first column - WBS_CODE
second column - PROJECT_CODE
first column and second column are matched and then printed in separate column (column F) using python code. Please find my below code,
import pandas as pd
A = pd.read_excel("D:\python_work\Sample_data.xlsx", sheet_name = '31Sep')
code = A['WBS_CODE'].tolist()
B = pd.read_excel("D:\python_work\Sample_data.xlsx", sheet_name = '4Dec')
code1 = B['PROJECT_CODE'].tolist()
for x in code1:
if x in code:
print(x)
else:
print("NA")
output:
NA
NA
NA
APP-ACI-PJ-APAC-EMEA-ENG
NA
NA
I have found a way to export the output and print them in a separate column in excel sheet. Below is the solution,
import pandas as pd
from openpyxl import load_workbook
# Reading the Excel file columns
A = pd.read_excel("D:\python_work\Sample_data.xlsx", sheet_name='4Dec')
code = A['PROJECT_CODE'].tolist()
B = pd.read_excel("D:\python_work\Sample_data.xlsx", sheet_name='31Sep')
code1 = B['WBS_CODE'].tolist()
# Comparison of columns
class test:
def loop(self):
result = []
for x in code1:
if x in code:
result.append(x)
else:
y = "NA"
result.append(y)
print(result)
# Printing data into Excel
try:
book = load_workbook('D:\python_work\Aew1.xlsx')
writer = pd.ExcelWriter('D:\python_work\Aew1.xlsx', engine='openpyxl')
writer.book = book
writer.sheets = dict((ws.title, ws) for ws in book.worksheets) # loading all the worksheets in opened Excel
df = pd.DataFrame.from_dict({'Column1': result})
df.to_excel(writer, sheet_name='Sheet1', startcol=19)
writer.save()
except FileNotFoundError:
print("File Not found: 'Check the Name or Existence of file specified/'")
except PermissionError:
print("File Opened/No-Access: Check whether you have access to file or file is opened")
test().loop()
steps that solved:
1. Appended the for loop output to a list
2. used openpyxl library to print the output to a column in excel worksheet.
Thanks guyz for help and support. Have a good day
Related
I use
df = pd.read_clipboard()
list_name = df['column_name'].to_list()
but this is a bit long method for me. I want to copy a column and convert in python and then apply some function so that the copied text is converted to a list.
this will read a excel column as list
import xlrd
book = xlrd.open_workbook('Myfile.xlsx') #path to your file
sheet = book.sheet_by_name("Sheet1") #Sheet name
def Readlist(Element, Column):
for _ in range(1,sheet.nrows):
Element.append(str(sheet.row_values(_)[Column]))
pass
column1 = [] # List name
Readlist(column1, 1) # Column Number is 1 here
pirnt(column1)
Read a specified column as list use Readlist function, intialize [] variable before using that.
Using Pandas:
import pandas as pd
df = pd.read_excel("path.xlsx", index_col=None, na_values=['NA'], usecols = "A")
mylist = list(df[0])
print(mylist)
I have extracted data from different excel sheets spread in different folders, I have organized the folders numerically from 2015 to 2019 and each folder has twelve subfolders (from 1 to 12) here's my code:
import os
from os import walk
import pandas as pd
path = r'C:\Users\Sarah\Desktop\IOMTest'
my_files = []
for (dirpath, dirnames, filenames) in walk(path):
my_files.extend([os.path.join(dirpath, fname) for fname in filenames])
all_sheets = []
for file_name in my_files:
#Display sheets names using pandas
pd.set_option('display.width',300)
mosul_file = file_name
xl = pd.ExcelFile(mosul_file)
mosul_df = xl.parse(0, header=[1], index_col=[0,1,2])
#Read Excel and Select columns
mosul_file = pd.read_excel(file_name, sheet_name = 0 ,
index_clo=None, na_values= ['NA'], usecols = "A, E, G, H , L , M" )
#Remove NaN values
data_mosul_df = mosul_file.apply (pd.to_numeric, errors='coerce')
data_mosul_df = mosul_file.dropna()
print(data_mosul_df)
then I saved the extracted columns in a csv file
def save_frames(frames, output_path):
for frame in frames:
frame.to_csv(output_path, mode='a+', header=False)
if __name__ == '__main__':
frames =[pd.DataFrame(data_mosul_df)]
save_frames(frames, r'C:\Users\Sarah\Desktop\tt\c.csv')
My problem is that when I open the csv file it seems that it doesn't store all the data but only the last excel sheet that it has read or sometimes the two last excel sheets. however, when I print my data inside the console (in Spyder) I see that all the data are treated
data_mosul_df = mosul_file.apply (pd.to_numeric, errors='coerce')
data_mosul_df = mosul_file.dropna()
print(data_mosul_df)
the picture below shows the output csv created. I am wondering if it is because from Column A to Column E the information are the same ? so that's why it overwrite ?
I would like to know how to modify the code so that it extract and store the data chronologically from folders (2015 to 2019) taking into accout subfolders (from 1 to 12) in each folder and how to create a csv that stores all the data ? thank you
Rewrite your loop:
for file_name in my_files:
#Display sheets names using pandas
pd.set_option('display.width',300)
mosul_file = file_name
xl = pd.ExcelFile(mosul_file)
mosul_df = xl.parse(0, header=[1], index_col=[0,1,2])
#Read Excel and Select columns
mosul_file = pd.read_excel(file_name, sheet_name = 0 ,
index_clo=None, na_values= ['NA'], usecols = "A, E, G, H , L , M" )
#Remove NaN values
data_mosul_df = mosul_file.apply (pd.to_numeric, errors='coerce')
data_mosul_df = mosul_file.dropna()
#Make a list of df's
all_sheets.append(data_mosul_df)
Rewrite your save_frames:
def save_frames(frames, output_path):
frames.to_csv(output_path, mode='a+', header=False)
Rewrite your main:
if __name__ == '__main__':
frames = pd.concat(all_sheets)
save_frames(frames, r'C:\Users\Sarah\Desktop\tt\c.csv')
This is what i try
from pathlib import Path
loc = Path('D:\DataSciSpec\Practice\Forloopindict.xlsx')
dict = pd.read_excel(loc,sheetname = None)
for i in dict.keys():
print(i)
I get the name of sheets
Sheet4
Sheet3
Sheet2
Sheet1
I can also see the sheet content one by one
for i in dict.keys():
print(dict[i].head())
But how put this data in n data frames (equal to no of sheets)
and then append one to another
This will create a single dataframe (df_full) with the data from all sheets.
import pandas as pd
loc = r'D:\DataSciSpec\Practice\Forloopindict.xlsx'
workbook = pd.read_excel(loc,sheet_name = None)
df_full = pd.DataFrame()
for _, sheet in workbook.items():
df_full = df_full.append(sheet)
# Reset index or you'll have duplicates
df_full = df_full.reset_index(drop=True)
I am trying to import certain columns of data from several different sheets inside of a workbook. However, while appending it only seems to append 'q2 survey' to a new workbook. How do I get this to append properly?
import sys, os
import pandas as pd
import xlrd
import xlwt
b = ['q1 survey', 'q2 survey','q3 survey'] #Sheet Names
df_t = pd.DataFrame(columns=["Month","Date", "Year"]) #column Name
xls = "path_to_file/R.xls"
sheet=[]
df_b=pd.DataFrame()
pd.read_excel(xls,sheet)
for sheet in b:
df=pd.read_excel(xls,sheet)
df.rename(columns=lambda x: x.strip().upper(), inplace=True)
bill=df_b.append(df[df_t])
bill.to_excel('Survey.xlsx', index=False)
I think if you do:
b = ['q1 survey', 'q2 survey','q3 survey'] #Sheet Names
list_col = ["Month","Date", "Year"] #column Name
xls = "path_to_file/R.xls"
#create the empty df named bill to append after
bill= pd.DataFrame(columns = list_col)
for sheet in b:
# read the sheet
df=pd.read_excel(xls,sheet)
df.rename(columns=lambda x: x.strip().upper(), inplace=True)
# need to assign bill again
bill=bill.append(df[list_col])
# to excel
bill.to_excel('Survey.xlsx', index=False)
it should work and correct the errors in your code, but you can do a bit differently using pd.concat:
list_sheet = ['q1 survey', 'q2 survey','q3 survey'] #Sheet Names
list_col = ["Month","Date", "Year"] #column Name
# read once the xls file and then access the sheet in the loop, should be faster
xls_file = pd.ExcelFile("path_to_file/R.xls")
#create a list to append the df
list_df_to_concat = []
for sheet in list_sheet :
# read the sheet
df= pd.read_excel(xls_file, sheet)
df.rename(columns=lambda x: x.strip().upper(), inplace=True)
# append the df to the list
list_df_to_concat.append(df[list_col])
# to excel
pd.concat(list_df_to_concat).to_excel('Survey.xlsx', index=False)
I have been working on code that takes rows from csv file and transfer them into the lists of integers for further mathematical operations. However, if a row turns out to be empty, it causes problems. Also, the user will not know which row is empty, so the solution should be general rather than pointing at a row and removing it. Here is the code:
import csv
import statistics as st
def RepresentsInt(i):
try:
int(i)
return True
except ValueError:
return False
l = []
with open('Test.csv', 'r') as f:
reader = csv.reader(f, delimiter=',')
for row in reader:
l.append([int(r) if RepresentsInt(r) else 0 for r in row])
for row in l:
row=[x for x in row if x!=0]
row.sort()
print(row)
I've tried l=[row for row in l if row!=[]] and ...
if row==[]:
l.remove(row)
... but both do nothing, and there is no error code for either. Here is my csv file:
1,2,5,4
2,3
43,65,34,56,7
0,5
7,8,9,6,5
33,45,65,4
If I run the code, I will get [] for row 4 and 6 (which are empty).
This worked on my machine:
import csv
def RepresentsInt(i):
try:
int(i)
return True
except ValueError:
return False
l = []
with open('Test.csv', 'r') as f:
reader = csv.reader(f, delimiter=',')
for row in reader:
l.append([int(r) for r in row if RepresentsInt(r)])
rows = [row for row in l if row]
for row in rows:
print(row)
It is unclear what you are doing with the statistics module, but the following program should you what you asked for. The pprint module is imported to make displaying the generated table easier to read. If this answer solves the problem presented in your question but you are having difficulty somewhere else, make sure you open another question targeted at the new problem.
#! /usr/bin/env python3
import csv
import pprint
def main():
table = []
# Add rows to table.
with open('Test.csv', newline='') as file:
table.extend(csv.reader(file))
# Convert table cells to numbers.
for index, row in enumerate(table):
table[index] = [int(value) if value.isdigit() else 0 for value in row]
# Remove zeros from the rows.
for index, row in enumerate(table):
table[index] = [value for value in row if value]
# Remove empty rows and display the table.
table = [row for row in table if row]
pprint.pprint(table)
if __name__ == '__main__':
main()