transform text file to data table using python - python-3.x

Question

I have text data available in the following format:
title:- sell product A,
description:- good quality product, fast service,
title:- sell product B,
description:- long lasting and good quality,
title:- buy product C,
description:- latest, good price
My goal is to get the text values into a dataframe with title and description values in corresponding columns.For example:
title
Description
sell product A
good quality product, fast service
sell product B
long lasting and good quality
buy product C
latest, good price

Answer 1

Edit Ver 1: Read file and Split to two columns
I am assuming that the pattern of your txt file is
title:-followed by data
description:-followed by data
title:-immediately after description line
description:-immediately after title line
If the pattern is not title:- and description:- sequence one after the other, the below code will NOT work.
I also assume there is no line blank line separating the title and description lines. If there are blank lines, I need to exclude the blank lines first before we do the splitting into title and description.
Read the full file into a list, then separate the data into two lists, then strip the leading text and load into pandas. I have added comments to each line so you can see what's being done.
import pandas as pd
with open('abc.txt','r') as f:
data = [line.rstrip() for line in f.readlines()]
#read all rows into a list while also striping \n
#[::2] selects alternate rows, starting from 0. [8:] excludes "title:-"
#adding rstrip(',') removes the trailing "," from the string
title = [d[8:].rstrip(',') for d in data[::2]]
#[1::2] selects alternate rows, starting from 1. [14:] excludes "description:-"
#adding rstrip(',') removes the trailing "," from the string
desc = [d[14:].rstrip(',') for d in data[1::2]]
df = pd.DataFrame({'Title':title,'Description':desc})
print (df)
The output of this will be:
Title Description
0 sell product A good quality product, fast service
1 sell product B long lasting and good quality
2 buy product C latest, good price
For the above example, the source data in abc.txt file is as follows:
title:- sell product A,
description:- good quality product, fast service,
title:- sell product B,
description:- long lasting and good quality,
title:- buy product C,
description:- latest, good price
Edit Ver 2: If title & description rows do not match
If you are unsure of the datafile and want to play it safe, you can create the dataframe as follows:
df = pd.DataFrame({'Title':pd.Series(title),'Description':pd.Series(desc)})
Here's an example:
I modified the input file to be as follows:
title:- sell product A,
description:- good quality product, fast service,
title:- sell product B,
description:- long lasting and good quality,
title:- buy product C,
description:- latest, good price
title:- buy product D,
The output of this will be:
Title Description
0 sell product A good quality product, fast service
1 sell product B long lasting and good quality
2 buy product C latest, good price
3 buy product D NaN
In the above example, the pattern breaks for the last row. We don't have a matching description value for the title. This results in a NaN in the Description column.
Edit Ver 3: Input file has blank line after each row
If your input file has a blank line in between Title and Description rows, then you can add this code immediately after reading the file.
with open('abc.txt','r') as f:
#read all rows into a list while also striping \n
data = [line.rstrip() for line in f.readlines()]
#removing the blank lines from data
data = data[::2]
Input sample file:
title:- sell product A,
description:- good quality product, fast service,
title:- sell product B,
description:- long lasting and good quality,
title:- buy product C,
description:- latest, good price
title:- buy product D,
Output DataFrame:
Title Description
0 sell product A good quality product, fast service
1 sell product B long lasting and good quality
2 buy product C latest, good price
3 buy product D NaN