Python3: counting and extracting letters out of a text - python-3.x

Question

I'm doing some processing tasks on a medium-sized (1.7 Mb) Persian text corpus. I want to make lists of three set of characters in the text:
alphabets
white spaces (including newline, tab, space, no-breaking space and etc.) and
punctuation.
I wrote this:
# -*- coding: utf8 -*-
TextObj = open ('text.txt', 'r', encoding = 'UTF8')
import string
LCh = LSpc = LPunct = []
TotalCh = TotalPunct = TotalSpc = 0
TempSet = 'ابپتثجچحخدذرزژسشصضطظعغفقکگلمنوهی'
#TempSet variable holds alphabets of Persian language.
ReadObj = TextObj.read ()
for Char in ReadObj:
if Char in TempSet: #This's supposed to count & extract alphabets only.
TotalCh += 1
LCh.append (Char)
elif Char in string.punctuation: #This's supposed to count puncts.
TotalPunct += 1
LPunct.append (Char)
elif Char in ('', '\n', '\t'): #This counts & extracts spacey things.
TotalSpc += 1
LSpc.append (Char)
else: #This'll ignore anything else.
continue
But when I try:
print (LPunct)
print (LSpc)
I tried this code on both Linux and Windows 7. On both of them, the result is not what I expected at all. The punctuation's and space's lists, both contains Persian letters.
Another question:
How can I improve this condition elif Char in ('', '\n', '\t'): so that it covers all kind of space family?

Answer 1

On line 3 you've assigned all the lists to be the same list!
Don't do this:
LCh = LSpc = LPunct = []
Do this:
LCh = []
LSpc = []
LPunct = []
The string class has whitespace built in.
elif Char in string.whitespace:
TotalSpc += 1
LSpc.append (Char)
In your example you didn't actually put a space in your '' character which also may be causing it to fail. Shouldn't this be ' '?
Also, take into account the other answer here, this code is not very pythonic.
I'd write it like this:
# -*- coding: utf8 -*-
import fileinput
import string
persian_chars = 'ابپتثجچحخدذرزژسشصضطظعغفقکگلمنوهی'
filename = 'text.txt'
persian_list = []
punctuation_list = []
whitespace_list = []
ignored_list = []
for line in fileinput.input(filename):
for ch in line:
if ch in persian_chars:
persian_list.append(ch)
elif ch in string.punctuation:
punctuation_list.append(ch)
elif ch in string.whitespace:
whitespace_list.append(ch)
else:
ignored_list.append(ch)
total_persian, total_punctuation, total_whitepsace = \
map(len, [persian_list, punctuation_list, whitespace_list])

Answer 2

First of all as a more pythonic way for dealing with files you better to use with statement for opening the files which will close the file at the end of the block.
Secondly since you want to count the number of special characters within your text and preserve them separately, you can use a dictionary with the list names as the keys and relative characters in a list as value. Then use len method to get the length.
And finally for check the membership in whitespaces you can use string.whitespace method.
import string
TempSet = 'ابپتثجچحخدذرزژسشصضطظعغفقکگلمنوهی'
result_dict={}
with open ('text.txt', 'r', encoding = 'UTF8') as TextObj :
ReadObj = TextObj.read ()
for ch in ReadObj :
if Char in TempSet:
result_dict['TempSet'].append(ch)
elif Char in string.punctuation:
result_dict['LPunct'].append(ch)
elif Char in string.whitespace:
result_dict['LSpc'].append(ch)
TotalCh =len(result_dict['LSpc'])