I have a spreadsheet (.ods format) that looks like this:
Column A Column B
abstemious marked by restraint
ameliorate to make better or more tolerable
amphora an ancient jar with 2 handles on the top
How can I convert it to a text file with the following format?
Q: abstemious
A: marked by restraint
Q: ameliorate
A: to make better or more tolerable
Q: amphora
A: an ancient jar with 2 handles on the top
Note the Q:, A:, and the new lines. I searched on Google and stack overflow and tried many things including converting to .csv as an intermediate step and using "find and replace", creating columns with "\n" in the spreadsheet, and also using "#13#10" which should indicate a new line but it doesn't seem to translate to the final .txt file.
Also, for formatting this post, I indented everything by 4 spaces and typed by hand, is there a better way to simulate a spreadsheet on stack overflow (eg if I was going to post something more complicated)? I looked at http://daringfireball.net/projects/markdown/syntax but it is not clear what I would do if I wanted to post part of a spreadsheet with multiple columns and rows. I searched on this topic and also didn't find anything. Thanks!
There is probably a much easier way to do it but I figured out one way. First, use a symbol that does not appear in the file (eg, '$') as a field separator and convert the .ods to a .csv. Next I wrote this code in Python:
def convert():
f = open(r'name_of_file.csv','r')
contents = f.read()
contents = contents.split('\n')
contents.pop()
p = []
for element in contents:
p.append(element.split('$'))
for element in p:
print('Q: ' + element[0])
print('A: ' + element[1])
print('')
No reason to post other answers as this should suffice now that the program is written.
Related
I've accumulated a list of over 10,000 text files in Octave. I've got a function which cleans up the contents of each files, normalizing things in various ways (lowercase, reducing repeated whitespace, etc). I'd like to distill from all these files a list of words that appear in at least 100 files. I'm not at all familiar with Octave data structs or cell arrays or the Octave sorting functions, and was hoping someone could help me understand how to:
initialize an appropriate data structure (word_appearances) to count how many emails contain a particular word
loop thru the unique words that appear in an email string and increment for each of those words the count I'm tracking in word_appearances -- ideally we'd ignore words less than two chars in length and also exclude a short list of stop_words.
reduce word_appearances to only contain words that appear some number of times, e.g, min_appearances=100 times.
sort the words in word_appearances alphabetically and either export this as a .MAT file or as a CSV file like so:
1 aardvark
2 albatross
etc.
I currently have this code to loop through my files one by one:
for i = 1:total_files
filename = char(files{i}(1));
printf("processing %s\n", filename);
file_contents = jPreProcessFile(readFile(filename))
endfor
Note that the file_contents that comes back is pretty clean -- usually just a bunch of words, some repeated, separated by single spaces like so:
email market if done right is both highli effect and amazingli cost effect ok it can be downright profit if done right imagin send your sale letter to on million two million ten million or more prospect dure the night then wake up to sale order ring phone and an inbox full of freshli qualifi lead we ve been in the email busi for over seven year now our list ar larg current and deliver we have more server and bandwidth than we current need that s why we re send you thi note we d like to help you make your email market program more robust and profit pleas give us permiss to call you with a propos custom tailor to your busi just fill out thi form to get start name email address phone number url or web address i think you ll be delight with the bottom line result we ll deliv to your compani click here emailaddr thank you remov request click here emailaddr licatdcknjhyynfrwgwyyeuwbnrqcbh
Obviously, I need to create the word_appearances data structure such that each element in it specifies a word and how many files have contained that word so far. My primary point of confusion is what sort of data structure word_appearances should be, how I would search this data structure to see if some new word is already in it, and if found, increment its count, otherwise add a new element to word_appearances with count=1.
Octave has containers.Map to hold key-value pairs. This is the simple usage:
% initialize map
m = containers.Map('KeyType', 'char', 'ValueType', 'int32');
% check if it has a word
word = 'hello';
if !m.isKey(word)
m(word) = 1;
endif
% increment existing values
m(word) += 1;
This is one way to extract most frequent words from a map like the one above:
counts = m.values;
[sorted_counts, indices] = sort(cell2mat(counts));
top10_indices = indices(end:-1:end-9);
top10_words = m.keys(top10_indices);
I must warn you though, Octave may be pretty slow at this task, considering that you have thousands of files. Use it only if running time isn't that important for you.
I'm quite new to GNU Octave, so can anyone help me for 2 things:
(1) How can I filter that huge dataset in such a way that it will only contain [1x1 struct] persons?
(2) Inside that value of struct, I only want to retain combined_categories. How can I delete the others?
Basically, my end goal is to have a dataset with 2 columns only (filename and combined_categories of the filtered 1x1 structs). And if I can convert that to csv, that would be more awesome.
Regarding your first question, how to filter a struct. First step is to create a vector which decides which ones to keep and which ones to delete:
%Get the data for the relevant field
persons={test.person}
%For each field, check if the size is 1
one_person=cellfun(#numel,persons)==1
%Select those you want
test=test(one_person)
About your second question, please check the documentation for rmfield
I have a single column of text in Excel that is to be used for translating into foreign languages. The text is automatically generated from an InDesign File. I would like to clean it up for the translator by removing rows that simply contain a number ("20", 34.5" etc), or if they contain a measurement "5mm", "3.5 µm", etc. I've found many posts (see link below) on how to remove a row with specific string, but none that use search strings, such as those I typically use with GREP searches: "\d+" and "\d.\d µm"
How would I do this? I am on Mac iOS if that helps.
Note that I would need to delete the row if the cell only contains a number or a measurement, not if the number is contained within a phrase, sentence, or paragraph, etc.
https://stackoverflow.com/a/30569969
It may not be what you are looking for, but how about just sorting the column and remove the rows starting with numbers? It is a manual approach but from what I understand this translation process only happens from time to time. Am I right?
I see two possible issues in your question:
How to work with regular expressions in Excel?
How to delete rows in a loop?
Let me start with the second question: when you want to create a for-loop in order to remove items from a list, you MUST start at the end and go back to the beginning (it's a beginner's trick, but a lot of people trip over it.
About the first question: this is a very useful post about this subject, it's too large to even give a summary here.
I am using Surveymonkey for a questionnaire. Most of my data has a regular scale from 0-6, and additionally an "Other" option that people can use in case they choose to not answer the item. However, when I download the data, Surveymonkey automatically assigns a value of 0 to that not-answer category, and it appears this cant be changed.
This leads to me not knowing when a zero in my numeric dataset actually means zero or just participants choosing to not answer the question. I can only figure that out by looking at another file that includes the labels of participants answers (all answers are provided by the corresponding labels, so this datafile misses all non-labeled answers...).
This leads me to my problem: I have two excel files of same size. I would need to find a way to find certain values in one dataset (text value, scattered randomly over dataset), and replace the corresponding numeric values in the other dataset (at the same position in the dataset) with those values.
I thought it would just be possible to find all values and copy paste in the same pattern, but I cannot seem to find a way to do that. I feel like I am missing an obvious solution, but after searching for quite a while I really could not find an answer to my specific question.
I have never worked with macros or more advanced excel programming before, but have a bit of knowledge about programming in itself. I hope I explained this well, I would be very thankful for any suggestions or scripts that could help me out here!
Thank you!
Alex
I don't know how your Excel file is organised, but if it's like the legacy Condensed format, all you should need to do is to select the column corresponding to a given question (if that's what you have), and search and replace all 0 (match entire cell) with the text you want.
I have a large spreadsheet with column data like:
ABC:1:I.0
ABC:1:I.1
ABC:1:I.2
ABC:1:I.3
ABC:2:I.0
ABC:2:I.1
ABC:2:I.2
ABC:2:I.3
ABC:3:I.0
ABC:3:I.2
ABC:3:I.3
ABC:4:I.0
ABC:4:I.1
ABC:4:I.2
ABC:4:I.3
ABC:5:I.0
ABC:5:I.1
ABC:5:I.2
ABC:5:I.3
ETC.
I need to replace the above with the following:
ABC:I.Data[1].0
ABC:I.Data[1].1
ABC:I.Data[1].2
ABC:I.Data[1].3
ABC:I.Data[2].0
ABC:I.Data[2].1
ABC:I.Data[2].2
ABC:I.Data[2].3
ABC:I.Data[3].0
ABC:I.Data[3].2
ABC:I.Data[3].3
ABC:I.Data[4].0
ABC:I.Data[4].1
ABC:I.Data[4].2
ABC:I.Data[4].3
ABC:I.Data[5].0
ABC:I.Data[5].1
ABC:I.Data[5].2
ABC:I.Data[5].3
ETC.
Here is a sample of the data, most of the data follows a similar format with the exception of the naming "ABC", which can vary in size, so it might be "ABCD" and also with the exception of the letter "I", it can be "O" as well. Also, some might be missing some values such as ABC:3:I.1 which is missing from the data. I am not too familiar with excel formulas or VBA code. Does anyone know how to do this? I have no preference on which method it has to be done in as I don't mind learning some VBA code if someone provides me with a VBA solution.
I was thinking of using some sort of loop along with some conditional statements.
Thanks!
Please try:
=LEFT(F11,FIND(":",F11))&MID(F11,FIND(":",F11,6)+1,1)&".Data["&MID(F11,FIND(":",F11,2)+1,1)&"]."&RIGHT(F11,1)
copied down to suit, assuming placed in Row11 and your data is in ColumnF starting in Row11.
Curiosities:
When this A was first posted it attempted to address only the tabulated example input and output. I temporarily deleted that version while addressing that what was in the table as ABC might at times be ABCD and that what was I might at times be O.
OP has posted an answer that I edited to make no visible change but which shows as the deletion of two characters. A copy of the OP’s formula exhibited a syntax error prior to my edit.
OP suggested an edit to my answer but this was rejected by the review process. As it happens, I think the edit suggestion was incorrect.
I have edited my answer again to include these ‘curiosities’ and to match the cell reference used by the OP in his answer.
=LEFT(A1,SEARCH(":",A1)) & MID(A1, SEARCH(".",A1)-1, 2) &
"Data[" & MID(A1,SEARCH(":",A1)+1,1) & "]" & RIGHT(A1,2)
With the help of pnuts I was able to come up with my own solution:
=LEFT(F11,LEN(F11)-5)&MID(F11,LEN(F11)-2,2)&"Data["&MID(F11,LEN(F11)-4,1)&"]"&RIGHT(F11,2)
My solution works based on the fact that the length of the last six values in the string ABC:1:I:0 will always be the same in size for all the data I have, hence you see LEN(F11)-some number in my code. The only part of the string that changes in size is the first part, in this case ABC which can also be ABCDEF, etc.
If you'd like to use formulas rather than VBA, an easy option is to split the data into 4 columns, using the Text To Columns option - first split using the colon as a delimiter, then using a full-stop / period as a delimiter.
Once you have 4 columns of data (one for each block), you can use the Concatenate function to join them and add in the extra characters: =CONCATENATE(A1,":",C1,".","Data[",B1,"].",D1)
This should still work if you have extra / alternative characters (eg ABCD instead of ABC), as long as you have the same delimiters, but obviously you'd need to test to make sure.