I am running Windows 7 and have MS Office installed. Any time I download a .csv file the "file type" line in the "save as..." dialog defaults to "Microsoft Office Excel comma separated values file". Is there actually a Microsoft specific format that is distinct from "plain" .csv?
Googling the relevant terms returns various incredibly uninformative pages such as this one. Is any information lost or gained, or anything encoded differently by using this format than by just treating a file as a .csv, conforming to the general standards?
Yes, there are almost certainly differences.
From the top of my head: English Excel uses "," as a seperator. German locale uses ";" as a seperator, requiring an additional importing step if you want to import a csv with a comma seperator. This is not unique to german locales, roughly 1/4 to 1/3 of the world uses ";".
Also, there might be differences in how complicated strings are escaped (; and " in texts) which are probably different from program to program.
This is not excels fault, since the csv "format" is not really standardised and there are uncountable numbers of programs which are rolling their own csv parser, which leads to all sorts of problems because they forgot to handle corner cases.
I once read the comment that csv is the plague of data exchange formats because it is so difficult to do right. I could not agree more, I have to deal with them on a daily basis and they are extremly annoying to work with.
Open source fans will hate me for this, but I think csv is a poor choice for data exchange, even xlsx is better because it has rules which are well defined.
There are two things going on. The abbreviation (and suffix) "CSV" can mean character-separated values or it can mean comma-separated values. "Microsoft Office Excel comma separated values file" is a disambiguation, and means that you have a number of values in a record, with the field values separated by a comma.
The values themselves, in comma-separated value files, may contain commas if they are properly stropped (quoted). Usually, the stropping is putting a double quote around some or all of the field.
MS Excel also supports newlines in the middle of fields, again being properly stropped.
Related
How to change delimiter semicolon to comma of a file in google sheets or other way? I have already tried text to column feature but my file is too large around 10 MB so it doesn't work.
I have been through that path quite a few times, since business logic likes to use Excel, and software likes CSV.
Excel, and probably Google Sheets etc, are a pain to use for CSV. Especially Excel because it tries to be "friendly" for you, so it typically exports with separators determined by the Windows Locale (ex: ";" or "," ) - and excel usually shifts between quotes and no quotes.
If you HAVE the text file, and wish to IMPORT into excel, find the combination of separators that suits your language variant, using a test file. To convert the big text file, a tool like AWK is simple and awesome.
To EXPORT from Excel, I would again just go with whatever Excel wants to do, then adapt the file after using AWK or similar. Easier for me than trying to find settings for separator and quoting.
There is an article here that I find helpful.
PS: I should mention that in most of European region, CSV IS semicolon by default.
I need to work on tabular data with some people who will edit it using Excel 2010, likely set up with , as decimal delimiter and ; as list separator.
The data needs to be under version control, and it must be easy to fork/share, merge, and compare different states of the data set. This includes low-barrier ways to contribute for people outside our work group who do not have me around to help them set the system up.
What setup will allow my co-workers to easily open the file using Excel and edit cells, and commit and compare with very few clicks and maybe entering a commit message?
In particular, I require that if I open the file in Excel, do a null-change and save it again, and then do whatever is the commit step for this setup, the commit will be empty.
The data contains non-ASCII characters, in particular IPA-symbols.
I expected that I would just be able to use git with csv files. But while for ASCII data, .csv with comma as separator seems to fulfill these conditions, for more complete encodings I cannot seem to get Excel keep data the same on round-trips, not even by appearance, not to mention binary compatibility – it either loses unicode characters upon saving or does not recognize the format upon reading.
In many EU countries a comma ',' is used as the decimal separator, whereas in the US a dot '.' is used.
CSV (Comma Separated Values) files are supposed to use the comma to separate cell values. However, often a tab '\t' or other characters are used instead.
What's interesting, Excel if you save a .csv file using Microsoft Excel in a EU country using the comma as a decimal separator, the value it uses to separate cell values is not an escaped comma, but a semicolon ';'. Looking on the net it seems that, if you are in the US, Excel will save .csv files using a proper comma (I can't verify this).
I'm trying to find a way to create a csv file that can be recognized by Excel without any user action, both in the EU and the US.
Here's an example using Excel with an Italian locale
The above, saved as .csv (MS-DOS), translates to
foo;foo bar;
foo'bar;"foo""bar";
foo,bar;foo.bar;
foo:bar;"foo;bar";
foo/bar;foo\bar;
"foo
bar";foo|bar;
foo;bar;foobar
this is to make the empty line appear
It may be possible that, depending on the local "list separator", this may not be recognized correctly.
I've read that the new Excel 2013 needs sep=; to be set as the first line in order to work correctly. This is an ugly hack, but it seems to also be working for Excel 2010 (except it gets overwritten on save)...
Does the above text work for you, if you save it as a csv?
Is there a less hacky way to tell Excel which character is the cell separator, without having the user to set things up?
Thanks.
Time to head back to a time before visual anything, and grab a command from the past. It will involve you manually writing the file out with VBA, but it has the criteria you expect: Write
Open "c:\tmp\myfile.csv" for output as #1
for i=1 to 100
write #1,range("A"&i),range("B"&i),range("C"&i)
next i
close #1
You will have to do a little manual work - it doesn't translate a single quote into a double quote, but the rest is as desired:
the Write # statement inserts commas between items and quotation marks around strings as they are written to the file
Numeric data is always written using the period as the decimal separator.
Dates are written as #yyyy-mm-dd hh:mm:ss#
The format of our member numbers has changed several times over the years, such that 00008, 9538, 746, 0746, 00746, 100125, and various other permutations are valid, unique and need to be retained. Exporting from our database into the custom Excel template needed for a mass update strips the leading zeros, such that 00746 and 0746 are all truncated to 746.
Inserting the apostrophe trick, or formatting as text, does not work in our case, since the data seems to be already altered by the time we open it in Excel. Formatting as zip won't work since we have valid numbers less than five digits in length that cannot have zeros added to them. And I am not having any luck with "custom" formatting as that seems to require either adding the same number of leading zeros to a number, or adding enough zeros to every number to make them all the same length.
Any clues? I wish there was some way to set Excel to just take what it's given and leave it alone, but that does not seem to be the case! I would appreciate any suggestions or advice. Thank you all very much in advance!
UPDATE - thanks everybody for your help! Here are some more specifics. We are using a 3rd party membership management app -- we cannot access the database directly, we need to use their "query builder" tool to get the data we want to mass update. Then we export using their "template" format, which is called XLSX but there must be something going on behind the scenes, because if we try to import a regular old Excel, we get an error. Only their template works.
The data is formatted okay in the database, because all of the numbers show correctly in the web-based management tool. Also, if I export to CSV, save it as a .txt and import it into Excel, the numbers show fine.
What I have done is similar to ooo's explanation below -- I exported the template with the incorrect numbers, then exported as CSV/txt, and copied / pasted THOSE numbers into the template and re-imported. I did not get an error, which is something I guess, but I will not be able to find out if it was successful until after midnight! :-(
Assuming the data is not corrupt in the database, then try and export from the database to a csv or text file.
The following can then be done to ensure the import is formatted correctly
Text file with comma delimiter:
In Excel Data/From text and selected Delimited, then next
In step 3 of the import wizard. For each column/field you want as text, highlight the column and select Text
The data should then be placed as text and retain leading zeros.
Again, all of this assumes the database contains non-corrupt data and you are able to export a simple text or csv file. It also assumes you have Excel 2010 but it can be done with minor variation across all versions.
Hopefully, #ooo's answer works for you. I'm providing another answer mainly for informational purposes, and don't feel like dealing with the constraints on comments.
One thing to understand is that Excel is very aggressive about treating "numeric-looking" data as actual numbers. If you were to open the CSV by double-clicking and letting Excel do its thing (rather than using ooo's careful procedure), those numbers would still have come up as numbers (no leading zeros). As you've found, one way to counteract this is to append clearly nonnumeric characters onto your data (before Excel gets its grubby hands on it), to really convince Excel that what it's dealing with is text.
Now, if the thing that uploads to their software is a file ending in .xlsx, then most likely it is the current Excel format (a compressed XML document, used by Excel 2007 and later). I suppose by "regular old Excel" you mean .xls (which still works with the newer Excels in "compatibility mode").
So in case what you've tried so far doesn't work, there are still avenues to explore before resorting to appending characters to the end of your data. (I'll update this answer as needed.)
You're on the right track with the apostrophe.
You'll need to store your numbers in excel as text at the time they are added to the file.
What are you using to create the original excel file / export from database?
This will likely be where your focus needs to be regarding your export.
For example one approach is that you could potentially modify the database export to include the ' symbol prefix before the numbers so that excel will know to display them as text.
I use the formula =text(cell,"# of zeros of the field") to add preceding zeros.
Example, Cell C2 has 12345 and I need it to be 10 characters long. I would put =text(c2,"0000000000").
The result will be 0000012345.
I want to generate a CSV file for user to use Excel to open it.
If I want to escape the comma in values, I can write it as "640,480".
If I want to keep the leading zeros, I can use ="001234".
But if I want to keep both comma and leading zeros in the value, writing as ="001,002" will be splitted as two columns. It seems no solution to express the correct data.
Is there any way to express 001, 002 in CSV for Excel?
Kent Fredric's answer contains the solution:
"=""001,002"""
(I'm bothering to post this as a separate answer because it's not clear from Kent's answer that it is a valid Excel solution.)
Put a prefix String on your data:
"N001,002","N002,003"
( As long as that prefix is not an E )
That notation ( In OpenOffice at least) above parses as a total of 2 columns with the N001,002 bytes correctly stored.
CSV Specification says that , is permitted inside quote strings.
Also, A warning from experience: make sure you do this with phone numbers too. Excel will otherwise interpret phone numbers as a floating point number and save them in scientific notation :/ , and 1.800E10 is not a really good phone number.
In OpenOffice, this RawCSV chunk also decodes as expected:
"=""001,002""","=""002,004"""
ie:
$rawdata = '001,002';
$equation = "=\"$rawdata\"";
$escaped = str_replace('"','""',$equation);
$csv_chunk = "\"$escaped\"" ;
Do
"""001,002"""
I found this out by typing "001,002" and then doing save-as CSV in Excel. If this isn't exactly what you want (you don't want quotes), this might be a good way for you to find what you want.
Another option might be use tab-delimited text, if this is an option for you.
A reader of my blog found a solution, ="001" & CHAR(44) & "002", it seems workable on my machine!
Pretty old thread but why don't you just add whitespace after your value. It will be then treated as string and no leading zeros will be stripped.
"001,002"." "
Since no-one mentioned it already, figured it was worth mentioning it in this old post.
If you add a horizontal tab character \t before the number, then MS Excel will also show the leading zero's. And the tab character doesn't show in the excel sheet. Even if it's surrounded by double-quotes. (F.e. \"\t001,002\")
It also looks nicer in Notepad++, compared to putting a \0 aka NULL before such number.
Looking more at the Excel spreadsheet it looks what you want can't be done using CSV.
This site http://office.microsoft.com/en-us/excel/HP052002731033.aspx says "If cells display formulas instead of formula values, the formulas are converted as text. All formatting, graphics, objects, and other worksheet contents are lost. The euro symbol will be converted to a question mark."
However, you can change how you load it to get the result you want. See this web page:
Microsoft import a text file.
The key thing is to choose Import External Data-Import Data-Text Files, go Next, Next, and then tick "Text" under column data format. This will prevent it being interpreted as a number, and losing formatting.
I was fiddling around with CSV to Excel (i use PHP to create the CSV, but i guess this solution works for any language. When you spot that a leading characters (such as + , - or 0 are disappearing, create the CSV with chr(13) as a prefix. This is a non printable character and it works wonders for my Excel Office 2010 version. I tried other non printable characters, but with no luck.
so i use Chirp Internet solution but tweaked with my prefix:
if (preg_match("/^0/", $str) || preg_match("/^\+?\d{8,}$/", $str) || preg_match("/^\d{4}.\d{1,2}.\d{1,2}/", $str)) {
$str = chr(13)."$str";
}
If you are using "Content-Disposition" and exporting from asp to excel using HTML tags,then you have to add "style='mso-number-format:\#;'" to that tag and making it to accept only Text values ,thereby leading zeroes omission will be avoided,If Forward slash"\" is accepted use double forward slash "\"
All the suggested answers don't seem to work for me right now ("=""blahblah""" and others) in all current Excel versions or Numbers app on OS X.
The only solution I found to be working by fiddling around is to add an escaped null character at the beginning of the string (which is \0 in PHP or C based languages). Everything ends up treated as is without being calculated or processed by the software when opening the calc sheet.
echo "\0" . $data;
Excel uses a default formatting for CSV columns depending on the content. So if you have 001 in a csv, excel will automatically turn it to 1.
The only way to keep the leading zeros in excel from a csv file is by changing the extension of the csv file to .txt, then just open excel, click on open, select the txt file, and you'll see the Text Import Wizard. Select your csv format (separated by commas), then just make sure you select "Text" as the format.
And that's it, now you can export that previous csv data to any other while keeping the leading zeros.
This is straightforward using Excel's Power Query functionality that allows you to perform step-by-step transformations.
Original File:
Add a Custom Column: