Exporting CSV with Diacritics causes strange characters to appear in output - linux

Question

I export a csv in Scala/Spray and it works nice on my Windows machine but fails on Linux machine.
The response from both OS are identical:
Access-Control-Allow-Credentials:true
Access-Control-Allow-Headers:X-Requested-With, Cache-Control, Pragma, Origin, Authorization, Content-Type, Auth-Token
Access-Control-Allow-Methods:GET, POST, DELETE, OPTIONS, PUT
Access-Control-Allow-Origin:*
Access-Control-Expose-Headers:Auth-Token
Content-Disposition:attachment; filename=Enter report title.csv
Content-Length:229
Content-Type:text/csv; charset=ISO-8859-1
Date:Fri, 07 Feb 2014 22:17:40 GMT
Server:spray-can/1.2.0
I am wondering why the OS can make a difference?
When exporting from linux after jar is deployed, diacritics are replaced with strange chars.
For instance this Café macchiato
is fine when exporting from Windows but it appears like Café macchiato
when exporting from Linux.

Answer 1

To help Excel recognize character encoding you can add a BOM to the beginning of the file. For example:
def prepareBomOutputStream(outputFile: String) = {
val os = new FileOutputStream(outputFile)
os.write(239)
os.write(187)
os.write(191)
os
}
You can also check if in both cases you get exact same encoding and not a subset of the encoding. For instance on Windows you might get ISO-8859-15 instead. You can most likely set encoding explicitly in your CSV export code/library. To check encoding on Linux you can use:
$ file -ib /tmp/test.csv
text/plain; charset=utf-8
or even something like hexdump.

Answer 2

Please never ever use Excel for text-oriented files. It screws up stuff. Use an editor like vim or Notepad++ where you can inspect the bytes and actually see whether your stuff is correct or not.