My file got the format Unix (LF) with UTF-8-BOM. It got quotation marks on each column:
Now i want to delete the last column. If i save the file with excel as .csv it is formatted in Windows (CR LF) in ANSI. It loses the quotation marks:
I wrote a short C# code to remove the last column manually with StreamReader/StreamWriter, but the output is also formatted in Windows (CR LF) UTF-8. Curiously it additionally loses some lines..
string newFilename = tboxFile.Text.Split(new string[] { ".csv" }, StringSplitOptions.None)[0] + "_replaced123.csv";
StreamReader streamReader = new StreamReader(tboxFile.Text);
StreamWriter streamWriter = new StreamWriter(newFilename);
string line = streamReader.ReadLine();
streamWriter.WriteLine(line.Split(new string[] { ";\"#timestamp\"" }, StringSplitOptions.None)[0]);
while ((line = streamReader.ReadLine()) != null)
streamWriter.WriteLine(line.Split(new string[] { ";\"2017-11-" }, StringSplitOptions.None)[0]);
So.. Is there a special excel-trick to save the output in same format (Unix LF UTF-8-BOM) or how can i help me?
Like my comment above says, the spreadsheet tool from LibreOffice offers to adjust filter settings when you do Save As on a csv file.
To get there check the box "Edit filter settings" in the Save As dialog. After pressing the save button a "Textexport" dialog will show up where you can select the text codec among other things.
Related
I have an Excel (xlsx) file that has 3 columns of data that is set to replace said data in a Photoshop file (PSD), to do so I need to load it into Photoshop in a txt format, encoded to ANSI, so that Photoshop can read that file, and export it a bunch of times each time with the next row's properties.
However my Excel file has some Hebrew text, that is lost when encoding to ANSI, I tried other encodings but Photoshop doesn't accept them, how can I still feed Photoshop with the Hebrew data? (It's a lot of photos so I can't do it manually one by one)
This works for me: I've got a simple text file, with some Hebrew text on it.
And from Photoshop:
var myfile = "D:\\temp\\hebrew.txt"; // change this
var text = read_it(myfile);
alert(text);
// השועל החום המהיר קופץ מעל הכלב העצלן.
// function READ IT (filename with path) :returns string
// ----------------------------------------------------------------
function read_it(afilepath)
{
var theFile = new File(afilepath);
//read in file
var words = ""; // text collection string
var theTextFile = new File(theFile);
theTextFile.open('r');
while(!theTextFile.eof)
{
var line = theTextFile.readln();
if (line != null && line.length >0)
{
words += line + "\n";
}
}
theTextFile.close();
// return string
return words;
}
I'm trying to convert XLS to CSV using job in AX2012. I have some non-ASCII characters in my XLS and I need to find out how can I set SysExcelWorkbook.saveAs method to use specific encoding (eg. UTF-8).
static void ExcelToCsv(Args _args)
{
SysExcelApplication application;
SysExcelWorkbooks workbooks;
SysExcelWorkbook workbook;
FileName xlsFile, csvFile;
;
application = SysExcelApplication::construct();
application.displayAlerts(false);
workbooks = application.workbooks();
xlsFile = #"C:\test.xlsx";
csvFile = #"C:\result.csv";
workbooks.open(xlsFile);
workbook = workbooks.item(1);
workbook.saveAs(csvFile, 6);
// workbook.saveAs(resFile, 22);
// workbook.saveAs(resFile, 23);
// workbook.saveAs(resFile, 24);
application.quit();
}
The code above generates CSV, but all non-ASCII characters are not displaying property when opening in text editor. I expect that I will be able to choose encoding for my CSV file programmatically or use source (XSL) encoding. Is there a way to achieve this with X++?
I don't think you can do this without some workarounds as it appears to be an Excel limitation. It's do-able though if you really need it.
It uses the Excel COM object to do the work, and you can see the reference here, where I can't find any options to specify encoding:
https://learn.microsoft.com/en-us/office/vba/api/excel.workbook.saveas
Here is the same issue, albeit in Powershell instead of X++ with solution (I think) being to export to UnicodeText instead of CSV, then replacing \t with , in the output file.
It looks like you could output to UnicodeText by making the below change to your code, then you could just use some other string-replace to update the final file.
#Excel
// workbook.saveAs(csvFile, 6); // 6 == #xlCSV
workbook.saveAs(csvFile, #xlUnicodeText);
I'm not sure if this truly fixes your encoding issue without testing. I'd also want to double-check how single/double quotes are handled.
I have encountered something really weird. When exporting to CSV my top line shows the quotation marks yet the lines below down.
I use UTF8 encoding and manually add the double quotation marks to the value so that it is encased with quotation marks.
the code being used is
Dim fs As New IO.FileStream(GenericValueEditorExportFilename.Value, IO.FileMode.Create)
Dim writer As New IO.StreamWriter(fs, Encoding.UTF8)
fs.Write(Encoding.UTF8.GetPreamble(), 0, Encoding.UTF8.GetPreamble().Length)
....
....
....
While reader.Read
If reader("TargetLanguageID") = targetLanguageID Then
writer.WriteLine(Encode(reader("SourcePhrase")) & ", " & Encode(reader("TargetPhrase")))
End If
....
....
....
Friend Shared Function Encode(ByVal value As String) As String
Return ControlChars.Quote & value.Replace("""", """""") & ControlChars.Quote
End Function
the result when displayed in excel is shown as (https://ibb.co/ntMYdw)
when i open the file in Notepad++ the text is shown as below. But each line is displayed differently. Why is it that the 1st row displays them and the 2nd does not. Notepad++ result is displayed as (https://ibb.co/fMkWWG)
Excel is treating the first line as headers.
https://stackoverflow.com/a/24923167/2319909
So the issue was being caused by the BOM that was created to manually set the encoding for the file as a start writing to the file.
fs.Write(Encoding.UTF8.GetPreamble(), 0, Encoding.UTF8.GetPreamble().Length)
Removing this resolves by issue and the file remains in the desired UTF8 encoding as it is set on the stream writer. so there is no need to add the BOM to set the encoding.
Something like this should work for you.
Dim str As New StringBuilder
For Each dr As DataRow In Me.NorthwindDataSet.Customers
For Each field As Object In dr.ItemArray
str.Append(field.ToString & ",")
Next
str.Replace(",", vbNewLine, str.Length - 1, 1)
Next
Try
My.Computer.FileSystem.WriteAllText("C:\temp\testcsv.csv", str.ToString, False)
Catch ex As Exception
MessageBox.Show("Write Error")
End Try
I would like to import product descriptions that need to be logically broken according by things like description, dimensions, finishes etc. How can I insert a line break so that when I import the file they will show up?
This question was answered well at Can you encode CR/LF in into CSV files?.
Consider also reverse engineering multiple lines in Excel. To embed a newline in an Excel cell, press Alt+Enter. Then save the file as a .csv. You'll see that the double-quotes start on one line and each new line in the file is considered an embedded newline in the cell.
I struggled with this as well but heres the solution. If you add " before and at the end of the csv string you are trying to display, it will consolidate them into 1 cell while honoring new line.
csvString += "\""+"Date Generated: \n" ;
csvString += "Doctor: " + "\n"+"\"" + "\n";
I have the same issue, when I try to export the content of email to csv and still keep it break line when importing to excel.
I export the conent as this: ="Line 1"&CHAR(10)&"Line 2"
When I import it to excel(google), excel understand it as string. It still not break new line.
We need to trigger excel to treat it as formula by:
Format -> Number | Scientific.
This is not the good way but it resolve my issue.
supposing you have a text variable containing:
const text = 'wonderful text with \n newline'
the newline in the csv file is correctly interpreted having enclosed the string with double quotes and spaces
'" ' + text + ' "'
On Excel for Mac 2011, the newline had to be a \r instead of an \n
So
"\"first line\rsecond line\""
would show up as a cell with 2 lines
I was concatenating the variable and adding multiple items in same row. so below code work for me. "\n" new line code is mandatory to add first and last of each line if you will add it on last only it will append last 1-2 character to new lines.
$itemCode = '';
foreach($returnData['repairdetail'] as $checkkey=>$repairDetailData){
if($checkkey >0){
$itemCode .= "\n".trim(#$repairDetailData['ItemMaster']->Item_Code)."\n";
}else{
$itemCode .= "\n".trim(#$repairDetailData['ItemMaster']->Item_Code)."\n";
}
$repairDetaile[]= array(
$itemCode,
)
}
// pass all array to here
foreach ($repairDetaile as $csvData) {
fputcsv($csv_file,$csvData,',','"');
}
fclose($csv_file);
I converted a pandas DataFrame to a csv string using DataFrame.to_csv() and then I looked at the results. It included \r\n as the end of line character(s). I suggest inserting these into your csv string as your row separation.
Depending on the tools used to generate the csv string you may need escape the \ character (\r\n).
I'm trying to write some double values to a text file the user creates via a SaveFileDialog, but everytime I do a streamWriterVariable->Write(someDoubleVariable), I instead see some kind of weird ASCII character in the text file where the double should be (music note, |, copyright symbol, etc). I'm opening the file with notepad if it's that of any significance. A basic outline of my code:
SaveFileDialog^ saveFileDialog1 = gcnew SaveFileDialog;
saveFileDialog1->Filter = "txt files (*.txt)|*.txt|All files (*.*)|*.*";
saveFileDialog1->Title = "Save File Here";
saveFileDialog1->RestoreDirectory = true;
if (saveFileDialog1->ShowDialog() == System::Windows::Forms::DialogResult::OK )
{
FileInfo ^fleTest = gcnew FileInfo(saveFileDialog1->FileName);
StreamWriter ^sWriter = fleTest->CreateText();
sWriter->AutoFlush = true;
double test = 5.635; //Some arbitrary double I made up for test purposes
sWriter->Write(test);
sWriter->Flush();
sWriter->Close();
}
Thanks for your help!
Have you tried to set the encoding explicitly?
StreamWriter^ sWriter = gcnew StreamWriter(saveFileDialog1->FileName, false, System::Text::Encoding::ASCII);
The code you've provided does exactly what you ask it to, that is to write a double to the file in the internal computer format. What you most likely want it to write out the textual representation of the double.
In other words you should try sWriter->Write(test.ToString()) or some variation over this, to get the textual version of your double. This also applies to bool and most other variable representation.