I have two related problems. I'm working on Arabic dataset using Excel. I think that Excel somehow reads the contents as ؟؟؟؟؟ , because when I tried to replace this character '؟' with this '?' it replaces the whole text in the sheet. But when I replace or search for another letter it works.
Second, I'm trying to edit the sheet using python, but I'm unable to write Arabic letters (I'm using jGRASP). For example when I write the letter 'ل' it appears as 0644, and when I run the code this message appears : "ُError encoding text. Unable to encode text using charset windows-1252 ".
0644 is the character code of the character in hex. jGRASP displays that when the font does not contain the character. You can use "Settings" > "Font" in jGRASP to choose a CSD font that contains the characters you need. Finding one that has those characters and also works well as a coding font might not be possible, so you may need to switch between two fonts.
jGRASP uses the system character encoding for loading and saving files by default. Windows-1252 is an 8-bit encoding used on English language Windows systems. You can use "File" > "Save As" to save the file with the same name but a different encoding (charset). Once you do that, jGRASP will remember it (per file) and you can load and save normally. Alternately, you can use "Settings" > "CSD Windows Settings" > "Workspace" and change the "Default Charset" setting to make the default something other than the system default.
saw some answers on this but not really useful
I do some coding with python using blender 2.6
and on occasion I copy text from internet page or from Libre office text document
so there might be some control characters UTF-8 copied to the text editor for python in blender
problem is I cannot detect where UTF-8 characters are located in blender text editor or even using outside text editor like notepad 2 or notepad ++
so my question is there a simple way to detect and remove these UNTF-8 characters ?
I mean on window 10 in blender using some python commands or using external text editor!
I need something quick or a simple trick here if possible!
sorry it is "utf-8"
in blender at least the error is given as "utf-8"
and very annoying
so when I run a script in bl with some unknown characters it gives an error
the problem is how can I find where it is and remove it !
The error in blender is given but does not always shows where it is located in the python script!
even if i had to use a text editor that can locate and then help to remove these characters somehow if possible at all
I have a CSV file with special accents and save it in Notepad by selecting UTF-8 encoding. When I read the file using Java, it reads the BOM characters too.
So I want to save this file in UTF-8 format without appending a BOM initially in Notepad.
Otherwise, is there a built-in class in Java that eliminates the BOM characters that present at beginning, when reading the contents in a file?
Use Notepad++ - it is free and much better than Notepad. It will help to save text without a BOM using Encoding → Encode in UTF-8 without BOM: Notepad++ v6 and olders:
Notepad++ v7+:
When I encountered this problem in Java, I didn't find any library to parse these first three bytes (BOM). So my advice:
Use PushbackInputStream(in, 3).
Read the first three bytes
If it's not BOM (EF BB BF), push them back
Process the stream as UTF-8
I just learned from this Stack Overflow post, as #martin-geisler points out, that you can save files without the BOM in Windows Notepad, by selecting ANSI as the encoding.
I'm assuming that for more advanced uses this won't work because the resulting file is probably not the end encoding wished, but actually ANSI; but I tested and confirmed this works to save a very small .php script without BOM using only Notepad.
I learned the long, hard way that Windows' Notepad is not a true editor, although I'd like to point out for others that, despite this, it is misleadingly called up when you type "editor" on newer Windows machines, at least on one of mine.
I am currently using Emacs and other editors to solve this problem.
Use Notepad++ instead. See my personal blog post on it. From within Notepad++, choose the "Encoding" menu, then "Encode in UTF-8 without BOM".
Notepad on Windows 10 version 1903 (May 2019 update) and later versions supports saving to UTF-8 without a BOM. In fact, UTF-8 is the default file format now.
Reference: Windows 10 Notepad is Getting Better UTF-8 Encoding Support
The answer is: Not at all. Notepad can't do that.
In Java you can just skip the first byte in your InputStream and be done.
You might want to try out Notepad2 or Notepad++. Those Notepad replacements have the option for you to choose whether to output BOM.
As for a Java solution, as far as I know, Java does not understand the standard UTF-8. I googled and found Java's UTF-8 and Unicode writing is broken - Use this fix that might be the solution.
We're using the utility BOMStripperInputStream.java to strip the BOM from our input if present.
I'm looking for a command line tool for Linux that will allow me to convert UTF-8 plain text files to various formats. My problem is that I'm working on a secure company-specific flavour of Linux, so the tool can't rely on other packages, such as Open Office, being present. Does anyone know of such a tool?
Gnu a2ps allows you convert from anything to postscript (designed for printing). Not exactly what you want but if you have utilites to display postscript files, you can convert them into pdf.
Another option is Gnu enscript which "converts text to Postscript, HTML or RTF with syntax highlighting". I'm not sure if it supports UTF-8.
Conversion into doc will be harder since it's a closed format. But I have in the past cheated by creating an HTML file with inline css and then renaming it to .doc. Worked back in the early 2000s. DOn't know about now.
Guys,
I am looking for a text editor or method to deal with the UTF8 encoded *.properties file created by the Netbeans (6.x and 7.0.x). Trust me I have tried Notepad++, PsPad, gVim, jedit, etc without luck. Or I have just missed somehing?
The items is in the form of followings:
update.text=\u4FEE\u6539
The reason for this is I am trying to use script to modify some property items during the build process.
At the moment only Netbeans itself can read and display the content correctly. Any ideas?
Thanks.
David
I use WinVi, which supports UTF-8, UTF-16, and hex/binary editing.