I have a table with hebrew characters.
I run a select command and get ??? instead of the hebrew result.
I connect using python3 on linux redhat8 and cx_Oracle using enconding='UTF-8'.
the string in the table is hebrew, I can see it correctly from pl/sql on windows.
how can I fix this
Thanks you
Tsvi
You probably have a discrepancy between the character set the database thinks the data is in and the actual character set of the data. This is covered in the documentation.
Check the database character set using this SQL:
SELECT value AS db_charset
FROM nls_database_parameters
WHERE parameter = 'NLS_CHARACTERSET'
Check the client character set using this SQL:
select distinct client_charset
from v$session_connect_info
where sid = sys_context('userenv', 'SID')
Check the client character set both where it is working correctly (PL/SQL on Windows) and not working correctly (Python on Linux). If this doesn't help you figure it out, post the results in your question and I'll adjust my answer accordingly.
Related
My base scenario is that I want to make an excel report with data from a PostgreSQL DB.
I get them via ODBC, making a simple linked table with PowerQuery.
For DSN I choose (None), then I write the connectio string and the SQL statement. Generally it works fine, but with one column, it doesn't. I recive the following error message:
ODBC: ERROR [22P05] ERROR: character with byte sequence 0xc2 0xb2 in encoding "UTF8" has no equivalent in encoding "WIN1250";Error while executing the query
So that is clear, the source is in UTF-8 with characters that are not compatible with Win1250.
What I am looking for is a general solution either on DB or excel site.
The used SQL statement is a simple SELECT * FROM [view], so I can use any replacement or converting or anything just to be able to hanle it with transformations on the column. I can replace the view with function if that is better.
But it would be better, if you can suggest an excel site solution.
With it there is some criteria. That scenario, when "I get the data first in text, then I convert it to Win1250, then import to excel" wont't fit, and I need something which connects to the excel file itself, so if I move it to an other pc, it need to work too without any more modification.
Thanks for all the help!
Environment:
Debian 11 with pl_PL locale(ISO-8859-2)
Postgresql 13 database created with ISO 8859-2 encoding
Plpython3u(Python 3.9.2)
A simple example of the problem:
CREATE OR REPLACE FUNCTION public.test()
RETURNS TEXT
LANGUAGE plpython3u
AS $function$
tmp = plpy.execute("SELECT field FROM table WHERE filter_pointing_at_a_single_row;")[0]['field']
plpy.execute("UPDATE table SET field='"+tmp+"' WHERE filter_pointing_at_a_single_row;")
$function$;
When the content is 'łódź', running this once results in it being changed to 'Ĺ‚ĂłdĹş'.
We have tens if not hundreds of functions that perform operations like that. The original solution used python2 with default encoding changed to iso8859-2, but it's time to upgrade and such trick won't work in python3.
Other observations:
CREATE OR REPLACE FUNCTION public.test()
RETURNS TEXT
LANGUAGE plpython3u
AS $function$
tmp = plpy.execute("SELECT field FROM table WHERE filter_pointing_at_a_single_row;")[0]['field']
ret = plpy.execute("SELECT '"+tmp+"' AS \"A\" ;")
return ret[0]['A']
$function$;
In psql with correct client-encoding as well as in DBeaver which forces utf-8 it returns 'Ĺ‚ĂłdĹş'.
Meanwhile:
CREATE OR REPLACE FUNCTION public.test()
RETURNS TEXT
LANGUAGE plpython3u
AS $function$
tmp = plpy.execute("SELECT field FROM table WHERE filter_pointing_at_a_single_row;")[0]['field']
return tmp
$function$;
Returns 'łódź' both in DBeaver and psql.
Edit:
I didn't mention what I need. We're looking for the best solution. We're considering moving the database to UTF-8, or rewriting all the functions that need it if we find other solution. There's also hope that some smart fix exists that would minimise the amount of work needed to solve this.
My table is sqlite3 is created with the following:-
'CREATE TABLE IF NOT EXISTS gig_program ( gig_program_id VARCHAR(20) PRIMARY KEY );'
When I try to insert data into the table using python 3.8 with the following:-
sql = 'INSERT INTO gig_program ( gig_program_id ) VALUES ( "20200524120727" );'
cur.execute(sql)
the following exception was thrown:-
near "gig_program": syntax error
When I cut and past the insert command to the sqlite3 console, it works.
I have also tried using another editor for the program (thinking that there may be hidden characters) but the result is the same.
I would appreciate help. I have used similar methods in other parts of the program to insert data and they work without issue.
Thank you for looking into my questions.
I found that it was actually my mistake. The exception was actually for a second sql statement which I missed out the "FROM" word.
Thank you everyone for your time.
Hope everyone is doing well.
Hi I am trying to upload data to the Heidi SQL table, but it returned "SQL Error (1366): Incorrect string value: '\xE3\x82\xA8\xE3\x83\xBC...'".
This issue is prompted by this string - "エーペックスレジェンズ" , and the source data file has a number of special characters. Want to know if there's a way to override this, so that all forms of character could be uploaded?
My default setting is utf8 and I have also tried utf8mb4, but neither of them would work.
That happens when you select the wrong file encoding in HeidiSQL's open-file dialog:
Never select "Auto-detect" - I wrote that auto-detection, and I can tell you it often detects the wrong encoding. Use the right encoding instead, which is mostly utf-8 nowadays.
I am trying to use excel to update a list of part numbers in my database:
UPDATE
stock s
SET
s.STC_AUTO_KEY = 2
WHERE s.WHS_AUTO_KEY = 2 AND
EXISTS(
SELECT
p.PNM_AUTO_KEY
FROM
PARTS_MASTER p
WHERE
s.PNM_AUTO_KEY=p.PNM_AUTO_KEY AND p.PN='102550');
UPDATE
stock s
SET
s.STC_AUTO_KEY = 2
WHERE s.WHS_AUTO_KEY = 2 AND EXISTS(
SELECT
p.PNM_AUTO_KEY
FROM
PARTS_MASTER p
WHERE
s.PNM_AUTO_KEY=p.PNM_AUTO_KEY AND p.PN='204-060-444-003');
The statements run without semicolons, but when I try to run more then one at once and use semicolons I get the error:
SQL Error [911] [22019]: ORA-00911: invalid character
java.sql.SQLSyntaxErrorException: ORA-00911: invalid character
so... it looks like I don't know how run run more then one basic statement at once.
I am using DBeaver to interact with a Oracle database.
Thanks guys, sorry if this a no-brainer.
Try adding a blank line between each update statement if possible. You can do this easily with a text editor that supports regular expressions, just replace ';\n' with ';\n\n'