I am attempting to connect to a firebird database running in windows from a linux client, and when trying to attache the database I get the following error:
bad parameters on attach or create database, CHARACTER SET UTF8 is not defined
I have googled and searched the answers here but can't seem to find a solution.
Any suggestions on how this can be overcome from the client side, or does it require the databse to be rebuilt with UTF8 support ?
Client-side I am using node-js with the node-firebird module, server side engine version is 2.5, ODS Version is 11.2
function dbConnect(cb){
fb.attach({
host: '192.168.42.233',
database: 'gi',
user: 'SYSDBA',
password: 'xxxxx'
}, function(err, db){
if (err) console.log(err.message);
else cb(db);
})
}
Harriv's comment prompted me to test my idea that this might be caused by a missing entry in RDB$CHARACTER_SETS. If I manually delete UTF8 from this system table I get the same error when I try to connect with UTF8:
SQL Message : -924
Connection error
Engine Code : 335544325
Engine Message :
bad parameters on attach or create database
CHARACTER SET UTF8 is not defined
The solution is to backup the database and restore it again. That will recreate the RDB$CHARACTER_SETS system table to include UTF8 again.
Note that this will only solve your problem if UTF8 is missing from RDB$CHARACTER_SETS. If it does, you should ask yourself why it was missing in the first place. Maybe a DBA or other developer deleted entries from RDB$CHARACTER_SETS, if so it is probably a good idea to find out why that was done.
For example: Maybe the database uses NONE as the default characterset (and for every column), and this was a way to ensure people only connect with the 'right' characterset by deleting all other options (a normal Firebird 2.5 database contains 52 entries in RDB$CHARACTER_SETS). If that is the case, make sure you fix this issue before connecting with UTF8, otherwise you will probably experience some form of data corruption (on read or on write) by incorrect transliteration.
The fix for this would be to create a new database with the right default characterset (and for the columns), and pump the data from the old to the new (making sure the read and write is done with the appropriate character set). After that Firebird can take care of transliterating between the database (or column) character set and the connection character set (with the exception of applications connecting using NONE as the connection character set).
I added to options encoding key 'NONE':
options.encoding = 'NONE'
Related
I use jooq to generate objects against a local database, but when running "for real" later in production the actual databases will have different names. To remedy this I use the <outputSchemaToDefault>true</outputSchemaToDefault> config option (maven).
At the same time, we have multiple databases (schemas), and are using a connection pool to the server like "jdbc:mysql://localhost:3306/" (without specifying a database here).
How do I tell jooq which database to use when running queries?
I have tried all config I can think of:
new Settings()
.withRenderSchema(true) // true/false seems to make no difference.
.withRenderCatalog(true) // true/false seems to make no difference.
.withRenderMapping(new RenderMapping()
.withDefaultSchema("my_database") // Seems to have no effect.
// The above 3 configs always give me an error saying "no database selected".
// Adding this gives me 'my_database.my_table' does not exist - while it actually does.
.withSchemata(new MappedSchema()
.withInputExpression(Pattern.compile(".*"))
.withOutput("my_database")
));
I have also tried using a database/schema name, as in not configuring outputSchemaToDefault. But then, adding the MappedSchema code above, but that gives me errors with "'my_databasemy_database.my_table' does not exist", which is correct. I have no clue why that code gives me the database/schema name twice?
Edit:
When jooq tells me that the db.table does not exist, if I put a break point in a good place and get the sql from jooq and run exactly that against my database it does work. But jooq fails to run it.
Also, I'm using version 3.15.3 of jooq.
I solved it. Instead of using .withInputExpression(Pattern.compile(".*")), it seems to work with .withInput("").
I'm still not sure why it works, or if this is the "correct" way of solving it. But at least it is a way forward.
No clue why using the pattern, I got the name twice though. But that one I'll leave alone.
I want to do a simple thing: get the database names on a RavenDB server. Looks straightforward according to the docs (https://ravendb.net/docs/article-page/4.1/csharp/client-api/operations/server-wide/get-database-names), however I'm facing a chicken-and-egg problem.
The problem comes because I want to get the database names without knowing them in advance. The code in the docs works great, but requires to have an active connection to a DocumentStore. And to get an active connection to a DocumentStore, is mandatory to select a valid database. Otherwise I can't execute the GetDatabaseNamesOperation.
That makes me think that I'm missing something. Is there any way to get the database names without having to know at least one of them?
The database isn't mandatory to open a store. Following code works with no problems:
using (var store = new DocumentStore
{
Urls = new[] { "http://live-test.ravendb.net" }
})
{
store.Initialize();
var dbs = store.Maintenance.Server.Send(new GetDatabaseNamesOperation(0, 25));
}
We send GetDatabaseNamesOperation to the ServerStore, which is common for all databases and holds common data (like database names).
I am new to CouchDB / PouchDB and until now I somehow could manage the start of it all. I am using the couchdb-python library to send initial values to my CouchDB before I start the development of the actual application. Here I have one database with templates of the data I want to include and the actual database of all the data I will use in the application.
couch = couchdb.Server()
templates = couch['templates']
couch.delete('data')
data = couch.create('data')
In Python I have a loop in which I send one value after another to CouchDB:
value = templates['Template01']
value.update({ '_id' : 'Some ID' })
value.update({'Other Attribute': 'Some Value'})
...
data.save(value)
It was working fine the whole time, I needed to run this several times as my data had to be adjusted. After I was satisfied with the results I started to create my application in Javascript. Now I synced PouchDB with the data database and it was also working. However, I found out that I needed to change something in the Python code, so I ran the first python script again, but now I get this error:
couchdb.http.ResourceConflict: (u'conflict', u'Document update conflict.')
I tried to destroy() the pouchDB database data and delete the CouchDB database as well. But I still get this error at this part of the code:
data.save(value)
What I also don't understand is, that a few values are actually passed to the database before this error comes. So some values are saved() into the db.
I read it has something to do with the _rev values of the documents, but I cannot get an answer. Hope someone can help here.
I have been using phantomjs for doing some heavy lifting for me in a server side dom environment. Till now I have been putting by data structures in-memory (i.e. doing nothing special with them) and everything was fine.
But recently under some use cases i started running into following problems:
memory usage becoming too high making swap to kick in and seriously effecting my performance.
not being able to resume from the last save point since in-memory data structures are not persistent (obviously)
This forced me to look for a database solution to be used on phantom but again I am running into issues while deciding on a solution:
I don't want my performance to get too effected.
it has to be persistent and queryable
how do i even connect to a database from inside phantom script.
Can anyone guide me to a satisfactory solution?
Note: I have almost decided on sqlite but connecting to it from phantom is still an issue. Nodejs provides sqlite3 node module, i am trying to browserify it for phantom.
Note Note: Browserify didn't worked! Back to ground zero!! :-(
Thanx in advance!
Phantomjs' filesystem API allows you to read and write binary files with:
buf = fs.read(FILENAME, 'b') and
fs.write(FILENAME, buf, 'b')
sql.js (https://github.com/kripken/sql.js/) gives you a javascript SQLite
implementation you can run in phantomjs.
Combine the 2 and you have a fast, persistent, queryable SQL database.
Example walkthrough
Get javascript SQLite implementation (saving to /tmp/sql.js)
$ wget https://raw.githubusercontent.com/kripken/sql.js/master/js/sql.js -O /tmp/sql.js
Create a test SQLite database using the command-line sqlite3 app (showing it is persistent and external to your phantomjs application).
sqlite3 /tmp/eg.db
sqlite> CREATE TABLE IF NOT EXISTS test(id INTEGER PRIMARY KEY AUTOINCREMENT, created INTEGER NOT NULL DEFAULT CURRENT_TIMESTAMP);
sqlite> .quit
Save this test phantomjs script to add entries to the test database and verify behaviour.
$ cat /tmp/eg.js
var fs = require('fs'),
sqlite3 = require('./sql.js'),
dbfile = '/tmp/eg.db',
sql = 'INSERT INTO test(id) VALUES (NULL)',
// fs.read returns binary 'string' (not 'String' or 'Uint8Array')
read = fs.read(dbfile, 'b'),
// Database argument must be a 'string' (binary) not 'Uint8Array'
db = new sqlite3.Database(read),
write,
uint8array;
try {
db.run(sql);
} catch (e) {
console.error('ERROR: ' + e);
phantom.exit();
}
// db.export() returns 'Uint8Array' but we must pass binary 'string' to write
uint8array = db.export();
write = String.fromCharCode.apply(null, Array.prototype.slice.apply(uint8array));
fs.write(dbfile, write, 'b');
db.close();
phantom.exit();
Run the phantomjs script to test
$ /usr/local/phantomjs-2.0.0-macosx/bin/phantomjs /tmp/eg.js
Use external tool to verify changes were persisted.
sqlite3 /tmp/eg.db
sqlite> SELECT * FROM test;
id created
1 2015-03-28 10:21:09
sqlite>
Some things to keep in mind:
The database is modified on disk only when you call fs.write.
Any changes you make are invisible to external programs accessing the same SQLite database file until you call fs.write.
The entire database is read into memory with fs.read.
You may want to have different OS files for different tables -- or versions of tables -- depending on your application and the amount of data in the tables, to address the memory requirements you mentioned.
Passing what is returned by sqlite3.export() to fs.write will corrupt the SQLite database file on disk (it will no longer be a valid SQLite database file).
Uint8Array is NOT the correct type for the fs.write parameter.
Writing a binary data in phantomjs works like this:
var db_file = fs.open(db_name, {mode: 'wb', charset: ''});
db_file.write(String.fromCharCode.apply(null, db.export()));
db_file.close();
You have to set the charset to '' because otherwise the writing goes wrong.
I have an account on a coldfusion/SQL Server shared server hosting service and have been targeted by what I believe to be a SQL injection attack. I have a absolute path to a javascript file in nearly every field of every table. It seems the script hides majority of the text and features on every page it is on. I'm lucky enough to be able to restore the database to a reasonable point, but I need to prevent this from happening again. Is there anything else I can do besides <cfqueryparam></cfqueryparam> ?
The recommended solution is to review ALL your code and make sure that you don't build your sql statements via string concatenation. Instead, use parametrized queries and sanitize user input. That's recommended for any kind of database you use.
Take a look at this post.
EDIT - Providing C# Example as requested:
string sql = "insert into table_a (cola,colb,colc) values (#value_a,#value_v,#value_c)";
using(SqlConnection conn = new SqlConnection("ConnectionString"))
{
conn.Open();
SqlCommand command = new SqlCommand(sql,conn);
command.CommandType = Commandtype.Text;
command.Parameters.AddWithValue("#value_a",txtFieldAFromForm.Text);
command.Parameters.AddWithValue("#value_b",txtFieldBFromForm.Text);
command.Parameters.AddWithValue("#value_c",txtFieldCFromForm.Text);
command.ExecuteNonQuery();
}
Checkout <cfqueryparams> tag: http://www.adobe.com/livedocs/coldfusion/7/htmldocs/wwhelp/wwhimpl/common/html/wwhelp.htm?context=ColdFusion_Documentation&file=part_cfm.htm
This is how to clean up user-submitted input for SQL queries.