I am trying to import a database via /dataimport?command=full-import.
solr-spec 4.8.0
My data-config.xml:
<dataConfig>
<dataSource type="JdbcDataSource"
driver="com.microsoft.sqlserver.jdbc.SQLServerDriver"
url="jdbc:sqlserver://dbservername/dbname;instance=sqlexpress2012;"
user="username"
password="pwd"
batchSize="10"
name="activeConnection"/>
<document name="Businesses">
<entity name="Business" pk="BusinessId"
query="select BusinessId from Business" >
<field column="BusinessId" name ="id"/>
<field column="BusinessId" name = "BusinessId"/>
</entity>
</document>
</dataConfig>
Log:
"verbose-output": [
"entity:Business",
[
"document#1",
[
"query",
"select BusinessId from Business",
"EXCEPTION",
"org.apache.solr.handler.dataimport.DataImportHandlerException:
Unable to execute query: select BusinessId from Business Processing Document # 1\r\n\tat
org.apache.solr.handler.dataimport.DataImportHandlerException.wrapAndThrow(DataImportHandlerException.java:71)\r\n\tat
org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.<init>(JdbcDataSource.java:281)\r\n\tat
org.apache.solr.handler.dataimport.JdbcDataSource.getData(JdbcDataSource.java:238)\r\n\tat
org.apache.solr.handler.dataimport.JdbcDataSource.getData(JdbcDataSource.java:42)\r\n\tat
org.apache.solr.handler.dataimport.DebugLogger$2.getData(DebugLogger.java:188)\r\n\tat
org.apache.solr.handler.dataimport.SqlEntityProcessor.initQuery(SqlEntityProcessor.java:59)\r\n\tat
org.apache.solr.handler.dataimport.SqlEntityProcessor.nextRow(SqlEntityProcessor.java:73)\r\n\tat
org.apache.solr.handler.dataimport.EntityProcessorWrapper.nextRow(EntityProcessorWrapper.java:243)\r\n\tat
org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:477)\r\n\tat
org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:416)\r\n\tat
org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:331)\r\n\tat
org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:239)\r\n\tat
org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:411)\r\n\tat
org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:483)\r\n\tat
org.apache.solr.handler.dataimport.DataImportHandler.handleRequestBody(DataImportHandler.java:179)\r\n\tat org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)\r\n\tat org.apache.solr.core.SolrCore.execute(SolrCore.java:1952)\r\n\tat org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:774)\r\n\tat
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:418)\r\n\tat
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:207)\r\n\tat
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419)\r\n\tat org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455)\r\n\tat org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)\r\n\tat org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557)\r\n\tat org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)\r\n\tat org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1075)\r\n\tat
org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384)\r\n\tat org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)\r\n\tat
org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009)\r\n\tat org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)\r\n\tat org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:255)\r\n\tat org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:154)\r\n\tat org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)\r\n\tat org.eclipse.jetty.server.Server.handle(Server.java:368)\r\n\tat org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489)\r\n\tat org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53)\r\n\tat org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:953)\r\n\tat org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:1014)\r\n\tat org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:861)\r\n\tat org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:240)\r\n\tat org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72)\r\n\tat org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264)\r\n\tat org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608)\r\n\tat org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543)\r\n\tat java.lang.Thread.run(Unknown Source)\r\nCaused by: com.microsoft.sqlserver.jdbc.SQLServerException: The TCP/IP connection to the host QBFBROKERS-DEV/CustomerConnect, port 1433 has failed. Error: \"null. Verify the connection properties. Make sure that an instance of SQL Server is running on the host and accepting TCP/IP connections at the port. Make sure that TCP connections to the port are not blocked by a firewall.\".\r\n\tat com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDriverError(SQLServerException.java:190)\r\n\tat com.microsoft.sqlserver.jdbc.SQLServerException.ConvertConnectExceptionToSQLServerException(SQLServerException.java:241)\r\n\tat com.microsoft.sqlserver.jdbc.SocketFinder.findSocket(IOBuffer.java:2243)\r\n\tat com.microsoft.sqlserver.jdbc.TDSChannel.open(IOBuffer.java:491)\r\n\tat com.microsoft.sqlserver.jdbc.SQLServerConnection.connectHelper(SQLServerConnection.java:1309)\r\n\tat com.microsoft.sqlserver.jdbc.SQLServerConnection.login(SQLServerConnection.java:991)\r\n\tat com.microsoft.sqlserver.jdbc.SQLServerConnection.connect(SQLServerConnection.java:827)\r\n\tat com.microsoft.sqlserver.jdbc.SQLServerDriver.connect(SQLServerDriver.java:1012)\r\n\tat org.apache.solr.handler.dataimport.JdbcDataSource$1.call(JdbcDataSource.java:151)\r\n\tat org.apache.solr.handler.dataimport.JdbcDataSource$1.call(JdbcDataSource.java:131)\r\n\tat
org.apache.solr.handler.dataimport.JdbcDataSource.getConnection(JdbcDataSource.java:394)\r\n\tat
org.apache.solr.handler.dataimport.JdbcDataSource.access$200(JdbcDataSource.java:42)\r\n\tat org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.<init>(JdbcDataSource.java:268)\r\n\t... 43 more\r\n",
"time-taken",
"0:0:29.422"
]
]
]
I have tried the query outside of this instance and all my database info is accurate
Any ideas about why dataimport is failing to execute my query?
solr-spec 4.8.0
I think your url statement has the ";" and which is causing the issue.
Please try by removing the ";" from the url statement.
It should be "url="jdbc:sqlserver://dbservername/dbname;instance=sqlexpress2012".
If this does not work then
check by changing the instance name to databaseName.
Url would be then
url="jdbc:sqlserver://dbservername;databaseName=sqlexpress2012;"
From the exception stacktrace you could see the following lines:
SQLServerException:
The TCP/IP connection to the host QBFBROKERS-DEV/CustomerConnect, port 1433 has failed.
Error: \"null. Verify the connection properties.
Make sure that an instance of SQL Server is running on the host and accepting TCP/IP
connections at the port. Make sure that TCP connections to the port are not blocked
by a firewall.
Could you please check connectivity between Solr instance and your MS SQL instance? Specifically if port 1433 is accessible by Solr instance.
Related
I'm trying to connect to a SAP SQLANYWHERE server using syqlanydb library and even sqlalchemy however both scenarios I get
sqlanydb.OperationalError: (b'Specified database not found', -83)
I can use the same connection details to connect using clients such as DBeaver but no matter what can't connect to this server, I've also tried ODBC drivers but server wasn't returning anything.
I have sqlanywhere 17 installed matching server and port 2638 is open. Have in mind this is a remote server.
My connection string:
sybase_connection_string = "sqlalchemy_sqlany://{user}:{pwd}#{host}:{port}/{db}".format(user=user, pwd=pwd, host=host, port=port, db=database)
engine = create_engine(sybase_connection_string)
engine.connect(sybase_connection_string)
Has anyone experienced this?
Thanks
I'm attempting to connect to a new Aurora PostgreSQL instance with Babelfish enabled.
NOTE: I am able to connect to the instance using the pg library through the normal port 5432 (the Postgres TDAS endpoint).
However, for this test, I am attempting to connect through the Babelfish TDS endpoint (1433) using the standard mssql package.
If I specify a database name (it is correct), I receive the error 'database "postgres" does not exist':
var config = {
server: 'xxx.us-east-1.rds.amazonaws.com',
database: 'postgres',
user: 'xxx',
password: 'xxx'
};
and the connection closes since the connection fails.
if I omit the database property in the config, like:
var config = {
server: 'xxx.us-east-1.rds.amazonaws.com',
user: 'xxx',
password: 'xxx'
};
It will connect. Also, I can use that connection to query basic things like SELECT CURRENT_TIMESTAMP and it works!
However, I can't access any tables.
If I run:
SELECT COUNT(1) FROM PERSON
I receive an error 'relation "person" does not exist'.
If I dot-notate it:
SELECT COUNT(1) FROM postgres.dbo."PERSON"
I receive an error "Cross DB query is not supported".
So, I can't connect to the specific database directly and if I connect without specifying a database, I can't cross-query to the table.
Any one done this yet?
Or, if not, any ideas on helping me figure out what to try next? I'm out of ideas.
Babelfish databases (that you connect to on port 1433) have nothing to do with PostgreSQL databases (port 5432). Essentially, all of Babelfish lives within a single PostgreSQL database (parameter babelfishpg_tsql.database_name).
You seem to have a single-db setup, because Cross DB query is not supported. With such a setup, you can only have a single database via port 1433 (apart from master and tempdb). You have to use CREATE DATABASE to create that single database (if it isn't already created; ask sys.databases).
I can't tell if it is supported to create a table in PostgreSQL (port 5432) and use it on port 1433 (the other way around is fine), but if so, you have to create it in a schema that you created with CREATE SCHEMA while connected on port 1433.
The answer was that I should be connecting to database "master".
Even though there is no database titled master in the instance, you still do connect to it.
Once connected, running the following:
select current_database();
This will indicate you are connected to database "babelfish_db".
I don't know how that works or why a database would have an undocumented alias.
The bigger answer here is that cross-DB object references are not currently supported in Babelfish, outside your current SQL Server database.
This is currently being worked on. Stay tuned.
Just got my brand new 6.8 DataStax Astra (Cassandra) and downloaded studio from https://www.datastax.com/dev/datastax-studio. My Nodejs connection works great. But trying connect from Studio - everything fails with SSL configurations:
All host(s) tried for query failed.. (com.datastax.driver.core.exceptions.TransportException:
It looks like port should be 29080 as per secure-connect .. /config.json
I used API User Admin Token for Client and Secret keys.
Not sure if it's related but python connection fails with:
...cassandra.cluster.NoHostAvailable: ('Unable to connect to any servers'
...Unauthorized('Error from server: code=2100 [Unauthorized] message="No SELECT permission on <table system_virtual_schema.keyspaces>"')
It can't connect to Astra because it's connecting to the wrong CQL port.
The correct port configuration is in the cqlshrc file in the [connection] section. For example:
[connection]
hostname = db-uuid-us-east1.db.astra.datastax.com
port = 39876
ssl = true
This is the correct CQL port to use to connect from Studio or other clients. Cheers!
Cassandra 6.8 Astra is currently using port 29042 by default.
Both Python connector and Studio work great under Admin User credentials, but not under the Admin API User. So the authorization error message was ligid. NodeJS works great with Admin API User credentials..
Change phppgadmin connection from adodb to php7.2-pgsql
Tried running an SQL query on database from phppgadmin and got error "Deprecated: The each() function is deprecated. This message will be suppressed on further calls in /usr/share/php/adodb/adodb-error.inc.php on line 114"
In my config.inc.php I don't see anything about defining a connection type
// Display name for the server on the login screen
$conf['servers'][0]['desc'] = 'PostgreSQL';
// Hostname or IP address for server. Use '' for UNIX domain socket.
// use 'localhost' for TCP/IP connection on this computer
$conf['servers'][0]['host'] = 'phppgadmin.com';
// Database port on server (5432 is the PostgreSQL default)
$conf['servers'][0]['port'] = 5432;
I would like to change the default connection from adodb to php7.2-pgsql
You need to run a newer version of phppgadmin with proper support for php7. I'd suggest just grabbing the code from git master at https://github.com/phppgadmin/phppgadmin
I have a Glue job script that does this (not showing imports and setup here) and it inserts the row into SQL Server RDS just fine:
columns = ['test']
vals = [("test")]
df = sqlContext.createDataFrame(vals, columns)
test = DynamicFrame.fromDF(df, glueContext, "test")
datasink = glueContext.write_dynamic_frame.from_catalog(frame = test,
database = "database-name", table_name = "table-name")
job.commit()
When I run with this same connection but for a larger test load (ends up being about 100 rows) I get this error:
An error occurred while calling o596.pyWriteDynamicFrame. The TCP/IP connection to the host , port 1433 has failed. Error: "Connection timed out: no further information. Verify the connection properties. Make sure that an instance of SQL Server is running on the host and accepting TCP/IP connections at the port. Make sure that TCP connections to the port are not blocked by a firewall
The thing is that I know there's no firewall or security group issue since one row inserts just fine. I've tried adding a loginTimeout parameter to the JDBC connection like so:
jdbc:sqlserver://<host>:<port>;databaseName=dbName;loginTimeout=600;
As it indicates you can do so here. But the connection fails with Glue when I do that but succeeds when I remove the loginTimeout parameter.
I've also checked the remote timeout configuration on my SQL Server instance and it shows as 600 seconds which is longer than any of my failed jobs so it couldn't be that.
How can I get around this connection timeout error? It seems to be a limitation built into Glue.
In order to do a JDBC connection with Glue you need to follow the steps in this documentation: https://docs.aws.amazon.com/glue/latest/dg/setup-vpc-for-glue-access.html
We had done that but it turns out that our self-referencing sec group wasn't actually self-referencing. Once we changed that it got resolved
I also had to create the connection as an Amazon RDS connection and not as a JDBC connection even though it's doing the same thing under the hood.
Even after doing all that I still had issues. Turns out that you need to add the sql connection specifically to the job outside of the script. If you hit "Edit Job" you'll see a list of sql connections there. If the connection you're trying to hit isn't on the list of required connections you will always timeout