I am looking for the exporting db table content into the csv file. For that I found that Apache Sqoop does it very fast. and also import into hdfs.
I am successfully able import to hdfs using Apache Sqoop for Oracle 11g. Now when I tried for Sybase database. It's not completing the task and situation like nothing happening.
command using...
"import"
, "--driver", "com.sybase.jdbc4.jdbc.SybDriver"
, "--connect"
, "jdbc:sybase:Tds:192.168.1.151:5002?ServiceName=test1"
, "--username", "tejkiran", "--password", "passw0rd"
, "--table", "PIN"
, "-m", "1"
, "--target-dir", "/tmp/HELLO/tej01"
Log Details...
16:51:38.249 [main] WARN org.apache.sqoop.tool.SqoopTool -
$SQOOP_CONF_DIR has not been set in the environment. Cannot check for additional configuration.
16:51:38.442 [main] DEBUG com.cloudera.sqoop.SqoopOptions - Generated nonce dir: /tmp/sqoop-hduser/compile/7c374682d26b0ab8b800a1bbbbccd726
16:51:38.547 [main] WARN org.apache.sqoop.tool.BaseSqoopTool - Setting your password on the command-line is insecure. Consider using -P instead.
16:51:38.586 [main] WARN org.apache.sqoop.ConnFactory - $SQOOP_CONF_DIR has not been set in the environment. Cannot check for additional configuration.
16:51:38.750 [main] DEBUG org.apache.sqoop.ConnFactory - Loaded manager factory: com.cloudera.sqoop.manager.DefaultManagerFactory
16:51:38.751 [main] DEBUG org.apache.sqoop.ConnFactory - Trying ManagerFactory: com.cloudera.sqoop.manager.DefaultManagerFactory
16:51:38.803 [main] INFO org.apache.sqoop.manager.SqlManager - Using default fetchSize of 1000
16:51:38.803 [main] DEBUG org.apache.sqoop.ConnFactory - Instantiated ConnManager org.apache.sqoop.manager.GenericJdbcManager#50246923
16:51:38.803 [main] INFO org.apache.sqoop.tool.CodeGenTool - Beginning code generation
16:51:40.279 [main] DEBUG org.apache.sqoop.manager.SqlManager - No connection paramenters specified. Using regular API for making connection.
16:51:41.147 [main] DEBUG org.apache.sqoop.manager.SqlManager - Using fetchSize for next query: 1000
16:51:41.164 [main] INFO org.apache.sqoop.manager.SqlManager - Executing SQL statement: SELECT t.* FROM PIN AS t WHERE 1=0
After this statement there is not log updated and
Related
How can I generate nodejs-server with OAS 2.0 as input with openapi-generator?
openapi-generator runs on two versions 3.3.4 and 4.0.0.
The results of my execution are listed below.
■in 3.3.4
java -jar openapi-generator-cli-3.3.4.jar generate -i petstore.json -g nodejs-server -o stub
[main] WARN o.o.c.ignore.CodegenIgnoreProcessor - Output directory does not exist, or is inaccessible. No file (.openapi-generator-ignore) will be evaluated.
[main] WARN o.o.c.languages.NodeJSServerCodegen -
=======================================================================================
Currently, Node.js server doesn't work as its dependency doesn't support OpenAPI Spec3.
For further details, see https://github.com/OpenAPITools/openapi-generator/issues/34
=======================================================================================
[main] INFO o.o.codegen.DefaultGenerator - Model Pets not generated since it's an alias to array (without property)
Exception in thread "main" java.lang.RuntimeException: Could not generate api file for 'Pets'
at org.openapitools.codegen.DefaultGenerator.generateApis(DefaultGenerator.java:651)
at org.openapitools.codegen.DefaultGenerator.generate(DefaultGenerator.java:891)
at org.openapitools.codegen.cmd.Generate.run(Generate.java:355)
at org.openapitools.codegen.OpenAPIGenerator.main(OpenAPIGenerator.java:62)
Caused by: java.lang.IllegalArgumentException: character to be escaped is missing
at java.util.regex.Matcher.appendReplacement(Matcher.java:809)
at java.util.regex.Matcher.replaceAll(Matcher.java:955)
at java.lang.String.replaceAll(String.java:2223)
at org.openapitools.codegen.languages.NodeJSServerCodegen.apiFilename(NodeJSServerCodegen.java:192)
at org.openapitools.codegen.DefaultGenerator.generateApis(DefaultGenerator.java:595)
... 3 more
■in 4.0.0
java -jar openapi-generator-cli-4.0.0.jar generate -i petstore.json -g nodejs-server -o stub
[main] WARN o.o.c.ignore.CodegenIgnoreProcessor - Output directory does not exist, or is inaccessible. No file (.openapi-generator-ignore) will be evaluated.
[main] INFO o.o.codegen.DefaultGenerator - OpenAPI Generator: nodejs-server (server)
[main] INFO o.o.codegen.DefaultGenerator - Generator 'nodejs-server' is considered stable.
[main] WARN o.o.c.languages.NodeJSServerCodegen -
=======================================================================================
Currently, Node.js server doesn't work as its dependency doesn't support OpenAPI Spec3.
For further details, see https://github.com/OpenAPITools/openapi-generator/issues/34
=======================================================================================
[main] INFO o.o.codegen.DefaultGenerator - Model Pets not generated since it's an alias to array (without property) and `generateAliasAsModel` is set to false (default)
Exception in thread "main" java.lang.RuntimeException: Could not generate api file for 'Pets'
at org.openapitools.codegen.DefaultGenerator.generateApis(DefaultGenerator.java:666)
at org.openapitools.codegen.DefaultGenerator.generate(DefaultGenerator.java:922)
at org.openapitools.codegen.cmd.Generate.run(Generate.java:396)
at org.openapitools.codegen.OpenAPIGenerator.main(OpenAPIGenerator.java:60)
Caused by: java.lang.IllegalArgumentException: character to be escaped is missing
at java.util.regex.Matcher.appendReplacement(Matcher.java:809)
at java.util.regex.Matcher.replaceAll(Matcher.java:955)
at java.lang.String.replaceAll(String.java:2223)
at org.openapitools.codegen.languages.NodeJSServerCodegen.apiFilename(NodeJSServerCodegen.java:181)
at org.openapitools.codegen.DefaultGenerator.generateApis(DefaultGenerator.java:611)
... 3 more
The input OAS uses the following.
https://raw.githubusercontent.com/OAI/OpenAPI-Specification/master/examples/v2.0/json/petstore.json
The execution environment is as follows.
・windows10
・java 1.8.0_202
It works for me:
java -jar modules/openapi-generator-cli/target/openapi-generator-cli.jar generate -g nodejs-server -i https://raw.githubusercontent.com/OAI/OpenAPI-Specification/master/examples/v2.0/json/petstore.json -o /tmp/nodejs-server
[main] INFO o.o.codegen.DefaultGenerator - OpenAPI Generator: nodejs-server (server)
[main] INFO o.o.codegen.DefaultGenerator - Generator 'nodejs-server' is considered stable.
[main] WARN o.o.c.languages.NodeJSServerCodegen -
=======================================================================================
Currently, Node.js server doesn't work as its dependency doesn't support OpenAPI Spec3.
For further details, see https://github.com/OpenAPITools/openapi-generator/issues/34
=======================================================================================
[main] INFO o.o.codegen.DefaultCodegen - Skipped overwriting README.md as the file already exists in /tmp/java2//README.md
[main] INFO o.o.codegen.DefaultGenerator - Model Pets not generated since it's an alias to array (without property) and `generateAliasAsModel` is set to false (default)
[main] INFO o.o.codegen.AbstractGenerator - writing file /tmp/java2/service/PetsService.js
[main] INFO o.o.codegen.AbstractGenerator - writing file /tmp/java2/controllers/Pets.js
[main] INFO o.o.codegen.AbstractGenerator - writing file /tmp/java2/utils/writer.js
[main] INFO o.o.codegen.AbstractGenerator - writing file /tmp/java2/api/openapi.yaml
[main] INFO o.o.codegen.AbstractGenerator - writing file /tmp/java2/index.js
[main] INFO o.o.codegen.AbstractGenerator - writing file /tmp/java2/package.json
[main] INFO o.o.codegen.AbstractGenerator - writing file /tmp/java2/.openapi-generator/VERSION
But as mentioned in the warning, the nodejs-server generator no longer works as expected as one of its dependencies does not support OpenAPI spec v3.
Please refer to https://github.com/OpenAPITools/openapi-generator/issues/2828 for the latest development of creating a new NodeJS Express generator.
UPDATE (2019/09): we've added a new nodejs-express-server generator. Please refer to https://twitter.com/oas_generator/status/1160000504455319553 for more information.
I am trying to import data from SAP HANA database onto Azure DataLake Store using SQOOP.
for this, I've downloaded the HDB client to connect to HANA database but I'm looking for the location to copy 'ngdbc.jar' to $SQOOP_HOME/lib. On HDInsight Cluster, am not able to see the environmental variable $SQOOP_HOME/lib, it seems to be blank. Can anybody point me to the right location on HDP - HDInsight Cluster.
Currently, I am encountering following error.
sshadmin#hn0-busea2:~$ sqoop import --connect 'jdbc:sap://XXXXXXX0004.ms.XXXXXXX.com:30015/?database=HDB&user=XXXXXXXXX&password=XXXXXXXXXXXXX' --driver com.sap.db.jdbc.Driver \
--query 'select * from XXX.TEST_HIERARCHY where $CONDITIONS' \
--target-dir 'adl://XXXXXXXXXXXXX.azuredatalakestore.net:443/hdi-poc-dl/SAP_TEST_HIERARCHY' \
--m 1;
Warning: /usr/hdp/2.4.2.4-5/accumulo does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
17/01/18 10:34:26 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6.2.4.2.4-5
17/01/18 10:34:26 WARN sqoop.ConnFactory: Parameter --driver is set to an explicit driver however appropriate connection manager is not being set (via --connection-manager). Sqoop is going to fall back to org.apache.sqoop.manager.GenericJdbcManager. Please specify explicitly which connection manager should be used next time.
17/01/18 10:34:26 INFO manager.SqlManager: Using default fetchSize of 1000
17/01/18 10:34:26 INFO tool.CodeGenTool: Beginning code generation
17/01/18 10:34:26 ERROR sqoop.Sqoop: Got exception running Sqoop: java.lang.RuntimeException: Could not load db driver class: com.sap.db.jdbc.Driver
java.lang.RuntimeException: Could not load db driver class: com.sap.db.jdbc.Driver
at org.apache.sqoop.manager.SqlManager.makeConnection(SqlManager.java:856)
at org.apache.sqoop.manager.GenericJdbcManager.getConnection(GenericJdbcManager.java:52)
at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:744)
at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:767)
at org.apache.sqoop.manager.SqlManager.getColumnInfoForRawQuery(SqlManager.java:270)
at org.apache.sqoop.manager.SqlManager.getColumnTypesForRawQuery(SqlManager.java:241)
at org.apache.sqoop.manager.SqlManager.getColumnTypesForQuery(SqlManager.java:234)
at org.apache.sqoop.manager.ConnManager.getColumnTypes(ConnManager.java:304)
at org.apache.sqoop.orm.ClassWriter.getColumnTypes(ClassWriter.java:1845)
at org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1645)
at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:107)
at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:478)
at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:605)
at org.apache.sqoop.Sqoop.run(Sqoop.java:148)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:184)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:226)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:235)
at org.apache.sqoop.Sqoop.main(Sqoop.java:244)
try this path /usr/hdp/current/sqoop-client/lib/
I am using presto-server-0.149 on macOS 10.11. Just for testing purpose, I run a single node and everything is okay. When I add etc/catalog/localfile.properties with:
connector.name=localfile
presto-logs.http-request-log-location=/var/log/apache2/access_log
I get the following error:
2016-07-04T12:02:45.435-0700 INFO main io.airlift.bootstrap.LifeCycleManager Life cycle starting...
2016-07-04T12:02:45.435-0700 INFO main io.airlift.bootstrap.LifeCycleManager Life cycle startup complete. System ready.
2016-07-04T12:02:45.436-0700 INFO main com.facebook.presto.metadata.CatalogManager -- Added catalog jmx using connector jmx --
2016-07-04T12:02:45.436-0700 INFO main com.facebook.presto.metadata.CatalogManager -- Loading catalog etc/catalog/localfile.properties --
2016-07-04T12:02:45.797-0700 INFO main Bootstrap PROPERTY DEFAULT RUNTIME DESCRIPTION
2016-07-04T12:02:45.797-0700 INFO main Bootstrap presto-logs.http-request-log.pattern null null If log location is a directory this glob is used to match the file names in the directory
2016-07-04T12:02:45.797-0700 INFO main Bootstrap presto-logs.http-request-log.location var/log/http-request.log var/log/http-request.log Directory or file where http request logs are written
2016-07-04T12:02:45.797-0700 INFO main Bootstrap
2016-07-04T12:02:45.797-0700 WARN main Bootstrap UNUSED PROPERTIES
2016-07-04T12:02:45.797-0700 WARN main Bootstrap presto-logs.http-request-log-location=/var/log/apache2/access_log
2016-07-04T12:02:45.797-0700 WARN main Bootstrap
2016-07-04T12:02:45.989-0700 ERROR main com.facebook.presto.server.PrestoServer Unable to create injector, see the following errors:
1) Configuration property 'presto-logs.http-request-log-location=/var/log/apache2/access_log' was not used
at io.airlift.bootstrap.Bootstrap.lambda$initialize$2(Bootstrap.java:235)
1 error
com.google.inject.CreationException: Unable to create injector, see the following errors:
1) Configuration property 'presto-logs.http-request-log-location=/var/log/apache2/access_log' was not used
at io.airlift.bootstrap.Bootstrap.lambda$initialize$2(Bootstrap.java:235)
1 error
at com.google.inject.internal.Errors.throwCreationExceptionIfErrorsExist(Errors.java:466)
at com.google.inject.internal.InternalInjectorCreator.initializeStatically(InternalInjectorCreator.java:155)
at com.google.inject.internal.InternalInjectorCreator.build(InternalInjectorCreator.java:107)
at com.google.inject.Guice.createInjector(Guice.java:96)
at io.airlift.bootstrap.Bootstrap.initialize(Bootstrap.java:242)
at com.facebook.presto.localfile.LocalFileConnectorFactory.create(LocalFileConnectorFactory.java:64)
at com.facebook.presto.connector.ConnectorManager.createConnector(ConnectorManager.java:315)
at com.facebook.presto.connector.ConnectorManager.addCatalogConnector(ConnectorManager.java:169)
at com.facebook.presto.connector.ConnectorManager.createConnection(ConnectorManager.java:162)
at com.facebook.presto.connector.ConnectorManager.createConnection(ConnectorManager.java:148)
at com.facebook.presto.metadata.CatalogManager.loadCatalog(CatalogManager.java:99)
at com.facebook.presto.metadata.CatalogManager.loadCatalogs(CatalogManager.java:77)
at com.facebook.presto.server.PrestoServer.run(PrestoServer.java:115)
at com.facebook.presto.server.PrestoServer.main(PrestoServer.java:63)
UPDATE
Based on Dain Sundstrom's answer below, I was able to fix my problem. It turned out that the fb documentation for Local File Connector is incorrect. And since I needed to get something to feed Presto to test the localfile connector, I changed the config filepath to Presto's own request log:
presto-logs.http-request-log.location=/var/presto/data/var/log/http-request.log
You have a typo in the configuration property. It should be:
presto-logs.http-request-log.location=/var/log/apache2/access_log
Also, this connector can only process the http log format created by Presto itself, so you would need to reconfigure your Apache2 server to output the same format.
Im trying to query a titan db 0.5.4 via mogwai, but when I run the following script i get the error: rexpro.exceptions.RexProScriptException: transaction is not open
and I found the same question here
P.S there is no tag for mogwai
script:
#!/usr/bin/env python3
from mogwai.connection import execute_query, setup
con = setup('127.0.0.1', graph_name="bio4j", username="re", password="re")
results = execute_query("2 * a",params={"a":2}, connection= con)
print(results)
results = execute_query("bio4j.E",params={}, connection= con)
print(results)
log:
$ ./bin/rexster.sh --start
0 [main] INFO com.tinkerpop.rexster.Application - .:Welcome to Rexster:.
93 [main] INFO com.tinkerpop.rexster.server.RexsterProperties - Using [/Users/Phoenix/Dropbox/Graph4Bio/Titan/rexhome/config/rexster.xml] as configuration source.
102 [main] INFO com.tinkerpop.rexster.Application - Rexster is watching [/Users/Phoenix/Dropbox/Graph4Bio/Titan/rexhome/config/rexster.xml] for change.
730 [main] INFO com.thinkaurelius.titan.graphdb.configuration.GraphDatabaseConfiguration - Generated unique-instance-id=0a69045d1736-AngryMac-local1
804 [main] INFO com.thinkaurelius.titan.diskstorage.Backend - Initiated backend operations thread pool of size 8
905 [main] INFO com.thinkaurelius.titan.diskstorage.log.kcvs.KCVSLog - Loaded unidentified ReadMarker start time Timepoint[1455128079919000 μs] into com.thinkaurelius.titan.diskstorage.log.kcvs.KCVSLog$MessagePuller#302c971f
908 [main] INFO com.tinkerpop.rexster.RexsterApplicationGraph - Graph [bio4j] - configured with allowable namespace [tp:gremlin]
932 [main] INFO com.tinkerpop.rexster.config.GraphConfigurationContainer - Graph bio4j - titangraph[berkeleyje:/Users/Phoenix/Dropbox/Graph4Bio/Bio4j/bio4j] loaded
939 [main] INFO com.tinkerpop.rexster.server.metrics.HttpReporterConfig - Configured HTTP Metric Reporter.
941 [main] INFO com.tinkerpop.rexster.server.metrics.ConsoleReporterConfig - Configured Console Metric Reporter.
2058 [main] INFO com.tinkerpop.rexster.server.HttpRexsterServer - HTTP/REST thread pool configuration: kernal[4 / 4] worker[8 / 8]
2060 [main] INFO com.tinkerpop.rexster.server.HttpRexsterServer - Using org.glassfish.grizzly.strategies.LeaderFollowerNIOStrategy IOStrategy for HTTP/REST.
2160 [main] INFO com.tinkerpop.rexster.server.HttpRexsterServer - Rexster Server running on: [http://localhost:8182]
2160 [main] INFO com.tinkerpop.rexster.server.RexProRexsterServer - Using org.glassfish.grizzly.strategies.LeaderFollowerNIOStrategy IOStrategy for RexPro.
2160 [main] INFO com.tinkerpop.rexster.server.RexProRexsterServer - RexPro thread pool configuration: kernal[4 / 4] worker[8 / 8]
2162 [main] INFO com.tinkerpop.rexster.server.RexProRexsterServer - Rexster configured with [DefaultSecurity].
2163 [main] INFO com.tinkerpop.rexster.server.RexProRexsterServer - RexPro Server bound to [0.0.0.0:8184]
2177 [main] INFO com.tinkerpop.rexster.server.ShutdownManager - Bound shutdown socket to /127.0.0.1:8183. Starting listener thread for shutdown requests.
152568 [Grizzly(2) SelectorRunner] INFO com.tinkerpop.rexster.protocol.EngineController - ScriptEngineManager has factory for: ECMAScript
152568 [Grizzly(2) SelectorRunner] INFO com.tinkerpop.rexster.protocol.EngineController - ScriptEngineManager has factory for: gremlin-groovy
152568 [Grizzly(2) SelectorRunner] INFO com.tinkerpop.rexster.protocol.EngineController - Registered ScriptEngine for: gremlin-groovy
152569 [Grizzly(2) SelectorRunner] INFO com.tinkerpop.rexster.protocol.EngineHolder - Initializing gremlin-groovy engine with additional imports.
153259 [Grizzly(2) SelectorRunner] INFO com.tinkerpop.rexster.protocol.EngineHolder - ScriptEngine initializing with a custom script
154074 [Grizzly(2) SelectorRunner] INFO com.tinkerpop.rexster.protocol.EngineController - ScriptEngineManager has factory for: Groovy
154076 [Grizzly(2) SelectorRunner] INFO com.tinkerpop.rexster.protocol.session.RexProSessions - RexPro Session created: a2b416ce-75ea-4ecb-9835-b287162c90cb
154354 [Grizzly(4)] INFO com.tinkerpop.rexster.protocol.session.RexProSessions - Try to destroy RexPro Session: a2b416ce-75ea-4ecb-9835-b287162c90cb
154355 [Grizzly(4)] INFO com.tinkerpop.rexster.protocol.session.RexProSessions - RexPro Session destroyed or doesn't otherwise exist: a2b416ce-75ea-4ecb-9835-b287162c90cb
154356 [Grizzly(5)] INFO com.tinkerpop.rexster.protocol.session.RexProSessions - RexPro Session created: 5b8a669f-615d-4f84-9d1e-2d10624347f0
154525 [Grizzly(7)] WARN com.tinkerpop.rexster.protocol.server.ScriptServer - Could not process script [bio4j.E] for language [groovy] on session [[B#6634722f] and request [[B#68f38099]
154527 [Grizzly(8)] INFO com.tinkerpop.rexster.protocol.session.RexProSessions - Try to destroy RexPro Session: 5b8a669f-615d-4f84-9d1e-2d10624347f0
154527 [Grizzly(8)] INFO com.tinkerpop.rexster.protocol.session.RexProSessions - RexPro Session destroyed or doesn't otherwise exist: 5b8a669f-615d-4f84-9d1e-2d10624347f0
154529 [Grizzly(1)] INFO com.tinkerpop.rexster.protocol.session.RexProSessions - Try to destroy RexPro Session: 00000000-0000-0000-0000-000000000000
154529 [Grizzly(1)] INFO com.tinkerpop.rexster.protocol.session.RexProSessions - RexPro Session destroyed or doesn't otherwise exist: 00000000-0000-0000-0000-000000000000
Maintainer of mogwai here.
What version of mogwai are you using? in 0.7.7 there is no return value for setup method and the connection object should not be passed around. In fact when you call setup it creates a connection pool (a synchronous rexpro connection pool since there was no concurrency option specified). So in general, just call setup once for the life of your app and you can use execute query without any references.
Also this message in particular stands out:
154525 [Grizzly(7)] WARN com.tinkerpop.rexster.protocol.server.ScriptServer - Could not process script [bio4j.E] for language [groovy] on session [[B#6634722f] and request [[B#68f38099]
Is your graph configured with a graph name of "bio4j"? The default titan graph name is "graph" and the default graph object name mogwai uses is "g". If you have a graph name of "bio4j" you wouldn't reference this directly, you'd use the graph object name associated to the transaction. You can think of a graph-name as a database name in a SQL database, and the graph object being the transactional reference to said database. This is configured in the xml configuration file when starting titan. Particularly:
<graphs>
<graph>
<graph-name>graph</graph-name>
....
</graph>
</graphs>
So assuming you changed that from "graph" to "bio4j" and left the default graph_obj_name in the setup function as "g", then your query should read "g.E".
I have a small Rexster/Titan cluster using Cassandra. A Rexster extension is used to query the graph. I did some benchmarking and did start and stop Rexster/Titan many times. But now I run into a strange issue: Rexster refuses to start but does not display any error message.
I tried to figure out what is causing this and reduced the cluster to a single node 192.168.0.4.
If I remove my extension Rexster manages to start up.
# console output
Forking Cassandra...
Running `nodetool statusthrift`..... OK
(returned exit status 0 and printed string "running").
Forking Titan + Rexster...
Connecting to Titan + Rexster (127.0.0.1:8184)...... OK
(connected to 127.0.0.1:8184).
Run rexster-console.sh to connect.
but when I place my extension uber JAR in the ext folder Rexster refuses to start.
# console output
Forking Cassandra...
Running `nodetool statusthrift`..... OK
(returned exit status 0 and printed string "running").
Forking Titan + Rexster...
Connecting to Titan + Rexster (127.0.0.1:8184)............................
timeout exceeded (60 seconds): could not connect to 127.0.0.1:8184
See /var/lib/titan/bin/../log/rexstitan.log for Rexster log output.
If I now check rexstitan.log, as suggested by the console output, I can not find any error message.
# rexstitan.log
0 [main] INFO com.tinkerpop.rexster.Application - .:Welcome to Rexster:.
73 [main] INFO com.tinkerpop.rexster.server.RexsterProperties -
Using [/var/lib/titan/rexhome/../conf/rexster-cassandra-cluster.xml]
as configuration source.
78 [main] INFO com.tinkerpop.rexster.Application - Rexster is watching
[/var/lib/titan/rexhome/../conf/rexster-cassandra-cluster.xml] for change.
244 [main] INFO com.netflix.astyanax.connectionpool.impl.ConnectionPoolMBeanManager -
Registering mbean: com.netflix.MonitoredResources:type=ASTYANAX,
name=ClusterTitanConnectionPool,ServiceType=connectionpool
252 [main] INFO com.netflix.astyanax.connectionpool.impl.CountingConnectionPoolMonitor -
AddHost: 192.168.0.4
537 [main] INFO com.netflix.astyanax.connectionpool.impl.ConnectionPoolMBeanManager -
Registering mbean: com.netflix.MonitoredResources:type=ASTYANAX,
name=KeyspaceTitanConnectionPool,ServiceType=connectionpool
538 [main] INFO com.netflix.astyanax.connectionpool.impl.CountingConnectionPoolMonitor -
AddHost: 192.168.0.4
1951 [main] INFO com.thinkaurelius.titan.graphdb.configuration.GraphDatabaseConfiguration -
Set cluster.partition=false from store features
1971 [main] INFO com.thinkaurelius.titan.graphdb.configuration.GraphDatabaseConfiguration -
Set default timestamp provider MICRO
2019 [main] INFO com.thinkaurelius.titan.graphdb.configuration.GraphDatabaseConfiguration -
Generated unique-instance-id=7f0000012902-node1
2045 [main] INFO com.netflix.astyanax.connectionpool.impl.ConnectionPoolMBeanManager -
Registering mbean: com.netflix.MonitoredResources:type=ASTYANAX,
name=ClusterTitanConnectionPool,ServiceType=connectionpool
2046 [main] INFO com.netflix.astyanax.connectionpool.impl.CountingConnectionPoolMonitor -
AddHost: 192.168.0.4
2053 [main] INFO com.netflix.astyanax.connectionpool.impl.ConnectionPoolMBeanManager -
Registering mbean: com.netflix.MonitoredResources:type=ASTYANAX,
name=KeyspaceTitanConnectionPool,ServiceType=connectionpool
2054 [main] INFO com.netflix.astyanax.connectionpool.impl.CountingConnectionPoolMonitor -
AddHost: 192.168.0.4
2228 [main] INFO com.thinkaurelius.titan.diskstorage.Backend -
Initiated backend operations thread pool of size 4
6619 [main] INFO com.thinkaurelius.titan.diskstorage.log.kcvs.KCVSLog -
Loaded unidentified ReadMarker start time Timepoint[1423479705116000 μs]
into com.thinkaurelius.titan.diskstorage.log.kcvs.KCVSLog$MessagePuller#212f3ff1
6625 [main] INFO com.tinkerpop.rexster.RexsterApplicationGraph -
Graph [graph] - configured with allowable namespace [*:*]
The only entry that looks strange to me is the one concerning the log:
6619 [main] INFO com.thinkaurelius.titan.diskstorage.log.kcvs.KCVSLog -
Loaded unidentified ReadMarker start time Timepoint[1423479705116000 μs]
into com.thinkaurelius.titan.diskstorage.log.kcvs.KCVSLog$MessagePuller#212f3ff1
My exception uses the logger for debugging. You can see the instantiation an usage on github: https://github.com/sebschlicht/titan-graphity-kribble/blob/master/src/main/java/de/uniko/sebschlicht/titan/extensions/GraphityExtension.java#L22
Though Rexster failed to start there is a process with the PID displayed in the console but curl fails to connect to Rexster:
$ curl 192.168.0.4:8182
curl: (7) Failed to connect to 192.168.0.4 port 8182: Connection refused
Why doesn't Rexster throw an exception? How can I debug this situation?
edit:
I removed any log messages in my code. I removed all exceptions that may be thrown during startup. Still Rexster refuses to start with my extension and the only hint in the log files is the unidentified read marker. I have to clue what prevents Rexster from starting.
The log message is nothing to worry about.
After rebuilding the application in another project step-by-step Rexster is now able to start with the extension. During this rebuild I noticed two situations, that can cause the behaviour described:
Missing dependency
If your project depends on a second project you might use Maven to inject it as a dependency. However, if you use
mvn clean package
to build the extension's JAR file it does not contain this dependency by default. You need to use a Maven plugin (e.g. maven-shade-plugin) to create a shaded JAR that contains all the dependencies your extension needs. Set the dependency scope to provided for all Titan/Rexster/Blueprints related dependencies. Use the shaded uber-JAR to deploy the extension to Rexster.
However, this was not new to me and should not have caused the problem in my case. There might be more situations that cause this problem or maybe there was a problem with Maven that messed up the shaded JAR. Feel free to browse the commit on github to catch this voodoo.
Missing extension
Another cause of this behaviour is a missing extension.
If you specify an extension in the com.tinkerpop.rexster.extension.RexsterExtension resource file, that is not present on startup, Rexster does neither log nor throw an exception, but refuses to start.