After running databricks-connect configure, when I run databricks-connect test, I am getting "The system cannot find the path specified." and then nothing happens, no error nothing. Please help me resolve this. Since there is no error message as well I am short pressed on what to google as well.
Update: I resolved this by matching the JAVA versions. The Databricks runtime in the cluster is 6.5 and on checking the documentation, it said JAVA 1.8.0_252 and so I had to look for a version closer to this and it is working now (both JDK and JRE are working).
There is still a caveat though. For tables that belong to a data lake I am still unable to make it work with
sparklyr::spark_read_parquet(sc = sc, path = "/.../parquet_table", header = TRUE, memory = FALSE)
It does work for the tables that belong to the "default" database in databricks. Not sure if this is just in my case but I am tired of all the tweaking I have been doing for the past week lol. Please comment if anyone has been able to get this working!
One of the hints is that you have JDK 15
Related
I got a HPCC community version 8.0.0.1 installation running. Whenever I try to add a new FilScope trhoug ECL Watch nothing happens. It simply doesn't create it.
I got some version 6.4 clusters running and this problem doesn't happen. Works fine there.
When I try to delete Scopes created on ver 6 through version 8 I got the error "BaseDN not specified". On the logs I see the same thing.
I am not missing any basedn configuration of the already in place basedn fields on environment.xml.
Thanks for posting this question. I just did some research, and found that this problem has been reported and is fixed in a newer release. Please try build 8.6.42 or newer.
FYI, it was fixed with this issue
https://track.hpccsystems.com/browse/HPCC-27892
Russ Whitehead
There are a lot of similar questions noting that they could not get Apache (2.4) started due to improperly created "PYTHONHOME" and "PYTHONPATH" variables.
I have installed mod_wsgi, I am using python 3.8.0, I am on Windows 10 x64 and have x64 bit installations of all my modules. When I start Apache, though, (httpd -k start, restart) the service does not start. I tried manually doing the same in the services tab of Task Manager, it also immediately stopped after I tried to run Apache.
Looking in the logs, it seems my issue is also related to the aforementioned variables, however in other peoples' cases, they were advised to remove the variables whereas it seems mine are not set to anything in the first place. I had read that setting these variables inadvertently masks system variables, but that does not seem to be the case for me.
The most explicit error in the logs seemed to be of the "encodings" module not being found. I have attached a screenshot of the log below.
Can anyone help me get Apache24 running?
P.S. I am using Apache Haus Distribution, I believe the name of the installation I have is.
Error.log Report
Yes, this error is reported when PYTHONHOME does not have a valid Python installation.
This answer may help: https://stackoverflow.com/a/69949005/2850706
I am facing below issue while connecting to DB server(Suse linux machine B) from app Server (Suse linux machine A) which is contains WEbSphere Application Server, configured with DB2. please find below details.
ASX7015E: Exception running command: "AdminControl.testConnection('DB_NAME(cells/cell_name/nodes/node_name/servers/server1|resources.xml#dataSource_ID)')"; exception information:
com.ibm.websphere.management.exception.AdminException
javax.management.MBeanException
java.sql.SQLNonTransientException: java.sql.SQLNonTransientException: null DSRA0010E: SQL State = 08001, Error Code = -1,639.
not able to find root-cause of this error
not able to find, from which server this error came.
Note : This issue came after doing some path update on production issues. WebSphere version is 9.0, using websphere internal java and openJPA 2.0 and DB2 version is 11.1.0.0.
The sqlode -1639 is caused by incorrect permissions on files on the Db2-server.
See this technote
The advice given there, in case of broken links, is:
Cause
This is caused by incorrect permissions and ownership of the following security files in ~/sqllib/security directory
db2chpw
db2ckpw
These files should have root as owner and must have permission -r-s--x--x
Resolving The Problem
Change the owner of the files to root
Change the permission of the files to -r-s--x--x
Update the instance using db2iupdt
Apart from the above, it is always unwise to use any "fixpack 0" of any Db2 product to production environments on linux/unix/windows. There are always too many silly bugs that have long since been fixed by subseqent cumulative fixpacks. Your question mentions v11.1.0.0 (this is fixpack 0 of Version 11.1 ). Best advice is to deploy the current most recent fixpack to the Db2-server, so get your DBA to do this first on your development/test environments before deploying to production.
Additionally consider ensuring to maintain the Db2 jdbc driver version on your WAS server, instead of continuing to use the (usually old version) that comes with WAS. Best practice is to keep the version of the jdbc driver the same between the WAS and the Db2-server, so keep those versions maintained.
I am trying to create an instance using the Configuration Manager of WCS 7. I am working on a Win 7 x64 machine with DB2 9.5 64 bit version.
I am struck with this Massloading error when the instance creation happens :
In createInstanceANT.log file :
[Massload] Massloading
C:\IBM\WebSphere\CommerceServer\schema\xml\wcs.keys.xml Error in
MassLoading, please check logs for details.
The error log shows the following error :
[jcc][10165][10044][4.3.111] Invalid database URL syntax:
jdbc:db2://:0/WCSDEMO. ERRORCODE=-4461, SQLSTATE=42815
C:\IBM\WEBSPH~1\COMMER~2\config\DEPLOY~1\xml\createBaseSchema.xml:185:
Error in massloading
WCSDEMO is the database name. The Massloader is not able to get the URL and port to connect. It is supposedly getting them from createInstance.properties file but it is not working. The createInstance.properties file has all the details of the DB to connect.
What could be the reason for this error and how to resolve it ? Is there any configuration change that I am missing ?
Can you provide some more details.
look inside the messages.txt file located in WC_install_dir/instances/instance_name/logs
and confirm what the exact issue is. If it is related to jdbc driver being wrong I may be able to help you.
I've been running into massloading problems with external systems. Eg. databases not on the same machine as the WAS installation.
In these cases I look for the
As you can see setting the loaderDBName to just the name of the database would look on the local machine. But by changing this statement so you load with the syntax
loaderDBName=[DATABASE_SERVER_NAME]:[PORT]/[DATABASE_NAME]
You'll be able to massload using the commerce standard scripts. These changes needs to be done in many scripts. Both for updating fixpacks and enabling features. If you run database updates without the changes it will crash at first and have done all the schema changes to the database that you then need to comment out before trying again.
IBM Software Support is your friend. They'll help you fix it.
Followed the steps to install the apache-cassandra latest build. Upon first startup (./cassandra -f), I get this:
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/cassandra/thrift/CassandraDaemon
Caused by: java.lang.ClassNotFoundException: org.apache.cassandra.thrift.CassandraDaemon
at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
at java.lang.ClassLoader.loadClass(ClassLoader.java:321)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294)
at java.lang.ClassLoader.loadClass(ClassLoader.java:266)
at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:334)
Could not find the main class: org.apache.cassandra.thrift.CassandraDaemon. Program will exit.
I exported the JAVA_HOME path, etc. What am I doing wrong? I should note that I am on an Ubuntu Lucid machine.
The first thing you should do is setup CASSANDRA_HOME path to the Cassandra root directory.
Try running cassandra cassandra -f and everything will run smooth. (Cassandra actually checks CASSANDRA_HOME environment variable to find the lib folder to run the deamon).
If using Ubuntu (Lucid), use the tutorial here: http://dustyreagan.com/installing-cassandra-on-ubuntu-linux/, which is based on the debian package. Building from Git didn't work.
This answer may help you if you don't technically need to build from source, and if you're just getting started with a fresh Cassandra install rather than upgrading an existing one.
I had the same problem when building from source. To get around it, I used a development build from the "Latest Builds (Hudson)" http://cassandra.apache.org/download/ link here.
The next problem you'll encounter is that no keyspaces will be set up on a fresh install. To get around that problem you can use the last release 0.6.3. That solution didn't work for me, because I wanted to use Pycassa which needs 0.7.
So what I had to do was the following steps:
Fire up a JMX console. Personally I'm not located with the server running Cassandra, so I needed to use ssh tunnels, like this:
jconsole -J-DsocksProxyHost=localhost -J-DsocksProxyPort=1080
Then used this funky looking url to connect:
service:jmx:rmi:///jndi/rmi://my.hostname.com:8080/jmxrmi
Then on the left side
expand org.apache.cassandra.service
expand Storage Service expand Operations
select loadSchemaFromYAML
at the top right, click the loadSchemaFromYAML button to invoke it.
You can use the same steps to add new keyspaces during development, once you figure out what you want your schema to look like. But the above steps only work if you have no data. So you would have to remove all your data using rm /var/lib/cassandra/* after taking down the server. (Of course, there are other steps you can take that are more complicated to migrate data without destroying it.)
I realize you didn't ask about creating keyspaces, but on a trunk version of cassandra, if your'e just getting started, that's the very next problem you'll have. I just spent a day solving it, and am hoping it helps.
Can you provide more details? Are you using ubuntu Open JDK 6?
Also, you don't have to build from source. Just get the binary from the following url:
http://mirror.nexcess.net/apache/cassandra/1.1.2/apache-cassandra-1.1.2-bin.tar.gz