Can anyone tell me why is the gp_dump utility is not available with greenplum database by default? If I have to use it then what is the source to download and way to enable it? I have gone through a lot of online resources but nothing relevant could be found
Are you using Greenplum v6.x?
The latest version of gpbackup is located here:
https://github.com/greenplum-db/gpbackup/releases
and if interested, the corresponding s3 plugin here:
https://github.com/greenplum-db/gpbackup-s3-plugin
gp_dump is a very old, deprecated backup utility for Greenplum.
The older python based gpcrondump/gpdbrestore utilities are still bundled in Greenplum 4.3.x and 5.x versions, but do not support Greenplum 6.x and thus removed.
The newer Golang based gpbackup/gprestore utilities support Greenplum 4.3.22 and later, Greenplum 5.5 and later and Greenplum 6.0 and later all in the same binary.
Let me know if you have additional questions.
oak
Try using pg_dump since Greenplum is a fork of postgres.
Related
We upgraded our Oracle DB and oracle client from 12.1 to 19.10.
Our application server have below setup
OS is Linux
Python version is 2.7.5 and there is no plan to upgrade the python version yet
cx_Oracle library version is 5.3
A quick test shows the application works fine but I am wondering if I need to upgrade the cx_Oracle library? Does it have a support expiry date? I tried to google it, read the information on its Github page but can't find any statement that answers my question.
This combination hasn't been tested. But since the Oracle Client libraries keep compatibility it is likely to work, subject to whatever underlaying enforced changes have been made to the Oracle libraries, perhaps to drop unsupported TLS versions or whatever has occurred in the 8 years since 12.1 was released that I don't know about. You application testing should be the final confirmation.
You should definitely set a goal to update to Python 3 so you can get the latest cx_Oracle version.
I have downloaded Spark source code(branch 2.4) and built the jars using the built instruction for Hadoop 2.7.4. I have also downloaded a pre-built version of Spark 2.4.4(Pre-built for Hadoop 2.7).
When I start spark-shell I see two different versions of Spark as shown in the picture below:
In the first picture, version is 3.0.0 for the jars built after downloading source code of branch 2.4. The second picture is from the pre-built version available from apache spark website. Not only that, the plans are using RelationV2 in first case and Relation logical node in second case.
Can anyone explain why is there such a difference?
Pretty sure you got mixed up, as 3.0.0 is the default choice for dowloading source or prebuilt version. Maybe I am mistaked, but, as of my comment, carefully check what version you have built.
Currently I have Spark version 1.6.2 installed.
I want to upgrade the Spark version to the newest 2.0.1. How do I do this without losing the existing configurations?
Any help would be appreciated.
If its maven or sbt application you simply change dependency version of spark and also migrate your code according to 2.0 so you will not lose you configurations. and for spark binary you can take backup of config folder.
There is no much change related to configuration, some method signatures are changed , major changes i observed was mapPartitions method signature and some changes to metrics/listener api, apart from new features.
I am trying to build a simple project with Spark+Cassandra for a SQL-analytics demo.
I need to use Cassandra v2.0.14 (can't upgrade it for now). I am unable to find the correct version of Spark and Spark-cassandra-connector. I referred to Datastax's git project at - https://github.com/datastax/spark-cassandra-connector, and I know that the Spark and Spark-cassandra-connector versions need to match and be compatible with Cassandra. Hence, would like anyone to help pointing out the exact versions for Spark, Spark-Cassandra-connector. I tried using v1.1.0 and v1.2.1 for both Spark and Spark-Cassandra-connector - but unable to build the spark-cassandra-connector jat jar with neither the supplied sbt (fails because the downloaded sbt-launch jar just contains a 404 not found html), nor my local sbt v0.13.8 (fails for compilation error for "import sbtassembly.Plugin.", "import AssemblyKeys.")
The connector works with Cassandra 2.0 and 2.1 but some features may also work fine with 2.2 and 3.0 (not officially supported yet) using the older Java driver 2.1. This is because C* Java driver supports a wide range of Cassandra versions. The newer driver works with older C* versions, but also the older driver versions work with newer C* versions, excluding new C* features.
However, there is a one minor caveat with using C* 2.0:
Since version 1.3.0, we dropped the thrift client from the connector. This move was to simplify connectivity code and make it easier to debug - debugging one type of connection should be easier than two. It either connects or not, no more surprises of a kind "it writes fine, but can't connect for reading". Unfortunately, not all of the thrift functionality was exposed by the native protocol in C* 2.0 nor in the system tables. Therefore, if you use C* prior to version 2.1.5, automatic split sizing won't work properly and you have to tell the connector the preferred number of splits. This is to be set in ReadConf object passed at the creation of the RDD.
As for the interface between the Connector and Spark, there is much less freedom. Spark APIs change quite often and you typically need a connector dedicated to the Spark version you use. See the version table in the README.
(fails because the downloaded sbt-launch jar just contains a 404 not found html)
This looks like an SBT problem, not a connector problem.
I just tried to do sbt clean assembly on all v1.2.5, v1.3.0, b1.4 and it worked fine.
if you can upgrade version of spark then you can connect with spark with cassandra .
put following maven dependency in pom file :-
cassandra-all
cassandra-core
cassandra-mapping
cassandra-thrift
cassandra-client
spark-cassandra-connector
spark-cassandra-connector-java
this will be work.
I am trying to upgrade a cassandra single node installation from 1.2.10 to 2.0.1.
I am using still the old RandomPartitioner, I change all the parameters in the cassandra.yaml file. But when I tried to start the node I get this error:
java.lang.RuntimeException: Can't open incompatible SSTable! Current version jb
In the notes from the documentation I can not find anything related with the change in the SSTables, did anyone deal with this problem?
If you guys had previously upgraded from an earlier version to 1.2.10, did you guys perform a 'nodetool upgradesstables'? This would have upgraded sstables to prevent possible incompatibilities.
The upgrade procedure from 1.2.9 to 2.x does not seem to recommend running it, so those sstables should be compatible, but trying it again might be worth it.
http://www.datastax.com/documentation/cassandra/1.2/webhelp/#upgrade/upgradeC_c.html#task_ds_gqm_vzr_ck