I would like to give this a try- https://github.com/zalando-incubator/spark-json-schema
However, I have no clue how to even get started. I literally have no idea what this means "Include the library under the following coordinates". I am new to spark and have only used it via Python/PySpark. Any pointers greatly appreciated. Thanks!
If want to include library in Azure Databricks
Please follow below steps:
Click on computer inside bar
Click on cluster name
And then, click on the libraries tab and install libraries
Reference:
Install a library on a Databricks cluster | microsoft
Related
Our version of SAP (ECC6 EHP 0) does not have the FM ZRFC_READ_TABLE2. We created a copy and called it ZFM_INTERNAL_ZRFC_READ_TABLE2; however, when trying to use it from ADF with the SAP connector (version 3.0.23 64bit) we get an error.
Type: SAP.Middleware.Connector.RfcInvalidParameterException
Message: Element QUERY_TABLE of container metadata XXXXX/ZFM_INTERNAL_ZRFC_READ_TABLE2 unknown.
I posted this question in the GitHub feedback page (link) and was told this is a better forum to answer this type of questions.
The FM has the QUERY_TABLE parameter and it has been tested to be working (using Python scripts).
Screenshots of the configuration and tests performed.
FM Config
Test Configuration
Test Results
Any ideas where the problem could be?
Thanks in advance for your help!
As discussed in Microsoft Q&A Platform the above issue can be resolved if we are specifying function module alone without the function group.
Under Attribute tab, make sure you select "Remote-Enabled Module".
I am building a a demo/training environment for one of our products which work with Hive & Spark. I am using HDP 2.6.5 and If I configure the hive settings I need (primarily these: ACID Settings) through the Ambari GUI it works fine. But I want to automate this and setting these in hive-site.xml is not working (I have found many copies of this file, so it could simply be I am using the wrong one? )
How can I change from the command line what changes when I make changes in Dashboard->Hive->Configs ?
Where are these changes stored? I am sure I have missed something obvious in the docs, but I can't find it.
Thanks!
#Leigh K You should check out the Ambari REST API to make changes to hive. I did not find a quick link to official documentation, but I was able to find this post that goes into detail using PIG:
https://markobigdata.com/2018/07/22/adding-service-to-hdp-using-rest-api/2/
I followed "Writing Your First Application" tutorial. when I get to this section "Enrolling the Admin User"enter link description here
it asks me to launch this command node enrollAdmin.js.
But I check out my current directory, this file just doesn't exist. and I try to google it on the internet, search it on my computer, I still can not find it.
enter image description here
How you guys work out this? Do you have corresponding files?
Any advice will be appreciated.
You want to follow http://hyperledger-fabric.readthedocs.io/en/release/write_first_app.html which is our stable documentation link.
Put another way, you should use the stable rather than latest version of the docs.
The doc link you used points to our latest code but has not updated to point to a preview release yet.
My goal is to create a Deployment Manager profile in my Websphere on Linux.
Reading tons of documentation pages gives just two methods:
1) using X GUI application WAS_root/bin/ProfileManagement/pmt.sh
In this case, according to manuals, i should choose Management option, click "Next" and choose Deployment Manager server type.
Actually when i run pmt.sh in my WAS installation there is no option to choose Deployment Manager in Management section and only one profile type that i'm able to create is AdminAgent.
2) using manageprofiles.sh script under WAS_root/bin directory.
The manuals say that i can use -serverType DEPLOYMENT_MANAGER option in order to create deployment manager profile.
Actually when i run the script:
./manageprofiles.sh -create -templatePath ../profileTemplates/management/ -ServerType DEPLOYMENT_MANAGER -isDefault -profileName dmgr -adminUserName websphere -adminPassword websphere1
I get the following message:
The following validation errors were present with the command line
arguments:
serverType: The value for this parameter must be within this set of values [ADMIN_AGENT]
That means that i don't have the ability to create Deployment Manager at all.
Please advise what steps i can perform to be able to install Deploy Manager except erase my WAS installation and install it from scratch once again.
Thanks a lot.
You have standalone aka base edition of WebSphere Application Server. To create Deployment Manager you need Network Deployment edition.
Unfortunately you will have to install it from scratch from different installation files (from ND, not base or developers edition).
i am new to drupal.
I am currently working on web services. Eventhough i selected REST and XMLRPC servers in services I got the following warning: A library is missing for rest_server to work (Currently using Missing library Please download spyc and create a file called spyc.php in rest_server/lib)
i downloaded spyc and followed the above instruction but it dint worked.
can any one help me to troubleshoot this issue.
put the spyc.php inside the services module. (services/servers/rest_server/lib/)
try to clear site cache from "Administrator - > Site configuration -> performance"