I'm looking for information on writing DataFlow jobs in NodeJS. The tutorials are all referring to Java or Python.
Any ideas if it's possible?
Until now(2.13.0), Beam allows writing pipeline by Python, Java and Go.
Beam community is working on cross language pipeline feature that allows call PTransforms written in another language, but NodeJS is not on the roadmap.
Check[1] for more information on it.
[1] https://beam.apache.org/roadmap/portability
There is support for TypeScript now.
As of February 2023. it's still experimental but main functionality is covered, from Apache Beam documentation:
The Typescript SDK supports Node v16+ and is still experimental.
You can install it via npm:
npm install apache-beam
More instructions and documentation can be found in the official sites at npm and github.
Link npm: https://www.npmjs.com/package/apache-beam
Link github (example): https://github.com/apache/beam-starter-typescript
Related
Amazon recently published its Amazon Product Advertising API v5.0 for Amazon Affiliate Program. From 31st October 2019, previous version 4.0 of API will be taken down and only v5.0 will be supported to access Amazon products programmatically.
With these changes, most available node modules and applications will no longer work with V5.0. For this Amazon PA-API 5.0, Amazon has released an SDK for PHP, NodeJs, Python and, Java. However, using the SDK for NodeJs may require time to configure.
That being said, is there an available Node Package module that we can use to easily start integrating our application?
There are a few NPM wrappers that are available today that help you use Paapi 5.0 for NodeJS. All of this module uses the amazon SDK for their code.
amazon-paapi - By far my favorite as it uses simple to understand node syntax. Adding parameters is much like how you do it using amazon scratchpad. It also support custom parameter options.
amazon-pa-api50 - Another great module. However, as of this writing, I found some limitations on optional parameters you can add to some operations.
apaw - alternative option.
I have been using Selenium, Testng, Java, Maven to automate browser and I know these compliment each other. We can use one tool's method (APIs) in with other.
Now I have to automate using Nightwatch.js. During its initial setup I was asked to install node.js, but never got to know that why I need it? Why an installation of Nightwatch.js itself not enough?
I have tried to find out on Nightwatch's official page, but couldn't find out. Seems I am asking a very basic, foundation level question.
In a Nutshell:
NodeJS is a Javascript runtime that runs on the server. It is built on V8, Google's Javascript engine. This is the equivalent of the JVM and the Java Platform.
npm is a very popular package manager for the Javascript world and it is tighly integrated with Node. This is probably on the npm repositories that you will download Nightwatch.js. This is the package manager part of Maven for Javascript.
Nightwatch.js is a Node library that wraps Selenium for use in a Javascript environment. It is also a test runner, like JUnit or TestNG are for Java.
Hope this helps.
I'm working on converting a library from full .NetFramework to .NetCore
I'm looking for a replacement for Microsoft.Azure.Management.HDInsight.Job, which hasn't been updated in over a year and is not compatible with .NetCore. I was hoping that the functionality would be rolled up into the much-more-recently-updated and netcore-compatible Microsoft.Azure.Management.HDInsight, but that doesn't appear to be the case.
I'm down to use the REST API, but I haven't been able to find the same functionality there. Any guidance would be appreciated.
You could try to install Microsoft.Azure.Management.HDInsight.Job with Package Manager to install some prerelease versions, so that its dependencies would not be conflict with your asp.net core.
I test them, no matter it is preview, it also have the functionality what you want.
Write in Package Manager Console such as:
Install-Package Microsoft.Azure.Management.HDInsight.Job -Version 1.0.7-preview
You could only install the version <= 1.0.7-preview. If not, you may could not install it.
For more detail, you could refer to this article.
I found the REST API I was looking for. It is the WebHCat API, not an Azure API.
MapReduce Job creation: https://cwiki.apache.org/confluence/display/Hive/WebHCat+Reference+MapReduceJar
Pig Job creation:
https://cwiki.apache.org/confluence/display/Hive/WebHCat+Reference+Pig
Hive Job creation:
https://cwiki.apache.org/confluence/display/Hive/WebHCat+Reference+Hive
Sqoop Job creation: https://learn.microsoft.com/en-us/azure/hdinsight/hadoop/apache-hadoop-use-sqoop-curl and https://sqoop.apache.org/docs/1.99.3/RESTAPI.html
Hopefully they will release 3.0.0 soon
https://github.com/Azure/azure-sdk-for-net/issues/9219
Yahoo just released a version of caffe that uses the latest version of Apache-Spark yesterday, the git repo is not well documented yet: git link
There is a scala test file which is suppose to run an example: Scala Example
but it requires the dependency com.yahoo.ml.caffe.{Config, CaffeOnSpark, DataSource} which I assume contains basically the data, the config and the API. Has this been made into a library yet? How could I build this using sbt?
Our CaffeOnSpark release contains all the code that you need to run. com.yahoo.ml.caffe.* are collection of Scala classes in caffe-grid folder. Please follow our guides at CaffeOnSpark wiki page, and ask questions at your mailing list.
I am interested in playing around with the Q & A service, the demo app uses the REST API and I am wondering if the service can be used with the newer watson-developer-cloud module and if so where can I find an example or documentation for those functions.
The demo is online at Q&A Node.js demo but this uses the older REST API.
Thanks,
Andy
Yes, you should be able to use the service with the newer watson-developer-cloud module, but there's no updated demo. Note, that the module was not changed drastically enough that it should really matter.
The README.md file for the Github repository containing the project has a full set of usage instructions.