Cloudera documentation talks about (and, I believe, recommends) use of an edge node (aka gateway node) as a means of controlling external access to the cluster.
I have recently deployed a Cloudera cluster on Azure using the provided ARM template and discovered that no edge node gets provisioned during the deployment. Why is that? If an edge node is recommended should that not be included in such a deployment?
Actually right now every node provisioned using this Cloudera ARM template is an edge node (has the gateway role). This is a simple configuration for smaller clusters. You can customize the template to provision dedicated edge nodes as needed.
Related
Azure Service Fabric can be run on Windows Server. Can Service Fabric Mesh be hosted that way as well?
The underline platform is the same service fabric binaries, the only difference is that on MESH you don't manage nodes and all definitions is based on Containers and Hardware Resources (Network, CPU, storage), you will be able to simulate a "Single Node" MESH cluster like you do with current SF and deploy you mesh applications in there.
If your plan is to have a production environment onPrem I haven't got in much details about it, now that it is becoming an opensource solution, I assume yes, but I don't think it is on their top priority.
For now, there is not much documentation about it, so the best you can find will be on these links:
https://learn.microsoft.com/en-gb/azure/service-fabric-mesh/
https://azure.microsoft.com/en-us/blog/azure-service-fabric-mesh-is-now-in-public-preview/
How to setup local development cluster:
https://learn.microsoft.com/en-gb/azure/service-fabric-mesh/service-fabric-mesh-howto-setup-developer-environment-sdk
I am beginner in elastic search.I have implemented elastic search for single node with NEST Client in my C# Web Application successfully.Now I want to scale my data horizontally and want to deploy this application on Azure cloud storage.so if any article or documentation(step by step) is available for how to effectively deploy Elasticsearch with proper configuration on Azure with multi node then inform me.
Any suggestion is appreciated.
Take a look at Elastic's Azure Marketplace offering. Within a few screens and minutes, you'll have a cluster deployed on Azure.
If you need more control over the deployed cluster such as
number and size of disks to attach to each data node
install Azure repository plugin for snapshot/restore
Java heap size for each node
take a look at the ARM template on GitHub. This is the same underlying template that powers the Marketplace offering, but deploying directly provides more features than are exposed in the Marketplace UI.
I have a requirement to use Docker containers in PCF deployed in Azure.
And now we want to use kubernetes as container orchestration.
Does kubernetes can be used here ?
Or PCF will take care of the container orchasteration ?
Which one would be the better approach here ?
PKS Is Pivotal's answer to running kubernetes in PCF (regardless of IaaS)
Pivotal Cloud Foundry (PCF) is a sophisticated answer from Microsoft to current cloud expectations. PCF offers the best platform to run Microsoft based technology like .NET, and smoothly supports enterprise Java application. You can run Kubernetes there with fine results, but to achieve comfortable orchestration and management of containers I suggest reading about GKE or setting up your own Kubernetes cluster using kubespray utility.
We are developing Microservices with .NET Core on top of Service Fabric.
We have two development environments that is part of our release process, Automated Test Environment and Functional Test Environment. Using two full instances of SF on Azure is costy given that we can tolerate availability and performance for our dev environment and given that our production environment is on Azure. We already have a VM that we can use.
Does SF work under Windows Server 2016 Core? (I couldn't find any affirmation online).
Is it possible to have two instances of our application running on one VM?
In order to setup a Service Fabric cluster you need at least 3 machines (or you cannot reach quorum in your cluster). If you run it on Azure then you also choose the Reliability and durability tier for your nodetypes https://learn.microsoft.com/en-us/azure/service-fabric/service-fabric-cluster-capacity#the-reliability-characteristics-of-the-cluster. A higher reliability tier (Silver, Gold or Platinum) means that you need further nodes (machines) in cluster.
You can run multiple instances of the same application and of different deployment versions in the same cluster. You need to consider how your services are assigned ports (for those that expose HTTP endpoints), otherwise these will conflict if you have multiple instances of the same application type in the same cluster. There is currently no way to provision new instances through Visual Studio, you need to use PowerShell, the API or the Service Fabric Explorer.
I am able to create a new Hadoop cluster through the interface, but need to create a new cluster on request. Does anyone know if an API exists to create a new cluster?
Not yet. As of now (Preview) you must be using Windows Azure Management Portal Interface to create Hadoop cluster in your windows Azure Subscription.
As most of the Windows Azure Management functionalities are available on Powershell, it is possible to have such functionality built into Powershell over REST as described here however I don't know any immediate plans.
Yes, you can.
The Azure CLI allows you to control HDInsight clusters from a batch file for example. You then get a bunch of HDInsight control functions. Type
azure hdinsight
to see all the built in help. It covers all the basics (listing, creating configuring clusters) and the multiple storage account functionality.
This is I believe based on the nodejs sdk. To get going, install nodejs and
npm install azure-cli
This should give you what you need to be able to manage the clusters from the command line.
By asking "Create a new Hadoop cluster", I believe you mean Hadoop HDInsight Cluster.
If yes, then we can create a powershell(.ps1) script, which can do the job for you.
Here is the sample script which might be useful.
http://mydailyfindingsit.blogspot.in/2016/01/create-script-hdinsight-cluster.html