I want to be able to configure placement constraints based on different node types (i.e. FrontEnd, BackEnd). Can I do this per-environment, so that in my test/local cluster there are no constraints, but on my production environments there is?
Yes, you can! The official docs describe a way to do this via code, but you can also do so with the service manifest files directly to get default & per-environment configuration. There is a blog post you can follow (found it in this answer).
Related
I am gonna deploy a node js service in openshift and there are few properties such as database configs and app properties which I need to externalize.
I have java applications running as part of solution which uses config server as config store and GIT as source. I have seen libs for npm to integrate with spring config server.
So, I am looking for best practices here, what would be best approach for externalizing configs in nodejs in orchestration tools like k8s or openshift. Or can we go with config server int the above scenario?
Please let know of any info , any pointers are highly appreciated.
There are multiple possibilities, one being the Cloud Config Server as you noted. However, the naive approach according to the Twelve-Factor App, the config should be stored in the environment:
The twelve-factor app stores config in environment variables
In OpenShift / Kubernetes, this means that we will store the configuration in the Deployment itself, in ConfigMaps or Secrets and then use these with envFrom.configMapRef (here is an example).
If you are moving towards orchestration tools, I would say use their offering. In k8s, you would typically use ConfigMaps to manage your application configs. The beauty of this solution is that you can also do Configuration as Code, so you keep your Configmaps version-controlled.
One more thing, NodeJs best practices is to use environment variables. So you can use orchestration offering to mount all your configs to the environment, plus you get secrets encryption for your sensitive info (API keys, etc..)
For anyone if it would help, we went for environment variable approach since we had very minimal parameters to work with and we don't see much change in this approach. If it grows we would be looking at the configmap approach (as also suggested by simon / obanby) above.
I want to create a modular node.js application stack containing a set of applications. The idea is that app1, app2, etc can use the controllers and models.
Inside each app folder, I can have app specific package.json, app.js, etc.
I am using express.
I have two issues:
Is it possible to have that structure?
Why I'm not able to deploy such an app set on GCP? When I try It throws 500 internal server error.
enter image description here
To create a similar architecture, even if I didn't find a way to have the same, you should use services. According to the official GAE doc:
Use services in App Engine to factor your large apps into logical
components that can securely share App Engine features and communicate
with one another. Generally, your App Engine services behave like
microservices. Therefore, you can run your whole app in a single
service or you can design and deploy multiple services to run as a set
of microservices.
Does this work for your use case?
Regarding question 2, you didn't provide any information about your current process, so I cannot help you. Please edit the question adding the deployment configuration (app.yaml, etc.) and how is it performed. Please delete any sensitive information before posting it.
There are multiple node services currently deployed and running through pm2 in aws environment.
Difficulty(in terms of maintenance) I see in my current code base is that each of these node services have a a separate configuration file (config\app.json) - Though, most of the properties in these configuration files are common for all the services, each of the property is mentioned in each individual service in code. If there is a change is any of these properties, I will have to modify the change in multiple places.
I would like to centralise the configurations across multiple node services. Is there a way to do that? Expectation is to have a centralised place for maintaining configurations. Any references would help.
I am not sure how your architecture is but if you do not mind creating a small library or microservice, which will just fetch you configurations from a small NoSQL database such as Redis which stores key-value pairs, then it will provide you with configurations at a centralized place.
Now the only configuration remains here is of redis which you can add while building the service by providing it's configuration as an environment variable using some thing like yargs.
Then in every service you'll have to make only one API call to fill up your config json in your case config/app.json
I have a Java web app running on EC2 under Tomcat (a WAR) that requires various sensitive configuration parameters - for example, the credentials associated with various other AWS services. I had been setting these as environment variables, but then discovered that running Tomcat as a service removes almost all environment variables. So currently I use a simple configuration file to store these values.
I don't believe this is a wise choice going forward, however, and would like to find an alternative. What is the right way to handle this kind of sensitive information?
IAM Roles are going to be your best friend here. The official docs here will point you in the right direction. There's also a post on the AWS security blog about it here.
I am completely new to elasticsearch but I like it very much. The only thing I can't find and can't get done is to secure elasticsearch for production systems. I read a lot about using nginx as a proxy in front of elasticsearch but I never used nginx and never worked with proxies.
Is this the typical way to secure elasticsearch in production systems?
If so, are there any tutorials or nice reads that could help me to implement this feature. I really would like to use elasticsearch in our production system instead of solr and tomcat.
There's an article about securing Elasticsearch which covers quite a few points to be aware of here: http://www.found.no/foundation/elasticsearch-security/ (Full disclosure: I wrote it and work for Found)
There's also some things here you should know: http://www.found.no/foundation/elasticsearch-in-production/
To summarize the summary:
At the moment, Elasticsearch does not consider security to be its job. Elasticsearch has no concept of a user. Essentially, anyone that can send arbitrary requests to your cluster is a “super user”.
Disable dynamic scripts. They are dangerous.
Understand the sometimes tricky configuration is required to limit access controls to indexes.
Consider the performance implications of multiple tenants, a weakness or a bad query in one can bring down an entire cluster!
Proxying ES traffic through nginx with, say, basic auth enabled is one way of handling this (but use HTTPS to protect the credentials). Even without basic auth in your proxy rules, you might, for instance, restrict access to various endpoints to specific users or from specific IP addresses.
What we do in one of our environments is to use Docker. Docker containers are only accessible to the world AND/OR other Docker containers if you explicitly define them as such. By default, they are blind.
In our docker-compose setup, we have the following containers defined:
nginx - Handles all web requests, serves up static files and proxies API queries to a container named 'middleware'
middleware - A Java server that handles and authenticates all API requests. It interacts with the following three containers, each of which is exposed only to middleware:
redis
mongodb
elasticsearch
The net effect of this arrangement is the access to elasticsearch can only be through the middleware piece, which ensures authentication, roles and permissions are correctly handled before any queries are sent through.
A full docker environment is more work to setup than a simple nginx proxy, but the end result is something that is more flexible, scalable and secure.
Here's a very important addition to the info presented in answers above. I would have added it as a comment, but don't yet have the reputation to do so.
While this thread is old(ish), people like me still end up here via Google.
Main point: this link is referenced in Alex Brasetvik's post:
https://www.elastic.co/blog/found-elasticsearch-security
He has since updated it with this passage:
Update April 7, 2015: Elastic has released Shield, a product which provides comprehensive security for Elasticsearch, including encrypted communications, role-based access control, AD/LDAP integration and Auditing. The following article was authored before Shield was available.
You can find a wealth of information about Shield here: here
A very key point to note is this requires version 1.5 or newer.
Ya I also have the same question but I found one plugin which is provide by elasticsearch team i.e shield it is limited version for production you need to buy a license and please find attached link for your perusal.
https://www.elastic.co/guide/en/shield/current/index.html