I am trying to deploy the Spring Cloud Data Flow Server to an enterprise Pivotal Cloud Foundry instance using an application manifest.yml.
My first concern is externalizing the SPRING_CLOUD_DEPLOYER_CLOUDFOUNDRY_USERNAME and SPRING_CLOUD_DEPLOYER_CLOUDFOUNDRY_PASSWORD properties from the manifest so I'm not storing them in plain text in version control. Is there a best practice to accomplish that? Where does that fit into the workflow of deploying a Data Flow Server and associated Streams/Tasks? My initial idea is to create a new Cloud Foundry user for the Data Flow Server that can deploy applications to the org/space I am using, then potentially use Spring Cloud Config to inject the credentials into the env. Is that a typical solution?
My second concern is requiring TLS/HTTPS for all of the Data Flow Server's web endpoints. I again attempted to omit the properties concerning the key store locations from the manifest in the SPRING_APPLICATION_JSON env entry, but that again caused initialization to fail. Does the Cloud Foundry Java buildpack have a standard location for those certificate stores, or is that specific to my Cloud Foundry instance? I also want to redirect all HTTP requests to HTTPS? Do I need to create a custom build of the Cloud Foundry Data Flow Server to do that?
Thanks for any help.
UPDATE:
Based on other SO posts, I have now resolved that I do not need to enable HTTPS/TLS from the Data Flow Server config in its embedded Tomcat server because it is sitting behind Cloud Foundry's proxy. However, I cannot see a way to require and redirect to HTTPS endpoints other than creating a customized build of the Data Flow Server for Cloud Foundry with the proper configuration.
You can refer the documentation for this here.
The Security section from the docs linked in the previous answer should be useful. I'll attempt to reply few specific questions.
I'm not storing them in plain text in version control
A few of our customers use Config-Server + Vault integration for encrypted password credentials.
My initial idea is to create a new Cloud Foundry user for the Data Flow Server that can deploy applications to the org/space I am using, then potentially use Spring Cloud Config to inject the credentials into the env. Is that a typical solution?
This whole experience will be automated with e2e OAuth/SSO workflow with Spring Cloud Data Flow's tile, which will be available as a BETA tile by this month. There won't be any clear-text passwords - everything will be OAuth token driven, and the service-broker will control the lifecycle of SCDF's service-instance. Please reach out to PCF account rep and we would gladly add you to the BETA program.
Related
I made a simple Node.js application that is using Google Cloud's Speech to Text API.
I could successfully authenticate locally, but how do I authenticate in production while keeping the credentials privates?
P.S. My application is not using any other server than Google's servers.
The credentials necessary for making client calls are based on a strategy called Application Default Credentials. These can be found through an environment variable called GOOGLE_APPLICATION_CREDENTIALS if the variable is present. If the the variable is not set, then the default credentials can be used. The following article provides detailed guidance on this topic:
Setting Up Authentication for Server to Server Production Applications
If your client is running outside of Google on your own premises (and hence not in a Compute Engine or GKE cluster) then the thinking will likely be that the credentials are contained in a file that is itself content protected by local OS access permissions. For example, contained in a file which has read permission restricted to either the user that runs your application or group that the user is a member of.
You need to be more specific to get more accurate answers
====================
OK, it's all about how you control the permission of your service account and there is a note in that page: "If you are developing a production app, specify more granular permissions than Project > Owner. For more information, see granting roles to service accounts."
You should use a specific service account needed to use this service. I don't have the particular role name(something like AutoML Viewer or Editor), and you need to test a minimum workable role in your Dev ENV.
After which, you can deploy your service into PROD and make sure the key with a 0400 file. So more or less the account can use limited service as expected
I am trying to build Serverless Microservice Architecture
Azure services used by me are:
Azure CDN
Azure Active Directory
Azure Logic Apps
Azure Functions
Azure Event Grid
Azure SignalR Service
Which below tools do I need mange and secure my API in Azure Serverless Microservice Architecture?
Azure Traffic Manager
Azure Application Gateway
Azure API Management
Azure Function Proxy
Links Referred by me are :
https://learn.microsoft.com/en-us/azure/architecture/reference-architectures/serverless/web-app
https://learn.microsoft.com/en-us/dotnet/standard/serverless-architecture/serverless-design-examples
Please help
Edit:
I understand above tools and it purpose but what I can't understand is do I require them, if yes in what order, all I am developing is an Angular 2+ app, post a Command Event to Azure Functions/Logic Apps using REST api returning RequestId (and triggering chain of events) and subscribing to that RequestId to listen for Domain Event.
This is very broad Architecture question. All the services you mentioned have specific purpose. You can even secure your functions without using any of them by simply turning on authentication on functions.
I would suggest reading all of them in details can help you identify which service may suit your needs in this case. e.g. Traffic manager is used for cross region traffic distribution and may not be required in your case. Function proxies and Api management overlap in few cases and really depends on what you are trying to achieve. To get better idea you may need to share your architecture diagram.
I have an app that I want to run in Cloud Foundry (specifically, in IBM Bluemix). This app will call out to a number of third-party services/APIs, most of which are not managed via CF services. One of these APIs requires my app to use an SSL key to authenticate, so my app needs access to a private SSL key.
My app is deployed from an automated pipeline (specifically, IBM Bluemix DevOps Pipelines) based off of source from a GitHub repo and some small scripts saved in the pipeline config.
It seems like a bad idea to keep the private key file in the source control, and I don't believe there's anywhere to keep it in the pipeline config, so what are my options for making it available to the app while keeping it secure?
I've read this article: Keeping secrets – how your cloud application should access credentials and other private data which says that secrets, passwords, etc, can be made available via environment variables or Cloud Foundry user-provided services.
Googling to find out if this is a suitable option for SSL keys, I find this Cloud Foundry Page: Understanding Cloud Foundry Security which says "Store SSL keys securely to prevent disclosure, and promptly replace lost or compromised keys.", but doesn't say anything about how to store them securely and make them available to your apps.
So what options are available to me for storing the private key and making it available to my CF app while not storing it in my source control?
Note: I believe I do not want the cf ssh functionality, as that is for connecting in to an app (if I understand it correctly), not for an app connecting out.
Agreed - keeping your key in source control is bad practice.
Bluemix has a service called Key Protect that should provide exactly what you're looking for.
Docs here
Alternatively there are a number of other 3rd party providers you could use to do this.
I am currently doing some research for the development of a mobile application for our company that should support offline data sync (on an iPad). We have explored many possibilities including PhoneGap/Cordova, Xamarin and simply native iOS development. Xamarin, for many different reasons, seems to be our best choice, so my question will assume we will develop in Xamarin.
I was looking into a library for managing offline data synchronization and the most obvious solution is Microsoft Azure MobileServices. However, my company is Canadian, and apparently it's hard to trust (legally) our data to clouds based in the US. Since we already deployed internally our WebApi on our intranet, I figured there was probably a way to point the MobileServices library to our own WebApi. I have read about the Azure Hybrid Connection possibility, but our data still conveying through Microsoft servers might not be a possibility. So, my question is this:
Is there a way to configure the Microsoft.WindowsAzure.MobileServices Client library to point directly to our intranet, RESTful WebApi backend, without going through any Microsoft Azure servers ?
I understand that, in order to be able to use the Client librairies seamlessly, we probably would have to adapt our WebApi to implement the necessary .net Backend interfaces. I'm mostly wondering if it's even possible as the MSDN documentation on the libraries all seem to point to direct connections to their servers (no possibilities to configure your own connection strings) and all instructions redirect you to their Azure Mobile Services website.
Thank you.
If you look at the API for your mobile client, you'll notice that the Azure Mobile Services Client SDK only cares about two things:
new AzureMobileClient( url, appkey)
...where it's hosted shouldn't be a concern. Everything else is just configuration.
If you want to host the Azure Mobile Services Backend on your own servers, technically you could do this, but there are likely a few caveats. Microsoft has announced that they will be launching a Canadian Azure data center, but we won't see it until 2016.
In the meantime, here's how you can host the services locally. Note that I have not tried to emulate all of the features of Azure Mobile Services (aka Zumo) so your mileage (or kilometerage) will vary.
Hosting Locally:
From a technical feasibility, you absolutely can run the services locally. I know this because you can create the Azure Mobile Services Backend project from within Visual Studio and run it locally for development purposes. This is what our development team does for testing their mobile applications.
Note that you can create the Azure Mobile Service backend directly from within Visual Studio: New Project -> Cloud -> Azure Mobile Service. You can also download the exact same template (pre-configured with your URL and ApplicationKey) directly from the Azure dashboard: Create -> Mobile Service.
Obviously, if you're hosting it on your server it will be up to you to configure and use a proper SSL certificate for your site.
ZUMO Permissions:
By default, the security roles on the server are turned off. So if you're locking down any of your methods using the [AuthorizeLevel] attribute these settings will be ignored at runtime. If you need to enable this feature you can do so by modifying the WebApiConfig.Register() method and marking the site as self-hosted: config.SetSelfHosted(true).
Configuration:
From a configuration perspective, the Azure Mobile Service dashboard provides several tabs for configuring Identity, Push Notifications, Connection Strings and App Settings. Sadly, you won't have a dashboard, but all of these settings have a corresponding value in the local web.config. Any value you provide here is automatically overwritten in Azure, but they're used when running locally.
The minimum settings you'll need to configure are listed here. The ApplicationKey you can distribute with your ZuMo client, but the MasterKey is for the Admin authorization level so you'll want to keep that secret. The MobileServiceName is used by the EntityFramework for your database schema and what appears in the URL of your site.
<add key="MS_MobileServiceName" value="myzumosite" />
<add key="MS_MasterKey" value="masterkey" />
<add key="MS_ApplicationKey" value="appkey" />
Values that start with a MS_ prefix map to corresponding values in the Azure Portal. MS_GoogleClientID and MS_GoogleClientSecret map to the Google Identity values in the dashboard, for example.
Any other value in the AppSettings node is immediately accessible via the ApiServices.Settings property and corresponds to the Settings node in the Azure dashboard.
Database connection strings continue to exist in the connectionStrings node. The same is true for azure notification hub.
Database:
Obviously, the database you configure will be up to you as well. Permissions and User accounts are also obvious. There may be some minor differences between the SQL Azure syntax for Entity Framework database migration scripts that you'll need to worry about. (I've discovered the database migration scripts don't work from the Package Manager, but they do work when the database scripts are run when your website starts)
Caveats:
You will not have a nice dashboard for monitoring performance of your site, reviewing logs or changing runtime settings
You will not be able to scale out your site immediately; Scaling and deployment will be your problem
Deployment configuration is your responsibility (Project -> Publish won't be available unless you configure it)
Not sure if you'll be able to use Azure Active Directory as an authentication scheme, though from the sounds of it that won't be a concern. You can write your own authentication providers: Microsoft's Zumo library only supports a handful, but the underlying Owin.Security package that Microsoft uses supports several dozen systems!
Your site will need to be publically visible to your mobile clients
Push Notifications should work, but you will be using Azure's notification hub for this.
I have no idea where ApiServices.Log will go
The easiest path to take would be to:
Create the Mobile Service in Azure to get the notification hub and settings preconfigured
Download the starter site from the dashboard
Configure the web.config as mentioned here.
It's not possible to simply configure WAMS Client library to work with your own WebApi Backend.
But WAMS library is available at github, so I'm sure you can reuse a lot of code from the WAMS project, especially if you want to use a PCL project.
To route your data securly through Azure, you could think about setting up express route. Additionally, for last weeks update, it's possible to apply a custom domain to the WAMS Backend, including your own certificate to secure your connection.
I'm starting to develop an application using node.js on azure. I'm using everyauth to provide authentication because I want to support lots of different authentication methods. I plan to deploy to Azure. The potential problem I have is that everyauth requires the connect.session helper. Will this work with azure when running multiple instances? Or do I need an alternative session provider?
I have never used Node.js on Azure, but:
everyauth
Looking at the documentation for everyauth there is a method for authenticating against a Windows Azure ACS. See the section entitled Setting up Windows Azure Access Control Service (ACS) Auth in the readme for more information. There are no notes there about it not working on Azure itself so I would infer from that that you can use it on Azure.
connect-azure
There is also a project called connect-azure, which appears to be using connect.session so again I would extrapolate from this that it will work on Azure.
Contact Azure support
If you are already a customer you can contact support for help.
Try it and see
So if you have the Azure environment setup I would definitely say it is worth trying it out.
This was asked a while ago, but I thought I would attempt an answer anyway. It seems that connect-session relies on cookies to maintain the session. Azure has a different load-balancing strategy depending on what you use:
WebRole/WorkerRole - the LB doesn't have any affinity so requests from your clients might end up at different backend instances. This will throw off whatever session management connect is doing. This is a side effect of a distributed cloud architecture: you don't want any backend node to be the source of truth since it can go down. So what you would have to do is figure out how to externalize connect's cookie store and have all backends share it. This way no matter what backend receives the request, it will be aware of the session.
Websites - in this case the LB will actually try to pin a client connection to a given backend instance, so cookie-based sessions may work without any changes. You are sacrificing failover, as described above.