I have a number of SSIS packages that are launched by Windows Services. These packages point to any number of different databases, and the connection information is known only by the service at run-time.
I know that I can't save connection strings inside the package, but it seems that I can't even pass in a complete "connection string" as a variable; when I do, the package errors out and it appears that the password is removed.
What I can do, however, is pass in the parts of the connection string that I need and then re-assemble them into a working connection string. I do NOT want to do this; it seems sloppy and ill-advised.
Anyone have any thoughts on how to accomplish this?
You can save XML package configuration files. They too will strip out the passwords, but you can add them in by hand. I just keep them on a secure location on the servers, since they are XML, and the passwords are not encrypted. Anyone who can get to XML file can get to the database on that server anyway.
When you're creating a job to run the package, you can specify the config file to use and its location. If you're developing, the config file is on your computer. Once it's on the server, have the config file put in a secure location on that same server.
The proper deployment and configuration of SSIS packages isn't as easy as it first seems. There's a really good walk-through on MSDN about the right way to do it (including connection string configuration and password storage):
Depoloying Integration Services Packages
...in the end you end up with a deployment package that you install on the server, and then use a combination of XML Configuration files and Environment variables to configure the package.
You can pass data in to SSIS packages as variables, and the values in those variables can be "passed on" to virtually any property or attribute within the package.
One way I've done it is to pass in a servername as a variable, and then for the requisite connection I used Expressions to assign the variable to the connection's ServerName property. I can't (quickly) find a website that describes in detail how to do this, but I learned it from going through the SSIS tutorials in Books Online. (And, looking at a sample package, it appears that yes, you could pass in an entire connection string and assign it to a connection via expressions.)
This is great stuff if you need a fine level of control over what goes into your packages. XML configuration files are effective if your requirements are more straightforward. And definitely be paranoid about storing passwords in C:\Login.txt files -- if you at all can, use NT authentication.
Related
I am using mongoose with MongoDB in a NodeJS application. Right now, in development, I have a configuration (.env) file which stores sensitive information my code needs to run. For example, the MongoDB password & URL, emails & passwords needed to email using the code, etc.
When I put it into production, it would obviously be wrong to upload this configuration file anywhere on the cloud, given the information in it. How can I make it so my production code, hosted somewhere such as Heroku, can access these needed variables without letting undue access to them?
Thanks in advance!
You are right, pushing your env file to production is pretty bad from a security perspective.
The way you would go with storing your environmental variables differs between cloud platforms, but essentially you should get a secure way of adding them through either an user interface or through terminal (You usually find these information easily by looking into your provider documentation).
To store them in a project deployed on Heroku, you will need to:
Log to Heroku
Open the newly deployed project
Head over the Settings tab
Find the section named Config Vars
Click on Reveal Vars
Add your variables in there
And you are good to go!
I've started working on an Azure project. In terms of config, I currently have three files: ServiceConfiguration.Cloud.cscfg, ServiceConfiguration.Local.cscfg and ServiceDefinition.csdef.
ServiceDefinition.csdef is the template file for the csfg files. ServiceConfiguration.Cloud.cscfg contains all the actual Azure configuration, including DB passwords, SAS keys etc.
Should ServiceConfiguration.Cloud.cscfg be checked into source control? I wouldn't have thought so but a quick search on github for the file shows that it is.
If it should be checked in, how should the sensitive password data be managed?
I typically check in the configurations. The reason is that the behavior of your application will change dramatically depending on these configurations. For example -> number of roles for a distributed application directly affects how you process incoming messages and the vmsize directly affects how much memory you have. You may encounter issues debugging problems if each developer is using a different configuration. This standardizes your deployment.
Anything with plain-text password information shouldn't be checked into a public repo unless you want people to have access to that information.
You can add this file to the .gitignore file and prevent it from being checked in.
Provide a different ServiceConfiguration.Cloud.cscfg named something like ServiceConfiguration.Cloud.cscfg.template with all the config info of your cloud service minus the password values. If someone forks your project they need to use that and fill in the appropriate values and rename the file.
Do this and change all your passwords to something else. Even if you delete this file from the repo, it still exists in the history and anyone can view it.
I'm trying out a yesod applications which I will eventually put up on github or similar.
I will use oauth2 with google which means I have to provide an email and secret token. Which I obviously do not want up on github.
What is a good place to store these in a yesod scaffolded application? I'm hoping to store it in a seperate, config/secret.yml for example, so I can put that into the ignore file of git/mercurial and never commit it.
But i can't find out how to include such a file. Or if such a file already is provided by yesod. config/settings.yml seemed possible, but there's entries there which I would like in github.
So my question is, in a yesod scaffolded application. Where can I store secret keys in a way I can easily exclude it from version control systems?
There are many approaches to this, mostly depending on what flavor of devops/hosting your prefer. One option is to put a dummy value in the config file and override it with an environment variable at runtime (see: https://github.com/yesodweb/yesod/wiki/Configuration#overriding-configuration-values-with-environment-variables). You can also having an extra settings file for production that overrides the values in the default config file, which is how the test suite works. A completely different approach would be to use a system like vault in production and query it for your secure credentials.
EDIT To spell out one of the approaches:
Create a new YAML file with the settings you won't to override, e.g. in config/production.yml:
copyright: This is a copyright notice for production
When you run the application, pass in a command line argument giving the location of the config file
What are the best steps to take to prevent bugs and/or data loss in moving servers?
EDIT: Solved, but I should specify I mean in the typical shared hosting environment e.g. DreamHost or GoDaddy.
Bootstrap config is the smartest method (Newism has a free bootstrap config module). I think it works best on fresh installs myself, but ymmv.
If you've been given an existing EE system and need to move it, there are a few simple tools that can help:
REElocate: all the EE 2.x path and config options, in one place. Swap one URL for another in setup, check what's being set and push the button.
Greenery: Again, one module to rule them all. I've not used this but it's got a good rating.
So install, set permissions, move files and and DB, and then use either free module. If you find that not all the images or CSS instantly comes back online, check your template base paths (in template prefs) and permissions.
I'm also presuming you have access to the old DB. If not, and you can't add something simple like PHPMyAdmin to back it up, try:
Backup Pro(ish): A free backup module for files and db. Easy enough that you should introduce it to the site users (most never consider backups). All done through the EE CP. The zipped output can easily be moved to the new server.
The EE User Guide offers a reasonably extensive guide to Moving ExpressionEngine to Another Server and if you follow all of these steps then you will have everything you need to try again if any bugs or data loss occur.
Verify Server Compatibility
Synchronize Templates
Back-up Database and Files
Prepare the New Database
Copy Files and Folders
Verify File Permissions
Update database.php
Verify index.php and admin.php
Log In and Update Paths
Clear Caches
As suggested by Bitmanic, a dynamic config.php file helps with moving environments tremendously. Check out Leevi Graham's Config Bootstrap for a quick and simple solution. This is helpful for dev/staging/prod environments too!
I'd say the answer is the same as any other system -- export your entire database, and download all of your files (both system and anything uploaded by users - images, etc). Then, mirror this process by importing/uploading to the new server.
Before I run my export, I like to use the Deeploy Helper module to change all of my file paths in EE to the new server's settings.
Preventing data loss primarily revolves around the database and upload directories.
Does your website allow users to interact with the database? If so at some point you'll need to turn off EE to prevent DB changes. If not that you don't have too much to worry about as you can track and changes on the database end between the old and new servers.
Both Philip and Derek offer good advice for migrating EE. I've also found that having a bootstrap config file helps tremendously - especially since you can configure your file upload directories directly via config values now (as of EE2.4, I think).
For related information, please check out the answers to this similar Stack Overflow question.
The majority of our C# projects configuration is kept in *.ini files. Mainly these files hold
many sections affecting all aspects of programs behaviour. But besides of regular configuration data some of sections are vulnerable like db connection string or server password. We try to keep this sections in following forms:
[Database]
user=testuser
database=testdb
password=
But when developer is testing application he must fill the config in order to start application. It is quite common that some of the passwords are commited into version control.
Because these files are indispensable for application they cannot be included in .svnignore.
Probably what I'm looking for is some kind of script (maybe in powershell). That would scan all *.ini files and erase all passwords. The most interesting solution would be adding some external password storage that can be used both to encode and decode passwords in *.ini files.
I always push to store configuration template files in subversion, but not actual configuration files. So if the configuration file is "config.ini" then I'll check in a "config.ini.template" populated with non-working sample data.
Then to prevent multiple developers from checking in their individual "config.ini" files, I'll add the actual configation file name to the svn:ignore properties list.
This forces the developer to copy the file and modify it appropriately for their environment, but eases the work of that task by not forcing them to find out which fields need to be present. If you have the time, you can even embed comments into the template file to simplify the meanings of some of the configuration options.
At the top of the file, include the directions of how to configure the system using the template, which should read something like:
# *** CONFIGURATION TEMPLATE --- DO NOT MODIFY THIS FILE ***
# 1. Make a copy of this file in the same directory with the command "copy config.ini.template config.ini"
# 2. Edit the new copy and follow the rest of the instructions
#
# Change "this.system.hostname" to the hostname of this system
Hostname = this.system.hostname
# Set the answer "23" to "42"
Answer = 23
You get the idea....
If you have problems (or think you might have problems) with people checking in their configuration options over the config.ini.template file, then I'd recommend using "svn lock" on the template file. However, with the appropriate warning, I've never found it necessary.
I'll not answer your question and instead recommend a different approach, assuming it's not too late to change the relevant design.
You should not store passwords in the same files as the rest. Have the application read a dedicated password file (or retrieve the password from a password storage service) in addition to the regular configuration file. This is not just about not storing passwords in svn, but also about not having passwords exposed to shoulder surfing, accidentally mailed or posted when someone asks for help with a non-working configuration, etc.