I want to run several docker containers in different regions (asia, eu, us) which host a nginx server.
However, they should all have the same configuration because I need to updated hostnames at runtime dynamically (one domain for every new tenant).
So I guess it would be the easiest way to just share one config file among all containers and reload them...
So how can I share data/files among n containers on azure?
In general, unless you want to use proprietary solutions specific to the platform at hand, the best way to synchronise files between multiple systems is with the help of rsync.
For example, in DNS, there exists a specialised protocol for transferring domain zones directly within the DNS software, called AXFR; one of the authors of a newer DNS implementation suggests that this AXFR protocol is crap, and rsync over ssh works much better — http://cr.yp.to/djbdns/tcp.html — and the ssh part is a nice thing about rsync, in that it can work over plain old ssh protocol as far as interconnection between the hosts goes, not requiring any special firewall considerations.
Have you considered using the Azure file share.
Related
I see that a number of people have set up Docker containers with Guacamole or other tools to allow them to remote in to GUI as if the container was a remote Linux desktop. A friend of mine had a conversation with a professor who told him that they set up Ubuntu desktop access for their students via ubuntu/rdp docker containers.
It's an attractive concept for efficiently packed cloned desktops since you don't need 50 copies of the guest OS, but how would you manage such a swarm without a connection broker like a VDI solution or a hypervizer console like a KVM setup? Would you simply use standard docker (or swarm) management tools to manage the containers themselves, then some separate remote client for the actual remote control connections?
I'm currently reading up on Docker, but unclear: If each desktop is the same, so say Firefox, LibreOffice, etc. Is there any way to gain efficiency by sharing these resources as well? For instance, could there be a container with those resources that the others all connect to... or have it shared on a lower level like the OS? Looking for any way to gain efficiency, lower overall cpu, ram, etc for all combined machines on server. Really looking for anything other than a separate copy of the same thing in each container.
I see that there are solutions for shared persistent storage in containers like Hatchway. Are there other issues caused by statelessness of the container that this does not address?
Also, I see a few ways people have cobbled together internet connectivity for docker containers (like IP per container), but most of the older posts are people frustrated with the process. Is there now a standard or preferred way to do something like this?
Or, if docker/containers are absolutely the wrong way to go about setting up the most efficient possible Linux remote desktop clones, I'd love to understand exactly what part does not work so I can find the right way.
I see after days of reading that LXD is actually what I'm looking for (Linux machine containers) instead of Docker (process containers).
I'm trying to set up a situation where I drop files into a folder on one Azure VM, and they're automatically copied to another Azure VM. I was thinking about mapping a drive from the receiver to the sender and using a file watch/copy program to send the files over the mapped drive.
What's a good recommendation for a file watch/copy program that's simple and efficient, and what security setups do I need to get the two Azure boxes to "talk" to each other? They're in the same account/resource group/etc, so I'm not going outside of a virtual network or anything like that.
By default, VMs in the same virtual network can talk to each other (this is true even if default NSGs are applied). So you wouldn't have to do anything special to get that type of communication working.
To answer the second part, you might want to consider just using built-in FCI rules to execute a short script to do the copy. See this link for a short intro into FCI rules.
Alternatively, you could use a service such as Azure files to have files shared between those servers using CIFS. It really depends on why you are trying to have a copy of the file on two servers.
Hope that helps!
I currently run a Red Hat Linux server with Plesk to host a hundred or so domains. For multiple reasons I'd like to transition away from Plesk and to Docker containers with each virtual host as one or more containers. I'm unclear from what I've read so far what would be the best approach to this.
A typical site includes the doc root file area and one or two MySQL databases. We run PHP on all the sites. Some sites may have constraints on the version of PHP they can run. Some of the sites use SSL. I don't believe there are any constraints on the MySQL versions, but it's of course possible that future MySQL versions could deprecate some feature that is needed. I don't believe there's any dependency on the Apache version, but I do rely on some specific Apache modules being installed. There may be a site or two that have dependencies outside of their doc root and not part of the basic virtual host setup, but I don't believe any require a specific version of Linux.
I would like the containers to have maximum portability so that I can have flexibility in moving sites to whatever server or cloud service I choose. Part of my goal is to retire the current server and move sites to servers which best fit them.
I would also like to try upgrading the PHP version after the containers are created.
So would a single container include the entire doc root file system, including the data directories where users can upload/ftp files? Would it include the MySQL database, or would that be separate? I assume I would include the current version of PHP so that I could upgrade each one when I was ready. Would it include Apache when specific Apache modules are required? Is there a reason to include Apache and/or MySQL in all containers?
One last piece. I'm looking into using CoreOS which utilizes Docker as an integral part.
Any and all inputs are appreciated.
The whole idea of Docker is running processes/components isolated, to keep them easily upgradable. I have tinkered with this in the past and have come up with the following.
Create four containers per instance (customer):
Apache or nginx
php-fpm
MySQL
Busybox (as a data container)
Link all of them together and set volumes to all data that should persist in the data container. MySQL data and /var/www plus site config files for example.
This way you can always switch out one of the components while keeping the others. It's questionable though if Docker is a solution to a full virtual server though, as Docker containers do not have a full init system and you'll have to resort to bending things quite a bit to resemble a full virtual machine. Think more of it as "application containers", hence the idea with the separation of concerns.
Update:
Newer Docker versions come with the docker-compose tool which greatly eases this task.
I am trying to solve the same issues with cPanel instead of Plesk.
We can try and accomplish this using the plugins for cpanel or plesk however we have to worry about few things.
and we have to create some premade template images for containers that our clients can use.. it cannot be just any container from dockerhub,user Dockerfiles,etc Because cPanel/Plesk will look for specific log files available on specifc locations for bw calculations, disk quota,etc.
Biggest advantage with this solution is that we can provide CloudLinux kind of isolation and easy resource allocation/ fair sharing. However it is not as easy.
To answer your question:
Every container will be nearly a complete system so you will need to have less clients per host, because each container might be like 1G and by default have to run its own webserver/php and hence more ram foot print.
Its painful to run a Mysql inside each container and it is better to use mysql on the host or 1 dedicated container and share it. this way the Plesk's tools will help.
You may also have to use the standard apache and then reverse proxy it to each container after ssl termination so Plesk's standard tools are used but then I think containers will have to run its own webserver itself or we may have to do some trickery with php-fpm to allow host's apache to talk to each container's php-fpm processes . This is more painful than allowing each container to just run its own Nginx but possible.
It doesnt prevent users from installing their own Mysql server within their container if they need.
This kind of stuff is easy for someone from cPanel or Plesk to do.. but for others it will need a lot of Dedicated development time + testing to make sure all this works.
I was going to invest some time in creating this kind of plugin for cPanel but still undediced. I may try this if I can rope in some investors.
You can see amount of interest , CPanel shows on this issue : http://features.cpanel.net/responses/dockerio-support
I will leave you to decide
Also as an alternative solution:
so Instead of playing to the Cpanel's tune I created this . https://github.com/paimpozhil/WhatPanel
Here every site runs in its own container ( and its own VM if needed.).
Migration is simple as exporting/importing a container with tools like : on github.com /paimpozhil/docker-volume-backup & acaranta/docker-backuper
I didnt complete the migrator/ php upgrade tools ,etc here but will do when i have free time.
I'm looking to launch a linux EC2 instance.
Although I understand linux quite well my ability to security/harden a linux OS would undoubtedly leave me vulnerable to attach. eg: there are others who know more about linux security than me.
I'm looking to just run Linux, Apache & PHP5.
Are there any recommended Amazon AMI's that would come pre-harden running linux/apache/php or something similar to this?
Any advice would be greatly appreciated.
thankyou
Here is an older article regarding this (I haven't read it, but it's probably a good place to start): http://media.amazonwebservices.com/Whitepaper_Security_Best_Practices_2010.pdf
I would recommend a few best practices off the top of my head
1) Move to VPC, and control inbound and outbound access.
2a) Disable password authentication in SSH & only allow SSH from known IP's
2b) If you cannot limit SSH access via IP (due to roaming etc) allow password authentication and use google authenticator to provide multi-factor authentication.
3) Put an elastic load balancer in front of all public facing websites, and disable access to those servers except from the ELB
4) Create a central logging server, that holds your logs in a different location in case of attack.
5) Change all system passwords every 3 months
6) Employ an IDS, as a simple place to start I would recommend tripwire.
7) check for updates regularly (you can employ a monitoring system like Nagios w/NRPE to do this on all your servers) If you're not a security professional you probably don't have time to be reading bugtraq all day, so use the services provided by your OS (CentOS/RHEL it's yum)
8) Periodically (every quarter) do an external vulnerability assessment. You can learn and use nessus yourself (for non-corporate use) or use a third party such as qualys.
If you're concerned and in doubt, contract a security professional for an audit. This shouldn't be to cost prohibitive and can give you some great insight.
Actually, you can always relaunch your server from pre-configured AMI, if something happened.
It can be done very easy with Auto Scaling, for example. Use SSH Without a Password. Adjust your Security Groups accordingly. Here's good article on Securing Your EC2 Instance.
You have to understand 2 things:
Tight security make life hard for attackers as well as for you...
Security is an on-going task.
having your server secure at specific point in time don't say anything about the future.
New exploits and patches published every day, and lot of "development" acts render security unstable.
Solution?
You might consider services like https://pagodabox.com/
Where you are getting specific PHP resources without having to manage Linux/Security and so...
Edit:
Just to empathize...
Running Production system, where you are responsible for the on going security of the site, force you to do much more than starting up with a secure instance!
Otherwise, your site will become much less secure as time passed by (and as more people will learn about it)
As I see it (for a real production site), you have 2 options:
Get a security expert (in house or freelance) that will check your site regularly and will apply needed patches and so.
Get hosting service that will manage the security aspect for you.
I pointed to one service like that, where you can put your PHP code in and they will take care of everything else for you.
I would check this type of service for every production site that don't have the ability to get real periodically security checkup/fixes
Security is a very complex field... do not underestimate the risks...
One of the things I like most about using Amazon is how quickly and easily I can restrict my attack surface. I've made a prioritized list here. Near the end it gets a bit advanced.
Launch in a VPC
Put your webserver behind a loadbalancer ELB or ALB (terminate SSL there too)
Only allow web traffic from your load balancer
Create a restrictive security group. The only things allowed into your host should be incoming traffic from the load balancer and ssh from your IP (or your dhcp subnet if your ISP does not offer a static address)
Enable automatic security updates
yum-cron (amazon linux)
or unattended-upgrades (ubuntu)
Harden ssh
disallow root login and default amazon accounts
disallow password login in favor of ssh keys
Lock down your aws root account with 2fa and a long password.
Create and use IAM credentials for day-to-day operations
If you have a data layer deploy encrypted RDS and put it in a private subnet
Explore connecting to RDS with IAM credentials (no more db password saved in a conf file)
Check out yubikey for 2fa ssh.
Advanced: For larger or more important deployments you might consider using something like ThreatStack. They can warn you of AWS misconfig (s3 bucket containing customer data open to the world?), security vulnerabilities in packages on your hosts. They also alert on signals of compromise and keep a command log which is useful for investigating security incidents.
I whant to make a web interface on a server that will manage a few dns servers on another servers.
How can i remotely manage a bind dns server programmaticly ?
I would like to add/edit/delete zones.
I see that there is rndc but that only allows reloading of zones and not adding/deleting.
I could NFS mount zones from dns servers and edit them but is there a better way?
If there isn't a hard requirement on writing something like this from scratch, why not simply use an already existing interface without having to reinvent the wheel? A simple Google search for the keywords bind dns web interface yields an entire list of good open source projects in the very first result link.
There is work at the IETF to define a standard for remote control of name servers based on the Netconf framework. See:
https://datatracker.ietf.org/doc/html/draft-ietf-dnsop-name-server-management-reqs
https://datatracker.ietf.org/doc/html/draft-dickinson-dnsop-nameserver-control-00.txt
The requirements include the ability to add/remove zones, etc.
You could set up something that does remote SSH commands? That may be a bit insecure, though, unless the server running the commands themselves is pre-authenticated with an SSH key, and that's the only way you can access the server.