I wanted to create noodpool with swap memory enabled in AKS, I have gone through
Terraform documentation there I can see swap_file_size_mb and vm_swappiness are the only thing related to swap. My question is
is there any way to use this flag --fail-swap-on
to false(or it will automatically set to false when we set swap_file_size_mb)
And is there any way to change MemorySwap.SwapBehavior to "UnlimitedSwap"
Are these things are possible in AKS, or Am I missing something, I want a working node
that has swap memory and should use for workload through terraform. Any suggestion appreciated. Thanks.
there is a kubelet_config block in AKS schema that allows settings failSwapOn: https://learn.microsoft.com/en-us/azure/aks/custom-node-configuration#virtual-memory
but that one is not exposed in terraform, but i think its hardcoded to false: https://github.com/hashicorp/terraform-provider-azurerm/blob/ddd6a9e2ef99f2e859b567badbed1aa829261caa/internal/services/containers/kubernetes_cluster_node_pool_resource.go#L1020
MemorySwap - i dont think so, at least I dont see it in the docs: https://learn.microsoft.com/en-us/azure/aks/custom-node-configuration#virtual-memory
Related
I'm looking at setting up a cluster of GPU nodes on AKS.
In order to avoid installing the nvidia device daemonset manually, apparently I can register for the GPUDedicatedVHDPreview and send UseGPUDedicatedVHD=true with AKS custom headers (https://learn.microsoft.com/en-us/azure/aks/gpu-cluster).
I understand how to do this on the command line, but I don't understand how I can do it using the azurerm_kubernetes_cluster terraform provider.
Is it even possible?
It looks like, at the time of writing, this isn't possible yet, as indicated by this open issue: https://github.com/terraform-providers/terraform-provider-azurerm/issues/6793
I'm currently trying out Azure AKS and during setup I obviously also want to make backups. For this the best practice seems to be velero. According to the documentation of velero to include pv snapshots you would annotate the pod/deployment. Example:
backup.velero.io/backup-volumes: wp-pv
Note the above is when using a static managed disk. I can see the snapshot is created. However, when I do a restore a new pv is created instead of using the one from the restore. Is this expected behavior?
Ideally, I would like to use dynamic pv's instead but this would make it even more trivial because I don't know what name the pv will have and thus can't add proper annotations beforehand.
How can I solve this in a clean way? My ideal situation would be to have scheduled backups using velero and in case of a recovery automatically have it use the snapshot as base for the pv instead of it creating a new one that doesn't contain my data. For now, it seems this is a manual procedure? Am I missing something?
This is by design.
PersistantVolumes by definitions can only ever belong to one PVC claimant. Even when set as dynamic.
I think what you want is to have the reclaim policy set to retain. See here:
https://kubernetes.io/docs/concepts/storage/persistent-volumes/#retain
A state of "Retain" should mean that the PVs data persists, it is just needing to be reclaimed by a new PV/PVC. The AKS should pick up on this... But I've only ever done this with AWS/Baremetal
In this case Velero, rightly, has to both recreate the PVC and PV for the volume to be released and reassigned to the new claimant.
I am new to terraform and have been playing with it for a while. In the scenario I am trying to execute, I am trying to create a stack for each deployment of mine. I am able to create the resources I need. However, when I want to deploy a new stack, it is trying to destroy the already existing stack. I have tried to use the lifecycle key with prevent destroy, but that throws an error saying the resource cannot be deleted.
the plan would destroy this resource, but it currently has lifecycle.prevent_destroy set to true. To avoid this error and continue with the plan, either disable lifecycle.prevent_destroy or adjust the scope of the plan using the -target flag.
Is there no way to skip or make terraform "forget" the resource it created? I understand this is tricky because of the state file that terraform maintains, but any leads would help.
Thanks in advance.
You can remove resources from the state using terraform state rm <resource>, maybe that's what you are looking for?
More information available at https://www.terraform.io/docs/commands/state/rm.html.
I think you need to use terraform workspaces for different deployments (environments ?). So you will be able to create different resources for different envs (this is mentioned in this comment Terraform forget resource and recreate?)
Information: https://www.terraform.io/docs/state/workspaces.html
I would like to add additional name servers to kube-dns in the kube-system namespace (solution provided here: https://stackoverflow.com/a/34085756/2461761), however doing this in an automated manner.
So I know I can create my own dns addon via https://github.com/kubernetes/kubernetes/tree/master/cluster/addons/dns and launch it in the system's namespace, however, I am trying to provide a simple script to developers to spin up their own clusters with DNS resolution built in and don't want them to concern with the system namespace.
Is there a way to modify/set the SKYDNS_NAMESERVERS environment variable without having to making a copy of the replication controller?
Or even set it via a command and re-launch the pods of the kube-dns deployment?
Thank you in advance.
I still think that "adding SKYDNS_NAMESERVERS to the manifest file" solution is a good choice.
Suppose the developers still need to spin up the cluster, it would be better to set up the upstream DNS servers ahead through the manifest file instead of changing them on the fly. Or is there any requirement that need this to be done after the cluster is up?
If this has to be done while everything is running, one way to do so is to modify the manifest file locates on the master node. For current version kubernetes(1.4), you will also need to modify the ReplicationController name to a new one and the Addon Manager will then update the resources for you. But notice that there would be kube-dns outage(probably seconds) in between because current Addon Manager executes the update in the delete->create manner.
What is the best way to set yaml settings? I am using docker containers and want to automate the process of setting cassandra.yaml settings like seeds, listen_address & rpc_address.
I have seen something like this in other yaml tools: <%= ENV['envsomething'] %>
Thanks in advance
I don't know about the "best" way but when I set up a scripted cluster of cassandra servers on a few vagrant vms I used puppet to set the seed and so on in cassandra.yaml.
I did write some scripting than used puppetdb to keep track of the addresses of the hosts but this wasn't terrifically successful. The trouble was the node that came up first only had itself in the list of seeds and so tended to make a cluster on it's own. Then the rest would come up as a seperate cluster. So I had to take down the solo node, clear it out and restart it with correct config
If I did it now I would set the addresses as static ip, then use them to fill in the templates for the cassandra.yaml files on all the nodes. Then hopefully the nodes would come up with the right idea about the other cluster members.
I don't have any experience with Docker but they do say the way to use puppet+Docker is to use puppet on the Docker container before starting it up
Please note that you need a lot of memory to make this work. I had a machine with 16GB and that was a bit dubious.
Thank you for information.
I was considering using https://github.com/go-yaml/yaml
But this guy did the trick: https://github.com/abh1nav/docker-cassandra
Thanks
If you're running Cassandra in Docker use this as an example: https://github.com/tobert/cassandra-docker You can override cluster name/seeds when launching so whatever config management tool you use for deploying your containers could do something similar.