problem statement.
as per my understanding, we can run an elastic search, kibana and logstash etc as a pod in kubernates cluster for log management. but it is also memory heavy intensive application. AWS provides various managed services like Cloudwatch, cloud trail and ELK stack for log management.
do we have a similar substitute in Azure as well i.e. some managed service?
you can use AKS with Azure Monitor (reading). I'm not sure you can apply this to not AKS cluster (at least not in a straight forward fashion).
Onboarding (for AKS clusters) is really simple and can be done using various methods (portal included).
You can read more on the docs I've linked (for example, about capabilities).
Azure Monitor for Containers is available now and once integrated some cluster metrics as well as the logs will be automatically collected and made available through log analytics.
Related
Given that I have a 24x7 AKS Cluster on AZURE, for which afaik Kubernetes cannot stop/pause a pod and then resume it standardly,
with, in my case, a small Container in a Pod, and for that Pod it can be sidelined via --replicas=0,
then, how can I, on-demand, best kick off a LINIX script packaged in that Pod/Container which may be not running,
from an AZURE Web App?
I thought using ssh should work, after first upscaling the pod to 1 replica. Is this correct?
I am curious if there are simple http calls in AZURE to do this. I see CLI and Powershell to start/stop AKS cluster, but that is different of course.
You can interact remotely with AKS by different methods. The key here is to use the control plane API to deploy your kubernetes resource programmatically (https://kubernetes.io/docs/concepts/overview/kubernetes-api/) .
In order to do that, you should use client libraries that enable that kind of access. Many examples can be found here for different programming languages:
https://github.com/kubernetes-client
ssh is not really recommended since that is sort of a god access to the cluster and its usage is not meant for your purpose.
I want to find the Node scalability time on Azure Kubernetes Service (AKS) using Logs.
It's possible with some assumptions.
This information is taken from Azure AKS documentation (consider getting familiar with it, it describes how to enable, where to look at and etc):
To diagnose and debug autoscaler events, logs and status can be
retrieved from the autoscaler add-on.
AKS manages the cluster autoscaler on your behalf and runs it in the
managed control plane. You can enable control plane node to see the
logs and operations from CA (cluster autoscaler).
The same cluster-autoscaler is used across different platforms, each of them can have some specific setup (e.g. for Azure AKS). Based on it, logs should have events like:
status, scaleUp, scaleDown, eventResult
We are designing an application which will be hosted on AKS(Azure kubernetes service). The application will consist of a set of services written in asp .net core running in docker containers. I want to monitor the services as well as the containers/nodes and have the observability across the cluster. Azure monitor for containers seems to be a good solution for monitoring containers, nodes and the cluster as a whole however I want the advanced monitoring capabilities of the application insights for the asp .net core services for example application maps, live metrics streams, transaction tracing and such features. Moreover, I don't want to have overlapping solutions. Is the Azure monitor for containers able to provide all or most of these application insights features or do I have to have both solutions in order to get proper cluster monitoring and also the advanced application monitoring?
Azure Monitor for containers provide infrastructure level monitoring and basic application logs with stdout and stderr, Kubernetes events captured out of the box.
It does not provide instrumentation for your apps or distributed tracing capabilities today, which is possible with Application Insights.
If you are looking for application map & instrumentation for events metrics and logs for your app, you can use both together and it's possible to correlate data from both and create dashboards and views.
The long term road map has Azure Monitor for containers & Application insights combined offering
In this (https://learn.microsoft.com/en-us/azure/azure-monitor/insights/container-insights-overview) guide, you can find all the features of AKS monitoring. If these covers all your use cases then you don't need to install any other tool. If not, then you cover only those features which are missing.
From whatever i read, i could not find a way to connect to master node in Azure kubernetes Service. I have a requirement to read some parameters like 'enable-admission-plugins' which is possible from master node. Is there any third party api available to get this info.
More specific i need to read the files 'kube-apiserver.yaml', 'kube-controller-manager.yaml'
No, this is not possible. Masters are managed by Microsoft and you dont have access to them. All the configurations are to be done through the AKS api (mostly when you create it).
Azure Kubernetes Service (AKS) makes it simple to deploy a managed
Kubernetes cluster in Azure. AKS reduces the complexity and
operational overhead of managing Kubernetes by offloading much of that
responsibility to Azure. As a hosted Kubernetes service, Azure handles
critical tasks like health monitoring and maintenance for you. The
Kubernetes masters are managed by Azure. You only manage and maintain
the agent nodes.
Does Azure Container Service integrate with Azure Monitor?
Wondering what the best way is to do logging/monitoring of kubernetes cluster?
If you are looking for monitoring tools on Azure, you may want to use Azure OMS (Opertation Management Suite). This gives you the ability to monitor the container inventory, performance, and logs in a single location. To my understanding, the stats of the container is only available for Linux nodes now, if you are deploying your k8s cluster on the Azure Portal.
To do this, you need to first create an OMS account. By this time, you should have the Workspace ID and the key available. The next step would
be to create the oms pod on each node using a DaemonSet.
For the detailed setup, take a look at https://learn.microsoft.com/en-us/azure/container-service/kubernetes/container-service-kubernetes-oms.
For third party tools, Grafana+influxdb is one of the ways I have tried before. Basically it provides you with the metrics on two levels: POD and NODE respectively. The displayed metrics included CPU Usage, Memory Usage, Network Usage and Filesystem Usage, etc. Of course, you can always alter your query to add extra metrics.
For the implementation of this approach, you can refer to https://github.com/Azure/acs-engine/blob/master/docs/kubernetes/monitoring.md.
Hope this helps :)
you can use this CLI command to browse through kubernetes cluster deployed using azure container service.
az acs kubernetes browse -g -n
This way you can see kubernetes webui
also you can use kubectl proxy command.