Find logs of POD in AKS using Log Analytics Query - azure

Question

There is a AKS running that is connected to Log Analytics in Azure.
I'm trying to view logs of named PODs using the following query snippet:
let KubePodLogs = (clustername:string, podnameprefix:string) {
let ContainerIdList = KubePodInventory
| where ClusterName =~ clustername
| where Name startswith strcat(podnameprefix, "-")
| where strlen(ContainerID)>0
| distinct ContainerID;
ContainerLog
| where ContainerID in (ContainerIdList)
| join (KubePodInventory | project ContainerID, Name, PodLabel, Namespace, Computer) on ContainerID
| project TimeGenerated, Node=Computer, Namespace, PodName=Name1, PodLabel, ContainerID, LogEntry
};
KubePodLogs('aks-my-cluster', 'my-service') | order by TimeGenerated desc
The above query does return rows of the matching PODs but not all that are actually available.
Trying to get results of the partial queries by inspecting POD details:
KubePodInventory
| where ClusterName =~ 'aks-my-cluster'
| where Name startswith 'my-service-'
| where strlen(ContainerID)>0
| distinct ContainerID;
gives me a container-id. Now feeding this container-id into another query shows more
results then the combined query from above. Why ?
ContainerLog
| where ContainerID == "aec001...fc31"
| order by TimeGenerated desc
| project TimeGenerated, ContainerID, LogEntry
One thing I noticed is that the later simple query result contain log results that have a LogEntry field parsed from JSON formatted output of the POD. In the results I can expand LogEntryto more fields corresponding to the original JSON data of that POD log output.
I.e. it seems like the combined query ( with a join ) skips those JSON LogEntry ContainerLog entries, but why ?
As far as I can see the combined query doesn't filter in any way on the LogEntry field.

Answer 1

A changed query seems to produce the results I would expect:
I exchanged the join with a lookup and used more columns to distinct the KubePodInventory results.
let KubePodLogs = (clustername:string, podnameprefix:string) {
let ContainerIdList = KubePodInventory
| where ClusterName =~ clustername
| where Name startswith strcat(podnameprefix, "-")
| where strlen(ContainerID)>0
| distinct ContainerID, PodLabel, Namespace, PodIp, Name;
ContainerLog
| where ContainerID in (ContainerIdList)
| lookup kind=leftouter (ContainerIdList) on ContainerID
| project-away Image, ImageTag, Repository, Name, TimeOfCommand
| project-rename PodName=Name1
};
KubePodLogs('aks-my-cluster', 'my-service') | order by TimeGenerated desc