Log rotation on IoT Edge Agent - azure

I am following Microsoft documentation and trying to set the logs for edge Agent through the container options. My deployment.template.json file is as follows:
{
"$schema-template": "2.0.0",
"modulesContent": {
"$edgeAgent": {
"properties.desired": {
"schemaVersion": "1.0",
"runtime": {
"type": "docker",
"settings": {
"minDockerVersion": "v1.25",
"loggingOptions": "",
"registryCredentials": {
"myRegistryName": {
"username": "$CONTAINER_REGISTRY_USERNAME",
"password": "$CONTAINER_REGISTRY_PASSWORD",
"address": "myRegistryAddress.azurecr.io"
}
}
}
},
"systemModules": {
"edgeAgent": {
"type": "docker",
"settings": {
"image": "mcr.microsoft.com/azureiotedge-agent:1.0",
"createOptions": {
"HostConfig": {
"LogConfig": {
"Type": "json-file",
"Config": {
"max-size": "10m",
"max-file": "3"
}
}
}
}
}
},
"edgeHub": {
"type": "docker",
"status": "running",
"restartPolicy": "always",
"settings": {
"image": "mcr.microsoft.com/azureiotedge-hub:1.0",
"createOptions": {
"HostConfig": {
"PortBindings": {
"5671/tcp": [
{
"HostPort": "5671"
}
],
"8883/tcp": [
{
"HostPort": "8883"
}
],
"443/tcp": [
{
"HostPort": "443"
}
]
},
"LogConfig": {
"Type": "json-file",
"Config": {
"max-size": "10m",
"max-file": "3"
}
}
}
}
}
}
},
"modules": {
"Module_Name": {
"version": "1.0",
"type": "docker",
"status": "running",
"restartPolicy": "always",
"settings": {
"image": "${MODULES.Module_Name}",
"createOptions": {
"HostConfig": {
"LogConfig": {
"Type": "json-file",
"Config": {
"max-size": "10m",
"max-file": "3"
}
}
}
}
}
}
}
}
},
"$edgeHub": {
"properties.desired": {
"schemaVersion": "1.0",
"routes": {
"route": "FROM /messages/* INTO $upstream"
},
"storeAndForwardConfiguration": {
"timeToLiveSecs": 7200
}
}
}
}
}
When I build and deploy it on my device, edge Hub and my module log rotation are applied but edgeAgent log rotation is not applied. I check the log rotation settings in the
/var/lib/docker/containers/{container_id}/hostconfig.json file.
What I have done so far:
Removed the image through sudo docker rmi sudo docker rmi mcr.microsoft.com/azureiotedge-agent:1.0 and removed all the containers including the edgeAgent container and restarted the edge environment by sudo systemctl restart iotedge. The log rotation is still not applied to the new container created by the edge run time. I am not sure, what am I missing ? Any help is appreciated. Please note, I don't want to apply log rotation by creating a daemon.json file and placing it in the edge run time folder. I need to do it through the container options specified in the deployment.template.json file.

This is a known bug where edge agent deployment does not apply if version number is identical to the one in config.yaml; please help to create a github issue for it. To workaround, please set the options in config.yaml.

Related

Mongo client in Node doesn't receive connection response

I currently have a set of nodejs modules (containers) that connect to a local mongodb (also a container).
All modules works perfectly but with one of the I have the following problem:
https://i.stack.imgur.com/xuFqw.png
The connection seems to be accepted by mongo but the client doesn't receive the response and keeps waiting until it times out after 30 second (watch timestamps in the log)
The code of the client is the following:
async function connectToMongo() {
try {
const mongoUrl = 'mongodb://mongo:27017/LocalDB'
let result = await MongoClient.connect(mongoUrl, { useUnifiedTopology: true })
logger.info("Connected to Mongo DB")
return result
} catch (err) {
logger.error("Cannot connect to Mongo - " + err)
throw err
}
}
And is exactly the same as the other modules (that works).
Mongo is started without any settings or authentication.
I have tried using the following hostnames:
mongo : name of the container in which mongo is running
host.docker.internal : to address the host since the 27017 port of the container is binded to the host port
I honestly don't know what to try anymore...does anybody have an idea about what could be the problem?
UPDATE: as requested i'm adding info about how i'm starting the containers. Anyway I have omitted this part because they are started as Azure IoT Edge Modules. Anyway here is the infos from the deployment template:
"modules": {
"mongo": {
"settings": {
"image": "mongodb:0.0.1-replset-noauth",
"createOptions": "{\"HostConfig\":
{\"PortBindings\":{\"27017/tcp\":[{\"HostPort\":\"27017\"}]}}}"
},
"type": "docker",
"version": "1.0",
"status": "running",
"restartPolicy": "always"
},
"healthcheck": {
"settings": {
"image": "${MODULES.HealthCheck.debug}",
"createOptions": {
"HostConfig": {
"PortBindings": {
"9229/tcp": [
{
"HostPort": "9233"
}
]
}
}
}
},
"type": "docker",
"version": "1.0",
"env": {
"MONGO_CONTAINER_NAME": {
"value": "mongo"
},
"MONGO_SERVER_PORT": {
"value": "27017"
},
"DB_NAME": {
"value": "LocalDB"
}
},
"status": "running",
"restartPolicy": "always"
}
},
While this is the configuration of another module that perfectly works:
"shovel": {
"settings": {
"image": "${MODULES.Shovel.debug}",
"createOptions": {
"HostConfig": {
"PortBindings": {
"9229/tcp": [
{
"HostPort": "9229"
}
]
}
}
}
},
"type": "docker",
"version": "1.0",
"env": {
"MONGO_CONTAINER_NAME": {
"value": "mongo"
},
"MONGO_SERVER_PORT": {
"value": "27017"
},
"DB_NAME": {
"value": "LocalDB"
}
},
"status": "running",
"restartPolicy": "always"
},
(it's basically the same)
The modules are currently being launched using the IoT Simulator or on an IoT Edge device as Single Device deployments.

HTTP POST request to external gateway from Azure runtime module

I am working on Azure edge module where I created a custom module that actually received a message from another module within edge runtime.
My requirement is to send data to an external gateway(example thingsboard) over the HTTP post method. In docker file, expose 80 port and bind with local machine 80 port.
The problem is whenever I tried to send message from a custom module to external gateway(thingsboards) using the HTTP post method, getting an error of HTTP 500.
If I run the container directly(not under the edge runtime) is working fine.
As my code in container want to send message to external world(external device or gateway) over internet. I am sending message to thingsboard (http://demo.thingsboard.io/api/v1/camera/telemetry).
Edge deployment manifest:
{
"modulesContent": {
"$edgeAgent": {
"properties.desired": {
"schemaVersion": "1.0",
"runtime": {
"type": "docker",
"settings": {
"minDockerVersion": "v1.25",
"loggingOptions": "",
"registryCredentials": {
"cameraedgedev": {
"username": "username",
"password": "password",
"address": "cameraedgedev.azurecr.io"
}
}
}
},
"systemModules": {
"edgeAgent": {
"type": "docker",
"settings": {
"image": "mcr.microsoft.com/azureiotedge-agent:1.0",
"createOptions": "{}"
}
},
"edgeHub": {
"type": "docker",
"status": "running",
"restartPolicy": "always",
"settings": {
"image": "mcr.microsoft.com/azureiotedge-hub:1.0",
"createOptions": "{\"HostConfig\":{\"PortBindings\":{\"5671/tcp\":[{\"HostPort\":\"5671\"}],\"8883/tcp\":[{\"HostPort\":\"8883\"}],\"443/tcp\":[{\"HostPort\":\"443\"}],\"8082/tcp\":[{\"HostPort\":\"8082\"}]}}}"
}
}
},
"modules": {
"lvaEdge": {
"version": "1.0",
"type": "docker",
"status": "running",
"restartPolicy": "always",
"settings": {
"image": "mcr.microsoft.com/media/live-video-analytics:2",
"createOptions": "{\"HostConfig\":{\"PortBindings\":{\"8082/tcp\":[{\"HostPort\":\"8082\"}]},\"LogConfig\":{\"Type\":\"\",\"Config\":{\"max-size\":\"10m\",\"max-file\":\"10\"}},\"Binds\":[\"/var/media:/var/media/\",\"/var/lib/azuremediaservices:/var/lib/azuremediaservices\"]}}"
}
},
"rtspsim": {
"version": "1.0",
"type": "docker",
"status": "running",
"restartPolicy": "always",
"settings": {
"image": "mcr.microsoft.com/lva-utilities/rtspsim-live555:1.2",
"createOptions": "{\"HostConfig\":{\"Binds\":[\"/home/cameraedgevm_user/samples/input:/live/mediaServer/media\"]}}"
}
},
"lvaSupport": {
"version": "1.0",
"type": "docker",
"status": "running",
"restartPolicy": "always",
"settings": {
"image": "cameraedgedev.azurecr.io/lvasupport:1.2",
"createOptions": "{\"HostConfig\":{\"PortBindings\":{\"80/tcp\":[{\"HostPort\":\"80\"}]}}}"
}
},
"ai_module": {
"version": "1.0",
"type": "docker",
"status": "running",
"restartPolicy": "always",
"env": {
"PYTHONUNBUFFERED": {
"value": "1"
},
"NVIDIA_VISIBLE_DEVICES": {
"value": "all"
}
},
"settings": {
"image": "cameraedgedev.azurecr.io/ai_module:1.2",
"createOptions": "{\"HostConfig\":{\"Runtime\":\"nvidia\",\"PortBindings\":{\"8082/tcp\":[{\"HostPort\":\"8082\"}]},\"Binds\":[\"/home/cameraedgevm_user/ai_module/data:/ai_module/data/\"]}}"
}
}
}
}
},
"$edgeHub": {
"properties.desired": {
"schemaVersion": "1.0",
"routes": {
"LVAToHub": "FROM /messages/modules/lvaEdge/outputs/* INTO $upstream",
"LVATolvaSupportModule": "FROM /messages/modules/lvaEdge/outputs/* INTO BrokeredEndpoint(\"/modules/lvaSupport/inputs/input1\")"
},
"storeAndForwardConfiguration": {
"timeToLiveSecs": 7200
}
}
},
"lvaEdge": {
"properties.desired": {
some properties
}
},
"lvaSupport": {
"properties.desired": {
"Gateway": "http://demo.thingsboard.io/api/v1/camera/telemetry",
"gatewayMethod": "POST"
}
}
}
}
How should I resolve this problem?
How can i send message to external gateway
Thanks.
By default, your modules in Azure IoT Edge will be able to access the internet, unless the configuration was changed, or a firewall is in place that restricts you from reaching thingsboards.io To verify if your module can reach that URL, you can try the following:
docker run -i --rm -p 80:80 --network="azure-iot-edge" alpine ping thingsboard.io
This will run a container on the same Docker network as your modules and see if you can resolve the URL.
You mentioned the HTTP status code 500, which indicates that a server (from Thingsboard) is having an issue with your request. Because you mentioned that this works if you just run the container manually, this could be because your payload is different when you run your module in the Azure IoT Edge runtime. An easy way to find out is to use logging to check what you're sending to Thingsboard in each scenario.
Note: in your question, you said
In docker file, expose 80 port and bind with local machine 80 port
This is only needed if you want your module to accept incoming requests over port 80. For outgoing requests (such as messages to Thingsboard) this is not needed.

How to get the running status of IOT Edge Modules via Azure REST API? (not the connectionState or status from the 'Get Modules on Device' API Call)

Context:
Azure REST - Modules - Get Modules On Device
By using this API call, I can get information about connectionState(connected/disconnected) & status(enalbled/disabled) of modules. We can check the runtime status of the modules deployed on the device by visiting the Azure Iot Hub web portal
portal.azure.com -> iot hub -> iot edge section -> select the device you wish to find the details for
Question:
How can I get this RUNTIME STATUS via the Azure API?(please refer to the picture).
If you check the module twin of edgeAgent it is like below:-
{
"deviceId": "edgeDevice",
"moduleId": "$edgeAgent",
"etag": "AAAAAAAAAEA=",
"deviceEtag": "NDU1OTY3MjA=",
"status": "enabled",
"statusUpdateTime": "0001-01-01T00:00:00Z",
"connectionState": "Disconnected",
"lastActivityTime": "0001-01-01T00:00:00Z",
"cloudToDeviceMessageCount": 0,
"authenticationType": "sas",
"x509Thumbprint": {
"primaryThumbprint": null,
"secondaryThumbprint": null
},
"version": 501,
"properties": {
"desired": {
"schemaVersion": "1.0",
"runtime": {
"type": "docker",
"settings": {
"minDockerVersion": "v1.25",
"loggingOptions": "",
"registryCredentials": {
}
}
},
"systemModules": {
"edgeAgent": {
"type": "docker",
"settings": {
"image": "mcr.microsoft.com/azureiotedge-agent:1.0",
"createOptions": "{}"
}
},
"edgeHub": {
"type": "docker",
"status": "running",
"restartPolicy": "always",
"settings": {
"image": "mcr.microsoft.com/azureiotedge-hub:1.0",
"createOptions": "{\"HostConfig\":{\"PortBindings\":{\"5671/tcp\":[{\"HostPort\":\"5671\"}],\"8883/tcp\":[{\"HostPort\":\"8883\"}],\"443/tcp\":[{\"HostPort\":\"443\"}]}}}"
}
}
},
"modules": {
"CustomModuleName": {
"version": "1.0",
"type": "docker",
"status": "running",
"restartPolicy": "always",
"settings": {
"image": "iotregdev300.azurecr.io/customModuleName:0.0.2-amd64",
"createOptions": "{\"HostConfig\":{\"PortBindings\":{\"8080/tcp\":[{\"HostPort\":\"8080\"}]}}}"
}
}
}
},
"reported": {
"schemaVersion": "1.0",
"version": {
"version": "1.0.9.4",
"build": "32971639",
"commit": "12d55e582cc7ce95c8abfe11eddfbbc938ed6001"
},
"lastDesiredStatus": {
"code": 200,
"description": ""
},
"runtime": {
"platform": {
"os": "linux",
"architecture": "x86_64",
"version": "1.0.9.4"
},
"type": "docker",
"settings": {
"minDockerVersion": "v1.25",
"loggingOptions": "",
"registryCredentials": {
}
}
},
"systemModules": {
"edgeAgent": {
"type": "docker",
"exitCode": 0,
"statusDescription": "running",
"lastStartTimeUtc": "2020-09-09T07:34:34.4585643Z",
"lastExitTimeUtc": "2020-09-09T07:34:26.9869915Z",
"runtimeStatus": "running",
"imagePullPolicy": "on-create",
"settings": {
"image": "mcr.microsoft.com/azureiotedge-agent:1.0",
"imageHash": "sha256:1a2fffc3c74a2b2510a3149bb2295b68a553e4c9aca90698879902f36fd6d163",
"createOptions": "{}"
}
},
"edgeHub": {
"type": "docker",
"status": "running",
"restartPolicy": "always",
"imagePullPolicy": "on-create",
"env": {},
"exitCode": 0,
"statusDescription": "running",
"lastStartTimeUtc": "2020-09-09T07:34:50.8012461Z",
"lastExitTimeUtc": "2020-09-09T07:34:26.9845717Z",
"restartCount": 0,
"lastRestartTimeUtc": "2020-09-09T07:34:26.9845717Z",
"runtimeStatus": "running",
"settings": {
"image": "mcr.microsoft.com/azureiotedge-hub:1.0",
"imageHash": "sha256:f531eb6c23f347c37ea8c90204e9cb12024aec77d8b2e68e93b14c38ec066520",
"createOptions": "{\"HostConfig\":{\"PortBindings\":{\"5671/tcp\":[{\"HostPort\":\"5671\"}],\"8883/tcp\":[{\"HostPort\":\"8883\"}],\"443/tcp\":[{\"HostPort\":\"443\"}]}}}"
}
}
},
"lastDesiredVersion": 64,
"modules": {
"CustomModuleName": {
"exitCode": 0,
"statusDescription": "running",
"lastStartTimeUtc": "2020-09-09T07:34:49.3923079Z",
"lastExitTimeUtc": "2020-09-09T07:34:26.9606688Z",
"restartCount": 0,
"lastRestartTimeUtc": "2020-09-09T07:34:26.9606688Z",
"runtimeStatus": "running",
"version": "1.0",
"status": "running",
"restartPolicy": "always",
"imagePullPolicy": "on-create",
"type": "docker",
"settings": {
"image": "iotregdev300.azurecr.io/custommodulename:0.0.2-amd64",
"imageHash": "sha256:e728d4b8804d2114beab7c1903f706d8152e404be3f5601ee5e7371e8ac32ecf",
"createOptions": "{\"HostConfig\":{\"PortBindings\":{\"8080/tcp\":[{\"HostPort\":\"8080\"}]}}}"
},
"env": {}
}
}
}
}
}
In the above json, CustomModuleName is the custom module and it has the field called runtimeStatus: "running". Same field exists in the edgeHub and edgeAgent modules too. So you need to just fetch the edgeAgentTwin through REST API or Azure Device/Service sdk.

Error creating a customContent on a confluence addon

Today I was trying to create a confluence addon for my company and I've try following atlassian documents.
My problem comes trying to run the express app when adding a new customContent to the atlassian-connect.json, after running npm start I get the following error.
Failed to register with host https‍://admin:xxx#xxx.atlassian.net/wiki (200)
{"type":"INSTALL","pingAfter":300,"status":{"done":true,"statusCode":200,"con
tentType":"application/vnd.atl.plugins.task.install.err+json","subCode":"upm.
pluginInstall.error.descriptor.not.from.marketplace","source":"https‍://1a0adc
8f.ngrok.io/atlassian-connect.json","name":"https‍://1a0adc8f.ngrok.io/atlassi
an-connect.json"},"links":{"self":"/wiki/rest/plugins/1.0/pending/b88594d3-c3
c2-4760-b687-c8d860c0a377","alternate":"/wiki/rest/plugins/1.0/tasks/b88594d3
-c3c2-4760-b687-c8d860c0a377"},"timestamp":1502272147602,"userKey":"xxx","id":"xxx"}
Add-on not registered; no compatible hosts detected
This is my atlassian-connect.json file:
{
"key": "my-add-on",
"name": "Ping Pong",
"description": "My very first add-on",
"vendor": {
"name": "Angry Nerds",
"url": "https://www.atlassian.com/angrynerds"
},
"baseUrl": "{{localBaseUrl}}",
"links": {
"self": "{{localBaseUrl}}/atlassian-connect.json",
"homepage": "{{localBaseUrl}}/atlassian-connect.json"
},
"authentication": {
"type": "jwt"
},
"lifecycle": {
"installed": "/installed"
},
"scopes": [
"READ"
],
"modules": {
"generalPages": [
{
"key": "hello-world-page-jira",
"location": "system.top.navigation.bar",
"name": {
"value": "Hello World"
},
"url": "/hello-world",
"conditions": [{
"condition": "user_is_logged_in"
}]
},
{
"key": "customersViewer",
"location": "system.header/left",
"name": {
"value": "Hello World"
},
"url": "/hello-world",
"conditions": [{
"condition": "user_is_logged_in"
}]
}
],
"customContent": [
{
"key": "customer",
"name": {
"value": "Customers"
},
"uiSupport": {
"contentViewComponent": {
"moduleKey": "customersViewer"
},
"listViewComponent": {
"moduleKey": "customerList"
},
"icons": {
"item": {
"url": "/images/customers.png"
}
}
},
"apiSupport": {
"supportedContainerTypes": ["space"]
}
}
]
}
}
Does anybody has an idea on whats going on?
The contentViewComponent can't find the generalPage it is referencing in moduleKey.
From the docs:
In the snippet above, the moduleKey “customersViewer” maps to a
generalPage module we have defined in our add-on. This generalPage is
passed the context parameters we specify, and visualizes our content
accordingly.
If you change the generalPage with the key hello-world-page-confluence to customersVieweryou be able to install and get up and running.

Unable to start a ubuntu container in openshift-origin

Am trying to bring up a ubuntu container in a POD in openshift. I have setup my local docker registry and have configured DNS accordingly. Starting the ubuntu container with just docker works fine without any issues. When I deploy the POD, I can see that my docker ubuntu image is pulled successfully, but doesnt succeed in starting the same. It fails with back-off pulling image error. Is this because my entry point does not have any background process running in side the container ?
"openshift.io/container.ubuntu.image.entrypoint": "[\"top\"]",
Snapshot of the events
Deployment-config :
{
"kind": "DeploymentConfig",
"apiVersion": "v1",
"metadata": {
"name": "ubuntu",
"namespace": "testproject",
"selfLink": "/oapi/v1/namespaces/testproject/deploymentconfigs/ubuntu",
"uid": "e7c7b9c6-4dbd-11e6-bd2b-0800277bbed5",
"resourceVersion": "4340",
"generation": 6,
"creationTimestamp": "2016-07-19T14:34:31Z",
"labels": {
"app": "ubuntu"
},
"annotations": {
"openshift.io/deployment.cancelled": "4",
"openshift.io/generated-by": "OpenShiftNewApp"
}
},
"spec": {
"strategy": {
"type": "Rolling",
"rollingParams": {
"updatePeriodSeconds": 1,
"intervalSeconds": 1,
"timeoutSeconds": 600,
"maxUnavailable": "25%",
"maxSurge": "25%"
},
"resources": {}
},
"triggers": [
{
"type": "ConfigChange"
},
{
"type": "ImageChange",
"imageChangeParams": {
"automatic": true,
"containerNames": [
"ubuntu"
],
"from": {
"kind": "ImageStreamTag",
"namespace": "testproject",
"name": "ubuntu:latest"
},
"lastTriggeredImage": "ns1.myregistry.com:5000/ubuntu#sha256:6d9a2a1bacdcb2bd65e36b8f1f557e89abf0f5f987ba68104bcfc76103a08b86"
}
}
],
"replicas": 1,
"test": false,
"selector": {
"app": "ubuntu",
"deploymentconfig": "ubuntu"
},
"template": {
"metadata": {
"creationTimestamp": null,
"labels": {
"app": "ubuntu",
"deploymentconfig": "ubuntu"
},
"annotations": {
"openshift.io/container.ubuntu.image.entrypoint": "[\"top\"]",
"openshift.io/generated-by": "OpenShiftNewApp"
}
},
"spec": {
"containers": [
{
"name": "ubuntu",
"image": "ns1.myregistry.com:5000/ubuntu#sha256:6d9a2a1bacdcb2bd65e36b8f1f557e89abf0f5f987ba68104bcfc76103a08b86",
"resources": {},
"terminationMessagePath": "/dev/termination-log",
"imagePullPolicy": "Always"
}
],
"restartPolicy": "Always",
"terminationGracePeriodSeconds": 30,
"dnsPolicy": "ClusterFirst",
"securityContext": {}
}
}
},
"status": {
"latestVersion": 5,
"details": {
"causes": [
{
"type": "ConfigChange"
}
]
},
"observedGeneration": 5
}
The problem was with the http proxy. After solving that image pull was successful

Resources