I have successfully deployed my first IoT Edge Device to Azure IoT Hub. The edge device is Ubuntu 16.04 I used the following link as a guide https://learn.microsoft.com/en-us/azure/iot-edge/quickstart-linux
The link uses tempSensor to generator realtime streaming data.
On my Ubuntu I have installed a realtime event stream generator of json using the following link http://acesinc.net/introducing-a-streaming-json-data-generator/
When I check the logs see events on my Ubuntu machine I can see the logs from the tempSensor but I can't see the events from my streaming event generator.
The problem is I can see the logs (traffic being generatored) from the tempSensor, however I can't see the traffic being generated from my realtime traffic.
I did some research and I discovered that in order to send events to the edge hub I would need to create a custom module that receives a message and send it to the Edge Hub. I therefore would need to modify the streaming generator to a module SDK to send the data. It was suggested that I follow the guide at the following link:
https://learn.microsoft.com/en-us/azure/iot-edge/tutorial-csharp-module
Now, the problem is the guide from link is based on simulated IoT edge device that uses data generator based on the tempSensor. Whereas my Ubuntu uses totally different generator. So I attempted to only as much from the guide that I think would enable to me get my Ubuntu to send events to the IoT Edge hub.
My configuration is as follows:
namespace mycsharpmodule4 {
using System.Collections.Generic; // for KeyValuePair<>
using Microsoft.Azure.Devices.Shared; // for TwinCollection
using Newtonsoft.Json; // for JsonConvert
using System;
using System.IO;
using System.Runtime.InteropServices;
using System.Runtime.Loader;
using System.Security.Cryptography.X509Certificates;
using System.Text;
using System.Threading;
using System.Threading.Tasks;
using Microsoft.Azure.Devices.Client;
using Microsoft.Azure.Devices.Client.Transport.Mqtt;
class MessageBody {
public Machine machine {get;set;}
public Ambient ambient {get; set;}
public string timeCreated {get; set;} } class Machine { public double temperature {get; set;}
} class Ambient { public double temperature {get; set;} public int humidity {get; set;} }
}
Just so you know, I have followed everything in guide:
https://learn.microsoft.com/en-us/azure/iot-edge/tutorial-csharp-module
When I try build the code by running the command, 'Build IoT Edge Solution' I get the following error:
CSC : error CS5001: Program does not contain a static 'Main' method suitable for an entry point [C:\app\mycsharpmodule4.csproj]
The command 'cmd /S /C dotnet publish -c Release -o out' returned a non-zero code: 1
See image for complete view of error in Visual Studio Code.
visualstudiocode
Can someone let me know where I'm going wrong.
I'm checked on Stack Exchange for similar questions and came across
How to send data from a device to IOT HUB? . However, I was unable to make it work for me.
I have been at this for nearly 7 days now, and I'm about to give up.
Any help would be greatly appreciated
Paul
Related
I am listening to Event hub for various events.
Each event is high value and cannot be missed.
Events are partitioned based on device id.
Events from one device id are sparse and not very frequent (couple of events per few dasy) . It only occurs in response to a user action which is infrequent.
The number of devices are huge , so I will have a lot of events for a variety of device Ids.
For each event , I need to make 3-4 API calls to Systems which are not super reliable. And since some of these are cross Geo Calls it might take some time.
I am planning to take the events from Event hub and put them into Service Bus. My reasons are as follows.
Event hub can be scaled to only 32 partitions and if one event takes time , the entire partition gets blocked.
Service bus on the other hand is more horizontally scalable. If the throughput drops I can just add more subscribers to the Service Bus.
I have been looking for patterns like this but I have not seen patterns where we get data from a log based messaging system and push them to a queue based one.
Are there better approach to handle such scenarios ?
I think you can use Event hub trigger and service bus output binding to achieve what you want.
For example, I want to monitor Event hub 'test' and I am using C# library:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using Microsoft.Azure.EventHubs;
using Microsoft.Azure.WebJobs;
using Microsoft.Extensions.Logging;
namespace FunctionApp68
{
public static class Function1
{
[FunctionName("Function1")]
[return: ServiceBus("test1", Connection = "ServiceBusConnection")]
public static string Run([EventHubTrigger("test", Connection = "str")] EventData[] events, ILogger log)
{
var exceptions = new List<Exception>();
string messageBodyt = "";
foreach (EventData eventData in events)
{
try
{
string messageBody = Encoding.UTF8.GetString(eventData.Body.Array, eventData.Body.Offset, eventData.Body.Count);
messageBodyt = messageBodyt + messageBody;
// Replace these two lines with your processing logic.
log.LogInformation($"C# Event Hub trigger function processed a message: {messageBody}");
//await Task.Yield();
}
catch (Exception e)
{
// We need to keep processing the rest of the batch - capture this exception and continue.
// Also, consider capturing details of the message that failed processing so it can be processed again later.
exceptions.Add(e);
}
}
// Once processing of the batch is complete, if any messages in the batch failed processing throw an exception so that there is a record of the failure.
if (exceptions.Count > 1)
throw new AggregateException(exceptions);
if (exceptions.Count == 1)
throw exceptions.Single();
return messageBodyt;
}
}
}
The above code will collect from event hub 'test' and save to service bus queue 'test1'.
Have a look of these doc:
https://learn.microsoft.com/en-us/azure/azure-functions/functions-bindings-event-hubs-trigger?tabs=csharp
https://learn.microsoft.com/en-us/azure/azure-functions/functions-bindings-service-bus-output?tabs=csharp#example
What you need is actually a private queue per device-Id. As soon as event comes to event hub, Pull events from it and put that into device-Id's private queue, and then process it serially.
How to build queue per device-Id:
Simple way to build queue is to use SQL database(mostly it works if request per second are not very very high, for sql-db 100 req/second are normal.)
another horizontally scalable way is to use azure append blobs(if your event processors are stateless).
You can also use advanced methodology like using Azure Service Fabric Reliable Queue.
I'm calling this code from Windows IoT Core on RPi3 and getting this error. I'm trying to send a message to a blob in Azure. However, it only does it once and silently fails.
The Code:
s_deviceClient = DeviceClient.Create(s_iotHubUri, new
DeviceAuthenticationWithRegistrySymmetricKey(s_myDeviceId, s_deviceKey),
TransportType.Mqtt);
await s_deviceClient.SendEventAsync(message);
The Error:
microsoft azure devices client "I/O Error Occurred".
I was told that a using SetPolicy/ExponentialBackoff might work but I haven't been successful in implementing it. I'm calling it from a static class if that means anything.
I found a solution with a dynamic class, but I'd have to change the architecture of my app to use it.
https://azureiot.wordpress.com/2018/05/03/azure-iot-hub-device-sdk-retry-policy/
Trying to get Azure Webjobs to react to incoming Service Bus event, Im running this by hitting F5. Im getting the error at startup.
No job functions found. Try making your job classes and methods
public. If you're using binding extensions (e.g. ServiceBus, Timers,
etc.) make sure you've called the registration method for the
extension(s) in your startup code (e.g. config.UseServiceBus(),
config.UseTimers(), etc.).
My functions-class look like this:
public class Functions
{
// This function will get triggered/executed when a new message is written
// on an Azure Queue called queue.
public static void ProcessQueueMessage([ServiceBusTrigger("test-from-dynamics-queue")] BrokeredMessage message, TextWriter log)
{
log.WriteLine(message);
}
}
I have every class and method set to public
I am calling config.UseServiceBus(); in my program.cs file
Im using Microsoft.Azure.WebJobs v 1.1.2
((Im not entirely sure I have written the correct AzureWebJobsDashboard- and AzureWebJobsStorage-connectionstrings, I took them from my only Azure storage-settings in Azure portal. If that might be the problem, where should I get them ))
According to your mentioned error, it seems that you miss parameter config for ininitializing JobHost. If it is that case, please use the following code.
JobHost host = new JobHost(config)
More detail info about how to use Azure Service Bus with the WebJobs SDK please refer to the document.The following is the sample code from document.
public class Program
{
public static void Main()
{
JobHostConfiguration config = new JobHostConfiguration();
config.UseServiceBus();
JobHost host = new JobHost(config);
host.RunAndBlock();
}
}
Having some trouble processing messages created in the Azure IoT Hub.
Getting the following eror: Exception thrown: 'Microsoft.ServiceBus.Messaging.Amqp.AmqpException' in Microsoft.ServiceBus.dll ("An AMQP error occurred (condition='amqp:link:redirect').")
Can anyone point me in the right direction?
Regards,
Jonas
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using Microsoft.ServiceBus.Messaging;
namespace IOTHubMessageProcessor
{
class Program
{
static string connectionString = "HostName=yaddaaaa.azure-devices.net;SharedAccessKeyName=iothubowner;SharedAccessKey=keydataasdss+tacsCxwkWQeUm9sMCc2GHnQkIZHM=";
static string iotHubD2cEndpoint = "messages/events";
static EventHubClient eventHubClient;
static void Main(string[] args)
{
Console.WriteLine("Receive messages\n");
eventHubClient = EventHubClient.CreateFromConnectionString(connectionString, iotHubD2cEndpoint);
var d2cPartitions = eventHubClient.GetRuntimeInformation().PartitionIds;
foreach (string partition in d2cPartitions)
{
ReceiveMessagesFromDeviceAsync(partition);
}
Console.ReadLine();
}
private async static Task ReceiveMessagesFromDeviceAsync(string partition)
{
var eventHubReceiver = eventHubClient.GetDefaultConsumerGroup().CreateReceiver(partition, DateTime.UtcNow);
while (true)
{
EventData eventData = await eventHubReceiver.ReceiveAsync();
if (eventData == null) continue;
string data = Encoding.UTF8.GetString(eventData.GetBytes());
Console.WriteLine(string.Format("Message received. Partition: {0} Data: '{1}'", partition, data));
}
}
}
}
I don't encounter this issue when testing with your code, so it might be related to something else. I find the duplicate thread Azure IoT hub basic receiving example, AMQP error, it suggests check on block port or proxy settings, you can try it.
Your iotHubD2cEndpoint has an incorrect format. You can find your compatible endpoint in your azure portal -> messages -> device to cloud settings.
I found this a very helpful example: https://github.com/ppatierno/codesamples/tree/master/IoTHubAmqp/IoTHubAmqp
Code looks fine to me. Mine is identical and works well.
Your best bet is to create a new IoT hub in Azure and replace the strings.
While building the connectionstring, can you try "Endpoint=" instead of "HostName=" on line 12?
Hope this helps!
Mert
A couple of comments:
Please ensure that you are using the latest version of the Service Bus dll. i.e., 3.1.7 (as of today).
Please do not mix async and sync method calls in your code.
Let us know if you are still encountering this issue.
Sometimes I see that the proxy filters amqp packets. Changing the transport type to http probably will solve the problem.
Now I don't have access to Visual Studio but I seem to remember that it is possible to set the transport under the client properties.
If you try it you can easily find if the problem is in the proxy or in the program.
Am trying to implement SLAB for my Azure Web app (In Process) and my listner is Azure table Storage (table conection string) ,
the problem am facing is -“EventSource.IsEnabled() = always returns false”
(Am running the application from VS2013 with IIS express)
my code
————global.asax
var listener2 = new ObservableEventListener();
listener2.EnableEvents(SBEvents.Log, EventLevel.Verbose,Keywords.All);
listener2.LogToWindowsAzureTable(“sdf”, “DefaultEndpointsProtocol=https;AccountName=********;AccountKey=****************);
———-Event Source
Public class SBEvents :EventSource {
public class keywords{...}
public class Tasks {..}
private static readonly Lazy Instance = new Lazy(() => new SBEvents());
public static SBEvents Log { get { return Instance.Value; } }
[Event(102, Message = “Bike started with Bike ID :{0}”, Keywords = Keywords.Application, Level = EventLevel.Informational)]
public void BikeStarted(String BikeID){
if (this.IsEnabled()) //// = always returns false
this.WriteEvent(102,BikeID);
It looks like 'Azure Web Apps' cannot listen to ETW events.
https://azure.microsoft.com/en-in/documentation/articles/choose-web-site-cloud-service-vm/
Areas of diagnostics logging and tracing that aren't available to web applications on Azure are Windows ETW events, and common Windows event logs (e.g. System, Application and Security event logs). Since ETW trace information can potentially be viewable machine-wide (with the right ACLs), read and write access to ETW events are blocked. Developers might notice that API calls to read and write ETW events and common Windows event logs appear to work, but that is because WEb Apps is "faking" the calls so that they appear to succeed. In reality, the web app code has no access to this event data
Thanks