Configuring notification tag for Azure Function - azure

I'm using an Azure function to pick up messages from an event hub and send out notifications via the Azure notification hub. Works great! Now I wanted to see whether I could add tags to those messages in order to allow user targeting via those tags.
The output for the notification hub has a "tag expression" parameter which you can configure. But this seems to be a static text. I need to dynamically set these tags based on the message received from the event hub instead. I'm not sure whether you can somehow put dynamic content in there?
I also found that the constructor of the GcmNotification object that I'm using has an overload which allows a tag string to be used. But when I try that I get a warning on compile time stating this is deprecated and when the function fires it shows an error because the Tag property should be empty.
So I'm not clear on a) whether this is at all possible and b) how to do it when it is. Any ideas?
Update: as suggested I tried creating a POCO object to map to my input string. The string is as follows:
[{"deviceid":"repsaj-neptune-win10pi","readingtype":"temperature1","reading":22.031614503139451,"threshold":23.0,"time":"2016-06-22T09:38:54.1900000Z"}]
The POCO object:
public class RuleMessage
{
public string deviceid;
public string readingtype;
public object reading;
public double threshold;
public DateTime time;
}
For the function I now tried both RuleMessage[] and List<RuleMessage> as parameter types, but the function complains it cannot convert the input:
2016-06-24T18:25:16.830 Exception while executing function: Functions.submerged-function-ruleout. Microsoft.Azure.WebJobs.Host: Exception binding parameter 'inputMessage'. Microsoft.Azure.WebJobs.Host: Binding parameters to complex objects (such as 'RuleMessage') uses Json.NET serialization.
1. Bind the parameter type as 'string' instead of 'RuleMessage' to get the raw values and avoid JSON deserialization, or
2. Change the queue payload to be valid json. The JSON parser failed: Cannot deserialize the current JSON array (e.g. [1,2,3]) into type 'Submission#0+RuleMessage' because the type requires a JSON object (e.g. {"name":"value"}) to deserialize correctly.
To fix this error either change the JSON to a JSON object (e.g. {"name":"value"}) or change the deserialized type to an array or a type that implements a collection interface (e.g. ICollection, IList) like List that can be deserialized from a JSON array. JsonArrayAttribute can also be added to the type to force it to deserialize from a JSON array.
Function code:
using System;
using Newtonsoft.Json;
using Newtonsoft.Json.Linq;
using Microsoft.Azure.NotificationHubs;
public static void Run(List<RuleMessage> inputEventMessage, string inputBlob, out Notification notification, out string outputBlob, TraceWriter log)
{
if (inputEventMessage == null || inputEventMessage.Count != 1)
{
log.Info($"The inputEventMessage array was null or didn't contain exactly one item.");
notification = null;
outputBlob = inputBlob;
return;
}
log.Info($"C# Event Hub trigger function processed a message: {inputEventMessage[0]}");
if (String.IsNullOrEmpty(inputBlob))
inputBlob = DateTime.MinValue.ToString();
DateTime lastEvent = DateTime.Parse(inputBlob);
TimeSpan duration = DateTime.Now - lastEvent;
if (duration.TotalMinutes >= 0) {
notification = GetGcmMessage(inputMessage.First());
log.Info($"Sending notification message: {notification.Body}");
outputBlob = DateTime.Now.ToString();
}
else {
log.Info($"Not sending notification message because of timer ({(int)duration.TotalMinutes} minutes ago).");
notification = null;
outputBlob = inputBlob;
}
}
private static Notification GetGcmMessage(RuleMessage input)
{
string message;
if (input.readingtype == "leakage")
message = String.Format("[FUNCTION GCM] Leakage detected! Sensor {0} has detected a possible leak.", input.reading);
else
message = String.Format("[FUNCTION GCM] Sensor {0} is reading {1:0.0}, threshold is {2:0.0}.", input.readingtype, input.reading, input.threshold);
message = "{\"data\":{\"message\":\""+message+"\"}}";
return new GcmNotification(message);
}
public class RuleMessage
{
public string deviceid;
public string readingtype;
public object reading;
public double threshold;
public DateTime time;
}
Update 28-6-2016: I've not managed to get it working by switching ASA output to line seperated to that it doesn't generate a JSON array any more. This is a temp. fix because the Function binding now fails as soon as there is more than one line in the output (which can happen).
Anyways, I now proceeded to set the tagExpression, as per instruction I changed it to:
{
"type": "notificationHub",
"name": "notification",
"hubName": "repsaj-neptune-notifications",
"connection": "repsaj-neptune-notifications_NOTIFICATIONHUB",
"direction": "out",
"tagExpression": "deviceId:{deviceid}"
}
Where {deviceid} equals the deviceid property on my RuleMessage POCO. Unfortunately this doesn't work, when I call the function it outputs:
Exception while executing function: Functions.submerged-function-ruleout. Microsoft.Azure.WebJobs.Host: Exception binding parameter 'notification'. Microsoft.Azure.WebJobs.Host: No value for named parameter 'deviceid'.
Which is not true, I know for sure the property has been set as I've logged it to the ouput window. I also tried something like {inputEventMessage.deviceid}, but that doesn't work either (as I didn't get how the runtime would map {deviceid} to the correct input object when there's more than one.

The tagExpression binding property supports binding parameters coming from trigger input properties. For example, assume your incoming Event Hub event has properties A and B. You can use these properties in your tagExpression using the parens syntax, e.g.: My Tag {A}-{B}.
In general, most of the properties across all the binding types support binding parameters in this way.

Related

How do I configure this JdbcMessageHandler to pull parameters from the message instead of static beans?

The Symptom:
When invoked, the cleanupMessageHandler component (a JdbcMessageHandler) of this subflow is not pulling parameters from the message, but is instead attempting to pull from a static bean.
The exception message:
org.springframework.dao.InvalidDataAccessApiUsageException: No value supplied for the SQL parameter 'ALC_startTime': Invalid property 'ALC_startTime' of bean class [org.springframework.integration.jdbc.JdbcMessageHandler$1]: Bean property 'ALC_startTime' is not readable or has an invalid getter method: Does the return type of the getter match the parameter type of the setter?
The SQL
Delete from SB_STREAM.TT_9321383
WHERE (LASTUPDATESTAMP >= to_timestamp (:ALC_startTime, 'YYYY-MM-DD HH24:MI:SS.FF')
and LASTUPDATESTAMP <= to_timestamp (:ALC_endTime, 'YYYY-MM-DD HH24:MI:SS.FF'))
AND ttkey = BIGINT(:ID)
AND (ttobjectname = :ALC_object AND ttschema = :ALC_schema)
The Flow:
return flowDef
.filter(getFilterExpression(rule)).channel(new DirectChannel())
.handle(inboundAdapter) // a JdbcOutboundAdapter
.split(insertDeleteSplitter)
.publishSubscribeChannel(taskExecutor, c ->
c.subscribe(s -> s
.filter ("....")
.transform(genericTransformer)
.handle(insertUpdateMessageHandler(rule))) // a JDBCMessageHandler
.subscribe(s -> s
.filter("....")
.transform(genericTransformer)
.handle(deleteMessageHandler(rule))) // a JDBCMessageHandler
.subscribe(sub -> sub
.handle(cleanupMessageHandler(rule))) // a JDBCMessageHandler
.errorHandler(new CustomErrorHandler() //
);
The cleanupMessagehandler
private MessageHandler cleanupMessageHandler(RuleMetadata rule) {
<snip ...>
SQLTextAndParamsList sql = sqlFactory.getCleanupSQL(rule);
JdbcMessageHandler handler = new JdbcMessageHandler(dbprx.getDatasource(), sql.getSql());
return handler;
}
Message as received by JdbcMessageHandler
SQLParameterSources at point where JDBCOperation is invoked
Alternate coding - batchArray of values at JDBCOperation is invoked (added ExpressionEvaluatingSqlParameterSourceFactory to the JdbcMessageHandler)
Debugger view at AbstractNestablePropertyAccessor.getPropertyValue()
Debugger view at CachedIntrospectionResults.getPropertyDecriptor()
Edit: code changes
The generated SQL was updated to this:
Delete from SB_STREAM.TT_9321383
WHERE (LASTUPDATESTAMP >= to_timestamp (:payload[ALC_startTime], 'YYYY-MM-DD HH24:MI:SS.FF')
and LASTUPDATESTAMP <= to_timestamp (:payload[ALC_endTime], 'YYYY-MM-DD HH24:MI:SS.FF'))
AND ttkey = :payload[ID]
AND (ttobjectname = :payload[ALC_object] AND ttschema = :payload[ALC_schema])
According to your first screenshot, the payload of your message is an ArrayList, which indeed does not have those ALC_startTime, ALC_endTime properties. There are just no those getters on the ArrayList!
To be able to read a content of your list, you need to configure an ExpressionEvaluatingSqlParameterSourceFactory on your JdbcMessageHandler. See docs for more info: https://docs.spring.io/spring-integration/docs/current/reference/html/jdbc.html#passing-parameters-by-using-spel-expressions.
Probably better to convert your list into a Map somehow or some POJO representation. With the list it is going to be hard for your to determine the target value by just index...
Edit:
This is a batch update, so the array list of maps is correct.
The SPEl expressions in the SQL parameters are malformed. The correct form is :payload[ALC_startTime].

ServiceStack Deserializing Interface property null in object

I want to use ServiceStack JsonSerializer.
I am setting IncludeTypeInfo property true before I serialize my object.And my serialized string contains all type informations like "__type":".Interfacesb,....
When I want to deserialize string that time my interface property null even though I have type information in my serialized string.Is there any other configuration need when deserializing object.
I use two methods
JsonSerializer.SerializeToString and JsonSerializer.DeSerializeFromString
Example:
JsConfig.IncludeTypeInfo = true;
Public Class MyObject
{
Public string a{get;set;}
Public interface b{get;Set;}
}
First, the version 4.* is the continued developed version. 3.9 is not actively maintained by anyone.
Test on servicestack.text 4.50
Secondly i don't think this this property was made to de-serialize it back practical objects.
i did the same in 4.50 and it just doesn't deserialize:
Alternative solutions
Here you can read what to if you want the types from the json: https://stackoverflow.com/a/21603948/1275832.
When you have the type:
I use the following code as an alternative solution (note its an extension method) as a solution for run-time dynamic types (v4.50):
public static object FromJson(this string json, Type deserializeType)
{
return typeof(JsonSerializer).GetMethod("DeserializeFromString", BindingFlags.Static)
.MakeGenericMethod(deserializeType)
.Invoke(null, new[] { json });
}
and usage as: var object = (MyInterface)jsonString.FromJson(Type.GetType(AssemblyQualifiedNameString));

Strange Out of Memory exception in C# with List or Strings or ado.net?

I have a big doubt. The problem is Out of Memory Exception in my class. But it seems something strange here. I have class in a dll.
public class MyClass : IDisposible
{
List<ClassA> a_classLists = new .....// new instance.
List<ClassB> b_classLists = new .....// new instance.
public string Method1(int IDValue)
{
// do here some web service call and get some XML data from it.
// parse the xml.
// Iterate through a for loop and add each node value to a_classLists
// Usually contains 10 or 15 items
Method2(); // from here calling another method
FinalSaveToDB(); // finally save the data to DB
return "";
}
private void Method2()
{
// do here some web service call and get some XML data from it.
// Iterate through a forloop.
// parse the xml. [large xml data. ie, image in binary format]
// For each loop add image binary data and other xml to b_classLists
// Usually it contains 50 or 60 such large lists.
}
private void FinalSaveToDB()
{
// using sqlbulkcopy, i am saving the data in the 2 lists to 2 different
// tables in the DB.
// Tab lock is mentioned in sqlbulk class.
// Actually 2 sqlbulkcopy class for 2 lists.
// Only 1 sql connection opens, then do the sqlbulkcopy. [there is no dataset or datareader]
// sqlconnection closes. I am using "using" clause for sqlconnection, bulkcopy etc
// these all are working fine.
}
private void Dispose()
{
// here nulling everything
// proxy null
// all the lists null....
}
}
This is the class I am instantiating 1000 times using reactive framework's Observable.Start
method as shown below...
private IObservable<string> SendEmpDetails(Employee emp)
{
using (MyClass p = new MyClass())
{
return Observable.Start(() => p.Method1(emp.ID), Scheduler.ThreadPool);
}
// here I hope it will call the Dispose and release all objects in the class.
}
// This EmployeeLists contains 1000 employee objects
EmployeeLists.ToObservable().Select(x => SendEmpDetails(x).Select(y => new { emp = x, retval = y }))
.Merge(10)
.ObserveOn(Scheduler.CurrentThread)
.Subscribe(x =>
{
SendStatus(x.retval.Item1, x.retval);
});
Even though, why i am getting out of memory exception ??? After starting the app, when it
process the 200th (or above) MyClass object, it throws error.
I forgot to mention 1 more thing, I am using VS 2010 and C# 4.0 (win7, 64 bit OS).
I need to log each activity. [ie, i need to understand the each and every process the app has gone through]. SO i declared a class [MyClass] level private string variable and assign each process details like "called this method", "got 5 records from this web service" etc.
logdata = Environment.Newline() + "This method has completed";
So the error is thrown here saying out of memory with some evalution failed.
So I turned off the string evaluation check box from Options in VS.
Again, there is no use.
So I changed the string to StringBuilder and tried to append the activity string each time.
Still no use. I dont understand what is the problem in it.
Is this because all the threads are working parallel, do they exchange the MyClass resources ??? Why the objects are not released ???
Please help me in this matter.

ServiceStack - Request Binding JSON encoded parameter

I have an existing application that sends a Request with a parameter named 'filters'. The 'filters' parameter contains a string that is JSON encoded. Example:
[{"dataIndex":"fieldName", "value":"fieldValue"}, {"dataIndex":"field2", "value":"value2"}].
Using ServiceStack, I would like to bind this as a property on a C# object (class Grid). Is there a preferred method to handle this? Here are the options I can think of. I don't think either 'feel' correct.
Option 1:
I do have a 'ServiceModel' project and this would create a dependency on it which I don't really like.
In AppHost.Configure() method add
RequestBinders[typeof(Grid)] => httpReq => {
return new Grid() {
Filters = new ServiceStack.Text.JsonSerializer<IList<Filter>>().DeserializeFromString(httpReq.QueryString["filters"])
}
}
Option 2:
Seems kind of 'hacky'
public class Grid
{
private string _filters;
public dynamic Filters {
get
{
ServiceStack.Text.JsonSerializer<IList<Filter().DeserializeFromString(_filters);
}
set
{
_filters = value;
}
}
}
You can send Complex objects in ServiceStack using the JSV Format.
If you want to send JSON via the QueryString you can access it from inside your Service of Request filters with something like:
public object Any(Request req) {
var filters = base.Request.QueryString["Filters"].FromJson<List<Filter>>();
}
Note: Interfaces on DTOs are bad practice.

Log4net EventLogAppender Log Event ID

Is there a way to log an event into the windows event log with a specified eventid per message? I am using log4net v 1.2.10.
Based on what I see in the EventLogAppender source code the following should do the trick:
log4net.ThreadContext.Properties["EventID"] = 5;
Just call this before you write your log messages (if you do not set it for all messages you should remove the "EventID" again from the Properties.
N.B the property key is case sensitive.
When one uses the native .net Event Log APIs in System.Diagnostics, the WriteEntry methods allow setting the eventID and category. In these APIs:
eventID is a 32 bit int, but its value must be between 0 and 65535
category is a 16 bit int, but its value must be positive. If the
event source includes a category resource file, the event viewer will
use the integer category value to lookup a localized “Task category”
string. Otherwise, the integer value is displayed.
The categories must be numbered consecutively, beginning with the
number 1
Log4net supports writing an EventID and a Category, but it isn’t straight forward. When log4net’s EventLogAppender logs an event, it looks at a dictionary of properties. The named properties "EventID" and "Category" are automatically mapped by the EventLogAppender to the corresponding values in the event log. I’ve seen a few good suggested ways to use log4net’s EventLogAppender and set the EventID and Category in the Windows event log.
a. Using log4net’s appender filtering, a filter may be registered that can add the EventID and Category properties. This method has a nice benefit that the standard log4net wrappers are used and so this can be implemented without changing existing logging code. The difficulty in this method is some mechanism has to be created to calculate the EventID and Category from the logged information. For instance, the filter could look at the exception source and map that source to a Category value.
b. Log4net may be extended so custom logging wrappers can be used that can include EventID and Category parameters. Adding EventID is demonstrated in the log4net sample “Extensibility – EventIDLogApp” which is included in the log4net source. In the extension sample a new interface (IEventIDLog) is used that extends the standard ILog interface used by applications to log. This provides new logging methods that include an eventId parameter. The new logging methods add the eventId to the Properties dictionary before logging the event.
public void Info(int eventId, object message, System.Exception t)
{
if (this.IsInfoEnabled)
{
LoggingEvent loggingEvent = new LoggingEvent(ThisDeclaringType, Logger.Repository, Logger.Name, Level.Info, message, t);
loggingEvent.Properties["EventID"] = eventId;
Logger.Log(loggingEvent);
}
}
c. Log4net supports a ThreadContext object that contains a Properties dictionary. An application could set the EventID and Category properties in this dictionary and then when the thread calls a logging method, the values will be used by the EventLogAppender.
log4net.ThreadContext.Properties["EventID"] = 5;
Some helpful references:
Log4net home page
Log4net SDK reference
Log4net samples
Log4net source
Enhancing log4net exception logging
Log4Net Tutorial pt 6: Log Event Context
Customizing Event Log Categories
EventSourceCreationData.CategoryResourceFile Property
Event Logging Elements
EventLog.WriteEntry Method
Well, the solution was to build the extension project "log4net.Ext.EventID" and to use its types: IEventIDLog, EventIDLogImpl and EventIDLogManager.
Another solution is to add a custom Filter as described here: Enhancing log4net exception logging (direct link to the Gist just in case).
As the author points out:
... EventLogAppender uses inline consts to check them. Once they are added they will be used by the mentioned EventLogAppender to mark the given entries with EventId and Category.
The filter implementation will look like the code below (stripped down gist) with the added benefit that if you make GetEventId method public, you can write some tests against it
public class ExceptionBasedLogEnhancer : FilterSkeleton
{
private const string EventLogKeyEventId = "EventID";
public override FilterDecision Decide(LoggingEvent loggingEvent)
{
var ex = loggingEvent.ExceptionObject;
if (ex != null)
{
loggingEvent.Properties[EventLogKeyEventId] = GetEventId(ex);
}
return FilterDecision.Neutral;
}
private static short GetEventId(Exception ex)
{
// more fancy implementation, like getting hash of ex properties
// can be provided, or mapping types of exceptions to eventids
// return no more than short.MaxValue, otherwise the EventLog will throw
return 0;
}
}
Extend ILog.Info() to take an event ID:
public static class LogUtils
{
public static void Info(this ILog logger, int eventId, object message)
{
log4net.ThreadContext.Properties["EventID"] = eventId;
logger.Info(message);
log4net.ThreadContext.Properties["EventID"] = 0; // back to default
}
}
Then call it like this:
using LogUtils;
private static readonly log4net.ILog _logger = log4net.LogManager.GetLogger(System.Reflection.MethodBase.GetCurrentMethod().DeclaringType);
_logger.Info(3, "First shalt thou take out the Holy Pin, then shalt thou count to three.");

Resources