I have a simple Azure Function which returns to a queue:
private readonly TelemetryClient _telemetryClient;
[return: Queue("%ReturnQueue%")]
public async Task<string> Run([QueueTrigger("%RequestQueue%")] string msg, ILogger log)
{
try
{
//Some dependency calls
}
catch(Exception ex)
{
var dic = new Dictionary<string,string>();
dic.Add("Id", someId);
dic.Add("CustomData", cusomData);
_telemetryClient.TrackException(ex, dic);
}
}
I obviously get a compilation error saying that not all code paths returns a value.
The problem is that if I add a throw at the end of the catch block the Azure Functions runtime replicate the excpetion on the appinsights portal. How can I add custom data to my exceptions like this?
You can create your own Exception type:
public class MyCustomException : Exception
{
public string Id {get;set;}
public string CustomData {get;set;}
public Exception RootException {get;set;}
public MyCustomException(string id, string customData, Exception ex)
{
Id = id;
CustomData = customData;
RootException = ex;
}
}
private readonly TelemetryClient _telemetryClient;
[return: Queue("%ReturnQueue%")]
public async Task<string> Run([QueueTrigger("%RequestQueue%")] string msg, ILogger log)
{
try
{
//Some dependency calls
}
catch(Exception ex)
{
//var dic = new Dictionary<string,string>();
//dic.Add("Id", someId);
//dic.Add("CustomData", cusomData);
var customEx = new MyCustomException(someId, cusomData, ex);
_telemetryClient.TrackException(customEx);
}
finally
{
return "";
}
}
PS: inside MyCustomException you can actually have Dictionary rather than string properties.
Related
There is a field called "metadata" (not to be confused with GRPC metadata) that is present in every request proto that comes to the GRPC service:
message MyRequest {
RequestResponseMetadata metadata = 1;
...
}
And the same field is also present in all responses:
message MyResponse {
RequestResponseMetadata metadata = 1;
...
}
I am trying to write a ServerInterceptor (or something else, if it works) to read the "metadata" field from the request, keep it somewhere, and then set it in the response once done processing the request.
Attempt 1: ThreadLocal
public class ServerInterceptor implements io.grpc.ServerInterceptor {
private ThreadLocal<RequestResponseMetadata> metadataThreadLocal = new ThreadLocal<>();
#Override
public <ReqT, RespT> ServerCall.Listener<ReqT> interceptCall(
ServerCall<ReqT, RespT> call,
final Metadata requestHeaders,
ServerCallHandler<ReqT, RespT> next) {
return new SimpleForwardingServerCallListener<ReqT>(
next.startCall(
new SimpleForwardingServerCall<ReqT, RespT>(call) {
#Override
public void sendMessage(RespT message) {
super.sendMessage(
(RespT)
MetadataUtils.setMetadata(
(GeneratedMessageV3) message, metadataThreadLocal.get()));
metadataThreadLocal.remove();
}
},
requestHeaders)) {
#Override
public void onMessage(ReqT request) {
// todo nava see if ReqT can extend GenericV3Message
var metadata = MetadataUtils.getMetadata((GeneratedMessageV3) request);
metadataThreadLocal.set(metadata);
super.onMessage(request);
}
};
}
}
I tried to use ThreadLocal, to later realise that sendMessage and onMessage need not necessary to be on the same thread.
Attempt 2: GRPC Context
public class ServerInterceptor implements io.grpc.ServerInterceptor {
public static final Context.Key<RequestResponseMetadata> METADATA_KEY = Context.key("metadata");
#Override
public <ReqT, RespT> ServerCall.Listener<ReqT> interceptCall(
ServerCall<ReqT, RespT> call,
final Metadata requestHeaders,
ServerCallHandler<ReqT, RespT> next) {
return new SimpleForwardingServerCallListener<ReqT>(
next.startCall(
new SimpleForwardingServerCall<ReqT, RespT>(call) {
#Override
public void sendMessage(RespT message) {
super.sendMessage(
(RespT)
MetadataUtils.setMetadata(
(GeneratedMessageV3) message, METADATA_KEY.get()));
}
},
requestHeaders)) {
#Override
public void onMessage(ReqT request) {
var metadata = MetadataUtils.getMetadata((GeneratedMessageV3) request);
var newContext = Context.current().withValue(METADATA_KEY, metadata);
oldContext = newContext.attach();
super.onMessage(request);
}
};
}
}
I am planning to detach the context in a onComplete(), but before it reaches there itself, METADATA_KEY.get() in sendMessage returns null, while I was expecting it to return the data.
Even before hitting the sendMessage() function, I get this in the console, indicating that I am doing something wrong:
3289640 [grpc-default-executor-0] ERROR i.g.ThreadLocalContextStorage - Context was not attached when detaching
java.lang.Throwable: null
at io.grpc.ThreadLocalContextStorage.detach(ThreadLocalContextStorage.java:48)
at io.grpc.Context.detach(Context.java:421)
at io.grpc.Context$CancellableContext.detach(Context.java:761)
at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:39)
at io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:123)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)
How do I read data when a request is received, store it somewhere and use it when the response is send back?
You can use Metadata to pass values from the request to the response:
public class MetadataServerInterceptor implements ServerInterceptor {
public static final Metadata.Key<byte[]> METADATA_KEY = Metadata.Key.of("metadata-bin", Metadata.BINARY_BYTE_MARSHALLER);
#Override
public <ReqT, RespT> ServerCall.Listener<ReqT> interceptCall(ServerCall<ReqT, RespT> call, Metadata headers, ServerCallHandler<ReqT, RespT> next) {
var serverCall = new ForwardingServerCall.SimpleForwardingServerCall<>(call) {
#Override
public void sendMessage(RespT message) {
byte[] metadata = headers.get(METADATA_KEY);
message = (RespT) MetadataUtils.setMetadata((GeneratedMessageV3) message, metadata);
super.sendMessage(message);
}
};
ServerCall.Listener<ReqT> listenerWithContext = Contexts.interceptCall(Context.current(), serverCall, headers, next);
return new ForwardingServerCallListener.SimpleForwardingServerCallListener<>(listenerWithContext) {
#Override
public void onMessage(ReqT message) {
byte[] metadata = MetadataUtils.getMetadata((GeneratedMessageV3) message);
headers.put(METADATA_KEY, metadata);
super.onMessage(message);
}
};
}
}
Note: Since it is not possible to put the instance of RequestResponseMetadata in the metadata (at least without implementing a custom marshaller), you can save it there as a byte array. You can use toByteArray() on your RequestResponseMetadata object to get byte[] and RequestResponseMetadata.#parseFrom(byte[]) to get the object from byte[].
I am having an existing webjob(V3.0) in .net core that has a function that is invoked by manual trigger, essentially by a webhook. I want to add another function to the same webjob that should be invoked on a Timer trigger every 20 mins. Is it possible to have both these in the same webjob. If it is possible what should the host configuration that I need to do. I tried going through Microsoft's documentation but there is barely any documentation with respect to the host configuration part with multiple triggers
Yes but your function would need to be triggered by something in Azure Storage like a Queue. This code is probably more then you might need. All of my services implement a custom interface IServiceInvoker. My CTOR asks for an
IEnumerable<IServiceInvoker>
which gets all of the services. Then I either use a constant or a passed in value to determine what service to run. Since I ONLY want ONE function to ever be running I am using the Singleton attribute passing in String.Empty. I also have the following settings on my Queues
b.AddAzureStorage(a =>
{
a.BatchSize = 1;
a.NewBatchThreshold = 1;
a.MaxDequeueCount = 1;
a.MaxPollingInterval = TimeSpan.FromSeconds(60);
a.VisibilityTimeout = TimeSpan.FromSeconds(60);
});
Finally I found that during testing I sometimes needed to turn off on or more functions hence the class ServiceConfigurationProvider.
Code sample follows I removed quite a bit of code so YMMV
public static async Task Main(string[] args)
{
await CreateHostBuilder(args).Build().RunAsync();
}
more code
public class Functions
{
/// <summary>
/// This scopes the singleton attribute to each individual function rather than the entire host
/// </summary>
const String SCOPESINGLETONTOFUNCTION = "";
readonly ILogger<Functions> _logger;
readonly Dictionary<String, IServiceInvoker> _services;
readonly IConfiguration _configuration;
private Functions()
{
_services = new Dictionary<string, IServiceInvoker>();
}
public Functions(IEnumerable<IServiceInvoker> services, ILoggerFactory loggerFactory, IServiceProvider serviceProvider, IConfiguration configuration) : this()
{
_logger = loggerFactory.CreateLogger<Functions>();
foreach (var service in services)
{
_services.Add(service.ServiceIdentifier, service);
}
_configuration = configuration;
}
[Disable(typeof(ServiceConfigurationProvider))]
[Singleton(SCOPESINGLETONTOFUNCTION)]
public async Task TimerTriggerFunction([TimerTrigger("%TimerTriggerFunctionExpression%")]TimerInfo myTimer, CancellationToken cancellationToken)
{
try
{
if (_services.TryGetValue("ServiceName", out IServiceInvoker serviceToInvoke))
{
await serviceToInvoke.InvokeServiceAsync(null, cancellationToken, false);
}
}
catch (Exception ex)
{
_logger?.LogError(ex, $"Unhandled exception occurred in method:'{nameof(TimerTriggerFunction)}'");
}
}
[Disable(typeof(ServiceConfigurationProvider))]
[Singleton(SCOPESINGLETONTOFUNCTION)]
public async Task ServiceInvokerQueueFunction([QueueTrigger("%ServiceInvokerQueueName%", Connection = "AzureWebJobsStorage")] ServiceInvokerMessage serviceInvokerMessage, CancellationToken cancellationToken)
{
if (serviceInvokerMessage is null || String.IsNullOrEmpty(serviceInvokerMessage.ServiceIdentifier))
{
_logger?.LogError("The queue message received in the ServiceInvokerQueueFunction could not be serialized into a ServiceInvokerMessage instance.");
}
else
{
Boolean serviceExists = _services.TryGetValue(serviceInvokerMessage.ServiceIdentifier, out IServiceInvoker serviceToInvoke);
if (serviceExists)
{
try
{
await serviceToInvoke.InvokeServiceAsync(null, cancellationToken, true);
}
catch (Exception exception)
{
_logger?.LogError(exception, $"Unhandled exception occurred in method:'{nameof(ServiceInvokerQueueFunction)}' for service:'{serviceInvokerMessage.ServiceIdentifier}'");
}
}
}
}
[Disable(typeof(ServiceConfigurationProvider))]
[Singleton(SCOPESINGLETONTOFUNCTION)]
public async Task RecordQueueFunction([QueueTrigger("%RecordQueueName%", Connection = "RecordConnectString")] string message, CancellationToken cancellationToken)
{
{
_logger?.LogInformation(message);
try
{
if (_services.TryGetValue("ServiceName", out IServiceInvoker serviceToInvoke))
{
await serviceToInvoke.InvokeServiceAsync(message, cancellationToken, false);
}
}
catch (Exception ex)
{
_logger?.LogError(ex, $"Unhandled exception occurred in method:'{nameof(RecordQueueFunction)}'");
throw;
}
}
}
}
public class ServiceConfigurationProvider
{
readonly IConfiguration _configuration;
public ServiceConfigurationProvider(IConfiguration configuration)
{
_configuration = configuration;
}
public bool IsDisabled(MethodInfo method)
{
Boolean returnValue = false;
String resultConfiguration = _configuration[$"{method.Name}Disable"];
if (!String.IsNullOrEmpty(resultConfiguration))
{
Boolean.TryParse(resultConfiguration, out returnValue);
}
return returnValue;
}
}
I want to call webservices via clients. The clients are instantiated as beans:
#Configuration
public class ServiceClientConfiguration {
#Bean
public Jaxb2Marshaller marshaller() {
Jaxb2Marshaller marshaller = new Jaxb2Marshaller();
marshaller.setContextPath(CONTEXT_PATH);
return marshaller;
}
#Bean
public Service1Client authenticate(Jaxb2Marshaller marshaller) {
Service1Client client = new Service1Client();
client.setDefaultUri("http://localhost:8292/service1");
client.setMarshaller(marshaller);
client.setUnmarshaller(marshaller);
return client;
}
#Bean
public Service2Client broker(Jaxb2Marshaller marshaller) {
Service2Client client = new Service2Client();
client.setDefaultUri("http://localhost:8192/service2");
client.setMarshaller(marshaller);
client.setUnmarshaller(marshaller);
return client;
}
Though the both of the services are different, the request and responses are defined via xsd-files to the same package name - which is provided here as String 'CONTEXT_PATH' to the marshaller.
The clients itself look like this:
public class Service1Client extends WebServiceGatewaySupport {
private static final Logger log = LoggerFactory.getLogger(Tools.getClassName());
public Service1Response process(Service1Request request) {
Service1Response response = null;
try {
response = (Service1Response) getWebServiceTemplate()
.marshalSendAndReceive("http://localhost:8292/service1", request);
} catch (Exception e) {
log.error("", e);
}
return response;
}
}
and
public class Service2Client extends WebServiceGatewaySupport {
private static final Logger log = LoggerFactory.getLogger(Tools.getClassName());
public Service2Response process(Service2Request request) {
Service2Response response = null;
try {
response = (Service2Response) getWebServiceTemplate()
.marshalSendAndReceive("http://localhost:8192/service2", request);
} catch (Exception e) {
log.error("", e);
}
return response;
}
}
While running Service1Client is fine, the Service2Client fails with
javax.xml.bind.JAXBException: class <package>.Service2Request nor any of its super class is known to this context.
I removed "Service1Client" from the code - but the error remained to be the same.
I have renamed the pathname of the CONTEXT_PATH. It was "types.". I changed it to "dto.". Now it works as expected.
For me the reason for the problem looks sort of strange - maybe somebody can explain it.
I followed this example https://learn.microsoft.com/pt-br/azure/app-service/webjobs-sdk-get-started and it is working fine. What I want to do is to make the connection strings (strongly typed) available in all methods within Functions class. My Connection Strings object:
namespace MyApp.Domain
{
public class Secrets
{
public class ConnectionStrings
{
public string SqlServer {get; set;}
public string Storage {get; set;}
public string SendGrid {get; set;}
public string AzureWebJobsDashboard { get; set; }
public string AzureWebJobsStorage {get; set;}
}
}
}
In web project I use (and it works perfectly):
services.Configure<Secrets.ConnectionStrings>(Configuration.GetSection("CUSTOMCONNSTR_ConnectionStrings"));
and in the classes' constructors I use:
public class EmailController: ControllerBase
{
private readonly MyEmail _myEmail;
public EmailController(MyEmail MyEmail)
{
_myEmail = MyEmail;
}
[HttpGet]
public async Task<ActionResult<string>> SendEmail()
{
try
{
...
return await _myEmail.SendMailMI3D(myMsg);
}
catch (System.Exception ex)
{
return ex.Message + " - " + ex.StackTrace;
}
}
[HttpGet("sendgrid")]
public string GetSendGrid(long id)
{
return _myEmail.SendGridConnStr();
}
}
But this way doesn't work on webjobs (console apps).
I tried to insert a simple Console.WriteLine in Functions' constructor but it doesn't work as well. So I think this is the problem: Functions' constructor is not being called. So when I insert a message in my queue I receive this error message related to DI Connection String:
Microsoft.Azure.WebJobs.Host.FunctionInvocationException: Exception while executing function: Functions.ProcessQueueMessage ---> System.NullReferenceException: Object reference not set to an instance of an object.
Can anybody please help me? Thanks a lot.
public Functions(IOptions<Secrets.ConnectionStrings> ConnectionStrings)
{
_connectionStrings = ConnectionStrings;
Console.WriteLine("Simple line");
Console.WriteLine($"Functions constructor: ${_connectionStrings.Value.SendGrid}");
}
Microsoft.Azure.WebJobs.Host.FunctionInvocationException: Exception while executing function: Functions.ProcessQueueMessage ---> System.NullReferenceException: Object reference not set to an instance of an object.
Dependency Injection is available in WebJobs but you do need to take the extra step to create an IJobActivator to define the injection.
namespace NetCoreWebJob.WebJob
{
public class JobActivator : IJobActivator
{
private readonly IServiceProvider services;
public JobActivator(IServiceProvider services)
{
this.services = services;
}
public T CreateInstance<T>()
{
return services.GetService<T>();
}
}
}
Inside Main()
var config = new JobHostConfiguration();
config.JobActivator = new JobActivator(services.BuildServiceProvider());
That should allow the runtime to utilize the parameterized constructor.
I create code to save log to azure tables.
And I override ActivateOptions method to create table if it does exist.
public override async void ActivateOptions()
{
base.ActivateOptions();
CloudStorageAccount storageAccount = CloudStorageAccount.Parse
(CloudConfigurationManager.GetSetting("StorageConnectionString"));
_tableClient = storageAccount.CreateCloudTableClient();
await CraeteTablesIfNotExist();
}
private async Task CraeteTablesIfNotExist()
{
CloudTable logCloudTable = _tableClient.GetTableReference(TableName);
await logCloudTable.CreateIfNotExistsAsync();
}
And code to save message to blob storage:
protected override async void Append(LoggingEvent loggingEvent)
{
try
{
CloudTable cloudTable = _tableClient.GetTableReference(TableName);
TableBatchOperation tableBatchOperation = new TableBatchOperation();
tableBatchOperation.InsertOrReplace(new LogEntry($"{DateTime.UtcNow:yyyy-MM}",
$"{DateTime.UtcNow:dd HH:mm:ss.fff}-{Guid.NewGuid()}")
{
LoggerName = loggingEvent.LoggerName,
Message = loggingEvent.RenderedMessage
});
await cloudTable.ExecuteBatchAsync(tableBatchOperation);
}
catch (DataServiceRequestException drex)
{
ErrorHandler.Error("Couldwrite log entry", drex);
}
catch (Exception ex)
{
ErrorHandler.Error("Exception log entry", ex);
}
}
It does not work! I don't know why but if I move code from ActivateOptions to constructor tables created success.
Code below runs my ActivateOptions method and logs a message:
[TestFixture]
public class Log4NetHandler : TableStorage
{
private TableStorage _storage;
[SetUp]
public void Init()
{
_storage = new TableStorage();
_storage.ActivateOptions();
BasicConfigurator.Configure(_storage);
}
[Test]
public void CheckLogger()
{
Append(new LoggingEvent(new LoggingEventData
{
LoggerName = "Taras",
Message = "Message"
}));
}
}
I don't understand why if I run ActivateOptions methods table in Azure is not created?
Can you try implementing the ActivateOptions method without the async calls? I have old code that is almost identical to yours which works fine.
public override void ActivateOptions()
{
base.ActivateOptions();
_account = CloudStorageAccount.Parse(ConnectionString);
_client = _account.CreateCloudTableClient();
_table = _client.GetTableReference(TableName);
_table.CreateIfNotExists();
}