I am writing a component that reads data from a specific filetype. Currently, it has a property for filepath - I would like for this block to quit as hard as possible when passed an invalid file/no file found.
Throwing an exception causes it to stop execution, but also deletes the block from the chalkboard while I am testing (?), which makes me think there is a more "approved" way to do it.
My current solution is something like:
LOG_ERROR( MyReader_i, "Unable to open file at " + Filepath );
return FINISH;
Is there another way to stop if something is wrong, that will hopefully stop all downstream processing as well?
Have you taken a look at the Data Reader component in the basic components? It also has a file path as an input. It deals with this during the onConfigure call as shown below:
def onconfigure_prop_InputFile(self, oldvalue, newvalue):
self.InputFile = newvalue
if not os.path.exists(self.InputFile):
self._log.error("InputFile path provided can not be accessed")
And then again in the service function by returning NOOP.
def process(self):
if (self.Play == False):
return NOOP
if not (os.path.exists(self.InputFile)):
return NOOP
This isn't the only way to deal with invalid input however. It's a design decision that is up to the developer.
If you'd like additional components down stream to know about an issue elsewhere in the chain, you have a few options. You could use the End of Stream bit, available in bulkio port implementations, to signal to down stream components that there is no additional data. They can then use this information to clean up and shut down. You could also use messaging to send a message out to an event channel and anyone who has subscribed to this event channel can be made aware of the message. Again, it's a design decision.
Related
I am using QFileSystemWatcher to control the log file changes.
For creating and updating the log file I am using boost library.
When I log few messages in one method file changing signal emits only ones (for last message), but I see that file updating every time after log message added.
So, the code for QFileSystemWatcher is
std::string fn = "app.log";
logging::init_log(fn);
QFileSystemWatcher* watcher = new QFileSystemWatcher();
auto success = QObject::connect(watcher, SIGNAL(fileChanged(QString)), this, SLOT(handleFileChanged(QString)));
Q_ASSERT(success);
watcher->addPath(QString::fromStdString(fn));
adding log messages
void a(){
/* some code */
logging::write_log("test error", logging::kError);
logging::write_log("test info", logging::kInfo);
}
QFileSystemWatcher emits signal only ones for Info level message.
In file manager I see that file updating after each call (test error, test info).
In log file initialization I use
sink->locked_backend()->auto_flush(true);
so the file updates immediately.
How can I fix this? Or maybe there is another approach how to handle log file updating to show message in GUI.
Similar filesystem event notifications are usually collapsed into one, unless they are consumed by a reader. For example, if the writer writes 10 bytes to a file, the thread that monitors that file for writes will typically see one event instead of 10. This is explicitly outlined in inotify description notes on Linux, which is likely used internally by QFileSystemWatcher.
This should not matter for any correct implementation of a filesystem monitoring software. The notification only allows the monitor to notice that some event happened (e.g. a write occurred), and it is up to the software to discover further details about the event (e.g. the amount of data that was written, and writing position).
If you aim to just display the written logs, you should be able to just read the file contents from the current reading position to the end of the file. That read operation may return one log record or more. It can return an incomplete log record, too, if the C++ standard library is implemented in a certain way (e.g. when auto_flush is disabled, and the stream buffer fills the internal buffer with part of the log record content before issuing write). The monitoring software should parse the read content to separate log records and detect incomplete log records (e.g. split data by newline characters).
I want to list blobs from container using azure java sdk version 12.0.6. In order to getting good performance, I use async method with class BlobContainerAsyncClient. But I cannot list blobs with the class and I do not get any error.
My code
ListBlobsOptions options = new ListBlobsOptions()
.setDetails(new BlobListDetails()
.setRetrieveDeletedBlobs(true)
.setRetrieveSnapshots(true));
client.listBlobs(options).subscribe(blob ->
System.out.printf("Name: %s, Directory? %b, Deleted? %b, Snapshot ID: %s%n",
blob.getName(),
blob.isPrefix(),
blob.isDeleted(),
blob.getSnapshot()));
Regarding the issue, please refer to the following code
ListBlobsOptions options = new ListBlobsOptions()
.setDetails(new BlobListDetails()
.setRetrieveDeletedBlobs(true)
.setRetrieveSnapshots(true));
lient.listBlobs(options).doOnNext(blob ->
System.out.printf("Name: %s, Directory? %b, Deleted? %b, Snapshot ID: %s%n",
blob.getName(),
blob.isPrefix(),
blob.isDeleted(),
blob.getSnapshot()))
.blockLast();
Besides, cannot call both subscribe() and block(). When you call subscribe() you get back a Disposable which allows you to cancel the async operation, but you cannot block on this object. If you want to ensure the method in question blocks, you simply call block() on the operation (the type which has a block() method is a Mono or possibly a Flux).
Two things you should try:
Use the sync APIs and ensure you get the response you expect there. If you get the results you expect, that rules out issues related to authentication, container names, and so on.
Because your code above is async, you need to be sure that the application is not exiting before the response is received from the service. If you could update your code listing above to show more fully what you're doing, we can rule that out. For the sake of increasing confidence, you may also choose to do one of the two things to temporarily increase confidence that your async code is correct:
Temporarily add a Thread.sleep(5000) call after your subscribe call.
Replace the .subscribe() call with a .block() call, essentially turning the request back into a sync call.
The Azure Service Bus supports a built-in retry mechanism which makes an abandoned message immediately visible for another read attempt. I'm trying to use this mechanism to handle some transient errors, but the message is made available immediately after being abandoned.
What I would like to do is make the message invisible for a period of time after it is abandoned, preferably based on an exponentially incrementing policy.
I've tried to set the ScheduledEnqueueTimeUtc property when abandoning the message, but it doesn't seem to have an effect:
var messagingFactory = MessagingFactory.CreateFromConnectionString(...);
var receiver = messagingFactory.CreateMessageReceiver("test-queue");
receiver.OnMessageAsync(async brokeredMessage =>
{
await brokeredMessage.AbandonAsync(
new Dictionary<string, object>
{
{ "ScheduledEnqueueTimeUtc", DateTime.UtcNow.AddSeconds(30) }
});
}
});
I've considered not abandoning the message at all and just letting the lock expire, but this would require having some way to influence how the MessageReceiver specifies the lock duration on a message, and I can't find anything in the API to let me change this value. In addition, it wouldn't be possible to read the delivery count of the message (and therefore make a decision for how long to wait for the next retry) until after the lock is already required.
Can the retry policy in the Message Bus be influenced in some way, or can a delay be artificially introduced in some other way?
Careful here because I think you are confusing the retry feature with the automatic Complete/Abandon mechanism for the OnMessage event-driven message handling. The built in retry mechanism comes into play when a call to the Service Bus fails. For example, if you call to set a message as complete and that fails, then the retry mechanism would kick in. If you are processing a message an exception occurs in your own code that will NOT trigger a retry through the retry feature. Your question doesn't get explicit on if the error is from your code or when attempting to contact the service bus.
If you are indeed after modifying the retry policy that occurs when an error occurs attempting to communicate with the service bus you can modify the RetryPolicy that is set on the MessageReciver itself. There is an RetryExponitial which is used by default, as well as an abstract RetryPolicy you can create your own from.
What I think you are after is more control over what happens when you get an exception doing your processing, and you want to push off working on that message. There are a few options:
When you create your message handler you can set up OnMessageOptions. One of the properties is "AutoComplete". By default this is set to true, which means as soon as processing for the message is completed the Complete method is called automatically. If an exception occurs then abandon is automatically called, which is what you are seeing. By setting the AutoComplete to false you required to call Complete on your own from within the message handler. Failing to do so will cause the message lock to eventually run out, which is one of the behaviors you are looking for.
So, you could write your handler so that if an exception occurs during your processing you simply do not call Complete. The message would then remain on the queue until it's lock runs out and then would become available again. The standard dead lettering mechanism applies and after x number of tries it will be put into the deadletter queue automatically.
A caution of handling this way is that any type of exception will be treated this way. You really need to think about what types of exceptions are doing this and if you really want to push off processing or not. For example, if you are calling a third party system during your processing and it gives you an exception you know is transient, great. If, however, it gives you an error that you know will be a big problem then you may decide to do something else in the system besides just bailing on the message.
You could also look at the "Defer" method. This method actually will then not allow that message to be processed off the queue unless it is specifically pulled by its sequence number. You're code would have to remember the sequence number value and pull it. This isn't quite what you described though.
Another option is you can move away from the OnMessage, Event-driven style of processing messages. While this is very helpful you don't get a lot of control over things. Instead hook up your own processing loop and handle the abandon/complete on your own. You'll also need to deal some of the threading/concurrent call management that the OnMessage pattern gives you. This can be more work but you have the ultimate in flexibility.
Finally, I believe the reason the call you made to AbandonAsync passing the properties you wanted to modify didn't work is that those properties are referring to Metadata properties on the method, not standard properties on BrokeredMessage.
I actually asked this same question last year (implementation aside) with the three approaches I could think of looking at the API. #ClemensVasters, who works on the SB team, responded that using Defer with some kind of re-receive is really the only way to control this precisely.
You can read my comment to his answer for a specific approach to doing it where I suggest using a secondary queue to store messages that indicate which primary messages have been deferred and need to be re-received from the main queue. Then you can control how long you wait by setting the ScheduledEnqueueTimeUtc on those secondary messages to control exactly how long you wait before you retry.
I ran into a similar issue where our order picking system is legacy and goes into maintenance mode each night.
Using the ideas in this article(https://markheath.net/post/defer-processing-azure-service-bus-message) I created a custom property to track how many times a message has been resubmitted and manually dead lettering the message after 10 tries. If the message is under 10 retries it clones the message increments the custom property and sets the en queue of the new message.
using Microsoft.Azure.ServiceBus;
public PickQueue()
{
queueClient = new QueueClient(QUEUE_CONN_STRING, QUEUE_NAME);
}
public async Task QueueMessageAsync(int OrderId)
{
string body = JsonConvert.SerializeObject(OrderId);
var message = new Message(Encoding.UTF8.GetBytes(body));
await queueClient.SendAsync(message);
}
public async Task ReQueueMessageAsync(Message message, DateTime utcEnqueueTime)
{
int resubmitCount = (int)(message.UserProperties["ResubmitCount"] ?? 0) + 1;
if (resubmitCount > 10)
{
await queueClient.DeadLetterAsync(message.SystemProperties.LockToken);
}
else
{
Message clone = message.Clone();
clone.UserProperties["ResubmitCount"] = ++resubmitCount;
await queueClient.ScheduleMessageAsync(message, utcEnqueueTime);
}
}
This question asks how to implement exponential backoff in Azure Functions. If you do not want to use the built-in RetryPolicy (only available when autoComplete = false), here's the solution I've been using:
public static async Task ExceptionHandler(IMessageSession MessageSession, string LockToken, int DeliveryCount)
{
if (DeliveryCount < Globals.MaxDeliveryCount)
{
var DelaySeconds = Math.Pow(Globals.ExponentialBackoff, DeliveryCount);
await Task.Delay(TimeSpan.FromSeconds(DelaySeconds));
await MessageSession.AbandonAsync(LockToken);
}
else
{
await MessageSession.DeadLetterAsync(LockToken);
}
}
I am trying to implement Paul Calhoun's Apache FOP solution for creating PDF's from Xpages (from Notes In 9 #102). I am getting the following java exception when trying to run the xAgent that does the processing --> Can't get a Writer while an OutputStream is already in use
The only changes that I have done from Paul's code was to change the package name. I have isolated when the exception happens to the SSJS line: var jce: DominoXMLFO2PDF = new DominoXMLFO2PDF(); All that line does is instantiate the class, there is no custom constructor. I don't believe it is the code itself, but some configuration issue. The SSJS code is in the beforeRenderResponse event where it should be, I haven't changed anything on the xAgent.
I have copied the jar files from Paul's sample database to mine, I have verified that the build paths are the same between the two databases. Everything compiles fine (after I did all this.) This exception appears to be an xpages only exception.
Here's what's really going on with this error:
XPages are essentially servlets... everything that happens in an XPage is just layers on top of a servlet engine. There are basically two types of data that a servlet can send back to whatever is initiating the connection (e.g. a browser): text and binary.
An ordinary XPage sends text -- specifically, HTML. Some xAgents also send text, such as JSON or XML. In any of these scenarios, however, Domino uses a Java Writer to send the response content, because Writers are optimized for sending Character data.
When we need to send binary content, we use an OutputStream instead, because streams are optimized for sending generic byte data. So if we're sending PDF, DOC/XLS/PPT, images, etc., we need to use a stream, because we're sending binary data, not text.
The catch (as you'll soon see, that's a pun) is that we can only use one per response.
Once any HTTP client is told what the content type of a response is, it makes assumptions about how to process that content. So if you tell it to expect application/pdf, it's expecting to only receive binary data. Conversely, if you tell it to expect application/json, it's expecting to only receive character data. If the response includes any data that doesn't match the promised content type, that nearly always invalidates the entire response.
So Domino in its infinite wisdom protects us from making this mistake by only allowing us to send one or the other in a single request, and throws an exception if we disobey that rule.
Unfortunately... if there's any exception in our code when we're trying to send binary content, Domino wants to report that to the consumer... which tries to invoke the output writer to send HTML reporting that something went wrong. Except we already got a handle on the output stream, so Domino isn't allowed to get a handle on the output writer, because that would violate its own rule against only using one per response. This, in turn, throws the exception you reported, masking the exception that actually caused the problem (in your case, probably a ClassNotFoundException).
So how do we make sure that we see the real problem, and not this misdirection? We try:
try {
/*
* Move all your existing code here...
*/
} catch (e) {
print("Error generating dynamic PDF: " + e.toString());
} finally {
facesContext.responseComplete();
}
There are two reasons this is a preferred approach:
If something goes wrong with our code, we don't let Domino throw an exception about it. Instead, we log it (instead of using print to send it to the console and log, you could also toss it to OpenLog, or whatever your preferred logging mechanism happens to be). This means that Domino doesn't try to report the error to the user, because we've promised that we already reported it to ourselves.
By moving the crucial facesContext.responseComplete() call (which is what ultimately tells Domino not to send any content of its own) to the finally block, this ensures it will get executed. If we left it inside the try block, it would get skipped if an exception occurs, because we'd skip straight to the catch... so even though Domino isn't reporting our exception because we caught it, it still tries to invoke the response writer because we didn't tell it not to.
If you follow the above pattern, and something's wrong with your code, then the browser will receive an incomplete or corrupt file, but the log will tell you what went wrong, rather than reporting an error that has nothing to do with the root cause of the problem.
I almost deleted this question, but decided to answer it myself since there is very little out on google when you search for the exception.
The issue was in the xAgent, there is a line importPackage that was incorrect. Fixing this made everything work. The exception verbage: "Can't get a Writer while an OutputStream is already in use" is quite misleading. I don't know what else triggers this exception, but an alternative description would be "Java class ??yourClass?? not found"
If you found this question, then you likely have the same issue. I would ignore what the exception actually says, and check your package statements throughout your application. The java code will error on its own, but your SSJS that references the java will not error until runtime, focus on that code.
Update the response header after the body can solve this kind of problem, example :
HttpServletResponse response = (HttpServletResponse) facesContext.getExternalContext().getResponse();
response.getWriter().write("<html><body>...</body></html>");
response.setContentType("text/html");
response.setHeader("Cache-Control", "no-cache");
response.setCharacterEncoding("UTF-8");
Background: I am using OmniThreadLibrary to load batch mode ADO stored procedures in the background. I am doing some slightly dodgy stuff by swapping the connection after opening the SP but that seems to be quite reliable. I'm using PostMessage to send messages back to the calling form and that works in my test applications. Primoz' comms channels work for me, I'm using those for inter-thread comms but for our main application I'm trying to avoid that dependency by using standard PostMessage calls as we do elsewhere in the app.
Problem: Unfortunately when I put this into our main application the PostMessage calls in the thread start failing with 1400:invalid window handle.
I have liberally added extra PostMessage calls and logging code to try to locate the problem, but I'm out of ideas now. The code is boilerplate:
const WM_PW_ADLQUEUEEMPTY = WM_USER + 11;
...
if PostMessage (OwnerHandle, WM_PW_ADLPROGRESS, QueueID, 10) then
pwDebugLog ('TADLQueue.Run WM_PW_ADLPROGRESS send to ' + IntToHex (OwnerHandle, 8) + ' (IsWindow '+BoolToStr(IsWindow(OwnerHandle),true)+') OK for Queue ' + IntToStr (QueueID))
else
pwDebugLog ('TADLQueue.Run WM_PW_ADLPROGRESS send to ' + IntToHex (OwnerHandle, 8) + ' (IsWindow '+BoolToStr(IsWindow(OwnerHandle),true)+') failed for Queue ' + IntToStr (QueueID));
But the log for a series of calls is not very revealing to me. note that the four hex digits after the time is the thread id from GetCurrentThreadID.
15:41:53.221 1614 TpwAsyncDataLoader.RunQueue WM_PW_ADLPROGRESS send to 00A5110C (IsWindow True) OK for Queue -6
15:41:53.265 13B4 TADLQueue.Run WM_PW_ADLPROGRESS send to 00A5110C (IsWindow True) OK for Queue -6
15:41:53.554 13B4 TADLQueueManager.WriteSysErrorMessageToDatabase Postmessage 00A5110C (IsWindow False) failed with 1400 Invalid window handle
Can anyone shed some light on this? I'm confused at how a window handle can become invalid while I'm looking at it, but that's what it looks like to me.
The one thing I can think of is that the form I'm showing here isn't processing messages and I'm seeing a "message queue full" failure rather than the IsWindow(handle) failure that it looks like. How can I test for that?
There are cases where a handle gets recreated, most notably when you change window flags. This might be what's happening in your application.
All I found so far about recreating windows handle is this post from Allen Bauer but I'm certain reading a more detailed one written by Peter Below. Unfortunatly I can't seem to find that one.
Finally, you need to be aware of cases
where your handle may need to get
recreated. This can happen if the
surrounding form or the parent
component's handle goes through a
recreate process. Up until more recent
releases of Windows, the only way to
change some window flags was to
destroy the handle and recreate with
new flags in the CreateWindowEx()
call. There are many components that
still do this. You know if you're in a
recreate situation by checking
(csRecreating in ControlState).
Edit
Not actually the posts from Peter I had in mind but it might give you some fresh ideas.
The form will not have a handle until
you show it the first time (unless
something in the form load sequence
request the handle) but the handle is
not destroyed when you hide the form
and unless you do something that
forces the form to recreate the
handle, e.g. change its border style
or border icons, or call RecreateWnd
yourself the handle will stay the
same.
It may not be desirable but it cannot
be avoided, at least not the way
Delphi drag&dock is currently
implemented. When you dock the dragged
form to another form it becomes a
control (with WS_CHILD window style)
and that means that its window handle
has to be destroyed and recreated with
the new style. And destroying the
window handle of a container control
automatically destroys the handles for
all child controls as well.
and
There is also the fact that the forms
window handle is destroyed and
recreated when you assign to its
Parent property. This also destroys
and recreates the handles for all
controls on the form.
I had a similar issue (but in VC++2010), and I did not find the solution on any forum, so I post it here, hope this will help:
Issue:
Creating a thread,
Passing the HWnd handle
In the thread, PostMessage
throws a 1400 error (invalid handle), although the pointer was equal
with the handle as seen from UI thread (with GetSafeHWnd()).
Solution:
Do not pass the handle, but the parent CDialog(Ex) class
This class has a m_hWnd member that will do the job
Here is a (Cpp) example, sorry for the cast mess.
// In the worker thread
ThreadParam *threadParam = (ThreadParam*)param
// This is ugly because my pointer is a void *, to avoid one more forward declaration
CCoreGenDlg *dlg = static_cast<CCoreGenDlg *>(threadParam->ptr);
// Post
bool b = PostMessage(dlg->m_hWnd ,1221,0,(LPARAM)message);
Cheers'