Sometimes I have environments, which can run into unhandled situations after choosing several bad actions, which yields exceptions.
Instead of crashing the agent, I'd like to just retry and ignore that episode.
How to achieve this?
From the docs:
Calling tune.run with max_failures=-1 as argument will try to recover a trial unlimited times e.g.:
tune.run(algorithm_name,
config=config_dict,
max_failures=-1)
Related
I am using the Firebase Admin SDK with Cloud Functions. The function does multiple writes to several Firestore collections, which need to be consistent.
Now i am not sure how the Firestore operation behave if a valid operation like a write to a document fails (maybe through cosmic radiation or something which is similar unlikely).
Does the operation instantly return an error or is there some kind of retry or error correction mechanism?
Maybe this is a silly question and has nothing to do with the SDK itself.
First of all, if you have multiple documents to write that all must land at the same time, atomically, you should be using a batch or transaction in order to make that happen. If any document would fail to write for any reason, then nothing will happen for any of the documents referenced. If you instead choose to do several write operations, you would have to figure out to reliably roll back each change individually, which is going to be a lot of work.
If you do get an error, I don't believe there are any guarantees about the conditions of that error. You would likely want to retry on your own, unless you're able to determine that the error is not transient. To make retries reliable, you could enable the retry configuration on the function, allow the error to escape the function (don't catch the error), and let Cloud Functions invoke it again for you.
It will throw an error. If you notice every method use has a callback with succes or error.
If you are using something like await on node, you should then try/catch
If you have more than one operation and the procedure should be atomical and/or all or nothing, then use batches
https://firebase.google.com/docs/firestore/manage-data/transactions
When a Functions encounters if the error is not handle then the Function crash, you can modify the retries for Functions
https://firebase.google.com/docs/functions/retries
I am launching page https://www.nasdaq.com/ . After that I am also waiting for 5 sec to load the page. After this I want to check whether the page exist or not, or gets loaded or not then throw the exception. So how and when to use exception handling in this scenario. see the image attached. I tried putting recover, resume, exception stage on launch stage as well as on wait stage. But I dont know where to put the exception.
1st of all, don't use arbitrary (fixed) wait stages until it's completely necessary. Use intelligent wait stages instead, which means wait for something to happen and then proceed or throw an exception if it times out. In your case, you can use intelligent wait stage for example to check if the website has been loaded.
When it comes to throwing an exception, in your case I would just simply launch, then wait for the document to be loaded and throw an exception if it times out. See below diagram.
Also, I would leave retry logic (recover - resume) for the process layer. Object should ideally contain small reusable actions and no business logic, so decisions if and how many times to retry should be taken in the Process.
I've been reading articles about methods/best practices to handle errors in node. Besides listening to process.on('uncaughtException'), which considered a bad practice, there is no method to properly handle exceptions troughout the entire application.
All the suggested solutions like using domains, try/catch blocks should be implemented per module (or worse if you use try/catch and not domains, per action).
Am i missing some article/documentation or domains/try-catch blocks are the best available solutions?
The best solution is to rely on asynchronous code.
Asynchronous code show not throw but rather pass the error to the Callback function as the first argument to be handled by the callee. The problem with a lot of modules is that people seem to throw whenever they think it is right to do so and this happens a lot more often than actual syntax exceptions, type exceptions or any exception is thrown by V8. If you don't want your app to crash in production use well-written modules that are truly async and rely on callbacks rather than try/catch and if you really need to rely on a sync module then wrap it around try/catch but there is almost always an async module out there that even though might not perform any I/O passes the error to the callback.
Moreover try/catch does have a performance overhead (albeit tiny) that would put strain on resources if overused (don't wrap every module in try/catch). From experience I suggest if you are developing something that goes into production then try to choose your modules carefully rather than catching their errors later on. process.on('uncaughtException') is a last resort and usually ideal for logging the exceptions.
As an ultimate solution I suggest that you use a process manager. My personal choice is PM2. It will attempt to gracefully restart your code and handles exceptions pretty well.
References:
Try/Catch performance overhead.
Try/Catch performance overhead (recent)
Using PM2 in production
Isaac Schlueter on async code and throw
I'm writing a command line tool for installing Windows services using Node JS. After running a bunch of async operations, my tool should print a success message then quit. Sometimes however, it prints its success message and doesn't quit.
Is there a way to view what is queued on Node's internal event loop, so I can see what is preventing my tool from quitting?
The most typical culprit for me in CLI apps is event listeners that are keeping the process alive. I obviously can't say if that's relevant to you without seeing your code, though.
To answer your more general question, I don't believe there are any direct ways to view all outstanding tasks in the event loop (at least not from JS-land). You can, however, get pretty close with process._getActiveHandles() and process._getActiveRequests().
I really recommend you look up the documentation for them, though. Because you won't find any. They're undocumented. And they start with underscores. Use at your own peril. :)
try to use some tools to clarify the workflow - for example, the https://github.com/caolan/async#waterfall or https://github.com/caolan/async#eachseriesarr-iterator-callback
so, you don't lose the callback called and can catch any erros thrown while executing commands.
I think you also need to provide some code samples that leads to this errors.
Usually an access violation terminates the program and I cannot catch a Win32 exception using try and catch. Is there a way I can keep my program running, even in case of an access violation? Preferably I would like to handle the exception and show to the user an access violation occurred.
EDIT: I want my program to be really robust, even against programming errors. The thing I really want to avoid is a program termination even at the cost of some corrupted state.
In Windows, this is called Structured Exception Handling (SEH). For details, see here:
http://msdn.microsoft.com/en-us/library/windows/desktop/ms680657%28v=vs.85%29.aspx
In effect, you can register to get a callback when an exception happens. You can't do this to every exception for obvious reasons.
Using SEH, you can detect a lot of exceptions, access violations included, but not all (e.g. double stack fault). Even with the exceptions that are detectable, there is no way to ensure 100% stability after the exception. However, it may be enough to inform the user, log the error, send a message back to the server, and gracefully exit.
I will start by saying that your question contains a contradiction:
EDIT: I want my program to be really robust, ... The thing I really want to avoid is a program termination even at the cost of some corrupted state.
A program that keeps on limpin' in case of corrupted state isn't robust, it's a liability.
Second, an opinion of sorts. Regarding:
EDIT: I want my program to be really robust, even against programming errors. ...
When, by programming errors you mean all bugs, then this is impossible.
If by programming errors you mean: "programmer misused some API and I want error messages instead of a crash, then write all code with double checks built in: For example, always check all pointers for NULL before usage, even if "they cannot be NULL if the programmer didn't make a mistake", etc. (Oh, you might also consider not using C++ ;-)
But IMHO, some amount of program-crashing-no-matter-what bugs will have to be accepted in any C++ application. (Unless it's trivial or you test the hell out of it for military or medical use (even then ...).)
Others already mentioned SEH -- it's a "simple" matter of __try / __catch.
Maybe instead of trying to catch bugs inside the program, you could try to become friends with Windows Error Reporting (WER) -- I never pulled this, but as far as I understand, you can completely customize it via the OutOfProcessException... callback functions.