This is more of a pragmatical question in regards to implementing pagination with the usage of Cassandra. I have read the documentation in regards to the Page state and that it should not be exposed to end user as it is not encrypted and creates the possibility of unwanted tampering.
So, with all that said, my question is what are the pragmatics and best practices in terms of using the PageState object when implementing an API that needs to fetch the next records in line upon scrolling?
After re-reading your question, I think I understand now what you're trying to achieve.
You are correct in saying that you shouldn't expose the page state to end-users of your application. I don't think there will be anything catastrophic if the page state is modified/tampered by a user but if they do, it will either return unpredictable results or be unusable and the driver will throw an exception.
On your question of what is best practice when implementing an API to retrieve the next "page" of results, I would recommend separating the implementation of your API from the logic which requests the records from the DB.
For example, don't use the driver's page state in your API but instead implement an option page=next so the HTTP endpoint looks like /api/users?page=next. Cheers!
I have an application that connects to an SQL Server Compact Edition 3.5 database. The application will randomly fail, throwing a SqlCeException with the message "bad hash." What is causing this exception and how can I prevent it? I can't find any information on google that might explain what's going on. The problem seems to be fixable with a reboot, but I would like to know more about what causes it and how to prevent it.
System.Data.SqlServerCe.SqlCeException: Bad Hash.
at System.Data.SqlServerCe.SqlCeConnection.ProcessResults(Int32 hr)
at System.Data.SqlServerCe.SqlCeConnection.Open(Boolean silent)
at System.Data.SqlServerCe.SqlCeConnection.Open()
How can I handle unexpected errors in my Express app to avoid downtime?
My first thought was to use try/catch statements to handle unexpected error, but that only works with synchronous code. I found that one possible option was to use Node.js's 'Domain Module', however, it turns out, that it is deprecated, and will remain deprecated.
Currently I am handling errors, by using middle-ware that is provided by the express generator. I am also using a process monitoring program called forever. Is the process-monitoring and Express's middle-ware enough to for successful unexpected error handling, or are there other options that I should be considering?
To answer you bluntly, you cannot handle your unexpected errors. This is not to say you can not handle errors, obviously you can, should and must. What I am saying is if an error is handled, then the program expected it, and it is not unexpected, if the program did not expect it then it is unexpected, so essentially if you do handle the unexpected errors, your not handling unexpected errors, your just handling errors.
As developers it is our job to thoroughly element unexpected errors as best we can, though none of us will ever succeed completely. Through devotion, and rigorous effort, a level can be achieved where the unexpected errors in your code that do occur are very minimal, and the situation is tolerable, at least to a certain degree. Error handling is a process that will often continue long after a beta version release, and even long after the first real production release. As data is collected, in a way I will soon talk about, bugs/errors are found and eliminated through versioning & patching, techniques designed to continue increasing the integrity of software even long after it has been released.
As you should already understand, there is not one single fix that will take care of all your unexpected errors. You as a developer should rid your app of all programmatic errors before production, and handle operational errors as best you can, which means you should understand what a Programmatic error, and operational error is.
#1. Programmatic-Errors: errors and/or bugs within the code itself.
#2. Operational-Errors - "Errors that occur in areas that are outside of the control of the programmer. Errors caused be the client, by the network, by the V8-Engine(this can be Programmatic too though).
There is no clear cut way to define every error as either operational or programmatic, and grey areas do exist. There are ways of looking at the errors in question that can help make a definitive determination. If you look at your code and figure that you couldn't solve the error, by writing different code, or more code, its likely not a programmatic error. If your getting errors trying to connect to a particular host, and your connecting everywhere else just fine, it is likely operational. Operational errors occur, even when the code of the application is written flawlessly, just make sure your correct in your determination of your codes integrity.
Here is a couple examples:
OPERATIONAL ERRORS
Socket Drops
Request timeout
Failed to authenticate client credentials
Any errors caused by legality issues
Here is a couple examples:
PROGRAMMATIC ERRORS
Invalid type-of variable passed into a function
Incorrect/Invalid Syntax
Failed to authenticate client credentials
Name clashing
As I already pointed out, the purpose of understanding these two error types, is so you as a developer can eliminate all the errors, that you are not only responsible for eliminating, but you need to eliminate in order to have a half decent working application. In the world of Node.js Application Development, this is the part that separates The Hero from the Zero, the Women from the girls, Glorious Chihuahuas from Sasquatch (Okay I am bias towards chihuahuas) You get the Idea.
This is really not a Question or Topic that can be answered in a one liner, mainly because, you can't just handle all unexpected errors with some 3rd party NPM API, but also because learning, and knowing what to do is not a matter of learning more JavaScript syntax, but instead learning a paradigm that every developer who wants to build solid, well built Node.js applications needs to know.
I know some of you are talking smack right now, thinking you have the answer and it is so simple. Well its not. Those of you who are using, the event listener that catches uncaught/unexpected errors are not putting great applications into production, if any applications. I sure hope any company you work for doesn't let you put out code that uses it. For those who don't know what I am talking about it bellow this text block. FYI, its already been suggested on this page, which blows me away.
//Catch uncaught exceptions
process.on('uncaughtException', function (err) {
// handle the error safely
console.log(err);
});
ABOVE IS THE PERFECT EXAMPLE OF WHAT NOT TO DO!!!!!!
If an error occurs unexpectedly the right thing to do is throw it as an Error Object. Not doing this threatens the integrity of your entire application. Remember you will be away from your server when clients have to deal with the results of your failure to do what you were told was the right thing to do. Using the above code snippet threatens the Security of your server. This includes the information needed to access your server, your pem keys, your passwords, all your databases and databases data, and worst of all any client info you store including any credit cards and passwords you keep (make sure you Blow-fish that kind of stuff). Every thing is put at risk the minute you launch an app that handles unexpected errors this way.
possible issues include:
Memory references, that are used and passed around on the V-8 c++ level, can potentially get majorly screwed up. This could result in massive memory leaks. You may end up without the necessary memory to fulfill requests, or do anything for that matter. Memory leaks are a major topic right now in the node community, as developers are having a heck of a time debugging them.
Some transactions can get left open, especially during memory leaks, and the wrong request and data can get sent to the wrong users. Random people can wind up getting sent the session ID's of other people. This is something hackers will quickly take advantage of, especially if there's any finances involved.
There are actually several more, and memory leaking is a big problem despite making it worse to try to make another problem better. You can make both better, you as a developer just need to have integrity, thus your app will have integrity.
The reason that event even occurs is for logging and monitoring. It is actually implemented by default in each node instance ran, and is part of how Error objects manage to inform you of, and about, unexpected errors. Below is the correct way to use it.
process.on('uncaughtExceptionMonitor', (err, origin) => {
MyMonitoringTool.logSync(err, origin);
// if needed pass the error to a function that will throw it...
});
The above is an example from the Node.js website, where you can read further about error handling, and unexpected errors.
Node.js Official Error Handling Documentation
Lastly we talking about Error monitors. For small applications, error monitors maybe all you need in order to do a satisfactory job. The only way to know if you have done enough for errors & unexpected errors to go production, is to get to the point where you feel you have error handling for all your operational errors, and your programmatic errors have all been corrected. Then you need a logging system that logs your errors. You can build your own logging system or purchase one for a regular fee. You will need a monitor, again you could technically build one, and for most stuff I would suggest trying, but for this I would pay a very small monthly fee, and it will restart your application if it crashes due to unexpected errors.
The answers above were really bad, but this has some good stuff in it. I did get help from the Node site that I liked to above. I also got help from here, a resource I have referenced time and time again, you should too.Handling Unexpected errors in Node Applications
I would recommend pm2 over forever as it has several great features. We use it in our production website and it works great. Enable logging to track your exceptions.
http://pm2.keymetrics.io/
PM2 allows you to manage logs easily. You can display all application logs in real-time, flush them, and reload them. There are also different ways to configure how PM2 will handle your logs (separated in different files, merged, with timestamp...) without modifying anything in your code.
I have handled this in my application by listing to uncaught error on process in my "index.js".
//Catch uncaught exceptions
process.on('uncaughtException', function (err) {
// handle the error safely
console.log(err);
});
You can use try-catch using a combination of generators and promises. - https://gist.github.com/swarajgiri/16202e32aa4d80d45c62
On catching an error, pass it to an error handler, in your server/app.js.
// errors handler.
app.use(function (err, req, res, next) {
if (! err) {
return next();
}
res.status(500);
res.send('500: Internal server error');
});
You will still need to have either forever or pm2 manage it.
I'm attempting to have a slightly more organized openswan logging system and I think I've over thought it. I'm really just trying to save username and public IPs. This is what I so far but it's still returning more data than I need and I'd rather not build a complicated parser. I imagine there has got to be a way to get just this info without getting to overly complicated.
ipsec auto --status | grep 'STATE_MAIN_R3' > /home/ipsec.log
Please help!
providing which protocol you're using for authentication will help
Since you mentioned "username and public IPs" I'm guessing you're doing some kind of road worrier IPSec connection. if this is true then with openswan it is usually L2TP/IPSec. In this case you will not see the userid in the "ipsec auto --status" since the user authentication is done by the L2TP/PPP stack. if you're using xl2tpd then it usually logs into /var/log/syslog
Hope this is helpful
We are trying out CQRS. We have a validation situation where a CustomerService (domain service) needs to know whether or not a Customer exists. Customers are unique by their email address. Our Customer repository (a generic repository) only has Get(id) and Add(customer). How should the CustomerService find out if the Customer exists?
Take a look at this blog post: Set based validation in the CQRS Architecture.
It addresses this very issue. It's a complex issue to deal with in CQRS. What Bjarte is suggesting is to query the reporting database for existing Customer e-mail addresses and issue a Compensating Command (such as CustomerEmailAddressIsNotUniqueCompensatingCommand) back to the Domain Model if an e-mail address was found. You can then fire off appropriate events, which may include an UndoCustomerCreationEvent.
Read through the comments on the above blog post for alternative ideas.
Adam D. suggests in a comment that validation is a domain concern. As a result, you can store ReservedEmailAddresses in a service that facilitates customer creation and is hydrated by events in your event store.
I'm not sure there's an easy solution to this problem that feels completely clean. Let me know what you come up with!
Good luck!
This issues does not have to be that complex:
Check your reporting store for customer uniqueness before submitting the UpdateCustomer command.
Add a constraint to your DB for uniqueness on email address. When executing the command, handle the exception and send a notification to the user using a reply channel. (hences never firing CustomerUpdated events to the reporting store.
Use the database for what it's good for and don't get hung up on ORM limitations.
This post by Udi Dahan http://www.udidahan.com/2009/12/09/clarified-cqrs/ contains the following paragraph:
"Also, we shouldn’t need to access the query store to process commands – any state that is needed should be managed by the autonomous component – that’s part of the meaning of autonomy."
I belive Udi suggested simply adding a unique constraint to the database.
But if you don't feel like doing that, based on the statement above, I would suggest just adding the "ByEmail" method to the repository and be done with it - but then again Udi would probably have a better suggestion..
Hope I am not too late...but we faced a similar situation in our project, we actually intercept the command executor and attach it with the set of rules created for that command, which in turn uses a query to fetch the data.
So in this case, We can have a class by the name, CustomerEmailMustBeUniqueRule which is fetched by the RuleEngine when the command "RegisterCustomerCommand" is about to be executed by RegisterCustomerCommandExecutor. This rule class has the responsibility to query the database to find if the email id exist and stop the execution by raising the invalid flag...