How can I collect user feedback without physically seeing my users? [closed] - user-experience

As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 10 years ago.
Like probably a lot of software developers, I almost never see real users using my software.
It is, of course, quite difficult to get good user feedback in this situation. Even if some users agree to give me some piece of information about the way they use the software, there's a huge difference between how they really use it and how they think they use it.
By chance, my software is client/server, which means I can quite easily technically collect some information on the server.
Of course, nothing equals looking at a real user using the software in real life, but I think it's better than nothing, or at least it's worth trying :)
While I log all the exceptions raised on the client in my database, I've not been beside this point yet.
Has anyone does that before?
What information would you log?
Are there some legal issues? How should I deal with those?

I face the same problem with the software I'm developing, though I have no users for it yet.
I generally think that monitoring should always be opt-in, and that you should have the ability to review before materials are being sent. I think most people would agree to that.
However, from a legal standpoint there are greater issues at stake. Some companies restrict users in installing software that has any components that "call home" for security reasons. Depending on the usage context, any monitoring data can potentially reveal secrets.
For example, my software annotates things in the IDE. If I transmitted "home" details about files that are open (rather than hashes), even without the content of these files, I would still possibly be sending confidential details. If your tool can be used to open images or documents, there may be similar issues.
I would suggest hashing or finding way of obfuscating results on the client side, and ensuring via sufficient tests that there cannot be a situation where your software sends information home without consent and obfuscation. If I'm not mistaken, if your software does so, even by mistake, you may be violating US federal laws.
Also, make sure to encrypt the details as you send them over the wire.
Finally, if some of your users are in the EU, where privacy laws are stronger, your database of exceptions may be legally considered a "database" in itself (e.g., if you store SQL statements as they were executed and failed and these contain production values). So you may have to follow a lot of the rules about personally identifiable information.

When I did UI development, I used to collect every user command (button push, menu selection) and log them to file with my own internal debug information, but auto-delete the log files after a few days. This information is invaluable when trying to debug your own software (user can rarely recall precisely the steps they took when a problem occurs). I also kept a record of every application startup, in case we had a compatibility problem with third party software.
The point is that the information wasn't used unless a problem did occur, it was kept locally with no remote access, and it automatically got deleted if there was no problem. Only if the customer called us in for a problem did we access the log data.
Actively tracking user operations and sending them back to base is a separate issue entirely, and I've always shied away from that.

This isn't exactly what you asked for, but you do have a few options that are not programming-related solutions:
1) Do some hallway usability testing (scroll down to #12).
2) Try a product like Morae to set up a more formal, but remote, viewing session.
3) Ask a client to watch over their shoulder, using something like GoToMeeting, CoPilot or WinVNC. Or go to their site for a day and hang out watching over their actual shoulder.
Any of these will give you a really good idea of what works and what doesn't.

You could do something like this, which captures mouse movements and replays them for you to see using javascript and ajax.

Related

Spoofing online activation protection [closed]

As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 11 years ago.
Ok I want to use online activation to protect my software. The users will need an internet connection and need to verify that they have an active account/license to continue using the software. So my software will send out some credentials to my website, where I will verify and send back SOMETHING to allow the program to run or not.
Here's the problem. How do I prevent someone from spoofing the simple SOMETHING that I send back, bypassing my verification altogether?
Two examples:
I remember seeing a program somewhere that allows you to monitor internet traffic and modify it before the traffic returns to the software. I cannot remember the name. In effect, its a man-in-the-middle between what goes out and comes into your computer, before the application sees it. For example, your browser sends an HTTP request, this program can intercept it and modify the out packet, or vice verse, some HTTP webpage is returned to the browser, and this program can intercept and change the incoming packet and then pass it along to the browser.
The HOSTS file can be changed to redirect traffic to any address. So if someone knows that my verification will return the text "ok_to_use" or "not_ok_to_use", and they know the url that I use to verify, then they can just use the HOSTS file to bypass dns lookup of my website, redirect it to their own site, and always return the string "ok_to_use"
The second example illustrates the problem. I think I need to somehow return something other than a static string. Obviously I can encrypt the string, but it won't take long to figure out that "ok_to_use" equals "7hgx3a" if that is the same encrypted string that always gets sent back. So I would need to send something back that changes all the time.
EDIT/
To the comments/responses saying that there will always just be a final if/then check in the app that flags whether the license is valid or not:
Yes thats true. I was hinting towards something like this in the last paragraph of the original post. Perhaps having the web php send back some critical functionality, so that even if the if/then is skipped, if the web check wasn't performed, then the algorithm for some function won't get downloaded, and the app won't work correctly. Anyone have ideas in this vain?
Although I basically agree with Gavin, but this is not the reason to leave your software prone to be hacked by the most stupid and noob hackers ever!
Well, you have many options to prune a large portion of the script kiddies and probably customer employees from the population that threaten your software's licensing mechanism.
I list some of them in the order they cross my mind:
You can use two-way SSL for your client-server connections, it has a little coding overhead but you can make sure that you have sold that version of software that is contacting your licensing server already. You can identify it by the public key that decrypts the sent licensing query correctly.
You can send more complicated tokens than OK_TO_USE, perhaps a value of a mathematical function that returns a number randomly every time it is asked about. Then the client software can be written so that identifies if the number is produced by that specific function.(this can be a more complicated thing than a single integer!)
At least use encryption! or register a certificate so that your client can verify that it has contacted the correct server.
Update your software in a timely manner and change the licensing mechanism every now and then!
Finally, no bullet-proof vest for a software license! you'll get cracked soon or late, but try to sell more and get hacked later. :D
I personally wouldnt bother - there are plenty of different copy protection systems used by professional software products, and every single one of them have been cracked.
No matter how complicated your copy protection system is, at some point in the binary you will have a check to see if the license is valid, all that is needed is to modify this part of the binary to always return true.
Just add a very simple activation system to deter people from passing it to friends, and then spend your time adding new features in order to sell more copies, rather than wasting time trying to prevent professional hackers from copying your software.
If they want to - they will!
You can use public/private key protection when sending messages from the server to the client. These messages cannot be altered and you rest assure the response came from the server.
You can check LicenseSpot that already provides what you're looking for. It has:
Free Licensing Component
Private/public key encryption
API to integrate your app and online store
Serial number generation
Revoke licenses

ColdFusion Security [closed]

As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 10 years ago.
What are the best practices for securing a coldfusion webpage from malicious users? (including, but not limited to, sql injection attacks)
Is cfqueryparam enough?
I use a modified portcullis, and filter all incoming var scopes (URL,FORM,COOKIE) onRequestStart.
http://portcullis.riaforge.org/
Pete Freitag has an awesome blog, especially this post on Hardening ColdFusion
Never trust the client.
The most ColdFusion specific "set and forget" is following server administrator hardening guidelines noted above, keeping the server up-to-date, and following ColdFusion on twitter to learn about any new issues immediately.
For app security, which is common across all languages, you should validate every piece of information that touches your server from the client. Forms are are obvious areas of tight control, but don't forget about URL parameters that you might use for application state management or control. Something like &startRow=10&tag=security which isn't "supposed" to be touched by the user is user input. Even if your application could never break with invalid data, you might not know how that data will be used in the future. Validation could be as simple as ensuring that someone isn't entering a 100 character long first name and doesn't contain programming characters or ensuring that &startRow is always a number. These are the little things that application developers sometimes skip because everything works OK as long as you are using the software as expected.
I believe you can look at the Sony Playstation hacking as an example. Unfortunately, they didn't expect someone to hack the client (playstation console) and manipulate the PlayStation console software to hack the server. The server trusted the client.
Never trust the client.
I would say best practices for ColdFusion are similar to those for programming web applications in any language.
I recently read Essential PHP Security Chris Shiflett and the majority of issues discussed affect ColdFusion as well, though the syntax for dealing with them may be slightly different. I expect there are other (possibly better) language agnostic books which contain principles which can easily be altered for use in ColdFusion.
Although using a prebuilt solution will work, I recommend knowing all the possible issues that must be protected. Check out Hack Proofing ColdFusion at Amazon.
Another great place to learn about security (and all kinds of other topics) is to check out Charlie Arehart's massive list of recorded user group presentations: http://www.carehart.org/ugtv/
Here is information on a good tool that can be used to prevent XSS.
https://www.owasp.org/index.php/Category:OWASP_AntiSamy_Project
http://www.petefreitag.com/item/760.cfm
Fairly easy to implement and Java based.
I recommend you the excellent talk by Justin McLean "ColdFusion Security and
Risk Management". It includes a case study.
PDF presentation http://cdn.classsoftware.com/talks/CFMeetupSecurity.pdf
Video streaming: http://experts.adobeconnect.com/p22718297
CfQueryParam is very important, but not nearly enough.
There is a boxed solution we use at my work: http://foundeo.com/security/. It covers most of the bases. And even if you don't want to buy it, you can take a look at it's feature set and get an idea of the things you should be considering.
You may like to check -
http://help.adobe.com/en_US/ColdFusion/10.0/Developing/WSe61e35da8d3185183e145c0d1353e31f559-8000.html

Easiest way to recommend users to upgrade their browser? [closed]

As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 9 years ago.
There have been a lot of buzz around IE6NoMore campaign.
What's the easiest way to recommend users to upgrade their browser?
Use conditional comments targeting IE. This way other browsers/bots (even when masquerading as IE) won't see it.
Make sure the message is polite and unobtrusive. Focus on benefits of change ("our website is faster and easier to use in up-to-date browsers") rather than negative aspects of IE (otherwise users could interpret this as ridicule of them or anti-Microsoft fanboyism).
Be as non-technical as possible. Average user doesn't know the difference between Internet, browser and search engine (← this video is painful to watch).
You can recommend things to people until you're blue in the face, but the reason people who don't instinctively upgrade their browser actually get around to doing it will always be the same. They do it because there's something they want to use, but can't unless they upgrade.
Want people to quit using an old browser ?
Quit supporting it.
It's that simple.
If no one supported them anymore then you wouldn't have to write code for them. For a certain percentage of people, they upgrade through necessity, not out of choice. If IE6 keeps working then they will keep using. Dont make the extra effort to support IE6.
From a psychological point of view, users will upgrade software due to one of the following reasons:
- Need (they have to have feature)
- Benefit (they'd like to have a feature)
- Fear (they perceive some sort of damage if they don't have a feature)
Based on how well you know your users, you can use one of these approaches to motivate them to upgrade. I.e., Provide extra functionality in your software that works only in new browsers (suggestion: use Ajax for better page update, without the need to call another page or refresh the current). Make those users aware that new, better, safer functionality is available, if they just upgrade their browser.
But remember the following caveats:
Not everyone can upgrade on his own. OS and browser versions are often dictated by IT policies in medium-large organizations.
Not everyone is tech-savvy enough to upgrade - you may need to explain the process.
If the user can dictate behavior, he may require you to provide the shiny new functionality in the old browser - be careful how you present those benefits!
The type of people who haven't yet upgraded their browser will almost certainly be people who don't have the technical knowledge to do so; those users who do know what a browser is, does and how to upgrade (and the benefits of upgrading) will most probably have done so. From this you can see how important it is to explain clearly and concisely both the benefits of upgrading and precisely how to do so for those who aren't technically minded.

issue/defect tracking software [closed]

As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 10 years ago.
Our group is currently reviewing our toolset and looking for new defect/issue tracking software in additional to source control, and project management software.
For issue tracking, we've looked at bugzilla, fogbugz, bugtracker.net, sourcegear fortres, and bugnet.
I'm not satisfied with the list we've come up with, so I'm curios to know what others are using.
We're looking for Active directory integration for security, although we'd settle for a windows app, a web interface may be preferential, visual studio integration is also a bonus. We need to prioritize defects, mark the version the defect was found in, mark the version the defect was fixed in, and hopefully be able to maintain a discussion around each issue/defect. We'd also like to categorize items as defect, enhancement request, etc. and document workarounds for defects.
Very similar question:
https://stackoverflow.com/questions/101774/what-is-your-bug-task-tracking-tool
Try Unfuddle. If you use their version control hosting (SVN and Git options) with their issue tracker, you get some good integration stuff going on. For example, you can enter a note in your commit message such as "fixes #384: Too much foo in the bar"*, and you not only get that turned into a hyperlink to the issue, but it also marks the ticket as fixed with a link back to the changeset. All good stuff. This is a web-based solution that is hosted by Unfuddle themselves, in a SaaS-type fashion.
Other than that, +1 for Trac which I've used in the past and like very much. It's quite an immature project feature-wise, although it's got a very healthy community around it that has developed plug-ins to do a lot of extra stuff (like the AD authentication you wanted). It also has similar integration with a number of source control systems, but it's much less feature-rich than the Unfuddle stuff. That is to say, you get to use an extended wiki syntax in your commit messages which is parsed by Trac when it's display to create links. It doesn't do any of the two-way stuff that Unfuddle does. Trac is available to host in-house; alternatively, if you want it hosted, there's a list of places that will do so on Trac's wiki.
*I can't remember the exact format off the top of my head.
On our current project, we've amazingly used 6 different tracking tools (2 versions of PVCS), mostly commercial. Here's my opinion on the ones that we've used. I've listed them in order of my most favored to least.
Serena Teamtrack - We use a web client. The interface is intuitive. Performance will vary across installations, but comparing with our same data in each tool, this works the fastest. It also works in Firefox.
HP Quality Center - This is also web based, but it is IE only. On the upside, it's well organized, easy to use, and full-featured. It has reasonable performance for us as well.
It has an odd feature where there isn't a save button. It saves automatically for you. To force a save, you have to navigate to another ticket. Also when you first use it, it has to install so many DLLs that it is practically a thick client. That being the case, IE sometimes gets locked up (usually when trying to reinitialize a session after session expiration). Once locked up, you occasionally have to kill IE to regain control.
Bugzilla - I didn't use this as thoroughly as the other tools, so this isn't a fair comparison. We used it briefly for some internal tickets. I suppose the big upside is the (lack of) cost. IMO, I just didn't find the interface as nice and easy to use as the other tools. Its been awhile so I apologize for lack of specifics for why I'm relegating it below the others.
Siebel - There wasn't much to like about their defect tracking tool apart from that it is better than PVCS. The interface seems hokey. It's as if the Siebel interface has a set of user interface controls and it tries to force all square pegs into its round holes. Another downside is that it uses lengthy generated IDs so its hard to reference them or search by them. Along with that, the ticket IDs aren't sequenced.
Merant PVCS - We had separate databases and used both the web client and thick client. Its been awhile now, so the details are fading. I recall there were bugs in the tool and they weren't getting fixed, for instance reports couldn't display certain fields. Performance was bad. It took a long time to load. It was slow to navigate through tickets.
Issue tracking for support is a different problem from tracking issues during development.
Trac http://trac.edgewall.org/ is a very capable tool which supports a number of large open source projects. You can find Trac hosting at places like http://www.wush.net
If you need more workflow and custom security, you'll want to look at JIRA which is from Atlassian http://www.atlassian.com. Atlassian has a number of products which you might also find useful.
For Issue tracking in a support setting, try RT http://bestpractical.com/rt. RT is deceptively simple, but I've seen it used in the largest environments and it does a good job making sure you are accountable to every you make a support commitment to.
An off-site (www) hosted solution with all the features you mentioned is NetResults Tracker
We use bugzilla, it suits us perfectly. We haven't investigated too many others because honestly it does everything we need and then some.
We don't use Visual Studio so I can't speak for integration compatibility.
Try out HappyFox ( http://www.happyfox.com), an issue and bug tracking software. The clean interface and automation features help you track and resolve bugs smoothly. HappyFox is free for a 2 member and priced economically for larger teams.

Is it ethical to monitor users? [closed]

As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 10 years ago.
I didn't know I would be getting too many replies so fast. I can provide more details. It is definitely for use within the company. I was looking for some info on whether I should be more careful or anything to watch out for...
My boss is asking me to put some tracking info on what users do with their application. It is not about collecting sensitive data but there might be some screenshots involved and I am not sure if this is a breach of privacy.
Would love to hear any thoughts on this or if you handled something similar.
At work, there is no privacy. Think of it this way, if you work for a financial institution, or a government one, monitoring users may be the difference between keeping sensitive information secret and not. (I want my personal information kept private). They are paid to do work at work. If they are afraid about what they are doing is wrong, then they shouldn't be doing it.
A comment brought up a good point. If you are selling the product and spying on end users, that is totally different. That is highly unethical to take screen shots and report them back to the company. Actually where I work, we'd have you arrested for it if we found out. (yes, you'd be violating a federal law, and I guarantee we'd go after everyone and sort out the mistakes later.) That is a very slippery slope.
If you mean users at large, yes it's a breach of privacy.
If you mean users internal to your company (workers), then no -- there should be no expectation of privacy in the workplace.
Sometimes it is good to collect some metrics and will help in enhancing the user experience. Once, we were able to prove that a certain functionality was never used and we were able to remove support for it. For screenshots, you should be careful to take only the required window instead of a full screen.
If the application is used internally within your organization, and you have a corporate policy that states "no expectation of privacy" that has been communicated to and signed by your users then there is no issue.
Monitoring the actions of employees within a business in the US is very common practice.
Legal issues aside, do you want to work at a company that takes screenshots of your desktop?
Even if legal, this behavior is sure to drive away developers. Remember, in a bad work environment often the best developers leave first; they have the best job prospects.
Here's a corollary example: would you want your boss taping and listening to phone calls you made from the office? You don't give up every right you have just by cashing a paycheck.
Even if this screen capture methodology is legal, it certainly isn't ethical and will absolutely damage the morale of employees by demonstrating that they cannot be trusted.
It's just a bad idea. There have got to be better ways of accomplishing your goals than this.
Screenshots? If it's not opt-in, I'd say that's a pretty clear breach of privacy.
I made a simple CMS in PHP and I had to store all actions of users, but it's a completely different situation. In my opinion what is asking your boss is a bit out of privacy, especially if in your application you don't mention to the user this kind of behavior.
On a work machine? Absolutely; as long as the users know the extent to which they are being monitored. It's their choice to work for the employer, and they are using the employer's equipment.If you don't notify them that they are being watched, then that is kind of a "grey area"....depending upon state lawss, it may even be illegal - depending on what sort of information you are monitoring.
Something that would help on clarification would be is this an internal company application or something that will be on user's personal computers.
Typically when it comes to computers that are owned by the company, if the company decides to do monitoring, it is their choice. Disclosure of the monitoring is often encouraged in an effort to be open and honest, but is not mandatory. A user should not have any expectation of privacy when using equipment owned and managed by the company.
This is not just a matter of custom built applications, but also web browsing, email, phone conversations, etc. If you are using company resources then you are releasing your privacy.
If this is an application going to users outside of the company, then yes it is wrong without permission by the users.
That is greatly depending on the country you are in and what information you are collecting and what you do with it.
There is a huge difference between the US and EU for instance.
The Law, jurisprudence, union contracts and company policy (when not in contradiction to the above) are what determines what is acceptable.
If its for an internal app its completely ethical.
Beyond disclosing to all users that their use of the apps is monitored there is no other obligation of disclosure(excepting federal contracts and union contracts).
What is most important about capturing this kind of data is to focus on capturing the absolute least amount necessary - capturing screenshots of all open windows plus any adjacent data streams does in fact incur liability issues (think HIPPA) as well as producing a mountain of data that no one will ever look thru until a lawyer requests it with a subpoena and you're asked to go thru it and redact all Names, DOB, and SSNs in 160GB of data.
Seems this has already been answered, but it should be noted that there are countries where this is illegal, even at a place of work.
For instance, in Switzerland it is illegal to track which websites each user has been visiting.
Other than specific laws to the contrary, I would agree that it is acceptable to do, since there should be no reasonable expectation of privacy at the workplace. That said, informing the users is the right thing to do.
One other caveat, if the data you are collecting is sensitive enough that an attacker would have use of it (say, the screenshots include CC numbers), then you must ensure that this information is well protected. (I'm not referring to the user's information, but say the bank's clients' account details.)
If it is done without the user's consent, then it is definitely a breach of privacy. Even with the user's consent, it must be made clear exactly what information is being passed back. If the screenshot was to grab the whole screen, not just a window, then you could potentially get all kinds of private info.
Is this an internal app or a something for the public? If it's internal, it's not unethical, even if it's scummy, to monitor users.
If it's something for the public, in order to not be sleazy:
the user has to be able to opt-out
no personally identifying data can be collected
only data about your app (not screenshots of the entire screen) can be collected
It really depends on exactly what is being collected, the disclosure, and if the program could be opted out of. If that passes the smell test, then ensuring the reporting does not provide an attack vector and the data is appropriately safeguarded becomes your concern. If things seem shady get some written 'feature request' to CYA. The basic idea, if done right, is nothing new. Microsoft, for instance, does it with some of their products.
In a work environment, I think it is OK as long as all employees know that they may be monitored. I've seen places (Intuit was one) where employees are tracked all day. Not my cup of tea, however.
In government facilities, there is typically some sort of login screen that states that anything and everything done on that machine is subject to monitoring.
If these are applications that are run by the general public, I'd say that it better be crystal clear that you are collecting data on them. Personally, I'd rather not have programs 'phoning home' with info about my activities, boring as they may be.
If the client is external, this should be disclosed to the client. Actually, if the client is internal OR external, if you do not disclose it, it is totally unethical.
An employment agreement that states that there can be no expectation of privacy constitutes disclosure.
Screenshots? If it's not opt-in, I'd
say that's a pretty clear breach of
privacy.
you've opted-in by cashing your paycheck :)
as many indicated, informing the user is the best the company can do. Informing, not asking to Opt-In.
I would suggest reading:
Privacy. My interpretation is that people will expect some things to be kept private such as their personal information. By interacting with your sites, users are sharing information with you that you should be able to use but not distribute or abuse as if it was your own.
Screen shots is obviously the hot button issue here. While users entering information into a text input field are knowingly giving you information, screen shots go beyond what a typical user would expect and therefore should be disclosed to the user through a privacy policy.
Collecting anonymous usage should be doable without screenshots.
If your app collects any data that is meant to be protected by privacy laws, then you will have to treat the screenshots as containing sensitive information and protect them accordingly. Data protection laws are pretty strict in most countries.
Unless you have a really really small company, privacy laws vary a lot between countries, and the feature is probably more trouble than it's worth. In any country I've even lived in, that idea would never fly.
But don't ask a bunch of hacks on a site like stack overflow. Seriously, ask a lawyer.
I think the question is still a bit vague as to who is going to be monitored for what. From what I understand who'll be monitored are the end users who are using the application and the gathered data will be used internally. Assuming this is the case, I think, I can contribute the following answer:
If you are going to monitor end users to see how they are using your product, you are in human factors/user experience business and what you want to do is really an experiment. Doing such an experiment requires consent of the subject (the end user). In an academic setting (and I think the same goes for industry as well), there is an Institutional Review Board (IRB) which grants permission for such experiments. I believe in the industry scene there are similar organizations (just not sure what they are called). A request for permission for such an experiment is accompanied by a report which details the user experiment in a very specific manner. The IRB than decides whether to issue a permit or not.
The important point is the consent here and users should know about the experiment and agree to be subjects. I think, in the absence of a user consent the experiment is neither ethical nor legal. Again, I approached this based on an assumption and tried to summarize my experience in such experiments.
Collecting screen shots may be illegal even if employees are notified. This is an issue of local law and federal law. You haven't said which country you are in. In California, for example, monitoring screens might violate both workplace privacy laws and wiretap laws. You should get an opinion of your corporate attorney before implementing this.

Resources