I have 500 Excel documents. I want users to keep working as if that was excel (I'll provide app for that) yet cross-reference data in-between that documents. What database can feet such needs?
So, if i get it ok then you need to get data from ~500 excel files while people may access and change them in real time! I can think of 4 ways of approach:
live links of all files to 1 workbook... hurts me to even think the maintenance and setting ... but it will be "live".
powerQuery: group them all in one data table using PowerQueries or PowerBI or similar, then load them on workbook OR save as csv... 1 button refresh, relatively quick, no actual coding needed
use VBA: access all files (or changed ones...) and get what you want, when you want it. If implemented expertly will only take a few seconds for full scan in modern pc, yet needs someone good at coding VBA.
setup 1) using VBA instead of manually, then using VBA to check for errors etc. Result will be "live" but requires again serious VBA coding...
I believe that 2) is the easiest choice with good maintenance features, ease of setting and good speed... (start in excel ...Data / new Query/from File/from Folder ...)
In my company we have all products in an Excel-database. When a customer (shops) wants to place an order, they fax a form with item No etc. The order is entered through a VBA form, returning info about "in stock" etc, and the order is subtracted from the stock and placed in worksheet.
Now I would like to know, if its possible - without too much fuss, to do this from the www, so a customer can access the database and see if the item is in stock (which is not the hard part) - AND place the order so the database is updated in our local workbook.
I'm not asking for actual code, but ideas to a general approach.
Is it possible to run VBA forms from a remote computer?
Will XML, ASP etc be the answer?
Other ideas?
I'll appreciate any help.
This is all fairly easy to do if you have a proper database, there are all kinds of reasons not to use Excel.
Your best approach would likely be to build a web system around a database and move all application logic there.
But in your specific case it might be easier to build it around Sharepoint.
I am hoping someone might be able to help me.
My problem is essentially this: A Colleague enters information into an excel, which I then have to check and pass on by email. This is fairly time-critical.
What I would like to do is
have the colleague press a button that calls the macros on my computer (worksheet running continuously),or
have my colleague email me and I have a macro in Outlook which checks for specific subject lines, or
he saves it on the network, and I check every minute for new files in that folder.
While the last two of these are possible, the outlook solution is - for several company policy reasons - the very last resort, and I would also like to avoid the ongoing checking for files as I am already having slight performance issues (large worksheet with lots of external links that are being feed real time).
I am also open to all other suggestions someone might have. Thanks a lot!!
I'd go with the third option.
Make his workbook save information to a 'queue' file or folder. Your workbook can query this queue, which should be on the network somewhere, and notify you when its changed. Wouldn't even have to open it unless it has changed if you set it to compare modify dates, and could be small if it is saved as text or in XML format.
First option won't work because VBA framework is pretty locked down. Cross workbook macro activation isn't possible from what i understand.
Second option is more work than necessary, and VBA/Outlook will warn you every 5 minutes that its trying to access your mail folder since that's what malicious software typically does.
Like i said, the third option would be best, and his macro could be set to only write, could even encrypt the text using simple encryption methods so that others can't easily modify it if that is a concern.
I am on my dissertation in my final year at university at the moment. One of the areas I need to research is security - for both websites and for databases. I currently have sections on the following:
Website
Form security - such as data validation. This section is more about preventing errors made by legitimate users as much as possible rather than stopping hackers, for example comparing a field to a regular expression and giving them meaningful feedback on any errors which did occur so as to stop it happening again.
Constraints. For example if a value must be true or false then use a checkbox. If it is likely to be one of several values then use a dropdown or a set of radio boxes, and so on. If the value is unpredictable then use regular expressions to limit what characters they are allowed to enter, and to restrict the length of the string, and sometimes to limit the format (such as for dates / times, post codes and so on).
Sometimes you can limit permissions to the form. This is on the occasion that you know exactly who (whether it be peoples names or a group of people - such as administrators or employees) is going to need access to the form. Restricting permissions will stop members of the public from being able to access the form.
Symbols or strings which could be used maliciously or cause the website to act incorrectly (such as the script tag) should be filtered out or html encoded.
Captcha images can be used to prevent automated systems from filling in and submitting the form.
There are some hacks for file uploads - such as using double extensions - which can allow hackers to upload malicious files.
Databases (this is nowhere near done yet but the sections I have planned are listed below)
SQL statements vs stored procedures
Throwing an error when one of the variables contains particular characters or groups of characters (I cant remember what characters they are, but I have seen a message thrown back at me before where I have tried to enter html or something into a text area).
SQL Injection - and ways around it, with some examples.
Does anyone have any hints and tips on where I could go for some decent, reliable information either about these areas or about other areas of security that I could cover?
Thanks in advance.
Regards,
Richard
PS I am a complete newbie when it comes to security, so please be patient with me. If any of the information I have put down is wrong or could be sub-sectioned then please feel free to say so.
To get you started on website security, I recommend you go through the following sources -
OWASP Top 10 - http://www.owasp.org/index.php/Category:OWASP_Top_Ten_Project
Common Weakness Enumeration - http://cwe.mitre.org/
Both list the top programming errors, and will give you a head-start in this field.
What I can think of is pre-populating certain form input elements based on the user's geographical information.
What are other ways can you think of to speed up user input on long application forms?
Or at least keep them focus on completing the application form?
If you have a long form, try to prune it down. Don't ask them to fill in fields that you don't really need.
If the form spans several pages, give the user some feedback as to how many more pages there are. We users hate clicking on the continue button wondering if this will be the last page.
Never lose a field that they filled in, no matter what they do. This could have security implications if passwords are involved.
Use dropdowns to provide the user with options unless there are a lot of options that the user would have to scroll through or if the terms in the dropdown aren't widely accepted (e.g. dropdown filled with Systems Engineer, Solution Developer, IT Application... I just want Programmer.).
Provide help for fields that might be hard to fill in (or provide examples).
If it is possible in your case, just collect the bare minimum up front and then allow the user to use the basic features of your service.
For the user to upgrade to a better level of service, they will need to fill in the 2nd form with more detail.
How important it is to you to collect ALL that information up front ? It is worth losing customers by demanding too much from them ? Why not demand it later at a time more convenient to the user.
Creating a multi-step wizard offering only a small number of input fields per step. Ensure that they are aware of how far they have progressed in the sequence.
The psychology is that once a user is 'invested' in a task, they are more likely to continue. If you present the whole list of input fields at once, you scare them off.
Offering musings at each step (cartoon, humor, sayings etc) makes them move to the next step out of curiosity.
Users won't mind filling in long forms if and only if they feel that the questions that you ask are important: otherwise they will be discouraged, and become impatient with it.
Remember, in a web application people have very, very short attention spans. When the user starts feeling that you are asking too much, they're usually right.
Keep required information as few as possible: other info should only be optional, and you have to give something in return to the user to compel them to complete that information.
However you implement it, please please please use some kind of Ajax hearbeat to store their progress server side and repopulate it if it's lost. There is nothing more infuriating to a user that working through a long form and having a browser or network hiccup lose their entire submission.
Whenever it happens to me I generally never give it a second shot, because at that point recreating my submission isn't worth whatever I was signing up for.
Checklist:
Explain clearly the purpose of the form. (What's in it for them?)
Prune, prune, prune, and keep questions clearly relevant!
Give the user feedback on his/her progress (if the form is split over multiple pages)
Ask for as little as you can up-front and leave the rest for later.
Clearly mark required fields
Group fields logically.
Keep labels/headings brief and easy to understand.
Prefill as much as possible - but not too much.
Spread super long forms over multiple pages and allow backtracking.
Cleverly placed "Back", "Save" and "Cancel" buttons put people's minds at ease - even when redundant.
Provide friendly (but clear!) validation error messages, in a timely manner.
Allow the user to reclaim half-filled in forms - don't lose their data!
No matter what you do, do not include a reset button. :-)
Finally:
Explicitly tell the user when the process is finished. ("Thank you! Your application has been sent.")
Tell the user what will happen next. ("A confirmation e-mail has been sent to your e-mail address, and we'll process your application within two working days.")
use Ajax to populate and update the controls asynchronously.It will speedup the filling of long application forms.
Split it up into multiple pages - there's nothing quite so discouraging as seeing that you have another 100 questions to go.
Put validation on each input and check it onblur(). If they get to the end of the page and then it says "question #2 was incorrect", chances are they've forgotten what that one was anyway and it'll be more difficult to return to it. Plus, if they answer a series of similar inputs in a particular, incorrect way, you should let them know straight away (eg: entering dates as mm/dd/yyyy when you want dd/mm/yyy)
Split the form into several steps. It's like how someone is much more likely to read five 3-sentence paragraphs than one big 15-sentence paragraph of the same length.
I agree with tim; just let them fill in the bare minimum information and then leave the rest to profile updates. If any data is necessary for the service offered on your site, ask for it when they try to avail of the service (and no earlier).
That said, I wouldn't advocate the kind of forcing function that adam suggests. It pays to give your users the warm, fuzzy feeling that they are privileged and can use ALL of the services on your site. Although, if you look at it hard enough, adam's and my suggestions are pretty much the same.
If the application needs to include a lot of information, then make sure the user can save at any point, and log off, and log in later to complete the form. This would make more sense if some of the information is not necessarily easily available. Tax returns are an obvious example, where some of the data may need to be calculated, or the user must find the relevant documentation.
In some cases the user might use the same information in multiple applications. In that case it might make sense for the user to register their details (Name, Address, Telephone numbers, etc), which are automatically filled in on each application. For example, if you had a website for a recruitment agency, they may allow users to register their details, and then to apply for a particular job, they can just include a personal statement that applies to that job in particular.
As another consideration, if some information may be incorrect (particular if this is not always clear, such as a CAPTCHA, or a user name that must be unique), either separate it from the rest of the data, or otherwise make it so a mistake doesn't mean the rest of the information must be reentered.
These are basically ways of avoiding the user having to enter the same information twice.