I’ve been working on a few small scale Access projects that have turned large scale rather quickly. The original designer implemented next to zero security and everyone can just walk in with a simple shift enter, way beyond just a security hole for nuclear submarines to dive through and that has always drove me bonkers.
With that said, users are currently on Office 2000, migrating slowly into 2003. I have taken this opportunity to convince higher parties to implement said security through the use of built in access tools.
Next I get to go through hundreds of functions and forms to pop in option explicit to define all the data types restricting the compile to MDE and clean up memory that was not done for some reason. There are some sensitive connection strings in the code that are plain as day that need to be compiled to reduce the risk factor.
My questions involve both the upgrade to 2003+ and the built in security. And yes, this is what I'm stuck with using unless I really want to redo everything in Visual FoxPro but building a porsche with rocks... not my idea of a good time.
When moving into office 2007, are
there any major holes that I should
be working around ahead of time?
Within the next year and a half the
whole business is supposedly
upgrading to this and I’ve only heard
horror stories about changed/obsolete
functions
Are there any major bugs that
can/will happen because of the use of
the workgroup file and permissions?
Tricks I should know ahead of time if
something crazy happens to lock
everyone out of it?
In the sandbox, I have not implemented the Encryption feature. Pros/Cons, Risks?
Any other good tips? I realize the broadness of this question and have a few good books on hand here (Professional Access 2000 Programming, Access Developers 2002, Developing Solutions with Office 2000 Components and VBA) but obviously these are before the time of current Access and Jet technology. If anything, a good book recommendation would be a booster for me, anything to give me a head start. Right now I really need to devour this security issue, its beyond just out of hand considering the sensitivity of the information at hand.
Thanks for reading my dreaded wall of text o.O
User level security does not exist for Access 2007 files (http://office.microsoft.com/en-us/access/HA101662271033.aspx). If the data is very sensitive, you may wish to consider a different back-end.
If the data is truly that sensitive it shouldn't be stored in an Access database file. Anyone can copy the entire data MDB/ACCDB and take it home with them to analyze at their leisure. Instead the data should be upsized to a database engine such as SQL Server.
Keep the current Access queries, forms and reports but get the data into a format that isn't so easy to steal.
Then think about limiting their views, logging the queries they run and such.
I would wait until A2010 is out before making any determination about upgrades beyond A2003. A2003 is fine for now, seems to me. I certainly wouldn't want to wade into targetting development to A2007 with A2010 coming out so soon and having so many really great new features (table-level data macros, really useful additions to Sharepoint integration that make a lot of really huge things possible, to name just two). My plan is to skip A2007 with clients (though I have it installed now and am playing with it so that I'll be better prepared when 2010 comes out).
One thing that doesn't often get mentioned about A2007 is that the Office FileSearch object was removed in Office 2007. If your app uses it, you can use my File Search class module to replace it. I've had it in production use since June (when I created it), but just released it more widely and am currently troubleshooting some issues that seem to be related to file names with odd characters.
Related
Is there a way of offering the flexibility of Excel/Access development that end users love while instilling centralised IT management so data and logic is secure, backed up, version controlled etc. The common options are to re-write in C#/ASP.Net/Java/Python/Your Choice, but that takes away control from the users. Is there a better way, and what do you do at your site?
There is a universal issue of users creating fantastically useful Excel/Access mini-apps that the IT department would like to bring under control. Users love the flexibility that Excel affords, especially on the fly changes, graphing and data import/export. In Access we have brilliant QBE. The downside is that after a short while there are legions of out of control spreadsheets/mdbs which are mission critical, with lots poorly understood business logic, and brittle code, they're a pain to support especially as staff move on.
This puts the IT dept in an awkward spot, they'd like to support these apps, but don't know enough about them. This is made more difficult as they are typically insecure with zero documentation.
Having been of both sides of the fence I would go after the root cause of the problem. Why do uses make their own little apps? Because it is too hard/expensive/time consuming/never turns out right when they go through the “proper” channels.
The other thing is they tend to know the business very well so whilst their coding might not be very good their knowledge of what needs doing is very good.
So what can we do to combat this problem? I personally think their should be a small team of people within IT whose job (or one of their jobs) is to develop these small applications. They should work very closely with the end users and not be locked in the ivory tower of IT.
In my current role I’m on the non-IT side of the fence, I have a few quite major applications that needed to be developed so I asked for an install of visual studio and some space on an SQL server. I had my request denied. So I just asked for SQL server space, again request denied (each request taking about a week to go through) So in the end I’m “stuck” in access.
Now these are very nice access apps with version control, comments in the (shock!) and all the other nice things but at the end of the day I was trying to do things the “right” way and ended up being forced down the access route. So when my apps try to get scaled up and I’m quoting a long time for a rewrite who is to blame?
Have you considered looking at SharePoint for department-level applications? Many professional developers will balk at the idea of using Sharepoint for "application development," but it truthfully can be a great way for "power users" to start putting their data and tools in a managed framework.
With SharePoint, you can manage the overall structure of the site and then set up users with elevated permissions within their respective departments. There are some great 3rd-party tools to help with keeping an eye on what's going on in your SharePoint site.
SharePoint is not a silver bullet by any means, but it is great for many multi-user applicatinos that need to keep up with a list of data.
(The following is not really related to my above answer, but your question really hit home and I thought I'd share my similar experiences and insights.)
Our company will be going through a similar process in the near future. I'm on the "end user" side of things and can sympathize with a lot of what Kevin Ross said. Sometimes Access and Excel are simply the best tools available for me to get the job done.
Here's an example: I was asked several years ago to come up with a system for creating Purchase Orders to a vendor in China for product for which there is a 3 month lead time. Our ERP software had a few features for procurement, but nothing that even came close to the complexity of the situation we were facing. Years later, after going through several iterations of the application in Excel (VLOOKUP was a lifesaver), Access ("So that is why people using relational databases. Awesome!), and back in Excel ("let's not make this so complicated"), I still find that these Micorosft Office apps are the best tools to get the job done.
What's the cost to not use these tools to get the job done?
Contract work to our ERP vendor to add a special feature for this ordering process: are you kidding me? We'd likely pay tens of thousands of dollars for an unflexible monolithic application with horrendous user experience...and we would still end up back in Excel.
Buy third party software designed for this exact process: I've seen an on-site demo of software that does exactly what I want for our procurement process. It starts at $100,000. There are probably other tools that we can get for a few thousand dollars, but at that price point, I've already emulated most of their features in my own application.
Try to finish the job "by hand." : Ha! I'm a programmer at heart, which means I'm lazy. If it takes a solid week of sitting at a desk to work up a purchase order (it actually did take this long), you can bet I'm going to work up a solution so that it only takes me a few hours (and now it does). Perhaps the guy after me will go back to doing most of it by hand, but I'll use the tools in my toolbox to save myself time and stress.
It's so hard to find the perfect application to allow for maximum creativity on the user end but still allow IT to "manage" it. Once you think you've found a solution for one thing, you realize it doesn't do something else. Can I write I printable report in this solution like I used to do in Access? Can I write complicated Excel formulas that tie multiple data sources together from different sheets ("You want me to learn what? No, I've never heard of a "SQuirreL query" before. VLOOKUP is just fine thankyouvermuch)? Can I e-mail the results to the people in my department? Can it automatically pull data from our back-end database like I do in Excel and Access? Can I write my own code, VBA or otherwise, to make my job easier? The list goes on.
In the end, the best advice I can give to any IT manager in your situation is to respect the other workers at your company. Let them know their work is important (even if it's only useful to them and the guy at the next desk over). Let them know you are not trying to make their job harder. Don't assume they are morons for creating mission-critical applications in office productivity software; they are just trying to get the job done with the tools at hand and are usually quite capable and intelligent people. Invite them to explore different solutions with you instead of just removing the tools they currently have in their toolbox and then replacing them with ones they don't know how to use.
At the end of the day, if you have users who are smart enough to shoot themselves in the foot by creating complicated apps in Excel and Access, they are probably smart enough to learn to use the appropriate tools to accomplish the same tasks. Invest the time and energy to involve them in the process and you will have a solution that works for everyone at the end.
You could try a hybrid approach: Allow your users to use Excel/Access to home-brew their own, specialized tools, but take the mission-critical stuff and put it under IT control. There are a few strategies that could help you with this:
Make sure that your IT department is firm on VBA. Not the "yeah-everybody-can-write-a-few-lines-of-basic" type of knowledge, but in-depth training, just like you would if it were a less simple programming language. Although "real programmers" will tell you otherwise, it is possible to write large, stable applications in VBA.
If you currently have the data in Access databases, move away from that and migrate it to an SQL Server. This allows you to do centralized backup and management, while still giving your power users the flexibility to "link" these SQL Server tables to their Access frontend.
Commonly used business logic should be under control of your IT department. This can be done either with VBA, by creating an Access library that is linked by your users, or in any of the .net languages, using COM interop. The latter sounds more complicated than it is, and it will increase the satisfaction of your IT department, since developing in .net is just much more rewarding than VBA (version control possible, etc.).
I would second one of Kevin Ross's main points:
I personally think their should be a
small team of people within IT whose
job (or one of their jobs) is to
develop these small applications. They
should work very closely with the end
users and not be locked in the ivory
tower of IT.
I think any IT department that has a lot of users using Access/Excel should have at least one properly trained and experienced specialist in developing apps on those platforms. That person would be the go-between to make sure that:
IT's priorities and policies get properly implemented in the home-grown apps.
the end users get expert help in converting their home-grown efforts into something more stable and well-designed.
I would second Tony's point that whoever works with the end users in revising these apps to meet IT standards should work side-by-side with the users. The Access/Excel specialist should be an advocate for the end users, but also for the IT policies that have to be followed.
I also think that an IT department could have a specialist or two on staff, but should also have a full-time professional Access and/or Excel developer as a consultant, since the on-staff people could probably handle day-to-day issues and management of the apps, while the professional consultant could be called in for planning and architecture and for the implementation of more complex feature sets.
But all of that would depend on the size of the organization and the number of apps involved. I don't know that it would be desirable to have someone on salary who is nothing but an Access/Excel specialist, precisely because of the problem you get with all salaried employees compared to consultants -- the employees don't see as wide a variety of situations as an active consultant with the same specialization is likely to see and thus the consultant is going to have broader experience.
Of course, I recognize that many companies do not like to outsource anything, or not something that important. I think that's unwise, but then again, I'm the person that gets hired by the people who decide to do it!
If it's mission critical, and it's in Access or Excel, is built poorly, and no one understands it, it is probably time to rebuild it properly.
When the 'users' are in control it usual means one particular person is in control of the architecture, design, coding and documentation... except they normally omit the documentation step. Source control and bug reporting, the touchstone of software development, is usually absent. Few instances of code reuse, due to the nature of Office apps (code modules usually embedded into documents) and VBA (little OOP, most VBA coders don't use Implements, etc). All this means that the resulting applications are not subject to get proper scrutiny and quality can suffer, meaning there are likely to be maintenace issues, escpecially when that one user leaves. I know because I used to be that person ;)
So in order to satisfy the IT department, the proper process needs to be applied. That one 'power' user can continue to own the design and coding but will get peer review, perhaps the serivces of a technical author and a dedicated tester, be required to use source control, perhaps consider integrating with enterprise systems, etc.
There is no getting around the use of Excel/Access. It's what's available, and still very powerful and flexible. The best thing to do is offer some guidelines as to how files should look and be set up. If everyone is using similar standards then the files will live longer and more productive lives, beyond the creator's tenure at the company.
You've got some excellent answers regarding dealing with the folks and the business side of things. So my response will be more technical.
If you are going to redesign the app have the developers work in the same offices as the users. Given the users updates every day or two. If the users have any minor suggestions give those to the users within a day or two. Ultra Frequent Application Deployment
Give the power users an Access MDB/ACCDB linked to the tables with a bunch of starter queries. Let them create the queries they need to export the data to Excel for their own purposes and distribution to clients.
A third party developer my boss has brought in, designed a "Better" System than our ASP.NET + MSSQL Server 2005 website we're using now.
Here are the relevant specs:
Excel + ODBC as the data store
Built using old school ASP, not ASP.NET
Is there any glaring problem with his solution short of the ancient tech? Thread safety etc?
Let me put it this way, "What can tell my boss (who's only partially technical) to blow this code out of the water?"
Thank you,
Vindictive Developer :)
Excel should never be used as a data store,
It is not a database
It will not handle multiple users at once at all
No support for transactions, so if an error occurs in the middle of a odbc call the excel file could end up trashed. (Even access would be better then using excel and that isn't saying much)
Excel is a spreadsheet, designed for analyzing data, not for storing data.
Straight from Microsoft: http://support.microsoft.com/kb/195951
IMPORTANT: Though ASP/ADO applications support multi-user access, an Excel spreadsheet does not. Therefore, this method of querying and updating information does not support multi-user concurrent access.
Allain, as well as the great technical reasons that have come out here, I think you need to ask yourself "why did the boss do this?"
Perception is reality, and if your boss is only partially technical, then purely technical reasoning may not get through.
Apart from the glaring architectural weaknesses, is there some functionality in this monster that makes it more appealing to your boss? Generally people don't do stupid things on purpose, it may serve you well to consider where you boss is coming from before you go making a CLM.
Ummm.... it lacks scalability: You could only have a few users. Is the data important?
Here's what you can tell him: Remind him of the nightmares that happen when two or more people need to edit the same spreadsheet at the same time. Now tell him to imagine that multiplied by a hundred people who can't call each other to tell them to "close the spreadsheet so I can update it". That's what it will be like.
Synching issues dealing with a seperate xls datastore and sql server 2005? On our IIS server, classic asp pages are prohibited by default. Maybe that's a sign lol.
How about terrible performance, since Excel is not designed to be used as a database? Tell your boss Excel isn't even a single-user database (that's what MS Access is), let alone a multi-user database designed for high, concurrent performance.
And of course by using pure ASP you're losing access to all of the libraries .NET framework (which of course is what all library developers in the MS ecosystem are focusing on). But you asked for 1 reason, and the first is better.
I would go with the mantra that those are the wrong tools for the job (assuming they are in your case). It'd be like using a screw driver as a hammer. For one nail, it might work with a lot of sweat and tears. For a real project, through, this is likely doomed.
I'd boast about the tools you are familiar with--how much better the tooling is in terms of performance, security, maintenance (esp. maintenance cost).
You could say something like he's paying someone to write a new app with decade old technology which may not be supported for much longer (if it still is...).
Ummm...row limit?
Is an Excel spreadsheet even going to handle concurrent transactions correctly? It wasn't designed for this kind of thing, and I wouldn't hold it responsible if it did something bad (like only letting one ODBC connection in at a time, or not properly locking concurrent updates).
That excel file's going to get corrupted in a hurry with many people hitting it at the same time. The scalability of Excel as a backend datastore is almost non-existent. It has a hard enough time keeping data integrity with its native Shared Workbook feature...
BTW- Is this third party a relative of your boss??? Yikes...
The ancient tech is itself a glaring problem. Will you be around forever? It will be very difficult for the boss to find new developers to maintain something like this. The tech world has moved on.
What open source toolkit does fatwire compare to and are there some particular advantages to fatwire?
How hard is fatwire to export out of and move to a free alternative?
How stable is it as a platform to write java extensions on?
From a development persepective, FatWire can be unfriendly. Having worked on a number of sites using this application it can easy bloat, and become difficult to maintain.
From a user perspective there has been alot of effort in the UI and this has led to a highly functional tool.
From a client perspective all clients bar 1 (a large news agency) were happy with the end result. FatWire can slow when using complex logic to generate menus or breadcumbs for example or when you have a large amount of content. This is the main reason the one client was unhappy. The FatWire site regularily struggled under the load. It sometimes seen as a solution to all web needs.
As such FatWire succeeds in serving Static Content & Semi Dynamic content, but can flounder when forced to do fully dynamic sites (from my experience).
From the original press release:
FatWire Software announced the rollout
of FirstSite, which is a set of tools
and best practices that helps
companies using FatWire Content Server
get their first Web site or
application running quickly while
providing a foundation for future
expansion. FirstSite includes a
collection of standard templates and
site components that are common to
most sites, combined with
documentation, training, a rich
developer community, and best
practices methodology. FatWire and its
solution partners are using FirstSite
as the basis for developing
content-centric applications for
specific vertical markets. With only
minor, cosmetic alterations,
developers can use the code in
FirstSite to implement a first site,
while simultaneously learning how to
utilize Content Server's capabilities,
such as dynamic content delivery,
personalization, caching, and product
catalogs.
Firstsite is not a product, unless this has changed since 2004 (unfortunately I cannot look, since their developer site is down). Fatwire's Content Server does not compare to any Open Source CMS that I know. It's scope goes much further. I will answer your questions one by one:
Advantages - There are many (or nobody would buy it, and it is not cheap)
On the delivery side: scalability, fine-grained cache control, stateless servlet architecture, ....
On the back office side: virtually no limit to asset types, dynamic content attributes, find-grained security and access control, ...
On the development side: Intelligently architected API with good coding productivity, tag library, ...
Openness
You cannot easily expect to migrate content between any two CMS products, open source or not. While there are ways to extract contant from the database in XML and other forms, using product tools, or simply at the database level, I don't think that this can be an argument for or against using a particular CMS. Ever tried to migrate from Drupal to Joomla?
Stable
I worked on several Fatwire implementations from 2000 to 2004 (back then it was OpenMarket Content Server, then Divine Content Server). It was stable enough for the Washington Post, the New York Times, and the S&P sites, and I would expect stability not to be an issue today.
Fatwire is really unique concept from developer point of view. It builds everything on a very abstract, extremely flexible clever asset modeling framework which is stored in relational database.
Application logic is based on "templates" which actually are pieces of JSP code. This JSP code is not like conventional Java, but tags instead. It takes very long from a developer to learn these tags and Fatwire asset api. Expect even months before skilled develpers start to be productive.
Almost nothing useable samples ships along the product. There is advertized "FirstSite" but it is way too simple for the purpose this product is used normally (huge complex sites). So pretty much everything has to be built from scratch.
Cache control is advertized to be one powerful feature. Yes it is, but we had extremely long learning curve and it never worked exactly like one assumed.
Wysiwyg editing has been missed from this product even it is advertized. At least during 2009 it had serious conceptual problems which practically prevented using it in live environments. But it was cool feature for demos and marketing of course. Today it might be fixed.
As a summary and if I were a customer with limited budget, I'd select any open source alternative instead. Mostly because development costs with Fatwire are high due the uniqueness of the product, lack of good documentation and extremely long learing curve. Of course the product price tag is also thing to consider.
And to answer to questions: you have to start from scratch if you move from Fatwire 6.0 to any open source alternative. And it is stable to build Java extensions on.
Fatwire stores content in relation database and file system. Depending on what type of content (structured/unstructured), Fatwire can be evaluated.
For the better part of 10 years + we have relied on various network mapped drives to allow file sharing. One drive letter for sharing files between teams, a seperate file share for the entire organization, a third for personal use etc. I would like to move away from this and am trying to decide if an ECM/Sharepoint type solution, or home grown app, is worth the cost and the way to go? Or if we should simply remain relying on login scripts/mapped drives for file sharing due to its relative simplicity? Does anyone have any exeperience within their own organization or thoughts on this?
Thanks.
SharePoint is very good at document sharing.
Documents generally follow a process for approval, have permissions, live in clusters... and these things lend themselves well to SharePoints document libraries.
However there are somethings that don't lend themselves well to living inside SharePoint... do you have a virtual hard drive (.vhd) file that you want to share with a workmate? Not such a good idea to try and put a 20GB file into SharePoint.
SharePoint can handle large files, and so can SQL Server behind it... but do you want your SQL Server bandwidth being saturated by such large files? Do you want your backup of SQL Server to hold copies of such large files multiple times?
I believe that there are a few Microsoft partners who offer the ability to disassociate file blobs from the SharePoint database, so that SharePoint can hold the metadata and a file system holds the actual files, and SharePoint simply becomes the gateway to manage access, permissions, and offer a centralised interface to files throughout an organisation. This would offer you the best of both worlds.
Right now though, I consider SharePoint ideal for documents, and I keep large files (that are not document centric) on Windows file shares.
Definetely, use a tool.
The main benefit here is version control. Being able to jump easily to a previous version, diff'ing and seeing who modified what (see most VCS' blame/annotate tool- it prints out a text file showing when/who modified each line in the text file).
Second, you can probably benefit from issue tracking/task tracking.
Other benefits include web access from the internet, having a wiki (which can be great in some situations), etc.
I use Subversion + Redmine at work, and I find it highly useful- test a few solutions and you will surely find out further advantages for you.
One thing that can be overlooked in the change to an document management tool is the planning required around how much is going to be stored and information architecture issues like where different content is going to end up.
SharePoint particularly is easy to setup without a good plan going forward and is particularly vulnerable to difficulties later on when things get to busy.
I would not recommend a home grown app for something like this. The problem has been solved by off the shelf tools and growing one from scratch is going to cost a huge amount and not get you any way near the features for the money.
Did I mention how important planning your security groups and document areas (IA) was?
If you need just document storage then sharepoint can do very well. WSS is ewen free and it provides very good document storage capabilities.
But you have to plan carefully as updating existing applications is painfull. If you decide to go with Sharepoint then I can give you few advices from top of my head
Pay attention to security configuration (user groups, privilegies,..)
Plan your document libraries well as it is not easy to just move documents betveen them
Also consider limiting number of versions that one document can have, because sharepoint stores full backups betveen verions, not just changes
Don't use infopath:) we have very bad experience with it (just don't tell this to managers)
If you don't really need to change graphical look of Sharepoint than don't bother with it as it brings many problems (I'm talking about custom masterpages and custom site templates)
Try to use as much OOB stuff as possible, because developing your own webparts not only cost more, but it can be quite complicated.
Make sure to turn-on search indexing. This is quite tricky, because it is by default turned off and then you will be as surprised that search is not working as I was :)
If you try to just deploy it and load 10.000 documents into it then you will surely have problems with it later. If you give a little thought about structure then you will end up with really good document storage.
Migrating is very probably worth the cost in the long term. You will gain reliability, versioning, traceability, and extensibility.
Be sure to first identify the groups/rights, and to identify which links need to be fixed (maybe you have applications that use links to the shares).
An open source alternative to SharePoint is Alfresco, it is very good for CIFS (Windows shares) too.
I am currently working on a project which requires migration of content from different content management Systems to SharePoint. Are there any good, preferably open source, tools that would help me do this? Also, what are the best practices that I would have to keep in mind when it comes to such projects. One more thing that i would like to factor here is reusablity, because we might have to work on similar migration projects, from other Content Management systems in future.
You can check http://www.codeplex.com/SPMigration (open source, project started by a Microsoft consultant).
This framework gives you an importer tool, as well as some exporter example (FileSystem for example). You'll problably have to code your own exporter.
This MSDN blog also goes into some detail about the Migration API and may be useful as its generally very had to do this sort of thing without getting your hands dirty
http://blogs.msdn.com/sharepointdeveloperdocs/archive/2007/11/30/content-migration-in-sharepoint.aspx
Also, IMHO you shouldn't dismiss proprietary products as although they can be expensive they may save you considerable time and therefore cost if you have a large conversion project.
http://www.tzunami.com/Pages/default.aspx
http://www.avepoint.com/products/sharepoint-migration
Tricks and tips -
http://www.parallelspace.net/portals/ALS305-mwherman-Content%20Migration-1-1-18-RC6_FINAL.ppt
We have had good mileage from going to the nearest university and grabbing some IT students to do a manual migration.
The students like the extra cash and it is sometimes easier when the Information Architectures of the site changes between systems.