I am trying to figure out a way to create a secure data entry process using Microsoft Office (cannot use Azure by the way). The obvious way is to use Microsoft Forms, but the form would be too large as there are over 150 data points that need to be entered.
I have instead created an Excel file that has the ability to load responses to a SharePoint list. This works as long as the list is public and viewers have contribute privileges. The downside to this is that the data is not secure and anyone can view the list. The data has phone numbers, ssn, passports, etc. I tried to make a work around for this by creating a flow that captured any new entries and moved them to my personal private SharePoint list, then deleted the original. This works great, but after testing with my colleague it seems that anyone can join the first team and make an identical flow to that team to capture data themselves.
Is there a way to accept data from anyone internally while also preventing them from accessing the data? I am capable of using SharePoint, Teams, Power Automate, and Access to accomplish this. I am less familiar with access but have used it before. I'm just not sure how secure it is.
I'll be happy to provide further information as needed. Thanks
You could consider Access forms, and then have the tables reside on SharePoint.
You can then say base the form on a query, and in that query you could provide (have) the user name included as part of the query where clause (thus a user removing or changing the forms filter will only provide a view of their own data.). However, such uses if they fired up Access (or even Excel) could wind up with seeing all rows in the table and not what the Access query restricts the data to (the one user).
Unfortantly with Access desktop? And like most desktop software? Well the goal and design and history is that of allowing ease of data editing and viewing. So, just like when you walk into a bank, and wait in line? Well, when you get to the teller, they will fire up a access form based like system and pull up your account information. And any and all tellers can see/view and pull up that data because that is what the software supposed to do, and do well. (get at any customer data).
Now, flip this problem, and create a user web banking system, or even a instant teller. Now the problem is backwards. The software's goal is not ease of editing data in a table, but ONLY allowing the user to see THEIR data. So a very different kind of problem.
It is very (beyond very) to note that web development tools do not out of the blue or automatics restrict data to JUST the one user. It is YOU the developer that makes that choice. So, if we could press a button and have a access form converted to the web? Well, then that access form would do its job in life. That job is to allow you to edit all that data in a table. Not just YOUR data.
For example, have you ever written a Access application that JUST displays each users own data? I am betting not! It is hard to do. (because the roots of Access is ease of editing data). As noted, I cannot stress that web development tools do the SAME thing. They can let you edit data. The editing of data is thus a software issue and development issue. The data system or Access or even SQL server? They just hold data. It is YOUR software that has to decide that ONLY data for the one user is to be displayed.
Now so very often web systems ONLY show YOUR data. But that VERY common case is due to the nature of web systems, and that your not a bank teller, or a company employee, but that of a consumer of that system. And each consumer now is 100% opposite of what most software systems do naturally (that is get data from a table).
So, I can't stress this concept enough. That concept is that web software and systems do NOT make this restricting of data choice for you. It is your developer tools that MUST have this ability and ALSO the choice of how you build such systems.
Web systems can do this data restricting data better for several reasons.
One big reason? Well, most of the time you adopt some kind of security and logon system.
Thus, parts of the web site now are restricted based on your logon system. (hey, maybe you use Facebook logons). So web systems have a very good and wide and broad system of supporting some kind of logon system. For desktop software? not so much.
But EVEN with that very robust and type of security system and logon system?
You the developer STILL have to make the choice of what data they see. Maybe it is traveling salesmen on the road. And thus every user of that web system still has to be able to edit, view and add customers to that system. This common case is not really different then a typical Access form to edit data.
So only SOME types of web applications need that restricting of data to JUST the one logged on user. But, because web systems have that "better" logon system? Then you the developer can thus now use that logon to restrict data with greater ease then what desktop software dev tools often offer (say like access).
So, a lot of this will come down to the web tools used. If you pulling data with Excel, or Access? Then it considerable more difficult to pull JUST data that belongs to a given user. And your software will require designs from day one with these restrictions in mind.
Again, at the risk of repeating myself?
the tools don't' make the data restrictions for you. If you need secure data for ONLY the given user, then you simply have to adopt a set of developer tools that allow this concept in mind. And that tends to mean the tools you use do NOT by default allow users to directly edit or link to some big table of data that contains all users data.
This is also why say a program written in VB6, or now vb.net can be more secure then Access. (because those tools by default don't provide forms that are designed from day one to edit all rows of data). So, the UI does not present default forms and things that gives users the ability to edit all rows. As a result, then the software can now control what records the users sees, and since they don't' have some default form that allows viewing of all rows.
It thus becomes more practical to write software that only dishes out their own data. But again, and I can't stress this:
The choice of what data to display is a choice of the software developer - not the database nor the web development tools.
It thus stands to reason then you have to chose the appropriate tools that will allow you to write software that restricts the data in a manor you require.
General questions on SO in regards to what flavor of ice cream or what dev tools to use tends to be frowned upon, and is quite much against the general rules for solving software and code issues you have and post on SO.
But, if you come from say Access with VBA skills? Then I would suggest adopting the free verison of Visual Studio and go with vb.net + asp.net web forms. Web forms are close to end of life, but they are by far and away the best choice, the least effort (coming from Access) and will give you a robust security model in which you can restrict data on a per user basis.
I would not duck tape and try to cobble together such a system with office tools, since they are not really designed for that type of data restricting you need. So, go with some good quality web tools like Visual Studio (free version is fine) and use SQL server (free version again) is really the way to go here. If you need a truck in place of a car, then get the right kind of truck, and don't try and use a car for that transport of goods, or in this case build a secure web database system.
Related
I've been asked to figure out a way to do this, so please fill me in on whether this is even possible or if it shouldn't be done.
The goal is to automate data entry into VMS (we use E-Term32 for connecting to VMS). Things that have been discussed for this purpose: Excel spreadsheets, Dynamic Data Exchange, the macro tools available in E-Term32 (Emulation Command Language - ECL), OLE Automation, etc.
The envisioned process would go like:
Receive Excel file (or other data format like a text file)
Connect to VMS
Run Macro
Macro navigates the menu system and uses data from Excel file to enter data
I know there are "better" or easier ways to do this like building an application to enter the data, but my supervisors are concerned about circumventing the business logic built into the "Blue Screen" menu/applications. They are also sticklers on building new applications for stuff like this anyways.
How is the data stored on OpenVMS, may we assume in native RMS (indexed) files? or some database (RDB, Oracle, Adabas,...) ?
Whichever, it is sure to be perfectly possible to write directly in the datastore through some ODBC or JDBC method. Freeware, or commercial (ConnX, Attunity,...). Just google: OpenVMS ODBC
Once you find a method for direct data access one should indeed be concerned about the business logic. Field formatting, values ranges, foreign keys,...
Thus access can only be granted to (software managed by) trusted team players.
But that can be perfectly manageable and you may find the new method can be made more robust than those green-screen apps.
If direct data access is no negotiable, then there are still many options.
Screen-scrapers have been build, you should not attempt to write from scratch.
Check out commercial terminal-centric modernization tools like: http://www.rocketsoftware.com/products/rocket-legasuite-gui
presentation: http://de.openvms.org/Spring2009/03-Dutch_Police_FINAL.pdf
(I am not associated with the fine folks # Rocket, it is just one example I am aware of. There are surely more (commercial) options.
Now about those business rules. How clean is this implement?
Strict form/function seperation? Probably not, otherwise you would not be asking.
There are several RPC, GSOAP, methods available, free and for fee, that allow one to call OpenVMS service routines, passing in external data. That's probably the best path forward.
The company I work for, Attunity, sells such 'legacy adapter' tools in addition to ODBC/JDBC data access to files directly.
For example, using Attunity Connect software you can connect a row in a table, to the call of a subroutine. The basic plan would be to just use an SQL INSERT statement on Linux or Windows to an ODBC datasource which is connected to an OpenVMS target. The connect software will then call an OpenVMS subroutine in a shareable library to process the row, using or at least re-using the existing business logic for validation, normalization or de-normalization as the case might be. And yes, they can also expose a SELECT or MODIFY for lookups that are more complex than can be described in SQL.
Everybody happy! You can use modern tools, they can use the old code and checks.
And ofcourse another time-honored method is to just have an FTP drop point for data to be entered. Some OpenVMS job scans an agreed upon directory for 'action' files, and runs an OpenVMS program to process the data in the fashion similar to the terminal UI app. Cobol, Basic, re-using as much of the existing terminal code and logic as possible
Good luck!
Hein
Looking for an IT data mapping solution that integrates with SharePoint 2013.
Need to be able to capture information about IT systems, how they relate, how the data flows through each system. Must be able to dynamically generate diagrams on the fly.
As far as I am aware, there is nothing like this that Integrates to SharePoint. You would have to create something bespoke.
Indeed, I would not recommend that approach as I think that the level of complexity and size of data is likely to quickly outstrip anything that would be especially performant in SP. SP is already a complex set of data and adding further complexity is not wise in my experience.
SP lists are good for simple data. I've been starting to create a service catalogue for ourselves in SP but it will be a very simple list & even as I've been thinking about it, I've quickly realised that it might not be suitable for long. Also, I'm not planning to hold much in the way of data interfaces.
Instead, I would recommend a solution that is more focussed with a database and a web front end that could easily be integrated to the SP site or simply linked.
These kind of systems tend to get very complex as you begin to discover links between systems and unless your organisation is relatively small and simple, I think you will regret it.
If your system architecture is at all complex, you would be better off with a fully-fledged Enterprise Architecture tool. That would allow you to begin to correctly map the logic between systems and even to map systems to infrastructure.
In short, to use SP, keep it very simple (simplistic) and forget the diagrams for now as you would have to create a bespoke interface anyway. Consider a bespoke database with web front end that can be linked to your AD for authentication and authorisation the same as SP and simply linked to from SP. Or for a more complex architecture, investigate the use of an Enterprise Architecture tool.
UPDATE:
OK, so you need something simple to draw out the data. That is a bit easier. I'm still not aware of a dedicated web part but to be honest, I've not looked.
My approach would probably be to use a JavaScript "Graph" tool - note the phrase "graph" not to be confused with "chart". A chart is a pictorial view of (often numeric) data. A graph is a set of nodes and "edges" (I probably haven't explained that very well). Graphviz is the most famous of such tools but that is a command line tool. However, there are plenty of JavaScript tools that might be used.
SharePoint is delightfully easy to integrate with JavaScript (apart from the bizarre field names and hopelessly overcomplex data structures!) & even with my limited skills, I was able to knock up a Google map "view" that uses the data from a normal SharePoint list. With a bit of help from a JavaScript library of course.
You simply hide the normal View Web Part. The data is available to the web page still and you can feed it straight into a JavaScript library that will do the graphing for you.
Here are some free and none-free libraries in case you chose to go this route:
Free:
SigmaJS
VisJS
Arbor
Springy
Not Free
GoJs - Not used it but this looks fab. Interactive Diagrams for the
Web - unlimited trial but watermarked, USD thousands
jGraph USD 10's thousands
ZoomCharts - USD hundreds
Worth also noting that, in theory, you could do this with Visio. However, there are not many examples out there because it is so complex.
I have programmed a system for internal behavior reporting for my company's intranet. I should not have access to its data (not being part of the controlling committee, but I have.
I've locked my account away from the data, but I could unlock it. I could store the data in an encrypted format, but, even if chosen by someone else, I should store the salt somewhere and hence read it -> decrypt the data.
From a theoretical point of view (I'm not talking about a particular system or framework or utility), how can I not have access to the data stored in a system I have complete control of?
Seems to me that you could just set passwords such that only one user has access to the database, then allow someone else to set that password. It would make maintenance a bit more tricky, but then again a database shouldn't need a ton of maintenance on a tool like this once all is said, done, and thoroughly tested.
If this is internal, it would be nothing to setup a dedicated, physically secure WAMP or similar machine that's solely dedicated to this purpose. Have someone else tweak root passwords and store them with the "committee" and you're off the hook, in theory.
I suppose if one was to be completely paranoid, one could build a web service to isolate the database completely on a separate network from the reporting functionality. In theory, you could setup the web service on a remote machine that your access is removed from, then use the front-end to collect data and pass it to the webservice. From there, it's completely out of your hands, with no "data out" webservice to retrieve data.
Security is always a messy subject. I've worked in banking, ecommerce, and sports (drug testing) environments where I'm knee-deep in confidential data and it is more than just a bit scary. At some point, you just have to do the best you can do, document your safeguards, be "read in" on proper protocol and required background checks, do thorough testing with independent testers, and then just maintain complete transparency. In the IT world we have access to a ridiculous amount of information, and that's never going to go away.
The basic answer is Mandatory Access Control. The kind of access control most computer user are familir with is Discressionary Access Control. In DAC (Discressionary Access Control) everything on the computer is owned by a user. Users can grant access of an object (file, service, peripheral, memory, etc) to another user. Users can even transfer ownership of an object to another user. In MAC (Manditory Access Control) at least some objects are not owned by any user. The rules governing how users can access or interact with these objects are fixed and unchangable by any user.
In your example the data generated by the reporting system should be protected by Manditory Access Control, but the reporting system configuration may be owned by you. So you can control how the system behaves but not have access to the data it generates.
Microsoft began implementing MAC with Windows Vista. In Vista it was called Mandatory Integrity Control (MIC).
Linux can implement MAC with SELinux or AppArmor.
Mac OS X uses an implementation of the TrustedBSD MAC.
So, why isn't MAC used more often?
I takes effort. It is not easy to set up MAC, and it is hard to change once it is set up. It can be complicated. Most systems and services are built on the DAC model. Turning on MAC often makes services stop working.
Is there a way of offering the flexibility of Excel/Access development that end users love while instilling centralised IT management so data and logic is secure, backed up, version controlled etc. The common options are to re-write in C#/ASP.Net/Java/Python/Your Choice, but that takes away control from the users. Is there a better way, and what do you do at your site?
There is a universal issue of users creating fantastically useful Excel/Access mini-apps that the IT department would like to bring under control. Users love the flexibility that Excel affords, especially on the fly changes, graphing and data import/export. In Access we have brilliant QBE. The downside is that after a short while there are legions of out of control spreadsheets/mdbs which are mission critical, with lots poorly understood business logic, and brittle code, they're a pain to support especially as staff move on.
This puts the IT dept in an awkward spot, they'd like to support these apps, but don't know enough about them. This is made more difficult as they are typically insecure with zero documentation.
Having been of both sides of the fence I would go after the root cause of the problem. Why do uses make their own little apps? Because it is too hard/expensive/time consuming/never turns out right when they go through the “proper” channels.
The other thing is they tend to know the business very well so whilst their coding might not be very good their knowledge of what needs doing is very good.
So what can we do to combat this problem? I personally think their should be a small team of people within IT whose job (or one of their jobs) is to develop these small applications. They should work very closely with the end users and not be locked in the ivory tower of IT.
In my current role I’m on the non-IT side of the fence, I have a few quite major applications that needed to be developed so I asked for an install of visual studio and some space on an SQL server. I had my request denied. So I just asked for SQL server space, again request denied (each request taking about a week to go through) So in the end I’m “stuck” in access.
Now these are very nice access apps with version control, comments in the (shock!) and all the other nice things but at the end of the day I was trying to do things the “right” way and ended up being forced down the access route. So when my apps try to get scaled up and I’m quoting a long time for a rewrite who is to blame?
Have you considered looking at SharePoint for department-level applications? Many professional developers will balk at the idea of using Sharepoint for "application development," but it truthfully can be a great way for "power users" to start putting their data and tools in a managed framework.
With SharePoint, you can manage the overall structure of the site and then set up users with elevated permissions within their respective departments. There are some great 3rd-party tools to help with keeping an eye on what's going on in your SharePoint site.
SharePoint is not a silver bullet by any means, but it is great for many multi-user applicatinos that need to keep up with a list of data.
(The following is not really related to my above answer, but your question really hit home and I thought I'd share my similar experiences and insights.)
Our company will be going through a similar process in the near future. I'm on the "end user" side of things and can sympathize with a lot of what Kevin Ross said. Sometimes Access and Excel are simply the best tools available for me to get the job done.
Here's an example: I was asked several years ago to come up with a system for creating Purchase Orders to a vendor in China for product for which there is a 3 month lead time. Our ERP software had a few features for procurement, but nothing that even came close to the complexity of the situation we were facing. Years later, after going through several iterations of the application in Excel (VLOOKUP was a lifesaver), Access ("So that is why people using relational databases. Awesome!), and back in Excel ("let's not make this so complicated"), I still find that these Micorosft Office apps are the best tools to get the job done.
What's the cost to not use these tools to get the job done?
Contract work to our ERP vendor to add a special feature for this ordering process: are you kidding me? We'd likely pay tens of thousands of dollars for an unflexible monolithic application with horrendous user experience...and we would still end up back in Excel.
Buy third party software designed for this exact process: I've seen an on-site demo of software that does exactly what I want for our procurement process. It starts at $100,000. There are probably other tools that we can get for a few thousand dollars, but at that price point, I've already emulated most of their features in my own application.
Try to finish the job "by hand." : Ha! I'm a programmer at heart, which means I'm lazy. If it takes a solid week of sitting at a desk to work up a purchase order (it actually did take this long), you can bet I'm going to work up a solution so that it only takes me a few hours (and now it does). Perhaps the guy after me will go back to doing most of it by hand, but I'll use the tools in my toolbox to save myself time and stress.
It's so hard to find the perfect application to allow for maximum creativity on the user end but still allow IT to "manage" it. Once you think you've found a solution for one thing, you realize it doesn't do something else. Can I write I printable report in this solution like I used to do in Access? Can I write complicated Excel formulas that tie multiple data sources together from different sheets ("You want me to learn what? No, I've never heard of a "SQuirreL query" before. VLOOKUP is just fine thankyouvermuch)? Can I e-mail the results to the people in my department? Can it automatically pull data from our back-end database like I do in Excel and Access? Can I write my own code, VBA or otherwise, to make my job easier? The list goes on.
In the end, the best advice I can give to any IT manager in your situation is to respect the other workers at your company. Let them know their work is important (even if it's only useful to them and the guy at the next desk over). Let them know you are not trying to make their job harder. Don't assume they are morons for creating mission-critical applications in office productivity software; they are just trying to get the job done with the tools at hand and are usually quite capable and intelligent people. Invite them to explore different solutions with you instead of just removing the tools they currently have in their toolbox and then replacing them with ones they don't know how to use.
At the end of the day, if you have users who are smart enough to shoot themselves in the foot by creating complicated apps in Excel and Access, they are probably smart enough to learn to use the appropriate tools to accomplish the same tasks. Invest the time and energy to involve them in the process and you will have a solution that works for everyone at the end.
You could try a hybrid approach: Allow your users to use Excel/Access to home-brew their own, specialized tools, but take the mission-critical stuff and put it under IT control. There are a few strategies that could help you with this:
Make sure that your IT department is firm on VBA. Not the "yeah-everybody-can-write-a-few-lines-of-basic" type of knowledge, but in-depth training, just like you would if it were a less simple programming language. Although "real programmers" will tell you otherwise, it is possible to write large, stable applications in VBA.
If you currently have the data in Access databases, move away from that and migrate it to an SQL Server. This allows you to do centralized backup and management, while still giving your power users the flexibility to "link" these SQL Server tables to their Access frontend.
Commonly used business logic should be under control of your IT department. This can be done either with VBA, by creating an Access library that is linked by your users, or in any of the .net languages, using COM interop. The latter sounds more complicated than it is, and it will increase the satisfaction of your IT department, since developing in .net is just much more rewarding than VBA (version control possible, etc.).
I would second one of Kevin Ross's main points:
I personally think their should be a
small team of people within IT whose
job (or one of their jobs) is to
develop these small applications. They
should work very closely with the end
users and not be locked in the ivory
tower of IT.
I think any IT department that has a lot of users using Access/Excel should have at least one properly trained and experienced specialist in developing apps on those platforms. That person would be the go-between to make sure that:
IT's priorities and policies get properly implemented in the home-grown apps.
the end users get expert help in converting their home-grown efforts into something more stable and well-designed.
I would second Tony's point that whoever works with the end users in revising these apps to meet IT standards should work side-by-side with the users. The Access/Excel specialist should be an advocate for the end users, but also for the IT policies that have to be followed.
I also think that an IT department could have a specialist or two on staff, but should also have a full-time professional Access and/or Excel developer as a consultant, since the on-staff people could probably handle day-to-day issues and management of the apps, while the professional consultant could be called in for planning and architecture and for the implementation of more complex feature sets.
But all of that would depend on the size of the organization and the number of apps involved. I don't know that it would be desirable to have someone on salary who is nothing but an Access/Excel specialist, precisely because of the problem you get with all salaried employees compared to consultants -- the employees don't see as wide a variety of situations as an active consultant with the same specialization is likely to see and thus the consultant is going to have broader experience.
Of course, I recognize that many companies do not like to outsource anything, or not something that important. I think that's unwise, but then again, I'm the person that gets hired by the people who decide to do it!
If it's mission critical, and it's in Access or Excel, is built poorly, and no one understands it, it is probably time to rebuild it properly.
When the 'users' are in control it usual means one particular person is in control of the architecture, design, coding and documentation... except they normally omit the documentation step. Source control and bug reporting, the touchstone of software development, is usually absent. Few instances of code reuse, due to the nature of Office apps (code modules usually embedded into documents) and VBA (little OOP, most VBA coders don't use Implements, etc). All this means that the resulting applications are not subject to get proper scrutiny and quality can suffer, meaning there are likely to be maintenace issues, escpecially when that one user leaves. I know because I used to be that person ;)
So in order to satisfy the IT department, the proper process needs to be applied. That one 'power' user can continue to own the design and coding but will get peer review, perhaps the serivces of a technical author and a dedicated tester, be required to use source control, perhaps consider integrating with enterprise systems, etc.
There is no getting around the use of Excel/Access. It's what's available, and still very powerful and flexible. The best thing to do is offer some guidelines as to how files should look and be set up. If everyone is using similar standards then the files will live longer and more productive lives, beyond the creator's tenure at the company.
You've got some excellent answers regarding dealing with the folks and the business side of things. So my response will be more technical.
If you are going to redesign the app have the developers work in the same offices as the users. Given the users updates every day or two. If the users have any minor suggestions give those to the users within a day or two. Ultra Frequent Application Deployment
Give the power users an Access MDB/ACCDB linked to the tables with a bunch of starter queries. Let them create the queries they need to export the data to Excel for their own purposes and distribution to clients.
I’m looking for a mechanism that can be used to distribute Microsoft Office content over the web without using Rights Managements Services whilst minimising the ability for it to be printed or redistributed. I know any solution is always going to be a compromise and never entirely secure but I’d be interested in any mechanisms which increase the degree of difficulty to redistribute. The frontrunner at the moment is to use Flashpaper with the print and selection tools disabled and convert the documents to this format as required.
The business need to provide sales reps with material about products which can not easily be redistributed externally in electronic format.
Has anyone come up with something more elegant? Any other thoughts about limiting the ability to redistribute without investing in other DRM technologies?
Question - are there any Open Source tools that read Flashpaper? Bear in mind that any restrictions that exist in the tool to read the document are only useful when there is no more permissive tool handy. If any tool becomes popular for limiting document use like that, there will be tools to grab the content. They may be illegal under the DMCA, but they will exist.
Unless all you want to do is discourage those who are not tech-savvy, you're going to have to go for DRM.
BTW, what is the intended use for this?