Looking for an IT data mapping solution that integrates with SharePoint 2013.
Need to be able to capture information about IT systems, how they relate, how the data flows through each system. Must be able to dynamically generate diagrams on the fly.
As far as I am aware, there is nothing like this that Integrates to SharePoint. You would have to create something bespoke.
Indeed, I would not recommend that approach as I think that the level of complexity and size of data is likely to quickly outstrip anything that would be especially performant in SP. SP is already a complex set of data and adding further complexity is not wise in my experience.
SP lists are good for simple data. I've been starting to create a service catalogue for ourselves in SP but it will be a very simple list & even as I've been thinking about it, I've quickly realised that it might not be suitable for long. Also, I'm not planning to hold much in the way of data interfaces.
Instead, I would recommend a solution that is more focussed with a database and a web front end that could easily be integrated to the SP site or simply linked.
These kind of systems tend to get very complex as you begin to discover links between systems and unless your organisation is relatively small and simple, I think you will regret it.
If your system architecture is at all complex, you would be better off with a fully-fledged Enterprise Architecture tool. That would allow you to begin to correctly map the logic between systems and even to map systems to infrastructure.
In short, to use SP, keep it very simple (simplistic) and forget the diagrams for now as you would have to create a bespoke interface anyway. Consider a bespoke database with web front end that can be linked to your AD for authentication and authorisation the same as SP and simply linked to from SP. Or for a more complex architecture, investigate the use of an Enterprise Architecture tool.
UPDATE:
OK, so you need something simple to draw out the data. That is a bit easier. I'm still not aware of a dedicated web part but to be honest, I've not looked.
My approach would probably be to use a JavaScript "Graph" tool - note the phrase "graph" not to be confused with "chart". A chart is a pictorial view of (often numeric) data. A graph is a set of nodes and "edges" (I probably haven't explained that very well). Graphviz is the most famous of such tools but that is a command line tool. However, there are plenty of JavaScript tools that might be used.
SharePoint is delightfully easy to integrate with JavaScript (apart from the bizarre field names and hopelessly overcomplex data structures!) & even with my limited skills, I was able to knock up a Google map "view" that uses the data from a normal SharePoint list. With a bit of help from a JavaScript library of course.
You simply hide the normal View Web Part. The data is available to the web page still and you can feed it straight into a JavaScript library that will do the graphing for you.
Here are some free and none-free libraries in case you chose to go this route:
Free:
SigmaJS
VisJS
Arbor
Springy
Not Free
GoJs - Not used it but this looks fab. Interactive Diagrams for the
Web - unlimited trial but watermarked, USD thousands
jGraph USD 10's thousands
ZoomCharts - USD hundreds
Worth also noting that, in theory, you could do this with Visio. However, there are not many examples out there because it is so complex.
Related
Is there a way of offering the flexibility of Excel/Access development that end users love while instilling centralised IT management so data and logic is secure, backed up, version controlled etc. The common options are to re-write in C#/ASP.Net/Java/Python/Your Choice, but that takes away control from the users. Is there a better way, and what do you do at your site?
There is a universal issue of users creating fantastically useful Excel/Access mini-apps that the IT department would like to bring under control. Users love the flexibility that Excel affords, especially on the fly changes, graphing and data import/export. In Access we have brilliant QBE. The downside is that after a short while there are legions of out of control spreadsheets/mdbs which are mission critical, with lots poorly understood business logic, and brittle code, they're a pain to support especially as staff move on.
This puts the IT dept in an awkward spot, they'd like to support these apps, but don't know enough about them. This is made more difficult as they are typically insecure with zero documentation.
Having been of both sides of the fence I would go after the root cause of the problem. Why do uses make their own little apps? Because it is too hard/expensive/time consuming/never turns out right when they go through the “proper” channels.
The other thing is they tend to know the business very well so whilst their coding might not be very good their knowledge of what needs doing is very good.
So what can we do to combat this problem? I personally think their should be a small team of people within IT whose job (or one of their jobs) is to develop these small applications. They should work very closely with the end users and not be locked in the ivory tower of IT.
In my current role I’m on the non-IT side of the fence, I have a few quite major applications that needed to be developed so I asked for an install of visual studio and some space on an SQL server. I had my request denied. So I just asked for SQL server space, again request denied (each request taking about a week to go through) So in the end I’m “stuck” in access.
Now these are very nice access apps with version control, comments in the (shock!) and all the other nice things but at the end of the day I was trying to do things the “right” way and ended up being forced down the access route. So when my apps try to get scaled up and I’m quoting a long time for a rewrite who is to blame?
Have you considered looking at SharePoint for department-level applications? Many professional developers will balk at the idea of using Sharepoint for "application development," but it truthfully can be a great way for "power users" to start putting their data and tools in a managed framework.
With SharePoint, you can manage the overall structure of the site and then set up users with elevated permissions within their respective departments. There are some great 3rd-party tools to help with keeping an eye on what's going on in your SharePoint site.
SharePoint is not a silver bullet by any means, but it is great for many multi-user applicatinos that need to keep up with a list of data.
(The following is not really related to my above answer, but your question really hit home and I thought I'd share my similar experiences and insights.)
Our company will be going through a similar process in the near future. I'm on the "end user" side of things and can sympathize with a lot of what Kevin Ross said. Sometimes Access and Excel are simply the best tools available for me to get the job done.
Here's an example: I was asked several years ago to come up with a system for creating Purchase Orders to a vendor in China for product for which there is a 3 month lead time. Our ERP software had a few features for procurement, but nothing that even came close to the complexity of the situation we were facing. Years later, after going through several iterations of the application in Excel (VLOOKUP was a lifesaver), Access ("So that is why people using relational databases. Awesome!), and back in Excel ("let's not make this so complicated"), I still find that these Micorosft Office apps are the best tools to get the job done.
What's the cost to not use these tools to get the job done?
Contract work to our ERP vendor to add a special feature for this ordering process: are you kidding me? We'd likely pay tens of thousands of dollars for an unflexible monolithic application with horrendous user experience...and we would still end up back in Excel.
Buy third party software designed for this exact process: I've seen an on-site demo of software that does exactly what I want for our procurement process. It starts at $100,000. There are probably other tools that we can get for a few thousand dollars, but at that price point, I've already emulated most of their features in my own application.
Try to finish the job "by hand." : Ha! I'm a programmer at heart, which means I'm lazy. If it takes a solid week of sitting at a desk to work up a purchase order (it actually did take this long), you can bet I'm going to work up a solution so that it only takes me a few hours (and now it does). Perhaps the guy after me will go back to doing most of it by hand, but I'll use the tools in my toolbox to save myself time and stress.
It's so hard to find the perfect application to allow for maximum creativity on the user end but still allow IT to "manage" it. Once you think you've found a solution for one thing, you realize it doesn't do something else. Can I write I printable report in this solution like I used to do in Access? Can I write complicated Excel formulas that tie multiple data sources together from different sheets ("You want me to learn what? No, I've never heard of a "SQuirreL query" before. VLOOKUP is just fine thankyouvermuch)? Can I e-mail the results to the people in my department? Can it automatically pull data from our back-end database like I do in Excel and Access? Can I write my own code, VBA or otherwise, to make my job easier? The list goes on.
In the end, the best advice I can give to any IT manager in your situation is to respect the other workers at your company. Let them know their work is important (even if it's only useful to them and the guy at the next desk over). Let them know you are not trying to make their job harder. Don't assume they are morons for creating mission-critical applications in office productivity software; they are just trying to get the job done with the tools at hand and are usually quite capable and intelligent people. Invite them to explore different solutions with you instead of just removing the tools they currently have in their toolbox and then replacing them with ones they don't know how to use.
At the end of the day, if you have users who are smart enough to shoot themselves in the foot by creating complicated apps in Excel and Access, they are probably smart enough to learn to use the appropriate tools to accomplish the same tasks. Invest the time and energy to involve them in the process and you will have a solution that works for everyone at the end.
You could try a hybrid approach: Allow your users to use Excel/Access to home-brew their own, specialized tools, but take the mission-critical stuff and put it under IT control. There are a few strategies that could help you with this:
Make sure that your IT department is firm on VBA. Not the "yeah-everybody-can-write-a-few-lines-of-basic" type of knowledge, but in-depth training, just like you would if it were a less simple programming language. Although "real programmers" will tell you otherwise, it is possible to write large, stable applications in VBA.
If you currently have the data in Access databases, move away from that and migrate it to an SQL Server. This allows you to do centralized backup and management, while still giving your power users the flexibility to "link" these SQL Server tables to their Access frontend.
Commonly used business logic should be under control of your IT department. This can be done either with VBA, by creating an Access library that is linked by your users, or in any of the .net languages, using COM interop. The latter sounds more complicated than it is, and it will increase the satisfaction of your IT department, since developing in .net is just much more rewarding than VBA (version control possible, etc.).
I would second one of Kevin Ross's main points:
I personally think their should be a
small team of people within IT whose
job (or one of their jobs) is to
develop these small applications. They
should work very closely with the end
users and not be locked in the ivory
tower of IT.
I think any IT department that has a lot of users using Access/Excel should have at least one properly trained and experienced specialist in developing apps on those platforms. That person would be the go-between to make sure that:
IT's priorities and policies get properly implemented in the home-grown apps.
the end users get expert help in converting their home-grown efforts into something more stable and well-designed.
I would second Tony's point that whoever works with the end users in revising these apps to meet IT standards should work side-by-side with the users. The Access/Excel specialist should be an advocate for the end users, but also for the IT policies that have to be followed.
I also think that an IT department could have a specialist or two on staff, but should also have a full-time professional Access and/or Excel developer as a consultant, since the on-staff people could probably handle day-to-day issues and management of the apps, while the professional consultant could be called in for planning and architecture and for the implementation of more complex feature sets.
But all of that would depend on the size of the organization and the number of apps involved. I don't know that it would be desirable to have someone on salary who is nothing but an Access/Excel specialist, precisely because of the problem you get with all salaried employees compared to consultants -- the employees don't see as wide a variety of situations as an active consultant with the same specialization is likely to see and thus the consultant is going to have broader experience.
Of course, I recognize that many companies do not like to outsource anything, or not something that important. I think that's unwise, but then again, I'm the person that gets hired by the people who decide to do it!
If it's mission critical, and it's in Access or Excel, is built poorly, and no one understands it, it is probably time to rebuild it properly.
When the 'users' are in control it usual means one particular person is in control of the architecture, design, coding and documentation... except they normally omit the documentation step. Source control and bug reporting, the touchstone of software development, is usually absent. Few instances of code reuse, due to the nature of Office apps (code modules usually embedded into documents) and VBA (little OOP, most VBA coders don't use Implements, etc). All this means that the resulting applications are not subject to get proper scrutiny and quality can suffer, meaning there are likely to be maintenace issues, escpecially when that one user leaves. I know because I used to be that person ;)
So in order to satisfy the IT department, the proper process needs to be applied. That one 'power' user can continue to own the design and coding but will get peer review, perhaps the serivces of a technical author and a dedicated tester, be required to use source control, perhaps consider integrating with enterprise systems, etc.
There is no getting around the use of Excel/Access. It's what's available, and still very powerful and flexible. The best thing to do is offer some guidelines as to how files should look and be set up. If everyone is using similar standards then the files will live longer and more productive lives, beyond the creator's tenure at the company.
You've got some excellent answers regarding dealing with the folks and the business side of things. So my response will be more technical.
If you are going to redesign the app have the developers work in the same offices as the users. Given the users updates every day or two. If the users have any minor suggestions give those to the users within a day or two. Ultra Frequent Application Deployment
Give the power users an Access MDB/ACCDB linked to the tables with a bunch of starter queries. Let them create the queries they need to export the data to Excel for their own purposes and distribution to clients.
I'm currently looking at developing an application, and have a choice between doing a standard ASP.NET web application or integrating it into SharePoint. Our client would like it to be SharePoint if possible, as they are under pressure to put all new development into it, but standard ASP.NET is still an option.
It is an application to manage and view data in a database with about 10 tables, including an approval workflow when certain new items are added. Referential integrity of the data is important.
I have experience of developing ASP.NET applications, but very little with SharePoint. Does anyone have any criteria they would apply to deciding between the two?
So far I am thinking along the lines of:
Referential integrity of the data is important and SP doesn't seem to handle this very well without writing lots of custom code
SharePoint doesn't seem very scaleable with the suggested limit of 2000 items in a list
The application has an approval workflow, which does seem to be something SP does well
On the wholem, it seems like we would end up writing lots of custom code and not really using any of the out of the box SP features. So my thinking is why not just write a standard ASP.NET application.
Are there any other key things we should consider?
By now, you may have already found this link: http://blogs.msdn.com/sanjaynarang/archive/2009/06/19/should-i-build-my-application-in-sharepoint-vs-asp-net.aspx. If not, it's a decent starting place with some good questions to ask.
What follows is my take as a long-time .NET developer (for as long as the platform has been around) and a SharePoint architect (since 2003). That's basically my way of saying that I've been on both sides of the fence.
In my opinion, SharePoint is a platform, not a product. As ASP.NET provides valuable web-based services to the core .NET framework, SharePoint supplies additional services and capabilities on top of ASP.NET. The platform removes the need for writing common code pieces that are a part of so many ASP.NET apps: security code, user profile management, personalization, UI/UX baseline, etc. When you get into the plumbing, you get even more: rich caching support that requires minimal configuration, customization modularity via Features, and more.
Should every application be built in SharePoint? I'd never push for that. With my current client, we use a mix of SharePoint-based and custom ASP.NET applications. Whether an application gets built in SharePoint vs. written from the ground-up in ASP.NET is a function of what we're doing. We conduct the same sort of exercise you are. If SharePoint's features and functionality can be brought to bear to reduce development time, it goes in SharePoint. If the need is too specific or we feel we'd be working around SharePoint, we go the custom app route.
You have some very specific concerns for your application, so let me take a crack at them with the little I understand about your requirements:
REFERENTIAL INTEGRITY: based on what you're saying, it sounds like your data model is pretty specific. Building your information architecture to natively leverage site columns, content types, and lists probably doesn't make sense. That doesn't throw SharePoint out, though. There's absolutely no reason why you couldn't build the data model you want (presumably in SQL Server) and then consume it with components that reside in SharePoint. If you're using MOSS, some of the BDC WebParts might work for you straightaway. If not, you'd still be writing controls and/or pages to work with the data, correct? There's nothing wrong with using SharePoint as your presentation layer for access to SQL directly or (in a more scalable, n-tier fashion) go against business services somewhere else.
2000 ITEM LIMIT: this is a common concern and one that is misunderstood. There is no 2000 list item limit; the actual measurement is 2000 items per view (and that's with out of the box views, by the way) or "container" (such as a folder). You can store many more times that (millions, if you like) in a list if you partition with folders, build your own view to page, etc. Again, given your data structure and the likely need you have to dodge SharePoint's lists, this wouldn't be an issue if you simply consumed data from SQL Server.
WORKFLOW: SharePoint is nice as a workflow host, and the OOTB workflows are handy. I'm assuming you're looking at MOSS (versus straight WSS), but just in case: the approval workflow comes with MOSS. If you're constrained to WSS, you only have one workflow available to you: the three-state workflow.
At the end of the day, SharePoint is .NET and built on top of ASP.NET. Much of the code you'd have to write in a SharePoint app you'd need to write in a custom .NET anyway. I'd look at things from the perspective of understanding whether the experience and features SharePoint affords you (as a developer) can help speed your development cycle and/or improve the user experience (something we, as developers, sometimes forget).
David in Dakota does have an excellent point, though, in that the dev experience for SharePoint is different from straight ASP.NET. The need (or rather, best practice) to deploy via Features, understand specific SharePoint concerns (e.g., lifetime and disposal of SharePoint objects), etc., mean that there will be ramp-up time if you do build in SharePoint. There are quite a few good resources out there (including folks here on StackOverflow) that can help, but you'll need to factor some learning into the equation of whether or not SharePoint makes sense.
One more parting thought: Microsoft is slowly shifting many of its own products and platforms to leverage SharePoint as their UI/UX layer, and the trend is picking up some steam. PerformancePoint, Project Server, Team Foundation Server, and Commerce Server all use SharePoint as their presentation tier. The trend will probably continue, though I don't know how far. If you use any of these products (or their on your technology roadmap), a SharePoint investment now might pay off later.
Despite all of my writing about and advocating for SharePoint, I don't think it's the right tool in every scenario. I still build WinForms apps, smart clients, command line apps, and more quite a bit. It just comes down to weighing "what I get" for "what I spend" (in both time and money).
I hope this helps!
Your evaluation is quite accurate. (It would help to have more detail about every feature your application needs but that's not really practical for this medium.)
The problems you mention have been largely solved but you would need to understand and implement the solutions. For example, there is a CodePlex project that can assist with referential integrity and there are recommendations on how to manage the number of items in a list. But using SharePoint is never going to give you the freedom of writing an ASP.NET application from scratch.
Another thing to consider is how you and/or the client expect the application will evolve in the future. If it will need more collaboration-style features or features such as version history on list items and integration with the Office client then SharePoint may be the better option.
You should also think about the complexity of deploying and updating your application on SharePoint.
What open source toolkit does fatwire compare to and are there some particular advantages to fatwire?
How hard is fatwire to export out of and move to a free alternative?
How stable is it as a platform to write java extensions on?
From a development persepective, FatWire can be unfriendly. Having worked on a number of sites using this application it can easy bloat, and become difficult to maintain.
From a user perspective there has been alot of effort in the UI and this has led to a highly functional tool.
From a client perspective all clients bar 1 (a large news agency) were happy with the end result. FatWire can slow when using complex logic to generate menus or breadcumbs for example or when you have a large amount of content. This is the main reason the one client was unhappy. The FatWire site regularily struggled under the load. It sometimes seen as a solution to all web needs.
As such FatWire succeeds in serving Static Content & Semi Dynamic content, but can flounder when forced to do fully dynamic sites (from my experience).
From the original press release:
FatWire Software announced the rollout
of FirstSite, which is a set of tools
and best practices that helps
companies using FatWire Content Server
get their first Web site or
application running quickly while
providing a foundation for future
expansion. FirstSite includes a
collection of standard templates and
site components that are common to
most sites, combined with
documentation, training, a rich
developer community, and best
practices methodology. FatWire and its
solution partners are using FirstSite
as the basis for developing
content-centric applications for
specific vertical markets. With only
minor, cosmetic alterations,
developers can use the code in
FirstSite to implement a first site,
while simultaneously learning how to
utilize Content Server's capabilities,
such as dynamic content delivery,
personalization, caching, and product
catalogs.
Firstsite is not a product, unless this has changed since 2004 (unfortunately I cannot look, since their developer site is down). Fatwire's Content Server does not compare to any Open Source CMS that I know. It's scope goes much further. I will answer your questions one by one:
Advantages - There are many (or nobody would buy it, and it is not cheap)
On the delivery side: scalability, fine-grained cache control, stateless servlet architecture, ....
On the back office side: virtually no limit to asset types, dynamic content attributes, find-grained security and access control, ...
On the development side: Intelligently architected API with good coding productivity, tag library, ...
Openness
You cannot easily expect to migrate content between any two CMS products, open source or not. While there are ways to extract contant from the database in XML and other forms, using product tools, or simply at the database level, I don't think that this can be an argument for or against using a particular CMS. Ever tried to migrate from Drupal to Joomla?
Stable
I worked on several Fatwire implementations from 2000 to 2004 (back then it was OpenMarket Content Server, then Divine Content Server). It was stable enough for the Washington Post, the New York Times, and the S&P sites, and I would expect stability not to be an issue today.
Fatwire is really unique concept from developer point of view. It builds everything on a very abstract, extremely flexible clever asset modeling framework which is stored in relational database.
Application logic is based on "templates" which actually are pieces of JSP code. This JSP code is not like conventional Java, but tags instead. It takes very long from a developer to learn these tags and Fatwire asset api. Expect even months before skilled develpers start to be productive.
Almost nothing useable samples ships along the product. There is advertized "FirstSite" but it is way too simple for the purpose this product is used normally (huge complex sites). So pretty much everything has to be built from scratch.
Cache control is advertized to be one powerful feature. Yes it is, but we had extremely long learning curve and it never worked exactly like one assumed.
Wysiwyg editing has been missed from this product even it is advertized. At least during 2009 it had serious conceptual problems which practically prevented using it in live environments. But it was cool feature for demos and marketing of course. Today it might be fixed.
As a summary and if I were a customer with limited budget, I'd select any open source alternative instead. Mostly because development costs with Fatwire are high due the uniqueness of the product, lack of good documentation and extremely long learing curve. Of course the product price tag is also thing to consider.
And to answer to questions: you have to start from scratch if you move from Fatwire 6.0 to any open source alternative. And it is stable to build Java extensions on.
Fatwire stores content in relation database and file system. Depending on what type of content (structured/unstructured), Fatwire can be evaluated.
I’m looking for a mechanism that can be used to distribute Microsoft Office content over the web without using Rights Managements Services whilst minimising the ability for it to be printed or redistributed. I know any solution is always going to be a compromise and never entirely secure but I’d be interested in any mechanisms which increase the degree of difficulty to redistribute. The frontrunner at the moment is to use Flashpaper with the print and selection tools disabled and convert the documents to this format as required.
The business need to provide sales reps with material about products which can not easily be redistributed externally in electronic format.
Has anyone come up with something more elegant? Any other thoughts about limiting the ability to redistribute without investing in other DRM technologies?
Question - are there any Open Source tools that read Flashpaper? Bear in mind that any restrictions that exist in the tool to read the document are only useful when there is no more permissive tool handy. If any tool becomes popular for limiting document use like that, there will be tools to grab the content. They may be illegal under the DMCA, but they will exist.
Unless all you want to do is discourage those who are not tech-savvy, you're going to have to go for DRM.
BTW, what is the intended use for this?
Our organization started on the SharePoint path about two years ago. Before that, we (the developers) wrote mostly asp.net front ends for SQL back ends. Now it seems like every time a new project comes up, we are asked to “make” it fit in SharePoint; and we have stuffed some things into SharePoint that probably should have been stand alone applications or web applications due to complexity and interactions with other technologies.
My question is: Where do you draw the line as to developing a project in SharePoint versus Web/Winform application, and how do you convince your manager(s) that SharePoint may not be the best solution for a particular project?
I sort of agree with you that this is sometimes a tough question. In general, though, I agree with the cliche that you just have to think about a sharepoint app a little differently. If your data can be considered as list-based, then SharePoint probably isn't a necessarily bad development framework. It may seem like more work on the surface, but IMO the challenges just move from one place to another. If you use things like custom field templates and web parts, you can relatively naturally handle all sorts of data. And you get the positive aspects of SharePoint for free (an already mature security framework, built-in searching, site and list templates/definitions, personalized page customizations, yada, yada).
I also I don't know what you mean by "complexity and interactions with other technologies" here, so it's hard to imagine what specific issues might be introduced when SharePoint is added to the mix.
If your dev team is relatively inexperienced with SharePoint and you care about quality and deadlines, I can definitely see your point. It's not an easy learning curve, but I think the SharePoint product is more naturally extensible than many people give it credit for.
There is, in some cases, a third option between a SharePoint application and an ASP.NET application. You can build custom site and application pages and deploy them to a SharePoint site. (The book Inside Windows SharePoint Services 3.0 gives a good overview of how to do this.) This will allow you to use ASP.Net and SQL Server within a SharePoint environment (which means you can also take advantage of things like SharePoint security). It's not as easy as developing a plain ASP.Net application, but it's a compromise.
Of course, this is sort of a technicality if they're wanting these new applications to be built on SharePoint technologies (lists, libraries, workflow, etc.), not just to be "inside" SharePoint.
One of the primary reasons why you might put an applicaiton in SP is when you want to take advantage of the building blocks SP gives you:
Security (share security with the site)
Data (store some or all of your data in lists)
Provisioning (if you want you app on multiple sites)
Some basic data UI e.g. Lists give you that and you dont need to build it.
One thing to consider when trying to 'integrate' a new app into the existing pool is whether there is any overlap in data (customers, inventory, etc) that would benefit from the merger.
There is also the benefit of being able to back up multiple applications and all of their respective data in one place.
Why are they asking for it all to go into SharePoint?
In my experience it is because the 'ole SharePoint intranet is being great as a portal to keep everything together and findable under the one information architecture.
Approach the issue from a uses perception of the application space in the organisation.
So long as the application looks and feels just like part of the intranet site and the user does not have to think about how to get to it (and how to get back out), you can pretty much take any architecture decisions necessary to get the best bang for the organisations buck when it comes to implementation and maintainence.
When we started thinking about the site less from SharePoint vs other stuff to the nice woolly concepts of Information Architecture, findability and usability, our decisions not to make it actually inside SharePoint, but still skin it like the Intranet became easier to sell.