How can I develop an Excel plug-in to edit external data in an Excel data table?
Excel can make connections to external data sources but as far as I am can see they are one-direction read-only data tables. What I am trying to do is something like TFS plug-in for excel. I am sure there are many more ones like that.
For those who do not know that plug-in:
When installed, TFS Excel plug-in takes place as a new menu in Excel. Through that menu you can open a connection to a TFS server and bring your (work item) records into Excel as an Excel table. You can add new rows or edit the data in the table. Some cells has drop down lists attached to them but only valid options are shown in the list and that is different for each record. You can edit rows in the table and you can bulk push those records back to server.
I don't know if it makes a difference but the connection and update operations on my datasource will be through web services.
I guess this would require some serious development but I am lost between web pages about external data ranges (which are only for reading). Can someone please direct me to some further reading on the topic?
External Data Ranges will not help you so you can stop reading web pages about them. You're correct that they're read only. You could use them for the read part of your operation, but you'll be doing so much coding around the write part, you might as well just control everything. You just won't get enough benefit from External Data Ranges to warrant using them at all for this type of situation. In my opinion, of course.
If you were reading and writing to a database, you would likely use ActiveX Data Objects (ADO). You would read in a recordset, monitor it's changes, then write back to the database using UPDATE, DELETE, and INSERT statements as necessary.
If you'll be interacting with the database through an API, as you seem to indicate, you will probably use Microsoft XML library, specifically the MSXML2.XMLHTTP object. You can use GET, POST, PUT, DELETE, and anything else you can do through HTTP.
If you've never used XMLHTTP before, you'll have a little learning to do. But it's not particularly difficult and there's a ton of info available. The hard part, in my opinion, is tracking the changes made to the Excel sheet. If you allow the user to use Excel's native editing features, it can be difficult to keep track of the changes made. If you go to a total lockdown situation where the user has to, say, use your menu item to delete a record, then you have to ask why you're using Excel (there still may be good reasons, but familiarity with Excel's interface won't be one of them because you'll be replacing it with yours).
Maybe you already have a strategy for this. But if not, search for "detect deleted row with worksheet change event" to get a feel for some of the challenges you'll face. If you have a way forward, then go read up on XMLHTTP and you should be all set.
Related
My client wants to store basic relational data in Access. So far, so good. However, ideally, he'd like for me to create an Excel spreadsheet that would allow users to create and modify data types without having to work with Access software or know about databases. To be more specific, he wants a single master spreadsheet that would let people manage data for several different "projects." Each project would have basic attributes and other related data such as employees working on it, numbered to do items with associated data, etc. I've worked with databases before and it's a neat, textbook example of a relational database. I have a model for the data already, and making an Access form to fill it in would be straightforward.
However, here's the thing: he wants creating new attributes and tables completely intuitive within the Excel spreasheet--as easy as clicking an "add student" button or even add a new category of data. For instance, in the future, he may add a list of contractors working on the project, and it would be nice to be able to have a button that would allow you to essentially create that new table. There won't be a great amount of data, though, and I'm not sure if referential integrity and normalization is crucial. For instance, the list of contractors he creates wouldn't need to be perfectly linked up so that each company only appears once in the database.
So, what should I do? Can I accomplish this within Excel spreadsheets using macros? Can you make buttons in Excel that would say "create a new table," which would (run a VBScript to) create a new database table to be associated with each project, and then allow you to format it? Should I not bother with Excel at all and basically write a Visual Basic program? I'm familiar with general programming and databases, but I am fairly new to Excel, Access, and Visual Basic. If you could point me in the right direction--to tutorials, examples, advice, general concepts, etc--it would be much appreciated.
Excel is essentially for analyzing data, while Access is essentially for storing and processing relational data. Now, having said that, what you are trying to do is probably possible but it is really not taking advantage of the features the software where optimized for.
Furthermore, adding "tables to be associated with each project" does not seem as the "relational way of doing it", like a complex solution for a simple problem.
Perhaps you should consider some alternatives:
If the amount of data is small and not very complex, would there really be any need for Access or could you just as well use Excel for data storage and data manipulation?
Depending on how the data is structured, perhaps you can create a view or stored procedure in Access and used it as an linked table in Excel?
Perhaps you can develop the set of forms you need in Access and turn it to an stand-alone application (no need for Access installed on the client's computers)
For a person entering data, Access is built so they don't need to understand relational data at all. If you let them enter data in Excel, you will have to excessively code it to give you the same control in Access or you run the risk of letting them free form data to the point you won't be able to import it back into Access.
Unless there are very complex calculations and a need for the user to 'tweak' the report layout, give them a data entry form.
Beware of the "I'm just so use to doing it in Excel I don't want to relearn it in Access" notion. The data entry can be made very intuitive and may save them time in the long-run.
Seems like there is an owner/manager who understands Excel and wants the ability to update it without you if needed.
A little background to my question. I work for a company that is charged with retrieving data from databases from all 50 states and DC. I take this data and reformat it in excel. Once it's reformatted I use SQL Server to upload it to our website vetportal.agdata.net. While some states are not so bad, retrieving information from others make it very painful to sort through.
I have 2 questions:
Can a code be written so that a new database can be crossed checked with the old database (our records) and update the information in the old database while also excluding duplicate information?
Can a code be written to take a number from an open excel sheet, switch over to an open website, input the number, search for the individual, and extract his/her information, and finally update the excel with that information then move on to the next person? Ex, WA State's website is set up so that you can only look up one person at a time which is very tedious when going through 1200+ individuals.
I have some experience with C++ and have written programs that draw code from other files, but mainly only equations or values which then get evaluated in my code so I know this is a bit different.
I guess if you have a repetitive technological problem you can solve it with some programming.
Your questions:
You can make that with a little app, that using SQL reads the information from the new database and checks/updates the information of the old database.
This code is a little more difficult to do, but i guess it can be done. In C++, I don't know if there is any library that can already open Excel files, but in Java you have the Apache POI, that way you can open your excel file in the application, then while iterating through the information you open the website in the application, and submit the form you want with your number, getting the response and parsing it.
If you want to make this in Java I think it will not waste you too much time if you know C++ . The only exception is opening the website in java and parsing it which will take more time to learn and do.
Hope it helps!
1) Yes. Depending on the databases, you may be able to do a db to db connection. You could then write a query using an INNER JOIN to update information in the old database and exclude duplicates.
2) A few ways to approach this problem. Depending on your language (mine is PHP) you could use an open source class such as PHPExcel to open the sheet and fetch & update website data (cURL). You could also write some VBA within Excel that could do similar functionality.
Following up on my previous post, I need to be able to query a database of 6M+ rows in the fastest way possible, so that this DB can be effectively used as a "remote" data source for a dynamic Excel report.
Like I said, normally I would store the data I need on a separate (perhaps hidden) worksheet and I would manipulate it through a second "control" sheet. This time, the size (i.e. number of rows) of my database prevents me from doing so (as you all know, excel cannot handle more than 1,4M rows).
The solution my IT guy put in place consists of holding the data on a txt file inside of a network folder. This far, I managed to query this file through ADO (slow but no mantainance needed) or to use it as a source to populate an indexed Access table, which I can then query (faster but requires more mantainance & additional software).
I feel both solutions, although viable, are sub-optimal. Plus it seems to me as all of this is but an unnecessary overcomplication. The txt file is actually an export from SAP BO, which the IT guy has access to through WEBI. Now, can't I just query the BO database through WEBI myself in a "dynamic" kind of way?
What I'm trying to say is, why can't I extract only bits of information at a time, on a need-to-know basis and directly from the primary source, instead of having all of the data transfered in bulk on a secondary/duplicate database?
Is this sort of "dynamic" queries even possible? Or will the "processing" times hinder the success of my approach? I need this whole thing to really feel istantaneuos, as if the data was already there and I'm not actually retrieving it all the times.
And most of all, can I do this through VBA? Unfortunately that's the only thing I will be having access to, I can't do this BO-side.
I'd like to thank you guys in advance for whatever help you can grant me!
Webi (short for Web Intelligence) is a front-end analytical reporting application from Business Objects. Your IT contact apparently has created (or has access to) such a Webi document, which retrieves data through a universe (an abstraction layer) from a database.
One way that you could use the data retrieved by Web Intelligence as a source and dynamically request bits instead of retrieving all information in one go, it to use a feature called BI Web Service. This will make data from Webi available as a web service, which you could then retrieve from within Excel. You can even make this dynamic by adding prompts which would put restrictions on the data retrieved.
Have a look at this page for a quick overview (or Google Web Intelligence BI Web Service for other tutorials).
Another approach could be to use the SDK, though as you're trying to manipulate Web Intelligence, your only language options are .NET or Java, as the Rebean SDK (used to talk to Webi) is not available for COM (i.e. VBA/VBScript/…).
Note: if you're using BusinessObjects BI 4.x, remember that the Rebean SDK is actually deprecated and replaced by a REST SDK. This could make it possible to approach Webi using VBA after all.
That being said, I'm not quite sure if this is the best approach, as you're actually introducing several intermediate layers:
Database (holding the data you want to retrieve)
Universe (semantic abstraction layer)
Web Intelligence
A way to get data out of Webi (manual export, web service, SDK, …)
Excel
Depending on your license and what you're trying to achieve, Xcelsius or Design Studio (BusinessObjects BI 4.x) could also be a viable alternative to the Excel front-end, thereby eliminating layers 3 to 4 (and replacing layer 5). The former's back-end is actually heavily based on Excel (although there's no VBA support). Design Studio allows scripting in JavaScript.
I have a ton of data in a sql database which I would like to be able to import and display in excel (I can already do this) and additionally modify or append to the dataset within excel and write the changes/additions back to the database.
What is the best way to go about doing something like this?
Please let me know, thanks!
The way to do this is via Sql Server's DTS/SSIS capabilities. Create SSIS packages for Excel import and export and execute them as needed.
However you still have the issue of people having to share this massive spread sheet. You should consider importing the data into the db permanently and providing a winforms interface for the data entry. You'd be surprised how quickly you could whip out an app with a databound grid view control that would give you decent, Excel-like ability to add/edit/delete table data.
Although Excel is great at displaying/reporting on data stored within a SQL DB, it has no built-in controls for updating the data.
I would recommend investigating using VBA (Visual Basic for Applications) or based on your coding experience/tools available to you, VSTO (Visual Studio Tools for Office).
This method will allow all of your users to share the spreadsheet at the same time and allow incremental updates plus validation of the data being entered by the user at the point they enter it.
All the usual gotchas apply though - mainly GIGO (Garbage In, Garbage Out). Correctly authenticate your users and what they are allowed to update
I run a sports program where i have a master roll of who is in which class in excel. I want to link this to a database in access that stores the other information about each athlete, e.g. address, parents name, school, medical details. I want to be able to add names to class in the excel speadsheet and have this automatically generate a record for that person in access. There also needs to be some failsafe for athletes that are in multiple classes. I was also doing class roles as pivot tables out of the access database so i need to code for classes and also have this allow for athletes in multiple classes/disciplines.
It is easy enough to update an Access table from Excel via ADO, after that it is very much about your tables and indexes. If you are not familiar with relational databases, you might like to read http://r937.com/relational.html. That being said, it would be a lot easier to work in Access and output to Excel when necessary.
I agree I think this is a classic case of trying to get excel to do something its not best for. If you try to create some kind of hybrid system with excel pushing data into access then it will end in tears at some point.
The best thing in this case would be to port the whole thing to some kind of database. If the number of uses and the usage falls into the range for access/jet then that would be a great choice. If more users/higher usage is going to be needed then maybe look to SQL express to hold the data and access as a front end.
There was a thread a few days ago about someone being sick when maintaining an access DB, he wanted to rewrite it in .net. The point of that thread boiled down to using the correct tool for the correct job. No one can blankly say “Access sucks, everything should be in SQL server/.net” because if used in the correct way and for the correct projects access is a great tool.
So to bring it back to this thread it looks like you have “outgrown” excel and should be looking at some kind of database with access being a strong candidate
If you want to display the data in Excel (so you can do sorts, filters, etc.) then you could store the data in Access as has been suggested, then instead of exporting a report every time you want to use it, link your Excel file to Access using a Database Query.
In Excel 2003 go to Data->Import External Data->New Database Query and create a new data source to your Access mdb.
That way your data is stored in a much better way, whilst still having the Excel viewability that everyone(?) loves.