My business works with a partner business. The partner business has a database we can access using web forms. We have to do hourly/daily metric reports which involves exporting the data & copy/pasting into a google sheet which then gets us our numbers.
My question is, is there a more efficient way of grabbing this data if backend access to the database has been rejected. Ideally I'd like to do my own queries but since I have no access I run the query on the webpage, export to excel, then copy paste the data into a google sheet and then use the query function to get what I need. What would be the solutions you would advise? Should I ask for a web service? Any way to automate exports? Any ideas?
What you are doing appears to be web-scraping. If so, you can scrape HTML tables from a website from within a Google sheet. Absolutely no backend access using the IMPORTHTML function.
See this excellent video: https://www.youtube.com/watch?v=95c0OlsjKgU
Related
I use Google Sheets API from googleapis to get a spreadsheet and its charts. So I have ChartSpec. How can be this spec used to visualize the chart in a web application, e.g. using Visualization API?
Use case
A simple web app written in HTML and JS which shows all charts from an arbitrary spreadsheet. In the ideal scenario, the charts are rendered just like in spreadsheets, so I can see the charts without the necessity to open a spreadsheet manually.
A user selects a spreadsheet from a list of predefined spreadsheets (menu)
Web app requests up-to-date data for the selected spreadsheet from Node.js backend
Node.js app gets all charts in the spreadsheet using Google Sheets API
Node.js app responds with necessary data
Web app visualizes the charts that should ideally be interactive (eg. using Visualization API)
Problem
I have a trouble with steps 4 and 5. In step 3 I receive ChartSpec which seems to contain all information but data. Instead of data, there are just references to ranges in a form of ChartData object.
Creating a custom implementation which gets all referenced data seems extremely complicated. Even if I created it on my own, I do not know how to create visualizations in step 5.
What I found and why it is not solution for me? There are similar questions and solutions that use Visualization API to query data from a spreadsheet and the visualize it, such as this answer. Why is this not solution for me? Because my spreadsheets are created by non-tech people who do not follow any good practices, so data cannot be easily queried using Query API. The spreadsheets are a mess. The charts are created by the same folks and they want to customize it by themselves. What I need is to show already existing charts without necessity to make any chart-wise code.
The only solution I came up with so far is to use the Chart Editor and allow the user to set up the chart himself. I would be interested in investing in such a solution or working on it together.
I am trying to create a data entry "app" to collect daily readings across our site. Here are the three biggest constraints:
Software - ideally, we would use some software within the Microsoft 365 Suite, mainly because those are the only approved apps on site. It may be possible to use open source software, but that might raise some flags in terms of security. So my thoughts are to use either Excel or Access.
Cost - ideally, we do not want purchase any additional software licenses. I would try and create something with Power Apps, but we do not have the licensing for an Azure or SQL server to store the data. I could be missing something here though.
Mobile-Friendly - finally, it needs to work on an Android tablet. Currently, we collect readings using pen and paper. The whole idea of this is to move towards using a tablet.
The easiest approach would be to create an Excel spreadsheet, save it on OneDrive, and edit the spreadsheet. I don't love this option because we are collecting 100's of data points each day. This would end up with a very wide spreadsheet that will be cumbersome to navigate.
The other option I looked into was creating an Access database and accompanying form and storing it on SharePoint. However, it seems Microsoft has stopped supporting Access databases on SharePoint.
I have created data entry forms using VBA, similar to this, but these do not work on mobile.
Is it possible to create a data entry form in Excel that also works on the Android version of Excel? Are there other alternatives I am not thinking of?
I am engaged in just this kind of project also. I have written an app in PowerApps, built an Excel spreadsheet and stored it in OneDrive, and am running it (the app) on an iPad. The design differs somewhat from your description of directly presenting a spreadsheet to the user (which I think PowerApps could do) because I don't want users having direct access to the data.
Edit: You do not need Azure or SQL, unless you are storing tons of data. Excel can be a satisfactory data storage location for modest uses.
I found the learning curve for PowerApps to be quite steep, as it's a different paradigm than line-by-line coding.
I think this is a more user friendly way to collect data than trying to run an Excel form, and once you get it made and polished, you'll look like a pro :)
I am by no means an expert but if you need some tips I'll do what I can to help. It sounds like we are at similar developmental stages.
Is it possible to create a data entry form in Excel that also works on the Android version of Excel? Are there other alternatives I am not thinking of?
Microsoft Forms does the job when created from OneDrive on mobile browser. Side note: the form I just created and the response I submitted have now disappeared from my OneDrive.
I also saw some people using Power Automate to save responses from a form into an Excel file (every reponse).
I am completely new Office 365 (and SharePoint) but have been asked to create a site that will display a range of data in the form of graphs and tables etc The data will change daily and therefore it must be possible for members of the team to enter new raw data, for the results to then be displayed through Office 365.
I realise this might sound a little vague but my initial thoughts are that SharePoint is what I should use to display the data and to have a SQL backend database that stores the data for SharePoint to connect to. Having done some reading on the topic and I am still a little unsure if this is common practice or even possible.
Any inital pointers would be greatly appreciated.
This can be done with Power BI. The data sources can by almost anything, SQL, spreadsheets, online sources, you name it. Create queries to get the data, model it (if required) and build reports and dashboards that display in a browser (or on a phone).
I need to give power-users of the web application I am working on the ability to create their own reports, analysis, etc. in Excel. Basically they need to create some Excel sheets, read some data from the web app and than mix the result with further data coming from other data sources.
The web application already exposes data in JSON/XML format through a web api (not fully REST but this doens't matter) and I would use this channel to get data in Excel.
At the moment I have these three options in my mind:
In the past I've solved a request like this with some VBA code and a COM object (that talked with the server/database) but I am not sure if today this is still the best solution to do this kind of job.
I have learned that today Excel has PowerPivot that can read data from a web service. I could develop an oData feed for PowerPivot but I am not sure if PowerPivot is what power-users need
Another solution could bean ad-hoc Excel add-in
How would you solve a request like this?
Power Query would be a better option than Power Pivot here. Power Pivot is a dimensional modeling and analytical database (it is exactly a private instance of SSAS Tabular running behind the Excel process).
Power Query is an end-user friendly ETL tool, developed as an add-in for Excel, and available natively as of Office 2016. It allows loading directly to an Excel worksheet or into a Power Pivot model. It will give more flexibility to your end users. It is a GUI-driven interface that is a front-end to the M query language, developed by Microsoft.
Unfortunately, I am not able to help with Power Query too much, but it fits your use case perfectly.
Edit: An additional feature of Power Query, likely not to meet your needs, but I thought I'd throw in.
Power Query can read directly from HTML tables. If you present data in HTML tables, your end users can simply load directly from a URL.
Power Query definitely the "correct" tool for this within Microsoft world. It can also handle JSON and XML (and Odata) directly. How well it manages your data will depend a bit on how nicely formatted it is, but it can work with most things with a little bit of effort.
It is a free Microsoft authored add-in for pre Excel 2016 and built in to Excel 2016.
I have a list in sharepoint which maintains particular month OnCall list,and we are maintaining employee directory in sql server. My requirement is to get complete data from sql server and show it in sharepoint and compare with sharepoint list and show small icon for the employees who are On Call for that particular Month. Can anyone please suggest me the waus of implementing this.
Thanks in advance.
Update: I have finished the part where I have to connect to the sqlserver database and get the employees information. For this we are using 3rd party web part to connect to the sql server and pull the data from the table. Now I have to show some kind of image on the employee name to show that he is on-call for that week. We are going to cretae custom list for maintaing the list of people who are on-Call. Can anyone please advise me on how to accomplish this.
Write a custom webpart which will pull the data from the list using sharepoint object model and SQL server using ADO.NET and do the said comparison.
If you were looking for out of the box, I am afraid there i too little information given here to analyze if its feasible out of the box or not.
If you have the SharePoint Enterprise version, you can look at using the Business Data Catalog. This will let you bind columns to external data sources. This might provide you with the functionality you're looking for.
If you do not have the Enterprise features, do you have access to deploy WSP packages and custom code?
You will have to write your own data access to your external data source. Your options would be to have a job that pulls data from the external data source and populates SharePoint list(s) or create a custom view that pulls the external data on-demand.
You'll have to come up with synchronization strategies. Meaning, is the data in the external SQL data source static, reference information that does not need to be updated depending on what a user does in SharePoint? This seems to be the case based on your question. If you do need to update the external data source, you'll have to hook into the on save event (so probably a custom event handler that listens for ItemAdding) to update the data, validate, and optionally cancel the operation with an error message.
If you can't deploy WSP packages / DLLs, you could take a look at the jQuery SharePoint library. This will let you interact with lists using jQuery. If you also write a WCF or Web Service wrapper around the data you need access to from your external data source that is accessible from the SharePoint environment, you can use hack together a solution.
To accomplish this you'd need to place a Content Editor Web Part on the page you need custom data access. In there you will write the code to reference the jQuery javascript library and jQuery SharePoint library. The code will have to make the calls to your external data service and make any updates you need.
This is the least reliable method to accomplish what you want since it's entirely page-based and can be broken by simply disabling script or someone editing the CEWP or removing it altogether.
If you don't have access to place a CEWP or any of the other solutions, then you have no options at all.
it relatively easy now to pull all the data using the third party webpart and saving it into a custom list. I would recommend you not only creating custom list but also creating the content types for this list. take a look at SharPoint MVP's post about creating a Custom List with Content Types