I am using excel's web query feature for the first time. The site I am using it on is here. I put a check mark next to the table TTM Yield Load etc.. I click import, but a different table than the one I selected is produced in excel. It's one far down the page. What am I doing wrong?
After further reflection, I discovered that it is the dynamic nature of this web site that appears to be causing some trouble. I have opted to use selenium for vba and so far the solution has proven to be effective, yet slow. If anyone out there would like to offer some other avenue to web scraping dynamic pages using vba for excel, I am all ears. Thanks.
Related
I would like to display a graph on the home page of my sharepoint online site. I can add in a quick chart but it does not fit my requirements.
Basically, I have a task logger where i log all incoming tasks, they are assigned a status - closed, completed, in progress, on hold, and pending.
I would like to display this visually on the home page so we can see how many tasks per status.
I think the x axis would the status and the y axis would be the value.
I'm not sure how to achieve this? i've seen multiple examples of coding for this but being inexperienced in coding, i'm not sure how to amend or adapt the codes for my requirements.
If someone could please provide a solution or have any ideas, that would be greatly appreciated.
We can use the solution below from GitHub to achieve it.
Modern Charts
Or we can use modern script editor web part with chart js plugin and REST API to achieve your requirement.
In classic site page, we can use script/content editor web part with custom JavaScript code to achieve it.
If You don't plan to code the solution I would suggest two different then quick chart webpart approaches:
File Viewer webpart - You could create some excel file to store all Your task list there. This file could be stored on some document library. The excel file could have data in one tab and chart in other. Using File viewer You could add this excel file on any SharePoint page. On one Page You could present the tab to change data (tasks), on other the tab that has the chart. I know..a bit of a workaround but this is one of the simplest solutions I thought of without coding :)
Power BI- this tool for sure may will fit any needs as in this tool You may do almost anything. Of Course You need Power BI server for this. You may create any chart/report with this also from data taken directly from SharePoint
Hope this will be of any help to You
I would like to create a spreadsheet that I can refresh and pull in each weeks English premier League fixtures, each week I would like to refresh this and see the weeks future fixtures. I have tried to use the import function from Data/From Web and selected the box with the table of fixtures however no data gets pulled into the spreadsheet.
The website I am using is - "http://data.7m.com.cn/matches_data/92/en/index.shtml"
I am open to understand a better way of doing this import and also if there is a better website to use I am also happy to change. I have chosen this one as it seems to have the most simplified listing of the fixtures.
I have also tried this website - https://www.premierleague.com/fixtures
When the import completes it actually skips all the fixtures and returns all the other information.
Should i be looking to some of the HTML elements within the script of the web page to extract the data?
For example on the following site - https://www.premierleague.com/fixtures I am looking for a file received by the website that updates the fixtures each week (after some direction from Google) I hit the F12 command and look within the "Network" tab however I cant understand how the website, this or the others quoted create the weekly fixtures.
Any suggestions on how to pull this into Excel or another tool would be fantastic.
Welcome to [so]! it sounds like you haven't done as much research as you could have. Your first link, in the top corner has links to "Free Feed" which take you to customizable widgets and from there is a link to a customizable live template.The first page also has a link to "Data" , I'm not sure what that consists of or whether it will help (since I'm not much of a sports fan on my continent, and even less on yours!
As for importing into Excel, I didn't have an issue with the table I could see, but once again I'm not clear on what data you're trying to get and what you want to do with it.
On the ribbon's Data tab click From Web.
Enter the first URL from your question and hit Enter
When the Navigator window loads, click "Table 1" and then click Load.
Below is what Excel then automatically loaded as a table:
If instead of clicking Load, you were to click Edit then you are brought into the Power Query Editor, where you can customizable tons of stuff. The one I was interested in was Use First Row on Headers. After choosing that, and clicking Close & Load, and 30 seconds of formatting later I had:
With Power Query you can choose, remove, split, or combine columns from this or other tables. It's fairly advanced but you should be able to find a good Power Query tutorial online, to see examples of what you can do, to learn about other ways you can customize the import and/or analysis of the data.
Edit:
More Information:
Here are the instructions for all versions:
Office Support : Connect to a web page (Power Query)
I am trying to fetch some data from the web pages using Excel VBA and have been fairly successful.
However I have realized that most of the pages do have data-layer available on the page and thus if I am able to use the same, lot of effort can be reduced in massaging the data to bring it in usable format.
I tried to call the data layer by Document.getElement method but this seems not to be working.
I am not a hard core developer just can swim for my needs so please let me know if this is possible as all of my search results so far have yielded nothing.
I normally use SeleniumBasic (Excel Plugin) for WebScraping Needs. They way it can be done is by using
driver.ExecuteScript("return dataLayer[x].variableName")
I have a document library set up with multiple different categories of document, and I'm using a metadata column to differentiate between them.
I want to be able to display two different document library web part on a page for different categories of file side by side. This is simple for one category, I just set up a list view filtered by the metadata column, but when I add a second web part alongside the first, it breaks the first one.
I have no idea why this is happening, but it seems like SharePoint isn't happy with pulling two sets of data from the same document library.
When I am editing the web parts, I can get them to both display the documents I want, but then when I click save, the first web part empties.
Not sure what other information would be useful for diagnosing or helping with the problem, so if I haven't given enough detail let me know. I am familiar with SPD as well as developing through the web interface, so if this needs a more complex solution that's fine with me!
Having spent some more time playing around with this, it struck me that I could probably achieve what I wanted using something other than a Document web part, and I was right.
Instead of using the somewhat inflexible document web part, I created a content query web part which only searched within the document library from my site, and filtered by the metadata column.
This way I can create as many queries as I like and they don't interact with each other in weird ways. It also has the advantage of being significantly easier to customise the output without needing to resort to SharePoint Designer.
Content Queries are the answer!
I have a link in one column and, based on it, I want
Number of Google searches in column 2
Page rank of first result in column 3
I know this can be done, as I saw a friend pulling google search result right in Excel. If anyone knows, please share how I could do that.
If I correctly interpret your question, one of the tasks you had to do is
How do I get programmatically the Google page rank for a list of URLs?
You can find the code to do this in this CodeProject article:
Request Google´s Pagerank programmatically
Regarding the Excel part: it depends which programming framework or platforms you could use. You could use to create a .NET extension for Excel using the Microsoft Visual Studio Tools for Office.
From Excel there is Data->Get External Data->New Web Query. Is this what you want?
You have two options, both of which are unfortunately poorly documented.
If you are comfortable in C/C++, you can write a special DLL called an "XLL" that you can call during Excel runtime. There is some sparse documentation available. Note that this stuff isn't very fun to use.
If you prefer .NET, there is a binding for the entire Office suite outlined here that allows you to write COM-based methods that you can call from Office. It is intended for automation, but you can write any managed code you want and have Excel call into it.
There is also what Remou just suggested; I don't actually own a copy of Excel to test that out, but it may be the easiest option.
By link i meant keywords and not URL. I want to put a keyword in one cell and pull number of searches and page rank in adjacent cells.
I tried doing the same with web query in excel but i can only reach till the number of searches. that too not in the proper cell (trying to figure out). But i have no clue about how to get the pageranks.
I am not that tech savvy to code a binder or plugin for myself. Although i am checking the link by splattne. Please focus more light on it. Is it gonna be time consuming if i try to make this one..?
Regards
Thinkjayant
There are some nice plugins for this (in various languages) on GitHub:
http://github.com/search?langOverride=&language=&q=pagerank&repo=&start_value=1&type=Repositories&x=0&y=0
I have a PR checker functions in my Excel plugin "SeoTools".
http://nielsbosma.se/projects/seotools/