Is it possible for me to create a list of google search results from a specific query and export it into excel? For example, I'd like to google orthodontists in Florida and be able to export the business name, phone number and address to an excel spreadsheet. I've done a lot of searching but I can't find any solutions. I'm looking for someone to point me in the right direction. Any help is appreciated, thanks.
An API is an Application Programming Interface and it's a way for your software to interact with the software on a server. Google has an API called the "Custom Search Engine" which you can use for 100 free queries per day. Other search engines may have more generous free APIs. With a search API you can write a code to download text that contain all the relevant data. You can read more about search engine APIs here.
Another way to collect data from google is to scrape their page. This means that you use a code to download the HTML, and from that HTML you collect the relevant pieces (wikipedia link). With a programming language like python, many people use the Beautiful Soup library for scraping. With code then you can take the relevant parts of the HTML and put it into a format like CSV that is readable by Excel. With python there are ways to write to Excel, directly, too (link).
Finally, here is a link from 2007 that says with Google Spreadsheets you can import HTML.
Update: here is the MS Excel version.
The following web app https://www.resultstoexcel.com/ allows you to download Google search results to a CSV file, a Microsoft Excel readable format, for free.
If you have any problem viewing the downloaded results correctly in MS Excel, please read the FAQs section where you will find how to open the file using the correct column separators.
Where are the results coming from?
A Google search on the topic retrieves many companies that offer online access to Google Search Results through an Application Programming Interface (API). This web app uses SERPSBOT API.
Related
Bellowing is example fuction in Microsoft Word 2016. Google Docs have function or any extension support select value from range?
I only few line need select from drop down. So i don't want using Google Sheet, difficult convert linked table of content, different page header footer or other to sheet.
I get reponse from Google Docs Editors Community .
Bellowing is the response.
That capability currently isn't available in Docs. To help influence future feature changes or additions in Docs, I encourage you to provide feedback about this directly to the Docs team by following these steps:
Open a document.
Click Help > Report a problem.
Enter your feedback.
Although the product team is unable to respond to individual users directly, they review this feedback regularly and use it to help improve the product over time. The more users who have this request, the more likely it is the team will make a change.
So, conclusion. Google doesn't support yet.
I am trying to fetch some data from the web pages using Excel VBA and have been fairly successful.
However I have realized that most of the pages do have data-layer available on the page and thus if I am able to use the same, lot of effort can be reduced in massaging the data to bring it in usable format.
I tried to call the data layer by Document.getElement method but this seems not to be working.
I am not a hard core developer just can swim for my needs so please let me know if this is possible as all of my search results so far have yielded nothing.
I normally use SeleniumBasic (Excel Plugin) for WebScraping Needs. They way it can be done is by using
driver.ExecuteScript("return dataLayer[x].variableName")
Having had the luxury of having my fun and earning my bread well away from MS products for years, I am today trying to programmatically search with Bing Azure (wearing gloves) basically because I thought getting a google api was complex. So I headed down Data Market and issued this (let's say with perl's LWP which has been used to pass credentials):
https://api.datamarket.azure.com/Bing/Search/v1/Composite?%24skip=0&%24top=10&%24format=json&Sources=%27web%27&Query=%27abc%27
which works.
What I am now trying to find out is
1) how to tell the Bing search api to restrict results to a specific domain (e.g. ".org" or even a single website "www.wikipedia.org").
2) how to tell the search engine to restrict results to a specific filetype, e.g. 'PDF', 'XML' (or PDF and XML if that's possible)
3) if there is a simple list of the features/keywords in the GET request of the latest bing search API. Please no MS links if you please - i am really tired.
I have seen "site:.org" working on the bing search website when doing a manual search. And read about "filetype:pdf" working too.
Any hints?
bliako
cracked it:
... Query=%27abc site:.com filetype:pdf%27
at the point when m$ realises it costs to be clumsy
bliako
we are currently building an application on the google cloud platform, which generates reports in Google Doc. For them, it is really important to have a table of content ... with page numbers. I know this is a feature request since a few years and there are add-ons (Paragraph Styles +, which didn't work for us) that provide this solution, butt we are considering to build this ourselves. if anybody has a suggestion on how we could start with this, it would be a great help!
thanks,
Best bet is to file a feature request on the product forums.
Currently the only way to do that level of manipulation of a doc to provide a custom TOC is to use Apps Script. It provides access to the document structure sufficient enough to build and insert a basic table of contents, but I'm not sure there's enough to do paging correctly (unless you force a page break on ever page...) There's no method to answer the question of "what page is this element on?"
Hacks like writing to a DOCX and converting don't work because TOCs are recognized for what they and show up without page numbers.
Of course you could write a DOCX or PDF with the TOC as you'd like and upload as a blob rather than as a Google Doc. They can still be viewed in Drive and such.
I have a list of books listed by their titles in a text file. I want to write a script which can use a web service like Google scholar or amazon to search for the books and return me a xml or bibtex file with citation info for each book.
Which programming tools can I use for this kind of automated search ?
Python would be my recommendation.
Get names from the text file, simple file reading
Construct a REST URL request to google's book API
http://books.google.com/books/feeds/volumes?q=Elizabeth+Bennet&start-index=21&max-results=10
Simple python code to get data from this URL (may need an API key, would advise using urllib2 with error handling rather than urllib)
Sample code,
import urllib
url = 'http://foo.api.request'
data = urllib.urlopen(url).read()
See the return schemas for this API (you can use the XML however you like).
See BibTeXML for conversion between the two formats.
HTH
I think it could be useful if you specify what kind of script you want to write!
Anyway... you could do some low level work and write your own HttpRequest for google and amazon or you could just rely on their API for example: http://code.google.com/apis/books/
There is a great project which does something similar what you want to do, it's called Shelves. It's written for Android but should give you some ideas how to handle your requests. Instead of downloading some citations it's downloading the cover.
http://code.google.com/p/shelves/
Just as a quick side note, saving your books in a xml file could be an option as well. In some cases it makes parsing them easier.