I am exploring webscraping to try and improve efficiency when inputting data. Unfortunately the website I wish to extract data from is now in tabular format and so I wish to use VBA to manipulate the website to the desired result.
I'm not very familiar with coding/VBA but so far I have got VBA to open a website and search for a provided value. In this case the CAS number 67-64-1 refs Acetone on the website.
The code for this is:
Sub BrowseToSite()
Dim IE As New SHDocVw.InternetExplorer
IE.Visible = True
IE.Navigate "https://apps.who.int/food-additives-contaminants-jecfa-database/Search.aspx#"
Do While IE.ReadyState <> READYSTATE_COMPLETE
Loop
IE.Document.forms("form1").elements("ctl00$ContentPlaceHolder1$txtSearch").Value = "67-64-1"
IE.Document.forms("form1").elements("ctl00$ContentPlaceHolder1$btnSearch").Click
End Sub
Ultimately I wish to create a list in an excel sheet of CAS numbers that this code can loop through and return either the found phrase (in this case No safety concern at current levels of intake when used as a flavouring agent) or simply return a "Not Found". Sometimes the search returns multiple results, for the time being I just wish to take the first result.
This raises 2 problems I'm not sure how to solve:
How can I modify my code to loop through values within a column of a worksheet instead of having to explicitly give each one.
2.I'm unsure how to pull the data into the adjacent column.
Below is an image of the desired output. Column A is inputted by the user and hopefully column B is created by VBA code.
Any help would be appreciated.
what you need is a step by step process for web scraping.
i highly recommend you get familiar with seleniumbasic for vba https://florentbr.github.io/SeleniumBasic/
you need to loop on your excel rows using Range() or Cells(i,1) to read the row.
you check the number of search results using collections
save as many results as you wish in the excel in front of the row using cells(i, k) k being number of returned search results.
unfortunately the website did not load for me to help you further
Related
I am creating a macro in Excel that inserts a table in Word at a specific bookmark. The table insertion process and the process that updates the table is working like I expect it to.
Now, I'm trying to align the data in the table. I'm using the following code to align the data in the table cells:
objTbl.Cell(i, 1).Range.ParagraphFormat.Alignment = wdAlignParagraphCenter
objTbl.Cell(i, 1).Range.ParagraphFormat.Alignment = wdAlignParagraphRight
This works great and is exactly how I want the data to appear.
The only issue I'm having is the data appears at the top of each cell and I would like it in the middle of each cell. Is this possible? I tried to adjust the row height with the following code:
WrdApp.ActiveDocument.Tables(1).Rows.Height = 15
I've tried both 15 and 20 and it didn't make a difference.
My VBA experience has been overwhelmingly within Excel. So this is a bit of a learning curve to program Word from Excel. Any suggestions would be appreciated. Thanks in advance for your help.......
Perhaps you should spend part of your 60 minutes of research in recording a macro. Whilst you wouldn't want to use the recorded code it will point you to the objects and properties you need to use. Of course you do need to know how to use the UI to achieve what you want first.
Sub CellAlignment()
'
' CellAligment Macro
'
'
Selection.SelectCell
Selection.ParagraphFormat.Alignment = wdAlignParagraphCenter
Selection.Cells.VerticalAlignment = wdCellAlignVerticalCenter
End Sub
Having recorded the code you can then use the online help or the object explorer to get a pointer to more usable code, e.g.
ActiveDocument.Tables(1).Range.Cells.VerticalAlignment = wdCellAlignVerticalCenter
I'm trying to create a macro that fetch for specific DATA in URL from Excel sheet and when the DATA is found , it copy the values and past it into the Excel File andmove on to the next DATA
I'm a little bit new with SCRAPPING WEB via EXCEL VBA so Anyone can light me and helps me to carry on ?
I made a lot of searches here in stack over flow but I didn't understand too much
I will explain below with Image what I want to Do and I will show My code which is nothing :(
First URL to use to access to MP :http://XXXX-XXXXX.eu.airbus.XXXX:XXXXX/XXXX/consultation/preSearchMP.do?clearBackList=true&CMH_NO_STORING_fromMenu=true
then Take a value from EXCEL FILE and put it in that Path to make search and tape enter
The last Picture IT will loop for MOD and Copy Value and Paste it into Excel File
Can Anyone guide me please with something
this is my first Code
Dim str As String: str = ThisWorkbook.Sheets("Feuil1").Range("A1").Value
Dim myURL as String
' The & symbol concatenates strings. The _ symbol is for line continuation.
myURL =
"http://Confidential.eu.airbus.Confidential:Confidential/Confidential/" _
& "consultation/preViewMP.do?" & str
If you have your URL then the next stage is to send it. To send it you need something to act as a connector, like Internet Explorer, or a background connector without the browser like XMLHttpServer objects - THIS is a good guide to hopefully get you started.
I am new to VBA and html coding in general. I apologise if I don't understand basic terms or use them incorrectly. I was looking to create and run a macro in excel for work that would make my job a lot easier. Essentially, I need to grab a bunch of information off of a real estate website. This includes address, list price, listing agency, auction date (if any) etc. I have spent the last 4 hours reading all about web scraping and I understand the processes, I just don't know how to code it. From what I have read, I need to write a code to automatically open the website, force-ably wait until it's loaded, then retrieve information by either tag, name or id. Is this correct? How can I go about this. What resources should I look to use.
TL;DR How to web scrape text from a webpage of search results (noob instructions).
I will not tell you all the details, you have to find them on your own. Some web pages are complicated, some are easy. Other are impossible, especially if the text is displayed not in HTML but in some other form - picture, Flash etc.
It is however quite simple to extract data from HTML web pages in Excel. First of all, you want to automate it. So click 'Record macro' on the 'Developer' ribbon. This way, you will have all the reproducible step recorded and then you can have a look at the macro a adjust some steps to suit your needs. I however can't teach you here how to program VBA.
When your macro is being recorded, click on 'From web' on the 'Data' ribbon. This will show up a new web query. Then you enter the address of the web page you want to read and try to select (with the little arrow or check-off mark) as narrow area you are interested in as possible. You can also explore some of the fine-tuning options in this wizard dialog.
When you are done, click 'Import' and you will have in some form the contents of the web page. If you are lucky, the data you are interested in will be always in the same cells. Then you can read the cells and store the values somewhere (possible using another macro). If the data are not in the same cells every time you refresh the query, then you have bad luck and have to use some complicated formulas or macros to find them.
Next stop the macro which you are recording and review the code which was recorded. Try to experiment and play around with it until you discover what you actually need. Then it is up to you, how you want to automate it. The options are many...
Otherwise Excel is maybe not the best tool. If I wanted to load HTML page and extract data from it, I would use some scripting e.g. Python which has much better tools than Excel and VBA. There are also tools for converting HTML to XHTML and then extracting data from it like from well-formed XML.
Below is a very basic example illustrating some of the concepts of web scraping. Other reading you should do, would be how to use the other element selectors such as getElementByID getElementByClassName getElementByName.
Here's some code to get you started.
Public Sub ExampleWebScraper()
Dim Browser As Object: Set Browser = CreateObject("InternetExplorer.Application")
Dim Elements As Object 'Will hold all the elements in a collection
Dim Element As Object 'Our iterator that will show us the properties
'Open a page and wait for it to load
With Browser
.Visible = True
.Navigate "www.google.com"
'Wait for the page to load
While .busy Or .readystate <> 4
Application.Wait (Now() + TimeValue("00:00:01"))
Wend
'Enumerate all Elements on the page
'It will store these elements into a collection which we can
'iterate over. The * is the key for ALL, here you can specify
'any tagName and it will limit your search to just those.
'E.g. the most common is Likely Input
Set Elements = .document.getElementsByTagname("*") ' All elements
'Iterate through all elements, and print out some properties
For Each Element In Elements
On Error Resume Next ' This is needed as not all elements have the properties below
' if you try and return a property that doesn't exist for that element
' you will receive an error
'The following information will be output to the 'Immediate Window'
'If you don't see this window, Press Ctrl+G, and it will pop up. That's where this info will display
Debug.Print "The Inner Text is: " & Element.InnerText
Debug.Print "The Value is: " & Element.Value
Debug.Print "The Name is: " & Element.Name
Debug.Print "The ID is: " & Element.ID
Debug.Print "The ClassName is: " & Element.Class
Next Element
End With
'Clean up, free memory
Set Browser = Nothing
Set Elements = Nothing
Set Element = Nothing
End Sub
Is there a way i can check if a US number is a cell phone or landline?
I have found several sites online, but they are paid per number. Can i use code to do the check?
Thanks
This is less of an Excel question and more of a general question about phone numbers. If there's some way (I don't know of one) that you can freely check a phone number against some registry, then we could use some code to do this in Excel - sure. However, I'm not sure how to check whether a line is mobile or land line.
I did find this SO thread, but it looks like the free options are limited. I did find this site just Googling around, but I have no idea how to check if that's accurate or not. There's this site which also charges, but could help you.
Again, if you (or someone) does find a way to check phone numbers, you could probably automate it with VB, but getting to that point is the tricky part.
(Note: the webpages I linked above I just found Googling around).
I have worked on an admittedly crude way to do what you've requested, using the free service phonevalidator.com. I cannot attest to the accuracy of this service, but I tried for several cell numbers I know and several landlines as well and it was accurate for them.
This option uses VBA to go to the website, automatically enter a number, and scrape the phone type off of the results page. I have made the scraping a function (probably could be optimized better, but it works).
In a standard code module, start with this function:
Public Function GetPhoneNumType(phoneNum As String) As String
Dim ie As Object
Dim objResult As Object
'Initialize IE
Set ie = CreateObject("InternetExplorer.Application")
'Go to the phone validator
ie.Navigate "http://www.phonevalidator.com/index.aspx"
'Wait for the page to load
Do While ie.ReadyState <> 4
DoEvents
Loop
'Enter the phone number
ie.document.getElementByID("ContentPlaceHolder1_txtPhone").Value = phoneNum
'Click the search button
ie.document.getElementByID("ContentPlaceHolder1_SearchButton").Click
'Wait until javascript executes and displays the PhoneTypeLabel
On Error Resume Next
Do Until Not objResult Is Nothing
Set objResult = ie.document.getElementByID("ContentPlaceHolder1_PhoneTypeLabel")
Loop
On Error GoTo 0
'Return the phone type result, either LANDLINE or CELL PHONE
GetPhoneNumType = objResult.innerText
End Function
This takes in a phoneNum as a string and returns the result either as LANDLINE or CELL PHONE (I'm not sure if there are any other return types).
For purposes of this example, let's assume that I entered two phone numbers (these can be with or without dashes, but should have no other symbols and no leading 1, etc.) in cells A1 and A2, I can call this function like so:
Public Sub TestPhoneNum()
Dim rngNumbers As Range, c As Range
Set rngNumbers = Sheets("Sheet1").Range("A1:A2")
For Each c In rngNumbers
c.Offset(0, 1).Value = GetPhoneNumType(c.Value)
Next c
MsgBox "Complete!"
End Sub
This will put the phone number type in the column adjacent (to the right) of each cell that contains a phone number. You can adjust A1:A2 to be whatever range has the phone numbers. Whichever range you enter, this subroutine will always place the phone number type in the right adjacent cell.
I added the message box because this code is slow, and that notified me when the code was done executing.
Hopefully this gets you at least started.
Sub TEST()
If cells(i, "R").Value <> "UK" Then
cells(i, "R").Interior.ColorIndex = 3
End If
End Sub
If I run this program it throws application defined error \
I am new to Excel (beginner)
How to correct this error!!!
Thanks In advance
I think the issue is "R" that I know of the cells method takes 2 parameters one is rows the other is columns (in that order) but this is done by number not letter so if you change it to cell(1,18) then the code above works fine.
This link may also be useful to learn more, among other things it describes how you would normally select a range first as I believe your code above will assume the currently selected page, however you might want to run in on a button click from another page or as soon as the spreadsheet opens.
http://msdn.microsoft.com/en-us/library/office/ff196273.aspx
The problem is that the variable i has not been assigned a value. VBA assumes that it is zero. Since i is used to determine the row of the cell, Excel throws an exception because there is no row 0!
First you have to define i variable
for example: Dim i as variant