Text retrieved on multiple rows on Excel VBA - excel

I'm a kind of new to Excel VBA. Here's the problem:
Given a identifier, I want to retireve some text from a web page. Ideally I want to store the text of the page in a single cell. I created a function that creates a QueryTable but, sometimes, the retrieved text is copied on multiple rows.
Is there a way to place all the text on a single cell?
Here's the code of my function:
Function Articolo(myRange As Range, code As String)
Dim myURL As String
Dim myName As String
myURL = "URL;http://techstore.runner.it/feed/dettagli_csv.php?codcli=111367&pwd=03142110786&sku=" & code
myName = "dettagli_csv.php?codcli=111367&pwd=03142110786&sku=" & code
With ActiveSheet.QueryTables.Add(Connection:= _
myURL _
, Destination:=myRange)
.Name = myName
.FieldNames = True
.RowNumbers = False
.FillAdjacentFormulas = False
.PreserveFormatting = False
.RefreshOnFileOpen = False
.BackgroundQuery = True
.RefreshStyle = xlOverwriteCells
.SavePassword = False
.SaveData = True
.AdjustColumnWidth = False
.RefreshPeriod = 0
.WebSelectionType = xlEntirePage
.WebFormatting = xlWebFormattingNone
.WebPreFormattedTextToColumns = False
.WebConsecutiveDelimitersAsOne = False
.WebSingleBlockTextImport = True
.WebDisableDateRecognition = False
.WebDisableRedirections = False
.WebConsecutiveDelimitersAsOne = True
.Refresh BackgroundQuery:=False
End With
End Function
As test you can use 8E4374 as code
Thanks for your help!

Querytables are often slow and cumbersome. If you use one of the httprequest objects instead it's much quicker and you have more control over how to parse the response. Below is basic example that doesn't manage sessions or check if the page is cached.
Option Explicit
Sub test()
Dim rng As Range
Dim code As String
Set rng = Sheet1.Range("A1")
code = "8E4374"
Articolo rng, code
End Sub
Sub Articolo(myRange As Range, code As String)
Dim myURL As String
Dim myName As String
myURL = "http://techstore.runner.it/feed/dettagli_csv.php?codcli=111367&pwd=03142110786&sku=" & code
myRange.Value = ExecuteWebRequest(myURL)
End Sub
Function ExecuteWebRequest(ByVal url As String) As String
Dim oXHTTP As Object
Set oXHTTP = CreateObject("MSXML2.XMLHTTP")
oXHTTP.Open "GET", url, False
oXHTTP.send
ExecuteWebRequest = oXHTTP.responseText
Set oXHTTP = Nothing
End Function
EDIT: the above code is designed to run as a Sub rather than a UDF. Since a UDF cannot affect other cells the only option is to return the string to the calling cell or call set up the code to run as either an event or from a control (eg a button)
Below is an example UDF, it's called from Excel using =Articolo(C1) where C1 is any cell containing the required code eg 8E4374
Option Explicit
Function Articolo(ByVal code As String) As String
Dim myURL As String
myURL = "http://techstore.runner.it/feed/dettagli_csv.php?codcli=111367&pwd=03142110786&sku=" & code
Articolo = ExecuteWebRequest(myURL)
End Function
Function ExecuteWebRequest(ByVal url As String) As String
Dim oXHTTP As Object
Set oXHTTP = CreateObject("MSXML2.XMLHTTP")
oXHTTP.Open "GET", url, False
oXHTTP.send
ExecuteWebRequest = oXHTTP.responseText
Set oXHTTP = Nothing
End Function
Also, as this is making http requests, everytime you force a full re-calc your UDF's will also re-calc which is probably not what you want as you could be making hundreds or thousands of requests. I would suggest running it once only from a defined loop such as
For Each code in Listofcodes : <Download Page> : Next code

It looks like the data coming from that URL has carriage return and line feeds embedded in it. That's why it's separating into different cells in excel.
One solution would be to run VBA code to do the query, strip out the carriage return/line feed characters and then put the results into a cell. The problem would be that you'd have to run the code to update, rather than excel taking care of the refreshes.
A simpler answer might be to add another cell with a formula like this:
=A1&" "&A2&" "&A3&" "&A4

Related

QueryTables.Add(Connection) ERROR , Expected '='

So, first of all, i am not familiar with VBA. I've been trying to learn VBA because at my job that's what drives the backbone of our database and our excel workflow (although no one at my division seems to know how any of it works). And i've been trying to write a macro that could automate some of the work i have to do, namely: Importing CSV files into an excel to process them. I've been trying to approach this problem step by step, by dividing into smaller subsets of problems.
So I have been able to add a query into my excel, that allows it to look into a folder to see which files are in there. This allows the macro to 'see' different file paths and filenames.
What i am trying to accomplish now is to have the macro loop through all the files it can see in the folder, and import them to a sheet called 'output'.
No matter what i've tried, and how much research i did, i can't figure out the import macro. It keeps giving me a compile error that it expects '='
(I know that the macro is able to read the filenames etc correctly, because i have created a sub that makes it 'log' whatever it reads, to test its ability to differentiate between file types)
Dim ImportFolder As String
Dim ImportRow As Range
Dim ImportFilename As Range
Dim ImportFilenameS As String
Dim ImportAccessDate As Range
Dim ImportFilePath As Range
Dim ImportExtension As Range
Dim ImportRange As Range
Dim ImportVar As Integer
Dim ImportLength As Integer
Dim L
Dim LogRow As Range
Dim LogFilename As Range
Dim LogAccessDate As Range
Dim LogFilePath As Range
Dim LogStatus As Range
Dim LogReason As Range
Dim OutputFolder As String
Dim OutputRange As Range
Sub FileTypeController()
Set ImputFolder = "H:\BLM.Workflow\CSV.Workflow-Input\"
Set ImportExtension = ThisWorkbook.Worksheets("Import").Range("B2")
Set ImportFilename = ThisWorkbook.Worksheets("Import").Range("A2")
Set ImportAccessDate = ThisWorkbook.Worksheets("Import").Range("C2")
Set ImportFilePath = ThisWorkbook.Worksheets("Import").Range("G2")
Set LogFilename = ThisWorkbook.Worksheets("Log").Range("A2")
Set LogAccessDate = ThisWorkbook.Worksheets("Log").Range("B2")
Set LogFilePath = ThisWorkbook.Worksheets("Log").Range("C2")
Set LogStatus = ThisWorkbook.Worksheets("Log").Range("D2")
Set LogReason = ThisWorkbook.Worksheets("Log").Range("E2")
Set ImportRange = ThisWorkbook.Worksheets("Import").Range("A:A")
ImportVar = WorksheetFunction.CountA(ImportRange)
ImportLength = (ImportVar - 1)
For L = 1 To ImportLength
If ImportExtension.Value = ".csv" Or ImportExtension.Value = ".txt" Then
Call CSVToOutput Else Call ExtensionTypeFailedImport
Set ImportExtension = ImportExtension.Offset(1, 0)
Set ImportFilename = ImportFilename.Offset(1, 0)
Set ImportAccessDate = ImportAccessDate.Offset(1, 0)
Set ImportFilePath = ImportFilePath.Offset(1, 0)
Set LogFilename = LogFilename.Offset(1, 0)
Set LogAccessDate = LogAccessDate.Offset(1, 0)
Set LogFilePath = LogFilePath.Offset(1, 0)
Set LogStatus = LogStatus.Offset(1, 0)
Set LogReason = LogReason.Offset(1, 0)
Next
ThisWorkbook.Worksheets("Log").Activate
End Sub
Sub CSVToOutput()
ImportFilenameS = "TEXT;" & ImportFilename.Value
Set OutputRange = ThisWorkbook.Worksheets("Output").Range("A1")
'Application.CutCopyMode = False
With ThisWorkbook.Worksheets("Output").QueryTables_
.Add(Connection:= ImportFilenameS,Destination:= OutputRange)
.Name = "Importfilename"
.FieldNames = True
.RowNumbers = False
.FillAdjacentFormulas = False
.PreserveFormatting = True
.RefreshOnFileOpen = False
.RefreshStyle = xlInsertDeleteCells
.SavePassword = False
.SaveData = True
.AdjustColumnWidth = True
.RefreshPeriod = 0
.TextFilePromptOnRefresh = False
.TextFilePlatform = 1252
.TextFileStartRow = 1
.TextFileParseType = xlDelimited
.TextFileTextQualifier = xlTextQualifierDoubleQuote
.TextFileConsecutiveDelimiter = False
.TextFileTabDelimiter = False
.TextFileSemicolonDelimiter = False
.TextFileCommaDelimiter = True
.TextFileSpaceDelimiter = False
.TextFileColumnDataTypes = Array(1)
.TextFileTrailingMinusNumbers = True
.Refresh BackgroundQuery:=False
Call SuccessfulImport
End Sub
It's still a first draft, so i haven't taken the time to clean it up a little yet.
Apologies if the code is trash, but, i'm giving it my all ;-)
At first I spot an error here:
Set ImputFolder = "H:\BLM.Workflow\CSV.Workflow-Input\"
Reason: you only use the reserved word Set with objects, and in this case, you are just assigning a value to a string variable.
Also, I didn't see that variable defined anywhere. Maybe you meant to use this one:
Dim ImportFolder As String
Those facts aside, I suggest that you look into Power Query. If you have Excel 2010, 2013 you can download it as an add-in or if you have 2016-2019 is already available in the Ribbon's Data tab.
Check Matt’s tutorial on how to combine files from a folder here

VBA code - connect to webpage and retrieve value

I have the following
Column A == FEdEX AWB #s
Column B == Delivery date (Empty)
I would like to write a function where it reads the tracking number on Column A and extracts the delivery date from the website - all AWB # are delivered - 100% sure
The code I have writes all the info found in the website into the sheet - not sure how to extract only the delivered date.
Sub Macro1()
With ActiveSheet.QueryTables.Add(Connection:= _
"URL;https://www.bing.com/packagetrackingv2?
packNum=727517426234&carrier=Fedex&FORM=PCKTR1" _
, Destination:=Range("$A$1"))
.Name = _
"https://www.bing.com/packagetrackingv2?
packNum=727517426234&carrier=Fedex&FORM=PCKTR1"
.FieldNames = True
.RowNumbers = False
.FillAdjacentFormulas = False
.PreserveFormatting = True
.RefreshOnFileOpen = False
.BackgroundQuery = True
.RefreshStyle = xlInsertDeleteCells
.SavePassword = False
.SaveData = True
.AdjustColumnWidth = True
.RefreshPeriod = 0
.WebSelectionType = xlEntirePage
.WebFormatting = xlWebFormattingNone
.WebPreFormattedTextToColumns = True
.WebConsecutiveDelimitersAsOne = True
.WebSingleBlockTextImport = False
.WebDisableDateRecognition = False
.WebDisableRedirections = False
.Refresh BackgroundQuery:=False
End With
End Sub
A function, getting passing the airway bill number and returning the date would be quite enough:
Function GetDateFromAwb(awbNumber As String) As String
Dim objIE As New InternetExplorer 'Microsoft Internet Controls library added
objIE.Visible = False 'Or put True, if you want to see the IE
objIE.navigate "https://www.fedex.com/apps/fedextrack/?tracknumbers=" & awbNumber
Do While objIE.Busy = True Or objIE.readyState <> 4: DoEvents: Loop
Application.Wait (Now + TimeValue("0:00:05"))
GetDateFromAwb = objIE.Document.getElementsByClassName("redesignSnapshotTVC snapshotController_date dest").Item.InnerText
objIE.Quit
End Function
The idea of the function is to append the airbill string number to the URL and to open the corresponding site. Then, using the class "redesignSnapshotTVC snapshotController_date dest", the corresponding date is taken.
This is a possible way to call the function, displaying the date in a MsgBox:
Sub Main()
Dim awbNumber As String
awbNumber = 727517426234#
Dim awbDate As String
awbDate = GetDateFromAwb(awbNumber)
MsgBox awbDate
End Sub
Make sure that the library "Microsoft Internet Controls" is added from the VBE menu>Extras>References:
Rather than using a browser you could use xmlhttp request which is quicker.
The page does a form XHR POST request which returns json you can parse (lots of info returned including a delivery date field). You can use this as a function in the sheet. I also show a test call. The id (tracking number) is passed as an argument to the function GetDeliveryDate.
Here is the request made when you submit your tracking number on the site:
As you can see from the above, and further detailed in code, the tracking number is part of the body sent in the request (data param); it is also part of one of the request headers.
I use jsonconverter.bas to parse the json response. After adding the code from there to your project you need go VBE > Tools > References and add a reference to Microsoft Scripting Runtime.
View the json response here
As you say all requests will return a delivery date, if you don't want to load this external library you could use split to isolate the date.
Relevant json:
You can see relevant part of json here:
I use the field actDeliveryDt for version of code using split as I can separate an unambiguous date yyyy-mm-dd from the datetime string. I use displayActDeliveryDt for json parsing though you could use either (removing time part with split if usnig the former as shown in examples below)
Caveat: I have had only one delivery id to use for testing.
TODO:
You could add in a test for whether a valid request was made as the json response includes a field for this.
If performing this for multiple requests I would recommend, for efficiency, to re-write using a sub which loops an array of the tracking numbers, stores results in an array and writes that array out in go at end.
VBA:
JSON parsing:
Option Explicit 'example test call from VBE
Public Sub test()
Debug.Print GetDeliveryDate(727517426234#)
End Sub
Public Function GetDeliveryDate(ByVal id As Double) As Date
Dim json As Object, body As String '< VBE > Tools > References > Microsoft Scripting Runtime
body = "data={""TrackPackagesRequest"":{""appType"":""WTRK"",""appDeviceType"":""DESKTOP"",""supportHTML"":true,""supportCurrentLocation"":true,""uniqueKey"":"""",""processingParameters"":{},""trackingInfoList"":[{""trackNumberInfo"":{""trackingNumber"":" & Chr$(34) & CStr(id) & Chr$(34) & ",""trackingQualifier"":"""",""trackingCarrier"":""""}}]}}"
body = body & "&action=trackpackages&locale=en_US&version=1&format=json"
With CreateObject("MSXML2.XMLHTTP")
.Open "POST", "https://www.fedex.com/trackingCal/track", False
.setRequestHeader "Referer", "https://www.fedex.com/apps/fedextrack/?tracknumbers=" & CStr(id)
.setRequestHeader "User-Agent", "Mozilla/5.0"
.setRequestHeader "X-Requested-With", "XMLHttpRequest"
.setRequestHeader "Content-Type", "application/x-www-form-urlencoded; charset=UTF-8"
.send body
Set json = JsonConverter.ParseJson(.responseText)
End With
GetDeliveryDate = json("TrackPackagesResponse")("packageList")(1)("displayActDeliveryDt")
End Function
Using split:
Option Explicit
Public Sub test()
Debug.Print GetDeliveryDate(727517426234#)
End Sub
Public Function GetDeliveryDate(ByVal id As Double) As Date
Dim s As String, body As String
body = "data={""TrackPackagesRequest"":{""appType"":""WTRK"",""appDeviceType"":""DESKTOP"",""supportHTML"":true,""supportCurrentLocation"":true,""uniqueKey"":"""",""processingParameters"":{},""trackingInfoList"":[{""trackNumberInfo"":{""trackingNumber"":" & Chr$(34) & CStr(id) & Chr$(34) & ",""trackingQualifier"":"""",""trackingCarrier"":""""}}]}}"
body = body & "&action=trackpackages&locale=en_US&version=1&format=json"
With CreateObject("MSXML2.XMLHTTP")
.Open "POST", "https://www.fedex.com/trackingCal/track", False
.setRequestHeader "Referer", "https://www.fedex.com/apps/fedextrack/?tracknumbers=" & CStr(id)
.setRequestHeader "User-Agent", "Mozilla/5.0"
.setRequestHeader "X-Requested-With", "XMLHttpRequest"
.setRequestHeader "Content-Type", "application/x-www-form-urlencoded; charset=UTF-8"
.send body
s = .responseText
End With
GetDeliveryDate = Split(Replace$(Split(Split(s, """actDeliveryDt"":""")(1), Chr$(34))(0), "\u002d", "-"), "T")(0)
End Function
Example usage in sheet:
Note: I have UK format dd/mm/yyyy in sheet

Using For loop through string +

First, I want to say that this is my first attempt at building vba code. I am trying to extract data from the web using a web query .Add(Connection,Destination,sql). What I want my code to do is to loop through the string 'str' containing stock tickers to be inserted into my url using a for loop and pasting the table data in the active sheet.
In addition, it would be an extra if I could create a new sheet for every url queried with the corresponding NYSE name.
Currently my code does not run because it is not extracting the data. I think the error is in how I am specifying the url using the loop index NYSE(i).
Thanks for any responses, advice, and suggestions.
Sub URL_Get_Query()
Dim NYSE(1 To 22) As String
NYSE(1) = "APC"
NYSE(2) = "APA"
NYSE(3) = "COG"
NYSE(4) = "CHK"
NYSE(5) = "XEC"
NYSE(6) = "CRK"
NYSE(7) = "CLR"
NYSE(8) = "DNR"
NYSE(9) = "DVN"
NYSE(10) = "ECA"
NYSE(11) = "EOG"
NYSE(12) = "XCO"
NYSE(13) = "MHR"
NYSE(14) = "NFX"
NYSE(15) = "NBL"
NYSE(16) = "PXD"
NYSE(17) = "RRC"
NYSE(18) = "ROSE"
NYSE(19) = "SD"
NYSE(20) = "SWN"
NYSE(21) = "SFY"
NYSE(22) = "WLL"
For i = 1 To 22
Debug.Print NYSE(i)
With ActiveSheet.QueryTables.Add(Connection:= _
"URL;http://finance.yahoo.com/q/ks?s=NYSE(i)+Key+Statistics", _
Destination:=Range("a1"))
.BackgroundQuery = True
.TablesOnlyFromHTML = True
.Refresh BackgroundQuery:=False
.SaveData = True
End With
Next i
End Sub
See how this works for you:
Dim NYSE_List As String, i As Long
Dim NYSE
NYSE_List = "APC,APA,COG,CHK,XEC,CRK,CLR,DNR,DVN,ECA,EOG,XCO,MHR,NFX,NBL,PXD,RRC,ROSE,SD,SWN,SFY,WLL"
' this is easier to maintain. Split the list at the commas.
' No need to count absolute numbers, either.
NYSE = Split(NYSE_List, ",")
For i = 0 To UBound(NYSE)
Dim ws As Worksheet
' Insert a new worksheet after the last one (each time)
Set ws = Worksheets.Add(after:=Worksheets(Worksheets.Count))
ws.Name = NYSE(i)
Debug.Print NYSE(i)
' assemble the variable into the string:
With ws.QueryTables.Add(Connection:= _
"URL;http://finance.yahoo.com/q/ks?s=" & NYSE(i) & "+Key+Statistics", _
Destination:=ws.Range("a1"))
' note that the range must address the proper worksheet object
.BackgroundQuery = True
.TablesOnlyFromHTML = True
.Refresh BackgroundQuery:=False
.SaveData = True
End With
Next i

Download eBay shipping charges through Excel 2010 VBA

I am trying to download non-free shipping charges from eBay. I have the item numbers of the pages. The links should go to the right pages on eBay.
While trying to go to the page and download the data, Excel hangs.
I have similar working code that gets the eBay item numbers on many pages from eBay.
If this code can't be fixed how can I get the info I need into Excel?
itemNumberAlone = Range("a" & eachItem).Value
With ActiveSheet.QueryTables.Add(Connection:= _
"URL;http://www.ebay.com/itm/" & itemNumberAlone & "?ru=http%3A%2F%2Fwww.ebay.com%2Fsch%2Fi.html%3F_from%3DR40%26_sacat%3D0%26_nkw%3D" & itemNumberAlone & "%26_rdc%3D1" _
, Destination:=Range("$bZ$1"))
.Name = "second ebay links"
.FieldNames = True
.RowNumbers = False
.FillAdjacentFormulas = False
.PreserveFormatting = True
.RefreshOnFileOpen = True
.BackgroundQuery = True
.RefreshStyle = xlOverwriteCells
.SavePassword = False
.SaveData = True
.AdjustColumnWidth = True
.RefreshPeriod = 0
.WebSelectionType = xlEntirePage
.WebFormatting = xlWebFormattingNone
.WebPreFormattedTextToColumns = True
.WebConsecutiveDelimitersAsOne = True
.WebSingleBlockTextImport = False
.WebDisableDateRecognition = False
.WebDisableRedirections = False
.Refresh BackgroundQuery:=False
End With
Do While Not IsError(Application.Match("Shipping and handling", Range("bz1:bz1000"), 0))
If IsError(Application.Match("Shipping and handling", Range("bz1:bz1000"), 0)) Then Exit Do
If Not IsError(Application.Match("Shipping and handling", Range("bz1:bz1000"), 0)) Then
shippingRow = Application.Match("Shipping and handling", Range("bz1:bz1000"), 0) + 1
shippingCell = Range("bz" & shippingRow).Value
If Left(shippingCell, 2) <> "US" Then
Range("bz" & shippingRow - 1).ClearContents
Else
Range("c" & eachItem).Value = Right(shippingCell, Len(shippingCell) - 2)
End If
End If
Loop
End If
Next
I think you will have to learn DOM automation to do this cleanly. I took a look at the HTML on the ebay pages and it might be a little much for someone who hasn't used DOM automation before. I wasn't planning on writing this but it sounds like you are in a bit of a pinch, so here you go. You can use it to learn from. Just keep in mind that this will work in the short-term but when they change their HTML, it will fail.
Option Explicit
Sub Get_Ebay_Shipping_Charges()
Dim IE As Object, DOM_DOC As Object
Dim URL$, SHIPPING_CHARGES$
Dim SHIPPING_AMOUNT
Dim i&, x&
Dim EL, EL_COLLECTION, CHILD_NODES, TABLE_NODES, TABLE_ROW_NODES, TABLE_DATA_NODES, ITEM_NUMBER_ARRAY
Dim WS As Excel.Worksheet
Dim ITEM_NOT_FOUND As Boolean
''You should change this to the worksheet name you want to use
''ie Set WS = ThisWorkbook.Sheets("Ebay")
Set WS = ThisWorkbook.Sheets(1)
''Create an Internet Explorer Object
Set IE = CreateObject("InternetExplorer.Application")
''Make it visible
IE.Visible = True
''You can replace this with an array that is built from your spreadsheet, this is just for demo purposes
ITEM_NUMBER_ARRAY = Array("290941626676", "130942854921", "400035340501")
''In your code, you can start your loop here to handle the list of items
''This code is a little different for demo purposes
For x = 0 To UBound(ITEM_NUMBER_ARRAY)
''Here is your URL
URL = "http://www.ebay.com/itm/" & ITEM_NUMBER_ARRAY(x) & "?ru=http%3A%2F%2Fwww.ebay.com%2Fsch%2Fi.html%3F_from%3DR40%26_sacat%3D0%26_nkw%3D" & ITEM_NUMBER_ARRAY(x) & "%26_rdc%3D1"
''Navigate to your URL
IE.navigate URL
''This loop will wait until the page is received from the server - the page was hanging for me too so I added a counter to exit after a certain number of loops (this is the i variable)
Do Until IE.readystate = 4 Or i = 50000
i = i + 1
DoEvents
Loop
i = 0
''This sets the DOM document
Set DOM_DOC = IE.document
''First get a collection of table names
Set EL_COLLECTION = DOM_DOC.GetElementsByTagName("table")
If IsEmpty(EL_COLLECTION) Then ITEM_NOT_FOUND = True: GoTo ERR_EXIT
''Then look for the table classname that matches the one we want (in this case "sh-tbl") and set the childnodes to a new collection
For Each EL In EL_COLLECTION
If EL.ClassName = "sh-tbl" Then
Set CHILD_NODES = EL.ChildNodes
Exit For
End If
Next EL
If IsEmpty(CHILD_NODES) Then ITEM_NOT_FOUND = True: GoTo ERR_EXIT
''Next look for the TBODY element in the childnodes collection and set the childnodes of the TBODY element when found
For Each EL In CHILD_NODES
If Not TypeName(EL) = "DispHTMLDOMTextNode" Then
If EL.tagname = "TBODY" Then
Set TABLE_NODES = EL.ChildNodes
Exit For
End If
End If
Next EL
If IsEmpty(TABLE_NODES) Then ITEM_NOT_FOUND = True: GoTo ERR_EXIT
''Find the TR element and set its childnodes to another collection
For Each EL In TABLE_NODES
If Not TypeName(EL) = "DispHTMLDOMTextNode" Then
If EL.tagname = "TR" Then
Set TABLE_ROW_NODES = EL.ChildNodes
Exit For
End If
End If
Next EL
If IsEmpty(TABLE_ROW_NODES) Then ITEM_NOT_FOUND = True: GoTo ERR_EXIT
''Find the first TD element and get it's childnodes
For Each EL In TABLE_ROW_NODES
If Not TypeName(EL) = "DispHTMLDOMTextNode" Then
If EL.tagname = "TD" Then
Set TABLE_DATA_NODES = EL.ChildNodes
Exit For
End If
End If
Next EL
If IsEmpty(TABLE_DATA_NODES) Then ITEM_NOT_FOUND = True: GoTo ERR_EXIT
''The first DIV element holds the shipping information so when it is found, get the innertext of that element
For Each EL In TABLE_DATA_NODES
If Not TypeName(EL) = "DispHTMLDOMTextNode" Then
If EL.tagname = "DIV" Then
SHIPPING_CHARGES = EL.INNERTEXT
Exit For
End If
End If
Next EL
''Make sure a shipping charge was found
If SHIPPING_CHARGES = vbNullString Then MsgBox "No shipping charges found for item " & ITEM_NUMBER_ARRAY(x): GoTo ERR_EXIT
If IsNumeric(Right(SHIPPING_CHARGES, InStr(SHIPPING_CHARGES, Chr(36)))) Then
SHIPPING_AMOUNT = Right(SHIPPING_CHARGES, InStr(SHIPPING_CHARGES, Chr(36)))
Else
SHIPPING_AMOUNT = SHIPPING_CHARGES
End If
''You may have to change this to fit your spreadsheet
WS.Cells(x + 1, 3).Value = SHIPPING_AMOUNT
ERR_EXIT:
If ITEM_NOT_FOUND = True Then MsgBox "No Page Was Found For Item " & ITEM_NUMBER_ARRAY(x): ITEM_NOT_FOUND = False
Next x
IE.Quit
Set IE = Nothing
End Sub
If you are stuck on using your existing code, you can also try deleting the querytables after the query.
Dim QRY_TABLE As QueryTable
For Each QRY_TABLE In ThisWorkbook.Sheets(1).QueryTables
QRY_TABLE.Delete
Next
This method will not delete the querytable values on your spreadsheet but it will kill the querytable connection. If you have too many of these, it could create a crash.
One final suggestion, if your workbook contains a lot of vlookups then this is probably the true culprit. Good Luck!
You can use xmlHTTP object which will download the data easier and wont make the excel stuck.
Sub xmlHttp()
Dim xmlHttp As Object
Set xmlHttp = CreateObject("MSXML2.ServerXMLHTTP.6.0")
Dim ITEM_NUMBER_ARRAY As Variant
ITEM_NUMBER_ARRAY = Array("290941626676", "130942854921", "400035340501")
For x = 0 To UBound(ITEM_NUMBER_ARRAY)
''Here is your URL
URL = "http://www.ebay.com/itm/" & ITEM_NUMBER_ARRAY(x) & "?ru=http%3A%2F%2Fwww.ebay.com%2Fsch%2Fi.html%3F_from%3DR40%26_sacat%3D0%26_nkw%3D" & ITEM_NUMBER_ARRAY(x) & "%26_rdc%3D1"
xmlHttp.Open "GET", URL, False
xmlHttp.setRequestHeader "Content-Type", "text/xml"
xmlHttp.send
Dim html As Object
Set html = CreateObject("htmlfile")
html.body.innerHTML = xmlHttp.ResponseText
Set objShipping = html.getelementbyid("shippingSection").getElementsbytagname("td")(0)
If Not objShipping Is Nothing Then
Set divShip = objShipping.ChildNodes(1)
Debug.Print divShip.innerHTML
Else
Debug.Print "No Data"
End If
Next
End Sub
Immediate Window (Ctrl + G)
US $2.55
No Data
US $6.50

Excel VBA: Looped Web Queries

I have a list of 100,000 URLs that I need to parse via an API call. I've sorted them into a list of 600+ concatenated strings, each containing 200 URLs - ready to be parsed.
I've written the code below to loop the process, places the returned information about the URLs in the last row of column C, one at a time. However, my loop seems to be broken and I don't know why (looking at it too long) but I suspect it's a rookie mistake. After doing the first two concatenated strings (400 URLs, it starts to rewrite the information from around row 200, processing only the first string.
The code is below and any help will be greatly appreciated. Regrettably, I can't share the URL that I'm attempting to parse because it's a propriety system built by my employers and isn't for public use.
Sub APIDataProcess()
Dim lURLsLastRow As Long
Dim lDataSetLastRow As Long
Dim rngURLDataSet As Range
Dim sURLArray As String
Dim lURLArrayCount As Long
Dim rngArrayCell As Range
lURLsLastRow = Cells(Rows.Count, 1).End(xlUp).Row
lDataSetLastRow = Cells(Rows.Count, 3).End(xlUp).Row
Set rngURLDataSet = Range("A1:A" & lDataSetLastRow)
lURLArrayCount = Range("B1").Value ' placeholder for count increments
sURLArray = Range("A" & lsURLArrayCount).Value
For Each rngArrayCell In rngURLDataSet
If lsURLArrayCount <= lURLsLastRow Then
With ActiveSheet.QueryTables.Add(Connection:="URL;http://test.test.org/test.php", Destination:=Range("C" & lDataSetLastRow))
.PostText = "urls=" & sURLArray
.RowNumbers = False
.FillAdjacentFormulas = False
.PreserveFormatting = True
.RefreshOnFileOpen = False
.BackgroundQuery = False
.RefreshStyle = xlOverwriteCells
.SavePassword = False
.SaveData = True
.AdjustColumnWidth = True
.RefreshPeriod = 0
.WebSelectionType = xlEntirePage
.WebFormatting = xlWebFormattingNone
.WebPreFormattedTextToColumns = True
.WebConsecutiveDelimitersAsOne = True
.WebSingleBlockTextImport = False
.WebDisableDateRecognition = False
.WebDisableRedirections = False
.Refresh BackgroundQuery:=False
End With
lURLArrayCount = lURLArrayCount + 1
Range("B1").Value = lURLArrayCount
Application.Wait Now + TimeValue("00:01:00")
Else
Exit Sub
End If
Next rngArrayCell
End Sub
You probably solved your own problem long ago but since the question is still open I will have a go.
I assume the intention is that B1 is initially 1 and is then stepped after each row is processed. This would allow you to stop the macro and carry on from where you had got to on the previous run.
But you do not use B1 or lURLArrayCount like that. The range you examine is always A1 to Amax. You step lURLArrayCount and store it in B1 but its value is not used within the loop.
You set sURLArray outside the loop but use it within.
The loop is For Each rngArrayCell but you never use rngArrayCell.
You do not step lDataSetLastRow after a result has been added.

Resources