VBA - Excel - Automation Error Unspecified Error - excel

Question

So I ran into a slight stumbling block and hopefully here someone can help me. In short, I need to visit a string of webpages (the list of the names on each page are already input, that code works fine). As my code visits each page, I need to pull back information. Unfortunately, there's a problem - it can't even make it through the "A" list before I get "Automation Error Unspecified Error" and it's never at the same spot.
I've tried the "normal" steps to fix this. I've installed the VB 6 Controls and I've unregistered and re-registered mscomctl.ocx, and including On Error Resume Next (which doesn't do anything).
It usually reaches over 100 cases before it dies (randomly as I said earlier). And AFTER the error pops up, when I try to re-run it (with or without changes) and it errors on the first one. If I restart my computer it will let me try again (for whatever reason) but it still doesn't finish.
Is the code too complex and I need to reduce it? I can probably find a way to make it only run for each letter at a time (run all A's, then do B's, etc) but I still can't even get it to complete the letter A.
I noticed in another thread someone had suggested instead of using IE to swap to xmlhttp - is that a fix for this? Is the problem that this script is too long? What exactly am I doing wrong here?
Sub Lookup()
Range("AI1").Value = "Unique ID"
Range("AJ1").Value = "Name"
Range("AK1").Value = "Birth Year"
Range("AL1").Value = "Title"
Range("AM1").Value = "State"
Range("AN1").Value = "Position"
Range("AO1").Value = "Country"
Range("AP1").Value = "Appointed"
Range("AQ1").Value = "Credentials"
Range("AR1").Value = "Terminations"
Dim i As Integer
For i = 1 To 26
If i = 24 Then
Range("X:X").End(xlUp).Select
ActiveCell.Value = ""
Else
Dim ic As String
ic = LCase(ConvertToLetter(i))
Range(ic & "5000").End(xlUp).Select
Dim J As Integer
J = ActiveCell.Row
Dim k As Integer
For k = 2 To J
Range(ic & k).Select
Dim Lookup As String
Lookup = ActiveCell.Value
Dim IE As Variant
Set IE = CreateObject("InternetExplorer.Application")
IE.Visible = False
IE.navigate "http://history.state.gov/departmenthistory/people/" & Lookup
Do
DoEvents
Loop Until IE.readyState = READYSTATE_COMPLETE
Dim Doc As HTMLDocument
Set Doc = IE.document
Dim Italics As Integer
Italics = 0
Dim EachA As Integer
For EachA = 64 To 100
Dim Position As String
Position = Doc.getElementsByTagName("a")(EachA).innerText
If Position = "Home" Then
Exit For
Else
Dim NameBY As String
NameBY = Doc.getElementsByTagName("h2")(1).innerText
Dim TitleST As String
TitleST = Doc.getElementsByTagName("p")(1).innerText
Range("AJ" & "90000").End(xlUp).Offset(1, 0).Select
ActiveCell.Value = NameBY
TitleState = Split(TitleST, vbLf)
ActiveCell.Offset(0, 2).Value = TitleState(0)
On Error GoTo 1037
ActiveCell.Offset(0, 3).Value = TitleState(1)
On Error GoTo 1037
1037
ActiveCell.Offset(0, 4).Select
ActiveCell.Value = Position
Dim EachLi As Integer
EachLi = EachA - 1
If Doc.getElementsByTagName("li").Item(EachLi + Italics).innerHTML Like "<em>*" Then
Italics = Italics + 1
Else
End If
Dim JobList As String
JobList = Doc.getElementsByTagName("li")(EachLi + Italics).innerText
Dim Job() As String
Job() = Split(JobList, vbLf)
Dim JCount As Integer
For JCount = LBound(Job) To UBound(Job)
ActiveCell.Offset(0, 1).Select
ActiveCell.Value = Job(JCount)
Next JCount
End If
Next EachA
Next k
End If
Next i
End Sub

Answer 1

One thing I notice is that you're continually creating new IE objects inside the loop, and you're never destroying them or setting to Nothing. It's pointless, expensive, and possibly a source of error to be creating 100+ instances of IE.
I think it will probably help to create a single instance of IE initially, and then use that same object inside the loop to navigate the desired URLs.
So instead of this:
Dim IE As Variant
Set IE = CreateObject("InternetExplorer.Application")
Do this:
Dim IE as Object
If IE Is Nothing Then Set IE = CreateObject("InternetExplorer.Application")