How to Loop webpage to find a specific element

This VBA code can be used to loop all the data on a webpage looking for a specific element.  This element can then be actioned.  Such actions as inputting value in text box, clicking button, scraping values, etc.

http://www.vba-market.com/IEAutomationExample.xlsm

Sub scrape_loop()
Dim doc, hcol, text As Variant
Dim ie As SHDocVw.InternetExplorer
Set ie = CreateObject("InternetExplorer.Application")
ie.Visible = True
ie.Navigate ("https://www.google.com")
Do While ie.ReadyState <> READYSTATE_COMPLETE
    DoEvents
    Loop
    While ie.Busy
    DoEvents
    Wend
Set doc = ie.Document
Set hcol = doc.getElementsByTagName("input")
    For Each text In hcol
         If text.ID = "lst-ib" Then
              text.Value = "www.vba-market.com"
        End If
    Next
Application.Wait (Now + #12:00:02 AM#)
Set doc = ie.Document
Set hcol = doc.getElementsByTagName("button")
    For Each text In hcol
         If text.ID = "_fZl" Then
              text.Click
        End If
    Next
End Sub
Set hcol = doc.getElementsByTagName("button")

This should be set to the Tag type your looping such as input, button, td, etc.

The macro loops all the tags of the specified type.  You can then check if the name, id, innerHTML, etc of the element matches what you’re looking for.

If text.ID = "lst-ib" Then

If text.title = "Search" then

The ID, name, title, etc can be found in View Source of the webpage (right click inspect element).  Use the select tool to grab the element you’re looking for.

Then read through the HTML code to find the ID, name, title, innerHTML, etc.

search

Set doc = ie.Document
Set hcol = doc.getElementsByTagName("input")
    For Each text In hcol
         If text.ID = "lst-ib" Then
              text.Value = "www.vba-market.com"
        End If
    Next

Then use this to loop the tags to find the input Search box.  Once it is found the value “www.vba-market.com” is input into the Search box.

*Note:  Microsoft Internet Controls Reference must be enabled.

Interested in learning new skills?