jeudi 29 octobre 2020

Selenium web driver statement not working within an if statement, but works fine outside of if statement, Python Selenium

Hi all I am working with selenium in python, I have the following code

a=26 #specifying the starting point 
b=73 #specifying the end point
x=a/25
print(x)
q=int(x)-1
p=b-a+1


if x<=1:
    school_urls = [link.get_attribute('href') for link in driver.find_elements_by_xpath("//*[@id='rankings']/tbody/tr/th/a")]
    print(school_urls)
    for i in range(a-1,len(school_urls)):
        school_links.append(school_urls[i])
    while len(school_links)<p:
        next_elements=driver.find_elements_by_xpath("//*[@id='layout_1']/section/ol/li[8]/a")
        print(next_elements)
        next_page=next_elements[0]
        next_link=next_page.get_attribute('href')
        if next_link in next_links:
            break
        next_links.append(next_link)
        driver.get(next_link)
        school_urls = [link.get_attribute('href') for link in driver.find_elements_by_xpath("//*[@id='rankings']/tbody/tr/th/a")]
        school_links=school_links+school_urls


this code chunk occurs after having navigated to a page via the code below

from selenium import webdriver
from selenium.webdriver.common.keys import Keys
import pandas as pd
import mysql.connector as mysql
import re 

options = webdriver.ChromeOptions()
options.add_experimental_option('excludeSwitches', ['enable-logging'])
driver = webdriver.Chrome(options=options)
driver.implicitly_wait(5)
driver.get("https://www.maxpreps.com/")
sign_in=driver.find_element_by_link_text('SIGN IN')
sign_in.click()
email=driver.find_element_by_id("ctl00_Navigation_Content_txtEmail")
email.send_keys('XXXXX')
password=driver.find_element_by_id("ctl00_Navigation_Content_txtPassword")
password.send_keys('XXXXX')
sign_in_submit=driver.find_element_by_id("ctl00_Navigation_Content_Button1")
sign_in_submit.click()
school_links=[]
next_links=[]

the problem is that the elements

school_urls = [link.get_attribute('href') for link in driver.find_elements_by_xpath("//*[@id='rankings']/tbody/tr/th/a")]

next_elements=driver.find_elements_by_xpath("//*[@id='layout_1']/section/ol/li[8]/a")


are returned as empty lists when inside the if statement sometimes and sometimes the code works perfectly fine, but if I take them out of the if statement they work perfectly fine every time I run it. This is a very weird error to have, since the web elements are always present and an if statement should not have an impact on this. Any ideas?

Aucun commentaire:

Enregistrer un commentaire