Hi all I am working with selenium in python, I have the following code
a=26 #specifying the starting point
b=73 #specifying the end point
x=a/25
print(x)
q=int(x)-1
p=b-a+1
if x<=1:
school_urls = [link.get_attribute('href') for link in driver.find_elements_by_xpath("//*[@id='rankings']/tbody/tr/th/a")]
print(school_urls)
for i in range(a-1,len(school_urls)):
school_links.append(school_urls[i])
while len(school_links)<p:
next_elements=driver.find_elements_by_xpath("//*[@id='layout_1']/section/ol/li[8]/a")
print(next_elements)
next_page=next_elements[0]
next_link=next_page.get_attribute('href')
if next_link in next_links:
break
next_links.append(next_link)
driver.get(next_link)
school_urls = [link.get_attribute('href') for link in driver.find_elements_by_xpath("//*[@id='rankings']/tbody/tr/th/a")]
school_links=school_links+school_urls
this code chunk occurs after having navigated to a page via the code below
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
import pandas as pd
import mysql.connector as mysql
import re
options = webdriver.ChromeOptions()
options.add_experimental_option('excludeSwitches', ['enable-logging'])
driver = webdriver.Chrome(options=options)
driver.implicitly_wait(5)
driver.get("https://www.maxpreps.com/")
sign_in=driver.find_element_by_link_text('SIGN IN')
sign_in.click()
email=driver.find_element_by_id("ctl00_Navigation_Content_txtEmail")
email.send_keys('XXXXX')
password=driver.find_element_by_id("ctl00_Navigation_Content_txtPassword")
password.send_keys('XXXXX')
sign_in_submit=driver.find_element_by_id("ctl00_Navigation_Content_Button1")
sign_in_submit.click()
school_links=[]
next_links=[]
the problem is that the elements
school_urls = [link.get_attribute('href') for link in driver.find_elements_by_xpath("//*[@id='rankings']/tbody/tr/th/a")]
next_elements=driver.find_elements_by_xpath("//*[@id='layout_1']/section/ol/li[8]/a")
are returned as empty lists when inside the if statement sometimes and sometimes the code works perfectly fine, but if I take them out of the if statement they work perfectly fine every time I run it. This is a very weird error to have, since the web elements are always present and an if statement should not have an impact on this. Any ideas?
Aucun commentaire:
Enregistrer un commentaire