vendredi 22 septembre 2017

Python - Error detecting/recording selenium element using for loop and if statement

I'm running into an error and I'm unsure why. I've also been stuck on this problem for a while now, and after a few days, I think I might know what the main issue is. Essentially, when I run a for loop to iterate over the possible clicking combinations in selenium, my function that detects a selenium element that indicates that the current iteration yields no data, which is conveniently labeled as No data fails to locate the element. I've tried using if no instead of if argument but it still doesn't work. Hopefully, someone can help, or at least point me in the right direction.

import os
from selenium import webdriver
from selenium.common.exceptions import NoSuchElementException
import zipfile
import pandas as pd
import time
import itertools

# Use itertools to create all possible combinations
to_loop = itertools.product(Suitability_Productivity_List, Crop_List, Water_Supply_List, Input_Level_List)

# Convert itertools.object into pandas dataframe named test
test = list(to_loop)
test = pd.DataFrame(data = test, columns=["Suitability_and_Productivity", "Crop", "Water_Supply", "Input_Level"])

# Create empty column for for loop
No_data = []

# Resave to_loop object as a generator for for loop
to_loop = itertools.product(Suitability_Productivity_List, Crop_List, Water_Supply_List, Input_Level_List)

for i in to_loop:

    link = 'http://ift.tt/2xpkSaq'

    chrome_options = webdriver.ChromeOptions()
    prefs = {'download.default_directory': 'C:/.../Download'}
    chrome_options.add_experimental_option('prefs', prefs)
    driver = webdriver.Chrome(
        executable_path='C:/.../Chrome-Driver/chromedriver.exe',
        chrome_options=chrome_options)

    driver.get(link)

    # Enter username and password
    driver.find_element_by_name('_username').send_keys(username)
    driver.find_element_by_name('_password').send_keys(password)
    driver.find_element_by_id('buttonSubmit__login').click()

    # Click on first link
    driver.find_element_by_name('_targetfieldmain=main_py&eventtype').click()

    # Click on second link
    driver.find_element_by_name('&fieldmain=main_py&idPS_eventtype').click()

    # Click on Suitability and productivity list
    driver.find_element_by_css_selector('input[value="{}"]'.format(i[0])).click()

    # Click on crop link
    driver.find_element_by_css_selector("input.linksubmit[value=\"▸ Crop\"]").click()
    driver.find_element_by_css_selector('input[value="{}"]'.format(i[1])).click()

    # Click on Water Supply Link
    driver.find_element_by_css_selector("input.linksubmit[value=\"▸ Water Supply\"]").click()
    driver.find_element_by_css_selector('input[value="{}"]'.format(i[2])).click()

    # Click on Input Level Link
    driver.find_element_by_css_selector("input.linksubmit[value=\"▸ Input Level\"]").click()

    driver.find_element_by_css_selector('input[value="{}"]'.format(i[3])).click()

    # Check if the webpage contains the `No data` element
    data_check = driver.find_element_by_xpath("//span[contains(text(),'No data')]")

    # If statement to record whether current iteration has data or not
    if not True:
        try:
            driver.find_element_by_xpath("//span[contains(text(),'No data')]")
            print("Yes data")
            No_data.append('0') # Store value into No_data column
        except NoSuchElementException:
            print("Yes")
    else:
        print("No data")
        No_data.append('1') # Store value into No_data column


# Append results to dataframe
test["No_data"] = No_data

This is my error:

Traceback (most recent call last):
  File "C:\Users\...\Anaconda3\lib\site-packages\IPython\core\interactiveshell.py", line 2881, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-2-067ec590ef4f>", line 120, in <module>
    data_check = driver.find_element_by_xpath("//span[contains(text(),'No data')]")
  File "C:\Users\...\Anaconda3\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 354, in find_element_by_xpath
    return self.find_element(by=By.XPATH, value=xpath)
  File "C:\Users\...\Anaconda3\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 832, in find_element
    'value': value})['value']
  File "C:\Users\...\Anaconda3\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 297, in execute
    self.error_handler.check_response(response)
  File "C:\Users\...\Anaconda3\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 194, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"xpath","selector":"//span[contains(text(),'No data')]"}
  (Session info: chrome=60.0.3112.113)
  (Driver info: chromedriver=2.31.488763 (092de99f48a300323ecf8c2a4e2e7cab51de5ba8),platform=Windows NT 10.0.15063 x86_64)

Thanks for reading.

Aucun commentaire:

Enregistrer un commentaire