dimanche 25 avril 2021

if statement not working with scraped web text

I understand this is a fairly common question, but the circumstances of this make it a perplexing one.

I am using beautifulsoup to scrape certain data from a website, using this I am also checking for a "Next page" link on the page I am currently scraping to see if I can scrape another.

next_page_button_finder = soup.find('ul', class_='navnext').text

To check how this comes out, I print it using:

print(next_page_button_finder)

The output is:

Next >>

However, and this is the curious part, when I then try to validate this by creating the simple if statement:

    if next_page_button_finder == "Next >>":
        print("yes")
    else:
        print("no")

"no" is printed.

Any help would be hugely appreciated.

Here is the code you can use to replicate the issue (any link from spareroom.com will work, however, for your ease you can use this link https://www.spareroom.co.uk/flatshare/?search_id=1034984872& ) :

from bs4 import BeautifulSoup
import requests

html_address = input("Paste page the address here:")

html_text = requests.get(html_address).text

soup = BeautifulSoup(html_text, 'lxml')
prices = soup.find_all('strong', class_='listingPrice')

next_page_button_finder = soup.find('ul', class_='navnext').text

print(next_page_button_finder)

if next_page_button_finder == "Next >>":
    print("yes")
else:
    print("no")

Aucun commentaire:

Enregistrer un commentaire