Im currently trying to create my own notification whenever I hit a exception x amount of times, it should notify to me through a function that will in the future be sent to either discord, SMS or whatever I will choose.
However that is not my problem, my problem is that I want to be able to gather more information such as which response status, own custom reason status, headers, cookies etc etc...
I have currently done something like this:
import time
import requests
from lib.exceptions import (
FailedRequests
)
def send_notification(msg):
print(f"Sending notification to discord later on! {msg}")
class Scraper:
def __init__(self):
self.failedCnt = 0
def simpleException(self, exception, msg):
if self.failedCnt > 2:
send_notification(msg)
raise exception(msg)
def setup_scraper(self, site_url):
r = requests.session()
while True:
response = r.post(site_url, timeout=5)
if response.ok:
print("Yay!")
exit()
if response.status_code in {429, 403, 405}:
print(f"Status -> {response.status_code}")
self.simpleException(
FailedRequests,
f"Too many {response.status_code} response"
)
self.failedCnt += 1
# What I would like to print to my discord/sms/whatever if I hit etc: 15x of retries
# reason=f"{response.status_code} response requests",
# url=str(site_url),
# headers=str(session.headers),
# cookies=str(session.cookies),
# response=str(response.text)
time.sleep(3)
continue
if __name__ == "__main__":
Scraper().setup_scraper("https://www.google.se/")
and I wonder if its possible to do a function which retries x amount of times and if it fails after etc 15 times, it should print the reason, url, headers etc etc to my def send_notification(msg): function and then raise the error.
Aucun commentaire:
Enregistrer un commentaire