mardi 6 avril 2021

How to raise exception with data after x retries

Im currently trying to create my own notification whenever I hit a exception x amount of times, it should notify to me through a function that will in the future be sent to either discord, SMS or whatever I will choose.

However that is not my problem, my problem is that I want to be able to gather more information such as which response status, own custom reason status, headers, cookies etc etc...

I have currently done something like this:

import time

import requests

from lib.exceptions import (
    FailedRequests
)


def send_notification(msg):
    print(f"Sending notification to discord later on! {msg}")


class Scraper:

    def __init__(self):
        self.failedCnt = 0

    def simpleException(self, exception, msg):
        if self.failedCnt > 2:
            send_notification(msg)
            raise exception(msg)

    def setup_scraper(self, site_url):

        r = requests.session()

        while True:

            response = r.post(site_url, timeout=5)

            if response.ok:
                print("Yay!")
                exit()

            if response.status_code in {429, 403, 405}:
                print(f"Status -> {response.status_code}")

                self.simpleException(
                    FailedRequests,
                    f"Too many {response.status_code} response"
                )

                self.failedCnt += 1

                # What I would like to print to my discord/sms/whatever if I hit etc: 15x of retries
                # reason=f"{response.status_code} response requests",
                # url=str(site_url),
                # headers=str(session.headers),
                # cookies=str(session.cookies),
                # response=str(response.text)

                time.sleep(3)
                continue


if __name__ == "__main__":
    Scraper().setup_scraper("https://www.google.se/")

and I wonder if its possible to do a function which retries x amount of times and if it fails after etc 15 times, it should print the reason, url, headers etc etc to my def send_notification(msg): function and then raise the error.

Aucun commentaire:

Enregistrer un commentaire