mardi 4 juin 2019

general function to handle errors api

I have created a generalised function using requests ratelimit and backoff libraries. The objective of this function is to implement the following logic:

  • if the status of the returned object is not 200 nor 404 => raise error (so that if I have connection errors backoff can try up to a certain amount of times)

  • if the status of the returned object is 404 => return error dictionary

  • else return r.json

here's my function

import requests
from requests import ConnectionError
from ratelimit import limits, sleep_and_retry
from backoff import on_exception, expo


@sleep_and_retry  # if we exceed the ratelimit imposed by @limits forces sleep until we can start again.
@on_exception(expo, ConnectionError, max_tries=10)
@limits(calls=500, period=FIVE_MINUTES)
def call_api(url, api_key):
    r = requests.get(url, auth=(api_key, ""))

    if not (r.status_code is 200 or r.status_code is 404):
        r.raise_for_status()

    elif r.status_code is 404:
        return dict({"error": "not found"})

    else:
        return r.json()

What actually happens is that if I have a 404 r.raise_for_status() gets activated - which hints me that there is something wrong with my logic.

I made an abstraction of this logic and indeed:

def logic_try(value):

    if not (value is 200 or value is 404):
        print("stopped with value {}".format(value))

    elif value is 404:
        return dict({"error":value})

    else:
        return dict({"correct": value})

# calls

logic_try(200)
§ {'correct': 200}

logic_try(404)
§ stopped with value 404 # should return {"error":value}

logic_try(400):
§ stopped with value 400 

I would like the function to first check that the r.status i not 200 nor 404 and raise the error status so the decorator can call again. Then check if r.status is 404, in which case return an error dictionary which I store in a pg table, finally, all other case should simply return r.json() as i assume the r.status is a 200.

Aucun commentaire:

Enregistrer un commentaire