Code goes through both if and else statement

I want to include some design changes on the product pages on my Wordpress site. I want to add specific classes when the purchase button is out of the viewport. For example here https://www.evacosmetics.eu/product/bio-dushgel-za-suha-kozha/.

At first, I thought of just doing it with a scroll event listener, but that was total hell. After that, I tried doing it with a setInterval, but that just had the same results.

var buttonFormToChange = document.querySelector("form.cart")
var buttonToChange = document.querySelector("button")
window.addEventListener("scroll", function() {
    if (parseFloat(document.querySelector("button").getBoundingClientRect().y) <
        parseFloat(-48)) {
        buttonFormToChange.classList.add('addToForm');
    } else {
        buttonFormToChange.classList.remove('addToForm');
    }
});

var buttonFormToChange = document.querySelector("form.cart")
var buttonToChange = document.querySelector("form > button")
setInterval(function() {
    if (buttonToChange.getBoundingClientRect().y <= -48) {
        buttonFormToChange.classList.add('addToForm');
    } else {
        buttonFormToChange.classList.remove('addToForm');
    }
}, 100)

The issue is that when I scroll down below the purchase button both the if and the else statements get fired and the classes get added and removed every millisecond. I did try several ways to do this including an example from here with a variable that checks how many times you've scrolled, however even if I don't do anything(don't scroll) the classes get added and removed.