Else Statement doesn't work in unity?

Alright so here is my code. This should be really simple but it doesn't want to work with me for some reason. Raycast sends a ray from the mouse, if it hits an object with a tag, it assigns a number to a variable. If it doesn't hit an object, then it sets the variable as -99. For some reason mine is hanging on:

A. I don't hit the objects, it outputs -99 the first time but after that it hangs on getting assigned 4.

B.I hit the objects and it will work just fine. After I click OFF the objects it hangs on the variable from the object I just hit previously.

RaycastHit hit; 
            Ray ray = camera.ScreenPointToRay (Input.mousePosition);
            if (Physics.Raycast (ray, out hit, 10000.0f)) {
                if (hit.collider.tag == "Box") {
                    Hitnum = 1;
                } else if (hit.collider.tag == "Sphere") {
                    Hitnum = 2;
                } else if (hit.collider.tag == "Pyramid") {
                    Hitnum = 3;
                } else if (hit.collider.tag == "Trapezoid") {
                    Hitnum = 4;
                } else {
                    Hitnum = -99;
                }
            }
            Debug.Log (Hitnum);
        }

Thanks in advance. This is driving me nuts because it should be simple.