Else and if Statements not working C

Well, I'm making a code and I noticed in Debug mode the else-if statements are not working as they should. It is a code that should encrypt a phrase by changing the word with another n times behind it in the alphabet. The number is determined by the user, which is stored in pattern. I tried to firstly check if the word subtracted by the pattern wouldn't be a character behind the alphabet, and if not, the subtraction would occur normally. However, it only enters in the if statement. I would like to know if I'm missing a basic logic concept or if my compiler has a problem.

    for(i = 0; i < size; i++)
    {
        if(phrase[i] - pattern < '65')
        {
            temp = 64 - (phrase[i] - pattern);
            phrase[i] = 'Z' - temp;
        }
        else
        {
            phrase[i] - pattern;
        }
    }