While loop with if/else statement in Python

So I am still in the process of learning Python and I am having difficultly with while loops. I have a sample of code below that includes while loop and if and else statements. What I want it to do is print 'Less than 2' and 'Greater than 4' which it does, but it keeps running. It does not print it just once each which is what I would want it to do. Any help would be greatly appreciated!

counter = 1
while (counter < 5):
    count = counter
    if count < 2:
        counter = counter + 1
    else:
        print('Less than 2')
    if count > 4:
        counter = counter + 1
    else:
        print('Greater than 4')
    counter = counter + 1