Why does this loop never accepts input first time round?

I'm creating a caesar Cipher and it works completely but there is one small bug. I have created a code snippet as to not post the whole algorithm. My problem is that when this is ran and you enter 1 or 2 it will not except, however on the second run it will. I am utterly baffled as to why, if you help I would be exceptionally greatful.

Choice = input('Encrypt or Decrypt? \n\nEnter 1 for Encrytion \nEnter 2 for Decrytion')

while Choice != '1' or Choice != '2':
    print('You must enter 1 or 2')
    Choice = input('Encrypt or Decrypt? \n\nEnter 1 for Encrytion \nEnter 2 for Decrytion')

    if Choice == '1':
        print('bananas')
        break

    if Choice == '2':
        print ('cake')
        break