If Then Statement doesn't read input [duplicate]

This question already has an answer here:

• How do I test one variable against multiple values? 16 answers

Just learning how to code, and wanted to make a small program to see what i know.

 n = int(input("Pick a number any Number: "))
if  n > 100:
    print ("No... Not that Number")
else:
    answer = input("Would you like to know your number?")
    if answer == "Y" or "Yes" or "y" or "yes":
        print ("Your number is %s" % (n))
    elif answer == "N" or "No" or "n" or "no" or "NO":
        print ("Oh, well that's a shame then.")
    else:
        print ("Please type Yes or No")

input("Press Enter/Return to Exit")

Everything works except for the second if then statement, which doesn't follow any of the data entered into input. Any reason why it does this?