How to exit loop when input is nothing

I'm trying to work out the average of numbers that the user will input. If the user inputs nothing (as in, no value at all) I want to then calculate the average of all numbers that have been input by the user upto that point. Summing those inputs and finding the average is working well, but I'm getting value errors when trying to break the loop when the user inputs nothing. For the if statement I've tried

if number == '' First attempt that didn't work, also tried if number == int("")

if len(number) == 0 This only works for strings

if Value Error throws up same error

Full code below

sum = 0

while True :
    number = int(input('Please enter the number: '))
    sum += number
    if number == '' :
        break

print(sum//number)

Error I'm getting is

number = int(input('Please enter the number: ')) ValueError: invalid literal for int() with base 10:>

Any help much appreciated!