Ruby if/elseif coding error

I'm writing a simple and small program in Ruby to output the ordinal of a number based on user input. Here is what I wrote:

puts "Enter a number"
number = gets.chomp.to_i

conversion = number % 10

if number == 11 || number == 12 || number == 13
  puts "That is #{number}th"

end

if conversion == 1
  puts "That is #{number}st"
elsif conversion == 2
  puts "That is #{number}nd"
elsif conversion == 3
  puts "That is #{number}rd"
else 
  puts "That is #{number}th"

end  

As my code shows, I wanted to take the "number" from the user input, cut off the last number, and use that single digit number to determine whether "number" should be a (fir)st, (seco)nd, (thi)rd, or (four)th type of ordinal. The "conversion" value should convert that for me. However, it can be seen that the numbers 11, 12, and 13 are exceptions. My program works fine except that when I input one of these three values, I get two outcomes, not one, in the terminal such as:

That is 11st
That is 11th

My intention is that "11th" should be displayed, not "11st". Even though I tried to write an exception into my code, the "conversion" still gets executed. I'm not sure what error I made, but is there a way I can isolate the first if statement, so that my conversion does not include 11, 12, and 13?