How to make my code asking question second time?

I'm trying to create simple game. Try to guess a number, if you guess wrong it spouse to ask you second time to guess number second time which is bigger or lesser than that you already guess. It depend of the random generated number.

The problem that i have is when i guess wrong answer it goes straight to the last else statement. Where is my error, why this happen and how to fix it?

let correctGuess = false;
let guess = prompt('Can you gess the number between 1 and 6?');
let randomNumber = Math.floor(Math.random() * 6) + 1;

if (parseInt(guess) === randomNumber) {
  correctGuess = true;
} else if (parseInt(guess) < randomNumber) {
  let guesMore = prompt('Try again. The number im thinking is more than ' + guess);
  if (parseInt(guesMore) === randomNumber) {
    correctGuess = true;
  }
} else if (parseInt(guess) > randomNumber) {
  let guesLess = parseInt('Try again. The number im thinking is less than ' + guess);
  if (parseInt(guesLess) === randomNumber) {
    correctGuess = true;
  }
}

if (correctGuess) {
  document.write('Congrats mate!');
} else {
  document.write('You wrong! Number was ' + randomNumber);
}