Bash if... then... syntax error

I am trying to remove something based on user input using bash/sh, here is my code :

echo "remove ? [Y/n]"
read REMOVE
if [ $REMOVE != "n" ] || [ $REMOVE !="N" ];then
  # ... do something ...
  echo "done"
fi

the error I am getting is something like:

./run.sh: line 8: syntax error near unexpected symbol « then »

./run.sh: line 8: `if [ $REMOVE != "n" ] || [ $REMOVE !="N" ];then'

I tried to add/remove spaces many times, and I still don't understand what happens. I also don't really understand all the differences between [[ statement ]] [statement] or ((statement)). If someone can help...