C# if statements. if not a or not b

I am trying to add a number to a list but only if the number is not 255 or not 0 id like to avoid a nested if. The code I have to do this is as follows.

if (!(r == 255 || r == 0))
{
    rlist.Add(r);
    listBox2.Items.Add(Math.Floor(r).ToString());
}

However I am still getting 255 and 0 added to the listbox and I can not figure out why. Can anyone point out what i am doing wrong? Thanks in advance for any help.