Why is the Or Operator not working?

I am trying to close every open form as long as it's name does not match specific form names (certain forms should remain open), but it is not working like I think it should and I can't figure out why.

I store the forms in a list using a for each loop and Application.OpenForms (the formatting is not working when I try to paste this code in).

I then use another foreach loop to compare the names of the forms in the list to the names I want to keep open. When the loop reaches frmMain the if statement returns true and the form closes which closes the application.

If I remove the OR part of the if statement so the if statement becomes: if(form.Name != frmMain) then it works just fine.

            //Close all user dependant forms that might be open.
            foreach (Form form in forms)
            {
                if(form.Name != "frmMain" || form.Name != "frmDice")
                {
                    form.Close();
                }
            }

Why does the code not work with the OR in the if statement?