Form submit - IF there is a class, OR option doesn't exist, submit form - but won't submit

I have a form, and when a user selects a clothing size, the form is submitted, but if they don't select any size, an error message appears. This part works perfectly, however there are times when no size options exists (for example, when buying a wallet). When there is a size option, an li element with a class of "size-attribute" is there, and within this li is another ul -> li with the class es-value, and when the user selects an es-value, it is given a class of selected. It looks something like this:

<li class="size-attribute>
 <ul>
  <li class="es-value"></li>
  <li class="es-value selected"></li>
  <li class="es-value"></li>
 </ul>
</li>

However, when there is No size to select (like for a wallet), the list item with the class size-attribute doesn't exist. I am trying to write my code so that when an es-value with a class of selected exists, or NO size-attribute exists at all, the form will submit. However, with the code I have written, when I click add to bag, nothing happens. Here is my code:

.on('click', '.modalAddToBagButton', function(e) {
            e.preventDefault();
            $form = $(this).closest("#dialog-addToBag").find('form');
            if( $( ".es-value.selected" ).length || !$("li .size-attribute")) {
                $form.submit();
            } else {
                $("#errormessage").show();
                $("#error-border").addClass("error-border");
            }

Can anyone figure out what I am doing wrong, and why the form won't submit?