Excel VBA running all If statements even though all conditions aren't being met

My sheet has a table with many columns, but only the first two columns are input fields and the remaining have formulas based on the values in the first two columns. Instead of manually entering the information into the first few columns, I've set up two input fields outside of the table that I would like, once a button with an assigned macro has been clicked, to have transferred into the table.

The VBA code I wrote has several If-Then statements. The first statement returns a msgbox if the first input field (B3) is blank. The second statement returns a msgbox if the second input field (B4) is blank. The third If-Then statement will transfer the info from B3 and B4 into the next available table row if B3 and B4 are not blank.

If I enter information into B3 and B4, the code works fine. The problem comes when B3 and B4 are blank - I receive the appropriate msgbox, but the blank information still gets added as a new row to the table. If I click my button when there is no info in B3 and B4, it will keep giving me the msgbox while also adding rows to my table. What am I doing wrong?

Sub AddToPortfolio()

Dim portfolio As Worksheet
Dim table_list_object As ListObject
Dim table_object_row As ListRow

Set portfolio = Sheets("Sheet1")
Set table_list_object = portfolio.ListObjects(1)
Set table_object_row = table_list_object.ListRows.Add

  If portfolio.Range("B3").Value = Empty Then
        MsgBox "Please Enter CUSIP ID"
     portfolio.Range("B3").Select
    Exit Sub
  End If

  If portfolio.Range("B4").Value = Empty Then
        MsgBox "Please Enter Quantity"
     portfolio.Range("B4").Select
  End If

  If portfolio.Range("B3").Value <> "" And portfolio.Range("B4").Value <> "" Then
     table_object_row.Range(1, 1).Value = portfolio.Range("B3").Value
     table_object_row.Range(1, 2).Value = portfolio.Range("B4").Value
  End If


End Sub