Excel VBA IF statement being skipped

I have two If statements that follow each other down below. The first checks if the previous step is marked as complete with an "X", and exits the macro as appropriate. The second If statement checks if that step has already been run. This displays the "Proceed?" question, but choosing "no" does not end the macro.

If Range("D4").Value = "" Then
Dim response As VbMsgBoxResult
response = MsgBox("Previous step is not marked as complete. Proceed?", vbYesNo)
If response = vbNo Then
    Exit Sub
End If
End If

If Range("D4").Value = "X" Then
Dim response2 As VbMsgBoxResult
response2 = MsgBox("Current step is already marked as complete, proceed?", vbYesNo)
If response = vbNo Then
    Exit Sub
End If
End If

Set Range1 = Sheets("Latest Open QNs Report Data").ListObjects("OpenQns").DataBodyRange.Offset(0, 1)
Set Range1 = Range1.Resize(Range1.Rows.Count, Range1.Columns.Count - 1)
Range1.ClearContents

Sheets("Instructions").Select
Range("D5", "D5").Value = "X"