How to properly ignore NA when comparing two dates?

I have two date variables

ID Date1       Date2      
1  2019-09-10  2019-09-09 
1  2019-12-11  2019-12-01
1  NA          2019-12-08
2  2019-09-11  2019-09-25
2  2019-11-05  2019-11-15
2  2019-12-05  NA
3  2019-09-19  2019-09-11
3  2019-10-22  2019-10-17

I want to compare if Date 1 is after Date 2 by running below codes

df %>% 
group_by(id) %>% 
mutate(Result = ifelse(!is.na(Date1) < !is.na(Date2), "Yes", 
"No")) 

What I want is:

ID Date1       Date2       Result
1  2019-09-10  2019-09-09  No
1  2019-12-11  2019-12-01  No
1  NA          2019-12-08  NA
2  2019-09-11  2019-09-25  Yes
2  2019-11-05  2019-11-15  Yes
2  2019-12-05  NA          NA
3  2019-09-19  2019-09-11  No
3  2019-10-22  2019-10-17  No

If I run this, I get "Yes" for the ones that contain NA.. I don't want to remove NAs because I still need this missing information later on. I am not sure why !is.na() does not work in this case. Any help would be appreciated!

• I also tried difftime() and same thing happened