R - ifelse function with date and time

this is my first question here and I'm pretty new with R and programming so probably it will be an obvious question. I have a data.frame with column "DateTime", column "Night_Starts" and column "Night_Ends", all of them in the format "YYYY-MM-DD hh:mm:ss". All of them are factors. Looks like this:

DateTime            Night_starts        Night_ends      
2018-01-31 01:52:32 2018-01-31 22:22:19 2018-01-31 04:30:08      
2018-02-02 03:24:28 2018-02-02 22:19:14 2018-02-02 04:33:47       
2018-02-03 08:29:52 2018-02-03 22:17:40 2018-02-03 04:35:37  
2018-03-16 21:45:23 2018-03-16 21:04:09 2018-03-16 05:38:54 

I would like to create a fourth column named "Day_or_night" where I want to have "NIGHT" if the "DateTime" is between "NightStarts" and "NightEnds" and "DAY" if it is not. If I code

mydata$Day_or_night<-ifelse(mydata$DateTime >= mydata$Night_Starts & mydata$DateTime <= mydata$Night_Ends,"NIGHT","DAY")

obviously the message I get is

Warning messages:
1: In Ops.factor(mydata$DateTime, mydata$Night_Starts) :
  ‘>=’ not meaningful for factors
2: In Ops.factor(mydata$DateTime, mydata$Night_Ends) :
  ‘<=’ not meaningful for factors

I tried to convert the factors in numeric or in characters but it's not working, because the result looks like this:

DateTime            Night_starts        Night_ends          Day_or_night
2018-01-31 01:52:32 2018-01-31 22:22:19 2018-01-31 04:30:08 DAY      
2018-02-02 03:24:28 2018-02-02 22:19:14 2018-02-02 04:33:47 DAY       
2018-02-03 08:29:52 2018-02-03 22:17:40 2018-02-03 04:35:37 DAY  
2018-03-16 21:45:23 2018-03-16 21:04:09 2018-03-16 05:38:54 DAY 

When clearly the third row should be "NIGHT". I know it's a dumb question and there's probably an easy answer but right now I cannot find it.