Using if_else, I can't return the column used as the conditional if the conditional is false

Like the title says, but basically, I've created a flag variable of different job descriptions in a dataset. I would like to change the each flag==true into the same row position from another variable. I've tried ifelse, if_else; all I've been able to get is a list of the changed values for the true conditional and NA's. Here is reproducible example below using the diamonds.

mydata <- diamonds[1:10,c(3,4)] 
mydata$position <- c('flag','cathy')
mydata
mydata$new.vary <- mydata %>% if_else(color=='E',position,color)