Can I apply consistent sample numbers using group_by function in conjunction?

I am trying to select a single value using sample() and apply it to my data where x = 0 using an ifelse statement, but apply this to multiple rows of data relating to a project using group_by() but together it is not producing what I would like. Here is the code I'm trying to use:

df %>% group_by(project) %>% mutate(VarB = ifelse(VarA == 0, sample(1:5, nrow(df), replace = TRUE), 1))

What I am getting is sample numbers between 1-5 that vary across all rows of project data when VarA == 0, instead of being consistent across all rows as I'm hoping to achieve by using the group_by() command.

Can anyone guide me in how to amend the code to achieve what I want?