replace values with NA across multiple columns if a condition is met in R

I'm trying to replace values across values with NA across multiple columns if a condition is met.

Here's a sample dataset:

library(tidyverse)
sample <- tibble(id = 1:6,
                 team_score = 5:10,
                 cent_dept_test_agg = c(1, 2, 3, 4, 5, 6),
                 cent_dept_blue_agg = c(15:20),
                 num_in_dept = c(1, 1, 2, 5, 100, 6))

I want the columns that contain cent_dept_.*_agg to be NA when num_in_dept is 1, so it looks like this:

library(tidyverse)
solution <- tibble(id = 1:6,
                   team_score = 5:10,
                   cent_dept_test_agg = c(NA, NA, 3, 4, 5, 6),
                   cent_dept_blue_agg = c(NA, NA, 17:20),
                   num_in_dept = c(1, 1, 2, 5, 100, 6))

I've tried using replace_with_na_at (from the nanier package) and na_if (from the dplyr package), but I can't figure it out. I know my selection criteria is correct (dplyr::matches("cent_dept_.*_agg"), but I can't figure out the solution.

In my actual dataset, I have many columns that start with cent_dept and end with agg, so it's very important that the selection users that matches component.

Thank you for your help!