Do I Have to Specialize Templates If Their Offending Code Is in an if(false)

Given the hierarchy:

struct base {};
struct a : public base {};
struct b : public base {};

I want to fill vector<base*> vecBase and vector<a*> aVec with this function:

template <typename T>
void foo(T* bar) {
    if (is_base_of_v<decltype(baseVec)::value_type, T>) baseVec.push_back(static_cast<decltype(baseVec)::value_type>(bar));
    if (is_base_of_v<decltype(aVec)::value_type, T>) baseVec.push_back(static_cast<decltype(aVec)::value_type>(bar));
}        

The problem here is that even though the static_cast will never be performed unless it's legal; calls like these fail to compile:

int myInt;
b myB;

foo(&myInt);
foo(&myB);

I know that I can specialize foo. Is that what I have to do here, or is there a way to tip the compiler off to the fact that the offending static_casts will never happen?