If-statement in RecyclerView ViewHolder causes item duplication

How do I implement if-statements with a custom viewholder without causing that view to appear multiple times in other items (duplication) as I scroll?

I have a RecyclerView with a custom viewholder that works as expected. However, I have a tiny image icon within each list item that should only appear if one of the list item's textView is filled out. When I try to implement an if-statement in the custom viewholder, it causes that view to duplicate when I scroll the RecyclerView.

And by the way, If I use

@Override
public int getItemViewType(int position) {
     return position;
}

It prevents the duplication, but it also causes an animation issue with "swipe to dismiss" functionality. It's no longer smooth but appears to blink or glitch a little bit. Below is my ViewHolder:

private class CustomViewHolder extends RecyclerView.ViewHolder {
        ImageView candidateMainImage;
        ImageView careerIcon;
        TextView candidateCareerText;
        TextView candidateBioText;

        CustomViewHolder(View itemView) {
            super(itemView);
            candidateMainImage = itemView.findViewById(R.id.imageview_swipe_profile_container);
            careerIcon = itemView.findViewById(R.id.imageview_swipe_career_icon);
            candidateCareerText = itemView.findViewById(R.id.textview_swipe_career_display);
            candidateBioText = itemView.findViewById(R.id.textview_swipe_bio_display);
            mContext = itemView.getContext();
        }

        void bind(SwipeCandidate candidate) { 

            StorageReference candidateImageReference = candidate.getCandidateImageReference(); 

            GlideApp.with(candidateMainImage)
                    .load(candidateImageReference)
                    .into(candidateMainImage);

            String textCareer = candidate.getCandidateCareer();
            candidateCareerText.setText(textCareer);
            candidateCareerText.setVisibility(View.VISIBLE);

            String bio = candidate.getCandidateBio();
            candidateBioText.setText(bio);
            candidateBioText.setVisibility(View.VISIBLE);

            if (!TextUtils.isEmpty(candidate.getCandidateCareer())) {
                careerIcon.setVisibility(View.VISIBLE); // This gets duplicated into unwanted items
            }
        }
    }