JS add text to input if it isn't in already

I'm trying to get a button click to add the data of one field to another if it isn't in there already, I think I'm close but, in my example, if my 'else if' parses, a second click will make my 'else' parse also. can't quite figure this one out.

$('#test').click(function(){
    var replied = $('#en_replied');
    var mem_id = $('#enr_member_id').val();

    if (mem_id.indexOf(replied.val())) {
        var replied_ids = replied.val();
    } else if (!replied.val()) {
        var replied_ids = mem_id;
    } else {
        var replied_ids = replied.val() + '|' + mem_id;
    }
    replied.val(replied_ids);
});

I've also set up a js fiddle: http://ift.tt/1P4NWq4

Thanks