How can I display a field value in MySQL only if another value in the same field doesn't exist? IF/ELSEIF or CASE?

I'm writing a report to show past due invoices, and I want it to include the billing address of the company. The data has multiple address types as possibilities (Bill To, Business Address, Royalty Invoices, etc.). These are all mapped to an id (address_name_type_id... 1 is "Business Address", 4 is "Bill To).

Every invoice has a Business Address. But I only want that address displayed if the others don't exist. This code correctly shows me the addresses I primarily want (where the ID is 4, 5, or 6). But it loads blank for the ones that only have a "Business Address" (because that isn't 4, 5, or 6).

NOTE: this is only a snippet of the relevant code. The entire report is massive, but I do have it set to only display one line per invoice. In this case, organization_id = 6303 doesn't show up at all, as it only has a Business Address.

    SELECT
    o.name AS 'Organization',
    org_add.address_name_type_id AS "Address Type|display_name:AddressNameType",
    ad.street1 AS "Address 1",
    ad.street2 AS "Address 2",
    ad.city AS "City",
    ad.state_id AS  "State|display_name:State",
    ad.zip_code AS "Zip Code",
    ad.country_id AS "Country|display_name:Country"

    FROM
    organization o
    LEFT JOIN
        (SELECT o2a.organization_id, o2a.address_name, o2a.address_name_type_id, o2a.address_id, o2a.active 
           FROM organization2address o2a 
           WHERE o2a.address_name_type_id IN (4,5,6)) AS org_add ON org_add.organization_id = o.organization_id 
    LEFT JOIN
    address ad ON org_add.address_id = ad.address_id

WHERE o.organization_id IN (6654, 6082, 6303)
AND
org_add.active = 1

Everything I've tried to fix it ends up returning ONLY the business address, since every invoice has one of those.

How can I tell it to ONLY return the Business Address IF there's NOT an address_name_type_id IN (4, 5, 6)? This CASE statement doesn't work - it only returns the address with an address_name_type_id of 1.

SELECT
   o.name AS 'Organization',
     (CASE WHEN o2a.address_name_type_id = 4 THEN 4 
           WHEN o2a.address_name_type_id = 5 THEN 5 
           WHEN o2a.address_name_type_id = 6 THEN 6 
            ELSE 1
             END)  AS "Address Type 2|display_name:AddressNameType",
   ad.street1 AS "Address 1",
   ad.street2 AS "Address 2",
   ad.city AS "City",
   ad.state_id AS  "State|display_name:State",
   ad.zip_code AS "Zip Code",
   ad.country_id AS "Country|display_name:Country"

    FROM
    organization o
    LEFT JOIN
    organization2address o2a ON o2a.organization_id = o.organization_id 
    LEFT JOIN
    address ad ON o2a.address_id = ad.address_id

WHERE o.organization_id IN (6654, 6082, 6303)
AND
o2a.active = 1
group by o.name

Is it even possible to do this? Am I overthinking it?