If you're just after the address, it shouldn't matter what other tags you use, you could just have `addr:housenumber` + `addr:street` without any building=* or landuse=* and it should still be picked up as an address...