I landed on building=outbuilding. According to the wiki, an outbuilding is "a less important building near to and on the same piece of land as a larger building".

With the current values for the building tag I feel this is as specific as you can get without getting too specific in an inaccurate way.