It's the pub!  It's never been referred to as a number in my experience.  There's no obvious number displayed on it either, whereas 58 and 60 have numbers displayed just to the north.
My guess is that many buildings here predate house numbers. House numbers came to Earswick to the north only around 65 years ago; many buildings here or at least their plots significantly predate that.