In some cases, your zip+4 is uniquely your address, too. My townhouse was a new development and after complaining for over a year that I wasn't able to sign up for Informed Delivery, I was assigned a new unused +4.
That said, most people don't use the +4 when getting directions or the like, it's just used for postal service.
Even then, to my understanding the USPS has for some time now not relied on zip codes at all. They have a really good address database, they match on that, and then stamp the mail with a routing barcode at the origin post office. The zip code is extra, mail flows just fine if you leave it off.
That said, most people don't use the +4 when getting directions or the like, it's just used for postal service.