Why pay a brokers fee if broker does nothing?

Negotiate. There are enough NO FEE rentals out there to make a compelling argument that the Broker will take 25-50% of his initial ask, or 0% (i.e. the owner pays). There's no basis for most RE prices, other than "that's what's been done in the past." You won't get won't you don't ask for though.

Why did you hire this broker? Rudeness and explaining the value does not release you from the legal obligation you created when you hired him. Welcome to New York. Don't like it? Leave. Or visit one of the many no-fee buildings, from Stuy Town to the brownstone owner who rents out the floor above his. Thanks to StreetEasy, they're easy to find.