Harlem over Caroll Gardens and Cobblehill area?

I keep reading on this forum that Harlem, West of Park and below 125 is a great neighborhood. I've visited for the past couple of weekends; walking and driving around and I just don't see it. There are way more boarded up buildings; SRO's; low rent buildings and dirty dilapidated streets in Harlem than in Cobble hill and Caroll Gardens. Caroll Gardens and Cobble hill has more brownstone blocks and just seems much safer to me. They have way more restaurants and amenities than in Harlem. I always see middle class families with strollers walking around in those neghborhoods in Brooklyn. Haven't seen any of that in south harlem? Am I missing something? If it matters, I'm looking for a brownstone/townhouse.