As a white kid growing up in Chicago you get confronted with race issues from a pretty early age. Early on your nickname on the playgrounds is pretty much guaranteed to be either white boy or wedo (which is just white boy in Spanish). As a white kid with a smart mouth sooner or later you end up calling other kids black boy or tamale boy and probably end up getting into a few scruffs because of it. Eventually, kids get a little older and start developing friendships and a sense of humor.
By the time I was in eighth in grade or so I had a pretty diverse group of friends and all we would really do was play sports and sit on our parent's porches talking s***. If you've ever had a group of male friends you realize that when the s*** talking starts there isn't really much that is out of bounds to use as fodder and race was no exception. I was a little uncomfortable at first, because I was white, but eventually I learned that I could pretty much say the most racist possible things I could think of and as long as it was funny no one really made a big deal about it. Sure feelings would get hurt from time to time, but those people would just get made fun of more. Just sort of how it works with young men.
Because of my somewhat unique experiences as a white male with race it's somehow gotten to the point that people know I'm not racist because I say so many racist things. Now I'm not saying you should suddenly start going up to black people you don't know and give them KFC coupons because that will probably get you punched in the mouth. But, if you've got a black friend that wouldn't get offended for you making that same bad joke chances are you aren't racist.
A lot of it comes down to comfort levels. If you're comfortable around blacks and Latinos then chances are blacks and Latinos will be comfortable around you. Black people like jokes about black people more than anyone if they're funny and without malice. We're the Barack Obama and Dave Chappelle generation, not the George Wallace generation. There is a lot less racial tension out there and it isn't as taboo as you think it is to talk and even joke about race. If we ever do make it to the point that we live in a post-racial world it won't be because we don't talk about racism. It will be because we think racism is just one big joke.
Image taken from here: http://funny.funnyoldplanet.com/strange/ant-racism/