Is there a difference between teaching kids from an underprivileged neighborhood than vice versa?

I was wondering, a person who plans to teach can either come across teaching kids/teens from a neighborhood that is poor and underprivileged or kids from a neighborhood that is rich and wealthy. Does anyone know or think there is a difference between teaching either of these types? Are underprivileged kids more rowdy and less well behaved than those who are wealthy? Are they harder to teach, generally of course.

The reason I ask this is because I will more than likely be teaching the first kind of children, and I have heard they are too difficult to teach. I’m not going to let this stop me, but I admit, I am nervous…

So hopefully some one out there can give me some information regarding this.