Hypothetical to muse: would extraterrestrials be villains if they colonized the earth but did not annihilate man?

Say, in the future (it will never happen, but roll with it) extraterrestrials landed on Earth. They see the infrastructure of humans, etc. but their intelligence next to ours is the way we see termites; they have no desire to try and communicate of see us as greatly intelligent at all. The areas they colonize they wipe out any humans or animals in the way, similar to how we move into the hills and get rid of any coyotes, wild boar, bears, etc. that eat our plants, nibble or shrubs, or root our lawns, even though they were there first. The areas they have not chosen to use they do not bother mankind or even pay us any mind. Would humans see them as villains? Would humans resign themselves to make do with the land the extraterrestrials have not bothered, would man do all he can to drive the aliens off the planet? Would we think we were in some way intellectually related as to try to make communication with them even though they show no interest?