They feared power they didn't have or understand. Eventually took to setting us on fire. Which was fairly useless, as we all know anti-burning charms, for the most part.

[It's almost boring, really, if one is prepared for it.]

Muggles fear us, once they understand what we can do and they cannot. Resentment is often common. Some adjust when it's family -- wizards can be born to the magicless.