Actually, after Chinese, English is generally considered one of the hardest languages to become fluent in. This is not because English has more conjugations or more tenses than most other languages (I agree, we have less than many). It is because English is not a language of rules - it is a language of exceptions to the rules it has.
English has more exceptions that require direct memorization of rules and their exceptions than any other language. That is why it is generally very easy to know who speaks English as their native language once you talk to them for a while - most non-native speakers do not know the minor exceptions (which is fine, and still totally comprehensible - but not considered completely fluent).