As a Spaniard, the main problem most of us have with English is phonetics. Most of the sounds sound too similar, or we can't even perceive them. It requires a lot of practice to fix that, and most of people never quite get it right.
It doesn't help at all that English has nearly zero redundancy in most common words, so if you don't get a sound, you are screwed.
Of course, spelling only makes it worse.
Fortunately, most of UX is not voice based. Unfortunately, most of voice based UX are English only.