So it turns out that there was a fairly comprehensive study about this very same topic published in 2006. Here is the article, for people who have nothing better to do and want to challenge themselves: (Noftle & Shaver, 2006). Note that the authors were testing more complicated hypotheses as well, but that's beyond the scope of the question that was asked.
As a quick and dirty summary, it turns out that people who are high in neuroticism are also much more likely to be anxious, as well as more likely to be avoidant (although not as much). Moreover, it turns out that people who are high in extraversion are especially likely to be LOW in avoidance, as well as low in anxiety (although not as much). So essentially, their conclusion is that extraverted people are less likely to be avoidant (or alternately, that introverted people are more likely to be avoidant).
Of course, these are just correlations, and they are not ridiculously high correlations (although they are significant). Here is the correlation table that they provided in case you want to see for yourself.