At SpeakerConf 2009, one of the speaker's wives asked me: Why is it that most speakers are confident, and some are simply arrogant. The question was also in the context of "software is nerdy, how can you speakers be so cocky". I pondered for a minute and replied: When no one knows what's "correct", people with confidence generally win the discussion.
Imagine the early days of medicine. Three different doctors give you three different surgery options. There's not enough experience in the industry to show which is the correct choice. Who do you trust? Probably the doctor who managed to be the most confident without being overly aggressive or cocky.
As I've said before, we're still figuring this stuff out. I constantly try to improve my processes for delivering software. I share those processes via speaking and blogging. However, I'm definitely aware that I could be doing it entirely wrong.
In general, I'm weary of developers who speak in absolutes. Most solutions I've seen are contextual, and even in context they tend to be fragile. We simply don't have the heuristics to determine what true best practices (for our field) are.
When pondering the original question I remembered when I wrote about Despised Software Luminaries. At the time I blamed passion; however, I'm guessing confidence and arrogance probably also weigh heavily on the source of animosity.
There's a direct correlation between being a luminary and your compensation package. Therefore, luminaries are enticed to gain as much market share as possible. Your luminary status is determined by the speed at which your ideas are adopted. And, without absolute proof of correctness, the speed at which your ideas are adopted can largely based on your confidence. I'm sure some luminaries see this situation and walk the line between confidence and arrogance.
Of course, not all luminaries are in it for compensation. I truly doubt Martin Fowler does what he does for money. But, the people who are looking to take market share from Martin may be motivated by a compensation package. Therefore, it's pretty hard to escape the the effects of confidence and arrogance.
The confidence and arrogance discussion is also interesting if you've ever met a luminary who you found to be completely incompetent. Perhaps they truly don't know what they were talking about, and they've just been confident enough to fool the majority of people they've met so far.