My relationship to speech tags as a reader is quite different: Only rarely do I take note of them at all. If I take note of them, it is usually a phenomenological indication that the writer has failed in a critical task, giving unique voice to the characters. As an illustration, I will point to the two novels I most recently finished reading: Iain Banks' Stonemouth and Helene Wecker's The Golem and the Jinni.
In the first, each character, once they had engaged in one or two substantive speech-acts, could be readily identified whenever they opened their mouths. In that case, there was no need to refer to the speech tags to keep straight who said what in a given scene; they existed instead as indicators of the speech-style of the first-person narrator and main character. Serving that purpose, they often took a form that, in the hands of a lesser writer than Banks, could well have been "annoying"--belated back-announcing of who said what, use of verbs and adverbs to convey elements of physical demeanor that are not readily evident from the quoted speech, etc.
Wecker's first novel, while it succeeds brilliantly as an example of fiction "in which multiple ontologies conflict" (if I may quote myself), has the critical weakness that nearly all characters, regardless of age, gender, mother tongue, ethnicity, occupation, ideology or species, speak in voices that are barely distinguishable one from the other. Thus, in dialogue between the wise old rabbi Meyer Levy and his Socialist Labor Party-member, social worker, 27-year-old apikoyres of a nephew Michael, one must refer continually to the speech tags to keep track of who is saying what. In reality, in Yiddish, a conversation between two such characters would have been marked by pious sprinklings of Hebrew and Aramaic on one side, and Marxian jargon rendered into daytshmerish on the other--and this could have been rendered well in English, just by walking in the footsteps of I.B. Singer. There are exceptions to this, of course, usually in the form of minor characters from whom I found myself wishing to hear more, such as Maryam Faddoul, the yenta of Little Syria. I found myself wishing I could spend more time lingering in her coffeehouse, listening to her gossip about Arbeely's strange new Bedouin apprentice Ahmad.
If speech tags are "annoying," then, it seems to me to be a symptom of a problem that takes longer to diagnose by close reading than it takes to skim a manuscript looking for deviations from the rules: That the characters function as roles rather than as persons with more-or-less complicated histories which have left identifiable marks on how they speak. Well-written dialogue is well-written whether it is framed by Joycean dashes, laconic saids, or a garrulous first-person narrator who talks over the words of the characters. Weak dialogue will call attention to any weaknesses in plot, narration or characterization surrounding it, regardless of how it is presented.