[saymaListserv] "munging"

Sharon Annis sharonannis at comcast.net
Tue Nov 25 21:26:16 JEST 2003


Dear Friends I don't know about any of the rest of you but I am now totally confused.  I try to reply to all when appropriate and to the individual when that is what I want to do.  So far this has worked and i don't want to learn any thing else.  shaon
  ----- Original Message ----- 
  From: Mary Calhoun 
  To: sayma 
  Sent: Tuesday, November 25, 2003 6:12 AM
  Subject: [saymaListserv] "munging"


  Dear Friends,

  A look at the "munging" issue --

  ^o^
  \_/
  Mary Calhoun, List Administrator
  AdminAsst at sayma.org
  276-628-5852
  POB 2191, Abingdon VA 24212-2191

  -------------------------------


  sayma mailing list administration
  General Options Section

  . . . . 

  Configuration Categories

  . . . .

  [General Options] 

  . . . .

  Reply-To: header munging

  Should any existing Reply-To: header found in the original message be stripped? If so, this will be done regardless of whether an explicit Reply-To: header is added by Mailman or not
        No Yes 

  Where are replies to list messages directed? Poster is strongly recommended for most mailing lists.
        Poster This list Explicit address 

   (Details for reply_goes_to_list)  

  [links to...] This option controls what Mailman does to the Reply-To: header in messages flowing through this mailing list. When set to Poster, no Reply-To: header is added by Mailman, although if one is present in the original message, it is not stripped. Setting this value to either This list or Explicit address causes Mailman to insert a specific Reply-To: header in all messages, overriding the header in the original message if necessary (Explicit address inserts the value of reply_to_address). 
  There are many reasons not to introduce or override the Reply-To: header. One is that some posters depend on their own Reply-To: settings to convey their valid return address. Another is that modifying Reply-To: makes it much more difficult to send private replies. See `Reply-To' Munging Considered Harmful for a general discussion of this issue. See Reply-To Munging Considered Useful for a dissenting opinion. 

  "Reply-To" Munging Considered Harmful
  An Earnest Plea to Mailing List Administrators

------------------------------------------------------------------------------

  An email message requires some amount of processing when it is redistributed to a mailing list. At the very least, the envelope must be rewritten to redirect bounces directly to the list administrator. While the message is being processed, the list administrator might take advantage of the opportunity to munge 

  [link....  ]  The page cannot be displayed 
        The page you are looking for is currently unavailable. The Web site might be experiencing technical difficulties, or you may need to adjust your browser settings. 

------------------------------------------------------------------------

        Please try the following:

          a.. Click the  Refresh button, or try again later.
       


  some of the message headers. 

  Some forms of header munging are helpful, such as special loop-detection headers. Others are questionable. Most are ill-advised or dangerous. Many list administrators want to add a Reply-To header that points back to the list. This transformation also is one of the most ill-advised. 

  Some administrators claim that Reply-To munging makes it easier for users to respond to the entire list, and helps encourage list traffic. These benefits are fallacious. Moreover, Reply-To can have harmful -- even dangerous -- effects. If you think Reply-To munging is a good idea, I hope I can change your mind. 

  The Principle of Minimal Munging
  Email processing is pretty tricky. Read through RFC-822, the Standard for the Format of ARPA Internet Text Messages, sometime. It is 47 pages of dense, dry detail. A lot of engineering and consideration went into this work. Even still, RFC-822 leaves many corner conditions and specialized circumstances poorly specified. RFC-1123, the commonly-called Internet Host Requirements document, adds a couple dozen more pages, and remedies some of the defects. Then there is MIME, X.400 mapping, and a handful of other standards and conventions -- some documented and some folklore. Email handling is surprisingly complicated, and even an innocuous-sounding change might have grave, unintended consequences. 


  The "Principle of Minimal Munging" is a good rule that will keep you out of trouble. It says you should not make any changes to an email header unless you know precisely what you want to do, why you want to do it, and what it will affect. Unless you can articulate a clear reason for munging and understand the full consequences of the action, you should not do it. 

  The "Principle of Minimal Munging" will help you avoid the sorts of problems we are about to discuss. This principle is a rule designed to be broken, but you can avoid some significant heartache by thinking hard and long before you do so. 

  It Adds Nothing
  Reply-To munging does not benefit the user with a reasonable mailer. People want to munge Reply-To headers to make "reply back to the list" easy. But it already is easy. Reasonable mail programs have two separate "reply" commands: one that replies directly to the author of a message, and another that replies to the author plus all of the list recipients. Even the lowly Berkeley Mail command has had this for about a decade. 

  Any reasonable, modern mailer provides this feature. I prefer the Elm mailer. It has separate "r)eply" and "g)roup-reply" commands. If I want to reply to the author of a message, I strike the "r" key. If I want to send a reply to the entire list, I hit "g" instead. Piece 'o cake. 

  I mention Elm here (and a lot later on) simply because that's the mailer I use everyday. This sort of support is not unique to Elm Any reasonable mailer provides it. The Pine mailer, for instance, asks directly, "Reply to all recipients?" when you use the "r" command. It doesn't get much easier than that! 

  Whichever mailer you choose, please read the fine manual that comes with it. Unless you are stuck with some decrepit mail system, I bet you'll find it has a similar feature. If so, you easily can choose to direct your responses either to the original author or the entire list. Mauling the mail headers doesn't make it any easier. 

  It Makes Things Break
  If you use a reasonable mailer, Reply-To munging does not provide any new functionality. It, in fact, decreases functionality. Reply-To munging destroys the "reply-to-author" capability. Munging makes this command act effectively the same as the "reply-to-group" function. We haven't added anything new, we've only taken away. Reply-To munging is not merely benign, it is harmful. It renders a useful mail capability inoperative. 

  Freedom of Choice
  Some administrators justify Reply-To munging by saying, "All responses should go directly to the list anyway." This is arrogant. You should allow me to decide exactly how I wish to respond to a message. If I feel a public response is justified, I'll hit the "g" key and tell Elm to do a group-reply. If I believe a private response is more appropriate, I'll use "r" to send one. Please allow me the freedom to decide how to handle a message. 

  Can't Find My Way Back Home
  It may be impossible to reply to the author of a message once the Reply-To header is munged. The Reply-To header was not invented on a whim. It is there for the sender of a mail message to use. If you stomp on this header, you can lose important information. 

  There are good reasons why the sender might insert a Reply-To header. The sender might not be the original author of the message (the name that appears in the From header). If responses should return to the sender and not the original author, then the sender will insert a Reply-To header. Or, maybe the sender added a Reply-To because he or she cannot receive email at the account from which the message was sent. There are many good reasons to place a Reply-To header into a mailing list message. 

  If the Reply-To is munged by the mailing list, the value provided by the original sender is lost. Reply-To munging can make it impossible to reach the sender of a message. 

  Coddling the Brain-Dead, Penalizing the Conscientious
  There are, unfortunately, poorly implemented mail programs that lack separate reply-to-author and reply-to-group functions. A user saddled with such a brain-dead mailer can benefit from Reply-To munging. It makes it easier for him or her to send responses directly to the list. 

  This change, however, penalizes the conscientious person that uses a reasonable mailer. This is a poor trade-off. As Internet list administrators, we should encourage people to run reasonable software. If a few people need to type in a full reply address so that everybody else can use all the features of their mailer, I say, "Fine!" We should not penalize the conscientious to coddle those who run brain-dead software. 

  Principle of Least Work
  Compare and contrast: the work required for me (or any other Elm user) to reply on lists that do and don't employ Reply-To munging. 

		Case One:		Case Two:
Action		Without Munging		With Munging
=============	=====================	=====================

Reply to	Hit the "g"		Probably hit the "r"
everybody.	key.			key, but maybe the "g"
					key if there were other
					recipients of the message.

Reply just	Hit the "r"		Look at the original
to author.	key.			message header, write
					down the sender's
					email address, hit the
					"r" key, call up the
					header editing menu,
					erase the current To:
					value, and type in the
					sender's full email
					address.  And pray the
					correct address wasn't
					wiped out when the Reply-To
					was munged.
Again, your preferred mailer probably implements this feature in a different fashion. Nonetheless, it should be easy. I'll take box number one, Monte. 

  Principle of Least Surprise
  When I hit the "r" key in Elm, it sends a response to the author of a message. When you munge the Reply-To header you change this action so that it does something entirely different from what I expect. This creates specialized behavior for your mailing list, which increases the potential for surprise. I'm not schooled in the science of human factors, but I suspect surprise is not an element of a robust user interface. 

  Private messages frequently are broadcast across lists that do Reply-To munging. That's an empirical fact. It's what happens when you violate the principle of least surprise. 

  Principle of Least Damage
  Consider the damage when things go awry. If you do not munge the Reply-To header and a list subscriber accidentally sends a response via private email instead of to the list, he or she has to follow up with a message that says, "Ooops! I meant to send that to the list. Could you please forward a copy for me." That's a hassle, and it happens from time to time. 

  What happens, however, when a person mistakenly broadcasts a private message to the entire list? If the message is a complaint about the personal hygiene of sender's boss, or the sex life of his or her roommate, a simple "Ooops!" won't cut it. About all you can do is send a followup with lots of retroactive smileys (weak). Or say your cat was dancing on the keyboard (better). Or start reading the classifieds for a new job/roommate/set of teeth (most likely). 

  Reply-To munging encourages catastrophic failure modes. Sure, you don't need Reply-To munging to create this sort of damage. A simple slip of the fingers will suffice. When, however, you violate the "Principle of Least Surprise" you invite this sort of disaster. A responsible list administrator will avoid creating avenues that lead to such extreme damage. 

  And in the End...
  If you are not convinced yet, then allow me one final plea. I contribute to the Elm mailer development team. I get to see a lot of the wants and requests from the user community. Guess what feature more and more people are asking for? A third reply command -- one that ignores any existing Reply-To header! Want to guess why people are asking for it? If you think you are doing your subscribers a service by munging Reply-To headers, you are kidding yourself. You are making your subscribers miserable. 

  Some list administrators, even after reading all this, seem to say, "Oh, it's not that bad. Besides, my subscribers like it!" If they do, it's probably because they haven't bothered to learn to use the "reply-to-group" feature of their mailer. Instead of going through all the trouble of making your list gateway scribble on email headers, how about making an effort to educate your subscribers? 

  Summary
  Many people want to munge Reply-To headers. They believe it makes reply-to-list easier, and it encourages more list traffic. It really does neither, and is a very poor idea. Reply-To munging suffers from the following problems: 

    a.. It violates the principle of minimal munging. 
    b.. It provides no benefit to the user of a reasonable mailer. 
    c.. It limits a subscriber's freedom to choose how he or she will direct a response. 
    d.. It actually reduces functionality for the user of a reasonable mailer. 
    e.. It removes important information, which can make it impossible to get back to the message sender. 
    f.. It penalizes the person with a reasonable mailer in order to coddle those running brain-dead software. 
    g.. It violates the principle of least work because complicates the procedure for replying to messages. 
    h.. It violates the principle of least surprise because it changes the way a mailer works. 
    i.. It violates the principle of least damage, and it encourages a failure mode that can be extremely embarrassing -- or worse. 
    j.. Your subscribers don't want you to do it. Or, at least the ones who have bothered to read the docs for their mailer don't want you to do it. 
  Addendum
  In case you are wondering, yes, I once thought Reply-To munging was a nifty idea. I got better though. 

  When I started running email lists, I munged 'em all. One day I accidentally sent a private, personal reply out over one of my own damn lists. If the list owner can't remember how to use the list properly, no way will the subscribers be able to sort it out. I stopped munging the very next day. 

  On the whole, it has worked out quite well. Yes, on occasion somebody mistakenly responds directly to the author of a message when they wanted to reply to the group. Most folks, however, seem to catch on pretty fast to how it works, and seem to appreciate the flexibility. Moreover, private responses mistakenly sent to the entire list have become an almost unheard-of event. 



------------------------------------------------------------------------------

  Chip Rosenthal
  <chip at unicom.com>

$Id: reply-to-harmful.html,v 1.20 2002/11/15 03:46:04 chip Exp $----------
    Reply-To Munging Considered Useful
    An Earnest Plea to Mailing List Administrators
    Last revised: 3 January 2000

----------------------------------------------------------------------------

    An email message requires some amount of processing when it is redistributed to a mailing list. At the very least, the envelope must be rewritten to redirect bounces directly to the list administrator. 

    While the message is being processed, the list administrator might take advantage of the opportunity to munge some of the message headers. Many list administrators want to add a Reply-To header that points back to the list. This transformation is also one of the most useful. 

    Some administrators claim that Reply-To munging can have harmful -- even dangerous -- effects. I assert the opposite, that not adding a Reply-To header has even more harmful effects. If you think Reply-To munging is a bad idea, I hope I can change your mind. 

    RFC 822 and "Text Message Teleconferencing"
    The first thing to consider is that RFC 822, the document which defines the standards and usages for email, specifically mentions this usage in section 4.4.3: 

        A somewhat different use may be of some help to "text message
        teleconferencing" groups equipped with automatic distribution
        services: include the address of that service in the "Reply-To"
        field of all messages submitted to the teleconference; then 
        participants can "reply" to conference submissions to guarantee
        the correct distribution of any submission of their own.
Aside from this official sanction, there are a number of reasons for munging the Reply-To header. The arguments which follow are my own. They may not be comprehensive, but I think they are compelling. 

    The Principle of Minimal Bandwidth
    The ``Principle of Minimal Bandwidth'' is a good rule that will keep you out of trouble. It says that you should make any changes which will reduce the amount of email traffic on the Internet. The ``Principle of Minimal Bandwidth'' will help you avoid the sorts of problems we are about to discuss. This principle is a rule designed to be broken, but you can avoid some significant heartache by thinking hard and long before you do so. 

    Reply-To Munging Adds Something
    Reply-To gives the respondent an option which would not otherwise exist: namely the ability to reply only to the list. Despite the fact that many (though not all) email clients have the ability to "reply to sender" or "reply to all recipients", many list subscribers want to reply only to the list, which is not the result of selecting either of these options. So, to ensure that the reply goes to the list, they select "reply to all recipients", which generally results in the sending of at least two email messages, one to the list, and one to the original sender. 

    This is frequently quite annoying to the original sender, who now receives two copies of the reply. Furthermore, in many cases the original sender has added additional recipients. Not only does "reply to all recipients" send the reply to each of these additional recipients (who are frequently also members of the list), it also propagates this list of recipients onto the reply to the list. 

    The effects of this snowball, as each additional person replies to the messages using "reply to all recipients", they become the sender, and thus get added to the list of recipients with the next reply. Thus the list of recipients grows and grows. Frequently, as the subject matter changes, members of the list find themselves receiving multiple copies of messages which have strayed from the topic in which they were originally interested, even after they have unsubscribed from the list. 

    Many people have pointed out that it is relatively easy to implement a procmail filter to remove duplicates. This attitude merely reveals a Unix-centric and US-centric viewpoint. Many users of inferior operating systems do not have a tool powerful enough to ensure the removal of duplicate messages. Furthermore, in many European countries, connect time is charged by the minute. Even with procmail, the duplicates have to be downloaded before they can be filtered, resulting in unnecessary additional expenses for some of our European list mates. 

    This last fact reveals that the issue is really related to bandwidth. By applying the ``Principle of Minimal Bandwidth'', we conclude that it is necessary to add a Reply-To header that points back to the list. 

    It Doesn't Break Reasonable Mailers
    If you use a reasonable mailer, Reply-To munging does provide new functionality, namely the ability to reply only to the list. Furthermore, it does not decrease functionality. In Pine, for example, when there is a Reply-To header, Pine will ask, ``Use "Reply-To:" address instead of "From:" address?'', easily allowing one to reply only to the original author. In KMail, it is even easier. One merely right-clicks on the hyperlinked From address. 

    If your mailer doesn't have this option, you should request it from its development team. Any mailer, whose development team refuses this simple request due to some ideological position, cannot be said to be reasonable. 

    Freedom of Choice
    Since Reply-To munging adds additional functionality, it actually increases freedom of choice. Not only can you now reply only to the list, you still have the option to reply to the original author, or to all recipients, easily and conveniently. 

    Some Mailers are Broken
    There are, unfortunately, some poorly implemented mail programs that lack separate "reply-to-author" and "reply-to-group" functions. A user saddled with such a mailer can benefit from Reply-To munging. It makes it easier for him or her to send responses directly to the list. 

    Furthermore, this change does not penalize the conscientious person that uses a reasonable mailer. Reasonable mailers give one the ability to reply to the From address. Therefore, it would be unkind to further penalize those with poorly implemented mail programs, since munging the Reply-To header causes no harm to those with reasonable mailers. 

    Principle of Least Total Work
    For discussion type lists, I would estimate that ninety percent of the time, people want to reply to the list. Without munging, they either have to break the ``Principle of Least Bandwidth'', or type in the list address. Many people, being lazy, will choose the former, sending unnecessary copies of emails to people who will either have to delete them, or take the time to set up a filter (if they are lucky enough to be running an operating system which facilitates this). 

    On the other hand, about ten percent of the time, replying to the sender might be more appropriate. Even if the respondent has an unreasonable mailer (a decision for which they are probably responsible), the worst case scenario is that they have to type in an address ten percent of the time. Of course, if they took the time to add this recipient to their address book, they could reduce the amount of typing to a minimum. 

    So, which produces least total work: typing in the list address ninety percent of the time (plus possibly taking the time to set up a filter), or typing in an individual's address ten percent of the time? 

    I'll take munged Reply-To headers every time, thanks. 

    People are Responsible for Their Own Mistakes
    Some administrators claim that munging Reply-To headers is harmful because it surprises people, and can cause damage when things go awry. They assert that administrators should prevent the possibility of a private message being mistakenly broadcast to the entire list. 

    This is simply not the responsibility of the administrator. People are responsible for their own mistakes. If someone is sending a private email which is derogatory, or otherwise embarrassing were it to be made public, they should probably be sending it directly, rather than as a reply to a public message. They should also pause and think about whether they should be sending it at all. This pause should be quite sufficient for a conscientious person using a reasonable mailer to catch any mistake that they might be about to make. 

    In any case, it is an entirely trivial matter for the list administrator to provide an obvious clue in the subject line of every message that the message was received from a mailing list. If your Mailing List Manager doesn't provide an option to prepend "[listname]" to the subject, then switch to one that does ( e.g. GNU MailMan or Majordomo ). 

    And in the End...
    If you are not convinced yet, then allow me one final plea. Most mailing lists are intended to facilitate discussion on a given topic. If this is indeed the primary purpose of your list, then you really should add a Reply-To header which directs replies to the list. This helps to ensure that the entire thread of the conversation is available to all who might be interested. 

    I can't count the number of times I have searched the archives of a list for a solution to a problem, only to find the question asked, but no solution. Yet, when I subsequently post the question to the list, the long-time members insist that it has already been discussed, and that I should search the archives. If I'm lucky, a newer member forwards to me the private reply which answered the question. 

    Thus, munging the Reply-To header benefits those lists which are intended for serious discussion. If your list is intended primarily for announcements or other one-way mailings, you may safely ignore these arguments. 

    It's What People Want
    I have been and am subscribed to both munging and non-munging mailing lists. On the non-munging lists, there are regular requests to change the list so that Reply-To replies to the list. On the munging lists which already do this, there are hardly any requests for change. 

    Summary
    Many people want to munge Reply-To headers. They believe it makes reply-to-list easier, and it encourages more list traffic. It really does both of these things, and is a very good idea. To reiterate: 

      a.. It adheres to the principle of minimal bandwidth. 
      b.. It provides additional functionality to the user. 
      c.. It increases a subscriber's freedom to choose how to direct a response. 
      d.. It does not reduce functionality for the user of a reasonable mailer. 
      e.. It aids and assists the user with a deficient mailer. 
      f.. It adheres to the principle of least total work. 
      g.. It helps to ensure that questions are answered on the list. 
      h.. Your subscribers want you to do it. 

    Addendum
    There are, of course, a few details that need to be addressed to make Reply-To munging more pleasant and productive for everyone. One potentially serious problem with Reply-To munging is the possibility of mail loops. It should be possible for the list server to detect and prevent this. If anyone has any patches to implement this feature, I would be happy to provide a link to the patch on your ftp server, or to make it available on my own ftp server. 

    Also, patches are needed for any mailers that do not implement the ability to reply to the From address. Please send links or patches to me at sdhill at metasystema.net. Thanks. 

    Patches
    A patch for Emacs rmail is available at: http://www.metasystema.net/pub/patches/emacs/rmail-query-reply-to.el. 

    Dissenting Opinion
    I originally wrote this essay as a response to Chip Rosenthal's Reply-To Munging Considered Harmful. 


    Simon Hill
    sdhill at metasystema.net
  HTTP/1.1 200 OK Date: Tue, 25 Nov 2003 10:44:54 GMT 



------------------------------------------------------------------------------


  _______________________________________________
  Southern Appalachian Yearly Meeting and Association mailing list
  sayma at kitenet.net
  http://kitenet.net/cgi-bin/mailman/listinfo/sayma
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://kitenet.net/pipermail/sayma/attachments/20031125/a46d1254/attachment.html>


More information about the sayma mailing list