Announcement

Symphony's issue tracker has been moved to Github.

Issues are displayed here for reference only and cannot be created or edited.

Browse

Closed#154: Email header encoding is wrong

Please see the Related discussion

It looks like the General::encodeHeader function performs base-64-encoding. According to RFC2045, section 6.7, the encoding should be quoted-printable.

As far as I remember, we once had a special function in class.general.php for this, which has been removed:

Method: encodeHeader
Description: Encodes parts of an email header if necessary, according to RFC2047;
             does not need the IMAP module installed (like the imap_8bit function);
Added by:    Michael Eichelsdoerfer
Source:      http://www.php.net/manual/en/function.imap-8bit.php (see comments);

***/
function encodeHeader($input, $charset = 'ISO-8859-1') {

   preg_match_all('/(s?w*[x80-xFF]+w*s?)/', $input, $matches);

   foreach ($matches[1] as $value) {
       $replacement = preg_replace('/([x20x80-xFF])/e', '"=" . strtoupper(dechex(ord("1")))', $value);
       $input = str_replace($value, '=?' . $charset . '?Q?' . $replacement . '?=', $input);
   }

   return wordwrap($input,75,"nt",true);
}

This function works much better, since it does the right thing. (See related discussion.)

It seems that even this function has a problem with wordwrapping, which may break the subject.

I will have to dig deeper. SwiftMailer is getting this right! So we should try and be inspired by it…

Just curious, what does the word wrapping solve? I managed to fix the garbage characters in the subject simply by removing the word wrapping from the original function.

From the SwiftMailer documentation:

Swift Mailer defaults to using 78 characters per line in a message. This is done for historical reasons and so that the message can be easily viewed in plain-text terminals.

And:

You should never set a maximum length longer than 1000 characters according to RFC 2822. Doing so could have unspecified side-effects such as truncating parts of your message when it is transported between SMTP servers.

(http://swiftmailer.org/docs/message-line-length)

So it is necessary to wrap (at 1000 characters per line, at least). So you can as well go the “traditional” way, wrapping at 75 characters or so. If you look at the output of email clients like Apple Mail, you will see that they still use “traditional” wordwrapping.

Yes, that’s all good for the body, but wordwrap doesn’t need to be applied to the headers. On Line 253 of class.general.php you can see that wordwrap is still applied to the mail body.

231d14d15eaaed248521a96e45d0b9e13ce14654 removes wordwrap from the encodeHeader function.

I thought it to be necessary, since SwiftMailer actually does it (without breaking encoded parts, of course).

Have you seen the related discussion? Obviously base64 encoding is not the right thing here…

(Solution: See the function I posted above.)

Ah, yes, I remember now. Yeah I’ve noticed in a few other email classes that the base64 isn’t applied unless it is necessary, looks like your solution should do the trick.

If you implement and commit it, I will be happy to perform some test cases as mentioned in the forum discussion.

changes are in. I won’t close this issue off until we’re all sure it’s working as intended. Thanks for going to the effort michael-e.

Alistair, unfortunately this function is still far from perfect. I spent many hours trying to figure out a really working solution, and here is what I finally developed - it’s dead-simple, but I am rather sure that this is working fine:

/***

Method: encodeHeader
Description: Encodes (parts of) an email header if necessary, according to RFC2047;
Added by: Michael Eichelsdoerfer

***/
public static function encodeHeader($input, $charset='ISO-8859-1')
{
    $m = preg_match_all('/(w*[x80-xFF]+w*)/', $input, $matches);
    if($m)
    {
        mb_internal_encoding($charset);
        $input = mb_encode_mimeheader($input, $charset, 'Q');
    }
    return $input;
}

I have posted some explanations in the forum thread.

The only problem is that:

mbstring is a non-default extension. This means it is not enabled by default. You must explicitly enable the module with the configure option

So, for some people, that function will cause fatal errors.

I will take another look at this issue.

I am afraid that I have to give up on this, since it is taking too much time. I have worked another 12 hours on this without success. There are many issues to solve here (e.g. different character sets for the input, regular expressions for line breaks etc.), and I am unable to get this right.

I may look into this much later, but for the moment (i.e. for the coming weeks) I won’t find the time. So I am afraid that we only have the following options:

  1. We implement the mbstring function which I proposed - this will help everybody who actually has mbstring in PHP. Everybody else will be “lost” (like it is the case at the moment – so nothing’s getting worse).
  2. We integrate a mail framework like SwiftMailer into the core. This framework has the ability to encode headers properly, but I think it’s even bigger than Symphony.
  3. Somebody else tries to implement all this logic. In this case I will be glad to help with testing, but I won’t find the time to do the coding.

Sorry, but I will get lost here if I don’t give up now.

P.S.: None of the classes or functions you will find in the PHP manual’s comments actually works as advertised. Looks like all those guys didn’t know how to set up test cases…

Totally understandable. Thanks a lot of the work you’ve put in thus far.

Thanks for your understanding.

The first (old) function might be a practicable fallback (if mbstring is absent) – if it is used without wordwrapping. In this case the output will not conform to the RFC specs, but according to my tests it should work in many email clients (like Apple Mail and Outlook Express).

Like this:

preg_match_all('/(s?w*[x80-xFF]+w*s?)/', $input, $matches);
foreach ($matches[1] as $value)
{
    $replacement = preg_replace('/([x20x80-xFF])/e', '"=" . strtoupper(dechex(ord("1")))', $value);
    $input = str_replace($value, '=?' . $charset . '?Q?' . $replacement . '?=', $input);
}
return($input);

So combining the two functions is the best I can do – it should work for 99 percent of the cases (and be much better than what we have now):

/***

Method: encodeHeader
Description: Encodes (parts of) an email header if necessary, according to RFC2047 if mbstring is available;
Added by: Michael Eichelsdoerfer

***/
public static function encodeHeader($input, $charset='ISO-8859-1')
{
    if(preg_match_all('/(s?w*[x80-xFF]+w*s?)/', $input, $matches))
    {
        if(function_exists('mb_internal_encoding'))
        {
            mb_internal_encoding($charset);
            $input = mb_encode_mimeheader($input, $charset, 'Q');
        }
        else
        {
            foreach ($matches[1] as $value)
            {
                $replacement = preg_replace('/([x20x80-xFF])/e', '"=" . strtoupper(dechex(ord("1")))', $value);
                $input = str_replace($value, '=?' . $charset . '?Q?' . $replacement . '?=', $input);
            }
        }
    }
    return $input;
}

EDIT: Removed the spam filter comment, since I couldn’t reproduce this. Might have been another problem.

Added forum comment with explanations and test results.

Awesome. Thanks for that michael-e. I will be sure to update integration.

Using General::encodeHeader() function written by ‘michael-e’. Closed by 0e12e019122d58f26d3a017fca403d736e41aa25

This issue is closed.

Symphony • Open Source XSLT CMS

Server Requirements

  • PHP 5.3-5.6 or 7.0-7.3
  • PHP's LibXML module, with the XSLT extension enabled (--with-xsl)
  • MySQL 5.5 or above
  • An Apache or Litespeed webserver
  • Apache's mod_rewrite module or equivalent

Compatible Hosts

Sign in

Login details