[WD23] Email text problems [Solved]

Hi all,

I'm building an email system for my ERP and have some problems that I hope somebody can help me to solve.

In my email reading window I have two text fields, one for plain text (from email.message) and one for html text (from email.html).

Problem 1:
The html text field is properly filled with the html contents (not always, see below), as it should, but the plain text field has the contents of them both; first the plain text and then the html contents. Furthermore, the plain text is all in one single text mass, so WD obviously cannot convert the LFs and CRs in it.
So, how can I cut the HTML part away from the plain text, and
How can I make the plain text more readable?
Correction: The "furthermore error" was my fault: the "plain text" field was actually defined also as an HTML field. When I changed it to plain text, the line feeds appeared. Actually there are now too many line feeds, which I obviously have to clean away in my conversion code. But the HTML is still there (as HTML code text) after the plain text section...

Problem 2:
In some cases WD cannot convert the extended characters correctly. For instance, when the header has a content type like this:

Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 8bit
or
Content-Type: text/html; charset=utf-8
Content-Transfer-Encoding: 8bit

the extended characters are coded like below, and they produce only "boxes" in the displayed text:
=C3=A4 (= ä - latin small letter a with diaeresis)
=C3=84 (= Ä - latin capital letter A with diaeresis)

=C3=B6 (= ö - latin small letter o with diaeresis)
=C3=96 (= Ö - latin capital letter O with diaeresis)

=C3=A5 (= å - latin small letter a with ring above)
=C3=85 (= Å - latin capital letter A with ring above)

=C3=BC (= ü - latin small letter u with diaeresis)
=C3=9C (= Ü - latin capital letter U with diaeresis)

Of course I could replace these codes to readable letters, but surely there must be a more efficient way? Is there?
Any suggestions will be appreciated, as always.

Best regards
Ola



Edited 2 time(s). Last edit at 10/13/2020 08:59PM by Ola.

Hi all,

My email plain text and html text fields are just strings. If the incoming email is UTF-8 coded, should I then do the UniCodeToAnsi conversion to the contents, when importing them to my email system? Or UTF8ToAnsi?

Best regards
Ola

Hi all,

I have now been wrestling with the WD email system for some time, and it really seems to me that WD is not able to correctly convert unicode UTF-8(hex) coded "extended Ascii characters" to Ansi or any readable format. They display as square boxes! And, when they are in an WD HTML variable, there is no way I could convert them myself, because they have already been "converted" to square boxes! However, when they are in the plaintext variable, I can convert them with simple replace commands:

//HTML text with UTF-8 Unicode(hex) also cannot be converted!!!!!!,
//but plain text can be coverted like this
IF Contains(StringFormat(sString,ccUpCase),"=C3=A4") THEN //"ä" is the most common in Finnish
	//Toast("Contains Älphåbéts!")
	sString = Replace(sString,"=C3=A4","ä",IgnoreCase)
	sString = Replace(sString,"=C3=B6","ö",IgnoreCase)
	sString = Replace(sString,"=C3=BC","ü",IgnoreCase)
	sString = Replace(sString,"=C3=84","Ä",IgnoreCase)
	sString = Replace(sString,"=C3=96","Ö",IgnoreCase)
	sString = Replace(sString,"=C3=9C","Ü",IgnoreCase)
END

I think this is really weird, because these days UTF-8 is one of the most widely used character coding. And I's suppose even the French suffer from this, as they too have a lot of these "funny" characters.

Of course I could take the raw contents of the WX email variable and try to extract everything from there, but it's a jungle out there and I'm no Tarzan...

Or maybe this is a problem only with WX23? Do more recent versions have the same problem?

Best regards
Ola

Hi all,

I am talking about incoming email messages that look like this:

"Copyright � 1999-2020 BuyPal. Kaikki oikeudet pid�tet��n. BuyPal (Europe) S.� r.l. et Cie, S.C.A., Soci�t� en Commandite par Actions. Rekister�ity toimipaikka: 26-28 Boulevard de la Madeleine..."

Not very nice, is it? Not even the copyright character comes out correctly. And there is no way I could fix it, because WD has already destroyed the original character coding in its email variable. Or is there? Am I missing something, like usual? Please let me know.

Best regards
Ola

Hi Ola,

Guess your project is still in ANSI and the email (encoded as UTF-8 most likely) is unicode...
I'm not doing any WX ANSI stuff anymore since WX17...

Today's standard is unicode and it is the default for mobile development and most web services for example.
So if you don't want to do a lot of ANSIToxxx and xxxToANSI translations or specific declarations (is ANSI string , is UNICODE string), I'ld suggest you move your project and database over to unicode. ANSI is a thing from the past...

Just my 2 cents...

Peter

Hi Peter,

Your guess is correct. The HTML file item was a memo-type field (Ansi) so I changed it to Unicode. And indeed, this seems to be the solution to my problem!

I'm much obliged!

Best regards
Ola

Hi Ola

There's an option in the analysis editor to set all string type fields to Unicode at once.
You might give it a shot and next modify your project configuration in Unicode if that is not already done.

Cheers

Peter

Hi Peter,

Yes, I know. We had this Unicode conversation earlier:
[www.wxforum.info]

I've just been too busy to get down to the Unicode conversion.
I'll concentrate on it when I get my email system fully tested
and integrated to the rest of the ERP -- unless something more
pressing comes up, as usual...

But thank you. Your "two cents" are always worth much more!

Best regards
Ola