• Do not register here on develop.twiki.org, login with your twiki.org account.
• Use View topic Item7848 for generic doc work for TWiki-6.1.1. Use View topic Item7851 for doc work on extensions that are not part of a release. More... Close
• Anything you create or change in standard webs (Main, TWiki, Sandbox etc) will be automatically reverted on every SVN update.
Does this site look broken?. Use the LitterTray web for test cases.

I dont know, if it is a bug or a configuration issue. May someone enlighten me. wink

Testcase:

  • $TWiki::cfg{Site}{CharSet} = 'iso-8859-15'
  • $TWiki::cfg{UseLocale} = "on" and "off"
  • $TWiki::cfg{Site}{Locale} = "de_DE.iso885915@euro"

  • Insert a euro sign into topic text.
  • Go through edit-cycle 1-2 more times.

Result:

  • The euro sign gets converted to a 164 entity and then not shown correctly by the browser


There are a couple of ways to get the euro sign:

  • numeric: € (widely supported)
  • entity: € (less widely supported, but degrades better)
  • char ref: € (not recommended)
  • iso encoding: ¤ (requires ISO 8859-15 support)

see http://www.ascii.cl/htmlcodes.htm

MD.


The euro sign needs to be insertable by the euro-key for non-programmers. -- OK - 07 Oct 2005


What happens is that the character #164 is the euro sign in iso-8959-15, and that little star (or someting like it) in iso-8959-1.

The problem is that browsers -- at least Firefox does -- seem to interpret ¤ as the little star even if under iso-8859-15 (I don't know if it's a bug os a feature). See this.

Since the text put inside the edit box is passed into TWiki::entityEncode, euro signs are converted into ¤. One possible sollution is to change TWiki::entityEncode this way:

=== TWiki.pm
==================================================================
--- TWiki.pm    (revision 10487)
+++ TWiki.pm    (local)
@@ -1760,12 +1760,15 @@
 This method encodes <, >, &, " and any 8-bit characters
 using numeric entities.

+In the case we are using iso-8859-15 charset, the euro sign is handled
+specially: instead of ¤ , we should use €.
+
 =cut

 sub entityEncode {
     my $text = shift;

-    $text =~ s/([^ -~\n\r]|[]["<>&])/'&#'.ord( $1 ).';'/ge;
+    $text =~ s/([^ -~\n\r]|[]["<>&])/'&'. (((lc($TWiki::cfg{Site}{CharSet}) eq 'iso-8859-15') && (ord($1) == 164))?('euro'):('#' . ord( $1 )) ).';'/ge;
     return $text;
 }

AT


Hm, why not render any standard html entitiies, not only euro, when needed, ie not depending of the server's charset.

MD


Antonio, I think it is unsafe to change entityEncode that way; after all, it is behaving correctly, and mapping 16 bit chars to their 7-bit numeric encodings. I also think it is unsafe to map characters to their symbolic equivalents - after all, I don't want the little star I inserted when editing under iso-8959-1 to be forced into a euro, do I?

The euro sign needs to be insertable by the euro-key for non-programmers. -- OK - 07 Oct 2005

Surely this is a translation that is totally dependent on the browser character set? You need to say somewhere: "if the browser character set was iso-8959-15 then translate character 164 to a symbolic euro when you read the browser input". That's not something you can hack around in entityEncode.

CC


The euro sign needs to be insertable by the euro-key for non-programmers. -- OK - 07 Oct 2005

right, and it needs to be edittable by non-programmers, too. that is, after entering a , i still want it to be there in the edit box (and not with some funking encoding).

WN

This bug is probably related to the recently posted one about UTF-8 malformed characters when running with Perl 5.6 and certain versions of Locale::Maketext (version 1.06 or higher, I believe, which includes a use utf8 near the top of the Guts.pm module). This was causing Perl 5.6 or lower to go into UTF-8 mode despite the locale character set being ISO-8859-15 as here.

Of course, fixing this with a solution for the Euro only would have been a bad idea. However, fortunately this all works now - the '' you see here is in ISO-8859-15 and not corrupted.

It's also worth Googling on TWiki for 'numeric character references' (NCRs), which I've mentioned a few times - these are the &#nnnn; entity references that actually generate Unicode characters, *no matter what your current character encoding is set to,* as required by the XML and HTML specs. These NCRs sometimes appear if you paste (say) KOI8-R character set data into an ISO-8859-1 page, i.e. the browser generates them as a way of trying preserve the data. However, they should really be considered an indicator of a problem. See TWiki:Codev.InternationalisationEnhancements, which has some links on UTF-8 as well.

I think this problem is not a bug in TWiki, but should be documented (i.e. install version 1.05 or lower of Locale::Maketext on Perl 5.6 or lower) - we should also create a patch to remove use utf8 from CPAN:Locale::Maketext::Guts and submit it to Sean Burke, the author of Locale::Maketext.

RD

The problem is that from version 1.06, CPAN:Locale::Maketext had a redesign that, at the end, requires Perl 5.8 (use utf8).

The sollution is to recomend versions older the 1.06. think we could close this now. Well, it's documented in TWiki:Codev.UserInterfaceInternationalisation (where the user is pointed to from DakarReleaseNotes).

AT

Item782 seems to address the same issue.

AT

Duplicate of 782 CC

ItemTemplate
Summary Euro sign gets lost through edit-cycles
ReportedBy OliverKrueger
AppliesTo Engine
Priority Normal
CurrentState No Action Required
WaitingFor

Checkins 7206
Topic attachments
I Attachment History Action Size Date Who Comment
HTMLhtml test-euro.html r0 manage 0.4 K 2005-10-08 - 16:31 AntonioTerceiro see also the HTML source
Edit | Attach | Watch | Print version | History: r16 < r15 < r14 < r13 < r12 | Backlinks | Raw View | Raw edit | More topic actions
Topic revision: r16 - 2005-11-06 - CrawfordCurrie
 
This site is powered by the TWiki collaboration platform Powered by PerlCopyright © 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback