• Do not register here on develop.twiki.org, login with your twiki.org account.
• Use View topic Item7848 for generic doc work for TWiki-6.1.1. Use View topic Item7851 for doc work on extensions that are not part of a release. More... Close
• Anything you create or change in standard webs (Main, TWiki, Sandbox etc) will be automatically reverted on every SVN update.
Does this site look broken?. Use the LitterTray web for test cases.

Item5710: trunk broken for UTF8

Item Form Data

AppliesTo: Component: Priority: CurrentState: WaitingFor: TargetRelease ReleasedIn
Engine I18N Urgent Closed   patch 4.2.1

Edit Form Data

Reported By:
Applies To:
Current State:
Waiting For:
Target Release:
Released In:


A change in TWikirev:16846 (for Item5656) has broken trunk for UTF8 sites.
+    # Tell CGI.pm which charset we are using if not default
+    if( defined $TWiki::cfg{Site}{CharSet} &&
+          $TWiki::cfg{Site}{CharSet} !~ /^iso-?8859-?1$/io ) {
+        $query->charset( $TWiki::cfg{Site}{CharSet} );
+    }

This causes CGI to automatically and silently translate URL parameters to unicode. You would expect that to be correct, but unfortunately the rest of TWiki assumes they are UTF8 encoded, and not unicode, and this causes a crash deep in RcsFile.pm viz Wide character in print.

To reproduce the problem, configure a site to {Site}{CharSet} = 'utf-8' and edit a new topic using the raw editor. Paste in the first line of http://en.wikipedia.org/wiki/Chinese_language (the line with the chinese characters on it) and save the topic.

The solution is to change this call back to CGI::charset instead of $query->charset, as this doesn't cause conversion of the parameters to unicode.

-- TWiki:Main/CrawfordCurrie - 17 Jun 2008

that change is in twiki42 branch as well :/

-- TWiki:Main.SvenDowideit - 17 Jun 2008

The behaviour is an undocument side-effect of CGI::charset, and has been reported to CPAN. http://rt.cpan.org/Ticket/Display.html?id=36817

-- CrawfordCurrie - 17 Jun 2008

One thing is the report to CPAN. Another TWiki.

I do not see the fix checked in???

-- TWiki:Main.KennethLavrsen - 01 Jul 2008

Fix checked in in agreement with Crawford.

Bug was never in released version so closing

-- TWiki:Main.KennethLavrsen - 15 Jul 2008

Summary trunk broken for UTF8
ReportedBy TWiki:Main.CrawfordCurrie

SVN Range TWiki-5.0.0, Sun, 01 Jun 2008, build 16865
AppliesTo Engine
Component I18N
Priority Urgent
CurrentState Closed

Checkins TWikirev:17020 TWikirev:17021
TargetRelease patch
ReleasedIn 4.2.1
Edit | Attach | Watch | Print version | History: r6 < r5 < r4 < r3 < r2 | Backlinks | Raw View |  Raw edit | More topic actions
Topic revision: r6 - 2008-07-15 - KennethLavrsen
This site is powered by the TWiki collaboration platform Powered by PerlCopyright © 2008-2023 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback