Item6896 < Bugs

AppliesTo:	Component:	Priority:	CurrentState:	WaitingFor:	TargetRelease	ReleasedIn
Engine		Normal	Closed		major	6.0.0

This is caused by CGI::textarea() in view() in core/lib/TWiki/UI/View.pm. Calling CGI::charset($TWiki::cfg{Site}{CharSet}) before CGI::textarea() call solves this problem. It's confirmed to be harmless with ISO-8859-1. It should do no harm to other character sets.

-- TWiki:Main/HideyoImazu - 2012-06-29

It turned out that UTF-8 is not the only character set affected. Specifically, character sets utilizing the \x8b or \x9b bytes in a different way from Windows CP1252 are affected. Such as...

Shift JIS (a.k.a. Windows CP932, very common for Japanese)
Big5 (a.k.a. Windows CP950, traditional Chinese)
Windows CP936 (Simplified Chinese)
Windows CP949 (Korean)

In CGI.pm, the following lines of escapeHTML() affect those bytes.

         my $latin = uc $self->{'.charset'} eq 'ISO-8859-1' ||
                     uc $self->{'.charset'} eq 'WINDOWS-1252';
         if ($latin) {  # bug in some browsers

Other character sets than ISO-8859-1 or WINDOWS-1252 may need some treatment here, but it's up to CGI.pm maintainer to take care.

-- TWiki:Main.HideyoImazu - 2012-07-02

ItemTemplate
Summary	When charset=UTF-8, viewing a page with non-ASCII character with raw=on parameter corrupts some characters
ReportedBy	TWiki:Main.HideyoImazu
Codebase
SVN Range
AppliesTo	Engine
Component
Priority	Normal
CurrentState	Closed
WaitingFor
Checkins	TWikirev:23066
TargetRelease	major
ReleasedIn	6.0.0

Attachments

Topic attachments
I	Attachment	History	Action	Size	Date	Who	Comment
png	utf-8-corruption.png	r1	manage	34.3 K	2012-06-29 - 09:45	HideyoImazu

Item6896: When charset=UTF-8, viewing a page with non-ASCII character with raw=on parameter corrupts some characters

Item Form Data

Edit Form Data

Detail