• Do not register here on develop.twiki.org, login with your twiki.org account.
• Use View topic Item7848 for generic doc work for TWiki-6.1.1. Use View topic Item7851 for doc work on extensions that are not part of a release. More... Close
• Anything you create or change in standard webs (Main, TWiki, Sandbox etc) will be automatically reverted on every SVN update.
Does this site look broken?. Use the LitterTray web for test cases.

Item7747: EDITFORMFIELD garbles some Unicode characters

Item Form Data

AppliesTo: Component: Priority: CurrentState: WaitingFor: TargetRelease ReleasedIn
Engine   Normal Closed   patch 6.1.0

Edit Form Data

Summary:
Reported By:
Codebase:
Applies To:
Component:
Priority:
Current State:
Waiting For:
Target Release:
Released In:
 

Detail

%EDITFORMFIELD{"FIELD" topic="TOPIC"}% garbles some Unicode characters. This is caused by CGI::* functions for form input fields used under the hood. Those functions call HTML::Entities::encode_entities(), which converts an "unsafe" character into an entity representation (&something;).

This may happen with any multi-byte character encoding, but let's take UTF-8 as an example. To this date, TWiki treats characters without setting the UTF-8 flag.

There is a character which is encoded to e3 82 8b in UTF-8. When EDITFORMFIELD handles such a character, CGI::* functions convert the 8b byte into ‹ partly because the UTF-8 flag is not set, which means that three byte sequence is not recognized as a single character but three characters under the hood.

When calling CGI::*, a Unicode string needs to be handed with the UTF-8 flag turned on.

-- TWiki:Main/HideyoImazu - 2016-07-08

ItemTemplate
Summary EDITFORMFIELD garbles some Unicode characters
ReportedBy TWiki:Main.HideyoImazu
Codebase ~twiki4
SVN Range TWiki-6.0.2-trunk, Thu, 05 May 2016, build 30177
AppliesTo Engine
Component

Priority Normal
CurrentState Closed
WaitingFor

Checkins TWikirev:30213 TWikirev:30214 TWikirev:30255 TWikirev:30256
TargetRelease patch
ReleasedIn 6.1.0
Edit | Attach | Watch | Print version | History: r8 < r7 < r6 < r5 < r4 | Backlinks | Raw View | Raw edit | More topic actions
Topic revision: r8 - 2018-07-17 - PeterThoeny
 
This site is powered by the TWiki collaboration platform Powered by PerlCopyright © 2008-2018 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback