Item7476: SEARCH with newline parameter adds extra newlines, breaking formatting in some cases

Example search:

%SEARCH{
   "Role$"
   scope="topic"
   type="regex"
   nosearch="on"
   nototal="on"
   newline="%BR%"
   header="| *Name* | *Purpose* | *Energizers* |"
   format="| [[$topic][$formfield(Name)]] | $formfield(Purpose) | $formfield(Energizers) |"
}%

When my 'Energizers' field contained newlines, they were replaced by <br /> followed by another newline, which added extra vertical spacing between the expected lines of text.

I tracked the problem down to a specific change made years ago:

$ svn log --diff -c 17828
------------------------------------------------------------------------
r17828 | SopanShewale | 2009-02-19 06:46:19 -0800 (Thu, 19 Feb 2009) | 1 line

Item6138: Bulletted lists in form fields are not rendered properly - skipped modifying Test Cases from RenderFormTests-needs to verify the way cases are written for trunk code

Index: Render.pm
===================================================================
--- Render.pm   (revision 17827)
+++ Render.pm   (revision 17828)
@@ -2166,11 +2166,14 @@
     }
 
     # change newlines
-    my $newline = '<br />';
+    my $newline;
     if( $attrs && defined $attrs->{newline} ) {
         $newline = $attrs->{newline};
         $newline =~ s/\$n/\n/gs;
     }
+$newline = '<br />
+' unless defined $newline;
+
     $value =~ s/\r?\n/$newline/gs;
 
     # change vbars

------------------------------------------------------------------------

This change looks suspect to me; the added code is not indented and defines a string literal containing a newline instead of \n.

This change also violates the claims of the protectFormFieldValue function header. The header states:

* newlines are replaced with <br> or the value of $attrs->{newline}

With this change in place newlines are actually replaced by <br />\n, and this is done regardless of the value of $attrs->{newline}.

In my testing on 6.0.0, reverting this change fixes the problem I have identified but also results in the return of the problem identified in Item6138. Given my current (vague) understanding of the code, this fix for Item6138 appears to be inappropriate and should be reverted and Item6138 reopened.

-- TWiki:Main.DavidDAllen - 2014-03-31

Confirmed, TWiki::Render::protectFormFieldValue does some funny stuff. It needs to be fixed to work according to the documentation:

• If SEARCH has no newline parmeter: No change in newline, e.g. breaks TWiki tables.
• If SEARCH has newline parameter use that, and nothing else.

-- TWiki:Main.PeterThoeny - 2014-04-01

This is now fixed in SVN trunk and 6.0 branch. Patch:

Index: TWiki/Render.pm
===================================================================
--- TWiki/Render.pm   (revision 27162)
+++ TWiki/Render.pm   (working copy)
@@ -2609,16 +2609,13 @@
     return $value if ( $attrs && $attrs->{encode} );
 
     # change newlines
-    my $newline;
     if( $attrs && defined $attrs->{newline} ) {
-        $newline = $attrs->{newline};
+        my $newline = $attrs->{newline};
         $newline =~ s/\$n/\n/gs;
+        $newline =~ s/\$br/<br \/>/gs;
+        $value =~ s/\r?\n/$newline/g;
     }
-$newline = '<br />
-' unless defined $newline;
 
-    $value =~ s/\r?\n/$newline/gs;
-
     # change vbars
     my $bar = '&#124;';
     if( $attrs && $attrs->{bar} ) {

-- TWiki:Main.PeterThoeny - 2014-04-01

Follow-up mini-feature: Item7477.

-- TWiki:Main.PeterThoeny - 2014-04-01