• Do not register here on develop.twiki.org, login with your twiki.org account.
• Use View topic Item7848 for generic doc work for TWiki-6.1.1. Use View topic Item7851 for doc work on extensions that are not part of a release. More... Close
• Anything you create or change in standard webs (Main, TWiki, Sandbox etc) will be automatically reverted on every SVN update.
Does this site look broken?. Use the LitterTray web for test cases.

Item3489: Formatted search breaks with formfield variables and when using nested search

Item Form Data

AppliesTo: Component: Priority: CurrentState: WaitingFor: TargetRelease ReleasedIn
Engine   Urgent Closed   patch 4.1.1

Edit Form Data

Reported By:
Applies To:
Current State:
Waiting For:
Target Release:
Released In:


Install Fix on TWiki-4.1.0 (fix updated 27 Jan 2007 with a more complete fix)

We have found some basic and important problems with the formatted search feature in 4.1.0.

The problem is mainly that the $dollar variable gets expanded too early and makes nested searching no longer working.

In addition some basic features with simple formatted searches are broken too all related to the $-character

Until we get 4.1.1 released you can fix your TWiki 4.1.0 by simply replacing two files.

And your searches should work fine again.

Watch out for 4.1.1 which will be released in a not so distant future.

-- TWiki:Main.KennethLavrsen - 27 Jan 2007

Nested search is broken. The variables such as $dollar no longer work. This breaks quite many of our TWiki applications among other our Quamity Management System forcing me to revert to 4.0.5. This is a very urgent bug. For test case see FormattedSearch#Nested_Search. The documented examples are all broken.

The bug occurs because Wiki::expandStandardEscapes() was called before the $dollar, $topic, etc vars have been expanded. But it must be the last thing to be replaced, i.e. not to remove $dollar too early.


Thank you Michael Daum for this quick fix. I have merged both your checkins into Patch04x01 branch so it will be in the 4.1.1 release.

I will attach the fixed search to this bug item so people can download it from KnownIssues.


Thanks for that quick turnaround lads but it seems I'm still having problems with $dollarformfield(MyField) in nested search. Seems to me like it's trying to get the formfield value from the top level SEARCH. The MD patched version works fine with $dollartopic though. I set the status back to New.


I compared the fixed Search.pm provided above with Search.pm as it was 4.0.2 and came up with the following patch. It fixes the problem I had above. I just put back the expandStandardEscapes() to the very place it was in 4.0.2. I hope it does not break anything. Maybe we should implement unit test for nested SEARCH to avoid regressions like this one.

-- TWiki:Main.StephaneLenclud - 24 Jan 2007

Please do not put in waiting for release unless the fix is checked into SVN.

-- KennethLavrsen - 24 Jan 2007

The patch attached need careful testing before checking in. Going back to 4.0.2 code will very likely just revoke an older bug

-- KennethLavrsen - 25 Jan 2007

Stephane, we can't simply go back to 4.0.2 code since this would re-introduce bug Item2837. This bug Item3489 was introduced when fixing that Item2837. So we need to make sure that we fix it properly.

-- PeterThoeny - 25 Jan 2007

There is another bug re-introduced: As you can see in LitterTray.FormTestTopic, $foo, $nop, $quot, $percnt, $dollar in a $formfield(FirstName) in a SEARCH gets displayed properly, but it is broken for $summary. Special chars in formfields survive an edit/save cycle, so that is OK.

-- PeterThoeny - 25 Jan 2007

I can confirm the bug Stephane is pointing out, see LitterTray.FormTest. A format with $dollarformfield(anything) gets evaluated as $formfield(anything) by the first SEARCH, thus preventing the neste search from ever evaluating the form field.

-- PeterThoeny - 25 Jan 2007

Seems like fixing both bugs might be tricky. We may have to make a choice here. Having/supporting escape character in field value (Item2837) sounds a bit weird to me. What's the use case behind that requirement? If it's just about supporting our bug tracking application here we (TWiki savvy) can always have something like $dollardollar in field value to make it work with SEARCH. But then it does not render nice outside of a SEARCH !?! IMHO and unless we come up with a proper fix we should leave Item2837 open and fix Item3489 in the next patch release.

On the long term I think we should provide TWiki application developer a way to implement same functionality as nested SEARCH but with a proper syntax. Ever tried to read a nested SEARCH code only a week after you wrote it? stick out tongue

-- StephaneLenclud - 25 Jan 2007

A solution to both bugs might be to $formfieldvalue =~ s/\$/\$dollar/g prior to replace $formfield(myfield) in output and leave the expandStandardEscapes at the very end as suggested in my patch.

-- StephaneLenclud - 25 Jan 2007

I think a safe way to fix this is to revert to the 4.0 version and to escape $something with $<nop>something in the proper places, such as data from a form field.

-- PeterThoeny - 26 Jan 2007

I have checked in TestCaseAutoFormattedSearch and some test topics that goes with it. I have also updated TestCaseAutoSearch

Please review these test cases. The Item3501 tracks the test case update itself.

And please make sure that code updates to fix this does not break other cases and make sure the current 3 failing test cases are all fixed.

-- KennethLavrsen - 26 Jan 2007

I am working on the fix for this.

For the record, the code in 4.0.X translates $variables in both formfields and summary. The fix I am working on addresses both and I will extend my test case to catch this as well.

-- KennethLavrsen - 27 Jan 2007

Closing time. All should be fixed now.

  • $variables in formfields are ed so they do not expand
  • $variables in summaries are ed so they do not expand
  • renderFormFieldArg got some extra inline doc so we can see what is does
  • expandStandardEscapes is run at the end of the formatted search where it originally was and belongs. It must be at the end! Otherwise the
$dollar etc are removed too early and nested formatted search breaks
  • test cases are updated
  • All merged to Patch04x01 ready for 4.1.1

-- KennethLavrsen - 27 Jan 2007

I have uploaded the new Search.pm and Render.pm which contains the final fixes.

I have hidden Stephanes patch file to avoid confusing people that come by from known issues. But I appreciated the patch. It was a great help Stephane.

-- TWiki:Main.KennethLavrsen - 27 Jan 2007

Summary Formatted search breaks with formfield variables and when using nested search
ReportedBy TWiki:Main.KennethLavrsen
Codebase 4.1.0, ~twiki4
SVN Range TWiki-4.1.0, Mon, 15 Jan 2007, build 12567
AppliesTo Engine

Priority Urgent
CurrentState Closed

Checkins 12600 12603 12604 12605 12637 12638
TargetRelease patch
ReleasedIn 4.1.1
Topic attachments
I Attachment History Action Size Date Who Comment
Perl source code filepm Render.pm r1 manage 65.1 K 2007-01-27 - 21:05 UnknownUser Replacement lib/TWiki/Render.pm for 4.1.0
Perl source code filepm Search.pm r2 r1 manage 41.4 K 2007-01-27 - 21:03 UnknownUser Replacement lib/TWiki/Search.pm for 4.1.0 (new version)
Edit | Attach | Watch | Print version | History: r28 < r27 < r26 < r25 < r24 | Backlinks | Raw View |  Raw edit | More topic actions
Topic revision: r28 - 2007-02-05 - JasonHill
This site is powered by the TWiki collaboration platform Powered by PerlCopyright © 2008-2024 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback