• Do not register here on develop.twiki.org, login with your twiki.org account.
• Use View topic Item7848 for generic doc work for TWiki-6.1.1. Use View topic Item7851 for doc work on extensions that are not part of a release. More... Close
• Anything you create or change in standard webs (Main, TWiki, Sandbox etc) will be automatically reverted on every SVN update.
Does this site look broken?. Use the LitterTray web for test cases.

This is really a strange problem, I have tested it out for now and it happens as follows:

I wanted add a Search to one of my webs, to just show a summary of topics which have attached a special named form. This works good for some days, but then one of my colleguages adds a total different single page which was then shown buggy in my search result table.

If have reduces the search now to following call

%SEARCH{ "." regex="off" format="%CALC{$IF(0 != 1, <nop>, | $summary |)}%" }% 

Even if no Document ever could be found by this try to create a topic with following content

xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx()

The text you enter is absolute uninteresting is just has to have 162 characters followed by an open and a close bracket "()".

After you have add this as a topic to your web the %SEARCH{}% will produce unexpected results.

What I also have tested:

  • you can append as many characters as you like after this 164 characters
  • the opening bracket could be at any position between 1 and 163
  • linefeeds are counted as on character smile
  • text could be everything

Mystics:

I thought that it must be a problem with a regex or something like this and so I checked following states:

  • regex="on" => things happen like above
  • regex="off" => things happen like above

  • if you remove the parameter regex in SEARCH statement, problem seemed to be far away

I really hope, someone could fix it!

If any questions left wink please let me know.


I set this to priority Urgent because I am not sure if this could cause a security risk problem.


The bug is not a security problem, but rather an unfortunate consequence of the syntax of SpreadSheetPlugin. You don't need to have 162 characters in your test topic: Just one is sufficient, if it is an opening parenthesis (.

What happens is that the summary is expanded first in the format string. If it contains an unbalanced paren, this breaks the $IF construct.

I have no idea how one can have a single ( within a text operand in SpreadSheetPlugin - maybe that is simply not possible.

-- haj


Hi Harald, I am not sure if I get your answer the right way:

"maybe its not possible"

what should I do instead ? Not use SEARCH anylonger, not use regex parameter? Do you have any good idea ? To say, customers are not allowed to use brackets inside their topic text is at least not possible smile

I have set the priority back to normal, if you think its not risky.


There is something special with this 164 characters. I have not checked this inside code but I guess this happens because then $summary will only copy the first (about) 164 characters from a topic ?

I think the best way to handle those things, is to add an type specific handling inside calc or to add some kind of delimiter:

%CALC{ $IF ( "1 != 0", "<nop>", "$summary" ) }%

or

%CALC{ $IF ( 1 != 0, <nop>, $TEXT( $summary ) ) }%

or with an additional delimiter like regex search

%CALC{ $IF ( 1 != 0, <nop>, $TEXT([ $summary ]) ) }%

Hope someone will fix this or have a solution how to write the SEARCH / format statement.


Sorry - I did not want to scare you from using formatted search. The fact that I have no idea how to fix that doesn't mean that there isn't a solution, I'm just a TWiki user like you smile

I had left the bug report open, and just reassigned it to SpreadSheetPlugin, hoping that someone with more experience in this module (PTh?) could have a look.

Using parentheses in topics is of course permitted. Usually, as you found out, parens "just work". And as you suspect, the character count of 162 is indeed an artifact of a setting in Render.pm, where a summary is, per default, defined to be the first 162 characters of a topic (not counting markup).

What happens in your example is that per pure "luck" the truncated summary has an unbalanced parenthesis: it consists of 161 x and one (, but the closing parenthesis is character number 163, and not included in the summary. So the SpreadSheetPlugin is receiving a text parameter with a ( and no ) in it. This breaks its parser.

Your example breaks the parser with any topic which has an unbalanced set of parentheses in the first 162 characters. The parser does not have dedicated delimiters for text parameters. It does handle () pairs within texts, but fails if only one of them is present.

Quoting strings, like you suggested, could be a solution, but this may break existing topics which depend on the quotes being in the text after %CALC{...}% expansion. So please be patient, this seems to need a SpreadSheetPlugin expert, which unfortunately I am not.

I've set the state to "New" again, because I don't know how to fix it.

-- haj

Hi Harald, you just scared me a bit smile Thanks for your answer, I will report this also in then SpreadSheetPluginDev and hope some of the good guys will take care of this.

-- Tom

HAJ's assessment is correct, this is an artefact of how the SpreadSheetPlugin works. I cannot think of a workaround at this time. You could patch $summary in Search.pm to filter out the parnethesis. Actually, not sure if this works, but you could try to use %IF{}% instead of %CALC{}%.

The SpreadSheetPlugin could be enhanced to be aware of string delimiters (such as $IF(0 = 1, , "| $summary |")), but that is a major change of the parser. The current regex based parser is at least 10 times faster than a real parser that builds a parse tree, so I am reluctant to switch to a parse tree. There might be a way to make the regex based parser aware of string delimiters.

-- PTh

I am discarding this request since I do not anticipate changing the fast parser at this time.

-- PTh

ItemTemplate
Summary %SEARCH{ }%, strange results when having ( inside topic text
ReportedBy TWiki:Main.ThomasFreudenberg
Codebase 4.0.5
SVN Range TWiki-4.1, Sat, 18 Nov 2006, build 12001
AppliesTo Extension
Component SpreadSheetPlugin
Priority Normal
CurrentState No Action Required
WaitingFor

Checkins

TargetRelease n/a
Edit | Attach | Watch | Print version | History: r9 < r8 < r7 < r6 < r5 | Backlinks | Raw View | Raw edit | More topic actions
Topic revision: r9 - 2006-12-19 - PeterThoeny
 
This site is powered by the TWiki collaboration platform Powered by PerlCopyright © 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback