• Do not register here on develop.twiki.org, login with your twiki.org account.
• Use View topic Item7848 for generic doc work for TWiki-6.1.1. Use View topic Item7851 for doc work on extensions that are not part of a release. More... Close
• Anything you create or change in standard webs (Main, TWiki, Sandbox etc) will be automatically reverted on every SVN update.
Does this site look broken?. Use the LitterTray web for test cases.

While looking through my access logs I noticed a strange looking bin/search request from what appeared to be googlebot. I entered it in my browser and after several minutes received a 500 Internal Server Error. Closer inspection of the logs revealed that the server ran out of memory.

Access log entries

66.249.65.132 - - [21/Feb/2006:08:29:43 -0800] "GET /twiki/bin/search/TWiki/?scope=topic&regex=on&bookview=on&search=%5C.* HTTP/1.1" 200 - "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
66.249.65.132 - - [21/Feb/2006:08:34:37 -0800] "GET /twiki/bin/search/TWiki/?scope=topic&regex=on&bookview=on&search=%5C.* HTTP/1.1" 200 - "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"

Sample error log entries

[Tue Feb 21 08:29:33 2006] search: Use of uninitialized value in pattern match (m//) at /home/rhill/family.qbfreak.net/twiki/lib/TWiki/If.pm line 271.
Out of memory!
[Tue Feb 21 08:29:43 2006] [error] [client 66.249.65.132] Premature end of script headers: /home/rhill/family.qbfreak.net/twiki/bin/search

The first line of the error log is repeated about 50 times for each of the sites I tested this on Dreamhost.

I tried the same on TWikiVMDebianStable (twiki-vm-debian-stable-4.0.1-vm02) with similar results. I did not run out of memory, and ended up with approximately 100 entries in my error.log looking much like the first line listed above. I also received a handful of errors that looked a lot like this:

********************************
OopsException(accessdenied/no_such_web web=>TWiki/\/twiki/pub/TWiki/TWikiDocGraphics/tip topic=>gif\ params=>view)
********************************
I don't know if they are related or not.

On the virtual machine, it took TWiki and Apache somewhere between 5 and 8 minutes to serve the page and it took Firefox another 15-20 minutes to render it. The resulting page consisted of every single topic in the TWiki web.

I do NOT recommend trying this on a production site, the URL I used on the VM was http://twiki-vm/twiki/bin/search/TWiki/?scope=topic&regex=on&bookview=on&search=%5C.* and the resulting entry in TWiki's log looked like:

22 Feb 2006 - 10:48 TWikiGuest search TWiki \.* Mozilla 192.168.237.1

I have confirmed that this occurs on 4.0.0 (build 8671), 4.0.1 (build 8740), and what I believe to be 4.0.0-b6 (build 7851). I attached the error log from the VM, I can attach or e-mail the other logs opon request.

I realize that TWiki is just doing exactly what it was intended to do, but the potential to use this as a Denial of Service attack is pretty big.

-- JasonHill - 23 Feb 2006

Use of ressources can (and should) be limited either through the webserver or through the respective TWiki:Codev.CategoryCgiAccelerators used.

That leaves the other error, "Use of uninitialized value".

When I perform the same, I get this:

In data/warn.txt :
| 26 Feb 2006 - 11:00 | Form: get find category template twikicatitems for Web TWiki

In apache error log:
[Sun Feb 26 10:58:18 2006] [error] [client 192.168.1.1] Premature end of script headers: /home/httpd/twiki/ibensverden.dk/bin/search

The browser reports "Internal Server Error" rather quickly, but search keeps running for a while.

-- SP

There is no specific fix for this, other than through Apache wihich is outside the scope of this bug DB, so I'm disarding it.

CC

ItemTemplate
Summary Bookview in search often runs out of memory when searching for all topics in a web
ReportedBy TWiki:Main.JasonHill
Codebase

SVN Range Wed, 08 Feb 2006 build 8740
AppliesTo Engine
Component

Priority Urgent
CurrentState No Action Required
WaitingFor

Checkins

TargetRelease n/a
Topic attachments
I Attachment History Action Size Date Who Comment
Unknown file formatlog error_twiki-vm-4.0.1.log r1 manage 14.8 K 2006-02-23 - 02:36 UnknownUser Error log from twiki-vm-debian-stable-4.0.1-vm02
Edit | Attach | Watch | Print version | History: r5 < r4 < r3 < r2 < r1 | Backlinks | Raw View |  Raw edit | More topic actions
Topic revision: r5 - 2006-05-15 - CrawfordCurrie
 
This site is powered by the TWiki collaboration platform Powered by PerlCopyright © 2008-2024 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback