• Do not register here on develop.twiki.org, login with your twiki.org account.
• Use View topic Item7848 for generic doc work for TWiki-6.1.1. Use View topic Item7851 for doc work on extensions that are not part of a release. More... Close
• Anything you create or change in standard webs (Main, TWiki, Sandbox etc) will be automatically reverted on every SVN update.
Does this site look broken?. Use the LitterTray web for test cases.

Item3749: Configure always fails now when downloading a tgz plugin - maybe server config problem

Item Form Data

AppliesTo: Component: Priority: CurrentState: WaitingFor: TargetRelease ReleasedIn
NotTWiki TWiki.org Urgent Closed   n/a  

Edit Form Data

Reported By:
Applies To:
Current State:
Waiting For:
Target Release:
Released In:


No matter what extention I try to install with configure I get the error

Warning: I can't install http://twiki.org/p/pub/Plugins/CompareRevisionsAddOn/CompareRevisionsAddOn.tgz because I don't recognise the download as a gzip file. 
Warning: Extension may not have been packaged correctly. Trying for a .zip file instead. 

It fails in EXTEND.pm

    } elsif ($ar !~ s!^.*Content-Type: application/x-gzip\r\n\r\n!!is) {

This worked until recently. Maybe it is the twiki.org setting that are still goofing up things.

Anyone trying to install extensions with configure will experience this.

I also get total errors (fail to install at all) when plugins are only uploaded a a zip file.

This is far too fragile.

We need to fix twiki.org to get existing customers going again.

And we need configure to not just rely on mime headers.

-- TWiki:Main/KennethLavrsen - 11 Mar 2007

With a packet sniffer it seems that downloading a tgz file directly from pub gives you a server reply with Content-Type: application/x-tar which is wrong.

It makes IE save the file as .tar and it makes configure fail installing extensions.

It seems a bit random which extensions configure can download. Most fails when trying the .tgz first. And quite many fails completely.

We need twiki.org setup properly now!

-- TWiki:Main.KennethLavrsen - 11 Mar 2007

The tests that must pass are

Additionally test that the files in all 4 cases can be downloaded by Internet Explorer and Firefox and can be unpacked afterwards.

-- TWiki:Main.KennethLavrsen - 11 Mar 2007

I removed the incorrect directive that mapped tgz to content type tar on twiki.org. Please try again.

-- TWiki:Main.MichaelDaum - 12 Mar 2007

The configure program still fails even though the mime type is now OK.

The headers that configure receives is now

HTTP/1.1 200 OK
Date: Mon, 12 Mar 2007 07:37:03 GMT
Server: Apache/2.2.0 (Unix)
Last-Modified: Sat, 03 Mar 2007 07:20:24 GMT
ETag: "2b399-45195-8ec1de00"
Accept-Ranges: bytes
Content-Length: 283029
Connection: close
Content-Type: application/x-gzip
Content-Encoding: x-gzip

When you look at lib/TWiki/Configure/UIs/EXTEND.pm line 63 (4.1.2)

    } elsif ($ar !~ s!^.*Content-Type: application/x-gzip\r\n\r\n!!is) {

So it is expecting the Content-Type to be the last header line followed by an empty line to indicate end of headers. This is pretty aggresive coding and should be changed. But to get all the already downloaded configures working we probably have to chase some AddEncoding statements as well.

You can easily test this Michael. Just user configure to install DirectedGraphPlugin

-- TWiki:Main.KennethLavrsen - 12 Mar 2007

Oh my gawd ... it must be the last line? This is insane. So being able to download a gzipped file from twiki.org (tgz archives) only worked by accident before. I have removed the x-gzip content type. Downloading tgz archives using IE6 is now broken again. So either the content encoding is set to make IE6 happy or the content encoding is not set and the installer is happy. But we can't have both. That the extra \r\n must be removed from the installer(s) code.

-- TWiki:Main.MichaelDaum - 12 Mar 2007

What the installer does is to replace everything from the beginning to the Content-Type: application/x-gzip\r\n\r\n.

So we cannot just change the code to not include the \r\n\r\n because that will leave some header stuff left and we end up with a corrupted zip or tgz file. But the fix is easy. First we test for the right header (or just simply replace it with a simpler test that just tests for valid download - for example by looking for the HTTP/1.1 200) and then cut from ^ to \r\n\r\n.

-- TWiki:Main.KennethLavrsen - 12 Mar 2007

I can confirm that the installer works now.

I keep this open to fix the configure to be more robust but since this is only relevant with non twiki.org repositories, and the default Apache setup most people will use in practical does not give this problem we can lower the urgentcy to normal.

-- TWiki:Main.KennethLavrsen - 14 Mar 2007

Problem is back again, 4.1.2 replies:

Warning: I can't install http://twiki.org/p/pub/Plugins/ChartPlugin/ChartPlugin.tgz because I don't recognise the download as a gzip file.

Raising to Urgent again (I understand from the above that it is possible to eliminate this error).

-- TWiki:Main.SteffenPoulsen - 09 Apr 2007

Duplicate of 3749 Closed.

-- TWiki:Main.KennethLavrsen - 02 Jul 2007

This is Item3749, link should be corrected to point to correct duplicate before closing.

-- TWiki:Main.SteffenPoulsen - 03 Jul 2007

Duplicate of 3 items: Item4161, Item3914, Item3885

The issue was that depending on how Apache felt like today the sequence of headers was not always the same and the old code looked for a specific header and it had to be the last one received. Crawford did several fixes and finally decided not to check headers at all - which I agree is the best to do. If a file is called .tgz or .zip then trust that this is true because the headers are not any better at knowing if a file is valid.

-- TWiki:Main.KennethLavrsen - 03 Jul 2007

Summary Configure always fails now when downloading a tgz plugin - maybe server config problem
ReportedBy TWiki:Main.KennethLavrsen

SVN Range TWiki-4.1.2, Sat, 03 Mar 2007, build 13043
AppliesTo NotTWiki
Component TWiki.org
Priority Urgent
CurrentState Closed


TargetRelease n/a

Edit | Attach | Watch | Print version | History: r12 < r11 < r10 < r9 < r8 | Backlinks | Raw View | Raw edit | More topic actions
Topic revision: r12 - 2007-07-03 - TWikiUserMapping_KennethLavrsen
This site is powered by the TWiki collaboration platform Powered by PerlCopyright © 2008-2019 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback