Item3575: Usability: Use pub url instead of viewfile url in attachment table

http://koala.ilog.fr/twikiirc/bin/irclogger_log/twiki?date=2007-02-06,Tue&sel=264#l260

The file links in the attachment table point to the viewfile script. This has drawbacks:

• Speed - cgi access is much much slower than a direct access
• Long URL - if URL is sent with Outluck, long URLs tend to break apart, rendering them useless
• Inaccessible - if a site prohibits wget and other robots from accessing TWiki scripts (as on twiki.org due to server load)

Older TWiki versions pointed directly to the pub URL, which avoids these issue. The spec was changed supposedly to protect content under access control. However, the pub url is unprotected by default, so it does not help to protect the content with viewfile. Also, if the pub URL is protected (as per supplemental doc), from a security perspective it does not make a difference if the pub URL is used or the viewfile is used.

Suggested fix: Revert back the llinks in the attachment table to the pub url, e.g. %PUBURL%/%WEB%/%TOPIC%/file.ext. Attention: This cannot be done in the attachment history table.

-- TWiki:Main/PeterThoeny - 06 Feb 2007

I would like to challenge the statement that "if the pub URL is protected (as per supplemental doc), from a security perspective it does not make a difference if the pub URL is used or the viewfile is used"

Can you explain this?

-- TWiki:Main.KennethLavrsen - 06 Feb 2007

Peter is right, insofar as if there are Apache protections ( require valid-user ) applied to the pub dir, then you are able to at least force a login for viewing of attachments. Peter is making the point that if the logged-in user is able to view a topic (and therefore the attachment table) they have already fulfilled the conditions for being able to view latest rev the attachment, and should therefore be shown a link, not the viewfile.

However the counter argument, and the real reason viewfile links were introduced, is that a security concious site will not open the pub dir to the public view. In this case a viewfile is the only way to view the content of the attachment. See TWiki:Codev/SecuringAttachments

The correspondent on IRC (harlan) is asserting out that downloading plugins from t.o. - which are stored in attachments - is "no longer possible using wget" because the viewfile link, rather than the pub link, is used. This is incorrect, as a viewfile link can be used to download as readily as a pub link (try it). OK, so you get a wierd filename, but that's the only problem, and one that could probably be solved by intelligent use of the HTTP headers in the response, or even mv on the client.

• No, I always get an authentication error. Somehow I need to pass username and password. -- TWiki:Main.ArthurClemens - 08 Feb 2007

At the end of the day, a site has to make the decision whether attachment security is more important to them than performance. Pattern and Classic skins both go for the "more secure" approach of using the viewfile link in the template.

The real issue here, and the reason I am not simply reassigning this issue to the skin components, is that the performance of viewfile is unacceptably slow. viewfile it is a full-blooded TWiki CGI script, which loads not only the perl interpreter, but also the whole of TWiki, preferences, plugins, everything, to make a simple decision on permissions. I addressed this issue in the WebDAVPlugin by cacheing topic permissions in a TDB which was accessed by a 'C' program - indeed, the WebDAVPlugin code could no doubt be used to hugely accelerate viewfile - but it is C-code, so not useable for default TWiki. However a similar approach (cacheing) could be taken in pure perl to accelerate viewfile. It wil never approach C-code speeds, but at least it would afford a more secure solution for sites that require it.

OK, so what are the options? As I see it:

1. Accelerate viewfile (and improve the HTTP headers) to the point where it is fast enough
   • I personally doubt this is possible without mod_perl, but no-one has had time to try yet
2. Allow a site to optionally select use of viewfile (for a protected pub) or plain links
   • Should be as simple as adding an %IF to the templates
3. Revert the templates to links in the skins
   • This requires open pub, which as Peter points out is the default.
   • However it will cause a major problem for users who run secure (no public pub) sites.
     • This will be the case for sites that are not open for TWikiGuest users, otherwise no stylesheets or javascripts will be loaded. We just don't have the possibility to secure the pub dir for general use. AC
   • The documention (in TWiki:Codev.SecuringYourTWiki) would have to be updated to explain to admins how to edit the skin templates to secure their attachments again, and a huge banner warning issued.
4. Code an apache module that checks TWiki permissions; that way you can go back to using links, no problem.
   • Could be an interesting experiment

Changing to Confirmed, as the problem is understood and susceptible to improvement.

CC

I've done (4) and proposed it in several places long time ago. Note however, that this requires mod_perl which is not available in hosted environments, and when TWiki is suexec'ed. The implementation I did is able to check any auxiliary files coming from .../pub/... and has the little extra feature to prevent the expensive ACL check based on a regex on the url. This way you can prevent permission checks for attachments in .../pub/TWiki/.* where the skins' stylesheets, javascripts and graphics are stored. You really don't want to check each of them again and again. So ACLs only get checked for those webs and topics that really need to be checked (e.g. your private p*** collection). I am selling this solution to all of my clients. I've got it running on all sites hosted on http://wikiring.de.

MD

Also note that attachment links inserted in the topic by default use the pub path, not the viewfile link.

AC

Right, there's no point of having viewfile but leaving the http://.../pub... url unprotected ... which is the state of the affairs. So, IMHO, viewfile urls in attachment are only an annoyance. Apache is doing fine in serving plain files already. Forcing each file through viewfile is no good idea. We really only need to teach apache TWiki ACLs.

MD

I can confirm that the viewfile links gives my users on my Motion TWiki a lot of trouble. I would not object to changing the attachment table links back to pub links.

I just want to make sure it is still possible to protect attachments in general in selected webs by using the Apache config redirecting to viewfile. And it seems this is possible.

-- TWiki:Main.KennethLavrsen - 10 Feb 2007

I've changed the default linkage of A_URL to pub path file; added A_VIEWFILE to be able to create a viewfile link in the template.

-- TWiki:Main.ArthurClemens - 22 May 2007

Per e-mail:

I'm really not happy about this. The use of viewfile versus a URL should be transparent to the code and to the templates; the template author shouldn't need to know. That was the reason for my SMELL remark.

I would much rather we made A_WEB, A_TOPIC and A_FILE available to the template, and allowed the construction of the URL to be done there, based on %PUBURL%, rather than adding more code to generate new temporary TWiki variables.

CC

Reverting this last change in favor of a solution in the attachtables template.

-- TWiki:Main.ArthurClemens - 25 May 2007