Item3727: Can't login; e-mail wiped out in .htpasswd entry

I got about 10 reports of people not being able to login. In addition, those that tried to reset the password had the e-mail address wiped out in the .htpasswd. One example entry: JasonAbner:xxxxxxxxxxxxx: (I obfuscated the password with x-es)

People known to have login problems: RaganHaggard, JasonAbner, StephaneLenclud, DenisBallant, KlausGestrich, ThomasWeigert, JasonHill, LynnwoodBrown.

Debug help is appreciated. You can access the 4.1.2 TWiki temporarily at http://twiki.org/cgi-bin/4x1/view/Sandbox/WebHome

These bugs might be related to not all TWiki users being listed in the TWikiUsers topic (which should not be a requirement.)

-- TWiki:Main/PeterThoeny - 06 Mar 2007

I doubt it's that; I'm not listed in TWikiusers, but I can log in.

-- TWiki:Main.CrawfordCurrie - 07 Mar 2007

After upgrading to 4.1.2, TWik crashed 2042 times on 6 Mar 2007 only:

• 1956 times view
• 20 times rdiff
• 24 times edit
• 17 times search
• 2 times save
• 5 times register
• 18 times the rest

Even the non-twiki script tzdate crashed 11 times.

Most probably TWiki took the .htpasswd file with it during one of these crashes. As there were 5 register calls that crashed it is very likely that it happened there.

-- TWiki:Main.MichaelDaum - 07 Mar 2007

There isn't enough information here to do anything with this report, and there has been no follow up since early March. So I am setting it no action.

CC

Please do not close a real bug. Current .htpasswd has these empty e-mail addresses:

$ egrep ':$' .htpasswd
AnneMarieBatrinca:xxxxxxxxxxxxx:
BrendanPattterson:xxxxxxxxxxxxx:
BryanCoville:xxxxxxxxxxxxx:
DanielBarnett:xxxxxxxxxxxxx:
DongpyoHong:xxxxxxxxxxxxx:
GarrettA:xxxxxxxxxxxxx:
IlayRaja:xxxxxxxxxxxxx:
JennyObray:xxxxxxxxxxxxx:
JongWookChoi:xxxxxxxxxxxxx:
JoshBrown:xxxxxxxxxxxxx:
KarlKnight:xxxxxxxxxxxxx:
MarkMankoff:xxxxxxxxxxxxx:
NandeepN:xxxxxxxxxxxxx:
NicCutean:xxxxxxxxxxxxx:
PeterHolenstein:xxxxxxxxxxxxx:
PraveenKoorse:xxxxxxxxxxxxx:
RyanBAnderson:xxxxxxxxxxxxx:
SandeepMR:xxxxxxxxxxxxx:
StephenCantoria:xxxxxxxxxxxxx:
StevenLin:xxxxxxxxxxxxx:
ThomasDaly:xxxxxxxxxxxxx:
VarunPabrai:xxxxxxxxxxxxx:
YanHey:xxxxxxxxxxxxx:

re-opening.

-- PTh

Peter, I'm sorry but I honestly don't know what you expect anyone to do about this. I fully accept that you saw a problem, but no-one reports being able to reproduce it. Michael was unable to identify any problem when he looked on that server - we even went so far as to reproduce the complete login environment on another server to try and reproduce a problem. AFAIK no one with ssh access to that server has tried to debug further. By any sensible definition, therefore, the problem cannot be reproduced. Yes, you have symptoms, and yes, you have a history of server crashes, but that isn't sufficient to do anything with. Lack of follow-up on this issue simply serves to underline that it can hardly be classed as "Urgent".

Even if the recent changes to the users code end up fixing the problem, we will never know, because we can't reproduce the problem. There really is no point in keeping reports open just for the sake of it. It just clutters up the bug DB, diffuses our efforts and makes TWiki look bad, and I don't think that's what you want. If you are able to demonstrate the problem occurring, then we should most certainly pursue it - but for my money, one of those server crashes resulted in corruption of the perl data structures during a write of the .htpasswd file during a password reset. But it's pure guesswork.

CC

First, a general alert: Try to avoid posting .htpasswd entries as-is, tools like "john" is quite effective at guessing (guesses 3 of the above in less than a second here), and some people have a bad habit of using the same password in many places, leaving them vulnerable when one is exposed.

Looking at the list we have

• Deleted users - GarrettA, IlayRaja, DanielBarnett, NandeepN
• 20-Jun-06 - Known problem after upgrade: AnneMarieBatrinca, JennyObraym, BrendanPattterson, BryanCoville, JongWookChoi, JoshBrown, KarlKnight, PeterHolenstein, SandeepMR, StevenLin, ThomasDaly
• 23-Feb-2007 - YanHey
• 05-Mar-2007 - NicCutean, VarunPabrai
• 06-Mar-2007 - RyanBAnderson, StephenCantoria
• 13-Mar-2007 - DongpyoHong
• 23-Mar-2007 - PraveenKoorse
• 06-Apr-2007 - MarkMankoff

The largest part of the users are either already deleted or can be related to a known (and fixed) error after the upgrade in June.

That leaves us with the last 8 users to speculate on. As there are known race conditions in the registration code (handling of the .htpasswd should be done in a synced monitor or using other means of mutual-exclusion) these are more likely to show up when the server is heavily loaded - I don't know if the 5th and 6th of March were especially hard hit to this end - but if there are any historical info on the twiki.org load this might be worth looking into.

The easiest way to provoke race conditions that I know of is to stress the code, so a test case would be: Try to reset the password of a number of users in parallel and run the testcase in a timeslot when twiki.org is heavily loaded. Remember to back up .htpasswd first

BTW: Some of the code relating to this has been refactored in the meantime (Item3838, Item3812). I agree this is not urgent.

-- TWiki:Main.SteffenPoulsen - 09 Apr 2007

I obfuscated the encryoted passwords with x'es, not sure why I did not think about doing this in the first place.

And I reprioritized this to normal.

We should try hard to raise the quality of TWiki, not lower. Closing a bug just because it is not convenient to debug does not really help. What I think should be done is to try to reproduce the bug with simultaneous accesses, lots of manual testing, and code review.

-- TWiki:Main.PeterThoeny - 09 Apr 2007

That's fine, but leaving a report that cannot be reproduced open doesn't help either. I have already spent a lot of time trying to reproduce race conditions in the password code, and have so far failed. I am not going to waste any more time on it. You will be the first to admit that t.o. is not a "standard TWiki install", and I can't rule out the possibility that something you have done on t.o. (such as splitting apart the TWikiUsers file, or perhaps the Apache config) is causing problems. The user management code has just been significantly simplified and cleaned up, but I still can't guarantee that this problem is fixed because it can't even be reproduced on the code it was reported against. Leaving an non-reproducible bug open is not SMART - we have enough other specific problems for people to be focusing on without distracting them with nebulous seen-once issues, however real they may have been at the time.

I have to set it "no action required" because all reasonable actions have already been taken to try and resolve it. Please feel free to re-open this as and when you have been able to demonstrate it reproducibly, preferably in clean-room conditions.

CC

Please do not close real bugs. Sven and Micha have access to twiki.org, so there are several people who can debug this.

-- TWiki:Main.PeterThoeny - 07 May 2007

Peter, do you ever bother to read what I write? I made it very clear under what conditions this can be regarded as a bug i.e. when you, or anyone else, are able to demonstrably reproduce it. otherwise it is impossible to fix. It is totally soul-destroying to have you continually re-opening issues where you are the only person who can demonstrate the problem, but you steadfastly refuse to help do so.

OK, since you seem determined, I will allow this to remain in a "Waiting for Feedback" state for another 30 days to give you, Sven and Michael a chance to reproduce the problem on TWiki.org, or whatever other public platform you think can be used to demonstrate it.

CC

RE: "you steadfastly refuse to help". I am not going to reply to personal attacks. I disagree with your statement, I already spend over 40 hours a week on open source TWiki. (I have a backlog on TWiki.org gardening work because I was out of town last week, also this was help the open source TWiki.)

-- TWiki:Main.PeterThoeny - 09 May 2007

Please note that we can't upgrade twiki.org to 4.2 because of this issue.

-- TWiki:Main.PeterThoeny - 10 May 2007

Sorry I missed the release of 4.2....

Seriously, though, what I said wasn't a personal attack; it was an observation. I don't doubt your contribution to open source TWiki, nor do you doubt mine, I believe. However, apart from a fifteen-minute debug session from Micha, I have had no help in tracking this down this issue beyond what is written in this topic. There have been many changes in the user mapper code, and it would be easy for me to say "this is fixed", because the mapper no longer depends on TWikiUsers in the way it did in 4.1.2 (or 4.1.1 for than matter). However it's still guesswork that splitting the users topic is the cause of the problem, and until it has been reproduced and isolated, it will remain guesswork.

-- TWiki:Main.CrawfordCurrie - 16 May 2007

I will be looking into this during the bug fixing stage of 4.2

-- SvenDowideit - 21 May 2007

ok, this bug should now be gone - I've re-done TWikiUserMapping to ignore the TWikiUsers topic when AllowLoginName == false (as on twiki.org), and made a few more unit tests - though i've just thought of a few more

-- SvenDowideit - 01 Jun 2007

Cleaned "WaitingFor" field.

-- TWiki:Main.GilmarSantosJr - 10 Aug 2008