• Do not register here on develop.twiki.org, login with your twiki.org account.
• Use View topic Item7848 for generic doc work for TWiki-6.1.1. Use View topic Item7851 for doc work on extensions that are not part of a release. More... Close
• Anything you create or change in standard webs (Main, TWiki, Sandbox etc) will be automatically reverted on every SVN update.
Does this site look broken?. Use the LitterTray web for test cases.

I did this speed analysis to see why i18n slows down Twiki4 so much.

Skin Plugins i18n Additional Page load starts Page fully loaded
Pattern 18 All - 2.61 3
Pattern None All - 2.38 2.7
Classic None All - 1.08 1.33
Pattern None No - 1.06 1.40
Classic None No - 0.85 1.10
Pattern None All Removed Language Selector 1.31 1.65
Pattern None All Disabled %LANGUAGES% in LanguageSelector 1.31 1.55
Pattern None All I18N::available_languages
Commented out foreach my $file ( @all ) block
1.03 1.31
Pattern None All I18N::available_languages
Commented out all except return
1.03 1.28

Conclusions.

  • The code that generates %LANGUAGES% takes longer to run than the code that renders the rest of a TWiki page.
  • It is I18N::available_languages that is the sinner


wbniv@ultiwiki:~/twiki/TWikiRelease04x00/lib/TWiki$ svn diff I18N.pm
Index: I18N.pm
===================================================================
--- I18N.pm     (revision 9488)
+++ I18N.pm     (working copy)
@@ -50,16 +50,10 @@

     my @available ;

-    if ( opendir( DIR, $TWiki::cfg{LocalesDir} ) ) {
-        my @all = grep { m/^(.*)\.po$/ } (readdir( DIR ));
-        foreach my $file ( @all ) {
-            $file =~ m/^(.*)\.po$/;
-            my $lang = $1;
-            if ($TWiki::cfg{Languages}{$lang}{Enabled}) {
-                push(@available, _normalize_language_tag($lang));
-            }
+    while ( my ( $langCode, $langOptions ) = each %{$TWiki::cfg{Languages}} ) {
+        if ( $langOptions->{Enabled} ) {
+            push(@available, _normalize_language_tag($langCode));
         }
-        closedir( DIR );
     }

     return @available;

[17:51] <wbniv> that eliminates the disk access, though i'm not sure i'm seeing a noticeable difference...
[17:53] <terceiro> wbniv: that would work, I think
[17:53] <wbniv> yes, it does work, i've already tested it :)
[17:53] <wbniv> i also tried adding bogus languages like 'xx'
[17:53] <wbniv> which didn't adversely affect anything
[17:53] <wbniv> which leads me to conclude it's pointless to have to check the disk to see what's there
[17:54] <wbniv> something else later on must deal with it correctly if it's not there
[17:54] <wbniv> BUT
[17:54] <wbniv> i'm not sure the above change actually made any difference in speed
[17:54] <terceiro> I think that avoiding disk access is always an enhancement
[17:55] <wbniv> you'd think, wouldn't you? :)
[17:56] <wbniv> guess it's just too small to notice
[17:56] <wbniv> anyway,
[17:57] <Soronthar> wbniv: the disk access is not the problem.
[17:57] <wbniv> nope
[17:57] <wbniv> it's not
[17:57] <wbniv> but that's where we started :)
[17:57] <Soronthar> :)
[17:57] <wbniv> ok, so, what _is_ the problem ?
[17:58] <Soronthar> the problem is in enabled_languages
[17:58] <Soronthar> foreach my $tag ( available_languages() ) {
[17:58] <Soronthar>         my $h = TWiki::I18N->get_handle($tag);
[17:58] <Soronthar>         my $name = $h->maketext("_language_name");
[17:58] <Soronthar>         $name = $this->toSiteCharSet($name);
[17:58] <Soronthar>         $this->_add_language($tag, $name);
[17:58] <Soronthar>     }
[17:58] <Soronthar> the line my $h = TWiki::I18N->get_handle($tag); is forcing the load of all the po files
[17:58] <Soronthar> and that's a known performance problem.
[17:59] <Soronthar> we need a way to "cache" the names of the languages
[17:59] <Soronthar> whithout reading all the po files
[17:59] <wbniv> ok, so that's all
[17:59] <wbniv> it's just the mapping from xx to name?
[17:59] <Soronthar> yep
[18:03] <wbniv> Soronthar: ok, sounds pretty straightforward then, actually.  just have configure write out the names as part of lang, another property like Enabled
[18:05] <Soronthar> yep
[18:10] <wbniv> hm, now that i think more about that, i'm not crazy about that solution, because it requires running configure
[18:10] <wbniv> i like to automate my installation
[18:10] <wbniv> ahhh, however,
[18:10] <wbniv> where that code is now is where the new code can be, too
[18:10] <wbniv> (well, mostly the same code)
[18:11] <wbniv> just there, if it's not in LocalSite.cfg, write the language name then
[18:11] <wbniv> so, it does happen during "runtime", but only once per language (ever)


I patched the I18N module to cache the names of the languages. These are the numbers in AthenMarks. The TWiki4 installation has no plugins, i18n enabled, and all the languages enabled.

Release Skin Plugins Runs Time per run AthensMarks
athens     11 0.466363636363636 100
cairo pattern   5 1.828 25.5122339367416
cairo classic   5 1.822 25.5962478794531
TWikiRelease04x00 (w/o patch) pattern   5 2.112 22.5120523415978
TWikiRelease04x00 pattern   5 1.518 30.7222421846928
TWikiRelease04x00 classic   5 1.512 30.8441558441558

up in r9526

-- RA

Hmm. Yet another report about Dakar being faster than Cairo which is obviously not correct. You can in fact "feel" that Dakar is slower just by browsing.

I have put below an updated table with all the numbers and I am happy to say

I even did my own Kenneth-Athensmarks based on the Ethereal numbers.

Skin Plugins i18n Additional Page load starts Page fully loaded Athensmarks
Athens Benchmark
Classic None None Some rendering errors 0.31 0.54 100
Cairo Benchmark
Pattern 1 * None - 0.67 0.90 46.2686567164179
Pattern 18 + 1 None - 0.99 1.20 31.3131313131313
TWiki 4.0.2
Pattern 18 All Before SVN 9526 2.61 3 11.8773946360153
Pattern None All Before SVN 9526 2.38 2.7 13.0252100840336
Classic None All Before SVN 9526 1.08 1.33 28.7037037037037
Pattern None No Before SVN 9526 1.06 1.40 29.2452830188679
Classic None No Before SVN 9526 0.85 1.10 36.4705882352941
Pattern None All Before SVN 9526
Removed Language Selector
1.31 1.65 23.6641221374046
Pattern None All Before SVN 9526
Disabled %LANGUAGES% in LanguageSelector
1.31 1.55 23.6641221374046
Pattern None All Before SVN 9526
I18N::available_languages
Commented out foreach my $file ( @all ) block
1.03 1.31 30.0970873786408
Pattern None All Before SVN 9526
I18N::available_languages
Commented out all except return
1.03 1.28 30.0970873786408
Pattern 18 All After SVN 9526 1.64 1.91 18.9024390243902
Pattern None All After SVN 9526 1.30 1.66 23.8461538461538
Classic None All After SVN 9526 1.08 1.36 28.7037037037037

  • In Cairo SessionPlugin is enabled to compare with TWiki4 which also runs with sessions (to compare apples with apples)
  • Athensmarks are calculated based on 'Page load starts' to measure mainly the code execution. But the true benchmark would be to look at the page fully loaded numbers.

Conclusions.

  • SVN 9526 really works. Great job.
  • The benchmark tool and its athensmarks needs fixing.

I have copied the results and more comments to TWiki:Codev.DakarPerformanceIssues

But again. Great great job Rafael.

KJL

Perhaps I should have stated more clearly: The twiki4 times are WITHOUT any plugin, while the Cairo times are with plugins (I just used my default benchmark config).

That is, TWiki4 without plugins is a little bit faster than Cairo with plugins. That's not surprising given the big overhead of plugin discovery in Cairo.

What I really wanted to measure by comparing with Cairo is the difference between Classic and Pattern. In both Cairo and TWiki4, the difference is not significative (from the Core point of view).

It would be interesting to have a tool that can measure "online AthenMarks" automatically and can be run from the cli.

btw, commited to DEVELOP in r9533.


RA

Closed with release of 4.0.2

KJL

ItemTemplate
Summary LANGUAGES variable very slow
ReportedBy TWiki:Main.KennethLavrsen
Codebase

SVN Range Thu, 23 Mar 2006 build 9479
AppliesTo Engine
Component

Priority Urgent
CurrentState Closed
WaitingFor

Checkins 11161 11163 9526
TargetRelease patch
Edit | Attach | Watch | Print version | History: r11 < r10 < r9 < r8 < r7 | Backlinks | Raw View | Raw edit | More topic actions
Topic revision: r11 - 2006-04-01 - KennethLavrsen
 
This site is powered by the TWiki collaboration platform Powered by PerlCopyright © 2008-2018 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback