I did this speed analysis to see why i18n slows down Twiki4 so much.

Skin	Plugins	i18n	Additional	Page load starts	Page fully loaded
Pattern	18	All	-	2.61	3
Pattern	None	All	-	2.38	2.7
Classic	None	All	-	1.08	1.33
Pattern	None	No	-	1.06	1.40
Classic	None	No	-	0.85	1.10
Pattern	None	All	Removed Language Selector	1.31	1.65
Pattern	None	All	Disabled %LANGUAGES% in LanguageSelector	1.31	1.55
Pattern	None	All	I18N::available_languages Commented out foreach my $file ( @all ) block	1.03	1.31
Pattern	None	All	I18N::available_languages Commented out all except return	1.03	1.28

Conclusions.

• The code that generates %LANGUAGES% takes longer to run than the code that renders the rest of a TWiki page.
• It is I18N::available_languages that is the sinner

---

wbniv@ultiwiki:~/twiki/TWikiRelease04x00/lib/TWiki$ svn diff I18N.pm
Index: I18N.pm
===================================================================
--- I18N.pm     (revision 9488)
+++ I18N.pm     (working copy)
@@ -50,16 +50,10 @@

     my @available ;

-    if ( opendir( DIR, $TWiki::cfg{LocalesDir} ) ) {
-        my @all = grep { m/^(.*)\.po$/ } (readdir( DIR ));
-        foreach my $file ( @all ) {
-            $file =~ m/^(.*)\.po$/;
-            my $lang = $1;
-            if ($TWiki::cfg{Languages}{$lang}{Enabled}) {
-                push(@available, _normalize_language_tag($lang));
-            }
+    while ( my ( $langCode, $langOptions ) = each %{$TWiki::cfg{Languages}} ) {
+        if ( $langOptions->{Enabled} ) {
+            push(@available, _normalize_language_tag($langCode));
         }
-        closedir( DIR );
     }

     return @available;

[17:51] <wbniv> that eliminates the disk access, though i'm not sure i'm seeing a noticeable difference...
[17:53] <terceiro> wbniv: that would work, I think
[17:53] <wbniv> yes, it does work, i've already tested it :)
[17:53] <wbniv> i also tried adding bogus languages like 'xx'
[17:53] <wbniv> which didn't adversely affect anything
[17:53] <wbniv> which leads me to conclude it's pointless to have to check the disk to see what's there
[17:54] <wbniv> something else later on must deal with it correctly if it's not there
[17:54] <wbniv> BUT
[17:54] <wbniv> i'm not sure the above change actually made any difference in speed
[17:54] <terceiro> I think that avoiding disk access is always an enhancement
[17:55] <wbniv> you'd think, wouldn't you? :)
[17:56] <wbniv> guess it's just too small to notice
[17:56] <wbniv> anyway,
[17:57] <Soronthar> wbniv: the disk access is not the problem.
[17:57] <wbniv> nope
[17:57] <wbniv> it's not
[17:57] <wbniv> but that's where we started :)
[17:57] <Soronthar> :)
[17:57] <wbniv> ok, so, what _is_ the problem ?
[17:58] <Soronthar> the problem is in enabled_languages
[17:58] <Soronthar> foreach my $tag ( available_languages() ) {
[17:58] <Soronthar>         my $h = TWiki::I18N->get_handle($tag);
[17:58] <Soronthar>         my $name = $h->maketext("_language_name");
[17:58] <Soronthar>         $name = $this->toSiteCharSet($name);
[17:58] <Soronthar>         $this->_add_language($tag, $name);
[17:58] <Soronthar>     }
[17:58] <Soronthar> the line my $h = TWiki::I18N->get_handle($tag); is forcing the load of all the po files
[17:58] <Soronthar> and that's a known performance problem.
[17:59] <Soronthar> we need a way to "cache" the names of the languages
[17:59] <Soronthar> whithout reading all the po files
[17:59] <wbniv> ok, so that's all
[17:59] <wbniv> it's just the mapping from xx to name?
[17:59] <Soronthar> yep
[18:03] <wbniv> Soronthar: ok, sounds pretty straightforward then, actually.  just have configure write out the names as part of lang, another property like Enabled
[18:05] <Soronthar> yep
[18:10] <wbniv> hm, now that i think more about that, i'm not crazy about that solution, because it requires running configure
[18:10] <wbniv> i like to automate my installation
[18:10] <wbniv> ahhh, however,
[18:10] <wbniv> where that code is now is where the new code can be, too
[18:10] <wbniv> (well, mostly the same code)
[18:11] <wbniv> just there, if it's not in LocalSite.cfg, write the language name then
[18:11] <wbniv> so, it does happen during "runtime", but only once per language (ever)

---

I patched the I18N module to cache the names of the languages. These are the numbers in AthenMarks. The TWiki4 installation has no plugins, i18n enabled, and all the languages enabled.

Release	Skin	Plugins	Runs	Time per run	AthensMarks
athens			11	0.466363636363636	100
cairo	pattern		5	1.828	25.5122339367416
cairo	classic		5	1.822	25.5962478794531
TWikiRelease04x00 (w/o patch)	pattern		5	2.112	22.5120523415978
TWikiRelease04x00	pattern		5	1.518	30.7222421846928
TWikiRelease04x00	classic		5	1.512	30.8441558441558

up in r9526

-- RA

Hmm. Yet another report about Dakar being faster than Cairo which is obviously not correct. You can in fact "feel" that Dakar is slower just by browsing.

I have put below an updated table with all the numbers and I am happy to say

I even did my own Kenneth-Athensmarks based on the Ethereal numbers.

Skin	Plugins	i18n	Additional	Page load starts	Page fully loaded	Athensmarks
Athens Benchmark
Classic	None	None	Some rendering errors	0.31	0.54	100
Cairo Benchmark
Pattern	1 *	None	-	0.67	0.90	46.2686567164179
Pattern	18 + 1	None	-	0.99	1.20	31.3131313131313
TWiki 4.0.2
Pattern	18	All	Before SVN 9526	2.61	3	11.8773946360153
Pattern	None	All	Before SVN 9526	2.38	2.7	13.0252100840336
Classic	None	All	Before SVN 9526	1.08	1.33	28.7037037037037
Pattern	None	No	Before SVN 9526	1.06	1.40	29.2452830188679
Classic	None	No	Before SVN 9526	0.85	1.10	36.4705882352941
Pattern	None	All	Before SVN 9526 Removed Language Selector	1.31	1.65	23.6641221374046
Pattern	None	All	Before SVN 9526 Disabled %LANGUAGES% in LanguageSelector	1.31	1.55	23.6641221374046
Pattern	None	All	Before SVN 9526 I18N::available_languages Commented out foreach my $file ( @all ) block	1.03	1.31	30.0970873786408
Pattern	None	All	Before SVN 9526 I18N::available_languages Commented out all except return	1.03	1.28	30.0970873786408
Pattern	18	All	After SVN 9526	1.64	1.91	18.9024390243902
Pattern	None	All	After SVN 9526	1.30	1.66	23.8461538461538
Classic	None	All	After SVN 9526	1.08	1.36	28.7037037037037

• In Cairo SessionPlugin is enabled to compare with TWiki4 which also runs with sessions (to compare apples with apples)
• Athensmarks are calculated based on 'Page load starts' to measure mainly the code execution. But the true benchmark would be to look at the page fully loaded numbers.

Conclusions.

• SVN 9526 really works. Great job.
• The benchmark tool and its athensmarks needs fixing.

I have copied the results and more comments to TWiki:Codev.DakarPerformanceIssues

But again. Great great job Rafael.

KJL

Perhaps I should have stated more clearly: The twiki4 times are WITHOUT any plugin, while the Cairo times are with plugins (I just used my default benchmark config).

That is, TWiki4 without plugins is a little bit faster than Cairo with plugins. That's not surprising given the big overhead of plugin discovery in Cairo.

What I really wanted to measure by comparing with Cairo is the difference between Classic and Pattern. In both Cairo and TWiki4, the difference is not significative (from the Core point of view).

It would be interesting to have a tool that can measure "online AthenMarks" automatically and can be run from the cli.

btw, commited to DEVELOP in r9533.

---

RA

Closed with release of 4.0.2

KJL