From: Timo Tijhof <krinklemail@gmail.com>
Date: Wed, 4 Sep 2019 20:35:10 +0000 (+0100)
Subject: localisation: Release data from memory in LCStoreStaticArray::finishWrite
X-Git-Tag: 1.34.0-rc.0~390^2
X-Git-Url: http://git.cyclocoop.org/?a=commitdiff_plain;h=55db848b773c449cb481feed31e105ae149c1c6a;p=lhc%2Fweb%2Fwiklou.git

localisation: Release data from memory in LCStoreStaticArray::finishWrite

With this change, the memory behaviour of LCStoreStaticArray
matches the other LCStore implementations. Specifically, that when
mass-rebuilding LocalisationCache entries for all language codes,
the computed data should be released from memory after
calling LCStore::finishWrite().

This doesn't affect user-facing web requests, even in the case
of stock MW where every once in a while a user request can lazy-
regenerate the LCStore, there is a process-cache in front of LCStore
in the LocalisationCache class.

The rebuildLocalisationCache.php clears that via
LocalisationCacheBulkLoad::unload(), but due to LCStoreStaticArray
internally holding on to the data, it was still leaking.

The leak was found by @Nikerabbit as part of testing for T218207.

To test this, amend rebuildLocalisationCache.php and add the
following on line 161, as the first line of the doRebuild/foreach/if
block:

  echo "[$code-start-mem] " . round(memory_get_usage(true)/1024/1024, 2) . " MB\n";

If you then have LocalSettings.php configured like so:

  $wgCacheDirectory = $wgTmpDirectory;
  $wgLocalisationCacheConf['store'] = 'array';

Then before this patch, running rebuildLocalisationCache.php,
shows memory starting at 12 MB and growing 2-3 MB for every language
until the very end, closing with 970 MB memory use.

After this patch, it starts at 12 MB and stops growing at 32 MB.

When configuring as `['store'] = 'files'`, which uses LCStoreCDB,
the memory starts at 12 MB and stops growing at 44 MB, both before
and after this patch.

Bug: T218207
Change-Id: I0d215efee5b31766776a068b16811d52f9879312
---

diff --git a/includes/cache/localisation/LCStoreStaticArray.php b/includes/cache/localisation/LCStoreStaticArray.php
index 59116561c0..53893bdbe4 100644
--- a/includes/cache/localisation/LCStoreStaticArray.php
+++ b/includes/cache/localisation/LCStoreStaticArray.php
@@ -121,6 +121,8 @@ class LCStoreStaticArray implements LCStore {
 			'Generated by LCStoreStaticArray.php -- do not edit!'
 		);
 		file_put_contents( $this->fname, $out );
+		// Release the data to manage the memory in rebuildLocalisationCache
+		unset( $this->data[$this->currentLang] );
 		$this->currentLang = null;
 		$this->fname = null;
 	}