Timo Tijhof [Fri, 31 Aug 2018 19:44:17 +0000 (20:44 +0100)]
mediawiki.user: Fix missing array initialization in generateRandomSessionId
Array was not properly initialized and thus browsers
that do not support Crypto API where displaying an error
on console.
The tests failed to catch this because assigning window.crypto
to `undefined` does not work (it is a read-only property). This
"fallback" test was actually testing the regular Crypto-based path
a second time.
Bug: T203275
Co-Authored-By: Timo Tijhof <krinklemail@gmail.com>
Change-Id: I8feecddf0878a739e560085f7897ebc3d8100c02
jenkins-bot [Fri, 31 Aug 2018 19:14:13 +0000 (19:14 +0000)]
Merge "Change @return-taint to use onlysafefor_html instad of escapes_html"
jenkins-bot [Fri, 31 Aug 2018 18:53:44 +0000 (18:53 +0000)]
Merge "Use annotations for taint in Parser & ParserOutput."
jenkins-bot [Fri, 31 Aug 2018 17:39:51 +0000 (17:39 +0000)]
Merge "Title: Fix isRawHtmlMessage() for messages with underscores"
Kunal Mehta [Tue, 28 Aug 2018 19:47:49 +0000 (12:47 -0700)]
Title: Fix isRawHtmlMessage() for messages with underscores
Title::getRootText() uses the text form (spaces) of the title, while
$wgRawHtmlMessages was specifying them in dbkey form (underscores).
And add tests while we're at it. Which spotted that the existing
code didn't work. Whoops. Fixed.
Change-Id: I05eea553c588e0f99f862e07ad15386507ed0728
jenkins-bot [Fri, 31 Aug 2018 16:41:15 +0000 (16:41 +0000)]
Merge "resourceloader: Use 'this' to access the mw.loader.store internally"
jenkins-bot [Fri, 31 Aug 2018 15:56:41 +0000 (15:56 +0000)]
Merge "resourceloader: Remove redundant '!!' from startup.js"
Brian Wolff [Fri, 31 Aug 2018 15:55:44 +0000 (15:55 +0000)]
Use annotations for taint in Parser & ParserOutput.
This replaces the builtin taints that are removed in
Ic1e1983a51c. Additionally, parse will no longer warn about
double escaping - there's many situations where such warnings
are wrong (e.g. Using Html::rawElement()). However this also
means that Parser::parse( wfMessage( 'foo' )->parse() ); will
no longer give a double escaping warning, which is unfortunate.
Bug: T202380
Change-Id: Ia52d37411beb62b112c6ff102438063c3d750769
jenkins-bot [Fri, 31 Aug 2018 12:44:54 +0000 (12:44 +0000)]
Merge "Minor cleanup in backup test cases"
jenkins-bot [Fri, 31 Aug 2018 11:38:15 +0000 (11:38 +0000)]
Merge "Make HTML generation in RenderedRevision optional"
jenkins-bot [Fri, 31 Aug 2018 11:38:10 +0000 (11:38 +0000)]
Merge "Add test for {{subst:REVISIONUSER}}"
jenkins-bot [Fri, 31 Aug 2018 11:25:15 +0000 (11:25 +0000)]
Merge "[MCR] Introduce RevisionRenderer"
daniel [Fri, 31 Aug 2018 10:49:19 +0000 (12:49 +0200)]
Add test for {{subst:REVISIONUSER}}
This tests that revision meta-data is available for Pre-Save Transform.
Change-Id: I62f73ea24784b539cdf8229aeb1f8efa62631248
daniel [Mon, 13 Aug 2018 20:33:31 +0000 (22:33 +0200)]
Make HTML generation in RenderedRevision optional
This allows optimization for situations in which a caller
needs the meta-data of a ParserOutput, and the respective
ContentHandler can provide that meta-data without generating
HTML output.
Bug: T194048
Change-Id: I786d294d18a6a2e3cea61577313e21b578c44f1e
Brian Wolff [Fri, 31 Aug 2018 09:47:24 +0000 (09:47 +0000)]
Change @return-taint to use onlysafefor_html instad of escapes_html
This prevents some double escaped warnings. Requires
I2f4e33656b9f94 to be effective. Follow up
faf2e14517b05f8.
Change-Id: I255c96592f3baff2df34e07c81510c8874908e28
Bug: T202797
Kunal Mehta [Fri, 31 Aug 2018 04:46:10 +0000 (21:46 -0700)]
Set @param-taint for Parser::internalParse()
This is not strictly accurate, because Parser::internalParse() actually
returns half-parsed HTML, which is not safe for output. But it is safe for
output from a parser tag.
Maybe phan-taint-check plugin needs to learn about half-parsed HTML as an
extra taint type, and make that an acceptable thing for parser tags to return,
but not other things.
But this fixes the failures for the Listings extension, so I think it's
worthwhile in the meantime.
Change-Id: Idf87f5c3dcf81dd210de73a4ff15e3b1aabd9f89
daniel [Fri, 31 Aug 2018 04:30:22 +0000 (14:30 +1000)]
Minor cleanup in backup test cases
Change-Id: Iab2ad5a19b32cd32c2ea9c9dd0b589428056c86d
Timo Tijhof [Fri, 31 Aug 2018 02:21:25 +0000 (03:21 +0100)]
resourceloader: Use 'this' to access the mw.loader.store internally
Shorter and more intuitive. All of these functions are always
called as methods on the mw.loader.store objects, not detached.
Change-Id: If26851eac1530f023228897392c5067c6e8927af
Timo Tijhof [Fri, 31 Aug 2018 01:50:40 +0000 (02:50 +0100)]
resourceloader: Remove redundant '!!' from startup.js
The outer expression already casts the result to a boolean.
Unit tests in startup.test.js also strictly assert that the
returned values are boolean.
Change-Id: I5709fcd0184b99d289b9cdfeccf8afa960806d59
jenkins-bot [Fri, 31 Aug 2018 00:48:34 +0000 (00:48 +0000)]
Merge "Allow tests to run with a non-writable source tree"
jenkins-bot [Thu, 30 Aug 2018 21:36:47 +0000 (21:36 +0000)]
Merge "resourceloader: Remove selective build optimisation from getModuleContent()"
Timo Tijhof [Thu, 30 Aug 2018 01:42:24 +0000 (02:42 +0100)]
resourceloader: Remove selective build optimisation from getModuleContent()
This follows
5ddd7f91c7, which factored out response building
from ResourceLoader.php to ResourceLoaderModule::buildContent.
As optimisation, I made this method only return the array keys
needed for the current response; based $context->getOnly().
The reason for this refactoring was the creation of the
'enableModuleContentVersion' option to getVersionHash(), which
would use this method to create a module response, and hash it.
During the implementation of that option, I ran into a problem.
getVersionHash() is called by the startup module for each
registered module, to create the manifest. The context for the
StartupModule request itself has "only=scripts". But, we must
still compute the version hashes for whole modules, not just
their scripts.
I worked around that problem in
aac831f9fa by creating a mock
context in getVersionHash() that stubs out the 'only' parameter.
This worked, but made the assumption that the scripts and styles
of a module cannot differ based on the 'only' parameter.
This assumption was wrong, because the 'only' parameter is part
of ResourceLoaderContext and available to all getters to vary on.
Fortunately, the 'enableModuleContentVersion' option is off by
default and nobody currently using it was differing its output
by the 'only' parameter.
I intend to make use of the 'enableModuleContentVersion' option
in StartupModule to fix T201686. And StartupModule outputs a
manifest if the request specifies only=scripts, and outputs
a warning otherwise. As such, it cannot compute its version
if the 'only' parameter is stubbed out.
* Remove the 'only' parameter stubbing.
* Remove the selective building from the buildContent() method.
This was not very useful because we need to build the whole
module regardless when computing the version.
As benefit, this means the in-process cache is now shared between
the call from getVersionHash and the call from makeModuleResponse.
Bug: T201686
Change-Id: I8a17888f95f86ac795bc2de43086225b8a8f4b78
Translation updater bot [Thu, 30 Aug 2018 20:06:08 +0000 (22:06 +0200)]
Localisation updates from https://translatewiki.net.
Change-Id: I5515e9768e1b1348f2983ed0fa8d6475f49110da
daniel [Tue, 7 Aug 2018 16:52:40 +0000 (18:52 +0200)]
[MCR] Introduce RevisionRenderer
RevisionRenderer is the MCR replacement for Content::getParserOutput,
as outlined in <https://www.mediawiki.org/wiki/User:Daniel_Kinzler_(WMDE)/MCR-PageUpdater>.
Note: This change also introduces quite a bit of code for
merging ParserOutput objects.
Bug: T194048
Change-Id: I871978bf79f67c9e7954fb3fc8528d6e365f2cc1
jenkins-bot [Thu, 30 Aug 2018 15:41:41 +0000 (15:41 +0000)]
Merge "Selenium: selenium-daily NPM script"
Kunal Mehta [Thu, 30 Aug 2018 05:08:32 +0000 (22:08 -0700)]
Linker: Add @return-taint for formatLinksInComment()
Works around a false positive in the phan-taint-check-plugin.
Bug: T202797
Change-Id: If7c9e729ca7624b3f791fe01d0b768791657277b
Kunal Mehta [Thu, 30 Aug 2018 05:06:39 +0000 (22:06 -0700)]
Document expected input and return value for Language::convert()
Bug: T202571
Change-Id: I1598f8a83d9cb2ab9d9e9ba96acd90f70edd59ad
jenkins-bot [Thu, 30 Aug 2018 02:54:03 +0000 (02:54 +0000)]
Merge "Fix some warnings from phan-taint-check"
jenkins-bot [Thu, 30 Aug 2018 02:43:59 +0000 (02:43 +0000)]
Merge "EditPage: Allow summary=0 in URL parameter"
jenkins-bot [Thu, 30 Aug 2018 01:01:13 +0000 (01:01 +0000)]
Merge "resourceloader: Refuse to preview content with </script>"
jenkins-bot [Thu, 30 Aug 2018 01:01:04 +0000 (01:01 +0000)]
Merge "Html: Reject </script> from inlineScript() and leave rest unescaped"
Tim Starling [Thu, 30 Aug 2018 00:48:50 +0000 (10:48 +1000)]
Allow tests to run with a non-writable source tree
It's insecure to allow apps to modify their own source, that's how file
write vulnerabilities escalate to code execution.
Change-Id: I0f79b2b7c7502405a62dcb176d8be4633ce4eda5
James D. Forrester [Wed, 29 Aug 2018 23:31:09 +0000 (16:31 -0700)]
resources: Deprecate jquery.localize, long-replaced by jquery.i18n
Bug: T202154
Change-Id: I2548880987145d41f6a0c6fa7466fb6405e1c5a1
Timo Tijhof [Mon, 20 Aug 2018 00:14:46 +0000 (01:14 +0100)]
resourceloader: Refuse to preview content with </script>
Bug: T200506
Change-Id: I4ab5fbb0f5413aad24360169ba635672ce8d9c8e
Timo Tijhof [Mon, 20 Aug 2018 00:42:15 +0000 (01:42 +0100)]
Html: Reject </script> from inlineScript() and leave rest unescaped
There are three problems with the CDATA approach:
1. It doesn't work.
HTML5 already interprets the contents of <script> tags as CDATA,
which means escaping of characters like & is not needed. In fact,
in HTML5 mode, a plain script tag with <script>0&1;</script>
would be a syntax error. Indicating it is not interpreted as
text, but as CDATA. Effectively, the only thing an HTML parser
looks for is </script>.
And that's exactly the problem. Producing an inline script
containing the characters "</string>" for legitimate reasons,
is currently broken.
No alternate wrapping or setting can make it work, either.
See also:
https://people.wikimedia.org/~krinkle/200506-html-inlinescript.html
which contains:
<script>/*<![CDATA[*/
if (true && true) {
console.log('This is a <script></script> tag (original)');
}
/*]]>*/</script>
In a browser, the script is terminated by the first "</script>",
leaving the code unfinished, throwing a SyntaxError, and outputting
the rest of the script as plain text on the page.
2. CDATA is only for XML mode, whereas MediaWiki does not support
the XML/XHTML output mode (since MediaWiki 1.22). Instead, we only
output HTML (5). Code that does need to produce XML, should use the
class from Xml.php instead.
3. It gives a false sense of security.
We could just remove the CDATA code as-is and that in itself would be an
improvement per point 2 and 3, and would break nothing per point 1.
However, this commit attempts to address the underlying bug by rejecting
the characters "</script>" from input. If this is needed in a literal,
it is the responsibility of the caller to escape it in a way that is
appropiate for how it is used (string, comment, regex, etc.).
There are two ways this can be used currently in core:
* User input as exported through JSON (e.g. mw.config, or mw.messages).
This is already fine as both FormatJson::encode and json_encode handle
escape either < or / in the string by default already.
* Previews of edits to user scripts. This is currently already broken and
causes the script to end early and produce arbitrary HTML on the page.
This commit limits the impact by refusing to output such script in a
broken way. I will further address that use case in a follow-up.
Bug: T200506
Change-Id: I67ceb34eabf2f62fd3f3841b8f1459289fad28fb
jenkins-bot [Wed, 29 Aug 2018 21:49:35 +0000 (21:49 +0000)]
Merge "jobqueue: Use explicit retry when refreshLinks can't get a lock"
jenkins-bot [Wed, 29 Aug 2018 21:49:29 +0000 (21:49 +0000)]
Merge "Add code to read from ct_tag_id in ChangeTags"
jenkins-bot [Wed, 29 Aug 2018 20:42:15 +0000 (20:42 +0000)]
Merge "HTMLForm: Deprecate parameters 'notice', 'notice-messages', 'notice-message'"
jenkins-bot [Wed, 29 Aug 2018 20:37:32 +0000 (20:37 +0000)]
Merge "sitemaps: absolute URL for sitemaps"
Ian Marlier [Wed, 29 Aug 2018 18:02:23 +0000 (14:02 -0400)]
sitemaps: absolute URL for sitemaps
Google, at least, considers sitemap indexes that provide relative URLs
as being broken.
Bug: T202321
Change-Id: I5509be4b165eea9eca36e3f4975f87285ef87911
Bartosz Dziewoński [Fri, 17 Aug 2018 21:04:57 +0000 (23:04 +0200)]
HTMLForm: Deprecate parameters 'notice', 'notice-messages', 'notice-message'
Bug: T197179
Change-Id: I603436e0720fdc0f08f35f3c0630b79865a9c82a
Translation updater bot [Wed, 29 Aug 2018 19:56:11 +0000 (21:56 +0200)]
Localisation updates from https://translatewiki.net.
Change-Id: I524cbcfe3d8d65c89ba38d60f7320304316ceede
Amir Sarabadani [Sat, 11 Aug 2018 18:34:27 +0000 (20:34 +0200)]
Add code to read from ct_tag_id in ChangeTags
Bug: T194162
Change-Id: I6c9e0c94cdd46fe46ccaf7feb78889f4ab5995f2
jenkins-bot [Wed, 29 Aug 2018 16:25:22 +0000 (16:25 +0000)]
Merge "Apply content wrapping in ParserOutput::getText()"
jenkins-bot [Wed, 29 Aug 2018 15:29:21 +0000 (15:29 +0000)]
Merge "Add tests for article viewing"
daniel [Tue, 28 Aug 2018 16:48:10 +0000 (18:48 +0200)]
Apply content wrapping in ParserOutput::getText()
Instead of applying wrapping the the parser and unwrapping in
ParserOutput::getText(), turn this around and apply wrapping in getText(),
and only if desired.
This avoids search&replace logic for unwrapping, and it also makes it a lot
easier to merge the output of multiple slots for MCR output.
This changes behavior in two hopefully irrelevant ways:
1) the limit report comments will be inside the wrapper div, instead of
following it.
2) if HTML with a wrapper div is explicitly injected into a ParserOutput
object, it will not be possible to unwrap the text.
Bug: T174035
Change-Id: I1641b7995af9bd297f1acd610d583fbf874f34e0
jenkins-bot [Wed, 29 Aug 2018 13:07:00 +0000 (13:07 +0000)]
Merge "Use "break" instead of "continue""
Željko Filipin [Wed, 29 Aug 2018 12:58:04 +0000 (14:58 +0200)]
Selenium: selenium-daily NPM script
selenium-daily just calls selenium-test. It's needed for daily Jenkins job targeting
beta cluster. The script might seem redundant, but it provides flexibility. In case
a repository does not want to run all tests daily, that's easily fixed by updating
the the script.
Bug: T188742
Change-Id: Idf86f94cc31abda4bfcdc1ac4eba29206d9c91f9
jenkins-bot [Wed, 29 Aug 2018 10:34:52 +0000 (10:34 +0000)]
Merge "resourceloader: Remove unused static SkinModule::getLogo method"
jenkins-bot [Wed, 29 Aug 2018 10:06:07 +0000 (10:06 +0000)]
Merge "Install giorgiosironi/eris as require-dev"
jenkins-bot [Wed, 29 Aug 2018 01:22:22 +0000 (01:22 +0000)]
Merge "resourceloader: Audit use of JSON encoding and use json_encode directly"
jenkins-bot [Wed, 29 Aug 2018 01:16:33 +0000 (01:16 +0000)]
Merge "Remove support for StartProfiler.php"
Timo Tijhof [Thu, 16 Aug 2018 19:48:59 +0000 (20:48 +0100)]
resourceloader: Audit use of JSON encoding and use json_encode directly
* Remove use of MediaWiki's FormatJson class. Ref T32956.
* FormatJson::decode() was a direct alias of json_decode.
* FormatJson::encode() is a wrapper for json_encode that
sets `JSON_UNESCAPED_SLASHES | JSON_HEX_TAG | JSON_HEX_AMP`
by default, plus a boolean parameter to add JSON_PRETTY_PRINT.
Instead, use json_encode() directly. By default json_encode() escapes
slashes and non-ASCII unicode.
* Audit our uses of JSON encoding, document their needs, and set flags
as needed.
* Remove $mapToJson abstraction from ResourceLoaderStartUpModule.
* Always pretty-print the list of module names in the error message,
regardless of debug mode.
* Remove hacky indentation-adder from ResourceLoaderStartUpModule.
This was relying on there not being line breaks in the json output
outside debug mode, and adding a level of indentation to roughly
match the code in the JSON is substituted. This didn't match and
just generally doesn't seem worth it.
Change-Id: I3e09ddeb4d53c8980195c1855303c0ee25bd4c20
jenkins-bot [Tue, 28 Aug 2018 23:07:00 +0000 (23:07 +0000)]
Merge "resourceloader: Move logo preload from OutputPage to SkinModule"
Timo Tijhof [Mon, 20 Aug 2018 23:58:41 +0000 (00:58 +0100)]
resourceloader: Remove unused static SkinModule::getLogo method
This existed for internal use by OutputPage, which is no longer
the case as of I11b390f2e4f5e7db.
Also move the unit tests from OutputPageTest,
to ResourceLoaderSkinModuleTest.
Change-Id: I8b23f976f5f89b1005b387a827f75031f5c96141
jenkins-bot [Tue, 28 Aug 2018 21:55:00 +0000 (21:55 +0000)]
Merge "resourceloader: Remove checkCssHandles for modules without styles"
jenkins-bot [Tue, 28 Aug 2018 21:44:31 +0000 (21:44 +0000)]
Merge "Notifications: Note that the ID is unnecessary and deprecated"
Timo Tijhof [Tue, 28 Aug 2018 21:14:25 +0000 (22:14 +0100)]
jobqueue: Use explicit retry when refreshLinks can't get a lock
While RefreshLinksJob is de-duplicated by page-id, it is possible
for two jobs to run for the same page ID if the second one was queued
after the first one started running. In that case they the newer
one must not be skipped or ignored because it will have newer
information to record to the database, but it also has no way
to stop the old one, and we can't run them concurrently.
Instead of letting the lock exception mark the job as error,
making it implicitly retry, do this more explicitly, which avoids
logspam.
Bug: T170596
Co-Authored-By: Aaron Schulz <aschulz@wikimedia.org>
Change-Id: Id2852d73d00daf83f72cf5ff778c638083f5fc73
jenkins-bot [Tue, 28 Aug 2018 20:30:02 +0000 (20:30 +0000)]
Merge "maintenance: Implement 'file' type and use for jquery and qunitjs"
jenkins-bot [Tue, 28 Aug 2018 20:25:12 +0000 (20:25 +0000)]
Merge "maintenance: Add 'verify' action to manageForeignResources.php"
jenkins-bot [Tue, 28 Aug 2018 20:25:06 +0000 (20:25 +0000)]
Merge "registration: Short-circuit if dependency constraint is '*'"
Translation updater bot [Tue, 28 Aug 2018 19:58:30 +0000 (21:58 +0200)]
Localisation updates from https://translatewiki.net.
Change-Id: I03f7833c356f02d665983839012db274c4548b4a
Kunal Mehta [Tue, 28 Aug 2018 19:29:04 +0000 (12:29 -0700)]
registration: Short-circuit if dependency constraint is '*'
Most extensions depend upon any version of other extensions/skins since
versioning is pretty inconsistent. Since '*' is so commonly used, explicitly
short-circuit that constraint since we only need to check whether the
dependency is loaded.
Bug: T198044
Change-Id: I5526c8068d3b9a6ee5ca71ea6bdbcd693f1ffb7a
Aaron Schulz [Tue, 28 Aug 2018 17:28:45 +0000 (10:28 -0700)]
Update $wgTranscludeCacheExpiry comment
Change-Id: Ic1f20c072174732f7fd8ecab9d366125d6f0c2d2
daniel [Thu, 16 Aug 2018 15:45:10 +0000 (17:45 +0200)]
Add tests for article viewing
Bug: T174035
Change-Id: I06dc78853169812b17e0bde733d9306ccd687564
Ed Sanders [Tue, 28 Aug 2018 14:50:14 +0000 (15:50 +0100)]
Notifications: Note that the ID is unnecessary and deprecated
Change-Id: I30a3a2bd0dce6391dc075807ffbfdf6fef49d88d
jenkins-bot [Tue, 28 Aug 2018 09:47:04 +0000 (09:47 +0000)]
Merge "Improve logging in MediaWikiTestCase"
jenkins-bot [Tue, 28 Aug 2018 09:39:15 +0000 (09:39 +0000)]
Merge "Set migration stages in RevisionStoreDbTestBase"
Pablo Grass [Mon, 27 Aug 2018 11:24:34 +0000 (13:24 +0200)]
Install giorgiosironi/eris as require-dev
Eris is a porting of QuickCheck and property-based testing tools to
the PHP and PHPUnit ecosystem.
It is used in WikibaseLexeme to generate inputs for tests cases.
These tests are skipped if the library is not present.
T188354 / Iba3da2820262ca8a879780ee7848cfc831df28b5 removed the
extension-specific phpunit run from composer scripts.
Due to the unavailability of eris in core composer, and the fact that
the composer merge plugin does not install require-dev of extensions,
all eris-based tests have been skipped since (2018-02-17).
Change-Id: I3909fb34025d9a9e07ccd6c85b30c34144714d04
jenkins-bot [Tue, 28 Aug 2018 07:22:19 +0000 (07:22 +0000)]
Merge "Replace underscores with spaces in DeletedContribs form"
jenkins-bot [Tue, 28 Aug 2018 06:03:08 +0000 (06:03 +0000)]
Merge "Remove PhanUndeclaredStaticMethod from blacklist"
jenkins-bot [Tue, 28 Aug 2018 05:47:28 +0000 (05:47 +0000)]
Merge "resourceloader: Optimise several map-like objects with Object.create(null)"
Tim Starling [Tue, 28 Aug 2018 02:39:33 +0000 (12:39 +1000)]
Improve logging in MediaWikiTestCase
Log test setup, start and end to the tests-phpunit log channel. This
helps when interpreting debug logs. The channel name was chosen by
analogy with the existing tests-parser log channel.
Change-Id: I2bc87564247f2f136b2244d426fa127d21663f92
Tim Starling [Mon, 27 Aug 2018 23:39:18 +0000 (09:39 +1000)]
Set migration stages in RevisionStoreDbTestBase
I confirmed that wgCommentTableSchemaMigrationStage needs to be set, and
getActorQueryFields() implies that the actor table migration stage needs
to be set as well.
Change-Id: I153ff985c8753b291d2201ebeed10515acb0cd36
Erik Bernhardson [Fri, 8 Jun 2018 20:51:44 +0000 (13:51 -0700)]
Remove PhanUndeclaredStaticMethod from blacklist
There was a single instance of this issue which was reasonably simple
to solve. The code that triggered the issue was not wrong, but calling
the grandparent constructor directly is not the cleanest code so it
seems right to fix it. Switch revision initialization into an
overridable function to remove the need to even have a constructor in
the class.
Change-Id: Ic2af0d93e8d55e93061e3f4b5c7a4122509543f0
jenkins-bot [Tue, 28 Aug 2018 04:51:27 +0000 (04:51 +0000)]
Merge "resourceloader: Simplify addEmbeddedCSS by using object refs"
Timo Tijhof [Sun, 26 Aug 2018 03:44:10 +0000 (04:44 +0100)]
resourceloader: Remove checkCssHandles for modules without styles
When execute() begins, we are in a stack that is suitable for
script execution. In the case of styles needing to be inserted,
we end that stack by scheduling the styles insertion via
requestAnimationFrame. When that styles insertion is completed,
we are currently continuing the script execution in that same
stack, which is bad for performance and also creates a confusing
call stack, given that none of that code is paint or animation
related.
Before we can address that, e.g. by unwinding the stack via
requestIdleCallback, we first need to clean up the code to not
needlessly involve checkCssHandles for modules without styles.
Most modules are small and have only scripts, no styles, so this
is important to keep in a single stack and rapidly executed in
one batch - without interuption.
This commit moves the logic that was previously inside checkCssHandles
to the bottom of execute(), which is where checkCssHandles used to
be called from.
Bug: T202703
Change-Id: I13a996e01b48bab477e68ce933a5e2ef05b361aa
Timo Tijhof [Sun, 26 Aug 2018 03:09:02 +0000 (04:09 +0100)]
resourceloader: Simplify addEmbeddedCSS by using object refs
* Previously, the same function was used both for adding to the buffer,
and flushing the buffer (via self-calling alternate signatures).
The flushing logic was split off to a flushCssBuffer function.
* Previously, when encountering an '@import' statement, it performed a
synchronous flush, instead of the usual asynchronous ones. There was
no reason for this, other than my laziness. I suspect because I was
using strings, which can't be passed by reference, and I didn't
think of another way.
I'm now storing the string in an object, which can be passed by
reference to the flush function. This means, as before, we can
keep appending to its string after the flush is scheduled. But,
unlike before, it also means we can reset our local reference and
start a new buffer at any time and schedule that one, too.
Bug: T202703
Change-Id: Ifc6dd59e9e8885d65ba425bc579ecbfb09f2ac64
Timo Tijhof [Tue, 28 Aug 2018 03:26:28 +0000 (04:26 +0100)]
resourceloader: Don't defer discovery of critical request via rIC()
== Overview
This reverses a small part of the big
dec800968eb commit. That
commit eliminated the 2nd of 3 JS requests required on a page.
The first JS request is now modules=startup with the registry and
mw.loader. The main JS request for modules (now 2nd) is triggered
from RLQ, which fires when startup.js calls 'startUp()'.
== Regression
In commit
dec800968eb, I thought I was being smart by letting the
browser "take a breath" at the end of startup.js before calling
'startUp()'. I now believe that this artifical is a mistake.
I thought it was a good idea because, before
dec800968eb startup.js
made an http request (for jquery). During that network request the
main thread was free to do rendering. Then, when jquery came back,
we called startUp() to trigger our main request for modules (3rd,
now 2nd). While this request no longer exists, the idea was to keep
that thread yield at the end to startup.js.
I now think it was a mistake because requestIdleCallback is not
a "small" thread yield. Rather, I now find in production that in
the majority of cases (all mobile, and desktop first views) the
rIC is not called until *after* the page load event fires.
This means that during the entire page rendering time, the network
and javascript thread were sitting idle. Prioritising rendering and
making the page load complete 0.5s faster is great. But,
underutilising the network and delaying time to JS interation by 1-2
seconds, is bad.
== Cost
The startUp() function is quite light. All it does is a bunch of
pure JavaScript processing (no DOM), and then creates 1 script tag
On desktop (MacBook) with 6x CPU slowdown and mobile emulation
(enwiki/Obama) it clocks startUp() at 35ms.
On a real mobile device (Nexus 5, Android 4.4, the oldest currently
supported Android version/device I could find on BrowserStack) it
clocks startUp() on enwiki/Obama at 44ms.
This seems small enough to make it worth doing right away, without
artificial delay, so that it can kick off the network request in
the background while we're rendering - which is exactly how things
were before commmit
dec800968eb was merged.
This means the 'load' event will once again include the main JS
request. After
dec800968eb, this request unintentionally started
after the 'load' event. I'm hoping that with the eliminated
request for jquery+mediawiki.base, we'll still keep some of the
winnings we had from a much earlier 'load' event.
Bug: T192623
Change-Id: I4942bfd236c72b0cf4c23b0e2a0d5e5e069c0de0
jenkins-bot [Tue, 28 Aug 2018 02:11:08 +0000 (02:11 +0000)]
Merge "resourceloader: Remove redundant check in mw.loader.store for Opera 12"
jenkins-bot [Tue, 28 Aug 2018 01:52:10 +0000 (01:52 +0000)]
Merge "resourceloader: Remove redundant calls to getState() in mw.loader.enqueue()"
Fomafix [Sat, 25 Aug 2018 09:57:25 +0000 (11:57 +0200)]
resourceloader: Optimise several map-like objects with Object.create(null)
Change-Id: Ia8a3159509536f58a97faadc2e72fb33f2d0e97f
Timo Tijhof [Mon, 20 Aug 2018 23:42:42 +0000 (00:42 +0100)]
resourceloader: Move logo preload from OutputPage to SkinModule
This was introduced in OutputPage before support for getPreloadLinks()
was added to ResourceLoader. The introduction in ResourceLoader was
actually inspired by this original implementation.
Now that we have it, we should make use of it for this module
as well. Doing so has several benefits:
* Makes the code cleaner by not requiring every skin to implement
the extra boolean method. Instead, it naturally works. If
the skin loads the SkinModule, it gets the preload as well.
If not (such as Minerva, which has a different logo config),
then it also doesn't get the preload link.
Naturally, automatic.
* Makes code cleaner by not having static methods, and by not
having OutputPage call into a Module class.
* Fixes the problem where, if a site's logo is changed, all cached
HTML is preloading the old logo whilst the stylesheet fetches
the newer one. Causing both to be downloaded.
* Still preloads the logo well before it can render.
Change-Id: I11b390f2e4f5e7db8b4506ab547839152888005c
Kunal Mehta [Sun, 19 Aug 2018 10:17:52 +0000 (03:17 -0700)]
Use StaticArrayWriter in LCStoreStaticArray
...instead of var_export(), which uses array() syntax and spaces for
indentation.
Also get rid of some unnecessary closure indirection.
Bug: T200626
Change-Id: I5db8ade50fcba5ecf394817b2d14295620314ea7
Kunal Mehta [Sun, 19 Aug 2018 10:15:31 +0000 (03:15 -0700)]
Support nested arrays in StaticArrayWriter
This is needed by LCStoreStaticArray.
Bug: T200626
Change-Id: I76abd3849381b3b98e350a4627f3515eeb00fa2d
jenkins-bot [Mon, 27 Aug 2018 21:44:23 +0000 (21:44 +0000)]
Merge "resourceloader: Restore mw.loader.store update postponing logic"
jenkins-bot [Mon, 27 Aug 2018 21:33:58 +0000 (21:33 +0000)]
Merge "rdbms: make LBFactorySingle/LoadBalancerSingle set the local domain from the connection"
jenkins-bot [Mon, 27 Aug 2018 21:17:04 +0000 (21:17 +0000)]
Merge "Make interwiki transclusion use the WAN cache"
Timo Tijhof [Thu, 23 Aug 2018 02:16:06 +0000 (03:16 +0100)]
resourceloader: Restore mw.loader.store update postponing logic
* Document the conditional that disables mw.loader.store if
localStorage is unavailable. `raw === undefined` is correct,
but can be confusing given it can also be null, and we
specifically don't want to disable the store in that case.
We only disable the store if raw is undefined.
* Remove the call to mw.loader.store.update() in init().
If 0 modules were to load on the current page, there is little
value in flushing an empty `items` object.
If any modules load, they will be set() in the store and
schedule an update() at that time, which is preferred and avoids
unexpected overhead.
* Remove store.enabled check from prune(). It is only called
from update(), which checks it already.
* Remove boolean return from set(). Not used.
* Remove boolean return from prune(). Not used.
* Restore the 2-second setTimeout debounce from 2013 (
c719401661e),
which got lost in the 2015 refactor (
4174b662f623).
This makes the documentation true again.
* Document the store singleton as @private. It will still be
indexed by JSDuck and listed in the sidebar (under "private"),
but will not be listed on the homepage, and the class' page
will have a notice about it being a private API.
Bug: T202598
Change-Id: Ic0d4a15c241df391ab5f824ca9e754c3938ea108
jenkins-bot [Mon, 27 Aug 2018 20:55:35 +0000 (20:55 +0000)]
Merge "resourceloader: Remove unused code from mw.loader.register()"
jenkins-bot [Mon, 27 Aug 2018 20:43:43 +0000 (20:43 +0000)]
Merge "resourceloader: Remove unused code in private mw.loader#enqueue()"
jenkins-bot [Mon, 27 Aug 2018 20:31:40 +0000 (20:31 +0000)]
Merge "Remove semicolon after class declaration"
jenkins-bot [Mon, 27 Aug 2018 20:31:34 +0000 (20:31 +0000)]
Merge "Replace "Bug37714" by "T39714""
jenkins-bot [Mon, 27 Aug 2018 20:17:02 +0000 (20:17 +0000)]
Merge "objectcache: Improve WANObjectCache doc for update strategies"
jenkins-bot [Mon, 27 Aug 2018 20:08:52 +0000 (20:08 +0000)]
Merge "API: Update examples to avoid MCR deprecation"
Translation updater bot [Mon, 27 Aug 2018 19:55:42 +0000 (21:55 +0200)]
Localisation updates from https://translatewiki.net.
Change-Id: I6a4ede0f5c74135b00c63d36f1a8fb9fb5ca323a
Aaron Schulz [Fri, 8 Jun 2018 20:43:03 +0000 (13:43 -0700)]
Make interwiki transclusion use the WAN cache
This means that now:
* Entries actually get deleted when expired
* The transclusion cache is shared across wikis
* Large blobs that do not fit in cache no longer cause DB errors
* DB writes are not triggered on GET requests
* Keys are hashed and no longer need to be so restrictive
Also, add a "check key" based purge system and process cache the
text/html values similar to how regular revision text is cached.
Bug: T189702
Change-Id: I8ac12b53c02bb26857175dd5a4af29d49e03dc33
jenkins-bot [Mon, 27 Aug 2018 19:32:01 +0000 (19:32 +0000)]
Merge "Allow admins to delete user JS/CSS pages"
jenkins-bot [Mon, 27 Aug 2018 19:10:41 +0000 (19:10 +0000)]
Merge "Paranoia, escape image alignment parameters before outputting."
Umherirrender [Mon, 27 Aug 2018 19:08:55 +0000 (21:08 +0200)]
Remove semicolon after class declaration
Change-Id: Iae57172138f8887e37ed401b443531259808e678