Add a mechanism to parserTests when run in --compare or --record mode, to give inform...

a test's history. This may help to differentiate tests that have never passed, versus new regressions.
Also for a regression the range of dates and SVN version numbers in which it first appeared will be
given, which may help isolate the cause of the regression; equally fixed tests will show a range
to indicate when the test was fixed.

Here is an example of the output (using a parserTests.txt modified specifically to cause all
the conditions to occur) :
---------------------------------------------------------------------------------------
root@bling:/var/www/hosts/mediawiki/wiki# php maintenance/parserTests.php --compare --quick --quiet --color=no
This is MediaWiki version 1.10alpha (r19799).

Reading tests from "maintenance/parserTests.txt"...
This is dvips(k) 5.95a Copyright 2005 Radical Eye Software (www.radicaleye.com)
' TeX output 2007.02.06:0630' ->
<tex.pro><texps.pro>. <cmr12.pfb>[1]
Reading tests from "../extensions/Cite/citeParserTests.txt"...
Reading tests from "../extensions/LabeledSectionTransclusion/lstParserTests.txt"...

   1 previously failing test(s) now PASSING! :)
      * URL-encoding in URL functions (single parameter)  [Fixed between 05-Feb-2007 23:58:17, 1.10alpha (r19799) and 06-Feb-2007 06:30:34, 1.10alpha (r19799)]

   1 previously PASSING test(s) removed o_O
      * Prevent conversion of text with -{}- tags (language variants)  [First recorded appearance: 02-Jan-2007 04:30:32, 1.9alpha (r18762)]

   1 new PASSING test(s) :)
      * Fake passing test  [Has never failed]

   1 previously passing test(s) now FAILING! :(
      * Plain link to URL  [Introduced between 05-Feb-2007 23:58:17, 1.10alpha (r19799) and 06-Feb-2007 06:30:34, 1.10alpha (r19799)]

   1 previously FAILING test(s) removed O_o
      * TODO: Link containing double-single-quotes '' (bug 4598)  [First recorded appearance: 15-Nov-2006 02:53:34, 1.9alpha (r17686)]

   1 new FAILING test(s) :(
      * Blah blah fake test  [Has never passed]

  18 still FAILING test(s) :(
      * URL-encoding in URL functions (multiple parameters)  [Has never passed]
      * TODO: Table security: embedded pipes (http://mail.wikipedia.org/pipermail/wikitech-l/2006-April/034637.html)  [Has never passed]
      * TODO: message transform: <noinclude> in transcluded template (bug 4926)  [Has never passed]
      * TODO: message transform: <onlyinclude> in transcluded template (bug 4926)  [Has never passed]
      * TODO: HTML bullet list, unclosed tags (bug 5497)  [Has never passed]
      * TODO: HTML ordered list, unclosed tags (bug 5497)  [Has never passed]
      * TODO: HTML nested bullet list, open tags (bug 5497)  [Has never passed]
      * TODO: HTML nested ordered list, open tags (bug 5497)  [Has never passed]
      * TODO: Inline HTML vs wiki block nesting  [Has never passed]
      * TODO: Mixing markup for italics and bold  [Has never passed]
      * TODO: 5 quotes, code coverage +1 line  [Has never passed]
      * TODO: dt/dd/dl test  [Has never passed]
      * TODO: Images with the "|" character in the comment  [Has never passed]
      * TODO: Parents of subpages, two levels up, without trailing slash or name.  [Has never passed]
      * TODO: Parents of subpages, two levels up, with lots of extra trailing slashes.  [Has never passed]
      * Blank ref followed by ref with content  [Introduced between 31-Jan-2007 23:52:43, 1.10alpha (r19702) and 05-Feb-2007 23:58:17, 1.10alpha (r19799)]
      * Regression: non-blank ref "0" followed by ref with content  [Introduced between 31-Jan-2007 23:52:43, 1.10alpha (r19702) and 05-Feb-2007 23:58:17, 1.10alpha (r19799)]
      * Regression sanity check: non-blank ref "1" followed by ref with content  [Introduced between 31-Jan-2007 23:52:43, 1.10alpha (r19702) and 05-Feb-2007 23:58:17, 1.10alpha (r19799)]

Passed 509 of 529 tests (96.22%)... 20 tests failed!
root@bling:/var/www/hosts/mediawiki/wiki#
---------------------------------------------------------------------------------------

Lastly please note that this is my first play with the MW database API, so if there are better
ways of doing any of the database query stuff than in this patch, then please don't be shy
about telling me how.

diff --git a/maintenance/parserTests.inc b/maintenance/parserTests.inc
index 6bcb490..197ad32 100644 (file)
--- a/maintenance/parserTests.inc
+++ b/maintenance/parserTests.inc
@@ -1021,8 +1021,8 @@ class DbTestRecorder extends TestRecorder  {
                                if( $differences ) {
                                        $count = count($differences);
                                        printf( "\n%4d %s\n", $count, $label );
-                                       foreach ($differences as $differing_test_name) {
-                                               print "      * $differing_test_name\n";
+                                       foreach ($differences as $differing_test_name => $statusInfo) {
+                                               print "      * $differing_test_name  [$statusInfo]\n";
                                        }
                                }
                        }
@@ -1034,8 +1034,8 @@ class DbTestRecorder extends TestRecorder  {
        }
 
        /**
-        ** @desc: Returns an array of the test names with changed results, based on the specified
-        **        before/after criteria.
+        ** Returns an array of the test names with changed results, based on the specified
+        ** before/after criteria.
         */
        private function compareResult( $before, $after ) {
                $testitem = $this->db->tableName( 'testitem' );
@@ -1065,14 +1065,77 @@ class DbTestRecorder extends TestRecorder  {
                $result = $this->db->query( $sql, __METHOD__ );
                $retval = array();
                while ($row = $this->db->fetchObject( $result )) {
-                       $retval[] = $row->t;
+                       $testname = $row->t;
+                       $retval[$testname] = $this->getTestStatusInfo( $testname, $after, $curRun );
                }
                $this->db->freeResult( $result );
                return $retval;
        }
 
        /**
-        ** @desc: Helper function for compareResult() database querying.
+        ** Returns a string giving information about when a test last had a status change.
+        ** Could help to track down when regressions were introduced, as distinct from tests
+        ** which have never passed (which are more change requests than regressions).
+        */
+       private function getTestStatusInfo($testname, $after, $curRun) {
+
+               // If we're looking at a test that has just been removed, then say when it first appeared.
+               if ( is_null( $after ) ) {
+                       $changedRun = $this->db->selectField ( 'testitem',
+                                                                                                  'MIN(ti_run)',
+                                                                                                  array( 'ti_name' => $testname ),
+                                                                                                  __METHOD__ );
+                       $appear = $this->db->selectRow ( 'testrun',
+                                                                                        array( 'tr_date', 'tr_mw_version' ),
+                                                                                        array( 'tr_id' => $changedRun ),
+                                                                                        __METHOD__ );
+                       return "First recorded appearance: "
+                              . date( "d-M-Y H:i:s",  strtotime ( $appear->tr_date ) )
+                              .  ", " . $appear->tr_mw_version;
+               }
+
+               // Otherwise, this test has previous recorded results.
+               // See when this test last had a different result to what we're seeing now.
+               $changedRun = $this->db->selectField ( 'testitem',
+                                                                                          'MAX(ti_run)',
+                                                                                          array( 
+                                                                                              'ti_name'    => $testname,
+                                                                                              'ti_success' => ($after ? "0" : "1"),
+                                                                                              "ti_run != " . $this->db->addQuotes ( $curRun )
+                                                                                               ), 
+                                                                                               __METHOD__ );
+
+               // If no record of ever having had a different result.
+               if ( is_null ( $changedRun ) ) {
+                       if ($after == "0") {
+                               return "Has never passed";
+                       } else {
+                               return "Has never failed";
+                       }
+               }
+
+               // Otherwise, we're looking at a test whose status has changed.
+               // (i.e. it used to work, but now doesn't; or used to fail, but is now fixed.)
+               // In this situation, give as much info as we can as to when it changed status.
+               $pre  = $this->db->selectRow ( 'testrun',
+                                                                               array( 'tr_date', 'tr_mw_version' ),
+                                                                               array( 'tr_id' => $changedRun ),
+                                                                               __METHOD__ );
+               $post = $this->db->selectRow ( 'testrun',
+                                                                               array( 'tr_date', 'tr_mw_version' ),
+                                                                               array( "tr_id > " . $this->db->addQuotes ( $changedRun) ),
+                                                                               __METHOD__,
+                                                                               array( "LIMIT" => 1, "ORDER BY" => 'tr_id' )
+                                                                        );
+
+               return ( $after == "0" ? "Introduced" : "Fixed" ) . " between "
+                               . date( "d-M-Y H:i:s",  strtotime ( $pre->tr_date ) ) .  ", " . $pre->tr_mw_version
+                               . " and "
+                               . date( "d-M-Y H:i:s",  strtotime ( $post->tr_date  ) ) .  ", " . $post->tr_mw_version ;
+       }
+
+       /**
+        ** Helper function for compareResult() database querying.
         */
        private function condition( $value ) {
                if( is_null( $value ) ) {
@@ -1094,4 +1157,4 @@ class DbTestPreviewer extends DbTestRecorder  {
        }
 }
 
-?>
+?>
\ No newline at end of file