Configurable log sampling via $wgDebugLogGroups

Make it possible to configure a log-group-specific sampling factor by allowing
$wgDebugLogGroups values to be associative arrays with 'destination' & 'sample'
keys.

The immediate use-case I have in mind is the memcached log, which is too
verbose to enable in production. We could sample on the receiving end, but
doing so would not help MediaWiki, which would still be on the hook for
processing and dispatching every single record.

Idea was vetted by Dr. Manhattan:

<TimStarling> I think you could have each element in $wgDebugLogGroups be an
  associative array
<TimStarling> e.g. array( 'sample' => 1000, 'target' => "$host:$port")
<TimStarling> that would allow for more features to be added to it in
  future

Change-Id: Ib0ece5d5d9c2aadef13b77ac38995b0e47ac086f

diff --git a/includes/DefaultSettings.php b/includes/DefaultSettings.php
index ebae110..6d0e36c 100644 (file)
--- a/includes/DefaultSettings.php
+++ b/includes/DefaultSettings.php
@@ -4895,10 +4895,29 @@ $wgDebugDBTransactions = false;
 $wgDebugDumpSql = false;
 
 /**
- * Set to an array of log group keys to filenames.
+ * Map of string log group names to log destinations.
+ *
  * If set, wfDebugLog() output for that group will go to that file instead
  * of the regular $wgDebugLogFile. Useful for enabling selective logging
  * in production.
+ *
+ * Log destinations may be string values specifying a filename or URI, or they
+ * may be filename or an associative array mapping 'target' to the desired
+ * filename. The associative array may also contain a 'sample' key with an
+ * integer value, specifying a sampling factor.
+ *
+ * @par Example:
+ * @code
+ * $wgDebugLogGroups['redis'] = '/var/log/mediawiki/redis.log';
+ * @endcode
+ *
+ * @par Advanced example:
+ * @code
+ * $wgDebugLogGroups['memcached'] = (
+ *     'target' => '/var/log/mediawiki/memcached.log',
+ *     'sample' => 1000,  // log 1 message out of every 1,000.
+ * );
+ * @endcode
  */
 $wgDebugLogGroups = array();
 

diff --git a/includes/GlobalFunctions.php b/includes/GlobalFunctions.php
index ff91ba0..d8fe38e 100644 (file)
--- a/includes/GlobalFunctions.php
+++ b/includes/GlobalFunctions.php
@@ -1008,7 +1008,12 @@ function wfDebugMem( $exact = false ) {
 
 /**
  * Send a line to a supplementary debug log file, if configured, or main debug log if not.
- * $wgDebugLogGroups[$logGroup] should be set to a filename to send to a separate log.
+ * To configure a supplementary log file, set $wgDebugLogGroups[$logGroup] to a string
+ * filename or an associative array mapping 'target' to the desired filename. The
+ * associative array may also contain a 'sample' key with an integer value, specifying
+ * a sampling factor.
+ *
+ * @since 1.23 support for sampling log messages via $wgDebugLogGroups.
  *
  * @param $logGroup String
  * @param $text String
@@ -1018,14 +1023,28 @@ function wfDebugMem( $exact = false ) {
 function wfDebugLog( $logGroup, $text, $public = true ) {
        global $wgDebugLogGroups;
        $text = trim( $text ) . "\n";
-       if ( isset( $wgDebugLogGroups[$logGroup] ) ) {
-               $time = wfTimestamp( TS_DB );
-               $wiki = wfWikiID();
-               $host = wfHostname();
-               wfErrorLog( "$time $host $wiki: $text", $wgDebugLogGroups[$logGroup] );
-       } elseif ( $public === true ) {
-               wfDebug( "[$logGroup] $text", false );
+
+       if ( !isset( $wgDebugLogGroups[$logGroup] ) ) {
+               if ( $public === true ) {
+                       wfDebug( "[$logGroup] $text", false );
+               }
+               return;
        }
+
+       $logConfig = $wgDebugLogGroups[$logGroup];
+       if ( is_array( $logConfig ) ) {
+               if ( isset( $logConfig['sample'] ) && mt_rand( 1, $logConfig['sample'] ) !== 1 ) {
+                       return;
+               }
+               $destination = $logConfig['destination'];
+       } else {
+               $destination = strval( $logConfig );
+       }
+
+       $time = wfTimestamp( TS_DB );
+       $wiki = wfWikiID();
+       $host = wfHostname();
+       wfErrorLog( "$time $host $wiki: $text", $destination );
 }
 
 /**