This provisional workaround runs a page at a time through UtfNormal::cleanUp() instead of running the entire file's dumped text at once. This avoids exploding memory too much during the preg_match_all() used to divide up ASCII and non-ASCII runs for validation, which is very wasteful for long texts in Latin languages with many mixed-in non-ASCII characters (like French and German text).
Won't fix legit cases of huge texts, such as realllllllllly long page text, which would still be subject to getting run through at web input time in a giant chunk.
$txt = wfShellExec( $cmd, $retval );
wfProfileOut( 'djvutxt' );
if( $retval == 0) {
- # Get rid of invalid UTF-8, strip control characters
- $txt = UtfNormal::cleanUp( $txt );
+ # Strip some control characters
$txt = preg_replace( "/[\013\035\037]/", "", $txt );
$reg = <<<EOR
/\(page\s[\d-]*\s[\d-]*\s[\d-]*\s[\d-]*\s*"
}
function pageTextCallback( $matches ) {
- return '<PAGE value="' . htmlspecialchars( $matches[1] ) . '" />';
+ # Get rid of invalid UTF-8, strip control characters
+ return '<PAGE value="' . htmlspecialchars( UtfNormal::cleanUp( $matches[1] ) ) . '" />';
}
/**