If us_props is too big for the database (> 64kb), don't store the
file handler metadata. This is usually by far the biggest portion
(For pdf/djvu it contains OCR data), and it's non-critical to the
upload process as it's only needed for display, and the file handler
will probably regenerate it anyway if it is missing.
At some point in the future, having data too big for a db field
is probably going to cause errors instead of silent truncation,
so we really don't want to be doing silent truncation.
An alternative way to solve T94562 might be to make it use us_sha1
instead of the one encoded in us_props, but seems like it's important
to fix us_props for other data in it too.
Bug: T94562
Change-Id: I8aed6a1a5c9a136090b36d4ad23d8429791f58c4
class UploadStash {
// Format of the key for files -- has to be suitable as a filename itself (e.g. ab12cd34ef.jpg)
const KEY_FORMAT_REGEX = '/^[\w-\.]+\.\w*$/';
class UploadStash {
// Format of the key for files -- has to be suitable as a filename itself (e.g. ab12cd34ef.jpg)
const KEY_FORMAT_REGEX = '/^[\w-\.]+\.\w*$/';
+ const MAX_US_PROPS_SIZE = 65535;
/**
* repository that this uses to store temp files
/**
* repository that this uses to store temp files
wfDebug( __METHOD__ . " inserting $stashPath under $key\n" );
$dbw = $this->repo->getMasterDb();
wfDebug( __METHOD__ . " inserting $stashPath under $key\n" );
$dbw = $this->repo->getMasterDb();
+ $serializedFileProps = serialize( $fileProps );
+ if ( strlen( $serializedFileProps ) > self::MAX_US_PROPS_SIZE ) {
+ // Database is going to truncate this and make the field invalid.
+ // Prioritize important metadata over file handler metadata.
+ // File handler should be prepared to regenerate invalid metadata if needed.
+ $fileProps['metadata'] = false;
+ $serializedFileProps = serialize( $fileProps );
+ }
+
$this->fileMetadata[$key] = array(
'us_id' => $dbw->nextSequenceValue( 'uploadstash_us_id_seq' ),
'us_user' => $this->userId,
'us_key' => $key,
'us_orig_path' => $path,
'us_path' => $stashPath, // virtual URL
$this->fileMetadata[$key] = array(
'us_id' => $dbw->nextSequenceValue( 'uploadstash_us_id_seq' ),
'us_user' => $this->userId,
'us_key' => $key,
'us_orig_path' => $path,
'us_path' => $stashPath, // virtual URL
- 'us_props' => $dbw->encodeBlob( serialize( $fileProps ) ),
+ 'us_props' => $dbw->encodeBlob( $serializedFileProps ),
'us_size' => $fileProps['size'],
'us_sha1' => $fileProps['sha1'],
'us_mime' => $fileProps['mime'],
'us_size' => $fileProps['size'],
'us_sha1' => $fileProps['sha1'],
'us_mime' => $fileProps['mime'],