Improve normalization and sanitization of memcached keys
The motivation for this patch came from seeing the following error in the
memcached log:
memcached ERROR: Memcached error for key "flowdb:flow_ref:wiki:by-source:\
v3:0:tawikinews:பூமிக்கு_அச்சுறுத்தலான_சிறுகோள்களைக்_கண்டுபிடிக்கும்_முயற்சியில்_தனியார்_நிறுவன0jம்:4.7" \
on server ":": A BAD KEY WAS PROVIDED/CHARACTERS OUT OF RANGE
I submitted a fix to Flow in I26e531f6, but I noticed that AbuseFilter had a
similar issue (fixed by Aaron in I27b51a4b), so I started thinking about how
to solve this more generally:
* The regular expression we current use to sanitize keys does not cover
characters outside the ASCII range, but such characters can be illegal
if one of their constituent bytes (when taken by itself) is an ASCII
control character. So change the regular expression to cover any and all
characters that fall outside the range \x22-\x7e (and '#' -- see below).
* Enforce a key length limit of 255 bytes, which is the maximum length
permitted by the memcached protocol. The Tamil segment in the key above is 84
characters, but 233 bytes in UTF-8, which become 684 characters when
URL-encoded. To fix this, try to shrink any segment that would push the total
key length over the limit by md5()ing it. If the end result is *still* over
the limit (this would happen if, for example, $args consists of many short
strings), then concatenate all args together and MD5 them.
* MD5'd arguments are prefixed with '#'. Any "organic" '#'s in the key segments
are URL-encoded.
Change-Id: Ia46987d3b0a09bb6b1952abd936d4c72ea7c56a0