Page MenuHomePhabricator

Migrate memcache serialization from igbinary
Closed, ResolvedPublic

Description

HHVM ships with an lz4 compression library -- see https://github.com/facebook/hhvm/tree/master/hphp/third_party/lz4. Is it a viable replacement for igbinary serialization? It would require a complicate migration in production, but I suspect the overall effort required would be less than adding igbinary support for HHVM.


Version: unspecified
Severity: normal
See Also:
https://bugzilla.wikimedia.org/show_bug.cgi?id=73467

Details

Reference
bz61227

Event Timeline

bzimport raised the priority of this task from to Medium.Nov 22 2014, 3:01 AM
bzimport set Reference to bz61227.
bzimport added a subscriber: Unknown Object (MLST).

We should stick with igbinary for now. Aaron has agreed to try to port it.

We don't need to port it.

MemcachedPeclBagOStuff stores the serializer in the bitfield accompanying each value, and will use the correct deserializer regardless of which serializer is currently configured. Switching to the PHP serializer means any new keys written from now on will use PHP serialization, while existing values will continue to be readable.

We've done this for Labs already.

TODOs:

  • Make the switch in prod
  • Devise a tool for assessing % of keys that still use igbinary (for example, tcpdump 1 min of memcached GETs and check each key's serialization flag)

Change 119461 had a related patch set uploaded by Ori.livneh:
Set serializer to 'php' for production memcache

https://gerrit.wikimedia.org/r/119461

Change 119461 merged by Ori.livneh:
Set serializer to 'php' for production memcache

https://gerrit.wikimedia.org/r/119461

Getting close to the 30 day mark of the conf change in https://bugzilla.wikimedia.org/show_bug.cgi?id=61227. Since parser cache is stored for 30 days (longer than most other stuff, rev cache is only 7) then most cache entries should be in the new format by now.