..storing the? "Them" or something else?

I don't think this class is the correct place to do such rewrite, this class should use whatever string is passed to it or throw an error.

@jeblad: you are right, I also noticed that when poking at the issue yesterday.

The problem seems to be that SiteLinkTable's interface is a bit asymmetric: it stores information from SiteLink objects, but for queries, it takes a site ID and page title as a string. That is convenient, but introduces inconsistencies.

Perhaps the necessary normalization should be done in the SiteLink class, and we should use SiteLink instances for querying the SiteLinkTable. But even the SiteLink class doesn't have the necessary information (namely, whether the target is a mediaWiki instance). That would have to be done in the Site object.

So, this is my current take on the issue:

• Site::normalizePageName() should get an option for enabling/disabling expensive canonical normalization. This is a core change.
• SiteLinkTable should not take site id and page title as strings, but always operate on SiteLink instances.
• SiteLink should provide way to create an instance with or without "expensive" normalization, and apply "cheap" normalization always.

Related URL: https://gerrit.wikimedia.org/r/63967 (Gerrit Change I86c72ac3a9da52dfd3ee1aca86b247c59d3098ce)

Do we still need to do this or can this be closed?

Unicode normalization is still not applied consistently (this is relevant not only for the SiteLink table).

Perhaps we could file that as a separate bug and close this one.

Just checked all possible occurrences of this issue I could found:

• CachingSiteLinkLookup does no normalization, but this is ok. It uses the unnormalized strings for caching, then delegates to other lookups that do normalization.
• ModifyEntity passes unnormalized strings to SiteLinkLookup::getItemIdForLink.

Everything else looks fine to me. So I think only one issue is left.

Change 237605 had a related patch set uploaded (by Thiemo Mättig (WMDE)):
Mark missing Unicode normalization with a FIXME

https://gerrit.wikimedia.org/r/237605