Add APIs for case folding to the standard library

[Jules-Bertholet]

Libs-api requested these, so here they are.

New public API (gated behind #[feature(casefold)]):

impl char {
    pub fn to_casefold(self) -> ToCasefold;
}

impl str {
    pub fn to_casefold(&self) -> String;
    pub fn eq_ignore_case(&self) -> bool;
}

pub struct ToCasefold { ... }
impl Iterator for ToCasefold { type Item = char; ... }
impl DoubleEndedIterator for ToCasefold { ... }
impl FusedIterator for ToCasefold { }
impl ExactSizeIterator for ToCasefold { ... }
impl fmt::Display for ToCasefold { ... }

Notes

• This only adds a negligible amount of static data to core::unicode. To accomplish that, we compute the case-folding for most characters as the lowercase of their uppercase; this double mapping adds some complexity to the implementation.
• No normalization (e.g. NFC) is performed, so visually and semantically equivalent strings can compare unequal.
• I have not put any effort into optimizing eq_ignore_case(); there may be a more performant implementation.
• char::eq_ignore_case() is left to future work—it's a potential footgun, so we may want to think more deeply about how to expose and document that API.

@rustbot label T-libs-api A-unicode

[rustbot]

These commits modify the Cargo.lock file. Unintentional changes to Cargo.lock can be introduced when switching branches and rebasing PRs.

If this was unintentional then you should revert the changes before this PR is merged.
Otherwise, you can ignore this comment.

library/core/src/unicode/unicode_data.rs is generated by the src/tools/unicode-table-generator tool.

If you want to modify unicode_data.rs, please modify the tool then regenerate the library source file via ./x run src/tools/unicode-table-generator instead of editing unicode_data.rs manually.

[rustbot]

r? @scottmcm

rustbot has assigned @scottmcm.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

Why was this reviewer chosen?

The reviewer was selected based on:

• Owners of files modified in this PR: @scottmcm, libs
• @scottmcm, libs expanded to 8 candidates
• Random selection from Mark-Simulacrum, jhpratt, scottmcm

[scottmcm]

@rustbot reroll

[rustbot]

This PR was rebased onto a different main commit. Here's a range-diff highlighting what actually changed.

Rebasing is a normal part of keeping PRs up to date, so no action is needed—this note is just to help reviewers.