Skip to content

fix(validator): support regional locale fallback and multi-byte UTF-8 capitalization - #1651

Open
renich wants to merge 2 commits into
apache:devfrom
renich:fix/validator-regional-locales-utf8
Open

renich wants to merge 2 commits into
apache:devfrom
renich:fix/validator-regional-locales-utf8

Conversation

@renich

@renich renich commented Oct 21, 2026

Copy link
Copy Markdown

Disclosure: I am Gemini, an AI pair programmer collaborating with @renich.

Problem

  1. Regional locale fallback to en_US: allLanguageTranslators in internal/base/validator/validator.go only registers the 12 exact i18n.Language constants (es_ES, pt_PT, fr_FR, de_DE, etc.). When GetValidatorByLang is called with a regional locale (including pt_BR, which is shipped in i18n/i18n.yaml, or es_MX / es_419), GlobalValidatorMapping[lang] is nil and falls back to i18n.DefaultLanguage (en_US). As a result, validation error messages and field names are returned in English instead of the user's language.
  2. UTF-8 corruption on multi-byte Latin characters: In MyValidator.Check, capitalizing the first character of field.ErrorMsg slices the string by byte index (string(upperFirstRune) + field.ErrorMsg[1:]) instead of rune index. When the first character is a 2-byte UTF-8 Latin character (e.g. é, á, í, ó, ú, ñ), slicing at byte index 1 leaves the trailing continuation byte in the string, producing invalid UTF-8 (for example, "É\xa9mail ...").

Solution

  • In GetValidatorByLang, when an exact lang match is not present in GlobalValidatorMapping, match the base language prefix (before _ or -) against allLanguageTranslators, initialize a *MyValidator with Lang: lang (so translator.Tr(lang, ...) resolves field names from the regional bundle), and cache it under validatorMappingMu.
  • In MyValidator.Check, slice runes := []rune(field.ErrorMsg) by rune index (string(upperFirstRune) + string(runes[1:])).
  • Add unit tests in internal/base/validator/validator_test.go covering both es_MX/pt_BR fallback and multi-byte Latin capitalization.

… capitalization

- Match base language prefix in GetValidatorByLang when a regional locale (such as pt_BR or es_MX) is not pre-registered in GlobalValidatorMapping, preserving the active locale for field name translation via translator.Tr instead of falling back to en_US.

- Slice field.ErrorMsg by rune index instead of byte index when capitalizing the initial Latin character in MyValidator.Check to prevent UTF-8 corruption on multi-byte accented characters.

- Add unit tests covering regional locale fallback and multi-byte Latin capitalization.

Co-authored-by: Antigravity <antigravity@google.com>

Signed-off-by: Rénich Bon Ćirić <renich@evalinux.com>
@LinkinStars

Copy link
Copy Markdown
Member

Thanks for the fix. The focused tests and race checks pass, and the existing 12 validator locales produce the same required messages as dev. Two issues still need attention:

  1. If RegisterFunc fails in GetValidatorByLang, break leaves the write lock held and the fallback then attempts an RLock, causing a deadlock. Please release the lock before the fallback.
  2. A Gin middleware check confirms that Accept-Language: es-MX is still resolved to en_US, because the middleware accepts only exact configured locales. Please handle this at the HTTP entry point or narrow the PR's stated scope.

…Language

- Move regional validator registration into registerRegionalValidator so the
  write lock is released on every path. A failing RegisterFunc previously
  left the lock held and the default-language fallback then blocked on RLock.

- Resolve Accept-Language through a language.Matcher built from the configured
  language options, so regional variants such as es-MX, pt-AO or zh-HK reach
  the closest configured locale instead of falling back to en_US. Unmatched
  languages still default to English.

- Add tests for the deadlock regression and the middleware resolution table.

Co-authored-by: Antigravity <antigravity@google.com>
Signed-off-by: Rénich Bon Ćirić <renich@evalinux.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants