fix(http): correct JsonFormat string escaping and decoding#6884
Closed
waynercheung wants to merge 1 commit into
Closed
fix(http): correct JsonFormat string escaping and decoding#6884waynercheung wants to merge 1 commit into
waynercheung wants to merge 1 commit into
Conversation
waynercheung
changed the title
escapeBytesSelfType escaped only double quotes. Backslashes and raw control characters could therefore produce invalid JSON, or alter the object seen by the lenient re-parse in Util.printTransactionToJSON. Use escapeText for valid UTF-8 and fall back to hex for malformed UTF-8. Unescape visible name-string bytes on input so print and merge are symmetric. Also fix newline-tail duplication in JsonGenerator.print, U+FFFF truncation and malformed-surrogate handling in escapeText, and support the escaped solidus while rejecting signed Unicode escapes in unescapeText. BREAKING CHANGE: for visible=true name-string byte fields, JSON escape sequences are now decoded before conversion to bytes, so a request sending "a\nb" stores one 0x0A byte instead of a backslash followed by 'n'. Such requests already complete today, they just store the undecoded text, so upgrading changes the bytes written on chain for clients that escape their payloads correctly. Malformed or unsupported escapes are now rejected, while the legacy "\'" extension stays accepted. The shared proto string parser also now accepts "\/" and rejects malformed escapes such as "\u+123", in both visible modes; the visible=false bytes/hex path itself is unchanged. During a rolling upgrade, avoid crossing node versions within an affected visible=true create/sign/broadcast flow, in either direction.
waynercheung
force-pushed
the
fix/jsonformat-name-string-escape
branch
from
July 20, 2026 15:50
30c457a to
c34ee46
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
escapeBytesSelfTypeescaped only the double quote, so backslashes and control characters0x00-0x1Fpassed through raw. ReuseescapeTextfor these fields, falling back to hex for malformed UTF-8, and decode escapes symmetrically on input so values surviveprint -> merge.Several pre-existing defects in the same file are fixed alongside:
JsonGenerator.printduplicated the tail on every newline;write()now takes only the data it writes, so its length cannot disagree with the range written.escapeTexttruncated values at U+FFFF, becauseCharacterIterator.DONEis that code point.escapeTextthrewIllegalArgumentExceptionon unpaired surrogates, which now render as?to match protobuf's own UTF-8 substitution.unescapeTextrejected the RFC 8259 escaped solidus\/.unescapeTextaccepted signed escapes such as\u+123, becauseInteger.parseIntallows a leading sign.Why are these changes required?
The old output was JSON that a strict RFC 8259 parser rejects, and it was not reproduced faithfully by the lenient re-parse in
Util.printTransactionToJSON. Because output escaped"while input never unescaped, values containing a quote never survivedcreate -> sign -> broadcast; that is why both halves must land together.This PR has been tested by:
JsonFormatEscapeTest, 40 cases: control chars, backslash, quote, U+FFFF, supplementary plane, malformed UTF-8, unpaired surrogates,print -> mergebyte round trip, inbound escape set, and a fixed-seed property sweep over all 1-byte and all 65536 2-byte inputs. RevertingJsonFormat.javato develop ate89c0d6652fails 28 of the 40, so the suite is not vacuous.org.tron.core.services.httpandorg.tron.core.services.jsonrpcpackages,checkstyleMain,checkstyleTest,git diff --check: all clean.protobuf-java-util3.25.8, the version this repo depends on.Consensus impact: none. No consensus validation, protobuf schema, txID algorithm or historical transaction replay is touched. All changes are in the HTTP JSON conversion layer.
Follow up
Not included, each a separate compatibility decision: HTML-safe escaping of
<>&='and of U+2028/U+2029 (protobuf-java-util3.25.8 does both, but=and&are too common in URLs to change here), tightening the tokenizer's outer leniency (single quotes, unquoted field names, trailing commas, raw control chars), the legacy\'escape, and thebytesmapping, which stays TRON'shex/ Base58Check / UTF-8 text rather than Base64.Extra details
BREAKING CHANGE, request handling. The
visible=falsebytes/hex path is unchanged, butunescapeTextis shared by every protostringfield with novisiblecheck, so\/is now accepted and\u+123rejected in both visible modes.For
visible=truename-string byte fields, valid JSON escapes are now decoded, while invalid escapes such as\dor a malformed\uXXXXare rejected instead of being stored literally. The rejection is a change in request acceptance, not only in the value stored."a\nb"615c6e62(a\nb)610a62(aLFb)"C:\\dir"433a5c5c646972(two backslashes)433a5c646972(one)"a\"b"615c2262612262"C:\dir"433a5c646972(stored literally)The new value is what the client meant per JSON, so this is a correction. Note that the create endpoints do not themselves write to the chain: the same HTTP JSON request can construct different
raw_data, which, if subsequently signed and broadcast, results in different contract bytes on chain. Downstream systems that deduplicate, compare or replay should be aware.For affected escape-containing payloads, avoid crossing node versions within one
visible=truecreate/sign/broadcast flow during a rolling upgrade, in either direction; payloads without escapes are unaffected.