qemu/qemu.git/qobject/json-lexer.c, branch master

json-parser: add location to JSON parsing errors

2026-07-02T11:56:35+00:00

Now that all calls to parse_error have a token, add the line and column to the message. As far as I can see the two important TODOs (better errors and better EOI handling) are done, and the others (token range information and "parsed size"?) do not really matter or are handled better by json-streamer.c. So remove the list, which had sat unchanged since 2009. This needs some adjustments to provide a good x and y for error messages. First of all, they switch from zero-based to one-based, which is safe because they were both sitting unused. Second, right now the x and y are those of the *last* character in the token. Modify json-lexer.c to freeze tok->x and tok->y at the first character added to the GString. Signed-off-by: Paolo Bonzini Message-ID: <20260626101727.1727389-7-pbonzini@redhat.com> Reviewed-by: Markus Armbruster Signed-off-by: Markus Armbruster

docs/interop: Convert qmp-spec.txt to rST

2023-05-22T08:21:01+00:00

Convert the qmp-spec.txt document to restructuredText. Notable points about the conversion: * numbers at the start of section headings are removed, to match the style of the rest of the manual * cross-references to other sections or documents are hyperlinked * various formatting tweaks (notably the examples, which need the -> and <- prefixed so the QMP code-block lexer will accept them) * English prose fixed in a few places Reviewed-by: Eric Blake Signed-off-by: Peter Maydell Message-Id: <20230515162245.3964307-2-peter.maydell@linaro.org> Reviewed-by: Markus Armbruster Signed-off-by: Markus Armbruster [.. code-block:: dumbed down to :: to work around CI failure]

json: Fix off-by-one assert check in next_state()

2019-03-26T07:10:11+00:00

The assert checking if the value of lexer->state in next_state(), which is used as an index to the 'json_lexer' array, incorrectly checks for an index value less than or equal to ARRAY_SIZE(json_lexer). Fix assert so that it just checks for an index less than the array size. Signed-off-by: Liam Merwick Message-Id: <1553169472-25325-1-git-send-email-liam.merwick@oracle.com> Reviewed-by: Markus Armbruster Reviewed-by: Li Qiang Reviewed-by: Stefano Garzarella Signed-off-by: Markus Armbruster

json: Eliminate lexer state IN_WHITESPACE, pseudo-token JSON_SKIP

2018-09-24T16:08:07+00:00

The lexer ignores whitespace like this: on whitespace on non-ws spontaneously IN_START --> IN_WHITESPACE --> JSON_SKIP --> IN_START ^ | \__/ on whitespace This accumulates a whitespace token in state IN_WHITESPACE, only to throw it away on the transition via JSON_SKIP to the start state. Wasteful. Go from IN_START to IN_START on whitespace directly, dropping the whitespace character. Signed-off-by: Markus Armbruster Reviewed-by: Eric Blake Message-Id: <20180831075841.13363-7-armbru@redhat.com>

json: Eliminate lexer state IN_ERROR

2018-09-24T16:08:07+00:00

Signed-off-by: Markus Armbruster Reviewed-by: Eric Blake Message-Id: <20180831075841.13363-6-armbru@redhat.com>

json: Nicer recovery from lexical errors

2018-09-24T16:08:07+00:00

When the lexer chokes on an input character, it consumes the character, emits a JSON error token, and enters its start state. This can lead to suboptimal error recovery. For instance, input 0123 , produces the tokens JSON_ERROR 01 JSON_INTEGER 23 JSON_COMMA , Make the lexer skip characters after a lexical error until a structural character ('[', ']', '{', '}', ':', ','), an ASCII control character, or '\xFE', or '\xFF'. Note that we must not skip ASCII control characters, '\xFE', '\xFF', because those are documented to force the JSON parser into known-good state, by docs/interop/qmp-spec.txt. The lexer now produces JSON_ERROR 01 JSON_COMMA , Update qmp-test for the nicer error recovery: QMP now reports just one error for input %p instead of two. Also drop the newline after %p; it was needed to tease out the second error. Signed-off-by: Markus Armbruster Reviewed-by: Eric Blake Message-Id: <20180831075841.13363-5-armbru@redhat.com> [Conflict with commit ebb4d82d888 resolved]

json: Make lexer's "character consumed" logic less confusing

2018-09-24T16:06:09+00:00

The lexer uses macro TERMINAL_NEEDED_LOOKAHEAD() to decide whether a state transition consumes the input character. It returns true when the state transition is defined with the TERMINAL() macro. To detect that, it checks whether input '\0' would have resulted in the same state transition, and the new state is not IN_ERROR. Why does that even work? For all states, the new state on input '\0' is either IN_ERROR or defined with TERMINAL(). If the state transition equals the one we'd get for input '\0', it goes to IN_ERROR or to the argument of TERMINAL(). We never use TERMINAL(IN_ERROR), because it makes no sense. Thus, if it doesn't go to IN_ERROR, it must be defined with TERMINAL(). Since this isn't quite confusing enough, we negate the result to get @char_consumed, and ignore it when @flush is true. Instead of deriving the lookahead bit from the state transition, make it explicit. This is easier to understand, and a bit more flexible, too. Signed-off-by: Markus Armbruster Reviewed-by: Eric Blake Message-Id: <20180831075841.13363-4-armbru@redhat.com>

json: Clean up how lexer consumes "end of input"

2018-09-24T16:06:09+00:00

When the lexer isn't in its start state at the end of input, it's working on a token. To flush it out, it needs to transit to its start state on "end of input" lookahead. There are two ways to the start state, depending on the current state: * If the lexer is in a TERMINAL(JSON_FOO) state, it can emit a JSON_FOO token. * Else, it can go to IN_ERROR state, and emit a JSON_ERROR token. There are complications, however: * The transition to IN_ERROR state consumes the input character and adds it to the JSON_ERROR token. The latter is inappropriate for the "end of input" character, so we suppress that. See also recent commit a2ec6be72b8 "json: Fix lexer to include the bad character in JSON_ERROR token". * The transition to a TERMINAL(JSON_FOO) state doesn't consume the input character. In that case, the lexer normally loops until it is consumed. We have to suppress that for the "end of input" input character. If we didn't, the lexer would consume it by entering IN_ERROR state, emitting a bogus JSON_ERROR token. We fixed that in commit bd3924a33a6. However, simply breaking the loop this way assumes that the lexer needs exactly one state transition to reach its start state. That assumption is correct now, but it's unclean, and I'll soon break it. Clean up: instead of breaking the loop after one iteration, break it after it reached the start state. Signed-off-by: Markus Armbruster Reviewed-by: Eric Blake Message-Id: <20180831075841.13363-3-armbru@redhat.com>

json: Fix lexer for lookahead character beyond '\x7F'

2018-09-24T16:06:09+00:00

The lexer fails to end a valid token when the lookahead character is beyond '\x7F'. For instance, input true\xC2\xA2 produces the tokens JSON_ERROR true\xC2 JSON_ERROR \xA2 This should be JSON_KEYWORD true JSON_ERROR \xC2 JSON_ERROR \xA2 instead. The culprit is #define TERMINAL(state) [0 ... 0x7F] = (state) It leaves [0x80..0xFF] zero, i.e. IN_ERROR. Has always been broken. Fix it to initialize the complete array. Signed-off-by: Markus Armbruster Reviewed-by: Eric Blake Message-Id: <20180831075841.13363-2-armbru@redhat.com>

json: Clean up headers

2018-08-24T18:26:37+00:00

The JSON parser has three public headers, json-lexer.h, json-parser.h, json-streamer.h. They all contain stuff that is of no interest outside qobject/json-*.c. Collect the public interface in include/qapi/qmp/json-parser.h, and everything else in qobject/json-parser-int.h. Signed-off-by: Markus Armbruster Reviewed-by: Eric Blake Message-Id: <20180823164025.12553-54-armbru@redhat.com>