1
0
mirror of https://github.com/nlohmann/json.git synced 2026-07-28 22:53:03 +04:00

Compare commits

..

4 Commits

Author SHA1 Message Date
Niels Lohmann 9ea2d28ef9 Reserve output capacity up front for binary serialization
The vector-returning to_cbor/to_msgpack/to_ubjson/to_bjdata/to_bson grew
the output buffer purely by geometric reallocation. Reserving an estimate
up front avoids the early reallocations, which is the dominant per-byte
cost for array/object-heavy output.

The estimate (binary_reserve_hint) is deliberately conservative and safe
against untrusted input: it consults only the top-level element count
(O(1), no walk of the DOM), guards the multiplication against overflow,
and clamps the result to a fixed 1 MiB ceiling, so a large or hostile DOM
can never force an oversized allocation here. The buffer still grows
geometrically past the hint, so an underestimate only costs a few later
reallocations; scalars/strings/binary are written in one shot and get no
hint. Reserving capacity does not change the bytes produced.

Throughput (g++/clang -O3, vs the previous commit):
  cbor int array     +10% / +13%
  cbor object array  +20% / +38%

Output is byte-for-byte identical to develop across the binary
differential corpus.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XAYM1qhSA2FDaDcGfPW3fG
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-07-22 11:01:03 +00:00
Niels Lohmann ac6b505923 Encode big-endian numbers with a byte swap instead of std::reverse
write_number() reordered multi-byte numbers for the big-endian formats
(CBOR/MessagePack/UBJSON) with std::reverse over the byte array. GCC
lowered only some sizes to a bswap; clang kept a scalar byte shuffle
(0 bswap instructions in the CBOR number path). Replace the reverse with
size-dispatched __builtin_bswap16/32/64 helpers (portable shift fallback
for other compilers; std::reverse retained for exotic sizes such as a
long double number_float_t).

Codegen: the CBOR number path now emits bswap on both compilers
(gcc 2 -> 16, clang 0 -> 4). Output is byte-for-byte identical to the
previous implementation across the binary differential corpus.

Throughput (isolated vs the std::reverse version, best of 9):
  CBOR int64 array   gcc +7%   clang +10%
  CBOR uint16 array  gcc +27%  clang flat

Modest but consistent on number-dense encodings; negligible on
string/blob-heavy output, as expected.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XAYM1qhSA2FDaDcGfPW3fG
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-07-22 10:20:33 +00:00
Niels Lohmann 7374730aed Fix CI failures from binary_writer output-sink change
Four CI jobs failed on the initial commit; all are addressed here without
changing any output (binary encodings remain byte-for-byte identical to
develop across the differential corpus):

1. ci_test_gcc / cuda (-Werror=duplicated-branches): for number_float_t ==
   float, static_cast<float>(n) is the identity, so write_compact_float's
   two branches are intentionally identical. Once the concrete vector sink
   is inlined, GCC constant-folds and diagnoses this (the type-erased path
   hid it behind a non-inlined virtual call). Silence -Wduplicated-branches
   for GCC (clang has no such warning) alongside the existing -Wfloat-equal
   pragma.

2. ci_static_analysis_clang (UBSan nonnull-attribute): binary_writer passes
   a null pointer with length 0 for empty strings/binary. output_vector_sink
   / output_adapter_sink declared write_characters JSON_HEDLEY_NON_NULL, so
   the sanitizer flagged the (harmless) zero-length call once the sink was
   called directly rather than through the attribute-free virtual base. Drop
   the attribute from both sinks, matching the pre-existing behavior.

3. ci_cpplint (build/include_what_you_use): output_adapter_sink uses
   std::move; add #include <utility>.

4. ci_cuda_example (nvcc 11.8): NVCC's front end rejects the default
   template argument on the binary_writer alias template. Revert the alias
   to its original single-parameter form (relying on binary_writer's own
   defaulted OutputSinkType) and spell out the full type in the vector-sink
   convenience functions.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XAYM1qhSA2FDaDcGfPW3fG
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-07-22 08:47:24 +00:00
Niels Lohmann ebb3abba41 Devirtualize binary_writer via a value-type output sink
to_cbor/to_msgpack/to_ubjson/to_bjdata/to_bson wrote every byte through
output_adapter_t, a shared_ptr<output_adapter_protocol> whose
write_character/write_characters are virtual. Unlike the lexer (templated
on a concrete InputAdapterType), the binary writer never got that
treatment, so binary output paid a vtable lookup per byte and a
make_shared per call.

Template binary_writer on an OutputSinkType and give it two concrete,
non-virtual sinks:

- output_vector_sink: appends straight into a std::vector (push_back /
  insert), used by the vector-returning to_* convenience functions. No
  vtable, no shared_ptr; the writes inline.
- output_adapter_sink: forwards to a type-erased output_adapter_t, so the
  existing to_*(j, output_adapter) overloads (streams, strings, custom
  adapters) keep working exactly as before -- one virtual call each,
  unchanged.

binary_writer keeps a convenience constructor taking output_adapter_t
(building the default output_adapter_sink), so the adapter overloads are
untouched; only the convenience functions switch to the vector sink. The
friend declaration and the basic_json binary_writer alias gain the new
(defaulted) template parameter.

Output is byte-for-byte identical: verified across ~3000 randomized
values plus curated edge cases (all scalar widths, strings with invalid
UTF-8, binary, nested arrays/objects) for CBOR, MessagePack, UBJSON (both
size/type settings), BJData, and BSON, plus the output_adapter path, in
C++11/17/20. Warning-clean under clang -Weverything and the gcc pedantic
set; clang-tidy clean on the changed headers; make check-amalgamation
clean.

Throughput (g++ -O3, vs develop): scalar-dense binary output such as
integer arrays ~1.4x; many small to_cbor calls ~1.04x (DOM traversal
bound); string/blob-heavy output unchanged (already bulk-bound). No
workload regressed.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XAYM1qhSA2FDaDcGfPW3fG
Signed-off-by: Niels Lohmann <mail@nlohmann.me>
2026-07-20 23:27:51 +00:00
25 changed files with 794 additions and 676 deletions
-8
View File
@@ -15,14 +15,6 @@ guidance.
For vulnerabilities in third-party dependencies or modules, please report them directly to the respective maintainers. For vulnerabilities in third-party dependencies or modules, please report them directly to the respective maintainers.
## Unofficial packages
This project does not publish an official npm package. The npm package
[`nlohmann-json`](https://www.npmjs.com/package/nlohmann-json) (or similarly named packages) is not maintained or
endorsed by this project. See the
[package managers documentation](https://json.nlohmann.me/integration/package_managers/#npm) for supported
integration options.
## Additional Resources ## Additional Resources
- Explore security-related topics and contribute to tools and projects through - Explore security-related topics and contribute to tools and projects through
+2 -2
View File
@@ -39,14 +39,14 @@ jobs:
egress-policy: audit egress-policy: audit
- name: Checkout pull request - name: Checkout pull request
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with: with:
path: main path: main
ref: ${{ github.event.pull_request.head.sha }} ref: ${{ github.event.pull_request.head.sha }}
persist-credentials: false persist-credentials: false
- name: Checkout tools - name: Checkout tools
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with: with:
path: tools path: tools
ref: develop ref: develop
+4 -4
View File
@@ -32,20 +32,20 @@ jobs:
egress-policy: audit egress-policy: audit
- name: Checkout repository - name: Checkout repository
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with: with:
persist-credentials: false persist-credentials: false
# Initializes the CodeQL tools for scanning. # Initializes the CodeQL tools for scanning.
- name: Initialize CodeQL - name: Initialize CodeQL
uses: github/codeql-action/init@7188fc363630916deb702c7fdcf4e481b751f97a # v4.37.1 uses: github/codeql-action/init@99df26d4f13ea111d4ec1a7dddef6063f76b97e9 # v4.37.0
with: with:
languages: c-cpp languages: c-cpp
# Autobuild attempts to build any compiled languages (C/C++, C#, or Java). # Autobuild attempts to build any compiled languages (C/C++, C#, or Java).
# If this step fails, then you should remove it and run the build manually (see below) # If this step fails, then you should remove it and run the build manually (see below)
- name: Autobuild - name: Autobuild
uses: github/codeql-action/autobuild@7188fc363630916deb702c7fdcf4e481b751f97a # v4.37.1 uses: github/codeql-action/autobuild@99df26d4f13ea111d4ec1a7dddef6063f76b97e9 # v4.37.0
- name: Perform CodeQL Analysis - name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@7188fc363630916deb702c7fdcf4e481b751f97a # v4.37.1 uses: github/codeql-action/analyze@99df26d4f13ea111d4ec1a7dddef6063f76b97e9 # v4.37.0
+1 -1
View File
@@ -22,7 +22,7 @@ jobs:
egress-policy: audit egress-policy: audit
- name: 'Checkout Repository' - name: 'Checkout Repository'
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with: with:
persist-credentials: false persist-credentials: false
- name: 'Dependency Review' - name: 'Dependency Review'
+2 -2
View File
@@ -32,7 +32,7 @@ jobs:
egress-policy: audit egress-policy: audit
- name: Checkout code - name: Checkout code
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with: with:
persist-credentials: false persist-credentials: false
@@ -43,6 +43,6 @@ jobs:
output: 'flawfinder_results.sarif' output: 'flawfinder_results.sarif'
- name: Upload analysis results to GitHub Security tab - name: Upload analysis results to GitHub Security tab
uses: github/codeql-action/upload-sarif@7188fc363630916deb702c7fdcf4e481b751f97a # v4.37.1 uses: github/codeql-action/upload-sarif@99df26d4f13ea111d4ec1a7dddef6063f76b97e9 # v4.37.0
with: with:
sarif_file: ${{github.workspace}}/flawfinder_results.sarif sarif_file: ${{github.workspace}}/flawfinder_results.sarif
+3 -3
View File
@@ -26,7 +26,7 @@ jobs:
DEVELOPER_DIR: /Applications/Xcode_${{ matrix.xcode }}.app/Contents/Developer DEVELOPER_DIR: /Applications/Xcode_${{ matrix.xcode }}.app/Contents/Developer
steps: steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with: with:
persist-credentials: false persist-credentials: false
- name: Run CMake - name: Run CMake
@@ -45,7 +45,7 @@ jobs:
DEVELOPER_DIR: /Applications/Xcode_${{ matrix.xcode }}.app/Contents/Developer DEVELOPER_DIR: /Applications/Xcode_${{ matrix.xcode }}.app/Contents/Developer
steps: steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with: with:
persist-credentials: false persist-credentials: false
- name: Run CMake - name: Run CMake
@@ -62,7 +62,7 @@ jobs:
standard: [11, 14, 17, 20, 23, 26] standard: [11, 14, 17, 20, 23, 26]
steps: steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with: with:
persist-credentials: false persist-credentials: false
- name: Run CMake - name: Run CMake
+1 -1
View File
@@ -31,7 +31,7 @@ jobs:
with: with:
egress-policy: audit egress-policy: audit
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
- name: Install virtual environment - name: Install virtual environment
run: make install_venv -C docs/mkdocs run: make install_venv -C docs/mkdocs
+2 -2
View File
@@ -41,7 +41,7 @@ jobs:
egress-policy: audit egress-policy: audit
- name: "Checkout code" - name: "Checkout code"
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with: with:
persist-credentials: false persist-credentials: false
@@ -76,6 +76,6 @@ jobs:
# Upload the results to GitHub's code scanning dashboard. # Upload the results to GitHub's code scanning dashboard.
- name: "Upload to code-scanning" - name: "Upload to code-scanning"
uses: github/codeql-action/upload-sarif@7188fc363630916deb702c7fdcf4e481b751f97a # v4.37.1 uses: github/codeql-action/upload-sarif@99df26d4f13ea111d4ec1a7dddef6063f76b97e9 # v4.37.0
with: with:
sarif_file: results.sarif sarif_file: results.sarif
+2 -2
View File
@@ -37,7 +37,7 @@ jobs:
egress-policy: audit egress-policy: audit
# Checkout project source # Checkout project source
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with: with:
persist-credentials: false persist-credentials: false
@@ -61,7 +61,7 @@ jobs:
# Upload SARIF file generated in previous step # Upload SARIF file generated in previous step
- name: Upload SARIF file - name: Upload SARIF file
uses: github/codeql-action/upload-sarif@7188fc363630916deb702c7fdcf4e481b751f97a # v4.37.1 uses: github/codeql-action/upload-sarif@99df26d4f13ea111d4ec1a7dddef6063f76b97e9 # v4.37.0
with: with:
sarif_file: semgrep.sarif sarif_file: semgrep.sarif
if: always() if: always()
+35 -43
View File
@@ -21,11 +21,11 @@ jobs:
runs-on: ubuntu-latest runs-on: ubuntu-latest
container: gcc:latest container: gcc:latest
steps: steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with: with:
persist-credentials: false persist-credentials: false
- name: Get latest CMake and ninja - name: Get latest CMake and ninja
uses: lukka/get-cmake@e6906078ebd1ccb8ce51ab4626ac46a1b5a517e3 # v4.4.0 uses: lukka/get-cmake@f5b8fbb4d77cec1acc5a5f9f0df4beffaf5d98d9 # v4.3.4
- name: Run CMake - name: Run CMake
run: cmake -S . -B build -DJSON_CI=On run: cmake -S . -B build -DJSON_CI=On
- name: Build - name: Build
@@ -43,11 +43,11 @@ jobs:
run: | run: |
wget -q -O - "https://github.com/facebook/infer/releases/download/v1.3.0/infer-linux-x86_64-v1.3.0.tar.xz" | sudo tar -C /opt -xJ wget -q -O - "https://github.com/facebook/infer/releases/download/v1.3.0/infer-linux-x86_64-v1.3.0.tar.xz" | sudo tar -C /opt -xJ
sudo ln -s /opt/infer-linux-x86_64-v1.3.0/bin/infer /usr/local/bin/infer sudo ln -s /opt/infer-linux-x86_64-v1.3.0/bin/infer /usr/local/bin/infer
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with: with:
persist-credentials: false persist-credentials: false
- name: Get latest CMake and ninja - name: Get latest CMake and ninja
uses: lukka/get-cmake@e6906078ebd1ccb8ce51ab4626ac46a1b5a517e3 # v4.4.0 uses: lukka/get-cmake@f5b8fbb4d77cec1acc5a5f9f0df4beffaf5d98d9 # v4.3.4
- name: Run CMake - name: Run CMake
run: cmake -S . -B build -DJSON_CI=On run: cmake -S . -B build -DJSON_CI=On
- name: Build - name: Build
@@ -66,11 +66,11 @@ jobs:
- name: Install Valgrind - name: Install Valgrind
run: sudo apt-get update ; sudo apt-get install -y valgrind run: sudo apt-get update ; sudo apt-get install -y valgrind
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with: with:
persist-credentials: false persist-credentials: false
- name: Get latest CMake and ninja - name: Get latest CMake and ninja
uses: lukka/get-cmake@e6906078ebd1ccb8ce51ab4626ac46a1b5a517e3 # v4.4.0 uses: lukka/get-cmake@f5b8fbb4d77cec1acc5a5f9f0df4beffaf5d98d9 # v4.3.4
- name: Run CMake - name: Run CMake
run: cmake -S . -B build -DJSON_CI=On run: cmake -S . -B build -DJSON_CI=On
- name: Build - name: Build
@@ -85,11 +85,11 @@ jobs:
steps: steps:
- name: Install git, clang-tools, iwyu (ci_single_binaries), and unzip - name: Install git, clang-tools, iwyu (ci_single_binaries), and unzip
run: apt-get update ; apt-get install -y git clang-tools iwyu unzip run: apt-get update ; apt-get install -y git clang-tools iwyu unzip
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with: with:
persist-credentials: false persist-credentials: false
- name: Get latest CMake and ninja - name: Get latest CMake and ninja
uses: lukka/get-cmake@e6906078ebd1ccb8ce51ab4626ac46a1b5a517e3 # v4.4.0 uses: lukka/get-cmake@f5b8fbb4d77cec1acc5a5f9f0df4beffaf5d98d9 # v4.3.4
- name: Run CMake - name: Run CMake
run: cmake -S . -B build -DJSON_CI=On run: cmake -S . -B build -DJSON_CI=On
- name: Build - name: Build
@@ -104,11 +104,11 @@ jobs:
steps: steps:
- name: Install build-essential - name: Install build-essential
run: apt-get update ; apt-get install -y build-essential unzip wget git libssl-dev run: apt-get update ; apt-get install -y build-essential unzip wget git libssl-dev
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with: with:
persist-credentials: false persist-credentials: false
- name: Get latest CMake and ninja - name: Get latest CMake and ninja
uses: lukka/get-cmake@e6906078ebd1ccb8ce51ab4626ac46a1b5a517e3 # v4.4.0 uses: lukka/get-cmake@f5b8fbb4d77cec1acc5a5f9f0df4beffaf5d98d9 # v4.3.4
- name: Run CMake - name: Run CMake
run: cmake -S . -B build -DJSON_CI=On run: cmake -S . -B build -DJSON_CI=On
- name: Build - name: Build
@@ -122,7 +122,7 @@ jobs:
with: with:
egress-policy: audit egress-policy: audit
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with: with:
persist-credentials: false persist-credentials: false
- name: Install dependencies and de_DE locale - name: Install dependencies and de_DE locale
@@ -163,15 +163,7 @@ jobs:
export DEBIAN_FRONTEND=noninteractive export DEBIAN_FRONTEND=noninteractive
apt-get update apt-get update
apt-get install -y --no-install-recommends software-properties-common ca-certificates gnupg make git apt-get install -y --no-install-recommends software-properties-common ca-certificates gnupg make git
# add-apt-repository resolves the PPA through the Launchpad API, add-apt-repository -y ppa:ubuntu-toolchain-r/test
# which intermittently times out or fails the team lookup (the plain
# "deb ..." sources below never hit Launchpad and never flake).
# Retry with backoff so a transient Launchpad blip does not fail CI.
for attempt in 1 2 3 4 5; do
add-apt-repository -y ppa:ubuntu-toolchain-r/test && break
echo "::warning::add-apt-repository ppa:ubuntu-toolchain-r/test failed (attempt ${attempt}/5); retrying"
sleep $((attempt * 10))
done
apt-add-repository -y "deb http://archive.ubuntu.com/ubuntu/ bionic main" apt-add-repository -y "deb http://archive.ubuntu.com/ubuntu/ bionic main"
apt-add-repository -y "deb http://archive.ubuntu.com/ubuntu/ bionic universe" apt-add-repository -y "deb http://archive.ubuntu.com/ubuntu/ bionic universe"
apt-add-repository -y "deb http://archive.ubuntu.com/ubuntu/ xenial main" apt-add-repository -y "deb http://archive.ubuntu.com/ubuntu/ xenial main"
@@ -180,11 +172,11 @@ jobs:
apt-add-repository -y "deb http://archive.ubuntu.com/ubuntu/ xenial-updates universe" apt-add-repository -y "deb http://archive.ubuntu.com/ubuntu/ xenial-updates universe"
apt-get update apt-get update
apt-get install -y --no-install-recommends g++-${{ matrix.compiler }} apt-get install -y --no-install-recommends g++-${{ matrix.compiler }}
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with: with:
persist-credentials: false persist-credentials: false
- name: Get latest CMake and ninja - name: Get latest CMake and ninja
uses: lukka/get-cmake@e6906078ebd1ccb8ce51ab4626ac46a1b5a517e3 # v4.4.0 uses: lukka/get-cmake@f5b8fbb4d77cec1acc5a5f9f0df4beffaf5d98d9 # v4.3.4
- name: Run CMake - name: Run CMake
run: CXX=g++-${{ matrix.compiler }} cmake -S . -B build -DJSON_CI=On run: CXX=g++-${{ matrix.compiler }} cmake -S . -B build -DJSON_CI=On
- name: Build - name: Build
@@ -198,11 +190,11 @@ jobs:
compiler: ['7', '8', '9', '10', '11', '12', '13', '14', '15', 'latest'] compiler: ['7', '8', '9', '10', '11', '12', '13', '14', '15', 'latest']
container: gcc:${{ matrix.compiler }} container: gcc:${{ matrix.compiler }}
steps: steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with: with:
persist-credentials: false persist-credentials: false
- name: Get latest CMake and ninja - name: Get latest CMake and ninja
uses: lukka/get-cmake@e6906078ebd1ccb8ce51ab4626ac46a1b5a517e3 # v4.4.0 uses: lukka/get-cmake@f5b8fbb4d77cec1acc5a5f9f0df4beffaf5d98d9 # v4.3.4
- name: Run CMake - name: Run CMake
run: cmake -S . -B build -DJSON_CI=On run: cmake -S . -B build -DJSON_CI=On
- name: Build - name: Build
@@ -215,11 +207,11 @@ jobs:
compiler: ['3.4', '3.5', '3.6', '3.7', '3.8', '3.9', '4', '5', '6', '7', '8', '9', '10', '11', '12', '13', '14', '15-bullseye', '16', '17', '18', '19', '20', 'latest'] compiler: ['3.4', '3.5', '3.6', '3.7', '3.8', '3.9', '4', '5', '6', '7', '8', '9', '10', '11', '12', '13', '14', '15-bullseye', '16', '17', '18', '19', '20', 'latest']
container: silkeh/clang:${{ matrix.compiler }} container: silkeh/clang:${{ matrix.compiler }}
steps: steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with: with:
persist-credentials: false persist-credentials: false
- name: Get latest CMake and ninja - name: Get latest CMake and ninja
uses: lukka/get-cmake@e6906078ebd1ccb8ce51ab4626ac46a1b5a517e3 # v4.4.0 uses: lukka/get-cmake@f5b8fbb4d77cec1acc5a5f9f0df4beffaf5d98d9 # v4.3.4
- name: Set env FORCE_STDCPPFS_FLAG for clang 7 / 8 / 9 / 10 - name: Set env FORCE_STDCPPFS_FLAG for clang 7 / 8 / 9 / 10
run: echo "JSON_FORCED_GLOBAL_COMPILE_OPTIONS=-DJSON_HAS_FILESYSTEM=0;-DJSON_HAS_EXPERIMENTAL_FILESYSTEM=0" >> "$GITHUB_ENV" run: echo "JSON_FORCED_GLOBAL_COMPILE_OPTIONS=-DJSON_HAS_FILESYSTEM=0;-DJSON_HAS_EXPERIMENTAL_FILESYSTEM=0" >> "$GITHUB_ENV"
if: ${{ matrix.compiler == '7' || matrix.compiler == '8' || matrix.compiler == '9' || matrix.compiler == '10' }} if: ${{ matrix.compiler == '7' || matrix.compiler == '8' || matrix.compiler == '9' || matrix.compiler == '10' }}
@@ -235,11 +227,11 @@ jobs:
matrix: matrix:
standard: [11, 14, 17, 20, 23, 26] standard: [11, 14, 17, 20, 23, 26]
steps: steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with: with:
persist-credentials: false persist-credentials: false
- name: Get latest CMake and ninja - name: Get latest CMake and ninja
uses: lukka/get-cmake@e6906078ebd1ccb8ce51ab4626ac46a1b5a517e3 # v4.4.0 uses: lukka/get-cmake@f5b8fbb4d77cec1acc5a5f9f0df4beffaf5d98d9 # v4.3.4
- name: Run CMake - name: Run CMake
run: cmake -S . -B build -DJSON_CI=On run: cmake -S . -B build -DJSON_CI=On
- name: Build - name: Build
@@ -255,11 +247,11 @@ jobs:
steps: steps:
- name: Install git and unzip - name: Install git and unzip
run: apt-get update ; apt-get install -y git unzip run: apt-get update ; apt-get install -y git unzip
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with: with:
persist-credentials: false persist-credentials: false
- name: Get latest CMake and ninja - name: Get latest CMake and ninja
uses: lukka/get-cmake@e6906078ebd1ccb8ce51ab4626ac46a1b5a517e3 # v4.4.0 uses: lukka/get-cmake@f5b8fbb4d77cec1acc5a5f9f0df4beffaf5d98d9 # v4.3.4
- name: Run CMake - name: Run CMake
run: cmake -S . -B build -DJSON_CI=On run: cmake -S . -B build -DJSON_CI=On
- name: Build with libc++ - name: Build with libc++
@@ -282,11 +274,11 @@ jobs:
cuda: ['11.8.0', '12.1.1', '12.6.3'] cuda: ['11.8.0', '12.1.1', '12.6.3']
container: nvidia/cuda:${{ matrix.cuda }}-devel-ubuntu22.04 container: nvidia/cuda:${{ matrix.cuda }}-devel-ubuntu22.04
steps: steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with: with:
persist-credentials: false persist-credentials: false
- name: Get latest CMake and ninja - name: Get latest CMake and ninja
uses: lukka/get-cmake@e6906078ebd1ccb8ce51ab4626ac46a1b5a517e3 # v4.4.0 uses: lukka/get-cmake@f5b8fbb4d77cec1acc5a5f9f0df4beffaf5d98d9 # v4.3.4
- name: Run CMake - name: Run CMake
run: cmake -S . -B build -DJSON_CI=On run: cmake -S . -B build -DJSON_CI=On
- name: Build - name: Build
@@ -299,14 +291,14 @@ jobs:
runs-on: ubuntu-latest runs-on: ubuntu-latest
container: ${{ matrix.container }} container: ${{ matrix.container }}
steps: steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with: with:
persist-credentials: false persist-credentials: false
# The module test uses `import std;`, which needs CMake's experimental # The module test uses `import std;`, which needs CMake's experimental
# import-std support. Its opt-in token is CMake-version-specific, so pin # import-std support. Its opt-in token is CMake-version-specific, so pin
# CMake to the version whose token is set in tests/module_cpp20/CMakeLists.txt. # CMake to the version whose token is set in tests/module_cpp20/CMakeLists.txt.
- name: Get pinned CMake and ninja - name: Get pinned CMake and ninja
uses: lukka/get-cmake@e6906078ebd1ccb8ce51ab4626ac46a1b5a517e3 # v4.4.0 uses: lukka/get-cmake@f5b8fbb4d77cec1acc5a5f9f0df4beffaf5d98d9 # v4.3.4
with: with:
cmakeVersion: 4.3.4 cmakeVersion: 4.3.4
# Clang: the std library module is provided by libc++ (the image's libstdc++ # Clang: the std library module is provided by libc++ (the image's libstdc++
@@ -328,11 +320,11 @@ jobs:
# Intel's own last officially published image that still includes it. # Intel's own last officially published image that still includes it.
container: intel/oneapi-hpckit:2023.2.1-devel-ubuntu22.04 container: intel/oneapi-hpckit:2023.2.1-devel-ubuntu22.04
steps: steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with: with:
persist-credentials: false persist-credentials: false
- name: Get latest CMake and ninja - name: Get latest CMake and ninja
uses: lukka/get-cmake@e6906078ebd1ccb8ce51ab4626ac46a1b5a517e3 # v4.4.0 uses: lukka/get-cmake@f5b8fbb4d77cec1acc5a5f9f0df4beffaf5d98d9 # v4.3.4
- name: Run CMake - name: Run CMake
run: cmake -S . -B build -DJSON_CI=On run: cmake -S . -B build -DJSON_CI=On
- name: Build - name: Build
@@ -345,9 +337,9 @@ jobs:
runs-on: ubuntu-latest runs-on: ubuntu-latest
container: intel/oneapi-hpckit:latest container: intel/oneapi-hpckit:latest
steps: steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
- name: Get latest CMake and ninja - name: Get latest CMake and ninja
uses: lukka/get-cmake@e6906078ebd1ccb8ce51ab4626ac46a1b5a517e3 # v4.4.0 uses: lukka/get-cmake@f5b8fbb4d77cec1acc5a5f9f0df4beffaf5d98d9 # v4.3.4
- name: Run CMake - name: Run CMake
run: cmake -S . -B build -DJSON_CI=On run: cmake -S . -B build -DJSON_CI=On
- name: Build - name: Build
@@ -357,9 +349,9 @@ jobs:
runs-on: ubuntu-latest runs-on: ubuntu-latest
container: nvcr.io/nvidia/nvhpc:25.5-devel-cuda12.9-ubuntu22.04 container: nvcr.io/nvidia/nvhpc:25.5-devel-cuda12.9-ubuntu22.04
steps: steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
- name: Get latest CMake and ninja - name: Get latest CMake and ninja
uses: lukka/get-cmake@e6906078ebd1ccb8ce51ab4626ac46a1b5a517e3 # v4.4.0 uses: lukka/get-cmake@f5b8fbb4d77cec1acc5a5f9f0df4beffaf5d98d9 # v4.3.4
- name: Run CMake - name: Run CMake
run: cmake -S . -B build -DJSON_CI=On run: cmake -S . -B build -DJSON_CI=On
- name: Build - name: Build
@@ -375,11 +367,11 @@ jobs:
- name: Install emscripten - name: Install emscripten
uses: mymindstorm/setup-emsdk@4528d102f7230f0e7b276855c01ea1159be0e984 # v16 uses: mymindstorm/setup-emsdk@4528d102f7230f0e7b276855c01ea1159be0e984 # v16
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with: with:
persist-credentials: false persist-credentials: false
- name: Get latest CMake and ninja - name: Get latest CMake and ninja
uses: lukka/get-cmake@e6906078ebd1ccb8ce51ab4626ac46a1b5a517e3 # v4.4.0 uses: lukka/get-cmake@f5b8fbb4d77cec1acc5a5f9f0df4beffaf5d98d9 # v4.3.4
- name: Run CMake - name: Run CMake
run: cmake -S . -B build -DCMAKE_TOOLCHAIN_FILE=$EMSDK/upstream/emscripten/cmake/Modules/Platform/Emscripten.cmake -GNinja run: cmake -S . -B build -DCMAKE_TOOLCHAIN_FILE=$EMSDK/upstream/emscripten/cmake/Modules/Platform/Emscripten.cmake -GNinja
- name: Build - name: Build
@@ -396,7 +388,7 @@ jobs:
with: with:
egress-policy: audit egress-policy: audit
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with: with:
persist-credentials: false persist-credentials: false
- name: Run CMake - name: Run CMake
+9 -9
View File
@@ -24,7 +24,7 @@ jobs:
architecture: [x64, x86] architecture: [x64, x86]
steps: steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with: with:
persist-credentials: false persist-credentials: false
- name: Set up MinGW - name: Set up MinGW
@@ -49,7 +49,7 @@ jobs:
runs-on: windows-2022 runs-on: windows-2022
steps: steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with: with:
persist-credentials: false persist-credentials: false
- name: Set extra CXX_FLAGS for latest std_version - name: Set extra CXX_FLAGS for latest std_version
@@ -86,9 +86,9 @@ jobs:
runs-on: windows-2025 runs-on: windows-2025
steps: steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
- name: Get latest CMake and ninja - name: Get latest CMake and ninja
uses: lukka/get-cmake@e6906078ebd1ccb8ce51ab4626ac46a1b5a517e3 # v4.4.0 uses: lukka/get-cmake@f5b8fbb4d77cec1acc5a5f9f0df4beffaf5d98d9 # v4.3.4
- name: Set extra CXX_FLAGS for latest std_version - name: Set extra CXX_FLAGS for latest std_version
# /wd5285 silences C5285 emitted by the bundled third-party doctest.h, which # /wd5285 silences C5285 emitted by the bundled third-party doctest.h, which
# specializes std::tuple (newly diagnosed by the VS2026 v145 toolset) # specializes std::tuple (newly diagnosed by the VS2026 v145 toolset)
@@ -122,7 +122,7 @@ jobs:
runs-on: windows-11-arm runs-on: windows-11-arm
steps: steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
- name: Run CMake (Release) - name: Run CMake (Release)
run: cmake -S . -B build -G "Visual Studio 17 2022" -A ARM64 -DJSON_BuildTests=On -DCMAKE_CXX_FLAGS="/W4 /WX" run: cmake -S . -B build -G "Visual Studio 17 2022" -A ARM64 -DJSON_BuildTests=On -DCMAKE_CXX_FLAGS="/W4 /WX"
if: matrix.build_type == 'Release' if: matrix.build_type == 'Release'
@@ -143,7 +143,7 @@ jobs:
version: [11.0.1, 12.0.1, 13.0.1, 14.0.6, 15.0.7, 16.0.6, 18.1.8, 19.1.7, 20.1.8] version: [11.0.1, 12.0.1, 13.0.1, 14.0.6, 15.0.7, 16.0.6, 18.1.8, 19.1.7, 20.1.8]
steps: steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with: with:
persist-credentials: false persist-credentials: false
- name: Install Clang - name: Install Clang
@@ -173,7 +173,7 @@ jobs:
architecture: [Win32, x64] architecture: [Win32, x64]
steps: steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with: with:
persist-credentials: false persist-credentials: false
- name: Run CMake - name: Run CMake
@@ -186,14 +186,14 @@ jobs:
ci_module_cpp20: ci_module_cpp20:
runs-on: windows-2022 runs-on: windows-2022
steps: steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 - uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # v7.0.0
with: with:
persist-credentials: false persist-credentials: false
# The module test uses `import std;`, which needs CMake's experimental # The module test uses `import std;`, which needs CMake's experimental
# import-std support. Its opt-in token is CMake-version-specific, so pin # import-std support. Its opt-in token is CMake-version-specific, so pin
# CMake to the version whose token is set in tests/module_cpp20/CMakeLists.txt. # CMake to the version whose token is set in tests/module_cpp20/CMakeLists.txt.
- name: Get pinned CMake and ninja - name: Get pinned CMake and ninja
uses: lukka/get-cmake@e6906078ebd1ccb8ce51ab4626ac46a1b5a517e3 # v4.4.0 uses: lukka/get-cmake@f5b8fbb4d77cec1acc5a5f9f0df4beffaf5d98d9 # v4.3.4
with: with:
cmakeVersion: 4.3.4 cmakeVersion: 4.3.4
- name: Run CMake (Debug) - name: Run CMake (Debug)
+1
View File
@@ -90,6 +90,7 @@ You can sponsor this library at [GitHub Sponsors](https://github.com/sponsors/nl
- [Steve Sperandeo](https://github.com/homer6) - [Steve Sperandeo](https://github.com/homer6)
- [Robert Jefe Lindstädt](https://github.com/eljefedelrodeodeljefe) - [Robert Jefe Lindstädt](https://github.com/eljefedelrodeodeljefe)
- [Steve Wagner](https://github.com/ciroque) - [Steve Wagner](https://github.com/ciroque)
- [Lion Yang](https://github.com/LionNatsu)
### Further support ### Further support
-11
View File
@@ -43,17 +43,6 @@ Strong guarantee: if an exception is thrown, there are no changes to any JSON va
Throws [`type_error.316`](../../home/exceptions.md#jsonexceptiontype_error316) if a string stored inside the JSON value Throws [`type_error.316`](../../home/exceptions.md#jsonexceptiontype_error316) if a string stored inside the JSON value
is not UTF-8 encoded and `error_handler` is set to `strict` is not UTF-8 encoded and `error_handler` is set to `strict`
!!! warning "Serializing untrusted input"
When serializing values that may contain invalid or untrusted UTF-8 (e.g., bytes taken directly from network
input), `dump()` throws [`type_error.316`](../../home/exceptions.md#jsonexceptiontype_error316) in the default
`strict` mode. To serialize such data without throwing, pass
[`error_handler_t::replace`](error_handler_t.md) (substitutes U+FFFD) or
[`error_handler_t::ignore`](error_handler_t.md). Callers that serialize untrusted input on a crash-sensitive path
should either choose a non-strict error handler or wrap `dump()` in a `#!cpp try`/`#!cpp catch`.
See the [FAQ](../../home/faq.md#serializing-untrusted-or-invalid-utf-8) for details.
## Complexity ## Complexity
Linear. Linear.
@@ -40,9 +40,6 @@ Strong guarantee: if an exception is thrown, there are no changes in the JSON va
is not an object; example: `"to serialize to BSON, top-level type must be object, but is string"` is not an object; example: `"to serialize to BSON, top-level type must be object, but is string"`
- Throws [`out_of_range.409`](../../home/exceptions.md#jsonexceptionout_of_range409) if a key in the JSON object contains - Throws [`out_of_range.409`](../../home/exceptions.md#jsonexceptionout_of_range409) if a key in the JSON object contains
a null byte (code point U+0000); example: `"BSON key cannot contain code point U+0000 (at byte 2)"` a null byte (code point U+0000); example: `"BSON key cannot contain code point U+0000 (at byte 2)"`
- Throws [`out_of_range.412`](../../home/exceptions.md#jsonexceptionout_of_range412) if the length of a document, array,
string, or binary value exceeds the range of the 32-bit BSON length field; example:
`"BSON length 2147483661 exceeds maximum of 2147483647"`
## Complexity ## Complexity
-16
View File
@@ -898,22 +898,6 @@ A JSON Patch `add` operation cannot be applied because the target location's par
This exception was added in version 3.13.0. Before that, this situation hit an internal assertion (aborting the program in debug builds) or was silently ignored when assertions were disabled. This exception was added in version 3.13.0. Before that, this situation hit an internal assertion (aborting the program in debug builds) or was silently ignored when assertions were disabled.
### json.exception.out_of_range.412
BSON stores the length of documents, arrays, strings, and binary values in a signed 32-bit integer. This exception is thrown when a value is too large to be described by such a length field.
!!! failure "Example message"
```
BSON length 2147483661 exceeds maximum of 2147483647
```
!!! note
This exception was added in version 3.13.0. Before that, the length was silently truncated, and
[`to_bson`](../api/basic_json/to_bson.md) produced documents with negative length prefixes that
[`from_bson`](../api/basic_json/from_bson.md) rejected.
## Further exceptions ## Further exceptions
This exception is thrown in case of errors that cannot be classified with the This exception is thrown in case of errors that cannot be classified with the
-21
View File
@@ -194,27 +194,6 @@ The library uses `std::numeric_limits<number_float_t>::digits10` (15 for IEEE `d
See [this section](../features/types/number_handling.md#number-serialization) on the library's number handling for more information. See [this section](../features/types/number_handling.md#number-serialization) on the library's number handling for more information.
### Serializing untrusted or invalid UTF-8
!!! question "Questions"
- Why does `dump()` throw when I serialize data that came from the network?
- Is CVE-2024-34363 a vulnerability in this library?
Crashes reported against this library that stem from an uncaught
[`type_error.316`](exceptions.md#jsonexceptiontype_error316) while serializing unvalidated input (e.g.,
CVE-2024-34363) are a usage issue, not a library vulnerability:
[`dump()`](../api/basic_json/dump.md) throws in its default `strict` mode because
[RFC 8259](https://datatracker.ietf.org/doc/html/rfc8259#section-8.1) requires JSON text to be valid UTF-8.
The recommended pattern is to pass a non-strict [`error_handler`](../api/basic_json/error_handler_t.md) or to handle the
exception:
```cpp
// replace invalid sequences with U+FFFD instead of throwing
const auto s = j.dump(-1, ' ', false, json::error_handler_t::replace);
```
### Using JSON values with `std::format` or `fmt` ### Using JSON values with `std::format` or `fmt`
!!! question !!! question
+1
View File
@@ -14,5 +14,6 @@ You can sponsor this library at [GitHub Sponsors](https://github.com/sponsors/nl
- [Steve Sperandeo](https://github.com/homer6) - [Steve Sperandeo](https://github.com/homer6)
- [Robert Jefe Lindstädt](https://github.com/eljefedelrodeodeljefe) - [Robert Jefe Lindstädt](https://github.com/eljefedelrodeodeljefe)
- [Steve Wagner](https://github.com/ciroque) - [Steve Wagner](https://github.com/ciroque)
- [Lion Yang](https://github.com/LionNatsu)
Thanks everyone! Thanks everyone!
@@ -930,12 +930,6 @@ If you are using [CocoaPods](https://cocoapods.org), you can use the library by
to your podfile (see [an example](https://bitbucket.org/benman/nlohmann_json-cocoapod/src/master/)). Please file issues to your podfile (see [an example](https://bitbucket.org/benman/nlohmann_json-cocoapod/src/master/)). Please file issues
[here](https://bitbucket.org/benman/nlohmann_json-cocoapod/issues?status=new&status=open). [here](https://bitbucket.org/benman/nlohmann_json-cocoapod/issues?status=new&status=open).
## npm
This project does not publish an official [npm](https://www.npmjs.com) package. The npm package
[`nlohmann-json`](https://www.npmjs.com/package/nlohmann-json) (or similarly named packages) is not maintained or
endorsed by this project. Use one of the package managers listed above, or integrate the single header directly.
## ESP-IDF and PlatformIO ## ESP-IDF and PlatformIO
There is no official package published to the [ESP-IDF Component Registry](https://components.espressif.com) or the There is no official package published to the [ESP-IDF Component Registry](https://components.espressif.com) or the
+1 -1
View File
@@ -2,7 +2,7 @@ wheel==0.47.0
mkdocs==1.6.1 # documentation framework mkdocs==1.6.1 # documentation framework
mkdocs-git-revision-date-localized-plugin==1.5.3 # plugin "git-revision-date-localized" mkdocs-git-revision-date-localized-plugin==1.5.3 # plugin "git-revision-date-localized"
mkdocs-material==9.7.7 # theme for mkdocs mkdocs-material==9.7.6 # theme for mkdocs
mkdocs-material-extensions==1.3.1 # extensions mkdocs-material-extensions==1.3.1 # extensions
mkdocs-minify-plugin==0.8.0 # plugin "minify" mkdocs-minify-plugin==0.8.0 # plugin "minify"
mkdocs-redirects==1.2.3 # plugin "redirects" mkdocs-redirects==1.2.3 # plugin "redirects"
@@ -163,39 +163,12 @@ class binary_reader
// BSON // // BSON //
////////// //////////
/*!
@brief Validate a BSON document's declared size against the bytes read.
A BSON document starts with an int32 that counts its own total length in
bytes, including that prefix and the trailing 0x00. The reader is driven
by the terminator rather than the declared length, so without this check a
nested document could declare a length that disagrees with where its
terminator actually falls and quietly hand the bytes in between to the
enclosing document. A well-formed document is at least 5 bytes (the prefix
plus the terminator); the equality also rejects those impossible sizes,
since at least 5 bytes are always consumed.
@param[in] document_start value of chars_read before the size prefix
@param[in] document_size the declared document size
@return whether the declared size matches the number of bytes read
*/
bool check_bson_document_size(const std::size_t document_start, const std::int32_t document_size)
{
if (JSON_HEDLEY_UNLIKELY(document_size < 0 || static_cast<std::size_t>(document_size) != chars_read - document_start))
{
return sax->parse_error(chars_read, get_token_string(), parse_error::create(112, chars_read,
exception_message(input_format_t::bson, concat("document size ", std::to_string(document_size), " does not match the number of bytes read (", std::to_string(chars_read - document_start), ")"), "document"), nullptr));
}
return true;
}
/*! /*!
@brief Reads in a BSON-object and passes it to the SAX-parser. @brief Reads in a BSON-object and passes it to the SAX-parser.
@return whether a valid BSON-value was passed to the SAX parser @return whether a valid BSON-value was passed to the SAX parser
*/ */
bool parse_bson_internal() bool parse_bson_internal()
{ {
const std::size_t document_start = chars_read;
std::int32_t document_size{}; std::int32_t document_size{};
get_number<std::int32_t, true>(input_format_t::bson, document_size); get_number<std::int32_t, true>(input_format_t::bson, document_size);
@@ -209,11 +182,6 @@ class binary_reader
return false; return false;
} }
if (JSON_HEDLEY_UNLIKELY(!check_bson_document_size(document_start, document_size)))
{
return false;
}
return sax->end_object(); return sax->end_object();
} }
@@ -429,7 +397,6 @@ class binary_reader
*/ */
bool parse_bson_array() bool parse_bson_array()
{ {
const std::size_t document_start = chars_read;
std::int32_t document_size{}; std::int32_t document_size{};
get_number<std::int32_t, true>(input_format_t::bson, document_size); get_number<std::int32_t, true>(input_format_t::bson, document_size);
@@ -443,11 +410,6 @@ class binary_reader
return false; return false;
} }
if (JSON_HEDLEY_UNLIKELY(!check_bson_document_size(document_start, document_size)))
{
return false;
}
return sax->end_array(); return sax->end_array();
} }
File diff suppressed because it is too large Load Diff
@@ -13,6 +13,7 @@
#include <iterator> // back_inserter #include <iterator> // back_inserter
#include <memory> // shared_ptr, make_shared #include <memory> // shared_ptr, make_shared
#include <string> // basic_string #include <string> // basic_string
#include <utility> // move
#include <vector> // vector #include <vector> // vector
#ifndef JSON_NO_IO #ifndef JSON_NO_IO
@@ -118,6 +119,72 @@ class output_string_adapter : public output_adapter_protocol<CharType>
StringType& str; StringType& str;
}; };
/// @brief non-virtual output sink writing into a std::vector
///
/// Unlike output_vector_adapter, this sink is not part of the virtual
/// output_adapter_protocol hierarchy: it is passed to binary_writer by value as
/// a template parameter, so write_character()/write_characters() are ordinary
/// (inlinable) calls with no vtable lookup and no shared_ptr. It is used for the
/// common `to_cbor`/`to_msgpack`/... into a std::vector.
template<typename CharType, typename AllocatorType = std::allocator<CharType>>
class output_vector_sink
{
public:
explicit output_vector_sink(std::vector<CharType, AllocatorType>& vec) noexcept
: v(vec)
{}
void write_character(CharType c)
{
v.push_back(c);
}
// no JSON_HEDLEY_NON_NULL here: binary_writer legitimately passes a null
// pointer with length 0 for empty strings/binary values. Appending an empty
// range is a no-op; the type-erased path tolerates this via the (unattributed)
// virtual base, and the concrete sink must do the same.
void write_characters(const CharType* s, std::size_t length)
{
v.insert(v.end(), s, s + length);
}
private:
std::vector<CharType, AllocatorType>& v;
};
/// @brief output sink forwarding to a type-erased output adapter
///
/// Wraps the polymorphic output_adapter_t so the same binary_writer template can
/// also target arbitrary adapters (output streams, strings, user-provided
/// adapters) via the `output_adapter`-based overloads. Each write still goes
/// through one virtual call, exactly as before; only the concrete sinks above
/// avoid it.
template<typename CharType>
class output_adapter_sink
{
public:
explicit output_adapter_sink(output_adapter_t<CharType> adapter)
: oa(std::move(adapter))
{
JSON_ASSERT(oa);
}
void write_character(CharType c)
{
oa->write_character(c);
}
// no JSON_HEDLEY_NON_NULL: forwards (null, 0) for empty payloads, exactly as
// the type-erased path already did before this sink existed
void write_characters(const CharType* s, std::size_t length)
{
oa->write_characters(s, length);
}
private:
output_adapter_t<CharType> oa = nullptr;
};
template<typename CharType, typename StringType = std::basic_string<CharType>> template<typename CharType, typename StringType = std::basic_string<CharType>>
class output_adapter class output_adapter
{ {
+16 -6
View File
@@ -140,7 +140,7 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
friend ::nlohmann::detail::serializer<basic_json>; friend ::nlohmann::detail::serializer<basic_json>;
template<typename BasicJsonType> template<typename BasicJsonType>
friend class ::nlohmann::detail::iter_impl; friend class ::nlohmann::detail::iter_impl;
template<typename BasicJsonType, typename CharType> template<typename BasicJsonType, typename CharType, typename OutputSinkType>
friend class ::nlohmann::detail::binary_writer; friend class ::nlohmann::detail::binary_writer;
template<typename BasicJsonType, typename InputType, typename SAX> template<typename BasicJsonType, typename InputType, typename SAX>
friend class ::nlohmann::detail::binary_reader; friend class ::nlohmann::detail::binary_reader;
@@ -4327,7 +4327,9 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
static std::vector<std::uint8_t> to_cbor(const basic_json& j) static std::vector<std::uint8_t> to_cbor(const basic_json& j)
{ {
std::vector<std::uint8_t> result; std::vector<std::uint8_t> result;
to_cbor(j, result); result.reserve(detail::binary_reserve_hint(j));
detail::binary_writer<basic_json, std::uint8_t, detail::output_vector_sink<std::uint8_t>>(
detail::output_vector_sink<std::uint8_t>(result)).write_cbor(j);
return result; return result;
} }
@@ -4350,7 +4352,9 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
static std::vector<std::uint8_t> to_msgpack(const basic_json& j) static std::vector<std::uint8_t> to_msgpack(const basic_json& j)
{ {
std::vector<std::uint8_t> result; std::vector<std::uint8_t> result;
to_msgpack(j, result); result.reserve(detail::binary_reserve_hint(j));
detail::binary_writer<basic_json, std::uint8_t, detail::output_vector_sink<std::uint8_t>>(
detail::output_vector_sink<std::uint8_t>(result)).write_msgpack(j);
return result; return result;
} }
@@ -4375,7 +4379,9 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
const bool use_type = false) const bool use_type = false)
{ {
std::vector<std::uint8_t> result; std::vector<std::uint8_t> result;
to_ubjson(j, result, use_size, use_type); result.reserve(detail::binary_reserve_hint(j));
detail::binary_writer<basic_json, std::uint8_t, detail::output_vector_sink<std::uint8_t>>(
detail::output_vector_sink<std::uint8_t>(result)).write_ubjson(j, use_size, use_type);
return result; return result;
} }
@@ -4403,7 +4409,9 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
const bjdata_version_t version = bjdata_version_t::draft2) const bjdata_version_t version = bjdata_version_t::draft2)
{ {
std::vector<std::uint8_t> result; std::vector<std::uint8_t> result;
to_bjdata(j, result, use_size, use_type, version); result.reserve(detail::binary_reserve_hint(j));
detail::binary_writer<basic_json, std::uint8_t, detail::output_vector_sink<std::uint8_t>>(
detail::output_vector_sink<std::uint8_t>(result)).write_ubjson(j, use_size, use_type, true, true, version);
return result; return result;
} }
@@ -4430,7 +4438,9 @@ class basic_json // NOLINT(cppcoreguidelines-special-member-functions,hicpp-spec
static std::vector<std::uint8_t> to_bson(const basic_json& j) static std::vector<std::uint8_t> to_bson(const basic_json& j)
{ {
std::vector<std::uint8_t> result; std::vector<std::uint8_t> result;
to_bson(j, result); result.reserve(detail::binary_reserve_hint(j));
detail::binary_writer<basic_json, std::uint8_t, detail::output_vector_sink<std::uint8_t>>(
detail::output_vector_sink<std::uint8_t>(result)).write_bson(j);
return result; return result;
} }
File diff suppressed because it is too large Load Diff
-87
View File
@@ -11,35 +11,12 @@
#include <nlohmann/json.hpp> #include <nlohmann/json.hpp>
using nlohmann::json; using nlohmann::json;
#include <cstdint>
#include <fstream> #include <fstream>
#include <limits> #include <limits>
#include <sstream> #include <sstream>
#include <vector>
#include "make_test_data_available.hpp" #include "make_test_data_available.hpp"
#include "test_utils.hpp" #include "test_utils.hpp"
namespace
{
// a binary container that reports a size beyond INT32_MAX without allocating
// that much memory, so the BSON length overflow can be tested cheaply
class huge_binary_t : public std::vector<std::uint8_t>
{
public:
using std::vector<std::uint8_t>::vector;
size_type size() const noexcept // NOLINT(readability-convert-member-functions-to-static)
{
// one byte more than the BSON length field can represent
return static_cast<size_type>((std::numeric_limits<std::int32_t>::max)()) + 1;
}
};
using huge_binary_json = nlohmann::basic_json <
std::map, std::vector, std::string, bool, std::int64_t, std::uint64_t,
double, std::allocator, nlohmann::adl_serializer, huge_binary_t, void >;
} // namespace
TEST_CASE("BSON") TEST_CASE("BSON")
{ {
SECTION("individual values not supported") SECTION("individual values not supported")
@@ -103,14 +80,6 @@ TEST_CASE("BSON")
#endif #endif
} }
SECTION("lengths exceeding INT32_MAX cannot be serialized to BSON")
{
huge_binary_json j;
j["b"] = huge_binary_json::binary(huge_binary_t{});
CHECK_THROWS_WITH_AS(huge_binary_json::to_bson(j), "[json.exception.out_of_range.412] BSON length 2147483661 exceeds maximum of 2147483647", huge_binary_json::out_of_range&);
}
SECTION("string length must be at least 1") SECTION("string length must be at least 1")
{ {
// from https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=11175 // from https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=11175
@@ -885,62 +854,6 @@ TEST_CASE("Unsupported BSON input")
CHECK(!json::sax_parse(bson, &scp, json::input_format_t::bson)); CHECK(!json::sax_parse(bson, &scp, json::input_format_t::bson));
} }
TEST_CASE("BSON document size mismatch")
{
json _;
SECTION("top-level document declaring more bytes than it contains")
{
// empty object, but the length prefix claims 6 bytes instead of 5
std::vector<std::uint8_t> const input = {0x06, 0x00, 0x00, 0x00, 0x00};
CHECK_THROWS_WITH_AS(_ = json::from_bson(input), "[json.exception.parse_error.112] parse error at byte 5: syntax error while parsing BSON document: document size 6 does not match the number of bytes read (5)", json::parse_error&);
CHECK(json::from_bson(input, true, false).is_discarded());
}
SECTION("top-level document with a negative size")
{
std::vector<std::uint8_t> const input = {0xFF, 0xFF, 0xFF, 0xFF, 0x00};
CHECK_THROWS_WITH_AS(_ = json::from_bson(input), "[json.exception.parse_error.112] parse error at byte 5: syntax error while parsing BSON document: document size -1 does not match the number of bytes read (5)", json::parse_error&);
CHECK(json::from_bson(input, true, false).is_discarded());
}
SECTION("embedded document whose size disagrees with its terminator")
{
// the embedded document "d" declares 0x7FFFFFFF bytes but its 0x00
// terminator falls right after {"a":null}; the length prefix would
// otherwise let the following "h" element be read as a member of the
// enclosing document instead of "d"
std::vector<std::uint8_t> const input =
{
0x00, 0x00, 0x00, 0x00, // outer size
0x03, 'd', 0x00, // entry: embedded document "d"
0xFF, 0xFF, 0xFF, 0x7F, // embedded size 0x7FFFFFFF
0x0A, 'a', 0x00, // entry: null "a"
0x00, // embedded end marker
0x08, 'h', 0x00, 0x01, // entry: bool "h" = true
0x00 // outer end marker
};
CHECK_THROWS_WITH_AS(_ = json::from_bson(input), "[json.exception.parse_error.112] parse error at byte 15: syntax error while parsing BSON document: document size 2147483647 does not match the number of bytes read (8)", json::parse_error&);
CHECK(json::from_bson(input, true, false).is_discarded());
}
SECTION("embedded array whose size disagrees with its terminator")
{
// array [42] is 12 bytes, but the length prefix claims 13
std::vector<std::uint8_t> const input =
{
0x00, 0x00, 0x00, 0x00, // outer size
0x04, 'a', 0x00, // entry: array "a"
0x0D, 0x00, 0x00, 0x00, // array size 13 (real is 12)
0x10, '0', 0x00, 0x2A, 0x00, 0x00, 0x00, // entry: int32 "0" = 42
0x00, // array end marker
0x00 // outer end marker
};
CHECK_THROWS_WITH_AS(_ = json::from_bson(input), "[json.exception.parse_error.112] parse error at byte 19: syntax error while parsing BSON document: document size 13 does not match the number of bytes read (12)", json::parse_error&);
CHECK(json::from_bson(input, true, false).is_discarded());
}
}
TEST_CASE("BSON numerical data") TEST_CASE("BSON numerical data")
{ {
SECTION("number") SECTION("number")