Merge pull request #28461 from Ron12777:opt-clean

Optimize calibrateCamera with Schur‑complement LM and parallel Jacobian accumulation #28461

## Summary

- Optimized `calibrateCamera` for faster runtime without changing outputs using Schur‑complement LM, Parallel Jacobian accumulation, alongside other optimizations.
- Reduced time complexity from O(n^3) to O(n)
- Add a perf test that uses a 500-image chessboard dataset for performance testing.

## Performance
<img width="1200" height="800" alt="base_vs_fast_results" src="https://github.com/user-attachments/assets/6dafa19f-f9cb-4f7f-ba40-0940373712e8" />
<img width="1200" height="800" alt="fast_vs_ceres_results" src="https://github.com/user-attachments/assets/7157af27-8a2b-4810-8b53-3cc9972a8493" />
<img width="1200" height="800" alt="base_vs_fast_param_deviation" src="https://github.com/user-attachments/assets/fe4f954c-34f9-4b9a-b1b2-46e4c76ce08c" />


[Testing repo
](https://github.com/Ron12777/OpenCV-benchmarking)
## Testing
- All local tests pass 

## Related

- [opencv_extra PR with test images](https://github.com/opencv/opencv_extra/pull/1312)


See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [ ] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake

doc/opencv.bib

@@ -860,6 +860,22 @@
   number = {8},
   publisher = {IOP Publishing Ltd}
 }
+@article{Lourakis2009_sba,
+  author = {Lourakis, Manolis I. A. and Argyros, Antonis A.},
+  title = {SBA: A Software Package for Generic Sparse Bundle Adjustment},
+  year = {2009},
+  month = mar,
+  journal = {ACM Transactions on Mathematical Software},
+  volume = {36},
+  number = {1},
+  articleno = {2},
+  pages = {2:1--2:30},
+  numpages = {30},
+  publisher = {Association for Computing Machinery},
+  doi = {10.1145/1486525.1486527},
+  url = {https://scispace.com/pdf/sba-a-software-package-for-generic-sparse-bundle-adjustment-1d4hp0z31z.pdf},
+  month_numeric = {3}
+}
 @article{LowIlie2003,
   author   = {Kok-Lim Low, Adrian Ilie},
   year = {2003},
@@ -1353,6 +1369,20 @@
   publisher = {Taylor \& Francis},
   url = {https://www.olivier-augereau.com/docs/2004JGraphToolsTelea.pdf}
 }
+@incollection{Triggs2000_bundle_adjustment,
+  author = {Triggs, Bill and McLauchlan, Philip F. and Hartley, Richard I. and Fitzgibbon, Andrew W.},
+  title = {Bundle Adjustment---A Modern Synthesis},
+  booktitle = {Vision Algorithms: Theory and Practice},
+  year = {2000},
+  pages = {298--372},
+  publisher = {Springer},
+  series = {Lecture Notes in Computer Science},
+  volume = {1883},
+  editor = {Triggs, Bill and Zisserman, Andrew and Szeliski, Richard},
+  doi = {10.1007/3-540-44480-7_21},
+  isbn = {978-3-540-67973-8},
+  url = {https://www.cs.jhu.edu/~misha/ReadingSeminar/Papers/Triggs00.pdf}
+}
 @article{Tsai89,
   author = {R. Y. Tsai and R. K. Lenz},
   journal = {IEEE Transactions on Robotics and Automation},

modules/calib3d/include/opencv2/calib3d.hpp

@@ -627,7 +627,8 @@ enum { CALIB_NINTRINSIC          = 18,
        // for stereo rectification
        CALIB_ZERO_DISPARITY      = 0x00400,
        CALIB_USE_LU              = (1 << 17), //!< use LU instead of SVD decomposition for solving. much faster but potentially less precise
-       CALIB_USE_EXTRINSIC_GUESS = (1 << 22)  //!< for stereoCalibrate
+       CALIB_USE_EXTRINSIC_GUESS = (1 << 22), //!< for stereoCalibrate
+       CALIB_DISABLE_SCHUR_COMPLEMENT = (1 << 23)  //!< disable Schur complement (use Bouguet calibration engine)
      };
 
 //! the algorithm for finding fundamental matrix
@@ -1659,6 +1660,7 @@ fx, fy, cx, cy that are optimized further. Otherwise, (cx, cy) is initially set
 center ( imageSize is used), and focal distances are computed in a least-squares fashion.
 Note, that if intrinsic parameters are known, there is no need to use this function just to
 estimate extrinsic parameters. Use @ref solvePnP instead.
+-   @ref CALIB_DISABLE_SCHUR_COMPLEMENT Disable Schur complement and use the Bouguet calibration engine (@cite Zhang2000, @cite BouguetMCT).
 -   @ref CALIB_FIX_PRINCIPAL_POINT The principal point is not changed during the global
 optimization. It stays at the center or at a different location specified when
  @ref CALIB_USE_INTRINSIC_GUESS is set too.
@@ -1693,7 +1695,9 @@ supplied distCoeffs matrix is used. Otherwise, it is set to 0.
 @return the overall RMS re-projection error.
 
 The function estimates the intrinsic camera parameters and extrinsic parameters for each of the
-views. The algorithm is based on @cite Zhang2000 and @cite BouguetMCT . The coordinates of 3D object
+views. By default, the optimization follows a sparse bundle adjustment formulation with Schur
+complement; see @cite Triggs2000_bundle_adjustment and @cite Lourakis2009_sba for background. Use
+@ref CALIB_DISABLE_SCHUR_COMPLEMENT to switch to the Bouguet calibration engine. The coordinates of 3D object
 points and their corresponding 2D projections in each view must be specified. That may be achieved
 by using an object with known geometry and easily detectable feature points. Such an object is
 called a calibration rig or calibration pattern, and OpenCV has built-in support for a chessboard as
@@ -1716,6 +1720,10 @@ The algorithm performs the following steps:
     the projected (using the current estimates for camera parameters and the poses) object points
     objectPoints. See @ref projectPoints for details.
 
+-   In practice, robust acquisition is essential for stable results: use multiple board poses with
+    significant tilt, avoid collecting all views at a single working distance, span the expected
+    working-distance range (a larger board with larger squares can help for longer distances).
+
 @note
     If you use a non-square (i.e. non-N-by-N) grid and @ref findChessboardCorners for calibration,
     and @ref calibrateCamera returns bad values (zero distortion coefficients, \f$c_x\f$ and
@@ -1801,8 +1809,8 @@ less precise and less stable in some rare cases.
 @return the overall RMS re-projection error.
 
 The function estimates the intrinsic camera parameters and extrinsic parameters for each of the
-views. The algorithm is based on @cite Zhang2000, @cite BouguetMCT and @cite strobl2011iccv. See
-#calibrateCamera for other detailed explanations.
+views. The object-releasing extension follows @cite strobl2011iccv and uses the same optimization
+core as #calibrateCamera. See #calibrateCamera for other detailed explanations.
 @sa
    calibrateCamera, findChessboardCorners, solvePnP, initCameraMatrix2D, stereoCalibrate, undistort
  */

modules/calib3d/perf/perf_calibratecamera.cpp

@@ -0,0 +1,100 @@
+// This file is part of OpenCV project.
+// It is subject to the license terms in the LICENSE file found in the top-level directory
+// of this distribution and at http://opencv.org/license.html
+#include "perf_precomp.hpp"
+
+#include "opencv2/core/utils/filesystem.hpp"
+
+namespace opencv_test {
+
+static std::vector<cv::String> loadBulkImages(size_t max_images)
+{
+    const std::string data_dir = findDataDirectory("perf/calib3d/bulk_n500", false);
+    std::vector<cv::String> image_paths;
+    cv::utils::fs::glob(data_dir, "*.png", image_paths, false, false);
+    if (image_paths.empty())
+        cv::utils::fs::glob(data_dir, "*.jpg", image_paths, false, false);
+    if (image_paths.empty())
+        throw SkipTestException("No images found in perf/calib3d/bulk_n500");
+
+    std::sort(image_paths.begin(), image_paths.end());
+    if (image_paths.size() > max_images)
+        image_paths.resize(max_images);
+    return image_paths;
+}
+
+static std::vector<Point3f> buildObjectPoints(const Size& pattern_size, float square_size)
+{
+    std::vector<Point3f> object_points;
+    object_points.reserve(pattern_size.area());
+    for (int y = 0; y < pattern_size.height; ++y)
+        for (int x = 0; x < pattern_size.width; ++x)
+            object_points.push_back(Point3f(x * square_size, y * square_size, 0.f));
+    return object_points;
+}
+
+PERF_TEST(CalibrateCamera, DISABLED_BulkImages_N500)
+{
+    applyTestTag(CV_TEST_TAG_LONG, CV_TEST_TAG_SIZE_HD);
+
+    const Size pattern_size(6, 8);
+    const size_t max_images = 500;
+
+    std::vector<cv::String> image_paths = loadBulkImages(max_images);
+    std::vector<Point3f> object_pattern = buildObjectPoints(pattern_size, 1.0f);
+
+    std::vector<std::vector<Point2f> > image_points;
+    std::vector<std::vector<Point3f> > object_points;
+    image_points.reserve(image_paths.size());
+    object_points.reserve(image_paths.size());
+
+    Size image_size;
+    for (const auto& path : image_paths)
+    {
+        Mat gray = imread(path, IMREAD_GRAYSCALE);
+        ASSERT_FALSE(gray.empty()) << "Can't read image: " << path;
+        if (image_size.empty())
+            image_size = gray.size();
+        else
+            ASSERT_EQ(gray.size(), image_size) << "Mismatched image size: " << path;
+
+        std::vector<Point2f> corners;
+        bool found = findChessboardCorners(
+            gray, pattern_size, corners,
+            CALIB_CB_ADAPTIVE_THRESH | CALIB_CB_NORMALIZE_IMAGE);
+        ASSERT_TRUE(found) << "Chessboard not found: " << path;
+
+        cornerSubPix(gray, corners, Size(11, 11), Size(-1, -1),
+                     TermCriteria(TermCriteria::EPS + TermCriteria::COUNT, 30, 0.1));
+
+        image_points.push_back(corners);
+        object_points.push_back(object_pattern);
+    }
+
+    ASSERT_FALSE(image_points.empty());
+
+    Mat camera_matrix = Mat::eye(3, 3, CV_64F);
+    Mat dist_coeffs = Mat::zeros(8, 1, CV_64F);
+    std::vector<Mat> rvecs;
+    std::vector<Mat> tvecs;
+    double rms = 0.0;
+
+    declare.in(image_points, object_points);
+    declare.out(camera_matrix, dist_coeffs);
+    declare.iterations(1);
+
+    TEST_CYCLE()
+    {
+        camera_matrix = Mat::eye(3, 3, CV_64F);
+        dist_coeffs = Mat::zeros(8, 1, CV_64F);
+        rvecs.clear();
+        tvecs.clear();
+        rms = calibrateCamera(object_points, image_points, image_size,
+                              camera_matrix, dist_coeffs, rvecs, tvecs, 0);
+    }
+
+    SANITY_CHECK_NOTHING();
+    EXPECT_GT(rms, 0.0);
+}
+
+} // namespace opencv_test