1
0
mirror of https://github.com/opencv/opencv.git synced 2026-07-30 15:53:03 +04:00

Merge pull request #26391 from Abdurrahheem:ash/lstm-new-graph-engine-latest

LSTM layer for new graph engine. #26391

Merge with extra: https://github.com/opencv/opencv_extra/pull/1218

This PR updates/creates LSTM layer compatible with new graph engine. It is based on previous LSTM implementation with some modification on how initializers blobs are processed.

Note: Following tests are currently are disabled 


Two following two tests are disbled since ONNNRuntime does not support `layout=1` attiribute inference. See a detailed issue #26456 on this.
- `LSTM_layout_seq` 
- `LSTM_layout_batch`

Following test fails with the new engine as it is not able to deal with shapes of the form [?, C, H, W]
- `LSTM_Activations`

Works:
- [x] One directional case any batch type 
 - [x] Fix directional case when batch size large than 1
 - [x] Add peepholes attribute

TODO with the next PRs:
 - [ ] Activation support

Note: 
  

> Currently `LSTM_layout_seq`, `LSTM_layout_batch` are disabled as the tests are incorrect. They do not comply with the ONNX standard. Particularly test outputs are of incorrect dimensionality. They produce 3-dimentinal output instead of 4-dimentional. 

------

### Pull Request Readiness Checklist

See details at https://github.com/opencv/opencv/wiki/How_to_contribute#making-a-good-pull-request

- [x] I agree to contribute to the project under Apache 2 License.
- [x] To the best of my knowledge, the proposed patch is not based on a code under GPL or another license that is incompatible with OpenCV
- [x] The PR is proposed to the proper branch
- [x] There is a reference to the original bug report and related work
- [x] There is accuracy test, performance test and test data in opencv_extra repository, if applicable
      Patch to opencv_extra has the same branch name.
- [x] The feature is well documented and sample code can be built with the project CMake
This commit is contained in:
Abduragim Shtanchaev
2024-11-20 15:12:58 +04:00
committed by GitHub
parent 2ed6d0f590
commit d0820dac38
6 changed files with 698 additions and 4 deletions
+50
View File
@@ -2740,4 +2740,54 @@ INSTANTIATE_TEST_CASE_P(TestLayerFusion, ConvolutionActivationEltwiseFusion, Com
TestLayerFusion::dnnBackendsAndTargetsForFusionTests()
));
TEST(Layer_LSTM, repeatedInference)
{
std::string onnx_file_path = findDataFile("dnn/onnx/models/onnxscript_lstm.onnx", false);
// Test parameters
const int batch_size = 1;
const int seq_length = 5;
const int input_size = 6;
const int hidden_size = 4;
const int num_directions = 1;
// Create random input tensors
int x_shape [] = {seq_length, batch_size, input_size};
int h_0_shape [] = {num_directions, batch_size, hidden_size};
int c_0_shape [] = {num_directions, batch_size, hidden_size};
Mat X(3, x_shape, CV_32F);
Mat h_0(3, h_0_shape, CV_32F);
Mat c_0(3, c_0_shape, CV_32F);
randu(X, 0, 1);
randu(h_0, 0, 1);
randu(c_0, 0, 1);
// Load and run ONNX model
dnn::Net net = dnn::readNet(onnx_file_path);
std::vector<std::string> outputNames = {"Y", "Y_h", "Y_c"};
std::vector<std::vector<Mat>> all_outputs;
// Run the model twice
for (int i = 0; i < 2; i++) {
net.setInput(X, "X");
net.setInput(h_0, "initial_h");
net.setInput(c_0, "initial_c");
std::vector<Mat> outputs;
net.forward(outputs, outputNames);
// std::cout << "........pass " << (i + 1) << " done........" << std::endl;
all_outputs.push_back(outputs);
}
// convert to assertions
double diff0 = cv::norm(all_outputs[0][0], all_outputs[1][0], NORM_L1);
double diff1 = cv::norm(all_outputs[0][1], all_outputs[1][1], NORM_L1);
double diff2 = cv::norm(all_outputs[0][2], all_outputs[1][2], NORM_L1);
EXPECT_EQ(diff0, 0.);
EXPECT_EQ(diff1, 0.);
EXPECT_EQ(diff2, 0.);
}
}} // namespace