• Products
  • Solutions
  • Made with Unity
  • Learning
  • Support & Services
  • Community
  • Asset Store
  • Get Unity

UNITY ACCOUNT

You need a Unity Account to shop in the Online and Asset Stores, participate in the Unity Community and manage your license portfolio. Login Create account
  • Blog
  • Forums
  • Answers
  • Evangelists
  • User Groups
  • Beta Program
  • Advisory Panel

Navigation

  • Home
  • Products
  • Solutions
  • Made with Unity
  • Learning
  • Support & Services
  • Community
    • Blog
    • Forums
    • Answers
    • Evangelists
    • User Groups
    • Beta Program
    • Advisory Panel

Unity account

You need a Unity Account to shop in the Online and Asset Stores, participate in the Unity Community and manage your license portfolio. Login Create account

Language

  • Chinese
  • Spanish
  • Japanese
  • Korean
  • Portuguese
  • Ask a question
  • Spaces
    • Default
    • Help Room
    • META
    • Moderators
    • Topics
    • Questions
    • Users
    • Badges
  • Home /
avatar image
0
Question by dman8723 · Jun 26, 2019 at 05:39 PM · opencvpanorama

how to optimize real-time stitching by OpenCV

Hello everyone! I am trying to make a real-time stitching function from camera in Unity. After watching some tutorials, I have success to make a simple real-time stitching from camera and this is my work. However, you may find there are some problems in the video. First, there is shaking every frame from the stitched video; second is the stitched video color is different from the left-hand side; and finally, there is a conspicuous line from the stitched video. Therefore, I want to know how to fix the problem and thanks everyone who seeing my question. Here is my code of the project, and I am using OpenCVforUnity for this project.

   void OnStitch1()
 {
     pr2 = Stitching(second_mat_gray, first_mat_gray);
 
     Utils.matToTexture2D(pr2, _Texture1);
     output.texture = _Texture1;
     output.GetComponent<AspectRatioFitter>().aspectRatio = (float)_Texture1.width / (float)_Texture1.height;
 }
 
 Mat Stitching(Mat first_mat_temp, Mat second_mat_temp)
 {
     //finding the keypoint between two streaming
     detector.detect(first_mat_temp, keypointsSrc);
     extractor.compute(first_mat_temp, keypointsSrc, descriptorsSrc);
 
     detector.detect(second_mat_temp, keypointsScene);
     extractor.compute(second_mat_temp, keypointsScene, descriptorsScene);

    //Matching the keypoints
     matcher.clear();
     matcher.match(descriptorsSrc, descriptorsScene, matches);
 
     matchesList = matches.toList();

     //After matching, trying to find the good match point by distance
     for (int i = 0; i < descriptorsSrc.rows(); i++)
     {
         dist = (double)matchesList[i].distance;
         if (dist < min_dist)
             min_dist = dist;
         if (dist > max_dist)
             max_dist = dist;
     }
 
     good_matches.Clear();
     for (int i = 0; i < matchesList.Count; i++)
     {
         if (matchesList[i].distance < 10 * min_dist)
             good_matches.Add(matchesList[i]);
     }
     GM = good_matches.Count;

     //Adding Keypoint to List for finding Homography with the OpenCV function
     keypoints_objectList = keypointsSrc.toList();
     keypoints_sceneList = keypointsScene.toList();
     keypointsSrc.release();
     keypointsScene.release();
 
     for (int i = 0; i < good_matches.Count; i++)
     {
         objList.Add(keypoints_objectList[good_matches[i].queryIdx].pt);
         sceneList.Add(keypoints_sceneList[good_matches[i].trainIdx].pt);
     }
 
     MatOfPoint2f obj = new MatOfPoint2f();
     MatOfPoint2f scene = new MatOfPoint2f();
 
     obj.fromList(objList);
     scene.fromList(sceneList);
     H = Calib3d.findHomography(obj, scene, Calib3d.RANSAC, 10);

    //Start warp the streaming from second camera to connect the streaming from first camera
    //Trying to stop the shaking, but not smooth enough
     warp_mat = second_mat.clone();
 
     if (GM - gmCount > 10)
     {
         gmCount = good_matches.Count;
         tempH = H.clone();
         Imgproc.warpPerspective(second_mat, warp_mat, H, ims);
     }
     else
     {
         Imgproc.warpPerspective(second_mat, warp_mat, tempH, ims);
     }
     if (gmCount > 400 || GM < 100)
         gmCount = GM;
 
     //Last Step for stitching as a whole Mat
     half = new Mat(warp_mat, _rect);
     first_mat.copyTo(half);
     return warp_mat;
 }

Comment
Add comment · Show 2
10 |3000 characters needed characters left characters exceeded
▼
  • Viewable by all users
  • Viewable by moderators
  • Viewable by moderators and the original poster
  • Advanced visibility
Viewable by all users
avatar image dman8723 · Jun 27, 2019 at 09:17 AM 0
Share

I find $$anonymous$$ Blending may be the solution to clear the conspicuous line from the stitched video, but I have no idea how to do that in Unity, does anyone can giving me some suggestion?

avatar image LionWare · Apr 10, 2021 at 04:50 AM 0
Share

Hey did you find anything PS. Which version of Unity were you using, on 2020.2, I am getting constant crashes even on this code.

0 Replies

· Add your reply
  • Sort: 

Your answer

Hint: You can notify a user about this post by typing @username

Up to 2 attachments (including images) can be used with a maximum of 524.3 kB each and 1.0 MB total.

Welcome to Unity Answers

If you’re new to Unity Answers, please check our User Guide to help you navigate through our website and refer to our FAQ for more information.

Before posting, make sure to check out our Knowledge Base for commonly asked Unity questions.

Check our Moderator Guidelines if you’re a new moderator and want to work together in an effort to improve Unity Answers and support our users.

Follow this Question

Answers Answers and Comments

108 People are following this question.

avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image

Related Questions

Need help stitching images from folder using OpenCV 0 Answers

OpenCV integration 2 Answers

Transform position of child object and look at it 0 Answers

Opencv2.framework not loaded? 1 Answer

Anyone know how to load/view Cardboard Camera 3D Jpeg's? 2 Answers


Enterprise
Social Q&A

Social
Subscribe on YouTube social-youtube Follow on LinkedIn social-linkedin Follow on Twitter social-twitter Follow on Facebook social-facebook Follow on Instagram social-instagram

Footer

  • Purchase
    • Products
    • Subscription
    • Asset Store
    • Unity Gear
    • Resellers
  • Education
    • Students
    • Educators
    • Certification
    • Learn
    • Center of Excellence
  • Download
    • Unity
    • Beta Program
  • Unity Labs
    • Labs
    • Publications
  • Resources
    • Learn platform
    • Community
    • Documentation
    • Unity QA
    • FAQ
    • Services Status
    • Connect
  • About Unity
    • About Us
    • Blog
    • Events
    • Careers
    • Contact
    • Press
    • Partners
    • Affiliates
    • Security
Copyright © 2020 Unity Technologies
  • Legal
  • Privacy Policy
  • Cookies
  • Do Not Sell My Personal Information
  • Cookies Settings
"Unity", Unity logos, and other Unity trademarks are trademarks or registered trademarks of Unity Technologies or its affiliates in the U.S. and elsewhere (more info here). Other names or brands are trademarks of their respective owners.
  • Anonymous
  • Sign in
  • Create
  • Ask a question
  • Spaces
  • Default
  • Help Room
  • META
  • Moderators
  • Explore
  • Topics
  • Questions
  • Users
  • Badges