• Products
  • Solutions
  • Made with Unity
  • Learning
  • Support & Services
  • Community
  • Asset Store
  • Get Unity

UNITY ACCOUNT

You need a Unity Account to shop in the Online and Asset Stores, participate in the Unity Community and manage your license portfolio. Login Create account
  • Blog
  • Forums
  • Answers
  • Evangelists
  • User Groups
  • Beta Program
  • Advisory Panel

Navigation

  • Home
  • Products
  • Solutions
  • Made with Unity
  • Learning
  • Support & Services
  • Community
    • Blog
    • Forums
    • Answers
    • Evangelists
    • User Groups
    • Beta Program
    • Advisory Panel

Unity account

You need a Unity Account to shop in the Online and Asset Stores, participate in the Unity Community and manage your license portfolio. Login Create account

Language

  • Chinese
  • Spanish
  • Japanese
  • Korean
  • Portuguese
  • Ask a question
  • Spaces
    • Default
    • Help Room
    • META
    • Moderators
    • Topics
    • Questions
    • Users
    • Badges
  • Home /
avatar image
1
Question by enekochan · Apr 04, 2012 at 05:53 PM · iostextureaugmented realityopencv

Read texture data for iOS PlugIn

Hi,

I'm trying to do a small Augmented Reality App with Unity for iOS. The goal is to capture data from the iOS device camera, put a Unity camera in orthographic projection and a plane in front of it with a texture of the real cameras data (with WebCamTexture for example), process that data with OpenCV on native iOS code and then again in Unity update the position of some 3D object (wich would be between the Unity camera and the plane) with the data processed by OpenCV.

Right now I don't have a real iOS device so I'm working with the Simulator. That's why, for the moment, I'm loading a JPG in a texture and trying to process its data for testing. When I have the real device I could use WebCamTexture, but I suppose the overall process would be similar.

The problem I have right now is that I don't know how to read the data of the texture so I can process it on OpenCV. I can get the NativeTextureID but in OpenGL ES there is no glGetTexImage call that would give me just what I want. I've also tried sending the GetPixels32 data to my iOS function but it looks like only the first RGBA pixel is correct, the next one is all 0's. Also I'm not sure if this would be extremely slow to process, may be it would load too many data on every frame.

In Unity:

 Texture2D tex = renderer.material.mainTexture as Texture2D;
 data = tex.GetPixels32();
 _LoadColor32Array( ref data ); // My iOS native function

In iOS:

 struct Color32 {
  Byte r;
  Byte g;
  Byte b;
  Byte a;
 };

 void _LoadColor32Array( Color32 *pixels[] ) {
     NSLog(@"_LoadColor32Array");
     printf("RGBA(%i,%i,%i,%i)\n", pixels[0]->r, pixels[0]->g, pixels[0]->b, pixels[0]->a);
     printf("RGBA(%i,%i,%i,%i)\n", pixels[1]->r, pixels[1]->g, pixels[1]->b, pixels[1]->a);
     NSLog(@"-> complete.");
 }

Is there a way to read the texture data in iOS native code? May be with some OpenGL code using the texture ID? I've read something about FrameBuffer Objects but don't know anything about that. Any other approach?

Thanks.

P.D.:I'm new to Unity and OpenGL so may be I'm right now in the wrong direction or there is a simpler way to do what I want.

Comment
Add comment · Show 2
10 |3000 characters needed characters left characters exceeded
▼
  • Viewable by all users
  • Viewable by moderators
  • Viewable by moderators and the original poster
  • Advanced visibility
Viewable by all users
avatar image Rambleramble · Sep 25, 2013 at 08:32 PM 0
Share

Sorry to dig out this old thread, but any update on the project? I am also facing a similar situation. Your experience on the issue will be much appreciated!

avatar image robertbu · Sep 25, 2013 at 08:35 PM 0
Share

Prime31's LiveTexture plugin will pipe the camera into a Unity app.

2 Replies

· Add your reply
  • Sort: 
avatar image
1

Answer by enekochan · Apr 05, 2012 at 12:38 PM

I made it work by not passing the Color32[] by reference (as it is an array I suppose it is always passed by reference as in C... or may be not, don't really know). So just deleting "ref" in the C# code and the "*" in the Objetive-C code did the trick.

I'm still not sure if this approach will be fast enough for an Augmente Reality App. Could somebody explain me how would it be to copy the texture data to a FBO and then read it with glReadPixels? Although this could also not be the fastest solution I don't know any other one.

Thank you.

Comment
Add comment · Show 4 · Share
10 |3000 characters needed characters left characters exceeded
▼
  • Viewable by all users
  • Viewable by moderators
  • Viewable by moderators and the original poster
  • Advanced visibility
Viewable by all users
avatar image SirGive · Apr 05, 2012 at 12:45 PM 0
Share

check out Qualcomm's Augmented reality plugin for unity, qcar. I don't know how much of their code is exposed, but their AR is pretty solid on the ipad. Hope that helps.

avatar image enekochan · Apr 10, 2012 at 09:26 AM 0
Share

I already knew Vuforia but thanks anyway. The thing is that I want to create my own AR plugin. $$anonymous$$ay be I'll use use their AR SD$$anonymous$$ for another project in the future.

avatar image reistiago · Jun 19, 2012 at 09:38 AM 0
Share

Did you ever found a better way to do this? I'm struggling with the same.

avatar image Rambleramble · Sep 25, 2013 at 08:30 PM 0
Share

Any new update for sharing? I am also facing a similar situation. Your experience would be very much appreciated!

avatar image
0

Answer by Fattie · Jan 18, 2019 at 01:18 PM

Seven years later ..

nowadays unity has TOTALLY CHANGED all of this:

https://github.com/keijiro/TextureUpdateExample

Enjoy

Comment
Add comment · Share
10 |3000 characters needed characters left characters exceeded
▼
  • Viewable by all users
  • Viewable by moderators
  • Viewable by moderators and the original poster
  • Advanced visibility
Viewable by all users

Your answer

Hint: You can notify a user about this post by typing @username

Up to 2 attachments (including images) can be used with a maximum of 524.3 kB each and 1.0 MB total.

Welcome to Unity Answers

If you’re new to Unity Answers, please check our User Guide to help you navigate through our website and refer to our FAQ for more information.

Before posting, make sure to check out our Knowledge Base for commonly asked Unity questions.

Check our Moderator Guidelines if you’re a new moderator and want to work together in an effort to improve Unity Answers and support our users.

Follow this Question

Answers Answers and Comments

4 People are following this question.

avatar image avatar image avatar image avatar image

Related Questions

Face Tracking using openCV is not working accurately 0 Answers

Download images from url, save to device, load into UGUI 4 Answers

Image size correlation between disk and memory 1 Answer

Identical material with different textures 0 Answers

Texture on instantiated asset bundle sometimes black only on iPhone 0 Answers


Enterprise
Social Q&A

Social
Subscribe on YouTube social-youtube Follow on LinkedIn social-linkedin Follow on Twitter social-twitter Follow on Facebook social-facebook Follow on Instagram social-instagram

Footer

  • Purchase
    • Products
    • Subscription
    • Asset Store
    • Unity Gear
    • Resellers
  • Education
    • Students
    • Educators
    • Certification
    • Learn
    • Center of Excellence
  • Download
    • Unity
    • Beta Program
  • Unity Labs
    • Labs
    • Publications
  • Resources
    • Learn platform
    • Community
    • Documentation
    • Unity QA
    • FAQ
    • Services Status
    • Connect
  • About Unity
    • About Us
    • Blog
    • Events
    • Careers
    • Contact
    • Press
    • Partners
    • Affiliates
    • Security
Copyright © 2020 Unity Technologies
  • Legal
  • Privacy Policy
  • Cookies
  • Do Not Sell My Personal Information
  • Cookies Settings
"Unity", Unity logos, and other Unity trademarks are trademarks or registered trademarks of Unity Technologies or its affiliates in the U.S. and elsewhere (more info here). Other names or brands are trademarks of their respective owners.
  • Anonymous
  • Sign in
  • Create
  • Ask a question
  • Spaces
  • Default
  • Help Room
  • META
  • Moderators
  • Explore
  • Topics
  • Questions
  • Users
  • Badges