• Products
  • Solutions
  • Made with Unity
  • Learning
  • Support & Services
  • Community
  • Asset Store
  • Get Unity

UNITY ACCOUNT

You need a Unity Account to shop in the Online and Asset Stores, participate in the Unity Community and manage your license portfolio. Login Create account
  • Blog
  • Forums
  • Answers
  • Evangelists
  • User Groups
  • Beta Program
  • Advisory Panel

Navigation

  • Home
  • Products
  • Solutions
  • Made with Unity
  • Learning
  • Support & Services
  • Community
    • Blog
    • Forums
    • Answers
    • Evangelists
    • User Groups
    • Beta Program
    • Advisory Panel

Unity account

You need a Unity Account to shop in the Online and Asset Stores, participate in the Unity Community and manage your license portfolio. Login Create account

Language

  • Chinese
  • Spanish
  • Japanese
  • Korean
  • Portuguese
  • Ask a question
  • Spaces
    • Default
    • Help Room
    • META
    • Moderators
    • Topics
    • Questions
    • Users
    • Badges
  • Home /
avatar image
1
Question by xandersteenbrugge · Jun 07, 2018 at 09:15 AM · meshaudioprocedural meshpythonprocedural animation

Generating a parameterized 3D mesh from Python

Hi All, I'm a hard-core Machine Learning programmer (Python, TensorFlow, PyTorch, Numpy, ...) and I want to dive into my first VR project so basically I need some pro tips to get started!

Here's what I want to do:


⦁ Create a parameterized 3D mesh (initially this could be built from simple 3D objects like spheres or pyramids with adjustable sizes and rotations, but in the end I would like to create a fully custom, connected grid of triangular meshes based on (x,y,z) coordinates that can be dynamically adjusted. (The goal is to generate these coordinates on the fly, using Machine Learning libraries in Python, send them over a local socket and use them in Unity to dynamically warp the 3D mesh)

⦁ Next, I want to map a single image (like this one: https://pbs.twimg.com/media/DdJ7iEOX4AAOiUp.jpg) (or a set of images) onto this 3D mesh. The image can be stretched, rotated, even cut into pieces and duplicated/tiled, but I want to place it all over the 3D mesh so that the 3D structure from the mesh is overlayed with the texture of the image.

⦁ Then, I want to render the entire scene from any two camera points in the 3D space (most likely just selecting standard two-eye viewpoints)

⦁ Finally, the two rendered spherical views need to be formatted as a split-screen square .mp4 video format so it can be played on a standard VR headset and the viewer can look around.


If you're wondering what the intended goal is, well it's kind of hard to explain but I want to try something creative, slightly inspired by this video: https://youtu.be/II3bE9nA5Z4

Basically, I am looking for the right VR libraries (Github?) to do all this. In the end I want to dynamically control the parameterized 3D mesh from an ML library like TensorFlow (Python) based on music and user controls. I think the Unity-ML agents library is probably where I need to start, but I don't really know the best way to approach this..

Anybody has some suggestions for me?

Comment
Add comment · Show 1
10 |3000 characters needed characters left characters exceeded
▼
  • Viewable by all users
  • Viewable by moderators
  • Viewable by moderators and the original poster
  • Advanced visibility
Viewable by all users
avatar image thelaplage · Jul 13, 2019 at 07:25 AM 0
Share

@xandersteenbrugge Hi, any progress with this? If so, I'd be curious to hear more about your approach, thanks!

0 Replies

· Add your reply
  • Sort: 

Your answer

Hint: You can notify a user about this post by typing @username

Up to 2 attachments (including images) can be used with a maximum of 524.3 kB each and 1.0 MB total.

Welcome to Unity Answers

The best place to ask and answer questions about development with Unity.

To help users navigate the site we have posted a site navigation guide.

If you are a new user to Unity Answers, check out our FAQ for more information.

Make sure to check out our Knowledge Base for commonly asked Unity questions.

If you are a moderator, see our Moderator Guidelines page.

We are making improvements to UA, see the list of changes.



Follow this Question

Answers Answers and Comments

118 People are following this question.

avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image

Related Questions

Unity Ui – Generating PixelPerfect Meshes 0 Answers

How to cut a hole in a wall? 3 Answers

Creating a dynamic mesh with audio = crash after 30secs ? 0 Answers

Why does Frame Selected toggle between position and bounds for procedural and imported meshes? 1 Answer

detect if two objects in a list intersect in editor. 0 Answers


Enterprise
Social Q&A

Social
Subscribe on YouTube social-youtube Follow on LinkedIn social-linkedin Follow on Twitter social-twitter Follow on Facebook social-facebook Follow on Instagram social-instagram

Footer

  • Purchase
    • Products
    • Subscription
    • Asset Store
    • Unity Gear
    • Resellers
  • Education
    • Students
    • Educators
    • Certification
    • Learn
    • Center of Excellence
  • Download
    • Unity
    • Beta Program
  • Unity Labs
    • Labs
    • Publications
  • Resources
    • Learn platform
    • Community
    • Documentation
    • Unity QA
    • FAQ
    • Services Status
    • Connect
  • About Unity
    • About Us
    • Blog
    • Events
    • Careers
    • Contact
    • Press
    • Partners
    • Affiliates
    • Security
Copyright © 2020 Unity Technologies
  • Legal
  • Privacy Policy
  • Cookies
  • Do Not Sell My Personal Information
  • Cookies Settings
"Unity", Unity logos, and other Unity trademarks are trademarks or registered trademarks of Unity Technologies or its affiliates in the U.S. and elsewhere (more info here). Other names or brands are trademarks of their respective owners.
  • Anonymous
  • Sign in
  • Create
  • Ask a question
  • Spaces
  • Default
  • Help Room
  • META
  • Moderators
  • Explore
  • Topics
  • Questions
  • Users
  • Badges