• Products
  • Solutions
  • Made with Unity
  • Learning
  • Support & Services
  • Community
  • Asset Store
  • Get Unity

UNITY ACCOUNT

You need a Unity Account to shop in the Online and Asset Stores, participate in the Unity Community and manage your license portfolio. Login Create account
  • Blog
  • Forums
  • Answers
  • Evangelists
  • User Groups
  • Beta Program
  • Advisory Panel

Navigation

  • Home
  • Products
  • Solutions
  • Made with Unity
  • Learning
  • Support & Services
  • Community
    • Blog
    • Forums
    • Answers
    • Evangelists
    • User Groups
    • Beta Program
    • Advisory Panel

Unity account

You need a Unity Account to shop in the Online and Asset Stores, participate in the Unity Community and manage your license portfolio. Login Create account

Language

  • Chinese
  • Spanish
  • Japanese
  • Korean
  • Portuguese
  • Ask a question
  • Spaces
    • Default
    • Help Room
    • META
    • Moderators
    • Topics
    • Questions
    • Users
    • Badges
  • Home /
avatar image
1
Question by xandersteenbrugge · Jun 07, 2018 at 09:15 AM · meshaudioprocedural meshpythonprocedural animation

Generating a parameterized 3D mesh from Python

Hi All, I'm a hard-core Machine Learning programmer (Python, TensorFlow, PyTorch, Numpy, ...) and I want to dive into my first VR project so basically I need some pro tips to get started!

Here's what I want to do:


⦁ Create a parameterized 3D mesh (initially this could be built from simple 3D objects like spheres or pyramids with adjustable sizes and rotations, but in the end I would like to create a fully custom, connected grid of triangular meshes based on (x,y,z) coordinates that can be dynamically adjusted. (The goal is to generate these coordinates on the fly, using Machine Learning libraries in Python, send them over a local socket and use them in Unity to dynamically warp the 3D mesh)

⦁ Next, I want to map a single image (like this one: https://pbs.twimg.com/media/DdJ7iEOX4AAOiUp.jpg) (or a set of images) onto this 3D mesh. The image can be stretched, rotated, even cut into pieces and duplicated/tiled, but I want to place it all over the 3D mesh so that the 3D structure from the mesh is overlayed with the texture of the image.

⦁ Then, I want to render the entire scene from any two camera points in the 3D space (most likely just selecting standard two-eye viewpoints)

⦁ Finally, the two rendered spherical views need to be formatted as a split-screen square .mp4 video format so it can be played on a standard VR headset and the viewer can look around.


If you're wondering what the intended goal is, well it's kind of hard to explain but I want to try something creative, slightly inspired by this video: https://youtu.be/II3bE9nA5Z4

Basically, I am looking for the right VR libraries (Github?) to do all this. In the end I want to dynamically control the parameterized 3D mesh from an ML library like TensorFlow (Python) based on music and user controls. I think the Unity-ML agents library is probably where I need to start, but I don't really know the best way to approach this..

Anybody has some suggestions for me?

Comment
Add comment · Show 1
10 |3000 characters needed characters left characters exceeded
▼
  • Viewable by all users
  • Viewable by moderators
  • Viewable by moderators and the original poster
  • Advanced visibility
Viewable by all users
avatar image thelaplage · Jul 13, 2019 at 07:25 AM 0
Share

@xandersteenbrugge Hi, any progress with this? If so, I'd be curious to hear more about your approach, thanks!

0 Replies

· Add your reply
  • Sort: 

Your answer

Hint: You can notify a user about this post by typing @username

Up to 2 attachments (including images) can be used with a maximum of 524.3 kB each and 1.0 MB total.

Welcome to Unity Answers

The best place to ask and answer questions about development with Unity.

To help users navigate the site we have posted a site navigation guide.

If you are a new user to Unity Answers, check out our FAQ for more information.

Make sure to check out our Knowledge Base for commonly asked Unity questions.

If you are a moderator, see our Moderator Guidelines page.

We are making improvements to UA, see the list of changes.



Follow this Question

Answers Answers and Comments

119 People are following this question.

avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image

Related Questions

Strange point light on generated mesh 1 Answer

Terrain heightmap isn't smooth 0 Answers

add mesh to trail renderer and MAKE IT SOLID !! 1 Answer

Generating a 3D Mesh from Audio data 0 Answers

Sticking object on mesh 0 Answers

  • Anonymous
  • Sign in
  • Create
  • Ask a question
  • Spaces
  • Default
  • Help Room
  • META
  • Moderators
  • Explore
  • Topics
  • Questions
  • Users
  • Badges