• Products
  • Solutions
  • Made with Unity
  • Learning
  • Support & Services
  • Community
  • Asset Store
  • Get Unity

UNITY ACCOUNT

You need a Unity Account to shop in the Online and Asset Stores, participate in the Unity Community and manage your license portfolio. Login Create account
  • Blog
  • Forums
  • Answers
  • Evangelists
  • User Groups
  • Beta Program
  • Advisory Panel

Navigation

  • Home
  • Products
  • Solutions
  • Made with Unity
  • Learning
  • Support & Services
  • Community
    • Blog
    • Forums
    • Answers
    • Evangelists
    • User Groups
    • Beta Program
    • Advisory Panel

Unity account

You need a Unity Account to shop in the Online and Asset Stores, participate in the Unity Community and manage your license portfolio. Login Create account

Language

  • Chinese
  • Spanish
  • Japanese
  • Korean
  • Portuguese
  • Ask a question
  • Spaces
    • Default
    • Help Room
    • META
    • Moderators
    • Topics
    • Questions
    • Users
    • Badges
  • Home /
avatar image
0
Question by Rich_XR · Nov 10, 2020 at 09:13 PM · raycastaudioruntimedatageometry

Using raycast to sonify point cloud data and geometry of shapes

Hello community,

I am working with the blind and the visually impaired to assist with object identification.

I am looking for development advice for the best approach to sonify shapes with accuracy using sound in a 3D unity scene at runtime.

I have been using existing datasets of simple objects and also iPad Pro and have seen the geometry mesh and point cloud information that can be captured. I am really interested in the communities opinion of how sonification of this data could be approached. I have a procedural and spatial audio background but am struggling with getting the dataset/data to a usable format for audio implementation.

I have looked into volumetric audio, however surfaces don’t have great definition with this method and if blindfolded you could not detect that object shape or detail of edges.

I feel as though a raycast, focused, almost an audio scrubbing method, from FPS view then spatialised would possibly be the best approach, although I want to know your thoughts, I am unsure dev wise how to implement this. Would be great to scan through the point cloud or geometry data and extrapolate it so that I could use those parameters to drive sound. I have attempted to draw this https://share.icloud.com/photos/0o81uJF3vmaz6YM5vOm-efzzQ

I would need to leverage the raw point cloud data or geometry to sonify, I am just really unsure on this and would appreciate any advice on this.

This video is unique but is actually quite interesting in terms of approach, if a user/player had control of this it would be an interesting way for a blind person to inspect a complicated object. https://vimeo.com/195054847

I would want to start with boxes and spheres first and the consideration of someone moving round the object needs consideration also.

Thanks in advance

R

Comment
Add comment · Show 1
10 |3000 characters needed characters left characters exceeded
▼
  • Viewable by all users
  • Viewable by moderators
  • Viewable by moderators and the original poster
  • Advanced visibility
Viewable by all users
avatar image elenzil · Nov 11, 2020 at 07:03 PM 0
Share

wow, interesting. I'm not super familiar with the data one gets back from ARKit etc, but one approach might be to have three separate sounds, one anchored to the left ear, one to the right, and one forward, and modulate them in some way based on corresponding raycasts. For example create a truncated cone or something extending in each of those three directions, and adjust the sound volume or pitch according to how many points are closest to the device, and adjust some other property like repeat rate based on proximity. .. hmm.. Actually I guess I'm thinking about head-mounted devices. For a handheld I think your idea of a single probe makes sense.

0 Replies

· Add your reply
  • Sort: 

Your answer

Hint: You can notify a user about this post by typing @username

Up to 2 attachments (including images) can be used with a maximum of 524.3 kB each and 1.0 MB total.

Welcome to Unity Answers

The best place to ask and answer questions about development with Unity.

To help users navigate the site we have posted a site navigation guide.

If you are a new user to Unity Answers, check out our FAQ for more information.

Make sure to check out our Knowledge Base for commonly asked Unity questions.

If you are a moderator, see our Moderator Guidelines page.

We are making improvements to UA, see the list of changes.



Follow this Question

Answers Answers and Comments

225 People are following this question.

avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image avatar image

Related Questions

help with multiple sounds 0 Answers

WAV byte[] to AudioClip? 3 Answers

streaming local sound files on iOS 1 Answer

Generating Mesh at runtime, vertex count different 0 Answers

AudioClip.Create() 1 Answer

  • Anonymous
  • Sign in
  • Create
  • Ask a question
  • Spaces
  • Default
  • Help Room
  • META
  • Moderators
  • Explore
  • Topics
  • Questions
  • Users
  • Badges