How to reliably send Kinect v2 data over MQTT?

Question

I am trying to send Microsoft's Kinect v2 body data over MQTT to effectively map skeletal data without direct connection to the Kinect, but I can't seem to deserialize a Body[] correctly. Essentially, the Kinect is connected to Computer A and processes the data and publishes it over MQTT, where Computer B listens and retrieves the data, then maps joints to its screen. I am just having trouble sending the Body[] over MQTT. I am publishing the Body[] every frame in Update().

My current setup is using Newtonsoft's JSON.Net to serialize a List taken from Body[] and publish it to MQTT (using https://github.com/vovacooper/Unity3d_MQTT). I used this since the Body class isn't serializable (so I can't use JSONUtility?). My current set up:

Computer A

 void Update() {
     ...
     //trackedBodies is a List<Body> that contains the tracked Bodys
     //client is MQTTClient that is connected
     string bodyData = JsonConvert.SerializeObject(trackedBodies);
     client.Publish("test", System.Text.Encoding.UTF8.GetBytes(bodyData));
     ...
 }

Computer B

 void client_MqttMsgPublishReceived(object sender, MqttMsgPublishEventArgs e) {
         //Check MQTT for data, then deserialize
        List<Body> bodyData = JsonConvert.DeserializeObject<List<Body>>(System.Text.Encoding.UTF8.GetString(e.Message));
         Debug.Log(bodyData);
 }

When there are no bodies being tracked, I get an empty Body[].

 System.Collections.Generic.List`1[Windows.Kinect.Body]

When there are bodies being tracked, MQTT seems to halt and stop working on the JsonConvert.DeserializeObject() line. I am not getting errors, and Update() is still being called.

My limitations are that I need to use Unity and MQTT.

I know that was a lot but I would appreciate it if anybody knows how to help with what I currently have, or suggest a better alternative to my problem.

Answer 1

Best Answer

Answer by char1997 · Feb 16, 2018 at 01:14 AM

I see that some people also may have the same question. I have finally gotten a working solution for myself. I personally just exposed the Body object by creating a custom 'pseudo' class to represent the body and hold the Joint and JointOrientation arrays. I was able to send a list of these custom objects through MQTT by serializing them with Json.NET

Cheers.

Add comment · Share

How to reliably send Kinect v2 data over MQTT?

1 Reply

Your answer

Welcome to Unity Answers

Follow this Question

Related Questions