• Runtimes
  • Unity Bounding Boxes and integration with Physics engine

Hi all,

I have been working on developing some additions to the Unity runtime which will allow bounding boxes to work with Unity's physics engine (Physx, but hopefully later Box2D). Hopefully Nate will be able to pop this work into the official runtimes after further testing. So here is a breakdown of it...

[Jump animation + physics interaction]

What has been implemented so far...

  1. Bounding Boxes are created individually as mesh colliders at a default size/rotation/position relative to a child GameObject representing their parent bone. This GameObject's local transform is then updated relative to the root GameObject each frame, so the collider's geometry/vertices don't have to be updated each frame, allowing the overhead to be minimal. Colliders are enabled/disabled at runtime based on the animation. Collider geometry is checked each time you run the game, so you can be sure any changes you make in the Spine Editor will be reflected in game.

  2. If bounding boxes are not present, then the impact on the runtime performance is minimal. A single boolean is checked each frame. If not enabled nothing happens 🙂

  3. The GameObjects which house the Bounding Box colliders are persisted even if you disable Bounding Boxes, remove a Bounding Box from your animation, etc. Instead the colliders are simply disabled and the GameObjects are no longer updated. This is so that you can attach scripts to the GameObjects and not have to worry about losing information associated with those scripts (variables you have set and so on). You can then decide what to do with those scripts before deleting the unwanted GameObjects for yourself from the heirarchy.

  4. It is very simple to toggle on/off all Bounding Box collisions. Yous simply disable/enable a script which sits on the root GameObject next to SkeletonComponent.cs/SkeletonAnimation.cs called SkeletonCollisions.cs. Easy peasy 🙂. You can also override (turn off) individual Bounding Box colliders if you would also like to do so.

  5. A Rigidbody is attached to the root GameObject, allowing for all of the Bounding Boxes to form one large collider. Collision scripts can be attached to the root and still detect which Bounding Box collider actually intersected. This also means that Bounding Boxes which intersect other Bounding Boxes of the same animation will not trigger collisions. However, one can attach Rigidbodies to the child colliders to allow collision scripts to be associated with those colliders only and this is required to mix trigger and non-trigger Bounding Boxes, as well as allow for triggers to provide information on with Bounding Box was triggered in the collection of Bounding Boxes forming the collider (not present when using Triggers 🙁 ).

Good news aside though, there are a few issues associated with the implementation 🙁

Issues...

  1. Currently, you can specify concave Bounding Boxes in the Spine Editor, and rightly so. However, in Unity (and most physics engines) concave colliders will not collide with other concave colliders. Therefore, I have started trying to look into Optimal Convex Decompositions and how I can go about breaking down concave Bounding Boxes into multiple convex polgons, which will sum up to represent the initial concave shape.

  2. Carrying on from 4. above, the introduction of individual Rigidbodies per Bounding Box creates issues. They introduce more overhead (more Rigidbodies) as well as allow Bounding Boxes which intersect other Bounding Boxes of the same animation to trigger/collide with each other. This could be overcome by modifying the layer collision matrix so certain layers can't collide in Unity and specifying different layers for the Bounding Boxes in an animation. However, you only have 30 or so layers to fiddle with and you don't want to end up with two different skeletons not colliding/triggering through trying to avoid self collision.

  3. The Bounding Boxes are created using Mesh Colliders. These are the most expensive physics collider in Unity, and may well destroy the speed of mobile games for example. It may be better to try and approximate the Bounding Boxes with a collection of Box Collider primitives, which are much faster for physics computations. However writing an algorithm to do this is going to be tricky. See the image below which touches on this.

I'm looking into automating the approximation as best I can with an algorithm, but for now will probably just provide an option for the user to create n number of Box Colliders in child GameObjects to approximate the Bounding Box shape and position/rotate/scale them by hand. This will obviously be optional to the existing Mesh Colliders created and will disable those mesh colliders while enabled. The parent bone GameObject will be transformed as necessary every frame, and the child Box colliders will automatically inherit the new scale/position/rotation so the overhead should still be minimal.

  1. Finally, you have to provide a depth for the Mesh Colliders created otherwise Unity throws a wobbly. This means that you cannot have a depth of zero 🙁 . This is not configurable just yet, but will be very shortly. I am pretty sure this won't become an issue with the introduction of Box2D.

And so...

Hopefully support for Box2D, when implemented in Unity, will be simpler to add than stooopid PhysX. Box2D is obviously more suited to what I imagine most Spine users will need (being 2D and all). However it is always good to provide support for both, as some people (like me) may need 3D physics in their game.

Anyway, I'm not going to make the code available right now, and instead I will be sending it to Nate for review when finished and hopefully integration into the Git Repository.

If anyone has any thoughts or suggestions then post them here and we can all have a chat about how to continue and improve this. Spine is awesome, so lets support these guys and help them make it even better!

Rob

Related Discussions
...

Thanks Rob! Looks very interesting. I'll dig into the code as soon as I have a chance.

Generating convex polygons from concave polygons is doable, if a little tricky. Approximating using box colliders is pretty nasty. 🙁

I wonder, maybe for Unity the colliders should be created manually in Unity? We could have an API (eg a ColliderAttachment) so they attach to bones and transform along with the bones. This isn't as nice, but converting Spine bounding boxes to colliders looks to be turning out pretty tricky and not all that flexible (eg, using box colliders is difficult).

One downside to attaching colliders in Unity might be if you want to change bounding boxes during animations. Maybe we could match up the names of bounding boxes in Spine with the names of colliders in Unity so the colliders could be attached/detached.

Thanks again for tackling this! I really appreciate it. 🙂 I'll give some feedback on the code as soon as I can.

Hi Nate,

Yeah I agree, Box Collider approximation would be horrible, which is why I was thinking the user could just create them manually if they really need performance increases. Optimal Convex Decomposition is definitely do-able, but just a little overkill for the application perhaps. I'm trying to wrap my head around it anyway. http://mnbayazit.com/406/files/OptimalC ... azelle.pdf

The collider attachment idea is a good one, it's sort of what I was trying to achieve. The current implementation of Bounding Boxes that I sent you do attach to child GameObjects representing Bones and transform along with the animation. The individual colliders are also switched on and off at runtime as stipulated within the animations to represent attach/detach. However you're right that it could be easier to build in Unity, and I am coding in an option to allow the Bounding Box to be built as a mesh collider as per above and then have the user go in and 'trace/build' over it with Box, Sphere and Capsule primitive colliders, which will then be used instead of the mesh collider at runtime. In fact we can even allow the user to toggle between the two if they have built the primitive version, based on what hardware they are running on. UnityBoundingBox.cs could easily be renamed to and house all of the functionality required by ColliderAttachment.cs.

At the moment when you want to change Bounding Boxes during an animation, given that both are present, one is simply switched off (but not destroyed) and the pother switched on. If the second one is not present it is created then switched on. Afterwards it is persisted, so that next time it needs only to be switched on.

I will be looking into the triangulation stuff that you sent me, I have to admit that I am still trying to wrap my head around it, and also I would love to investigate a surefire solution to Concave triangulation. Given that the meshes/Bounding Boxes people are likely to generate going to have a low number of vertices, I imagine that you triangulation won't mess up though. Also given this expected low number of vertices, an exhaustive search to optimally triangulate may be okay too. I'm not sure though.

Also, at the moment it is worth mentioning that bounding boxes are matched on name only, so unique names are required for bounding boxes. Also, if Bounding Boxes are not found upon skeleton initialisation their geometry will have to be created at runtime when they are encountered. I would therefore like to include an option to quickly step through all of the animations and determine all Bounding Boxes that will be present as well as their default size and orientation. At the moment I use bone.WorldScaleX and so on to do my initialisation and mesh vertice creation. Is there a way to iterate through like I want to find the colliders and have their calculate their correct scales, rotations etc. Or will I have to somehow grab the information from the .json file and work out the relative unity sizes, rotations, etc associated with the instance of the skeleton within Unity myself?

Many thanks,

Rob

Attachment names are unique per slot. Slot names are unique per skeleton. You could use slot + "-" + attachment name to be unique for the whole skeleton.

Not sure what you mean, that you use bone.WorldScaleX? You can use SkeletonBounds to collect the polygons and bounding box attachments, then create colliders from those.

I wonder if we could make a way to attach arbitrary game objects to a slot. This could be done by implementing a GameObjectAttachment, but that isn't too useful since the gameobject won't use the skeleton's draw order. Maybe it would be simpler to have the attached gameobjects always be drawn on top of the skeleton, and have an API as simple as skeleton.attach(gameobject, bone). I guess the attachment offset would be the gameobject's transform relative to the bone. After attachment, the gameobject would translate/rotate/scale along with the bone. To solve the need for colliders, you just attach them to the bones and you're all done. It could also be useful for particle effects, etc.

Yeah that sounds great Nate. I had thought about doing a generic implementation of creating child GameObjects for different bones, allowing the user to attach various bits and pieces to them, and you mentioning attaching particle emitters and so on makes it sound like it could be quite useful.

Regarding the bone.WorldScaleX thing, sorry for my poor explanation. At the moment what I do is create a child GameObject and place it at a location equivalent to the Bounding Boxes parent Bone. I do this by setting its transform.localPosition to the Bone WorldX/Y, it's transform.localRotation is set to Bone WorldRotation and scale equivelantly too. I am able to do this because they (WorldX/Y) are relative to the root GameObject, which is the direct parent of this child GameObject.

I then create the mesh from the supplied polygon, however I create it relative to the Bone GameObject I just created rather than the root GameObject, something like this when grabbing the vertices...

vertices.Add(new Vector2(polygon.Vertices[i] - parentBone.WorldX, polygon.Vertices[i + 1] - parentBone.WorldY));

Above, you can see that I offset against the Bounding Box's bone's position. Given the above setup, I then only have to update the position/rotation/scale of the child GameObject each frame to reflect the Bounding Box's bone's position/rotation/scale. The Mesh representing the Bounding Box is inherently transformed by doing so. Therefore, I avoid having to update the mesh collider's vertices each frame to match the Bounding Box polygon. It seemed like a good solution at the time as I thought it would have less overhead. But the issue is that if a Bounding Box is not present on initialisation (switched off) then the mesh geometry will not be created. When it then needs to be switched on for the first time in an animation, the mesh geometry will need to be caculated at runtime. I just wanted a way to iterate through all potential Bounding Boxes, even those switched off at the start, so I can create them. However, I figured looking in the .json file for them might not be sufficient for setup given that I also need the WorldX/Y and so on calculations for the initial position, rotation and scale of the parent Bone.

Hmmmm... I hope you can make sense of the above, because reading it back, it does sound a little convoluted. Also if you think the above method is stupid then I can move to updating vertices each frame (which is simply copying your polygon vertices each frame).

Again many thanks for your time Nate

Rob

The bounding box vertices never change, they only get transformed by the bones. It seems there are two approaches. One is to create a gameobject per bounding box and use the gameobject to transform the collider. The other is to have a single mesh collider on the skeleton gameobject. Is that right? Fewer gameobjects probably performs better, but that is just a guess. I think using a single mesh was a good idea for the skeleton images, I'm not sure about for colliders.

To go through all the bounding boxes and create the colliders, you can go through the SkeletonData's skins and all the attachments for each one. Any bounding box attachments you find will need a collider. However, creating them at runtime might not be so bad, since a SkeletonData could have many skins with many colliders that are never used. Or, if a single collider mesh is used I guess it isn't an issue to just add more vertices, sort of how the SkeletonComponent mesh stuff works.

Aha, of course the bounding box vertices do not change, my mistake. I think with my current implementation though, there is not even a need to calculate the polygon each frame, instead we just update the Bone GameObject transform to match that of the bone. I looked at the 'one giant mesh' approach, but put it aside as it doesn't allow the user to tell which bounding box actually did the collision, which the current method does. I also don't think moving GameObjects has too much overhead, but without trying both and benchmarking I cannot be sure.

Thanks for the SkeletonData skin tip, I will look into that 🙂 The reason I wanted this is in case people wanted to have all Bounding Boxes, even infrequently used ones, available in the editor. Then they could attach collision scripts to them, other scripts or components, fiddle with them (e.g. approximate with box/sphere/capsule colliders) and so on before running the game. The, when they are encountered in the game, even if for only one frame in one animation, everything is setup for them to work as the user wants. Was just an idea though, I know I would find it useful.

Thanks for all your help, advice and tips so far Nate, I really appreciate you taking the time 🙂

Rob

I'm not super familiar with the Unity workflow, but attaching scripts to things that are created dynamically seems a little fragile. Maybe you could get similar functionality with an callback/C# event?

True a giant mesh doesn't tell you which collider hit. Hmm. Can you attach multiple colliders to the same gameobject?

Unity is a massive pain. It does not allow attaching multiple colliders to a single GameObject, which is why I went for the 'attaching individual ones as children' method. I agree that it may seem a bit fragile attaching scripts to the GameObjects that are created for the Bounding Boxes, but I thought it would be alright since they aren't going anywhere. That is, once they are created, they aren't removed unless done so by the user. It's also another reason why I wanted to iterate through all Bounding Boxes. I figured if I found one not being references anymore, then it would be disabled (but not removed) and a warning would be logged informing the user that this Bounding Box is now redundant. Then they could decide what to do with it and when. If they were to re-implement it in their animation at some point, and they had left it there doing nothing, then it would simply be re-enabled and mesh geometry updated to new geometry. All their initial scripts and so on would still be intact and ready to roll. This is why I figured it would be okay, but I will look into callbacks/delegates as a way of implementing collision functionality instead. I guess I just personally wanted the ability to store scripts on the GameObjects, but others may not want this. Also, at the moment the implementation would obviously not support FFD Bounding Boxes, so I'm guessing I will also need to provide an implementation with the updating mesh vertices every frame based on polygons calculated in order to support this feature when it is implemented.

Cheers Nate, I wish Unity wasn't such a pain in the ass but that's kinda why I figured I would help at tackling it, I seem to have run into quite a few of its annoying flaws by this point 🙂

2 tháng sau

Just in case anyone is re-directed here for Bounding Box physics integration please find attached the state of my experimentation so far. It is highly untested and I haven't been able to work on it in ages but because of recent interest in physics integration I figured I would post for others.

I had to modify the following files:

// Modified to include Bounding Box support
SkeletonComponent.cs

// Added store/retrieval of Bounding Box Attachment Bone which I needed
// Pretty much a one line code addition in each
SkeletonBounds.cs
BoundingBoxAttachment.cs

Then the rest of the new files are in the root of the project. in the existing runtime files i modified i'm pretty sure i managed to mark everything I slotted in there with an /** Added **/ comment. Searching for that should bring up all modifications/additions. I also used the triangulator from the Unity forum.

Adding/toggling the SkeletonCollisions.cs script on the root gameObject (the one with SkeletonAnimation) should enable/disable collisions. Automatic Rigidbody params just sets the Rigidbody to kinematic and the constraints to 2D. The rest is as in the above posts, so boxes should update/disable/enable as required by animations. Currently only works with 3D physics engine, will create concave colliders which will not collide with other concave colliders (need to work on convex decompositions or some other solution).

Will continue to work on this soon, as my project is starting to need robust Spine && Unity physics integration. Can't wait to get these two playing together nicely.

Kind regards to all,

Rob

Thanks for sharing Rob! I wish I had the bandwidth to dig into this right now. 🙁

một tháng sau

you are really my Jesus Christ!Rob!
and I got a question here
when I use the script to detect player hit, for example, a otaku is hitting a bookmonster with a mouse sword, when the otaku is playing attack animation, I enable a attack bounding box in the attack animation from 8 frame to 10 frame(it's about 0.1 s I think), and the attack bouding box does intersect with the monster's body, but, I recieved the OnCollisionEnter message many times in one attacking animation, which is not needed. so what can I do? 🙂

and sometimes, I can't get the message even all my attack bouding box is intersect with the monster...

Maybe I can do someting useful!

finally I solve my question through this way...
firstly, I create two attachment in spine, and
er...
set three rect in SkeletonComponent.cs
public Rect _rect; //collide box
public Rect _rectAtt; //attack box
public Rect rectVis; //vision box
and then when the anim init, get the attachment data, and set those rect, when the anim update,
update these rect, and then, in then enemy update, check if its "
rect" overlap with attacker's _rectAtt,
and I got my answer

say something...aha?

What can I say? 🙂 Sounds like your stuff is working, which is great.

2 tháng sau

Could this work that has been done here be used to impliment a "Ragdoll" effect with a spine character in Unity?

For that, you would need code that causes physics to influence the skeleton, not just the other way around as bounding boxes do. But I guess it could build on it. There's a lot of work to do though.

Yea lots of work I'm sure. Ragdoll with spine is something that would be AMAZING. Sounds like its time to start a Kickstarter! :clap:

You don't need anything special for that besides what Spine and the runtimes can already do. I think somebody already did it before in unity. Don't quote me on that though