Spine player - React Native - Expo

anthomaxcool

Hi, I'm trying to use the spine-player in a react-native app using Expo. I have this code

import React, { useRef } from "react";
import { ExpoWebGLRenderingContext, GLView } from 'expo-gl';
import spine from '@esotericsoftware/spine-player';

const Character = () => {
  const glViewRef = useRef<GLView>(null);

  const onContextCreate = async (gl: ExpoWebGLRenderingContext) => {

    const myString = glViewRef.current!.nativeRef!.toString();

    new spine.SpinePlayer(myString, {
      jsonUrl: "assets/spine/spineboy-pro.json",
      atlasUrl: "assets/spine/spineboy-pro.atlas",
      animation: "walk",
      showControls: false,
      premultipliedAlpha: true,
      backgroundColor: "#cccccc",
      alpha: true,
      preserveDrawingBuffer: true,
      defaultMix: 1,
      controlBones: ["root"],
      viewport: {
        debugRender: true,
      },
    });
  };

  return (
    <GLView
      style={{ flex: 1 }}
      onContextCreate={onContextCreate}
      ref={glViewRef}
    />
  );
};

export default Character;

It seems like myString is undefined. Since the SpinePlayer requires a string or a HTMLElement, I can't figure out how to make this work. Is it possible? If so, how?

Thank you

Nate

I'm not familiar with React, but can't you pass a string ID of an HTML element or the HTML element itself? The HTML element should be a DIV, not a GLView. The player does the rest.

anthomaxcool

Problem is that it's not "React", but rather "React-Native". There's no such thing as an ID referencing an HTML element.. Actually, I initially thought that myString would reference the GLView element, which would in turn act as a div. I tried another approach

const Character = () => {
  useEffect(() => {
    new spine.SpinePlayer("test-id", {
      jsonUrl: "assets/spine/spineboy-pro.json",
      atlasUrl: "assets/spine/spineboy-pro.atlas",
      animation: "walk",
      showControls: false,
      premultipliedAlpha: true,
      backgroundColor: "#cccccc",
      alpha: true,
      preserveDrawingBuffer: true,
      defaultMix: 1,
      controlBones: ["root"],
      viewport: {
        debugRender: true,
      },
    });
  }, []);

  return (
    <View nativeID="test-id">
    </View>
  );
};

export default Character;

But I get the following : undefined is not an object (evaluating 'new _spinePlayer.default.SpinePlayer')

Actually I just read the source code for SpinePlayer and it uses the document variable, which doesn't exist in react-native, since it's made to build on mobile devices, not a web page. However, I see that a library exists for Flutter, which is also for mobile development.. So I guess there's a way to make this work for react-native.. ?

Nate

anthomaxcool Problem is that it's not "React", but rather "React-Native".

Yeeeah, I'm afraid that doesn't mean anything to me. 😬 If your UI toolkit is not actually HTML, then SpinePlayer is not going to work. It looks like you can use a WebView to render HTML, so you could try putting the SpinePlayer into a WebView.

anthomaxcool I see that a library exists for Flutter, which is also for mobile development.. So I guess there's a way to make this work for react-native.. ?

Sure, there are Spine runtimes for many languages, game engines, UI toolkits, etc:
http://esotericsoftware.com/spine-runtimes/
I'm not sure how React Native renders or which generic runtime would get you closest. Ideally you use our generic Spine Runtimes and just add the rendering layer on top. My colleague, Mario, may have more input.

Mario

The Spine web player is build on top of WebGL. React Native has no in-built support for WebGL. Expo comes with GLView but is not compatible with WebGL entirely. The most problematic part is context creation and management, which works differently in Expo compared to browsers. It also requires a call to gl.endFrameEXP() to actually swap buffers, and it seems there's no way to use requestAnimationFrame either.

In short, the web player will not work with React Native (with and without Expo) as it's not a fully compatible WebGL environment. You could try to hack the existing sources to take a WebGL2RenderingContext, but that's not for the faint of heart. Especially since there also seem to be heavy limitations on what and how resources like textures can be loaded with Expo.

anthomaxcool

Could I possibly render the animation server side and return the result to the app? Do every runtime actually need the "frontend" to be able to render the animation itself?

Nate

Anything is possible. What would be returned to the app? An image? GIF? If not an image, what would you be rendering in the app?

anthomaxcool

In my specific case, all I need is to render an idle animation or a walking animation. There's no control for the player, therefor a GIF would be ideal. Would I need an additional lib to "simulate" a browser or something like that? If generating a GIF backend is an option, I would do it from an AWS Lambda, so serverless which supports c# or python that I saw in the available runtimes.. it also supports javascript, but as I understand it, I cannot use javascript without using "html variables" (like "document")?

Nate

Can you export GIF from the Spine editor? That can work if you don't need to highly customize your skeleton for each GIF.

To make a GIF with the Spine runtimes, you need a way to render images to a buffer so you can write GIF frames. For example:
https://github.com/EsotericSoftware/spine-runtimes/blob/4.1/spine-libgdx/spine-libgdx-tests/src/com/esotericsoftware/spine/PngExportTest.java

You could run Spine with CLI parameters each time you need to render a GIF, but it may not be fast. Spine uses OpenGL to render frames for the GIF, so your server would need that capability.

anthomaxcool

I'll see what I can do with that 🙂 However, I do need to customise the character.. like to add a hat. I do believe I can use OpenGL, so I'll try that! I'll come back here when I have a solution to offer.. or ran out of ideas! Thank you!

anthomaxcool

I'm back! Sadly not with the news I initially wanted.

Before I say anything, just let me say that I'm new to rendering anything and before this project I never really used react-native expo. It's quite possible that someone with more experience would have no problem making it work.

So, I gave up on the GIF. Even if it could have worked, I didn't really like this solution. I tried few runtimes, like spine-canvas, spine-webgl, spine-player, spine-three and was about to test pixi-spine. I feel like spine-webgl and spine-three were really close to work. When I gave up debugging, if I recall correctly, the problem was that it was trying to fetch the image using a localhost url. Expo uses Metro to create a server that exposes the app on a "exp" url which allow us to run the app on our cellphone without having to install the app. So trying to fetch an image using localhost just couldn't work. If I recall correctly it was always in the AssetManager. I probably should have been able to create my own AssetManager and require('myFile') instead of using a url.

I ended up using ThreeJS by itself and I could read the file as a string and load the file using a base64 string, so no url involved.

The part where it render something in GLView, I feel like with spine it wouldn't be a problem like mentionned previously since I could use both requestAnimationFrame(render) and gl.endFrameEXP(). This structure was identical for both spine and three.

const onContextCreate = async (gl: ExpoWebGLRenderingContext) => {
  [...]
  // Method to call to start everything
  const animate = () => {
    requestAnimationFrame(animate);
    render();
    gl.endFrameEXP();
  };
}

return (
  <View>
    <GLView
      onContextCreate={onContextCreate}
      style={{ width: windowWidth, height: windowHeight }}
    />
  </View>
)

Sorry I couldn't make it work! I hope it does one day!

Nate

FWIW, you can use base64 with AssetManager:

assetManager.setRawDataURI("your.json", "data:application/json;base64,eyJza2VsZXRvbiI6ey<snip>");

You can use this for all the assets: JSON, binary, atlas, PNG images.

anthomaxcool

Oh that's interesting I didn't see that! Well, with the onContextCreate example and this way of loading asset, it probably would have worked. That's good to know thank you!

HauHau

@anthomaxcool Were you able to make spine working with expo-gl/react-native?

I'm trying to run spine-threejs on react-native + expo-three + expo-gl setup.
I have already rewrited AssetManager, AssetDownlaoder and ThreeJsTexture.
At my current point React-Native is loading correctly atlas file and json file.
However, I have still problem with texture. Looks that any way of loading texture is working, I'm always getting "WebGLRenderer: Texture marked for update but no image data found" error.

Is there anybody who managed how to run Spine on expo-three?

adamgerthel

It's probably possible to put together a Spine library for React Native considering there's an official C++ runtime and C++ seems to be recommended with TurboModules. See for instance.

HauHau

adamgerthel
Yes, it is what I did, just instead of turbo modules I have choose fabric component (with additional support for old arch).
Initially I have tried approach with rendering via Android's Canvas through JNI, however this approach was problematic and slow.
So, because of that I have ended with implementation based on Skia which works very well.

adamgerthel

HauHau Wait - so you have Spine working in React Native through Skia?

HauHau

Yup I have.
Integration spine2d with Skia is relatively simple.
In my case I have used SFML implementation of Spine2D as a base for further code changes. Glueing together SFML impl. with Skia is easy, hardest part is just to find equivalent operations in Skia to properly render vertexes.
And as I don't wanted to waste time on figuring out how to compile Skia, I have used precompiled binaries of Skia which comes with @shopify/react-native-skia package.

On Android/Java side I have view with Bitmap. This Bitmap object is passed down to C++ through JNI. Then Skia can draw directly on Bitmap's pixels.
Android's choreographer is controlling frames rate and animation loop.

At the moment I'm bit busy, but when I'll find more time I can open source my solution on GitHub. I just need finish JS<->Java bindings and do some cleanup.

--
Forgot to add, my solution supports only Android as I don't have any Apple device. However integration with iOS should be straightforward.

adamgerthel

HauHau Interesting!

Off topic, but what's your use case for RN + Spine? Game development?

HauHau

Initially game development - I believe that RN can be suitable platform for specific types of games.
Also I'm very interested in experimenting with Spine2D for interface elements and as Lottie/Rive replacement.

adamgerthel

HauHau Heh, same here, that's what led me to this forum thread. Our games are made in React Native. Check out https://play.google.com/store/apps/details?id=com.gotterdammerung.redtide

Trang sau »