Swap Spine Images from Web Player runtime

Davide

Webstorm is perfect for the job.

The easiest way to setup a TS+React environment is by using some tools like create-react-app. Once you have installed it, you just npx create-react-app my-app --template typescript and your environment is ready to be used.

I personally use vite when I have to do something with react.

The other way is to setup a typescript project from scratch and then add missing react types.

EnricDelgado

Davide
Thank you for your kind help!! Now I've finally managed to successfully retrieve the attachments!

Now I get to the complicated part of getting the images attached to those atttachments for them to be displayed in the screen, and change them by the ones the user provide as input. I'm trying to follow the Loading Skelleton Data guide, but I'm getting kind of lost when it comes to the Texture Loader.

Where are the images stored? - Atlas, I'm asuming - and how can I access them, retrieve them and modify them? Because once I have the new user input's images, I think I can add them by sending the new image through the setAttachment method as you suggested. - maybe by modifyng the page or region property of the current attachment? I'm still trying to figure that out, sorry for asking so many questions! 😵‍💫

Davide

EnricDelgado

Doing that is not trivial. You might expect that since you are seeing your single images displayed, you could simply swap them by providing yours. Unfortunately, it's harder than that.

Your single images are actually loaded from the png files that pack together all the images. The atlas txt file allows detection of how the single images are organized in your png atlas.

The spine player (and the underlying spine-webgl) uses WebGL to render your images. The canvas where you are seeing the images is actually a webgl context. You cannot directly extract the images and display them outside that context.
Have a look at the dress-up example of spine-webgl.
In that example, a thumbnail of some attachments is shown to the user outside the webgl context. In order to do that an actual image is created by using the toDataURL method.
You could show individual images also by using other techniques.

The other thing to do would be the actual image swap. In order to do that, you have to load your image into webgl, then swap the region of the desired attachment or create a new attachment with that image.
To do that, it might be easier for you to wait for us to implement this feature. I can work on that, but need to close a couple of things before.

EnricDelgado

Davide
Thank you again for your response! I'm getting quite confused on how to work with it, for I'm getting difficulties understanding how Spine works on the back. I'll follow your suggestion I'll wait until the example is created.

Thank you so much for your help! 🙂

Davide

I didn't have the time to make an example yet, but you can use the following code to get an idea on how to do what you want.

As far as I understand, your goal is to allow a user to add their own texture to an attachment in a slot.
As in Spine, a slot can have 0 or 1 attachments enabled. An attachment has a region property that contains the information about the texture that has to be rendered.
What you want to do is probably to change the image on one of these attachments, or to create a new attachment with the desired image. To make things simpler, the code below shows the former case.

First, we patch the asset manager to load an image from a URL:

spine.AssetManager.prototype.loadTextureAtlasForSingleImageUrl = function (imgUrl) {
	const atlasPath = imgUrl.replace(/\.[^.]+$/, '.atlas');
	if (this.assets[atlasPath]) return Promise.resolve(this.assets[atlasPath]);
	this.start(atlasPath);

	this.start(imgUrl);
	let image = new Image();
	image.crossOrigin = "anonymous";
	image.src = imgUrl;
	return new Promise((resolve) => {
		image.onload = () => {
			const region = new spine.TextureAtlasRegion(new spine.TextureAtlasPage(imgUrl), imgUrl);
			region.u = region.v = 0;
			region.u2 = region.v2 = 1;
			region.page.width = region.width = region.originalWidth = image.width;
			region.page.height = region.height = region.originalHeight = image.height;

			const atlas = new spine.TextureAtlas("");
			atlas.pages.push(region.page);
			atlas.regions.push(region);

			// set the loaded texture into the page (and internally into all the regions)
			region.page.setTexture(this.textureLoader(image));

			this.success(() => {}, imgUrl, region.page.texture);
			this.success(() => {}, atlasPath, atlas);

			resolve(atlas);
		};
		image.onerror = () => this.error(() => {}, imgUrl, `Couldn't load image: ${imgUrl}`);
	});
}

Then, we use that function to load an image into a new atlas and get the first region:

const atlas = await player.assetManager.loadTextureAtlasForSingleImageUrl("assets/my-texture.png");
const region = atlas.regions[0];

Finally, we select a slot, take its attachment, change its region, and ask the attachment to update based on the new region information:

const slot = player.skeleton.slots.find(slot => slot.data.name == "raptor-body");
const attachment = slot.attachment;
if (attachment) {
	attachment.region = region;
	slot.attachment.updateRegion();
}

I tried this code on the basic example of spine-player.
This works for both region and mesh attachments.

EnricDelgado

Davide Thank you so much for your help!! I'll give a try to this and I'll get back with my results! Thank you so much! 🙂 <3

EnricDelgado

Davide

Thank you for your code and your help! I've managed to generate a ReactJS / Typescript functional variant of this - I'm adding the component's code for the project is too large to get uploaded.

By creating a new ReactJS project and adding this component - as well as the CSS for the Spine Viewer - the results can be checked.

// @ts-ignore
import React, {useEffect, useRef, useState} from 'react';
import {SpinePlayer, Skeleton, AssetManager, TextureAtlasRegion, TextureAtlasPage, TextureAtlas} from "@esotericsoftware/spine-player";
import './SpinePlayer.css'

function SpineViewer ({ jsonUrl, atlasUrl }) {
    const playerRef = useRef(null);
    const [imageAttachments, setImageAttachments] = useState([]);
    const [player, setPlayer] = useState<SpinePlayer>(null);
    const [skeleton, setSkeleton] = useState<Skeleton>(null);

    useEffect(() => {
        if(playerRef.current){
            if(player !== null) {
                player.dispose();
            }

            AssetManager.prototype.loadTextureAtlasForSingleImageUrl = function (imgUrl: string) {
                const atlasPath = imgUrl;
                if (this.assets[atlasPath]) return Promise.resolve(this.assets[atlasPath]);
                this.start(atlasPath);
            
                this.start(imgUrl);
                let image = new Image();
                image.crossOrigin = "anonymous";
                image.src = imgUrl;
                
                return new Promise((resolve) => {
                    image.onload = () => {
                        const region = new TextureAtlasRegion(new TextureAtlasPage(imgUrl), imgUrl);
                        region.u = region.v = 0;
                        region.u2 = region.v2 = 1;
                        region.page.width = region.width = region.originalWidth = image.width;
                        region.page.height = region.height = region.originalHeight = image.height;
            
                        const atlas = new TextureAtlas("");
                        atlas.pages.push(region.page);
                        atlas.regions.push(region);
            
                        // set the loaded texture into the page (and internally into all the regions)
                        region.page.setTexture(this.textureLoader(image));
            
                        this.success(() => {}, imgUrl, region.page.texture);
                        this.success(() => {}, atlasPath, atlas);
            
                        resolve(atlas);
                    };
                    image.onerror = () => this.error(() => {}, imgUrl, `Couldn't load image: ${imgUrl}`);
                });
            }

            updateSpinePlayer(jsonUrl, atlasUrl, spinePlayerStatesSetup);

            return() => {
                player?.dispose();
            }
        }
    }, []);

    useEffect(() => {
        if(skeleton === null) return;
        populateAttachments();
    }, [skeleton]);


    function populateAttachments(){
        const currentAttachments = [...imageAttachments];

        skeleton.slots.forEach(slot => {
            if(slot.attachment === null) return;
            
            const attachment = skeleton.getAttachmentByName(slot.data.name, slot.attachment.name);
            currentAttachments.push(attachment);
        });

        setImageAttachments(currentAttachments);
    }

    function updateSpinePlayer(skeleton: string, atlas: string, succesCallback?) {
        //@ts-ignore
        new SpinePlayer(playerRef.current, {
            skeleton: skeleton,
            atlas: atlas,
            success: succesCallback
        })
    }

    function spinePlayerStatesSetup(pl: SpinePlayer){
        setPlayer(pl);
        setSkeleton(pl.skeleton);
    }

    async function imageToAtlasRegion(attachment, imageURL) {
        const atlas = await player.assetManager?.loadTextureAtlasForSingleImageUrl(imageURL);
        updateAttachment(atlas.regions[0], attachment);
    }

    function updateAttachment(region, attachmentName) {
        if(skeleton === null) return;

        const slot = skeleton.slots.find(slot => slot.data.name == attachmentName);

        const attachment = slot?.attachment;

        if (attachment) {
            attachment.region = region;
            slot.attachment?.updateRegion();
        }
        else{
            alert("NO ATTACHMENT FOUND!");
        }
    }

    function generateInputFields(){
        return(
            <div style={{width: '100%'}}>
                <h2>Attachments</h2>
                <div style={{maxHeight: '50vh', overflow:'scroll'}}>
                    {
                        imageAttachments.map((item, index) => {
                            return(
                                <div>
                                    <h3>{item.name}</h3>
                                    <button id="attachmentInput" onClick={() => imageToAtlasRegion(item.name, '/logo192.png')}>Load Attachment</button>
                                </div>
                            );
                        })
                    }
                </div>
            </div>
        );
    }


    return (
        <div style={{padding: '3rem'}}>
            <div>
                <h1>Spine Slot Swap Example</h1>
            </div>

            <div style={{display: 'flex', flexDirection: 'row', justifyContent: 'center'}}>
                <div style={{width: '100%'}}>
                    <div ref={playerRef} style={{ width: '800px', height: '600px' }}></div>
                </div>
                {generateInputFields()}
            </div>
        </div>
    );
}

export default SpineViewer;

Now I've got one issue: would it be possible to generate a new spine project that includes the modified attachments? For download purposes! I couldn't find any references to this in the API </3

Davide

EnricDelgado

I'm happy you were able to achieve your desired result about texture swapping!

EnricDelgado Now I've got one issue: would it be possible to generate a new spine project that includes the modified attachments? For download purposes! I couldn't find any references to this in the API </3

You cannot create a .spine file project file, but you can create a new json file that can be imported into Spine. Here you can find the documentation on how to create a json containing skeleton data.

EnricDelgado

Davide
Oh I thought about that! But then, how do you manage to keep the newly-swapped images?

Davide

EnricDelgado

If you created a new attachment, you need to create new attachment entry for it in the json. Otherwise, if you just swapped an image, you might want to change the attachment path of the region/mesh attachment and other properties accordingly.

You might also provide as a download option a zip file containing the json and a folder with the newly added images. Then, when the user imports the json, they have to add the old and new images into the image folder of the project.

EnricDelgado

Davide
Sounds nice! But don't I need also the atlas too for a project to be able to generate the images inside spine?

Is there an example or a way on how to retrieve all the images currently displayed in the player, as well as the atlas? 🙂

Davide

EnricDelgado Sounds nice! But don't I need also the atlas too for a project to be able to generate the images inside spine?

You need an atlas file (and the respective atlas pages PNG), when you don't have access to the original image. In such a case, the Spine Editor offers you the texture unpacker feature to generate single images from the atlas pages. And that should be your case, since your application uses the player that needs atlas TXT/PNGs.

As you can see from the code above, you are creating a runtime atlas with a single page having a single region, without any padding, rotation, and so on. So basically, the atlas page PNG of your runtime atlas is your original image. Using this approach, you don't need to unpack anything. Even if you created an atlas text file from your runtime atlas, the texture unpack process would output your original image. If you don't have access to the original images, you might need to use the original atlas TXT/PNGs to obtain them.

Eventually, you could provide as a download option a zip containing: the new JSON, a folder with the new images, and the initial atlas TXT/PNGs. With that, the user should be able to: import the JSON, add the new images to the project's image folder, unpack the atlas, and also add the output images to the project image folder.

EnricDelgado Is there an example or a way on how to retrieve all the images currently displayed in the player, as well as the atlas? 🙂

As I already explained in previous messages, even if you see the region/mesh images as single pieces, what you are currently seeing is WebGL showing you portions of textures, which are the atlas PNG pages. So, unfortunately, the answer is no, there is no example that cleanly does that (other than the dress-up example I mentioned above that creates thumbnails), because there isn't any texture unpacking feature in spine-ts and most other runtimes, like in the editor.

If you really need to unpack the atlas, you can do it by yourself using the runtime atlas generated by the player during the initialization process contains all the information about the position, rotation, padding, etc., of each image. You might also use the atlas TXT file directly and interpret it, if you prefer. Be aware that there might be several use cases to support.

« Trang trước