Esoteric Spine, changing art of a slot with multiple attachment regions

[timetocode]

I've got a spine animation where the sprite of a certain slot changes. This is essentially the same thing as in the spine demo where they make a character blink by having two different images for the eye and having an animation that swaps between them.

Currently, due to spritesheets + pixi being superior imo to how spine wants to work, I use `hackTextureBySlotName` for everything.

Now my question is how I handle the situation where a slot has multiple images that are changed during an animation?

For example I have code like this:

model.hackTextureBySlotName('head', Texture.from(`/images/${ skin }/head.png`))
model.hackTextureBySlotName('thorax', Texture.from(`/images/${ skin }/torso.png`))
model.hackTextureBySlotName('pelvis', Texture.from(`/images/${ skin }/pelvis.png`))
// etc for all bodyparts of my skin, and gear held in the hands

But I would hypothetically need something like this for the slots that had multiple images:

model.hackTexturesBySlotName('left-upper-leg',[
    Texture.from(`/images/${ skin }/left-upper-leg-frame-1.png`),
    Texture.from(`/images/${ skin }/left-upper-leg-frame-2.png`),
    Texture.from(`/images/${ skin }/left-upper-leg-frame-3.png`)
])

Then my spine animation which has multiple images for the leg which it switches through would work.

I've attached a gif showing my animation, and how it defaults to the back of the legs turning white where the original animation would have "attached" a different piece of artwork -- the other frames of my leg art. The picture from the spine hierarchy is pointing to the two images that are switched between with red and blue.

 

Also here's the source code for hackTextureBySlotName incase anyone could point me to a way to modify it. I was surprised that the region was a single object and not an array, but maybe I don't really understand how spine structures the skeleton.

 

hackTextureBySlotIndex(slotIndex: number, texture: PIXI.Texture = null, size: PIXI.Rectangle = null) {
    let slot = this.skeleton.slots[slotIndex];
    if (!slot) {
        return false;
    }
    let attachment: any = slot.getAttachment();
    let region: core.TextureRegion = attachment.region;
    if (texture) {
        region = new core.TextureRegion();
        region.texture = texture;
        region.size = size;
        slot.hackRegion = region;
        slot.hackAttachment = attachment;
    } else {
        slot.hackRegion = null;
        slot.hackAttachment = null;
    }
    if (slot.currentSprite && slot.currentSprite.region != region) {
        this.setSpriteRegion(attachment, slot.currentSprite, region);
        slot.currentSprite.region = region;
    } else if (slot.currentMesh && slot.currentMesh.region != region) {
        this.setMeshRegion(attachment, slot.currentMesh, region);
    }
    return true;
}

Thank you!

[ivan.popelyshev]

Usually people just added extra attachments in spine and hacked them in pixi.

Unfortunately, I, as pixi-spine author, cant support this feature without input from community. I mean, someone have to solve it, and i'll put it in https://github.com/pixijs/pixi-spine/tree/master/examples

My estimation - probability that someone will answer you is low. You have to do it yourself and share with community

[timetocode]

Okay, I'll post back if I can figure out something decent.

[timetocode]

As it stands currently the issue is two parts:

1) `hackTextureBySlotIndex` will modify the *current* texture of an attachment in a slot, however if a slot has multiple attachments the attachment whose texture gets changed is whichever one is currently active. So this method gives an instant result, but won't change the hidden attachments. If called during an animation that changes attachments (esp if multiple frames), the results can be a bit random as it'll simply replace the texture in one frame of the animation.

2) using `getAttachmentByName` + `attachment.region.texture = newTexture` one can change the textures of attachments (instead of the texture of whichever current attachment is active in a slot) however this has no instant visual effect, and the new texture only appears the next time the spine entity changes to this texture

So the solution for me was to mix the methods. The attachment's texture should be changed, which will do a deep reskin of the skeleton including its hidden attachments and something like hackTextureBySlot should be used to change all of the currently visible attachments. Note: when i say deep reskin i'm referring to changing all of the textures in the model with new textures using pixi, rather than any of the spine skin features which I'm not using. The second part was to make  a version of  hackTextureBySlotIndex that would only change the texture of the attachment if the attachment was active.

const changeAttachment = (spine, slotName, attachmentName, texture) => {
    // changes the texture of an attachment at the skeleton level
    // (will not change currently visible attachments)
    const slotIndex = spine.skeleton.findSlotIndex(slotName)
    const attachment = spine.skeleton.getAttachmentByName(slotName, attachmentName)
    attachment.region.texture = texture
    // changes currently visible attachements
    // note: this is a modified version of hackTextureBySlotIndex
    changeAttachmentTextureIfActive(spine.skeleton, slotIndex, attachmentName, texture)
}

// modified to not change the texture of an attachment unless that attachment is currently active
const changeAttachmentTextureIfActive = (skeleton, slotIndex, attachmentName, texture, size = null) => {
    const slot = skeleton.slots[slotIndex]
    if (!slot) {
        return false
    }
    const attachment = slot.getAttachment()

    if (attachmentName !== attachment.name) {
        // do not change the texture of this attachment
        return
    }
    let region = attachment.region
    if (texture) {
        region = new TextureRegion()
        region.texture = texture
        region.size = size
        slot.hackRegion = region
        slot.hackAttachment = attachment
    } else {
        slot.hackRegion = null
        slot.hackAttachment = null
    }
    if (slot.currentSprite && slot.currentSprite.region != region) {
        setSpriteRegion(attachment, slot.currentSprite, region)
        slot.currentSprite.region = region
    } else if (slot.currentMesh && slot.currentMesh.region != region) {
        console.log('mesh regions are disabled for changeAttachmentTextureIfActive')
        //this.setMeshRegion(attachment, slot.currentMesh, region)
    }
    return true
}

I'm going to probably combine the code from the above into another function, but I figured I'd paste it to the forum while it still resembles the existing `hackTextureBySlotIndex` closely. One could probably combine the two and call it `hackAtachmentByName(slotName, attachmentName, newTexture)` if it warranted adding to the pixi-spine api, though it only would be helpful for people who do a lot of attachment swapping in their animations. Publicly exposing TextureRegion might be helpful too for more hacks. It's also worth noting that attachments in spine are not freely nameable -- they take on the name of the images, which makes me feel like i'm sort of hacking an extra feature into spine that was unintended by doing this.

(screenshot showing a spine skeleton whose attachments are named for programmatic use, as opposed to things like "iron-pick-axe.png")

An animation with changing attachments (the legs use different sprites for front/back at different points in the run):

In-game animations and gear changing randomly during the animation using the above code:

 

 

[ivan.popelyshev]

ok, so we have to rename it "hackTextureAttachment" and put into Spine.ts, right?

[timetocode]

I rewrote it in typescript and put it into a PR: https://github.com/pixijs/pixi-spine/pull/347

I'm sorry that I don't actually know how to build pixi, so I was unable to test the final version, you may want to make sure it works. The version of it right before I pasted it back into Spine.ts was correctly able to change attachments in my own game.

 

For reference for anyone reading this thread for attachment-hacking related purposes, here's the added code which contains the logic from earlier but refactored to more closely follow the style of Spine.ts' hackTextureBySlotName/index

/**
 * Changes texture of an attachment
 *
 * PIXI runtime feature, it was made to satisfy our users.
 *
 * @param slotName {string}
 * @param attachmentName {string}
 * @param [texture = null] {PIXI.Texture} If null, take default (original) texture
 * @param [size = null] {PIXI.Point} sometimes we need new size for region attachment, you can pass 'texture.orig' there
 * @returns {boolean} Success flag
 */
hackTextureAttachment(slotName: string, attachmentName: string, texture, size: PIXI.Rectangle = null) {
    // changes the texture of an attachment at the skeleton level
    const slotIndex = this.skeleton.findSlotIndex(slotName)
    const attachment: any = this.skeleton.getAttachmentByName(slotName, attachmentName)
    attachment.region.texture = texture

    const slot = this.skeleton.slots[slotIndex]
    if (!slot) {
        return false
    }

    // gets the currently active attachment in this slot
    const currentAttachment: any = slot.getAttachment()
    if (attachmentName === currentAttachment.name) {
        // if the attachment we are changing is currently active, change the the live texture
        let region: core.TextureRegion = attachment.region
        if (texture) {
            region = new core.TextureRegion()
            region.texture = texture
            region.size = size
            slot.hackRegion = region
            slot.hackAttachment = currentAttachment
        } else {
            slot.hackRegion = null
            slot.hackAttachment = null
        }
        if (slot.currentSprite && slot.currentSprite.region != region) {
            this.setSpriteRegion(currentAttachment, slot.currentSprite, region)
            slot.currentSprite.region = region
        } else if (slot.currentMesh && slot.currentMesh.region != region) {
            this.setMeshRegion(currentAttachment, slot.currentMesh, region)
        }
        return true
    }
    return false
}

 

[timetocode]

I have a bug where I can create two different spine instances and hacking their attachments is linked. In other words if I change the attachment in spineA the attachment in spineB may change as well. I say "may" because it doesn't always happen. I think it follows the same rules discussed above where during an animation with multiple attachments some methods affect only the active slot whereas others affect the textures associated with a skeleton.

My guess is the bug is one of two things:

1. my code above that changes the attachment texture of the skeleton is using the wrong approach
2.  when two `new Spine(spineData)` are created from the same spine data some of this state is shared instead of cloned into the spine instances (so two different Spine share the same skeleton, not just conceptually but literally in a state-ful manner)

If it is #1 I'm not sure, I guess I'll find another angle. If it is #2 then perhaps adding a function like skeleton.clone() or slot.clone() or treating the spineData immutably on initial construction of the spine object (copying the slots maybe) might fix the bug.

Does anyone have any idea?

I also volunteer to figure this out if someone can explain to me how to build/test pixi-spine . All of my code is using things like `import { Sprite, spine } from 'pixi.js'` but i'm guessing es6 is not the way to do this? To get pixi spine running I have some weird code:
 

import * as PIXI from "pixi.js"
window.PIXI = PIXI
import "pixi-spine"
/*... normal es6 code from here on, though warnings appear about 'spine' not being exported by pixi */

 

Edit:

If I use  yarn build in pixi-spine, and then change my import from `import "pixi-spine"` to the local folder in which i've checked out spine and built it, I end up with an error of pixi not being found presumably from pixi-spine.

 

Edited August 11, 2020 by timetocode

[timetocode]

I've found the specific piece of state that is "shared" between multiple spine instances and thus is resistant to having its texture changed directly. You will probably not be surprised to hear that this shared state is the "Skin" because that's pretty much how it is supposed to work. Interestingly `hackTextureBySlotName` is immune to this problem because it never messed with the skin itself, it just changed the texture of one active slot of a spine instance. Skins are only involved when a spine entity is first created, when it has its skin programmatically changed, or when an animation changes the texture of an attachment.

Meanwhile `hackAttachmentTexture` is essentially a skin hack -- it changes the skin. In order to use the same skeleton+animations over and over again but with different skins which are textured via pixi instead of spine atlases, the association of one skin:skeleton has to be broken. I was able to accomplish this just by cloning the skin object before creating my spine entity -- that way if I change the skin on it, i'm not changing the skin of everything else that uses that skeleton.

I'm not sure if anyone would want this to be honest. I don't think that it is worth a PR or if it is then my recommending is just to add a .clone method to Skin that creates a new skin, attachments, and textures (those are the parts that are currently shared mutably). I mean it's a great feature but it simply isn't how spine works, it's basically replacing the concept of spine skins with a do-whatever-you-want-skins via pixi textures. It's hard for me to imagine any significant number of people learning spine, thinking it's awesome, but then only disliking the way skins are done and trying to hack that. I guess I was lead down that path so maybe someone will want this feature. In any case below is my incredibly hacky code that performs a partial clone of `spineData` (the thing that loads when you load a spine file) and clones all of the textures in the skin so they can be changed programmatically.

// copies clone spineData loaded by pixi
// .. but creates a unique instance of its skin + attachments + regions + textures
// leaves bones, animations, etc all shared and mutable as before
const cloneSpineData = (spineData) => {
    const {
        animations, bones, defaultSkin, events, fps, hash,
        height, ikConstraints, imagesPath, pathConstraints,
        skins, slots, transformConstraints, version, width, x, y } = spineData

    const copy = {
        animations, bones, defaultSkin, events, fps, hash,
        height, ikConstraints, imagesPath, pathConstraints,
        skins, slots, transformConstraints, version, width, x, y
    }

    // ^ this could just be one Object.assign, but I left it unfolded
    // in case anyone wants to clone anything else about spineData (animations? bones?)

    const newSkin = {
        attachments: [],
        bones: [],
        constraunts: [],
        name: 'default'
    }

    Object.setPrototypeOf(newSkin, defaultSkin.__proto__)

    defaultSkin.attachments.forEach(attachment => {
        const newAttachment = {}
        for (let prop in attachment) {
            // Attachment is an object like { 'head': RegionAttachment }
            // RegionAttachment
            const regionAttachment = attachment[prop]

            const newRegionAttachment = {}
            Object.setPrototypeOf(newRegionAttachment, regionAttachment.__proto__)
            Object.assign(newRegionAttachment, regionAttachment)

            // objects of value therein...

            // color
            newRegionAttachment.color = cloneObject(regionAttachment.color)
            newRegionAttachment.tempColor = cloneObject(regionAttachment.tempColor)
            newRegionAttachment.offset = cloneArray(regionAttachment.offset)
            newRegionAttachment.uvs = cloneArray(regionAttachment.uvs)

            // region object
            const newRegion = cloneObject(regionAttachment.region)
            // this is the most important item -- give this skin's pieces a unique texture
            newRegion.texture = regionAttachment.region.texture.clone()
            newRegionAttachment.region = newRegion
            // skipped: region.page (did not seem to be used in rendering)
            newAttachment[prop] = newRegionAttachment
        }
        newSkin.attachments.push(newAttachment)

        const attachmentName = Object.keys(attachment)[0]
        const attachmentObject = attachment[attachmentName]
        Object.setPrototypeOf(newAttachment, attachmentObject.__proto__)
    })


    // and here use our new skin instance instead of a globalish skin
    copy.defaultSkin = newSkin
    copy.skins = [newSkin]

    Object.setPrototypeOf(copy, spineData.__proto__)
    return copy
}

// usage: don't want two Spines to share the same skin when you change it programmatically? do this
const spineData = /* spine data you loaded */

const spineA = new Spine(cloneSpineData(spineData))
const spineB = new Spine(cloneSpineData(spineData))
// now spineA and spineB have the same-looking skin, but if you 
// change any part of their skin it only affects one of them

 

I apologize for the hacks. Objects from the classes Skin, RegionAttachment, Region, etc are created via copying their prototypes (they're not public and I couldn't figure out how to rebuild pixi-spine). The most important class to create a copy of was Texture, and for that pixi's good ol' texture.clone() method was used.

 

 

Here's a gif of a knight with randomly changing weapons+armor pieces, and also skeletons. They all are using the same humanoid.spine skeleton but with different skins that are applied via pixi. Prior to this feature changing parts of the knight would sometimes change parts of the skeletons.

Edited August 11, 2020 by timetocode
added image

[ivan.popelyshev]

cloning spineData? yes, why not. cloning skin? yes.

Make a PR.

Those things are gold because they are workarounds for many problems. I also can talk to esotericsoftware about adding those in other runtimes.

Official API is not enough, I know it for a long time, but I'm not a spine user to actually do those things.