Pixi web app slows down after loading >20MB of images on iPad Mini/2

[Peg Digital]

Hi there, 

 

We've used Pixi.js a couple of times already in commercial projects because its so beautifully fast. Unfortunately one of the projects we're currently working on seems to be bucking the trend and we are struggling to get to the bottom of it. 

 

The app consists of a continuous strip of panels that scroll left to right - we're using DisplayObjectContainers for each panel and then loading in a bunch of images (in layers) and displaying them as sprites within that. Performance was excellent until we started to build more panels where it felt like some sort of memory threshold was crossed on the iPad Mini/2.

 

The issue we have is that we are going to need 100+ panels within our strip - so memory management is of the upmost importance. So far we have 38 and the frame rate has already dropped to an unusable level. We are of course loading in each set of layers only as the user scrolls to that point in the strip and destroying panels they have already moved past but this doesn't seem to release the memory back to the browser fully. We are removing the sprites from the display list, destroying all the PIXI.Textures, removing them form the PIXI.Texture Cache and nulling everything but the frame rate just slows the further in you get. If you skip straight to the end (skipping the need to load all the images on the way) it stays fast.  

 

Here's part of the 'unbuild' function that exists within each panel:

 

        //console.log(PIXI.TextureCache)        for (var pane in this._panes)        {            //console.log(this._panes[pane].art.children.length)            for (var layer in this._paneData[pane].layers)//go through all the layers in this pane            {                PIXI.Texture.removeTextureFromCache(Game.path+'panel-assets/'+this._paneData[pane].layers[layer].art)                //console.log('removing', Game.path+'panel-assets/'+this._paneData[pane].layers[layer].art)            }            for (var bubble in this._paneData[pane].bubbles)//go through all the layers in this pane            {                PIXI.Texture.removeTextureFromCache(Game.path+'bubbles/'+this._paneData[pane].bubbles[bubble].art)                //console.log('removing', Game.path+'bubbles/'+this._paneData[pane].bubbles[bubble].art)            }            this._panes[pane].art.alpha = 0            while(this._panes[pane].art.children.length > 0)            {                this._panes[pane].art.getChildAt(0).texture.destroy()                this._panes[pane].art.removeChild(this._panes[pane].art.getChildAt(0))            }            for (var layer in this._paneData[pane].layers)//go through all the layers in this pane                this._loadedLayers[this._paneData[pane].layers[layer].art] = null            for (var bubble in this._paneData[pane].bubbles)                this._loadedBubbles[this._paneData[pane].bubbles[bubble].art] = null            //console.log(this._paneTimelines[pane].getChildren())            var timelines:any[] = this._paneTimelines[pane].getChildren()            for (var timeline in timelines)            {                timelines[timeline].kill()                timelines[timeline] = null            }        }                timelines = null        this._loadedLayers = {}        this._loadedBubbles = {}        if(this.timelineAcross)        {            this.timelineAcross.clear()            this.timelineAcross.kill()            this.timelineAcross = null        }        if(this.timelineAmbient)        {            this.timelineAmbient.clear()            this.timelineAmbient.kill()            this.timelineAmbient = null        }

 

It will also run beautifully smoothly (60fps) if we generate 100 panels and reuse the same image assets for each one. I realise this may be a larger issue with mobile safari and its memory limitations such as I have read about here:

 

http://stackoverflow.com/questions/10582502/javascript-runs-slowly-in-safari-ipad2-after-loading-200mb-worth-of-new-images

 

and here: http://engineering.linkedin.com/linkedin-ipad-5-techniques-smooth-infinite-scrolling-html5

 

But these articles refer to HMTL5 image objects in the DOM. I was wondering if there is anything else I can do in terms of getting mobile safari to forget image data that has been loaded and viewed via the PIXI ImageLoader or if we are going to have to completely change approach?

 

Any suggestions would be really appreciated!

 

Thanks

[rich]

How are you loading the images in the first place? I suspect it's creating a brand new Image() object for each one of them. That is almost certainly the resource that isn't being cleared, rather than anything specific to Pixi.

 

You could try recycling the Image objects, or killing them - but you've still no control at all over when the browser will actually run gc on those. I've not tested it but I'd be curious to know what would happen if you changed the src of the image. Technically the new one would load in, but I don't know at which point the old will be purged from memory. Got to be worth testing though.

[xerver]

If you are using WebGL, you will also need to destroy your base texture when you are done with it to free the WebGL memory used by the texture.

[Mat Groves]

Hi there! 

 

I have had this kind off issue before - Ipad gets real slow with too many big images in memory.

 

You should try:   this._panes[pane].art.getChildAt(0).texture.destroy(true);

 

Passing true to the texture will make sure to destroy the base texture (which contains the reference to the image) too.

 

Might do the trick? Let me know how it goes for ya!

 

cheers!

[Peg Digital]

Thanks for your responses guys! 

 

I'm actually just using PIXI's built in image loader to load the sprites on the fly at the moment:

 var image:PIXI.Sprite = PIXI.Sprite.fromImage(Game.path+'panel-assets/'+this._paneData[pane].layers[layer].art); //create sprites

I'm glad you all agree that I'm on the right track and that this is to do with images clogging up the memory and resulting in the iPad slowing right down. It sounds like getting the gc to remove the base image data underneath PIXI is the goal. 

 

I've tried passing true into the texture destroy function but I must be doing something wrong - I get this error that seems to come from the PIXI render cycle:

 

1. Uncaught TypeMismatchError: The type of an object was incompatible with the expected type of the parameter associated to the object. pixi.js:13
   1. f.CanvasRenderer.renderDisplayObjectpixi.js:13
   2. f.CanvasRenderer.renderpixi.js:13
   3. Game.tickGame.ts:499
   4. s.dispatchEventTweenMax.min.js:16
   5. f
2.  

  

 

Is there something else I need to do in order to prevent PIXI from trying to render the object who's texture I've destroyed? 

 

Thanks again for your help!

[rich]

Could this be a case of when you're trying to delete it? (i.e. in the middle of an update loop, or during the render loop when it still holds onto the reference as it iterates?)

[Peg Digital]

Yes I'm sure you're right. I have tried removing the sprites from the display list first but the result is the same. I'm going to have a look at the PIXI docs and see how to wait for the right time in the render cycle to destroy textures. 

[Peg Digital]

Still can't seem to find out how to safely destroy textures / how to make sure its done at the right time - can anyone advise on this?

 

Thanks

[Peg Digital]

Instead of destroying them there and then I have tried pushing the texture references to an array which is iterated through inside my game tick. If I call destroy(true) here (after the Game.renderer.render(Game.currentScene); call) I get exactly the same error. I've tried this with the pixi.dev.js build so I can get more info about whats happening. It's coming from the render display object method on the context.drawImage line.

 

PIXI.CanvasRenderer.prototype.renderDisplayObject = function(displayObject){// no loger recurrsive!var transform;var context = this.context;context.globalCompositeOperation = 'source-over';// one the display object hits this. we can break the loop var testObject = displayObject.last._iNext;displayObject = displayObject.first;do {transform = displayObject.worldTransform;if(!displayObject.visible){displayObject = displayObject.last._iNext;continue;}if(!displayObject.renderable){displayObject = displayObject._iNext;continue;}if(displayObject instanceof PIXI.Sprite){var frame = displayObject.texture.frame;if(frame){context.globalAlpha = displayObject.worldAlpha;context.setTransform(transform[0], transform[3], transform[1], transform[4], transform[2], transform[5]);context.drawImage(displayObject.texture.baseTexture.source,   frame.x,  frame.y,  frame.width,  frame.height,  (displayObject.anchor.x) * -frame.width,   (displayObject.anchor.y) * -frame.height,  frame.width,  frame.height);}       }

I'm guessing there is something else that I'm not doing that takes the sprite out of the render cycle so it doesn't try and draw it on the next iteration. I'm removing it from the display list and I've even tried setting it to renderable = false and visible = false but that doesn't seem to make any difference. I feel like we're on the brink of sorting this!  

[rich]

As well as destroying the texture, you're destroying the Sprite too, yes?

[Mat Groves]

Good stuff, You will need to remove the sprite from the stage too (as it will break because the texture is no longer valid)

I will look to add something that checks for this situation as it might be quite useful down the line.

[Peg Digital]

Thanks guys. I've been able to verify this works fine on something simple like example 1 (spinning bunny) within the PIXI repo.

 

Remove the sprite from the display list first, then call destroy on the texture - thats all there is to it. 

 

I'm still having trouble with what I'm doing but as I can verify on the simple demo that this should work, my problems must be related to the surrounding complexity of what I'm doing. At least I know this works in practice and hopefully should allow the app to minimise its memory use and keep running smoothly!

 

Wish me luck - thanks for your help.

[Mat Groves]

Hello!

 

I realised that when destroying a texture it does not get removed from the texture cache. So I tweaked the dev branch so that when destroy is called I remove it from the cache too, as there is no point in having a dead texture kicking around in there

 

Cheers and Good luck sir!!

[Peg Digital]

Thanks thats a handy addition :-)

 

I've still got some issues and I've been able to re-create them with the simple 'example-1' bunny demo. Essentially the problems seems to arise when I try to rebuild a sprite after I have just removed it and destroyed its texture. 

 

Here's the example and the code:

 

<!DOCTYPE HTML><html><head><title>pixi.js example 1</title><style>body {margin: 0;padding: 0;background-color: #000000;}</style><script src="pixi.js"></script></head><body><script>// create an new instance of a pixi stagevar stage = new PIXI.Stage(0x66FF99);var texturevar bunny // create a renderer instancevar renderer = new PIXI.CanvasRenderer(400, 300);// add the renderer view element to the DOMdocument.body.appendChild(renderer.view);this.buildBunny()requestAnimFrame( animate );setTimeout(function(){stage.removeChild(bunny);texture.destroy(true)}, 1000)setTimeout(function(){buildBunny()}, 2000)function buildBunny() {// create a texture from an image paththis.texture = PIXI.Texture.fromImage("bunny.png");// create a new Sprite using the texturethis.bunny = new PIXI.Sprite(texture);// center the sprites anchor pointbunny.anchor.x = 0.5;bunny.anchor.y = 0.5;// move the sprite t the center of the screenbunny.position.x = 200;bunny.position.y = 150;stage.addChild(bunny);}function animate() {   requestAnimFrame( animate );   // just for fun, lets rotate mr rabbit a little   this.bunny.rotation += 0.1;   // render the stage      renderer.render(stage);}</script></body></html>

Any Idea's on why this might be?

 

I really appreciate the help!

[powerfear]

From a quick look at it the first time you call buildBunny directly from the this pointer, the other time it get called from a setTimeout so the context is the window. But there is no need for this at all, bunny, texture and etc are all defined as local variable within the same code block so just remove the few this. you are using and it might work or atleast you'll have this part fixed.

 

EDIT: I tested the code myself using latest pixi build from dev_simple_batch branch containing Mat Groves latest fix for the cache and I noticed it is not working. Seems to be a path problem it is nulling the fullpath while its the shortpath that is in the cache.

 

http://i.imgur.com/CzS04dj.png

 

Also you might need to remove the Texture object from the TextureCache as when recreating a texture using fromImage it will get the old one from the cache that is pointing to a deleted baseTexture

[Peg Digital]

Thanks a lot Powerfear, that's some really interesting info there! I will investigate this stuff today....

[Peg Digital]

I've had another look at this and I seem to have found the issue. The image was still being stored in the BaseTextureCache even though it was being removed from the TextureCache. This meant it wasn't reloading it from scratch on the second time around and was instead still referencing the now nulled texture. 

 

Adding an additional line to the removeTextureFromCache Method seems to have fixed it:

 

PIXI.Texture.removeTextureFromCache = function(id){var texture = PIXI.TextureCache[id]PIXI.TextureCache[id] = null;PIXI.BaseTextureCache[id] = null;return texture;}

 

http://www.pegdigital.co.uk/demoarea/pixi/fixed/

[Mat Groves]

Nice fix sir! 

 

Will make sure to add it to the dev build