Issue with GUI control / attaching to mesh / click through

[thrice]

So, I am having an issue with GUI controls / click through events when attaching to mesh. I'm not sure if this is technically a bug, or if there is a built in way to accomplish what I am trying to do which I am missing, so figured I'd ask here first before going to GH.

Use case is:

I have a card plane. When you hover the card plane, I have an actions plane (with advancedDynamicTexture) which appears above the card plane slightly, which contains multiple action buttons (which do not fill the plane, parts of the card are still visible) - Basically trying to make it so when the actions buttons are visible, you can click the button, without triggering the card planes action manager click actions, but if you click inbetween the actions, you are able to trigger the cards action manager. 

Example is here:

https://www.babylonjs-playground.com/#6KVU7X

Basically, if I set the actions plane to isPickable, can't click the plane behind it. If I set it to false, can't click the buttons.

Is there an easy way to achieve this and or is this a bug?

I am thinking maybe it would make sense to have a property on the dynamic texture itself which would default to the parent mesh, but then could be overwritten, to allow us to have:

mesh.isPickable = false

texture.isPickable = true

and that distinction could allow click through on areas which are empty, while still allowing the buttons to be clicked? (unless there is some easy property setting I am missing or whatever).

[Wingnut]

Hiya thrice!  Interesting observation, and you are correct.  If AdvanceDynamicTexture (ADT) is .CreateForMesh(mesh), and if mesh.isPickable = false... then none of the GUI clicks will work.

I don't think this is a good idea (design-wise), but there's surely many things that I have not taken into consideration. (I don't see "the big picture" real well.) 

Buttons are set ._isPointerBlocker = true... by default.  I think they MIGHT be failing to block, if I understand pointer blocking.

https://www.babylonjs-playground.com/#6KVU7X#1

There's a totally hacked-up mess if there ever was one.    Messy Wingnut.

I eliminated "plane" and kept only bg_plane... and applied the ADT to bg_plane (temporarily, for testing).  bg_plane will likely need to be a non-ADT texture... for project completion.  So "plane" will likely return, and plane.setEnabled(true/false) will likely be used to toggle actions menu.  We could probably rename 'plane' to 'menu_plane', or something similar.

I created a GUI "stackPanel" to replace 'plane' (line 60).  I put a ref to that stackPanel ... into bg_plane.gui (in line 62), just to keep it handy.    ADT now has ONE control... the stackPanel.  The stackPanel has TWO controls... both buttons.

In lines 29-45, which is bg_plane's click handler, I toggle the stackPanel's visibility ON/OFF.  So each time the bg_plane is clicked, its actionManager toggles the visibility of the stackPanel.

No joy!    We can still click "thru" the 2 buttons, and trigger the stackPanel visibility toggle.  The buttons are not blocking the picks from arriving at bg_plane.  hmm.

So, I accomplished nothing.  I think there MIGHT be a bug in button._isPointerBlocker operations.  Not sure.  We need more eyes, experts, and time... to see if things are working as expected.  Thanks for the report, Thrice.  Interesting find.  Stay tuned... more talking and playgrounding on the way, I'm sure.

Extra:  I think there is something wrong with GUI button ENTER and OUT events.  Activate lines 85-92 (highlight entire code block, hit control /) re-RUN, and then move mouse pointer slowly across either button.  Watch console.  It is an ENTER/OUT festival!  Not looking healthy, but ENTER/OUT events are not a useful solve for the main issue.  We can't set bg_plane.isPickable = false, ever, because that disables clicks on the mesh-based ADT (the buttons).

[dbawel]

Without ceating and destroying event listeners, you'll not accomplish your tasks.

I'm not back.....

[GameMonetize]

No you're not

[thrice]

 @Wingnut

Thanks for the reply. I actually don't disagree that in this case, it's a bit of a poor ui decision. Reason I did it that way was mainly simplicity. The better way would probably be to make the menu appear to the right or left of the mesh, however that brings a number of complications, i.e.:

1. I am showing a list of keywords to the left of the card, which leaves me showing it to the right of the card.If the board gets too full, the menu will be stretching beyond the bounds of the window (which I haven't solved yet)

2. I have to now offset the actions plane to be slightly within the parent plane, then when mouse enters the actions plane, store a variable on the actions plane to indicate that mouse is over the actual actions plane (otherwise hovering out of the main card plane would cause the actions plane to close)

With all that said: I still think the behavior is surprising.

[Wingnut]

Hi Thrice.  I think YOU have made good UI decisions, here, actually.  A stackPanel with a list of keywords on the left, a menu on the right... and they would be on their own planes... so you could FLY them into view, or pivot them into position like window shutters.  You have lots of power, here.

But... I think there is an (possible) issue with .isPointerBlocker on both stackPanels and simpleButtons.  Perhaps issue with broken advancedTexture._shouldBlockPointer = true;   Not sure.

I DID put a click-observer on the stackPanel, and clicking buttons does NOT trigger the observable on stackPanel. 

SO, buttons .isPointerBlocker IS working for GUI things. The buttons did not pass their clicks to the stackPanel.  So perhaps... .isPointerBlocker ONLY pertains to Babylon.GUI things... and has NO AFFECT on mesh.isPickable or mesh.actionManager.   Perhaps all is normal with that.

Let's just try a BIG BEG to @Deltakosh, and see if he will visit and comment.

Deltakosh, got a moment? 

Re: This playground - Clicking plane -> actionManager - toggles 2-button stackPanel ON/OFF

Buttons and stackPanel are set .isPointerBlocker = true;   (buttons are default = true.  stackPanel.isPointerBlocker set true in line 63)

* Is it normal for plane actionManager to trigger, when buttons (or area between buttons) is clicked? (considering there are 2 layers if isPointerBlocker=true... atop)

(Thrice, I'll start digging code tomorrow to see if I can discover an answer to this click-thru issue, if nobody else has info, soon.  Sorry it has taken so long to determine if we have a framework bug, here.  I'm on it, but will try to delegate the task, first. I'm just that kind of lazy.)  heh.

[GameMonetize]

Hello!

this is expected as the isPointerBlocker only works for fullscreen UI and not for projected UI.

Projected UI uses the scene onPointerObservable and has no way to block pointer propagation unfortunately.

I'm not sure to be really clear here ;(

 

But TLDR: IT is the expected behavior

[Wingnut]

Alright, there we go.  Deltakosh has rescued us!   We got the "Expected Behavior" stamp from DK, and that means... no bug.  YAY!

And I learned a new term:  "projected UI".  (See why I LOVE hanging around this joint?)    "projection".  yeah!

In DK's fine shaders tutorial, in a section called "The voodoo behind matrices"... there's an illustration.

worldViewProjection = worldMatrix * viewMatrix * projectionMatrix.

hmm.  Seems like there MIGHT BE a correlation, here.  Materials...  textures... projected.  Yep, Babylon.GUI has some unique characteristics... because it is a texture.

ANYWAY... moving right along... (thank goodness)... I think we have this issue... understood.   Agree, Thrice? 

Your keywords-list and pop-open menu could be projected onto separate (possibly invisible) planes, each with their own AdvancedDynamicTexture [ADT].  Set both those planes to .isPickable = false;  Then you can overlap them atop the main "card"... and all clicks on keywords panel or menu panel... won't pass-thru to the card's actionManager.

It is a 3-plane solution... card, keywords, menu.  Three ADT's, too.  (Does the card itself need an ADT?  Maybe not.)  We/You use the two extra planes' .isPickable... as the pointer-blocker system.  Genius.

(Thrice came to this conclusion a couple-of days ago.  Wingnut just-now catching up.) 

[Wingnut]

Oh wait... if we set isPickable to false on the two extra ADT planes... the GUI controls won't click.  Crap.  hmm.  I forgot about that.

I need to build one of these.  I had one almost complete yesterday... and then I went to hit RUN in the playground, and I moused too high and hit a bookmark on my bookmark toolbar.  The playground's "Are you sure you want to leave?"-check... doesn't work in this case, and you'll lose the unsaved playground.  That's what I did... and then cried a little bit.

On another PG project, I missed the F12 button and instead, accidentally hit BACKspace button.  This will load the previously-used playground with no concern about having un-saved changes, too.  I cried on that one, too.

Losing unsaved changes in the PG app... hurts.  I wish we could somehow prevent those two "things" from happening.  Ok, party on.  I'm going to code this two-menu-planes-overlapping-the-card test... in a little while (after some coffee).  I'll be saving often. 

[Wingnut]

https://www.babylonjs-playground.com/#6KVU7X#5

Not complete yet.  Action manager on center cardPlane is not toggling .isVisible on the two stack panels (lines 43-65).

It's ok... because 43-65 probably needs changing.  No need to toggle gui stackpanel .isVisible... because we should probably be toggling keysPlane and menuPlane .setEnabled(true/false).  Then the menuPlane and keysPlane will appear/disappear along with the gui they contain.  Wiser.  Next version.

Good thing seen on console.  The parts of the green buttons... that are overlapping cardPlane, are NOT passing their clicks "thru" to the cardPlane's actionManager.

I think this is on-objective.  By putting these two stackPanels on their own planes (keysPlane and menuPlane), we don't NEED to set their .isPickable to false.  The clicks on the buttons (portions that overlap cardPlane) are not clicking-thru to cardPlane.actionManager.  Yay!  So far, so good, I guess. 

I need to remove the verticalAlignment crap from createButton, or do something different.  I want each added button to stack-from-center, like DK's stackPanel demo from the Babylon.GUI docs... stackPanel section.  In that demo, no verticalAlignment is set... so the added controls center-stack automatically.  More versions coming soon.  Anyone is free to jump-in and do tests, and make more PG saves. 

Update: [link]  Click central cardPlane to toggle both stackPanel .isVisible... working well now.  But menuPlane and keysPlane remain visible, so I will be changing to mesh.setEnabled method... next.

Update2: [link]  Click cardPlane now sets keysPlane and menuPlane disabled.  Works great.  But clicking cardPlane AGAIN.... should re-enable the two side-planes... and it doesn't, so far.  hmm.  SimpleButtons seem to have altered their verticalAlignment, too.  Not sure why... I didn't work-on that part yet. 

Update3: [link]  There we go.  I needed ._isEnabled  (in lines 50 and 58).  I forgot the underscore.    Next-up, add 20 keywords and menu buttons, and make overflow-handling and scrolling-arrows for each side-plane, and and and... *sigh* 

[thrice]

Nice! I think I need to familiarize myself with stack panel,, thought it was only part of the full screen api - that could simplify some of my code I think. however: the original objective I was trying to accomplish remains - Basically, in your example, it would be the equivalent of being able to click through to the plane, in the small portion of the stack panels that overlaps the plane behind it, that does not have a stack panel button in that space.

But ya, I have it working the slightly more complicated way so not the end of the world, and design wise probably not the best UI doing it that way, but would have worked in this instance just because the buttons themselves aren't very tall so it's easy to click the plane behind it while hovered.

P.S. Overflow handling/scrollbar would be amazing btw! Just throwing that out there @Deltakosh  

[Wingnut]

Cool.  Hey, have you followed the last 4-5 posts in The Wingnut Chronicles?  I was trying text scrolling, via my normal turn-every-knob-on-textBlock-object-until-something-works methodology, and I ended-up using a total kludge to simulate Babylon.GUI vertical text scrolling  (related to overflow/scrollbar container, right?)

Anyway, suddenly genius @adam said THIS, which @Deltakosh agreed-with.

So... ahem... a guy like you could sneak right in there, and beat them to the punch.  Not only could you create a scrolling container control before Adam or Deltakosh even got an IDE fired-up, but you could scribble-out a little tutorial on "Making Custom Controls for Babylon GUI"  (yay!)  Maybe a couple nice playgrounds to go with that.

Part of the "Thrice Krispees" series of "Customizing Babylon GUI" documentation/demos.  I'd help you get it all started... if you wanted.  Maybe another tutorial... Canvas Commandos - Stroking and Filling (ick, that sounds dirty, somehow)    Context2d canvas painting expertise... is a VERY IMPORTANT part of creating custom controls.  ANIMATING context2d paintings... is a miserable bitch... often needing "blit" animation... like animating a sprite.  (Does the world still use the term 'blit'?  Maybe not, eh?)    So, the "Canvas Commandos" is a prestigious and honorable group... sprite-ish animators... who can wipe and redraw the needle on that old-style volt meter... quickly and nicely, over and over again.  Almost all Babylon.GUI power-animating... will be done by paint, overpaint, paint, overpaint, over and over.  It is a fairly exhaustive task, and only the most patient of artists... will be able to take the heat.  heh.

Think about a dial, like a volume knob on a guitar amplifier. Its outer ring might be numbered 1 thru 10, and to turn that knob from 1-10 smoothly... we will need some SERIOUS paint/wipe/paint/wipe/paint/wipe/paint, etc etc.  We might be looking at 100 frames of sprite/blit... just to turn that dial smoothly from 1-10.  Erf.

But anyway, ignoring all THAT drama... I think there is an avenue of opportunity available... to open-up "custom controls" much wider... to us BJS experimenters.  It almost takes a "jig"... a custom control starter kit... part of Babylon.Tools, perhaps.  Stroke assistant and fill assistant funcs.  "Thrice shows us what we can do... with a blank GUI control" 

Thoughts?  I think Deltakosh could use a little assistance... in fleshing-out Babylon GUI, so if we could build a team of Babylon GUI experts/experimenters... then THAT would kind-of ROCK, I think.  I think a crap-load of people would love to AT LEAST see some playgrounds of user-made custom controls, containers, stacks, dials, meters, radial/linear fillers, whatever we can imagine-up.  Someone would definitely get a "woot" from Deltakosh... if they were to successfully spearhead that team/project.  Ready to get started?  GUI PARTY!!!