Jump to content

Simulating mouse pointers in VR


Recommended Posts

Hi there, I'm in the middle of creating a website that works with multiple control inputs including a virtual cursor for VR using VRDeviceOrientationFreeCamera. The method I'm currently using is a bit of a hack i think.
I've created a long cylinder and attached it to the camera which i use to check for collisions and then fire the same functions I have in OnPointerOverTrigger, OnPointerOutTrigger .etc... 

Is this the best solution or is there a way to simulate a mouse pointer that is attached to a camera?


Link to comment
Share on other sites

My pleasure.  I'd like to pass a "thanks" to the original coder of that playground, whomever he/she be.  It works great.  BJS picking rays work great.

Rays are pretty good at getting distances.  I wonder if they could be used as a 3D scanner of mesh?  We have that skull model thing...


(allow some time for skull model to load)

I wonder... if "sweeping" a pickingRay across the surface of the skull... is possible.  As we sweep, we gather distance data into a buffer, and then use that buffer as a heightMap or displaceMap.  Weird.

Sorry, I wandered-off, mentally, for a moment.  :)

Link to comment
Share on other sites

Hi again!  That's a great question/topic!


Here, even though I have no texture on the "wall" (the target), line 33 is reporting pickingInfo.textureCoordinates to console.  The numbers are the fractional distance between 0, 0 in the lower left corner... and 1,1 in the upper right corner (of the no-texture texture on the wall plane) (huh?).  :)

Programmers can easily convert those numbers to "percentage of up-ness" and "percentage of right-ness".

I didn't even go that far.  I just sent the raw coordinates to the console. (I'm lazy)  :) 

Your question is so good, it should probably be a new forum topic.  Possibly titled...  'Eyedropper - Get color under picked point'. 

If you DO start this new topic, possibly include 'eyedropper' and 'colorUnderPointer' as tags, if you please.  I searched forum and playgrounds... finding no information about this.  It is a worthy subject to discuss further, probably in a separate thread.

POSSIBLY... the way to do this... is create a renderTargetTexture (rtt) from the current camera view.  RTT's are like a camera view... that can be used as a texture.  For example, below is a playground with 4 RTT's from 4 different cameras.... textured onto 4 planes... that are parented to the camera (so they stay in the same place).


I know only one way to get RGB data from an image/texture.  Paint/Put the image (our RTT) into a context2D canvas, and then use the context2D's pixel manipulation features.  The getImageData function allows us to retrieve imageData.data from a (tiny) rectangular area.  We would want to position that tiny rectangular area... a certain distance upward and right-ward... into the canvas.   Hey, we HAVE up-ness (Y-amount) and right-ness (X-amount) values... from our pickingInfo.textureCoordinates, yes?  I see hope!  :)

Note: When I mention "canvas" above and below, I do NOT mean the renderCanvas for the entire scene.  I mean a separate HTML canvas where we "put" a renderTargetTexture (RTT) gotten from our primary camera.

If we CAN get imageData.data from the tiny rectangle... which is position X/Y into the context2D canvas... then perhaps we will be farting through silk (pardon my Klingon dialect).  We could be successful with our "eyedropper" color getter.  According to some document somewhere... imageData.data contains "a Uint8ClampedArray representing a one-dimensional array containing color data in RGBA order, with integer values between 0 and 255 (inclusive)." 

Sounds like something usable, eh?  What a complicated procedure, though.  erf.  Let's wait for more comments, and then perhaps start a new topic about this... a bit later.  Does that sound like a good plan?

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.


  • Recently Browsing   0 members

    • No registered users viewing this page.
  • Create New...