Fix/chorded button event ordering

[sudomakeinstall]

Context

In native VTK, mouse button events are registered independently of each other; i.e., if you press the left button, then press the right, then release the left, then release the right, you would get:

• LeftButtonPress
• RightButtonPress
• LeftButtonRelease
• RightButtonRelease

However, currently in vtk.js, if you went through the same sequence, you would only register the events which occur when no other button is pressed:

• LeftButtonPress
• RightButtonRelease

This is undesirable because:

• It makes it difficult to capture multi-button events (such as right-left click together).
• It may leave the interactor in an unexpected mode (for example, in the above sequence, it would be left in the "Left Down" mode since the LeftButtonRelease was never registered). Although this PR provides an example which shows the detected mouse events, note that you can currently demonstrate this behavior in any example with LeftDown, RightDown, LeftUp, RightUp, which will leave you with no buttons depressed but the object still rotating as you move the mouse.
• It is a change from the native VTK behavior.

The underlying reason for this seems to be the web specification for Chorded Button Interactions:

Some pointer devices, such as mouse or pen, support multiple buttons. In the [UIEVENTS] Mouse Event model, each button press produces a mousedown and mouseup event. To better abstract this hardware difference and simplify cross-device input authoring, Pointer Events do not fire overlapping pointerdown and pointerup events for chorded button presses (depressing an additional button while another button on the pointer device is already depressed).

Instead, chorded button presses can be detected by inspecting changes to the button and buttons properties. The button and buttons properties are inherited from the MouseEvent interface, but with a change in semantics and values, as outlined in the following sections.

I believe that this is the underlying cause of an issue I previously opened in trame (here: Kitware/trame#803) before I determined the source.

Results

This PR adds new logic to RenderWindowInteractor which inspects event.buttons to detect and correctly fire chorded events. As far as I can tell, the behavior now matches native VTK.

Changes

• New example: Examples/Rendering/MouseEvents/index.js

• New test: Sources/Rendering/Core/RenderWindowInteractor/test/testChordedButtons.js

• Modified: Sources/Rendering/Core/RenderWindowInteractor/index.js

• Documentation and TypeScript definitions were not updated, since in my mind this fix brings behavior in line with expectation, though happy to add additional information to the docs if requested.

PR and Code Checklist

• semantic-release commit messages
• Run npm run reformat to have correctly formatted code

Testing

• This change adds a unit test
• Tested environment:
  • vtk.js version 34.16.3 (latest master)
  • OSx version 15.7.3
  • **FireFox version 147.0.4 **

[jourdain]

@finetjul the code looks reasonable but it would be great if you could test it. Thx

[finetjul]

It works fine ! It seems to also coexist well with touch/gesture. (did not thoroughly tested though).

I think you could factorize by having a unique function called by handlePointerUp, handlePointerCancel and handlePointerMove
e.g.

handleChordedButtons(previous, current){
// Process Release events before Press events ????
if (previous ^ current & previous & 1) {
  leftButtonReleaseEvent();
}
if (previous ^ current & previous & 2) {
  rightButtonReleaseEvent();
}
...
if (previous ^ current & current & 1) {
  leftButtonPressEvent();
}
...

If handleMouseUp() must really be kept in handlePointerUp(), you could do:

handlePointerUp () {
...

const ignoreButton = event.button == 0 ? ~1 : event.button == 1 ? ~4 : ~2;
handleChordedButtons(previousMouseButtons & ignoreButton, event.buttons & ignoreButton);
handleMouseUp();
previousMouseButtons = event.buttons;
}

[sudomakeinstall]

@finetjul Thanks so much for the review and feedback! I've refactored into a helper function and tried to simplify the logic as you suggested--on my machine the test I added is passing and the example still seems to work as I expect it to. CI is running now. Let me know what you think!

[sudomakeinstall]

@finetjul Done--new commits pushed! Thanks again for the reviews/feedback.

[finetjul]

LGTM

[github-actions]

🎉 This PR is included in version 35.3.1 🎉

The release is available on:

• npm package (@latest dist-tag)
• npm package (@latest dist-tag)
• GitHub release

Your semantic-release bot 📦🚀