MouseLock & FullScreen

Just before the mouselock patch was submitted to review, some new changes landed on the mozFullScreen implementation, and that affected some of the mouselock behaviour, since a great part of its functionality relies on the fullscreen code. However, David Humphrey was able to make the necessary changes to adapt the new fullScreen implementation to handle mouselock, and the patch was successfully submited.

After the patch was submitted, I was curious to see what were the changes to the fullscreen implementation, plus we needed some new mochitests to test this new patch, so I went to mxr and started inspecting some code.

From what I understood, the functionality added to fullscreen was to cascade fullscreen between nested elements, for example:

<div id="parent">
<div id="child"></div>
</div>

So if parent gets fullscreen, and then child gets fullscreen, parent would still be in fullscreen mode, and if child loses fullscreen, parent would regain it.

The Problem

What happened was that if the parent element had the mouse locked, and the child received fullscreen, the mozfullscreenchange event was never dispatched, since the parent didn’t actually lose fullscreen, so it would still have the mouse locked to it, even though the child would be in fullscreen mode.

Solution

When nsDocument::RequestFullScreen is called, it checks if there is already an element in fullscreen mode, if it does, it unlocks the mouse before setting the new element to fullscreen.
code

Bellow, I created a simple class diagram with some of the classes/methods that handle fullscreen requests.

Here are some printf’s that helped me understand the flow of the fullscreen.
The first two stacks are from a parent and a child element receiving fullscreen
The last two are from exiting fullscreen by calling document.mozCancelFullScreen and pressing ESC

Fullscreen Parent Element

nsDocument::AsyncRequest
nsCallRequestFullScreen::nsCallRequestFullScreen
nsCallRequestFullScreen::Run
nsDocument::RequestFullScreen
sFullScreenDoc? FALSE
nsDocument::FullScreenStackPush
calling nsEventStateManager::SetFullScreenState(aElement, true)
returning from nsDocument::FullScreenStackPush
while ((parent = child->GetParentDocument()))
nsDocument::FullScreenStackPush
calling nsEventStateManager::SetFullScreenState(aElement, true)
returning from nsDocument::FullScreenStackPush
if (static_cast(parent)->FullScreenStackPush(element))
static DispatchFullScreenChange
static DispatchFullScreenChange
static SetWIndowFullScreen
nsSetWindowFullScreen::nsSetWindowFullScreen
nsSetWindowFullScreen::Run
nsGlobalWindow::SetFullScreen
setting fullscreen to: TRUE
nsGlobalWindow::SetFullScreen
setting fullscreen to: TRUE

Fullscreen Child Element

nsDocument::AsyncRequest
nsCallRequestFullScreen::nsCallRequestFullScreen
nsCallRequestFullScreen::Run
nsDocument::RequestFullScreen
sFullScreenDoc? TRUE
nsDocument::FullScreenStackPush
top exists, calling nsEventStateManager::SetFullScreenState(top, false)
calling nsEventStateManager::SetFullScreenState(aElement, true)
returning from nsDocument::FullScreenStackPush
static DispatchFullScreenChange
static SetWindowFullScreen
nsSetWindowFullScreen::nsSetWindowFullScreen
nsSetWindowFullScreen::Run
nsGlobalWindow::SetFullScreen
setting fullscreen to: TRUE
nsGlobalWindow::SetFullScreen
setting fullscreen to: TRUE

Exit FullScreen By Calling document.mozCancelFullScreen()

nsDocument::MozCancelFullScreen
nsDocument::RestorePreviousFullScreenState
nsDocument::MaybeUnlockMouse
nsDOMMouseLockable::Unlock
static DispatchFullScreenChange
nsDocument::MaybeUnlockMouse
nsDOMMouseLockable::Unlock
static DispatchFullScreenChange
static SetWindowFullScreen
nsSetWindowFullScreen::nsSetWindowFullScreen
nsSetWindowFullScreen::Run
nsGlobalWindow::SetFullScreen
setting fullscreen to: FALSE
nsGlobalWindow::SetFullScreen
setting fullscreen to: FALSE
nsIDocument::ExitFullScreen
nsDocument::ExitFullScreen

Exit FullScreen By Pressing ESC

nsIDocument::ExitFullScreen
nsCallExitFullScreen::Run
nsDocument::ExitFullScreen
nsDocument::MaybeUnlockMouse
nsDOMMouseLockable::Unlock
static ResetFullScreen
nsDocument::ClearFullScreenStack
static ResetFullScreen
static ResetFullScreen
nsDocument::ClearFullScreenStack
static ResetFullScreen
static ResetFullScreen
static ResetFullScreen
static DispatchFullScreenChange
static DispatchFullScreenChange
static static SetWindowFullScreen
nsSetWIndowFullScreen::nsSetWindowFullScreen
nsSetWindowFullScreen::Run
nsGlobalWindow::SetFullScreen
setting fullscreen to: FALSE
nsIDocument::ExitFullScreen
nsDocument::ExitFullScreen

Advertisements

MouseLock – DOM Validation

I’ve been trying to implement the validation on MouseLock to check if the element being locked belonged to the “right” DOM.
What happens is that if the parent document tries to lock an element that belongs to a different document, for example: the document of an iframe, the mouse shouldn’t lock. So it’s only possible to lock an element if he belongs to the same DOM as the one the MouseLockable object has access to.

The Fix

 nsCOMPtr<nsIDOMDocument> targetDoc;
  parentNode->GetOwnerDocument(getter_AddRefs(targetDoc));
  if (targetDoc != domDoc) {
    return false;
  }

Code

While testing, I came across some interesting data:

All the objects from the parent window have a different memory address from the ones on the iframe.
I always knew that the iframe had a different scope from the parent page, but I never realized that having a different scope literally meant creating all the objects needed in a page, pretty much loading a page inside a page.
It’s amazing how many things we take for granted when using a browser, when in fact, a simple mousemove generates a call stack of more than 100 calls.

**I would assume that the iDoc would have the same memory address as the iWindow.document, but apparently they don’t


MouseLock on Dual Monitor – Ubuntu

Usually, I run firefox on my laptop and leave the second monitor just for coding. I’ve been doing that for the past three weeks. However, I don’t know why, I decided to run firefox on my second monitor. This is what happened:

Video

If I lock the mouse when firefox is on my second monitor, the cursor gets repositioned back to my first monitor, and when I try to leave the boundaries of the first monitor, it repositions the cursor back to the middle of the screen. That’s exactly how MouseLock should work. However, it should be doing it on the second monitor, where firefox is running, not on the first.

Brainstorming with Raymond (rhung) on #seneca, we found that the bug was only happening on Linux, since he wasn’t able to reproduce it on windows.
Looking further, we started inspecting the screenX/Y and clientX/Y coords for both monitors

On Linux:

On the first monitor:
screenX: 641 screenY: 782
clientX: 641 clientY: 160

On the second:
screenX: 2006 screenY: 524
clientX: 640 clientY: 158
It looks like the screenX/Y are not relative to the monitor size, but to the total size of the display. On the other hand, clientX/Y are relative to each specific monitor.

My guess is that when the mouse is being locked, it’s using the clientX/Y coords to set the screenX/Y coords. So instead of the cursor being reseted to the middle of the screen of the second monitor it’s being reseted to the middle of the screen of the first monitor, since screenX:641 and screenY:158 are in the boundaries of the first monitor