MouseLock – DOM Tree & mozFullScreen

For the moment, MouseLock only works if the element being locked is in fullscreen mode. However, the w3c spec says that any element, in fullscreen or not could have the mouse locked into.
It was decided on  the Bug 6333602 discussion, that the first implementation of MouseLock would only work in fullscreen mode, but probably future versions will support non fullscreen mode as weel. With that in mind, I went back revising all the features that MouseLock was relying on the mozFullScreen API.

One of those, was the unlocking of the mouse.
If the locked element is removed from the DOM Tree, because the mouse can only be locked if the element is in fullscreen mode, and when the element is removed from the DOM Tree it automatically loses fullscreen, that would trigger the unlocking of the mouse.
So in the future, if an element that is not in fullscreen gets locked, and then removed from the DOM Tree, the mouse wouldn’t unlock it.

The Fix:

nsCOMPtr node = do_QueryInterface(aOldChild);
  if (node) {
    nsCOMPtr domDoc;
    node->GetOwnerDocument(getter_AddRefs(domDoc));
    if (domDoc) {
      nsCOMPtr lockedElement;
      domDoc->GetMozFullScreenElement(getter_AddRefs(lockedElement));

      nsCOMPtr lockedNode = do_QueryInterface(lockedElement);
      // If the element being removed is the same element with the mouse locked
      // Then unlock the element
      if (node == lockedNode) {
        // Get Window
        nsCOMPtr window;
        domDoc->GetDefaultView(getter_AddRefs(window));
        //Get Navigator
        nsCOMPtr navigator;
        window->GetNavigator(getter_AddRefs(navigator));
        if (navigator) {
          // Get Pointer
          nsCOMPtr pointer;
          navigator->GetPointer(getter_AddRefs(pointer));
          if (pointer) {
            // Unlock the mouse
            pointer->Unlock();
          }
        }
      }
    }
  }

So now, if the locked element is removed from the DOM Tree, it will be unlocked in the nsINode::RemoveChild method not in the nsDocument::CancelFullScreen.
Not relying on the mozFullScreen API anymore

The second feature that MouseLock was relying on mozFullScreen, was that only elements in the DOM Tree could have the mouse locked. However, the element can only be locked if in fullscreen mode, and to get fullscreen mode, the element must be in the DOM Tree. So MouseLock wasn’t validating that, because mozFullScreen was filtering.

The Fix:

  nsCOMPtr targetNode(do_QueryInterface(aTarget));
  nsCOMPtr parentNode;
  targetNode->GetParentNode(getter_AddRefs(parentNode));
  if (!parentNode) {
    return false;
  }

We were also considering validating if the element being locked was a DOM element. However, it seems that only a nsIDOMElement can be locked, and if anything else gets passed as an argument an exception is thrown.

p.lock()
p.lock(true)
p.lock(“text”)
p.lock(12345)
p.lock(canvas, 1, 2)
p.lock(document)
p.lock(window)

[02:03:22.566] [Exception… “Could not convert JavaScript argument arg 0 [nsIDOMMouseLockable.lock]” nsresult: “0x80570009 (NS_ERROR_XPC_BAD_CONVERT_JS)” location: “JS frame :: Web Console :: :: line 1” data: no]

So I guess we get for free the validation to check if the element being locked is a DOM element, since when trying to lock anything but a DOM element the browser throws an exception

I saved a few printf’s showing the flow of MouseLock when an element gets locked

Locking an element that is not in the DOM Tree

nsDOMMouseLockable::Lock
nsMouseLockableRequest::nsMouseLockableRequest
Calling ShouldLock..
nsDOMMouseLockable::ShouldLock
Element is not in the DOM Tree, returning false….
Dispatching failure callback to main thread
nsRequestMouseLockEvent::nsRequestMouseLockEvent
Failure request dispatched
Returning from Lock
RequestAllowMouseLockEvent::Run
nsMouseLockableRequest::SendFailure

Locking an element that is not in fullscreen mode

nsDOMMouseLockable::Lock
nsMouseLockableRequest::nsMouseLockableRequest
Calling ShouldLock..
nsDOMMouseLockable::ShouldLock
Element is in the DOM Tree
Element is not in fullscreen mode, returning false…
Dispatching failure callback to main thread
nsRequestMouseLockEvent::nsRequestMouseLockEvent
Failure request dispatched
Returning from lock

Locking an element that it’s in the DOM tree and in fullscren mode

nsDOMMouseLockable::Lock
nsMouseLockableRequest::nsMouseLockableRequest
Calling ShouldLock..
nsDOMMouseLockable::ShouldLock
Element is in the DOM Tree
Element is in fullscreen mode
returning true…
ShouldLock returned true
Dispatching success callback to main thread
nsRequestMouseLockEvent::nsRequestMouseLockEvent
Success request dispatched
Returning from lock
RequestAllowMouseLockEvent::Run
nsMouseLockableRequest::SendSuccess
 

I pushed the changes here

Advertisements

Inspecting nsGeolocation callbacks

Before implementing callbacks into MouseLock, I followed David Humphrey’s suggestion and decided to take a closer look at how the nsGeolocation was implementing callbacks.
The nsGeolocation class has a lot more functionality than the MouseLock, so at the beginning it was hard to pin point the exact place where the callbacks were handled.
After a few hours searching on mxr and a bunch of printf’s , I was able to find how the callbacks were being stored and fired

I wrote down all the steps involved in firing a callback

nsGeolocation::GetCurrentPosition

  • Ensures that a callback is passed, the failure callback is optional, but the successful is mandatory
  • Checks if geolocation is enabled by the user
  • When the nsGeolocation is instantiated, it creates a nsGeolocationService object, the geolocationService object calls the static function GeoEnabledChangedCallback, that sets the static bool variables sGeoEnabled(by default its set to true)
  • The geolocation preference is called “geo.enabled” and can be set via the browser on the about:config page.
  • Check if the pending callbacks are bigger than the max number of geo requests per window(1500)

nsGeolocationRequest::nsGeolocationRequest()

  •  Creates a nsGeolocationRequest object
  •  Checks if the request object was successfully created

nsGeolocationRequest::Init()

  • Checks if the request object was successfully initialized
  • Not sure why this check happens, since the Init method for the nsGeolocationRequest returns NS_OK

nsGeolocation::RegisterRequestWithPrompt

  • Calls nsGeolocation::RegisterRequestWithPrompt and check if the action was successful
  • Checks if the geo.prompt.testing preference is enabled( I couldn’t access this preference in the about:config page)
  • Preferences::GetBool
  •  Check if the XRE_GetProcessType() equals to GeckProcessType_Content
  • The XRE_GetProcessType returns a mozilla::startup::sChildProcessType, that it is a GeckProcessType
  • GeckoProcessType_Content it’s an enum.
  • I didn’t go any further on this topic, so I’m not sure what is exactly doing.
  • If the XRE_GetProcessType() is equal to GeckoProcessTypeContent, it then get a reference to a nsPIDOMWindow, by querying mOwener(a reference to the current inner window)
  • Gets a TabChild from the nsPIDOMWindow
  • It calls the AddRef method of the nsGeolocationRequest object, here is the comment in the source code: “Retain a reference so the object isn’t deleted without IPDL’s knowledge.
  • Corresponding release occurs in DeallocPContentPermissionRequest.” When implementing the callbacks for MouseLock I got stuck in this part, I’ll comeback later and inspect to see what really happends when the AddRef mehtod is called
  •  The TabChild calls the SendPContentPermissionRequestConstructor
  • After doing a search on mxr for SendPCOntentPermissionRequestConstructor, the only other file that used this method was the nsDesktopNotification.cpp. Looking at the code in the DesktioNotification file I saw that was very similar to the one used in the Geolocation, probably because was the same person who wrote it Doug Turner(dougt). Anyway, the DesktopNotification class seems like a good class to dive into next.
  • Back to the SendPContentPermissionRequestConstructor, the closest method I could find on mxr was SendPContentDialogConcstructor. Digging a little bit more I found the nsIContentPermissionPrompt.idl.
  •  The nsGeolocationRequest extends the nsIContentPermissionRequest, that’s where the nsIContentPermissionRequest is defined.
  • My guess is that SendPContentPermissionRequestConstructor allows something make requests from a browser tab, in this case, probably requesting the geolocation status.
  •  Calls Sendpromt on the nsGeolocationRequest object

RequestPromptEvent::RequestPromptEvent

  • Creates a new RequestPromptEvent object

nsGeolocation::RegisterRequestWithPrompt – Dispatch to Main Thread

  •  Dispatches the object to the main thread
  • This is the furthest I go into inspecting threads

RequestPromptEvent::Run()

  • RequestPromptEvent::Run method is called
  • It creates a nsIContentPermissionPrompt object by getting the service from NS_CONTENT_PERMISSION_PROMPT_CONTRACTID
  • Some comments for the interface: “Interface provides a way for the application to handle the UI prompts associated with geo position.”
  • If we implement prompts on MouseLock, we may need to create an interface as well, or maybe just reuse this one?
  • If the nsIContentPermissionPrompt was successfully created, it then calls the method Prompt on the object, passing the nsGeolocationRequest as an argument.
  • I searched on mxr but couldn’t find the Prompt method
  • **I don’t know why or where the following methods are called, because I couldn’t find the definition for the Prompt method, but by examining the stack that’s the chain of method calls that happens

nsGeolocationRequest::GetType()

  • nsGeolocationRequest::GetType gets called. It returns the string “geolocation”

nsGeolocationRequest::GetUri()

  • nsGeolocationRequest::GetUri gets called.
  • It then calls the GetURI method of the nsGeolocation, since the nsGeolocationRequest holds a reference to the nsGeolocation object.
  • The GetURI method of the nsGeolocation returns the mURI.
  • mURI is of type nsIURI.
  • To get mURI on the Init method, it gets from the doc->NodePrincipal->GetURI
  • After it has a nsIURI object, it calls the forget method passing the aRequestingURI as a parameter.
  • Here is the comments for the nsHtml5RefPtr::forget method:
  • “Set the target of rhs to the value of mRawPtr and null out mRawPtr. Useful to avoid unnecessary AddRef/Release pairs with “out” parameters where rhs bay be a T** or an I** where I is a base class of T.”
  • Not sure what is happening there

nsGeolocationRequest::GetWindow()

  •  nsGeolocationRequest::GetWindow gets called.
  • Follow the same steps as the GetUri method.
  •  Waits for user response.

nsGeolocationRequest::Allow()

  •  After the user clicks OK on the prompt window, and shares the geolocation, the nsGeolocationRequest::Allow method gets called.
  • I don’t know how the Allow gets called, since the only places in the nsGeolocation.cpp code that call Allow are: nsGeolocationRequest::Recv__delete__, nsGeolocationRequest::WatchPosition and RequestAllowEvent::Run. I’ll keep debugging to see if I can find the stack trace for the Allow call, but I suspect it has something to do with the Prompt called in the RequestPromptEvent::Run.
  • Anyway, the Allow gets called and this is what happens:

nsGeolocationService::GetInstance

  •  Calls thensGeolocationService::GetIncstance method

nsGeolocationService::StartDevice

  • Starts the GeolocationService device
  • Checks if the geolocation is enabled

nsGeolocationService::SetDisconnectTimer

    • Calls SetDisconnectTimer
    •  If a nsTimer doesn’t exist, creates one by do_CreateInstance(“@mozilla.org/timer;1”)
    • Looks like do_CreateInstance is another way to instantiate objects
    •  The timer is initialized passing as arguments, ‘this’ referring to the nsGeolocationService object, the delay for the timer, with a flag of TYPE_ONE_SHOT, that means it fires only once
    • Checks if the XRE_GetProcessType() equals to GeckProcessType_Content.
    • If so, creates a ContentChild object and calls the SendAddGeolocationListener method
    • **DISALLOW_EVIL_CONSTRUCTORS
    •  Creates a nsIObserverService object. Still not clear what a nsIObserver does.
    • It loops through the number of nsIGeolocationProvider the nsGeolocationService has it stored.
    • It then starts the provider.
    • Set the provider to watch the nsGeolocationService object
    • Then start observing the provider.
    • Some comments from the nsIGeolocationProvider.idl:
    • “Interface provides location information to the nsGeolocator via the nsIDOMGeolocationCallback interface. After startup is called, any geo location change should call callback.update().”
    • I guess that’s why the Update method gets called later on.
    • Still lots of different components, but things are starting to make sense

nsGeolocationRequest::Allow()

    •  Checks if the geoService was successfully started.
    • If not it Calls the NotifyError method passing as an argument nsIDOMGEOPositionError::POSITIONUNAVAILABLE
    • nsIDOMGeoPositionError seems to be an idl with the error codes for the Geolocation class
    • Notifying an error is how the failure callback gets called
    • First thing that happens on the nsGeolocationRequest::NotifyError method is the creating of a nsDOMGeoPositionError object.
    • It checks if the object was successfully created

nsGeolocationRequest::NotifyError

    • Calls the nsDOMGeoPositionError::NotifyCallback method passing the callback provided by the user as an argument

nsDOMGeoPositionError::NotifyCallback

    •  On the nsDOMGeoPositionError:NotifyCallback. First thing it checks to see if the callback exists, because the failure callback is optional, sometimes it might be null. if it doesn’t exist, it returns.
    •  Creates a nsIJSContextStack by passing do_GetService(“@mozilla.org/js/xpc/ContextStack;1”) to its constructor.
    •  Checks if the stack was successfully created, if not, returns
    • Calls the HandleEvent method of the error callback: nsIDOMGeoPositionCallback
    • Removes the JSContext stack
    • (Not sure what goes on here)

nsGeolocationRequest::Allow()

    •  Creates a nsIDOMGeoPosition object by calling the nsGeolocationService::GetCachedPosition
    • GetCachedPosition returns the last GeoPosition stored in the GeolocationService
    •  Gets the cached position time by calling the GetTimeStamp.
    •  Checks if the cached value can be used by calling the GetMaximumAge of the nsIDOMGeoPositionOptions
    • More checking if the cached position can be returned
    • Then for some reason, the nsGeolocation::Update method gets called
    •  It checks if the window still exists. If not, it calls the nsGeolocationRequest::Shutdown.
    • The Shutdown method cancels the nsITimer and sets the callbacks to null, and clear flag to true
    • Loops through all pending callbacks(a nsGeolocationRequest array) and call Update() on all of them

nsGeolocationRequest::Update()

  • Checks if the callback should be allowed

RequestSendLocationEvent::RequestSendLocationEvent()

  • Creates a RequestSendLocationEvent object.

nsGeolocationRequest::Update – Dispatch to Main Thread

  •  Dispatches the object to the main thred

RequestSendLocationEvent::Run()

  •  Calls the RequestSendLocationEvent::Run.

nsGeolocationRequest::SendLocation()

  •  Calls the nsGeolocationRequest::SendLocation method
  •  Checks if the request was cleared, or if its not allowed
  • Checks if the timer exists If so, cancel it.
  •  Checks if the nsIDOMGeoPostion is not null, since it can’t send a null position ins a successful callback.
  • If the position is null, it calls the nsGeolocationRequest::NotifyError method, that will fire the failure callback
  •  Checks the nsIJSContextStack
  •  Calls the HandleEvent method on the success callback nsIDOMGeoPositionCallback
  • Removes the JSContext from the stack
  •  If the watchPositionRequest is set to true, then calls the nsGeoLocationRequest::SetTimeoutTImer

nsGeolocation::RemoveRequest

  • Back to the RequestSendLocationEvent::Run.
  • If the object was initialized with a reference to the Geolocation object, it then calls the nsGeolocation::RemoveRequest method
  • The Request method removes the request from the pendingCallback array, and then call the nsGeolocationRequest::MarkCleared method

nsGeolocationRequest::MarkCleared()

  •  nsGeolocationRequest::MarkCleared method sets the mCleared to true

Back to nsGeolocation::GetCurrentPosition

  • If the RegisterRequestWithPrompt was successful, it then stores the request on the mPendingCallbacks array
  • Checks if the caller is the Chrome, I’m still not 100% of what Chrome means on firefox, but my guess here is that it’s checking if it’s actually the Chrome(Browser) is making the request and not somebody else?
  • IsCallerChrome
  • SubjectPrincipalIsSystem
  • Curious that the value returned by the IsCallerChrome depends on the result of SubkectPrincipalIsSystem, and on the SubjectPrincipalIsSystem the argument passed it’s always set to true. Not sure why that is done, since it will always evaluate to true.
  •  Appends the request to the mPendingCallbacks.
  •  Creates a new RequestAllowEvent
  • Dispatches the event to the main thread
  • **Step 7 all the way to step 8 only happens if the mOwner exists, not sure why the mOwner wouldn’t exist, since here validates mOwner, and if it doesn’t exist returns NS_ERROR_FAILURE

Here is a simple class diagram I made for the nsGeolocation.cpp file.


Unlocking the mouse when fullscreen is canceled

Getting the mouse to unlock whenever the window loses fullscreen mode wasn’t an easy task.
Mostly because I’m not very familiar with the firefox code, and I’m still trying to understand how all the different components are able to communicate between them.
A few times I thought about giving up on the task, but I’m glad I didn’t. Looking back, the amount of new things I learnt was worth every minute spent.

1. Finding where to write the code

This is the call call stack for a fullscreen request:
http://pastebin.com/cKbtHxP2

I just went as low as the nsWindow::MakeFullScreen, since there is where the call to native libraries happens.
Because the nsWindow is platform specific, I decided to go one level of above. To the nsGlobalWindow::SetFullScreen

The SetFullScreen method of the nsGlobalWindow is called whenever the status of the screen needs to be changed(to enter or exit fullscreen).
I thought that placing the code to unlock the mouse on the SetFullScreen would be a good option, since it’s the last method call before going into platform specific files.

2. The problem

At first I thought that if the screen exited fullscreen mode the nsGlobalWindow::SetFullScreen would be called, then I would get a reference to the navigator, then to the pointer, and then unlock the mouse.
Here is the stack for canceling the fullscreen
Cancel fullscreen stack

On the stack, I saw that the nsDocument::MozCancelFullScreen was called, then the nsDocument::CancelFullScreen, and then kept going until the nsGlobalWindow::SetFullScreen

The only problem was that when the window exited the fullscreen mode, the nsGlobalWindow:SetFullScreen would only set the fullscreen status of the root window. However, the navigator was attached to the outer window, so it wasn’t possible to access the navigator when the fullscreen mode was canceled.

Different memory address.

3. Inspecting the problem

I saved some printf’s

First I opened the js console an accessed the navigator by typing window.navigator
Accessing window.navigator from js console

Line 7 gets the navigator for window7
Line 17 creates the navigator, since its the first time its being referred to

Then on the same page I requested an element to go fullscreen, and later canceled the fullscreen
Requesting and Canceling fullscreen

Line 7 is setting the window7 to fullscreen (the outer window, same as the one that created the navigator
Line 11 checks to see if window7 is the root window, because isn’t, it calls SetFullScreen on the root window, that has an id of 1

Line 20 the document is canceling the fullscreen
Line 21 it’s the id of the window of the document that is being canceled.
Line 29 is existing fullscreen mode of window1
Line 37 gets the navigator for the window1
Line 57 unlocks the mouse for window1

When the window exited the fullscreen mode, it was window1(root) that was set to false, not the window7(outer), and because each window has a different navigator, the navigator for window1 wasn’t the same as the one for window7

4. The solution

My solution, was to write the code to unlock the mouse on nsDocument::CancelFullScreen. On the CancelFullScreen method, there is a loop that iterates through the documents, and sets their fullscreen status to false. Inside the loop, one of the iterations gives the document which has the locked element. So with a reference to the doc, it’s possible to get the window for that document, the navigator, the pointer, and finally, unlock it.
Solution

while (doc != nsnull) {
    if (::SetFullScreenState(doc, nsnull, false)) {

      nsCOMPtr window = doc->GetWindow();
      nsCOMPtr navigator;
      window->GetNavigator(getter_AddRefs(navigator));

      if (navigator) {
        nsCOMPtr pointer;
        navigator->GetPointer(getter_AddRefs(pointer));
        if (pointer) {
          pointer->Unlock();
        }
      }

      DispatchFullScreenChange(doc);
    }
    doc = doc->GetParentDocument();
  }

Now whenever the element loses fullscreen mode, the mouse is automatically unlocked.

*The only reason I got this working, was because of the great help of the mozilla community.
Big thanks to cpearce!


Testing MouseLock FullScreen

After a lot of tries and fails I got the navigatorPointer and mozFullScreen tests working.

I didn’t think that the process of getting the tests to work would be so tricky.

1. Running navigatorPointer test

I tried to run the navigatorPointer test, but nothing would appear on the browser screen. However, on the terminal, I could see the output for all the tests. Browsing through the Mochitest wiki I found this page that lets you run mochitests without building firefox. I ran the navigatorPointer test there and the output appeared on the screen.
So I knew the test was working, but for some unknown reason it only worked on the mochitester page, not when building the mochitest-plain suite.

My first thought was to run other tests from the mochitest-plain suite and see if they worked, I found and ran the outerWindow test, and the output was successfully displayed on the browser.
I then opened side by side the outerWindow and the navigatorPointer tests and started inspecting them to see what was different. The only major difference was that on the outterWindow, the test started with:

SimpleTest.waitForExplicitFinish()

and ended with

SimpleTest.finish();

I added those two lines in the navigatorPointer.html and the mochitest worked 🙂
I’m not sure why the test needs to explicity call the SimpleTest.finisht() since the test itself doesn’t have any asynchronous calls.

2. mozFullScreen API

Before writing the mochitest to check if the mouse was being locked when the element was in fullscreen mode, I started hacking around the mozFullScreen api to see how I could implement the tests.
The first problem I noticed, was that the mozfullscreenchange event was only fired when there was some sort of user interaction, for example:

document.body.mozRequestFullScreen();

wouldn’t work, and:

document.getElementById("test").addEventListener("click", function () {
      document.body.mozRequestFullScreen();
}, false);

would.

I first thought that the mozRequestFullScreen() could only be invoked if some sort of event was fired, and because in the mochitest, there is no user interaction, and all the tests are automated, I needed to find a way to create and fire an event in javascript. I asked that question on the #seneca channel and James(jboelen) pointed me in the right direction: document.createEvent

However, even creating and firing a synthetic event still didn’t trigger the mozfullscreenchange, I believe that for security reasons, to request a fullscreen change on the browser, the user must give some sort of acknowledgement by clicking the mouse or pressing a key.

I then started thinking of different ways to test the fullscreen status for the mouselock, but then I realized that when they wrote the mochitests for the mozfullscreen api they must have faced the same problems. So I did a quick search on mxr and found these two files: file_fullscreen-esc-exit.html file_fullscreen-denied.html
Inspecting the mochitests, I saw how they requested a fullscreen change without user interaction:

SpecialPowers.setBoolPref("full-screen-api.allow-trusted-requests-only", false);

3. MouseLock fullscreen tests

When the browser goes fullscreen, the mozfullscrenchange event is sent to the document and the element that requested the fullscreen change, but when the window looses the fullscreen mode, the mozfullscreenchange event is only sent to the document, so to listen for both the entering and exiting of fullscreen mode, the mozfullscreenchange event needs to be attached to the document.

The solution I came up for the test, was to add the event listener to the document, and inside the event listener check if the screen was entering, or existing the fullscreen mode. If the screen was entering, I would check if the element in fullscreen mode was the one with the mouse locked and if the screen was exiting, I would test to make sure the mouse wasn’t locked.
The problem with that, was that because the test was asynchronous, and inside the event listener the SimpleTest.finish() needed to be invoked. But because the event was fired more than once, I didn’t know where to place it.
To make sure the test was only finished when all the test cases were completed, I created two variables to keep track of how many tests needed to be performed, and how many were completed.

       document.addEventListener("mozfullscreenchange", function (e) {
        if (document.mozFullScreen) {
          if (document.mozFullScreenElement === canvas) {
            // Mouse should be lock if the element is in fullscreen mode
            pointer.lock(canvas);
            is(pointer.islocked(), true, "Mouse should be locked");
            nTests += 1;
            document.mozCancelFullScreen();
          }
          else {
            // Mouse shoudn't lock if element isn't in fullscreen mode
            pointer.lock(canvas);
            isnot(pointer.islocked(), true, "Mouse should only be locked if the locked element is in fullscreen mode");
            nTests += 1;
            document.mozCancelFullScreen();
          }
        }
        else {
          // Mouse should be unlocked if winodw is not in fullscreen mode
          is(pointer.islocked(), false, "Mouse should be unlocked if winodw is not in fullscreen mode");

          if (nTests === totalTests) {
            // Finish test
            SimpleTest.finish();

          }
        }
      }, false);

So every time the screen lost the fullscreen status, a test would run to check if the mouse was unlocked. If all the tests were completed, the SimpleTest.finish() would be invoked.

I then ran the mochitest. It didn’t work. The screen just stayed blank.
I started debugging, login some messages to the console, and I found out that the mozfullscreenchange event was being fired only once, even though two elements were requesting fullscreen mode. I thought that maybe because the requests were being made one after another, the events were getting mixed up. I added a setTimeout for the second request and then all the tests ran successfully.

Some tests are failing because the functionality still needs to be implemented in the MouseLock.

  • The MouseLock needs to unlock the mouse when the element looses the fullscreen mode
  • The mouse should only be locked if the element is in fullscreen mode

I committed the changes here.


MouseLockLost Event

Since we began hacking firefox, I was curious to find out how the events were handled internally.
I saw the mouselocklost event as an opportunity to hack around the event chain in firefox and see how all the differents events worked.
First I started tracking the hierarchy of the nsDOMMouseEvent, that led me to nsDOMUIEvent and nsDOMevent. On the nsDOMEvent.h file, there was an enum with the name of all firefox events. Next I looked at the nsDOMEvent.cpp, I saw that there was an array, again, with the names of all firefox events, still in the nsDOMEvent.cpp there was a method called GetEventName that used a switch statement to find which event was fired in the browser and return its name.

switch(aEventType) {
1118   case NS_MOUSE_BUTTON_DOWN:
1119     return sEventNames[eDOMEvents_mousedown];
1120   case NS_MOUSE_BUTTON_UP:
1121     return sEventNames[eDOMEvents_mouseup];
.......
.........
..............

and the list just kept getting bigger and bigger, I think in total, there were 134 different events.
An interesting point:

  • For some reason, every time I clicked on the browser, the NS_MOUSE_BUTTON_UP event got fired, but the NS_MOUSE_BUTTON_DOWN, and NS_MOUSE_CLICK didn’t.

So my next move was to search on mxr for the macros name to see where they were declared.
All the macros were declared in the nsGUIEvent.h file.
I wasn’t sure where to declare, and which value to give to the NS_MOUSELOCKLOST macro, so I just placed together with all the other mouse event macros and incremented by one the value of the last mousemacro:

#define NS_MOUSE_MESSAGE_START          300
#define NS_MOUSE_MOVE                   (NS_MOUSE_MESSAGE_START)
#define NS_MOUSE_BUTTON_UP              (NS_MOUSE_MESSAGE_START + 1)
#define NS_MOUSE_BUTTON_DOWN            (NS_MOUSE_MESSAGE_START + 2)
#define NS_MOUSE_ENTER                  (NS_MOUSE_MESSAGE_START + 22)
#define NS_MOUSE_EXIT                   (NS_MOUSE_MESSAGE_START + 23)
#define NS_MOUSE_DOUBLECLICK            (NS_MOUSE_MESSAGE_START + 24)
#define NS_MOUSE_CLICK                  (NS_MOUSE_MESSAGE_START + 27)
#define NS_MOUSE_ACTIVATE               (NS_MOUSE_MESSAGE_START + 30)
#define NS_MOUSE_ENTER_SYNTH            (NS_MOUSE_MESSAGE_START + 31)
#define NS_MOUSE_EXIT_SYNTH             (NS_MOUSE_MESSAGE_START + 32)
#define NS_MOUSE_MOZHITTEST             (NS_MOUSE_MESSAGE_START + 33)
#define NS_MOUSEENTER                   (NS_MOUSE_MESSAGE_START + 34)
#define NS_MOUSELEAVE                   (NS_MOUSE_MESSAGE_START + 35)
#define NS_MOUSELOCKLOST                (NS_MOUSE_MESSAGE_START + 36)

After declaring the mouselocklost macro, I kept searching on mxr for any other occurrences of the event macros in the source code, I found that they were also used in the nsEventNameList.h. I wasn’t sure which event name type I should use, so I added EventNameType_All, since it was used by most other mouse events.

EVENT(mouseup,
      NS_MOUSE_BUTTON_UP,
      EventNameType_All,
      NS_MOUSE_EVENT)
EVENT(mozfullscreenchange,
      NS_FULLSCREENCHANGE,
      EventNameType_HTML,
      NS_EVENT_NULL)
EVENT(mozfullscreenerror,
      NS_FULLSCREENERROR,
      EventNameType_HTML,
      NS_EVENT_NULL)
EVENT(mouselocklost,
      NS_MOUSELOCKLOST,
      EventNameType_All,
      NS_EVENT_NULL)

I then decided it was time to build to see if the mouselocklost event was being registered. However, I got some compile errors:

In file included from /home/diogogmt/mozilla-central/dom/base/nsGlobalWindow.cpp:10733:
../../dist/include/nsEventNameList.h:278: error: no ‘nsresult nsGlobalWindow::GetOnmouselocklost(JSContext*, jsval*)’ member function declared in class ‘nsGlobalWindow’
In file included from /home/diogogmt/mozilla-central/dom/base/nsGlobalWindow.cpp:10733:
../../dist/include/nsEventNameList.h:278: error: no ‘nsresult nsGlobalWindow::SetOnmouselocklost(JSContext*, const jsval&)’ member function declared in class ‘nsGlobalWindow’

Those errors were familiar, they meant that the getter and setter for the event didn’t exist, and I remembered from the GetScreenX and GetScreenY example that the Get and Set were added just for the c++ code, and that in javascript would be the just the method name, without Get and Set.
I thought about adding those methods in the nsGlobalWindow.cpp, but I looked at the file and didn’t find any getter/setter for the other events, so I knew I was missing something.
I went back searching on mxr, but this time without success. My next move was go  to IRC. I followed David Humphrey suggestion and asked first on the #seneca channel, then I went to the #introduction channel, and there they told me it would be best if I asked the question on the #developers channel. That’s what I did, and in less than 5 min I got an answer, I’m amazed about the speed and quality of the answers on IRC, I think that’s one of the reasons for the huge success of Mozilla, having a very strong community of developers.

So the answer I got on the channel, was that I was missing adding the mouselocklost event to two files, nsGkAtomList.h and nsIInlineEventHandlers.idl

They also recommended that I look at this patch.

I made the necessary changes on the code, and this time it worked 🙂
I’ve commited the changes to here and here

I’m not even sure that if that is how the mouselocklost event should be implemented, but I remembered the quote from today’s class:

The perfect is the enemy of the good

and instead of trying to write the perfect line of code, I just kept hammering around lots of files, and in the end, even though it was not a final solution, is was start.

What’s my next plan?

  • Find how to dispatch an event
  • Callback functions

Mouselock FullScreen

After reading the MouseLock wiki page I thought that the option 8 “Mouse lock should only work when in Full Screen Mode” would be a good place to start hacking.

My first searches led me to the GetFullScreen method of nsGlobalWindow http://mxr.mozilla.org/mozilla-central/source/dom/base/nsGlobalWindow.cpp#4453

Knowing that to get the fullscreen status of a browser window I would need to use the nsGlobalWindow::GetFullScreen the next step was to find how to call that method from the MouseLock class. I struggled a bit with that, since to get the FullScreen status of the window I would first need to have access to the window itself. After hitting a dead wall I went to irc looking for some help. Luckly, David Humphrey was there and was able to provide some good explanations for my problem. He suggested to create an Init method on MouseLock, similar to what the Geolocation class is doing.

After a good amount of hours and a good amount of errors….it worked! 🙂

So what happens is that on the Navigator::GetPointer, a MouseLock object is created and the Init method is called passing a reference to the current window as an argument.
The MouseLock::Init stores the window reference as a member variable, so then all the other methods can access the current window and check for its fullscreen status.

nsresult
nsDOMMouseLockable::Init(nsIDOMWindow* aContentWindow)
{
  NS_ENSURE_ARG_POINTER(aContentWindow);
  mWindow = aContentWindow;
  return NS_OK;
}

NS_IMETHODIMP nsDOMMouseLockable::Islocked(bool *_retval NS_OUTPARAM)
{
  // Check the status of the window
  bool* isFullScreen = new bool;
  mWindow->GetFullScreen(isFullScreen);
  printf("\nisFullScreen? %s\n", *isFullScreen ? "true" : "false");
  *_retval = mIsLocked;
  return NS_OK;
}
NS_IMETHODIMP
Navigator::GetPointer(nsIDOMMouseLockable** aPointer)
{
  NS_ENSURE_ARG_POINTER(aPointer);

  if (!mPointer) {
    mPointer = new nsDOMMouseLockable();
  }

  if (!mDocShell) {
    return NS_ERROR_FAILURE;
  }

  nsCOMPtr<nsIDOMWindow> contentDOMWindow = do_GetInterface(mDocShell);
  if (!contentDOMWindow) {
    return NS_ERROR_FAILURE;
  }

  if (NS_FAILED(mPointer->Init(contentDOMWindow))) {
    mPointer = nsnull;
    return NS_ERROR_FAILURE;
  }


  NS_ADDREF(*aPointer = mPointer);
  return NS_OK;
}

I committed all the changes on github

Here are a couple of screenshots testing if the MouseLock was getting the right values of the fullscreen status:


Implementing Mouse Lock P1 – Extending MouseEvent

To be honest, I still don’t fully understand how firefox handles all the layers between the OS and the javascript on the browser, anyway, I decided to take guess.
I did a little bit of research and narrowed down to two possible files that I would need to make changes to add the new properties(movementX/Y) to the MouseEvent object:

content/events/src/nsDOMMouseEvent.cpp
dom/interfaces/events/nsIDOMMouseEvent.idl

I declared the properties on the IDL file, and defined the getters on the cpp file.
After building….. it worked!! 🙂
There isn’t much happening though, since I’m just assigning the values of screenX/Y to movementX/Y, but it kind of gives a little bit of clarity on how firefox operates, because by modifying some c++ code it dynamically creates a javascript object and maps the communication between that object and the native OS APIs

It seems a bit unreal to think that we are implementing a new feature on firefox, looking 3 months back, the thought of modifying firefox never crossed my mind.

Changes: https://github.com/diogogmt/mozilla-central/commit/cd7478ed9fbe99ef9eeb88469999acfd21176177