Starlight Touch Documentation

0.2

This document covers Touch implementation for Windows 7 in Qt / WebKit.

Overview

The life of a touch event begins when the user performs a touch interaction. The front-end handles the touch event, and passes it to WebCore::EventHandler::handleTouchEvent.

WebCore::EventHandler::handleTouchEvent first updates its gesture context by calling WebCore::GestureContext::touchEvent. The gesture context updates the state of all GestureRecognizer objects it holds. After that, WebCore::EventHandler::handleTouchEvent just returns if the page does not have any touch event listeners (raw touch events or manipulate events).

WebCore::EventHandler::handleTouchEvent constructs touch lists corresponding to what has happened to tracked touch points (pressed, released, moved, etc). If no touch sequence was yet tracked, the first touch point press target is stored and will receive all subsequent touch events until all touch points are released.

After raw touch events are dispatched using above constructed touch lists, WebCore::EventHandler::handleManipulateEvent is called. It dispatches a manipulate event to one DOM node. If a manipulation is already in progress, all subsequent manipulate events are dispatched to its stored target node, regardless of where the touch points hit.

Next WebCore::EventHandler::handleTouchMouse is called, which generates emulated mouse events from the touch event, and dispatches them to DOM nodes.

After WebCore::EventHandler::handleTouchEvent returns, the frontend can perform page manipulation, such as scrolling or zooming, unless the touch event has started a manipulation for which the DOM handler declined the default handler execution by calling preventDefault on the event, or the same is done to the dispatched touchMove event.

DOM events

TouchEvent

Definition

module events {
    interface [
        Conditional=TOUCH_EVENTS
    ] TouchEvent : UIEvent {
        readonly attribute TouchList touches;
        readonly attribute TouchList targetTouches;
        readonly attribute TouchList changedTouches;
        readonly attribute boolean ctrlKey;
        readonly attribute boolean shiftKey;
        readonly attribute boolean altKey;
        readonly attribute boolean metaKey;
        void initTouchEvent(in TouchList touches,
                            in TouchList targetTouches,
                            in TouchList changedTouches,
                            in DOMString type,
                            in DOMWindow view,
                            in long screenX, 
                            in long screenY, 
                            in long clientX, 
                            in long clientY,
                            in boolean ctrlKey,
                            in boolean altKey,
                            in boolean shiftKey,
                            in boolean metaKey);
    };
}
module events {
    interface [
        Conditional=TOUCH_EVENTS,
        HasIndexGetter
    ] TouchList {
        readonly attribute unsigned long length;
        Touch item(in unsigned long index);
    };
}
module events {
    interface [
        Conditional=TOUCH_EVENTS
    ] Touch {
        readonly attribute long             clientX;
        readonly attribute long             clientY;
        readonly attribute long             screenX;
        readonly attribute long             screenY;
        readonly attribute long             pageX;
        readonly attribute long             pageY;
        readonly attribute EventTarget      target;
        readonly attribute unsigned long    identifier;
    };
}

See detailed descriptions in WebCore::TouchEvent, WebCore::TouchList or WebCore::Touch class reference pages.

Behaviour

A TouchEvent can be of the following types: touchstart, touchend, touchmove. Each such event contains three lists of touch points: touches, targetTouches, changedTouches. The touches list contains all the touches that are pressed at the time. The targetTouches list contains the touches that are on the element that received the first touch event. The changedTouches list contains the touches that are relevant for the touch event. For example, a touchend event contains all the touch points that are released in that touch event in its changedTouches list.

A touch point list can be indexed using the indexing operator, [i], where i is an integer from 0 up to and excluding the list's length property.

Each touch point contains the identifier for the touch and positional information.

ManipulateEvent

Definition

module events {
    interface [
        Conditional=TOUCH_EVENTS,
        GenerateConstructor
    ] ManipulateEvent : UIEvent {
        readonly attribute long screenX;
        readonly attribute long screenY;
        readonly attribute long clientX;
        readonly attribute long clientY;

        readonly attribute boolean ctrlKey;
        readonly attribute boolean shiftKey;
        readonly attribute boolean altKey;
        readonly attribute boolean metaKey;

        readonly attribute long panX;
        readonly attribute long panY;
        readonly attribute long panSpeedX;
        readonly attribute long panSpeedY;
        readonly attribute float scale;
        readonly attribute float scaleSpeed;
        readonly attribute float rotation;
        readonly attribute float rotationSpeed;

        [OldStyleObjC] void initManipulateEvent(in DOMString type,
                                                in boolean canBubble,
                                                in boolean cancelable,
                                                in DOMWindow view,
                                                in long detail,
                                                in long screenX,
                                                in long screenY,
                                                in long clientX,
                                                in long clientY,
                                                in long pageX,
                                                in long pageY,
                                                in boolean ctrlKey,
                                                in boolean altKey,
                                                in boolean shiftKey,
                                                in boolean metaKey,
                                                in long panX,
                                                in long panY,
                                                in long panSpeedX,
                                                in long panSpeedY,
                                                in float scale,
                                                in float scaleSpeed,
                                                in float rotation,
                                                in float rotationSpeed);
   };
}

See detailed descriptions in WebCore::ManipulateEvent class reference page.

Behaviour

The target element for manipulatestart event is the element that was pressed first. All manipulatemove events and manipulateend event are fired to the same element than manipulatestart.

manipulatestart

Happens when a finger or fingers appear on the screen for the first time.

The attribute values are set as follows:

  • Positional values according to the first finger on the screen.
  • panX = panY = panSpeedX = panSpeedY = 0
  • scale = 1.0, scaleSpeed = 0.0
  • rotation = 0.0, rotationSpeed = 0.0

manipulateend

Happens when all the remaining fingers are released from the screen. The attribute values are kept the same as in the latest move event.

manipulatemove

In all cases, fingers that are higher order than two are simply ignored. Only the two oldest are always considered.

Scale:

  • One finger on the screen: in use is the latest value from when two fingers were on the screen.
  • Two fingers on the screen: the scale is multiplied with the relative change of distance from the previous distance.
  • Speed is calculated from the difference between the new and the previous scale factor.

Rotation:

  • One finger on the screen: in use is the latest value from when two fingers were on the screen.
  • Two fingers on the screen: rotation is added, with the change between the current and the previous rotation.
  • Speed is calculated from the difference between the new and the previous rotations.

Pan:

  • One finger on the screen:
    • positional values are set according to the position of the first finger.
    • pan is incremented according to the difference between the current and the previous position.

  • Two fingers on the screen:
    • positional values are set according to the position centre point between the two fingers.
    • pan is incremented according to the difference between the current and the previous position.

Sending DOM events

Introduction

DOM events coming from the platfrom are processed synchronously. When the dispatch functions return, JavaScript has processed the event completely. There is also an asynchronous function for calling specific JavaScript functions from platform called MainThread::callOnMainThread, but this is of no concern in regard to event sending. Unless this specific function is called, communication between WebCore and JavaScriptCore is synchronous.

WebCore

What happens when DOM event is sent?

The following:

  1. Instance of Node gets message dispatchEvent. Let us call this instance N.

  2. N sends message dispatchGenericEvent to itself.
  3. N traverses DOM-tree up to the root and sends message handleLocalEvents to every node until the root of the DOM tree is reached.

Node has a list of EventListener instances. Node::handleLocalEvents calls EventListener::handleEvent for each EventListener instance. In the context of this specification, the following assumption can be made: EventListener instances are always JSEventListener instances.

JavaScriptCore

Let us call JSEventListener instance as E for this example. In handleEvent, E configures a JavaScript object that represents the handleEvent message for the counterpart JavaScript object. After that, E calls indirectly Interpreter::execute through a global function call defined in CallData.h. This is the function that actually executes the byte code.

Mouse Emulation

Mouse emulation passes touch events to corresponding mouse event handlers in WebKit. Before passing the event it however:

  • Uses smart algorithm to do hit testing on bigger area than just the exact point and shifts the coordinates accordingly so that the optimal element is hit. See "Smart hit testing algorithm"
  • Checks if the user performed long tap
  • Checks if the user performed double tap

Mouse emulation is cancelled if more than one fingers touch the screen. Mouse emulation will then be active only after all fingers have been lifted from the screen.

Selecting element from area

Criteria for best element selection:

  • Element nearest to the centre point
  • Prioritising of links, clickable and focusable elements

Smart hit testing algorithm

  • Do maximum N*M hit tests on an area of a circle of radius R.
    • N: Number of circles to scan through. Circles are distributed on equal distance on radius R. For example, if circle has radius of 5 and N=3, then radiuses are 5/3, 10/3 and 15/3.
    • M: Number of points on each circle to scan through.
  • For each point:
    1. Do a hit test
    2. If node is a link, return the coordinates and stop.
    3. If node is a clickable element, return the coordinates and stop.

    4. If node is a focusable element, return the coordinates and stop.
  • In the beginning of the algorithm do the same tests to the touch point and return the coordinates if it passes without starting the loop at all.
  • If none of the nodes satisfy the tests, return the original coordinates.

Long tap

  • Long tap is performed by pressing the screen a while without moving the finger noticeably
  • After a successful long tap:

    • Enter hover mode
    • Propagate contextmenu event if the pressed element doesn't have click and mouseover listeners

  • In hover mode:
    • Mousemove events are sent without any buttons down
    • Click event is not sent

Hover mode ends when the finger is released from the screen.

Page manipulation

  • Page manipulation is entered when default action for onmanipulatestart or touchmove is not prevented in their respective event handlers and finger moves a small threshold or more than one fingers are pressed on screen.

  • Page manipulation is not entered if mouse emulation enters into hover mode.
  • Page manipulation is not entered if scrollbar is hit
  • Page manipulation is not entered if the pressed element has mousedown and mousemove listeners. This is to enable dragging of elements without page starting to scroll.
  • Page manipulation does scrolling and zooming according to pan and pinch.

Deployment

MODIFIED:

ADDED:

Generated by  doxygen 1.6.2-20100208