Handling Multi-touch and Mouse Input in All Browsers

Touch interaction with Web sites and apps has the opportunity to improve their usability and ubiquity as the Web and Windows 8 Metro style apps play a key role on tomorrow’s touch-enabled devices.

This post explains how Web developers can use the new IE10 pointer event model along with the iOS touch event model and the W3C mouse event model (as extended) to create a cross-browser, common-code handler for pointer, touch, and mouse input.

A little background: I’m fortunate to have a Samsung 700T Windows Developer Preview tablet PC. With it, I’ve been able to enjoy the IE Test Drive multi-touch demos Touch Effects and Lasso Birds. Like me, you may have noticed that Lasso Birds works across different devices and browsers in addition to IE10. For example, its multi-touch works on iOS devices. In this post, I’ve taken some of the patterns from Lasso Birds and generalized and extended them to include older versions of browsers.

The result of my experimentation is below. It should work in your browser. A discussion of the coding patterns and lessons learned follows below the demo.

The Code

The basic algorithm for drawing with the mouse model is straightforward:

var drawingStarted = false;

function DoEvent(eventObject) {

if (eventObject.type == "mousedown") {

drawingStarted = true;

startDraw(eventObject.pageX, eventObject.pageY);

}

else if (eventObject.type == "mousemove") {

if (drawingStarted) {

extendDraw(eventObject.pageX, eventObject.pageY);

}

}

else if (eventObject.type == "mouseup") {

drawingStarted = false;

endDraw();

}

}

The only change needed to make this work with IE10’s pointer events is to add awareness that multiple pointers can be down at the same time, each identified by a different pointerId value. The IE10 pointer model fires separate MSPointerDown, MSPointerMove, and MSPointerUp events for each pointer that changes state.

var drawingStarted = {};

function DoEvent(eventObject) {

eventObject.preventManipulation(); // without this, instead of drawing, you pan

var pointerId = eventObject.pointerId;

if (eventObject.type == "MSPointerDown") {

drawingStarted[pointerId] = true;

startDraw(pointerId, eventObject.pageX, eventObject.pageY);

}

else if (eventObject.type == "MSPointerMove") {

if (drawingStarted[pointerId]) {

extendDraw(pointerId, eventObject.pageX, eventObject.pageY);

}

}

else if (eventObject.type == "MSPointerUp") {

delete drawingStarted[pointerId];

endDraw(pointerId);

}

}

Adapting the original mouse model to Apple’s iOS touch event model requires that you iterate through the list of changedTouches for each touchstart, touchmove, and touchend event because, in the iOS model, state changes that occur at the same time are bundled into one event. Like the IE10 pointer model, a unique identifier identifies each touch point.

var drawingStarted = {};

function DoEvent(eventObject) {

eventObject.preventDefault(); // without this, instead of drawing, you pan

for (var i = 0; i < eventObject.changedTouches.length; ++i) {

var touchPoint = eventObject.changedTouches;

var touchPointId = touchPoint.identifier;

if (eventObject.type == "touchstart") {

drawingStarted[touchPointId] = true;

startDraw(touchPointId, touchPoint.pageX, touchPoint.pageY);

}

else if (eventObject.type == "touchmove") {

if (drawingStarted[touchPointId]) {

extendDraw(touchPointId, touchPoint.pageX, touchPoint.pageY);

}

}

else if (eventObject.type == "touchend") {

delete drawingStarted[touchPointId];

endDraw(touchPointId);

}

}

}

Merging these three individual algorithms requires noting the differences between the event names and the unique pointer identifier attribute names and accounting for the lack of an identifier in the mouse model.

In the merged model, below, I also add a check that the “move” position has actually changed because the IE10 pointer model streams MSPointerMove events with the same x, y position when a touch point is held down but not moved. By filtering out these redundant moves, I eliminate calls to extendDraw() that do nothing. I implemented this check by storing the last x,y position from a start or move in the lastXY object and, by checking that a lastXY entry exists for a particular id, lastXY replaces the drawingStarted object used in the previous two examples.

var lastXY = { };

function DoEvent(eventObject) {

// stop panning and zooming so we can draw

if (eventObject.preventManipulation)

eventObject.preventManipulation();

else

eventObject.preventDefault();

// if we have an array of changedTouches, use it, else create an array of one with our eventObject

var touchPoints = (typeof eventObject.changedTouches != 'undefined') ? eventObject.changedTouches : [eventObject];

for (var i = 0; i < touchPoints.length; ++i) {

var touchPoint = touchPoints;

// pick up the unique touchPoint id if we have one or use 1 as the default

var touchPointId = (typeof touchPoint.identifier != 'undefined') ? touchPoint.identifier : (typeof touchPoint.pointerId != 'undefined') ? touchPoint.pointerId : 1;

if (eventObject.type.match(/(down|start)$/i)) {

// process mousedown, MSPointerDown, and touchstart

lastXY[touchPointId] = { x: touchPoint.pageX, y: touchPoint.pageY };

startDraw(touchPointId, touchPoint.pageX, touchPoint.pageY);

}

else if (eventObject.type.match(/move$/i)) {

// process mousemove, MSPointerMove, and touchmove

if (lastXY[touchPointId] && !(lastXY[touchPointId].x == touchPoint.pageX && lastXY[touchPointId].y == touchPoint.pageY)) {

lastXY[touchPointId] = { x: touchPoint.pageX, y: touchPoint.pageY };

extendDraw(touchPointId, touchPoint.pageX, touchPoint.pageY);

}

}

else if (eventObject.type.match(/(up|end)$/i)) {

// process mouseup, MSPointerUp, and touchend

delete lastXY[touchPointId];

endDraw(touchPointId);

}

}

}

The examples above specifically ignore the issues of registering to receive the events or ensuring that they apply to the drawing target. Making this work for real and with all browsers—including versions of Internet Explorer before IE9—requires a bit more work. Interested parties can peruse the final version of my multi-browser, multi-touch drawing class here.

By coding for touch alongside mouse, Web developers can assure their sites work with all browsers—whether desktop, tablet, or phone.

—Ted Johnson, Graphics Program Manager Lead, Internet Explorer

[attach=full]5165[/attach]

Continue reading...