Haptic Technology: Mult-itouch & Gestures

Folks, I have been quite busy lately with regular offline work coupled with articles for Handsonlabs Software Academy website. I’m working on balancing the offline and online work. Besides, I am also working on the new Handsonlabs book in Java, which should be published shortly. In addition, series of Android apps are underway, to be made known on GooglePlay. Do stay tuned.

 

memo-hd-7-front

 

Recently as previously discussed, there has been an explosion of innovation utilizing the latest research in haptic technology. From touch based iphone to Microsoft surface to tangible user interfaces such as the Reactable, to gesture based motion tracking as Microsoft Kinect.
Graphical User Interface based keyboard and mouse formerly accustomed to be already outdated. Little wonder why there’s been a heavy drop in PC sales worldwide. Who knows, maybe the human hand as well as the entire body might be the interaction of the future.
The first multitouch came around in 1982 by Nimish Mehta of the University of Toronto. Bell Labs as well as others picked up from here before iphone stepped in. Mind you, future ATMs, DVDs as well as other consumer centric items will soon become fully touch based.

A simple Definition:
1. Multi-touch: It’s all about detecting one or more touches.
2. Gesture: Series of touches maybe recognized or coded as a pattern, registered as a single event.

Types:
Resistive Technology: Consists of 2 flexible sheets, electrically conductive, vertical horizontal lined for precision location, accurate, fairly expensive and does not support multi-touch gestures.

Capacitive Technology: consists of insulator like glass (ground plane), coated with a transparent conductor), voltage surface, finger (capacitor simulator) and electrostatic fielded.
Others. Like non mobile devices or equipment, in a controlled environment. Include infrared, acoustic digitizer, and in cell.

Now, quickly let me dive into the core technical concepts.
The Multi-touch Class: Flash platform breaks down gestures across platforms and the tools as such. The factuality is that support varies from Windows, iOS, Mac OS X, Android and RIM.
Flash.ui.Multitouch class is a modern addition to the Action script Language to support user input. Android supports multitouch and could be a great testing ground.

Algorithm Code instance:

import flash.ui.Multitouch;
if (Multitouch.supportsGestureEvents == true) {
trace(Multitouch.supportedGestures);
}

Let’s not forget the principles underlining object oriented programming:
Object.property.method (Noun.Adjective.verb) structure
Supported Gestures property return an array of gestures that specific device understands. For example, some of gestures for Nexus One understands are:

GestureZoom, gestureRotate, gesturePpan, gestureswipe, gestureTwoFingerTap.

Testing for mutitouch:

if (Multitouch.supportsTouchEvents == true) {
trace(Multitouch.maxTouchPoints);
}

1.  Gesture Mode None    – Mouse Event
2. Gesture mode               – GestureEvent
3. Touchpoint                   – Touch Event
These 3 can be changed at run-time only if it’s necessary and is not confusing to the user.
The GestureEvent Class: these are multiple points as a recognizable pattern. Adobe Air Android platform has three major  GestureEvent Classes namely:
GestureEvent, TransformGestureEvent, and PressAndTapGestureEvent.
Gesture Evens have phase property to indicate gesture progress these are: Begin, Update, End and All.

Algorithm code sample:

import flash.ui.MultitouchInputMode;
import flash.events.GesturePhase;
import flash.events.TransformGestureEvent
function onZoom(event:TransformGestureEvent):void {
if (event.phase == GesturePhase.BEGIN) {
// play hello sound
} else if (event.phase == GesturePhase.END) {
// play good bye sound
}
}

Categories of TransformGestuture Event
1. The Zoom Gesture: It’s pinching; let’s say increase/decrease distance between fingers to scale an object up or down.
The Zoom Gesture Algorithm:

zoom_g

import flash.ui.Multitouch;
import flash.ui.MultitouchInputMode;
import flash.display.Sprite;
import flash.events.TransformGestureEvent;
var sprite:Sprite;
Multitouch.inputMode = MultitouchInputMode.GESTURE;
sprite = new Sprite();
sprite.x = stage.stageWidth * 0.5;
sprite.y = stage.stageHeight * 0.5;
var g:Graphics = sprite.graphics;
g.beginFill(0xFF6600);
g.drawCircle(0, 0, 150);
g.endFill();
sprite.addEventListener(TransformGestureEvent.GESTURE_ZOOM, onZoom);
sprite.x = stage.stageWidth * 0.5;
sprite.y = stage.stageHeight * 0.5;
function onZoom(event:TransformGestureEvent):void {
sprite.scaleX *= event.scaleX;
sprite.scaleY *= event.scaleY;
}

2. The Rotate Gesture: Objects rotated in two paths, 1. Finger on, 2nd finger around. 2. Two fingers apart, one clocked other anti clocked.

 

gesturerotate

The Rotate Gesture Algorithm Code:

import flash.display.Sprite;
import flash.events.TransformGestureEvent;
import flash.ui.Multitouch;
import flash.ui.MultitouchInputMode;
var sprite:Sprite;
Multitouch.inputMode = MultitouchInputMode.GESTURE;
sprite = new Sprite();
sprite.x = stage.stageWidth * 0.5;
sprite.y = stage.stageHeight * 0.5;
var g:Graphics = sprite.graphics;
g.beginFill(0xFF6600);
g.drawRect(-150, -150, 300, 300);
g.endFill();
sprite.addEventListener(TransformGestureEvent.GESTURE_ROTATE, onRotate);
sprite.x = stage.stageWidth * 0.5;
sprite.y = stage.stageHeight * 0.5;
function onRotate(event:TransformGestureEvent):void {
event.currentTarget.rotation += event.rotation;
}

3. The Pan Gesture: Simply reveals object off screen if it’s larger than the screen. Two fingers are intuitively used.

pan_g

The Pan Gesture Code Algorithm:

Multitouch.inputMode = MultitouchInputMode.GESTURE;
var sprite:Sprite;
function createArt():void {
sprite = new Sprite();
addChild(sprite);
var g:Graphics = sprite.graphics;
g.beginFill(0xFFCCFF);
g.drawRect(0, 550, 1000, 200);
g.endFill();
g.lineStyle(3, 0xFF0000);
// 650 is an arbitrary pixel position
g.moveTo(2, 650);
// draw a sin wave
var xpos:Number = 0;
var ypos:Number = 0;
var angle:Number = 0;
for (var i:int = 0; i < 200; i++) {
xpos += 5;
ypos = Math.sin(angle)*100 + 650;
angle += 0.20;
sprite.graphics.lineTo (xpos, ypos);
}
stage.addEventListener(TransformGestureEvent.GESTURE_PAN, onPan);
}
function onPan(event:TransformGestureEvent):void {
// move the sprite along with the motion
sprite.x += event.offsetX;
}

4. The Swipe Gesture: Discharges an element as though pushing off screen. Direction of the swipe is determined by a single integer. Left to Right, bottom to top is minus one(-1) . While Right to Left and bottom to top gives one (1).

 

swipe_g

 

Swipe Gesture Algorithm Code:
import flash.events.TransformGestureEvent;
import flash.text.TextField;
import flash.text.TextFormat;
var pageText:TextField;
var counter:int = 1;
function createArt():void {
var textFormat:TextFormat = new TextFormat();
textFormat.size = 90;
textFormat.color = 0xFF6600;
// create a text field to display the page number
pageText = new TextField();
pageText.x = 100;
pageText.y = 200;
pageText.autoSize = TextFieldAutoSize.LEFT;
pageText.defaultTextFormat = textFormat;
pageText.text = "Page " + counter;
addChild(pageText);
// create a listener for a swipe gesture
stage.addEventListener(TransformGestureEvent.GESTURE_SWIPE, onSwipe);
}
function onSwipe(event:TransformGestureEvent):void {
counter -= event.offsetX;
if (counter < 1) counter = 1;
pageText.text = "Page " + counter;
}

 

5. The Press and Tap Gesture: More advanced than others mentioned so far. One finger pressed down while the other taps.

gettingstarted-concept-press-and-hold-gesture

 

Press and Tap Example Code

import flash.events.PressAndTapGestureEvent
stage.addEventListener(PressAndTapGestureEvent.GESTURE_PRESS_AND_TAP,
onPressAndTap);
function onPressAndTap(event:PressAndTapGestureEvent):void {
trace(event.tapLocalX);
trace(event.tapLocalY);
trace(event.tapStageX);
trace(event.tapStageY);
}

6. The Two-Finger Tap Gesture: It has only a type, Gesture_Two_Finger_Tap. Similar to mouse click only limited to a special area with two fingers near each other. It could be applied to pausing videos or playing them.

Two-Finger Tap_g

Two-Finger Tap Gesture Sample Algorithm Code.

import flash.events.PressAndTapGestureEvent;
sprite.addEventListener(GestureEvent.GESTURE_TWO_FINGER_TAP, onTwoFinger);
function onTwoFinger(event:GestureEvent):void {
// play or pause video
}

The Touch Event Class: Just like the mouse event, with multiple events at once.
Code:

import flash.ui.Multitouch;
import flash.ui.MultitouchInputMode;
import flash.events.TouchEvent;
Multitouch.inputMode = MultitouchInputMode.TOUCH_POINT;
import flash.events.TouchEvent;
Multitouch.inputMode = MultitouchInputMode.TOUCH_POINT;
stage.addEventListener(TouchEvent.TOUCH_TAP, onTouchTap);
function onTouchTap(event:TouchEvent):void {
var sprite:Sprite = new Sprite();
sprite.graphics.lineStyle(25, Math.random()*0xFFFFFF);
sprite.graphics.drawCircle(0, 0, 80);
sprite.x = event.stageX;
sprite.y = event.stageY;
addChild(sprite);
}

var touches:Object = {};
stage.addEventListener(TouchEvent.TOUCH_BEGIN, onTouchBegin);
stage.addEventListener(TouchEvent.TOUCH_END, onTouchEnd);
function onTouchBegin(event:TouchEvent):void {
var sprite:Sprite = createCircle(event.stageX, event.stageY);
addChild(sprite);
// store the touchPointID and the sprite
touches[event.touchPointID] = sprite;
// drag the sprite
sprite.startTouchDrag(event.touchPointID, true);
}
function onTouchEnd(event:TouchEvent):void {
// retrieve the sprite using the touchPointID
var sprite:Sprite = touches[event.touchPointID];
// stop drag and destroy
stopTouchDrag(event.touchPointID);
sprite.graphics.clear();
removeChild(sprite);
touches[event.touchPointID] = null;
}
function createCircle(x:int, y:int):Sprite {
var sprite:Sprite = new Sprite();
sprite.graphics.lineStyle(25, Math.random()*0xFFFFFF);
sprite.graphics.drawCircle(0, 0, 100);
sprite.x = x;
sprite.y = y;
return sprite;
}

TouchPointID has a special ID associated with it, from Touch_Begin to Touch_End, it provides identification and point storage with method or data associated with it.
These are the technical basic fundamentals of the touch devices that are making the rounds right now all over the world and effectively outdating the Personal Computer (PC) which we used to know. Don’t be left out.