The Language of Motion

Advanced Touch Processing in Android

Collin Flynn
Published on Medium•June 29th, 2016

The physical world has physical rules. Your app does not.

Material Design is effective because it leverages what people already understand about the world. Navigating real space involves an internal model of your surroundings. Turning around, left, or right means something to you within that model. If you get lost somewhere unfamiliar, it’s possible to accept that your model is incomplete or mistaken.

That account settings screen was around here somewhere.

But in an application, the same type of misunderstanding can be excruciating. You probably can’t prevent yourself from unconsciously making the same misstep again, and there is no physical arbiter of truth to say you’re wrong.

Implementing convincing spacial relationships between the parts of your application can take considerable effort, but consider what you’ll get for free: the user’s own intuitions will guide them as they interact.

For your part, you have to mimic some subtle characteristics of real objects. We’re going see how to use tension, inertia, and visual boundary cues to your advantage.

A brief aside: If you are new to touch handling there are some great resources that explain the raw mechanics of the Android View hierarchy. For a tour of relevant classes and methods, check out Mastering the Android Touch System. The official documentation also has some best practices for handling touch events. Finally, the Material Design Guidelines for Motion give clear do’s and don’ts.

The purpose of this article is to provide real examples of transforming the touch event stream into meaningful motion. Everything that follows is available for your inspection in the demo project on Github.

Processing motion events often involves a cycle of tweaking variables, feeling the results with your own hands on a real device, then tweaking again. Therefore it’s useful to implement visual feedback directly on screen.

To begin, here’s a View we can touch that will pass its MotionEvents along to an arbitrary listener.

Listening to those events, there is a View that will show them as a debug log below the touch target.

Pretty simple. Each time our finger meets the screen we get an ACTION_DOWN, followed by an arbitrary number of ACTION_MOVE events. When we lift, we can see one final ACTION_UP.

public class NoisyMotionEventView extends View {

    private MotionEventListener mListener;// ...@Override
    public boolean onTouchEvent(MotionEvent event) {
        if (mListener != null) {
            mListener.onMotionEvent(event); // Log all events.
        }
        return super.onTouchEvent(event);
    }
}

If you’re new to touch handling, consider these questions:

  • What happens when our finger drags outside the boundary of the view? Does the stream of touch events stop?
  • Do we always get an ACTION_UP at the end?
  • What happens when you put a second finger on the touch target view? Outside it?

If you don’t know any of those answers, try it for yourself! Run the source code on your own device. If any outcome is surprising, check out those resources linked above.

Getting Sloppy

You might notice that Android’s default scrolling behavior gives a little wiggle room before it starts tracking a horizontal or vertical swipe. Fingers are fat compared to a pixel, so the OS calculates an affordance radius based on the device screen density.

With custom touch processing, it’s important to respect that affordance so that any internal child Views can still receive touch events without being intercepted by the parent View. Keep this in mind if you are overriding onInterceptTouchEvent().

Expanding on the previous sample, here’s the same touch target but each ACTION_DOWN event is recorded and drawn with a circle that shows the touch slop area. A slider increases the radius.

public class TouchSlopView extends View {

    private float mLastDownX;
    private float mLastDownY;
    private int mScaledTouchSlop;
    private Paint mPaint;    public TouchSlopView(Context context) {
        super(context);
        mScaledTouchSlop = ViewConfiguration.get(context)
                           .getScaledTouchSlop();
    }
    
    //...

    @Override
    public boolean onTouchEvent(MotionEvent event) {
        switch (event.getAction()) {
            // ...
            case MotionEvent.ACTION_DOWN:
                mLastDownX = event.getX(); // Capture the DOWN event
                mLastDownY = event.getY();
                break;
            // ...
        }
        return super.onTouchEvent(event);
    }

    @Override
    protected void onDraw(Canvas canvas) {
        super.onDraw(canvas);
        canvas.drawCircle(mLastDownX, // Draw the touch slop circle
                          mLastDownY,
                          mScaledTouchSlop,
                          mPaint);
    }
}

Visualizing Elasticity

What if the circle drawn above was stretchy? Could we drag it it around or pull its boundary along with our finger?

We’ll need to track a few facts about the touch event stream. Similar to how we were passing each MotionEvent to the Log, this time pass them into a processor that will write down the state as it changes from each consecutive MotionEvent.

The state holder object:

public class TouchState {
    /**
     * No-Value for state.
     */
    public static final float NONE = -1f;
    /**
     * The relative x coordinate where the motion event started.
     */
    public float xDown = NONE;
    /**
     * The relative y coordinate where the motion event started.
     */
    public float yDown = NONE;
    /**
     * The current x coordinate
     */
    public float xCurrent = NONE;
    /**
     * The current y coordinate
     */
    public float yCurrent = NONE;
    /**
     * The distance between the down and current coordinates.
     */
    public float distance = NONE;

    /**
     *
     */
    public void reset() {
        xDown = NONE;
        yDown = NONE;
        xCurrent = NONE;
        yCurrent = NONE;
        distance = NONE;
    }
}

And here is a simple touch event processor that writes down state as each MotionEvent is passed in:

public class TouchStateTracker implements TouchProcessor {

    private TouchState mState;    // ...    @Override
    public void onTouchEvent(View v, MotionEvent event) {
        switch (event.getAction()) {
            case MotionEvent.ACTION_UP:
            case MotionEvent.ACTION_CANCEL:
                mState.reset();
                break;
            case MotionEvent.ACTION_DOWN:
                mState.xDown = event.getX();
                mState.yDown = event.getY();
                break;
            case MotionEvent.ACTION_MOVE:
                mState.xCurrent = event.getX();
                mState.yCurrent = event.getY();
                mState.distance = Geometry.distance(mState.xDown,
                   mState.yDown, mState.xCurrent, mState.yCurrent);
                break;
        }
    }

}

The View can reference the recorded state at any time to draw itself. Later on, this processor will let us introduce state mutations, while keeping the view ignorant of how the processing is accomplished.

TouchState contains enough information for the View to calculate a Path, then draw it to the Canvas.

    private TouchState mState;
    private Path mPath;    //...    @Override
    public boolean onTouchEvent(MotionEvent event) {       
        mTouchProcessor.onTouchEvent(this, event); // track state
        calculatePath(mState);
        invalidate();
        return super.onTouchEvent(event);
    }

    @Override
    protected void onDraw(Canvas canvas) {
        super.onDraw(canvas);
        canvas.drawPath(mPath, mPaint);
    }    /**
     * Use the current touch state values to re-plot the path.
     */
    protected final void calculatePath(TouchState s) {
        mPath.reset();
        mPath.moveTo(s.xCurrent, s.yCurrent);
        mPath.quadTo(
             /* quadratic to a tangent point on the circle */);
        mPath.arcTo(
             /* arc around the circle */);

        mPath.moveTo(s.xCurrent, s.yCurrent);
        mPath.quadTo(
             /* quadratic to a mirror tangent point */);
    }
}

Some details of how to calculate the points on our Path have been omitted for brevity, but if you want to inspect the math you can view the source code. Notably, the Path is calculated as if the pull gesture is dragging exactly along the x axis. Before drawing the Path, the canvas is rotated by the angle between the ACTION_DOWN coordinates and the current ACTION_MOVE.

This stretchy action feels good, but we can do better.

A Few Problems

  • It’s infinitely elastic. It doesn’t resist the pull at all.
  • It’s infinitely tense. Upon releasing your finger it snaps back into position with unbounded velocity and zero inertia.

We can use our touch processor to solve both of these problems.

A resting state

Animate to a Resting State

We’ve established the the View knows how to look at any given TouchState and calculate a new Path to draw. Recall ValueAnimator — let’s use a ValueAnimator to continuously throw updated TouchStates at the View after ACTION_UP.

public class SettleAnimator {

    private ValueAnimator mAnimator;
    private Interpolator mInterpolator;
    private long mAnimationDuration;
    private Path mLineToCenter = new Path();
    private PathMeasure mPathMeasure = new PathMeasure();    //...    /**
     * Mutate the TouchState's current [x,y] position towards
     * the xDown and yDown position over the animation time.
     *
    public void start(final TouchState s, final TouchStateView v) {
        // Use a Path from the current [x,y] to the ACTION_DOWN 
        // point to animate the current [x,y] along that line.
        mLineToCenter.reset();
        mLineToCenter.moveTo(s.xCurrent, s.yCurrent);
        mLineToCenter.lineTo(s.xDown, s.yDown);
        final float[] points = new float[2];
        final float fromDistance = s.distance;
        mAnimator = ValueAnimator.ofFloat(1f, 0f);
        mAnimator.setInterpolator(mInterpolator);
        mAnimator.setDuration(mAnimationDuration)
                .addUpdateListener((a) -> {
                        if (v == null) {
                            cancel();
                            return;
                        }
                        final float fraction =
                               a.getAnimatedFraction();
                        // update the state's distance
                        s.distance = (1 - fraction) * fromDistance;
                        // Find the point that is fraction 
                        // percentage of mLineToCenter's distance.
                        // Store the result in points[]
                       setPointFromPercent(mLineToCenter, 
                           fromDistance, fraction, points);
                        // Mutate the state's current position
                        // to points[]
                        s.xCurrent = points[0];
                        s.yCurrent = points[1];
                        // Now that the state is mutated, tell the 
                        // View to draw it.
                        v.drawTouchState(s);
                    }
                });
        mAnimator.start();
    }
}

By invoking the SettleAnimator, we can send updated TouchStates to the View at any time. How does it look?

Animating to a resting state.

Wow! The View doesn’t have to know anything about interpolating or animation. If it can draw one TouchState it can draw them all. Furthermore, any disjointed state transition is the fault of the processor or animator, not the View. This helps keep the View less concerned with state, and focused on correctly drawing.

The stretchy boundary will snap back into position with inertia, but it’s still infinitely elastic. It has no limit on how far it will stretch. Is it possible to constrain the pull with tension?

Simple Tension by a Constant Factor

A dead-simple way to introduce a tension is by a constant fraction multiple of the pull distance. With a tension factor of 0.5, if you move 100 pixels the view moves 50.

Back inside our Touch Processor,

private float mTensionFactor;
private TouchState mState;//...@Override
public void onTouchEvent(View v, MotionEvent event) {
    mTracker.onTouchEvent(v, event);

    switch (event.getAction()) {
        case MotionEvent.ACTION_UP:
        case MotionEvent.ACTION_CANCEL:
        case MotionEvent.ACTION_DOWN:
            return;
        default:
            break;
    }
    final float deltaX = mState.xCurrent - mState.xDown;
    final float deltaY = mState.yCurrent - mState.yDown;
    // reduce the total change in [x,y] by the tension factor
    final float tensionDeltaX = deltaX * (1 - mTensionFactor);
    final float tensionDeltaY = deltaY * (1 - mTensionFactor);
    mState.xCurrent = tensionDeltaX + mState.xDown;
    mState.yCurrent = tensionDeltaY + mState.yDown;
    mState.distance = Geometry.distance(mState.xDown, mState.yDown, mState.xCurrent, mState.yCurrent);
}

public void setTension(@FloatRange(from=0,to=1) float tension) {
    mTensionFactor = tension;
}

Our stretchy border now resists a pull. This is easy to implement and the effect is somewhat convincing.

However, it resists equally at all distances. Short pulls may be reduced below the circle’s resting radius, and longer pulls will never stop tracking the pointer.

By contrast, real stretchy materials will approach a boundary where it gets harder to keep stretching. Likewise, there may be an inner boundary before which tension is small or non-existent.

Interpolated Tension

Let’s use two boundaries that divide three zones:

  1. An inner zone where there is zero tension.
  2. A middle zone where tension increases with the distance of the pull.
  3. An outer zone where the tension prevents the view from further stretching.

Inside the Touch Processor, we can calculate the pull distance for each of these respective zones.

private int mMinRadius; // no tension inside this radius
private int mMaxRadius; // infinite tension outside this radius
// reusable storage for calculating a point location.
private float[] mCoords = new float[2];
private Path mPath = new Path();
private PathMeasure mPathMeasure = new PathMeasure();@Override
public void onTouchEvent(View v, MotionEvent event) {
    mTracker.onTouchEvent(v, event);

    switch (event.getAction()) {
        case MotionEvent.ACTION_UP:
        case MotionEvent.ACTION_CANCEL:
        case MotionEvent.ACTION_DOWN:
            return;
        default:
            break;
    }
    // instead of using the "real" pull distance, 
    // calculate it based on which zone we're in.
    final float interpolatedDistance =  
                              interpolatedDistance(mState.distance);
    // find the point at the interpolated distance and 
    // store it in the array mCoords[]
    interpolatedCurrent(mState, interpolatedDistance, mCoords);
    mState.xCurrent = mCoords[0];
    mState.yCurrent = mCoords[1];
    mState.distance = Geometry.distance(mState.xDown, mState.yDown, mState.xCurrent, mState.yCurrent);
}private float interpolatedDistance(float realDistance) {
    if (realDistance < mMinRadius) {
        // if we're in the inner zone, don't apply tension.
        return realDistance;
    }

    final float radiusSurplus = realDistance - mMinRadius;
    final float tensionZone = mMaxRadius - mMinRadius;
    final float tensionZoneRequiredPullDistance = 
                       tensionZone * (mTensionFactor + 1);

    if (realDistance >= 
                 (tensionZoneRequiredPullDistance + mMinRadius)) {
        // If this is the outer zone, cap the distance at maximum.
        return mMaxRadius;
    }    // Otherwise, apply a tension factor based on how far we've gone
    final float realProgress = 
                    radiusSurplus / tensionZoneRequiredPullDistance;
    final float interpolatedProgress = 
                       mInterpolator.getInterpolation(realProgress);
    return mMinRadius + (interpolatedProgress * tensionZone);
}// Find the point at {@code distance} from s.xDown and s.yDown
private void interpolatedCurrent(TouchState s, float distance, float[] coords) {
    mPath.reset();
    mPath.moveTo(s.xDown, s.yDown);
    mPath.lineTo(s.xCurrent, s.yCurrent);
    mPathMeasure.setPath(mPath, false);
    mPathMeasure.getPosTan(distance, coords, null);
}

The inner boundary is shown in blue, and the outer boundary is shown in orange. The stretchy border is color blended to indicate the relative amount of applied tension at the current distance. Widening the middle boundary increases the distance over which tension is ramped. If the blue and orange boundaries touch, there is effectively zero ramp-time and the tension goes infinite instantaneously.

By adjusting the inner and outer boundaries and the tension factor inside the middle zone, we can reproduce a wide range of physical elasticity.

This will be really useful if I’m ever asked to implement a stretchy line

But we can use this!

Take that exact same Touch Processor we just used for interpolated tension, and apply it to something else:

This uses the same touch processor
(Source)

From the graphic above you can sense the same forces of tension, elasticity, and boundary, but I’d encourage you to run it on your device and feel the effect for yourself.

The elevation change is a visual cue that triggers as the top card is pulled beyond the max radius: the resting card gently lifts upward while the pulled card descends. Releasing your finger will send the cards back into the resting state, but with their positions reversed. If the threshold radius is never crossed, the top card simply settles back on top.

This represents a translation from a flood of MotionEvents to movements that match familiar physical rules. This is meaningful motion.

We talk the talk.
See how we walk the walk.

Our Work