Tricks to Making a Realtime SurfaceView Actually Perform in Realtime - Maarten Edger

Tricks to Making a Realtime
SurfaceView Actually Perform in
Realtime
Maarten Edgar

Hello, my name is …
Maarten Edgar

What we’ll cover
SurfaceViews:
• Why
• When
• What
• How
• Hard earned lessons

Why use a SurfaceView?
SurfaceView
GL_SurfaceView
TextureView
SurfaceTexture
View

What is a SurfaceView?
A View which gives you access to a
Surface using .getHolder(), which is
drawn on a seperate thread and is
double/triple buffered behind the
scenes.
It cuts holes and displays underneath
the window it is in.

How to use it:
• Setup
• Threads vs Runnables and other
control mechanisms
• Loops
• UI communication
• Tips

Setup
• Activity/View
• SurfaceView and its Thread

Setup: Activity and View
@Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
// set flags as needed
getWindow().setFormat(PixelFormat.RGBA_8888);
this.setVolumeControlStream(AudioManager.STREAM_MUSIC);
setContentView(R.layout.activity_game);
// get handles to the View from XML, and its Thread
mCSurfaceView = (MySurfaceView) findViewById(R.id.surfaceview);
setSurfaceType(View.LAYER_TYPE_SOFTWARE);
mSurfaceViewThread = mSurfaceView.getThread();
createInputObjectPool();

Your SurfaceView class
public class ChiBlastSurfaceView extends SurfaceView implements SurfaceHolder.Callback {
public ChiBlastSurfaceView(Context context) {
super(context);
mSurfaceCreated = false;
touchBool = true;
// register our interest in hearing about changes to our surface
SurfaceHolder holder = getHolder();
holder.addCallback(this);
myHandler = new MyInnerHandler(this);
// create thread only; it's started in surfaceCreated()
thread = new ChiBlastSurfaceViewThread(holder, context, myHandler);
setFocusable(true); // make sure we get key events
}

Your SurfaceView callbacks 1/3
SurfaceHolder.Callback:
@Override
public void surfaceCreated(SurfaceHolder holder) {
// start the thread here so that we don't busy-wait
in run() waiting for the surface to be created
if (mSurfaceCreated == false)
{
createThread(holder);
mSurfaceCreated = true;
touchBool = true;
}
}

@Override
public void surfaceDestroyed(SurfaceHolder holder) {
mSurfaceCreated = false;
cleanupResource();
terminateThread();
}

@Override
public void surfaceChanged(SurfaceHolder holder, int format, int width, int height) {
thread.setSurfaceSize(width, height);
}

Setup: driving the SurfaceView
Runnables, thread and loops, oh my!

Setup: Thread
public class ChiBlastSurfaceViewThread extends Thread {
public ChiBlastSurfaceViewThread(SurfaceHolder surfaceHolder,
Context context, Handler handler) {
// get handles to some important objects
mSurfaceHolder = surfaceHolder;
mSurfaceHolder.setFormat(PixelFormat.RGBA_8888);
mContext = context;
res = context.getResources();
//any other initialization:
ops = new BitmapFactory.Options();
ops.inPurgeable = true;
ops.inDensity = 0;
ops.inDither = false;
ops.inScaled = false;
ops.inPreferredConfig = Bitmap.Config.ARGB_8888;
ops.inJustDecodeBounds = false;
}

@Override
public void run() {
while (mRun) {
Canvas c = null;
try {
// update game state
processInput();
//if (mMode == STATE_SCROLL_MAP)
if (mMode != STATE_PAUSE)
{
updatePhysics(timeDiff);
}
c = mSurfaceHolder.lockCanvas(null);
synchronized (mSurfaceHolder) {
doDraw(c);
}
} finally {
// do this in a finally so that if an exception is thrown
// during the above, we don't leave the Surface in an
// inconsistent state
if (c != null) {
mSurfaceHolder.unlockCanvasAndPost(c);
}
}
}
}
Setup: Thread

The Thread and your Activity
What does this now mean for your
Activity?
or
How do we make this fit into the
Android Lifecycle?

@Override
protected void onPause() {
super.onPause();
// pause game when Activity pauses
mSurfaceView.getThread().pause();
mSurfaceView.terminateThread();
System.gc();
}

@Override
protected void onResume()
{
super.onResume();
if (mSurfaceView.mSurfaceCreated)
{
mSurfaceView.createThread(mSurfaceView.getHolder());
}
mSurfaceView.SetTouch(true);
}

@Override
protected void onRestoreInstanceState(Bundle inState) {
// just have the View's thread load its state from our Bundle
if (mSurfaceView.mSurfaceCreated)
{
mSurfaceView.createThread(mSurfaceView.getHolder());
}
mSurfaceViewThread.restoreState(inState);
}

The main loop
• AFAFP
• Fixed step

@Override
public void run() {
long beginTime; // the time when the cycle begun
long timeDiff; // the time it took for the cycle to execute
int sleepTime; // ms to sleep (<0 if we're behind)
int framesSkipped; // number of frames being skipped
timeDiff = System.currentTimeMillis()+50;
sleepTime = 0;
while (mRun) {
Canvas c = null;
try {
beginTime = System.currentTimeMillis();
framesSkipped = 0; // resetting the frames skipped
// update game state
processInput();
//if (mMode == STATE_SCROLL_MAP)
{
}
c = mSurfaceHolder.lockCanvas(null);
synchronized (mSurfaceHolder) {
doDraw(c);
}
The main loop 1/3

The main loop 2/3
// calculate how long did the cycle take
timeDiff = System.currentTimeMillis() - beginTime;
// calculate sleep time
sleepTime = (int)(FRAME_PERIOD - timeDiff);
if (sleepTime > 0) {
// if sleepTime > 0 we're OK
try {
// send the thread to sleep for a short period
// very useful for battery saving
Thread.sleep(sleepTime);
} catch (InterruptedException e) {}
}
while (sleepTime < 0 && framesSkipped < MAX_FRAME_SKIPS) {
// we need to catch up
// update without rendering
processInput();
{
}
// add frame period to check if in next frame
sleepTime += FRAME_PERIOD;
framesSkipped++;
}

The main loop 3/3
} finally {
// do this in a finally so that if an exception is thrown
// during the above, we don't leave the Surface in an
// inconsistent state
if (c != null) {
mSurfaceHolder.unlockCanvasAndPost(c);
}
}
}
}

UI Communication
Use a Handler:

static class MyInnerHandler extends Handler {
private final WeakReference<ChiBlastSurfaceView> mView;
MyInnerHandler(ChiBlastSurfaceView aView) {
mView = new WeakReference<ChiBlastSurfaceView>(aView);
}
@Override
public void handleMessage(Message m) {
ChiBlastSurfaceView theView = mView.get();
theView.mStatusText.setText(m.getData().getString("text"));
if (m.getData().getInt("viz") == View.VISIBLE)
{
theView.mStatusText.setVisibility(View.VISIBLE);
//mStatusText.setAnimation(displayTextAnim);
//mStatusText.startAnimation(displayTextAnim);
}
else
{
if (m.getData().getInt("viz") == View.INVISIBLE)
{
theView.mStatusText.setVisibility(View.INVISIBLE);
theView.mStatusText.setAnimation(null);
}
else if (m.getData().getInt("viz") == View.GONE)
{
theView.mStatusText.setVisibility(View.GONE);
}
}
theView.mStatusText.invalidate();
}
}

Setup: Cleanup
public void terminateThread ()
{
boolean retry = true;
thread.setRunning(false);
while (retry) {
try
{
thread.join();
retry = false;
}
catch (InterruptedException e)
{
}
//break; //THIS BREAKS IT ON PUSHING HOME
}
//thread = null; //THIS BREAKS IT ON PUSHING HOME
}

Tips
• Input buffer
• Object creation
• Scaling
• Drawing, bitmaps and other dirty
things

Tips: input buffer in SVActivity
private void createInputObjectPool() {
inputObjectPool = new ArrayBlockingQueue<InputObject>(INPUT_QUEUE_SIZE);
for (int i = 0; i < INPUT_QUEUE_SIZE; i++) {
inputObjectPool.add(new InputObject(inputObjectPool));
}
}

public class InputObject {
public static final byte EVENT_TYPE_KEY = 1;
public static final byte EVENT_TYPE_TOUCH = 2;
public static final int ACTION_KEY_DOWN = 1;
public static final int ACTION_KEY_UP = 2;
public static final int ACTION_TOUCH_DOWN = MotionEvent.ACTION_DOWN;
public static final int ACTION_TOUCH_POINTER_DOWN = MotionEvent.ACTION_POINTER_DOWN;
//public static final int ACTION_TOUCH_POINTER_2_DOWN = MotionEvent.ACTION_POINTER_2_DOWN;
public static final int ACTION_TOUCH_MOVE = MotionEvent.ACTION_MOVE;
public static final int ACTION_TOUCH_UP = MotionEvent.ACTION_UP;
public static final int ACTION_TOUCH_POINTER_UP = MotionEvent.ACTION_POINTER_UP;
//public static final int ACTION_TOUCH_POINTER_2_UP = MotionEvent.ACTION_POINTER_2_UP;
public ArrayBlockingQueue<InputObject> pool;
public byte eventType;
public long time;
public int action;
public int keyCode;
public int x;
public int y;
public int x2;
public int y2;
public int pointerID;
public int pointerIndex;
public int pointerIndex2;
InputObject 1/5

InputObject 2/5
public InputObject(ArrayBlockingQueue<InputObject> pool) {
this.pool = pool;
}
public void useEvent(KeyEvent event) {
eventType = EVENT_TYPE_KEY;
int a = event.getAction();
switch (a) {
case KeyEvent.ACTION_DOWN:
action = ACTION_KEY_DOWN;
break;
case KeyEvent.ACTION_UP:
action = ACTION_KEY_UP;
break;
default:
action = 0;
}
time = event.getEventTime();
keyCode = event.getKeyCode();
}

public void useEvent(MotionEvent event) {
eventType = EVENT_TYPE_TOUCH;
int a = event.getAction();
switch (a) {
case MotionEvent.ACTION_DOWN:
action = ACTION_TOUCH_DOWN;
break;
case MotionEvent.ACTION_POINTER_DOWN:
action = ACTION_TOUCH_POINTER_DOWN;
break;
case MotionEvent.ACTION_POINTER_2_DOWN:
action = ACTION_TOUCH_POINTER_DOWN;
break;
case MotionEvent.ACTION_MOVE:
action = ACTION_TOUCH_MOVE;
break;
case MotionEvent.ACTION_UP:
action = ACTION_TOUCH_UP;
break;
case MotionEvent.ACTION_POINTER_UP:
action = ACTION_TOUCH_POINTER_UP;
break;
case MotionEvent.ACTION_POINTER_2_UP:
action = ACTION_TOUCH_POINTER_UP;
break;
default:
action = -1;
}
InputObject 3/5

InputObject 4/5
time = event.getEventTime();
pointerIndex = (event.getAction() &
MotionEvent.ACTION_POINTER_ID_MASK) >>
MotionEvent.ACTION_POINTER_ID_SHIFT;
pointerID = event.getPointerId(pointerIndex);
x = (int) event.getX(pointerIndex);
y = (int) event.getY(pointerIndex);
if (event.getPointerCount() > 1)
{
pointerIndex2 = pointerIndex== 0 ? 1 : 0;
x2 = (int)event.getX(pointerIndex2);
y2 = (int)event.getY(pointerIndex2);
}
}

InputObject 5/5
public void useEventHistory(MotionEvent event, int historyItem) {
eventType = EVENT_TYPE_TOUCH;
action = ACTION_TOUCH_MOVE;
time = event.getHistoricalEventTime(historyItem);
pointerIndex = (event.getAction() &
MotionEvent.ACTION_POINTER_ID_MASK) >>
MotionEvent.ACTION_POINTER_ID_SHIFT;
pointerID = event.getPointerId(pointerIndex);
x = (int) event.getHistoricalX(pointerIndex, historyItem);
y = (int) event.getHistoricalY(pointerIndex, historyItem);
if (event.getPointerCount() > 1)
{
pointerIndex2 = pointerIndex== 0 ? 1 : 0;
x2 = (int) event.getHistoricalX(pointerIndex2, historyItem);
y2 = (int) event.getHistoricalY(pointerIndex2, historyItem);
}
}
public void returnToPool() {
pool.add(this);
}

@Override
public boolean onTouchEvent(MotionEvent event) {
try {
// history first
int hist = event.getHistorySize();
if (hist > 0)
{
// add from oldest to newest
for (int i = 0; i < hist; i++)
{
//for (int i = hist-1; i > -1; i--) {
InputObject input = inputObjectPool.take();
input.useEventHistory(event, i);
mSurfaceViewThread.feedInput(input);
}
}
// current last
InputObject input = inputObjectPool.take();
input.useEvent(event);
mSurfaceViewThread.feedInput(input);
} catch (InterruptedException e) {
}
// don't allow more than 60 motion events per second
try {
Thread.sleep(16);
}
return true;
}
Back to the activity:

public void feedInput(InputObject input) {
synchronized(inputQueueMutex) {
try {
inputQueue.put(input);
//Log.e(TAG, e.getMessage(), e);
}
}
}
private void processInput() {
synchronized(inputQueueMutex) {
ArrayBlockingQueue<InputObject> inputQueue = ChiBlastSurfaceView.inputQueue;
while (!inputQueue.isEmpty()) {
try {
InputObject input = inputQueue.take();
if (input.eventType == InputObject.EVENT_TYPE_KEY) {
//processKeyEvent(input);
} else if (input.eventType == InputObject.EVENT_TYPE_TOUCH) {
processMotionEvent(input);
}
input.returnToPool();
//Log.e(TAG, e.getMessage(), e);
}
}
}
}
And in the SurfaceView.Thread:

Tips: object creation
Just don’t do it.

Tips: object creation
Or do it up front.
No matter how odd that sometimes
may seem.

Tips: scaling
Two types of scaling:
• Realtime whole view SV scaling
only works from Android N
• Fixed scaling (as done in Unreal
Tournament 3)

Tips: scaling
@Override
public void surfaceChanged(SurfaceHolder holder, int format, int width, int height) {
//thread.setSurfaceSize(width, height);
if (mCanvasWidth != width)
{
int scaledWidth = (int)(width*0.75f);
int scaledHeight = (int)(height*0.75f);
if (scaledHeight != height)
{
yRatio = (float)(scaledHeight / (float)height);
xRatio = (float)(scaledWidth / (float)width);
}
holder.setFixedSize(scaledWidth, scaledHeight);
thread.setSurfaceSize(scaledWidth, scaledHeight);
}
}

Tips: drawing, bitmaps and other
dirty things

Tips: drawing, bitmaps and other
dirty things
In SurfaceView.Thread doDraw():
canvas.drawBitmap(mBackgroundImage,
null, fullscreenRect, mPicPaint);

Thank you!
Maarten Edgar
lifeboatsoft@gmail.com
Resources:
https://source.android.com/devices/graphics/architecture.html
https://github.com/google/grafika

Tricks to Making a Realtime SurfaceView Actually Perform in Realtime - Maarten Edger

Recommended

More Related Content

What's hot (19)

Viewers also liked (17)

Similar to Tricks to Making a Realtime SurfaceView Actually Perform in Realtime - Maarten Edger (20)

More from DroidConTLV (20)

Recently uploaded (20)

Tricks to Making a Realtime SurfaceView Actually Perform in Realtime - Maarten Edger