Learn to Develop Virtual Reality Android Apps from Scratch

This article will guide you on how to develop Virtual Reality Android Apps from scratch, assuming that you are already aware of the basics of Android App development.

Requirements

• Android Studio 1.0 or higher
• Version 19 of Android SDK
• A physical Android device running Android 16(Jelly Bean) or Higher

 

The Manifest File

<manifest ...

<uses-permission android:name="android.permission.NFC" />

<uses-permission android:name="android.permission.READ_EXTERNAL_STORAGE" />

<uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" />

 

<uses-permission android:name="android.permission.VIBRATE" />

...

<uses-sdk android:minSdkVersion="16" android:targetSdkVersion="19"/>

<uses-feature android:glEsVersion="0x00020000" android:required="true" />

<application

...

<activity

android:name=".MainActivity"

android:screenOrientation="landscape">

...

 

<intent-filter>

<action android:name="android.intent.action.MAIN" />

<category android:name="android.intent.category.LAUNCHER" />

<category android:name="com.google.intent.category.CARDBOARD" />

</intent-filter>

</activity>

</application>

</manifest>

 

Explaining the permissions:

• <uses-sdk android:minSdkVersion=”16″/> indicates that the device must be running API Level 16 (Jellybean) or higher.
• <uses-sdk android:targetSdkVersion=”19″/> indicates our app is targetting API Level 19 (KitKat).
• <uses-feature android:glEsVersion=”0x00020000″ android:required=”true” /> indicates that the app uses graphics and so must support OpenGL ES 2.0.
• android:screenOrientation=”landscape” indicates that the activity’s required screen orientation is “landscape.” This is the orientation you must set for VR apps. The view used by the Cardboard SDK, CardboardView, only renders on fullscreen and landscape (landscape, reverseLandscape, sensorLandscape) modes.
• The setting android:configChanges=”orientation|keyboardHidden” is also recommended, but not mandatory.
• android.permission.NFC permission is required by the Cardboard SDK to access Cardboard’s NFC tag.
• android.permission.READ_EXTERNAL_STORAGE and android.permission.WRITE_EXTERNAL_STORAGE. These permissions are required by the Cardboard SDK to pair the user’s phone to their VR viewer.
• android.permission.VIBRATE permission is required by our demo app to make the phone vibrate to inform the user that something has happened.
• The intent-filter and specifically com.google.intent.category.CARDBOARD state that this activity is compatible with Cardboard-like viewers. This category is used by the Cardboard app to list compatible apps installed on the user’s phone.

 

Extend Cardboard Activity:

CardboardActivity is the starting point for coding a cardboard app. CardboardActivity is the base activity that provides easy integration with Cardboard devices. It exposes events to interact with Cardboards and handles many of the details commonly required when creating an activity for VR rendering.

 

Note that CardboardActivity uses sticky immersive mode, in which the system UI is hidden, and the content takes up the whole screen. This is a requirement for a VR app, since CardboardView will only render when the activity is in fullscreen mode.

Android 4.4 (API Level 19) introduces a new SYSTEM_UI_FLAG_IMMERSIVE flag for setSystemUiVisibility() that lets your app go truly “full screen.” This flag, when combined with the SYSTEM_UI_FLAG_HIDE_NAVIGATION and SYSTEM_UI_FLAG_FULLSCREEN flags, hides the navigation and status bars and lets your app capture all touch events on the screen.

 

Define a CardBoardView:

All user interface elements in an Android app are built using views. The Cardboard SDK for Android provides its own view, CardboardView, which is a convenience extension of GLSurfaceView that can be used for VR rendering. CardboardView renders content in stereo. You can see how the demo app defines aCardboardView to in its activity layout xml file in the following way:

<com.google.vrtoolkit.cardboard.CardboardView    
android:id="@+id/cardboard_view    
android:layout_width="fill_parent"    
android:layout_height="fill_parent" />

Then in the main activity class it initializes the CardboardView in the onCreate() method:

**
* Sets the view to our CardboardView and initializes the transformation matrices we will use
* to render our scene. 
* @param savedInstanceState 
*/
@Override
public void onCreate(Bundle savedInstanceState) {    
super.onCreate(savedInstanceState);    
setContentView(R.layout.common_ui);    
CardboardView cardboardView = (CardboardView) findViewById(R.id.cardboard_view);    
// Associate a CardboardView.StereoRenderer with cardboardView.    
cardboardView.setRenderer(this);    
// Associate the cardboardView with this activity.    
setCardboardView(cardboardView);     
// Initialize other objects here
....
}

 

Render the view

Once you get the CardboardView you associate it with a renderer, and then you associate theCardboardView with the activity. Cardboard supports two kinds of renderers, but the quickest way to get started is to use CardboardView.StereoRenderer, which is what the demo app uses.

CardboardView.StereoRenderer includes these key methods:

• onNewFrame(), called every time that app renders.
• onDrawEye(), called for each eye with different eye parameters.

Implementing these is similar to what you would normally do for an OpenGL application. These methods are discussed in more detail in the following sections.

 

Implement onNewFrame

Use the onNewFrame() method to to encode rendering logic before the individual eyes are rendered. Any per-frame operations not specific to a single view should happen here. This is a good place to update your model. In this snippet, the variable mHeadView contains the position of the head. This value needs to be saved to use later to tell if the user is looking at treasure:

/**

* Prepares OpenGL ES before we draw a frame.

* @param headTransform The head transformation in the new frame.

*/

@Override

public void onNewFrame(HeadTransform headTransform) {

...

headTransform.getHeadView(mHeadView, 0);

...

}

 

Implement onDrawEye

Implement onDrawEye() to perform per-eye configuration.

This is the meat of the rendering code, and very similar to building a regular OpenGL ES2 application. The following snippet shows how to get the view transformation matrix, and also the perspective transformation matrix. You need to make sure that you render with low latency. The Eye object contains the transformation and projection matrices for the eye. This is the sequence of events:

• The treasure comes into eye space.
• We apply the projection matrix. This provides the scene rendered for the specified eye.
• The Cardboard SDK applies distortion automatically, to render the final scene.

/**

* Draws a frame for an eye.

*

* @param eye The eye to render. Includes all required transformations.

*/

@Override

public void onDrawEye(Eye eye) {

GLES20.glClear(GLES20.GL_COLOR_BUFFER_BIT | GLES20.GL_DEPTH_BUFFER_BIT);

...

// Apply the eye transformation to the camera.

Matrix.multiplyMM(mView, 0, eye.getEyeView(), 0, mCamera, 0);

 

// Set the position of the light

Matrix.multiplyMV(mLightPosInEyeSpace, 0, mView, 0, LIGHT_POS_IN_WORLD_SPACE, 0);

 

// Build the ModelView and ModelViewProjection matrices

// for calculating cube position and light.

float[] perspective = eye.getPerspective(Z_NEAR, Z_FAR);

Matrix.multiplyMM(mModelView, 0, mView, 0, mModelCube, 0);

Matrix.multiplyMM(mModelViewProjection, 0, perspective, 0, mModelView, 0);

drawCube();

 

// Draw rest of the scene.

...

}

 

Handling inputs

The Cardboard viewer includes a push button which uses a magnet. When you push the magnet, the magnetic field changes and is detected by the magnetometer of your phone. These magnet events are detected by the Cardboard SDK for you.

To provide custom behavior when the user pulls the magnet, overrideCardboardActivity.onCardboardTrigger() in your app’s activity. In the treasure hunt app, if you found the treasure and pull the magnet, you get to keep the treasure:

/**

* Increment the score, hide the object, and give feedback if the user pulls the magnet while

* looking at the object. Otherwise, remind the user what to do.

*/

@Override

public void onCardboardTrigger() {

if (isLookingAtObject()) {

mScore++;

mOverlayView.show3DToast("Found it! Look around for another one.\nScore = " + mScore);

...

} else {

mOverlayView.show3DToast("Look around to find the object!");

}

// Always give user feedback

mVibrator.vibrate(50);

}

 

Start your own project

Now that you’re more familiar with the Cardboard SDK for Android, it’s time to create your own applications.

Whether it’s a new project you start from scratch or an existing one, here’s what you should do.

1)    Download the two JAR files. Click here to download

2)    Copy and paste it into your app/libs folder (Find this in Project view of Android Studio Project Structure)

3)    Right click on the libraries and select “Add as library”

Then, make sure the following lines are present in your project’s app/build.gradle file:

dependencies {    …    compile fileTree(dir: ‘libs’, include: [‘*.jar’])  }

You’re ready to give a kick start to Virtual Reality app development!

 

Are you an Android Developer? Join Truelancer today & Start Freelancing