Basic

QuickBlox Video Calling API is built on top of WebRTC. It allows adding real-time video communication features into your app similar to Skype using API easily. The communication is happening between peers representing camera devices. There are two peer types:

• Local peer is a device running the app right now.
• Remote peer is an opponent device.

Establishing real-time video communication between two peers involves 3 phases:

1. Signaling. At this phase, the peers’ local IPs and ports where they can be reached (ICE candidates) are exchanged as well as their media capabilities and call session control messages.
2. Discovery. At this phase, the public IPs and ports at which endpoints can be reached are discovered by STUN/TURN server.
3. Establishing a connection. At this phase, the data are sent directly to each party of the communication process.

Visit our Key Concepts page to get an overall understanding of the most important QuickBlox concepts.

​

Before you begin

1. Register a QuickBlox account. This is a matter of a few minutes and you will be able to use this account to build your apps.
2. Configure QuickBlox SDK for your app. Check out our Setup page for more details.
3. Create a user session to be able to use QuickBlox functionality. See our Authentication page to learn how to do it.
4. Connect to the Chat server to provide a signaling mechanism for Video Calling API. Follow our Chat page to learn about chat connection settings and configuration.

​

Initialize WebRTC

To be able to receive incoming video chat calls, you should initialize QBRTCClient and add WebRTC signaling to it.

// add signalling manager
chatService.getVideoChatWebRTCSignalingManager().addSignalingManagerListener(new QBVideoChatSignalingManagerListener() {
    @Override
    public void signalingCreated(QBSignaling signaling, boolean createdLocally) {
    if (!createdLocally) {
        rtcClient.addSignaling(signaling);
    }
}
});

​

Manage calls

Add SessionCallbacksListener to your QBRTCClient to define session states. Learn more details about the event listener configuration in the Event listener section.

QBRTCClient.getInstance(getApplicationContext()).addSessionCallbacksListener(new QBRTCClientSessionCallbacks() {
    @Override
    public void onReceiveNewSession(QBRTCSession session) {

    }

    @Override
    public void onUserNoActions(QBRTCSession session, Integer integer) {

    }

    @Override
    public void onSessionStartClose(QBRTCSession session) {

    }

    @Override
    public void onUserNotAnswer(QBRTCSession session, Integer integer) {

    }

    @Override
    public void onCallRejectByUser(QBRTCSession session, Integer integer, Map<String, String> map) {

    }

    @Override
    public void onCallAcceptByUser(QBRTCSession session, Integer integer, Map<String, String> map) {

    }

    @Override
    public void onReceiveHangUpFromUser(QBRTCSession session, Integer integer, Map<String, String> map) {

    }

    @Override
    public void onSessionClosed(QBRTCSession session) {

    }
});

You should allow QBRTCClient to process calls. To be sure that your app is ready for calls processing and Activity exists, use the following snippet in the Activity class.

rtcClient.prepareToProcessCalls();

Now the application is ready for processing calls.

​

Local/remote video view

Set up two video chat layouts for remote and local video tracks to be able to show the video.

• A remote video track represents a remote peer video stream from a remote camera app. Specify userId for the remote camera app of the remote peer.
• A local video track represents a local peer video stream from a local camera app. Specify userId for the local camera app of the local peer.

<com.quickblox.videochat.webrtc.view.QBRTCSurfaceView
    android:id="@+id/remote_video_view"
    android:layout_width="match_parent"
    android:layout_height="match_parent"/>

<com.quickblox.videochat.webrtc.view.QBRTCSurfaceView
    android:id="@+id/local_video_view"
    android:layout_width="match_parent"
    android:layout_height="match_parent"/>

QBRTCSurfaceView allows using several views on the screen layout and overlapping each other. This is a good feature for group video calls.

QBRTCSurfaceView is a surface view (it extends org.webrtc.SurfaceViewRenderer class) that renders video track. It has its own lifecycle for rendering. It uses init() method for preparing to render and release() to release resources when the video track does not exist anymore.

QBRTCSurfaceView is automatically initialized after the surface is created (in surfaceCreated() method callback). You can manually initialize QBRTCSurfaceView using EGLContext getting from QBRTCClient. Use this only when Activity is alive and GL resources exist.

QBRTCSurfaceView surfaceView = (QBRTCSurfaceView) findViewById(R.id.remote_video_view);

EglBase eglContext = QBRTCClient.getInstance(getContext()).getEglContext();
surfaceView.init(eglContext.getEglBaseContext(), null);

Method release() should be called when video track is no more valid, for example, when you receive onConnectionClosedForUser() callback from QBRTCSession or when QBRTCSession is going to close. But you should call release() method before Activity is destroyed and while EGLContext is still valid. If you don’t call this method, the GL resources might leak.

Here are the few methods of the QBRTCSurfaceView:

// set if the video stream should be mirrored or not
surfaceView.setMirror(true);

// set how the video will fill the allowed layout area
surfaceView.setScalingType(RendererCommon.ScalingType.SCALE_ASPECT_FIT);

// request to invalidate view when something has changed
surfaceView.requestLayout();

// releases all related GL resources
surfaceView.release();

To render received video track from an opponent, you should have QBRTCSurfaceView, QBRTCVideoTrack and use:

videoTrack.addRenderer(videoView);

To stop rendering video track, you should simply use:

videoTrack.removeRenderer(videoTrack.getRenderer());

​

Initiate a call

To call other users, you should create a call session with the user first using createNewSessionWithOpponents() method. After that, you can start calling using startCall() method.

// create collection of opponents ID
List<Integer> opponents = new ArrayList<>();
for (QBUser user : users) {
    opponents.add(user.getId());
}

// you can set any string key and value in user info
// then retrieve this data from sessions which is returned in callbacks
// and parse them as you wish
Map<String, String> userInfo = new HashMap<>();
userInfo.put("key", "value");

// there are two call types: Audio or Video Call
QBRTCTypes.QBConferenceType conferenceType = QBRTCTypes.QBConferenceType.QB_CONFERENCE_TYPE_AUDIO;
// or
QBRTCTypes.QBConferenceType conferenceType = QBRTCTypes.QBConferenceType.QB_CONFERENCE_TYPE_VIDEO;

// init session
QBRTCSession session = QBRTCClient.getInstance(this).createNewSessionWithOpponents(opponents, conferenceType);

// start call
session.startCall(userInfo);

Now, your opponents will receive a call request callback onReceiveNewSession() via QBRTCClientSessionCallbacks (read above).

@Override
public void onReceiveNewSession(QBRTCSession session) {
    Map<String, String> userInfo = new HashMap<>();
    userInfo.put("key", "value");

    session.acceptCall(userInfo);
    session.rejectCall(userInfo);
}

​

Accept a call

If you accept an incoming call, your opponent receives an onCallAcceptByUser() callback.

@Override
public void onCallAcceptByUser(QBRTCSession session, Integer userId, Map<String, String> userInfo) {

}

​

Reject a call

If you reject an incoming call, your opponent receives an onCallRejectByUser() callback.

@Override
public void onCallRejectByUser(QBRTCSession session, Integer userId, Map<String, String> userInfo) {

}

​

Ignore a call

If you neither accept nor reject an incoming call, the caller will receive an appropriate callback after a specific period of time. You can set interval using QBRTCConfig.setAnswerTimeInterval() method.

@Override
public void onUserNotAnswer(QBRTCSession session, Integer userId) {

}

​

Render video stream to view

For managing video tracks, you should use the QBRTCClientVideoTracksCallbacks interface.

session.addVideoTrackCallbacksListener(this);
session.removeVideoTrackCallbacksListener(this);

new QBRTCClientVideoTracksCallbacks() {
    @Override
    public void onLocalVideoTrackReceive(BaseSession session, QBRTCVideoTrack videoTrack) {

    }

    @Override
    public void onRemoteVideoTrackReceive(BaseSession session, QBRTCVideoTrack videoTrack, Integer userId) {

    }
}

Once you’ve got an access to video track, you can render them to some view in your app UI.

private void fillVideoView(QBRTCSurfaceView videoView, QBRTCVideoTrack videoTrack) {
    // to remove renderer if Video Track already has another one
    videoTrack.cleanUp();

    if (videoView != null) {
        videoTrack.addRenderer(videoView);
        updateVideoView(videoView);
    }
}

private void updateVideoView(SurfaceViewRenderer videoView) {
    RendererCommon.ScalingType scalingType = RendererCommon.ScalingType.SCALE_ASPECT_FILL;
    videoView.setScalingType(scalingType);
    videoView.setMirror(false);
    videoView.requestLayout();
}

​

Obtain audio tracks

For managing audio tracks, you should use the QBRTCClientAudioTracksCallback interface.

session.addAudioTrackCallbacksListener(this);
session.removeAudioTrackCallbacksListener(this);

new QBRTCClientAudioTracksCallback() {
    @Override
    public void onLocalAudioTrackReceive(BaseSession session, QBRTCAudioTrack audioTrack) {

    }

    @Override
    public void onRemoteAudioTrackReceive(BaseSession session, QBRTCAudioTrack audioTrack, Integer userId) {

    }
}

Then you can use these audio tracks to mute/unmute audio.

​

End a call

To end a call, use thehangUp()method.

Map<String, String> userInfo = new HashMap<>();
userInfo.put("key", "value");

session.hangUp(userInfo);

After this, the call session is going to close and your opponent will receive an onReceiveHangUpFromUser() callback.

@Override
public void onReceiveHangUpFromUser(QBRTCSession session, Integer userId, Map<String, String> userInfo) {

}

​

Release resource

When you don’t want to receive and process video calls, for example, when a user is logged out, you have to destroy QBRTCClient. Call the destroy() method to unregister from receiving any video chat events and close existing signaling channels.

QBRTCClient.getInstance(this).destroy();

​

Event listener

To process events such as incoming call, call reject, hang up, etc. you need to set up the event listener. The event listener processes various events that happen with call session or peer connection in your app.

Using the callbacks provided by the event listener, you can implement and execute the event-related processing code. For example, the onCallAcceptByUser callback of QBRTCClient is received when a call has been accepted. This callback receives information about the call session, user ID who accepted the call, and additional key-value data about the user.

QuickBlox Android SDK persistently interacts with the server via XMPP connection that works as a signaling transport for establishing a call between two or more peers. It receives the callbacks of the asynchronous events which happen with the call and peer connection. This allows you to track these events and build your own video calling features around them.

To track call session events, you should use SessionCallbacksListener. The inherited event listeners for a call session are QBRTCClientSessionCallback and QBRTCSessionConnectionCallbacks.

QBRTCClientSessionCallback

The supported call session event callbacks of QBRTCClientSessionCallback along with their parameters as well as shows how to add the listener.

The following code lists all supported event callbacks for a call session along with their parameters as well as shows how to add the listener.

QBRTCClient.getInstance(this).addSessionCallbacksListener(new QBRTCClientSessionCallbacks() {
   @Override
   public void onReceiveNewSession(QBRTCSession session) {

   }

   @Override
   public void onUserNoActions(QBRTCSession session, Integer userId) {

   }

   @Override
   public void onSessionStartClose(QBRTCSession session) {

   }

   @Override
   public void onUserNotAnswer(QBRTCSession session, Integer userId) {

   }

   @Override
   public void onCallRejectByUser(QBRTCSession session, Integer userID, Map<String, String> userInfo) {

   }

   @Override
   public void onCallAcceptByUser(QBRTCSession session, Integer userID, Map<String, String> userInfo) {

   }

   @Override
   public void onReceiveHangUpFromUser(QBRTCSession session, Integer userID, Map<String, String> userInfo) {

   }

   @Override
   public void onSessionClosed(QBRTCSession session) {

   }
});

Go to the Resources section to see a sequence diagram for a regular call workflow.

QBRTCSessionConnectionCallbacks

The supported call session event callbacks of QBRTCSessionConnectionCallbacks listener are listed in the table below.

The following code lists all supported event callbacks for a call session along with their parameters as well as shows how to add the listener.

session.addSessionCallbacksListener(new QBRTCSessionConnectionCallbacks() {
   @Override
   public void onStartConnectToUser(QBRTCSession session, Integer userId) {

   }

   @Override
   public void onDisconnectedTimeoutFromUser(QBRTCSession session, Integer userId) {

   }

   @Override
   public void onConnectionFailedWithUser(QBRTCSession session, Integer userId) {

   }

   @Override
   public void onStateChanged(QBRTCSession session, BaseSession.QBRTCSessionState sessionState) {

   }

   @Override
   public void onConnectedToUser(QBRTCSession session, Integer userId) {

   }

   @Override
   public void onDisconnectedFromUser(QBRTCSession session, Integer userId) {

   }

   @Override
   public void onConnectionClosedForUser(QBRTCSession session, Integer userId) {

   }
});

Go to the Resources section to see a sequence diagram for a regular call workflow.

​

Call session states

The following table lists all supported call session connection states.

​

Peer connection states

To get peer connection state for a particular peer, use the code snippet below.

QBRTCSession session = WebRtcSessionManager.getInstance(getApplicationContext()).getCurrentSession();
QBRTCTypes.QBRTCConnectionState peerConnectionState = session.getPeerConnection(userId).getState();

The following table lists all supported peer connection states.

​

Resources

A regular call workflow.