Video Calling

QuickBlox Video Calling API is built on top of WebRTC. It allows adding real-time video communication features into your app similar to Skype using API easily. The communication is happening between peers representing camera devices. There are two peer types:

Local peer is a device running the app right now.
Remote peer is an opponent device.

Establishing real-time video communication between two peers involves 3 phases:

Signaling. At this phase, the peers’ local IPs and ports where they can be reached (ICE candidates) are exchanged as well as their media capabilities and call session control messages.
Discovery. At this phase, the public IPs and ports at which endpoints can be reached are discovered by STUN/TURN server.
Establishing a connection. At this phase, the data are sent directly to each party of the communication process.

🚧
In order to start using Video Calling Module, you need to connect to QuickBlox Chat first. The signaling in the QuickBox WebRTC module is implemented over the XMPP protocol using QuickBlox Chat Module. So QuickBlox Chat Module is used as a signaling transport for Video Calling API.

📘
Please use this WebRTC Video Calling to make the Group Calls with 4 or fewer users. Because of Mesh architecture we use for multi-point where every participant sends and receives its media to all other participants, the current solution supports group calls with up to 4 people.

Visit our Key Concepts page to get an overall understanding of the most important QuickBlox concepts.

Before you begin

Register a QuickBlox account. This is a matter of a few minutes and you will be able to use this account to build your apps.
Configure QuickBlox SDK for your app. Check out our Setup page for more details.
Create a user session to be able to use QuickBlox functionality. See our Authentication page to learn how to do it.
Connect to the Chat server to provide a signaling mechanism for Video Calling API. Follow our Chat page to learn about chat connection settings and configuration.

Initialize WebRTC

WebRTC module allows to process calls. When a call is initiated or received, a call session is created. If the module is not initialized, it will not be able to create a call session and process calls consequently. To initialize WebRTC module call init() method.

try {
  await QB.webrtc.init();
} on PlatformException catch (e) {
  // Some error occurred, look at the exception message for more details
}

📘
If you miss calling init() method, a Exception is returned: The call service is not connected.

Manage calls

🚧
Each WebRTC session is assigned a unique session identifier (sessionId). You can get the sessionId from the session value returned from call(), accept(), reject(), hangUp() methods or from any event emitted by QB.webrtc module. See a full list of callbacks here.

To process events such as incoming call, you should subscribe to events first and assign an event handler. When you have subscribed to the event, you receive StreamSubscription that you should unsubscribe when you need in dispose() method. Learn more details about the event handler configuration in the Event handler section.

// WebRTC Events
// QBRTCEventTypes.CALL
// QBRTCEventTypes.CALL_END
// QBRTCEventTypes.NOT_ANSWER
// QBRTCEventTypes.REJECT
// QBRTCEventTypes.ACCEPT
// QBRTCEventTypes.HANG_UP
// QBRTCEventTypes.RECEIVED_VIDEO_TRACK

StreamSubscription? _callSubscription;

...

@override
  void dispose() {
    if(_callSubscription != null) {
      _callSubscription!.cancel();
      _callSubscription = null;
    }
  }

...

String event = QBRTCEventTypes.CALL;

try {
  _callSubscription = await QB.webrtc.subscribeRTCEvent(event, (data) {
    Map<dynamic, dynamic> payloadMap = Map<dynamic, dynamic>.from(data["payload"]);
    Map<dynamic, dynamic> sessionMap = Map<dynamic, dynamic>.from(payloadMap["session"]);

    String sessionId = sessionMap["id"];
    int initiatorId = sessionMap["initiatorId"];
    int callType = sessionMap["type"];
  } on PlatformException catch (e) {
  // Some error occurred, look at the exception message for more details  
}

📘
The example above shows assigning one handler for all module events but you can assign separate handlers for each event.

Initiate a call

To call users, you should create a call session and start calling using call() method.

// Session types
// QBRTCSessionTypes.VIDEO
// QBRTCSessionTypes.AUDIO

List<int> opponentIds = [22345, 23521];
int sessionType = QBRTCSessionTypes.VIDEO;

try {
  QBRTCSession? session = await QB.webrtc.call(opponentIds, sessionType);
} on PlatformException catch (e) {
  // Some error occurred, look at the exception message for more details 
}

Argument	Description	Description
opponentsIds	yes	A list of opponents IDs.
sessionType	yes	Call type: `QB.webrtc.RTC_SESSION_TYPE.VIDEO`, `QB.webrtc.RTC_SESSION_TYPE.AUDIO`.

📘
Note
After this, your opponents will receive an event QB.webrtc.EVENT_TYPE.CALL.

Accept a call

To accept a call request, call accept() method and pass sessionId to tell SDK which call session to accept.

String sessionId = "5d4175afa0eb4715cae5b63f";
Map<String, Object> userInfo = new Map();

try {
  QBRTCSession? session = await QB.webrtc.accept(sessionId, userInfo: userInfo);
} on PlatformException catch (e) {
  // Some error occurred, look at the exception message for more details   
}

Argument	Required	Description
sessionId	yes	Call session identifier.
userInfo	no	Custom user data.

Reject a call

To reject a call request, use reject() method and pass sessionId parameter to tell SDK which call session to reject.

String sessionId = "5d4175afa0eb4715cae5b63f";
Map<String, Object> userInfo = new Map();

try {
  QBRTCSession? session = await QB.webrtc.reject(sessionId, userInfo: userInfo);
} on PlatformException catch (e) {
  // Some error occurred, look at the exception message for more details 
}

Argument	Required	Description
sessionId	yes	Call session identifier.
userInfo	no	Custom user data.

End a call

To end a call, use hangUp() method and pass sessionId parameter to tell SDK which call session to end.

String sessionId = "5d4175afa0eb4715cae5b63f";
Map<String, Object> userInfo = new Map();

try {
  QBRTCSession? session = await QB.webrtc.hangUp(sessionId, userInfo: userInfo);
} on PlatformException catch (e) {
  // Some error occurred, look at the exception message for more details 
}

Argument	Required	Description
sessionId	yes	Call session identifier.
userInfo	no	Custom user data.

Release resource

When you don't want to receive and process video calls, for example, when a user is logged out, you have to release QB.webrtc module. Call release() method that allows to unregister QB.webrtc module from receiving any video chat events and closes existing signaling channels.

try {
  await QB.webrtc.release();
} on PlatformException catch (e) {
  // Some error occurred, look at the exception message for more details 
}

📘
release() method should be called when a video track is no more valid. If you don't call this method, you will get a memory leak.

Local/remote video view

Set up two RTCVideoView for remote and local video tracks to be able to show the video.

A remote video track represents a remote peer video stream from a remote camera app. Specify initial value to RTCVideoView - RTCVideoViewController for the remote camera app of the remote peer.
A local video track represents a local peer video stream from a local camera app. Specify initial value to RTCVideoView is RTCVideoViewController for the local camera app of the local peer.

Subscribe to QBRTCEventTypes.RECEIVED_VIDEO_TRACK manually. Thus, once the SDK receives data that a remote video track was received, it creates the event of RECEIVED_VIDEO_TRACK type with userId and sessionId properties.

//Widget

RTCVideoViewController? _localVideoViewController;
RTCVideoViewController? _remoteVideoViewController;
...
child: Container(
  margin: new EdgeInsets.fromLTRB(10.0, 10.0, 10.0, 10.0),
  width: 160.0,
  height: 160.0,
  child: RTCVideoView(
    onVideoViewCreated: _onLocalVideoViewCreated,
  ),
  decoration: BoxDecoration(color: Colors.black54),
)
 
...

child: Container(
  margin: new EdgeInsets.fromLTRB(10.0, 10.0, 10.0, 10.0),
  width: 160.0,
  height: 160.0,
  child: RTCVideoView(
    onVideoViewCreated: _onRemoteVideoViewCreated,
  ),
  decoration: BoxDecoration(color: Colors.black54),
)
  
...

void _onLocalVideoViewCreated(RTCVideoViewController controller) {
  _localVideoViewController = controller;
}

void _onRemoteVideoViewCreated(RTCVideoViewController controller) {
  _remoteVideoViewController = controller;
}

After this, invoke method play() and pass sessionId and userId parameters to it. If userId matches with the one in properties, the video starts playing.

String sessionId = "5d4175afa0eb4715cae5b63f";
int userId = 7832;
int opponentId = 3928;

Future<void> play() async {
  _localVideoViewController!.play(sessionId, userId);
  _remoteVideoViewController!.play(sessionId, opponentId);
}

Argument	Required	Description
sessionId	yes	Call session identifier.
userId	yes	The ID of the local peer.
opponentId	yes	The ID of the remote peer.

Event handler

To process events such as incoming call, call reject, hang up, etc. you need to set up the event handler. The event handler processes various events that happen with the call session or peer connection in your app. The events are emitted by the WebRTC module of QuickBlox Flutter SDK.

📘
Once the WebRTC module is initialized, it can start emitting events, so you can assign event handler even before module initialization.

Using the callbacks provided by the event handler, you can implement and execute the event-related processing code. For example, the accept() method is called when your call has been accepted by the user. This method receives information about the call session and additional key-value data about the user.

QuickBlox Flutter SDK persistently interacts with the server via XMPP connection that works as a signaling transport for establishing a call between two or more peers. It receives the callbacks of the asynchronous events which happen with the call and peer connection. This allows you to track these events and build your own video calling features around them.

Call session events

The table below lists all supported call session event types.

Event type	Description
QBRTCEventTypes.CALL	An incoming call event has been received by the peer after the call session has been initiated.
QBRTCEventTypes.ACCEPT	An incoming call has been accepted by the peer.
QBRTCEventTypes.REJECT	An incoming call has been rejected by the remote peer without accepting the call.
QBRTCEventTypes.HANG_UP	An accepted call has been ended by the peer by pressing the hang-up button.
QBRTCEventTypes. RECEIVED_VIDEO_TRACK	A remote video track has been received by the remote peer.
QBRTCEventTypes. PEER_CONNECTION_STATE_CHANGED	A peer connection state has been changed. View all available peer connection states in the Peer connection states section.
QBRTCEventTypes.NOT_ANSWER	No answer received from the remote peer within the timer expiration period.
QBRTCEventTypes.CALL_END	An accepted call has been ended. A call session was closed.

To track call session events, you should add the event handler.

//QBRTCEventTypes.CALL
String event = QBRTCEventTypes.CALL;

try {
  _callSubscription = await QB.webrtc.subscribeRTCEvent(event, (data) {
    Map<dynamic, dynamic> payloadMap = Map<dynamic, dynamic>.from(data["payload"]);
    Map<dynamic, dynamic> sessionMap = Map<dynamic, dynamic>.from(payloadMap["session"]);
    
  	String sessionId = sessionMap["id"];
    int initiatorId = sessionMap["initiatorId"];
    int callType = sessionMap["type"];
} on PlatformException catch (e) {
  // Some error occurred, look at the exception message for more details
}

//QBRTCEventTypes.CALL_END
String event = QBRTCEventTypes.CALL_END;

try {
  _callEndSubscription = await QB.webrtc.subscribeRTCEvent(event, (data) {
    Map<dynamic, dynamic> payloadMap = Map<dynamic, dynamic>.from(data["payload"]);
    Map<dynamic, dynamic> sessionMap = Map<dynamic, dynamic>.from(payloadMap["session"]);
    
  	String sessionId = sessionMap["id"];
 } on PlatformException catch (e) {
   // Some error occurred, look at the exception message for more details
}

//QBRTCEventTypes.REJECT
String event = QBRTCEventTypes.REJECT;

try {
  _rejectSubscription = await QB.webrtc.subscribeRTCEvent(event, (data) {
    int userId = data["payload"]["userId"];
   });
} on PlatformException catch (e) {
  // Some error occurred, look at the exception message for more details
}

//QBRTCEventTypes.ACCEPT
String event = QBRTCEventTypes.ACCEPT;

try {
  _acceptSubscription = await QB.webrtc.subscribeRTCEvent(event, (data) {
    int userId = data["payload"]["userId"];
  });
} on PlatformException catch (e) {
  // Some error occurred, look at the exception message for more details
}

//QBRTCEventTypes.HANG_UP
String event = QBRTCEventTypes.HANG_UP;

try {
  _hangUpSubscription = await QB.webrtc.subscribeRTCEvent(event, (data) {
    int userId = data["payload"]["userId"];
  });
} on PlatformException catch (e) {
  // Some error occurred, look at the exception message for more details
}

//QBRTCEventTypes.RECEIVED_VIDEO_TRACK
String event = QBRTCEventTypes.RECEIVED_VIDEO_TRACK;

try {
  _videoTrackSubscription = await QB.webrtc.subscribeRTCEvent(event, (data) {
    Map<dynamic, dynamic> payloadMap = Map<dynamic, dynamic>.from(data["payload"]);
    int opponentId = payloadMap["userId"];
  });
} on PlatformException catch (e) {
  // Some error occurred, look at the exception message for more details
}

//QBRTCEventTypes.NOT_ANSWER
String event = QBRTCEventTypes.NOT_ANSWER;

try {
  _notAnswerSubscription = await QB.webrtc.subscribeRTCEvent(event, (data) {
    int userId = data["payload"]["userId"];
  });
} on PlatformException catch (e) {
  // Some error occurred, look at the exception message for more details
}

//QBRTCEventTypes.PEER_CONNECTION_STATE_CHANGED
try {
  _peerConnectionSubscription = await QB.webrtc.subscribeRTCEvent(
    QBRTCEventTypes.PEER_CONNECTION_STATE_CHANGED, (data) {
      int state = data["payload"]["state"];
      //QBRTCPeerConnectionStates.NEW = 0
      //QBRTCPeerConnectionStates.CONNECTED = 1
      //QBRTCPeerConnectionStates.FAILED = 2
      //QBRTCPeerConnectionStates.DISCONNECTED = 3
      //QBRTCPeerConnectionStates.CLOSED = 4
  });
} on PlatformException catch (e) {
  // Some error occurred, look at the exception message for more details
}

Each event emitted by WebRTC is an Object type having the following properties:

Argument	Required	Description
type	yes	The name of the event you have subscribed to.
payload	yes	Available if the event transmits the data. Almost all events contain session (to identify in which session this event occurred) and userId (to indicate initiator of the event) properties.

Go to the Resources section to see a sequence diagram for a regular call workflow.

Peer connection state

The peer connection state can change. To monitor the states of your peer connections (users), you need to add an event handler for QB.webrtc.EVENT_TYPE.PEER_CONNECTION_STATE_CHANGED event type.

//Peer connection states
//QBRTCPeerConnectionStates.NEW
//QBRTCPeerConnectionStates.CONNECTED
//QBRTCPeerConnectionStates.FAILED
//QBRTCPeerConnectionStates.DISCONNECTED
//QBRTCPeerConnectionStates.CLOSED

String event = QBRTCEventTypes.PEER_CONNECTION_STATE_CHANGED;

try {
  _peerConnectionSubscription = await QB.webrtc.subscribeRTCEvent(event, (data) {
    int state = data["payload"]["state"];
    String userId = map["userId"];
    int state = map["state"];
  });
} on PlatformException catch (e) {
  // Some error occurred, look at the exception message for more details 
}

The following table lists all supported peer connection states.

Connection state	Description
QBRTCPeerConnectionStates.NEW	Gathering information to establish connection.
QBRTCPeerConnectionStates.CONNECTED	A peer is connected to the call session.
QBRTCPeerConnectionStates.FAILED	A peer failed to join the call session.
QBRTCPeerConnectionStates.DISCONNECTED	A peer is disconnected from the call session.
QBRTCPeerConnectionStates.CLOSED	A call session is closed by the peer.

Resources

A regular call workflow.

🚧

📘

Before you begin

Initialize WebRTC

📘

Manage calls

🚧

📘

Initiate a call

📘Note

Accept a call

Reject a call

End a call

Release resource

📘

Local/remote video view

Event handler

📘

Call session events

Peer connection state

Resources

📘
Note