Symbl.ai

This guide is provided by Symbl.ai. Agora is planning a documentation upgrade program for all extensions on the marketplace. Please stay tuned.

This extension allows you to embed real-time conversation intelligence into your mobile application providing contextual analysis of conversation data along with speech recognition without any upfront data training requirements.

Prerequisites

Agora Credentials: Make sure you have the Agora credentials and required information (App ID, Channel Name, Temp Token and Meeting ID). This information can be retrieved from the Agora Console.

Learn more about the Agora required credentials in Secure authenticatino..

Symbl API Credentials: You will also need the Symbl API credentials which are available on the Symbl Conversation Intelligence Extensions Marketplace product details page. Navigate to the Agora Extensions Marketplace and click on the Symbl card.

Activate the Symbl Conversation Intelligence Extension.

After activating the Symbl Conversation Intelligence Extension, click the View button under the Credentials column to retrieve the required Symbl credentials (App ID and App Secret).

Android Mobile Application: This guide assumes you have an Android mobile application with the Agora Video SDK enabled. Follow the steps Get Started with Extensions Marketplace. to set it up.

Integrate Symbl Conversation Intelligence

This section walks you through the steps necessary to set up the Symbl Conversation Intelligence extension in your mobile application.

Add the following information into your build.gradle module file:

_3implementation 'com.squareup.okhttp3:okhttp:3.10.0' _3implementation 'org.java-websocket:Java-WebSocket:1.5.1' _3implementation 'ai.symbl:android.extension:0.0.2'
Implement the interface io.agora.rtc2.IMediaExtensionObserver

_1public class MainActivity extends AppCompatActivity implements io.agora.rtc2.IMediaExtensionObserver {
Add the following method to set all the necessary information to initialize the Symbl configuration. You can find description for the parameters used in the table below:

_62private void setSymblPluginConfigs(JSONObject pluginParams)throws JSONException{ _62 try{ _62 // Symbl main extension config objects _62 SymblPluginConfig symblParams=new SymblPluginConfig(); _62 _62 // Set the Symbl API Configuration _62 ApiConfig apiConfig=new ApiConfig(); _62 apiConfig.setAppId("<symbl_app_id>"); _62 apiConfig.setAppSecret("<symbl_app_secret>"); _62 apiConfig.setTokenApi("https://api.symbl.ai/oauth2/token:generate"); _62 apiConfig.setSymblPlatformUrl("api-agora-1.symbl.ai"); _62 symblParams.setApiConfig(apiConfig); _62 _62 // Set the Symbl Confidence Level and Language Code _62 RealtimeStartRequest realtimeFlowInitRequest=new RealtimeStartRequest(); _62 RealtimeAPIConfig realtimeAPIConfig=new RealtimeAPIConfig(); _62 realtimeAPIConfig.setConfidenceThreshold(Double.parseDouble("<symbl_confidence_threshold>")); _62 realtimeAPIConfig.setLanguageCode("en-US"); _62 _62 // Set the Speaker information _62 Speaker speaker=new Speaker(); _62 speaker.setName("<symbl_meeting_user_Name>"); _62 speaker.setUserId("<symbl_meeting_UserId>"); _62 realtimeFlowInitRequest.setSpeaker(speaker); _62 _62 // Set the meeting encoding and speaker sample rate hertz _62 SpeechRecognition speechRecognition=new SpeechRecognition(); _62 speechRecognition.setEncoding("<symbl_meeting_encoding>")); _62 speechRecognition.setSampleRateHertz(Double.parseDouble("<symbl_meeting_sampleRateHertz>")); _62 realtimeAPIConfig.setSpeechRecognition(speechRecognition); _62 _62 // Set the redaction content values _62 Redaction redaction=new Redaction(); _62 redaction.setIdentifyContent(true); _62 redaction.setRedactContent(true); _62 redaction.setRedactionString("*****"); _62 realtimeAPIConfig.setRedaction(redaction); _62 _62 // Set the Tracker (custom business intent) information _62 realtimeFlowInitRequest.setConfig(realtimeAPIConfig); _62 Tracker tracker1=new Tracker(); _62 tracker1.setName("Budget"); _62 List<String> vocabulary=new ArrayList<> (); _62 vocabulary.add("budget"); _62 vocabulary.add("budget decision"); _62 tracker1.setVocabulary(vocabulary); _62 List<Tracker> trackerList=new ArrayList<> (); _62 trackerList.add(tracker1); _62 _62 // Set the Symbl conversation parameters _62 realtimeFlowInitRequest.setTrackers(trackerList); _62 realtimeFlowInitRequest.setType("start_request"); _62 realtimeFlowInitRequest.setId("<symbl_unique_meetingId>"); _62 realtimeFlowInitRequest.setSentiments(true); _62 realtimeFlowInitRequest.setInsightTypes(Arrays.asList("action_item","question","follow_up")); _62 symblParams.setRealtimeStartRequest(realtimeFlowInitRequest); _62 Gson gson=new Gson(); _62 pluginParams.put("inputRequest",gson.toJson(symblParams)); _62 }catch(Exception ex){ _62 Log.e(TAG,"ERROR while setting Symbl extension configuration"); _62 } _62}

The following table lists the parameters and their descriptions used in the sample code above:

Parameter	Description
`symbl_meeting_UserId`	Used to identify the user in the real-time meeting.
`symbl_meeting_user_Name`	The name of the user attending the real-time meeting.
`symbl_unique_meetingId`	Unique identifier for the meeting.
`symbl_platform_url`	The dedicated URL for the Symbl platform. Use `api-agora-1.symbl.ai`.
`symbl_app_id`	The Symbl App ID.
`symbl_app_secret`	The Symbl App Secret.
`symbl_meeting_language_code`	The language code. Currently, `en-US` (English US) is the only language supported.
`symbl_token_api`	The URL for secure token generation.
`symbl_meeting_encoding`	The audio encoding in which the audio will be sent over the WebSocket connection.
`symbl_meeting_sampleRateHertz`	The rate of the incoming audio stream.
`symbl_confidence_threshold`	The minimum confidence score that you can set for an API to consider it as valid insight. The minimum confidence score should be in the range >=0.5 to <=1 (greater than or equal to 0.5 and less than or equal to 1.0.). Default value is 0.5.

You would have to call the method above during the initialization of the Agora Engine in your application.

On your onEvent() method, you should receive all the transcription information and the analyzed conversation events where you can parse the JSON responses.

For a basic Speech-to-Text (recognition result) type of response, you should expect the following payload:

{
    "type": "recognition_result",
    "isFinal": true,
    "payload": {
        "raw": {
            "alternatives": [{
                "words": [{
                    "word": "Hello",
                    "startTime": {
                        "seconds": "3",
                        "nanos": "800000000"
                    },
                    "endTime": {
                        "seconds": "4",
                        "nanos": "200000000"
                    }
                }, {
                    "word": "world.",
                    "startTime": {
                        "seconds": "4",
                        "nanos": "200000000"
                    },
                    "endTime": {
                        "seconds": "4",
                        "nanos": "800000000"
                    }
                }],
                "transcript": "Hello world.",
                "confidence": 0.9128385782241821
            }]
        }
    },
    "punctuated": {
        "transcript": "Hello world"
    },
    "user": {
        "userId": "emailAddress",
        "name": "John Doe",
        "id": "23681108-355b-4fc3-9d94-ed47dd39fa56"
    }
}

For a Topic type of response, you should expect the following payload:

[
    {
        "id":"e69a5556-6729-XXXX-XXXX-2aee2deabb1b",
        "messageReferences":[
            {
                "id":"0df44422-0248-47e9-8814-e87f63404f2c",
                "relation":"text instance"
            }
        ],
        "phrases":"auto insurance",
        "rootWords":[
            {
                "text":"auto"
            }
        ],
        "score":0.9,
        "type":"topic"
    }
]

For a Tracker type of response, you should expect the following payload:

{
    "type":"tracker_response",
    "trackers":[
        {
            "name":"budget",
            "matches":[
                {
                    "type":"vocabulary",
                    "value":"budget",
                    "messageRefs":[
                        {
                            "id":"0f3275d7-d534-48e2-aee2-685bd3167f4b",
                            "text":"I would like to set a budget for this month.",
                            "offset":22
                        }
                    ],
                    "insightRefs":[
                    ]
                }
            ]
        }
    ],
    "sequenceNumber":0
}

Given below is a complete sample of MainActivity.java class which includes the Agora RTC and Symbl settings.

package agoramarketplace.symblai.cai;
import androidx.annotation.NonNull;
import androidx.annotation.RequiresApi;
import androidx.appcompat.app.AppCompatActivity;
import androidx.core.app.ActivityCompat;
import androidx.core.content.ContextCompat;
import android.Manifest;
import android.content.Context;
import android.content.pm.PackageManager;
import android.os.Build;
import android.os.Bundle;
import android.util.Log;
import android.view.SurfaceView;
import android.view.View;
import android.view.inputmethod.InputMethodManager;
import android.widget.Button;
import android.widget.EditText;
import android.widget.FrameLayout;
import android.widget.TextView;
import com.google.gson.Gson;
import ai.symbl.android.extension.model.request.ApiConfig;
import ai.symbl.android.extension.model.request.RealtimeAPIConfig;
import ai.symbl.android.extension.model.request.RealtimeStartRequest;
import ai.symbl.android.extension.model.request.Redaction;
import ai.symbl.android.extension.model.request.Speaker;
import ai.symbl.android.extension.model.request.SpeechRecognition;
import ai.symbl.android.extension.model.request.SymblPluginConfig;
import ai.symbl.android.extension.model.request.Tracker;
import ai.symbl.android.extension.model.response.SymblResponse;
import org.json.JSONException;
import org.json.JSONObject;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;
import java.util.UUID;
import ai.symbl.android.extension.ExtensionManager;
import ai.symbl.android.extension.SymblAIFilterManager;
import io.agora.rtc2.Constants;
import io.agora.rtc2.IRtcEngineEventHandler;
import io.agora.rtc2.RtcEngine;
import io.agora.rtc2.RtcEngineConfig;
import io.agora.rtc2.UserInfo;
import io.agora.rtc2.video.VideoCanvas;
import io.agora.rtc2.video.VideoEncoderConfiguration;
public class MainActivity extends AppCompatActivity implements io.agora.rtc2.IMediaExtensionObserver {
    private final static String TAG = "Agora_SymblTag java :";
    private static final int PERMISSION_REQ_ID = 22;
    // Set Agora platform configuration
    private final static String AGORA_CUSTOMER_CHANNEL_NAME = "Demo";
    private static final String AGORA_CUSTOMER_APP_ID = "XXXXXXXXXX";
    public static final String TOKEN_VALUE = "XXXXXXXXXX";
    public static final String AGORA_MEETING_ID = "Sample Meeting";
    // Set Symbl configuration in the code or optionally in "strings.xml"
    public static final String symbl_meeting_UserId = "user@mail.com";
    public static final String symbl_meeting_user_Name = "userName";
    private FrameLayout localVideoContainer;
    private FrameLayout remoteVideoContainer;
    private RtcEngine mRtcEngine;
    private SurfaceView mRemoteView;
    private TextView infoTextView;
    private Button wmButton;
    private EditText et_watermark;
    private static final String[] REQUESTED_PERMISSIONS = {
        Manifest.permission.RECORD_AUDIO,
        Manifest.permission.CAMERA
    };
    @RequiresApi(api = Build.VERSION_CODES.M)
    @Override
    protected void onCreate(Bundle savedInstanceState) {
        Log.d(TAG, "onCreate");
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_main);
        initUI();
        checkPermission();
        infoTextView.setText("Symbl Sample Conversation Application");
        wmButton.setEnabled(true);
    }
    @Override
    protected void onDestroy() {
        Log.d(TAG, "onDestroy");
        super.onDestroy();
        mRtcEngine.leaveChannel();
        mRtcEngine.destroy();
    }
    @RequiresApi(api = Build.VERSION_CODES.M)
    private void checkPermission() {
        Log.d(TAG, "checkPermission");
        if (checkSelfPermission(REQUESTED_PERMISSIONS[0], PERMISSION_REQ_ID) &&
            checkSelfPermission(REQUESTED_PERMISSIONS[1], PERMISSION_REQ_ID)) {
            initAgoraEngine();
        }
    }
    @Override
    public void onRequestPermissionsResult(int requestCode, @NonNull String[] permissions, @NonNull int[] grantResults) {
        super.onRequestPermissionsResult(requestCode, permissions, grantResults);
        if (requestCode == PERMISSION_REQ_ID && grantResults.length > 0 &&
            grantResults[0] == PackageManager.PERMISSION_GRANTED) {
            initAgoraEngine();
        }
    }
    private void initUI() {
        localVideoContainer = findViewById(R.id.view_container);
        remoteVideoContainer = findViewById(R.id.remote_video_view_container);
        infoTextView = findViewById(R.id.infoTextView);
        et_watermark = findViewById(R.id.et_watermark);
        wmButton = findViewById(R.id.enable_button);
        wmButton.setOnClickListener(new View.OnClickListener() {
            @Override
            public void onClick(View v) {
                InputMethodManager imm = (InputMethodManager) getSystemService(Context.INPUT_METHOD_SERVICE);
                imm.hideSoftInputFromWindow(et_watermark.getWindowToken(), 0);
                if (wmButton.isSelected()) {
                    wmButton.setSelected(false);
                    wmButton.setText(R.string.string_enable);
                    disableEffect();
                } else {
                    wmButton.setSelected(true);
                    wmButton.setText(R.string.string_disable);
                    enableEffect();
                }
            }
        });
    }
    private boolean checkSelfPermission(String permission, int requestCode) {
        if (ContextCompat.checkSelfPermission(this, permission) !=
            PackageManager.PERMISSION_GRANTED) {
            ActivityCompat.requestPermissions(this, REQUESTED_PERMISSIONS, requestCode);
            return false;
        }
        return true;
    }
    private void initAgoraEngine() {
        try {
            RtcEngineConfig config = new RtcEngineConfig();
            config.mContext = this;
            config.mAppId = AGORA_CUSTOMER_APP_ID;
            config.addExtension(ExtensionManager.EXTENSION_NAME);
            config.mExtensionObserver = this;
            config.mEventHandler = new IRtcEngineEventHandler() {
                @Override
                public void onJoinChannelSuccess(String s, int i, int i1) {
                    Log.d(TAG, "on Join Channel Success");
                    mRtcEngine.startPreview();
                    JSONObject pluginParams = new JSONObject();
                    try {
                        setSymblPluginConfigs(pluginParams, AGORA_CUSTOMER_CHANNEL_NAME);
                        mRtcEngine.setExtensionProperty(ExtensionManager.EXTENSION_VENDOR_NAME, ExtensionManager.EXTENSION_FILTER_NAME, "init", pluginParams.toString());
                    } catch (JSONException e) {
                        Log.d(TAG, "ERROR while joining channel " + e.getMessage());
                    }
                }
                @Override
                public void onFirstRemoteVideoDecoded(final int i, int i1, int i2, int i3) {
                    super.onFirstRemoteVideoDecoded(i, i1, i2, i3);
                    runOnUiThread(new Runnable() {
                        @Override
                        public void run() {
                            setupRemoteVideo(i);
                        }
                    });
                }
                @Override
                public void onUserJoined(int i, int i1) {
                    super.onUserJoined(i, i1);
                }
                @Override
                public void onUserOffline(final int i, int i1) {
                    super.onUserOffline(i, i1);
                    runOnUiThread(new Runnable() {
                        @Override
                        public void run() {
                            onRemoteUserLeft();
                        }
                    });
                }
            };
            mRtcEngine = RtcEngine.create(config);
            mRtcEngine.enableExtension(ExtensionManager.EXTENSION_VENDOR_NAME, ExtensionManager.EXTENSION_FILTER_NAME, true);
            setupLocalVideo();
            VideoEncoderConfiguration configuration = new VideoEncoderConfiguration(640, 360,
                VideoEncoderConfiguration.FRAME_RATE.FRAME_RATE_FPS_24,
                VideoEncoderConfiguration.STANDARD_BITRATE,
                VideoEncoderConfiguration.ORIENTATION_MODE.ORIENTATION_MODE_FIXED_PORTRAIT);
            mRtcEngine.setVideoEncoderConfiguration(configuration);
            mRtcEngine.setClientRole(Constants.AUDIO_ENCODED_FRAME_OBSERVER_POSITION_MIC);
            mRtcEngine.enableLocalAudio(true);
            mRtcEngine.setEnableSpeakerphone(true);
            mRtcEngine.setAudioProfile(1);
            mRtcEngine.enableAudio();
            mRtcEngine.addHandler(new IRtcEngineEventHandler() {
                @Override
                public void onUserInfoUpdated(int uid, UserInfo userInfo) {
                    super.onUserInfoUpdated(uid, userInfo);
                }
                @Override
                public void onUserJoined(int uid, int elapsed) {
                    super.onUserJoined(uid, elapsed);
                }
                @Override
                public void onUserOffline(int uid, int reason) {
                    super.onUserOffline(uid, reason);
                }
                @Override
                public void onConnectionStateChanged(int state, int reason) {
                    super.onConnectionStateChanged(state, reason);
                }
                @Override
                public void onConnectionInterrupted() {
                    super.onConnectionInterrupted();
                }
                @Override
                public void onConnectionLost() {
                    super.onConnectionLost();
                }
                @Override
                public void onApiCallExecuted(int error, String api, String result) {
                    super.onApiCallExecuted(error, api, result);
                }
                @Override
                public void onTokenPrivilegeWillExpire(String token) {
                    super.onTokenPrivilegeWillExpire(token);
                }
                @Override
                public void onRequestToken() {
                    super.onRequestToken();
                }
                @Override
                public void onActiveSpeaker(int uid) {
                    super.onActiveSpeaker(uid);
                }
                @Override
                public void onFirstLocalAudioFramePublished(int elapsed) {
                    super.onFirstLocalAudioFramePublished(elapsed);
                }
                @Override
                public void onAudioPublishStateChanged(String channel, STREAM_PUBLISH_STATE oldState, STREAM_PUBLISH_STATE newState, int elapseSinceLastState) {
                    super.onAudioPublishStateChanged(channel, oldState, newState, elapseSinceLastState);
                }
                @Override
                public void onAudioSubscribeStateChanged(String channel, int uid, STREAM_SUBSCRIBE_STATE oldState, STREAM_SUBSCRIBE_STATE newState, int elapseSinceLastState) {
                    super.onAudioSubscribeStateChanged(channel, uid, oldState, newState, elapseSinceLastState);
                }
                @Override
                public void onAudioRouteChanged(int routing) {
                    super.onAudioRouteChanged(routing);
                }
                @Override
                public void onAudioQuality(int uid, int quality, short delay, short lost) {
                    super.onAudioQuality(uid, quality, delay, lost);
                }
                @Override
                public void onRtcStats(RtcStats stats) {
                    super.onRtcStats(stats);
                }
                @Override
                public void onLocalAudioStats(LocalAudioStats stats) {
                    super.onLocalAudioStats(stats);
                }
                @Override
                public void onAudioMixingFinished() {
                    super.onAudioMixingFinished();
                }
                @Override
                public void onLocalAudioStateChanged(LOCAL_AUDIO_STREAM_STATE state, LOCAL_AUDIO_STREAM_ERROR error) {
                    super.onLocalAudioStateChanged(state, error);
                }
                @Override
                public void onEncryptionError(ENCRYPTION_ERROR_TYPE errorType) {
                    super.onEncryptionError(errorType);
                }
                @Override
                public void onPermissionError(PERMISSION permission) {
                    super.onPermissionError(permission);
                }
            });
            mRtcEngine.joinChannel(TOKEN_VALUE, AGORA_CUSTOMER_CHANNEL_NAME, AGORA_MEETING_ID, 0);
            mRtcEngine.startPreview();
        } catch (Exception e) {
            e.printStackTrace();
            Log.e(TAG, " ERROR:: RTC engine startup error " + e.getMessage());
        }
    }
    private void setupLocalVideo() {
        SurfaceView view = RtcEngine.CreateRendererView(this);
        view.setZOrderMediaOverlay(true);
        localVideoContainer.addView(view);
        mRtcEngine.setupLocalVideo(new VideoCanvas(view, VideoCanvas.RENDER_MODE_HIDDEN, 0));
        mRtcEngine.setLocalRenderMode(Constants.RENDER_MODE_HIDDEN);
    }
    private void setupRemoteVideo(int uid) {
        // Only one remote video view is available for this sample application.
        int count = remoteVideoContainer.getChildCount();
        View view = null;
        for (int i = 0; i < count; i++) {
            View v = remoteVideoContainer.getChildAt(i);
            if (v.getTag() instanceof Integer && ((int) v.getTag()) == uid) {
                view = v;
            }
        }
        if (view != null) {
            return;
        }
        Log.d(TAG, " setupRemoteVideo uid = " + uid);
        mRemoteView = RtcEngine.CreateRendererView(getBaseContext());
        remoteVideoContainer.addView(mRemoteView);
        mRtcEngine.setupRemoteVideo(new VideoCanvas(mRemoteView, VideoCanvas.RENDER_MODE_HIDDEN, uid));
        mRtcEngine.setRemoteRenderMode(uid, Constants.RENDER_MODE_HIDDEN);
        mRemoteView.setTag(uid);
    }
    private void onRemoteUserLeft() {
        removeRemoteVideo();
    }
    private void removeRemoteVideo() {
        if (mRemoteView != null) {
            remoteVideoContainer.removeView(mRemoteView);
        }
        mRemoteView = null;
    }
    private void enableEffect() {
        JSONObject pluginParams = new JSONObject();
        try {
            setSymblPluginConfigs(pluginParams, AGORA_CUSTOMER_CHANNEL_NAME);
            mRtcEngine.setExtensionProperty(ExtensionManager.EXTENSION_VENDOR_NAME, ExtensionManager.EXTENSION_FILTER_NAME, "start", pluginParams.toString());
        } catch (JSONException e) {
            e.printStackTrace();
        }
    }
    /**
     * This method is used to set the Symbl configuration
     *
     * @param pluginParams
     * @param channelName
     * @throws JSONException
     */
    private void setSymblPluginConfigs(JSONObject pluginParams, String channelName) throws JSONException {
        try {
            String symbl_unique_meetingId = UUID.randomUUID().toString();
            pluginParams.put("secret", getString(R.string.symbl_app_secret));
            pluginParams.put("appKey", getString(R.string.symbl_app_id));
            pluginParams.put("meetingId", symbl_unique_meetingId);
            pluginParams.put("userId", symbl_meeting_UserId);
            pluginParams.put("name", symbl_meeting_user_Name);
            pluginParams.put("languageCode", "en-US");
            // Symbl main extension config objects
            SymblPluginConfig symplParams = new SymblPluginConfig();
            // Set the Symbl API Configuration
            ApiConfig apiConfig = new ApiConfig();
            apiConfig.setAppId(getString(R.string.symbl_app_id));
            apiConfig.setAppSecret(getString(R.string.symbl_app_secret));
            apiConfig.setTokenApi(getString(R.string.symbl_token_api));
            apiConfig.setSymblPlatformUrl(getString(R.string.symbl_platform_url));
            symplParams.setApiConfig(apiConfig);
            // Set the Symbl Confidence Level and Language Code
            RealtimeStartRequest realtimeFlowInitRequest = new RealtimeStartRequest();
            RealtimeAPIConfig realtimeAPIConfig = new RealtimeAPIConfig();
            realtimeAPIConfig.setConfidenceThreshold(Double.parseDouble(String.valueOf(R.string.symbl_confidence_threshold)));
            realtimeAPIConfig.setLanguageCode(getString(R.string.symbl_meeting_language_code));
            // Set the Speaker information
            Speaker speaker = new Speaker();
            speaker.setName(symbl_meeting_user_Name);
            speaker.setUserId(symbl_meeting_UserId);
            realtimeFlowInitRequest.setSpeaker(speaker);
            // Set the meeting encoding and speaker sample rate hertz
            SpeechRecognition speechRecognition = new SpeechRecognition();
            speechRecognition.setEncoding(getString(R.string.symbl_meeting_encoding));
            speechRecognition.setSampleRateHertz(Double.parseDouble(getString(R.string.symbl_meeting_sampleRateHertz)));
            realtimeAPIConfig.setSpeechRecognition(speechRecognition);
            // Set the redaction content values
            Redaction redaction = new Redaction();
            redaction.setIdentifyContent(true);
            redaction.setRedactContent(true);
            redaction.setRedactionString("*****");
            realtimeAPIConfig.setRedaction(redaction);
            // Set the Tracker (custom business intent) information
            realtimeFlowInitRequest.setConfig(realtimeAPIConfig);
            Tracker tracker1 = new Tracker();
            tracker1.setName("Budget");
            List < String > vocabulary = new ArrayList < > ();
            vocabulary.add("budgeted");
            vocabulary.add("budgeted decision");
            tracker1.setVocabulary(vocabulary);
            List < Tracker > trackerList = new ArrayList < > ();
            trackerList.add(tracker1);
            // Set the Symbl conversation parameters
            realtimeFlowInitRequest.setTrackers(trackerList);
            realtimeFlowInitRequest.setType("start_request");
            realtimeFlowInitRequest.setId(symbl_unique_meetingId);
            realtimeFlowInitRequest.setSentiments(true);
            realtimeFlowInitRequest.setInsightTypes(Arrays.asList("action_item", "question", "follow_up"));
            symplParams.setRealtimeStartRequest(realtimeFlowInitRequest);
            Gson gson = new Gson();
            pluginParams.put("inputRequest", gson.toJson(symplParams));
        } catch (Exception ex) {
            Log.e(TAG, "ERROR while setting Symbl extension configuration");
        }
    }
    private void disableEffect() {
        JSONObject o = new JSONObject();
        mRtcEngine.setExtensionProperty(ExtensionManager.EXTENSION_VENDOR_NAME, ExtensionManager.EXTENSION_FILTER_NAME, "stop", o.toString());
    }
    @Override
    public void onEvent(String vendor, String extension, String key, String value) {
        Log.i(TAG, "Symbl conversation Event \n \n  " + vendor + "  extension: " + extension + "  key: " + key + "  value: " + value);
        final StringBuilder sb = new StringBuilder();
        sb.append(value);
        if ("result".equals(key)) {
            try {
                Gson json = new Gson();
                SymblResponse symblResponse = json.fromJson(value, SymblResponse.class);
                if (symblResponse.getEvent() != null && symblResponse.getEvent().length() > 0) {
                    switch (symblResponse.getEvent()) {
                        //this is the conversation response from Symbl platform
                    case SymblAIFilterManager.SYMBL_START_PLUGIN_REQUEST:
                        break;
                    case SymblAIFilterManager.SYMBL_ON_MESSAGE:
                        try {
                            if (symblResponse.getErrorMessage() != null && symblResponse.getErrorMessage().length() > 0) {} else {}
                        } catch (Exception ex) {
                            Log.e(TAG, "ERROR on Symbl message on message transform error " + ex.getMessage());
                        }
                        break;
                    case SymblAIFilterManager.SYMBL_CONNECTION_ERROR:
                        if (symblResponse != null)
                            Log.i(TAG, "SYMBL_CONNECTION_ERROR error code %s , error message " + symblResponse.getErrorCode());
                        break;
                    case SymblAIFilterManager.SYMBL_WEBSOCKETS_CLOSED:
                        if (symblResponse != null)
                            Log.i(TAG, "SYMBL_CONNECTION_ERROR " + symblResponse.getErrorCode());
                        break;
                    case SymblAIFilterManager.SYMBL_TOKEN_EXPIRED:
                        break;
                    case SymblAIFilterManager.SYMBL_STOP_REQUEST:
                        break;
                    case SymblAIFilterManager.SYMBL_ON_CLOSE:
                        break;
                    case SymblAIFilterManager.SYMBL_SEND_REQUEST:
                        break;
                    case SymblAIFilterManager.SYMBL_ON_ERROR:
                        break;
                    }
                } else { // all error cases handle here
                    if (symblResponse != null) {
                        sb.append("\n Symbl event :" + symblResponse.getEvent());
                        sb.append("\n Error Message :" + symblResponse.getErrorMessage());
                        sb.append("\n Error code :" + symblResponse.getErrorCode());
                    }
                }
            } catch (Exception exception) {
                System.out.println("result parse error ");
            }
        }
        this.runOnUiThread(new Runnable() {
            @Override
            public void run() {
                infoTextView.setText(sb.toString());
            }
        });
    }
}

Run the demo

The following sample project provides you an Android mobile app using the Agora Video SDK and the Symbl.ai Extension and it can be used as a reference. Follow the instructions in the README file for setting up, configuring and running the sample mobile app in your own device. See Sample Android App Project.

Reference

Find comprehensive information about our REST APIs in the API Reference section.

Prerequisites​

Integrate Symbl Conversation Intelligence​

Run the demo​

Reference​

Prerequisites

Integrate Symbl Conversation Intelligence

Run the demo

Reference