AIPlayer Basic Speaking Features

1. Send Speaking

After the aiPlayer initialization is completed(AIPlayer’s state change from INITIALIZE to IDLE), you can call the send method.

You can just input string to make AI speak but json format (AIClipSet object)is also available. When using object(converted into AIClipSet inside AIPlayer), you can make AI speak with gesture. For example, you can order AI to wave and say "hello!" This is called gesture speech. Details are described in the Gesture part.

If the text to speak is too long, it may not be possible to synthesize the resources required for the speaking. There are some models that can synthesize long sentences. Although it varies from AI to AI, it is generally recommended that sentences be cut to an appropriate length in Korean, usually within 100 characters, and at a similar level in English.

//Case1. One Gesture Speak
const AIClipSet = { text: "Nice to meet you", gst: "hi" };
AI_PLAYER.send(AIClipSet);

//Case2. One Sentence Speak
AI_PLAYER.send("Nice to meet you");

You can test it from the SDK demo page if you want to.

2. Monitoring of speaking behavior

After calling the send method, you can check the feedback on the operation event in the registered listener. This feedback is returned by calling the event related method (onAIPlayerEvent) of the listener. For example, when you call send method, the following events will be sent as an argument of onAIPlayerEvent sequencially.

• AIEventType.AICLIPSET_PLAY_PREPARE_STARTED
• AIEventType.AICLIPSET_PLAY_PREPARE_COMPLETED
• AIEventType.AICLIPSET_PLAY_STARTED
• AIEventType.AICLIPSET_PLAY_COMPLETED
• AIEventType.AICLIPSET_PLAY_FAILED //if there is a failure,

If there are some errors while send, the onAIPlayerErrorV2 will be called with AIError that contains which error has occured.

AI_PLAYER.onAIPlayerEvent = function (aiEvent) {
  // TODO: event handling

  //example
  switch (aiEvent.type) {
    case AIEventType.AIPLAYER_STATE_CHANGED:
      onAIStateChange();
      console.log("AI AIPLAYER_STATE_CHANGED :", aiEvent);
      break;
    case AIEventType.AICLIPSET_PLAY_PREPARE_STARTED:
      console.log("AI started preparing to speak. speech :", aiEvent);
      break;
    case AIEventType.AICLIPSET_PLAY_PREPARE_COMPLETED:
      console.log("AI finished preparing to speak. speech :", aiEvent);
      break;
    case AIEventType.AICLIPSET_PLAY_STARTED:
      console.log("AI started speaking. normal state: ", aiEvent);
      break;
    case AIEventType.AICLIPSET_PLAY_COMPLETED:
      console.log("AI finished speaking. normal state: ", aiEvent);
      break;
    case AIEventType.AICLIPSET_PLAY_FAILED:
      console.log("AI AICLIPSET_PLAY_FAILED: ", aiEvent);
      break;
    case AIEventType.AICLIPSET_PRELOAD_FAILED:
      console.log("AI AICLIPSET_PRELOAD_FAILED: ", aiEvent);
      break;
    //...
  }
};

AI_PLAYER.onAIPlayerErrorV2 = function (aiError) {
  // TODO: error handling

  if (aiError.code >= AIError.RESERVED_ERR) {
    //You've encountered a reserved error. Please check the error list!
    console.log("RESERVED_ERR :", aiError.message);
  } else if (aiError.code >= AIErrorCode.AICLIPSET_PLAY_ERR) {
    console.log("AICLIPSET_PLAY_ERR :", aiError.message);
  } else if (aiError.code >= AIErrorCode.INVALID_AICLIPSET_ERR) {
    console.log("INVALID_AICLIPSET_ERR :", aiError.message);
  } else if (aiError.code >= AIErrorCode.AI_SERVER_ERR) {
    console.log("AI_SERVER_ERR :", aiError.message);
  } else if (aiError.code > AIErrorCode.UNKNOWN_ERR) {
    //0 ~ 9999
    console.log("BACKEND_ERR :", aiError.message);

    if (aiError.code == 1402) {
      //refresh token
      refreshTokenIFExpired();
    }
  } else {
    console.log("UNKNOWN_ERR :", aiError.message);
  }
};

3. Make resposive UI with AIPLAYER_STATE_CHANGED event

The AIPlayer has several states that a user can notice and respond to. When the state is changed, the 'onAIPlayerEvent' callback is called with AIEventType.AIPLAYER_STATE_CHANGED and AIPlayer.getState() method return the current state.

Before an AI initialized, the state is AIPlayerState.NONE and it changes to INITIALIZE when the 'AIPlayer.init()' method called. If the AI initialization is done, The state will change to IDLE. In this state, one can send AIClipSet and the state will be 'PLAY'. Finally, if the AIPlayer is released by calling 'release', the state will be RELEASE state. Please refer the full definition in here.

AI_PLAYER.onAIPlayerEvent = function (aiEvent) {
  // TODO: event handling

  //example
  switch (aiEvent.type) {
    case AIEventType.AIPLAYER_STATE_CHANGED:
      onAIStateChange();
      console.log("AI AIPLAYER_STATE_CHANGED :", aiEvent);
      break;
    //...
  }
};

function onAIStateChange() {
  //make resonsive app by enable or disble UI or etc...
  console.log("onAIStateChange state:", AI_PLAYER.getState());
}

4. Pause Speaking

AI_PLAYER.pause();

5. Resume Speaking (Resume from Pause)

AI_PLAYER.resume();

6. Stop Speaking (Resets all the data you have. No resume allowed)

Stop speaking and reset all data. (cannot resume)

AI_PLAYER.stopSpeak();