Android Ultimate Plugin tutorial part 5 – Speech to text

Hi everyone, it’s tutorial time again and now we will show you on how to use Speech to text feature and before anything else we want to tell you that this features will only work when there’s an internet connection on  your device else it will just throw an error event.

Permission needed on your Android Manifest file without this it will not work or it will produce an error.

<uses-permission android:name="android.permission.INTERNET"/>
<uses-permission android:name="android.permission.READ_EXTERNAL_STORAGE"/>
<uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE"/>
<uses-permission android:name="android.permission.ACCESS_NETWORK_STATE"/>
<uses-permission android:name="android.permission.RECORD_AUDIO"/>

Let’s do it!

Firs step is to get the instance of SpeechPlugin

private SpeechPlugin speechPlugin = SpeechPlugin.GetInstance();

To stop Debug toast message from showing,  pass 0 on SetDebug method

speechPlugin.SetDebug(0);

We need to Initialized it

speechPlugin.Init();

Next we need to check first if the current device has a support for Speech Recognizer, we can do this check by call this method, if it’s true then its supported else it’s not supported.

bool isSupported = speechPlugin.CheckSpeechRecognizerSupport();

After we make sure that Speech Recognizer is supported on your Android Device, we can now setup a Listener for Speech Recognizer Events, by doing this code

//speech recognizer listener
speechPlugin.onReadyForSpeech+=onReadyForSpeech;
speechPlugin.onBeginningOfSpeech+=onBeginningOfSpeech;
speechPlugin.onEndOfSpeech+=onEndOfSpeech;
speechPlugin.onError+=onError;
speechPlugin.onResults+=onResults;

And here’s the sample method receivers fro Speech Recognizer Events

private void onReadyForSpeech(string data){
     string receivedData = data.ToString();
}

private void onBeginningOfSpeech(string data){
     string receivedData = data.ToString();
}

private void onEndOfSpeech(string data){
    string receivedData = data.ToString();
}

private void onError(string data){
    string receivedData = data.ToString();
}

private void onResults(string data){        
    string[] results =  data.Split(',');
    Debug.Log(" result length " + results.Length);
            
   //when you set morethan 1 results index zero is always the closest to  the words the you said

  //but it's not always the case so if you are not happy with index zero result you can always check the other index
            
  //sample on checking other results
  foreach( string possibleResults in results ){
       Debug.Log( " possibleResults " + possibleResults );
  }
           
  //sample showing the nearest result
  string firstResult = results.GetValue(0).ToString();  
}

 

To Start Listening for Speech Input call this method

//the number if possible results
int numberOfResults = 5; 

//the actual call when starting to listen from speech input
speechPlugin.StartListening(numberOfResults);

To stop listening from Speech Recognizer events call this method

speechPlugin.RemoveSpeechRecognizerListener();

and when you are done using Speech Recognizer Destroy it to release the Speech Recognizer service from Android Device

speechPlugin.DestroySpeechController();

To  Cancel Speech

speechPlugin.Cancel();

To Stop Listening

speechPlugin.StopListening();

For Stopping and Cancel

speechPlugin.StopCancel();

New Feature

Start Listening No Beep

speechPlugin.StartListeningNoBeep(numberOfResults,true);

Experiment Features but still not complete to listen and search using web browser

speechPlugin.StartListeningWithSearch(numberOfResults);

For Reference here’s the code that we used on Speech To Text Demo

Note: you will noticed on this code that Text to Speech is also used, the purposed is only for presentation. Also to make this code run you need Text UI just drag it on this code inspector and Button UI to call Start Listening. 

using UnityEngine;
using UnityEngine.UI;
using System.Collections;
using System;
using System.Linq;

public class SpeechRecognizerDemo : MonoBehaviour {

	private const string TAG = "[SpeechRecognizerDemo]: ";

	private SpeechPlugin speechPlugin;	
	private bool hasInit = false;
	public Text resultText;
	public Text statusText;

	private TextToSpeechPlugin textToSpeechPlugin;

	// Use this for initialization
	void Start (){		
		speechPlugin = SpeechPlugin.GetInstance();
		speechPlugin.SetDebug(0);
		speechPlugin.Init();

		//speech recognizer listener
		speechPlugin.onReadyForSpeech+=onReadyForSpeech;
		speechPlugin.onBeginningOfSpeech+=onBeginningOfSpeech;
		speechPlugin.onEndOfSpeech+=onEndOfSpeech;
		speechPlugin.onError+=onError;
		speechPlugin.onResults+=onResults;

		//new
		SpeechPlugin.OnSpeechSearchComplete+=OnSpeechSearchComplete;
		SpeechPlugin.OnSpeechSearchFail+=OnSpeechSearchFail;

		textToSpeechPlugin = TextToSpeechPlugin.GetInstance();
		textToSpeechPlugin.SetDebug(0);
		textToSpeechPlugin.Initialize();

		//text to speech listener
		textToSpeechPlugin.OnInit+=OnInit;
		textToSpeechPlugin.OnGetLocaleCountry+=OnGetLocaleCountry;
		textToSpeechPlugin.OnChangeLocale+=OnSetLocale;
		textToSpeechPlugin.OnStartSpeech+=OnStartSpeech;
		textToSpeechPlugin.OnEndSpeech+=OnEndSpeech;
		textToSpeechPlugin.OnErrorSpeech+=OnErrorSpeech;
	}

	private void OnApplicationPause(bool val){
		//for text to speech events
		if(textToSpeechPlugin!=null){
			if(hasInit){
				if(val){
					textToSpeechPlugin.UnRegisterBroadcastEvent();
				}else{
					textToSpeechPlugin.RegisterBroadcastEvent();
				}
			}
		}
	}

	public void StartListening(){
		bool isSupported = speechPlugin.CheckSpeechRecognizerSupport();

		if(isSupported){
			//number of possible results
			//Note: sometimes even you put 5 numberOfResults, there's a chance that it will be only 3 or 2
			//it is not constant.
			
			int numberOfResults = 5;
			speechPlugin.StartListening(numberOfResults);
			
			//by activating this, the Speech Recognizer will start and you can start Speaking or saying something 
			//speech listener will stop automatically especially when you stop speaking or when you are speaking 
			//for a long time
		}else{
			Debug.Log(TAG + "Speech Recognizer not supported by this Android device ");
		}
	}

	public void StartListeningWithSearch(){
		bool isSupported = speechPlugin.CheckSpeechRecognizerSupport();
		
		if(isSupported){
			//number of possible results
			//Note: sometimes even you put 5 numberOfResults, there's a chance that it will be only 3 or 2
			//it is not constant.
			
			int numberOfResults = 5;
			speechPlugin.StartListeningWithSearch(numberOfResults);
			
			//by activating this, the Speech Recognizer will start and you can start Speaking or saying something 
			//speech listener will stop automatically especially when you stop speaking or when you are speaking 
			//for a long time
		}else{
			Debug.Log(TAG + "Speech Recognizer not supported by this Android device ");
		}
	}

	public void StartListeningNoBeep(){
		bool isSupported = speechPlugin.CheckSpeechRecognizerSupport();
		
		if(isSupported){
			//number of possible results
			//Note: sometimes even you put 5 numberOfResults, there's a chance that it will be only 3 or 2
			//it is not constant.
			
			int numberOfResults = 5;
			speechPlugin.StartListeningNoBeep(numberOfResults,true);
			
			//by activating this, the Speech Recognizer will start and you can start Speaking or saying something 
			//speech listener will stop automatically especially when you stop speaking or when you are speaking 
			//for a long time
		}else{
			Debug.Log(TAG + "Speech Recognizer not supported by this Android device ");
		}
	}

	//cancel speech
	public void CancelSpeech(){
		if(speechPlugin!=null){
			bool isSupported = speechPlugin.CheckSpeechRecognizerSupport();

			if(isSupported){			
				speechPlugin.Cancel();
			}
		}

		Debug.Log( TAG + " call CancelSpeech..  ");
	}

	public void StopListening(){
		if(speechPlugin!=null){
			speechPlugin.StopListening();
		}
		Debug.Log( TAG + " StopListening...  ");
	}

	public void StopCancel(){
		if(speechPlugin!=null){
			speechPlugin.StopCancel();
		}
		Debug.Log( TAG + " StopCancel...  ");
	}

	private void OnDestroy(){
		speechPlugin.StopListening();
		speechPlugin.DestroySpeechController();

		//call this of your not going to used TextToSpeech Service anymore
		textToSpeechPlugin.ShutDownTextToSpeechService();
	}

	private void UpdateStatus(string status){
		if(statusText!=null){
			statusText.text = String.Format("Status: {0}",status);	
		}
	}

	//SpeechRecognizer Events
	private void onReadyForSpeech(string data){
		UpdateStatus(data.ToString());
	}

	private void onBeginningOfSpeech(string data){
		UpdateStatus(data.ToString());
	}

	private void onEndOfSpeech(string data){
		UpdateStatus(data.ToString());
	}

	private void onError(int data){
		SpeechRecognizerError error = (SpeechRecognizerError)data;
		UpdateStatus(error.ToString());
		//statusText.text =  String.Format("Status: {0}",error.ToString());

		if(resultText!=null){
			resultText.text =  "Result: Waiting for result...";
		}
	}

	private void onResults(string data){
		if(resultText!=null){
			string[] results =  data.Split(',');
			Debug.Log( TAG + " result length " + results.Length);
			
			//when you set morethan 1 results index zero is always the closest to the words the you said
			//but it's not always the case so if you are not happy with index zero result you can always 
			//check the other index
			
			//sample on checking other results
			foreach( string possibleResults in results ){
				Debug.Log( TAG + " possibleResults " + possibleResults );
			}
			
			//sample showing the nearest result
			string whatToSay  = results.GetValue(0).ToString();
			string utteranceId  = "test-utteranceId";
			resultText.text =  string.Format("Result: {0}",whatToSay); 
			
			//check if Text to speech has initialized
			if(hasInit){
				//Text To Speech Sample Usage
				textToSpeechPlugin.SpeakOut(whatToSay,utteranceId);
			}
		}
	}

	public void OnSpeechSearchComplete(string val){
		Debug.Log(TAG + "OnSpeechSearchComplete val: " + val);
	}

	public void OnSpeechSearchFail(string val){
		Debug.Log(TAG + "OnSpeechSearchFail val: " + val);
	}
	//SpeechRecognizer Events


	//TextToSpeech Events
	private void OnInit(int status){
		Debug.Log(TAG + "OnInit status: " + status);
		
		if(status == 1){
			hasInit = true;
			textToSpeechPlugin.SetLocale(SpeechLocale.US);
		}
	}

	private void OnGetLocaleCountry(string localeCountry){
		Debug.Log( TAG + "OnGetLocaleCountry localeCountry: " + localeCountry);
	}
	
	private void OnSetLocale(int status){
		Debug.Log(TAG + "OnSetLocale status: " + status);
		if(status == 1){
			textToSpeechPlugin.SetPitch(1f);
		}
	}
	
	private void OnStartSpeech(string utteranceId){
		Debug.Log(TAG + "OnStartSpeech utteranceId: " + utteranceId);
	}
	
	private void OnEndSpeech(string utteranceId){
		Debug.Log(TAG + "OnEndSpeech utteranceId: " + utteranceId);
	}
	
	private void OnErrorSpeech(string utteranceId){
		Debug.Log(TAG + "OnErrorSpeech utteranceId: " + utteranceId);
	}
	//TextToSpeech Events
}

And there you have it, Speech Recognizer tutorial is done.

Thanks for reading our humble tutorial and Happy Coding 🙂

For questions or comments or suggestions or any feed back just email us at gigadrillgames.com

4 thoughts on “Android Ultimate Plugin tutorial part 5 – Speech to text

  • May 27, 2016 at 4:42 am
    Permalink

    in the SpeechPlugin, how can change the language? exists some method simillar to .StartListening (3, “en-US”);

    Reply
    • June 17, 2016 at 11:13 am
      Permalink

      Hi you can change it now it’s a new feature download the new version please

      Reply
  • June 15, 2017 at 12:40 pm
    Permalink

    how can the app recognize speech in background mode?

    Reply
    • July 6, 2017 at 12:29 pm
      Permalink

      we don’t have this feature as of now sorry

      Reply

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.