Speech to Text Conversion in React Native – Voice Recognition

React Native Speech to Text Conversion

This is an example to show how to do Speech to Text Conversion in React Native – Voice Recognition. This is a very demanding feature from many of the customers after the success of intelligent voice assistances like Google Home and Amazon Alexa. To make your app different from another app you can implement voice recognition features while making search filters. In the current scenario, many applications have voice search with text input search where they provide either search by text input or can search by voice.

Here we are going to see how can we convert voice to text and can get the texts as a result with the help of voice recognition.

Voice Recognition in React Native

For the voice recognition in React Native or we can say that for the speech to text conversion we are going to use Voice component provided by react-native-voice library which has a number of events which can be used to start or stop the voice recognition and to get the status and of voice recognition.

When we initialize the screen we set some event callbacks in the constructor which looks like the code snippet given below. You can see we are providing a function for the SpeechStart or for the Speech End and so on. So this is the callbacks basically which will be called automatically when the event happens.

Voice.onSpeechStart = this.onSpeechStart;
Voice.onSpeechEnd = this.onSpeechEnd;
Voice.onSpeechError = this.onSpeechError;
Voice.onSpeechResults = this.onSpeechResults;
Voice.onSpeechPartialResults = this.onSpeechPartialResults;
Voice.onSpeechVolumeChanged = this.onSpeechVolumeChanged;

So these were the callback events to get the status of voice recognition. Now let’s see how to start, stop, cancel and destroy the voice recognition process.

Start Voice Recognition

const startRecognizing = async () => {
  this.setState({
    pitch: '',
    error: '',
    started: '',
    results: [],
    partialResults: [],
    end: '',
  });

  try {
    await Voice.start('en-US');
  } catch (e) {
    //eslint-disable-next-line
    console.error(e);
  }
};

Stop Voice Recognition

const stopRecognizing = async () => {
  try {
    await Voice.stop();
  } catch (e) {
    //eslint-disable-next-line
    console.error(e);
  }
};

Cancel Voice Recognition

const cancelRecognizing = async () => {
  try {
    await Voice.cancel();
  } catch (e) {
    //eslint-disable-next-line
    console.error(e);
  }
};

Destroy the session of Voice Recognition

const destroyRecognizer = async () => {
  try {
    await Voice.destroy();
  } catch (e) {
    //eslint-disable-next-line
    console.error(e);
  }
  this.setState({
    pitch: '',
    error: '',
    started: '',
    results: [],
    partialResults: [],
    end: '',
  });
};

I think this is enough to know about the library, now let’s move towards the code. In this example, we are going to make a screen with a mike icon which will be the clickable button for us and after clicking onto that we will start voice recognition. In the whole process, we will get the status of everything in the functions we have set for the callback. We can also stop the speech to text conversion by using stop,  cancel and destroy button.

One more point I want to add in this is we will get two types of results while and after voice recognition which are:

1. Result: It comes when Speech Recognizer finishes recognization.
2. Partial Results: It comes during the computation of results, so this is the kind of words that voice recognizer recognizes before the final result. Partial results can be many for a single recognition as it is an intermediate result.

Now let’s get started with the example and see how to convert speech to text.

To Make a React Native App

Getting started with React Native will help you to know more about the way you can make a React Native project. We are going to use react native command line interface to make our React Native App.

If you have previously installed a global react-native-cli package, please remove it as it may cause unexpected issues:

npm uninstall -g react-native-cli @react-native-community/cli

Run the following commands to create a new React Native project

npx react-native init ProjectName

If you want to start a new project with a specific React Native version, you can use the --version argument:

npx react-native init ProjectName --version X.XX.X

Note If the above command is failing, you may have old version of react-native or react-native-cli installed globally on your pc. Try uninstalling the cli and run the cli using npx.

This will make a project structure with an index file named App.js in your project directory.

Installation of Dependency

To use Voice component we have to install react-native-voice dependency. To install the dependency open the terminal and jump into your project

cd ProjectName

Now install the dependency

npm install react-native-voice --save

CocoaPods Installation

After the updation of React Native 0.60, they have introduced autolinking so we do not require to link the libraries but need to install pods. In this example, we need to install the pods for react-native-voice.

Please use the following command to install CocoaPods

npx pod-install

Permission to use microphone and speech recognition for IOS

Please follow the below steps to add the permission in the iOS project to use the microphone and speech recognition.

Open the project SpeechToTextExample -> ios -> yourprj.xcworkspace in Xcode.

1. After opening the project in Xcode click on the project from the left sidebar and you will see multiple options in the right workspace.

2. Select info tab which is info.plist

3. Now add two permissions key “Privacy-Microphone Usage Description” and “Privacy-Speech Recognition Usage Description”. You can also set the value which will be visible when permission dialog pops up. Here are the screenshots below for the reference.

Code to Convert Speech to Text

Now Open App.js in any code editor and replace the code with the following code

App.js

// Speech to Text Conversion in React Native – Voice Recognition
// https://aboutreact.com/speech-to-text-conversion-in-react-native-voice-recognition/

// import React in our code
import React, {useState, useEffect} from 'react';

// import all the components we are going to use
import {
  SafeAreaView,
  StyleSheet,
  Text,
  View,
  Image,
  TouchableHighlight,
  ScrollView,
} from 'react-native';

// import Voice
import Voice from 'react-native-voice';

const App = () => {
  const [pitch, setPitch] = useState('');
  const [error, setError] = useState('');
  const [end, setEnd] = useState('');
  const [started, setStarted] = useState('');
  const [results, setResults] = useState([]);
  const [partialResults, setPartialResults] = useState([]);

  useEffect(() => {
    //Setting callbacks for the process status
    Voice.onSpeechStart = onSpeechStart;
    Voice.onSpeechEnd = onSpeechEnd;
    Voice.onSpeechError = onSpeechError;
    Voice.onSpeechResults = onSpeechResults;
    Voice.onSpeechPartialResults = onSpeechPartialResults;
    Voice.onSpeechVolumeChanged = onSpeechVolumeChanged;

    return () => {
      //destroy the process after switching the screen
      Voice.destroy().then(Voice.removeAllListeners);
    };
  }, []);

  const onSpeechStart = (e) => {
    //Invoked when .start() is called without error
    console.log('onSpeechStart: ', e);
    setStarted('√');
  };

  const onSpeechEnd = (e) => {
    //Invoked when SpeechRecognizer stops recognition
    console.log('onSpeechEnd: ', e);
    setEnd('√');
  };

  const onSpeechError = (e) => {
    //Invoked when an error occurs.
    console.log('onSpeechError: ', e);
    setError(JSON.stringify(e.error));
  };

  const onSpeechResults = (e) => {
    //Invoked when SpeechRecognizer is finished recognizing
    console.log('onSpeechResults: ', e);
    setResults(e.value);
  };

  const onSpeechPartialResults = (e) => {
    //Invoked when any results are computed
    console.log('onSpeechPartialResults: ', e);
    setPartialResults(e.value);
  };

  const onSpeechVolumeChanged = (e) => {
    //Invoked when pitch that is recognized changed
    console.log('onSpeechVolumeChanged: ', e);
    setPitch(e.value);
  };

  const startRecognizing = async () => {
    //Starts listening for speech for a specific locale
    try {
      await Voice.start('en-US');
      setPitch('');
      setError('');
      setStarted('');
      setResults([]);
      setPartialResults([]);
      setEnd('');
    } catch (e) {
      //eslint-disable-next-line
      console.error(e);
    }
  };

  const stopRecognizing = async () => {
    //Stops listening for speech
    try {
      await Voice.stop();
    } catch (e) {
      //eslint-disable-next-line
      console.error(e);
    }
  };

  const cancelRecognizing = async () => {
    //Cancels the speech recognition
    try {
      await Voice.cancel();
    } catch (e) {
      //eslint-disable-next-line
      console.error(e);
    }
  };

  const destroyRecognizer = async () => {
    //Destroys the current SpeechRecognizer instance
    try {
      await Voice.destroy();
      setPitch('');
      setError('');
      setStarted('');
      setResults([]);
      setPartialResults([]);
      setEnd('');
    } catch (e) {
      //eslint-disable-next-line
      console.error(e);
    }
  };

  return (
    <SafeAreaView style={styles.container}>
      <View style={styles.container}>
        <Text style={styles.titleText}>
          Speech to Text Conversion in React Native |
          Voice Recognition
        </Text>
        <Text style={styles.textStyle}>
          Press mike to start Recognition
        </Text>
        <View style={styles.headerContainer}>
          <Text style={styles.textWithSpaceStyle}>
            {`Started: ${started}`}
          </Text>
          <Text style={styles.textWithSpaceStyle}>
            {`End: ${end}`}
          </Text>
        </View>
        <View style={styles.headerContainer}>
          <Text style={styles.textWithSpaceStyle}>
            {`Pitch: \n ${pitch}`}
          </Text>
          <Text style={styles.textWithSpaceStyle}>
            {`Error: \n ${error}`}
          </Text>
        </View>
        <TouchableHighlight onPress={startRecognizing}>
          <Image
            style={styles.imageButton}
            source={{
              uri:
                'https://raw.githubusercontent.com/AboutReact/sampleresource/master/microphone.png',
            }}
          />
        </TouchableHighlight>
        <Text style={styles.textStyle}>
          Partial Results
        </Text>
        <ScrollView>
          {partialResults.map((result, index) => {
            return (
              <Text
                key={`partial-result-${index}`}
                style={styles.textStyle}>
                {result}
              </Text>
            );
          })}
        </ScrollView>
        <Text style={styles.textStyle}>
          Results
        </Text>
        <ScrollView style={{marginBottom: 42}}>
          {results.map((result, index) => {
            return (
              <Text
                key={`result-${index}`}
                style={styles.textStyle}>
                {result}
              </Text>
            );
          })}
        </ScrollView>
        <View style={styles.horizontalView}>
          <TouchableHighlight
            onPress={stopRecognizing}
            style={styles.buttonStyle}>
            <Text style={styles.buttonTextStyle}>
              Stop
            </Text>
          </TouchableHighlight>
          <TouchableHighlight
            onPress={cancelRecognizing}
            style={styles.buttonStyle}>
            <Text style={styles.buttonTextStyle}>
              Cancel
            </Text>
          </TouchableHighlight>
          <TouchableHighlight
            onPress={destroyRecognizer}
            style={styles.buttonStyle}>
            <Text style={styles.buttonTextStyle}>
              Destroy
            </Text>
          </TouchableHighlight>
        </View>
      </View>
    </SafeAreaView>
  );
};

export default App;

const styles = StyleSheet.create({
  container: {
    flex: 1,
    flexDirection: 'column',
    alignItems: 'center',
    padding: 5,
  },
  headerContainer: {
    flexDirection: 'row',
    justifyContent: 'space-between',
    paddingVertical: 10,
  },
  titleText: {
    fontSize: 22,
    textAlign: 'center',
    fontWeight: 'bold',
  },
  buttonStyle: {
    flex: 1,
    justifyContent: 'center',
    marginTop: 15,
    padding: 10,
    backgroundColor: '#8ad24e',
    marginRight: 2,
    marginLeft: 2,
  },
  buttonTextStyle: {
    color: '#fff',
    textAlign: 'center',
  },
  horizontalView: {
    flexDirection: 'row',
    position: 'absolute',
    bottom: 0,
  },
  textStyle: {
    textAlign: 'center',
    padding: 12,
  },
  imageButton: {
    width: 50,
    height: 50,
  },
  textWithSpaceStyle: {
    flex: 1,
    textAlign: 'center',
    color: '#B0171F',
  },
});

To Run the React Native App

Open the terminal again and jump into your project using.

cd ProjectName

1. Start Metro Bundler

First, you will need to start Metro, the JavaScript bundler that ships with React Native. To start Metro bundler run following command:

npx react-native start

Once you start Metro Bundler it will run forever on your terminal until you close it. Let Metro Bundler run in its own terminal. Open a new terminal and run the application.

2. Start React Native Application

To run the project on an Android Virtual Device or on real debugging device:

npx react-native run-android

or on the iOS Simulator by running (macOS only)

npx react-native run-ios

Output Screenshots

Android

iOS

This is how you can do speech to text conversion / Voice Recognition in React Native. If you have any doubts or you want to share something about the topic you can comment below or contact us here. There will be more posts coming soon. Stay tuned!

Hope you liked it. 🙂