Text-to-speech and speech recognition are both important elements of the web-based server platform and therefore both tested extensively. TTS accesses the html pages and reads out the data to callers and speec rec is used as the primary means of caller navigation between the menus and for accepting data such as credit card numbers, etc.

Speech Recognition Command Evaluation

During each test call, the automated call generator would speak a word or a phrase and expect to receive a particular series of DTMF digits in return. The automated call generator program gathered performance data (time for vendor server to respond to a spoken command) and the accuracy of the expected response with twelve lines of call load.

DTMF Command Performance Evaluation

During the test call, the automated call generator simulates the pressing of a digit or series of digits and expects a specific string of DTMF digits in response from the server under test. If the code that is sent based on the VoiceXML application does not match the expected code, an error is logged in the automated call generator program. The automated call generator program accumulated performance data (time for vendor server to respond to a DTMF command) and the accuracy of the expected response with twelve lines of call load.

The results of the Speech Recognition and DTMF Command Evaluations can be found here.

Text-to-Speech (TTS) Quality Evaluation

This test evaluated the quality of each vendor server's ability to read ASCII text to callers. A menu option was selected to hear a description of a certain college class. The text of the class description follows:

MIS 116. Systems Development Life Cycle. A business information system of moderate complexity will be analyzed and designed by student teams. Various analysis and design techniques will be studied and used in this project. Both oral and written reports are required. CASE methodology will be used. The project will be continued in MIS 116B; therefore, taking MIS116A and 116B in consecutive semesters is highly recommended. Prerequisite: :MIS 030, MIS 101. 3 units.

A panel of six CT Labs personnel judged the quality of these recordings on a scale of 1-10, where 1 is Bad and 10 is Excellent. Their scores were averaged to produce the final TTS score. On listening quality, BeVocal scored the highest with 7.1, followed by VoiceGenie with 6.8, Tellme with 6.7 and HeyAnita with 5.7.

To hear the text-to-speech example, click on a Real Media file below:

BeVocal
HeyAnita
Tellme
VoiceGenie

Back to Introduction: Voice Portals Using VoiceXML
See CT Labs Full Report
See Product Overview
See Product Review Results Grid
See Samples Of Voice XML Script