Batch Container

High Level Summary

This release provides new improved language packs for all Speechmatics' commercially available languages. Two new language packs Cantonese (yue) and Indonesian (id) are released. It improves on existing punctuation in several of these languages, as well as existing speaker diarization and custom dictionary features. Common numerals (entities) are now transcribed in a consistent and predictable fashion for a subset of languages. Additional data about these entities can be requested via the API.

Important Notices

It is now necessary to use processors that support Advanced Vector Extensions 2 (AVX2) when running the container in all scenarios in order to take advantage of latest performance optimisations.

It is also recommended when using the enhanced model to use hardware that supports the AVX512_VNNI flag for optimal processing performance. The enhanced model also has increased compute requirements and will run more slowly than the standard model. For more information please see the quick start guide.

What's New

9.0.0

• Improved accuracy for all 31 language packs. Gains will be for both standard and enhanced operating points
• New Cantonese (yue) and Indonesian (id) language packs
• Improved formatting of numeric entities such as dates, currencies and large numbers for 11 languages, which are as following
  • Cantonese (yue)
  • Chinese Mandarin (cmn)
  • English (en)
  • French (fr)
  • German (de)
  • Hindi (hi)
  • Italian (it)
  • Japanese (ja)
  • Portuguese (pt)
  • Russian (ru)
  • Spanish (es)
• Additional metadata about the entities can be requested by using the new enable_entities config parameter. For more information please see our documentation for entities here
• Improvements to speaker diarization functionality in scenarios where two speakers are labelled when it is only a single speaker
• Improvements to custom dictionary functionality. Custom dictionary entries should now have less false positives
• Languages with updated punctuation marks
  • Japanese (。 、)
  • Italian (. ? , !)
  • Portuguese (. ? , !)
  • Russian (. ? , !)
  • Mandarin (。 ？ ！ 、)
  • Hindi (। ? , !)
  • All other languages will not see a change in outstanding punctuation marks
• The JSON-v2 output format version is now 2.7
• The transcription can now output words containing non-breaking spaces as a single result
• Speaker Diarization sensitivity parameters (previously deprecated in March 2021) are now removed from the API
  • Jobs will now be rejected if these parameters are included in the job config
  • This includes speaker_diarization_params, new_speaker_sensitivity, segment_boundary_sensitivity

Known Limitations

Issue ID	Summary	Detailed Description and Possible Workarounds
REQ-1409	Proteus HCL with <unk> causes out of memory error	A custom dictionary list that contains the word '' causes the worker to crash.
REQ-10160	Advanced punctuation for Spanish (es) does not contain inverted marks.	Inverted marks [ ¿ ¡ ] are not currently available for Spanish advanced punctuation.
REQ-10627	Double full stops when acronym is at the end of the sentence	If there is an acronym at the end of the sentence, then a double full stop will be output, for example: "team G.B.."
REQ-10634	Putting "-" as an item in additional vocab configuration will cause the container to fail	Do not enter just a "-" on its own in Custom Dictionary either as an additional vocab item or in the sounds_like property. Hyphens are still supported when entered as part of phrases or words

Supported Platforms

Docker (17.06.0+) running on Ubuntu, Debian, Fedora or CentOS.

Installation

Pull the Batch Container Docker image from the Speechmatics Docker repository.

Pre-requisites

You have a login (URL, username and password) for the Speechmatics Docker repository, and have a Docker environment (version 17.06.0 or above) running.

Related Documentation

• Speechmatics Batch Container Quick Start Guide version 9.0.0
• Speechmatics Batch Container API Guide version 9.0.0

Supported Languages

Below is the complete list of languages supported by Speechmatics:

Language	ISO Code
Arabic	ar
Bulgarian	bg
Catalan	ca
Mandarin	cmn
Czech	cs
Danish	da
German	de
Greek	el
Global English	en
Global Spanish	es
Finnish	fi
French	fr
Hindi	hi
Croatian	hr
Hungarian	hu
Indonesian	id
Italian	it
Japanese	ja
Korean	ko
Lithuanian	lt
Latvian	lv
Malay	ms
Dutch	nl
Norwegian	no
Polish	pl
Portuguese	pt
Romanian	ro
Russian	ru
Slovakian	sk
Slovenian	sl
Swedish	sv
Turkish	tr
Cantonese	yue

Container images are labelled using the following scheme, where language codes adhere the ISO-639 standard:

batch-asr-transcriber-<language>:<version>

For example,

batch-asr-transcriber-en:9.0.0