/
Batch Container
/
Migration - Technical Guide

Batch Container Migration Guide

Overview

This is a guide for customers who are updating to V8.0.0 or later (October 2020). It documents changes in the batch container, and how you, a customer, may need to reintegrate your batch container with any other systems. It is provided in addition to our standard release notes and documentation pack.

As part of this upgrade, some V1 features that are no longer supported have been completely deprecated, and will cease to work as announced in the v6.2.0 release.

In all cases, replacements are supported via our V2 input, and are documented in our Speech API Guide.

The changes below should show no loss of any feature or functionality as a result of the migration.

Scope

The scope of this document shows:

  • What changes you, the customer, must make to use the Speechmatics batch container v8.0.0 if you have been using previous versions of the container
  • If you are still using deprecated V1 features, this document will show which ones are no longer supported, and what you must use instead to ensure output
  • Examples of our V2 output, and how it differs from our V1 output
  • What changes have been made to licensing, and how you, the customer, must license a container prior to using it

The scope of this document excludes

  • How to start the Batch Container - this is documented in our quick start guide
  • Our Speech API - this is documented in the Speech API guide
  • List of software packages used - this is covered in our release and attribution list
  • Recommendations for any custom workflows or integrations you have built

What has changed

License File

Previously Speechmatics built batch containers with their own license file integrated within the container for each language required by a customer. For simplicity and replicability we have moved to a generic customer-agnostic container for each language, with each customer now receiving a separate license file to use with the container(s) they are licensed for.

Please note: The contents of the license file are confidential. They should be shared on the principles of least privilege. Speechmatics is not responsible for how you handle, store, or share licensing information.

Speechmatics Support will provide you with a new license file. The license is a JSON file called license.json and has the following JSON structure:

ItemDescription
Customer nameThis is your company's name
IdThis is internal to Speechmatics
Is-TrialWhether the license is for a trial use of Speechmatics or not
MetadataWhat Features a container is licensed to use. These can include:

Speaker Diarization

Channel Diarization

Speaker Change

Batch Container use

Real-time container use

Language: any supported language

Language: A supported individual language (e.g. English)
NotValidAfterThe date after which the license expires and can no longer be used to run the container. The date is in ISO format
ValidFromThe date from which this license is valid.
Signed Claims TokenA unique reference number used to validate the license file when running the container. Generated by Speechmatics

The values in this license file will reflect each customer's individual contract arrangement with Speechmatics.

An example license file is below:

{
   "contractid": 1,
   "creationdate": "2020-03-24 17:43:35",
   "customer": "Speechmatics",
   "id": "c18a4eb990b143agadeb384cbj7b04c3",
   "is_trial": true,
   "metadata": {
       "key_pair_id": 1,
       "request": {
           "customer": "Speechmatics",
           "features": [
               "MAPBA",
               "LANY"
           ],
           "isTrial": true,
           "notValidAfter": "2021-01-01",
           "validFrom": "2020-01-01"
       }
   },
   "signedclaimstoken": "example",
}

How this affects you

Previously the batch container was licensed by use of the environment variable LICENSE_KEY. This is no longer a valid variable and will not license the product. Instead you may either license the product via the two methods described below:

  • Volume mapping the license file into the container. Volume map the location of the license file into the container when running transcription jobs, like the Configuration Object. Please see below for an example:

docker run -i -v $AUDIO_FILE:/input.audio -v $CONFIG_JSON:/config.json -v /my_license.json:/license.json batch-asr-transcriber-en:8.0.0

  • Use the value of the ‘signed claims token’ from the license file and pass it as the value of the LICENSE_TOKEN variable when running a transcription job. See an example of using LICENSE_TOKEN below:

docker run -i -v $AUDIO_FILE:/input.audio -v $CONFIG_JSON:/config.json -e LICENSE_TOKEN='example' batch-asr-transcriber-en:8.0.0

If you lose a license file or it is no longer secure, Speechmatics can generate a new one. Please contact Speechmatics support if this is the case.

V1 Deprecation

In the Speechmatics container you can still process a media file for transcription without use of the V2 configuration object. This will generate our JSON v2 output without any alteration or changes to the text.

From the V8.0.0 release, the configuration file is now the only way by which you can modify the transcription output in the Speechmatics container. If you want to use features such as diarization, punctuation overrides, output locale etc. you must use the configuration object to request these features.

If you already do so, then you do not need to make any changes to how you use the container.

All JSON transcription output will now be in the V2.4 output.

As part of the v7.0.0 release support for V1 features was withdrawn. As part of this release all V1 features have now since been removed. Where applicable, these have been replaced by options within the configuration object. This includes the following:

V1 ItemTypeReplaced By
DIARIZE. Enables speaker diarizationenvironment variableUse the diarization:speaker parameter within the configuration object
DIARISE. Enables speaker diarizationenvironment variableUse the diarization:speaker parameter within the configuration object
CHANNEL_DIARISATION. enables channel diarization on stereo filesenvironment variableUse the diarization:channel parameter within the configuration object
CHANNEL_DIARISATION_LABELS. Provides labels to different speakers when using channel diarizationenvironemnt variableReplaced by the parameter channel_diarization_labels in the configuration object
LICENSE_KEY. used to license the batch containerenvironment variableReplaced by LICENSE_TOKEN
/extra_words.txt. Used as a custom dictionary to generate additional vocabulary objectstext fileUse the additional vocab parameter within the configuration object to generate a custom dictionary
/build_date. Documents the date the batch container was built bytext fileReplaced by the new licensing file, and no longer needed
/license_days. How many days the license has to runtext fileReplaced by the new licensing file, and no longer needed

Changes to Notifications

Notifications are still supported in the batch container as before. There are a few changes in how single and multi-part notifications are generated and encoded, and this is noted below for integration purposes:

  • If you request transcript, this will now be output in the JSON-V2 format rather than the deprecated V1 JSON format
  • If you want to request an empty notification, you must specify contents to be blank by using []. An example is provided below
  • Notifications now have the charset=utf8 on all transcript types. Ensure that your workflow can support this
  • For receiving notifications, Content-Type header's used to be set always to application/octet-stream . This value now corresponds to actual content of the notification and is application/json in case of JSON-v2 content, text/plain in case of an SRT contetn, and application/octet-stream for TXT content

An example notification configuration that would generate a notification with no contents is shown below. This is a change from the previous version of batch container.

{
    "notification_config": [{
        "url": "http://localhost:8080",
        "contents": []
    }]
}