Caption Translator with Python

Jesse Olchawa
11 minutes ago
8 min read

Introduction:

Hello there! This post will discuss a handy python script I utilise whilst teaching at De Montfort University. This script is easily run from its executable file and takes in txt files of Panopto captions. Panopto automatically generates English captions from lectures after they are recorded so this handy tool can help increase accessibility of recordings by translating them into other languages students may speak. Its also translates completely for free!

How to use the script:

To use it download the .zip folder below. It contains the executable script file and a test .txt file to see if the program runs correctly. You will need an internet connection as this script utilises Google Translates free API libraries to translate text.

DISCLAIMER: You are responsible for checking the accuracy of the translated content. The accuracy of the English original captions also depends on how well your voice was recorded so proofread your caption file for any mistakes for a better translated result.

Downloading Captions from Panopto:

If your administrator has enabled the default English captions AND you have editing privileges for the video file you can click onto the settings cog icon. Select captions and click on English to download it. If any of these settings are missing you may not be the owner of the file, not have permission or captions have not been created.

Translating:

Once downloaded rename the file a shorter name that makes sense you to like unreal engine demo English.txt. Then open the script and select the output language you desire for this example I have selected Polish as I can natively proofread it too. Click on select caption file and select your English file. Now in the terminal you will get a confirmation that the translation has started. Keep an eye on the timestamp and after 5-8 minutes your file will be complete.

Embedding Captions into Panopto:

Return back to Panopto and to the caption’s menu. At the bottom of the page, you can select the language a caption file should be in and click upload. Now your file will be embedded into the original video.

Making of the Script:

This section dives more into how I made the script and the challenges I faced.

To see the the full script checkout my PasteCodeio here: https://pastecode.io/s/upeezidx

Goals and Structure:

Now my original problem was that Panopto was not generating other captions apart from English. The reality is that Panopto CAN make automatic translated captions, it actually list them on their site as an official feature here: https://support.panopto.com/s/article/How-to-Enable-Translated-Captioning

You may not notice it even uses a machine to translate most likely a provider like Google Translate or some sort of AI. However, if you go down to the fine print it discloses this:

The catch is that the minimum a university would have to spend is $2000 for a credit package. For most demos I teach my recordings stem around 40 minutes. So that is 0.50 cents a video at 2-3 videos a week so $1.50 per week at the highest end. It would take 4000 videos to make that large budget worthwhile so the chances of approving that for me during the term are extremely slim. So, I decided to take matters into my own hands and explore how I can translate these files faster and for free.

My main goals for this script were:

· Be easy to use.

· Be self-contained not requiring any or external sites.

· Be quick to translate.

API and Libraries:

My first idea was using LibreTranslate a free program that you can self-host on a server with a web UI and upload files to translate to and from. As someone who dabbles in servers as a hobby, I was able to setup a docker image of this site to use. I’ve still kept it up as its excellent for translating PowerPoint files without affecting the formatting and image content.

Moving back to the caption file the site did support translating it as it’s a .txt file. So, I uploaded it and hit translate. After I got it back and checked I realised it translated EVERYTHING including the initial two lines of “this is an automatic caption” header from Panopto. The numbers as well started to be weirdly translated into the actual words at certain points. Figuring it was fine I uploaded it to Panopto and was hit by a embed error due to the number issue. This is where I began scripting my Python script to correct the issue outlining patterns in how the caption numbering is generated.

We first need to access the text line by line correctly by reading it. There are different read and write modes, but I used R+ to read and append the lines to my list. By storing all the data, we can open it in write mode W which will have the numbers corrected.

Let’s first calculate how many lines exist in total in a text file to ensure the renumbering function runs as many times as needed. For this I divided the number of captions in a file versus the final number of lines it has so for example a caption file with 636 rows = 159 captions. Sometimes this equation would have decimal values so using ceil() I was able to round up.

Now to correctly add the numbers to over the old caption ones the equation is: line number = i*4-3 apart from the first line 0. For example, the first number 1 is at line 0 in Python. But the numbers after that are located at 5, 9 and 13. Between these spaces the difference is 4. But if we want the location of number 2 so we times 4 by 2 we get 8 for location, so we need to subtract 3 to compensate for the text and spacer in between. Now we can add all that to an array to store where the integer needs to be.

This allows us to overwrite the number until the end of the file and make the captions finally become embeddable back into Panopto.

I also am using TKInter as I have on previous projects to create a button file dialog prompt. This makes selecting files easier for the user. However, at this point I was going against one of my primary goals: being self-contained. The issue with using my server was that is posed making the script useless if it went down or the web address changed. It would be quite annoying to have to manually update as I wanted to share this script to others in my faculty. The service it uses for translation should be reliable.

So, I went looking for other options found out that Google actually offers free queries to their Google Translate service via Python allowing line by line translation.

Using the same line reading logic to get numbers, I imported Googletrans Translator and sent off queries to be translated by breaking down where text would be. The numerical formula for this was easier as I could list the two discrepancies at 2 and 3 at the start. Then to get to the text on line 3 which would become 11 I could times 3 by 4 then take away one. Here’s how it looks in the terminal when working:

I was also surprised at how quickly the responses were and that translations looked accurate for the most part. Unfortunately, since the translations are line by line some contexts are missing or program names like 3DSMax might be jumbled into other words so its not as good as a person but good enough for captions.

Now the script can be self-enclosed and not rely on another site that may go down. Now it was time to polish and implement more things into the script like a language selector before running it through varying caption files for testing.

To allow language selection, TKInter can create combo boxes the values of which return a number of selections. If the string list in this combo box matches another string list of letter combinations such as pl for Polish or zh-cn for Chinese simplified the variable can be passed into the translation line.

Test Run at the University:

I copied my python file over to an external drive and bring it to the office the next day. I had used Python inside Unreal Engine before and assumed this meant the computers had it installed natively. Unfortunately, I was wrong. I could run the script via Visual Studio, but it once again imposes another step the user would need to take. So, I went home and decided to package it instead.

Building an Executable:

After some troubleshooting I managed to install Pyinstaller into my Windows terminal. This allows the easy packaging of any Python script into an executable by running it inside the folder like so: Pyinstaller caption_translator.py. After packaging there are 2 folders made, one for build and another called dist or distribution. The build folder contains settings or configs when you package but the other folder has what you want. The executable! At last, I ran it finally outside of Visual Studio and got a successful translation. Before sharing it however I decided to add a few more things: time to translate and confirmation of program state.

For creating a time to translate I was able to record the time started when the function of translation was called to the and take it out of the finish time to get the total amount of seconds that have passed. This number is very high and not useful, so I divided it by 60 to get the minutes. As for confirming the language and state I added extra print lines that use the combo box selection and put it in a string. The entire function runs in a try file which allows error handling for failure to translate. This prints out a small error message. By removing all the other printing of line-by-line translated text the program worked faster and here is how the final terminal looks like:

Bugs and Limitations:

Through lots of testing and frequent usage I’ve encountered some issues and limitations my script has. The most common being failed translation messages immediately starting. As I had removed a lot of print calls to clean up the terminal the speed of the failure made me think the file was just not being read to begin with. So, I did some testing in Visual Studio code and uncovered that after packaging the script can fail on files with long file names. As part of the script requires it to add a new name to it at the end the string values may be failing prematurely. An easy fix would be to force the new file to have a shorter name but if it is failing on reads it may need to duplicate the file and rename it altogether. However, I haven’t fixed it yet as the failure is not too common and easy to fix manually by renaming the file.

Another odd quirk that I did fix was the timing function. Initially the way the time taken to translate was calculated with a time variable at the beginning of the script. However, this value was not overwritten, it was set on when the script was first run. Due to this if you kept the script open and decided to translate multiple files the time was inaccurate the real time taken by a large value. This was quite easy to fix as it required me to recall the value and update it instead.

Future Improvements:

There are some revisions I do want to add in the future as a bulk translate mode. This would be easy to implement in theory but would need a different UI overhaul to allow selecting folders or multiple files. I could then queue the script to repeat for each file in the array. Another option that would help boost the usability is to allow translating from a different starting language. As Panopto is set to English (USA) by our admin all automatic captions are guaranteed to be in English. However, if those abroad use the same captioning software they may have a different auto caption language. Implementing this would expand the accessibility of this tool to others all around the world.

Final Closing Note:

To conclude I learnt quite a bit regarding translation and handling data from and to text files in this project. The script is in a good functional state but there is still a lot of optimisations that could be applied to get it to work even faster. For example, I could collect multiple strings and separate them into paragraphs to get better translations. This would also speed up the request calls as there would be fewer total requests. However, this would also need a numbering system to decipher where the strings would need to be arranged back into the caption file format to be read properly by Panopto. It would a large rewrite but would promise quicker translation on really long demos.