Workshop: Instantly Dub videos Into Any Language

Join Smartcat for our 45-minute webinar and see how to quickly and easily AI-dub videos in any language with remarkable quality.

Q: Does this work for calls and call recordings? 

A: The transcription service we are using is quite robust and should handle varying sound qualities. It is advisable to try it first to be sure. As the webinar mentions, this service is primarily targeted at translating videos of good quality, but it should also be fine for translating call recordings.

Q: Do we have to request the Drive and the video dubbing? I don't see them on my dashboard. 

A: Yes, please contact your customer success manager to request access. But it will be activated for everyone soon.

Q: After finishing with the video, can it be exported in a way that I can obtain an embedded code? 

A: Upon the completion of the AI dubbing, you will already have the video downloaded locally. Additionally, a copy of the video will reside on the drive site. You can download the subtitle file as part of the video translation process if needed. The subtitle file is also downloadable and usable.

If you want to share the file, there is an option to do so. Go to the folder on the drive, select the language version of the file you want to share, click on the context menu, and select the "copy link" option. For example, if you have a Korean file, you can copy the link to that file, and then send it to your colleagues. When your colleague clicks the link, they will be directed right to the file you are sharing. There is no need to download the file if you want someone to view it; it can be shared directly within Smartcat.

Q: Is there any additional cost to use the dubbing “feature”? 

A: There are no additional requirements. As long as you have a subscription to Smartcat, you'll have access to the feature.

Q: Does the translation only work for the script or will it change the wording within the video, if you have text on screen in the video? 

A: No, but we have plans with Smartcat to introduce image translation. This will allow you to upload images, and we will translate the text on the images. Additionally, we plan to embark on video text translations in the future, but no promises yet.

Q: Is it possible to hear a preview of the voiceover for the full video – in the translated language - before downloading?

A: It's a good question. We would call it a suggestion. Let us make a note of that, and we will consider and discuss implementing it.

Q: Will this result in two mp4 files, one per language or will the end user be able to toggle between two subtitle options within one video? 

A: No, we don't have that capability.

Q: Can you choose the dubbing voice? 

A: Currently, that's the limitation of the available version. We've analyzed the videos recently uploaded by Smartcat users and found that many videos have only one speaker, typically a male voice. Taking this feedback into account, we've implemented this in the current release. By early Q3, we plan to introduce voice selection, allowing you to choose from libraries integrated into Smartcat. At present, we support voices in 28 languages, and in the coming quarter, we aim to integrate additional service providers for AI voices.

Q: Is it possible to arrange the subtitle timing via drag & drop in the timeline? 

A: For now, we'll be collecting requests to implement that feature.

Q: What is the video file maximum size allowed for the activity? 

A: We artificially limited that to 4 GB for each file.

Q: Is it possible to teach the tool the pronunciation of certain words, for example, the name of a product/a company? 

A: Not at the moment.

Q: Does the platform ensure that the length of the video is appropriate for the length of the translated narration if the narration becomes longer or shorter when translated? 

A: Not at the moment.

Q: Is there any way to slow down the voiceover or prolong pauses between phrases? 

A: So you can map the language into the timeframe of the cue, but this isn't the ideal solution because varying voice speeds in the video can disrupt user experience. Instead, we recommend rephrasing the translated cue to ensure a consistent experience throughout the video while maintaining the meaning. Some solutions use AI, like GPT, to adjust segment duration by rephrasing to be longer or shorter. While this option exists in the legacy Smart Editor, it's not available in the current editor. If you find this feature beneficial or would like it added to the subtitle editor, please let us know so we can prioritize its development.

Q: Do you have voice cloning? And how is the variety of voices? Do you have at least female and male voices for each language? 

A: Currently, we have integrated one service provider, which is 11Labs, known for its voice cloning capabilities. We are exploring the potential to implement voice cloning on Smartcat with 11Labs for videos. However, this process is not straightforward. In the future, we may consider enabling the integration of third-party providers, allowing you to connect your own 11Labs account for cloned voices. This isn't currently a critical priority, and we do not aim to compete with language or service providers for AI voices. Therefore, this feature is not on the critical path for voice warnings at this time.

Q: Can you export the caption files separately from the video file? 

A: Yes, that's one of the options: to download either captions or the video.

Q: Can you invite a marketplace linguist to review the translation quality? 

A: Not at the moment. Right now, you can only invite colleagues who are part of the workspace.

Q: Will the voices improve? They sound a bit robotic. 

A: Yes, they will improve. We are constantly working with our providers to enhance the quality of the voices.

Q: Can you download the subtitles as a VTT file? 

A: Currently, we offer SRT files, but VTT will also become an option soon. Please help us understand the importance of this feature. By default, we generate SRT files, but converting them to VTT shouldn't be a major issue for us. Your feedback will help us prioritize this feature. If you believe VTT is more important, we may consider switching from SRT to VTT for the original transcription process.

Q: Is there lip sync? 

A: Please let us know how important this feature is to you. At the moment, we don't see lip-syncing as very crucial. While it's a nice addition, compared to features like voice cloning, different accents, and automatic adjustment of cue lengths, lip-syncing may provide less value to end users. However, your comments will help us analyze and consider your feedback.

Q: How stable is the dubbing functionality? Is it still in beta? 

A: It's not in beta anymore; it's now fully released. Everyone should have access to it eventually.

Q: Can we edit the naming of files before downloading to keep the same naming as the original used in the course? 

A: If you upload an MP4 file with a specific name used in a course when you download it, it will retain the same name, which should be straightforward.

Alternatively, you can change the file name in the drive after uploading. Once transcribed, enable the renaming option, and upon download, you'll receive the file with a new name. This option is also available.

Q: You mentioned videos could be translated into 280 languages. To confirm, is that for both subtitles and AI dubbing/audio? 

A: That applies to both subtitles and AI dubbing, although AI dubbing is currently limited to a certain number of languages.

Q: Is it possible to upload a VTT/SRT file together with the video, so that the tool doesn't have to transcribe the audio and the manager of the project does not have to proofread the automatic transcription in the first stage of the workflow?

A: Yes, you can do that on Drive. Upload a video and a matching VTT or SRT file with the same filename, and they'll be paired in Drive. From there, you can open them for editing.

After uploading the VTT or subtitle file and the video to Drive with matching names, you can manually select the video when opening the VTT file. This process differs slightly—you begin on Drive rather than with the video upload but achieve the desired result for dubbing.

Q: When will the feature be released? 

A: It’s been released already.

Q: Do you know if the video changes the appearance of characters after translation? I have an example about an animated video and with another AI tool it changed the appearance of the main character according to language, e.g., Chinese appearance. 

A: So we don't change the video content itself, just the dubbing.

Q: Will you be including the voices of children, youth, or elderly persons? 

A: Different providers will offer us various options for that.

Q: OK, sharing the editing platform, but can you share the Drive with colleagues? 

A: Anyone who has access to the workspace will also have access to Drive.