Introducing Smartcat TQS - The Next Generation of AI Translation

Smartcat AI translation is fast, affordable, and of high quality. Now with the introduction of Smartcat Translation Quality Score (TQS) we’ve added an objective, consistent and continuous measurement layer, designed for non-linguists.

• How do I achieve the highest quality for my AI translations?

• How do I consistently and objectively measure the quality of my AI translation?

• How can I do so in an easy and scalable way, without requiring coding skills or in-depth knowledge of AI?

Q: Any vision for translating a question or search criteria into an AI prompt?

A: Currently, Smartcat's AI Actions operate on a per-segment basis, making it impractical to employ questions or search criteria at this point. However, in future iterations, Smartcat users will have the ability to interact with entire documents, allowing them to engineer prompts that explicitly direct AI Actions to seek answers within the underlying document.

Q: How does this figure in custom libraries/translation memory as a qualifier to determine quality?

A: Translation memory (TM) segments receive a TQS score that is boosted by their pre-approval status. However, the TQS boost is contingent on the segment's leverage. TM segments with 103% leverage receive a higher TQS boost than those with 100% leverage. To achieve a TQS of 100, TM segments must be manually reviewed and confirmed.

Q: Is the score only assessing the machine translation? Once the human checks it it’s assumed it’s “100”? Or could you use it to check the quality of the human post edit if you were the project owner or manager?

A: Yes, that's correct. That's where we want the overall score to be or could you use it to check the quality of the human post edits if you were the project owner or the project manager. So this is yes and no. So even though I've said that we are not measuring the quality of the human translators. What we measure, we measure the confidence of the of the translations. So if we see that the translation score is low meaning that translation quality is not so good. But we also see a few edits to the segment. It means that this is improbable situation, and whoever is supervising it probably should have that segment or set of segments on the radar so we'll be bringing that segment or set of segments to their attention. So we do bring the troublesome segments that are triggered by the TQS core, and the amount of post edits to to the reviewers for the further analysis.

Q: Does TQS work well for segments with tags and placeholders?

A: So what we've seen in our assessment of the technologies, we're testing this on different types of content. We do see when formatting is changed, the TQS score is lowered. So it should be working with tags, because TQS also picks up not only the linguistic aspects of the translations, but also things like formatting. So I'll encourage you to try and see by yourself with your specific specific translation, specific strings.

Q: Does TQS only work in projects with "AI + PE" custom template? Wouldn't it work in old "MT+PE" projects if we switch MT to AI in settings?

A: No, it works on everything essentially. Whenever there is a change to translation in any segment, in any project, with any configuration. We are calculating this translation quality score. When you are reviewing your translation, we show you that the estimated TQS is 100, meaning that if you are as a linguist, are confident that the translation is right. This is why you confirm the segment. We're showing you the highest number. But even with that we are storing the original TQS for each of the segment revisions and we'll be helping you later, when you'll be reusing your translations from TM. Showing that translation quality score in the editor will also show you the history of your edits and translation quality scores there.

Q: Instead of translating say, English into another language. Can the software look at the English someone uses and return something which uses simpler English so people who can speak English a little, can understand what you are trying to say (easy English)?

A:  Yes, this is something that can be implemented in Smartcat. You would prompt the engine specifically to say: I need this text to be much simpler, I need it to be, for people that have basic reading skills in English, for example. So this is definitely something that's possible to do, and not just for English, but also for other languages. 

Q: How is the calculation different from other algorithmic calculations such as BLEU, COMET etc?

A: Great question! We use a combination of different technologies: LLM-based one plus a customized version of COMET score. We also have means to analyze post-editing done in Smartcat to tune TQS further.

Q: Can the video translation also translate the visuals in the video, or will that have to be done separately and paired with the translated audio?

A: No, the images are not processed. You would need to process them separately.

Q: Is this TQS tool already available for SmartCat Unite plan? Or do we need to ask it separately? If available, it'll appear everytime we create a project with AI + PE stages?

A: No restrtictions on access, but paid accounts should request that this be enabled by their CSM.

Q: Persuading clients that human quality translations are worth paying for! 

A: The human in the middle component is without question important today, and will be even more important in the future. It's the function of that human that is shifting. So with automated tools like AI translation and and AI quality scorin, you enable the humans to focus on the areas where they contribute the most. So whether that's having domain expertise in a particular language or language variant or a technology business area or a specific business function like marketing, learning and development sales. Our our goal is to make the most of the humans in in the mix. And truly, we don't see any scenario. We're not hearing of any scenario through organizations like CSA, where humans go away. But it really is the way in which humans work is going to change. So there's definitely opportunity for everybody, ourselves included, to adopt AI tools and really learn them. What is it that I individually can do, using AI to make me better at my job. We're we're actually pretty positive that this is overall a very good thing for the majority of people. And there could be roles common that are not, for example, may be that important right now, like cultural adaptation, where the human interaction will be also very important, and translators will have maybe a role that's more enriching, maybe something that's more interesting than just doing translation. But they'll be using AI to to adapt the content of for different cultures, different lookouts, and that could be something where the human will not be replaced very quickly.

Q: How can it know the quality if it is only 50% processed?

A: The 50% represents the 1st step of a 2 step process. The automatic translaton was 100% completed.

Q: I’d love to see a demo with Articulate eLearning course, and an example of voiceover quality (human-like or robotic?)

A: We had a webinar about this a few weeks back but you can schedule a call with our team and we can demo Artculate and voiceover.

Q: How are the AI-translation scores calculated?

A: We use a combination of different technologies: LLM-based one plus a customized version of COMET score. We also have means to analyze post-editing done in Smartcat to tune TQS further.

Q: I don't see the TQS on my dashboard. Do I have to enable it somewhere?

A: Yes, please reach out to your Smartcat account manager to have this feature enabled.

Q: Is there an upcomming project where you can translate audios directly using another audio file? thank you.

A: You can process audio and video files in Smatcat and the audio will be transcribed.