Facebook introduced TextStyleBrush, an artificial intelligence research project that can replicate the style of text in a photograph using only a single word. Facebook said that you can modify and replace text in photographs with this AI model. it further stated this is a research project, it has the potential to open up new avenues for creative self-expression such as personalised message and subtitles and provides the framework for future advancements such as photo-realistic language translation in augmented reality.
Picture demonstrating how the TextStyleBrush transfer style .
Facebook researcher Praveen Krishnan a post doctoral researcher said most AI systems are capable of this for well-defined, specialized tasks, developing an AI system that is flexible enough to comprehend the intricacies of both real-world text and handwriting is a considerably more difficult AI issue. It outperforms state-of-the-art accuracy for any type of text in both automated tests and user studies. Unlike prior systems, which focused on individual characteristics such as typeface or goal style supervision, they took a more holistic approach to training by disentangling the content of a text picture from all other characteristics of its appearance in the full word box. The representation of the overall appearance can then be used as a one-shot transfer on the novel source style samples without retraining.
Praveen Krishnan further stated that by disclosing this information publicly, we intend to stimulate additional study and debate aimed at preventing deepfake text attacks, just like we do with deepfake faces. If AI researchers and practitioners can stay one step ahead of adversaries in developing this technology, we can improve our ability to detect and battle this new breed of deepfakes. While this technology is still in its infancy, it has the potential to power a variety of useful applications in the future, including translating text in images to multiple languages, creating personalized messaging and captions, and perhaps one day enabling real-world translation of street signs via augmented reality.
A innovative, comprehensive method to the acquisition of text style representations
Text style transfer requires training a model on supervised data in terms of source and target content with similar styles, as well as explicit text segmentation. However, developing an efficient text segmentation system for real-world photos is challenging. We make no assumptions about the availability of supervision over the representation of styles or the availability of segmented text labels. In the research Facebook extracted an opaque latent style representation from a recognized text box carrying a source style and improve the representation to enable photo realistic display of additional content in the source style using a single source sample
Second, they extracted the additional issue of stylized text pictures’ distinct nature. Representing text styles requires a blend of global data and particular, fine-scale data, such as the minute variances in individual penmanship. The styled text generator component from the top graphic is described in detail. Indeed, because text styles are virtually endless in their diversity, it is unclear what high-level characteristics may be utilized to capture these styles.
To solve this issue, they offer a novel self-supervised training criterion that preserves both source style and target content. They accomplished this by combining a typeface classifier, a text recognizer, and an adversarial discriminator. They begun by evaluating our generator’s ability to accurately reproduce the style of the input text using a pre-trained typeface classification network. Separately, they examined the content of a created image using a pre-trained text recognition network to determine how effectively the generator captured the intended material.
Reduced impediments to studying Deepfake text
Facebook researcher Praveen said that they are continuing to improve their system by working around some of the constraints they’ve encountered, such as writing written on metallic items or characters with varying hues. And they hope that this work contributes to the continued reduction of hurdles to photo realistic translation, creative self-expression, and the investigation of deepfake text attacks. This research involves expanding beyond synthetic faces to text and collaborating on benchmark data sets such as the Deepfake Detection Challenge. They anticipate that by making their work and methodologies for synthetically generated text styles publicly available, the broader field of artificial intelligence will be able to build on it and achieve cumulative forward progress.
You can read Complete Research paper here.
Source: Facebook Blog