April 1, 2021

Introducing AI Dungeon Translate

AI Dungeon players can now translate their stories into emojis by just clicking a button.

Latitude is proud to announce AI Dungeon Translate, a novel text-to-emoji tool, available for a limited time to all AI Dungeon players. We built it after conducting extensive analysis in which we found that while many people use emojis in their day-to-day life (e.g. texting, emails), there’s a distinct lack of emojis in long form text. We believe that AI Dungeon Translate can help bridge what we are now calling the ‘emoji gap’.

You can access AI Dungeon Translate in two ways: first, inside of any story there’s a new icon in the actions bar which you can click for translation.

AI Dungeon game screenshot. The user’s action is, “You run away.” The game responded: “You pick up another vase and use it to smash in the face of yet another attacker. You run to the balcony and jump over the side. You must escape. (New line here). You are not a hero.” Below is the toolbar: Flag, World Info, Pin, Edit, Undo, Redo, Restore, Retry, Help, and at the end is a new tool called Spell with a smiley face icon. The Spell icon is circled to highlight the new AI Dungeon Translate feature. — Selecting AI Dungeon Translate from the Action Icons

A few seconds later, the newly translated version will pop-up in a modal!

I Dungeon game screenshot showing results of using the new AI Dungeon Translate feature. It is a modal with the header titled: “Hmm…”, repeating the game’s response: “You pick up another vase and use it to smash in the face of yet another attacker. You run to the balcony and jump over the side. You must escape. (New line here). You are not a hero.” Below is a new line showing five emojis, “Translation: wilted flower, face with monocle, face screaming in fear, world map, person running.” — AI Dungeon Translate Output

Secondly, in the Play and Edit modes you can click on any piece of text and translate it directly from there.

A screenshot of an AI Dungeon game showing the game’s response in edit mode with the options menu inside clicked. The menu options are: Cancel, Undo, Delete, Report, Undo To Here, Spellcheck. The Spellcheck option is highlighted visually with a square to show the new AI Dungeon Translate feature. — AI Dungeon Translate from Edit

It can even translate into complex sentences relatively well. Yeet!

AI Dungeon game screenshot showing results of using the new AI Dungeon Translate feature. It is a modal repeating the game’s response: “You turn to head back in and then pause. Why should you? The king has a dozen men. He doesn’t need you. You could run and start a new life, maybe somewhere without war and violence. ” Below is a new line showing nine emojis, “Translation: top hat, prince, artist palette, person: beard, door, stop sign, globe showing Asia-Australia, triangular flag, prohibited.” — AI Dungeon Translate Output

How it Works

This feature builds on Andrew Mayne’s prior work showing that GPT-3 could be used to summarize movie plots. We realized how emojis can profoundly summarize content and decided to give it a glow up representing novel text with emojis as the clearly next logical step here.

Well, the problem with translating to emojis is it’s rather hard to figure out where each emoji should go. You could do a naive translation which works successfully with English → French to try for a few-shot attempt, but this doesn’t work so well for emojis as we found out from our first attempt:

[Few-shots] English: I'm late for work
Emoji: ⏰ 🏃 💻

English: I'm so full from lunch I might throw up
Emoji: 🤢 🍤
…
English: You turn to head back in and then pause. Why should you? The king has a dozen men. He doesn't need you. You could run and start a new life, maybe somewhere without war and violence.
Emoji: [Out] '💰 👣 🙏 💰 🛑'

Notice how the few-shots leaked into the output and didn’t really know what to do? We can get a bit of a performance boost by including an intermediary step that re-words the initial content to what it’ll need for the final output.

[Fewshots]: English: I'm late for work
Meaning: An alarm clock for time and a person running with a briefcase
Emoji: ⏰ 🏃 💻

English: I'm so full from lunch I might throw up
Meaning: A person with a full belly and a puke
Emoji: 🤢 🍤
…
English: You turn to head back in and then pause. Why should you? The king has a dozen men. He doesn't need you. You could run and start a new life, maybe somewhere without war and violence.
Meaning: [Out] 'A king, a man, a dozen men, a head, a pause, a back, a new life
Emoji: 🤴 🤴 🤴 🤴 🤴'

While it got better by adding a king, it was a bit extra repeating it five times and still didn’t really understand what the longer sentence meant. We realized it needed a little help to figure out how to map emojis to its meaning, so we had GPT-3 generate the emojis in the intermediary step as well.

[Fewshots] English: I'm late for work
Meaning: An alarm clock (⏰) and a person (🏃) running with a briefcase (💻)
Emoji: ⏰ 🏃 💻

English: I'm so full from lunch I might throw up
Meaning: A puke (🤢) and food (🍤)
Emoji: 🤢 🍤

...

English: You turn to head back in and then pause. Why should you? The king has a dozen men. He doesn't need you. You could run and start a new life, maybe somewhere without war and violence.
Meaning: [Out]'A crown (🎩), a man (🧔), a dozen (🎨), a man (🧔), a stop (🛑), a new life (🌏), a war (🏹), a violence (🔪)
Emoji: 🎩 🧔 🎨 🛑 🌏 🏹 🔪'

Low key this is a neat example of how important the prompt is for GPT-3: if the prompt doesn’t focus the model to generate the information it needs in a format that it’s able to parse, it can lead to nonsensical outputs. We think this methodology can be applied to multiple other GPT-3 generation tasks and we’re excited to see where it takes us! With a bit of toying around we’re finally vibing with some really useful results.

And for our friends who prefer emoji-based communication: 👀📝😅🖥️🤔🥺👉👈👁️👄👁️🧠 🤖🥰🎉✨