- September 28, 2022
- AI Projects
Text-to-Pokémon is an AI for creating Pokémon from text. It is possible to enter the name of a character, a celebrity or even a physical description to let the artificial intelligence generate pocket monsters in the style of the famous Nintendo franchise. Find out everything you need to know about this awesome tool, and how to use it!
For many months, AI-based Text-to-Art image generators are experiencing viral success. From MidJourney to DALL-E, these tools automatically create great drawings from just a few words. Learn all about it in our full guide to Text-to-Art AIs.
These artificial intelligence are exploited by graphic artists and designers, and an architect has even used them to create the skyscrapers of the future. However, these programs also spark controversy, such as when an AI-created drawing won an art competition in Colorado.
Additionally, since these AIs are trained on existing images, they can cause copyright issues. This is the reason why Getty has banned this type of content from its image bank…
However, far from the polemics and other debates, machine learning researcher Justin Pinkney from Lambda Labs has created an image synthesizer that will make everyone agree: Text-to-Pokémon.
As the name suggests, this tool uses AI to create a new Pokemon from a text description. It can be simple words, an object, or even the name of a celebrity or famous person.
Just enter a text, and the artificial intelligence model automatically generates a character in pure Pokemon-style drawing from its database of images and captions.
A version of the Stable Diffusion AI trained on Pokémon
Fine-tuning just takes a few hours on a couple of A6000s (I used Lambda GPU cloud of: https://t.co/K7rYN2UwE7)
During training the model learns to produce things in Pokemon style, but retains its previous knowledge for a while, eventually, it overfits the dataset. pic.twitter.com/2k3Ao2QYm4
— Justin Pinkney (@Buntworthy) September 20, 2022
This program is executed with an API on Replicate, the platform for training your own model’s machine learning and running them on the cloud. This is a version of the Stable Diffusion AI image generator, trained specifically on Pokemon.
To create the model, Justin Pinkney has “honed” Stable Diffusion by training it on Pokémon as a dataset. The AI was trained on captioned BLIP images of Pokémon with two NVIDIA RTX A6000 GPUs on the Lambda GPU Cloud for approximately 15,000 steps.
According to the researcher, Stable Diffusion is a great general-purpose model, but getting a certain style of result can be tricky and usually requires a lot of prompt engineering. The “prompt engineering” consists in modifying the texts entered many times until the AI produces a satisfactory result in terms of style or appearance.
Alternatively, the expert explains that ” refining the model itself is a simple approach to focus only on what you want, provided you have data “. Through this process, Text-to-Pokémon only generates Pokémon.
If you want to know more about the Stable Diffusion refinement process for Text-to-Pokémon, Justin Pinkney has posted a full explanation. on GitHub at this address.
A Pokemon Creator Without Limits
While some Text-to-Art AIs set limits and censor certain content, like MidJourney and DALL-E, this is not the case with Stable Diffusion on which Text-to-Pokémon is based. This fully open-source tool has no restrictions.
It is therefore possible to let your imagination run wild, to use famous people as a starting model, and to create as many Pokémon as you want based on anything and everything. Freedom is absolute.
As with all AI image generators, the results can be very successful or on the contrary completely off the mark. Anyway, this tool is extremely entertaining.
As an example on the website, Pinkney used the Yoda character from Star Wars. In addition to fictional or existing character names, it is entirely possible to enter a physical description of the Pokémon that you want to create.
How do I use Text-to-Pokémon?
Creating your own pocket monsters with Text-to-Pokémon couldn’t be simpler. Just surrender on the website and connect with GitHub. You can then enter your text and let the AI generate the image.
You should wait about 56 seconds before the image is produced. To increase the chances of obtaining a satisfactory result, you can choose “4” in the “num_outputs” field to generate four images instead of one.
