Original size 1140x1600

Learning Stable Diffusion for Creating Game Concept Art

PROTECT STATUS: not protected
Longread translated automatically

Description

Long ago, I drew a character I grew fond of, which sparked the idea for my own game—a 2D platformer with an atmospheric narrative.

Soul Catcher

This game is about the journey of Soul, a dog exhausted by insomnia, through snowy forests and caves. Along the way, he meets many characters in the form of anthropomorphic animals, each carrying a piece of a soul. Soul must collect these pieces by building trusting relationships with the characters to find peace and bring summer to the valley.
big
Original size 3072x1024

Project Goal

In my project, I utilized the generative network Stable Diffusion to train it to draw in my personal style and develop concept art for characters and locations for a game.

During the process, in addition to Stable Diffusion, I also employed: the generative network Midjourney to upscale and enhance image quality; Google Colab for code execution; and Adobe Photoshop and Figma for compositing the resulting images.

Source Images

For the project, I drew nine characters in the form of stylized animals, aiming to maintain consistency in the placement of body parts while incorporating variation in the details.

Style characteristics include a specific character pose, a smooth, sinuous line, general principles of body construction, animal features, and certain clothing attributes.

big
Original size 1456x1057

Source Images

Process

In the preparatory stage before training Stable Diffusion XL, I uploaded the necessary libraries (bitsandbytes, transformers, accelerate, peft, diffusers) and the SDXL DreamBooth Lora into a Google Colab Notebook.

0
1/2

Code with library installation

Next, I will import my dataset into Google Colab. Image resolution: 512×512 I resize it to 256 and display the image previews.

Original size 1671x1039

To train the model, you must describe every imported image. I use the BLIP model to generate captions and associate them with the images.

Afterward, I create a file containing the image captions and use the token «my» in the prompt to train the neural network on my own style.

0
1/2

Code linking descriptions and images

Finally, I move on to model training and prompt writing. In the settings, I set the output to 512 pixels, establish the training iteration count at 500 steps, and save intermediate values every 250 steps.

Original size 1680x1145

Code for model training and prompting

Final Images

The trained model handled the conveyance of the overall style—smooth, curved lines, intertwined textures, vibrating strokes, and repeated animal motifs—quite well.

It also excelled at generating clothing and objects held by the characters.

Original size 3072x1024

prompts: 1. «photo collage in my style, rabbit with a hat» 2. «photo collage in my style, a dog in armor, sword in hand» 3. «photo collage in my style, a dog that is bandaged all over»

Problems arose with inheriting the black and white style. The model used colored backgrounds and colored the character or their clothing.

Moreover, even when specifying «black and white color» in the prompt, the results lacked sufficient overall form quality.

The output contained numerous illogical lines and artifacts, and the results became homogenized, heavily tied to a single image.

Original size 3072x1024

prompts: «photo collage in my style, black and white drawing, side view, …» 1. «… dog in a raincoat» 2. «… a goat in a fur coat» 3. «… dragon in pajamas»

Attempts to clothe characters in a characteristic persona using costumes often yielded excellent results, perfectly suited for concept art. Occasionally, the generation process produced images featuring multiple characters in a single frame.

Original size 3072x1024

prompts: 1. «photo collage in my style, a crocodile in a suit» 2. «photo collage in my style, a horse in a jacket» 3. «photo collage in my style, a dog wrapped in a chain»

Style Development

Original size 2088x1024

prompt: «a tree in my style»

After successfully generating the characters, I decided to test how the model would handle other objects that did not fit the strictly defined template of my dataset. First, I tried generating trees and flowers for the environment, and then the rock pattern for use in assets.

Original size 2088x1024

prompt: «art in my style, background, rock»

Synopsis

In subsequent generations, I designed locations that served as keys to the protagonist’s narrative path.

Sol, having found himself deep within an icy cavern, ventures into the snowy woods with curiosity, where he discovers a cozy cabin concealing frightening underground passages. But at the end of his journey awaits an endless summer and a sunny meadow where he can finally rest.

Learning Stable Diffusion for Creating Game Concept Art
Project created at 11.02.2026
brnd
For us, a startup is a creative and innovative project with great potential for growth. We are dedicated to finding the best projects and helping them refine their concept, develop branding, and create effective promotion strategies. We also invest in production and replication to ensure their success. Are you ready to take your business to the next level? Join our startup incubator program and let us help you grow.
Brand accelerator
Startup ideas
Marketplace shops
We use cookies to improve the operation of the website and to enhance its usability. More detailed information on the use of cookies can be fo...
Show more