Search
Magical automated way of doing animations – with Stable diffusion with Deforum extension

Let’s explore the magical of doing animations – automated and unpredicable.

Yes, you input the text prompt and have general control, but the magical part is, that you let the “black box” of neutral matrices do the unexpected art for you.

These kinds of animations are sure to draw attention and can be used for marketing, music videos, social posts etc.

These animations were done with “txt2img” (text to image) mode, so without reference video and are often times more artistic, because you let the Stable Diffusion to use its imagination.

Tech for this was – “Automatic1111” – you can get it on github + “deforum extension for automatic1111”.

 

 

 

 

Stable diffusion AI – high resolution generated art

What is Stable Diffusion AI?

Stable Diffusion is a deep learning, text-to-image model released in 2022. It is primarily used to generate detailed images conditioned on text descriptions, though it can also be applied to other tasks such as inpainting, outpainting, and generating image-to-image translations guided by a text prompt.

Practically, it is a tool, like we had a abacus -> calculator -> computer -> quantum computer.

Each tool, has expanded capability and options of use. Computers, especially Nvidia graphics cards (RTX series, A series etc.) are adapted to speeding up model training and generating AI generated outputs from the model.

This includes audio, data, pictures, videos.

For Stable Diffusion, we have a publicly available project, on GitHub – like “Automatic1111”. A magical thing is that you can run it locally on your GPU and test various “checkpoints” (models trained on different art). With those, you can extend the SD model with different styles or art, that were additionally trained on different sets of images.

Regarding checkpoints, it is truly wild wild west at the moment, because are so many of them, and you can “merge” them together, so you get, checkpoints that are merges or different merges. So tracking the original checkpoint from a few months ago can be hard. You can google “civitai”, or “hugging face”, and be amazed how many “checkpoints” there are. You can also google “Lora” that are considered “mini checkpoints” that can extended the current main checkpoint that you are using.

Some uses of generated art:

  • Creative Expression: AI-generated art can be appreciated for its aesthetic value and artistic expression. It offers a new form of artistic creation, blurring the boundaries between human and machine-generated artwork. It can be exhibited in galleries, museums, or shared online as a means of artistic exploration and innovation.
  • Design and Visual Communication: AI-generated art can be utilized in design fields such as graphic design, advertising, and branding. It can provide unique and eye-catching visuals for advertisements, logos, packaging, and other design elements. Generated art can offer designers fresh sources of inspiration and assist in creating visually appealing and engaging content.
  • Personal and Social Expression: Generated art can be used as a means of personal expression, allowing individuals to create unique artworks, designs, or avatars that reflect their personality or interests. It can also be shared on social media platforms, fostering creativity, and encouraging artistic collaborations and discussions.
  • Entertainment and Gaming: AI-generated art finds applications in the entertainment industry and gaming. It can contribute to generating game assets, virtual environments, characters, and animations. Generated art can enhance the visual appeal and realism of video games, virtual reality experiences, and augmented reality applications.
  • Data Visualization: Generated art can be employed in data visualization to represent complex data sets in a visually appealing and intuitive manner. AI algorithms can transform raw data into visually striking representations, aiding in the understanding and interpretation of information across various domains like scientific research, finance, and analytics.
  • Concept Exploration and Prototyping: AI-generated art can be utilized as a tool for generating ideas and exploring new design concepts. It can help in the rapid generation of visual prototypes, enabling designers and innovators to iterate and refine their ideas more efficiently.
  • Collaborative Art and Co-creation: Generated art can be used as a collaborative tool, allowing artists and designers to work alongside AI algorithms in a co-creative process. Artists can provide input and guidance to the AI system, shaping the generated outputs and creating unique collaborative artworks.

Checkout the video below to see what are methodologies and techniques to render high-res renders. You have to be familiar with Automatic1111 if you wish to follow the workflow.

If you need interesting, eye-catching art for your marketing campaigns, blog thumbnails …,  email me on matija@2gika.si

 

Here are some random examples – I like to experiment with watercolour drawing and modifying them, or creating different nature scenes – click on image to enlarge it:

Modified watercolor painting - that drew face and cat :)