AI Artwork, The Road Ahead
By Neil Blevins
Created On: May 8th 2022

If you've been following social media these last few months, you've probably come across a ton of artwork generated by Artificial Intelligence. AI can take two different images and mix them in surprising ways, it can shade a line drawing, or it can even generate artwork by providing it a line of descriptive text. Tons of artists are starting to play with this sort of software, including myself, and so I felt now was a good time to go over some of what these sorts of applications can do, and have a brief discussion of where this may all be leading us.

Note, this is not a lesson on how to use a particular piece of AI software, this is about my experiments with them, giving you some idea of the sorts of things you can achieve with the techniques.

You have two choices with this lesson, watch me discuss the issue in the video below, or read the full text.

Nvidia Canvas

The first piece of AI art software I ever tried was Nvidia's Canvas. The way this software works is you paint a layout painting (seen on the left), where each color represents a thing, like trees, mountains, clouds, etc. For example, the color Purple means a river. Then the algorithm, which has been fed millions of nature images as examples of the result it should aim for, takes that knowledge and makes a final piece of work based on that layout you provided. So as you can see, I painted 3 mountains in the layout, and it creates 3 much more photorealistic mountains in approximately the same place. The software also includes other functions such as lighting styles, so you can switch from day or night, or cloudy or sunset, and it will change the result painting to look like that sort of time of day / weather.


Next comes Artbreeder (I believe called GAN breeder originally). On the artbreeder site, you can take 2 images and mix them together in an interesting way. For this example, I uploaded one of my own paintings, seen here:

It then prepared an image generated from my original painting, and then gave me a number of parameters I could tweak. So for example, one of the "genes" it provided is "river", so if I started increasing the value of river, it would add more rivers to my painting based on all the other paintings the AI algorithm had previously been fed.

Here's a bunch of images all based on my original image, adding genes such as "snow", "architecture" and "mountain". While there's some funky stuff added here, you could totally imagine using one of these paintings as a starting spot for a more detailed painting. And each of these rough paintings were generated in seconds.

In some ways, this sort of software makes the artist more of an editor. You give it some input, some parameters to give it some idea what to do, it can then create 100 ideas in no time at all, and then as the artist you can throw away the ideas that are bad, keep the few good ones, and then paint on top of the good idea to get your final image. But it lets you explore ideas you may not have come up with on your own.

Vizcom's S2R AI Shading Tool

With this software, you supply a drawing, and Vizcom's S2R AI shading Tool will do the shading in 1 click. First image is a rough pen drawing and the 1 click results.

Image 2 is the result with 2 min of extra hand paint, cleaning up and adding some details.

Image 3 is another robot head test.

And the final image is trying it on something flatter. The AI seems to work best on rounded surfaces, maybe because it trained on a lot of car sketches? Still, some neat results with a more rectilinear spaceship.

Disco Diffusion

Disco Diffusion is probably the most commonly used piece of software currently on my social media platforms. It lets you either generate artwork based on just a descriptive line of text, or you provide it an input image, and it warps and modifies the input image based on the descriptive line of text you give it. (here's a great video tutorial on how to use the software BTW)

Here's once again my input image.

And here are some of the things it generated based on the phrase I provided it below each image.

Note that this process is random, so each image uses a different seed value, meaning that if I run the process twice with the same image and text line, it will create a completely new image with the same input.

My next Disco Diffusion test was to simulate some potential client work. Say the client asks you for alien plant life for a film. I provided Disco Diffusion a single seed image of two flowers, and it generated all of these variations based on my text prompt. Text prompts include stuff like tentacles, alien, spikes, mandelbulb, etc

None of these would be perfect as is, but there are elements in each one that I could take and expand upon to create some interesting designs, basically using the visual result they create as photobash elements to my own paintings.

AI, Good or Bad?

So now that I've shown some experiments with AI, it's time to have a good chat about the pros and cons of this technology.

1) The main pro is giving you unique ideas, suggesting shapes you wouldn't normally have thought of. Especially with this current level of AI, nothing it produces is good enough on it's own to be the final product. But it can provide you some great starting spots.

2) The AI does the "boring" part. In the case of the S2R shading tool, you still have to provide it was a drawing, and it colors it in with shading. So if the fun part or creative part for you is the drawing, then the AI has done the boring and tedious part for you. The same with paintings, maybe making the painting photorealistic is the boring part for you. But the part you love is coming up with the idea or general layout. You can paint the landscape layout you want, and then the AI makes it look real, eliminating the part you don't find fun.

Now the cons, or at least issues that we as an artistic community need to discuss.

1) Copyright. Modern concept art has generally skirted the line of copyright, with a lot of concepts being generated using bits and pieces of photos online. While I try as much as possible to only use my own photographs or photos from stock image sites, many people go to google and grab pieces of photos and even other artists paintings and use them as bases for their own work. I have a directory on my hard drive with over 100 images where people have taken my artwork and painted over top of it, used it as a texture, etc. In fact, one of my pieces of scifi art even got taken and had some shall we say Rated X material composited over top. What AIs are currently doing is a variation of this process, taking photos and artwork that may or may not be copyrighted and learning from them to then create a final image. So a big legal question comes up, does using copyrighted material in the process of training an AI model = copyright infringement? Many argue that it constitutes fair use, but if you input say pictures of Mickey Mouse into your algorithm under the idea of fair use, and the AI then chooses to paint a painting that looks a lot like Mickey Mouse, has copyright been violated? These are legal questions that are unlikely to be answered definitively for quite some time.

2) Is it art? This sort of question is nothing new, and in fact anyone who's gone to art school probably had this conversation already. I always liked my friend Tom Dillon's view on this debate, he argued that art is a "Context". So for example, a toilet that's meant to be used by the public before you enter the art gallery is just a toilet. But if the toilet is inside the museum on a little stage, and has a card beside it saying the artist's name, and is meant to be viewed through the lens of art, then that toilet is actually art. Now you can argue whether its good or bad art, but you can't argue whether its art, because it's in the art context. I make concept art for film and games, and many fine artists say that's not art because its made to satisfy a commercial client. But under the context definition, the art I produce that ends up in a film is also art, it's just in the context of "film art", instead of in the context of "fine art".

So is AI artwork actually art? I would therefor argue yes. Whether it's good or bad art, that's a different argument. Since it uses an algorithm, one could argue it lacks intention or human motivation, there isn't really an emotional or intellectual foundation to the art. But others will say that art is in the eye of the beholder, so if the viewer sees meaning, does it matter that the artist isn't capable of adding that meaning in the first place? Also the imagery will activate all the same visual receptors that a human painting will activate. So say what you want, but AI art is eye candy in the same way a painting or even a natural landscape can be.

3) Will we all be out of a job? So while the current AIs I've played with aren't producing anything that can replace us yet, check out this image by an AI called DALL-E, the text prompt it was provided was "A rabbit detective sitting on a park bench and reading a newspaper in a victorian setting".

Now while not the best work of art I've seen, if I saw this while quickly flipping through a magazine, it wouldn't seem out of place. And that means if part of your business is making editorial art for a magazine, that's one job you're going to have a harder time getting.

Many people make the argument that the artwork isn't as good as the best human art, and while I agree, if its just good enough, it may eliminate enough work to make being a graphic artist no longer viable. Remember, say your living expenses are $100,000 a year, if say 20% of your work goes away because of AIs, it's not that you're going to continue to be an artist but with less money, you can no longer pay your bills, which means you are likely to give up being an artist altogether and become something else instead. So even a small decrease in the number of art related jobs can have much larger effects of the viability of the industry and the number of people who can make a living from doing the job.

Now there's many good arguments on why this may not happen.

First off, AIs may get 80% of the way to good enough quickly, but achieving that last 20% may take a long time. Take a look at the self driving car, the engineers went from 0% to 90% reliability in a matter of years. Tons of articles were written about how fully self driving cars being widely used was just a year or two away. Those articles were written a decade ago. I'm not saying self driving cars won't eventually replace human driven cars, just that getting that last little bit has taken way longer than they originally thought.

Another argument my friend made is that most of the work that needs only be "good enough" is already being done by people working for free or "exposure". And so people who are professional artists won't see their career go away because the types of artwork an AI will do are already not the kinds of jobs a professional artist gets.

And lastly, in the past, when technology makes one job obsolete, it frequently creates new job types. Now this may no longer be the case, after all, while new technology has always done this like the car replacing the horse and buggy, the speed at which new technology can be created with today's computers are way faster than ever before. But even if that weren't the case, remember that those "new job types" take time to manifest themselves. So maybe art as a career isn't doomed for everyone long term, but for current artists, it may be a huge problem. Take for example Jurassic Park, which is the moment hollywood realized that effects were no longer going to be made traditionally, they'd be made digitally. In short order, the big FX houses fired like half their employees, and they hunted for people who knew how to use computers. And while there's more jobs now making digital FX than there ever was doing practical FX, a lot of those people who were fired in the original switch over never recovered. For some it took them too long to learn the digital tools, and so spent years unemployed. Remember, paying the rent doesn't stop just because you have to devote years to learning new skills. Or they did learn the tools, but companies weren't interested in hiring them because they were "old school". Or their joy was doing the FX using traditional means, and they just didn't enjoy working digitally. Same thing when the animation industry moved from 2d cell animation to 3d once Toy Story hit. So many people laid off, people who had rent and families.

Now I started my professional career in the early days of digital art, so I missed these big early disruptions, but here I am 20 years later looking at AI, and wondering if this will be my switch from 2d to 3d or practical to digital. Someone mentioned that there will still be plenty of work for artists training the AIs. Maybe a 5 year old of today will say "When I grow up, I want to train an AI!" But for me, I just don't think I'll find much fulfillment in a job like that.

Anyways, after all those cons, you might think I am anti AI, but that is not the case. Whether we like it or not, change is inevitable, and so you have to learn to adapt, and hopefully help shape some of that change in a positive direction. If AIs are here to assist an artist, I am all for it. If they end up replacing the artist, I may have to adapt by learning how to make beer and open up a brew pub. So we'll just have to see where all this stuff goes.

This site is ©2023 by Neil Blevins, All rights are reserved. Twitter Mastodon Bluesky Instagram Blogger Facebook LinkedIn ArtStation Kickstarter Gumroad YouTube IMDB