Artwork, The Road Ahead
By Neil Blevins
Created On: May 8th 2022
If you've been following social media these last few months, you've
probably come across a ton of artwork generated by
Artificial Intelligence. AI can take two different images and mix them
in surprising ways, it can
shade a line drawing, or it can even generate artwork by providing it a
line of descriptive text. Tons of artists are starting to play with
this sort of software, including myself, and so I felt now was a good
time to go over some of what these sorts of applications can do, and
have a brief discussion of where this may all be leading us.
Note, this is not a lesson on how to use a particular piece of
AI software, this is about my experiments with them, giving you some
the sorts of things you can achieve with the techniques.
You have two choices with this lesson, watch me discuss the issue in
the video below, or read the full text.
The first piece of AI art software I ever tried was Nvidia's Canvas.
The way this software works is you paint a layout painting (seen on the
left), where each color represents a thing, like trees, mountains,
clouds, etc. For example, the color Purple means a river. Then the
algorithm, which has been fed millions of nature images as examples of
the result it should aim for, takes that knowledge and makes a final
piece of work based on that layout you provided. So as you can see, I
painted 3 mountains in the layout, and it creates 3 much more
photorealistic mountains in approximately the same place. The software
also includes other functions such as lighting styles, so you can
switch from day or night, or cloudy or sunset, and it will change the
result painting to look like that sort of time of day / weather.
Next comes Artbreeder
(I believe called GAN breeder originally). On the artbreeder site, you
can take 2 images and mix them together in an interesting way. For this
example, I uploaded one of my own paintings, seen here:
It then prepared an image generated from my original painting, and then
gave me a number of parameters I could tweak. So for example, one of
the "genes" it provided is "river", so if I started increasing the
value of river, it would add more rivers to my painting based on all
the other paintings the AI algorithm had previously been fed.
Here's a bunch of images all based on my original image, adding genes
such as "snow", "architecture" and "mountain". While there's some funky
stuff added here, you could totally imagine using one of these
as a starting spot for a more detailed painting. And each of these
rough paintings were
generated in seconds.
In some ways, this sort of software makes the artist more of an editor.
You give it some input, some parameters to give it some idea what to
do, it can then create 100 ideas in no time at all, and then as the
artist you can throw away the ideas that are bad, keep the few good
ones, and then paint on top of the good idea to get your final image.
But it lets you explore ideas you may not have come up with on your own.
Vizcom's S2R AI Shading Tool
With this software, you supply a drawing, and Vizcom's
S2R AI shading Tool will do the shading in 1 click. First image is
a rough pen drawing and the 1 click results.
Image 2 is the result with 2 min of extra hand paint, cleaning up and
adding some details.
Image 3 is
another robot head test.
And the final image is trying it on something
flatter. The AI seems to work best on rounded surfaces, maybe because
it trained on a lot of car sketches? Still, some neat results with a
more rectilinear spaceship.
Diffusion is probably the most commonly used piece of software
currently on my social media platforms. It lets you either generate
artwork based on just a descriptive line of text, or you provide it an
input image, and it warps and modifies the input image based on the
descriptive line of text you give it. (here's a great video
tutorial on how to use the software BTW)
Here's once again my input image.
And here are some of the things it generated based on the phrase I
provided it below each image.
Note that this process is random, so each image uses a different seed
value, meaning that if I run the process twice with the same image and
text line, it will create a completely new image with the same input.
My next Disco Diffusion test was to simulate some potential client
work. Say the client asks you for alien plant
life for a film. I provided Disco Diffusion a single seed image of two
flowers, and it generated all of these variations based on my text
prompt. Text prompts include stuff like tentacles, alien, spikes,
None of these would be perfect as is, but there are elements in
each one that I could take and expand upon to create some interesting
designs, basically using the visual result they
create as photobash elements to my own paintings.
AI, Good or Bad?
So now that I've shown some experiments with AI, it's time to have a
good chat about the pros and cons of this technology.
1) The main pro is giving you unique ideas, suggesting shapes you
wouldn't normally have thought of. Especially with this current level
of AI, nothing it produces is good enough on it's own to be the final
product. But it can provide you some great starting spots.
2) The AI does the "boring" part. In the case of the S2R shading tool,
you still have to provide it was a drawing, and it colors it in with
shading. So if the fun part or creative part for you is the drawing,
then the AI has done the boring and tedious part for you. The same with
paintings, maybe making the painting photorealistic is the boring part
for you. But the part you love is coming up with the idea or general
layout. You can paint the landscape layout you want, and then the AI
makes it look real, eliminating the part you don't find fun.
Now the cons, or at least issues that we as an artistic community need
1) Copyright. Modern concept art has generally skirted the line of
copyright, with a lot of concepts being generated using bits and pieces
of photos online. While I try as much as possible to only use my own
photographs or photos from stock image sites, many people go to google
and grab pieces of photos and even other artists paintings and use them
as bases for their own work. I have a directory on my hard drive with
over 100 images where people have taken my artwork and painted over top
of it, used it as a texture, etc. In fact, one of my pieces of scifi
art even got taken and had some shall we say Rated X material
composited over top. What AIs are currently doing is a variation of
process, taking photos and artwork that may or may not be copyrighted
and learning from them to then create a final image. So a big legal
question comes up, does using copyrighted material in the process of
training an AI model = copyright infringement? Many argue that it
constitutes fair use, but if you input say pictures of Mickey Mouse
into your algorithm under the idea of fair use, and the AI then chooses
to paint a painting that looks a lot like Mickey Mouse, has copyright
been violated? These are legal questions that are unlikely to be
answered definitively for quite some time.
2) Is it art? This sort of question is nothing new, and in fact anyone
who's gone to art school probably had this conversation already. I
always liked my friend Tom Dillon's view on this debate, he argued that
art is a "Context". So for example, a toilet that's meant to be used by
the public before you enter the art gallery is just a toilet. But if
the toilet is inside the museum on a little stage, and has a card
beside it saying the artist's name, and is meant to be viewed through
the lens of art, then that toilet is actually art. Now you can argue
whether its good or bad art, but you can't argue whether its art,
because it's in the art context. I make concept art for film and games,
and many fine artists say that's not art because its made to satisfy a
commercial client. But under the context definition, the art I produce
ends up in a film is also art, it's just in the context of "film art",
instead of in the context of "fine art".
So is AI artwork actually art? I would therefor argue yes. Whether it's
good or bad art, that's a different argument. Since it uses an
algorithm, one could argue it lacks intention or human motivation,
there isn't really an emotional or intellectual foundation to the art.
But others will say that art is in the eye of the beholder, so if the
viewer sees meaning, does it matter that the artist isn't capable of
adding that meaning in the first place? Also the imagery will activate
all the same visual receptors that a human painting will activate. So
say what you want, but AI art is eye candy in the same way a painting
or even a natural landscape can be.
3) Will we all be out of a job? So while the current AIs I've played
producing anything that can replace us yet, check out this image by an
AI called DALL-E, the text prompt it was provided was "A rabbit
detective sitting on a park bench and reading a newspaper in a
Now while not the best work of art I've seen, if I
saw this while quickly flipping through a magazine, it wouldn't
seem out of place. And that means if part of your
business is making editorial art for a magazine, that's one job you're
going to have a harder time getting.
Many people make the argument that the artwork isn't as good as the
best human art, and while I agree, if its just good enough, it may
eliminate enough work to make being a graphic artist no longer viable.
Remember, say your living expenses are $100,000 a year, if say 20% of
your work goes away because of AIs, it's not that you're going to
continue to be an artist but with less money, you can no longer pay
bills, which means you are likely to give up being an artist altogether
and become something else instead. So even a small decrease in the
number of art related jobs can have much larger effects of the
viability of the industry and the number of people who can make a
living from doing the job.
Now there's many good arguments on why this may not happen.
First off, AIs may get 80% of the way to good enough
quickly, but achieving that last 20% may take a long time. Take a look
at the self driving car, the engineers went from 0% to 90% reliability
in a matter of years. Tons of articles were written about how fully
self driving cars being widely used was just a year or two away. Those
articles were written a decade ago. I'm not saying self driving cars
won't eventually replace human driven cars, just that getting that last
little bit has taken way longer than they originally thought.
Another argument my friend made is that most of the work that needs
only be "good enough" is already being done by people working for free
or "exposure". And so people who are
professional artists won't see their career go away because the types
of artwork an AI will do are already not the kinds of jobs a
professional artist gets.
And lastly, in the past, when technology makes one job obsolete, it
frequently creates new job types. Now this may no longer be the case,
after all, while new technology has always done this like the car
replacing the horse and buggy, the speed at which
new technology can be created with today's computers are way faster
than ever before. But even if that weren't the case, remember that
those "new job types" take time to manifest themselves. So maybe art as
a career isn't doomed for everyone long term, but for current artists,
it may be a huge problem. Take for example Jurassic Park, which is the
moment hollywood realized that effects were no longer going to be made
traditionally, they'd be made digitally. In short order, the big FX
houses fired like half their employees, and they hunted for people who
knew how to use computers. And while there's more jobs
now making digital FX than there ever was doing practical FX, a lot of
those people who were fired in the original switch over never
recovered. For some it took them too long to
learn the digital tools, and so spent years unemployed. Remember,
paying the rent doesn't stop just because you have to devote years to
learning new skills.
Or they did learn the tools, but companies weren't interested in hiring
them because they were "old school". Or their joy was doing the FX
using traditional means, and they just didn't enjoy working digitally.
Same thing when the animation industry moved from 2d cell animation to
3d once Toy Story hit. So many people laid off, people who had rent and
Now I started my professional career in the early days of digital art,
so I missed these big early disruptions, but here I am 20 years later
looking at AI, and wondering if this will be my switch from 2d to 3d or
practical to digital. Someone mentioned that there will still be
plenty of work for artists training the AIs. Maybe a 5 year old of
today will say "When I grow up, I want to train an AI!" But for me, I
just don't think I'll find much fulfillment in a job like that.
Anyways, after all those cons, you might think I am anti AI, but that
is not the case. Whether we like it or not, change is inevitable, and
so you have to learn to adapt, and hopefully help shape some of that
change in a positive direction. If AIs are here to assist an artist, I
am all for it. If they end up replacing the artist, I may have to adapt
by learning how to make beer and open up a brew pub. So we'll just have
to see where all this stuff goes.