People who say that AI can’t be used as a tool to help artists clearly have never tried using AI as a tool. Everything you’ve written here is untrue.
Artists can manually curate unique datasets to create LoRAs. They can draw from their own photographs, drawings, paintings, etc., and then coordinate prompts and parameters to blend their custom LoRAs with other creators’ LoRAs/models/checkpoints to craft something unique. The process can be even more involved with tools like ControlNet, where artists can sketch an outline of the scene by hand. I.e., you can have precise control over where the pencil strokes go.
You’re wrong. You can’t have the same level of control as the person above describes.
Even if you train the model on your own work, it will still be the one generating every “stroke” of the pencil. If trained for it, it will do it based on how you must often do so, but you can’t clearly control it. You can’t control granular details of the process of creating the image. It’s all broad strokes.
I don’t know what your level of experience with art is, but so much of what makes art is tied up in the process of having to think through every little addition you make to an image. And by little addition I don’t mean “let’s add a person here” but “let’s do these 200 individual strokes that make up that person”. The involvement in the process is the point, and when an image is generated for you, you remove so much of the involvement and granular decision making, that the actual point is lost.
It’s like cooking with premade, pre-prepared ingredients. You can pick the dish and put it together with the stuff you buy, but you can’t control the whole process, because you’ve given up that for the sake of speed and convenience, and the dish will be different for it.
With ControlNet, you can control that very finely though. It partially combines computer graphics into the mix which is definitely not easy to get into.
But it’s different from drawing, that is for sure. Different decisions with different outcomes and different possibilities. But you see similar differences with every form of art. You don’t make 200 strokes for a photograph, or for a cut out collage, or algorithmic art, or sculpting. But you do make different decisions that are similar in nature. They shape the end product, and in a way no other person would do exactly the same. You still have to be involved in every step of the process, even if some steps are no longer done by yourself, they are replaced by other decisions.
Again, this is assuming they aren’t just clicking generate and calling it good to go, but if someone is using ControlNet they should be well above that, since the quality of blank generations is often not the best and demands refinement to anyone with an creative outlook.
This is why we shouldn’t confuse AI art, AI assisted art, and other forms of visual art, even if they all end up making an image as the end result. Something can be impressive when drawn, but mediocre if made with AI. Just like painting a scene isn’t the same as taking a photograph of that same scene, even if they end up being visually similar. Everything exists in context.
And yes, you are giving up some creative control for sake of convenience. But the question is how much. A painter that hand crafts their brushes will make a different painting had they used a pre-made brush. But we can agree I think that the creative control they lost by doing so is negligible. Artists generally lean on what they have produced before as references too. 3D sculptors can start with a cube too, but if they’re going to be making a person they will start with a mesh of a human figure, male or female depending on what they want. There is no shame in taking shortcuts in steps where it’s possible, even if it’s commendable if you don’t.
Art takes long and is expensive to produce, it would be unsustainable in this modern day to do everything from scratch every time. And as long as you focus on the parts where your decisions make the artistic intent happen, you can still make something unique and valuable in it’s own right.
That’s a cool visualisation of what kind of visual input you can feed into the process with ControlNet.
And it really makes it clear that what AI images is good for if communicating a general idea.
I think comparing AI generated or Assisted images or videos to photography is probably the closest analogous medium we have, but I think AI images are stort if in-between that and more classical art. You have more control over the more technical aspects of the image, as you can alter those things with big strokes, but you’ve given up too much control to really infused it with artistic intent.
Even when photography, where you are generally limited by reality, you can better infused artistic intent into the picture, because you carefully examine what makes that object of the picture unique.
Even if you try to direct AI models, it limit their scope they will always add whether the most average expression of what they’re adding, because that what it looks for in the training; the commonalities/averages of whatever it was trained on.
Even ControlNet is just a way to claw back a little more control over the process. I wouldn’t actually call the examples I’ve seen of ControlNet to be examples of fine control. I’m struggling to find a way to clearly communicate it, but it’s like the difference between 3D art that is trying to look like 2D, and actual 2D.
There’s always something lost in the translation.
Most artistic disciplines are their own language, and I just don’t think we have a way to communicate that language without actually doing the art, and art requires artistic intent, which I don’t think is possible with the current AI tools. Maybe it will be at some point, but artistic intent and control over the process are so interconnected that the balance becomes very difficult.
While I kind of get what you’re trying to say, I do personally think you have some point that the expression of the AI is more generic, but that’s kind of where the value is. Certain actions, even if you are manually doing everything, are repetitive and provide a low level of artistic expression. Like coloring in large surfaces, background characters, background buildings. It’s impressive to do so yourself and you can even get very good at it, but in a regular scenario they are unimportant by design. Sometimes you just want to get to the core of where your ideas matter. You can also use AI to upscale your own drawings to allow yourself to add more detail and work on a larger scale. I personally find it horrible and demotivating to manually upscale something I’ve drawn already. You’ll be tracing lines for hours if you want to do it well.
For the more detailed things that no AI is sophisticated enough to be guided towards, that’s something I would also draw myself and leave the AI out of, exactly because I want that level of finer control there. To me it’s about using AI for it’s strengths and not as a catch all, just like you don’t use a sledge hammer to kill a fly.
I do disagree with the framing that ControlNet ‘claws back’ control over the process. I see it more as it enhancing the control you already have. Because you are specifically priming the AI with very fine parameters. The amount of information you can encode in a string of text is just miniscule compared to being able to provide a texture that could realistically be 2K in resolution where you have 2048*2048*4(For every RGBA value) = 16777216 individual pixels that you could fine tune. Same thing with image to image, even doing a couple of iterations with that creates possibilities beyond human understanding of scale, same as other art. Now not every one of those permutations will be valuable, but the same can be said about drawn art. And driving it to the valuable creations is what an artist does.
A big part of my process is reflecting on the value the AI added, and whether or not I can still call something my own by the end of it. I even compare images I’ve completely made myself to the ones that I produced with the AI to ensure that. Especially when I started out I binned some ideas because I didn’t feel they were expressive enough. To me it is a requirement to be happy with what I created. And I think that’s something a lot of people understand in their own way. So I guess we must agree to disagree on that, based on our different experiences.
People who say that AI can’t be used as a tool to help artists clearly have never tried using AI as a tool. Everything you’ve written here is untrue.
Artists can manually curate unique datasets to create LoRAs. They can draw from their own photographs, drawings, paintings, etc., and then coordinate prompts and parameters to blend their custom LoRAs with other creators’ LoRAs/models/checkpoints to craft something unique. The process can be even more involved with tools like ControlNet, where artists can sketch an outline of the scene by hand. I.e., you can have precise control over where the pencil strokes go.
The tools available right now do that
I’ve used it and continue to research it.
You’re wrong. You can’t have the same level of control as the person above describes. Even if you train the model on your own work, it will still be the one generating every “stroke” of the pencil. If trained for it, it will do it based on how you must often do so, but you can’t clearly control it. You can’t control granular details of the process of creating the image. It’s all broad strokes. I don’t know what your level of experience with art is, but so much of what makes art is tied up in the process of having to think through every little addition you make to an image. And by little addition I don’t mean “let’s add a person here” but “let’s do these 200 individual strokes that make up that person”. The involvement in the process is the point, and when an image is generated for you, you remove so much of the involvement and granular decision making, that the actual point is lost.
It’s like cooking with premade, pre-prepared ingredients. You can pick the dish and put it together with the stuff you buy, but you can’t control the whole process, because you’ve given up that for the sake of speed and convenience, and the dish will be different for it.
With ControlNet, you can control that very finely though. It partially combines computer graphics into the mix which is definitely not easy to get into.
But it’s different from drawing, that is for sure. Different decisions with different outcomes and different possibilities. But you see similar differences with every form of art. You don’t make 200 strokes for a photograph, or for a cut out collage, or algorithmic art, or sculpting. But you do make different decisions that are similar in nature. They shape the end product, and in a way no other person would do exactly the same. You still have to be involved in every step of the process, even if some steps are no longer done by yourself, they are replaced by other decisions.
Again, this is assuming they aren’t just clicking generate and calling it good to go, but if someone is using ControlNet they should be well above that, since the quality of blank generations is often not the best and demands refinement to anyone with an creative outlook.
This is why we shouldn’t confuse AI art, AI assisted art, and other forms of visual art, even if they all end up making an image as the end result. Something can be impressive when drawn, but mediocre if made with AI. Just like painting a scene isn’t the same as taking a photograph of that same scene, even if they end up being visually similar. Everything exists in context.
And yes, you are giving up some creative control for sake of convenience. But the question is how much. A painter that hand crafts their brushes will make a different painting had they used a pre-made brush. But we can agree I think that the creative control they lost by doing so is negligible. Artists generally lean on what they have produced before as references too. 3D sculptors can start with a cube too, but if they’re going to be making a person they will start with a mesh of a human figure, male or female depending on what they want. There is no shame in taking shortcuts in steps where it’s possible, even if it’s commendable if you don’t.
Art takes long and is expensive to produce, it would be unsustainable in this modern day to do everything from scratch every time. And as long as you focus on the parts where your decisions make the artistic intent happen, you can still make something unique and valuable in it’s own right.
That’s a cool visualisation of what kind of visual input you can feed into the process with ControlNet.
And it really makes it clear that what AI images is good for if communicating a general idea. I think comparing AI generated or Assisted images or videos to photography is probably the closest analogous medium we have, but I think AI images are stort if in-between that and more classical art. You have more control over the more technical aspects of the image, as you can alter those things with big strokes, but you’ve given up too much control to really infused it with artistic intent. Even when photography, where you are generally limited by reality, you can better infused artistic intent into the picture, because you carefully examine what makes that object of the picture unique. Even if you try to direct AI models, it limit their scope they will always add whether the most average expression of what they’re adding, because that what it looks for in the training; the commonalities/averages of whatever it was trained on.
Even ControlNet is just a way to claw back a little more control over the process. I wouldn’t actually call the examples I’ve seen of ControlNet to be examples of fine control. I’m struggling to find a way to clearly communicate it, but it’s like the difference between 3D art that is trying to look like 2D, and actual 2D. There’s always something lost in the translation.
Most artistic disciplines are their own language, and I just don’t think we have a way to communicate that language without actually doing the art, and art requires artistic intent, which I don’t think is possible with the current AI tools. Maybe it will be at some point, but artistic intent and control over the process are so interconnected that the balance becomes very difficult.
While I kind of get what you’re trying to say, I do personally think you have some point that the expression of the AI is more generic, but that’s kind of where the value is. Certain actions, even if you are manually doing everything, are repetitive and provide a low level of artistic expression. Like coloring in large surfaces, background characters, background buildings. It’s impressive to do so yourself and you can even get very good at it, but in a regular scenario they are unimportant by design. Sometimes you just want to get to the core of where your ideas matter. You can also use AI to upscale your own drawings to allow yourself to add more detail and work on a larger scale. I personally find it horrible and demotivating to manually upscale something I’ve drawn already. You’ll be tracing lines for hours if you want to do it well.
For the more detailed things that no AI is sophisticated enough to be guided towards, that’s something I would also draw myself and leave the AI out of, exactly because I want that level of finer control there. To me it’s about using AI for it’s strengths and not as a catch all, just like you don’t use a sledge hammer to kill a fly.
I do disagree with the framing that ControlNet ‘claws back’ control over the process. I see it more as it enhancing the control you already have. Because you are specifically priming the AI with very fine parameters. The amount of information you can encode in a string of text is just miniscule compared to being able to provide a texture that could realistically be 2K in resolution where you have 2048*2048*4(For every RGBA value) = 16777216 individual pixels that you could fine tune. Same thing with image to image, even doing a couple of iterations with that creates possibilities beyond human understanding of scale, same as other art. Now not every one of those permutations will be valuable, but the same can be said about drawn art. And driving it to the valuable creations is what an artist does.
A big part of my process is reflecting on the value the AI added, and whether or not I can still call something my own by the end of it. I even compare images I’ve completely made myself to the ones that I produced with the AI to ensure that. Especially when I started out I binned some ideas because I didn’t feel they were expressive enough. To me it is a requirement to be happy with what I created. And I think that’s something a lot of people understand in their own way. So I guess we must agree to disagree on that, based on our different experiences.