<aside> 💡

Research and find a project (experiments, websites, art installations, games, etc) that utilizes machine learning in a creative way. Consider the following:

• What type of machine learning models did the creator use?
• What data might have been used to train the machine learning model?
• Why did the creator of the project choose to use this machine learning model?

</aside>

I recently visited the Arte Museum and inside, there was a project called "LIVE SKETCHBOOK Guardians.” This project basically asks us, the participants, to choose a drawing of our liking and then fill the shape in with colors that we desire. After we’re done, we put it under a scanner and it projects it onto a screen. What gets projected the is animal that we choose and the colors that we blocked in also makes up the animal. I had chosen a phoenix and what’s interesting is that the parts where I left blank were just white. I tried doing gradients as well and it translated over pretty nicely too. All it in all, the project is scanning an image and making a 3D render of it.

Some machine learning models that the creators might’ve used was ComfyUI and ControlNet. While I didn’t find much information on the actual project, what I did find was an architect called Hamza Shaikh that guides their sketches and designs through AI iterations, using tools like Krita with ComfyUI and ControlNet. Midjourney is also used to further fine tune details, which I can see being used for “LIVE SKETCHBOOK Guardians” due to the amount of details that I saw from the different options we got to color in.

To train the machine learning model, I think that there’s definitely many different data on how the animals were moving to give it a more realistic feel, even though the animals may be folktale, and giving them the fluidity of movements, different angles and probably an original 3D version given to the machine to learn of the different shapes that belong on the animal. There’s probably also image recognition that can connect it to which model is being used to be render and color recognition as well so that the user’s picked colors are being used to color the animal of their choosing.

The creator of the project chose to use this machine learning model because many already has backed up success of using these model and these models also made it very inter-actable with their audience.

Some video(s) and picture(s):

IMG_6053.mov

Oh, forgot to mention but the squares on the corner is like a restriction to what can be scanned and the camera only scans what’s within the borders.