• Skip to primary navigation
  • Skip to main content
  • Skip to primary sidebar
  • Skip to footer
Codemotion Magazine

Codemotion Magazine

We code the future. Together

  • Magazine
  • Dev Hub
    • Community Manager
    • CTO
    • DevOps Engineer
    • Backend Developer
    • Frontend Developer
    • Web Developer
    • Mobile Developer
    • Game Developer
    • Machine Learning Developer
    • Blockchain Developer
    • Designer – CXO
    • Big Data Analyst
    • Security Manager
    • Cloud Manager
  • Articles
    • Stories
    • Events
  • Sign In
Home » Videos » AI art and design: from deep dream to GANs
Machine Learning Developer

AI art and design: from deep dream to GANs

We take a look at AI and art and design including the evolution of GANs and helpful tools and resources for artists and developers.

Last update June 22, 2020 by

Margaret Maynard Reid is a machine learning engineer at Tiny Peppers. She currently works on research to showcase how AI can be used to create art. “I’m a machine learning GDE. I contribute to TensorFlow, write tutorials, and blog posts and speak at conferences. I am a community leader. I’m the lead organiser of GDE Seattle, and Seattle Data analytics, and Machine learning. This article offers an overview of some of the topics covered including a deep dive into GANs and their application in art and design, you’ll really want to take a look at the video below to watch the presentation and get the full experience, especially of the images!

Loading the player...

Understand the terminology

We hear these words a lot, AI, machine learning, deep learning and computer vision. So what’s the difference?

  • AI is the broadest playing that aims to create intelligent machines
  • Machine learning is one way to achieve that intelligence.
  • Deep Learning is a subset of machine learning, which has become quite popular in recent years.

We can use deep learning, machine learning, or non-machine learning algorithms to solve problems, such as computer vision. For natural language processing, computer vision is one way to interpret images with computers.

Computer vision overview

An overview of computer vision tasks:

  • Image classification: Classify whether an image belongs to a particular one class
  • Image classification (multi-label): Classify and image to multi classes
  • Feature extraction: Extracting latent features of an image with CNN models.
  • Object detection: Identify one or multiple objects within an image and their locations with bounding boxes.
  • Segmentation: Classify whether each pixel of the image belong to a certain class
  • Generative models (GANs): Two or more models trained simultaneously

Generative models to create art and design

Deep Dream (2015)

4ZSWD4L-e1436336191130

Margaret explains, “

Deep dream was an experiment that visualises the patterns learned by a neural network similar to when the child watches clouds tries to interpret random shapes, deep dream over, interprets and enhances the pattern to see in an image. It was really the first time we used a neural network to create some artistic art representation.

Image to Image Translation

In 2015 Margaret and her team were able to perform image style tests for by using convolution on your network feature extraction and apply the style of a style image and to a content image, and then produce a new image.

Generative Adversarial Networks

In 2014 Ian Goodfellow created generative adversarial networks. And since then, there are all kinds of variations of Gans. GANs are fun but difficult to train. Generative adversarial models have at least two network models which compete against each other, like a chess game. It’s a zero-sum game.

Deep convolutional GANs (DCGAN)

One of the earliest kinds of GANs. As Margaret explains, “The network uses random noise as input, then it has a generator that generates images and the discriminator critiques the degenerated images made by the generator, and as a result, teaches his generator how to become better at creating these fake images.

So we have random noises input and then the output are actually the images.”

Pix to Pix

Pix to Pix was introduced in 2016 pics and it was able to convert black and white images to colour images, sketches to high res photos, day to night, etc.

When you do training, you must have paired images which may not be readily available.

Cycle GAN (2017)

Similar to Pix to Pix without the need for paired training images. For example, Cycle GAN can convert between horses and zebras, winter and summer, and then a picture at regular photo converted to a  Van Gogh painting.

Big GAN

In 2018, Google researchers experimented with Big GAN, “which is a large scale game that generates high fidelity class conditional images. One of the main challenges is again, that you don’t have a whole lot of control over the images that you’re generating with conditional GANS that I mentioned earlier, that you can condition based on a particular class or perhaps a text or image. ” However Style GAN was to change things:

Style GAN (2018)

StyleGAN is a novel generative adversarial network introduced by Nvidia researchers in December 2018. It allows for fine control of images. For example, you can create faces based on a particular pose, face shape, age, gender, hair colour, etc.

header_1.fa124956

There are many other variations of GANs such as Image Inpainting which lets you edit images with a smart retouching brush. NVIDIA GPUs and deep learning algorithms make it possible to replace any portion of the image.

Image to text

Fashion++ (2019)

In 2019, Facebook AI came up with Fashion++, capable of recommending fashion changes. Not only is it generating images fashion, but it also may have some sort of sense of whether the changes in the image is fashionable or not.

Sketch2Code (2018)

In 2018, Microsoft AI lab created Sketch2Code. This is a web-based solution that uses AI to transform a handwritten user interface defined from a picture to a HTML markup code, which produces high-resolution design.

Datasets, Tools and Resources

As Margaret notes, “To train machine learning models you will need data” She has included in her presentation (and slides) extensive data resource lists including art and UI and UX design. Some of her examples include the Art Institute of Chicago which provides 5000 free high res images and art data on Kaggle. 

According to Margaret, in terms of UI and UX design, data sets are a bit more limited.  Rico mobile app datasets have UI layouts as well as segmentation to help with mobile application UI design. Kaggle’s 50 icon has 10,000 icons with 50 categories.

Project magenta: This is a project open-source research project by Google that uses machine learning to make music and art. You can find various project demos and code.

RunwayML: Runway ML is a community platform for creatives of all kinds to use machine learning tools in intuitive ways without any coding experience.

Artists and Machine Intelligence: Artists and machine learning intelligence is a programme that Google created to bring artists and engineers together to realise projects using machine intelligence.

Watch the rest of Margaret’s talk above for more resources and tools, including resources around ethics around AI design and designing human-centred AI products. 

Tagged as:GANs

Primary Sidebar

Whitepaper & Checklist: How to Organise an Online Tech Conference

To help community managers and companies like ours overcome the Covid-19 emergency we have decided to share our experience organizing our first large virtual conference. Learn how to organise your first online event thanks to our success story – and mistakes!

DOWNLOAD

Latest

Decoding Adaptive Vs. Responsive Web Design

Decoding Adaptive Vs. Responsive Web Design

Web Developer

How to Implement Data Version Control and Improve Machine Learning Outcomes

How to Implement Data Version Control and Improve Machine Learning Outcomes

Machine Learning Developer

7 Ways to Use UX Design to Enhance User Data Security

7 Ways to Use UX Design to Enhance User Data Security

Security Manager

What are the Main Areas of Development for Programmers to Land Their Dream Job? Codemotion

What are the Main Areas of Development for Programmers to Land Their Dream Job?

Backend Developer

How to Contribute to an Open-Source Project

How to Contribute to an Open-Source Project

Backend Developer

Subscribe to our platform

Subscribe

Share and learn. Launch and grow your Dev Community. Join thousands of developers like you and code the future. Together.

Footer

  • Learning
  • Magazine
  • Community
  • Events
  • Kids
  • How to use our platform
  • About Codemotion Magazine
  • Contact us
  • Become a contributor
  • How to become a CTO
  • How to run a meetup
  • Tools for virtual conferences

Follow us

  • Facebook
  • Twitter
  • LinkedIn
  • Instagram
  • YouTube
  • RSS

DOWNLOAD APP

© Copyright Codemotion srl Via Marsala, 29/H, 00185 Roma P.IVA 12392791005 | Privacy policy | Terms and conditions

  • Learning
  • Magazine
  • Community
  • Events
  • Kids
  • How to use our platform
  • About Codemotion Magazine
  • Contact us
  • Become a contributor
  • How to become a CTO
  • How to run a meetup
  • Tools for virtual conferences

Follow us

  • Facebook
  • Twitter
  • LinkedIn
  • Instagram
  • YouTube
  • RSS

DOWNLOAD APP

CONFERENCE CHECK-IN