AI applications and machine learning algorithms are playing an increasingly important role in the IT industry. For example, projects like Tensorflow allow us to exploit a technological stack to create not only AI learning models but also to manage and manipulate data.
TensorFlow is the open source software library for numerical calculation using data-flow graphs. The project was originally developed by the Google Brain Team in the research organisation called Google‘s Machine Intelligence for development of machine learning and research on deep neural networks. The first stable release of TensorFlow was released in February 2017 and has continued to develop rapidly thanks to the huge community developed around this project. There are over 21,000 commits on the Github repository, many of which by external contributors.
TensorFlow have a multiplatform nature and it can be used on almost all GPUs / CPUs, including SoCs of mobile devices and other embedded systems. Google has also created a new platform called TPU (Tensor Processing Unit) an ASIC (Integrated Circuit for Specific Applications) specific for Machine Learning applications studied precisely for TensorFlow.
One of the most important components of TensorFlow is its distributed execution engine, which abstracts away the many supported devices and provides a high performance core implemented in C++ for the TensorFlow platform. On top of that sit the Python and C++ frontends, the Layers API which provides a simplified interface for the most commonly used layers in deep learning models, and on top of that sit higher-level APIs, including Keras and Estimator API. And finally, a number of commonly used models are ready to use out of the box. The structure of TensorFlow is therefore quite simple.
The potential of a similar project is really vast. For example, during Milan Codemotion 2018, Martin Musiol, data scientist for Frog, held a keynote where he illustrated the applications of TensorFlow in the field of deep learning and in particular in style transfer operation. Neural Style Transfer is one of the most interesting and fun techniques that can be exploited through deep learning technologies.
Style transfer is that operation that allows you to combine different styles in an image, basically performing a mix of two images. For example, you can identify the style models present inside a Van Gogh painting and apply them in a modern photo. The result is a mix of style and data that create a unique image. This operation is part of a complex data management system, obviously all of this has a certain artistic relevance. That’s why it immediately caught the attention of many companies in the sector.
So we start from an image that makes “content” and from one that makes “style” to generate a completely new image (generated). To carry out this operation, the algorithms that use this technique are based on a previously trained convolutional network. This concept is called transfer learning, using a previously trained network to perform a task different from the one for which it was initially programmed.
This technique has direct repercussions on graphic design, in advertising and in the fashion sector. Interpolating more styles and more 3D images allows the user to create new styles and unique products completely automatically.
For example, in August 2018, the first self-generated picture from an AI was sold during an auction. Obviously this is possible thanks to the models previously trained through a vast set of data and this makes us immediately realise how important data is During his talk, Martin Musiol put the accent precisely on this aspect, while also focusing on Data Augmentation, the process that adds value to base data by adding information derived from internal and external sources within an enterprise. Data is one of the core assets for an enterprise, making data management essential. Data augmentation can be applied to any form of data, but may be especially useful for customer data, sales patterns and product sales, where additional information can help provide more in-depth insight. So data is one of the main resources available to modern enterprise. Data augmentation can help reduce the manual intervention required to develop meaningful information and insight of business data, as well as significantly enhancing data quality.
These techniques and technologies are used inside Generative AI. This form of AI will drive the next generation of apps for autoprogramming, content development, visual arts and other creative, design and engineering activities.
During his talk, Martin Musiol dove down into the details of Neural Style Transfer. This technique is made up essentially of three ingredients: the autoencoder, the compressed representation and the transformation feature.
The substantial autoencoder takes care of reconstructing the images that the user selects, compressing them and eliminating the imperfections. The autoencoder is essentially formed by an encoder and a decoder that respectively encode the image and then encode it so that it is more detailed, with fewer imperfections and compressed.
The compressed representation allows us to view the images acquired and compressed before the interpolation and transformation into another image.
The transformation feature is the interpolation and transformation action itself. At this point the user must indicate to the algorithm which of the two images is used for the contest and which for the style. This mix process consists of three phases: whitening, colouring, stylisation.
The entire process of Neural Style Transfer can also be performed on mobile devices through TensorFlow Lite, the variant of the main project dedicated to mobile platform. This version was designed to ensure more agility and speed than the main release, inheriting several features of the main branch such the Mobile API. The work necessary for its implementation has focused mainly on the lightening of the package, the speeding up of the initialisation times of the processes, the improvement of the loading times and on the support of hardware acceleration.
TensorFlow Lite allows the access to a Java API that acts as a container for the Android C ++ interface, which loads the TensorFlow Lite Model File and invokes the Interpreter. This last component executes the model using a set of operators. The interpreter supports selective operator loading; without operators it is only 70KB, and 300KB with all the operators loaded. This is a significant reduction from the 1.5M required by TensorFlow Mobile (with a normal set of operators). So the main advantages of using TensorFlow Lite is:
- The C ++ API allows TensorFlow Lite to work on any platform
- Calling up all available operators involves a load equal to just 300kb against the 1.5Mb of the standard version
- Work offline
- Power efficient
- All sensor data accessible on-device
- Low latency (no server calls)
- The data remains on the device and this improves the privacy and data security
Google has also developed TOCO (TensorFlow Lite Optimizing Converter), a graph converter for TensorFlow Lite. This software converts convert and optimise the software for mobile platforms, just as if they were developed through TensorFlow Lite.
