Introduction

Once upon a time, a fabulous machine learning idea was conceived. Let me introduce you to it. Do you ever remember recording a video of your family enjoying the vue in front of a beautiful landscape and realising, back home, that a ugly electricity pole ruined all your lovely video? Well, now it is possible, thanks to a new machine learning technology, to supress all the ugly objects you don't want to appear in your photos, but also your videos, as if they had never existed! Fantastic, isn't it? Thanks to machine learning, this technology is analyzing the environment of the video, in order to fill the blanks of the object you want to remove with the perfect environment. Thanks to machine learning, this technology is analyzing the environment of the video, in order to fill the blanks of the object you want to remove with the perfect environment.

How does it work ?

There are two main steps :
1) AI has to identify the object you want to remove and follow it in its move. This step corresponds to machine-learning because the machine needs to learn how to recognize and then suppress them
2) Then, the machine suppresses the unwanted object by creating a mask, but instead of letting a dark hole on the video, it analyses the environment of this one to make a continuation to fulfill the blank that the unwanted object let. Incredible! It seems like the object has never existed! This functionality is very impressive because instead of removing every single image of the object, it uses learning machine in order to analyses the frames over time to generate something that doesn’t exist in the video : the environment behind the unwanted object with neighbouring pixels from a seemless frame in the video.

How to create it ?

We can take the example of Adobe after effects
1- Introduction :
You have to use the Content-Aware Fill feature to remove any single object you want from the video. Then, After Effect analyses the image and draw a mask around it in order to replace it.
2- The tools of Content-Aware Fill panel :
Fill Target : with that you can have the preview of the area you want to remove. This area is then outlined in pink.
Alpha expansion : with this tool you can increase the size of the preview area.
Range : you can chose wether to fill layer for the work area or the entire duration of the video.
Generate fill layer : it creates new layers to fill the blank let by the image removed.
Fill Method : you can chose between three types of fill :
- Object : When you want to remove an entire object. As said before, it suppresses the image and fill the hole with pixels took just next to the image and from images back or forward in time.
- Surface : it replaces all the surface of an object
- Edge Blend : it mixes pixels around the area and fills the area with those blended pixels.