Using artificial intelligence, one of Rembrandt van Rijn’s most iconic works of art, Militia Company of District II under the Command of Captain Frans Bannick Cocq – better known as The Night Watch – has had missing sections reconstructed to give viewers’ an insight into how Rembrandt’s painting originally looked.

The Night Watch is one of the most famous paintings of the Dutch Golden Age and was completed by Rembrandt in 1642. It was commissioned by Captain Bannick Cocq and seventeen members of the Kloveniers (civic militia guards) for the banquet hall of the newly built Kloveniersdoelen (Musketeers’ Meeting Hall) in Amsterdam. Less than 75 years after the work was created, the painting was moved to the Arquebusiers Guild Hall in Amsterdam’s City Hall. City officials wanted the painting to sit between two doors, however it was too large and they made the decision, which may seem shocking today, to trim the painting down. A panel measuring 60cm in width was removed from the left hand side, 22cm was removed from the top, 12cm from the bottom and 7cm from the right hand side.

Taking advantage of twenty-first century technology, the Rijksmuseum has undertaken a project to restore the missing edges using artificial intelligence. Senior scientist at the museum, Robert Erdmann, used a seventeenth-century copy of Rembrandt’s painting attributed to Gerrit Lundens, which was created prior to the removal of the panels, and the original painting, to reconstruct how Rembrandt’s missing sections may have looked. Erdmann said: “Our attempt here is to make a best guess, without the hand of an artist, into what The Night Watch looked like”.

The director of the Rijksmuseum, Taco Dibbits, explained some of the process to the The Associated Press, saying: “We made an incredibly detailed photo of the Night Watch and through artificial intelligence or what they call a neural network, we taught the computer what color Rembrandt used in the Night Watch, which colors, what his brush strokes looked like.” Dibbits did, of course, understand the limitations of the technique, stating: “Rembrandt would have definitely done it more beautifully, but this comes very close”.

This project has formed a part of the Rijksmuseum’s ‘Operation Night Watch’ – a multi-year, multi-million dollar restoration project which the museum started in 2019. Whilst the use of AI to reconstruct the missing panels was not initially part of the project plan, when eminent Rembrandt scholar Erst van der Wetering suggested it, the museum enthusiastically took up the challenge. The additional AI elements allow visitors to view three figures on the left hand side which were previously missing, a complete helmet on the right hand side, a clearer view of the boy in the left foreground and a repositioning of the entire composition, with the central two figures moving slightly to the right.

The AI panels have been positioned alongside the original work, and will be exhibited at the museum until September.