Smithsonian.com

The Repentir app reveals an artist’s creative process by allowing users to peel back layers of paint with the touch of their fingertips. Photo courtesy of Jonathan Hook. Artwork © Nathan Walsh

An artist’s studio is usually a private space, and the hours spent with a paint-dipped brush in hand mostly solitary. So, the final products we gaze at on gallery walls are just the tip of the iceberg when it comes to the makers’ creative processes.

For Nathan Walsh, each of his realist paintings is a culmination of four months of eight to 10-hour days in the studio. Now, thanks to a new app, we can go back in time and see how his work came to be, stroke by stroke.

Repentir, a free app for smartphones and the iPad, provides a hand-controlled time-lapse of Walsh’s oil painting, Transamerica. It compresses months of sketching and revision into interactive pixels, allowing users to peel back layers of paint and deconstruct Transamerica to its original pencil sketches.

The app, developed by researchers at Newcastle and Northumbria universities in England, uses computer vision algorithms to recognize the painting in photographs taken from various perspectives. When you take a photo of any part of Transamerica (or the entire work), the app replaces your image with those captured in the studio as Walsh painted. Every day for four months, a digital camera set up in his York-based studio snapped a shot of his progress, accumulating roughly 90 images.

Researcher Jonathan Hook demonstrates how to use the Repentir app in front of Nathan Walsh’s Transamerica. Photo courtesy of Jonathan Hook. Artwork © Nathan Walsh

Users can view the painting’s layers in two ways. A slider feature at bottom allows viewers to see the piece in its beginning stages to the final product by swiping from left to right (think “slide to unlock”). They can also use their fingers to rub away at a given spot on the painting on the screen, revealing earlier stages in the process.

“Where their fingers have been, we basically remove pixels from the image and add pixels from older layers until they’re rubbed away,” says Jonathan Hook, a research associate at Newcastle who studies human-computer interaction. “It’s like how you add paint to the canvas—we’re doing the opposite.”

Repentir was unveiled this week at the ACM SIGCHI Conference on Human Factors in Computing in Paris, an annual science, engineering and design gathering. This year’s theme is “changing perspectives.” Transamerica will be on display there until tomorrow, when it moves to the Bernarducci Meisel Gallery, a realist painting collection in New York.

But you don’t have to visit the gallery to test out the app for yourself—you can pull up this print of the painting and snap a shot of your computer screen.

Realist painter Nathan Walsh drew inspiration from a visit to San Francisco’s Chinatown to create Transamerica, which took nearly four months to complete. © Nathan Walsh

The app relies on a process known as scale invariant feature matching, technology that’s similar to that of augmented reality. Researchers trained the app against a high-resolution image of Transamerica to identify and create markers for certain features. These markers can then be used to find matching features in a user’s photo of the painting and the artwork itself—even in a tiny piece of it.

“If you take a picture of the bottom right-hand corner, it will find the features in the bottom right-hand corner of the image and match them against those same features in the source image,” Hook says. “If there’s at least three or four features matched, you’re able to work out the perspective and the difference in image position on those features.”

Ninety images worth of layers may not sound like a lot when you factor in today’s smartphone scrolling speeds, but if you’re viewing Transamerica in person, there’s more than enough of it to explore. The canvas measures roughly 71 by 48 inches. It would take a massive number of screen grabs to rub away the layers of the entire work.

Transamerica is a colorful composite of elements that caught Walsh’s eye during a trip to San Francisco’s Chinatown, the largest Chinese community outside of Asia. Several years ago, Walsh traveled across America, stopping in major cities, including San Francisco, New York and Chicago, sketching and taking photographs of the urban landscapes.

Walsh spends about a month on sketching alone before he begins adding paint to the canvas. Here, Transamerica is in its beginning stages. © Nathan Walsh

Walsh says he’s often accused of stitching photographs together or touching up in Photoshop because of the realistic look of his paintings. He aims to convey a sense of three-dimensional space in his work. In Transamerica, the juxtaposition of different objects and designs create almost palpable layers of paint.

“There’s always an assumption that there’s some sort of trickery involved,” Walsh says. “Getting involved in a project like this explains literally how I go about constructing these paintings. It shows all the nuts and bolts of their making.”

Hook says the researchers chose Walsh’s work to expose those “nuts and bolts.” “Lots of people, when they see his paintings, they think he’s cheated, when in reality what Nathan does is just get a pencil and a ruler and draws these really amazing photorealistic pictures from scratch,” he says. “The idea behind the app was to reveal Nathan’s process and show people how much hard work he does.”

In this way, Walsh believes using Repentir in front of the actual work will make the gallery experience more educational for visitors. “For me, the exciting thing is that you’re getting close, as close as you can, to my experience of making the painting,” he says.

While the app is free, Hook believes the tool could lead to a new business model for artists. In the future, app users could purchase a print of a configuration of layers they like best.

A difference of nearly four decades: at top, a ski area in Aspen, Colorado last year, captured by Ron Hoffman; at bottom, the same location in 1974, shot by Dustin Wesley. Credit: US EPA

In 1971, about 70 photographers, commissioned by the newly formed Environmental Protection Agency, set out to document the American landscape on just 40 rolls of film each. They trudged through coal mines and landfills, traversed deserts and farms and discovered big cities’ small corridors. The end result was DOCUMERICA, a collection of more than 15,000 shots capturing the country’s environmental problems—from water and air pollution to industrial health hazards—over six years.

Decades later, a new generation of photographers is collecting ”after” pictures. In the past two years, the EPA has collected more than 2,000 photos, all of which loosely depict the environment. The State of the Environment Photography Project, as the effort is called, asks photographers to take shots that match scenes from DOCUMERICA, to show how the landscape has changed since the 1970s. It also asks photographers to capture new or different environmental issues, with the idea that these modern scenes could in turn be re-photographed in the distant future; the EPA has released several of these shots for this year’s Earth Day. The project will accept submissions through the end of 2013.

The EPA explains that DOCUMERICA became a baseline for America’s environmental history, and that tracking change is key for public eco-consciousness.

Both images, taken by Michael Philip Manheim, show a section of East Boston in the 1970s and present day. Decades ago, rows of triple-deckers lined the streets of the neighborhood. Today, only one remains, the sole survivor of nearby airport expansion. Credit: Michael Philip Manheim/US EPA

There’s more to capturing environmental issues on camera than shooting smoke stacks and nuclear plants. The most effective way to convey them is to photograph people, says Michael Philip Manheim. Manheim, one of DOCUMERICA’s photographers, documented noise pollution in East Boston in the ’70s, portraying the deterioration of a close-knit community as nearby Logan Airport expanded its runways. That’s what made DOCUMERICA strike a chord with the public years ago, providing closeups of miners suffering from black lung and kids playing basketball in cramped housing developments.

“Meet the affected people, let them know how you care, find out what impacts them the most,” advises Manheim about matching his photos today. He still has the cameras he used for his assignment, which he treats as “sculptures” that stay hidden in closets. “After that, it’s time to energize a camera, and not by posing pictures but by reacting candidly to what is going on in the lives of your subjects.”

At left, DOCUMERICA photographer David Falconer’s shot of the Weyerhaeuser Paper Mills and Reynolds Metal Plant along the Columbia River in Washington State. At right, Craig Leaper’s re-creation. Credit: US EPA

Though some landscapes remain the same, Manheim says what’s changed since DOCUMERICA is the level of awareness of environmental issues. The photographer attributes this increase to the rapid spread of digital information, a visual online petition that he says Bostonians could have used to fight back in the 1970s.

At left, the Great Falls of Maine’s Androscoggin River, with the city of Lewiston in the background, captured by Charles Steinhacker in 1973. At right, a replication of the same scene by Munroe Graham. Credit: US EPA

The “now” and “then” photos show varying degrees of change when placed side-by-side, funky fashions and clunky cars aside. Clumps of unnatural foam continue to bob along polluted waters near industrial buildings, but considerably less smog hangs in the air of some urban cities. In an “after” shot of a section of John Day Dam between Oregon and Washington State, a set of wind turbines appear on the background terrain.

At left, the John Day Dam viewed from the Washington side of the Columbia River, photographed by David Falconer in 1973. At right, a similar view, including wind turbines along the ridge, taken by Scott Butner in 2012. Credit: US EPA

The ease of digital photography will help propel the current iteration of an environmental snapshot, Manheim says. When shooting on film, photographers can’t know right away whether they’ve taken “the shot.” Digital allows them to examine the first few shots of a scene, and then find better ways to convey its details.

“You don’t stand around, waiting for something to happen. You exert mental and physical energy,” Manheim says. For anyone wanting to participate in the State of the Environment project, the photographer has some advice: “Set the scene in your coverage, and then you go for the ‘good stuff.’ You get close, closer, closest. You move in to explore and find the epitomizing image, close and meaningful, that symbolizes the situation.”

In the 1970s, Manheim got to know the people who lived in the colorful triple-decker row houses lining Neptune Road in East Boston. Planes soared overhead nearly every three minutes, prompting the nearby residents to cover their ears from the deafening roar of the engines. He captured one of these low-flying planes in a photograph, shown above. In 2012, Manheim returned to the site to document it yet again. The “then” and “now” pairing tells a story that has played out over decades. Eventually, the adjacent airport built runways flush to the streets’ backyards and driveways, and today, only one home remains.

South Boston’s Moakley Park. At left, Ernst Halberstadt smog-heavy shot in 1973; at right, Roger Archibald’s 2012 take. Once a muralist for the Works Progress Administration (WPA), Halberstadt documented city life in Boston for DOCUMERICA. Credit: US EPA

For Nathalie Miebach, the stars aligned with this sculpture, inspired by a Hertzsprung-Russell diagram. © Nathalie Miebach

In 2000, Nathalie Miebach was studying both astronomy and basket weaving at the Harvard Extension School in Cambridge, Massachusetts. She was constantly lugging her shears and clamps with her into the room where she’d study projections of stars and nebulas on the wall.

Understanding the science of space could be tricky, she found. “What was so frustrating to me, as a very kinesthetic learner, is that astronomy is so incredibly fascinating, but there’s nothing really tactile about it,” says Miebach. “You can’t go out and touch a star.”

Soon, something in the budding artist clicked. Her solution? Turn space data into visual art, so that she and other learners like her could grasp it.

Miebach’s final project for her basket weaving class was a sculpture based on the Hertzsprung-Russell diagram, a well-known astronomy scatter plot measuring stars’ luminosities against their surface temperatures. Temperature readings travel downward from left to right, and the wider the diameter of the star, the higher the luminosity. The graph is used to track stars as they evolve, showing how they move along the diagram as shifts in their structure cause changes in temperature, size and luminosity.

Miebach translated the relationship between star luminosity and temperature into a thick, funnel-shaped sculpture (shown above) with tightly interwoven reeds. She uses the temperature and luminosity values of specific stars on the diagram to inform the manner in which she weaves  the reeds.

Basket weaving involves a three-dimensional grid with vertical spokes that create structure and horizontal weavers that fill in the sides of the work. The sculpture achieves its shape through the interaction of the materials—usually, straw, grass or reeds—and the amount of pressure exerted on the grid by the artist’s hand.

“Antarctic Explorer – Darkness to Lightness” © Nathalie Miebach

Miebach’s next project involved transforming scientific data of solar and lunar cycles into sculpture. In the piece pictured above, the artist transferred three months of moon, twilight and sun data from Antarctica into layers of woven reeds. She assigned the vertical and horizontal reeds of the basket grid specific variables, such as temperature, wind and barometric pressure. Changes in these variables naturally altered the tension exerted on the reeds, and the varying tensions created bulges within the piece. The changing values of these variables distorted the tension between the reeds, driving the warped shapes that emerged in the piece.

Reeds are not unbreakable; if too much pressure is exerted, they snap. If Miebach used wire, she’d be completely in charge of the process, and no tension would exist to guide the piece into its final shape.

“Because these cycles change every day, you are working this grid in different ways,” she says.

The thick, ribbon-like blue lines circumventing each bulge are segmented into hours of the day. The naturally colored reeds representmoon data, the yellow reeds sun data and the green reeds twilight.

The yellow spheres on the exterior of the shape signify sunrise and the smaller navy balls represent moon phases. The orange spokes protruding from each bulge of the sculpture represent solar azimuth, or the spherical angle of the sun, and solar hours, which measure the passage of time based on the sun’s position in the sky. Red spokes designate the ocean’s high tide and yellow spokes, the low tide. The basket grid becomes a pattern representing the changes of these variables.

How elements like wind, temperature and barometric pressure, assigned to vertical spokes based on values from low to high, look in a woven representation of two months of Cape Cod weather. © Nathalie Miebach

This weaving process remained the same when Miebach’s subject changed from sky to sea during an artist residence on Cape Cod several years ago. Armed with basic measuring tools like thermometers purchased at the hardware store, Miebach studied the Gulf of Maine every day for 18 months, checking and recording temperature, wind speeds, barometric pressure and other climate indicators. She gleaned additional data from weather stations, satellites and anchored buoys bobbing up and down in open water.

“Changing Waters” portrays meteorological and oceanic interactions within the Gulf of Maine. © Nathalie Miebach

The result was multiple woven sculptures examining different aspects of the Gulf of Maine. A 33-foot-wide wall installation called “Changing Waters” (pictured above) depicts the geography of the gulf. The blue material represents its currents, streams and basins, delineated by changes in the water that Miebach recorded and assigned to each tiny segment.

“To Hear an Ocean in a Whisper” (pictured below) examines the effects of currents, temperature and tidal patterns on krill living in the Georges Bank of the Gulf of Maine. The roller coaster represents the Labrador Current, which flows from the Arctic Ocean and along Nova Scotia’s eastern coast. The merry-go-round inside shows how krill activity changes as temperature, salinity and wave height vary, and the Ferris wheel tracks the diurnal cycle of the tiny crustaceans. A swinging ship-style ride follows the tidal patterns of the Bay of Fundy on the northeast end of the gulf and nearby whale sightings.

“Everything is some sort of data point,” Miebach says. “There’s nothing there just for whimsy or aesthetic purpose only.”

“To Hear an Ocean in a Whisper.” © Nathalie Miebach

The artist has taken this same approach with her latest project: translating scientific data into musical scores. When Miebach relocated from the coast of Maine to Omaha and then Boston in 2006, she realized the cityscape influenced weather dramatically, and not in the same way that the shoreline did.

“In an urban environment, you have infrastructure, you have heat bubbles that hover over cities, you have the lack of vegetation, and all these create very localized fluctuations in weather data that the weather instruments are very sensitive in picking up,” she says.

Miebach found that she could not accurately express in her basket weaving the subtle fluctuations in weather that cities foster. Instead, she began experimenting with musical notation as a medium, which she says provided the flexibility she needed in artistically representing weather data at the street level.

“Navigating Into a New Night” © Nathalie Miebach

In the score pictured above, the royal blue squiggly lines represent cloud cover. The notes signify weather variables: orange is humidity, red is temperature and green is barometric pressure. The sky blue lines zigzagging across the sheet indicate wind direction, and the pink shading represents tempo for musicians to interpret.

Interpreting scientific data in this way allowed Miebach to translate the nuance of weather she felt was present in a city environment without altering the information in any way. “One thing that has been very dear to my heart from the very beginning is that I don’t change information for any aesthetic purpose,” she says. “I want the information to stay true, so that when you look at the sculpture, you’re still seeing the weather.”

Musical score for Hurricane Noel. © Nathalie Miebach

In her musical score for Hurricane Noel, which swept along the Atlantic Ocean in 2007, Miebach correlated each change in a given weather variable she had measured with a note on the piano keyboard. The piano scale is drawn as black-and-white column on the left-hand side of the sheet music (pictured above). Shaded regions represent shifting cloud cover during the storm.

Miebach says she transposed wind speed into the upper two octaves because howling winds are a dominant aspect of any storm. Each note on the scale receives a range, from zero to two miles per hour, two to four miles per hour and so on. The same goes for temperature and barometric pressure readings.

The Nineteen Thirteen, a group of cellists and percussionists, performed Hurricane Noel at the Milwaukee Art Museum in 2011 (listen to the ominous-sounding song here). Another cellist group offered up a different interpretation.

But transforming the musical scores into live performances isn’t the end. Once she feels that she has captured the nuances of weather data from urban settings, Miebach then uses her melodious blueprints to create woven sculptures such as the one pictured below.

What Hurricane Noel looks like in three-dimensional music. © Nathalie Miebach

The amusement-park themed “To Hear an Ocean in a Whisper” that Miebach made in collaboration with Jon Fincke, an oceanography graduate student at MIT, is on display in “Ocean Stories: A Synergy of Art and Science,” an exhibition at Boston’s Museum of Science through June 2. Her latest piece, “The Last Ride,” translates weather and ocean data from Hurricane Sandy, which destroyed the Jersey Shore’s Star Jet roller coaster. It will be featured in the Massachusetts College of Art and Design’s annual art auction on April 13.

This origami structure, called “Green Cycles,” by Erik Demaine and his father Martin required a week of improvisation to assemble. Credit: Renwick Gallery

The shape of a Pringle, mathematically speaking, is called a hyperbolic paraboloid. Artists have been folding paper into this shape for years. The twist? Hyperbolic paraboloids shouldn’t exist in origami—it’s impossible to make such a 3D shape using only the creases pressed into paper by hand.

By that logic, some of Erik Demaine’s artwork shouldn’t exist either.

Demaine, the world’s top computational origami theorist, has created a series of sculptures by folding concentric squares into square pieces of paper, alternating mountain and valley, and folding the diagonals. With each sculpture, the paper pops into a saddle shape called a hyperbolic paraboloid and stays there. Its accordion-like folds are pretty to look at, but Demaine, a computer science professor at MIT, isn’t sure how it works.

A paper structure featuring multiple hyperbolic paraboloids. Credit: Erik Demaine

Once the paper is folded, the entire structure settles into a natural form. “Physics finds that balance,” Demaine says. But, the mechanisms of the Pringle-like shape are still poorly understood. Demaine posits there must be little creases in the paper invisible to the naked eye, as handmade folds alone can’t account for the end shape.

Trying to solve this mystery means marrying sculpture and mathematics.

“We’ve come up with a math problem that inspires new art—and an art problem that inspires new math,” says Demaine. The 31-year-old artist creates his origami sculptures with his father Martin.

The final product, “Green Cycles” (pictured at top), was created using two different colored sheets of French-made Mi-Teintes watercolor paper, bonded together. Using a ball burnisher, which is essentially a ballpoint pen without the ink, the Demaines pushed the two-layer sheet into rings of concentric circles carved into a wood template. The paper is scored along the circular creases and cut into a donut shape, before it springs into a three-dimensional form. The artist creates several of these models and loops them together into an interlocking paper sculpture. The younger Demaine says the hardest part is assembly, which takes up to a week, because they can’t predict if the resulting shapes will twist around one another to create a solid, aesthetically pleasing piece.

“We get them to interlock, let go and let them relax, sometimes overnight, if we think we have a candidate sculpture,” he says. If the structure droops or falls apart, the pair tries again.

“Natural Cycles” by Erik and Martin Demaine

Written instructions for paper folding first appeared in 1797 in Japan. Akisato Rito published a book, Sembazuru Orikata, with lessons for 1,000 paper cranes. Adachi Kazuyuki published a more comprehensive how-to collection in 1845. By the late 1800s, kindergarteners across Europe began folding colored squares in class.

The concept was simple: no scissors, no glue, no tape—just nimble fingers bending and twisting paper into novel shapes. Origami became a modern art form in the 1950s, when Akira Yoshizawa, a Japanese artist, combined the mechanics of the craft with the aesthetic of sculpture. He created more than 50,000 paper models, never selling one. Since then, artist Eric Joisel’s crinkled lifelike animal and human figures appeared on display at the Louvre and physicist-artist Robert Lang’s detailed compositions have been exhibited at the Museum of Modern Art.

But paper folding doesn’t just create something we can ooh and aah at. It also plays a role in answering long-standing questions in math, like the fold-and-cut problem.

The first known record of the problem appeared in 1721 in a Japanese book of brain teasers, one of which asked the reader to fold a rectangular piece of paper flat and make only one straight cut to produce a Japanese crest called sangaibisi, which translates to “three-folded rhombics.” The author offered a solution through a diagram, but the problem remained an open question for centuries—how many shapes are possible?—until Demaine solved it.

As it turns out, any shape is possible—swans, horses, five-pointed stars. All that’s needed is a geometric blueprint, a guide about folding here and bending there.

“Hugging Circles” by Erik and Martin Demaine

The use of such blueprints added complexity to origami. In the 1960s, folding diagrams involved 20 to 30 steps. Now, a model could require 200 to 300 steps from start to finish. That’s a lot of folding for a single piece of paper. But, the trick is using super thin paper with long fibers, which give it strength to withstand all the pulling and tugging.

Computer programs have only added to the fun. TreeMaker, a free software program created by artist Robert Lang, takes user-generated line drawing and churns out patterns that can be printed out and folded to create the shapes. Origamizer allows users to design a 3D model and alter its crease patterns on the screen, exploring different shapes and forms.

With the help of computer software, origami has expanded beyond the art world. Scientists and engineers have found practical applications for paper folding. Car manufacturers, for instance, use origami mathematics to compute a crease pattern for folding airbags into flattened shapes. Demaine says origami structures could influence nanomanufacture, spurring the creation of flat intel chips that can spring into 3D shapes. He also met with members of the National Institutes of Health last year to discuss how the craft could help design synthetic virus-fighting proteins.

Linking mathematics and art does carry some occupational hazards, though.

“A few paper cuts a year,” Demaine says.

Three works by the father-son team are on display in “4o Under 40: Craft Futures,” an exhibition at the Smithsonian’s Renwick Gallery through February 3, 2013.