Imagine a flag flapping gently in the wind on a sunny day. Now think of it flapping more aggressively as the wind gets up. Our imaginations are powerful simulators. Indeed, experienced observers can get a good sense of the wind speed just be looking at flags.
This ability requires a powerful mental model that can simulate the real word, and humans are well equipped to do this. We constantly use mental models to predict everything from the trajectory of a football to the necessity of carrying an umbrella.
But computer models are not nearly as powerful in this respect. Computer scientists can realistically simulate the way a flag flaps using a model that includes variables such as the speed of the wind and the viscosity of air, along with the strength, weight and geometry of the flag material. The model also relies on the equations of fluid flow that describe how these parameters are linked.
But show the same computer a flapping flag and it will give it a blank stare. There is no way the computer can use its powerful model to determine the speed of the wind.
Until now. Today, Tom Runia and colleagues at the University of Amsterdam in the Netherlands, show how exactly this is possible, They to start with the movement of a flag — from a video, say — and then use a computer model to determine the physical properties of the cloth and the air.
In other words, they use a simulation to measure the wind speed. Their work is part of broader effort in computer science that aims to replace ordinary measurements by simulations, at least in part.
First, some background. Flag motion is complex because the air exerts forces on the cloth through air pressure and viscosity while the flag reacts with its own inertial and elastic forces, says the team. The complex interplay of all this produces the flag motion, from wavelike movements across the fabric to a rolling motion in the corners to violent flapping at the edges and more.
“Untangling the dynamics of fabric is challenging due to the involved nature of the air-cloth interaction,” say Runia and colleagues. But the team has made substantial progress.
In theory, a simulation can reveal the wind speed if it can reproduce the real-world behavior exactly. The team’s new approach is to automate the process of simulating this real behavior.
The overall approach is straightforward in principle. The idea is to compare a real-world video of a flag with a simulated flag video. If the motion is the same in both, then the wind speed in both must match.
But comparing the real and simulated videos is not straightforward because they may have different viewpoints, lighting conditions and so on.
To solve this, the team created a database of flag simulations in which they vary not only the wind speed, but also the virtual camera angle and distance from the flagpole, the lighting angle and so on. In this way, the team built a database of 14,000 simulated flag videos.
Next, the team trained a machine learning algorithm to recognize flags flying in the same wind speed, even when the camera angle, distance and lighting are different. All this is done with simulated videos, without the machine algorithm ever seeing a real flag.
The final step was to set the machine learning algorithm loose on a database of real flag videos. The team created this by recording real flags while measuring the wind speed in a variety of weather conditions. In this way, they created 4,000 short video sequences to act as a ground-truth data set.
The machine learning algorithm then compares a real video with a simulated video and recommends a change in the simulated parameters to make the simulation more real. It then compares the real video against the revised simulation and recommends further fine-tuning. It repeats this process so the simulation becomes more and more like the ground-truth example.
The end result is impressive. “We note that the wind speed converges towards the ground-truth wind speed within a few iterations,” say Runia and colleagues. In other words, this process automatically simulates the movement of a real flag and uses this simulation to determine the real wind speed, just as an experienced sailor might.
That’s an interesting result because it shows how physical parameters in real videos can be measured using simulations. And it has numerous other applications. Runia and his colleagues say they hope to apply the same approach to the spread of fire, smoke and mechanical problems.
But the potential is greater still. If this approach reveals wind speed, there is no reason that it could not reveal other parameters. The properties of cloth, for example, depend on all kinds of details related to the type of weave, the material used to make the yarn and so on. Indeed, the team show how the machine can determine some of the material properties of the real flag.
But the same approach might be useful to automatically determine the type of cloth used to make a suit or a dress, just by looking at the way the material moves.
And beyond that, there are the laws of physics themselves. Might it be possible to create simulations based on different laws of physics — different mathematical equations — to find the ones that actually govern the behavior of flags?
That’s not as far-fetched as it sounds. Back in 2009, a group at Cornell University used an entirely automated approach to extract the basic laws of physics from raw data taken from simple mechanical experiments. Without any prior knowledge of these laws, their algorithm found the laws of conservation of energy and of momentum.
The engine at the heart of this work was the process of evolution, which has the almost-magical ability to find powerful solutions to extremely complex problems.
It would be ambitious to suggest that a similar approach applied to flag-flying simulations might reveal the laws of fluid mechanics, for example, or perhaps the way they need to be modified to model cloth.
But it is not beyond imagination — and, indeed, there has been some progress in this area. If models play an important role in understanding the universe, it is just possible that there is a great deal more to come from this kind of automated learning.
Ref: arxiv.org/abs/2003.05065 : Cloth in the Wind: A Case Study of Physical Measurement through Simulation
