Two University of Maine researchers received a $500,000 National Science Foundation award to advance the ability to analyze massive data samples gathered by real-time sensors.
Today, sensors can only take samples at discrete points in space, such as taking hundreds of individual photographs. Many times the data has to be physically retrieved on a memory card or from the research site, which can be inconvenient.
The age of sensor data collection is moving toward collecting data in real-time, similar to what we think of as a video. It is a stream of images strung together, continually updating in the blink of an eye. Real-time sensors can bring in thousands of new samples every few second. This dramatically enhances the possibility of data available to scientists.
“You can do your science faster and better without having to worry about the cumbersome technical aspects of collecting and integrating data. The information system can do that part for you,” Silvia Nittel, professor in the UMaine School of Computing and Information Science, told the Bangor Daily News (BDN).
The project involves the use of fields, which is a new abstraction for geographic information systems. It can be used to model a range of environmental occurrences like air pollution, smog levels, pollen distribution and humidity.
A large part of the project is dealing with computational challenges with processing the data that uploads so quickly and being able to transform the data into fields to be analyzed.
Nittel’s team created software to help a computer keep up with the thousands of new data samples updating every few seconds. The innovative computational framework complements the field data model, allowing researchers to analyze fields with a click of a button.
“Then you can say, ‘Show me the visual representations.’ And there it is,” Nittel told BDN.
The researchers hope by spreading sensor data streams to fields, scientists can work with high-level abstractions to assess extreme weather events, environmental disasters and chemical accidents fast and precisely.
“That is what we try to do in computer science, we try to come up with something like the Google search engine. Anyone can search for anything, there is one mechanism behind it,” Nittel said. “It doesn’t matter if you’re looking for sports news, world politics, research or places to go out. It’s just one concept behind it. With the field, it’s very similar,”
Nittel said the technology could help UMaine scientists in forestry, ecology and marine science get the most out of the data collected by lower numbers of expensive sensors. Though mathematically complex, the new information system will make data analysis and collection more convenient for the user.
“The concept of fields is also applicable for data that is not real-time, but in this project we specifically address the computational challenges posed through the rapid arrival of data through real-time streams and instant analysis based on fields.”
The UMaine group is working in collaboration with researchers at the University of New York at Buffalo and using their data to test the techniques to see if the new system is effective and accurate.
