NSL spindle map-reduce at the edge in a V2V environment

Abstract

We present Spindle, a hybrid vehicle-to-cloud architecture that we believe overcomes the problem of performing cloud analytics on vehicle data, given network constraints. Spindle achieves this by automatically moving cloud-computing processes to the vehicles rather than moving all of the vehicle data to the cloud. Spindle takes advantage of new standards for dedicated wireless ad-hoc communications between vehicles, and leverages existing work using these networks to create clusters of vehicles in which resources can be shared. Spindle builds on the cloud/big data Spark Streaming software. Spindle adapts Spark Streaming to provide an edge computing framework that can run arbitrary, user-defined, Spark Streaming map/reduce functions on individual vehicles and inside vehicle clusters. Spindle allows general-purpose user-defined, cloud-oriented, analyses to take place on vehicle data by performing the user-defined aggregations inside the vehicle clusters and sending only the computation's results. Spindle's novelty and power lies in its ability to allow an end-user to leverage edge computing optimizations with dynamic clusters of connected vehicles without having to leave the familiar Spark Streaming ecosystem and syntax. By sending only the aggregated results of a user-defined analysis to the cloud, Spindle reduces the amount of internet bandwidth required by the analysis to a manageable amount.

In the coming years, automobiles are expected to become increasingly connected and are also expected to become sources for vast amounts of data. Each vehicle has a wealth of information about the local environment and groups of vehicles can convey valuable information about mobility patterns. By combining vehicle data with other data sources, one can potentially glean information not only about traffic conditions, but also weather and climate, infrastructure usage over time, and possibly even social and economic information derived from long term movement patterns. Real-time access to vehicle data could facilitate everything from traffic management, navigation, emergency response, and short-term infrastructure maintenance planning to weather forecasting. While aggregate data from these connected vehicles is valuable, it is expensive and potentially impractical to transmit to and store in the cloud all data produced by all operating vehicles all the time. While the cloud offers invaluable resources for extracting useful information from large volumes of data, cloud resources are not free and must be shared among a growing set of organizations with an ever-expanding list of use cases. Connected vehicles must also compete with smartphones, cellular modems, and other IoT devices for bandwidth on shared networks such as the internet. The cloud has powerful data analytics resources and vehicles have (and will increasingly have) valuable data, but there is currently no cost-effective or practical way to get the data to where it can be processed.

We evaluate Spindle using a custom-built simulator that combines vehicle velocity and connectivity trace data with clustering information to provide a realistic representation of how a Spindle system would operate on the vehicle and cluster level. Our results show Spindle can produce bandwidth savings of 99.6% compared to sending raw vehicle data to the cloud and over 75% compared to performing only map/reduce without vehicle clusters.