《系统工程》课程教学资源（英文文献）Connected Vehicle Safety Science, System, and Framework

Connected Vehicle Safety Science, System, andFrameworkGuide Words:Connected vehicle; intelligent transportation system, driver assistance system,internet-of-thingsAbstract:In this paper, we propose a framework to develop an M2M-based (machine-to-machine)proactive driver assistance system. Unlike traditional approaches, we take the benefits of M2M inintelligenttransportationsystem(ITS):1)expansionof sensorcoverage,2)increaseoftimeallowedto react, and 3)mediation of bidding for right of way,to help driver avoiding potential trafficaccidents.Todevelopsuchasystem,wedivideitintothreemainparts:1)driverbehaviormodelingand prediction, which collects grand driving data to learn and predict the future behaviors of drivers; 2)M2M-based neighbor map building, which includes sensing, communication, and fusion technologiesto build a neighbor map, where neighbor map mentions the locations of all neighboring vehicles; 3)design of passive information visualization and proactive warning mechanism, which researches onhow to provide user-needed information and warning signals to drivers without interfering theirdrivingactivities.I.INTRODUCTIONThe most profound technologies are those that disappear. They weave themselves into the fabricof everyday life until they are indistinguishable from it, dubbed by Mark Weiser. Theinternet-of-things (IOT) is a realization of the ubiquitous computing vision, whereas (1) the bestcomputer is a quiet, invisible servant; (2) the computer should extend your unconscious; (3)technology informs but does not demand our attention. The usefulness of IOT will emerge whenproducts, applications, and services are connected and interacting with each other. Intelligenttransportation system (ITs), which has been extensively researched in the last decade, compliesadvanced mechanisms to provide innovative,proactive services relating to traffic management anddriving safety. For example, drivers'behaviors are limited to their line of sight. Connected vehiclescannot only share their sensory information, but also actively send out alerts to nearby vehicles indange. Forming an even larger vehicular network, comprising connected vehicles and infrastructures,make it possible to proactively perform load balancing across multiple routes. It is anticipated thattraffic accidents can be eliminated from one of the leading causes of death and the catastrophic onescanbeeffectivelyprevented.In this paper, we present the challenges arise from realizing intelligent transports, and provide

Connected Vehicle Safety Science, System, and Framework Guide Words：Connected vehicle; intelligent transportation system, driver assistance system, internet-of-things Abstract：In this paper, we propose a framework to develop an M2M-based (machine-to-machine) proactive driver assistance system. Unlike traditional approaches, we take the benefits of M2M in intelligent transportation system (ITS): 1) expansion of sensor coverage, 2) increase of time allowed to react, and 3) mediation of bidding for right of way, to help driver avoiding potential traffic accidents. To develop such a system, we divide it into three main parts: 1) driver behavior modeling and prediction, which collects grand driving data to learn and predict the future behaviors of drivers; 2) M2M-based neighbor map building, which includes sensing, communication, and fusion technologies to build a neighbor map, where neighbor map mentions the locations of all neighboring vehicles; 3) design of passive information visualization and proactive warning mechanism, which researches on how to provide user-needed information and warning signals to drivers without interfering their driving activities. I. INTRODUCTION The most profound technologies are those that disappear. They weave themselves into the fabric of everyday life until they are indistinguishable from it, dubbed by Mark Weiser. The internet-of-things (IOT) is a realization of the ubiquitous computing vision, whereas (1) the best computer is a quiet, invisible servant; (2) the computer should extend your unconscious; (3) technology informs but does not demand our attention. The usefulness of IOT will emerge when products, applications, and services are connected and interacting with each other. Intelligent transportation system (ITS), which has been extensively researched in the last decade, complies advanced mechanisms to provide innovative, proactive services relating to traffic management and driving safety. For example, drivers’ behaviors are limited to their line of sight. Connected vehicles cannot only share their sensory information, but also actively send out alerts to nearby vehicles in dange. Forming an even larger vehicular network, comprising connected vehicles and infrastructures, make it possible to proactively perform load balancing across multiple routes. It is anticipated that traffic accidents can be eliminated from one of the leading causes of death and the catastrophic ones can be effectively prevented. In this paper, we present the challenges arise from realizing intelligent transports, and provide

insights on resolution in thepresence of machine-to-machine (M2M) communications,includingvehicle-to-vehicle(V2V),vehicle-to-infrastructure(V21)and vehicle-to-cloud (V2C).Interms ofubiquitous computing, the internet-of-things in ITS (1) is a large-scale distributed computing server,(2) can extend human perception; (3) interacts with one another and, most importantly, with humanbeings to ensure against potential traffic violations and accidents.II.PROBLEMFORMULATIONTraffic violations do not necessarily lead to traffic collisions, if timely warnings can be sent outin accordance with the traffic situation.However, due to the line of sight, the perceptual capabilities ofany individuals are limited. In terms of ITS, things (or devices) work not just as individuals, but asmembers of a hierarchy.Thus, it is necessary to consider the problem of not just individual groups, butalsotheproblemof setsofgroupsasawholeX,Fig. 1. The hierarchy of the ITS problemAs shown in Fig. 1, lines are used to indicate roads. Analytics (A) optimizes decision making inthe cloud by aggregating every bit of information, whereas communication (C) and user experience (X)addressconnectivityandusability,respectively,inanadhocmanner,toensureagainstpotentialtrafficcollisions. Apart from the above problems, crowdsourcing also plays an essential role in developmentof analytics. Learning from crowdsourcing can serve as the key to making ITS reality.To develop such a proactive driver assistance system based on M2M communications, we divideit into three main parts and several technical components, as shown in Fig. 2. The three parts are 1)driver behavior modeling and prediction, 2) M2M-based neighbor map building, where neighbor mapmentions the locations of all neighboring vehicles, and 3) design of passive information visualizationand proactive warning mechanism

insights on resolution in the presence of machine-to-machine (M2M) communications, including vehicle-to-vehicle (V2V), vehicle-to-infrastructure (V2I) and vehicle-to-cloud (V2C). In terms of ubiquitous computing, the internet-of-things in ITS (1) is a large-scale distributed computing server; (2) can extend human perception; (3) interacts with one another and, most importantly, with human beings to ensure against potential traffic violations and accidents. II. PROBLEM FORMULATION Traffic violations do not necessarily lead to traffic collisions, if timely warnings can be sent out in accordance with the traffic situation. However, due to the line of sight, the perceptual capabilities of any individuals are limited. In terms of ITS, things (or devices) work not just as individuals, but as members of a hierarchy. Thus, it is necessary to consider the problem of not just individual groups, but also the problem of sets of groups as a whole. Fig. 1. The hierarchy of the ITS problem As shown in Fig. 1, lines are used to indicate roads. Analytics (A) optimizes decision making in the cloud by aggregating every bit of information, whereas communication (C) and user experience (X) address connectivity and usability, respectively, in an ad hoc manner, to ensure against potential traffic collisions. Apart from the above problems, crowdsourcing also plays an essential role in development of analytics. Learning from crowdsourcing can serve as the key to making ITS reality. To develop such a proactive driver assistance system based on M2M communications, we divide it into three main parts and several technical components, as shown in Fig. 2. The three parts are 1) driver behavior modeling and prediction, 2) M2M-based neighbor map building, where neighbor map mentions the locations of all neighboring vehicles, and 3) design of passive information visualization and proactive warning mechanism

1)MachineLearning3)UserExperienceDriverBehaviorData CollectionLearningUser-CentricSimulationDriver Behavior ModelAnticipatoryReasoningHCI DesignNeighbor MapNeighbor MapGPS,CameraPassive InformationBuildingLidar,etc.Proactive WarningDriver2)NeighborMapCommunicationFig. 2. System flowchart of the proactive driver assistance system.The rest of the paper is organized as follows. Next, the analytics and reasoning methodology isdescribed, prior to which the data collection process, as well as the data, is presented. In Sec. V, theconstraints and limitations in communication will be addressed. In Sec. VI, we explore the variety ofdesign challenges of the frontend exposed to end-users. Finally, we present the concluding remarksand future outlook in Sec. VIl.IILDATACOLLECTIONScooters are one of themost important transportation means in Taiwan.Out of 22millionregistered vehicles in Taiwan, scooters account for 67.2% of the vehicles - every 1.56 persons inTaiwan own a scooter. It is popular due to its higher fuel efficiency, lower sale price, and better abilitytomove through heavy traffic jams in theurban area, compared to a regular passenger car.Howeveralso due to its lower sale price, which results in less safety features incorporated in it, and its highermobility,which increases theprobability of a collision with other vehicles,scooters havecontributedto more than 80% of deaths in traffic accidents in Taiwan, causing more than 2,000 fatalities annuallyin the past decade.It is therefore crucial to develop a safety system that can helpto improve the safetyof the scooters on the road, while the solution needs to be able to be implemented within the costmargin of a regular scooter, which usually sells for approximately 2,000 U.S. dollars, about one tenthof that of a regular passenger car.Onepossiblesolutiontoutilizeamobiledevice,suchasasmartphone,to implementsomeofthese safety features. As the market penetration rate of smartphones grows to be over the 50% markglobally, they are owned by the majority of the drivers and thus, when the safety features areimplemented on the smartphone, it does not increase the cost of the vehicle. In addition, smartphoneshave many built-in sensors that can be usedtoobservethe driving behavior of the scooter drivers andthe surrounding vehicles; these sensors include gyroscopes, accelerometers, cameras, GPS, etc. If

Fig. 2. System flowchart of the proactive driver assistance system. The rest of the paper is organized as follows. Next, the analytics and reasoning methodology is described, prior to which the data collection process, as well as the data, is presented. In Sec. V, the constraints and limitations in communication will be addressed. In Sec. VI, we explore the variety of design challenges of the frontend exposed to end-users. Finally, we present the concluding remarks and future outlook in Sec. VII. III. DATA COLLECTION Scooters are one of the most important transportation means in Taiwan. Out of 22 million registered vehicles in Taiwan, scooters account for 67.2% of the vehicles - every 1.56 persons in Taiwan own a scooter. It is popular due to its higher fuel efficiency, lower sale price, and better ability to move through heavy traffic jams in the urban area, compared to a regular passenger car. However, also due to its lower sale price, which results in less safety features incorporated in it, and its higher mobility, which increases the probability of a collision with other vehicles, scooters have contributed to more than 80% of deaths in traffic accidents in Taiwan, causing more than 2,000 fatalities annually in the past decade. It is therefore crucial to develop a safety system that can help to improve the safety of the scooters on the road, while the solution needs to be able to be implemented within the cost margin of a regular scooter, which usually sells for approximately 2,000 U.S. dollars, about one tenth of that of a regular passenger car. One possible solution to utilize a mobile device, such as a smartphone, to implement some of these safety features. As the market penetration rate of smartphones grows to be over the 50% mark globally, they are owned by the majority of the drivers and thus, when the safety features are implemented on the smartphone, it does not increase the cost of the vehicle. In addition, smartphones have many built-in sensors that can be used to observe the driving behavior of the scooter drivers and the surrounding vehicles; these sensors include gyroscopes, accelerometers, cameras, GPS, etc. If

behavior models can be established and used to predict hazardous behaviors in advance with thecollected smartphone sensor data, then advance warning can be provided to the driver of that vehicleor,via someforms of communications,tothe driver of a neighboring vehicle.The behaviors of thescooters are significantly different from the behaviors of cars, due to its smaller dimensions and that ithas one more degree of freedom in its movement - the lean angle of its body, ie., roll angle. Althoughthere have been many efforts in collecting driving behavior data for cars, to the best of ourknowledge,thereisalmostnoeffortsincollectingextensivedrivingbehaviordataforscootersormotorcyclesTable I. Description of the Collected Sensor TypesrsopneVideo Carideo that is split into 10-min segments, The resolution of the video depends on t1020x10,640x480,320x240,ofollowing fou06x144.ThevideousesH.264/AdvancedVideoCoding(AVC)recorded audio is recorded as part of the video file, using the Adaptive Multi-RateMicrophonAD:CDvelocity,and bearine (vehicle)are )10 - 30 Hz, depending on the phone modelAccelerometerin m/s that is applied to the device on all three physicalz),includingthefon10 - 30 Hz, depending on the phone modelacceleationforceinm/sthat isappliedtothedeviceonallthreephysicalLineartheGyroscope10 - 30 Hz, depending on the phone modethe device'srate of rotationn rad/s around each of the three physical axes (x,y.sticfieldforall threephvsaxes(x,y,z)inpendineonthephreesofrotationthatthedevicemakesaroundallthreephvsicalaTo obtain the necessary data for developing various scooter driver behavior model, in June toSeptember2013,wehaveconductedalarge-scaledata collectionevent.in which100 scooterdriversare hired to collect sensor data during their daily use of scooters, using an app that we developed thatis executed on theirown Android smartphone.Before the event, wehave also distributed phonemountstoall participants,sothattheirsmartphonescanbeplacedonthehandlebarofthescootersandthe back camera of the smartphones can be used to capture video of the surrounding environment ofthe scooters.The app functions as a video event data recorder for the user, but in addition to recording thevideo and the audio, it also collects data from many sensors in the phone. Table I shows the type ofsensorsthat weused inthesmartphonefordata collection andtheirdescription.Notethat some of thelistedsensorsarevirtual sensors,whosedataiscalculatedwiththerawdatacollectedbyothersensors.Data collected by the smartphones is uploaded to a back-end server via cellular data connections orWi-Fi connections in real-time

behavior models can be established and used to predict hazardous behaviors in advance with the collected smartphone sensor data, then advance warning can be provided to the driver of that vehicle, or, via some forms of communications, to the driver of a neighboring vehicle. The behaviors of the scooters are significantly different from the behaviors of cars, due to its smaller dimensions and that it has one more degree of freedom in its movement - the lean angle of its body, i.e., roll angle. Although there have been many efforts in collecting driving behavior data for cars, to the best of our knowledge, there is almost no efforts in collecting extensive driving behavior data for scooters or motorcycles. Table I. Description of the Collected Sensor Types To obtain the necessary data for developing various scooter driver behavior model, in June to September 2013, we have conducted a large-scale data collection event, in which 100 scooter drivers are hired to collect sensor data during their daily use of scooters, using an app that we developed that is executed on their own Android smartphone. Before the event, we have also distributed phone mounts to all participants, so that their smartphones can be placed on the handlebar of the scooters and the back camera of the smartphones can be used to capture video of the surrounding environment of the scooters. The app functions as a video event data recorder for the user, but in addition to recording the video and the audio, it also collects data from many sensors in the phone. Table I shows the type of sensors that we used in the smartphone for data collection and their description. Note that some of the listed sensors are virtual sensors, whose data is calculated with the raw data collected by other sensors. Data collected by the smartphones is uploaded to a back-end server via cellular data connections or Wi-Fi connections in real-time

Fig. 3. The footprints of the participating scooter drivers over the 3-month event duration.Overthe3-monthperiod, a large amountofdatawas collected.Thefollowing summarizes somestatisticsofthecollecteddata:(1) 10,858 video files, with a total size of 473.8 GB, were collected. Most of the files are 10minutes in length.(2) In total, we collected 28,273 kilometers of driving behavior data. Out of the 100 participants.8 of them collectedmore than 1,000 kilometers of data,while 22 of them collected 100-1,000kilometersofdata(3) The majority of the participants operate the vehicle in the urban area of Taipei city, whilesome of them operate thevehicle in other parts of Taiwan.IV.ANALYTICSANDREASONINGMETHODOLOGYIn this section, we will present two main technical components for anticipatory reasoning: driverbehavior learning and neighbor map building.A.DriverBehavior LearningIn the past few years, researchers have spent lots of money and human efforts to study how toimprove the quality of driving and to avoid traffic accidents caused by improper drivingbehaviorwiththeaidfromcomputers.In2oo9,a studyreportedbytheAmericanAutomobileAssociation(AAA)Foundation for Traffic Safety shows that there are 56% of deadly crashes between 2003 and 2007involve one or more unsafe driving behaviors typically associated with aggressive driving. In thiswork, we want to analyze whether it is possible to predict dangerous events and to alert in advanceusing heterogeneous sensor data. Also we want to learn whether being able to recognize the drivingstyles of drivers can boost the above performance.In this project, we have collected the heterogeneous sensor data of 100 drivers, which bring some

Fig. 3. The footprints of the participating scooter drivers over the 3-month event duration. Over the 3-month period, a large amount of data was collected. The following summarizes some statistics of the collected data: (1) 10,858 video files, with a total size of 473.8 GB, were collected. Most of the files are 10 minutes in length. (2) In total, we collected 28,273 kilometers of driving behavior data. Out of the 100 participants, 8 of them collected more than 1,000 kilometers of data, while 22 of them collected 100 –1,000 kilometers of data. (3) The majority of the participants operate the vehicle in the urban area of Taipei city, while some of them operate the vehicle in other parts of Taiwan. IV. ANALYTICS AND REASONING METHODOLOGY In this section, we will present two main technical components for anticipatory reasoning: driver behavior learning and neighbor map building. A. Driver Behavior Learning In the past few years, researchers have spent lots of money and human efforts to study how to improve the quality of driving and to avoid traffic accidents caused by improper driving behavior with the aid from computers. In 2009, a study reported by the American Automobile Association (AAA) Foundation for Traffic Safety shows that there are 56% of deadly crashes between 2003 and 2007 involve one or more unsafe driving behaviors typically associated with aggressive driving. In this work, we want to analyze whether it is possible to predict dangerous events and to alert in advance using heterogeneous sensor data. Also we want to learn whether being able to recognize the driving styles of drivers can boost the above performance. In this project, we have collected the heterogeneous sensor data of 100 drivers, which bring some

handy benefits as well as some challenges. It is possible to use some rules to automatically generatesome lower-level driver behavior, such as whether the driver stops at particular intersection at giventimeorwhetherthederivermakesa U-turn.Theformer canbe obtainedbycheck whetherthe speed isreduced to zero when approaching the intersection, while the later can be checked by whether thedirection of the driver is changed to the opposite within a short amount of time.Given the drivers'dataset with such automatically behavior, we try to create some forecastingmodels that utilize the existing data to prediction whether in the near future the drivers will performsuch behaviors of interests. That is, we want to build a system knowing that a driver is not going tostop in the intersection (or is performing U-turn) several seconds before this driver conducts suchaction.With such mechanism,we can than forecast somedangerous behavior (e.g.red-light runner)and issuewarning to thenearbyvehicles.There are some other events of interests that we can hardly extract from data through writingsimple rules, for instance, sudden change of lines or aggressive left turn. Usually these are morecomplicated behavior that might require human judgment to label. The second goal of this project is tocreate a semi-automatic framework that assists the users to identify or label such event moreefficiently. With a faithful label, then we can again design supervised system to model such behavior.The idea is to exploit semi-supervised or active learning to create a hypothesis of labels to query theusers.Ideas such as dynamic time warping (DTW) based sequence matching can be also useful. Toimprovefurther of the above short-term learning method, we in cooperate with another learningmodule that can also learn the long-term behavior of motorcyclists. In this study, we assume thatlong-termbehaviorsofmotorcvclistscanrevealtheirtendencyintheirdrivingtrajectoriesIntuitively, we understand that some types of vehicle drivers tend not to obey traffic rules and wehope to extract patterns of the“bad"driving behaviors given the drivers'trajectories. We plan to findbad driving behaviors by detecting anomaly trajectories from a collection of trajectory set. Theassumption is that most drivers are likely to follow traffic rules most of the time and we consider theirtrajectories the normal part of the trajectory set; on the other hand, some drivers may break rules bychanging lanes frequently, speeding, sharply turn to the left, etc., and we consider those behaviorsanomalies in the set. To detect anomalies for bad driver prediction, we first find a dissimilaritymeasure or a “" distance" to describe how different between each pair of two trajectories, then by usingthe dissimilarity measure, we can cluster trajectories into several groups and hopefully each groupcontains trajectories of similar patterns. Given the clustering result, we can find anomalies from theoutlier group, minor group, or trajectories not belonging to any groups, and we can then find baddriversfrom theanomaliesl.To combine the short-term and long-term learning modules together, we simply transform the

handy benefits as well as some challenges. It is possible to use some rules to automatically generate some lower-level driver behavior, such as whether the driver stops at particular intersection at given time or whether the deriver makes a U-turn. The former can be obtained by check whether the speed is reduced to zero when approaching the intersection; while the later can be checked by whether the direction of the driver is changed to the opposite within a short amount of time. Given the drivers’ dataset with such automatically behavior, we try to create some forecasting models that utilize the existing data to prediction whether in the near future the drivers will perform such behaviors of interests. That is, we want to build a system knowing that a driver is not going to stop in the intersection (or is performing U-turn) several seconds before this driver conducts such action. With such mechanism, we can than forecast some dangerous behavior (e.g. red-light runner) and issue warning to the nearby vehicles. There are some other events of interests that we can hardly extract from data through writing simple rules, for instance, sudden change of lines or aggressive left turn. Usually these are more complicated behavior that might require human judgment to label. The second goal of this project is to create a semi-automatic framework that assists the users to identify or label such event more efficiently. With a faithful label, then we can again design supervised system to model such behavior. The idea is to exploit semi-supervised or active learning to create a hypothesis of labels to query the users. Ideas such as dynamic time warping (DTW) based sequence matching can be also useful. To improve further of the above short-term learning method, we in cooperate with another learning module that can also learn the long-term behavior of motorcyclists. In this study, we assume that long-term behaviors of motorcyclists can reveal their tendency in their driving trajectories. Intuitively, we understand that some types of vehicle drivers tend not to obey traffic rules and we hope to extract patterns of the “bad” driving behaviors given the drivers' trajectories. We plan to find bad driving behaviors by detecting anomaly trajectories from a collection of trajectory set. The assumption is that most drivers are likely to follow traffic rules most of the time and we consider their trajectories the normal part of the trajectory set; on the other hand, some drivers may break rules by changing lanes frequently, speeding, sharply turn to the left, etc., and we consider those behaviors anomalies in the set. To detect anomalies for bad driver prediction, we first find a dissimilarity measure or a “distance” to describe how different between each pair of two trajectories, then by using the dissimilarity measure, we can cluster trajectories into several groups and hopefully each group contains trajectories of similar patterns. Given the clustering result, we can find anomalies from the outlier group, minor group, or trajectories not belonging to any groups, and we can then find bad drivers from the anomalies1. To combine the short-term and long-term learning modules together, we simply transform the

above dissimilarity measure to coordinates by either Multidimensional scaling (MDS), Isomap or anysimilar techniques, and feed the resultant coordinates to the attribute set that is belonged to theshort-term learning part and we can have a complete attribute set for the final learning task.B.NeighborMapBuildingData from a number of heterogeneous sensors such as GPS, odometer, inertial measurement unit(IMU), laser scanners, cameras and RGB-D cameras used by connected vehicles and moving entitieshas to be fused properly and efficiently. There are two levels to fuse the heterogeneous data. First, forfusion between the nodes, an algorithm for simultaneous localizing and tracking vehicles [13] [14] isutilized to obtain sub-meter accurate localization which is necessary for driver warning systems.Second, for fusion within the nodes, algorithms to detect moving objects from laser scanner andstationary cameras is exploited to provide pedestrians, motorcycles, bikes, and cars information in theheterogeneoussensorfusion scheme.For ITS applications, a sensor fusion scheme is composed by a road-side unit with cameras andlaser scanners, and moving vehicles, including several motorcycles with GPs information, onemotorcycle with laser scanners, and one car.Each node (vehicle or infrastructure) processes data retrieved from its sensors and detects thenearby moving objects. All the information is fused into local believes to represent the traffic scene.These beliefs are shared (as shown in Fig. 5) and propagated by communication modules to nearbyvehicles or roadside units. All received believes are fused via the belief merge module with eachnodes own believes and the representation of the traffic environment is obtained which can be used byother applications, such as driver warning systems. The sharing and fusion of each nodes believes canavoid the delays or data lost problems within the unstable traffic environments, and it is beneficial forimprovingthe driving safetyratherthan sharing sensor measurements

above dissimilarity measure to coordinates by either Multidimensional scaling (MDS), Isomap or any similar techniques, and feed the resultant coordinates to the attribute set that is belonged to the short-term learning part and we can have a complete attribute set for the final learning task. B. Neighbor Map Building Data from a number of heterogeneous sensors such as GPS, odometer, inertial measurement unit (IMU), laser scanners, cameras and RGB-D cameras used by connected vehicles and moving entities has to be fused properly and efficiently. There are two levels to fuse the heterogeneous data. First, for fusion between the nodes, an algorithm for simultaneous localizing and tracking vehicles [13] [14] is utilized to obtain sub-meter accurate localization which is necessary for driver warning systems. Second, for fusion within the nodes, algorithms to detect moving objects from laser scanner and stationary cameras is exploited to provide pedestrians, motorcycles, bikes, and cars information in the heterogeneous sensor fusion scheme. For ITS applications, a sensor fusion scheme is composed by a road-side unit with cameras and laser scanners, and moving vehicles, including several motorcycles with GPS information, one motorcycle with laser scanners, and one car. Each node (vehicle or infrastructure) processes data retrieved from its sensors and detects the nearby moving objects. All the information is fused into local believes to represent the traffic scene. These beliefs are shared (as shown in Fig. 5) and propagated by communication modules to nearby vehicles or roadside units. All received believes are fused via the belief merge module with each nodes own believes and the representation of the traffic environment is obtained which can be used by other applications, such as driver warning systems. The sharing and fusion of each nodes believes can avoid the delays or data lost problems within the unstable traffic environments, and it is beneficial for improving the driving safety rather than sharing sensor measurements

MergedBeliefreceiveBelief-MergeBeliefsshareCommunicationSimultaneousLocalization and Tracking(SLAT)OdometryCameraLaserGPSScannerFig.5.An overviewof belief-based sensor fusionV.COMMUNICATIONOur systemusestheIEEE 802.1l solutiontosupport communicationbetweenvehicles androadside units (RSUs).In the infrastructure or ad hoc mode of an IEEE802.11 network,devices canonlyreceiveMAC-layerframeswithinthe samebasicservice set(BSS).AlthoughsuchMAC-layerfiltering improves efficiency and energy consumption, it will become a serious problem for movingvehicles since vehicles in the vicinity of each other may not necessarily belong to the same Bss. TheIEEE802.1lpstandard(alsoknownasDedicated ShortRangeCommunicationorDSRC)solvestheproblem by introducing a wildcard BSS ID. By using a wildcard BSS ID, a vehicle can receive allframes from nearby vehicles in the same channel without association or authentication.Unfortunatelythe cost and availability are always the deal breakers for DSRC in vehicular communications. Insteadof usingDSRCradios,we relyonoff-the-shelfIEEE802.11b/gradiosand enabletheso-calledmonitoringmodeandinjection.Themonitoring modeallows an IEEE 802.11network interface card (NIC)to captureMAc-layer frames without associatingwith an access point or ad hoc network,while the injectionallows aNIC totransmita framewithno intended recipient.WehavefoundthatNICs usingAtherosAR9271,RealtekRTL8187LorIntelJC82546MDE chipsetswithmodifieddrivers supportthemonitoring mode and injection.The achievablethroughput of theresulting802.11b/gNICs is shownin Fig. 6. The result shows that when transmitting at 54 Mbps (11g) with a payload size of 1523 bytes,two vehicles can achieve a throughput of up to 17 Mbps, which is much higher than our per-vehiclethroughput requirement of 1.1Mbps

Fig. 5. An overview of belief-based sensor fusion. V. COMMUNICATION Our system uses the IEEE 802.11 solution to support communication between vehicles and roadside units (RSUs). In the infrastructure or ad hoc mode of an IEEE 802.11 network, devices can only receive MAC-layer frames within the same basic service set (BSS). Although such MAC-layer filtering improves efficiency and energy consumption, it will become a serious problem for moving vehicles since vehicles in the vicinity of each other may not necessarily belong to the same BSS. The IEEE 802.11p standard (also known as Dedicated Short Range Communication or DSRC) solves the problem by introducing a wildcard BSS ID. By using a wildcard BSS ID, a vehicle can receive all frames from nearby vehicles in the same channel without association or authentication. Unfortunately, the cost and availability are always the deal breakers for DSRC in vehicular communications. Instead of using DSRC radios, we rely on off-the-shelf IEEE 802.11b/g radios and enable the so-called monitoring mode and injection. The monitoring mode allows an IEEE 802.11 network interface card (NIC) to capture MAC-layer frames without associating with an access point or ad hoc network, while the injection allows a NIC to transmit a frame with no intended recipient. We have found that NICs using Atheros AR9271, Realtek RTL8187L or Intel JC82546MDE chipsets with modified drivers support the monitoring mode and injection. The achievable throughput of the resulting 802.11b/g NICs is shown in Fig. 6. The result shows that when transmitting at 54 Mbps (11g) with a payload size of 1523 bytes, two vehicles can achieve a throughput of up to 17 Mbps, which is much higher than our per-vehicle throughput requirement of 1.1Mbps

PHYdatarateto actual injection throughput?/S10W0n0EEE802.11bIEEE802.11g102030405060datarate(Mbits/sec)Fig. 6. Achievable throughput when using IEEE 802.11 b/g NICs with the monitoring and injection.In order to make our implementation transparent to theupper layers,four applicationprogramming interfaces (API) are also developed.For example, CommOpen(interface, nodeID)specifies the NIC and transmitter node ID. A pcap, a structure that links to the queues of received andbuffered packets, is then created. Comm_Send(pcap, belief) encapsulates the belief into a packet andinserts the packet into the buffer queue for transmission.If the size of a belief is larger than1523 bytesthe belief is fragmented. Finally, Comm_Receive(pcap) retrieves the belief from a received packet. Byusing these APIs, a vehicle can easily broadcast and receive belief from any other vehicle in the sameneighborhood.VL.USEREXPERIENCERearview mirrors exploit human peripheral vision to eliminate driver's attention blind spots forsafety driving. However, users only check the rearview mirrors when they want to, rather than thecritical situations that they need to. In the busy traffics, users tend to pay more attention on the roadcondition in the frontthan the potential hazards behind them.To increasedriver's awareness on theirattention blind spots, some vehicle manufacturers have started to provide blind spot warningmechanism on the rearview mirror Conventional blind spot warning mechanism uses a blinking pointlight, vibrating the steering wheel, or making audible sound alert to notify driver to take a glance atthe rear mirror while lane switching. Nevertheless, when the driver checks the view in the mirror, theystillneedtimetocomprehendthereal scenebeforereacttoit.Theinsufficientinformationof thecurrent warning mechanism neither helps drivers making a just-in-time decision, nor takes anappropriate reaction. The slow reaction time also hinder the primary driving tasks.ThetrafficmonitoringITScan analyzethetraffics to recognizethevarious potential hazards.Thesystem can detect a dangerous events such as a rush driving out of drivers' sight, and forwardcorresponding proactive warning message, including sufficient information of the fact and thesuggested action to perform, to the drivers who may concerned.By utilizing this capability of ITS, wecan provide a more informative warning messages for drivers to take proper reactions in shorter time

Fig. 6. Achievable throughput when using IEEE 802.11 b/g NICs with the monitoring and injection. In order to make our implementation transparent to the upper layers, four application programming interfaces (API) are also developed. For example, Comm_Open(interface, node ID) specifies the NIC and transmitter node ID. A pcap, a structure that links to the queues of received and buffered packets, is then created. Comm_Send(pcap, belief) encapsulates the belief into a packet and inserts the packet into the buffer queue for transmission. If the size of a belief is larger than 1523 bytes, the belief is fragmented. Finally, Comm_Receive(pcap) retrieves the belief from a received packet. By using these APIs, a vehicle can easily broadcast and receive belief from any other vehicle in the same neighborhood. VI. USER EXPERIENCE Rearview mirrors exploit human peripheral vision to eliminate driver’s attention blind spots for safety driving. However, users only check the rearview mirrors when they want to, rather than the critical situations that they need to. In the busy traffics, users tend to pay more attention on the road condition in the front than the potential hazards behind them. To increase driver's awareness on their attention blind spots, some vehicle manufacturers have started to provide blind spot warning mechanism on the rearview mirror. Conventional blind spot warning mechanism uses a blinking point light, vibrating the steering wheel, or making audible sound alert to notify driver to take a glance at the rear mirror while lane switching. Nevertheless, when the driver checks the view in the mirror, they still need time to comprehend the real scene before react to it. The insufficient information of the current warning mechanism neither helps drivers making a just-in-time decision, nor takes an appropriate reaction. The slow reaction time also hinder the primary driving tasks. The traffic monitoring ITS can analyze the traffics to recognize the various potential hazards. The system can detect a dangerous events such as a rush driving out of drivers' sight, and forward corresponding proactive warning message, including sufficient information of the fact and the suggested action to perform, to the drivers who may concerned. By utilizing this capability of ITS, we can provide a more informative warning messages for drivers to take proper reactions in shorter time

In this work, we propose an AR-based visualization technique, Augmented Rearview, to visualizethe potentially hazardous events detected by the ITs. The visualization of hazardous events, such asswitching lanes, dangerous driving, etc., are directly overlaid on the real scenes displayed in theelectronic rearview mirror. The simple visualization is seamlessly integrated with the real scene,providing semantic means that the type of events can be comprehensible in a glimpse.Thehigh-saliency features that we used in visualization also help users perceive the warnings by theirperipheral vision effectively, without hindering the focus driving tasks.We have implemented a VR system with simulated reality through immersive driving simulationA 7-inch display is used to serve as the electronic rearview mirror. The rear-view obtained from thecamera that set in the simulated graphic context is shown on the rearview mirror, In the simulatedtraffics, the screen visualizes the proactive warning message on the rearview in real-time. We alsobuild the proof-of-concept device of AR electronic rearview mirror using 5-inch smartphoneembeddeda2Mega-Pixelcameraandmotionsensorsinside.Visualizationisdirectlyaddedonthereal view obtainedfrom the camera,andtheconsistencybetween the cameraviewportsandtheoverlaid graphic events is maintained by the device's built-in compass.Early users feedbacks gathered from a pilot user study show that users are positive on perceivingthe proactive warning as soon as possible.Also, they can comprehend the semantic meanings of theprovided visualization in a glimpse, and appreciate for the sufficient information provided beforereactingtotheevents.In future work, we will conduct a formal user study to evaluate the efficiency of thesevisualization techniques.We will also attempt transplanting or incorporating our visualizationtechniques with wearable display, such as Google glass, or providing always glance-abledriver-centered information on the helmet glass to increase driver's peripheral awareness.VIL.CONCLUSIONANDFUTUREWORKIn this paper, we propose a framework to develop an M2M-based proactive driver assistancesystem. Unlike traditional approaches, we take the benefits of M2M in ITS: 1) expansion of sensorcoverage, 2) increase of time allowed to react, and 3) mediation of bidding for right of way, to helpdriver avoiding potential traffic accidents. To accomplish such a system, several technical componentsare proposed, such as data collection, driver behavior modeling and prediction, sensor fusionneighbor map building, communication, and HCI design. Although this is an ongoing project andthere still can be some improvements for each component, this paper gives a beginning of how toachieveconnected vehicle safety andfurtherprovidesfuturedirectionsfornewresearch.Inthefuturefirst we will keep improving each component and the system integration performance. Second,compare to current passive warning, we will further research on the problem of proactive trafficaccident avoidance, for example right of way mediation for traffic events, such as change lanes, turnleft vs. go straight, make a U-turn, etc., to improve driving safety

In this work, we propose an AR-based visualization technique, Augmented Rearview, to visualize the potentially hazardous events detected by the ITS. The visualization of hazardous events, such as switching lanes, dangerous driving, etc., are directly overlaid on the real scenes displayed in the electronic rearview mirror. The simple visualization is seamlessly integrated with the real scene, providing semantic means that the type of events can be comprehensible in a glimpse. The high-saliency features that we used in visualization also help users perceive the warnings by their peripheral vision effectively, without hindering the focus driving tasks. We have implemented a VR system with simulated reality through immersive driving simulation. A 7-inch display is used to serve as the electronic rearview mirror. The rear-view obtained from the camera that set in the simulated graphic context is shown on the rearview mirror. In the simulated traffics, the screen visualizes the proactive warning message on the rearview in real-time. We also build the proof-of-concept device of AR electronic rearview mirror using 5-inch smartphone embedded a 2 Mega-Pixel camera and motion sensors inside. Visualization is directly added on the real view obtained from the camera, and the consistency between the camera viewports and the overlaid graphic events is maintained by the device's built-in compass. Early users feedbacks gathered from a pilot user study show that users are positive on perceiving the proactive warning as soon as possible. Also, they can comprehend the semantic meanings of the provided visualization in a glimpse, and appreciate for the sufficient information provided before reacting to the events. In future work, we will conduct a formal user study to evaluate the efficiency of these visualization techniques. We will also attempt transplanting or incorporating our visualization techniques with wearable display, such as Google glass, or providing always glance-able driver-centered information on the helmet glass to increase driver's peripheral awareness. VII. CONCLUSION AND FUTURE WORK In this paper, we propose a framework to develop an M2M-based proactive driver assistance system. Unlike traditional approaches, we take the benefits of M2M in ITS: 1) expansion of sensor coverage, 2) increase of time allowed to react, and 3) mediation of bidding for right of way, to help driver avoiding potential traffic accidents. To accomplish such a system, several technical components are proposed, such as data collection, driver behavior modeling and prediction, sensor fusion, neighbor map building, communication, and HCI design. Although this is an ongoing project and there still can be some improvements for each component, this paper gives a beginning of how to achieve connected vehicle safety and further provides future directions for new research. In the future, first we will keep improving each component and the system integration performance. Second, compare to current passive warning, we will further research on the problem of proactive traffic accident avoidance, for example right of way mediation for traffic events, such as change lanes, turn left vs. go straight, make a U-turn, etc., to improve driving safety