Overview
American Society of Civil Engineers is estimating that $2.2 trillion is needed over five years to repair and retrofit U.S. infrastructure to a good condition. The civil infrastructure in other countries is also aging and failing. Although managing and maintaining infrastructure is not a new problem, nonetheless in recent decades significant expansion in size and complexity of infrastructure networks have posed several new engineering/management problems on how existing infrastructure can be monitored, prioritized and maintained in a timely fashion. One of the grand challenges in restoring and improving urban infrastructure as identified by U.S. National Academy of Engineering (NAE 2010) is to devise techniques to efficiently create records of the locations and up-to-date status of the civil infrastructure. The need for frequent tracking and condition assessment is not only specific to existing infrastructure, but new construction projects are also suffering from the lack of techniques that can easily and quickly track, analyze and visualize as-built status of a project and monitor performance metrics.
To address these inefficiencies in an all-inclusive manner, this research looks into creating a new technique by using two emerging sources of information: (1) digital photographs and (2) Infrastructure Information Models, and explores how current challenges of creating up-to-date records of new/existing civil infrastructure can be proactively addressed. This research proposes creating and developing a new image-based 3D reconstruction and recognition system in which un-calibrated digital images are captured from fixed cameras installed on a vehicle. Using these images (1) a 3D point cloud model of the highway and all other infrastructure is reconstructed; (2) Using a new approach based on Structure-from-Motion, Semantic Texton Forests and Support Vector Machine, all assets are identified and their conditions are assessed by comparing the data to the underlying expected infrastructure information model and reconstructions from initial visual data collections;(3) Finally expected and actual status of highways and assets are visualized in an augmented reality environment (Golparvar-Fard et al. 2009) wherein elements that their conditions are changed, are highlighted with a traffic-light metaphor. Compared to other technologies such as laser scanning, RFID and Barcode tags as well as GPS, this work (1) does not add any new task to project management, (2) has easier, low-cost and safe implementation, (3) enables more frequent assessment, (4) does not need expertise for operation and (5) communicates information easily to all stakeholders, addressing yet another major challenge identified by NAE that the current virtual representations are not realistic enough to maintain a proper depiction of an actual site. Pending a successful formation of this technology, it can highly impact condition assessment as even improving condition by 0.01% can result in $200 million of cost savings.
Product
Research Publications
Software and Patents
Datasets
Videos
Research Areas
Semantic Texton Forests Segmentation and Recognition of Highway Assets
Given video frames collected from a car-mounted camer the proposed set of algorithms (1) takes the captured frames and using a pipeline of Structure from Motion and Multi View Stereo reconstructs a 3D point cloud model of the highway and surrounding assets; (2) using a Semantic Texton Forest classifier, each geo-registered 2D video frame at the pixel-level is segmented based on shape, texture, and color of the highway assets; and finally (3) based on the results of the 2D segmentation and a new voting scheme, each reconstructed 3D point in the cloud is also categorized for one type of asset and is color coded accordingly.
Video-Based Highway Asset Recognition and 3D Localization
A key element towards development of asset management program is an efficient data collection on high-quantity low-cost highway assets. Despite the importance, current practices of asset data collection are still manual and time-consuming. There is a need for a well-managed asset data collection that can provide useable asset inventories to departments of transportations (DOTs) for further analysis and condition assessment purposes. In this paper we present a novel video-based recognition and 3D reconstruction algorithm.
Scalable Non-Parametric Parsing for Segmentation and Recognition of High-Quantity Low-Cost Highway Assets from Car-Mounted Video Streams
Systematic condition assessment on high-quantity low-cost highway assets requires frequent reporting on location and up-to-date status of these assets. Recent research on video-based assessment of assets have primarily focused on detecting traffic signs during data collection, and are not applicable to other assets such as guardrails and light poles. To overcome such limitations, this paper presents fast graph-based segmentation and super-parsing algorithms which efficiently segment highway assets from 2D video streams.
Video-based Detection and Classification of US Traffic Signs and Mile Markers using Color Candidate Extraction and Feature-based Recognition
Traffic sign and mile-marker detection and classification are among the important components of highway asset management systems. The significant number of these high-quantity and low-cost assets in US highways can negatively impact on quality of any manual data collection and analysis. To address these challenges, this paper presents an efficient pipeline for video-based detection and classification of traffic signs and mile-markers based on color and shape criteria.