Prof. Ing. Francescomaria Marino

Associate Professor

DEE Politecnico di Bari

Rail inspection is a very important task in railway maintenance and it is periodically needed for preventing dangerous situations. Inspection is operated manually by trained human operator walking along the track searching for visual anomalies. This monitoring is unacceptable for slowness and lack of objectivity, because the results are related to the ability of the observer to recognize critical situations.
With the growing of the high-speed railway traffic, companies over the world are interested to develop automatic inspection systems which are able to detect rail defects, sleepers’ anomalies, as well as missing fastening elements. These systems could increase the ability in the detection of defects and reduce the inspection time in order to guarantee more frequently the maintenance of the railway network.

VISyR, is a patented [1] real time Visual Inspection System for Railway maintenance, able to analyse defects of the rails and detect the presence/absence of the fastening bolts that fix the rails to the sleepers, and it is partially supported by the research project PON “RAILSAFE” funded by the Italian Ministry of University and Research (MIUR).

It is based on the hardware implementation of RailScanner, a software tool developed by the CNR-ISSIA [2], [3]. The development board, employed for the prototypal realization of VISyR, has been the Altera’s PCI High-Speed Development Kit, Stratix™ Professional Edition [4], which among other features, presents a Stratix™ EP1S60F1020C6 FPGA [5], 256-MByte PC333 DDR SDRAM, 32-bit or 64-bit PCI and 8/16-bit differential I/O up to 800 Mbps. The hardware design has been carried out by Gianluigi and Pasquale De Ruvo under the scientific direction of Prof. F. Marino at the DEE of the Politecnico di Bari. A detailed description of VISyR can be found in [6], [7].

Shortly,VISyR acquires images of the rail by means of a DALSA PIRANHA 2 line scan camera [8] having 1024 pixels of resolution (maximum line rate of 67 kLine/s) and using the Cameralink protocol [9]. Furthermore, it is provided with a PC-CAMLINK frame grabber (Imaging Technology CORECO) [10].
A long video sequence captured by the acquisition system is fed into Prediction Algorithm Block (PAB). Moreover, PAB receives a feedback from the Bolt Detection Block, as well as the coordinates of the railways geometry by the Rail Detection & Tracking Block. PAB exploits this knowledge for extracting 24x100 pixel windows where the presence of a bolt is expected .
These windows are provided to the 2-D DWT Preprocessing Block (DWTPB). DWTPB reduces these windows to two sets of 150 coefficients (features) resulting respectively from a Daubechies DWT and a Haar DWT. The features are therefore provided respectively to the Daubechies Classifier and to the Haar Classifier. The output from DC and HC are combined in a logical AND in order to produce the final response of the classification. This “cross validated” response avoids (practically-at-all) false positive, and reveals the presence/absence of the fastening bolts with an accuracy of 99.6% in detecting visible bolts and of 95% in detecting missing bolts.

In order to speed-up the processing, the Interface extracts from the video sequence the image areas candidate to contain the hexagonal-headed bolts, i.e., only those windows requiring inspection. More in detail, the system uses two kinds of search: Exhaustive search and Jump search. In the first kind of search, a window exhaustively slides on the areas at a (well-known) distance Dx from the rail location, until it finds contemporaneously (at the same y) the first occurrence of the left and of the right bolts. At this point, it determines and stores this position and continues in this way until it finds the second occurrence of both the bolts. Now, it calculates the distance along y between these two positions along y (Dy) and the process switches on the Jump search. In fact, as it is well known, the distance along y between two adjacent sleepers is fixed. Therefore, the Jump search uses Dy to jump only in those areas candidate to enclose the windows containing the hexagonal-headed bolts, saving computational time and speeding-up the performance of the whole system. If, during the Jump search, VISyR does not find the bolts in the position where it expects them, then it stores the position of fault (this is cause of alarm) in a log-file and restarts the Exhaustive search. The position of these windows is computed thanks to the knowledge of the geometry of the rail, with respect to the position of the head of the rail.

The task of detecting and tracking the head of the rail is performed by a second module of VISyR, which is based on the Principal Component Analysis and the Single Value Decomposition. This technique allows the detection of the coordinates of the center of the rail analyzing a single row of the acquired video sequence (and not a rectangular window having more rows). This keeps extremely low the time for I/O. The accuracy reached in this task is 98.5%. Basically, rail tracking is achieved by a Multi Layer Perceptron Network which implement the Tracking Classifier and using as features the result of a SVD pre-processing. If the analyzed strip is classified as not centered on the head of the rail, the system shifts the research alternatively on left and on right, until the analyzed row does not fit the center of the rail. the module provides the coordinates of the center of the rail to the system that uses this data to extractr the windows to be inspected. This task is therefore halted and it will be resumed (starting from the last detected coordinates when the system will turn on the Exhaustive search.

In order to save hardware, Tracking Classifier is implemented on the same logic as the Daubechies Classifier: this does not decrease the performance, since rail tracking and bolts detections are not concurrent tasks. The shared hardware works during these two distinct steps using as coefficient of the multipliers of the perceptrons two different sets, as determined during the training of the two classifiers.

The implemented FPGA-based architecture allow to speed up the system performance in terms of the inspection velocity: VISyR analyses video at a velocity higher than 450 km/h (Jump search) and approximately at 5 km/h (Exhaustive search), reaching a composite velocity higher than 160 km/h for the tested video sequence covering more than 9 km.
If the system remains in the Jump phase for a long time, performance can increase subsequently. Next work will be addressed in this direction, for example, automatically skipping those areas where the fastening elements are covered by asphalt (i.e., level crossing, where Exhaustive search is executed in continuous).

VISyR constitutes a significant aid to the personnel in the railway safety issue because of its high reliability, robustness and accuracy (99.6% visible bolts and 95% absent bolts correctly detected). Moreover, its computing performance allows a more frequent maintenance of the entire railway network. Detection of different types of fastening bolts employed in railway infrastructures has been also implemented because of the flexibility of its FPGA-based implementation. In particular, detection of hook bolts have been even tested downloading onto the FPGA different set of neural weights, generated by a proper training step.


[1]   A. Distante, F. Marino, P.L. Mazzeo, M. Nitti and E. Stella, “Metodo e Sistema Automatico di Ispezione Visuale di una Infrastruttura”, Italian Industrial Patent N. RM2005A000381, owned by the Italian National Research Council, 2005.
[2]   E. Stella, P.L. Mazzeo, M. Nitti, G. Cicirelli, A. Distante and T. D’Orazio, “Visual recognition of missing fastening elements for railroad maintenance,” IEEE-ITSC International Conference on Intelligent Transportation System, pp. 94-99, Singapore, 2002.
[3]   P.L. Mazzeo, M. Nitti, E. Stella and A. Distante, “Visual recognition of fastening bolts for railroad maintenance,” Pattern Recognition Letters, vol. 25 n. 6, pp. 669-677, 2004.
[6]   F. Marino, A. Distante, P.L. Mazzeo and E. Stella, A Real Time Visual Inspection System for Railway Maintenance: Automatic Hexagonal Headed Bolts Detection, to be published on IEEE Transactions on Systems, Man and Cybernetics-Part C.
[7]   F. Marino, et al., “A Real Time Visual Inspection System for Railway Maintenance: Automatic Rail Detection and Tracking”, sottomesso per la pubblicazione su IEEE Transactions on Systems, Man and Cybernetics-Part C.
[9]   “CAMERALINK: specification for camera link interface standard for digital cameras and frame grabbers,”

poliba Site Map | mova::webdesign | ©2006 Politecnico di Bari