Real-Time Queries on Large Volumes of Safety Text
conference paper
It is often necessary to parse large volumes of text in the process of carrying out Safety and Risk Management duties. One example of this is the Close Call system, operated in the UK to log safety related incidents on the GB railways. Approximately 300,000 unstructured text reports are added each year. Traditionally, locating and categorizing potential risk indicators in the Close Call text (and other systems like it) has been a human task. Though steps have been taken towards augmenting this with computer-based analysis, real-time feedback has not been possible. This paper will discuss a platform which allows real-time queries on large volumes of text. A novel application of Integer based hashing is applied to n-grams of the text. Using this method, in combination with search optimizations such as binary searching (which would be cumbersome or impossible to perform on unmodified text) it can be shown that pattern matching performance is improved by several orders of magnitude when compared to Brute force matching, or even more developed methods such as the Boyer-Moore algorithm.
TNO Identifier
947020
ISBN
978-981-11-2724-3
Publisher
Research Publishing Services
Source title
Proceedings of the 29th European Safety and Reliability Conference
Editor(s)
Beer, M.
Zio, E.
Zio, E.
Place of publication
Singapore
Pages
1800-1803
Files
To receive the publication files, please send an e-mail request to TNO Repository.