Development of safety-critical software exhaustive testing framework for nuclear power plant digital i&c system

Abstract

By its nature, the software is a logical matter and determines the function of hardware in the digitalized environment. The space that digitalized input and internal variables construct can be considered as the domain that the software may encounter during system operation, which may be very large but not infinite. For many safety-critical systems, the size and complexity of the program are relatively small and simple, and the number of inputs and internal variables is limited to finite resolution. Therefore, if we can perform the software testing over the whole of this space, the issues related to input selection and model parameter estimation can be resolved. To make it practically achievable, the speed of software testing also needs to be drastically improved compared to the conventional hardware-based test methods.

An important characteristic of the proposed software test approach is that the test sets can be systematically and quantitatively derived to achieve exhaustive testing of the safety-critical software. In addition, the developed simulation-based test-bed can effectively reduce the software testing time per test case compared to the existing black-box testing from few minutes to few milliseconds by emulating the software behavior given the software input and internal states at the machine language level and automating the process from providing the software input to verifying the output. The proposed software test method is expected to be used to support the software reliability quantification of NPP safety-critical I&C applications and further ensure the safety of software-based digital systems.

The proposed software testing method was applied to the safety-critical trip logic software of an integrated digital protection system-reactor protection system (IDiPS-RPS), a fully digitalized reactor protection system developed under the Korea Nuclear Instrumentation and Control Systems project. Among 15 trip logics of IDiPS-RPS, the pressurizer pressure low trip logic, which is one of the most complicated logics among IDiPS-RPS trip logics, was chosen as a case study to demonstrate the effectiveness of the proposed software test method. As a result, a total of 4,206,164,480 exhaustive test cases were generated for the case study and were tested using a simulation-based test-bed where all test cases generated the pressurizer pressure low trip logic signal. Compared to the traditional approach where all possible combinations of each software variable are exercised and the test cases are derived by extracting the cases which generate the desired software output, the proposed framework showed 2.94 times faster performance in generating each exhaustive test case. In addition, the test execution time per each test case using the simulation-based software test-bed was 6.205 milliseconds in average which is much faster compared to the that of the hardware-based test-bed which ranges from few seconds to few minutes.

As the use of digital I&C systems triggered a challenge in incorporating the software failure into the risk assessment of digitalized NPPs, the software testing for safety-critical systems in nuclear power plants (NPPs) has become an important issue. In response, various quantitative software reliability methods such as software reliability growth model, Bayesian belief network model, and test-based methods have been proposed and adopted in the nuclear field. However, the limitations of the state-of-the-art methods include: (1) the uncertainty in estimating model parameters, (2) the limitation on demonstrating the tested inputs match to the actual operation profile, and (3) a long testing time for each case.

In this research, an exhaustive software testing method based on an automated test case generation framework for the function block diagram (FBD) programs used in NPP safety systems combined with the simulation-based test-bed was developed. As the software output is determined by the combinations of the states of software input and internal variables, generating the exhaustive test cases can be considered as a problem of finding the solutions that satisfy the on-demand situation of a software. The proposed test case generation framework translates FBD program to semantically equivalent SMT formula based on the formal definition of FBD and generates exhaustive test cases given desired software output by iteratively solving the SMT formula. In addition, an emulation-based software test-bed was developed which emulates the microprocessor architecture and memory map of a safety-critical programmable logic controller (PLC) used in NPP digital I&C system and captures its behavior at each machine instruction while the software executes its dedicated safety function. The test-bed can be used to execute the FBD program given the test cases and generate the test results by comparing the software output generated by the test-bed and the expected output.