# Introduction

oftware is an important part of a computer system. Being complex or created by incompetent developers, faults might be introduced to the software. There are faults that cause violating the system security. These faults are called vulnerability. There has been much research on preventing, detecting and analyzing software vulnerabilities.

By the time of writing this paper there is a number of surveys on the methods of mitigating vulnerabilities, i.e. [1], [2], [3] and [4]. Among them, [4] surveys the static analysis vulnerability detection methods that are applied in three areas that are associated with sources of vulnerabilities, i.e., accesscontrol, information-flow and application-programmingconformance. It reviews around 88 papers. The studied methods, however, do not cover all the software vulnerability classes. Static analysis methods are also surveyed in [3]. It reviews 23 papers and classifies their methods with a different point of view. In [1] static and dynamic analysis methods are classified and 18 papers are briefly reviewed. The classification for static analysis methods presented in that paper is similar to the on in [3]. The most comprehensive survey is presented in [2] by Shahriar et al. in 2012. They review 173 papers and classify their methods in four classes, i.e., static analysis, dynamic analysis, monitoring and hybrid analysis.

In this paper, we present a new definition for software vulnerability. Based on this definition, vulnerability mitigation methods are classified and reviewed with a new point of view. We use the general classification of fault mitigation methods as a base and extend it to a detailed classification of software vulnerability mitigation methods.

Our comprehensive classification aggregates many of the classification presented in the previous surveys, i.e., [1], [3], [2] and [5]. Also, the general perspective applied in our survey helps to identify the fault mitigation methods that are not yet used in mitigating software vulnerabilities. Since we consider the software vulnerability as a type of fault, these methods may be helpful in mitigating software vulnerabilities. We suggest new directions for the future researches based on our analysis during the review of the proposed vulnerability mitigation methods.

In this paper, our definition of software vulnerability is presented in section II. Based on this definition, software vulnerability mitigation methods are classified in section III. In this section, each class is described in details and some examples are reviewed. Section IV concludes the paper and presents some future directions.


# II.


# Defining Software Vulnerability

To review vulnerability mitigation methods, a precise definition of software vulnerability is required. Different researchers have suggested definitions for this term which are nearly analogous but have differences. Matt Bishop et al. define software vulnerability by modeling the software as a state machine in [6], [7], [8] and [9]. In this model, a vulnerable state is the state that let unauthorized reads, changes or accessibility modifications to a source. They define vulnerability as a property in the system that let it enter into a vulnerable state. In [8] Bishop defines vulnerability as a weakness that makes it possible for a threat to occur, where a threat is a potential violation of security policy. Amoroso defines vulnerability as an unfortunate characteristic that allows a threat to potentially occur [10]. There are other definitions of software vulnerability in relation with S security policy, e.g. [11] and [1]. Most of them define it as a property, characteristic or weakness that may cause compromising the security policy.

In order to clarify the terms property and security policy compromise, we redefine "software vulnerability". We use the precise definitions for the concepts in software security and reliability that are presented in [12] and construct our definition of software vulnerability. The taxonomy in [12] is presented in 2004 for the concepts of software security and reliability, such as fault, error, failure, vulnerability and attack. The authors define fault as the cause of error, while error is a state of the system that is probable to failure. Failure -or service failure is an event in which the delivered service is deviated from the correct service. In fact, a fault may become active and produce an error. Also the error may propagate inside the system and produce more errors. If the propagated error reaches system boundaries and affects the services, it becomes a failure.

A service is defined in [12] as the behavior perceived by users in system boundaries. Correct services are determined by the system specification. Some parts of the system behavior are specified by the security policy, which is a partial system specification. Thus when a system deviates from the security policy, a security failure occurs. This means that compromising security policy causes a security failure.

Faults are classified in [12] based on eight criteria, such as the phase of creation or occurrence, the objective, the phenomenological cause, the system boundary and the dimension. All combinations of the eight elementary fault classes would result in 256 different combined classes. The authors, however, believe that not all combinations are possible. For example, there is no malicious non-deliberate faults, or all the natural faults are non-malicious.

An attack is defined in [12] as a malicious external fault. An attack may be either an external hardware malicious fault, such as heating the RAM with a hairdryer to cause memory errors, or an external software malicious fault, such as a Trojan horse [12]. The term vulnerability is also defined in [12] as an internal fault that enables an external fault to harm the computer system, although harming the computer system is not clearly defined.

According to the previous definitions, we consider software vulnerabilities as:

We have concluded this definition, out of the definitions in [12], [8], [10], [11] and [1], since looking a vulnerability as a fault, instead of a property, better clarifies the concept of vulnerability by considering its relation to error, security failure and thus security policy. Like faults, a vulnerability may be dormant and never be activated. It also may be activated and propagated in the system. The activated vulnerability might never reach the boundaries. As an example, suppose that a buffer overflow occurs and the value of a return address in the stack changes as a result. But using a monitoring procedure, the unauthorized change is detected and the program halts. Thus, the security policy is not violated. Monitoring the program, as a vulnerability detection method, is explained in section III-B. When an active vulnerability reaches the system boundaries, it causes a security failure. For example, an attacker may activate the format string vulnerability in a program and make it print some confidential data from the memory [13]. Since the active vulnerability has reached the system boundaries, it has made a security failure.


# III. Vulnerability Mitigation Methods

Since vulnerability mitigation is a well-known research area, a structured approach is required to review the previous related works. In this paper, we review vulnerability mitigation methods using a new point of view. We classify and review these methods based on how we define software vulnerability. In the previous section, software vulnerability is defined as an internal software fault. Since we considered vulnerability as a type of fault, the classifications of fault mitigation methods can be used as a base for classifying vulnerability mitigation methods. Avizienis et al. present a classification for the means of mitigating the faults to achieve a secure and dependable system in [12]. We use this general classification as a base and extend it into a detailed classification of vulnerability mitigation methods. Our classification is illustrated in figure 1. The vulnerability mitigation classes that are shown in figure 1 are described in more details in the following sections. This figure presents a comprehensive view of the previous efforts in mitigating software vulnerabilities. Our classification also aggregates the classifications presented in the previous surveys, such as the ones presented in [1], [3], [2]. Moreover, this classification helps to identify the fault mitigation methods that can be applied to improve current software vulnerability mitigation methods. This helps to suggest new directions for the future research.


# a) Vulnerability prevention

Generally, fault prevention means avoiding the fault introduction and occurrence in the application during the development. A fault may be introduced during any of the development phases: requirement analysis, design and implementation. To prevent the occurrence of software vulnerabilities during these phases, software security is emerged. Software security is the process of designing, building and testing software for security [14]. It aims at designing and implementing a secure software and educating developers, architects and users to build security in the software [14]. There are various secure software development methods presented by now, such as Microsoft Security Development Lifecycle (SDL) [15], Security Quality Requirement Engineering (SQUARE) [16] and McGraw's secure development method [14]. Also, there are secure coding best practices that are suggested for different programming languages. These best practices educate the programmers to prevent introduction of well-known vulnerabilities during the coding phase, such as [17] for .NET framework, [18] for C/C++ and [19] for Java.

The programmers' lack of security knowledge is an important reason for the introduction of vulnerabilities. Transferring the related information to the developers is an issue in vulnerability prevention. The SHIELDS project was an example of the attempts in this area [20]. The goal in this project was to create a database of security related information for programmers that can be used automatically. A unified modeling language was proposed in SHIELDS for representing this information [21]. Using this language, it is possible to specify a vulnerability class and its relations to the well-known attacks. It also helps to define the methods of preventing a vulnerability class. Thus, it helps the developers to learn how to prevent vulnerabilities in order to achieve the security goals of the application. Some tools were also developed based on this language in that project, such as GOAT [20] and TestInv-Code [22].


# b) Vulnerability Tolerance

In spite of vulnerability prevention efforts, vulnerabilities are created. Thus, vulnerability tolerance is required. Generally, fault tolerance methods accept the existence of faults and focus on preventing the activated faults from reaching the system boundaries and causing a failure. Fault tolerance is usually performed in two steps: error detection and recovery [12]. Therefore, we study monitoring methods based on three aspects: the applied error detection, error handling and fault handling techniques. Please note that since we look a vulnerability as a fault, we consider error as an active vulnerability. Thus, the mentioned three aspects are also named as active vulnerability detection, active vulnerability handling and vulnerability handling techniques respectively. Fig. 1: Our classification of vulnerability mitigation methods according to the classification of fault mitigation methods in [12]. The boxes with dashed borders show the methods that have not been used in mitigating software vulnerabilities yet.


# C Error detection (active vulnerability detection)

There are vulnerability mitigation methods that control the execution of a program and detects active vulnerabilities at run-time. These methods are also called monitoring methods [2]. Various active vulnerability detection techniques have been used in the proposed monitoring methods. Some examples are monitoring the memory and validating its integrity [23], [24], [25], controlling the flow of user provided data (taint analysis) [26], [27], [28], [29] and validating the arguments of specific functions [30], [31], [32].

For example, the return addresses of functions in the stack memory of the program are monitored in [23], [24] and [25] to detect stack overflows at run-time. If any unauthorized changes of the return addresses is detected, it is concluded that a buffer overflow vulnerability has become active in the program. Some monitoring methods track the flow of user provided un-trusted data at run-time and react appropriately if the untrusted data reach sensitive statements in the program, such as [26], [27], [28], [29]. This method is used to tolerate various vulnerabilities, such as DOM-based XSS [29], SQL injection [26], [27], [28], buffer overflow [26], [27], [28] and format string [26], [27], [28]. Some monitoring methods locate specific functions in the program and control their arguments during the program execution, such as [30], [31] and [32].

For example, in [31] the program code is analyzed statically and the query strings, that are used as the arguments of SQL functions, are parsed to extract the ASTs of legitimate queries. In this method, the code is instrumented to control the values of SQL queries before executing the relevant functions. Before executing a query with un-trusted data, the monitoring procedure extracts the AST of the query. It then compares the extracted AST with the AST of the legitimate queries. Any inconsistency between the two ASTs might reveal a malicious query. Thus, an appropriate reaction is taken by the monitoring procedure to prevent security failures.

Detecting the errors may be performed during the normal service delivery (concurrent detection). Also, it may be performed in specific times in which the application does not deliver services (preemptive detection). The latter is usually applied to eliminate the negative effects of software aging. All the studied monitoring methods detect active vulnerabilities during the normal service delivery. However, preemptive error detection can be used to detect the activation of vulnerabilities that makes the program overuse the system resources, like the memory leakage vulnerability.


# Error handling (active vulnerability handling)

After an error is detected, it is handled in one of three ways: rollback, roll-forward and compensation.

Many of the presented monitoring methods focus on detecting active vulnerabilities, but less attention is paid to handling the active vulnerabilities. It seems that more effort is required on designing appropriate handling methods for active vulnerabilities. Although halting the program and throwing an exception prevents a successful attack, they violate the availability of the software to the legitimate users. Thus, it may result in deniable of service. Therefore, more intelligent active vulnerability handling techniques should be designed for the monitoring methods. Since the rollback technique is usually used for the transient faults and software vulnerability is a permanent fault, this technique cannot be applied in the monitoring methods. Thus, the roll-forward and compensation techniques can be used to design more intelligent active vulnerability handling methods.


# Fault handling (vulnerability handling)

After handling the error, sometimes fault handling is performed to remove the fault and prevent the similar errors in the future. Of course, sometimes the fault is handled immediately after error detection. Fault handling is performed by first recognizing causes of the Using the rollback method, the system is restored to a previously stored error-free state. Then, the program continues normal execution from the restored state. In some applications, such as real-time applications, there is no time to rollback. Thus, roll forwarding is performed to change the system state into a degraded new state that contains no errors. Then, the program executes normally from the degraded state. Roll-forwarding is applicable for predictable errors. Another error handling method is compensation. In this method, the redundancy in the current state is used to mask the error and let the program continues the execution. Many of the monitoring methods halt the program and generate an error message when they detect an active vulnerability, e.g. [32], [27], [33], [24]. In other words, many of the monitoring methods do not perform error handling. Some monitoring methods call an exception handler and take the program to a pre-defined state [34], [26], [28]. Most of the monitoring methods that are used for web applications ignore the requests that result in errors and continue normal execution [30], [31], [35], [29]. Calling exception handlers and ignoring the malicious requests can be considered as simple rollforwarding actions, since the erroneous state is changed into an error-free state and the program continues normal execution. However, more intelligent reactions can be performed after detecting active vulnerabilities. For example, in [36] the stack content and return addresses are stored to compensate for buffer overflow errors. When a buffer overflow error is detected, the monitoring procedure uses the stored data to help the program continue execution securely.

error. Usually, the faulty component is isolated to prevent the future activation of the fault. A spare faultfree component is then replaced by the faulty one. The system is reconfigured based on the new structure. We are not aware of any monitoring method that consists of a vulnerability handling procedure. However, there are some specific methods for automatically patching the software vulnerabilities, such as [37], [38], [39], [40] and [41]. These methods might be usable in the proposed vulnerability tolerance methods to handle the vulnerabilities. The automatic patching methods analyze the malicious data that is used in an attack and modify the program to filter similar data in the future. These methods can be combined with preemptive active vulnerability detection techniques to generate a complete vulnerability tolerance solution.

Table I summarizes the presented vulnerability tolerance methods so that the reader can review them easier. To sum up, there are various monitoring methods with enhanced error detection mechanisms presented by now. These methods pay more attention to detecting the errors. This might be due to the difference between software vulnerability and the other faults. Usually, software vulnerability is activated by malicious external faults. Therefore, detecting an active vulnerability reveals an ongoing attack. The software should resist the attack as soon as possible to prevent further damages. Thus, the quick detection of the active vulnerability is very important. Halting the program is the fastest low-risk response to the attack. However, it makes the program unavailable to the legitimate users as well. Thus, more intelligent error handling and vulnerability handling techniques should be added to the monitoring methods. To do so, a good starting point is inspiring by the current fault handling and error handling techniques and designing software vulnerability handling techniques.


# c) Vulnerability removal

Vulnerability removal is performed to detect and remove the vulnerabilities that are created in software despite the vulnerability prevention efforts. Based on figure 1, the fault removal process consists of four steps: verification, diagnosis, correction and nonregression verification. During the verification step, it is verified if the system adheres to the specification. If not, the reason (fault) is diagnosed and corrected. After removing the fault, the verification is repeated to check if the removal was effective. The verification at this step is called non-regression verification.

Most of the vulnerability removal methods focus on the verification step and don't suggest any diagnosis or correction methods for the detected vulnerabilities. There are, however, special vulnerability diagnosis methods that diagnose the vulnerabilities that are exploited by malicious users. For example, in [42] exploitation of memory corruption vulnerabilities is detected and then the exploited vulnerability is automatically diagnosed. The result of diagnosis consists of the instruction that are exploited by an attacker to corrupt critical program data, the stack trace at the time of memory corruption and the history that the corrupted data are propagated after the initial corruption. This information helps the developers to remove the diagnosed vulnerabilities. We could not find any vulnerability diagnosis or correction procedure that is used after the verification step of a vulnerability removal method. We need vulnerability diagnosis and correction procedures that can be used after the verification step, not after detecting an attack. In other words, these procedures should not be based on the attack information, but based on the information achieved during the verification step.

Some vulnerability detection methods perform the verification step by checking if the software adheres to the security specification, while some of them verify if specific vulnerabilities exist in the software. Figure 1 illustrated our classification of vulnerability verification methods. We divide the verification methods into three main classes: static, dynamic and hybrid methods.

i. Static analysis Static analysis methods do not execute the program. Instead, they examine the program code and study its possible behaviors. Therefore, the result of static analysis is true regardless of the input data and static methods are usually sound and conservative [43]. A sound method is able to detect any specified vulnerability in the program. In other words, if a vulnerability is defined for the static analyzer and exists in a program, the analyzer will surely find it. In order to be sound, the analyzer produces conservative results that are weaker than the actual ones and may not be very useful [43]. In fact, static analysis is appropriate in proving the absence of a specific vulnerability. Usually static analyzers create many false alarms, hence they cannot be very useful in proving the existence of a specific vulnerability. Static analysis may be performed on the program or on the behavior model of the program [12]. Thus, static analysis methods are divided into two main classes: program-based and modelbased methods. 


# ii. Program-based methods

As figure 1 shows, these methods are classified into seven subclasses. Each class is explained as follows.


# Pattern Matching

The most basic static analysis method is pattern matching. A pattern matcher considers the program as a text file. It may not even distinguish between the code and the comments. The pattern matcher searches for vulnerable functions or patterns in the text of the program code. Thus, this method can be implemented using any pattern matching utility, such as grep. Such a tool needs a database of the vulnerability patterns. As an example, Flawfinder [44] scans C/C++ programs to detect buffer overflow or format string in them. This tool ignores the text inside the comments and strings. However, it does not recognize the type of function parameters and control flow or data flow of the program. This lack of knowledge results in many false decisions. Thus, it makes many false positive and false negative alarms. 


# Lexical analysis

In this method, source code of the program is tokenized in order to recognize the variables and function arguments. Thus, the results of a lexical analyzer can be more accurate than the results of a pattern matcher. As an example, the tool ITS4 applies lexical analysis to detect buffer overflow, format string and race condition vulnerabilities in C or C++ programs [45]. ITS4 scans the source code statically and breaks it into series of lexical tokens. These tokens are compared with the token streams that are defined in a vulnerability database. The vulnerability database contains several handlers for well-known vulnerable functions in C/C++.


# Parsing

In this method, source code of the program is parsed and represented in Abstract Syntax Trees (AST). The ASTs are then used to analyze the program syntactically and semantically. For example, Lint uses this method to detect vulnerabilities in programs written in C [46]. As another example, in [47] the ASTs of the source code are extracted and compared to the ASTs of different vulnerable codes. The main idea in [47] is that different vulnerabilities in software may be related to the same flawed programming pattern. Thus, the suggested method uses the ASTs of known vulnerable codes and searches for similar patterns in the target program. When a similar pattern is found in the program, it may reveal an unknown vulnerability.


# Data flow and taint analysis

In this method, the flow of data among the instructions is analyzed to determine possible values that a variable holds during the run time. Two wellknown program representations are used in this method: control flow and data flow graphs. In a control flow graph, each node represents an instruction and a


# Global Journal of Computer Science and Technology

Volume XVII Issue I Version I  [48] extracts the control flow and data flow graphs from the source code. It then compares extracted graphs with some patterns of known vulnerabilities. In this method, known vulnerabilities are specified as simple patterns of vulnerable functions or more complex flow-based rules.

A subclass of data flow analysis is called taint analysis. A taint analyzer only tracks the flow of data that come from un-trusted resources. The un-trusted resources include the network protocols, keyboard, touchpad, webcam, files, etc. Since most of the vulnerabilities are exploited by un-trusted input data, this method pays attention to the flow of un-trusted input data in the program. If such data reach sensitive statements in the program, a vulnerability may be reported by the taint analyzer. The sensitive statements, called sinks, are defined according to the specified vulnerabilities. For example, the functions that execute SQL queries are usually defined as the sinks for SQL injection vulnerability. The propagation of tainted data among the instructions is determined based on some predefined rules. For example, if the data in a tainted variable is assigned to an un-tainted variable, the untainted variable will get tainted too.

Taint analysis is used in many of the proposed vulnerability detection solutions, e.g. [49], [50], [51], [52] and [53], to detect various vulnerability classes. Since this method focuses on the flow of tainted data, it does not consider the execution paths in the program that are not affected by malicious data. This feature reduces the time of analysis and number of produced false positives. However, there are vulnerability classes that cannot be specified in such a source-sink structure, e.g. logic vulnerabilities. Although an attacker exploits logic vulnerabilities with malicious data, the sinks cannot be easily specified for this class of vulnerability. For example, the sinks for SQL injection vulnerability are the query execution statements. But a sink for logic vulnerabilities may be any statement that manipulates the input data.


# Annotation-based methods

Annotation is a comment that the programmer makes in the code about the desired behavior of a function or an instruction. It may be defined as a set of pre-and post-conditions or as simple pre-execution conditions. An annotation-based analysis algorithm reads the annotations, analyzes the code statically and verifies if the conditions are met in the program. There are plenty of annotation languages presented so far, such as SPLINT [54], MECA [55], Sparse [56], SAL [57] and a Comment [58].

Since there is a huge number of statements and functions in the programs, manual annotation is usually very time consuming and fault prone [58]. There are annotation languages that provide some facilities to annotate the program more easily, such as MECA [55] and aComment [58]. Among them, aComment is designed to help in detecting concurrency faults in the operating systems and allows the programmers to define the pre-and post-conditions that are related to the interrupts in each function. It also infers the annotation of some functions automatically to reduce the programmers' workload. In this way, the programmers are not supposed to annotate all the functions manually.

Although some of these languages help in reducing the required time and effort for annotating the programs, they usually have a different syntax and directed edge between two nodes represents their semantics than the applied programming languages. Therefore, the programmers and verifiers have to make extra efforts to learn another language in order to use this method. Also, the programmers should be familiar with the security requirements of the programs and the vulnerability classes to annotate the program appropriately. Therefore, the success of this method depends on the programmers' knowledge of software security. Moreover, this method is not helpful in analyzing the COTS 1 software and third party components since their source code is not available.


# Constraint analysis

In this method, the program is analyzed statically and some constraints are calculated for specific objects in it. The constraints are defined according to specific vulnerabilities and are solved to verify if the program suffers from those vulnerabilities. Constraint analysis was first proposed by Wagner et al. in [50]. The resulted tool, called BOON, considers the strings in a C program as an abstract data type. There are also predefined functions that manipulate this data type, such as strcpy(), strcat(), etc. BOON summarizes the state of each string by two integer values: the allocated size for the string and its current length. For each string in the buffer, it analyzes the string manipulating statements in the program to verify if the length of the string exceeds its allocated size. If such condition is inferred, the program might contains buffer overflow vulnerabilities.

It is important to note that the constraints are determined by the analyzer in this method, not by the programmer. This makes the constraint analysis method different from the annotation-based analysis method.

Moreover, constraint analysis does not increase the programmer's workload since generation of the constraints is performed automatically and does not involve the programmer. Of course, it cannot profit the programmers' knowledge of the code to do a more efficient analysis.


# Theorem proving

In this method, the software and its specification are expressed as some formulas of logics or algebraic systems. Also, the security requirements of software are expressed as some theorems. Proving these theorems demonstrates the satisfaction of the security requirements. Otherwise, there is a fault (vulnerability) in the program. As an example, in [59] the source code of target program is statically analyzed and some firstorder formulas are generated that assert the absence of certain faults and vulnerabilities, such as out-of-bounds array access. If the generated asserts are proved, the program does not contain such faults and vulnerabilities.

Although the results of analysis are accurate in the theorem proving methods, they demand expertise and enough experience. In fact, theorem proving is difficult to be achieved automatically and requires highquality staff to apply this method, which is very timeconsuming. So it is generally used to verify correct design rather than the actual code [60]. Simple, fast.
A
Does not have any idea about the types of function parameters and control or data flow of the program and so generates many false alarms. [44].


# Lexical Analysis

Tokenizes the code to recognize variables and function arguments.

Variables and function arguments are recognized. More accurate than the pattern matching method.

Lack of knowledge about the syntax and semantics of the code causes false alarms. Requires the high level source code. [45].


# Parsing

Parses the code and represents it in Abstract Syntax Trees (AST) to be analyzed syntactically and semantically.

Understands the code syntactically and semantically, less false alarms in comparison with the above two methods.

Requires the high level source code.

[47],

[46], [30], [31], [66].


# Annotation-based methods

Comments that the programmer makes about the desired behavior of the code. The code is then analyzed statically to verify if the conditions are met.

Profits the programmers' knowledge to do a focused analysis.

The programmer must learn an additional language to do the annotation.

[54],

[55], [56],

[57], [58].


# Theorem proving

The security requirements of software are expressed as some theorems. Proving these theorems, demonstrates the satisfaction of the security requirements or existence of vulnerabilities.


# Accuracy.

Difficult to be achieved automatically and requires high-quality staff to apply this method [59].


# Data flow analysis (Taint analysis)

Tracks the flow of the data that comes from un-trusted resources and warns if the data reaches sensitive program points.

Reduces the analysis time and number of false positives by not considering the execution paths in the program that are not affected by un-trusted data.

Cannot detect vulnerabilities that are not defined specifically in a source-sink structure.

[51], [52], [53],

[67], [50],

[68], [69], [70].


# Constraint analysis

Analyzes the program, associates constraints with some objects in the code and solves them to verify if the program is vulnerable.

Constraints are generated automatically and do not increase the programmer's workload.

Does not profit the programmer's knowledge of the code (in comparison with the annotation method).

[50],

[51], [52], [53].


# Model checking

Models the program and then checks the model to verify if it satisfies specified requirements.

Only the modeling and requirement specification is performed manually by the human analyzer, rest of the analysis is done automatically.

Modeling the program and specifying its security requirements-if done manuallyis time consuming and fault prone. State-explosion problem when the number of program states is large.

[61],

[62], [63],

[71], [64]. In this method, the program is modeled and then analyzed to verify if it complies with its specifications, e.g. [61], [62], [63] and [64]. If a specific requirement is not satisfied in the software, this method provides some counter examples. Model checking helps the human analyzers by automating a noticeable part of the analysis. Although modeling and specifying the requirements may be done manually, analyzing all possible states of the program and verifying the requirements are done automatically. This is a great help in analyzing large programs. A well-known example of using this method for detecting vulnerabilities is MOPS [63]. Using MOPS, the program is modeled as a push-down automaton 2 . Also, the requirements are defined through safety properties. A safety property is represented as a finite state automaton. It defines the ordering constraints on security related operations. After modeling the program and defining its constraints, MOPS searches exhaustively through possible program states to check if a reachable state violates the safety properties.

Abstracting the program in a model is a challenging task in this method. The model should be expressive enough to have a precise analysis. Thus, some model checking methods 2A push down automaton is a type of computational model. It is similar to NFAs except that it uses an additional component called a stack. In this model, state transitions are chosen based on three components; input signal, current state and what is at the top of the stack. Thus, the stack plays the role of an additional memory for it. also help the analyzer to model the target program. For example, MOPS uses the control flow of the program to build its automata. Also, in [64] the GCC compiler is used to automatically model and verify the programs that are written in any language supported by this compiler, i.e. C, C++, Java, etc. This is done by employing an intermediate language of GCC, called GIMPIL, that is common to all the supported languages. The model is extracted from the intermediate representation of the program and is checked against the defined specification by the use of Moped. Moped is a model checking tool for push down systems 3 65].

There are, however, some shortcomings in the model checking method. Although the modeling phase is performed automatically in some model checking methods, the analyzer should manually specify the security requirements in this method. This is again time consuming and may cause errors in the results. Also, the method suffers from state-explosion problem for large programs.

Table II summarizes the reviewed static analysis methods. Note that all these methods inherit the general advantages and shortcomings of static analysis. iv.


# Dynamic Analysis

By executing the program with actual data, dynamic analysis studies the exact run-time behavior of the program. Dynamic analysis can be as fast as the execution of the program, whereas static analysis generally requires more computation time to obtain accurate results [43]. The main challenge in dynamic analysis methods is executing all the possible execution paths in the program and activating all vulnerabilities in those paths. In fact, acquiring an appropriate test data set, that make the program behave more diversely, is an issue in these methods. The most important shortcoming of dynamic analysis methods is that they components; input signal, current state and what is at the top of the stack. Thus, the stack plays the role of an additional memory for it. are unable to guarantee the analysis of all feasible execution paths. Therefore, the dynamic analysis is not sound and is mostly used to prove the existence of specific vulnerabilities in the programs. The dynamic methods are classified into two main classes in [12]: methods that use symbolic input values and methods that use actual (concrete) input values to test the program. Based on the recent advances in dynamic analysis methods, we classify these methods in three classes based on the type of applied input values: concrete execution, symbolic execution and concolic (concrete + symbolic) execution methods. The following subsections describe each class in more details.


# a) Concrete execution

In this method, the program is executed with actual data and its behavior is analyzed to detect vulnerabilities. There are four dynamic analysis methods that use actual data to execute the program during the analysis: fault injection, mutation-based analysis, dynamic taint analysis and dynamic model checking.

i. Fault injection In this method, the external faults are injected to the program to examine its behavior. According to our definition in section II, the external faults abuse the internal faults and cause unauthorized behaviors in the program. In other words, internal faults are activated by the external fault and are propagated to reach the program boundaries. Therefore, inability to handle external faults may reveal a vulnerability in the program.

The external faults may be injected by corrupting input data to verify if the program is able to handle them. Most of the blackbox vulnerability scanners corrupt input data and analyze the reaction of the program, such as [72] and [73]. The black-box scanners have access to the inputs and outputs of the program. They might also have very little knowledge about the program internal structure [74]. They usually create the corrupted data based on known attack patterns to study if the program can resist these attacks or suffers from the relevant vulnerabilities. Another group of dynamic vulnerability detectors that inject corrupted input data to the programs are fuzzers. Takanen et al. introduced fuzzing for detecting vulnerabilities for the first time. They suggested injecting unexpected random input data to the program and studying its behavior [74]. The difference between fuzzers and black-box vulnerability scanners is that fuzzers don't corrupt input data exactly based on a list of attack patterns. In fact, they generate numerous random faulty data hoping that some data make the program crash. The main advantages of this method were simplicity and independence from the analyzed program. Thus, the method could be used easily to detect vulnerabilities in different programs. However, fuzzers were not intelligent enough to corrupt input data effectively and cover most of the execution paths. In order to have better program coverage, new fuzzers focus on producing well-formed corrupted data [75], satisfying data validation checks in the program like checksums [76], being aware of the state of the program during the fuzzing [77] and producing consistent input data with the path conditions to make the program execute all the branches [78], [79], [80], [81]. All these enhancements made fuzzers play an effective role in detecting vulnerabilities during the recent years [82]. Injecting faults into the program can be done randomly or intelligently. By the word random, we mean that faulty data are generated semi-randomly based on predefined patterns. For example, in order to detect buffer overflow, random input data with different lengths are generated. Here the predefined pattern determines the length of input data and the other properties are set randomly. Takanen et al. consider random fuzzers as the ones that make small random changes into the valid data. For example, a FTP fuzzer may randomly add valid/invalid commands to the test data or chose the arguments of the commands randomly [74]. Random fuzzers sometimes use evolutionary algorithms to guide random choices and extend the program coverage, e.g. [83], [84]. Random corruption of data is simple and independent from the logic and structure of the programs. Moreover, randomness helps to reveal a wide range of behaviors of the programs while the designed testcases by the human analyzer may not. This is because the designed test-cases are prepared by a human analyzer who may not think of all possible behaviors of the program.

Corrupting the data intelligently is performed based on a previous analysis of the program. Although it requires more analysis efforts, it helps in extending the program coverage. For example, imagine a program that compares one of the input values with an integer value and exits if they are not equal. Using the random method, the possibility of passing this constraint is one out of 232. By analyzing the code before injecting faulty data, the analyzer is able to extract the constraint and generate the data in a way that complies with the constraint. This helps the intelligent corruption method have more reliable program coverage [74].


# ii. Mutation-based analysis

As mentioned before, acquiring appropriate test data is an issue in dynamic analysis. When the program behaves normally during the test process, it means that either there is no vulnerability in the program or the test data don't reveal the vulnerabilities in the program. In the latter case, the data set is not diverse enough to activate the vulnerabilities. Mutation is a method that is concerned with enhancing the data set during the dynamic analysis. In this method, specific vulnerabilities are injected into the program code intentionally. If the current data set does not detect the injected vulnerability, it will not detect similar vulnerabilities in the original version of the program. Thus, the analyzer augments the data set so that it can detect the vulnerability. A version of a program in which a specific vulnerability is created, is called a mutant. For example, in a mutant the function strncpy( ) is replaced with strcpy( ) to make it buffer overflow vulnerable. A good test data set distinguishes the mutants from the original version of the program and kills them. If no test-case kills the mutants, the data set must be augmented [85].

This method is effective in detecting software vulnerabilities [85], though it requires considerable amount of time and effort. If the changed statements in a mutant are executed by the test data, the mutant would be effective. Otherwise, the result of analysis does not reveal the difference between the mutant and the original version of the program. Therefore, some computations are required to generate appropriate testcases that make the program execute the intended path which contains the vulnerability.

Also, automatic creation of mutants for complex vulnerabilities is a challenge. As an example, the strncpy ( ) functions are automatically changed to strcpy ( ) for creating mutants to detect buffer overflow in [85]. There are, however, more complicated buffer overflow scenarios like copying an array in a loop that causes overflow. Moreover, creating mutants for logic vulnerabilities requires a deep understanding of the logic of the program. Thus, automatic generation of mutants may not be feasible.

iii. Dynamic model checking This method, which is also called executionbased model checking [86], [87], is a model checking method that executes the program exhaustively and checks if it satisfies the specifications. For example, the tools VeriSoft [88], JavaPathFinder [89], CMC [90], Bogor [91] and DART [92] apply this method in their analysis. Random execution in dynamic model checking is mostly the result of two factors: program inputs and scheduling choices of a scheduler [87]. For each random input and schedule choice, the resulted behavior of the program is analyzed by monitoring the process and its environment, e.g. registers and the stack. Here, each state consists of the entire machine state. When the execution reaches a state, in which the specification is compromised, the related input value and schedule choice are presented as a counterexample.

An advantage of dynamic model checking is that by executing the program, the machine handles the semantics of the instructions. In other words, there is no need to formally represent the semantics of the programming language and the machine instructions [87]. However, there is a time-state-soundness tradeoff in this method. Since the states represent the entire machine state, they contain many details and require more storage space. Thus, storing all the states might be infeasible for large programs. At the same time, exploring the states without a history of visited ones may cause visiting similar states again and again. When no state is recorded, the model checker spends too much time to make sure it has traversed all possible states. Storing the states reduces the verification time by making sure that no state is revisited. Yet, it requires too much space [87].

iv. Dynamic taint analysis This method is similar to static taint analysis as it tracks the flow of information from un-trusted sources to the sinks. However, it tracks the flow of tainted data during the execution of the program, some examples are [93], [94], [95] and [96]. Schwartz et al. describe this method precisely in [93]. They introduce a language, named SIMPIL, that formally defines the algorithms of dynamic taint analysis. Before the execution, all the variables are considered untainted. While executing the program, variables may get tainted according to a predefined policy. This policy defines how the taint data propagate from a variable to other variables. For example, when tainted data are used in an argument of an arithmetic operation, the policy defines that the result of this operation should be considered tainted. If a tainted value reaches a sink, the analyzer reports a vulnerability.

The basic taint analysis methods limit taint propagation to the direct assignments. This might make the results of the analysis inaccurate [97]. Sarwar et al. present some scenarios in [97] to show how basic taint analysis can be ineffective. An example scenario is that the tainted data are used in a conditional statement (without any direct assignment to other variables) and affect on the control flow of the program. Also, tainted data might be used to define the number of an iterative action or as the index of an un-tainted array. The taint analysis method should pay attention to these indirect effects of the tainted data in calculating the taint propagation. Considering such effects is not always easy. For example, the tainted variable might cause information leakage through a side channel. To detect such vulnerability, the analyzer should taint a large amount of variables that results in many false alarms [97].

Table III summarizes and compares the advantages and disadvantages of the concrete execution methods. Each method inherits the advantages and shortcomings of dynamic analysis.


# b) Symbolic execution

Using the symbolic execution method, the program is executed with symbolic input values instead of concrete data values [98], [99]. Thus, the values of program variables are represented as symbolic expressions over the symbolic input. During the symbolic execution, the state of the program and the conditions of the current path are calculated symbolically. The path conditions are updated any time a branch instruction is executed. At the end of an executed path, the path conditions are solved using a constraint solver. There are various constraint solvers presented by now, such as STP [100] and Z3 [101] that solve the constraints on binary vectors and Hampi [102] and S3 [103] that solve the constraints on string variables. If the constraint solver solves the path conditions, it generates some concrete input data that are used to execute the intended path in the program.

There are several challenges with the symbolic execution method. For example path explosion, the overhead of constraint solving for complicated paths, non-determinism of concurrent programs and the tradeoff between precision and scalability of modeling the memory are some of the challenges in applying symbolic execution [104]. Cadar and Sen present the challenges of this method and mention some solutions for them [104].

To overcome these challenges, a solution is combining symbolic execution with concrete execution. The result is a new method that is called concolic execution. This method is described in the next section.


# c) Concolic execution

A problem with pure symbolic execution is that the constraints of complex loops and recursive functions may get very complicated and cannot be resolved in an acceptable time [105]. Concrete execution applies real data to execute the program. There is a little chance to traverse all the feasible paths in this method. Using the combined method, concolic + symbolic execution, the concrete data is used to simplify the complex constraints that are generated by the symbolic execution. This method was first presented by Godefroid et al. in [92]. Concolic execution is performed by changing some symbols in the complex constraints into the concrete values. This helps to achieve better program coverage with much less computation overhead.

A New View on Classification of Software Vulnerability Mitigation Methods Concolic execution is used in many of the recent fuzzers to extend their knowledge about the program, such as KLEE [78], EXE [80], Simfuzz [75], CUTE [106], SAGE [79], Taintscope [105] and [107]. For example, CUTE combines symbolic execution with concrete execution to create input data traverse deeper paths in the program. It first executes the program with concrete input data. During the execution, it calculates symbolically the constraints of the executed path. The calculated constraints are then negated one by one, from the last to the first. After each negation, the resulted constraints are queried from a constraint solver. If the constraint solver solves the new constraints, the result is used to generate new test data that traverse other execution paths in the program.


# Global


# Table 3: Concrete execution methods: a comparison

Concolic execution is also used in other dynamic vulnerability detection methods. For example in [108] a dynamic model checking method is applied that uses concolic execution for state-space exploration of the analyzed application. In [108], concolic execution helps to model the application as a finitestate automata and to guide further state-space exploration.


# d) Hybrid analysis

The previous sections described static and dynamic analysis methods and their advantages and shortcomings. The idea of combining static and dynamic analysis was first proposed by Ernst in [43]. He suggested that hybrid analysis can combine the static and dynamic analysis methods to generate a new analysis method that profits a great amount of soundness and accuracy advantages of each method with little sacrifices.

Monitoring and static analysis methods are also combined in [110] to detect SQL injection errors and [83], [84], [75], [76], [78], [79], [80].


# Mutation-based Analysis

Injects vulnerability into the program code. If the current data set does not reflect the injected vulnerability, it would not detect similar vulnerabilities in the original version of the program.

Reduces false negatives by enriching test data.

Expensive in time and computation. Automatic mutation of complicated vulnerabilities is a challenge. [85].


# Dynamic taint analysis and sanitization

Tracks the flow of information from input sources to the sinks during the run-time.

Reduces the analysis time and number of false positives by not considering the paths in the program that are not affected by malicious data.

Cannot detect vulnerabilities that are not defined in specific source-sink structure.

[93], [96].


# Dynamic Model checking

A model checking method in which the program is executed with concrete input values exhaustively.

No need to formally represent the semantics of the programming languages and machine instructions. Time-state-soundness tradeoff.

[88], [89], [90], [91], [92].

From then, many researchers have combined these methods, in different manners, to make up for each other's shortcomings. For example, Monga et al. combine static and dynamic analysis to detect XSS and SQL injection vulnerabilities in PHP applications in [109]. The suggested method first analyzes the code statically and extracts the control flow graph of the functions in it. These graphs are then connected together to obtain an inter-procedural control flow graph (iCFG). The iCFG is analyzed to extract the possible paths from the tainted sources to the sinks in it. For each sink, backward slicing is used to detect the statements that affect the tainted argument. These statements are monitored at run time. When a tainted value is used in a sink, the monitoring procedure passes it to an oracle to verify if it can exploit a vulnerability. The oracle have a database of well-known attack patterns that are used to exploit different vulnerabilities. For example, the implemented oracle for mysql query() performs a limited syntactically analysis on the SQL queries and searches for the tainted characters in unsafe positions. In this method, the sanitizing procedures are assumed to be perfect. prevent the successful attacks. In the static analysis phase the hotspots, that are statements in the program that execute a SQL query, are identified. Also the control flow of the program is extracted. Then, the query strings in the hotspots are parsed. Considering the control flow of the program, a FSA for each hotspot is created to model the legitimate queries. During the monitoring phase, the queries are checked against the relative FSA to prevent execution of malicious queries.

As the last example, hybrid analysis is used in [111] to detect logic vulnerabilities in web applications. The logic vulnerabilities are usually related to the intended functionality of an application. Thus, there is no general specification for them that can be used in different applications. For example, consider an online store that allows the users to use coupons for having discount on specific items. It has a policy which determines that each coupon should be used only once. A logic vulnerability, however, allows the users to reuse a coupon and reduce the cost to zero. Since logic vulnerabilities are created based on the functionality of the application, the vulnerability detection method requires the specification of the program. The proposed method in [111] consists of two steps. First, the web application is executed with normal input data. The executed traces are then analyzed to infer the specification of the application. This is based on the intuition that normal behavior of the program reflects the properties that are intended by the programmer. In fact, because the specification of the program is not always available, this method uses dynamic analysis to obtain it. The inferred specification is presented in the form of likely invariants. In the second step, model checking is used to analyze the web application based on the inferred specification.


# e) Vulnerability forecasting

Generally, fault forecasting is used to predict the quality or quantity of the faults that are left in the system and will be activated in the future. It is mainly concerned with estimating the current reliability of the system and predicting its future reliability. This prediction may be qualitative or quantitative (usually probabilistic). The qualitative forecasting identifies and ranks the future failure modes. Also, the event combinations that lead to the failures are identified.

The quantitative forecasting is performed by modeling or operational testing. These methods are complementary, since the results of operational tests are usually used to model the system more accurately. Software Reliability Growth Models (SRGM) are used generally for fault forecasting. In fact, SRGMs model the testing process [112]. In most of these models, the rate of fault detection gradually reduces and the cumulative number of faults eventually approaches a fixed value. These models help to predict the number of left faults in the software and determine when the software is ready to be released. They are also used to estimate the required efforts for future maintenance.

There are probabilistic models for predicting the rate of vulnerability detection, named Vulnerability Detection Models (VDM). Alhazmi and Malaiya proposed a specific model, named AM 4 for vulnerability detection in [113]. In this model the rate of vulnerability detection depends on two factors: one of these factors reduces as the number of remaining undetected vulnerabilities declines. The second factor increases with the time. In this way, the rate of vulnerability detection is modeled in a S-shaped form. In fact, AML is created based on the observation that the detectors 5  pay little attention to the newly published software. Gradually people become familiar with the software and the detectors pay more attention to it. Thus, the rate of vulnerability detection increases by time and peaks at some period. By the introduction of newer versions of the program, the detectors' interest becomes lower and the rate of vulnerability detection decreases. Alhazmi and Malaiya examined the applicability of this model to various operating systems in [113] and [114]. The results demonstrated that AML fits the data of several operating systems.

All the mentioned models are time-based. It means that they determine the detection rate based on the calendar time. An effort-based model, named AME 2,   is proposed by Alhazmi and Malaiya in [113]. They believe that time-based models do not consider the changes that occur in the environment during the lifetime of the system. Thus, they consider the number of installations as an important environmental factor that affects the rate of vulnerability detection. It is based on the observation that the detectors are more interested in the software that is installed in many computers. Therefore, the rate of vulnerability detection is modeled in AME based on the number of installations. Sungwhan Woo et al. explore the applicability of AML and AME to some HTTP servers, i.e., IIS and Apache. The results indicate that these models are applicable to the HTTP servers in addition to the operating systems [112]. Of course, this method does not consider many of the effective factors on the detection trend. For example, it only calculates the cyclomatic complexity to estimate the code complexity. There are other complexity metrics that can be considered in addition to A problem with the studied VDMs is that they are parametric models that should be fitted to real vulnerability data [115]. To model the vulnerability detection rate in a specific application, a large amount of historical vulnerability data is required. Therefore, it is necessary that many of the vulnerabilities be discovered already. Hence, these models cannot be applied to predict the detection rate for newly released software. Also, it is shown in [116] that the precision of VDMs depend on the number of known vulnerabilities. The precision of the VDMs are usually very low at the early stages in the lifecycle of the program. It seems that the problem is because the models don't consider the features of each application in their predictions. Thus, they need a history of detected vulnerabilities to estimate the security level of the program. There are many features in each application and its environment that affect the rate of vulnerability detection. Rahimi and Zargham present a VDM in [115] based on two effective factors: code complexity and code quality. The code complexity is defined based on the cyclomatic complexity. Also, the code quality determines its compliance with secure coding practices. They believe that more vulnerabilities are detected in the applications with lower code quality. Also, the possibility of detecting vulnerabilities is less in the applications with complicated codes. Thus, the source code of the application is statically analyzed to compute the two factors. The computed data are then used to model the vulnerability detection rate. Since this model does not need a database of detected vulnerabilities, it can be used for newly released applications. The authors analyze four applications to study the impact of these factors on the vulnerability detection trend. The analysis results show that the proposed method can predict vulnerabilities even in early stages of the application's lifecycle.

this one. The environmental parameters can also be considered for a good prediction. As an example, even the seasonal changes affect the rate of vulnerability discovery. It is shown in [117] that more vulnerabilities are reported during the mid-end and year-end months. Also, the presented method in [115] is based on analyzing the source code of the application. So it cannot be helpful when the source code is not available. There are some qualitative methods for estimating the current security level of the application. For example OWASP ASVS consists of several check lists that helps to determine the security level of a web application [118]. It classifies the check lists in thirteen classes, such as authentication, access control, session management, etc. In each class the check lists are grouped into three security levels. If an application passes all the check lists of a group, it is achieves the respective security level. These methods only estimate the current security level. Based on the current level, it is possible to predict the future failure modes. However, we could not find any qualitative method that predicts and ranks the future security failures based on the current state.

V.


# Conclusions

During the past decades various methods have been presented for mitigating software vulnerabilities. A comprehensive classification of the proposed methods helps to achieve a general understanding of this research area. In this paper, we defined software vulnerability as an internal fault. By considering software vulnerability as a type of fault, we classified the vulnerability mitigation methods based on the general classification of the fault mitigation methods. We extended the general classification of fault mitigation methods, represented it in the context of software vulnerability and added more detailed subclasses into it. We divided vulnerability mitigation methods into four main classes: vulnerability prevention, vulnerability tolerance, vulnerability removal and vulnerability forecasting. The vulnerability prevention methods attempt to prevent the occurrence of software vulnerability. Software security and the secure coding best practices are examples of these efforts. The question is why, despite the vulnerability prevention efforts, vulnerabilities are still created. Oliveira et al. believe that educating the developers is not enough for preventing the vulnerabilities [119], because security is not an issue for the developers. They believe that the human's memory is limited and can only keep a limited number of mental elements available at a time. The programmers are also supposed to create applications with correct functionality and acceptable performance. Under the time pressure, an ordinary situation in software programming, the programmers usually chose the simplest solutions for developing the software and pay little attention to the security concerns. Oliveira et al. suggest developing assistant tools that remind the educated programmers the security concerns during the development.

Besides educating the programmers, intelligent assistant tools are required to notify the security concerns at specific statements or functions. Thus, in the future we should work on designing and implementing intelligent assistant tools that help the programmers to avoid generating vulnerabilities during the design and implementation of the applications. These tools should be intelligent enough not to bother the developers with many false alarms. They may use the enhanced static analysis methods to analyze the code during the coding phase and warn the programmers at sensitive situations. This will help the programmers to use their security knowledge more effectively in preventing the vulnerabilities.

Vulnerability tolerance methods accept the existence of vulnerabilities in the programs and prevent the active vulnerabilities from making security failures. In this paper, the vulnerability tolerance methods were studied based on three aspects: the applied active vulnerability detection technique, active vulnerability A New View on Classification of Software Vulnerability Mitigation Methods handling technique and vulnerability handling technique. All the reviewed vulnerability mitigation methods detect active vulnerabilities during the normal execution of the program (concurrently). However, there are active vulnerabilities that overuse system resources and make the resources unavailable to legitimate users after a period of time, such as the memory leakage vulnerability. The security failure as a result of these vulnerabilities can be prevented by checking the system resources periodically. Thus, preemptive error detection can be applied to detect if such vulnerabilities are active.

Most of the vulnerability tolerance methods focus on detecting the active vulnerabilities. However, less attention is paid to handling the (active) vulnerability. In the proposed methods, active vulnerabilities are handled by halting the program, restarting the program or invoking an exception handler. Although these mechanisms limit the negative effects of the active vulnerability, they violate the availability of the application to the legitimate users. Thus, more intelligent vulnerability handling techniques are required for the current vulnerability tolerance methods. A good starting point is inspiring by the current fault tolerance methods. As an example, software diversity is a fault tolerance method that is used to make the programs reliable. In this method, various versions of software with the same specification but different design or implementationprocess the same request and the correct result is achieved by voting the results of the different versions. The result of such system is more reliable, since it is less possible that all the versions of software suffer from the same fault and so a request does not cause errors in all versions. This method can be used in software security to tolerate malicious requests. For example, recently software diversity has been used in [120] to tolerate active vulnerabilities in web browsers. In the proposed method, different browsers are used to process the user's requests. Since the browsers are designed and implemented differently, it is less probable that all the applied browsers contain similar vulnerabilities. Thus, malicious data cannot compromise all the browsers. The correct response to the client's request is achieved by voting the responses of the browsers. Also, some protection mechanisms, such as ASLR that protects system against memory corruption vulnerabilities [121], are inspired by the idea of using diversity to make the program unpredictable for the attackers. A new direction for the future research could be adopting the current fault tolerance methods to handle different software vulnerabilities. As there is no vulnerability handling mechanism in the current proposed methods, we should work on designing complete vulnerability tolerance methods that contain appropriate active vulnerability handling and vulnerability handling mechanisms.

Vulnerability removal is performed to detect and remove the vulnerabilities in the software. The focus of most of the current vulnerability removal methods is on verifying the vulnerabilities. In fact, less attention is paid on designing appropriate methods for diagnosis, correction and regression verification of software vulnerabilities. There are some vulnerability diagnosis and correction methods that are used after detecting the exploitation of a vulnerability. But specific methods are required for diagnosis and correction of the vulnerabilities that are detected during the verification of the program. Currently, automatic patching methods analyze an attack and generate software patches based on the pattern of malicious data that are used in the attack. These methods can be modified to automatically generate patches based on the results of analyzing the program and according to the mechanism of detected vulnerabilities.

There are numerous vulnerability detection methods presented by now. Most of the recent vulnerability detection methods tend to combine the previous methods in order to profit their advantages at the same time. For example, the concolic execution method combines the concrete and symbolic execution methods to reduce the complexity of pure symbolic execution and increase the program coverage. As another example, static taint analysis is used with the constraint analysis method to limit the overhead of program analysis and compute the constraints only on the tainted data [51]. Also, the control and data flow of the program are extracted in [122], [62] and [63] to model the program automatically and detect vulnerabilities by performing the model checking. There are more possible combinations that are not applied yet and might be effective in detecting the vulnerabilities more accurately. For example, the annotation can be used in model checking to profit the programmers' knowledge for modeling the program.

Also, most of the vulnerability removal and vulnerability tolerance methods consider a specific vulnerability class based on their own definition of the relevant software vulnerability. Therefore, an imprecise definition of the intended vulnerability would cause inaccurate results in the proposed method. In addition, some methods only consider a limited number of vulnerability classes. To handle new vulnerability classes, the algorithm of these methods has to be changed. Many of the presented methods or tools are not able to detect all of the vulnerability classes [123], [124]. By now, the researchers' focus was mainly on designing more accurate methods.

A new research trend is making the vulnerability detection methods extendable. In this way, an accurate method for detecting a specific vulnerability can also be used to detect other vulnerabilities. To make a vulnerability detection method extendable, we suggest designing vulnerability detection algorithms that are A New View on Classification of Software Vulnerability Mitigation Methods independent from the sought vulnerability classes. Such algorithms are able to detect any specified vulnerabilities in the program. Designing such methods requires a general model for specifying the vulnerabilities that encompasses any vulnerability classes, even the future ones. Based on this model, various vulnerabilities are specified for the detection algorithms to be detected automatically in the programs.

There are a few extendable vulnerability removal methods, such as [125], [126] and [124]. However, these methods are limited to specific programming languages. Also, some of them are not expressive enough to specify any vulnerability classes. For example, in [127] an extendable vulnerability method is presented for detecting the vulnerabilities in web applications that are written in Java. The specification method of [127] does not support some data types, e.g. integer, float and character. Therefore, it is not possible to define certain operations, such as mathematical operations or comparing the characters, in specifying a vulnerability. This is not a limitation for specifying vulnerabilities in object-oriented programs, such as Java. Because they encapsulate these operations in certain methods for each data type. It is, however, a limitation for specifying vulnerabilities in other languages, such as C. For example, it is not possible to specify integer overflow vulnerabilities in C programs with this method.

The vulnerability forecasting methods predict the number of left vulnerabilities in the software and determine when the software is ready to be released. They are also used to estimate the required efforts for the future maintenance. Some vulnerability forecasting methods use the vulnerability detection models to predict the rate of vulnerability detection during the lifecycle of the software. The first vulnerability detection models were in fact software reliability growth models that were applied for predicting the vulnerability detection rate. The next models were designed especially for the vulnerability detection rate. These models consider effective parameters on the detection of vulnerabilities, such as time and the number of installations. Since these models do not consider characteristics of the software in their predictions, they need a history of the detected vulnerabilities to predict the vulnerability detection rate in the future. These models are not accurate especially at the early stages in the lifecycle of programs. New vulnerability detection models consider the characteristics of the software to achieve more accurate predictions. In this paper, we reviewed a vulnerability detection model that is based on two characteristics of the program: cyclomatic complexity of the source code and the level of compliance with secure coding practices. There are, however, other characteristics that affect on the rate of vulnerability detection, such as software support, version of the program, availability of the source code or usage of third-party components. For example, the rate of vulnerability detection decreases in time for a program with effective support that periodically presents patches and resolves problems in the program. Also, it might be more difficult to detect vulnerabilities in the higher versions of a program than in its lower versions. The future models can use other characteristics of software to model the vulnerability prediction rate more accurately. Also, they can combine software characteristics with the effective environmental factors, such as time and number of installations, to generate more accurate models.

The possibility of analyzing the program and the program analysis method become important when the vulnerability detection models consider the characteristics of software. For example, a model may be based on some characteristics in the source code of the program. Thus, it is not possible to use such model when the source code of the program is not available. Also, vulnerability forecasts based on inaccurate software analysis are not reliable. In the future, static and dynamic analysis methods that are proposed for detecting the vulnerabilities can be used to better analyze the current characteristics of a program and predict the future rate of vulnerability detection accurately. 
![New View on Classification of Software Vulnerability Mitigation Methods Global Journal of Computer Science and Technology Volume XVII Issue I Version I](image-2.png "A")





Reviewed PaperActive vulnerability detectionActive vulnerability handling Vulnerability handlingVulnerability[23]Detect unauthorized changes of return addresses. (preemp-tive)Halts or restarts the program. (rollback)NoneBuffer overflow[24]Detect unauthorized changes of return addresses. (preemp-Halts the program.NoneBuffer overflowtive)[25]Detect unauthorized changes of return addresses. (preemp-Halts the program.NoneBuffer overflowtive)Any vulnerability[26]Monitor the flow of un-trusted data. (preemptive)Invokes exception handlers. (roll-forward)Nonethat is exploitable by malicious inputdata.Any vulnerability[27]Monitor the flow of un-trusted data. (preemptive)Halts the program.Nonethat is exploitable by malicious inputdata.[30]Monitor the argument of SQL-related functions. (preemptive)Ignores the request. (roll-forward)NoneSQL injection[31]Monitor the argument of SQL-related functions. (preemptive)Ignores the request. (roll-forward)NoneSQL injectionMonitor the format argument[32]of printing functions. (pre-Halts the program.NoneFormat stringemptive)[28]Monitor the flow of un-trusted data. (preemptive)Invokes exception handlers. (roll-forward)NoneSQL injection Buffer overflow Format string[36]Detect unauthorized changes of return addresses. (preemp-tive)Recovers the stack. (compen-sation)NoneBuffer overflow[29]Monitor the flow of un-trusted data. (preemptive)Ignores the request. (roll-forward)NoneDOM-based XSS
1Year 201746)( C
Analysis MethodDescriptionAdvantagesshortcomingsExamplesPattern MatchingConsiders the program as a text file and searches for vul-nerability patterns in the text.
2
Analysis MethodDescriptionAdvantagesshortcomingsExamplesFault injectionFaults are generated semi-randomly. (random corrup-tion) Fault injection is based on some previous analysis of the program. (intelligent corrup-tion )Simplicity and independence in random corruption of in-puts. (random corruption) More reliable code coverage, less false negatives. (intelli-gent corruption)Cannot detect logic vulner-ability. Less reliable code coverage. (random corrup-tion) More effort is required for testing each single program. (intelligent corruption)
			© 2017 Global Journals Inc. (US)
			Commercial off-the-shelf
		
		
			

				


* 
	
		Software vulnerabilities, prevention and detection methods: a review
		
			WJimenez
		
		
			AMammar
		
		
			ACavalli
		
		
			RFourier
		
	
	
		Proceeding of the first International Workshop on Security in Model Driven Architecture
				eeding of the first International Workshop on Security in Model Driven Architecture
		
			SEC-MDA
			2009
		
	




* 
	
		Mitigating program security vulnerabilities: Approaches and challenges
		
			HShahriar
		
		
			MZulkernine
		
	
	
		ACM Computing Surveys (CSUR)
		
			44
			3
			11
			2012
		
	




* 
	
		Static techniques for vulnerability detection
		
			KZafar
		
		
			AAli
		
		
			Sweden
		
		
			Linkoping University
		
	




* 
	
		A survey of static analysis methods for identifying security vulnerabilities in software systems
		
			MPistoia
		
		
			SChandra
		
		
			SJFink
		
		
			EYahav
		
	
	
		IBM Systems Journal
		
			46
			2
			
			2007
		
	




* 
	
		Software vulnerability discovery techniques: A survey
		
			BLiu
		
		
			LShi
		
		
			ZCai
		
		
			MLi
		
	
	
		Proceeding of the Fourth International Conference on Multimedia Information Networking and Security (MINES)
				eeding of the Fourth International Conference on Multimedia Information Networking and Security (MINES)
		
			IEEE
			2012
			
		
	




* 
	
		
			MBishop
		
		CSE-95- 10
		A taxonomy of unix system and network vulnerabilities
				
			1995
		
		
			Department of Computer Science, University of California at Davis
		
	
	Technical Report




* 
	
		A critical analysis of vulnerability taxonomies
		
			MBishop
		
		
			DBailey
		
		CSE-96-11
		
			1996
		
		
			Department of Computer Science, University of California at Davis.
		
	
	Technical Report




* 
	
		Computer security: art and science
		
			MBishop
		
		
			2002
			Addison-Wesley
		
	




* 
	
		A general framework for categorizing vulnerabilities regarding their impact on security policy
		
			HRShahriari
		
		
			RJalili
		
		
			MBishop
		
		
			Computers and Security
		
	




* 
	
		Software vulnerability analysis
		
			IVKrsul
		
		
			1998
		
		
			Purdue University
		
	
	Ph.D. thesis




* 
	
		A structured approach to classifying security vulnerabilities
		
			RCSeacord
		
		
			ADHouseholder
		
		CMU/SEI-2005-TN-003
		
	
	Technical report




* 
	
		Carnegie-mellon univ pittsburgh pa software engineering inst
		
			2005
		
	




* 
	
		Basic concepts and taxonomy of dependable and secure computing, Dependable and Secure Computing
		
			J.-CAvizienis
		
		
			BLaprie
		
		
			CRandell
		
		
			Landwehr
		
	
	
		IEEE Transactions on
		
			1
			1
			
			2004
		
	




* 
	
		Preventing formatstring attacks via automatic and efficient dynamic checking
		
			MFRingenburg
		
		
			DGrossman
		
	
	
		Proceedings of the 12 th ACM conference on Computer and communications security
				the 12 th ACM conference on Computer and communications security
		
			ACM
			2005
			
		
	




* 
	
		
			GMcgraw
		
		Software security: building security in
				
			Addison-Wesley Professional
			2006
			1
		
	




* 
	
		How do they do it? a look inside the security development lifecycle at microsoft, MSDN Magazine
		
			MHoward
		
		
			2005
			
		
	




* 
	
		
			NRMead
		
		
			TStehney
		
		Security quality requirements engineering (SQUARE) methodology
				
			ACM
			2005
			30
		
	




* 
	
		
		Secure Coding Guidelines
				
	
	Online; accessed 2016-10-14




* 
	
		
			RCSeacord
		
		Secure Coding in C and C++
				
			Pearson Education
			2005
		
	




* 
	
		
			FLong
		
		
			DMohindra
		
		
			RCSeacord
		
		
			DFSutherland
		
		
			DSvoboda
		
		The CERT Oracle Secure Coding Standard for Java
				
			Addison-Wesley Professional
			2011
		
	




* 
	
		
		The Shields Project
				
			2012
		
	
	Online; accessed 2016-10-23




* 
	
		Unified modeling of attacks, vulnerabilities and security activities
		
			DByers
		
		
			NShahmehri
		
	
	
		Proceedings of the 2010 ICSE Workshop on Software Engineering for Secure Systems
				the 2010 ICSE Workshop on Software Engineering for Secure Systems
		
			ACM
			2010
			
		
	




* 
	
		An advanced approach for modeling and detecting software vulnerabilities
		
			NShahmehri
		
		
			AMammar
		
		
			EMontes De Oca
		
		
			DByers
		
		
			ACavalli
		
		
			SArdi
		
		
			WJimenez
		
	
	
		Information and Software Technology
		
			54
			9
			
			2012
		
	




* 
	
		Stackguard: Automatic adaptive detection and prevention of buffer-overflow attacks
		
			CCowan
		
		
			CPu
		
		
			DMaier
		
		
			JWalpole
		
		
			PBakke
		
		
			SBeattie
		
		
			AGrier
		
		
			PWagle
		
		
			QZhang
		
		
			HHinton
		
	
	
		Usenix Security
		
			98
			
			1998
		
	




* 
	
		Rad: A compile-time solution to buffer overflow attacks
		
			TChiueh
		
		
			F.-HHsu
		
	
	
		Proceeding of the 21st International Conference on Distributed Computing Systems
				eeding of the 21st International Conference on Distributed Computing Systems
		
			IEEE
			2001. 2001
			
		
	




* 
	
		Stackoffence: a technique for defending against buffer overflow attacks
		
			BBMadan
		
		
			SPhoha
		
		
			KSTrivedi
		
	
	
		Proceeding of International Conference on Information Technology: Coding and Computing. ITCC 2005
				eeding of International Conference on Information Technology: Coding and Computing. ITCC 2005
		
			IEEE
			2005
			1
			
		
	




* 
	
		
			MDalton
		
		
			HKannan
		
		
			CKozyrakis
		
	
	
		Raksha: a flexible information flow architecture for software security
				
			ACM
			2007
			35
			
		
	




* 
	
		Secure program execution via dynamic information flow tracking
		
			GESuh
		
		
			JWLee
		
		
			DZhang
		
		
			SDevadas
		
	
	
		ACM SIGOPS Operating Systems Review
		
			38
			
			2004
			ACM
		
	




* 
	
		Dytan: a generic dynamic taint analysis framework
		
			JClause
		
		
			WLi
		
		
			AOrso
		
	
	
		Proceedings of the 2007 international symposium on Software testing and analysis
				the 2007 international symposium on Software testing and analysis
		
			ACM
			2007
			
		
	




* 
	
		Precise clientside protection against dom-based cross-site scripting
		
			BStock
		
		
			SLekies
		
		
			TMueller
		
		
			PSpiegel
		
		
			MJohns
		
	
	
		Proceedings of the 23rd USENIX security symposium
				the 23rd USENIX security symposium
		
			2014
			
		
	




* 
	
		A method of detecting sql injection attack to secure web applications
		
			SManmadhan
		
		
			TManesh
		
	
	
		International Journal of Distributed and Parallel Systems
		
			3
			6
			1
			2012
		
	




* 
	
		The essence of command injection attacks in web applications
		
			ZSu
		
		
			GWassermann
		
	
	
		ACM SIGPLAN Notices
		
			41
			
			2006
			ACM
		
	




* 
	
		Formatguard: Automatic protection from printf format string vulnerabilities
		
			CCowan
		
		
			MBarringer
		
		
			SBeattie
		
		
			GKroah-Hartman
		
		
			MFrantzen
		
		
			JLokier
		
	
	
		USENIX Security Symposium
				Washington, DC
		
			2001
			91
		
	




* 
	
		Multi-variant program execution: Using multi-core systems to defuse buffer-overflow vulnerabilities
		
			BSalamat
		
		
			AGal
		
		
			TJackson
		
		
			KManivannan
		
		
			GWagner
		
		
			MFranz
		
	
	
		proceedings of International Conference on Complex, Intelligent and Software Intensive Systems. CISIS 2008
				International Conference on Complex, Intelligent and Software Intensive Systems. CISIS 2008
		
			IEEE
			2008
			
		
	




* 
	
		Countering code-injection attacks with instruction-set randomization
		
			GSKc
		
		
			ADKeromytis
		
		
			VPrevelakis
		
	
	
		Proceedings of the 10 th ACM conference on Computer and communications security
				the 10 th ACM conference on Computer and communications security
		
			ACM
			2003
			
		
	




* 
	
		An implementation of the binding mechanism in the web browser for preventing xss attacks: introducing the bind-value headers
		
			GIha
		
		
			HDoi
		
	
	
		Availability, Reliability and Security, 2009. ARES'09. International Conference on
				
			IEEE
			2009
			
		
	




* 
	
		Dynamic code instrumentation to detect and recover from return address corruption
		
			SGupta
		
		
			PPratap
		
		
			HSaran
		
		
			SArun-Kumar
		
	
	
		A New View on Classification of Software Vulnerability Mitigation Methods 36
				
			ACM
			2006
			
		
	
	Proceedings of the 2006 international workshop on Dynamic systems analysis




* 
	
		Automatic generation of vulnerability specific patches for preventing component hijacking attacks in android applications
		
			MZhang
		
		
			HYin
		
		
			Appsealer
		
	
	
		Proceedings of the 21th Annual Network and Distributed System Security Symposium (NDSS 2014)
				the 21th Annual Network and Distributed System Security Symposium (NDSS 2014)
		
			2014
		
	




* 
	
		Shieldgen: Automatic data patch generation for unknown vulnerabilities with informed probing
		
			WCui
		
		
			MPeinado
		
		
			HJWang
		
		
			MELocasto
		
	
	
		IEEE Symposium on
		
			
			2007. 2007
			IEEE
		
	
	SP'07




* 
	
		Diagnosis and emergency patch generation for integer overflow exploits
		
			TWang
		
		
			CSong
		
		
			WLee
		
	
	
		Detection of Intrusions and Malware, and Vulnerability Assessment
				
			Springer
			2014
			
		
	




* 
	
		Automatic patch generation for buffer overflow attacks, in: Information Assurance and Security
		
			ASmirnov
		
		
			TChiueh
		
	
	
		IAS 2007. Third International Symposium on
				
			IEEE
			2007. 2007
			
		
	




* 
	
		Automatic synthesis of filters to discard buffer overflow attacks: A step towards realizing self-healing systems
		
			ZLiang
		
		
			RSekar
		
		
			DCDuvarney
		
		
	




* 
	
		Automatic diagnosis and response to memory corruption vulnerabilities
		
			JXu
		
		
			PNing
		
		
			CKil
		
		
			YZhai
		
		
			CBookholt
		
	
	
		Proceedings of the 12th ACM conference on Computer and communications security
				the 12th ACM conference on Computer and communications security
		
			ACM
			2005
			
		
	




* 
	
		Static and dynamic analysis: Synergy and duality
		
			MDErnst
		
	
	
		WODA 2003: ICSE Workshop on Dynamic Analysis
				
			2003
			
		
	




* 
	
		
		
			DAWheeler
		
		
			Flawfinder
		
		
		
			2001
		
	
	Online; accessed 2016-10-23




* 
	
		Its4: A static vulnerability scanner for c and c++ code
		
			JViega
		
		
			J.-TBloch
		
		
			YKohno
		
		
			GMcgraw
		
	
	
		Proceedings of the 16th Annual Computer Security Applications Conference, ACSAC'00
				the 16th Annual Computer Security Applications Conference, ACSAC'00
		
			IEEE
			2000
			
		
	




* 
	
		Lint, a C program checker
		
			SCJohnson
		
		
			1977
		
		
			Bell Telephone Laboratories
		
	




* 
	
		Generalized vulnerability extrapolation using abstract syntax trees
		
			FYamaguchi
		
		
			MLottmann
		
		
			KRieck
		
	
	
		Proceedings of the 28 th Annual Computer Security Applications Conference
				the 28 th Annual Computer Security Applications Conference
		
			ACM
			2012
			
		
	




* 
	
		Applying dataflow analysis to detecting software vulnerability
		
			HKim
		
		
			T.-HChoi
		
		
			S.-CJung
		
		
			H.-CKim
		
		
			OLee
		
		
			K.-GDoh
		
	
	
		Proceeding of the 10th International Conference on Advanced Communication Technology. ICACT
				eeding of the 10th International Conference on Advanced Communication Technology. ICACT
		
			IEEE
			2008. 2008
			1
			
		
	




* 
	
		Pixy: A static analysis tool for detecting web application vulnerabilities
		
			NJovanovic
		
		
			CKruegel
		
		
			EKirda
		
	
	
		Proceeding of the 2006 IEEE Symposium on Security and Privacy
				eeding of the 2006 IEEE Symposium on Security and Privacy
		
			IEEE
			2006
			6
		
	




* 
	
		A first step towards automated detection of buffer overrun vulnerabilities
		
			DWagner
		
		
			JSFoster
		
		
			EABrewer
		
		
			AAiken
		
		
			2000
			NDSS
			
		
	




* 
	
		Buffer overrun detection using linear programming and static analysis
		
			,SV Ganapathy
		
		
			DJha
		
		
			DChandler
		
		
			DMelski
		
		
			Vitek
		
	
	
		Proceedings of the 10th ACM conference on Computer and communications security
				the 10th ACM conference on Computer and communications security
		
			ACM
			2003
			
		
	




* 
	
		Detecting memory access errors with flow-sensitive conditional range analysis
		
			YXia
		
		
			JLuo
		
		
			MZhang
		
	
	
		Embedded Software and Systems
				
			Springer
			2005
			
		
	




* 
	
		Symbolic string verification: Combining string analysis and size analysis, in: Tools and Algorithms for the Construction and Analysis of Systems
		
			FYu
		
		
			TBultan
		
		
			OHIbarra
		
		
			2009
			Springer
			
		
	




* 
	
		Improving security using extensible lightweight static analysis, software
		
			DEvans
		
		
			DLarochelle
		
	
	
		IEEE
		
			19
			1
			
			2002
		
	




* 
	
		Meca: an extensible, expressive system and language for statically checking security properties
		
			JYang
		
		
			TKremenek
		
		
			YXie
		
		
			DEngler
		
	
	
		Proceedings of the 10th ACM conference on Computer and communications security
				the 10th ACM conference on Computer and communications security
		
			ACM
			2003
			
		
	




* 
	
		Accelerating the annotation of sparse named entities by dynamic sentence selection
		
			YTsuruoka
		
		
			JTsujii
		
		
			SAnaniadou
		
	
	
		BMC bioinformatics
		
			9
			11
			S8
			2008
		
	
	Suppl




* 
	
		
			MHoward
		
		2016-10-22
		
		A brief introduction to the standard annotation language (SAL)
				
			2006/05/19/602077.aspx. 2006
		
	




* 
	
		acomment: mining annotations from comments and code to detect interrupt related concurrency bugs
		
			LTan
		
		
			YZhou
		
		
			YPadioleau
		
	
	
		Proceedings of the 33rd international conference on software engineering
				the 33rd international conference on software engineering
		
			ACM
			2011
			
		
	




* 
	
		Simplify: a theorem prover for program checking
		
			DDetlefs
		
		
			GNelson
		
		
			JBSaxe
		
	
	
		Journal of the ACM (JACM)
		
			52
			3
			
			2005
		
	




* 
	
		
			GTian-Yang
		
		
			SYin-Sheng
		
		
			FYou-Yuan
		
	
	
		Research on software security testing
				
			2010
			70
			
		
	




* 
	
		Precise buffer overflow detection via model checking
		
			SChaki
		
		
			SHissam
		
		
			2005
		
	




* 
	
		Merging static analysis and model checking for improved security vulnerability detection
	
	
		A New View on Classification of Software Vulnerability Mitigation Methods USENIX Annual Technical Conference
				
			. W.-SR¨odiger
		
		
			2005. 2011
			
		
		
			Dept. of Com. Sc. Augsburg University
		
	
	General Track. Ph.D. thesis, Master thesis




* 
	
		Mops: an infrastructure for examining security properties of software
		
			HChen
		
		
			DWagner
		
	
	
		Proceedings of the 9th ACM conference on Computer and communications security
				the 9th ACM conference on Computer and communications security
		
			ACM
			2002
			
		
	




* 
	
		Modelchecking for software vulnerabilities detection with multi-language support
		
			RHadjidj
		
		
			XYang
		
		
			STlili
		
		
			MDebbabi
		
	
	
		Proceeding of the sixth Annual Conference on Privacy, Security and Trust, PST'08
				eeding of the sixth Annual Conference on Privacy, Security and Trust, PST'08
		
			IEEE
			2008
			
		
	




* 
	
		Efficient algorithms for model checking pushdown systems
		
			JEsparza
		
		
			DHansel
		
		
			PRossmanith
		
		
			SSchwoon
		
	
	
		Computer Aided Verification
				
			Springer
			2000
			
		
	




* 
	
		A method for detecting software vulnerabilities based on clustering and model analyzing
		
			JRen
		
		
			BCai
		
		
			HHe
		
		
			CHu
		
	
	
		Journal of Computational Information Systems
		
			7
			4
			
			2011
		
	




* 
	
		Finding security vulnerabilities in java applications with static analysis
		
			BLivshits
		
		
			MSLam
		
	
	
		Proceedings of the 14th conference on USENIX Security Symposium
				the 14th conference on USENIX Security Symposium
		
			2005
			14
		
	




* 
	
		Static detection of cross-site scripting vulnerabilities
		
			GWassermann
		
		
			ZSu
		
	
	
		Proceeding of the ACM/IEEE 30th International Conference on Software Engineering. ICSE'08
				eeding of the ACM/IEEE 30th International Conference on Software Engineering. ICSE'08
		
			IEEE
			2008
			
		
	




* 
	
		Dynamic taint analysis for automatic detection, analysis, and signature generation of exploits on commodity software
		
			JNewsome
		
		
			DSong
		
	
	
		Proceedings of the 12th Network and Distributed System Security Symposium (NDSS05)
				the 12th Network and Distributed System Security Symposium (NDSS05)
		
			2005
		
	




* 
	
		Fast and precise sanitizer analysis with bek
		
			PHooimeijer
		
		
			BLivshits
		
		
			DMolnar
		
		
			PSaxena
		
		
			MVeanes
		
	
	
		Proceedings of the 20 th USENIX conference on Security, USENIX Association
				the 20 th USENIX conference on Security, USENIX Association
		
			2011
			
		
	




* 
	
		Automated detection of code vulnerabilities based on program analysis and model checking
		
			LWang
		
		
			QZhang
		
		
			PZhao
		
	
	
		Proceeding of the Eighth IEEE International Working Conference on Source Code Analysis and Manipulation
				eeding of the Eighth IEEE International Working Conference on Source Code Analysis and Manipulation
		
			IEEE
			2008
			
		
	




* 
	
		Toward black-box detection of logic flaws in web applications
		
			GPellegrino
		
		
			DBalzarotti
		
	
	
		Proceedings of the Network and Distributed System Security (NDSS) Symposium
				the Network and Distributed System Security (NDSS) Symposium
		
			2014
		
	




* 
	
		Secubat: a web vulnerability scanner
		
			SKals
		
		
			EKirda
		
		
			CKruegel
		
		
			NJovanovic
		
	
	
		Proceedings of the 15th international conference on World Wide Web
				the 15th international conference on World Wide Web
		
			ACM
			2006
			
		
	




* 
	
		
			JDA Takanen
		
		
			CDemott
		
		
			Miller
		
		Fuzzing for software security testing and quality assurance
				
			Artech House
			2008
		
	




* 
	
		Simfuzz: Test case similarity directed deep fuzzing
		
			DZhang
		
		
			DLiu
		
		
			YLei
		
		
			DKung
		
		
			CCsallner
		
		
			NNystrom
		
		
			WWang
		
	
	
		Journal of Systems and Software
		
			85
			1
			
			2012
		
	




* 
	
		Taintscope: A checksum-aware directed fuzzing tool for automatic software vulnerability detection
		
			TWang
		
		
			TWei
		
		
			GGu
		
		
			WZou
		
	
	
		Proceeding of 2010 IEEE Symposium on Security and Privacy (SP)
				eeding of 2010 IEEE Symposium on Security and Privacy (SP)
		
			IEEE
			2010
			
		
	




* 
	
		Enemy of the state: A state-aware black-box web vulnerability scanner
		
			LDoup´e
		
		
			CCavedon
		
		
			GKruegel
		
		
			Vigna
		
	
	
		USENIX Security Symposium
				
			2012
			
		
	




* 
	
		Klee: Unassisted and automatic generation of high-coverage tests for complex systems programs
		
			DCadar
		
		
			DRDunbar
		
		
			Engler
		
		
			2008
			OSDI
			8
			
		
	




* 
	
		Automated whitebox fuzz testing
		
			PGodefroid
		
		
			MYLevin
		
		
			DAMolnar
		
	
	
		NDSS
		
			8
			
			2008
		
	




* 
	
		Exe: automatically generating inputs of death
		
			CCadar
		
		
			VGanesh
		
		
			PMPawlowski
		
		
			DLDill
		
		
			DREngler
		
	
	
		ACM Transactions on Information and System Security
		
			12
			2
			10
			2008
		
	
	TISSEC)




* 
	
		Dowsing for overflows: A guided fuzzer to find buffer boundary violations
		
			AHaller
		
		
			MSlowinska
		
		
			HNeugschwandtner
		
		
			Bos
		
	
	
		Usenix Security
				
			2013
			
		
	




* 
	
		Vulnerability detection syst ems:Think cyborg, not robot
		
			SHeelan
		
	
	
		IEEE Security and Privacy
		
			9
		
	




* 
	
		Automated vulnerability analysis: Leveraging control flow for evolutionary input crafting
		
			SSparks
		
		
			SEmbleton
		
		
			RCunningham
		
		
			CZou
		
	
	
		Proceeding of Twenty-Third Annual Computer Security Applications Conference. ACSAC
				eeding of Twenty-Third Annual Computer Security Applications Conference. ACSAC
		
			IEEE
			2007. 2007
			
		
	




* 
	
		Revolutionizing the field of greybox attack surface testing with evolutionary fuzzing
		
			JDemott
		
		
			REnbody
		
		
			WFPunch
		
		
	
	BlackHat and Defcon




* 
	
		Music: Mutation-based sql injection vulnerability checking
		
			HShahriar
		
		
			MZulkernine
		
	
	
		Proceeding of the Eighth International Conference on Quality Software. QSIC'08
				eeding of the Eighth International Conference on Quality Software. QSIC'08
		
			IEEE
			2008
			
		
	




* 
	
		Extending model checking with dynamic analysis
		
			RGroce
		
		
			Joshi
		
	
	
		Verification, Model Checking, and Abstract Interpretation
				
			Springer
			2008
			
		
	




* 
	
		Software model checking
		
			RJhala
		
		
			RMajumdar
		
	
	
		ACM Computing Surveys (CSUR)
		
			41
			4
			21
			2009
		
	




* 
	
		Model checking for programming languages using verisoft
		
			PGodefroid
		
	
	
		Proceedings of the 24th ACM SIGPLAN-SIGACT symposium on Principles of programming languages
				the 24th ACM SIGPLAN-SIGACT symposium on Principles of programming languages
		
			ACM
			1997
			
		
	




* 
	
		Model checking java programs using java pathfinder
		
			KHavelund
		
		
			TPressburger
		
	
	
		International A New View on Classification of Software Vulnerability Mitigation Methods Journal on Software Tools for Technology Transfer
				
			2000
			2
			
		
	




* 
	
		Cmc: A pragmatic approach to model checking real code
		
			MMusuvathi
		
		
			DYPark
		
		
			AChou
		
		
			DREngler
		
		
			DLDill
		
	
	
		ACM SIGOPS Operating Systems Review
		
			36
			SI
			
			2002
		
	




* 
	
		Bogor: an extensible and highly-modular software model checking framework
		
			MBDwyer
		
		
			JHatcliff
		
	
	
		ACM SIGSOFT Software Engineering Notes
		
			28
			
			2003
			ACM
		
	




* 
	
		Dart: directed automated random testing
		
			PGodefroid
		
		
			NKlarlund
		
		
			KSen
		
	
	
		ACM Sigplan Notices
		
			40
			
			2005
			ACM
		
	




* 
	
		All you ever wanted to know about dynamic taint analysis and forward symbolic execution (but might have been afraid to ask
		
			EJSchwartz
		
		
			TAvgerinos
		
		
			DBrumley
		
	
	
		Proceeding of the 2010 IEEE Symposium on Security and Privacy (SP)
				eeding of the 2010 IEEE Symposium on Security and Privacy (SP)
		
			IEEE
			2010
			
		
	




* 
	
		25 million flows later: large-scale detection of dom-based xss
		
			SLekies
		
		
			BStock
		
		
			MJohns
		
	
	
		Proceedings of the 2013 ACM SIGSAC conference on Computer & communications security
				the 2013 ACM SIGSAC conference on Computer & communications security
		
			ACM
			2013
			
		
	




* 
	
		Dta++: Dynamictaint analysis with targeted controlflow propagation
		
			MGKang
		
		
			SMccamant
		
		
			PPoosankam
		
		
			DSong
		
		
			2011
			NDSS
		
	




* 
	
		Saner: Composing static and dynamic analysis to validate sanitization in web applications
		
			DBalzarotti
		
		
			MCova
		
		
			VFelmetsger
		
		
			NJovanovic
		
		
			EKirda
		
		
			CKruegel
		
		
			GVigna
		
	
	
		Proceeding of IEEE Symposium on Security and Privacy
				eeding of IEEE Symposium on Security and Privacy
		
			IEEE
			2008. 2008
			
		
	




* 
	
		On the effectiveness of dynamic taint analysis for protecting against private information leaks on android-based devices
		
			GSarwar
		
		
			OMehani
		
		
			RBoreli
		
		
			DKaafar
		
	
	
		Proceeding of the 10th International Conference on Security and Cryptography (SECRYPT)
				eeding of the 10th International Conference on Security and Cryptography (SECRYPT)
		
			2013
		
	




* 
	
		A program testing system
		
			LAClarke
		
	
	
		Proceedings of the 1976 annual conference
				the 1976 annual conference
		
			ACM
			1976
			
		
	




* 
	
		Fie on firmware: Finding vulnerabilities in embedded systems using symbolic execution
		
			DDavidson
		
		
			BMoench
		
		
			TRistenpart
		
		
			SJha
		
	
	
		USENIX Security
				
			2013
			
		
	




* 
	
		A decision procedure for bitvectors and arrays
		
			DLV Ganesh
		
		
			Dill
		
	
	
		Computer Aided Verification
				
			Springer
			2007
			
		
	




* 
	
		Z3: An efficient smt solver, in: Tools and Algorithms for the Construction and Analysis of Systems
		
			LDe Moura
		
		
			NBjørner
		
		
			2008
			Springer
			
		
	




* 
	
		Hampi: a solver for string constraints
		
			VKiezun
		
		
			PJGanesh
		
		
			PGuo
		
		
			MDHooimeijer
		
		
			Ernst
		
	
	
		Proceedings of the eighteenth international symposium on Software testing and analysis
				the eighteenth international symposium on Software testing and analysis
		
			ACM
			2009
			
		
	




* 
	
		S3: A symbolic string solver for vulnerability detection in web applications
		
			M.-TTrinh
		
		
			D.-HChu
		
		
			JJaffar
		
	
	
		Proceedings of the 2014 ACM SIGSAC Conference on Computer and Communications Security
				the 2014 ACM SIGSAC Conference on Computer and Communications Security
		
			ACM
			2014
			
		
	




* 
	
		Symbolic execution for software testing: three decades later
		
			CCadar
		
		
			KSen
		
	
	
		Communications of the ACM
		
			56
			2
			
			2013
		
	




* 
	
		Linear obfuscation to combat symbolic execution
		
			ZWang
		
		
			JMing
		
		
			CJia
		
		
			DGao
		
	
	
		Computer Security-ESORICS 2011
				
			Springer
			2011
			
		
	




* 
	
		Dynamic test generation to find integer bugs in x86 binary linux programs
		
			KSen
		
		
			DMarinov
		
		
			GAgha
		
		
			; DMolnar
		
		
			XCLi
		
		
			DAWagner
		
	
	
		Proceedings of the 18 th conference on USENIX security symposium
				the 18 th conference on USENIX security symposium
		
			ACM
			2005. 2009
			30
			
		
	
	CUTE: a concolic unit testing engine for C




* 
	
		Model-inference-assisted concolic exploration for protocol and vulnerability discovery
		
			CYCho
		
		
			DBabic
		
		
			PPoosankam
		
		
			KZChen
		
		
			EXWu
		
		
			DSong
		
		
			Mace
		
	
	
		USENIX Security Symposium
				
			2011
			
		
	




* 
	
		A hybrid analysis framework for detecting web application vulnerabilities
		
			MMonga
		
		
			RPaleari
		
		
			EPasserini
		
	
	
		Proceedings of the 2009 ICSE Workshop on Software Engineering for Secure Systems
				the 2009 ICSE Workshop on Software Engineering for Secure Systems
		
			IEEE Computer Society
			2009
			
		
	




* 
	
		Combining static analysis and runtime monitoring to counter sql-injection attacks
		
			WGHalfond
		
		
			AOrso
		
	
	
		ACM SIGSOFT Software Engineering Notes
		
			30
			
			2005
			ACM
		
	




* 
	
		Toward automated detection of logic vulnerabilities in web applications
		
			LFelmetsger
		
		
			CCavedon
		
		
			GKruegel
		
		
			Vigna
		
	
	
		USENIX Security Symposium
				
			2010
			
		
	




* 
	
		
			S.-WWoo
		
		
			HJoh
		
		
			OHAlhazmi
		
		
			YKMalaiya
		
	
	
		Modeling vulnerability discovery process in apache and iis http servers
				
			2011
			30
			
		
	




* 
	
		
			OHAlhazmi
		
		
			YKMalaiya
		
		Proceedings of annual reliability and maintainability symposium
				annual reliability and maintainability symposium
		
			2005
			
		
	
	Quantitative vulnerability assessment of systems software




* 
	
		
			OAlhazmi
		
		
			YMalaiya
		
		
			IRay
		
		Security vulnerabilities in software systems: A quantitative perspective
				
			Springer
			2005
			
		
	
	Data and Applications Security XIX




* 
	
		Vulnerability scrying method for software vulnerability discovery prediction without a vulnerability database, Reliability
		
			SRahimi
		
		
			MZargham
		
	
	
		IEEE Transactions on
		
			62
			2
			
			2013
		
	




* 
	
		Application of vulnerability discovery models to major operating A New View on Classification of Software Vulnerability Mitigation Methods systems, Reliability
		
			OHAlhazmi
		
		
			YKMalaiya
		
	
	
		IEEE Transactions on
		
			57
			1
			
			2008
		
	




* 
	
		Seasonal variation in the vulnerability discovery process
		
			HJoh
		
		
			YKMalaiya
		
	
	
		Proceedings of ICST'09 International Conference on Software Testing Verification and Validation
				ICST'09 International Conference on Software Testing Verification and Validation
		
			IEEE
			2009. 2009
			
		
	




* 
	
		
		Application Security Verification Standard (ASVS)
				
	
	Online; accessed 2016-10-8




* 
	
		It's the psychology stupid: how heuristics explain software vulnerabilities and how priming can illuminate developer's blind spots
		
			DOliveira
		
		
			MRosenthal
		
		
			NMorin
		
		
			K.-CYeh
		
		
			JCappos
		
		
			YZhuang
		
	
	
		Proceedings of the 30th Annual Computer Security Applications Conference
				the 30th Annual Computer Security Applications Conference
		
			ACM
			2014
			
		
	




* 
	
		Using replicated execution for a more secure and reliable web browser
		
			HXue
		
		
			NDautenhahn
		
		
			STKing
		
		
			2012
			NDSS
		
	




* 
	
		Eternal war in memory
		
			LSzekeres
		
		
			MPayer
		
		
			TWei
		
		
			DSong
		
	
	
		IEEE Symposium on Security and Privacy
				
			2013
		
	




* 
	
		Multi-module vulnerability analysis of web-based applications
		
			DBalzarotti
		
		
			MCova
		
		
			VVFelmetsger
		
		
			GVigna
		
	
	
		Proceedings of the 14th ACM conference on Computer and communications security
				the 14th ACM conference on Computer and communications security
		
			ACM
			2007
			
		
	




* 
	
		State of the art: Automated black-box web application vulnerability testing
		
			JBau
		
		
			EBursztein
		
		
			DGupta
		
		
			JMitchell
		
	
	
		Proceeding of the 2010 IEEE Symposium on Security and Privacy (SP)
				eeding of the 2010 IEEE Symposium on Security and Privacy (SP)
		
			IEEE
			2010
			
		
	




* 
	
		Supporting automated vulnerability analysis using formalized vulnerability signatures
		
			MAlmorsy
		
		
			JGrundy
		
		
			ASIbrahim
		
	
	
		Proceedings of the 27th IEEE/ACM International Conference on Automated Software Engineering
				the 27th IEEE/ACM International Conference on Automated Software Engineering
		
			ACM
			2012
			
		
	




* 
	
		Modeling and discovering vulnerabilities with code property graphs
		
			FYamaguchi
		
		
			NGolde
		
		
			DArp
		
		
			KRieck
		
	
	
		Proceedings of 2014 IEEE Symposium on Security and Privacy (SP)
				2014 IEEE Symposium on Security and Privacy (SP)
		
			IEEE
			2014
			
		
	




* 
	
		Vdc-based dynamic code analysis: Application to c programs
		
			WMallouli
		
		
			AMammar
		
		
			ACavalli
		
		
			WJimenez
		
	
	
		Journal of Internet Services and Information Security
		
			1
			2/3
			
			2011
		
	




* 
	
		Improving software security with precise static and runtime analysis
		
			BLivshits
		
		
			2006
		
		
			Stanford University
		
	
	Ph.D. thesis




* 
	
		Detecting format string vulnerabilities with type qualifiers
		
			UShankar
		
		
			KTalwar
		
		
			JSFoster
		
		
			DWagner
		
	
	
		USENIX Security Symposium
				
			2001
			
		
	




* 
	
		Proving properties of security protocols by induction
		
			LCPaulson
		
	
	
		Proceedings of the 10th workshop on Computer Security Foundations
				the 10th workshop on Computer Security Foundations
		
			IEEE
			1997
			
		
	




* 
	
		A logic of authentication
		
			MBurrows
		
		
			MAbadi
		
		
			RMNeedham
		
	
	
		Proceedings of the Royal Society of London A: Mathematical, Physical and Engineering Sciences
		
			426
			
			1989
			The Royal Society
		
	




* 
	
		
			TNipkow
		
		
			LCPaulson
		
		
			MWenzel
		
		Isabelle/HOL: a proof assistant for higher-order logic
				
			Springer Science & Business Media
			2002
			2283
		
	




* 
	
		Statically detecting likely buffer overflow vulnerabilities
		
			DLarochelle
		
		
			DEvans
		
	
	
		USENIX Security Symposium
				
			.C ELandwehr
			ARBull
			JPMcdermott
			WSChoi
		
		Washington DC
		
			2001. 1994
			32
			
		
	
	A taxonomy of computer program security flaws




* 
	
		Security analysis and enhancements of computer operating systems
		
			RPAbbott
		
		
			JSChin
		
		
			JEDonnelley
		
		
			WLKonigsford
		
		
			STokubo
		
		
			DAWebb
		
		NBSIR-76- 1041
	
	
		National bureau of standards Washington inst for computer sciences and technology
				
			1976
		
	
	Technical report




* 
	
		
			RBisbey
		
		
			DHollingworth
		
		Protection analysis: Final report
				
			1978
		
	




* 
	
		
			PAnderson
		
	
	
		Codesurfer/path inspector, Proceeding of the 20th IEEE International Conference on Software Maintenance
				
			2004. 2004
		
	




* 
	
		Software verification with blast
		
			TAHenzinger
		
		
			RJhala
		
		
			RMajumdar
		
		
			GSutre
		
	
	
		Model Checking Software
				
			Springer
			2003
			
		
	




* 
	
		
			SRawat
		
		
			DCeara
		
		
			LMounier
		
		
			M.-LPotet
		
		arXiv:1305.3883
		Combining static and dynamic analysis for vulnerability detection
				
	
	arXiv preprint




* 
	
		Security in open versus closed system the dance of boltzmann, coase and moore
		
			RAnderson
		
	
	
		Open Source Software Economics
		
	




* 
	
		Is finding security holes a good idea?
		
			ERescorla
		
	
	
		Security & Privacy
				
			2005
			3
			
		
	




* 
	
		A logarithmic poisson execution time model for software reliability measurement
		
			JDMusa
		
		
			KOkumoto
		
	
	
		Proceedings of the 7 th i nternational conference on Software engineering
				the 7 th i nternational conference on Software engineering
		
			IEEE Press
			1984
			
		
	




* 
	
		Simulation of built-in php features for precise static code analysis
		
			JDahse
		
		
			THolz
		
	
	
		Symposium on Network and Distributed System Security (NDSS)
				
			2014