Alert Lida: Security Testing Automation
Good afternoon, dear readers. My name is Victor Burov, I am a developer in the ISPsystem. In the last post I talked about tool for creating autotests , today I will share the experience of automating security testing.
At first, an individual employee was looking for vulnerabilities in our products. Manual testing took a lot of time and did not guarantee that all vulnerabilities would be found. Having found out the basic laws of testing, we came to the conclusion that it can be automated. Then we decided to write a utility that will facilitate the life of the tester, save his time and allow you to check the products after each change. Since the tester was called Lida, we called the new application in her honor. In general, in our company it has become a tradition to call testing tools by the names of testers.
After analyzing the vulnerability search utilities, I came to the conclusion that they all need to specify the functions to call and the parameters used. We again decided to take advantage of the unified interface and formulated the requirements for Lida.
Requirements at the start:
Automatic building lists of functions. 3r350.
Autocomplete parameters. 3r350.
Making requests to the API. 3r350.
Analysis of data output after performing functions. 3r350.
Search for data vulnerabilities. 3r350.
Formation of reports. 3r350.
Flexible settings. 3r350.
Implementing all this was not easy.
3r3163. Bypassing forms and lists
To find vulnerabilities in a function, it must be executed by passing the necessary parameters. Our interface is built on the basis of lists and forms, so it is possible to automate data collection by processing xml documents describing the structure of interface elements.
I decided to start a crawl from the main menu, recursively going into all nested lists. "Lida" opens the list of the first level. As a rule, it has several buttons that call some functions.
If the button opens a form, then the result of the call will be an xml-document with nodes containing information about the fields: name, validator, range of acceptable values. Based on this information, field values are generated. For example, an int will generate a number. If there is no validator, a random string is generated. After filling in all the form fields, a request is sent.
If the function is a list, it will be opened and functions associated with the buttons will be called for its elements.
When checking lists there is a problem - all lists must have a set of entries that will ensure that all buttons in the list are clickable.
3r3163. Search for SQL injection
SQL injection is probably one of the most common problems for applications, including ours. Many function calls spawn various queries to the DBMS. An error occurs when parameters that come from outside are inserted into the request body "as is". The consequences of such errors can be sad: from unauthorized data acquisition to the removal of tables.
To start searching for SQL injections, I organized the output of all SQL queries to a file. After executing the function, the application searches for the values of the passed parameters in the resulting SQL queries.
It was possible to use the log of the SQL server itself. But in our case there is only one method for executing requests and adding logging to it was easy. Because of this, we know exactly what call originated a particular request. 3r33232.
When the value of the passed parameter is found, the utility passes in this parameter a value containing a single quote. If the quotation mark is found in the same sequence, it means that the parameter is not escaped - we have found a place for SQL injections.
3r3163. Analysis of system calls
A similar problem occurs when we make any system calls. I decided to look for them with strace and selected the optimal launch parameters for it. Example of launching ISPmanager:
3r3117. 3r3118. strace -f -s 1024 -e trace = file, process bin /core ispmgr
As with SQL injections, the “Lida” performs the function and analyzes the output of strace in order to fall into the function 3r-3231. open, unlink, rmdir, chmod chown, chflags, mknod, mkfifo, fcntl, symlink, link, execve, mkdir the values of the passed parameters.
If the parameter is found, the utility passes in it a value containing, for example, the path with the transition to the directory above. If it hits as-is, then a potential vulnerability is found.
The analysis of the execve function was very useful. It allows you to determine in which function the launch arguments of executable files are not screened. This error is very expensive, because through it you can get root access to the server by simply changing the password.
When users find a vulnerability in our products, the company pays a cash reward, the amount of which depends on the category of vulnerability. This approach may be cheaper than searching for errors on its own (the tester may not find an error, but will receive a salary). 3r3138.
Another interesting test: checking the order of calling the stat functions and others. Often, access is first checked through stat, and then any unsafe actions, which leaves room for substitution. But we did not automate this. 3r33232.
3r3163. Check access to other objects 3r3r164.
We check access to foreign objects from under the user for entities of another user and administrator. In this mode, we check the ability to read, modify and view lists of items of another user.
"Lida" bypasses all functions available on behalf of the owner or administrator and remembers their elements. Then it calls the same functions with elements from under another user. In an ideal situation, the answer should be an error like Access or Missed. If this error is not received, therefore, with high probability you can read the data of another user.
In some cases, the absence of an error does not mean that you can get direct access to the objects of another user. At the beginning of such functions, we add a check for access rights to the item. This is how not only security is checked, but also the correctness of server responses.
3r3163. Validation API
Validating our APIs is an added bonus of function checking for vulnerabilities. Analyzing reports on Lida's work, we learned how to return the correct types of errors, made our API more convenient and logical. As a result, as with the tape recorder, we received not only a security check, but once again checked our API for "consistency."
The utility can work with all our products, therefore, it checks many different functions. Mostly false positives appeared in the mode of checking access to other objects. This is due to the peculiarities of the buttons and functions.
False alarms were Lida’s biggest problem. It seemed that it was completely debugged, but when checking on different panels all new false positives appeared. As a result, there were several stages of their correction.
Creating entities 3r33228.
The main part of the action in the panel is performed on any entity (domain name, database, etc.). To achieve maximum automation, Lida had to create these entities automatically. But in practice it turned out to be difficult to implement. Sometimes validation is performed in code, so it is not always possible to substitute the value of the parameter automatically. The second reason is dependent entities. For example, to create a mailbox you need to create a mail domain.
Therefore, we decided not to fight for full automation. Before starting, the tester creates the entities manually and takes a snapshot of the machine, since after checking the entities will be changed. This allows you not to skip checking the group of functions in the case of an unsuccessful creation of an entity.
Call of destructive functions
Almost every list has functions to delete or turn off an entity. If you perform them in the order, then the entity will be deleted before performing other functions. I define such functions and perform after others. Additionally added a key that prohibits the execution of such functions.
Some functions reboot the server. They have to track and add to the list of ignored.
Due to the peculiarities of the operation logic, some functions restart the panel. During security testing, this leads to the fact that the panel is launched without a trace of SQL queries or strace - further verification becomes meaningless. It is necessary to monitor and re-launch the panel in trace mode.
Check dependent parameters
On forms there are text entry fields, checkboxes, drop-down lists, the values of which determine the availability of the values of other fields. For each value of the dependent field there may be a separate section of the code, therefore, parts when they can remain untested.
To solve this problem, I added algorithms for analyzing dependent fields and checking all combinations of dependent controls.
Check for functions not available in theinterface.
Service functions are not available for migration, but may contain vulnerabilities. To identify and verify them, we have a special function that returns a list of all registered functions. We compare this list with the list of proven functions. There are no metadata for service functions, therefore it is impossible to check the parameters processed within them. In order to check such functions at least somehow, I pass our standard parameters 3r33231 to them. elid, plid other.
We included Lida in every Jenkins nightly build. According to the results of her work, a verification report is generated, information about the suspicious function is displayed in it with all parameters.
The task closed by the developer is now checked not only by the tester, but also by Lida. The tester processes the report received: copies the parameters of the suspicious function to the browser and analyzes the behavior and log of the panel. If the vulnerability is confirmed, log the error to the developer.
It may be interesting