Applications for Tarantool. Part 3. Testing and launching

The application for Tarantool is, in essence, a set of stored procedures used as an API. The data is processed on the storage side, which can significantly improve performance. However, support for stored procedures can turn into a nightmare.
 
Can. But not today.
 
Today we will consider the issues of ensuring the quality of the application. In particular, let's talk about testing, understand how to launch in production, how to use connectors, and also talk about the intricacies of migration of the data schema.
 
Applications for Tarantool. Part 3. Testing and launching Part 1. Stored procedures
 
- Part 2. OAuth2-authorization
 
- Part 3. Testing and running
 
Let me remind you that all the examples in this series of articles are based on the application for authorizing users tarantool-authman .
 
Testing
 
Testing is only one way to improve quality.
 
In the case of stored procedures, functional tests are a good way to support the current API. For testing, the built-in Tarantool is used. module tap (Test Anything Protocol) . It includes several methods for basic checks and comparison of Lua objects. Let's test two user registration cases auth.registration (email) .
 
    local case = {}
local tap = require ('tap')
local response = require ('authman.response')
local error = require ('authman.error')
local db = require ('authman.db')
local config = require ('test.config')
- Initialize the application for testing
local auth = require ('authman'). api (config)
local test = tap.test ('registration_test')
- Clean the space before each start of the test
function case.before () db.truncate_spaces () end
function case.after () end
function test_registration_succes ()
local ok, code
ok, code = auth.registration ('[email protected] ')
- Let's check that the registration was successful, and the type of the returned parameter is line
test: is (ok, true, 'test_registration_succes user created')
test: isstring (code, 'test_registration_succes code returned')
end
function test_registration_user_already_exists ()
local ok, code, got, expected, user
ok, code = auth.registration ('[email protected] ')
- One of the API methods that activates the user with the password
ok, user = auth.complete_registration ('[email protected] ', code,' password ')
- Let's check that the value corresponds to the expected error
- The method returns two parameters, so to simplify the test, the call is wrapped in the initialization of the Lua table
got = {auth.registration ('[email protected] '),}
expected = {response.error (error.USER_ALREADY_EXISTS),}
- is_deeply checks the match of keys and data in Lua tables
test: is_deeply (got, expected, 'test_registration_user_already_active')
end
case.tests = {
test_registration_succes,
test_registration_user_already_exists
}
return case

 

Now write a small script to run the tests. Do not forget to take into account the methods before and after , which will simplify testing.


 
    - First of all, you need to run Tarantool
box.cfg {
listen = 333?
}
local TEST_CASES = {
'test.case.registration',
}
function run ()
for case_index = ? #TEST_CASES do
local case = require (TEST_CASES[case_index])
for test_index = ? # case.tests do
- Calling a particular test
case.before ()
case.tests[test_index]()
case.after ()
end
end
end
run ()

 

The result of running the tests:


 
    $ tarantool test /authman.test.lua
TAP version 13
ok - test_registration_succes user created
ok - test_registration_succes code returned
ok - test_registration_user_already_active

 

Starting Instance


 

The application is written and tested, which means it's time to deploy it in the production environment.


 

To manage instances of Tarantool, use built-in utility tarantoolctl . First, create an instance with the application. Do not forget to add the user and give him the rights.


 
    box.cfg {
listen = 3331;
}
local function bootstrap ()
box.schema.user.create ('my_user', {password = '123'})
box.schema.user.grant ('my_user', 'read, write, execute', 'universe')
end
box.once ('init_user', bootstrap)
config = {
- Application configurations
}
- Declare the variable auth globally
auth = require ('auth'). api (config)

 

Now create a symbolic link to this file from the instances.enabled directory:


 
    sudo ln -s /etc/tarantool/instances.available/auth.lua /etc/tarantool/instances.enabled/auth.lua    

 

Run tarantoolctl and check that we can connect to Tarantool and use the application methods:


 
    $ tarantoolctl start auth
$ tarantoolctl enter auth
connected to unix /: /var /run /tarantool /auth.control
unix /: /var /run /tarantool /auth.control> ok, user = auth.registration ('[email protected] ')
unix /: /var /run /tarantool /auth.control> ok, user = auth.complete_registration ('[email protected] ', user.code,' 123 ')
unix /: /var /run /tarantool /auth.control> user
---
- is_active: true
email: [email protected]
id: 8cd27d26-3974-43d6-a2b2-87202664753d

 

If something went wrong, you should look at the logs /var/log/tarantool/auth.lua . For more information on administering the Tarantool server, read in the documentation .


 

Connector


 

One of the advantages of stored procedures is the provision of a common interface for services written in different programming languages.


 


 

Consider an example of using Tarantool asynchronously using Python connector .


 
    import asyncio
import asynctnt
async def create_user ():
tnt_connection = asynctnt.Connection (
host = '???.1', port = '3367', username = 'my_user', password = '123'
)
# Asynchronous user registration
user_data = await tnt_connection.call (
'auth.registrtion',['[email protected]', ].
)
await tnt_connection.disconnect ()
# Call the user creation
loop = asyncio.get_event_loop ()
loop.run_until_complete (create_user ())

 

It is worth noting that you can also interact with Tarantool from another Tarantool instance using the built-in module. net.box . Read more about it in the documentation .


 

Data Migration


 

A common problem when an application is already running in combat is changing the data schema. For example, the user needs to add a field. gender .


 


 

To do this, create a script that modifies all data in space auth_user . The script itself can be added to the initialization call of the application, and the function box.once will limit the repeated call.


 
    local fiber = require ('fiber')
local function migrations ()
- Perform the migration only once
box.once ('20171101_add_gender', function ()
local counter = 0
- Bypassing the space with the iterator will reduce the use of RAM
- Only the current object will be in memory
.for _, tuple in box.space .auth_user: pairs (
nil, {iterator = box.index.ALL}
) do
local user_tuple = tuple: totable ()
- Write new data into the table
user_tuple[4]= get_user_gender ()
Box.space.auth_user: replace (user_tuple)
.
Counter = counter + 1
.end
End)
end
migrations ()

 

Although migration can take a long time, the main thread of execution is not blocked. This is due to the fact that the method replace implicitly calls yield . In order to explicitly release the execution thread, you need to call the method fiber.sleep (0) . Read more about this in the documentation .
 
What's next?
 
Tarantool does not stand still and is actively developing.
 
Beyond this training series of articles, there are several important issues that you should pay attention to when writing your own services:
 
 
Charding and replication. Use of module vshard .
 
Support for SQL syntax in Tarantool. Optimizing stored procedures using SQL.
 
Overview of the disk engine Vinyl. Comparison with Memtx, or "What should I do if the RAM is out?".
 
Optimization and profiling of stored procedures written in C.
 
 
I remind you that all code samples are taken from application tarantool-authman . It also develops, and can now be used as Oauth2-server.
 
If you have any questions beyond the scope of the articles, you can ask them in telegram-chat . Responsive community Tarantool, most likely, will be able to help you.
 
Thanks to all who have waited for the third part.
 
Until next time!
 
The Tarantool community conducts the first conference on in-memory computing June 21 in Moscow. The reports are still accepted, the call for papers is open.

+ 0 -

Add comment