Greedy Dwarf: As I wrote market analytics in Lineage 2

Here comes the autumn-winter season. Outside the window, the rains and the desire to spend time outdoors are less and less. And here comes a message to me from a friend "
And let's play in 3r3135. Lineage 2 3r3r136. 3r?108.?". And again, I succumbed to nostalgia, agreed. We chose a fresh server on our site and created characters.
 3r33333. Greedy Dwarf: As I wrote market analytics in Lineage 2  3r33333.
 3r33333. Unlike World of Warcraft in Lineage 2 is a completely different game currency mining system. Need day and night to hunt monsters in order to obtain profit. For me it was even a discovery that for some people RMT (Real money trading) is something of a job. Also, the game has an economy that players form. In other words, you can make money on buy-sell or buy cheap resources, from them to do things and sell at a premium. Since for us the game remains something like a rest, this is the way to get game currency was chosen by us.
 3r33333.
To buy and sell items the player must be online (screenshot above). Accordingly, someone wants to quickly sell (cheaper) and someone quickly buy (more expensive). And what if the difference to sell is to buy a positive one? Just this example will be considered in the article as a result.
 3r33333.
However, prices in the market are quite unstable and often change. By this, there is a chance to buy something "cheap" and then sell it even cheaper and have a negative profit. This is what we are trying to avoid. In general, it was decided to write
market analytics system 3r3108. and deal with a couple of interesting technologies to me.
 3r33333.
Spoiler 3r3136. :
 3r33333. The article will use the following technologies 3r3322.  3r33333. Docker, DigitalOcean, NodeJs, Ktor, Prometheus, Grafana, Telegram bot notification
Ktor - server solution on Kotlin.
 3r33333. I did not begin to use the database, and decided to store the latest data in Singleton. Yes, the decision is not the most elegant, but it’s a quick fix, and we’ll always have time to optimize.
 3r33333.
So the second module of the system appeared -
Harvester
.
 3r33333. Harvester provides the user with two endpoint'a /item /{id} and /metrics
 3r33333. If everything is clear with the first, the latter returns the data in the format for the next system - 3r3135. Prometheus
 3r33333. 3r33140.
 3r33333. 3r3143.
 3r33333.

Data storage for analytics


 3r33333.
Intermediate link was selected Prometheus - Open Source database for analytics which works through a pull approach. In other words, during the configuration, you need to specify in the yaml file the set of data providers and the polling frequency. In our case, this is the same /metrics endpoint.
 3r33333.
We try to run Prometheus (the default is port 9090) and if we see in Target something similar to:
 3r33333.
 3r33333. 3r3162.
 3r33333.
 3r33333. So we are going the right way. This means that every 30 seconds Prometheus goes to Harvester and picks up the latest status on all products of interest to us.
 3r33333.

Displaying data


 3r33333.
The next stage is a beautiful display of graphs
 3r33333.
was chosen for drawing. Grafana which is also open source.
 3r33333. Pros Grafana and Prometheus - they are available in the form of Docker containers that require a minimum of user actions.
 3r33333.
When you first start Grafana (standard port 3000), it will ask you to specify an available database. Choose Prometheus and prescribe the address. If everything goes well, we will see:
 3r33333.
 3r33333. 3r3193.
 3r33333.
 3r33333. This means it's time to start drawing graphs.
 3r33333.
Sample request for drawing sales graphics:
 3r33333.
 3r33333. 3r3208.
 3r33333.
 3r33333. Thus, at any moment we see the average price for buying and selling, as well as price dynamics.
 3r33333.
 3r33333.  3r33333.
 3r33333. However, there are times when the minimum sale price is higher than the maximum purchase price. This means that we can get easy money in the form of "buy sell." Telegram was selected for the notification channel. Create a bot and add its token to Grafana (yes, yes, it supports notifications) 3r3322.  3r33333.
 3r33333.  3r33333.
 3r33333. Simply set the condition under which this notification will come to us and just wait
 3r33333.
 3r33333.  3r33333.
 3r33333. As we see from the schedule, such situations occur in the market.
 3r33333.
 3r33333.  3r33333.
 3r33333.

Cloud


 3r33333.
We pack each subsystem into a Docker container and load it into DigitalOcean or other services of your choice. But this does not prevent us from running the whole system without a dedicated IP. Now the minimum container for DO costs $ 5 per month.
 3r33333.
We start at first Scrapper
 3r33333. docker run -d -p 6661: 6661 --name scrapper l2 /scrapper: latest
 3r33333.
Behind him is the Harvester
 3r33333. docker run -d -p 6662: 6662 -v /root /harvester: /res --link scrapper: scrapper l2harvester: latest
 3r33333. The folder /harvester should contain an ids.txt file with the format
 3r33333.
id1 name1
 3r33333. id2 name2
 3r33333.

Conclusion


 3r33333.
Ultimately, the system looks like this:
 3r33333.  3r33333.
It is planned in the future to add an agent to update google docs, and count the cost of crafting on the fly.
 3r33333.
I do not know whether this development will bring any benefits, but for me personally it was a good experience to refresh my knowledge in the applied field. My main specificity is mobile apps. Development of the server part is an additional skill and curiosity.
 3r33333.
As an addition, I attach links to get familiar with the code:
 3r33333. 3r3309. Scrapper
 3r33333. 3r33333. Harvester
 3r33333. (Containers can be assembled with the commands
 3r33333. docker build -t scrapper.
 3r33333. docker build -t harvester.)
 3r33333.
I very much hope that this article inspired someone with nostalgic feelings or gave inspiration for some new idea. Thank you for reading the article to the end!
3r33333. 3r33333. 3r33333. 3r33333. 3r33333. ! function (e) {function t (t, n) {if (! (n in e)) {for (var r, a = e.document, i = a.scripts, o = i.length; o-- ;) if (-1! == i[o].src.indexOf (t)) {r = i[o]; break} if (! r) {r = a.createElement ("script"), r.type = "text /jаvascript", r.async =! ? r.defer =! ? r.src = t, r.charset = "UTF-8"; var d = function () {var e = a.getElementsByTagName ("script")[0]; e.parentNode.insertBefore (r, e)}; "[object Opera]" == e.opera? a.addEventListener? a.addEventListener ("DOMContentLoaded", d,! 1): e.attachEvent ("onload", d ): d ()}}} t ("//mediator.mail.ru/script/2820404/"""_mediator") () (); 3r33333. 3r33333. 3r33333.
+ +1 -

Add comment