Why write your game engine?
In December last year, at the Games Gathering 2017 conference, we did report , in which they talked about whether it is necessary for companies working in the gaming industry to write their own engines.
Unity in a variety of game engines
Why in the modern world, in which there are such giants as Unity, do something your own, write your own game engines?
Before you is a slide with data from Unity Technologies on the use of various game engines in 2017. Next year, as you understand, the situation will change. We are interested in what is assigned to 41%, that is - engines of our own design. If you look at the 5 largest companies represented at the conference, each of them will have its own engine. It turns out that in most of the companies that represent the game industry, some internal developments are used. Far from always this is the most reasonable decision in the world, but it is so.
What basic technologies can companies choose to think about developing a game project?
The army of the seven nations
The pyramid that you see on the slide can be continued up and down. It absolutely clearly limits you. Bottom - the operating system, even lower - some basic technology. Above are addons on the engines, a ready-made toolkit that comes with games, such as Neverwinter Nights. Whatever you do, you, one way or another, end up inside this thing.
Suppose I want to be on the second level of this pyramid from below. However, all the paths, in the end, lead to the upper part of the pyramid. We will also use something from the existing one, but it is also very important that we can see in the upper left corner of the top level of the pyramid, that is, the engines of our own design.
Now we will analyze the time, effort and money needed to develop the game.
Back to the basics
If all the pie chart shown on the slide is a game, then the engine is its blue sector. In an ideal world, when developing a game, you can take out this blue sector and put it red, or yellow, or green. You can change one big "U" for another large "U", or take a small "x", and what you have chosen, there will immediately work. In reality, this is not so, but we must strive for this. A similar picture, if it comes to real projects, is observed in the gaming industry constantly. It also happens that the whole diagram is occupied by the engine, but this is true only for some demonstration products.
At the same time, money, time and everything else is distributed exactly this way. Whatever you do, you will have to do everything else that falls into the green area of the diagram. It does not change from engine to engine. As a result, if you have enough resources to support the entire process of working on a game project, then the development of the engine will not be so expensive. However, if someone in the company starts talking about developing their own engine, it is likely to encounter a certain set of objections. Consider them.
Suppose you decided to create your own engine and informed the person making the decisions about it. In response, you will hear about the same thing that can be seen on the slide: "You will do this long, it's very risky, it's irrational, it will not help us achieve our goals." What can I say to this? The point here is that all these objections, except the last one, do not withstand any criticism.
Long engine development? But if the development with its own engine will go faster, then this is not an argument. Perhaps no one knows what "rational" is. Therefore, all these objections are very subjective.
The last point concerning whether the engine contributes to the achievement of goals is very important. If your goal is to earn by receiving a grant from Unreal ? then probably your engine does not need to do it, because it does not lead to this goal. If you have a goal within which you need to do something on certain technologies, then do not write your engine - you have to take what is. But it's not always possible to use the ready engines effectively. Why, all the same, write your engine?
13 reasons why
When might you need your own engine? Let's take a look at this slide:
Time to Market - the time period for the product to enter the market. It's really serious. Half of those engines that are now used in large companies were developed precisely because at the moment when this company needed to quickly quickly occupy a niche, it was faster to develop its own than to take something alien and master it.
Here's a story for you. In one company on the site there was a task for programmers, looking something like this: "Guys, if you want to work with us, please write" Asteroids ", which should be run on the Android platform without using external libraries. We believe that this is enough for you 8 hours. Time has gone. " Then the time was increased to 12 hours. Maybe it looks ridiculous. At first I watched this from the outside, then I looked into this company and asked to tell me about what was sent as a response to this task. As it turned out, more than two hundred programs were selected, that is - these programs were launched and worked. This means that if you suddenly want to publish "Asteroids" for Android, then there will be at least 200 people who can do it in 8 hours. I'm not saying that it can be sold. But I told this story to the fact that very often, in fact, the engine - this is Time to Market. For example, just because you have such small needs, that you will study the documentation for the same Unreal longer than just take it and make it your own.
Lord of the Platform is the lord of the platform. There are platforms for which there are no ready-made tools. For example, STB, set-top box (digital television receiver) is a box for cable television, which is on the table of every third American.
Warner has 120 million users of this service. If you write there software, and games including, then you have a dollar off the box. This is $ 120 million a year. In addition to you, no one else can write for this thing. Because there is no DirectX, there's nothing at all. If you know how to write programs for STB - then you are the lord of the platform, you have these hundred and twenty million dollars a year. What is not the reason to write your engine?
It is clear that we are now talking more about something like console toys, but, nevertheless, in some situations this may be necessary. Here, for example, slot machines. Of course, there are now mostly computers. But before us is a separate iron device and a huge market, for which it is quite possible to write something of its own.
We can say that we are interested in telephones, but we are talking about millions of dollars. Why not write for other devices? As a result, there are absolutely clear reasons to do this.
Or, for example, we have the newest smart watch. The SDK has not yet come out to them, no one understands what to do with them, and if you can write a quality product for them on your own, then you, for example, will earn a dollar from each such device. If they are sold two million, then you will get two million dollars. It's easy to write, but you have to make your engine for this, because there are no strangers, and the manufacturers of such devices will not make public engines for development under them.
Weak but proud devices are small but proud devices. If you make games for mobile phones, collect at least some statistics, then you know that with the hardware from the devices from Apple everything is more or less normal, but with the Android platform - just trouble.
Half of the devices on the market are built on the basis of the Mali-400 chipset from ARM, any budget phone is the Mali-400. And if you are paid for doing telephone applications, then you should write for these little ones, but proud devices that are not going to leave the market yet and will not leave it very soon.
In this case, in the case of the iPhone, you can do at least some bets on progress. For example, it is expected that Unreal will work under the iPhone 1? although until all this is brought to mind, there will already be some iPhone 1? 15 or 17. But in the case of the world in general in the short term, progress is harder to put. Because the device upgrade is very slow.
If you want a competitive picture, and if this is very important, you do not go to low-power devices. But you have to take into account that all modern engines do not scale very much down. If you do not want a competitive picture, then, accordingly, consider the possibilities of weak phones. Therefore, if you are in a situation where you are not interested in the fastest devices, for example, you are the only distributor somewhere in Portugal or in Brazil, then you will have to think about it.
Own language for own ideas - own languages for own ideas. When you do the engine yourself - you can use this concept. The fact is that the main problem of our industry is that the domain of game designer is philology. He thinks in ordinary language. He knows nothing else. A programmer thinks in the programming domain and there is a gap between them. As a result, an iteration, which has to be repeated continuously, costs, for example, two dollars. And you constantly spend this money.
Standard engines try to cover everyone. In fact, we see how they try to make natural domain transformations from language to language and from space to space. But for everyone. And you have your own ideas, and you can implement them directly, making your own set of tools. It is clear that all this, in the form of plug-ins, can be done on top of existing engines, but its engine opens up completely different possibilities.
Unique Mechanics = Unique engines - unique mechanics = unique engines. My friends wrote Minecraft in the fifteenth year using Unity. Whether there was a sense to do all this is an open question. But they chose the engine and started to work. However, the engine obviously hindered them. It was hard for them. They had very long iterations. After we had consulted them, they wrote, in just three days, their own render. And all the rest of the code, responsible, say, for building the world, of course, has not gone anywhere. Simply, it all ceased to be in C #, ceased to be in Unity. And the work began to boil. I do not know if they managed to make money on it, but the main conclusion from this story is that they did not need to use Unity from the beginning.
That is, there are a large number of mechanics for which standard, large, universal engines are not suitable simply because they are designed for everything. Therefore, if you come up with something special tomorrow, some complicated voxel mechanisms, then it will be inconvenient to work with standard engines. That is, there are mechanics for which standard engines are not suitable, and which are simple enough to implement independently.
The game is not a render - the game is everything else - the game is not a render, the game is everything else. We have already spoken about this. If you have a problem just in drawing something, or, say, using sound, making a multiplatform game, then in the previously considered pyramid you could see similar stories. If you say: "I want to play sound on three platforms," then you do not need a large "U" for this - a small "c" will be enough.
The reasons for developing our own engine, we considered. Let us now dwell on the advantages that the company gives to the development of its own engine.
Advantages of developing your engine
Consider the advantages of developing your engine, drawing on the main ideas made on the slide:
Buying is often better than mortgage - buying is often preferable to a mortgage. Game development is, one way or another, money. There are ways to monetize, when using which the purchase is not simply better than a mortgage, and this is simply the only possible option.
If someone worked on mobile technologies, then you understand everything. If the box says "10 percent royalty", then it is absolutely unacceptable, you will not earn so much. You can have a profit of one hundred percent, but you work from 2. That is, if you have royalties, then this is purely an economic reason for abandoning the engine. And I must say that three, more precisely - two of the most popular at this time engines - it's just a matter of deductions. That is, this option immediately disappears.
Specificity is better than universality in the long term - at a long distance, specialized tools are always better than universal ones. Obviously, universality is always slower, it is less productive and less specific than specialized things. The engine, written for a specific task, will cope with it better and faster than a universal tool. At a long distance, special tools are much more profitable than universal ones.
Tools and pipelines are developed within - the paylines and development tools created within the company. Any engine, invented by people outside your company, is guided by several things. The first is best practices. That is, the engine of another company is oriented not on how your artist draws, but on how artists paint, ideal from their point of view. It is possible that your artists paint differently. They have their own pipelines and they get it.
You have 2 options for action: either re-train them in the way they should in terms of best practices, or use your own. There are simple examples. Suppose you say: "We import 3D models". You do not know what's on that side. Therefore, you need an intermediate format. The intermediate format is FBX. Flaws FBX everyone who does this, know. And you have nowhere to go, because you do not know what will be done there, you will not write plug-ins for 450 3D modeling tools.
When you work inside your company, then you can realize the very same payline that you already have and put it on top of what you are doing. In fact, this is very important. The fact is that all this has to do with the time of development and, as a consequence, the cost. Therefore, when you develop at home, you can recycle the one that already exists. Otherwise, you have a document called "The rule of unloading 3D models and creating materials for artists" will be more than a design document, which is wrong.
Reaction time - reaction time. It's about the reaction time of the engine manufacturer to your requests, the possibility of equipping the engine with new functionality, and the operational research of new technologies.
Say, in the next office people are sitting, who make the engine. Everyone who tried to fix a bug in the universal engine, that is, write to the bugtracker Unity or Epic, knows that it's better not even to start. And if the developers are sitting in the next office, then you can address them and solve the problem in 15 minutes.
The same applies to the offer of a new functional, if you have the right to do so. Research of new technologies is also simplified when using your own engines.
Suppose your programmer went to a conference, listened to a lecture about something new. He immediately tried it, you got an idea about this new one and you know whether you need it or not. You can directly try to try something interesting from the world of science. And this, by the way, means that there can be a person in the company who will be called a "researcher". At the same time, research can be done on the same Unreal, since it is supplied in the source code.
Performance - performance. The gaming industry is always productivity. The first approach to achieving high performance is to use special solutions. The more specific the solutions are, the more productive they are. The second approach is also, by the way, respected, it is the optimization of ready engines. How it will look is very dependent on these engines.
Developing your own engine is not only an advantage. This is also a risk. Consider them.
Risks associated with the development of its engine
Consider the risks accompanying the development and use of their own engines:
Development time - development time. This concept overlaps with what we talked about the time to market. The development of the engine can be very long, and fast enough. But the development time of the engine, in any case, contributes to the overall development time of the project. Therefore, this is also a risk. For example, I know of teams whose development time for the engine tends to infinity.
Terminal cases of vendor-lock - terminal cases of binding to the vendor. This applies not only to large companies, but also to small ones. For example, you hired Vasya, he wrote the engine, then fell in love, left you, and nobody understands what he wrote. As a result, you have vendor-lock pohlesche Google. Because Google can still write a letter, although they will not answer, but here with the departure of the programmer it all ended. As a result - the lost time for development and other unpleasant consequences. We must be able to avoid these risks.
Reinvent the wheel - the invention of the wheel. The thing is that we live in a world in which you still invent bicycles. When developing the engine, it turns out to transfer the bicycle plant from the game code to the engine code, although there is no place for them.
Closed ecosystem - closed ecosystem. Everything that is done inside the company belongs to this company. I know a lot of companies that have something like their own scripting language. It can be any XScript that only works as part of their solution.
Programmers who know this technology, in fact, do not know how to do anything else. This can be considered one of the factors that help to keep employees. Therefore, the answer to the question of whether it is good or bad, whether use of own technologies is a risk, depends on the specific situation. We, for example, try not to use the concepts of our own invention. For example, I know of one company that has Lua, strongly typed, with Pascal syntax. It can be mastered, but this knowledge is dead, like the Greek language. We try so hard not to do it.
The main question of life, the universe and all such
Let's consider a very important question. What resource is required first of all for developing your own engine? A resource, without which it makes no sense to think about whether or not you need to make your own engine. The answer is, of course, not 42. The question is - what do you need to just at least have the opportunity to say: "Yes, we have something, we can start doing something." The answer to this question is that programmers are needed to develop their own engine.
In order to create your own engine, you need programmers. If you do not know - google the difference between the words "developer" (developer) and "programmer" (programmer). It is very important. Developers are the main group. The game industry is so organized that most people in it can not be called programmers. Sorry, but they are developers. Developers are not able to make the engine competently. Again, if you look at the difference between the first and second, then the developers make the games, and the programmers make the tools. Developers make a product, the company earns from the product, but the tools must be made by programmers, otherwise they just will not work.
On the one hand, it's a very open world. I, for example, know the code Unreal 4 and CryEngine, it is open. Anyone who wants to know can learn the Unity code, there is a huge amount of relevant materials. This means that it is possible to do this by someone who is a programmer and reads in English. There's no rocket science. But on the other hand, as it turned out, programmers who read in English, it is very difficult to find. Therefore, you must know where they are found, must be able to recruit, use, encourage. If you know how to do this, you can even think about your engine. If you do not have such people, then you still will not succeed. Examples are darkness. It's not that there are bad and good solutions. It's just that there are things that can not work from the beginning.
Programmers need to be able to hire. You know how to hire - you can do the engine. You do not know how to hire - then you have to take something ready. Moreover, the funniest thing is that when you take something ready, then to you, if you are a big company, you still need two people from these programmers who read in English.
If you need programmers to develop the engine, then we immediately have several questions. Where to look for programmers? How to organize their work? What is worth paying attention to, thinking about the ways of development of gaming companies?
Now it is appropriate to recall one more aspect of employee search, which concerns, mainly, large companies. Such companies have several approaches to recruitment.
First, you can recruit people, junos, arranging internships, and, training them inside the company, somehow raise to the desired level. This is a normal approach. At the same time, a lot of technological problems are solved, because with a person who initially perceives corporate culture and studies certain technologies, it is easier to find a common language.
Secondly, there is an approach that we, in principle, profess. How does kefir work? He turns everything around into himself. Take milk, throw kefir there - and there is no milk, everything turned into kefir. At us it looks so: "Children, let's take 5 strong programmers, it will be the internal technological center". And I tell everyone that if you can afford to do RnD-department, if you are a big company, then do it. Let them even do nothing, in terms of money, do not do useful. If the company is strengthened in the market and the question arises about where to expand, the answer to this question may be the creation of the RnD-department. When a company is already rich, it is not a loss of money for it, because it loses so much money already at the efficiency at which our gaming industry is currently operating, that it will simply not notice the costs of research.
Now consider the approach, which is that the company is organizing a team that makes the engine or some other interesting things. This is a work for the future. You can conduct interviews, talk about that you give money, you have an interesting task, you are doing the engine. You can choose from applicants, people go to you, and within the company you always have an atmosphere where you can motivate, encourage and, as a result, achieve your goals.
I, for example, have some modules developed by grocery companies, because they need it. That is, the physics is ordered, and instead of sawing some of our pieces, we say: "Let's do it. We just come up with common interfaces, several generalized ones, and you will do it. " And within the framework of typical tasks this is very good. That is, in principle, it is good to spread the technology within the company.
If the company is already so large that it can afford to do something interesting inside itself, then it pays off even in terms of money. So if you can - then try it. It can look like anything - let's say, we make our Unreal branch and process everything there as you want. I, for example, in one of the companies did a browser that is placed in 2.5 megabytes of memory. And he worked. Why - I do not know, but it was infinitely interesting.
Above we mentioned the problem of gaming companies, which consists in organizing effective interaction of programmers and designers. Let us dwell on this problem in more detail.
It's time to show you the working place of our game designer. This is a real picture. It shows some demo, the implementation of the behavior, a little later we will dwell on this in more detail.
In the gaming industry, there are two worlds. People are either focused on solving technological problems, or on narrative. And in the middle - some kind of bushes. That is, there is practically no toolkit. Words, words, words - bach - code. Again the words - and again the code. We believe that we need tools that allow you to connect to what will result from the work on the game, the game designer, the manager, other employees who are not programmers.
On the slide you can see the behavior tree, the tree of behavior. In principle, it's a thing, just taken from Wikipedia, but before it from there it was taken by Unreal. Nothing wrong with that. So, the documentation for this lies on the Unreal site, it was easy for us to make a suitable interface that is compatible with what is done in Unreal. That is, you can take any example from the Anrilov action site, an example of the behavior itself, since the format is quite the same, and that's how to rewrite it, and it will work. This means that I have made my life easier, I do not write documentation. And there are many such pieces.
In the slide example, something happens, the crab runs, someone catches, in general, it does not matter. Inside the programmer solves problems that look like "go to ", "shoot at ", "count the distance" - that's all. All other behavior is written in this editor by a person who has absolutely nothing to do with programming. And it works, as opposed to translating text into code. At the same time, if we talk, say, about the balance. What is balance? These are 15 coefficients that can be changed. And here we describe behavior, not coefficients.
That is, for example, the behavior of "patrolling" is described by the game designer, and not by the programmer. This means that we took the step that most people do not do. They just write in a design document: "patrols." A programmer can translate it in 50 different ways. What is patrolling in general? And here the game designer writes exactly what he means. And this is a victory, my friends. That's why you need your own tools. In order to smooth the transition from the verbal definition of your visionary, who sees the game, so to speak, inside the head, to the programmers. Otherwise, they will cease to be programmers, become developers and spend their entire life planting grass.
Let's sum up. We talked about the reasons to write your engine. For example, if you look back at an outdated device, then this is not good or bad. That is, you want your games to run on a certain range of devices that are no longer supported by commercial engines. At the same time, you want to look modern. How to achieve this? Write your own.
Do you want to own a platform? Do you have a specific project that simply does not require the use of universal solutions? Or do you, on the contrary, have a very large and complex project with a specific picture? In these situations, again, you can think about your engine. In order to make your engine, you need resources. And resources are programmers.
As a result, if you have reasons to write your engine, and you have the resources - take and write.
Questions and Answers [/b]
What is the cost of your engine, if you evaluate it in money and labor?
How much it cost money, I probably can not say right now. And if we talk about work, it's nine months of a team of six people. But we must understand that this is our latest development, and here we aim high enough. I do not talk about our toolset, what we do with Lua, or what we do with jаvascript such that no one at all does. We plan to release a couple of modules in the open source, which, if not turned over, then, at least, will help people to understand what scripting languages are. Our previous engine was done by two people for four months. He is also 3D, but simpler, a telephone. It is possible faster, I have already told you how "Asteroids" write for 8 hours, but this, of course, is far from reality.
How long did it take to implement the behavior tree?
This I can say for sure, it was done quite recently. Two people were engaged in this month. But the problem here is wider than creating a tree of behavior. The fact is that the action is written in Lua, it probably takes four days, and write an editor for Qt - a month.
As far as I understand, you use Lua, do you have an action system that you originally wrote, and the behavior tree just pulls these actions?
Yes, that is right. In fact, we are working on bringing this to the open source, writing documentation, the assembly system, examples.
Our company has very similar views with yours, and we have interesting problems too. I would like to know - what is your ratio of labor to the game, to the engine, to the tools? That is, how many people, for example, working on the engine, on the game, how many games use one engine?
We now have two engines, the previous version and a new one. That is, this is not a refactoring. This is a completely new engine. If we talk about labor, then we can say that the company we have large, employs about 500 people, programmers about 25? 5 offices. The project team is working on games. A project is a kind of game, and a number of people are engaged in it. The engine development team is a separate team. This is the same kefir, about which I spoke, elite units, there a little bit other money and approaches to formation of commands. Now we are a little ahead of the development. Two new games are launched on a new engine. This is quite painful, as the game developers are not very comfortable, because they work in a situation where they can literally get something out of hand and explode. And we have a team of the engine - it's 6 people. Product teams are, on average, a man of four programmers, they do not overlap.
Under the engine, do you mean tools?
Yes, we have a separate team for developing tools. We had a very unfortunate example. Very poorly developed GUI-tools. Because any normal programmer believes that it is very simple. We tried to outsource this. Because it's understandable - you give the full interface, you have everything, you say: "Create a window, draw a button - and that's it." But this venture failed, that's why we do it ourselves, we work with Qt painfully, because it's important for us that it works on all three desktop platforms. That's why we do it ourselves. And we have 6 people doing both, and the third. But we are still slightly ahead of the product requests.
Is it realistic now to sell your engine?
No. Now you can not sell your engine. You can sell the ecosystem. That is, work under the scheme "give me money, but I'll give you an engine" can not. Pay attention to how many companies we have our own engine, and how many of them the engine sells. In fact - none of them engines do not sell. For starters, this is a rather big headache in terms of what it should be turned into a product. What you work inside the company, you can not sell. You must, at a minimum, write the documentation that others will understand. You should hire just some army of volunteers who will evangelize this business. And it is not entirely clear what you get from it. And if you make a mobile game on this engine, then you can fully wake up a millionaire. Therefore, in order to do such things, one must be a fan, one must be sure of what you are doing. I talked about the reasons that can lead to the development of your engine, and you have here - another reason. Let's say you think you'll make the engine better than Unreal. If this is so, go to the market. But I do not think I'll do better than Unreal.
I understand that your new engine is C ++ and Lua?
Yes, C ++, Lua and more jаvascript.
And why C ++? Were there any variants, or did you clearly know what exactly you would take?
Look, there is such a fashion. Every second person you meet says to you: "Golang", or says to you: "Rust". And if it was now, I would basically think about it. But when you come to the company as the head of the engine development process, but it was a year ago, you need to make some plans, and you'll have to include the "Read about Go" item in these plans. Here, after all, performance is important, but in C ++ we have been working for a long time, we know how to use it.
And why do we use Lua? Because it's an underrated language, it's great for embedding. Why jаvascript? Because it happened. Because there is nothing on the market other than V8 and Webkit. And these are monsters. As I said, we did a browser that takes 2.5 megabytes of memory, there is a jаvascript engine that passes all the tests. We have it, and that's why - jаvascript. As a result, for example, you can take those who know JS and wrote sites on React.
Tell me, do you use the behavior tree only for controlling behavior, or are you using it for managing game mechanics and for promoting game progress?
Now for the behavior, but we still have several alternative mechanics. We also have, say, Petri nets with the editor, and here the problem is slightly different, which is that Petri nets are hard to explain to the game designer. There are still some things with editors that allow you to draw, say, a finite state machine. And we are trying to do something from this all. I now need to get those people who write scripts to write it in this form. So the behavior tree worked, and the rest is not yet in the workflow.
How difficult is it to predict the development of technology for the future? That is, how difficult is it to foresee the appearance of some kind of reefs and similar things?
At the moment I see one problem. Now the technology of WebAssembly looks interesting. Flash died. We want, of course, to be published somewhere else on the web. Porting a game, say, with Unity to WebGL is a task that can not be solved by pressing one button. That is, now we are looking at the WebAssembly and it's unclear whether it will be a standard or not, start now with this work, or wait. In mobile technology, too, nothing special happens. There are not yet any singular explosions, but if they do, we will prepare.
As a result, I want to say that under normal modular design, with normal design, and I very much hope that we have them normal, when new technologies appear, you can simply remove the old from the engine, put it new, and everything will work.
It may be interesting