Methodology for assessing knowledge of an engineer. The path of the architect and the path of the expert

Is it possible to represent the level of knowledge and skills of an engineer or even a team on one picture?
Parameters, in fact, a lot. But we have a wonderful example of how difficult it can be visualized in a rather simple and visual way.
I'm talking about (I think everyone knows) the Gartner quadrant.
This article is about how this visualization approach was applied to assess the level of knowledge and skills of network engineers in a Russian company.

Two-dimensional approach

When conducting the above survey, it was decided to display the level of knowledge of the engineer in two dimensions. The value deposited on the OX axis characterizes the depth of knowledge and on the OY axis - the breadth of the horizon.
Of course, it is important to initially limit the range of topics addressed, in accordance with the requirements of the company. In our case, these were the themes of the R & S, Security, Service Provider topics.
Methodology for assessing knowledge of an engineer. The path of the architect and the path of the expert
the four stages of development of the engineer 3r32r6262.
Experience has shown that there is ambiguity in such an assessment, and the subjective assessment of interviewers is a big factor. In principle, when we speak of the Gartner quadrant, subjectivity is always present. But in this case, we can formalize the task and eliminate subjectivity.
An example of a formalized (objective) approach 3r3224.
You have a questionnaire consisting of 100 questions. The questionnaire corresponds to 10 topics - 10 questions for each topic. All questions and topics are equivalent in terms of importance.
After completing the test, you will have a distribution of points (the number of answered questions in each topic) of 10 topics. For example, it could be something like:
Q1: 9
Q2: 7
Q3: 4
Q4: 6
Q5: 0
Q6: 3
Q7: 6
Q8: 1
Q9: 1
Q10: 3
Good. And what place in our two-dimensional visualization will the evaluation of this engineer take?
We see that on 4 questions the level is above average, and on 6 - below. In principle, it can be assumed that the engineer is located in the upper part of quadrant IV or in the lower part of quadrant III. Where to add points for knowledge - in the "width" or "depth"? Ambiguity. And, it is not very clear how to correctly.
Different algorithms can be selected. For example, I wrote a simple
python script
and set it up (by changing the factor parameter) in accordance with my intuitive perception.
So, for example, for this case, this script gives the result 3r-3281.  
x = ? y = 5
That is, this is the top of quadrant IV.
This means that on the path of the architect this engineer is on the verge between the 2nd and 3rd steps.
But do not trust the formal approach too much. Nothing replaces personal communication. Even a very strong engineer can not pass the test. Just because recently he worked in a different direction and just forgot something, or he just doesn’t like tests or slept poorly And on the other hand, a good result may be due to the fact that the engineer just knew the test questions in advance.
Therefore, further in the oral interview, you can determine more precisely which stage the engineer is at.
+ 0 -

Add comment