Top reports of JPoint 2018: Java /JVM and its performance, Kotlin, Spring, Docker

 
3r3-31.
We have already posted videos of JPoint 2018 reports on YouTube and, specifically for the Java hub on Habré, we have made a traditional selection of the best of them according to conference visitors.
 
As usual, the top “junior” reports, in the end - with the highest rating. Of course, this does not mean that one report is much worse than another: if you change the calculation method, places can easily change. In reality, we changed it, now the “soft quorum” version of the rating is used, taking into account the number of participants present at the report. This approach has its drawbacks (for example, more people come to the keyout than to the regular report, simply because the audience has no choice), but in general it gives a better picture of what happened.
 
Under the cut - and videos of the best reports, and links to their presentations, and short descriptions, and a link to the full playlist.
 
Top reports of JPoint 2018: Java /JVM and its performance, Kotlin, Spring, Docker Available under the link .
 
10. Linux container performance tools for JVM applications
 
Download slides
 
3r33966.
3r3-1028. 3r3-1028. 3r3-1028.
 
The container revolution has overwhelmed everyone - the world of Java has not been without it. Nevertheless, Java has always had some problems and peculiarities with it, which will be discussed in the report of Sasha Goldstein.
 
Sasha is a serial creator of performance hardcore. Last year at the JPoint, he gave a great talk about using 3r3356. Berkeley Packet Filter for JVM
(I desperately recommend watching the recording on YouTube), and it was only a matter of time before he gets to the detailed parsing of the containerization. The world goes to the clouds and dockers, which in turn brings us many new problems. As you can see, most low-level debugging and profiling systems, after being applied to containers, are overgrown with various features and jambs. Sasha went through the main scenarios with a skating rink (CPU load, IO responsiveness, access to shared databases, etc.) through the prism of using modern tools on the GNU /Linux platform, including BCC and perf.
 
This specificity is needed not by all. This is one of the reasons why the report was only in tenth place - it was attended by about two hundred people (compare with more than a thousand from Tolkachev and Borisov), and our rating calculation algorithm is very sensitive to the size of the audience.
 
For those who are somehow connected with containers, this report is a must-see. Save a lot of time on an independent search for all the details using the method of complete enumeration of the countless number of utilities and ways to use them
 
I would also like to note that Sasha’s report from all the top 10 is almost the densest in terms of the amount of practical information per unit of time. Almost every slide you can copy yourself some useful things. In order not to engage in a detailed retelling, I will show only the surface structure of the presentation:
 
3r3741. 3r3742. Report Layout 3r34343. 3r3744. 3r3994.  
3r3999. How are containers arranged?
 
3r3994.  
3r3999. Control groups: CPU, memory, block I /O; 3r3-3000.  
3r3999. Namespaces: pid namespace, mount namespace, network namespace; 3r3-3000.  
3r3-3000.  
3r3999. The difference between the problems:
 
3r3994.  
3r3999. On the host; 3r3-3000.  
3r3999. In the container; 3r3-3000.  
3r3999. Examples of problems with solutions:
 
3r3994.  
3r3999. Connect to JVM? 3r3-3000.  
3r3999. Performance data JVM? 3r3-3000.  
3r3-3000.  
3r3999. Demo JVM tools on the host; 3r3-3000.  
3r3-3000.  
3r3999. Tools for resource information: 3r3-31015.  
3r3994.  
3r3999. sidecar for monitoring; 3r3-3000.  
3r3999. docker stats; 3r3-3000.  
3r3999. systemd-cgtop; 3r3-3000.  
3r3999. htop + cgroup ID; 3r3-3000.  
3r3999. nsenter or docker exec; 3r3-3000.  
3r3999. Container resource demo; 3r3-3000.  
3r3-3000.  
3r3999. Profiling of CPU containers:
 
3r3994.  
3r3999. perf on the host: -G, symbol mapping, PID mapping, sharing perf map (perf-map-agent); 3r3-3000.  
3r3999. Demo with flamegraphs; oos; 3r3-3000.  
3r3999. perf in a container in unprivileged mode; 3r3-3000.  
3r3999. honest-profiler, async-profiler, perf vs async-profiler; 3r3-3000.  
3r3999. Demo async-profiler; 3r3-3000.  
3r3999. BCC is a profiling tool (it solves a number of problems on its own, but it only works on Linux 4.9+); 3r3-3000.  
3r3999. Throttling; 3r3-3000.  
3r3-3000.  
3r3999. Other monitoring methods:
 
3r3994.  
3r3999. cAdvisor, SysDig, New Relic, DataDog 3r3-31000.  
3r3-3000.  
3r3-1028. 3r3-1028.
 
3r3195. 9. Once a year, gardens bloom: parsing the “exactly-once” semantics Apache Kafka
 
Download slides
 
3r33966.
3r3208. 3r3-1028. 3r3-1028. 3r3-1028.
 
Victor Gamov has a completely unfair advantage over the other speakers. This is the man who released almost two hundred issues of the podcast Debriefing, the author of a heap of reports, texts, posts and even the book “Enterprise Web Development” of O'Reilly. The very presence of Gamow makes anything better. I’m sure he could, with the same intonation, tell not about Kafka, but about the growth of geranium on the window sill, and it would still be fascinating.
 
Here we have a special case: initially holivornaya topic about the semantics of "exactly once". Kafka was rushed to use everything on an extraordinary scale, which required reworking the traditional semantics of message delivery. Kafka didn’t put a finger in his mouth - they changed their shoes on the go under the correct protocol and message format and did everything they needed. Throughout the report, Victor tells how it works inside and what it affects.
 
Two news, good and bad. Good: in Kafka, everything is configured. The bad: in Kafka everything is configured. To live with this, you need to understand how it works and where to climb with your dirty hands.
 
One would expect some kind of smoothie from Viktor, but it turned out to be a desperate hardcore, and a plan like this:
 
3r3741. 3r3742. Report Layout 3r34343. 3r3744. 3r3994.  
3r3999. Introduction to Kafka:
 
3r3994.  
3r3999. Topics 3-3331000.  
3r3999. Parts 3r3-3000.  
3r3999. Logs and offsets
 
3r3999. Replicas and leaders 3-3331000.  
3r3999. Clients: producers and consumers 3-33-3000.  
3r3-3000.  
3r3999. Producer
 
3r3994.  
3r3999. How it works on examples 3-3331000.  
3r3999. Protocol producer 3r3-3000.  
3r3-3000.  
3r3999. Counselor
 
3r3994.  
3r3999. How it works on examples 3-3331000.  
3r3999. Protocol Consumer 3r3-31000.  
3r3-3000.  
3r3999. Processing model
 
3r3994.  
3r3999. They read, they counted, they recorded 3-33-3000.  
3r3999. Types of failures (everything broke, zombies) 3-3331000.  
3r3-3000.  
3r3999. Processing Semantics
 
3r3994.  
3r3999. At-least-once, at-most-once, exactly-once
 
3r3-3000.  
3r3999. The semantics is “exactly once”
 
3r3994.  
3r3999. With examples 3r3-3000.  
3r3999. Weaknesses
 
3r3994.  
3r3999. insecure retries 3r3-331000.  
3r3999. non-atomic record with offset 3r3-31000.  
3r3999. zombies 3r3-3000.  
3r3-3000.  
3r3-3000.  
3r3999. We fix the first weakness: idempotent producer 3r3-31000.  
3r3999. We fix the second weakness:
 
3r3994.  
3r3999. Chengde Lamport snapshots
 
3r3994.  
3r3999. Principle 3r3-31000.  
3r3999. How to live malfunctioning?
 
3r3994.  
3r3999. Hard: crash during processing
 
3r3999. Soft: fail before or during snapshot 3r3-331000.  
3r3-3000.  
3r3-3000.  
3r3999. Transactions in Kafka
 
3r3994.  
3r3999. Two types of markers (COMMIT, ABORT) 3-3331000.  
3r3999. Atomic writing to multiple partitions (Including! _Consumer_offsets) 3-3331000.  
3r3-3000.  
3r3-3000.  
3r3999. We fix the third weakness: zombie fencing
 
3r3999. Isolation of reading at the bureau 3r3-31000.  
3r3999. end-to-end eos
 
3r3994.  
3r3999. Kafka Connect Source 3r3-31000.  
3r3999. Kafka Streams
 
3r3999. Kafka Connect Sink 3r3-31000.  
3r3-3000.  
3r3-1028. 3r3-1028.
 
3r33434. 8. Fight Russian Hackers with Kafka Streams and Firehose API
 
3r33424. Download slides
 
3r33966.
3r33434. 3r3-1028. 3r3-1028. 3r3-1028.
 
Again Kafka! Again Gamov! Yes, and with Baruch Sadogursky (3r33440. Jbaruch 3r3-31019.) Now. Nevertheless, this report is not a smoothie, but a very concrete practical thing about how on the Bintray platform (of which Baruch is the god) using Apache Kafka (for which Victor is responsible) and Firehose to analyze behavioral patterns, process large amounts of data.
 
If the previous report about Kafka's insides had 245 slides, then this one is only 17. That's because you need to watch it! This is primarily a lively dialogue between the leading and live demos. Do not linger, open vidosik and look.
 
7. Hardware transactional memory in Java 3r3-31010.
 
Download slides
 
3r33966.
3r33462. 3r3-1028. 3r3-1028. 3r3-1028.
 
Nikita Koval ( Ndkoval 3r3-31019.) Is a researcher at JetBrains on the Kotlin team and a PhD student at IST Austria. His report contrasts strongly with the “Russian hackers,” because this is not a light entertaining read, but a story about the intricate insides of a VM. If you look at
slides
, you can find 150 sheets, much of which - the code.
 
If you were at JBreak in the beginning of 201? you could find Nikita’s story about completely different things - writing a fast multi-threaded hash table using the power of modern multi-core architectures and special algorithms. Agree, Nikita has
style. . I immediately recall Shipilev.
 
This time we will talk about transactional memory, which gradually appears in modern processors, but which is not yet clear how to use an ordinary person from the JVM world. Outside the JVM, of course, everything is easier. But imagine how you tell the usual Spring web developer: "Yes, you just edit the vmstructs, load your intrinsics, rebuild OpenJDK and you're done!" And he said: "Well, of course, I do it every day!". Nikita just very clearly talks about how to use it, what optimization already exists in OpenJDK and how to perform transactions directly from Java code.
 
3r3741. 3r3742. Report Layout 3r34343. 3r3744. 3r3994.  
3r3999. Introduction: Why we need multithreading 3-33-3000.  
3r3999. Approaches to the construction of algorithms 3r3-31015.  
3r3994.  
3r3999. Rough lock 3-3-331000.  
3r3999. Thin lock 3r3-31000.  
3r3999. Non-blocking synchronization 3-3-331000.  
3r3999. All three views are illustrated on task.example of a toy bank 3r3-31000.  
3r3-3000.  
3r3999. Multithreading is hard. What to do?
 
3r3994.  
3r3999. Transactions in a perfect world. Just write atomic! 3r3-3000.  
3r3999. Where to get atomic:
 
3r3994.  
3r3999. Software Transactional Memory (STM). Scala STM, NOrec, Corutin. 3r3-3000.  
3r3999. Hardware Transactional Memory (HTM). Haswell, Power 8. 3r3-31000.  
3r3999. Hybrid Transactional Memory 3r3-31000.  
3r3-3000.  
3r3999. Intel RTM on examples 3-3331015.  
3r3994.  
3r3999. XBEGIN, XEND, XABORT, XTEST 3r3-331000.  
3r3999. Intel RTM + Java
 
3r3994.  
3r3999. Lock elision
 
3r3999. java.util.concurrent.RTMSupport
 
3r3-3000.  
3r3999. Intrinsics: interpreter, C? C2 3-3-331000.  
3r3999. Coarse-grained /Lock-free + RTMSupport on charts
 
3r3-3000.  
3r3-3000.  
3r3-1028. 3r3-1028.
 
3r38080. 6. We profile up to microseconds and instructions of the processor 3r-31010.
 
3r33585. Download slides
 
3r33966.
3r3-3593. 3r3-1028. 3r3-1028. 3r3-1028.
 
Sergey Melnikov from Raiffeisenbank brought us the second profiling report. Interestingly, before taking up the low-latency code in Java, he worked at Intel as a performance engineer of compilers for C /C ++ /FORTRAN languages. This report also has a perf! :-) There is also about the hardware features of processors and Intel Processor Trace technology, which allows you to take the next step in profiling accuracy and to reconstruct the execution of the program section. There are quite a few such reports (for example, you can find 3-333601. Andi Kleen report 3-3331019. At the Tracing Summit 2015), they usually leave a lot of questions and do not shine with practicality for Java. Here we not only have a person who has visited both worlds (both Intel and Java in the bank), but also who is able to clearly explain complex topics.
 
3r3741. 3r3742. Report Layout 3r34343. 3r3744. 3r3994.  
3r3999. What it is and why you need
 
3r3994.  
3r3999. The subject area is the low latency of application 3r3-331000.  
3r3999. Moscow Exchange Example 3-33-3000.  
3r3-3000.  
3r3999. Choose a profiler
 
3r3994.  
3r3999. How to profile? Sampling and instrumenting profilers 3-3331000.  
3r3999. async-profiler
 
3r3-3000.  
3r3999. We learn the profiler to collect a detailed profile
 
3r3994.  
3r3999. How to run perf
 
3r3999. How to look in a profile, in call stack
 
3r3999. perf-map-agent, sysctl, dmesg
 
3r3-3000.  
3r3999. We use PMU /PEBS events in perf 3r3-3000.  
3r3999. Intel Processor Trace - what it is and how to profile Java
 
3r3994.  
3r3999. Requirements: packages, hardware, operating system 3-33-3000.  
3r3999. How to run on Skylake-X (Xeon and i9) 3r3-31000.  
3r3-3000.  
3r3-1028. 3r3-1028.
 
5. VMStructs: why should an application know about the internals of the JVM
 
Download slides
 
3r33966.
3r3-1028. 3r3-1028. 3r3-1028.
 
Andrei ( Apangin ) Is a person who collects the most profound and powerful reports over and over again. On the past Joker, he had a little less than a thousand people - a record in terms of audience size in a regular non-keyout report. In this he is helped by a decade of experience in virtual machines and the ability to explain technical hardcore so that it can be repeated in practice.
 
Many people do not quite understand why digging into a virtual machine if you have a regular application on a computer, with a database and everything is as it should be. This report has a good argument at the level of "how to understand which query will pull a lot of data?" If you try to instrument the code with the help of JMX, then something strange is happening with the performance. First, it is possible to understand why this is happening, and secondly, what can be done with it. In my opinion, this report is valuable not so much as a set of tools (although there is a little about helfy), as a demonstration of the correct mindsets developer OpenJDK and how to behave in difficult situations. It says the existence and explanation of the specific things like TLAB, Code Cache, Constant Pool, etc.
 
4. Korutin in Kotlin
 
Download slides
 
3r33966.
3r3-1028. 3r3-1028. 3r3-1028.
 
Only the lazy one does not know about Kotlin now, and you, the reader, once read to the fourth point - obviously not one of the lazy. The novel ( Elizarov ) Is a former developer of high-performance trading software, and now is a team leader in the Kotlin libraries. We already did with Roma An interview about Korutin 3-3331019. , and it may be worth rereading before or after viewing the report. Korutins are a very old concept, from the time of the Simula, but not all mainstream languages ​​support them, they will not appear in Java soon. But in Kotlin they are already in a stable version.
 
This report answers all current questions about Korutin, that is, all current issues of our time in general :-) 3r3-31014.
 
3r3741. 3r3742. Report Layout 3r34343. 3r3744. 3r3994.  
3r3999. The overall picture of the development of languages ​​3-3-31000.  
3r3999. Asynchronous programming with callbacks 3-3-331000.  
3r3999. Futures /Promises /Rx
 
3r3999. Korutinas in Kotlin
 
3r3994.  
3r3999. regular loops, exception handling, higher-order functions
 
3r3999. custom higher-order functions
 
3r3999. everything looks like in blocking code! 3r3-3000.  
3r3-3000.  
3r3999. How it works?
 
3r3994.  
3r3999. suspending functions, code with suspension points
 
3r3-3000.  
3r3999. Integration
 
3r3994.  
3r3999. Retrofit async
 
3r3999. Callbacks and Continuations (call /cc from Scheme!) 3-3-31000.  
3r3999. contlinx-coroutines-core
 
3r3994.  
3r3999. jdk, guava, nio, reactor, rx? rx2
 
3r3-3000.  
3r3-3000.  
3r3999. How to run a korutiny? 3r3-3000.  
3r3999. async /await
 
3r3994.  
3r3999. Why is there no await keyword in Kotlin? 3r3-3000.  
3r3999. Concurrency is hard. You need to do this explicitly. 3r3-3000.  
3r3999. Kotlin's approach to async 3r3-31000.  
3r3-3000.  
3r3999. The concept of korutin: very light threads 3–3–31000.  
3r3999. Interop with Java 3-3-31000.  
3r3999. Synchronous coroutines - generate /yield
 
3r3994.  
3r3999. Example with Fibonacci numbers 3r3-31000.  
3r3-3000.  
3r3999. Communicating Sequential Processes (CSP) 3-3331000.  
3r3999. Library vs Language
 
3r3994.  
3r3999. The core language should be small! 3r3-3000.  
3r3999. Keywords: async /await, generate /yield
 
3r3999. Modifiers: suspend
 
3r3999. kotlinx-coroutines: launch, async, runBlocking, future, delay, Job, Deferred,
 
3r3-3000.  
3r3-1028. 3r3-1028.
 
3. On the shoulders of giants: the languages ​​in which Kotlin studied 3r3-31010.
 
Download slides

 
3r33966.
3r3887. 3r3-1028. 3r3-1028. 3r3-1028.
 
A report about Kotlin from one of the creators of the language - what else is needed for happiness? The essence and structure of the presentation is completely different from that of the previous report of Elizarov. Roman talked about specific things - what, how and why in the design of korutin, and how to use it, and I need it to improve my programming skills on Kotlin. Here, Andrew ( Abreslav ) Tells things that improve knowledge of life in general and give an understanding of their place in the world. If you have seen all these languages ​​with your own eyes - Java, C #, Scala, Groovy, Python, Gosu - this is even more interesting, because there is a reason for discussion (conference visitors probably could have taken the opportunity and really discussed their understanding of things live in the discussion area ). It is 3r33897. “Seven languages ​​in seven weeks” 3r3-31019. , but only in one hour.
 
By the way, we recently did with with Andrei. separate interview 3-3-31019. , but it is not only about Kotlin, but about a lot of different things.
 
3r3908. 2. The adventures of Senor Holmes and Junior Watson in the world of software development
 
3r3393913. Download slides

 
3r33966.
3r? 3921. 3r3-1028. 3r3-1028. 3r3-1028.
 
Making reports in which more than one speaker participates is unusually difficult. Often it looks like this: half of the time one of them stands on the stage and is bored, and it looks very sad. What can be said about the joint performance of Baruch Sadogursky and Yevgeny Borisov (3r33929. EvgenyBorisov 3r3-331019.) - on the contrary, it was made magnificently, it is a work of art. The stars came together in the right order, and two top speakers appeared on the scene with a wealth of experience and practice in conducting paired reports to discuss an interesting topic for both of them. Why I emphasize it this way - usually, viewers have no idea what the creation of such a presentation is worth, and take everything for granted.
 
But the result justified the investment. Judge for yourself, this is the evening keyout, the most recent conference report, during which people are already exhausted by two days of tin, they want to sleep, someone needs to fly home right now, and so on. And yet, in the hall there are more than 600 people for sure.
 
This is not a reference report, but a report-show, you need to watch it. In it, Holmes and Watson will uncover several puzzles that you have encountered, encountered or will encounter in everyday development. There will be no gutters of garbage collectors and bytecode, but there will be tools, libraries and frameworks that perplex ordinary developers in their everyday routines, leading to downtime, dead footing and prolonged depressions. In fact, in this report, Sherlock and Watson save your forehead from the facepalm and the rake that someone has already attacked.
 
Therefore, the summary content here again will not be - just start the video and watch.
 
1. Boot yourself, Spring is coming
 
3r3951. Download slides
 
3r33966.
3r3r9959. 3r3-1028. 3r3-1028. 3r3-1028.
 
3r33966.
3r33939. 3r3-1028. 3r3-1028. 3r3-1028.
 
Baruch + Gamov, Baruch + Borisov, who is missing among the authors of the popular pair reports? That's right, Borisov + Tolkachev (3r3-3977. Tolkkv 3r3-31019.). Another cool report that, not being a keyout, gathered a record number of participants - more than a thousand. The material turned out to be so much, and it is so interesting that they allocated as many as two slots in the conference program - and you will have to go to YouTube,respectively, watch the whole two videos.
 
Many years ago, Java programmers used “new” to create services. They did a lot of manual actions and mixed configuration with business logic. They even used copy-paste techniques. It was written many lines of poor code, which at times even worked.
 
Then Spring appeared. With him, much has changed We got a lot of "magic" from the magic cylinder of Spring, and our code has become more clean, simple and maintained.
 
And there was Spring Boot. On the one hand, it solves thousands of previously existing problems: version conflicts, configuration tasks, work with infrastructure bins, the problem of setting up the environment and, of course, launching or deploying the application, including building jar /war archives. On the other hand, Spring Boot added even more magic to our magic top hat. As a result, there are two scenarios:
 
3r3994.  
3r3999. Everything works fine, although no one knows how. 3r3-3000.  
3r3999. Nothing works, and no one knows why. 3r3-3000.  
 
This report reveals the basic secrets of Spring Boot magic. You are aware of the basic principles and conventions by which typical Spring Boot applications work. The task of Cyril and Eugene is that all the magic from the cylinder turns into a much more transparent process for you and you can not only have fun when everything works, but also understand the essence of the problems that arise and can solve them without using ambulance and other services available urgent calls.
 
Conclusion 3r3-31010.
 
It took me a few days to watch all the videos from the list. It was not easy, but it is clear that it was worth it. In reality, there were much more reports at the conference, and watching the top ten of them was a good start for a much larger way. If you also watch these reports - do not be lazy to write your feedback in the comments on Habré!
 
Meanwhile, it is already possible to purchase tickets for the next JPoint. It will be held April 5-? 2019 at the WTC Congress Center. Until January ? there is still an opportunity to purchase tickets at low prices ! 3r3-1020. 3r3-1028.
3r31025. ! function (e) {function t (t, n) {if (! (n in e)) {for (var r, a = e.document, i = a.scripts, o = i.length; o-- ;) if (-1! == i[o].src.indexOf (t)) {r = i[o]; break} if (! r) {r = a.createElement ("script"), r.type = "text /jаvascript", r.async =! ? r.defer =! ? r.src = t, r.charset = "UTF-8"; var d = function () {var e = a.getElementsByTagName ("script")[0]; e.parentNode.insertBefore (r, e)}; "[object Opera]" == e.opera? a.addEventListener? a.addEventListener ("DOMContentLoaded", d,! 1): e.attachEvent ("onload", d ): d ()}}} t ("//mediator.mail.ru/script/2820404/"""_mediator") () (); 3r31026.
3r3-1028.
+ 0 -

Add comment