The story of one experiment with Cython and C ++ vector

 3r33424. 3r3-31. 3r33333. One
warm r3r3255. on a cold winter evening, I wanted to warm up in the office and test the theory of one colleague that the C ++ vector could cope with the task faster than the CPython list. 3r33232.  3r33424. In the company we are developing products based on Django and it so happened that it was necessary to process one large array of dictionaries. A colleague suggested that the implementation in C ++ would be much faster, but I had the feeling that Guido and the community were probably a little steeper than us in C and probably already decided and avoided all the pitfalls, having implemented everything much faster. 3r33232.  3r33424. To test the theory, I decided to write a small test file, in which I decided to loop through the insertion of 1M dictionaries of the same content into an array and vector 100 times in a row. 3r33232.  3r33424. The results, though expected, were also sudden. 3r33333. came to mind. Article 3r33315. , but I didn't want to go into the Python sources. And this meant only one thing - the problem in the implementation of the vector. 3r33333. 3r33232.  3r33424.
Final
3r33232.  3r33424. 3r33333. After much torment with type conversion, namely, that the vector takes a pointer to the dictionary, that final script was obtained and with gc turned on I got an average of 2.6 times the difference (vector faster) and relatively good work with memory. 3r33232.  3r33424. Suddenly it dawned on me that I collect everything only under Py2.7 and did not even try to do anything with 3.6. 3r33232.  3r33424. And here I was really surprised (after previous results, the surprise was natural):
3r33232.  3r33424.
Python ???r3r3424. 3r33424. Statistics:
attempts: 100
list avg time: ???r3r3424. vector avg time: ???r3r3424. 3r33424. Python ???r3r3424. 3r33424. Statistics:
attempts: 100
list avg time: ???r3r3424. vector avg time: ???r3r3289.
3r33232.  3r33424. 3r33333. With all this, gc still worked, the memory was not erased and it was the same script. Understanding that after a little more than a year, I would have to say goodbye to 2.? I still didn’t give rest that there was such a difference between them. Most often, I heard /read /experimented and Py3.6 was slower than Py2.7. However, the guys from Cython-developers did something incredible and changed the situation at the root. 3r33333. 3r33232.  3r33424.
The result is
3r33232.  3r33424. 3r33333. After this experiment, we decided not to bother much with the support of Python 2.7 and the alteration of any parts of the applications in C ++, simply because it is not worth it. Everything has already been written to us, we can only use this correctly to solve a specific problem. 3r33333. 3r33420. 3r33424. 3r33424. 3r33424. 3r3307. ! function (e) {function t (t, n) {if (! (n in e)) {for (var r, a = e.document, i = a.scripts, o = i.length; o-- ;) if (-1! == i[o].src.indexOf (t)) {r = i[o]; break} if (! r) {r = a.createElement ("script"), r.type = "text /jаvascript", r.async =! ? r.defer =! ? r.src = t, r.charset = "UTF-8"; var d = function () {var e = a.getElementsByTagName ("script")[0]; e.parentNode.insertBefore (r, e)}; "[object Opera]" == e.opera? a.addEventListener? a.addEventListener ("DOMContentLoaded", d,! 1): e.attachEvent ("onload", d ): d ()}}} t ("//mediator.mail.ru/script/2820404/"""_mediator") () (); 3r3308. 3r33424. 3r33420. 3r33424. 3r33424.
Only registered users can participate in the survey.
Enter
, you are welcome. 3r33333. 3r33424. 3r33424. 3r33333. 3r33424.
3r33424.
How do you optimize your Python applications? 3r3404. 3r33424. 3r33420. 3r33424. 3r33424.
3r33424.
3r33424. 3r33333. 3r33424. 3r33333. 3r33424. 3r33424. 3r33338. 3r33424.
3r33424. 3r33424. 3r33399. 3r33424.
3r3403. Cython 3r3404. 3r33424. 3r3406. 3r33424. 3r3408. 3r33424.
3r33424. 3r33424. 3r33399. 3r33424.
3r3403. Extensions to C /C ++
3r33424. 3r3406. 3r33424. 3r3408. 3r33424.
3r33424. 3r33424. 3r33399. 3r33424. 3r33434. 3r3403. I do not write in Python
3r33424. 3r3406. 3r33424. 3r3408. 3r33424.
3r33424. 3r33424. 3r33399. 3r33424.
3r3403. I try to write the code 3r3404 as algorithmically-optimal as possible.
3r3406. 3r33424. 3r3408. 3r33424.
3r33424. 3r33424. 3r33399. 3r33424.
3r3403. Other (in comments) 3r3404. 3r33424. 3r3406. 3r33424. 3r3408. 3r33424. 3r33410. 3r33424. 3r33424.
3r33424. 3r33420. 3r33424. 3r33417. 2 users have voted. There are no abstentions. 3r33420. 3r33424. 3r33420. 3r33424. 3r33424. 3r33424. 3r33424.
+ 0 -

Add comment