Developers Club geek daily blog

Synchronous code in asynchronous Twisted, or the narration how to cross a hedgehog with uzhy

2 years, 11 months ago
Everything is good

Twisted — asynchronous (it is event - oriented) the framework written on Python. A powerful tool for rapid development network (and not only) services. It is developed with use of a pattern of design of Reactor. The services created with use of Twisted are fast and reliable, the framework allows not to write the macaroni code sated with unclear kollbeka, has in itself beautiful helper (Deferred, Transport, Protocol etc). In other words, does our life bekend developers better.

But there are also problems

The main problem is that the numerous, reliable, ottestirovanny, convenient libraries using synchronous Python modules in the basis (socket, os, ssl, time, select, thread, subprocess, sys, signal etc), will just take and will block to us the main process, a cycle of the reactor and there will come the trouble. Such libraries, for example, are psycopg2, request, mysql and others. In particular, psycopg2 is used in Django ORM as one of bekend of databases.

What to do?

There are three ways. Difficult, acceptable and good. Difficult — to implement analog of library on Twisted. Accepted — to use deferToThread and to start a synchronous code in separate flows (using the thread pool implemented in Twisted). About a good way (in my opinion) the speech in a note will also go.
Скрестить ежа с ужом

Read more »


Assembly of XGBoost for Python under Windows

2 years, 11 months ago
Windows is so evil that consumes extra energy to make the things running.



The XGBoost library rattles at all machine learning competitions and helps to take prizes. However, the owner of this packet for Python under Windows to become not so simple.

Installation process is poorly described on GitHub and the little is broader at the forum Kaggle. Therefore I will try to describe step by step in more detail. I hope it will help to save a lot of time to novice users.

Read more »


Selenium for Python. Chapter 6. Objects of the Page

2 years, 11 months ago
Continuation of transfer of unofficial documentation of Selenium for Python.
The original can be found here.

Contents:


1. Installation
2. First steps
3. Navigation
4. Search of elements
5. Expectations
6. Objects of the Page
7. WebDriver API
8. Application: Frequently asked questions

6. Objects of the page


This section is introduction to the guide to patterns of design of objects of the page. The object of the page represents area of the user interface in a web application with which your test will interact.

Pluses of use of patterns of objects of the page:

  • Creation of repeatedly used code which can be used among a set of test cases
  • Reduction of quantity of the duplicated code
  • If the user interface changes, editing in only one place is required

Read more »


Parsing of formulas in 50 lines on Python

2 years, 11 months ago

Read more »


We study the graph - the oriented Neo4j DBMS on the example of the lexical Wordnet base

2 years, 11 months ago
Neo4j DBMS is NoSQL the database oriented to storage of graphs. A highlight of a product is declarative language of requests Cypher.

Cypher borrowed a key word like WHERE, ORDER BY from SQL; syntax from such different languages as Python, Haskell, SPARQL; and as a result there was a language allowing to make requests to graphs in a visual form like ASCII art. For example, I would present heading of this article in the form of the graph (Neo4j) — [we study]-> (Wordnet). And it is almost ready database request!


Read more »


Verification of the theory of six handshakes

2 years, 11 months ago

Read more »


PyLOO — Library for reports generation, accounts of Libre/Open Office on Python

2 years, 11 months ago
Occasionally there are tasks when there is a need to create reports and other documents. In my practice this task arose more than once.

Projects in which there was this task:

  • warehouse accounting
  • accounting of real estate objects
  • document flow

My first experience was with MS Office, but there was a need for development of cross-platform solutions later, therefore the choice fell on OpenOffice. Since it was in most cases necessary to create tables, OpenOffice Calc was selected.

Read more »


Selenium for Python. Chapter 5. Expectations

2 years, 11 months ago
Continuation of transfer of unofficial documentation of Selenium for Python. The original can be found here.

Contents:


1. Installation
2. First steps
3. Navigation
4. Search of elements
5. Expectations
6. Objects of the Page
7. WebDriver API
8. Application: Frequently asked questions

5. Expectations


Today the majority of web applications use AJAX technologies. When the page is loaded in the browser, elements on this page can be loaded with different time intervals. It complicates search of elements if the element is not present at DOM, there is ElementNotVisibleException exception. Using expectations, we can solve this problem. Waiting gives a certain time interval between the made actions — search of an element or any other operation with an element.

Read more »


As I debugged python httplib and httplib2

2 years, 11 months ago
It was required to me at itself in the project once to implement work with file storage with use of HTTP REST API. The project is developed on python, besides the http-client with use of httplib2 library therefore it was decided to expand functionality of the http-client was already implemented and to work with file storage through a touch library. The problem arose when loading files on the server. The first PUT request is executed, further all subsequent requests refuse to be executed — 500 Internal Server Error.

I look Wireshark'om it becomes clear that after the first request the server sends in headings of the answer connection: keep-alive and a trace in 5 seconds closes connection. Everything is simple — it the timeout of keep-alive is set on the server.



And here is how it looks on the client:

Read more »


We lift the difficult project on Django with use of Docker

2 years, 11 months ago
Good afternoon, colleagues.

Today I will tell about not absolutely simple concept fast (to an hour after several trainings) expansions of the project for work of the command consisting at least from separate a frontend and a backend of developers.

Basic data at us such: development of the project in which "the thin backend" is planned begins. I.e. bek at us executing controller role consists of the cached pages (are rendered by any shablonizator), volume models with the accompanying logic (ORM) and REST API. Actually, View in such system is reduced and taken out in JS, the benefit is different reakta, angulyara and other things which allow frontendshchik to consider themselves as "white people".

Read more »