2 years, 11 months ago
The field of activity of our company extends far beyond game development. In parallel with it we keep tens of internal projects, and Player Relationship Management Platform (PRMP) – one of the most ambitious.
Player Relationship Management Platform (PRMP) – special system which by means of the analysis of data bulks in real time allows to personify interaction with the player through the recommendations arriving to the user proceeding from a context of its last play experience.
PRMP allows our players to derive more pleasure from game, improves their user experience, and also relieves of viewing unnecessary advertizing and promo-messages.
In this article I will tell about Twitter Scalding – a framework for data handling process description in Apache Hadoop. I will begin from far away, with history of frameworks over Hadoop. Then I will give the overview of opportunities Scalding. In end I will show the code samples available to understanding to that who knows Java, but is almost not familiar with Scala.
2 years, 11 months ago
I decided to tighten the English here. In particular, there was a wish to expand a lexicon considerably. I know that there is a mass of programs which in a game form help to make it. The hitch is that I do not love a geymfikation. I prefer in the old manner. A paper leaflet where the table with words, a transcription and transfer. Also we teach it we learn. Also we check the knowledge, for example, closing a column with transfer. Generally, as I learned it at university.
I heard about the fact that there are 3000 most often used loss for words which is been on OxfordDictionary the website. Here this list of words: www.oxfordlearnersdictionaries.com/wordlist/english/oxford3000/Oxford3000_A-B And transfer into Russian I decided to take from here: www.translate.ru/dictionary/en-ru only One problem, all to be on these websites well at all not in that format which can be printed and learned. As a result the idea all this was born to program. But to make it not as consecutive algorithm, and to parallelize everything. What pumping out and parsing of all words occupied not (3000 words * 2 websites) / 60 seconds = 100 minutes. It if to give on 1 second on pumping out and rasparsivany pages for extraction of transfer and a transcription (in reality I think it by 3 times longer until we open connection until we close both td and tp).
How to force Parboiled to work even quicker? What mistakes it is better not to make? What to do with inheritance in the form of Parboiled1? On these, and also other questions the concluding article of a series is urged to answer.
In this article we will construct the parser for already described by us before a format of configuration files. Also we implement small DSL for the simplified access to elements got a tree. From this article you learn about types of rules, operations of the parser, and also about "dark matter" Parboiled — a stack of values.
In the second part of a cycle we will talk about the basic rules of comparison of characters in Parboiled. We will not concern all rules — for this purpose there is documentation, I only want that you felt confident with the basic syntax of rules used in Parboiled.
For fixing of knowledge we will write the simple recognizer for simple grammar. The recognizer (recognizer), but not the full-fledged parser as it will only compare the input text with the rules described by us (who are also called by produktion), but will not retrieve any values from the compared text. The recognizer can be useful and in itself as can work as the validator: if the input was incorrect, the recognizer will let about it know and will tell what went not and where. And absolutely cool our recognizer will become when we learn how to retrieve the sorted values and and here some "value stack". Well, went?
Today, in the light of rapid growth of popularity of functional programming languages, schemers of parsers — the tools facilitating analysis of the text by the mere mortal even more often find for themselves application. It is already good to prove such libraries as Parsec (Haskell) and Planck (OCaml) in the ecosystems. Their convenience and opportunities pushed the creator of the language Scala, Martina Oder in due time, to enter to standard library their analog — Scala Parser Combinators (are taken nowadays out in scala-modules), and knowledge and ability to use similar tools — to carry to obligatory requirements to Scala-developers of the A3 level.
This series of articles is devoted to Parboiled library — strong alternative and possible replacement of Scala Parser Combinators. In it we will in detail consider work with the current version of library — Parboiled2, and also we will pay attention to Parboiled1 as the most part of the existing code still uses it.
Few months ago I undertook the obligation for self-education. There is in other offices such subject — the employee, time in half a year is revyyuit and speak "give to the following revyyu you will study Spring, patterns (what?) and functional programming!" The idea would be quite good if the purpose was not set it is so indistinct. Let's pass Spring so far and patterns — at the weekend I rushed to attack to FP.
I will begin with simple exercises on Scala — I solved. Selected Scala as the main working tool of Java we — well had still Clojure, Groovy and Java8 options (something else?) — but with them perhaps I will try then.
Set to herself the objects (and whether correctly I understood FP?):
To solve problems in functional style
I.e. whenever possible not to use explicit cycles and branchings
And also to avoid mutable collections, etc.
One exercises turned out easily, others to put it mildly not really. Now I will try to tell in brief about they are to arrange new knowledge. It is hard to say whether this article can help in the future someone or more likely someone will help me, having pointed to errors or having offered improvements.
3 years agoNote of the translator. In future Scala version ("Don Giovanni") support of types of consolidation (union types) is announced. Miles Sabin widely known in narrow circles as the creator of Shapeless, shows in this article of 2011 how to create consolidation types already now.
Scala has very expressive type system. However it does not include (at least as primitives) all desired elements. There are several really useful types falling under this category are types of polymorphic functions of the highest rank (higher-rank) and recursive structural types. But I will tell about them in more detail in the following posts, and today I am going to show you as in Scala we can create types of consolidation (union types). During an explanation I will shed a little light on Carry-Howard's isomorphism and I will show how to use it in our purposes.
3 years, 2 months ago
In Scala, in addition to normal accessibility modifier of private there is also private [this] modifier. These two modifiers are quite similar at each other. Besides in Java there is only simple private. Therefore they can easily cause confusion or conviction that simple private — not such and private, and everywhere it is necessary to use private[this] for security dense forest. But let's understand, how are you are actually.