Program in pep8 not in python. That’s a lesson I’ve recently learned.

First of pep8 is the a standard for writing python code [1], it is actually very useful and used. But my review here it is just to pinpoint that for me it was learning a new thing. I might actually say that learning python was a new thing, after learning c/c++/java but learning pep8 is the next step.

Examples:

# Aligned with opening delimiter.
foo = long_function_name(var_one, var_two,
                         var_three, var_four)

My highlights, several standards are really useful but it took me time to learn it. The import rules sometimes it’s annoy though, if you have several importings in different lines. !:

They follow Donal Knuth suggestion for operator simplification! hehe! DK hehe:

income = (gross_wages
          + taxable_interest
          + (dividends - qualified_dividends)
          - ira_deduction
          - student_loan_interest)

Yes, use python, yes use pep8. It might be annoying in the begin but it worth.

[1] https://www.python.org/dev/peps/pep-0008/

 

Since I learned English and live in a French neighbourhood, nothing better than learning my 4th language. Also, Austria and Germany are specially nice, so one more reason.

In the beginning was extremely difficult I couldn’t understand much, but if you practive every day for several months, you will start understand some stuff.

There are some tricks to learn fast though:

> Learning everyday

> Find reasons to learn it, e.g. songs you like

> Watch you favourite movies in the new language

> Listen to songs several times

The last tips is better if you have a good song, and CRO, the German rapper, or ropper, is my suggestion today.  He always use a Panda mask and is approx 1year older than me, I was born in 91 and him in 90. Both in January.

So my tips of the day are those, please when you have time, listen to Einmal um die Welt, by CRO. The song makes you want to dance, almost instantaneously.

To compare stack traces of a program Brendan Gregg suggested the so called Differential Flamegraph, but actually it might no be so straight forward to use it.

Brendan Gregg diff flamegraph

On his diff alg, Brendan G, the difference of the two profiles is highlighted using a color scheme. Basically, it takes the first profile and put the colors using a delta from the second profile! This delta might cause an issue for comparing since we would need to wonder which one of the frames is which.

TraceCompare Flamegraph

Francois Doray’s TraceCompare developed another differential flamegraph comparing the first and the second groups in the javascript.

When faced with the same problem, he talks about two solutions: use 3 flamegraphs (input 1, input 2 and output) or use a new black/white solution, which adds a region for missing code paths in white. Basically highlighting a Venn diagram solution of the result would be the first tree, and inside of it the difference of the second.

A problem with this approach is that some code paths present in the A profile may be missing entirely in the B profile, and so will be missing from the final visualization. This could be misleading.

Differential flame graph used in my masters implementation

To simplify the question I developed a diff flamegraph that basically used three colors for each stack using the average of it.

Reference

[1] http://cacm.acm.org/magazines/2016/6/202665-the-flame-graph/fulltext#R2

 

Small Context

During my final months at the Dorsal Lab, in Montreal, I started co-supervising Gabriel’s work in Anhembi Morumbi [1]. After writing some lines of code in python, we actually came to the conclusion that the scripts could be adopted for several open source projects and then we combined all the ideas in the MazelTov framework. Part of the work is not implemented yet but we have the drafts summarized and several scripts in git already.

TCC

From all the MazelTov framework, we came to the conclusion that it would be too big for a undergrad project so we divided in phases and the first was actually the one we presented as his undergrad thesis.

The so called TCC, aka honor thesis, was presented at Anhembi with excitement among the judges, but for us was just part of the main idea to develop a framework combining several performance tools/methods that we actually worked before.

The slides of his thesis can be found here:

MazelTov Framework

The framework is intended to be compiled in scons [cof cof], and integrated to other tools such as Lttng. The idea is to have an independent tool for performance comparison using the methods that I developed in my master, but with other scripts as Differential FlameGraph. It is in the stack but will be delayed for some papers we have been working on recently.

I remember discussing with a mathematician that couldn’t believe there was another regression besides all the methods he knew!!  So nice guy! But yeah, regression and software regression are different subjects (but they cross paths sometimes).

Software Regression

Broadly speaking a performance regression occurs when a later version is slower than the previous one. So the software performed, in terms of the time process, worst in the latest version.

Methods to detect it

Well, it is possible to use several methods to detect performance regression but one we can highlight is the use of t-test, in a scalable way to measure one version to another the performance difference. The problem with this technique is obviously the need for some method to validate the scale without carrying error in several comparisons.

Google, for example, uses this. I’m pretty sure more companies do to on their test suites and integration tests.

KW

I and Abderrahmane applied some techniques such as KW to find regressions and found that actually this technique can be used for performance regression comparison. It might be even better than other tools such as direct comparison, i.e. t-tests. The results were not published yet.

 

 

Anybody can learn!

Teaching using Code.org

I was able to teach kinds in private and in a NGO organization back in Brazil using Code.org in 2015 and I cannot complain on the opposite actually. I think it’s a very powerful and fun learning tool for kids, k-12 up to 15 or 16 years.

My conclusions

My conclusions in this matter were compiled and presented in the conference ECGBL 2017 in Graz, Austria. I might add actually that my main conclusion was that for less skilled kids, motivation was a key aspect of the learning process to deal with the frustations of learning.

Framework

In my next works in education I will certainly talk more about motivation and frustration. How frustration/expectation and motivation makes a key in the learning curve of kids and how to improve that, so that’s why I developed the framework.

My second conclusion is to follow the flow, if the kids learning a competitive or cooperative manner, the teacher should just follow the flow instead of going agains it.

The slides can be found here [2]

Results

Nowadays the platform is actually much more interesting for kids and the additional features they added were awesome. The several parts added, like Minecraft clearly makes a different to teach kids. I might add that Code.org improved so much on those last years. Minecraft part, the drawing and the small challanges were the parts the studesnts most liked.

Citation

de Melo Jr, I. F., Mendes, G. A. R., & de Paula, R. (2017, October). A Game Framework to Teach Computer Science Concepts. In European Conference on Games Based Learning (pp. 138-147). Academic Conferences International Limited.

REFs

[1] Code.org

[2] https://www.slideshare.net/fjdemelo/comparative-framework-for-education

 

Problem

Lttng is the best trace by far.

But I remember doing the performance tests with some open source codes, using the Lttng tracer to record performance metrics, by enabling perf event counters. However, when we enabled three perf counters, lttng would crash!!

But why we needed more than two perf events enable at the same time

Well, the answer is simple, to make sure that it wouldn’t be multiple influences, i.e. noise, in the measures. We make sure that that run had that event with that value. For our experiments with several runs, this is very important.

Solution

The solution that we did was to use a cross-validation scheme in which we would measure pairs for performance metrics in order to avoid this crash and than compare is in a table. Not was as Apriori algorithm, but more like a cross-validation table.

Intro (Play-by-play)

I remember while working at AllApps, in Sao Paulo Brazil we implemented the system to develop software (mobile apps) from the time the client enter to the time we deliver the software. The draw of the whole process took several weeks and actually was very interesting to me after all.

Scrum Master

Although it was my first job after the bachelor, it was also my first job as Scrum Master. I might add, it was all new to me. People reporting you what they have done is something very interesting and

Lesson

In the end we did not implemented it by-the book, but it was good to have done it. For our small group, we didn’t need 5 minutes everyday to know what each grouop member was doing. Just a quick chat or email was enough to know if the person accomplised the issues of the previous day. Also everybody working in the same room we could see the progress ourselves.

The computers distributions, I mean people distribution makes comunication easy. It is very naively draw below.

_________________________________
| p1                   p2                      p3|

|                                                       |

|                  TABLE                         |

| p4                                              p5|

|                                                        |

_______________[BOSS1 & BOSS2]

Certification

Anyways, for certification I recommend [1].

[1] https://www.scrumalliance.org

Another topic which I studied during my Bachelor, with prof Airton Deppman, was Random number generation. At that time I was quite impressed by the fact that a computer cannot generate a sequence of total random numbers, aka pseudo random generation.

So kids, do not trust those sites the promise REAL random generation cause they might be using slaves to roll the dices.

Break down MegaSena

But I faced this problem of random number generation when I first created a Program to Break “MegaSena”! My break MegaSena was a simple counting c program that generated a billion numbers “randomly” count the number of appearances and display to me the histogram and 6 numbers so I could win the MegaSena.

It was bizarre when the numbers were always 1,2,3,4,5 and 7!! or 6!! How come? So I increased the number of generated numbers for 10 billions, and after some hours the discrepancy was even lower!

So yeah, coming back to the random numbers generation, I found that Mersenne Twister was the best generator and it is the default for several languages, as C++11, MatLab, Excel and Julia.

Details of the algorithm can be found here[1] but basically it was developed in 97 by Makoto Matsumoto (松本 眞) and Takuji Nishimura(西村 拓士).

[1] https://en.wikipedia.org/wiki/Mersenne_Twister

Intro

I started my Bachelor in 2009, after 6 months studying Chemistry at Oswaldo Cruz. At Mackenzie, I learned many things, but I basically became an adult. In 2012 I received a scholarship from the Federal Government to do an exchange program to Canada, in Halifax, Nova Scotia at Saint Mary’s University.

I’m very glad I came to Canada, it’s an amazing country, a land of opportunities! After the exchange we were suppose to come back, for contractual reasons, so that’s what I did. I went back and finished my Honour Thesis, which was based on some Psy tests, the list is below:

Tests

The selected tests were: PVT test, Ebbinghaus test and Stroop effect.  There were actually many tests we could implement, but those seem the most famous ones.

Result

So to do it, I grabbed some materials I had from the work with Prof Jason Ivanoff, I compiled it and created an mobile application for Android and iOS. The app in Android can be found here [1]. By the way, I miss prof Jason, he was really really nice! He used to say Canadians save their emotions to emoticons 😀 haha!

Presentation

Below is my bachelor presentation, which was presented in December 2014.

App Android and iOS

The Android App can be found [1] it is called PVT test and brings Decartes. It was developed using Eclipse and uses MVC model. The process to build it Waterfall Model. I got some help of Jovian Trotman, who taught me the basic lines in Android programming.

The iOS app was developed by me with some help of Leao Bicalho, a developer from the north part of Brazil.  The iOS used the  COCOS2D library and the app is composed of a main menu with the three tests.

Source Code

The source codes can be found in my GitHub account[1], together with all the content I have i.e. reports in Portuguese/English and presentation.

[1] : https://github.com/FranciscoMeloJr/TCC