Tools

The Simian Army

All, Suggestion, Tools

I remember once in an interview I was asked in how could I improve the system that there were developing in terms of security and performance.

The answer was quite fast, I would not reinvent the wheel but rather use Doctor Monkey from the Simian Army.

I was quite surprise that actually they did not know what was the Simian Army and how Netflix was using it to assure Amazon services and the cloud services were actually working. THEY DIDN’T KNOW! HOW COME RIGHT?

Basically the tool was I talking about use several external tools for health checks in the instances.

The tools are an analogy to a simian being released from a cage, this analogy is specially interesting as the Chaos Monkey, this tools is very interesting. It actually disable some random instances to verify how the system would react!!

Netflix thanks for that!!

 

[1] https://medium.com/netflix-techblog/the-netflix-simian-army-16e57fbab116

[2] https://github.com/Netflix/SimianArmy/wiki

Distributed Tracing

All, Tools

Although my master thesis was in local tracing, e.g. Lttng and dtrace, there are very powerful tools called end-to-end tracers that basically work tracing a request from top to bottom in distributed systems. In other words, it allows to log information of distributed system.

The example in [3] is the wikipedia, but any major platform nowadays is distributed in several places and could potentially use this tool.

An example of end-to-end tracer is X-trace[1], which was developed by Rodrigo Fonseca, and presented at his PhD Thesis and it’s available in his github account [2].

Currently there are several tools, in this so called end-to-end tracers but also techniques and tools to complement this, e.g. Pivot tracing technique.

This is clearly a hot-topic and has received for attention from several companies, e.g. Google and others.

I will be posting more about end-to-end tracers soon.

[1] http://xtrace.sourceforge.net/

[2] https://github.com/rfonseca

 

 

 

Root Framework

All, Tools

Another framework that is very interesting and powerful, although not very used beyond the physicits and the Cern people is the ROOT Framework [1].

I have the experience to play with it, using its functions to generate meaningful results.

Root is very optimized, written in C++ but nowadays you can used it combined with python and R.

I personally don think its interface is easy, as other libraries/frameworks, but its optimized and the community is great. People from several backgrounds helping specific questions.

At Instituto de Fisica in USP University, we used it a lot. But not rarely we faced situations that it played some trick answers on us, e.g. some of the results were inconsistents.  But anyways.

It is interesting to try it straight on the browser too.

Is always important to watch CERN and its tools, since they become a tendency in the world, the WWW is an CLEAR example.

[1] https://root.cern.ch/

Qt

All, Suggestion, Tools

Well, during my graduate studies, for developing the C/C++ code I started using Qt and it was the best thing I did.

It is very straightforward, simple and nice. Much less complex than Eclipse Framework, which I also used.

Formally speaking, Qt is this cross-platform framework for graphical user interfaces and applications. It has everything embedded on it and it’s developed by the Qt Company, which I was surprise to learn that was based in Finland.

Qt also provides simple UI interfaces, from which I’m creating the MazelTov framework for testing and analysing performance of open software projects.

I forgot to mention, of course, Qt is also the library that combines several utilities for developers and embedded functions. Qt library is the thing!

 

[pt]

Durante meus estudos de pós-graduação, para desenvolver o código C / C ++, comecei a usar o Qt e foi a melhor coisa que fiz. É muito simples, simples e agradável. Muito menos complexo que o Eclipse Framework, que eu também usei. Formalmente falando, Qt é essa framework cross-platform para interfaces gráficas de usuário e aplicativos. Tem tudo incorporado e é desenvolvido pela Qt Company, que fiquei surpreso ao saber que foi baseado na Finlândia. O Qt também fornece interfaces de interface do usuário simples, a partir das quais eu estou criando a estrutura do MazelTov para testar e analisar o desempenho de projetos de software aberto. Eu esqueci de mencionar, é claro, o Qt também é a biblioteca que combina vários utilitários para desenvolvedores e funções embutidas. Qt biblioteca é a oq ha!

O que é Tracing?

All, Tools

Introducao basica – o que eh tracing?

Gravacao de eventos .

Nao eh logging, pois esta ligado a eventos em baixo nivel e muito mais rapidos nao sendo tambem profiling 

Pode ser no lado em eventos de usuario e eventos do kernel.  A gravacao eh feita em runtime e portanto ha uma interferencia, que é chamada de overhead. O numero de eventos gravados tambem varia, ocorrendo por vezes perda de alguns eventos a depender da ferramenta de tracing.

     Basicamente o que deve ser feito?

Rodar o aplicativo, ligar o trace e fazer a analise.

Ferramentas

LTTNG

Eh um tracing dos eventos do kernel e dominio usuario com baixo overhead.

Eh possivel analisar apenas alguns eventos do kernel ou todos os eventos, assim como eh possivel fazer a analise de eventos padrao no kernel ou criar pontos de tracing no proprio codigo (aplicativo) para fazer a gravacao.

Para fazer a analise dos traces, eh necessaria a ferramenta de analise dos traces, que pode ser o BabelTrace, ou o TraceCompass. É desenvolvido pela EfficiOS.

Dtrace

É um framework para analises de performance e deteccao de erros que eh incluso por padrao em diversos sistemas operacionais, incluindo Solaris, Mac OS X and FreeBSD.

perf

É uma ferramenta para profiling para Linux 2.6+. Um interessante tutorial pode ser encontrado aqui: https://perf.wiki.kernel.org/index.php/Tutorial

strace

É uma ferramenta para debbuing e diagnostico Pode alterar sua aplicacao a medida que usa ptrace no kernel, tendo um overhead grande.

   ftrace

É um framework para tracing em Linux desenvolvido por Steven Rostedt e que foi incorporado ao kernel do Linux.