All

Design Doc

All

Intro

I remember starting developing my own tools, like some python scripts like Mazeltov and stop with the issues. So this time, for Log Helper I have created a Design Doc, following what Techled suggested – btw, he is very funny.

Design Doc

There are many good templates for Design Doc, and Angela Zung is able to describe here very well some details about it – though I might disagree on the comedy part though.

I think this model here is very to the point, but seems almost like a read.me kind of thing.

This reference here, from Klariti.com, is the one I’d think it’s the complete solution. Too big for a small script as Log Helper though.

Remember that:  The design doc is written as part of the development of the software design

REFs

inline

GC [pt 3]

All

Intro

Again learning about Garbage collection, I found about ZGC collector. A quite interesting concurrent region-based collector that aims for 10ms or less pause times.

ZGC

Experimental with JDK  11. The Oracle developers video.

Goals

The aim is to keep it easy to tune.

Multi-terabyte heaps

15% throughput reduction

10ms pause time

   >> Pause times do not increase with heap or live-set size

   >> Pause times do increase with root-set size

 

Enabling  THP

For use THP on ZGC, we must enable using -XX:+UseTransparentHugePages

REFs

   inline

Spoiler – Another Flaw in Intel processers

All

Intro

Similar to Spectre and Meltdown( and Foreshadow), that were discovered in Jan 2018, now it was found another Spoiler. The original paper is that describes Spoiler here

Spoiler leakage

This flaw attacks Memory Order Buffer  – MOB and provides sandbox environment access, including JavaScript.

 

According to the original paper

Knowledge of the physical address enables adversaries to bypass OS protections [25] and ease other microarchitectural] attacks [31]. For instance, the procfs filesystem exposes
physical addresses [31], and Huge pages allocate contiguous physical memory

 

I will read more about it and I will post my findings.

REFS

https://www.zdnet.com/article/all-intel-chips-open-to-new-spoiler-non-spectre-attack-dont-expect-a-quick-fix/

Shenandoah GC

All

Intro

Reviewing here about Concurrent GC’s and GC’s strategy, I saw about Shenandoah GC quite interesting topic. The phases are basically related to marking the regions. Basically in summary:

Concurrent → the application runs together with the GC
Regional based GC – Forwarding pointers enable Shenandoah to collect each region independently without remembered sets.
Not generational based (this means the log does not have Young, Tenure, Old) → don’t look for young gen on the logs and .
Shenandoah compacts concurrently –> CMS :/

Phases
Operates in 3 or 2 concurrent phases (deprecated traversal operates in one concurrent mode)

OpenJDK versions

In regards to the OpenJDK versions, it can be used in OpenJDK 1.8 & OpenJDK 11 & OpenJDK 17 (update 2021). Targeting for pauses below 10ms (Similar to ZGC Oracle – soft goal).

Shenandoah GC

Basically, Shenandoah is about regional collection – if this sentence makes sense.  It has 9 phases but also 5 heuristics. The paper can be accessed here (paper describes Shenandoah1 not Shenandoah2).

The usage is pretty simple: just use the flag: XX:+UseShenandoahGC

The phases are enumerated below: 

   Concurrent Marking

   Concurrent Evacuation

   Concurrent Update References

Failure Modes

Basically Shenandoah is a run to clean more memory than the application is generating – a concurrent GC. But sometime you start to lose this race, so then first you start to clean everything but still with the threads running, and if still does not work, so then you stop everything to clean the heap. So then Pacing –> Degenerative –> Full GC/STW.

Pacing: First it will pace the application allocation →  up to a certain ShenandoahPacingMaxDelay (default max is 10ms)

Degenerative GC ~ STW occurs together with the concurrent cycle. It can turn to Full GC if the concurrent gc do not happen (if a failure is detected after some phase) → yes 

Full GC/ STW – Finally as the last resource it can be used to avoid an OOME – which can be the case for ZGC. Stop everything, including the concurrent threads, and clean the heap.

Heuristics:

Basically will define when the GC should happen, as in the heuristic will decide to start GC cycle on certain heap occupancy.

Static heuristic ~ First then if we set a goal, as in a hard set percentage, like 50%, and stick with that . But this can be too pessimistic, meaning you clean too much in advance from the actual application usage  == heap occupancy.

Adaptive heuristic ~ still used ~ It sets some boundaries but adapts according to the application usage of the memory. There are three options in case the application is filling faster than cleaning.

The heuristics can be used as:

-XX:ShenandoahGCHeuristics=<name>

And tells the GC how to actually start the mechanism. They are: adaptative, static, compact, passive and aggressive.

Reddit thread

Shenandoah: The Garbage Collector That Could by Aleksey Shipilev from java

Shenandoah Visualizer

It is possible to use the Shenandoah visualizer tool to understand more about it.

REFs

Youtube Video from Christine Flood

Httpd

All

Intro

The apache http server – d standands for daemon, i.e. runs in the process.

httpd

Apachectl is actually used to listen to the http requests and responds to them, basically a daemon as any other in Linux.

The server guide can be found here

Apache changes a lot man – Vj

    httpd.conf

It’s a bit confusing because that httpd.conf was removed.

    magic file

Interesting that there is a file called magic: /etc/apache2/magic

 

Heap dumps in JDK 11

All

Introduction

OutOfMemoryError is an exception handled by the JVM and usually means the application is not properly sized or it is taking more than what its memory intended size. It can be direct development team issues, sometimes an actual leak (in native or heap) or an issue on the deployment with an undersized heap. Regardless of how it came up, OOME should be addressed.

Automatic generation For automatically generating heap dumps on JDK 11, in case of OOME, is pretty straight forward, following the previous versions – just add the flag when calling java – and when the OOME happens the heap dump will be automatically generated:

    -XX:+HeapDumpOnOutOfMemoryError  

This is interesting given some container have ExitOnOutOfMemoryerror flag, so on OOME event, the JVM will exist and most of the times we will see the pod terminating code153 or code 0 – given only one container per pod so the container just exists.

Manual generation: given jcmd avaliable one can generate a heap dump or VM.info as well – both very useful for heap and native investigations respectively.

Generating

$ java -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/path/JDK11/java_application

Generated file

The generated file will be java_pid[number].hprof – hprof is a type of file. For its analysis I most of the times use Eclipse MTA – pretty straightforward opening the file. I think I have blog posts about querying the data from the heap and completing the analysis.

Testing

For testing, I used the scripts from here – but be aware that some tests have perm generation, which is deprecated on JDK 8 forward, now it is metaspace, so native memory.

Actually, the second example is pretty straight forward –  Error 2 – GC Overhead limit exceeded

// Java program to illustrate
// GC Overhead limit exceeded
import java.util.*;
 
public class Wrapper {
public static void main(String args[]) throws Exception
    {
        Map m = new HashMap();
        m = System.getProperties();
        Random r = new Random();
        while (true) {
            m.put(r.nextInt(), "randomValue");
        }
    }
}

But I didn’t run with his suggestion and instead, run it without it. Just to see how long it would take. I regretted because the file is huge, given the hprof was a photochart of the entire jvm heap memory at the time is generated.

References

  Scripts

Selenium

All

Intro

It’s so interesting how much stuff it’s possible to be done just using some python script and libs.

I was doing some small stuff with Selenium here and it’s very useful actually, just play around with automating web browsing stuff.

This site brings a neat tutorial on this case

Selenium

    Webdriver

It is this API to make the navigation easy:

 WebDriver driver = new FirefoxDriver();

So then you can use, as Colin did, find_element_by_id directly, like javascript.

browser.find_element_by_id('search_form_input_homepage')

    Bottom click

next_button = [e for e in browser.find_elements_by_class_name('item-page')
                   if e.text.lower().find('next') > -1]
>>> next_button.click()

REFs

  Colin’s tutorial