GC [pt 3]

All

Intro

Again learning about Garbage collection, I found about ZGC collector. A quite interesting concurrent region-based collector that aims for 10ms or less pause times.

ZGC

Experimental with JDK  11. The Oracle developers video.

Goals

The aim is to keep it easy to tune.

Multi-terabyte heaps

15% throughput reduction

10ms pause time

   >> Pause times do not increase with heap or live-set size

   >> Pause times do increase with root-set size

 

Enabling  THP

For use THP on ZGC, we must enable using -XX:+UseTransparentHugePages

REFs

   inline

Spoiler – Another Flaw in Intel processers

All

Intro

Similar to Spectre and Meltdown( and Foreshadow), that were discovered in Jan 2018, now it was found another Spoiler. The original paper is that describes Spoiler here

Spoiler leakage

This flaw attacks Memory Order Buffer  – MOB and provides sandbox environment access, including JavaScript.

 

According to the original paper

Knowledge of the physical address enables adversaries to bypass OS protections [25] and ease other microarchitectural] attacks [31]. For instance, the procfs filesystem exposes
physical addresses [31], and Huge pages allocate contiguous physical memory

 

I will read more about it and I will post my findings.

REFS

https://www.zdnet.com/article/all-intel-chips-open-to-new-spoiler-non-spectre-attack-dont-expect-a-quick-fix/

Shenandoah GC

All

Intro

Reviewing here about Concurrent GC’s and GC’s strategy, I saw about Shenandoah GC quite interesting topic. The phases are basically related to marking the regions. Basically in summary:

Concurrent → the application runs together with the GC
Regional based GC – Forwarding pointers enable Shenandoah to collect each region independently without remembered sets.
Not generational based (this means the log does not have Young, Tenure, Old) → don’t look for young gen on the logs and .
Shenandoah compacts concurrently –> CMS :/

Phases
Operates in 3 or 2 concurrent phases (deprecated traversal operates in one concurrent mode)

OpenJDK versions

In regards to the OpenJDK versions, it can be used in OpenJDK 1.8 & OpenJDK 11 & OpenJDK 17 (update 2021). Targeting for pauses below 10ms (Similar to ZGC Oracle – soft goal).

Shenandoah GC

Basically, Shenandoah is about regional collection – if this sentence makes sense.  It has 9 phases but also 5 heuristics. The paper can be accessed here (paper describes Shenandoah1 not Shenandoah2).

The usage is pretty simple: just use the flag: XX:+UseShenandoahGC

The phases are enumerated below: 

   Concurrent Marking

   Concurrent Evacuation

   Concurrent Update References

Failure Modes

Basically Shenandoah is a run to clean more memory than the application is generating – a concurrent GC. But sometime you start to lose this race, so then first you start to clean everything but still with the threads running, and if still does not work, so then you stop everything to clean the heap. So then Pacing –> Degenerative –> Full GC/STW.

Pacing: First it will pace the application allocation →  up to a certain ShenandoahPacingMaxDelay (default max is 10ms)

Degenerative GC ~ STW occurs together with the concurrent cycle. It can turn to Full GC if the concurrent gc do not happen (if a failure is detected after some phase) → yes 

Full GC/ STW – Finally as the last resource it can be used to avoid an OOME – which can be the case for ZGC. Stop everything, including the concurrent threads, and clean the heap.

Heuristics:

Basically will define when the GC should happen, as in the heuristic will decide to start GC cycle on certain heap occupancy.

Static heuristic ~ First then if we set a goal, as in a hard set percentage, like 50%, and stick with that . But this can be too pessimistic, meaning you clean too much in advance from the actual application usage  == heap occupancy.

Adaptive heuristic ~ still used ~ It sets some boundaries but adapts according to the application usage of the memory. There are three options in case the application is filling faster than cleaning.

The heuristics can be used as:

-XX:ShenandoahGCHeuristics=<name>

And tells the GC how to actually start the mechanism. They are: adaptative, static, compact, passive and aggressive.

Reddit thread

Shenandoah Visualizer

It is possible to use the Shenandoah visualizer tool to understand more about it.

REFs

Youtube Video from Christine Flood

Httpd

All

Intro

The apache http server – d standands for daemon, i.e. runs in the process.

httpd

Apachectl is actually used to listen to the http requests and responds to them, basically a daemon as any other in Linux.

The server guide can be found here

Apache changes a lot man – Vj

    httpd.conf

It’s a bit confusing because that httpd.conf was removed.

    magic file

Interesting that there is a file called magic: /etc/apache2/magic

 

Heap dumps in JDK 11

All

Intro

For generating heap dumps on JDK 11 is pretty straight forward, following the previous versions – just add the flag when calling java:

    -XX:+HeapDumpOnOutOfMemoryError  

Generating

[@your_computer] java -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/path/JDK11/java_application

Generated file

The generated file will be java_pid[number].hprof

Tests

For testing, I used the scripts from here

Actually, the second example is pretty straight forward –  Error 2 – GC Overhead limit exceeded 

// Java program to illustrate
// GC Overhead limit exceeded
import java.util.*;
 
public class Wrapper {
public static void main(String args[]) throws Exception
    {
        Map m = new HashMap();
        m = System.getProperties();
        Random r = new Random();
        while (true) {
            m.put(r.nextInt(), "randomValue");
        }
    }
}
  But I didn’t run with his suggestion and instead, run it without it. Just to see how long it would take. I regretted because the file is huge!

REFs

  Scripts

Selenium

All

Intro

It’s so interesting how much stuff it’s possible to be done just using some python script and libs.

I was doing some small stuff with Selenium here and it’s very useful actually, just play around with automating web browsing stuff.

This site brings a neat tutorial on this case

Selenium

    Webdriver

It is this API to make the navigation easy:

 WebDriver driver = new FirefoxDriver();

So then you can use, as Colin did, find_element_by_id directly, like javascript.

browser.find_element_by_id('search_form_input_homepage')

    Bottom click

next_button = [e for e in browser.find_elements_by_class_name('item-page')
                   if e.text.lower().find('next') > -1]
>>> next_button.click()

REFs

  Colin’s tutorial