Our engineers recently gathered to discuss a topic that's becoming increasingly urgent: sovereign cloud infrastructure. Our conversation covered cloud solutions in Europe, the Netherlands' digital independence, and our current reliance on American cloud providers. This blog post shares the key insights from that meeting and outlines our planned next steps.

Our discussion began with an observation that's hard to ignore: media coverage consistently focuses on the "big three" American cloud providers - Microsoft Azure, Amazon Web Services, and Google Cloud. While these platforms dominate the market, recent geopolitical developments have raised important questions about Europe's digital sovereignty.

The re-election of Donald Trump has introduced new tensions in European-American relations. While the United States remains a valued ally, our almost exclusive dependence on American cloud infrastructure is becoming a point of concern for many organizations and governments across Europe.

A Real-World Example of Digital Dependency

This concern moved from theoretical to concrete when Trump's administration took action against a Chief Prosecutor at the International Criminal Court in The Hague, blocking access to email services. This was possible because the ICC relies on Office 365 - a Microsoft service subject to American jurisdiction and policy decisions.

The incident illustrates a troubling reality: our digital infrastructure can be disrupted by foreign policy disagreements, even when those disagreements don't directly involve us. When an ally can unilaterally disable critical services we depend on, it raises serious questions about our digital autonomy.

We're not alone in recognizing this challenge. The website european-alternatives.eu, launched in December 2024, has seen a significant surge in traffic (as shown in image below), indicating growing interest in European cloud alternatives. Additionally, cybersecurity expert Bert Hubert a former AIVD supervisor and employee - has repeatedly called on the Dutch government to address this digital dependency.

From Politics to Practice: Time to Nerd Out

While geopolitics sparked our initial interest, we're technical people at heart - so we did what we do best: we rolled up our sleeves and started exploring the technical landscape.

Since we primarily use Azure (with occasional AWS and Google Cloud deployments), we first needed to identify which functionalities are essential to our operations. Given our DevOps team's focus on Kubernetes, the answer became clearer. Beyond container orchestration, we also rely on Azure Functions and various data services. And lastly our CTO still has that Java applet running weekly for €0.01 per month.

After our discussion, we identified these core cloud platform functions:

Networking (public IP addresses, routing, firewalls, load balancing)

Using european-alternatives.eu as our starting point, we found 12 alternative providers listed at the time of writing - a number that continues to grow. Three stood out immediately:

OVH Cloud: Hosted in France and trusted by the French government, now available to public companies. Their established presence and government backing caught our attention.

Stackit: Primarily hosted in Germany and owned by the Schwarz Group (which also owns Lidl). Sometimes playfully called "LidlCloud," this platform has serious enterprise backing.

Cyso Cloud: Based in the Netherlands and formerly known as Fuga Cloud, offering local hosting with familiar regulatory frameworks.

Hands-On Testing: Our Initial Experiments

Our colleague Noud took the lead on testing OVH Cloud's Kubernetes capabilities. The results were encouraging - deploying applications proved straightforward and user-friendly. Using Terraform, he successfully spun up a Kubernetes cluster, leveraging OVH's dedicated Terraform provider to create the necessary resources efficiently.

After deploying a simple nginx application to the cluster, everything worked as expected. The sign-up process was smooth, and deployment times were reasonable. One minor observation: the region naming conventions could be more intuitive, which would simplify initial setup and potential migrations from other providers.

Our colleague Dion began exploring Stackit, though this proved more complex initially. Unlike other providers, Stackit isn't open to the general public - it originated as an internal cloud solution for the Schwarz Group's companies, including Lidl and Kaufland.

Dion connected with Stackit engineers at KubeCon London in April, and we received access just after our session was held. The platform shows significant promise, built on OpenStack for compute, storage, and Kubernetes infrastructure. Their enterprise heritage suggests robust, scalable solutions. We will be testing this soon.

Our CTO Bouke set himself an interesting challenge: migrating his weekly Java applet from Azure Functions to a European alternative. This seemingly simple task revealed some current limitations in the European cloud ecosystem.

While several alternatives offer script scheduling capabilities, Java support remains limited. Other providers require running scripts within Kubernetes clusters, which significantly increases costs for simple, infrequent tasks. This highlighted a potential path forward: adapting to different programming languages to better align with European providers' current offerings.

Based on our initial exploration, we've outlined several concrete next steps:

Deeper technical evaluation: We're scheduling another session in a few months, with teams assigned to thoroughly test each promising cloud provider. This will give us hands-on experience with real-world workloads.

Knowledge sharing: We're planning to make our findings public through an open session, sharing our discoveries with the broader technical community and demonstrating what's already available in the European cloud landscape.

Practical preparation: Rather than advocating for immediate wholesale migration, we're positioning ourselves to support informed decisions when the time comes.

European cloud providers are undeniably behind the "big three" American companies in terms of feature breadth and ecosystem maturity. However, the services we've examined show genuine promise for core functionalities, and some are already expanding into more specialized areas.

What European providers offer that their American counterparts cannot is something increasingly valuable: native European hosting with built-in compliance to European standards and regulations. This isn't just about data sovereignty - it's about predictable, stable access to critical infrastructure.

The current momentum is encouraging, with interesting developments happening at both the technical and governmental levels. However, we're realistic about the timeline ahead.

Conclusion: Prepared for Multiple Scenarios

We don't expect massive migrations away from American cloud providers in the immediate future. Enterprise migrations require significant time, effort, and planning. Political landscapes change, and many organizations are comfortable with their current solutions.

But as CINQ, we're preparing for various scenarios:

Customer-driven moves to European cloud solutions

Regulatory requirements that favor European hosting

Geopolitical disruptions that affect American cloud access

Continued reliance on current providers where appropriate

The key is being ready: understanding the landscape, knowing the alternatives, and having the technical expertise to advise and implement when the time comes. Whether that's next year or in five years, we'll be prepared to help our clients navigate whatever the future of cloud infrastructure brings.

Discover why sovereign cloud matters more than ever. Learn about Europe's digital independence, the risks of U.S. cloud reliance, and our hands-on tests with OVH, Stackit, and Cyso as promising European cloud alternatives.

Why sovereign cloud matters more than ever

Een van de redenen waarom ik het vak van developer zo bijzonder vind, is dat je niet alleen aan technische oplossingen werkt, maar ook de kans krijgt om te leren over onderwerpen en werelden waar je anders misschien nooit mee in aanraking zou komen. Een mooi voorbeeld hiervan is hoe belangrijk de binnenvaart is en hoe IT-systemen een cruciale rol spelen om dit mogelijk te maken.

Digitale Waterwegen: De Rol van Informatiesystemen in het Binnenvaartbeheer bij Rijkswaterstaat

Nederland staat bekend als een waterland. Met een uitgebreid netwerk van rivieren, kanalen en meren speelt de binnenvaart een onmisbare rol in onze economie en samenleving. Achter de schermen zorgt Rijkswaterstaat (RWS) ervoor dat dit complexe netwerk efficiënt, veilig en duurzaam functioneert. Binnen RWS speelt de afdeling Scheepvaartmanagement een sleutelrol in het dagelijks beheer van de binnenvaart.

Rijkswaterstaat (RWS) is een overheidsorganisatie die valt onder het Ministerie van Infrastructuur en Waterstaat. Sinds de oprichting in 1798 heeft RWS zich ontwikkeld tot een van de belangrijkste spelers op het gebied van infrastructuurbeheer in Nederland. Van snelwegen tot waterwegen, RWS zorgt ervoor dat Nederland in beweging blijft.

De afdeling Scheepvaartmanagement van RWS houdt zich bezig met het beheer van de Nederlandse vaarwegen en het waarborgen van een veilige en vlotte doorstroming van het scheepvaartverkeer. Dit omvat het monitoren van waterstanden, verkeersmanagement op rivieren, evenals onderhoud en de bediening (op afstand) van sluizen en bruggen.

Daarnaast levert de afdeling essentiële diensten zoals actuele scheepvaartberichten, scheepvaartverkeersbegeleiding en nautisch beheer. Ook biedt zij voorlichting en begeleiding aan zowel beroeps- als pleziervaart, met nadruk op veiligheid en gastvrijheid. Zo blijven de waterwegen efficiënt en toegankelijk voor alle gebruikers.

Binnenvaart: De Ruggengraat van de Nederlandse Logistiek

De binnenvaart is van groot belang voor de Nederlandse economie. Elk jaar worden miljoenen tonnen goederen vervoerd over waterwegen zoals de Rijn, Maas en Waal. Deze transportwijze is niet alleen aanzienlijk goedkoper dan wegvervoer – tot wel 75% voordeliger op langere afstanden – maar ook veel duurzamer. Waar vrachtwagens gemiddeld 60-150 gram CO₂ per tonkilometer uitstoten, ligt dit bij binnenvaartschepen tussen de 15 en 25 gram CO₂ per tonkilometer. Bovendien helpt de binnenvaart om verkeersdrukte op de wegen te verminderen, wat bijdraagt aan een efficiënter transportnetwerk.

Rijkswaterstaat maakt gebruik van meerdere, vaak in eigen beheer ontwikkelde informatiesystemen om zijn taken uit te voeren. Zonder deze systemen zouden de vele medewerkers van Rijkswaterstaat hun werk niet kunnen uitvoeren. Twee voorbeelden hiervan zijn BICS (Berichtencentrum voor de Binnenvaart) en IVS Next (Informatie- en Volgsysteem voor de Scheepvaart). Beide systemen worden intern ontwikkeld en beheerd.

BICS: Slimme Communicatie tussen Schippers en Beheerders

BICS is een digitale applicatie waarmee schippers eenvoudig en veilig reis- en ladinggegevens kunnen communiceren met vaarwegbeheerders zoals Rijkswaterstaat en havenautoriteiten. Het systeem vergemakkelijkt de uitwisseling van gestandaardiseerde gegevens en voldoet aan internationale meldplichtnormen, zoals de elektronische meldplicht voor schepen.

Elektronisch aanmelden: Schippers melden hun gegevens digitaal aan, waarna deze direct worden verwerkt in IVS Next of andere systemen.

Internationale standaardisatie: Ondersteuning voor meerdere talen (Engels, Frans, Duits en Nederlands) bevordert integratie met Europese vaarwegsystemen, met als doel een brede uitrol in Europa.

Gratis beschikbaarheid: Rijkswaterstaat biedt BICS kosteloos aan voor schippers, wat de toegankelijkheid vergroot.

Toepassing in Europa: Het systeem wordt al gebruikt langs de Rijn en wordt gefaseerd ingevoerd in andere Europese landen, zoals België en Roemenië.

Door de gegevensuitwisseling via BICS verloopt de communicatie tussen schippers en vaarwegbeheerders sneller en efficiënter, wat bijdraagt aan een veiligere en duurzamere scheepvaart.

IVS Next: Real-time Monitoring en Beheer van Scheepvaart

IVS Next is een geavanceerd informatiesysteem dat vaarwegbeheerders (sluis- en brugbedienaars en/of verkeersleiders) een actueel overzicht biedt van scheepvaartverkeer op de Nederlandse waterwegen. Het systeem, operationeel sinds 2019, verzamelt gegevens via onder andere AIS-signalen* en aanmeldingen vanuit BICS. Hierdoor kunnen schepen real-time worden gevolgd en wordt de doorstroming geoptimaliseerd.

Real-time monitoring: IVS Next biedt een actueel overzicht van beroeps- en zeevaart dankzij verplichte AIS-signalen, BICS-aanmeldingen en andere systemen.

Gegevensbeheer: Het systeem deelt scheepvaartdata met stakeholders zoals waterbeheerders en havens, waarbij privacy en vertrouwelijkheid hoog in het vaandel staan.

Efficiëntie: IVS Next verbindt bruggen, sluizen en verkeersposten, zodat schepen sneller kunnen passeren zonder hun gegevens telkens opnieuw door te geven. Zo kunnen sluis- en brugbedienaren het verkeer efficiënt en veilig plannen, terwijl verkeersleiders een volledig overzicht van reis- en ladingsgegevens hebben om potentieel gevaarlijke situaties te voorkomen.

*   AIS-signalen (Automatic Identification System) zijn automatische scheepvaartuitzendingen die informatie delen over positie, snelheid, koers en identiteit van schepen, bedoeld voor navigatie en verkeersbeheer.

Ontdek hoe IT een een cruciale rol speelt in de wereld van de binnenvaart.

Opdracht in de spotlight: Rijkswaterstaat

Op dinsdag 22 juli organiseren we samen met onze vrienden van Sysdig een AI Meetup bij ons op kantoor. Dit event staat volledig in het teken van AI. De eerste workshop gaat over hoe je LLMs kan gebruik om K8s te troubleshooten. De tweede workshop zal gaan over het maken van workflows binnen n8n waarin we gebruik maken van AI-agents. Beide workshops zullen gebruiken maken van een lokale LLM. De afsluiting is een spannende hands-on Capture the Flag workshop waarin je als team met behulp van Sysdig’s AI assistant Sage probeert een hack te voorkomen en als eerste de kluis moet zien te kraken om te zegevieren.

Voor wie is deze workshop?
Voor iedereen met interesse in AI en enige ervaring in de IT.

Hoe kan ik me voorbereiden?
Deze informatie vind je binnenkort op deze Meetup pagina.

18:00 | Workshop - Mastering K8s Troubleshooting with K8sGPT door Sander Lambrechts (CINQ)

18:30 | Workshop - Building Agentic Workflows with n8n door Dennis Vermeulen (CINQ)

19:15 | Workshop - AI Security Challenge: Capture the Flag met Sysdig Sage door Roald Ruchti en Marcel Claassen (Sysdig)

Op dinsdag 22 juli organiseren we een Meetup in het teken van AI. In drie hands-on workshops verkennen we verschillende LLM's en toepassingen van AI.

Meetup door Sysdig & CINQ: AI workshops

Java has come a long way since its early days as an object-oriented, imperative programming language. Over time, it has steadily adopted features inspired by functional programming, a paradigm focused on immutability, pure functions, and type safety—principles long embraced by languages like Haskell.

From lambda expressions and functional interfaces in Java 8 to records in Java 11, Java has been evolving to make functional programming more accessible. However, one crucial feature has been missing—the ability to define algebraic data types (ADTs) in a concise and type-safe way. With the introduction of sealed classes, Java now fills this gap, enabling developers to write code that mirrors the expressiveness and immutability of functional languages like Haskell.

In this blog, I’ll explore how Java is embracing functional programming principles and demonstrate why sealed classes are the missing piece of the puzzle for writing fully functional code in Java, using the purely functional language Haskel as an example.

A sealed class in Java restricts which other classes or interfaces can extend or implement it, enabling developers to define a controlled hierarchy and enforce inheritance rules. This restriction is explicitly enforced by the Java compiler, ensuring that only specified, predefined subclasses are allowed to inherit from the sealed class. 

To declare a sealed class (or interface), you use the `sealed` keyword on the class definition, followed by a `permits` clause that lists all allowed subclasses:

public sealed class Shape permits Circle, Triangle, Rectangle {
    // class contents
}

In this example, `Shape` is a sealed class that is extended by `Circle`, `Triangle`, and `Rectangle`.

When a sealed class defines its permitted subclasses, those subclasses must be declared within the same module or package as the sealed class and compiled together. Each subclass of a sealed class must include one of the following modifiers. The final modifier indicates that the subclass is the last type in the hierarchy and cannot be extended further. The sealed modifier allows the subclass to define its own permitted subclasses, continuing the sealed hierarchy. The non-sealed modifier removes inheritance restrictions, allowing any class to extend this subclass.

public sealed class Shape permits Circle, Triangle, Rectangle {}

final class Circle extends Shape {}
sealed class Triangle extends Shape permits Pyramid {}
non-sealed class Rectangle extends Shape {} 
final class Pyramid extends Triangle {} 

// And an unknown, at runtime, number of children of Rectangle e.g.
final class Square extends Rectangle {} 

In this example, Shape is the sealed class that is extended by ‘Circle’, ‘Triangle’ and ‘Rectangle’. Circle is final and cannot be subclassed. Triangle is defined as sealed, and therefore must declare permitted subclasses. Rectangle is non-sealed meaning it can be subclassed, meaning it can be subclassed without restriction.

Haskell is a purely functional programming language that emphasizes immutability, lazy evaluation, and strong static typing. It supports higher-order functions, allowing functions to take other functions as arguments or return them—similar to lambda expressions and functional interfaces introduced in Java 8. Haskell also includes algebraic data types (ADTs), which enable expressive and type-safe data modeling. Its product types resemble records introduced in Java 11, while its sum types are similar to Java’s sealed classes introduced more recently.

Java has increasingly adopted features inspired by functional programming, particularly to address challenges in concurrency. Functional programming minimizes the use of variables and promotes immutability by default, ensuring computations are thread-safe. This makes functional programs highly scalable and reduces the risk of bugs caused by shared mutable state, offering significant advantages for modern, parallel systems.

In the following sections, I’ll walk you through how a (simple) List is implemented in Haskell, followed by its functional equivalent in Java. I chose the List because it’s often considered the "Hello, World" of Haskell programming—simple, fundamental, and a great starting point for understanding functional concepts. 

-- Define a List data type 
data List a = Empty             -- Represents an empty list 
            | Cons a (List a)   -- Represents a non-empty list 
                                -- (head + tail) 

-- Add an element to the list 
addElement :: a -> List a -> List a 
addElement element list = Cons element list

In the first two lines, the List type is defined with two constructors: Empty, representing an empty list, and Cons (Constructor) , representing a value (the head) followed by the rest of the list (the tail). Notice the use of recursion. The letter ‘a’ is used in the same way as ‘T’ in Java to represent a generic type.

Next, a single global function called addElement is declared to add an element to the beginning of a list. This function is defined in two parts. The first line specifies its type signature—it takes a value of type a and a List of type a, then returns a new List of the same type. The second line provides the implementation, which is notably concise. It creates a new Cons entity, setting the provided value as the head and the existing list as the tail.

In the following code, a list is declared, and a new list is created by prepending an element. Notice that addElement returns a new list object, leaving the original list unchanged. This design ensures that the list is immutable by definition.

myList :: List Int
myList = Cons 1 (Cons 2 (Cons 3 Empty))       -- [1 2 3]

let newList = addElement 0 myList             -- [0 1 2 3]

Now let's write the same code in Java, in a pure functional way.

// Define the sealed class hierarchy
sealed interface List<T> permits Empty, Cons{} 

// Represents an empty listrecord 
Empty<T>() implements List<T> {}

// Represents a non-empty list (head + tail)record  
Cons<T>(T head, List<T> tail) implements List<T> {}

// Utility class to work with the List ADT
class ListUtils {
    // A useful to String method
    public static <T> String toString(List<T> list) {
        return switch (list) {
            case Empty<T> ignored -> "";
            case Cons<T> cons -> cons.head().toString() + " " 
               + toString(cons.tail());        
    };    
}    
    // Add an element to the list    
    public static <T> List<T> addElement(T element, List<T> list) {     
        return new Cons<T>(element, list);    
    }
}

By using a sealed interface List that only permits the records Empty and Cons, we can create an immutable algebraic data type (ADT) in Java. While some of Haskell’s compactness and conciseness is lost in translation, the result is still quite readable and maintains the functional programming style.

The global functions, toString and addElement, are defined in a utility class, although they could also be implemented directly within the List interface. Notably, the toString method uses pattern matching and ensures exhaustiveness in the switch statement, making the code both safe and expressive. This design maintains immutability and type safety, showcasing how Java can incorporate functional programming concepts inspired by languages like Haskell.

public class Main {
    public static void main(String[] args) {
        // Create a list: [ 1 2 3 ]
        var myList = new Cons<>(1, new Cons<>(2, 
                     new Cons<>(3, new Empty<>())));
        var newList = ListUtils.addElement(0, myList);

        // Will print: [ 0 1 2 3 ]
        System.out.println("[" + ListUtils.toString(newList) + "]"); 
    }
}

This code closely mirrors the previous Haskell example. It demonstrates how to initialize a List and create a new one by prepending a zero to the beginning. While Java already offers a variety of predefined thread-safe data structures, this example highlights how Java has taken further steps toward embracing functional programming.

In this blog, I demonstrated how Java has steadily adopted features from functional programming languages like Haskell. With the introduction of lambda expressions, records, and most recently, sealed classes, Java has evolved to support immutability, type safety, and expressive data modeling.

I also showed that Java can achieve many of the same benefits as Haskell—such as thread safety andscalability—while retaining its object-oriented foundation. With sealed classes filling the final gap, Java is now fully equipped to support a functional programming style, allowing developers to write cleaner, safer, and more maintainable code.

Sealed classes and functional programming in Java

Haskell Did It First: Sealed classes and functional programming in Java

Cribl Stream is recognized for its ability to efficiently structure data flows while providing capabilities to mask, transform, reduce, and route data. The platform's straightforward implementation and operation make it particularly suitable for enterprise environments, which has contributed to Cribl’s rapid adoption among Fortune 100 companies globally. Cribl Stream offers immediate functionality that many organizations have desired to incorporate for a long time.

Our experience with this solution spans several years, during which we have consistently been impressed by its organizational impact. As a partner we are working together with Cribl to enhance the products themselves and for the features outside of the core functionality we have developed an open-source solution: CriblUtilities - A comprehensive tool to streamline migrations and validate configurations. Our open source solution ensures a smooth implementation of Cribl Stream within your organization.

We started our development with the core functionality of database migration. Splunk provides DBConnect as a supplementary application for Splunk Heavy Forwarder. However, the architecture of the Heavy Forwarder does not support high availability. While it is possible to construct a high-availability solution around it, Cribl Stream natively supports database queries at defined intervals with built-in high availability. Making this a quick win for organizations that are starting their Cribl Stream implementation.

All you need is a user on your Cribl API and the three configuration files from Splunk and our CriblUtilities solution takes care of the rest.

The development process required substantial testing and coding efforts. Splunk utilizes a modified implementation of TOML (Tom's Obvious, Minimal Language). The standard Python TOML module proved to be incompatible with this implementation, and requires data reformatting to ensure compatibility. The migration process transforms three distinct Splunk files (db_connections.conf, identities.conf, and db_inputs.conf) into two Cribl outputs (Database Connection Knowledge items and Database Collectors). Followed by verification and additional configuration adjustments, such as schedule activation, the new configurations are deployed to Cribl via the Cribl API.

Currently, this represents the sole migration path incorporated into the utility. It serves as a foundation for the continued expansion of migration capabilities in future development.

Above is a picture shown with a couple of steps. In step 1 the setup of CriblUtilities is started. A few required parameters are asked from the user via the prompt, this results in a ".env" file. Alternatively, you can also set this file manually. In step 2 the empty screens are shown to prove the environment is empty. In step 3 the actual migration is done. The script gives insight into what is done. In step 4 the proof is shown that the database connections and collectors are created in Cribl Stream.

Cribl Stream provides native support for GitOps, which we consider very beneficial as it facilitates a unified approach to configuration promotion, testing, and administration. The implementation process is straightforward, requiring only a single API command. Our clients typically establish a workflow wherein designated individuals review merge requests between environments to identify potential configuration anomalies.

CriblUtilities does not aim to replace these manual reviews but rather to reduce their scope. The utility can be integrated into the pipeline to perform configuration verification prior to manual review, thereby reducing reviewer workload and implementing consistent validation procedures.

Our initial development focused on YAML file linting. As Cribl utilizes standard YAML formatting, we leverage the Python YAML module to identify unsupported modifications, such as improperly formatted local changes. The prevention of improperly formatted YAML files is critical, as they may compromise Cribl Stream in production.

Establishing naming conventions for Cribl Stream configuration elements is considered best practice. We have implemented naming convention verification as an additional feature in CriblUtilities. Organizations typically maintain conventions for worker groups, sources, destinations, data routes, pipelines, and packs. As configuration types store naming information in various locations, our utility simplifies verification through commands such as:

cribl-utilities check naming --conf /opt/cribl --field sources --regex "src_.*"

For pipeline configurations, similar verification can be performed with:

cribl-utilities check naming --conf /opt/cribl --field pipelines --regex "ppl_.*"

In this example there are four worker groups created. Only one applies to the new naming convention. Which states that the name of a worker group should start with "wgp_".

In this example you see the default worker group. We created an exception for this worker group in the naming convention since this group is always there. This gives two workers groups that do not apply to the naming convention:

Use the debug tag to check whether this checks all configurations you expect. Especially filtered_fields are interesting in this case:

We have decided to launch this initiative as an open-source project due to our commitment to transparency. CriblUtilities does not transmit data to external sources with the exception of your designated Cribl environment. Our open approach enables verification of these assurances. We encourage developer participation to enhance functionality, and welcome issue reports from all users to facilitate ongoing improvement.

CribUtilitites is a comprehensive tool to streamline migrations and validate configurations.

Streamline migrations and validate configurations with CriblUtilities

In early 2024, the 1 Billion Row Challenge (1BRC) went viral. The goal was simple yet daunting: process a file with 1 billion rows of temperature measurements and calculate the minimum, maximum, and mean temperatures for each weather station in the dataset. This benchmark pushed developers to explore extreme optimization techniques. The winning solution completed this challenge in just a mere 1.535 seconds, decisively debunking the long-standing myth that Java is inherently slow.

The question I aim to answer in this blog is: how was this incredible feat achieved, and what techniques can we apply to make Java code faster?

The first and simplest approach is to leverage the ability of modern processors to handle multiple tasks simultaneously—also known as parallelization. Parallelization makes work faster by dividing a task into smaller parts that can be completed simultaneously. Instead of processing everything one step at a time (sequentially), parallelization assigns different parts of the work to multiple processors or cores.

Java offers powerful libraries for parallelization, making it easy to divide tasks and process them simultaneously. Key tools include the Fork/Join Framework for divide-and-conquer problems, Parallel Streams for effortless parallel data operations, and the Executor Framework for managing threads. Features like CompletableFuture enable asynchronous parallel tasks, while third-party libraries like Akka and RxJava support advanced concurrency. These tools simplify parallelization, boosting performance by leveraging modern multi-core processors.


The fastest solutions took this approach to the next level. A common bottleneck in parallel processing is that files are often read sequentially, even when the data is processed in parallel. Without delving too deeply into the details, the fastest developers overcame this by writing custom code to split the text into chunks optimized for the number of cores and CPU cache sizes, allowing the file to be read and processed simultaneously across all cores—fully maximizing their utilization. Understanding your architecture is key.

Working hard isn’t enough—you need to work smart. While reading each byte of every line, the data must also be understood. Since they were dealing with a CSV file, this involves identifying the separators and efficiently parsing each line into meaningful pieces for further processing.

A simple way to handle this is by using regular expressions, often the go-to approach in everyday programming. However, for extreme performance, regular expressions are too slow due to their sequential nature. To optimize processing, it’s necessary to break down each line into bytes and process them in parallel, leveraging the capabilities of modern CPUs.

The first approach is using Java’s Vector API, which enables SIMD (Single Instruction, Multiple Data) operations. SIMD allows a single instruction to process multiple bytes simultaneously, significantly boosting performance. For example, instead of checking each character one by one, the Vector API processes entire chunks of data (e.g., 128 bits) in parallel, identifying target values like commas in a single operation. This method is ideal for maximizing CPU core utilization;  however, it does have some limitations, such as incompatibility with GraalVM.

The second approach involves masks, which efficiently process data at the byte level. A mask is an array that marks positions of target values (e.g., 1 for a match, 0 otherwise) using branchless comparisons. Masks avoid the performance penalties of conditional logic, ensuring predictable, high-speed processing. While less abstract than the Vector API, masks provide a scalable and hardware-agnostic solution for parallel data parsing.

3. Efficient Data Structures – Be Organized

When processing data, temporary results need to be stored efficiently while handling the next piece of data. A commonly used data structure for this purpose is the standard Java hash map (HashMap), a reliable solution for key-value storage. However, while HashMap is versatile, it can become a performance bottleneck due to hash collisions and the overhead involved in resolving them.

To address this, some developers explored alternative storage methods. One particularly intriguing approach was creating a custom hash map with forward probing. A standard Java HashMap works by transforming a key into a hash value and storing the value at an index determined by the hash. However, hash collisions occur when multiple keys are assigned the same index. In such cases, the HashMap stores these values in a list. Retrieving a value then requires accessing the list and traversing it to find the desired value, which can slow down performance.

In contrast, a hash map using forward probing avoids storing values with the same index in a list. Instead, it searches for the next available slot in its array and places the value there. This approach is faster, provided collisions are kept to a minimum. Since the set of weather stations in this case was fixed, developers could design a custom hash function optimized to minimize collisions. By fully understanding the constraints of their data, they were able to implement extremely efficient organized data.

This approach is less about optimizing the code itself and more about optimizing how Java runs the code. As you may know, Java source code is first compiled into an intermediate format called bytecode, which is platform-independent. At runtime, the JVM (Java Virtual Machine) uses a Just-In-Time (JIT) compiler to convert this bytecode into native machine code specific to the underlying hardware. This process allows Java programs to achieve performance close to that of native applications.

However, JIT compilation happens during program execution, which can create overhead. The JVM spends time analyzing and optimizing code paths at runtime, leading to initial "warm-up" delays. This can slow down execution, especially for performance-critical or short-lived applications.

To address this, contestants used GraalVM, which improves runtime performance by leveraging an Ahead-Of-Time (AOT) compiler. GraalVM compiles Java bytecode into native machine code before the program runs, eliminating the need for JIT compilation during execution. This reduces warm-up times and improves startup performance. Additionally, GraalVM applies advanced optimization techniques to generate highly efficient native code, resulting in faster execution overall, particularly for applications that demand high performance or quick startup times.

For some developers, even this wasn’t fast enough. As you may know, Java uses a Garbage Collector (GC) to clean up objects that are no longer referenced. Without this process, a program running long enough would eventually run out of memory. However, while the Garbage Collector is working, the CPU is occupied and not running your program, which can cause slowdowns.

To remedy this, some developers simply turned off the Garbage Collector. (Technically, they used the Epsilon GC that just doesn’t do anything.) This is feasible for short-lived programs because all memory is automatically released once the program finishes execution. By disabling the GC, developers eliminated this overhead, allowing the CPU to focus entirely on the program and achieve faster execution times.

One of the key benefits of using Java over languages like C or C++ is its strong emphasis on safety. Java is designed to minimize the risk of causing harm to your system while running code. Each Java application operates within its own dedicated memory space and cannot access or interfere with memory allocated to other processes. This isolation, often referred to as a sandbox, ensures that Java programs cannot disrupt other processes or compromise system stability.

Java is also memory-safe, meaning it prevents unauthorized access to memory. Unlike C or C++, where developers manage memory manually, Java uses built-in mechanisms to handle memory allocation and cleanup. For example, Java’s automatic Garbage Collector ensures that unused objects are safely removed without risking memory leaks or invalid memory references. Combined with its robust runtime checks and strict type system, Java significantly reduces vulnerabilities, making it a highly secure choice for developers.

However, this level of safety comes at a cost: it can slow down your code. To achieve even greater performance, some developers bypassed these safety features by using Java's Unsafe library for direct memory manipulation. The Unsafe library allows developers to perform low-level operations, such as directly allocating and accessing memory, circumventing Java’s built-in memory management and safety checks.

One specific optimization involved bypassing the unmapping of data, which is typically handled automatically by Java’s memory management system. By taking control of these processes, developers were able to reduce overhead and execute operations significantly faster. 

The above-mentioned techniques sacrifice safety for speed and should, therefore, be used only in extreme cases when a developer truly knows what they are doing.

Parallelization – Teamwork: Break tasks into smaller pieces and execute them in parallel to maximize CPU utilization, ensuring no processing power goes to waste.

Efficient Parsing – Work Smart: In programming, solving problems efficiently matters. While there are many ways to achieve a solution, some paths are significantly faster—choose the shortest road to your goal.

Efficient Data Structures – Be Organized: Every data structure has strengths and weaknesses. Understand your data and select the structure that best fits the problem for optimal performance.

GraalVM – Skip Unnecessary Steps: Fast code isn’t everything; how your code is executed matters too. Tools like GraalVM can eliminate unnecessary runtime overhead, allowing your code to run more efficiently.

Unsafe – Don’t Try This at Home: In extreme cases, bypassing Java’s safety features with tools like the Unsafe library can deliver higher performance. However, this approach should only be used with careful consideration and a strong justification.

https://www.morling.dev/blog/one-billion-row-challenge/

Inside the Fastest Solutions of the 1 Billion Row Challenge

Bekijk hieronder de aftermovie van de CINQ Conferentie. De jaarlijkse afsluiting van de CINQ Sprekersopleiding was weer een feestje!

We hebben genoten van drie mooie presentaties:
🔸 'Deployen op vrijdagmiddag zonder stress' door Bas Stoker
🔸 'Wirespec vs OpenAPI' door Jerre van Veluw
🔸 'Immunologie door de lens van een ontwikkelaar' door Alex Witkamp

Aftermovie CINQ Conferentie 2025

De volgende CINQ sprekersopleiding gaat waarschijnlijk in het najaar van 2025 van start. 

Volg ons op Linkedin om op de hoogte te blijven: https://www.linkedin.com/company/cinqict/ 

Bekijk de aftermovie van de CINQ Conferentie 2025

Were you ever curious about how companies like Uber, Just Eat, and Strava show you the location of your delivery, ride, or run in real-time? Well, in this tutorial we will go over the two main ways that map data can be displayed and interacted with on a web page. 

a discussion of the two main ways to display map data on a web page, and their pros and cons

inserting actual map data into the database

spinning up a geoserver to serve the map data

spinning up a geographical REST API to serve the map data

displaying the map data on a web page using streamlit and folium


In the following video demo our resident Python Developer, James Twose, will show you how it is done!

Displaying map data on a web page: geoserver vs geoapi

We hope you enjoyed this tutorial and now have a better understanding of how to display map data on a web page.

If you have any questions or comments, please feel free to reach out to us on our social media channels or via email.

You can find the code for this tutorial here: https://github.com/jameshtwose/geoserver-vs-pygeoapi

Were you ever curious about how companies like Uber, Just Eat, and Strava show you the location of your ride, delivery, or run in real-time? In this tutorial we will go over the two main ways that map data can be displayed and interacted with on a web page.

Displaying map data on a web page: Geoserver vs geoapi

Geoserver vs geoapi

Heb je altijd al willen spreken op een conferentie maar weet je niet waar je moet beginnen? Of wil je je simpelweg je presentatievaardigheden verbeteren? Meld je dan aan voor de CINQ Sprekersopleiding! 

Na het succes van twee interne edities met onze CINQ collega’s, gaat in oktober 2024 de CINQ Sprekersopleiding open voor het publiek. 

De opleiding bestaat uit vier avonden waarin je wordt klaargestoomd voor je eerste optreden. In oktober, november, januari en februari zijn de opleidingsavonden en in maart vindt de CINQ Conferentie plaats: daar ga jij je presentatie voor publiek geven! 

hoe schrijf je een goed voorstel (paper)?

hoe maximaliseer je je kansen om geselecteerd te worden?

Tijdens de avonden word je voorzien van de benodigde kennis, en ga je vooral ook veel oefenen met presenteren. De deelnemers gaan elkaar beoordelen en van feedback voorzien. Aan het einde van het traject heeft iedereen een goed beeld van wat er nodig is om op een conferentie te kunnen spreken. Niet alleen het spreken zelf, ook alle stappen die voorafgaan aan het spreken op een conferentie heb je dan onder de knie.

We sluiten het traject af met de CINQ conferentie waarbij je je nieuwe vaardigheden in een presentatie voor publiek gaat laten zien. Hierna blijft Bouke beschikbaar als je coach en zal hij je begeleiden naar je eerste “optreden” als spreker op een conferentie.

Deelname aan de CINQ Sprekersopleiding kost €250 exclusief btw. Hiervoor krijg je:

vier avonden opleiding inclusief avondeten

training en begeleiding door een ervaren internationaal spreker: Bouke Nijhuis

begeleiding na het traject naar je eerste conferentie

Klik hier om je aan te melden. Je zit nog nergens aan vast. Na je aanmelding nemen we zo snel mogelijk contact met je op. 

Bekijk een impressie van de vorige CINQ Conferentie:

CINQ CONFERENTIE AFTERMOVIE 2024

Heb je altijd al willen spreken op een IT conferentie maar weet je niet waar je moet beginnen? Of wil je je simpelweg je presentatievaardigheden verbeteren? Meld je dan aan voor de CINQ Sprekersopleiding!

Meld je aan voor de CINQ Sprekersopleiding

CINQ Sprekersopleiding

Kun je al (een beetje) programmeren en ben je benieuwd naar Python? Kom langs en leer van Bart van Dijk waarom Python een fijne taal is om snel mee te kunnen developen.

De workshop
In de workshop leer je over de datatypes binnen Python en een aantal geavanceerdere concepten. Vervolgens laat Bart zien hoe je Clean Code kunt schrijven in Python.

Als afsluiter kan er gekozen worden tussen een set opdrachten voor Object-Georienteerd Programmeren, Functioneel Programmeren, of algoritmische opdrachten a la Advent of Code.

Voor wie is deze workshop?
Developers, DevOps- en Data-engineers met al enige ervaring in programmeren.

Hoe kan ik me voorbereiden?
Zorg dat je je favoriete IDE (bijvoorbeeld VSCode) paraat hebt die Python ondersteunt. (Of Notepad als je van hardcore code schrijven houdt)

Het programma
17.00u - inloop met eten
18.00u - start workshop
20.00u - borrel / netwerken

Het is een workshop, dus neem je laptop mee!

Kom naar de Python workshop op 10 september

Splunk 9.3 has been released and  there are some new features that we would like to share with you. In the release notes you can find all the features added to this release.

Dashboard Studio - Scheduled PDF and PNG export

It is now possible to schedule PDF or PNG exports for Dashboard Studio dashboards.  For users to be able to do this you have to give them the ‘list_settings’ capability. There are a few limitations that you should consider, see the Splunk Documentation for the details.

A nice bonus is the ability to use the API to fetch a PNG, this means that you can use your automation tooling to fetch a PNG of a dashboard and display that on a website. This is much easier than using embedded reports.

Instead of Python 3.7 the new default is now Python 3.9, you could override this in your app settings but we highly recommend testing all your Python scripts in a test environment with Splunk 9.3 before upgrading on production. 

On busy search heads it is now possible to apply a memory reduction measure for the Splunkd process. In the Splunk conf description it does state that you should consult Splunk support before enabling this setting. 

If you used role-based field filters then you should know that after upgrading these filters will no longer work. You might want to check out the new field filter option, but keep in mind that you can no longer use the following commands: mpreview, mstats, tstats, typeahead, and walklex

If you use Splunk Enterprise Security you probably won’t be able to use this since Splunk Enterprise Security relies heavily on tstats. For more information click this link. 

Splunk 9.3 has been released and we highlight some of the new features in this blogpost.

Onze laatste blogs