cloud – Zukunft-Innovation-Technik (ZuInnoTe)

Revisiting Big Data Formats: Apache Iceberg, Delta Lake and Apache Hudi

Aug 5, 2023

—

von

in analytics, big data, cloud, flink, hadoop, hive, spark

Novel Big Data formats, such as Apache Parquet, Apache ORC or Apache Avro have been years ago the game changer for processing massive amounts of data efficiently as I wrote in a previous blog post (aside of the Big Data platforms leveraging them). Nowadays we see the emergence of new Big Data formats, such as…

eBPF for modern cloud-based data centres

Mrz 5, 2023

—

von

Jörn Franke

in artificial intelligence, big data, cloud, devops, emerging trends, serverless computing

I have also published the source code related to this article on Codeberg (EU Git hosting) and Github (US Git hosting). Modern cloud data centres offer different level of abstractions to run applications, such as virtual machines, containers, micro-vms (see also my article on Unikernels), functions, virtual data-processing clusters and so on. As those abstractions…

Modern Cloud Application Delivery: WASM and WASI

Feb 7, 2022

—

von

Jörn Franke

in artificial intelligence, cloud, emerging trends, inter-cloud, serverless computing, tech

I described in a previous blog post that modularity will play a key role in future enterprise applications. This is demonstrated in the current trends of serverless functions or containerized architectures. However, those solutions are not perfect: Given the trend of many different computing architectures, such as ARM on servers, Internet of Things (IoT) Edge…

Unikernels, Software Containers and Serverless Architecture: Road to Modularity

Sep 14, 2019

—

von

Jörn Franke

in business, emerging trends, inter-cloud, serverless computing

This blog post is discussing the implications of Unikernels, Software Containers and Serverless Architecture on Modularity of complex software systems in a service mesh as illustrated below. Modular software systems claim to be more maintainable, secure and future proven compared to software monoliths. Software containers or the alternative MicroVMs have been proven as very successful…

GPUs, FPGAs, TPUs for Accelerating Intelligent Applications

Feb 5, 2019

—

von

Jörn Franke

in big data, business, data science, emerging trends, machine learning

Intelligent Applications are part of our every day life. One observes constant flow of new algorithms, models and machine learning applications. Some require ingesting a lot of data, some require applying a lot of compute resources and some address real time learning. Dedicated hardware capabilities can thus support some of those, but not all. Many…

HadoopCryptoLedger library a vision for the coming Years

Dez 29, 2017

—

von

Jörn Franke

in analytics, big data, blockchain, flink, hadoop, hive, spark, streaming, tech

The first commit of the HadoopCryptoLedger has been on 26th March of 2016. Since then a lot of new functionality has been added, such as support for major Big Data platforms including Hive / Flink / Spark. Furthermore, besides Bitcoin, Altcoins based on Bitcoin (e.g. Namecoin, Litecoin or Bitcoin Cash) and Ethereum (including Altcoins) have…

Lambda, Kappa, Microservice and Enterprise Architecture for Big Data

Nov 11, 2016

—

von

Jörn Franke

in big data, business, emerging trends

A few years after the emergence of the Lambda-Architecture several new architectures for Big Data have emerged. I will present and illustrate their use case scenarios. These architectures describe IT architectures, but I will describe towards the end of this blog the corresponding Enterprise Architecture artefacts, which are sometimes referred to as Zeta architecture. Lambda…

DevOps for your business? – About Uniting Development and Operations

Mai 18, 2014

—

von

Jörn Franke

in big data, business, devops, emerging trends, enterprise application integration

DevOps has become in recent years a term for a new paradigm of integrating and managing development as well as operations of software within and cross organizations. I will describe in this blog entry what DevOps is and relate it to existing methodologies, such as agile development, and organizational structures. Basically, DevOps is a broad…

Scenarios for Inter-Cloud Enterprise Architecture

Okt 27, 2013

—

von

Jörn Franke

in business, emerging trends, enterprise application integration, inter-cloud

The unstoppable cloud trend has arrived at the end users and companies. Particularly the first ones openly embrace the cloud, for instance, they use services provided by Google or Facebook. The latter one is more cautious fearing vendor lock-in or exposure of secret business data, such as customer records. Nevertheless, for many scenarios the risk…

Schlagwort: cloud