The Question of Maintenance of pre-trained Machine Learning Embeddings

I will address in this post the issue of maintenance of large pretrained embeddings within Artificial Intelligence (AI) services. While this issue has some links to ethical aspects (see for example the European Commission’s guidelines on trustworthy AI or here), the focus here is on maintainability of those embeddings as part of MLOps. Software Maintenance… The Question of Maintenance of pre-trained Machine Learning Embeddings weiterlesen

Secure Blockchain Analytics

Blockchain analytics has become a trending topic in recent years. This topic is of interest not only for public blockchains, such as Bitcoin or Ethereum and their Altcoins, but also for private/permissive blockchains based on various technologies. Nevertheless, there are many challenges involved, such as the large data volumes, the inefficient format for analytics, state… Secure Blockchain Analytics weiterlesen

AI Applications and Systems for Deep Logic and Probabilistic Networks

This blog post describes the integration of deep learning, logic and probabilistic reasoning to enable advanced artificial intelligence tasks. The combination of completely different set of AI approaches will be one of the key advances to support AI driven business processes in the coming years. Furthermore, I describe challenges for operating such complex AI systems… AI Applications and Systems for Deep Logic and Probabilistic Networks weiterlesen

Unikernels, Software Containers and Serverless Architecture: Road to Modularity

This blog post is discussing the implications of Unikernels, Software Containers and Serverless Architecture on Modularity of complex software systems in a service mesh as illustrated below. Modular software systems claim to be more maintainable, secure and future proven compared to software monoliths. Software containers or the alternative MicroVMs have been proven as very successful… Unikernels, Software Containers and Serverless Architecture: Road to Modularity weiterlesen

GPUs, FPGAs, TPUs for Accelerating Intelligent Applications

Intelligent Applications are part of our every day life. One observes constant flow of new algorithms, models and machine learning applications. Some require ingesting a lot of data, some require applying a lot of compute resources and some address real time learning. Dedicated hardware capabilities can thus support some of those, but not all. Many… GPUs, FPGAs, TPUs for Accelerating Intelligent Applications weiterlesen

Collaborative Data Science: About Storing, Reusing, Composing and Deploying Machine Learning Models

Why is this important? Machine Learning has re-emerged in recent years as new Big Data platforms provide means to use them with more data, make them more complex as well as allowing combining several models to make an even more intelligent predictive/prescriptive analysis. This requires storing as well as exchaning machine learning models to enable… Collaborative Data Science: About Storing, Reusing, Composing and Deploying Machine Learning Models weiterlesen

Automated Machine Learning (AutoML) and Big Data Platforms

Although machine learning exists already since decades, the typical data scientist – as you would call it today – would still have to go through a manual labor-intensive process of extracting the data, cleaning, feature extraction, regularization, training, finding the right model, testing, selecting and deploying it. Furthermore, for most machine learning scenarios you do… Automated Machine Learning (AutoML) and Big Data Platforms weiterlesen

Blockchain Consensus Algorithms – Proof of Anything?

Blockchains have been proven over the last years to be stable distributed ledger technologies. Stable refers to the fact that they can recover from attacks and/or bugs without compromising their assets. They are most commonly known for enabling transaction with virtual cryptocurrencies not issued by a central authority. Popular examples are Bitcoin and Ethereum. However,… Blockchain Consensus Algorithms – Proof of Anything? weiterlesen

Lambda, Kappa, Microservice and Enterprise Architecture for Big Data

A few years after the emergence of the Lambda-Architecture several new architectures for Big Data have emerged. I will present and illustrate their use case scenarios. These architectures describe IT architectures, but I will describe towards the end of this blog the corresponding Enterprise Architecture artefacts, which are sometimes referred to as Zeta architecture. Lambda… Lambda, Kappa, Microservice and Enterprise Architecture for Big Data weiterlesen

Batch-processing & Interactive Analytics for Big Data – the Role of in-Memory

In this blog post I will discuss various aspects of in-memory technologies and describe how various Big Data technologies fit into this context. Especially, I will focus on the difference between in-memory batch analytics and interactive in-memory analytics. Additionally, I will illustrate when in-memory technology is really beneficial. In-memory technology leverages the fast main memory… Batch-processing & Interactive Analytics for Big Data – the Role of in-Memory weiterlesen