Abstract

This report documents the program and outcomes of Dagstuhl Seminar 23302, "Software Architecture and Machine Learning". We summarize the goals and format of the seminar, results from the breakout groups, key definitions relevant to machine learning-enabled systems that were discussed, and the research roadmap that emerged from the discussions during the seminar. The report also includes the abstracts of the talks presented at the seminar and summaries of open discussions.

Abstract

Microservice architectures have gained enormous popularity due to their ability to be dynamically added/removed, replicated, and updated according to run-time needs. However, the dynamic nature of microservices introduces uncertainty, which in turn can affect the provided Quality of Service (QoS). This calls for novel service discovery mechanisms able to adapt to the variability of the QoS attributes and further perform effective service discovery and selection. To this end, this paper combines machine learning and self-adaptation techniques to perform service discovery and selection by trading off different QoS attributes. The results of our validation on a state-of-the-art microservices exemplar show that our ML-enabled approach can perform service discovery with 35% higher effectiveness with respect to existing baselines.

Abstract

Urbanization, driven by technological advancements, has brought about improved connectivity and efficiency, especially with the rise of Internet of Things (IoT) devices. Smart cities use these innovations to manage resources better and enhance resident’s quality of life. However, implementing smart city projects comes with challenges like monitoring, maintaining, and testing urban infrastructure. Digital Twin(DT) entails the connection of physical facilities or devices with their digital counterparts, facilitating real-time monitoring, manipulation, and predictive analysis of their behavior. This innovative concept offers a virtual replica of assets, processes and systems, enabling insights into their real-time performance and predictive behaviors. By simulating real-world scenarios, DT aids in planning maintenance activities and conducting comprehensive testing, thereby enhancing the resilience and efficiency of Smart City Systems. Particularly in the context of managing water networks, DT technology holds significant promise. Visualization capabilities provide intuitive insights, while actuation enables control actions based on predictive analytics and optimization algorithms. To this end, in this paper, we present the architecture of WaterTwin, a DT we developed for water quality networks in smart city systems. We demonstrate our approach through the use case of a water quality network in the smart city living lab of the IIIT Hyderabad campus.

Abstract

While documenting Architectural Knowledge (AK) is crucial, it is frequently neglected in many projects, and existing manual tools are underutilized. Although undocumented, Archi-tecture Knowledge (AK) is dispersed across various sources such as source code, documentation, and runtime logs. To address this, automated tools for efficient AK extraction and documentation are essential. Even after generating AK, navigating through vast the Architectural Records can be overwhelming. Building on that, we propose an automated Architectural Knowledge Management (AKM) System using Information Extraction and Generative AI, which generates AK from various source for a given system and answers architectural queries with respect to the given system. The development of an efficient Architectural Knowledge Management (AKM) system, which is both effective and user-friendly, entails the resolution of numerous challenges. It requires consolidating diverse AK data sources scattered across code, dia-grams, repository commits, and online platforms. The integration of Multimodal AI for AK extraction, incorporation of global AK, and leveraging Generative AI for AK documentation further compounds the problem. Moreover, generating contextually appropriate query responses adds another layer of complexity. To this end, we performed an initial exploratory study on generating Architectural Design Decisions using generative Large Language Models (LLM) in the context of Architecture Decision Records (ADR). Our initial results have been promising indicating the potential impact of GenAI for architectural knowledge management.

Abstract

Edge computing allows for reduced latency and operational costs compared to centralized cloud systems. In this context, serverless functions are emerging as a lightweight and effective paradigm for managing computational tasks on edge infrastructures. However, the placement of such functions in constrained edge nodes remains an open challenge. On one hand, it is key to minimize network delays and optimize resource consumption; on the other hand, decisions must be made in a timely manner due to the highly dynamic nature of edge environments. In this paper, we propose POSEIDON, a solution based on Deep Reinforcement Learning for the efficient placement of functions at the edge. POSEIDON leverages Proximal Policy Optimization (PPO) to place functions across a distributed network of nodes under highly dynamic workloads. A comprehensive empirical evaluation demonstrates that POSEIDON significantly reduces execution time, network delay, and resource consumption compared to state-of-the-art methods.

Abstract

Modern software systems are subjected to various types of uncertainties arising from context, environment, etc. To this end, self-adaptation techniques have been sought out as potential solutions. Although recent advances in self-adaptation through the use of ML techniques have demonstrated promising results, the capabilities are limited by constraints imposed by the ML techniques, such as the need for training samples, the ability to generalize, etc. Recent advancements in Generative AI (GenAI) open up new possibilities as it is trained on massive amounts of data, potentially enabling the interpretation of uncertainties and synthesis of adaptation strategies. In this context, this paper presents a vision for using GenAI, particularly Large Language Models (LLMs), to enhance the effectiveness and efficiency of architectural adaptation. Drawing parallels with human operators, we propose that LLMs can autonomously gen- erate similar, context-sensitive adaptation strategies through its advanced natural language processing capabilities. This method allows software systems to understand their operational state and implement adaptations that align with their architectural requirements and environmental changes. By integrating LLMs into the self-adaptive system architecture, we facilitate nuanced decision-making that mirrors human-like adaptive reasoning. A case study with the SWIM exemplar system provides promising results, indicating that LLMs can potentially handle different adaptation scenarios. Our findings suggest that GenAI has signifi- cant potential to improve software systems’ dynamic adaptability and resilience.

Abstract

In today’s dynamic technological landscape, sustainability has emerged as a pivotal concern, especially with respect to architecting Machine Learning enabled Systems (MLS). Many ML models fail in transitioning to production, primarily hindered by uncertainties due to data variations, evolving requirements, and model instabilities. Machine Learning operations (MLOps) offers a promising solution by enhancing adaptability and technical sustainability in MLS. However, MLOps itself faces challenges related to environmental impact, technical maintenance, and economic concerns. Over the years, self-adaptation has emerged as a potential solution to handle uncertainties. This paper introduces a novel approach employing self-adaptive principles integrated into the MLOps architecture through a MAPE-K loop to bolster MLOps sustainability. By autonomously responding to uncertainties, including data, model dynamics, and environmental variations, our approach aims to address the sustainability concerns of a given MLOps pipeline identified by an architect at design time. Further, we implement the method for a Smart City use case to display the capabilities of our approach.

Abstract

Architectural Knowledge Management (AKM) involves the organized handling of information related to architectural decisions and design within a project or organization. An essential artefact of AKM is the Architecture Decision Records (ADR), which documents key design decisions. ADRs are documents that capture decision context, decision made and various aspects related to a design decision, thereby promoting transparency, collaboration, and understanding. Despite their benefits, ADR adoption in software development has been slow due to challenges like time constraints and inconsistent uptake. Recent advancements in Large Language Models (LLMs) may help bridge this adoption gap by facilitating ADR generation. However, the effectiveness of LLM for ADR generation or understanding is something that has not been explored. To this end, in this work, we perform an exploratory study which aims to investigate the feasibility of using LLM for the generation of ADRs given the decision context. In our exploratory study, we utilize GPT and T5-based models with 0-shot, few-shot, and fine-tuning approaches to generate the Decision of an ADR given its Context. Our results indicate that in a 0-shot setting, state-of-the-art models such as GPT-4 generate relevant and accurate Design Decisions, although they fall short of human-level performance. Additionally, we observe that more cost-effective models like GPT-3.5 can achieve similar outcomes in a few-shot setting, and smaller models such as Flan-T5 can yield comparable results after fine-tuning. To conclude, this exploratory study suggests that LLM can generate Design Decisions, but further research is required to attain human-level generation and establish standardized widespread adoption.

Abstract

The advent of the Internet of Things (IoT) has ushered in transformative possibilities for smart cities, with the potential to revolutionize urban living through enhanced connectivity and data-driven decision-making. However, the effective realization of IoT in smart cities hinges upon the seamless interoperability of diverse devices and systems. To address this critical need, the oneM2M standards initiative has emerged as a foundational framework for IoT interoperability. In this research paper, we perform an exploratory analysis of three prominent open-source oneM2M based interoperability systems—Mobius, OM2M, and ACME. We leverage an existing large-scale system provided by our Smart City Living Lab deployed at IIIT Hyderabad, sprawling a 66-acre campus featuring over 370 nodes across eight verticals. We investigate the architectural characteristics of each solution, considering their strengths and limitations in facilitating IoT interoperability. Through this analysis, our paper aims to provide valuable insights for stakeholders seeking to implement IoT interoperability solutions in the context of smart cities. By evaluating the strengths and limitations of Mobius, OM2M, and ACME, we seek to offer guidance for selecting the most suitable solution. Our analysis reveals that the optimal framework choice depends on specific quality constraints: Mobius excels in performance, while ACME offers advantages in ease of setup for smaller-scale implementations.

Abstract

Addressing runtime uncertainties in Machine Learning-Enabled Systems (MLS) is crucial for maintaining Quality of Service (QoS). The Machine Learning Model Balancer is a concept that addresses these uncertainties by facilitating dynamic ML model switching, showing promise in improving QoS in MLS. Leveraging this concept, this paper introduces SWITCH, an exemplar developed to enhance self-adaptive capabilities in such systems through dynamic model switching in runtime. SWITCH is designed as a comprehensive web service catering to a broad range of ML scenarios, with its implementation demonstrated through an object detection use case. SWITCH provides researchers with a flexible platform to apply and evaluate their ML model switching strategies, aiming to enhance QoS in MLS. SWITCH features advanced input handling, real-time data processing, and logging for adaptation metrics supplemented with an interactive real-time dashboard for enhancing system observability. This paper details SWITCH's architecture, self-adaptation strategies through ML model switching, and its empirical validation through a case study, illustrating its potential to improve QoS in MLS. By enabling a hands-on approach to explore adaptive behaviors in ML systems, SWITCH contributes a valuable tool to the SEAMS community for research into self-adaptive mechanisms for MLS and their practical applications.