Abstract

Residential electricity consumption datasets are essential for applications such as smart grid management, home automation, renewable energy integration, infrastructure planning, and policy-making. However, obtaining high-resolution residential datasets remains challenging due to the high costs and complexities of sensor installation, monitoring, and maintenance, obtaining approvals and related human factors, among other issues. To address this issue with a focus on the Indian residential context, where such data is quite limited, we propose a Residential Electricity Usage Simulator (REUS), to generate synthetic residential electricity usage data. Our approach models electricity usage for 7 different categories of homes using data collected over one year at an hourly interval, from 65 residences. In addition to energy data, we also collected 18 different features for each home to improve our modeling. The data and features are preprocessed using feature selection with Probabilistic Finite State Machines (validated through Multiple Correspondence Analysis) and further refined through systematic data cleaning and imputation. We built simulation models using the popular Machine learning techniques such as Long Short-Term Memory networks, including Vanilla, Stacked, BiDirectional, and Encoder-Decoder LSTMs and Transformer model, including Vanilla Transformer and Temporal Fusion Transformer. In addition, statistical techniques such as Markov Chains of orders (0, 1, 2, 3) and ARIMA were used as benchmarks to evaluate the models’ ability to generate a synthetic residential electricity dataset that is close to real data. Via extensive experimentation and analysis, our results show that (Bi-di) LSTMs capture the trends in electricity consumption more effectively (with the lowest RMSE) than the other models. Simulation and analysis of this nature enables broader, regionspecific energy research, reducing the need for costly or intrusive data collection.

Abstract

A significant task that municipalities or equivalent government organizations across the world perform is street addressing including naming of roads. Due to unstructured ways in which towns and cities grow, this process needs significant thought and effort which in reality results in major delays in terms of action taken on these issues. Meanwhile, the local populace which needs some naming convention for practical needs may take the initiative to arrive at reasonable names for the roads within their vicinity based on their local knowledge. This can result in significant sections of the road network without an official name or named after a landmark or a famous personality in many places across the world. The past 2-3 decades have seen an increasing digitisation of government activities. However, a lot of the street addressing and road naming systems and conventions were derived before the digitisation era, which in most cases means that computational parsability of the addresses or road names that would be important for digital processing would not be accounted for. There has been a significant increase in importance and usage of location-based geocoding services like what3words, Zippr and plus codes in recent times. However, these services provide alphanumeric code-based addressing which is not intuitive for human understanding, lacks directionality, does not account for existing address information nor the geometric relations of road topology. In this paper, we propose usage of (modified versions of) well known AI search algorithms for automated road numbering and evaluate them based on spatial proximity and connectivity on roadmaps of three different cities.

Abstract

Effective decision-making in complex environments with multi-discrete action spaces poses significant challenges for agent architectures, particularly in image-based settings. While Decision Transformers have shown promise in various domains, their performance often suffers in environments where agents must handle multi-discrete actions. Existing enhancements to Decision Transformer architectures have yet to address this critical issue, limiting their ability to support agents in learning robust policies in these environments. To address this gap, we propose Multi-State Action Tokenisation (M-SAT), a novel approach designed to improve agent decision-making by tokenising actions at the individual action level and incorporating auxiliary state information. This disentanglement of actions improves both the performance of agents and the interpretability of individual actions within attention layers, fostering better visibility into agent decision processes. Importantly, M-SAT facilitates the development of more interpretable and transparent agents capable of making complex decisions in dynamic environments involving multi-discrete action spaces. We evaluate M-SAT on the challenging ViZDoom environments, focusing on scenarios with multi-discrete action spaces and image-based observations, such as Deadly Corridor, My Way Home and Death Match. Our approach demonstrates superior performance compared to baseline Decision Transformers, with no additional data or significant computational overheads. Furthermore, we observe that M-SAT does not require positional encoding to achieve high performance, with its removal occasionally leading to further improvements. These findings suggest that M-SAT enables more efficient and interpretable agent-based decision-making in multi-discrete action spaces.

Abstract

Moving Target Defense (MTD) has emerged as a proactive and dynamic framework to counteract evolving cyber threats. Traditional MTD approaches often rely on assumptions about the attacker’s knowledge and behavior. However, real-world scenarios are inherently more complex, with adaptive attackers and limited prior knowledge of their payoffs and intentions. This paper introduces a novel approach to MTD using a Markov Decision Process (MDP) model that does not rely on predefined attacker payoffs. Our framework integrates the attacker’s real-time responses into the defender’s MDP using a dynamic Bayesian Network. By employing a factored MDP model, we provide a comprehensive and realistic representation of the system and its dependencies. We also incorporate incremental updates to an attack response predictor as new data emerges, ensuring an adaptive and robust defense mechanism. Additionally, we consider the costs of switching configurations in MTD, integrating them into the reward structure to balance execution and defense costs. We first highlight the challenges of the problem through a theoretical negative result on regret. However, our empirical evaluations, conducted in network and web application settings, demonstrate the framework’s effectiveness in environments characterized by high uncertainty and dynamically changing attack surfaces.

Abstract

Emergency vehicles (EVs) perform a critical task of attending medical emergencies and delay in their operations can result in loss of lives to long term or permanent health implications. Therefore, it is very important to design strategies that can reduce the delay of EVs caused by slow moving traffic. Most of the existing work on this topic focuses on assignment and dispatch of EVs from different base stations to hospitals or finding the appropriate routes from dispatch location to hospital. However, these works ignore the effect of lane changes when EV is travelling on a stretch of a road. In this work, we focus on lane level dynamics for EV traversal and showcase that a pro-active picking of lanes can result in significant reductions in traversal time. In particular, we design a Reinforcement Learning (RL) model to compute the most optimal lane for an EV to travel at each timestep. We propose RLLS (Reinforcement Learning based Lane Search) algorithm for a general purposes EV trave

Abstract

In literature, various time-frequency representation methods were investigated for automatic detection of voice disorders. Stockwell-Transform (S-Transform) provides good time-frequency localization; hence, it may efficiently capture the voice disorder related information from speech signal. With this motivation, we investigated different variants of S-Transform for the classification of voice disorders. This study proposed the S-Transform based cepstral coefficients for voice disorder detection and assessment. The performance of the proposed feature was compared with baseline features on SVD and HUPA databases. Compared to baseline features, proposed features performed best in terms of classification accuracy of 80.2% and 79.8% on HUPA and SVD databases, respectively for voice disorder detection task. Also, the proposed features performed better in case of assessment task. Further, the experimental results reveal that combining cepstral coefficients derived from S-Transform with baseline features improved the performance of proposed systems by 8% and 4% for detection and assessment tasks, respectively which highlights complementary nature of the explored features. We also analysed the effectiveness of S-Transform based spectral representation in capturing the acoustic characteristics for various voice qualities like breathiness, harshness, creakiness, and falsetto phonations. This representation was also compared with other time-frequency based methods such as STFT, ZTW and SFF. It was observed that S-Transform effectively captures the acoustic variations associated with different voice qualities compared to other baseline methods, which may be due to better spectro-temporal resolution offered by the S-Transform.

Abstract

In safety-critical RL settings, the inclusion of an additional cost function is often favoured over the arduous task of modifying the reward function to ensure the agent's safe behaviour. However, designing or evaluating such a cost function can be prohibitively expensive. For instance, in the domain of self-driving, designing a cost function that encompasses all unsafe behaviours (eg, aggressive lane changes, risky overtakes) is inherently complex, it must also consider all the actors present in the scene making it expensive to evaluate. In such scenarios, the cost function can be learned from feedback collected offline in between training rounds. This feedback can be system generated or elicited from a human observing the training process. Previous approaches have not been able to scale to complex environments and are constrained to receiving feedback at the state level which can be expensive to collect. To this end, we introduce an approach that scales to more complex domains and extends beyond state-level feedback, thus, reducing the burden on the evaluator. Inferring the cost function in such settings poses challenges, particularly in assigning credit to individual states based on trajectory-level feedback. To address this, we propose a surrogate objective that transforms the problem into a state-level supervised classification task with noisy labels, which can be solved efficiently. Additionally, it is often infeasible to collect feedback for every trajectory generated by the agent, hence, two fundamental questions arise:(1) Which trajectories should be presented to the human? and (2) How many trajectories are necessary for effective learning?

Abstract

Preserving the privacy of preferences (or rewards) of a sequential decision-making agent when decisions are observable is crucial in many physical and cybersecurity domains. For instance, in wildlife monitoring, forest rangers must conduct surveillance without revealing animal locations to poachers. This paper addresses privacy preservation in planning over a sequence of actions in MDPs, where the reward function represents the preference structure to be protected. Observers can use Inverse RL (IRL) to learn these preferences, making this a challenging task. Current research on Differential Privacy (DP) in this setting fails to ensure a lower bound on the minimum expected reward and offers theoretical guarantees that are inadequate against IRL-based observers. To bridge this gap, we propose a novel approach rooted in the theory of deception. Deception includes two models: dissimulation (hiding the truth) and simulation (showing the wrong). As our first contribution, we theoretically demonstrate a significant privacy leak in the current dissimulation-based method. Our second contribution is a novel RL-based planning algorithm that uses simulation to effectively address these privacy concerns while ensuring a guarantee on the expected reward. Through experimentation on multiple benchmark problems, we show that our proposed approach outperforms existing methods in preserving the privacy of reward functions.

Abstract

Decision Transformers have demonstrated impressive performance in offline reinforcement learning, but their complex internal mechanisms remain challenging to interpret. This paper presents a comprehensive analysis of Decision Transformers(DT) trained on various MuJoCo environments using different interpretability techniques. Our analysis includes: (a) positional encoding (PE) analysis to assess the impact of temporal information on performance in different types of environments, (b) return-to-go (RTG) analysis to explore the variability in achieving specific returns, (c) embedding analysis revealing the models' ability to learn detailed representations of the environments, (d) attention visualizations to understand the role of attention in behaviour guidance, and (e) perturbation studies to test robustness to input noise. Our analysis reveals the extent to which positional information is necessary for achieving high performance in different MuJoCo environments and provides insights into the model's adaptability to varying task requirements. In addition, the embedding analysis shows hierarchical abstractions captured by the model across layers. Through these analyses, we provide novel insights into the decision-making process of transformers in continuous control domains. These insights are pivotal for designing more interpretable and reliable decision-making systems and deepen our comprehension of the capabilities and limitations of DT in complex tasks.

Abstract

Moving Target Defense (MTD) has emerged as a proactive and dynamic framework to counteract evolving cyber threats. Traditional MTD approaches often rely on assumptions about the attackers knowledge and behavior. However, real-world scenarios are inherently more complex, with adaptive attackers and limited prior knowledge of their payoffs and intentions. This paper introduces a novel approach to MTD using a Markov Decision Process (MDP) model that does not rely on predefined attacker payoffs. Our framework integrates the attackers real-time responses into the defenders MDP using a dynamic Bayesian Network. By employing a factored MDP model, we provide a comprehensive and realistic system representation. We also incorporate incremental updates to an attack response predictor as new data emerges. This ensures an adaptive and robust defense mechanism. Additionally, we consider the costs of switching configurations in MTD, integrating them into the reward structure to balance execution and defense costs. We first highlight the challenges of the problem through a theoretical negative result on regret. However, empirical evaluations demonstrate the frameworks effectiveness in scenarios marked by high uncertainty and dynamically changing attack landscapes.