Abstract

Temporal Action Localization (TAL) is a fundamental and challenging problem in video understanding that involves identifying action instances in untrimmed videos and precisely predicting their
temporal boundaries along with corresponding class labels. Unlike trimmed video classification, TAL
requires accurate detection of start and end times of actions within long, real-world video streams that
may contain multiple activities, background clutter, ambiguous transitions, and variable-duration events.
Existing state-of-the-art approaches predominantly rely on visual modalities such as RGB frames and
optical flow, leveraging proposal-based architectures and sophisticated temporal modeling strategies.
However, despite the rich multimodal nature of videos, the audio modality remains largely underexplored in fully supervised TAL frameworks.
This thesis presents a systematic and comprehensive study of audio-augmented Temporal Action
Localization, proposing simple yet effective fusion strategies that integrate audio features into existing
video-only TAL pipelines. Motivated by the observation that many actions exhibit distinctive acoustic
signatures, such as the sound of a bowling ball striking pins, a splash during diving, or keystrokes while
typing, we hypothesize that audio can provide complementary cues for improving temporal boundary
precision, reducing background confusion, and disambiguating visually similar actions. To the best of
our knowledge, this work constitutes the first dedicated attempt to incorporate audio modality into fully
supervised TAL in a plug-and-play fashion, without requiring fundamental architectural modifications
to established video-based models.
We introduce two principal fusion strategies: Proposal Fusion and Encoding Fusion. Proposal Fusion operates at the decision level by merging candidate proposals generated independently from audio
and visual streams, followed by Non-Maximum Suppression to obtain refined predictions. In contrast,
Encoding Fusion integrates audio and visual representations at the feature level prior to proposal generation. Within Encoding Fusion, multiple alignment mechanisms are explored to address modality
sequence length mismatches, including concatenation-based fusion, Duplicate-and-Trim, Average-andTrim, and a novel Residual Multimodal Attention (RMAttn) mechanism. The proposed RMAttn module
enables dynamic cross-modal interaction, allowing one modality to selectively attend to informative regions of the other, thereby producing temporally aligned, information-rich multimodal representations.
Our framework is designed to seamlessly integrate with state-of-the-art TAL architectures such as
BMN, BSN, GTAD, and PGCN. Importantly, the core proposal generation and refinement modules
remain unchanged, ensuring generalizability and ease of adoption. Audio features are extracted usingpretrained VGGish embeddings, while visual features are derived from standard backbones such as I3D
and TSN. Extensive experiments are conducted on two large-scale benchmark datasets, ActivityNet1.3 and THUMOS14, using standard mean Average Precision (mAP) evaluation protocols at multiple
Intersection-over-Union (IoU) thresholds.
Experimental results demonstrate consistent and measurable improvements over strong video-only
baselines across both datasets. Our best-performing configurations achieve state-of-the-art performance,
attaining 54.34 mAP@0.5 on ActivityNet-1.3 and 57.18 mAP@0.5 on THUMOS14. Among the proposed strategies, Encoding Fusion with Residual Multimodal Attention shows superior performance
on larger and acoustically diverse datasets, while simple concatenation-based fusion performs competitively in scenarios where audio-visual correlations are weaker. These findings highlight that even
straightforward multimodal integration can yield substantial gains when designed carefully.
Beyond aggregate performance metrics, we conduct comprehensive ablation studies and in-depth
analyses to understand when and why audio contributes most effectively. Class-wise analysis reveals
that actions characterized by distinctive acoustic transitions benefit significantly from audio augmentation. Instance-level analysis further indicates that audio fusion particularly improves detection of
short-duration and low-coverage actions, where visual cues alone may be insufficient. False positive
analysis demonstrates a marked reduction in background errors, suggesting that audio signals enhance
the model's ability to discriminate between action and non-action segments in untrimmed videos.
In addition to algorithmic contributions, this thesis presents a Graphical User Interface (GUI) tool
for interactive visualization and qualitative analysis of TAL predictions. The tool supports synchronized video playback, timeline-based visualization of ground truth and predicted segments, multimodal
comparison, and detailed error inspection. This system facilitates granular

Abstract

Clustering is the task of grouping a set of objects such that objects in the same group (often
called clusters) are more similar to each other than to those in other groups. It is the main task
of exploratory data mining and a common technique for statistical data analysis, used in many
fields, including pattern recognition, image analysis, information retrieval, bioinformatics, data
compression, computer graphics, and machine learning.
With increasing dimensionality, pairwise distances between two data points converge to a
constant--a phenomenon known as distance concentration. This leads to difficulty in identifying nearest neighbors. Further, each new dimension increases the volume of the feature space,
giving data points higher differentiation and thus greater likelihood of becoming more spread
out in higher-dimensional spaces than in lower-dimensional ones. Consequently, estimating
density reliably and detecting cluster boundaries becomes challenging. This phenomenon of
distance concentration and data sparsity in higher dimensions makes clustering particularly difficult. Another challenge is setting appropriate parameter values for clustering algorithms. We
require algorithms that are robust against changes in parameter values and dimensionality increases, ensuring greater confidence in clustering results while remaining scalable with reduced
computing time. This dissertation addresses these challenges and provides better solutions.
It is vital to understand the nature of points in any dataset to conceptualize algorithms meeting these requirements. We observe that reverse nearest neighbors efficiently estimate dense
and sparse regions in higher dimensions. We base our understanding of points on reverse nearest neighbors, using different thresholds to study their impact on clustering results. We study
their properties through experiments on real and synthetic datasets. This analysis identifies
points possessing favorable qualities for understanding clusters. Furthermore, we use these
points to formulate the Strongly Connected Components Based Clustering (SCCC) algorithm.
We further formulate an optimization problem whose solution yields optimal parameters for
the SCCC algorithm, removing manual parameter identification version of SCCC. We present
extensive experimental evaluation comparing our approach against state-of-the-art clustering
algorithms across various real-world datasets. The results demonstrate that our algorithm, with
automatically generated parameters, achieves comparable or superior clustering quality compared to competing methods requiring hand-tuned parameters, particularly in high-dimensional
settings.Furthermore, we extend our method with an online variant of the SCCC algorithm, which
leverages these identified points to handle streaming data efficiently while preserving clustering
performance. Finally, we propose a technique for generating representative points based on
core points and use this representation to develop a sublinear version of the SCCC algorithm,
offering improved scalability without compromising result quality for large datasets.

Abstract

Can machine learning algorithms be implemented using chemistry? We demonstrate that this is
possible in the case of support vector machines (SVMs). SVMs are powerful tools for data classification,
leveraging Vapnik-Chervonenkis theory to handle high-dimensional data and small datasets effectively.
In this thesis, we propose a chemical reaction network scheme for implementing SVMs, utilizing the
steady-state behavior of reaction network dynamics to model key computational aspects of SVMs. This
approach introduces a novel biochemical framework for implementing machine learning algorithms in
non-traditional computational environments.

Abstract

Molecular dynamics (MD) simulations provide a powerful framework for studying the time evolution of molecular systems, enabling the characterization of both thermodynamic and kinetic properties.
However, many biologically and chemically relevant processes, including protein conformational transitions and ligand-binding events, occur on timescales that remain inaccessible to conventional atomistic
simulations. Monte Carlo and enhanced sampling methods can efficiently explore configurational space
and accurately reproduce equilibrium distributions, but they do so at the expense of physical time,
thereby limiting access to kinetic information. This thesis investigates whether physically meaningful molecular trajectories and dynamical information can be recovered without performing extensive
molecular dynamics simulations. Three complementary approaches are developed and evaluated, each
relying on data that is substantially less expensive to generate than fully converged MD trajectories.
The first approach introduces a velocity-based trajectory reconstruction scheme that seeks to recover
the temporal ordering of shuffled molecular dynamics frames. The method combines Maxwell-Boltzmann
velocity likelihoods with a velocity continuity criterion, and a greedy optimization algorithm is employed to identify the most probable sequence of configurations. The approach performs well for simple, low-dimensional systems such as n-pentane, although its effectiveness decreases for more complex
molecular systems.
The second and principal contribution of this thesis is DIME (Dynamics Inferred from Monte Carlo
Ensembles), a framework that reconstructs molecular kinetics from static, Boltzmann-weighted ensembles. Metastable states are identified using density-based clustering, and a rate matrix satisfying detailed
balance is generated by construction, with the transition rate between two states set by the free energy
barrier separating them, read off the free energy surface of the same ensemble. Absolute timescales
are calibrated using a short MD reference trajectory. DIME is validated on six systems of increasing
difficulty: n-pentane and alanine dipeptide as low-dimensional benchmarks, a WLALL pentapeptide,
the achiral peptide AIB9, whose left- and right-handed helices are related by an exact symmetry that
provides a reference-free test of the inferred kinetics, wild-type chignolin as a beta-hairpin folder, and
the millisecond native-state dynamics of bovine pancreatic trypsin inhibitor. Across systems whose
slow processes span picoseconds to a millisecond, the inferred models reproduce MD-derived rate matrices, mean first-passage times, state occupancy autocorrelation functions, and implied timescales with
quantitative accuracy following calibration.
The third component of this work explores the use of deep generative models for trajectory generation in continuous configurational space. Normalizing flows trained on molecular configurations are
v
vi
employed to interpolate within latent space and generate low-barrier transition pathways. In a second approach, continuous normalizing flows are combined with Hamiltonian Neural Networks in latent
space to produce dynamical trajectories. Both methods are demonstrated on the Muller-Brown potential ¨
energy surface.
Overall, the results presented in this thesis demonstrate that meaningful kinetic information can
be extracted from computational approaches that require substantially less effort than fully converged
molecular dynamics simulations. Among the methods developed, DIME provides the most comprehensive and rigorously validated framework for recovering molecular kinetics from equilibrium ensembles.

Abstract

In this thesis, we study extinction in dynamical systems generated by reaction networks. We introduce two notions: weak extinction and strong extinction, and relate them to the structure of the underlying network through Lyapunov functions and LaSalle's invariance principle. In particular, for all
deficiency-zero networks that are not weakly reversible, we provide a geometric construction of linear
Lyapunov functions. Using these functions, we establish that if these networks have bounded invariant
subspaces, then they must exhibit weak extinction within every such subspace. Also, for linear networks
that are not weakly reversible, we show that every species outside a terminal strongly connected component undergoes strong extinction. Moreover, in order to further emphasize the difference between weak
and strong extinction, we construct an example of a reaction system (based on the Ivanova network) that
exhibits weak extinction for all the species, but does not exhibit strong extinction in any species.

Abstract

Descent methods form a fundamental class of algorithms for solving unconstrained optimization
problems. The most basic among them is the steepest descent method, where one moves in the direction
of the negative gradient of the objective function. Newton's method, on the other hand, is a secondorder approach that uses curvature information through the Hessian to select more informed descent
directions. In this work, we introduce Constrained Gradient Descent (CGD), a gradient-regularized
descent method that modifies the descent direction by reshaping the optimization landscape.
CGD is motivated by considering a constrained version of the original optimization problem, where
the constraint is imposed on a function of the gradient. Instead of solving the constrained problem directly, we incorporate this constraint through a Lagrangian penalty leading to an unconstrained gradientregularized objective. Gradient descent on this modified objective yields an update direction controlled
by the regularization parameter, allowing CGD to incorporate gradient-norm information and encouraging movement through regions with smaller gradient magnitudes compared to standard steepest descent.
We establish global linear convergence guarantees for CGD under suitable assumptions and validate its performance through extensive numerical experiments on the practical variants, CGD-FD (a
Hessian-free implementation) and CGD-QN (a quasi-Newton implementation), both of which demonstrate promising empirical performance on synthetic benchmark functions. We additionally evaluate
CGD in a neural-network training setting, where the proposed methods remain competitive with the
baselines, providing further evidence of its practical applicability.
Building on these results, we further explore theoretical interpretations and extensions of CGD. In
particular, we interpret CGD as a gradient descent method under a matrix-induced norm and also place
it within the broader mirror descent framework, thereby highlighting its geometric underpinnings.
Finally, we investigate the applicability of CGD in the context of Bayesian optimization. In this
setting, CGD is used to modify the optimization of acquisition functions, incorporating gradient-based
regularization into the acquisition optimization step.
Through these developments, CGD is positioned as a versatile framework connecting classical descent methods with modern perspectives on geometry, regularization and black-box optimization.

Abstract

Modern speech technology often relies on orthographic text such as transcripts, spellings, and characterlevel edit distances, even though human listeners judge speech by phonological and semantic content.
For morphologically rich and low-resource languages, this orthographic bottleneck produces unreliable evaluation and weak pronunciation modelling. This thesis explores phonetic abstraction: the
deliberate reduction of surface script to shared phonological structure before comparison or transfer.
Automatic Speech Recognition (ASR) evaluation for morphologically complex languages remains
an open and practically significant problem. Standard metrics such as Word Error Rate (WER) and
Character Error Rate (CER) operate purely at the lexical level, assigning uniform penalties to all substitutions regardless of whether the resulting output is intelligible to a human listener. In agglutinative and
fusional languages with rich phonological inventories, such as Tamil, Telugu, Hindi, and Marathi, these
metrics routinely misalign with human intelligibility.
This thesis proposes and evaluates a semantic-phonetic hybrid evaluation framework that combines a deterministic phonetic skeleton, derived by mapping consonant characters across languagespecific Indic scripts into shared articulatory equivalence classes, with contextual semantic embeddings
from large pre-trained multilingual models. A linear combination of these two components, optimised
via Ordinary Least Squares regression against expert human scores on 1,287 challenging transcription
pairs, yields a metric that consistently outperforms string-edit and LLM-based baselines across all evaluated languages.
Arabic, Persian, and Urdu share phonology but not computational resources. As a complementary contribution, the thesis also presents a Common Phone Space (CPS-APU) for Arabic, Persian,
and Urdu, which applies cross-lingual phonological transfer to the problem of Grapheme-to-Phoneme
conversion for low-resource Perso-Arabic languages. It is a three-stage Grapheme-to-Phoneme (G2P)
pipeline (expert rules, corpus-mined rules, Large Language Model (LLM)-powered generation), and
shows that cross-lingual transfer from high-resource Arabic improves G2P for low-resource Persian and
Urdu.
Together, these works argue that underlying phonetic and phonological structures offer a robust computational resource that can be systematically exploited to overcome text-level limitations. Both contributions instantiate the same principle: related languages share phonological structure that can be
encoded as equivalence classes (evaluation-time bucketing in Indic scripts; CPS labels in Perso-Arabic
G2P). Phonetic abstraction is a transferable resource for building fair and effective speech technology
for the world's linguistic diversity beyond surface orthography.

Abstract

The classical setting of private information retrieval (PIR) involves the retrieval of one desired message by a client, from a library of K messages stored across N distributed servers, without revealing the
identity of the desired file from any particular server. In graph-based replication, the storage configuration is described by a (simple, undirected) graph with N vertices and K edges, where the edges denote
the files and the nodes denote the servers. Each edge corresponds to a single distinct file, stored exactly
in the two servers indexed by the vertices that are its end points. A query-response protocol between the
client and the servers which enforces (a) the client is able to retrieve his desired file and (b) the identity
of the desired file is not revealed to any individual server constitutes a valid PIR scheme. Such protocols
involve utilizing private randomness at the client, which enables the masking of the index of the desired
file from the server.
The rate of PIR for a given storage scenario is the ratio of the number of symbols in a file, to the
number of symbols downloaded from all servers by the client for successful recovery of the desired
file. The optimal (largest) rate of PIR for a given setting is known as the PIR capacity. Prior work
in high-rate PIR for graph-based systems typically require large subpacketization (which indicates the
number of parts each file has to be divided into, indicating the complexity of the scheme). A good
number of these schemes are fixed-download schemes, i.e., they have the property that the number of
bits downloaded from each server remains unchanged, irrespective of the query realization. In this
thesis, we present four high-rate PIR schemes for graph-based storage systems.
• A unit-subpacketization (which is minimal possible) variable-download scheme on star graphs.
• A fixed-download scheme on star graphs.
• A variable-download scheme for general graphs.
• A fixed-download scheme for general graphs.
The fixed-download scheme for general graphs is obtained from the variable-download scheme, via a
generic method of transformation, known from literature.
Our variable-download PIR scheme on star graphs is the first scheme in the literature that employs
unit subpacketization (L = 1) while also having orderwise optimal rate. Our PIR schemes for general
graphs use a decomposition of the graph via independent sets. They achieve rates lower than prior
schemes for the complete graph, however they can achieve higher rates than previously known for some
specific graph classes.

Abstract

Functional materials with tailored properties are essential for advancements in crucial areas such
as energy storage and semiconductor technologies. Despite progress in generative models, producing
materials for specific target properties remains challenging because of their vast chemical space and
complexities introduced by atomic structures and material properties.
In this work, an approach to the generative design of inorganic materials is introduced, employing a string-based invertible representation called the Simplified Line-Input Crystal Encoding System
(SLICES). This representation adheres to crystallographic invariances, facilitating accurate modeling
and reversible generation of crystal structures. A transformer decoder model with approximately 100
million parameters is integrated with a property-guided generation mechanism based on scaled propertytoken embeddings, following the continuous value encoding methodology. For the first time, special
tokens corresponding to material properties (formation energy, band gap, bulk modulus, and crystal
system) are introduced as control codes, and the embedding of each token is scaled by the desired
property value, allowing fine-grained and continuous control over the generated structures. Unlike
prior approaches such as MatterGPT, which concatenated property embeddings with SLICES tokens,
or SLICES-PLUS, which required architectural modifications and an enhanced crystal representation
for symmetry control, the present work achieves both property and crystal system conditioning within a
single unified framework.
Trained on the Alex-20 dataset (280,033 structures) for formation energy and band gap, and on the
Materials Project dataset (13,024 structures) for bulk modulus, the model consistently achieved validity
exceeding 90% across single-property generation experiments, with competitive uniqueness and novelty.
The framework was extended to dual-property conditioning, demonstrating simultaneous targeting of
formation energy with bulk modulus and formation energy with band gap across broad grids of target
combinations. Crystal system conditioning was further introduced as a new capability, achieving match
rates of up to 93.44% for triclinic systems and 57.06% for cubic systems when co-conditioned with
formation energy, and generalizing consistently across band gap and bulk modulus conditioning as
well. A multi-step validation strategy incorporating structural and compositional plausibility checks,
predictive property assessments via the MEGNet model followed by Density Functional Theory (DFT)
calculations confirmed the physical realism of the generated structures. The results show that MatGPT
generates valid, unique, and novel inorganic crystal structures while enabling controlled generation
toward prescribed formation energy, band gap, bulk modulus, and crystal system targets. Representative
DFT relaxations further indicate that the generated structures can retain their structural integrity after optimization, supporting the physical plausibility of the proposed materials. These results establish
scaled property-token conditioning as a simple and unified strategy for controllable inverse design of
inorganic crystals.
In addition, an agentic workflow for materials science is presented, integrating the Qwen language
model with domain-specific computational tools through a modular Model Context Protocol (MCP)
architecture. The framework augments the language model with tools for structure relaxation, thermodynamic calculations, and retrieval-augmented generation, and is evaluated on the MaScQA benchmark
for materials science question answering. Together, the generative MatGPT framework and the agentic
verification system establish foundations for end-to-end AI-driven materials discovery pipelines.

Abstract

Automated interpretation of high-resolution satellite imagery using deep learning has advanced
rapidly, yet its application to aviation surface mapping remains limited. Existing studies primarily focus
on roads, buildings, or coarse runway-level features, while the fine-grained surface markings required
for airport ground navigation and aerodrome mapping remain underexplored.
This thesis addresses the automated extraction of fine-grained airport surface features from satellite
imagery, with emphasis on supporting digital moving maps and Aerodrome Mapping Database update
workflows within the aeronautical data update (AIRAC) cycle. Unlike conventional object detection
tasks, airport feature extraction requires grouping spatially contiguous and repeating markings--such
as runway thresholds or chevrons--into semantically meaningful aviation objects.
Two case studies first establish feasibility and expose the core challenges. An initial single-class
detection study demonstrates that machine learning can localize airport features from satellite imagery,
but reveals the limitations of small, domain-specific training data and weak generalization. A second
study extends this to multi-class detection and segmentation, achieving strong performance on image
patches yet failing at full-scene scale, where false positives proliferate and conventional sliding-window
inference breaks down. These failures motivate the Airport Ground Navigation Object Detection
Dataset (AGNODD), which addresses the granularity and domain-specific data gap in existing datasets.
To overcome the limitations of sliding-window inference on large satellite scenes, a multi-level-ofdetail tiling strategy is then developed and extended into RAHI (Raster-Aware Hyper Inference),
a memory-mapped, geometry-preserving inference framework that delivers substantial tiling speedups
while improving detection fidelity over conventional slicing. An exploratory generative-AI-based validation stage is also investigated but found to be unreliable and computationally expensive at scale,
motivating a dedicated model-driven solution.
The proposed AERO-DETR pipeline introduces a dual-perspective approach for runway surface
extraction. A top-down stage employs oriented object detection to identify candidate runway corridors,
while a bottom-up stage uses transformer-based set prediction to assemble precise runway polygons
from orientation-standardized markings. Spatial post-processing enforces shape and adjacency constraints aligned with ICAO aviation standards. Experimental results demonstrate improved full-scene
robustness, reduced false positives compared to tiling-only baselines, and a mean Intersection-overUnion of approximately 0.89 against ground-truth data.
Overall, the thesis demonstrates that aviation-aware computer vision pipelines can enable more reliable and scalable mapping of airport surface features from satellite imagery.