Abstract

Text as signs, labels, or instructions is a critical element of real-world scenes as they can convey important contextual information. 3D representations such as 3D Gaussian Splatting (3DGS) struggle to preserve fine-grained text details, while achieving high visual fidelity. Small errors in textual element reconstruction can lead to significant semantic loss. We propose STRinGS, a text-aware, selective refinement framework to address this issue for 3DGS reconstruction. Our method treats text and non-text regions separately, refining text regions first and merging them with non-text regions later for full-scene optimization. STRinGS produces sharp, readable text even in challenging configurations. We introduce a text readability measure OCR Character Error Rate (CER) to evaluate the efficacy on text regions. STRinGS results in a 63.6% relative improvement over 3DGS at just 7K iterations. We also introduce a curated dataset STRinGS-360 with diverse text scenarios to evaluate text readability in 3D reconstruction. Our method and dataset together push the boundaries of 3D scene understanding in text-rich environments, paving the way for more robust text-aware reconstruction methods.

Abstract

Neural Style Transfer (NST) traditionally applies a fixed style to a
target image, offering limited control over its degree, progression, or
variations. Recent methods have introduced various control mech-
anisms like spatial attention, entropy regularization, and latent
space manipulation. However, these approaches lack interpretable,
human-understandable controls for fine-grained stylistic control as
intermediate stylization is typically achieved by adjusting content
and style losses. Other works explore interpolation across feature
CAV Styler, a human-interpretable style transfer approach. We ex-
ploit human interpretable concepts such as textures, blurs, and style
details to enable fine-grained control over stylization. To our knowl-
edge, ours is among the first in image-to-image style transfer to offer
explicit user guidance through semantically meaningful parame-
ters. Our method demonstrates superior controllability compared
to existing approaches while maintaining competitive stylization
quality, opening new possibilities for intuitive and precise artistic
style manipulation.
layers, but they operate through abstract parameters that require
domain expertise and often lack intuitive control, as the relation-
ship between parameters and stylization remains unclear due to
the black-box nature of the architecture. In this paper, we introduce

Abstract

Existing self-supervised mesh retrieval methods suffer from sensitivity to mesh triangle ordering, limiting real-world deployment where meshes from different tools have arbitrary face orderings. We address this fundamental limitation by leveraging mesh convolutions for robust, efficient retrieval that achieves permutation and rotation invariance. Our method learns class-agnostic, shape-aware embeddings that capture geometric similarity without semantic bias, enabling retrieval of geometrically similar objects across different categories. Extensive experiments demonstrate that while existing methods degrade drastically under permutation (>50% performance loss), our method maintains consistent performance (94.22% vs 94.87% mAP) with 200x fewer parameters (1.34M vs 278M), enabling practical deployment across diverse mesh sources.

Abstract

A system and processor-implemented method for three-dimensional (3D) object retrieval using a self-supervised model is provided. The present system learns an embedding space of the 3D mesh objects in a self-supervised manner without the need for objects annotated with their class or other properties. Effective embeddings of 3D mesh objects are learned using the self-supervised method for ranked retrieval from a large collection of 3D objects. A simple representation of mesh objects and a standard neural network model is used to learn the embedding. The results are retrieved on the basis of the shape of the object which may not belong to the same category but look similar in shape using the embeddings generated by self-supervised model. The system is independent of class labels and uses the entire 3D model for better information extraction.

Abstract

Digital 3D models play a pivotal role in engineering, entertainment, education, and various domains. However, the search and retrieval of these models have not received adequate attention compared to other digital assets like documents and images. Traditional supervised methods face challenges in scalability due to the impracticality of creating large, labeled collections of 3D objects. In response, this paper introduces a self-supervised approach to generate efficient embeddings for 3D mesh objects, facilitating ranked retrieval of similar objects. The proposed method employs a straightforward representation of mesh objects and utilizes an encoder-decoder architecture to learn the embedding. Extensive experiments demonstrate the competitiveness of our approach compared to supervised methods, showcasing its scalability across diverse object collections. Notably, the method exhibits transferability across datasets, implying its potential for broader applicability beyond the training dataset. The robustness and generalization capabilities of the proposed method are substantiated through experiments conducted on varied datasets. These findings underscore the efficacy of the approach in capturing underlying patterns and features, independent of dataset-specific nuances. This self-supervised framework offers a promising solution for enhancing the search and retrieval of 3D models, addressing key challenges in scalability and dataset transferability.

Abstract

Deep learning models are susceptible to backdoor attacks
involving malicious attackers perturbing a small subset of
training data with a trigger to causes misclassifications.
Various triggers have been used including semantic triggers that are easily realizable without requiring attacker
to manipulate the image. The emergence of generative AI
has eased generation of varied poisoned samples. Robustness across types of triggers is crucial to effective defense.
We propose Prototype Guided Backdoor Defense (PGBD),
a robust post-hoc defense that scales across different trigger types, including previously unsolved semantic triggers.
PGBD exploits displacements in the geometric spaces of activations to penalize movements towards the trigger. This
is done using a novel sanitization loss of a post-hoc finetuning step. The geometric approach scales easily to all
types of attacks. PGBD achieves better performance across
all settings. We also present the first defense against a new
semantic attack on celebrity face images.

Abstract

Creating photorealistic, animatable, and relightable 3D head avatars traditionally requires expensive Lightstage with multiple calibrated cameras, making it inaccessible for widespread adoption. To bridge this gap, we present a novel, cost-effective approach for creating high-quality relightable head avatars using only a smartphone equipped with polaroid filters. Our approach involves simultaneously capturing cross-polarized and parallel-polarized video streams in a dark room with a single point-light source, separating the skin's diffuse and specular components during dynamic facial performances. We introduce a hybrid representation that embeds 2D Gaussians in the UV space of a parametric head model, facilitating efficient real-time rendering while preserving high-fidelity geometric details. Our learning-based neural analysis-by-synthesis pipeline decouples pose and expression-dependent geometrical offsets from appearance, decomposing the surface into albedo, normal, and specular UV texture maps, along with the environment maps. We collect a unique dataset of various subjects performing diverse facial expressions and head movements.

Abstract

Linearly Transformed Spherical Distributions (LTSDs), a superset of the commonly known Linearly Transformed Cosines (LTCs) for analytic area light rendering, applied in the context of analytic area light rendering with participating media.

Abstract

Large-scale capturing of real-world scenes as 3D point clouds (e.g., using LIDAR scanning) generates billions
of points that are challenging to visualize. High storage requirements prevent the quick and easy inspection
of captured datasets on user-grade hardware. The fastest real-time rendering methods are limited by the
available GPU memory and render only around 1 billion points interactively. We show that we can achieve
state-of-the-art in both while simultaneously supporting datasets that surpass the capabilities of other methods.
We present an on-the-fly point cloud decompression scheme that tightly integrates with software rasterization
to reduce on-chip memory requirements by more than 4×. Our method compresses geometry losslessly and
provides high visual quality at real-time framerates. We use a GPU-friendly, clipped Huffman encoding for
compression. Point clouds are divided into equal-sized batches, which are Huffman-encoded independently.
Batches are further subdivided to form easy-to-consume streams of data for massively parallel execution.
The compressed point clouds are stored in an access-aware manner to achieve coherent GPU memory access

Abstract

The complex, glinty appearance of detailed normal-mapped surfaces at different scales requires expensive per-pixel Normal Distribution Function computations. Moreover, large light sources further compound this integration and increase the noise in the Monte Carlo renderer. Specialized rendering techniques that explicitly express the underlying normal distribution have been developed to improve performance for glinty surfaces controlled by a fixed material roughness. We present a new method that supports spatially varying roughness based on a neural histogram that computes per-pixel NDFs with arbitrary positions and sizes. Our representation is both memory and compute efficient. Additionally, we fully integrate direct illumination for all light directions in constant time. Our approach decouples roughness and normal distribution, allowing the live editing of the spatially varying roughness of complex normal-mapped objects. We demonstrate that our approach improves on previous work by achieving smaller footprints while offering GPU-friendly computation and compact representation.