Abstract

The disclosed embodiments illustrate method and system for auto-generation of the sketch notes-based visual summary of the multimedia content. The method includes determining one or more segments, based on one or more transitions in the multimedia content. The method further includes generating a transcript based on audio content associated with each determined segment. The method further includes retrieving a set of images from an image repository based on each of the identified one or more keywords. The method further includes generating a sketch image of each of one or more of the retrieved set of images associated with each of the identified one or more keywords. The method further includes rendering the sketch notes-based visual summary of the multimedia content, generated based on at least generated one or more sketch images, on a user interface displayed on a display screen of the user …

Abstract

The Interactions website (interactions.acm.org) hosts a stable of bloggers who share insights and observations on HCI, often challenging current practices. Each issue we'll publish selected posts from some of the leading and emerging voices in the field.

Abstract

Recommender systems are a key component of music sharing platforms, which suggest musical recordings a user might like. People often have implicit preferences while listening to music, though these preferences might not always be the same while they listen to music at different times. For example, a user might be interested in listening to songs of only a particular artist at some time, and the same user might be interested in the top-rated songs of a genre at another time. In this paper we try to explicitly model the short term preferences of the user with the help of Last.fm tags of the songs the user has listened to. With a session defined as a period of activity surrounded by periods of inactivity, we introduce the concept of a subsession, which is that part of the session wherein the preference of the user does not change much. We assume the user preference might change within a session and a session might have multiple subsessions. We use our modelling of the user preferences to generate recommendations for the next song the user might listen to. Experiments on the user listening histories taken from Last.fm indicate that this approach beats the present methodologies in predicting the next recording a user might listen to.

Abstract

Recommender Systems are an integral part of music sharing platforms. Often the aim of these systems is to increase the time, the user spends on the platform and hence having a high commercial value. The systems which aim at increasing the average time a user spends on the platform often need to recommend songs which the user might want to listen to next at each point in time. This is different from recommendation systems which try to predict the item which might be of interest to the user at some point in the user lifetime but not necessarily in the very near future. Prediction of next song the user might like requires some kind of modeling of the user interests at the given point of time. Attentive neural networks have been exploiting the sequence in which the items were selected by the user to model the implicit short-term interests of the user for the task of next item prediction, however we feel that features of the songs occurring in the sequence could also convey some important information about the short-term user interest which only the items cannot. In this direction we propose a novel attentive neural architecture which in addition to the sequence of items selected by the user, uses the features of these items to better learn the user short-term preferences and recommend next song to the user.

Abstract

The purpose of this study is to examine the differences between different types of news stories. Given the huge impact of social networks, online content plays an important role in forming or changing the opinions of people. Unlike traditional journalism where only certain news organizations can publish content, online journalism has given chance even for individuals to publish. This has its own advantages like individual empowerment but has given a chance to a lot of malicious entities to spread misinformation for their own benefit. As reported by many organizations in recent history, this even has influence on major events like the outcome of elections. Therefore, it is of great importance now, to have some sort of automated classification of news stories. In this work, we propose a deep hierarchical attention neural architecture combining sentiment and semantic embeddings for more accurate fine grained classification of news stories. Experimental results show that the sentiment embedding along with semantic information outperform several state-of-the art methods in this task.

Abstract

Variational autoencoders were proven successful in domains such as computer vision and speech processing. Their adoption for modeling user preferences is still unexplored, although recently it is starting to gain attention in the current literature. In this work, we propose a model which extends variational autoencoders by exploiting the rich information present in the past preference history. We introduce a recurrent version of the VAE, where instead of passing a subset of the whole history regardless of temporal dependencies, we rather pass the consumption sequence subset through a recurrent neural network. At each time-step of the RNN, the sequence is fed through a series of fully-connected layers, the output of which models the probability distribution of the most likely future preferences. We show that handling temporal information is crucial for improving the accuracy of the VAE: In fact, our model beats the current state-of-the-art by valuable margins because of its ability to capture temporal dependencies among the user-consumption sequence using the recurrent encoder still keeping the fundamentals of variational autoencoders intact.

Abstract

Ensemble POS taggers are a good choice to integrate and leverage benefits of various types of POS taggers. This can help the large number (6500+) of resource-poor languages which do not have much annotated training data by providing ways to integrate semi-supervised/unsupervised taggers with supervised taggers. In this paper we present our experiments of developing ensemble POS taggers using a decision tree. We integrate a semi-supervised data mining approach that uses context based lists (CBLs) for POS tagging with supervised (1) Support Vector Machine based POS tagger, called SVMTool and (2) Conditional Random Field based POS tagger. The results are enhanced semi-supervised ensemble POS taggers which outperform the base methods. In these POS taggers, we use a decision tree to decide when to rely on the output of supervised tagger, and when to rely on the semi-supervised CBL method. The CBL based tagger uses rich contextual information which helps in tagging both existing and unseen words and uses no domain knowledge while supervised taggers give good performance for words present in the training model and can include domain based features. Hence, these algorithms have complementary strengths and in our ensemble we are able to combine these strengths. Enhanced performance of our new POS taggers over the base methods suggests that integrating these methods combines the qualities of these in the new tagger which enhances the performance. Therefore, these new semi-supervised ensemble taggers are more suitable for resource-poor languages.

Abstract

Encoding models can reveal and decode neural representations in the visual and semantic domains. However, a thorough understanding of how distributed information in auditory cortices and temporal evolution of music contribute to model performance is still lacking in the musical domain. We measured fMRI responses during naturalistic music listening and constructed a two-stage approach that frst mapped musical features in auditory cortices and then decoded novel musical pieces. We then probed the infuence of stimuli duration (number of time points) and spatial extent (number of voxels) on decoding accuracy. Our approach revealed a linear increase in accuracy with duration and a point of optimal model performance for the spatial extent. We further showed that Shannon entropy is a driving factor, boosting accuracy up to 95% for music with highest information content. These fndings provide key insights for future decoding and reconstruction algorithms and open new venues for possible clinical applications.

Abstract

There has been growing interest towards naturalistic neuroimaging experiments, which deepen our understanding of how human brain processes and integrates incoming streams of multifaceted sensory information, as commonly occurs in real world. Music is a good example of such complex continuous phenomenon. In a few recent fMRI studies examining neural correlates of music in continuous listening settings, multiple perceptual attributes of music stimulus were represented by a set of high-level features, produced as the linear combination of the acoustic descriptors computationally extracted from the stimulus audio. New method: fMRI data from naturalistic music listening experiment were employed here. Kernel principal component analysis (KPCA) was applied to acoustic descriptors extracted from the stimulus audio to generate a set of nonlinear stimulus features. Subsequently, perceptual and neural correlates of the generated high-level features were examined.

Abstract

The main challenge in studying cognition & designing effective tasks for children with learning disorders is creating personalized & adaptive tasks in line with the current abilities & mood of the child. The current study confronts this challenge by testing a new paradigm to access the current state of mind and adapting the tasks based on the current mood & abilities of the child. Children were given chess puzzles with various levels of difficulty (from just identifying the pieces, legal moves and eventually even capturing pieces with depth=1). while the children were performing the tasks the pupil-metric data (for cognitive load), facial expressions and the head pose were used to gauge the current-state and adapt the puzzles accordingly. Further development of dynamic feedback and providing rewards for looking at the right squares are also underway. custom software with off the shelf web-cameras were used as the current solutions in the market are prohibitively expensive for testing on large scale.