Abstract

A system and a method for recommending background music that corresponds to an extracted text from a book based on emotion and a topic that is relevant to the extracted text using machine learning models provided. The method includes,(i) determining, using a first trained machine learning model, the emotion from the extracted text that corresponds to the paragraph of the book,(ii) assigning, using a word similarity technique, a similarity score for

Abstract

Personalized Interactive Music Systems (PIMSs) are emerging as promising devices for enhancing physical activity and exercise outcomes. By leveraging real-time data and adaptive technologies, PIMSs align musical features, such as tempo and genre, with users' physical activity patterns, including frequency and intensity, enhancing their overall experience.

Abstract

Background
Despite showing remarkable performance on tasks across modalities, deep neural networks remain opaque, with limited insight on how they internally organize information. A learned hierarchy is suggested in vision (Zhou et al., 2015) and language (Tenney et al., 2019) models in deep learning.
Bregman (1990) describes the process of auditory scene analysis (ASA), where the auditory system abstracts information based on perceptual properties such as pitch and timbre, among others, in order to construct auditory streams that are then integrated to form higher order percepts like musical genre. This implies that the representation of audio goes through a series of hierarchical abstractions in the process. While hierarchical representations are indicated in human auditory processing (Kell et al., 2018), the same has not been thoroughly demonstrated in deep learning models for audio.

Aims
We study how convolutional neural networks (CNNs) represent the hierarchy of audio tasks, hypothesizing that representations at earlier layers of a CNN perform better at lower level tasks, and those at later layers perform better at higher level tasks.

Methods
On the basis of Bregman's ASA model, we choose tasks that are arranged in a hierarchy from the domains of speech and music. We select three hierarchical musical tasks - note identification, instrument classification, and genre classification, representative of low-, mid-, and high-level tasks. We use class-balanced subsets of NSynth (Engel et al., 2017) with 1800 instances across 12 classes, Medley-solos-DB (Lostanlen et al., 2019) with 965 instances across 7 classes , and GTZAN (Tzanetakis et al., 2001) with 1000 instances across 10 classes, for each of the respective tasks.
We inspect three models - VGGish (Hershey et al., 2017), CLAP (Elizalde et al., 2023), and MobileNetV3 (Schmid et al., 2023). We use k-Nearest Neighbour classifiers with class-balanced five-fold cross-validation to assess the model's accuracy on each task with intermediate representations extracted at six equally spaced convolutional blocks and the first fully-connected layer. We repeat this for three hierarchically related speech tasks - consonant classification, keyword recognition, and speaker count estimation, using similarly class-balanced subsets of PCVC (Malekzadeh et al., 2020) with 1794 instances across 23 classes, Speech Commands (Warden, 2018) with 1750 instances across 35 classes, LibriCount (Stöter et al., 2018) with 1100 instances across 11 classes respectively.

Results
Our results show that the CNN's early layers learned low-level tasks and later layers learned high-level tasks even without explicit training. On CLAP, for note identification, the first layer's accuracy (47%) is greater than later layers (<30%). For instrument classification, we see peak accuracy at the last layer (97%), but a remarkable change occurs at the fourth layer (71%) compared to the third layer (55%). Genre classification accuracy at the last layer (75%) is better than earlier layers (<66%). These trends hold across all models. We see similar trends for the speech tasks.

Discussion
The results support our hypothesis, providing strong evidence that CNNs implicitly learn a hierarchy in sound. This mirrors the human brain's hierarchical encoding and deserves further investigation if other deep learning architectures and training methodologies also encode similar hierarchies. We also see how low-level tasks are implicitly learnt.

Abstract

Background

Music Emotion Recognition (MER) systems primarily rely on audio features (Wang et al., 2021), with recent approaches incorporating lyrics to analyze sentiment and structure for improved accuracy (Agrawal et al., 2021). Musical content, particularly melody, has been found to have a stronger capacity to convey perceived emotional expression than lyrics, although lyrics tend to enhance negative emotions more easily than positive emotions (Ali & Peynircioğlu, 2006). This variability raises important questions: Which component carries the greatest emotional information? Do listeners rely on music, vocals, or lyrics when assessing emotional content?


Aim

Here, we systematically analyse the contribution of music, vocals, and lyrics to perception of emotion in music.


Methods

For this study, we used the DEAM (Soleymani et al., 2013) and PMEmo (Zhang et al., 2018) datasets, comprising a total of 2,700 songs. The DEAM dataset includes 1,802 audio items annotated for both dynamic and static emotion using continuous valence and arousal ratings. Participants utilised a two-dimensional interface based on the Self‑Assessment Manikins (SAM) to continuously rate the emotional content at a 2‑Hz sampling rate. The PMEmo dataset contains 794 popular music choruses sourced from international music charts (Billboard Hot 100, iTunes Top 100, and UK Top 40) and was annotated using a similar interface to capture continuous dynamic ratings for valence and arousal at the same sampling rate.


Songs were source-separated using Ultimate Vocal Remover (Takahashi et al., 2017) to isolate music and vocals, while lyrics were transcribed using OpenAI's Whisper model (Radford et al., 2022) and verified manually. Deep learning models trained on diverse song and speech datasets predicted VA values for musical and vocal components, while lyrics were analyzed using models trained on general and lyrically annotated texts (Çano, 2017). Spearman's correlation was calculated between predicted and human-annotated (HA) VA values. Additionally, quadrant-based (Q1: positive V, high A; Q2: negative V, high A; Q3: negative V, low A; Q4: positive V, low A) concurrency analyses evaluated prediction alignment with human-identified emotional quadrants.


Results

The musical component exhibited the highest correlation with HA ratings (valence: r = 0.70, arousal: r = 0.74), followed by vocals (valence: r = 0.54, arousal: r = 0.65; all p < .001), while lyrics contributed the least (valence: r = 0.11, arousal: r = 0.01). In terms of overall quadrant concurrency, the musical component showed the highest concurrency (60.41%) followed by vocals (48.84%) and then lyrics (31.45%). Specifically, the highest quadrant concurrency was observed in Q3 (musical: 90.53%, vocal: 93.53%, lyrics: 66.16%) followed by Q1 for musical (64.51%) and vocals (39.10%), and Q4 for lyrics (42.50%).


Discussion

Our findings emphasize the dominant role of the musical component in shaping perceived emotional expression in Western tonal music, aligning with prior work highlighting melody's emotional salience (Ali & Peynircioğlu, 2006). Furthermore, for music signifying negative valence and low arousal (Q3), there is a higher degree of congruence between the components in conveying sadness and its related emotions. These results suggest that MER systems could prioritize musical and vocal components over lyrical content (Wang et al., 2021; Agrawal et al., 2021), as they appear to primarily drive the perception of emotional expression.


References

Agrawal, Y., Shanker, R. G. R., & Alluri, V. (2021). Transformer-based approach towards music emotion recognition from lyrics. In V. U. Rao & R. Kailash (Eds.), Advances in information retrieval (pp. 167-175). Springer. https://doi.org/10.1007/978-3-030-72240-1_12


Ali, S. O., & Peynircioğlu, Z. F. (2006). Songs and emotions: Are lyrics and melodies equal partners? Psychology of Music, 34(4), 511-534. https://doi.org/10.1177/0305735606067168


Çano, E. (2017). MoodyLyrics: A sentiment-annotated lyrics dataset. In Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2017) (pp. 118-124). Association for Computational Linguistics. https://doi.org/10.18653/v1/S17-1017


Radford, A., Kim, J. W., Xu, T., Brockman, G., McLeavey, C., & Sutskever, I. (2022). Robust speech recognition via large-scale weak supervision. arXiv. https://doi.org/10.48550/arXiv.2212.04356


Soleymani, M., Caro, M., Schmidt, E., Sha, C.-Y., & Yang, Y.-H. (2013). 1000 songs for emotional analysis of music. In P. Herrera, E. Gómez, & T. Kaneko (Eds.), Proceedings of the 2nd ACM International Workshop on Music Information Retrieval with User-Centered and Multimodal Strategies (pp. 1-6). Association for Computing Machinery.


Takahashi, N., Huang, P.-S., & Lee, P. (2017). Multi-scale multi-band DenseNets for audio source separation. arXiv. https://doi.org/10.48550/arXiv.1706.09588


Wang, S., Xu, C., Ding, A. S., & Tang, Z. (2021). A Novel Emotion-Aware Hybrid Music Recommendation Method Using Deep Neural Network. Electronics, 10(15), 1769. https://doi.org/10.3390/electronics10151769


Zhang, K., Zhang, H., Li, S., Yang, C., & Sun, L. (2018). The PMEmo dataset for music emotion recognition. In Proceedings of the 2018 ACM International Conference on Multimedia Retrieval (pp. 135-142). Association for Computing Machinery. https://doi.org/10.1145/3206025.3206037

Abstract

Background

Heggli et al. (2021) noted that circadian rhythms and individual traits influence diurnal patterns in music preference. Depressive individuals often experience more negative moods in the evening, known as the evening-worsening effect (Rusting & Larsen, 1998). As music is often used for emotion focused-coping during such mood states (Stewart et al., 2019), depressive individuals' music consumption in the evening are unclear.

Aims

This study explores the evening-worsening effect in music listening, examining if depressive individuals prefer low-valence, low-arousal music in the evening.

Methods

A survey of 290 Indians (M=20.35yrs, SD=2.08, 211 males) collected Kessler Psychological Distress scores (K10) and Healthy-Unhealthy music scale (HUMS) scores along with Spotify listening histories over one year. Additional measures like trait empathy and life satisfaction were collected but not analysed. Participants were classified as No-Risk (K10<20, n=114, 82 males) or At-Risk (K10≥29, n=81, 52 males) of depression (Andrews & Slade, 2001). Spotify valence and energy features for top 100-250 songs over the past 4-8 weeks from the time they filled in the survey were analyzed alongside timestamps. K-Means clustering divided the day into early hours, morning, afternoon and evening, and group differences in hourly valence and energy averages were assessed using two-way ANOVA and t-tests.

Results

At-Risk had significantly higher Unhealthy scores than No-Risk (t=8.15, p<0.001). Significant differences in valence and energy were found between At-Risk and No-Risk only in the afternoon (valence: t=-9.47, energy: t=-6.53) and evening (valence: t=-2.98, energy: t=-11.56)(all p≤ 0.05). Two-way ANOVA showed significant main effects of time, group, and their interaction for valence (time: F(3, 40)=22.86; group: F(1, 40)=12.36; interaction: F(3, 40)=6.65) and energy (time: F(3, 40)=23.22; group: F(1, 40)=98.73; interaction: F(3, 40)=13.57)(all p<0.001), consistent across the top 100-250 songs and the past 4-8 weeks of listening histories.

Conclusions

This study confirms diurnal patterns in music preferences, with At-Risk individuals favouring low-valence and low-energy music, especially in the afternoon and evening. These results align with Rusting and Larsen's (1998) evening-worsening effect, linking depressive tendencies to heightened negative moods in the evening. The findings suggest potential maladaptive listening behaviors, needing further research.

Abstract

Background and Aims
Trait empathy, measured by the Interpersonal Reactivity Index (IRI; Davis, 1980), influences musical preferences and emotional responses to music. Individuals high on the IRI subscales Empathic Concern (EC) and Fantasy Seeking (FS) are theorised to prefer sad music (characterised by negative valence and low arousal) as it elicits feelings of sympathy and increases listener's absorption, respectively (Huron & Vuoskoski, 2020). We examine these hypotheses in a naturalistic setting (music streaming platforms). Also, we explore dynamic music listening patterns by examining song sequencing.
Methods
Data from 290 Indian university students (age M = 20.35, SD = 2.08; 211 males) included IRI scores and one year of Spotify history. Tracks played for <15 seconds were removed, and users with <500 listening events were excluded. Each event logged track name, artist, duration, and timestamp. The final dataset had 1.7M events, 136K tracks, and 86K hours. Valence and arousal (via Spotify's energy feature) were extracted and correlated with EC and FS scores.
Participants were divided into groups based on median splits of EC and FS scores.
We performed quadrant-based analysis by dividing music into four quadrants: positive valence-high arousal (Q1), negative valence-high arousal (Q2), negative valence-low arousal (Q3), and positive valence-low arousal (Q4). Quadrant Prevalence Scores (QPS), the percentage of songs from that quadrant in the listening history, were calculated per participant. Finally, transition probabilities (TP) between quadrants were calculated based on sequential track plays. Permutation tests assessed group differences in QPS and TP.
Results
Significant negative correlations were observed between EC and the average valence (r = -0.15, p = 0.011) and arousal values (r = -0.17, p = 0.003) of the tracks played by the user. Overall, each group had the highest QPS in Q1 followed by Q2. Significant between-group differences were observed only in Q3. High EC (p = 0.001), high FS (p = 0.024), and high PT (p = 0.001) were associated with high QPS in Q3. High EC individuals showed higher transition probabilities from Q1 to Q3 and within Q3 (p < .05), while low EC individuals more often transitioned into Q1 from Q2, Q3, and Q4. No significant transition differences were observed for FS. PT differences emerged in transitions into Q3 from Q1, Q2, and Q4 (p < .05).
Discussion
This study extends music and empathy research to a naturalistic setting. The findings support our hypotheses: Individuals with greater empathic concern and fantasy seem to listen to sad music more than their low empathy counterparts. While music from Q1 is most frequently consumed, the increased chances that highly empathic individuals transition to Q3 might reflect their need for a deeper emotional engagement through music. Despite the ecological validity of using real-world Spotify data, limitations include Spotify's coarse emotion metrics and lack of control over contextual factors like time of day or social setting. However, long-term trends may help mitigate transient effects and capture stable preferences. Future research could explore these factors using experience sampling (Larson & Csikszentmihalyi, 2014) and wearable sensors to capture real-time emotional responses to music.

Abstract

Empathy has been linked to music preferences in controlled laboratory settings, but naturalistic settings like music streaming platforms remain unexplored. This study investigates how trait empathy influences music preferences and sequential listening behaviors in real-world settings. To this end, we collected and analyzed one-year Spotify listening histories of 290 Indian university students alongside their trait empathy scores, measured using the IRI scale. Our results reveal that individuals who score high on the IRI subscales of Empathic Concern, Fantasy, and Perspective Taking prefer sad music. Moreover, those scoring high on Empathic Concern or Perspective Taking were found to be more likely to transition from happy to sad music. These findings partially align with previous lab-based research, specifically for the subscales of Empathic Concern and Fantasy, while providing novel insights into the relations between the Perspective Taking subscale and music consumption. The study also provides novel insights into sequential listening behaviours, thus, strengthening the evidence that empathy shapes musical preferences and listening behaviors across diverse contexts.

Abstract

Background: Sensory Processing Sensitivity (SPS), a temperament trait reflecting deeper cognitive processing and heightened emotional/physiological reactivity, is often linked to emotional dysregulation and vulnerability to stress-related problems. SPS includes three subscales: Ease of Excitation (EOE) reflects emotional sensitivity to internal (e.g., thoughts, bodily states) and external (e.g., social demands) stimuli; Low Sensory Threshold (LST) reflects sensitivity to intense sensory input; and Aesthetic Sensitivity (AES) denotes appreciation for subtleties (Smolewska et al., 2006). Music can sometimes contribute to maladaptive outcomes like rumination, avoidance coping, and amplification of negative affect (Alluri et al., 2022). Highly sensitive persons, especially with elevated EOE, may be more prone to such patterns due to their emotional reactivity. However, SPS's interactive role in maladaptive music use remains underexplored.
Aim: This study investigates the relationship between SPS and maladaptive music use, hypothesizing that individuals with high EOE, particularly those sensitive to external demands (External EOE), are more prone to unhealthy music engagement. Network analysis was employed to reveal how sensitivity traits, distress, and empathy interact in shaping music use patterns.
Methods: A total of 680 Indian participants(mean age=20.69, s.d.=1.9; 180 females) completed a battery of self-report measures including Highly Sensitive Persons Scale (HSPS) for SPS traits, the Kessler Psychological Distress Scale (K10) (Andrews & Slade, 2001), Interpersonal Reactivity Index (IRI) for empathy traits (Keaton, 2017), and Healthy-Unhealthy Music Scale (Saarikallio et al., 2015) assessing adaptive and maladaptive music use. The IRI included four subscales: Empathic Concern (EC), Perspective Taking (PT), Personal Distress (PD), and Fantasy Scale (FS), capturing both cognitive and affective empathy. Using exploratory factor analysis we split EOE into Internal and External factors. Regularized Partial Correlation Networks (RPCNs) and Bayesian Networks were constructed to model interactions and infer relationships among variables.
Results: Unhealthy music use correlated with External EOE(r=0.25) while Healthy music use correlated with AES(r=0.32) among SPS traits. RPCNs identified External EOE as a central node in the network(strength=0.69; betweenness=116), bridging SPS traits with distress-related personal feelings(PD from IRI), psychological symptoms (K10), and Unhealthy music use. Bayesian Networks confirmed K10 as a direct predictor of Unhealthy use(B=0.50), with PD predicting K10(B=0.31) and External EOE predicting PD(B=0.50). Mediation analysis revealed that K10 significantly mediated the pathway from External EOE to Unhealthy music use, accounting for 55% of the total effect (all p<0.01), suggesting distress amplifies the impact of emotional sensitivity on maladaptive behaviors.
Discussion: While music can support emotion regulation, our results suggest that highly sensitive persons experiencing distress may turn to music as a maladaptive coping strategy to escape overwhelming external demands. In India, where population density and sensory overstimulation are high, such patterns may be intensified. These findings add to the emerging literature on trait-environment interactions in music behavior and suggest that understanding EOE may inform future interventions or wellness strategies tailored to individual sensitivity profiles.

Abstract

Sensory processing sensitivity (SPS), a trait marked by
heightened reactivity to stimuli, is linked to emotional
dysregulation and stress-related problems. Its subscales include
Ease of Excitation that reflects emotional sensitivity to internal
and external demands, Low Sensory Threshold reflects sensory
overload susceptibility, and Aesthetic Sensitivity denotes
appreciation for subtleties. While music can sometimes
amplify maladaptive outcomes (rumination, avoidance), SPS's
role in such behaviours, especially in non-Western contexts,
remains underexplored. This study examines how SPS drives
maladaptive music listening in 673 Indian adults. Network
analysis and structural equation modelling revealed sensitivity
to external demands elevated psychological distress which in
turn predicted maladaptive music use. Reactivity to internal
demands may reduce maladaptive music use but exacerbated it
when mediated by external demands, reflecting preference for
avoidant coping in Indian context. Findings emphasize SPS's
role in maladaptive behaviours, demonstrating how sensory
reactivity interacts with traits to shape distress-driven music
use as an emotional regulation mechanism.

Abstract

Personalised Interactive Music Systems (PIMS) are emerging as promising devices for enhancing physical activity and exercise outcomes. By leveraging real-time data and adaptive technologies, PIMS align musical features, such as tempo and genre with users' physical activity patterns, including frequency and intensity, enhancing their overall experience. This systematic review and exploratory meta-analysis evaluates the effectiveness of PIMS across physical, psychophysical, and affective domains. Searches across nine databases identified 18 eligible studies, of which six (comprising 17 intervention arms) contained sufficient data for meta-analysis. Random-effects meta-analyses and meta-regression were performed to assess outcomes for physical activity levels, physical exertion, ratings of perceived exertion (RPE), and affective valence. Results showed significant improvements in physical activity levels (g = 0.49, CI [0.07, 0.91], p = .02, k = 4) and affective valence (g = 1.68, CI [0.15, 3.20], p = .03, k = 4), with faster music tempo identified as a significant moderator (p = 0.04). No significant effects were observed for RPE (g = 0.72, CI [-0.14, 1.59], p = .10, k = 3) or physical exertion (g = 0.79, CI [-0.64, 2.10], p = .28, k = 5). Substantial heterogeneity and limited study quality indicate the need for more robust, randomised controlled trials to establish the efficacy of PIMS in diverse populations.