A topic that has been studied in my lab is extracting meaningful representations from unlabeled, paired, multimodal data. These datasets are collected from various measurement devices, each capturing a different aspect of the same underlying phenomenon, such as images paired with text, clinical data combined with genomic information, or various sensor streams. Learning from this unlabeled data presents challenges but is valuable for many applications, including clustering and anomaly detection. 🧠 We recently published two papers on this topic (at ICLR and TMLR): 🔹 COPER: Correlation-Based Permutations for Multi-View Clustering We present a self-supervised method that learns LDA-like projections from unlabeled multi-view data by permuting samples within predicted clusters. This approach results in more discriminative and cluster-friendly representations, all without any supervision. COPER achieves state-of-the-art clustering performance across ten benchmarks. 🔹 SpecRaGE: Robust and Generalizable Multi-View Spectral Representation Learning In this study, we developed a scalable deep framework that simultaneously diagonalizes graph Laplacians from different views in order to create unified representations. Our approach, SpecRaGE, is resilient to noise and outliers due to a meta-learned fusion module, and it generalizes effectively to unseen data. These works enhance our understanding of learning structured, robust, and generalizable representations from paired data, without depending on labels or ideal conditions. Please check the papers for more details or reach out if you have questions. Kudos to all collaborators for these great works. #ICLR2025 #RepresentationLearning #MultiModal #UnsupervisedLearning #MultiViewLearning #DeepLearning

Multi-view biomedical foundation models for molecule-target and property prediction @IBMResearch • The paper introduces MMELON, a multi-view molecular foundation model combining graph, image, and text views to enhance prediction of molecular properties. Unlike single-view models, MMELON leverages multiple representations for a richer, more versatile molecular embedding. • The model performs exceptionally well on 18 diverse tasks, including ligand-protein binding, molecular solubility, metabolism, and toxicity, balancing the strengths of each modality. This versatility is critical in drug discovery and computational chemistry. • MMELON integrates three views—graph, image, and text—to learn comprehensive molecular representations. The image view uses ImageMol (pre-trained on 10 million molecules), while the graph and text views are based on advanced transformer architectures, pre-trained on datasets of 200 million molecules. • A novel aspect is the “late fusion” of these different modalities, ensuring each modality contributes optimally depending on the downstream task. This approach yields interpretable results and allows for an analysis of how each view supports different predictions. • For validation, MMELON was applied to screen compounds against a large set of G Protein-Coupled Receptors (GPCRs). Of these, 33 GPCRs related to Alzheimer’s disease were identified, and strong binders were predicted, validated through in silico structure modeling. • The multi-view model shows strong correlations between predicted and experimental affinities, achieving a Pearson correlation of 0.78 for GPCR binding. This suggests the model’s robust application for identifying new therapeutics. • Compared to single-view models, MMELON delivers superior performance across classification and regression tasks, making it an essential tool for complex molecular property predictions in drug discovery. @jamorrone3 @jianying_hu @FeixiongCheng @jeriscience @BCKwon @timrumbell @dplatt_maths @YunguangQiu @diwakarmahajan 💻Code: github.com/BiomedSciAI/biome… 📜Paper: arxiv.org/abs/2410.19704 #biomedicalAI #drugdiscovery #foundationmodel #multiviewlearning #GPCR #Alzheimers #machinelearning #bioinformatics

Join us tomorrow for this week's robot learning seminar, in which @imankitgoyal from @nvidia will present his talk titled "View Transformers for 3D Manipulation in Robotics". See you there at 11:30AM ET! YouTube.com/@MontrealRobotic… #Manipulation #multiViewLearning #transformers

🔔 New article: Ye et al. propose #VCANet (Multi-View Contextual Adaptation Network) for improved #ObjectDetection in remote sensing imagery. 🔗 doi.org/10.1080/01431161.202… #IJRS #RemoteSensing #MultiViewLearning

Robust Visual Ship Tracking with an Ensemble Framework via Multi-View Learning and Wavelet Filter mdpi.com/1424-8220/20/3/932 #VisualShipTracking #WaveletFilter #MultiViewLearning

At another round of our #JournalClub, we discussed #MultiViewLearning framework for #multiomics data. There are methods in our lab such as #GGLasso that perfectly fits this framework described scrupulously by Nam D. Nguyen @namtrk, D. Wang @daifengwang in journals.plos.org/ploscompbi…

A #multiviewlearning method for #malware #threathunting: #windows, #IoT and #android as case studies, #WorldWideWeb Journal link.springer.com/article/10… #UTSAResearch #CyberSecurity #cybercrime #AI #MachineLearning #api #opcodes #iotmalware #cyberthreats #CTI

@EmilieMorvant @ECMLPKDD #ECML 2017 : talk on #machineLearning #PACBayesian Theory for #MultiviewLearning hal.archives-ouvertes.fr/hal…

And now A. Goyal are talking about #multiviewLearning and #PACBayes theory #MachineLearning #Cap16 #LabCurien

It's time for #PhDSeminar #LabCurien! I. Redko talk about #optimalTransport & #multiviewLearning #MachineLearning