Projects | Yik Yu Ng

Privacy-Preserving Multimodal AI for MS Prognosis

Sun, 31 May 2026 00:00:00 +0000

Overview

This research develops a privacy-preserving multimodal large language model system for personalized multiple-sclerosis lesion-activity prognosis. The system combines clinical narratives and MRI while keeping model inference local.

Research direction

A retrieval-synthesis-critique workflow assembles grounded evidence for the final assessment.
The design emphasizes privacy, traceability, and cautious use in a clinical decision-support setting.
Current work studies systematic failure subgroups through multimodal decision embeddings.

This project is conducted at Mila - Quebec AI Institute under the supervision of Professor Tal Arbel.

Honour Research Project: Bias Detection through Unsupervised Learning

Thu, 01 May 2025 00:00:00 +0000

Bias Detection through Unsupervised Learning

30 Nov 2024 – Current

Executive Summary

This innovative research project addresses a critical challenge in today’s digital information ecosystem: detecting and quantifying political bias in news articles and social media content without requiring labeled training data. The framework introduces a novel fully unsupervised pipeline that combines geometric embedding-based techniques with a multi-agent evaluation system to provide a comprehensive, nuanced assessment of media bias across two dimensions - political leaning and rhetorical bias intensity.

Note: This work is still in progress, we hope to push it to a formal conference paper. Proposal can be found here

Problem Statement

In today’s digital landscape, social media platforms have become the primary source of news for billions worldwide, fundamentally altering information flow and public discourse. This shift presents unprecedented challenges as algorithms selectively amplify content, often prioritizing engagement over accuracy and potentially exposing users to increasingly polarized and biased information. Such bias can profoundly influence public opinion formation, policy preferences, and voting behavior in democratic societies.

While traditional approaches to media bias detection typically rely on supervised learning with large labeled datasets, our research introduces a framework that eliminates this dependency, making bias detection more scalable, adaptable, and transparent.

Methodology: A Two-Component Approach

Our framework consists of two complementary components that work together to provide a comprehensive bias assessment:

1. Embedding-Based Political Leaning Estimation

This component quantifies the political orientation of articles on a continuous left-right spectrum through several innovative techniques:

Article Embedding: Content is transformed into dense vector representations using INSTRUCTOR, an instruction-finetuned text embedding model that captures semantic information relevant to political analysis.
Pseudo-Topic Clustering: Articles are grouped into clusters based on topical similarity to normalize for content variation, ensuring political leaning is assessed within comparable contexts rather than across disparate subjects.
Contrastive Adversarial Fine-Tuning: Embeddings are enhanced through triplet learning, where semantically aligned articles are pulled closer together while divergent articles are pushed further apart, amplifying fine-grained ideological distinctions within topical groups.
Directional Vector-Based Scoring: A political spectrum axis is defined within the embedding space using prototypical left-leaning and right-leaning seed articles. Each article’s political leaning score is computed as the normalized projection onto this axis, ranging from -1 (strongly left-leaning) to +1 (strongly right-leaning).

2. Multi-Agent LLM-Based Rhetorical Bias Analysis

This component evaluates the presence of biased rhetorical techniques regardless of political orientation:

Agent Configuration: Rather than simulating political diversity, three specialized agents examine different aspects of bias:
- Structural Analysis Agent: Examines article organization, framing, headline analysis, and source selection
- Content Analysis Agent: Evaluates language choice, fact-opinion separation, and evidence assessment
- Contextual Analysis Agent: Focuses on omissions, historical patterns, and comparison to mainstream reporting
Evaluation Protocol: The process occurs in three sequential phases:
1. Independent article analysis by each specialized agent
2. Structured deliberation facilitated by a moderator agent
3. Consensus formation, producing a five-point rhetorical bias rating (1=minimal bias, 5=extreme bias) with supporting evidence

Integrated Bias Assessment

The outputs from both components are combined into a two-dimensional bias representation:

X-axis: Political leaning from -1 (strongly left) to +1 (strongly right)
Y-axis: Rhetorical bias intensity from 0 (minimal bias) to 1 (extreme bias)

This representation allows for nuanced distinctions between different bias profiles. For example, an article might be identified as strongly right-leaning but employing relatively few biased rhetorical techniques, while another might be moderately left-leaning but using highly biased rhetorical devices.

Visualization

Our framework generates several types of visualizations to help users understand and interpret bias profiles:

2D Bias Maps: Plot media outlets in a two-dimensional space showing both political leaning and rhetorical bias intensity
Score Distributions: Histograms showing the distribution of political leaning scores across all articles
Source Comparisons: Bar charts comparing mean political leaning scores across different media outlets
Beeswarm Plots: SHAP-like visualizations showing individual article distributions by source
Violin Plots: Probability density visualizations showing the distribution shape for each source

Key Contributions

Our framework offers several significant advantages over existing approaches:

No Labeled Data Requirement: Eliminates the need for expensive human-annotated datasets, enabling analysis at scale
Domain Generalizability: Can be applied to any domain with opposing viewpoints, not limited to political content
Transparent Assessment: Provides explicit reasoning and evidence for bias evaluations, fostering trust
Bias Mitigation: Combines geometric methods with deliberative LLM analysis to reduce potential biases inherent in either approach alone

Applications and Impact

This research contributes to several important objectives:

Providing tools for media literacy and critical information consumption
Enabling more balanced content recommendation systems
Supporting researchers and policymakers with insights into information flow
Strengthening democratic discourse by helping citizens identify bias in an increasingly complex digital information ecosystem

Future Directions

Future work will address several promising avenues:

Expanding the framework to incorporate multimodal analysis of visual and textual elements
Developing methods for continuous learning and adaptation to evolving political discourse
Exploring cross-cultural adaptations for international media analysis
Further optimizing the multi-agent protocol to reduce computational requirements

By providing accessible tools for bias detection without relying on costly labeled data or domain-specific customization, this project aims to contribute to a more informed and discerning public discourse in the digital age.

Parameter-Efficient Reinforcement Learning via Dynamic Sparse Training

Sun, 08 Dec 2024 00:00:00 +0000

Enhancing A2C with DAPD across Classic Control and Atari Environments

1 Oct 2024 – 10 Dec 2024

Project Overview

This project investigates parameter-efficient reinforcement learning through the integration of Dynamic Sparse Training with Pathways (DAPD) into the Advantage Actor-Critic (A2C) algorithm. We compare A2C+DAPD against standard dense A2C and Proximal Policy Optimization (PPO) across three benchmark environments—LunarLander-v2, Breakout, and Pong—to evaluate trade-offs between computational efficiency and learning reliability.

Full Report

Access the full details about the project and result on the .

Presentation

Objectives

Assess the impact of DAPD on A2C’s training speed and parameter footprint.
Compare performance and stability of sparse and dense agents across classic and visual control tasks.
Derive empirical insights into the trade-offs inherent in dynamic sparse training within RL.

Technical Approach

A2C: Advantage Actor-Critic with actor and critic networks sharing a two-layer feed-forward architecture (64 ReLU units), optimized via Adam.
PPO: Dense baseline using a clipped objective for enhanced stability and sample efficiency.
A2C+DAPD: Incorporates a binary mask to maintain 90% sparsity, updating prune–regrow operations periodically (every 1,000 steps for LunarLander; 100,000 for Atari).

Agents are trained for 10,000 episodes in each environment, recording episodic rewards, average performance over the last 100 episodes, and wall-clock training time. Reward curves are smoothed using a 100-episode moving average.

Experiments

We evaluate:

LunarLander-v2: Classic control task, low-dimensional state space.
Breakout & Pong: Visual environments from the Atari suite.

Key findings:

DAPD reduces wall-clock training time by approximately 30–40%, converging in ~8,000–14,000 seconds versus ~14,000–25,000 seconds for dense A2C.
Sparse training introduces significant variance and instability, particularly in control tasks.
PPO consistently outperforms both A2C variants in final reward and stability across all environments.

Outcomes and Future Work

Our results highlight the computational advantages of dynamic sparse training at the expense of convergence reliability. While DAPD accelerates early learning, its instability underscores the need for sparsity-aware optimization strategies. Future directions include:

Thorough evaluation of the Policy Pruning and Shrinking (PoPS) method.
Exploration of structured sparsity, adaptive pruning schedules, and hybrid dense–sparse models.
Deployment studies on resource-constrained hardware for real-time applications.

Medial Axis Transform: Geometric Cues in Scene Categorization

Fri, 01 Nov 2024 00:00:00 +0000

Medial Axis Transform

1 Nov 2024 – 15 Dec 2024

Project Overview

The Medial Axis Transform project is a final project for COMP 558 (Computer Vision) at McGill University. The main goal is to replicate and build upon the findings of the paper “Shape-Based Measures Improve Scene Categorization” by exploring additional geometric properties in images.

Full Report and source code

For a comprehensive project overview, please . Source code would be available at .

Project Presentation

Objectives

Replication: Recreate the experiments from the referenced paper to establish a baseline.
Innovation: Propose a new geometric property, Convexity, to complement traditional Gestalt cues.
Evaluation: Investigate the impact of these geometry cues on enhancing CNN model performance through rigorous experiments.

Technical Approach

Data Analysis: Employ image processing techniques to extract and analyze relevant geometric features.
Model Training: Integrate the new geometric property into CNN models and evaluate performance gains.
Comparative Study: Compare results with baseline approaches to ascertain the effectiveness of the proposed cue.

Outcomes

This project aims to bridge theoretical insights and practical implementation, ultimately contributing to the field of computer vision by offering improved methods for scene categorization.

Quick Draw: Interactive Animal Recognition Web App

Sun, 01 Oct 2023 00:00:00 +0000

Quick Draw

1 Oct 2023 – 25 Nov 2023

Project Overview

The Quick Draw project is a collaborative initiative developed during the MAIS202 Bootcamp. The project focuses on implementing a machine learning model capable of predicting animal labels based on user-uploaded images. Leveraging the Google Quick Draw dataset, the application accurately classifies images into 10 distinct animal categories.

Demo

Key Features

Real-Time Prediction: Upload an image and receive immediate, accurate predictions of the animal depicted.
High Accuracy: Achieves approximately 83% accuracy across 10 animal classes.
Collaborative Development: Developed by a two-member team, integrating strong model development with full-stack web application expertise.

Technical Approach

Model Development: Utilized CNN architecture, training on the Quick Draw dataset to optimize performance.
Web Integration: Engineered a responsive and user-friendly web interface that seamlessly integrates with the machine learning model for real-time predictions.
Optimization: Fine-tuned both the model and web application to ensure efficient and smooth operation under live conditions.

Contributions

Machine Learning: Contributed 50% towards model development, ensuring robust performance and accuracy.
Web Application: Took full responsibility for designing and implementing the web app, from UI/UX design to backend integration.