Poster Session

All sections at a glance

Life Sciences

Physical Sciences


Humanities and Social Sciences

Life Sciences

Deep Learning of Mass Spectrometry Imaging
 Chau Tran, Ph.D.
New York University


Mass spectrometry imaging (MSI) is a rapidly advancing technique which can provide spatial resolution and quantification of metabolites, lipids, peptides, and glycans in snap frozen tissue cryo-sections. In contrast, traditional pathology techniques such as immunohistochemistry (IHC) target specific analytes by utilizing the exquisite specificity of antibodies to detect a single marker protein. The format of sample preparation and data acquisition for MSI mirrors traditional histopathology, but detection is achieved by high-resolution mass spectrometry which can simultaneously detect a wide spectrum of hundreds or thousands of individual molecules and provide relative quantification. Therefore, the overarching hypothesis of our research is that metabolites and lipids provide more information about cell and molecular composition of cancer tissues than traditional histopathology staining approaches. We aim to couple these high-dimensional data to machine learning (ML) to identify discriminating features. For example, ML has been applied to traditional pathology data (optical H&E images) by extracting the latent information present in the stained tissue. Past research has successfully developed and published such a Convolutional Neural Network (CNN) which predicts the histological subtype and even molecular subtype of endometrial cancer tissue sections using H&E images. Named Panoptes, this CNN architecture is sensitive to subtle patterns in image data, which is the basis for its ability to unbiasedly classify image tiles. We therefore expect that MSI data, which contains thousands of individual markers, can be exploited to outperform existing CNN models with respect to tissue discrimination and molecular annotation.

Gain Modulation and Plasticity Lead to Perceptual Learning: A Neural Network Model of the Prolonged Effects of Visual Attention

Thomas Maher
New York University

Perceptual learning, characterized by consistent long-term changes in perception resulting from prolonged practice on perceptual tasks such as orientation discrimination, has garnered significant interest due to its implications for understanding the mechanisms of attention and neural plasticity. The feature similarity gain model (FSGM) of attention proposes that neuronal activity is scaled (likely by signals from the prefrontal cortex) based on preference for features in an attended stimulus, leading to enhanced sensitivity to those features. The increased firing of upscaled neurons over a period of time may lead to the effect of perceptual learning due to the synaptic plasticity of the neurons in the visual cortex. Building on FSGM, we present a computational study using a convolutional neural network (CNN) model of the visual cortex capable of expressing perceptual learning effects. We incorporate multiplicative modulations within the network to simulate attentional effects and implement a Hebbian update rule to capture the changes in neuronal activity induced by attention. Our results demonstrate that Hebbian updating during attentional modulation improves the model's performance on orientation discrimination tasks compared to baseline conditions. This study contributes to our understanding of how attentional mechanisms and Hebbian plasticity interact to facilitate perceptual learning in the visual system.

How Math and AI are Revolutionizing Biosciences
Rui Wang, Ph.D.
New York University


The ongoing COVID-19 pandemic has underscored the critical need for innovative analytical techniques to interpret complex biological data and enhance our understanding of viral spread and treatment efficacy. This study presents a pioneering integration of computational topology and artificial intelligence (AI), showcasing a novel approach to epidemiological research.


Specifically, we delve into the capacities of persistent spectral graphs (PSGs) for analyzing the intricate topological and geometric properties of high-dimensional biological data. Our approach begins with the formulation of persistent Laplacian matrices (PLMs), constructed through the application of a dynamic filtration parameter. The harmonic spectra derived from the null spaces of these PLMs encapsulate the underlying topological features, while the non-harmonic spectra elucidate the geometry of high-dimensional datasets. Through our open-source software, HERMES, we streamline the computation of these spectra, catalyzing progress across biology, medicine, and engineering fields.


During the COVID-19 pandemic, we integrated PSG, genomics, and deep learning into a Math-AI model (TopNetmAb) to predict the binding free energy (BFE) changes caused by mutations in the interaction between the virus's Spike protein and the human ACE2 receptor or antibodies. Such a Math-AI model has successfully forecasted the predominance of Omicron variants BA.2, and BA.4/BA.5 one or two months ahead of their presence, offering a glimpse into a future where mathematical techniques and biology converge to combat viral threats more effectively.

System and Method for Automating Dental Implant Fixtures (ADIF) using Artificial Intelligence and Machine Learning for Computer-assisted Implant Surgery

Young K Kim, DMD, DMSc
New York University


The prevalence of edentulism and the burgeoning dental implant market underscore the critical need for innovative solutions in implantology. Traditional computer-assisted implant placement workflows face challenges of preparatory fatigue and cognitive burden, necessitating interventions to streamline planning processes. Leveraging recent advancements in AI and machine learning, we propose an Auto-Distribution of Implant Fixtures (ADIF) system to automate implant planning procedures. This system integrates anatomical and biomechanical guidelines with machine learning algorithms, enhancing efficiency and precision. ADIF's seamless integration with existing software ensures minimal disruption to established workflows. Our approach progresses from deterministic algorithms to machine learning-based solutions, promising transformative enhancements in dental implant planning. Future developments include cloud-based learning systems and sequential sophistications to meet evolving clinical demands. ADIF holds significant potential to revolutionize implant surgery practices, improving patient outcomes and clinician efficiency.

A knowledge-graph-based pharmaceutical engineering chatbot for drug discovery
Naz Pinar Taskiran
Columbia University

Despite their great success in day-to-day applications, ChatGPT and other large language models (LLMs) have not covered as much ground in scientific and engineering domains. One key challenge is the abundance of domain-specific terminology, which an LLM is not trained to extract in accordance with the underlying physical laws. This can lead to unreliable results or hallucinations. To address these challenges, we have developed SUSIE, an ontology-based pharmaceutical information extraction tool that is built to extract semantic triples and present them to the user as knowledge graphs (KGs). While KGs help visualize the relationships between different entities, they are not easily accessible for user questions, yet serve as structured inputs for LLMs. Thus, KGs can be used to efficiently query a corpus of pharmaceutical documents, streamlining drug discovery and manufacturing processes. In this work, we present a customized question-and-answering module that enables the user to query from generated KGs, and get an answer in natural language. We show that with the integration of prompt engineering into a model based on the set of rules defined by the ontology, the output is sensible and explainable.


Deep speech-to-text models capture the neural basis of spontaneous speech in everyday conversations
Haocheng Wang
Princeton University


Humans effortlessly use the continuous acoustics of speech to communicate rich linguistic meaning during everyday conversations. In this study, we leverage 100 hours (half a million words) of spontaneous open-ended conversations and concurrent high-quality neural activity recorded using electrocorticography (ECoG) to decipher the neural basis of real-world speech production and comprehension. Employing a deep multimodal speech-to-text model named Whisper, we develop encoding models capable of accurately predicting neural responses to both acoustic and semantic aspects of speech. Our encoding models achieved high accuracy in predicting neural responses in hundreds of thousands of words across many hours of left-out recordings. We uncover a distributed cortical hierarchy for speech and language processing, with sensory and motor regions encoding acoustic features of speech and higher-level language areas encoding syntactic and semantic information. Many electrodes—including those in both perceptual and motor areas—display mixed selectivity for both speech and linguistic features. Notably, our encoding model reveals a temporal progression from language-to-speech encoding before word onset during speech production and from speech-to-language encoding following word articulation during speech comprehension. This study offers a comprehensive account of the unfolding neural responses during fully natural, unbounded daily conversations. By leveraging a multimodal deep speech recognition model, we highlight the power of deep learning for unraveling the neural mechanisms of language processing in real-world contexts.

Cluster-aware machine learning of multiomics and neuroimaging for precision neuroscience and psychiatry

Amanda Buch, Ph.D.
Cornell University


Explainable machine learning of complex multimodal data is transforming the landscape of neuroscience research. In many cases, data heterogeneity across samples due to biological clusters is an important component of variation, and revealing these clusters and the biological factors that explain them is an important research approach. For example, in medical diagnoses, interpretable clustering of patients into distinct subtypes improves personalization of treatment. However, a combination of the well-known “curse of dimensionality” and the clustered structure of biomedical data together present a unique challenge in the high dimensional and limited observation regime common in datasets used in neuroscience. Embedding followed by clustering is popular, but this two-stage process often results in both suboptimal embeddings and degraded cluster separation, motivating a need for joint clustering and embedding approaches that are explainable, robust to technical variability, and generalizable. To overcome both challenges simultaneously we propose a simple and scalable approach to joint clustering and embedding that combines standard embedding methods with a convex clustering penalty in a modular way. Through both numerical experiments and real-world examples, we show that our approach outperforms traditional and contemporary clustering methods on highly underdetermined problems (e.g., with just tens of observations) as well as on large sample datasets. Thus our approach improves significantly on existing methods for identifying patient subgroups in multiomics and neuroimaging data.



Deep learning Assisted Biomechanics Analysis based on Optical Coherence Tomography in the Mouse Oviduct In Vivo
Tianqi Fang
Stevens Institute of Technology

The mammalian oviduct (or fallopian tube) is a tubular organ connecting the ovary and the uterus. It is essential for natural pregnancy, carrying out the transport of preimplantation embryo toward the uterus for implantation. This transport process has traditionally been perceived as a simple, gradual movement of embryos primarily driven by the beat of motile cilia lining the epithelium of the oviduct lumen. However, our recent work on the first high-resolution in vivo 3D time-lapse imaging in the mouse model showed complex dynamics of oocytes/embryos, suggesting multiple roles of the oviduct muscle contraction in the transport process. Understanding the mechanism of embryo transport and the oviduct-embryo interactions during this process is critical to elucidate the important biomechanical factors underlying a successful natural pregnancy and is also vital to uncover functional causes of associated female reproductive disorders, such as tubal ectopic pregnancy. In this work, we applied the in vivo imaging approach and developed deep learning assisted image processing pipeline to facilitate the biomechanic analysis of oviduct structures from a 4-D OCT dataset. Multiple image volumes at different time points were selected as key volumes and their oviduct structures including muscle wall and lumen, were manually labeled and then used to train a U-Net for oviduct structure segmentation. A semi-automatic post-processing method was applied for assessing the oviduct strain from the 4D data and validated through a computational phantom. This work sets the stage for a quantitative understanding of the oviduct dynamics in vivo, and the proposed deep-learning assisted method fills in a gap in OCT-based biomechanics analysis. 


AI-Driven Quantification of Dendritic Spine Dynamics: Expanding Insights into Synaptic Plasticity
 Esra Sefik, Ph.D., Callan O’Shea
Princeton University

Dendritic spines play a pivotal role in synaptic transmission and plasticity, serving as key sites for the formation, strengthening, and elimination of synapses, which enable the communication and integration of information within neural circuits. These micron-scale protrusions undergo dynamic structural changes that reach their peak during early postnatal development, gradually diminishing with age, under the influence of both intrinsic genetic programs and synaptic activity or environmental cues. Systematic mapping of changes in dendritic spine structure can shed light on the mechanisms underlying learning, memory, and cognitive flexibility, and offer insights into neurological and psychiatric disorders and potential therapeutic interventions. For dendritic spine imaging, we employ cutting-edge tools for live longitudinal multi-photon microscopy of cortical neurons in awake mice over a period of hours up to weeks, alongside confocal microscopy of ex vivo mouse brain slices encompassing cortical and subcortical regions. Despite the sophistication of these optical methods, the manual nature of spine identification and categorization remains prohibitively slow and variable when performed by humans, consuming significant time and labor and imposing limitations on sample size, statistical power, and reproducibility, hence, restricting the insights we can derive. To address this challenge, we present both longitudinal and cross-sectional examples of utilizing the state-of-the-art, deep learning-based framework, DeepD3, to robustly quantify dendritic spine appearance, disappearance, and morphology in our microscopy data in a fully automated manner. With the aid of AI, we are currently testing the hypothesis that changes in spine density contribute to the enhancement of cognitive flexibility in evidence-accumulation based task acquisition triggered by psilocybin. In another project, we are employing this AI-based approach to test the hypothesis that cerebellar damage in juvenile mice induces developmental diaschisis, leading to long-distance changes in spine turnover within neocortex on the other side of the brain from the cerebellum. As showcased across these applications, automated dendritic spine quantification empowered by AI promises to accelerate scientific discovery in neuroscience, amplifying the scale of data analysis, expediting processing speed, and mitigating inter- and intra-rater subjective biases, thus bolstering the reliability and reproducibility of research findings into fundamental structural processes underlying neural plasticity.


Wavelet Transform Embedding and Masked Autoencoder Transformer for signal classification
Haozhe Pang
Stevens Institute of Technology

Recent advancements in deep learning have highlighted the efficiency of the Transformer model in complex data analysis. Despite its success, applying Transformers to signal processing remains challenging due to long data sequences and sparse features. In this work, we introduce the Wavelet Transform-based Masked Autoencoder Transformer (WT-MAE), which utilizes wavelet transform for embedding within the whole model. Wavelet transform is learning-free and decomposes signal data into sub-series by frequency, effectively reducing sparsity. The complexities of long data sequences are simplified by generating a limited number of much shorter sub-series as input. Additionally, to tackle information redundancy in the series, we develop a Masked Autoencoder structure adopting Transformer modules. The ablation experiments with our wavelet-based approach in gesture recognition tasks show that the embedding brings over 10 percent performance improvements.

Physical Sciences

Discovering design rules for solvent selection in ternary organic blends

Hao Liu, Ph.D.
Fordham Univesity


During solvent-based fabrication of organic blends, the blend composition changes significantly as solvent is removed from the blend triggering phase transformations and structure formation. The choice of the solvent in conjunction with the processing pathways (i.e. quench profile) affects this process. Using a combination of solvents as well as solvent additives has produced favorable structures for organic electronic applications. For example, in the context of organic solar cells, changing solvents, utilizing solvent blends, including solvent additives, and adding thermal annealing as a post-processing step, have all produced significant jumps in photovoltaic efficiency. With the number of possible solvents potentially in the hundreds, choosing solvent blends, solvent additives and similar strategies represent a combinatorically large set of possibilities. Most processing variants reported have invariably been chosen through a trial-and-error approach, suggesting that only a small subset of the selection set has been explored. The selection of solvents has a rich history of empirical and theoretical design rules. The solubility sphere (HSP sphere) technique is an often used approach that allows the down selection of solvents when dealing with blends of materials. However, this becomes impractical with an increasing number of components and has no straightforward way to consider quench depths, crystalline phases or processing pathways as no additional thermodynamics (let alone kinetics) is considered. Consequently, it has remained challenging to establish reliable design rules that can be used for solvent blend selection, as well as selecting (and expanding) the processing space. In this work, we consider using the full phase diagram as a signature for solvent selection. The phase diagram is a comprehensive representation of the thermodynamic characteristics of the multi-component material system. Using high throughput exploration and clustering method, we learn simple and interpretable rules of phase diagram type and its sensitivity to the changes in the interaction parameter. This work has implications for the solvent selection but also solvent replacement for the efficient devices.


Multi-objective Bayesian optimization for design of Pareto-optimal current drive profiles in STEP
Alexander Terenin, Ph.D.
Cornell University


The safety factor profile is a key property in determining the stability of tokamak plasmas. To improve the safety factor profile in the United Kingdom's proposed Spherical Tokamak for Energy Production (STEP), we apply multi-objective Bayesian optimisation - a machine learning technique in which an uncertainty-aware predictive model guides the optimisation process based on the observed data - to design electron-cyclotron heating profiles. The resulting procedure generates sets of solutions that represent optimal tradeoffs between objectives, enabling decision makers to understand the compromises made during the design process. The solutions from our method score as well as those generated in previous work by a genetic algorithm; however, our method leads to a greater degree of solution diversity and interpretability, providing more information to tokamak designers without compromising performance. Our results suggest that the region of reversed central safety factor in STEP can be reduced at the cost of letting rational safety factor surfaces move slightly inwards, while keeping the minimum safety factor above two.


Perspectives of machine learning for acoustic methods of insect detection in stored products
Daniel Kadyrov
Stevens Institute of Technology

Stored products, such as grains and processed foods, are susceptible to infestation by various insects. Early detection of insects in the supply chain is crucial, as introducing invasive pests to new environments may cause disproportionate harm. While technologies and standards exist to aid detection, many food storage facilities and ports of entry still rely on manual visual inspection. Stevens developed the Acoustic Stored Product Insect Detection System to expedite the process. This system utilizes piezoelectric sensors to detect insect movements in infested commodities while featuring insulation and external sensors to mitigate against the influence of external noise. A large database of acoustic signatures was generated from testing the systems with the Callosobruchus maculutus, the Tribolium confusum, and the larval and adult stages of the Tenebrio molitor in various materials. Preliminary analysis and a detection algorithm were developed based on the extraction of time and frequency features of the received signals. Machine learning algorithms can leverage the dataset to improve the detection algorithm, suppress external noise, and allow insect classification and density estimation.


Mapping the Urban Heat: High-Resolution Air Temperature Analysis
AIfSR Team
New York University

As urban areas become increasingly vulnerable to rising temperatures, there is a critical need for targeted climate mitigation strategies. Our research focuses on the development of detailed air temperature and humidity maps for New York City and other metropolitan hubs, utilizing a combination of land surface temperature readings, atmospheric data, and additional environmental parameters. By producing these high-resolution maps, we aim to pinpoint communities facing the highest health risks due to extreme urban heat. The granularity of the data allows for a comprehensive understanding of temperature and humidity distribution patterns within the urban fabric, facilitating the assessment of existing mitigation efforts. Our poster presents the methodology, preliminary findings, and potential applications of these maps in formulating effective climate resilience policies and interventions designed to safeguard the most vulnerable urban populations. Join us in exploring the intersection of urban geography, public health, and environmental science as we chart a course towards sustainable urban living. 


Machine learning based analysis of movement of antiferromagnetic domain walls
AIfSR Team
New York University

While modern memory technology is based on ferromagnetic materials, there is a big interest in the use of antiferromagnetic materials due to advantages such as faster dynamics, stability and robustness, and higher density. However, there are many questions about the dynamics of the motion of antiferromagnetic domains. In order to study that, researchers perform experiments on such materials exposed to various conditions. Such work often produces a lot of graphical content that is hard to analyze with traditional software technologies. In this work, we show an example of an AI-based soft solution that enables physicists to test their hypotheses, formulate new ones, and open opportunities for novel experiments.



Automating Chemical Reaction Mechanism Reduction using Artificial Intelligence for Atmospheric Chemistry

Arijit Chakraborty

Columbia University


The objective of chemical reaction mechanism reduction is to obtain a condensed reaction mechanism that is a fairly accurate surrogate of the complete reaction mechanism and is considerably more memory and computationally efficient for subsequent tasks, such as real-time concentration prediction. When formulated as a search problem, the search space of potential candidate mechanisms could be larger than those of games such as Chess and Go combined. In this work, we present an artificially intelligent (AI) automated mechanism reduction of the gas-phase isoprene oxidation mechanism, termed Genetic Algorithmic - Automated Model Reduction (GA-AMORE). We highlight the efficacy of our approach in a well-established case study: atmospheric gas-phase isoprene oxidation. Formulated as a search problem through an astronomically large search space, we explore the same under mechanistic constraints, and subject matter expert rules, thus highlighting the benefits accrued from such a hybrid AI approach. We contrast the resultant mechanism(s) obtained to state-of-the-art manually tuned mechanisms of comparable size. Further, we obtain the optimal stoichiometric coefficients, and rate parameters, of the reduced reaction mechanisms using a derivative-free optimization strategy that significantly improves its accuracy. We posit that our approach can be extended to other mechanism reduction case studies by mere modification of the domain-specific rules, while keeping the underlying algorithm unchanged.



Decision-Focused Prediction of Strategic Energy Storage Behaviors
Ming Yi, Ph.D.
Columbia University


With the rapid increase in grid-scale energy storage deployment, it has become crucial for power system operators to accurately predict strategic energy storage behaviors in electricity markets, specifically the timings of charging and discharging. This prediction task is complex due to the intricate interplay of market fluctuations and decision-making processes,  constrained by physical power and energy limits. We introduce a novel, decision-focused, end-to-end methodology that directly forecasts storage charge and discharge profiles at sub-hourly resolutions, leveraging past pricing data and storage activity. This approach does not rely on future price predictions or  intermediary price forecasting models. Our method incorporates the prior knowledge of the storage model and infers the hidden reward that incentivizes the energy storage decisions. This is achieved through a dual-layer framework, combining a predictive layer with an optimization layer. We have also developed a  hybrid loss function for effective model training. The numerical experiments on synthetic and real-world energy storage data show that our approach achieves the best performance against existing benchmark methods, which shows the effectiveness of our method.



Interpretable Machine Learning for multimodal analyses of materials: X-ray absorption spectra and pair distribution functions

Tanaporn Na Narong, Ph.D.

Columbia University


Interpretable machine learning techniques can help material scientists and chemists extract valuable information and improve the prediction of materials’ properties by combining data from different modes of measurement. To demonstrate this application, we trained random forest models on X-ray near-edge absorption spectra (XANES) and atomic pair distribution function (PDF) data to predict coordination numbers and charge states of transition metal oxides. We analyzed the predictive performance and feature ranking to identify where important information lies in XANES and PDF and when it is desirable to use either or both data types for prediction. Using data available on the Materials Project database, our work demonstrates the potential of interpretable machine learning for multimodal material analyses and provides a framework to help scientists optimize their design of experiments.



A data-driven approach for star formation parameterization

Diane Salim

Rutgers University


Whilst star formation (SF) in the interstellar medium (ISM) and the physics that govern it are some of the most fundamental mechanisms needed to paint a nuanced understanding of galaxy evolution, attempts to construct closed-form analytic expressions that connect SF and physical variables that have been observed to influence it, such as the density and turbulence properties of surrounding gas, still exhibit substantial intrinsic scatter.

In this work we leverage recent advancements in machine learning (ML) and use neural network symbolic regression (SR) techniques to produce the first data-driven, ML-discovered analytic relations for SF using the publicly available FIRE-2 simulation suites, which have no explicit numerical sub-grid recipe for SF. We employ a genetic algorithm-based SR pipeline that assembles analytic functions to model a given dataset called PySR, training it to predict symbolic representations of a model for the star formation rate surface density (ΣSFR) at both 10 mega-years (Myr) and 100 Myr based on extracted variables from FIRE-2 galaxies. These variables include those dominated by small-scale characteristics such as gas surface densities, gas velocity dispersions and surface density of stars, as well as large-scale environmental properties like the dynamical time and the potential of gas. The equations that PySR finds to describe ΣSFR at both 10 and 100 Myr exhibit more influence from properties that show more consistency over the entire galaxy such as the dynamical time, pointing to the efficacy of “top-down” analytic models for describing SF. Furthermore, the equations found for the longer SFR timescale better capture the intrinsic physical scatter of the data within the Kennicutt-Schmidt plane, indicating that training on longer SFR timescales leads to less overfitting. Future applications of this process include investigation of equations found for observational data and input of such found equations into semi-analytic models of galaxies.



Data-Driven Verification of the Korteweg-De Vries Equation for Vacuum Quasi-axisymmetric Stellarators

Rishin Madan

Princeton University


Stellarators are a class of magnetic fusion devices which, unlike tokamaks, achieve confinement of plasma through 3D shaping of the magnetic field. Quasi-axisymmetric stellarators, where the magnetic field strength (in a particular choice of coordinates) is independent of one of the coordinates, have confinement properties equivalent to a tokamak. Finding integrable, quasiaxisymmetric fields is a difficult problem, both computationally and analytically. Here, we show the Korteweg-De Vries equation to be an effective model for modelling the magnetic field strength on a flux surface, hinting at a hidden lower-dimensionality. For this, we used the sparse regression tool pySINDy [1, 2], which recovers governing differential equations directly from data.

References:

[1] Brian M. de Silva, Kathleen Champion, Markus Quade, Jean-Christophe Loiseau, J. Nathan Kutz, and Steven L. Brunton., (2020). PySINDy: A Python package for the sparse identification of nonlinear dynamical systems from data. Journal of Open Source Software, 5(49), 2104, https://doi.org/10.21105/joss.02104

[2] Kaptanoglu et al., (2022). PySINDy: A comprehensive Python package for robust sparse system identification. Journal of Open Source Software, 7(69), 3994, https://doi.org/10.21105/joss.03994



Deep Learning-based Multimodal Analysis for Transition Metal Dichalcogenides

Shivani Bhawsar

Stevens Institute of Technology


In this study, we present a novel approach to enable high-throughput characterization of transition metal dichalcogenides (TMDs) across various layers, including mono-, bi-, tri-, four, and multilayers, utilizing a generative deep learning (DL)-based image-to-image translation method. Graphical features, including contrast, color, shapes, flake sizes, and their distributions, were extracted using color-based segmentation of optical images, and Raman and photoluminescence (PL) spectra of chemical vapor deposition (CVD)-grown and mechanically exfoliated TMDs. The labeled images to identify and characterize TMDs were generated using the Pix2Pix conditional generative adversarial network (cGAN), trained only on a limited dataset. Furthermore, our model demonstrated versatility by successfully characterizing TMD heterostructures, showing adaptability across diverse material compositions. 

Humanities and Social Sciences

GANs for Causal Inference: Harnessing Conditional Independence

Palak Bansal

New York University


This interdisciplinary research harnesses the power of Generative AI in Causal Inference and its subsequent applications in Economics and Political Science, showcasing the symbiotic relationship between AI and these scientific fields. The rigorous year-long research aims to develop a state-of-the-art Causal Inference technique: CausalGANs. The generative model used here is ‘Generative Adversarial Networks’ which dominates the field of image generation. We harness the essence of GANs to create,  from scratch, a causal inference technique that modifies the architecture of GANs to solve the fundamental problem of “Missing Counterfactuals” in Causal Inference.

The GAN algorithm simultaneously trains two models: a generator and a discriminator. At the core of the GAN algorithm is the search for a neural network model that can generate fake data, whose distribution is independent of the labeling of real versus fake data. Independence restrictions of this kind are front and center in causal inference models, where the distribution of potential outcomes under treatment and control, conditional on contextual variables, are independent of the realized treatment. This makes the GANs apparatus a good method for causal inference, where instead of pitting real versus fake data, we now strive to get distributions of potential outcomes for treated and non-treated as close as possible.


This research involves creating a new framework, mathematical proofs, and thorough experimentation with different data-generating processes, generator, and discriminator models. It contains robust results produced by running over 200 parallelized experiments for each different set of parameters on High Power Computing.

We could empirically verify these mathematical theorems defined for the framework:

(1) We can recover the parameters of the data-generating process through this adversarial framework, (2) The minimum of the loss function is attained close to the true data parameters, and (3) The minimizer provides the best estimator of the propensity score.


Retooling Histories of Alchemy through Machine Learning

Farzad Mahootian, Ph.D.

New York University


A surge of scholarship in alchemy, especially its early modern European variant, has brought to the fore a new and interesting approach to the history of chemistry. The vast body of non-European alchemies have yet to be explored in such detail. A robust evidence-based response to this situation is unimaginable without the assistance of advanced textual analysis techniques. Digital humanities and machine learning present powerful potentialities for framing and exploring a truly global history of alchemy. As a first step on this long-term goal, I selected a sample of 180 documents of 17th century English alchemy from the Early English Books Online database. Initial findings bring previously unavailable quantitative precision to certain well-known trends in the historical timeline of European alchemy. Assurances about the accuracy of machine learning analysis of the data can be secured only by iterative involvement of teams of interdisciplinary scholars. I discuss some of the complications and opportunities presented by choices made in various framings of the problem and issues involved in their execution. I’ll also articulate next steps, requirements for advancement, and conjectures about interactive crossovers between the content and form of this research project.

LUWA Dataset: Learning Lithic Use-Wear Analysis on Microscopic Images

Jing Zhang, Ph.D.

New York University


Lithic Use-Wear Analysis (LUWA) using microscopic images is an underexplored vision-for-science research area. It seeks to distinguish the worked material, which is critical for understanding archaeological artifacts, material interactions, tool functionalities, and dental records. However, this challenging task goes beyond the well-studied image classification problem for common objects. It is affected by many confounders owing to the complex wear mechanism and microscopic imaging, which makes it difficult even for human experts to identify the worked material successfully. In this paper, we investigate the following three questions on this unique vision task for the first time:(i) How well can state-of-the-art pre-trained models (like DINOv2) generalize to the rarely seen domain? (i) How can few-shot learning be exploited for scarce microscopic images? (i) How do the ambiguous magnification and sensing modality influence the classification accuracy? To study these, we collaborated with archaeologists and built the first open-source and the largest LUWA dataset containing 23,130 microscopic images with different magnifications and sensing modalities. Extensive experiments show that existing pre-trained models notably outperform human experts but still leave a large gap for improvements. Most importantly, the LUWA dataset provides an underexplored opportunity for vision and learning communities and complements existing image classification problems on common objects.

Certified Edge Unlearning for Graph Neural Networks

Kun Wu
Stevens Institute of Technology


The emergence of evolving data privacy policies and regulations has sparked a growing interest in the concept of "machine unlearning", which involves enabling machine learning models to forget specific data instances. In this paper, we specifically focus on edge unlearning in Graph Neural Networks (GNNs), which entails training a new GNN model as if certain specified edges never existed in the original training graph. Unlike conventional unlearning scenarios where data samples are treated as independent entities, edges in graphs exhibit correlation. Failing to carefully account for this data dependency would result in the incomplete removal of the requested data from the model. While retraining the model from scratch by excluding the specific edges can eliminate their influence, this approach incurs a high computational cost. To overcome this challenge, we introduce CEU, a Certified Edge Unlearning framework. CEU expedites the unlearning process by updating the parameters of the pre-trained GNN model in a single step, ensuring that the update removes the influence of the removed edges from the model. We formally prove that CEU offers a rigorous theoretical guarantee under the assumption of convexity on the loss function. Our empirical analysis further demonstrates the effectiveness and efficiency of CEU for both linear and deep GNNs - it achieves significant speedup gains compared to retraining and existing unlearning methods while maintaining comparable model accuracy to retraining from scratch.


Data Pipelines as a Foundation for AI Enabled Research Grant Management
Priyanka Bose
New York University

Data Pipelines are necessary for creating a foundation for AI Enabled Research Grant Management. We present the work of NYU data management is doing to support research and student success by enabling development of the pipelines using enterprise data available in NYU Administrative systems.


Disparate Vulnerability in Link Inference Attacks against Graph Neural Networks

Da Zhong

Stevens Institute of Technology


Graph Neural Networks (GNNs) have been widely used in various graph-based applications. Recent studies have shown that GNNs are vulnerable to link-level membership inference attacks (LMIA) which can infer whether a given link was included in the training graph of a GNN model. While most of the studies focus on the privacy vulnerability of the links in the entire graph,  none have inspected the privacy risk of specific subgroups of links (e.g., links between LGBT users). In this paper, we present the first study of disparity in subgroup vulnerability (DSV) of GNNs against LMIA. First, with extensive empirical evaluation, we demonstrate the existence of non-negligible DSV under various settings of GNN models and input graphs. Second, by both statistical and causal analysis, we identify the difference between three specific graph structural properties of subgroups as one of the underlying reasons for DSV. Among the three properties, the difference between subgroup density has the largest causal effect on DSV. Third, inspired by the causal analysis, we design a new defense mechanism named FairDefense to mitigate DSV while providing protection against LMIA. At a high level, at each iteration of target model training, FairDefense randomizes the membership of edges in the training graph with a given probability, aiming to reduce the gap between the density of different subgroups for DSV mitigation. Our empirical results demonstrate that FairDefense outperforms the existing defense methods in the trade-off between defense and target model accuracy. More importantly, it offers better DSV mitigation.


Unveiling Minds: AI-powered Morphological Analysis for Psychological Assessment & Therapeutic Understanding

AIfSR team

New York University


Despite the progress in computational language models, existing Large Language Models (LLMs) face limitations in effectively capturing morphological structures due to their tokenization methods. This proposal introduces a novel approach that amalgamates cutting-edge research in neurolinguistics with the innovative development of a generative model for multimorphemic English words. The goal is not only to advance our understanding of morphological processing but also to revolutionize therapeutic interventions by incorporating linguistic insights into psychological evaluations.

Why morphemes?

Linguistic morphemes, though often overlooked, offer valuable insights for psychological evaluations: 1. Language Development: Morpheme analysis aids in identifying developmental delays and language disorders in children. 2. Cognitive Processing: Morpheme usage reflects cognitive abilities, aiding in understanding neurodevelopmental and cognitive disorders. 3. Emotional Expression: Morphemes convey emotional nuances, aiding in assessing emotional states and coping mechanisms. 4. Communication Styles: Analysis of morpheme usage reveals communication patterns and social interaction dynamics. 5. Predictive Modeling: Morpheme analysis informs predictive modeling for risk assessment and treatment outcomes. 6. Neurolinguistic Research: Understanding morpheme processing informs neurolinguistic research and rehabilitation strategies. Hence, incorporating morpheme analysis will enrich psychological assessments, enabling tailored interventions and holistic approaches to mental health. Thus, this proposal introduces a novel approach that unites groundbreaking research in generative linguistics and neurolinguistics, creating a paradigm shift in understanding and applying linguistic morpheme insights. By merging these two innovative approaches, we aim to not only advance scientific knowledge but also to transform therapeutic interventions, offering more personalized and effective treatments for individuals.


CreateAI Platform
Ayat Sweid

Arizona State University


The CreateAI Platform is a platform that accelerates world-class AI innovation at ASU by empowering the ASU community to build and engage with AI-enabled products in a secure environment. It takes the challenging parts of building AI and transforms them into easy-to-understand tools for those who want to use AI but might not have engineering backgrounds. This marks a milestone in bridging the gap between cutting-edge research and tangible, real-world AI applications.