👋 Hi, I’m Sam!

• I’m:

  • currently a computational scientist at Argonne National Laboratory (ALCF).

  • generally interested in the application of AI + HPC to computational problems in science¹,

  • usually working on scaling large (language, vision, multi-modal) models across thousands of GPUs 🚀

    • If this sounds like something you’d be interested in doing, please feel free to reach out to me

 

🎤 Recent Talks

You can get a live view of some of my recent talks here

Now Playing:

📍 How I got here

My current research focuses on using deep generative modeling to help build better sampling algorithms in lattice gauge theory. In particular, I’m interested in building gauge equivariant neural network architectures and using inductive priors to incorporate physical symmetries into machine learning models.

I received my PhD in Physics from the University of Iowa in 2019 and my thesis was on Learning Better Physics: A Machine Learning Approach to Lattice Gauge Theory. Prior to this, I completed two bachelors degrees (Engineering Physics and Applied Mathematics, 2015) at The University of Illinois at Urbana-Champaign. My undergraduate dissertation was titled Energy Storage in Quantum Resonators and was supervised by Professor Alfred Hübler within the Center for Complex Systems Research at UIUC².

🤔 What I work on

As a member of the AI / ML Group at ALCF, I work on:

• AI + Science

  • Building better sampling methods for Lattice QCD

  • GenSLM: Genome-Scale Language Models

  • Foundation models for long term climate forecasting

• Scaling Large Language Models

• Optimizing distributed training across thousands of GPUs

• Building new parallelism techniques for efficient scaling

• Generative modeling (esp. for physical systems)

📝 Work

• Intro to HPC Bootcamp: Engaging New Communities Through Energy Justice Projects, Journal of Computational Science, 2024

• Thorough Characterization and Analysis of Large Transformer Model Training At-Scale, Proc. ACM Meas. Anal. Comput. Syst. 8 March 2024

• MLMC: Machine Learning Monte Carlo for Lattice Gauge Theory, S. Foreman, X.Y. Jin, & J.C. Osborn, Lattice, 2023 (Proceedings), Dec 2023

• Protein Generation via Genome-scale Language Models with Bio-physical Scoring, SC’23, Nov 2023

• DeepSpeed4Science Initiative: Enabling Large-Scale Scientific Discovery […], NeurIPS 2023 AI For Science Workshop, Oct 2023

  • DeepSpeed4Science.ai Blog Post
  • Loooooooong Sequence Lengths

• A Comprehensive Performance Study of Large Language Models on Novel AI Accelerators, M. Emani, S. Foreman, et al., IPDPS 2024, Oct 2023

• Exploratory Analysis of Climate Data with ClimRR, S. Foreman, Intro to HPC Bootcamp @ NERSC, August 7, 2023

• GenSLMs: Genome-scale language models reveal SARS-CoV-2 evolutionary dynamics, Oct 2022

  • ACM Gordon Bell Special Prize for HPC-Based COVID-19 Research

• Lattice QCD and Particle Physics, A.S. Kronfeld et al., July 15, 2022

• Applications of ML to Lattice QFT, arXiv:2202.05838, D. Boyda, S. Calí, S. Foreman, et al., Feb 2022

• LeapFrogLayers: Trainable Framework for Effective Sampling, S. Foreman, X.Y. Jin, J.C. Osborn, Lattice, 2021

• HMC with Normalizing Flows, slides, S. Foreman et al., Lattice, 2021

• Deep Learning Hamiltonian Monte Carlo (+ poster), S. Foreman, X.Y. Jin, & J.C. Osborn, @ SimDL Workshop @ ICLR, 2021

• Machine Learning and Neural Networks for Field Theory, S. Foreman, X.Y. Jin, & J.C. Osborn, SnowMass, 2020

• Examples of renormalization group transformations for image sets, S. Foreman et al., Physical Review E., 2018

• RG inspired Machine Learning for lattice field theory S. Foreman et al., arXiv:1710.02079, 2017

• Large Energy Density in Three-Plate Nanocapacitors due to Coulomb Blockade, S. Foreman et al., J. Appl. Phys, 2018

🎤 Talks

• 🪧Parallel Training Techniques @ AI-4-Science Training Series (03/2024)
• 🪧 LLMs from Scratch @ LLM Tutorial Workshop

• 🪧 Creating Small(-ish) LLMs @ LLM Tutorial Workshop (1), (11/2023)
• 🪧 Exascale Science on Aurora @ Intel oneAPI Workshop @ UIC, (10/2023)
• 🪧 LLM Lunch Talk @ ALCF Hands On HPC Workshop, (10/2023)
• 🪧 Scaling LLMs 4 Science @ Data-Intensive Computing and AI/ML at Scale, (08/2023)
• 🪧 MLMC: Machine Learning Monte Carlo for Lattice Gauge Theory, @ Lattice 2023, (07/2023)
• 🪧 Generative Modeling and Efficient Sampling, @ PASC23, (07/2023)
• 🪧 Efficient Sampling for Lattice Gauge Theory, @ Deep Fridays @ U. Bologna, (04/2023)
• 🪧 Large Scale Training, @ AI-4-Science on Supercomputers, (02/2023)
• 🪧 Hyperparameter Management, @ 2022 ALCF Simulation, Data, and Learning Workshop, (10/2022)
• 🪧 Statistical Learning, @ ATPESC 2022, (08/2022) 📕 accompanying notebook
• 🪧 Scientific Data Science: An Emerging Symbiosis @ Argonne National Laboratory, (05/2022)
• 🪧 Machine Learning in HEP, @ UNC Greensboro, (03/2022)
• 🪧 Accelerated Sampling Methods for Lattice Gauge Theory @ BNL Workshop: “DWQ @ 25”, (12/2021)
• 🪧 Training Topological Samplers for Lattice Gauge Theory @ ML4HEP @ ECT* Trento, (09/2021)
• 🪧 l2hmc-qcd @ MIT Lattice Group Seminar, (2021)
• 🪧 Deep Learning HMC for Improved Gauge Generation @ ML Techniques in LQCD, (2021)
• 🪧 Machine Learning for Lattice QCD @ University of Iowa, (2020)
• 🪧 Machine learning inspired analysis of the Ising model transition @ Lattice, 2018

• Machine Learning Analysis of Ising Worms @ Brookhaven National Laboratory, (2017)

📦 Projects

ezpz 🍋 [web]

Distributed training, ezpz

Megatron-DeepSpeed 🤖

For the largest of large language models.

wordplay 🎮💬 [web]

[nanoGPT]³+{🤗datasets,DeepSpeed}

ai-science-training-series 👨🏻‍🏫 [web]

Lecture series, “AI-4-Science-on-Supercomputers”

enrich 💸 [web]

Python’s logging, with Rich

ambivalent 🤷🏻‍♂️ [web]

Clean, simple style for Matplotlib figures⁴.

l2hmc-qcd 🔲 [web]

Accelerated samplers for Lattice QCD

climate-analysis 🌍 [web]

Climate Analysis project using ClimRR data

GitHub Stats

GitHub Stats (saforem2):

l2hmc-qcd

GitHub repo

🎪 Events

• Organizer for SC23 Workshop: High Performance Python for Science at Scale (HPPSS), November 2023

• Organizer for Machine Learning and Quantum Computing for Earth Sciences at 17th U. S. National Congress on Computational Mechanics, July 2023

🥷 Experience

Table 1: 📟 Experience

Assistant Computational Scientist	ALCF	2022	–
Postdoc	ALCF	2019	2022
Graduate Researcher	ANL	2018	2019

🎒 Education

Table 2: 🎓 Education

PhD	Physics	University of Iowa	2019
B.Sc	Physics	UIUC	2015
B.Sc	Math	UIUC	2015

❤️‍🩹 Status

Last Updated: 04/20/2024 @ 14:19:31

Footnotes

1. So far, for: {Lattice QCD, Quantum Mechanics, Biology (Protein Generation, Drug Discovery), and Climate Modeling / Weather Forecasting}

2. And resulted in a patent !!

3. ⚡ powered by ezpz 🍋

4. Forked from saforem2/opinionated