A few of the ideas, topics, and commonplaces that have been gaining steam on arXiv during the last few months (yes, I do a lot of filtering to keep away most of the genAI stuff; explainer).
1. High Bandwidth Memory: One of the foundational axioms of the contemporary economy is Moore's Law; it's also not really true going forward (there's an analogy here with fossil fuels). Part of the response is physical scaling up — the colossal ongoing build-up of data centers — but also technical shifts to focus more and more on parallelism, network, and memory performance.
Some recent articles:
- GeoPipe: a Geo-distributed LLM Training Framework with enhanced Pipeline Parallelism in a Lossless RDMA-enabled Datacenter Optical Transport Network
- Thermal Analysis of 3D GPU-Memory Architectures with Boron Nitride Interposer
- Effect of non-Fourier heat transport on temperature distribution in High Bandwidth Memory
- DL-PIM: Improving Data Locality in Processing-in-Memory Systems
- Stratum: System-Hardware Co-Design with Tiered Monolithic 3D-Stackable DRAM for Efficient MoE Serving
2. Causal attribution: We're probably entering the state where we no longer have to push for the use of causal analysis methods but have to push against their misuse. In any case, research has certainly exploded, which is a good thing.
Some recent articles:
- Evaluating Reasoning Faithfulness in Medical Vision-Language Models using Multimodal Perturbations
- Counterfactual explainability and analysis of variance
- Exploring Large Language Model as an Interactive Sports Coach: Lessons from a Single-Subject Half Marathon Preparation
- Hide-and-Shill: A Reinforcement Learning Framework for Market Manipulation Detection in Symphony-a Decentralized Multi-Agent System
- An Interventional Approach to Real-Time Disaster Assessment via Causal Attribution
3. Primordial non-Gaussianities: One of the mind-boggling facts about contemporary cosmology is that we are studying empirically questions about the isometry of the observable Universe and the mechanisms of its origins and deviations. I can't put it better than the first phrase of the abstract of the first paper linked below: The advent of Stage IV galaxy redshift surveys such as DESI and Euclid marks the beginning of an era of precision cosmology, with one key objective being the detection of primordial non-Gaussianities (PNG), potential signatures of inflationary physics.
Some recent articles:
- Hierarchical summaries for primordial non-Gaussianities
- Validating the Galaxy and Quasar Catalog-Level Blinding Scheme for the DESI 2024 analysis
- Primordial Physics in the Nonlinear Universe: mapping cosmological collider models to weak-lensing observables
- Primordial Physics in the Nonlinear Universe: signatures of inflationary resonances, excitations, and scale dependence
- Beyond the two-point correlation: constraining primordial non-gaussianity with density perturbation moments