Self-Hosted Particle Physics Data Analysis: ROOT vs uproot vs Awkward Array

Introduction

Particle physics experiments at CERN’s Large Hadron Collider (LHC) generate petabytes of collision data annually — the ATLAS and CMS detectors alone produce over 100 petabytes each year. Analyzing this data requires specialized frameworks designed for the unique challenges of high-energy physics (HEP): hierarchical event structures, jagged arrays of varying-length particle collections, four-vector mathematics, and statistical inference at the boundaries of the Standard Model.

This guide compares three open-source frameworks — ROOT, uproot, and Awkward Array — that power particle physics data analysis, from the LHC to neutrino observatories and dark matter searches.

Why Self-Host Particle Physics Analysis?

Data locality is paramount in HEP. Raw collision data runs to exabytes, but derived analysis datasets (NTuples, NanoAOD) are typically 100 GB–10 TB. Transferring these over institutional networks to cloud providers introduces days of latency and significant egress costs. Self-hosting analysis infrastructure on a local cluster with 100 TB of NVMe storage allows physicists to iterate on analyses in minutes rather than days.

Computational reproducibility is essential for results that claim 5-sigma discoveries. Self-hosted environments with pinned software versions, containerized analysis frameworks, and documented data processing pipelines allow entire analyses to be reproduced years later — a requirement for peer-reviewed publication. The CERN Analysis Preservation portal and REANA platform both emphasize self-hosted reproducibility as a core principle.

Specialized hardware requirements set HEP apart from typical cloud workloads. ROOT file I/O benefits enormously from NVMe SSDs (5-10× faster than cloud block storage), while vectorized analysis with Awkward Array leverages AVX-512 instructions available on modern server CPUs but rarely exposed in virtualized cloud instances. Self-hosting gives you control over the hardware configuration that optimizes for these domain-specific workloads.

For HPC cluster management, see our HPC workload managers guide. For containerized deployment, our HPC container runtimes comparison covers Singularity/Apptainer setups. For scientific data management, check our scientific data management guide.

ROOT: The CERN Standard

ROOT is the foundational framework for particle physics data analysis, developed at CERN with 3,221 GitHub stars. For over 25 years, ROOT has been the primary tool for every major LHC physics result. It provides a complete ecosystem: a columnar file format (.root), a C++ interpreter (Cling), statistical analysis tools (RooFit, RooStats), visualization (histograms, graphs, 2D/3D), and a Python interface (PyROOT).

ROOT’s .root file format is the universal data exchange format in HEP. It stores hierarchical, compressed data with efficient random access — you can read a single branch from a 10 TB file without decompressing the rest. The TTree data structure is optimized for columnar access patterns typical in HEP: iterate over all events, extract specific branches, apply selections.

Docker Deployment

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16

# docker-compose.yml — ROOT analysis server
version: "3.8"
services:
  root:
    image: rootproject/root:6.32.04-ubuntu24.04
    container_name: root-analysis
    volumes:
      - ./data:/data
      - ./analysis:/analysis
      - ./results:/results
    working_dir: /analysis
    environment:
      - ROOT_HIST=0
      - ROOT_INCLUDE_PATH=/analysis
    entrypoint: ["root"]
    command: ["-l", "-q", "analyze.C"]

ROOT analysis in C++ (macro):

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45

// analyze.C — ROOT analysis macro for LHC data
void analyze() {
    // Open input NanoAOD file
    TFile *f = TFile::Open("/data/nanoaod.root");
    TTree *events = (TTree*)f->Get("Events");

    // Define branch readers
    UInt_t nMuon;
    Float_t Muon_pt[100], Muon_eta[100], Muon_phi[100];
    Int_t Muon_charge[100];

    events->SetBranchAddress("nMuon", &nMuon);
    events->SetBranchAddress("Muon_pt", Muon_pt);
    events->SetBranchAddress("Muon_eta", Muon_eta);
    events->SetBranchAddress("Muon_phi", Muon_phi);
    events->SetBranchAddress("Muon_charge", Muon_charge);

    // Output histogram
    TH1F *h_mass = new TH1F("h_mass", "Z boson mass;m_{#\mu\mu} [GeV];Events",
                            80, 60, 120);

    // Event loop
    Long64_t nentries = events->GetEntries();
    for (Long64_t i = 0; i < nentries; i++) {
        events->GetEntry(i);

        for (UInt_t j = 0; j < nMuon; j++) {
            for (UInt_t k = j + 1; k < nMuon; k++) {
                if (Muon_charge[j] * Muon_charge[k] > 0) continue;

                TLorentzVector p1, p2;
                p1.SetPtEtaPhiM(Muon_pt[j], Muon_eta[j], Muon_phi[j], 0.10566);
                p2.SetPtEtaPhiM(Muon_pt[k], Muon_eta[k], Muon_phi[k], 0.10566);

                float mass = (p1 + p2).M();
                h_mass->Fill(mass);
            }
        }
    }

    // Draw and save
    TCanvas *c = new TCanvas("c", "Z mass", 800, 600);
    h_mass->Draw();
    c->SaveAs("/results/z_mass_peak.png");
}

uproot: ROOT I/O in Pure Python

uproot (269 GitHub stars) is part of the Scikit-HEP ecosystem that brings modern Python to particle physics. uproot reads and writes ROOT files without requiring the ROOT C++ library — it’s a pure Python implementation of the ROOT I/O specification using NumPy for array operations.

uproot’s key advantage is its seamless integration with the Python data science ecosystem. You can load ROOT data directly into NumPy arrays, pandas DataFrames, or Awkward Arrays with a single function call, then use the full power of Matplotlib, SciPy, and scikit-learn for analysis.

Docker Deployment

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17

# docker-compose.yml — uproot analysis server
version: "3.8"
services:
  uproot:
    image: python:3.12-slim
    container_name: uproot-analysis
    volumes:
      - ./data:/data
      - ./analysis:/analysis
      - ./results:/results
    working_dir: /analysis
    command:
      - bash
      - -c
      - |
        pip install uproot awkward numpy scipy matplotlib hist mplhep
        python analyze.py

uproot analysis in Python:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60

# analyze.py — Z boson mass peak with uproot
import uproot
import numpy as np
import hist
import mplhep as hep
import matplotlib.pyplot as plt

# Open ROOT file (no C++ ROOT required!)
file = uproot.open("/data/nanoaod.root")
events = file["Events"]

# Load branches as Awkward Arrays (lazy, memory-efficient)
muon_pt = events["Muon_pt"].array()
muon_eta = events["Muon_eta"].array()
muon_phi = events["Muon_phi"].array()
muon_charge = events["Muon_charge"].array()

# Select events with at least 2 muons
mask = (uproot.AsJagged(muon_pt).counts >= 2)

# Compute di-muon invariant mass
# For each event, pair the two highest-pT opposite-sign muons
mass_hist = hist.Hist.new.Reg(80, 60, 120, name="mass").Double()

for i in range(len(muon_pt)):
    n_mu = len(muon_pt[i])
    best_mass = 0

    for j in range(min(n_mu, 3)):  # Check top 3 by pT
        for k in range(j + 1, min(n_mu, 3)):
            if muon_charge[i][j] * muon_charge[i][k] > 0:
                continue

            # Four-momentum from pt, eta, phi
            px1 = muon_pt[i][j] * np.cos(muon_phi[i][j])
            py1 = muon_pt[i][j] * np.sin(muon_phi[i][j])
            pz1 = muon_pt[i][j] * np.sinh(muon_eta[i][j])
            E1 = np.sqrt(px1**2 + py1**2 + pz1**2 + 0.10566**2)

            px2 = muon_pt[i][k] * np.cos(muon_phi[i][k])
            py2 = muon_pt[i][k] * np.sin(muon_phi[i][k])
            pz2 = muon_pt[i][k] * np.sinh(muon_eta[i][k])
            E2 = np.sqrt(px2**2 + py2**2 + pz2**2 + 0.10566**2)

            mass = np.sqrt((E1 + E2)**2 - (px1 + px2)**2 -
                          (py1 + py2)**2 - (pz1 + pz2)**2)
            if mass > best_mass:
                best_mass = mass

    if best_mass > 0:
        mass_hist.fill(mass=best_mass)

# Plot with HEP styling
plt.style.use(hep.style.CMS)
fig, ax = plt.subplots(figsize=(10, 7))
mass_hist.plot(ax=ax)
ax.set_xlabel("$m_{\mu\mu}$ [GeV]")
ax.set_ylabel("Events")
ax.set_title("Z $\rightarrow \mu^+\mu^-$ Candidate Mass")
plt.savefig("/results/z_mass_peak_uproot.png", dpi=150)

Awkward Array: Columnar Data for Jagged Structures

Awkward Array (962 GitHub stars) addresses the fundamental data structure challenge in particle physics: collision events produce jagged arrays — each event contains a variable number of particles (muons, electrons, jets), each with multiple properties (pT, eta, phi, mass). Traditional rectangular arrays (NumPy) can’t efficiently represent this.

Awkward Array provides NumPy-like operations on jagged, nested data structures while maintaining columnar memory layout for cache efficiency and vectorization. Operations like ak.sum(array, axis=1) or ak.combinations(array, 2) are expressed declaratively and compiled to optimized loops using the Awkward-CPP backend.

Docker Deployment

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16

# docker-compose.yml — Awkward Array analysis server
version: "3.8"
services:
  awkward:
    image: python:3.12-slim
    container_name: awkward-analysis
    volumes:
      - ./data:/data
      - ./analysis:/analysis
    working_dir: /analysis
    command:
      - bash
      - -c
      - |
        pip install awkward uproot numpy scipy vector hist mplhep
        python analyze_awkward.py

Awkward Array vectorized analysis:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49

# analyze_awkward.py — fully vectorized Z mass analysis
import uproot
import awkward as ak
import numpy as np
import vector
import hist
import matplotlib.pyplot as plt
import mplhep as hep

# Load data as Awkward Arrays
file = uproot.open("/data/nanoaod.root")
events = file["Events"].arrays(
    ["Muon_pt", "Muon_eta", "Muon_phi", "Muon_charge", "Muon_mass"],
    library="ak"
)

# Filter events with >= 2 muons
has_dimuon = ak.num(events.Muon_pt) >= 2
events = events[has_dimuon]

# Build Lorentz vectors (vectorized!)
muons = vector.zip({
    "pt": events.Muon_pt,
    "eta": events.Muon_eta,
    "phi": events.Muon_phi,
    "mass": events.Muon_mass,
})

# Take the two highest-pT muons per event
mu1 = muons[:, 0]  # Already sorted by descending pT
mu2 = muons[:, 1]

# Require opposite charges
opp_sign = events.Muon_charge[:, 0] != events.Muon_charge[:, 1]

# Compute di-muon invariant mass (fully vectorized!)
zmass = (mu1[opp_sign] + mu2[opp_sign]).mass

# Plot
plt.style.use(hep.style.CMS)
fig, ax = plt.subplots(figsize=(10, 7))
h = hist.Hist.new.Reg(80, 60, 120).Double()
h.fill(ak.to_numpy(zmass))
h.plot(ax=ax)
ax.set_xlabel("$m_{\mu\mu}$ [GeV]")
ax.set_ylabel("Events / 0.75 GeV")
hep.cms.label(ax=ax, data=True, lumi=59.8, year=2024)
plt.savefig("/results/z_mass_awkward.png", dpi=150)
print(f"Events in Z peak (80-100 GeV): {ak.sum((zmass > 80) & (zmass < 100))}")

Performance Comparison

Benchmarks on a nanoAOD file with 10 million events (8 GB compressed, ~35 GB decompressed), 2× Xeon Gold 6338 (64 cores total), 512 GB RAM, NVMe SSD:

The Awkward Array approach achieves near-C++ performance with dramatically fewer lines of code through vectorized operations, while uproot with pure NumPy loops shows the cost of Python-level iteration over per-event data.

Comparison Table

Choosing the Right Framework

Choose ROOT when:

• You’re working within an established HEP collaboration (ATLAS, CMS, LHCb, ALICE)
• You need the full analysis ecosystem (RooFit, RooStats, TMVA)
• Performance is critical and you’re comfortable with C++
• You need to produce publication-quality ROOT files
• Your analysis involves complex statistical models beyond simple event counting

Choose uproot when:

• You want to use Python’s rich data science ecosystem (pandas, scikit-learn, Jupyter)
• You only need to read ROOT files (not write complex output structures)
• You’re building analysis pipelines that mix HEP data with other data sources
• Team members are more comfortable with Python than C++
• You’re doing exploratory analysis or prototyping

Choose Awkward Array when:

• Your data has complex jagged/nested structures (jets with variable constituents)
• You want to write highly concise, vectorized analysis code
• Performance matters but you want to stay in Python
• You’re building reusable analysis functions that work across different datasets
• You need to interoperate with ML frameworks (JAX, PyTorch) via dlpack

FAQ

Can I use these tools without installing the full ROOT framework?

Yes. uproot and Awkward Array are pip-installable (pip install uproot awkward) and have no dependency on the ROOT C++ library. They implement the ROOT file format specification in pure Python (with optional C++ extensions for Awkward). For reading ROOT files and performing standard analyses, you can use them as completely standalone tools.

How do these tools handle very large datasets?

ROOT’s TTree with TChain transparently handles multi-terabyte datasets spread across hundreds of files. uproot supports lazy loading — branches are only decompressed and read into memory when you access them. Awkward Array uses lazy dask-awkward integration for out-of-core computation on datasets that exceed RAM, automatically partitioning work across files and parallelizing on multi-core systems.

What about GPU acceleration?

Awkward Array provides a ak.to_backend() function that converts data to CuPy or JAX arrays for GPU computation. Combined with JAX’s JIT compilation and GPU-accelerated vector operations, this enables 10-50× speedups for compute-bound analyses. ROOT has experimental CUDA support via the ROOT::RDataFrame with DefinePerSample for GPU-accelerated operations, though adoption remains limited.

How do I convert between ROOT and other formats?

uproot converts ROOT → pandas (arrays(library="pd")), ROOT → NumPy, ROOT → Awkward, and ROOT → Parquet (via ak.to_parquet()). For ROOT → HDF5, use h5py with NumPy intermediates. For large-scale format conversion, Apache Spark with the spark-root connector can parallelize ROOT → Parquet/ORC transformations across clusters.

Which framework should I recommend to new graduate students?

Start with the uproot + Awkward Array + hist stack for the first 3-6 months. The Python-first approach, Jupyter notebook integration, and readable syntax reduce the learning curve dramatically. Once they understand the physics, introduce ROOT’s RooFit/RooStats for advanced statistical modeling in their analysis. Most LHC analyses now use a hybrid approach: data loading and event selection with uproot/Awkward, final statistical fits with ROOT’s RooFit.

💰 想测试你的市场判断力？我用 Polymarket 做预测市场交易——这是全球最大的预测市场平台，从大选结果到技术监管时间线，什么都可以押注。和赌博不同，这是真正的信息市场：你懂的信息越多，胜率越高。我靠预测技术相关事件的走向已经赚了不少。用我的邀请链接注册：Polymarket.com