1. Physics-Informed Machine Learning (PIML)

This document provides comprehensive examples demonstrating Physics-Informed Neural Networks (PINNs) and their variants for solving partial differential equations (PDEs) in plasma physics and computational science.

1.1. Overview

Physics-Informed Machine Learning combines the power of neural networks with physical laws encoded as PDEs. Instead of relying solely on data, PINNs incorporate governing equations, boundary conditions, and initial conditions directly into the loss function, enabling:

  • Mesh-free solutions: No discretization or meshing required

  • Inverse problems: Simultaneous parameter identification and forward solving

  • Data efficiency: Physics constraints reduce data requirements

  • Continuous solutions: Differentiable solutions at arbitrary points

  • Multi-physics coupling: Natural handling of coupled PDEs

1.1.1. Why PINNs for Plasma Physics?

Plasma physics involves complex multi-scale phenomena with:

  • Highly nonlinear PDEs with temperature-dependent properties

  • Multi-physics coupling (thermal, electromagnetic, fluid dynamics)

  • Wide range of spatial and temporal scales

  • Limited experimental data in extreme conditions

PINNs offer a flexible framework for modeling these challenges without expensive grid-based simulations.

1.1.2. Available Implementations

AI4Plasma provides several PINN variants optimized for different scenarios:

  • Standard PINN: Basic implementation for general PDEs

  • CS-PINN: Coefficient-Subnet PINN for arc discharge with complex material properties

  • RK-PINN: Runge-Kutta PINN for time-dependent problems with better temporal accuracy

  • Meta-PINN: Meta-learning framework for fast adaptation across related problems

1.1.3. Common Features

All scripts in this section share:

  • Device Management: Automatic CPU/GPU selection via ai4plasma.config.DEVICE

  • Reproducibility: Fixed random seeds via ai4plasma.utils.common.set_seed

  • Monitoring: TensorBoard integration for real-time training visualization

  • Checkpointing: Automatic model saving for resumption and deployment

  • Visualization: Comparison plots and animated GIF generation

1.1.4. Output Organization

app/piml/{method}/
├── runs/       # TensorBoard event files for training monitoring
├── models/     # Saved model checkpoints (.pth files)
├── results/    # Output figures, GIFs, and CSV files
└── data/       # Input data (material properties, reference solutions)

1.2. Standard PINN

1.2.1. Theory

Physics-Informed Neural Networks [1] solve PDEs by minimizing a composite loss function:

\[ \mathcal{L} = \mathcal{L}_{\text{PDE}} + \lambda_{\text{BC}} \mathcal{L}_{\text{BC}} + \lambda_{\text{IC}} \mathcal{L}_{\text{IC}} \]

where:

  • \(\mathcal{L}_{\text{PDE}}\): PDE residual at interior collocation points

  • \(\mathcal{L}_{\text{BC}}\): Boundary condition residual

  • \(\mathcal{L}_{\text{IC}}\): Initial condition residual (for time-dependent problems)

  • \(\lambda_{\text{BC}}, \lambda_{\text{IC}}\): Weighting hyperparameters

Key Advantage: Uses automatic differentiation to compute derivatives required for PDE evaluation—no finite differences needed.

1.2.2. Examples

1.2.2.1. 1. One-Dimensional Poisson Equation

File: app/piml/pinn/solve_1d_pinn.py

Problem: Solve the second-order ODE with Dirichlet boundary conditions:

\[ \frac{d^2 u}{dx^2} = -\sin(x), \quad x \in [0, 1] \]

Boundary Conditions:

  • \(u(0) = 0\) (left boundary)

  • \(u(1) = 0\) (right boundary)

Analytical Solution: $\( u(x) = \sin(x) - x\sin(1) \)$

Network Architecture:

layers = [1, 50, 50, 50, 50, 1]  # Input: x → Output: u(x)
network = FNN(layers, act_fun=nn.Tanh())

Loss Function Weights:

  • PDE weight: \(w_{\text{PDE}} = 1.0\)

  • Boundary weight: \(w_{\text{BC}} = 10.0\) (higher to enforce BCs strictly)

Features:

  • Custom visualization callback with solution evolution

  • Animated training process (saved as GIF)

  • Memory-efficient history recording

  • Comparison with analytical solution

Training Configuration:

  • Epochs: 20,000

  • Learning rate: 0.001 (Adam optimizer)

  • Learning rate decay: 0.5× at epochs 10,000 and 15,000

  • Collocation points: 100 (interior), 2 (boundaries)

Run:

python app/piml/pinn/solve_1d_pinn.py

Expected Output:

  • Final relative error: < 0.1%

  • Training time: ~1-2 minutes

  • Output files:

    • results/1d_pinn_final.png: Final solution comparison

    • results/1d_pinn_animation.gif: Training evolution

    • runs/1d_pinn/: TensorBoard logs

1.2.2.2. 2. Two-Dimensional Poisson Equation (Rectangular Domain)

File: app/piml/pinn/solve_2d_rect_pinn.py

Problem: Solve 2D Poisson equation on rectangular domain:

\[ \nabla^2 u = \frac{\partial^2 u}{\partial x^2} + \frac{\partial^2 u}{\partial y^2} = -8\pi^2\sin(2\pi x)\sin(2\pi y) \]

Domain: \(\Omega = [0, 1] \times [0, 1]\)

Boundary Conditions: \(u = 0\) on all boundaries \(\partial\Omega\)

Analytical Solution:

\[ u(x, y) = \sin(2\pi x)\sin(2\pi y) \]

Network Architecture:

layers = [2, 50, 50, 50, 50, 1]  # Input: (x,y) → Output: u(x,y)
network = FNN(layers, act_fun=nn.Tanh())

Geometry: Uses GeoRect2D class for rectangular domain sampling

Sampling:

  • Interior points: \(50 \times 50 = 2500\) collocation points

  • Boundary points: \(50\) per edge × 4 edges = 200 points

Features:

  • 2D heatmap and contour plot visualization

  • Real-time monitoring of solution evolution

  • Automatic domain sampling with GeoRect2D

  • Colorbar and axis labels for publication-ready figures

Training Configuration:

  • Epochs: 30,000

  • Learning rate: 0.001 with exponential decay

  • Batch processing: All collocation points processed simultaneously

Run:

python app/piml/pinn/solve_2d_rect_pinn.py

Expected Output:

  • Final relative error: < 1%

  • Training time: ~5-10 minutes

  • 2D solution visualization with contours

1.2.2.3. 3. Two-Dimensional Poisson Equation (Polygonal Domain)

File: app/piml/pinn/solve_2d_poly_pinn.py

Problem: Same 2D Poisson equation as above, but on a polygonal domain.

Key Difference: Uses GeoPoly2D for arbitrary polygon shapes

Geometry Definition:

# Define polygon vertices (unit square as example)
points = np.array([[0.0, 0.0], [1.0, 0.0], [1.0, 1.0], [0.0, 1.0]])
geo = GeoPoly2D(points)

Advantages:

  • Handle complex geometries (L-shaped, triangular, arbitrary polygons)

  • Automatic boundary detection and sampling

  • No need for structured grids

Use Cases:

  • Irregular computational domains

  • Complex electrode geometries in plasma devices

  • Multi-region coupling problems

Run:

python app/piml/pinn/solve_2d_poly_pinn.py

1.3. CS-PINN (Coefficient-Subnet PINN)

1.3.1. Theory

CS-PINN extends standard PINNs for problems with complex nonlinear coefficients (e.g., temperature-dependent material properties). It employs:

  1. Coefficient Subnets: Separate neural networks or interpolators for material properties

  2. Automatic Boundary Enforcement: Network architecture guarantees boundary conditions by construction

  3. Spline Interpolation: Cubic splines for temperature-dependent plasma properties

Architecture for Arc Discharge:

Input: r (radius)  →  Neural Network  →  Temperature T(r)
                                            ↓
                            Material Properties: κ(T), σ(T), ε(T)
                                            ↓
                            PDE Residual: ∇·(κ∇T) - σE² + 4πε

Boundary Condition Enforcement:

Instead of using loss terms, CS-PINN constructs the network output as:

\[ T(r) = (r - R) \cdot N(r) + T_b \]

where \(N(r)\) is the neural network output. This guarantees \(T(R) = T_b\) regardless of network parameters.

1.3.2. Physical Model: Arc Discharge

Arc discharge is a high-temperature plasma phenomenon occurring in electrical devices (circuit breakers, welding arcs, lightning). The energy balance equation governs temperature distribution:

\[ \frac{1}{r}\frac{d}{dr}\left(r \kappa(T) \frac{dT}{dr}\right) = \sigma(T) E^2 - 4\pi \varepsilon_{\text{nec}}(T) \]

Physical Terms:

  • \(\kappa(T)\): Thermal conductivity [W/(m·K)] — heat conduction

  • \(\sigma(T)\): Electrical conductivity [S/m] — Joule heating

  • \(E\): Electric field [V/m] — determined by arc current \(I\)

  • \(\varepsilon_{\text{nec}}(T)\): Net emission coefficient [W/m³] — radiation loss

Material Property Interpolation:

  • Data from NIST, NASA, or experimental measurements

  • Cubic spline interpolation in log-log space

  • Automatic clamping to physical range [300 K, 30,000 K]

1.3.3. Examples

1.3.3.1. 1. Steady-State Arc Discharge

File: app/piml/cs_pinn/solve_1d_arc_steady_cs_pinn.py

Physical Problem: Solve steady-state energy balance for SF₆ arc plasma

Gas: SF₆ (Sulfur hexafluoride) — widely used in power systems

Physical Parameters:

  • Arc radius: \(R = 10\) mm

  • Arc current: \(I = 200\) A

  • Boundary temperature: \(T_b = 2000\) K

  • Temperature normalization: \(T_{\text{red}} = 10,000\) K

Network Architecture:

layers = [1, 50, 50, 50, 50, 50, 50, 1]  # 6 hidden layers, 50 neurons each
network = FNN(layers, act_fun=nn.Tanh())

Training Configuration:

  • Epochs: 100,000

  • Learning rate: 0.001

  • Collocation points: 500 (uniformly sampled)

  • Evaluation points: 600

Material Properties:

  • Thermodynamic properties: sf6_p1.dat (κ, σ, ρ, Cp vs T)

  • Radiation properties: sf6_p1_nec.dat (net emission coefficient)

Numerical Methods:

  • Gauss-Legendre quadrature (100 points) for arc conductance integral

  • Automatic differentiation for spatial derivatives

  • Adaptive learning rate scheduling (decay at milestones)

Features:

  • Real-time visualization with reference data comparison

  • Material property evolution plots (κ, σ, ε vs radius)

  • Automatic checkpoint saving every 10,000 epochs

  • TensorBoard logging for comprehensive monitoring

Run:

python app/piml/cs_pinn/solve_1d_arc_steady_cs_pinn.py

Expected Output:

  • Peak temperature: ~22,000-25,000 K at arc center

  • Training time: ~1-2 hours (100k epochs, GPU)

  • Relative error vs FVM: < 2%

  • Output files:

    • results/sta/arc_temperature.png: Temperature profile

    • results/sta/arc_properties.png: Material properties

    • results/sta/training_history.gif: Evolution animation

    • models/sta/model_final.pth: Trained model

Physical Insights:

  • Highest temperature at arc center (r = 0)

  • Sharp temperature gradient near boundary

  • Joule heating dominates in core, radiation in periphery

  • Electric field determined by conductance integral

1.3.3.2. 2. Transient Arc Discharge

File: app/piml/cs_pinn/solve_1d_arc_transient_cs_pinn.py

Physical Problem: Solve time-dependent energy balance with radial flow

Governing Equations:

Energy equation:

\[ \rho C_p \frac{\partial T}{\partial t} + \rho C_p v \frac{\partial T}{\partial r} = \frac{1}{r}\frac{\partial}{\partial r}\left(r\kappa\frac{\partial T}{\partial r}\right) + \sigma E^2 - 4\pi\varepsilon \]

Continuity equation (mass conservation):

\[ \frac{\partial (\rho r)}{\partial t} + \frac{\partial (\rho v r)}{\partial r} = 0 \]

Additional Parameters:

  • Time normalization: \(t_{\text{red}} = 1\) ms

  • Simulation duration: 10 ms (normalized: 10)

  • Initial condition: Steady-state solution from previous example

Network Architecture:

layers = [2, 300, 300, 300, 300, 300, 300, 2]  # Input: (r,t) → Output: (T,v)
network = FNN(layers, act_fun=nn.Tanh())

Key Features:

  • Coupled temperature and velocity fields

  • Automatic initial condition enforcement

  • Multi-time snapshot visualization

  • Higher network capacity (300 neurons/layer) for complexity

Training Configuration:

  • Epochs: 150,000

  • Learning rate: 0.0001 (lower for stability)

  • Spatial points: 200

  • Temporal points: 100

  • Evaluation times: t = 0.1, 0.5, 0.9 (normalized)

Run:

python app/piml/cs_pinn/solve_1d_arc_transient_cs_pinn.py

Expected Output:

  • Time-dependent temperature evolution

  • Radial velocity field development

  • Training time: ~3-5 hours (150k epochs, GPU)

  • Multi-panel plots showing different time snapshots

1.3.3.3. 3. Transient Arc Without Velocity

File: app/piml/cs_pinn/solve_1d_arc_transient_noV_cs_pinn.py

Simplification: Energy equation only (no radial flow)

Governing Equation:

\[ \rho C_p \frac{\partial T}{\partial t} = \frac{1}{r}\frac{\partial}{\partial r}\left(r\kappa\frac{\partial T}{\partial r}\right) + \sigma E^2 - 4\pi\varepsilon \]

Use Case: Early-stage arc development where flow is negligible

Advantages:

  • Faster training (single output variable)

  • Lower network capacity required

  • Clearer physical interpretation

Run:

python app/piml/cs_pinn/solve_1d_arc_transient_noV_cs_pinn.py

1.3.3.4. 4. Resume Training

File: app/piml/cs_pinn/resume_1d_arc_transient_cs_pinn.py

Purpose: Continue training from saved checkpoint

Features:

  • Load model state and optimizer state

  • Resume from specific epoch

  • Useful for long training runs or hardware interruptions

Usage:

# Specify checkpoint file
checkpoint_file = 'app/piml/cs_pinn/models/tra/model_epoch_50000.pth'

# Load and continue training
arc_model.load_checkpoint(checkpoint_file)
arc_model.train(num_epochs=100000, resume=True)

Run:

python app/piml/cs_pinn/resume_1d_arc_transient_cs_pinn.py

1.4. RK-PINN (Runge-Kutta PINN)

1.4.1. Theory

RK-PINN improves temporal accuracy for time-dependent PDEs by using implicit Runge-Kutta schemes. Instead of treating time as just another input coordinate, RK-PINN discretizes time explicitly while keeping spatial derivatives continuous.

Key Idea: At each time step \(t_n\), the solution \(u^{n+1}\) satisfies:

\[ u^{n+1} = u^n + \Delta t \sum_{i=1}^{q} b_i k_i \]

where \(k_i\) are stage derivatives computed from the PDE. The network outputs all stages simultaneously, enabling efficient implicit time stepping.

Advantages over Standard PINN:

  • Higher temporal accuracy (order q)

  • Better stability for stiff problems

  • Explicit time discretization provides clearer physical interpretation

1.4.2. Example: Corona Discharge

File: app/piml/rk_pinn/solve_1d_corona_rk_pinn.py

Physical Problem: Simulate time-dependent corona discharge in Argon gas

Corona Discharge: Non-equilibrium plasma near sharp electrodes (e.g., power lines, corona treaters)

Governing Equations:

Electron continuity:

\[ \frac{\partial n_e}{\partial t} = \nabla \cdot (D_e \nabla n_e - \mu_e n_e \mathbf{E}) + \alpha |\mu_e n_e \mathbf{E}| - \beta n_e n_p \]

Ion continuity:

\[ \frac{\partial n_p}{\partial t} = \nabla \cdot (D_p \nabla n_p + \mu_p n_p \mathbf{E}) + \alpha |\mu_e n_e \mathbf{E}| - \beta n_e n_p \]

Poisson equation (electric field):

\[ \nabla^2 \Phi = -\frac{e}{\epsilon_0}(n_p - n_e) \]

Physical Variables:

  • \(n_e\): Electron density [m⁻³]

  • \(n_p\): Positive ion density [m⁻³]

  • \(\Phi\): Electric potential [V]

  • \(\mathbf{E} = -\nabla \Phi\): Electric field [V/m]

Transport Coefficients (field-dependent):

  • \(D_e, D_p\): Diffusion coefficients [m²/s]

  • \(\mu_e, \mu_p\): Mobilities [m²/(V·s)]

  • \(\alpha\): Ionization coefficient [m⁻¹]

Physical Parameters:

  • Gas: Argon (Ar)

  • Radius: \(R = 10\) mm

  • Temperature: \(T = 600\) K

  • Pressure: \(P = 1\) atm

  • Applied voltage: \(V_0 = -10\) kV

  • Secondary emission coefficient: \(\gamma = 0.066\)

Network Architecture:

q = 300  # Number of RK stages (determines temporal accuracy)
layers = [1, 300, 300, 300, 300, 2*(q+1)]  # Output: (Φ, n_e) at all stages
network = FNN(layers, act_fun=nn.Tanh())

Normalization:

  • Density: \(N_{\text{red}} = 10^{15}\) m⁻³

  • Time: \(t_{\text{red}} = 5\) ns

  • Voltage: \(V_{\text{red}} = 10\) kV

Training Configuration:

  • Epochs: 100,000

  • Learning rate: 0.0001

  • Collocation points: 500

  • Time step: \(\Delta t = 1.0\) (normalized)

Features:

  • Implicit RK integration for stiffness

  • Multi-stage output prediction

  • Field-dependent transport coefficient interpolation

  • Initial condition from reference solution

Run:

python app/piml/rk_pinn/solve_1d_corona_rk_pinn.py

Expected Output:

  • Electron avalanche development

  • Space charge field evolution

  • Training time: ~2-4 hours (GPU)

  • Output files:

    • results/corona/electron_density.png

    • results/corona/potential_distribution.png

    • results/corona/electric_field.png

Physical Insights:

  • Ionization wave propagation from cathode

  • Space charge effects on electric field distortion

  • Avalanche-to-streamer transition at critical density

1.5. Meta-PINN

1.5.1. Theory

Meta-PINN applies meta-learning (learning-to-learn) to PINNs for rapid adaptation to new related tasks. Instead of training from scratch for each parameter configuration, Meta-PINN pre-trains on multiple tasks and fine-tunes quickly on new ones.

MAML Algorithm (Model-Agnostic Meta-Learning):

  1. Meta-Training: Learn initialization parameters \(\theta^*\) that work well across tasks

  2. Meta-Testing: Fine-tune from \(\theta^*\) with few gradient steps on new task

Benefits:

  • Fast adaptation to new parameter values (e.g., different gas mixtures)

  • Reduced training time for parametric studies

  • Better generalization across related physics problems

1.5.2. Example: Multi-Gas Arc Discharge

File: app/piml/meta_pinn/solve_1d_arc_steady_meta_pinn.py

Problem: Learn to solve steady arc discharge for various SF₆-N₂ gas mixtures

Gas Mixtures: SF₆:N₂ with varying nitrogen fractions (0%, 10%, 20%, …, 50%)

Why Meta-Learning?:

  • Each mixture has different material properties

  • Traditional approach: Train separate model for each mixture (~100k epochs each)

  • Meta-learning approach: Pre-train once, fine-tune for new mixture (~5k epochs)

Training Tasks: 5 SF₆-N₂ mixtures (10%, 20%, 30%, 40%, 50% N₂)

Test Tasks: 6 interpolated/extrapolated mixtures (5%, 15%, 25%, 35%, 45%, 55% N₂)

Meta-Training Configuration:

  • Outer loop (meta-epochs): 1,000

  • Inner loop (task adaptation): 5 gradient steps

  • Support points: 500 per task

  • Query points: 600 per task

  • Meta learning rate: 0.001

  • Task learning rate: 0.01

Network Architecture:

layers = [1, 50, 50, 50, 50, 50, 50, 1]
backbone_net = FNN(layers, act_fun=nn.Tanh())

Features:

  • Multi-task training with task sampling

  • Support/query set splitting for meta-learning

  • Rapid adaptation demonstration

  • Property table management for different mixtures

Run:

python app/piml/meta_pinn/solve_1d_arc_steady_meta_pinn.py

Expected Output:

  • Meta-trained model: models/meta/meta_model.pth

  • Fine-tuned models for test tasks

  • Comparison plots: meta-learning vs. training from scratch

  • Adaptation curves showing fast convergence

  • Training time: ~2-3 hours (meta-training)

  • Fine-tuning time: ~5 minutes per new task

Applications:

  • Parametric design studies (varying pressure, current, gas composition)

  • Uncertainty quantification with parameter variations

  • Real-time simulation with adaptive property updates

1.6. Best Practices

1.6.1. 1. Network Architecture Selection

Depth vs. Width:

  • Shallow & Wide (e.g., [1, 100, 100, 1]): Fast training, good for smooth solutions

  • Deep & Narrow (e.g., [1, 30, 30, 30, 30, 30, 1]): Better at learning complex patterns

Activation Functions:

  • Tanh: Smooth gradients, traditional choice for PINNs

  • Sin: Excellent for periodic solutions

  • SoftPlus: Avoids vanishing gradients, good for deep networks

Guidelines:

  • Start with 3-5 hidden layers, 50-100 neurons per layer

  • Increase depth for complex physics (multi-scale, sharp gradients)

  • Use wider networks for higher-dimensional problems

1.6.2. 2. Loss Function Weighting

Balancing Terms:

\[ \mathcal{L} = w_{\text{PDE}} \mathcal{L}_{\text{PDE}} + w_{\text{BC}} \mathcal{L}_{\text{BC}} + w_{\text{IC}} \mathcal{L}_{\text{IC}} \]

Typical Values:

  • \(w_{\text{PDE}} = 1.0\) (reference)

  • \(w_{\text{BC}} = 10.0\) - \(100.0\) (enforce strictly)

  • \(w_{\text{IC}} = 10.0\) - \(100.0\) (important for transient problems)

Adaptive Weighting:

  • Monitor individual loss components

  • Increase weight if corresponding term doesn’t decrease

  • Use gradient-based balancing (e.g., GradNorm)

1.6.3. 3. Collocation Point Sampling

Strategies:

  • Uniform: Simple, works for regular domains

  • Latin Hypercube Sampling (LHS): Better space-filling properties

  • Adaptive: Increase density where PDE residual is high

Number of Points:

  • 1D: 100-1000 points

  • 2D: 1000-10000 points

  • 3D: 10000-100000 points

Rule of Thumb: More points → better accuracy, but slower training

1.6.4. 4. Training Strategy

Learning Rate:

  • Initial: 0.001 - 0.01 (Adam optimizer)

  • Decay schedule: Exponential or step decay

  • Fine-tuning: Reduce to 0.0001 towards convergence

Epochs:

  • Simple problems: 10,000 - 30,000

  • Complex multi-physics: 100,000 - 500,000

  • Monitor validation loss to avoid overfitting

Optimization Tips:

  • Use learning rate warmup for first few hundred epochs

  • Apply gradient clipping (e.g., max_norm = 1.0) for stability

  • Check for loss stagnation and adjust learning rate

1.6.5. 5. Validation and Verification

Verification Methods:

  1. Analytical Solutions: Compare with known solutions for test cases

  2. Reference Data: Compare with FEM/FVM/experimental results

  3. Grid Convergence: Evaluate on increasingly fine evaluation grids

  4. Conservation Laws: Check mass, energy, momentum conservation

Error Metrics:

  • Relative L2 Error: \(\frac{\|u_{\text{pred}} - u_{\text{true}}\|_2}{\|u_{\text{true}}\|_2}\)

  • Pointwise Max Error: \(\max_i |u_{\text{pred}}(x_i) - u_{\text{true}}(x_i)|\)

  • PDE Residual Norm: \(\|\mathcal{F}[u_{\text{pred}}]\|_2\)

Target Accuracy:

  • Simple problems: < 0.1% relative error

  • Complex multi-physics: < 5% relative error

  • Engineering applications: < 10% typically acceptable

1.6.6. 6. Debugging Common Issues

Loss Not Decreasing:

  • Check loss function implementation

  • Verify PDE residual computation (use finite differences to validate autodiff)

  • Reduce learning rate

  • Increase network capacity

  • Check input/output normalization

Loss Plateaus Early:

  • Increase boundary condition weights

  • Add more collocation points

  • Use adaptive sampling

  • Try different activation functions

  • Implement learning rate warmup

Unphysical Solutions:

  • Verify boundary conditions are enforced correctly

  • Check physical parameter ranges (e.g., temperature clipping)

  • Increase boundary/initial condition weights

  • Use exact boundary enforcement (like CS-PINN)

Training Instability (NaN/Inf):

  • Apply gradient clipping

  • Reduce learning rate

  • Check for division by zero in PDE terms

  • Normalize inputs/outputs to [0, 1] or [-1, 1]

  • Use float64 precision instead of float32

1.6.7. 7. Computational Efficiency

GPU Acceleration:

  • Essential for large networks and many collocation points

  • Speedup: 10-100× compared to CPU

  • Recommended for problems with >10k collocation points

Memory Management:

  • Batch collocation points if GPU memory is limited

  • Use gradient checkpointing for very deep networks

  • Delete intermediate tensors explicitly

Parallelization:

  • Train multiple parameter configurations in parallel

  • Use data parallelism for large-scale sampling

1.7. Performance Benchmarks

Typical performance on standard hardware (single NVIDIA GPU):

Example

Network Size

Collocation Points

Epochs

Training Time

Final Error

1D Poisson

[1,50,50,50,1]

100

20k

~2 min

<0.1%

2D Poisson

[2,50,50,50,1]

2500

30k

~10 min

<1%

Steady Arc

[1,50×6,1]

500

100k

~2 hours

<2%

Transient Arc

[2,300×6,2]

20k

150k

~5 hours

<5%

Corona RK

[1,300×4,600]

500

100k

~4 hours

<10%

Meta-PINN

[1,50×6,1]

500×5 tasks

1k meta

~3 hours

<5%

Hardware: NVIDIA RTX 3090 or A100 GPU with PyTorch

1.8. Visualization and Monitoring

1.8.1. TensorBoard Integration

All examples support real-time monitoring via TensorBoard:

# Start TensorBoard server
tensorboard --logdir=app/piml

# Open browser to http://localhost:6006

Logged Metrics:

  • Total loss and individual components (PDE, BC, IC)

  • Learning rate schedule

  • Parameter histograms

  • Gradient norms

  • Custom metrics (temperature extrema, conservation errors)

1.8.2. Output Files

Checkpoints (.pth files):

  • Contain model state, optimizer state, epoch number

  • Enable training resumption

  • Can be loaded for inference without retraining

Figures (.png, .pdf):

  • Final solution comparisons

  • Error distributions

  • Material property plots

Animations (.gif):

  • Solution evolution during training

  • Multi-time snapshot sequences

  • Useful for presentations and debugging

CSV Files:

  • Numerical solution values

  • Error metrics over training

  • Reference data for comparison

1.9. Advanced Topics

1.9.1. Hard vs. Soft Boundary Conditions

Soft BCs (standard PINN): Include BC residual in loss

  • Pros: Simple implementation

  • Cons: BCs may not be satisfied exactly

Hard BCs (CS-PINN approach): Construct network to satisfy BCs exactly

  • Pros: Guaranteed satisfaction, faster convergence

  • Cons: Requires special network architecture

1.9.2. Transfer Learning

Scenario: Use trained model as initialization for related problem

Examples:

  • Different parameter values (current 200A → 300A)

  • Similar geometries (radius 10mm → 15mm)

  • Related physics (steady → transient initialization)

Approach:

  1. Load pre-trained model weights

  2. Fine-tune on new problem (typically 10-20% of original training)

  3. Significant time savings

1.9.3. Inverse Problems

PINNs naturally handle inverse problems: infer unknown parameters from data

Example: Infer electrical conductivity \(\sigma(T)\) from temperature measurements

\[ \mathcal{L} = \underbrace{\mathcal{L}_{\text{PDE}}}_{\text{physics}} + \underbrace{\lambda_{\text{data}} \sum_i (T_{\text{pred}}(x_i) - T_{\text{obs}}(x_i))^2}_{\text{data fidelity}} \]

Applications:

  • Material property identification

  • Boundary condition estimation

  • Source term reconstruction

1.10. Troubleshooting Guide

Problem

Possible Causes

Solutions

Loss not decreasing

Poor initialization, wrong gradients

Check autodiff, reduce LR, verify PDE

Slow convergence

Insufficient capacity, poor sampling

Increase network size, use LHS sampling

Unphysical oscillations

Under-constrained BCs

Increase BC weight, use hard BCs

NaN/Inf values

Numerical instability

Gradient clipping, normalize data, reduce LR

High memory usage

Too many collocation points

Batch processing, smaller network

GPU out of memory

Large network or batch

Reduce network size or collocation points

1.11. Further Reading

  • PINN Theory: Karniadakis et al., “Physics-informed machine learning,” Nature Reviews Physics, 2021

  • Plasma Physics: Fridman and Kennedy, “Plasma Physics and Engineering,” 2nd ed., 2011

  • Neural Networks: Goodfellow et al., “Deep Learning,” MIT Press, 2016

  • Meta-Learning: Hospedales et al., “Meta-learning in neural networks: A survey,” IEEE TPAMI, 2021

1.12. Quick Start Guide

For Beginners: Start with these examples in order:

  1. pinn/solve_1d_pinn.py — Learn basic PINN concepts

  2. pinn/solve_2d_rect_pinn.py — Extend to 2D problems

  3. cs_pinn/solve_1d_arc_steady_cs_pinn.py — Apply to realistic physics

For Plasma Physicists: Jump directly to:

  • Arc discharge: cs_pinn/solve_1d_arc_steady_cs_pinn.py

  • Corona discharge: rk_pinn/solve_1d_corona_rk_pinn.py

  • Multi-gas studies: meta_pinn/solve_1d_arc_steady_meta_pinn.py

For Method Developers: Explore advanced features:

  • Custom geometries: pinn/solve_2d_poly_pinn.py

  • Time integration: rk_pinn/solve_1d_corona_rk_pinn.py

  • Meta-learning: meta_pinn/solve_1d_arc_steady_meta_pinn.py