Machine Learning for NuWro
The project within the NCN grant, UMO-2021/41/B/ST2/02778, led by Prof. Jan Sobczyk.
The goal is to optimize the NuWro Monte Carlo generator utilizing machine learning tools, particularly deep learning techniques.

• Empirical fits to inclusive electron-carbon scattering data obtained by deep-learning methods,
  Beata Kowal, Krzysztof M. Graczyk, Artur M. Ankowski, Rwik Dharmapal Banerjee, Hemant Prasad, Jan T. Sobczyk,
  arxiv:2312.17298
  Abstract:
  
• Employing the neural network framework, we obtain empirical fits to the electron-scattering cross section for carbon over a broad kinematic region, extending from the quasielastic peak, through resonance excitation, to the onset of deep-inelastic scattering. We consider two different methods of obtaining such model-independent parametrizations and the corresponding uncertainties: based on the NNPDF approach [J. High Energy Phys. 2002, 062], and on the Monte Carlo dropout. In our analysis, the ?2 function defines the loss function, including point-to-point uncertainties and considering the systematic normalization uncertainties for each independent set of measurements. Our statistical approaches lead to fits of comparable quality and similar uncertainties of the order of 7% and 12% for the first and the second approaches, respectively. To test these models, we compare their predictions to a~test dataset, excluded from the training process, a~dataset lying beyond the covered kinematic region, and theoretical predictions obtained within the spectral function approach. The predictions of both models agree with experimental measurements and the theoretical predictions. However, the first statistical approach shows better interpolation and extrapolation abilities than the one based on the dropout algorithm.
  

Physics Informed Neural Networks

• Bayesian Reasoning for Physics Informed Neural Networks,
  Krzysztof M. Graczyk, Kornel Witkowski,
  arxiv:2308.13222
  Abstract:
  
• Physics informed neural network (PINN) approach in Bayesian formulation is presented. We adopt the Bayesian neural network framework formulated by MacKay (Neural Computation 4 (3) (1992) 448). The posterior densities are obtained from Laplace approximation. For each model (fit), the so-called evidence is computed. It is a measure that classifies the hypothesis. The most optimal solution has the maximal value of the evidence. The Bayesian framework allows us to control the impact of the boundary contribution to the total loss. Indeed, the relative weights of loss components are fine-tuned by the Bayesian algorithm. We solve heat, wave, and Burger's equations. The obtained results are in good agreement with the exact solutions. All solutions are provided with the uncertainties computed within the Bayesian framework.
  

Deep Learning in Porous Media

• Deep learning for diffusion in porous media,
  K. M. Graczyk, D. Strzelczyk and M. Matyka, Sci Rep 13, 9769 (2023)
  Abstract:
• We adopt convolutional neural networks (CNN) to predict the basic properties of the porous media. Two different media types are considered: one mimics the sand packings, and the other mimics the systems derived from the extracellular space of biological tissues. The Lattice Boltzmann Method is used to obtain the labeled data necessary for performing supervised learning. We distinguish two tasks. In the first, networks based on the analysis of the system’s geometry predict porosity and effective diffusion coefficient. In the second, networks reconstruct the concentration map. In the first task, we propose two types of CNN models: the C-Net and the encoder part of the U-Net. Both networks are modified by adding a self-normalization module [Graczyk et al. in Sci Rep 12, 10583 (2022)]. The models predict with reasonable accuracy but only within the data type, they are trained on. For instance, the model trained on sand packings-like samples overshoots or undershoots for biological-like samples. In the second task, we propose the usage of the U-Net architecture. It accurately reconstructs the concentration fields. In contrast to the first task, the network trained on one data type works well for the other. For instance, the model trained on sand packings-like samples works perfectly on biological-like samples. Eventually, for both types of the data, we fit exponents in the Archie’s law to find tortuosity that is used to describe the dependence of the effective diffusion on porosity.
  



• Predicting Porosity, Permeability, and Tortuosity of Porous Media from Images by Deep Learning,
  K. M. Graczyk and M. Matyka, Sci Rep 10, 21488 (2020)
  Abstract:
• Convolutional neural networks (CNN) are utilized to encode the relation between initial configurations of obstacles and three fundamental quantities in porous media: porosity (?), permeability (k), and tortuosity (T). The two-dimensional systems with obstacles are considered. The fluid flow through a porous medium is simulated with the lattice Boltzmann method. The analysis has been performed for the systems with ??(0.37,0.99) which covers five orders of magnitude a span for permeability k?(0.78,2.1×105) and tortuosity T?(1.03,2.74). It is shown that the CNNs can be used to predict the porosity, permeability, and tortuosity with good accuracy. With the usage of the CNN models, the relation between T and ? has been obtained and compared with the empirical estimate.
  

Uncertainties in Deep Learning Systems

• MOZART GRANT (WCA ): Opracowanie metod oceny niepewności w klasyfikacji próbek mikrobiologicznych
  (eng.: The estimate of uncertainties in the classification of microbiological samples)
  Project from 01.10.2019 to 30.09.2020, work done at NeuroSys
• Self-Normalized Density Map (SNDM) for Counting Microbiological Obejcts,
  Krzysztof M. Graczyk, Jarosław Pawlowski, Sylwia Majchrowska, Tomasz Golan,
  Sci Rep 12, 10583 (2022)
  Abstract:
  
• The statistical properties of the density map (DM) approach to counting microbiological objects on images are studied in detail. The DM is given by U 2-Net. Two statistical methods for deep neural networks are utilized: the bootstrap and the Monte Carlo (MC) dropout. The detailed analysis of the uncertainties for the DM predictions leads to a deeper understanding of the DM model’s deficiencies. Based on our investigation, we propose a self-normalization module in the network. The improved network model, called Self-Normalized Density Map (SNDM), can correct its output density map by itself to accurately predict the total number of objects in the image. The SNDM architecture outperforms the original model. Moreover, both statistical frameworks—bootstrap and MC dropout— have consistent statistical results for SNDM, which were not observed in the original model. The SNDM efficiency is comparable with the detector-base models, such as Faster and Cascade R-CNN detectors.
  

Electromagnetic and Weak Structure of the Nucleon Investigated with Bayesian Neural Networks

• Nucleon axial form factor from a Bayesian neural-network analysis of neutrino-scattering data,
  Luis Alvarez-Ruso, Krzysztof M. Graczyk, Eduardo Saul-Sala, Phys. Rev. C99, 025204 (2019)
  Abstract:
• The Bayesian approach for feed-forward neural networks has been applied to the extraction of the nucleon axial form factor from the neutrino-deuteron scattering data measured by the Argonne National Laboratory (ANL) bubble chamber experiment. This framework allows to perform a model-independent determination of the axial form factor from data.. When the low $0.05 < Q^2 < 0.10$ GeV$^2$ data is included in the analysis, the resulting axial radius disagrees with available determinations. Furthermore, a large sensitivity to the corrections from the deuteron structure is obtained. In turn, when the low-$Q^2$ region is not taken into account, with or without deuteron corrections, no significant deviations from the dipole ansatz have been observed. A more accurate determination of the nucleon axial form factor requires new precise measurements of neutrino-induced quasielastic scattering on hydrogen and deuterium.
  



• Zemach moments of proton from Bayesian inference,
  Krzysztof M. Graczyk and C. Juszczak, Phys. Rev. C91, 045205 (2015)
  Abstract:
• The first and the third Zemach moments are obtained, $\langle r \rangle_{(2)}= 1.1108\pm 0.0021 $ fm and $\langle r^3\rangle_{(2)}=2.889 \pm 0.008$ fm$^3$, from the Bayesian analysis of the elastic $ep$ scattering data. The quantitative discussion of the dependence of the results on the parametrization choice is presented and the corresponding systematic uncertainties are estimated -- about 0.6\% and 1.6\% for the first and the third Zemach moments respectively.
  



• Applications of Neural Networks in Hadron Physics,
  Krzysztof M. Graczyk and C. Juszczak, J.Phys. G42 (2015) 3, 034019
  invited contribution to special issue of J.Phys. G: Nucl. Phys., "Enhancing the interaction between nuclear experiment and theory through information and statistics" (ISNET).
  Abstract:
• The Bayesian approach for the feed-forward neural networks is reviewed. Its potential for usage in hadron physics is discussed. As an example of the application the study of the the two-photon exchange effect is presented. We focus on the model comparison, the estimation of the systematic uncertainties due to the choice of the model, and the over-fitting. As an illustration the predictions of the cross sections ratio $d \sigma(e^+ p\to e^+ p)/d \sigma(e^- p\to e^- p)$ are given together with the estimate of the uncertainty due to the parametrization choice.
  



• Proton Radius from Bayesian Inference,
  Krzysztof M. Graczyk and C. Juszczak, Phys. Rev. C90, 054334 (2014).
  Abstract:
• The methods of Bayesian statistics are used to extract the value of the proton radius from the elastic $ep$ scattering data in a model independent way. To achieve that goal a large number of parametrizations (equivalent to neural network schemes) are considered and ranked by their conditional probability $P(\mathrm{parametrization}\,|\,\mathrm{data})$ instead of using the minimal error criterion. As a result the most probable proton radii values ($r_E^p=0.899\pm 0.003$ fm, $r_M^p=0.879\pm 0.007$ fm) are obtained and systematic error due to freedom in the choice of parametrization is estimated. Correcting the data for the two photon exchange effect leads to smaller difference between the extracted values of $r_E^p$ and $r_M^p$. The results disagree with recent muonic atom measurements.
  



• Comparison of Neural Network and Hadronic Model Predictions of Two-Photon Exchange Effect,
  Krzysztof M. Graczyk, Phys. Rev. C88, 065205 (2013)
  Abstract:
• Predictions for the two-photon exchange (TPE) correction to unpolarized $ep$ elastic cross section, obtained within two different approaches, are confronted and discussed in detail. In the first one the TPE correction is extracted from experimental data by applying the Bayesian neural network (BNN) statistical framework. In the other the TPE is given by box diagrams, with the nucleon and the $P_{33}$ resonance as the hadronic intermediate states. Two different form factor parametrizations for both the proton and the $P_{33}$ resonance are taken into consideration. Proton form factors are obtained from the global fit of the full model (with the TPE correction) to the unpolarized cross section data. Predictions of both methods agree well in the intermediate $Q^2$ range, $(1,3)$ GeV$^2$. Above $Q^2=3$ GeV$^2$ the agreement is on $2\sigma$ level. Below $Q^2=1$ GeV$^2$ the consistency between both approaches is broken. The values of the proton radius extracted within both models are given. In both cases predictions for VEPP-3 experiment have been obtained and confronted with the preliminary experimental results.




• Two-Photon Exchange Effect Studied with Neural Networks,
  Krzysztof M. Graczyk, Phys. Rev. C84, 034314 (2011)
  Abstract:
• An approach to the extraction of the two-photon exchange (TPE) correction from elastic ep scattering data is presented. The cross-section, polarization transfer (PT), and charge asymmetry data are considered. It is assumed that the TPE correction to the PT data is negligible. The form factors and TPE correcting term are given by one multidimensional function approximated by the feedforward neural network (NN). To find a model-independent approximation, the Bayesian framework for the NNs is adapted. A large number of different parametrizations is considered. The most optimal model is indicated by the Bayesian algorithm. The obtained fit of the TPE correction behaves linearly in ? but it has a nontrivial Q2 dependence. A strong dependence of the TPE fit on the choice of parametrization is observed.



• The analytical form of the fits fit.pdf and the covariance matrix (order of parameters the same as in fit.pdf )
  Analysis done with:
• GraNet - the feedworward neural network C++ library (will be avialable soon).



• Neural Network Parameterizations of Electromagnetic Nucleon Form Factors,
  Krzysztof M. Graczyk, Piotr Płoński, Robert Sulej, JHEP (2010) 053
  Abstract:
• The electromagnetic nucleon form-factors data are studied with artificial feed forward neural networks. As a result the unbiased model-independent form-factor parametrizations are evaluated together with uncertainties. The Bayesian approach for the neural networks is adapted for chi2 error-like function and applied to the data analysis. The sequence of the feed forward neural networks with one hidden layer of units is considered. The given neural network represents a particular form-factor parametrization. The so-called evidence (the measure of how much the data favor given form factor model) is computed with the Bayesian framework and it is used to determine the best form factor parametrization.

Analysis is done with:
• GraNet - the feedworward neural network C++ library (will be avialable soon).
• NetMaker (written in C#) by Robert Sulej and Piotr Płoński.