Upgrade to Pro — share decks privately, control downloads, hide ads and more …

PFN Internship 2023 / Hagai Masaya: Towards Neural Network Potential for Excited states

PFN Internship 2023 / Hagai Masaya: Towards Neural Network Potential for Excited states

Towards Neural Network Potential for Excited states. PFN Internship 2023 by Hagai Masaya

Preferred Networks
PRO

October 13, 2023
Tweet

More Decks by Preferred Networks

Other Decks in Technology

Transcript

  1. Towards Neural Network
    Potential for Excited states
    Hagai Masaya
    @ Preferred Networks Summer Internship 2023

    View Slide

  2. Light - related phenomena
    2
    Photosynthesis[1] Human vision[2]
    Photodamage[3]
    [1] https://en.wikipedia.org/wiki/Photosynthesis
    [2] http://light.physics.auth.gr/enc/vision_en.html
    [3] Facial plastic surgery : FPS 25 5 (2009): 337-46 .

    View Slide

  3. Light - related devices
    Solar cell[1] OLED[2] Bioimaging[3]
    3
    [1] https://www.science.org/content/article/amp-solar-cells-scientists-ditch-silicon
    [2] https://en.wikipedia.org/wiki/OLED
    [3] https://www.k-ishiilab.iis.u-tokyo.ac.jp/research/theme3/3-1_en.html

    View Slide

  4. Theory of absorption and emission
    4
    S
    0
    (Ground State)
    S
    1
    (Excited State)
    S
    2
    (Excited State)
    Energy
    Nuclear Coordinates


    1. Absorption
    2. Structural relaxation in S
    1
    3. Emission
    4. Structural relaxation in S
    0




    Radiative process
    Nonradiative process
    ● Internal conversion
    ● Intersystem crossing
    ● Photoreaction

    View Slide

  5. Problems in traditional ways of excited state calc.
    5
    Calculation time of excited state[1]
    Conventional methods v.s. NNP
    Calculation scaling against
    number of atoms in molecule[2]
    Conventional method takes much time, O(N4)~O(N!)
    (N is molecule size)
    Large number of molecules for new photo-related materials
    Big molecule such as protein and molecular complex
    [1] Chem. Rev. 121, 9873–9926 (2021). [2] http://www.chem.waseda.ac.jp/nakai/?page_id=1291

    View Slide

  6. Conventional neural network potential (NNP)
    6
    Input:
    Molecules
    Atomic numbers + Coordinates
    Z Rx Ry Rz

    Output : DFT’s property
    (Density Functional Theory)
    - Energy
    - Gradient
    - Dipole moment
    S
    0
    Ground State
    Neural Network[1]
    [1] Chem. Sci. 8, 3192–3203 (2017).

    View Slide

  7. Objective of this study : NNP for excited states
    7
    7
    Input:
    Molecules
    Atomic numbers + Coordinates
    Z Rx Ry Rz

    Output:
    DFT’s & TDDFT’s property
    Neural Network[1]
    - Energy
    - Gradient
    - Dipole moment
    S
    0
    Ground State
    - Energy
    - Gradient
    - Dipole moment
    - Transition Dipole
    S
    1
    ,S
    2
    Excited State
    [1] ANI-1 : Chem. Sci. 8, 3192–3203 (2017).

    View Slide

  8. Extensive property and Intensive property
    8
    Methane(n=1) Ethane (n=2) Hexane (n=6)
    (S
    0
    opt. / S
    1
    opt.)
    C
    n
    H
    2n+
    2
    Ground state energy [eV]
    Excitation energy [eV]
    -1100
    12.86
    -2168
    11.12
    -6440 / -6439
    10.06 / 7.09
    ~2 ~3
    Ground state energy is extensive, Excitation energy is intensive
    O(N) O(1)

    View Slide

  9. No size consistency for excited state
    9
    Excited state energy = Ground state energy + Excitation energy
    extensive property intensive property
    System A System B
    ΔEe=12.9 eV ΔEe=10.1 eV
    System A+B
    ΔEe=10.1 eV
    1000Å

    View Slide

  10. Overview of this study
    10
    Dataset (Made by myself)
    ● n=1~6 Alkane (n_sample=4,000)
    ● QM5 (n_sample=57,000)
    NNP model
    ● SchNet
    ● M3GNet
    ➔ Comfirm better performance
    M3GNet compared to SchNet
    Extrapolaion
    ● Heptane (n=7)
    ● Octane (n=8)
    Readout layer
    ● Output E(S1) directry
    ○ Sum model
    ● Output ΔE(S1) as E(S1)=E(S0)+ΔE(S1)
    ○ Softmin model
    ○ Softmin + self-attention model (Didn’t improve)
    Loss
    1. E(S1) loss
    2. ΔE(S1) loss

    View Slide

  11. Dataset preparation
    11
    S
    0
    (Ground State)
    S
    1
    (Excited State)
    S
    2
    Energy
    Nuclear Coordinates
    1. Find equilibrium structure of
    S
    0
    and S
    1
    (  )
    2. Sampling structures around
    each equilibrium structure
    (Wigner sampling[1]
    (≒Normal mode sampling))
    [1] Wigner sampling : M. Pinheiro Jr, S. Zhang, P. O. Dral, M. Barbatti, Scientific Data. 10, 1–11 (2023).

    View Slide

  12. Alkane dataset / QM5 dataset
    12
    Alkane dataset (total 4,000 data)
    C
    n
    H
    2n+2
    (n=1,2,3,4,5,6)
    ● For n=1,2,3,4, sampling 500 structure from S0 opt. structure
    ● For n=5,6, sampling 500 structures from each of the S0 and S1 opt. structures
    ● TDDFT PBE0/6-31G(d) using Gaussian 16
    QM5 (total 57,000 data)
    Max. 5 heavy atoms (C,N,O,F)
    ● 177 S0 opt. structures and 108 S1 opt. structures
    ● Sampling 200 structures from each optimized structre
    Heptane(n=7) and Octane(n=8) can be predicted from these data?
    Made TDDFT dataset for NNP by myself in this internship

    View Slide

  13. M3GNet NNP model
    13
    M3GNnet: C. Chen, S. P. Ong, Nature Computational Science. 2, 718–728 (2022).
    Hyper parameter
    ● N block = 3
    ● cutoff = 5.0Å
    ● 3body cutoff = 4.0Å
    ● node / edge embedding=64dim
    Readout layer

    View Slide

  14. Readout layer
    14
    Node feature of atom i
    1. Sum model
    2. Softmin model
    3. Softmin + SelfAttention
    Ground state energy
    Excited state energy

    View Slide

  15. Test of ΔE(S1-S0) with Sum / Softmin readout
    15
    Sum readout (MAE=140meV) Softmin readout (MAE=66meV)
    ❏ Softmin readout shows smaller test error

    View Slide

  16. Sum readout model (extrapolation)
    Heptane (n=7) Octane (n=8)
    ❏ Large MAE compared to TDDFT error (0.24eV)
    ❏ MAE error scales with the size of molecule

    View Slide

  17. Softmin readout model (extrapolation)
    17
    Heptane (n=7) Octane (n=8)
    ❏ Small MAE (0.11~0.33 eV)
    ❏ MAE errors don’t scale with the size of molecule

    View Slide

  18. ΔE(S1) loss & Softmin readout
    18
    Heptane (n=7) Octane (n=8)
    Test MAE = 18.6 meV
    ❏ Test dataset MAE (18.6meV) using ΔE(S1) loss is better than
    one (66meV) using E(S1) loss

    View Slide

  19. App 1. Size consistency problem
    19
    A = Methane, B = Methane
    ΔE(S1) in A+B ΔE(S2) in A+B
    A = Methane, B = Pentane
    (Softmin readout)
    ❏ Failed to reproduce ΔE(S1)
    ❏ between ΔE(S1)(Methane) and
    ΔE(S1)(Pentane)
    ❏ Success to reproduce ΔE(S1)
    ❏ Failed to reproduce ΔE(S2)
    ❏ (Must be same as ΔE(S1))

    View Slide

  20. App 2. Geometry optimization of Excited State
    20
    Geometry optimization of S1 takes long time
    Optimization using NNP for excited state
    C-C-C angle vary greatly
    (S0 opt. 113.5°, S1 opt. 96.5°)
    Hexane S1 opt. using NNP from S0 opt. init structure
    C-C-C angle of NNP S1 opt. is 97.8°
    Fail hydrogen position
    ● RMSD between initial and S1(TDDFT) = 0.31Å
    ● RMSD between S1(NNP) and S1(TDDFT) = 0.19Å

    View Slide

  21. Test of QM5 result (Softmin readout model)
    21
    S0/S1 energy (MAE=54/58meV) S1 Excitation energy (MAE=22meV)
    ΔE(S1) Loss

    View Slide

  22. QM5 & [QM5 + hexane] extrapolation
    22
    Heptane
    Heptane Octane Octane
    QM5 dataset QM5+Hexane dataset
    ❏ QM5 only, octane’s predictions are poor (MAE=0.47/0.75eV)
    ❏ By adding hexane, extrapolability improved (MAE=0.25/0.14eV)

    View Slide

  23. TDDFT error & Summary
    23
    TDDFT/TZVP MSE [eV] MAE [eV]
    PBE0 -0.05 0.24
    [1] A. D. Laurent, D. Jacquemin, Int. J. Quantum Chem. 113, 2019–2039 (2013).
    TDDFT vertical transition energy error[1](Ref. CASPT2/TZVP)
    Test dataset & Heptane & Octane ΔE(S1) MAE
    Type of Readout layer
    Type of Loss
    Sum readout
    E(S1) Loss
    Softmin
    E(S1) Loss
    Softmin
    ΔE(S1) Loss
    Softmin,ΔE(S1)
    QM5 data
    Softmin, ΔE(S1)
    QM5 + Hexane
    Test dataset MAE [meV] 140 66 19 22 Not yet
    Heptane/Octane MAE[eV] 0.73/1.40 0.33/0.32 0.40/0.19 0.47/0.75 0.25/0.14
    ❏ Achieve errors comparable to TDDFT's own errors

    View Slide

  24. Conclusion
    24
    Conclusion
    ● Made TDDFT dataset by myself
    ● Designing readout layer and loss function to reflect excited
    state property improve performance
    ● Our best excited state NNP shows errors comparable to
    TDDFT’s own error (0.24eV)
    ● Calculation of Octane S1 energy and force using
    NNP takes only 0.26s (GPU), TDDFT 10s (32cpu)

    View Slide