michael-herbst.comhttps://michael-herbst.com/2022-10-12T00:30:00+02:00Research and projectsNumerical stability and efficiency of response property calculations in density functional theory2022-10-12T00:30:00+02:002022-10-12T00:30:00+02:00Michael F. Herbsttag:michael-herbst.com,2022-10-12:/2022-schur-response.html<p>The goal of quantum-chemical calculations is the simulation
of materials and molecules. In density-functional theory (DFT) the first step
along this line is obtaining the electron density minimising an energy functional.
However, since energies and the density are usually not very tractable quantities
in an experimental setup, comparison to experiment …</p><p>The goal of quantum-chemical calculations is the simulation
of materials and molecules. In density-functional theory (DFT) the first step
along this line is obtaining the electron density minimising an energy functional.
However, since energies and the density are usually not very tractable quantities
in an experimental setup, comparison to experiment and scientific intuition
also requires the computation of properties.
Important properties include the forces (i.e. the energetic change due to a displacement
of the structure) polarisabilities (change in dipole moment due to an external electric field)
or phonon spectra (which can be measured using infrared spectroscopy).
Therefore an efficient and reliable property computation is crucial to make quantum-chemical
simulations interpretable and to close the loop back to experimentalists.</p>
<p>In DFT property calculations are done using density-functional perturbation theory (DFPT),
which essentially computes the linear response of the electronic structure to
the aforementioned changes in external conditions (external field, nuclear displacements etc.).
Solving the equations underlying DFPT can become numerically challenging as
(especially for metallic systems) the equations are ill-conditioned.</p>
<p>In a collaboration with my former PostDoc advisor Benjamin Stamm
and my old group at the CERMICS at École des Ponts,
including Eric Cancès, Antoine Levitt, Gaspard Kemlin,
we just published an article, where we provide a more mathematical take on DFPT.
In our work we provide an extensive review of various practical setups employed in main-stream codes
such as <a href="https://www.abinit.org">ABINIT</a> and <a href="https://quantum-espresso.org">QuantumEspresso</a>
from a numerical analysis point of view, highlighting the differences and similarities
of these approaches.
Moreover we develop a novel approach approach to solve the so-called Sternheimer equations
(a key component of DFPT), which allows to make better use of the byproducts
available in standard SCF schemes (the algorithm used to obtain the DFT ground state).
With our approach we show savings up to 40% in the number of matrix-vector products required
to solve the response equations. Since these are the most expensive step in DFPT this
implies a similar saving in computational cost overall.
Naturally our algorithm has been implemented as the default
response solver in our <a href="https://dftk.org">DFTK</a> code, starting from version 0.5.9.</p>
<p>Most of this work was done during a two-month visit of Gaspard Kemlin with Benjamin and myself
here in Aachen. I think I speak for the both of us when I say that it has been a great
pleasure to have Gaspard around, both on a professional as well as a personal level.</p>
<p>The full abstract of the paper reads</p>
<blockquote>
<p>Response calculations in density functional theory aim at computing the
change in ground-state density induced by an external perturbation. At finite
temperature these are usually performed by computing variations of orbitals,
which involve the iterative solution of potentially badly-conditioned linear
systems, the Sternheimer equations. Since many sets of variations of orbitals
yield the same variation of density matrix this involves a choice of gauge.
Taking a numerical analysis point of view we present the various gauge
choices proposed in the literature in a common framework and study their
stability. Beyond existing methods we propose a new approach, based on a
Schur complement using extra orbitals from the self-consistent-field
calculations, to improve the stability and efficiency of the iterative
solution of Sternheimer equations. We show the success of this strategy on
nontrivial examples of practical interest, such as Heusler transition metal
alloy compounds, where savings of around 40% in the number of required
cost-determining Hamiltonian applications have been achieved.</p>
</blockquote>CECAM flagship workshop: Error control in first-principles modelling2022-07-16T12:00:00+02:002022-07-16T12:00:00+02:00Michael F. Herbsttag:michael-herbst.com,2022-07-16:/2022-cecam-error-control.html<p><em>(Cross-post from our report published in the <a href="https://psi-k.net/workshop-error-control-in-first-principles-modelling">Psi-k blog</a>)</em></p>
<p>From 20th until 24th June 2022 I co-organised a workshop on the theme of
<a href="https://www.cecam.org/workshop-details/1115">Error control in first-principles modelling</a>
at the CECAM Headquarters in Lausanne
(<a href="https://www.cecam.org/workshop-details/1115">workshop website</a>).
For one week the workshop unified like-minded researchers from a range of communities,
including …</p><p><em>(Cross-post from our report published in the <a href="https://psi-k.net/workshop-error-control-in-first-principles-modelling">Psi-k blog</a>)</em></p>
<p>From 20th until 24th June 2022 I co-organised a workshop on the theme of
<a href="https://www.cecam.org/workshop-details/1115">Error control in first-principles modelling</a>
at the CECAM Headquarters in Lausanne
(<a href="https://www.cecam.org/workshop-details/1115">workshop website</a>).
For one week the workshop unified like-minded researchers from a range of communities,
including quantum chemistry, materials sciences, scientific computing and
mathematics to jointly discuss the determination of errors in atomistic
modelling. The main goal was to obtain a cross-community overview of ongoing
work and to establish new links between the disciplines.</p>
<p>Amongst others we discussed topics such as: the determination of errors in
observables, which are the result of long molecular dynamics simulations, the
reliability and efficiency of numerical procedures and how to go beyond
benchmarking or convergence studies via a rigorous mathematical understanding of errors.
We further explored interactions with the field of uncertainty quantification to link
numerical and modelling errors in electronic structure calculations or to
understand error propagation in interatomic potentials via statistical
inference.</p>
<h2>Organisers</h2>
<ul>
<li><a href="http://www.eng.cam.ac.uk/profiles/gc121">Gabor Csanyi</a> (University of Cambridge)</li>
<li><a href="https://gdusson.perso.math.cnrs.fr/">Genevieve Dusson</a> (CNRS & Université Bourgogne Franche-Comté)</li>
<li><a href="https://michael-herbst.com">Michael Herbst</a> (RWTH Aachen University)</li>
<li><a href="https://uqgroup.mit.edu/">Youssef Marzouk</a> (Massachusetts Institute of Technology)</li>
</ul>
<h2>Participants</h2>
<div class="centered">
<img width=400px src="https://michael-herbst.com/images/2022.06.23_CECAM_error_group_picture.jpg" alt="group picture"/>
</div>
<p>A primary objective of the conference was to facilitate networking and exchange
across communities. Thanks to the funds provided by CECAM and Psi-k we managed to
get a crowd of 30 researchers, including about 15 junior researchers, to come
to Lausanne in person. Moreover we made an effort to enable a virtual
participation to the smoothest extent possible. For example we provided
a conference-specific Slack space, which grew into a platform
for discussion involving both in-person as well as virtual participants
during the conference.
In this way in total about 70 researchers from 18 countries could participate in
the workshop. The full list of participants is available
on the <a href="https://www.cecam.org/workshop-details/1115">workshop website</a>.</p>
<h2>Workshop programme</h2>
<p>The workshop programme was split between the afternoon sessions, in which we
had introductory and topic-specific lectures, as well as the morning sessions,
which were focussed on informal discussion and community brainstorming.</p>
<h3>Afternoon lectures</h3>
<h5>Monday June 20th 2022</h5>
<ul>
<li><em>Uncertainty quantification for atomic-scale machine learning.</em> (Michele Ceriotti, EPFL)<br>
<a href="https://michael-herbst.com/2022-cecam-error-control/2022.06.20_1_Michele_Ceriotti.pdf">[slides]</a>
<a href="https://michael-herbst.com/2022-cecam-error-control/2022.06.20_1_Michele_Ceriotti.mp4">[recording]</a></li>
<li><em>Testing the hell out of DFT codes with virtual oxides.</em> (Stefaan Cottenier, Ghent University)<br>
<a href="https://michael-herbst.com/2022-cecam-error-control/2022.06.20_2_Stefaan_Cottenier.pdf">[slides]</a>
<a href="https://michael-herbst.com/2022-cecam-error-control/2022.06.20_2_Stefaan_Cottenier.mp4">[recording]</a></li>
<li><em>Prediction uncertainty validation for computational chemists.</em> (Pascal Pernot, Université Paris-Saclay)<br>
<a href="https://michael-herbst.com/2022-cecam-error-control/2022.06.20_3_Pascal_Pernot.pdf">[slides]</a>
<a href="https://michael-herbst.com/2022-cecam-error-control/2022.06.20_3_Pascal_Pernot.mp4">[recording]</a></li>
<li><em>Uncertainty driven active learning of interatomic potentials for molecular dynamics</em> (Boris Kozinsky, Harvard University)<br>
<a href="https://michael-herbst.com/2022-cecam-error-control/2022.06.20_4_Boris_Kozinski.mp4">[recording]</a></li>
<li><em>Interatomic Potentials from First Principles</em> (Christoph Ortner, University of British Columbia)<br>
<a href="https://michael-herbst.com/2022-cecam-error-control/2022.06.20_5_Christoph_Ortner.pdf">[slides]</a>
<a href="https://michael-herbst.com/2022-cecam-error-control/2022.06.20_5_Christoph_Ortner.mp4">[recording]</a></li>
</ul>
<h5>Tuesday June 21st 2022</h5>
<ul>
<li><em>Numerical integration in the Brillouin zone</em> (Antoine Levitt, Inria Paris)<br>
<a href="https://michael-herbst.com/2022-cecam-error-control/2022.06.21_1_Antoine_Levitt.pdf">[slides]</a>
<a href="https://michael-herbst.com/2022-cecam-error-control/2022.06.21_1_Antoine_Levitt.mp4">[recording]</a></li>
<li><em>Sensitivity analysis for assessing and controlling errors in theoretical spectroscopy and computational biochemistry</em> (Christoph Jacob, <br>
TU Braunschweig)<br>
<a href="https://michael-herbst.com/2022-cecam-error-control/2022.06.21_2_Christoph_Jacob.pdf">[slides]</a></li>
<li><em>Uncertainty quantification and propagation in multiscale materials modelling</em> (James Kermode, University of Warwick)<br>
<a href="https://michael-herbst.com/2022-cecam-error-control/2022.06.21_3_James_Kermode.pdf">[slides]</a>
<a href="https://michael-herbst.com/2022-cecam-error-control/2022.06.21_3_James_Kermode.mp4">[recording]</a></li>
<li><em>Uncertainty Quantification and Active Learning in Atomistic Computations</em><br>
(Habib Najm, Sandia National Labs)</li>
<li><em>Nuances in Bayesian estimation and active learning for data-driven interatomic potentials for propagation of uncertainty through molecular dynamics</em><br>
(Dallas Foster, MIT)<br>
<a href="https://michael-herbst.com/2022-cecam-error-control/2022.06.21_5_Dallas_Foster.pdf">[slides]</a>
<a href="https://michael-herbst.com/2022-cecam-error-control/2022.06.21_5_Dallas_Foster.mp4">[recording]</a></li>
</ul>
<h5>Wednesday June 22nd 2022</h5>
<ul>
<li><em>The BEEF class of xc functionals</em> (Thomas Bligaard, DTU)<br>
<a href="https://michael-herbst.com/2022-cecam-error-control/2022.06.22_1_Thomas_Bligaard.mp4">[recording]</a></li>
<li><em>A Bayesian Approach to Uncertainty Quantification for Density Functional Theory</em> (Kate Fisher, MIT)<br>
<a href="https://michael-herbst.com/2022-cecam-error-control/2022.06.22_2_Kate_Fisher.pdf">[slides]</a>
<a href="https://michael-herbst.com/2022-cecam-error-control/2022.06.22_2_Kate_Fisher.mp4">[recording]</a></li>
<li><em>Dielectric response with short-ranged electrostatics</em> (Stephen Cox, Cambridge)<br>
<a href="https://michael-herbst.com/2022-cecam-error-control/2022.06.22_3_Stephen_Cox.pdf">[slides]</a></li>
<li><em>Fully guaranteed and computable error bounds for clusters of eigenvalues</em> (Genevieve Dusson, CNRS)<br>
<a href="https://michael-herbst.com/2022-cecam-error-control/2022.06.22_4_Genvieve_Dusson.pdf">[slides]</a>
<a href="https://michael-herbst.com/2022-cecam-error-control/2022.06.22_4_Genvieve_Dusson.mp4">[recording]</a></li>
<li><em>Practical error bounds for properties in plane-wave electronic structure calculations</em> (Gaspard Kemlin, Ecole des Ponts)<br>
<a href="https://michael-herbst.com/2022-cecam-error-control/2022.06.22_5_Gaspard_Kemlin.pdf">[slides]</a>
<a href="https://michael-herbst.com/2022-cecam-error-control/2022.06.22_5_Gaspard_Kemlin.mp4">[recording]</a></li>
<li><em>The transferability limits of static benchmarks</em> (Thomas Weymuth, ETH)<br>
<a href="https://michael-herbst.com/2022-cecam-error-control/2022.06.22_6_Thomas_Weymuth.pdf">[slides]</a>
<a href="https://michael-herbst.com/2022-cecam-error-control/2022.06.22_6_Thomas_Weymuth.mp4">[recording]</a></li>
</ul>
<h5>Thursday June 23rd 2022</h5>
<ul>
<li><em>An information-theoretic approach to uncertainty quantification in atomistic modelling of crystalline materials</em> (Maciej Buze, Birmingham)<br>
<a href="https://michael-herbst.com/2022-cecam-error-control/2022.06.23_1_Maciej_Buze.pdf">[slides]</a>
<a href="https://michael-herbst.com/2022-cecam-error-control/2022.06.23_1_Maciej_Buze.mp4">[recording]</a></li>
<li><em>Hyperactive Learning</em> (Cas van der Oord, Cambridge)<br>
<a href="https://michael-herbst.com/2022-cecam-error-control/2022.06.23_2_Cas_van_der_Oord.pdf">[slides]</a>
<a href="https://michael-herbst.com/2022-cecam-error-control/2022.06.23_2_Cas_Van_der_Oord.mp4">[recording]</a></li>
<li><em>Benchmarking under uncertainty</em> (Jonny Proppe, TU Braunschweig)</li>
<li><em>Model Error Estimation and Uncertainty Quantification of Machine Learning Interatomic Potentials</em> (Khachik Sargsyan, Sandia National Labs)<br>
<a href="https://michael-herbst.com/2022-cecam-error-control/2022.06.23_4_Khachik_Sargsyan.pdf">[slides]</a>
<a href="https://michael-herbst.com/2022-cecam-error-control/2022.06.23_4_Khachik_Sargsyan.mp4">[recording]</a></li>
<li><em>Committee neural network potentials control generalization errors and enable active learning</em> (Christoph Schran, Cambridge)<br>
<a href="https://michael-herbst.com/2022-cecam-error-control/2022.06.23_5_Christoph_Schran.pdf">[slides]</a>
<a href="https://michael-herbst.com/2022-cecam-error-control/2022.06.23_5_Christoph_Schran.mp4">[recording]</a></li>
</ul>
<h3>Morning discussion sessions</h3>
<p>The discussion sessions were centred around broad multi-disciplinary topics to
stimulate cross-fertilisation. Key topics were active learning techniques for
obtaining interatomic potentials on the fly as well as opportunities to connect
numerical and statistical approaches for error estimation.</p>
<p>A central topic of the session on Thursday morning was the development of a
common cross-community language and guidelines for error estimation. This
included the question how to establish a minimal standard for error control and
make the broader community aware of such techniques to ensure published results
can be validated and are more reproducible. Initial ideas from this discussion are
summarised in a <a href="https://github.com/mfherbst/error-atomistic-simulations">public github repository</a>.
With this repository we invite everyone to contribute concrete examples of the
error control strategies taken in their research context. In the future we hope
to community guidelines for error control in first-principle modelling based on
these initial ideas.</p>
<h2>Feedback from participants</h2>
<p>Overall we received mostly positive feedback about the event. Virtual
participants enjoyed the opportunity to interact with in-person
participants via the zoom sessions and Slack. For several in-person
participants this meeting was the first physical meeting since the pandemic and
the ample opportunities for informal interchange we allocated in the
programme (discussion sessions, poster sessions, social dinner, boat trip
excursion) have been much appreciated.</p>
<p>A challenge was to keep the meeting accessible for both researchers from
foreign fields as well as junior participants entering this interdisciplinary
field. With respect to the discussion sessions we got several suggestions for
improvement in this regard. For example it has been suggested to (i) set and
communicate the discussion subject well in advance to allow people to get
prepared, (ii) motivate postdocs to coordinate the discussion, which would be
responsible to curate material and formulate stimulating research questions and
(iii) get these postdocs to start the session with an introductory presentation
on open problems.</p>
<h2>Conclusions and outlook</h2>
<p>During the event it became apparent that the meaning associated to the term
“error control” deviates between communities, in particular between
mathematicians and application scientists. Not only did this result in a
considerable language barrier and some communication problems during the
workshop, but it also made communities to appear to move at different paces. On
a first look this sometimes made it difficult to see the applicability of
research results from another community.
But the heterogeneity of participants also offered opportunities to learn from
each other's viewpoint: for example during the discussion sessions we actively
worked towards obtaining a joint language and cross-community standards for
error control. Our initial ideas on this point are
available in a <a href="https://github.com/mfherbst/error-atomistic-simulations">public github repository</a>,
where we invite everyone to participate via opening issues and pull requests to continue
the discussion.</p>RWTH Julia workshop 20222022-02-20T12:00:00+01:002022-02-20T12:00:00+01:00Michael F. Herbsttag:michael-herbst.com,2022-02-20:/2022-rwth-julia-workshop.html<p>Last Thursday and Friday (17/18 February)
I taught an <a href="https://michael-herbst.com/teaching/2022-rwth-julia-workshop/">introductory course to the Julia programming language</a>.
The course took place in virtual format and to my great surprise around 90 people
from all over the world ended up joining. Luckily I had a small support team
consisting of Gaspard …</p><p>Last Thursday and Friday (17/18 February)
I taught an <a href="https://michael-herbst.com/teaching/2022-rwth-julia-workshop/">introductory course to the Julia programming language</a>.
The course took place in virtual format and to my great surprise around 90 people
from all over the world ended up joining. Luckily I had a small support team
consisting of Gaspard Kemlin and Lambert Theissen (thanks!) who took care of some
of the organisational aspects in running the zoom session. Overall it was a lot of fun to
spread the word about the Julia programming language with so many curious listeners
with interested and supporting questions.</p>
<p>Thanks to everyone who tuned in
and thanks to everyone who gave constructive feedback at the end. I'm very much encouraged
by the fact that all of you, unanimously, would recommend the workshop to your peers.
In that sense: Please go spread the word as I'm already looking forward to the next occasion
I'll have to teach about Julia!</p>GdR nbody general meeting2022-01-25T12:00:00+01:002022-01-25T12:00:00+01:00Michael F. Herbsttag:michael-herbst.com,2022-01-25:/2022-gdr-nbody-toulouse.html<p>About two weeks ago, from 10 till 13 Jan 2022 I was at the <a href="https://lcpq.github.io/gdr_nbody_2021">annual meeting of the French research group on many-body phaenomena</a>,
the GDR nbody. Originally scheduled to take place in person in Toulouse
the Corona-related developments unfortunately caused the organisers to switch to a
virtual event on …</p><p>About two weeks ago, from 10 till 13 Jan 2022 I was at the <a href="https://lcpq.github.io/gdr_nbody_2021">annual meeting of the French research group on many-body phaenomena</a>,
the GDR nbody. Originally scheduled to take place in person in Toulouse
the Corona-related developments unfortunately caused the organisers to switch to a
virtual event on short notice. Albeit I would have loved to return to Toulouse
and see everyone in person, it was still an opportunity to catch up.
In my talk at the occasion I presented on the
<a href="adaptive damping procedure">{filename}/articles/Publications/2021-adaptive-damping.md</a>,
which Antoine Levitt and myself recently developed,
see the <a href="https://arxiv.org/abs/2109.14018">submitted article on arxiv</a>.</p>
<table>
<thead>
<tr>
<th>Link</th>
</tr>
</thead>
<tbody>
<tr>
<td><a href="https://michael-herbst.com/talks/2022.01.13_gdrnbody_linesearch.pdf">A robust and efficient line search for self-consistent field iterations</a> (Slides)</td>
</tr>
</tbody>
</table>Outlook to 20222021-12-23T12:00:00+01:002021-12-23T12:00:00+01:00Michael F. Herbsttag:michael-herbst.com,2021-12-23:/2021-workshops-outlook.html<p>A quick teaser to some workshops I will organise next year.</p>
<ul>
<li>
<p><strong>17/18 Feb 2022: Introduction to the Julia programming language</strong> (virtual).<br>
In two half-day sessions I will provide a concise overview of the Julia programming
language and offer to get some hands-on practice.
The selection of exercises and small …</p></li></ul><p>A quick teaser to some workshops I will organise next year.</p>
<ul>
<li>
<p><strong>17/18 Feb 2022: Introduction to the Julia programming language</strong> (virtual).<br>
In two half-day sessions I will provide a concise overview of the Julia programming
language and offer to get some hands-on practice.
The selection of exercises and small projects makes the course
particularly well-suited for interdisciplinary researchers in the computational sciences,
but is free and open to everyone.
<a href="https://michael-herbst.com/teaching/2022-rwth-julia-workshop/">Course website.</a>
<a href="https://terminplaner4.dfn.de/D9vWdxIpaLTS3ukk">Registration link.</a></p>
</li>
<li>
<p><strong>20-24 Jun 2022: CECAM workshop: Error control in first-principles modelling</strong>
(Lausanne, Switzerland).<br>
In this workshop, which I organise jointly
with Gábor Csányi, Geneviève Dusson, Youssef Marzouk,
we plan to bring together mathematicians and simulation scientists
to discuss error control and error estimation
in first-principles simulations,
an aspect which to date has seen too little attention in our opinion.
We want to bring together experts on numerical analysis and uncertainty quantification
on the one hand and researchers working on electronic-structure and
molecular-dynamics methods on the other to identify promising
directions of research to make progress in this topic.
<a href="https://www.cecam.org/workshop-details/1115">Website and registration.</a></p>
</li>
<li>
<p><strong>29-31 Aug 2022: DFTK school: Numerical methods for density-functional-theory simulations</strong>
(Paris, France).<br>
Antoine Levitt and Eric Cancès and myself will organise an interdisciplinary
summer school next year, centred around our joint work on DFTK and numerical
developments in density-functional theory (DFT). With the school we want to bridge
the divide between simulation practice and fundamental research in
electronic-structure methods: It is
is intended <em>both</em> for researchers with a background in mathematics
and computer science interested to learn the numerics of DFT and physicists or chemists
interested in modern software development methodologies and the mathematical
background of DFT.
<a href="https://school2022.dftk.org">Course website.</a>
<a href="https://school2022.dftk.org/registration">Registration link.</a></p>
</li>
</ul>GdR REST Discussion meeting on Machine Learning2021-12-13T12:00:00+01:002021-12-13T12:00:00+01:00Michael F. Herbsttag:michael-herbst.com,2021-12-13:/2021-gdr-rest-ml.html<p>Last week on 9th and 10th December 2021 I participated in the
<a href="http://gdr-rest.polytechnique.fr/Discussion_Meeting_Machine_Learning">Discussion meeting on Machine Learning</a>
been organised by the <a href="http://gdr-rest.polytechnique.fr">French research group REST</a>,
which is centred around theoretical spectroscopy in solids and molecules.
While most participants joined remotely I was fortunately able to
travel to École Polytechnique in …</p><p>Last week on 9th and 10th December 2021 I participated in the
<a href="http://gdr-rest.polytechnique.fr/Discussion_Meeting_Machine_Learning">Discussion meeting on Machine Learning</a>
been organised by the <a href="http://gdr-rest.polytechnique.fr">French research group REST</a>,
which is centred around theoretical spectroscopy in solids and molecules.
While most participants joined remotely I was fortunately able to
travel to École Polytechnique in Palaiseau (near Paris).
This gave me the opportunity to interact with some of the speakers and local organisers.
Since to date I have not yet taken a detailed look at applying machine learning in
chemistry and materials science I took the chance to
discuss with both practitioners as well as the other on-site
speakers during the breaks and the social dinner.
Overall this meeting has been extremely helpful and I feel I managed to get a good
impression of the challenges and current research in this exciting topic. I am very
grateful to the organisers Francesco Sottile and Jack Wetherell for the invitation
and I already look forward to my next interaction with the GdR REST.</p>
<p>In my talk I gave an <strong>introduction to algorithmic differentiation (AD) approaches</strong>
and their <strong>application in DFTK</strong> as well as density-functional theory simulations in general.
I motivated our work both from data-driven approaches for the design of novel
DFT functionals as well as the computation of properties, sensitivities
and uncertainties. Summarised in one sentence the key advantage
of getting a code algorithmically differentiable (AD-able) is to be able to <strong>automatically
compute derivatives</strong> of <em>arbitrary</em> output quantities (band gaps, forces, ...) with respect
to <em>arbitrary</em> input quantities (pseudo parameters, XC parameters, positions, temperature, ...)
within an acceptable computational cost and without the need to code analytical gradients.</p>
<p>AD approaches are not new in the electronic-structure context.
However, the successful existing AD-able
codes are either centred around simplified settings (e.g. 1D systems)
or Gaussian basis sets (thus primarily molecular systems).
In contrast our focus in DFTK are solid-state systems.
In particular for cases with vanishing
band gaps (e.g. metals) this setting is more involved and one needs to be overall a bit
more careful in the implementation. Another distinction from previous efforts
is that our implementation in DFTK has not been written from scratch just for AD.
Effectively the ability to make DFTK AD-able with relatively little effort
is a side effect from our flexible design as well as our seamless integration
with the composable Julia package ecosystem.
To emphasise this let me mention that the largest part of the work I presented upon
has been achieved in only 12 weeks by our excellent
Google Summer of Code student Niklas Schmitz (Thanks very much Niklas!).</p>
<p>To give a practical demonstration I showed how to use forward-mode algorithmic
differentiation to (a) compute polarisabilities,
(b) the variation of the dipole moment with respect to changing parameters in the exchange
functional and (c) a work-in-progress example using adjoint-mode differentiation.
As usual my slides are attached below.</p>
<table>
<thead>
<tr>
<th>Link</th>
</tr>
</thead>
<tbody>
<tr>
<td><a href="https://michael-herbst.com/talks/2021.12.10_gdrrest_ml.pdf">DFTK: An algorithmically differentiable density-functional theory framework</a> (Slides)</td>
</tr>
<tr>
<td><a href="https://www.youtube.com/watch?v=35rIzLUjqII">Recording on Youtube</a></td>
</tr>
<tr>
<td><a href="https://docs.dftk.org/v0.4.2/examples/forwarddiff/">Forward-mode algorithmic differentiation example</a></td>
</tr>
</tbody>
</table>Surrogate models for quantum spin systems based on reduced order modeling2021-11-04T00:30:00+01:002021-11-04T00:30:00+01:00Michael F. Herbsttag:michael-herbst.com,2021-11-04:/2021-quantum-spin-reduced-basis.html<p>The simulation of quantum spin models is an actively researched field.
Albeit rather basic these many-body systems are inherently strongly
correlated and as such feature a rich variety of
phaenomena including involved patterns of ordering / discordering,
topological order or varieties of phase changes.
Furthermore these models often provide a good …</p><p>The simulation of quantum spin models is an actively researched field.
Albeit rather basic these many-body systems are inherently strongly
correlated and as such feature a rich variety of
phaenomena including involved patterns of ordering / discordering,
topological order or varieties of phase changes.
Furthermore these models often provide a good approximation
to the low-temperature regime of real physical systems
justifying their detailed study.
One approach is to consider parametrised quantum spin models
as a low-complexity proxy for real systems and use them to understand
which parameter values (e.g. which spin coupling strengths) lead to interesting
behaviours. From this one can deduce inversely how novel materials ought to be
designed in order to probe and study these behaviours experimentally.</p>
<p>In a recent work my mentor Benjamin Stamm and myself
teamed up with <a href="https://www.solidtheory.rwth-aachen.de/go/id/fpbi">Stefan Wessel</a>
(RWTH physics department) and
<a href="http://www.rizzi-matteo.com/">Matteo Rizzi</a>
(Universität Köln, Forschungszentrum Jülich)
to work on cheap surrogate models for accelerating the study of such
parametrised quantum spin models.
Our key assumption is that the Hamiltonian
of these models as well as the deduced quantities of interest
(e.g. the structure factor)
can be decomposed affinely in the parameters.
For many standard models this is indeed the case.
Exploiting the affine structure of the Hamiltonian our
approach constructs a reduced-basis surrogate,
which effectively represents the full problem in a basis of the exact solutions
at a carefully chosen set of parameter values.
As we demonstrate for two examples
(a chain of Rydberg atoms as well as a sheet of coupled triangles)
the information in relatively small reduced bases,
which are orders of magnitude smaller than the dimensionality of the Hilbert space,
sufficient information is accumulated by the reduced basis
in order to reproduce key quantities of interest
over the full parameter domain to an absolute error of 10⁻⁴ or less.</p>
<p>For me this was the first time working with quantum spin models.
Even more so I enjoyed this interdisciplinary collaboration and the associated
diving into a new subject in the discussions we had. Along the work on this
paper we actually identified a number of possibilities for future work.
In fact a number of the problems typically encountered when numerically
modelling quantum spin models
(e.g. due to highly degenerate ground states or issues with the iterative eigensolvers)
are closely related to the challenges for modelling difficult quantum-chemical systems.</p>
<p>The full abstract of our paper reads</p>
<blockquote>
<p>We present a methodology to investigate phase-diagrams of quantum models based
on the principle of the reduced basis method (RBM). The RBM is built from a
few ground-state snapshots, i.e., lowest eigenvectors of the full system
Hamiltonian computed at well-chosen points in the parameter space of interest.
We put forward a greedy-strategy to assemble such small-dimensional basis,
i.e., to select where to spend the numerical effort needed for the snapshots.
Once the RBM is assembled, physical observables required for mapping out the
phase-diagram (e.g., structure factors) can be computed for any parameter value
with a modest computational complexity, considerably lower than the one
associated to the underlying Hilbert space dimension. We benchmark the method
in two test cases, a chain of excited Rydberg atoms and a geometrically
frustrated antiferromagnetic two-dimensional lattice model, and illustrate the
accuracy of the approach.· In particular, we find that the ground-state
manifold can be approximated to sufficient accuracy with a moderate number of
basis functions, which increases very mildly when the number of microscopic
constituents grows --- in stark contrast to the exponential growth of the
Hilbert space needed to describe each of the few snapshots. A combination of
the presented RBM approach with other numerical techniques circumventing even
the latter big cost, e.g., Tensor Network methods, is a tantalising outlook of
this work.</p>
</blockquote>Quantum Chemistry Common Driver and Databases (QCDB) and Quantum Chemistry Engine (QCEngine): Automation and Interoperability among Computational Chemistry Programs2021-11-03T00:30:00+01:002021-11-03T00:30:00+01:00Michael F. Herbsttag:michael-herbst.com,2021-11-03:/2021-qcengine.html<p>As part of <a href="https://michael-herbst.com/tag/adcc.html">my previous work</a> on the <a href="https://adc-connect.org">adcc</a> code
for computational spectroscopy based on the algebraic-diagrammatic construction (ADC),
we also integrated the package with <a href="https://github.com/MolSSI/QCEngine">QCEngine</a>.
This package aims at integrating different quantum-chemistry codes
under a common interface for end users,
which is an effort I fully support.
Recently the …</p><p>As part of <a href="https://michael-herbst.com/tag/adcc.html">my previous work</a> on the <a href="https://adc-connect.org">adcc</a> code
for computational spectroscopy based on the algebraic-diagrammatic construction (ADC),
we also integrated the package with <a href="https://github.com/MolSSI/QCEngine">QCEngine</a>.
This package aims at integrating different quantum-chemistry codes
under a common interface for end users,
which is an effort I fully support.
Recently the design and structure of QCEngine and the related QCDB
packages have been summarised in a publication. Its full abstract reads:</p>
<blockquote>
<p>Community efforts in the computational molecular sciences (CMS) are evolving
toward modular, open, and interoperable interfaces that work with existing
community codes to provide more functionality and composability than could be
achieved with a single program. The Quantum Chemistry Common Driver and
Databases (QCDB) project provides such capability through an application
programming interface (API) that facilitates interoperability across multiple
quantum chemistry software packages. In tandem with the Molecular Sciences
Software Institute and their Quantum Chemistry Archive ecosystem, the unique
functionalities of several CMS programs are integrated, including CFOUR,
GAMESS, NWChem, OpenMM, Psi4, Qcore, TeraChem, and Turbomole, to provide
common computational functions, i.e., energy, gradient, and Hessian
computations as well as molecular properties such as atomic charges and
vibrational frequency analysis. Both standard users and power users benefit
from adopting these APIs as they lower the language barrier of input styles
and enable a standard layout of variables and data. These designs allow
end-to-end interoperable programming of complex computations and provide best
practices options by default.</p>
</blockquote>A robust and efficient line search for self-consistent field iterations2021-10-15T00:30:00+02:002021-10-15T00:30:00+02:00Michael F. Herbsttag:michael-herbst.com,2021-10-15:/2021-adaptive-damping.html<p>In an ongoing effort with Antoine Levitt
our aim is to develop reliable density-functional theory (DFT) methods
for computational materials design.
Recently we looked into a strategy to automatically
select the damping parameter for the self-consistent field iterations (SCF).
Our <strong>adaptive damping</strong> approach is based on
a theoretically sound quadratic …</p><p>In an ongoing effort with Antoine Levitt
our aim is to develop reliable density-functional theory (DFT) methods
for computational materials design.
Recently we looked into a strategy to automatically
select the damping parameter for the self-consistent field iterations (SCF).
Our <strong>adaptive damping</strong> approach is based on
a theoretically sound quadratic model for the DFT energy,
which is used to fix the step size (damping) adaptively
along the search directions suggested by an underlying algorithm
(such as Pulay mixing, Kerker mixing, etc.).
Our algorithm is fully automatic, i.e. an <em>a priori</em> damping selection
is no longer required. In our work we test our method successfully
on a range of challenging systems including
supercells, transition-metal alloys or metallic surfaces.
Overall our study shows adaptive damping to provide superior robustness
over the traditional fixed-damping approach.</p>
<p>As I have reported in
<a href="https://michael-herbst.com/2021-virtual-materials-design.html">previous</a>
<a href="https://michael-herbst.com/2021-siamms.html">blog</a>
<a href="https://michael-herbst.com/2021-siamla.html">articles</a>
and we also discussed in
our previous publication on <a href="https://michael-herbst.com/2020-ldos-preconditioning.html">black-box mixing strategies for inhomogeneous systems</a>
the main motivation of our work is to design numerical methods,
which are parameter-free and automatically self-adapt to the simulated material.
In modern simulation scenarios where millions of DFT calculations are required
in order to generate training data or screen over large design spaces,
robustness and automation are the key requirements.
Often it is in fact less the computational time of the individual calculations,
which limits overall throughput.
Much rather it is the human factor, i.e. the human time required to
setup, check and verify computations.</p>
<p>Clearly at the level of millions of calculations
computational parameters can no longer be selected manually.
Instead elaborate heuristics are employed to select basis set size,
k-point sampling, SCF algorithm or the damping parameter.
In case a calculation fails heuristics are also employed for automatic restart.
However, this approach is far from perfect and even an optimistic 1% failure
rate easily equals thousands of calculations, which require human attention.
With our work
(both the <a href="https://michael-herbst.com/2020-ldos-preconditioning.html">previous paper</a>
as well as this one) we want to replace heuristic approaches to parameter selection
by algorithms that employ a mixture of mathematical and physical insight
to automatically adapt to the simulation at hand.
As we demonstrate in this work,
such algorithms might be associated with an increased effort compared to the
best possible parameter setting,
however it also makes calculations overall more robust.
Therefore one saves (a) on the repeated effort to find a suitable parameter set
by trial and error and (b) reduces the fraction of calculations,
which need to be considered by a human.
Overall the maximally attainable throughput can therefore be expected to increase
from such a robust scheme despite the fact that an individual calculation
might be more costly.</p>
<p>In this work in particular we considered the question of choosing the damping parameter.
For this our <strong>adaptive damping approach</strong> is based on constructing
an approximate quadratic model for the DFT energy
and using this model within a line search procedure.
Since this procedure is associated with an additional cost,
we only employ it in case the proposed
SCF step would either increase the DFT energy or SCF residual
Notably our approach introduces no changes to the SCF in case each proposed
SCF step by the mixing procedure is already perfect (i.e. energy or residual decreasing).
Therefore adaptive damping can be considered a safeguared,
which only comes into play if the proposed steps are noisy or erroneous.
Adaptive damping is by construction orthogonal to any existing
mixing and convergence acceleration technique for DFT methods
and in our work we demonstrate it to integrate readily into an Anderson-accelerated
SCF for various challenging systems.
Overall we managed to increase performance and robustness at only a minor extra cost.
The full abstract of our paper reads</p>
<blockquote>
<p>We propose a novel adaptive damping algorithm for the
self-consistent field (SCF) iterations of Kohn-Sham
density-functional theory, using a backtracking line search to
automatically adjust the damping in each SCF step. This line search
is based on a theoretically sound, accurate and inexpensive model
for the energy as a function of the damping parameter. In contrast
to usual SCF schemes, the resulting algorithm is fully automatic
and does not require the user to select a damping. We
successfully apply it to a wide range of challenging systems,
including elongated supercells, surfaces and transition-metal alloys.</p>
</blockquote>Q-Chem 5 paper2021-08-29T00:30:00+02:002021-08-29T00:30:00+02:00Michael F. Herbsttag:michael-herbst.com,2021-08-29:/2021-qchem5.html<p>About two years ago I integrated my
open-source <a href="/articles/Research/ctxlibrary.md">ctx library</a>
into the <a href="https://q-chem.com">Q-Chem</a> quantum-chemistry software suite.
Quickly <code>ctx</code> became part of the core stack for managing
computational results inside Q-Chem.
In particular inside the <code>ccman</code> and <code>adcman</code> modules,
which are responsible for most of the coupled-cluster
and algebraic-diagrammatic construction methods …</p><p>About two years ago I integrated my
open-source <a href="/articles/Research/ctxlibrary.md">ctx library</a>
into the <a href="https://q-chem.com">Q-Chem</a> quantum-chemistry software suite.
Quickly <code>ctx</code> became part of the core stack for managing
computational results inside Q-Chem.
In particular inside the <code>ccman</code> and <code>adcman</code> modules,
which are responsible for most of the coupled-cluster
and algebraic-diagrammatic construction methods available in Q-Chem,
<code>ctx</code> is widely used.</p>
<p>In a recently published paper by all the Q-Chem authors
the developments inside the Q-Chem package
leading up the major version 5 of the software are now summarised.
The full abstract reads</p>
<blockquote>
<p>This article summarizes technical advances contained in the fifth major release
of the Q-Chem quantum chemistry program package, covering developments since
2015. A comprehensive library of exchange-correlation functionals, along with a
suite of correlated many-body methods, continues to be a hallmark of the Q-Chem
software. The many-body methods include novel variants of both coupled-cluster
and configuration-interaction approaches along with methods based on the
algebraic diagrammatic construction and variational reduced density-matrix
methods. Methods highlighted in Q-Chem 5 include a suite of tools for modeling
core-level spectroscopy, methods for describing metastable resonances, methods
for computing vibronic spectra, the nuclear–electronic orbital method, and
several different energy decomposition analysis techniques. High-performance
capabilities including multithreaded parallelism and support for calculations
on graphics processing units are described. Q-Chem boasts a community of well
over 100 active academic developers, and the continuing evolution of the
software is supported by an "open teamware" model and an increasingly modular design.</p>
</blockquote>JuliaCon BoF discussion session: Building a Chemistry and Materials Science Ecosystem2021-08-04T12:00:00+02:002021-08-04T12:00:00+02:00Michael F. Herbsttag:michael-herbst.com,2021-08-04:/2021-juliacon-bof-chemistry-materials.html<p>The second event I co-organised at this year's JuliaCon
(see <a href="https://michael-herbst.com/2021-juliacon-workshop-dftk.html">this article</a> for the other) was a
Birds of Feather (BoF) discussion session
titled <em>Building a Chemistry and Materials Science Ecosystem in Julia</em>.
In this session Rachel Kurchin and I wanted to gather the various
stakeholders working on Julia codes …</p><p>The second event I co-organised at this year's JuliaCon
(see <a href="https://michael-herbst.com/2021-juliacon-workshop-dftk.html">this article</a> for the other) was a
Birds of Feather (BoF) discussion session
titled <em>Building a Chemistry and Materials Science Ecosystem in Julia</em>.
In this session Rachel Kurchin and I wanted to gather the various
stakeholders working on Julia codes for chemistry and materials simulations
and discuss possible overlaps and plan future joint efforts.</p>
<p>This has been the first time a meeting dedicated to this scientific field
has been conducted within the Julia community and so we were quite curious
about who would turn up.
In the end we had a pretty mixed crowd consisting
of Julia users tackling research problems in chemistry and materials
as well as plenty of maintainers of various Julia packages related to the field,
but also some veteran Julia users joined the discussion.
This mix of people provoked a rather rich and lively debate
about the perspectives of Julia in this respective field
and the 90 minutes which were given to us passed almost in an instance.</p>
<p>A central discussion point within the session was the need
for joint interfaces shared amongst the key packages of the ecosystem
both to leverage Julia's unique composability between the various packages
and to furthermore enhance the interoperability and lead to a good user experience.
As many have pointed out during the session, a good first step is the design
of an interface for representing the structure of the chemical system
or the material to be studied. In particular this would allow to deveop unified
approches to share data between packages, setup calculations and plainly compare
between different approaches.
Additionally annoying aspects such as file parsing, data export,
plotting or other post-processing could then be easily implemented once
using the general interface and used by everyone in the Julia community.
Naturally a time slot of 90 minutes is just about sufficient to get the
discussion started and scratch the surface, so the session has not yet yielded
anything conclusive.
However, following up from the conference the debate has definitely intensified
amongst participants and I would not be suprised if some progress will be made.</p>
<p>In case you are interested to participate in these developments
or plainly want to get in touch with Julia users and developers
from chemistry, molecular or materials science, here are a number of relevant resouces:</p>
<ul>
<li><a href="https://juliamolsim.github.io/">Website of the JuliaMolSim github organisation</a>:
We plan to use this to gather information about the
the chemistry and materials simulations ecosystem in Julia
and to communicate events.</li>
<li><a href="https://join.slack.com/t/juliamolsim/shared_invite/zt-tc060co0-HgiKApazzsQzBHDlQ58A7g">JuliaMolSim slack workspace</a>:
The place where discussions are ongoing and some package developers
provide chat support (e.g. DFTK's primary support channel is there).</li>
</ul>JuliaCon DFTK workshop: A mathematical look at electronic structure theory2021-08-03T12:00:00+02:002021-08-03T12:00:00+02:00Michael F. Herbsttag:michael-herbst.com,2021-08-03:/2021-juliacon-workshop-dftk.html<p>From 13th July till 30th July this year's JuliaCon finally took place virtually.
The first week (13th till 27th) hosted a number of three-hour live-streamed
sessions of workshops, while the "regular" conference with a number of prerecorded
talks started on 28th.</p>
<p>After my <a href="https://michael-herbst.com/2020-juliacon.html">introductory talk</a>
to electronic structure theory and …</p><p>From 13th July till 30th July this year's JuliaCon finally took place virtually.
The first week (13th till 27th) hosted a number of three-hour live-streamed
sessions of workshops, while the "regular" conference with a number of prerecorded
talks started on 28th.</p>
<p>After my <a href="https://michael-herbst.com/2020-juliacon.html">introductory talk</a>
to electronic structure theory and our DFTK code
at <a href="https://michael-herbst.com/2020-juliacon.html">last year's Juliacon</a>,
this year I participated at the conference with two events.
One BoF <strong>discussion session</strong> gathering the people working
<strong>on materials-science and electronic-structure</strong>
codes in Julia about which I will write
some more in a <a href="https://michael-herbst.com/2021-juliacon-bof-chemistry-materials.html">follow-up blog article</a>.</p>
<p>My second event was a three-hour workshop titled
<em>A mathematical look at electronic structure theory</em>
in which I prepared a broadly accessible introduction
into density-functional theory (DFT),
the numerical procedures to solve DFT
as well as some tools from numerical analysis to understand
the convergence properties of these methods.
As the tool to conduct the relevant calculations,
code up and study the respective self-consistent field (SCF) algorithms
we used our <a href="https://dftk.org">density-functional toolkit (DFTK)</a>.
The workshop therefore also provides a great showcase for the merits
of this code and how it leverages the broader Julia ecosystem
to gain its unique features (arbitrary floating-point types,
flexible and composable algorithms,
automatic differentiation,
numerical analysis techniques to investigate convergence failures, etc. ).
For <strong>more details</strong> on the workshop
see the <a href="https://michael-herbst.com/teaching/2021-juliacon-workshop-dftk/">dedicated teaching page</a>.</p>
<p>What surprised me very positively during conducting the workshop
was the large number of viewers that followed the workshop live and actively engaged
by asking questions or posting comments on Youtube.
Since the workshop was hosted at Juliacon I wouldn't have thought
this topic would capture this many people, so in retrospect I am very
happy I did it.
In that sense also a big thanks to everyone who participated
and provided me with feedback afterwards.
(BTW: I'm still happy to take any in case you have some comments or suggesitons).</p>
<p>In case you missed the workshop the <strong>complete materials</strong> are
<a href="https://github.com/mfherbst/juliacon_dft_workshop">available on github</a>
and the <strong>full recording</strong> of the workshop
is <a href="https://www.youtube.com/watch?v=HvpPMWVm8aw">available on Youtube</a>.</p>Virtual materials design 2021: Black-box density-functional theory methods2021-08-01T12:00:00+02:002021-08-01T12:00:00+02:00Michael F. Herbsttag:michael-herbst.com,2021-08-01:/2021-virtual-materials-design.html<p>On 20th and 21st July 2021 the
<a href="https://www.cecam.org/workshop-details/1093">Virtual Materials Design 2021 CECAM workshop</a>
took place virtually.
I was excited about this workshop and the
opportunity to get in touch with researchers working on
high-throughput computational materials design.
While I am not actively working in this field
the special requirements of …</p><p>On 20th and 21st July 2021 the
<a href="https://www.cecam.org/workshop-details/1093">Virtual Materials Design 2021 CECAM workshop</a>
took place virtually.
I was excited about this workshop and the
opportunity to get in touch with researchers working on
high-throughput computational materials design.
While I am not actively working in this field
the special requirements of the multitude of calculations running
in this field clearly have been one of the main motivations for my work on
<a href="https://michael-herbst.com/2021-dftk-proceedings.html">DFTK</a>,
<a href="https://michael-herbst.com/2020-nonscf-error-estimates.html">error control</a>
and <a href="https://michael-herbst.com/2020-ldos-preconditioning.html">black-box SCF algorithms</a>.
In advance of the workshop I asked the organisers to participate with a contributed talk
to present my work to this community for the first time, which thankfully got accepted.</p>
<p>Due to the virtual format the workshop it was unfortunately rather packed,
which allowed for little time to engage in discussion during the presentation slot.
However, the organisers arranged multiple longer poster sessions in a GatherTown
virtual world, which allowed for almost realistic face-to-face discussions.
In these GatherTown sessions I talked with a number of scientists
working on high-throughput studies as well as designing the
large software infrastructures, which are commonly used to conduct these.
At the level of performing millions of individual calculations
in a screening study this naturally poses especial demands
on the workflow software as well and I was curious
to learn about some of the details.</p>
<p>With my focus on advocating a more mathematical look at screening and
DFT simulations I represented a minority viewpoint at the meeting
and I was very curious about the general feedback and critique
of the more applied scientists in response to our recently proposed ideas.
In general people were indeed quite interested to learn about our work on reliable
<a href="https://michael-herbst.com/2020-ldos-preconditioning.html">SCF methods for inhomogeneous systems</a>,
but being confronted with our recent
<a href="https://michael-herbst.com/2020-nonscf-error-estimates.html">error estimation perspectives</a>,
some had doubts about the required effort being really worth it for DFT simulations.
I certainly understand that concern.
However, I think one should keep in mind the successes and potential,
which has been unlocked by error estimation techniques in other fields,
such as finite-element modelling or aerospace design.
In these fields simulation methods have both become more efficient
due to the lessons learned from uncertainty quantification and error estimation
and the nowadays well-established error estimation techniques
have furthermore contributed to prevent accidents from trusting faulty simulation data
(such as the <a href="https://en.wikipedia.org/wiki/Sleipner_A">Sleipner A oil rig collapse</a>).
While clearly not all aspects of macroscopic modelling apply in the microscopic world,
it is not hard to imagine that error bars establishing a guaranteed trustworthiness
can make screening decisions more robust, thus potentially preventing costly
manufacture of less useful compounds.
Furthermore I expect a careful introduction of numerical errors
(e.g. by lowering the floating-point type)
to balance numerical error against the (usually much larger) DFT model error
to allow for notable computational savings when performing on the order
of millions of DFT calculations.</p>
<p>Overall I have enjoyed the two afternoons with many discussions
in the high-throughput design community.
As usual my slides are attached below.</p>
<table>
<thead>
<tr>
<th>Link</th>
</tr>
</thead>
<tbody>
<tr>
<td><a href="https://michael-herbst.com/talks/2021.07.21_vmd_black_box_dft.pdf">Towards error-controlled, black-box density-functional theory methods</a> (Slides)</td>
</tr>
</tbody>
</table>SSD Seminar: Accelerating the discovery of tomorrow's materials by robust and error-controlled simulations2021-07-15T12:01:00+02:002021-07-15T12:01:00+02:00Michael F. Herbsttag:michael-herbst.com,2021-07-15:/2021-ssd-seminar-scf-materials-discovery.html<p>A couple of days ago, on 12th July, I was invited to present my research
in the <a href="https://blog.rwth-aachen.de/irtg-mip/ssd-seminar-series-ss-2021/">SSD Seminar Series</a> of RWTH Aachen.
Being part of the <a href="https://blog.rwth-aachen.de/irtg-mip/">research training group on modern inverse problems</a>
as well as the School for Simulation and Data Science (SSD) the SSD seminars
are interdisciplinary …</p><p>A couple of days ago, on 12th July, I was invited to present my research
in the <a href="https://blog.rwth-aachen.de/irtg-mip/ssd-seminar-series-ss-2021/">SSD Seminar Series</a> of RWTH Aachen.
Being part of the <a href="https://blog.rwth-aachen.de/irtg-mip/">research training group on modern inverse problems</a>
as well as the School for Simulation and Data Science (SSD) the SSD seminars
are interdisciplinary and feature researchers as well as Master-level students
from a couple of departments at RWTH (mathematics, computer science, simulation sciences, ...).</p>
<p>To make my recent work on error estimation and the design of robust algorithms
for density-functional theory broadly accessible I started by motivating
the need for density-functional theory (DFT) and high-throughput methods
for the discovery and design of novel materials.
Afterwards I briefly hinted at the mathematical structure of the equations,
which need to be solved to obtain DFT properties.
With this in mind I presented current research questions at the edge
of mathematics and electronic-structure modelling and presented some of my recent results.
As usual the slides are attached below.</p>
<table>
<thead>
<tr>
<th>Link</th>
</tr>
</thead>
<tbody>
<tr>
<td><a href="https://michael-herbst.com/talks/2021.07.12_ssd_materials_discovery.pdf">Accelerating the discovery of tomorrow's materials by robust and error-controlled electronic-structure simulations</a> (Slides)</td>
</tr>
</tbody>
</table>Talk at many-body seminar at RWTH2021-07-15T12:00:00+02:002021-07-15T12:00:00+02:00Michael F. Herbsttag:michael-herbst.com,2021-07-15:/2021-many-body-methods.html<p>On 29th June I was invited to present a short summary of my research
at the seminar of the research training group
<a href="https://www.rtg1995.rwth-aachen.de">Quantum Many-Body Methods</a>
at RWTH Aachen University.
In the talk I give a overview over my ongoing work about
reliable black-box self-consistent field schemes for high-throughput DFT calculations …</p><p>On 29th June I was invited to present a short summary of my research
at the seminar of the research training group
<a href="https://www.rtg1995.rwth-aachen.de">Quantum Many-Body Methods</a>
at RWTH Aachen University.
In the talk I give a overview over my ongoing work about
reliable black-box self-consistent field schemes for high-throughput DFT calculations.
My slides are attached below.</p>
<table>
<thead>
<tr>
<th>Link</th>
</tr>
</thead>
<tbody>
<tr>
<td><a href="https://michael-herbst.com/talks/2021.06.29_qmb_reliable_scf.pdf">Reliable black-box self-consistent field schemes for high-throughput DFT calculations</a> (Slides)</td>
</tr>
</tbody>
</table>Errors and uncertainty quantification in density-functional theory2021-06-21T12:00:00+02:002021-06-21T12:00:00+02:00Michael F. Herbsttag:michael-herbst.com,2021-06-21:/2021-marzouk-uq.html<p>On 8th June I was invited to the seminar of the Uncertainty Quantification (UQ) group
of <a href="https://uqgroup.mit.edu/">Prof. Youssef Marzouk</a> at MIT.
Youssef and I planned to have this seminar since my involvement
with MIT's <a href="https://cesmix.mit.edu/">CESMIX project</a>
last February (see also <a href="https://michael-herbst.com/2021-cesmix-tst.html">this blog article</a>),
but it took use quite some time …</p><p>On 8th June I was invited to the seminar of the Uncertainty Quantification (UQ) group
of <a href="https://uqgroup.mit.edu/">Prof. Youssef Marzouk</a> at MIT.
Youssef and I planned to have this seminar since my involvement
with MIT's <a href="https://cesmix.mit.edu/">CESMIX project</a>
last February (see also <a href="https://michael-herbst.com/2021-cesmix-tst.html">this blog article</a>),
but it took use quite some time to get it arranged.
Finally I managed to present my point of view on UQ
in density-functional theory (DFT),
sneakily re-using most of the slides I had already prepared
for my recent UQ-in-DFT talk
at RWTH Aachen's UQ group <a href="https://michael-herbst.com/2021-math4uq.html">the week earlier</a>.</p>
<p>Similar to the Aachen talk I've put strong emphasis on engaging
audience participation and discussion.
I first introduced the UQ group
to electronic structure theory and DFT,
allowing for enough time to discuss the key ideas of the physics.
Then I pointed out current research in error estimation and UQ
in DFT and provided a number of opportunities for interesting
future UQ-related research.
The discussion was very lively and I hardly made
it beyond a slide without a question, which was just great.
Since a lot could be gained from stronger
uncertainty quantification tools in DFT in my opinion,
I hoped this talk made DFT more accessible to the UQ group
and made some people curious to look into the details.
On my end I would definitely enjoy to learn more about UQ in the future
and look forward to my future UQ-related involvements in the CESMIX project.
As usual my slides are attached below.</p>
<table>
<thead>
<tr>
<th>Link</th>
</tr>
</thead>
<tbody>
<tr>
<td><a href="https://michael-herbst.com/talks/2021.06.08_mit_dft_uq.pdf">Errors and uncertainty quantification in electronic-structure theory</a> (Slides)</td>
</tr>
</tbody>
</table>Talk at MATH4UQ seminar series at RWTH2021-06-15T12:00:00+02:002021-06-15T12:00:00+02:00Michael F. Herbsttag:michael-herbst.com,2021-06-15:/2021-math4uq.html<p>On 1st June I was invited to the <a href="https://www.uq.rwth-aachen.de/cms/UQ/Forschung/~gbopj/MATH4UQ-Forschungseminar/">MATH4UQ seminar series</a>
of the <a href="https://www.uq.rwth-aachen.de">Mathematics of Uncertainty Quantification</a>
chair of Prof. Raul Tempone at RWTH Aachen University.</p>
<p>Over the past months I got more and more interested in
mathematical methods for uncertainty quantification (UQ)
as an opportunity to estimate and understand …</p><p>On 1st June I was invited to the <a href="https://www.uq.rwth-aachen.de/cms/UQ/Forschung/~gbopj/MATH4UQ-Forschungseminar/">MATH4UQ seminar series</a>
of the <a href="https://www.uq.rwth-aachen.de">Mathematics of Uncertainty Quantification</a>
chair of Prof. Raul Tempone at RWTH Aachen University.</p>
<p>Over the past months I got more and more interested in
mathematical methods for uncertainty quantification (UQ)
as an opportunity to estimate and understand errors
in density-functional theory (DFT) calculations.
In particular I imagine UQ methods to be useful to estimate
the model error of a DFT model itself.
At this level statistical approaches are likely the only feasible option
for a practical error estimation,
since the mathematical complexity of modern DFT models
beyond the local density approximations
very likely make <em>a posteriori</em> error analysis strategies extremely infeasible.</p>
<p>In my talk I explain the basics of DFT and provide a rough overview
of present UQ developments in this method.
Since I know very little about UQ and my audience knew
very little about DFT, I intended the talk to be
more of a Q&A session, where the slides are around to stimulate discussion.
This turned out to work very well
and I am very grateful to the many interesting
questions from the audience
and the enjoyful discussion.
As usual my slides are attached below.
Additionally a recording of my talk can be found <a href="https://www.youtube.com/watch?v=S6Vjh06a2A8">on youtube</a>.</p>
<table>
<thead>
<tr>
<th>Link</th>
</tr>
</thead>
<tbody>
<tr>
<td><a href="https://michael-herbst.com/talks/2021.06.01_dft_uq.pdf">Errors in electronic-structure theory: Status and directions for future research</a> (Slides)</td>
</tr>
<tr>
<td><a href="https://www.youtube.com/watch?v=S6Vjh06a2A8">Youtube recording of the talk</a></td>
</tr>
</tbody>
</table>SIAM LA: Robust and efficient accelerated methods for density-functional theory2021-05-30T18:01:00+02:002021-05-30T18:01:00+02:00Michael F. Herbsttag:michael-herbst.com,2021-05-30:/2021-siamla.html<p>Just one day after my talk at the SIAM Materials Science conference
(<a href="https://michael-herbst.com/2021-siamms.html">blog article</a>) I gave another talk at a SIAM meeting,
this time at SIAM Linear Algebra.
I was very much looking forward to participate in SIAM LA,
firstly because it was the first time I attended this conference …</p><p>Just one day after my talk at the SIAM Materials Science conference
(<a href="https://michael-herbst.com/2021-siamms.html">blog article</a>) I gave another talk at a SIAM meeting,
this time at SIAM Linear Algebra.
I was very much looking forward to participate in SIAM LA,
firstly because it was the first time I attended this conference,
but also secondly because it was a good opportunity to talk about
our recent algorithmic work on robust DFT methods to an international
crowd of mathematicians.</p>
<p>I presented as part of the minisymposium
<a href="https://meetings.siam.org/sess/dsp_programsess.cfm?SESSIONCODE=70908"><em>Theory and Practice of Extrapolation and Acceleration Methods</em></a>,
which consisted of three interesting sessions
of historic and recent talks about extrapolation and convergence acceleration
in the broadest sense of the word.
Both topics about iterative methods as well as summation theory and sequence summation
were discussed, which turned out to be a very enjoyful mix.
In that sense I am really grateful for the mini organisers,
Agnieszka Miedlar and Yousef Saad,
for the invitation and for allowing me to be part of the great sessions.</p>
<p>Beyond the mini I enjoyed a number of talks about emerging topics
in numerical linear algebra such as mixed-precision computation,
low-rank tensor approximations or randomised methods.
Even though the time zone difference meant that the conference was mostly
running during the afternoon and late evening for me
and even though the collision with SIAM Materials Science made it quite a busy week,
I took a lot from SIAM LA and I'm already looking forward to next time.</p>
<table>
<thead>
<tr>
<th>Link</th>
</tr>
</thead>
<tbody>
<tr>
<td><a href="https://michael-herbst.com/talks/2021.05.20_siamla_dft_acceleration.pdf">Robust and Efficient Accelerated Methods for Kohn-Sham Density-Functional Theory</a></td>
</tr>
</tbody>
</table>SIAM MS: Using the density-functional toolkit to design black-box DFT methods2021-05-30T18:00:00+02:002021-05-30T18:00:00+02:00Michael F. Herbsttag:michael-herbst.com,2021-05-30:/2021-siamms.html<p>After being moved by one year due to the pandemic,
the last two weeks (from 17th to 28th May)
the <a href="https://wp.bcamath.org/siamms21/">SIAM materials science conference</a>
finally took place in virtual form.
Unfortunately this meant that the conference was scheduled in
parallel to the SIAM Linear Algebra
virtual conference,
where I also …</p><p>After being moved by one year due to the pandemic,
the last two weeks (from 17th to 28th May)
the <a href="https://wp.bcamath.org/siamms21/">SIAM materials science conference</a>
finally took place in virtual form.
Unfortunately this meant that the conference was scheduled in
parallel to the SIAM Linear Algebra
virtual conference,
where I also presented (<a href="https://michael-herbst.com/2021-siamla.html">blog article</a>),
which made my past two weeks rather busy.</p>
<p>At SIAM Materials I was invited to talk in
the Minisymposium <em>Numerics of electronic structure calculations</em>,
which was organised by the steering committee of the
GAMM activity group <a href="https://moansi.wixsite.com/gamm/">moansi (Modelling, Analysis and Simulation of Molecular Systems)</a>.
Besides interesting sessions about some mathematical insights
to electronic-structure methods, this gave the mini
the additional feature of a spring gathering for the usual crowd of the activity group,
which I already had the pleasure to meet at <a href="https://michael-herbst.com/2019-munich-moansi-domcke.html">previous</a> <a href="https://michael-herbst.com/2020-moansi.html">moansi workshops</a>.</p>
<p>In my talk I gave a broad overview of the recent projects
we realised with the <a href="https://dftk.org">density-functional toolkit</a>
for making self-consistent field calculations for density-functional theory
more robust and reliable.
Apart from our work on preconditioning inhomogeneous systems
with the <a href="https://michael-herbst.com/2020-ldos-preconditioning.html">LDOS preconditioner</a>
I also presented first work-in-progress results
on an adaptive damping strategy we recently came up with.
The idea of our method is to use a line search based on an approximate
quadratic model for cases where a proposed SCF step is not successful
(i.e. increases energy and SCF residual).
This firstly allows to automatically choose the damping parameter
(instead of requiring the user to choose one by trial and error).
Secondly it makes the SCF procedure more robust, especially for tricky cases.
For example in our experiments on Heusler alloys
our adaptive damping approach was the only method that managed to converge on some cases.</p>
<table>
<thead>
<tr>
<th>Link</th>
</tr>
</thead>
<tbody>
<tr>
<td><a href="https://michael-herbst.com/talks/2021.05.19_siamms_dftk_blackbox.pdf">Using the density-functional toolkit (DFTK) to design black-box methods in density-functional theory</a></td>
</tr>
</tbody>
</table>Infomath seminar: A one-hour introduction to Julia2021-05-11T12:01:00+02:002021-05-11T12:01:00+02:00Michael F. Herbsttag:michael-herbst.com,2021-05-11:/2021-infomath-julia.html<p>Just one day after <a href="https://michael-herbst.com/2021-luechow-research-overview.html">my talk at the Lüchow group</a>
in Aachen, on 6th May I was asked to give a short introduction to Julia
at the <a href="https://infomath.pages.math.cnrs.fr">Infomath</a> seminar series at Sorbonne Université.
While virtual seminars certainly don't share the same spirit as in-person ones do,
the ability to quickly …</p><p>Just one day after <a href="https://michael-herbst.com/2021-luechow-research-overview.html">my talk at the Lüchow group</a>
in Aachen, on 6th May I was asked to give a short introduction to Julia
at the <a href="https://infomath.pages.math.cnrs.fr">Infomath</a> seminar series at Sorbonne Université.
While virtual seminars certainly don't share the same spirit as in-person ones do,
the ability to quickly hop between seminar series organised all across the world
has advantages, too.</p>
<p>In my one-hour talk I gave a short introduction into Julia,
focusing on the perspective of applied mathematicians.
I gave a short speed comparsion of Julia, python and C
on a simple example and presented some of its strengths
in different application scenarios (numerical linear algebra,
numerical methods for solving PDEs, data science and statistical learning).
For future reference I have put the Jupyter notebooks
I used during the lecture <a href="https://github.com/mfherbst/julia-one-hour-intro">on github</a>.</p>
<table>
<thead>
<tr>
<th>Link</th>
</tr>
</thead>
<tbody>
<tr>
<td><a href="https://github.com/mfherbst/julia-one-hour-intro">An introductory hour to the Julia programming language</a> (Github repository)</td>
</tr>
</tbody>
</table>Talk at Lüchow group seminar at RWTH2021-05-11T12:00:00+02:002021-05-11T12:00:00+02:00Michael F. Herbsttag:michael-herbst.com,2021-05-11:/2021-luechow-research-overview.html<p>On 5th May I was invited to present a short summary of my research
at the local theoretical chemistry research group of
<a href="https://www.ipc.rwth-aachen.de/cms/IPC/Das-Institut/IPC-Arbeitsgruppen/~lbfz/Prof-Luechow-Theoretische-Chemie/">Prof. Dr. Arne Lüchow</a>
at RWTH Aachen. Because I wanted to give a broad overview of topics
that I worked on over the past few years, I did …</p><p>On 5th May I was invited to present a short summary of my research
at the local theoretical chemistry research group of
<a href="https://www.ipc.rwth-aachen.de/cms/IPC/Das-Institut/IPC-Arbeitsgruppen/~lbfz/Prof-Luechow-Theoretische-Chemie/">Prof. Dr. Arne Lüchow</a>
at RWTH Aachen. Because I wanted to give a broad overview of topics
that I worked on over the past few years, I did not really go into many details.
Nevertheless my talk lead to interesting and lively discussions,
which I enjoyed very much. Clearly the pandemic makes it difficult
to get in touch with other researchers, such that I was pretty happy
to be able to get to get in touch with some more chemists
from the RWTH during the seminar.</p>
<table>
<thead>
<tr>
<th>Link</th>
</tr>
</thead>
<tbody>
<tr>
<td><a href="https://michael-herbst.com/talks/2021.05.05_high_throughput_reliability.pdf">High-throughput electronic-structure simulations: Where reliability really matters</a> (Slides)</td>
</tr>
</tbody>
</table>DFTK: A Julian approach for simulating electrons in solids2021-05-08T00:30:00+02:002021-05-08T00:30:00+02:00Michael F. Herbsttag:michael-herbst.com,2021-05-08:/2021-dftk-proceedings.html<p>Following my talk at Juliacon about our <a href="https://dftk.org">DFTK</a> code last year
(<a href="https://michael-herbst.com/talks/2020.07.29_juliacon_dftk.pdf">slides</a>,
<a href="https://www.youtube.com/watch?v=-RomkxjlIcQ">recording</a>,
<a href="https://michael-herbst.com/2020-juliacon.html">blog article</a>),
we have now published an extended abstract in the JuliaCon proceedings,
which you can find below.
The JuliaCon proceedings use the same open journals software stack
to manage their publication infrastructure as the Journal of …</p><p>Following my talk at Juliacon about our <a href="https://dftk.org">DFTK</a> code last year
(<a href="https://michael-herbst.com/talks/2020.07.29_juliacon_dftk.pdf">slides</a>,
<a href="https://www.youtube.com/watch?v=-RomkxjlIcQ">recording</a>,
<a href="https://michael-herbst.com/2020-juliacon.html">blog article</a>),
we have now published an extended abstract in the JuliaCon proceedings,
which you can find below.
The JuliaCon proceedings use the same open journals software stack
to manage their publication infrastructure as the Journal of Open-source Software.
This stack is actually
pretty impressive since it reduces the effort both on the reviewer as well
as on the author side to comments within github issues.
Since thus the complete exchange
(including the <a href="https://github.com/JuliaCon/proceedings-review/issues/69">review process</a>)
is public,
this is not only convenient, but also leads to truly transparent publication process.
I wish publishing with all journals was like that ...</p>Thoughts on initial guess methods for DFT2021-05-01T12:00:00+02:002021-05-01T12:00:00+02:00Michael F. Herbsttag:michael-herbst.com,2021-05-01:/2021-aced-dft-guesses.html<p>On Thursday I gave a brief talk in our weekly
<a href="https://www.cmu.edu/aced/index.html">ACED differentiate</a>
group meeting about initial guess methods for starting self-consistent field calculations
in methods such as density-functional theory.
For preparing the talk I did a little digging into both the standard approaches
used by many molecular and solid-state codes …</p><p>On Thursday I gave a brief talk in our weekly
<a href="https://www.cmu.edu/aced/index.html">ACED differentiate</a>
group meeting about initial guess methods for starting self-consistent field calculations
in methods such as density-functional theory.
For preparing the talk I did a little digging into both the standard approaches
used by many molecular and solid-state codes and did a literature review
of some recent ideas motivated from reduced-order modelling or data science.
The slides of my talk (which include most references I found) are attached below.</p>
<table>
<thead>
<tr>
<th>Link</th>
</tr>
</thead>
<tbody>
<tr>
<td><a href="https://michael-herbst.com/talks/2021.04.29_dft_guess_overview.pdf">Thoughts on initial guess methods for DFT</a> (Slides)</td>
</tr>
</tbody>
</table>A novel black-box preconditioning strategy for high-throughput density-functional theory2021-04-09T18:00:00+02:002021-04-09T18:00:00+02:00Michael F. Herbsttag:michael-herbst.com,2021-04-09:/2021-gamm.html<p>A couple of weeks ago, from 15th to 19th March, I participated in the
virtual <a href="https://jahrestagung.gamm-ev.de/index.php">annual meeting</a>
of the German Association of Applied Mathematics and Mechanics (GAMM).
For me this meeting was the first time I presented my work to an audience
of applied mathematicians with a broad background and …</p><p>A couple of weeks ago, from 15th to 19th March, I participated in the
virtual <a href="https://jahrestagung.gamm-ev.de/index.php">annual meeting</a>
of the German Association of Applied Mathematics and Mechanics (GAMM).
For me this meeting was the first time I presented my work to an audience
of applied mathematicians with a broad background and no inherent
interest in quantum chemistry.
With only 15 minutes for my talk in the "scientific computing" track
preparing the material was quite a challenge.
I hope I still managed to convey the main ideas of
our recently published <a href="https://michael-herbst.com/2020-ldos-preconditioning.html">LDOS preconditioner</a>
in a broadly accessible way.
My slides are attached below.</p>
<table>
<thead>
<tr>
<th>Link</th>
</tr>
</thead>
<tbody>
<tr>
<td><a href="https://michael-herbst.com/talks/2021.03.17_gamm_scf_preconditioning.pdf">A novel black-box preconditioning strategy for high-throughput density-functional theory</a> (Slides)</td>
</tr>
</tbody>
</table>PostDoc position at Appl. & Comput. Mathematics lab, RWTH Aachen University2021-03-05T00:00:00+01:002021-03-05T00:00:00+01:00Michael F. Herbsttag:michael-herbst.com,2021-03-05:/2021-acom.html<p>This week I started my new position as a postdocotoral researcher
at the <a href="http://www.acom.rwth-aachen.de/">Applied and Computational Mathematics (ACoM) research lab</a>
at RWTH Aachen University.
The lab consists of two interdisciplinary research groups,
namely the group of <a href="http://www.acom.rwth-aachen.de/2research/0mms/start">Prof. Dr. Manuel Torrilhon</a>,
who works on the mathematical modelling and simulation of technical …</p><p>This week I started my new position as a postdocotoral researcher
at the <a href="http://www.acom.rwth-aachen.de/">Applied and Computational Mathematics (ACoM) research lab</a>
at RWTH Aachen University.
The lab consists of two interdisciplinary research groups,
namely the group of <a href="http://www.acom.rwth-aachen.de/2research/0mms/start">Prof. Dr. Manuel Torrilhon</a>,
who works on the mathematical modelling and simulation of technical processes
(e.g. plasma or gas flow processes) as well as
the group of <a href="http://www.acom.rwth-aachen.de/5people/stamm/start">Prof. Dr. Benjamin Stamm</a>,
which I am now joining.
Ben's research focus is the numerical analysis of
PDEs and linear algebra problems,
which arise e.g. in electrostatics or quantum chemistry.
This includes principle questions related to eigenvalue problems,
but also concrete applications such as improving
the performance of the polarisable continuum model,
a standard solvation model in electronic structure theory.
I see a good fit between our respective research backgrounds
and I am happy for this opportunity to extend my research horizon
and contribute to the research in Ben's group and the ACoM
over the next years.</p>
<p>During my time in Aachen Ben and I want to continue
to work on the numerical analysis and the development of mathematically-motivated
methods for density-functional theory (DFT).
One aspect we have in mind, for example,
is to port Ben's recent work for constructing good initial
guesses for the self-consistent iterations in molecular DFT (and Gaussian basis functions)
to plane-wave DFT.
As part of this research we will make use and extend the
<a href="https://dftk.org">density-functional toolkit (DFTK)</a>,
the density-functional theory code I started in Paris.
Beyond our work at the ACoM I expect DFTK and its suitability
for multidisciplinary research also to be helpful for reaching out to other
researchers in the mathematics, computer science and physics departments in Aachen.
In particular I see a good fit of DFTK
within the <a href="https://www.jara.org/en/research/center-for-simulation-and-data-sciences">JARA-CSD</a>,
a joint research initiative between RWTH Aachen and the Jülich research centre.</p>
<p>Having known Aachen already a little from my previous visits
in Ben's group I am very much looking forward to work here.
Not only is the city very pretty and welcoming,
but also the interdisciplinary orientation of RWTH Aachen resonates well with me.
I'm looking forward to the many interesting discussions to come
and to becoming part of strengthening the interdisciplinary links in Aachen,
while at the same time continuing to work
at the boundary of chemistry, physics and mathematics.</p>Gator: a Python-driven program for spectroscopy simulations2021-02-27T00:30:00+01:002021-02-27T00:30:00+01:00Michael F. Herbsttag:michael-herbst.com,2021-02-27:/2021-gator.html<p>A bit over a year ago we published our <a href="https://adc-connect.org">adcc code</a>.
In this work the aim was to develop a toolkit for computational spectroscopy methods
focused on rapid development and interactive hands-on usage
(see the <a href="https://michael-herbst.com/2019-adcc.html">blog article</a> for details).
Our target back then was to simplify method development involving
the …</p><p>A bit over a year ago we published our <a href="https://adc-connect.org">adcc code</a>.
In this work the aim was to develop a toolkit for computational spectroscopy methods
focused on rapid development and interactive hands-on usage
(see the <a href="https://michael-herbst.com/2019-adcc.html">blog article</a> for details).
Our target back then was to simplify method development involving
the algebraic-diagrammatic construction approach (ADC)
to compute excited states energies and properties.
ADC has been a research focus both of myself as well as the group of Andreas Dreuw
and the ADC family of methods have proven in the past to be greatly suited
for describing photochemistry and spectroscopic results.</p>
<p>Employing mainly thread-based parallelism and (apart from our recent inclusion
of <a href="https://github.com/ilyak/libxm">libxm</a>) basically no options for swapping stored
tensors to disk, adcc is naturally restricted to problems that fit
into the main memory of a single cluster node. This is fine for developing and
testing new ADC methods, but can be limiting for employing ADC methods
in practice: The code can currently only treat small-sized to medium-sized molecules.</p>
<p>In parallel to adcc we therefore started working on the <strong>Gator project</strong>
in collaboration with the groups of Patrick Norman
and Zilvinas Rinkevicius (both KTH Stockholm).
We now release in a first version.
Apart from an interface to adcc, Gator features a
response library capable of the <strong>complex polarisation propagator (CPP) approach</strong>
for simulating properties such as excited-states polarisabilities
or enabling a direct computation of spectra including broadening.
Additionally it contains a newly developed ADC(2) module
with <strong>MPI-based distributed computing</strong> capabilities.
For this the integral driver of the <a href="https://veloxchem.org">Veloxchem</a> code from KTH
is used, which allows the ADC(2) computation to be performed in a direct fashion
(i.e. without storing the two-electron-integral tensor).
This makes <strong>ADC(2) simulations</strong> in Gator more memory efficient and allows them
to be <strong>distributed over a few cluster nodes</strong>.
In this publication we provide an overview of Gator's current
capabilities. The full abstract reads</p>
<blockquote>
<p>The Gator program has been developed for computational spectroscopy and
calculations of molecular properties using real and complex propagators at the
correlated level of wave function theory. At present, the focus lies on methods
based on the algebraic diagrammatic construction (ADC) scheme up to third-order
of perturbation theory. A Fock matrix-driven implementation of the second-order
ADC method for excitation energies has been realized with an underlying hybrid
MPI/OpenMP parallelization scheme suitable for execution in high-performance
computing cluster environments. With a modular and object-oriented program
structure written in a Python/C++ layered fashion, Gator enables, in addition,
time-efficient prototyping of novel scientific approaches as well as
interactive notebook-driven training of students in quantum chemistry.</p>
</blockquote>CESMIX TST meeting: DFTK.jl: A multidisciplinary Julia code for density-functional theory development2021-02-04T20:00:00+01:002021-02-04T20:00:00+01:00Michael F. Herbsttag:michael-herbst.com,2021-02-04:/2021-cesmix-tst.html<p>These past two days I have participated
in the Tri-Lab Support Team (TST) meeting of the CESMIX, the newly founded
<a href="https://computing.mit.edu/CESMIX/">Center for the Exascale Simulation of Material Interfaces in Extreme Environments</a>
at the Massachusetts Institute of Technology.
Within the next few years the idea of the CESMIX is to develop …</p><p>These past two days I have participated
in the Tri-Lab Support Team (TST) meeting of the CESMIX, the newly founded
<a href="https://computing.mit.edu/CESMIX/">Center for the Exascale Simulation of Material Interfaces in Extreme Environments</a>
at the Massachusetts Institute of Technology.
Within the next few years the idea of the CESMIX is to develop
a multi-level simulation stack all the way up from DFT over MD to
flow simulations to be able to discover novel materials suitable
for extremely high temperatures under atmospheric conditions.
The prototypical application for such materials would be
heat shields for example in space crafts returning to earth or supersonic planes.</p>
<p>One novel aspect of the project is to include
progress from modern compiler techniques and programming language design
when building the software stack.
In particular the challenge is that multiple codes will be involved in the project
that feature a large variety of programming languages (FORTRAN, C++, Julia, python, ...).
On top of that one goal is to keep track of simulation errors
using uncertainty quantification (UQ) and use that insight
to construct a multi-fidelity workflow.
In such an approach the data generation for the simulation does not only employ
a single accurate model,
but in fact features multiple simulation layers
based on cheaper and cruder models as well as more costly and accurate ones.
Using the deduced knowledge of the error one can dynamically switch between these models
and reach a compromise between accuracy and computational cost,
but in a way that the result still has the quality to be comparable to experiments.
As the employed fidelity layers the CESMIX project
targets classical molecular dynamics or ab initio molecular dynamics
with various kinds of density-functional theory (DFT) methods
... and this is the angle of my involvement in the project.</p>
<p>In particular using our density-functional toolkit (<a href="https://dftk.org">DFTK</a>)
the idea is to be able to quickly prototype parts of the workflow.
Then, building on DFTK's design as a multi-disciplinary platform
(see my <a href="/tag/dftk.html">related blog articles</a>),
we want to start incorporating new techniques
(GPU platforms, UQ, multi-fidelity) to see how they could fit.
I am excited about the opportunity to contribute to a project,
which shares a lot in philosophy to DFTK itself.
In particular I am looking forward to seeing how DFTK
will play out in a real-world research scenario
for connecting the needs from the
modelling side with the approaches of the computer science folks.</p>
<p>With respect to the TST meeting, I briefly gave an overview about DFTK
to show where we are. Slides and a short demo are attached below.</p>
<table>
<thead>
<tr>
<th>Link</th>
</tr>
</thead>
<tbody>
<tr>
<td><a href="https://michael-herbst.com/talks/2021.02.03_dftk_tst_cesmix.pdf">DFTK.jl: A multidisciplinary Julia code for density-functional theory development</a> (Slides)</td>
</tr>
<tr>
<td><a href="https://michael-herbst.com/talks/2021.02.03_dftk_tst_cesmix_demo.tar.gz">Benchmarks and DFTK demo</a> (Tarball)</td>
</tr>
</tbody>
</table>High-throughput density-functional theory calculations: An interdisciplinary challenge2020-12-23T18:00:00+01:002020-12-23T18:00:00+01:00Michael F. Herbsttag:michael-herbst.com,2020-12-23:/2020-kl-scicomp.html<p>Last Thursday I was invited to give a virtual talk at the <a href="https://www.scicomp.uni-kl.de/teaching/scientific-computing-seminar/?event_id1=2613">Scientific Computing
Seminar</a>
of working group of Prof. Nicolas Gauger at TU Kaiserslautern. Since the research
in Prof. Gauger's group mostly concerns topics which are not directly related to
electronic structure theory and density-functional theory (DFT), I chose …</p><p>Last Thursday I was invited to give a virtual talk at the <a href="https://www.scicomp.uni-kl.de/teaching/scientific-computing-seminar/?event_id1=2613">Scientific Computing
Seminar</a>
of working group of Prof. Nicolas Gauger at TU Kaiserslautern. Since the research
in Prof. Gauger's group mostly concerns topics which are not directly related to
electronic structure theory and density-functional theory (DFT), I chose to present my current research
from a rather broad and introductory angle.
Main focus of my talk was thus to hint at the interdisciplinary challenges
arising in high-throughput methods in DFT simulations,
followed by a summary of a few of my recent projects in the field.</p>
<p>I was glad for the opportunity to spread the word about the difficulties
with high-throughput methods in DFT.
Firstly because I think it is an absolutely fascinating topic,
but secondly because it is one where input from fields beyond the standard "culprits"
of chemistry, physics and materials science
is beneficial to solve the upcoming problems.
In fact exactly this hope to get other fields and people with non-standard backgrounds
involved was one of the driving forces behind our <a href="https://dftk.org">density-functional toolkit</a>
DFT code, which I also briefly presented.</p>
<p>I hope that my talk got some of the audience more interested in DFT
and I look forward to continue the discussions at a later point,
hopefully meeting the Gauger group in person in Kaiserslautern.</p>
<table>
<thead>
<tr>
<th>Link</th>
</tr>
</thead>
<tbody>
<tr>
<td><a href="https://michael-herbst.com/talks/2020.12.17_scientific_computing_kl.pdf">High-throughput density-functional theory calculations: An interdisciplinary challenge</a> (Slides)</td>
</tr>
</tbody>
</table>Challenges and prospects of a posteriori error estimation in density-functional theory2020-10-21T18:00:00+02:002020-10-21T18:00:00+02:00Michael F. Herbsttag:michael-herbst.com,2020-10-21:/2020-braunschweig-error.html<p>Last week Wednesday I was invited to give a talk
at the group seminar of the <a href="https://www.tu-braunschweig.de/pci/agjacob">AG Christoph Jacob</a>
at TU Braunschweig, Germany.
Christoph was especially interested in our
<a href="https://michael-herbst.com/2020-nonscf-error-estimates.html">recent publication on a posteriori error estimation</a>
in Kohn-Sham problems and so I decided to use the opportunity to give a …</p><p>Last week Wednesday I was invited to give a talk
at the group seminar of the <a href="https://www.tu-braunschweig.de/pci/agjacob">AG Christoph Jacob</a>
at TU Braunschweig, Germany.
Christoph was especially interested in our
<a href="https://michael-herbst.com/2020-nonscf-error-estimates.html">recent publication on a posteriori error estimation</a>
in Kohn-Sham problems and so I decided to use the opportunity to give a broad introduction
into the topic.
Since the main audience for my talk were chemists I motivated
our work from the context
of high-throughput density-functional theory calculations,
which are becoming more and more of interest in practice.
In the second part of my talk I tried to lay out the basic ideas and challenges
of a posteriori error estimation and what are the key aspects to understand
and overcome if one wants to provide a useful error bound for a particular problem
(see also <a href="https://michael-herbst.com/2020-nonscf-error-estimates.html">this article for more details</a>).
In the last part of the talk I turned the attention specifically to our
recent contribution, discussing error estimates
in the context of simple Kohn-Sham problems.
Many of the delicate details of our work I did not touch upon,
but nevertheless this last section of my talk gives a very good overview of our approach,
our main underlying assumptions and our results.
While we have focused solely on plane-wave basis sets in our work,
I briefly hinted whether extensions to other basis sets with similar methods
can be achieved.</p>
<p>Unfortunately the whole "visit" to Braunschweig was completely virtual,
which really was a shame as I would have loved to return to the city
and spend a day with the group.
Despite these circumstances though the discussions with Christoph
and his group did not not really fall short.
During the day I had many video call sessions where I had ample
opportunity to discuss with PhD students about their projects
and in this way get a good idea of the interesting research going on in Braunschweig.
As that's typically my favourite part about visiting a group
I'm very happy this worked out so well (thanks to everyone who I managed to talk to!)
and I already look forward to
properly meeting everyone in person
when the usual conference schedules are back in place.</p>
<p>The slides from my talk are attached below.</p>
<table>
<thead>
<tr>
<th>Link</th>
</tr>
</thead>
<tbody>
<tr>
<td><a href="https://michael-herbst.com/talks/2020.10.14_aposteriori_error.pdf">Challenges and prospects of a posteriori error estimation in density-functional theory</a> (Slides)</td>
</tr>
</tbody>
</table>Moansi: Inhomogeneous preconditioning for density-functional theory2020-09-30T22:00:00+02:002020-09-30T22:00:00+02:00Michael F. Herbsttag:michael-herbst.com,2020-09-30:/2020-moansi.html<p>Last week (24th and 25th September) I attended the 4th annual meeting
of the <a href="https://moansi.wixsite.com">Moansi</a>
(Modelling, analysis and simulation of Molecular Systems)
work group of the <a href="gamm-ev.de">GAMM</a>.
I already attended the Moansi meeting
<a href="https://michael-herbst.com/2019-munich-moansi-domcke.html">last year</a>,
where I very much enjoyed both the broad range of talks
at the interdisciplinary border …</p><p>Last week (24th and 25th September) I attended the 4th annual meeting
of the <a href="https://moansi.wixsite.com">Moansi</a>
(Modelling, analysis and simulation of Molecular Systems)
work group of the <a href="gamm-ev.de">GAMM</a>.
I already attended the Moansi meeting
<a href="https://michael-herbst.com/2019-munich-moansi-domcke.html">last year</a>,
where I very much enjoyed both the broad range of talks
at the interdisciplinary border of maths, chemistry and physics
as well as the familial atmosphere with lively and stimulating discussions.
In the same spirit I was looking forward to this year's addition,
where in light of the global pandemic, however, the workshop had to be virtual.
Nevertheless I was able to take away many stimulating impressions
from the two days of the meeting and I already look forward
to next year, where we hopefully meet again in person.</p>
<p>During the meeting I myself presented our
<a href="https://michael-herbst.com/2020-ldos-preconditioning.html">recently published work</a>
on an LDOS-based SCF preconditioner,
which is especially suitable for DFT calculations in inhomogeneous systems
(metallic slabs or clusters).
See <a href="https://michael-herbst.com/2020-ldos-preconditioning.html">this blog article</a>
and my slides for details.</p>
<table>
<thead>
<tr>
<th>Link</th>
</tr>
</thead>
<tbody>
<tr>
<td><a href="https://michael-herbst.com/talks/2020.09.24_ldos_preconditioning.pdf">Black-box inhomogeneous preconditioning for density-functional theory</a> (Slides)</td>
</tr>
</tbody>
</table>Faraday Discussions: New horizons in density functional theory2020-09-15T15:00:00+02:002020-09-15T15:00:00+02:00Michael F. Herbsttag:michael-herbst.com,2020-09-15:/2020-faraday-dft.html<p>Following the submission of our
<a href="https://michael-herbst.com/2020-nonscf-error-estimates.html">paper on a posteriori error estimation in the Kohn-Sham equations</a>
a few months ago
I was recently invited to present our work at the
<a href="https://www.rsc.org/events/detail/37136/new-horizons-in-density-functional-theory-faraday-discussion">Faraday Discussions on New horizons in density functional theory</a>.
Being amongst speakers such as Kieron Burke, Andreas Savin or Weitao Yang …</p><p>Following the submission of our
<a href="https://michael-herbst.com/2020-nonscf-error-estimates.html">paper on a posteriori error estimation in the Kohn-Sham equations</a>
a few months ago
I was recently invited to present our work at the
<a href="https://www.rsc.org/events/detail/37136/new-horizons-in-density-functional-theory-faraday-discussion">Faraday Discussions on New horizons in density functional theory</a>.
Being amongst speakers such as Kieron Burke, Andreas Savin or Weitao Yang
this was truly a great honour.</p>
<p>Even though the conference had to be virtual,
I enjoyed it very much, especially because of its very unusual format.
Unlike most other conferences where the presenting author typically does his thing
for like 30 minutes, followed by just a few questions,
the situation is completely reversed for the Faraday discussions.
Since we had to submit our paper already months in advance
(and this was shared with the other participants)
the content of my talk was already known to the audience.
The main chunk of the time at the conference was therefore allocated
to the discussion and not to the presentation.
My few slides therefore only briefly recap our work
and hint at the general motivation and outlook.
As I hoped our work did indeed stimulate an intense discussion
with interested and stimulating questions, which I really appreciated
(thanks to everyone who asked or commented).
As far as I understand both the paper as well a transcript of the discussion
will be part of the official Faraday Discussions conference proceedings,
which will be published by the Royal Society of Chemistry soon.
As per usual my slides are attached below.</p>
<table>
<thead>
<tr>
<th>Link</th>
</tr>
</thead>
<tbody>
<tr>
<td><a href="https://michael-herbst.com/talks/2020.09.03_faraday_error_nonscf_kohn_sham.pdf">A posteriori error estimation for the non-self-consistent Kohn-Sham equations</a> (Slides)</td>
</tr>
</tbody>
</table>Black-box inhomogeneous preconditioning for self-consistent field iterations in density-functional theory2020-09-04T00:30:00+02:002020-09-04T00:30:00+02:00Michael F. Herbsttag:michael-herbst.com,2020-09-04:/2020-ldos-preconditioning.html<p>For the past half a year or so Antoine Levitt and myself have been looking
at a particular tricky busyness for solid-state density-functional theory
(DFT) calculations,
namely how to design efficient self-consistent field (SCF) schemes for
large inhomogeneous systems.
I have already
<a href="https://michael-herbst.com/2020-emc2-dftk-scf-preconditioning.html">previously reported</a>
on this matter in a short …</p><p>For the past half a year or so Antoine Levitt and myself have been looking
at a particular tricky busyness for solid-state density-functional theory
(DFT) calculations,
namely how to design efficient self-consistent field (SCF) schemes for
large inhomogeneous systems.
I have already
<a href="https://michael-herbst.com/2020-emc2-dftk-scf-preconditioning.html">previously reported</a>
on this matter in a short talk at the seminar
of our interdisciplinary working group,
but now our results have reached a stage suitable for publication.</p>
<p>The underlying problem we are tackling in our work is that
<strong>for large systems</strong>, meaning increased sizes of the unit cell,
the <strong>SCF iterations become harder and harder</strong> to solve.
Mathematically speaking the (spectral) condition number of the
fixed point iterations underlying the SCF procedure increase
rather drastically in such cases, leading to very slow convergence.
For example in aluminium the number of iterations required to converge an SCF
with a damped iteration scheme (the most simple one)
increases quadratically with the system size.
This quickly makes calculations intractable and multiple
more sophisticated approaches have therefore been developed over the years.
As is detailed in our work there are mainly two orthogonal directions of attack.
The first is to black-box "accelerate" the convergence by using the so-called
Anderson (or Pulay or DIIS) scheme. This reduces the growth of iterations
with system size from quadratic to linear (in the aluminium example),
which is a good start.
The second approach is to use a <strong>carefully designed preconditioner</strong> for the SCF
in order to tame the SCF iterations.
Figuratively speaking this approach makes use of known physics to prevents the SCF
from looking in the wrong direction for the solution.
If done right, meaning that the physics modelled by the preconditioner
fits the system at hand,
this allows the <strong>SCF iteration count</strong> to become <strong>independent of system size</strong>.
This latter approach is clearly the more important route to cure the problem,
but both approaches are orthogonal and are therefore typically combined
in order to get the fastest convergence.</p>
<p>Now what does it mean the preconditioner has to fit the system?
As we detail in the paper, the convergence of an SCF
is intimately linked to the dielectric behaviour of the material one models with the SCF.
For <strong>homogeneous cases</strong> (i.e. bulk insulators, metals and semiconductors)
people have devised <strong>very good models</strong> for their dielectric behaviour
and have used them to construct preconditioners.
As is well known (and confirmed by our study) these models show exactly
the desirable property of a size-independent iteration count.
The caveat is only that metals, insulators
and semiconductors have deviating dielectric properties, meaning
that each of these calls for a <em>different</em> preconditioning strategy.
In return this means that <strong>heterogeneous cases</strong> where multiple of these materials
are combined are <strong>difficult to treat in practice</strong> because none of the
bulk recipes fully fit.</p>
<p>The main aim of <strong>our work</strong> was therefore to <strong>design
a preconditioner</strong> which automatically and <strong>locally adapts</strong> to the system at hand,
meaning that for heterogeneous cases it treats metallic regions
like metals, insulating regions like insulators and so on.
As we demonstrate with a number of test cases
our preconditioner is able to do this completely <strong>black-box and parameter-free</strong>
and performs well also for <strong>large heterogeneous systems</strong>.
This is in contrast to previous approaches
to tackle this problem, which were not as general
as our approach and sometimes required complex hand-tuning of the involved parameters.</p>
<p>While our preconditioner solves the problem of <strong>efficiently treating</strong> cases
like <strong>metallic slabs, metal clusters</strong> and basically any combination of
metallic parts, <strong>insulators and vacuum</strong>,
it is not fully capable of distinguishing insulators and semiconductors.
We show that this can be cured at the expense of introducing
another parameter to our algorithm.
This works, but is not completely satisfactory to us.
Part of our ongoing work is therefore to
extend our scheme to treat mixed systems involving semiconductors as well.
Another aspect we have so far neglected is spin,
which is a constant annoyance for converging SCFs.
Having a solid dielectric model as we propose it,
also opens way to adapt preconditioning to each spin component
differently. We hope to use this in the future to
tackle convergence issues with spin
in a hopefully more rigorous way than this is done to date.</p>
<p>The full abstract of our paper reads</p>
<blockquote>
<p>We propose a new preconditioner for computing the self-consistent problem in
Kohn-Sham density functional theory, based on the local density of states. This
preconditioner is inexpensive and able to cure the long-range charge sloshing
known to hamper convergence in large, inhomogeneous systems such as clusters
and surfaces. It is based on a parameter-free and physically motivated
approximation to the independent-particle susceptibility operator, appropriate
for both metals and insulators. It can be extended to semiconductors by using
the macroscopic electronic dielectric constant as a parameter in the model. We
test our preconditioner successfully on inhomogeneous systems containing
metals, insulators, semiconductors and vacuum.</p>
</blockquote>DFTK: A Julian approach for simulating electrons in solids2020-07-31T22:00:00+02:002020-07-31T22:00:00+02:00Michael F. Herbsttag:michael-herbst.com,2020-07-31:/2020-juliacon.html<p>Since last Friday I have been attending <a href="https://juliacon.org/">JuliaCon</a>,
the annual conference for the Julia language.
Naturally given the current situation the event did not take place "on location",
but was instead converted into a virtual event.
Albeit the different feel compared to a real-life conference
the organisers did a very …</p><p>Since last Friday I have been attending <a href="https://juliacon.org/">JuliaCon</a>,
the annual conference for the Julia language.
Naturally given the current situation the event did not take place "on location",
but was instead converted into a virtual event.
Albeit the different feel compared to a real-life conference
the organisers did a very good job to maintain the social component into the event.
Talks were pre-recorded and speakers available in a chat room to discuss
during and after the presentation in written form.
Birds of feather brainstorming sessions took place using audio discussions
and at the end of every day there was a Gather Town virtual social,
where one could videochat with fellow attendees by meeting up in a
beautifully animated world, where each attendee was represented by a tiny avatar.</p>
<p>Apart from attending and listing to the great talks about the Julia
language and its plenty of applications,
I also had the chance to actively participate
by giving a lecture about our package <a href="https://dftk.org">DFTK.jl</a>.
While I have presented on DFTK <a href="/tag/dftk.html">a few times before</a>
in front of expert audiences of the field,
it was really the first time I presented DFTK as a released package
to the broader Julia audience.
That meant that I could, for once, give up on my usual storyline
where I try and convince people into using Julia
and instead focus on providing insight into the fascinating
challenges of electronic-structure theory and
how DFTK and Julia are ideal tools to tackle these.</p>
<p>In my talk I start easy by a general introduction
into electronic-structure theory illustrating why an exact solution
for electronic structures in molecules or solids is just not possible
in realistic timeframes.
Therefore one needs to live with approximate models,
one example being density-functional theory (DFT), which we use in DFTK.
As I detail in the talk an almost immediate consequence
of the complexity of the problem
is that advances in electronic-structure theory
can typically only be realised if multiple disciplines join forces.
An interdisciplinary project, however,
brings some practical problems
just quite frankly due to the fact that different fields have different
approaches when tackling a problem.
Being able to support such multidisciplinary
motions in a common software platform for DFT,
is one of the key aims of DFTK.</p>
<p>Related to this point we wanted DFTK to have
a low entrance barrier for novel researchers.
As time and money in research is tight programs should be easy to use
and code simple and self-explanatory,
such that new PhD students or researchers from foreign fields
do not have a tough time to get started.
In my talk I mention a few recent projects
(an undergrad internship and a master project),
where a noteworthy result could be achieved
albeit students had little prior experience
with neither Julia nor electronic-structure theory.
A similar success story emphasising our ability to rapidly realise
novel ideas in DFTK
includes our recently published
<a href="https://michael-herbst.com/2020-nonscf-error-estimates.html">Faraday paper</a>,
where it only took 10 weeks from starting the project to submitting the paper.</p>
<p>Lastly, I discussed challenges arising
from the so-called high-throughput screening methods,
which are recently gaining popularity
in computational materials design.
In this particular research direction
algorithms need to be particularly robust and tunable to find a sweet spot
between accuracy and computational cost.
This demands extremely stable and reliable algorithms,
which poses interesting mathematical problems
in numerical analysis and e.g. with respect to
designing estimators for discretisation error.
Especially in this area of application-oriented
mathematical research we expect DFTK to be a handy tool in the future.</p>
<p>If you are interested in the full story
a recording of the talk is available
<a href="https://www.youtube.com/watch?v=-RomkxjlIcQ">on youtube</a>.</p>
<table>
<thead>
<tr>
<th>Link</th>
</tr>
</thead>
<tbody>
<tr>
<td><a href="https://michael-herbst.com/talks/2020.07.29_juliacon_dftk.pdf">DFTK: A Julian approach for simulating electrons in solids</a> (Slides)</td>
</tr>
<tr>
<td><a href="https://www.youtube.com/watch?v=-RomkxjlIcQ">Youtube recording of the talk</a></td>
</tr>
</tbody>
</table>SCF preconditioning for mixed systems2020-06-26T18:00:00+02:002020-06-26T18:00:00+02:00Michael F. Herbsttag:michael-herbst.com,2020-06-26:/2020-emc2-dftk-scf-preconditioning.html<p>Fortunately the general lockdown due to the Corona pandemic slowly starts
to ease around Paris as well. While basically all seminars are only virtual
it is good to see old procedures and habits to slowly return.
From my end I gave the first talk after the forced break today
in …</p><p>Fortunately the general lockdown due to the Corona pandemic slowly starts
to ease around Paris as well. While basically all seminars are only virtual
it is good to see old procedures and habits to slowly return.
From my end I gave the first talk after the forced break today
in the <a href="https://erc-emc2.eu/group-meetings/">EMC2 group meeting</a>.</p>
<p>Since it was the first time discussing DFTK in the EMC2 synergy group
I decided to talk about it taking the angle of tackling an actual research problem.
After presenting briefly DFT methods and DFTK in the first half of the talk,
I therefore focused on one of my ongoing projects with Antoine Levitt,
namely constructing better preconditioners for the self-consistent field (SCF)
iterations in mixed systems.
What is meant by mixed systems are systems where locally differing dielectric
properties are found, i.e. where some parts of the material are insulating,
others may be metallic or semiconductors.
Since the dielectric properties are closely related to the spectrum of the SCF
fixed-point map, they therefore also control the convergence properties of SCF
procedures.
For metals and (to a minor extent) semiconductors simple SCF procedures,
where one just applies the SCF cycle over and over
require extremely small step sizes (i.e. small damping values).
As a result the SCF converges only very slowly.
The remedy is to precondition the spectrum of the SCF map itself
by using so-called mixing techniques.
In state-of-the-art approaches these are usually material-specific,
i.e. different mixings are used for insulators, metals or semiconductors.
This is fine for bulk materials, but fails in case of mixed systems,
since one has to globally select a single approach.
Our recent work has been to investigate the spectrum of the SCF map
and to try and construct a preconditioner which locally adapts
and as a result is able to properly treat mixed systems as well.
The results I presented today are, however, not yet final and more investigations
are to be underdone for our approach to work as reliable as we want.</p>
<table>
<thead>
<tr>
<th>Link</th>
</tr>
</thead>
<tbody>
<tr>
<td><a href="https://michael-herbst.com/talks/2020.06.26_dftk_scf_emc2.pdf">SCF preconditioning for mixed systems: A DFTK case study</a> (Slides)</td>
</tr>
<tr>
<td><a href="https://michael-herbst.com/talks/2020.06.26_dftk_scf_emc2_tutorials.ipynb">A few DFTK examples</a> (Jupyter notebook)</td>
</tr>
<tr>
<td><a href="https://michael-herbst.com/talks/2020.06.26_dftk_scf_emc2_preconditioners.ipynb">SCF preconditioners in 1D</a> (Jupyter notebook)</td>
</tr>
</tbody>
</table>First release of the density-functional toolkit (DFTK)2020-06-05T10:00:00+02:002020-06-05T10:00:00+02:00Michael F. Herbsttag:michael-herbst.com,2020-06-05:/2020-dftk-updates.html<p>After we released a <a href="https://michael-herbst.com/2019-release-dftk.html">preliminary snapshot</a> of DFTK last year
our focus in the first half of this year was on using it
<a href="https://michael-herbst.com/2020-nonscf-error-estimates.html">for some new science</a>.
Recently, however, we got back into polishing
<a href="https://dftk.org">our code base</a> and <a href="https://docs.dftk.org">our documentation</a>
in order to get DFTK ready for a wider audience …</p><p>After we released a <a href="https://michael-herbst.com/2019-release-dftk.html">preliminary snapshot</a> of DFTK last year
our focus in the first half of this year was on using it
<a href="https://michael-herbst.com/2020-nonscf-error-estimates.html">for some new science</a>.
Recently, however, we got back into polishing
<a href="https://dftk.org">our code base</a> and <a href="https://docs.dftk.org">our documentation</a>
in order to get DFTK ready for a wider audience.
Today I am proud to announce that DFTK 0.1.0
has been accepted into the <a href="https://juliahub.com/ui/Packages/DFTK/zs81u/0.1.0">Julia Package repository</a>,
such that the package can now be <a href="https://docs.dftk.org/dev/guide/installation/">readily installed</a>
within the Julia ecosystem.</p>
<p>Let me take this opportunity to recapitulate on DFTK:
I started the code with <a href="antoine.levitt.fr/">Antoine Levitt</a>
about <a href="https://michael-herbst.com/2019-dftk-julia-paris.html">a year ago</a> when I moved to Paris.
What we had in mind was to create a simple platform for methodological developments
in density functional theory (DFT).
Clearly our code should support the interdisciplinary requirements of the field,
where advances are often the result from devising
chemically and physically sound models, using mathematical insight
for suggesting stable algorithms
and then scaling them up to the high-performance regime.
This means that we would need both
(a) the flexibility to mix and match models and numerical approaches
by keeping the code high-level and similar to a scripting language
and (b) access to the usual tricks (vectorisation, GPUs, threading, distributed computing)
to tweak performance down to the metal.</p>
<p>In <a href="https://julialang.org/">Julia</a> we found a language
which suits these aims perfectly.
This is illustrated by the fact that after only a good year of
development we already support a <a href="https://docs.dftk.org">sizable number of features</a>
in only about 5k lines of source code.
Right now the focus of DFTK is on DFT ground-state simulations for solids
(LDA/GGA in a plane-wave basis with GTH pseudopotentials)
with more to come.
Special care is taken to have a simple and clean codebase,
well-commented and suitable for teaching or extensions (other models, basis, etc.).
DFT is not hard-coded, and other similar models can be
computed with DFTK (<a href="https://docs.dftk.org/dev/examples/gross_pitaevskii_2D">for instance</a>,
the 2D Gross-Pitaevskii equation with a magnetic field).
Nevertheless, the performance is comparable with that of established plane-wave DFT
codes, usually within a factor of 2.
DFTK is fully multithreaded, although not distributed (yet).
We also include interfaces with various codes
(ASE, pymatgen, abipy...) for easy workflows and to integrate to
the world beyond the Julia ecosystem.
See for example the <a href="https://github.com/mfherbst/asedftk">asedftk</a> python package,
which integrates DFTK into the atomistic simulation environment.</p>
<p>The code is of course fully open source
and <a href="https://docs.dftk.org/dev/guide/installation/">installation</a> is easy.
Since it is intended as a platform for
multidisciplinary collaboration, we welcome any question, suggestion or addition.
Feel free to get in touch by <a href="https://github.com/JuliaMolSim/DFTK.jl/issues">opening an issue</a>
at any time.</p>Recent developments in adcc2020-06-03T19:00:00+02:002020-06-03T19:00:00+02:00Michael F. Herbsttag:michael-herbst.com,2020-06-03:/2020-adcc-news.html<p>Since the publication of the
<a href="https://michael-herbst.com/2019-adcc.html">adcc paper</a>
a few months back, there are a few updates to report briefly:</p>
<ul>
<li>adcc can now be interactively tried in the browser at
<a href="https://try.adc-connect.org">try.adc-connect.org</a>
using the infrastructure from the binder project.</li>
<li>Binary installation of adcc is now available via conda for Linux …</li></ul><p>Since the publication of the
<a href="https://michael-herbst.com/2019-adcc.html">adcc paper</a>
a few months back, there are a few updates to report briefly:</p>
<ul>
<li>adcc can now be interactively tried in the browser at
<a href="https://try.adc-connect.org">try.adc-connect.org</a>
using the infrastructure from the binder project.</li>
<li>Binary installation of adcc is now available via conda for Linux and MacOS,
see <a href="https://adc-connect.org/installation.html">the adcc documentation</a>.</li>
<li>Calculation of rotatory strengths at all ADC levels.</li>
<li>adcc is now fully integrated
into the <a href="https://psicode.org/">Psi4</a> quantum chemistry package.
This means that ADC calculations in adcc can now be directly started
from within the Psi4 ecosystem,
including Psi4's python frontend and input files.
This effectively equips Psi4 with all ADC capabilities adcc offers.
See some details in the <a href="https://doi.org/10.1063/5.0006002">recent Psi4 paper</a>.</li>
<li>Tensor evaluations in adcc are now lazy, which means that complex tensor
evaluation expressions can be coded up in python without being evaluated.
Only once results are needed the complete expression is evaluated
in a batch using the underlying linear algebra frameworks.</li>
</ul>Quantifying the error of the core-valence separation approximation2020-05-13T00:30:00+02:002020-05-13T00:30:00+02:00Michael F. Herbsttag:michael-herbst.com,2020-05-13:/2020-cvs-relaxation.html<p>After only a few months of publishing our <a href="https://michael-herbst.com/2019-adcc.html">adcc paper</a>,
which introduces the novel algebraic-diagrammatic construction (ADC) code <a href="https://adc-connect.org">adcc</a>,
the package has already found mention in a number of related articles.
One example is a manuscript on updates in the
<a href="https://doi.org/10.1063/5.0006002">Psi4 quantum chemistry package</a>,
which features adcc as one of …</p><p>After only a few months of publishing our <a href="https://michael-herbst.com/2019-adcc.html">adcc paper</a>,
which introduces the novel algebraic-diagrammatic construction (ADC) code <a href="https://adc-connect.org">adcc</a>,
the package has already found mention in a number of related articles.
One example is a manuscript on updates in the
<a href="https://doi.org/10.1063/5.0006002">Psi4 quantum chemistry package</a>,
which features adcc as one of the highlighted community modules.
As the paper discusses, the close integration of adcc into the Psi4
ecosystem now allows to start ADC calculations in adcc directly
form Psi4's python frontend and its input files.
This effectively extends Psi4 by all ADC capabilities adcc offers.
Another example is a
paper on <a href="https://doi.org/10.26434/chemrxiv.12179289.v1">computing complex polarisabilities for excited states</a>,
which especially emphasises the importance adcc has played
for simplifying the implementation of the method.</p>
<p>On my end, I recently conducted a study with Thomas Fransson
on the <strong>error of the core-valence separation (CVS) approximation</strong>.
This approximation is very important for the simulation
of <strong>X-ray absorption spectra</strong> using accurate wave-function methods
like coupled-cluster or ADC.
In the literature the error of the CVS approximation
is widely accepted to be negligible compared to the error with respect to experiment.
This statement is, however, based on previous investigations of the CVS error,
which were limited by the methodologies to only small, non-representative basis sets.</p>
<p>In our work we present an iterative <strong>post-processing scheme</strong> in the ADC context,
which is able to <strong>undo the CVS approximation</strong> and remove the CVS error.
Our procedure is still basic (essentially just Rayleigh-Quotient iteration),
but it allowed us to study the CVS error
for a much larger range of systems and basis sets.
This includes augmented and/or core-polarised triple-zeta basis sets
as well as other bases prominent in the community to carry
out simulations of core-excitations and X-ray spectra
(read <code>6-311++G**</code> and variants).
Based on a representative compounds from elements of the second and third
period we managed to confirm that the CVS error is not only small compared
to experiment but additionally the spread across elements and compounds
is even smaller, such that the impact on energy differences is small, too.
This is an important finding, since most aspects of chemistry (spectroscopy,
reaction barrier heights, thermochemistry) are dominated by energy differences
and while relative errors in energies might be small,
relative errors in energy differences can be much larger if the error spread
across compounds is not uniform.</p>
<p>In particular our work identified the main contributions to the CVS error
to originate from two classes of couplings,
which are neglected by the CVS approximation
and which moreover contribute with opposite sign.
We demonstrate that basis sets providing a balanced description
of core and valence regions of the electron density are also capable
of describing these neglected couplings in a balanced fashion,
thus providing not only a good description
of the core-excitation process, but also a small CVS error
in terms of absolute value and spread.
Based on these findings we were able to conclude that especially
tight polarising functions are key for describing core-excitations.
Along our study we suggest appropriate modifications
for the popular <code>6-311++G**</code> basis to reduce its CVS error.
The full abstract of our paper reads</p>
<blockquote>
<p>For the calculation of core-excited states probed through X-ray absorption
spectroscopy, the core-valence separation (CVS) scheme has become a vital tool.
This approach allows to target such states with high specificity,
albeit introducing an error.
We report the implementation of a post-processing step
for CVS excitations obtained within the algebraic-diagrammatic construction
scheme for the polarisation propagator (ADC),
which removes this error.
Based on this we provide a detailed analysis of the CVS scheme,
identifying its accuracy
to be dominated by an error balance between two neglected couplings, one between
core and valence single excitations and
one between single and double core excitations.
The selection of the basis set is shown to be vital for a proper description
of both couplings, with tight polarising functions being necessary
for a good balance of errors.
The CVS error is confirmed to be stable across multiple systems,
with an element-specific spread of only about ±0.02 eV.
A systematic lowering of the CVS error by 0.02-0.03 eV is
noted when considering excitations to extremely diffuse states, emulating ionisation.</p>
</blockquote>A posteriori error estimation for the non-self-consistent Kohn-Sham equations2020-04-28T16:30:00+02:002020-04-28T16:30:00+02:00Michael F. Herbsttag:michael-herbst.com,2020-04-28:/2020-nonscf-error-estimates.html<p>After about a year of work on the <a href="https://dftk.org">density-functional toolkit</a> (DFTK)
we finally started using the code for some new science.
Given the mathematics background of the CERMICS
the first two DFTK-related articles were not so much about large-scale applications,
but rather deal some fundamental questions of
Kohn-Sham density-functional theory …</p><p>After about a year of work on the <a href="https://dftk.org">density-functional toolkit</a> (DFTK)
we finally started using the code for some new science.
Given the mathematics background of the CERMICS
the first two DFTK-related articles were not so much about large-scale applications,
but rather deal some fundamental questions of
Kohn-Sham density-functional theory (DFT).
The <a href="https://arxiv.org/abs/2004.09088">first submission</a>
provides a detailed analysis of self-consistent iterations
and direct-minimisation approaches for solving equations such as DFT.
My main focus in the past weeks, however,
was rather the second project,
an invited submission for the <strong>Faraday discussions</strong>
<a href="https://www.rsc.org/events/detail/37136/new-horizons-in-density-functional-theory-faraday-discussion"><strong>New horizons in density functional theory</strong></a>,
which are to take place in September this year.</p>
<p>In this article we deal with <strong>numerical error</strong> in DFT calculations.
More precisely we want to address the question:
What is the remaining error in the result obtained by a particular
DFT simulation, which has already been performed?</p>
<p>This is naturally a rather broad statement, which cannot realistically
be addressed in the scope of a single article.
As we detail we only address the question
of bounding the error of a particular quantity of interest,
namely band energies near the Fermi level. Other quantities,
such as the forces or the response to an external perturbation,
might benefit from our ideas, but will need extensions on top.
Also one needs to keep in mind that
there are plenty of sources for error in a DFT calculation,
including:</p>
<ol>
<li>The <strong>model error</strong> due to replacing the (almost) exact model
of the many-body Schrödinger equation by a reduced,
but more feasible model like DFT.</li>
<li>The <strong>discretisation error</strong> due to employing only finitely
many basis functions instead of solving analytically
in an infinite-dimensional Hilbert space.</li>
<li>The <strong>algorithmic error</strong> resulting from using non-zero convergence
tolerances in the eigensolvers as well as the SCF procedure.</li>
<li>The <strong>arithmetic error</strong> obtained by doing computation
only in finite-precision floating-point arithmetic.</li>
</ol>
<p>Of course in practice people often have a good ballpark idea
of the error of DFT as a method
or the errors obtained from a particular basis cutoff.
Rightfully one might ask, why one should go through
the effort of deriving provable bounds to the error
in a DFT result?
While there are surely many takes on this question,
I only want to highlight two aspects in this summary:</p>
<ul>
<li><strong>Educated guesses</strong> for convergence
parameters taken from experience
<strong>can fail</strong>. Typically they fail exactly
when interesting things happen in a system
and thus the usual heuristic breaks down.
In other words converging a simulation to
investigate what's going on becomes difficult
when it's most needed.
Detailed error analysis splitting up the error
during an SCF iteration
into contributions like the errors 1 to 4
(or even finer) can help to hint at the parameters
worth tweaking or can provide insight into which
error term behaves unusual in an SCF.</li>
<li>Thinking such aspects one step further,
a good bound to the individual terms even allows
to equilibrate sources of error during a running calculation.
The outlook of this idea would be a <strong>fully black-box
scheme for DFT</strong> calculations where typical convergence parameters
are automatically while the calculation progresses
in order to yield the cheapest path to a desired target accuracy.</li>
</ul>
<p>With respect to the errors 1 to 4 mentioned above one would of course
like to be able to provide an upper bound to each of them.
Unfortunately especially
obtaining an estimate for the model error is a rather difficult task.
In our work we have therefore concentrated on the errors 2 to 4
and moreover we only focused on Kohn-Sham models <em>without</em> self-consistency,
i.e. where none of the terms in the Hamiltonian depend on the density / orbitals.
It goes without saying that especially this last restriction
needs to be lifted to make our results useful in practice.
This angle we have left for future work.</p>
<p>Already at the stage of the current reduced model, however,
there are a few aspects to consider when finding an upper bound:</p>
<ul>
<li>We wanted our bound to be <strong>fully-guaranteed</strong>,
which means that we wanted to design a bound where
we are able to <em>prove</em> that the exact
answer must lie inside the bounds we give.
This means when we provide error bars for band energies it is
mathematically guaranteed to find the exact answer of
the full-dimensional (complete-basis set limit) calculation
at infinite accuracy and at zero convergence tolerances inside
our bound.</li>
<li>To be useful our bound should be (cheaply) <strong>computable</strong>,
because otherwise plainly checking the calculation at
vastly increased precision might end up being
the better option. Notice that our bound does require
to use a finer discretisation for some terms,
but this is only a one-shot <em>a posteriori</em> step and not an iterative one.</li>
<li>Ideally the bound should be <strong>sharp</strong>
meaning that the upper bound to the error we report
should not be too far off the true error.
Even better would be an <strong>optimal bound</strong>,
where we are as close to the true error as possible
(given proposed structure in the error estimate).
Such considerations are very important to not
end up with completely correct but also useless statements
like: "The error in the band energy is smaller than 10 million Hartree".</li>
</ul>
<p>Finding a balance between these three aspects is not always easy
and in our work we often take the pragmatic route
to obtain a simpler, albeit less sharp error.
Still, our bound is fully computable and allowed us,
for the first time, to report <strong>band structure diagrams</strong> of silicon
annotated with <strong>fully-guaranteed error bars</strong>
of combined discretisation, algorithm and arithmetic errors.
Details of our methodologies are given in the paper.
Its full abstract reads</p>
<blockquote>
<p>We address the problem of bounding rigorously the errors in the numerical solution of
the Kohn-Sham equations due to (i) the finiteness of the basis set, (ii) the
convergence thresholds in iterative procedures, (iii) the propagation of
rounding errors in floating-point arithmetic. In this contribution, we
compute fully-guaranteed bounds on the solution of the non-self-consistent
equations in the pseudopotential approximation in a plane-wave basis set. We
demonstrate our methodology by providing band structure diagrams of silicon
annotated with error bars indicating the combined error.</p>
</blockquote>1st GDR NBODY meeting in Lille: Applications of DFTK2020-01-28T19:00:00+01:002020-01-28T19:00:00+01:00Michael F. Herbsttag:michael-herbst.com,2020-01-28:/2020-gdr-meeting-lille.html<p>Earlier this month the <a href="https://gdr-nbody-lille.sciencesconf.org">first annual meeting</a>
of the French <a href="https://wiki.lct.jussieu.fr/gdrnbody">working group NBODY</a> (GDR NBODY)
took place in Lille.
Since the GDR NBODY is an
association of interdisciplinary scientists working on <em>N</em>-body problems
in chemistry an physics,
the about 80 participants came from a broad background ranging from
quantum …</p><p>Earlier this month the <a href="https://gdr-nbody-lille.sciencesconf.org">first annual meeting</a>
of the French <a href="https://wiki.lct.jussieu.fr/gdrnbody">working group NBODY</a> (GDR NBODY)
took place in Lille.
Since the GDR NBODY is an
association of interdisciplinary scientists working on <em>N</em>-body problems
in chemistry an physics,
the about 80 participants came from a broad background ranging from
quantum chemistry, materials science, mathematics or nuclear physics.
The overall atmosphere of the conference was extremely relaxed,
such that during the talks vivid discussions frequently arose.
I particularly enjoyed the presentations about <em>N</em>-body effects in nuclear physics
and the first-principle simulations of the structure of nuclei,
since that topic was completely new to me.
Fortunately there was one introductory talk
for each of the four mayor topics of the working group
bringing everyone up to speed in each others' subject.</p>
<p>As part of the program I presented about <a href="https://dftk.org">DFTK</a>
and our recent advances with the code. I first gave a brief rationalisation
why we started DFTK as a new code for working on electronic-structure problems
from the mathematical perspective, then I briefly presented two applications,
where we hope our code could be useful towards developing new approaches
for practical calculations.
One is an investigation of increased precision and more generally
estimates for the floating-point error in density-functional theory calculations.
The other was a discussion of our
ongoing work on SCF preconditioning techniques.
In this I showed our first steps towards developing
a mathematically justified SCF preconditioner
suitable for tackling systems containing
both a conducting and an insulating part.
Our first results indicate that our approach could be more
suitable than established methods,
which usually rely on interpolating empirically
between the established preconditioning strategies
for metals and insulators.
Our hope would be that our preconditioner could allow to apply
DFTK in the context of simulating the electronic structure
of catalytic metal surfaces in the future.
In this application a challenge for SCF schemes is that
employed catalysts are usually coated with an insulating oxide layer
or are interfacing with more insulating organic compounds or air.</p>
<table>
<thead>
<tr>
<th>Link</th>
</tr>
</thead>
<tbody>
<tr>
<td><a href="https://michael-herbst.com/talks/2020.01.09_dftk_fp_scf_lille.pdf">Using the density-functional toolkit (DFTK) to investigate floating-point error and SCF convergence</a> (Slides)</td>
</tr>
</tbody>
</table>36c3: Getting to know Julia session2019-12-31T12:00:00+01:002019-12-31T12:00:00+01:00Michael F. Herbsttag:michael-herbst.com,2019-12-31:/2019-36c3-getting-to-know-julia.html<p>At this year's 36th Chaos Communication Congress in Leipzig,
I signed up to give a short
<a href="https://events.ccc.de/congress/2019/wiki/index.php/Session:Getting_to_know_Julia">introductory workshop for Julia</a>.
To my surprise the room was completely packed yesterday,
even though the meeting was scheduled for 11am on day 4 ...
Everyone at the session was extremely engaged and
packaged with …</p><p>At this year's 36th Chaos Communication Congress in Leipzig,
I signed up to give a short
<a href="https://events.ccc.de/congress/2019/wiki/index.php/Session:Getting_to_know_Julia">introductory workshop for Julia</a>.
To my surprise the room was completely packed yesterday,
even though the meeting was scheduled for 11am on day 4 ...
Everyone at the session was extremely engaged and
packaged with plenty of supporting and challenging questions about Julia.
Luckily a bunch of masters from the Julia community were
in the audience to help out with providing the in-depth details
where I failed
(tip of the hat to Keno Fischer and Valentin Churavy).
Thanks for everyone in the room!
I really enjoyed my time and as usual with these things
I learned a lot.</p>
<p>As I mentioned during the session, the Jupyter notebooks I used
are all available <a href="https://github.com/mfherbst/course_julia_day">on github</a>.
They were originally made for a one-day introductory Julia course,
which I presented in Paris a few weeks ago
(<a href="https://michael-herbst.com/teaching/2019-julia-day-jussieu/">details here</a>).
The material, however, should be self-explanatory
and accessible to people completely new to Julia,
but which already have some familiarity with programming of some kind.
Main point is to get an idea of what Julia is like,
seed a little curiosity about the language
and provide links to plenty of further info
(<a href="https://michael-herbst.com/teaching/2019-julia-day-jussieu/#tutorials">see also these links</a>).
Feel free to spread the word about the repo if you like the material.</p>
<p>With that said, I am happy to receive any feedback you
might have about the session,
the notebooks and the other material.
I plan to keep doing such courses in the future
and I'm always looking for ways how to improve ;).</p>
<table>
<thead>
<tr>
<th>Link</th>
</tr>
</thead>
<tbody>
<tr>
<td><a href="https://github.com/mfherbst/course_julia_day">https://github.com/mfherbst/course_julia_day</a></td>
</tr>
</tbody>
</table>Introduction to Julia session at 36c32019-12-21T22:00:00+01:002019-12-21T22:00:00+01:00Michael F. Herbsttag:michael-herbst.com,2019-12-21:/36c3-julia.html<p>If you missed the <a href="https://michael-herbst.com/teaching/2019-julia-day-jussieu/">Julia day at Jussieu</a>
in Paris last week,
but still want to get to know the programming language Julia a little,
have no fear:
I'll be doing an <a href="https://events.ccc.de/congress/2019/wiki/index.php/Session:Getting_to_know_Julia">introductory Julia session</a>
at the <a href="https://events.ccc.de/congress/2019/wiki/index.php">36th Chaos Communication Congress</a>
in Leipzig this year (on <strong>day 4, 11am</strong>, so …</p><p>If you missed the <a href="https://michael-herbst.com/teaching/2019-julia-day-jussieu/">Julia day at Jussieu</a>
in Paris last week,
but still want to get to know the programming language Julia a little,
have no fear:
I'll be doing an <a href="https://events.ccc.de/congress/2019/wiki/index.php/Session:Getting_to_know_Julia">introductory Julia session</a>
at the <a href="https://events.ccc.de/congress/2019/wiki/index.php">36th Chaos Communication Congress</a>
in Leipzig this year (on <strong>day 4, 11am</strong>, so be prepared to get up early ;) ).
<a href="https://events.ccc.de/congress/2019/wiki/index.php/Session:Getting_to_know_Julia">Find all the infos here</a>.</p>
<p>The plan is to do something similar to the <a href="https://michael-herbst.com/teaching/2019-julia-day-jussieu/">Julia day</a>,
just a little more condensed with more focus on showing things rather than explaining things.
Overall the workshop will not be so much centred on really teaching Julia,
much rather to show you the potential of the language
and get you all set to start exploring Julia on your own.</p>
<p><strong>Update:</strong> <a href="https://michael-herbst.com/2019-36c3-getting-to-know-julia.html">Details here</a></p>Talk in Toulouse: adcc and DFTK2019-12-06T19:00:00+01:002019-12-06T19:00:00+01:00Michael F. Herbsttag:michael-herbst.com,2019-12-06:/2019-toulouse-lcpqt.html<p>A few weeks ago on 21st and 22nd November of this year
Pierre-Francois Loos invited me for two days
to the <a href="http://www.lcpq.ups-tlse.fr">Laboratoire de Chimie et Physique Quantiques</a>
of the Université Paul Sabatier Toulouse.
On Thursday the 21st I had the chance to present our current
status in DFTK and adcc …</p><p>A few weeks ago on 21st and 22nd November of this year
Pierre-Francois Loos invited me for two days
to the <a href="http://www.lcpq.ups-tlse.fr">Laboratoire de Chimie et Physique Quantiques</a>
of the Université Paul Sabatier Toulouse.
On Thursday the 21st I had the chance to present our current
status in DFTK and adcc at the
<a href="http://www.lcpq.ups-tlse.fr/spip.php?article1940">lab's seminar</a>.
My talk ended up being an updated version of my
<a href="https://michael-herbst.com/2019-munich-moansi-domcke.html">talk at the Domcke group in Munich</a>,
which I held just in the previous month.
After the talk and on Friday I had plenty of time to discuss with Titou,
Anthony Scemama and Michel Caffarel about adcc, DFTK and their respective projects.
The discussions were interesting from my end and ended up being a rather fruitful
exchange of ideas. During the two days I certainly learned a lot.</p>
<p>As usual my slides and examples are attached below:</p>
<table>
<thead>
<tr>
<th>Link</th>
</tr>
</thead>
<tbody>
<tr>
<td><a href="https://michael-herbst.com/talks/2019.11.21_software_toulouse.pdf">Interdisciplinary software for electronic structure theory: adcc and DFTK</a> (Slides seminar LCPQ)</td>
</tr>
<tr>
<td><a href="https://michael-herbst.com/talks/2019.11.21_software_toulouse_DFTK.tar.gz">DFTK and Julia examples</a> (Tarball)</td>
</tr>
<tr>
<td><a href="https://michael-herbst.com/talks/2019.11.21_software_toulouse_adcc_method_cascade.ipynb">adcc examples</a> (Notebook)</td>
</tr>
</tbody>
</table>Preview release of the density-functional toolkit (DFTK)2019-11-15T10:00:00+01:002019-11-15T10:00:00+01:00Michael F. Herbsttag:michael-herbst.com,2019-11-15:/2019-release-dftk.html<p>As part of my research stay at the
<a href="https://team.inria.fr/matherials/">Matherials team</a>
of ENPC Paris Tech and Inria
I have been developing the density-functional toolkit,
<a href="https://github.com/JuliaMolSim/DFTK.jl">DFTK.jl</a>,
together with Antoine Levitt and Eric Cancès.
Finally after about six months of joint effort
on this Julia code, we just released a first preview …</p><p>As part of my research stay at the
<a href="https://team.inria.fr/matherials/">Matherials team</a>
of ENPC Paris Tech and Inria
I have been developing the density-functional toolkit,
<a href="https://github.com/JuliaMolSim/DFTK.jl">DFTK.jl</a>,
together with Antoine Levitt and Eric Cancès.
Finally after about six months of joint effort
on this Julia code, we just released a first preview in the form of
<a href="https://github.com/JuliaMolSim/DFTK.jl/releases">version <code>0.0.1</code></a>.</p>
<p>As I mentioned already in
<a href="https://michael-herbst.com/2019-lille-software-development-techniques.html">previous</a>
<a href="https://michael-herbst.com/2019-munich-moansi-domcke.html">articles</a>
and <a href="https://michael-herbst.com/talks_posters/#pwdft">talks</a>
the aim of DFTK is to bridge between mathematicians,
computer scientists and materials scientists
and to simplify mathematically-motivated
research in this interdisciplinary subject.
Currently we exclusively focus on a plane-wave discretisation
of density-functional theory (PWDFT)
as implemented in plenty of mature packages in the community
(<a href="https://www.abinit.org/">Abinit</a>, <a href="http://quantum-espresso.org/">Quantum Espresso</a>,
<a href="https://www.vasp.at/">VASP</a>, ...).</p>
<p>DFTK is mainly born from two realisations:
Firstly, existing PWDFT progams have huge code bases,
where reducing or altering the physical model to a case,
which can be treated with rigorous mathematics
is very challenging.
As a result mathematical research is done in custom codes
developed mostly for the purpose of a single project or research
questions and findings hardly make it into a setting useful
for practitioners.
Secondly, such single-purpose custom codes are usually are not fast
enough to treat the setting of real-world applications.
This prevents verification of obtained results in the context
actually relevant in practice.
With DFTK we want to overcome this issue by allowing
<em>in one code</em> to both implement toy problems and upscale
them to the high-performance level needed in practice.</p>
<p>As of now, we're certainly not fully there,
but given the short time, we're still proud of our feature list:</p>
<ul>
<li>Plane-wave basis sets</li>
<li>All LDA and GGA functionals from <a href="https://tddft.org/programs/libxc/">libxc</a></li>
<li>Modelling of Insulators and metals (Fermi-Dirac or Methfessel-Paxton smearing)</li>
<li><a href="https://doi.org/10.1103/PhysRevB.54.1703">GTH</a>
or <a href="https://doi.org/10.1103/PhysRevB.58.3641">HGH</a> Pseudopotentials</li>
<li>Exploitation of Brillouin zone symmetry for <code>k</code>-Point sampling</li>
<li>Band structure computation</li>
<li>Full access to intermediate quantities (density, Bloch wave)</li>
<li>Three SCF algorithms
(DIIS, <a href="https://github.com/JuliaNLSolvers/NLsolve.jl">NLsolve</a>, damping)</li>
<li>Close agreement with <a href="https://www.abinit.org/">Abinit</a> for a few
thoroughly tested cases (silicon, graphite, manganese).</li>
<li>Support for both single and double precision throughout the
library for a small set of functionals.
Support for <strong>arbitrary floating point types</strong>
is <a href="https://github.com/JuliaMolSim/DFTK.jl/pull/59">on the way</a>.</li>
</ul>
<p>Recently we moved the code to <a href="https://github.com/JuliaMolSim/">JuliaMolSim</a>
a github organisation, where we want to collect Julia codes
for performing molecular simulations in quantum chemistry
and materials science.
Along with this we registered the release
with the <a href="https://github.com/JuliaMolSim/MolSim">MolSim</a>
Julia registry, which means that you can <strong>install DFTK</strong> by two simple steps:</p>
<ol>
<li>Add the MolSim registry to your Julia installation. Type from a Julia REPL:</li>
</ol>
<div class="highlight"><pre><span></span><code><span class="p">]</span><span class="w"> </span><span class="n">registry</span><span class="w"> </span><span class="n">add</span><span class="w"> </span><span class="nl">https:</span><span class="c1">//github.com/JuliaMolSim/MolSim.git</span>
</code></pre></div>
<ol>
<li>Install DFTK as usual, again from a REPL:</li>
</ol>
<div class="highlight"><pre><span></span><code>] add DFTK
</code></pre></div>
<p>For the time being, DFTK will probably stay in the
pre-release stage, as we are not yet completely happy with the
internal code structure and the API and we think
some more restructuring should follow.
Still, we consider the code now
sufficiently advanced to suggest you
to <strong>have a look and try it out :)</strong>.
With this naturally comes a small plea: If you discover something,
which bothers you or something, which does not work,
<strong>please <a href="https://github.com/JuliaMolSim/DFTK.jl/issues">open an issue</a></strong>
on github and pose it for discussion.
Also we are very happy if you want to contribute something,
please just get going!</p>
<p>For further details and the DFTK source code, see the
<a href="https://github.com/JuliaMolSim/DFTK.jl">DFTK project page</a>
on github. Citations to the DFTK source code are possible
using <a href="https://doi.org/10.5281/zenodo.3541724">DOI 10.5281/zenodo.3541724</a>.</p>adcc: A versatile toolkit for rapid development of algebraic-diagrammatic construction methods2019-10-23T23:00:00+02:002019-10-23T23:00:00+02:00Michael F. Herbsttag:michael-herbst.com,2019-10-23:/2019-adcc.html<p>Over the part year or so a side project I have been working off and on
has been the <a href="https://adc-connect.org">adcc code</a>.
This python module allows to perform excited states calculation
for simulating molecular spectra
based on algebraic-diagrammatic construction (ADC) methods.
The speciality of the adcc code is that
(a) multiple …</p><p>Over the part year or so a side project I have been working off and on
has been the <a href="https://adc-connect.org">adcc code</a>.
This python module allows to perform excited states calculation
for simulating molecular spectra
based on algebraic-diagrammatic construction (ADC) methods.
The speciality of the adcc code is that
(a) multiple host programs are supported for supplying a Hartree-Fock reference,
right now <a href="http://www.pyscf.org/">pyscf</a>, <a href="http://psicode.org/">Psi4</a>,
<a href="https://molsturm.org">molsturm</a> and <a href="https://veloxchem.org">veloxchem</a>,
and (b) that its hybrid C++/python design allows to implement
a large part of the ADC procedure <em>including iterative solver schemes</em>
in python and still achieve the speed required for larger ADC calculations.
The code development has been a collaboration with multiple members
from the old group of my PhD times,
the <a href="https://dreuw.qchem.de">Dreuw group</a> from Heidelberg.
Finally, today, we are publicly releasing the code
following the submission of our paper on
adcc last week (<a href="http://localhost:8000/publications/2019.10.23_adcc.pdf">download link</a>).</p>
<p>I have already
<a href="https://michael-herbst.com/2019-munich-moansi-domcke.html">talked about adcc</a>
and <a href="https://michael-herbst.com/2018-franco-german-aachen.html">ADC methods</a> before
(see also my <a href="https://michael-herbst.com/research/cvs_error/">research interests</a>),
so I'll be brief with my summary about these.
The ADC scheme builds upon the fact that the poles of the polarisation
propagator, i.e. the two-particle Green's function, contains the physical
information for describing excited electronic states and their properties.
Using a Møller-Plesset partitioning of the Schrödinger Hamiltonian
one may construct a perturbative expansion for said polarisation propagator.
In the ADC approach this is done using a particular representation basis
for the many-body states of the Schrödinger Hamiltonian,
the so-called intermediate state basis.
The outcome are a perturbative series of matrices, the so-called ADC matrices
corresponding to the ADC(n) series of methods.
The ADC matrices contain exactly the spectral information of the
polarisation propagator consistent to order n of a Møller-Plesset
expansion of the correlated ground state.
Diagonalising this matrix, thus allows to obtain excitation energies and
excited-state properties. For more details, see for example the
<a href="https://adc-connect.org/latest/theory.html">theory section in the adcc documentation</a>.</p>
<p>As we discuss in our paper, in adcc the basic procedure of building and diagonalising
the ADC matrix is implemented in a hybrid C++/python code.
The aim was to keep everything,
including intermediate results, accessible from the python layer,
while implementing the computationally intensive parts in C++.
As a result,
the code is comparable to C++-only implementations in speed,
such that it can be used to e.g. treat biomolecular systems.
The iterative diagonalisation procedure
and other aspects of the numerics are implemented completely in python,
facilitating experimenting on the numerical level in the future.
Multiple variants of ADC are available up to and including ADC(3),
i.e. a treatment of the polarisation propagator consistent to third-order.
Originally adcc grew out of my desire to use another ADC code,
which is developed in the Dreuw group, namely the <code>adcman</code> code,
interactively from python.
In its current state, however,
adcc has well grown beyond being a plain wrapper around <code>adcman</code>.
The internal code structure and workflows of both packages have
very little in common, albeit the general methodology to solve ADC of both
packages is certainly related.
The design of adcc aims to simplify working with ADC methods
as a developer, but also facilitating the use and extension of existing workflows
by everyday users.
For more details and a couple of examples, see the paper,
or the <a href="https://adc-connect.org/latest/calculations.html">Performing calculations with adcc</a>
section of the adcc documentation.
<a href="https://adc-connect.org/latest/installation.html">Installing adcc</a> is simple.
Given that a few requirements (such as openblas on Linux) are available,
it boils down to using pip:</p>
<div class="highlight"><pre><span></span><code>pip install pybind11
pip install adcc
</code></pre></div>
<p>The full abstract reads</p>
<blockquote>
<p>ADC-connect (<code>adcc</code>) is a hybrid python/C++ module for performing excited state
calculations based on the algebraic-diagrammatic construction scheme for the
polarisation propagator (ADC). Key design goal is to restrict adcc to this single
purpose and facilitate connection to external packages, e.g., for obtaining the
Hartree-Fock references, plotting spectra, or modelling solvents. Interfaces
to four self-consistent field codes have already been implemented,
namely <code>pyscf</code>, <code>psi4</code>, <code>molsturm</code>, and <code>veloxchem</code>. The computational workflow,
including the numerical solvers, are implemented in python, whereas the working
equations and other expensive expressions are done in C++. This equips <code>adcc</code> with
adequate speed, making it a flexible toolkit for both rapid development of ADC-based
computational spectroscopy methods as well as unusual computational workflows. This
is demonstrated by three examples. Presently, ADC methods up to third order in
perturbation theory are available in adcc, including the respective core-valence
separation and spin-flip variants. Both restricted or unrestricted Hartree-Fock
references can be employed.</p>
</blockquote>JuliaParis: Electronic structure simulations using Julia2019-10-19T00:00:00+02:002019-10-19T00:00:00+02:00Michael F. Herbsttag:michael-herbst.com,2019-10-19:/2019-dftk-julia-paris.html<p>Yesterday we had the first instance of the
<a href="https://julia-users-paris.github.io/">Julia Paris Meetup</a>.
As the name suggests a couple of us Julia enthusiasts in the French capital
got together for one evening of talks and chatting about everyone's favourite
programming language. It was a great occasion to finally meet some people
in …</p><p>Yesterday we had the first instance of the
<a href="https://julia-users-paris.github.io/">Julia Paris Meetup</a>.
As the name suggests a couple of us Julia enthusiasts in the French capital
got together for one evening of talks and chatting about everyone's favourite
programming language. It was a great occasion to finally meet some people
in person and gather around for some technical chit-chat till the late hours
at one of the bars of the 11th arrondissement after the talks.</p>
<p>I myself was pretty excited about the evening and not quite sure what
kind of people to expect. Unsurprisingly most people, who turned
up had an academic background with focus on number crunching of some sort,
mostly high-performance linear algebra or optimisation problems.
To my big surprise, the number of people working in a industrial context
was a lot larger than I expected, even though most did not use the language
to a large extend in their everyday work. Still, after the evening
I am yet again convinced that Julia is spreading beyond dusty desks in universities.
Certainly, a great part in this plays the dedicated community of people,
willing to invest a Sunday evening here and there to work on just
the very thing required to advance the language and the Julia ecosystem step by step.
Even though this has already been clear to me to some extent before Thursday,
actually meeting the community and listening to stories and experiences
from the Julia trenches, make the whole impression more vivid.
I am already looking forward to the next meeting
(details as always on <a href="https://julia-users-paris.github.io">https://julia-users-paris.github.io</a>).</p>
<p>On that note, I was very happy, that I had the chance to play an active part
in the evening by being one of the first to present some stories of my own.
Naturally I talked about my endeavours with Julia
while developing the <a href="https://github.com/JuliaMolSim/DFTK.jl">DFTK</a> code.
Since I was not too sure about the audience,
I tried to get across what makes Julia an ideal language for our efforts,
what things we are currently working on and what will be our focus next
and how we expect Julia to help. I'm not going to repeat the details
about DFTK here, because these can be found in a
<a href="https://michael-herbst.com/2019-lille-software-development-techniques.html">previous article</a>.
See also the second half of <a href="https://michael-herbst.com/2019-munich-moansi-domcke.html">this article</a>.
The slides of my talk can be downloaded here as usual:</p>
<table>
<thead>
<tr>
<th>Link</th>
</tr>
</thead>
<tbody>
<tr>
<td><a href="https://michael-herbst.com/talks/2019.10.17_dftk_julia_paris.pdf">Electronic-structure simulations using Julia</a> (Slides 1st Julia Paris Meetup)</td>
</tr>
</tbody>
</table>Teasers for some upcoming Julia activities2019-10-01T00:00:00+02:002019-10-01T00:00:00+02:00Michael F. Herbsttag:michael-herbst.com,2019-10-01:/2019-julia-activities.html<p>A quick teaser for some Julia-related activities around Paris,
I am involved with in the next months.</p>
<ul>
<li>
<p><strong>17 Oct 2019, 19:00</strong>, QuantStack (27 Rue du Chemin Vert 75011, Paris):
The first <strong>Julia meet-up</strong> of Paris.
We want to gather the people in and around Paris, who are interested in …</p></li></ul><p>A quick teaser for some Julia-related activities around Paris,
I am involved with in the next months.</p>
<ul>
<li>
<p><strong>17 Oct 2019, 19:00</strong>, QuantStack (27 Rue du Chemin Vert 75011, Paris):
The first <strong>Julia meet-up</strong> of Paris.
We want to gather the people in and around Paris, who are interested in
or working with Julia and stimulate exchanging experiences and ideas.
I'll give a short talk about <a href="https://github.com/JuliaMolSim/DFTK.jl"><code>DFTK.jl</code></a>
and François Févotte will talk about <a href="https://github.com/JuliaMath/AccurateArithmetic.jl"><code>AccurateArithmetic.jl</code></a>.
More details on <a href="https://julia-users-paris.github.io">https://julia-users-paris.github.io</a>.
Feel free to drop by if you are around!</p>
</li>
<li>
<p><strong>13 Dec 2019, 09:00</strong>, Sorbonne Université, <a href="http://ljll.math.upmc.fr/">LJLL</a>
and <a href="http://lct.jussieu.fr/">LCT</a> labs of Jussieu Campus.
<strong>Julia Day</strong> (Journée Julia): I'll do an introductory session to Julia
in the morning and talk a little bit about useful and helpful Julia
packages (linear algebra, non-linear solvers, etc.) in the afternoon.
followed by an introduction to
<a href="https://github.com/JuliaMolSim/DFTK.jl"><code>DFTK.jl</code></a>.
For more details on the day
<a href="https://michael-herbst.com/teaching/2019-julia-day-jussieu/">see the website</a>.</p>
</li>
</ul>Interdisciplinary software for electronic structure theory: adcc and DFTK2019-09-16T19:00:00+02:002019-09-16T19:00:00+02:00Michael F. Herbsttag:michael-herbst.com,2019-09-16:/2019-munich-moansi-domcke.html<p>Last week, from Wednesday to Friday, I paid the Technical University of Munich (TUM)
a short visit for two great opportunities to present my work:
Once in the theoretical chemistry group of
<a href="https://www.department.ch.tum.de/theo/startseite/">Professor Wolfgang Domcke</a>
and once at the annual meeting of the
modeling, analysis and simulation of molecular systems …</p><p>Last week, from Wednesday to Friday, I paid the Technical University of Munich (TUM)
a short visit for two great opportunities to present my work:
Once in the theoretical chemistry group of
<a href="https://www.department.ch.tum.de/theo/startseite/">Professor Wolfgang Domcke</a>
and once at the annual meeting of the
modeling, analysis and simulation of molecular systems (MoAnSi) interest group,
a crowd consisting mostly of mathematicians and fellow modelling scientists.</p>
<h2>Talk at Domcke group</h2>
<p>The <a href="https://michael-herbst.com/talks/2019.09.11_adcc_dftk_munich.pdf">first presentation</a>
I gave last Wednesday in the Domcke group,
where
I talked about challenges and approaches to interdisciplinary software projects.
I also presented two projects of my own, namely
<a href="https://github.com/JuliaMolSim/DFTK.jl">DFTK</a> and <a href="https://adc-connect.org">adcc</a>.
About these codes I already gave a few details
in <a href="https://michael-herbst.com/2019-lille-software-development-techniques.html">previous</a>
<a href="https://michael-herbst.com/2019-mqm-heidelberg.html">articles</a>
and thus be rather brief with respect to details about them.</p>
<p>In the part of the talk on the plane-wave density-functional theory code DFTK
I mainly focused on the question why we decided to use Julia
as a programming language.
For this I summarised Julia's capabilities and performance
and gave a short teaser of Julia code in a
<a href="https://michael-herbst.com/talks/2019.09.11_julia_intro_davidson.ipynb">Jupyter notebook</a>.
This notebook shows a Julia implementation of a Jacobi-Davidson
iterative eigensolver and discusses concrete instantiations of the code
in various situations.
This includes a mixed-precision algorithm, which transfers intermediate results
obtained in a lower precision (e.g. single precision) to higher
(i.e. double) precision and then continues the iterations at this level.
Other examples were diagonalisations using sparse matrices or GPU computations.
Most notably, for all these cases the actual Jacobi-Davidson code
was only written once and not explictly adapted
to fit the respective datastructures or computational backends.
Last, but not least the notebook teasered a small automatic
differentiation example as well.</p>
<p>The main focus of the talk, however, was the <a href="https://adc-connect.org">adcc</a> code,
which implements the algebraic-diagrammatic construction scheme for
the polarisation propagator (ADC) and allows to drive this computation
from any SCF code using a python-based interface.
In the presentation I sketch content from our upcoming software paper
on adcc and present three examples for calculations, which we are able
to do with only few lines of python code.
This includes an analysis of the core-valence separation (CVS) error for water,
a sophisticated combination of freezing extra core and valence orbitals
alongside the CVS
and a case of an ADC(2) modelling of nile red (426 basis functions)
in an explicit solvent model (polarisable embedding).
The code for the second example, namely the combination of CVS with freezing
other orbitals, was also shown in an
<a href="https://michael-herbst.com/talks/2019.09.11_adcc_demo_cvs_frozen_spaces.ipynb">example notebook</a>.
Especially this part of the talk was well-received,
providing fruit for many ADC-related and adcc-related discussions afterwards.</p>
<h2>MoAnSi workshop</h2>
<p>Directly after the day at the Domcke group I attended
the <a href="https://www.ma.tum.de/en/department/news-events/modelling-analysis-simulation-molecular-systems-meeting.html">MoAnSi meeting</a>,
which took place last Thursday and Friday,
bringing together a bunch of mathematicians interested in computational methods
for molecular science.
Similar to other interdisciplinary meetings overlapping chemistry and maths,
like the <a href="https://michael-herbst.com/2018-oberwolfach.html">one in Oberwolfach</a>,
the atmosphere was friendly and open.
Plenty of discussions were stimulating for thought about fundamental
aspects of quantum chemistry,
one usually does not think about as a chemist.</p>
<p>For this reason my <a href="https://michael-herbst.com/talks/2019.09.13_dftk_moansi.pdf">second talk</a> was tailored
a bit more to the mathematician crowd,
which meant that I focused more on <a href="https://github.com/JuliaMolSim/DFTK.jl">DFTK</a>
and how it could
be used in the context of mathematically-driven research
of density-functional theory.
Apart from the slides and the
<a href="https://michael-herbst.com/talks/2019.09.11_julia_intro_davidson.ipynb">Julia examples</a>
I also presented some actual DFTK code,
emphasising how Julia leads to extremely concise and readable programs.
During the Q&A, I mentioned a blog article
about a Julia case study using the language for a petascale computation.
The respective project is
the <a href="https://juliacomputing.com/case-studies/celeste.html">Celeste code</a>
and a summary of the hurdles and difficulties
can be found
<a href="https://juliacomputing.com/blog/2019/04/12/Supercomputing-julia.html">in a post by Keno Fischer</a>
on the Julia Computing Blog.</p>
<p>Slides and notebooks for both talks are linked in the main text above
or can be downloaded here:</p>
<table>
<thead>
<tr>
<th>Link</th>
</tr>
</thead>
<tbody>
<tr>
<td><a href="https://michael-herbst.com/talks/2019.09.11_adcc_dftk_munich.pdf">Interdisciplinary software for electronic structure theory: adcc and DFTK</a> (Slides seminar Domcke group)</td>
</tr>
<tr>
<td><a href="https://michael-herbst.com/talks/2019.09.11_julia_intro_davidson.ipynb">Two Julia examples</a> (Notebook)</td>
</tr>
<tr>
<td><a href="https://michael-herbst.com/talks/2019.09.11_adcc_demo_cvs_frozen_spaces.ipynb">adcc Demo: Combining CVS and frozen orbitals</a> (Notebook)</td>
</tr>
<tr>
<td><a href="https://michael-herbst.com/talks/2019.09.13_dftk_moansi.pdf">DFTK: The density-functional toolkit</a> (Slides MoAnSi talk)</td>
</tr>
</tbody>
</table>Interdisciplinary software development in electronic structure theory (MQM Poster)2019-07-30T13:00:00+02:002019-07-30T13:00:00+02:00Michael F. Herbsttag:michael-herbst.com,2019-07-30:/2019-mqm-heidelberg.html<p>At the beginning of the month, from 30th June till 5th July, I attended the
<a href="https://mqm2019.org/">9th Molecular Quantum Mechanics Conference</a>
in my old home Heidelberg.
This occasion was a great opportunity to catch up with friends and colleges
both in science and from my old circle in Heidelberg as well …</p><p>At the beginning of the month, from 30th June till 5th July, I attended the
<a href="https://mqm2019.org/">9th Molecular Quantum Mechanics Conference</a>
in my old home Heidelberg.
This occasion was a great opportunity to catch up with friends and colleges
both in science and from my old circle in Heidelberg as well.</p>
<p>At the conference I presented a poster entitled
<strong>Modern interdisciplinary software development in electronic-structure theory</strong>.
which was highly related to my former
<a href="https://michael-herbst.com/2019-lille-software-development-techniques.html">talk in Lille</a>
a couple of weeks ago.
In the poster I wanted to to motivate the use of modern languages and
software development techniques and demonstrate its opportunities for
enabling different communities (like mathematicians and chemists)
to work jointly on the same codes.
As examples I discuss three projects of my own.</p>
<p>Out of these, the <strong><a href="https://molsturm.org">molsturm</a></strong>
code for basis-function-independent
method development I have described in detail
in our <a href="https://michael-herbst.com/molsturm-design.html">paper last year</a>.
Some words about our mathematically-driven approach
in the development of the
<strong><a href="https://github.com/JuliaMolSim/DFTK.jl">density-functional toolkit (DFTK)</a></strong>
I already gave in
<a href="https://michael-herbst.com/2019-lille-software-development-techniques.html">my previous article</a>.
In this article I will thus focus on the third code
presented on the poster, namely <a href="https://adc-connect.org">adcc</a>,
short for <strong>ADC connect</strong>.</p>
<p>The <strong>adcc project</strong> is a project I started a few years ago
during my time at the <a href="https://dreuw.qchem.de">Dreuw group</a> in Heidelberg.
I continue to work on it as a side project during my time at the CERMICS,
jointly with some of my old group members.
Our main aim is to provide one independent
building block for excited states computations
using the algebraic-diagrammatic construction (ADC).
This means that adcc does not implement any self-consistent field
algorithm itself, much rather it runs on top of any Hartree-Fock code.
In practice as of now four SCF codes have been connected to adcc
via our Python interface, including
<a href="https://pyscf.github.io/">pyscf</a>,
<a href="http://www.psicode.org/">Psi4</a> and
<a href="https://molsturm.org">molsturm</a>.
But Python is not only used as a glue language between
SCF and ADC, much rather it allows to orchestrate
the full computational procedure.
See <a href="https://michael-herbst.com/poster/2019.07.01_MQM_Software_Development.pdf">the attached poster</a>
or the <a href="https://adc-connect.org/latest/calculations.html">introductory chapter</a>
of the adcc documentation for some examples.</p>
<p>A third aspect where Python plays a crucial role in adcc
are the iterative solver algorithms.
For example the Davidson procedure we use for solving the ADC
equations is implemented purely in high-level Python code.
This allows to rapidly investigate new numerical schemes
and approaches in the context of ADC.
We are, however, not loosing too much performance by this choice,
because the time-consuming tensor contractions we require
are still done in a <code>C++</code> core library and only called from Python.
Inside our core library we in turn use the
<a href="https://github.com/epifanovsky/libtensor">libtensor</a> code for
tensor computations.
Overall adcc therefore shows a comparable performance
to the <a href="http://doi.org/10.1080/00268976.2013.859313">adcman code</a>,
a pure <code>C++</code> implementation of ADC also developed in Heidelberg
on top of libtensor.</p>
<p>Currently adcc is not yet publicly available,
but we are in the stage of finalising adcc for a first public release
within the Gator framework. A first standalone release of the code
is planned within the upcoming months as well.
The <a href="https://adc-connect.org">adcc documentation</a>, however,
is already available and gives an idea of adcc in practice.</p>
<p>As usual I attach my poster as a pdf below.</p>
<table>
<thead>
<tr>
<th>Link</th>
</tr>
</thead>
<tbody>
<tr>
<td>Poster <a href="https://michael-herbst.com/poster/2019.07.01_MQM_Software_Development.pdf"><em>Modern interdisciplinary software development in electronic structure theory</em></a></td>
</tr>
</tbody>
</table>Modern software-development techniques in electronic structure theory2019-06-09T00:00:00+02:002019-06-09T00:00:00+02:00Michael F. Herbsttag:michael-herbst.com,2019-06-09:/2019-lille-software-development-techniques.html<p>A few weeks ago, on 23rd May I received an invitation
to visit the
<a href="http://www.phlam.univ-lille1.fr/">Laboratoire de Physique des Lasers, Atomes et Molécules</a>
at the Université de Lille and present about my recent work.
When it came to selecting a topic, me and my host, Andre Gomes,
quickly agreed to focus …</p><p>A few weeks ago, on 23rd May I received an invitation
to visit the
<a href="http://www.phlam.univ-lille1.fr/">Laboratoire de Physique des Lasers, Atomes et Molécules</a>
at the Université de Lille and present about my recent work.
When it came to selecting a topic, me and my host, Andre Gomes,
quickly agreed to focus on discussing
more modern approaches to scientific software development
and why these can be useful for electronic-structure codes.
In retrospect I am very happy for this opportunity
to summarise a few ideas and learned lessons
from my previous and ongoing software projects.
I also took the liberty to sketch possible challenges
when it comes to future directions of scientific software
from my personal point of view as well as
propose some ideas how to tackle them.</p>
<p>From this angle I ended up introducing the talk
by a review of the fundamental difficulties of electronic structure
theory itself, namely the inherent complexity of the problem
(large dimensionality and non-linearity of the respective
partial-differential equations). Added to that in recent years
the available high-performance computing (HPC) environments have become
more heterogeneous and more complex as well. Mixed general-purpose GPU / CPU
cluster setups are now standard and making use of both CPU and GPU
for computations has become a strict requirement for applying for access
to the top-ranked clusters of the world. Projecting into the future,
where further "accelerators" such as field-programmable gate arrays (FPGAs),
or even more long term quantum computers, might come up,
this heterogeneity is only going to increase.</p>
<p>For quickly testing the usefulness of such technologies in the context of
quantum chemistry, a key aspect is to be able to reuse the code,
which already exists. This means that code needs to be sufficiently high-level
in order to be able to adapt to changing hardware architectures.
Let be be clear on this: I am not talking about achieving 100% peak performance
on every hardware setup, much rather about having a good compromise
between the required effort to get the code to work on new hardware
and the achieved performance. The point is to quickly get an idea
on what is theoretically possible and if further investigation even
makes sense.</p>
<p>As I sketch in the talk,
changing the hardware, does have an impact on questions such as the
optimal basis function, numerical algorithm or even the best-suited
physical model. In this "suitable" and "optimal" refers to
a simulation procedure, which agrees with the available
computational architecture and at the same time is physically meaningful.
A consequence of this observation is that experimentation on all levels
(basis functions, algorithms, numerics, linear algebra backend) is
required, which often calls for interdisciplinary expertise.
In an ideal world experts from different fields can thus work
on the same code base, approaching the problem from various different angles.</p>
<p>Without a doubt this is a huge task and chances are, that this goal
will in fact never be reached. Still I think,
there are some opportunities to get a little closer than presently.
For this purpose, key aspects are
high-level and dynamic programming languages
like Julia and python and a clear, modular code design,
where individual aspects of the simulation procedure
can be easily modified or swapped.
Such an approach helps in practice to investigate
different basis functions or swap computational backends
with only negligible changes to the code base,
as I discuss with reference to the
molsturm and the DFTK codes (see below).
The ability to approach the physical problem on a high level
allows mathematicians and physicists to interact
with the code abstractly, while computer scientists still have the freedom
to tweak performance on a lower level.</p>
<p>I have already discussed
the basis-function independent quantum chemistry package
<a href="https://michael-herbst.com/molsturm-design.html">molsturm</a>
<a href="https://michael-herbst.com/2018-cs-atomic-hf-convergence.html">and related</a>
<a href="https://michael-herbst.com/2018-jussieu-lct-molsturm.html">work</a>
<a href="https://michael-herbst.com/2019-metz-coulomb-sturmian-quantum-chemistry.html">a few times</a>,
so I'll skip that in this article.
Instead I want to detail some aspects of
a more recent project, <a href="https://github.com/JuliaMolSim/DFTK.jl">DFTK.jl</a>,
the density-functional toolkit.
The idea of this code is to be simple and minimalistic,
using existing libraries and codes as much as possible.
This makes the code a lot more accessible, which facilitates
to construct reduced models, which can be treated in mathematical proof.
The hope is that the lessons learned can than be scaled in the same
code base to larger and more realistic problems and an HPC environment.
The choice of programming language for this project is julia,
since it is a high-level and dynamical, but strongly typed language
with an impressive performance and out-of-the-box compatibility with existing
libraries written in C++, Fortran, python or R.
Using features such as multiple dispatch and JIT (just-in-time)
compilation of code, Julia seems to be a big step forwards in the direction,
where code is written once and then can be translated many times
for specific computational backends or hardware.
I already presented about Julia
<a href="https://michael-herbst.com/2019-inria-julia-overview.html">in a previous talk</a>
a few weeks ago.</p>
<p>All in all I am very thankful for Andre for giving me the opportunity
to gather some thoughts about this matter and eventually present them
in this talk.
The audience in Lille was surprisingly open about the topic and to my
big surprise many interesting discussions arose.
Throughout the afternoon I spent time with PhD students and staff researchers
discussing my ideas in their application context,
leading to loads of interesting feedback, helpful ideas, questions and comments.
The time in Lille truly flew by very quickly with may aspects still
undiscussed after the day. Luckily we all agreed to stay in touch,
such that the discussions will likely continue during other
meetups in the future.</p>
<p>The slides and the Jupyter notebooks I used during the talk are attached below.</p>
<table>
<thead>
<tr>
<th>Link</th>
</tr>
</thead>
<tbody>
<tr>
<td><a href="https://michael-herbst.com/talks/2019.05.23_software_development_lille.pdf">Modern software-development techniques in electronic structure theory</a> (Slides)</td>
</tr>
<tr>
<td><a href="https://michael-herbst.com/talks/2019.05.23_software_development_lille.ipynb">A 5-minute introduction to Julia</a> (Jupyter notebook)</td>
</tr>
</tbody>
</table>Coulomb Sturmian basis functions in electronic structure theory2019-05-22T19:00:00+02:002019-05-22T19:00:00+02:00Michael F. Herbsttag:michael-herbst.com,2019-05-22:/2019-metz-coulomb-sturmian-quantum-chemistry.html<p>Earlier this month, on 3rd of May, I was invited for a day to the
<a href="https://lpct.univ-lorraine.fr/">Laboratoire de Physique et Chimie Théoriques</a>
of the Université de Lorraine in Metz.
Unlike my earlier stay in Metz stay, during the
<a href="https://michael-herbst.com/2018-mes-metz.html">MES conference</a>,
last August, I did not manage to enjoy this beautiful city …</p><p>Earlier this month, on 3rd of May, I was invited for a day to the
<a href="https://lpct.univ-lorraine.fr/">Laboratoire de Physique et Chimie Théoriques</a>
of the Université de Lorraine in Metz.
Unlike my earlier stay in Metz stay, during the
<a href="https://michael-herbst.com/2018-mes-metz.html">MES conference</a>,
last August, I did not manage to enjoy this beautiful city much.
Instead I passed my day with science at the lab
and presented at their seminar in a nicely informal setting:
Around a communal table, eating <em>brioche</em> and drinking coffee.</p>
<p>I have known my host, Ugo Ancarani,
already since the MES.
His research mostly revolves around
the modelling of collisions,
more precisely
the interaction of light and matter leading to the ejection
of one or more electrons.
Similar to my research in molecular problems he employs
a Sturmian-based approach to solve the problem.
It was quite interesting to discuss similarities and differences
between the physics of bound states and of collision problems,
a theoretical setting I have not yet
spend a lot of time thinking about.</p>
<p>In general I was quite surprised by the large number of research directions,
which are dealt with in this lab.
The range covers for example quantum dynamics,
collision theory,
biochemistry, solvation and mixed phase problems.
I did not manage to chat with everyone,
unfortunately,
but I think I managed to catch at least some insight
into the research in Metz.</p>
<p>Originating from the relation
with Ugo's work I ended up presenting
on our recent Coulomb-Sturmian
<a href="https://michael-herbst.com/2018-cs-atomic-hf-convergence.html">convergence results at HF level</a>
with some outlook into future directions.
Overall I really enjoyed the atmosphere at the seminar
and the lab and
I already look forward to anther time when I might return to Metz.
As usual my slides are attached below.</p>
<table>
<thead>
<tr>
<th>Link</th>
</tr>
</thead>
<tbody>
<tr>
<td><a href="https://michael-herbst.com/talks/2019.05.03_coulomb_sturmians_metz.pdf">Coulomb Sturmian basis functions in electronic structure theory</a> (Slides seminar talk)</td>
</tr>
</tbody>
</table>Julia lightning talk2019-04-17T11:00:00+02:002019-04-17T11:00:00+02:00Michael F. Herbsttag:michael-herbst.com,2019-04-17:/2019-inria-julia-overview.html<p>Yesterday I gave a short lightning talk at the
<a href="https://sed-paris.gitlabpages.inria.fr/developer-meetups/">Inria Devmeetup</a>
about <a href="https://julialang.org">Julia</a>.
My intention was to show a few of the design aspects
of this rather recent programming language
and motivated from my experiences
in my
<a href="https://michael-herbst.com/postdoc-cermics.html">current PostDoc project at the MATHERIALS team</a>
hint at interesting ongoing developments.
For …</p><p>Yesterday I gave a short lightning talk at the
<a href="https://sed-paris.gitlabpages.inria.fr/developer-meetups/">Inria Devmeetup</a>
about <a href="https://julialang.org">Julia</a>.
My intention was to show a few of the design aspects
of this rather recent programming language
and motivated from my experiences
in my
<a href="https://michael-herbst.com/postdoc-cermics.html">current PostDoc project at the MATHERIALS team</a>
hint at interesting ongoing developments.
For this I chose to show some (hopefully) representative
code examples in a <a href="http://jupyter.org/">Jupyter notebook</a>,
interactively running the Julia code
using an <a href="https://github.com/JuliaLang/IJulia.jl">IJulia</a> kernel.</p>
<p>For demonstrating Julia's <strong>multiple dispatch</strong> paradigm,
I hinted how functionality (like the <code>mysquare</code>
function in my example) can be easily implemented in a way
such that code can be combined with multiple different
computational backends.
Such backends include distributed array storage for large
chunks of data or arrays of static size,
where the size information may be used at compile-time
to speed up the byte code for small problems.
GPU backends are possible as well, but I did not go into this
in my presentation.
With respect to speed I show <strong>some timings</strong> from a heat equation
example (courtesy <a href="http://antoine.levitt.fr/">Antoine Levitt</a>)
comparing a <code>Julia</code>, a <code>python</code> and a <code>C</code> implementation.
Last but not least I hinted at the <strong>interoperability</strong>
with python packages and Fortran code
and showed a plotting example,
where I used <a href="https://github.com/FluxML/Zygote.jl">Zygote</a>
to automatically compute and show the gradient
of a function using <strong>adjoint-mode automatic differentiation</strong>.</p>
<p>The notebook I used for the presentation is both attached
below and can be found
<a href="https://github.com/mfherbst/inria-devmeetup-julia-overview">on github</a>
as well.
For a rendered version of the notebook you can
<a href="https://nbviewer.jupyter.org/github/mfherbst/inria-devmeetup-julia-overview/blob/master/Presentation.ipynb">open it in nbviewer</a>.</p>
<table>
<thead>
<tr>
<th>Link</th>
</tr>
</thead>
<tbody>
<tr>
<td><a href="https://michael-herbst.com/talks/2019.04.16_Julia_Overview.ipynb">Julia: A numerical programming language</a> (Notebook lightning talk)</td>
</tr>
<tr>
<td><a href="https://nbviewer.jupyter.org/github/mfherbst/inria-devmeetup-julia-overview/blob/master/Presentation.ipynb">View notebook in nbviewer</a></td>
</tr>
</tbody>
</table>Coulomb Sturmian basis functions in electronic structure theory2019-04-12T11:00:00+02:002019-04-12T11:00:00+02:00Michael F. Herbsttag:michael-herbst.com,2019-04-12:/2019-lct-coulomb-sturmian-quantum-chemistry.html<p>After my first
<a href="https://michael-herbst.com/2018-jussieu-lct-molsturm.html">Coulomb Sturmian-related talk</a>
at the <a href="http://lct.jussieu.fr/">LCT</a> last November,
I was approached by Julien Toulouse and asked
whether I could give another talk providing a little more insight
into Coulomb Sturmians and their potential.</p>
<p>A few weeks ago, on 29th March, it was thus about time for a …</p><p>After my first
<a href="https://michael-herbst.com/2018-jussieu-lct-molsturm.html">Coulomb Sturmian-related talk</a>
at the <a href="http://lct.jussieu.fr/">LCT</a> last November,
I was approached by Julien Toulouse and asked
whether I could give another talk providing a little more insight
into Coulomb Sturmians and their potential.</p>
<p>A few weeks ago, on 29th March, it was thus about time for a second
Coulomb Sturmian seminar at LCT.
In my recent talk I focused more on stressing the differences between
Coulomb Sturmians and Gaussian basis sets
and motivated by our
<a href="https://michael-herbst.com/2018-cs-atomic-hf-convergence.html">convergence results at HF level</a>
provide some hints, what could be expected from Coulomb Sturmians
in the future.
Naturally this last part turned out to be rather far-fetched
and speculative and little if any bullet-proof evidence
for the mentioned aspects can be given at the moment.</p>
<p>I really enjoyed giving this talk in the arising
rather informal setting, where many stimulating questions
came up in the discussion with plenty of opportunity for
further research. As usual the slides are attached below.</p>
<table>
<thead>
<tr>
<th>Link</th>
</tr>
</thead>
<tbody>
<tr>
<td><a href="https://michael-herbst.com/talks/2019.03.29_coulomb_sturmians_jussieu.pdf">Coulomb Sturmian basis functions in electronic structure theory</a> (Slides seminar talk)</td>
</tr>
</tbody>
</table>ctx: Key-value datastructure for organised hierarchical storage2019-03-18T00:00:00+01:002019-03-18T00:00:00+01:00Michael F. Herbsttag:michael-herbst.com,2019-03-18:/ctxlibrary.html<p>As an aside from my <a href="https://michael-herbst.com/postdoc-cermics.html">plane-wave DFT</a> project
at CERMICS, I spent some time in the last few weeks polishing up
the <a href="https://github.com/mfherbst/ctx"><code>ctx</code> context library</a>
and publishing it via <a href="https://github.com/mfherbst/ctx">github</a>.
The <code>ctx</code> library provides one approach how to tackle a common challenge
in larger scientific simulation codes, namely how to …</p><p>As an aside from my <a href="https://michael-herbst.com/postdoc-cermics.html">plane-wave DFT</a> project
at CERMICS, I spent some time in the last few weeks polishing up
the <a href="https://github.com/mfherbst/ctx"><code>ctx</code> context library</a>
and publishing it via <a href="https://github.com/mfherbst/ctx">github</a>.
The <code>ctx</code> library provides one approach how to tackle a common challenge
in larger scientific simulation codes, namely how to organise
simulation data.</p>
<p>Often each substep of a larger simulation code by itself already requires
a large number of input data in order to produce its results.
To increase the flexibility and modularity of the code base,
the data/parameters required for one part of the procedure
are best hidden from the respective others.
Naturally, complete separation cannot be achieved,
since the steps do need to exchange some results or input.
The question is therefore where and how to find a good balance
between accessing data from other simulation steps and shielding them.</p>
<p>Typically, simulation procedures are inherently hierarchical.
E.g. the computation of a particular property
of an electronic structure might be achieved by solving
linear-response equation,
which in turn requires to solve a linear system of equations,
e.g. by a contraction-based iterative algorithm,
which in turn requires the computation of matrix-vector products.
In such an hierarchy of algorithms
data only needs to flow up or down, but not sideways.
This is to say that, e.g. computing a second property via
another linear-response equation,
might again be using a similar tree of methods,
but does not necessarily need access to all individual details
of the first tree.
This immediately leads to the approach
of a tree-like, hierarchical storage for data and parameters,
where a simulation substep may only work on its own subtree,
but not on any of its parents or siblings.</p>
<p>This is where the library <code>ctx</code> comes in.
It offers a <code>C++</code> implementation of a tree-like
string-to-value mapping, called <code>CtxMap</code>,
which allows to store
data of arbitrary type.
In line with above approach,
a key in a <code>CtxMap</code> is a path-like string such as
<code>/this/is/a/path/to/a/value</code>.
By means of functionality such as views into subtrees
or <code>C++</code> iterators over ranges of keys,
navigating and accessing the data from the <code>CtxMap</code>
from different parts of the code is greatly facilitated.
Since all key objects are stored as <code>std::shared_ptr</code>
objects, memory safety as well as good integration into the
<code>C++</code> standard library are assured.</p>
<p>Originally I started working on <code>ctx</code>,
when I wanted to connect our
<a href="https://michael-herbst.com/molsturm-design.html"><code>molsturm</code></a>
package to the <code>adcman</code> code
also developed
at the <a href="https://dreuw.qchem.de">Dreuw group in Heidelberg</a>.
In the design of <code>ctx</code> I took strong inspirations from both the
<a href="https://github.com/mfherbst/pammap">PamMap</a>
and the
<a href="https://github.com/lazyten/krims#genmap-a-hierachical-dictionary-for-managing-data-of-arbitrary-type">GenMap</a>
data structures I have been playing with,
as well as the <code>libctx</code> library by E. Epifanovsky <em>et. al.</em>.
The latter code had similar goals of providing
hierarchical storage of data in mind
and was used for this purpose inside the
<a href="https://q-chem.com">Q-Chem</a> quantum chemistry code.
From this respect I am very happy that
<code>ctx</code> has now become the successor of <code>libctx</code> inside Q-Chem,
providing <code>ctx</code> with an application in
the context of a larger code base.</p>
<p>For further details and the <code>ctx</code> source code, see the
<a href="https://github.com/mfherbst/ctx"><code>ctx</code> project page</a>
on github. <code>ctx</code> can be cited using
<a href="https://doi.org/10.5281/zenodo.2590706">DOI 10.5281/zenodo.2590706</a>.</p>PostDoc position at CERMICS / Sorbonne Université2019-01-09T00:00:00+01:002019-01-09T00:00:00+01:00Michael F. Herbsttag:michael-herbst.com,2019-01-09:/postdoc-cermics.html<p>Last week I started my new position as a postdoctoral fellow at
the <a href="https://team.inria.fr/matherials">MATHERIALS</a> team
of the <a href="https://cermics-lab.enpc.fr">Centre d'Enseignement et de Recherche en Mathématiques
et Calcul Scientifique (CERMICS)</a> laboratory and
<a href="https://www.inria.fr">Inria Paris</a> with an additional affiliation to the
<a href="http://iscd.upmc.fr/">Institut des Sciences du Calcul et des Données</a>
at <a href="https://www.sorbonne-universite.fr">Sorbonne Université</a>.
As …</p><p>Last week I started my new position as a postdoctoral fellow at
the <a href="https://team.inria.fr/matherials">MATHERIALS</a> team
of the <a href="https://cermics-lab.enpc.fr">Centre d'Enseignement et de Recherche en Mathématiques
et Calcul Scientifique (CERMICS)</a> laboratory and
<a href="https://www.inria.fr">Inria Paris</a> with an additional affiliation to the
<a href="http://iscd.upmc.fr/">Institut des Sciences du Calcul et des Données</a>
at <a href="https://www.sorbonne-universite.fr">Sorbonne Université</a>.
As can already be guessed from the names of the aforementioned labs,
each cover research topics in an interdisciplinary research field.
At CERMICS, for example, applied mathematicians are working
on topics related to the molecular sciences, optimisation
or finance.</p>
<p>Along this line my new direction of research stays interdisciplinary as well:
Using the programming language <a href="https://julialang.org"><code>julia</code></a>
we would like to provide a highly modular and flexible
plane-wave density-functional theory code for the modelling
of solid-state materials.
The main aim of the project is to allow for novel mathematical approaches
in this context to be readily implemented and tested on a scale
going beyond simple toy problems.
In order to achieve this, the code needs to stay simple and accessible,
but still perform decently
--- a challenge, which can in our opinion be tackled
rather well using <code>julia</code> and its rapidly-growing number of modules.
As <a href="https://michael-herbst.com/selfconsistentfieldjl.html">mentioned previously</a>
I like the modern concepts of the <code>julia</code> language very much
and thus I am eager about this opportunity to try it
for coding a complete framework for modelling a scientific problem.</p>
<p>Right now I know rather little about the topic of
modelling solid-state materials,
but fortunately I am not alone on this project.
Rather I will be working jointly with Antoine Levitt,
Eric Cancès, Laura Grigori and Eleonora Luppi
from either CERMICS, Inria or Sorbonne Université
on this project.</p>SelfConsistentField.jl: A julia toolbox for coding self-consistent field algorithms2018-12-29T00:00:00+01:002018-12-29T00:00:00+01:00Michael F. Herbsttag:michael-herbst.com,2018-12-29:/selfconsistentfieldjl.html<p>Over the past half a year I became interested
in the <a href="https://julialang.org"><code>julia</code></a> programming language.
It is a rather recent language (version 1.0.0 just released last August),
which promises to provide a modern approach to scientific computing.
From my experience so far, I like <code>julia</code> a lot.
All aspects …</p><p>Over the past half a year I became interested
in the <a href="https://julialang.org"><code>julia</code></a> programming language.
It is a rather recent language (version 1.0.0 just released last August),
which promises to provide a modern approach to scientific computing.
From my experience so far, I like <code>julia</code> a lot.
All aspects of the code, let it be parallelisation, vectorisation,
macros, generic code and so on, are consistently designed and fit together very well.
In contrast to e.g. <code>python</code>, it is thus not required
to employ a specialised external package
with its associated way of doing the computation in parallel for a
large-scale problem.
In <code>julia</code> parallelisation is usually automatic or can be added with little manual effort.
Whilst <code>julia</code> code might not necessarily beat C++ code with respect to
performance in all cases,
it certainly is written much faster.</p>
<p>To test <code>julia</code> in a larger setting than just a toy project,
I decided to work towards a <code>julia</code> toolbox for coding and experimenting
with self-consistent field algorithms.
Over the summer Tobias Wackenhut participated in the development
during his internship with us in Heidelberg.
After about two solid months of coding, mostly from his end,
a first version with support for solving Hartree-Fock problems
has now been implemented.</p>
<p>For further details, the project code and some examples, see the
<a href="https://github.com/mfherbst/SelfConsistentField.jl">SelfConsistentField.jl project page</a>
on github.</p>Design of the molsturm quantum-chemistry framework2018-12-23T18:00:00+01:002018-12-23T18:00:00+01:00Michael F. Herbsttag:michael-herbst.com,2018-12-23:/2018-jussieu-lct-molsturm.html<p>A few weeks ago, on 23rd November, I was invited to speak
at the
<a href="https://www.ljll.math.upmc.fr/fr/seminaires/article/gdt-mathematiques-et-chimie?lang=fr">mathematics and chemistry seminar</a>
(GdT mathématiques et chimie) at Sorbonne Université in Paris.
This seminar is a joint meeting of theoretical chemists and mathematicians from the
<a href="http://ljll.math.upmc.fr/">LJLL (Laboratoire Jacques-Louis Lions)</a>
and the <a href="http://www.lct.jussieu.fr/">LCT (Laboratoire de Chimie …</a></p><p>A few weeks ago, on 23rd November, I was invited to speak
at the
<a href="https://www.ljll.math.upmc.fr/fr/seminaires/article/gdt-mathematiques-et-chimie?lang=fr">mathematics and chemistry seminar</a>
(GdT mathématiques et chimie) at Sorbonne Université in Paris.
This seminar is a joint meeting of theoretical chemists and mathematicians from the
<a href="http://ljll.math.upmc.fr/">LJLL (Laboratoire Jacques-Louis Lions)</a>
and the <a href="http://www.lct.jussieu.fr/">LCT (Laboratoire de Chimie Théorique)</a>
of Sorbonne Université
and the <a href="https://cermics-lab.enpc.fr/">CERMICS laboratory</a>
of <a href="https://inria.fr/centre/paris">Inria Paris</a>
and <a href="http://www.enpc.fr">ENPC (Ecole des Ponts Paris Tech)</a>.
Such a mixed audience provided a good opportunity to present
our recent work on
the <a href="https://michael-herbst.com/molsturm-design.html"><code>molsturm</code></a>
quantum chemistry framework and our recent
<a href="https://michael-herbst.com/2018-cs-atomic-hf-convergence.html">Coulomb-Sturmian convergence results</a>.
Both the feedback and the discussion accompanying
the talk were fruitful and some ideas
how to provide a more mathematical backbone for our empirical observations emerged.</p>
<p>My travels to Paris were supported by the
<a href="https://daad.de">DAAD</a>,
which I gratefully acknowledge.
The slides of my talk are attached below:</p>
<table>
<thead>
<tr>
<th>Link</th>
</tr>
</thead>
<tbody>
<tr>
<td>Slides <a href="https://michael-herbst.com/talks/2018.11.22_molsturm_LCT_Jussieu.pdf"><em>Design of the <code>molsturm</code> quantum-chemistry framework</em></a></td>
</tr>
</tbody>
</table>Quantum chemistry with Coulomb Sturmians: Construction and convergence of Coulomb Sturmian basis sets at Hartree-Fock level2018-11-15T11:00:00+01:002018-11-15T11:00:00+01:00Michael F. Herbsttag:michael-herbst.com,2018-11-15:/2018-cs-atomic-hf-convergence.html<p>This week we submitted a paper related to our ongoing work
regarding the use of
<a href="https://michael-herbst.com/research/coulomb_sturmians/">Coulomb Sturmian basis functions in electronic structure theory</a>.
Given the recent progress, which has been made with regards to the evaluation
of molecular integrals based on these exponential-type functions,
molecular calculations based on Coulomb Sturmians …</p><p>This week we submitted a paper related to our ongoing work
regarding the use of
<a href="https://michael-herbst.com/research/coulomb_sturmians/">Coulomb Sturmian basis functions in electronic structure theory</a>.
Given the recent progress, which has been made with regards to the evaluation
of molecular integrals based on these exponential-type functions,
molecular calculations based on Coulomb Sturmians are within reach.
This is a very promising prospect due to the abilities of the Coulomb Sturmians
to represent the features of the wave function both at the nucleus
as well as at large distances.</p>
<p>Our work takes a first look at the convergence of Coulomb Sturmian
discretisations in electronic structure theory,
expanding on ideas suggested in my
<a href="https://michael-herbst.com/phd-thesis.html">PhD thesis</a>.
We suggest a simple construction scheme for Coulomb Sturmian basis sets
and discuss its properties.
A key aspect is to connect the basis set parameters to physical features
of the wave function or other chemically intuitive quantities
such as the exponents obtained by Clementi and Raimondi,
i.e. the <strong>effective nuclear charge</strong>.
The abstract of the paper reads</p>
<blockquote>
<p>The first discussion
of basis sets consisting of exponentially
decaying Coulomb Sturmian functions
for modelling electronic structures is presented.
The proposed basis set construction
selects Coulomb Sturmian functions using separate upper limits to
their principle, angular momentum and magnetic quantum numbers.
Their common Coulomb Sturmian exponent is taken as a fourth parameter.
The convergence properties of such basis sets
are investigated
for second and third row atoms at the Hartree-Fock level.
Thereby important relations between the values of the basis set parameters
and the physical properties of the electronic structure are recognised.
For example, an unusually large limit for the angular momentum quantum
number in unrestricted Hartree-Fock calculations
can be linked to the breaking of spherical symmetry in such cases.
Furthermore, a connection between the optimal,
i.e. minimum-energy, Coulomb Sturmian exponent
and the average Slater exponents values
obtained by Clementi and Raimondi (<a href="https://doi.org/10.1063/1.1733573">E. Clementi and D. L. Raimondi, J. Chem. Phys. 38, 2686 (1963)</a>)
is made.
These features of Coulomb Sturmian basis sets
emphasise their ability to correctly reproduce
the physical features of Hartree-Fock wave functions.</p>
</blockquote>Simulating chemistry: Enabling novel approaches for modelling the electronic structure of molecules2018-09-30T18:00:00+02:002018-09-30T18:00:00+02:00Michael F. Herbsttag:michael-herbst.com,2018-09-30:/2018-hlf-simulating-chemistry.html<p>Last week Wednesday I was invited to present my research
in front of a bunch of young researchers from the
<a href="https://www.heidelberg-laureate-forum.org/event_2018/">Heidelberg Laureate Forum</a>
visiting our institute. Whilst I would have loved to attend the full week
of the forum, I am happy that in this way I was able to …</p><p>Last week Wednesday I was invited to present my research
in front of a bunch of young researchers from the
<a href="https://www.heidelberg-laureate-forum.org/event_2018/">Heidelberg Laureate Forum</a>
visiting our institute. Whilst I would have loved to attend the full week
of the forum, I am happy that in this way I was able to contribute a little
to this meeting and participate in discussions with other participants.</p>
<p>Instead of too much going into the details of my work, I decided to provide a general
overview of modern quantum-chemical approaches towards modelling
chemical properties and reactions.
Only very briefly, at the end of my talk, I very briefly talk about
the work concerning
our recent <a href="https://michael-herbst.com/molsturm-design.html"><code>molsturm</code></a> paper.
As usual my slides are attached below:</p>
<table>
<thead>
<tr>
<th>Link</th>
</tr>
</thead>
<tbody>
<tr>
<td>Slides <a href="https://michael-herbst.com/talks/2018.09.26_laureate_forum_simulating_chemistry.pdf"><em>Simulating chemistry</em></a></td>
</tr>
</tbody>
</table>Franco-German Workshop 2018: Open problems related to the algebraic-diagrammatic construction scheme2018-09-18T18:00:00+02:002018-09-18T18:00:00+02:00Michael F. Herbsttag:michael-herbst.com,2018-09-18:/2018-franco-german-aachen.html<p>In some of my recent projects, e.g. the <a href="https://michael-herbst.com/pe-adc-prospect.html">PE-ADC paper</a>,
I got a little more into method development
surrounding the algebraic-diagrammatic construction scheme (ADC).
Even though quite a few people from the Dreuw group in Heidelberg
are involved with ADC development and we have regular talks on ADC,
I …</p><p>In some of my recent projects, e.g. the <a href="https://michael-herbst.com/pe-adc-prospect.html">PE-ADC paper</a>,
I got a little more into method development
surrounding the algebraic-diagrammatic construction scheme (ADC).
Even though quite a few people from the Dreuw group in Heidelberg
are involved with ADC development and we have regular talks on ADC,
I did not yet have the chance to really take a deeper look
at this method myself.</p>
<p>Last week I was invited to the
<a href="http://www.mathcces.rwth-aachen.de/5people/stamm/workshop2018">Franco-German Workshop on mathematical aspects in computational chemistry 2018</a>,
where a selection of chemists and mathematicians from France and Germany
and a few other places got together to talk about their recent advances,
but most importantly to discuss.
From the ADC side I had already collected a few potential angles for improvement
in recent month and this occasion thus seemed ideal
to both brush up on my knowledge
of ADC and at the same time present on ADC.</p>
<p>In the spirit of such a rather hands-on-type workshop,
I only quickly glanced over the current status of ADC as an excited-states method.
Afterwards introduced common numerical procedures
and pointed at currently open problems and challenges.
From the discussions directly after the lecture as well as the
remaining days of the workshop,
a few interesting ideas emerged and I am already looking forward
to trying them out.
As per usual the slides of my talk are attached below.</p>
<table>
<thead>
<tr>
<th>Link</th>
</tr>
</thead>
<tbody>
<tr>
<td>Slides <a href="https://michael-herbst.com/talks/2018.09.11_franco_german_adc.pdf"><em>Challenges and open problems related to the algebraic diagrammatic construction scheme</em></a></td>
</tr>
</tbody>
</table>MRMCD18: Pitfalls for performance: Latencies to keep in mind2018-09-09T20:00:00+02:002018-09-09T20:00:00+02:00Michael F. Herbsttag:michael-herbst.com,2018-09-09:/mrmcd18-pitfalls-performance.html<p>This weekend I attended the <a href="https://2018.mrmcd.net">MRMCD18</a>
in Darmstadt, which is one of the many chaos events where a few hundred people
interested in computer science, hacking, making, politics and society
gather for a weekend of talks, coding and discussion. Having participated
in this event regularly in the past few years …</p><p>This weekend I attended the <a href="https://2018.mrmcd.net">MRMCD18</a>
in Darmstadt, which is one of the many chaos events where a few hundred people
interested in computer science, hacking, making, politics and society
gather for a weekend of talks, coding and discussion. Having participated
in this event regularly in the past few years, I was very happy
that this year I was given the chance to present a lecture.</p>
<p>I chose to talk about lazy matrices and contraction-based algorithms.
Even though this topic is discussed briefly in my
<a href="https://michael-herbst.com/phd-thesis.html">PhD thesis</a>,
I typically keep it rather short in my typical talks.
Similarly the
<a href="https://michael-herbst.com/molsturm-design.html"><code>molsturm</code> design paper</a>
was only brief with respect to this subject.
In this talk I therefore took the chance to expand
on contraction-based methods a bit further,
tackling the matter by taking a look at latency numbers in modern hardware
and from there drawing conclusions with respect to numerical linear algebra.</p>
<p>Thanks to the great <a href="https://c3voc.de">C3VOC</a> team
the talk was recorded live and can be
<a href="https://media.ccc.de/v/2018-134-pitfalls-for-performance-typical-latencies-to-keep-in-mind">watched on media.ccc.de</a>
or downloaded below, along with my slides.</p>
<p><strong>Note:</strong> Unfortunately the batteries of the microphone gave up
in the middle of my talk, around 25:20.
However, if you seek the video forward to minute 26:17,
I switch to a handheld and audio is back.</p>
<table>
<thead>
<tr>
<th>Link</th>
</tr>
</thead>
<tbody>
<tr>
<td><a href="https://michael-herbst.com/talks/2018.09.08_mrmcd18_pitfalls_performance_latencies.mp4">Recording <em>Pitfalls for performance</em></a>: <br /><em>Latencies to keep in mind</em></td>
</tr>
<tr>
<td><a href="https://michael-herbst.com/talks/2018.09.08_mrmcd18_pitfalls_performance_latencies.pdf">Slides <em>Pitfalls for performance</em></a></td>
</tr>
</tbody>
</table>MES 2018 in Metz: molsturm Towards quantum-chemical method development for arbitrary basis functions2018-08-28T18:00:00+02:002018-08-28T18:00:00+02:00Michael F. Herbsttag:michael-herbst.com,2018-08-28:/2018-mes-metz.html<p>Currently I am attending the
<a href="https://mesm.event.univ-lorraine.fr/">Molecular Electronic Structure 2018</a>
conference in Metz, an electronic-structure theory conference
intended for researchers working on novel methodologies,
such as explicit correlation, density matrix theory, stochastic approaches
or on discretisation techniques
going beyond Gaussian-type orbitals.</p>
<p>Especially the latter aspect fits well to the kind of …</p><p>Currently I am attending the
<a href="https://mesm.event.univ-lorraine.fr/">Molecular Electronic Structure 2018</a>
conference in Metz, an electronic-structure theory conference
intended for researchers working on novel methodologies,
such as explicit correlation, density matrix theory, stochastic approaches
or on discretisation techniques
going beyond Gaussian-type orbitals.</p>
<p>Especially the latter aspect fits well to the kind of research direction
we want to support with our <a href="https://molsturm.org"><code>molsturm</code></a> program package.
Thanks to the organisers of the conference,
namely Ugo Ancarani and his committee,
I was able to present the design and ideas of <code>molsturm</code> in a
short contributed talk.
The talk essentially became a summary of our
<a href="https://michael-herbst.com/molsturm-design.html"><code>molsturm</code> design paper</a>,
which just got published.</p>
<p>Unfortunately 15 minutes are not enough to explain everything
in detail, so I had to glance over many aspects in my talk.
The focus ended up being mainly our basis-function independent,
contraction-based self-consistent field scheme,
which sits at <code>molsturm</code>'s heart, bringing together integral
libraries for various basis function types and Post-HF routines.
I furthermore briefly hinted at <code>molsturm</code>'s lightweight
structure with easy-to-use interfaces that allow to combine
existing integral and Post-HF codes with <code>molsturm</code>.
The driving force here is to avoid implementing
all quantum-chemical methods from scratch
for each new basis function type and much rather use
what codes already exists.
As I outlined in my talk, a basis-function independent SCF
code is a key step to achieve this.</p>
<p>I am thankful for all the questions and comments I received
during the talk and the conference so far. As per usual the slides
are attached below.</p>
<table>
<thead>
<tr>
<th>Link</th>
</tr>
</thead>
<tbody>
<tr>
<td>Slides <a href="https://michael-herbst.com/talks/2018.08.28_MES_2018_molsturm_development_arbitrary_basis_function.pdf"><em><code>molsturm</code>: Towards quantum-chemical method development for arbitrary basis functions</em></a></td>
</tr>
</tbody>
</table>Embedding Combined with the Algebraic Diagrammatic Construction: Tackling Excited States in Biomolecular Systems2018-08-08T23:00:00+02:002018-08-08T23:00:00+02:00Michael F. Herbsttag:michael-herbst.com,2018-08-08:/pe-adc-prospect.html<p>Yesterday the journal of chemical theory and computation
accepted our manuscript titled <strong>Polarizable Embedding Combined with the
Algebraic Diagrammatic Construction: Tackling Excited States in Biomolecular Systems</strong>.
In this work we describe the combination of the polarisable embedding (PE) scheme for
including environmental effects with the
algebraic-diagrammatic construction (ADC) approach.</p>
<p>The …</p><p>Yesterday the journal of chemical theory and computation
accepted our manuscript titled <strong>Polarizable Embedding Combined with the
Algebraic Diagrammatic Construction: Tackling Excited States in Biomolecular Systems</strong>.
In this work we describe the combination of the polarisable embedding (PE) scheme for
including environmental effects with the
algebraic-diagrammatic construction (ADC) approach.</p>
<p>The main idea of <a href="https://doi.org/10.1021/ct1003803"><strong>polarisable embedding</strong></a>
is to model the effect an environment
has on a core system by means of modelling the classical electrostatic
interactions between both.
For this the potential generated by the electron density of the environment
is expanded in a multipole series, which is typically truncated
at the level of dipoles. Furthermore effect of the core
on the environment itself is taken into account by allowing for
induced dipoles in the environment next the aforementioned static
dipoles of the multipole expansion.
Instead of modelling the full details, the environment
is thus reduced to a set of static dipole moments and polarisabilities,
which are located at some predefined sites,
typically the position of the nuclei.
During the SCF procedure of the core region
the dipoles induced by the core are optimised self-consistently
along the SCF, such that after the SCF procedure
a consistent electrostatic potential of the environment
is included inside the core density.</p>
<p>The reduction of the environment to a set of dipoles and polarisabilities
has the advantage, that these parameters can be determined
<em>a priori</em> based on a comparatively cheap computational method
and a standardised
fitting procedure (e.g. <a href="https://doi.org/10.1063/1.1778131">LoProp</a>).
This implies that one can reduce the size of the system,
which needs to be modelled by a more expensive method.
One example would be the
<a href="https://doi.org/10.1002/wcms.1206"><strong>algebraic-diagrammatic construction scheme for the polarisation propagator</strong></a>
for computing so-called excited states,
which we connect with polarisable embedding in this work.
In order to achieve this properly one has to do a little more
than self-consistently finding the electrostatic embedding potential.
Namely, one has to keep in mind that an electronic excitation
changes the electronic structure of the core, thus changes
the polarisation on the embedding and thus again the electronic
structure of the core. Taking this effect fully into account
would require another self-consistent iteration back and forth
between ADC and the PE scheme.
One can avoid this by treating this effect only approximately
via linear response as well as perturbative corrections
on top of an ADC calculation.</p>
<p>Apart from describing these corrections and their connections
to physical quantities,
we also hint at another important aspect
of the simulation procedure, namely finding sensible
arrangements for the environment atoms. Since the environment
often consists of a solvent or a protein,
it tends to be much more flexible compared to the core system.
For this reason one cannot merely pick a single random molecular arrangement
for the environment, but one needs to sample and perform the full
calculation (that is ADC on the core) multiple times
in order to gain some statistical results, which are a meaningful model
for the exact physical system.</p>
<p>Last but not least we demonstrate the applicability of
PE-ADC using three examples,
namely a water cluster environment around para-nitroaniline,
lumiflavin in bulk water
and lumiflavin inside a dodecin protein.
For each of the cases a detailed analysis of the results
and comparison with existing methods is presented including
statistical analysis of the sampling employing
<a href="https://en.wikipedia.org/wiki/Box_plot">box plots</a>.</p>
<p>The full abstract of the paper reads as follows:</p>
<blockquote>
<p>We present a variant of the algebraic diagrammatic construction (ADC) scheme
by combining ADC with the polarizable embedding (PE) model. The presented
PE-ADC method is implemented through second and third order and is designed
with the aim of performing accurate calculations of excited states in large
molecular systems. Accuracy and large-scale applicability are demonstrated
with three case studies, and we further analyze the importance of both
state-specific and linear-response-type corrections to the excitation
energies in the presence of the polarizable environment. We demonstrate
how our combined method can be readily applied to study photo-induced
biochemical processes as we model the charge-transfer (CT) excitation which
is key to the photoprotection mechanism in the dodecin protein with PE-ADC(2).
Through direct access to state-of-the-art excited state analysis, we find that
the polarizable environment plays a decisive role by significantly increasing
the CT character of the electronic excitation in dodecin. PE-ADC is thus
suited to decipher photoinduced processes in complex, biomolecular systems at
high precision and at reasonable computational cost.</p>
</blockquote>Coulomb Sturmians in electronic structure theory (Poster at 16th ICQC)2018-06-25T00:00:00+02:002018-06-25T00:00:00+02:00Michael F. Herbsttag:michael-herbst.com,2018-06-25:/2018-icqc-menton.html<p>Last week from 18th till 23rd June I attended the
<a href="https://icqc16.sciencesconf.org">16th International Congress of Quantum Chemistry</a> (ICQC)
in Menton, France.
Apart from the absolutely stunning scenery at the French Riviera
as well as the historic conference building,
the conference was a great opportunity to meet up and discuss with
people …</p><p>Last week from 18th till 23rd June I attended the
<a href="https://icqc16.sciencesconf.org">16th International Congress of Quantum Chemistry</a> (ICQC)
in Menton, France.
Apart from the absolutely stunning scenery at the French Riviera
as well as the historic conference building,
the conference was a great opportunity to meet up and discuss with
people from the community.
Often the discussions continued
in one of the plenty of bars in the old town
or right next to the beach.
Even though the program was pretty packed,
I fortunately still had the chance
to walk around and enjoy the Mediterranean atmosphere
in Menton as well as the sun and the sea.</p>
<p>During the conference I presented a poster titled
<strong>Coulomb Sturmians in electronic structure theory</strong>,
where I discussed my recent convergence results on Coulomb Sturmians,
just recently published in my
<a href="https://michael-herbst.com/phd-thesis.html">PhD thesis</a>.
The poster pdf is attached below.</p>
<table>
<thead>
<tr>
<th>Link</th>
</tr>
</thead>
<tbody>
<tr>
<td>Poster <a href="https://michael-herbst.com/poster/2018.06.19_ICQC_Convergence_CS_functions.pdf"><em>Coulomb Sturmians in electronic structure theory</em></a></td>
</tr>
</tbody>
</table>Towards quantum-chemical method development for arbitrary basis functions2018-06-24T00:00:00+02:002018-06-24T00:00:00+02:00Michael F. Herbsttag:michael-herbst.com,2018-06-24:/molsturm-design.html<p>Last week we submitted the <a href="https://molsturm.org">molsturm</a>
design paper titled <strong>Towards quantum-chemical method development for arbitrary basis functions</strong>.
This manuscript is a condensed summary of the arguments about design
and implementation of a basis-function independent self-consistent field scheme (SCF)
already presented in my <a href="https://michael-herbst.com/phd-thesis.html">PhD thesis</a>.</p>
<p>Our approach to a basis-function-independent electronic …</p><p>Last week we submitted the <a href="https://molsturm.org">molsturm</a>
design paper titled <strong>Towards quantum-chemical method development for arbitrary basis functions</strong>.
This manuscript is a condensed summary of the arguments about design
and implementation of a basis-function independent self-consistent field scheme (SCF)
already presented in my <a href="https://michael-herbst.com/phd-thesis.html">PhD thesis</a>.</p>
<p>Our approach to a basis-function-independent electronic structure theory code
is to employ <strong>contraction-based methods</strong>
for solving the Hartree-Fock (HF) problem.
The idea of this ansatz is to avoid storing the Fock matrix,
i.e. the matrix representation of the operator underlying the HF problem,
in memory.
Instead one focuses on only employing optimised expressions for the
product of this matrix with a set of trial vectors
in the SCF procedure for solving HF.
As the paper discusses in more detail
this is possible,
since all SCF algorithms known to us can be performed
based upon iterative procedures,
which only require a matrix-vector product.</p>
<p>An advantage of the contraction-based SCF
is that the basis-function-dependent structure of the Fock matrix
can be hidden inside the expression for the matrix-vector product.
As a result the SCF code becomes independent of the basis function choice
and new basis functions can be added
to our SCF procedure in a plug and play fashion just by linking to
appropriate integral libraries.
The abstract of the paper reads</p>
<blockquote>
<p>We present the design of a flexible quantum-chemical
method development framework,
which supports
employing any type of basis function.
This design has been implemented in the
light-weight program package <code>molsturm</code>,
yielding a basis-function-independent self-consistent field scheme.
Versatile interfaces,
making use of open standards like <code>python</code>,
mediate the integration of <code>molsturm</code> with existing third-party packages.
In this way both rapid extension of the present set of
methods for electronic structure calculations
as well as adding new basis function types
can be readily achieved.
This makes <code>molsturm</code> well-suitable for testing
novel approaches for discretising the electronic wave function
and allows comparing them
to existing methods using the same software stack.
This is illustrated by two examples,
an implementation of coupled-cluster doubles
as well as a gradient-free geometry optimisation,
where in both cases,
an arbitrary basis functions could be used.
<code>molsturm</code> is open-source and can be obtained from <a href="http://molsturm.org">http://molsturm.org</a>.</p>
</blockquote>Aachen: Beyond Gaussian-type orbitals2018-06-03T12:00:00+02:002018-06-03T12:00:00+02:00Michael F. Herbsttag:michael-herbst.com,2018-06-03:/aachen-beyond-gaussian-type-orbitals.html<p>From Monday till Wednesday this week I was invited to the computational mathematics
research group of
<a href="http://www.mathcces.rwth-aachen.de/5people/stamm/start">Prof. Dr. Benjamin Stamm</a>
at the RWTH Aachen University.
In retrospect I am very thankful for the invitation,
since the three days turned out to be filled with
fruitful discussions, scientific exchange and brainstorming …</p><p>From Monday till Wednesday this week I was invited to the computational mathematics
research group of
<a href="http://www.mathcces.rwth-aachen.de/5people/stamm/start">Prof. Dr. Benjamin Stamm</a>
at the RWTH Aachen University.
In retrospect I am very thankful for the invitation,
since the three days turned out to be filled with
fruitful discussions, scientific exchange and brainstorming of ideas.
I really enjoyed the interdisciplinary spirit of the group,
where research is conducted
on a broad spectrum of topics
ranging from developing fundamental models for classical dynamics,
electrostatics and physical processes in batteries
to domain decomposition methods for electronic structure theory.
I overall found it very enlightening to talk to the
PhD students and Post Docs of the group, since the background
of the group members is almost as diverse as the topics these people work on.</p>
<p>Last Tuesday I was invited to give a talk as part of the
<a href="http://www.mathcces.rwth-aachen.de/4events/start">MathCCES lunch seminar</a>
talk series. I decided to present an overview of the discretisation methods
commonly employed in electronic structure theory.
In my talk titled
<strong>State of the basis in electronic structure theory:
Sketching the scene beyond Gaussian-type orbitals</strong>
I discussed and contrasted a range of basis function types,
which may be used for numerical modelling in quantum chemistry.
Special focus was put on uncommon basis function types,
that is basis sets different from contracted Gaussian-type orbitals.
Overall this talk could be considered an extension of <strong>section 5.3</strong> of my
<a href="https://michael-herbst.com/phd-thesis.html"><strong>PhD thesis</strong></a>,
where a subset of the basis function types mentioned in the talk
are discussed in greater depth and elaborating more on the details
with respect to applying them for
solving the Hartree-Fock self-consistent field problem.
As usual the slides are attached below.</p>
<table>
<thead>
<tr>
<th>Link</th>
</tr>
</thead>
<tbody>
<tr>
<td>Slides <a href="https://michael-herbst.com/talks/2018.05.29_aachen_beyond_gaussian_type_orbitals.pdf"><em>Beyond Gaussian-type orbitals</em></a></td>
</tr>
</tbody>
</table>Publication of my PhD thesis2018-06-01T08:00:00+02:002018-06-01T08:00:00+02:00Michael F. Herbsttag:michael-herbst.com,2018-06-01:/phd-thesis.html<p>Finally, after quite some years of work I have successfully defended my PhD thesis
with the title <strong>Development of a modular quantum-chemistry framework
for the investigation of novel basis functions</strong> on 22nd May.
An electronic version of the text has now been <a href="https://michael-herbst.com/phd-thesis.html#documents">published online</a>.</p>
<p>To make the topic of my …</p><p>Finally, after quite some years of work I have successfully defended my PhD thesis
with the title <strong>Development of a modular quantum-chemistry framework
for the investigation of novel basis functions</strong> on 22nd May.
An electronic version of the text has now been <a href="https://michael-herbst.com/phd-thesis.html#documents">published online</a>.</p>
<p>To make the topic of my work accessible both for chemists as well as mathematicians
the thesis introduces the numerical modelling of chemical systems
from a rather mathematical perspective.
For example the first chapters are devoted to <strong>spectral theory</strong> as well
as <strong>Ritz-Galerkin</strong> discretisation techniques and
iterative <strong>diagonalisation algorithms</strong>.
In the following chapters the main equations underlying quantum chemical modelling,
i.e. the <strong>Schrödinger equation</strong> as well as the <strong>Hartree-Fock approximation</strong>,
are presented stating known results about their mathematical properties.
Afterwards the focus is shifted to numerical techniques for <strong>solving Hartree-Fock</strong>
and further to developing a unified algorithmic approach for dealing
with <strong>self-consistent field</strong> problems irrespective of the basis function type employed.
This leads to a discussion of <strong>contraction-based methods</strong>,
<strong>lazy matrices</strong> as well as the <a href="https://molsturm.org"><code>molsturm</code></a> package.
Last but not least convergence results for Coulomb-Sturmian-based quantum chemistry
are presented.<br>
For more details see the <a href="https://michael-herbst.com/phd-thesis.html#abstract">full abstract</a> given below.</p>
<h1>Published thesis versions</h1>
<p>My thesis is made available in three forms.
Firstly the officially published documents can be accessed using the
DOI <a href="https://doi.org/10.11588/heidok.00024519">10.11588/heidok.00024519</a>.
Please use this DOI as well as
<a href="https://michael-herbst.com/publications/2018.05_phd.bib">this bibtex entry</a> for citations.</p>
<p>Secondly the <a href="https://github.com/mfherbst/dissertation">LaTeX source code</a>
is made available on github.
I will keep amending this repository as errata and typos are discovered.
Note, that I deliberately do <em>not</em> publish this code as free software yet,
since there are a few legal issues I want to clearify first.
Please let me know, however, if you wish to use some of the LaTeX and CMake code
and I am sure we can work something out.</p>
<p>Thirdly, from time to time I plan on making new "releases" out of the
amended LaTeX code on github. The link
<a href="https://michael-herbst.com/publications/2018.05_phd_corrected.pdf">https:/michael-herbst.com/publications/2018.05_phd_corrected.pdf</a>
will be used to point to the most recent version.</p>
<h1 id="abstract">Full thesis abstract</h1>
<p>State-of-the-art methods
for the calculation of electronic structures of molecules
predominantly use Gaussian basis functions.
The algorithms employed inside existing code packages
are consequently often highly optimised keeping
only their numerical requirements in mind.
For the investigation of novel approaches,
utilising other basis functions, this is an obstacle,
since requirements might differ.
In contrast, this thesis develops
the highly flexible program package <code>molsturm</code>,
which is designed in order to facilitate rapid design,
implementation and assessment
of methods employing different basis function types.
A key component of <code>molsturm</code> is
a Hartree-Fock (HF) self-consistent field (SCF) scheme,
which is suitable to be combined
with any basis function type.</p>
<p>First
the mathematical background of quantum mechanics
as well as some numerical techniques are reviewed.
Care is taken to emphasise the often overlooked subtleties when
discretising an infinite-dimensional spectral problem
in order to obtain a finite-dimensional eigenproblem.
Common quantum-chemical methods such as full configuration interaction
and HF are discussed providing insight
into their mathematical properties.
Different formulations of HF are contrasted
and appropriate (SCF) solution schemes formulated.</p>
<p>Next discretisation approaches based on four different types of basis functions
are compared
both with respect to the computational challenges as well as
their ability to describe the physical features of the wave function.
Besides (1) Slater-type orbitals and (2) Gaussian-type orbitals,
the discussion considers (3) finite elements,
which are piecewise polynomials on a grid,
as well as (4) Coulomb-Sturmians,
which are the analytical solutions to a Schrödinger-like equation.
A novel algorithmic approach based on matrix-vector contraction
expressions is developed,
which is able to adapt to the numerical requirements of all basis functions considered.
It is shown that this ansatz not only allows to formulate
SCF algorithms in a basis-function independent way,
but furthermore improves the theoretically achievable
computational scaling for finite-element-based discretisations
as well as performance improvements for Coulomb-Sturmian-based discretisations.
The adequacy of standard SCF algorithms with respect to a contraction-based
setting is investigated
and for the example of the optimal damping algorithm
an approximate modification to achieve such a setting is presented.</p>
<p>With respect to recent trends in the development of modern computer hardware
the potentials and drawbacks of contraction-based approaches are evaluated.
One drawback, namely the typically more involved and harder-to-read code,
is identified and a data structure named lazy matrix is introduced
to overcome this.
Lazy matrices are a generalisation of the usual matrix concept,
suitable for encapsulating contraction expressions.
Such objects still look like matrices from the user perspective,
including the possibility to perform operations like matrix sums and products.
As a result programming contraction-based algorithms
becomes similarly convenient as working with normal matrices.
An implementation of lazy matrices in the <code>lazyten</code> linear algebra library
is developed in the course of the thesis, followed by an example
demonstrating the applicability in the context of the HF problem.</p>
<p>Building on top of the aforementioned concepts the design of <code>molsturm</code> is outlined.
It is shown how a combination of lazy matrices and a contraction-based SCF scheme
separates the code describing the SCF procedure
from the code dealing with the basis function type.
It is discussed how this allows to add a new basis function type
to <code>molsturm</code> by only making code changes in a single integral interface library.
On top of that, we demonstrate by the means of examples
how the readily scriptable interface of <code>molsturm</code>
can be employed to implement and assess novel quantum-chemical methods
or to combine the features of <code>molsturm</code> with existing third-party packages.</p>
<p>Finally, the thesis discusses an application of <code>molsturm</code>
towards the investigation of the convergence properties
of Coulomb-Sturmian-based quantum-chemical calculations.
Results for the convergence
of the ground-state energies at HF level
are reported for atoms of the second and the third period
of the periodic table.
Particular emphasis is put on a discussion about the required
maximal angular momentum quantum numbers
in order to achieve convergence
of the discretisation of the angular part of the wave function.
Some modifications required for a treatment at correlated level are suggested,
followed by a discussion of the effect of the Coulomb-Sturmian exponent.
An algorithm for obtaining an optimal exponent is devised
and some optimal exponents for the atoms of the
second and the third period of the periodic table at HF level are given.
Furthermore, the first results of a Coulomb-Sturmian-based excited
states calculation based on the algebraic-diagrammatic construction
scheme for the polarisation propagator are presented.</p>
<h1>Acknowledgements</h1>
<p>No doubt the research leading up to a PhD thesis cannot be successfully achieved
without the help and support from many people, both scientifically
as well as on a personal level. In the past years I have been very lucky
to be surrounded by people, who created an environment
for fruitful discussion. I am happy for everyone from related fields
of science and technology, who broadened my own perspective by getting me in touch
with new and unusual ideas and who helped me escape the box of my own narrow thinking.
Many people should be named in this respect,
most of them are already given credit in the acknowledgement section of my
<a href="https://michael-herbst.com/publications/2018.05_phd_corrected.pdf">PhD thesis</a>,
so let me be rather brief here and simply say thanks!</p>
<h1 id="documents">Download thesis</h1>
<p>To <a href="https://michael-herbst.com/publications/2018.05_phd_corrected.pdf">download the most recent version</a>
including <strong>corrections</strong> for all known errors and typos
use <a href="https://michael-herbst.com/publications/2018.05_phd_corrected.pdf">this link</a>.
To <strong>cite the thesis</strong> please use
<a href="https://michael-herbst.com/publications/2018.05_phd.bib">this bibtex entry</a>. </p>Oberwolfach reports: Molsturm modular electronic structure theory framework2018-04-16T18:00:00+02:002018-04-16T18:00:00+02:00Michael F. Herbsttag:michael-herbst.com,2018-04-16:/2018_oberwolfach_report_molsturm.html<p>As announced in my
<a href="https://michael-herbst.com/2018-oberwolfach.html">previous post</a>
about the Oberwolfach workshop on
<a href="https://www.mfo.de/occasion/1812">Mathematical Methods in Quantum Chemistry</a>,
the proceedings of each Oberwolfach workshop are published
in the form of an article in the
<a href="https://doi.org/10.4171/owr">Oberwolfach reports</a>.
For this purpose each speaker is asked for an extended abstract
summarising the content of his …</p><p>As announced in my
<a href="https://michael-herbst.com/2018-oberwolfach.html">previous post</a>
about the Oberwolfach workshop on
<a href="https://www.mfo.de/occasion/1812">Mathematical Methods in Quantum Chemistry</a>,
the proceedings of each Oberwolfach workshop are published
in the form of an article in the
<a href="https://doi.org/10.4171/owr">Oberwolfach reports</a>.
For this purpose each speaker is asked for an extended abstract
summarising the content of his or her talk.
I have finally submitted the summary of my contribution,
namely some quick introduction into Coulomb-Sturmians,
lazy matrices and the <code>molsturm</code> framework.
The introductory paragraph reads as follows.</p>
<blockquote>
<p>The talk gives an overview of our recent efforts to simplify the investigation of
novel types of basis functions for quantum-chemical calculations. Motivated by
Coulomb-Sturmians as basis functions for quantum-chemistry simulations, the design
of the flexible and light-weight quantum-chemical method development framework
<code>molsturm</code> is outlined, where new discretisation methods for electronic
structure theory calculations can be easily implemented and tested.</p>
</blockquote>
<p>A preprint of the full text is available below
and some more details regarding the Oberwolfach seminar
as well as my talk can be found
in my <a href="https://michael-herbst.com/2018-oberwolfach.html">previous post</a>.</p>Oberwolfach workshop: Mathematical Methods in Quantum Chemistry2018-03-28T00:00:00+02:002018-03-28T00:00:00+02:00Michael F. Herbsttag:michael-herbst.com,2018-03-28:/2018-oberwolfach.html<p>Last week I was invited to the
<a href="https://www.mfo.de">Mathematical Research Institute Oberwolfach</a>
(Mathematisches Forschungsinstitut Oberwolfach)
for a workshop about
<a href="https://www.mfo.de/occasion/1812">Mathematical Methods in Quantum Chemistry</a>.
I am really grateful to the organisers
Eric Cancès,
Gero Friesecke,
Trygve Helgaker
and Lin Lin
for asking me to join this meeting
as it turned out …</p><p>Last week I was invited to the
<a href="https://www.mfo.de">Mathematical Research Institute Oberwolfach</a>
(Mathematisches Forschungsinstitut Oberwolfach)
for a workshop about
<a href="https://www.mfo.de/occasion/1812">Mathematical Methods in Quantum Chemistry</a>.
I am really grateful to the organisers
Eric Cancès,
Gero Friesecke,
Trygve Helgaker
and Lin Lin
for asking me to join this meeting
as it turned out to be a week of
interesting talks, stimulating discussions
and getting to know many interesting people.</p>
<p>Unlike the typically rather large and tightly scheduled
quantum-chemistry conferences,
which I went to in the past years,
the Oberwolfach workshop was not extremely dense.
Much rather there was a lot of free time,
where one could talk to other participants
and discuss each other's topic very well.
Overall I learned a lot about current research
regarding the mathematical aspects of quantum chemistry.</p>
<p>One should mention that Oberwolfach,
being located in the middle of the Black Forest,
has a very unique atmosphere with
numerous black boards, comfortable chairs
and a huge library scattered around the institute.
Whilst it might seem a bit hard to believe in the beginning,
this environment really has an inspiring effect,
where one starts to sit with fellow researchers and discuss
all kinds of interesting topics for many hours and sometimes till late at night.
If I am ever offered to return to this place for another meeting or workshop,
I would hesitate no second.</p>
<p>On the last day of the workshop, namely last Friday,
I was asked to present my current research.
For this I choose to give a rather broad overview over molsturm and
our recent take on Coulomb-Sturmian-based quantum chemistry.
The slides are attached below and an extended abstract about the talk
will soon become available as part of the
<a href="http://www.ems-ph.org/journals/all_issues.php?issn=1660-8933">Oberwolfach reports</a>. </p>
<p><strong>Update:</strong>
The <a href="https://michael-herbst.com/2018_oberwolfach_report_molsturm.html">preprint of the extended abstract</a> is now available.</p>
<table>
<thead>
<tr>
<th>Link</th>
</tr>
</thead>
<tbody>
<tr>
<td>Slides <a href="https://michael-herbst.com/talks/2018.03.23_molsturm.pdf"><em><code>molsturm</code>: Modular electronic structure theory framework</em></a></td>
</tr>
</tbody>
</table>Annual Colloquium 2017: Introduction to Bohrium2017-12-03T00:00:00+01:002017-12-03T00:00:00+01:00Michael F. Herbsttag:michael-herbst.com,2017-12-03:/annual-colloquium-2017-bohrium.html<p>Last Thursday the PhDs of my graduate school gathered again for our
self-organised mini conference, named
<a href="http://www.mathcomp.uni-heidelberg.de/annual-colloquium-2017/">Annual Colloquium</a>.
Having organised this event myself
as well a <a href="https://michael-herbst.com/annual-colloquium-2014.html">couple of years back</a>,
I had rather mixed feelings this time,
since I most likely not be around in Heidelberg for another AC.</p>
<p>For …</p><p>Last Thursday the PhDs of my graduate school gathered again for our
self-organised mini conference, named
<a href="http://www.mathcomp.uni-heidelberg.de/annual-colloquium-2017/">Annual Colloquium</a>.
Having organised this event myself
as well a <a href="https://michael-herbst.com/annual-colloquium-2014.html">couple of years back</a>,
I had rather mixed feelings this time,
since I most likely not be around in Heidelberg for another AC.</p>
<p>For this reason I am very happy that the organisers
gave me the chance to once again make a contribution to this great event.
This time I was asked to repeat my interactive introductory talk
about the <a href="http://bh107.org">Bohrium</a> automatic parallelisation framework.
I already presented about Bohrium in one of my c¼h lectures
at the Heidelberg Chaostreff <a href="https://michael-herbst.com/c14h-bohrium.html">earlier this year</a>
and in fact this talk turned out to be very similar to the previous one.</p>
<p>Compared to the points mentioned in my
<a href="https://michael-herbst.com/c14h-bohrium.html">earlier blog post</a>,
one should probably add a few things,
which have changed in Bohrium in the recent months.
First of all Bohrium has made quite some progress
regarding the interoperability with other efforts like cython, pyopencl
and pycuda for improving the performance of python scripts.
In fact Bohrium and these projects can now be used side-by-side
and will work together flawlessly to accelerate algorithms written in python.
Along a similar line, Bohrium started to look into mechanisms,
which could be used to speed up places where one would typically require
a plain python <code>for</code>-loops.
Whilst this destroys the full compatibility with numpy on the one hand,
this allows on the other hand to increase performance in settings,
which are hard to write only as array operations.</p>
<p>On top of that the recent integration into the <a href="https://spack.io">Spack</a>
package manager makes it comparatively easy to install Bohrium
on any machine (including HPC clusters) to give it a try in a production
environment.
See the
<a href="https://bohrium.readthedocs.io/installation/spack.html">Spack section of the Bohrium documentation</a>
for more details.</p>
<p>If you want to find out more about Bohrium,
I suggest you <a href="https://michael-herbst.com/c14h-bohrium.html">read my previous post</a>
or
watch the <a href="https://youtu.be/PC_glGSK2Kg">recording of my previous talk</a>.
For completeness I attach below the demonstration script I
used for both Bohrium presentations.</p>
<table>
<thead>
<tr>
<th>Link</th>
</tr>
</thead>
<tbody>
<tr>
<td><a href="https://michael-herbst.com/talks/2017.07.13_moments.py">Bohrium moments example script</a></td>
</tr>
</tbody>
</table>Lazy matrices talk from the IWR school 20172017-11-11T00:00:00+01:002017-11-11T00:00:00+01:00Michael F. Herbsttag:michael-herbst.com,2017-11-11:/iwr-school-mathematical-methods-talk.html<p>As mentioned in a <a href="https://michael-herbst.com/iwr-school-mathematical-methods.html">previous post</a>
on this matter last month, from 2nd to 6th October, the IWR hosted the school
<a href="https://www.iwr.uni-heidelberg.de/events/iwr-school-2017/">Mathematical Methods for Quantum Chemistry</a>,
which I co-organised together with my supervisors Andreas Dreuw and Guido Kanschat
as well as the head of my graduate school Michael Winckler.</p>
<p>From …</p><p>As mentioned in a <a href="https://michael-herbst.com/iwr-school-mathematical-methods.html">previous post</a>
on this matter last month, from 2nd to 6th October, the IWR hosted the school
<a href="https://www.iwr.uni-heidelberg.de/events/iwr-school-2017/">Mathematical Methods for Quantum Chemistry</a>,
which I co-organised together with my supervisors Andreas Dreuw and Guido Kanschat
as well as the head of my graduate school Michael Winckler.</p>
<p>From my personal point of view the school turned out to be a major success,
where I had the chance to meet a lot of interesting people
and got a bucket of ideas to try out in the future.
The feedback we got from the participants was positive as well
and so I am very happy that all the effort in the past half a year
really turned to be worth the while for all of us.</p>
<p>Even though all relevant documents from the school,
including the slides from the lectures and most contributed talks,
are finally published at the
<a href="https://www.iwr.uni-heidelberg.de/events/iwr-school-2017/documents">school's website</a>,
I nevertheless want to include a pointer to the slides of my lazy matrices talk
from this blog for reference.</p>
<p>The topic and structure of the talk is very similar
to the talks of the previous months.
I motivate the use of contraction-based methods from the realisation,
that storing intermediate computational results in memory
can be less optimal than recomputing them in a clever way when needed.
Then I present lazy matrices as a solution to the issue
that the code needed for performing calculations
in the sense of contraction-based methods
can become very complicated.
Afterwards I hint how we use lazy matrices in the context of the
quantum chemistry program <a href="https://molsturm.org"><code>molsturm</code></a>
and how <code>molsturm</code> itself really facilitates the implementation
of new quantum-chemical methods.
For this I show in slide 20
a comparison of parts of a working
<a href="https://en.wikipedia.org/wiki/Coupled_cluster">Coupled-Cluster doubles (CCD)</a>
code based on <code>molsturm</code> with the relevant part of
the equation for the CCD residual.</p>
<table>
<thead>
<tr>
<th>Link</th>
</tr>
</thead>
<tbody>
<tr>
<td><a href="https://michael-herbst.com/talks/2017.10.04_iwr_school.pdf">Lazy matrices for contraction-based algorithms</a> (IWR school talk)</td>
</tr>
</tbody>
</table>Coulomb-Sturmians and molsturm2017-09-19T00:00:00+02:002017-09-19T00:00:00+02:00Michael F. Herbsttag:michael-herbst.com,2017-09-19:/kwt-2017-coulomb-sturmian-molsturm.html<p>Yesterday I gave a talk at our annual group retreat at the Darmstädter Haus in Hirschegg,
Kleinwalsertal. For this talk I expanded the slides from
<a href="https://michael-herbst.com/kiel-sturmian-lazymat.html">my lazy matrix talk in Kiel</a>,
incorporating a few of the more recent Coulomb-Sturmian results I obtained.</p>
<p>Most importantly I added a few slides to …</p><p>Yesterday I gave a talk at our annual group retreat at the Darmstädter Haus in Hirschegg,
Kleinwalsertal. For this talk I expanded the slides from
<a href="https://michael-herbst.com/kiel-sturmian-lazymat.html">my lazy matrix talk in Kiel</a>,
incorporating a few of the more recent Coulomb-Sturmian results I obtained.</p>
<p>Most importantly I added a few slides to highlight the issues with standard
contracted Gaussians and to discuss the fundamental properties of the Coulomb-Sturmians.
Furthermore I gave some details why a <code>contraction</code>-based scheme is advantageous if
Coulomb-Sturmian basis functions are employed and discussed the design of our molsturm
program package and how it allows to perform calculations in a fully
<code>contraction</code>-based manor.</p>
<table>
<thead>
<tr>
<th>Link</th>
</tr>
</thead>
<tbody>
<tr>
<td><a href="https://michael-herbst.com/talks/2017.09.18_Kleinwalsertal_Sturmians.pdf">Coulomb-Sturmians and molsturm</a> (Slides Kleinwalsertal talk)</td>
</tr>
</tbody>
</table>Advanced bash scripting 20172017-09-11T10:00:00+02:002017-09-11T10:00:00+02:00Michael F. Herbsttag:michael-herbst.com,2017-09-11:/advanced-bash-scripting-block-course-2017.html<p>I am very happy to announce
that <a href="http://www.mathcomp.uni-heidelberg.de/">my graduate school</a>
asked me to repeat the block course about <code>bash</code> scripting,
which I first taught in 2015.</p>
<p>In the course we will take a structured look at UNIX shell scripting from the bottom up.
We will revise some elements about the …</p><p>I am very happy to announce
that <a href="http://www.mathcomp.uni-heidelberg.de/">my graduate school</a>
asked me to repeat the block course about <code>bash</code> scripting,
which I first taught in 2015.</p>
<p>In the course we will take a structured look at UNIX shell scripting from the bottom up.
We will revise some elements about the UNIX operating system
infrastructure, discuss handy features of the shell
as well as its syntax elements.
Whilst the main focus of the course is the <code>bash</code> shell,
we will look at other common utility programs like
<code>grep</code>, <code>sed</code> and <code>awk</code> as well
as they are very handy in the context of scripting.
Last but not least we will discuss common pitfalls and how they can be avoided.</p>
<p>The course runs from <strong>6th till 10th November 2017</strong> at Heidelberg University.
You can find further information on the
<a href="https://michael-herbst.com/teaching/advanced-bash-scripting-2017/">"Advanced <code>bash</code> scripting" course website</a>.</p>
<p>As usual all course material will be published both on the
<a href="https://michael-herbst.com/teaching/advanced-bash-scripting-2017/">course website</a>
as well as the course <a href="https://github.com/mfherbst/bash-course">github repository</a>
afterwards.</p>
<p><strong>Update (29/09/2017):</strong>
<a href="http://hgs.iwr.uni-heidelberg.de/Portfolio_HGS/VERANSTALTUNGEN/reg_form/reg_form.php?id=196">Registration</a>
is now open.</p>[c¼h] Parallelised numerics in Python: An introduction to Bohrium2017-07-25T00:00:00+02:002017-07-25T00:00:00+02:00Michael F. Herbsttag:michael-herbst.com,2017-07-25:/c14h-bohrium.html<p>Thursday a week ago I gave a brief introductory talk in our Heidelberg Chaostreff
about the <a href="http://bh107.org">Bohrium</a> project.
Especially after the
<a href="https://michael-herbst.com/hpc-day-nbi.html">HPC day at the Niels Bohr Institute</a>
during my recent visit to Copenhagen,
I became rather enthusiastic about Bohrium
and wanted to pass on some of my experiences.</p>
<p>The …</p><p>Thursday a week ago I gave a brief introductory talk in our Heidelberg Chaostreff
about the <a href="http://bh107.org">Bohrium</a> project.
Especially after the
<a href="https://michael-herbst.com/hpc-day-nbi.html">HPC day at the Niels Bohr Institute</a>
during my recent visit to Copenhagen,
I became rather enthusiastic about Bohrium
and wanted to pass on some of my experiences.</p>
<p>The main idea of Bohrium is to build a fully <a href="http://numpy.org">numpy</a>-compatible
framework for <strong>high-performance computing</strong>,
which can automatically parallelise numpy array operations and/or
execute them on a general-purpose graphics cards.
The hope is that this eradicates the step of rewriting a prototypical python
implementation of a scientific model in more low-level languages like C++ or
<a href="https://en.wikipedia.org/wiki/CUDA">CUDA</a> before dealing with the actual
real-world problems in mind.</p>
<p>In practice Bohrium achieves this by translating the python code (via some
intermediate steps) into small pieces of C or CUDA code.
These are then automatically compiled at runtime of the script,
taking into account the current hardware setup, and afterwards executed.
The results of such a <strong>just-in-time</strong> compiled <strong>kernel</strong> are again
available in numpy-like
arrays and can be passed to other scripts for post-processing,
e.g. plotting in <a href="https://matplotlib.org/">matplotlib</a>.</p>
<p>It is important to note, that the effect of Bohrium is limited to
array operations. So for example the usual Python <code>for</code> loops
are not touched.
This is, however, hardly a problem if the practice of
so-called <a href="https://en.wikipedia.org/wiki/Array_programming"><strong>array programming</strong></a>
is followed.
In array programming one avoids plain <code>for</code>-loops and similar traditional
python language elements in preference for special
syntax which works on blocks (numpy <em>arrays</em>) of data at once.
Examples of such operations is pretty much the typical
numpy workflow:</p>
<ul>
<li>views and slices: <code>array[1:3]</code></li>
<li><a href="https://docs.scipy.org/doc/numpy/user/basics.broadcasting.html">broadcasting</a>: <code>array[:, np.newaxis]</code></li>
<li>elementwise operations: <code>array1 * array2</code></li>
<li>reduction: <code>np.sum(array1, axis=1)</code></li>
</ul>
<p>A slightly bigger drawback of Bohrium is, that the
just-in-time compilation takes time, where no results are produced.
In other words Bohrium does only start to pay of at larger problem sizes or if exactly the
same sequence of instructions is to be executed many times.</p>
<p>In my c¼h I demonstrate Bohrium by the means of this example script</p>
<div class="highlight"><pre><span></span><code><span class="ch">#!/usr/bin/env python3</span>
<span class="kn">import</span> <span class="nn">numpy</span> <span class="k">as</span> <span class="nn">np</span>
<span class="kn">import</span> <span class="nn">sys</span>
<span class="kn">import</span> <span class="nn">time</span>
<span class="k">def</span> <span class="nf">moment</span><span class="p">(</span><span class="n">n</span><span class="p">,</span> <span class="n">a</span><span class="p">):</span>
<span class="n">avg</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="kp">sum</span><span class="p">(</span><span class="n">a</span><span class="p">)</span> <span class="o">/</span> <span class="n">a</span><span class="o">.</span><span class="kp">size</span>
<span class="k">return</span> <span class="n">np</span><span class="o">.</span><span class="kp">sum</span><span class="p">((</span><span class="n">a</span> <span class="o">-</span> <span class="n">avg</span><span class="p">)</span><span class="o">**</span><span class="n">n</span><span class="p">)</span> <span class="o">/</span> <span class="n">a</span><span class="o">.</span><span class="kp">size</span>
<span class="k">def</span> <span class="nf">compute</span><span class="p">(</span><span class="n">a</span><span class="p">):</span>
<span class="n">start</span> <span class="o">=</span> <span class="n">time</span><span class="o">.</span><span class="n">time</span><span class="p">()</span>
<span class="kp">mean</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="kp">sum</span><span class="p">(</span><span class="n">a</span><span class="p">)</span> <span class="o">/</span> <span class="n">a</span><span class="o">.</span><span class="kp">size</span>
<span class="kp">var</span> <span class="o">=</span> <span class="n">moment</span><span class="p">(</span><span class="mi">2</span><span class="p">,</span> <span class="n">a</span><span class="p">)</span>
<span class="n">m3</span> <span class="o">=</span> <span class="n">moment</span><span class="p">(</span><span class="mi">3</span><span class="p">,</span> <span class="n">a</span><span class="p">)</span>
<span class="n">m4</span> <span class="o">=</span> <span class="n">moment</span><span class="p">(</span><span class="mi">4</span><span class="p">,</span> <span class="n">a</span><span class="p">)</span>
<span class="n">end</span> <span class="o">=</span> <span class="n">time</span><span class="o">.</span><span class="n">time</span><span class="p">()</span>
<span class="n">fmt</span> <span class="o">=</span> <span class="s2">"After </span><span class="si">{0:8.3f}</span><span class="s2">s: </span><span class="si">{1:8.3f}</span><span class="s2"> </span><span class="si">{2:8.3f}</span><span class="s2"> </span><span class="si">{3:8.3f}</span><span class="s2"> </span><span class="si">{4:8.3f}</span><span class="s2">"</span>
<span class="nb">print</span><span class="p">(</span><span class="n">fmt</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">end</span> <span class="o">-</span> <span class="n">start</span><span class="p">,</span> <span class="nb">float</span><span class="p">(</span><span class="kp">mean</span><span class="p">),</span> <span class="nb">float</span><span class="p">(</span><span class="kp">var</span><span class="p">),</span>
<span class="nb">float</span><span class="p">(</span><span class="n">m3</span><span class="p">),</span> <span class="nb">float</span><span class="p">(</span><span class="n">m4</span><span class="p">)))</span>
<span class="k">def</span> <span class="nf">main</span><span class="p">(</span><span class="kp">size</span><span class="p">,</span> <span class="kp">repeat</span><span class="o">=</span><span class="mi">6</span><span class="p">):</span>
<span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="kp">repeat</span><span class="p">):</span>
<span class="n">compute</span><span class="p">(</span><span class="n">np</span><span class="o">.</span><span class="n">random</span><span class="o">.</span><span class="n">rand</span><span class="p">(</span><span class="kp">size</span><span class="p">))</span>
<span class="k">if</span> <span class="vm">__name__</span> <span class="o">==</span> <span class="s2">"__main__"</span><span class="p">:</span>
<span class="kp">size</span> <span class="o">=</span> <span class="mi">30</span>
<span class="k">if</span> <span class="nb">len</span><span class="p">(</span><span class="n">sys</span><span class="o">.</span><span class="n">argv</span><span class="p">)</span> <span class="o">>=</span> <span class="mi">2</span><span class="p">:</span>
<span class="kp">size</span> <span class="o">=</span> <span class="nb">int</span><span class="p">(</span><span class="n">sys</span><span class="o">.</span><span class="n">argv</span><span class="p">[</span><span class="mi">1</span><span class="p">])</span>
<span class="n">main</span><span class="p">(</span><span class="kp">size</span><span class="p">)</span>
</code></pre></div>
<p>which is also <a href="https://michael-herbst.com/talks/2017.07.13_moments.py">available for download</a>.
The script performs a very simple analysis of the (randomly generated)
input data: It computes some statistical moments and displays them to
the user.
For bigger arrays the single-threaded numpy starts to get very slow,
whereas the multi-threaded Bohrium version wins even thought it needs
to compile first.
Running the script with Bohrium does not require one to change even
a <em>single</em> line of code!
Just</p>
<div class="highlight"><pre><span></span><code>python3 -m bohrium ./2017.07.13_moments.py
</code></pre></div>
<p>does kick off the <strong>automatic parallelisation</strong>.</p>
<p>The talk has been <a href="https://youtu.be/PC_glGSK2Kg">recorded</a> and is available on
<a href="https://youtu.be/PC_glGSK2Kg">youtube</a>.
Note, that the title of the talk and the description are German,
but the talk by itself is in English.</p>IWR School: Mathematical Methods for Quantum Chemistry2017-07-07T00:00:00+02:002017-07-07T00:00:00+02:00Michael F. Herbsttag:michael-herbst.com,2017-07-07:/iwr-school-mathematical-methods.html<p>From <strong>2nd to 6th October 2017</strong> the IWR hosts the school
<a href="https://www.iwr.uni-heidelberg.de/events/iwr-school-2017/">Mathematical Methods for Quantum Chemistry</a>,
which aims at shining some light into the simulation of quantum chemical problems
and their mathematical properties.
I am very happy to be part of the organising committee for this school,
especially since I …</p><p>From <strong>2nd to 6th October 2017</strong> the IWR hosts the school
<a href="https://www.iwr.uni-heidelberg.de/events/iwr-school-2017/">Mathematical Methods for Quantum Chemistry</a>,
which aims at shining some light into the simulation of quantum chemical problems
and their mathematical properties.
I am very happy to be part of the organising committee for this school,
especially since I have been missing an opportunity like this myself in the
recent years.</p>
<p>Starting from an introduction into the basics of quantum chemistry as well as the
required concepts about optimisation and non-linear eigenvalue problems,
we will look at ways to tackle the relevant differential equations from a numerical
and algorithmic viewpoint. Hands-on practical sessions allow to apply
the treatment introduced in the lectures to simple practical problems.</p>
<p>After the school participants will have an idea of the required
interplay between the physical modelling, the mathematical representation
and the computational treatment needed in this highly interdisciplinary
field of research.</p>
<p>Speakers of the school include</p>
<ul>
<li><a href="https://cermics.enpc.fr/~cances/">Eric Cances, Ecole des Ponts Paris Tech and INRIA Paris</a></li>
<li><a href="http://www.ltm.uni-erlangen.de/lehrstuhl/mitarbeiter/personal.shtml/denis-davydov.shtml">Denis Davydov, FAU Erlangen-Nürnberg</a></li>
<li><a href="http://www.iwr.uni-heidelberg.de/groups/compchem/andreas_dreuw.php">Andreas Dreuw, Heidelberg University</a></li>
<li><a href="http://simweb.iwr.uni-heidelberg.de/~gkanscha/de/">Guido Kanschat, Heidelberg University</a></li>
<li><a href="https://www-m16.ma.tum.de/Allgemeines/KatharinaKormann">Katharina Kormann, Max Planck Institute for Plasma Physics</a></li>
<li><a href="http://www.math.tu-berlin.de/fachgebiete_ag_modnumdiff/fg_modellierung_simulation_und_optimierung_in_natur_und_ingenieurswissenschaften/v_menue/mitarbeiter/prof_dr_reinhold_schneider/modellierung/parameter/en/">Reinhold Schneider, TU Berlin</a></li>
</ul>
<p>The school mainly targets postgraduate students, postdocs and young
researchers from mathematics and computational chemistry.
Everyone is encouraged to participate actively by giving a short
presentation of his or her work during the available sessions of
contributed talks.</p>
<p>For further information as well as the program check out our
<a href="https://www.iwr.uni-heidelberg.de/events/iwr-school-2017/">website</a>.
The <strong>deadline</strong> for the <strong>application</strong> is <strong>8th August 2017</strong>
(<a href="http://hgs.iwr.uni-heidelberg.de/Portfolio_HGS/FGEN/form/fgen_form.php?id_form=39">direct registration link</a>).</p>
<p>We intend to <a href="https://www.iwr.uni-heidelberg.de/events/iwr-school-2017/documents">publish the material</a>
of the school as we go along
<a href="https://www.iwr.uni-heidelberg.de/events/iwr-school-2017/documents">at this location</a>,
so feel free to check back regularly for updates.</p>Lazy matrices in quantum chemistry2017-06-19T00:00:00+02:002017-06-19T00:00:00+02:00Michael F. Herbsttag:michael-herbst.com,2017-06-19:/kiel-sturmian-lazymat.html<p>On my way back from Copenhagen last Thursday I stopped in Kiel to visit Henrik Larsson
and some other people from last year's Deutsche Schülerakademie in Grovesmühle.</p>
<p>Henrik is doing his PhD at the group of <a href="https://ravel.pctc.uni-kiel.de/">Prof. Bernd Hartke</a>
and so I took the opportunity and gave a short talk …</p><p>On my way back from Copenhagen last Thursday I stopped in Kiel to visit Henrik Larsson
and some other people from last year's Deutsche Schülerakademie in Grovesmühle.</p>
<p>Henrik is doing his PhD at the group of <a href="https://ravel.pctc.uni-kiel.de/">Prof. Bernd Hartke</a>
and so I took the opportunity and gave a short talk at their group seminar
on my progress in Copenhagen (slides are attached below).</p>
<p>Most people in the Hartke group work on global optimisation of structures or
reaction pathways or on fitting reactive force fields.
Hence my talk turned out to be a little off-topic.
On the other hand I had an afternoon of enjoyful and rather unusual discussions,
where I learned quite a lot about their work.</p>
<table>
<thead>
<tr>
<th>Link</th>
</tr>
</thead>
<tbody>
<tr>
<td><a href="https://michael-herbst.com/talks/2017.06.15_sturmian_lazy_matrices_kiel.pdf">Lazy-matrices in quantum chemistry</a> (Slides group seminar talk)</td>
</tr>
</tbody>
</table>HPC day at Niels Bohr Institute2017-05-20T00:00:00+02:002017-05-20T00:00:00+02:00Michael F. Herbsttag:michael-herbst.com,2017-05-20:/hpc-day-nbi.html<p>For the second time I currently have the pleasure of staying a couple of weeks at the
<a href="http://www.migrid.org/vgrid/eScience/">eScience</a>
group of Prof. Brian Vinter at the Niels Bohr Institute in Copenhagen.
Same as last time the atmosphere in the group is very welcoming and
I really enjoy the many fruitful discussions …</p><p>For the second time I currently have the pleasure of staying a couple of weeks at the
<a href="http://www.migrid.org/vgrid/eScience/">eScience</a>
group of Prof. Brian Vinter at the Niels Bohr Institute in Copenhagen.
Same as last time the atmosphere in the group is very welcoming and
I really enjoy the many fruitful discussions I had so far
with the PhDs and PostDocs of the group.</p>
<p>Many of them work on the <a href="http://bh107.org">Bohrium</a> project,
which aims at providing a simple-to-use high-level interface for performing
computations on all sorts of hardware, i.e. both CPUs as well as GPUs or mixed systems.
Even though there surely are some performance drawbacks compared to a
fully native implementation, many manhours can be saved by just implementing
an algorithm once, which then automatically runs on whatever hardware
happens to be around.
I really like this idea and hopefully we will manage to integrate Bohrium
into
our <a href="https://lazyten.org"><code>linalgwrap</code></a> linear algebra wrapper libary very soon.</p>
<p>The main purpose of my visit, however, is to continue the work with
my long-term collaborator <strong>James Avery</strong> on our <a href="https://molsturm.org"><code>molsturm</code></a>
modular quantum chemistry code and of course on <a href="https://lazyten.org"><code>linalgwrap</code></a>
as well. So far the progress is very good and it seems we are soon ready
to do a couple of very simple calculations on closed shell atoms
or ions.</p>
<p>Just yesterday I furthermore had the pleasure of introducing
<a href="https://lazyten.org"><code>linalgwrap</code></a> and our <strong>lazy matrices</strong>
to a wider audience at the <strong>high performance computing day</strong>
here at the Niels Bohr Institute. The slides of my presentation
are attached below.</p>
<table>
<thead>
<tr>
<th>Link</th>
</tr>
</thead>
<tbody>
<tr>
<td><a href="https://michael-herbst.com/talks/2017.05.19_HPC_Day_NBI.pdf">Lazy-matrices for apply-based algorithms</a> (Slides HPC day 2017)</td>
</tr>
</tbody>
</table>A new home in a new design2017-04-29T00:00:00+02:002017-04-29T00:00:00+02:00Michael F. Herbsttag:michael-herbst.com,2017-04-29:/new-home.html<p>Finally I successfully managed to migrate my blog articles
from the old wordpress blog at <a href="http://blog.mfhs.eu">blog.mfhs.eu</a>
to their new home right here.
Thanks to the awesome static site generator
<a href="https://getpelican.com/">pelican</a>
this site now works without any of this horrible <code>php</code> in the background,
just static html and some …</p><p>Finally I successfully managed to migrate my blog articles
from the old wordpress blog at <a href="http://blog.mfhs.eu">blog.mfhs.eu</a>
to their new home right here.
Thanks to the awesome static site generator
<a href="https://getpelican.com/">pelican</a>
this site now works without any of this horrible <code>php</code> in the background,
just static html and some css.</p>
<p>Note that I have not yet replicated all pages from the old
<a href="http://blog.mfhs.eu">blog.mfhs.eu</a> site,
but new content, especially new blog articles, will only
appear here from now on.</p>
<p>Finally ... things might still be broken, beware ;).</p>look4bas: Small script to search for Gaussian basis sets2017-04-25T00:00:00+02:002017-04-25T00:00:00+02:00Michael F. Herbsttag:michael-herbst.com,2017-04-25:/look4bas-small-script-to-search-for-gaussian-basis-sets.html<p>In the past couple of days I hacked together a small Python script,
which searches through the <a href="https://bse.pnl.gov/bse/portal">EMSL basis set exchange</a>
library of Gaussian basis sets via the commandline.</p>
<p>Unlike the webinterface it allows to use <code>grep</code>-like <strong>regular expressions</strong> for
matching the names and descriptions of the basis sets …</p><p>In the past couple of days I hacked together a small Python script,
which searches through the <a href="https://bse.pnl.gov/bse/portal">EMSL basis set exchange</a>
library of Gaussian basis sets via the commandline.</p>
<p>Unlike the webinterface it allows to use <code>grep</code>-like <strong>regular expressions</strong> for
matching the names and descriptions of the basis sets.
Of course limiting the selection by requiring basis functions for specific
elements to be defined is possible as well.</p>
<p>All matching basis sets can be easily downloaded at once.
Right now only downloading the basis set files in Gaussian94 format is implemented, however.</p>
<p>The code, further information and some examples can be found on
<a href="https://github.com/mfherbst/look4bas">https://github.com/mfherbst/look4bas</a>.</p>The design of molsturm2016-12-10T00:00:00+01:002016-12-10T00:00:00+01:00Michael F. Herbsttag:michael-herbst.com,2016-12-10:/design-of-molsturm.html<p>Yesterday I was invited to the give a talk at the group seminar of
<a href="https://chemie.uni-paderborn.de/en/arbeitskreise/technische-chemie/kuehne/">Prof. Thomas Kühne</a>
at the University of Paderborn.
I was really happy about this invitation, since this gave me a great opportunity
to talk about the recent direction my PhD has taken.</p>
<p>Motivated from the problems …</p><p>Yesterday I was invited to the give a talk at the group seminar of
<a href="https://chemie.uni-paderborn.de/en/arbeitskreise/technische-chemie/kuehne/">Prof. Thomas Kühne</a>
at the University of Paderborn.
I was really happy about this invitation, since this gave me a great opportunity
to talk about the recent direction my PhD has taken.</p>
<p>Motivated from the problems we had with finite-element based Hartree-Fock
we wanted to seek a way to design a <strong>modular quantum chemistry</strong> program package,
which enables quick experimentation on all levels.
From the choice of the SCF algorithm, the integral library as well as the
linear algebra backend: Everything should be ideally configurable at runtime.
For this I once again joined forces with <strong>James Avery</strong> from the University of Copenhagen.</p>
<p>We started by developing <a href="https://lazyten.org"><code>linalgwrap</code></a>:
A lightweight linear algebra wrapper library, which easily allows
switching between different linear algebra backends.
For this we employ so-called <strong>lazy matrices</strong>, i.e. data structures, which offer
a matrix-like interface, but where the matrix elements may no longer need to be
stored in memory.
This has a couple of consequences:
On the one hand it might no longer be possible
to obtain elements of a matrix very cheaply, but on the other hand
this simplifies writing algorithms based on matrix-vector applications
very much.
It turns out that for some cases --- including finite elements --- these type of
algorithms can reduce the scaling in storage and time by an order of magnitude.</p>
<p>Currently we will not use <code>linalgwrap</code> together with finite elements, however,
but instead we will take a look at Coulomb-Sturmian basis functions,
which James and his father John Avery have developed in the recent years.
These share a lot of properties with finite elements, for example
very sparse matrix structures, but unlike finite elements
they only require small basis sizes (comparable to Gaussian basis sets)
for reasonable results and hence are easier to deal with.</p>
<p>The presentation I gave in Paderborn both talks about the lessons we learned
from trying our luck with finite-element based Hartree-Fock as well as the
design of our new modular program package <a href="https://molsturm.org"><code>molsturm</code></a>.</p>
<table>
<thead>
<tr>
<th>Link</th>
</tr>
</thead>
<tbody>
<tr>
<td><a href="https://michael-herbst.com/talks/2016.12.09_Design_Molsturm.pdf">Design of molsturm</a> (Slides)</td>
</tr>
</tbody>
</table>Introduction to awk programming block course2016-05-26T00:00:00+02:002016-05-26T00:00:00+02:00Michael F. Herbsttag:michael-herbst.com,2016-05-26:/introduction-to-awk-programming-block-course.html<p>Last year I taught a course about
<a href="https://michael-herbst.com/teaching/advanced-bash-scripting-2015/">bash scripting</a>,
during which I briefly touched on the scripting language <code>awk</code>. Some of the attending
people wanted to hear more about this, so I was asked by <a href="http://www.mathcomp.uni-heidelberg.de/">my graduate
school</a> to prepare a short block
course on <code>awk</code> programming for this year …</p><p>Last year I taught a course about
<a href="https://michael-herbst.com/teaching/advanced-bash-scripting-2015/">bash scripting</a>,
during which I briefly touched on the scripting language <code>awk</code>. Some of the attending
people wanted to hear more about this, so I was asked by <a href="http://www.mathcomp.uni-heidelberg.de/">my graduate
school</a> to prepare a short block
course on <code>awk</code> programming for this year.</p>
<p>The course will be running
from 15th till 17th August 2016 and
<a href="https://michael-herbst.com/teaching/introduction-to-awk-programming-2016/">registration is now open</a>.
You can find an outline and further information on the
<a href="https://michael-herbst.com/teaching/introduction-to-awk-programming-2016/">"Introduction to <code>awk</code>" course page</a>.
If you cannot make the course in person, do not worry: All course material will
be published both on the course website and on
<a href="https://github.com/mfherbst/awk-course">github</a> afterwards.</p>[c¼h] Testen mit Rapidcheck und Catch2016-03-14T00:00:00+01:002016-03-14T00:00:00+01:00Michael F. Herbsttag:michael-herbst.com,2016-03-14:/c14h-testen-mit-rapidcheck-und-catch.html<p>Last Thursday, I gave another short talk at the Heidelberg
Chaostreff NoName e.V. This time I talked about writing tests in <code>C++</code>
using the testing libraries
<a href="https://github.com/emil-e/rapidcheck/">rapidcheck</a> and
<a href="https://github.com/philsquared/Catch/">Catch</a>.</p>
<p>In the talk I presented some ideas how to incorporate <em>property-based
testing</em> into test suites for <code>C++</code> programs. The idea …</p><p>Last Thursday, I gave another short talk at the Heidelberg
Chaostreff NoName e.V. This time I talked about writing tests in <code>C++</code>
using the testing libraries
<a href="https://github.com/emil-e/rapidcheck/">rapidcheck</a> and
<a href="https://github.com/philsquared/Catch/">Catch</a>.</p>
<p>In the talk I presented some ideas how to incorporate <em>property-based
testing</em> into test suites for <code>C++</code> programs. The idea of property-based
testing originates from the Haskell library
<a href="https://en.wikipedia.org/wiki/QuickCheck">QuickCheck</a>, which tries to
use properties of the input data and of properties of the output data in
order to generate random test cases for a code unit. So instead of
testing a piece of code with just a small number of static tests, which
are the same for each run of the test suite, we test with randomly
seeded data. Additionally, if a test fails, QuickCheck/rapidcheck
automatically simplifies the test case in order to find the simplest
input, which yields a failing test. This of cause eases finding the
underlying bug massively.</p>
<p>Since this was our first Treff in the new <a href="https://www.noname-ev.de/anfahrt.html">Mathematikon
building</a>, the University just
<a href="https://www.uni-heidelberg.de/presse/news2015/pm20151130_festveranstaltung_eroeffnung_mathematikon.html">opened
recently</a>,
we had a few technical difficulties with our setup. As a result there is
no recording of my talk available this time, unfortunately. The example
code I used during the presentation, however, is available
<a href="https://github.com/mfherbst/c14h-rapidcheck-catch">on github</a>. It contains
a couple of buggy classes and functions and a <code>Catch</code> based test
program, which can be used to find these bugs.</p>51th Symposium on Theoretical Chemistry (Potsdam)2015-10-02T00:00:00+02:002015-10-02T00:00:00+02:00Michael F. Herbsttag:michael-herbst.com,2015-10-02:/51th-symposium-on-theoretical-chemistry-potsdam.html<p>Last week I went to the annual Symposium on Theoretical Chemistry. This
time the <a href="http://tcb16.chem.uni-potsdam.de/cms/wordpress/">51th STC</a>
took place from the 20th till the 24th of September in Potsdam.</p>
<p>Unlike last year, where the conference business was all new to me, this
year it was more like returning to a place …</p><p>Last week I went to the annual Symposium on Theoretical Chemistry. This
time the <a href="http://tcb16.chem.uni-potsdam.de/cms/wordpress/">51th STC</a>
took place from the 20th till the 24th of September in Potsdam.</p>
<p>Unlike last year, where the conference business was all new to me, this
year it was more like returning to a place with a familiar atmosphere
and many familiar faces. Whilst the conference surely was again a great
opportunity to present my work and to learn about theoretical chemistry
in the lectures, this time it had the additional aspect of catching up
with the people I already met last year. Most of all I enjoyed the
poster sessions this year, probably because of the many discussions I
had with other researchers and PhD students about their recent advances.</p>
<p>Concerning our project we still were not able to obtain noteworthy
results with our current implementation of finite-element Hartree-Fock
(see Poster attached below). This is mainly due to the extreme memory
requirements the calculation of the Hartree-Fock exchange matrix imposes
on the program: Right now we store this whole beast in memory and hence
get quadratic scaling with respect to the number of finite elements in
the amount of memory we require. In other words even for extremely small
test cases we need gigabytes of memory to achieve only very inadequate
accuracy.</p>
<p>Together with <a href="http://www.nbi.ku.dk/ansatte/?pure=da%2Fpersons%2Fjames-emil-avery%2841079901-31dc-4e1b-9e75-86ad49e73136%29.html">James
Avery</a>,
who visited us for a few days in July, we recently managed to come up
with a new scheme for implementing Hartree-Fock exchange within the
finite-element method. This looks very promising, since it decreases
both the memory as well as the computational cost to linear scaling.
Right now we struggle with the implementation, however. In November I
will visit James in Kopenhagen for 3 weeks. If all goes well we will
hopefully overcome these problems and work out a good structure for a
FE-HF program, that incorporates everything we learned so far.</p>
<table>
<thead>
<tr>
<th>Link</th>
</tr>
</thead>
<tbody>
<tr>
<td><a href="https://michael-herbst.com/poster/2015.09.23_STC_P42_New_approaches_towards_a_finite-element_based_Hartree-Fock_method.pdf">Poster P42 STC 2015</a></td>
</tr>
</tbody>
</table>[c¼h] Einführung in die Elektronenstrukturtheorie2015-09-30T00:00:00+02:002015-09-30T00:00:00+02:00Michael F. Herbsttag:michael-herbst.com,2015-09-30:/c14h-einfuhrung-in-die-elektronenstrukturtheorie.html<p>The week before my bash scripting course I gave another short talk for
the weekly meeting of the Heidelberg Chaostreff <a href="https://www.noname-ev.de/">NoName
e.V</a>. Unlike <a href="c14h-hartere-crypto-fur-unsere-services.html">last
time</a>
the talk was not concerned with a traditional "Hacker" topic, but much
rather I tried to give a brief introduction into my own research …</p><p>The week before my bash scripting course I gave another short talk for
the weekly meeting of the Heidelberg Chaostreff <a href="https://www.noname-ev.de/">NoName
e.V</a>. Unlike <a href="c14h-hartere-crypto-fur-unsere-services.html">last
time</a>
the talk was not concerned with a traditional "Hacker" topic, but much
rather I tried to give a brief introduction into my own research field.</p>
<p>Of cause 15 to 20 minutes are not enough to go deep into finite-element
Hartree-Fock, so I ended up giving a small introduction into electronic
structure theory instead. Questions which were addressed:</p>
<ul>
<li>Why is electronic structure theory useful? What kind of information
can I retrieve using it?</li>
<li>What are the fundamental physical and chemical concepts that lead to
electronic structure theory?</li>
<li>What kind of approximations and physical ideas do we need to do in
order to get to a method which is used in practice, e.g. the
so-called Hartree-Fock method?</li>
</ul>
<p>The talk was held in German and a recording is available
<a href="http://youtu.be/RWHc1qZ9LSk">on youtube</a>.
Please be aware, however, that not everything I mention is,
scientifically speaking, correct.</p>Advanced bash scripting block course2015-08-16T00:00:00+02:002015-08-16T00:00:00+02:00Michael F. Herbsttag:michael-herbst.com,2015-08-16:/advanced-bash-scripting-block-course.html<p>Currently I am busy preparing the lecture notes and the exercises for
the
<a href="https://michael-herbst.com/teaching/advanced-bash-scripting-2015/">advanced bash scripting</a>
course that I will teach for PhD students of <a href="http://www.mathcomp.uni-heidelberg.de/">my graduate
school</a>. The course will be a
block course running from 24th till 28th August and there are still some
spaces. So in case …</p><p>Currently I am busy preparing the lecture notes and the exercises for
the
<a href="https://michael-herbst.com/teaching/advanced-bash-scripting-2015/">advanced bash scripting</a>
course that I will teach for PhD students of <a href="http://www.mathcomp.uni-heidelberg.de/">my graduate
school</a>. The course will be a
block course running from 24th till 28th August and there are still some
spaces. So in case you are interested in learning how to write shell
scripts in a structured way, you will find more information
<a href="https://michael-herbst.com/teaching/advanced-bash-scripting-2015/">here</a>.
You can obtain the course material either from
<a href="https://michael-herbst.com/teaching/advanced-bash-scripting-2015/">this page</a>
or from my github account
<a href="https://github.com/mfherbst/bash-course">mfherbst</a>.</p>Tor and Tor Browser update scripts2015-08-07T00:00:00+02:002015-08-07T00:00:00+02:00Michael F. Herbsttag:michael-herbst.com,2015-08-07:/tor-and-tor-browser-update-scripts.html<p>A while ago I setup a <a href="https://github.com/mfherbst/">github account</a> in
order to contribute a BugFix to the <a href="https://www.dealii.org/">dealii</a>
Finite Element software I am using for my PhD project. Since then not
very much had happened to the account until I decided to add the
repository <a href="https://github.com/mfherbst/update-scripts">update-scripts</a>
in which I plan to …</p><p>A while ago I setup a <a href="https://github.com/mfherbst/">github account</a> in
order to contribute a BugFix to the <a href="https://www.dealii.org/">dealii</a>
Finite Element software I am using for my PhD project. Since then not
very much had happened to the account until I decided to add the
repository <a href="https://github.com/mfherbst/update-scripts">update-scripts</a>
in which I plan to maintain the scripts I use to update some 3rd party
software on my LinuX machines.</p>
<p>Currently not very many applications are supported. Pretty much only Tor
and the <a href="https://www.torproject.org/projects/torbrowser.html.en">Tor
Browser</a>, both
applications from <a href="https://www.torproject.org/">torproject.org</a>. Since
both are updated fairly often, the packages of my distribution (Debian)
are not recent enough. Especially the Tor Browser is installed on pretty
much all my machines: It is really easy to use and perfect if one just
needs to do some quick checks on the internet. So I decided to develop
scripts that automatically download, cryptographically check and install
the current Tor Browser to the user's HOME directory. If you are
interested to use or contribute to this project, feel free to download
it below or <a href="https://github.com/mfherbst/update-scripts">visit the project
page</a> on github.</p>
<table>
<thead>
<tr>
<th>Link</th>
</tr>
</thead>
<tbody>
<tr>
<td><a href="https://github.com/mfherbst/update-scripts/archive/master.zip">update-scripts repository</a></td>
</tr>
</tbody>
</table>A short return to Cambridge2015-03-26T00:00:00+01:002015-03-26T00:00:00+01:00Michael F. Herbsttag:michael-herbst.com,2015-03-26:/a-short-return-to-cambridge.html<p>About two weeks ago, on Friday the 13th of March, I was invited back to
the <a href="http://www.ch.cam.ac.uk">Chemistry Department</a> at the University of
Cambridge. As part of their Theoretical Chemistry Informal Seminar talk
series I was asked to give a short talk
(<a href="http://talks.cam.ac.uk/talk/index/58012">Abstract</a>) about my current
research. Needless to say that …</p><p>About two weeks ago, on Friday the 13th of March, I was invited back to
the <a href="http://www.ch.cam.ac.uk">Chemistry Department</a> at the University of
Cambridge. As part of their Theoretical Chemistry Informal Seminar talk
series I was asked to give a short talk
(<a href="http://talks.cam.ac.uk/talk/index/58012">Abstract</a>) about my current
research. Needless to say that I am very grateful that Lucy Colwell and
the other organisers provided me with this great opportunity to share
some of the insights we got in the past year. Over afternoon tea I also
had the chance to catch up with some familiar faces from the theory
section and meet some new researchers as well. With great pleasure I
realised that very little has changed in Cambridge when it comes to the
open and welcoming atmosphere that I enjoyed so much during my undergrad days.</p>
<p>The first part of my presentation is very similar to the one I prepared
for our <a href="https://michael-herbst.com/phd-project-finite-element-methods-in-quantum-chemistry.html">group seminar in Kleinwalsertal</a>.
In the section titled "Building the matrices" I describe our approach to
build the relevant stiffness and mass matrices and what difficulties
arise. At this point we cannot really provide acceptable solutions to
these problems, however.</p>
<table>
<thead>
<tr>
<th>Link</th>
</tr>
</thead>
<tbody>
<tr>
<td><a href="https://michael-herbst.com/talks/2015.03.13_Solving_HF_using_FEM.pdf">Invited Talk Cambridge 13.03.2015</a></td>
</tr>
</tbody>
</table>A thankful goodbye2015-03-23T00:00:00+01:002015-03-23T00:00:00+01:00Michael F. Herbsttag:michael-herbst.com,2015-03-23:/a-thankful-goodbye.html<p>As announced on our <a href="http://www.iwr.uni-heidelberg.de/groups/compchem/index.php">group
website</a>,
Dr. Michael Wormit, passed away in a tragic accident a week ago
Saturday, the 14th of March. Michael's loss leaves a gap behind that
cannot be closed — both on a scientific and on a personal level.</p>
<p>I myself am thankful and proud that I …</p><p>As announced on our <a href="http://www.iwr.uni-heidelberg.de/groups/compchem/index.php">group
website</a>,
Dr. Michael Wormit, passed away in a tragic accident a week ago
Saturday, the 14th of March. Michael's loss leaves a gap behind that
cannot be closed — both on a scientific and on a personal level.</p>
<p>I myself am thankful and proud that I had the chance to work with this
fine man for the past year and a bit. I do not think we would have made
as much progress in our research without his outstanding intuition and
expertise. His advice and the many spontaneous discussions we had in
front of our whiteboard in the evenings will be missed most of all. In
these sad hours my thoughts are with his wife and his family.</p>
<p>Goodbye, Michael. You will be remembered.</p>[c¼h] Härtere Crypto für unsere Services2015-02-02T00:00:00+01:002015-02-02T00:00:00+01:00Michael F. Herbsttag:michael-herbst.com,2015-02-02:/c14h-hartere-crypto-fur-unsere-services.html<p>Last Thursday I gave a talk (in German) in our local
<a href="https://www.ccc.de/de/club/chaostreffs">Chaostreff</a> in Heidelberg — the
<a href="https://www.noname-ev.de/">NoName e.V</a>. The main topic was to introduce
the various cryptographic algorithms used in modern cryptography and to
give practical advice how to improve the default configuration even
further. The talk mainly focuses on …</p><p>Last Thursday I gave a talk (in German) in our local
<a href="https://www.ccc.de/de/club/chaostreffs">Chaostreff</a> in Heidelberg — the
<a href="https://www.noname-ev.de/">NoName e.V</a>. The main topic was to introduce
the various cryptographic algorithms used in modern cryptography and to
give practical advice how to improve the default configuration even
further. The talk mainly focuses on <code>openssl</code> ciphers and TLS
connections as well as the settings applicable to the <code>openssh</code> server
and client applications.</p>
<p>As usual you can find the slides attached below. Furthermore thanks to
the camera, which NoName e.V. bought some time ago, there is a recording
of the talk available <a href="http://youtu.be/PtzZM_HB3Wk">on youtube</a>.</p>
<h2>Corrections</h2>
<p>In my talk I got the purpose of MAC codes a little confused. I argue
that a MAC is required in order to prevent a plaintext recovery attack
by using forged ciphertext. That is not quite true, however. The main
purpose is simply to provide integrity for the connection. A good
discussion why it is nevertheless a good idea to use Encrypt-then-MAC
can be found on
<a href="https://crypto.stackexchange.com/questions/202/should-we-mac-then-encrypt-or-encrypt-then-mac">StackExchange Cryptography</a>.</p>
<table>
<thead>
<tr>
<th>Link</th>
</tr>
</thead>
<tbody>
<tr>
<td><a href="https://michael-herbst.com/talks/2015.01.29_c14h_Hartere_Crypto.pdf">[c¼h] Vortrag: Härtere Crypto für unsere Services</a></td>
</tr>
</tbody>
</table>31st Chaos Communication Congress2015-01-08T00:00:00+01:002015-01-08T00:00:00+01:00Michael F. Herbsttag:michael-herbst.com,2015-01-08:/31st-chaos-communication-congress.html<p>Just at the end of last year I went to the
<a href="https://events.ccc.de/congress/2014/wiki/Main_Page">31c3</a>, the annual
hacker's conference organised by the <a href="https://ccc.de">Chaos Computer
Club</a>. I've been following this event via the
<a href="http://streaming.media.ccc.de/">streams</a> and by <a href="https://media.ccc.de/">watching the recorded
videos</a> for many years. This time, however, I
actually made my way to Hamburg. Even …</p><p>Just at the end of last year I went to the
<a href="https://events.ccc.de/congress/2014/wiki/Main_Page">31c3</a>, the annual
hacker's conference organised by the <a href="https://ccc.de">Chaos Computer
Club</a>. I've been following this event via the
<a href="http://streaming.media.ccc.de/">streams</a> and by <a href="https://media.ccc.de/">watching the recorded
videos</a> for many years. This time, however, I
actually made my way to Hamburg. Even though I did not know many people
there, it was a truly great experience, since the atmosphere is very
open and welcoming.</p>
<p>At the first day I felt a little lost in the CCH building with its
labyrinth of confusing escalators and staircases. Also I did not quite
understand the character of the different areas in this gigantic
hackspace. It might seem obvious, but if feels very different if one
chats to someone about his crypto setup at one place called Noisy Square
or if you get someone to explain to you how his new 3D printer works in
the hardware hacking hall — just because of the surroundings.</p>
<p>Most of the four days I spent walking around, chatting to people about
their projects and participating in a few workshops (Lockpicking,
Keysigning Party, Cryptoparty). I also went to a few of the ongoing
talks. From what I have seen so far (either in the audience or recorded)
I can highly recommend the following:</p>
<ul>
<li><a href="https://events.ccc.de/congress/2014/Fahrplan/events/6258.html">Reconstructing
narratives</a><br>
Jacob Appelbaum and Laura Poitras about narratives of the
surveillance state and the most recent batch of Snowden documents.</li>
<li><a href="https://events.ccc.de/congress/2014/Fahrplan/events/6450.html">Ich sehe also bin ich
Du</a><br>
Scary talk about how unreliable biometric authentication
methods are.</li>
<li><a href="https://events.ccc.de/congress/2014/Fahrplan/events/6121.html">What Ever Happened to Nuclear
Weapons</a><br>
History and current status of Nuclear Weapons</li>
<li><a href="https://events.ccc.de/congress/2014/Fahrplan/events/6369.html">ECCHacks</a><br>
Very good talk that made me understand the basics of elliptic
curve cryptography.</li>
<li><a href="https://events.ccc.de/congress/2014/Fahrplan/events/6251.html">State of the
Onion</a><br>
The last year with respect to anonymity in the net and the Tor
Project</li>
<li><a href="https://events.ccc.de/congress/2014/Fahrplan/events/6173.html">Source Code and Cross-Domain Authorship
Attribution</a><br>
How documents and source code can be accurately attributed to
people, and how to protect against it.</li>
<li><a href="https://events.ccc.de/congress/2014/Fahrplan/events/6602.html">The case of Chelsea
Manning</a><br>
Alexa O'Brien discussing the Chelsea Manning case with her lawyers.</li>
<li><a href="https://events.ccc.de/congress/2014/Fahrplan/events/6294.html">Vor Windows 8 wird
gewarnt</a><br>
Why Windows 8 and secure boot are screwed up.</li>
<li><a href="https://events.ccc.de/congress/2014/Fahrplan/events/6574.html">Why are computers so @#!*, and what can we do about
it?</a><br>
Funny talk by one of my former Computer Science lecturers at
Cambridge about the strange design that computers and programming
languages have historically adopted.</li>
</ul>
<p>You can either watch these talks <a href="https://media.ccc.de/">online</a>, or you
can download these or likewise all other talks from this year's congress
using a small Python script, which I wrote. For each talk it
automatically downloads the recorded video, the abstract and all files
which were attached in the
<a href="https://events.ccc.de/congress/2014/Fahrplan">Fahrplan</a>. Essentially
all you need is a plain text file which contains all four-digit talk-ids
line by line. This file should then be passed to the script. For example
to download all talks I mention above, you simply create a file
<code>31c3talks.list</code> with the ids</p>
<div class="highlight"><pre><span></span><code><span class="mf">6258</span><span class="w"></span>
<span class="mf">6450</span><span class="w"></span>
<span class="mf">6121</span><span class="w"></span>
<span class="mf">6369</span><span class="w"></span>
<span class="mf">6251</span><span class="w"></span>
<span class="mf">6173</span><span class="w"></span>
<span class="mf">6602</span><span class="w"></span>
<span class="mf">6294</span><span class="w"></span>
<span class="mf">6574</span><span class="w"></span>
</code></pre></div>
<p>and then you execute</p>
<div class="highlight"><pre><span></span><code>down_frab_videos.py 31c3talks.list
</code></pre></div>
<p>Note, that the script requires the Python modules
<a href="https://pypi.python.org/pypi/requests"><code>requests</code></a>,
<a href="https://pypi.python.org/pypi/beautifulsoup4"><code>beautifulsoup4</code></a>
and some others (see the
<a href="https://github.com/mfherbst/down-frab-videos/blob/master/README.md">README.md</a>).
Feel free to download it <a href="https://github.com/mfherbst/down-frab-videos">from github</a>.</p>Annual Colloquium 20142014-11-26T00:00:00+01:002014-11-26T00:00:00+01:00Michael F. Herbsttag:michael-herbst.com,2014-11-26:/annual-colloquium-2014.html<p>Last Monday and Tuesday this year's most important event of the <a href="http://www.mathcomp.uni-heidelberg.de/">HGS
Mathcomp</a> (my graduate school)
took place — the <a href="http://www.mathcomp.uni-heidelberg.de/index.php?id=254">Annual
Colloquium</a>.
Being part of the organising committee we stepped into the footsteps of
the previous ACs and planned it as a short two-day conference with
graduate students presenting their project …</p><p>Last Monday and Tuesday this year's most important event of the <a href="http://www.mathcomp.uni-heidelberg.de/">HGS
Mathcomp</a> (my graduate school)
took place — the <a href="http://www.mathcomp.uni-heidelberg.de/index.php?id=254">Annual
Colloquium</a>.
Being part of the organising committee we stepped into the footsteps of
the previous ACs and planned it as a short two-day conference with
graduate students presenting their project and invited speakers from
science and industry giving insight into a topic of current interest.
This year we chose the topic for the external talks to be "Small World —
Big Data" with speakers from SAP, IBM, the
<a href="http://www.kit.edu/index.php">KIT</a> in Karlsruhe and the
<a href="http://www.h-its.org/english/">HITS</a> in Heidelberg. You can take a look
at the program in the <a href="http://www.mathcomp.uni-heidelberg.de/fileadmin/Redakteure_Annual_Colloquium/AC2014/boa_2014_online.pdf">Book of
Abstracts</a>.</p>
<p>For me personally those days were quite stressful keeping things going
as smoothly as possible, but it was really nice to see that the previous
weeks of careful planning were finally paying off. We had positive
feedback from many attendees and overall I really had a great time as
well.</p>
<p>Apart from being part in the organising committee I gave a talk about
free software and cryptography, titled "<strong>Everyday cryptography</strong>". In
this talk I explained, why the currently used methods to chat or
exchange emails are flawed and why and how cryptography can be employed
to make the situation better. The slides with many links can be found
below.</p>
<table>
<thead>
<tr>
<th>Link</th>
</tr>
</thead>
<tbody>
<tr>
<td><a href="https://michael-herbst.com/talks/2014.11.24_Everyday_Cryptography.pdf">O3.1 Everyday cryptography HGS AC 2014</a></td>
</tr>
</tbody>
</table>50th Symposium on Theoretical Chemistry (Vienna)2014-10-09T00:00:00+02:002014-10-09T00:00:00+02:00Michael F. Herbsttag:michael-herbst.com,2014-10-09:/50th-symposium-on-theoretical-chemistry.html<p>Last month, from 14th to the 18th of September, I attended my first
conference. Together with many members of my group I was at the <a href="http://stc2014.univie.ac.at/">50th
Symposium on Theoretical Chemistry</a> in
Vienna. It was a great opportunity to present my work, get feedback on
some ideas and get to know …</p><p>Last month, from 14th to the 18th of September, I attended my first
conference. Together with many members of my group I was at the <a href="http://stc2014.univie.ac.at/">50th
Symposium on Theoretical Chemistry</a> in
Vienna. It was a great opportunity to present my work, get feedback on
some ideas and get to know the German speaking quantum chemistry
community. Most lectures were very interesting and I learned a lot about
cutting-edge electronic structure theory.</p>
<p>I myself also presented a poster titled "<strong>New approaches towards a
finite-element based Hartree-Fock method</strong>" (attached below). I was
surprised how many people were interested and I pretty much talked
continuously for the whole of the poster session. In the end I left
Vienna with a bag full of new ideas and with the pleasant feeling that I
became part of the science community.</p>
<table>
<thead>
<tr>
<th>Link</th>
</tr>
</thead>
<tbody>
<tr>
<td><a href="https://michael-herbst.com/poster/2014.09.16_STC_P73_New_approaches_towards_a_finite-element_based_Hartree-Fock_method.pdf">Poster P73 STC 2014</a></td>
</tr>
</tbody>
</table>PhD Project: Finite Element Methods in Quantum Chemistry2014-08-29T00:00:00+02:002014-08-29T00:00:00+02:00Michael F. Herbsttag:michael-herbst.com,2014-08-29:/phd-project-finite-element-methods-in-quantum-chemistry.html<p>In January I officially started my PhD in the <a href="https://www.iwr.uni-heidelberg.de/groups/compchem/index.php">Theoretical Chemistry
group of Prof. Dr. Andreas
Dreuw</a> at
the <a href="https://www.iwr.uni-heidelberg.de/">Interdisziplinäres Zentrum für wissenschaftliches
Rechnen</a> of the
Ruprecht-Karls-Universität Heidelberg. The principle aim of the project
is to replace the common formalism of using atom-centered basis
functions (e.g. Gaussian atomic orbitals …</p><p>In January I officially started my PhD in the <a href="https://www.iwr.uni-heidelberg.de/groups/compchem/index.php">Theoretical Chemistry
group of Prof. Dr. Andreas
Dreuw</a> at
the <a href="https://www.iwr.uni-heidelberg.de/">Interdisziplinäres Zentrum für wissenschaftliches
Rechnen</a> of the
Ruprecht-Karls-Universität Heidelberg. The principle aim of the project
is to replace the common formalism of using atom-centered basis
functions (e.g. Gaussian atomic orbitals) by a finite element basis. The
<a href="https://en.wikipedia.org/wiki/Finite_element_method">finite element
method</a> (FEM),
which was originally developed in the engineering community, has been
applied to problems all over the physical sciences in recent years. In
electronic structure theory, however, it has not really been considered
in great depth so far.</p>
<p>Together with Dr. Michael Wormit — a Postdoc in our group — and Prof.
Dr. Guido Kanschat from the <a href="http://mathsim.eu">Mathematical Methods of Simulation
group</a> we are currently working on a simple first
program that solves the famous
<a href="https://en.wikipedia.org/wiki/Hartree-Fock">Hartree-Fock</a> (HF)
equations using finite elements (FEs). Surely this is only the first
step towards finite-element based Quantum Chemistry and many things can
be done to go beyond HF. We hope. however, that simple calculations will
already give us some understanding of the very different properties of
FEs compared to an Gaussian basis and how these can be used effectively.</p>
<p>A short presentation, which I compiled for our group's yearly seminar in
Hirschegg (Kleinwalsertal, Austria), can be downloaded below.
In said talk I give a brief introduction into
the mathematical formalism of FEs and how the FEM could be applied in an
SCF calculation. </p>
<table>
<thead>
<tr>
<th>Link</th>
</tr>
</thead>
<tbody>
<tr>
<td><a href="https://michael-herbst.com/talks/2014.08.14_Finite_element_method_in_quantum_chemistry.pdf">The finite-element method in quantum chemistry</a></td>
</tr>
</tbody>
</table>Automatically encrypt root mails sent off-site2014-02-14T00:00:00+01:002014-02-14T00:00:00+01:00Michael F. Herbsttag:michael-herbst.com,2014-02-14:/automatically-encrypt-root-mails.html<h1>Motivation</h1>
<p>Especially for server's which are required to run 24/7 it is absolutely
crucial that one stays updated on the machine's status. E.g. using
programs like the <code>smartd</code> daemon of
<a href="http://sourceforge.net/apps/trac/smartmontools/wiki" title="smartmontools">smartmontools</a>
provides a good method to detect ageing hard disk long before failures
actually occur. Like many other …</p><h1>Motivation</h1>
<p>Especially for server's which are required to run 24/7 it is absolutely
crucial that one stays updated on the machine's status. E.g. using
programs like the <code>smartd</code> daemon of
<a href="http://sourceforge.net/apps/trac/smartmontools/wiki" title="smartmontools">smartmontools</a>
provides a good method to detect ageing hard disk long before failures
actually occur. Like many other daemons <code>smartd</code> primarily logs its
findings to syslog. So in order to detect possible problems in advance
we need to crawl through tons of lengthy logfiles each day, right? —
Fortunately not.</p>
<p>A script library called
<a href="http://www.logcheck.org/" title="logcheck"><code>logcheck</code></a> is built to solve
exactly this problem. It looks through the <code>syslog</code> and <code>auth.log</code> (and
in principle many more) logs, filters and summarises the entries and
sends off hourly reports to an email address. As these emails usually
contain sensitive information about the system's internals it is a good
idea to encrypt these using <code>gnupg</code> and public-key cryptography. As it
turns out this process is fully automatable and — if implemented right —
does not only encrypt logcheck emails but in fact all other emails sent
to the root user as well.</p>
<h1>Prerequisites</h1>
<p>On the servers on which we finally want to use the package, we need the
programs</p>
<ul>
<li><code>make</code></li>
<li><code>mime-constuct</code></li>
<li><code>gnupg</code></li>
</ul>
<p>Futhermore a <a href="https://en.wikipedia.org/wiki/Mail_delivery_agent" title="Message transfer agent">mail delivery
agent(MDA)</a>
that understands and interprets sendmail-style <code>.forward</code> files is
required. The recommended
MDA/<a href="https://en.wikipedia.org/wiki/Message_transfer_agent" title="Message Transfer Agent">MTA</a>
system is <a href="http://www.postfix.org/" title="postfix.org"><code>postfix</code></a>. Others
were not tested.</p>
<p>The machine which is used for the preliminary steps (see below) should
have your public key in the current user's <code>gnupg</code> keyring. <code>make</code> is
also required on this machine.</p>
<h1>Setup guide</h1>
<p>The preliminary steps bundle your public key with the <code>encrypt_mail.sh</code>
script and produce a tarball which can be copied to and installed on as
many systems as you want via pretty much a simple <code>make install</code>.</p>
<ol>
<li>Find the <strong>long keyid</strong> of the key you want to use. The long keyid
consists of the last four 2-byte blocks of the fingerprint. E.g. for
<a href="https://michael-herbst.com/contact/">my key</a> this would be
<code>3887DB6744A9DF9F</code>. (Note the omitted spaces)</li>
<li>
<p>Download, untar and run <code>make install-package</code>:</p>
<div class="highlight"><pre><span></span><code>wget https://github.com/mfherbst/root-encrypt/archive/05603e08705e10a493541911dbb27ed7a67e9555.zip -O root-encrypt_dist.zip
unzip root-encrypt_dist.zip
make install-package
</code></pre></div>
<p>In the process of creating the install package, the script will ask
you for the email address you want to deliver the mails to and the
long keyid you determined above:</p>
<div class="highlight"><pre><span></span><code><span class="n">Please</span><span class="w"> </span><span class="n">enter</span><span class="w"> </span><span class="n">the</span><span class="w"> </span><span class="n">email</span><span class="w"> </span><span class="n">address</span><span class="w"> </span><span class="k">to</span><span class="w"> </span><span class="n">forward</span><span class="w"> </span><span class="n">root</span><span class="w"> </span><span class="n">mail</span><span class="w"> </span><span class="k">to</span><span class="err">:</span><span class="w"> </span><span class="n">mail</span><span class="nv">@example</span><span class="p">.</span><span class="n">com</span><span class="w"></span>
<span class="n">Please</span><span class="w"> </span><span class="n">enter</span><span class="w"> </span><span class="n">the</span><span class="w"> </span><span class="n">long</span><span class="w"> </span><span class="k">key</span><span class="w"> </span><span class="n">id</span><span class="w"> </span><span class="k">of</span><span class="w"> </span><span class="n">the</span><span class="w"> </span><span class="k">key</span><span class="w"> </span><span class="n">used</span><span class="w"> </span><span class="k">for</span><span class="w"> </span><span class="nl">encryption</span><span class="p">:</span><span class="w"> </span><span class="mi">3887</span><span class="n">DB6744A9DF9F</span><span class="w"></span>
</code></pre></div>
</li>
</ol>
<p>The resulting install package contains both the current date as well as
your long keyid in its name.</p>
<p>To finish the installation ...</p>
<ol>
<li>Copy the install package to the target hosts.</li>
<li>
<p>(As <code>root</code>) Untar and <code>make install</code></p>
<div class="highlight"><pre><span></span><code>tar xzf *-root-encrypt_*.tar.gz
cd root-encrypt/
make install
</code></pre></div>
</li>
<li>
<p>Test setup by sending a mail to root:</p>
</li>
</ol>
<h1>Detailed description</h1>
<p>The installation creates a new user <code>root-encrypt</code> with home directory
<code>/var/lib/root-encrypt</code> and adds the selected public key to this users
gpg keyring. Whenever mail is sent to this user, the forward file
<code>/var/lib/root-encrypt/.forward</code> instructs the MDA to pipe this mail to
<code>encrypt_mail.sh</code>. The latter script takes this message, encrypts it
with <code>gnupg</code> and your key and sends it off to your email address.
Internally this script buffers the emails in temporary files and uses a
very simple queue-like system to ensure that mails do not get lost if
e.g. power outages or something else happens during processing: Whenever
the next message is sent to <code>root-encrypt</code> all older messages are
processed as well. Note, that this can lead to duplication of messages
if the MDA also re-sends the messages after an error. In short the user
<code>root-encrypt</code> serves as a encrypt-wrapper for mail sent to it.</p>
<p>The missing link to achieve our goal is that all mail sent to <code>root</code> is
forwarded to <code>root-encrypt</code>. Exactly this is achieved by adding the line</p>
<div class="highlight"><pre><span></span><code><span class="n">root</span><span class="o">:</span><span class="w"> </span><span class="n">root</span><span class="o">-</span><span class="n">encrypt</span><span class="w"></span>
</code></pre></div>
<p>to the <code>/etc/aliases</code> file.</p>
<p>Finally one might wonder why it is even necessary to add an extra user.
At first sight it seems more direct to have <code>root</code>'s <code>/root/.forward</code>
file instruct the MDA to pipe all mails to <code>encrypt_mail.sh</code> straight
away. As it turns out this does not quite work as intended. For security
reasons the MDA runs scripts given in <code>/root/.forward</code> not as the <code>root</code>
user, but as <code>nobody</code> instead. And since the <code>nobody</code> user tends to be
used by other services as well for low-privilege work, I figured it was
a wiser choice to add a different user for processing <code>root</code>'s emails.</p>
<h1>Attachments</h1>
<table>
<thead>
<tr>
<th>Link</th>
</tr>
</thead>
<tbody>
<tr>
<td><a href="https://github.com/mfherbst/root-encrypt/archive/05603e08705e10a493541911dbb27ed7a67e9555.zip">root-encrypt_dist.zip</a></td>
</tr>
</tbody>
</table>SSH through SSH SOCKS Proxy2013-10-08T00:00:00+02:002013-10-08T00:00:00+02:00Michael F. Herbsttag:michael-herbst.com,2013-10-08:/ssh-through-ssh-socks-proxy.html<h1>Setting up a SOCKS Proxy using ssh</h1>
<p>Port forwarding in ssh is really great. Next to local and remote
forwarding there is also <em>dynamic port forwarding</em> that essentially
turns the ssh server into a SOCKS Proxy server. This means that you can
actually route all the traffic of a local …</p><h1>Setting up a SOCKS Proxy using ssh</h1>
<p>Port forwarding in ssh is really great. Next to local and remote
forwarding there is also <em>dynamic port forwarding</em> that essentially
turns the ssh server into a SOCKS Proxy server. This means that you can
actually route all the traffic of a local application through the ssh
connection to the host — given that the application supports SOCKS
Proxies. Dynamic port forwarding is set up using the <code>-D</code> switch
followed by a port. The SOCKS proxy is then listening to outgoing
traffic on this chosen port. For example</p>
<div class="highlight"><pre><span></span><code><span class="n">ssh</span><span class="w"> </span><span class="o">-</span><span class="n">N</span><span class="w"> </span><span class="o">-</span><span class="n">C</span><span class="w"> </span><span class="o">-</span><span class="n">D</span><span class="w"> </span><span class="mi">1234</span><span class="w"> </span><span class="k">user</span><span class="nv">@host</span><span class="w"></span>
</code></pre></div>
<p>establishes a proxy under <code>localhost:1234</code> that directs traffic first to
<code>host</code> and then to the desired destination. The <code>-C</code> enables
compression, which increases the performance if mainly text is
transferred. For transferring binary data, compression is almost useless
and one should probably leave out the <code>-C</code> to save the overhead. The
<code>-N</code> tells ssh that you don't want to run any commands, so it won't
allocate a shell. This circumvents issues with hosts where port
forwarding is allowed, but where proper user logins have been disabled
(e.g. by allocating the shell <code>/bin/false</code> to the user).</p>
<h1>ssh connections via a SOCKS proxy</h1>
<p>Amongst the programs that speak SOCKS are all common web browsers and
almost all programs shipped with the major LinuX desktop environments
(e.g. Evolution, Thunderbird, Transmission, ...). SSH itself, however,
cannot connect via a SOCKS proxy out of the box, but with a little help
from the <a href="https://en.wikipedia.org/wiki/Netcat">TCP/IP swiss army knife</a>
<code>netcat</code> it still can be done. E.g. in order to connect to <code>host2</code> via the
proxy we established above, simply run</p>
<div class="highlight"><pre><span></span><code><span class="n">ssh</span><span class="w"> </span><span class="o">-</span><span class="n">o</span><span class="w"> </span><span class="s1">'ProxyCommand nc -x localhost:1234 %h %p'</span><span class="w"> </span><span class="n">user2</span><span class="nv">@host2</span><span class="w"></span>
</code></pre></div>
<p>Here the <code>ProxyCommand</code> option tells ssh to pipe all traffic through the
command that follows. The variables <code>%h</code> and <code>%p</code> are replaced by ssh
with the host and port of the ssh connection, respectively. A few other
such variables exist as well: Check out the manual for further
information. In this case ssh would hence pipe the traffic through
<code>nc -x localhost:1234 host2 22</code>. The latter command, however, is
netcat-speak for opening a plain tcp connection to <code>host2</code> on port <code>22</code>
via the SOCKS Proxy listening on <code>1234</code> on the local machine. Overall we
achieve what we want: ssh through an ssh SOCKS Proxy.</p>
<p>Note that there are two common netcat versions out there: You will need
the OpenBSD version, which is the package <strong>netcat-openbsd</strong> on Debian.</p>
<h1>Why is this useful?</h1>
<p>Many companies and universities have the understandable policy to only
expose a few machines to the public internet, whereas the majority can't
be reached outside a network of trusted IPs. In the terminology of
above, <code>host</code> would be one of the publicly exposed machines. Having
established the SOCKS proxy to <code>host</code> we are part of the trusted network
and can hence use all the internal services offered. For example we can
now logon to the <code>host2</code>, even if it can't be reached directly from the
Internet.</p>
<p>Note, that in principle we only need to be able to forward ports on
<code>host</code>, no rights to properly login on this machine are required.</p>LinuX, SSDs and disk encryption2013-09-15T00:00:00+02:002013-09-15T00:00:00+02:00Michael F. Herbsttag:michael-herbst.com,2013-09-15:/linux-ssds-and-disk-encryption.html<p>Last week I updated the OS on my SSD-only laptop to Debian Jessie and I
thought that would be a good opportunity to finally set up full-disk
encryption(FDE) on that machine as well. Against my usual policy I did
not do so initially fearing the possible performance drawbacks from …</p><p>Last week I updated the OS on my SSD-only laptop to Debian Jessie and I
thought that would be a good opportunity to finally set up full-disk
encryption(FDE) on that machine as well. Against my usual policy I did
not do so initially fearing the possible performance drawbacks from not
being able to use a technology called TRIM. Since cryptsetup 1.4 and a
3.1 kernel it is now possible to use TRIM in such a case as well, giving
the option for a fast SSD-setup with full-disk encryption using
<a href="https://en.wikipedia.org/wiki/LUKS">cryptsetup/LUKS</a>.</p>
<p>Although there are a lot of articles and posts on SSDs out there, I
could not really find a good place summarising all the tweaks I used, so
I decided to add one of my own. Note, that this guide is mainly intended
for single-user machines, which have plenty of RAM — e.g. my system has
4GB. Some of the things mentioned might not be so suitable in other
cases, but feel free to try.</p>
<p>The first two paragraphs are about the internal structure of an SSD, the
TRIM command and how it can partially screw up full disk encryption. The
remaining part of the article is intended to give hints and ideas how to
set up your own system. Finally I provide some references for further
reading.</p>
<h1>Solid-state drives and the TRIM command</h1>
<p>Unlike usual hard drives <a href="https://en.wikipedia.org/wiki/Solid-state_drive">Solid-state
drives</a> or SSDs contain
no "disk" of any kind nor any mechanical system driving them. They are
just made out of flash memory blocks. This makes them a lot quieter, a
lot faster on random read/write access and a lot more energy efficient.
This all has a drawback, however: Each sector has only a specific write
endurance, meaning that having written to it a given number of times, a
sector can't be used any more. Consequently the available drive space of
an SSD shrinks as it gets older. Nowadays the write count we need for
this to happen is quite large. With the correct configuration and some
countermeasures an SSD can easily last for <a href="http://www.storagesearch.com/ssdmyths-endurance.html">tens of
years</a>.</p>
<p>An obvious aim is to try to get all sectors to wear out equally fast,
the so-called <a href="https://en.wikipedia.org/wiki/Write_levelling">write
levelling</a>. To achieve
this the SSD needs to know which sectors are empty on a hardware level.
This is where the <a href="https://en.wikipedia.org/wiki/TRIM" title="TRIM">TRIM</a>
command comes in. It is a way for the
<a href="https://en.wikipedia.org/wiki/Filesystem">filesystem</a> to tell the SSD
which sectors contain no files. So when a file is deleted on the
filesystem level a TRIM is issued and the SSD knows that the respective
sectors can be added to the pool of empty sectors — for use in a later
write operation. TRIM has further effects on some SSD operations like
<a href="https://en.wikipedia.org/wiki/Garbage_collection_(SSD)">garbage
collection</a>.
Without proper garbage collection write operations tend to become much
longer(<a href="https://en.wikipedia.org/wiki/Write_amplification">Write
Amplification</a>) and
hence performance decreases. In summary TRIM both increases the SSD's
lifetime and performance.</p>
<h1>TRIM and disk encryption</h1>
<p><a href="https://en.wikipedia.org/wiki/Dm-crypt">Dm-crypt</a> — the cryptographic
system underlying cryptsetup — adds an additional layer to our picture.
Let <code>/dev/sda1</code> be the partition on the SSD, which contains the
encrypted data. When we open this partition using dm-crypt a <em>virtual
disk</em> is created that contains the decrypted version of the data of
<code>/dev/sda1</code>. Let us refer to this virtual disk as <code>/dev/dm-0</code>. Usually
dm-crypt takes all requests of the filesystem received on <code>/dev/dm-0</code>
and forwards them to <code>/dev/sda1</code>, just de/encrypting data transferred
between them. So if the filesystem issues a TRIM to <code>/dev/dm-0</code> to
indicate that some sectors have been emptied (i.e. a file deleted), we
would expect dm-crypt to forward this request to <code>/dev/sda1</code>. Cryptsetup
versions after 1.4 are capable of doing this, but it is disabled by
default and there is a good reason for that.</p>
<p>One of the features of disk-encryption is that the data written to the
disk looks totally random. So in absence of the appropriate key all
internal structure is hidden. Following the standard procedure we would
first fill a partition with random data and only set up the encryption
afterwards. The result is that an attacker can't even distinguish a
sector that is filled with encrypted data from a sector that is unused,
i.e. still filled with the random data from the set-up. This status is
called <a href="https://en.wikipedia.org/wiki/Plausible_deniability#Use_in_cryptography">plausible
deniability</a>:
We could always deny the existence of any encrypted data on the
partition and argue that the whole partition is just random noise and no
one without the key can disprove this.</p>
<p>Now what happens if we force dm-crypt to pass the TRIM command onto the
SSD? All sectors which are marked as empty on <code>/dev/dm-0</code> via the
filesystem issuing a TRIM will also be marked as empty on <code>/dev/sda1</code>
via dm-crypt forwarding the TRIM. This means that they become well
distinguishable from filled sectors (with the encrypted random data) and
plausible deniability is lost. In extreme cases extreme cases one might
even be able to guess the file system used on the plaintext device
<code>/dev/dm-0</code>!</p>
<p>In summary TRIM for encrypted devices greatly enhances performance but
we loose plausible deniability. I personally consider this to be ok for
my root filesystem, but I chose to keep it disabled for <code>/home</code>. If
performance gets worse I might change my mind later. In any case just be
aware what you are doing.</p>
<h1>During installation — Partitioning and full-disk encryption</h1>
<p>For efficient write levelling as described above we need a sufficiently
large pool of empty sectors. Most SSDs therefore already have a
so-called sparse area of sectors, which are invisible to the operating
system and are hence always empty. Furthermore we can easily leave a
part of the SSD unpartitioned to artificially increase the sparse area.
Most articles I read recommend about <strong>10% of unpartitioned space</strong>.
Especially when not enabling TRIM for all encrypted partitions you might
want to increase this value a little.</p>
<p>Now the <strong>filesystem</strong> choice. One might first think that a good idea
would be a filesystem without a journal in order to avoid the overhead
of metadata writes when files are modified or deleted. As Theodore Ts’o
points out in <a href="http://thunk.org/tytso/blog/2009/03/01/ssds-journaling-and-noatimerelatime/">his blog
[5]</a>,
this is not quite the case. After his analysis he recommends using
<strong>ext4 with journal</strong> and using some mount options I'll go into in the
section about the <code>/etc/fstab</code>. His analysis further shows that
especially write-intensive operations (in his case the <code>make clean</code>)
still place a much higher write load if journaling is used. We will
therefore try to avoid all unnecessary writes by mounting various
folders as ramdisks.</p>
<p>If you have enough RAM available (as I said I have 4GB), I would
consider <strong>omitting a SWAP partition</strong>. If you still want SWAP you
should <a href="http://blog.philippklaus.de/2011/02/ssd-optimizations-on-linux/">disable hibernation for all
users</a>
and also <a href="https://lwn.net/Articles/83830/">set the swappiness</a> to zero
after the installation. This will tell the kernel that the SWAP should
only be touched if there is no physical memory left. For the remaining
article I assume you have chosen to not create a SWAP partition.</p>
<p>Keeping the above in mind, use the installer of your distribution to do
your favourite <strong>full disk encryption</strong> setup! It should automatically
overwrite the disk with random data before creating the encrypted
partitions (at least the Debian installer does this). I don't like the
extra overhead of a <a href="https://en.wikipedia.org/wiki/Logical_Volume_Manager_(Linux)">Logical Volume
Manager</a>,
so I don't use it. Instead I just create 3 separate partitions, one for
the root filesystem(about
20<a href="https://en.wikipedia.org/wiki/Binary_prefix">GiB</a>), one for
<code>/boot</code>(about 100MiB) and a <code>/home</code> partition. On my 120GB SSD this
yields:</p>
<table>
<thead>
<tr>
<th>Size</th>
<th>Mount point</th>
<th>File system</th>
<th>Purpose</th>
</tr>
</thead>
<tbody>
<tr>
<td>20GiB</td>
<td><code>/</code></td>
<td>ext4 inside LUKS</td>
<td>Root filesystem (encrypted)</td>
</tr>
<tr>
<td>75GiB</td>
<td><code>/home</code></td>
<td>ext4 inside LUKS</td>
<td>Home filesystem (encrypted)</td>
</tr>
<tr>
<td>100MiB</td>
<td><code>/boot</code></td>
<td>ext4</td>
<td>Boot partition</td>
</tr>
<tr>
<td>ca. 17GiB</td>
<td></td>
<td></td>
<td><em>unpartitioned</em></td>
</tr>
</tbody>
</table>
<p>With this setup you will need to provide two separate passwords for the
encrypted <code>/home</code> and root partition (Which can surely be the same). In
the initial setup you will need to type both of these to start up your
system, but we will change this in the next section. The only password
you will need after this is the one for the root partition. Keep that in
mind when choosing your passwords.</p>
<h1>After installation — Configuring dm-crypt for TRIM and use LUKS key derivation</h1>
<p>For both of the these things we need to edit <code>/etc/crypttab</code> (as root).
Your installer should have created entries for the encrypted partitions
automatically and the file should look similar to</p>
<div class="highlight"><pre><span></span><code>#/dev/mapper name Device UUID req_devs cryptdevice options
sda1_crypt UUID=000000000-...00 none luks
sda2_crypt UUID=000000000-...00 none luks
</code></pre></div>
<p>To instruct dm-crypt to pass on the TRIM on, you simply need to add an
<code>discard</code> to the block of cryptdevice options:</p>
<div class="highlight"><pre><span></span><code>#/dev/mapper name Device UUID req_devs cryptdevice options
sda1_crypt UUID=000000000-...00 none luks,discard
sda2_crypt UUID=000000000-...00 none luks,discard
</code></pre></div>
<p>As promised we now <strong>use key derivation</strong> to reduce the number of
passwords we need to provide at start-up to only one. This is done using
the pre-installed script <code>/lib/cryptsetup/scripts/decrypt_derived</code>. It
takes an unlocked LUKS partition and uses its (static) LUKS header to
derive a password from it. If this password is added as a key to other
LUKS partitions we only need to unlock the first partition and we can
then use the script to unlock the others automatically. Since the root
partition needs to be unlocked first in either case, it makes sense to
derive the other keys from this partition. In my case
<code>/dev/mapper/sda1_crypt</code> is the mapper for the root partition and I run
the commands</p>
<div class="highlight"><pre><span></span><code># <span class="nv">create</span> <span class="nv">temporary</span> <span class="nv">ramdisk</span>
<span class="nv">mkdir</span> <span class="o">/</span><span class="nv">mnt</span><span class="o">/</span><span class="nv">ram</span> <span class="o">&&</span> <span class="nv">mount</span> <span class="o">-</span><span class="nv">t</span> <span class="nv">ramfs</span> <span class="o">-</span><span class="nv">o</span> <span class="nv">size</span><span class="o">=</span><span class="mi">1</span><span class="nv">m</span> <span class="nv">ramfs</span> <span class="o">/</span><span class="nv">mnt</span><span class="o">/</span><span class="nv">ram</span>
<span class="nv">chmod</span> <span class="mi">600</span> <span class="o">/</span><span class="nv">mnt</span><span class="o">/</span><span class="nv">ram</span>
# <span class="nv">derive</span> <span class="nv">a</span> <span class="nv">key</span> <span class="nv">from</span> <span class="nv">root</span> <span class="nv">partition</span> <span class="nv">and</span> <span class="nv">save</span>
# <span class="nv">to</span> <span class="nv">temporary</span> <span class="nv">storage</span>
<span class="o">/</span><span class="nv">lib</span><span class="o">/</span><span class="nv">cryptsetup</span><span class="o">/</span><span class="nv">scripts</span><span class="o">/</span><span class="nv">decrypt_derived</span> <span class="nv">sda1_crypt</span> <span class="o">></span> <span class="o">/</span><span class="nv">mnt</span><span class="o">/</span><span class="nv">ram</span><span class="o">/</span><span class="nv">tmp</span>.<span class="nv">key</span>
# <span class="nv">Add</span> <span class="nv">key</span> <span class="nv">to</span> <span class="nv">other</span> <span class="nv">encrypted</span> <span class="nv">partitions</span>
# <span class="nv">Needs</span> <span class="nv">a</span> <span class="nv">valid</span> <span class="nv">Key</span> <span class="k">for</span> <span class="nv">this</span> <span class="nv">partition</span>
<span class="nv">cryptsetup</span> <span class="nv">luksAddKey</span> <span class="o">/</span><span class="nv">dev</span><span class="o">/</span><span class="nv">sda2</span> <span class="o">/</span><span class="nv">mnt</span><span class="o">/</span><span class="nv">ram</span><span class="o">/</span><span class="nv">tmp</span>.<span class="nv">key</span>
# <span class="nv">Add</span> <span class="nv">to</span> <span class="nv">more</span> <span class="k">if</span> <span class="nv">required</span> ...
# <span class="nv">Remove</span> <span class="nv">the</span> <span class="nv">key</span> <span class="nv">and</span> <span class="nv">clean</span> <span class="nv">up</span>
<span class="nv">rm</span> <span class="o">/</span><span class="nv">mnt</span><span class="o">/</span><span class="nv">ram</span><span class="o">/</span><span class="nv">tmp</span>.<span class="nv">key</span>
<span class="nv">umount</span> <span class="o">/</span><span class="nv">mnt</span><span class="o">/</span><span class="nv">ram</span>
<span class="nv">rmdir</span> <span class="o">/</span><span class="nv">mnt</span><span class="o">/</span><span class="nv">ram</span>
</code></pre></div>
<p>to add the key derived off the unlocked root partition to the other
encrypted LUKS partitions. Note that it is a good idea to keep the old,
non-derived keys of the devices as well in case the LUKS header of the
root partition gets damaged.</p>
<p>Finally we need to tell the system how to unlock itself at start-up,
which is again done in the <code>/etc/crypttab</code> file:</p>
<div class="highlight"><pre><span></span><code>#/dev/mapper name Device UUID req_devs cryptdevice opts
sda1_crypt UUID=000000000-...00 none luks,discard
sda2_crypt UUID=000000000-...00 sda1_crypt luks,discard,keyscript=/lib/cryptsetup/scripts/decrypt_derived
</code></pre></div>
<p>By putting the entry <code>sda1_crypt</code> in the <code>req_devs</code> column we indicate
that the <code>sda1_crypt</code> partition should be used to derive the key for
<code>sda2_crypt</code> using the keyscript.</p>
<p>Now <strong>update the <a href="https://en.wikipedia.org/wiki/Initrd">initial
ramdisk</a></strong>: Run as root</p>
<div class="highlight"><pre><span></span><code>update-initramfs -u -k all
</code></pre></div>
<p>and then reboot the machine to check if unlocking the devices works as
intended. Afterwards you can also verify if dm-crypt indeed allows TRIM
on the devices by running</p>
<div class="highlight"><pre><span></span><code>dmsetup table /dev/mapper/sda1_crypt
dmsetup table /dev/mapper/sda2_crypt
#...
</code></pre></div>
<p>on the encrypted devices. The output should contain a <code>allow_discards</code>
flag for each of them.</p>
<h1>After installation — Editing <code>/etc/fstab</code></h1>
<p>For each partition that is located on an SSD drive you should at least
add the <strong>mount options</strong> <code>noatime</code> and <code>nodiratime</code> (see
<a href="http://thunk.org/tytso/blog/2009/03/01/ssds-journaling-and-noatimerelatime/">[5]</a>).
These options will suppress the usual bookkeeping of access times for
files and directories, respectively. Usually accessing a file implies a
write as well since the current date and time will be written to the
so-called <a href="https://en.wikipedia.org/wiki/Inode">inode</a> table of the
filesystem. For most applications this is not required and can therefore
be safely disabled using the above options.</p>
<p>On top of that we can further provide the <code>discard</code> option selectively
to those partitions where we want TRIM to be used — keeping in mind the
consequences this has for plausible deniability (see above). In my case
the <code>/etc/fstab</code> than looks similar to</p>
<div class="highlight"><pre><span></span><code>#file system point type options dump pass
#My root partition:
/dev/mapper/sda1_crypt / ext4 noatime,nodiratime,discard,errors=remount-ro 0 1
#
#Boot partition:
UUID=00000000-...00 /boot ext4 noatime,nodiratime,discard 0 2
#
#Home partition (no TRIM)
#/dev/mapper/sda2_crypt /home ext4 noatime,nodiratime 0 2
</code></pre></div>
<h1>After installation — Setting up ramdisks using <code>tmpfs</code></h1>
<p>Next to the mount options a good paradigm is to use
<a href="https://en.wikipedia.org/wiki/Ramdisk">ramdisks</a> for <strong>temporary
files</strong> to avoid wasting writes for files we don't want to keep anyway.
Looking at the UNIX <a href="https://en.wikipedia.org/wiki/Filesystem_Hierarchy_Standard">Filesystem Hierarchy
Standard</a>
we note that temporary files are stored at <code>/tmp</code> and (in practice to a
lesser extent) at <code>/var/tmp</code>. Moving the files in these locations to RAM
can now be easily achieved by mounting the corresponding paths with the
filesystem type <code>tmpfs</code>. This will create a <em>dynamical</em> ramdisk, meaning
that only as much RAM is used up by the disk as it requires to store the
files. A maximum size can also specified via the mount option
<code>size=XXX</code>.</p>
<p>If you can spare a little more RAM you should furthermore consider the
use of ramdisks for locations in the file system tree where <strong>frequent
writes</strong> take place, e.g. <code>/var/log</code> and <code>/var/spool</code>. You might be
aware that <code>/var/log</code> contains the system's logfiles, meaning that by
applying these settings you will loose all logs on system reboot. For
servers this would surely be a problem, but in the case of laptops I
don't want to loose this option of saving a lot of writes. And honestly:
How often do you really need to check logs on a laptop system that are
days old?</p>
<p><code>/var/spool</code> is the location where files are saved that wait for a task
to finish, e.g. printer queues or queues for <code>cron</code> or
<a href="https://en.wikipedia.org/wiki/At_%28Unix%29"><code>at</code></a>. So for example
printer queues or <code>at</code> jobs won't be kept across reboots any more if
<code>/var/spool</code> is a ramdisk — again this is something I don't really need
on a laptop.</p>
<p>A further problem with having <code>/var/log</code> and <code>/var/spool</code> in RAM is that
the respective folder structures will also be discarded. When the system
starts up many services and init-scripts will, however, assume the
presence of certain folders with certain permissions in <code>/var/log</code>. The
simple solution is to backup the folder structure on shutdown and to
restore it when the system starts up. In principle this is what the
attached script <a href="#attachments">varFolders</a>
does. On top of that it also offers an option to keep all log files
(e.g. for debugging purposes) and a check to avoid updating the backup
if nothing has changed.</p>
<p>For the setup of the discussed ramdisks, I first added the following
lines to the <code>/etc/fstab</code>:</p>
<div class="highlight"><pre><span></span><code><span class="c1">#file systems point type options dump pass</span><span class="w"></span>
<span class="n">none</span><span class="w"> </span><span class="o">/</span><span class="n">tmp</span><span class="w"> </span><span class="n">tmpfs</span><span class="w"> </span><span class="n">noatime</span><span class="p">,</span><span class="n">size</span><span class="o">=</span><span class="mi">1024</span><span class="n">m</span><span class="w"> </span><span class="mi">0</span><span class="w"> </span><span class="mi">0</span><span class="w"></span>
<span class="n">none</span><span class="w"> </span><span class="o">/</span><span class="k">var</span><span class="o">/</span><span class="n">tmp</span><span class="w"> </span><span class="n">tmpfs</span><span class="w"> </span><span class="n">noatime</span><span class="p">,</span><span class="n">size</span><span class="o">=</span><span class="mi">256</span><span class="n">m</span><span class="w"> </span><span class="mi">0</span><span class="w"> </span><span class="mi">0</span><span class="w"></span>
<span class="n">none</span><span class="w"> </span><span class="o">/</span><span class="k">var</span><span class="o">/</span><span class="n">spool</span><span class="w"> </span><span class="n">tmpfs</span><span class="w"> </span><span class="n">noatime</span><span class="p">,</span><span class="n">size</span><span class="o">=</span><span class="mi">512</span><span class="n">m</span><span class="w"> </span><span class="mi">0</span><span class="w"> </span><span class="mi">0</span><span class="w"></span>
<span class="n">none</span><span class="w"> </span><span class="o">/</span><span class="k">var</span><span class="o">/</span><span class="nb">log</span><span class="w"> </span><span class="n">tmpfs</span><span class="w"> </span><span class="n">noatime</span><span class="p">,</span><span class="n">size</span><span class="o">=</span><span class="mi">512</span><span class="n">m</span><span class="w"> </span><span class="mi">0</span><span class="w"> </span><span class="mi">0</span><span class="w"></span>
</code></pre></div>
<p>Remember that <code>tmpfs</code> is a dynamic ramdisk, so this does not mean you
are actually filling roughly 2.2GiB of RAM — especially when emptying
the logs on reboot you need hardly more than a few tens of MB for
<code>/var/tmp</code>, <code>/var/spool</code> and <code>/var/log</code> together.</p>
<p>Then I executed the following as root to make use of varFolders:</p>
<div class="highlight"><pre><span></span><code><span class="c1"># download and install varFolders:</span><span class="w"></span>
<span class="n">wget</span><span class="w"> </span><span class="o">-</span><span class="n">O</span><span class="w"> </span><span class="o">/</span><span class="n">etc</span><span class="o">/</span><span class="n">init</span><span class="o">.</span><span class="n">d</span><span class="o">/</span><span class="n">varFolders</span><span class="w"> </span><span class="n">https</span><span class="p">:</span><span class="o">//</span><span class="n">gist</span><span class="o">.</span><span class="n">githubusercontent</span><span class="o">.</span><span class="n">com</span><span class="o">/</span><span class="n">mfherbst</span><span class="o">/</span><span class="n">c38d5acb0b6c48e90054c53d805f73e7</span><span class="o">/</span><span class="n">raw</span><span class="o">/</span><span class="mi">35218</span><span class="n">ca60bd1012bc1128b8cec8d7aee71ba359c</span><span class="o">/</span><span class="n">varFolders</span><span class="w"></span>
<span class="n">update</span><span class="o">-</span><span class="n">rc</span><span class="o">.</span><span class="n">d</span><span class="w"> </span><span class="n">varFolders</span><span class="w"> </span><span class="n">defaults</span><span class="w"></span>
<span class="c1"># create first image:</span><span class="w"></span>
<span class="o">/</span><span class="n">etc</span><span class="o">/</span><span class="n">init</span><span class="o">.</span><span class="n">d</span><span class="o">/</span><span class="n">varFolders</span><span class="w"> </span><span class="n">stop</span><span class="w"></span>
<span class="c1"># Move directories and create empty ones</span><span class="w"></span>
<span class="k">for</span><span class="w"> </span><span class="n">DIR</span><span class="w"> </span><span class="ow">in</span><span class="w"> </span><span class="o">/</span><span class="k">var</span><span class="o">/</span><span class="n">tmp</span><span class="w"> </span><span class="o">/</span><span class="k">var</span><span class="o">/</span><span class="nb">log</span><span class="w"> </span><span class="o">/</span><span class="k">var</span><span class="o">/</span><span class="n">spool</span><span class="p">;</span><span class="w"> </span><span class="n">do</span><span class="w"></span>
<span class="w"> </span><span class="n">mv</span><span class="w"> </span><span class="o">$</span><span class="n">DIR</span><span class="w"> </span><span class="o">$</span><span class="n">DIR</span><span class="o">.</span><span class="mi">1</span><span class="w"> </span><span class="o">&&</span><span class="w"> </span>\<span class="w"></span>
<span class="w"> </span><span class="n">mkdir</span><span class="w"> </span><span class="o">$</span><span class="n">DIR</span><span class="w"> </span><span class="o">&&</span><span class="w"> </span>\<span class="w"></span>
<span class="w"> </span><span class="n">chmod</span><span class="w"> </span><span class="o">--</span><span class="n">reference</span><span class="o">=$</span><span class="n">DIR</span><span class="o">.</span><span class="mi">1</span><span class="w"> </span><span class="o">$</span><span class="n">DIR</span><span class="w"> </span><span class="o">&&</span><span class="w"> </span>\<span class="w"></span>
<span class="w"> </span><span class="n">chown</span><span class="w"> </span><span class="o">--</span><span class="n">reference</span><span class="o">=$</span><span class="n">DIR</span><span class="o">.</span><span class="mi">1</span><span class="w"> </span><span class="o">$</span><span class="n">DIR</span><span class="w"> </span><span class="o">&&</span><span class="w"> </span>\<span class="w"></span>
<span class="w"> </span><span class="n">mount</span><span class="w"> </span><span class="o">--</span><span class="n">bind</span><span class="w"> </span><span class="o">$</span><span class="n">DIR</span><span class="o">.</span><span class="mi">1</span><span class="w"> </span><span class="o">$</span><span class="n">DIR</span><span class="w"> </span><span class="o">&&</span><span class="w"> </span>\<span class="w"></span>
<span class="w"> </span><span class="n">echo</span><span class="w"> </span><span class="n">successfully</span><span class="w"> </span><span class="n">prepared</span><span class="w"> </span><span class="o">$</span><span class="n">DIR</span><span class="w"></span>
<span class="n">done</span><span class="w"></span>
<span class="c1"># If all directories are successfully prepared</span><span class="w"></span>
<span class="c1"># ie we get the three echos "successfully prepared $DIR"</span><span class="w"></span>
<span class="c1"># then proceed -- if not find the error and correct it!</span><span class="w"></span>
<span class="n">reboot</span><span class="w"></span>
</code></pre></div>
<p>After the reboot check if all services come up properly and if
<code>/var/log</code> is populated with folders as expected. If that is the case
the temporary folders <code>/var/tmp.1</code>, <code>/var/log.1</code> and <code>/var/spool.1</code> can
be removed.</p>
<h1>After installation — Changing the disk scheduler</h1>
<p>The <a href="https://en.wikipedia.org/wiki/I/O_scheduling">disk scheduler</a>
controls how the kernel schedules access to storage drives. For rotating
disks data access is obviously a slow process and it needs to be
distributed fairly between the many processes that might want to read or
write something to/from different parts of the disk. The default
scheduler called <a href="https://en.wikipedia.org/wiki/CFQ">cfq</a> is optimised
for the latter scenario, but not for SSDs. Performance can be greatly
improved if <a href="https://en.wikipedia.org/wiki/Noop_scheduler">noop</a> or
<a href="https://en.wikipedia.org/wiki/Deadline_scheduler">deadline</a> are used
instead. Noop is the most simple one: It basically handles requests in
the order they are submitted. Deadline in some sense implements a timer
on top and stops requests that take too long. Which one to use pretty
much depends on the underlying hardware and the use case, but for a
normal laptop and a normal use it should not make much of a difference.</p>
<p>Changing the scheduler can be done in two ways. First, one can tell the
kernel to use an alternative scheduler for all disks via a <strong>boot
option</strong>. This is done by editing <code>/etc/default/grub</code> and adding
<code>elevator=deadline</code> to the variable <code>GRUB_CMDLINE_LINUX_DEFAULT</code>, e.g.</p>
<div class="highlight"><pre><span></span><code>GRUB_CMDLINE_LINUX_DEFAULT="quiet splash elevator=deadline"
</code></pre></div>
<p>Afterwards run <code>update-grub2</code> as root. Whist this is nice and simple, it
implies that all drives are scheduled in that way and hence it is only
really applicable for an SSD-only system.</p>
<p>The second option dynamically assigns the disk scheduler depending on
the type of disk. This can be achieved using
<strong><a href="https://en.wikipedia.org/wiki/Udev">udev</a> rules</strong>. For example create
the file <code>/etc/udev/rules.d/60-schedulers.rules</code></p>
<div class="highlight"><pre><span></span><code># <span class="nv">set</span> <span class="nv">deadline</span> <span class="nv">scheduler</span> <span class="k">for</span> <span class="nv">non</span><span class="o">-</span><span class="nv">rotating</span> <span class="nv">disks</span>
<span class="nv">ACTION</span><span class="o">==</span><span class="s2">"</span><span class="s">add|change</span><span class="s2">"</span>, <span class="nv">KERNEL</span><span class="o">==</span><span class="s2">"</span><span class="s">sd[a-z]</span><span class="s2">"</span>, <span class="nv">ATTR</span>{<span class="nv">queue</span><span class="o">/</span><span class="nv">rotational</span>}<span class="o">==</span><span class="s2">"</span><span class="s">0</span><span class="s2">"</span>, <span class="nv">ATTR</span>{<span class="nv">queue</span><span class="o">/</span><span class="nv">scheduler</span>}<span class="o">=</span><span class="s2">"</span><span class="s">deadline</span><span class="s2">"</span>
# <span class="nv">set</span> <span class="nv">cfq</span> <span class="nv">scheduler</span> <span class="k">for</span> <span class="nv">rotating</span> <span class="nv">disks</span>
<span class="nv">ACTION</span><span class="o">==</span><span class="s2">"</span><span class="s">add|change</span><span class="s2">"</span>, <span class="nv">KERNEL</span><span class="o">==</span><span class="s2">"</span><span class="s">sd[a-z]</span><span class="s2">"</span>, <span class="nv">ATTR</span>{<span class="nv">queue</span><span class="o">/</span><span class="nv">rotational</span>}<span class="o">==</span><span class="s2">"</span><span class="s">1</span><span class="s2">"</span>, <span class="nv">ATTR</span>{<span class="nv">queue</span><span class="o">/</span><span class="nv">scheduler</span>}<span class="o">=</span><span class="s2">"</span><span class="s">cfq</span><span class="s2">"</span>
</code></pre></div>
<p>to use cfq and deadline.</p>
<h1>After installation — Enabling device-level write cache</h1>
<p>Turning on the write cache of a drive causes the drive to use some
internal RAM to cache files until they are written to the disk properly.
This improves performance in both SSDs as well as traditional hard
drives. A disadvantage is that in the case of an abrupt power loss
enabling this feature could lead to data loss. Since data is usually
only kept in the disk cache for extremely short times this is hardly a
problem for normal use cases.</p>
<p>First check if write-caching is not enabled already by running (as root)</p>
<div class="highlight"><pre><span></span><code>hdparm -W /dev/sda
</code></pre></div>
<p>If this command fails your drive either has no write cache feature or
<code>hdparm</code> can' access it. In either case you can't change the setting and
should proceed to the next section. If it returns something along</p>
<div class="highlight"><pre><span></span><code>/dev/sda:
write-caching = 1 (on)
</code></pre></div>
<p>write caching is already enabled. Otherwise (i.e. for
<code>write-caching = 0</code>) you can enable it by issuing</p>
<div class="highlight"><pre><span></span><code>hdparm -W1 /dev/sda
</code></pre></div>
<p>Note that this setting is not preserved across reboots, so in order to
make this change permanent, you have to add the line issued above to the
file <code>/etc/rc.local</code> before the final <code>exit 0</code>.</p>
<h1>After installation — User-level tweaks</h1>
<p>On a user level it is mostly the browser cache which is very predictable
to cause a lot of writes. Now that <code>/tmp</code> is located in RAM it is a good
idea to make the browser drop its cache at this location. For firefox
you do the following:</p>
<ol>
<li>Navigate to <code>about:config</code> and confim that you know what you are
doing</li>
<li>Left-click > New > String</li>
<li>Call the new entry "browser.cache.disk.parent_directory"</li>
<li>Give it the value "/tmp"</li>
<li>Restart the browser to have it write the cache to <code>/tmp</code></li>
</ol>
<p>Something similar should be possible in other browsers like chromium as
well.</p>
<p>Last but not least: Use <code>/tmp</code> frequently, e.g.:</p>
<ul>
<li>Unpack tarballs to <code>/tmp</code> if you do not intent to keep the files
for long.</li>
<li>Place temporary downloads in <code>/tmp</code> (if they are small enough).</li>
<li>Compile in <code>/tmp</code> if you are only interested in the
final executable.</li>
</ul>
<h1>References</h1>
<ul>
<li>[1] <a href="http://wiki.ubuntuusers.de/SSD/Grundlagen">http://wiki.ubuntuusers.de/SSD/Grundlagen</a> (in German)</li>
<li>[2] <a href="http://www.thomas-krenn.com/de/wiki/Solid_State_Drive">http://www.thomas-krenn.com/de/wiki/Solid_State_Drive</a> (in German)</li>
<li>[3] <a href="http://chriseiffel.com/everything-linux/how-to-set-up-an-ssd-on-linux/">http://chriseiffel.com/everything-linux/how-to-set-up-an-ssd-on-linux/</a></li>
<li>[4] <a href="http://asalor.blogspot.com/2011/08/trim-dm-crypt-problems.html">http://asalor.blogspot.com/2011/08/trim-dm-crypt-problems.html</a></li>
<li>[5] <a href="http://thunk.org/tytso/blog/2009/03/01/ssds-journaling-and-noatimerelatime/">http://thunk.org/tytso/blog/2009/03/01/ssds-journaling-and-noatimerelatime/</a></li>
<li>[6] <a href="http://worldsmostsecret.blogspot.com/2012/04/how-to-activate-trim-on-luks-encrypted.html">http://worldsmostsecret.blogspot.com/2012/04/how-to-activate-trim-on-luks-encrypted.html</a></li>
</ul>
<h1 id="attachments">Attachments</h1>
<ul>
<li><a href="https://gist.github.com/mfherbst/c38d5acb0b6c48e90054c53d805f73e7"><code>varFolders</code></a></li>
</ul>
<div class="highlight"><pre><span></span><code><span class="ch">#!/bin/bash</span>
<span class="c1">### BEGIN INIT INFO</span>
<span class="c1"># Provides: varFolders</span>
<span class="c1"># Required-Start: $local_fs</span>
<span class="c1"># Required-Stop: $local_fs</span>
<span class="c1"># Default-Start: 1 2 3 4 5</span>
<span class="c1"># Default-Stop: 0 6</span>
<span class="c1"># Short-Description: Backup and restore var folder structures</span>
<span class="c1">### END INIT INFO</span>
<span class="c1">#</span>
<span class="c1"># Script Version 1.0 (14/09/2013)</span>
<span class="c1">#</span>
<span class="c1">##################################################################################</span>
<span class="c1"># Licence:</span>
<span class="c1">#</span>
<span class="c1"># (C) 2013 Michael F. Herbst <info@michael-herbst.com></span>
<span class="c1">#</span>
<span class="c1"># This script is free software: you can redistribute it and/or modify</span>
<span class="c1"># it under the terms of the GNU General Public License as published by</span>
<span class="c1"># the Free Software Foundation, either version 3 of the License, or</span>
<span class="c1"># (at your option) any later version.</span>
<span class="c1">#</span>
<span class="c1"># It is distributed in the hope that it will be useful,</span>
<span class="c1"># but WITHOUT ANY WARRANTY; without even the implied warranty of</span>
<span class="c1"># MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the</span>
<span class="c1"># GNU General Public License for more details.</span>
<span class="c1">#</span>
<span class="c1"># A copy of the GNU General Public License can be found</span>
<span class="c1"># at <http://www.gnu.org/licenses/>.</span>
<span class="c1">#</span>
<span class="c1">##################################################################################</span>
<span class="c1"># Usage:</span>
<span class="c1">#</span>
<span class="c1"># run with "stop" to create a tar archive representing the structure of a sub-</span>
<span class="c1"># directory of var. This means that no actual files, only symbolic links and</span>
<span class="c1"># folders are kept. Run with "start" to restore this structure back to the original</span>
<span class="c1"># location.</span>
<span class="c1">#</span>
<span class="c1"># The idea is that this allows write-intensive subfolders of var (eg. log, spool .. )</span>
<span class="c1"># to be mounted as ramdisks using tmpfs in a SSD-only system to reduce the amounts</span>
<span class="c1"># of writes the SSD suffers.</span>
<span class="c1">#</span>
<span class="c1"># DIRS are the subfolers of var which are subject to this treatment. With the </span>
<span class="c1"># KEEPFILES the user can control if only the structure or everything should be </span>
<span class="c1"># stored. (Eg. if something is debugged and files should be kept across power cyles)</span>
<span class="c1">#</span>
<span class="c1"># The script also checks for presence of /etc/var_folders_keep_files. If this file</span>
<span class="c1"># is present the script will automatically set KEEPFILES="y".</span>
<span class="c1">#</span>
<span class="c1"># If KEEPFILES="n" the archive will only be updated if the names of the subfolders /</span>
<span class="c1"># links have changed or if subfolders / links have been created / deleted. Other</span>
<span class="c1"># changes (eg permission changes, owner changes, ...) won't trigger an update</span>
<span class="c1"># automatically. You can however force an update at the next invocation of "stop"</span>
<span class="c1"># by calling varFolders with the option "force-update".</span>
<span class="c1">#</span>
<span class="c1">##################################################################################</span>
<span class="c1"># Settings:</span>
<span class="nv">KEEPFILES</span><span class="o">=</span><span class="s2">"n"</span> <span class="c1">#Keep the files or only the folder structure?</span>
<span class="nv">DIRS</span><span class="o">=</span><span class="s2">"log spool"</span> <span class="c1">#dirs under /var to extract folder structure/files from</span>
<span class="nv">ARFILE</span><span class="o">=</span><span class="s2">"var_folders.tar"</span> <span class="c1">#Archive to keep stuff in (stored under /var/local)</span>
<span class="c1">##################################################################################</span>
. /lib/lsb/init-functions
<span class="nb">set</span> -e
<span class="nv">PATH</span><span class="o">=</span>/sbin:/bin:/usr/sbin:/usr/bin
<span class="nv">PROG</span><span class="o">=</span>varFolders
initialise<span class="o">(){</span>
<span class="c1">#set DIRS, DIR1 and KEEPFILES</span>
<span class="nb">local</span> <span class="nv">DTMP</span><span class="o">=</span>
<span class="k">for</span> DIR <span class="k">in</span> <span class="nv">$DIRS</span><span class="p">;</span> <span class="k">do</span>
<span class="k">if</span> <span class="o">[</span> <span class="s2">"</span><span class="nv">$DIR</span><span class="s2">"</span> <span class="o">==</span> <span class="s2">"local"</span> <span class="o">]</span><span class="p">;</span> <span class="k">then</span>
<span class="nb">echo</span> <span class="s2">"Can't store the directory structure of /var/local!"</span>
<span class="nb">exit</span> <span class="m">1</span>
<span class="k">fi</span>
<span class="o">[</span> -z <span class="s2">"</span><span class="nv">$DIR1</span><span class="s2">"</span> <span class="o">]</span> <span class="o">&&</span> <span class="nv">DIR1</span><span class="o">=</span><span class="s2">"</span><span class="nv">$DIR</span><span class="s2">"</span> <span class="c1">#The first valid dir</span>
<span class="o">[</span> -d <span class="s2">"/var/</span><span class="nv">$DIR</span><span class="s2">"</span> <span class="o">]</span> <span class="o">&&</span> <span class="nv">DTMP</span><span class="o">=</span><span class="s2">"</span><span class="nv">$DTMP</span><span class="s2"> </span><span class="nv">$DIR</span><span class="s2">"</span>
<span class="k">done</span>
<span class="nv">DIRS</span><span class="o">=</span><span class="s2">"</span><span class="nv">$DTMP</span><span class="s2">"</span>
<span class="k">if</span> <span class="o">[</span> -z <span class="s2">"</span><span class="nv">$DIRS</span><span class="s2">"</span> <span class="o">]</span><span class="p">;</span> <span class="k">then</span>
<span class="nb">echo</span> <span class="s2">"No valid directory in DIRS"</span>
<span class="nb">exit</span> <span class="m">1</span>
<span class="k">fi</span>
<span class="c1">#We go to /var</span>
<span class="nb">cd</span> /var
mkdir -p /var/local
<span class="o">[</span> -e /etc/var_folders_keep_files <span class="o">]</span> <span class="o">&&</span> <span class="nv">KEEPFILES</span><span class="o">=</span><span class="s2">"y"</span>
<span class="nb">true</span>
<span class="o">}</span>
update_archive<span class="o">()</span> <span class="o">{</span>
<span class="k">if</span> <span class="o">[</span> <span class="s2">"</span><span class="nv">$KEEPFILES</span><span class="s2">"</span> <span class="o">==</span> <span class="s2">"y"</span> <span class="o">]</span><span class="p">;</span> <span class="k">then</span>
update_archive_keepfiles
<span class="k">return</span> <span class="nv">$?</span>
<span class="k">fi</span>
update_archive_normal
<span class="k">return</span> <span class="nv">$?</span>
<span class="o">}</span>
update_archive_keepfiles<span class="o">()</span> <span class="o">{</span>
<span class="c1">#Create tar with all files</span>
tar cf /var/local/<span class="s2">"</span><span class="nv">$ARFILE</span><span class="s2">"</span> <span class="nv">$DIRS</span>
chmod <span class="m">640</span> /var/local/<span class="s2">"</span><span class="nv">$ARFILE</span><span class="s2">"</span>
find <span class="nv">$DIRS</span> -type d -o -type l > /var/local/<span class="s2">"</span><span class="nv">$ARFILE</span><span class="s2">"</span>.list
<span class="o">}</span>
update_archive_normal<span class="o">()</span> <span class="o">{</span>
<span class="nv">TMP</span><span class="o">=</span><span class="sb">`</span>mktemp<span class="sb">`</span>
find <span class="nv">$DIRS</span> -type d -o -type l > <span class="s2">"</span><span class="nv">$TMP</span><span class="s2">"</span>
<span class="nv">SUMNEW</span><span class="o">=</span><span class="sb">`</span>cat <span class="s2">"</span><span class="nv">$TMP</span><span class="s2">"</span> <span class="p">|</span> sha512sum<span class="sb">`</span>
<span class="nv">SUMOLD</span><span class="o">=</span>
<span class="o">[</span> -f /var/local/<span class="s2">"</span><span class="nv">$ARFILE</span><span class="s2">"</span>.list <span class="o">]</span> <span class="o">&&</span> <span class="nv">SUMOLD</span><span class="o">=</span><span class="sb">`</span>cat /var/local/<span class="s2">"</span><span class="nv">$ARFILE</span><span class="s2">"</span>.list <span class="p">|</span> sha512sum<span class="sb">`</span>
<span class="k">if</span> <span class="o">[[</span> <span class="s2">"</span><span class="nv">$SUMOLD</span><span class="s2">"</span> <span class="o">==</span> <span class="s2">"</span><span class="nv">$SUMNEW</span><span class="s2">"</span> <span class="o">]]</span><span class="p">;</span> <span class="k">then</span>
rm <span class="s2">"</span><span class="nv">$TMP</span><span class="s2">"</span>
<span class="k">return</span>
<span class="k">fi</span>
mv <span class="s2">"</span><span class="nv">$TMP</span><span class="s2">"</span> /var/local/<span class="s2">"</span><span class="nv">$ARFILE</span><span class="s2">"</span>.list
tar cf /var/local/<span class="s2">"</span><span class="nv">$ARFILE</span><span class="s2">"</span> --no-recursion <span class="sb">`</span>cat /var/local/<span class="s2">"</span><span class="nv">$ARFILE</span><span class="s2">"</span>.list<span class="sb">`</span>
chmod <span class="m">640</span> /var/local/<span class="s2">"</span><span class="nv">$ARFILE</span><span class="s2">"</span>
<span class="o">}</span>
force_update<span class="o">()</span> <span class="o">{</span>
<span class="c1">#removing the list file will force the update if stop is called</span>
rm -f <span class="s2">"/var/local/</span><span class="si">${</span><span class="nv">ARFILE</span><span class="si">}</span><span class="s2">.list"</span>
<span class="o">}</span>
extract_archive<span class="o">()</span> <span class="o">{</span>
<span class="o">[</span> ! -f /var/local/<span class="s2">"</span><span class="nv">$ARFILE</span><span class="s2">"</span> <span class="o">]</span> <span class="o">&&</span> <span class="k">return</span> <span class="m">1</span>
<span class="c1">#Archive has already been extracted:</span>
<span class="o">[</span> -f /var/<span class="si">${</span><span class="nv">DIR1</span><span class="si">}</span>/archive_extracted <span class="o">]</span> <span class="o">&&</span> <span class="k">return</span> <span class="m">0</span>
<span class="c1">#Extract:</span>
tar xf /var/local/<span class="s2">"</span><span class="nv">$ARFILE</span><span class="s2">"</span>
touch /var/<span class="si">${</span><span class="nv">DIR1</span><span class="si">}</span>/archive_extracted
<span class="o">}</span>
<span class="c1">##################################################################################</span>
<span class="k">case</span> <span class="nv">$1</span> <span class="k">in</span>
start<span class="o">)</span>
log_begin_msg <span class="s2">"Starting </span><span class="nv">$PROG</span><span class="s2">"</span>
<span class="k">if</span> ! initialise<span class="p">;</span> <span class="k">then</span>
log_end_msg <span class="m">1</span>
<span class="nb">exit</span> <span class="m">1</span>
<span class="k">fi</span>
<span class="k">if</span> ! extract_archive<span class="p">;</span> <span class="k">then</span>
log_end_msg <span class="m">1</span>
<span class="nb">exit</span> <span class="m">1</span>
<span class="k">fi</span>
log_end_msg <span class="m">0</span>
<span class="p">;;</span>
stop<span class="o">)</span>
log_begin_msg <span class="s2">"Stopping </span><span class="nv">$PROG</span><span class="s2">"</span>
<span class="k">if</span> ! initialise<span class="p">;</span> <span class="k">then</span>
log_end_msg <span class="m">1</span>
<span class="nb">exit</span> <span class="m">1</span>
<span class="k">fi</span>
<span class="k">if</span> ! update_archive<span class="p">;</span> <span class="k">then</span>
log_end_msg <span class="m">1</span>
<span class="nb">exit</span> <span class="m">1</span>
<span class="k">fi</span>
log_end_msg <span class="m">0</span>
<span class="p">;;</span>
force-update<span class="o">)</span>
initialise <span class="o">||</span> <span class="nb">exit</span> <span class="m">1</span>
force_update
<span class="nb">exit</span> <span class="nv">$RET</span>
<span class="p">;;</span>
restart<span class="p">|</span>force-reload<span class="p">|</span>status<span class="o">)</span>
<span class="nb">exit</span> <span class="m">0</span>
<span class="p">;;</span>
*<span class="o">)</span>
<span class="nb">echo</span> <span class="s2">"Usage: </span><span class="nv">$0</span><span class="s2"> {start|stop|restart|force-reload|force-update|status}"</span> ><span class="p">&</span><span class="m">2</span>
<span class="nb">exit</span> <span class="m">1</span>
<span class="p">;;</span>
<span class="k">esac</span>
<span class="nb">exit</span> <span class="m">0</span>
</code></pre></div>Master Thesis: Tunnelling in water clusters2013-08-10T00:00:00+02:002013-08-10T00:00:00+02:00Michael F. Herbsttag:michael-herbst.com,2013-08-10:/master-thesis-tunnelling-in-water-clusters.html<p>In my first post I'll briefly talk about my first proper piece of
research, my <em>Master thesis</em>. As part of my last year of Cambridge I
spent 6 months doing a project in the theoretical chemistry <a href="http://www-stuart.ch.cam.ac.uk/" title="Stuart Althorpe group">group of
Prof. Dr. Stuart
Althorpe</a>.
Although I've done a few smaller research projects …</p><p>In my first post I'll briefly talk about my first proper piece of
research, my <em>Master thesis</em>. As part of my last year of Cambridge I
spent 6 months doing a project in the theoretical chemistry <a href="http://www-stuart.ch.cam.ac.uk/" title="Stuart Althorpe group">group of
Prof. Dr. Stuart
Althorpe</a>.
Although I've done a few smaller research projects in previous summers,
it is still quite different if one works on a topic for 6 months rather
than just a few weeks. At the beginning it is much less clear what will
be achieved and throughout it is quite difficult to keep everything
ordered in your mind. From the little research I've done up to now, I'd
say that staying on top of what you have tried and what has actually
worked is quite crucial in order to not get lost in the ocean of
possibilities. Most of the time what helped me was to go and talk to
other members of the group about my work, trying to get ideas of the
conversations. Luckily my group was very open about that and within just
a few days I found myself taking part in passionate discussions.</p>
<p>What fascinated me most about the topic of <a href="https://en.wikipedia.org/wiki/Quantum_tunnelling" title="wiki quantum tunnelling">quantum
tunnelling</a>
were the mathematical and physical challenges. Firstly, we never really
talked about tunnelling in lectures in a quantitative way, meaning that
we never considered ways to actually calculate the resulting splittings
in the spectra. So this project offered a good chance to gain a more
thorough understanding of this quantum mechanical phenomenon. Next, the
method to calculate the splittings uses the so-called <a href="https://en.wikipedia.org/wiki/Feynman_path_integral" title="wiki Feynman">Feynman path
integral</a>
description of Quantum Mechanics and ever since my Physics friends
talked about this over lunch, I was curious to find out more about it.
This thesis provided an excellent opportunity to do so.</p>
<p>I will probably write a little bit more about ring-polymer instanton
theory in a later post, but for now I will just provide the thesis
itself. My first draft massively exceeded the allowed word limit, so I
had to remove some details -- mostly from the theory section. This
shorter version is the version I handed in and got my degree for. It can
be found by clicking the link "Dissertation" in the table
below. A proof-read extended version including
everything from the first draft is given below as well.</p>
<table>
<thead>
<tr>
<th>Link</th>
</tr>
</thead>
<tbody>
<tr>
<td><a href="https://michael-herbst.com/publications/2013.04.28_dissertation.pdf">Dissertation</a></td>
</tr>
<tr>
<td><a href="https://michael-herbst.com/publications/2013.04.28_dissertation_extended.pdf">Dissertation (extended version)</a></td>
</tr>
</tbody>
</table>