Merge branch 'feature/handbook_scripts' into 'master'

Handbook section on scripts

Closes #1019

See merge request !2587

doc/handbook/0_dumux-handbook.tex

@@ -154,6 +154,7 @@ concepts (\ref{sc_linearsystem}).
 \input{5_restartsimulations}
 \input{5_developingdumux}
 \input{5_externaltools}
+\input{5_scripts}
 \input{5_assemblinglinearsystem}
 
 \chapter{Advanced \Dumux\ -- Detailed Instructions}

doc/handbook/5_externaltools.tex

@@ -59,20 +59,12 @@ of \texttt{Dumux::GnuplotInterface}, or the header file itself (\texttt{dumux/io
 
 \subsection{Gstat}
 Gstat is an open source software tool which generates geostatistical random fields (see \url{www.gstat.org}).
-In order to use gstat, execute the \texttt{bin/installexternal.sh} from your \Dumux root
+In order to use gstat, execute the \texttt{bin/installexternal.py} from your \Dumux root
 directory or donwload, unpack and install the tarball from the gstat-website.
 Then, rerun cmake (in the second case set \texttt{GSTAT\_ROOT} in your input file to the
 path where gstat is installed).
 
 
-\subsection{ParaView}
-\paragraph{Reload Button:}
-There are scripts to reload PVD or series of VTU files since ParaView 4.2.
-The scripts can be found
-\href{http://markmail.org/message/exxynsgishbvtngg#query:+page:1+mid:rxlwxs7uqrfgibyv+state:results}{\texttt{under this link}}.
-Just save the specific code portion in a file and load it via \texttt{Macros} $\rightarrow$ \texttt{Add new macro}.
-
-\paragraph{Guide:}
-Since ParaView 4.3.1, The ParaView Guide is partly
-available for free download, see \url{http://www.paraview.org/documentation/}.
-It contains similar content as the ParaView book.
+\subsection{ParaView} \label{ssec:paraview}
+To visualize the simulation data you have produced using \Dumux, we recommend using \href{https://www.paraview.org/}{Paraview}. 
+This open-source software supports \Dumux~'s standard data formats, and can be operated either with a GUI or with batching tools. 
\ No newline at end of file

doc/handbook/5_scripts.tex

+\section{Scripts}
+\label{sc_scripts}
+
+A suite of scripts is available within the dumux repository to assist in the performance of common tasks. These scripts can be found in the \texttt{dumux/bin/} directory. 
+The majority of these scripts began as bash (*.sh) scripts, but most have been replaced with more portable and user-friendly Python versions. 
+Some of these scripts are used in the installation or the creation of working dumux-dependent repositorities; these are outlined in section \ref{ssec:repoDevScripts}.
+Others can be used to perform post-processing tasks after simulation data has been produced; these are outlined in section \ref{ssec:postprocessingScripts}. In order to maintain the suite of unit tests and examples within this repository, some testing scripts are also required; these are outlined in section \ref{ssec:testingScripts}. All remaining tests are mentioned in section \ref{ssec:miscScripts}.
+
+\subsection{Repository development Scripts:} \label{ssec:repoDevScripts}
+
+\begin{itemize}
+  \item Installation Scripts: \texttt{installdumux.py} and \texttt{installexternal.py} both have been developed to make the installation of the base Dumux suite and any external dependencies as simple as possible. 
+  \item Module Development: When developing within the dumux simulation environment, it is recommended to do so within your own module. When sharing this module with others, or installing it on multiple machines, it is often difficult to find exactly which version you are using so that your work will run in different places. To simplify this, the following scripts are available in the \texttt{util/} directory: 
+  	\begin{itemize}
+  		\item The script \texttt{createdockerimage.py} can help to create a docker image to put your work in a container for simple delivery.
+		\item In order to extract the content and dependencies within your model, \texttt{extractmodulepart.py} will collect all headers used to run a specified test and collect them all in a functioning external module.
+		\item To document the exact versions that have been used, the scripts \texttt{getusedversions.py} is available to search each through an existing simulation dumux suite and find the branch, commit hash, and commit date of the current implemention. 
+		\item The script \texttt{create\_cmakelists.py} can be used to integrate new header files to the dune/dumux cmake system.
+		\item In order to produce an installscript that will install your repository and its dependencies on another computer, the \texttt{makeinstallscript.py} is available. This script will collect all of the dependencies and their versions using the \texttt{getmoduleinfo.py} and the other scripts listed above and write a script to install a new version in the same configuration. In the past it has been recommended to run the script \texttt{test\_dumux.sh} to ensure that the final dunecontrol and compilation pass have worked. 
+  	\end{itemize} 
+\end{itemize}
+
+\subsection{Post-processing Scripts:} \label{ssec:postprocessingScripts}
+Data visualization and post processing is an integral part of any exercise in simulation. A few scripts are available to begin with in the \texttt{postprocessing/} subdirectory. 
+\begin{itemize}
+	\item Paraview data collection scripts: Although there are many tools available within the open-source visualization program paraview (see \ref{ssec:paraview}), performing consistent data collection from your simulations can be tedious when analyzing many output files. The scripts \texttt{exportscreenshot2d.py}, \texttt{extractlinedata.py}, and \texttt{extractpointdataovertime.py} are each scripts that can help to perform paraview data collection commands consistently and each produce easy to use \texttt{.csv} files or images. 
+	\item Error convergence script: When evaluting errors to a reference solution across grid refinements, the L2 error is typically evaluated. A script to help perform this is available in \texttt{l2error.py}.
+\end{itemize}
+
+\subsection{Testing Scripts:} \label{ssec:testingScripts}
+In order to ensure consistency when developing in dumux, developers are encouraged to introduce unit tests and reference tests that can be run to ensure new code introductions do not break existing functions. While existing simulations should each test something when the \texttt{ctest} command is called, a few scripts exist in the \texttt{testing/} subdirectory to make this simple to develop and quick to evaluate.
+\begin{itemize}
+	\item Run tests: In order to automatically find and run tests that fit a specific critera, the scripts \texttt{findtests.py}, \texttt{runtest.py} and \texttt{runselectedtests.py} each are available. These are each used to automate the testing process.  
+	\item Reference Solution Comparisons: The scripts \texttt{fuzzycomparevtu.py} and \texttt{fuzzycomparedata.py} are both used to compare simulation results against reference simulation results. When running a test, one can use these files to compare the output from the test against reference simulation results. Should these results differ, the difference for each field is shown. These scripts are typically directly called during testing. 
+\end{itemize}
+
+\subsection{Miscellaneous:} \label{ssec:miscScripts}
+In addition, a few miscellaneous scripts are available.
+\begin{itemize}
+	\item Parameter scripts: The scripts \texttt{doc/compareparameters.sh} and \texttt{doc/getparameterlist.py} are both available for tracking all of the runtime parameters used. 
+	\item Other: \texttt{doc/getcontributors.sh} can be used to find the names and contact information for all commit authors over a given time period. 
+\end{itemize}
\ No newline at end of file