dumux-repositories issueshttps://git.iws.uni-stuttgart.de/groups/dumux-repositories/-/issues2019-10-04T11:07:26Zhttps://git.iws.uni-stuttgart.de/dumux-repositories/dumux/-/issues/760Tests for flashs don't test anything2019-10-04T11:07:26ZBeatrix BeckerTests for flashs don't test anythingThey never fail, just print an error.They never fail, just print an error.3.1https://git.iws.uni-stuttgart.de/dumux-repositories/dumux/-/issues/753[tests] Newon line search not tested in test suite2019-11-08T09:42:13ZTimo Kochtimokoch@math.uio.no[tests] Newon line search not tested in test suiteWe could just enable if for some test, ideally where it also improves convergence.We could just enable if for some test, ideally where it also improves convergence.3.2Roman WinterRoman Winterhttps://git.iws.uni-stuttgart.de/dumux-repositories/dumux/-/issues/481test solidsystems and solid components2018-05-24T13:48:09ZTimo Kochtimokoch@math.uio.notest solidsystems and solid componentsOnce we have solidsystem (!903) we should also have a solid system unit test.Once we have solidsystem (!903) we should also have a solid system unit test.3.0https://git.iws.uni-stuttgart.de/dumux-repositories/dumux/-/issues/646Tests: Replace problem functions postTimeStep/preTimeStep2019-02-27T13:52:59ZTimo Kochtimokoch@math.uio.noTests: Replace problem functions postTimeStep/preTimeStepFunctions with such names are due to the version 2 way of implementing simulation flow control. In version 3 such functions should not exist because
* the function name doesn't say anything about what the function does
* the functionali...Functions with such names are due to the version 2 way of implementing simulation flow control. In version 3 such functions should not exist because
* the function name doesn't say anything about what the function does
* the functionality in those functions probably doesn't belong in the problem
The function might be removed by moving the functionality into the main file, or renamed to some appropriate name if the functionality really belongs to the problem (connected to BC/IC/Sources or maybe an analytical solution).3.1https://git.iws.uni-stuttgart.de/dumux-repositories/dumux/-/issues/682Test tabulation doesn't actually test anything2019-12-20T12:43:35ZTimo Kochtimokoch@math.uio.noTest tabulation doesn't actually test anythingNo matter if success is false or true the test always returns 0 exit code.No matter if success is false or true the test always returns 0 exit code.3.2Timo Kochtimokoch@math.uio.noTimo Kochtimokoch@math.uio.nohttps://git.iws.uni-stuttgart.de/dumux-repositories/dumux/-/issues/813[test] Unit test for VanGenuchten / Brooks-Corey material laws missing2020-03-31T09:31:36ZTimo Kochtimokoch@math.uio.no[test] Unit test for VanGenuchten / Brooks-Corey material laws missingWe should test (for raw and regularized version):
* if derivatives match numerical derivatives
* unregularized same as regularized in the non-regularized range
* endPointPc() is the same as evaluation at Sw=1
* entryPressure is the same...We should test (for raw and regularized version):
* if derivatives match numerical derivatives
* unregularized same as regularized in the non-regularized range
* endPointPc() is the same as evaluation at Sw=1
* entryPressure is the same as evaluation at Sw=1
* test against some precomputed reference values (a good regression test)
* test eff to abs law
add more invariants that could be unit tested if you have an idea.3.2https://git.iws.uni-stuttgart.de/dumux-repositories/dumux/-/issues/751[test] Unit test intersectspointgeometry incomplete2020-04-04T18:09:54ZTimo Kochtimokoch@math.uio.no[test] Unit test intersectspointgeometry incompletepoint-tetrahedron and point-pyramid is not tested
Maybe a dedicated unit test for this header would be good and easy to implement.point-tetrahedron and point-pyramid is not tested
Maybe a dedicated unit test for this header would be good and easy to implement.3.2Tim JupeTim Jupehttps://git.iws.uni-stuttgart.de/dumux-repositories/dumux/-/issues/373Tricky tests2018-07-20T14:57:08ZBeatrix BeckerTricky testsSome of our tests fail depending e.g. on the machine or the solvers that are used. For those tests, small changes seem to make a large difference. This could be a bug in a test, could also be a very sensitive system. For the latter more ...Some of our tests fail depending e.g. on the machine or the solvers that are used. For those tests, small changes seem to make a large difference. This could be a bug in a test, could also be a very sensitive system. For the latter more stable tests need to be defined. A list of the tests that failed on my computer (but did not fail on the buildbot) is below. Please add additional tests that seem to show such a behavior in the comments.
* test_zeroeq (works with SuperLU, not with Umfpack)
* test_el1p2c
* test_boxadaptive2p (works with SuperLU, not with Umfpack)
* test_2cstokes2p2c
* test_2cnistokes2p2cni
* test_2cnistokes2p2cni_boundarylayer
* test_2czeroeq2p2c
* test_2cnizeroeq2p2cni
* test_forchheimer2p
* test_boxmpnckinetic
* test_cc2pncmin
* test_stokes
* lens2pexercise3
* co2plumeshapeexercise
* fuelcell
* naplinfiltration3p3chttps://git.iws.uni-stuttgart.de/dumux-repositories/dumux/-/issues/739Unit test for compositional flash2019-10-04T09:42:27ZBeatrix BeckerUnit test for compositional flashThere is none currentlyThere is none currently3.1https://git.iws.uni-stuttgart.de/dumux-repositories/dumux/-/issues/968Use gitlab ci for automated testing2021-04-29T09:49:22ZDennis GläserUse gitlab ci for automated testingIt could be useful to integrate build and test jobs into our ci such that one can conveniently let the runner test everything in a merge request. That makes it less probable that stuff gets merged that breaks anything.
I would suggest t...It could be useful to integrate build and test jobs into our ci such that one can conveniently let the runner test everything in a merge request. That makes it less probable that stuff gets merged that breaks anything.
I would suggest to create pipelines for commits to master, tags and merge requests. Regarding automatic start of the pipelines, I would suggest the following strategy:
- add manual trigger for build & test pipeline for merge requests. Especially on Dumux days, when there is a lot of pushing to open MRs, we would trigger a lot of unneccesary pipeline runs, I guess. I would prefer a manual trigger here.
- automatic start of the pipeline for master and tags. The first is useful as you always see which commit destroyed the pipeline on master. This is of course a bit redundant as one maybe already triggered and verified a successful pipeline on the branch just before merging. In some cases, it could still break on master if someone didn't rebase before running the pipeline. For safety reasons, I think it could be nice to always test master.
I am thinking of some default setup (Dune2.7, release opts) to be used, but we could schedule one nightly build with different setups as on buildbot. However, that is basically currently done on buildbot, so this would only make sense if we want to stop using buildbot at one point. So, this is probably irrelevant for now.
Another point is that our build and test times are quite long, and it would be nice to build and run only those tests that are necessary, at least for MRs. Here's a first idea for merge requests:
- 1 use `git diff-tree -r --name-only $CI_COMMIT_SHA $CI_MERGE_REQUEST_TARGET_BRANCH_SHA` to list the modified files
- 2 use `ctest -N` to list all tests we currently have
- 3 do something like in `extractmodulepart.sh` to determine required headers of the tests
- 4 select those tests which use any of the files obtained in (1)
- 5 compile and run those tests only
point (2) and (3) might be expensive. We could also store that info in a json file or so, but on the other hand, that file would need to be up-to-date, and maybe someone adds a test in the MR thats not in the file. Or, maybe someone adds a dependency with the MR. For now, I'd opt for always doing that, should be much less expensive than building and running a lot of tests.
_Docker images_
Dune's Docker images for testing are built here: https://gitlab.dune-project.org/docker/ci
It would be nice to reuse some of the base images they use there. Also I propose a `dumux-docker-ci` project that
* automatically builds the dumux test images in GitLabCI (services images from Dune, e.g. `buildah` can be hopefully reused for that purpose)
* pushes the images to the `dumux-docker-ci` container registry (see gitlab-ci.yaml in the dune project)
* reuse base images from Dune if possible
_BuildBot_
Our current images are in https://git.iws.uni-stuttgart.de/timok/dumux-bb-ci
and our current buildbot setup is here: https://git.iws.uni-stuttgart.de/timok/buildbot-master
_Multi module setup_
After new commits in dumux we also want to trigger builds of dumux-lecture (maybe dumux-course). There is also (a bit outdated) dumux-testing with the first system test for dumux. See !2396.
_Coverage reports_
The coverage report is generated here in a weekly build. It already uses GitlabCI.
https://git.iws.uni-stuttgart.de/dumux-repositories/dumux-coverage3.5Dennis GläserDennis Gläser