Using optional lead to issues probably because the assignment operator were not correcty implemented. However there is not need for optional here since the member is in fact not optional but always there when the object is validly constructed. We can simply use a unique_ptr to work around the fact that not all geometry imeplementation are default-constructible and/or copy-assignable.
Fixes #930 (closed).
Should be also backported to release 3.2 since the bug already occurred there but hasn't been detected by the CI.