variant flattening is broken for variants with different buffer sizes and no type restrictions

[Inkrementator]

Intro

How does variant handle state

Instances Variant have a buffer (which either contains the payload if it fits, or a pointer to it on the heap) and an function ptr to a handler function. This handler function is templatized on the current payload. This means that the variant struct actually has no knowledge of the type, only the handler function.

The Variant type itself is templatized on the size of the internal buffer and a restriction of allowed types. The handler function is inside the Variant struct, so has access to these symbols.

The problem

If the VariantN you want to wrap has a different buffer size OR if it doesn't provide a restriction of allowed types, this flattening will fail. See the following example.

import std.stdio;
import std.variant;
import std.exception;

void main(){
  auto a = Variant(Variant(4));
  a.get!int; // this works

  alias Restricted = VariantN!(16, int, string);
  auto b = Restricted(4);
  auto c = Variant(b);
  c.get!int.writeln; // also works

  alias V16 = VariantN!16;
  alias V8 = VariantN!8;

  auto d = V16(V8(5));
  d.type.writeln; // VariantN!8

  d.get!int.assertThrown;
  d.get!V8.get!int.writeln;
}

Solutions

Naive ways to try to solve this don't work, because there is no way to get the type wrapped by a variant at compile time, since that type is only inside of a function that we only have the pointer to.

You can get the size of the wrapped type at runtime, but currently we need the type at compile time to instantiate the handler template with the correct type.

Easier way

Make the wrapped VariantN types an implementation detail and collapse the chain every time we access the value. This would probably need Variant to store another parameter (whether a Variant is stored inside of it).

Now every time get, opIndex or any function is called, these functions would first recursively unpack every Variant inside of it to finally get the value out.

It would be easy to miss something here, so an extensive test suite would be necessary.

Hard way

After writing it down, I realized this would be way to much effort without breaking changes to the API, which is not worth it for this problem. I still included my thoughts for completeness

Move `handler` from outside of the `VariantN` template to module scope. Also, do not templatize this function on the type of the payload anymore. Instead, pass in the size of the VariantN.storage array and typeinfo of the payload (which VariantN now would have to store) into the handler function as runtime argument.

Emplace and friends are extensively used to run postblits, but these can be accessed from the typeinfo.

There would probably be some trouble when trying to coerce types. To do this without changing the api, maybe it could be achieved by generating a few conversion functions at opAssign when we still know the type and index them by typeinfo.

Variant would probably also have to store the type qualifications so i.e. immutability is honored.

associative arrays and slices would have to be reimplemented from the runtime too. Yeah this is too hard

[Inkrementator]

And of course there is the trivial solution of just throwing in opAssign if flattening doesn't work. Breaking change but I would be astonished if anyone is actually chaining get out there to save a few bytes of stack space.