Is there a way to add custom per-element parsers to std::vectors?

[joshuamaiche]

There are a number of situations where parsing a vector can't be done by inserting a blank object then parsing the object:

• The object could have immutable members.
• The object could need a special constructor before being parsed.
• The parsing could be complex, needing to parse one part of the data, then using that to construct and parse the object.

These situations all seem bespoke, and best handled by the user. For example, here's a simple struct:

struct ImmutableStruct
{
    const int val1;
    const float val2;
};

struct MyStruct
{
    std::vector<ImmutableStruct> vals;
};

And here's a wrapper that can insert into a vector of ImmutableStructs:

struct ImmutableStructInserter_t
{
    std::vector<ImmutableStruct>& container;
    int parsedInt;
    float parsedFloat;
};

template <auto Member>
inline constexpr decltype(auto) ImmutableStructInserter_impl()
{
    return [](auto&& v) { return ImmutableStructInserter_t{ v.*Member }; };
}

template <auto Member>
constexpr auto ImmutableStructInserter = ImmutableStructInserter_impl<Member>();

However, I didn't see a great "batteries included" way in Glaze to run this wrapper on every element. I ended up making my own, with a few requirements:

• I wanted something that would automatically work with multiple formats, so no json-specific code.
• I wanted to hook into existing Glaze mechanisms instead of reinventing the wheel.
• I wanted to use the template compile-time code instead of the generic runtime code like glz::json_t.

What I ended up was basically a type that looks like a vector, but is actually just a passthrough to an adapter, that does whatever it wants on a per-element basis:

template <typename Adapter>
struct per_element_adapter_t
{
    struct dummy_iterator
    {
        using difference_type = std::ptrdiff_t;
        using value_type = Adapter;

        Adapter& operator*() const {
            return *adapter;
        }

        bool operator != (const dummy_iterator& rhs) const {
            return position != rhs.position;
        }

        dummy_iterator& operator++() { ++position; return *this; }
        dummy_iterator operator++(int) { ++position; return dummy_iterator{ adapter, position - 1 }; }
        Adapter* adapter;
        size_t position;
    };

    using reference = Adapter&;

    void clear() {};
    void resize(size_t newSize) { m_size = newSize; }
    reference emplace_back() { return adapter; }
    size_t size() { return m_size; }

    dummy_iterator begin() { return dummy_iterator{ &adapter, 0 }; }
    dummy_iterator end() { return dummy_iterator{ &adapter, m_size }; }

    Adapter adapter;
    size_t m_size;
};

template <auto Adapter>
inline constexpr decltype(auto) per_element_adapter_impl()
{
    return [](auto&& v) { return per_element_adapter_t{ Adapter(v) }; };
}

template <auto Adapter>
constexpr auto per_element_adapter = per_element_adapter_impl<Adapter>();

Usage is pretty much what you would expect:

template <>
struct glz::meta<MyStruct>
{
    using T = MyStruct;
    static constexpr auto value = object(
        "vals", glz::custom<
            per_element_adapter<ImmutableStructInserter<&T::vals>>,
            &T::vals
        >
    );
};

Here's a godbolt link showing the full example: https://godbolt.org/z/ahbzbKKMf

I still have an issue with this implementation--it depends on just passing whatever concepts that indicate that this should be deserialized like a vector. When Glaze bumped up to V5, for instance, I had to modify the code to comply with Glaze's internals. This leads to my question.

Is there a format-agnostic way to pass a wrapper to every element being parsed in an array? If not, could something like per_element_adapter be added to Glaze, so I don't have to maintain it on my side? I don't know what typical Glaze usage is, but I would imagine other users would also benefit from controlling how elements are inserted into their array-like collections.

[stephenberry]

Thanks for sharing this solution! I really like this approach and would be glad to add it as a tool for Glaze. Feel free to add a pull request, or I'll get to it when I'm able. It will just need some unit tests and documentation.

[joshuamaiche]

I'll try to set up a PR, but you'll probably have to get it through the finish line; something might come up in a week or so that will get in the way of me coding.

[stephenberry]

No problem, I can run with whatever you want to put together. And, no pressure to add the PR, it just might speed up the process.

[joshuamaiche]

Maybe I'll let you do it then, thanks. I'm in no rush--I'm just happy I don't have to maintain the solution on my side!