OP2Utility Serializer Help

[Vagabond]

Thanks to Leeor's help fixing my issue with multiple inheritance, I was continuing work on adding a Serializer interface to OP2Utility.

Unfortunately I've hit a roadblock and am not sure how to proceed.

Below I have the prototype of the Serializer interface. Notice that Serialize takes a const char* buffer as an argument.

Code: [Select]

class Serializer {
public:
	virtual ~Serializer() = 0 {};
	virtual void Serialize(const char* buffer, size_t size) = 0;
};

This works well for Writing streams, as it allows passing both const and non-const sources of data for writing.

Code: [Select]

void MemoryStreamWriter::Serialize(const char* buffer, size_t size)
{
	if (offset + size > streamSize) {
		throw std::runtime_error("Size of bytes to write exceeds remaining size of buffer.");
	}

	memcpy(streamBuffer + offset, buffer, size);
	offset += size;
}

However, this does not work for the opposite case of reading from memory into a buffer (StreamReader). Since the buffer is const to comply with the definition of Serializer::Serialize, nothing can be read into the buffer.

Code: [Select]

void MemoryStreamReader::Serialize(const char* buffer, size_t size) 
{
	if (offset + size > streamSize) {
		throw std::runtime_error("Size of bytes to read exceeds remaining size of buffer.");
	}

	memcpy(buffer, streamBuffer + offset, size);
	offset += size;
}

One solution would be to remove the const qualifier from Serialize(const char* buffer, size_t size). This would allow for both reading and writing. The problem here is we can no longer write from a const source.

We could include a function overload of Serialize that only exists within StreamReader that has a const buffer. Then reading from const sources would be allowed in StreamReader. Both StreamReader and StreamWriter would comply with the Serialize interface

However, if you were reading from a source by referencing the Serialize interface, the class would not recognize the existence of the Serialize function overload containing the const buffer. This could be overcome by casting a Serializer as a StreamReader (assuming C++ supports this sort of casting like C# does?).

However, this makes me question the usefulness of having a base Serializer interface if we are casting it to read const data. This wrinkle would be in addition to checking if the Serializer is a StreamReader or StreamWriter when the size of a container needs to be initialized for writing. Between these two special case situations I'm questioning the usefulness of a Serializer interface.

Perhaps someone has a better solution for the Serialize function?

If not but people still see incorporating the Serialize interface as a net win, I will continue working on it.

Thoughts?

Thanks,
Brett

[leeor_net]

If you're reading from a serialized stream, you're not serializing it, you're deserializing it so you should probably have a deserialize interface function that takes a non-const pointer to a char* buffer (or preferably a void pointer buffer).

Another option is to const_cast<char*>(buffer) which defeats the purpose of the const keyword and also violates the documentation purpose of using a const parameter to begin with (though it would work).

[Hooman]

You're nicely hitting many of the same difficulties I ran into when I first toyed with this idea. I also had my doubts at times.

One solution would be to remove the const qualifier from Serialize(const char* buffer, size_t size). This would allow for both reading and writing. The problem here is we can no longer write from a const source.

Correct. According to this idea, Serialize must be an abstraction of both reading and writing, and so the parameter must not be const so as to support reading into a necessarily non-const buffer.

To support writing const objects, the Writer derived interfaces may provide an overload of the Serialize function with a const buffer parameter. One caveat is not being able to override the base class function when making the parameter const, since the function signature has changed. However, one method could delegate to the other method, and the delegation could be done as an inline method in the header file to avoid the overhead of an extra function call.

Here was roughly what I tried, using whisper notation. I used "operator()" rather than a "Serialize" function, and I'm returning a reference to the class to allow for easy call chaining.

Code: [Select]

WriterStream& operator()(void* buffer, std::size_t length);
// Provide helper methods to write const objects (writing them to the stream does not modify them)
inline WriterStream& operator()(const void* buffer, std::size_t length) {
	return operator()(const_cast<void*>(buffer), length);
}

One downside of the above, is the actual write code won't be type checked by the compiler to ensure it doesn't modify the buffer. You could potentially swap implementations to allow for that. I'm not sure how smart the compiler is at optimizing chained calls from a virtual function to a non-virtual function, but you might still be able to eliminate the extra call overhead if things were swapped.


Unrelated, but thinking ahead, the next big stumbling block I encountered was supporting slices of existing streams. I believe the Curiously Recurring Template Pattern (CRTP) may be helpful there. I believe I alluded to this pattern when I brought up mixins a few days ago. Warning though, it feels a bit convoluted, with rather twisted looking logic.