Converting Intergers to Vector in C++
I have been working on improving the storage footprint on disk for a system I am working on for the university.
The system is currently using JSON even for binary data, which was not a good design choice for multiple reasons.
I have been working on replacing JSON with pure binary data without the overhead of textual representation used in JSON.
One of my problems have been numbers, though luckily only natural numbers and mainly unsinged integers and I know that there are a lot of tools like BOOST that provides serialisation for this.
But, first I do not really like BOOST, no offence, and second other solutions have depended heavily on the usage of if-else chains, which is bad for predictive optimisation provided by -O2
and -O3
, so I didn’t really get why they used if-else chains.
So I decided to play a bit with templates and see if I could make a converter functions from type Type
which is a natural number to a std::vector<uint8_t>
and back, without using if-else chains.
Though this is not necessarily faster, it will be cleaner.
Now, I will take a bit of inspiration from other solutions and use left and right shifts and I wanna make the functions as simple to use as possible. Also for those who have not used C++ before, you can see templates as Generics.
To do the conversion, we need a natural number as input and it must be able to be any type of natural number in C++, so int
, long
, uint8_t
, and so on, and we need the output as a std::vector<uint8_t>
.
Now in C and C++ we have the function sizeof
which can give us the size of any type or variable in bytes.
Therefore we know that our output vector must have the size of our input in bytes.
Next we will make the assumption that all natural number types has a size such that size % 8 = 0
.
Then we can construct the function:
template<typename Type>
void convert_natural_number(const Type input, std::vector<uint8_t>& output)
{
out = std::vector<uint8_t>(sizeof(input);
for (size_t i = 0; i < sizeof(input); ++i)
{
out.at(i) = (output >> (i * 8));
}
}
A few comments on the function.
1) In the for-loop we have out.at(i) = (input >> (i * 8));
what this does is that it takes the input and right shift with n
bytes where is 8 times some offset i
where i
is depended on the size of our input in bytes.
This allows us to avoid having many if-statements which contains specialised shift statements for each length of an integer.
2) The function header: void convert_natural_number(const Type input, std::vector<uint8_t>& input)
might be weird for some that are used to use function return values.
What I do here is that I parse a reference to our output vector and use it as the output parameter of the function.
To “reverse” the conversion I have made a similar function, though with the output and input types swapped and moving from right shifts to left shifts:
template<typename Type>
void convert_natural_number(const std::vector<uint8_t>& input, Type& output)
{
output = 0;
for (size_t i = 0; i < input.size(); ++i)
{
Type next = input.at(i);
next = next << (i * 8);
output ^= next;
}
}
The only real comment I have to this function, is that you should always zero your output for such functions, just to be on the safe side.
A final remarks, I have only tested the function with natural numbers, but there should be no problem with floating points either or strings, but please test first.
If you have improvement ideas please write me, I am always ready to learn something new.
./Lars