Habemus C++20!

The first blog posts have started to appear and Herb Sutter already posted a trip report.

C++20 brings about a King-Kong-number of changes, and of course, the Big Kahuna:

  • Concepts
  • Ranges
  • Coroutines
  • Modules

You can find an (unofficial) summary of the features for C++20 on cppreference.com or your preferred website for C++ documentation.

En español

Los primeros blogs ya empezaron a aparecer y Herb Sutter ya postuló su informe de viaje.

C++20 trae consigo un número de cambios tamaño King Kong y, por supuesto, la Gran Kahuna:

  • Conceptos
  • Rangos
  • Corutinas
  • Módulos

Puedes encontrar un resumen (extraoficial) de las características de C++20 en es.cppreference.com (en español) o tu sitio web favorito para documentación de C++.

Posted in C++, C++20, software | Tagged , | Leave a comment

C++17 – La guía completa

Actualizado el 10 de febrero de 2020
Los siguientes son los enlaces en Amazon:
Amazon España
Amazon México
Amazon Estados Unidos
Amazon Canadá

Como un estudiante de ingeniería en mi amada alma mater, el Instituto Tecnológico de Chihuahua, recuerdo claramente durante una de nuestras clases de electrónica al Profesor Nevárez dándonos el material, disponible en la biblioteca de la escuela…en inglés. También recuerdo claramente esos libros amarillentos y gruesos, con un montón de dibujos de circuitos escritos en un lenguaje que, aunque podía leerlo, era difícil captar todo el contenido.

Caray, en ese tiempo hablaba poco inglés, habiéndolo aprendido a través de la secundaria, la preparatoria, mis viajes con mi abuela y mis tías durante el verano, traduciendo canciones, y cualquier otra manera que tenía a mi alcance, pero nunca intenté leer un libro técnico.

Sin embargo, tenía una ventaja en el sentido que el esfuerzo para mí de entender el material sería menor. Pero no tenía que haber sido de esa manera si hubiera habido material fácilmente disponible en español, pero no lo había–al menos no para esa clase en particular.

Sacando fruto de esa experiencia, me puse en contacto con Nico. Le había estado suministrando retroalimentación de su libro, C++17 – The Complete Guide, y le propuse la idea de traducirlo al español. Como le expliqué a Nico ese día, mi propósito principal era y todavía es traer un libro de C++ de alta calidad para estudiantes y profesionistas hispanohablantes.

Puedes ver (o mejor dicho, leer) el resultado de esa colaboración.

Trabajar con Nico has sido una experiencia instructiva. Es un profesional en toda la extensión de la palabra y siempre se conduce de una manera profesional. Además, entender la cantidad de trabajo que conlleva crear un libro, inclusive después que ha sido traducido, a un producto final, realmente me abrió los ojos. En una conversación casual que tuve con Rainer Grimm en CPPCON 2018, mencionó que después que Nico publicaba un libro, había poco material que no se cubría, o huecos que no se llenaran–o palabras similares. Viniendo de Rainer, un autor renombrado, es mucho que decir acerca de un colega. Habiendo visto el trabajo de Nico de cerca, sólo puedo agregarme a ese comentario.

Traducir un libro es una tarea desafiante. No sólo tienes que traducir las palabras, pero tienes que hacerlo tomando en cuenta el estilo y los términos técnicos, y con las diferentes variantes del español, asegurarte que no utilices términos que son específicos para un país–p. ej., México. Es por eso que tengo que dar gracias a José Daniel García. José Daniel también es un miembro del Comité de Normalización de C++ en España, y respondió pacientemente a mis preguntas en cuanto a uso de términos técnicos, sugerencias y un mejor flujo del material.

Creo firmemente que este libro llena un vacío en el material fácilmente disponible para C++ en español, y es mi esperanza que asista a otros al aprender este hermoso lenguaje de programación que he usado desde 1996 y durante años para programar sistemas para el mercado de seguridad pública, 9-1-1, para salvar vidas diariamente.

Si te interesa comprarlo o recomendarlo a tus amigos hispanohablantes, puedes encontrarlo en Leanpub, en Amazon, y su sitio web de soporte, con todos los ejemplos de código.

Y sí, se habla C++.

Posted in software | Tagged , , | 1 Comment

C++17 – La guía completa

Updated on February 10, 2020
The following are the links at Amazon:
Amazon Spain
Amazon Mexico
Amazon USA
Amazon Canada

As an engineering student at my beloved alma mater, the Instituto Tecnológico de Chihuahua, I remember clearly during one of our electronics classes Professor Nevárez giving us the material, available at the school library…in English. I also remember clearly those thick, yellowish books, with a bunch of circuit drawings written in a language that, while I could read, it was difficult to grasp all the meaning.

Alas, I spoke little English at the time, having learned it through my middle school, high school, my summer visits to grandma and aunts, translating songs, reading the occasional book, and whatever other means I had at my disposal, but never did I attempt to read a technical book.

Nonetheless, I had an advantage in the sense that the effort for me to understand the material would be less. But it didn’t have to be that way if there had been material readily available in Spanish, but there wasn’t–at least not for that particular class.

Drawing from that experience, I got in touch with Nico. I had been providing him with feedback on his book, C++17 – The Complete Guide, and pitched the idea of translating it to Spanish. As I outlined to Nico that day, my main purpose was and still is to bring a quality C++ book to Spanish-speaking students and professionals.

You can see (or rather, read) the results of that collaboration.

Working with Nico has been an enlightening experience. He is a professional in every extent of the word and always conducts himself in a professional manner. Furthermore, understanding the amount of work that it takes to take a book, even after translated, to a finished product is really an eye opener. In a chat I had with Rainer Grimm in CPPCON 2018 he mentioned that after Nico published a book, there was little material left out, or gaps to fill, if any–or words to that effect. Coming from Rainer, a renowned author, that’s a lot to say about a colleague. Having seen Nico’s work up close, I can only add myself to that comment.

Translating a book is a challenging task. Not only do you have to translate the words, but you have to do so accounting for style and technical terms, and with the different variations in Spanish, make sure that you don’t use terms that are specific to a country–e.g., Mexico. For that I have José Daniel García to thank for. José Daniel is also a member of the C++ Committee and patiently answered my questions on usage of technical terms, suggestions, and better flow.

I strongly believe that this book fills a gap in the material readily available for C++ in Spanish, and it is my hope that it assists others in learning this beautiful programming language that I have used since 1996 and over the years to program systems for the 9-1-1 public safety market, to save lives on a daily basis.

Should you be inclined to buy it or recommend it further to your Spanish-speaking amigos, you can find it on Leanpub, on Amazon, and its support website, with all of the code examples.

And yes, se habla C++.

Posted in software | Tagged , , | 1 Comment

A Conversion Story: Improving from_chars and to_chars in C++17

The Dome of The Pantheon, Rome

Note: You can get source code and examples on Github at https://github.com/ljestrada/charconvutils and watch my CPPCON2019 Lightning Challenge talk on YouTube here, and the Spanish version of this post here.

Nope, this is not about a religious experience, epiphany, change of faith or meeting my CMaker. Rather, it’s about C++.

C++17 sports two low-level character conversion functions,  std::from_chars and std::to_chars, but they have a usage model that can be easily improved using the power of templates.

from_chars and to_chars are appealing because they have a number of nice guarantees: locale-independent, non-allocating and non-throwing. Compare std::to_chars to std::to_string, which is locale-dependent and can throw bad_alloc. Saying that, the functions are in reality sets of overloaded functions for integer types, char, and floating point types (float, double and long double). Notice that bool is not supported, and neither are the other character types.

Nicolai Josuttis has excellent coverage and devotes a full chapter to these functions in his C++17: The Complete Guide eBook. There you can find examples for from_chars and to_chars using structured bindings and if with initialization, two other new additions to C++17. Moreover, you can reference the documentation at cppreference.com for a few examples.

The function signatures for the character conversion functions from_chars are as follows:

std::from_chars_result from_chars(
  const char* first, const char* last, /*see below*/& value,
  int base = 10);

std::from_chars_result from_chars(
  const char* first, const char* last, float& value, 
  std::chars_format fmt = std::chars_format::general);

std::from_chars_result from_chars(
  const char* first, const char* last, double& value,
  std::chars_format fmt = std::chars_format::general);

std::from_chars_result from_chars(
  const char* first, const char* last, long double& value,
  std::chars_format fmt = std::chars_format::general);

struct from_chars_result {
    const char* ptr;
    std::errc ec;
};  

The first signature is for integer types. The /*see below*/ comment refers to the integer types that are supported, as well as char (again, but not bool or any of the other character types). The rest of the signatures are for floating point types. The structure from_chars_result is used to hold the result. Albeit the function signatures are not marked with noexcept, the documentation states that they do not throw–errors are communicated via the ec member in the from_chars_result structure. Anecdotally, Microsoft’s implementation “strengthens” the functions by marking them as noexcept.

Similarly, the function signatures for the character conversion functions to_chars are as follows:

std::to_chars_result to_chars(
  char* first, char* last, /*see below*/ value, int base = 10);

std::to_chars_result to_chars(char* first, char* last,
  float value);

std::to_chars_result to_chars(char* first, char* last,
  double value);

std::to_chars_result to_chars(char* first, char* last,
  long double value);

std::to_chars_result to_chars(char* first, char* last,
  float value, std::chars_format fmt);

std::to_chars_result to_chars(char* first, char* last,
  double value, std::chars_format fmt);

std::to_chars_result to_chars(char* first, char* last,
  long double value, std::chars_format fmt);

std::to_chars_result to_chars(char* first, char* last,
float value, std::chars_format fmt, int precision);

std::to_chars_result to_chars(char* first, char* last,
  double value, std::chars_format fmt, int precision);

std::to_chars_result to_chars(char* first, char* last,
long double value, std::chars_format fmt, int precision);

struct to_chars_result {
    char* ptr;
    std::errc ec;
}; 

The usage model requires that you always provide two pointers to char. Consider converting a character sequence to a floating point type using from_chars:

#include <charconv>

int main() {
  char a[] = "3.141592";
  float pi;
  std::from_chars_result res = std::from_chars(a, a+9, pi);
  if (res.ec != std::errc{} ) {
    // CONVERSION FAILED
  }
}

The overloaded function taking a float will be selected and the result will be placed in the variable pi.

Similarly, to turn a floating point variable into a character sequence using to_chars:

#include <charconv>

int main() {
  char a[10];
  float pi = 3.141592f;

  std::to_chars_result res = std::to_chars(a, a+10, pi);
  if (res.ec != std::errc{} ) {
    // CONVERSION FAILED
  }
}

See the documentation for the value of the ptr member upon success or failure.

While the usage model is consistent, one cannot help but notice that for cases where the array size is known, one must pass one pointer too many. What if we could use instead:

#include "charconvutils.hpp" // templates and pixie dust

using charconvutils::from_chars, charconvutils::to_chars;

int main() {
  char in[] = "3.141592";
  char out[50]; 
  float pi;

  // Look, ma, no tracking array size
  std::from_chars_result res1 = from_chars(in, value);
  std::to_chars_result   res2 = to_chars(out, pi);
}

The usage model is simplified: you do not need neither to keep track of the array size. Furthermore, you could use other classes such as std::array, std::string, std::string_view or std::vector. Interested? Read on.

Function templates to the rescue

By looking carefully at the function signatures, you can come up with the following observations:

  • All the functions take a pair [const] char* that constitute a valid range. In the case of from_chars, the range is read-only, in the case of to_chars, it is where the output will be placed.
  • The functions are overloaded for integer types, char and floating point types.

A first cut at a function template for from_chars could look like this (the ??? is information that will be provided later):

template<std::size_t N, typename T>
std::from_chars result
from_chars(const char(&a)[N], T& value, ???)
{
  return std::from_chars(a, a+N, value, ???);
}

This initial stub takes advantage of template argument deduction to determine the array size. Because the array is passed by reference (that’s what the syntax const char(&a)[N] does), it won’t decay onto a pointer, and that’s what we want.

Similarly, a first cut onto a function template for to_chars could look like this:

template<std::size_t N, typename T>
std::to_chars result
to_chars(const char(&a)[N], T& value, ???)
{
  return std::to_chars(a, a+N), value, ???);
}

Here’s a non-working version of the function templates using this approach, limited to from_chars for simplicity:

// For integral types
template<std::size_t N, typename T>
std::from_chars_result
from_chars(const char(&a)[N], T& value, int base = 10)
{
  return std::from_chars(a, a + N, value, base);
}

// For floating-point types
template<std::size_t N, typename T> 
std::from_chars_result
from_chars(const char(&a)[N], T& value,
           std::chars_format fmt = std::chars_format::general)
{
  return std::from_chars(a, a + N, value, fmt);
}

Running this version of the function templates somewhat works, but it breaks when you use a floating point type and don’t provide a base argument, defaulting to 10, and thinking that the second function template will be instantiated, but that is not the case. When called with the same number of arguments, the first function template will be instantiated with a float/double/long double, and the call to std::from_chars does not match.

Reversing the function template definitions–first the one for floating point types, then the one for integral types–does not work, either.

What we want is a way to instantiate the first function template when we pass an integer type, and the second when we pass a floating point type. What we need are templates and a little bit of pixie dust:

#include "charconvutils.hpp" // templates and pixie dust

 using charconvutils::from_chars, charconvutils::to_chars; 

int main() {
  char a[] = "365";
  int daysPerYear;

  // Instantiates a function template for integer types
  std::from_chars_result res1 = from_chars(a, daysPerYear);

  char b[] = "3.141592";
  float pi;

  // Instantiates a function template for floating point types
  std::from_chars_result res2 = from_chars(b, pi);
}

Of course, using the function with additional parameters would let the compiler pick the right function, too, such as providing a base for the integer types or fmt for the floating point types.

#include "charconvutils.hpp" // templates and pixie dust

 using charconvutils::from_chars, charconvutils::to_chars; 

 int main() {
  char a[] = "365";
  int daysPerYear;

  // Instantiates a function template for integer types
  std::from_chars_result res1 = from_chars(a, daysPerYear, 10);

  char b[] = "3.141592";
  float pi;

  // Instantiates a function template for floating point types
  std::from_chars_result res2 = from_chars(b, pi);
}

Enter enable_if

The solution is to use the type trait enable_if, available in the <type_traits> header file, or rather its alias template enable_if_t. This type trait yields void when passed one argument, or a given type when passed two arguments and the first argument yields true.

The pixie dust behind enable_if is that if the instantiation fails, the template in question will not be instantiated. Nifty. By using two other type traits, is_integral and is_floating_point, we define two helper alias templates that will let us discriminate between integral types and floating point types and select the proper template function. A minor caveat is that the is_integral includes bool and character types other than char, but for our purposes it should suffice–we can let the compiler reject other instantiations. Moreover, we will use the alias templates is_integral_v and is_floating_point_v, available since C++17, for simplification.

// Alias template for integer types types
template<typename T>
using EnableIfIntegral = 
  std::enable_if_t<std::is_integral_v<T>>;

// Alias template for floating point types
template<typename T>
using EnableIfFloating =
  std::enable_if_t<std::is_floating_point_v<T>>;


NOTE: Integer or integral? The type trait std::is_integral encompasses bool, character types and integer types. The std::from_chars and std::to_chars character conversion functions consider only integer types, char and floating point types. Keep in mind that the alias template EnableIfIntegral will filter anything allowed by the std::is_integral template.

With the alias templates handy, we define two function templates for the from_chars character conversion function (we will extend to types other than arrays of characters later):

template<std::size_t N, typename T, typename=EnableIfIntegral<T>>
std::from_chars_result
from_chars(const char(&a)[N], T& value, int base = 10)
{
  return std::from_chars(a, a + N, value, base);
}

template<std::size_t N, typename T  typename=EnableIfFloating<T>>
std::from_chars_result
from_chars(const char(&a)[N], T& value,
   std::chars_format fmt = std::chars_format::general)
{
  return std::from_chars(a, a + N, value, fmt);
}

Similarly, we define three function templates for the to_chars character conversion function. Notice that the function templates for floating types do not have a default format or precision:

template<std::size_t N, typename T, typename=EnableIfIntegral<T>>
std::to_chars_result
to_chars(char(&a)[N], T value, int base = 10)
{
  return std::to_chars(a, a + N), value, base);
}

template<std::size_t N, typename T, typename=EnableIfFloating<T>> 
std::to_chars_result
to_chars(char(&a)[N], T value, std::chars_format fmt)
{
  return std::to_chars(a, a + N), value, fmt);
}

template<std::size_t N, typename T, typename=EnableIfFloating<T>>  
std::to_chars_result
to_chars(char(&a)[N], T value, std::chars_format fmt,
         int precision)
{
    return std::to_chars(a, a + N), value, fmt, precision);
}

Extending to std::array

It is easy to extend the functionality to std::array. The corresponding from_chars and to_chars for std::array are straightforward. Here are a couple of examples for integer types:

template<std::size_t N, typename T, typename=EnableIfIntegral<T>>
std::from_chars_result
from_chars(const std::array<char, N>& a, T& value, int base = 10)
{
  return std::from_chars(a.data(), a.data() + N, value, base);
}

template<std::size_t N, typename T, typename=EnableIfIntegral<T>> 
std::to_chars_result
to_chars(std::array<char, N>& a, T value, int base = 10)
{
  return std::to_chars( a.data(), a.data() + N, value, base);
}

Extending to Random Access Containers

In Generic Programming and the STL, Matt Austern describes random access containers. The book relies on the SGI STL documentation (or vice versa) and Martin Broadhurst kindly preserved it here. You can find the description of random access containers here. With that said, std::string, std::vector, std::deque and std::array are random access containers and could be used as the source or destination of from_chars and to_chars. Simply allocate the storage needed for the conversion ahead of time. std::string_view, although strictly speaking not a random access container, qualifies because it provides the same interface, accessing its underlying std::string. Of course, one must guarantee that the containers won’t change while the conversion takes place.

With the definitions that follow, the function templates for std::array defined above can be removed and replaced.

The function template calculates the range at runtime using data() and size(). In addition, there’s a check to avoid passing containers whose value type is not char. Here are a couple of example for integer types:

template<typename Cont, typename T, typename=EnableIfIntegral<T>>
std::from_chars_result
from_chars(const Cont& c, T& value, int base = 10)
{
  static_assert(std::is_same_v<char, typename Cont::value_type>,
                "Container value type must be char.");
  return std::from_chars(c.data(), c.data() + c.size(), 
                         value, base);
}

template<typename Cont, typename T, typename=EnableIfIntegral<T>> 
std::to_chars_result
to_chars(Cont& c, T value, int base = 10)
{
  static_assert(std::is_same_v<char, typename Cont::value_type>,
                "Container value type must be char.");
  return std::to_chars(c.data(), c.data() + c.size(),
                       value, base);
}

Kicking the tires…

You would expect that after all of this template brouhaha things would just magically work, but alas, the compiler support for C++17 is, let’s say, unbalanced? I ran into different problems and decided to see if support for these character conversion functions was really there. I ran two programs with Compiler Explorer, one to test from_chars, which you can find here, and one to test to_chars, which you can find here.

I tested with C++ Compiler Explorer using the latest versions of MSVC, Clang and GCC, which support for C++17 (and therefore <charconv>).

For MSVC I used the flags /std:c++17, for Clang I used the flags -std=c++17 -stdlib=libc++ and for GCC I used the flags -std=c++17. If you run Clang with this flag effectively you’re using Clang as a front end to GCC.

The test program for from_chars is reproduced here. I’ve used the problematic portions with numbers (1 through 4). These tests are not exhaustive and you may try other types (e.g., wchar_t) to see if there are other inconsistencies in the support.

#include <charconv>

using std::from_chars, std::from_chars_result, 
      std::chars_format;

int main() {
  char const bstr[] = "true";
  bool bValue;

  // 1: bool should not be supported
  from_chars_result bres = from_chars(bstr, bstr+5, bValue);

  char const istr[] = "12345";
  int iValue;
  from_chars_result ires = from_chars(istr, istr+6, iValue);

  char const lstr[] = "12345";
  long lValue;
  from_chars_result lres = from_chars(lstr, lstr+6, lValue);

  char const fstr[] = "3.141592";
  float fValue{};

  // 2: Should pick overload that takes float
  from_chars_result fres = from_chars(fstr, fstr+9, fValue);

  char const dstr[] = "3.14159265358979";
  double dValue;

  // 3: Should pick overload that takes double
  from_chars_result dres = from_chars(dstr, dstr+17, dValue,
                                      chars_format::general);

  char const ldstr[] = "1234567890.50";
  long double ldValue;
  
  // 4: Should pick overload that takes long double
  from_chars_result ldres = from_chars(ldstr, ldstr+14, ldValue);
}

The test program for to_chars is reproduced here.

#include <charconv>

using std::to_chars, std::to_chars_result, std::chars_format;

int main() {
  char bstr[10];
  bool bValue{true};

  // 1: bool should not be supported
  to_chars_result bres = to_chars(bstr, bstr+5, bValue);

  char istr[6];
  int iValue{12345};
  to_chars_result ires = to_chars(istr, istr+6, iValue);

  char lstr[6];
  long lValue{12345};
  to_chars_result lres = to_chars(lstr, lstr+6, lValue);

  char fstr[9];
  float fValue{3.141592f};

  // 2: Should pick overload that takes float, chars_format
  to_chars_result fres = to_chars(fstr, fstr+9, fValue,
                                  chars_format::general);

  char dstr[17];
  double dValue{3.14159265358979};

  // 3: Should pick overload that takes double, chars_format,
  //    precision
  to_chars_result dres = to_chars(dstr, dstr+17, dValue,
                                  chars_format::general, 10);

  char ldstr[14];
  long double ldValue{1234567890.50};
  
  // 4: Should pick overload that takes long double, chars_format 
  to_chars_result ldres = to_chars(ldstr, ldstr+14, ldValue,
                                   chars_format::general, 10);
} 

Testing charconv support in Clang 8.0.0

The first thing I found is that there seems to be a bug in Clang’s implementation when using libc++. It correctly flags the conversion to bool, on both tests, but incorrectly flags the calls using floating point types at #2, #3 and #4 .

Testing charconv support in GCC 9.1

Right off, GCC cannot find std::chars_format. Similar results when using Clang as a front end to GCC. std::chars_format seems to be in another header and most probably has not been ported fully. It correctly flags the conversion to bool at #1, but incorrectly flags the calls using floating point types at #2, #3 and #4.

Testing charconv support in MSVC 19.20

The best results. I used flags /std:c++17. It correctly flags the conversion to bool at #1 for the from_chars test, but allows conversion from bool to char[] for the to_chars test (Incidentally, Visual Studio 2019 v16.2 Preview 1 adds additional support for floating point to_chars overloads and the feature test macro __cpp_lib_to_chars, a welcome improvement).

…and taking it for a spin

With the compilers out of the way, we can now turn towards testing the function templates from_chars and to_chars. Here’s the full "charconvutils.hpp" file, which you can find in Compiler Explorer here:

#ifndef CHARCONVUTILS_HPP
#define CHARCONVUTILS_HPP

#include <charconv>
#include <type_traits>

namespace charconvutils {

// -----
// Helper alias templates

template<typename T>
using EnableIfIntegral = 
  std::enable_if_t<std::is_integral_v<T>>;

template<typename T>
using EnableIfFloating = 
std::enable_if_t<std::is_floating_point_v<T>>;

// -----
// from_chars function templates for C-style arrays

template<std::size_t N, typename T,
         typename =  EnableIfIntegral<T>>
std::from_chars_result
from_chars(const char(&a)[N], T& value, int base = 10) {
  return std::from_chars(a, a + N, value, base);
}

template<std::size_t N, typename T, 
         typename =  EnableIfFloating<T>>
std::from_chars_result
from_chars(const char(&a)[N], T& value,
           std::chars_format fmt = std::chars_format::general) {
  return std::from_chars(a, a + N, value, fmt);
}

// -----
// from_chars function templates for random access containers

template<typename Cont, typename T,
         typename = EnableIfIntegral<T>>
std::from_chars_result
from_chars(const Cont& c, T& value, int base = 10) {
  static_assert(std::is_same_v<char, typename Cont::value_type>, 
                "Container value type must be char.");
  return std::from_chars(c.data(), c.data() + c.size(),
                         value, base);
}

template<typename Cont, typename T,
         typename = EnableIfFloating<T>>
std::from_chars_result
from_chars(const Cont& c, T& value,
           std::chars_format fmt = std::chars_format::general) {
  static_assert(std::is_same_v<char, typename Cont::value_type>, 
                "Container value type must be char.");
  return std::from_chars(c.data(), c.data() + c.size(),
                         value, fmt);
}

// -----
// to_chars function templates for C-style arrays

template<std::size_t N, typename T,
         typename = EnableIfIntegral<T>>
std::to_chars_result
to_chars(char(&a)[N], T value, int base = 10) {
  return std::to_chars(a, a + N, value, base);
}

template<std::size_t N, typename T,
         typename = EnableIfFloating<T>>
std::to_chars_result
to_chars(char(&a)[N], T value, std::chars_format fmt) {
  return std::to_chars(a, a + N, value, fmt);
}

template<std::size_t N, typename T,
         typename = EnableIfFloating<T>>
std::to_chars_result
to_chars(char(&a)[N], T value, std::chars_format fmt,
         int precision) {
  return std::to_chars(a, a + N, value, fmt, precision);
}

// -----
// to_chars function templates for random access containers

template<typename Cont, typename T,
         typename = EnableIfIntegral<T>>
std::to_chars_result
to_chars(Cont& c, T value, int base = 10) {
  static_assert(std::is_same_v<char, typename Cont::value_type>, 
                "Container value type must be char.");
  return std::to_chars(c.data(), c.data() + c.size(),
                       value, base);
}

template<typename Cont, typename T,
         typename = EnableIfFloating<T>>
std::to_chars_result
to_chars(Cont& c, T value, std::chars_format fmt) {
  static_assert(std::is_same_v<char, typename Cont::value_type>, 
                "Container value type must be char.");
  return std::to_chars(c.data(), c.data() + c.size(),
                       value, fmt);
}

template<typename Cont, typename T,
         typename = EnableIfFloating<T>> 
std::to_chars_result
to_chars(Cont& c, T value, std::chars_format fmt, int precision) {
  static_assert(std::is_same_v<char, typename Cont::value_type>,
                "Container value type must be char.");
  return std::to_chars(c.data(), c.data() + c.size(),
                       value, fmt, precision);
}

} // namespace charconvutils

#endif // CHARCONVUTILS_HPP

With this file in place, you can run the sample programs outlined above.

Afternotes

Jens Maurer proposed the character conversion functions in this paper. It’s interesting to see the proposed interfaces, and I could not find out why templates were not considered (albeit std::string_view was). Perhaps in a different iteration of the paper this idea was rejected and I’m missing something (e.g., considerations for the containers not to be accessed by multiple threads while the conversion takes place).

What about structured bindings? Sure, you can use structured bindings to capture the resulting structure–as previously mentioned, Nicolai Josuttis has several examples.

Another aspect that seems to be missing is that if the functions as declared are non throwing, why not mark them with noexcept? That’s what Microsoft did, and it seems only logical. Perhaps a consideration is that the range would become invalid while performing the conversion?

In closing, this post explored how one can improve the usage model for the character conversion functions by investing in function templates.

And remember, vote for Pedro.

Updated on June 20, 2019

Posted a bug report to Microsoft for taking the bool parameter and they report that the behavior is as designed (that is, the bool is promoted to an integer). You can find the problem report here.

The approach taken by Clang is that they mark the with delete, so it does not allow calling std::to_chars by passing a bool.

While logical, this discrepancy among compilers is problematic–which implementation is correct?

The problem I see with allowing the conversion to chars from bool is that there is no symmetry: Calling std::from_chars with “true” or “false” does not result in an int 1 or 0, or a bool. Now, that’s understandable, since there can be many representations for true and false strings (e.g., “yes” and “no”), but the opposite should hold, too.

Updated on August 28, 2019

An issue has been entered in the LWG that will delete the std::to_chars overload for bool overload. It can be found here:
https://cplusplus.github.io/LWG/issue3266
and the corresponding documentation in cppreference.com reflects it.

Updated on September 16, 2019

Reflects comments and simplification by using the data() member for containers, and now preserving the same semantics as the std::from_chars and std::to_chars functions.

Posted in C++, C++17, software | Tagged , , , , | 1 Comment

Mary had a Little Lambda (Architecture)

 

ardillaNursery rhyme aside, I’ve been looking avidly at Big Data Lambda Architectures. Nathan Marz introduced the term back in 2012, which is reminiscent of λ-Calculus. The first time you hear the term it brings memories of high-order functions in programming languages (functional or imperative, applications or systems). It is a layered architectural style, similar in nature, since its layered, to Pipes and Filters…but for Big Data.

The “lambda” portion of the term refers to data immutability at the Batch Layer (similar to pure functions). Quoting Nathan, “The batch layer needs to be able to do two things to do its job: store an immutable, constantly growing master dataset, and compute arbitrary functions on that dataset.”

Lambda Architectures are not new, but I think that Nathan had the great idea of giving it a name. Reminds me of the Allegory of the Cave by Plato, where while in the cave, we professionals have most likely seen a lambda architecture in one way or another, or participated in developing, or even created products with similar characteristics, but could not describe it fully. Nathan departed from the cave and saw the general aspects of the architectural style and then returned to the cave and described the real thing, giving a name that, whether you like it or not, is here to stay. Hey, it’s catchy and intriguing!

Lately I’ve been thinking that Lambda Architectures come in two flavors: Big Lambda Architectures (𝚲-Architecture), which is what Nathan describes, and Little Lambda architectures (λ-Architecture), and the differentiation has to do with how much the underlying technologies can scale, and one would choose such technologies on purpose instead of “scaling down” Big Data products. Otherwise, how would architectures that have the three layers and same functions, but don’t scale as much be called? I can think of analytics products that have evolved from little lambda to big lambda, too, so little lambdas must exist.

Take it with a little grain of data salt 😉

Posted in big data, lambda architecture, software, software architecture | Leave a comment

Customizing Priority Queues in C++

seagull
A nifty feature of priority queues in C++ is that you can implement a max heap or a min heap by simply changing the compare type. For most purposes, std::less and std::greater can do the job, provided that you use built-in types or custom types that provide a < or > operator and strict weak ordering.

A priority_queue can be implemented by a couple of different containers, with vector being the default.  In addition, you can provide a Compare type that must comply with strict weak ordering.

 

template<typename T, typename Container=std::vector<T>,
    typename Compare=std::less<typename Container::value_type>> 
        class priority_queue;

Here’s a priority queue with a max heap, followed by a priority queue with a min heap (and yes, it’s counterintuitive that max heap uses std::less and min heap uses std::greater).

priority_queue<int, vector<int>, less<int>> max_pq;
priority_queue<int, vector<int>, greater<int>> min_pq;

max_pq.push(1); max_pq.push(2); max_pq.push(3);
min_pq.push(1); min_pq.push(2); min_pq.push(3);

// prints 3 2 1
while (!max_pq.empty()) {
  cout << max_pq.top() << ' ';
  max_pq.pop();
}
cout << '\n';

// prints 1 2 3
while (!min_pq.empty()) {
  cout << min_pq.top() << ' ';
  min_pq.pop();
}
cout << '\n'

Nothing earth-shattering here. However, if you have a user-defined type, you have several mechanisms at your disposal. Assume that we have the following type

struct student {
  student(double gpa, int age) : gpa{gpa}, age{age} {}
  
  double gpa;
  int age;
};

A student has a GPA and a given age.  We want to give priority to students with the highest GPA.  Let’s look at the options:

Provide comparison operators

Comparison operators can be done as a member function, as a friend function if it accesses private member variables of the type, or as a free function if it makes use of only publicly accessible parts of the type. Once you have comparison operators, the type can be safely used with std::less and std::greater (should you provide both operators).

struct student {
  student(double gpa, int age) : gpa{gpa}, age{age} {}
  
  double gpa;
  int age;

  // Define it as a member function
  bool operator < (const student& rhs) const { return gpa < rhs.gpa; }

  // Define it as as a friend if it accesses private member variables,
  // or as a free function if it does not.
  // (in this case it does not, but it's for illustration).
  friend bool operator >(const student& lhs, const student& rhs);
};

bool operator >(const student& lhs, const student& rhs) {
  return gpa > rhs.gpa;
}

int main() {
  priority_queue<student, vector<student>, std::less> max_gpa;
  priority_queue<student, vector<student>, std::greater> min_gpa;

  student alice(3.98,15), bob(3.97, 18), charlie(3.95, 16);
  max_gpa.push(alice);
  max_gpa.push(bob);
  max_gpa.push(charlie);

  // prints 3.98 3.97 3.95
  while (!max_gpa.empty()) {
    cout << max_gpa.top().gpa << ' ' << endl;
    max_gpa.pop();
  }

  min_gpa.push(alice);
  min_gpa.push(bob);
  min_gpa.push(charlie);

  // prints 3.95, 3.97, 3.98
  while (!min_gpa.empty()) {
    cout << min_gpa.top().gpa << ' ' << endl;
    min_gpa.pop();
  }
}

Roll your own  function object…

Simply define a type with a function call operator as shown below. The signature for the function call operator makes it a binary predicate.

struct student {
  student(double gpa, int age) : gpa{gpa}, age{age} {}
  
  double gpa;
  int age;
};

struct less_gpa {
  bool operator() (const student& lhs, const student& rhs) { 
    return lhs.gpa < rhs.gpa;
  }
};

struct greater_gpa {
  bool operator() (const student& lhs, const student& rhs) { 
    return lhs.gpa > rhs.gpa;
  }
};

int main() {
  priority_queue<student, vector<student>, less_gpa> max_gpa;
  priority_queue<student, vector<student>, greater_gpa> min_gpa;

  student alice(3.98,15), bob(3.97, 18), charlie(3.95, 16);
  max_gpa.push(alice);
  max_gpa.push(bob);
  max_gpa.push(charlie);

  // prints 3.98 3.97 3.95
  while (!max_gpa.empty()) {
    cout << max_gpa.top().gpa << ' ';
    max_gpa.pop();
  }
  cout << '\n';

  min_gpa.push(alice);
  min_gpa.push(bob);
  min_gpa.push(charlie);

  // prints 3.95, 3.97, 3.98
  while (!min_gpa.empty()) {
    cout << min_gpa.top().gpa << ' ';
    min_gpa.pop();
  }
  cout << '\n';
}

…Or use std::binary_function as the base class for your function object

The only benefit to using a binary function object is if you need the predefined typedefs for the argument and return types and use them in generic programming.  You still must provide the function call operator and be able to access the necessary elements.

struct student {
  student(double gpa, int age) : gpa{gpa}, age{age} {}
  
  double gpa;
  int age;
};

struct less_gpa : public std::binary_function<student, student, bool> {
  bool operator() (const student& lhs, const student& rhs) { 
    return lhs.gpa < rhs.gpa;
  }
};

struct greater_gpa : public std::binary_function<student, student, bool> {
  bool operator() (const student& lhs, const student& rhs) { 
    return lhs.gpa > rhs.gpa;
  }
};

Use a lambda expression

Technically, you will use a copy of the closure generated by the lambda expression and assigned to the f variable. Scott Meyers gives a great explanation here.

struct student {
  student(double gpa, int age) : gpa{gpa}, age{age} {}
  
  double gpa;
  int age;
};

int main()  
 // Binary predicate with lambda
  auto f = [](const student& lhs, const student& rhs) -> bool {
    return lhs.gpa < rhs.gpa;
  };

  // Notice the decltype and passing f in the constructor
  priority_queue<student, vector<student>, decltype(f)> max_gpa(f);

  student alice(3.98,15), bob(3.97, 18), charlie(3.95, 16);

  max_gpa.push(alice);
  max_gpa.push(bob);
  max_gpa.push(charlie);

  // prints 3.98 3.97 3.95
  while (!max_gpa.empty()) {
    cout << max_gpa.top().gpa << ' ';
    max_gpa.pop();
  }
  cout << '\n';

And if you need two or more criteria…

All of the options above will work when you have only one criterion for the priority queue, but what happens if you want to apply two or more criteria?  Say that you want to give the highest priority to the student with the highest GPA, but in case of a tie, you want to give it to the younger student. In this case you can apply any of the methods above, but in your comparison, you have to consider the additional criteria.  In the code below, when the age GPA is the same, the return value is true if the lhs student is older–and in a max heap priority queue, the younger student will be given higher priority.

struct student {
  student(double gpa, int age) : gpa{gpa}, age{age} {}
  
  double gpa;
  int age;
};

int main()  
  // Binary predicate with lambda
  auto f = [](const student& lhs, const student& rhs) -> bool {
    if (lhs.gpa < rhs.gpa)
      return true;
    else if (lhs.gpa == rhs.gpa && 
             lhs.age > rhs.age)
      return true;
    else
      return false;
  };

  // Notice the decltype and passing f in the constructor
  priority_queue<student, vector<student>, decltype(f)> max_gpa(f);

  student alice(3.98,15), bob(3.97, 18), charlie(3.98, 16);

  max_gpa.push(alice);
  max_gpa.push(bob);
  max_gpa.push(charlie);

  // prints (3.98, 15) (3.98, 16) (3.97, 18)
  while (!max_gpa.empty()) {
    const student& s = max_gpa.top();
    cout << '(' << s.gpa << ", " << s.age << ')' << ' ';
    max_gpa.pop();
  }
 cout << '\n';

 

Posted in algorithms, data structures, software | Leave a comment

The (White) Elephant in the Room

lizard

All successful software development projects are alike; each unsuccessful software development project is unsuccessful in its own way…and some become white elephants. Paraphrasing and extending the first sentence of Anna Karenina, a white elephant is not only a failed project, but also a project that fails miserably, severely late to market–if at all–, severe cost overruns, severe quality problems or a combination of the above. As a software professional, chances are you’ve participated in a white elephant project.

Why do white elephants happen?

Nobody wants a white elephant. So how is it that, as software professionals and very frequently being the most expensive resources for a company, we (as in the collective we) manage to produce white elephants?

There’s a combination of factors, but projects that turn into white elephants are typically very large, either for the size of the company, or multi-million, multi-year projects that span multiple, possibly geographically disperse teams. Large product rewrites are typical, but you can also find white elephants when investing in new technology, too.

While researching for this post, I came across this article, entitled The Most Common Reasons Why Software Projects Fail.  Unfortunately, not much has changed from previous years, and the reasons are similar to works published by other authors.  A couple of the points in that article that resonate with me have to do with schedules and process.

Mandated completion dates originate from the need to deliver products that are time-sensitive: Products sensitive to time to market can become irrelevant if they’re not delivered on time, and this is perfectly understood by the collective we, so why are these schedules still imposed and accepted?  For a green team–and I’m not talking about an energy-efficient team–it is easier to impose a mandated completion date.  Few dare to say no or negotiate a different outcome, and the team embarks in crazy hours.  These teams usually have high turnover rate and replacing members is expensive to the project.  This white elephant is easier to understand, but how about experienced teams savvy teams that can negotiate a schedule?

Software development processes are necessary in any product that is going to be in front of customers…yet, when we fail to follow them chaos ensues and things become like a Running of the Bulls in Pamplona, where everyone runs for their lives. I remember a C++seminar I went to in 2003, where the speaker, Herb Sutter, asked the audience how many of us had a software development process in place in our companies.  I remember raising my hand…only to find myself alone. I don’t think things have changed dramatically in the last (ouch!) 13 years.  What used to be iterative processes, such as Rational Unified Processes (and referred by people that has joined the ranks after the Agile manifesto as “waterfall”), turned into Agile, but implementations true to the spirit of Agile are scarce (and some are turned into Fragile) and now DevOps.

Two other reasons I’ve seen for white elephants are gullibility and panacea. Gullibility is an over-promised and under-delivered completion date. It flows from the bottom layers to upper management, who buys the completion date with a complete disregard for data that shows the opposite. Panacea can be anything that will deliver your project, be it a technology that is a silver bullet (and the one you want to add to your resume) or compressing a schedule by bringing another team, but failing to manage it properly. For some reason it seems that throwing people at a problem is akin to throwing hardware at a problem–heck, add more RAM and while you’re at it add more disk space. Any additional resources that you throw at a problem will increase the amount of communication needed and the number of “collisions” will increase, too. Nothing new under the sun.

So..have you seen a white elephant lately? I didn’t think so. They don’t exist, right?

Posted in software | Leave a comment

My Sister From Another Mister

No, this is not about family infidelities.  Rather, it is about copyright laws.  You see, a couple of months ago the debate team in my daughter’s school had to prepare for a topic on copyright laws and how effective or ineffective they are.

I’ve always thought that copyright laws mean something, and given that my friend Catherine is always calling me “brother from another mother”, I only thought it was natural to start calling her sister from another mister and see if copyright really works.

The purpose of this post is to claim a copyright in LinkedIn (and now WordPress) over this simple phrase, an experiment of sorts.  Who knows, perhaps it’ll become something like the smiley–albeit I don’t expect to make any money on it.  I must also disclose that I did my due diligence and Googled for the phrase, and it did not exist.  So there you have it.  You can use it in social gatherings to show your smart wits, T-shirts, or whatever floats your boat and to your heart’s content and don’t have to pay any royalties or attributions–as long as you don’t claim it as yours.

Sister From Another Mister, © Javier Estrada, 2016

Sista’ From Another Mista’ © Javier Estrada, 2016

Posted in software | Leave a comment

The Leprechaun Trap

A few years ago and on occasion of St. Patrick’s Day, my daughter built a leprechaun trap. She wrapped a shoe box in shiny green paper and cut a slot through it, decorated it with a border with jingle bells, added a ladder made of Q-tips so the leprechaun would be able to climb, and then added a gold box with the legend “gold inside”.  To trick the leprechaun, she hung a piece of
“gold” from a pole. Once he reached for the piece of gold, he would fall through the slot and he’d be caught. Who could argue with such piece of engineering? It was virtually impossible for the leprechaun to escape once tempted.

The night before she asked me impatiently where would be the best place to place the trap.  I mentioned that next to the front door would be best, since the leprechaun would have to come in from somewhere.  She placed the trap and we all went to sleep.  Next morning, as I got off to work–I’m an early riser, so it was still dark–and opened the door, I almost tripped because of some object on the floor.  I uttered words that I cannot repeat here, and as I turned on the light I realized I had rattled the leprechaun trap and had messed it up a little.

After realizing my mistake, I decided to leave the trap as it was and just left.  Sure enough, later in the day my daughter called me very excited: “Dad, I almost caught the leprechaun!”, and she described what had happened, and how she had found the box. When I came back, she described why the trap had failed and started thinking of improvements–she kept the box and used it the following year, again, with no success.

There are leprechaun traps in software. Very frequently we’re tempted by “gold”whether it is in the form of:

  1. Latest and greatest programming language X…it doesn’t matter if X has a few percentage points in global usage and has remained so for several years. The problem is not in the language itself, it’s mostly in the long-term viability, tools and libraries around it.
  2. Latest technology…even though there is scarce support for it.  How often have you looked at an open source library that looks promising only to find out that the community
    around it is a couple of developers and it has not maintained for years, or that once you look under the hood the code quality leaves a lot to be desired?
  3. Latest and greatest development process…DevOps anyone, anyone, Bueller? The problem is not in the development process–after all DevOps  responds to a set of needs–, it is in the edicts that move teams towards development processes that are  misunderstood or inappropriate, often without training and simply as a trend that starts who knows where.

It does not mean that there are not authentic opportunities for striking “gold” and that one shouldn’t be looking elsewhere for satisfactory solutions not currently met with existing technologies.  So why, as software professionals, do we often fall in a leprechaun trap? A coworker of mine long ago mentioned that we as engineers are pleasers.  We’re problem solvers, and often times overly optimistic.  It’s hard to say no to “gold”, to resist the impulse and think back carefully before we offer our educated guesses and commit to schedules that are unrealistic–but there’s a shiny box that reads “gold inside”. We fall into the leprechaun trap, and once inside, we know it’s hard to climb out of late schedules, budget overruns and buggy software: whole teams have to work overtime to compensate for the late schedules and the budget overruns, and with buggy software you just build a backlog that you–or some unlucky soul–will have to work on for the next year because of software “gold”.

The secret not to fall into a leprechaun trap?  Don’t be a leprechaun.

Happy Saint Patrick’s Day!

Posted in software | Tagged | Leave a comment

Observations on Observer

zopiloteMy first encounter with the Observer design pattern was circa 1998.  The GoF book was still new–published in 1994–and the community was still assimilating design patterns. Java  the Hot* even offered an Observer interface and an Observable class. What could be simpler?

However, I’ve used Observer exactly once.  The main reason is that Observer is a scaffolding pattern–it is good to understand the concepts, but once you try to implement a robust version of it, it becomes complicated very quickly.  Furthermore, you often graduate to a better version of it, and often times there are language-specific facilities or libraries that implement a variation on Observer. In essence, not worth your squeeze.

Consider this naïve implementation in C++:

struct observer {
  virtual void update() = 0;
};

class subject {
 set<observer> observers;
 public:
  void notify() { 
   for_each(begin(observers), end(observers), mem_fn(&observer::update));
  }

  add(const observer& o) { observers.insert(o); }
  remove(const observer& o) { observers.erase(o); } 
};

class concrete_subject : public subject {
 ...
};

The problems start very quickly.  They’ve been discussed ad naseam over the years. One of the most complete discussions I’ve read are by Alexandrescu [1], [2] and Sutter [3].

  1. What type of container is best? For example, can an observer register more than once? Can observers be notified in a specific order?
  2. Is the container thread-safe? For example, can observers be added or removed concurrently from different threads.
  3. What is the type a container stores?  Storing an observer requires it is copy-constructible, so perhaps storing an observer* is best.  However, if the observer goes out of scope without being removed from the subject one will end up with a dangling pointer.  How about using a shared_ptr<observer>?  If all but the reference stored in a subject are released, unwanted notifications could be coming in.
  4. Can observers call remove from within the update method?  This could invalidate the iterator in subject::notify.
  5. What happens if an exception is thrown from the a concrete observer’s update member function? You don’t pass Go, you don’t collect $200.

These questions are not specific to C++–the same problems need to be solved Java or C#, for that matter, even when using delegates**, which behind the scenes implement a variation on Observer: It provides a collection to store the targets, add/remove methods with operators +=, -=, and a notification mechanism that executes on the thread where the delegate is invoked.

Java’s  Observer interface and Observable are not perfect, either. If you read the fine print, you’ll notice that the notifications can come in different threads.  Ouch. This means that you better make sure your event handler is thread-safe.

Notifications per Method Ratio (N:M)

A taxonomy for Observer that can let you choose what to use is the ratio of notifications per method.  Assume N is the number of notifications from a specific Subject and M is the number of methods to handle the notification.  When choosing, you can look at the N:M ratio and see if it makes sense to you.

In Observer, the N:M ratio is many:1, since all notifications are funneled through the update method.

In the Delegation Event Model (DEM) design pattern, all the notifications are grouped into one interface, with one method per notification. The N:M ratio is 1:1.

// Methods in Java interfaces are public.  Ditto in C#
// Events are grouped in one 
public interface XxxListener {
  void event1(XxxEvent e);
  void event2(XxxEvent e); 
}

public class MyXxxListener implements XxxListener {
  public void event1(XxxEvent e) { ... }
  public void event2(XxxEvent e) { ... }
}

public class MySubject {
  public addXxxListener(XxxListener lis) { ... }
  public removeXxxListener(XxxListener lis) { ... }
}

In C#, using delegates is a language-specific version of Observer,  where each delegate, while it can address several targets, it will support a notification per delegate, so the N:M ratio is also 1:1.

If you want notifications à la carte, in C++ you can make use of Boost.Signals2, a signals and slots, thread-safe library where you can get a notification per signal. You can consider signals a C++ alternative to delegates in C#. Added flexibility is that you can specify a priority for each slot. Just like in C#, there will be a signal per slot (or a notification per method). The N:M ratio is also 1:1.

In essence, when shopping around for Observer, beware of the consequences and trade-offs of the implementation, whether it is a language-specific solution, a library or a roll-your own implementation.

References

[1] Alexandrescu, Andrei,  Generic<Programming>: Prying Eyes: A Policy-Based Observer (I), C/C++ Users JournalApril 2005.

[2] Alexandrescu, Andrei,  Generic<Programming>: Prying Eyes: A Policy-Based Observer (II), C/C++ Users Journal, June 2005.

[3] Sutter, Herb,  Generalizing Observer, C/C++ Users JournalSeptember 2003.

* Could not resist paying homage to Star Wars character Jabba the Hutt.

** As an anecdote, when learning about delegates in the first-ever Guerrilla .NET at DevelopMentor before the .NET framework was released, I asked the instructor, Brent Rector, what happened if the object receiving the delegate call threw an exception.  He tried it, and the program crashed.

Posted in design patterns, software | Leave a comment