Last modified 2024-11-04

Understanding the Standard - part 1

What the words mean

The ISO C Standard, and the ANSI C Standard before it, has been available for some years now, and it is now possible to purchase it quite cheaply. However, this has meant that suddenly many more people are having to come to grips with actually understanding it. Reading the C Standard is not an easy task. The wording has to be read carefully, and you always have to be remembered that certain words have special technical meanings, and may not mean what you think they do. Some apparently simple questions are only answered by assembling pieces from several different sections. And finally, sometimes the Standard just doesn't give an answer, but leaves it up to "Quality of Implementation".

With all this in mind, your editor has asked me to put together a guide to some of the commonest pitfalls in Understanding the Standard. This first article explains how to tell what kind of answer you're being given; what the words mean.

You've got it wrong

When your programs are wrong, the Standard doesn't say "you ought to get an error message". Instead, it talks about "diagnostics". Whenever the Standard requires the compiler to give you an error message, it says that "a diagnostic is required". A diagnostic is Standardese for a certain kind of error message; your compiler manual should tell you how to tell what error messages are diagnostics, and a good compiler will tell you where in the Standard to look to see what you've done wrong.

However, there's one big loophole in this. The compiler has to generate a diagnostic for the errors described by the Standard, but it is also allowed to generate them for other errors, and to warn you about dubious practices, or even just to annoy you! So as well as:

Error 42: octal constant contains 8 or 9

Error 176: variable "x" used before it is initialized

Error 833: spelling mistake in comment

However, you will always get an error message when the Standard requires one. There are two kinds of place that this happens.

Firstly there are the syntax sections of the Standard. These describe what the text of a program looks like. For example, the syntax section of 6.1.3.2 describes integer constants. Two forms of these are decimal and octal constants. The syntax for these is:

    decimal-constant:
        non-zero-digit
        decimal-constant digit

    octal-constant:
        0
        octal-constant octal-digit

This says that a decimal constant can contain any digit from 0 to 9, but cannot begin with a zero, while an octal constant can only contain digits from 0 to 7, but must begin with a zero. Now "081" is not a decimal constant, because it begins with a zero, but is not an octal constant, because it contains an 8. This means that it is not a constant at all, but is a "syntax error", and so a diagnostic will be generated.

The other sort of place that promises you a diagnostic are the sections of the Standard labelled "Constraints". These sections describe things that the Standard requires you to do or to not do when you write your program. For example, section 6.3.3.3 has a constraint that reads:

The operand of the unary + or - operator shall have arithmetic type; of the ~ operator, integral type; of the ! operator, scalar type.

This means that, for example, you can write -5.2 but not ~5.2, and (!&x) but not (+&x). If you do write either of the second forms, you have done something that a constraint has said you shall not do, and so you are promised a diagnostic. However, note that this promise only applies to the constraint sections. There are plenty of other "shall"s in the Standard, but if you disobey them, you won't necessarily get an error message, though the compiler writer might be kind and give you one.

I'm not telling you

There are lots of times when the authors of the Standard decided not to make a decision. This may seem odd, but it is actually the result of attempting to strike a balance between two opposing viewpoints. On the one hand, the application programmer (that's you) wants to know exactly what will happen when you write something. You know that 2+2 is always 4, and you want 7/-2 to always be -3. On the other hand, the compiler writers want lots of freedom; on one machine the divide instruction rounds towards zero, so 7/-2 is indeed -3, while on another machine it rounds downwards, so 7/-2 is -4 instead. In each case, if the Standard required one value, the compiler on the other machine would have to produce lots of extra code to get the right answer, and this against the "spirit of C". So the Standard leaves it up to the compiler which answer it generates, and you must be ready to handle either case. In this case, however, it does give you one piece of assistance: the manual for the compiler must tell you which choice it made.

Division is one example where the Standard doesn't answer your question; there are dozens more. However, in their wisdom, the authors decided that sometimes you should get part of an answer, and the most important ability you need in reading the Standard is to know how much of an answer you are getting.

There are three terms that appear time and time again in the Standard and that tell you this. These are implementation-defined, unspecified, and undefined, and each represents a different level of detail in what the Standard is telling you.

If something is implementation-defined, then there are several possible ways of doing something, and the compiler writer only has to choose one; he or she does, however, have to tell you in the manual what they chose. We have seen one example already: integer division. If the result of a division is exact the compiler must get it correct, and if both operands are positive it must round it down. However, if either operand is negative the compiler can choose whether to round up or to round down. Thus 6 / 2 == 3, 7 / 2 == 3, and 6 / -2 == -3, but 7 / -2 can be either -3 or -4. Which it is will depend on your compiler, and in the manual you should find a statement like "Inexact division of integers which are not both positive will be rounded towards the more negative choice", meaning that this compiler will produce -4.

Another example of something that is implementation-defined is the number of bits in a byte, or equivalently a variable of type char. The Standard requires that there be at least 8, but does not put any other limitation on it. Since it is implementation-defined, the actual value must be documented, and in this case it can also be found by looking at the symbol CHAR_BITS in the header <limits.h>.

There are 76 different items of implementation-defined behaviour described by the standard; they are listed in annex G.3. Some of them are shown in the box (together with references to where in the standard they are described). You may find it instructive to see how well your favourite compiler documents them; you might get some interesting surprises.

When you come across something that is implementation-defined, you should take care not to rely on the value of any specific compiler. Doing so will almost certainly lead you into problems at some point in the future, either because you need to move your code to another machine, or because the compiler has changed between releases. Instead, take a little more care and write your code to avoid the problem. For example, never assume that a byte holds exactly 8 bits. Instead, write "defensive code" that works for any size of byte. This can be done in any of three ways, and the right choice will depend on exactly what you're doing.

Suppose you need room to hold N bits. The first approach is to use CHAR_BITS to gain exactly the right amount of space:

    unsigned char n_bits [(N + CHAR_BITS - 1) / CHAR_BITS];

This has the advantage that all the bits of n_bits are actually used, and no space is wasted. In turn this means that bits shifted into the left end of a variable will definitely be zero; this may make the code slightly simpler.

The second approach is to use the guarantee of 8 bits per byte:

    unsigned char n_bits [(N + 7) / 8];

The final approach is not so good, but can be used where the code is initially developed on a system with 8 bits per byte. At some point in the code, write:

    #if CHAR_BITS != 8
    #error This code assumes 8 bits per byte and needs rewriting.
    #endif

If you ever port the code to a machine with more than 8 bits per byte, the #error will cause the compiler to complain, and you will then have to rewrite the code. This is only worth it if you think that the rewriting is unlikely, and you can afford the time to do it when it is; rewriting will be more expensive than doing it right in the first place.

When the standard says that something is unspecified, this is rather similar to implementation-defined, except that the choice made doesn't have to be documented, and can change as the program runs. Whenever possible, you should avoid unspecified behaviour. If you can't, you should ensure that you allow for every possibility.

For example, the definition of the qsort function states that, if two elements of the array compare equal, their order in the sorted array is unspecified. What this implies is that if you sort the same array twice, you might not get the same result if any two elements compare equal. [If you're unfamiliar with qsort, this might not make sense to you. The qsort function takes as one of its arguments a comparison routine, which can compare the elements of an array. Suppose that the elements are pointers to a complex structure; the routine might only look at some fields of the structure, and ignore others. Elements that differ only in the ignored fields will compare as equal.] Therefore you must not assume that the result will be the same. If it matters, you should modify the comparison routine to distinguish elements that used to be identical.

Note that it is not sufficient to ensure that the comparison routine never returns "equal". If two elements sometimes compare one way and sometimes the other, the qsort function might never terminate. Instead you must actually distinguish them. If the elements of the array are pointers to a structure, then you could add an extra field for this purpose. When creating the array, set this field in every element to zero. Whenever two elements would otherwise compare equal, change any zero fields to unique values, and then compare them. An example of this is shown here:

    /* Here's the definition of the structure */
    struct info
    {
        char * name;  /* We'll sort the array by name */
        /* Insert other fields here that we'll ignore when sorting */

        /* This is the extra field used to ensure things sort correctly.
           Ensure that it is always initialized to zero. */
        unsigned int extra;
    };

    /* Here's the array we're going to sort */
    struct info *array [100];

    /* Here's the comparison routine */
    int compare_info (const void *p1, const void *p2)
    {
        /* The arguments point to array elements, so they are (struct info **) */
        struct info *ip1 = *(struct info **) p1;
        struct info *ip2 = *(struct info **) p2;
        /* Compare the name fields */
        int r = strcmp (ip1->name, ip2->name);

        /* Now deal with elements that compare equal */
        if (r == 0)
        {
            static unsigned int last_extra = 0;

            if (ip1->extra == 0)
                ip1->extra = ++last_extra;
            if (ip2->extra == 0)
                ip2->extra = ++last_extra;
            r = ip1->extra < ip2->extra ? -1 : 1;
        }
        return r;
    }

The last of the three special terms is undefined. This is rather harder, because there are three different ways that it is described in the standard:

• The standard says that something is undefined. For example, talking about the / and % operators:

  if the value of the second operand is zero, the behaviour is undefined.

• The standard says that something "shall" or "shall not" happen, and you do the opposite (however, if the "shall" is in a Constraint section, then this requires a diagnostic instead, as I described earlier). For example, subclause 6.5.1 says:

  The declaration of an identifier for a function that has block scope shall have no explicit storage-class specifier other than extern.

  What this means is that the following code is undefined:

           int f_twice (void)
           {
               static int f (void);  /* This is undefined */
  
                   f ();
                   f ();
           }
       

  because the declaration of f had an explicit storage class (static) other than extern.
• The standard doesn't say what happens. This is probably the hardest to detect, and even the experts get confused sometimes. For example, subclause 6.1.2.1 says:

  An identifier is visible (i.e. can be used) only within a region of program text called its scope.

  and then goes on to explain the different kinds of scope. However, it never says what happens if the identifier is used outside its scope. Since it doesn't say, whatever happens is undefined.

Okay, so now we've seen how to tell when something is undefined, what does it actually mean? Well, the answer is: absolutely anything! You can get the right result, or the wrong one, or a totally nonsensical one. Your program can stop running, or it can jump to a random point. The computer can delete some of your files, or just crash. It's even allowed to phone up Strategic Air Command and ask them to drop an ICBM on you. As I said: absolutely anything.

People read the previous paragraph and ask "Why? Why does the standard allow anything? Why doesn't it limit it to something sensible?" Well, that's an interesting question. Let's look at one particular example of undefined behaviour: arithmetic overflow. Most of you are used to machines with two's-complement and no error checking, and on these machines the result just has the top bits thrown away. So on a 16-bit system, 32760+10 is -32766. This is often a sensible thing to do: it's easy to implement, fast, and can sometimes generate the right result anyway (32760+10-42 will generate the correct result of 32728). However, it's not the only solution. Floating-point arithmetic often uses "infinities"; if a result is larger than the maximum value that can be represented, it is set to plus or minus infinity. If you add any finite number to infinity, the result is still infinity. So an infinity in the answer is a sign that something overflowed. A third approach is the "trap"; the hardware or software detects the overflow and causes something special to happen. Perhaps your program is sent a signal, which it can catch and handle, or perhaps a warning is written out on the console. On the first computer I ever used, a floating-point overflow caused the whole computer to stop and a light to flash; the operator (we weren't allowed to touch it ourselves) had to write down all the information displayed on the control panel and then press either the RESTART button or the NEXT JOB button, depending on what instructions we had left him.

Then there are more esoteric possibilities. Some early computers didn't have fixed size registers, but instead the variables in memory were separated by markers (rather like strings are in C). If the result of a calculation got too big, it might overwrite the contents of another variable; one that is nothing to do with the calculation. This would be very hard to track down (you can do much the same thing in C by messing around with pointers). A similar effect was put to use by the author of the Internet Worm; now there's an example of undefined behaviour at its most dangerous.

Sometimes undefined behaviour can be put to good use. For example, a compiler writer can pick something that's undefined, and use it to implement a new feature. One example of this is printf: the effect of any % sequence not described by the standard is undefined. This means that extra options can be added, like one compiler I have used that had an automatic plural feature: %z would print an "s" unless the corresponding argument equalled 1, so:

    printf ("You scored %d point%z\n", score, score);

Good hunting

This article has acted as an introduction to the ISO C standard and how to read it. Hopefully it will guide you when you have to use the standard for real. In the meantime, the next article in this series will discuss some of the features of the standard that even the experts have trouble understanding.

Back to the intro.

Back to the C index.

Next article

Back to Clive's home page.