integer overflow c | buffer overflow in c | buffer overflow example

C grants the developer full control over the memory management and because of this buffer overflow scenarios are very common. Either it is the usage of unsafe methods or developer’s mistakes to calculate the length of a buffer, the damage will be the same. Most of buffer overflow vulnerabilities are related to the wrong calculation of a buffer length or an offset. These issues are attributed to the C’s Integer characteristics which remain hidden to the developers. A professional code auditor should be able to detect such logic and sometimes complex vulnerabilities. In this article first I am going to talk about root causes of buffer overflows (mainly related to integers) and then provide several source code examples.

Most of novice hackers are familiar with the term buffer overflow. However a naïve viewpoint is that, developers’ usage of unsafe methods such as strcpy is the main cause for such vulnerabilities. Well maybe 15 years ago this perspective was right but over time codes have become more secure and hackers found new ways to leverage a bug! In recent years a great number of vulnerabilities were logic and complex vulnerabilities related to integer overflows, integer boundary issues and integer conversions. These issues were vulnerabilities because they explicitly cause a buffer overflow.

After you review the vulnerabilities I introduce in this article you probably know how an attacker leverage an integer issue such as an integer overflow but if you look for some preconditions here are a couple of conditions that may lead to a vulnerability:

  • Either a dynamic allocation(from the heap using malloc) or a reserved amount allocation(from the stack using variable definition) is performed
  • An input which is controllable from user is going to be read into the buffer
  • A calculation for the amount of the buffer to be allocated or size of the data to be copied is being performed (either using a length variable from the user or an implicit code to do so)

The real issue is the number passing for the copy operation (for example input to the memset) being bigger than the size of the buffer or sometimes the malloc’s size argument being lesser than the real value. The first two mentioned conditions are common between all memory management modules and you should have spotted them in your pre-assessment phase. Out of all memory management modules those which have the 3td condition are better to start with because they are likely to have integer overflow issues. For example a simple length +1 addition is all that may cause an buffer overflow.

Integer overflow c

An integer overflow either leads to a negative number or to a much smaller number. Integer overflows by themselves are not vulnerability but most of time they result in a buffer overflow because the calculation for buffer allocation becomes incorrect. For the first example let’s examine an OpenSSL 0.9.6l buffer overflow example (The code is taken from The Art of Software Security Assessment: Identifying and Preventing Software Vulnerabilities book) because of a simple addition:

The Art of Software Security Assessment (Listing 6-6)

 

c.inf=ASN1_get_object(&(c.p),&(c.slen),&(c.tag),&(c.xclass),

                      len-off);

 

...

{

    /* suck in c.slen bytes of data */

    want=(int)c.slen;

    if (want > (len-off))

    {

        want-=(len-off);

        if (!BUF_MEM_grow(b,len+want))

        {

            ASN1err(ASN1_F_ASN1_D2I_BIO,

                    ERR_R_MALLOC_FAILURE);

            goto err;

        }

        i=BIO_read(in,&(b->data[len]),want);

 

To see the vulnerability consider c.slen to be 2147483647, len= 200(number of bytes already read), off=50(the number of bytes that are parsed). The first If statement passes because (2147483647>150) and the Grow-Buffer at line 21 does not grow the buffer because the len+want=-2147483449 and the last line writes 2147483647 bytes into in buffer! 

Although a simple addition of two big numbers can cause an integer overflow, addition is not the only cause; a multiplication is another source to cause integer overflow. In C for dynamic allocation malloc function is used. This function needs the number of bytes to be allocated as a parameter. To calculate the size, developers use:

bufferSize = length * [Type’s size]

If all of the variables are of type int32 and the user is able to specify a large length value, this multiplication can cause an integer overflow. Consider a network packet that contains a header field to indicate the length parameter. If the user specifies 1073741825 for this length value, on a 32 bit system that the type is int32 and 4 bytes, this multiplication leads to an integer overflow. This integer probably leads to a buffer overflow vulnerability because the bufferSize is 4 and malloc allocates 4 bytes. But the packet reader reads length parameter which leads to writing 1073741825 (4 bytes) after the 4th byte of the buffer! A huge buffer over flow!

Integer Conversion issues

A conversion between types can be either result of an explicit conversion or an implicit compiler conversion hidden from developer’s eyes. Explicit conversion vulnerabilities are mainly the result of the assignment of a smaller signed integer (short type) to an unsigned int. In this scenario a negative number such as -1 can become the biggest unsigned int! In these scenarios the memory is very small than the write operation. Another common explicit conversion is the conversion of an unsigned integer to its sign type. In this case a big number becomes a small negative number and probably bypass the following boundary length check. Implicit conversions mostly happen during operations; in fact every smaller type than int and unsigned int become int in arithmetic operations! A typical issue is when an integer value and an unsigned int are in an arithmatic operation. In this case the integer automatically is converted to an unsigned int! This is exactly why safe methods such as snprintf(), strncpy(), memcpy(), read(), or strncat() cannot be safe at all at certain situations. These functions accept an unsigned int as the “n” parameter to copy and passing a negative number ends up becoming a very large positive number.

Always remember that char and short types can be negative so passing these types as the parameter to the methods working with size_t can lead to a potential vulnerability. Look at a buffer overflow example in AntiSniff tool vulnerability from packetstormssecurity:

The Art of Software Security Assessment (adopted from listing 6-8)
  char *indx;
  unsigned int count;
  char nameStr[MAX_LEN]; //256
...
  memset(nameStr, '\0', sizeof(nameStr));
...
  indx = (char *)(pkt + rr_offset);
  count = (char)*indx;

  while (count){
    if (strlen(nameStr) + count < ( MAX_LEN - 1) ){
      (char *)indx++;
      strncat(nameStr, (char *)indx, count);
      indx += count;
      count = (char)*indx;
      strncat(nameStr, ".",
              sizeof(nameStr)  strlen(nameStr));
    } else {
      fprintf(stderr, "Alert! Someone is attempting "
                      "to send LONG DNS packets\n");
      count = 0;
    }

 }
 nameStr[strlen(nameStr)-1] = '\0';

 

When there is a conversion from the same-length unsigned type to the sign, there is a fat chance that an integer overflow vulnerability exists. In this case the large lengths are seen as a sign integer after the conversion and the following boundary checking is easily bypassed. It is worth to mention that many of these buffer overflow issues can be identified easier by a fuzz testing than by a source code analysis. For example let’s examine following piece of code:count is unsigned int which is correct, indx is used  to read characters including the count of next block. Indx is a char type and it makes sense, right? Well it doesn’t make sense since a char is signed and one can pass -1 as length. The -1 then will be converted (by dereferencing indx in line 7) to the hugest 32 bit integer and it passed the while condition. Then the if-statement at line 11 is going to prevent a buffer overflow attacks but since a small number (strlen(nameStr)) is added to a big integer (count) an integer overflow takes place and the result is less than MAX_LEN and the control flow goes to the copy statement in line 13. Here the big count number is passed as length argument and a buffer overflow happens. The If-Statement is an enhancement to the previous vulnerable version that there was no protection mechanism at all. But as you see the real problem is the conversion of a signed byte to an unsigned integer. Making both namestr and indx variables unsigned fix the problem of this buffer overflow example.

The Art of Software Security Assessment (Listing 6-12)

 

unsigned short read_length(int sockfd)

{

    unsigned short len;

 

    if(full_read(sockfd, (void *)&len, 2) != 2)

        die("could not read length!\n");

 

    return ntohs(len);

}

 

int read_packet(int sockfd)

{

    struct header hdr;

    short length;

    char *buffer;

 

    length = read_length(sockfd);

 

    if(length > 1024){

        error("read_packet: length too large: %d\n", length);

        return 1;

    }

 

    buffer = (char *)malloc(length+1);

    if((n = read(sockfd, buffer, length) < 0){

        error("read: %m");

        free(buffer);

        return 1;

    }

 

    buffer[n] = '\0';

 

    return 0;

}

 

In line 33 a simple conversion from unsigned short (result of read_length) to the signed short makes a big number like 65535 negative (-1) and therefor the length > 1024 check is bypassed. After that the malloc function surprisingly allocates 0 byte because the addition makes an integer overflow and the parameter of malloc becomes 0. 

While auditing it is important not to be misled by a complex algorithm and then leave it. One of the approaches to overcome the complexity is to look over a source code several times, and following just one goal in each iteration. To find integer conversion issues you should pay attention to variable definitions and the following assignment for each variable. If you find an assignment that the right side’s type is different than the left side then you should check different test cases. Let’s examine an integer overflow example from a vulnerability in SSH:

The Art of Software Security Assessment (adopted from listing 6-19)
/* Detect a crc32 compensation attack on a packet */
int
detect_attack(unsigned char *buf, u_int32_t len,
              unsigned char *IV)
{
    static u_int16_t *h = (u_int16_t *) NULL;
    static u_int16_t n = HASH_MINSIZE / HASH_ENTRYSIZE;
    register u_int32_t i, j;
    u_int32_t l;
    register unsigned char *c;
    unsigned char *d;

    if (len > (SSH_MAXBLOCKS * SSH_BLOCKSIZE) ||
        len % SSH_BLOCKSIZE != 0) {
        fatal("detect_attack: bad length %d", len);
    }
for (l = n; l < HASH_FACTOR(len / SSH_BLOCKSIZE); l = l << 2)
       ;
if (h == NULL) {
    debug("Installing crc compensation "
          "attack detector.");
    n = l;
    h = (u_int16_t *) xmalloc(n * HASH_ENTRYSIZE);
} else {
    if (l > n) {
        n = l;
        h = (u_int16_t *)xrealloc(h, n * HASH_ENTRYSIZE);
    }
}
if (len <= HASH_MINBLOCKS) {
        for (c = buf; c < buf + len; c += SSH_BLOCKSIZE) {
            if (IV && (!CMP(c, IV))) {
                if ((check_crc(c, buf, len, IV)))
                    return (DEATTACK_DETECTED);
                else
                    break;
            }
            for (d = buf; d < c; d += SSH_BLOCKSIZE) {
                if (!CMP(c, d)) {
                    if ((check_crc(c, buf, len, IV)))
                    return (DEATTACK_DETECTED);
                else
                    break;
               }
           }
       }
       return (DEATTACK_OK);
    }
memset(h, HASH_UNUSEDCHAR, n * HASH_ENTRYSIZE);

    if (IV)
        h[HASH(IV) & (n - 1)] = HASH_IV;

    for (c = buf, j = 0; c < (buf + len); c += SSH_BLOCKSIZE, j++) {
        for (i = HASH(c) & (n - 1); h[i] != HASH_UNUSED;
             i = (i + 1) & (n - 1)) {
            if (h[i] == HASH_IV) {
                if (!CMP(c, IV)) {
                    if (check_crc(c, buf, len, IV))
                        return (DEATTACK_DETECTED);
                    else
                        break;
                 }
             } else if (!CMP(c, buf + h[i] * SSH_BLOCKSIZE)) {
                 if (check_crc(c, buf, len, IV))
                     return (DEATTACK_DETECTED);
                 else
                     break;
             }
          }
          h[i] = j;
       }
    return (DEATTACK_OK);
}

 

 This is fairly a complex code and finding the vulnerability in the first glance is hard but you should concentrate on your goal. For example from the memset (line 66) we suspect a buffer overflow vulnerability and our goal is to check the code for any buffer overflow vulnerabilities. In one of the iterations we check the code for type conversion issues and for that we check variable definition and following assignments. After looking at the lines 6 to 12 we should focus on finding any assignments between u_int16_t variables (n and h) and u_int32_t variable (l). We quickly find a conversion in line 26 and further analysis shows that by sending a packet of size 262,144 we can cause a buffer overflow. Because the n variable is used for buffer allocation and the buffer will be used for saving network data in line 71.

While auditing always check operations that involve size_t (or unsigned int32) type which is the result type of most libc functions.  Any operations involving this variable leads to an unsigned integer type and so any implication to consider the result of operations as signed integer is incorrect. A probable mistake is using a size_t type in subtraction and then comparing the result with a negative value! Comparisons that are supposed to protect an allocation or an indexing are also candidate statements to look for vulnerabilities.  So always check these kinds of comparisons and have mind that unsigned values cannot be negative!

C vulnerable functions

As a rule of thumb you always should examine the possibility of passing negative numbers for read(), recvfrom(), memcpy(), memset(), bcopy(), snprintf(), strncat(), strncpy(), and malloc() methods. You should have seen subtle vulnerabilities in this article and by now you should be convinced that at certain situations these functions cannot be safe at all. The real problem is the calculation of the buffer these functions use and you should closely check them.

Also pay careful attention to the usage of sizeof() operator. Remember that the result of sizeof is unsigned int and operations including such result do not produce negative values. sizeof() operator should be closely examined in buffer allocation functions. One of the sizeof issues is its usage on a buffer pointer value, expecting it to return the size of buffer; the result of sizeof on a pointer type is the size of variable. For example result of sizeof on c (a char pointer) is 4! Another mistake is usage of sizeof to guard an allocation. For example:

int buf[1024];

int *b=buf;

 

while (havedata() && b < buf + sizeof(buf))

{

    *b++=parseint(getdata());

}

 

Here buf + sizeof(buf) does not guard the loop from buffer overflow because the loop executes 4092 times not 1024! This is because ++ on pointer advances the pointer to the next element not next byte.

Read 1641 times Last modified on Thursday, 18 June 2015 07:42
Rate this item
0
(0 votes)
More in this category: « Vulnerability analysis
About Author
Leave a comment

Make sure you enter the (*) required information where indicated. HTML code is not allowed.

Advanced Programming Concepts
News Letter

Subscribe our Email News Letter to get Instant Update at anytime