Read bits between bytes

Question

I am trying to read bits from a buffer of bytes. So far masking and shifting has worked however I am not sure what to do about a value that appear between bytes, such as,

... 11001001 11101101 ... (read 8 bits from the buffer with 4 bit offset)
        |_______|

8 and 4 are not static, can differ, for example:

... 11001001 11101101 ... (read 7 bits from the buffer with 6 bit offset)
    ^     |______|       
    |
  Buffer

I have a starting offset, and a total of bits to read. However I have a char buffer, and I can't bit shift two bytes into one by using chars (Or maybe I can ?), so I thought of doing this:

uint16_t two_byte_buffer;
memcpy(&two_byte_buffer, real_buffer, 2); // May need to endian convert ?

And now I can bit-shift the 16-bit integer value to read the necessary 8 bits. Would this method work ? And is there a better method ?

Edit: So far I have used this method to extract bits from a single char.

Reading n-bits from a 32bit chunk

`And now I can bit-shift the 16-bit integer value to read the necessary 8 bits` - and you can't do it with `real_buffer`? How is `real_buffer` defined? `I can't bit shift two bytes into one by using chars` `uint8_t value = ((real_buffer[1] << 4) & 0xf0) | ((real_buffer[0] >> 4) & 0x0f)` ? — KamilCuk, Nov 22 '19 at 15:31
@KamilCuk A char is 8-bits, I don't need to shift a 8 bit value, I need to shift an at least 16-bit value to get the bits in the next char. `real_buffer` is just a `char*`, but it doesn't represent ascii values, just binary data. — Max Paython, Nov 22 '19 at 15:32
Writing to a `uint16_t` seems like the best method. Instead of `memcpy` I'd use `two_byte_buffer = (real_buffer[n] << 8) | real_buffer[n + 1]`. — Fiddling Bits, Nov 22 '19 at 15:35
@KamilCuk The offset and the read_count are variable, I would probably need a mask table to write what you written in the comment programmatically. — Max Paython, Nov 22 '19 at 15:36
Please clarify your question then. You want to write a function like `int extract(char *result, char *buffer, size_t offset_in_bits, size_t length_in_bits)` that would place in the buffer pointed to by result the bits extract from the buffer pointed to bu `buffer` from specified bit position in specified bits length? That is very different job (and a much much bigger one) then extracting just const 8 bits from a const 4 bits offset. So you said: `now I can bit-shift ...` and `So far masking and shifting has worked` - please post the code that worked. — KamilCuk, Nov 22 '19 at 15:37
@KamilCuk The buffer is not being pointed to by a bit index. It's first_byte + bit_offset — Max Paython, Nov 22 '19 at 15:45

score 1 · Answer 1 · answered Nov 22 '19 at 15:48

How about the following:

#include <stdint.h>
#include <stdio.h>

#define BIG_ENDIAN_BUFFER

#ifdef BIG_ENDIAN_BUFFER
  #define EXTRACT(BUFFER, OFFSET) ((((BUFFER[0] << 8) | BUFFER[1]) >> (8 - OFFSET)) & 0xFF)
#else
  #define EXTRACT(BUFFER, OFFSET) ((((BUFFER[1] << 8) | BUFFER[0]) >> (8 - OFFSET)) & 0xFF)
#endif

int main(void)
{
    uint8_t real_buffer[] = {0xC9, 0xED};

    printf("0x%02X, 0x%02X, 0x%02X\n", real_buffer[0], real_buffer[1], EXTRACT(real_buffer, 0));
    printf("0x%02X, 0x%02X, 0x%02X\n", real_buffer[0], real_buffer[1], EXTRACT(real_buffer, 1));
    printf("0x%02X, 0x%02X, 0x%02X\n", real_buffer[0], real_buffer[1], EXTRACT(real_buffer, 2));
    printf("0x%02X, 0x%02X, 0x%02X\n", real_buffer[0], real_buffer[1], EXTRACT(real_buffer, 3));
    printf("0x%02X, 0x%02X, 0x%02X\n", real_buffer[0], real_buffer[1], EXTRACT(real_buffer, 4));
    printf("0x%02X, 0x%02X, 0x%02X\n", real_buffer[0], real_buffer[1], EXTRACT(real_buffer, 5));
    printf("0x%02X, 0x%02X, 0x%02X\n", real_buffer[0], real_buffer[1], EXTRACT(real_buffer, 6));
    printf("0x%02X, 0x%02X, 0x%02X\n", real_buffer[0], real_buffer[1], EXTRACT(real_buffer, 7));
    printf("0x%02X, 0x%02X, 0x%02X\n", real_buffer[0], real_buffer[1], EXTRACT(real_buffer, 8));

    return 0;
}

When BIG_ENDIAN_BUFFER is defined:

0xC9, 0xED, 0xC9
0xC9, 0xED, 0x93
0xC9, 0xED, 0x27
0xC9, 0xED, 0x4F
0xC9, 0xED, 0x9E
0xC9, 0xED, 0x3D
0xC9, 0xED, 0x7B
0xC9, 0xED, 0xF6
0xC9, 0xED, 0xED

When BIG_ENDIAN_BUFFER is NOT defined:

0xC9, 0xED, 0xED
0xC9, 0xED, 0xDB
0xC9, 0xED, 0xB7
0xC9, 0xED, 0x6E
0xC9, 0xED, 0xDC
0xC9, 0xED, 0xB9
0xC9, 0xED, 0x72
0xC9, 0xED, 0xE4
0xC9, 0xED, 0xC9

Thanks you for your answer, just to inform, the value being tried to extracted can be smaller than 8 bits. — Max Paython, Nov 22 '19 at 15:53

ilmiacs · Answer 2 · 2019-11-22T16:16:01.960

You don't need memcpy to 16 bit to be able to extract. Try this, it works on arbitrary length char arrays. It allocates sufficient memory to hold the extracted bits.

#import <stdlib.h>
#import <stdio.h>
#import <string.h>

char buffer[] = "whatever";
int offset_bits = 12;
int count_bits = 34;
char* result;

void print_bits(char* buf, int nbits){
    for (int i = 0; i<nbits; i++){
        printf("%i",(buf[i/8]>>(i%8))&1);
    }
    printf("\n");
}

int main(){
    int offset_bytes = offset_bits/8;
    int shift_bits = offset_bits%8;
    int count_bytes = (count_bits-1)/8+1;
    result = (char*) malloc( sizeof(char)*count_bytes );
    for(int i=0; i<count_bytes ;i++){
        result[i] = (buffer[i+offset_bytes]>>shift_bits) ^ (buffer[i+offset_bytes+1]<<(8-shift_bits));
    }
    print_bits(buffer, 8*strlen(buffer));
    for(int i=0; i<offset_bits ;i++){printf(" ");}
    print_bits(result, count_bits);

    return 1;
}

This should output

1110111000010110100001100010111010100110011011101010011001001110
            0110100001100010111010100110011011

I have aligned the extracted bits (34 in this example) by the offset, so one can visually check the result.

0___________ · Answer 3 · 2019-11-22T16:37:23.960

1

#include <stdint.h>
#include <stdio.h>

uint32_t extractBytes(void *val, unsigned startBit, unsigned nBits)
{
    uint32_t *v32 = val;
    uint32_t mask = (1 << nBits) - 1;

    return (*v32 >> startBit) & mask;
}

int main()
{
    uint32_t x = 0b111101000;

    printf("0x%x\n", extractBytes(&x, 3, 4));
}

https://godbolt.org/z/vqfMDK

for 64 bits just change the types

edited Nov 22 '19 at 16:37

answered Nov 22 '19 at 16:09

0___________

60,014
4
34
74

This supports a maximum of 32 bits. OP seeks a general solution for arbitrary length arrays. – ilmiacs Nov 22 '19 at 16:22
general solution for 64bits+ is not practiacally needed – 0___________ Nov 22 '19 at 16:37
How can you tell without knowing about the use case? – ilmiacs Nov 22 '19 at 16:41
@ilmiacs because it is easly done with smaller chunks and C does not support any data >64bits at this moment – 0___________ Nov 22 '19 at 17:07

Read bits between bytes

3 Answers3