encryption and decryption with openssl AES on Linux - linux

Question

I want to use AES to communicate tcp/ip. However, difficulties arose in making the AES function.
In the process of decoding, dummy values are generated or the values are deleted.
I'd appreciate it if you could give me a little help.
int main(void)
{
unsigned char mykey[] = "01234567890123456789012345678\0";
unsigned char iv[] = "0123456789012\0";
char buf[BUF_SIZE]="hi";
char enc[BUF_SIZE];
char dec[BUF_SIZE];
AES_encryption(buf,enc,mykey,iv);
AES_decryption(enc,dec,mykey,iv);
printf("buf : %s\n",buf);
printf("enc: %s\n",enc);
printf("dec: %s\n", dec);
return 0;
}
void AES_encryption(char plainfn[], char cipherfn[], unsigned char key[],unsigned char iv[])
{
EVP_CIPHER_CTX ctx;
int in_len, out_len=0;
in_len=strlen(plainfn);
EVP_CIPHER_CTX_init(&ctx);
EVP_CipherInit_ex(&ctx,EVP_aes_128_cbc(),NULL,key,iv,AES_ENCRYPT);
EVP_CipherUpdate(&ctx,cipherfn,&out_len,plainfn,in_len);
EVP_CipherFinal_ex(&ctx,cipherfn,&out_len);
EVP_CIPHER_CTX_cleanup(&ctx);
}
void AES_decryption(char cipherfn[], char plainfn[], unsigned char key[], unsigned char iv[])
{
EVP_CIPHER_CTX ctx;
int in_len, out_len=0;
in_len=strlen(cipherfn);
EVP_CIPHER_CTX_init(&ctx);
EVP_CipherInit_ex(&ctx,EVP_aes_128_cbc(),NULL,key,iv,AES_DECRYPT);
EVP_CipherUpdate(&ctx,plainfn,&out_len,cipherfn,in_len);
EVP_CipherFinal_ex(&ctx,plainfn,&out_len);
EVP_CIPHER_CTX_cleanup(&ctx);
}
These results come out.
buf : hi
enc: U▒▒B▒ac▒▒]▒▒▒▒Y▒-
dec: hi▒?!▒

Answer 1

The main problem is that AES_encryption is likely put NULL chars to the enc buffer. You then count the scrambled enc buffer "string length" via strlen() in AES_decryption. This is certainly wrong since decryption can stop too early thus not reading the entire input buffer.
You should probably pass an buffer size argument to encrypt and decrypt functions to properly encrypt/decrypt the buffer(s).
Calculate the string length before encryption and some how pass the same buffer length also to the decryption stage. You probably have to encode the string length in your buf before the actual data.
Also since enc buffer is by definition scrambled you can't just printf("%s",enc) it for the same reason strlen() doesn't work for it. You need to print the chars one by one by putchar() or some other way that is immune to null chars.