CS50/pset5/speller memory leak - memory-leaks

Can someone help me with this? What am I doing wrong? I don't exactly know what part of this I need to fix, I think it's somewhere in the unload function.
Here is my code:
// Implements a dictionary's functionality
#include <ctype.h>
#include <stdbool.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <strings.h>
#include "dictionary.h"
#define HASHTABLE_SIZE 10000
// Defines struct for a node
typedef struct node
{
char word[LENGTH + 1];
struct node *next;
}
node;
node *hashtable[HASHTABLE_SIZE];
// Hashes the word (hash function posted on reddit by delipity)
int hash_index(char *hash_this)
{
unsigned int hash = 0;
for (int i = 0, n = strlen(hash_this); i < n; i++)
{
hash = (hash << 2) ^ hash_this[i];
}
return hash % HASHTABLE_SIZE;
}
// Initializes counter for words in dictionary
int word_count = 0;
// Loads dictionary into memory, returning true if successful else false
bool load(const char *dictionary)
{
// Opens dictionary
FILE *file = fopen(dictionary, "r");
if (file == NULL)
{
return false;
}
// Scans dictionary word by word
char word[LENGTH + 1];
while (fscanf(file, "%s", word) != EOF)
{
// Mallocs a node for each new word (i.e., creates node pointers)
node *new_node = malloc(sizeof(node));
node *cursor;
node *tmp;
// Checks if malloc succeeded, returns false if not
if (new_node == NULL)
{
unload();
return false;
}
// Copies word into node if malloc succeeds
strcpy(new_node->word, word);
// Initializes & calculates index of word for insertion into hashtable
int h = hash_index(new_node->word);
// Initializes head to point to hashtable index/bucket
node *head = hashtable[h];
// Inserts new nodes at beginning of lists
if (head == NULL)
{
hashtable[h] = new_node;
word_count++;
}
else
{
new_node->next = hashtable[h];
hashtable[h] = new_node;
word_count++;
}
}
fclose(file);
return true;
}
// Returns true if word is in dictionary else false
bool check(const char *word)
{
// Creates copy of word on which hash function can be performed
int n = strlen(word);
char word_copy[LENGTH + 1];
for (int i = 0; i < n; i++)
{
word_copy[i] = tolower(word[i]);
}
// Adds null terminator to end string
word_copy[n] = '\0';
// Initializes index for hashed word
int h = hash_index(word_copy);
// Sets cursor to point to same address as hashtable index/bucket
node *cursor = hashtable[h];
// Sets cursor to point to same location as head
while (cursor != NULL)
{
// If strcasecmp returns true, then word has been found
if (strcasecmp(cursor->word, word_copy) == 0)
{
return true;
}
// Else word has not yet been found, advance cursor
else
{
cursor = cursor->next;
}
}
// Cursor has reached end of list, word not found in dictionary (misspelled)
return false;
}
// Returns number of words in dictionary if loaded else 0 if not yet loaded
unsigned int size(void)
{
return word_count;
}
// Unloads dictionary from memory, returning true if successful else false
bool unload(void)
{
node *head = NULL;
node *cursor = head;
// freeing linked lists
while (cursor != NULL)
{
node *temp = cursor;
cursor = cursor->next;
free(temp);
}
return true;
}
These are the results from check50:
:) dictionary.c, dictionary.h, and Makefile exist
:) speller compiles
:) handles most basic words properly
:) handles min length (1-char) words
:) handles max length (45-char) words
:) handles words with apostrophes properly
:) spell-checking is case-insensitive
:) handles substrings properly
:( program is free of memory errors
valgrind tests failed; rerun with --log for more information.
These are the results when I run valgrind:
==595== Memcheck, a memory error detector
==595== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==595== Using Valgrind-3.13.0 and LibVEX; rerun with -h for copyright info
==595== Command: ./speller texts/cat.txt
==595==
MISSPELLED WORDS
WORDS MISSPELLED: 0
WORDS IN DICTIONARY: 143091
WORDS IN TEXT: 6
TIME IN load: 1.39
TIME IN check: 0.00
TIME IN size: 0.00
TIME IN unload: 0.00
TIME IN TOTAL: 1.40
==595==
==595== HEAP SUMMARY:
==595== in use at exit: 8,013,096 bytes in 143,091 blocks
==595== total heap usage: 143,096 allocs, 5 frees, 8,023,416 bytes allocated
==595==
==595== 8,013,096 bytes in 143,091 blocks are still reachable in loss record 1 of 1
==595== at 0x4C2FB0F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==595== by 0x401175: load (dictionary.c:52)
==595== by 0x4009B4: main (speller.c:40)
==595==
==595== LEAK SUMMARY:
==595== definitely lost: 0 bytes in 0 blocks
==595== indirectly lost: 0 bytes in 0 blocks
==595== possibly lost: 0 bytes in 0 blocks
==595== still reachable: 8,013,096 bytes in 143,091 blocks
==595== suppressed: 0 bytes in 0 blocks
==595==
==595== For counts of detected and suppressed errors, rerun with: -v
==595== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
I'm really stuck on this, and I would greatly appreciate some help!!

You are correct, the problem is unload. Look at this sequence carefully:
node *head = NULL;
node *cursor = head;
// freeing linked lists
while (cursor != NULL)
What is the value of cursor when it first reaches the while?
hashtable is the head of the linked list. It needs to mentioned somewhere in the unload function.

Related

what does deallocation function in xv6's allocation function

in case 1 and 2, what does the deallocation function do in an allocation function?
case 1: if(mem == 0)
// does this condition mean physical memory has not space?
case 2: if(mappages(pgdir, (char*)a, PGSIZE, V2P(mem), PTE_W|PTE_U) < 0)
// does this condtion mean pagetable entry has not allocate in physical memory?
I attached the deallocation function and the allocation function.
reference:
https://github.com/fernandabonetti/xv6/blob/master/vm.c
int
allocuvm(pde_t *pgdir, uint oldsz, uint newsz)
{
char *mem;
uint a;
if(newsz >= KERNBASE)
return 0;
if(newsz < oldsz)
return oldsz;
a = PGROUNDUP(oldsz);
for(; a < newsz; a += PGSIZE){
mem = kalloc();
if(mem == 0){
cprintf("allocuvm out of memory\n");
deallocuvm(pgdir, newsz, oldsz);
return 0;
}
memset(mem, 0, PGSIZE);
if(mappages(pgdir, (char*)a, PGSIZE, V2P(mem), PTE_W|PTE_U) < 0){
cprintf("allocuvm out of memory (2)\n");
deallocuvm(pgdir, newsz, oldsz);
kfree(mem);
return 0;
}
}
return newsz;
}
int
deallocuvm(pde_t *pgdir, uint oldsz, uint newsz)
{
pte_t *pte;
uint a, pa;
if(newsz >= oldsz)
return oldsz;
a = PGROUNDUP(newsz);
for(; a < oldsz; a += PGSIZE){
pte = walkpgdir(pgdir, (char*)a, 0);
if(!pte)
a = PGADDR(PDX(a) + 1, 0, 0) - PGSIZE;
else if((*pte & PTE_P) != 0){
pa = PTE_ADDR(*pte);
if(pa == 0)
panic("kfree");
char *v = P2V(pa);
kfree(v);
*pte = 0;
}
}
return newsz;
}
allocuvm is a short of Allocate User Virtual Memory. This function is responsible to increase the user's virtual memory in a specific page directory.
There are indeed 2 cases where this function can fail:
Case 1: kalloc function failed. kalloc is a short of kernel allocation. This function is responsible to return an address of a new, currently unused, page in RAM. If it returns 0, that means there are no available unused pages currently.
Case 2: mappages function failed. This function is responsible of making the new allocated page to be accessible by the process who uses the given page directory by mapping that page with the next virtual address available in the page directory.
If this function fails that means it failed in doing so, probably due to the page directory being already full.
In both cases, allocuvm didn't managed to increase the user's memory to the size requested, Therefore, it is undoing all allocations until the point of failure, so the virtual memory will remain unchanged, and returns an error it self.

custom memory allocator - segfault

me and my friend are trying to develop custom memory allocator in linux ubuntu 16.04.
We got stuck because of an error, btw its our first time
that we are trying to code something like that so we are not the best debuggers the error is : Segmentation fault (core dumped)
and here is the code.
can anybody help us understand whats wrong ?
Thank you!
#include <unistd.h>
#include <string.h>
#include <pthread.h>
#include <stdio.h>
struct header_t {
size_t size;
unsigned is_free;
struct header_t *next; };
struct header_t *head = NULL, *tail = NULL;
pthread_mutex_t global_malloc_lock;
struct header_t *get_free_block(size_t size)
{
struct header_t *curr = head;
while(curr) {
/* see if there's a free block that can accomodate requested size */
if (curr->is_free && curr->size >= size)
return curr;
curr = curr->next;
}
return NULL;
}
void free(void *block)
{
struct header_t *header, *tmp;
/* program break is the end of the
process's data segment */
void *programbreak;
if (!block)
return;
pthread_mutex_lock(&global_malloc_lock);
header = (struct header_t*)block - 1;
/* sbrk(0) gives the current program break address */
programbreak = sbrk(0);
/*
Check if the block to be freed is the last one in the
linked list. If it is, then we could shrink the size of the
heap and release memory to OS. Else, we will keep the block
but mark it as free. */
if ((char*)block + header->size == programbreak) {
if (head == tail) {
head = tail = NULL;
} else {
tmp = head;
while (tmp) {
if(tmp->next == tail) {
tmp->next = NULL;
tail = tmp;
}
tmp = tmp->next;
}
}
/* sbrk() with a negative argument decrements the program break.
So memory is released by the program to OS. */
sbrk(0 - header->size - sizeof(struct header_t));
/* Note: This lock does not really assure thread
safety, because sbrk() itself is not really
thread safe. Suppose there occurs a foregin sbrk(N)
after we find the program break and before we decrement
it, then we end up realeasing the memory obtained by
the foreign sbrk(). */
pthread_mutex_unlock(&global_malloc_lock);
return;
}
header->is_free = 1;
pthread_mutex_unlock(&global_malloc_lock);
}
void *malloc(size_t size)
{
size_t total_size;
void *block;
struct header_t *header;
if (!size)
return NULL;
pthread_mutex_lock(&global_malloc_lock);
header = get_free_block(size);
if (header) {
/* Woah, found a free block to accomodate requested memory. */
header->is_free = 0;
pthread_mutex_unlock(&global_malloc_lock);
return (void*)(header + 1);
}
/* We need to get memory to fit in the requested block and header
from OS. */
total_size = sizeof(struct header_t) + size;
block = sbrk(total_size);
if (block == (void*) -1) {
pthread_mutex_unlock(&global_malloc_lock);
return NULL;
}
header = block;
header->size = size;
header->is_free = 0;
header->next = NULL;
if (!head)
head = header;
if (tail)
tail->next = header;
tail = header;
pthread_mutex_unlock(&global_malloc_lock);
return (void*)(header + 1);
}
void *calloc(size_t num, size_t nsize)
{
size_t size;
void *block;
if (!num || !nsize)
return NULL;
size = num * nsize;
/* check mul overflow */
if (nsize != size / num)
return NULL;
block = malloc(size);
if (!block)
return NULL;
memset(block, 0, size);
return block;
}
void *realloc(void *block, size_t size)
{
struct header_t *header;
void *ret;
if (!block || !size)
return malloc(size);
header = (struct header_t*)block - 1;
if (header->size >= size)
return block;
ret = malloc(size);
if (ret) {
/* Relocate contents to the new bigger block */
memcpy(ret, block, header->size);
/* Free the old memory block */
free(block);
}
return ret;
}
The problem occurred because the functions were not prototyped [decalred].
Once I added functions prototype. The code worked.
For more information about prototyping: http://www.trytoprogram.com/c-programming/function-prototype-in-c/
mutex variable should be initialized before using it for applying lock. your global_malloc_lock is not initialized.
you can't initialize mutex variable as of normal variable.
pthread_mutex_t global_malloc_lock = 0 ;// invalid .. you may thinking since it's it declared as global it's initialized with 0 which is wrong
Initialize the mutex variable by calling pthread_mutex_init() or using PTHREAD_MUTEX_INITIALIZER ;
for your code add this
pthread_mutex_t global_malloc_lock = pthread_mutex_t global_malloc_lock;

Building a Simple character device but device driver file will not write or read

I am trying to write a simple character device/LKM that reads, writes, and seeks.
I have been having a lot of issues with this, but have been working on it/troubleshooting for weeks and have been unable to get it to work properly. Currently, my module makes properly and mounts and unmounts properly, but if I try to echo to the device driver file the terminal crashes, and when i try to read from it using cat it returns killed.
Steps for this module:
First, I make the module by running make -C /lib/modules/$(uname -r)/build M=$PWD modules
For my kernel, uname -r is 4.10.17newkernel
I mount the module using sudo insmod simple_char_driver.ko
If I run lsmod, the module is listed
If I run dmesg, the KERN_ALERT in my init function "This device is now open" triggers correctly.
Additionally, if I run sudo rmmod, that functions "This device is now closed" KERN_ALERT also triggers correctly.
The module also shows up correctly in cat /proc/devices
I created the device driver file in /dev using sudo mknod -m 777 /dev/simple_char_driver c 240 0
Before making this file, I made sure that the 240 major number was not already in use.
My device driver c file has the following code:
#include<linux/init.h>
#include<linux/module.h>
#include<linux/fs.h>
#include<linux/slab.h>
#include<asm/uaccess.h>
#define BUFFER_SIZE 1024
MODULE_LICENSE("GPL");
//minor nunmber 0;
static int place_in_buffer = 0;
static int end_of_buffer = 1024;
static int MAJOR_NUMBER = 240;
char* DEVICE_NAME = "simple_char_driver";
typedef struct{
char* buf;
}buffer;
char *device_buffer;
static int closeCounter=0;
static int openCounter=0;
ssize_t simple_char_driver_read (struct file *pfile, char __user *buffer, size_t length, loff_t *offset){
int bytesRead = 0;
if (*offset >=BUFFER_SIZE){
bytesRead = 0;
}
if (*offset + length > BUFFER_SIZE){
length = BUFFER_SIZE - *offset;
}
printk(KERN_INFO "Reading from device\n");
if (copy_to_user(buffer, device_buffer + *offset, length) != 0){
return -EFAULT;
}
copy_to_user(buffer, device_buffer + *offset, length);
*offset += length;
printk(KERN_ALERT "Read: %s", buffer);
printk(KERN_ALERT "%d bytes read\n", bytesRead);
return 0;
}
ssize_t simple_char_driver_write (struct file *pfile, const char __user *buffer, size_t length, loff_t *offset){
int nb_bytes_to_copy;
if (BUFFER_SIZE - 1 -*offset <= length)
{
nb_bytes_to_copy= BUFFER_SIZE - 1 -*offset;
printk("BUFFER_SIZE - 1 -*offset <= length");
}
else if (BUFFER_SIZE - 1 - *offset > length)
{
nb_bytes_to_copy = length;
printk("BUFFER_SIZE - 1 -*offset > length");
}
printk(KERN_INFO "Writing to device\n");
if (*offset + length > BUFFER_SIZE)
{
printk("sorry, can't do that. ");
return -1;
}
printk("about to copy from device");
copy_from_user(device_buffer + *offset, buffer, nb_bytes_to_copy);
device_buffer[*offset + nb_bytes_to_copy] = '\0';
*offset += nb_bytes_to_copy;
return nb_bytes_to_copy;
}
int simple_char_driver_open (struct inode *pinode, struct file *pfile)
{
printk(KERN_ALERT"This device is now open");
openCounter++;
printk(KERN_ALERT "This device has been opened this many times: %d\n", openCounter);
return 0;
}
int simple_char_driver_close (struct inode *pinode, struct file *pfile)
{
printk(KERN_ALERT"This device is now closed");
closeCounter++;
printk(KERN_ALERT "This device has been closed this many times: %d\n", closeCounter);
return 0;
}
loff_t simple_char_driver_seek (struct file *pfile, loff_t offset, int whence)
{
printk(KERN_ALERT"We are now seeking!");
switch(whence){
case 0:{
if(offset<= end_of_buffer && offset >0){
place_in_buffer = offset;
printk(KERN_ALERT" this is where we are in the buffer: %d\n", place_in_buffer);
}
else{
printk(KERN_ALERT"ERROR you are attempting to go ouside the Buffer");
}
break;//THIS IS SEEK_SET
}
case 1:{
if(((place_in_buffer+offset)<= end_of_buffer)&&((place_in_buffer+offset)>0)){
place_in_buffer = place_in_buffer+offset;
printk(KERN_ALERT" this is where we are in the buffer: %d\n", place_in_buffer);
}
else{
printk(KERN_ALERT"ERROR you are attempting to go ouside the Buffer");
}
break;
}
case 2:{//THIS IS SEEK END
if((end_of_buffer-offset)>=0&& offset>0){
place_in_buffer = end_of_buffer-offset;
printk(KERN_ALERT" this is where we are in the buffer: %d\n", place_in_buffer);
}
else{
printk(KERN_ALERT"ERROR you are attempting to go ouside the Buffer");
}
break;
}
default:{
}
}
printk(KERN_ALERT"I sought %d\n", whence);
return place_in_buffer;
}
struct file_operations simple_char_driver_file_operations = {
.owner = THIS_MODULE,
.read = simple_char_driver_read,
.write = simple_char_driver_write,
.open = simple_char_driver_open,
.llseek = &simple_char_driver_seek,
.release = simple_char_driver_close,
};
static int simple_char_driver_init(void)
{
printk(KERN_ALERT "inside %s function\n",__FUNCTION__);
register_chrdev(MAJOR_NUMBER,DEVICE_NAME, &simple_char_driver_file_operations);
device_buffer = kmalloc(BUFFER_SIZE, GFP_KERNEL);
return 0;
}
static void simple_char_driver_exit(void)
{
printk(KERN_ALERT "inside %s function\n",__FUNCTION__);
unregister_chrdev(MAJOR_NUMBER, DEVICE_NAME);
kfree(device_buffer);
}
module_init(simple_char_driver_init);
module_exit(simple_char_driver_exit);
As I said before, this file makes properly with no errors or warnings.
However, currently if I try to echo to the device file
using: echo "hello world" >> /dev/simple_char_driver
The terminal I am using crashes
If I then reopen a terminal, and use: cat /dev/simple_char_driver
then the terminal returns killed.
I am completely lost as to what is going wrong, and I have been searching for a solution for a very long time without success. If anyone has any insight into what is going wrong, please let me know.
Edit: As a user below suggested, I removed all code from my read and write methods except for the printk and the return, to make sure the functions were being triggered.
When I then used echo, dmesg showed that the write printk was triggered, and the device(which I had had open) closed. When I then tried to cat the device file, dmesg showed that the device reopened, the "ready from device" printk showed up succesfully, and then the device closed again. However, echo did not actually find anything to read from the device file, despite my having echoed "Hello world" into it immediately before.
edit
Final functioning read and write functions are as follows:
ssize_t simple_char_driver_read (struct file *pfile, char __user *buffer, size_t length, loff_t *offset)
{
if (*offset > BUFFER_SIZE)
{
printk("offset is greater than buffer size");
return 0;
}
if (*offset + length > BUFFER_SIZE)
{
length = BUFFER_SIZE - *offset;
}
if (copy_to_user(buffer, device_buffer + *offset, length) != 0)
{
return -EFAULT;
}
*offset += length;
return length;
}
ssize_t simple_char_driver_write (struct file *pfile, const char __user *buffer, size_t length, loff_t *offset){
/* *buffer is the userspace buffer where you are writing the data you want to be written in the device file*/
/* length is the length of the userspace buffer*/
/* current position of the opened file*/
/* copy_from_user function: destination is device_buffer and source is the userspace buffer *buffer */
int nb_bytes_to_copy;
if (BUFFER_SIZE - 1 -*offset <= length)
{
nb_bytes_to_copy= BUFFER_SIZE - 1 -*offset;
printk("BUFFER_SIZE - 1 -*offset <= length");
}
else if (BUFFER_SIZE - 1 - *offset > length)
{
nb_bytes_to_copy = length;
printk("BUFFER_SIZE - 1 -*offset > length");
}
printk(KERN_INFO "Writing to device\n");
if (*offset + length > BUFFER_SIZE)
{
printk("sorry, can't do that. ");
return -1;
}
printk("about to copy from device");
copy_from_user(device_buffer + *offset, buffer, nb_bytes_to_copy);
device_buffer[*offset + nb_bytes_to_copy] = '\0';
*offset += nb_bytes_to_copy;
return nb_bytes_to_copy;
}
Your code in general leaves much to be desired, but what I can see at the moment is that your .write implementation might be dubious. There are two possible mistakes - the absence of buffer boundaries check and disregard of null-termination which may lead to undefined behaviour of strlen().
First of all, you know the size of your buffer - BUFFER_SIZE. Therefore, you should carry out a check that *offset + length < BUFFER_SIZE. It should be < and not <= because anyhow the last byte shall be reserved for null-termination. So, such a check shall make the method return immediately if no space is available (else branch or >=). I can't say for sure whether you should return 0 to report that nothing has been written or use a negative value to return an error code, say, -ENOBUFS or -ENOSPC. Anyhow, the return value of the method is ssize_t meaning that negative value may be returned.
Secondly, if your first check succeeds, your method shall calculate actual space available for writing. I.e., you can make use of MIN(A, B) macro to do this. In other words, you'd better create a variable, say, nb_bytes_to_copy and initialise it like nb_bytes_to_copy = MIN(BUFFER_SIZE - 1 - *offset, length) so that you can use it later in copy_from_user() call. If the user, say, requests to write 5 bytes of data starting at the offset of 1021 bytes, then your driver will allow to write only 2 bytes of the data - say, he instead of hello. Also, the return value shall be set to nb_bytes_to_copy so that the caller will be able to detect the buffer space shortage.
Finally, don't forget about null termination. As soon as you've done with
copy_from_user(device_buffer + *offset, buffer, nb_bytes_to_copy);
you shall pay attention to do something like
device_buffer[*offset + nb_bytes_copy] = '\0';
Alternatively, if I recall correctly, you may use a special function like strncopy_from_user() to make sure that the data is copied with an implicit null termination.
Also, although a null-terminated write shall not cause problems with subsequent strlen(), I doubt that you ever need it. You can simply do *offset += nb_bytes_to_copy.
By the way, I'd recommend to name the arguments/variables in a more descriptive way. *offset is an eyesore. It would look better if named *offsetp. If your method becomes huge, an average reader will unlikely remember that offset is a pointer and not a value. offsetp where p stands for "pointer" will ease the job of anyone who will support your code in future.
To put it together, I doubt your .write implementation and suggest that you rework it. If some other mistakes persist, you will need to debug them further. Adding debug printouts may come in handy, but please revisit the basic points first, such as null-termination and buffer boundary protection. To make my answer a little bit more useful for you, I furnish it with the link to the section 3.7 of "Linux Device Drivers 3" book which will shed light on the topic under discussion.

Malloc fails and the logging, isn't this foolish?

Here's the code, isn't this pointless?
unsigned char * rngBuf = malloc(nBytes);
if(!rngBuf) {
DDLogError(#"Unable to allocate buffer for random bytes.");
[self displayHUDMessage:#"Memory Error."];
return;
}
If the malloc fails, why would logging or displaying a hud succeed?
What's the best way to deal with this situation?
It would be foolish only if displaying the error required nbytes or more memory from the heap (vs from the stack, static or quoted strings, or compile-time resolved string macros like FILE and FUNCTION).
I'm not familiar with your API, but I make sure to use statically allocated print buffers in my code. Here's an example.
// ... file management data structures...
typedef struct {
FILE *fp;
size_t knt; // record counter for use when needed
ULLNG bytes; // total number of bytes written to .fp file handle
char name[128];
char mode_str[8];
} FILE_DESC, *pFILE_DESC;
typedef struct {
FILE_DESC *fargs; // for the price of a FILE pointer we get all the details :)
size_t bytes;
char buff[1<<20];
} SMART_PRN_BUFF, pSMART_PRN_BUFF;
// ---- allocate 4 static/global smart write-buffers here
static SMART_PRN_BUFF FiPrn[4];
Paired with an initializer near the top of main() like...
FILE_DESC File[4] = {{NULL,0,0,"","w+t"},{NULL,0,0,"","w+t"},{NULL,0,0,"","r+b"}, {NULL,0,0,"","w"}};
printf("\nProcessing %i file names in \n%s", argc -1, argv[0]);
for(int i=0; i<argc; i++) {
if(NULL != argv[i+1]) {
strcpy(File[i].name, argv[i+1]);
File[i].fp = fopen(File[i].name, File[i].mode_str);
if(NULL==File[i].fp) {
printf("\nFile %s could not be opened in %s mode at line %i in %s\n--- ABORTING PROGRAM ---\n\n",
File[i].name, File[i].mode_str, __LINE__, __FILE__);
exit(EXIT_FAILURE);
} else {
printf("\nFile %s opened in %s mode", File[i].name, File[i].mode_str);
}
}
FiPrn[i].fargs = &File[i];
}
Usage is like ...
FiPrn[ERR].bytes += sprintf(FiPrn[ERR].buff,
"\nSize of FiPrn structure is %llu", (ULLNG)sizeof(FiPrn));
WriteToFile(&FiPrn[ERR]);
As you might have noticed, I have to use printf() vs sprintf() until I get the files opened and can write to them, but the static memory is allocated as SMART_PRN_BUFF.buff[1<<20], which gives me 1 megabyte for each of the 4 files I've provisioned here before I even attempt to open the files they will write to.
So I am completely confident I can print returns from malloc() and calloc() a few lines later, like this...
if(NULL == Sequencer)
FiPrn[ERR].bytes += sprintf(FiPrn[ERR].buff + FiPrn[ERR].bytes,
"\nCalloc() for Sequencer returned %p at line %i in file %s in function %s()\n",STD_ERR(Sequencer));
Where STD_ERR() is a macro that helps insure the code emits uniform error messages, like this...
#define STD_ERR(rtn) rtn,__LINE__,__FILE__,__FUNCTION__ // uniform error handling

ffmpeg/libavcodec memory management

The libavcodec documentation is not very specific about when to free allocated data and how to free it. After reading through documentation and examples, I've put together the sample program below. There are some specific questions inlined in the source but my general question is, am I freeing all memory properly in the code below? I realize the program below doesn't do any cleanup after errors -- the focus is on final cleanup.
The testfile() function is the one in question.
extern "C" {
#include "libavcodec/avcodec.h"
#include "libavformat/avformat.h"
#include "libswscale/swscale.h"
}
#include <cstdio>
using namespace std;
void AVFAIL (int code, const char *what) {
char msg[500];
av_strerror(code, msg, sizeof(msg));
fprintf(stderr, "failed: %s\nerror: %s\n", what, msg);
exit(2);
}
#define AVCHECK(f) do { int e = (f); if (e < 0) AVFAIL(e, #f); } while (0)
#define AVCHECKPTR(p,f) do { p = (f); if (!p) AVFAIL(AVERROR_UNKNOWN, #f); } while (0)
void testfile (const char *filename) {
AVFormatContext *format;
unsigned streamIndex;
AVStream *stream = NULL;
AVCodec *codec;
SwsContext *sws;
AVPacket packet;
AVFrame *rawframe;
AVFrame *rgbframe;
unsigned char *rgbdata;
av_register_all();
// load file header
AVCHECK(av_open_input_file(&format, filename, NULL, 0, NULL));
AVCHECK(av_find_stream_info(format));
// find video stream
for (streamIndex = 0; streamIndex < format->nb_streams && !stream; ++ streamIndex)
if (format->streams[streamIndex]->codec->codec_type == AVMEDIA_TYPE_VIDEO)
stream = format->streams[streamIndex];
if (!stream) {
fprintf(stderr, "no video stream\n");
exit(2);
}
// initialize codec
AVCHECKPTR(codec, avcodec_find_decoder(stream->codec->codec_id));
AVCHECK(avcodec_open(stream->codec, codec));
int width = stream->codec->width;
int height = stream->codec->height;
// initialize frame buffers
int rgbbytes = avpicture_get_size(PIX_FMT_RGB24, width, height);
AVCHECKPTR(rawframe, avcodec_alloc_frame());
AVCHECKPTR(rgbframe, avcodec_alloc_frame());
AVCHECKPTR(rgbdata, (unsigned char *)av_mallocz(rgbbytes));
AVCHECK(avpicture_fill((AVPicture *)rgbframe, rgbdata, PIX_FMT_RGB24, width, height));
// initialize sws (for conversion to rgb24)
AVCHECKPTR(sws, sws_getContext(width, height, stream->codec->pix_fmt, width, height, PIX_FMT_RGB24, SWS_FAST_BILINEAR, NULL, NULL, NULL));
// read all frames fromfile
while (av_read_frame(format, &packet) >= 0) {
int frameok = 0;
if (packet.stream_index == (int)streamIndex)
AVCHECK(avcodec_decode_video2(stream->codec, rawframe, &frameok, &packet));
av_free_packet(&packet); // Q: is this necessary or will next av_read_frame take care of it?
if (frameok) {
sws_scale(sws, rawframe->data, rawframe->linesize, 0, height, rgbframe->data, rgbframe->linesize);
// would process rgbframe here
}
// Q: is there anything i need to free here?
}
// CLEANUP: Q: am i missing anything / doing anything unnecessary?
av_free(sws); // Q: is av_free all i need here?
av_free_packet(&packet); // Q: is this necessary (av_read_frame has returned < 0)?
av_free(rgbframe);
av_free(rgbdata);
av_free(rawframe); // Q: i can just do this once at end, instead of in loop above, right?
avcodec_close(stream->codec); // Q: do i need av_free(codec)?
av_close_input_file(format); // Q: do i need av_free(format)?
}
int main (int argc, char **argv) {
if (argc != 2) {
fprintf(stderr, "usage: %s filename\n", argv[0]);
return 1;
}
testfile(argv[1]);
}
Specific questions:
Is there anything I need to free in the frame processing loop; or will libav take care of memory management there for me?
Is av_free the correct way to free an SwsContext?
The frame loop exits when av_read_frame returns < 0. In that case, do I still need to av_free_packet when it's done?
Do I need to call av_free_packet every time through the loop or will av_read_frame free/reuse the old AVPacket automatically?
I can just av_free the AVFrames at the end of the loop instead of reallocating them each time through, correct? It seems to be working fine, but I'd like to confirm that it's working because it's supposed to, rather than by luck.
Do I need to av_free(codec) the AVCodec or do anything else after avcodec_close on the AVCodecContext?
Do I need to av_free(format) the AVFormatContext or do anything else after av_close_input_file?
I also realize that some of these functions are deprecated in current versions of libav. For reasons that are not relevant here, I have to use them.
Those functions are not just deprecated, they've been removed some time ago. So you should really consider upgrading.
Anyway, as for your questions:
1) no, nothing more to free
2) no, use sws_freeContext()
3) no, if av_read_frame() returns an error then the packet does not contain any valid data
4) yes you have to free the packet after you're done with it and before next av_read_frame() call
5) yes, it's perfectly valid
6) no, the codec context itself is allocated by libavformat so av_close_input_file() is
responsible for freeing it. So nothing more for you to do.
7) no, av_close_input_file() frees the format context so there should be nothing more for you to do.

Resources