Essentially a tail -f
implementation: I’m updating a program that works fine on an older system that will continuously read an open file: after EOF, wait for the file to grow and keep reading. On newer systems, it is not able to read the file past EOF and I can’t understand why. I stripped out all unrelated code and am able to reproduce with the following simplified example, which is nearly identical to the pattern given here.
// COMPILE WITH: gcc a.c -o a.out
#include <stdlib.h>
#include <stdio.h>
#include <string.h>
#include <errno.h>
#include <unistd.h>
int main(int argc, char* argv[]) {
//----------------------------------------------------------------
char* file_path = "/var/log/mail.log"; // any file that's active
int seek_to_head = 0;
int seek_to_tail = 1;
int loop_sleep_seconds = 4;
int use_getline = 1; // if 0, uses fgets()
//----------------------------------------------------------------
size_t line_size_fgets = 1024;
char line_fgets[line_size_fgets];
char* line = NULL;
size_t line_size = 0;
// Open the file
FILE* FILE_HANDLE;
if (!(FILE_HANDLE = fopen(file_path, "r"))) {
printf("Failed to open file [%s]: %s\n", file_path, strerror(errno));
return 1;
}
// Unbuffer input
if (setvbuf(FILE_HANDLE, NULL, _IONBF, 0) != 0)
// if (setvbuf(FILE_HANDLE, NULL, _IOLBF, 0) != 0)
{
printf("Failed to unbuffer\n");
return 1;
}
// Seek to head
if (seek_to_head) {
if (fseek(FILE_HANDLE, 0, SEEK_SET)) {
printf("Failed to seek to beginning of file\n");
return 1;
}
printf("Seek to start of file -- SUCCESS\n");
}
// Seek to tail
if (seek_to_tail) {
if (fseek(FILE_HANDLE, 0, SEEK_END)) {
printf("Failed to seek to end of file\n");
return 1;
}
printf("Seek to end of file -- SUCCESS\n");
}
printf("Begin tracking file %s\n", file_path);
// Main loop
while (1) {
// Read all remaining file contents til exhausted for now
if (use_getline)
while (getline(&line, &line_size, FILE_HANDLE) != -1)
printf("\nLINE: %s\n", line);
else
while (fgets(line_fgets, line_size_fgets, FILE_HANDLE) != NULL)
printf("\nLINE: %s\n", line_fgets);
// Make sure there wasn't an error reading file
if (ferror(FILE_HANDLE)) {
printf("Failed to read from file [%s]: %s\n", file_path, strerror(errno));
return 1;
}
// Pause for file to grow (inotify code goes here)
sleep(loop_sleep_seconds);
printf("Try reading again\n");
}
free(line);
printf("Exiting\n");
return 0;
}
This example has a variable you can flip to use getline() or fgets() but both have the same problem on newer systems I’ve tried. If seeking to the beginning of the file, all the file contents are printed to date, but anything after that is never printed. If seeking to the end of the file, no output is ever produced. The real program polls for file changes instead of sleeping and I confirmed that the poll is working correctly, but that seems unrelated; the errant behavior is the same both ways.
Am I missing something obvious?
You can (and should) explicitly clear the EOF before attempting to read further data. Something like:
// Pause for file to grow (inotify code goes here)
if( feof( FILE_HANDLE ) )
{
clearerr( FILE_HANDLE ) ;
sleep(loop_sleep_seconds);
printf("Try reading again\n");
}
Otherwise EOF will normally only be cleared by call to one of rewind, fseek, fsetpos and freopen on the same stream. So you could alternatively or setpos
to the current position.
What exact OS version(s) are involved?
@Someprogrammerdude Sorry, I don’t see that example doing anything with the input stream (fp) except getc() and an error check. Does flushing stdout somehow clear the EOF state on the input FILE stream? My original program isn’t printing anything out so that won’t be an option.