event handling - Is rearming file descriptors for epoll thread safe? -


from this question know can call epoll_ctl(2) while thread blocking on epoll_wait(2). still have question though.

when using epoll epolloneshot flag 1 event fired , fd has rearmed using epoll_ctl(2). necessary 1 thread read fd , handle result appropriately.

the following timeline visualizes supposed problem:

thread1:                       thread2:                  kernel: ----------------------------------------------------------------------- epoll_wait();                                                          receives chunk dispatch chunk thread 2 epoll_wait();                  handle chunk                                still handle chunk        receives chunk                                rearm fd epoll ? 

what happens on question mark when fd rearmed after chunk received? epoll fire epollin event, or block indefinitely although socket readable? architecture @ sensible?

your architecture sensible, , work: epoll mark file descriptor readable , fire epollin event.

the documentation on scarce , subtle; q/a section of man 7 epoll briefly mentions this:

q8 operation on file descriptor affect collected not yet reported events?

a8 can 2 operations on existing file descriptor. remove meaningless case. modify reread available i/o.

the 2 operations can on existing file descriptor (an existing file descriptor file descriptor has been added epoll set in past - includes file descriptors waiting rearmed) delete , modify. manpage mentions, delete meaningless here, , modify re-evaluate conditions in file descriptor.

nothing beats real world experiment though. following program tests edge case:

#include <stdio.h> #include <pthread.h> #include <signal.h> #include <stdlib.h> #include <assert.h> #include <semaphore.h> #include <sys/epoll.h> #include <unistd.h>  static pthread_t tids[2]; static int epoll_fd; static char input_buff[512]; static sem_t chunks_sem;  void *dispatcher(void *arg) {     struct epoll_event epevent;      while (1) {         printf("dispatcher waiting more chunks\n");         if (epoll_wait(epoll_fd, &epevent, 1, -1) < 0) {             perror("epoll_wait(2) error");             exit(exit_failure);         }          ssize_t n;         if ((n = read(stdin_fileno, input_buff, sizeof(input_buff)-1)) <= 0) {             if (n < 0)                 perror("read(2) error");             else                 fprintf(stderr, "stdin closed prematurely\n");             exit(exit_failure);         }          input_buff[n] = '\0';         sem_post(&chunks_sem);     }      return null; }  void *consumer(void *arg) {     sigset_t smask;     sigemptyset(&smask);     sigaddset(&smask, sigusr1);      while (1) {         sem_wait(&chunks_sem);         printf("consumer received chunk: %s", input_buff);         /* simulate processing... */         sleep(2);         printf("consumer finished processing chunk.\n");         printf("please send sigusr1 after sending more data stdin\n");          int signo;         if (sigwait(&smask, &signo) < 0) {             perror("sigwait(3) error");             exit(exit_failure);         }          assert(signo == sigusr1);          struct epoll_event epevent;         epevent.events = epollin | epolloneshot;         epevent.data.fd = stdin_fileno;          if (epoll_ctl(epoll_fd, epoll_ctl_mod, stdin_fileno, &epevent) < 0) {             perror("epoll_ctl(2) error when attempting readd stdin");             exit(exit_failure);         }          printf("readded stdin epoll fd\n");     } }  int main(void) {      sigset_t sigmask;     sigfillset(&sigmask);     if (pthread_sigmask(sig_setmask, &sigmask, null) < 0) {         perror("pthread_sigmask(3) error");         exit(exit_failure);     }      if ((epoll_fd = epoll_create(1)) < 0) {         perror("epoll_create(2) error");         exit(exit_failure);     }      struct epoll_event epevent;     epevent.events = epollin | epolloneshot;     epevent.data.fd = stdin_fileno;      if (epoll_ctl(epoll_fd, epoll_ctl_add, stdin_fileno, &epevent) < 0) {         perror("epoll_ctl(2) error");         exit(exit_failure);     }      if (sem_init(&chunks_sem, 0, 0) < 0) {         perror("sem_init(3) error");         exit(exit_failure);     }      if (pthread_create(&tids[0], null, dispatcher, null) < 0) {         perror("pthread_create(3) error on dispatcher");         exit(exit_failure);     }      if (pthread_create(&tids[1], null, consumer, null) < 0) {         perror("pthread_create(3) error on consumer");         exit(exit_failure);     }      size_t i;     (i = 0; < sizeof(tids)/sizeof(tids[0]); i++) {         if (pthread_join(tids[i], null) < 0) {             perror("pthread_join(3) error");             exit(exit_failure);         }     }      return 0; } 

it works follows: dispatcher thread adds stdin epoll set , uses epoll_wait(2) fetch input stdin whenever becomes readable. when input arrives, dispatcher wakes worker thread, prints input , simulates processing time sleeping 2 seconds. in meantime, dispatcher goes main loop , blocks in epoll_wait(2) again.

the worker thread won't rearm stdin until tell sending sigusr1. so, write more stuff stdin, , send sigusr1 process. worker thread receives signal, , rearms stdin - readable time, , dispatcher waiting on epoll_wait(2).

you can see output dispatcher correctly awaken , works charm:

dispatcher waiting more chunks testing 1 2 3 // input dispatcher waiting more chunks // dispatcher notified worker , waiting again consumer received chunk: testing 1 2 3 consumer finished processing chunk. please send sigusr1 after sending more data stdin hello world // input readded stdin epoll fd // rearm stdin; dispatcher waiting dispatcher waiting more chunks // dispatcher saw new input , waiting again consumer received chunk: hello world consumer finished processing chunk. please send sigusr1 after sending more data stdin 

Comments