I've been helping out with the Great Git Migration effort lately, particularly the queue infrastructure. As a result of this work, I created the Waiting Queue module. A design goal of the module is to make it easy to run a queue with workers that wait indefinitely for jobs. A worker that waits indefinitely will pull jobs off the queue as soon as they are available, which keeps the time a job waits in the queue very small. The Drupal.org queue infrastructure will be based around Beanstalkd, a simple, highly efficient queue implementation. This queue system works well with workers that wait indefinitely for jobs. This sort of queue processing is not compatible with the design of the core Drupal Queue module, because a single queue with workers that wait indefinitely will block all other queues called via a site's cron run. This is why we felt the need to write a new module. The Drupal.org infrastructure will use Supervisord to manage our long running processes, making it easy to guarantee that there will always be a worker ready to process jobs as soon as they are added to the queue. The Waiting Queue module includes a sample Supervisord configuration file.
With nearly two decades of professional technology experience, Alex oversees all business functions at Zivtech and Probo.ci, from sales and marketing, to client relations, human resources, and business strategy.
View the discussion thread.
Tell us about your project