Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Note that periodic execution, one of the more useful aspects of celery, is not supported on all backends. If you're using rabbitMQ, you're golden. If you're using most or all other transports (including SQS), you're likely SOL and will have to resort to either bare cron jobs or something like django-cronograph that accomplishes the same thing.

celery is definitely nice, but it does have its limitations (like everything else).



I'm really interested in peoples opinion on Celery vs Others when using Django.

I'm about to hit a point in my pet project where I need something but I havn't chosen my approach yet. I've ehard Pyres is better than Celery.


Once you wrap your head around one (embarrassingly, it took me a while to really understand what celery was doing -- I had an easier time getting comfortable with the multiprocessing library), it shouldn't be difficult to pick up another and compare them locally.

Celery does what it's supposed to; that's about all I can say for it. I'd suggest writing a few simple jobs and testing them in similar environments under as much load as you can. Look for things like consistency of logging and running tasks exactly once across multiple consumers despite the semantics of the queue you pick.


This is good to know. I've just started using it with redis.

I never really saw the point of that part of celery anyway. Cron's perfectly adequate on its own (and installed just about everywhere).


The nice thing about it from our point of view is the ability to keep everything in source control. Granted, you can keep your cron jobs as scripts, but celery+rabbitMQ allows you to keep it all in Python without the extra deployment step. It's not a huge deal, but it does help a bit with maintainability and getting new devs up to speed.


We keep all the deployment stuff in source control anyway. For me, it would just mean shifting it from being declared by salt in the "infrastructure" folder to being written in python in the django folder.

I agree that you should keep the code itself in python, but that can be handled just by making it a management command that gets called by cron.

Sysadmins understand and love cron, too. I think celery reinventing the wheel might make some of them a bit mad.


I'm using cron instead of celery daemon for those tasks, but here are two advantages:

- being able to modify the task frequency from the admin

- cron needs special care with setting the environment variables like PATH




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: