This is a nicely consumable lightweight summary of how the node event loop works. In my experience this is one of the hardest things for new JS developers to actually understand, especially those coming to the language with a limited asynchronous programming background. Here's a good link to how the browser event loop works for those interested: http://blog.carbonfive.com/2013/10/27/the-javascript-event-l...

IMHO, there's just a few things you need to understand to start getting productive with js fast, #1 on the list is to know how the event loop in the browser/node works and this is a really good intro to the node side of things. #2 is event bubbling/capturing in the DOM (http://javascript.info/tutorial/bubbling-and-capturing), and #3 would be everything about functions / function scoping.

Those are the parts I always try to explain first to devs I've mentored who were just jumping in to javascript.

The only thing that annoys me with this (and many other popular posts in our field recently) is that the author talks about "magic". I believe this started with the advent of Rails where some things seemed "magical" at first due to Ruby's meta-programming features. Many writers and speakers have adopted it since.

However, there is nothing occult about software development and we shouldn't use "magic" as an excuse for not understanding or describing things thoroughly. Just imagine your doctor explaining you a surgery in these terms.

Otherwise, a very good and refreshing read.

I think its important, especially in posts like this that are targeted to beginners, that certain information be left out and not clarified (essentially making the masked parts "magic"). Otherwise, the reader is overwhelmed with information and really can't grasp the overall picture. I think this article succeeds on its goal of shedding some light on how the node event loop works at a high level, and its straightforward about its goals in that regard.

Imagine trying to explain to someone how math operations work in js, you wouldn't really want to explain how numbers are represented in the computer until you get to something that necessitates that understanding - like bit operations.

So, I'd just say it depends on your goals, if your goal is to explain precisely how every detail of something works, then yes I'd agree with you, there's no excuse for magic. But if you want to explain how a general concept works I'm fine with loads of "magic" holes you can fill in for yourself later.

The use of the term "magic" to describe spooky abstractions or odd behavior caused by referential opacity far, far predates Rails.

To be fair, the productivity gains that using Rails gets you is pretty magical.

Right, I agree with this. There's nothing magic about any of this -- ultimately it's all just software, hardware and physics.

To describe something as magic in a serious context is toxic. We're not sleight of hand performers, we're engineers. It's more helpful to state something like, "this library is complicated and the internals are beyond the scope of this post". A pointer to documentation and source code on libuv would be even better. In face, this series of articles on libuv are a great start for those who are interested: https://nikhilm.github.io/uvbook/introduction.html

If all occurrences of "this is [magic]" are treated as "this is [sufficiently complicated internally and beyond the scope of this post]," then what's the difference? The former makes for a point that's expressed more succinctly and eloquently at little real loss.

Definition 2a seems related: "Producing [...] remarkable results, like those attributed to magic [...]; effecting or permitting change, success, etc., as if by magic."

Also the 'magic box'.

The Urban Dictionary (the place where I look for colloquialisms) has this definition: "with or through complexity either spurios to the contemproary context or to long to be reasonable or desired of explanation". [1]

1: http://www.urbandictionary.com/define.php?term=magic&defid=9...

This is what we like to call a "figure of speech". It means that we do not believe to be literally magic, but are using hyperbole to describe a situation where something happens and it's not immediately apparent why.

It's older than Rails, I remember the interpreter being described as magic in SICP. It was mentioned as such several times over the course of the book, and was somewhat thematic.

Until, of course, the final chapter (or was it the chapter before the final chapter), where you learned that magic isn't real.

There's a great talk from jsconfeu that describes the event loop in more detail. It helped me understand what I subconsciously had picked up about how Javascript works: http://2014.jsconf.eu/speakers/philip-roberts-what-the-heck-...

Thank you. That was a fantastic talk. Short, to the point and useful :)

Another important concept for JS devs to understand is (macro)tasks and microtasks. This is becoming especially important in ES6 and beyond (due to Promises leveraging microtasks).

http://stackoverflow.com/questions/25915634/difference-betwe... has a decent overview

Case in point which tripped me up recently. Knowing that promise.then() callbacks aren't called synchronously[1], even if the promise is already resolved, I assumed that this:

    var tick = Promise.resolve();
    async function cpuBoundJob() {
      for (...many iterations...) {
        // crunch some bits
        await tick;
      }
    }

...would spread out a heavy calculation and allow other things to break in occasionally and do work. But based on tests, nothing ever got a chance to run until the whole job finished.

My current understanding (correct me if I'm wrong) is that promise.then() calls you back from a microtask, which means "await tick" doesn't teleport you into the next tick, just out of the current call stack into a new one at the end of the same tick, regardless of how many things are piled up waiting on the next tick. In other words the entire calculation remains one big CPU-blocking chunk. This modified version ended up working as expected, since setImmediate is (apparently) not a microtask.

    var tick = () => new Promise(res => setImmediate(res))
    async function cpuBoundJob() {
      for (...many iterations...) {
        // crunch some bits
        await tick();
      }
    }

[1] https://promisesaplus.com/#point-34

Nice description. The Node event loop pretty much defines the general shape of all APIs that work in Node. When we were porting HyperDex to Node, the greatest challenge was to integrate the data store with the async event loop [1]. The resulting callback oriented programs look strange at first to people coming from a traditional thread-oriented background, but they have an elegance of their own.

[1] http://hyperdex.org/doc/latest/NodeAPI/

btw, the params order for callbacks is not very node-y in HyperDex. It should be (err, result) instead of (result, err).

Putting the error first would enable Async or Promise libs to work...

> emitter.emit('fire')

I think it should be this.emit('fire')

Great

> Node utilizes as much of what's already available from the operating system's kernel as possible.

And yet we've seen a proliferation of fibre/coroutine/green thread libraries for Node which all reinvent the concept of threading poorly.

Node's event loop is inherently cooperative multitasking. It's not trivial to simulate preemptive multitasking without writing/compiling your code to giant state machines. There are different use cases for the two, especially since any CPU-bound task performed in the event loop will cause stuttering in your app.

In a node process you can't run 2 things at the same time without forking an external process. That's why these libraries exist. A complex synchronous computation will block the only thread available in nodejs. Sure you can spawn workers, but your node workers will have the exact same issue.

> all reinvent the concept of threading poorly

because "user scripts" in node have no concept of threading to begin with. There is no concurrency in a node process, only non blocking I/O.