Page Background

Understanding the event driven nature of Node.js

Fundamental Operators #

To understand at the fundamental level how the event driven mechanism in node.js works, lets start with the fundamental design of the computer and understand how the computer works. Still looking at the very high level and ignoring the magic that the OS provides, there is something called the instruction pointer in the CPU. This points to the next instruction in the assembly language that is executed. All programming languages eventually go down to assembly as that is what the hardware understands. The assembly language provides very basic bare-bones primitives which are used to build the entire feature set of all programming languages. So we have a special region in the memory where the executing code resides (in dynamic languages like JavaScript even this is very murky but still makes it easy to understand what’s going on) and the instruction pointer goes line by line through this code. You might have seen such a thing in the debugger when you debug your software application. The pointer jumps slightly in javascript because we have multiple executable statements in one line of code. In assembly it is not the case and we will find it running sequentially like shown in the Figure 1.

One basic operator fundamentally provided at the hardware layer is the conditional jump operator. This provides the basics of all the other functionality that traditional programming brings. What this provides is the ability to jump to a different line in code if a certain condition is true. We use it almost as is for the if-else conditional like shown in Figure 2.

Sequential Execution

Sequential Execution

Conditional Execution

Conditional Execution

Loop Execution

Loop Execution

Function Execution

Function Execution

We also use it for the looping dispatch where we have a jump back to somewhere earlier in code. This is understood cleanest in the do..while loop in some programming languages, but functionally the for and the while loops are built over the same construct. This type of construct is shown in Figure 3.

The other constructs like functions are just two jump operators one goes into the special piece of code and the other comes back to the main executing code. But it still fundamentally is the same operator. The functional construct is shown in Figure 4.

Asynchronous programming #

Asynchronous programming is fundamentally different from regular programming. In asynchronous programming, we have two things happening in parallel. These two things not necessarily mean two CPUs/threads running in parallel. There could be multiple things that happen together. The hard disk has a spooler that actually writes from the RAM to the disk. The CPU is available to do something else while the write is happening. Similar is the case with fetch being done via the network card or console being written to the screen. There are also multiple CPU threads or physical machines in certain APIs like crypto in node.js. the concept of two things done in parallel is not alien to node/javascript which is built primarily as the language around optimized network communication. Most machines are multi-core and therefore there are multiple CPUs running in parallel. We also have GPUs that can do processing(GPGPU) in parallel to the CPU working on something else.
A piece of asynchronous code logically looks like what is shown in Figure 5. This is exactly how we do multi-threaded programming in most use cases. When the control reaches an asynchronous method, the second thread or hardware picks up the task and continue running in parallel. It could access the same resources and run at same or different speeds which the OS does not give the control over. It is very difficult to understand it across compiler optimizations and the assembly format of the hardware anyways. In traditional multi-threaded programming, at some point if we want to do some work in parallel we would fork a thread and then have the two threads running in parallel until we are done with the parallel operation. The join operation is what follows next where the faster thread waits for the slower thread to catch up and the resources are merged and the execution continues.
In most programming contexts, there is little we could do in parallel to the regular operations like filesystem access or even connection to a database. Therefore even though the CPU could potentially be used for something else, the program lets the OS make this decision as it does not have anything else to do. In most I/O bound use cases like those of doing CRUD operations on a database in response to network queries and we have to read from a file on disk or from the database connection over a high speed local network, the CPU wait is not too much of a problem. There is very little the CPU can do without the data and the Operating System is efficient in reallocating resources to another threads if a particular thread is waiting for the some I/O processing. The problem arises when we have hundreds of work items done in parallel like in the case of an application server responding to network requests. Having hundreds of threads is resource intensive and the OS is not able to do a good job with it. Javascript the language was built around extremely slow network requests and therefore instead of hiding the complexity of multiple hardware pieces doing multiple jobs, cherishes and exposes the concept in full flow. That is why most of the core methods in node are asynchronous and provide a promise or a callback.

To achieve the behavior that logically looks like the multi-threaded behavior in Figure 5 but can reuse the same flow of execution, the node runtime maintains a queue of tasks that the other pieces of hardware complete. So it never waits for the disk pooler or the network card. As soon as it reaches a statement that needs to send some work to the disk pooler, it takes the scope of the callback and ensures that it remains in the RAM by marking that used. Then the runtime continues. The other task is triggered and node does not wait for it. When the node reaches an end of the current task or things like an await statement, it looks at the queue of tasks that are completed by the other pieces of hardware that are waiting for further execution. It picks up the first task and attaches its scope to start execution. Therefore the main thread only halts if there is no work to do and in those cases if there is no timers or other hardware scheduled to do work, the node program quits. Figure 6 shows how this thing works. As programmers the act of picking up other task by the node process is something that may not impact us and therefore we remain oblivious to this event driven nature underneath. We can continue to believe that the current flow waits for the other hardware when we write await as logically that is correct. But instead of another thread modifying another variable, it is the same thread that performs this logic.

Logical Async

Logical Async

Event Driven Async

Event Driven Async

If you have done lower level C++ application programming, you must feel this thing to be nothing new. The application run loop in all Operating systems works like this. They are all event driven. The uniqueness of node/javascript is not the event driven architecture but instead the adoption of this reality. Whereas languages hide this complication under synchronous read calls from the filesystem and try their best to make the programmer not worry about this detail, node/javascript embraces this. All methods are asynchronous and the runtime itself provides this queuing mechanism. In languages like C, we have the build it ourselves. Nginx is an example of great open source software that embraces this event driven architecture. The thing with JS is that this is forced upon every program by the standard library. The concept of scopes which all can reside in the heap memory, self contained objects that do not depend on any pointers to a class structure are designs to make the task of maintaining the queue easier. Therefore every program is fast, even though it may not be maintainable and may feel ugly. Early version of node software was ugly, error prone and confusing to use. Performance is the most difficult thing to patch on to any runtime. The other problems are solvable. Features like async await enable programming like Figure 5 without the overhead of making the thread wait. This concept has been so powerful that almost all languages have upended their standard libraries to add support for these features.

Node’s event driven architecture is neither new nor unique. But node’s standard library was the first to embrace the event driven reality of working with multiple pieces of hardware at a large scale and forced all programmers to care about it. It has reaped the rewards to be hottest technology on the servers 10+ years and still running.

This post is accompanying my speech at Byteconf JS 2019.

More in Async Await

Comments

Post a new comment

We get avatars from Gravatar. You can use emojis as per the Emoji cheat sheet.