From callbacks to async - await
From callbacks to async - await - A migration guide #
I originally wrote this article for Morning Cup of Coding. Morning Cup of Coding is a newsletter for software engineers to be up to date with and learn something new from all fields of programming. Curated by Pek and delivered every day, it is designed to be your morning reading list. Learn more.
Why do we have callbacks? #
XMLHttpRequest.send() if it waits for the response to come over, the UI would hang as it takes significant time for the request to be sent over to the server, be processed and the response to come back. Therefore it made sense to take a function as an argument that would be called when the response is available, freeing the CPU for other things while we wait for the response. With Node.js, this was taken one step further as even requests to the database or to the hard disk, are also slower in comparison to the CPU, and waiting for them even in a background thread is a waste of resources. That is why in Node.js almost every call is asynchronous and after the success of Node.js, other languages also brought in asynchronous constructs int heir core.
Problems with callbacks #
Callbacks are very good for the CPU but they are not so good for the programmer for a variety of reasons. Programmers, unlike computers, think in a sequence and therefore while the CPU should not wait for the database response, the programmer needs to think about the task after the database has responded. Therefore, the logical next step for the program after the database call is handling of the response. With the presence of callback after callback, changing the code into subroutines becomes extremely difficult as the flow of the program does not remain linear. The next line may be called much earlier than the previous one in execution, forcing the programmer to constantly monitor the execution order while writing code. Here is an excerpt of the problem.
Synchronous Code(Slow but clear)
Asynchronous Version(Fast but difficult to follow) **
The asynchronous code, though looks ugly, has much better performance than its synchronous counterpart. Once you have more than one independent parallel call to the
calculate method, the synchronous code has to resort to threads and that requires creation of huge data structures in memory. While that has been the approach for the first decade of the internet, the costs of that approach have proven to be prohibitively too high and the economics has pushed for adopting the asynchronous version.
The callback-based asynchronous code is more verbose and prone to errors:
- If one of the return statements is missing, the callback can be called twice which can potentially break everything.
- Errors are easier to ignore as they have repetitions that are useless and cause more confusion.
- You are more likely to hit the width of your monitor screen because of the indentation.
- You are more likely to turn, say handling the second response into its own named function which would need to be moved somewhere above the entire method making it even more difficult to follow.
db.sqlcould go into the previous
db.sqland then there is no way to understand anything from the call stack.
- Once it gets beyond 3-4 levels deep, you don’t really understand what happens when. Accidental parallel code where the response is not present is a side effect.
What is async await? #
Async await is a simple solution to all of the above problems. While the code in async await works just like in the world of callbacks where the CPU is not wasted, it looks like synchronous code that we understand and love, and the extra verbosity is lost.
Now all those calls that were giving callbacks are still asynchronous. If you put a breakpoint in line 5, it might take a while after line 4 to actually be reached. But in the meanwhile, if you have other JS code, say some timeout elsewhere, those still get hit as the CPU is free to work on other things. Async Await is the cleanest solution to the problem of having lots of callbacks known as callback hell where we had to live with ugly code for the performance benefits that it provides.
Trivia Async Await is not the first solution to the callback hell problem. Generator Functions, Promises and even strict coding conventions have been used in many places. Node.js had to introduce a huge set of sync versions of its various APIs because writing asynchronous code was so much pain that developers sometimes preferred to just use the inefficient synchronous API to get around it. Async Await provides a proper solution where the code looks like the synchronous API but works like the callback version without the loss of any expressiveness, performance or power.
The elephant in the room - Promises #
Promises are complicated and at this point an internal detail. Async await is a wrapper syntax and works based on promises. The concept of promises is therefore required to be known if you ever want to go deeper into async await. We will not be going into the nitty-gritty of promises. A great introduction on Promises can be found in the google developer documentation and at MDN. This article explains how async await are built over promises. Promises are ways for synchronous and asynchronous functions to work together. If you ever need to convert callback based code to async outside of the pattern that we will just be learning or want to use async code in their synchronous counterparts, promises are the way to go. Guru99 provides a good tutorial on converting to promises which could be used as an intermediatory step before going into async await.
Steps to go async #
- Convert all callbacks to be of the
callback(err, data)format that takes one error and one data argument. The task is not very complicated. With ES6, the wrapper may actually end up being extremely small and useful.
- Use node’s
util.promisifymethod to convert existing method to async one by one. If you are not in the node environment you can use a polyfill.
- Handle one off errors or errors that can be skipped with a
Note that after performing a
.catch(cb) based special processing, you can still throw to reach the global handler at the try catch. If you want to do different error handling in all cases which was possible with the callback based code, you can use a
.catch clause everywhere. The await calls could still be wrapped in try..catch to handle exceptions in the catch block or otherwise.
- Once all methods are converted, run a replace all call through the code base to replace new method
task_pwith the original method name
taskif we had used the naming convention like described in step 2 above.
Common Patterns #
- Wrap parallel code in Promise.all. If some code is meant to be run in parallel, we should not make it sequential. Promise.all call take an array of asynchronus functions and run them in parallel. Since we can now work with array, all the array manipulation functions like
forEachcan now be used. There is also Promise.race for the cases where we need to wait for any one of the methods to respond and do not need to wait for all of them to continue execution.
Note that with
Promise.all and a sequence of
await calls all serial and parallel combinations of requests should be handled.
- Leave the side effects without await or use the next tick. Sometimes we do have side effects like leaving a call to analytics that we do not wish to wait for or logging that can happen after the response is sent from the server. For these we can ignore the await call and it works just like before.
The above assumes that
writeToDisk is not CPU expensive. If it is and there is no way to fix it, we could wrap it in a next tick call, like
process.nextTick(() => writeToDisk(resp)) . It might still be hit before the response is actually sent as there might be multiple async calls after
handleRequest but that is not something the old code was doing anyways.
What async await is not #
- Async await does not magically make the code synchronous. Everything is still asynchronous. The programmer still needs to understand the concept of event driven programming and the event loop.
- Async await is not a performance booster and will not speed up execution of the code. It might help in the cases where accidental synchronous APIs had been used which was slowing down code but if you are looking for a massive performance boost, this is not the right place. Instead are cases where async await can lead to more RAM consumption and can be difficult to debug if we need to transpile the code and do not have source maps.
Async await is a cleaner way to express the same ideas as callbacks and do not incur the performance overhead of synchronous code. They may be promises internally but apart from various method names like
util.promisify, there is no need to understand promises in depth to use async await in production. All the code can be converted piece by piece to async functions with simple steps. After the completion is done, it would be discovered that not only is the newer code a lot less verbose and easier to navigate but also fixes many bugs that might have crept up over time into the callback based methods. There is little reason to not move to async await especially as all major browsers as well as Node.js support them natively and they provide the same feature set with a much cleaner API as do callbacks.