A few days ago, I wrote a server in Node.js, the purpose of which is to email me when an API found in Nuget is updated. I thought the project would take a few hours, but as it turned out, it took several days. You may ask “How is this possible for such an easy problem?” Unfortunately, I spent a lot of time trying to figure out what API to use to read a text file, and how to use the API. It led me to an understanding of a fundamental problem with Javascript known as “callback hell” (1, 2).
In languages like C# and Java, file system I/O requests are usually blocking: the function does not return until the request is complete. A call to System.IO.File.ReadAllText opens a file, reads all the lines of text in the file into a string, closes the file, and returns the string. Control flow is transferred to the function and does not return until completion. Blocking functions are easy to use and understand in an imperative or functional programming context.
However, in Javascript, most APIs are non-blocking in order to boost performance and responsiveness of an application. Non-blocking function calls are implemented using a callback function, a computation that is executed by another function. For example, fs.readFile is a non-blocking function that opens a file, reads the text in the file, calls a callback function with two arguments, one of which contains the text, and closes the file. Code that textually follows fs.readFile cannot depend on the data because the callback is invoked asynchronously. Since the data is available through the callback function, a sequence of callback functions invocations is required, one dependent on the result of an enclosing callback. When callbacks are written as anonymous functions, the dependency is a nesting.
Deeply-nested callback functions
In my project, I wanted to create an array of URLs for APIs in Nuget.org from a file. Each URL in the file is on a separate line, and may contain trailing spaces, tabs, new lines, and carriage returns. Using nested callback functions, the Javascript code for this may look like the following:
As we implement the code that further processes the list of URLs (to request the webpage and extract dates of when the API was updated), the depth of the nesting callback functions increases. Callback chains is an implementation of a DU-chain of asynchronous variables, which is, unfortunately, the result of the design of Javascript.  At some point the code becomes unmanageable (3).
Alternatives to deeply-nested callback functions
Over the years, a number of solutions have been developed to solve deep nesting (4-9).
Declare a function for each nesting
Perhaps the easiest solution is to place each block corresponding to the callback in a separate function. The advantage of this that each block is now at the same lexical level.
One must judiciously decide between placing short blocks into separate functions that may be far removed from the context of the enclosing/calling block. However, this solution does not really solve the main problem of long chains of callback functions: a function is still a function regardless if it is has a name or is anonymous.
Function chaining using Promises
Promises for Javascript is an extension of a callback (10, 11). Functions that return a Promise can be chained together using the “then” method at the same lexical level.
Support for Asynchronous Programming
Support for asynchronous operations within the language are available for many languages, including Javascript in ES8/ES2017. (NB: async/await was dropped from ES7/ES2016. See the spec.) The async and await keywords eliminate the need for explicit callback functions. With the await function, continuation is registered as a callback with the awaiting expression. async/await is a feature of ES8, but is available in Node.js using Babel. (12-16)
Await/Async in ES8/ES2017
This solution works with Node.js 4.6+ and Babel. To set up Webstorm, follow these instructions.
Asyncawait API
A similar solution is available for ES6 Javascript using the API “asyncawait”, available through NPM. It provides a similar syntax and works well.
Further Information
- Callback Hell. http://callbackhell.com/
- Pyramid of doom (programming). https://en.wikipedia.org/wiki/Pyramid_of_doom_(programming).
- Harrison, Warren, and Curtis Cook. “Are deeply nested conditionals less readable?.” Journal of Systems and Software 6.4 (1986): 335-341. http://www.sciencedirect.com/science/article/pii/0164121286900038
- Computer Programming/Coding Style/Minimize nesting. https://en.wikibooks.org/wiki/Computer_Programming/Coding_Style/Minimize_nesting
- Handling Synchronous Asynchronous Loops In Javascript/Node.JS. https://zackehh.com/handling-synchronous-asynchronous-loops-javascriptnode-js/
- Green, Thomas R. G. “Ifs and thens: Is nesting just for the birds?.” Software: Practice and Experience 10.5 (1980): 373-381. http://onlinelibrary.wiley.com/doi/10.1002/spe.4380100505/abstract
- Clarke, Lori A., Jack C. Wileden, and Alexander L. Wolf. “Nesting in Ada programs is for the birds.” ACM Sigplan Notices. Vol. 15. No. 11. ACM, 1980. http://dl.acm.org/citation.cfm?id=948651
- https://www.quora.com/How-do-you-make-an-async-function-synchronous
- http://stackabuse.com/avoiding-callback-hell-in-node-js/
- Futures and promises. https://en.wikipedia.org/wiki/Futures_and_promises
- https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise
- http://stackabuse.com/node-js-async-await-in-es7/
- http://es6-features.org/
- Bierman, Gavin, et al. “Pause’n’Play: Formalizing Asynchronous C^\ sharp.”European Conference on Object-Oriented Programming. Springer Berlin Heidelberg, 2012. http://link.springer.com/chapter/10.1007/978-3-642-31057-7_12
- Tasirlar, Sagnak, and Vivek Sarkar. “Data-driven tasks and their implementation.” 2011 International Conference on Parallel Processing. IEEE, 2011. http://dl.acm.org/citation.cfm?id=2066922
- Cordemans, Piet, Eric Steegmans, and Jeroen Boydens. “Task Parallel Paradigms: a Comparative Case Study.” Annual Journal of Electronics. Vol. 7. Technical Univ. of Sofia, 2013. https://lirias.kuleuven.be/handle/123456789/416602
- Callback heaven for Node.js with async/await. https://github.com/yortus/asyncawait