fritzthecat-blog: Yet Another JavaScript AMD Loader

JavaScript dependency management via AMD is something that will stay. At least in our memories. It is an impressive insight into functional thinking.

The question rises "How do they do that?", provide dependency management by means of the language itself, not by some built-in mechanism?

On the requireJs page you can read about the basic ideas how to load scripts by JS. RequireJs itself uses adding script tags to HTML page head. RequireJs has about 2000 lines of code, and it is a standard library.

I was curious how many lines of code would be needed to have a basic AMD loader. Here are my sketches. My target was understandable source code. No hasOwnProp(), isFunc() etc.

WARNING: This article presents an AMD loader that really works, but it has restrictions:

you can not have more than one define() call per HTML page (put together all JavaScript of a page into just one initialization define())

you can not have more than one define() call per module file

Any "imperative" call to define(), after the first call, could confuse the loader in that it does wrong module-name associations for scripts that call define() because they were loaded as modules ("non-imperative" call).

The core problem is the fact that you can not associate a name to a module that calls define() except by an onload listener that holds the expected name. This listener is called asynchronously after the script has been executed. In the meantime, an "imperative" call to define() would put a wrong module into the variable the listener is waiting for its expected name.

Building Blocks

Load a Script using JS

Here is a JS implementation that adds a script tag to HTML head and registers a listener for finished-loading.

Mind that this does not work on IE-8 and older due to missing addEventListener. My aim is good readable code, not solutions that work on every browser. Understandable JS code is rare, so enjoy it :-)

  var loadScript = function(scriptUrl, scriptLoadedCallback) {
    var scriptElement = document.createElement("script");
    
    var finishedLoading = function(event) {
      if (event.type === 'load' || // seen in require.js, checks if script is loaded
            ("/^(complete|loaded)$/".test((event.currentTarget || event.srcElement).readyState)))
      {
        scriptElement.removeEventListener('load', finishedLoading, false);
        if (scriptLoadedCallback)
          scriptLoadedCallback(scriptUrl);
      }
    };
      
    scriptElement.addEventListener('load', finishedLoading, false);
    scriptElement.type = "text/javascript";
    scriptElement.src = scriptUrl;
    
    document.head.appendChild(scriptElement);
  };

It is fun to write readable source-code. Hard to read is the part where the event is checked whether it is a finished-loading event. I took this from requireJs, and tried to make it comprehensible by commenting it.

Try this out, it actually works. Although it is readable code :-)

But most likely it won't work with an external URL like http://site/beautiful/script.js. Most browsers will not let you load scripts from external sites through JavaScript. This is called cross-site-scripting and is a security hazard. Only URLs relative to the HTML page will work through JavaScript, like e.g. js/app/main.js. Nevertheless you can add an external script-URL to the HTML page in the traditional way.

Load Dependencies Recursively

Caused me some effort. Loading a script is nice, but this script could load other scripts, which again could load ... a recursive task. A dependency tree, hopefully without cycles. Recursive thinking always takes some time.

Here is an example of what is to be done. This "module" defines one dependency, and then gives a factory function which creates the module when executed. The anonymous function(message) will be the module finally. The dependency "app/messages.js" must be loaded and passed as parameter to the factory function function(appMessages). This is the AMD way to import modules.

define(
  [
    "app/messages.js"
  ],
  function(appMessages) {
    return function(message) {
      alert(appMessages.infoHeader+": "+message);
    };
  }
);

Why is there a factory function? Wouldn't it be better to immediately return the function that should represent the module?
The AMD specification does not give reasons for that, but I guess it is there to give the caller a chance to configure the module before actually executing some functionality of it.
In this case the alert dialog uses a message from the parameter of the outer factory function.

And here is a way to implement define(), in a hopefully readable manner:

  var currentModule;

  var define = function(dependencies, moduleFactory) {
    var parameters = [];
    
    if (dependencies.length) // the first parameter is an array
      parameters = evaluateParametersFromDependencies(dependencies, moduleFactory);
    else if ( ! moduleFactory ) // happens when no dependencies given
      moduleFactory = dependencies; // parameter in fact is the moduleFactory
    else
      throw "Dependencies must be array of strings, but was: "+dependencies;
    
    var dependenciesResolved = (parameters !== null);
    if (dependenciesResolved)
      if (typeof moduleFactory === "function")
        currentModule = moduleFactory.apply(context, parameters);
      else
        currentModule = moduleFactory;
    else
      currentModule = undefined;
  };

This allows only two parameters, the dependencies string array and the module (which is different from the AMD specification that allows the module name as first of three parameters).

If the dependencies are all present, define() calls either the factory function, or it takes the given object as module. It puts the module, whatever it is, into the instance variable currentModule. But if a null parameter array is returned from evaluateParametersFromDependencies(), the module is set to undefined.

Let me explain later what happens with the variable currentModule. Here is an implementation of evaluateParametersFromDependencies().

  var modules = {};
  var unresolved = [];
  var currentDependant = "main.js"; // to be initialized on load

  var evaluateParametersFromDependencies = function(dependencies, moduleFactory) {
    var parameters = [];
    for (var i = 0; i < dependencies.length; i++) {
      var dependency = dependencies[i];
      if (typeof dependency === "string") { // dependency is an identifier
        var loadedModule = modules[dependency]; // try to get it from modules map
        if (loadedModule) { // is present
          parameters[i] = loadedModule; // adopt it as parameter
        }
        else { // must load module
          unresolved.push({ // schedule this for later
            dependant: currentDependant,
            dependencies: dependencies,
            moduleFactory: moduleFactory
          });
          loadDependency(dependency);
          return null; // cancel define() as this is going asynchronous
        }
      }
      else {
        parameters[i] = dependency; // simply pass through anything else than string
      }
    }
    return parameters; // respond to calling define() with valid parameters
  };

This function evaluates the parameter array from dependencies. Its arguments are exactly the same as those of the calling define().

In best case all parameters can be evaluated to already loaded modules, then the define will succeed.
When one of the dependencies has not yet been loaded, it returns null, and the define will cancel. But before returning null it pushes the unresolvable definition onto a stack and calls loadDependency() for the missing dependency. This adds a script tag and thus goes asynchronous. Meaning the JS execution is over for now, and the browser will call us when the script is loaded.

Here is an implementation of loadDependency().

  var loadDependency = function(dependency) {
    loadScript(dependency, function() { // finished-loading callback
      if (currentModule) { // dependency was loaded successfully
        modules[dependency] = currentModule; // store the module to map

        while (currentModule && unresolved.length > 0) { // try to define dependants
          var toResolve = unresolved.pop();
          currentDependant = toResolve.dependant;
          define(toResolve.dependencies, toResolve.moduleFactory);
          if (currentModule)
            modules[currentDependant] = currentModule;
        }
      }
      // else: another finished-loading callback will continue
    });
  };

The finished-loading callback does nothing when the definition of the dependency is delayed to a subsequent dependency (currentModule undefined). But when the dependency could be defined, it

puts the new module from currentModule variable into the modules map
resolves queued definitions by calling define() as long as modules can be created

The stack is the recursion that processes the whole dependency tree.

Because only one script tag is added to page head at a time, no concurrency problems around currentModule can occur.

That's all, from these building blocks an AMD loader can be implemented.
Surely some more logic and convenience will be needed, especially concerning script URLs and module names, and to detect cyclic dependencies.

All Together Now

Here comes my solution with an example. It should work in all browsers except IE-8 and older.

Strategy:

browser loads the define.js script through an HTML script tag
that script loads the script referenced in data-main attribute
the first define() call there triggers dependency resolution
any define() call always checks the modules map for dependencies already loaded before actually loading one
any CSS will be loaded without waiting for its arrival (CSS is recognized by extension .css)
any JS will be loaded with a registered finished-loading listener, but only the first JS dependency will be loaded, not all
- the current dependency parameters are pushed onto a stack
- then the running define() is aborted
the dependency calls define() as soon as it was loaded
when it has dependencies, it goes the same way as 3. went, but when not, it puts its module into the global currentModule variable
the finished-loading listener checks whether a module was defined and puts the currentModule into global modules map when defined
the finished-loading listener again calls define() with the dependency parameters taken from stack, actually starting from 4. again.

Features:

everything is a define(), no require() here
checks for dependency cycles and throws an error in such a case
can load CSS
can load JS that was not written as AMD define; to avoid global variables completely, use the define.getLoadedModule override like shown in demo.js
dependency module names are paths relative to the data-main script (demo.js), or, when data-path was declared, relative to that path (see demo.html example below)
data-main attribute in demo.html points to the main script
data-allow-cyles attribute enables loading when the dependency graph has cycles (default does not allow)
data-logging attribute switches on logging (default is off)
go to some JS minifier and compress it, it has 2395 bytes

You can not load external scripts with this loader! It is just for resources relative to the calling HTML page.

What is the result of the example page below?
A centered title "INFO: Hello World" should be displayed on the page, below will be the HTML source of the page after all modules were loaded, that source is syntax-highlighted by highlight.js, then a dialog should appear saying "INFO: Hello World", and finally "INFO: Good Bye World" should appear as footer.
This is what is implemented in demo.js.

Example Directory Structure

html
- demo.html
- scripts
  - demo.js
js
- app
  - messages.js
  - ui
    - dialog.js
- lib
  - define.js
  - highlight.min.js
- util
  - output.js

The js directory is the JavaScript library root. With this AMD loader you will be forced to hold all your JS files under that root and reference it from all your web pages by relative paths.

Example Files

html/demo.html

<!DOCTYPE html>
<html>
<head>
  <meta charset="UTF-8">
</head>

<body>

  <script type="text/javascript"
      src="../js/lib/define.js"
      data-main="scripts/demo.js"
      data-path="../js/"
      data-logging="true"
      data-allow-cycles="false"
  >
  </script> 

</body>
</html>

html/scripts/demo.js

/**
 * Script referenced in "data-main" attribute.
 */

define(
  [
    "app/ui/dialog.js",
    "util/output",    // trailing .js is optional
    "app/messages.js",
    "lib/highlight-default.min.css",
    "lib/highlight.min.js"
  ],
  function(
      print,
      out,
      msgs)
  {
    out(msgs.hello);
    
    var scriptCode = document.createElement("pre");
    scriptCode.textContent = document.documentElement.outerHTML;
    document.body.appendChild(scriptCode);
    scriptCode.setAttribute("class", "html");
    hljs.highlightBlock(scriptCode);
  
    print(msgs.hello);
    
    out(msgs.goodBye);
  }
);

This variant is using the global variable hljs.
If you want to avoid global variables in any case, use the one to the right.

/**
 * This is for those who dislike global variables.
 * Pass highlightjs, which is not an AMD module,
 * to the main script as an argument (hljs)
 * by overriding the define.getLoadedModule() function.
 */
(function() {
  var getLoadedModule = define.getLoadedModule;

  define.getLoadedModule = function(dependency) {
    if (dependency === "lib/highlight.min") // dependencies do not have ".js"
      return hljs;
    return getLoadedModule(dependency);
  };
}());

define(
  [
    "app/ui/dialog.js",
    "util/output",
    "app/messages.js",
    "lib/highlight-default.min.css",
    "lib/highlight.min.js"
  ],
  function(
      print,
      out,
      msgs,
      cssDummy,
      hljs)
  {
    out(msgs.hello);
    
    var scriptCode = document.createElement("pre");
    scriptCode.textContent = document.documentElement.outerHTML;
    document.body.appendChild(scriptCode);
    scriptCode.setAttribute("class", "html");
    hljs.highlightBlock(scriptCode);
  
    print(msgs.hello);
    
    out(msgs.goodBye);
  }
);

js/app/messages.js

define(
  /* comment this in to test cycle detection
  [
    "app/ui/dialog"
  ],
   */
  {
    infoHeader: 'INFO',
    hello:      'Hello World',
    goodBye:    'Good Bye World'
  }
);

js/app/ui/dialog.js

define(
  [
    "app/messages"
  ],
  function(appMessages) {
    return function(message) {
      alert(appMessages.infoHeader+": "+message);
    };
  }
);

js/util/output.js

define(
  [
    "app/messages"
  ],
  function(appMessages) {
    return function(message) {
      var paragraph = document.createElement("div");
      paragraph.innerHTML = "<i>"+appMessages.infoHeader+":</i> <b>"+message+"</b>";
      paragraph.style.cssText += "text-align: center; font-size: 200%;";
      document.body.appendChild(paragraph);
    };
  }
);

js/lib/highlight.min.js
js/lib/highlight-default.min.css

Download both files from highlightjs website.

The Loader Implementation

js/lib/define.js

"use strict";

/**
 * This is a very basic AMD script loader.
 * It does not load scripts concurrently (although it loads asynchronously).
 * It has no require() function, just define(dependencies, moduleOrFactory).
 * It does not allow to pass the module name as first parameter to define().
 * The extension ".js" in module names is optional.
 * 
 * CAUTION:
 * Call define() only once from a HTML page!
 * The loader assumes that define(), after the initial call, is called
 * by modules only. Calling it a second time could break module assignment.
 * The same could happen when calling define() more than once from a module!
 * 
 * See AMD specification at https://github.com/amdjs/amdjs-api/wiki/AMD
 * 
 * @param context the execution context for any factory-function given by a module,
 *     in other words this will be the first parameter to factoryFunction.apply().
 */

var define = (function(context)
{
  /* Base URL of the directory where modules are in, set by data-path attribute. */
  var baseJsUrl = "";
  
  /* The currently loaded module as left by define(), object or function. */
  var currentModule;
  var currentModuleName = "anonymous script";
  
  /* Module map with key = script path, value = loaded module. */
  var modules = {};
  
  /* List of modules still to load. */
  var unresolved = [];
  
  /**
   * This is the returned define() implementation. It resolves given dependencies
   * and stores the defined module into currentModule variable when no
   * unresolved dependencies existed, else leaves it undefined and defers
   * module definition until resolution of the depencency (asynchronous).
   * @param dependencies array of String containing the module identifiers
   *     this module depends on.
   * @param moduleOrFactory an object representing the module, or a factory function
   *     creating a function or object representing the module,
   *     can also be undefined to just load some scripts.
   */
  var define = function(dependencies, moduleOrFactory) {
    var loadingModuleName = currentModuleName;
    log("Define called by '"+loadingModuleName+"'");
    
    var parameters = [];
    if (dependencies.length) /* the first parameter is an array */
      parameters = evaluateParametersFromDependencies(dependencies, moduleOrFactory);
    else if ( ! moduleOrFactory ) /* happens when no dependencies given */
      moduleOrFactory = dependencies; /* parameter in fact is the module */
    else
      throw "Dependencies must be array of strings, but was: "+dependencies;
    
    var dependenciesResolved = (parameters !== undefined);
    if (dependenciesResolved)
      if (typeof moduleOrFactory === "function")
        currentModule = moduleOrFactory.apply(context, parameters);
      else
        currentModule = moduleOrFactory;
    else
      currentModule = undefined;
  
    log("Define of '"+loadingModuleName+"' "+(dependenciesResolved ? "finished" : "was delayed"));
  };

  /**
   * Tries to pick up all dependencies from module map and returns a ready-made
   * parameter array when this succeeds, else returns undefined and asynchronously
   * loads the dependency.
   * @param dependencies same as in define().
   * @param moduleOrFactory same as in define().
   * @returns null when not all modules are present, else a parameter
   *     array containing all dependencies as modules.
   */
  var evaluateParametersFromDependencies = function(dependencies, moduleOrFactory) {
    log("Module '"+currentModuleName+"' has dependencies "+dependencies);
    
    var parameters = [];
    for (var i = 0; i < dependencies.length; i++) {
      var dependency = dependencies[i];
      if (typeof dependency === "string") { /* dependency is an identifier */
        dependency = cutExtension(dependency);
        var loadedModule = modules[dependency]; /* try to get it from modules map */
        if (loadedModule) /* is present*/
          parameters[i] = loadedModule; /* adopt it as parameter */
        else if (continueDeferred(dependency, dependencies, moduleOrFactory))
          return undefined; /* will be continued by a finished-loading event */
        else
          parameters.push({}); /* dummy module when finished-loading can not be caught */
      }
      else {
        parameters.push(dependency); /* simply pass through anything else than string */
      }
    }
    return parameters; /* respond to calling define() with valid parameters */
  };
  
  var continueDeferred = function(dependency, dependencies, moduleOrFactory) {
    var url = baseJsUrl + dependency;

    if (dependency.lastIndexOf(".css") >= 0) {
      loadStyle(url);
      modules[dependency] = {}; /* dummy module for CSS */
    }
    else { /* is a JavaScript */
      var schedule = { /* schedule for later */
          moduleName: currentModuleName,
          dependency: dependency,
          dependencies: dependencies,
          moduleOrFactory: moduleOrFactory
      };
      if (noCycle(schedule)) {
        unresolved.push(schedule);
        loadScriptDependency(dependency, url);
        return true; /* cancel define() as this is going asynchronous */
      }
    }
    return false; /* do not wait for finished-loading */
  };

  var noCycle = function(schedule) {
    for (var i = unresolved.length - 1; i >= 0; i--) {
      if (unresolved[i].dependency === schedule.dependency) {
        var message = "ERROR: Cycle detected, or more than one define() calls, when trying to load "+schedule.dependency;
        for (var j = unresolved.length - 1; j >= i; j--)
          message += " after "+unresolved[j].dependency;
        
        log(message);
        if (define.allowCycles)
          return false;
      
        /* throw exception and prevent subsequent work */
        unresolved = [];
        throw message;
      }
    }
    return true;
  };
  
  /**
   * Loads given dependency asynchronously, installs a finished-loading listener.
   * Resolves unresolved modules when loading finished and a module was created.
   * @param dependency the identifier of the module to load.
   * @param scriptUrl the URL from where to load the dependency, with prepended base-URL.
   */
  var loadScriptDependency = function(dependency, scriptUrl) {
    if (scriptUrl.lastIndexOf(".js") < 0)
      scriptUrl = scriptUrl+".js"; /* append extension when necessary */

    currentModule = undefined; /* for the case that define() won't be called */
    currentModuleName = dependency;
    var currentUnresolved = unresolved.length; /* remember current state */
    log("Deferring to receival of script: "+dependency);
    
    loadScript(scriptUrl, function() { /* finished-loading callback */
      /* define() should have been called by the module, currentModule should have a value */
      log("Finished loading "+dependency);
      
      /* when no unresolved dependency was pushed, and also no module was defined,
         map a dummy module so that loading can continue */
      if ( ! currentModule && currentUnresolved === unresolved.length)
        currentModule = define.getLoadedModule(dependency);
      
      putModuleAndResolve(dependency); /* try to pop unresolved */
    });
  };
  
  var putModuleAndResolve = function(dependency) {
    if (currentModule) { /* a module was loaded successfully */
      modules[dependency] = currentModule; /* store the module to map */
        
      if (unresolved.length > 0) {
        var toResolve = unresolved.pop();
        currentModuleName = toResolve.moduleName;
      
        define(toResolve.dependencies, toResolve.moduleOrFactory);
      
        putModuleAndResolve(toResolve.moduleName);
      }
    }
    /* else: define() had to load another dependency */
  };

  /**
   * Loads a script by appending a script element to HTML document head
   * and registering a load-finished callback. This is asynchronous.
   * @param scriptUrl the script to load.
   * @param scriptLoadedCallback the function to call when loading finished, can be null.
   */
  var loadScript = function(scriptUrl, scriptLoadedCallback) {
    var scriptElement = document.createElement("script");
    
    var finishedLoading = function(event) {
      if (event.type === 'load' || /* checks if script is loaded, seen in require.js */
            ("/^(complete|loaded)$/".test((event.currentTarget || event.srcElement).readyState)))
      {
        scriptElement.removeEventListener('load', finishedLoading, false);
        if (scriptLoadedCallback)
          scriptLoadedCallback();
      }
    };
      
    scriptElement.addEventListener('load', finishedLoading, false);
    scriptElement.type = "text/javascript";
    scriptElement.src = scriptUrl;
    
    document.head.appendChild(scriptElement);
  };
  
  /**
   * Loads a CSS file by appending a link element to HTML document head
   * and registering a load-finished callback. This is asynchronous.
   * @param styleUrl the CSS file to load.
   */
  var loadStyle = function(styleUrl) {
    var linkElement = document.createElement("link");
    linkElement.type = "text/css";
    linkElement.rel = "stylesheet";
    linkElement.href = styleUrl;
    document.head.appendChild(linkElement);
  };
  
  var cutExtension = function(dependency) {
    var extensionIndex = dependency.lastIndexOf(".js");
    if (extensionIndex > 0)
      return dependency.substring(0, extensionIndex);
    return dependency;
  };
  
  var log = function(message) {
    if (define.logging && typeof console !== "undefined")
      console.log(message);
  };


  /* initialize, executed once when this factory function is called */
  
  (function() {
    /* find "data-main" and other attributes in document */
    var scripts = document.getElementsByTagName('script');
    
    var mainAttributeValue;
    var loggingAttributeValue;
    var allowCyclesAttributeValue;
    for (var i = 0; ! mainAttributeValue && i < scripts.length; i++) {
      var script = scripts[i];
      mainAttributeValue = script.getAttribute("data-main");
      loggingAttributeValue = script.getAttribute("data-logging");
      allowCyclesAttributeValue = script.getAttribute("data-allow-cycles");
      baseJsUrl = script.getAttribute("data-path");
    }
  
    if (mainAttributeValue) {
      if ( ! baseJsUrl ) {
        var lastSlashIndex = mainAttributeValue.lastIndexOf("/") + 1;
        baseJsUrl = mainAttributeValue.substring(0, lastSlashIndex);
      }
      loadScript(currentModuleName = mainAttributeValue); /* is asynchronous */
    }
    
    if (baseJsUrl && baseJsUrl.length > 0 && baseJsUrl.substring(baseJsUrl.length - 1) !== "/")
      baseJsUrl = baseJsUrl+"/";
  
    define.logging = (loggingAttributeValue === "true") ? true : false;
    define.allowCycles = (allowCyclesAttributeValue === "true") ? true : false;
  
    log("main="+mainAttributeValue);
    log("path="+baseJsUrl);
    log("allowCycles="+define.allowCycles);
    
    /**
     * Default fallback for loaded non-AMD modules, to be overridden.
     * @param dependency the non-AMD script loaded, without trailing ".js".
     * @return the loaded module object or function to store to modules map.
     */
    define.getLoadedModule = function(dependency) {
      if (dependency.indexOf("jquery") >= 0 && typeof $ !== "undefined")
        return $;
      return {}; /* empty dummy object */
    };
  }());
  
  
  return define; /* factory result */
  
}(typeof window !== "undefined" ? window : this)); /* pass a global JS context */

I hope I demystified AMD a little, and showed how JS code can be written in a readable and well-documented manner. This is what we need: simple and clear solutions. Long names for variables and functions. Header comments for modules, functions and instance variables. Code for humans, not for interpreters.

Have fun!

fritzthecat-blog

Blog-Archiv

Samstag, 28. Februar 2015

Yet Another JavaScript AMD Loader