Skip navigation
All People > jpeterson-esristaff > Joshua Peterson's Blog > 2017 > March > 07

Joshua Peterson's Blog

March 7, 2017 Previous day Next day
First, the links:

Crowdsource Reporter Repo: https://github.com/esri/crowdsource-reporter

My Migration Repo: https://github.com/jpeterson/crowdsource-reporter

 

Where we are

This is the sixth part in a seven-part blog series called How to Tame your Web App. Here’s where we are:

  1. The Plan
  2. ES6 and Modern Modules
  3. Module Bundling 101
  4. Module Bundling 201 - Optimization
  5. Linting
  6. UX and DX: The Best of Both Worlds (you are here)
  7. Legacy Support - coming soon

 

Today's place: Rotterdam

Header: De Kuip, home of Feyenoord Rotterdam

 

Erasmusbrug

Erasmusbrug, Rotterdam

 

The Plan

Optimize for both Developer Experience and User Experience. We'll create a second webpack config file so we can target our development environment and the production environment separately.

 

The Reason

We want a great developer experience as well as the ability to deploy our code in the most optimized way possible. These two things are not always possible without branching the build pipeline.

 

The Commits

https://github.com/jpeterson/crowdsource-reporter/pull/7/commits

 

Webpack offers a few options for developer tooling. We’ll use webpack-dev-server, but you can read this doc for an explanation of the other options. We’ll also use HtmlWebpackPlugin to send our index.html through Webpack’s bundling process.

 

> npm install webpack-dev-server html-webpack-plugin –-save-dev

 

All we need to do with the dev-server for now is add a new npm script to run our app in development mode:

"start": "webpack-dev-server --open"

 

Running webpack-dev-server is now as easy as:

 

> npm start

 

Next we’ll use HtmlWebpackPlugin in our config file:

new HtmlWebpackPlugin({
template: 'index.html',
inject: false
})

 

We can do many more powerful things with this plugin by injecting templated values into our index.html file at build time – but we’ll just use it to copy the file into our output folder for now.

 

Important note – we need to specify inject: false to tell Webpack not to inject a script tag into our html (as it does by default). We need to do this because we have to invoke our code using a require() statement to placate the Dojo loader.

 

Markthal (Rotterdam)

The awesome Markthal in Rotterdam

 

At this point we’ll duplicate our webpack.config.js and create a webpack.production.config.js, then modify our build script to reference this new config file:

"build": "webpack -p --config webpack.production.config.js"

 

Notice I slipped that -p into our build script. This is the production flag which tells webpack to run its awesome optimization routine on all our code and assets.

 

From this point on, we’ll be managing two webpack configs. It is highly recommended that you take advantage of the fact that these config files are simple CommonJS modules which can be composed rather easily. For this exercise though, I’m going to keep them separate.

 

The final curveball Dojo threw at me was internationalization (i18n) support. Crowdsource Reporter has great i18n support via the dojo/i18n! Dojo Loader plugin. I fought with this for a bit before realizing I needed to just let it do its thing. Handling i18n files in the Webpack build would mean packaging up every language together for download by every user – not ideal. Instead, I used a Webpack plugin called CopyWebpackPlugin to just pipe those files as-is into our /dist folder and let Dojo handle them at runtime.

 

I only added the CopyWebpackPlugin to my production config file (since the dev server runs at the root of my project and can directly access the /nls folder):

new CopyWebpackPlugin([{ from: 'js/nls', to: 'js/nls' }])

 

Montevideo (Rotterdam)

Montevideo, Rotterdam

 

So now we have two fully functioning environments for our app. To test that your prod build is working, run your build, then copy the /dist folder into an unrelated web server location and see if it runs. If you run into issues, it’s likely that you forgot to output some files/folders into your /dist folder.

 

I know I didn't talk much about why we want to use webpack-dev-server... I'll just point you to the doc page and have you check out things like hot module replacement and stats.

 

Alright, onto the last post! Legacy Support - coming soon (no, the irony is not lost on me).

First, the links:

Crowdsource Reporter Repo: https://github.com/esri/crowdsource-reporter

My Migration Repo: https://github.com/jpeterson/crowdsource-reporter

 

Where we are

This is the fifth part in a seven-part blog series called How to Tame your Web App. Here’s where we are:

  1. The Plan
  2. ES6 and Modern Modules
  3. Module Bundling 101
  4. Module Bundling 201 - Optimization
  5. Linting (you are here)
  6. UX and DX: The Best of Both Worlds
  7. Legacy Support - coming soon

 

Today's place: Yosemite National Park

Header: Rainy day in Yosemite driving down Tioga Road from Tuolumne Meadows.

 

Half Dome

Half Dome, bathed in alpenglow

 

Linting our Build 

 

The Plan

Implement linting (we’ll use ESLint) in our build process.

 

The Reason

Linting enhances our ability to catch bugs before they reach production.

 

The Commits

https://github.com/jpeterson/crowdsource-reporter/pull/6/commits

 

As a preface, it's highly likely you already have a linter installed in your editor. While inline-linting is absolutely useful and recommended, adding it to our build cycle makes it a bit more official, and forces us to play by the rules. I prefer ESLint because it is pluggable, highly configurable, and doesn’t promote any particular coding styles. I encourage you to read more about ESLint’s philosophy.

 

So how do we add ESLint to webpack? You guessed it – another loader.

> npm install eslint-loader –-save-dev

 

We’ll add it to webpack.config.js just like any other loader – but using a new option pre, which will ensure ESLint gets run on our code before webpack does any transformations.

 

Side note: You’ll notice I configured eslint-loader to output a report file (which contains all the broken rules). This is useful, but after going through the process I’d recommend just doing this via the command line so you don’t need to involve the rest of the build process just to find/fix the cacophony of rules your app will be breaking if this is the first time you’ve implemented a linter.

 

One of the most amazing features of ESLint is its ability to automatically fix some issues. The Rules page on the ESLint website has a wrench icon next to the rules that can be auto-fixed.

 

Crowdsource Reporter didn’t have any linting before, so I added an .eslintrc file to tell both my editor and webpack which rules to follow, then ran eslint from the command line to see what I was working with. Here is what my .eslintrc looks like.

> eslint -o ./temp/eslint-log.html

 

17,167 problems reported. Wow.

 

That’s kind of a shocking number – but chill... the vast majority of those are because the original developers simply had a different preference when it comes to indent-style and quote-style. Luckily, both of those are auto-fixable. Let’s run eslint again, but use the --fix flag.

> eslint --fix -o ./temp/eslint-log.html

 

187 problems reported. Talk about efficiency!

 

Most of the remaining errors break these 3 rules:

 

Grand Canyon of the Tuolomne

Tuolumne River, in the Grand Canyon of the Tuolumne

 

At this point, you must decide what it means to “fix” these remaining issues. In the case of Crowdsource Reporter, it was a mix of manually fixing some things (like when modules were being imported but are never actually used), or simply relaxing my ESLint rules a bit (like saying it’s okay to use $ as a global, and it’s okay to handle errors with console.warn and console.error). I relaxed those rules by modifying my .eslintrc.

 

I probably could have spent more time standardizing the codebase by applying more rules – but that was outside the scope of this exercise.

 

Now that we have a clean slate with no errors being reported, let’s modify our webpack.config.js again.

{
  test: /\.js$/,
  enforce: 'pre',
  exclude: /node_modules/,
  use: {
    loader: 'eslint-loader',
    options: {
      failOnWarning: false,
      failOnError: true
    }
  }
}

 

With that, we stop outputting a report on every build, and tell Webpack that the build should fail if an ESLint error is reported.

 

So now that we've added a layer of error-checking to our build/deployment process, let's push on to the 6th part of the series where we'll focus on user and developer experience - UX and DX: The Best of Both Worlds.

First, the links:

Crowdsource Reporter Repo: https://github.com/esri/crowdsource-reporter

My Migration Repo: https://github.com/jpeterson/crowdsource-reporter

 

Where we are

This is the fourth part in a seven-part blog series called How to Tame your Web App. Here’s where we are:

  1. The Plan
  2. ES6 and Modern Modules
  3. Module Bundling 101
  4. Module Bundling 201 - Optimization (you are here)
  5. Linting
  6. UX and DX: The Best of Both Worlds
  7. Legacy Support - coming soon

 

Today's place: Austria

Header: Overlooking the Danube from the hills above Dürnstein.

 

Stephensdom

Stephensdom, in Stephensplatz, Vienna

 

Style/Asset Bundles 

 

The Plan

Our app still uses CSS and images the same way it always did – most likely via <link> tags in our index.html. Let’s bundle that up so it can be shipped in a more optimized package.

 

The Reason

CSS and image files are inefficient in the browser for the same reason JS files are. We’ll bundle and optimize our CSS so that it can be delivered more quickly to our users.

 

The Commits

https://github.com/jpeterson/crowdsource-reporter/pull/4/commits

 

For CSS, all we need to do is remove the <link> tag(s) in our index.html and instead import our CSS files via our JS entry file. In the future, you could refactor your CSS in such a way that you import only the styles needed for a module in that module – therefore making your bundling process even more streamlined.

 

Next, let’s address image files. This will require 2 more loaders:

> npm install file-loader url-loader –-save-dev

 

file-loader is dead simple – it takes a file and puts it in your output folder.

 

url-loader does a similar thing, but it turns the files into Data URIs instead. This is useful for smaller images that can be turned into Base64 encoded strings in our code, eliminating a network request. Base64, however, is only more efficient up to a certain file size – this is where the url-loader’s limit option helps by falling back to file-loader for files larger than a given size.

 

We’ll use url-loader as our “catch all” loader. Instead of configuring it to only include certain file types, we’ll tell it to exclude the file types we’re already handling with other loaders, and it will attempt to load everything else. Here is the config I used.

 

Quick note: you may not want to configure this “catch all” loader until you’ve taken account of all the file types your app uses – seeing Webpack build errors for unhandled file types will force you to think about how you want to load them.

 

Go ahead and fire off another build:

> npm run build

 

It works, right? If you look at your network request, you might feel like we've introduced some black magic. There are no CSS files being loaded, but your styles are still there… The reason for this is that Webpack is loading your CSS into JS files and injecting the styles for you. This may or may not be ideal for you. The biggest drawback is that it inhibits the browser’s ability to load CSS asynchronously and in parallel – so your page won’t be styled until all your JS is loaded. Let’s add one more step to get a real CSS file.

 

This time, instead of a loader, we’ll be using a Webpack Plugin:

> npm install extract-text-webpack-plugin –-save-dev

 

 Add the plugin to your config file (remember that you need to import it). Then tell your css-loader to use ExtractTextPlugin.extract.

 

Now simply add a <link> tag back into your index.html, referencing the name of the css file in your config.

 

That’s it! You now have an optimized CSS bundle being served as a single file. If all your CSS was in a single file to begin with, this might seem like a waste of time – but it’s not. You’ve laid the foundation to add more optimizations (minification, PostCSS, etc…) and to write your CSS in a more modular fashion.

 

That deserves a photo break.

 

Burg Aggstein

Burg Aggstein and the Danube in Wachau, Austria

 

Vendor Bundle

 

The Plan

To this point, we are still using vendor libraries by including them as <script> tags in our HTML – we will use Webpack to bundle them in a more intelligent manner.

 

The Reason

A single vendor bundle means fewer network requests; we can also take advantage of browser caching to serve this bundle faster when it doesn’t change.

 

The Commits

https://github.com/jpeterson/crowdsource-reporter/pull/5/commits

 

A quick note about this section: due to the issues with the Dojo Loader mentioned above, our vendor bundle will not include any Dojo-loaded resources (including the JSAPI). This is okay! The JSAPI is already optimized and served out via a speedy CDN, so let it do its thing. If you have a custom build of the JSAPI - that should work exactly the same way.

 

Again, this process will be specific to your app, but for Crowdsource Reporter, the 3rd party libraries have been downloaded and placed in a folder called /vendor, then included in the app via <script> tags in index.html.

 

First thing to do is start managing those dependencies properly using NPM. I’ll wait while you find all your dependencies on npmjs.com and install them (remember the --save argument). Make sure you download the same versions you were using previously! We don’t want to confuse some library’s breaking changes with Webpack issues.

 

Here is the dependency list for Crowdsource Reporter (excluding the JSAPI):

jQuery, Moment.js, Bootstrap, Bootstrap Datepicker, Bootstrap TouchSpin

 

Hopefully your dependency tree is simpler than this one – this particular app has a bit of a mess of dependencies. Dependencies 3, 4, and 5 all depend on jQuery (and check for its existence in different ways), and dependencies 4 and 5 depend on Bootstrap. Fun stuff.

 

My first inclination was to use webpack’s ProvidePlugin, which checks a module to see if it needs some external dependency (that you define), and makes that dependency available to it. So, I thought I could do something like this: 

new webpack.ProvidePlugin({  
  '$': 'jquery', 
  'jQuery': 'jquery'
})

 

That didn’t work. The reason is that Webpack re-imports the jQuery module any time it's needed as a dependency, but the way these jQuery plugins work is by attaching their functionality onto the jQuery reference they have. This works well in a setting where you have a single jQuery global reference, but the concept breaks down when you start passing a bunch of different jQuery references around. We need to provide the same instance of jQuery to our dependent modules.

 

This turned out to be the most painstaking part of this entire migration process – but eventually I figured it out. I’ll try to turn my curse-word-laden notes into a succinct explanation of the issues I had and how to resolve them .

 

Let me start by showing you the 4 lines of code I eventually needed: 

import jquery from 'script-loader!jquery';
import bootstrap from 'bootstrap';
import bootstrapDatetimepicker from 'imports-loader?moment,this=>window,define=>undefined,exports=>undefined!eonasdan-bootstrap-datetimepicker';
import bootstrapTouchspin from 'bootstrap-touchspin';

 

script-loader is a Webpack loader that sort of acts as a last line of defense against legacy “modules” (which aren't really modular at all). It essentially injects your module as if it were a <script> tag in an html file. I’m not going to blame jQuery for this – rather the jQuery plugin architecture. Anyway, the first line puts jQuery and $ in the global namespace. Bootstrap and Bootstrap TouchSpin are happy now – they don’t throw errors on load, and they function just fine in the app. Almost there.

 

Bootstrap Datepicker, on the other hand, is a bit cantankerous. This library tries to be intelligent about the module environment it is being used within, but in this case it just screws up what we’re trying to do. I ended up using imports-loader (which lets you pass specific "global" variables to a module) to trick the library into thinking we don’t know about modules: this=>window,define=>undefined,exports=>undefined. This made the library play nicely with the global jQuery. Lastly, we also pass it a reference to the moment module (not a global one, one that will get imported by Webpack).

 

Phew. That was a royal pain, but all our 3rd party libraries are now being loaded successfully by Webpack!

 

Melk Abbey

Spiral staircase inside Melk Abbey outside Melk, Austria 

 

Uuuuuugh. One more curveball from Bootstrap Datepicker. The npm package doesn’t ship with any CSS (just Sass). It would be overkill right now to involve a Sass compiler in our app just to support what I would consider an edge-case… so I’ll just import the CSS file that was jammed in that /vendor folder from before.

 

In case you think I forgot Moment.js, I didn’t. I just didn’t mention it yet because it’s a proper module that can be imported and used as necessary – none of this global namespace nonsense.

 

Running the build should work now, and looking closely we should see all our vendor code bundled up into bundle.js.

 

Let's stop and see where we are. We’ve now got our vendor dependencies being loaded by Webpack, enabling all its optimization goodies. That’s a win all by itself – but let’s go one step further and split the vendor code into its own bundle. This will let us take advantage of browser caching for that hefty vendor code that doesn’t change very often (i.e. your users won’t even need to download this code on repeat visits to your app).

 

To add this optimization, let’s head back to our webpack.config.js. We’re going to modify 3 sections: entry, output, and plugins.

 

entry

We need to add a second entry point that will correspond to our vendor bundle. It took me a while to figure this out, but we essentially need to just copy all the paths we were importing here into our config file – inline loaders and all. Don’t forget to throw moment in here as well.

 

Note that Webpack treats separate entry points as totally separate dependency trees – this is why we need to include that custom inline loader logic. It’s also why we will need to involve another webpack plugin soon... 

entry: {      
  bundle: './js/bootstrapper.js',
  vendor: [
    'script-loader!jquery',
    'moment',
    'bootstrap',
    'bootstrap-touchspin',
    'imports-loader?moment,this=>window,define=>undefined,exports=>undefined!eonasdan-bootstrap-datetimepicker'
  ]
}

  

output

Webpack supports a few placeholders in our config file to insert dynamic strings. We’ll use [name] here to make our output bundle match the name of the entry point. 

output: {
  path: path.resolve(__dirname, 'dist'),
  filename: '[name].js',
  publicPath: './',
  libraryTarget: 'amd'
}

  

plugins

If we run the build now, we get 2 separate bundles, which is what we want. But since Webpack treats the 2 entry points as separate dependency trees – it still thinks all our vendor dependencies are needed within the primary bundle. This is where the CommonsChunkPlugin is useful. This plugin will make sure the code in our vendor bundle isn’t duplicated in our primary bundle. Read over this page if you want to know more about how this plugin works. CommonsChunkPlugin is part of webpack core, so don’t forget to import Webpack at the top of your config. 

new webpack.optimize.CommonsChunkPlugin({ name: ['vendor'] })

 

Lastly, make sure you include the new vendor.js file in your index.html, and remove the import statements from your app’s entry file (bootstrapper.js in my case).

 

Now you’ve got a dedicated bundle for your 3rd party code!

 

On to part 5: Linting.

First, the links:

Crowdsource Reporter Repo: https://github.com/esri/crowdsource-reporter

My Migration Repo: https://github.com/jpeterson/crowdsource-reporter

 

Where we are

This is the third part in a seven-part blog series called How to Tame your Web App. Here’s where we are:

  1. The Plan
  2. ES6 and Modern Modules
  3. Module Bundling 101 (you are here)
  4. Module Bundling 201 - Optimization
  5. Linting
  6. UX and DX: The Best of Both Worlds
  7. Legacy Support - coming soon

 

Today's place: Colorado

Header: San Juan Mountains above Ouray, Colorado.

 

Mt. Sneffels

Enjoying the view of Mount Sneffels one of Colorado's 53 fourteeners.

 

Module Bundling 101 

 

The Plan

Implement a module bundler (we’ll use Webpack 2 here) to turn our beautifully architected and readable code into a slim, optimized package for our users to consume.

 

The Reason

Modules are an amazing tool for architecting applications, but the way the web works today, loading a bunch of files in a web app is detrimental to the performance of the app. A module bundler will allow us to maintain our great developer experience while still providing the most optimized user experience possible by building, or bundling, our app for production.

 

The Commits

https://github.com/jpeterson/crowdsource-reporter/pull/3/commits

 

Before we get started, I’ll admit that this is by far the most involved step. Webpack is designed to be modular and leverage the work of others where possible – this makes it powerful, but can also make the setup process seem confusing. If you’ve worked with the Dojo Build system before, you know build/bundle systems are inherently complex beasts. If you get stuck, ask for help!

 

First off, we’ll be installing some npm modules locally with our project, so we need a package.json. Let's scaffold out a package.json with a simple command in the project root:

> npm init

 

Next, install Webpack:

> npm install webpack --save-dev

 

Webpack processes our assets with the help of loaders. These are simply npm packages that handle specific responsibilities within the webpack bundling process. For now, we will install loaders to handle HTML and CSS files (Webpack can handle JS on its own for now):

> npm install html-loader style-loader css-loader --save-dev

 

Next, we’ll create a configuration file for Webpack. If you’ve used Grunt or Gulp before, this is very similar to a Gruntfile.js or Gulpfile.js. The default name for a Webpack config file is webpack.config.js. Create this file in the project root and copy the contents of this file into yours. You can keep everything the same except for the entry property – this will need to point to the entry point of your app.

 

The next step is going to be a little more specific to the structure of your app. We need to modify our index.html and entry file to be more JS-centric (i.e. don't invoke your JS from index.html). In the case of the Crowdsource Reporter, here was the situation:

 

First, here’s my commit diff for reference. And a photo.

 

Dallas Peak

Dallas Peak, one of Colorado's 637 thirteeners.

 

Here’s the scenario in the existing app: index.html contained a require() which imported bootstrapper.js (the entry file in this app). It then instantiated a new instance of that module. Finally, it called the startup() method on bootstrapper.js.

 

Instead, we need index.html to simply require our bundle.js (the output from Webpack). We then need our entry file to invoke itself. In this case, I named the Dojo module (Bootstrapper) and invoked the constructor and startup methods within bootstrapper.js.

 

While your setup may be slightly different, the basic idea is that you need to kick off your app from within the JS file you define as your entry point – not from index.html.

 

At this point, we might be done. However, you’re likely using the Dojo Loader’s dojo/text! plugin, which Webpack isn’t aware of. This is why we installed Webpack’s html-loader earlier – it does almost exactly the same thing as the Dojo Loader plugin. One beautiful thing about Webpack loaders is that when we configure them properly in webpack.config.js, we don’t need to reference them in our JS, so it’s a simple matter of finding and replacing “dojo/text!” with “” to get what we need.

 

To avoid overcomplicating things, I'll address the externals section of the Webpack config by saying that it tells Webpack to ignore all modules beginning with dojo, dojox, dijit, and esri in the bundling process, and assume that they will be available for loading at runtime. If you want to know more about this - read the following tangential section...

 

<-- Begin Tangent -->

 

Okay – in some ways, this is the crucible of this entire exercise. Getting Webpack to play nicely with the Dojo Loader. The problem this section of the configuration file addresses is that the Dojo Loader does not implement the AMD standard like most module loaders (including Webpack), so lots of issues arise when attempting to run Dojo and JSAPI modules through the Webpack (or any other module loader’s) bundling process.

 

Luckily, a lot of very smart people in the ArcGIS developer community have come up with some creative ways to work around the Dojo Loader. The most succinct illustration of this problem and the various solutions comes from Tom Wayson’s blog post: Using the ArcGIS API for JavaScript in Applications built with webpack. If you’re interested in the details of the problem and gaining a deeper understanding of the solutions and workarounds – start there, and make sure to click all the links.

 

A few shoutouts to those smart people, in case you don't dive into that blog post... 

tomwayson (Tom Wayson) · GitHub 

lobsteropteryx (Ian Firkin) · GitHub 

gund (Alex Malkevich) · GitHub 

Robert-W (Robert) · GitHub 

 

You stuck with me through all that? Here's a photo.

 

Denver, Colorado

The beautiful mile-high city of Denver, Colorado. As seen from my roof.

 

<!-- End Tangent -->

 

To get our app back to functional, we could simply run webpack from the command line. Instead though, let’s add it as a script to our package.json so we don’t have to remember any command line arguments later. For now, we’ll simply run webpack with no arguments by adding the following property to the “scripts” section of package.json:

"scripts": {    
  "build": "webpack"
}

 

The moment of truth! In your terminal, try running:

> npm run build

 

If you’re fist pumping at your awesome app right now, you had more luck than I did! You deserve a photo.

 

Waterfall   Waterfalls are awesome. Waterfalls in the Uncompahgre National Forest are the awesomest.

 

I ran into a few issues here. You may or may not have run into them as well, but I’m going to list them out (along with how I resolved them) just in case.

 

First, I missed a few modules when manually converting package names to relative paths. The webpack build nicely alerted me, so I just had to go fix them.

 

The second issue took much longer to figure out. Webpack successfully gave me a bundle, so that was positive. But when I opened my app in the browser, I saw this in the console: 

TypeError: 'caller', 'callee', and 'arguments' properties may not be accessed on strict mode functions or the arguments objects for calls to them

 

This error is complaining about things related to strict mode. I wasn’t even aware my code was being run in strict mode, so to Google I went. I found out that the ES6 Module spec requires code to be in strict mode – who knew? Given the fact that this is in spec, Webpack naturally adds use strict to any ES6 modules it finds (i.e. all our code).

 

I guessed that shouldn’t be a big problem though – strict mode is a good idea anyway, right? Well, apparently, Dojo doesn’t think so. The error I was seeing can be traced to the numerous places in the app where Dojo modules do inheritance the Dojo way, using this.inherited(arguments). It’s a known fact that Dojo’s inheritance model is straight-up incompatible with strict mode.

 

I have an admission of guilt: I cheated to fix this problem. I followed a hunch that the original developers had added this.inherited(arguments) more as a matter of practice than actual necessity. Therefore, I simply commented out all occurrences of that statement, and the app worked. I admit, this is a total cop out! But it is also a possible solution – if your app doesn’t need this type of inheritance, simply remove it.

 

Luckily, I did stumble across an alternative workaround using Babel. If you want/need to do this, jump to the Legacy Support section below to add Babel to your Webpack config, and include the additional preset es2015-without-strict.

 

That was it for me – after resolving those issues, my app was back to a fully functioning state. Step back and take some joy from what you have so far. Even if you stop here, you have:

  • Moved away from AMD to JavaScript Modules
  • Implemented some ES6 in your codebase (arrow functions, let/const, template strings, object shorthand)
  • foundational webpack setup, with most of your code being bundled into a single file.

 

I’d recommend taking account of your app here. Are there other ES6 features you could take advantage of?

  • Revisit our earlier discussion of those Dojo language functions – would you rather use idiomatic JavaScript?
  • Peruse the remaining Lebab transforms – would any of them improve your codebase?
  • If you aren’t using Dojo’s class system much, adopting JavaScript Classes might be an option

 

Once you’re happy with things, let’s move on. With this foundation, we can start unlocking some of the bonus features of modern web development!

 

Next up is part 4 of this series, Module Bundling 201 - Optimization.

First, the links:

Crowdsource Reporter Repo: https://github.com/esri/crowdsource-reporter

My Migration Repo: https://github.com/jpeterson/crowdsource-reporter

 

Where we are

This is the second part in a seven-part blog series called How to Tame your Web App. Here’s where we are:

  1. The Plan
  2. ES6 and Modern Modules (you are here)
  3. Module Bundling 101
  4. Module Bundling 201 - Optimization
  5. Linting
  6. UX and DX: The Best of Both Worlds
  7. Legacy Support - coming soon

 

Today's place: Jordan

Header: Olive trees along the countryside north of Amman.

 

AmmanAmman, Jordan (taken from Al Qasr Metropole Hotel)

 

Modern Modules, from AMD to ES6

The Plan

Rewrite all our AMD modules to use ES6 Module syntax.

 

The Reason

The ES6 Module specification has been approved and is part of JavaScript. While an ES6 Module loading spec has not been finalized – we can prepare for the future by writing code according to the new spec now and using a module bundler to fill the temporary gap.

 

The Commits

https://github.com/jpeterson/crowdsource-reporter/pull/1/commits

 

The tool we’re going to use here is called amd-to-es6. It simply converts AMD modules into ES6 modules via the command line (I’m assuming you have NodeJS already installed, and are at least familiar with npm).

 

It doesn’t get much easier than this. Simply:

> npm install amd-to-es6 –g

 

Then run the tool. In my case, the command was:

> amdtoes6 --dir js/ --out js/

 

Note that if your app uses Dojo’s Module Identifiers (i.e. the “package” property in your dojoConfig), you will need to update your new import statements to use relative paths instead of package names. The Crowdsource Reporter used this feature of the Dojo Loader, and here is the commit where I had to manually update module paths.

 

That’s it for the AMD to ES6 Modules step. Relatively painless, right? Unfortunately, since browsers don't support ES6 Modules yet, we’ve entered a dark time where our app is inoperable, which means we can’t really test our progress. Let’s trudge ahead; we’ll have a functioning app back up in no time.

 

Arch of Hadrian

Arch of Hadrian (in Jerash, Jordan)

Migrating to ES6 syntax

 

The Plan

Convert some of the more basic constructs in our application from ES5 to ES6.

 

The Reason

ES6 is the current version of JavaScript. Writing ES6 is delightful, and I think you’ll enjoy it.

 

The Commits

https://github.com/jpeterson/crowdsource-reporter/pull/2/commits

 

For this step, we’re going to use an awesome tool called Lebab (yes, that is Babel backwards). As the name aptly implies, this tool does the opposite of Babel – it transpiles your ES5 code into ES6. It works by running individual transforms on your code to address one ES6 feature at a time. You can pick and choose which transforms you want to run. Grab Lebab with a quick:

> npm install lebab -g

 

Next, we’ll run some transforms (if you're following along with my commits: my apologies for running all the transforms in one go… I highly recommend you run them one at a time and check the output to make sure things look okay).

 

Here is my recommended order:

  1. arrow
  2. let
  3. template
  4. obj-shorthand

 

Feel free to run any additional transforms, but I’ll stop there.

 

Kanefeh

Delicious Kanefeh in downtown Amman

 

I want to call out something Dojo-specific here. Dojo is over 10 years old, which means it does some things that were very necessary at the time, but have since been accomplished natively by JavaScript. Let’s call these “language functions” (hint, a lot of them live in dojo/_base/lang).

 

Probably the most infamous of these language functions is dojo/_base/lang:hitch, which helps handle scope. There are others though, like most of dojo/_base/array. Many of these functions were written when JS couldn’t do certain things (efficiently) on its own, but ES6 (and even ES5) has added basic support for a lot of them.

 

Understandably, Lebab doesn’t know about Dojo, so it can’t convert lang.hitch into an arrow function or array.forEach into a plain array.prototype.forEach. If you’re interested in teaching Lebab about Dojo, it would be pretty rad to build some Dojo-specific Lebab transforms… The Lebab project welcomes contributions.

 

If you are interested in converting some of these language functions, I would recommend putting a bookmark on it and continuing at least until we have a functioning app again, then revisit them.

 

Alright, we’ve brought our app up to date with the current version of JavaScript!

 

In the next post, we’ll get our app back up and running by translating our awesome ES6 modules into something browsers can understand.

 

On to part 3 of this series, Module Bundling 101.

First, a couple links:

Crowdsource Reporter Repo: https://github.com/esri/crowdsource-reporter

My Migration Repo: https://github.com/jpeterson/crowdsource-reporter

 

TL;DR

In this blog series, I go through the process of “migrating” an existing Dojo/JSAPI app to use a more modern tooling infrastructure. I did this for real, then wrote about my experience. I added in ES6 (including modules), Webpack, ESLint, Babel, and a few other goodies. If you want to skip my write-up and just look at the code – go here.

 

Since I'm a *GIS* developer...

I have to throw some geography around. I enjoy exploring new places in the real world just as much as I love exploring new ideas in web development. Each post in this series is going to have a destination. I'm gonna share photos I've taken there. It's gonna be sweeeet.

 

Today's place: Paris

Header: Paris from the terrace rooftop of Printemps Haussman.

 

Eiffel Tower

The Eiffel Tower

 

What's the point of this?

Web development is full of rainbows and unicorns these days. While attending conferences, reading blog posts, and scrolling your social media feed, it’s easy to get excited about the new ideas being implemented in our industry. Unfortunately, a lot of us who get excited by these ideas go back to our offices and must deal with more pressing matters; maintenance, deadlines, profitability, clients... the list goes on.

 

Adopting new ideas into one’s workflow is difficult – it requires a real effort, and there is always the possibility that it just won’t work. Most of us decide we’ll try something new on that next app. The promise of a more streamlined workflow is worth the risk. The reason I went through this exercise (and documented it) is because I think adopting modern tooling is a worthwhile endeavor, and I hope this will help give someone the inspiration to take the plunge.

 

One last note on the point of this exercise. This post was created as a companion to an Esri International Developer Summit technical session with Gavin Rehkemper in March 2017. We love tools and the modern web development ecosystem so much that we talk about it every year at Dev Summit, but we’ve received feedback that while people get excited by our talks, there is a large gap between some of the things we talk about and the “real world problems” people go home to. Hopefully this post will bridge that gap so everyone can take advantage of the modern web.

 

Grand Arche

La Grande Arche de la Défense (my favorite Parisien arche)

 

Aren't you just hyping this year's crop of shiny objects?

I hear this concern a lot. I pontificate about the modern web quite often – to ArcGIS users, to my Esri coworkers, sometimes even to my dog (if only he would embrace tree-shaking). There is a lot of negativity surrounding the proliferation of tools and frameworks on the web. While I empathize with the point being made there, I see this explosion of new things as a sign of progress. Sure, the waters are muddied, but it’s important to remember that these tools and frameworks (or at least the good ones) are aiming for the same target: to make the web a better place for both developers and users.

 

Anyway, the point I’m trying to make is that you shouldn’t get distracted by the names of the hottest new tools and frameworks. Rather, try to see through the kitschy names and shiny websites to the fundamental progress being made; things like writing idiomatic JS, faster websites and apps, a more efficient development cycle. Also try to remember what you’re doing – if you are building a robust web-GIS application, that is a far different endeavor than making a content-driven website, so you can ignore the folks yelling about how everyone is over engineering the simplest websites.

 

The LouvreThe Louvre

 

Get to the goods, what did you actually do?

I chose Esri’s popular Crowdsource Reporter web app for this exercise because it is a well-written Dojo/JSAPI centric application that is like those belonging to many Esri customers I have worked with. It is a medium-sized application that doesn’t use many tools (i.e. no build system, CSS Preprocessors, linting, etc), so it provides a relatively blank slate to work with.

 

My plan was to modernize the toolchain used with this application with a twofold goal: (1) make the developer experience more efficient and (2) optimize the user experience by increasing performance.

 

By “modernize”, here's what I mean:

  • Migrate from AMD to ES6 modules, and sprinkle in some other ES6 syntax
  • Add a module-bundling system (Webpack) to optimize everything the client’s browser will have to download
  • Implement intelligent linting (error-checking)
  • Branch our build workflow so that we can achieve the best possible developer experience while maintaining the most optimized production version of our app
  • Maintain support for legacy browsers

 

I did my best to approach this effort one piece at a time. I went through the whole process in a git repository, and mirrored the list above to Pull Requests. If you want to correlate the list to the actual commits I made, here’s a good place to start.

 

This was originally intended to be a single blog post, but I quickly realized that post would be way too long. So I made it a series instead. Here's how I ended up breaking it down:

 

  1. The Plan (you are here)
  2. ES6 and Modern Modules
  3. Module Bundling 101
  4. Module Bundling 201 - Optimization
  5. Linting
  6. UX and DX: The Best of Both Worlds
  7. Legacy Support - coming soon

 

If you’re ready to dive in, head to the second post in the series: ES6 and Modern Modules.

Filter Blog