Alfred - A Modular Toolchain for JavaScript

Amila Welihinda

What is Alfred? TL;DR

  • An alternative to boilerplates / starter kits
  • Better tooling, out of the box
  • A solution to brittle and complex JS infrastructure

The Background

Maintaining over 200 open source JavaScript projects for the last 6 years has exposed me to the best and worst parts of JavaScript infrastructure. JavaScript infrastructure shines when it comes to flexibility. They make few assumptions about the user of the tools, but they don't assume you'll be using other tools. Making no assumptions is nice because it allows users to use whatever tools they want. Infrastructure usually doesn't come with integration for other tools out of the box support so integration is usually added by 3rd party plugins. This model for infrastructure is convenient because it allows developers to write plugins which extend the functionality of the tools they use, therefore allowing the tools to be used in a much wider range of scenarios than anticipated by maintainers. The downside of this customizability is that the tools usually don’t work out of the box. This comes at a huge cost to the user experience of beginners who have never used that tool before.

The JS ecosystem responds to this complexity by creating ‘boilerplates’, or ‘starter kits’, which are essentially template projects that have their infrastructure preconfigured for the tools the template project supports. For example, electron-react-boilerplate is a boilerplate that configures electron, react, eslint, webpack, and jest. Boilerplates solve the out-of-the-box problem but they sacrifice extensibility because they don’t expect users to change the tools they come with or even change the configurations of those tools.

Alfred proposes a new solution that allows each tool to configure itself with respect to other tools. To better understand Alfred’s solution, it is important to dive deeper into the current problems of JS infrastructure.

The Problem

JS Tooling is Brittle

When trying new tools, newcomers often spend a significant amount of time configuring tools. The difficulty of configuring a tool or library can discourage the user from using the it alltogether. Even users of widely adopted libraries and tools tend to experience issues related to configuration. The tweet below describes the situation well:

"Looking at the issues on storybooks/storybook:

  • 2,479 total issues
  • 732 mention "webpack" (30%)
  • 428 mention "babel" (17%)

That's crazy! Other keywords that come up that often would be treated as requiring architectural change, but those are just for configuration."

It's a little mind-boggling how many issues are purely for dealing with Babel or Webpack configuration. And those are by far some of the most frustrating issues to debug when you do run into them. So much time wasted. Makes you really understand the "zero config" movement.

Incorrect, Suboptimal Infrastructure

One of the great strengths of JS tooling is its customizability. This customizability allows JS tools to be used in a wide different use cases. But leaving the configuration of tools up to users allows for the possibility of misconfiguring tools, which often results in tools that are used sub-optimally or even incorrectly. For example, it is common for libraries to ship with polyfills or compiled code. This is considered an anti pattern because this makes it harder for applications to optimize apps which use the libraries (tree shaking) and it increases the bundle sizes of apps that use the libraries.

The Solution

Alfred aims to solve these problems by enabling tools to configure themselves out of the box. Each tool should know how to configure itself so that it can be compatible with other tools the user is using. Alfred achieves this 'out of the box' solution by generating minimal configurations for the user's tools. Advanced users can override or extend generated configurations. Alfred tests each combination of tools before publishing new versions.

Skills

A skill is an abstraction over a tool that allows it to configure itself with respect to other tools. For example, a babel ‘skill’ which wants to add react support would add the babel-preset-react preset to its config if the user is using the react skill.

Here is an example of a skill:

export default {
name: "eslint",
devDependencies: {
"eslint-config-airbnb": "18.0.0"
},
configs: [
{
// Config's filename
filename: ".eslintrc.js",
// The base eslint config
config: {
plugins: ["eslint-plugin-prettier"]
}
}
],
transforms: {
// Make eslint config compatible with react
react(eslintSkill) {
return eslintSkill
.extendConfig("eslint", {
plugins: ["eslint-plugin-react"]
})
.addDevDeps({
"eslint-plugin-react": "7.18.0"
});
}
}
};

For more on skills, see the skills section of the docs.

Alfred comes with skills out of the box but it also allows users to use 3rd party skills as well. Users can customize configs through Alfred's configs:

Here is an example of what an Alfred config looks like:

// package.json
{
// ...
"alfred": {
"skills": [
[
"@alfred/skill-eslint",
// A which extends the generated config
{
"extends": ["eslint-config-airbnb"],
"rules": {
"no-console": "off"
}
}
]
]
}
}

Entrypoints

Alfred formalizes the concept of entrypoints, which are files that determine the project type and platform a project will run on. For example, the entrypoint src/app.browser.js will be built as a browser app, app being the project type and 'browser' being the platform. Entrypoints determine which skills should be used to act on the entrypoint for a specific subcommand. Running the build subcommand on a project that has a ./src/lib.browser.js entrypoint should build the entrypoint with rollup, a bundler that is optimal for libraries.

Skills can declare which project types, platforms, and environments they support. Here's how the parcel skill defines which environments, platforms, and projects it supports:

const supports = {
envs: ["production", "development", "test"],
platforms: ["browser", "node"],
projects: ["lib"]
};
export default {
name: "rollup",
tasks: [
["@alfred/task-build", { supports }],
["@alfred/task-start", { supports }]
],
// ...
};

Tasks

Tasks determine which skill should be used when a certain subcommand is called. For example, when alfred run build is called, either the parcel, webpack, or rollup skill could be used. They also specify how skills are called and provide information about the task. Below is an example of a task:

// @alfred/task-build
export default {
subcommand: "build",
description: "Build, optimize, and bundle assets in your app",
runForEachTarget: true,
resolveSkill(skills, target) {
// return whichever skill you want to resolve...
}
};

Skills can then implement certain tasks:

export default {
name: "parcel",
tasks: ["@alfred/task-build", "@alfred/task-start"],
// ...
};

Tasks allow for skills to be interchanged while maintaining a consistent developer workflow. For example, all skills that lint a user’s project will use the @alfred/task-lint task so all of these skills are invoked through the lint subcommand that the task registers.

Alfred comes with the following tasks built-in: build, start, lint, format, and test.

Files and Directories

Sometimes, adding or changing configuration may not be enough to add support for a certain tool or library. Redux, for example, requires configureStore.js, root.js, and other files. To allow skills to fully add out of the box support for tools they wrap, Alfred allows them to define files and directories which are added to the user's project. Similar to configs, files can also be modified by skill transforms. Below is an example of how the redux skill transforms the configureStore.prod.js file to be compatible with typescript:

export default {
name: "redux",
files: [
{
alias: "configureStore.prod",
src: path.join(__dirname, "../boilerplate/store/configureStore.prod.js"),
dest: "src/store/configureStore.prod.js"
}
// ...
],
transforms: {
typescript(skill) {
skill.files
.get("configureStore.prod")
.rename("configureStore.prod.ts")
.applyDiff(
`@@ -12 +12 @@
-function configureStore(initialState) {
- return createStore(rootReducer, initialState, enhancer);
+function configureStore(initialState?: State): Store {
+ return createStore(rootReducer, initialState, enhancer);`
);
return skill;
}
}
};

Files can be transformed by either applying diffs to files or by replacing strings that match a regular expression.

Getting Started with Alfred

To get started with alfred,

# Create a new project
npx alfred new my-project
cd my-project
# Build your project
npx alfred run build

Here is an example of what an Alfred config looks like:

// package.json
{
// ...
"alfred": {
// Extend a shared Alfred config
"extends": "alfred-config-web-app",
// 3rd party skills the project uses
"skills": ["@alfred/skill-react"],
// The package manager to be used
"npmClient": "yarn"
}
}

For more details on how to use Alfred, see the docs.

Directory Structure

The following is an example of the directory structure of an Alfred browser app project:

my-project/
├── .gitignore
├── README.md
├── src/
│ └── app.browser.js
├── targets/
│ └── app.browser.dev/
│ └── index.js
│ └── app.browser.prod/
│ └── index.js
└── package.json

Exciting Opportunities

Alfred creates some exciting new oppertunities for workflows and tooling integration. Expect to see the oppertunities below in future releases!

Entrypoint specific commands

Sometimes, it is useful to run subcommands for a specific entrypoint. Alfred will allow user's to do so through the CLI:

# Building a specific entrypoint
alfred entrypoint lib.browser run build
# Building all app entrypoints
alfred entrypoint app.* run build

This will make maintaining apps with multiple entrypoints much easier.

Publishing/Deploying with Alfred

Alfred integration with publishing and deploying can significantly simplify the deployment process for many web developers. Ideally, developers can deploy to their platform of choice just by learning a skill:

# Publishing a specific entrypoint
# By default, libs and node apps are published to npm registry
alfred entrypoint lib.browser run publish
# Publish all entrypoints
alfred run publish
# Publishing all app entrypoints
alfred entrypoint app.* run publish
# Publish app to GitHub Pages
alfred learn @alfred/skill-github-pages
alfred entrypoint app.browser run publish
# Publish app to Now
alfred learn @alfred/skill-now
alfred entrypoints app.browser app.node run publish

Documentation Tooling

It can be said that one of the most undervalued pieces of JS tooling is its documentation tooling. There is much to be learned from the success of docs.rs, the standard for documentation generation for Rust. A JS equivalent of docs.rs might be of much use to the JS ecosystem.

Plugins for Alfred

It is very common for JS tools to allow plugins to add extra functionality to tools. Take, for example, a simple rollup plugin that prints the sizes of chunks after bundling:

9c0KN6W

A useful plugin indeed! But if we want to use this plugin across all our bundlers of all our entrypoints (parcel or webpack if you have an app entrypoint) we would need to create a new plugin for both webpack and parcel.

Alfred can reduce the duplication of plugins with skills and tasks. Skills can return metadata from the tools they wrap. Tasks can provide an interface which skills should conform their metadata responses to.

Here is an example of what a task may look like:

// @alfred/task-build
export default {
subcommand: "build",
description: "Build, optimize, and bundle assets in your app",
runForEachTarget: true,
resolveSkill(skills, target) {
// return whichever skill you want to resolve...
},
metadataInterface: {
ast: type.object,
output: type.array.of(type.string)
}
};

Below is an example of a skill that takes rollup's AST and return an object that conforms to the interface defined by the task.

// @alfred/skill-rollup
export default {
name: "rollup",
tasks: ["@alfred/task-build"],
// ...
metadata(rollupAst) {
// ...
return {
ast: {...},
outputs: [...]
};
}
};

A plugin can now receive this metadata from any skill that uses the build task:

// @alfred/plugin-size
export default {
name: "size",
hooks: {
afterBuild({ metadata }) {
const outputSizes =
metadata.outputs.map(output =>
`${output.name} size: ${output.size}`
);
console.log(outputSizes);
}
}
}

Shared ASTs

Sharing AST's between tools is an interesting area for investigation that can provide significantly improve the quality of JS tooling. This can improve the performance and developer experience of all JS tooling. Rome is already doing this!

Acknowledgements

Prior Art

Inspiration