npm 1.0: link

npm 1.0 is in release candidate mode. Go get it!

In npm 0.x, there was a command called link. With it, you could “link-install” a package so that changes would be reflected in real-time. This is especially handy when you’re actually building something. You could make a few changes, run the command again, and voila, your new code would be run without having to re-install every time.

Of course, compiled modules still have to be rebuilt. That’s not ideal, but it’s a problem that will take more powerful magic to solve.

In npm 0.x, this was a pretty awful kludge. Back then, every package existed in some folder like:

prefix/lib/node/.npm/my-package/1.3.6/package

and the package’s version and name could be inferred from the path. Then, symbolic links were set up that looked like:

prefix/lib/node/my-package@1.3.6 -> ./.npm/my-package/1.3.6/package

It was easy enough to point that symlink to a different location. However, since the package.json file could change, that meant that the connection between the version and the folder was not reliable.

At first, this was just sort of something that we dealt with by saying, “Relink if you change the version.” However, as more and more edge cases arose, eventually the solution was to give link packages this fakey version of “9999.0.0-LINK-hash” so that npm knew it was an imposter. Sometimes the package was treated as if it had the 9999.0.0 version, and other times it was treated as if it had the version specified in the package.json.

A better way

For npm 1.0, we backed up and looked at what the actual use cases were. Most of the time when you link something you want one of the following:

  1. globally install this package I’m working on so that I can run the command it creates and test its stuff as I work on it.
  2. locally install my thing into some other thing that depends on it, so that the other thing can require() it.

And, in both cases, changes should be immediately apparent and not require any re-linking.

Also, there’s a third use case that I didn’t really appreciate until I started writing more programs that had more dependencies:

  1. Globally install something, and use it in development in a bunch of projects, and then update them all at once so that they all use the latest version.

Really, the second case above is a special-case of this third case.

The first step is to link your local project into the global install space. (See global vs local installation for more on this global/local business.)

I do this as I’m developing node projects (including npm itself).

cd ~/dev/js/node-tap  # go into the project dir
npm link              # create symlinks into {prefix}

Because of how I have my computer set up, with /usr/local as my install prefix, I end up with a symlink from /usr/local/lib/node_modules/tap pointing to ~/dev/js/node-tap, and the executable linked to /usr/local/bin/tap.

Of course, if you set your paths differently, then you’ll have different results. (That’s why I tend to talk in terms of prefix rather than /usr/local.)

When you want to link the globally-installed package into your local development folder, you run npm link pkg where pkg is the name of the package that you want to install.

For example, let’s say that I wanted to write some tap tests for my node-glob package. I’d first do the steps above to link tap into the global install space, and then I’d do this:

cd ~/dev/js/node-glob  # go to the project that uses the thing.
npm link tap           # link the global thing into my project.

Now when I make changes in ~/dev/js/node-tap, they’ll be immediately reflected in ~/dev/js/node-glob/node_modules/tap.

Let’s say I have 15 sites that all use express. I want the benefits of local development, but I also want to be able to update all my dev folders at once. You can globally install express, and then link it into your local development folder.

npm install express -g  # install express globally
cd ~/dev/js/my-blog     # development folder one
npm link express        # link the global express into ./node_modules
cd ~/dev/js/photo-site  # other project folder
npm link express        # link express into here, as well

                        # time passes
                        # TJ releases some new stuff.
                        # you want this new stuff.

npm update express -g   # update the global install.
                        # this also updates my project folders.

Caveat: Not For Real Servers

npm link is a development tool. It’s awesome for managing packages on your local development box. But deploying with npm link is basically asking for problems, since it makes it super easy to update things without realizing it.

Caveat 2: Sorry, Windows!

I highly doubt that a native Windows node will ever have comparable symbolic link support to what Unix systems provide. I know that there are junctions and such, and I’ve heard legends about symbolic links on Windows 7.

When there is a native windows port of Node, if that native windows port has `fs.symlink` and `fs.readlink` support that is exactly identical to the way that they work on Unix, then this should work fine.

But I wouldn’t hold my breath. Any bugs about this not working on a native Windows system (ie, not Cygwin) will most likely be closed with wontfix.

Aside: Credit where Credit’s Due

Back before the Great Package Management Wars of Node 0.1, before npm or kiwi or mode or seed.js could do much of anything, and certainly before any of them had more than 2 users, Mikeal Rogers invited me to the Couch.io offices for lunch to talk about this npm registry thingie I’d mentioned wanting to build. (That is, to convince me to use CouchDB for it.)

Since he was volunteering to build the first version of it, and since couch is pretty much the ideal candidate for this use-case, it was an easy sell.

While I was there, he said, “Look. You need to be able to link a project directory as if it was installed as a package, and then have it all Just Work. Can you do that?”

I was like, “Well, I don’t know… I mean, there’s these edge cases, and it doesn’t really fit with the existing folder structure very well…”

“Dude. Either you do it, or I’m going to have to do it, and then there’ll be another package manager in node, instead of writing a registry for npm, and it won’t be as good anyway. Don’t be python.”

The rest is history.

This entry was posted in npm. Bookmark the permalink.

4 Responses to npm 1.0: link

  1. Bert says:

    Have faith my friend. Windows has symlinks.

    • I will be happy to support npm link wherever fs.symlink and fs.readlink are present.

      • Paul Betts says:

        Actually, NTFS symlinks were explicitly designed to work just like *nix symlinks by design. The support needs to be added to fs and you have to run npm elevated, but it’s definitely possible. Check out mklink on a >= Vista box

  2. Pingback: Node Roundup: npm link, mongoq, EventEmitter | tips & tricks

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s