Attention SPAMMERS: Don't advertise your services on my blog. If you do then you will have implicitly given me permission to disable any service you advertise via my blog by any means necessary.
Here are some related quotes from some notable people:
“To be is to do.” — Socrates
“To do is to be.” — Jean-Paul Sartre
“The way to be is to do.” — Dale Carnegie
“The way to do is to be.” — Leo-tzu, Chinese philosopher
“We act as though comfort and luxury were the chief requirements of life. All that we need to make us happy is something to be enthusiastic about.” — Albert Einstein
“Success consists of going from failure to failure without loss of enthusiasm.” — Winston Churchill
“Nothing great was ever achieved without enthusiasm.” — Ralph Waldo Emerson
I think they are all correct.
Reflection
You need to be in at least two of the circles above to stay employed.
You need to be in all three of the circles to thrive at work.
At the heart of success in your endeavors is your enthusiasm.
So, find what you truly enjoy doing, deliver good work and be nice and you will be successful at it.
If your are running Node.js v0.11.0 to v0.12.5 then you need to upgrade to v0.12.6 ASAP.
That's typically what happens with buffer exploits.
The Exploit
A bug in the way the V8 engine decodes UTF strings has been discovered. This impacts Node at the Buffer to UTF8 String conversion and can cause a process to crash. The security concern comes from the fact that a lot of data from outside of an application is delivered to Node via this mechanism which means that users can potentially deliver specially crafted input data that can cause an application to crash when it goes through this path. We know that most networking and filesystem operations are impacted as would be many user-land uses of Buffer to UTF8 String conversion.
Buffers
Here's some background information on how buffers work in NodeJS.
Buffers are instances of the Buffer class in node, which is designed to handle raw binary data. Each buffer corresponds to some raw memory allocated outside V8. Buffers act somewhat like arrays of integers, but aren’t resizable and have a whole bunch of methods specifically for binary data. In addition, the “integers” in a buffer each represent a byte and so are limited to values from 0 to 255 (2^8 – 1), inclusive.
There are a few ways to create new buffers:
var buffer = new Buffer(8);
This buffer is uninitialized and contains 8 bytes.
var buffer = new Buffer([ 8, 6, 7, 5, 3, 0, 9]);
This initializes the buffer to the contents of this array. Keep in mind that the contents of the array are integers representing bytes.
var buffer = new Buffer("I'm a string!", "utf-8")
Writing to Buffers
Given that there is already a buffer created:
var buffer = new Buffer(16);
We can start writing strings to it:
buffer.write("Hello", "utf-8")
The first argument to buffer.write is the string to write to the buffer, and the second argument is the string encoding. It happens to default to utf-8 so this argument is extraneous.
buffer.write returned 5. This means that we wrote to five bytes of the buffer. The fact that the string “Hello” is also 5 characters long is coincidental, since each character just happened to be 8 bits apiece. This is useful if you want to complete the message:
buffer.write(" world!", 5, "utf-8")
When buffer.write has 3 arguments, the second argument indicates an offset, or the index of the buffer to start writing at.
Reading from Buffers
Probably the most common way to read buffers is to use the toString method, since many buffers contain text:
buffer.toString('utf-8')
'Hello world!u0000�kt'
Again, the first argument is the encoding. In this case, it can be seen that not the entire buffer was used! Luckily, because we know how many bytes we’ve written to the buffer, we can simply add more arguments to “stringify” the slice that’s actually interesting:
buffer.toString("utf-8", 0, 12)
'Hello world!'
Using Buffers in the Browser
The Buffer exploit mainly affects backend server running NodeJS (or old versions of IO.JS), but the use of Buffers is not limited to the backend.
You can work also with buffers in the Browser by using: https://github.com/toots/buffer-browserify.
However, its performance is poor, mainly due to Buffer design decisions.
Equivalent functionality, with better performance metrics, in the browser is provided by TypedArrays or https://github.com/chrisdickinson/bops.
bops
bops presents a JavaScript API for working with binary data that will work exactly the same in supported browsers and in node. due to the way that Buffer is implemented in node it is impossible to take code written against the Buffer API and make it work on top of binary data structures (Array Buffers and Typed Arrays) in the browser.
Instead, you have to fake the API on top of Object, but Object isn't designed for holding raw binary data and will be really slow/memory inefficient for many common binary use cases (parsing files, writing files, etc).
Upgrade NodeJS
If your target operating system is OSX, then you probably have 3 main packages to consider:
NodeJS
NPM
... and probably these as well:
Homebrew
NVM
If you're a Homebrew user and you installed node via Homebrew, there are issues with the way Homebrew and NPM work together stemming from the fact that both homebrew and npm are package management solutions.
If you're a Homebrew user and you installed node via Homebrew, there is a major philosophical issue with the way Homebrew and NPM work together.
Microsoft recommends customers plan to migrate to one of the above supported operating systems and browser combinations by January 12, 2016.
IE8 Issues
HTML5 CSS3 Incompatibilities
Remember having to insert the following into your head tag to help fix the lack of support IE8 has for html5 tags and CSS3 properties? (or using Modernizer)
Remember having to use es5-shim because IE8 did not implement lastIndexOf, map, filter, every, forEach, etc. functions?
Security Vulnerabilities
Did you know the IE8 has over 500 known security vulnerabilities? (that will never get fixed)
IE8's continued reliance on ActiveX makes it vulnerable to the core.
Unforgiving Parser
Back in the days of IE5, IE was very forgiving when it came to HTML syntax.
IE8 is unforgiving in regards to HTML syntax and javascript.
I'm not saying that I approve of a lax enforcement of standards, but I do recall how quickly a web developer could crank out a web application when the user base were all IE users. Not so for IE8.
In many cases, IE8 would make your site break, even if it were coded perfectly.
Good bye, IE8. (and good riddance!)
p.s. Unbelievably, IE8 was actually somewhat better than IE7.
What is your mitigation strategy when things go wrong?
Today, github had a major outage.
Granted, github was only down for under 30 minutes or less, but that can still wreak havoc for scripts that depend on github and don't have 30 minute+ retries built in.
Between approx. 7:40 a.m. and 7:54 a.m. EST, if you were to try to reach any resource with github.com in the url this is what you'd see:
My local NodeJS build started breaking. It caused me heartache (and is no doubt affecting others). I learned that it was because some employees at Joyent--the company that took ownership of NodeJS--ticked off core NodeJS software engineers; They left to form a new truly open source software package called IOJS (that's a fork of NodeJS and now is far better than NodeJS).
Where does all good software go to die?
Corporations that put politics, political correctness, and profits ahead of creating great software.
Here are a few examples:
Oracle - MySQL
Oracle - OpenOffice
Joyent - NodeJS
There are no doubt many more, but these are the ones that percolate to the top of my mind.
Decline in Interest in MySQL
Interest peaked before Oracle acquired MySQL:
MySQL was left to the roadside, usually, since it was considered a useless appendage that prevents people from using the Oracle DB software. There were several community blunders (not making source public, not accepting patches, long-standing bugs with existing patches, etc) that forced MySQL guys to move to MariaDB. There was a big renaissance after the move, with many new features added and many bugs fixed. Sort of like the party when the house drops on the witch in the Wizard of Oz.
~ reddit
LibreOffice Forked from OpenOffice
Interest peaked before Oracle acquired OpenOffice:
OpenOffice. Oracle botched this so hard. No patches accepted, no timelines, no community communication. Oracle only paid attention to Fortune 500 contributors. Eventually, OpenOffice heads formed a foundation to start correcting some of these compounded issues. Oracle responded by kicking the members out of the project, telling them they couldn't use the OpenOffice trademark, etc. So all the experts left and formed LibreOffice. Another renaissance was had, and many long-standing issues were fixed. Code was maintained. The LibreOffice guys now regularly publish updates, statistics, reports, etc. It's a great example of how a professional FOSS project should be.
~ reddit
Number of NodeJS Releases
The following chart shows the number of stable NodeJS releases, per year:
The rest of this article will focus on NodeJS.
Implications
The number of stable releases of a software package is a good indication of its health.
It's clear that NodeJS should be in the ER. STAT.
As with other open source projects, a decline in the number of stable releases immediately precedes a major decline in public interest in it.
Common Thread
Mostly poor management decisions caused the best software development talent to leave the project, which directly related to the decline in that software's quality, interest and significance in the industry.
Joyent Calls Prolific NodeJS Contributor an "Asshole"
First, it would help to understand how software development works using the Git Workflow that NodeJS was using.
I explained the pertinent part of it in this snippet from this post.
Developer creates feature branch, commits file changes and then submits a Pull Request
Other developers are notified of Pull Request, perform code review and the last one merges the feature branch to master
Here's what happened:
A code reviewer noticed that Ben Noordhuis wrote the pronoun, "him", a few times instead of "him/her" or "them", in an inline comment that described part of the NodeJS logic and submitted a Pull Request (PR) to change the pronouns. See patch here.
Ben rejected the PR, providing this comment: "Sorry, not interested in trivial changes like that."
A shit storm of bullying comments ensued from, "...always assumed to be male first on the internet. I'm +1 on this documentation change." to the more direct, "Stop pissing around and merge the damn PR."
Some other contributor undeleted the trivial pronoun-changing PR and force pushed it.
A stream of praise was given to that committer that pushed the PR to replace "him" with "them", e.g., "I believe these kinds of things do make a difference. Same for speakers, presenters, organisers etc. at events making an effort to e.g. switch between gendered pronouns (because yes, for many this is indeed still an effort, and probably even more so for non-native speakers of English, who are often not so aware of the finer points of the language or "accepted" alternative ways to express things). I'm always happy when someone does this!"
Ben left the NodeJS community to help form IO.JS (an improved version of NodeJS) and is back to being highly productive.
Joyent calls Ben an "asshole".
Cyberbullying
Cyberbullying is a global term that means the harassment of someone by use of electronic media, usually but not always social media.
The Thread That Took Down NodeJS
Whoever said NodeJS was fault tollerant was wrong.
Can you find any technical merit in any of the above (un-edited) comments?
Does that sort of dialog belong in a source code repository?
How does any of that help Joyent sell more NodeJS services?
There is a clear lack of vision from the technical management team at Joyent.
Joyent's behavior (lack of leadership/poor management practices) has replaced NodeJS core contributors with individuals that are obviiously more interested in the proper use of pronouns in comments than improving what matters, the NodeJS software.
Is NodeJS doomed to the same fate as other similar, significant open source software products?
Joyent and the Future of NodeJS
Currently, it's not clear what will become of NodeJS.
Technically minded, merit-based software engineers are going to have a hard time getting behind a company that pushes its social agenda ahead of software development.
The vast majority of NodeJS' core developers left the NodeJS community to form IO.JS
Joyent is making the appearance of mending fences, but time will tell ...
But the task force may find it difficult to reconcile with reasoning like the following from the IOJS community:
i'd rather not reconcile. the benefits are not substantial, and i'm very happy with how iojs has been run. i don't want iojs to change organizationally in the name of reconciliation. for me, iojs' organization is an ultimatum.
i don't really care about naming and recognition. i'd rather just start pushing #!/usr/bin/env iojs and iojs-only (specifically, ES6+) support everywhere.
MIT License
There is a big difference. Node is MIT. And other companies with power and interest in node could simply fork if Joyent were to act foolishly. ~ Tim Caswell
One of the major components in most of my current front-end application architectures is JSDOM.
JSDOM is the first component in my stack to formally declare a divergence from NodeJS.
Here's what it says at the top of their README:
Note that as of our 4.0.0 release, jsdom no longer works with Node.js™, and instead requires io.js. You are still welcome to install a release in the 3.x series if you use Node.js™.
Available ES6 features in IO.JS
The following list of features are available without using any flags:
Block scoping (let, const)
Collections (Map, WeakMap, Set, WeakSet)
Generators
Binary and Octal literals
Promises
New String methods
Symbols
Template strings
Those ES6 features are very important. I'm sure I'll blog more about them in the future.
When the other NodeJS dependencies (besides JSDOM) support IO.JS I'll jump ship.
Personal Value Driven Decisions
I chose to move from MySQL to PostgreSQL
I chose to drop OpenOffice in favor of LibreOffice
I will very likely drop NodeJS in favor of IO.JS
The NodeJS Debacle - Lessons Learned for CTOs
Here are some suggestions, if implemented, could help with some issues of software development, socially aware employees and profitability:
Retain your real talent
Streamline software development governance
Do not allow the political correctness dept, finance dept., etc. to make decisions that impact software quality
Keep politics and social agendas out of your Git Workflow
Create an internal social media site for your writers and comment editors to discuss their non-technical beliefs
Hire a management team that can reconcile all time spent (X) to the question, "Does X help us sell more product?"
When you perceive that one of your software vendors is offering more and more discounts (frequently in the form of a recurring revenue scheme), you should look into the technical viability of the product being pushed.
If there is another open source alternative that has sprung out of the discontent of the lead developers of the software you're considering (or have been sold), beware.
Personal Opinion
I think that Joyent will continue to take the lead in gender-neutral-pronoun political correctness.
NodeJS may have the most politically correct, properly conjugated documentation, but that's not important to me.
What is important is being able to rely on the current and future stability of my software platform.
I bet that the interest curve and hence the viability of NodeJS is going to take a much steeper dive into insignificance than either MySQL or OpenOffice.
I personally applaud Ben for his professionalism and technical and social contributions.
I see the IOJS / NodeJS situation as a David / Goliath story.
It's only a matter of time before Joyent and its flagship product, NodeJS, fall to the feet of IOJS.
Reality and Drama
Money talks.
So, here's what I think is going to happen:
Joyent financial backers will soon understand why their cash cow is dying
Today, while I was installing a debugger for Node the install failed.
That made me take a long look at how I installed node, npm and nvm.
Here's what I decided:
Node comes with it's own package manager. It's called npm.
Brew is an awesome package manager for software you install on a mac, but should not be used to install node.
I had a bunch of cruft left over from previous installs of node, npm and nvm; Some of which was getting in the way of cleanly installing node, npm and nvm.
So, I wrote a bash script that does the following:
Remove previous installation cruft
Use brew to install nvm
Use nvm to install node (with also installs npm)
Here it is:
install-nvm-npm-node
# filename: install-nvm-npm-node
# author: Lex Sheehan
# purpose: To cleanly install NVM, NODE and NPM
# dependencies: brew
NOW=$(date +%x\ %H:%M:%S)
CR=$'\n'
REV=$(tput rev)
OFF=$(tput sgr0)
BACKUP_DIR=$HOME/backups/nvm-npm-bower-caches/$NOW
MY_NAME=$(basename $0)
NODE_VER_TO_INSTALL=$1
if [ "$NODE_VER_TO_INSTALL" == "" ]; then
NODE_VER_TO_INSTALL=v0.12.2
fi
if [ "`echo "$NODE_VER_TO_INSTALL" | cut -c1-1`" != "v" ]; then
echo """$CR""Usage: $ $MY_NAME "
echo "Example: $ $MY_NAME v0.12.1"
echo "Example: $ $MY_NAME $CR"
exit 1
fi
echo """$CR""First, run: $ brew update"
echo "Likely, you'll need to do what it suggests."
echo "Likely, you'll need to run: $ brew update$CR"
echo "To install latest node version, run the following command to get the latest version: $ nvm ls-remote"
echo "... and pass the version number you want as the only param to $MY_NAME. $CR"
echo "Are you ready to install the latest version of nvm and npm and node version $NODE_VER_TO_INSTALL ?$CR"
echo "Press CTL+C to exit --or-- Enter to continue..."
read x
echo """$REV""Uninstalling nvm...$CR$OFF"
# Making backups, but in all likelyhood you'll just reinstall them (and won't need these backups)
if [ ! -d "$BACKUP_DIR" ]; then
echo "Creating directory to store $HOME/.nvm .npm and .bower cache backups: $BACKUP_DIR"
mkdir -p $BACKUP_DIR
fi
set -x
mv $HOME/.nvm $BACKUP_DIR 2>/dev/null
mv $HOME/.npm $BACKUP_DIR 2>/dev/null
mv $HOME/.bower $BACKUP_DIR 2>/dev/null
{ set +x; } &>/dev/null
echo "$REV""$CR""Uninstalling node...$CR$OFF"
echo "Enter your password to remove user some node-related /usr/local directories"
set -x
sudo rm -rf /usr/local/lib/node_modules
rm -rf /usr/local/lib/node
rm -rf /usr/local/include/node
rm -rf /usr/local/include/node_modules
rm /usr/local/bin/npm
rm /usr/local/lib/dtrace/node.d
rm -rf $HOME/.node
rm -rf $HOME/.node-gyp
rm /opt/local/bin/node
rm /opt/local/include/node
rm -rf /opt/local/lib/node_modules
rm -rf /usr/local/Cellar/nvm
brew uninstall node 2>/dev/null
{ set +x; } &>/dev/null
echo "$REV""$CR""Installing nvm...$CR$OFF"
echo "++brew install nvm"
brew install nvm
echo '$(brew --prefix nvm)/nvm.sh'
source $(brew --prefix nvm)/nvm.sh
echo "$REV""$CR""Insert the following line in your startup script (ex: $HOME/.bashrc):$CR$OFF"
echo "export NVM_DIR=\"\$(brew --prefix nvm)\"; [ -s \"\$NVM_DIR/nvm.sh\" ] && . \"\$NVM_DIR/nvm.sh\"$CR"
NVM_DIR="$(brew --prefix nvm)"
echo """$CR""Using nvm install node...$CR"
echo "++ nvm install $NODE_VER_TO_INSTALL"
nvm install $NODE_VER_TO_INSTALL
NODE_BINARY_PATH="`find /usr/local/Cellar/nvm -name node -type d|head -n 1`/$NODE_VER_TO_INSTALL/bin"
echo "$REV""$CR""Insert the following line in your startup script (ex: $HOME/.bashrc) and then restart your shell:$CR$OFF"
echo "export PATH=\$PATH:$NODE_BINARY_PATH:$HOME/.node/bin"
echo """$CR""Upgrading npm...$CR"
echo '++ install -g npm@latest'
npm install -g npm@latest
{ set +x; } &>/dev/null
echo "$REV""$CR""Insert following line in your $HOME/.npmrc file:$OFF"
echo """$CR""prefix=$HOME/.node$CR"
echo "Now, all is likley well if you can run the following without errors: npm install -g grunt-cli$CR"
echo "Other recommended global installs: bower, gulp, yo, node-inspector$CR"
