Thursday, October 23, 2014

Prevent incorrectly-formatted Go code from being committed to Git

Problem

Even though Go is beautifully straightforward and blatantly obvious as to what is going on, there is still room for coding style differences.

And what happens when you're working in a multi-developer project not every developer uses the same coding styles?

Examples

  • Use TAB character rather than spaces
  • Use 2 spaces per TAB (v. 3 or 4 spaces)
  • Open brace on new line (v. at end of line)
  • etc...
...You get to experience needless merging and git diffs, that look like every line changed.

Enough of that nonsense... Just use the following pre-commit script to force each developer to run the standard gofmt against each source file before they git committed to git.

misc/git/pre-commit script

Simply create a file named pre-commit in your project's .git/hooks/ directory with the following contents:

pre-commit


#!/bin/sh
# Copyright 2012 The Go Authors. All rights reserved.
# Use of this source code is governed by a BSD-style
# license that can be found in the LICENSE file.

# git gofmt pre-commit hook
#
# To use, store as .git/hooks/pre-commit inside your repository and make sure
# it has execute permissions.
#
# This script does not handle file names that contain spaces.

gofiles=$(git diff --cached --name-only --diff-filter=ACM | grep '.go$')
[ -z "$gofiles" ] && exit 0

unformatted=$(gofmt -l $gofiles)
[ -z "$unformatted" ] && exit 0

# Some files are not gofmt'd. Print message and fail.

echo >&2 "Go files must be formatted with gofmt. Please run:"
for fn in $unformatted; do
 echo >&2 "  gofmt -w $PWD/$fn"
done

exit 1

Make it executable


$ chmod +x .git/hooks/pre-commit

See the pre-commit hook work

Add and extra TAB.

$ git diff
diff --git a/errors.go b/errors.go
index fc62d4c..dc3addf 100644
--- a/errors.go
+++ b/errors.go
@@ -15,8 +15,8 @@ 
     defaultErrMsg := "Oops! We had an issue with your request."
-    for _, errMsg := range optionalDefaultErrMsg {
-        defaultErrMsg = errMsg
+    if len(optionalDefaultErrMsg) > 0 {
+            defaultErrMsg = errMsg
     }

$ git add-commit 'Add extra TAB to break formatting'
Go files must be formatted with gofmt. Please run:
  gofmt -w /<PROJECT_DIRECTORY_HERE>/errors.go

See the pre-commit hook work

Run gofmt -w /<PROJECT_DIRECTORY_HERE>/errors.go and try again.

This time, your commit should work and your properly formatted file will be saved to your local git repo.

Notes

Since the file pre-commit file is in your .git/hooks directory, by default it will not get checked into your repo.

So, it's safe to use. No worries about an extra commit because you created the new pre-commit file.

Others on your team could rely only on their IDE to provide gofmt functionality (and choose not use this pre-commit hook).

But you can rest assured that all of your code will always be properly formatted before any push into the team git repo.

Configure gofmt Globally

Since git 2.9+ (2016-06-13), we can place our pre-commit file in a global git/hooks directory to configure all of our git repos to execute it.


$ git config --global core.hooksPath /path/to/global/git/hooks

Example

If you put your pre-commit script in $HOME/dev/global/.git/hooks/pre-commit then you can run the following command to configure git to run gofmt on the Go files that you want to commit to git:

$ git config --global core.hooksPath ~/dev/global/.git/hooks

Log for Configure gofmt Globally

Notes:

  • main.go has some formatting issues on the func main() line
  • We can build our Go app locally (with the formatting issues)
  • We are unable to commit our poorly formatted Go code to our local repository due to our core.hooksPath git configuration
  • After running gofmt -w we are able to commit our file and push it to the remote repo
  • The pre-commit script looks for Go files in your index, i.e., your local git staging area, and will only run if you are attempting to commit one or more .go files
  • See https://github.com/l3x/gofmt-demo.git


$ cat main.go
package main

import "fmt"

 func main()  {
 fmt.Println("formatting errors above")
}

$ gs
On branch master
Your branch is up-to-date with 'origin/master'.

Untracked files:
  (use "git add ..." to include in what will be committed)

 main.go

nothing added to commit but untracked files present (use "git add" to track)

$ git add .

$ go build

$ ll
total 4128
drwxr-xr-x   6 lex  staff      192 Mar 21 11:36 ./
drwxr-xr-x   3 lex  staff       96 Mar 21 11:34 ../
drwxr-xr-x  13 lex  staff      416 Mar 21 11:36 .git/
-rw-r--r--   1 lex  staff        0 Mar 21 11:34 README.md
-rwxr-xr-x   1 lex  staff  2106512 Mar 21 11:36 gofmt-demo*
-rw-r--r--   1 lex  staff       86 Mar 21 11:36 main.go

$ ./gofmt-demo
formatting errors above

$ cat main.go
package main

import "fmt"

 func main()  {
 fmt.Println("formatting errors above")
}

$ git commit -m 'demo use of global git precommit hook'
Go files must be formatted with gofmt. Please run:
  gofmt -w /Users/lex/tmp/20180321-113307/gofmt-demo/main.go

$ gofmt -w /Users/lex/tmp/20180321-113307/gofmt-demo/main.go

$ gs
On branch master
Your branch is up-to-date with 'origin/master'.

Changes to be committed:
  (use "git reset HEAD ..." to unstage)

 new file:   main.go

Changes not staged for commit:
  (use "git add ..." to update what will be committed)
  (use "git checkout -- ..." to discard changes in working directory)

 modified:   main.go

Untracked files:
  (use "git add ..." to include in what will be committed)

 gofmt-demo

$ git add .

$ go build

$ ./gofmt-demo
formatting errors above

$ git commit -m 'demo use of global git precommit hook'
[master f024a20] demo use of global git precommit hook
 2 files changed, 7 insertions(+)
 create mode 100755 gofmt-demo
 create mode 100644 main.go

$ git push
Counting objects: 4, done.
Delta compression using up to 8 threads.
Compressing objects: 100% (4/4), done.
Writing objects: 100% (4/4), 737.15 KiB | 6.41 MiB/s, done.
Total 4 (delta 0), reused 0 (delta 0)
To github.com:l3x/gofmt-demo.git
   6cff8f6..f024a20  master -> master

$ cat main.go
package main

import "fmt"

func main() {
 fmt.Println("formatting errors above")
}

References

http://blog.golang.org/go-fmt-your-code
https://golang.org/doc/effective_go.html

This work is licensed under the Creative Commons Attribution 3.0 Unported License.

Thursday, September 25, 2014

Push to a Different Branch

Problem

I made updates to files, added them to the local git repo and committed them to the index (git's local staging repo).

Then I realized that I had failed to create a new branch.

So the files I just updated are in an old branch.

How do I move these files to a new branch on the remote repo in one command?

Solution

Use one of my handy git aliases, of course!

$ git push-current-branch-to-another <CURRENT_OLD_BRANCH_NAME> <NEW_BRANCH_NAME>


References

http://lexsheehan.blogspot.com/2014/06/handy-git-aliases.html
http://stackoverflow.com/questions/8550586/how-to-push-from-one-branch-to-another-and-checkout

This work is licensed under the Creative Commons Attribution 3.0 Unported License.

Thursday, September 18, 2014

Atrocious API Architecture (AAA)

Summary

Use the following planning, design, implementation and testing pointers to create a terrible API.

Or do the opposite of each recommendation to create an API that rocks.

PLANNING

  • Start building your app before you clearly define your system requirements
  • Do not try to understand your problem domain
  • Do not identify constraints
  • Focus on on individual steps, rather than overall system throughput
  • Do not try to improve efficiency of your biggest constraints (see previous rule)
  • Refer only the articles listed in the References section below

INTERFACE DESIGN

  • Do not use RESTful APIs
  • Use web services and assume your integration partners will not change their APIs
  • Make your API as big as possible
  • Use SOAP or better yet, roll your own request/response protocol
  • Do not apply the Fascade Pattern (expose your implementation details via your API)
  • Do not use default args, varargs, generics or enums
  • Do not version your API (change your API at will without notifying clients)
  • Never use JSON (Use a custom message protocol or at least SOAP)
  • Do not implement rate limiting (allow a single client to hog system resources)
  • Use nonsensical API operation names
  • Use inconsistent API operation names (ex: use both AddProduct and insertClient)

REQUEST DESIGN

  • Permit non SSL access to various end points
  • Require as much data as possible from your clients to satisfy simple requests
  • Never use interface types for input parameters (use classes)
  • Require generic input parameter types
  • Always use the String type for inputs and outputs
  • Expose access to all internal data fields and methods
  • Use inconsistent parameter ordering (and lots of them!)
  • Force clients to use exceptions for flow control
  • Force client to specify data type of fields in each request
  • Use session based security for long running transactions
  • Do not provide authentication options (do everything in the clear)
  • Do not notify clients when data has been modified (force them to perform bulk reads and compare)
  • Do not filter any of the fields from your database tables
  • Do not allow clients to indicate which fields they want returned in the response
  • Do not return updated_at attribute (make the client hit your API repeatedly for the same data)
  • Do not use HTTP status codes in your response

PROJECT MANAGEMENT

  • No Agile development (that's a given!)
  • Do not use project management software like Jira, Pivotal Tracker, Rational Team Concert
  • Hold hour-long meetings hourly
  • Micro manage your developers
  • Never allow time for research or prototyping
  • Assume everybody will get everything right the first time and plan accordingly

IMPLEMENTATION

  • Do not use feature branches
  • Merge all code into the single, master branch
  • Do not tag your commits with a ticket/reference number
  • Do not make your API pluggable
  • Provide the same level of service regardless of business requirements
  • Design such that small specific changes will require lots of system upgrades
  • Always copy and paste code (no code reuse!)
  • Avoid writing modular code
  • Create only mutable objects (avoid stateless design)
  • Avoid functional programming
  • Do not fail early (process as much of the request as possible before you return an error)
  • Use Java (always overload your methods and use subclassing as much as possible)
  • Fail silently when possible
  • Use only global variables
  • Use floating points for monetary values

TESTING

  • If you do test, do not try to test individual components (which would be nearly impossible, given your needlessly complicated API)
  • Do not write unit tests
  • Do not provide an API test framework for your clients to use
  • Ensure that your tests take so long to run that it is impossible for developers to test their new code before they merge it into the master branch
  • Do not document (and if you do add non-valuable comments like: // This is a loop)

References



This work is licensed under the Creative Commons Attribution 3.0 Unported License.

Thursday, September 11, 2014

Fix Go SDK Version for go-lang-idea-plugin

Problem

Go works fine from the command window, but within IntelliJ you get a message about incorrect SDK version.

Assumption

You installed Go using homebrew.

Go Environment

Here, we see that Go 1.3.1 is the current installed version:

$ go env
GOARCH="amd64"
GOBIN=""
GOCHAR="6"
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/lex/dev/go/samples"
GORACE=""
GOROOT="/usr/local/Cellar/go/1.3.1/libexec"
GOTOOLDIR="/usr/local/Cellar/go/1.3.1/libexec/pkg/tool/darwin_amd64"
CC="clang"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fno-common"
CXX="clang++"
CGO_ENABLED="1"


Fix IntelliJ Problem

The go-lang-idea-plugin reported that it expected Go version 1.3, but got 1.3.1. So, just remove 1.3 (assuming that you don't need it) and create a symlink fro 1.3 to 1.3.1:

$ cd /usr/local/Cellar/go
  /usr/local/Cellar/go $ tdml
.
├── 1.2.2 -> /usr/local/Cellar/go/1.3/
├── 1.3
│   ├── bin
│   ├── etc
│   ├── libexec
│   └── share
└── 1.3.1
    ├── bin
    ├── etc
    ├── libexec
    └── share

11 directories
  /usr/local/Cellar/go $ rm -rf 1.3
  /usr/local/Cellar/go $ ln -s /usr/local/Cellar/go/1.
1.2.2  1.3.1/
  /usr/local/Cellar/go $ ln -s /usr/local/Cellar/go/1.3.1 /usr/local/Cellar/go/1.3


Discussion

This feels like such a hack, but I could not find a way to configure the version of the Go SDK for the go-lang-idea-plugin.

If anybody finds a better solution, please let us know. Thanks!

The Rest of the Story

I eventually found where to conigure the Go SDK in IntelliJ.

File > Folder Structure > Project Settings > Modules > Dependencies

Click the Module SDK dropdown and select the latest Go SDK (In my case it is currently Go SDK go1.3.3 darwin/adm64)

If that is not an option, you may need to click the New... button and select the path that is $GOPATH.

Also, verify that you selected the correct Go SDK in File > Project Structure > Project Settings > Project

References

http://lexsheehan.blogspot.com/2014/08/install-godoc-command.html
https://github.com/go-lang-plugin-org/go-lang-idea-plugin/blob/master/Missing%20ENV.md
https://github.com/go-lang-plugin-org/go-lang-idea-plugin/issues/696
http://brew.sh/

This work is licensed under the Creative Commons Attribution 3.0 Unported License.

Thursday, September 4, 2014

Jasmine Now Supports Pending Status

Summary

Jasmine is a BDD-style testing framework for automated testing of your web UI using headless browsers.

By default, it is configured to test Chrome and Firefox browsers.

History

Until recently, Jasmine did not allow you to indicate that any of your tests are in a pending state.

As of version 2.0, that is now possible.

Install Jasmine 2.0

You may run into issues updating your Jasmine installation.

Below, are the commands that worked for me:

$ rm -rf node_modules/karma
~/dev/angularjs/test $ npm cache clean
~/dev/angularjs/test $ npm install karma-jasmine@2_0 --save-dev
npm WARN package.json todomvc-angular-tests@1.0.0 No repository field.
npm WARN package.json karma-chrome-launcher@0.1.4 No README data
npm WARN package.json karma-phantomjs-launcher@0.1.4 No README data

> fsevents@0.2.1 install ~/dev/angularjs/test/node_modules/karma/node_modules/chokidar/node_modules/fsevents
> node-gyp rebuild

  CXX(target) Release/obj.target/fse/fsevents.o
  SOLINK_MODULE(target) Release/fse.node
  SOLINK_MODULE(target) Release/fse.node: Finished

> ws@0.4.32 install ~/dev/angularjs/test/node_modules/karma/node_modules/socket.io/node_modules/socket.io-client/node_modules/ws
> (node-gyp rebuild 2> builderror.log) || (exit 0)

  CXX(target) Release/obj.target/bufferutil/src/bufferutil.o
  SOLINK_MODULE(target) Release/bufferutil.node
  SOLINK_MODULE(target) Release/bufferutil.node: Finished
  CXX(target) Release/obj.target/validation/src/validation.o
  SOLINK_MODULE(target) Release/validation.node
  SOLINK_MODULE(target) Release/validation.node: Finished
karma@0.12.23 node_modules/karma
├── di@0.0.1
├── graceful-fs@2.0.3
├── rimraf@2.2.8
├── colors@0.6.2
├── mime@1.2.11
├── q@0.9.7
├── minimatch@0.2.14 (sigmund@1.0.0, lru-cache@2.5.0)
├── optimist@0.6.1 (wordwrap@0.0.2, minimist@0.0.10)
├── glob@3.2.11 (inherits@2.0.1, minimatch@0.3.0)
├── source-map@0.1.38 (amdefine@0.1.0)
├── lodash@2.4.1
├── useragent@2.0.9 (lru-cache@2.2.4)
├── log4js@0.6.20 (semver@1.1.4, async@0.1.15, readable-stream@1.0.31)
├── http-proxy@0.10.4 (pkginfo@0.3.0, utile@0.2.1)
├── chokidar@0.8.4 (recursive-readdir@0.0.2, fsevents@0.2.1)
├── connect@2.12.0 (uid2@0.0.3, methods@0.1.0, cookie-signature@1.0.1, debug@0.8.1, pause@0.0.1, fresh@0.2.0, qs@0.6.6, bytes@0.2.1, buffer-crc32@0.2.1, raw-body@1.1.2, batch@0.5.0, cookie@0.1.0, negotiator@0.3.0, send@0.1.4, multiparty@2.2.0)
└── socket.io@0.9.17 (base64id@0.1.0, policyfile@0.0.4, redis@0.7.3, socket.io-client@0.9.16)

karma-jasmine@0.2.2 node_modules/karma-jasmine


Test Code

Here's how to indicate that you have a Pending test:

Continuous TDD - Console Output


Notes

Instead of saying PENDING: 1 in an intuitive yellow color the console output says, "Skipped 1".

Mocha/Chai is another alternative that also works well and has a more flexible plugin architecture.

References

https://jasmine.github.io/2.0/introduction.html#section-Pending_Specs
http://lostechies.com/derickbailey/2012/08/18/jasmine-async-making-asynchronous-testing-with-jasmine-suck-less/
http://www.yearofmoo.com/2013/01/full-spectrum-testing-with-angularjs-and-karma.html
https://github.com/karma-runner/karma-jasmine
https://www.npmjs.org/package/npm-check-updates

This work is licensed under the Creative Commons Attribution 3.0 Unported License.

Thursday, August 28, 2014

Rietveld and Other Code Review Tools

Rietveld

Rietveld is a web application that manages the process of code reviews for software projects with large developer communities.

Rietveld is used today by developers of the standard distribution of the following:
  • Go (Golang)
  • Python
  • Chromium browser

Follow Golang Development

If you want to follow Go development on Rietveld, go to: golang-dev



In this example, I clicked the first issue, Issue 13454043: os/user: group lookup functions

Description


os/user: group lookup functions

Adding group lookup functions analogous to the user lookup ones to the
os/user package.
Updates issue 4589049.


Comments for src/pkg/os/user/lookup.go

Below, we see two comments:

bradfitz 2014/01/16 18:43:36
In reports whether ...



bradfitz 2014/01/16 18:43:36
let's leave this one out for now.  We can add it later if needed.  Also, the
return type is weird ([]string) ... why not []*User?

Otherwise this should clarify that it returns userids, not users.


Code and Comments for "group lookup functions" Issue


Summary

Rietveld is an effective code review tool for a large development community and it's great for groking Golang code and learning from code review comments.

However, when it comes to a typical development team, Github is good enough Typicial Github Workflow:
  1. Developer pushes changes to Github
  2. Developer creates pull request (PR) to team (sometimes needs to nudge reviewers)
  3. Reviewer accepts changes in PR and merges code from PR

Github Code Review Alternatives

Other code review alternatives include:

References

https://codereview.appspot.com/user/golang-dev
https://www.codereviewhub.com
https://code.google.com/p/gerrit

This work is licensed under the Creative Commons Attribution 3.0 Unported License.

Wednesday, August 27, 2014

Install godoc command

Installation


$ go get code.google.com/p/go.tools/cmd/godoc


Verify Installation


$ godoc fmt Println
func Println(a ...interface{}) (n int, err error)
    Println formats using the default formats for its operands and writes to
    standard output. Spaces are always added between operands and a newline
    is appended. It returns the number of bytes written and any write error
    encountered.


Alternative

A previous post, Install godoc showed you how to install the godoc package with the go instal command. This example shows you how to install the godoc command.

View Source

Add the -src flag and see the source code:

$ godoc -src fmt Println
// Println formats using the default formats for its operands and writes to standard output.
// Spaces are always added between operands and a newline is appended.
// It returns the number of bytes written and any write error encountered.
func Println(a ...interface{}) (n int, err error) {
    return Fprintln(os.Stdout, a...)
}



References

http://lexsheehan.blogspot.com/2014/07/install-godoc.htm

This work is licensed under the Creative Commons Attribution 3.0 Unported License.