Archive for July, 2013

I always find it fairly amazing how many questions appear on Programmers.SE, and Stack Overlow, with the subject of “Which x is better?”. Equally, I’ve been involved in a conversation with co-workers and friends on many occasions where the subject line starts with “What do you think of x?”. The commonality of these two separate threads is that the x is always a language, framework, or tool.

I want to go out on a limb here, and state my conclusion at the start of my essay:

Learning a new tool does not make you a better developer.

Learning a new technique makes you a better developer, tools are just there for convenience.

I’ll start with an example from my own experience. I started off as an ASP.NET WebForms developer straight out of University. I quickly became very knowledgeable about the framework itself, learning how to force it to work for any edge-case I needed, and learning how to build extensions and re-use previous components effectively. I considered myself a talented developer, and was considered to be very highly skilled by my employers. Upon reflection, it is now clear to me that I was a terrible developer.

After a while, the ASP.NET MVC framework was publicly released, to much fanfare. I scoffed, safe in the knowledge that I could make any website faster, and with more functional stability, than any other developer that I knew could develop things in the MVC framework. I began to learn MVC, and being unused to the pattern, struggled to really understand the principles behind how to develop on it. I was a brilliant ASP.NET WebForms developer; I was a terrible developer.

A little while after that, my WebForms project came to an end, and I surveyed the carnage. Bad use of JavaScript was the really key example, because I didn’t understand that putting your JS inside the page and running it as-and-when a control was used was costly and slow. Code reuse in the backend was there, but not ideal, as many actions were done specifically for the page that called them, as opposed to sharing a common data access methodology. The UI code was bloated and ugly. I took this as a good opportunity to start using the MVC framework; clearly it could result in cleaner code, I had seen other coders achieve this for myself. I became an excellent MVC developer. I was still a terrible developer.

Eventually, I came to the realisation that the MVC framework was not ideal. It had some clunky and unwieldy features (such as Microsoft AJAX), and, while it could lead to very clean code, it often did not.^[1] It was around this point that I began to learn NUnit, and the principles of TDD; I quickly became a Unit Testing aficionado, and the code I produced jumped up in quality.

At around the same time as learning TDD, I began to investigate the Agile methodologies, and how they could help my development skills. This was the real turning point; I started using my own little Kanban board to manage a project backlog, and watched the quality jump again due to the increased visibility of what I would be building and the increased ability to group tasks with other important tasks. I’d used a technique that I’d never considered to be part of development, as it was neither a tool nor a language, to improve what I was developing. I had learned how something worked, not just how to use something, and it had made my code better.^[2]

Since that turning point, I’ve become much more introspective, examining my skills and trying to see if I really understand how something works, not just how to use it. I’ve written my own MVC framework in node.js to prove to myself that I understood not just how to use the MVC pattern, or how to code JavaScript, but that I really understood why the pattern pushes you to better code. In the process, I became intimately familiar with the HTTP spec, and now really understand how the web works, not just how to build things that run on it.

This brings me back to my initial conclusion. It’s clear to me, in retrospect, that I have progressed as a developer not by learning how to use ASP.NET MVC, or how to use jQuery, or how to use node.js, but by delving a little deeper; learning how to apply the MVC pattern in a different language with different principles; learning how the underlying framework of WebForms really interacts with the web browser, not just how to write a custom datepicker control.

It’s very easy to get entranced by a new tool. Perhaps you’re enamoured by the features of a framework, being certain it’ll let you build your apps faster, or more robustly. Or perhaps you’re certain that Silverlight will let you build perfectly acceptable rich client applications without having to learn a new language^[3]. I’ve done it myself many times. The trick is to realise that, yes, new frameworks are great, and you should broaden your horizons as much as possible. Just don’t forget that frameworks are transient, and tools become outdated.

We can really see this attitude at work in the recruitment area of the IT industry. Every job advert I’ve seen in the past year has “$language$ developer required”, or “$framework$ developer required” as its title, and contains a list of frameworks and tools that you are expected to have at least 3 years of experience in using^[4]. What recruiters and HR managers cannot seem to understand is that each and every framework, tool, and language, has a number of parallels.

Let’s take C# for an example. Yes, C# is very Microsoft-centric, and the .NET framework has a specific set of tools you’re likely to use, NUnit for example. However, at its heart, C# is very similar to Java. Yes, you’re going to declare an anonymous function in a completely different way, and you’re going to use JUnit, as opposed to NUnit, but any C# developer worth his salt should become fully operational in Java in a very short period of time.

On the other hand, your average C# developer may well struggle with Ruby, for example. Dynamic languages and static languages are so different in usage that they require totally different techniques and patterns to use^[5]. Dependency Injection? Don’t really need that in a dynamic language. Interfaces? Nah, not like you’re used to.

How do we fix this problem? Well, there are a number of root issues here. The first is developers truly believing that being intimately familiar with, say, rails, makes them a good developer. We fix that by encouraging newer devs to be multi-disciplinary. I firmly believe that the best way to enact change in such a broad area as development is from the bottom; if every new developer comes into the game believing that they need to be familiar with more than just the language, and framework, that’s directly in front of their face all day at work then many of them will go out of their way to learn a broader range of techniques, instead of just learning their framework of choice more intimately, or new a framework every now and again.

The second issue is the issue of organisational change. Most development houses do just one framework, and are very keen to continue to do so. On one hand this makes perfect sense, why change what already works? I recently had a conversation with a developer at another organisation about node.js: he claimed that node.js was perfect for every use case, I assured him that it was not^[6]. This attitude of one-size-fits-all is problematic, as developers in small companies aren’t going to be exposed to the broader range of approaches you’ll see in large companies where they recognise that sometimes you want your backend system to be written in a different manner to your API. This is, again, something that needs to come from the bottom. If you’re pushing your own boundaries, discovering new techniques, you’ll begin to learn which are the most appropriate for which scenario. Don’t always punt for the new and shiny, try and keep a level head and really think “do I want to use this tool because it’s the best thing for the job, or because I want to use it”.

The final issue is hiring. This one can only change from the top, so I implore those of you who are in a position to influence the hiring process in your organisation to look at a developer’s skills, not just their résumé^[7]. It still surprises me that most development houses either don’t have a development test, or have one that only deals with their core technologies, not the skills that lie behind their use. It would be amazing to see a community resource containing not just a list of “good questions” to ask, but of good skills to look for, and how to look for them effectively.

So, I’ve spoken about how I’ve developed as a programmer not by learning new languages, but by pushing my boundaries in other ways with new management techniques and improving my understanding of the underlying principles of the languages I do know. I hope that a few people who read this can look at their own skills and think “you know, I understand how to use this, but do I understand why?”. Real change is slow and difficult to enact, but in such a young, and social, industry we have an incredible opportunity to make changes as a community.

[1] I have seen developers write five thousand line actions inside a controller, through both laziness and lack of knowledge. Those of you with MVC knowledge should shudder at the thought.

[2] Don’t get me wrong, I’ve not yet hit the pinnacle of my development abilities, I doubt I’m anywhere near (or that I will ever really get there). Everyone has more skills to learn, myself included, and I dedicate a large amount of time to investigation of new things, and practise of the old.

[3] I once had a fairly involved argument with a colleague about whether Silverlight was worth using. I, unfortunately, lost, and the now-legacy Silverlight app has caused a myriad of deployment issues and at least one developer to leave the organisation.

[4] I particularly love ones that are “$cms$ developer required”. Those go straight into my trash.

[5] Yes, C# has the dynamic keyword. No, that doesn’t make it dynamic; dynamic in C# is just a slightly more ducky version of var.

[6] Batch processes was where I finally won that one. Anyone writing enterprise-sized batch processes in node.js has a serious case of myopia, in my opinion.

[7] I recently went to an interview where the only technical question I was asked was “What is the difference between “visibility: hidden” and “display: none” in CSS. This is not an accurate gauge of anything.

So you’ve been using Stack Exchange for a while now and, like most other programmers, expect to be able to customize the website. I’ve seen many feature requests on the main meta, and a large fraction of them never get implemented because:

• It’s not a feature that everyone will like — Stack Exchange tries not to provide too many customization options, so features that will be annoying to some won’t be implemented
• It’s a rather minor feature that may be only useful to some people (or will be used rarely), and not worth the developer time
• The developers are all busy working on something else

It could be something as simple as adding a small reply button to comments. It could also be something more elaborate; like a script that lets one manage and use a bank of boilerplate comments or a script that makes editing long posts easy.

Whatever it is, it’s not always easy to get a feature implemented. In many such cases, userscripting proves to be a viable alternate solution. For those who haven’t heard of userscripts, a userscript is a block of JavaScript code loaded into the browser by the user. It contains some metadata, including instructions for the browser on what type of pages it is meant to be loaded on. They’re very useful if you want to tweak the behavior of a website.

Tips for creating a userscript for Stack Exchange

Firstly, here’s the userscript template that I use:

// ==UserScript==
// @name Script name here
// @description Short description goes here
// @namespace Manishearth (This is to avoid conflicts with userscripts that share a name)
// @author Your name here
// @license GNU GPL v3 (http://www.gnu.org/copyleft/gpl.html)
// @include http://stackoverflow.com/*
// @include http://serverfault.com/*
// @include http://superuser.com/*
// @include http://meta.stackoverflow.com/*
// @include http://meta.serverfault.com/*
// @include http://meta.superuser.com/*
// @include http://stackapps.com/*
// @include http://.stackexchange.com/
// @include http://askubuntu.com/*
// @include http://meta.askubuntu.com/*
// @include http://answers.onstartups.com/*
// @include http://meta.answers.onstartups.com/*
// @include http://mathoverflow.net/*
// @include http://meta.mathoverflow.net/*
// @include http://discuss.area51.stackexchange.com/*
// @exclude http://chat./


// ==/UserScript==
function with_jquery(f) {
    var script = document.createElement("script");
    script.type = "text/javascript";
    script.textContent = "(" + f.toString() + ")(jQuery)";
    document.body.appendChild(script);
};

with_jquery(function($) {

// Your custom code goes here

});

The @include directives tell the browser which sites to load the script on. You may wish to tweak these to your needs. You may omit the with_jquery bits if you’re not planning on using jQuery, and instead just write your code after the // ==/UserScript== line. If the file is saved as a .user.js file and loaded into your browser (instructions for each browser here), then it will run the code on each SE site you visit.

Make sure you’re familiar with the Chrome Developer tools (or Firebug) as well. Remember that building a userscript is different from using JS for making a dynamic webpage. In the former case, there are many more unknowns, so one can’t always accurately predict how the page will react to your code. With this in mind, I find that it is easier to develop userscripts bit-by-bit on the console, instead of writing a large portion of the script at once and then testing it. I start by testing smaller code snippets on the console, and slowly assembling them into larger code snippets. Then I assemble the larger snippets off-console. This way, you can catch bugs before they get buried in large chunks of code. For example, while writing this script, there were a few issues with the predefined behavior of the preview and edit history that would have been quite hard to catch if I had written the code in bulk.   Another thing to remember is that Stack Exchange has many dynamically-created elements. If, for example, you want to add a  keyboard shortcut to the markdown editor, you will need to make sure that the corresponding event handlers are delegated. This post (and this documentation page, under the “direct and delegated events” section) are good places to learn event delegation via jQuery.

The Stack Exchange API is currently limited to mostly read actions, but it’s another useful tool for writing userscripts and apps. You may have to register for an API key, depending on the nature of the script.

A final, very useful resource is the unminified javascript. The normal Stack Exchange javascript files that one sees in the debugger  look like this (there is also stub,js, full-anon.js, and wmd.js), but by modifying the cdn in the URL to cdn-dev, one gets this page, which is much easier to read. The URLs may change — if the above links change, go to http://dev.stackoverflow.com/ and look for the URL of /js/full-anon.js in the source code or the sources tab of the Chrome developer tools.

Publishing your script on StackApps

StackApps is a repository of apps and scripts for Stack Exchange. You can just post your script by asking a question tagged with the [script] tag. Give links to the script source code, and instructions for installation and usage. Note that installing a userscript on Chrome is currently a slightly roundabout process, so it is advisable to link to the [script] tag wiki for further instructions. Here’s an example of a complete StackApps script post.

Some great userscripts

There are a lot  of  awesome userscripts on StackApps, which can be directly installed. Here are some of those that I find quite useful:

• Keyboard Shortcuts for Stack Exchange
• “Reply” links in comments
• Auto-scrolling in the preview for editing long posts
• Make a post printer friendly
• Better handling of the tab key for code indentation
• Boilerplate comment manager
• Punching users through the internet

Well, there you have it. You should now be able to tweak the behavior of SE to your tastes with relative ease.