What Should Be Part Of Your Automated Build

Personal reasons have kept me away from this blog for a very long time. It has been longer than I expected, but it seems that finally things are settling down, and I would like to resume this fine and fun practice. And what better way to do it than to resume from where I left.

I started this blog writing about a series of strategies you must define in any software development project. I wrote about the importance of a source code management strategy, and the reasons why you should define a documentation strategy. Now, it is time to talk about your automated build process.

Let me start by saying, if you do not have an automated build process you are doomed to fail. The reasons are simple. Automation will help you to reproduce every necessary step to build your product as many times as you need removing human factors that most likely lead you to failure. People tend to make mistakes; scripts do not. When an automated build process fails is indicative of some human error not a script making a mistake (assuming your script works). Therefore, you want to have a complete automated build process.

An automated build process means different things to different people. In my mind, it only has one definition: the set of automated steps that grab everything needed for your application from you source repository, and operates in it to compile, test, and package your application. Yes, you read it all right; compile, test and package your application.

An ideal build process downloads all the required files from your source repository, compiles and links the source code to create your binaries, runs all your automated tests, and packages your application files into your deployment package. One single click, one single scheduled task launches your build process and spits a package that you can release to your customers at the end of it. So let us talk about what your build process needs to do.

Your Source Repository Is The Beginning Of It All

Every project file should be in your source repository. This includes your source code, documentation source, application data and content, and build scripts. The only file that is not part of your repository is the one and only script that triggers everything, and this script should only download the source and invoke your build process.

Many people see the build process as a separate entity from the product. This is a big mistake. Your build process is going to change in the same way your project changes. New dependencies are added, new files are added to the application, new application content is added to your project. The same way all those files reflects a particular state of the application, your build scripts reflect a particular state that allows you to build your project at a specific point in time. Keeping the project files and the build process separate will only add a maintenance burden to your process, so it is always better to keep them together.

Obviously, It Build Your Binaries

Needless to say, your build process is going to produce the binaries that your application needs. But the process of building the binaries needs to do more than just that.

Binaries will need to be versioned in a consistent way with each new build. Company information might need to be embedded into the binaries, and the binaries might need to be signed. Your documentation and help files might also need to be compiled.

Make sure you build every configuration of the files you need to ship or that are used internally and for every platform you support. This means you should build your release configurations, as well as your debug configurations.

This is what most people understand as the build process. To me, this is just the first step of a bullet-proof build process.

Run Test Automation

Once you have all the files you need for your application, it is wise to test them. If you have automated tests, and you should, run them against your newly created binaries. Make this test run part of your build process to insure your application is tested to some extent with every build.

This does not mean you should not rely on the manual testing your QA department does. As a matter of fact, you are insuring your application runs and can be tested by QA. You will eliminate problems where your application seems to compile but crashes on start-up. Including your automated acceptance tests into your build process will alleviate the headaches QA gets when the build is broken.

Package Your Application

Your automated build process should generate the images for all your deployment packages. Ideally, it will also cut the CD or DVD you intent to ship, but most of the time, and ISO will do.

If you are at the end stages of your project when you are building your release candidates, you can use the packages you produced to ship your product once QA has given the OK.

Many people does not include the ISO generation as part of the build process, and they manually create it afterwards. Guess what, they make mistakes, and they have to recreate the ISO's more than once. Not a good idea when you are in a hurry to ship.

Make Sure You Can Execute All Previous Steps Independently

It is a good idea to isolate each of the previous steps and design them so that they can run by themselves. This will give you a lot of flexibility, and there will be times when you will need this.

You want the ability to just run the compilation steps to create a check build that runs with every check-in against the source repository. This will validate the state of your source with every changes and will allow you to catch errors quicker.

There will be times, especially close to release, where you will only want to re-master your application to include an updated license agreement, help file, or content file. You can achieve this if the application packaging is design in a way that does not depend on the compilation or testing of your application.

You may want to test your application every few hours to insure that changes have not affected stability. In this case, you will want to get the latest source, compile it, and run your automated tests.

Of course, once or twice a day, you want to run through the entire process to validate your production build. This does not mean you are going to release it or give it to QA, but you want to make sure your whole process works.

Notify Your People

It is inevitable. Your build process is going to break. Someone is going to check-in code that does not compile, files are going to be changed or deleted, dependencies are going to change. When this happens, you want to notify immediately to the right people.

A broken build process should be the first priority of your team, and no further development should be done (or at least checked-in into your source repository) until is fixed.

An automated build process is the best way to insure your application is stable at al times. It is the quickest way to get feedback about the changes that are going on in your project. And, you need to build your application to be able to release it. If you rely on manual steps to build your application, things will fail when you least want them to fail. Automation will help you to reproduce the same steps over and over again, and insure that you end up with a working product at the end of each cycle.

Technorati Tags: Software Development,Build Process,Agile Development,Programming

5 Reasons To Have A Documentation Strategy

Documentation is one of those tasks where is very hard to calculate the ROI (return on investment). How can you know how much money you are saving by having good documentation if you do not have an un-documented version of the same project. In the same way, how are you going to compare the time you are saving by not documenting your projects if you have not documented your project and see how long it took.

No, I am not proposing you implement your projects twice, once documented and once undocumented to be able to compare. But the truth is, we do not think too much about documentation until we need it, and by then, it is way too late. So just in case we need it, let's implement a documentation strategy.

First and foremost let me explain that I am not talking about the help files you should write for your application users. This task is better left in the hands of professionals; your tech-pubs department. Your documentation strategy is for the code that goes into your application. This is a more technical documentation, and should be written by the same experts that write the code.

I haven't convinced you yet? Well, let me give you the reasons why you should have a strategy to document your code, and let's see what happens.

1. Complexity Is Best Explained In Plain Language

Despite all the post and articles you read before about how good code is self documenting, how good programmers are able to read and understand code like if it was their first language, and how a clear use of patterns will help you to understand the design and architecture of your application. The truth is that, with the exception of a very small elite group of programmers to which I do not belong, most programmers understand plain language better than any code.

Can a programmer with the right experience go through hundreds or thousands of lines of code and understand what they are doing? Sure he can. But, it is going to take him a lot longer than if he had a few UML class diagrams and some HTML help files describing the API's and their use.

"Wait a minute. Are you proposing to use UML?" Not really, but there are things that are better expressed in a diagram than in plain words (you know, a picture is worth a thousand words).

2. There Are Things Code Cannot Express

Customization points for classes; assumptions being made; side effects. These are all examples of things that cannot be expressed just with code. There are many more, and they will play a very important role once your code moves to maintenance mode.

You might be thinking, "that is what comments are for." Maybe, comments are your documentation strategy. If so, you should define it and make sure everyone understands it, so comments are actually added to the code.

Documentation and code can get out of synch if they are not maintained. Your team must understand the importance of having up-to-date documentation and make sure pertinent changes are made to it when the code changes.

3. Documentation As Training Material

Once in a while, you are forced to hire a less experienced developer, and you will be glad you have good documentation to get him up-to-speed. As he glances through the documentation, he might learn a few tricks that will allow him to become a better programmer. He can still learn a lot looking at code, but documentation is a supplement that can make his life and yours, easier.

New developers usually need some kind of Mentor that guides them through the learning process. You can save many mentoring hours if you have good documentation, and everyone will be more productive.

4. Finding What You Are Looking For

How many times have you implemented some functionality just to discover a few days later that someone had already written the same thing years ago? Undocumented code silently leads to duplication because in medium and big size teams, it is impossible that everyone knows about every single line of code written by other members.

There are many tools out there that will generate automatic documentation in a way that is easy to search and find exactly what you are looking for. This is a big time saver and promotes code reusability.

5. Because We All Forget And We Should Forget

No matter how many items you can keep in your memory (and I am not talking about RAM), sooner or later you will forget how a particular feature you wrote works. It is inevitable. If you did a good job writing the code, things will come back as you look at it, but most of the time, you will be glad you documented it.

There are times when you really want to forget. We all have code we wrote in the past that we do not want to remember. If you don't, most likely it means you have not learned much lately. In my experience, a developer presented with the exact same set of requirements at two different points in time will come out with two completely different solutions. Developers evolve and learn new things that makes them improve. This will drive them to solve the same problem in different ways.

Remember the assumptions I talked about in point two? These will also affect the outcome of the solution. Because we know now what we did not know then, we probably make different assumptions, and many times this will prevent us from understanding what we did in the past unless we have something to refresh our minds: documentation.

Conclusion

Documenting code is not fun. Maintaining documentation is a nightmare. You should plan your documentation strategy before starting a new project, so you can write the documentation from the beginning.

Your documentation strategy should be part of your code writing process. Merging both things together will make it less painful. Select a strategy that is accessible from the source code, find the right tools for you, and make sure everyone understand the importance of this task.

Documentation can bring new members up-to-speed; it can help you to refresh your mind when you need it; and it will prevent you from making the same mistakes over and over. It removes code duplication by exposing existing functionality to your development team, and it will help you to express the concepts that are hard to find and understand by just simply looking at the source code.

7 Strategies Software Development Processes Must Define

In my previous post, I talked about the necessary flexibility in a software development process. There are different needs in different teams and projects, and the process needs to be flexible to adapt to those needs. I like to think about processes as a set of guidelines as opposed to a set of rules, but those guidelines need to be categorized in groups of strategies. Today, I will talk about what strategies you need to define for any software development process, no matter how simple your project or small your team is.

1. Source Control Management.

No matter how small your team is or how simple your project is going to be, you need a source control management strategy. Simple projects tend to be come more complex are changes are required, and changes always happen no matter how simple the project once started.

You need a strategy to manage changes in a project and understand who made the change, what was changed, why was changed, and how was changed. A source control management strategy will provide the information you need to manage changes and understand better the evolution of your project.

Even if you are a single person team, and you think you do not, you do need a strategy to manage your source, you do. Notice I am saying you need a strategy not a system. Keep in mind, I am not talking about a source control management system, which is a tool that can help you, but a strategy that you must follow to understand the changes in your code.

2. Documentation.

Any project to be successful needs a strategy to document the project. The level of documentation created might change depending on your actual needs, but any project, even a single person project needs to define a strategy for documentation.

What the project does, how was implemented, and why was implemented in that way might be very clear while your mind is deep involved in the project, but it only takes a few days after the project completion to forget a huge chunk of what was done.

Documentation will help you to remember. It will also help you with the maintenance of the project, and as a tool for new members of the team to understand your applications.

Again, I am talking about having a strategy to document your project. I am not suggesting a particular way of doing it. How you do documentation will change depending on your needs, but always keep flexibility in mind.

3. Automated Build Process.

Any software development project will produce a set of deliverables that will end up in a user machine in a way or another. An automated build process will help you to generate those deliverables faster and more accurately.

Manual processes are error prone, and we tend to make more mistakes when we are working under pressure. You do not want to risk a manual mistake the day you have to provide your customers with your final product. An automated build process will help you to deliver your project faster and insuring no manual errors were introduced during the production of your application. It will save you a lot of hassle, especially in the final stages of your project as you get closer to release date.

4. Coding Standards.

I have read many things in favor and against coding standards. What I usually see is that people against a coding standard do not really understand what a coding standard is, and that misleads them to think they are not necessary.

When I talk about coding standards, I am not talking about where you should place your brackets or what level of indentation you should use in your code. It is not about whether you should use the TAB character or spaces when indenting code. Those are very personal preferences, and that is why there are so many arguments about them and no real consensus.

A coding standard should define the guidelines and idioms to be used while writing the code. A coding standard can help to decide, not dictate, whether something should be implemented in a way or another, and it usually eases the understanding of the code by all members of the team.

5. Testing.

How your are going to test your code at the different levels is one of the most important decisions you need to make to make sure your project is successful. You must define a testing strategy that accommodates the different levels of complexity in your project, from unit tests for low level functions, methods, and classes, to system tests that prove the application as a whole.

The sooner you start testing, the better. The sooner you define your testing strategy, the sooner you can start testing. The better you define your strategy, the more chances you have people will follow it.

Testing is the best way to find defect in your software, and the faster you find those defects the easier it is to fix them. Never start a software development project without a testing strategy in place; otherwise, by the time you start testing, it might be too late, and your project will go out the door without the quality customers expect.

6. Code Reviews.

Code reviews are very important, not only because they provide a way to catch defects soon, but because they also serve to keep people informed about what is happening in other parts of the application.

They are also an invaluable resource when it comes to coaching and mentoring less experienced members of the team. The experience a new comer can acquire from a more experienced programmer through a code review is invaluable. Code reviews will help your team to gain experience, to do things better, and to understand what is going on.

Your code review is tight together with your coding standards. The review process should insure your coding standard guidelines and idioms have been follow, and should stretch the importance of following those guidelines and idioms.

Do not underestimate the power of code reviews. I know they take time and are a real pain. That is why you need a code review strategy that adapts to the needs of your team.

7. Communication.

The way your team communicates is of high importance on any software development project. Establishing a strategy to communicate within the team is important to insure all communication is as efficient as possible.

Many software projects have failed because of lack of communication. In today's world, there are many ways you can communicate, all of them have their strengths and weaknesses. It is important to define which way we will communicate under which circumstances, and your team most be clear on the communication protocol.

There are many reasons why people do not communicate with others. There are also times where people communicate too much in a highly inefficient manner. Your communication strategy should be clear so everyone understands it and uses it in the best possible way.

These seven strategies are indispensable in any software development process. Unfortunately, sometimes we do not think about them way in advance, and by the time they cross our minds is more difficult to implement them.

In subsequent post, I will dive more into each of these strategies to make very clear what I really mean about them and why I consider them so important. Let me know if you find I am missing some important strategy or if you disagree with any of the ones a expose.