This is an old post written in November 2012 and published my old blog. I am publishing it here because I believe most of the thoughts presented have become true. We have Vagrant and Docker for fast onboarding, modern frameworks allow applications to be built very rapidly, the rise of continuous integration tools proves rapid release cycles and automated testing are popular, and the domination of Slack shows communication is as important as ever.
Application scalability has evolved. I think a lot of technologists, from developers to operations, believe that application scaling means to simply add more servers behind a load balancer and call it a day. That might have been the case when writing web applications was a new medium of software development in the late 1990s and early 2000s, but the last several years have completely changed what it means to write a scalable application.
Modern application scalability is all about speed. With as competitive as writing software is today, speed is not only a feature, but a complete game changer.
On-Boarding New Developers
Writing a scalable application means you need to be able to allow new developers to work on it as quickly as possible. The delta between when a developer joins your organization or project to when they can begin hacking on it should be measured in minutes, not weeks, days, or even hours.
Have you ever started at a new company and not felt useful on the project you are working on until several weeks into working on it? I have. That application is not scalable. Ideally you should be able to arrive at work on your first day, open your new computer, clone the project you are working on, adjust some build settings, and immediately build, test, and deploy the project (locally).
A scalable application allows developers to work on their own local machines, running their own local databases, services, cron jobs, and other necessary applications. A scalable application allows developers to work entirely in their own sandbox.
The ease at which you can build a new version of your application determines how scalable it is. A slow build time means you can not respond well to increased load. Large applications (500,000 LOC or larger) should take several minutes to build. Smaller applications should take several seconds.
If you are constantly fighting the build process, and are afraid of deployments, your application is not scalable. Take the time to reduce the delta from when a change is made to when it is pushed to production. It is worth spending the time to build a configuration system so you can turn on and off features in production. This way, you can build out features, push them into production, and turn them on and off as needed. This is a powerful mechanism to reduce the number of errors you have after releasing code that has been under development for several months.
A scalable application has rapid release cycles. Rapid release cycles find bugs quickly and respond to new requirements faster. As soon as your code is written, has passed all of its tests, and been reviewed, push it to production. Having tools already in place to monitor failures makes it easy to determine how your code is operating in production. Your goal should be to release more high quality production code as fast as possible.
We have better tools, development practices, and hardware now to continue allowing developers build in a silo for three months and then spend four sleepless days releasing their code.
Teams must communicate well to build a scalable product. This includes non-development teams as well. Inefficient communication leads to poor product development. We all have been members of email threads longer than a book. Information is lost or disguised, and building a product that scales to your customers demands becomes more and more difficult.
Communication needs to be efficient to build a scalable product. Developers need to be able to communicate clearly and efficiently between themselves. That means quiet working conditions and the best tools possible. A developer can hopefully express herself most efficiently with code, so having good code review tools and practices in place is a must. Code reviews primary purpose is not to catch bugs, it is to spread knowledge.
How quickly can sales managers communicate to your customers about production ready features? The faster you can onboard new customers to your product will help you build a scalable platform. You will find your pain points quicker and patch them faster. The Marketing and Development departments should work hand in hand. Everyone at the company is responsible for marketing the product, and everything they do markets the product.
A scalable software platform has great test coverage. With a great suite of tests, developers can easily mimic different load scenarios with the product without having to release it into production first. They will immediately know if their changes affect the speed of their product negatively.
Customers pay your company for quality work. Releasing software without tests is not releasing quality work.
Spend the (short) amount of time it takes to have your developers learn testing. Spend the (short) amount of time it takes to write tests for your application. It will pay off dividends in the end
Ability to Change
Application scalability boils down to one thing: the ability for a team to change rapidly. Each of the sections above relates to rapid change. All of it can seem overwhemling if you are used to a slower development pace. As you add more of the above suggestions into your development flow, you will notice a snowball effect.
Each new change will piggy back on top of the previous one and the delta of time to get it successfully integrated into your team will be smaller than the previous change. For example, getting your developers to write tests may take two weeks. But as soon as they are writing tests, their code reviews will increase in quality because they are reviewing the code and test changes. Suddenly, bugs that were not clear earlier scream out of the screen. Knowledge will spread quicker, and developers will on-boarded faster. Releases will take seconds rather than hours. Communication between developers and sales managers will increase because developers will be excited to show off their new working features.
Application scalability has evolved. To be a successful software engineer, you must evolve with it.