You might think that once you kick off the development process, most of the work should be done by developers that you’ve hired. While that's true to some extent you still have to spend plenty of time road mapping, planning, and managing the process. A big chunk of work that needs to be done from the day one is continuous testing, bugfixing, and gathering feedback. In an agile project, you need to plan to deliver the feature that is useful to the users and can be tested by them. Once the sprint ends, you need to get busy reviewing what’s been done, testing for bugs and getting customer feedback on new features.
There are always bugs
One of the biggest reasons why you need to start testing early on is the number of bugs that software has right after implementation. Bugs are a fact of life for any software product, no matter how big the team is and how professional team members are. Some bugs exist because of the complexity of underlying systems and features that affect each other. Others are introduced because of unclear of plainly missing details in specs. Finally, even the most seasoned developer makes stupid mistakes every so often. There’s nothing you really can do about bugs, the only way to address them is to plan for them, start testing early on and fix the most critical ones as soon as possible.
Smoke tests
Every feature that has been implemented during the sprint have to be tested end-to-end, from the user perspective. That's where your lack of technical skill will be a benefit because you will think the same way as your users will think (unless you are building apps for software developers). The problem is that developers, no matter how good they are, have professional deformation. For a developer it’s ok to send a password reset email with a long password reset url that is not clickable — something that is so obvious from a UX perspective (just make the link clickable!). For a developer it’s a minor feature and something that’s not important — the password reset does work, doesn’t it? Take your user stories and follow ‘happy paths’ of your users — actions they need to perform to complete their task in your app. This way you will ensure that the feature is working the way you want it to work.
Regression tests
With every sprint, you will have more features implemented and more code written. More features and code means more interdependent functionality and more moving parts that have to be tested. To make sure that your app still works as it should you need to test features that were implemented previously periodically. That can be quite time-consuming as your codebase and list of features grow, but it’s the only way to ensure that something that you implemented four months ago is still working. Otherwise, you might figure out that login through facebook stopped working after login with email was introduced. It mightbe late to be able to fix it quickly at that point. Take the user stories that have been implemented and go through them at least once a month.
Hire QA engineer
Testing might quickly become a cumbersome activity if the amount and complexity of features you are implementing are high. If you have the budget consider hiring a QA engineer who will do the testing for you. A QA engineer takes user stories and uses them as test cases to perform the actions on user behalf. He can perform regression tests and smoke tests weekly to free up your time while ensuring that the app is stable and working as expected. Note that you still need to perform manual testing every so often — sometimes QA engineers make a wrong judgment about UX which can lead to problems if not spot on time.
User testing
Testing for bugs is necessary, but proving that you are building a usable product that is valuable to the end users is essential too. Don’t wait for the launch date to start testing the product with the users, or you might be surprised how far your product is from what users need. The goal of an agile project is to build a valuable set of features every sprint, so after second or third sprint you should be able to show something. If you create a launch team of early adopters you were interviewing at early stages, they will feel privileged to get early access and influence the features you implement. Of course, you need to make sure that most critical bugs are fixed before testing with users.
There are several ways you can perform user testing. The easiest one is to show a demo of the implemented features during the in-person or online meeting. You can guide the user through what’s been done, look at their reaction and listen for the feedback. This way of getting feedback allows you to skip problems or bugs that your early-stage product has intentionally.
The other way is to let the user use the app himself. It requires more effort than just showing a demo but gives you much more valuable feedback — you can see the user using your product. You can even get the user using it periodically to solve his problems — it would be the best validation that the product is useful and valuable.
Prioritize and iterate
Implementing features, testing for bugs, fixing bugs, testing with users, evaluating and revising the roadmap — that's what the lifecycle of an agile project looks like. You can see that implementing new features is just the part of what needs to be done every sprint. After you test and learn what bugs you have and what needs to be done differently from the user perspective you need to prioritize. It’s impossible to address all bugs and user requests at the same time, no matter how hard you try. You need to fix most critical bugs first and select user request that has most business value, leaving everything else for the future. The chances are that with the end of next iteration you will get more user requests and more bugs. Big projects have bugs and requests sitting in the queue for years just because there’s always something more important.
