As a company, NML organically grew over time from a small group of talented developers working on one or two projects, to a company with 60+ people with complementary skills working on multiple projects over long-term contracts. Organic growth is necessary from such humble beginnings, but it tends to breed bad habits and can blind you to flaws in processes and technologies.

Of course, bad Software Development Life Cycle (SDLC) from organic growth is by no means an inevitable consequence and companies may find that they have issues only in their processes or their technology stack. In our case, however, it turned out to be both!

Organic SDLC

We have always endeavored to be an agile process company, and we have implemented SCRUM concepts and at times Kanban on our projects. Of course, "agile" being a nearly meaningless buzzword these days, and SCRUM being, at best, very poorly implemented industry-wide, our results varied wildly from project to project.

Our SDLC at about a year ago was roughly some variation of the following:

[EDIT: To be sure, the following describes only some worst-case scenarios, and is not a reflection of the status quo across even most of the projects at NML at the time.]

1. Requirements gathering and analysis
2. Planning and story breakdown
3. Development
4. Testing
5. Bug fixing
6. Deployment
7. Chaos
8. Hotfixes
9. Testing
10. Start at 7, or if lucky, proceed to 11
11. Retrospective bemoaning insufficient or inaccurate requirements, too little time, not enough testing, not enough visibility on work and progress, etc, etc
12. Start at 1

Looking at the list, one can almost choose any number at random, and find problems within that area. Which is exactly what I started doing when it became part of my responsibilities at NML.

In short, we had the following key issues (by no means a complete list):

No process template

Agile is strong on independent, self-organizing teams, but throughout IT, it seems we interpret that as "figure out what works and just go with it". NML interpreted this no better at times, and we had as many processes as we had projects.

Personally, I dislike processes and structures, as more often than not, they seem arbitrary and aimed at constraining instead of guiding. A guideline is there to help you along a path and sanity check your progress as you move along, without dictating every step and decision. As such, it should be able to serve in more than just a singular context and help problem-solve exceptions.

I have learned that if processes and structures are developed collaboratively, and not dictated, they tend to serve as better understood guidelines. I think one of the main reasons for this is that in order to be collaborative, you need to place good communication high on the list. Goodness knows I have missed that key aspect a couple of times this past year!

Insufficient clarity

The quote above is from the Agile manifesto. I am a fan, really, I am. This particular line, however, has been abused to within an inch of its life, and it generally has come to mean no documentation.

I actually am a huge proponent of super lightweight requirements gathering and little documentation. That is not the same as superficial requirements gathering and no documentation though!

The key to successfully minimize heavy requirements gathering and reams of documentation lie within another line from the Agile manifesto:

It is imperative to have excellent customer collaboration. I feel that "contract negotiation" here includes requirements gathering, and ensuring that customers' invested involvement is the only way to have any chance of achieving working software over comprehensive documentation. Poor team cohesion

If you ask most developers what the make up an average-sized agile development team is, invariably the first role listed will be "developer". Now I have not done this exercise, so the following is purely based on my experience as both a developer, a team lead, a project manager and a CTO, but if you ask a tester, or PM the same question, they will give the same answer.

A huge shortcoming at NML was the fact that development teams were developer-centric. Of course, nothing can get developed without a developer's involvement, but I think one can argue that no effective development can occur without at least a project manager (replace scrum master and product owner if you will), developers, testers, and the customer.

It is not good enough to just develop software. The goal should be to develop software effectively. This spans all disciplines within a team, from effective team management to coding and most definitely also testing.

Without excellent cohesion between these disciplines, being effective becomes nearly impossible.

Stale tooling

The argument could be made that the technology stack and tooling one uses for driving your SDLC is not important for the success of delivering projects, and perhaps it is not. But I think it would be a grave mistake to underestimate the value add of using the right tools for your specific circumstances.

At NML, we have always used Atlassian JIRA. It is a very comprehensive project delivery tool with great community support and loads of useful third-party applications. However, in our context, it was completely counterproductive to our efforts. It was not the right tool for us.

Another example was our testing automation efforts with Katalon studios. In this case, however, there is nothing wrong with the tool, but we were addressing the wrong problems. We opted for tooling over understanding our actual shortcomings. And now? What now?

Here are how we addressed the above concerns:

• We aligned our project SDLC across the board. We still do not have a "document" that lays down the law, but we collaborated across teams to identify the aspects that work, the aspects that do not, and the aspect that cannot change. We now have a mostly identifiable and familiar process across all our teams.
• We have always been strong on customer collaboration. One of our strengths is that the responsibility for gathering and understanding customer requirements lies with technical team leads and project managers. They form a unit that has both the technical insight and communication know-how to work with the customer and the team. So the major ticket for us to fix with regards to clarity was to specify a minimum set of standards teams should comply with to be able to effectively address requirements. Decent acceptance criteria is a must.
• Our testers are often multi-project team members, and we fell into the habit of excluding them from project planning. The result was that there was very little understanding between the developers and the testers. We addressed this by pulling testers back into planning and making them more integral to the SDLC. The team must identify and add tasks for the tester, which will result in one or more test cases. It improved our overall project delivery visibility and forced us to look at our testing efforts in a whole new light.
• We adopted Azure DevOps. We are a Microsoft technology stack company, and most of our projects end up in the Azure ecosystem. Our project SDLC is much more efficient as we do not need to hop between different tools to push various steps of our SDLC along. Just the efficiency of not having to log into 3 different systems with different credentials is saving us oodles of time and frustration.

Conclusion

We are nowhere near the end of a journey, which to be honest, should have no end.

I can without a shadow of a doubt say that we are in a much better position than a year ago, and I feel that the teams themselves are happier with how they work currently.

We continue to improve and look out for more opportunities to become more effective. From time to time some old habits creep up but along with the amazing people at NML, I look out for these and re-evaluate our processes again.