Inner Source - the key to improving software asset reusability

Some background first

Can you afford to re-invent the wheel? I’m sure most of you will say a big NO. Similarly, in software development, it’s not very common to build a piece of software from scratch unless you are building something which was not done by anyone before you in the world. Typically, a development team gets help from a platform like GitHub, or get help from other similar projects running within your organization. They copy the base code (let’s call it the initial codebase) from others and start developing on top of that. This is re-using the wheel.

This is one way of looking at re-usability, since they have re-used someone else’s code. When we talk about improving the re-usability of software assets (code, libraries, etc.) within your organization then it inherits a few inefficiencies. Especially when you do not have a mechanism of “Giving Back”.

For example, what if a project team who is using your initial codebase improves it over a period of time to make it more efficient? what if they find a few bugs in the initial code and fixes them? what if they address a few new security vulnerabilities? How do you now make sure that your initial codebase receives all these improvements? and this team can communicate back to the original creator of the base code? how do you make sure that if some new project team wants to re-use this asset, they get all the improvements as opposed to code their own improvements on top of the initial base code? how do you make sure you pass on these continuous improvements forward to your future software engineering teams?

Well, you need to build a culture within your software engineering practice to “Give Back”. It is giving back to the original codebase by continuously contributing the improvements. It is like helping the community (your entire software engineering practice) with their contributions.

Inner Source is a way of software development which allows you to create this “Giving Back” culture within your organization. A transformative cultural shift towards gaining efficiency and improve software asset reusability. Let’s see how –

Understanding Opensource and Inner Source

To understand Inner Source, you need to understand how Opensource software development works.

In the opensource, the entire community of students, hobbyists, professional programmers contribute from across the globe abiding by opensource licensing norms. The entire community participates in contributing in coding, bug fixes, providing feedback to build great software like Android, Kubernetes, Visual Studio Code, etc.

In Inner Source, it’s not open for the entire world. It is open to all of your software engineering teams within your organization. Be it co-located teams or remote teams. It’s like running opensource development within your organization’s firewall. Where your entire software engineering team contributes, collaborates, and innovates together.

In short — Opensource is open to all, Inner source is open to all your employees within your organization.

Inner Source is not a new kid in the block, this terminology is there for almost a decade now. It was originally coined by Tim O’Reilly in 2000.

As per Wikipedia – Inner source is the use of open source software development best practices and the establishment of an open source-like culture within organizations for the development of its non-open-source and/or proprietary software

Now the question is — can you also embrace Inner Source development techniques for your engineering team? Let’s understand next a few of the challenges with typical software development methodology and see how Inner Source can help.

What is not working so well?

To start a new project, you need a base to start with. For example, an initial code structure that supports all of your organization’s best practices. You need to have some cross-cutting code — such as authentication, authorization, logging, code to remediate various security issues, etc.

In a typical software development practice either you start writing all the code from scratch, or there is a high chance that you take some sample code from a GitHub-like platform, or if you are lucky enough you copy it from other existing similar projects in your organization.

It gets worst for a newly joined team member or for a newly formed team. They do not know a lot of people to ask about reusable components; they don’t know whether there is any already existing code repository present or not? they can’t search your organization’s proprietary code repositories (because there is no such common platform open to all for searching). So, they end up writing their own code from scratch. Kind of re-inventing the wheel. Isn’t it?

Let’s see how it affects in reusability and continuous improvement. In the above illustration, each box represents a project team. In this example, the Initial project Team may have some base code that can be re-used in Project 1. However, if the Project 1 team is aware this base code, then they may copy the entire repository over and start writing their code on top of that. Now if they find some bug on that, they fix it, if they want to add some new feature or enhancements then they commit those in their project-specific private repository. Now, if Project 2 team needs the same base code to start their project with, then they do not have a streamlined process to get all the enhancements/ updates developed by Project Team 1. It’s just because these teams are working in silos, they are not committing the potential re-usable code to any shared repository. It remains closed within their project boundary. This prevents the flow of continuous improvements over a period of time. And finally, each and every team ends up spending a lot of time doing the same thing again and again their own way.

This is how any organization loses the flow of knowledge from one project to another. Now you can imagine how much productivity you are losing just by repeating the same thing again and again in every project. You lose control over your organization’s coding standards, practices, security policies, etc. which are crucial for you to build re-usable software assets.

How Inner Source can fix it?

Now think a bit differently… you have a GitHub-like platform where your developers constantly commit all of their code improvements to a common repository and is searchable. Your code is always updated with the latest bug fixes, improvements. Now for a new project team, he doesn’t have to start all over again. They just take the reference to this updated repository and they are good to go.

If you look at the above illustration, the Initial Project team has committed their code in a shared repository. Now Project Team 1 forked the initial Codebase and started adding their bug fixes (if any), new enhancements, and a new feature to it. Once they commit the enhancements to their project-specific private repository, they create a pull request to contribute their improvements to the main branch of the code. The main branch of the code repository will be maintained by a group/induvial team members called as maintainer. If the maintainer accepts the pull request, then all the enhancements added by Project Team 1 gets committed to the main branch. Now if tomorrow one other Project Team 2 wants to start a new project, they just fork from the main code branch and they get all the improvements/ updates added by Project Team 1 automatically. Similarly, if Project Team 2 wants to contribute their improvements to the main branch, then they do the same thing, a pull request. Once the maintainers approve the Pull request, the main branch gets updated with the latest improvements added by Project Team 2. This is how the continuous improvements keep on flowing forward over time.

This is how your organization’s intellectual property code gets matured over a period of time. The code repository in this case is open for all to look at, submit enhancement ideas, raise a bug, use the code in their project and enhance it and submit contributions through a pull request. This gives your developers an open playground to collaborate, communicate their innovative idea with other developers, architects in your organization. This helps you to develop, reuse and improve your next big product, service, or software asset together.

Implementing Inner Source in your organization

With all the benefits of Inner Source, it comes with its own challenges. Inner Source development is a fundamental shift from how we generally do software development within our organization to a more open, contribute heavy software development. It needs a cultural change within your software engineering discipline by being open to contribute (Giving Back).

Since it needs a cultural change, participation from all the level such as the leadership team, project management, development team is required.

Here are a few common challenges of Inner Source

You may not be able to justify the funding from your client to improve your internal intellectual property
For any small organization having a dedicated team of the maintainer is not a cost-effective option
Getting maintainer’s constant attention to maintain the repository all the time at times becomes challenging because they may be on some other client billable project
Similarly, oftentimes the contributors may also think what is in there for me?

To address the above challenges, the leadership team and project management teams

Should not look at it as a short-term investment
Promote the culture of openness
Reward maintainers and contributors periodically based on various matrices
Use simple matrices like the number of successful pull requests, build quality improvements, number of bugs fixes, number of downloads of a library, the highest number of contributions to recognize maintainers and contributors

This needs change in the way a developer thinks as well

Developers should be looking at the bigger picture
Develop modular code with reusability in mind
They should be passionate about giving back
This is the greatest way to pass on their knowledge and learnings forward
Learning from someone else’s mistake or the success is the key to self-improvement

As the organization matures over time, they build many such reusable components, patterns, practices for the future development team. Moreover, it allows you to promote an open development culture where everyone’s ideas, suggestions are welcome to build a huge software asset library.

If you have come so far in this article then you must be wondering are there any patterns and practices for implementing Inner Source in your organization? well, the answer is Yes. InnerSource common is a community that is dedicated to create and spread knowledge about Inner Source. Their beautifully crafted Inner Source patterns are documented here. This will give a you very good stat to embark on the Inner Source journey.

Wrap up

This article is more on software asset reusability, but Inner Source is not limited to reusability only. Inner Source helps you to improve efficiency, productivity, it improves the quality of your software assets over time, it creates an open culture of innovation. Many big companies such as IBM, PayPal, Microsoft are benefiting from Inner Source for many years. It has the potential to transform your entire software engineering practice if it is you are committed to change the development culture.

In today’s context when industry is moving towards the Everything as Code approach, if you combine it with DevOps then you can take Inner Source to the next level. Especially if you are building a new software engineering team then in my opinion you must adopt Inner Source. In a long run, it will help you to come up with your own GitHub-like platform with all reusable proprietary software assets for your employees to use, contribute, collaborate and innovate together.