What exactly the software estimation is and why it’s so important?
Cost estimation is very important element of software development process & it helps enterprises to make cost-effective business decisions. However 50% of the time effort & cost estimations get wrong that results into overshooting of cost and efforts of the project. We generally estimate project efforts & cost in response to the RFP (Request For Proposal) if it is a fixed cost project or during the initiation phase of the software development if it is a T&M project.
Custom or COTS (Commercial-Off-The-Shelf) is a common question we hear from our customers when they come to us with a request to develop a solution. The exponential growth of everything digital makes the market an increasingly tight environment for businesses to compete within. The time to market becomes one of the main goals for business to pursue when developing a new digital product. To cater to the needs of customers, software development companies are continuously evolving their delivery methodologies (with convenient development shortcuts) to increase velocity of project release and its roll-out to other branch offices of customer.
More accurate cost estimation keeps the software development company highly motivated with higher project success rate. A good project estimation can make things easier during the project execution cycle. In other words, many a times, a good project estimation will lead to a successful delivery of the project, while a wrong estimation may end up in project failure. Several factors are involved in producing precise cost estimates for the software project rather than just estimating some numbers in a Excel file.
Cost estimation for a software project depends on various factors which software architects generally consider during efforts estimation. Those factors are like below :
- Number of features in the software
- Complexity of above features
- Different layers of the solution
- Complexity level of non-functional requirements (application performance, security layer)
- Programming languages
- Technology to use
- Licensed / open-source software libraries / tools / components (whether customer arranges them or we need to procure and supply them in solution)
- Re-usability factor
Now, let’s see why software project estimation process is so complex !!!
Complexity #1: Gathering detailed user requirements
Whenever any software development project is started, it starts with an objective which has some defined requirements and functionalities underlying it. One simple example is – development of Industry 4.0 point solution for a manufacturing industry. A complex example could be – development of Manufacturing Execution System (MES) for a manufacturing industry. So in both examples, we have a clear objective to achieve but if we go bit deeper into this objective then we will find more detailed requirements. In case of the point solution example, besides technical stuff of deployment / hosting and domain etc., we need to define various pages, sections, contents, images and reports etc. On the other hand, in case of complex MES system development, more complexities are involved in terms of various stakeholders, technical and functional details, machine connectivity needs, types of securities and system integration needs etc.
To build a customized software, the requirements are generally never clear prior to requirement gathering phase. Requirements keep evolving. Even if, they are well-documented in RFP (Request for Proposal), they are never synchronously understood by all the stakeholders. Additionally, because of the new on-premise web or cloud platform, there will arise various non-functional requirements like performance, multiple browsers, multiple devices, dependency on other IT-OT system, data security, network bandwidth etc. which also invite complexity in the project. When you are quoting during software sales, your estimate can go off by more than 50 %.
Complexity #2: Unclear requirements
Whether the project is small or big, every project with some objective will have huge set of clear and unclear requirements. These requirements may be clear at the beginning of the project or will be clear as we progress on the project development. The uncertainties about the complete requirement at the beginning of the project makes the project estimation a complex process. Sometimes an unclear objective will also add up to complexity in estimation!!! Another biggest issue is the uncertainty involved at the beginning of the project. Many times even the customer is not clear about the whole requirement of the solution needed. If there is no complete clear requirement then how it’s possible to estimate it in term of effort and time?
Complexity #3: Integration of new solution with existing IT-OT systems
Many a times in complex projects, system integration task plays a key role. But software companies sometimes underestimate efforts for system integration tasks. Explaining complexities involved in system integration to a customer is tricky. They may say that you can just take the API data and push it through. Yes, you could. But you then have to test it and make sure everything works together. And then, when you throw in multiple devices and platforms – like web applications, backend services, Windows applications, mobile applications, HMIs of machines — all of this suddenly exponentially grows to the point where you turn around and say to a customer that to test all of this properly is going to cost additional thousands of person-hours!!! Read my blog on System Integration Complexities for more details.
Complexity #4: Prediction of reusability factor
This is applicable when roll-out of developed solution is expected to happen in multiple branch offices of customer with some level of customization in the functionality is foreseen for each branch office.
At the time of proposal submission, it is very difficult to predict “% code reusability” factor where the solution is expected to be deployed at multiple branch offices of customer that involves customizations in software functionalities for each branch office. Normally software companies put lot of efforts while developing a “product” where “generalization” of functionality becomes very important which becomes “base layer” of the product. However software companies will not get sufficient time & cost to do the same in “project” development.
Complexity #5: Choosing right technology stack
These days, technology is moving very fast & continuously changing. There are patches and versions of each technologies which you need to keep an eye on. Patch or new version of technology might have resolved some vulnerability or major issue in the technology stack. By the time you start software development, some new technology or a newer version is beckoning you and it is tough to decide whether you opt for the current stable version and compromise on the new features or try the new version with some amount of risk.
Complexity #6: Outsourcing of work packages sounds easy, but difficult to manage
No one should deny that outsourcing of project can bring several benefits to a company. But no one also should forget about the challenges that we face by outsourcing the work (whole or part of project). Challenges are like – misunderstandings in communication, mistakes regarding technology, misinterpretation of bad quality requirement specification document, incompetent engineers in outsourced team, outsourced team sitting remote location and many more like this. This way challenges will occur at every stage of the outsourcing service – starting with the selection of the vendor, through the implementation development process, up to the delivery of the final product.
The key to success is to choose the right IT supplier and determine the course of collaboration in detail. Also, ensuring high quality requirement specification is very important while outsourcing any part of the project. If you take care of all such aspects seriously, you will not only avoid problems but also receive a high-quality deliverables from them.
Software project outsourcing complexities
Common mistakes we do while choosing our IT partners are highlighted below. Hints to overcome those problems is also briefed below :
We choose the outsourcing partner based only on the price of the service. Unfortunately, for many businesses executives choosing an outsourcing partner focuses mostly on the cheapest offer. We have to remember that a well-performing final product is a key to success. While choosing a partner we should evaluate the aspects that are crucial for successful delivery of the solution. Therefore you should pay particular attention to various aspects like – their experience in the domain, their testimonials and customer feedback, technologies they work in, etc. Well-done research will narrow down your choices quite a bit and make it easier for you to avoid working with an inexperienced vendor whose skillset won’t meet your expectations. Remember that, nothing can spoil cooperation like sudden information about additional costs while working with our partners. To avoid disappointment, ask at the very beginning of the partnership about any costs that could increase. Find out how to handle any billings and overtime hours that weren’t included in the software project estimated completion time. Be sure that all of these arrangements are written into the contract and set a notification interval for changes.
Outsourcing brings communication challenges also. Business executives decide to outsource project because they are not capable of having an in-house development team. But often, in addition to geographical or language barriers, we see differences in the culture of the organization, which can cause many misunderstandings and obstacles in the workflow process. The key to this problem is to ensure regular communication with outsourced team and they do have QA team to check the quality of deliverables. Creating a clear information flow will help us avoid communication problems.
Supplying poor quality requirement specification will make your outsourcing partner to fail. We cannot treat the requirement specification as a sideline. If we don’t know how the finished solution has to look like, we can’t expect that outsourcing partner will fulfill our blurry vision. Make sure you are clear about the goals of the solution delivery and the accuracy of your expectations in the requirement specification documents.
One another major problem that faces companies choosing to outsource software development services is taking care of the organization’s data security. To take care of confidential information and intellectual property, several steps need to be taken. Experienced software development partners can propose solutions that reduce this threat to zero. Be sure to share your concerns as well as any regulations the third-party provider should be aware of. It’s also necessary to sign a non-disclosure agreement (NDA) that will ensure your company protects confidential business data.
Few tips for overcoming estimation related issues
Last but not the least, few tips for overcoming estimation related issues are given below based on my personal experience…
Split the bigger tasks and estimate
Many times the estimation is taken keeping in mind the bigger tasks instead of splitting it into smaller tasks for proper estimation. Such estimation will definitely will lead to the overhead tasks at a later stage. A method is to create WBS, work breakdown structure, which enlist all the sub-tasks and their efforts in detail for complete effort estimation.
Involve technical team for estimation
Estimation must be done by the technical person or in assistance with the senior developer. Sometimes the estimation is not done by the technical team which may lead to huge mismatch in the estimation. Remember that estimation should come in the range of “time frame” rather a “particular timeline“.
Example: 5 to 6 months is much better than a 5 months estimation. Customer always calculates deadline for the project delivery based on your estimation. If technical risks or critical dependencies are foreseen at the time of estimation, it is always good to highlight them to customer by showcasing possible efforts variation in the beginning of the project itself.
Take care of risk buffer & dependencies
In addition to standard technical risk buffer of 15-20%, also consider the buffer for things like skillsets/experience of the team and complexity of the project. Dependency of project’s internal as well as external factors are not considered most of the time. It can be in terms of some functionality like external IT systems integration or some license cost for some OSS component(s) etc.
Asking questions, obtaining clarifications and documenting them
Software development team should ask as much questions as possible to customer to go deeper to clarify all the requirements. More clarity will always lead to better estimation. Customer will also appreciate this in some way as they will come to know about some aspects which they didn’t even think about till now. Customer must clarify all functional, non-functional and technical requirements separately for better understanding. PM & architect must drive to clear up all uncertain gray areas in the project requirements.
+++ Please feel free to give feedback on this blog (written based on my personal experience) in comments field below. Many thanks in advance 🙂 +++