Zero Defects Code: Vertex Launches Custom
Project Management Solution for a Fortune 50 Customer

Each profession has its state of nirvana (eternal bliss). A baseball pitcher’s nirvana is throwing a perfect game, not allowing a single opposing batter to get on base. A cricket bowler’s is getting a hat trick – three outs in three consecutive balls. These feats are rare, so when they’re attained, they are celebrated.

Software developers also have a state of nirvana. That is producing zero-defect code. As in baseball and cricket, this feat is seldom achieved. Even the titans of our industry – Microsoft, Google, Apple, IBM – talk about zero-defect code as a holy grail they only dream of.

Vertex recently achieved this holy grail by developing a zero-defect software system for a custom application we developed for a Fortune 50 US-based enterprise. Our feat is even more satisfying because we were able to produce high-quality software for a highly complex business problem for which the customer had only a strawman of what they wanted to do and had no specific requirements.

This is a story of how the talented Vertex team achieved a feat that few have done in our industry.

The Customer and the Problem Statement

For confidentiality reasons, the customer cannot be named, but to set the context, let me just say that they are a Fortune 50 CPG company with over 80 brands operating in most of the global regions. Most products and brands are sold worldwide, with some variance for regional demographics and preferences. Product launches (called initiatives) are a big deal with the customer; it takes bringing together globally distributed teams to plan, design, and execute product launches.

When a new initiative is launched, there are usually around 1,000 collaborators spread across several world regions, and functional roles and technical expertise that come together to gate the decisions that need to be made to keep steady and timely progress. The customer had no effective tool to capture, communicate, track, and keep up with such a humongous undertaking.

This is where we entered and built a custom solution.

Diagram

 

Building the Vision and Evolving Through Discovery

All engineering projects are guided by a few basic principles. When you build a ship, you start by designing the hull of the ship. When you build a house, you start with building the foundation. Software development projects, like other engineering disciplines, should start with defining the requirements of the solution. In our situation, the customer only knew about the problem that they wanted to solve. They did not have a vision of what a solution would look like. We used the requirements gathering stage to define the key problem statements and build a vision of what the solution should do. We spent about 30% of the entire duration on the requirements definition. This helped us to align with the customer to build a holistic scope of the system. The requirements document thoroughly defined the project boundaries and were built iteratively with customer’s inputs.

Design First

In a trade school, carpenters are taught to measure twice and cut once. In software engineering, it should be first design, then build. Our approach was thorough and detailed. We were launching the project for the Initiatives team of just one product category. Our customer had told us that if the solution was successful, they were going to open it up for all categories. We knew that our design had to accommodate changes in the future.

Data Model Design: The primary goal of the data model was to keep the database lean with lesser objects, denormalized to reduce network latency and the number of calls. We avoided direct relations in tables, limited applying constraints, and eliminated all kinds of cascading operations to reduce over-dependency on the database. All kinds of database programming were eliminated. Constraints and on-demand relations were controlled from the program’s entity framework model to allow more flexibility to the overall design.

 

Application/API Design: The application design went through phased stages (refer to diagram), something we did not try before. The principle was to establish a working prototype or template solution using few modules (as API or services). Once the prototype was tested and benchmarked the team was ready for continuous programming for the rest of the application building and focus on the functional aspects. The primary concept of API architecture was to identify services from business contexts, define micro-modules adhering to the core principles of microservices, build communication patterns between modules and yet develop it for a single Cloud Platform-As-A-Service (PAAS) solution. Each module represented a business functionality in the product and was designed to work independently. 90% of the system was designed to be Administrator controlled, hence a loosely coupled service-based architecture was the need.

Our Approach

Our Approach

We saved time by reducing a large upfront effort on mockup or wireframe by sharing an outline of the UX using PowerPoint and engaged with the customer in designing the system flow. This provided more touchpoints with the customer during the design stage, helping them visualize their system before it was built.

Agile Development

Before development started, the development team was fully immersed in the requirements document, which gave them the roadmap for the entire system. We adopted an agile development approach, with sprint cycles defined well in advance with tasks to be achieved. Each sprint consisted of 10 days. The building principle was to develop fast, fail fast, and recover fast. For each sprint cycle, we dedicated 75% of the time for development, 15% for integration testing, and 10% for issue fixing before turning it over to the users as a beta release. The development team was engaged in daily scrum calls, beyond other technical brainstorming sessions to reduce understanding gaps. Communication helped in reducing friction between modules built by multiple developers to ensure smoother integration. This ensured that each sprint cycle was defect-free and the users were asked to just validate the system. We presumed that about 20% of the requirements would change as we were building the system. Our agile method and our modular design approach gave us the flexibility to accommodate these changes without introducing defects.

Our Approach

Picture

Customer Insights

We kept engaging with the customer closely during the construction stage to exchange thoughts and demonstrate the continuous progress of the product. Wherever certain things were either technically not feasible or contradicted good design principles, we logically explained the reasons and adjusted our design accordingly. The customer was assured of the best design principles being adopted at every stage of the build. Using agile development principles helped in including our customer’s thoughts and adopting late changes in the product lifecycle easily. Including customer insights at key stages of the development helped us reach a high level of customer satisfaction at the end.

Analyze and Remediate

A key component towards a zero-defect journey was to predict defects before they arose. Our vast experience in custom development helped us to conceive a checklist of potential defects from a typical web development process. Developers treated the checkpoints as part of their development process, thus strengthening the unit testing practice. Quality software checks included performance checks and negative validation. Fixed issues which impacted core design were repeatedly checked after every sprint to ensure the particular functional area was not impacted due to other changes.

It is Not UAT Anymore, but UVA

Most custom software falls into the trap of a User Acceptance Test (UAT) stage. We did not want the user to test; we wanted them to validate the function. We started by calling the stage UVA – User Validation and Acceptance. This name change shifted the mindset of the developers and users. The users came into this stage with the mindset that they were being provided a functioning system and they just had to validate that the functions existed.

Yes, Zero-Defect Code is Possible

Custom software development often scares organizations. They are unsure of what needs to be built, how it should be built, and whether they will get what they want.  Our journey proved that with the right approach, we can build software that will deliver with zero defects. Our journey is repeatable. It is just a matter of discipline.

Building an Organizational Culture That Supports DevOps 

DevOps is at its core is a set of best practices for developers and IT operations that aims to shorten release cycles and increase software quality. As such, it is closely related to other development methodologies such as Agile Software Development, CI/CD (Continuous Integration/Continuous Deployment), and Kaizen.

As DevOps has matured, several toolsets have emerged to support the application development lifecycle, providing version control, reporting, requirements management, project management, automated builds, testing, and release management capabilities. The best tools in the world, however, will not ensure success if the development and IT organizations are not committed to instilling the culture necessary to support it.

Click here to read another article about how a strong DevOps culture at Vertex achieved the development paragon of zero defects code.

Our top five practices with the strongest correlation to organizational culture are:

  1. Organizational investment
  2. Team leaders’ experience and effectiveness
  3. Continuous delivery
  4. Organizational performance
  5. Lean management practices

Organizational Investment

DevOps success begins at the top. IT and development managers need to break down the silos between development, testing, deployment, and operations to create a cross-functional, unified team capable of working across the entire development lifecycle. While technology stacks and tooling aid in the quick and reliable development of applications, it is really the shared responsibility and ownership that increases the team’s velocity.

Experience and Effectiveness

Leaders of DevOps teams need to be experienced across the spectrum of IT disciplines in order to break down potential barriers and ensure that their teams are performing at the highest potential.

As a DevOps engineer at IBM said in an article published by InfoWorld, “We needed to answer some basic questions and determine the problems we were trying to solve…If you don’t know how the work is actually done, you don’t know which problems are worth solving.”

Continuous Delivery

Continuous delivery is the DevOps approach that aims to produce software in short cycles. By developing in short cycles, teams can reliably release their software at any time. With CD, development teams can build, test, and release software faster and more frequently. As a result, they can reduce the cost, time, and risk of delivering each change. A repeatable deployment process is important for continuous delivery. Rather than throwing code ‘over the wall’ to QA and Operations, integrated teams can achieve heretofore unexpected release cycles.

Organizational Performance

The transformation to a DevOps organization takes a large investment in time, money, and resources. The ability to clearly and accurately measure the performance of your DevOps team is critical to improving efficiency and tracking success.

The following metrics are a great start for measuring DevOps performance and progress:

  • Lead Time – The time it takes to implement, test, and deliver code
  • Deployment Frequency – The number of software deployments over a period of time.
  • Change Failure Rate – While shortening and increasing the frequency of release cycles is the goal of DevOps, management should also look at failure rates.
  • Mean Time to Recovery (MTTR) – How long it takes the organization to recover from a failure and release a fix.
  • Customer Ticket Volume – This is ultimately a measure of end-user satisfaction. Bugs and errors that are missed in QA are often detected by the end-user. The number of customer tickets marked as problems or bugs is an important indicator of application reliability.

Lean Management Practices

A recent article in Communications of the ACM stated that “DevOps is about good development practices that continually deliver product features (Agile) effectively with minimal wasted efforts (Lean) which are overseen by good governance controls (Information Technology Service Management, or ITSM). To that end, a growing consensus within the information technology community is that DevOps = Agile + Lean + ITSM.”

Contrary to popular belief, while DevOps may eliminate the traditional ‘waterfall’ method of project management, it does not brush aside Agile/Scrum either. Nor does it do away with traditional IT organizations rather than force a transformational shift.

Vertex Computer Systems Has The Practices AND The Tools

Look to the DevOps experts at Vertex to guide your way to the efficiencies of DevOps and to help you choose the right tools to create repeatable, clean releases. We Got IT.

How The Internet of Things (IoT) is Transforming the Way We Do Business

In just a few short years, small internet-enabled devices, known collectively as the Internet of Things (IoT), have transformed how we live in ways that continue to grow and evolve.

The IoT revolution really began with consumer-facing products – smartwatches, thermostats, and television remotes. While today your IoT-enabled refrigerator might tell you that you need to get more milk or your smart garage door opener may send you a text asking if you meant to leave the door open, the IoT is now set to transform and revolutionize business processes and operations.

While the roots of Industry 4.0 started in the 1990s, it is the rise of interconnected systems communicating via the internet that is truly leading to a revolution in manufacturing. Not only can machines on factory floors communicate more intelligently with each other across physical and geographical barriers, but these smart machines are also able to monitor, detect and predict faults, suggesting preventive measures and remedial action before downtime occurs.

The IoT also allows manufacturing processes to be completely virtually visualized, monitored, and managed from remote locations. Industry 4.0 puts machines, people, processes, and infrastructure into a single, connected manufacturing process, which provides businesses with full disclosure over the entire workings of their manufacturing and production. Using this information makes overall management highly efficient.

Supply Chain Management

Just one way that the IoT will create efficiencies for the world of business, be it B2B or B2C, is in supply chain management.

From factory to shelf, the IoT cannot only make existing processes more efficient, but it can also detect potentially expensive problems in advance of them impacting your business. Picture a small component in a factory. As a ‘dumb’ cog in the machinery, the part could fail without notice, shutting down an assembly line for several days. As a ‘smart’ cog, a similar part could tell engineers that it was set to fail days in advance, automatically place an order for its replacement, and direct staff to its location in the factory to replace it.

The same technology makes it easier for businesses to track inventory around the world, creating opportunities for supply chain and logistics optimization. Connected manufacturing and the IoT provide employees with visibility over company assets worldwide. Standard asset management tasks such as transfers, disposals, and adjustments can be streamlined and managed centrally and in real-time.

Similarly, the IoT has been used to handle logistics for the maintenance of a shipper’s fleet. With equipment requiring regularly scheduled maintenance, knowing where a given truck is on a given day, scheduling it for service, and making sure that there are enough trucks in the right place to cover customer needs. Easier scheduling, reduced downtime, and balanced fleet usage all translate to savings.

Asset Tracking & Waste Reduction

IoT sensors on both vehicles and product packaging can provide insight into where a company’s inventory is at any given time. When a company always knows exactly where their inventory is, they can be agile with moving it to where the need is the greatest. Furthermore, the same sensors are able to detect changes in temperature, light, and other environmental factors, ensuring that potentially perishable items do not go to waste.

With real-time insight into buyer behavior, retailers can stay up to date not only with on-shelf product stocking, but also by tracking which goods are most popular in a given setting, allowing them to increase their profits with efficient sales and stock management.

Advanced Workforce

Frontline workers need access to accurate and up-to-date information in order to solve problems and increase productivity. Access to information, guidance, training, and support which was previously delivered in person can now be delivered directly to the shop floor.

Manufacturers and industrial companies of every size can now access digital transformation initiatives in order to maintain business operations and business continuity. Technologies such as artificial intelligence (AI) and augmented reality (AR) can now be a part of the toolkit available to workers on the shop floor and are crucial to making the most of a workforce that is increasingly spread across the world.

Advanced Analytics & AI

With information coming in from sources as disparate IoT sensors on factory floors, lighting systems, sales data, supply chain, and customer demand, there are massive amounts of data to sift through. Thanks to advances in data and analytics, manufacturers are more capable than ever of using that data to make informed decisions to improve internal processes.

Taking it to the next level, Artificial Intelligence (AI) and machine learning can be leveraged to further process your data, using it to reach conclusions that go beyond the obvious. Some examples include forecasting market changes and predicting machine downtime.

Customer Engagement and Real-Time Insights

Stepping outside of the shop floor, as the IoT and its associated cloud platforms become more prevalent, companies will be able to gain deeper insight into how their products are being used, potentially leading to new marketing channels, improved customer service, product improvements, and ultimately, increased customer satisfaction.

Transform Your Business

As the IoT landscape continues to evolve, market research from Gartner and Cisco predicts that the Industrial Internet of Things (IIoT) will grow even larger, improving asset management, operational visibility, safety, and security. With all of these interconnected devices, the amount of data grows exponentially. By partnering with a technology company experienced in supply chain management, big data, machine learning, and data visualization, you can transform your business for the 21st Century.

loader
Vertex Computer Systems is Hiring!Join the Team »
+