Building an e-commerce omnichannel inventory solution with open-source

After an hour of online research, Rachel bought the perfect raincoat for her upcoming trip in Southeast Asia – and then discovered that it didn’t exist. Unfortunately, when she tried to add it to her cart, she was informed that the raincoat was actually out of stock, and could not be purchased. This made her frustrated: why, she wondered, couldn’t the site have told her that the raincoat wasn’t in stock when she first saw it?

Consumers like Rachel want to see accurate levels of inventory when they’re shopping online, so they can avoid these kinds of frustrations. The earlier the customer sees the correct inventory during the browsing/ordering process, the better. If users are able to see the correct inventory levels, they won’t buy items that are out of stock, and thus won’t waste their time. Unfortunately, Rachel’s pain is a frequent problem for online consumers. What is stopping retailers from displaying inventory levels in real-time on an e-commerce site?

The actual level of inventory for retailers is found on the fulfillment system, which is the system of record. Retailers want this fulfillment system to be presented to customers, as it has the accurate inventory levels available for purchase. However, fulfillment systems were created for inventory managers on the backend only, not for customers. As traffic has continued to rise on retail websites, these systems have become overloaded by the amount of users, causing them to crash. These crashes and the slowness of the platform means that it is very difficult to provide accurate inventory to users. The dilemma of retailers is that neither e-commerce nor inventory systems provide a workable solution.

One of our customers was having similar problems with their inventory system. Their inventory system would crash multiple times on high traffic days, causing lots of revenue to be lost. ATG, the client’s customer experience platform, could not provide a real-time inventory solution to this problem. Therefore, the customer wanted to build a custom solution that could display accurate inventory levels to shoppers. Due to our extensive experience with the retail industry and with replatforming, Grid Dynamics was a natural choice to implement this solution.

In order to fix the client’s inventory troubles, we first evaluated their legacy system and its technical challenges. The customer used IBM Sterling for inventory and Oracle ATG for its digital customer experience platform. There were several problems with these systems. The first was that real-time inventory was not possible due to the lag in updates from Sterling to ATG. Second, by serving more users than originally planned due to increases in online traffic, the systems were prone to crashing. These crashes were catastrophic, as users could not buy anything while the site was down. Finally, the systems had low performance, which led to issues with processing requests during peak times. All these problems resulted in lost revenue and user dissatisfaction. Therefore, our initial goal was to “stop the bleeding” by improving the customer experience platform.

Phase one of the project was to limit overbuying and underselling with a stable inventory messaging solution that had shorter batch periods. We did this by building a cache for inventory updates so that the website had inventory levels that were close to real-time. The architecture of the system looks like this:

To provide accurate inventory levels to consumers, we used in-memory data grids (such as Apache Ignite or Hazelcast) as caches for the data. These transferred the accurate inventory information (provided by Kafka) into the customer experience database, which in this case was Cassandra. This meant that the system of record didn’t have to send information to the customer experience database directly. Online shoppers therefore got inventory levels that were closer to real-time. By using in-memory data grids and a more scalable database, the customer experience platform had more stability and was less likely to crash than its predecessor.

However, this implementation was just phase one, not the real-time inventory solution the customer needed. Because the message queue provided updates in real-time, while the system of record sent them in batches, there were still lags in synchronizing the inventory. When these lags occurred, the inventory levels shown to users were inaccurate, and items were oversold once more. Additionally, the fulfillment platform itself was unchanged. It remained unscalable, slow-performing, and unable to provide clients with an omnichannel inventory management experience. Thus, more substantial changes were needed: replatforming off the legacy systems entirely.

Our ultimate goal was to create an omnichannel, real-time inventory management service. This would be an inventory system that could handle the entire process (ordering, managing, and presenting to customers) on one platform. The reference architecture is shown here:

This solution removes the issue of synchronizing the fulfillment platform and customer experience platform by combining them on the same system. The omnichannel inventory service then sends information to a data grid such as Ignite or Hazelcast, which passes it onto a scalable database built on the cloud (Cassandra or mongoDB). By connecting the system of record directly to all three clients, real-time inventory is available from the start of the online product discovery process. Here are some of the functional requirements of the complete system:

• Real-time sourcing responses for digital users
• Optimal solution for shipment cost and duration
• Multiple carriers
• Multiple shipment options
• Custom sourcing rules

This modern, omnichannel platform is the real solution to inventory issues. The retailer has a scalable, sturdy, easy-to-update system, and no longer has to pay license fees to rent server space. The customers receive real-time inventory levels during the entirety of the shopping process, which reduces their frustration substantially. Everything is handled under one umbrella, simplifying the process immensely. Everyone wins.

Inventory management issues are common for retailers, and these frustrations are getting worse as mobile traffic increases. Fortunately, we have devised a method to combat these problems. By building a cache, we provided close to real-time inventory for our client’s online users. With the user experience improved, we could then focus on fixing the more complicated backend issues by building an omnichannel inventory services platform. The end result is a system that can provide positive user experiences for now, and has scalability for the foreseeable future.

This project and similar ones have given us much experience in working on inventory systems. If you’re a retailer that is having inventory management issues, give us a call!