Developer’s insights on Shopware bulk synchronization

Care to share?

When building your eCommerce on Shopware, you usually deal with a huge number of integrations between various external systems. You might have to connect to enterprise resource planning (ERP), master data management (MDM), and many other systems containing data on product prices, stock levels, and other data crucial for any eCommerce solution.

In this article, I would like to share a few insights with you about one-way integration between Shopware and product information management (PIM) class software. Here’s a couple of issues you have to focus on to keep your eCommerce stable, reliable, and efficient. 

Common data in PIM integrations

In each of your integrations, a similar set of data must be migrated and fed to the eCommerce system in real time or once during project initialization. It’s crucial to get it right.

The data that you always need to synchronize are categories, products, photos, additional assets, and product attributes. This is a standard set of data that your PIM system must share with the eCommerce platform.

Admin API

The implementation of synchronization in Shopware will be significantly facilitated by the available API. The current Admin API documentation  could be improved to describe all the possibilities, but it’s enough for us to integrate with the PIM system. You should definitely check it out before going further.

Bulk synchronization

When browsing the Shopware documentation, you have certainly come across the endpoint /api/_action/sync and other endpoints for each entity separately. For us, /api/_action/sync is particularly noteworthy because it’s a dedicated endpoint for this type of task.

It allows you to send large data sets to Shopware for each entity. It works in two modes. They are UPSERT and DELETE. Of course, UPSERT is responsible for inserting or updating an entity, and DELETE is for removing it. It’s also worth paying attention to the indexing-behavior parameter that allows you to control how data is indexed after sending.

Shopware, depending on its infrastructure, can consume large amounts of data, but you must be careful not to overload it because it will destabilize the system. This means that the amount of data must be selected appropriately for the available resources.

Categories and products

As I mentioned at the beginning, in most of the projects, you always have to synchronize categories and products. You can solve this problem by using the sync API.

It’s important to adopt a strategy to generate object IDs in Shopware because this will serve as a reference for creating relationships in the future. Let's assume it will be the md5 hash function of the object ID from our PIM system, as proposed in the documentation. You can also use another way to control reference to the object from Shopware by saving an ID of the object that will be returned to you from API after creation. This solution requires some additional implementation on the PIM side.

Now that you know what to pay attention to, you must map the available entities of the product category and the product itself and send it in JSON format. It’s good to base such operations on asynchronous solutions and perform them in the background, like for example, by using a queuing system such as RabbitMQ.

Product properties and variant configurations

One of the biggest challenges you may face in integrating our systems will probably be the synchronization of attributes that are part of product properties or are used as the configuration for variants. The challenge may be the amount of data of this type, the units that are part of them, and the method of their subsequent processing within the product. On the Shopware side, all attributes are stored in the ProductGroupProperty entity, and it’s nothing more than a data dictionary.

Product properties 

The ProductProperty entity is used to assign product characteristics. If you want to use values from this dictionary, you can use an identifier that can be built similarly as for category and product objects. This is the result of the hash function for the attribute key ID and value. You pass it in a specific field, which in this case, is in the properties field.

Variant configuration settings

When you want to define what variants a given product has, you also have to use the ID of the specific value you want to use and pass it in the configurationSettings field.

The challenge that you may encounter when synchronizing attributes is data consistency between the PIM system and Shopware. If one of the PIM users wants to edit product features, you’ll have to add logic in our application that will remove the old value and add a new one.

It creates additional overhead on the integration side and may generate problems in the future if our solution allows for data inconsistencies. For example, a delete request might fail, but a insert request a new value does work. With this approach, it’s worth queuing such operations and repeating those that were not successful, and if it fails, note such a state in the logs to be able to monitor the status on an ongoing basis.

Takeaways

Shopware and bulk payload support via API is a perfect mechanism that makes it much easier to integrate it with other systems. The API documentation provides essential insight into how to use bulk updates. 

It’s definitely worth getting interested in object versioning using the API. Unfortunately, you won’t find anything about it in the documentation. The libraries available on GitHub for PHP allow you to use the API easily. All you need to do is configure the access data correctly.

Of course, the method of implementation will vary depending on the project and business requirements. This does not change the fact that the Shopware API gives us a full range of work with objects that significantly facilitate data migration.

Because of this, it can be safely assumed that it’s a platform that is adapted to carry out migration and integrate with other systems.

Published May 14, 2023