Ucommerce
  • Ucommerce Next Gen
    • Getting Started
      • Prerequisites
      • Licensing
      • Ucommerce Templates
      • Headless Template
      • MVC Template
    • Headless
      • Postman Collection
      • Headless API Authentication
        • Token endpoint - Authorization Header
        • Authorization Scopes
        • Refreshing the Access Token
      • Reference
        • Cart
        • Cart / Order Line Items
        • Shipment
        • Billing
        • Promotion Codes
        • Price Groups
        • Payment Methods
        • Countries
        • Shipping Methods
        • Catalogs
        • Cart Custom Properties
        • Line Item Custom Properties
        • Orders
        • Views for Cart modifying operations
      • Custom Headless APIs
      • Error Handling
      • Pagination
      • Deprecation
    • Backoffice Authentication
      • Microsoft Entra ID Example
      • Auth0 Authentication Example
    • Definitions
      • What is a Definition
    • Search and indexing
      • Configuration
      • Indexing
        • Index Definitions
        • Facets
        • Indexing Prices
        • Suggestions
        • Custom Data
      • Searching
    • Payment Providers
      • Stripe Provider Integration
      • Implementing a custom payment provider
    • Data Import
    • Miscellaneous
      • Media
      • Price Group Inheritance
      • Price Group Criteria
      • Soft Deletion Of Entities
      • Logging
      • OpenTelemetry
    • Extensions
      • Extending Pipelines
        • Order Processing Pipelines
        • Checkout Pipelines
      • Changing Service Behavior
        • Images
        • Content
      • Custom Headless APIs
      • Extend the Backoffice
        • Custom UI Components
      • Custom Editor UI
      • Custom Promotion Criteria
      • Custom Price Group Criteria
    • How-To
      • Migrate from Classic
        • Common database issues
      • Entities from code
        • Bootstrapping data on startup
        • Product Definitions & Fields
      • Discover pipelines and their tasks
      • Executing a pipeline
    • Integrations
      • Umbraco Media Delivery API
      • App Slices
        • Product Picker
  • Release Notes
  • Contact Us
Powered by GitBook
On this page
  • What the Data Importer Does
  • Getting Started
  • Registering the Fetcher(s)
  • Setting up the Data Importer
  • Running the Data Importer
  • Community contributed implementations

Was this helpful?

  1. Ucommerce Next Gen

Data Import

The data importer is a great tool for importing data from external sources. It can be used for the initial setup of a project or scheduled synchronization with a third-party application.

Currently, the data importer supports products, prices, price groups, and currencies.

What the Data Importer Does

The data importer inserts data that have not been previously imported into the Ucommerce database. If an entity already exists, it updates the existing entity. The only exception is the currency entity; if a currency with the same ISO code already exists in the database, that entity will not be updated.

Getting Started

To use the data importer, install the Ucommerce.DataImport.Core NuGet package:

dotnet add package Ucommerce.DataImport.Core

After installing the package, implement the following interfaces to fetch data from the external source(s):

  • ICurrencyFetcher

  • IPriceGroupFetcher

  • IPriceFetcher

  • IProductFetcher

Each interface has a method to fetch the data. The method takes batchNumber and batchSize as parameters. Use these to fetch a specific amount of data on each call. For example, if batchNumber is 2 and batchSize is 100, the fetcher should skip the first 200 elements and take the next 100:

var elements = data.Skip(batchNumber * batchSize).Take(batchSize);

Below is an example of how CurrencyFetcher could be implemented by reading from a CSV file:

public class MyCurrencyFetcher : ICurrencyFetcher
{
    public virtual Task<IReadOnlyList<CurrencyData>> GetCurrencyData(int batchNumber, int batchSize)
    {
            var filePath = "currencies.csv";
            using var reader = new StreamReader(filePath);
            using var csv = new CsvReader(reader, config);
            csv.Context.RegisterClassMap(new CurrencyMap());
            var records = csv.GetRecords<CurrencyData>();
            records = records.Skip(batchNumber * batchSize)
                .Take(batchSize);
            return Task.FromResult<IReadOnlyList<CurrencyData>>(records.ToImmutableList());
    }
}

Registering the Fetcher(s)

The fetcher(s) need to be registered in the service collection. It's recommended to create a helper method to register all the fetchers for reusability:

public static class MyPIMExtensions
{
    public static IServiceCollection AddPIMFetchers(this IServiceCollection services)
    {
        services.AddSingleton<ICurrencyFetcher, MyCurrencyFetcher>();
        services.AddSingleton<IProductFetcher, MyProductFetcher>();
        ...
        return services;
    }
}

Setting up the Data Importer

Add the necessary settings to the appsettings.json file (or similar) under the Ucommerce section to set up the data importer. The two key settings are BatchSize (optional - default is 1000), which determines the size of data chunks to be fetched, and ConnectionString (required), which should point to the Ucommerce database the data will be imported into. Here is an example of the app settings for the data importer:

{
  "Ucommerce": {
    "DataImport": {
      "BatchSize": 100,
      "ConnectionString": "YourConnectionString"
    }
  }
}

Running the Data Importer

To run the data importer, inject it where you want to execute it and call the Run method. Here is an example of an API Controller with an endpoint for executing the data importer on demand:

public class DataImportController : Controller
{
    private readonly DataImporter _dataImporter;

    public DataImportController(DataImporter dataImporter)
    {
        _dataImporter = dataImporter;
    }

    public virtual async Task<IActionResult> ImportData()
    {
        await _dataImporter.Run();
        return Ok();
    }
}

The importer should run on a regular schedule, such as a background service within the website, or even better, as a separate console application outside of the website. Especially if the importer need access to the external source of truth.

Community contributed implementations

PreviousImplementing a custom payment providerNextMiscellaneous

Last updated 9 months ago

Was this helpful?

If you have implemented a data importer for Ucommerce that you would like to share with the community .

- A flexible importer for CSV files, just tell it where to look for the files, which column is which, and optionally a few other things, and it will take care of the rest.

please let us know
UcommerceCsvImporter