Leveraging Global Trade Data-as-a-Service: Part One
on May 10, 2018
In most organizations, different types of trade data tend to be siloed in their own separate systems, only available for separate, narrow use cases. Trade DaaS allows trade data to be shared, delivered across the enterprise, directly to the various systems used daily by various functions across the company, right at the point they are making important relevant decisions.
Full Article Below -
( This article is excerpted from the complimentary report:
Using Global Trade Data-as-a-Service for Competitive Advantage, available for download here. )
The Age of Mega-Fines
Since around the turn of this century, we have entered the age of mega-fines for trade violations:
$24M is the average size of fines issued by OFAC since 2009.
A $964M fine was issued in 2014 to a company that processed transactions involving countries and entities subject to sanctions.
A $1.19B fine was issued in 2017 for illegal shipments of telecommunications equipment to Iran and North Korea in violation of the Export Administration Regulations (EAR) and the Iranian Transactions and Sanctions Regulations (ITSR). This was the largest civil penalty ever levied in a Commerce Export Control case.
Fines for misclassifying imported items can be millions or tens of millions of dollars as well.
What is Trade Data-as-a-Service?
Data-as-a-Service (DaaS) unbundles and decouples data from a specific application or underlying platform and delivers it as a service. In this paper, we are specifically referring to Trade DaaS—i.e. delivering trade-related data as a service, rather than keeping it ‘locked up’ within one specific application.
The preparation and delivery of trade data is decoupled from the applications using it. Delivery is accomplished via some combination of APIs, web services, and/or file downloads in formats to be consumed by various applications.
The data is much more up-to-date and dynamic compared with buying a list that is updated on a periodic (e.g. monthly or quarterly) basis.
A large team of people at the service provider continuously prepares and updates the data; extracting, cleansing, normalizing, doing QA, refreshing the trade data, and delivering it in easy-to-consume formats.
Trade DaaS should include not just the data, but professional services needed to successfully integrate the data into key enterprise applications.
Leveraging Global Trade Data-as-a-Service
From Compliance to Competitive Advantage
Ensuring compliance with trade regulations has become a significant expense for many companies that import and export goods. Trade regulations have mushroomed in response to terrorism and security concerns, the proliferation of trade agreements, protectionist actions and responses to them, and legal actions to settle trade matters. At the same time, we’ve seen more robust enforcement and the more frequent issuing of mega-fines (often tens to hundreds of millions of dollars), forcing companies to get serious about and invest in trade compliance.
Ensuring compliance requires access to a wide variety of trade data contents such as: duties and tariffs with the associated harmonization codes for countries across the world; an increasing variety of denied parties’ lists; trade data on imports and exports by country pair (from/to), commodity, and buyer/seller; and results of various rule making and legal cases. Many if not most companies strive for ‘compliance at the lowest cost’; minimizing the time, energy, and money spent dealing with trade data and contents. Far fewer appreciate how trade data and content can be a source of competitive advantage, well beyond compliance, while simultaneously lowering compliance costs and risks. Leveraging global trade data, delivered as a service (i.e. ‘Trade Data-as-a-Service’), is a key element for realizing these advantages. With Trade DaaS (see sidebar) global trade data can be leveraged for many potential uses, such as:
ERP/GTM Integration—Bringing trade data directly into ERP Global Trade Management workflows.
Supply Chain Risk Management—Preventing supply chain disruptions due to non-compliance or unknowing use of a restricted or debarred supplier. Diversifying your supply base by discovering new sources of supply.
Supplier Management—Supplier discovery; learning which suppliers others are using.
Sourcing Optimization—Clear understanding of total landed cost and the options for minimizing it.
Ecommerce—Real-time duties estimation, denied party screening.
Lead Gen, CRM Integration, and KYC—Finding new leads, auto-screening of customers and orders.
Competitive Intelligence—Seeing who competitors are selling to and the products they are selling, size of various markets.
Bringing Trade Data to the Point of Decision
In most organizations, each type of trade content tends to be siloed in a single system and only used for a narrow set of uses. This makes it difficult to achieve many of the higher value use cases we’re discussing. Trade DaaS allows the data to be delivered directly into the various systems that are used on a daily basis by the different functions across the company. It brings the right data to the right user right at the right time; at the point they are making an important relevant decision.
Figure 1 - Trade DaaS Brings Trade Data into Everyday Workflows and Systems at the Point of Decision
For example, trade data can be incorporated directly into the systems used by a sourcing professional to find and vet suppliers, invite suppliers to bid, evaluate bids and RFQ responses, and place purchase orders. The latest complete duties and tariffs can be incorporated into total cost comparisons and duty optimization processes. Denied and restricted parties can be flagged at the earliest decision point, rather than after the fact, saving effort and issues. New alternate sources can be discovered using trade data. These kinds of capabilities can be interwoven right into the day-to-day workflows and systems used by sourcing and procurement managers. Similarly, using Trade DaaS, trade data can be brought into the appropriate workflows and systems for ecommerce, HR/onboarding, sales and marketing, and service.
Plethora of Agencies Regulating Trade
The US alone has scores of legislative acts, hundreds of rules regulating trade, enforced by more than 20 agencies including:
Bilateral Trade Affairs
Bureau of Industry and Security
Committee on Foreign Investment in the US
Customs and Border Protection
Department of the Treasury International Office
Directorate of Defense Trade Controls
EPA Climate Change
EPA International Programs
FDA Office of International Programs
Foreign Agricultural Service
Intellectual Property Enforcement
International Trade Administration
Multilateral Trade and Agricultural Affairs
Office of Foreign Asset Controls
Overseas Private Investment Corporation
Policy and International Affairs
SBA International Trade
Trade Adjustment Assistance
U.S. Commercial Service
U.S. International Trade Commission
U.S. Trade and Development Agency
Enabling Cross-functional Collaboration
Often the functional responsibilities of people using that data are siloed and influenced by which organization they report into. A trade compliance manager who is part of legal or risk management, tends to focus mostly on legal compliance and penalty avoidance. If part of the supply chain group, their focus is on customs clearance and ensuring the uninterrupted flow of goods. If part of the financial organization or sourcing, they might focus on reducing duties and tariffs. In fact, things work best when the different functional areas collaborate together. Bringing a common, shared set of trade data directly into the systems that each of these groups uses can help facilitate that collaboration.
Challenges of a Do-it-yourself Approach
The vast majority of trade content is available directly from various government agencies. Larger businesses may try and go directly to these sources for the trade data they need, but they usually vastly underestimate the effort involved. Good Trade Data-as-a-Service providers can have a team of 50-100 people or more dedicated full time just to collecting and preparing trade content. There are many major challenges to trying to do this task without the help of an experienced Trade DaaS provider:
Volume and Variety of Data Sources—There are over 190 countries in the world, each with their own interests, circumstances, and agencies resulting in a mind-boggling array of country-specific regulations, tariffs and duties, denied parties lists, security schemes, classifications, rules of origin, court rulings (international and domestic), and other dimensions of trade compliance and the content necessary to achieve it.
Denied Parties Lists—There are hundreds of lists of denied, restricted, debarred, and sanctioned parties around the world, published on over 500 websites. Descartes’ collection of Denied and Restricted Parties Lists gives some sense of the scale. In aggregate, these lists typically contain hundreds of thousands of entities and it keeps increasing.
Permutations—Many trade rules involve complicated permutations such as the Export Administration Act, where exports are restricted by item, country, and recipient; and for dual use items you also need to factor in the intended use.
Pace of Change—An ever-changing landscape of bi-lateral and multi-lateral agreements, rulings, actions, updates to lists, duties, tariffs. A single global company can see hundreds of changes annually to tariffs and duties including whole new categories and rulings on classification. Reliance on government online systems, such as CROSS or EBTI,1 may not adequately demonstrate reasonable care.
Manual Searching or Screening—Manual methods can be extremely labor intensive, requiring looking at several government websites per shipment. Denied parties searches can yield false positives (loss of business) or false negatives (potential fines). Raw trade data is difficult to read or analyze.
Figure 2 - Example Dump of Raw US Trade Data
Cleaning, Formatting, Integrating, Enriching the Data—Bulk downloading of data is also problematic. You will need to file a FOIA request. There are non-trivial fees. Government data often contains errors. Cleaning up this data is a full-time job, requiring special analytics and years of experience to really get it right.
Integration Challenges—Integrating trade data into enterprise systems entails tricky mapping of fields, e.g. integrating denied and restricted party screening into subcontractor onboarding and payment processes. Cleaning and formatting trade data must be done continually for real-time and batch feeds.
Industry-specific Challenges—In addition, specific industries face specific challenges, such as:
Aerospace and Defense requires feature-based classifications which can be challenging. They also must consider their customer’s end-use and obtain and manage various export licenses. Value Added Tax (VAT) relief options can save substantial money, but adds complexity to classification and shipment processes. Alloys based on rare earth elements often have multiple rulings, explanatory notes, environmental policies or quotas that need to be heeded.
Medical device manufacturers deal with dozens of agencies and agency-specific documentation, such as sterilization records and clinical trial records. Export regulations differ between countries. Duties fluctuate. There are industry-specific excise taxes and exemptions.2 The correct HS code or FDA 510K must be used to execute a compliant recall.
Pharmaceutical and Biotech—60% of countries levy tariffs for Active Pharmaceutical Ingredients (APIs). For biotech companies, samples and items, e.g. laboratory-cultured cells, are subject to various agency regulation3 or require extensive permitting.
Chemical companies may rely on specialist consultants for classification of new products.
Industry-specific government agencies (such as the FDA, Dept. of Agriculture, and DoE) place additional demands on imports and exports. Some trade-related agencies, such as the Customs & Border Protection (CBP), are opening industry-specific sub-departments.
Import/Export Data Loading, Cleansing, Enriching
Here are some of the steps Descartes does to prepare import/export data for consumption:
Raw data loaded, checked, all duplicates deleted
Format the data and extract information including (from ISO Code) container #, size, type (dry, reefer, flat rack); cargo type, container vs. bulk
Perform master data management on consignee data to ensure that different names for the same company (i.e. different spellings, parent-child company) are represented once in the database
Calculate TEU with adaptive algorithm based on declaration, historical list of containers with lengths or estimated average length
Combine and compare data from House BOL vs. Master BOL to achieve more accuracy and detail
Find and apply all modifications (ups or downs) to already loaded bills of lading
Quality control and consistency check; compare with historical data to ensure correct loading
Enrich with value-add data and analysis including:
Port pair analysis
Sales territory construction and analysis
User-friendly representation of each BoL
50 other Customs data fields for Latin American BoLs
Searches by State, City or zip codes
Search by entire bill (descriptions and mark and numbers)
Searches by dry, refrigerated, container size
Unified Company names (D&B)
Calculate TEUs by ISO mode
You can outsource trade compliance tasks, but not the legal responsibility or financial and reputational consequences. It is often wisest (and most cost effective) to use a Trade DaaS provider, rather than do it yourself.
What a Good Trade DaaS Provider
Does for You
To get a better feel for the work behind the scenes that goes into delivery trade content, consider what Descartes does to collect, clean up, enrich and deliver trade data across their various services:
Monitors over 550 URLs at least eight times per day for changes to denied parties lists. For non- monitorable websites, they subscribe to newsletters which they scan manually for updates.
Receives import-export declarations from over 160 countries into their database (which already has about five billion records). Import data is downloaded daily, export data weekly, with rigorous loading, cleansing, and enriching (see sidebar).
This level of data diligence, coupled with well-documented enforced processes, demonstrates reasonable care, provides consistent results, and reduces costly errors and delays. Even more value can be realized via applications of Trade DaaS within various processes and existing systems.