Deliverlitics

Risk Predictive Model Comparison

An overview of two key predictive models powering Deliverlitics, highlighting their strengths and use cases.

Our Data Enrichment Process

The enrichment layer identifies all the items that were sent for free and linked to previous orders. This normally indicates an issue with an order that the seller decided to deal with by sending an item again, without challenging the claim or issuing a refund. Our current system has some ability to recognize repetition, to avoid misclassifying situations like items sent periodically to reviewers or promoters of the brand

Model Alpha: Address based model
Takes mostly into account historial information geographically grouped around addresses, as well as “risk trends” computed from previous orders.

Inputs

Inputs: Address , Product Category and order

Key Performance Metrics

Accuracy

97.76%

Precision

62%

Recall

99.77%

Fraud level

2.92%

Data used for Training

1.3MM orders

# false positives

1.1%

Interpretability

Medium-High

Decisions are traceable identifying the main factor used in the regression model

Pros

  • Medium-High transparency and explainability.
  • Faster to implement and iterate.
  • Lower computational cost for training.
  • Easier to expan to new historical dataset
  • Can uncover non-obvious risk indicators.

Cons

  • May not capture highly complex, non-linear patterns.
  • Depends on having previous information about the address
  • Less transparent ('black box' nature).

Ideal Use Case

Best for scenarios for those customer that manage the historical delivery information instead of transactional order data.

Model Beta: Order centric model
Based around detecting unusual amount/price patterns in orders in combination with the seasonal “risk trends” we identify from historical data.

Inputs

The input requires price, quantity for a given order and product category

Key Performance Metrics

Accuracy

98.80%

Precision

75%

Recall

99.84%

Fraud level

3.6%

Data used for Training

1.3MM orders

# false positives

0.89%

Interpretability

High

This model is trained explicitally to improve interpretability, providing an explanation for the score assigned to a given order

Pros

  • Superior accuracy in complex scenarios.
  • Adapts to evolving patterns with retraining.
  • Can uncover non-obvious risk indicators.

Cons

  • Requires having an idea of what's the average sale for the store.
  • Sensitive to data quality and volume.
  • Still under development.

Ideal Use Case

Suited for large-scale with limited historical delivery data