Setting up a data warehousing solution for High Voltage Analytics using Amazon Redshift

Who are High Voltage Analytics?

High Voltage Analytics is a technology consultancy operating in the e-Commerce space. They provide the necessary resources so their clients can fully leverage the latest technology to scale successfully and build quality infrastructure using the modern industry techniques and best practices.

Why Lambert Labs?

High Voltage Analytics decided to collaborate with Lambert Labs because of our skills and experience using, developing and architecting extract, transform, load (ETL) solutions for business intelligence (BI) workloads.

High Voltage Analytics chose us to build an end-to-end BI prototype for their client, a major player in the E-Scooters industry, because of our ability to quickly understand the problem and outline appropriate solutions given the project’s objectives and time/budget constraints. Crucial to this was our understanding of key trade-offs, e.g. using custom vs off the shelf solutions.

What we did – technical implementation

  • Setup a Stitch integration to copy data from multiple Shopify accounts into Amazon Redshift on a regular schedule.
  • Wrote custom SQL to generate business-critical KPIs and metrics for each Shopify account as specified by the end client.
  • Wrote custom SQL to aggregate data at the account level and produce KPIs and metrics for the entire business.
  • Integrated the resulting data with a frontend BI visualisation tool, Grow BI, so that end users could view and analyse the data via the web.
  • Identified the limitations of the prototype and the ways that these could be addressed in the future: greater automation, CI/CD, templated data transformations, integration with AWS services.

What we did – non-technical implementation

  • Organised regular calls for progress updates.
  • Organised calls with external vendors ensuring participation and feedback from relevant stakeholders.
  • Produced documentation giving a high level overview of the prototype architecture.
  • Produced marketing / sales material for future projects.