Case Study

ScorpionTrack Fleet

ScorpionTrack Fleet Management System

The ScorpionTrack Fleet system is a complex feature rich product that provides thousands of customers with real time vehicle tracking and fleet management services. Millions of rows of data streams into the system everyday with over 8 billion rows of data in total.

Problem:
Scorpion became a victim to its own success and the system started to creak under the enormous pressure of new vehicles being added on a daily basis. It’s initial MVP had served its purpose of finding a saleable market fit for such a product but there were daily outages that needed to be identified and rectified.
Technology:
Agile, QA, Devops, Java, Gradle, Pub Sub, Kafka, Hikari, Golang, PHP, Codeigniter, Lumen, ELK, Kibana, Google Cloud, Rackspace, Serverless, Continuous Integrations, Gitlab Pipelines, Datadog, Big Data, Data Warehousing, Redis, Docker, Grafana, Percona PMM, Firebase, Protocol Buffers
Our Solution:
Microservice architecture redesign built on a scalable cloud based infrastructure

We were tasked to overhaul the site with performance in mind to scale on demand and be as efficient and cost effective as possible. With the added requirement of keeping the existing website up whilst integrating new software with old. We also continued to build in new features to ensure the product is still competitive in the market.

Some of the standout issues we overcame where:-

Reporting

Due to the sheer scale of the billions of rows data in the database, the reports were taking an inordinate amount of time to produce. We analysed the bottlenecks within the reports and set about pulling out and rewriting the queries. Smaller reports are now near instantaneous and the larger reports taking a few seconds. Some of these reports even brought down the server.

Contextual Speed

One of the features that was implemented was a Golang microservice that on the fly takes the lat lng coordinates of the vehicle along with its speed and determines whether that user is breaking the speed limit according to their local speed limit. This is then reported back to the fleet manager.

Driver Behaviour

We gave Scorpion two options as a proposal for incorporating user experience with the current website UI.
1). By Vehicle View - Scores applied to vehicles. 2). By Driver - Scores by driver and journey.
The Driver based solution was applied. Using this flow ensures a natural growth of a feature for add ons as it focuses on journeys and drivers. This means moving forwards the driver behaviour can be applied to journey reports and fleet managers and have a much clearer accurate view of driver styles. The solution also utilised the assign driver functionality. It meant that if no drivers were assigned to a vehicle the user could search by vehicle or filter the results to assign journeys to drivers quickly.
The driver behaviour functionality gives customers the ability to:-
- Dismiss-able alerts.
- To filter the results, the user can select start and end dates.
- Viewing the vehicle and driver score by journeys. The user can edit the driver on a journey to ensure the correct journey and score information.
- Driver behaviour incidents - ie it shows the severity of incidents on a particular journey. They are displayed by band, if the band has had no incidents it shows a 0 and be faded.
- Selected driver view. When the journeys have been selected the ‘reassign Driver’ button will be available.

Address Lookups

The system requires millions of address lookups from the Google Maps API for features such as reports and live telemetry which became an issue with the size of the system. The inefficient use of these was causing Google Maps to reject the lookups due to going over the query limit further exacerbating the problem. We set about understanding where the bottlenecks and how the address lookups were used. We found that most lookups didn’t need to be done unless the user specifically requested it by an action so we modified the website to only request an address when needed via an API microservice that manages the requests to Google using queues. This solved the problem with capacity to spare for continued growth.

Migration software

Because of the size of the database and the ongoing stream of millions of rows of data coming into a fixed dedicated server which had no room to grow but couldn’t be taken down for maintenance due to monitoring stolen vehicles, it was a difficult problem to solve. We did this by building bespoke migration software that handled the transfer of data in a managed way.

Tracker Simulator

To test effectively we developed a simulator that could replicate trackers in the wild. This could be spun up with thousands of phantom trackers to load test the infrastructure and imitate peak times such as rush hour.

API

Due to the expansion of the fleet product to cater for custom solutions along with advanced mobile apps an API was developed and retrofitted to the existing system. This required rewriting legacy code to handle the modern API stack.

Systems Monitoring

We implemented a sophisticated monitoring solution that could let us understand what was happening to the system over time periods and if there was any specific bottlenecks. Once an incident took place we then investigated and put in place precautions to prevent it happening again.

iOS & Android Apps

100% web based application can be accessed online 24-7 from any location with our IOS and Android apps. This means no software downloads/installation.

With our mobile Apps ScorpionTrack has full support for hand held tablet devices such as the Apple iPad and Android. Access to real-time information allows you to instantly locate and track vehicles from anywhere, at any time.

The Future

The product is being methodically broken down into microservices to cater for expansion and modular nature to provide the ability to add new products seamlessly.

Testimonial

I worked with Chris and his team closely for a number of years when they became our outsourced digital department and key technology partner. With innovation and technology close to mind, we were frequently faced with the challenging requirements of our customers. From Internet of Things devices, vehicle tracking systems to new systems for improved operations, Chris managed the whole process for us.

One such process we found difficult is relaying digital tasks in a coordinated approach, Chris implemented a project management system to streamline our internal departments and effectively manage our digital problems and expectations.

Chris was also able to act on urgent priorities rapidly and provided 24-hour support for mission-critical devices to ensure minimal impact on our customers. When scoping out new projects for ourselves or our customers, Chris was the first port of call to discuss possible solutions.

I recommend Chris and his team to be able to deliver effective transformational change through digital especially in the face of uncertain requirements.