.svg "logo-sketch-development"){kind=logo}
Because this client was still new to AWS Cloud and serverless technologies, they needed a partner who could walk alongside them and lead by example. Training alone wasn’t going to be enough, especially because this client had low risk tolerance and limited learning time. The engagement would have to involve a high degree of collaboration and co-development.
The massive scope of this client’s operations created a second challenge. Because the client was processing large amounts of data, run times were becoming unacceptably long. The client needed a way to reduce or eliminate these problematic delays.
With the team sufficiently upskilled, we did the following to equip them for success by improving the tools at their disposal:
The remaining effort largely came down to providing leadership in the form of training and mentoring. We helped the client establish best practices and adopt new techniques. Everyone on the team became comfortable with what Sketch had delivered and co-developed with them. Video training resources ensure future developers can come up to speed quickly, too.
When it was all said and done, the client had 12 new systems at their disposal. This included various APIs, exporters, and loaders. Sketch co-created and delivered training sessions for each of these new systems alongside the client to drive adoption and implement best practices.
One of the systems, a cloud load platform, managed to use more than 5,000 concurrent lambdas to process 4.3TB of data (12.5 billion records) into parquet in only 43 minutes. This was the largest run the client had ever attempted. It cost about $7,700 and 12+ hours of cluster processing to generate projections using their 3rd party software, but this new parquet conversion process using the serverless technology transformed this data for a cost of only $7.69 and in a fraction of the time.
The creation of the Database load system was extremely important for the company to report its rapidly growing financial projections. This was delivered in a very short period of time with a small team. There was no room for error, and the timeline was critical. All other alternative options tried by the client in parallel had failed to pan out. The resulting system was delivered on time. It scaled extremely well and worked flawlessly, loading the database at the highest rate it could handle without overloading.
A common factor across these systems is how resilient they are and how they just do their job, day in and day out without needing much attention. Using good architectural design and strong functional development practices, these systems just work.
In the time leading up to this engagement, the client had struggled to develop systems that met its advanced needs. Because Sketch was able to deliver results that met the client’s lofty standards, the partnership continued for multiple years.
We hope these are the kinds of outcomes you were looking for. If you want to learn more about the work that went into this project, we're happy to answer your questions.
Better yet, tell us about your goals for your own cloud environment. We love to talk shop, and the first few conversations are always free.
