In my previous post, I talked about how I was looking into actually implementing Visualisations as a Service. I didn't really run into technical issues, but I did potentially uncover some cost-related ones. In this post, I'm going to describe my money-no-object architecture.

The obvious way to get around the video generation problem is to throw compute at it. So, a cluster of beefy EC2 instances would be required.

On these, I would run a Windows Service wrapper around my visualisation executable. When the service was idle, it should poll an SQS queue. The cluster could even autoscale on demand.

Nice. We have a mechanism to generate and zip video files. This is the core problem solved. Now, how do we make the service available?

Quick aside, instead of zipping the AVIs, there may be a possibility of using AWS' Amazon Elastic Transcoder service.

Update: maybe not. It seems AVI as a source isn't supported.

To make the service available, I think I'd expose an API Gateway REST API with an endpoint that accepts a puzzle id and input file. It would save the input file to S3 and pop a message on the SQS queue to notify an EC2 instance that there is something to be processed.

Once the EC2 instance has generated the video and zipped or transcoded it, it can write the result to S3. This object could have an expiration, so it is not hanging around in the bucket for too long.

Notifying the user that the visualisation is complete? Assuming we're avoiding taking their email address, the best thing is probably to store some state somewhere such as Dynamo DB. The API could expose a status endpoint that will check Dynamo and let the user know when their file is ready for download. They can then check back at their leisure.

All good so far. But users don't want to interact with an API! True, but once you have an API, slapping a UI on the front of it isn't hard.

In fact, though I've never done it, I assume you could probably host the web front-end using API Gateway also. It's just HTTP requests and responses ultimately...

Anyway, that's all I've got for today, go away now.