AWS S3 Storage

In my last entry I discussed the MT43News move from Open Vistsa to a Vultr unmanaged server to a Hosting.com managed server. It was a long journey, but we are now fully on the Hosting.com server and all of our processing is very smooth.  

In addition to MT43News, I have several other websites on the Hosting.com server -- this one in particular. When I moved MT43News to Hosting.com, I already had quite a bit of experience in setting up websites in that environment. But, one of the things I needed to address was the use of AWS S3 storage. Part of the reason we were evicted from the Open Vistas server was storage. I was concerned about the amount of storage available; in particular, I was trying to setup a search indexing for our news articles using python and an AI LLM.  Installing python packages was using up storage.

When the Open Vistas manager evicted us he did setup a new server on Vultr. One of the things he did to help keep the storage usage down was setup backups stored on AWS S3. Using AWS S3 storage was convenient and pretty cheap.  After taking over management of the server, I moved the archived PDF versions of the MT43News issues to a bucket on our AWS S3 account. Over the last 3 years I added another bucket for storing .mp3 videos for another website I manage.

Unfortunately, the server backup usage of AWS S3 did not include actually managing the storage on the AWS S3 server. The backup storage kept growing. While AWS S3 storage is quite inexpensive, it does start to grow if the backups are not managed. So, part of my process was periodically clearing ALL of the backup storage on the AWS S3 server manually. Not excruciating, but still yet another task to be done.

I was quite pleased how easy it was to work with the AWS S3 interface. When we publish a new issue of MT43News (weekly) I automatically archive the PDF version of the paper on the AWS S3 storage using the AWS S3 interface for linux.

However, when we moved to the Hosting.com server, I had to install the AWS S3 interface. It turned out to be a very simple easy install and I got the archiving of the newspaper issues working quickly and easily on the Hosting.com server. However, I wanted to keep the old server in case something came up that I needed something from that server. The server backups are still being done on the AWS S3 storage.  

I decided to write a python program to run on my Windows laptop to manage the backup storage. It took almost no time to implement management of the AWS Storage from my laptop. A morning worth of work and I was able to set up a scheduled task on my laptop that will cleanup the AWS S3 server backup buckets automatically.

AWS S3 storage is amazingly easy to use and very affordable. I do recommend it!