Infrastructure plan

Introduction

I decided to take on the challenge of building software privately and on a fixed, minuscule budget. So I should first explain exactly what I mean by this and why I think it's important.

Firstly, by "private" I mean that the infrastructure won't be accessible to the public (other than that which I explicitly allow), and source code to my projects will be kept private. Services requiring that I share source code in exchange for using their platforms are therefore excluded.

• This is a disruptive start-up - things need to be kept away from prying eyes (competitors etc).

Secondly, a fixed financial budget, in this case, means the total cost of ownership and operation is fixed and known in advance. i.e. No recurring costs, licence fees, etc. I have plenty of hardware capacity on old servers, and 100% usage of them costs only the electricity.

• No on-going fees - budget is fixed
• Productivity/services do not stop when budget is exhausted
• Hardware is not an issue
  • My local dev server is an 8-core I7 with 24Gb RAM and 4Tb disk

Why not just use the cloud and accept the costs?

Because I don't have to. I can build physical computers quite happily, and I've enough parts to last years. My development stack can be entirely disconnected from the Internet, should I so wish. Oh, and those cloud costs are not as low as people think.

Technology selection

Before considering the applications, I know I'll need the following:

Containerisation

As I want to be able to move things around as I please, I'll choose Docker to handle the containerisation of absolutely everything. Yes, I know Docker is unstable in many situations, but I'll cross that bridge later. It's very well supported by the community, so its selection allows me to fast-track setup and configuration.

This necessitates the creation of containers/machines for the following purposes:

• Docker host for running containers
• Docker repository containing my images (this is containerised)

CI/CD tooling

For continuous integration & delivery, I'll choose Jenkins. It's somewhat dated these days, but it is sophisticated enough for my use, has a very good collection of plugins, a huge community and, frankly, I like it. Jenkins will be deployed inside a Docker container.

This involves creating a container and a volume:

• Jenkins CI for running build jobs

Logging

Logging is a critical service, and I'll use a standard ELK stack. Again, this will be deployed inside Docker containers.

This involves the creation of three containers:

• Elastic search (database) container
• Logstash container for processing log messages from my applications
• Kibana container for its GUI

Monitoring & alerting

Monitoring is also a critical service. I considered a TICK stack, but I'm leaning towards a Grafana solution as it's far easier to setup. Again, this will be completely containerised.

Routing HTTP(S)

For the most part, traffic will be HTTP and HTTPS, and I'll want to access a load of services from the Internet. For the time being, I'll simply use apache2 and http_proxy to route traffic to internal ports, and to handle HTTPS encryption. This is a temporary solution, and will be replaced with something more convenient and resilient at a later time.

The environments

There are three main environments, and they will share many aspects. Being able to throw away an environment and completely rebuild it automatically is important.

The development environment

This is where the bulk of the work will be done, and costs are to be kept down.

• Jenkins CI
• Artifactory
• Docker repository
• Docker containers

The demo environment

This environment is for demonstrating the product, getting user feedback and performance tests etc.

The live environment

This is the actual environment where the product(s) will run.

Source code

There are two main parts. Firstly, the development environment at home. This consists of a big server whirring away all day in my basement. This will have all my work on, except my source code. The source code will be stored in Git at Bit Bucket as it's free for private repositories with very few developers.