The repositories in the Calvinverse organization are divided into image repositories, containing code for the creation of VM and Docker images, and configuration repositories, which contain some form of configuration information. The configuration repositories either create ZIP and ISO artefacts or they contain scripts that allow pushing data to the Consul key-value store in the environment.
In order to execute builds that create artefacts of some kind you will for the very least need:
Specific repository types may have additional demands on the environment they are build in. More details on these specific demands will be provided in the sections below.
Most of the resources in Calvinverse are VM images although there are a few repositories containing code for Docker containers. There are currently two types of VM hypervisors supported, Azure and Hyper-V. Since images are created with Packer it would be relatively easy to create images for AWS and other hypervisors.
All VM resources are based on either one of the 'base' images, one for the Linux based resources and one for the Windows based resources. These base images consist of an operating system install combined with a number of default applications, e.g. Consul for service discovery, Envoy for service mesh and Telegraf for metrics collection. So in order to build any of the VM resources you will first have to build and store a copy of these base images.
Additional prerequisites for building the virtual machine images are:
Once the base images are created you can start creating the VM images. To build an image you can invoke the following command line.
msbuild entrypoint.msbuild /t:build
Additional properties should be provided depending on the type of hypervisor, i.e. Hyper-V or Azure, you are targeting.
Once the image has been created you can run a series of simple smoke tests to verify that the image is valid. In order to execute these tests invoke the following command line.
msbuild entrypoint.msbuild /t:test
This will deploy the image to your selected hypervisor, connect to the VM, execute the test scripts and finally delete the test VM.
When you are building images for Azure you will need, in addition to the standard application installs:
In general building one of the images will take about 10 - 30 minutes depending on what is being installed in the image. So while the costs of creating images should be fairly small there will still be a cost, especially if you create lots of images.
Once all the prerequisites have been satisfied you can start the build process with the following command line
msbuild entrypoint.msbuild /t:build /P:ShouldCreateAzureImage=true /P:AzureLocation=<AZURE_LOCATION> /P:AzureClientId="<AZURE_CLIENT_ID>" /P:AzureClientCertPath="<PATH_TO_CERT_FILE_FOR_AZURE_CLIENT>" /P:AzureSubscriptionId="<AZURE_SUBSCRIPTION_ID>" /P:AzureImageResourceGroup="<RESOURCE_GROUP_NAME>"
This command line applies to both the base images and the normal resource images. At the end of the build process the image will be placed in the selected resource group and the following tags will be applied
ProductNameproperty in the
ops.propsMsBuild file in the root of the repository.
packerto indicate the image was created by Packer.
qato indicate that the image is ready for QA.
To test the newly created image you can execute the following command line
msbuild entrypoint.msbuild /t:test /P:ShouldCreateAzureImage=true /P:AzureLocation=<AZURE_LOCATION> /P:AzureClientId="<AZURE_CLIENT_ID>" /P:AzureClientCertPath="<PATH_TO_CERT_FILE_FOR_AZURE_CLIENT>" /P:AzureSubscriptionId="<AZURE_SUBSCRIPTION_ID>" /P:AzureImageResourceGroup="<RESOURCE_GROUP_NAME>" /P:AzureTestImageResourceGroup="<TEST_RESOURCE_GROUP_NAME>"
This will create a new VM in Azure with the originally created image and then run a series of smoke
tests. Note that if the tests pass a test image will be dropped in a resource group with the
TEST_RESOURCE_GROUP_NAME name.These images should periodically be removed.
In order to build images for Hyper-V you will need the following additional prerequisites
VM-LANand can be changed in the
Once all the tools are installed you can start the build process of a base image with the following command line
msbuild entrypoint.msbuild /t:build /P:ShouldCreateHypervImage=true /P:IsoDirectory=<PATH_TO_ISO_DIRECTORY>
If you want to build a normal resource image then you should use the following command line
msbuild entrypoint.msbuild /t:build /P:ShouldCreateHypervImage=true /P:RepositoryArchive=<PATH_TO_DIRECTORY_CONTAINING_BASE_IMAGE_ARTEFACTS>
Once the build process completes it will place a ZIP artefact containing the Hyper-V files in the
build/deploy directory of the workspace. To test the image that was created you can run the
smoke tests by executing the following command line
msbuild entrypoint.msbuild /t:test /P:ShouldCreateHypervImage=true
This assumes that the image artefacts will be available in the
build/deploy directory in the
Some of the Calvinverse services are packaged in Docker containers. As with the virtual machine images, the Docker containers for these services are also created from a base image. Once again this base container contains the default applications.
In order to build these Docker images you will need the following additional prerequisites
Once the prerequisites have been satisfied you can run the build by executing the following command line
msbuild entrypoint.msbuild /t:build /P:DirConsulCertificates="<LOCATION_OF_CONSUL_CERTIFICATE_BUNDLE>"
Tags for the Docker containers can be specified in the
docker - pack section of the
The final set of repositories which may need to be build are the configuration repositories. When using Hyper-V as the hypervisor you will need to create a set of ISO files which contain the initial provisioning files. These files can be found in the Calvinverse.Configuration repository. In addition this repository also stores
The other configuration repositories store
The configurations in these repositories can be applied to an environment by pushing the configuration files up to the
Consul Key-Value store. These can be pushed up by using the
Set-ConsulKV.ps1 script in the