What Is Ensemble CI?
Ensemble CI is designed to help businesses improve the quality, reliability and timeliness of their data pipelines by implementing an automated but controlled development lifecycle around their data transformation code.
For more details, please visit our introductory blog post.
Ensemble CI Setup Instructions
In this section we will walk through the process of configuring your Ensemble CI deployment.
Install Docker and Docker Compose
Ensemble CI is packaged and deployed using Docker and Docker Compose. You should therefore install a recent version on your deployment server.
Setup Your Github Application
Ensemble CI integrates with Github in order to build your repos and branches.
There is a small amount of manual configuration in Github to enable this interaction to take place with your own account.
- Visit the OAuth Apps area of Developer Settings - https://github.com/settings/developers
- Create a new app with the settings below, substituting in the publically accessible IP which you will use to host Ensemble CI
- Register your app
- Note down the Client ID and Client ID Secret for use in a subsequent step
Ensemble CI will interact with this application, meaning that you have no dependence on any centralised service hosted by us.
Clone This Repo
The next step is to clone the main Ensemble CI repository. This contains the docker-compose.yml file which we will use to start the system.
git clone https://github.com/ensembleanalytics/ensembleci.git
cd ensembleci
Set Details In The .env File
Copy the example .env file:
cp .env.example .env
Edit the .env file and set the values correctly using the keys saved earlier, replacing the values below. You'll also need to set the publically accessible IP address of the server you are running on.
SERVER_IP_ADDRESS=123.123.123.123
GITHUB_CLIENT_ID=cc7666ffbafb59c4XXXX
GITHUB_CLIENT_SECRET=377c0ae980a454082c7c4992968f56e881c9XXXX
Run The Application
Start the application using Docker compose.
docker-compose up
If everything has gone well, the three services, frontend, backend, and CI server should start with healthy logs.
Access The Application
The frontend applications should be available at http://localhost:3000.
Port Forwarding
Github will call the CI server with webhooks whenever new code is checked into your monitored repos.
Depending on your networking setup, you may need to setup port forwarding to connect from your public facing router to the server where Ensemble CI is running.
Self Managed Deployment From Source
If you would like to deploy from source instead of using Docker, please visit our contributing guide for details of how to clone and run the Ensemble CI repos.