How To: Set up n-node Cassandra environments for testing in Docker
A short guide on how to spin up a parameterized environment definition for a n-node Cassandra cluster locally over Docker
.jpg)
How to set up an n-node Cassandra environment
This guide is also available in our docs!
Introduction
In this guide, you will set up a 3-node Cassandra cluster in Docker and parameterize the environment definition so it can easily be modified for use in different tests that require an n-node Cassandra cluster. Then we will show you how to run remotely hosted packages authored by others, and go through how to package and publish our work to Github for others to use as well.
Specifically, you're going to configure your test environments with a way that allows you to both:
- Parameterize the environment so another developer using the environment can specify how many nodes they’d like for their system to have, and
- Make the environment definition composable so that your environments can be included in tests with other services for different scenarios & use cases
Without Kurtosis
One way to accomplish the above would be to write shell scripts over docker, or over binaries running on bare metal. In this case, we’d have to build these capabilities into our scripts and handle cross-system issues (laptop vs CI) ourselves.
With Kurtosis
In this guide, we’re going to use Kurtosis. Kurtosis has composability, parameterizability, and portability built into its environment definition system and runtime. With Kurtosis we can ensure that these environments are runnable on your own laptop or in your favorite CI provider.
Setup
Before you proceed, make sure you have:
- Installed and started the Docker engine on your local machine
- Installed the Kurtosis CLI (or upgraded it to the latest release, if you already have the CLI installed)
Instantiate a 3-node Cassandra cluster
First, create and cd into a directory to hold the project we’ll be working on:
Next, create a Starlark file called main.star inside your new directory with the following contents:
Finally, save your newly created file and, from the working directory we created, run the following command:
Kurtosis will run validation checks against your code to ensure that it will work before spinning up the containers for our 3-node Cassandra cluster. We won’t dive into the details of how validation checks are used by Kurtosis in this guide, but you can read more about them here.
Your output will look something like:
Congratulations! You’ve used Kurtosis to spin up a three-node Cassandra cluster over Docker.
Review
In this section, you created a Starlark file with instructions for Kurtosis to do the following:
- Spin up 3 Cassandra containers (one for each node),
- Bootstrap each node to the cluster,
- Map the default Cassandra node container ports to ephemeral local machine ports (described in their respective ServiceConfig), and
- Verify using nodetool that the cluster is up, running, and has 3 nodes (just as we specified) before returning the names of our nodes.
You now have a simple environment definition for Kurtosis to spin up a 3-node Cassandra cluster. You may now be wondering: but what if I need to change the number of nodes?
Fortunately, Kurtosis environment definitions can be parameterized. We’ll see just how easy it is to do so in the next section.
Parameterize our Cassandra cluster
Kurtosis enables users to parameterize environment definitions out-of-the-box. If you need to run tests against multiple different configurations of your environment, you'll be to do so without needing maintain different Bash scripts or docker-compose.yml files per test.
You can parameterize our Cassandra cluster environment definition by adding 2 lines of code to your main.star Starlark file.
Let’s add in those extra lines now.
In your main.star file, add the following lines in the code block that describes the plan object:
Now save your newly edited main.star file and run the following command to blow away your old enclave and spin up a new one with 5 nodes:
It may take a while, but you should now see the following result:
Congratulations! You’ve just parameterized your Cassandra cluster environment definition and used it to instantiate a 5-node Cassandra cluster. You can run the same command with 2 nodes, or 4 nodes, and it will just work. Kurtosis environment definitions are completely reproducible and fully parametrizable.
Note: Depending on how many nodes you wish to spin up, the max heap size of each node may cumulatively consume more memory on your local machine than you have available, causing the Starlark script to time-out. Modifying the MAX_HEAP_SIZE property in the ServiceConfig for the Cassandra nodes may help, depending on your needs.
Review
How did that work?
The plan object contains all enclave-modifying methods like add_service, remove_service, upload_files, etc. To use arguments, you accessed them via the second parameter of the run function, like so:
…which is what you used in your main.star file originally!
What you did next was add an if statement with the hasattr() function to tell Kurtosis that if an argument is passed in at runtime by a user, then override the default num_nodes variable, which we set as 3, with the user-specified value.
In our case, you passed in:
which told Kurtosis to run your main.star file with 5 nodes instead of the default of 3 that we began with.
Note that to pass parameters to the run(plan,args) function, a JSON object needs to be passed in as the 2nd position argument after the script or package path:
In this section, we showed how to use Kurtosis to parameterize an environment definition. Next, we’ll walk through another property of Kurtosis environments: composability.
Using and making composable environment definitions
You’ve now written an environment definition using Starlark to instantiate a 3-node Cassandra cluster over Docker, and then modified it slightly to parameterize that definition for other use cases (n-node Cassandra cluster).
However, the benefits of parametrized and reproducible environment properties are amplified when you’re able to share your definition with others to use, or when you use definitions that others (friends, colleagues, etc) have already written.
To quickly demonstrate the latter capability, run:
which should give you the same result you got when you ran your local main.star file with 5 nodes specified as an argument. However this time, you’re actually using a main.star file hosted remotely on Github!
Any Kurtosis environment definition can be converted into a runnable, shareable artifact that we call a Kurtosis Package. What we just did was run a remotely-hosted Kurtosis package.
While this guide won’t cover the steps needed to convert your Starlark file and export it for others to use as a Kurtosis Package, you can check out our docs to learn more about how to turn a Starlark file into a runnable Kurtosis package.
Review
You just executed a remote-hosted main.star from a Kurtosis package on Github. That remote-hosted main.star file has the same code as our local main.star file and, with only a locator, Kurtosis knew that you were referencing a publicly-hosted runnable package.
The entirety of what you wrote locally in your main.star file can be packaged up and pushed to the internet (Github) for anyone to use to instantiate an n-node cassandra cluster by adding a kurtosis.yml manifest to your directory and publishing the entire directory to Github. Read more about how to do this here.
Turning your Starlark code into a runnable Kurtosis package and making it available on Github means any developer will be able to use our Kurtosis package as a building block in their own environment definition or to run different tests using various configurations of nodes in a Cassandra cluster.
This behavior is bi-directional as well. Meaning, we can import any remotely hosted Kurtosis package and use it’s code inside our own Kurtosis package with the import_module instruction like so:
Package distribution via remote-hosted git repositories is one way in which Kurtosis enables environments to be easily composed and connected together without end users needing to know the inner workings of each setup
Conclusion
And that’s it! To recap this short guide, you:
- Wrote an environment definition that instructs Kurtosis to set up a 3 node Cassandra cluster in an isolated environment called an Enclave (over Docker),
- Added a few lines of code to our main.star file to parametrize your environment definition to increase the flexibility and use cases with which it can be used, and,
- Executed a remotely-hosted Starlark file that was part of what's called a Kurtosis Package to demonstrate how your environment definition can be shared with other developers.
We’d love to hear from you on what went well for you, what could be improved, or to answer any of your questions. Don’t hesitate to reach out via Github or email us!
Other exercises you can do with your Cassandra cluster
With your parameterized, reusable environment definition for a multi-node Cassandra cluster, feel free to:
- Turn your local main.star file into a runnable Kurtosis package and upload it on Github for others to use following these instructions
- Use our Go or Typescript SDK to write backend integration tests, like this network partitioning test
Other examples
We encourage you to check out our quickstart (where you’ll build a Postgres database and API on top) and our other examples in our awesome-kurtosis repository where you will find other Kurtosis packages for you to check out as well, including a package that spins up a local Ethereum testnet or one that sets up a voting app using a Redis cluster.
Wrapping Up
Get early access and updates
We're in build mode at Kurtosis. Follow along for the latest.
.svg)