- OpsFlow Newsletter
- Posts
- Streamlining Distributed System Testing With .NET Aspire
Streamlining Distributed System Testing With .NET Aspire
How we adopted .NET Aspire on a brownfield project
Modern software applications are built as distributed systems. Even the most simple applications still have a frontend, backend APIs, databases, etc. Larger systems will have even more dependencies, systems, micro-services that increase the complexity.
While these modern architectures bring many benefits, they are not without trade-offs. Local development and testing of the entire system gets more complicated as you have to spin up every service to test how the system works as a whole. Which in turn usually means you have lots of commands or scripts to run. You need to know what order everything should start in and how everything is connected together. You often also need to install databases, emulators, etc. before you can start running anything. Not to mention managing database state and ensuring all migrations have been applied.
Often you end up working around this by deploying a shared dev environment in the cloud, this allows you to have some semblance of the platform running, but that comes with its own trade-offs. You often end up stepping on each other's toes but having lots of concurrent work in progress all updating those shared resources. Which leads to upset developers when unrelated code breaks because someone just broke the dev database.
About 18 months ago, after struggling with this ourselves, we decided to adopt Aspire (formerly .Net Aspire) to orchestrate our entire stack and tackle this problem.
In this post, I'll share how we adopted Aspire in a fairly large brownfield project. I'll cover our journey, what it looked like and what we ultimately achieved.
Hopefully, it will be useful for engineering teams considering adopting Aspire and give a realistic idea of what to expect.
Software landscape before adopting Aspire
For some context I work with a small very technical team. We’re a small team of software engineers: 3 frontend engineers, 3 backend engineers and one overworked Automated Tester.
Our software estate is relatively simple by some standards. We have a mostly monolithic setup with a .Net API that powers a variety of JS SPA front end systems. We’ve got some backend systems that run automated reports and background jobs. Plus a shiny new ReactNative mobile application. With a couple of legacy .Net WebMVC projects. All centred around a primary database, Redis cache and storage.
Before Aspire (like many others), we faced the usual problems with trying build and run our platform locally. Even with this relatively simple setup, spinning everything up for local testing was too cumbersome to be feasible. There was no easy way to get the database running locally, let alone trying to get those background services setup to run at the correct times. So to get around this we were using a shared development environment we hosted in Azure. We had an API, database, etc. that would get updated when code is merged.
This was mostly ok for the FE developers as they can focus on writing their code and have an API that’s running 24/7 for them. It wasn’t too bad for the backend developers either as we had a hosted database we could write against that we knew had a lot of data in, so we never had to really worry about seed data.
This approach worked, but it came with a few big trade-offs.
Shared dev environment in the Cloud meant no one on the team was really running the whole system locally to test their work. You had to rely on those services being available, and you need to be very strict in how updates are applied.
It was very easy for someone to accidentally break the development environment, as you could be making DB changes for your code that end up breaking everyone else until that code is merged. This limits creativity and hinders the performance of the team.
This approach also impacted our ability to test. Our tester could be in the middle of running a test suite and get false errors because someone else has pushed a change or implementing some new feature.
Aspire as a solution to our development problems
For us adopting Aspire wasn’t just about improving the local development experience, it also played into a bigger project we were undertaking to modernise how we used Azure and how we hosted our platform. We wanted to move towards something like Docker and containers so that we could have a consistent development and runtime experience across environments. We spent some time reviewing the pros and cons between Aspire and vanilla Docker, and although we were very early adopters of Aspire, it paid dividends since adoption.
By taking the time to learn Aspire, we were able to properly model our entire platform in the AppHost, which has been super helpful for the whole team. Everyone now has a really easy way to see what dependencies there are between services, what configuration we need, etc. It’s been a huge help when it comes to onboarding new developers and opening a lot of new possibilities when it comes to testing.
When we set about implementing Aspire, we wanted to make local development as painless as possible. Free up our development environment so it could be used for proper integration and E2E testing. And finally to have a modern way to deploy our platform to the cloud in a consistent approach, removing the whole “It works on my machine” argument as every environment would be the same.
Implementing Aspire on a brownfield project
So how did we go about implementing Aspire?
Implementing Aspire in a new project is very easy as you can use the starter projects to get everything going. With no existing code you can also use all the lovely Aspire packages in their vanilla form. This means you can get going insanely fast now with very little setup time to get all your core services ready.
However, when it comes to adding Aspire to an existing project it can take a bit more care. It’s still relatively easy to implement (and the payoff is well worth any pain getting there) as you can adopt it at your own pace. You don’t have to use everything in Aspire from day 1, you can take your time to adopt the features you want, when you want.
For us it was a 4-6 month project. We wanted to focus first on the local developer experience and then slowly adopt more of the Aspire features as we got our Azure estate ready for the switch to Containers. As such we took a very agile iterative approach to our adoption and rollout of Aspire. The process wasn’t without its hiccups but there were no major problems and I recommend everyone gives it some consideration.
I led the adoption, and below I share what was required to get our system up and running with Aspire so you can get an idea of what you might face.
Mapping our software estate
To begin adopting Aspire you need to understand how your platform interacts with all its components. What configuration do you need, where are the secrets, what 3rd party services are we using, etc. Once you know this you can begin to plan out what you need to tell Aspire.
I did this by doing a review of all our deployed services. This meant I could see what was set as Environment Variables, what was in KeyVault and what was in App Configuration store. This also helped to see what differences we have between our Development and Production environments. Pulling all this together meant I had that nice clear picture of what was where and what I needed to setup in Aspire.
Moving to a mono-repo
Although Aspire can work across multiple repo’s, it does add some complexity as everyone needs to make sure they have the repos’s cloned into the same structure as everyone else. We decided this was a complexity we didn’t want. So we also set about merging our handful of repositories into a single mono-repo.
The big advantage here is that the Aspire setup is the same for everyone. Everyone has the same codebase in the same location. It also means we can centralise things like our GitHub workflows. The biggest challenge here was for our Mobile application, which needed some care to migrate into the mono-repo. It was worth the effort to get everything uniform and together.
The need for Docker
Aspire is built to manage containers, so we needed to adopt Docker (or Podman). This was a bit of a change from how we were working before. Prior to Aspire and Docker we had some small C# MVC wrapper projects that essentially hosted our JavaScript SPAs. This added some unnecessary complexity that we could avoid.
We decided to move our SPAs out into their own projects with a Dockerfile to manage their eventual deployment to Azure. Aspire has native support for most JavaScript apps, ours all use Vite which is built into Aspire. So this meant we could kill off the old C# projects and have nice vanilla SPAs that could be deployed directly to Azure Container Apps.
Luckily Aspire and .Net can automatically create dockerfiles in the background for .Net projects. So we didn’t have any extra work to get Azure Container App support for our .Net projects.
The Aspire AppHost
In Aspire, the AppHost is your orchestrator. It’s the core to everything Aspire gives you. You use the AppHost to model your application. It’s in here that you define all your resources and how they interact with each other. You can define all the relationships between resources and importantly for local development, you can define the start-up order and persistence for the containers. This is where the real power of Aspire comes into play.
I've written more in-depth articles on how to do it here:
https://intrepid-developer.com/blog/adopting-aspire and https://intrepid-developer.com/blog/getting-started-with-aspire
Creating a seed dataset
Now that we have Aspire setup, we needed to start thinking about local data. This was a big undertaking for us (and still isn’t complete) as we needed to start building our core seed scripts. This included creating things like core users, etc. By going through this we now have the ability to have a consistent setup for everyone. This also really helps when it comes to testing, as we can use this to ensure we always start from a known state.
Getting the team on board with Aspire
It took a little while to get the team setup with Aspire. As with all big changes there were hiccups and this meant adoption was split across the team. The backend developers were quicker to adapt to it. For our front end team it took a little longer as Aspire needed to catch up with the FE tooling.
The big win for us was the Aspire CLI, this made it a lot easier for the team to start really using Aspire. We now have a nice consistent approach for the whole team. We also heavily invested in documenting how to run and use Aspire. All of this makes it a game changer for us, the ability to onboard devs is a whole lot easier now.
Reaping the benefits of Aspire
So now that we have Aspire, what are the other benefits we get from it? There are 3 main benefits that we’ve gained from adoption. Each of these have helped us become more efficient and performant as a team. We’re able to work more independently and as a team our performance has increased. We can get more work done and have a much more stable platform because of it.
1. Better observability
By adopting Aspire we also moved to using their NuGet packages for things like SQL, EF Core, etc. By using their packages we get a nice default setup that includes things like health checks and Open Telemetry. This gives us a huge amount of observability into our application, the Aspire Dashboard does an amazing job of exposing this to us as well.
2. Consistent environments
We now have a very consistent environment with Aspire. Every developer has the exact same setup which for the most part removes the whole “it works on my machine”, we all have the same setup and it ‘just’ works. We no longer step on each other's toes with shared databases and API’s. It’s all local and we can iterate at a much higher rate.
This extends beyond local development too, we also get a nice consistent setup in deployed environments as well. I’m not saying all our deployed environments are identical, but the fact everything is now in containers we have a much higher confidence level when we do push code.
3. Testing
The final advantage has been with testing. The Aspire test framework is really good, it allows us to write E2E tests using a known state that is super easy to reset at any time. We can now have very performant tests that we can use to test almost any aspect of our platform. We can use this to run overnight E2E tests directly in our CI/CD without having to provision an environment for it. Plus again we have a consistent test approach when we work locally. It’s been a win-win for us.
Current Limitations of Aspire (for us at least)
There are some limitations with Aspire at the moment that mean we can’t really use it for the final deployment step. Our production setup is over several regions with redundancies. This means we have a more involved release and provisioning process. For now we’ve decided to use good old Bicep and PowerShell to manage the actual Azure resources. This allows us to have a staged release process and support having multiple active regions.
Aspire will certainly get there with this, each release of Aspire gives us more control over the provisioning and deployment process. Our ultimate end goal is to use Aspire for the whole process. I think if you have a platform that is a single region without things like Azure Front Door, etc., you should really use it. I personally use Aspire E2E for personal projects (like my blog) as it is so easy and simple to do aspire deploy and you’re done.
De-risking Aspire adoption in brownfield projects
It can be a scary thought to overhaul your whole setup to use Aspire. But hopefully by reading this you can see how you can slowly adopt Aspire. You don’t have to go full hog and use everything in one go. But what you can do is add that AppHost and begin modelling your application. Slowly add in the extra features as you go and start reaping the benefits.
Aspire isn’t perfect (yet), but it’s evolving at a crazy pace. So if it doesn’t give you everything right now, I’m sure it will in the not-too-distant future. You should totally give it a shot!
Chris is a full-stack software engineer and the guy behind intrepid-developer.com
He has been building web applications for over 15 years, specialising in the .NET ecosystem, with a strong focus on technologies such as Blazor, PowerShell, and Azure.
<Intrepid-Developer/> is a place where Chris gives back to the dev community and helps others take their next step in coding.
If you're looking to level up your engineering skills in the .NET ecosystem, I highly recommend checking out intrepid-developer.com
You can learn more and connect with Chris here:
Edited by Vova Pylypchatin.