If you're reading this article I'd hazard a guess that you know what a cloud development environment is, so I'll spare you the intro. I've been using GitHub Codepsaces and Gitpod in a professional environment for nearly 6 months. Here's what I learned.
TLDR: The only competitive advantage of Gitpod is price.
The similarities
At a glance, GitHub's and Gitpod's cloud environments are nearly identical. They both promise a solution to onboarding, a boost in DX and an end to cross-platform development environments. A Linux instance with Visual Studio Code on top of it is a click away. Both platforms achieve that goal so well that it helped me revolutionize technical interviews.
Unless you have a high-end PC, either service will be a guaranteed improvement over your current desktop environment. Not only are you able to offload all the heavy lifting, but also with a minimal learning curve (if you're already comfortable with a Linux terminal). You hardly need to change your workflow. You don't need to learn any new keyboard shortcuts. You do need to slightly change how you start Visual Studio Code.
Both GitHub and Gitpod have a prebuilding feature. GitHub's is still experimental, but works well(I have access to the beta). Gitpod likes to market their prebuilds as borderline magic, and though they are well-thought-out, hardly make a difference as I'll demonstrate below.
The differences
There are only two that matter:
- GitHub uses virtual machines, while Gitpod relies on Kubernetes
- GitHub is owned by Microsoft, so they get unfettered access to MS's ecosystem, while Gitpod does not
The best way to illustrate what this translates to IRL is to compare a cold startup.
Which is the fastest to boot?
Methodology: The test was done on the same computer, using the same (100 Mbit/s) internet connection and while opening the main branch of the same repo. I measured the time it took from opening a new workspace to VSCode rendering items in the file explorer (insert Adam Savage quote about writing things down).
Codespaces
To start a codespace I go to my recently opened workspaces and select the Codespace I last used.
15 seconds later I'm back to where I left off - my environment variables are there, the git branch I was working on is checked out, my opened files are open, even unsaved changes are still there.
Gitpod
To start a Gitpod workspace I go to gitpod.io/workspaces and select Open
A new tab is opened and you can spend the next 3 minutes doing some stretches while you wait for the container image to come alive.
Then you can choose if you want to open VSCode on your desktop or in your browser (why anyone would choose that option is a mystery)
It takes less than 30 seconds for the SSH connection to be established. This is when I stopped my stopwatch but you have to do an extra step before to start work - open my VSCode workspace, because it is not opened by default. Once that's done your IDE is back where you left it.
Results
- Codespaces: 15 seconds
- Gitpod: 3 minutes and 35 seconds
What about those fancy prebuilds and claims for <1min start times on Gitpod's website? Because Gitpod haven't disclosed their method of testing, it's hard to determine what their claims are. Even opening an already working workspace isn't as fast as booting GitHubs VMs.
There's a very logical explanation about the performance difference - it's not a question of caching, but a question of workflows. Take off your developer hat for a moment and put on your user hat. What's a better way of working:
The GitHub way:
- Open your IDE they way you would on a PC
- Everything is as you left it
- Any changes to your cloud environment are persisted and waiting for you next time you start work
The Gitpod way:
- Say goodbye to any terminal or other customizations that you use
- Don't open your IDE, but go to a webpage
- Click a bunch of buttons
- Wait for your IDE to become available
- Your files are as you left them, but everything is reset to what your gitpod.yml thinks you should be using.
Gitpod seems to think that developers prefer to start fresh workspaces regularly. So they've invested a lot of engineering and marketing effort into their prebuilds. And they do their job. Every new environment is the same as the previous one and thanks to their handy tasks and recently beta dotfiles support you can get a lot of minutia out of the way. You CAN get used to working that way, but why should you?
Github has created a Personal Cloud Environment. The experience is 99% identical to what you're already used to on your desktop. Gitpod has tried and failed to reinvent the wheel.
Performance after boot (it gets worse for Gitpod)
7 vCPUs, 12GB of RAM, 30GB diskspace. That's all you get on Gitpod. Want more? Host your own infrastructure.
Even the most budget-friendly of budget laptops comes with 8-core CPUs. Why would anyone think that 7 threads are enough?
If you're thinking that you can be thrifty with the pitiful amount of resources and make the most of them, don't get your hopes up. If Gitpod detects you're hitting their k8s node too hard they start killing your processes without warning.
Benchmark
Methodology:
- To make the comparison fair I used an 8-core Codespace (instead of my preferred 16-core VM).
- To make it even fairer I used maxWorkers and max_old_space_size to limit jest to 7 threads and 12 GB RAM.
- I used the same versions for all dependencies (node, npm, jest, etc).
- I did not disable jest caching, because I think that'd make the comparison less realistic.
- To collect the results is a somewhat scientific way I used hyperfine.
Gitpod is 38.8% slower.
After half a year of using both cloud platforms, the results don't surprise me. I haven't found a case where a Gitpod workspace is as fast as a Github codespace. I was lucky this time, often tests fail to finish on Gitpod but work in other environments (I suspect their process-killing policy is at play but haven't debugged it).
Coding in a browser
We've all been there-you're at a trendy beach bar in Bali, reading technical documentation on your tablet. Suddenly, an uncontrollable urge to write code has overcome you. What now?! You can't code on your tablet, can you?
Whatever your reason for wanting to use VSCode through a browser - GitHub codespaces is your better option.
When you open a Codespace in your browser it's nearly identical to your desktop experience. The only difference is that you still see your browser's top bar, so you get a tad less screen real-estate.
Initially, things look promising in the Gitpod camp. We see a the familiar UI of everyone's favourite code editor (albeit with an ugly colour theme). However, because the IDE is running outside of a Microsoft-approved environment you lose Settings Sync and their official extension marketplace. Instead, you are confined to manually duplicating your settings and re-installing all your extensions, but only if they are available in the VSX Registry.
Customer Support
I am happy to report Gitpod has good customer support. Their response time is slow, but every interaction I've had was pleasant and professional. I am unable to report on the quality of GitHub's support because I haven't had to use it.
Conclusion
I love a David vs. Goliath story as much as the next guy. The Gitpod vs. GitHub story is one where David loses. Let's at least enjoy using it for free courtesy of their VC funding.