For the past 6 months or so I have been researching open source semiconductors and open hardware business models. It was really difficult when I got started, the build environments especially for CPU cores are fairly intricate and huge.
Somewhere along the way I stumbled across GitLab and I’m very glad that I did.
GitLabCI completely changed the way that I approached this project. It gave me the tools that I needed to visualize the process. not only did it help my own comprehension, but it helps me to explain the project to others as well.
You can check it out here:
In short, matter compiler will soon allow users to design a computer starting with the CPU itself, without needing to deal with a complex development environment at all.
Docker images for toolchains and build environments ensure that users are able to develop against the exact environment that will be used in CI. Artifacts allow users to take compiled CPU cores or software images and run them on FPGAs or on physical hardware.
I’ve been able to make the jobs fairly granular, instead of building in one giant linear process. This has taken the development environment build time down from 6 hours to just about 20 minutes. I’m interested in accelerating it further.
today I read about matrix builds, and I will likely try to implement those for a few of the cores that we support.
I’m also planning to pick up a 32 core, 256 GB, 4x 1 tb nvme machine from hetzner this week, to accelerate the pipeline.
What do I need help with?
Optimization:
My guess is that the current CI configuration that I’m using is not optimal. Currently, it is two very large files: the traditional .gitlab-ci.yml and another file called “extra”.
Beyond reimplementing ccache, is there anything else I can do to speed builds?
Design: am I thinking about this the right way? Can I make it scale?
Visualization are there any third party tools for visualizing GitLab CI pipelines or gitlab CI generally?
Use: I would love to get feedback from people actually using the system. Because get lab is open source, the entire project from end to end is forkable. All you should need is a get lab instance, and a few large runners.to become a user, and use our infrastructure to do so, simply request to join the group or the repository. Make changes to a branch, and you will see those changes immediately built. this is quite different from the usual experience developing open semiconductors.
Jobs are set up so that they default to public runners, only larger jobs use the build cluster.
Thank you!