I included the current month in the title of this article because we truely live in interesting times and the rate of progress in the LLM space including the open weights models that you can run locally on "consumer" hardware is truely phenomenal!
This is infact the same words that I opened with in my prior article back in May and is no less true today as the rate of improvement in the last 5 months has continued to be break neck and with the release of the latest Qwen LLMs that can run locally is again a bit of a watershed moment worth documenting.
In this article, somewhat like a field report, I will update with an example, a single but I feel useful data point to show currently what is possible in the area of agentic software development.
So lets get down to specifics: what exactly have I been using? On the software side I have used:
Additionally a small guest appearance by the latest impressive model from Qwen: Qwen 3.8-Flash-Next.
On the hardware side I recently put together a very low-budget machine (more details on that coming in soon!) while for running Qwen Flash-Next I am fortunate to have a Framework AMD strix halo 128GB.
The task I decided to give my agent+model combo was something to both "smoke test" the new machine I had just setup and to solve a practical task: writing a Bash shell script to automate a tedious part of the software product release workflow that I had til now being doing by hand.
Here's the prompt:
From a given branch name (thats also passed as argument to the script) and then will get a list of all PRs that were tagged with "cherrypick" label and the given milestone, find the matching commits for each PR in master branch, apply each of those commits in order as a cherrypick to the current new branch and then print summary of all the cherrypicks done, call the script: make-hotfix-branch.sh
The reason I chose this particular task is that while it may seem simple, its both small and self contained enough that I could accept the local model to achieve in a reasonable amount of time yet complex enough with edge cases and error handling required that meant it was not trivial. It also turned out that it was generic enough that it could be done in a webchat window as well as by CLI agents tools, which turned out to be quite handy as you will see below.
Once I had started on this I then got the idea to treat this as a bit of comparison (my methodology was no where good enough to call it an "eval") of the local model vs cloud hosted models. Qwen 3.8 27B now benchmarks at near Opus 4.6 levels but I don't have access to it or its predacessors so I instead chose to give the same prompt to both Qwen and Gemini 3.7 Flash (High) which is currently the latest model available from Google in Antigravity. As an addition, really as an afterthought, I also decided to use Qwen 3.8 via both the llama.cpp web chat UI as well as via the pi agent running in the projects local source code directory, just as I did with using Gemini with Antigravity. This proved to be a very useful lucky accident as you will see in the results below. Because I was testing Qwen on the new "budget" machine I had just built the quant used was unsloth/Qwen3.8-27B-GGUF:Q4_K_M.
Once each agent+model had completed the script I had codex using GPT 5.6 (High) rank the versions of the script, guess which is which...
The result was very interesting indeed! In third place was Qwen 3.8 running via the webchat UI, but it was Gemini 3.7 in second place with Qwen 3.8 via pi agent in first place! Whats just as interesting as the ranking order was just how huge a difference there was in the result between with Qwen result via pi vs in a chat window! Or at least it was surprising until I compared the enormous amount of work and tokens that Qwen model expended via pi vs the much quicker but far less token/work intensive result in the webchat. In pi, Qwen really went to work, not just generatign far more thinking tokens, but also the work done: because it had tools available and was running in the directory of the source code of the project in question, it ran multiple unit tests and integration/smoke tests of the script, doggedly hunting down and fixing mistakes and issues it discovered by doing so. It even went so far as to create test stubs for GH API responses!
Then to add a bit more flavour to this data point, since I had just finished downloading Qwen 3.8 Flash Next (unsloth/Qwen3.8-Flash-Next-GGUF:Q3_K_XL)
Qwen 3.8 Flash Next came out on top, which is not surprising nor the fact that it only caem out a little ahead of Qwen 3.8 27B as I was using it in very heavily quantised "3bit" form and that even though its a 125 billion param model, its a MOE (mixture of experts) with only 6 billion activated.
Of course there are other contenders for the crown of best local open weights LLM though of course that title varies for everyone based as much on their use cases and the hardware they have to run the models locally as much as the "raw" performance results of the models themeselves and of course as you can see from the results above, the agent or at least the ability for the model to do tool calling via an agent makes a huge difference in the performance of the current generation of local models.
Things are changing fast, so its quite likely there will be new, more capable open weights models released by the time you read this, even if you are reading this only days after I publish it! For example the Flash Next "dense" version of Qwen 3.8 was released recently as I said above but I not yet had the time to put it to daily use to judge it properly but this and other initial tests do look very promising!
This experiment of course is no news to people using the latest local models like Qwen, DeepSeek Flash V4 0307 etc. But what is signficant to me is that the rate of improvement in these models has now progressed so far that while they still are slower and use way more tokens than the GIANT frontier cloud models, for the first time these local models are now able to get done a whole range of agentic coding work that was previously the exclusing domain of the giant cloud models. The fact that they run on my own hardware and not a service that can have unpredicatable degraded performance, can be price gouged by any amount or even can be cancelled on me at any time makes my local LLMs as the ads say "priceless"!
I'll have more articles coming up in the near future documenting my further use and exploration of using local LLMs with agentic harnesses so do subscribe if you want to be notified.
Until then, I wish you all a happy and safe today and tomorrow in these very uncertain times.
Get new posts delivered straight to your inbox. No spam, unsubscribe anytime.