Well ‘thanks’ for the ‘simplified’ statement, I guess?
Author: Dan Q
AI Software Development – What Does The Data Say?
This is a repost promoting content originally published elsewhere. See more things Dan's reposted.
…
- The maximum effective context limits of LLMs… beyond which model outputs become unusably inaccurate is orders of magnitude smaller than advertised limits…
- LLMs cannot distinguish between recent and out-of-date information in the context, and information in the model itself, learned during training (“dominant priors”), can often “outweigh” information we give it…
- Repo-level .md files tend to make model performance worse…
- LLMs struggle with negation…
- LLM inference is more accurate when we give them examples (demonstrations) rather than just describing what we want…
- Large/long-scale industry studies show a clear trend – output is up (more code, more commits, bigger diffs), but outcomes don’t reflect that trend. If anything, the average team is taking longer to ship worse software…
- The psychological and cognitive factors in LLM use are a growing field of serious research. One study found a significant correlation between confidence in AI output and belief in the paranormal…
- Deep neural networks, including LLMs, struggle to learn patterns with long-range dependencies, at any scale of model…
- The energy and compute needed to train an LLM to be an order of magnitude more reliable – e.g., wrong 3% of the time instead of 30% – is 10^20 times what the current frontier models require. Don’t expect significantly more reliable models any time soon….
- …many published benchmarks that do indeed show LLMs getting better and better. But other research finds that we might wish to be more skeptical of benchmark performance… they’re not really like real-world problems… [and] increasingly models are being “trained to the test”.
…
This is a really great summary of the state of the art in research into the efficacy of AI-driven software engineering, and it parallels with much of my experience. In particular, my personal experience has been that, properly-used, LLMs can be…
- …great at summarisation, which can make them helpful at tasks like parsing a large codebase to “trace” inconsistent logic, or explaining how a bug report represents a condition that is not covered by an automated test. This is most-valuable for unfamiliar codebases, where the developer’s intuition might be slower than the LLM’s token analysis: however this does introduce a risk compared to a more-systematic debugging process that an edge case (or parallel functionality) might be missed.
- …good at extrapolation. Given a solid example of what you’re trying to achieve, it’s relatively simple for an LLM to extrapolate that to other cases throughout your codebase. This can be useful when switching between many cases with slightly-differing logic, or when mirroring front- and back-end functionality in different programming language, where the LLM’s extrapolation capability may outperform the developer’s context-switching capability.
- …mediocre at creation: greenfield development from a “spec” to “code” is something LLMs appear to do pretty well, particularly for common and well-understood concepts (or, for “deep thinking” models, for tasks that they’re able to summarise correctly as a series of well-understood concepts). But when extending existing functionality, or when implementing functionality that is not well-represented by examples in the training data (e.g. truly new concepts, or use of new features of programming languages), LLMs routinely get stuck in flip-flopping loops, produce invalid solutions, or otherwise cause significantly more harm than they do good.
(They’re also okay for rubberducking, but so’s a rubber duck so that’s not really a good comparison.)
Since I first echoed Molly White’s observations that AI isn’t useless… but I’m not certain that they’re “worth it” several years ago, I’ve continued to experiment with different AI-assisted coding methodologies and technologies, but my fundamental position hasn’t changed: there are clearly some things that a coding LLM can help with, as described above, but there are problems when they get used for other tasks (which they routinely do). And I’m still not convinced that they’re “worth it”, and I’m concerned by those whose anthropomorphise GenAI and by the fact that a lot of industry is being built upon technologies whose runway is running out…
Anyway: great summary, Jason; thanks!
Re: Please let me interact with your blog post!
This is a repost promoting content originally published elsewhere. See more things Dan's reposted.
I love my RSS reader. It’s the first thing I reach for when I’m “catching up.” As I’m reading through my list of feed items, I often come across blog posts that do not give me any way to interact with the post!
Here are some options in the order of my own preferences.
- Webmentions
…- Reply email link directly in feed
…- Reply email link at the bottom of your post
…- Social media links to the blog post
…- Contact page or form
…
I’m with Michael… mostly. And I try to follow all of his suggestions and more here on my site. Let’s break it down from Mike’s list:
✅ Webmentions are fantastic. It’s been a long journey to get there: we had trackbacks. Then we had pingbacks. And now we’ve got webmentions. Each an improvement upon the last. I support all three. (If everything works properly, this post should result in a webmention being sent to Mike so he’ll know about this comment, even though I wrote it on my blog, and it’ll appear as a comment or at least a link on his blog). Webmentions are magical.
❌ Reply email links in feeds is an interesting idea. I don’t currently do this. Except on this post, where I have, experimentally! While I’m a big fan of reducing the barriers to engagement, I figure it’s acceptable to expect people to click-through to see their contact options, so long as you’re already supporting Webmentions. Maybe I’ll add these in a future iteration.
✅ Reply email link at the bottom of your post – yup, I do this every single time. I use dynamically-generated email addresses per-post (mixed in with some hidden honeypots) as an anti-spam mechanism, as well as to help me work out what something was a reply to (it’s sometimes pretty ambiguous: I don’t know why, but often people will email me about years- or decades-old blog posts and not specify what post it was, as if I’d written them just this morning and they were fresh in my mind!).
🤷 Social media links to the blog post: I mean… kinda. I don’t do much in the way of centralised social media, but I automatically push a summary to the
Fediverse at @blog@danq.me and selectively push via my personal account @dan@danq.me too. And occasionally
others. They get tucked-away in a <details> called “syndications”. I’ll get around to connecting to Bubbles eventually, although
only my articles and not e.g. replies like this make it to Bubbles, because apparently I’m too high-traffic for Bubbles otherwise (!). Someday I’d like to
negotiate something a bit better with them, e.g. all of my articles plus anything else I specifically choose so I can include Bubbles-targetting replies like
this!
✅ Contact page or form – I use both, assuming “form” here means a comments form.
Mike’s list is, of course, incomplete. Here’s a few options Mike hasn’t listed, and I think are worthy of mention:
- 🦛 Reactions – I use “reactions” to enable ultra-low-effort engagement for people who just want to say “hey, I read this” without doing anything more. Look at the bottom of this post on my blog and you’ll see that you can click on, say, a thumbs up, or a star, or… a hippopotamus?… icon to give me some feedback. Other people use other kinds of reaction/heart buttons: e.g. Bear Blog comes with one built-in, and there are third-party services that are basically like hit-counters but on-click.
- ✍️ Guestbooks – I don’t have one of these, but I’m considering it. They’re properly retro, and they’re more about responding publicly to a whole site than a specific post… but sometimes that’s a thing that a person wants to do!
- ✉️ Postal mail? Yes, really! For almost a year now I’ve been encouraging people to send physical mail postcards to by blog! Right now you have to click-through to my contact page for the details, but in a future version of my blog the postal address will be right there next to the comment form.
I’m sure there are others too, but I just wanted to look at Mike’s list, compare it to my features, and think a bit outside the box to boot.
Three(ish) Weeks In Spain
Ruth, JTA, the kids and I are all back in the UK1 after a three week period during which some or all of us were consistently in Spain. The whole adventure was the kind of masterpiece of epic planning that only Ruth could have pulled off, and it worked spectacularly2.
San Sebastián
Through clever permutations of remote-working from overseas (mostly by Ruth) and overlapping/intersecting stretches of annual leave (mostly by JTA and I), we were able to avoid having to send the dog to the dogsitter for a block of more than a single week3, didn’t need to break the bank on our collective leave budgets, and managed to give the kids the longest possible block of continuous immersion in the language and culture of Spain.
In the first week (when I wasn’t there), the kids got to take a mixture of surfing classes and Spanish language lessons in San Sebastián, among other activities. The Bordeaux wildfires caused some travel chaos for the others getting there, but they made it work in the end.
Tarragona
Later, I flew out to meet up with them and we switched to the opposite coast of Spain, staying in a delightful little villa North of Tarragona. The summer heatwaves that are affecting the continent made the middle of the day especially oppressive, so we did as the locals do and sensibly sheltered indoors during the hottest part of the day… but the kids in particular were glad to be able to take a dip in the pool to cool off.
As has become my tradition when travelling and on family holidays – because I’m an earlier riser than most, but also because I wanted to make the most of the coolest part of the day – I got out early most days for a whole bunch of geocaching expeditions5. But once others were awake we found plenty of other opportunities to explore the area.
Having come from, remarkably, a similar level of temperature and humidity in the UK, I was interested to see how much greener everything was here on the Mediterranean coast. The plants here, better-adapted to hot and dry summers than those that we plant on our little island, were clearly more-suited to these kinds of conditions. Is it perhaps time that Britain started ditching her lawns and other year-round thirsty vegetation… instead planting more tropical-adjacent plants to provide shade (and carbon capture!) as the climate continues to shift?
I enjoyed that we were able to split out time between a diversity of activities. A week-long jaunt can be dominated by a single activity – a skiing holiday, for example – but by the time you’re spending most of a month exploring a country, you want to be able to mix things up a bit, right?
Sometimes we kept it slow-paced and relaxing, like reading by the pool while the children invent splashing-based games or playing board games in the shade. Other times we explored history and culture, like our visit to the Ferreres Aqueduct, the Catedral de Tarragona, or the old Roman amphitheatre.
And other times still we even did the typical “British tourist in Spain” things, like a trip to the beach or an expedition out on a catamaran to swim in the sea while the sun set.
Obviously the biggest reason for our trip was to catch sight of the total solar eclipse as it passed over the region, and our villa’s garden/vineyard was without a doubt a superlative spot for this. We’d initially been concerned that visibility of the low-in-the-sky/close to sunset eclipse might be obstructed by the hills and that, therefore, we’d need to find another spot from which to spectate, but an early-in-the-week experiment of watching the sunset (with an eye on the clock) reassured us that this wasn’t the case. So – as long as we were lucky with the cloud cover – we should be set.
I’ve been fortunate enough to have had great visibility of a total solar eclipse once before, in 1999, but honestly: this one beat it. The perfect location, the reds of the setting sun, the “moon illusion” exaggerating the perceived size of the moon as it approached the sun, the joy of vicariously sharing in the childrens’ wonder… all of these made the whole experience pretty mind-blowing.
We watched, we awed, and then we saw the still half-obscured sun gradually disappear over the horizon. And then, a round of hugs and cheers and conversation later, we sat up late to enjoy the Perseids begin their nocturnal display.
PortAventura
After our week in the villa we delivered JTA to the airport to return home. Then Ruth, the kids and I drove on to PortAventura, perhaps our favourite theme park anywhere in the world. Since our last visit, the younger kid has reached a milestone of height that permits him to ride all of the largest rides, and he was excited to take advantage of this newfound privilege.
Both the elder child and I were particularly excited to share with him my favourite non-looping rollercoaster, Shambhala. Shambhala is simply excellent: it’s a highly-vertical smooth ride with a free and open-feeling restraint that, collectively, give amazing “air” and a sensation of “flying” that’s hard to compare to almost any other coaster ride I’ve been on. I could sing its praises aplenty.
Going to Disneyland Paris and PortAventura in the same summer gives me the confidence to say: PortAventura is the better theme park. The rides, the theming, the food, the queue management, the accommodation, the value-for-money… it’s all better. So if you’re considering a European theme park trip and you don’t care about getting your photo taken with Goofy or whatever… PortAventura would be my suggestion.
Barcelona
Finally, we made our way to Barcelona for a “wind-down” couple of days at a hotel we’ve always enjoyed there. Aside for being mistaken for staff while I was using my laptop by the rooftop pool, there’s little of note to say about it: we took it easy, did as little as we could get away with, and generally “unplugged” before our journey back to the UK and a return to normal life (boo!).
Altogether a fabulous trip… and a really interesting experiment in a model that we could replicate if we ever opted for a three week (but drop-in/drop-out/telework) holiday in future. Though… I’d again want (our scheduling queen) Ruth to wrangle the logistics, because it gets complicated when there are so many moving parts!
Footnotes
1 Although not in our house, of course, which is still being repaired following the flood in February; it’s now F-Day plus 191 days… more on that in a future blog post, I’ve no doubt.
2 For a while, she was talking about trying to engineer it into a five-week expedition spanning both this and our trip to France earlier in the summer (to which JTA, the kids and I drove/LeShuttle’d, and to which Ruth flew following a work-related event in Austria: my, what a cosmopolitan life we’re leading this year!). But that’d have been an enormously more-complicated affair and I’m glad we settled on this “simpler” trip in the end!
3 Not that she cares: the dog loves hanging out with her little doggy friends, and is always excited to be taken to our dogsitter‘s, where she also “commutes” for a workday or two most weeks.
4 During out post-holiday ‘retro’ (because of course that’s what we do), the lack of connectivity appeared as both a negative and a positive on some of our lists! Obviously we’re all aware that a little digital disconnection is healthy and valuable, but sometimes it takes being in the middle of nowhere to impose the requisite self-discipline to keep off your devices!
5 Geocaching logs: 1, 2, 3, 4, 5, 6, 7, 8, 9 (my 1000th successful find) – not that Geocaching.com can count that correctly! – 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37.
6 Obviously, the Devil’s Bridge nearest where I used to live and at which Ruth and I spent our first anniversary – back when we had no idea that we’d be able to make our relationship last… what…? 19 years this month!) – will always have a special place in my heart.
7 Reaching the top of King Khajuna’s tower, you can feel it swap in the breeze, and when you get to the slide itself its ‘horizon’ disappears precipitously until you shuffle completely up to it, at which point there’s little choice about your destination! While we queued for our first goes, we witnessed several people reach the top, change their minds, and turn back!
A Little Bit Straighter
The clasp on my ‘bi pride’ watch strap is failing because, in an ironic twist, part of it needs to be made a little bit more straight.
Fishsmith
I’m a little fishsmith,
Short and stout.
Here’s my anvil,
Here’s my trout.
Dan Q wrote note for GC448J6 EL TRESOR DEL POU DEL MON
This checkin to GC448J6 EL TRESOR DEL POU DEL MON reflects a geocaching.com log entry. See more of Dan's cache logs.
When I visited two years ago I thought I’d solved the puzzle but couldn’t find the cache. I’m in the area again today and thought I’d make another attempt, but the recent ‘enabled’ seems to suggest that it isn’t actually available?
Maybe I can try again in 2028!
I don’t work here!
I guess it’s the laptop and the pile of towels that cause the confusion, but people keep asking me questions (usually in Spanish) about where to get towels from and whether they can use the pool at this hotel.
And I have to keep explaining that nope, I don’t work here, I’m just watching my kids play while I catch up on my RSS reader!
Dan Q found GC9Q6G6 PORTA DE BARCELONA
This checkin to GC9Q6G6 PORTA DE BARCELONA reflects a geocaching.com log entry. See more of Dan's cache logs.
A quick find exactly where it should be while driving from Tarragona to Barcelona. My family and I were in the area for the eclipse and then for a trip to PortAventura and are heading to Barcelona for a few days before flying home.
My partner and our kids went to the bathroom and snack bar while I jogged up to the bridge. The coordinates put my close but the hint moved me exactly where I needed to be. Strangely, I suddenly needed to tie my laces, and then just as suddenly the cache was in my hand!
Thanks for breaking up this long journey (full of bad traffic!) for us. TFTC, and greetings from Oxfordshire, UK!
Geocaching.com Can’t Count? (my 1000+ geocache finds!)
This week I’m in Spain to see a spectacular eclipse, among other things. I’ve also done a reasonable amount of geocaching, including on Tuesday finding my 1000th geocache.
Which led me today to ask: how many caches does Geocaching.com think I’ve found? It’s the biggest, but not the only, site I use, and so I didn’t expect it to show the same total as me. A difference is expected.
Well… it turns out how you ask it. In the top-right corner of the page, when I log in, it says that I have… 1,035 finds:
That turns out to be significantly higher than I’d anticipated, which initially made me wonder: did I fail to count some in my tally?
My tally comes from those which have been imported right here onto my blog. I use a computer-assisted but still-manual approach1 to PESOS my geocaching logs from various sites – Geocaching.com, OpenCache.uk2, TerraCaching, etc.3 – onto my blog, and then I count them here. Could my data be wrong?
So I went about checking it. Which turned out to be challenging, because the “foreign key” I use to associate the caching logs on my website is the log ID that appears in the links on Geocaching.com… and those changed format somewhere along the way.4 So first I had to fix all of my metadata spanning about 750 rows. Then I was able to check.
But while I did that, I noticed something strange. When I click on that link that says I have “1,035 finds”, the resulting page has a link to see what they are. And it claims there are… 1,005 finds:
Stranger still, a different link from the same page on Geocaching.com told me that I had 1,006 finds!
Even before I’d corrected all of my metadata and rechecked my totals, I was becoming increasingly convinced that I couldn’t trust any of Geocaching.com’s totals. How many finds is it, exactly: is it 1,035? Or 1,006? Or 1,005?
It doesn’t really matter, of course. I keep my own count, and I’m now even-more confident than ever that it’s accurate. The correct answer to to the question “how many geocaches have I found?” is 1,019.
And for some reason, Geocaching.com can’t count even the ones that are logged on their site three times and get the same number twice. 🤣
Footnotes
1 This helps me sidestep Geocaching.com’s draconian rules on scraping and API access.
2 I got in trouble once with a Geocaching.com reviewer for hinting at the existence of other geocaching listing sites, once. I don’t understand why: they’re unlikely to be a commercial threat to Geocaching.com’s business, and those other sites typically allow you to talk about Geocaching.com! I guess that they’re mostly afraid that it weakens their trademark on the word “geocaching”, which I’ve never been convinced is legitimate anyway because it was being widely used online to describe the activity it represents before they tried to start centralising and hoovering-up all of the cache listings. It’s just sneaky monopolistic silo behaviour on the part of Geocaching.com, really. I’m not a fan. I can list plenty of examples of this kind of dipshittery on the part of Geocaching.com, but I’ll spare you for now.
3 I do a similar thing with my geohashing logs, although without the same anti-anti-scraping mitigations because geohashing.site’s site owner is a much more-chill dude.
4 Many of my old logs used a GUID, but newer ones use a GL-ID. It’s a whole thing.
Dan Q found GCAYFMG Més enllà de la via II (13) – Granja Fortuny
This checkin to GCAYFMG Més enllà de la via II (13) - Granja Fortuny reflects a geocaching.com log entry. See more of Dan's cache logs.
Retreived the cache from beneath its insect guardian (pictured). TFTC!
Dan Q found GCBBYYH Més ellà de la via II – (12) Caçar entre pedres
This checkin to GCBBYYH Més ellà de la via II - (12) Caçar entre pedres reflects a geocaching.com log entry. See more of Dan's cache logs.
Coordinates spot on! Exactly where it should be. From the right angle, the cache can quickly be spotted. TFTC!
Dan Q found GCAY62F Més enllà de la via II (11) – Quina pila
This checkin to GCAY62F Més enllà de la via II (11) - Quina pila reflects a geocaching.com log entry. See more of Dan's cache logs.
Quick find in the second place I looked while walking this trail as a shortcut back to town from the Passejant amb la Boira caches. TFTC, and greetings from Oxfordshire, UK!
Dan Q wrote note for GCAY628 Més enllà de la via II (10) – Em sento penjat
This checkin to GCAY628 Més enllà de la via II (10) - Em sento penjat reflects a geocaching.com log entry. See more of Dan's cache logs.
Not attempted. Lots of litter around the GZ, didn’t feel like walking, let alone poking, through it. 😢
Dan Q found GCB0EGT Passejant amb la Boira – Un altre túnel?
This checkin to GCB0EGT Passejant amb la Boira - Un altre túnel? reflects a geocaching.com log entry. See more of Dan's cache logs.
Found in the fourth place I looked. Nice container size, like the last letterbox hybrid.
This is where we’ll part ways. As much as I’ve loved this series so far and would enjoy exploring more of it, I need to get back to my accommodation where, apparently, my kids are waking up and ready for another fun day. So I’ll be switching to the Més enllà de la via II trail and finishing my walk soon FP awarded for this series as a whole; I’ve loved every moment of it (even that one frustrating cache from a few minutes back!).
TFTC, and if you the CO ever find yourself over my way, can I plug my cache GC9GTV3 Drive Slowly; Fox Crossing. It’s 100% the kind of hide you’d enjoy based on what I’ve seen of your work here!