Hey, I wrote this! There are a couple of reasons that I included the disclosure.
The main one is to set reader expectations that any errors are entirely my own, and that I spent time reviewing the details of the work. The disclosure seemed to me a concise way to do that -- my intention was not any form of anti-AI virtue signaling.
The other reason is that I may use AI for some of my future work, and as a reader, I would prefer a disclosure about that. So I figured if I'm going to disclose using it, I might as well disclose not using it.
I linked to other thoughts on AI just in case others are interested in what I have to say. I don't stand to gain anything from what I write, and I don't even have analytics to tell me more people are viewing it.
All in all, I was just trying to be transparent, and share my work.
Your actor analogy in your other post about AI doesn't really work when it comes to using LLMs for coding, at least. LLMs are pretty good at writing working code, especially given suitable guidance. An actor wouldn't be able fake their way through that.
That's nice to hear. For me personally, I don't really care what tools the author uses to write the article, as long as the author takes responsibility! Yes, that means I'll blame you for everything I see in the article :P
If I recall correctly, he used miniKanren along with formalized, structured data extracted from medical research. Unfortunately, his son has since passed away.
Note that this article is by the same Greg Egan who wrote Permutation City, a (in my opinion) really good, deeply technical, hard science fiction novel exploring consciousness, computation, and the infinite nature of the universe.
If that sounds interesting, I recommend not reading too much about the book before starting it; there are spoilers in most synopses.
You don't necessarily need a background in programming and theoretical computer science to enjoy it. But you'll probably like it better if you already have some familiarity with computational thinking.
Funnily enough I went into it with a background in math and was surprised about one specific claim that I couldn't quite understand, and it turns out it was subtly incorrect in such a way that it actually adds an interesting twist to the story (Greg Egan acknowledged it). I can't quite find the web page with the discussion (ETA: found it, it's the addendum at the end of the FAQ about the book [0]) but it's about <spoilers>the Garden of Eden configuration of the automaton.</spoilers>
ETA: I realize this sounds nitpicky and stickler-y so I just want to point out that I loved the book (and Greg Egan's work in general) and figuring out the automaton stuff was genuinely some of the most fun I've had out of a book.
This post recommends the Newsit extension to view Hacker News discussion associated with a page.
In the same vein, a few years ago, I made a Firefox extension for users who want a privacy-preserving way to see if pages have associated HN discussion:
Most other extensions probably hit an external API (such as Algolia) to check submission status, which means they send every page you visit to that API. Instead, my extension uses Bloom filters compiled from every link ever submitted (updated daily from the Hacker News BigQuery dataset) to check the current page's submission status. By using Bloom filters, my extension only hits the API when you click the button to view the discussion.
I use Chrome as my main browser and was going to ask if there was a Chrome version. I was sad to see there's a technical reason you couldn't make it work :(
Probably worth me revisiting! Web tech has changed a bit since I last investigated this in 2021, and I'm also not sure if I considered options like IndexedDB at the time.
Oh totally I wasn't suggesting that as a direct replacement but more an alternate approach and that privacy factor is important to consider. Apologies if that wasn't clear. Personally I don't check that often so the occasional connection on click isn't serious vs the complexities of adding another extension even if it's a neat one.
I think you may be overlooking some nuance related to that extension vs my solution though. Extensions like that check every page you visit whereas my approach only checks on click so there is a much greater need for privacy solutions with extensions like that since they would normally be sending out your entire browser history in real time to the api.
Jstriebs solution seems pretty neat though and it's definitely something I'll keep in mind for similar use cases even if I skip using it to minimize my extension usage. I was happy to hear about it and read how it worked.
> I think you may be overlooking some nuance related to that extension vs my solution though. Extensions like that check every page you visit whereas my approach only checks on click so there is a much greater need for privacy solutions with extensions like that since they would normally be sending out your entire browser history in real time to the api.
I'm not overlooking; this extension doesn't do that. That's what's so cool about the Bloom filter approach: all the checks can be done locally, never revealing your interests to a third party. So if the metric is privacy, it's superior to the bookmarklet, even if it checks every page you visit in real time.
(In principle, that is. I haven't reviewed the implementation. :)
The point from the end of the post that AI produces output that sounds correct is exactly what I try to emphasize to friends and family when explaining appropriate uses of LLMs. AI is great at tasks where sounding correct is the essence of the task (for example "change the style of this text"). Not so great when details matter and sounding correct isn't enough, which is what the author here seems to have rediscovered.
The most effective analogy I have found is comparing LLMs to theater and film actors. Everyone understands that, and the analogy offers actual predictive power. I elaborated on the idea if you're curious to read more:
I like this analogy a lot for non-technical...erm...audiences. I do hope that anyone using this analogy will pair it with loud disclaimers about not anthropomorphizing LLMs; they do not "lie" in any real sense, and I think framing things in those terms can give the impression that you should interpret their output in terms of "trust". The emergent usefulness of LLMs is (currently at least) fundamentally opaque to human understanding and we shouldn't lead people to believe otherwise.
> It's not a coincidence that I train a model on healthcare regulations and it answers a question about healthcare regulations
If you train a model on only healthcare regulations it wont answer questions about healthcare regulation, it will produce text that looks like healthcare regulations.
I don't think you're the right person to make any claim of "complete misunderstanding" when you claim that training an LLM on regulations would produce a system capable of answering questions about that regulation.
LLMs are trained on text, only some of which includes facts. It's a coincidence when the output includes new facts not explicitly present in the training data.
> It's a coincidence when the output includes facts,
That's not what a coincidence is.
A coincidence is: "a remarkable concurrence of events or circumstances without apparent causal connection."
Are you saying that training it on a subset of specific data and it responding with that data "does not have a causal connection"> Do you know how statistical pattern matching works?
It's not coincidence that the answer contains the facts you want. That is a direct consequence of the question you asked and the training corpus.
But the answer containing facts/Truth is incidental from the LLMs point of view, in that the machine really does not care, nor even have any concept of whether it gave you the facts you asked for or just nice-sounding gibberish. The machine only wants to generate tokens, everything else is incidental. (To the core mechanism, that is. OpenAI and co obviously care a lot about quality and content of the output)
Totally agree with that. But the problem is the phrase "coincidence" makes it into something it absolutely isn't. And it's used to try and detract from what these tools can actually do.
They are useful. It's not a coin flip as to whether Bolt will produce a new design of a medical intake form for me if I ask it to. It does. It doesn't randomly give me a design for a social media app, for instance.
At my work, we have an X-ray machine for PCB reverse engineering. On Fridays we throw in random stuff from around the office, and sometimes make videos about what we find inside.
A few weeks ago we released an X-ray teardown of several other, older chargers. Very interesting to compare with these fancy new ones!
A CT is, simplifying, an x-ray machine that takes lots of images in slices, then analyze them with certain algorithms to reconstruct 2D and 3D images of the interior of the 'subject'.
Yeah true, I was still half asleep and couldn't explain further. It's more like you image from lots of different viewpoints, then calculate a slice, then join the slices to form a volume.
It's better explained visually (at least to me) :P
I implied that on purpose. I'm generally hesitant to anthropomorphize LLMs, but in this case, I disagree with you that they don't have intention. They were developed to output likely tokens, and tuned such that their big tech developers approve the output. That is their intention. Their one and only intention.
That being said, I completely agree on the agency point. They don't make decisions, and certainly don't "think" like people.
Still, I believe the benefits of the analogy outweigh the potential loss of nuance here.
I have been explaining LLMs to friends and family by comparing them to actors. They deliver a performance in-character, and are only factual if it happens to make the performance better.
https://itch.io/jam/langjamgamejam/entries
There were some really impressive submissions in spite of the short time frame!
reply