Leo, You are Living Under a Rock When It Comes to AI

[something_else]

39 minutes ago, Butters said:

Yes this is true, but... it writes the logic in a language it knows (like Python) and translates it line-by-line into your new syntax on the fly.

Ehm, I'd be very surprised if that were literally the case. I suspect it will store the syntax rules in its context and use that to write code.

[Leo Gura]

1 hour ago, something_else said:

They literally can. Especially in agentic form. You keep making the same assertions that are just categorically wrong.

No.

This has been recently tested. They cannot code obscure languages with insuffucient coding examples.

You guys love to accuse me of the very thing you are guilty of! Your position is not scientic nor empirical.

Edited 5 hours ago by Leo Gura

[Leo Gura]

2 hours ago, CosmicExplorer said:

AI is able to solve some levels.

It is not my argument that LLMs have zero intelligence or merely spit out memorized data.

LLMs do very sophisticated data processing. However, it will never get them to AGI without a totally new architecture that is not based on mass data processing.

Intelligence is not just mass data processing.

Edited 5 hours ago by Leo Gura

[something_else]

1 hour ago, Leo Gura said:

This has been recently tested. They cannot code obscure languages with insuffucient coding examples.

You've changed the claim.

You started with "LLMs cannot code in a language not in its training data", now it's "they cannot code obscure languages with insufficient examples", those are different claims. Of course if the model can't code in a language for which it has no examples of. The same is true of humans, so that point doesn't really do the work you need it to do in an AGI argument.

The point is that you can load examples into the model's context and it will use those to write in a language it hasn't seen during training.

That's what they did in this paper (https://arxiv.org/pdf/2603.14501) and you can see an excellent summary in the snippet image below.

-----

On top of this, 53.8% is the result a 'chatbot' LLM gets. When they gave the task to a coding agent LLM and provided it with documentation it could search as needed (like a human would) it jumped to 77.6% accuracy.

The coding agent can also employ trial and error approach where it will compile some code, check the output of the compiler, edit its attempt to fix any error messages, remember it's mistake in context, and then try again, so it will build up an even better picture of how the new language works as it goes.

 

Edited 4 hours ago by something_else

[Girzo]

58 minutes ago, something_else said:

You started with "LLMs cannot code in a language not in its training data", now it's "they cannot code obscure languages with insufficient examples", those are different claims. Of course if the model can't code in a language for which it has no examples of.

My man, I am not a native English speaker, but, uh... Insufficient examples means exactly there are no examples in its training data. And it struggles a lot because it can't think, it defaults to repeating patterns reinforced during the learning phase. That you put it into its context, you need some special curated version of the rules, while a human can read the documentation version every other human reads and process it, learn it. Then if you get for example Fable and give it those snippets, it WON'T perform at the level it does in other languages. Because it doesn't have that many solved examples of problems to pattern-match. There's no thinking.

Even if you do social science, it can't come up with a good idea every paragraph when writing long-form texts, because it doesn't think. It can give a good idea if you ask it for ideas, then you have to trim 6 or 7 bad ones using your own intelligence. The only time it produces a good essay Is when I basically create the whole outline and then put the insights, all the important info into the paragraphs, and just ask the AI to process it into a fully finished, flowing prose. 

But still, I am the one doing the thinking, AI is like a stupid assistant who happens to know English and various formatting standards (or coding languages) really well, who can organise your thoughts.

If you give it too much of its own agency it defaults to boring, useless slop, the longer it processes something.

Edited 4 hours ago by Girzo

[Leo Gura]

59 minutes ago, something_else said:

You've changed the claim.

You started with "LLMs cannot code in a language not in its training data", now it's "they cannot code obscure languages with insufficient examples", those are different claims.

I often speak loosely on this forum because this is not a techncial paper.

I meant the later. I didn't change it, I just stated it more precisely.

Quote

Of course if the model can't code in a language for which it has no examples of. The same is true of humans

Not of course. A human can.

Humans don't need 10 million examples to learn a point. AI does.

A human teen can learn to drive from 20 hours of data. An AI still cannot drive after 10 billion hours of driving data. This isn't just a matter of degree, this is a difference in kind.

Edited 4 hours ago by Leo Gura

[Girzo]

All this AI stuff just shows how over-dignified programmers are in today's economy, if most of them up to this moment have been doing stuff that can be done by a mindless machine.

What do you get the money for? Almost as bad as MDs.

Edited 4 hours ago by Girzo

[something_else]

1 hour ago, Leo Gura said:

Not of course. A human can.

 

A human can't suddenly just program in C++ without somebody teaching them how to do it, or at least having access to documentation. So yes, we need examples too.

Quote

Humans don't need 10 million examples to learn a point. AI does.

Billions of rows of training data is to an LLM what billions of years of evolution is to a human brain.

The end result is the same, both end up in a state where they are capable of learning a previously unseen programming language from documentation and trial and error.

 

 

Edited 2 hours ago by something_else

[Girzo]

2 minutes ago, something_else said:

Billions of rows of training data is to an LLM what a billions of years of evolution is to a human brain.

It's precisely not. It's a coping mechanism to deal with the lack of evolution and intelligence.

[something_else]

2 hours ago, Girzo said:

It's precisely not. It's a coping mechanism to deal with the lack of evolution and intelligence.

Why?

[NewKidOnTheBlock]

2 hours ago, Girzo said:

It's precisely not. It's a coping mechanism to deal with the lack of evolution and intelligence.

And toilet paper is a coping mechanism to deal with the lack of natural bidet evolution within trees. Trees would eventually grow bidets if we weren't so keen on cutting them down to produce toilet papers