AI is taking over - ChatGPT

[pquinn]

Glade said:

It does significantly reduce time googling for "How to" tutorials.

Amazing how many times an LLM AI response spits out something incredibly similar to the top few responses on Google (or whatever), even if that turns out not to be particularly great as an actual answer. It's almost funny how weak a lot of the models are at dressing up the underlying data.

Lots of the code generation and other stuff seems to be clearly in the GIGO zone of operation; the answers aren't necessarily good, just popular, and the fun will really start when the AI generated stuff starts to reinforce its own prejudices on answers by polluting the input to the models via a nice feedback loop of published generated material.

A lot of this is promising to a point but it's not anywhere near where the hype puts it.

Still some money to be made but it's a very crowded market in the AI bubble right now...

[768]

pquinn said:

Lots of the code generation and other stuff seems to be clearly in the GIGO zone of operation; the answers aren't necessarily good, just popular, and the fun will really start when the AI generated stuff starts to reinforce its own prejudices on answers by polluting the input to the models via a nice feedback loop of published generated material.

Others have had the opposite experience to me, but I find if I ask it to solve some relatively trivial but real problem (not something for an exam scenario) in full that might require one or two hundred lines of code, it gives me garbage. It'll use completely made up APIs that sound entirely plausible but don't exist, at all.

Github Copilot though I find quite useful, it frequently surprises me without how good it is at predicting the rest of a line of code. Not the most complicated scenario and perhaps the least "intelligent", but very useful. Sometimes it's so good that it irritates me when I have to feed it what feels like an obvious letter or two to elicit a prediction I know it will then predict correctly.

[Mr Whippy]

pquinn said:

Glade said:

It does significantly reduce time googling for "How to" tutorials.

Amazing how many times an LLM AI response spits out something incredibly similar to the top few responses on Google (or whatever), even if that turns out not to be particularly great as an actual answer. It's almost funny how weak a lot of the models are at dressing up the underlying data.

Lots of the code generation and other stuff seems to be clearly in the GIGO zone of operation; the answers aren't necessarily good, just popular, and the fun will really start when the AI generated stuff starts to reinforce its own prejudices on answers by polluting the input to the models via a nice feedback loop of published generated material.

A lot of this is promising to a point but it's not anywhere near where the hype puts it.

Still some money to be made but it's a very crowded market in the AI bubble right now...

That’s because the LLM doesn’t know what’s right or wrong or good or bad, just what the common answer is… and a lot of the time on the internet the common answer is wrong or shallow etc.


Until these AI can self-audit against other reference material they’re only as good as the training data.

And much like FB etc, who is gonna police/moderate and meta-data ALL that training data?


It’s all great when you just scoop data willy nilly, or let others generate it for you.
But once you become responsible for it… yeah… suddenly your overheads are £££ hundreds of billions or even trillions.


AI will be a thing but it’s not gonna end with everyone being a winner this cycle!

[pquinn]

Mr Whippy said:

That’s because the LLM doesn’t know what’s right or wrong or good or bad, just what the common answer is… and a lot of the time on the internet the common answer is wrong or shallow etc.

But LLM and other generative output looks so shiny...

Problem is when you try to use it for solutions that have definite known rules and solid patterns it's maybe not the best approach compared to the other options that are also AI but of a different flavour.

Had this chat a few times recently when being demo'd some 'clever' AI; it looks OK at first glance, then you spot the invented/nonsense behaviour, then you ask why in a well known domain with well known rules & models why were approaches X/Y/Z not tried 'cos what these things were doing were relatively useless beyond a demo, and there was no obvious route to get them to be something useful especially beyond the easy bits that were started with.

You can reproduce a plausible looking pattern easily enough but that's not quite the same as it actually generating the proper output with the proper patterns behind it.

[Olivera]

768 said:

pquinn said:

Lots of the code generation and other stuff seems to be clearly in the GIGO zone of operation; the answers aren't necessarily good, just popular, and the fun will really start when the AI generated stuff starts to reinforce its own prejudices on answers by polluting the input to the models via a nice feedback loop of published generated material.

Others have had the opposite experience to me, but I find if I ask it to solve some relatively trivial but real problem (not something for an exam scenario) in full that might require one or two hundred lines of code, it gives me garbage. It'll use completely made up APIs that sound entirely plausible but don't exist, at all.

Github Copilot though I find quite useful, it frequently surprises me without how good it is at predicting the rest of a line of code. Not the most complicated scenario and perhaps the least "intelligent", but very useful. Sometimes it's so good that it irritates me when I have to feed it what feels like an obvious letter or two to elicit a prediction I know it will then predict correctly.

I'm highly sceptical that AI will be used (at least in the short-medium term) for anything other than hobby coding projects, or in the enterprise as 'suggest my next statement or short method'.

Firstly most large organisations I work with currently have a complete and comprehensive moratorium on adding AI generated code to the codebase, primarily because the licensing and intellectual property concerns are muddled and unclear.

Secondly, how can we be sure that the generated code does what is intended? Yes the AI can also generate test cases, but a human is still required to interpret this (and the results) for accuracy. As the AI generates more and more code it's likely that human understanding falls behind, especially when we are dealing with a large and possibly AI generated codebase. Bugs, unexpected results and requirement ambiguities will become manifest. No serious organisation is (at least currently) going to let this loose on their codebase.

[deckster]

Olivera said:

I'm highly sceptical that AI will be used (at least in the short-medium term) for anything other than hobby coding projects, or in the enterprise as 'suggest my next statement or short method'.

Firstly most large organisations I work with currently have a complete and comprehensive moratorium on adding AI generated code to the codebase, primarily because the licensing and intellectual property concerns are muddled and unclear.

Secondly, how can we be sure that the generated code does what is intended? Yes the AI can also generate test cases, but a human is still required to interpret this (and the results) for accuracy. As the AI generates more and more code it's likely that human understanding falls behind, especially when we are dealing with a large and possibly AI generated codebase. Bugs, unexpected results and requirement ambiguities will become manifest. No serious organisation is (at least currently) going to let this loose on their codebase.

You're right, of course.

But I would expect that statement to look quite different in 10 years time.

[Mr Penguin]

I was working on something recently and found that the package other people shared code for won't install from pip. ChatGPT gives code for it, and it looks real. Turns out ChatGPT invented the package and function and people copied it online without testing. No doubt ChatGPT will pick this up in the next training run and reinforce usage even more. What will be interesting is if developers try to get around it by creating the package with the syntax ChatGPT has created.

[768]

deckster said:

You're right, of course.

But I would expect that statement to look quite different in 10 years time.

Yeah. I've worked with plenty of self-styled "serious" organisations.

They won't be able to compete without it, it's coming even to the slowest moving of large organisations.

[DanL]

768 said:

deckster said:

You're right, of course.

But I would expect that statement to look quite different in 10 years time.

Yeah. I've worked with plenty of self-styled "serious" organisations.

They won't be able to compete without it, it's coming even to the slowest moving of large organisations.

I work for a good sized multinational software company, and we’re fully embracing AI. It’s unlikely to generate fully formed code (although it can, as I understand it), but it should accelerate coding in general - either by autocompleting to speed up typing, or by generating code snippets to deliver functionality.

We don’t intend to replace programmers with AI, but we do hope to get more out of the ones we have due to anticipated productivity improvements. As the technology improves (and I appreciate there’s a baked in assumption there that it will!) we might well get to a point where some code is automatically generated.

[Olivera]

DanL said:

I work for a good sized multinational software company, and we’re fully embracing AI. It’s unlikely to generate fully formed code (although it can, as I understand it), but it should accelerate coding in general - either by autocompleting to speed up typing, or by generating code snippets to deliver functionality.

I agree with regards to auto-complete and generation of code snippets, but good IDEs have been able to do this for years, albeit AI may improve efficacy. Full code generation for serious applications should be an absolute non-no for the time being, despite what hype believers and fantasists say,

[DanL]

Olivera said:

DanL said:

I work for a good sized multinational software company, and we’re fully embracing AI. It’s unlikely to generate fully formed code (although it can, as I understand it), but it should accelerate coding in general - either by autocompleting to speed up typing, or by generating code snippets to deliver functionality.

I agree with regards to auto-complete and generation of code snippets, but good IDEs have been able to do this for years, albeit AI may improve efficacy. Full code generation for serious applications should be an absolute non-no for the time being, despite what hype believers and fantasists say,

Quite agree on AI generation for full code, and for the reasons you listed earlier - if nothing else, debugging and error correction would be unnecessarily painful.

Code generation with review and tweaking by a coder, is where it’s at for us at the moment. Speeds up the thought process and code creation, but with a human touch and overall understanding.

[Olivera]

deckster said:

But I would expect that statement to look quite different in 10 years time.

The fundamental issue will not have changed in 10 years, namely that the precise meaning and constructs of a program (in a formal programming language) *cannot* easily be replicated by describing a domain problem in plain English. At least not without introducing misunderstandings, bugs and errors at a rate rising exponentially with the size of the codebase. Yes AI will be able to iteratively improve things with human feedback and get *most* things right, but most isn't good enough when this leaves a load of bugs, defects and security exploits that are completely unknown to the 'developers', doubly so when it's any kind of mission critical real world application.

[Mr Penguin]

Olivera said:

I agree with regards to auto-complete and generation of code snippets, but good IDEs have been able to do this for years, albeit AI may improve efficacy. Full code generation for serious applications should be an absolute non-no for the time being, despite what hype believers and fantasists say,

We use it for small chunks or small functions but not to write the whole thing. The main architecture should be designed by a human but if you are good at prompt engineering and know enough to be able to review it then it can be done in some cases.

[Frimley111R]

From a non technical viewpoint, how do you make an AI program? I don't mean from scratch but, if you wanted an Ai tool (does it even need to be AI?) to send me the top 3 most viewed videos on YT daily for specific search terms (e.g. Samsung Fold or Cute Kittens) and post them into a WP site?

[boxedin]

Frimley111R said:

From a non technical viewpoint, how do you make an AI program? I don't mean from scratch but, if you wanted an Ai tool (does it even need to be AI?) to send me the top 3 most viewed videos on YT daily for specific search terms (e.g. Samsung Fold or Cute Kittens) and post them into a WP site?

AI is not required for that.

The AI moniker is thoroughly misused/misunderstood.

This is AI: https://www.ted.com/talks/jim_fan_the_next_grand_c...

If you want to feel uncomfortable, watch the robot training sections: https://www.youtube.com/watch?v=SBoen3q5AoQ

Edited by boxedin on Wednesday 24th January 10:04

[SpudLink]

boxedin said:

Frimley111R said:

From a non technical viewpoint, how do you make an AI program? I don't mean from scratch but, if you wanted an Ai tool (does it even need to be AI?) to send me the top 3 most viewed videos on YT daily for specific search terms (e.g. Samsung Fold or Cute Kittens) and post them into a WP site?

AI is not required for that.

The AI moniker is thoroughly misused/misunderstood.

This is AI: https://www.ted.com/talks/jim_fan_the_next_grand_c...

If you want to feel uncomfortable, watch the robot training sections: https://www.youtube.com/watch?v=SBoen3q5AoQ

Edited by boxedin on Wednesday 24th January 10:04

I cut and and pasted Frimley111R's post into ChatGPT 3.5:
"This task may not necessarily require sophisticated AI techniques, but it does involve automation and programming skills."

It does also suggest an answer, but I won't paste it all here.

Edited by SpudLink on Wednesday 24th January 10:18

[Hoofy]

DanL said:

Quite agree on AI generation for full code, and for the reasons you listed earlier - if nothing else, debugging and error correction would be unnecessarily painful.

Code generation with review and tweaking by a coder, is where it’s at for us at the moment. Speeds up the thought process and code creation, but with a human touch and overall understanding.

Same goes for generic copy creation. Need to still apply your brain. I thought it would be funny to launch a social media channel that's just unedited AI, doesn't matter what the output is, I use it as is, just to see the response.

[Frimley111R]

SpudLink said:

boxedin said:

Frimley111R said:

From a non technical viewpoint, how do you make an AI program? I don't mean from scratch but, if you wanted an Ai tool (does it even need to be AI?) to send me the top 3 most viewed videos on YT daily for specific search terms (e.g. Samsung Fold or Cute Kittens) and post them into a WP site?

AI is not required for that.

The AI moniker is thoroughly misused/misunderstood.

This is AI: https://www.ted.com/talks/jim_fan_the_next_grand_c...

If you want to feel uncomfortable, watch the robot training sections: https://www.youtube.com/watch?v=SBoen3q5AoQ

Edited by boxedin on Wednesday 24th January 10:04

I cut and and pasted Frimley111R's post into ChatGPT 3.5:
"This task may not necessarily require sophisticated AI techniques, but it does involve automation and programming skills."

It does also suggest an answer, but I won't paste it all here.

Edited by SpudLink on Wednesday 24th January 10:18

Ok, I'll do some googling...

[SpudLink]

OpenAI have a new text-to-video generator, called SORA. It's not available to the general public yet, but there are examples of what it can do on their website.

The technology that was always "10 years away" is now here. Not always perfect, but what it can do is amazing.

[Frimley111R]

Cool but like a lot of these AI tools, when you do it yourself you never seem to get the same quality of results.