Uh… no its just the free models being free, theyre lower cost intentionally to provide free options for people who dont wanna pay subscription fees.

(context is (V)RAM)

Eh sort of, its more operating costs, the larger the context size the more expensive the model is to run, literally in terms of power consumption.

Keep in mind we are on the scale of fractions of cents here, but multiply that by millions of users and it adds up fast.

But the end result is that the agent will fuck stuff up, and will even quickly /forget/ it fucked that up if you dont catch it asap

A lot of them have a context window that can be wiped out within like, 2 minutes of steady busywork…

They dont lol

Pretty much always this is just the fact cheaper, especially free, chatbots, have very limited context windows.

Which means the initial restrictions you set like “dont do this, dont touch that” etc get dropped, the LLM no longer has them loaded. But it does have in the past history the very clear and urgent directives of it trying to do this task, its important, so it’ll do whatever it autocompletes its gotta do to accomplish the task. And then… fucks something up.

When you react to their fuck up, it *reloads the context back in

So now the LLM has in its history just this:

1. It doing a thing against the rules
2. The user yelling at it
3. The users now getting loaded after that on top

So now the LLM is going to autocomplete its generated text on top being very apologetic and going on about how it’ll never happen again.

Thats all there is to it.

Itd be weird to be exempt

Its really sounding like the FCC banned all routers and companies can only comply by running their own they install software on manually XD

TIL! I assume theres gotta be at least some US manufacturing of routers… I hope… otherwise the US just banned using the internet in effect lol

Isn’t TP Link a US company?

Basically yep.

No they mean routers.

And running your own router with its own OS is allowed.

Buying a prefab router from Shenzhen chock full of foreign malware is not.

Only locally produced malware is now allowed in prefab routers!

Folks definitely should be picking up learning how to setup their own router and wifi hubs, but they wont do it.

Its more and more seeming less like “Fuvk AI” and more and more like “I am bad at using AI and Im gonna blame my skill issues on everyone else”

This entire post sits in pure “git gud” territory, wait-

git: unrecognized command "good"

Dang

…wat?

LLMs are completely capable of invoking npm install, the fuck you smoking lol.

Theyre perfectly able to read docs and use existing libraries fine.

Arguably I find better, because when I bump into a bug in a lib, I can fork it, fix the bug, get my patch up, and use the patch in my project in like 15min flat, no longer having to even worry about “ugh is it worth the effort to fix it”

Yes, its so easy now. I just do it.

Furthermore agents can look up documentation in mere seconds and read it (better than Id say a lot of junior devs tend to approach problems ngl)

If the agent is equipped with adequate tools and instructions, its extremely productive under strict guidance.

The same way.

The result the LLM produces is a link to the relevant information directly I can click and go to it.

Example would be a giant collection of files, think like, 10gb+ of many pdfs, or more. I want any relevant sections on a topic, it quickly can aggregate on it and give me links I click to open up straight to relevant sections in specific files, and then read rhem.

This stuff comes up a lot in my industry (software dev) as we often inherit huge data pools of legacy documentation on massive codebases.

When I am tasked with fixing 1 small specific piece of the system, it could take me hours to find the specific stuff Im looking for on the (often poorly maintained) docs.

But also vector db setup to map to that data, and an LLM wired up to it, can search it in milliseconds and pull up relevant sections asap, and I can click and dig deeper from there as much as I need.

This sort of “fuzzy searching” vectorization of tokens is what an LLM does very well. Its part of how it produces its output, but you can reverse the process to create search indexes (effectively reversing the data through the LLM to turn the data into deterministic vectors)

Amd its important to note, the fault tolerance you perceive here doesnt apply. Thos specific type of searching with vector DBs will always produce the same results from the same input, everytime. Its deterministic.

What fault tolerance?

I tell it to find me the info, it searches for it via provided tools, locates it, and presents it to me.

Ive very very rarely seen it fail at this task even on large sets.

Usually if theres a fail point its in the tools it uses, not the LLM itself.

But LLMs often are able to handle searching via multiple methods, if they have the tools for it. So if one tool fails they’ll try another.

I have not had this experience tbh, Ive found summarizing to be one of the few things they are good at out of the box.

If your LLM summarizes something poorly you probably just fucked something up and got a “shit in, shit out” problem.

That sounds like a tooling problem.

Either your tooling was outright broken, or not present.

It should be a very trivial task to provide an agent with a MCP tool it can invoke to search for stuff like that.

Searching for a known specific value is trivial, now you are back to just basic sql db operations.

These types of issues arise when either:

A: the tool itself just gave the LLM bad info, so thats not the LLMs fault. It accurately reported on wrong data it got handed.

B: the LLM just wasnt given a tool at all and you prompted it poorly to give room for hallucinating. You just asked it “who has this license plate” instead of “use your search tool to look up who has this license plate”, the latter would result in it reporting the lack of a tool to search with, the former will heavily encourage it to hallucinate an answer.

They dont impact it at all, its not relevant to using MCP vector searching for info.

What Model Context Protocols have you tried that you had issues with?

Ive found most vector db search MCPs are pretty solid.

Hallucinations aren’t relevant as an issue when it comes to fuzzy searching.

Im not talking about the LLM generating answers, Im talking about sifting through vector databases to find answers in large datasets.

Which means hallucinations arent a problem now.

Can you come up with better ways to quickly search and summarize massive amounts of data?

Thats what I find their best use case is, and theres no better solution for it, so I use it for that heavily.

No… thats not how training models… works…

Thats not directly engineered.

They arent themselves making it that way, they’re just selecting the ones that happen to be that way.

Its like genetic breeding.

If I breed chickens and specifically breed red ones so every generation I get redder and redder chickens, thats not “engineering” them to be red.

I just kept the ones that happened to be red for the next generation.

Thats not engineering, its not deterministic, its just pulling the slot machines handle enough times until it pays out.

If I say “Im gonna keep pulling this slot machines handle until I hit the jackpot”, then do that, hit the jackpot after 2000 pulls, and stop there, would you say I “engineered” the jackpot occurring just cuz I stopped and my last pull was on the jackpot outcome?

Seemingly conscious AI is produced by developers who deliberately engineer behaviours that create the illusion of inner life.

False, the mimicry isnt engineered or even deliberate.

Its an outcome of the training process, LLMs are declarative in design, you create a rubric for success and then pure RNG eventually rolls the ball into the target.

You dont make it happen, you just let it randomly roll around until it hits the target by chance.